Skip to main content
Version: 0.4.x 🚧

SeaStreamer Architecture

The architecture of sea-streamer is constructed by a number of sub-crates:

sea-streamer-types: Traits & Types​

This crate defines all the traits and types for the SeaStreamer API, but does not provide any implementation.

sea-streamer-socket: Backend-agnostic Socket API​

Akin to how SeaORM allows you to build applications for different databases, SeaStreamer allows you to build stream processors for different streaming servers.

While the sea-streamer-types crate provides a nice trait-based abstraction, this crates provides a concrete-type API, so that your program can stream from/to any SeaStreamer backend selected by the user on runtime.

This allows you to do neat things, like generating data locally and then stream them to Redis / Kafka. Or in the other way, sink data from server to work on them locally. All without recompiling the stream processor.

sea-streamer-kafka: Kafka / Redpanda Backend​

This is the Kafka / Redpanda backend implementation for SeaStreamer. This crate provides a comprehensive type system that makes working with Kafka easier and safer.

All API (many are sync) are properly wrapped as async. Methods are also marked &mut to eliminate possible race conditions.

This crate depends on rdkafka, which in turn depends on librdkafka-sys, which itself is a wrapper of librdkafka.

sea-streamer-redis: Redis Backend​

This is the Redis backend implementation for SeaStreamer. This crate provides a high-level async API on top of Redis that makes working with Redis Streams fool-proof:

  • Implements the familiar SeaStreamer abstract interface
  • A comprehensive type system that guides/restricts you with the API
  • High-level API, so you don't call XADD, XREAD or XACK anymore
  • Mutex-free implementation: concurrency achieved by message passing

This crate is built on top of redis.

sea-streamer-stdio: Standard I/O Backend​

This is the stdio backend implementation for SeaStreamer. It is designed to be connected together with unix pipes, enabling great flexibility when developing stream processors or processing data locally.

You can connect processors together with pipes: processor_a | processor_b.

sea-streamer-file: File Backend​

This is very similar to sea-streamer-stdio, but the difference is Stdio works in real-time, while sea-streamer-file works in real-time and replay. That means, File has the ability to traverse a .ss (sea-stream) file and seek/rewind to a particular timestamp/offset.

In addition, Stdio can only work with UTF-8 text data, while File is able to work with binary data.

sea-streamer-runtime: Async runtime abstraction​

This crate provides a small set of functions aligning the type signatures between async-std and tokio, so that you can build applications generic to both runtimes.