Streamer
The Streamer trait defines the common interface of streaming clients.
Implemented by:
Kafka semantics
Consult https://kafka.apache.org/documentation/#intro_concepts_and_terms for a gentle introduction.
Redis semantics
SeaStreamer Redis aims to provide a Kafka-like client experience, but there are some fundamental differences between Redis and Kafka:
- In Redis sequence numbers are not contiguous
- In Redis messages are dispatched to consumers among group members in a first-ask-first-served manner, which leads to the next point
- In Redis ACK has to be done per message
Stdio semantics
The Stdio backend spawn two dedicated threads to handle stdin and stdout respectively. The host part of the Streamer URI is always empty, i.e. in stdio://, the host is . There is only shard ZERO.
File semantics
In File, unlike Stdio, there can be multiple independent Streamers in the same process.
In File, unlike other backends, it is possible to signify end of stream with a EOS message. See the end_with_eos option.
Currently it only streams to Shard ZERO.
If the file is removed from the file system, the streamer would stop.
ConnectOptions
timeout
Set the default network timeout for all connections.
connect
Establish a connection to the streaming server. The Streamer implementation does not have to maintain an open connection to the server.
disconnect
Disconnect from the streaming server. The intention is to flush remaining messages and exit gracefully. You have to await this operation until it completes. Once you called this method, all producers and consumers created will become unusable.
create_producer
Create a producer that streams to the specified stream.
create_generic_producer
Create a producer that can stream to any stream.
create_consumer
Create a consumer subscribing to the specified streams.