Amazon Kinesis
The AWS-native event stream. Producers write records; the stream holds them in shards for 24 hours to 365 days; many consumers read the same records independently, each at their own position. You run it in on-demand mode, where AWS scales the shards, or provisioned mode, where you set the shard count yourself.
Producers write once. Every consumer reads the same records, at its own pace.
A record carries a partition key; Kinesis hashes it to one shard, which fixes ordering per key and read capacity. Standard (shared) consumers call GetRecords and split one shard's 2 MB/s read budget between them, with a 5-calls-per-second ceiling. Enhanced fan-out gives each registered consumer its own 2 MB/s per shard over a push stream (SubscribeToShard), at extra cost. S3 Tables and plain-S3 delivery require on-demand mode.
Lambda, Flink, Firehose or your own code.
Lambda
Kinesis invokes your function with a batch of records. No code to poll or checkpoint; good for transforms, filtering and fan-out to other services. Scales with shards.
Managed Flink
Apache Flink for aggregations, joins and windows over the stream, with state kept across records. Use it when a single record is not enough to decide.
Firehose
Reads the stream and lands it in S3, Redshift, OpenSearch or Splunk, with buffering, format conversion and compression. No consumer code at all.
KCL app
The Kinesis Client Library handles shard assignment, checkpointing and rebalancing across workers, so your code just processes records. The most control, the most to run.
Capacity mode and read pattern set the bill.
On-demand or provisioned capacity
On-demand bills per GB in and out plus a per-stream hour; it absorbs spiky or unknown traffic with no sizing. Provisioned bills per shard-hour; it is cheaper for steady, predictable load you can forecast. You can flip a stream between the two twice in 24 hours.
How many shards
In provisioned mode each shard gives 1 MB/s or 1,000 records/s of write and 2 MB/s of read. Throughput and cost scale linearly with the shard count, so size to peak write rate and to the number of shared-read consumers.
Shared reads or enhanced fan-out
Shared consumers divide one shard's 2 MB/s between them and poll; add a third and they all slow down. Enhanced fan-out gives each registered consumer its own 2 MB/s per shard, pushed with lower latency, at a per-consumer-shard-hour and per-GB charge. Up to 20 registered consumers per stream.
Prices are US East (N. Virginia), provisioned and On-demand Standard; On-demand Advantage has its own rates. Visual values are illustrative.
A Kinesis Data Streams bill, by mode.
Industrial-grade data pipelines
Use cases in production, not POCs: tests, CI/CD, lineage, on-call that sleeps at night.
Read the approach →The one-page Amazon Kinesis field guide.
Best practices, cost traps and performance checks, in the order you should run them. Printable, no fluff.