De Banane Atomic
Aller à la navigationAller à la recherche


Allows to gather data from different sources using data stream or fire hoses, and get that data to a destination in the expected format.
Allows to collect, process and analyze data in near real-time at scale.
Kinesis data streams are used in places where an unbounded stream of data needs to worked on in real time.
And Kinesis Firehose delivery streams are used when data needs to be delivered to a storage destination, such as S3.

Common use cases

  • pattern detection
  • click stream analysis (who is clicking on what and when)
  • log processing for machine learning
  • anomyly detection in IoT devices


  • fully managed service (focus on data and don't worry about the underlying system)
  • can handle large amount of data
  • can consume process and buffer data in real-time
  • allows production of real-time metrics and reporting

Data Stream

Move data from sources to a destination while having analytics, monitoring alerts, and connections to other services.

Common use cases

  • log and data intake and procesing
  • real-time metrics, reporting, and data analytics


  • ensures data durability and elasticity


Producers put records into Kinesis Data Streams.

  • EventBridge
  • CloudFront, collect views and analyze real-time metrics of CDN
  • CloudWatch
var kinesisDataStreamName = "input-stream";
var region = RegionEndpoint.USEast1;
var kinesisConfiguration = new AmazonKinesisConfig
    Profile = new Profile("..."),
    RegionEndpoint = region
var kinesisClient = new AmazonKinesisClient(kinesisConfiguration);

for (var i = 1; ; i++)
    var message = new Message(i, DateTime.Now);
    var data = JsonConvert.SerializeObject(message);

    using var memoryStream = new MemoryStream(Encoding.UTF8.GetBytes(data));
    var request = new PutRecordRequest
        StreamName = kinesisDataStreamName,
        Data = memoryStream,
        PartitionKey = "partitions1"

    var response = kinesisClient.PutRecordAsync(request).Result;
    Console.WriteLine($"Shard ID: {response.ShardId}, Sequence Number: {response.SequenceNumber}, HTTPStatusCode: {response.HttpStatusCode}");

    await Task.Delay(1000); // every 1 second

class Message(int id, DateTime date)
    public int Id { get; set; } = id;
    public DateTime Date { get; set; } = date;


Consumers get records from Kinesis Data Streams and process them.

  • Data Firehose


  • ingest data from different sources
  • transform source records using AWS lambda functions
  • deliver data to a specified destination

Similar to Data Stream but acts as an ETL.


  • Kinesis Data Streams


  • S3
  • MongoDB

Data Analytics

  • Studio Notebook
  • Glue catalog