Kinesis

Description

Allows to gather data from different sources using data stream or fire hoses, and get that data to a destination in the expected format.
Allows to collect, process and analyze data in near real-time at scale.
Kinesis data streams are used in places where an unbounded stream of data needs to worked on in real time.
And Kinesis Firehose delivery streams are used when data needs to be delivered to a storage destination, such as S3.

Common use cases

• pattern detection
• click stream analysis (who is clicking on what and when)
• log processing for machine learning
• anomyly detection in IoT devices

Benefits

• fully managed service (focus on data and don't worry about the underlying system)
• can handle large amount of data
• can consume process and buffer data in real-time
• allows production of real-time metrics and reporting

Data Stream

Move data from sources to a destination while having analytics, monitoring alerts, and connections to other services.

Common use cases

• log and data intake and procesing
• real-time metrics, reporting, and data analytics

Benefits

• ensures data durability and elasticity

Producers

Producers put records into Kinesis Data Streams.

Consumers

Consumers get records from Kinesis Data Streams and process them.

• Data Firehose

Firehose

• ingest data from different sources
• transform source records using AWS lambda functions
• deliver data to a specified destination

Similar to Data Stream but acts as an ETL.

Source

• Kinesis Data Streams

Destination

• S3
• MongoDB