You need to use machine learning to produce real-time analysis of streaming data from IoT devices out in the field.
These devices monitor oil well rigs for malfunction.
Due to the safety and security nature of these IoT events, the events must be analyzed by your safety engineers in real-time.
You also have an audit requirement to retain your IoT device events for 7 days since you cannot fail to process any of the events.
Which approach would give you the best solution for processing your streaming data?
Click on the arrows to vote for the correct answer
A. B. C. D.Answer: B.
Option A is incorrect.
The Amazon Kinesis Data Streams Producer Library is not meant to be used for real-time processing of event data since, according to the AWS developer documentation, “it can incur an additional processing delay of up to RecordMaxBufferedTime within the library”
Therefore, it is not the best solution for a real-time analytics solution.
(See the AWS developer documentation titled Developing Producers Using the Amazon Kinesis Producer Library)
Option B is correct.
The Amazon Kinesis Data Streams API PutRecords call is the best choice for processing in real-time since it sends its data synchronously and does not have the processing delay of the Producer Library.
Therefore, it is better suited to real-time applications.
(See the AWS developer documentation titled Developing Producers Using the Amazon Kinesis Data Streams API with the AWS SDK for Java)
Option C is incorrect.
The Amazon Kinesis Data Streams Client Library interacts with the Kinesis Producer Library to process its event data.
Therefore, you'll have the same processing delay problem with this option.
(See the AWS developer documentation titled Developing Consumers Using the Kinesis Client Library 1.x)
Option D is incorrect.
The Amazon Kinesis Data Firehose service directly streams your event data to your S3 bucket for use in your real-time analytics model.
However, Amazon Kinesis Data Firehose retries to send your data for a maximum of 24 hours, but you have a 7-day retention requirement.
(See the Amazon Kinesis Data Firehose FAQs)
Reference:
Please see the Amazon Kinesis Data Streams documentation.
For real-time analysis of streaming data from IoT devices, Amazon Kinesis is a suitable solution. Kinesis Data Streams and Kinesis Data Firehose are two of the services offered by Amazon Kinesis.
Kinesis Data Streams is a scalable and durable real-time streaming data service that allows for the processing of high volumes of data per second. In this scenario, Kinesis Data Streams is the best option because it allows for real-time analysis of data from IoT devices. The Kinesis Producer Library can be used to pass events from producers to the Kinesis stream. The Kinesis Client Library can be used to process the data in real-time and then send it to the appropriate destination.
On the other hand, Kinesis Data Firehose is a managed service that allows for the loading of streaming data into data stores such as Amazon S3 and Amazon Redshift. It is also capable of transforming and enriching the data before it is loaded into the destination.
Therefore, option A is the best approach for this scenario as it provides real-time analysis of streaming data using Kinesis Data Streams and the Kinesis Producer Library for passing events from producers to the Kinesis stream.
Option B is incorrect because the Kinesis API PutRecords call is not the recommended way to pass events to Kinesis Data Streams as it can result in higher latency and decreased performance.
Option C is incorrect because the Kinesis Client Library is used for processing the data in real-time and not for passing events from producers to the Kinesis stream.
Option D is incorrect because it involves passing events directly to an S3 bucket using Kinesis Data Firehose, which is not the best option for real-time analysis of data. Although, using Kinesis Data Firehose could be a good option for storing the data for audit purposes for 7 days.