Load Kinesis Stream Data to Splunk
Question
One application of your company was configured to send all application logs to a Kinesis Stream and the logs can exist in the Stream shards for 24 hours.
A recent company strategy is to use Splunk Enterprise to search, monitor and analyze application logs and a Splunk server has already been deployed in an EC2 instance.
Which approach is the best to load the streaming data in Kinesis Stream to the Splunk instance?
Answers
Explanations
Click on the arrows to vote for the correct answer
A. B. C. D.Correct Answer - B.
Amazon Kinesis Data Firehose is a suitable data consumer for the data producer which is the Kinesis Stream in this case.
According to.
https://docs.aws.amazon.com/firehose/latest/dev/what-is-this-service.htmlKinesis Firehose is used to deliver logs to destinations including Amazon S3, Amazon Redshift, Amazon Elasticsearch Service (Amazon ES), and Splunk.
Option A is incorrect: Because Kinesis Stream cannot automatically deliver logs to Splunk.
There is no such feature in Kinesis Stream.
Option B is CORRECT: Streaming data can be delivered to Splunk through Kinesis Firehose as below:
Option C is incorrect: Because Kinesis analytics is mainly used to run continuous analysis.
There is no such requirement in this case.
Option D is incorrect: It needs to maintain an EC2 instance for this option.
However, Kinesis Firehose is fully managed by AWS and can operate the data in real time.
Option B is better from this perspective.
The best approach to load the streaming data in Kinesis Stream to the Splunk instance is to use Amazon Kinesis Data Firehose as a fully managed service to deliver real-time streaming data in Kinesis Stream to the Splunk instance.
Option A is incorrect because configuring the Kinesis Stream to auto deliver the received logs to the Splunk destination is not possible. Kinesis Stream is a data streaming service that allows real-time data processing, whereas Splunk is a log management and analysis tool. Therefore, the two services need a bridge to work together.
Option C is also incorrect because Amazon Kinesis Analytics is used to analyze and process streaming data in real-time. It is not used for delivering the data to the Splunk instance.
Option D is incorrect because using the AWS SDK in an EC2 instance to get the records from Kinesis Stream and forwarding the records to the Splunk instance is a manual process that requires custom code to be developed, deployed, and maintained. Moreover, it may not be scalable and reliable.
Option B is the best approach because Amazon Kinesis Data Firehose is a fully managed service that can load streaming data from Kinesis Stream to various destinations, including Splunk. With Amazon Kinesis Data Firehose, you can easily configure the delivery of streaming data to Splunk without the need for custom code. It also provides options for data transformation, buffering, and compression. Additionally, it can automatically handle scaling, fault tolerance, and retries to ensure high availability and reliability.
