A company is planning on using an EMR cluster to process data from their On-premise log files.
They need to perform SQL queries on the underlying data.
Which of the following can be used along with the EMR cluster to satisfy this requirement? Choose 2 answers from the options given below.
Click on the arrows to vote for the correct answer
A. B. C. D.Answer - A and B.
The AWS Documentation mentions the following.
Hive is an open-source, data warehouse, and analytic package that runs on top of a Hadoop cluster.
Hive scripts use a SQL-like language called Hive QL (query language) that abstracts programming models and supports typical data warehouse interactions.
Hive enables you to avoid the complexities of writing Tez jobs based on directed acyclic graphs (DAGs) or MapReduce programs in a lower level computer language, such as Java.
Presto is a fast SQL query engine designed for interactive analytic queries over large datasets from multiple sources.
Option C is incorrect since this tool is used to access Hive metastore tables.
Option D is incorrect since this tool is used for transferring data between Amazon S3, Hadoop, HDFS, and RDBMS databases.
For more information on hive and presto, please refer to the below URL.
https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hive.html https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-presto.htmlThe correct answers are A. Hive and B. Presto.
EMR (Elastic MapReduce) is a web service provided by Amazon Web Services (AWS) to run big data processing frameworks such as Apache Hadoop, Apache Spark, and others. EMR cluster allows processing of large amounts of data using a distributed computing architecture.
To perform SQL queries on the underlying data in EMR cluster, there are several options available, including Hive, Presto, HCatalog, and Sqoop.
Hive is a data warehousing framework that allows querying and analysis of large datasets stored in Hadoop Distributed File System (HDFS). Hive translates SQL-like queries into MapReduce jobs, which can be executed on the EMR cluster. It also supports the creation of tables, loading data into tables, and data manipulation using SQL-like queries. Hive is widely used in the Hadoop ecosystem and is a popular tool for data analysts and data scientists.
Presto is another distributed SQL query engine that allows querying of data from various data sources such as Hadoop HDFS, Amazon S3, and others. Presto is optimized for running interactive SQL queries and is designed to handle large-scale datasets. Presto is also widely used in the big data ecosystem, and it supports a variety of data sources, including Hadoop HDFS, Cassandra, MongoDB, and others.
HCatalog is a metadata and table management system for Hadoop that allows sharing of data between different Hadoop components such as Pig, Hive, and MapReduce. HCatalog provides a unified view of data stored in Hadoop and allows users to define and manage metadata, including tables and partitions.
Sqoop is a tool used to import and export data between Hadoop and external data sources such as relational databases. Sqoop allows users to import data from a relational database into Hadoop or export data from Hadoop to a relational database. Sqoop does not support querying of data using SQL.
In summary, to perform SQL queries on the underlying data in an EMR cluster, Hive and Presto are the best options. Both tools provide SQL-like querying capabilities and are optimized for processing large datasets. HCatalog can also be used for metadata management, while Sqoop is used for data import/export but does not support querying.