AWS Certified Big Data - Specialty Exam: Ideal Number of Dedicated Master Nodes for Elasticsearch

Ideal Number of Dedicated Master Nodes for Elasticsearch

Question

A company is planning on using the ElastiSearch service.

This needs to be setup in their production environment.

They need to come up with the ideal number of dedicated master nodes.

What is the recommended number of master nodes that should be setup for an ES domain?

Answers

Explanations

Click on the arrows to vote for the correct answer

A. B. C. D.

Answer - C.

This is given in the AWS Documentation.

########

Dedicated Master Nodes.

Amazon Elasticsearch Service uses dedicated master nodes to increase cluster stability.

A dedicated master node performs cluster management tasks, but does not hold data or respond to data upload requests.

This offloading of cluster management tasks increases the stability of your domain.

We recommend that you allocate three dedicated master nodes for each production Amazon ES domain:

One dedicated master node means that you have no backup in the event of a failure.

Two dedicated master nodes means that your cluster does not have the necessary quorum of nodes to elect a new master node in the event of a failure.

A quorum is Number of Dedicated Master Nodes / 2 + 1 (rounded down to the nearest whole number), which Amazon ES sets to

discovery.zen.minimum_master_nodes.

when you create your domain.

In this case, 2 / 2 + 1 = 2

Because one dedicated master node has failed and only one backup exists, the cluster does not have a quorum and cannot elect a new master.

Three dedicated master nodes, the recommended number, provides two backup nodes in the event of a master node failure and the necessary quorum (2) to elect a new master.

Four dedicated master nodes is no better than three and can cause issues if you use zone awareness.

If one master node fails, you have the quorum (3) to elect a new master.

If two nodes fail, you lose that quorum, just as you do with three dedicated master nodes.

If each Availability Zone has two dedicated master nodes and the zones are unable to communicate with each other, neither zone has the quorum to elect a new master.

########

Since this is clearly mentioned, all other options are invalid.

For more information on dedicated master nodes for Amazon ES, please refer to the below URL.

https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-managedomains-dedicatedmasternodes.html

When setting up an Amazon Elasticsearch Service domain in a production environment, it's essential to have dedicated master nodes. These nodes handle cluster management tasks and are responsible for maintaining the state of the cluster. The number of dedicated master nodes required depends on the size of the cluster and the level of fault tolerance needed.

The recommended number of dedicated master nodes for an Elasticsearch cluster is three. This configuration ensures that there is enough redundancy to handle failures while still maintaining cluster stability.

With three dedicated master nodes, the cluster can tolerate the failure of one node without any impact on the overall stability of the cluster. In contrast, having only one dedicated master node is not recommended as it creates a single point of failure. On the other hand, having two dedicated master nodes is a good option, but it can create a split-brain scenario if one node fails.

Split-brain happens when two nodes cannot communicate with each other and they both assume to be the master node. This can result in data loss, inconsistent cluster state and cause service disruption. Having three dedicated master nodes provides a higher level of redundancy, preventing split-brain scenarios and maintaining cluster stability.

In summary, the recommended number of dedicated master nodes for an Elasticsearch Service domain in a production environment is three.