Ben is a Data Security Analyst of Adatum Inc.
He's evaluating the LZO compression codec for Azure Databricks.
There's a licensing restriction for the LZO codec in the Azure Databricks cluster.
So, the LZO compression codec is not available as default in the Databricks cluster.
Hence, as a workaround, an init script is required to install the LZO codec for performing compressions on Azure Databricks cluster.
What are the two kinds of init scripts supported by Azure Databricks?
Click on the arrows to vote for the correct answer
A. B. C. D. E.Correct Answers: B and C.
Azure Databricks supports two kinds of init scripts: global init scripts and cluster-scoped init scripts.
Global init scripts are executed on every node in the Databricks environment, including driver nodes, worker nodes, and all types of cluster nodes. These scripts are designed to set up global configurations or install global dependencies that are required across all clusters in the workspace. Global init scripts can be added to the Databricks workspace using the Databricks CLI or the API.
Cluster-scoped init scripts, on the other hand, are executed only on the nodes of a specific cluster. These scripts are useful for setting up dependencies or configurations that are required only by a specific cluster. For example, a cluster-scoped init script can be used to install a compression codec that is required only by a particular cluster. Cluster-scoped init scripts can be added to a cluster by specifying them as a configuration setting when creating or modifying the cluster.
In the scenario described in the question, Ben needs to install the LZO compression codec for Azure Databricks cluster. As the LZO codec is not available as default in the Databricks cluster, Ben needs to use an init script to install it. Since the licensing restriction prevents the LZO codec from being installed globally, Ben can use a cluster-scoped init script to install it on the nodes of the specific cluster that requires it.