Redshift Architecture System Tables and Views

Understanding Redshift Metadata

Question

Parson Fortunes Ltd is an Asian-based department store operator with an extensive network of 131 stores, spanning approximately 4.1 million square meters of retail space across cities in India, China, Vietnam, Indonesia and Myanmar. Parson built a VPC to host their entire enterprise infrastructure on cloud.

Parson has large assets of data around 20 TB's of structured data and 45 TB of unstructured data and is planning to host their data warehouse on AWS and unstructured data storage on S3

The files sent from their on premise data center are also hosted into S3 buckets.

Parson IT team is well aware of the scalability, performance of AWS services capabilities.

Parson hosts their web applications, databases and the data warehouse built on Redshift in VPC.

The administrator wants to understand the system tables and views that support the team to understand the metadata of the Redshift architecture.

Please advice.

select 4 options.

Answers

Explanations

Click on the arrows to vote for the correct answer

A. B. C. D. E. F. G.

Answer : A, B, E, G.

Option A is correct -STL system tables are generated from Amazon Redshift log files to provide a history of the system.

These files reside on every node in the data warehouse cluster.

The STL tables take the information from the logs and format them into usable tables for system administrators.

To manage disk space, the STL log tables only retain approximately two to five days of log history, depending on log usage and available disk space.

If you want to retain the log data, you will need to periodically copy it to other tables or unload it to Amazon S3

https://docs.aws.amazon.com/redshift/latest/dg/c_intro_STL_tables.html

Option B is correct -STV tables are actually virtual system tables that contain snapshots of the current system data.

https://docs.aws.amazon.com/redshift/latest/dg/c_intro_STV_tables.html

Option C is incorrect -STV tables are actually virtual system tables that contain snapshots of the current system data.

https://docs.aws.amazon.com/redshift/latest/dg/c_intro_STV_tables.html

Option D is incorrect -STL system tables are generated from Amazon Redshift log files to provide a history of the system.

These files reside on every node in the data warehouse cluster.

The STL tables take the information from the logs and format them into usable tables for system administrators.

To manage disk space, the STL log tables only retain approximately two to five days of log history, depending on log usage and available disk space.

If you want to retain the log data, you will need to periodically copy it to other tables or unload it to Amazon S3

https://docs.aws.amazon.com/redshift/latest/dg/c_intro_STL_tables.html

Option E is correct - There are two types of system tables: STL and STV tables.

STL tables are generated from logs that have been persisted to disk to provide a history of the system.

STV tables are virtual tables that contain snapshots of the current system data.

They are based on transient in-memory data and are not persisted to disk-based logs or regular tables.

System views that contain any reference to a transient STV table are called SVV views.

Views containing only references to STL tables are called SVL views.

System tables and views do not use the same consistency model as regular tables.

It is important to be aware of this issue when querying them, especially for STV tables and SVV views

https://docs.aws.amazon.com/redshift/latest/dg/c_types-of-system-tables-and-views.html

Option F is incorrect -There are two types of system tables: STL and STV tables.

STL tables are generated from logs that have been persisted to disk to provide a history of the system.

STV tables are virtual tables that contain snapshots of the current system data.

They are based on transient in-memory data and are not persisted to disk-based logs or regular tables.

System views that contain any reference to a transient STV table are called SVV views.

Views containing only references to STL tables are called SVL views.

System tables and views do not use the same consistency model as regular tables.

It is important to be aware of this issue when querying them, especially for STV tables and SVV views

https://docs.aws.amazon.com/redshift/latest/dg/c_types-of-system-tables-and-views.html

Option G is correct -There are two types of system tables: STL and STV tables.

STL tables are generated from logs that have been persisted to disk to provide a history of the system.

STV tables are virtual tables that contain snapshots of the current system data.

They are based on transient in-memory data and are not persisted to disk-based logs or regular tables.

System views that contain any reference to a transient STV table are called SVV views.

Views containing only references to STL tables are called SVL views.

System tables and views do not use the same consistency model as regular tables.

It is important to be aware of this issue when querying them, especially for STV tables and SVV views

https://docs.aws.amazon.com/redshift/latest/dg/c_types-of-system-tables-and-views.html

Sure, I'd be happy to explain the system tables and views that support the metadata of the Redshift architecture.

Amazon Redshift is a data warehouse service that is designed to handle large-scale data analytics. Redshift provides a range of system tables and views that can be used by administrators and users to understand the metadata of the Redshift architecture.

The system tables and views in Redshift can be divided into two categories: STL tables and STV tables.

  1. STL Tables: STL tables are generated from logs that have been persisted to disk to provide a history of the system. These files reside on every node in the data warehouse cluster. The STL tables take the information from the logs and format them into usable tables for system administrators. These tables are used to monitor the activity and performance of the Redshift cluster. Some of the common STL tables are:
  • stl_alert_event_log: Contains information about all alerts that have been generated by the Redshift cluster.
  • stl_connection_log: Contains information about all connections that have been made to the Redshift cluster.
  • stl_query: Contains information about all queries that have been executed on the Redshift cluster.
  • stl_load_commits: Contains information about all successful data loads into the cluster.
  • stl_error: Contains information about all errors that have occurred on the Redshift cluster.
  1. STV Tables: STV tables are virtual tables that contain snapshots of the current system data. These tables are used to monitor the current state of the Redshift cluster. Some of the common STV tables are:
  • stv_blocklist: Contains information about blocks that have been marked as corrupted or inaccessible.
  • stv_partitions: Contains information about all partitions in the Redshift cluster.
  • stv_tbl_perm: Contains information about all permissions that have been granted on tables in the Redshift cluster.
  • stv_tbl_stats: Contains statistics about the tables in the Redshift cluster.
  • stv_inflight: Contains information about queries that are currently executing on the Redshift cluster.

It's important to note that system tables and views do not use the same consistency model as regular tables. Data in system tables and views may not be immediately consistent with the data in regular tables, so it's important to understand the limitations of these tables and views.

Lastly, system views that contain any reference to a transient STV table are called SVV views and references to STL tables are called SVL views. This naming convention helps to differentiate between system views that rely on STL and STV tables.

In summary, system tables and views are an important aspect of monitoring and understanding the metadata of the Redshift architecture. STL tables are generated from logs and provide a history of the system, while STV tables contain snapshots of the current system data. It's important to understand the consistency model of these tables and views, and the naming convention for system views that rely on STL and STV tables.