Background In Redshift, there are 2 primary methods for loading data into a Redshift table: INSERT and COPY commands. One might ask, what’s the difference? When is it best to use which one? INSERT Command Per the official AWS docs, INSERT is a SQL command used to, well, insert data row-by-row into a…

Background As of March 2021, I shifted my career domain from full-stack web development into the data track. Fast-forward to now, the three main data storage services I’ve used most heavily include Redshift, RDS, and DynamoDB. In the beginning, although I was quite sure how each of them was unique in…

Use Case Requirements You have an occasional need to copy and / or unload data from an S3 bucket from one account to a Redshift cluster in another account. This can be achieved through Redshift’s COPY & UNLOAD commands in combination with IAM Roles and Role chaining. Background IAM Role chaining also works…

The diagram above shows how the Redshift Data Share feature enables you to share data across Redshift clusters in different AWS accounts. Here’s how it works: Redshift data share creates a shared VPC endpoint in the producer account that acts as a gateway for the consumer account to access the producer Redshift cluster. The…

Use Case / Requirements You are tasked with joining / integrating data across two separate data stores (i.e. Redshift and S3) stored in a Single AWS account for deriving custom analytics. High-Level Solution Design

Background With any cloud service, it is important to secure your credentials to protect your data/service from unauthorized access. In this segment, I will demonstrate how to use Python to generate ephemeral credentials for authenticating to Amazon Redshift. And also explain why ephemeral credentials are safer than static credentials. What are Ephemeral Credentials? 🤔 Ephemeral credentials…

For the past few months, I have been developing with Amazon Redshift. Redshift is a data warehousing service that uses SQL to analyze structured and semi-structured data across data warehouses, operational and analytical data sources, and data lakes. Unlike traditional data warehouse solutions, Redshift offers a wide range of features…

Redshift offers Data sharing, Data mesh, and Spectrum. But, when to implement which one? Background Cloud data consumption patterns are essential for any organization’s business intelligence operations. The ability to access and analyze data quickly, accurately, and cost-effectively is paramount for success. …

Background Amazon Redshift is a cloud-based data warehousing solution that allows organizations to store and analyze large volumes of data efficiently. One of the key offerings of Redshift is its ability to handle data sharing between different Redshift clusters, which is critical for organizations that need to share data between multiple…

Background One of Redshift’s key features is the ability to create different types of Views. Views are virtual tables that are based on the result of a SQL query. In this post, I’ll do a basic walkthrough of the three common types of views available in Redshift, and when to use…

Background Redshift is a powerful cloud-based data warehousing solution provided by AWS. It is designed to handle large amounts of data (petabytes), and support complex analytical queries. While many of its features are well-known, there are several aspects of Redshift that are not widely understood. In this segment, I will explore…