The goal of this article is to clarify what data governance is and how to apply it.
I decided to write it because I saw a lot of articles and blog posts where data governance is pictured as an enemy whose only goal is to slow down any data related activity like analytics. Nothing could be further from the truth.
Data governance is your foundamental tool for any data related activities! This what I will demonstrate in this article.
This article contains:
- a definition of data governance and its goal, and
- a data governance framework
I took inspiration for the…
In my previous article I have described a POC to build a data warehouse using:
- S3, to store the data
- Athena, to ingest and transform the data
- Step functions, to orchestrate the data ingestion and transformation
- Data vault 2.0, as a methodology to implement and design the data warehouse
In this article I’m testing the performance of such design.
The article starts with disclaimers, continues with a quick recap of the previous article, and then deep dive into testing. Conclusions and next steps are at the end.
I assume that you have a good knowledge of AWS (S3, Athena, CLI…
Introduction
This article describes a proof-of-concept architecture for a data warehouse / Data Lake built in AWS using:
- S3, to store the data
- Athena, to ingest and transform the data
- Step functions, to orchestrate the data ingestion and transformation
- Data vault 2.0, as a methodology to implement and design the data warehouse
The article assumes that you have a good knowledge of AWS (S3, Athena, CLI, and Cloudformation).
The article starts with some disclaimers, continues with describing the motivation for such design, what we are going to build, and then how to build it. Conclusions and next steps close the article.
Daniel Mannino
I am a cloud-native analytics architect and my goal is to bring companies from drowning in data to swimming in innovation
