Hands-On Training For Cloud Data Engineers
Timeflow Academy is an online, hands-on platform for learning about Data Engineering and modern Cloud Native Databases including tools such as DBT, Clickhouse, Snowflake, Kafka, Spark and Airflow...
The Modern Data Stack
Data Lakes In The Modern Data Stack
In this lesson we will learn more about Data Lakes and their role as part of the Modern Data Stack.
ViewIntroduction To The Modern Data Stack
In this lesson we will introduce the concept of the Modern Data Stack or Modern Data Platform
ViewArchitecture Of The Modern Data Stack
In this lesson we will discuss architectural considerations and patterns when building a Modern Data Platform
ViewCloud And The Modern Data Stack
In this lesson we will discuss how cloud influences and drives the modern data platform
ViewIngesting Data Into A Modern Data Stack
In this lesson we will discuss how data is typically sourced and integrated into Modern Data Stack platforms
ViewData Transformations
In this lesson we will discuss the process of Extract, Load and Transform as it is used in the modern data stack
ViewAnalytics and Business Intelligence
In this lesson we will consider Data analysis and Visualisation in the Modern Data Platform
ViewData Lakes In The Modern Data Stack
In this lesson we will learn more about Data Lakes and their role as part of the Modern Data Stack.
ViewIntroduction To The Modern Data Stack
In this lesson we will introduce the concept of the Modern Data Stack or Modern Data Platform
ViewArchitecture Of The Modern Data Stack
In this lesson we will discuss architectural considerations and patterns when building a Modern Data Platform
ViewCloud And The Modern Data Stack
In this lesson we will discuss how cloud influences and drives the modern data platform
ViewIngesting Data Into A Modern Data Stack
In this lesson we will discuss how data is typically sourced and integrated into Modern Data Stack platforms
ViewData Transformations
In this lesson we will discuss the process of Extract, Load and Transform as it is used in the modern data stack
ViewAnalytics and Business Intelligence
In this lesson we will consider Data analysis and Visualisation in the Modern Data Platform
ViewData Lakes In The Modern Data Stack
In this lesson we will learn more about Data Lakes and their role as part of the Modern Data Stack.
ViewDBT For Data Engineers
Introduction To DBT
In this lesson we will introduce DBT, explain the value it brings and explain some of the core concepts behind the platform
ViewCreating A DBT Project
In this lesson we will begin using the DBT Command Line Interface. We will then create and configure our first DBT project.
ViewConfiguring DBT Profiles
In this lesson we will learn more about the DBT profile system and best practices for managing profiles for maintainable code.
ViewExecuting Your First Transformations
In this lesson we will implement and run our first transformations using DBT models that build both tables and views.
ViewDBT Views and Tables
In this lesson we will learn how DBT can materialise to tables and views, and the associated incremental and ephemeral options.
ViewIncremental Views
In this lesson we will learn about DBT incremental updates or incremental views.
ViewSources and Exposures
In this lesson we will learn about DBTs source and exposure features.
ViewIntroduction To DBT
In this lesson we will introduce DBT, explain the value it brings and explain some of the core concepts behind the platform
ViewCreating A DBT Project
In this lesson we will begin using the DBT Command Line Interface. We will then create and configure our first DBT project.
ViewConfiguring DBT Profiles
In this lesson we will learn more about the DBT profile system and best practices for managing profiles for maintainable code.
ViewExecuting Your First Transformations
In this lesson we will implement and run our first transformations using DBT models that build both tables and views.
ViewDBT Views and Tables
In this lesson we will learn how DBT can materialise to tables and views, and the associated incremental and ephemeral options.
ViewIncremental Views
In this lesson we will learn about DBT incremental updates or incremental views.
ViewSources and Exposures
In this lesson we will learn about DBTs source and exposure features.
ViewKafka For Data Engineers
What Is Kafka Streams?
Kafka Streams is the component of the Kafka ecosystem which can be used for stream processing. This involves taking messages from Kafka, usually as they are produced in real time, and processing or responding to them in some way.
ViewIntroduction To Kafka
Introduction to Apache Kafka including some of the core concepts and use cases.
ViewCore Concepts Of Kafka
In this lesson, we introduce some of the core concepts associated with Kafka.
ViewSetting Up Your Kafka Broker
In this lesson we will setup our Kafka broker in standalone mode.
ViewAdministering Kafka Topics
In this lesson we will learn about Kafka Topics, including what they are and how to configure and optimise them.
ViewConsuming Data From Kafka
In this lesson we will learn about consuming data in Kafka.
ViewKafka Partitions
In this lesson we will learn about Kafka partitions, which are the one of the main ways in which we can improve parallelism and therefore performance of your Kafka deployment.
ViewKafka Performance Test Scripts
In this lesson we will learn about the Kafka performance test scripts, which can be used for measuring the performance of your Kafka deployment for both producers and consumers.
ViewWhat Is Kafka Streams?
Kafka Streams is the component of the Kafka ecosystem which can be used for stream processing. This involves taking messages from Kafka, usually as they are produced in real time, and processing or responding to them in some way.
ViewIntroduction To Kafka
Introduction to Apache Kafka including some of the core concepts and use cases.
ViewCore Concepts Of Kafka
In this lesson, we introduce some of the core concepts associated with Kafka.
ViewSetting Up Your Kafka Broker
In this lesson we will setup our Kafka broker in standalone mode.
ViewAdministering Kafka Topics
In this lesson we will learn about Kafka Topics, including what they are and how to configure and optimise them.
ViewConsuming Data From Kafka
In this lesson we will learn about consuming data in Kafka.
ViewKafka Partitions
In this lesson we will learn about Kafka partitions, which are the one of the main ways in which we can improve parallelism and therefore performance of your Kafka deployment.
ViewKafka Performance Test Scripts
In this lesson we will learn about the Kafka performance test scripts, which can be used for measuring the performance of your Kafka deployment for both producers and consumers.
ViewWhat Is Kafka Streams?
Kafka Streams is the component of the Kafka ecosystem which can be used for stream processing. This involves taking messages from Kafka, usually as they are produced in real time, and processing or responding to them in some way.
ViewClickhouse For Data Engineers
Materialised Views
In this lesson we will introduce materialised views, and demonstrate how they are more powerful than materialsied views as implemented by other databases.
ViewIntroduction To Clickhouse
In this lesson we will introduce Clickhouse and cover it's distinctive features.
ViewConnecting To Clickhouse
In this lesson we will connect to Clickhouse, and learn about the different deployment options.
ViewAdministering Database Objects
In this lesson we will learn about some of the core database objects in Clickhouse.
ViewClickhouse Role Based Access Control (RBAC)
In this lesson we will look at the Clickhouse users, role and permission model
ViewConnecting Clickhouse To Kafka
In this lesson we will learn how to connect Clickhouse to Kafka to ingest in real time streams of messages.
ViewMaterialised Views
In this lesson we will introduce materialised views, and demonstrate how they are more powerful than materialsied views as implemented by other databases.
ViewIntroduction To Clickhouse
In this lesson we will introduce Clickhouse and cover it's distinctive features.
ViewConnecting To Clickhouse
In this lesson we will connect to Clickhouse, and learn about the different deployment options.
ViewAdministering Database Objects
In this lesson we will learn about some of the core database objects in Clickhouse.
ViewClickhouse Role Based Access Control (RBAC)
In this lesson we will look at the Clickhouse users, role and permission model
ViewConnecting Clickhouse To Kafka
In this lesson we will learn how to connect Clickhouse to Kafka to ingest in real time streams of messages.
ViewMaterialised Views
In this lesson we will introduce materialised views, and demonstrate how they are more powerful than materialsied views as implemented by other databases.
ViewSnowflake For Data Engineers
Resource Monitors
In this lesson we will learn about Snowflakes Resource Monitor feature
ViewIntroduction To Snowflake
In this lesson we will introduce Snowflake and describe some of it's key differentiators and architectural features.
ViewSnowflake Accounts
In this lesson we will learn about the different types of Snowflake account and the choices that you need to make.
ViewSnowflake Tasks
In this lesson we will learn about Snowflake tasks, which allow us to run database jobs on a schedule.
ViewSnowflake Streams
In this lesson we will learn about snowflake streams, which allow us to resopnd to database DML statements.
ViewReferential Integrity In Snowflake
In this lesson we will learn about referential integrity in Snowflake.
ViewUnderstanding Snowflake Load and Performance
In this lesson we will learn about how to understand snowflake load and performance.
ViewLoading Data Into Snowflake
In this lesson we will learn about batch loading data into snowflake and how to use staging areas.
ViewResource Monitors
In this lesson we will learn about Snowflakes Resource Monitor feature
ViewIntroduction To Snowflake
In this lesson we will introduce Snowflake and describe some of it's key differentiators and architectural features.
ViewSnowflake Accounts
In this lesson we will learn about the different types of Snowflake account and the choices that you need to make.
ViewSnowflake Tasks
In this lesson we will learn about Snowflake tasks, which allow us to run database jobs on a schedule.
ViewSnowflake Streams
In this lesson we will learn about snowflake streams, which allow us to resopnd to database DML statements.
ViewReferential Integrity In Snowflake
In this lesson we will learn about referential integrity in Snowflake.
ViewUnderstanding Snowflake Load and Performance
In this lesson we will learn about how to understand snowflake load and performance.
ViewLoading Data Into Snowflake
In this lesson we will learn about batch loading data into snowflake and how to use staging areas.
ViewResource Monitors
In this lesson we will learn about Snowflakes Resource Monitor feature
ViewSpark For Data Engineers
From Spark To Databricks
In this lesson we will learn about the differences between Spark and Databricks, and the advantages of moving from Spark to Databricks.
ViewIntroduction To Apache Spark
In this lesson we will introduce Apache Spark and cover some of the core concepts and use cases.
ViewLoading Static Files
In this lesson we will introduce Apache Spark and describe how to load data from static CSV and JSON files.
ViewGroup By and Aggregation
In this lesson we will learn about basic group by and aggregation operations in Spark.
ViewSpark Query Plan
In this lesson we will learn about the Spark query plan and how to interpret it.
ViewFrom Spark To Databricks
In this lesson we will learn about the differences between Spark and Databricks, and the advantages of moving from Spark to Databricks.
ViewIntroduction To Apache Spark
In this lesson we will introduce Apache Spark and cover some of the core concepts and use cases.
ViewLoading Static Files
In this lesson we will introduce Apache Spark and describe how to load data from static CSV and JSON files.
ViewGroup By and Aggregation
In this lesson we will learn about basic group by and aggregation operations in Spark.
ViewSpark Query Plan
In this lesson we will learn about the Spark query plan and how to interpret it.
ViewFrom Spark To Databricks
In this lesson we will learn about the differences between Spark and Databricks, and the advantages of moving from Spark to Databricks.
View