Course Overview
The Modern Data Stack

Stream Processing

Lesson #7

In this lesson we will:

  • Learn about batch processing;
  • Learn about stream processing;
  • Cover the difficulties in moving from a batch to a streaming model.

Batch Data Exchange

There are many situations where businesses need to move data from one location to another. For example:

  • Copying data from one application into another e.g. eCommerce into a fullfilment system
  • Copying data into a repository e.g. a Data Warehouse or a Data lake for later analysis
  • Copying data for a partner organisation e.g. Sending orders to a third party warehouse who will fulfil the order
  • Copying data into a location for backup, archive or audit purposes

Traditionally, businesses have done this using a batch model, whereby every hour, day or week, data.

From Batch To Streaming

Streaming technology involves moving data from source to destination as a continuous stream, typically immediately after it has been created.

By doing this, business can get instant visibility and response to situations much earlier.

Join our mailing list for our latest insights on Data Engineering:

Timeflow Academy is the leading online, hands-on platform for learning about Data Engineering using the Modern Data Stack. Bought to you by Timeflow CI

© 2023 Timeflow Academy. All rights reserved