Search Results

3 matches found for 'Apache Hadoop'

Big Data Processing: Batching vs. Streaming

Intro In data processing, we often have to work with large amounts of data. The way in which this data is gathered comes in a few variants: batching, where we aggregate a collection of data (e.g., by hourly time), streaming for data that needs to be processed in real-time, and a unified variant which simply does not distinguish the technical difference between batching and streaming, allowing you to programmatically use the same API for both.

January 3, 2022

A primer on MapReduce

... itself, but more so a concept. The most popular implementation of Google's MapReduce paper is Apache Hadoop. Although the code implementation of MapReduce is rather cryptic, it represents the underlying foundation of tools such as Hadoop or Spark.

July 11, 2020

Data stores in Software Architectures

Use Cases There are many ways to store your data. In this article we'll walk through some examples of data storage in common system designs. Reminder: There is no single best storage choice and they may vary heavily depending on things such as access patterns and scale.

July 19, 2021