Search Results

4 matches found for 'Apache Spark'

Big Data Processing: Batching vs. Streaming

... vs. alternatives Nowadays, a popular framework for batch data processing is Hadoop using Apache Spark for the dataflow engine. Spark was developed as a more performant alternative to the traditional Hadoop + MapReduce, by leveraging the memory (RAM) of distributed machines to compute workflows.

January 3, 2022

A primer on MapReduce

... you to define mappers and reducers without writing code. A more popular alternative to use is Apache Spark, which is about 100x faster than Hadoop, while also allowing you to bypass the need to write explicit mappers and reducers.

July 11, 2020

Comparison Charts of File Storage Formats

Big Data Encodings These encodings are often used with HDFS or some other distributed file system. Since the data can be as large as terabytes or petabytes, it is crucial to encode files in a space optimal way and also allow themselves to be read or written in an optimal way.

January 24, 2022

Data stores in Software Architectures

Use Cases There are many ways to store your data. In this article we'll walk through some examples of data storage in common system designs. Reminder: There is no single best storage choice and they may vary heavily depending on things such as access patterns and scale.

July 19, 2021