@rkenmi - Search Results

Sharding Techniques

... up the load into multiple machines helps prevent this. Challenges include: Management of sharding, especially in RDBMS where ACID compliance is desired Can be complicated to implement In this article, I cover three sharding techniques to improve the read/write performance of databases.

July 31, 2020

Data Sharding: Twitter Posts

... it incorporates both a sequence AND a timestamp, so that we can achieve uniform distribution via sharding and also have efficient lookup (with sort) by the database servers. To do this, we can use the Twitter Snowflake ID pattern to reserve at least the first 41 bits for the epoch timestamp and 12 bits for the sequence ID.

December 28, 2021

Sharding User IDs of Celebrities

Problem When you are partitioning (or sharding) database writes across multiple nodes based on a User ID, a typical partitioning algorithm is to use a basic hash like MD5 to have a reasonably compact (as in, low number of bits) partition ID.

January 15, 2022

Local Secondary Index vs. Global Secondary Index

Secondary Index A secondary index is used in databases to help speed up queries when we want to grab data from popular columns or if we want to do some type of key range lookup efficiently. Secondary indices are used in relational databases (e.

January 15, 2022

Distributed scaling with Relational Databases

... about how to scale databases. Typically, they will talk about the purpose and the general idea of sharding and replication, but often times these topics are explained separately and not so much in conjunction.

January 19, 2022

RDBMS Optimization

Indexing Probably the easiest tweak to implement. It can usually be done with one SQL command. However, an index should be made based on a good column. For example, if you are frequently querying your rows by timestamp, then the timestamp can be chosen for an index.

June 23, 2020

Data stores in Software Architectures

... for this, meaning we would need a multi-master setup to horizontally scale the SQL database with sharding and consistent hashing. Solutions: Document Databases / Columnar Databases (ex: MongoDB, DynamoDB, Cassandra) + Distributed Cache Runner-ups: - Relational Databases (ex: SQL) + Distributed Cache Crypto trends within the last year In this problem, we want to figure out the statistical data of cryptocurrency within the last year, down to a minute's granularity.

July 19, 2021

Design Concepts

In this article, I want to go over some fundamental design concepts that are useful for coming up with system design. Requirements Functional Requirements Describes specific behaviors i.e. If a URL is generated, it is composed of a Base64 encoded alias Non-functional Requirements Describes architectural requirements i.

November 20, 2019