Search Results

8 matches found for 'sharding'

Sharding Techniques

... up the load into multiple machines helps prevent this. Challenges include: Management of sharding, especially in RDBMS where ACID compliance is desired Can be complicated to implement In this article, I cover three sharding techniques to improve the read/write performance of databases.

Data Sharding: Twitter Posts

... it incorporates both a sequence AND a timestamp, so that we can achieve uniform distribution via sharding and also have efficient lookup (with sort) by the database servers. To do this, we can use the Twitter Snowflake ID pattern to reserve at least the first 41 bits for the epoch timestamp and 12 bits for the sequence ID.

Sharding User IDs of Celebrities

Problem When you are partitioning (or sharding) database writes across multiple nodes based on a User ID, a typical partitioning algorithm is to use a basic hash like MD5 to have a reasonably compact (as in, low number of bits) partition ID.

Local Secondary Index vs. Global Secondary Index

Secondary Index A secondary index is used in databases to help speed up queries when we want to grab data from popular columns or if we want to do some type of key range lookup efficiently. Secondary indices are used in relational databases (e.

Distributed scaling with Relational Databases

... about how to scale databases. Typically, they will talk about the purpose and the general idea of sharding and replication, but often times these topics are explained separately and not so much in conjunction.

RDBMS Optimization

Indexing Probably the easiest tweak to implement. It can usually be done with one SQL command. However, an index should be made based on a good column. For example, if you are frequently querying your rows by timestamp, then the timestamp can be chosen for an index.

Data stores in Software Architectures

... for this, meaning we would need a multi-master setup to horizontally scale the SQL database with sharding and consistent hashing. Solutions: Document Databases / Columnar Databases (ex: MongoDB, DynamoDB, Cassandra) + Distributed Cache Runner-ups: - Relational Databases (ex: SQL) + Distributed Cache Crypto trends within the last year In this problem, we want to figure out the statistical data of cryptocurrency within the last year, down to a minute's granularity.

Design Concepts

In this article, I want to go over some fundamental design concepts that are useful for coming up with system design. Requirements Functional Requirements Describes specific behaviors i.e. If a URL is generated, it is composed of a Base64 encoded alias Non-functional Requirements Describes architectural requirements i.