@rkenmi - Search Results

Storing passwords into a database

... in the database Don't put encoded passwords in the database (i.e. Base64) Don't put simple hashed passwords in the database (i.e. MD5, SHA-256) Whys For obvious reasons, putting raw passwords means that the DBA or anyone who has access to the database can steal the passwords.

January 30, 2021

Authentications

... increase number of machines Its 128 bits so it can be too big SHA256 It's a reliable and fast hash It's a hash so you need some input (i.e. you could hash an auto-incremented ID) Still have to worry about hash collisions It's even bigger than UUID (256 bits) Twitter Snowflake It's compact with only 64 bits It's a time ordered ID, so some databases can take advantage of this property Passwords Note: If you are using 3rd party OAuth providers for authorization, and not storing user ids and passwords yourself, then you don't need to read on.

February 3, 2021

Data Sharding: Twitter Posts

... User IDs are around 64 bits in size. Scaling We can start off by considering a basic hash of the Tweet ID and modulo that value with the total number of servers, to determine which server to store that Tweet.

December 28, 2021

Bloom Filter

A set data structure uses a hashing function to store values and to verify if a value exists. Bloom filters are similar in that it uses multiple hashing functions to store values and to verify if a value exists.

October 14, 2019

Sharding Techniques

... very essence of partitioning is to just call a partitioning function that takes the modulo of the hashed key of the database row (i.e. primary ID) by the number of machines \(n\) available for sharding.

July 31, 2020

Unique Permutations

... usage. The better solution is actually not far-fetched from the brute force approach: use a hash table to store new permutations. If we encounter a permutation that is already stored in the hash table, skip the adding of that permutation.

February 18, 2019

Sharding User IDs of Celebrities

... across multiple nodes based on a User ID, a typical partitioning algorithm is to use a basic hash like MD5 to have a reasonably compact (as in, low number of bits) partition ID. For the majority of users, this doesn't pose a problem.

January 15, 2022

Counting the path of sums

... to make our algorithm faster. Basically, we are going to store the running total values into a hash table as we visit each node in the tree. If the values already exist in the hash table, we can look them up in constant time O(1).

June 9, 2017

Algorithm Handbook

... order Hash Tables There are multiple implementations that use a hash table, and they can be quite different. For example, a set and a dictionary (i.e.

June 18, 2017

Java Essentials

... have enough memory to have their own non-static methods. Thus, they do not have a .equals() or .hashCode() implementation, which is needed in order to store them into a HashMap or ArrayList.

March 29, 2019

Design Concepts

... to a specific DB. What if you add more DB hosts? Or remove existing DB hosts? The validity of the hashes will break, since the increase/decrease in host size will affect the hashing algorithm.

November 20, 2019

NoSQL - the Radical Databases

... but we can't say the same for these NoSQL databases. Key-value store What its like: A hash map Keys are maintained in lexicographic order, allowing efficient retrieval of key ranges \(O(1)\) time lookup and writes Hash maps are pretty basic by itself, so it relies on application code if more operations are needed The basis for more complex NoSQL stores such as document stores Examples: Memcached, Redis, DynamoDB Document store What its like: A hash map but with document objects as values (i.

December 31, 2020

Perfect Squares

... - perfSquares[j]), i+1), f(total, i)) \) One optimization is to cache results, using a hash table, and recursively iterate from the smallest perfect square to the biggest perfect square.

February 18, 2019

B-Trees vs. LSM Trees

... as Bitcask, MongoDB and SQLite4. A simple log structured storage works by having an in-memory hash table (or hash index) that keeps track of keys and values. The values would be a byte offset reference inside an append-only log file (that is on disk, not in-memory), which contains the actual value string.

March 14, 2018

Distributed scaling with Relational Databases

... \(A\) is ambiguous. The sharding idea is simple - you can partition your keys based on some hash so that some rows go to one partition, and other rows go to some other partition.

January 19, 2022

Web Development 101

... JWT is typically generated by the server, and signed by the server. The header specifies what the hashing algorithm is to sign the token. The payload indicates information such as which user is logged in and the iat value or the "issue at time".

November 9, 2017

Javascript Essentials

... Empty array let list = []; Array with values let list = [1, 2, 3]; Empty hash map let map = new Map(); // or alternatively let object = {}; Hash map with values let map = new Map(['a', 'b']); let object = {a : 'b'}; Empty set let s = new Set(); Set with values let s = new Set([1, 2, 3]); Min heap goog.

November 12, 2017

NFT from a Software Developer's perspective

... your NFT loses all its value if Dropbox ever goes out of business. NFTs should have a one-way hash of the digital asset in the metadata, which proves validity. Theoretically a hashing algorithm should be non-reversible and represents the output of a single unique input (this would be the digital asset for trade).

December 26, 2021

Python Essentials

... Empty array list = [] Array with values list = [1, 2, 3] Empty hash map hashmap = {} Hash map with values hashmap = {'a': 'b'} Empty set s = set() Set with values s = set([1, 2, 3]) Min heap import heapq q = [] heapq.

April 3, 2019

RDBMS Optimization

... B-Tree, LSM. This works very well when your DB becomes too big. You can do partitioning based on hashes or under custom if/else conditions. Replication Allows you to add more DB servers that are all consistent.

June 23, 2020

Data stores in Software Architectures

... need a multi-master setup to horizontally scale the SQL database with sharding and consistent hashing. Solutions: Document Databases / Columnar Databases (ex: MongoDB, DynamoDB, Cassandra) + Distributed Cache Runner-ups: - Relational Databases (ex: SQL) + Distributed Cache Crypto trends within the last year In this problem, we want to figure out the statistical data of cryptocurrency within the last year, down to a minute's granularity.

July 19, 2021

Useful Links

This is a personal list of useful resources for improving web stacks, frameworks, development, UX, whatever that I come across! Software Engineering The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) Encryption vs.

September 4, 2017

Search Results