@rkenmi - Search Results

Scaling Instragram Infrastructure

Notes Sending notifications to a person whose photo you liked: RabbitMQ -> Celery Django / Python for web server / application PostgreSQL to store users, medias, friendships, etc.

December 26, 2020

Data stores in Software Architectures

... into Spark Streaming and store the output into a database like Cassandra or a message queue like RabbitMQ for notifications. Services can then query the output database or receive the messages from the message queue to return real time data back to the user Since we have big data with respect to time, we can use something like a Time Series Database (TSDB) such as InfluxDB, rather than Data Warehouse or Cassandra.

July 19, 2021

Design Concepts

... Publisher/Subscriber Model is a pretty popular design pattern, and message queues such as RabbitMQ are also a very popular implementation of that. With message queues, subscribers can constantly poll for the result of their request.

November 20, 2019

Apache Kafka and Event Streaming

... to a topic. In Kafka, you can swap broker implementations to your liking - if you want to use RabbitMQ, then go for it Producer The producer pushes events to the topics and their partitions Consumer The consumer pulls events from the topics The consumer needs to also commit the pulled events (similar to an ACK) so that old events do not get replayed over and over Partitions in a topic can be load-balanced within a group of consumers (consumer group) Use Cases There are numerous real-world use cases for Kafka, such as: Logging Service Emit logging events on every transaction.

March 31, 2021

Traditional Message Queues vs. Log-based Message Brokers

... but also some very important key differences. Message Queue (JMS / AMQP based) Examples: RabbitMQ, Amazon SQS Messages are not durable, typically Messages are inserted and removed into the queue with low latency No ordering guarantee! Messages are ACK'ed by consumers before they are removed from the queue.

January 20, 2022