How Instagram Scaled to 14 Million Users With Only 3 Engineers

Highlights

• Instagram scaled from 0 to 14 million users in just over a year, from October 2010 to December 2011. They did this with only 3 engineers. (View Highlight)
• Instagram’s Guiding Principles (View Highlight)
• Keep things very simple. (View Highlight)
• Don’t re-invent the wheel. (View Highlight)
• Use proven, solid technologies when possible. (View Highlight)
• Session: A user opens the Instagram app. (View Highlight)
• (View Highlight)
• Session: After opening the app, a request to grab the main feed photos is sent to the backend, where it hits Instagram’s load balancer. (View Highlight)
• (View Highlight)
• Session: The load balancer sends the request to the application server, which holds the logic to process the request correctly. (View Highlight)
• Instagram’s application server used Django and it was written in Python, with Gunicorn as their WSGI server. (View Highlight)
• As a refresher, a WSGI (Web Server Gateway Interface) forwards requests from a web server to a web application. (View Highlight)
• (View Highlight)
• Session: The application server sees that the request needs data for the main feed. For this, let’s say it needs: (View Highlight)
• latest relevant photo IDs (View Highlight)
• the actual photos that match those photo IDs (View Highlight)
• user data for those photos. (View Highlight)
• Session: The application server grabs the latest relevant photo IDs from Postgres. (View Highlight)
• An interesting challenge that Instagram faced and solved is generating IDs that could be sorted by time. Their resulting sortable-by-time IDs looked like this: (View Highlight)
• 41 bits for time in milliseconds (gives us 41 years of IDs with a custom epoch) (View Highlight)
• 13 bits that represent the logical shard ID (View Highlight)
• 10 bits that represent an auto-incrementing sequence, modulus 1024. This means we can generate 1024 IDs, per shard, per millisecond (View Highlight)
• Thanks to the sortable-by-time IDs in Postgres, the application server has successfully received the latest relevant photo IDs. (View Highlight)
• Session: To get the user data from Postgres, the application server (Django) matches photo IDs to user IDs using Redis. (View Highlight)
• Instagram used Redis to store a mapping of about 300 million photos to the user ID that created them, in order to know which shard to query when getting photos for the main feed, activity feed, (View Highlight)
• For general caching, Instagram used Memcached. They had 6 Memcached instances at the time. Memcached is relatively simple to layer over Django. (View Highlight)
• Session: The user now sees the home feed, populated with the latest pictures from people he is following. (View Highlight)
• (View Highlight)
• Session: Now, let’s say the user closes the app, but then gets a push notification that a friend posted a photo. (View Highlight)
• This push notification was sent using pyapns, (View Highlight)
• Pyapns is an open-source, universal Apple Push Notification Service (APNS) provider. (View Highlight)
• Session: The user really liked this photo! So he decided to share it on Twitter. (View Highlight)
• Session: Uh oh! The Instagram app crashed because something erred on the server and sent an erroneous response. The three Instagram engineers get alerted instantly. (View Highlight)
• Final Architecture Overview (View Highlight)
• (View Highlight)