High Level Design

Design a News Feed System

Design a Facebook-style news feed for 10M DAU, balancing fast reads against write amplification with hybrid push/pull fanout, ID-only feed caches and celebrity-aware delivery.

December 1, 2025·Updated October 24, 2026

A news feed is the heart of any social media platform: a constantly updating list of status updates, photos, videos, likes and activity from friends, groups and pages. In this post we design a feed similar to Facebook News Feed, Instagram Feed and Twitter Timeline, for both web and mobile, covering fanout strategies, caching and the optimisations needed at scale.


Step 1: Requirements#

Clarifying Questions#

QuestionAnswer
Mobile or Web?Both
Core features?Users can publish posts & view friends' posts
Feed sorting?Reverse chronological order
Max friends per user?5,000
Traffic?10M Daily Active Users
Content types?Images, videos, text

Functional Requirements#

#Requirement
1Users can publish a post (text, images, videos)
2Users can view a feed of their friends' posts
3The feed is in reverse chronological order
4Users can mute friends or restrict who sees a post, and the feed respects those settings
5Friends are notified when a user posts

Non-Functional Requirements#

#RequirementDetail
1Low read latencyFeed loads in under 200 ms; reads far outnumber writes
2Scalability10M DAU; users with up to 5,000 friends, plus celebrity accounts with millions of followers
3High availabilityThe feed should keep loading even if parts of the write path are degraded
4Eventual consistencyA new post can take a few seconds to reach friends' feeds
5DurabilityA published post must never be lost, even if fanout fails and has to retry

Estimations#

Assuming 10M DAU, each loading their feed ~10 times a day, ~20% of users posting once a day, and an average of ~200 friends:

MetricCalculationResult
Feed reads/day10M × 10100M
Feed read QPS (peak ≈ 3× avg)100M / 86,400 × 3~3,500/s
Posts/day10M × 20%2M
Post write QPS (peak)2M / 86,400 × 3~70/s
Fanout cache writes/day2M × 200 friends400M (~14,000/s at peak)
Feed cache memory10M users × 500 IDs × ~64 B per entry~320 GB
Post metadata storage/day2M × 1 KB~2 GB/day

The numbers explain the shape of the design. Reads are ~50× writes, and every post becomes ~200 cache writes, so feeds should be precomputed at write time and kept in a cache that holds only IDs.


Step 2: Entities#

EntityResponsibility
UserProfile (name, avatar), settings such as muted friends
PostContent, author ID, creation time, visibility settings, media references
FriendshipAn edge in the social graph between two users (or a follow edge for celebrities)
Feed EntryA <post_id, user_id> pair in a user's precomputed feed, ordered by time
MediaReference (URL, type, size) to an image or video in object storage
ActionA like, comment or share on a post, along with its counters

Step 3: APIs#

Publish a post#

POST /v1/feed

ParamDescription
contentText or media of the post
auth_tokenAuthenticates the request

Returns the new post_id. The post is persisted right away, and fanout to friends happens asynchronously.

Retrieve the feed#

GET /v1/feed

ParamDescription
auth_tokenAuthenticates the request
before(optional) Cursor: return posts older than this post (scrolling down)
since(optional) Cursor: return posts newer than this post (refreshing)
limit(optional) Page size, default 20

Returns a page of fully hydrated posts (author name, avatar, content, media URLs, counts) as JSON, plus the cursors for the next request.


Step 4: Data Flow#

The system splits into two flows.

Feed publishing#

When a user makes a post:

  1. The post is stored in the database and cache.
  2. It's delivered (or "fanned out") to friends' feeds.
  3. Friends receive notifications.

Feed Publishing Flow

Feed retrieval#

When a user opens the app:

  1. Their feed (a list of post IDs) is fetched from the news feed cache.
  2. Posts are assembled with metadata (user info, content, images, etc.) and returned.

Feed Retrieval Flow


Step 5: High-Level Design#

Starting simple: one server, one database#

The simplest version is a single web server and a single relational database. Publishing a post is one INSERT. Building a feed is one query: find the user's friends, join against the posts table, sort by created_at DESC and take the top 20.

That breaks down quickly:

ProblemWhy
Slow feed readsEvery feed load joins up to 5,000 friends against the posts table and sorts the result, ~3,500 times a second at peak
Single point of failureIf the server or the database goes down, nobody can post or read
Reads and writes competeA burst of posts slows down feed reads, and the two can't be scaled separately
CelebritiesA celebrity's posts appear in millions of feeds, so the same rows are read over and over

Improved design#

Feed Publishing Deep Dive

The improved design moves the expensive work from read time to write time, and puts caches in front of everything:

  • Load balancer + stateless web servers handle authentication and rate limiting, and can be scaled horizontally.
  • Post service persists posts to the Post DB and post cache.
  • Fanout service looks up the author's friends in the graph DB and puts a fanout job on a message queue.
  • Fanout workers consume the queue and write the post ID into each friend's news feed cache.
  • Notification service tells friends there's a new post.
  • News feed service reads post IDs from the news feed cache and hydrates them from the user and post caches.

Feed reads never touch the friends × posts join. They are a cache lookup plus a hydration step.


Step 6: Deep Dive#

Web Servers#

ConcernDetail
AuthenticationOnly users signed in with a valid auth_token are allowed to make posts.
Rate LimitingLimits the number of posts per user within a time window to prevent spam and abuse.

Fanout models#

Fanout = distributing a user's post to all their friends. There are two models:

Fanout on Write (Push Model)#

The system precomputes the feed during the write operation.

When Alice posts → her post is immediately pushed into all her friends' feed caches.

✅ ProsSuper-fast read performance; very low latency for feed retrieval
❌ ConsHigh write amplification; causes hotkey problems; large memory usage

Hotkey Problem: A key that gets disproportionately high reads/writes compared to others. A celebrity with 10M followers posts → the system must update 10M feed caches, overwhelming the cluster.

Fanout on Read (Pull Model)#

The system computes the feed during read, not write.

When a user opens their feed → posts are pulled from all friends' timelines on demand.

✅ ProsLess storage; no hotkey issue; good for inactive users
❌ ConsSlower reads; expensive during peak traffic; high read amplification

Hybrid Approach (Used by Instagram, Twitter)#

User TypeStrategyReason
Regular usersFanout on WriteFast reads for active, well-connected users
Celebrity usersFanout on ReadAvoids hotkey issues with consistent hashing

This approach provides the best balance between write efficiency and read performance.

Fanout Service — Closer Look#

Fanout Service

Here's exactly what happens inside the fanout service when a post is published:

  1. Fetch friend IDs from the graph database. Graph databases are purpose-built for managing friend relationships and friend recommendations — they make traversals over social graphs fast.

  2. Get friends' info from the user cache. The system then filters out friends based on user settings. For example, if you've muted someone, their posts won't show up on your feed even though you're still friends. Similarly, a user can selectively share a post with specific friends or hide it from others.

  3. Send the friends list and new post ID to the message queue. This decouples the fanout work from the write path — the post is persisted immediately, and the distribution happens asynchronously.

  4. Fanout workers consume from the message queue and store news feed data in the news feed cache. Think of the news feed cache as a <post_id, user_id> mapping table. Only IDs are stored (not full objects) to keep memory consumption manageable. A configurable limit caps how many entries are kept per user — most users only care about the latest content, so the cache miss rate stays low.

  5. Store <post_id, user_id> pairs in the news feed cache.

NewsFeed Retrieval — Deep Dive#

Feed Retrieval Deep Dive

When a user opens their app and the feed loads, here's the full journey:

  1. The user sends a request to retrieve their news feed: GET /v1/feed
  2. The load balancer redistributes the request to an available web server.
  3. The web server calls the news feed service to fetch the feed.
  4. The news feed service retrieves a list of post IDs from the news feed cache (the <post_id, user_id> store built by the fanout workers).
  5. A user's feed is more than just IDs — it contains usernames, profile pictures, post content, images, etc. So the news feed service fetches the complete user and post objects from the user cache and post cache to construct a fully hydrated feed.
  6. The fully hydrated feed is returned as JSON to the client for rendering.

This two-step approach (IDs first, then hydration) keeps the cache lean while still serving rich responses.

Cache Architecture#

The system heavily relies on caching to achieve low latency.

Cache TypePurpose
News Feed CacheStores <post_id, user_id> pairs for each user's feed
Content CacheFull post data; popular content stored in hot cache
Social Graph CacheFriend/follow relationships
Action CacheLikes, replies, and other user interactions on posts
Counters CacheLike count, follower count, reply count, etc.

Step 7: DB Choices#

Different components in the news feed system have very different access patterns — choosing the right database per use case is critical for performance at scale.

ComponentDatabaseWhy
Post dataMySQL / PostgreSQLStructured schema; relational queries on post metadata (author, timestamp, content)
User profilesMySQL / PostgreSQLRelational data; fast indexed lookups by user ID
Social graph (friendships)Neo4j / Amazon NeptuneGraph traversals (friends-of-friends, mutual connections) are natively efficient — far cheaper than SQL joins at scale
News feed cacheRedis (Sorted Set)O(log N) inserts and range reads; sorted sets keep feed IDs in chronological order by timestamp score
Post & user cacheRedisLow-latency key-value lookups for hydrating feed responses
Counters (likes, followers)RedisAtomic INCR / DECR operations avoid race conditions at high write throughput
Media (images, videos)S3 / Blob StorageCheap, durable object storage; CDN sits in front for fast global delivery
Message queue (fanout)Kafka / RabbitMQDurable, high-throughput pub/sub; decouples the write path from fan-out workers

Why separate SQL from cache? SQL is the source of truth — durable, consistent, queryable. Redis is the fast path — it serves the hot read traffic so the database never gets hammered on every feed load.

Why not store full posts in the news feed cache? The feed cache only holds <post_id, user_id> pairs. Full post objects live in the Content Cache (Redis) and the Post DB. This keeps the feed cache small and lets post data be updated in one place without invalidating every user's feed.


Step 8: Follow-Ups#

1. A friend posts while your feed is already open#

Fanout on write puts Bob's new post ID into Alice's news feed cache within seconds. But Alice loaded her feed a minute ago, and her client is still showing that old snapshot. Nothing tells the client the cache has changed. How do we get the post to her?

Step 1: Tell the client something is new. There are three common ways to do it:

ApproachHow it worksTrade-off
Short pollingClient calls GET /v1/feed?since=<newest_post_id> every N secondsSimple and stateless, but most calls come back empty, and that adds up at 10M DAU
Long pollingServer holds the request open until something new arrives or a timeout firesFewer empty responses, but each held request ties up a server connection
WebSocket / SSEServer pushes an event over a persistent connectionReal-time and efficient, but needs a stateful connection tier

Most large platforms use a persistent connection for active sessions and fall back to polling when it drops.

Step 2: Wire it into the fanout path. Fanout workers already know who each post goes to. After writing <post_id, user_id> to the news feed cache, a worker checks a presence service to see whether that friend is online. If they are, it publishes a small event to the real-time gateway, the server tier that holds the open WebSocket connections:

json
{ "type": "new_posts", "count": 1, "newest_post_id": "98213" }

The event is only a hint. It carries no post content, so the gateway stays thin and no hydration happens unless the user actually asks for the posts.

Step 3: Don't move the feed under the user. If new posts are inserted at the top while Alice is reading, everything she's looking at jumps down the screen. Instead, the client shows a "1 new post" pill. When she taps it (or pulls to refresh), the client calls:

GET /v1/feed?since=<newest_post_id_on_screen>

and puts the returned posts above what she already has.

Step 4: Paginate with cursors, not offsets. If the client fetches older posts with ?page=2 while new ones keep arriving at the top, every page boundary shifts and posts show up twice. Using before=<oldest_post_id_on_screen> for scrolling down and since=<newest_post_id_on_screen> for refreshing keeps both directions stable however many posts arrive.

The celebrity case. Celebrity posts use fanout on read, so they never land in Alice's news feed cache, and pushing an event to millions of online followers at once is the same hotkey problem again. For these accounts, skip the push. The client checks on a jittered timer (e.g. every 60–120s), and the news feed service merges the celebrity's latest posts into the since= response.

ScenarioWhat happens
Friend (regular user) posts, Alice onlineFanout worker → presence check → gateway pushes hint → "New posts" pill
Friend posts, Alice offlineOnly the news feed cache is updated; she sees it on her next feed load
Celebrity postsNo push; picked up by the next jittered refresh and merged on read
Several friends post in a burstGateway coalesces hints per user (e.g. at most one per 10s) and only bumps the count
Post deleted before Alice refreshesHydration skips IDs missing from the post cache/DB, so it never appears

2. How is the feed kept in chronological order?#

The requirements ask for reverse chronological order. The feed is never sorted at read time. It's stored in order to begin with, and the structure that does this is a Redis sorted set, one per user.

PartValue
Keyfeed:{user_id}
Memberpost_id
ScoreThe post's creation time in milliseconds

Redis keeps every sorted set ordered by score, so the newest posts are always at the top. The operations line up with the feed's needs:

OperationRedis commandCost
Fanout worker adds a postZADD feed:alice 1729760000000 98213O(log N)
Cap the feed at the latest 500 IDsZREMRANGEBYRANK feed:alice 0 -501O(log N + removed)
First page (newest 20)ZREVRANGE feed:alice 0 19O(log N + 20)
Scroll down (before= cursor)ZREVRANGEBYSCORE feed:alice (<cursor> -inf LIMIT 0 20O(log N + 20)
Refresh (since= cursor)ZRANGEBYSCORE feed:alice (<cursor> +infO(log N + new)

Why not a plain list? Fanout is asynchronous. If Bob posts at 10:00:01 and Carol at 10:00:02, Carol's fanout message might be processed first. With a list (LPUSH), posts end up in the order they arrived, which is wrong. A sorted set orders them by the score they carry, so a late write still lands in the right position.

Score choice. A time-sortable ID such as a Snowflake ID would seem like the obvious score, but Redis scores are 64-bit floats and are only exact up to 2⁵³. A full 64-bit Snowflake ID can lose precision and sort wrongly. Use the millisecond timestamp (or just the timestamp bits of the ID) as the score. Redis orders members with the same score lexicographically, which gives a stable tie-break.

Celebrity posts. Celebrities are on fanout on read, so their posts aren't in feed:alice. Each celebrity has their own sorted set, posts:{celebrity_id}, with the same timestamp scores. At read time the news feed service takes the top 20 from Alice's feed and from each celebrity she follows, then does a k-way merge by score. Every input is already sorted, so the merge is cheap.

What's the source of truth? The sorted set is a cache. Posts live in the Post DB, indexed on (author_id, created_at DESC). If a user's feed cache is evicted (e.g. an inactive user), the news feed service rebuilds it: fetch friend IDs from the graph DB, read each friend's latest posts through that index, merge them by created_at, and repopulate the sorted set. If you want a durable store of precomputed feeds too, Cassandra works well: partition by user_id with clustering order created_at DESC, and each user's feed is stored on disk already sorted.

Ranked feeds: Nothing here is tied to time. Switch the score from created_at to a ranking score (engagement, affinity, recency decay) and the same sorted set can serve a ranked feed. The catch is that ranking scores change over time (a post gains likes, recency decays), so they need to be recomputed periodically instead of being written once at fanout.

3. Deletes, unfriends and mutes after fanout#

Fanout on write has a downside: by the time Bob deletes a post, its ID is already sitting in up to 5,000 feed caches. Walking all of them to remove it is the same write amplification again, in reverse.

EventApproach
Post deletedMark the post deleted in the Post DB and evict it from the post cache. The stale ID stays in feed caches, but hydration drops any ID whose post is missing or deleted. A low-priority background job can clean the feed caches later with ZREM.
Alice unfriends or mutes BobFilter at read time: the news feed service drops posts whose author is in Alice's muted/unfriended set (from the user cache). Optionally run a one-off ZREM of Bob's recent post IDs from feed:alice, which only touches one feed.
Post visibility changedRe-check visibility during hydration, so a post narrowed to "close friends" disappears for everyone else on their next load.

The pattern is the same each time: write-time fanout is an optimisation, read-time filtering is what guarantees correctness. One side effect is that a page can come back with fewer than 20 posts after filtering, so the news feed service over-fetches (e.g. reads 30 IDs to return 20).


Summary#

Designing a scalable news feed system requires balancing:

Trade-offOptions
Write efficiency vs read latencyPush (fast reads) vs Pull (fast writes)
Cache size vs data freshnessBounded ID caches + hydration on read
Push vs pull modelsHybrid fanout based on user type
Throughput vs storage costStore IDs only, not full objects

The hybrid fanout model stands out as the most practical for real-world social platforms. This architecture ensures fast feed loading, efficient handling of celebrity users, and scalable performance at millions of DAU.