There is a particular kind of engineering problem that looks deceptively small from the outside. You paste a long URL into a box, click a button, and get back something like https://bit.ly/3xKp9Ld. The whole interaction takes less than a second. Behind that second, though, is a distributed system that has to do quite a lot of work — generate a globally unique short code, write it durably, cache it for fast retrieval, serve hundreds of thousands of redirects per second, collect analytics events without slowing down the redirect, scan for malicious links, and stay available across multiple data centers.

That is the honest shape of a URL shortener at scale. A toy version you could build in an afternoon with a SQLite database and a single Flask server. A production version used by millions of people is something else entirely.

This post walks through that production version — the architecture, the tradeoffs, the engineering decisions, and the failure modes. Whether you are preparing for a system design interview, building your own shortener, or just curious how Bitly or TinyURL actually work at scale, this should give you a real picture of what is going on inside.

Why URL Shortening Became a Distributed Systems Problem

The original use case was simple enough: URLs on the web can get long and ugly, especially after query parameters and tracking strings pile up. Early shorteners were literally just a database with two columns — a short code and a long URL — and a web server that did a lookup and issued a 301 redirect. That works perfectly fine at small scale.

What changes everything is traffic. The redirect path is hit every single time someone clicks a link. A popular link posted on social media might receive hundreds of thousands of clicks in minutes. A link embedded in a marketing email to ten million subscribers gets hammered the moment that email lands. Unlike an application where different users hit different endpoints, a URL shortener has a pathological access pattern: the same few “hot” URLs get hit by enormous volumes of traffic, all at once.

That single reality drives almost every architectural decision in this post. Low latency on the redirect path is non-negotiable, which means you cannot afford a database round trip for every request. The system has to be stateless so it scales horizontally. Short codes have to be generated without coordination bottlenecks. Analytics has to be collected without blocking the redirect. The system has to survive regional outages. And it has to defend itself against people who will try to use it to distribute malware.

A simple redirect becomes a distributed systems problem because of scale, latency requirements, and the adversarial nature of public infrastructure.

Core Features of URL Shorteners

Before diving into architecture, it helps to be clear about what a production URL shortener actually does. The redirect is the obvious part, but modern platforms offer considerably more.

URL shortening is the core: accept a long URL, return a short one. The short code needs to be as compact as possible while still providing enough space for billions of unique values.

Redirect handling is the primary read path. When someone visits a short URL, the system looks up the destination and issues an HTTP redirect. The choice between 301 (permanent) and 302 (temporary) redirect has real consequences for analytics, as we will see.

Custom aliases let users choose their own short code — company.io/sale instead of company.io/7fG2k. This introduces uniqueness conflicts and namespace management problems.

Link analytics track who clicked, from where, on what device, through what referrer, and at what time. This is where a lot of engineering complexity lives.

Expiring links have a time-to-live after which they stop redirecting. Implementing this correctly at scale requires careful thinking about TTL propagation through caches.

QR code generation is straightforward but needs to be fast and cacheable, since the same QR code will be requested many times.

Rate limiting prevents abuse — both on the creation side and the redirect side.

Click tracking is the raw event capture that feeds the analytics pipeline. It has to be asynchronous; you cannot let analytics processing delay the redirect response.

Geo and device analytics tell you where traffic comes from and what devices people are using. This feeds dashboards and helps with fraud detection.

API support means everything above needs to be accessible programmatically with proper authentication, rate limiting, and error handling.

High-Level Architecture

The system breaks naturally into a few major subsystems: the creation path, the redirect path, and the analytics path. Each has different performance requirements and scaling characteristics.

The creation path is write-heavy but relatively low volume. People create far fewer short URLs than they click on them. The redirect path is read-heavy and latency-sensitive — this is where the bulk of engineering effort goes. The analytics path is write-heavy but can tolerate higher latency because it runs asynchronously.

URL Creation Lifecycle

When a user submits a long URL, the creation service validates it, checks for duplicates, generates a short code, writes the mapping to the database, and pre-populates the cache. The response to the user is the new short URL.

Redirect Lifecycle

When a user clicks a short link, the request hits the CDN edge first. If the edge has the mapping cached, it issues the redirect directly without touching the origin. If not, the request reaches the redirect service, which checks Redis, then falls back to the database if needed. Regardless of where the lookup succeeds, a click event is emitted asynchronously to the analytics queue.

Analytics Flow

Click events land in a message queue. Analytics workers consume from the queue, enrich events with geo and device information, and write to an analytics database optimized for time-series aggregation. Real-time dashboards query this database.

URL Creation Pipeline

The creation pipeline is more interesting than it appears. Let us walk through what actually happens.

Input validation is the first gate. The system checks that the submitted value is actually a URL, that it uses an allowed scheme (typically http or https), and that the domain is not on a blocklist. This is where you reject empty inputs, non-URL strings, and javascript: scheme URLs.

Malicious URL scanning happens next. URL shorteners are heavily abused for phishing and malware distribution because they obscure the real destination. At minimum, the system checks the URL against known blocklists like Google Safe Browsing. More sophisticated implementations use ML-based classifiers to catch novel phishing pages. This step adds latency to creation, which is acceptable because creation is not as latency-sensitive as redirects.

Duplicate detection checks whether this long URL has already been shortened. If it has, returning the existing short code is more efficient than generating a new one. This is where idempotency matters: the same long URL submitted twice by the same user should return the same short URL. The lookup is typically on a hash of the normalized URL.

Short code generation is its own topic and gets its own section below.

Database write needs to be durable. The system writes to the primary database with at minimum one synchronous replica acknowledgment before returning success to the user. Losing a just-created URL mapping would be a bad user experience.

Cache population pre-warms the cache immediately after the write. If a user creates a short URL and immediately shares it, the first redirect should hit the cache, not the database.

The creation endpoint needs to be idempotent for reliable API clients. If a client creates a URL and the network drops before receiving the response, retrying the same request should return the same short code rather than creating a duplicate.

Short URL Generation System

This is one of the most discussed sections in system design interviews, and for good reason. Generating billions of unique, short, collision-free identifiers in a distributed system without coordination bottlenecks is a genuinely interesting problem.

Base62 encoding is the standard approach. Base62 uses the characters A-Z, a-z, and 0-9 — 62 characters total. A 7-character Base62 string gives you 62⁷ unique values, which is approximately 3.5 trillion. That is enough for a very long time even at large scale. The advantage of Base62 over Base64 is that it avoids +, /, and = characters that are problematic in URLs.

The generation strategy you choose matters a lot:

Sequential IDs with Base62 encoding means you auto-increment an integer counter and encode it in Base62. ID 1 becomes “1”, ID 62 becomes “Z”, ID 63 becomes “10”. This is simple, compact, and collision-free. The problem is that sequential IDs are predictable — an attacker can enumerate short URLs by incrementing the identifier. For a public shortener, this exposes every URL ever created.

Random codes pick characters randomly until you have a string of the desired length, then check the database for collisions. At low volumes this is fine. At high volumes, collision probability increases and the database round trip for every generation is expensive. You need to handle retry logic when a collision occurs.

Hash-based generation takes an MD5 or SHA-256 hash of the long URL and uses the first N characters of the hash encoded in Base62. This is deterministic, which means the same long URL always produces the same short code — useful for deduplication. The downside is collision risk (two different URLs producing the same first-N characters of hash) and the difficulty of supporting multiple short codes for the same long URL.

Snowflake-style ID generation is what production systems at scale actually use. Twitter’s Snowflake approach generates 64-bit integers composed of a timestamp, a machine ID, and a sequence number. Each machine generates IDs independently without coordination. The IDs are time-ordered, which helps with database write performance. The system never generates collisions as long as machine IDs are unique. Encoding a Snowflake ID in Base62 gives you a compact, unique, non-predictable-enough short code.

One practical optimization: pre-generate a pool of short codes in background workers and store them in a queue. When a creation request comes in, it pops a code from the queue rather than generating one on the fly. This removes ID generation latency from the critical path entirely.

Redirect Pipeline Deep Dive

The redirect path is where most requests go, and it needs to be fast. The engineering goal is to serve the vast majority of redirects without touching the origin database at all.

DNS resolution is the first step and it happens client-side. There is nothing you can directly optimize here, but you can use a short domain name to reduce the size of the URL. bit.ly is better than shorturl-service.example.com for both aesthetics and DNS lookup time.

CDN edge handling is where you can eliminate most of the redirect latency. A CDN with edge nodes distributed globally can cache URL mappings close to users. Cloudflare Workers, for example, let you run code at the edge that performs the Redis lookup or serves a cached redirect directly. The round trip from a user in Tokyo to an origin in Virginia is ~150ms. A CDN edge node in Tokyo reduces that to ~5ms.

Cache lookup hits Redis if the edge misses. Redis lookup times are measured in microseconds for local lookups, single-digit milliseconds for remote. The redirect service checks Redis for the short code and, on a hit, issues the redirect immediately.

Database lookup is the last resort. On a cache miss, the service queries the primary database. This should be a rare event for any reasonably popular URL — the cache should have everything hot. For brand new URLs, the creation pipeline pre-warms the cache, so even the first redirect after creation should hit the cache.

HTTP redirect response is either a 301 or 302. This choice matters for analytics. A 301 (Moved Permanently) tells the browser to cache the redirect. The browser will then redirect future clicks to the same short URL directly, without contacting your server at all. This is great for performance but catastrophic for analytics — you will never see repeat clicks from the same browser. A 302 (Found/Temporary) means the browser always asks your server, ensuring every click is tracked. Most commercial URL shorteners use 302 for this reason.

Async click event emission happens after the redirect response is sent. The response goes back to the client immediately; the click event is enqueued in the background. The user experiences no latency from analytics collection.

The redirect service should be completely stateless. Any instance should be able to handle any request. This means you can scale horizontally by adding more instances behind a load balancer without any coordination between them.

Database Architecture

The storage layer has two distinct concerns: the URL mapping store and the analytics store. They have very different access patterns and benefit from different database choices.

URL Mapping Storage

The URL mapping table is accessed on every redirect that misses the cache. It is almost entirely read traffic with occasional writes (URL creation). The schema is straightforward:

CREATE TABLE short_urls (
    id              BIGINT PRIMARY KEY,
    short_code      VARCHAR(12) NOT NULL UNIQUE,
    long_url        TEXT NOT NULL,
    user_id         BIGINT REFERENCES users(id),
    custom_alias    BOOLEAN DEFAULT FALSE,
    created_at      TIMESTAMP NOT NULL,
    expires_at      TIMESTAMP,
    is_active       BOOLEAN DEFAULT TRUE,
    url_hash        CHAR(64) NOT NULL,
    click_count     BIGINT DEFAULT 0
);

CREATE INDEX idx_short_code ON short_urls(short_code);
CREATE INDEX idx_url_hash ON short_urls(url_hash);
CREATE INDEX idx_expires_at ON short_urls(expires_at) WHERE expires_at IS NOT NULL;

The short_code index is the hot path — it gets hit on every cache-miss redirect. The url_hash index supports deduplication during creation. The expires_at partial index is used by background cleanup jobs.

SQL vs NoSQL is a real tradeoff here. A relational database (PostgreSQL) gives you strong consistency, easy querying, and mature tooling. For a URL shortener where the primary access pattern is a single key lookup, a key-value store (DynamoDB, Redis with persistence, or Cassandra) can offer better write throughput and simpler horizontal scaling. In practice, many URL shorteners use PostgreSQL for the URL mapping store because the read load is handled by the cache layer, not the database directly.

Sharding becomes necessary when your URL mapping table grows to billions of rows. The natural shard key is the short code. You can use consistent hashing to distribute short codes across database shards. Each shard handles a fraction of the total redirect traffic. The redirect service hashes the short code to determine which shard to query.

Read replicas are essential. Route all read queries to replicas; route writes to the primary. This is standard practice and gives you read scalability proportional to the number of replicas you add.

Analytics Storage

Analytics data has completely different characteristics. You are writing millions of click events per hour and reading them in aggregated form for dashboards. The raw events table:

CREATE TABLE click_events (
    id              BIGINT,
    short_code      VARCHAR(12) NOT NULL,
    clicked_at      TIMESTAMP NOT NULL,
    ip_address      INET,
    country_code    CHAR(2),
    city            VARCHAR(100),
    device_type     VARCHAR(20),
    os              VARCHAR(50),
    browser         VARCHAR(50),
    referrer        TEXT,
    user_agent      TEXT
) PARTITION BY RANGE (clicked_at);

Time-based partitioning lets you drop old partitions efficiently and keeps recent data fast to query. For analytics at serious scale, a columnar store like ClickHouse or Apache Cassandra is a better fit than PostgreSQL. ClickHouse in particular is excellent for time-series analytics — it ingests high-volume writes and executes aggregate queries (count by country, clicks by hour, device breakdown) very efficiently.

Caching System Deep Dive

The caching layer is arguably the most critical component of the redirect pipeline. If the cache is working correctly, the database almost never sees redirect traffic.

Redis as the primary cache is the standard choice. Redis is fast (sub-millisecond for local, low-single-digit milliseconds for remote), supports TTL on keys, and handles high concurrency well. The redirect service stores URL mappings as simple key-value entries: short_code -> long_url. It also stores metadata like expiration time.

Cache TTL is an interesting decision. If you cache a URL mapping with a TTL of 24 hours, you will not see database traffic for that URL for 24 hours — which is great. But if the URL is deleted or deactivated during that window, the cache will continue serving the redirect. Most systems accept this eventual consistency: a deleted URL might continue redirecting for up to the cache TTL. If you need immediate effect (for abuse takedowns, for example), you need an explicit cache invalidation mechanism.

Cache invalidation is one of those problems that sounds trivial but is not. When a URL is updated or deleted, you need to evict it from every layer of cache: the Redis cluster and the CDN edge. Most CDNs support cache purge APIs. Redis supports direct key deletion. The challenge is doing this reliably across a distributed system without missing any cache node.

Hotspot handling is the problem of what happens when a single URL gets millions of hits per second. A single Redis key becomes a bottleneck. Solutions include local in-process caching in the redirect service (a small in-memory LRU cache that lives inside the application), which avoids the Redis round trip entirely for the hottest URLs. The tradeoff is that this local cache is not invalidation-friendly — you rely on its short TTL to eventually expire stale entries.

Edge caching via CDN is a force multiplier. If you configure your CDN to cache the redirect response (the HTTP 302 with a Location header), the CDN edge nodes absorb the traffic without any request reaching your infrastructure. The CDN becomes your first-line cache. The challenge is that redirect responses include a Cache-Control header, and the logic for setting appropriate TTLs (especially for expiring links) needs to be carefully implemented.

Analytics Infrastructure

The analytics pipeline is one of those systems where the naive approach (write to the database on every click) breaks down quickly. At 100,000 redirects per second, synchronous database writes would immediately overwhelm any relational database. The solution is an asynchronous event-driven pipeline.

Click event emission happens immediately after the redirect response is sent. The redirect service publishes a lightweight event to a Kafka topic. The event contains the short code, timestamp, IP address, user agent, referrer, and any other raw request metadata. Publishing to Kafka is fast — it is a fire-and-forget write to a local buffer that Kafka flushes asynchronously.

Kafka as the event backbone is the right choice here because Kafka provides durable, ordered, replayable event streams. If the analytics consumer goes down, events accumulate in Kafka until the consumer recovers. You can replay events if a downstream system needs to be rebuilt. Multiple consumers can read the same stream independently.

Analytics consumers read from Kafka, parse the user agent string to extract device type, OS, and browser, call a geo-IP service to resolve the IP address to country and city, and write enriched events to the analytics database. This processing pipeline can run in parallel across many consumer instances.

Aggregation runs periodically (every minute, every hour) to produce pre-computed summaries: clicks per URL per hour, clicks by country, clicks by device. These aggregated tables are what dashboards query, because querying billions of raw events for every dashboard load would be prohibitively slow.

Real-time dashboards add complexity. If a user expects to see their click count update in near-real-time, you need to either query the event stream directly (possible with Kafka Streams or Flink) or maintain a hot counter in Redis that gets incremented on every click and is periodically persisted to the database. The Redis counter approach is fast and simple, but you risk losing recent counts if Redis fails before persistence.

Expiration and Cleanup Systems

Expiring links need careful handling across multiple system layers.

When a short URL has an expires_at timestamp, several things need to happen at expiration time:

The redirect service needs to return a 410 Gone or 404 Not Found response rather than redirecting. The cache entry needs to reflect the expiration. The database entry can eventually be soft-deleted or marked inactive. Storage can be reclaimed by eventually deleting or archiving old records.

TTL propagation is the tricky part. If you cache a URL mapping in Redis with a TTL of 24 hours, but the URL expires in 2 hours, the Redis entry needs a TTL of 2 hours, not 24. The caching layer must be aware of URL expiration times and set cache TTLs accordingly.

CDN purging at expiration time is hard to do precisely. If a URL expires at 3:00 PM and you have cached the redirect at the CDN edge with a long TTL, users will continue to get the redirect until the CDN TTL expires. The solution is either to use short CDN TTLs for expiring links, or to call the CDN purge API at expiration time.

Background cleanup workers handle soft-deletion and storage reclamation. They run periodically, query the database for URLs past their expiration time, mark them inactive, and optionally archive or delete the records. These workers need to be careful not to reclaim short codes that might still be in flight (cached somewhere with a long TTL).

Custom Alias System

Custom aliases — where a user chooses their own short code — introduce a set of problems that random code generation avoids.

Namespace conflicts are the core problem. If user A creates company.io/sale, user B cannot also create company.io/sale. The system needs to enforce global uniqueness across all aliases. This is a straightforward database unique constraint, but at scale you want to avoid hot spots on the uniqueness check.

Reserved keywords must be blocked from use as aliases. Words like admin, api, login, health, static, docs, and similar terms are typically used by the system itself and must be protected. The list of reserved words is surprisingly long when you account for all the paths a web application might use.

Validation needs to be strict: only alphanumeric characters plus hyphens and underscores, minimum and maximum length, no leading or trailing hyphens, no consecutive hyphens. The alias should be case-insensitive in lookup but case-preserved in display.

Abuse of custom aliases is a real problem. People will try to register brand names, celebrity names, and other valuable identifiers speculatively. You need both automated detection (keyword lists, brand name lists) and a takedown process.

Security and Abuse Prevention

URL shorteners are a uniquely attractive target for abuse. The core feature — hiding the destination URL — is precisely what makes them useful for phishing, malware distribution, and spam. A production shortener needs layered defenses.

Malicious URL scanning at creation time uses blocklists (Google Safe Browsing, PhishTank) to reject known bad URLs. This is table stakes. More sophisticated systems also scan the destination page content, look at the domain’s reputation and age, and use ML classifiers trained on phishing page characteristics.

Rate limiting on the creation endpoint prevents bulk URL generation. Unauthenticated requests get a very low rate limit (perhaps 10 per minute per IP). Authenticated users get higher limits based on their account tier. API clients get limits based on their API key tier. Rate limiting should happen at the API gateway layer, before requests reach the creation service.

Click fraud detection matters for analytics integrity. Bot traffic inflating click counts is a common problem. Signals for bot detection include: user agent strings that look like scrapers, IP addresses in known bot ranges, click patterns that are too regular, and clicks without corresponding browser fingerprints. Filtering bot clicks from analytics requires either real-time filtering in the click event pipeline or post-hoc analysis.

Redirect abuse — using a legitimate shortener to distribute malware — happens when the destination URL changes after shortening. Some systems re-scan destination URLs periodically. Others allow the destination to be reported by users and trigger a rescan. Takedown processes need to be fast; a phishing page that is live for hours causes real harm.

API key abuse — one API key used across many different clients — is detected by analyzing the diversity of creation patterns, IP addresses, and user agents associated with a single key.

CDN and Edge Infrastructure

The CDN layer deserves more attention than it typically gets in system design discussions. Done correctly, a CDN can absorb the vast majority of redirect traffic without any request reaching your servers.

The approach is to push URL mappings to the CDN edge as part of the creation pipeline. When a URL is created, the system can proactively notify edge nodes (via a CDN API or a push invalidation mechanism) that a new mapping exists. Alternatively, edges learn about mappings on first access and cache them for subsequent requests.

Modern CDN platforms (Cloudflare Workers, Lambda@Edge, Fastly Compute) let you run actual code at the edge. You can implement the full redirect logic — cache lookup, TTL check, analytics event emission — at the edge node, completely eliminating round trips to origin servers for cached URLs.

DNS optimization is another lever. Using a CDN also means your DNS TTL can be set appropriately for geographic routing — short enough to fail over quickly, long enough to avoid excessive DNS lookups.

Regional caching at the CDN level means that a popular link shared in Europe is served by European edge nodes without touching your US-based origin. This reduces both latency for European users and load on your origin.

Event-Driven Architecture

The analytics pipeline illustrates a broader principle: the system uses event-driven architecture to decouple the redirect path from analytics processing.

The redirect service does not care what happens to click events after it emits them. It fires the event and moves on. The analytics consumer does not care about the redirect service’s implementation details. The Kafka topic is the contract between them.

This decoupling has practical benefits. You can scale the redirect service and the analytics consumer independently. You can update the analytics pipeline without touching the redirect service. If the analytics system goes down, events accumulate in Kafka and are processed when it recovers — the redirect service is unaffected.

Event durability means Kafka retains events for a configurable period (days or weeks). If you discover a bug in your analytics processing three days after deployment, you can fix the bug and replay the events to produce correct analytics. This is not possible with a synchronous write-and-forget approach.

Consumer groups in Kafka let multiple independent consumers read the same event stream. Your analytics pipeline reads the stream. A fraud detection system reads the same stream looking for suspicious patterns. A real-time alerting system reads the stream looking for sudden traffic spikes. All three are independent consumers of the same events, with no coordination between them.

Scalability Deep Dive

Let us talk about where the bottlenecks actually are at different scales.

At small scale (thousands of redirects per day), a single server with a database and a Redis instance handles everything comfortably. The bottleneck is usually nothing at all — the system has plenty of headroom.

At medium scale (millions of redirects per day), the database becomes a concern for reads. Adding read replicas and ensuring the cache hit rate is high (above 95%) keeps the database comfortable. The redirect service can run as multiple stateless instances behind a load balancer.

At large scale (hundreds of millions of redirects per day), CDN edge caching becomes essential. The cache hit rate needs to be very high (above 99%) because even 1% of traffic reaching your Redis cluster is millions of requests per day. The Redis cluster needs to be horizontally sharded. The analytics pipeline needs dedicated infrastructure.

At extreme scale (tens of billions of redirects per day, in the league of Bitly at peak), you are running multi-region active-active deployments, purpose-built ID generation infrastructure, globally distributed CDN caching, and dedicated analytics compute clusters. You are also dealing with hotspot URL problems that require per-instance in-process caches.

Reliability and Availability

A URL shortener has a reliability characteristic that makes it feel more critical than it is: when it goes down, every short link on the internet that points to it breaks. That creates real pressure on the team to maintain high availability.

Multi-region deployment means running the redirect service in multiple regions (US East, EU West, AP Southeast) with independent databases and caches. A regional outage affects only users routed to that region. Global traffic routing (via DNS-based geo-routing or CDN) directs users to the nearest healthy region.

Cache as a reliability buffer is an underappreciated benefit. If your database goes down completely, the Redis cache and CDN cache will continue serving redirects for cached URLs until their TTLs expire. For a heavily trafficked system where most URLs are cached, this might mean the redirect service continues working for hours without a database — long enough to recover.

Graceful degradation means deciding ahead of time what the system should do when components fail. If the analytics queue is down, should the redirect fail? No — emit a log line and continue. If the malicious URL scanner is down during creation, should creation fail? Maybe — if the scanner is down, you stop accepting new URLs until it recovers, rather than accepting potentially malicious URLs.

Monitoring needs to cover cache hit rates (a sudden drop indicates a problem), redirect latency percentiles (p50, p99, p999), database query times, Kafka consumer lag, and error rates by type. Alerts should fire before users notice problems, which means setting thresholds below the point of user impact.

Engineering Tradeoffs

These are the decisions where reasonable engineers disagree, and where the right answer depends on your specific situation.

Sequential IDs vs random codes: Sequential IDs are simpler and slightly more compact. Random codes prevent enumeration. If your shortener is for internal use, sequential is fine. If it is public-facing and you care about URL privacy, use random codes or Snowflake IDs.

301 vs 302 redirects: 301 is better for performance and reduces server load over time. 302 is necessary for click tracking. Commercial shorteners almost universally use 302. If you are building an internal tool and do not need analytics, 301 is fine.

Synchronous vs asynchronous analytics: Synchronous analytics (write to the database before returning the redirect response) guarantees no event loss but adds latency to every redirect. Asynchronous analytics via Kafka risks losing events if the queue fails but adds near-zero latency to redirects. The answer at scale is always asynchronous.

SQL vs NoSQL for URL mappings: PostgreSQL is simpler to operate and gives you strong consistency. DynamoDB or Cassandra gives you better write throughput and easier horizontal scaling. At small to medium scale, PostgreSQL with read replicas and a caching layer in front is entirely sufficient. At extreme scale, a purpose-built key-value store makes more sense.

Consistency vs availability in caching: Do you want a cache that reflects reality exactly (always consistent with the database) or one that might serve stale data for some TTL window (eventually consistent)? For URL redirects, eventual consistency is almost always acceptable. For expiring or deleted links, you need faster invalidation, which requires more complexity.

Real-World Technology Stack

What does this look like in practice? Here is a realistic technology stack for a production URL shortener.

Go for the redirect service. Go’s performance characteristics — fast startup, low memory overhead, excellent concurrency with goroutines — make it ideal for a high-throughput, low-latency service. Many performance-sensitive web services at companies like Cloudflare and Uber are written in Go for exactly these reasons.

Java or Go for the creation and management services. These are less latency-sensitive and benefit from rich ecosystems. Java with Spring Boot is a common choice for business logic services.

Redis for the primary cache. Redis Cluster for horizontal scaling. Redis’s built-in TTL support makes it natural for URL expiration. Redis Streams or a separate Kafka cluster for the analytics event queue.

PostgreSQL for the URL mapping database at medium scale. Well-understood, excellent tooling, strong consistency. Add PgBouncer for connection pooling. Add read replicas for read scaling.

Cassandra or ClickHouse for analytics data. Cassandra handles high-volume writes well and is operationally straightforward. ClickHouse is exceptional for analytical queries and dashboard workloads.

Kafka for the analytics event stream. Kafka handles high-throughput, durable event streaming. Kafka’s consumer group model makes it easy to add new consumers (fraud detection, real-time alerting) without touching existing consumers.

Kubernetes for container orchestration. The redirect service, creation service, and analytics consumers all run as Kubernetes deployments with horizontal pod autoscaling. Traffic spikes trigger automatic scale-out.

Cloudflare or Fastly as the CDN. Both support edge computing (Workers, Compute) for running redirect logic at the edge.

Elasticsearch for log analysis and abuse detection. Storing and querying structured logs at scale is where Elasticsearch excels.

System Design Interview Perspective

URL shorteners appear constantly in system design interviews because they cover an enormous range of distributed systems concepts in a problem that is easy to explain and hard to solve at scale.

How interviewers frame the question: You will hear “Design a URL shortening service like Bitly” or “Design a system that can handle 100 billion short URL redirects per day.” The second framing is better because it forces you to think about scale from the start.

What strong candidates do:

They start by clarifying requirements. How many URLs created per day? How many redirects? Is analytics a hard requirement? What is the acceptable redirect latency? What availability SLA do we need? These questions signal that you understand that architecture follows requirements.

They calculate back-of-envelope numbers before drawing boxes. If you are handling 100 billion redirects per day, that is roughly 1.15 million per second at peak. A single Redis instance can handle ~100,000 operations per second, so you immediately know you need a Redis cluster. Walking through this math shows the interviewer that you reason quantitatively.

They explain the redirect path in detail, because that is where the interesting engineering is. Walk through DNS resolution, CDN edge handling, Redis lookup, database fallback, and the async analytics event. Explain why each layer exists and what it costs to skip it.

They discuss ID generation not as trivia but as a real engineering decision. Why Base62? Why 7 characters? Why Snowflake-style over a simple counter? Each answer connects to a real concern (URL length, global uniqueness, predictability).

They proactively discuss failure modes. What happens if Redis is down? What happens if the analytics queue is full? What happens if the database primary fails during a write? Showing that you think about failure scenarios signals production experience.

Common mistakes:

Jumping to solutions before establishing requirements. Drawing a database box without discussing SQL vs NoSQL or how you would handle sharding at scale.

Using a 301 redirect without explaining the analytics tradeoff. This is a detail that signals whether someone has actually thought about the problem.

Ignoring the CDN layer entirely. Many candidates design a system where every redirect hits an origin server, which simply does not work at scale.

Treating analytics as an afterthought. At production scale, analytics is a significant engineering investment that deserves its own discussion.

Forgetting about security entirely. URL shorteners are abuse vectors and interviewers will often probe here.

Strong answers explicitly discuss tradeoffs. Not “use Redis” but “we can use Redis here; the tradeoff is that we need to handle cache invalidation carefully when URLs are deleted or expired.” Not “use Kafka” but “Kafka gives us durability and replay capability for click events, which is worth the operational complexity because losing analytics data is not acceptable.”

The goal is not to arrive at the one correct architecture — there is no such thing — but to demonstrate that you understand why each component exists, what it costs, what breaks without it, and how you would adapt if the requirements changed.

A URL shortener is a small system that contains large distributed systems problems inside it. When you genuinely understand how it works, you have touched database sharding, distributed caching, event streaming, edge computing, ID generation at scale, and asynchronous pipeline design. That is a lot of ground from a very small URL.