High Level Design
Security in Distributed Systems
Authentication mechanisms (token-based, SSO, OAuth), authorization patterns (ACLs, rule engines), attack vectors, and how to secure CDN-delivered content.
Security in distributed systems refers to the measures and mechanisms used to protect data, communication, and resources across multiple interconnected nodes.
Verification#
CAPTCHA#
Completely Automated Public Turing test to tell Computers and Humans Apart. Protects against spam and brute-force attacks.
Problem: With modern AI, bots can now solve most CAPTCHA challenges with 95%+ accuracy.
SSL/TLS#
- SSL (Secure Sockets Layer) — encrypts data transmitted over the web; outdated and insecure (no longer used)
- TLS (Transport Layer Security) — successor to SSL; current standard (TLS 1.3)
- Provides privacy, integrity, and authentication
- Used in HTTPS, email, messaging apps
When people say "SSL certificates" today, they almost always mean TLS certificates.
Encryption Algorithms#
- RSA — asymmetric encryption; key exchange
- ECC (Elliptic Curve Cryptography) — smaller keys, same security as RSA
- Diffie-Hellman — key exchange protocol
Authentication#
Token-Based Auth#
- User sends username + password
- Server validates and generates a signed token
- Token sent with each subsequent request (no need to resend credentials)
- Server verifies token using a public key
Best practices:
- Use timestamps/version numbers to limit token validity
- Logging out = invalidate the token
- Never store tokens in localStorage (use httpOnly cookies)
SSO — Single Sign-On#
Authentication is delegated to an external service (Google, Okta).
- External service validates credentials and returns a token
- Server decrypts and verifies permissions
- Useful for enterprise systems managing many users
OAuth — Open Authorization#
Authorization framework allowing users to grant limited access to their resources without sharing passwords.
- User is prompted to grant specific permissions
- OAuth tokens generated after permissions are granted
- Server can access only the approved scopes (e.g., name, profile photo)
Always plan a fallback scenario for when the external OAuth service is unavailable.
Authorization#
ACLs — Access Control Lists#
Lists that define allowed actions on objects:
| Type | Description |
|---|---|
| User-Based | Specific users have certain permissions |
| Role-Based (RBAC) | Users in a role inherit those permissions |
| Group-Based | Groups of users share permissions |
✅ Simple for static, straightforward permission models
❌ Difficult to maintain as the system grows
Rule Engines#
Use conditional rules to decide if a user has permission:
- ✅ Handles complex authorization
- ✅ Rules can be updated without modifying ACLs directly
- ✅ Centralized maintenance
- Best for dynamic, context-aware permission models
Secret Keys#
Client keys add an extra layer of authentication:
- Must be securely stored and rotated regularly
- Not sufficient as the sole security mechanism
- Best for service-to-service authentication and API access
Attack Vectors#
DDoS Attacks (Hackers)#
Mitigation:
- Distributed rate limiting
- Web Application Firewalls (WAF)
Insider Threats (Employees)#
Mitigation:
- ACLs to limit employee actions
- Open only necessary entry points
- Minimize attack surface
Malicious Code#
Mitigation:
- Resource restrictions
- Rule engines to prevent illegal modifications
- Regular integrity checks and code reviews
Securing Videos on a CDN#
| Approach | How It Works | Pros | Cons |
|---|---|---|---|
| Token-Based | CDN forwards to main server for auth; server issues token; CDN serves on verification | Strong auth | Extra round-trip; CDN sees token |
| Domain-Based | CDN allows access only from approved domains | Simple and fast | Vulnerable to domain spoofing |
| Server-Side | Server generates token signed with private key; CDN verifies with public key; periodic refresh | Balanced security + efficiency | Complex backend integration |