📁 Volume IV: Distributed Systems

🔺 Topic 27: CAP Theorem

Consistency vs Availability vs Partition Tolerance — You can only have two.

"In a distributed system, network partitions WILL happen.
When they do, you have a choice:
Be Consistent (CP) — Wait until data is correct.
Be Available (AP) — Serve what you have, even if it's stale.
You cannot have both. Choose wisely."

⚠️ THE DISTRIBUTED SYSTEMS TRUTH

Many developers assume their database is always consistent. In a single-node setup, it is. But once you scale horizontally (multiple servers, replicas, shards), network partitions are inevitable. The CAP theorem states you can only guarantee two of three properties. Understanding this is essential for designing reliable distributed systems.

🔍 The Three Properties of CAP Theorem

┌─────────────────────────────────────────────────────────────────────────────┐ │ │ │ CONSISTENCY (C) │ │ All nodes see the same data │ │ at the same time │ │ │ │ │ │ │ │ ┌───────────────┴───────────────┐ │ │ │ │ │ │ │ │ │ │ ▼ ▼ │ │ ┌─────────┐ ┌─────────┐ │ │ │ CP │ │ AP │ │ │ │ (Bank) │ │ (Social │ │ │ │ │ │ Media) │ │ │ └────┬────┘ └────┬────┘ │ │ │ │ │ │ │ │ │ │ ▼ ▼ │ │ ┌─────────┐ ┌─────────┐ │ │ │PARTITION│ │PARTITION│ │ │ │TOLERANCE│ │TOLERANCE│ │ │ │ (P) │ │ (P) │ │ │ └─────────┘ └─────────┘ │ │ ▲ ▲ │ │ │ │ │ │ └───────────────┬───────────────┘ │ │ │ │ │ │ │ │ AVAILABILITY (A) │ │ Every request gets a response │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ C = Consistency → Every read receives the most recent write A = Availability → Every request receives a (non-error) response P = Partition Tolerance → System continues to operate despite network drops YOU CAN ONLY GUARANTEE TWO OF THREE!

Property	Definition	Real-World Example
Consistency (C)	All nodes see the same data at the same time. After a write, all reads return that value.	Bank balance: You see your exact balance after a transaction.
Availability (A)	Every request gets a response (not an error), even if the data is stale.	Twitter feed: You always see something, even if it's from 5 seconds ago.
Partition Tolerance (P)	The system continues to work when network connections between nodes fail.	Global CDN: US servers can't reach EU servers, but both still serve users.

🌐 The Partition Scenario (What Actually Happens)

NETWORK PARTITION (REAL SCENARIO) ═══════════════════════════════════════════════════════════════════ Normal Operation: ┌─────────────────────────────────────────────────────────────────┐ │ │ │ User A (US) → US Server ←───Replication───→ EU Server ← User B (EU) │ │ │ User A updates data → Replication to EU → User B sees update │ └─────────────────────────────────────────────────────────────────┘ NETWORK PARTITION (Cable cut between US and EU): ┌─────────────────────────────────────────────────────────────────┐ │ │ │ User A (US) → US Server ✗✗✗ EU Server ← User B (EU) │ │ │ │ │ │ │ │ │ User A updates: │ User B updates: │ │ "My name is John" │ "My name is Jane" │ │ │ │ │ │ ▼ ▼ │ │ US Server has "John" EU Server has "Jane" │ │ │ │ │ │ └──────────✗✗✗─────────────┘ │ │ (No communication) │ │ │ └─────────────────────────────────────────────────────────────────┘ NOW YOU MUST CHOOSE: ═══════════════════════════════════════════════════════════════════ Option 1 (CP - Consistency over Availability): ┌─────────────────────────────────────────────────────────────────┐ │ • US Server REJECTS User A's update (return error) │ │ • US Server says: "Cannot update, can't reach other nodes" │ │ • Data stays consistent (both servers have same old data) │ │ • But User A can't update (Availability sacrificed) │ └─────────────────────────────────────────────────────────────────┘ Option 2 (AP - Availability over Consistency): ┌─────────────────────────────────────────────────────────────────┐ │ • US Server ACCEPTS User A's update (returns success) │ │ • EU Server ACCEPTS User B's update (returns success) │ │ • Both users are happy (Availability preserved) │ │ • But when network recovers: CONFLICT! Who is correct? │ │ • Data becomes inconsistent (Consistency sacrificed) │ └─────────────────────────────────────────────────────────────────┘

⚖️ CP vs AP: Which One Do You Choose?

System Type	Sacrifice	Real-World Examples	When to Use
CP (Consistency + Partition Tolerance)	Availability	Banking systems, Payment processing, Inventory management, Booking systems	When incorrect data costs money or safety
AP (Availability + Partition Tolerance)	Consistency	Social media feeds, Product catalogs, DNS, Caching systems	When stale data is acceptable, downtime is not

CA IS IMPOSSIBLE

CA (Consistency + Availability) without Partition Tolerance is impossible in a distributed system because network partitions are guaranteed to happen eventually. The only way to have CA is to run on a single node (no distribution).

🗄️ Databases and Their CAP Choices

Database	CAP Type	Explanation
MySQL (Single node)	CA	No distribution, so CAP doesn't apply (but no partition tolerance)
MySQL (Master-Slave)	CP (default)	If master can't reach slave, writes still work, reads might be stale
PostgreSQL (Sync Replication)	CP	Waits for replicas to confirm before returning success
Cassandra	AP	Always available, eventual consistency (tunable)
MongoDB (default)	CP	Reads from primary by default, waits for consistency
DynamoDB	AP (configurable)	Prioritizes availability, tunable consistency
Redis (Cluster)	AP	Asynchronous replication, eventual consistency

🎯 CAP in Laravel Applications

COMMON LARAVEL SCENARIOS

🏦 Payment Processing (CP Required)

// You CANNOT use eventual consistency here
DB::transaction(function () {
    $user->balance -= $amount;
    $user->save();
    $this->createTransaction($user, $amount);
});

// Needs strong consistency:
// - Use master DB for reads after write
// - Use synchronous replication
// - Consider distributed locks

📱 Social Media Feed (AP Acceptable)

// Eventual consistency is fine
$posts = Cache::remember('feed', 30, function () {
    return Post::latest()->take(50)->get();
});
// If data is 30 seconds old, user doesn't notice
// Better to show stale data than to show an error

HOW TO HANDLE BOTH IN LARAVEL

// Critical operations (CP)
class PaymentController
{
    public function process(Order $order)
    {
        // Force master DB connection
        DB::connection('mysql_write')->transaction(function () use ($order) {
            $order->update(['paid' => true]);
            $order->payment()->create([...]);
        });
    }
}

// Non-critical operations (AP)
class FeedController
{
    public function index()
    {
        // Can read from replica (might be stale)
        $posts = Post::on('mysql_replica')
            ->latest()
            ->paginate(20);
        
        // Cache the result
        return Cache::remember('feed', 30, fn() => $posts);
    }
}

📊 Consistency Models (Spectrum of CAP)

表 Consistency Model Description Latency Availability Strict Consistency Every read sees the latest write immediately High Low Sequential Consistency Operations appear in order, but not necessarily instant Medium Medium Causal Consistency Causally related operations seen in order Medium High Eventual Consistency All nodes eventually converge (AP systems) Low Very High

THE TRADE-OFF SPECTRUM

As you move from strict consistency to eventual consistency:

✅ Availability increases
✅ Latency decreases
✅ Partition tolerance improves
❌ Data becomes potentially stale
❌ Need conflict resolution strategies

📚 PACELC Theorem (CAP Extension)

PACELC: Even when there's no Partition (P), you still choose between Latency (L) and Consistency (C) ┌─────────────────────────────────────────────────────────────────┐ │ │ │ IF Partition (P) exists: │ │ ┌─────────────────────────┐ │ │ │ Choose: C (CP) or A (AP)│ │ │ └─────────────────────────┘ │ │ │ │ ELSE (System is healthy): │ │ ┌─────────────────────────┐ │ │ │ Choose: L (Low Latency)│ │ │ │ or │ │ │ │ C (Consistency) │ │ │ └─────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────┘ Examples: ┌─────────────────────────────────────────────────────────────────┐ │ DynamoDB: AP/EL (Available + Eventual Latency) │ │ MongoDB: CP/EC (Consistent + Extra Consistency) │ │ Cassandra: AP/EL (Available + Eventual Latency) │ │ HBase: CP/EC (Consistent + Extra Consistency) │ └─────────────────────────────────────────────────────────────────┘

🎯 How to Choose for Your Laravel Application

表 Question Answer → Choose Can your app show stale data for 1-5 seconds? Yes → AP (Availability) No → CP (Consistency) Is downtime more expensive than incorrect data? Yes → AP (Keep serving, even if stale) No → CP (Wait for correct data) Are you handling money or inventory? Yes → CP (Strong consistency required) No → AP (May be acceptable) Is the system user-facing (social, content)? Yes → AP (Users prefer fast responses) No → CP (Internal systems need accuracy)

📌 THE RULE: Money, inventory, and bookings = CP (Consistency). Social feeds, product catalogs, analytics = AP (Availability). Choose based on your business needs, not technical preference.

📝 Topic 27 Summary: CAP Theorem

表 System Type Sacrifice When to Use Example CP (Consistency + Partition Tolerance) Availability Data accuracy is critical Banking, Payments, Inventory AP (Availability + Partition Tolerance) Consistency Uptime is critical, stale data acceptable Social media, Feeds, Caching

📌 THE RULE: Network partitions are inevitable. Decide in advance: do you serve stale data (AP) or serve errors (CP)? There is no right answer — only the right answer for YOUR business.

NEXT TOPIC PREVIEW

Topic 28: Distributed Locking — Cache::lock is not enough for multi-server environments. How Redlock, ZooKeeper, and etcd solve distributed mutex problems.