Every term the playground uses, in plain English.
- Alert
- An automatic notification that a metric crossed a threshold for a while.
- API gateway
- The front door for API traffic: authentication, rate limits and routing.
- Backlog / consumer lag
- How many messages are waiting in a queue, or how old the oldest one is.
- Cache
- Fast memory that stores recent answers so the slow source (database) is asked less often.
- Cache stampede
- Many keys expire at once, so a flood of misses hits the database at the same moment.
- Canary deploy
- Releasing a new version to a small slice of servers first and watching it before continuing.
- Capacity
- The most throughput a component can sustain: roughly concurrency × 1000 ÷ time per request (ms).
- Time spent waiting on dependencies counts too, because the slot is held while waiting.
- Cascading failure
- One slow or broken component makes its callers slow or broken, and so on up the chain.
- CDN
- Content Delivery Network: caches responses on servers close to users.
- Circuit breaker
- Stops calling a dependency that keeps failing, and fails instantly instead, then tries again later.
- Cold cache
- A cache with nothing in it, usually after a restart. Every read misses.
- Concurrency
- How many requests a component can work on at the same time: threads, workers, or connections.
- Connection pool
- The limited set of connections a database accepts at once. Queries wait when it is full.
- Dead-letter queue
- Where messages go after failing several times, so they do not block everything else.
- Eventual consistency
- Copies of data agree eventually, not instantly. You might briefly read stale data.
- Exponential backoff
- Waiting longer before each retry: 100ms, 200ms, 400ms…
- Headroom
- Spare capacity kept free for spikes and failures.
- Health check
- A periodic 'are you alive?' probe. Between checks, a dead server can still receive traffic.
- Hit ratio
- The fraction of reads a cache answers itself. The rest (misses) go to the next layer.
- Hot key / hot shard
- One key (or shard) that gets far more traffic than the others.
- Jitter
- Randomness added to wait times so that many clients do not all retry at the same instant.
- Latency
- How long one request takes, from sending it to getting the answer.
- Usually measured in milliseconds (ms). Users notice anything above ~100ms and get annoyed above ~1s.
- Load balancer
- Spreads requests across replicas and stops sending to ones that fail health checks.
- Load shedding
- Deliberately rejecting some work when overloaded so the remaining work succeeds.
- Logs
- Timestamped messages written by components describing what they did or what went wrong.
- Message queue
- A buffer between producers and consumers. Work is accepted instantly and processed later.
- Turns spikes into a steady stream. The price is that work finishes later than it was accepted.
- Metrics
- Numbers over time: request rate, errors, latency, utilisation.
- Network partition
- Two components can no longer reach each other, although both are running.
- Noisy neighbour
- Another program on the same physical machine using resources you expected to have.
- p50 / p99
- Percentiles of latency. p99 = 300ms means 99 of every 100 requests finish in under 300ms.
- Averages hide pain. The slowest 1% are often your most active users, because they make the most requests.
- Packet loss
- Some network messages never arrive. The sender only finds out by timing out.
- Queue (wait queue)
- Where requests wait when every slot is busy. A growing queue means demand exceeds capacity.
- Rate limiting
- Rejecting requests above a set rate, quickly and cheaply, to protect the rest of the system.
- Read replica
- A read-only copy of the database. Takes read load off the primary, but may lag behind it.
- Replica
- An identical copy of a component. More replicas give more capacity and survive a crash.
- Replication lag
- How far a replica is behind the primary. Reads from it may return old data.
- Retry
- Trying a failed call again. Great for blips, dangerous when the dependency is overloaded.
- Retry storm
- Retries add load, which causes more failures, which cause more retries.
- Saturation
- A component running at (or near) 100% of its capacity. Requests start to wait.
- Sharding
- Splitting data across several databases by key, so each handles only part of the load.
- Single point of failure
- A component with no backup. If it fails, everything that depends on it fails.
- SLO
- Service Level Objective: the target you promise, e.g. p99 under 300ms and fewer than 1% errors.
- Throughput (RPS)
- How many requests are handled per second.
- Thundering herd
- Many clients or jobs doing the same thing at exactly the same moment.
- Timeout
- How long a caller waits for an answer before giving up.
- Trace
- The path of one request through every component it touched, with timings for each hop.
- Utilisation
- What fraction of a component's capacity is in use right now.
- Latency climbs sharply as utilisation approaches 100%. Most teams aim for 50–70% at peak.