#
11 terms3NF
third normal form
- First used in
- Denormalization and SQL tuning
201 Created
The HTTP status code saying the request has been fulfilled and has created one or more new resources. The response names the main one in its Location header, or else it is the URI the request was sent to.
- Also called
- 201
- Taught in
- Requirements, estimates and the API
- Source
202 Accepted
The HTTP status code meaning the request was accepted for processing, which is not yet complete.
- Taught in
- Scaling from one user to millions
- Source
301 Moved Permanently
A redirect saying the resource has a new permanent URI, which future references ought to use. It is heuristically cacheable, and a client following it may change a POST into a GET.
- Also called
- 301
- Taught in
- Redirects, caching and visit analytics
- Source
302 Found
307 Temporary Redirect
A redirect saying the resource is for now at a different URI, where a client following it automatically must not change the request method. The client ought to go on using the original URI for future requests.
- Also called
- 307
- Taught in
- Redirects, caching and visit analytics
- Source
308 Permanent Redirect
A redirect saying the resource has a new permanent URI, which future references ought to use. It is heuristically cacheable, and it was added later to say plainly that the request method is kept.
- Also called
- 308
- Taught in
- Redirects, caching and visit analytics
- Source
404 Not Found
The HTTP status code saying the server found no current representation of the resource, or will not reveal that one exists. It does not say whether that is temporary or permanent.
- Also called
- 404
- Taught in
- Storage: SQL or NoSQL, object storage and expiry
- Source
410 Gone
The HTTP status code saying the resource is no longer available at the server, and probably never will be again. A server that cannot tell whether that is permanent ought to send 404 Not Found instead.
- Also called
- 410
- Taught in
- Storage: SQL or NoSQL, object storage and expiry
- Source
429 Too Many Requests
The HTTP status code for rate limiting: the user has made too many requests within some period. It may say, with Retry-After, how long to wait.
- Also called
- 429
- First used in
- Reverse proxies and edge security
- Source
503 Service Unavailable
The HTTP status code meaning the server cannot handle the request for now, because of a temporary overload or planned maintenance.
- Also called
- 503
- First used in
- Reverse proxies and edge security
- Source
A
46 termsAAAA record
The DNS record that stores a single IPv6 address for a name.
- Also called
- AAAA
- First used in
- DNS and global traffic routing
- Source
abort
The failure of a transaction: instead of committing, it rolls back every change it made that was not yet visible.
- Also called
- rollback
- Taught in
- Distributed transactions: two-phase commit and sagas
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 8: Chapter 8. Transactions
- Database Internals, 1st edition, chapter 5: Chapter 5. Transaction Processing and Recovery
ACID
atomicity, consistency, isolation, durability
The safety guarantees that transactions are commonly described as giving.
- First used in
- Performance versus scalability
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 8: The Meaning of ACID
active defragmentation
Redis getting back memory lost to fragmentation while the server runs: it copies values into contiguous memory and releases the old copies. It is off by default, and works only with the copy of jemalloc shipped with Redis, which Linux builds use by default.
- Taught in
- Eviction algorithms and memory management
- Source
active-active
Failover in which both servers handle traffic, sharing the load. The book uses active/active for multi-leader replication.
- Taught in
- Availability patterns and failover
- Source
- The System Design Primer
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Multi-Leader Replication
active-passive
Failover with an active server and a passive one on standby. They exchange heartbeats; if those stop, the passive server takes over the IP address of the active one and resumes service. The book uses active/passive for single-leader replication.
- Taught in
- Availability patterns and failover
- Source
- The System Design Primer
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Single-Leader Replication
Adamic/Adar
A score for two vertices: over each neighbor they share, add 1 divided by the logarithm of that neighbor’s number of neighbors. It adapts a measure Adamic and Adar used for the features two personal home pages share, which weights rarer features more heavily than a plain count of shared features does.
- Taught in
- People you may know: ranking mutual friends
- Source
- Liben-Nowell and Kleinberg, The Link Prediction Problem for Social Networks
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Graph-Like Data Models
AES
Advanced Encryption Standard
A cryptographic algorithm, approved by the U.S. Government, that can encrypt and decrypt electronic data.
allocator
A library or operating system routine that hands out memory to a program and takes it back. It can make the program faster, or slower if its memory becomes fragmented and wasted.
- Also called
- memory allocator
- Taught in
- Eviction algorithms and memory management
- Source
- Systems Performance, 2nd edition, chapter 7: 7.2.8 Allocators
allowed lateness
Amazon API Gateway
An AWS service to create, publish, maintain, monitor and secure APIs. It limits the rate of requests to them with a token bucket.
- Also called
- AWS API Gateway
- First used in
- Reverse proxies and edge security
- Source
AMQP
Advanced Message Queuing Protocol
A specification for message-queuing systems, worked out from 2006 to change a situation in which such systems were in large part proprietary.
- Taught in
- Queues, task queues and log-based message brokers
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 4: Example: Advanced Message Queuing Protocol (AMQP)
analytics
Queries that aggregate over many records.
- First used in
- Scaling from one user to millions
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 1: Characterizing Transaction Processing and Analytics
anti-entropy
A background process that compares the data held by replicas and fills in whatever one of them lacks. Unlike a replication log, it copies writes in no particular order, and the copy may come much later.
- Taught in
- Tunable consistency and quorums
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Catching up on missed writes
anycast
One address shared by several machines in different places: a packet sent to it goes to one of them, the nearest by the routing protocol’s measure of distance.
- Taught in
- DNS and global traffic routing
- Source
- IETF RFC
- Understanding Distributed Systems, 2nd edition, chapter 18: Transport layer load balancing
AP
available under network partitions
Choosing availability during a network partition: the application stays available, but its behavior is no longer linearizable.
- Taught in
- Availability, consistency, CAP and PACELC
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 10: The CAP theorem
API
application programming interface
What a server exposes over the network so that clients can connect to it and make requests.
- First used in
- NoSQL data models and access patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: Dataflow Through Services: REST and RPC
- Systems Performance, 2nd edition, glossary
Application Load Balancer
The AWS load balancer that works at the application layer, layer 7: rules you set decide where each request goes.
- Also called
- ALB
- First used in
- Load balancing: layer 4 versus layer 7
- Source
ARC
adaptive replacement cache
An eviction policy that keeps two LRU lists, one of items seen only once and one of items seen at least twice, and keeps shifting the balance between them, recency against frequency, as the workload changes. Extra lists (ghost lists) show how each would do with all the memory.
- Taught in
- Eviction algorithms and memory management
- Source
- USENIX
- Systems Performance, 2nd edition, chapter 8: 8.4.5 File System Types
arrival rate
The average rate at which requests arrive at a system. Little's law writes it as λ.
- Taught in
- Response time versus throughput
- Source
- Systems Performance, 2nd edition, chapter 2: 2.6.5 Queueing Theory
ASCII
American Standard Code for Information Interchange
- Also called
- US-ASCII
association (in TAO)
In TAO, a typed edge with a direction, from one object to another. It is named by its source object, its type and its destination object, and two objects can have at most one association of a type between them. It has a time, and may hold data.
- Taught in
- Requirements, estimates and the graph model
- Source
asynchronous
Going on without waiting for something to finish, such as data sent to another node over the network, and assuming nothing about how long it will take.
- First used in
- Scaling from one user to millions
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
asynchronous replication
Replication in which the leader sends a write to a follower without waiting for a reply. There is no guarantee how long the follower takes to apply it.
- First used in
- Single-leader and multi-leader replication
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Synchronous Versus Asynchronous Replication
at-least-once delivery
A delivery guarantee: the sender keeps sending the message again until the receiver acknowledges it, so it arrives at least one time, but possibly more.
- Also called
- at-least-once
- First used in
- Partition keys, skew and polyglot persistence
- Source
- Database Internals, 1st edition, chapter 8: Exactly-once delivery
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 8: RPC semantics in the presence of failures
at-most-once delivery
A delivery guarantee: the sender sends each message and expects no confirmation. Messages may be lost, but are never delivered again.
- Also called
- at-most-once
- Taught in
- Delivery guarantees, ordering and idempotency
- Source
- Database Internals, 1st edition, chapter 8: Exactly-once delivery
- Apache Kafka documentation
atomic commit
Making a transaction that involves several database nodes either commit on all of them or abort on all of them. Two-phase commit is an algorithm for it.
- Taught in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
- Designing Data-Intensive Applications, 2nd edition, chapter 8: Two-Phase Commit
atomicity
ACID's guarantee that a transaction's writes happen all together or not at all: if a fault stops the transaction from committing, it is aborted and the database discards the writes it had made.
- First used in
- Event sourcing, CQRS and the transactional outbox
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 8: Atomicity
attention head
One of several attention calculations a model runs for the same token, each with its own projections. Together they let it capture different kinds of relationship between tokens at once.
- Taught in
- Batching and the KV cache
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 2: Multi-head attention
audit log
A chronological record of a system’s activities, including records of accesses to the system and of the operations performed in a given period.
- First used in
- Credential vaults and envelope encryption
- Source
Aurora
A fully managed relational database engine, part of Amazon RDS, compatible with MySQL and PostgreSQL. All the database instances of a cluster share its cluster volume, and Aurora's underlying storage grows automatically as needed.
- Also called
- Amazon Aurora
- Taught in
- Availability zones, failover and auto scaling
- Source
Aurora global database
An Aurora database that spans several AWS Regions: one Region takes the writes, and as many as 10 others are read-only. Aurora copies every change to them automatically.
- Taught in
- Multiple regions and disaster recovery
- Source
Aurora Replica
A read-only database instance in an Aurora cluster. It reads the cluster volume that the writer instance writes to, and as that volume is shared, replicating to it takes little additional work. If the writer fails, Aurora promotes a replica in its place.
- Also called
- reader instance
- Taught in
- Availability zones, failover and auto scaling
- Source
authentication tag
A cryptographic checksum on data, designed to reveal both accidental errors and deliberate changes to the data. GCM generates one when it encrypts, and its authenticated decryption, given the tag with the IV, the ciphertext and any additional authenticated data, fails unless the tag is the correct one.
- First used in
- Credential vaults and envelope encryption
- Source
auto-increment
A unique ID given to each new database record by a counter: each request for an ID increments it. Common in databases on a single node, where the order of the IDs is the order the records were created in. A single node that generates the IDs is a single point of failure.
- Also called
- autoincrementing
- Taught in
- Short links: Base62, MD5 and a key service
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 10: ID Generators and Logical Clocks
autoregressive
Generating text one token at a time, each predicted from the prompt and every token generated before it, until a stopping condition such as a maximum length or a stop token.
- Also called
- autoregressive generation
- Taught in
- How a model answers a request
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 2: The Autoregressive Nature of Transformers
autoscaling
What Amazon EC2 Auto Scaling does: it can launch or terminate server instances as demand on an application rises or falls, to help keep the right number there to handle the load.
- Taught in
- Scaling from one user to millions
- Source
availability
How ready a system is to be used: the probability that it is working correctly at a given moment. It is often measured as the share of time it is up, or as the share of requests that succeed.
- Taught in
- Availability in numbers
- Also means
- availability (in CAP)
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 8: Basic concepts
- Site Reliability Engineering (Google)
availability (in CAP)
In CAP, availability means every node that has not failed responds to every request. This formal meaning is not the word's usual one.
- Taught in
- Availability, consistency, CAP and PACELC
- Also means
- availability
- Source
- Database Internals, 1st edition, chapter 11: Infamous CAP
- Designing Data-Intensive Applications, 2nd edition, chapter 10: The Unhelpful CAP Theorem
availability zone
One of several separate, isolated locations within an AWS region.
- Also called
- AZ
- First used in
- Scaling from one user to millions
- Source
Avro
A binary encoding format, started in 2009, that works differently from Protocol Buffers in some ways.
- Also called
- Apache Avro
- Taught in
- Transport protocols, gRPC and service meshes
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: Avro
AWS
Amazon Web Services
Amazon's set of cloud products, such as computing, storage and databases.
- First used in
- Scaling from one user to millions
- Source
AWS Lambda
An AWS compute service that runs your code without you setting up or managing servers.
- Also called
- Lambda
- First used in
- From one user to millions on AWS
- Source
AWS Region
One of AWS's separate geographic areas, each designed to be isolated from the others. Each holds several isolated locations, its availability zones.
AWS Step Functions
A way to create workflows on AWS, which Step Functions also calls state machines, for example to orchestrate microservices. Each step of a workflow is called a state.
- Also called
- Step Functions
- Taught in
- Distributed transactions: two-phase commit and sagas
- Source
B
25 termsB-tree
The structure databases use most to store records and look them up by key. It keeps keys in order, which makes lookups and range queries fast, and divides the database into fixed-size pages that it can overwrite where they lie.
- First used in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: B-Trees
back queue
In Mercator’s URL frontier, one of the FIFO queues of its back end, which is responsible for politeness. Each is nonempty and holds the URLs of only one host; a table maps hosts to back queues, and a heap records when each queue’s web server may be contacted again.
- Also called
- back-end queue
- Taught in
- The URL frontier and politeness
- Source
backend
The part of a web application that runs on servers and handles users' requests, as opposed to the code that runs in the web browser.
- First used in
- Response time versus throughput
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 1: Terminology: Frontends and Backends
backpressure
Making a sender slow down when its recipient cannot keep up, for example by blocking the producer, which lets a queue apply backpressure upstream, or by sending responses that ask clients to slow down.
- Also called
- flow control
- Taught in
- Backpressure, queue lag and dead-letter queues
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
- Release It!, 2nd edition, chapter 5: Create Back Pressure
- Designing Data-Intensive Applications, 2nd edition, chapter 2: When an Overloaded System Won't Recover
backup and restore
The disaster recovery strategy at the low-cost, low-complexity end: making backups. It is a suitable approach for mitigating data loss or corruption, and can also be used against a disaster in a Region, by replicating the data to other AWS Regions.
- Taught in
- Multiple regions and disaster recovery
- Source
bandwidth
The maximum rate at which data can be sent over a communication channel. It is often misused for the rate being sent at the moment, which is throughput.
- First used in
- Performance versus scalability
- Source
- Systems Performance, 2nd edition, glossary
BASE
basically available, soft state, eventual consistency
A name sometimes given to systems that do not meet the ACID criteria. It is even less precise than ACID: about the only sensible reading of it is "not ACID".
- Taught in
- NoSQL data models and access patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 8: The Meaning of ACID
Base64
An encoding that represents any sequence of bytes as text, six bits to a character, in an alphabet that uses letters in both cases as well as + and /. Its characters that are not letters or digits, / in particular, may cause problems in file names and URLs.
- Taught in
- Step 3: Core components
- Source
batch (of requests to a model)
Requests a model server processes together. A decode step still makes one token for each request, but makes them for all of them at once, doing more calculation for each reading of the weights.
- Taught in
- Batching and the KV cache
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 6: Why Do We Need Batching in Real-Time Serving?
batch processing
A computation that takes a fixed, usually large, set of data as its input and produces other data as its output, leaving the input unchanged.
- Also called
- batch job
- First used in
- Step 2: A high level design
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
benchmark
A tool that runs a workload as an experiment and measures its performance.
- First used in
- Denormalization and SQL tuning
- Source
- Systems Performance, 2nd edition, glossary
BFS
BGP
Border Gateway Protocol
The Internet's core routing protocol, between the networks it is made of: each BGP system tells the others which networks can be reached through it. It judges a path mainly by its number of hops, not by its latency.
- First used in
- DNS and global traffic routing
- Source
- IETF RFC
- Understanding Distributed Systems, 2nd edition, chapter 15: Overlay network
BigQuery
Google Cloud’s data warehouse, one of the newer ones that run only in the cloud.
- Also called
- Google BigQuery
- First used in
- Step 4: Scale the design
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Cloud Data Warehouses
Bigtable
Google's database that made the wide-column data model popular.
- First used in
- NoSQL data models and access patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Data locality for reads and writes
binlog
MySQL's log of events that describe changes to the database, such as changes to table data. With row-based replication, MySQL keeps it as its replication log, in addition to its write-ahead log.
- Also called
- binary log
- Taught in
- Single-leader and multi-leader replication
- Source
- MySQL documentation
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Logical (row-based) log replication
Bloom filter
A fast but approximate check of whether a key is in a set. When it says no, the key is certainly absent; when it says yes, the key is probably present. LSM-trees often keep one per SSTable, to skip the files that cannot hold a key.
- Taught in
- Storage engines: B-trees versus LSM-trees
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Bloom filters
bottleneck
The resource that limits a system's performance. Finding and removing bottlenecks is a key part of performance work.
- First used in
- Performance versus scalability
- Source
- Systems Performance, 2nd edition, chapter 2: 2.1 Terminology
branch mispredict
A wrong guess by the processor. At a conditional branch the code goes one way or another depending on a test, so the processor guesses the outcome and starts on it; when the guess proves wrong, that work is thrown away, which hurts performance.
- Also called
- mispredict
- First used in
- Back-of-the-envelope estimation
- Source
- Systems Performance, 2nd edition, chapter 6: 6.3.3 Instruction Pipeline
breadth-first search
brute force
Getting in by trying one combination of password after another.
- First used in
- Reverse proxies and edge security
- Source
buffer pool
The part of a database that keeps pages read from disk in memory, so that reading them again need not go to disk.
- First used in
- Single-leader and multi-leader replication
- Source
- Database Internals, 1st edition, chapter 5: Buffer Management
bulkhead
Splitting a system into separate parts, as a ship is divided into watertight compartments, so that a failure in one part can be kept from destroying the rest. Physical redundancy is the most common form; separate groups of threads inside one process are another.
- Taught in
- Circuit breakers, retries and bulkheads
- Source
- Release It!, 2nd edition, chapter 5: Bulkheads
byte
A unit of digital data: eight bits, each bit a zero or a one.
- First used in
- Response time versus throughput
- Source
- Systems Performance, 2nd edition, glossary
Byzantine fault
A fault in which a node behaves wrongly in an arbitrary way, for example by sending other nodes contradictory or malicious messages.
- Also called
- Byzantine, Byzantine failure
- Taught in
- Consensus and leader election: Raft and Paxos
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
C
70 termsCA
consistent and available
- First used in
- Availability, consistency, CAP and PACELC
cache
A component that keeps recently used data so that reading it again is faster. It usually holds only part of the data; anything missing is fetched from a slower store that has all of it.
- First used in
- Performance versus scalability
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
cache hit
A request for data that can be answered from what the cache holds.
- First used in
- Federation and functional partitioning
- Source
- Systems Performance, 2nd edition, glossary
cache invalidation
Removing cached data once the data it came from has changed: a web server using Facebook's memcache, for one, sends it a delete request for the stale data after each write to the database. It is very hard in practice: a cached query result must be invalidated whenever any data it touched changes; a TTL is a workaround.
- Also called
- invalidation
- Taught in
- Cache invalidation and update strategies
- Source
- Understanding Distributed Systems, 2nd edition, chapter 20: Policies
- USENIX
cache miss
A request for data that is not found in the cache.
- First used in
- Content delivery networks (CDN)
- Source
- Systems Performance, 2nd edition, glossary
cache stampede
What happens when a popular cached item expires: several requests miss the cache and all recompute the same item at the same time. The recomputations slow the system down, making each one slower.
- Also called
- stampede, dog-piling
- Taught in
- Stampedes, penetration and avalanches
- Source
cache-aside
A way of using a cache in which the application is responsible for reading from and writing to storage, and the cache does not deal with storage directly: on a miss the application loads the entry from the database and adds it to the cache.
- Also called
- lazy loading
- Taught in
- Cache invalidation and update strategies
- Source
Cache-Control
The HTTP header that lists instructions for the caches along the way between client and server.
- Also called
- Cache-Control header
- Taught in
- Content delivery networks (CDN)
- Source
canary
A partial, time-limited rollout of a change to a service, together with its evaluation. Testing new software on one instance with a share of the real workload is canary testing.
- Also called
- canary testing
- Taught in
- DNS and global traffic routing
- Source
- Site Reliability Engineering (Google)
- Systems Performance, 2nd edition, chapter 1: 1.3 Activities
candidate (Raft)
CAP theorem
When a network fault cuts some replicas off from the rest, a system must choose: stay linearizable, so the cut-off replicas become unavailable (they wait or return errors), or stay available and give up linearizability. Network faults are not chosen; they happen.
- Also called
- CAP
- Taught in
- Availability, consistency, CAP and PACELC
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 10: The CAP theorem
- Designing Data-Intensive Applications, 2nd edition, chapter 10: The Unhelpful CAP Theorem
CAS
check and set, or compare and swap
In Memcached, an operation that stores data only if nobody has updated it since the client last read it, which helps resolve race conditions when updating cached data. The gets command, which reads data for use with CAS, returns with each item its CAS identifier, a unique 64-bit number.
- First used in
- Memory: slab allocation, calcification and jemalloc
- Source
cascading failure
A failure in one layer of a system that sets off a failure in the layer that calls it; a database failure is the obvious example.
- Taught in
- Circuit breakers, retries and bulkheads
- Source
- Release It!, 2nd edition, chapter 4: Cascading Failures
Cassandra
A distributed NoSQL database. Its replication has no leader, a design inspired by Dynamo.
- First used in
- Availability, consistency, CAP and PACELC
- Source
- Apache Cassandra documentation
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Leaderless Replication
causality
The dependency between events when one happens before another: a later event that responds to an earlier one, builds on it, or must be understood in its light.
- First used in
- Tunable consistency and quorums
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
CDN
content delivery network
A hosting service that distributes and copies the web documents of many sites across the internet, automatically, given its size. In some, nodes hold copies so that clients can reach pages nearby, and so quickly.
- First used in
- Scaling from one user to millions
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 7: Example: Caching and replication in the Web
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 2: Hierarchically organized peer-to-peer networks
Celery
A task queue that supports scheduling, mainly for programs written in Python.
certificate
A record that proves who owns a public key. It carries the owner's details, an expiry date, the public key itself, and a digital signature from the third party that issued it.
- Taught in
- Transport protocols, gRPC and service meshes
- Source
- Understanding Distributed Systems, 2nd edition, chapter 3: Authentication
change data capture
Observing every change written to a database and extracting it so that other systems, such as a search index, can apply the same changes to their own copy. It makes the source database the leader and the others its followers.
- Also called
- CDC
- Taught in
- Partition keys, skew and polyglot persistence
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Change Data Capture
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Implementing CDC
checkpoint
The state of a computation, which a supercomputer’s large batch jobs typically write to disk from time to time. When a node fails, one common response is to halt the whole workload, fix the node, and resume from the last checkpoint.
- Taught in
- Fetching, DNS caching and fault tolerance
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 1: Cloud Computing Versus Supercomputing
chunk
In Memcached, one of the pieces a page is cut into, all of the size its slab class sets. An item goes into the slab class that fits it most nearly.
chunked prefill
Splitting a long prompt’s prefill into smaller chunks, so the running requests’ decode steps are not held up by it. It improves the time per output token at the cost of prefill, which makes the time to first token longer.
- Taught in
- Batching and the KV cache
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 6: Continuous Batching with Chunked Prefill
- Hands-On LLM Serving and Optimization, 1st edition, chapter 4: Latency Metrics
ciphertext
Data in its encrypted form: the output of encryption.
circuit breaker
A component wrapped around risky calls. It notes each failure, and once failures pass a threshold it opens: calls then fail at once without being tried. After a while it goes half-open and lets the next call through; success closes it again, failure opens it again.
- Also called
- breaker
- Taught in
- Circuit breakers, retries and bulkheads
- Source
- Release It!, 2nd edition, chapter 5: Circuit Breaker
client
Whatever uses a service over the network: the program doing so, or the machine it runs on.
- First used in
- Response time versus throughput
- Source
- Systems Performance, 2nd edition, glossary
clock skew
The difference between the times two clocks show at the same moment. No two clocks run at exactly the same rate, so the clocks of different computers drift apart.
- First used in
- Tunable consistency and quorums
- Source
- Understanding Distributed Systems, 2nd edition, chapter 8: Physical clocks
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 5: Physical clocks
CloudFront
An AWS web service that speeds up delivering web content, static or dynamic, to users.
- First used in
- Scaling from one user to millions
- Source
CloudWatch
The AWS service that watches your AWS resources, and the applications running on them, in real time. Many AWS services report metrics to it, and alarms you set up can watch metrics against thresholds you choose.
- Also called
- Amazon CloudWatch
- First used in
- From one user to millions on AWS
- Source
cluster endpoint
The address of an Aurora cluster that connects to the cluster's current writer instance; it is the only one through which writes can be made. If the writer fails, Aurora fails over to a new one, and the cluster endpoint goes on serving connections from it, with minimal interruption.
- Also called
- writer endpoint
- Taught in
- Availability zones, failover and auto scaling
- Source
clustering key
In Cassandra, the columns of a primary key that come after the partition key: they set the order of the rows within a partition.
- Also called
- clustering column
- Taught in
- Partition keys, skew and polyglot persistence
- Source
CNAME
canonical name
A DNS record saying that a name is an alias, and naming the canonical name it stands for.
- Also called
- CNAME record
- First used in
- DNS and global traffic routing
- Source
cold cache
A cache that is empty, or holds data nobody needs, so its hit ratio is zero, or near zero while it begins to warm up. Caches start cold and warm up over time.
- Taught in
- Stampedes, penetration and avalanches
- Source
- Systems Performance, 2nd edition, chapter 2: 2.3.14 Caching
cold start
The wait of a request for a model that is not loaded: it is held until the model is loaded into memory, which means fetching the model’s files and loading them and, if the cache is full, possibly evicting another model.
- Taught in
- Reusing work and starting fast
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 1: Routing and autoscaling in multi-model service
- Hands-On LLM Serving and Optimization, 1st edition, chapter 3: Challenges
commit
The successful completion of a transaction. A transaction that cannot be completed is aborted instead, and the database discards the writes it had made.
- First used in
- Single-leader and multi-leader replication
- Also means
- committed (Raft)
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 8: Atomicity
- Database Internals, 1st edition, chapter 5: Chapter 5. Transaction Processing and Recovery
committed (Raft)
In Raft, said of a log entry that is safe to apply to the state machines: the leader that created it has copied it to a majority of the servers. Raft keeps such entries, and every available state machine will carry them out in time.
- Taught in
- Consensus and leader election: Raft and Paxos
- Also means
- commit
- Source
common neighbors
A score for how likely two vertices of a network are to become linked: the number of neighbors they have in common. Pairs are predicted in order of their score, highest first.
- Taught in
- People you may know: ranking mutual friends
- Source
- Liben-Nowell and Kleinberg, The Link Prediction Problem for Social Networks
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Graph-Like Data Models
commutative
Said of updates that give the same result in whatever order they are applied, as incrementing a counter or adding an item to a set do.
- Taught in
- Distributed transactions: two-phase commit and sagas
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 8: Conflict resolution and replication
compaction
A background process that merges an LSM-tree's sorted files into fewer, keeping only the most recent value of each key and dropping values that were overwritten or deleted.
- Taught in
- Storage engines: B-trees versus LSM-trees
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Constructing and merging SSTables
compensable transaction
A step of a saga that another transaction with the opposite effect can undo, or compensate for. When one of the saga’s steps fails, compensating transactions reverse what the compensable ones changed.
- Also called
- compensable, compensable step
- Taught in
- Distributed transactions: two-phase commit and sagas
- Source
compensating transaction
A transaction that corrects a mistake made by an earlier one. In a saga, each local transaction has a compensating transaction that undoes its changes.
- Also called
- compensating local transaction
- First used in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 13: Loosely interpreted constraints
- Understanding Distributed Systems, 2nd edition, chapter 13: Sagas
composite index
An index on several columns, which combines their values into one key, appending one column to another, in the order the index names them. It works best when a query constrains the leading (leftmost) columns.
- Also called
- concatenated index, multicolumn index
- Taught in
- Denormalization and SQL tuning
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Multidimensional and Full-Text Indexes
- PostgreSQL documentation
compression
Reducing the size of data. It saves space and bandwidth, at the cost of some CPU time.
- First used in
- Reverse proxies and edge security
- Source
- Systems Performance, 2nd edition, chapter 8: 8.3.12 Logical vs. Physical I/O
- Designing Data-Intensive Applications, 2nd edition, chapter 4: The SSTable file format
compute-bound
Describes a task whose time to complete is set by the computation it needs. Spreading it over more chips, or chips with more computational power, can speed it up.
- Taught in
- How a model answers a request
- Source
- AI Engineering, 1st edition, chapter 9: Computational bottlenecks
- AI Engineering, 1st edition, chapter 9: Terminology Ambiguity: Memory-Bound Versus Bandwidth-Bound
connection
A TCP channel between two programs. It has to be opened before any data is sent over it, and a server must be listening for clients asking to open one.
- First used in
- Performance versus scalability
- Source
- Understanding Distributed Systems, 2nd edition, chapter 2: Connection lifecycle
connection pool
A set of open connections that a program keeps so it need not open a new one for every call. Threads check connections out of the pool when they need one.
- First used in
- Response time versus throughput
- Source
- Understanding Distributed Systems, 2nd edition, chapter 2: Connection lifecycle
- Release It!, 2nd edition, chapter 4: Blocked Threads
consensus
Getting several nodes to agree on something, such as which node should lead a database cluster. The problem is much harder than it first looks.
- First used in
- Performance versus scalability
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
consistency (in CAP)
In CAP, consistency means linearizability; the theorem says nothing about weaker consistency models.
- Taught in
- Availability, consistency, CAP and PACELC
- Also means
- consistency model
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 10: The Unhelpful CAP Theorem
consistency level
How many nodes must acknowledge a write, or respond to a read, for it to succeed. Raising it slows requests and needs more nodes up; lowering it makes the system more available, at the cost of consistency.
- Taught in
- Tunable consistency and quorums
- Source
- Database Internals, 1st edition, chapter 11: Tunable Consistency
consistency model
What processes can expect when they read and write a shared data store spread over several machines: whether, and when, one process's writes become visible to the reads of the others.
- Taught in
- Consistency patterns
- Also means
- consistency (in CAP)
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, glossary
consistent hashing
A way of mapping keys to shards that gives each shard about the same number of keys and, when shards are added or removed, moves the fewest keys it can.
- Taught in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Consistent hashing
Consul
A coordination service like ZooKeeper and etcd, one of the main users of consensus. It is not meant for heavy write loads or for storing data in general, and it is often used for service discovery.
- Taught in
- Transport protocols, gRPC and service meshes
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 10: Coordination Services
- Designing Data-Intensive Applications, 2nd edition, chapter 10: Service discovery
consumer
A process that receives the messages added to a queue, or reads a stream of changes.
- First used in
- Partition keys, skew and polyglot persistence
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: Message brokers
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Change Data Capture
consumer group
A group of consumers that read a topic together: each message goes to one consumer in the group, and each group gets every message. In a log-based broker, the number of consumers sharing a topic's work can be at most the number of its shards.
- Taught in
- Queues, task queues and log-based message brokers
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Multiple consumers
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Logs compared to traditional messaging
contention
Competition for a resource. As a component nears full utilization, contention for it can seriously degrade performance.
- First used in
- Performance versus scalability
- Source
- Systems Performance, 2nd edition, glossary
- Systems Performance, 2nd edition, chapter 2: 2.3.11 Utilization
continuous batching
Batching in which a waiting request joins the batch as soon as a running request in it finishes, and each response is returned as soon as it is complete.
- Also called
- in-flight batching, iterative batching
- Taught in
- Batching and the KV cache
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 6: Continuous Batching for LLM Online Inference
- AI Engineering, 1st edition, chapter 9: Batching
control plane
Software that helps manage infrastructure and applications rather than serving users directly, such as the part of a system that manages its configuration. Keeping it apart from the data plane is a common pattern in distributed systems.
- Taught in
- Transport protocols, gRPC and service meshes
- Source
- Release It!, 2nd edition, chapter 4: Outage Amplification
- Understanding Distributed Systems, 2nd edition, chapter 10: Chain replication
cooldown
In Auto Scaling, a wait after a group launches or terminates instances: further scaling started by simple scaling policies cannot start until it ends. Its intention is to let the group stabilize, and to keep it from launching or terminating more instances before the last activity has had a visible effect.
- Also called
- cooldown period, scaling cooldown
- Taught in
- Availability zones, failover and auto scaling
- Source
coordinator
In two-phase commit, the component that asks every node in a transaction whether it can commit, then sends them all the decision to commit or abort.
- Also called
- transaction manager
- First used in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 8: Two-Phase Commit
- Designing Data-Intensive Applications, 2nd edition, chapter 8: A system of promises
cosine similarity
A distance function: how close two vectors are, judged by the cosine of the angle between them. Search engines compare vectors with such functions.
- First used in
- Duplicate elimination and SimHash
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Vector Embeddings
covering index
An index that stores some of a table's columns as well as its keys, so that some queries can be answered from the index alone.
- Also called
- index with included columns
- Taught in
- Denormalization and SQL tuning
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Storing Values Within the Index
CP
consistent under network partitions
Choosing consistency during a network partition: replicas cut off from the rest cannot process requests, so they wait until the network is fixed or return an error.
- Taught in
- Availability, consistency, CAP and PACELC
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 10: The CAP theorem
CPU
central processing unit
The part of a computer that executes a program's instructions. The word is also used for the processor as a whole, or for a virtual CPU.
- Also called
- processor
- First used in
- Denormalization and SQL tuning
- Source
- Systems Performance, 2nd edition, glossary
CPU core
One execution pipeline on a processor. The operating system may show each core as a CPU of its own.
- First used in
- Performance versus scalability
- Source
- Systems Performance, 2nd edition, glossary
CQRS
command query responsibility segregation
Keeping separate representations of the data, each shaped for the reads it serves, and deriving them from the representation that is written to.
- Taught in
- Event sourcing, CQRS and the transactional outbox
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Event Sourcing and CQRS
CRC16
16-bit cyclic redundancy check
A cyclic redundancy check 16 bits wide. A cyclic redundancy check is a method of making sure data was not changed on its way through a communication channel; Redis Cluster uses the CRC16 of a key to find the key’s hash slot.
credential
Evidence that someone has a right to credit or authority.
- First used in
- Requirements, estimates and the data model
- Source
cryptographic hash
A function that turns a message of any length into a string of bits of fixed length, made so that no one can feasibly find two different inputs that give the same result.
- Also called
- cryptographic hash function
- First used in
- Performance versus scalability
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 9: Hash functions
cryptographic key
A parameter that decides how a cryptographic algorithm operates, so that anyone who knows the key can repeat or reverse the operation, and anyone who does not, cannot.
CSS
Cascading Style Sheets
The language of the web for adding style, such as fonts, colors and spacing, to web documents.
- First used in
- Content delivery networks (CDN)
- Source
D
34 termsdata key
In AWS KMS, a key that can encrypt data, large amounts of it included, and other data keys. When AWS KMS generates one, it returns an encrypted copy that can safely be stored with the data, and can return the data key in plaintext for immediate use.
- Taught in
- Credential vaults and envelope encryption
- Source
data model
The general form data takes when it is stored: documents in JSON, rows in the tables of a relational database, or a graph's vertices and edges. Each model suits some data and queries and makes others awkward.
- Taught in
- NoSQL data models and access patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Chapter 3. Data Models and Query Languages
data parallelism
Running copies of a model across GPUs and nodes, to scale throughput.
- Taught in
- Batching and the KV cache
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 7: Multi-GPU and Multi-Node Inferencing
data plane
The part of a system on the critical path, which takes each client request itself. Keeping it apart from the control plane, the part that manages configuration, is a common pattern in distributed systems.
- Taught in
- Transport protocols, gRPC and service meshes
- Source
- Understanding Distributed Systems, 2nd edition, chapter 10: Chain replication
data warehouse
A database that combines data from several operational systems and prepares it for analytics.
- First used in
- Scaling from one user to millions
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
database
A store of information or data. It may be a traditional relational database, or some other kind.
- Also called
- DB
- First used in
- Performance versus scalability
- Source
database per service
A pattern in which each microservice stores and reads its data in a data store of its own. It keeps the services loosely coupled, but complex transactions and queries that span several of them can be challenging to build, and two-phase commit is not an option.
database transaction
Several reads and writes grouped into one logical unit, to make errors, and several things happening at once, simpler to handle.
- First used in
- Performance versus scalability
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
DDoS
distributed denial of service
An attack in which many computers, spread across the Internet, generate load on one site at the same time: a denial of service carried out from many hosts.
- First used in
- DNS and global traffic routing
- Source
- NIST glossary and publications
- Release It!, 2nd edition, chapter 4: Malicious Users
dead-letter queue
A separate queue that a message is moved to when it is not processed successfully, instead of being retried forever, so the consumers are not blocked. An operator then drops it, changes and resends it, or fixes the consumer's code; Amazon SQS calls moving messages out of it redrive.
- Also called
- DLQ
- Taught in
- Backpressure, queue lag and dead-letter queues
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Acknowledgments and redelivery
- AWS documentation
deadlock
Transactions stuck each waiting for another to release its lock, so that none of them can go on.
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 8: Implementation of 2PL
Debezium
An open-source project for change data capture. It connects to the replication logs of databases such as MySQL, PostgreSQL and Cassandra, and publishes the changes in a standard form.
- First used in
- Partition keys, skew and polyglot persistence
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Implementing CDC
decode
The phase in which a language model generates its output one token at a time, repeating for every new token. Each step typically loads large matrices such as the weights, so it is memory-bandwidth-bound.
- Also called
- decode phase
- Taught in
- How a model answers a request
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 2: The Prefill and Decode Phases
- AI Engineering, 1st edition, chapter 9: Terminology Ambiguity: Memory-Bound Versus Bandwidth-Bound
decryption
Changing ciphertext back into plaintext with a cryptographic algorithm and a key.
- First used in
- Reverse proxies and edge security
- Source
denormalization
Storing a copy of the same information wherever it is used, instead of an ID that refers to one copy. Reads are usually faster, with fewer joins, but writes cost more, as every copy must be updated.
- Taught in
- Denormalization and SQL tuning
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Normalization, Denormalization, and Joins
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Trade-offs of normalization
deregistration delay
How long Elastic Load Balancing waits before it completes the deregistration of a target, to which it has stopped sending requests, which can help requests already in flight to finish. It is 300 seconds by default.
dirty read
One transaction reading writes that another has made but not yet committed.
- Taught in
- Distributed transactions: two-phase commit and sagas
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 8: Single-Object and Multi-Object Operations
disaggregation
Running the prefill and decode stages on different GPUs: prefill instances produce each request’s KV cache, which is sent to decode instances. The resources for each stage can then be scaled apart, so the time to first token and the time between tokens can each be improved independently.
- Also called
- prefill-decode disaggregation, PD disaggregation
- Taught in
- Reusing work and starting fast
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 7: Prefill-Decode Disaggregation
- Hands-On LLM Serving and Optimization, 1st edition, chapter 7: Overall Architecture
disk
A physical device that stores data.
- First used in
- Back-of-the-envelope estimation
- Source
- Systems Performance, 2nd edition, glossary
distributed system
A system made of several machines that communicate over a network. Each process taking part in it is called a node.
- First used in
- Availability, consistency, CAP and PACELC
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 1: Distributed Versus Single-Node Systems
distributed transaction
A transaction that involves several nodes, such as one that touches several shards of a sharded database.
- Taught in
- Distributed transactions: two-phase commit and sagas
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 8: Distributed Transactions
DNS
Domain Name System
The system that resolves a domain name to a network address.
- First used in
- Scaling from one user to millions
- Source
- IETF RFC
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 3: Server clusters
DNS record
One piece of data DNS holds about a name; a name can have none or several. Each carries a time to live, which tells caches how long the record stays valid.
- Also called
- resource record
- Taught in
- DNS and global traffic routing
- Source
- IETF RFC
- Understanding Distributed Systems, 2nd edition, chapter 4: Discovery
document database
A database that stores each record as a document, usually in JSON, and usually does not enforce a schema on it. Unlike a key-value store, it reads and indexes the documents, so they can be queried by their internal structure.
- Also called
- document store, document model, document-oriented store
- Taught in
- NoSQL data models and access patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Relational Versus Document Models
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Schema flexibility in the document model
- Understanding Distributed Systems, 2nd edition, chapter 19: NoSQL
domain name
A name in DNS's tree of names, such as www.example.com: the path from a domain to the tree's root. DNS looks up the network address that goes with it.
- Taught in
- DNS and global traffic routing
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 6: Example: The Domain Name System
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 4: Layered Protocols
doubly linked list
A list in which each item links to the item before it as well as the one after, so it can be walked in either direction and any item deleted in constant time.
draft model
In speculative decoding, the small model that generates candidate tokens for the large target model.
- Taught in
- Reusing work and starting fast
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 7: Speculative Decoding
DSR
direct server return
A way for the servers behind a load balancer to bypass it and answer clients directly. Responses usually carry more data than requests, so it can take much of the load off the load balancer.
- Taught in
- Load balancing: layer 4 versus layer 7
- Source
- Understanding Distributed Systems, 2nd edition, chapter 18: Transport layer load balancing
dual write
Keeping several systems in step by having the application write to each of them whenever data changes. Two such writes can race each other, or one can fail while another succeeds, leaving the systems inconsistent.
- Taught in
- Partition keys, skew and polyglot persistence
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Keeping Systems in Sync
duplicate URL eliminator
The component that guards a crawler’s URL frontier, so that it does not download and process one document many times: extracted links are submitted to it, and it passes on only those not submitted before.
durability
Storing data so that you can trust it will not be lost, even when faults happen.
- First used in
- Storage engines: B-trees versus LSM-trees
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
dynamic batching
Batching with a maximum batch size and a maximum delay: the batch is sent as soon as it is full, or when the delay is reached, even with only one request in it.
- Taught in
- Batching and the KV cache
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 6: Dynamic Batching in Online Inference
Dynamo
A highly available key-value storage system built inside Amazon. Riak and Cassandra are open-source data stores inspired by it.
- First used in
- Availability, consistency, CAP and PACELC
- Source
- All Things Distributed (the Dynamo paper)
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Leaderless Replication
DynamoDB
A distributed NoSQL database that AWS runs as a fully managed service. Despite its name, it is not built like Dynamo: it uses single-leader replication.
- Also called
- Amazon DynamoDB
- First used in
- NoSQL data models and access patterns
- Source
- AWS documentation
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Leaderless Replication
E
24 termsEarlybird
Twitter’s real-time retrieval engine, which is built on the open-source Lucene search engine. Tweets are spread over its servers by hash partitioning, and a server indexes each tweet as soon as it has been processed; a front-end server passes each query to several of them, then merges and re-ranks their results.
EC2
Elastic Compute Cloud
An AWS service that provides computing capacity on demand: you start however many virtual servers you need.
- First used in
- Availability in numbers
- Source
edge (in a graph)
One of the two kinds of object in a graph: a link between two vertices, such as which people know each other. Edges are also called relationships or arcs.
- Taught in
- NoSQL data models and access patterns
- Also means
- POP
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Graph-Like Data Models
Elastic IP address
A static public IPv4 address that AWS allocates to your account: it does not change over time, and it is yours until you release it.
- Also called
- Elastic IP
- Taught in
- From one user to millions on AWS
- Source
ElastiCache
An AWS web service that sets up and runs, in the cloud, an in-memory data store or cache spread over several machines.
- First used in
- Scaling from one user to millions
- Source
elasticity
Scaling up or down to meet demand, as a cloud deployment can, so that you pay only for the resources you are using.
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 1: Distributed Versus Single-Node Systems
ELB
Elastic Load Balancing
The AWS service that automatically spreads incoming traffic over several targets, such as EC2 instances or IP addresses, across one or more availability zones.
- First used in
- From one user to millions on AWS
- Source
election timeout
encryption
Changing plaintext into ciphertext with a cryptographic algorithm and a key, so that the data’s original meaning is concealed. Where the change can be reversed, the reverse is decryption.
endpoint
What code calling a web service's API queries over HTTP. The code has to know which endpoint to query, and what data to send and expect back.
- First used in
- DNS and global traffic routing
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: Web services
envelope encryption
Encrypting data with a data key, and that data key under a second key. The encrypted data key can safely sit beside the encrypted data, and one can encrypt just the data keys again under other keys, not the raw data. AWS KMS’s ReEncrypt changes the root key data is encrypted under, as when one is rotated by hand.
Envoy
A layer 7 proxy designed for large systems made of many services.
- First used in
- Load balancing: layer 4 versus layer 7
- Source
epoch
The moment a clock counts from. Time-of-day clocks on Linux and in Java return the seconds or milliseconds since midnight UTC on 1 January 1970, not counting leap seconds, which is called the epoch; some systems count from another date.
- First used in
- Storage, sharding and the timeline cache
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 9: Time-of-day clocks
erasure coding
A way of keeping data on several machines from which lost data can be recovered with less storage overhead than keeping full copies. The disks of one machine can be protected the same way.
- Taught in
- Erasure coding and durability
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 11: Distributed Filesystems and Network Storage
etcd
A strongly consistent, distributed key-value store for data a distributed system needs. When a network partition splits it, the side with a majority of members stays available and the minority side does not.
EU
European Union
- First used in
- DNS and global traffic routing
event
A small, self-contained record, never changed once written, of something that took place at one moment, such as an action a user took; it usually carries a timestamp. A producer generates it once, and several consumers may then process it.
- Taught in
- Queues, task queues and log-based message brokers
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Transmitting Event Streams
event sourcing
Treating events as the source of truth: every change to the state is stored as an event in a log, where events are never changed or deleted, only appended. Views of the data are derived from the log, and can be deleted and rebuilt by running the same events through again, in their original order, with the same code.
- Taught in
- Event sourcing, CQRS and the transactional outbox
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Event Sourcing and CQRS
event stream
Data, such as events, made available bit by bit over time. An event stream is unbounded, the counterpart of batch data, and is processed bit by bit too. Related events are usually grouped into a topic, or stream.
- Taught in
- Queues, task queues and log-based message brokers
- Also means
- stream (HTTP/2)
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Chapter 12. Stream Processing
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Transmitting Event Streams
event time
The time an event happened, on the device that produced it, as against processing time, the clock of the machine that processes it. Confusing the two leads to bad data.
- Taught in
- Stream processing versus batch recomputation
- Source
- Apache Flink documentation (release 1.20)
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Event time versus processing time
eventual consistency
Replicas that disagree for a while, then agree: if writes stop for a while, the followers catch up with the leader. The disagreement is temporary.
- Taught in
- Availability, consistency, CAP and PACELC
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Problems with Replication Lag
eviction
Removing entries from a full cache to make room for new ones. Which entry goes depends on the cache's eviction policy, such as least recently used (LRU), and on how the entries are accessed.
- Also called
- eviction policy
- Taught in
- Eviction algorithms and memory management
- Source
- Understanding Distributed Systems, 2nd edition, chapter 20: Policies
exactly-once semantics
Processing in which, though failures may make some records be processed more than once, the visible effect is as if each had been processed exactly once; effectively-once would be a better name. For a message, it can be had by committing the message and the effects of processing it atomically, so retries are harmless.
- Also called
- exactly-once, exactly-once processing
- Taught in
- Delivery guarantees, ordering and idempotency
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Fault Tolerance
- Designing Data-Intensive Applications, 2nd edition, chapter 8: Exactly-once message processing
exponential backoff
Waiting longer between one retry and the next, so that retries do not overload a service. One way is a capped exponential function: the wait is the initial backoff multiplied by a constant that grows exponentially with each attempt, up to a cap.
- Taught in
- Backpressure, queue lag and dead-letter queues
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: When an Overloaded System Won't Recover
- Understanding Distributed Systems, 2nd edition, chapter 27: Retry
F
21 termsfail fast
Failing an operation at once when the system can tell in advance that it will fail, so the caller does not tie up its capacity waiting.
- Taught in
- Circuit breakers, retries and bulkheads
- Source
- Release It!, 2nd edition, chapter 5: Fail Fast
failover
Moving the leader role from one node to another, in a system with a single leader.
- First used in
- Scaling from one user to millions
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
failover routing
A DNS routing policy for active-passive failover: DNS answers with the primary resources while they are healthy, and with the secondary ones, kept on standby, once all the primary ones are unhealthy.
false positive
A wrong yes from a Bloom filter: it reports a key as there when it is not. How likely one is depends on how many keys the filter holds, how many bits each key sets, and how many bits the filter has in all.
- Taught in
- Duplicate elimination and SimHash
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Bloom filters
fan-out
How many downstream requests one request turns into, when serving it means carrying out several other requests. On a social network, fan-out is also the delivering of each new post to every follower of the user who made it.
- First used in
- Response time versus throughput
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Materializing and Updating Timelines
fault tolerance
The property of a system that keeps working correctly when some of its components fail.
- Taught in
- Consensus and leader election: Raft and Paxos
- Source
- Database Internals, 1st edition, chapter 11: Chapter 11. Replication and Consistency
federation
Splitting one database into several by function, such as users, products and orders, so each carries less traffic. Designing Data-Intensive Applications uses "federated databases" for something else: one query interface over several different data stores.
- Also called
- functional partitioning
- Taught in
- Federation and functional partitioning
- Source
- The System Design Primer
- Designing Data-Intensive Applications, 2nd edition, chapter 13: The meta-database of everything
fencing token
A number that a lock service hands out with each lock, larger every time. Every write to storage must carry the writer's current token, so storage can reject a write whose token is lower than one it has already processed.
- Taught in
- Single-leader and multi-leader replication
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 9: Fencing off zombies and delayed requests
FIFO
first in, first out
Messages taken in the order they were sent: a receiver can restore that order with their sequence numbers, and an Amazon SQS FIFO queue preserves it. As a policy for replacing pages in a database's full buffer of pages, FIFO evicts the page added longest ago, ignoring later use, which makes it impractical for most real systems.
- Taught in
- Eviction algorithms and memory management
- Source
- Database Internals, 1st edition, chapter 5: FIFO and LRU
- Database Internals, 1st edition, chapter 8: Message order
- AWS documentation
FIPS
Federal Information Processing Standard
A standard published by NIST for federal departments and agencies to adopt and use.
firewall
A device or program that lets through only the network traffic its configured rules allow, to protect a network.
- Source
- Systems Performance, 2nd edition, chapter 10: 10.4.2 Hardware
- NIST glossary and publications
Flink
A distributed engine and framework for computations that keep state, over streams of data with or without an end.
- Also called
- Apache Flink
- First used in
- Stream processing versus batch recomputation
- Source
floating point
PostgreSQL’s floating-point types, real and double precision, are inexact: some values are stored as approximations, so a value read back may differ slightly. For exact storage and calculations, as for amounts of money, PostgreSQL’s documentation says to use the numeric type
- Also called
- binary floating point
- First used in
- Requirements, estimates and the data model
- Source
follower (in TAO)
In TAO, a cache server in one of a region’s follower tiers. Clients talk to the closest follower tier and never to a leader directly; a follower sends its read misses and writes on to its region’s leader tier, and the leader tells it of updates asynchronously.
- Also called
- follower cache, follower tier
- Taught in
- Graph partitioning and the TAO cache
- Also means
- read replicafollower (Raft)follower (social network)
- Source
follower (Raft)
In Raft, the state of a server that makes no requests of its own and only answers the leader and candidates. Servers start as followers, and one that hears nothing for its election timeout starts an election.
foreign key
A rule that the values in a column must match the values in some row of another table. It keeps references from one table to the other valid, which is called referential integrity.
- Also called
- referential integrity
- Taught in
- Federation and functional partitioning
- Source
fragmentation
Memory split into many unused regions, left behind by allocations of many different sizes, which together waste memory. An allocator can try to merge them so that larger allocations can use them.
- Also called
- memory fragmentation
- Taught in
- Eviction algorithms and memory management
- Source
- Systems Performance, 2nd edition, chapter 7: 7.3.3 Process Virtual Address Space
frame
A message at the data link layer, the level of the physical network. One level up, at the network layer, a message is called a packet.
- First used in
- Load balancing: layer 4 versus layer 7
- Source
- Systems Performance, 2nd edition, glossary
- Systems Performance, 2nd edition, chapter 10: 10.1 Terminology
- Systems Performance, 2nd edition, chapter 10: 10.2.3 Protocol Stack
freshness
In HTTP caching, a stored response is fresh until its age passes its freshness lifetime, the time from when the origin server made it until it expires; after that it is stale.
- Also called
- freshness lifetime
- Taught in
- Content delivery networks (CDN)
- Source
front queue
In Mercator’s URL frontier, one of the FIFO queues of its front end, which is responsible for prioritizing URLs: one queue for each priority. When a back queue empties, it is refilled from a front queue chosen at random, with a bias towards the high-priority ones.
- Also called
- front-end queue
- Taught in
- The URL frontier and politeness
- Source
G
15 termsgateway endpoint
A way for a VPC to reach Amazon S3 and DynamoDB without an internet gateway or a NAT device. Using one costs nothing extra.
- Also called
- gateway VPC endpoint
- Taught in
- Cost: compute, storage and network
- Source
GB
gigabyte
- First used in
- Response time versus throughput
GCM
Galois/Counter Mode
A mode of operation of AES, or another approved block cipher with 128-bit blocks, for authenticated encryption: it encrypts plaintext into ciphertext and generates an authentication tag on the ciphertext and any additional authenticated data. NIST requires the chance of one IV and key ever encrypting two different inputs to be at most 1 in 2³²
- Also called
- AES-GCM, AES-256-GCM
- Taught in
- Credential vaults and envelope encryption
- Source
geolocation routing
A DNS routing policy that answers according to where the user is.
- Also called
- geolocation
- Taught in
- DNS and global traffic routing
- Source
geoproximity routing
A Route 53 routing policy that routes traffic by where your resources are, and can shift traffic from resources in one place to resources in another.
- Also called
- geoproximity
- Taught in
- Multiple regions and disaster recovery
- Source
GET method
The HTTP request method that asks for a current representation of the target resource. It is the main way information is retrieved over HTTP.
- Also called
- GET
- First used in
- Redirects, caching and visit analytics
- Source
goodput
The number of requests a second that meet the application’s objectives, such as a limit on the time to first token and on the time per output token. Some teams focus on it instead, since judging a service on throughput and cost alone can lead to a bad user experience.
- Taught in
- Throughput, cost and hosted models
- Source
- AI Engineering, 1st edition, chapter 9: Throughput and goodput
GPU
graphics processing unit
A graphics processing unit, which can also be used for other workloads, such as machine learning. On servers and cloud instances it can run the part of a workload suited to highly parallel work on data, such as transforming matrices.
- Taught in
- How a model answers a request
- Source
- Systems Performance, 2nd edition, glossary
- Systems Performance, 2nd edition, chapter 6: 6.4.1 Hardware
graph database
A database that models data as a graph: vertices, such as people, joined by edges, such as who knows whom. It becomes the more natural choice when many-to-many relationships are common and the connections in the data grow complex.
- Also called
- graph model, graph data model
- Taught in
- NoSQL data models and access patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Graph-Like Data Models
graph partitioning
GraphQL
A query language, deliberately restricted, with which client software on a user’s device, for example a mobile app, asks for a JSON document of a particular shape, holding the fields it needs to show its user interface. Despite “graph” in its name, it can be built on any kind of database: relational, document or graph.
- First used in
- Requirements, estimates and the graph model
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: GraphQL
grouped-query attention
A form of attention that divides the query heads into groups, each group sharing one key and value. It was introduced to cut the KV cache’s size while keeping the model’s quality.
- Also called
- GQA
- Taught in
- Batching and the KV cache
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 6: Scalable Attention Mechanisms
- Hands-On LLM Serving and Optimization, 1st edition, chapter 6: Scaling Attention and Kernel Optimization
growth factor
gRPC
An open-source RPC framework: a client can call a method on a server on another machine as though it were a local object. By default it describes the service and its messages with Protocol Buffers.
GTID
global transaction identifier
In MySQL, a unique ID given to each transaction committed on the server where it began. GTIDs are one of MySQL’s ways of naming a position in the replication log: a new follower, set up from a snapshot taken at that position, asks the leader for every change made since.
- Taught in
- Single-leader and multi-leader replication
- Source
- MySQL documentation
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Setting Up New Followers
H
30 termsHamming distance
The number of bits that differ between two strings of bits.
handshake
The messages two sides exchange to set up a connection. TCP opens a connection with a three-way handshake; a TLS handshake then has client and server agree on how to secure it.
- Also called
- three-way handshake
- First used in
- DNS and global traffic routing
- Source
- Understanding Distributed Systems, 2nd edition, chapter 2: Connection lifecycle
- Understanding Distributed Systems, 2nd edition, chapter 3: Handshake
HAProxy
A software load balancer and reverse proxy that runs on an ordinary machine.
- First used in
- Load balancing: layer 4 versus layer 7
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: Load balancers, service discovery, and service meshes
- Release It!, 2nd edition, chapter 9: Software Load Balancing
hardware
The material, physical parts of a system.
- First used in
- Performance versus scalability
- Source
hardware security module
A physical computing device that keeps cryptographic keys safe, manages them, and does cryptographic processing.
- Also called
- HSM
- First used in
- Credential vaults and envelope encryption
- Source
hash chain
hash collision
Two or more different inputs for which a function gives the same output. Hash functions can have collisions.
- Also called
- collision
- Taught in
- Step 3: Core components
- Source
- NIST glossary and publications
- Database Internals, 1st edition, chapter 12: Digest Reads
hash function
A function that turns data of any length into a bit string of fixed length, its hash.
- Also called
- hash
- First used in
- Content delivery networks (CDN)
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 9: Hash functions
hash sharding
Sharding that first hashes the partition key and then maps the hash to a shard, used when it does not matter whether keys sit near each other. A good hash function spreads keys evenly, even keys that are bunched together.
- Also called
- hash-based sharding, hash partitioning
- Taught in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Sharding by Hash of Key
hash slot
One of the 16,384 slots that Redis Cluster splits its key space into, in practice a set of keys. Each master node handles a subset of them. Unless a key has a hash tag, its slot is the CRC16 of the key, mod 16,384.
hash table
A data structure that keeps items in an array, placing each key at the position a hash function gives it. Its lookups can take constant time, if the table is large enough or is allowed to grow.
hash tag
In Redis Cluster, a way to put several keys in the same hash slot: when a key holds a part written between { and }, only that part is hashed to find its slot. With hash tags, clients can use commands on several keys at once.
HBase
A database that uses the wide-column data model Bigtable made popular, and stores its data with the LSM-tree algorithm.
- First used in
- NoSQL data models and access patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Data locality for reads and writes
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Constructing and merging SSTables
head-of-line blocking
Slow requests at the front delaying everything queued behind them. A server works on only a few things at once, so a handful of slow requests is enough, and the requests behind respond slowly even if their own service time is short.
- First used in
- Backpressure, queue lag and dead-letter queues
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Latency and Response Time
health check
An end-to-end route that every application and service should include. A load balancer can use it to tell whether a machine has crashed. Route 53, a DNS service, runs health checks too, gathering what its health checkers report to decide whether an endpoint is healthy.
- First used in
- Scaling from one user to millions
- Source
- Release It!, 2nd edition, chapter 13: Rollout
- Release It!, 2nd edition, chapter 8: Health Checks
- AWS documentation
- AWS documentation
heap
A complete tree of keys in which every node’s key is greater than or equal to its parent’s, or else every node’s key is less than or equal to its parent’s.
heartbeat
A signal a server sends at regular intervals to say it is still up and running.
- First used in
- Availability patterns and failover
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: Load balancers, service discovery, and service meshes
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 5: Example: Leader election in Raft
heuristically cacheable
Said of an HTTP response whose status code lets a cache reuse it with no explicit expiration time, by estimating one from other fields, such as when the resource was last modified, unless the request method's definition or explicit cache controls say otherwise.
- Taught in
- Redirects, caching and visit analytics
- Source
hinted handoff
While a replica is unavailable, another replica may keep the writes meant for it, as hints. When the replica comes back, the hints are sent to it and then deleted.
- Taught in
- Tunable consistency and quorums
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Catching up on missed writes
hit ratio
The ratio of cache hits to hits plus misses: the share of requests a cache answers itself, usually as a percentage. Higher is better.
- Also called
- cache hit ratio, hit rate
- Taught in
- Content delivery networks (CDN)
- Source
- Systems Performance, 2nd edition, glossary
home timeline
What a user sees of the people they follow: their recent posts. It can be computed ahead of time, by inserting each new post into the home timeline of every one of its author’s followers, and then served from a cache.
- Taught in
- Requirements, estimates and the timeline API
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Representing Users, Posts, and Follows
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Materializing and Updating Timelines
hopping window
A window of fixed length, a new one starting every slide. Consecutive windows can overlap, when the slide is shorter than the window, to smooth the results. Hopping windows are also known as sliding windows.
- Taught in
- Stream processing versus batch recomputation
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Types of windows
- Apache Flink documentation (release 1.20)
horizontal scaling
Growing what a system can handle by adding more, smaller machines, instead of moving it to one bigger machine.
- Also called
- scaling out
- Taught in
- Performance versus scalability
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Pros and Cons of Sharding
host splitter
In a Mercator crawl spread over several processes, the component that checks, for each extracted URL, whether this process was assigned its host name. The URLs it was assigned go on to its duplicate URL eliminator; the others are sent to the appropriate peer process.
hosted model
A model used through a model API instead of hosted yourself: an inference service hosts it, runs it for each query and returns the responses, and the model API is the interface users reach it through. Model APIs are offered by model providers, cloud service providers and third-party API providers, and charge per use.
- Also called
- model API
- Taught in
- Throughput, cost and hosted models
- Source
- AI Engineering, 1st edition, chapter 4: Open source models versus model APIs
- AI Engineering, 1st edition, chapter 4: API cost versus engineering cost
hot key
A key with far more load than the others, such as the key of a celebrity in a social network.
- Taught in
- Stampedes, penetration and avalanches
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Sharding of Key-Value Data
hot spot
A shard with far more load than the others. Imbalanced load of this kind is also called skew.
- Also called
- hot shard
- Taught in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Sharding of Key-Value Data
- Designing Data-Intensive Applications, 2nd edition, glossary
HTML
Hypertext Markup Language
The core markup language of the World Wide Web.
- First used in
- Content delivery networks (CDN)
- Source
HTTP
Hypertext Transfer Protocol
A family of stateless request/response protocols that share one generic interface.
- First used in
- Scaling from one user to millions
- Source
HTTP header
A name and a value sent in the header section of an HTTP request or response, to give data about the message.
- Also called
- header field
- First used in
- Content delivery networks (CDN)
- Source
I
29 termsI/O
input/output
Transferring data in or out, such as reading from and writing to a disk.
- First used in
- Performance versus scalability
- Source
- Systems Performance, 2nd edition, glossary
- Systems Performance, 2nd edition, chapter 2: 2.1 Terminology
IAM
AWS Identity and Access Management
An AWS web service that controls access to AWS resources: who is signed in, and who has permission to use which resources.
- First used in
- Credential vaults and envelope encryption
- Source
ICDE
International Conference on Data Engineering
- First used in
- Real-time search and scatter-gather
ID
identifier
A name that refers to exactly one thing. Each thing has exactly one, and it is never given to anything else.
- First used in
- Horizontal sharding and consistent hashing
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 6: Summary
idempotency key
A unique identifier, such as a UUID, attached to a request so the server can tell a repeat of it from a new one. The server stores every identifier it has seen; kept in the same database as the server's data, they can be written in one ACID transaction with the work.
- Taught in
- Delivery guarantees, ordering and idempotency
- Source
- Understanding Distributed Systems, 2nd edition, chapter 5: Idempotency
idempotent
Describes an operation that is safe to retry: running it several times leaves things as running it once would.
- First used in
- Partition keys, skew and polyglot persistence
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
IIPC
International Internet Preservation Consortium
The consortium from whose discussions and experience came the motivation to extend an older format into the WARC format. Its members include national libraries and the Internet Archive.
- First used in
- Fetching, DNS caching and fault tolerance
- Source
index
A data structure that makes it fast to find every record with a given value in a given field.
- First used in
- Performance versus scalability
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
index-only scan
In PostgreSQL, answering a query from an index alone, without reading the table's rows themselves.
- Taught in
- Denormalization and SQL tuning
- Source
inference server
infrastructure as code
Applying the rigor of developing application code to setting up infrastructure: all configurations should be defined declaratively and kept in source control, as application code is.
- First used in
- Multiple regions and disaster recovery
- Source
InnoDB
MySQL's default storage engine.
- First used in
- Federation and functional partitioning
- Source
instance
A virtual server. Cloud computing provides servers as instances.
- First used in
- Performance versus scalability
- Also means
- instance (of a service)
- Source
- Systems Performance, 2nd edition, glossary
instance (of a service)
One copy of a service's program, running on one machine (physical, virtual or a container), out of a group of identical copies that a load balancer spreads requests over.
- First used in
- Transport protocols, gRPC and service meshes
- Also means
- instance
- Source
- Release It!, 2nd edition, chapter 8: Chapter 8 Processes on Machines
instance warmup
In Auto Scaling, how long an instance waits, once it is in service, before its usage data counts in the group's metrics, so that instances not yet handling application traffic do not affect dynamic scaling. AWS strongly recommends it for target tracking and step scaling.
- Also called
- default instance warmup
- Taught in
- Availability zones, failover and auto scaling
- Source
interface definition language
A formal language for describing the interfaces of a distributed system's components, from which code can be generated in different programming languages.
- Also called
- IDL
- Taught in
- Transport protocols, gRPC and service meshes
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, glossary
inverted index
The data structure many search engines use for full-text search. It is a key-value structure: each key is a term, and its value holds the IDs of every document containing that term, the term’s postings list.
- Taught in
- Real-time search and scatter-gather
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Full-Text Search
IOPS
input/output operations per second
How many I/O operations happen each second. For a disk, it counts its reads and its writes.
- First used in
- Single-leader and multi-leader replication
- Source
- Systems Performance, 2nd edition, glossary
- Systems Performance, 2nd edition, chapter 2: 2.1 Terminology
IP
Internet Protocol
The protocol for sending blocks of data across interconnected networks, from a source to a destination. Both are hosts, each identified by a fixed-length address.
- Also called
- IP address
- First used in
- Availability patterns and failover
- Source
IP hash
A way to spread requests in which a hash of the client's IP address chooses the server, so that one client's requests keep going to the same server.
- Taught in
- Load balancing: layer 4 versus layer 7
- Source
IP-based routing
A Route 53 routing policy that routes traffic by where users are, when you know the IP addresses their traffic comes from.
- Also called
- IP-based
- Taught in
- Multiple regions and disaster recovery
- Source
IPv6
Version 6 of the Internet Protocol, the successor to version 4 (IPv4). Its addresses are 128 bits long instead of 32.
- First used in
- DNS and global traffic routing
- Source
IPVS
Load balancing at the transport layer, inside the Linux kernel: running on one host, it stands in front of a cluster of real servers and directs requests to them.
- First used in
- Load balancing: layer 4 versus layer 7
- Source
isolation
How far transactions running at the same time can interfere with each other. Databases offer several levels of it, from a strongest guarantee to weaker ones.
- First used in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
ISP
Internet service provider
An organization that connects other networks, such as a business's, to the Internet.
- First used in
- DNS and global traffic routing
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 2: The edge-cloud architecture
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 1: Scalability
Istio
One service mesh. Those whose services run in a very dynamic environment under an orchestrator often choose a service mesh such as Istio.
- Taught in
- Transport protocols, gRPC and service meshes
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: Load balancers, service discovery, and service meshes
istiod
In Istio, the component that provides service discovery, configuration and certificate management. It turns high level routing rules into configuration specific to Envoy and sends it to the sidecars at runtime.
item (in Memcached)
In Memcached, a stored piece of data with its key. An item takes space for its key, its data and Memcached’s own internal record of it, and goes into the slab class whose chunks fit it most nearly.
IV
J
7 termsJaccard coefficient
A measure of how alike two things are, common in information retrieval. For predicting links between two vertices, it is the number of neighbors they share divided by the number of neighbors either of them has.
- Also called
- Jaccard's coefficient
- Taught in
- People you may know: ranking mutual friends
- Source
- Liben-Nowell and Kleinberg, The Link Prediction Problem for Social Networks
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Graph-Like Data Models
jemalloc
A memory allocator, available for Linux among other systems. It uses caches for each thread and slabs for small objects, among other techniques, to scale better and to reduce memory fragmentation.
- Taught in
- Eviction algorithms and memory management
- Source
- Systems Performance, 2nd edition, chapter 7: 7.3.3 Process Virtual Address Space
jitter
Small random amounts of time added to the backoff before a retry, so that clients are less likely to retry all at once.
- Taught in
- Stampedes, penetration and avalanches
- Source
- Database Internals, 1st edition, chapter 8: Cascading Failures
join
Bringing together records that have something in common, most often when one record refers to another (by a foreign key, say) and a query needs the record it refers to.
- First used in
- Federation and functional partitioning
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
JS
JavaScript
A lightweight programming language, best known as the scripting language of web pages, though many environments outside the browser use it too.
- First used in
- Content delivery networks (CDN)
- Source
JSON
JavaScript Object Notation
A lightweight, text-based format for exchanging data, independent of any programming language.
- First used in
- NoSQL data models and access patterns
- Source
JWT
JSON Web Token
A compact token, safe to put in a web address, that carries claims between two parties. Its claims can be digitally signed, or encrypted.
- First used in
- Performance versus scalability
- Source
K
10 termsKafka
A persistent event bus that combines a message queue with a distributed log: events stay in the log until the space runs out.
- Also called
- Apache Kafka
- First used in
- Partition keys, skew and polyglot persistence
- Source
- Release It!, 2nd edition, chapter 16: Messages, Events, and Commands
Kappa architecture
An approach, sometimes known by this name, in which batch computations, which reprocess past data, and stream computations, which handle events as they come in, can be implemented in one system. That needs the ability to replay past events through the same engine that handles the recent ones.
- Taught in
- Stream processing versus batch recomputation
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 13: Unifying batch and stream processing
KB
kilobyte
In computing, usually 1,024 bytes. The International System of Units makes a kilobyte 1,000 bytes.
- First used in
- Back-of-the-envelope estimation
- Source
- Systems Performance, 2nd edition, glossary
Ketama
A library that maps keys to servers, written to replace how its authors’ Memcached clients did it. It hashes each server to several numbers, conceptually placed on a circle, and each key to one number, owned by the server of the next number after it. Adding or removing a server then moves only a small share of keys.
- Also called
- libketama
- Taught in
- Partitioning: consistent hashing and hash slots
- Source
key-range sharding
Sharding in which each shard holds every partition key between a lowest and a highest value, like the volumes of a paper encyclopedia. Range scans are easy, but you can easily get a hot shard when there are many writes to nearby keys.
- Also called
- range sharding, range-based sharding, range partitioning
- Taught in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Sharding by Key Range
key-value store
A data store that maps each key to a value. In its pure form it treats both as bytes it does not interpret; Dynamo, for one, offers simple reads and writes of an item identified by its key.
- Also called
- key-value
- Taught in
- NoSQL data models and access patterns
- Source
- Understanding Distributed Systems, 2nd edition, chapter 19: NoSQL
- All Things Distributed (the Dynamo paper)
key/value head
A key and value head of a model’s attention. In the original design each query needs one of its own; with grouped-query attention, a group of query heads shares one.
- Also called
- KV head
- Taught in
- Batching and the KV cache
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 6: Scalable Attention Mechanisms
KGS
key generation service
KMS
AWS Key Management Service
An AWS service that protects its users’ root keys. Everything done with its KMS keys, from creating them to using and deleting them, happens inside it, and they never leave it unencrypted. To use or manage one, a caller calls AWS KMS.
- Also called
- AWS KMS
- First used in
- Credential vaults and envelope encryption
- Source
KV cache
The attention keys and values a model has computed at each layer for earlier tokens, stored so that decoding skips computing them again: more memory for less computation. Space for it is kept in GPU memory beside the weights, and it grows with longer prompts and longer answers.
- Also called
- KV
- Taught in
- Batching and the KV cache
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 2: Chapter 2. Large Language Model Serving
- Hands-On LLM Serving and Optimization, 1st edition, chapter 2: Enable the KV Cache to Boost Performance
- Hands-On LLM Serving and Optimization, 1st edition, chapter 5: Estimating KV Cache Size
L
41 termsL/C
L1
level 1 cache
The first of a CPU's caches for main memory: very fast, but small.
- First used in
- Back-of-the-envelope estimation
- Source
- Systems Performance, 2nd edition, chapter 2: 2.3.14 Caching
L2
level 2 cache
The second level of a CPU's caches for main memory: larger than level 1, and slower to reach.
- First used in
- Back-of-the-envelope estimation
- Source
- Systems Performance, 2nd edition, chapter 2: 2.3.14 Caching
L4
layer 4
The transport layer of the network stack, where TCP runs. A layer 4 load balancer works at this level: it generally looks at addresses and ports, not at what the packets carry.
- Also called
- transport layer
- Taught in
- Load balancing: layer 4 versus layer 7
- Source
- Understanding Distributed Systems, 2nd edition, chapter 18: Transport layer load balancing
- The System Design Primer
L7
layer 7
The application layer of the network stack, where HTTP runs. A layer 7 load balancer is an HTTP reverse proxy: it reads each request and sends it to a server of its choice.
- Also called
- application layer
- Taught in
- Load balancing: layer 4 versus layer 7
- Source
- Understanding Distributed Systems, 2nd edition, chapter 18: Application layer load balancing
- AWS documentation
Lambda architecture
An early proposal for bringing batch processing and stream processing together. It had several problems, and has fallen out of use.
- Taught in
- Stream processing versus batch recomputation
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 13: Unifying batch and stream processing
LAN
local area network
A group of computers and other devices in a fairly small area, such as a building, linked so that any of them can talk to any other.
- First used in
- Load balancing: layer 4 versus layer 7
- Source
- NIST glossary and publications
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 1: Introduction
language model
A model holding statistics about one or more languages: the chance of each word in a given context. Given a text, the prompt, it tries to complete it.
- Taught in
- How a model answers a request
- Source
- AI Engineering, 1st edition, chapter 1: Language models
last write wins
Resolving conflicting writes by keeping the one with the greatest timestamp and discarding the others. When writes are concurrent, which one counts as last is in effect random, so writes can be lost.
- Also called
- LWW
- Taught in
- Single-leader and multi-leader replication
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Last write wins (discarding concurrent writes)
- Designing Data-Intensive Applications, 2nd edition, chapter 9: Timestamps for ordering events
latency
Waiting time: the part of a request's life when nothing is working on it, such as a queueing delay or time crossing the network. Sometimes used to mean the whole response time.
- Taught in
- Response time versus throughput
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Latency and Response Time
latency-based routing
A DNS routing policy for resources in several regions: each query is answered with the region that gives the best latency.
- Also called
- latency routing
- Taught in
- DNS and global traffic routing
- Source
leader (in TAO)
In TAO, the cache server in a region’s leader tier that hosts a shard. All writes to the shard go through it, it reads from and writes to the database, and it sends refills and invalidations to the followers asynchronously.
- Also called
- leader cache, leader tier
- Taught in
- Graph partitioning and the TAO cache
- Also means
- leader (replication)
- Source
leader (replication)
When data is copied across several nodes, the one replica allowed to make changes. Also called the primary, or the source; Redis calls it the master.
- Also called
- master
- First used in
- Scaling from one user to millions
- Also means
- leader (in TAO)
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
- Redis documentation
leader election
Choosing which process is the leader, the one that coordinates the steps of a distributed algorithm. It must leave at most one leader at any time, and finish even when there are failures. In one algorithm that Database Internals describes, an election starts when a process notices there is no leader, or the leader has stopped responding.
- First used in
- Single-leader and multi-leader replication
- Source
- Database Internals, 1st edition, chapter 10: Chapter 10. Leader Election
- Database Internals, 1st edition, chapter 10: Bully Algorithm
- Understanding Distributed Systems, 2nd edition, chapter 9: Leader election
leaderless replication
Replication with no leader: any replica accepts writes straight from clients, and a client writes to several nodes and reads from several at once, to detect and correct stale data. Riak and Cassandra work this way, after Dynamo, so it is also called Dynamo-style.
- Also called
- leaderless, masterless
- Taught in
- Tunable consistency and quorums
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Leaderless Replication
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Summary
leaf page
A page at the bottom of a B-tree. It holds individual keys, each with its value or with references to the pages where the values are kept.
- Taught in
- Storage engines: B-trees versus LSM-trees
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: B-Trees
leaky bucket
A rate-limiting method. As NGINX applies it, requests above the set rate are held back and handled at that rate, and once more are waiting than the burst size allows, further ones are refused with an error.
- Taught in
- Reverse proxies and edge security
- Source
lease
In Facebook's memcache, a token for one key, given to a client that missed and shown when it sets the value. The server checks it to decide whether to store the value, and the check fails if a delete has invalidated it. Handing out a key's tokens only once every 10 seconds also limits thundering herds.
- Taught in
- Cache invalidation and update strategies
- Source
least connections
A way to spread requests: each one goes to the server that has the fewest active connections at that moment.
- Taught in
- Load balancing: layer 4 versus layer 7
- Source
LFU
least frequently used
An eviction policy that removes the item used least often. One such policy orders items by how often they are used rather than how recently.
- Taught in
- Eviction algorithms and memory management
- Source
- Systems Performance, 2nd edition, chapter 2: 2.3.14 Caching
- Database Internals, 1st edition, chapter 5: LFU
lifecycle rule
A rule in an S3 Lifecycle configuration: an action S3 takes on a group of objects, such as moving them to another storage class a set number of days after they were created.
- Also called
- S3 Lifecycle
- Taught in
- Cost: compute, storage and network
- Source
linearizability
As soon as a write completes, every client that reads the data must see the value just written.
- Also called
- strong consistency
- Taught in
- Consistency patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 10: Linearizability
listener
In an AWS load balancer, a process that checks for connection requests on the protocol and port you set. A load balancer with no listener receives no traffic.
- First used in
- Load balancing: layer 4 versus layer 7
- Source
Little's law
On average, the number of requests in a system equals the arrival rate times the average time each spends there: L = λW. It holds for a whole system, or for its queue alone.
- Taught in
- Response time versus throughput
- Source
- Systems Performance, 2nd edition, chapter 2: 2.6.5 Queueing Theory
LLM
large language model
The name for strong pre-trained language models, with tens to hundreds of billions of parameters each.
- First used in
- Batching and the KV cache
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 2: LLM Evolution
load
The work applied to a system, usually described as a rate: requests per second to a service, say, or gigabytes of new data a day.
- First used in
- Performance versus scalability
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Understanding Load
- Systems Performance, 2nd edition, chapter 2: 2.1 Terminology
load balancing
Spreading requests across several instances of a service, running on different machines, any of which can handle a request.
- Taught in
- Performance versus scalability
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: Load balancers, service discovery, and service meshes
load testing
A test that runs for a set length of time and then stops, with load generators configured for it. Real production traffic cannot be duplicated, so a load test aims to simulate reality as closely as possible.
- Taught in
- Step 4: Scale the design
- Source
- Release It!, 2nd edition, chapter 3: Extending Your Life Span
- Release It!, 2nd edition, chapter 15: Load Testing
locality
An optimization for performance: keeping pieces of data that are often needed at the same time in the same place.
- Also called
- data locality
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
lock
A way to make sure only one thread, node or transaction can use something at a time; anyone else who wants it waits for the lock to be released.
- First used in
- Performance versus scalability
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
log
An append-only file for storing data. A write-ahead log makes a storage engine survive crashes, a log-structured storage engine keeps its data in logs, a replication log copies writes from a leader to its followers, and an event log can hold a stream of data; a message broker can keep its messages in one.
- Also called
- append-only log
- First used in
- Single-leader and multi-leader replication
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Using logs for message storage
log shipping
Moving a database's write-ahead log records directly from one server to another. In PostgreSQL the same capability is also called warm standby.
log-based message broker
A message broker that keeps messages in a log: a producer appends each message to the end, and consumers read the log in order; reading a message does not delete it. It joins the durable storage of a database with the quick notifications of messaging; Kafka is one.
- Also called
- log-based broker
- Taught in
- Queues, task queues and log-based message brokers
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Log-Based Message Brokers
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Using logs for message storage
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Logs compared to traditional messaging
logical clock
Instead of measuring time, an algorithm that counts events as they occur. Its timestamps do not tell the time, but comparing two of them shows which came first.
- Taught in
- Consensus and leader election: Raft and Paxos
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 10: Logical Clocks
logical replication
Replication through a log kept apart from the storage engine's own format, usually recording writes to tables row by row. MySQL's binlog is such a log when set to row-based replication, and PostgreSQL implements logical replication by decoding its WAL.
- Also called
- logical log, logical replication log
- First used in
- Partition keys, skew and polyglot persistence
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Logical (row-based) log replication
long polling
LoRA
low-rank adaptation
A way to adapt a model to a task by inserting small low-rank layers into it and training them, without changing its original weights. Several such LoRA adapters can be loaded in GPU memory at once, so that one model serving instance serves the requests of different adapters together.
- Also called
- adapter, LoRA adapter, multi-LoRA
- Taught in
- Throughput, cost and hosted models
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 10: Multi-LoRA Serving
lost update
When two clients each read a value, change it and write it back at the same time, and one write replaces the other without including its change, so that change is lost.
- Taught in
- Distributed transactions: two-phase commit and sagas
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 8: Summary
LRU
least recently used
An eviction policy that removes the item that has gone longest without being used. It keeps items in the order they were added, and moves an item to the back each time it is used again.
- Taught in
- Eviction algorithms and memory management
- Source
- Systems Performance, 2nd edition, chapter 2: 2.3.14 Caching
- Database Internals, 1st edition, chapter 5: FIFO and LRU
LSM-tree
log-structured merge-tree
A way of storing data that writes sorted files, never changes a file once written, and later merges files together. Storage engines built this way are often called LSM storage engines.
- Also called
- LSM
- First used in
- Denormalization and SQL tuning
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Constructing and merging SSTables
Lucene
The full-text indexing engine behind Elasticsearch and Solr. It keeps each term’s postings list in sorted files, like SSTables, and merges those files in the background.
- First used in
- Real-time search and scatter-gather
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Full-Text Search
M
32 termsM/M/1
Markovian arrivals, Markovian service times, one server
A queueing model with one server and unlimited room to wait, in which the arrival rate is lower than the server's service rate.
- Also called
- M/M/1 queue, M/M/1 model
- Taught in
- Response time versus throughput
- Source
- Systems Performance, 2nd edition, chapter 2: 2.6.5 Queueing Theory
- MIT OpenCourseWare
MAC address
media access control address
- Also called
- MAC
- First used in
- Load balancing: layer 4 versus layer 7
many-to-many relationship
A relationship in which a record on either side can be linked to many on the other: a person may have worked at several companies, and each company employs many people.
- Also called
- many-to-many
- Taught in
- NoSQL data models and access patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Many-to-One and Many-to-Many Relationships
MapReduce
A batch processing algorithm published by Google in 2004. A job runs two functions you write: a mapper, called once for every input record to extract a key and a value, and a reducer, called with all the values that belong to the same key.
- Taught in
- Redirects, caching and visit analytics
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 11: Chapter 11. Batch Processing
- Designing Data-Intensive Applications, 2nd edition, chapter 11: MapReduce
materialized view
A copy of the results of a query, written to disk. It has to be updated when the data under it changes. In event sourcing, materialized views (also called projections or read models) are brought up to date as each new event is added to the log.
- Also called
- read model
- Taught in
- Denormalization and SQL tuning
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Materialized Views and Data Cubes
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Event Sourcing and CQRS
MB
megabyte
In computing, usually 1,048,576 bytes (2 to the 20th). The International System of Units makes a megabyte 1,000,000 bytes.
- First used in
- Back-of-the-envelope estimation
- Source
- Systems Performance, 2nd edition, glossary
MD5
Message Digest 5
A hash function that takes a message of any length and produces a 128-bit message digest of it. MongoDB, for example, uses it to shard data.
- Taught in
- Step 3: Core components
- Source
- NIST glossary and publications
- IETF RFC
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Sharding by Hash of Key
median
The halfway point of a list of response times sorted from fastest to slowest: with a median of 200 ms, half the requests take less than 200 ms and half take longer.
- Also called
- p50, 50th percentile
- First used in
- Fan-out on write, on read, or both
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Average, Median, and Percentiles
memcache
Facebook's name for its distributed caching system, as against Memcached, the program it runs. It takes read load off Facebook's databases: a web server asks memcache for a value, and if it is not cached, reads it from the database and adds it to the cache.
- Taught in
- Cache invalidation and update strategies
- Source
Memcached
A key-value store held in memory, for small pieces of data such as what a database query or an API call returned.
- First used in
- Scaling from one user to millions
- Source
memory
A system's primary memory storage, usually built from RAM.
- Also called
- main memory
- First used in
- Performance versus scalability
- Source
- Systems Performance, 2nd edition, glossary
memory barrier
What a program uses to make sure that writes become visible, in order, to threads running at the same time. Without one, a thread on one CPU core may not read a value that a thread on another core has just written, because every core keeps its own memory cache and store buffer.
- First used in
- Real-time search and scatter-gather
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 10: Linearizability and network delays
- Database Internals, 1st edition, chapter 11: Sequential Consistency
memory hierarchy
The kinds of storage ranked by size and by distance from the CPU: the fastest and closest are its registers, followed by caches, main memory, disk, tape and so on. Most databases work with two of its levels: slower disk and faster main memory.
- Taught in
- Caching tiers and cache placement
- Source
- Release It!, 2nd edition, chapter 4: Off-Heap Memory, Off-Host Memory
- Database Internals, 1st edition, chapter 5: Buffer Management
memory-bandwidth-bound
Describes a task limited by how fast data moves within the system, such as between memory and processors; often called memory-bound. Chips with higher bandwidth can speed it up.
- Also called
- bandwidth-bound, memory-bound
- Taught in
- How a model answers a request
- Source
- AI Engineering, 1st edition, chapter 9: Computational bottlenecks
- AI Engineering, 1st edition, chapter 9: Terminology Ambiguity: Memory-Bound Versus Bandwidth-Bound
memtable
The in-memory structure, kept sorted by key, into which an LSM-tree puts each write. Once it grows past a threshold, it is written out to disk, in key order, as an SSTable.
- Taught in
- Storage engines: B-trees versus LSM-trees
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Constructing and merging SSTables
Merkle tree
A tree of hashes over a replica's data: the lowest level hashes ranges of records, which can be as small as one key's value, as in Dynamo, and each level above hashes the level below it. Comparing two replicas' trees from the top down finds the ranges that differ.
- Also called
- hash tree
- Taught in
- Tunable consistency and quorums
- Source
- Database Internals, 1st edition, chapter 12: Merkle Trees
- All Things Distributed (the Dynamo paper)
message queue
A server that programs send messages through, in effect a kind of database optimized for handling streams of messages.
- Also called
- message broker, broker
- First used in
- Scaling from one user to millions
- Also means
- queuequeue (data structure)
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Message brokers
metadata
Information about data rather than the data itself: for files, their contents are data, and what describes the files is metadata.
- First used in
- NoSQL data models and access patterns
- Source
- Systems Performance, 2nd edition, chapter 8: 8.3.11 Metadata
metric
A series over time of raw measurements of how a resource is used, such as CPU utilization, or of how something behaves, such as the number of requests that failed. Each sample is a number with a timestamp.
- First used in
- From one user to millions on AWS
- Source
- Understanding Distributed Systems, 2nd edition, chapter 31: Metrics
microservice
A small, independent service that runs as its own process on the network; many are put together to form a larger application. Microservices mainly let different teams make progress without coordinating with each other.
- First used in
- Event sourcing, CQRS and the transactional outbox
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 2: Service-oriented architectures
- Designing Data-Intensive Applications, 2nd edition, chapter 1: Microservices and Serverless
modulo hashing
Choosing a key's node by taking its hash modulo the number of nodes. When the number of nodes changes, most of the keys have to move to another node.
- Also called
- mod N
- Taught in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Hash modulo number of nodes
MongoDB
A document database, one of those that made the document model popular.
- First used in
- NoSQL data models and access patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Relational Versus Document Models
monotonic reads
A guarantee that a user who reads several times in a row never sees time go backward: after reading newer data, they do not read older data. It is weaker than strong consistency, and stronger than eventual consistency.
- Taught in
- Consistency patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Monotonic reads
Multi-AZ deployment
In Amazon RDS, a Multi-AZ DB instance deployment: RDS keeps a standby replica of the database, updated synchronously, in another availability zone, and that standby cannot serve reads.
- Also called
- Multi-AZ
- Taught in
- From one user to millions on AWS
- Source
multi-leader replication
Replication with more than one leader, each also acting as a follower of the others. Also called active/active or bidirectional replication.
- Also called
- multi-leader, master-master, master-master replication, multi-master, multi-master replication
- Taught in
- Availability patterns and failover
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Multi-Leader Replication
multi-site active/active
A disaster recovery strategy that runs the workload in several Regions at once, each serving traffic. It is the most complex and costly strategy, but, with the right technology choices and implementation, it can bring recovery time close to zero for most disasters; data corruption may still need backups.
- Taught in
- Multiple regions and disaster recovery
- Source
multiplexing
In HTTP/2, carrying many requests on one connection at the same time, each request and its response on a stream of its own. The streams are largely independent, so one that is stalled does not hold up the rest.
- Taught in
- Transport protocols, gRPC and service meshes
- Source
multivalue answer routing
A Route 53 routing policy that answers DNS queries with as many as eight healthy records, chosen at random.
- Also called
- multivalue answer
- Taught in
- Multiple regions and disaster recovery
- Source
mutex
A mutual exclusion lock. Mutexes can become a performance bottleneck.
- First used in
- Back-of-the-envelope estimation
- Source
- Systems Performance, 2nd edition, glossary
mutual TLS
TLS with authentication in both directions, of the client as well as the server; TLS itself always checks the server's identity and only optionally the client's. With Amazon API Gateway, for example, clients present a certificate to prove who they are.
- Also called
- mTLS
- Taught in
- Transport protocols, gRPC and service meshes
- Source
MX
mail exchange
A DNS record that names a mail server for the domain: a host that accepts mail sent to its name.
- Also called
- MX record
- First used in
- DNS and global traffic routing
- Source
MySQL
A relational database management system: it stores data in separate tables rather than all in one place.
- First used in
- Single-leader and multi-leader replication
- Source
N
21 termsname server
Generally, in RFCs, a server on the Internet that takes DNS queries and responds to them. Authoritative servers and recursive resolvers alike are often called name servers, though their roles differ.
- Also called
- DNS server
- First used in
- Fetching, DNS caching and fault tolerance
- Source
NAT
network address translation
Mapping IP addresses from one address space to another, giving the hosts at either end transparent routing.
- First used in
- Load balancing: layer 4 versus layer 7
- Source
NAT gateway
An AWS network address translation service. Through it, instances in a private subnet can reach services beyond their VPC, while those services cannot open a connection to the instances.
- Taught in
- Cost: compute, storage and network
- Source
near-duplicate
A web document whose content is identical to another’s except for a small part, such as advertisements, counters or timestamps: differences that do not matter to web search.
- Taught in
- Duplicate elimination and SimHash
- Source
negative caching
Caching the fact that something does not exist, so that looking it up again fails faster. One file system cache does it, remembering lookups of entries that do not exist.
- Taught in
- Stampedes, penetration and avalanches
- Source
- IETF RFC
- Systems Performance, 2nd edition, chapter 8: 8.4.3 File System Caches
Neo4j
A graph database: it stores data as vertices joined by edges.
- First used in
- NoSQL data models and access patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Graph-Like Data Models
network
A group of connected hosts, each known by its network protocol address. Messages, called packets, are routed across networks to reach their destination.
- First used in
- Performance versus scalability
- Source
- Systems Performance, 2nd edition, chapter 10: 10.3.1 Networks and Routing
network jitter
Variation in network delay.
- First used in
- Single-leader and multi-leader replication
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Average, Median, and Percentiles
Network Load Balancer
The AWS load balancer that works at layer 4. It can take millions of requests a second.
- Also called
- NLB
- First used in
- Load balancing: layer 4 versus layer 7
- Source
network partition
A network fault that cuts one part of the network off from the rest.
- Also called
- netsplit
- Taught in
- Availability, consistency, CAP and PACELC
- Also means
- shardingpartition (Cassandra)partition (Kafka)
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 9: Network partitions
nginx
A software load balancer and reverse proxy that runs on an ordinary machine, built to handle thousands of connections at once.
- First used in
- Load balancing: layer 4 versus layer 7
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: Load balancers, service discovery, and service meshes
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 3: Server clusters
nines
A shorthand for availability as a percentage made of nines: three nines is 99.9%, five nines 99.999%.
- Taught in
- Availability in numbers
- Source
- Release It!, 2nd edition, chapter 1: The Scope of the Challenge
- Site Reliability Engineering (Google)
NIST
National Institute of Standards and Technology
- First used in
- Credential vaults and envelope encryption
- Source
node
One participant in a distributed system: software running on a computer that works with other nodes over a network. The word is also used for the machine, a system connected to the network.
- First used in
- Performance versus scalability
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
- Systems Performance, 2nd edition, glossary
nonrepeatable read
Reading the same row twice in one transaction and getting different results; it can happen, for example, when another transaction changes the row and commits in between.
- Also called
- fuzzy read
- Taught in
- Distributed transactions: two-phase commit and sagas
- Source
- Database Internals, 1st edition, chapter 5: Read and Write Anomalies
normalization
Storing each piece of information people care about in one place only, and referring to it elsewhere by an ID. Writes are usually faster, as there is one copy, but reads need joins.
- Taught in
- Denormalization and SQL tuning
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Normalization, Denormalization, and Joins
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Trade-offs of normalization
NoSQL
A loose set of ideas for databases other than the dominant relational kind: new data models, flexible structure, scalability and open-source licensing.
- First used in
- Scaling from one user to millions
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Relational Versus Document Models
NS
A DNS record naming an authoritative server for a domain.
- Also called
- NS record
- First used in
- DNS and global traffic routing
- Source
NTP
Network Time Protocol
The most common way to synchronize computers' clocks: each adjusts its clock to the time a group of servers reports, and those servers take their time from a more accurate source.
- First used in
- Tunable consistency and quorums
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 9: Unreliable Clocks
numeric type
PostgreSQL’s type for numbers that can have very many digits, recommended for amounts of money and other quantities that must be exact. Decimal and numeric are the same type.
- Also called
- decimal, decimal type
- First used in
- Requirements, estimates and the data model
- Source
NVMe
Non-Volatile Memory Express
A specification for how storage devices connect to a computer.
- First used in
- Performance versus scalability
- Source
- Systems Performance, 2nd edition, glossary
O
7 termsobject (in TAO)
In TAO, a typed vertex of the graph, such as a user, a check-in or a comment. Each has a 64-bit ID that no other object shares, whatever its type.
- Taught in
- Requirements, estimates and the graph model
- Source
- USENIX
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Graph-Like Data Models
object storage
A storage service for large files, such as Amazon S3. It offers fewer operations than a file system, but spreads the data across many machines for you.
- First used in
- Scaling from one user to millions
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 1: Layering of cloud services
objectively down
In Redis Sentinel, the condition a master reaches when enough Sentinels, at least its quorum, find it subjectively down. No failover can start without it, and starting one also needs the authorization of a majority.
- Taught in
- Replication, failover and client routing
- Source
offset
A message's sequence number within a shard (partition) of a log-based message broker, rising with each message. A consumer has processed every message below its current offset; after a failure it resumes from the last recorded offset, so it can see some messages twice.
- Taught in
- Queues, task queues and log-based message brokers
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Using logs for message storage
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Consumer offsets
On-Demand Instance
An EC2 instance you pay for by the second, with no long-term commitment.
- Also called
- On-Demand
- Taught in
- Cost: compute, storage and network
- Source
origin server
The server that can answer with authority for a piece of content: where it originates. A CDN fetches from it what it does not hold, caches it and returns it to the client.
- Also called
- origin
- Taught in
- Content delivery networks (CDN)
- Source
- IETF RFC
- Understanding Distributed Systems, 2nd edition, chapter 15: Content delivery networks
OS
operating system
The software and files installed on a computer so that it can start up and run programs.
- Taught in
- Caching tiers and cache placement
- Source
- Systems Performance, 2nd edition, chapter 3: 3.1 Terminology
P
53 termsp99
99th percentile
Shorthand for the 99th percentile. A p99 response time of 200 ms means 99 requests in 100 finish faster than that. p95 and p999 are the 95th and 99.9th percentiles.
- Also called
- p95
- Taught in
- Response time versus throughput
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Average, Median, and Percentiles
PA/EL
on a partition, availability; else, latency
A system that gives up consistency for availability when a partition occurs, and gives it up for lower latency the rest of the time.
- Also called
- PA
- First used in
- Availability, consistency, CAP and PACELC
- Source
PACELC
if partitioned, availability or consistency; else, latency or consistency
During a partition, a system trades availability against consistency; otherwise, while it runs normally, it trades latency against consistency.
- Taught in
- Availability, consistency, CAP and PACELC
- Source
- Daniel Abadi (the PACELC paper)
- Designing Data-Intensive Applications, 2nd edition, chapter 10: The CAP theorem
packet
A message as a network carries it: routing delivers packets across networks to their destination.
- First used in
- Back-of-the-envelope estimation
- Source
- Systems Performance, 2nd edition, glossary
- Systems Performance, 2nd edition, chapter 10: 10.3.1 Networks and Routing
page (in Memcached)
One of the blocks, 1 MB by default, into which Memcached divides the memory it keeps for items. Each page is assigned to a slab class when needed, and cut into chunks of the size set for that class.
page (storage)
One of the fixed-size blocks a B-tree breaks the database into. Each page has a number, so one page can refer to another on disk.
- First used in
- Denormalization and SQL tuning
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: B-Trees
page cache
The file system cache. As first introduced, in 1985, it cached pages of virtual memory, including pages of files, making file and directory I/O faster.
- Taught in
- Queues, task queues and log-based message brokers
- Source
- Systems Performance, 2nd edition, chapter 7: 7.3.2 Software
- Systems Performance, 2nd edition, chapter 8: 8.4.3 File System Caches
page signature
A value that the primer’s crawler would generate from the contents of a page and compare with the signatures of the pages it has crawled, to find duplicate content. The primer’s crawler lowers the priority of a link whose page has a signature similar to one already crawled.
- Also called
- signature, content signature
- Taught in
- Requirements, estimates and a crawler's parts
- Source
PagedAttention
A way of managing the KV cache, inspired by paging in operating systems: the cache is split into fixed-size blocks that a lookup table maps to, so it need not sit in contiguous memory.
- Taught in
- Batching and the KV cache
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 6: PagedAttention
PageRank
An algorithm that can be used on the web graph, in which the vertices are web pages and the edges are links to other pages, to determine how popular each page is and so where it ranks in search results.
- First used in
- Requirements, estimates and a crawler's parts
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Graph-Like Data Models
partial resynchronization
In Redis, how a replica tries to catch up when its link to the master breaks: it reconnects and asks for only the commands it missed while disconnected, sending the master’s replication ID and the offsets it has processed. When that is not possible, it asks for a full copy of the data.
- Taught in
- Replication, failover and client routing
- Source
participant
In a simple distributed commit, each of the processes taking part other than the coordinator, which tells them whether to carry out the operation. In two-phase commit they are the database nodes in the transaction.
- Taught in
- Distributed transactions: two-phase commit and sagas
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 8: Distributed commit
- Designing Data-Intensive Applications, 2nd edition, chapter 8: Two-Phase Commit
partition (Cassandra)
In Cassandra, all the rows whose partition key has the same value. A hash of the partition key decides where those rows are stored.
partition (Kafka)
In Kafka, one shard of a topic: a separate log, read and written independently of the others. The broker gives each message in it a rising sequence number, its offset. Since a partition is append-only, its messages are in one total order; there is no guarantee of order across partitions.
- Taught in
- Queues, task queues and log-based message brokers
- Also means
- network partitionshardingpartition (Cassandra)
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Using logs for message storage
partition key
The value that decides which shard a record goes in: records that share it always land on the same shard.
- Also called
- shard key
- Taught in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Pros and Cons of Sharding
partition tolerance
Continuing to work while the network is split into parts whose processes cannot communicate, so that some messages between them never arrive.
- Taught in
- Availability, consistency, CAP and PACELC
- Source
- Database Internals, 1st edition, chapter 11: Infamous CAP
partitioner
In Cassandra, the part that spreads groups of rows across the nodes of a cluster by their partition key. It cannot be changed unless all the data is loaded again.
Paxos
A consensus algorithm, well established and highly robust.
- First used in
- Consistency patterns
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 8: Summary
PB
petabyte
- First used in
- Back-of-the-envelope estimation
PCI DSS
Payment Card Industry Data Security Standard
An information security standard for organizations handling the branded credit cards of the major card schemes. The Payment Card Industry Security Standards Council administers it.
PDF
Portable Document Format
- First used in
- Queues, task queues and log-based message brokers
percent-encoding
Writing a data octet in a component of a URI as a percent sign and the two hexadecimal digits of its numeric value, as %20 for the octet of a space in US-ASCII. It is used when the octet's character falls outside the allowed set, or serves as a delimiter.
- Also called
- escaping
- Taught in
- Short links: Base62, MD5 and a key service
- Source
percentile
A threshold that a stated share of measurements fall below. If the 95th percentile response time is 200 ms, then 95% of requests finished in under 200 ms and 5% took longer.
- First used in
- Fan-out on write, on read, or both
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
performance problem
A system that is slow even for a single user.
- Taught in
- Performance versus scalability
- Source
permissive mode
In Istio, the setting under which a service accepts both plain text traffic and mutual TLS traffic at the same time. A mesh-wide policy that sets no mode uses it by default.
- Also called
- permissive
- Taught in
- Transport protocols, gRPC and service meshes
- Source
pessimistic view
A countermeasure against anomalies in sagas: reordering the steps of a saga so that data is updated in retryable transactions, to eliminate dirty reads.
pilot light
A disaster recovery strategy that replicates data to another Region and provisions a copy of the core infrastructure there. Data stores are always on, while servers such as application servers are switched off, so they must be turned on and scaled up before the Region can take traffic.
- Taught in
- Multiple regions and disaster recovery
- Source
pipeline parallelism
Splitting a model’s layers across several GPUs, each responsible for a contiguous block of them. When one stage is slow, the stages after it may be left waiting.
- Taught in
- Batching and the KV cache
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 7: Tensor Parallelism and Pipeline Parallelism
pipelining
Sending several requests without waiting for each response. In HTTP, a client may pipeline requests over a persistent connection, and the server must answer them in order. Redis supports it: a client can send many commands, then read all the replies in one step; in Redis’s example, it pays the RTT once for three commands, not once for each.
- Also called
- HTTP pipelining
- Taught in
- Transport protocols, gRPC and service meshes
- Source
pivot transaction
The point of no return in a saga: once it succeeds, the compensable transactions are no longer relevant. It can be the last step that can be undone, or the first that can be retried.
- Also called
- pivot
- Taught in
- Distributed transactions: two-phase commit and sagas
- Source
plaintext
Data that is input to encryption: data whose meaning can be read without decryption.
- First used in
- Credential vaults and envelope encryption
- Source
poison pill
A message that is received but cannot be processed, in Amazon SQS's words; a dead-letter queue reduces the chance of being exposed to such messages. Understanding Distributed Systems uses the name for malformed requests that crash the servers handling them because of a bug.
- Taught in
- Backpressure, queue lag and dead-letter queues
- Source
- AWS documentation
- Understanding Distributed Systems, 2nd edition, chapter 26: Fault isolation
politeness
polyglot persistence
Giving each part of a system its own data store, and choosing the storage technology of each by its data access patterns and other needs. The stores then usually end up only eventually consistent, and transactions, keeping data in step, and joins across them become challenges to weigh.
POP
point of presence
A place where a CDN keeps cached content on edge servers close to its users, to cut latency. Amazon CloudFront also calls them edge locations: DNS sends each request to the one that can serve it best, usually the nearest in latency, and when the object is not in the cache there, CloudFront forwards the request to the origin server.
- Also called
- edge location
- Taught in
- Content delivery networks (CDN)
- Also means
- edge (in a graph)
- Source
port
The end point on a machine that clients send their requests to. Each server listens on a specific one.
- First used in
- Load balancing: layer 4 versus layer 7
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 3: General design issues
POST method
The HTTP request method that asks the target resource to process the data enclosed in the request by its own rules. Creating a new resource is one of its uses.
- Also called
- POST
- First used in
- Redirects, caching and visit analytics
- Source
PostgreSQL
An open-source object-relational database management system, descended from a database first written at the University of California at Berkeley.
- First used in
- Single-leader and multi-leader replication
- Source
postings list
In an inverted index, the value kept for a term: the IDs of every document that contains it.
- Taught in
- Real-time search and scatter-gather
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Full-Text Search
precision
The data type of a model’s parameters: how many bits store each one, 16 bits taking 2 bytes and 8 bits 1. Lowering it generally costs some accuracy, but makes the model smaller and improves serving performance.
- Taught in
- How a model answers a request
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 5: Estimating Model Size
predictive scaling
Auto Scaling that looks for daily or weekly patterns in past load, forecasts the capacity needed from them, and adds capacity ahead of the load it expects.
prefill
The phase in which a language model processes the whole input prompt at once, its tokens in parallel. It is compute-bound.
- Also called
- prefill phase, prompt processing
- Taught in
- How a model answers a request
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 2: The Prefill and Decode Phases
- AI Engineering, 1st edition, chapter 9: Terminology Ambiguity: Memory-Bound Versus Bandwidth-Bound
prefix caching
Matching the beginning of a new prompt, its prefix, against the prompts a model has already processed; where they match, the KV cache of the matched part is reused instead of computed again. Also called a prompt cache or context cache: overlapping segments of prompts are stored so they are processed once.
- Also called
- prompt caching, context cache
- Taught in
- Reusing work and starting fast
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 6: Prefix Caching
- AI Engineering, 1st edition, chapter 9: Prompt caching
primary key
A value, usually a number or a string, that identifies a record uniquely. It is often generated by the system when the record is created.
- First used in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
prioritizer
In Mercator’s URL frontier, the pluggable component that gives each URL added a priority between 1 and k, from the URL and its download history (such as whether the page changed since it was last downloaded), and puts it in the front queue for that priority.
probabilistic early expiration
A defense against cache stampedes: each request decides on its own, at random, whether to recompute an item before it expires, with a chance that grows as the expiry nears. XFetch makes the decision with an exponential function, whose setting need not depend on the request rate.
- Also called
- XFetch
- Taught in
- Stampedes, penetration and avalanches
- Source
producer
Whatever generates events; a common approach is to send each in a message that a messaging system pushes to consumers.
- Also called
- publisher
- Taught in
- Queues, task queues and log-based message brokers
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Transmitting Event Streams
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Messaging Systems
profiling
Collecting data that describes the performance of a target. In systems performance it usually means using tools that take a sample of measurements to give a rough picture of the target; CPUs are a common target.
- Taught in
- Step 4: Scale the design
- Source
- Systems Performance, 2nd edition, glossary
- Systems Performance, 2nd edition, chapter 1: 1.7.2 Profiling
Protocol Buffers
A binary encoding library from Google that needs a schema for any data it encodes. gRPC uses it by default.
- Also called
- protobuf
- Taught in
- Transport protocols, gRPC and service meshes
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: Protocol Buffers
- gRPC documentation
proxy
A go-between that forwards messages, chosen by the client. A reverse proxy, also called a gateway, differs: to its clients it acts as the origin server, but it passes the requests it receives on to one or more other servers.
- Also called
- proxy server
- First used in
- Content delivery networks (CDN)
- Source
pull CDN
A CDN that fetches content from your server the first time a user requests it, and keeps it for a time to live. Until it is cached, requests for it are slower.
- Taught in
- Content delivery networks (CDN)
- Source
Pulsar
A log-based messaging system.
- Also called
- Apache Pulsar
- Taught in
- Queues, task queues and log-based message brokers
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Acknowledgments and redelivery
push CDN
A CDN that receives new content whenever it changes on your server: content is placed on it once, rather than fetched again at intervals. It suits sites with little traffic, or content that is rarely updated.
- Taught in
- Content delivery networks (CDN)
- Source
Q
5 termsqueue
Where work waits its turn, such as tasks waiting to run on a CPU. In a queueing model the queue is the waiting area, not the whole system.
- First used in
- Response time versus throughput
- Also means
- message queuequeue (data structure)
- Source
- Systems Performance, 2nd edition, glossary
- MIT OpenCourseWare
queue (data structure)
Items kept so that only the one added earliest can be accessed: each is added at the tail and removed from the head, first in, first out.
- Taught in
- The URL frontier and politeness
- Also means
- queuemessage queue
- Source
queueing delay
Time a request or its response spends waiting: for a CPU to become free before it is processed, say, or in a buffer before going out over the network. It is separate from the service time, and it grows sharply as throughput nears the hardware's limit.
- Taught in
- Response time versus throughput
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Latency and Response Time
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Describing Performance
quorum (in Redis Sentinel)
The number of Sentinels that must agree a master cannot be reached before it is marked as failing. It is used only to detect the failure: to carry out a failover, one Sentinel must also be elected for it by a majority of the Sentinel processes.
- Taught in
- Replication, failover and client routing
- Also means
- quorum (replication)
- Source
quorum (replication)
The smallest number of nodes that must vote for an operation before it counts as successful. With n replicas, writes confirmed by w nodes and reads asking r nodes, w + r > n means a read should reach an up-to-date copy.
- Taught in
- Consistency patterns
- Also means
- quorum (in Redis Sentinel)
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Using quorums for reading and writing
R
58 termsRabbitMQ
A message broker built on the traditional model that the AMQP standard sets out, in which messages are deleted soon after they are delivered to consumers. Using it means adopting AMQP and managing your own nodes.
- Taught in
- Queues, task queues and log-based message brokers
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Message brokers compared to databases
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Log-Based Message Brokers
- The System Design Primer
race condition
A concurrency problem that can arise when clients work on the same database records at the same time, and that shows only when their steps happen to be timed badly: two clients each add 1 to a counter, and it goes up by only one.
- First used in
- Fan-out on write, on read, or both
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 8: Isolation
- Designing Data-Intensive Applications, 2nd edition, chapter 8: Weak Isolation Levels
Raft
A consensus algorithm designed to be easy to understand.
- First used in
- Consistency patterns
- Source
RAM
random-access memory
Its dynamic kind is in common use as main memory. That kind is volatile, which means it forgets everything once the power goes off.
- First used in
- Performance versus scalability
- Source
- Systems Performance, 2nd edition, glossary
- Systems Performance, 2nd edition, chapter 7: 7.3.1 Hardware
range scan
Fetching every record whose key falls in a range, such as all the readings from one month.
- Also called
- range query
- First used in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Sharding by Key Range
rate limiting
Rejecting a request once a quota is exceeded; quotas are typically applied to particular users, API keys or IP addresses. Over HTTP the refusal is usually 429 Too Many Requests.
- Also called
- throttling
- Taught in
- Reverse proxies and edge security
- Source
- Understanding Distributed Systems, 2nd edition, chapter 28: Rate-limiting
RDBMS
relational database management system
Software that stores data by the relational model and is queried with SQL. By the mid-1980s it had become the usual tool for storing and querying data with a regular structure.
- First used in
- Single-leader and multi-leader replication
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Relational Versus Document Models
RDS
An AWS web service for setting up, running and scaling a relational database in the cloud.
- First used in
- Scaling from one user to millions
- Source
read repair
Repairing replicas as data is read: a client that reads from several replicas at once spots one returning an old value and sends it the new one. It suits values that are read often.
- Taught in
- Tunable consistency and quorums
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Catching up on missed writes
read replica
A replica that takes no writes from clients directly, only the changes it receives from the leader.
- Taught in
- Scaling from one user to millions
- Also means
- follower (Raft)follower (social network)follower (in TAO)
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
read-through cache
A cache that, in the example Release It! gives, returns the cached object on a hit, and on a miss does the query, caches the result and then returns it.
- Also called
- read-through
- Taught in
- Cache invalidation and update strategies
- Source
- Release It!, 2nd edition, chapter 4: Spot the Blocking
read-your-writes
A guarantee that if users reload the page, they always see the updates they submitted themselves.
- Also called
- read-after-write consistency, read-your-writes consistency
- Taught in
- Consistency patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Reading your own writes
rebalancing
Moving shards between nodes as nodes are added or removed, to balance the system's load. It is expensive: requests must be rerouted and a lot of data moved.
- Taught in
- Horizontal sharding and consistent hashing
- Source
- Understanding Distributed Systems, 2nd edition, chapter 16: Range partitioning
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Operations: Automatic Versus Manual Rebalancing
redirect
Redis
A data store that keeps its data structures in memory.
- First used in
- Performance versus scalability
- Source
Redis Cluster
A distributed implementation of Redis. Each of its master nodes handles a subset of the hash slots the keys are split into, replication is asynchronous, and nodes do not pass a command on to the right node: they redirect the client to it.
Redis Cluster redirection
How a Redis Cluster node answers a query it cannot handle: a
MOVEDerror naming the hash slot and the node serving it, for the client to send the query there; or, for a slot it is migrating away, anASKerror when the key is not on it, which sends only the next query to the other node.- Taught in
- Replication, failover and client routing
- Source
Redis Sentinel
A distributed system of processes that work together, which keeps checking Redis masters and replicas, can tell administrators or programs when something is wrong, and, when a master is not working as expected, can start a failover: a replica is promoted to master and the applications are told its address.
- Also called
- Sentinel
- Taught in
- Replication, failover and client routing
- Source
redo log
In MySQL's InnoDB storage engine, a structure on disk that crash recovery uses to correct the data that unfinished transactions wrote. In normal running it records requests to change table data. Amazon Aurora writes only redo log records to its storage.
Redshift
Amazon’s data warehouse, one of the newer ones that run only in the cloud.
- Also called
- Amazon Redshift
- First used in
- Step 4: Scale the design
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Cloud Data Warehouses
redundancy
Using several machines (or parts), so that when one fails another can take over. It works best when their faults are independent: one failing does not make another more likely to fail.
- Taught in
- Availability patterns and failover
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 1: Distributed Versus Single-Node Systems
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Tolerating hardware faults through redundancy
refresh-ahead
A cache that refreshes recently used entries on its own before they expire. If it predicts accurately which entries will be needed, it can respond faster than a read-through cache; predicting badly can make performance worse than not refreshing at all.
relational model
A data model in which data is organized into relations, called tables, each an unordered collection of rows. Probably the best-known data model today is based on it.
- Also called
- relational
- First used in
- Scaling from one user to millions
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Relational Versus Document Models
replicated log
A log of commands that every server keeps a copy of and applies in order; because each server's state machine is deterministic, each computes the same state. Keeping the copies the same is the job of the consensus algorithm.
replication
Keeping copies of the same data on several nodes, called replicas, so that the data stays reachable when a node does not.
- First used in
- Availability, consistency, CAP and PACELC
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
replication backlog
Memory a Redis master keeps for partial resynchronization: from it, a replica that reconnects can be sent just the part it missed. When there is not enough of it, the replica gets a full copy of the data instead.
- Taught in
- Replication, failover and client routing
- Source
replication factor
How many nodes store a copy of each piece of data.
- Taught in
- Tunable consistency and quorums
- Source
- Database Internals, 1st edition, chapter 11: Tunable Consistency
replication ID
In Redis, a large pseudo-random string that marks one history of a master’s data. Redis generates a new one when an instance restarts from scratch as a master, and when a replica is promoted to master. Together with a replication offset, it identifies an exact version of the master’s data.
- Taught in
- Replication, failover and client routing
- Source
replication lag
The delay between a write reaching the leader and the same write showing on a follower.
- First used in
- Single-leader and multi-leader replication
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Problems with Replication Lag
replication offset
In Redis, a count a master increases for every byte it produces to send to its replicas. Two instances with the same replication ID and offset have exactly the same data; for one replication ID, the offset tells which holds the most up-to-date data.
- Taught in
- Replication, failover and client routing
- Source
request
What a client sends to a server through the server's API, asking it for something. A web browser, for example, makes requests to web servers to download pages and images.
- First used in
- Performance versus scalability
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: Dataflow Through Services: REST and RPC
request coalescing
Letting a client have at most one request out at a time for a given object, to soften a thundering herd.
- Taught in
- Stampedes, penetration and avalanches
- Source
- Understanding Distributed Systems, 2nd edition, chapter 20: Local cache
request method
reread values
A countermeasure against anomalies in sagas: confirming that the data is unchanged before updating it.
resharding (changing the shard count)
Changing the number of shards when the number first set turns out wrong. With a fixed number of shards it is expensive: each shard is split and written to new files, and some systems cannot do it during writes.
- First used in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Fixed number of shards
resolver
A program that gets information from name servers in answer to clients' requests. The resolver a client's request goes to is typically run by its ISP, and it resolves the name iteratively for its clients. A recursive resolver is expected to keep the answers it gets in a cache.
- Also called
- recursive resolver, DNS resolver
- Taught in
- DNS and global traffic routing
- Source
- IETF RFC
- Understanding Distributed Systems, 2nd edition, chapter 4: Discovery
response time
How long the client waits from sending a request to getting the answer. It includes every delay anywhere in the system, and the service time too.
- Taught in
- Response time versus throughput
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Latency and Response Time
- Systems Performance, 2nd edition, chapter 2: 2.1 Terminology
REST
Representational State Transfer
A way of building a distributed system as a large collection of resources, each managed by a component, which remote applications can add, remove, retrieve and modify. It is widely used on the Web.
- First used in
- NoSQL data models and access patterns
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 2: Service-oriented architectures
retry storm
When a system is close to overload, clients that time out resend their requests, so the request rate rises even further and the overload grows worse.
- Taught in
- Circuit breakers, retries and bulkheads
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: When an Overloaded System Won't Recover
Retry-After
An HTTP header a server sends to say how long the client should wait before making another request.
- First used in
- Reverse proxies and edge security
- Source
retryable transaction
A step of a saga that follows the pivot. Retryable transactions are idempotent, and help make sure the saga can still reach its final state despite temporary failures.
- Also called
- retryable, retryable step
- Taught in
- Distributed transactions: two-phase commit and sagas
- Source
revalidation
A cache asking the server whether a stored response it cannot serve as it is, because it is no longer fresh, may still be used or must be replaced by a new one.
- Taught in
- Content delivery networks (CDN)
- Source
reverse proxy
A server in front of other servers: it takes clients' requests, forwards each to a server that can answer it, and returns that server's response. Clients cannot tell it from the server itself.
- Taught in
- Reverse proxies and edge security
- Source
- The System Design Primer
- Understanding Distributed Systems, 2nd edition, chapter 14: Reverse proxies
- IETF RFC
RFC
Request for Comments
A document in the series that describes Internet standards, protocols and other technical work on the Internet.
- First used in
- Horizontal sharding and consistent hashing
- Source
Riak
An open-source data store whose replication has no leader, a design inspired by Dynamo.
- First used in
- Availability, consistency, CAP and PACELC
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Leaderless Replication
robots.txt
The file in which a service makes available its rules for how crawlers may access URIs, under the Robots Exclusion Protocol. Crawlers are requested to honor the rules, which are not a form of access authorization. A crawler may cache the file, but should not use a cached copy for more than 24 hours unless the file is unreachable.
- Also called
- Robots Exclusion Protocol
- Taught in
- Requirements, estimates and a crawler's parts
- Source
RocksDB
A key-value store that can be embedded inside a program. It stores its data with the LSM-tree algorithm.
- First used in
- Storage engines: B-trees versus LSM-trees
- Source
- Understanding Distributed Systems, 2nd edition, chapter 20: Local cache
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Constructing and merging SSTables
root key
In envelope encryption, the top-level key that encrypts other keys and is itself kept in plaintext, so the keys and the data can be decrypted. AWS KMS calls the root keys it stores KMS keys, and they never leave its hardware security modules unencrypted.
- Also called
- KMS key, AWS KMS key
- Taught in
- Credential vaults and envelope encryption
- Source
root server
A DNS server at the top of the hierarchy. Asked about a name, it answers with the address of the name server for the name's top-level domain, such as .com. There are 13, each one made of many computers.
- Also called
- root name server
- Taught in
- DNS and global traffic routing
- Source
- Understanding Distributed Systems, 2nd edition, chapter 4: Discovery
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 1: Distributed versus decentralized systems
round robin
The simplest way to spread requests: each one goes to the next server on the list.
- Taught in
- Load balancing: layer 4 versus layer 7
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 3: Server clusters
round-trip
A message going from one endpoint to another and back. Network latency is sometimes measured as the time a round-trip takes.
- First used in
- Back-of-the-envelope estimation
- Source
- Systems Performance, 2nd edition, chapter 10: 10.1 Terminology
Route 53
The AWS DNS web service, built to be highly available and scalable.
- First used in
- Scaling from one user to millions
- Source
routing tier
A layer in front of a sharded database that works out which node should handle each request and forwards it there. It handles no requests itself, acting as a load balancer that knows the shards.
- Also called
- shard router
- Taught in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Request Routing
RPC
remote procedure call
A model of remote calls that tries to make a request to a service over the network look like an ordinary call to a function or method inside the same process.
- Taught in
- Transport protocols, gRPC and service meshes
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: The problems with remote procedure calls
RPO
recovery point objective
The longest acceptable time since the last data recovery point. It sets how much data loss is acceptable, between that point and an interruption of service; the organization decides it.
- Taught in
- Multiple regions and disaster recovery
- Source
RPS
requests per second
- First used in
- Response time versus throughput
RTO
recovery time objective
The longest acceptable delay between an interruption of service and its restoration. It sets how long service may acceptably be unavailable; the organization decides it.
- Taught in
- Multiple regions and disaster recovery
- Source
RTT
round-trip time
The time a network request takes to go from one endpoint to the other and back, including the time the signal takes to travel and the time spent processing it at each hop on the way.
- First used in
- DNS and global traffic routing
- Source
- Systems Performance, 2nd edition, chapter 10: 10.3.5 Latency
S
87 termsS3
Simple Storage Service
Amazon's object storage service.
- Also called
- Amazon S3
- First used in
- Scaling from one user to millions
- Source
S3 Glacier Instant Retrieval
An S3 storage class for long-term data that is rarely read and must be retrievable in milliseconds. Like the other Glacier classes, it has a minimum storage duration and retrieval fees.
- Also called
- Glacier Instant Retrieval
- Taught in
- Cost: compute, storage and network
- Source
S3 Standard-IA
S3 Standard-Infrequent Access
An S3 storage class for data that lives long and is read rarely. Objects are available in milliseconds, as in S3 Standard, but S3 charges a fee to retrieve them.
- Also called
- Standard-IA, IA
- Taught in
- Cost: compute, storage and network
- Source
SaaS
software as a service
Software provided as a service, so its users need not set up servers or the application themselves. Such products are often multitenant: each tenant is a customer, with data kept separate from the others.
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Sharding for Multitenancy
- Systems Performance, 2nd edition, chapter 11: 11.5 Other Types
saga
A distributed transaction made of local transactions, each with a compensating transaction that undoes it. Either all the local transactions succeed, or the compensating ones undo those that did.
- Taught in
- Horizontal sharding and consistent hashing
- Source
- Understanding Distributed Systems, 2nd edition, chapter 13: Sagas
saga choreography
Running a saga with no orchestrator: each service acts on the events the others publish. It suits a saga with only a few participants, and has no single point of failure.
- Also called
- choreography
- Taught in
- Distributed transactions: two-phase commit and sagas
- Source
saga orchestration
Running a saga from a central coordinator, the orchestrator, which knows every step and carries them out in the order they must follow. Because it coordinates the whole saga, it can become a single point of failure.
- Also called
- orchestration
- Taught in
- Distributed transactions: two-phase commit and sagas
- Source
saturation
How much work is queued at a resource because it cannot keep up. It begins once the resource is at 100% of its capacity, when extra work can no longer be processed and waits.
- First used in
- Performance versus scalability
- Source
- Systems Performance, 2nd edition, chapter 2: 2.1 Terminology
- Systems Performance, 2nd edition, chapter 2: 2.3.12 Saturation
Savings Plans
AWS's pricing model that gives lower rates than On-Demand in exchange for committing to use a set amount of compute, measured per hour, for one or three years. Usage is charged at the plan's rates until the commitment is used up, and the rest at On-Demand rates; an hour's commitment cannot be carried over.
scalability
Keeping performance the same as the load grows. If doubling the resources lets a system handle twice the load at the same performance, its scalability is linear.
- Taught in
- Performance versus scalability
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Summary
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Understanding Load
- All Things Distributed (the Dynamo paper)
scalability problem
A system that is fast when one person uses it, but slows down when the load is heavy.
- Taught in
- Performance versus scalability
- Source
scatter-gather
Sending a query to every shard and combining the results, needed when the records it wants could be on any shard because the query does not name their partition key.
- Taught in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Local Secondary Indexes
scheduled scaling
Auto Scaling for predictable changes in load: scheduled actions raise or lower a group's desired capacity at set times.
schema
A description of the structure of data: its fields and their types. A schema can change over time.
- First used in
- Federation and functional partitioning
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
secondary index
An extra data structure, kept alongside the main copy of the data, with which a database can efficiently find the records that meet a certain condition.
- Taught in
- NoSQL data models and access patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
semantic lock
A countermeasure against anomalies in sagas: application-level locks, used when a saga's compensable transaction signals that an update is in progress.
semaphore
A variable that limits how many operations may run in parallel. A counting semaphore allows a set number of them, a binary one only one at a time.
- Taught in
- Circuit breakers, retries and bulkheads
- Source
- Systems Performance, 2nd edition, chapter 5: 5.2.5 Concurrency and Parallelism
semisynchronous replication
Replication in which one follower is synchronous and the rest asynchronous. In MySQL, a commit on the leader waits until at least one follower acknowledges receiving the transaction; if none does before a timeout, the leader goes back to asynchronous replication.
- Also called
- semisynchronous
- Taught in
- Single-leader and multi-leader replication
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Synchronous Versus Asynchronous Replication
- MySQL documentation
sequential write
Writing to disk in a few large writes, as an LSM-tree does, rather than in many small, scattered ones, called random writes, as a B-tree does. Disks generally write faster sequentially than at random.
- Taught in
- Storage engines: B-trees versus LSM-trees
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Sequential versus random writes
serializability
An isolation guarantee: transactions that run at the same time behave as they would had they run one at a time, in some order.
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
serialization
Translating data from its form in memory into a sequence of bytes, also called encoding. The reverse is deserialization, or decoding.
- Taught in
- Transport protocols, gRPC and service meshes
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: Formats for Encoding Data
server
service
service discovery
Finding out the address of the service a client needs to connect to. Systems such as ZooKeeper, etcd and Consul are often used for it.
- Taught in
- Transport protocols, gRPC and service meshes
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: Load balancers, service discovery, and service meshes
- Designing Data-Intensive Applications, 2nd edition, chapter 10: Service discovery
service mesh
A form of load balancing that combines software load balancers with service discovery. Instead of running on a machine of its own, its load balancer usually runs beside each client and each server, as a library inside the program or as a separate sidecar process.
- Taught in
- Transport protocols, gRPC and service meshes
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: Load balancers, service discovery, and service meshes
service rate
The rate at which a server completes requests, written μ. The average service time is one divided by the service rate.
- Taught in
- Response time versus throughput
- Source
service time
How long a service is actually working on a request. Time the request spends waiting in a queue is not counted.
- Taught in
- Response time versus throughput
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Latency and Response Time
session state
Information about a series of operations by one user, which has to be kept for some time, though not forever.
- First used in
- Performance versus scalability
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 3: General design issues
SHA-1
Secure Hash Algorithm 1
A cryptographic hash function. Two documents that differ by a single byte, for example, get completely different SHA-1 hashes.
- Also called
- SHA
- First used in
- Duplicate elimination and SimHash
- Source
SHA-256
Secure Hash Algorithm 256
A hash algorithm that can produce digests of messages, used to detect whether a message has changed since its digest was made.
- First used in
- Credential vaults and envelope encryption
- Source
shortest path
shuffle
A distributed sorting algorithm, whose input and output are both sharded, that batch processors need for sorting datasets too large to fit on one machine.
- First used in
- Stream processing versus batch recomputation
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 11: Shuffling Data
sidecar
A proxy running as its own process on the same machine as a client, so that all of the client's network traffic passes through it.
- Also called
- sidecar proxy
- Taught in
- Transport protocols, gRPC and service meshes
- Source
- Understanding Distributed Systems, 2nd edition, chapter 18: Application layer load balancing
SimHash
Charikar’s technique for reducing a web page’s weighted features to a small fingerprint, with the property that the fingerprints of near-duplicates differ in only a few bit positions. The paper “Detecting Near-Duplicates for Web Crawling” found 64-bit fingerprints with k = 3 reasonable for 8 billion web pages, k being the most bit positions a match may differ in.
- Taught in
- Duplicate elimination and SimHash
- Source
simple routing
A Route 53 routing policy for one resource that does a given job for a domain, such as a web server that serves a website.
- Taught in
- Multiple regions and disaster recovery
- Source
simple scaling
An Auto Scaling policy that raises or lowers a group's capacity by a single adjustment, then waits out a cooldown before the next. AWS recommends against it.
single point of failure
A part of a system whose fault brings the whole system down, because the system cannot tolerate that part failing.
- Also called
- SPOF
- Taught in
- Performance versus scalability
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Fault Tolerance
single-leader replication
Replication in which one replica, the leader, takes the writes and the others follow it. One major downside: every write goes through the one leader.
- Also called
- leader-based replication, single-leader, master-slave, master-slave replication, master-replica, master-replica replication
- Taught in
- Availability patterns and failover
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Single-Leader Replication
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Multi-Leader Replication
skew
Load spread unevenly across shards: some get many requests or much data, and others far less. It makes sharding much less effective. Skew is also called hot spots.
- Taught in
- Partition keys, skew and polyglot persistence
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Sharding of Key-Value Data
skip list
An in-memory data structure that keeps keys in order: they can be inserted in any order, looked up efficiently and read back sorted. A memtable can be one.
- First used in
- Storage engines: B-trees versus LSM-trees
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Constructing and merging SSTables
SLA
service level agreement
A contract saying what happens if a service misses its targets for performance and availability: for example, customers may be entitled to a refund.
- First used in
- Availability in numbers
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Use of Response Time Metrics
slab allocation
How Memcached lays out the memory it keeps for items: it breaks it into pages, assigns each page to a slab class and cuts it into chunks of that class’s size, and puts each item in the class whose chunks fit it most nearly. An item smaller than its chunk loses the rest of the chunk as overhead.
- Also called
- slab class
- Taught in
- Eviction algorithms and memory management
- Source
slab reassignment
Moving pages of memory from one slab class to another in Memcached, added in release 1.4.11 for a problem long-running instances can meet: all the memory has gone to one class, and the application starts storing items of another size. It can be started by hand; that release also began an automatic algorithm for it.
- Also called
- slab rebalancing
- Taught in
- Memory: slab allocation, calcification and jemalloc
- Source
sloppy quorum
Letting any reachable replica accept a write, even one that is not among the usual replicas for its key. A later read is not certain to see the write, but that can be better than having the write fail. Cassandra's name for it is the consistency level
ANY.- Taught in
- Tunable consistency and quorums
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Single-Leader Versus Leaderless Replication Performance
Snowflake
X’s ID generator. It puts a timestamp in an ID’s most significant bits and fills the rest so that the ID is unique even when the timestamp is not.
- Also called
- Twitter Snowflake
- Taught in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 10: ID Generators and Logical Clocks
sort key
In DynamoDB, the second attribute of a composite primary key, after the partition key: the items that share a partition key are stored together, in order by it.
- First used in
- NoSQL data models and access patterns
- Source
sorted set
A Redis collection of unique strings, its members, ordered by a score that goes with each one. Members with the same score are ordered lexicographically, as in a dictionary.
- First used in
- Requirements, estimates and a crawler's parts
- Source
SP
NIST Special Publication
- First used in
- Credential vaults and envelope encryption
- Source
Spanner
A distributed database Google built to run at global scale while staying highly available.
- First used in
- Availability, consistency, CAP and PACELC
- Source
speculative decoding
Speeding up decode with a small draft model that proposes the next tokens, which the large target model verifies instead of generating them one by one.
- Taught in
- Reusing work and starting fast
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 7: Speculative Decoding
split brain
When two nodes both believe they are the leader at the same time. If both accept writes and nothing resolves the conflicts, data can be lost or corrupted.
- Taught in
- Single-leader and multi-leader replication
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Leader failure: Failover
split vote
In Raft, an election in which several followers have become candidates and none gets votes from a majority. Raft uses random timers to make split votes less likely in the elections that follow.
- Taught in
- Consensus and leader election: Raft and Paxos
- Source
- Database Internals, 1st edition, chapter 14: Failure Scenarios
spot instance
A cheaper, low-priority EC2 instance that uses spare capacity, for less than the On-Demand price. EC2 can stop or terminate it, with a warning two minutes before.
- Taught in
- Cost: compute, storage and network
- Source
- AWS documentation
- AWS documentation
- Designing Data-Intensive Applications, 2nd edition, chapter 11: Handling faults
SQL
Structured Query Language
The language of what is probably the best-known way to store and query data today, the relational model, in which data is organized in tables of rows.
- First used in
- Federation and functional partitioning
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Relational Versus Document Models
SQL injection
An attack on websites that pass input from users on to their databases without processing it enough: the input is written so that one SQL statement becomes several.
- Also called
- SQLi
- First used in
- Reverse proxies and edge security
- Source
- NIST glossary and publications
- Release It!, 2nd edition, chapter 11: Injection
SQS
Amazon's hosted message queue, for connecting the parts of a distributed system without tying them directly to each other.
- First used in
- Scaling from one user to millions
- Source
SRE
site reliability engineering
Google's approach to running services: software engineers are hired to run its products, and build systems to do work that would otherwise often be done by hand.
- First used in
- Availability in numbers
- Source
SSD
solid-state drive
A storage device usually built on flash memory.
- First used in
- Back-of-the-envelope estimation
- Source
- Systems Performance, 2nd edition, glossary
SSTable
Sorted String Table
A file of keys and their values, sorted by key, with each key appearing only once. LSM-trees often keep their data on disk in SSTables, which are never changed once written.
- Taught in
- Storage engines: B-trees versus LSM-trees
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: The SSTable file format
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Constructing and merging SSTables
- Database Internals, 1st edition, chapter 7: Sorted String Tables
stable storage
In a model of a system, the nonvolatile disk storage each node is assumed to have, which is preserved across crashes while what the node held in memory is lost.
- First used in
- Fetching, DNS caching and fault tolerance
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 9: System Model and Reality
stale set
In Facebook's memcache, a set of a value into the cache that is no longer the latest one that should be cached. It can happen when concurrent updates reach the cache out of order.
- Taught in
- Cache invalidation and update strategies
- Source
standby (Amazon RDS)
In Amazon RDS, a copy of the database kept up to date in another availability zone, which the deployment fails over to. It cannot serve reads.
- First used in
- Scaling from one user to millions
- Also means
- standby (failover)standby (DNS failover)standby (PostgreSQL)
- Source
standby (DNS failover)
In DNS failover routing, a secondary resource that DNS answers with only when all the primary resources have become unavailable.
standby (failover)
In active-passive failover, the passive server that waits to take over. A hot standby is already running; a cold one has to start up first. Which it is decides how long the outage lasts.
standby (PostgreSQL)
In PostgreSQL, a server that tracks the changes made on the primary. A hot standby also serves read-only queries; a warm standby accepts no connections until it is promoted to primary.
state machine
A model of a process as something that changes from state to state as each input, such as an operation, arrives.
- Taught in
- Consensus and leader election: Raft and Paxos
- Source
- Understanding Distributed Systems, 2nd edition, chapter 10: Replication
state machine replication
Keeping replicas alike by having each one apply the same writes in the same order with deterministic logic, so that they all end up in a consistent state.
- Taught in
- Consensus and leader election: Raft and Paxos
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 10: Using shared logs
stateless
Describes a server that remembers nothing about its clients' state, so it is free to change its own without informing any of them. A web server is one.
- Taught in
- Performance versus scalability
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 3: General design issues
static batching
Batching in which the server waits until enough requests have arrived to fill a set batch size before it processes them, so every request waits for the batch to be full.
- Taught in
- Batching and the KV cache
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 6: Dynamic Batching in Online Inference
- AI Engineering, 1st edition, chapter 9: Batching
static stability
AWS's name for a system going on working without depending on its control plane for configuration or for recovery. For example, rather than have Auto Scaling replace the instances lost with a zone, which depends on the control plane to launch them, instances are provisioned in advance in the remaining zones to absorb the load.
step scaling
An Auto Scaling policy that raises or lowers a group's capacity by step adjustments, which vary with how far an alarm is breached.
storage class
In Amazon S3, the class an object is stored in, chosen for its use and how it must be accessed. Objects are in S3 Standard unless another class is chosen.
- Taught in
- Cost: compute, storage and network
- Source
storage engine
The part of a database that carries out queries on its own node. It is built on a data structure, such as a B-tree, and adds caching, recovery and transactions on top.
- First used in
- Storage engines: B-trees versus LSM-trees
- Source
- Database Internals, 1st edition, chapter 1: DBMS Architecture
- Database Internals, 1st edition, chapter 1: Buffering, Immutability, and Ordering
stream (HTTP/2)
In HTTP/2, an independent flow in both directions between client and server inside one connection. A connection can hold many open streams at once, and each request and its response get a stream of their own.
- Taught in
- Transport protocols, gRPC and service meshes
- Also means
- event stream
- Source
stream processing
A computation that runs continually, taking an endless stream of events as its input and deriving output from them.
- First used in
- Redirects, caching and visit analytics
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
streaming RPC
In gRPC, a call in which the server, the client or both send a sequence of messages: server streaming, client streaming and bidirectional streaming. A unary call is a single request and a single response, like a normal function call.
- Also called
- server streaming, client streaming, bidirectional streaming
- Taught in
- Transport protocols, gRPC and service meshes
- Source
strict mode
In Istio, the setting under which every connection must use mutual TLS. It is useful for making sure that no connection goes around the mesh.
subjectively down
In Redis Sentinel, the condition of an instance that one Sentinel has had no valid reply from for a set number of seconds. It is local to that Sentinel and is not enough to start a failover.
- Taught in
- Replication, failover and client routing
- Source
subnet
A range of IP addresses in a VPC. Resources in a public subnet can reach the public internet; resources in a private subnet need a NAT device to do so.
- Also called
- public subnet, private subnet
- Taught in
- From one user to millions on AWS
- Source
switchover
In an Aurora global database, a planned move of the writes to another Region, for maintenance and other planned work while everything is healthy. The other clusters are brought up to date first, so no data is lost.
- Taught in
- Multiple regions and disaster recovery
- Source
synchronous
Waiting for something to finish before going on: the opposite of asynchronous.
- First used in
- Performance versus scalability
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
synchronous replication
Replication in which the leader waits until a follower confirms it has a write before reporting success. That follower then certainly has the latest data.
- First used in
- Availability zones, failover and auto scaling
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Synchronous Versus Asynchronous Replication
system call
The interface through which a process asks the kernel to carry out a privileged action, such as device I/O.
- Also called
- syscall
- First used in
- Requirements, estimates and the query cache
- Source
- Systems Performance, 2nd edition, glossary
- Systems Performance, 2nd edition, chapter 3: 3.1 Terminology
system of record
The system that holds the authoritative version of some data. Changes are written there first, and other datasets may be derived from it.
- Also called
- source of truth
- First used in
- Partition keys, skew and polyglot persistence
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
T
33 termstag (Protocol Buffers)
In Protocol Buffers, the number written before each encoded field: a varint made from the field's number and its wire type. The wire type tells the reader how large the value after it is.
tail latency
The slow end of the response time distribution, measured by high percentiles such as p99. It matters because these are delays that real users experience.
- Taught in
- Response time versus throughput
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Average, Median, and Percentiles
tail latency amplification
When an end-user request needs several backend calls, the chance that one of them is slow goes up. So even when only a small share of backend calls are slow, a higher share of end-user requests end up slow.
- Taught in
- Response time versus throughput
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Use of Response Time Metrics
tamper-evident log
A log whose changes are easy to detect. In the paper “Efficient Data Structures for Tamper-Evident Logging”, the logger alone writes and signs the log and is not trusted: auditors challenge it to prove it behaves correctly, against the signed commitments it publishes as events are added. A log nobody examines can be tampered with undetected
- Also called
- tamper-evident
- Taught in
- Credential vaults and envelope encryption
- Source
TAO (Facebook’s graph store)
Facebook’s data store for its social graph: a graph store in data centers across several regions, built for reads and eventually consistent. It keeps the data in MySQL, divided into shards, and handles all of the clients’ communication with the database through a cache of its own that understands the graph.
- Also called
- TAO
- Taught in
- Requirements, estimates and the graph model
- Source
target group
A group of registered targets that an Application Load Balancer routes requests to. Each listener rule names a target group, and requests that meet the rule's conditions are sent to it.
- First used in
- Availability zones, failover and auto scaling
- Source
target tracking scaling
An Auto Scaling policy that raises or lowers a group's capacity from a CloudWatch metric and a target value, the way a thermostat keeps a home at the temperature you choose. AWS recommends it.
- Also called
- target tracking
- Taught in
- Availability zones, failover and auto scaling
- Source
task queue
A queue that receives tasks with their data, runs them and delivers their results. It can schedule tasks, and run heavy jobs in the background.
TB
terabyte
- First used in
- Back-of-the-envelope estimation
TCP
Transmission Control Protocol
A protocol between hosts for reliable communication across connected networks. It gives applications a stream of bytes, delivered reliably and in order.
- First used in
- DNS and global traffic routing
- Source
tenant
In a service shared by many customers, one customer. Several users may log in to the same tenant, and each tenant's data is kept separate from the others'.
- First used in
- Partition keys, skew and polyglot persistence
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 7: Sharding for Multitenancy
tensor parallelism
Splitting each layer of a model across several GPUs: each holds a slice of the layer’s weights and does part of its computation, and the GPUs must communicate at every layer.
- Taught in
- Batching and the KV cache
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 7: Tensor Parallelism and Pipeline Parallelism
term (Raft)
In Raft, one of the numbered periods time is divided into. Each begins with an election, and the candidate that wins leads for the rest of it. Terms act as a logical clock that lets servers spot stale leaders.
thread
One running line of a program, which the operating system can schedule to run on a CPU. A process holds one or more threads.
- First used in
- Response time versus throughput
- Source
- Systems Performance, 2nd edition, glossary
thread pool
A set of threads a server keeps ready to handle work, instead of creating a thread each time one is needed.
- First used in
- Performance versus scalability
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 3: Object servers
Thrift
Similar to Protocol Buffers, and first developed at Facebook.
- Also called
- Apache Thrift
- Taught in
- Transport protocols, gRPC and service meshes
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: Protocol Buffers
throughput
How much work a system gets through per unit of time, counted as requests per second or as data volume per second. Any fixed set of hardware has an upper limit.
- Taught in
- Response time versus throughput
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Describing Performance
thundering herd
A sudden spike of requests at a cache's origin because the cache cannot answer them: caches start empty, an object suddenly becomes popular, or writes keep invalidating a heavily read key. Memcached's documentation calls it a stampeding herd.
- Also called
- stampeding herd
- Taught in
- Stampedes, penetration and avalanches
- Source
- Understanding Distributed Systems, 2nd edition, chapter 20: Local cache
- USENIX
- Memcached documentation
timeout
A limit on how long to wait for an answer: once it passes, you stop waiting, judging that the answer will not come.
- First used in
- Single-leader and multi-leader replication
- Source
- Release It!, 2nd edition, chapter 5: Timeouts
TLD
top-level domain
The top of a domain name, such as .com in example.com. Its name server answers with the address of the authoritative name server of each domain under it.
- First used in
- DNS and global traffic routing
- Source
- Understanding Distributed Systems, 2nd edition, chapter 4: Discovery
TLS
Transport Layer Security
A protocol that lets clients and servers talk over the Internet in a way designed to stop others listening in, changing their messages or sending false ones. It runs on top of TCP, and HTTP uses it to communicate securely.
- First used in
- Content delivery networks (CDN)
- Source
- IETF RFC
- Understanding Distributed Systems, 2nd edition, chapter 3: Secure links
TLS termination
Decrypting requests and encrypting responses at a load balancer or reverse proxy, so that the servers behind it do not have to do this potentially expensive work.
- Taught in
- Reverse proxies and edge security
- Source
- The System Design Primer
- Understanding Distributed Systems, 2nd edition, chapter 18: Application layer load balancing
token (of text)
The basic unit a language model works in: a character, a word or part of a word, depending on the model. The set of tokens a model can work with is its vocabulary.
- Taught in
- How a model answers a request
- Source
- AI Engineering, 1st edition, chapter 1: Language models
token bucket
A rate-limiting algorithm in which each request uses up a token. Tokens are added to the bucket at a set rate, up to its capacity: the rate is the steady rate it allows, and the capacity the largest burst.
- Also called
- token bucket algorithm
- Taught in
- Reverse proxies and edge security
- Source
tokenizer
The algorithm that breaks text into a model’s tokens. Its method and the size of the vocabulary are chosen by the model’s developers.
- Taught in
- How a model answers a request
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 2: Summary
- AI Engineering, 1st edition, chapter 1: Language models
topic
A group of messages that many producers can send to and many consumers read from. In a log-based message broker, a topic is a group of shards, which Kafka calls partitions, all carrying messages of the same type.
- Taught in
- Queues, task queues and log-based message brokers
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Messaging Systems
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Using logs for message storage
TPOT
time per output token
The time a model takes to generate each token of its answer after the first. It corresponds to the decode phase, where the model generates tokens one after another.
- Taught in
- How a model answers a request
- Source
- AI Engineering, 1st edition, chapter 9: Latency, TTFT, and TPOT
- Hands-On LLM Serving and Optimization, 1st edition, chapter 4: Latency Metrics
transactional outbox
Writing a message to an outbox table in the same local transaction as the change it reports, so it is stored if and only if the transaction commits. A relay process then forwards the messages, through a channel such as Kafka; it may send one more than once, so each carries an idempotency key.
- Also called
- outbox pattern, transactional outbox pattern, outbox
- Taught in
- Event sourcing, CQRS and the transactional outbox
- Source
- Understanding Distributed Systems, 2nd edition, chapter 13: Outbox pattern
TTFT
time to first token
How soon the first token of the answer comes after a query is sent. On the model, it corresponds to the prefill phase.
- Taught in
- How a model answers a request
- Source
- AI Engineering, 1st edition, chapter 9: Latency, TTFT, and TPOT
- Hands-On LLM Serving and Optimization, 1st edition, chapter 4: Latency Metrics
TTL
time to live
How long a cache may keep a copy before it counts as expired. Every DNS record carries one, the most time a cache may keep it.
- Taught in
- DNS and global traffic routing
- Source
- Understanding Distributed Systems, 2nd edition, chapter 4: Discovery
- IETF RFC
- The System Design Primer
tumbling window
A window of fixed length that does not overlap the next one: every event belongs to exactly one.
- Taught in
- Stream processing versus batch recomputation
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Types of windows
- Apache Flink documentation (release 1.20)
Twemproxy
two-phase commit
An algorithm that makes several database nodes all commit a transaction or all abort it. A coordinator first asks each node whether it can commit, then sends them all the decision; if the coordinator fails, the nodes can be stuck waiting for it to recover.
- Also called
- 2PC
- Taught in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, glossary
- Designing Data-Intensive Applications, 2nd edition, chapter 8: Two-Phase Commit
- Designing Data-Intensive Applications, 2nd edition, chapter 8: A system of promises
- Designing Data-Intensive Applications, 2nd edition, chapter 8: Three-phase commit
U
15 termsUDP
unicast
An address that names a single network interface: a packet sent to it goes to that one place.
- First used in
- DNS and global traffic routing
- Source
unique constraint
A constraint that no two rows of a table hold the same data in a given column, or group of columns. In PostgreSQL, adding one creates an index on those columns.
- First used in
- Requirements, estimates and the data model
- Source
URI
Uniform Resource Identifier
A short string of characters that identifies a resource, abstract or physical. A URL is a URI that also says how to locate the resource.
- Taught in
- Redirects, caching and visit analytics
- Source
URL
Uniform Resource Locator
An address that identifies a resource and says how to reach it.
- First used in
- Content delivery networks (CDN)
- Source
URL frontier
The data structure in which a web crawler keeps the URLs still to be downloaded. The crawler adds URLs to it, and the frontier decides the order in which it hands them out, so it controls the crawl’s download schedule. Mercator’s also keeps each URL’s download history.
URL normalization
Rewriting a URI before comparing it with others. With the generic syntax, the scheme and host are case-insensitive, so they should be lower-cased; and as http has a default port of 80 and treats an empty path as “/”, several spellings of one http URI are equivalent. Spiders and indexing engines often normalize URIs to reduce repeated requests.
- Also called
- URI normalization
- Taught in
- Duplicate elimination and SimHash
- Source
US
United States
- First used in
- DNS and global traffic routing
USD
United States dollar
- First used in
- Bank extraction and idempotent ingestion
user timeline
The timeline of one user’s own activity: all its tweets come from that user, not from the people the user follows, whose activity makes up the home timeline.
UTC
Coordinated Universal Time
UTF-8
A way of encoding text in which each character takes from one to four bytes, the number depending on the character.
- Also called
- UTF
- Taught in
- Transport protocols, gRPC and service meshes
- Source
utilization
How busy a server is: the arrival rate divided by the service rate, written ρ. It is the share of the time the server is busy, so at 0.9 it is working 90% of the time. The word is also used for the share of a component's capacity in use.
- Taught in
- Response time versus throughput
- Source
- MIT OpenCourseWare
- Systems Performance, 2nd edition, chapter 2: 2.3.11 Utilization
UUID
universally unique identifier
A 128-bit identifier that any node can make on its own, without asking a central registry. In version 4 the bits are random, so comparing two of them says nothing about which is newer.
- First used in
- Horizontal sharding and consistent hashing
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 10: ID Generators and Logical Clocks
- IETF RFC
UUIDv7
A version of UUID that begins with a timestamp, the number of milliseconds since 1 January 1970, and fills the rest so that each ID is unique.
- Also called
- version 7 UUID
- Taught in
- Horizontal sharding and consistent hashing
- Source
- IETF RFC
- Designing Data-Intensive Applications, 2nd edition, chapter 10: ID Generators and Logical Clocks
V
11 termsvarint
In Protocol Buffers, an integer written in a variable number of bytes, fewer for small values. Each byte has a continuation bit saying whether the next byte belongs to the same integer.
- Also called
- variable-width integer
- Taught in
- Transport protocols, gRPC and service meshes
- Source
vCPU
A virtual CPU. A modern processor can offer several of them on each of its cores.
- Taught in
- Transport protocols, gRPC and service meshes
- Source
- Systems Performance, 2nd edition, glossary
vector clock
A structure that records a partial order of events across processes, so that a system can tell whether one event happened before another, or whether two versions have diverged.
- First used in
- Single-leader and multi-leader replication
- Source
- Database Internals, 1st edition, chapter 11: Vector clocks
- Distributed Systems (van Steen and Tanenbaum), 4th edition, glossary
vertex
One of the two kinds of object in a graph: a thing such as a person, a web page or a road junction. Vertices are also called entities, and edges join them.
- Taught in
- NoSQL data models and access patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Graph-Like Data Models
vertical scaling
Moving a service to a more powerful machine, with more CPU cores, memory and disk space.
- Also called
- scaling up
- Taught in
- Performance versus scalability
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 2: Shared-Memory, Shared-Disk, and Shared-Nothing Architectures
virtual node
A unit that looks like a single node to the system. One physical node can be responsible for several, the number set by its capacity, and when a node becomes unavailable its load spreads evenly over the remaining nodes.
- Also called
- vnode
- Taught in
- Horizontal sharding and consistent hashing
- Source
visibility map
A record PostgreSQL keeps for each table of which pages hold only rows that every active transaction can see.
- First used in
- Denormalization and SQL tuning
- Source
visibility timeout
In Amazon SQS, how long a message that a consumer has received stays in the queue but hidden from other consumers, so that they cannot process it at the same time. If it is not deleted before the timeout expires, it becomes visible again and another consumer can take it. The default is 30 seconds.
- Taught in
- Cost: compute, storage and network
- Source
VLDB
Very Large Data Bases
- First used in
- Stampedes, penetration and avalanches
VoIP
voice over Internet Protocol
- First used in
- Consistency patterns
- Source
VPC
virtual private cloud
In AWS, a logically isolated virtual network you define yourself, in which you launch your AWS resources.
- Also called
- Amazon VPC
- Taught in
- From one user to millions on AWS
- Source
W
27 termsWAF
web application firewall
A firewall at the application layer, layer 7: it can detect and block suspicious requests, and be loaded with the signatures of well-known attacks.
- Taught in
- Reverse proxies and edge security
- Source
- Release It!, 2nd edition, chapter 11: Insufficient Attack Protection
WAL
write-ahead log
An append-only file to which a database writes every change before applying it to its data files, so that it can recover from a crash.
- Taught in
- Single-leader and multi-leader replication
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Making B-trees reliable
- PostgreSQL documentation
WARC
Web ARChive
A file format that joins many records, each a set of simple text headers and a block of data, into one long file. It extends an older format that has been used to store web crawls.
warm standby
A disaster recovery strategy that keeps a scaled-down but fully working copy of production in another Region. It can take traffic at once, at reduced capacity, and only has to scale up.
- Taught in
- Multiple regions and disaster recovery
- Source
watermark
In stream processing, a declaration that, at that point in the stream, every event with a timestamp up to a certain time should have arrived: event time has reached that time. Flink uses watermarks to keep track of how far event time has progressed.
weak consistency
A consistency model with weak guarantees, of which eventual consistency is one. The System Design Primer uses the name for a pattern in which, after a write, a read may or may not see it, on a best-effort basis.
- Taught in
- Consistency patterns
- Source
- Distributed Systems (van Steen and Tanenbaum), 4th edition, chapter 7: Client-centric consistency models
- The System Design Primer
weak politeness guarantee
web crawler
An automated client. Search engines, for example, have crawlers that follow links recursively, for indexing.
- Also called
- crawler
- Taught in
- Requirements, estimates and a crawler's parts
- Source
webhook
An HTTP request that serves as a push notification. Plaid, for example, sends one to an endpoint on the app’s server when new data is available, keeps sending it for up to 24 hours when the endpoint does not answer successfully, and asks apps to handle duplicate webhooks and to make what they do on receiving one idempotent.
- Taught in
- Bank extraction and idempotent ingestion
- Source
WebSocket
A protocol for two-way communication between a client and a remote server, so either side can send a message when it has one.
- Taught in
- Connections that stay open
- Source
weighted round robin
Round robin in which each server can be assigned a weight, a whole number that stands for its processing capacity, so servers of different capacities are better handled: higher weights get more connections. Some DNS services can route traffic by weighted round robin, to keep it from servers under maintenance, balance clusters of different sizes and run A/B tests.
weighted routing
A Route 53 routing policy that sends traffic to several resources in proportions you set.
- Taught in
- Multiple regions and disaster recovery
- Source
weights (of a model)
The parameters a model holds. The memory they take follows from their count and their data type, its precision; once loaded, they stay in GPU memory to serve requests.
- Taught in
- How a model answers a request
- Source
- Hands-On LLM Serving and Optimization, 1st edition, chapter 5: Estimating Model Size
- Hands-On LLM Serving and Optimization, 1st edition, chapter 5: The Model Loading Process
wide column
A data model that lets each row hold thousands of columns, with no need for every row to have the same ones. Despite the name, it stores all the values of a row together. Bigtable popularized it.
- Also called
- wide column store, wide-column data model, column-family
- Taught in
- NoSQL data models and access patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Column compression
- Designing Data-Intensive Applications, 2nd edition, chapter 3: Data locality for reads and writes
window (stream processing)
In stream processing, a period of time whose events can be taken together, for example to count them or to find the average of their values.
- Taught in
- Stream processing versus batch recomputation
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 12: Types of windows
- Apache Flink documentation (release 1.20)
workflow
A sequence of steps, each called a task, usually defined as a graph of tasks.
- Taught in
- Distributed transactions: two-phase commit and sagas
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: Durable Execution and Workflows
working set
The data a program uses often in its work. Performance should improve greatly if it fits in the CPU caches rather than main memory.
- First used in
- Federation and functional partitioning
- Source
- Systems Performance, 2nd edition, chapter 7: 7.2.10 Working Set Size
- Systems Performance, 2nd edition, chapter 8: 8.5.8 Micro-Benchmarking
workload
The input to a system: the load applied to it. For a database, it is the queries and commands that clients send.
- First used in
- Scaling from one user to millions
- Source
- Systems Performance, 2nd edition, chapter 2: 2.1 Terminology
write amplification
How many times over a storage engine writes its data: the total bytes written to disk, divided by the bytes an append-only log with no index would have needed. Both B-trees and LSM-trees have it.
- Taught in
- Storage engines: B-trees versus LSM-trees
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 4: Write amplification
write conflict
Writes made at the same time on different leaders that clash and have to be resolved, as can happen with multi-leader replication.
- Also called
- conflict
- First used in
- Availability patterns and failover
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 6: Dealing with Conflicting Writes
write forwarding
In an Aurora global database, a secondary cluster passing on the SQL statements that write to the cluster in the Region that takes writes, which runs them and changes the data first; Aurora then replicates the changes to all the secondary clusters in the other Regions.
write global
A strategy for a workload in several Regions that sends all writes to one Region; if that Region fails, another is promoted to take writes.
- Taught in
- Multiple regions and disaster recovery
- Source
write local
A strategy for a workload in several Regions that sends each write, like each read, to the closest Region. DynamoDB global tables allow it, and settle concurrent updates by letting the last writer win.
- Taught in
- Multiple regions and disaster recovery
- Source
write partitioned
A strategy for a workload in several Regions that assigns each write to one Region by a partition key, such as the user ID, to avoid write conflicts.
- Taught in
- Multiple regions and disaster recovery
- Source
write sharding
Spreading the writes for one partition key over many by adding a suffix to its values. A random suffix spreads them evenly, but reading all the items needs one query per suffix; a suffix calculated from something you query by makes single items easier to read.
- Also called
- salting
- Taught in
- Partition keys, skew and polyglot persistence
- Source
write-behind
A cache that reports a write as done once the data is in the cache, and writes it to the slower storage behind it, such as the database, later and asynchronously. Writes are faster, but data can be lost if the cache goes down before its contents reach the database.
- Also called
- write-back, write-back cache, write-behind cache
- Taught in
- Cache invalidation and update strategies
- Source
- The System Design Primer
- Systems Performance, 2nd edition, chapter 9: 9.2.2 Caching Disk
write-through
A cache that completes a write only once the data has also been written to the next level of storage.
- Also called
- write-through cache
- Taught in
- Cache invalidation and update strategies
- Source
- Systems Performance, 2nd edition, chapter 9: 9.2.2 Caching Disk
X
2 termsXML
Extensible Markup Language
A widely supported text format for encoding data, which many programming languages can write and read. It is often criticized as verbose and needlessly complicated.
- First used in
- NoSQL data models and access patterns
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 5: JSON, XML, and Binary Variants
- W3C
XSS
cross-site scripting
A flaw that lets attackers put code of their own into an otherwise harmless website, which then runs in other users' browsers.
- First used in
- Reverse proxies and edge security
- Source
Z
2 termsZipf’s law
The pattern in which the probability of words or other items occurring starts high and tapers off: a few occur very often, and many others rarely.
ZooKeeper
A coordination service, one of the main users of consensus. On the surface it looks like a key-value store, but it is not meant for heavy write loads or for storing data in general, and it is often used for service discovery.
- Also called
- Apache ZooKeeper
- Taught in
- Transport protocols, gRPC and service meshes
- Source
- Designing Data-Intensive Applications, 2nd edition, chapter 10: Coordination Services
- Designing Data-Intensive Applications, 2nd edition, chapter 10: Service discovery