A working technical dictionary

An experiment that compares two versions by randomly routing users to each and measuring which performs better on a chosen metric. It is used to validate model or product changes in production.

An experiment that removes or alters individual components of a system one at a time to measure their contribution to overall performance. It is a standard way to understand which design choices matter.

The fraction of all predictions a model gets right; simple but misleading when the classes are heavily imbalanced.

A function applied at each neuron that adds nonlinearity, letting networks learn complex patterns rather than just linear relationships.

An approach in which the model iteratively selects the most informative unlabeled examples for a human to label, aiming to reach high accuracy with as few labels as possible. It is useful when labeling is costly.

A reinforcement learning architecture that combines a policy (the actor) that chooses actions with a value estimator (the critic) that evaluates them. The critic's feedback stabilizes and speeds up the actor's learning.

An early boosting algorithm that combines many weak learners into a strong one by repeatedly reweighting training examples so that misclassified points get more attention in later rounds.

An optimizer that adapts the learning rate for each parameter based on the accumulated sum of its past squared gradients, giving smaller updates to frequently updated parameters. It suits sparse data but can slow down over time.

A popular optimization algorithm that adapts the learning rate for each parameter using running estimates of the first and second moments of the gradients. It combines the benefits of momentum and per-parameter scaling.

A deliberate attempt to fool or manipulate a machine learning model by crafting malicious inputs, poisoning training data, or exploiting its weaknesses. Defending against such attacks is a key security concern.

An input crafted with small, often imperceptible changes designed to make a model produce a wrong output with high confidence. Such examples reveal the brittleness of many neural networks.

An AI system that pursues a goal over multiple steps, deciding actions, calling tools, and observing results in a loop rather than producing a single answer.

The mechanism by which an AI agent stores and recalls information across steps or sessions, such as past actions, results, or user facts. It lets agents maintain context beyond a single model call.

A process in which an AI system autonomously plans and carries out a multi-step task, deciding which tools to call and when, rather than answering in a single turn. It underpins modern AI agents that take actions in the world.

The study of the moral principles and societal impacts of building and deploying AI, including fairness, accountability, transparency, and privacy. It guides responsible development and use.

The field concerned with ensuring AI systems behave reliably and do not cause unintended harm, both today and as capabilities grow. It spans robustness, alignment, monitoring, and governance.

Systematic and unfair skew in a model's outputs that disadvantages certain groups, often arising from biased training data or design choices. Detecting and mitigating it is central to responsible AI.

The effort to make an AI system's behavior match human intentions and values, so it is helpful, honest, and safe rather than merely capable.

The task of identifying data points, events, or observations that deviate significantly from the expected pattern. It is used for fraud detection, fault monitoring, and network security.

The tendency to attribute human traits, intentions, or emotions to an AI system based on its fluent behavior. Being aware of it helps set realistic expectations about what models actually understand.

A hypothetical AI that can understand, learn, and perform any intellectual task a human can, across domains, rather than being specialized to narrow tasks. Whether and when it is achievable is widely debated.

A mechanism that lets a model weigh which parts of the input matter most for each output, dynamically focusing on the relevant pieces.

The area under the ROC curve, a single number between 0 and 1 summarizing a classifier's ability to rank a random positive above a random negative. A value of 0.5 is no better than chance and 1.0 is perfect.

A network that learns to compress input into a compact code and reconstruct it, useful for dimensionality reduction, denoising, and learning representations.

An AI system that pursues a goal by taking a sequence of actions with little or no human intervention, observing results and adjusting as it goes. It typically combines a language model with tools, memory, and planning.

Describing a model that generates output one token at a time, feeding each prediction back as input to produce the next; the mode most language models run in.

The main pretrained feature-extracting portion of a model, on top of which task-specific heads are added. Reusing a strong backbone is central to transfer learning.

The algorithm that trains neural networks by computing how much each parameter contributed to the error and passing those gradients backward through the layers.

A simple text representation that describes a document by the counts of the words it contains, ignoring order and grammar. It is easy to compute but loses sequence information.

Short for bootstrap aggregating, an ensemble method that trains models on different random samples of the data (drawn with replacement) and averages their outputs to reduce variance. Random forests are a well-known example.

A simple reference model or heuristic used as a point of comparison to judge whether a more complex model adds real value. Common baselines include predicting the majority class or the mean.

A subset of training examples processed together in one step; using small batches (mini-batches) balances speed and stability during training.

Running a model over a large set of inputs all at once on a schedule, rather than responding to individual requests in real time. It is efficient for tasks that do not need immediate results.

A technique that rescales the outputs of a layer across a batch to stabilize and speed up training of deep networks.

The number of training examples processed together before the model's weights are updated once. It affects training speed, memory use, and the stability of the gradient estimate.

A formula that describes how to update the probability of a hypothesis given new evidence, relating the posterior to the prior, the likelihood, and the evidence. It is the foundation of Bayesian methods and naive Bayes classifiers.

A framework for updating beliefs about unknown quantities by combining prior assumptions with observed data via Bayes' theorem, producing a posterior distribution. It treats parameters as probability distributions rather than fixed values.

A hyperparameter tuning method that builds a probabilistic model of the objective and uses it to choose the most promising settings to try next. It finds good configurations with fewer evaluations than grid or random search.

A decoding strategy that keeps track of several of the most promising partial sequences at each step instead of committing to just the single best token. It often yields more fluent output than greedy decoding at higher compute cost.

A recursive equation that expresses the value of a state in terms of the immediate reward plus the discounted value of successor states. It is the mathematical backbone of many reinforcement learning algorithms.

A standardized dataset and scoring method used to compare models on a task, enabling apples-to-apples measurement of capability.

An influential transformer language model trained with masked language modeling to produce deep bidirectional representations of text. It is fine-tuned for tasks like classification, question answering, and named entity recognition.

The balance between a model too simple to fit the data (high bias) and one too sensitive to training noise (high variance); good models find the middle.

A recurrent network that processes a sequence in both forward and backward directions and combines the results, so each position has context from both sides. It improves performance on tasks like tagging and translation.

A resampling method that estimates the variability of a statistic by repeatedly drawing samples with replacement from the observed data. It underpins techniques like bagging and confidence-interval estimation.

A rectangle drawn around an object in an image to indicate its location, defined by its coordinates. It is the standard output format for object detection.

A subword tokenization algorithm that starts from characters and repeatedly merges the most frequent adjacent pairs to build a vocabulary of common subwords. It is widely used to tokenize text for language models.

The degree to which a model's predicted probabilities match observed frequencies, so that events predicted with 70% confidence actually occur about 70% of the time. Well-calibrated probabilities are important for decision making.

A rollout strategy that releases a new model or version to a small fraction of traffic first, monitoring it before expanding to everyone. It limits the blast radius of a bad release.

The tendency of a neural network to lose previously learned knowledge when trained on new data or tasks. It is a central challenge for continual and sequential learning.

A prompting technique that asks a model to reason step by step before answering, which often improves accuracy on complex or multi-step problems.

A software system that converses with users in natural language to answer questions or perform tasks. Modern chatbots are typically powered by large language models.

Splitting a long document into smaller passages so they can be embedded, indexed, and retrieved individually in a retrieval-augmented pipeline. Chunk size and overlap affect retrieval quality.

A situation where some classes appear far more often than others in the training data, which can bias a model toward the majority class. It calls for special metrics and techniques like resampling or reweighting.

A technique for handling imbalanced data by giving underrepresented classes more influence in the loss function, so the model does not ignore them. It is an alternative to resampling the data.

A predictive task where a model assigns inputs to discrete categories, such as spam versus not spam or which animal is in a photo.

An unsupervised task that groups data points by similarity without predefined labels, revealing natural segments in the data.

The difficulty a system faces in making good recommendations or predictions for new users or items about which it has little or no data. It is a common challenge for recommender systems.

A recommendation technique that predicts a user's preferences based on the preferences of similar users or items, without needing explicit content features. It powers many product and media recommender systems.

A gradual or sudden change over time in the relationship between inputs and the target a model predicts, causing its accuracy to decline. Detecting it signals that the model needs retraining.

A generative adversarial network that conditions generation on extra information such as a class label or text, so it can produce outputs of a chosen type. It enables controllable image and content generation.

A range of values, computed from data, that is likely to contain a true unknown quantity with a stated probability such as 95%. It expresses the uncertainty around an estimate.

A number a model outputs alongside a prediction to indicate how certain it is, often a probability. It supports thresholding, ranking, and deferring uncertain cases to humans.

A table showing a classifier's correct and incorrect predictions broken down by category, exposing exactly where and how it makes mistakes.

An alignment approach in which a model critiques and revises its own outputs against a set of written principles, reducing the need for direct human feedback on harmful content. The principles act as a guiding constitution for behavior.

A recommendation approach that suggests items similar to those a user has liked before, based on item features rather than other users' behavior. It works even for users with little overlap with others.

Techniques that shorten or summarize the information placed in a model's context window so more can fit or costs drop, while preserving the essentials. It helps manage long documents and conversation history.

The maximum amount of text, measured in tokens, that a model can consider at once, covering both the input prompt and its generated response.

A word or token representation that changes depending on the surrounding context, so the same word gets different vectors in different sentences. Transformer models produce these, unlike static embeddings such as Word2Vec.

A self-supervised technique that learns useful representations by pulling similar examples together and pushing dissimilar ones apart in the embedding space. It underlies many strong image and text encoders.

The point in training at which a model's parameters and loss stop changing meaningfully, indicating it has settled into a solution. Slow or failed convergence often signals problems with the learning rate or data.

An operation that slides a small filter across an input such as an image, computing weighted sums to detect local patterns like edges or textures. It is the core operation of convolutional neural networks.

A neural network that scans inputs with small sliding filters to detect local patterns; especially effective for images and other grid-like data.

A large, structured collection of text used to train or evaluate language models and other NLP systems. Its size, quality, and diversity strongly influence model performance.

The property of an AI system that reliably permits humans to correct, redirect, or shut it down without resisting. It is a desirable safety trait for powerful autonomous systems.

A measure of how alike two embedding vectors are based on the angle between them; widely used to rank matches in semantic search.

An attention mechanism in which one sequence attends to a different sequence, such as a decoder attending to an encoder's output. It lets models condition generation on separate inputs like a source sentence or an image.

A loss function for classification that measures the difference between the predicted probability distribution and the true labels, penalizing confident wrong predictions heavily. It is the standard objective for training classifiers and language models.

An evaluation method that repeatedly splits data into training and testing portions to estimate how well a model will generalize to unseen data.

A training strategy that presents examples to a model in a meaningful order, typically from easier to harder, to improve learning speed and final performance. It is inspired by how humans learn progressively.

Expanding a training set by creating modified copies of existing examples, such as rotating or cropping images, to improve robustness and reduce overfitting.

A change over time in the distribution of a model's input data compared with what it was trained on, which can degrade performance even if the underlying relationship is unchanged. Monitoring for it is a standard MLOps practice.

The process of annotating raw data with the correct answers a supervised model should learn to predict, such as tagging images or categorizing text. Its cost and quality strongly affect model outcomes.

A modeling error where information from outside the training data, often the target or future data, sneaks into the features and inflates performance during development. It leads to models that fail in production.

A density-based clustering algorithm that groups together points packed closely in space and marks points in low-density regions as outliers. Unlike k-means, it finds clusters of arbitrary shape and does not need the cluster count specified.

A supervised learning model that makes predictions by following a branching series of yes/no questions about the input features, with each leaf of the tree giving a final class or value. It is easy to interpret but prone to overfitting when grown deep.

Machine learning using neural networks with many layers, which learn increasingly abstract features and power most modern AI, including image and language models.

A reinforcement learning method that uses a deep neural network to approximate the value of actions in each state, enabling agents to learn from high-dimensional inputs like pixels. It was a landmark in playing Atari games from raw images.

Synthetic media, typically video or audio, in which a person's likeness or voice is realistically fabricated using generative models. Deepfakes raise concerns about misinformation, fraud, and consent.

An autoencoder trained to reconstruct clean inputs from deliberately corrupted versions, forcing it to learn robust features. It is used for representation learning and data denoising.

A mathematical framework that lets systems learn useful patterns from data while provably limiting how much can be inferred about any single individual, typically by adding calibrated noise. It gives formal privacy guarantees.

A generative model that learns to reverse a gradual noising process, turning random noise into coherent images or other data; the basis of many AI image generators.

The process of reducing the number of input variables in a dataset while preserving as much meaningful structure as possible. It helps with visualization, speeds up training, and can reduce noise.

An alignment method that fine-tunes a language model directly on human preference pairs without training a separate reward model or running reinforcement learning. It simplifies the RLHF pipeline while achieving similar results.

A number between 0 and 1 in reinforcement learning that determines how much future rewards are valued relative to immediate ones. Lower values make the agent more short-sighted.

In a generative adversarial network, the model that tries to distinguish real data from the fakes produced by the generator. Its feedback drives the generator to improve.

Training a smaller student model to imitate a larger teacher model, keeping much of the capability while cutting size and cost.

A regularization technique that randomly ignores some neurons during training, forcing the network to build redundant, more robust representations.

A regularization strategy that halts training once performance on a validation set stops improving, preventing the model from overfitting the training data. It uses the validation curve to pick the best stopping point.

Running AI models directly on local devices such as phones, cameras, or sensors rather than in the cloud, reducing latency and preserving privacy. It requires compact, efficient models.

A regularized regression method that combines the L1 penalty of lasso and the L2 penalty of ridge, balancing feature selection with stability when features are correlated.

A representation of text, images, or other data as a list of numbers (a vector) positioned so that similar items sit close together in that numeric space.

The multi-dimensional vector space into which items such as words, images, or users are mapped so that geometric closeness reflects semantic similarity. Operations like nearest-neighbor search happen within it.

Capabilities that appear in large models once they cross a certain scale but are largely absent in smaller ones, sometimes seeming to arrive abruptly. Examples include multi-step reasoning and in-context learning of new tasks.

An architecture pattern where an encoder compresses the input into an internal representation and a decoder expands that representation into an output. It is common in translation, summarization, and image captioning.

A strategy that combines the predictions of multiple models to produce a result more accurate and robust than any single model. Common forms include bagging, boosting, and stacking.

In information theory, a measure of the uncertainty or disorder in a distribution, highest when outcomes are equally likely. In machine learning it appears in decision-tree splitting and in cross-entropy loss.

One complete pass of the training algorithm over the entire training dataset; models usually train for many epochs.

A simple exploration strategy that picks the best-known action most of the time but chooses a random action with a small probability epsilon. It provides a basic balance between exploration and exploitation.

A test or suite of tests that measures how well a model performs on specific tasks or behaviors, used to track quality and catch regressions.

An iterative algorithm for finding maximum-likelihood parameter estimates when the data has hidden or missing variables. It alternates between estimating the hidden values (E-step) and updating parameters (M-step).

The practice of systematically recording the configurations, code, data, metrics, and results of machine learning experiments so they can be compared and reproduced. Dedicated tools log these runs automatically.

An early AI program that encodes the knowledge of human experts as a set of if-then rules to make decisions in a narrow domain. Such systems were prominent in the 1980s before machine learning became dominant.

The extent to which the reasons behind an AI system's outputs can be presented in human-understandable terms. It is important for trust, debugging, and regulatory compliance.

A set of methods and tools designed to make the decisions of complex machine learning models understandable to humans. It is often abbreviated XAI.

A training problem where gradients grow uncontrollably large as they propagate through many layers, causing unstable updates and numerical overflow. Gradient clipping and careful initialization help address it.

The core tradeoff in reinforcement learning between trying new actions to discover their outcomes and choosing known good actions to maximize reward. Balancing the two is essential for effective learning.

A single accuracy metric that combines precision and recall into their harmonic mean, useful when classes are imbalanced and both error types matter.

The goal of ensuring an AI system treats individuals and groups equitably and does not produce discriminatory outcomes. Multiple, sometimes conflicting, mathematical definitions of it exist.

An individual measurable property or input signal a model uses to make predictions, such as a pixel value, a word, or a customer's age.

The craft of selecting, transforming, and creating input features from raw data to help a model learn more effectively.

A measure of how much each input feature contributes to a model's predictions, used to interpret models and guide feature selection. Different methods, such as permutation importance or tree-based scores, can give different rankings.

The output produced when a convolutional filter is applied across an input, highlighting where a particular pattern appears. A convolutional layer produces many feature maps, one per filter.

Adjusting numeric input features to a common range or distribution so that no single feature dominates due to its units. Common methods are normalization to a 0–1 range and standardization to zero mean and unit variance.

The process of choosing the most relevant subset of input variables for a model, discarding redundant or uninformative ones. It can improve accuracy, reduce overfitting, and lower computational cost.

A central repository that stores, manages, and serves curated features for machine learning, ensuring the same feature definitions are used in training and in production. It reduces duplication and training-serving skew.

A training approach in which a shared model is improved across many devices or servers that keep their data local, sending only model updates instead of raw data. It enables learning while preserving data privacy.

A neural network in which information flows in one direction from input to output with no loops or cycles. It is the most basic form of deep network, contrasted with recurrent architectures.

Guiding a model by including a handful of worked examples in the prompt so it infers the desired pattern without any additional training.

Further training a pretrained model on a smaller, task-specific dataset so it specializes in a particular domain, style, or behavior.

An efficient, memory-aware implementation of the attention operation that avoids storing the large intermediate attention matrix, speeding up training and inference on long sequences. It computes exact attention with lower memory use.

A large model pretrained on broad data that serves as a general base, adaptable to many downstream tasks through prompting or fine-tuning.

A term for the largest and most capable general-purpose AI models at the leading edge of what is currently possible. Their scale and capabilities raise both new opportunities and safety considerations.

Model parameters that are held fixed and not updated during a training or fine-tuning run. Freezing a pretrained backbone while training only new layers is common in transfer learning.

A neural network layer in which every neuron is connected to every output of the previous layer. It is also called a dense layer and is common near the end of many architectures.

A type of recurrent network cell that uses gating mechanisms to control information flow, capturing long-range dependencies with fewer parameters than an LSTM. It is a common alternative to LSTMs for sequence modeling.

A probabilistic clustering model that assumes the data is generated from a mixture of several Gaussian distributions with unknown parameters. Each point is assigned a probability of belonging to each cluster.

The Gaussian Error Linear Unit, a smooth activation function that weights inputs by their value under a Gaussian distribution. It is widely used in transformer models.

A model's ability to perform well on new, unseen data rather than just the examples it was trained on; the real goal of machine learning.

A pair of networks trained against each other, one generating fake data and one trying to detect it, producing increasingly realistic synthetic outputs.

AI that creates new content such as text, images, audio, or code, rather than only classifying or predicting from existing inputs.

In a generative adversarial network, the model that creates synthetic data samples from random noise, trying to fool the discriminator into judging them real. It improves by learning to produce increasingly realistic outputs.

A measure of how often a randomly chosen element would be misclassified if labeled by the class distribution in a node. Decision trees minimize it to pick the best splits.

A method for learning word embeddings by factorizing a matrix of global word co-occurrence counts across a corpus. It combines the strengths of count-based and prediction-based approaches.

A family of decoder-only transformer models trained to predict the next token, generating text autoregressively. The architecture underpins many modern large language models.

A graphics processing unit, a parallel processor originally built for rendering images that is now the workhorse hardware for training and running neural networks. Its ability to do many operations at once suits the matrix math of deep learning.

A vector showing how the loss changes as each parameter changes; it points the direction to adjust parameters to reduce error.

An ensemble technique that builds models sequentially, where each new model is trained to correct the errors of the combined previous models by fitting to their residuals. It often produces very accurate predictors for tabular data.

A technique that caps the magnitude of gradients during training to prevent excessively large updates that destabilize learning. It is especially useful for recurrent networks prone to exploding gradients.

The core optimization method for training models, iteratively nudging parameters in the direction that most reduces the loss, one small step at a time.

A neural network that operates on graph-structured data by passing and aggregating messages between connected nodes. It is used for molecules, social networks, recommendation, and other relational data.

A text-generation strategy that always selects the single highest-probability token at each step. It is fast and deterministic but can produce repetitive or suboptimal sequences.

A hyperparameter tuning method that exhaustively tries every combination of values from a predefined grid and selects the best. It is simple but scales poorly as the number of hyperparameters grows.

The reference labels or measurements assumed to be correct, against which a model's predictions are compared during training and evaluation. Its quality directly limits how good a model can become.

Connecting a model's outputs to verifiable external information such as retrieved documents, databases, or sensor data so its responses are factual and traceable. It is a key technique for reducing hallucination.

Rules, filters, or checks placed around a model to keep its inputs and outputs safe, on-policy, and within intended bounds.

When a model produces fluent but false or fabricated information, stating it confidently as if it were true.

The set of techniques used to reduce a language model's tendency to state false or fabricated information, such as retrieval grounding, citation, and verification steps. It is central to making generative AI trustworthy.

A weight initialization scheme designed for ReLU-style activations that scales initial weights by the number of input units to preserve signal variance. It helps very deep networks train reliably.

A layer of neurons in a neural network that sits between the input and output layers and learns intermediate representations of the data. Networks with many hidden layers are called deep.

An unsupervised method that builds a tree of nested clusters by repeatedly merging the closest groups (agglomerative) or splitting them (divisive). The result is often visualized as a dendrogram.

A design in which people review, correct, or approve an AI system's outputs at key steps rather than letting it act fully autonomously. It improves safety and quality for high-stakes decisions.

A retrieval approach that combines keyword-based (lexical) search with vector-based (semantic) search to get the strengths of both. It improves recall on exact terms while still capturing meaning-level matches.

A setting chosen before training that controls how a model learns, such as the learning rate or number of layers, as opposed to values the model learns itself.

The process of searching for the configuration settings that are not learned from data, such as learning rate or tree depth, to maximize a model's performance. Methods include grid search, random search, and Bayesian optimization.

A flat decision boundary of one dimension less than its surrounding space, such as a line in 2D or a plane in 3D. Linear classifiers separate classes using a hyperplane.

A statistical procedure for deciding whether observed data provides enough evidence to reject a default assumption (the null hypothesis) in favor of an alternative. It is used to assess whether effects are likely real or due to chance.

The task of automatically generating a natural-language description of the content of an image. It sits at the intersection of computer vision and natural language generation.

The computer vision task of assigning a whole image to one of a set of predefined categories. It is a foundational benchmark task in the field.

A model's ability to adapt to a task purely from the examples and instructions in its prompt at inference time, without updating its weights.

Using a trained model to generate predictions or outputs on new inputs; the phase after training when the model is actually put to work.

The reduction in uncertainty (entropy) about the target achieved by splitting data on a particular feature. Decision trees use it to choose which feature to split on at each node.

A computer vision task that identifies each individual object in an image at the pixel level, distinguishing separate instances of the same class. It combines object detection with per-pixel masks.

Fine-tuning a pretrained language model on examples of instructions paired with desired responses, so it learns to follow natural-language directions. It is a key step in turning a base model into a helpful assistant.

The degree to which a human can understand how a model reaches its decisions, either because the model is inherently simple or through analysis techniques. Higher interpretability aids trust and oversight.

A metric that measures how well a predicted region overlaps a ground-truth region by dividing the area of their intersection by the area of their union. It is the standard way to score object detection and segmentation.

A crafted prompt or interaction that tricks a language model into bypassing its safety guidelines and producing content it was trained to refuse. Defending against jailbreaks is an ongoing safety challenge.

A validation technique that splits data into k equal folds, training on k-1 of them and testing on the remaining one, rotating until each fold has been the test set. Averaging the results gives a more reliable performance estimate.

An unsupervised algorithm that partitions data into k groups by iteratively assigning each point to the nearest cluster center and then recomputing those centers. It requires choosing the number of clusters in advance.

A simple algorithm that classifies or predicts a new point based on the labels or values of its k closest examples in the training data. It does no explicit training and stores the whole dataset for lookup at prediction time.

A method that lets algorithms like SVMs operate in a high-dimensional feature space without explicitly computing coordinates there, by using a kernel function to compute similarities directly. It enables efficient non-linear modeling.

A structured representation of facts as entities connected by typed relationships, forming a network of knowledge that machines can query and reason over. It is used to ground AI systems in verified information.

A memory optimization for transformer generation that stores the key and value vectors of previously processed tokens so they need not be recomputed for each new token. It greatly speeds up autoregressive decoding at the cost of memory.

A penalty added to the loss proportional to the sum of the absolute values of the model's weights. It encourages sparsity, driving some weights exactly to zero.

A penalty added to the loss proportional to the sum of the squared values of the model's weights. It discourages large weights and improves generalization without forcing weights to zero.

The correct answer attached to a training example in supervised learning; the target the model learns to predict.

A model that assigns probabilities to sequences of words or tokens, allowing it to predict likely next tokens and generate text. Large versions trained on huge corpora power modern generative AI.

A model trained on vast amounts of text to predict and generate language, capable of tasks like answering questions, writing, and reasoning over text.

A linear regression variant that adds an L1 penalty on the coefficients, which can shrink some of them exactly to zero and thereby perform automatic feature selection.

The time it takes for a model or system to return a response after receiving a request. Low latency is critical for interactive applications like chat and real-time recommendations.

A diffusion generation approach that runs the denoising process in a compressed latent space rather than on raw pixels, greatly reducing computation. It is the basis of many efficient text-to-image models.

A popular topic-modeling algorithm that represents each document as a mixture of topics and each topic as a distribution over words. It infers these hidden structures from word co-occurrence patterns.

The compressed internal representation a model learns, where each point is a vector capturing meaningful features of the data in an abstract numeric form.

A technique that normalizes the activations across the features of each individual example, stabilizing and speeding up training. It is the standard normalization used inside transformer layers.

A variant of the ReLU activation that allows a small, non-zero slope for negative inputs instead of outputting exactly zero. This helps prevent neurons from becoming permanently inactive.

A plot showing how model performance changes as a function of training experience, such as dataset size or training time. It helps diagnose whether more data or more training would help.

A hyperparameter that controls how large a step gradient descent takes on each update; too high overshoots, too low trains slowly.

A plan for changing the learning rate during training, such as decaying it over time or warming it up at the start. A good schedule can improve convergence speed and final accuracy.

A text-normalization technique that reduces words to their dictionary base form using vocabulary and grammar rules, so different inflections map to one lemma. It is more accurate but slower than stemming.

The probability of the observed data given a particular set of model parameters, viewed as a function of those parameters. Maximizing it is a common way to fit models.

A technique that explains a single prediction of any model by fitting a simple, interpretable model to the behavior of the complex one in the neighborhood of that input. Its name stands for Local Interpretable Model-agnostic Explanations.

A model that predicts a continuous value as a weighted linear combination of input features, fitting the line or plane that best matches the data. It is the foundational technique for regression problems.

A point in the loss landscape where the loss is lower than at all nearby points but not necessarily the lowest possible overall. Optimization can get stuck in poor local minima, though in high dimensions this is less common than once feared.

A statistical model for binary or categorical classification that estimates the probability of a class by applying a logistic (sigmoid) function to a linear combination of the input features.

The raw, unnormalized scores a model outputs for each possible class or token before they are converted into probabilities by a softmax function. They can be any real number, positive or negative.

A type of recurrent network with gating that lets it remember information over longer sequences, mitigating the forgetting problem of plain RNNs.

A lightweight fine-tuning method that trains small add-on matrices instead of all a model's weights, cutting the cost and storage of customizing large models.

A plot of a model's loss over training steps or epochs, used to diagnose whether training is progressing, converging, or overfitting. Comparing training and validation loss curves reveals generalization problems.

A formula that measures how wrong a model's predictions are, producing a single number that training tries to minimize.

A field where systems learn patterns from data to make predictions or decisions, rather than following rules written explicitly by a programmer.

The automatic conversion of text or speech from one human language to another. Modern systems use neural sequence-to-sequence and transformer models.

A mathematical framework for sequential decision making defined by states, actions, transition probabilities, and rewards, where the future depends only on the current state. It is the foundation of reinforcement learning theory.

A self-supervised training objective where random tokens in the input are hidden and the model learns to predict them from the surrounding context. It is the pretraining task used by models like BERT.

A pooling operation that keeps only the largest value in each small region of a feature map, preserving the strongest activations while reducing resolution. It is the most common pooling method in vision models.

A method of estimating a model's parameters by choosing the values that make the observed data most probable under the model. It underlies the training of many statistical and machine learning models.

A regression metric equal to the average absolute difference between predicted and actual values. It is more robust to outliers than mean squared error and is expressed in the target's units.

A regression loss and metric equal to the average of the squared differences between predicted and actual values. Squaring penalizes large errors more heavily and makes it sensitive to outliers.

A research area that aims to reverse-engineer the internal computations of neural networks, identifying the circuits and features that produce specific behaviors. It seeks a detailed, causal understanding of how models work inside.

A form of gradient descent that computes updates on small batches of examples, balancing the stability of full-batch training with the efficiency of single-example updates. It is the standard approach for training deep networks.

A training technique that uses lower-precision number formats such as 16-bit floats for most computations while keeping key parts in higher precision, cutting memory use and speeding up training. Modern accelerators are optimized for it.

An architecture that routes each input to only a few specialized sub-networks (experts) instead of the whole model, boosting capacity without proportional compute.

The set of practices for reliably deploying, monitoring, and maintaining machine learning models in production, adapting DevOps principles to ML. It covers data pipelines, versioning, testing, and continuous retraining.

A failure mode of generative adversarial networks where the generator produces only a few similar outputs instead of the full diversity of the data. It results in a narrow, repetitive set of samples.

The learned artifact produced by training, containing the parameters that map inputs to outputs for a given task.

A short standardized document that describes a machine learning model's intended use, performance, limitations, training data, and ethical considerations. It promotes transparency and responsible deployment.

A degradation that can occur when generative models are repeatedly trained on data produced by earlier models, causing them to lose diversity and drift from the real distribution. It is a concern as AI-generated content fills training corpora.

A set of techniques, including pruning, quantization, and distillation, that reduce a model's size and computational cost so it can run faster or on constrained hardware. The goal is efficiency with little loss of accuracy.

An open standard for connecting AI models to external tools and data sources through a common interface, so any compliant client can use any compliant server.

The process of putting a trained model into a production environment where it can serve predictions to applications or users. It involves packaging, serving infrastructure, and monitoring.

The ongoing tracking of a deployed model's performance, inputs, and outputs to detect problems like accuracy decay, drift, or errors. It triggers alerts and retraining when quality degrades.

A centralized system for storing, versioning, and cataloging trained models along with their metadata and stage (such as staging or production). It provides governance and reproducibility for ML teams.

Making a trained model available to respond to prediction requests, typically behind an API, with attention to latency, scalability, and reliability. It is the runtime side of deployment.

An approach in which the agent learns or is given a model of the environment's dynamics and uses it to plan ahead. It can be more sample-efficient than model-free methods but depends on model accuracy.

An approach in which the agent learns a policy or value function directly from experience without building an explicit model of the environment. Q-learning and policy gradients are examples.

An optimization technique that accumulates a moving average of past gradients to keep updates moving in a consistent direction, speeding convergence and smoothing out oscillations. It is a common enhancement to gradient descent.

A family of techniques that estimate quantities by averaging the results of many random samples or simulations. In reinforcement learning they estimate value by averaging returns from complete episodes.

A setup in which several AI agents interact, coordinate, or compete to solve a problem, sometimes with specialized roles. Coordination protocols and communication become central design concerns.

A transformer mechanism that runs several attention operations in parallel, each with its own learned projections, then combines them. Multiple heads let the model attend to different relationships at once.

A basic feedforward neural network with one or more hidden layers of neurons between the input and output, using non-linear activations. It can approximate complex functions and is a standard baseline architecture.

Describing models that work across more than one type of data at once, such as understanding images and text together or generating audio from a description.

A model that can process and relate information from more than one type of data, such as text, images, audio, or video, within a single system. It enables tasks like describing an image or answering questions about a chart.

A contiguous sequence of n items, usually words or characters, used to capture short-range context in text. Bigrams and trigrams are common in language modeling and text features.

A probabilistic classifier based on Bayes' theorem that assumes all features are conditionally independent given the class. Despite this simplifying assumption, it works well for tasks like spam filtering and text classification.

An NLP task that finds and classifies named things in text, such as people, organizations, locations, and dates. It is a building block for information extraction.

AI systems designed and trained for a specific, limited task, such as recognizing faces or recommending products, without general reasoning ability. Essentially all AI in use today is narrow.

The subfield of NLP focused on producing coherent, human-like text from data or internal representations. It covers tasks like summarization, dialogue, and report writing.

The field of AI concerned with enabling computers to understand, interpret, and generate human language. It spans tasks from translation and summarization to sentiment analysis and question answering.

The subfield of NLP focused on extracting meaning, intent, and structure from text, such as identifying entities, relationships, and sentiment. It emphasizes comprehension rather than generation.

The task of finding the items in a dataset whose vectors are closest to a query vector, used for retrieval and recommendation. Approximate versions trade a little accuracy for large speed gains at scale.

A model built from layers of interconnected nodes loosely inspired by the brain, where each connection has a weight adjusted during training.

The training objective of autoregressive language models, which learn to predict the following token given all preceding ones. Repeating this prediction generates coherent text.

A post-processing step in object detection that removes redundant overlapping boxes for the same object, keeping only the highest-scoring one. It cleans up the raw set of candidate detections.

Rescaling numeric values to a fixed range, typically 0 to 1, so features with different units contribute comparably. In deep learning the term also refers to layer techniques that stabilize activations.

A generative model that transforms a simple probability distribution into a complex one through a sequence of invertible functions, allowing exact likelihood computation. It is used for density estimation and generation.

The default assumption in a statistical test, typically that there is no effect or no difference. Tests attempt to gather enough evidence to reject it.

The computer vision task of locating and classifying multiple objects within an image, typically by drawing a bounding box around each and labeling it. It combines localization with classification.

The quantity that training seeks to optimize, either minimized as a loss or maximized as a reward. Choosing it defines what the model is actually trying to achieve.

A way to represent a categorical variable as a set of binary columns, one per category, where exactly one column is 1 and the rest are 0. It lets models that expect numeric input handle non-numeric categories.

A training paradigm where the model updates its parameters continuously as new data arrives, one example or small batch at a time, rather than training once on a fixed dataset. It suits streaming or changing data.

A model whose trained parameters are publicly released so anyone can run, inspect, or fine-tune it, though its training data or code may not be fully open. It contrasts with closed models accessed only through an API.

The technology that converts images of typed, printed, or handwritten text into machine-readable characters. It is used to digitize documents and read text from photos.

The algorithm that adjusts a model's parameters during training to reduce the loss, using the gradients from backpropagation. Common choices include SGD, Adam, and RMSprop.

Coordinating multiple models, tools, or agent steps into a coherent pipeline that accomplishes a larger task. An orchestrator decides the order of operations and passes information between components.

The standard method for fitting a linear regression by choosing the coefficients that minimize the sum of squared differences between predictions and observed values.

An estimate of a bagged model's error computed using, for each training point, only the trees that did not see it during training. It provides a built-in validation score without a separate holdout set.

When a model memorizes its training data, including noise and quirks, so it performs well on those examples but poorly on new, unseen data.

Recognizing when a model has learned noise in the training data by observing a growing gap between strong training performance and weaker validation performance. It signals the need for regularization or more data.

Building a model with far more parameters than the training data strictly requires, which, counterintuitively, often improves how well large networks learn.

In hypothesis testing, the probability of observing data at least as extreme as what was seen if the null hypothesis were true. Small p-values are taken as evidence against the null hypothesis.

Adding extra values, usually zeros, around the border of an input so that a convolution can process edge pixels and control the output size. Same padding keeps dimensions unchanged; valid padding uses none.

A family of methods that adapt a large pretrained model to a new task by training only a small number of added or selected parameters while keeping most of the model frozen. It cuts the memory and storage cost of customization.

The total count of learned weights in a model, often in the billions; roughly indicates capacity, though architecture and training data matter just as much.

An NLP task that labels each word in a sentence with its grammatical category, such as noun, verb, or adjective. It supports parsing and downstream language understanding.

The simplest artificial neuron, which computes a weighted sum of its inputs and outputs one of two values based on a threshold. It is the historical building block that led to modern neural networks.

A metric for language models measuring how surprised a model is by a text; lower perplexity means the model predicts the words more confidently.

In reinforcement learning, the strategy that maps states to actions, telling the agent what to do in each situation. It can be deterministic or a probability distribution over actions.

A family of reinforcement learning methods that directly optimize the parameters of a policy by following the gradient of expected reward. They handle continuous action spaces well.

A downsampling operation in convolutional networks that summarizes small regions of a feature map, for example by taking the maximum or average. It reduces spatial size and adds a degree of translation invariance.

The computer vision task of detecting the positions of a person's or object's key points, such as joints, to infer body posture or orientation. It is used in motion capture, fitness apps, and robotics.

Information added to a transformer's inputs so the model knows the order of tokens, since attention itself has no built-in sense of sequence.

In Bayesian statistics, the updated probability distribution over a quantity after combining the prior with observed data through Bayes' theorem. It represents what is believed after seeing the evidence.

Of all the items a model flagged as positive, the fraction that were actually correct; it measures how trustworthy the positive predictions are.

A plot of precision against recall across decision thresholds, especially informative when classes are imbalanced. It highlights performance on the rare positive class better than the ROC curve.

A parameter-efficient fine-tuning method that prepends a small set of trainable vectors to each layer's input while leaving the base model frozen. The learned prefixes steer the model toward a task.

The initial, resource-intensive phase where a model learns general patterns from a huge unlabeled corpus before any task-specific tuning.

A dimensionality-reduction technique that projects data onto a new set of axes (principal components) ordered by how much variance they capture. It compresses correlated features into fewer uncorrelated ones.

In Bayesian statistics, the probability distribution that expresses beliefs about a quantity before observing the current data. It is combined with the likelihood to form the posterior.

The input text given to a language model to guide its response, ranging from a short question to detailed instructions and examples.

Composing a task from a sequence of prompts where each step's output feeds the next, breaking a complex job into manageable stages. It improves reliability over asking for everything in one prompt.

The practice of crafting and refining prompts, including instructions, examples, and formatting, to reliably get better outputs from a language model.

An attack where malicious instructions hidden in user input or fetched content hijack a model, causing it to ignore its intended rules or leak data.

An attack in which a user coaxes a language model into revealing its hidden system prompt or confidential instructions. It is a variant of prompt injection and a security concern for deployed applications.

A reusable, parameterized text pattern with placeholders that are filled in with specific inputs to build a consistent prompt for a language model. Templates make prompting repeatable and easier to maintain.

A lightweight adaptation method that learns a small set of continuous prompt vectors prepended to the input, keeping the base model unchanged. It is a simple form of parameter-efficient fine-tuning.

A widely used reinforcement learning algorithm that improves a policy in small, stable steps by limiting how far each update can move from the previous policy. It is a common choice for the reinforcement stage of RLHF.

Simplifying a trained model by removing parts that add little value, such as branches of a decision tree or weights in a neural network. It reduces overfitting and model size.

Compressing a trained neural network by removing weights, neurons, or channels that contribute little to its output, shrinking the model with minimal accuracy loss. It speeds up inference and reduces memory.

A model-free reinforcement learning algorithm that learns the value of taking each action in each state, gradually improving its estimates to derive an optimal policy. It does not require a model of the environment.

A fine-tuning technique that combines low-rank adapters with a quantized (compressed) base model, letting large models be adapted on a single consumer GPU. It makes efficient fine-tuning far more accessible.

Reducing the numeric precision of a model's weights (for example from 16-bit to 4-bit) to shrink memory use and speed up inference, usually with minor accuracy loss.

The three learned vectors that drive attention: the query is compared against keys to produce weights, which are then used to combine the values. This mechanism lets each token gather information from others.

A regression metric, also called the coefficient of determination, that measures the fraction of variance in the target explained by the model. A value of 1 means perfect fit and 0 means no better than predicting the mean.

An ensemble method that trains many decision trees on random subsets of the data and features, then averages or votes their predictions. Combining many diverse trees reduces overfitting and usually improves accuracy over a single tree.

An agent pattern in which a language model interleaves reasoning steps with actions such as tool or search calls, using the results to inform its next thought. It combines chain-of-thought reasoning with acting in an environment.

A language model trained to spend extra computation working through a problem step by step before answering, trading latency for stronger performance on hard tasks.

Of all the truly positive items, the fraction the model correctly caught; it measures how much of the target the model finds.

The region of the input that influences a particular neuron's activation in a convolutional network. Deeper layers have larger receptive fields and capture broader context.

A system that suggests items a user is likely to want, using patterns in past behavior, item attributes, or both. Approaches include collaborative filtering, content-based filtering, and hybrids.

A neural network that processes sequences one step at a time while carrying a memory of previous steps, historically used for text and time series.

The practice of deliberately probing an AI system for harmful, unsafe, or policy-violating behavior by simulating adversaries. Findings are used to strengthen safeguards before and after deployment.

A predictive task where a model outputs a continuous numeric value, such as a price, temperature, or expected demand.

Techniques that discourage a model from becoming too complex, penalizing large parameters or adding constraints to reduce overfitting.

A method where an agent learns by trial and error, taking actions in an environment and adjusting its behavior based on rewards and penalties.

A tuning method that uses human preference rankings to train a reward signal, then optimizes the model to produce responses people prefer.

A widely used activation function that outputs the input if positive and zero otherwise; simple, fast, and effective for deep networks.

A second-pass step that reorders an initial set of retrieved results using a more precise model, surfacing the most relevant items to the top.

A shortcut that adds a layer's input directly to its output, letting gradients flow more easily and enabling training of very deep networks. It is the key idea behind residual networks (ResNets).

A deep convolutional architecture built from residual blocks that use skip connections, allowing networks hundreds of layers deep to train effectively. It set accuracy records in image classification and remains a common backbone.

The step of finding and returning relevant documents or passages from a knowledge source in response to a query, typically using keyword or vector similarity search. It supplies external context to grounded and retrieval-augmented systems.

A pattern where a model first fetches relevant documents from an external source, then uses them as context to generate a grounded, up-to-date answer.

The function in reinforcement learning that assigns a numeric score to the outcomes of an agent's actions, defining what it should try to achieve. Designing it well is crucial and mis-specification can cause unintended behavior.

When an agent finds a way to maximize its reward signal that satisfies the letter of the objective but not the intent, producing unintended or degenerate behavior. It highlights the difficulty of specifying good objectives.

A model trained to score how good a response is according to human preferences, used to guide reinforcement learning of a language model. It converts subjective human judgments into a numeric signal.

A linear regression variant that adds an L2 penalty on the size of the coefficients to reduce overfitting and handle correlated features. The penalty shrinks coefficients toward zero without eliminating them.

Reinforcement Learning from AI Feedback, a method that trains a model using preference judgments generated by another AI model instead of humans. It aims to scale alignment by reducing reliance on human labelers.

An optimizer that divides each parameter's learning rate by a running average of the magnitudes of its recent gradients, helping training progress steadily on both steep and flat directions.

A model's ability to maintain reliable performance when inputs are noisy, shifted, or adversarially manipulated. It is a central property for safe and dependable AI systems.

A plot of a classifier's true positive rate against its false positive rate across all decision thresholds. It shows the tradeoff between catching positives and raising false alarms.

The square root of the mean squared error, expressing regression error in the same units as the target variable. It is a common summary of how far predictions typically fall from the truth.

A method of encoding token positions in transformers by rotating the query and key vectors according to position, which generalizes well to longer sequences. It is used in many modern large language models.

Empirical relationships showing how model performance improves predictably as model size, dataset size, and compute increase. They guide decisions about how large to make models and how much data to use.

Attention applied within a single sequence, letting each element relate to every other element so the model captures context and relationships across the input.

A prompting technique that samples multiple reasoning paths for the same question and selects the most common answer among them. It improves accuracy on reasoning tasks over taking a single chain of thought.

A training approach where the model creates its own supervision signal from unlabeled data, for example by predicting a hidden part of the input from the rest. It underpins the pretraining of most modern language and vision models.

Search that matches on meaning rather than exact keywords, comparing embeddings so a query finds relevant results even when the wording differs.

The computer vision task of labeling every pixel in an image with the class of the object it belongs to, without distinguishing separate instances of the same class. It produces a dense, per-pixel map.

A learning setting that uses a small amount of labeled data together with a large amount of unlabeled data. It aims to get better performance than labeled data alone would allow when labels are expensive.

Another name for recall, the fraction of actual positives that a classifier correctly identifies. It is widely used in medical and diagnostic testing.

A single vector that represents the meaning of an entire sentence or passage, enabling similarity comparison and semantic search. It is produced by models designed to encode whole texts.

A tokenization toolkit and method that treats text as a raw stream, including spaces, so it works across languages without pre-tokenization. It supports byte-pair encoding and unigram tokenization.

An NLP task that determines the emotional tone or opinion expressed in text, often classifying it as positive, negative, or neutral. It is widely used for analyzing reviews and social media.

The task of learning from and predicting ordered data such as text, audio, time series, or DNA, where the order of elements carries meaning. Recurrent networks, convolutional models, and transformers are all used for it.

A modeling framework that maps an input sequence to an output sequence of possibly different length, using an encoder to read the input and a decoder to generate the output. It underlies machine translation and summarization.

Running a new model alongside the current one on live traffic without serving its results to users, to compare its behavior safely before promotion. It surfaces problems without user impact.

A method for explaining individual model predictions by assigning each feature a contribution value based on cooperative game theory. It provides consistent, locally accurate feature attributions.

An activation function that squashes any real number into the range 0 to 1 using an S-shaped curve. It is useful for outputting probabilities but can cause vanishing gradients in deep networks.

The challenge and set of techniques for transferring a model or policy trained in simulation to work reliably in the real world, bridging the gap between simulated and physical conditions. It is important in robotics.

A function that converts a set of raw scores into probabilities that sum to one, commonly used to produce a model's confidence across categories.

A model in which many parameters or activations are zero, so only a small portion is used or stored at a time, reducing compute and memory. Mixture-of-experts models are one popular way to build sparsity into large networks.

A classification metric measuring the fraction of actual negatives that the model correctly identifies as negative. It complements recall (sensitivity), which measures correctly identified positives.

An inference speedup where a small, fast model drafts several tokens that a larger model then verifies in parallel, accepting the correct ones. It reduces latency without changing the larger model's output distribution.

The technology that converts spoken audio into written text, also called automatic speech recognition. It powers voice assistants, transcription, and dictation.

Transforming a feature so it has zero mean and unit standard deviation, expressing each value as how many standard deviations it sits from the mean. It is a common preprocessing step for many algorithms.

A text-normalization technique that strips words down to a rough root form by chopping off suffixes, so related words are treated the same. It is fast but can produce non-words.

An optimization method that updates model parameters using the gradient computed on one training example or a small batch at a time, rather than the whole dataset. Its noisy updates are efficient and can help escape poor solutions.

Very common words such as the, is, and of that are often filtered out before text processing because they carry little distinguishing meaning. Removing them can reduce noise in some tasks.

A sampling method that preserves the proportion of each class or group when splitting or subsampling data. It ensures training and test sets have representative class distributions.

The step size by which a convolutional filter or pooling window moves across the input. A larger stride reduces the output size and computation.

Constraining a language model to produce responses in a defined format such as JSON matching a schema, so downstream software can parse them reliably. It is essential for using LLMs in automated pipelines.

A tokenization approach that breaks words into smaller reusable units so the model can handle rare or unseen words as combinations of known pieces. It balances vocabulary size against sequence length.

Training a model on examples that are each paired with a correct answer (a label), so it learns to predict that answer for new, unlabeled inputs.

A supervised algorithm that finds the boundary (hyperplane) that best separates classes by maximizing the margin between them. Using kernels, it can also model non-linear boundaries.

The training points that lie closest to the decision boundary in a support vector machine and determine its position. Only these points influence the final model.

An approach to AI based on explicit rules and manipulation of human-readable symbols and logic, dominant before the rise of statistical machine learning. It is strong at reasoning but brittle and hard to scale to messy real-world data.

Artificially generated data, created by simulations or generative models, used to augment or replace real data for training. It helps when real data is scarce, sensitive, or expensive to label.

A special instruction that sets a model's role, rules, and behavior for a conversation, applied before and above the user's messages.

A non-linear dimensionality-reduction technique used mainly to visualize high-dimensional data in two or three dimensions by keeping similar points close together. It is good for exploration but its distances between clusters should not be over-interpreted.

An activation function shaped like sigmoid but outputting values between -1 and 1, centered at zero. Its zero-centered output often helps optimization compared with sigmoid.

A setting that controls randomness in a model's output; low values make responses focused and predictable, high values make them more varied and creative.

A reinforcement learning approach that updates value estimates using the difference between successive predictions, learning from each step without waiting for a final outcome. It blends ideas from Monte Carlo and dynamic programming.

A multi-dimensional array of numbers that is the fundamental data structure for deep learning, generalizing scalars, vectors, and matrices to any number of dimensions. Neural networks compute by transforming tensors.

The NLP task of producing a shorter version of a text that preserves its key information. Extractive methods select existing sentences, while abstractive methods generate new phrasing.

The generative task of producing an image that matches a natural-language description. Modern systems use diffusion or transformer models trained on paired image-text data.

The technology that synthesizes natural-sounding spoken audio from written text. Neural approaches have made synthesized voices increasingly lifelike.

A weighting scheme that scores a word in a document by how often it appears there, scaled down by how common it is across all documents. It highlights words that are distinctive to a document.

The number of requests or tokens a model or system can process per unit of time. It measures capacity and is often traded off against latency.

Predicting future values of a quantity based on its past observations ordered in time, accounting for trends, seasonality, and noise. It is used in demand planning, finance, and operations.

The unit of text a language model reads and generates, often a word fragment; models process and are priced by counts of these tokens.

The step that breaks raw text into tokens the model can process, mapping words and word-pieces to numeric ids.

A capability letting a language model invoke external functions or APIs, such as search or calculations, by emitting a structured request the surrounding program runs.

A text-generation method that restricts each step's choices to the k most likely tokens and then samples among them. It adds controlled randomness while avoiding very unlikely words.

A generation method that samples the next token from the smallest set of choices whose probabilities add up to a threshold p, balancing coherence and variety.

An unsupervised NLP technique that discovers abstract themes running through a collection of documents by grouping words that tend to co-occur. It helps organize and summarize large text corpora.

A tensor processing unit, a specialized chip designed specifically to accelerate the tensor operations used in machine learning. It offers high throughput for training and inference of large models.

Dividing data into three parts: one to train the model, one to tune settings, and a held-out set to fairly estimate final performance.

The process of feeding data to an algorithm so it adjusts a model's parameters to reduce error and improve at a task.

Reusing knowledge from a model trained on one task as a starting point for a related task, cutting the data and compute needed.

A neural network architecture built around attention that processes sequences in parallel; it underpins nearly all modern large language models.

A prompting and search framework where a model explores multiple branching reasoning steps like a tree, evaluating and backtracking among them. It extends chain-of-thought reasoning to problems requiring exploration.

A proposed test of machine intelligence in which a human judge converses with a machine and a human and tries to tell them apart; a machine passes if the judge cannot reliably distinguish it. It shaped early thinking about artificial intelligence.

A false positive: rejecting a true null hypothesis, or flagging something as present when it is not. Its rate is controlled by the significance level of a test.

A false negative: failing to reject a false null hypothesis, or missing something that is actually present. Reducing it usually increases the risk of Type I errors.

A convolutional architecture with a symmetric encoder-decoder shape and skip connections, originally designed for biomedical image segmentation. It is also widely used as the backbone of image diffusion models.

A dimensionality-reduction and visualization technique that preserves both local and some global structure of high-dimensional data, often faster than t-SNE. It is widely used to map embeddings into two dimensions.

When a model is too simple to capture the real patterns in the data, performing poorly on both the training set and new inputs.

A theoretical result stating that a feedforward network with a single hidden layer and enough neurons can approximate any continuous function to arbitrary accuracy. It explains why neural networks are so expressive, though it says nothing about learnability.

Training on data with no labels, letting the model discover structure on its own, such as grouping similar items or finding underlying patterns.

In reinforcement learning, a function estimating the expected long-term reward of being in a state, or of taking an action in a state, under a given policy. It guides the agent toward high-reward behavior.

A training problem in deep networks where gradients shrink toward zero as they propagate backward, stalling learning in the earliest layers.

A statistical measure of how spread out a set of values is around their mean. In the bias-variance framing it also refers to how much a model's predictions change with different training data.

A generative autoencoder that learns a probabilistic latent space and can generate new data by sampling from it. It encodes inputs as distributions rather than fixed points, enabling smooth interpolation and generation.

A database that stores embeddings and finds the most similar ones to a query vector quickly, the backbone of semantic search and retrieval-augmented generation.

An image model that applies the transformer architecture to sequences of image patches instead of using convolutions. It has matched or exceeded convolutional networks on large-scale image tasks.

A multimodal model that understands images and text jointly, able to describe pictures, answer questions about them, or follow instructions referencing visuals.

The fixed set of tokens a language model knows and can read or generate, established during tokenizer training. Its size affects sequence length, memory, and how well rare words are handled.

A training technique that starts with a very small learning rate and gradually increases it over the first steps before applying the main schedule. It stabilizes early training, especially for large transformer models.

Embedding a detectable signal into AI-generated text, images, or audio so its machine origin can later be identified. It is one proposed tool for tracing and moderating synthetic content.

A regularization technique that shrinks model weights slightly at each update by adding a penalty on their magnitude, discouraging overly large parameters and reducing overfitting. It is closely related to L2 regularization.

The choice of starting values for a neural network's weights before training begins. Good initialization schemes keep signal and gradient magnitudes stable across layers and are essential for training deep networks.

The learned numeric values inside a model that determine how strongly inputs influence outputs; training is the process of finding good weights.

A representation that maps each word to a dense vector of numbers such that words with similar meanings sit close together in the space. It replaced sparse one-hot representations in modern NLP.

An early, influential technique for learning word embeddings by training a shallow network to predict words from their context or vice versa. It popularized the idea that word meaning can be captured by vector arithmetic.

A subword tokenization method that builds a vocabulary by choosing merges that most increase the likelihood of the training text. It is used by models such as BERT.

A weight initialization scheme that sets the initial scale based on the number of input and output units in a layer to keep activation variance stable. It works well with sigmoid and tanh activations.

A widely used, optimized implementation of gradient-boosted decision trees known for speed and strong performance on structured, tabular data. It adds regularization and engineering optimizations to standard boosting.

A family of real-time object detection models that predict bounding boxes and classes in a single pass over the image, prioritizing speed. The name stands for You Only Look Once.

A distributed training technique that shards a model's parameters, gradients, and optimizer states across devices to remove memory duplication, enabling training of very large models. It underlies popular large-scale training libraries.

A prompting trick that elicits step-by-step reasoning from a model without worked examples, often by simply adding an instruction like think step by step. It improves reasoning on many tasks at no extra data cost.

Asking a model to perform a task from instructions alone, with no examples provided, relying on knowledge gained during pretraining.

A class that cannot be instantiated directly and is meant to be subclassed, often declaring methods without implementations. It defines a common interface and shared behavior for its descendants.

A creational design pattern that provides an interface for creating families of related objects without specifying their concrete classes. It keeps a group of products consistent with one another.

A tree representation of a program's syntactic structure that omits superfluous details like punctuation and focuses on meaningful constructs. Compilers and tools operate on the AST during later analysis and transformation.

Exposing only the essential behavior of a component while hiding its implementation details, letting callers work at a higher conceptual level.

When the underlying details an abstraction was meant to hide surface and force callers to understand them, undermining the benefit of the abstraction.

Testing that validates whether a system meets business requirements and is ready for delivery, often performed from the user's perspective. Passing it signals the software satisfies the agreed criteria.

A keyword such as public, private, or protected that controls the visibility and reachability of a class member. Access modifiers enforce encapsulation boundaries.

A method that reads or writes the value of a private field, commonly called a getter or setter. Accessors let a class control and validate access to its internal state.

An acronym for the four guarantees of reliable database transactions: Atomicity, Consistency, Isolation, and Durability. Together they ensure transactions behave predictably even amid failures and concurrency.

A concurrency model in which independent actors each own private state and communicate only by sending asynchronous messages. Avoiding shared memory sidesteps many locking problems.

A structural design pattern that wraps an existing interface with a new one so incompatible classes can work together. It acts as a translator between two systems.

A graph representation that stores, for each node, a list of the nodes it connects to. It is memory-efficient for sparse graphs compared with an adjacency matrix.

A graph representation using a two-dimensional grid where each cell indicates whether an edge exists between two nodes. It allows constant-time edge checks but uses space proportional to the square of the node count.

An iterative approach to software development that delivers working increments in short cycles, favoring collaboration, feedback, and adaptation over fixed long-range plans.

The compilation of source or intermediate code into native machine code before execution rather than during it. It yields fast startup and predictable performance at the cost of build-time work.

A composite type formed by combining other types, typically as a sum of alternatives (variants) or a product of fields. It pairs naturally with pattern matching to model structured data.

A finite, well-defined sequence of steps that transforms input into a desired output, judged by correctness and by its time and space efficiency.

A method of analyzing the average cost per operation over a sequence of operations, even when individual operations occasionally cost much more. It shows that expensive steps are rare enough to keep the average low.

Application Programming Interface: a defined contract of operations, inputs, and outputs through which one piece of software requests services from another.

A server that acts as a single entry point for a set of backend services, handling concerns such as routing, authentication, rate limiting, and aggregation. It simplifies client access to a microservices system.

A contiguous, indexed collection of elements of the same type offering constant-time access by position, foundational to most other data structures.

A file produced by the build process, such as a compiled binary, library, or container image, that can be deployed or distributed. Artifacts are often stored and versioned in a repository.

A paradigm that isolates cross-cutting concerns such as logging or security into separate modules called aspects, which are woven into the main code at defined points. This keeps core logic free of repetitive boilerplate.

A tool that translates assembly language, a human-readable form of machine instructions, into the corresponding machine code. Each assembly instruction typically maps to a single processor operation.

A statement in code or a test that declares a condition expected to be true, raising an error if it is not. Assertions verify test outcomes and document assumptions in production code.

A collection that maps unique keys to values, allowing values to be retrieved by key rather than by position. It is commonly implemented with hash tables or balanced trees.

Language syntax for writing non-blocking asynchronous code in a sequential style; await pauses a coroutine until a pending operation completes without blocking the thread.

An operation that completes entirely as a single indivisible step, so other threads never observe a partial result. Atomics enable safe lock-free updates of shared variables.

A self-balancing tree that keeps data sorted and allows nodes to have many children, minimizing disk reads for large datasets. It is widely used in databases and file systems.

A prioritized list of work items, features, and fixes that a team may take on in the future. The product backlog is refined and reordered over time as priorities and understanding change.

A flow-control mechanism in which a slow consumer signals a fast producer to slow down, preventing unbounded buffering and resource exhaustion. It keeps streaming and reactive systems stable under load.

An algorithmic technique that incrementally builds candidate solutions and abandons a candidate as soon as it cannot lead to a valid solution. It is used for constraint problems like puzzles and pathfinding.

A notation for expressing the grammar of a language as a set of production rules with nonterminals and terminals. It and its extended variant are widely used to define programming language syntax.

A synchronization point at which threads wait until all participating threads have reached it before any proceeds. Barriers coordinate phases of parallel computation.

The execution of a series of jobs on a collection of data all at once, without user interaction, typically scheduled to run when convenient. It contrasts with real-time or interactive processing.

A development practice that expresses requirements as concrete examples of behavior in a readable format shared by developers, testers, and business stakeholders. It builds on test-driven development with a focus on communication.

A software system that lacks a discernible architecture, having grown haphazardly into a tangle of shared state and ad hoc dependencies. It is a widely observed anti-pattern in long-lived codebases.

Two conventions for ordering the bytes of a multi-byte value in memory: big-endian stores the most significant byte first, little-endian the least significant first. Mismatched endianness must be handled when systems exchange binary data.

A notation describing how an algorithm's running time or memory grows relative to input size in the worst case, ignoring constants, e.g. O(n) linear or O(n log n).

A search algorithm that finds a target in a sorted array by repeatedly halving the search interval based on comparison with the middle element. It runs in logarithmic time.

A binary tree in which each node's left subtree holds smaller keys and right subtree holds larger keys, enabling efficient ordered search, insertion, and deletion. Performance degrades to linear time if the tree becomes unbalanced.

A tree data structure in which each node has at most two children, referred to as the left and right child. It is the foundation for many specialized trees and traversal algorithms.

A debugging technique, automated in Git, that performs a binary search through commit history to find the commit that introduced a bug. It quickly narrows down the culprit among many commits.

An operation that manipulates the individual bits of a value, including AND, OR, XOR, NOT, and bit shifts. Bitwise operations are used for flags, low-level protocols, and performance-critical code.

A version control command that annotates each line of a file with the commit and author that last changed it. It helps trace when and why a particular line was introduced.

A space-efficient probabilistic data structure that tests whether an element is in a set, allowing false positives but never false negatives. It trades a small error rate for very low memory use.

A release strategy that runs two identical production environments, sending live traffic to one while deploying and testing the other, then switching over. It enables near-zero-downtime releases and quick rollback.

Repetitive, structurally required code that must be written to satisfy a language or framework but carries little unique logic of its own.

A primitive data type with exactly two values, true and false, used for logical conditions and control flow decisions.

In compilers, the process of writing a compiler in the language it compiles, using an existing compiler to build the first version and then using that to compile itself. More generally it means a self-starting process that needs minimal external input.

In ownership-based memory models, the temporary use of a value through a reference without taking ownership, subject to rules that prevent conflicting access. It enables safe sharing while the compiler enforces validity.

In domain-driven design, an explicit boundary within which a particular domain model and its terms apply consistently. Different bounded contexts can define the same term differently without conflict.

Wrapping a primitive value in an object so it can be used where an object reference is required; unboxing is the reverse operation. It bridges value types and reference-based collections at some performance cost.

A movable named line of development that diverges from the main history, letting you work on features or fixes in isolation before merging back.

A graph traversal algorithm that explores all neighbors at the current depth before moving to the next level, typically using a queue. It finds the shortest path in unweighted graphs.

A structural design pattern that separates an abstraction from its implementation so the two can vary independently. It replaces a rigid class hierarchy with composition.

A simple sorting algorithm that repeatedly steps through a list, swapping adjacent out-of-order elements until no swaps are needed. It is easy to understand but inefficient for large data with quadratic time.

A defect where a program writes data past the end of an allocated buffer, corrupting adjacent memory. Buffer overflows can crash programs and are a classic security vulnerability.

The process of transforming source code and assets into a runnable or distributable artifact, typically involving compilation, bundling, and packaging steps.

A creational design pattern that constructs a complex object step by step, separating the construction process from the final representation. It is useful when an object has many optional parts.

A visual chart that tracks remaining work against time over an iteration or project, sloping toward zero as work is completed. It helps teams see whether they are on pace to finish.

A compact, portable intermediate instruction set produced by a compiler and executed by a virtual machine, sitting between source code and native machine code.

The process of removing or updating cached entries when the underlying data changes so the cache does not serve stale results. It is famously one of the harder problems in computer science.

Storing copies of frequently used data in a fast-access layer to avoid repeated expensive computation or retrieval. Effective caching improves performance but introduces the challenge of keeping data fresh.

A function passed to another routine to be invoked later when an event occurs or an operation completes, a core pattern for asynchronous and event-driven code.

A deployment strategy that rolls out a change to a small subset of users first and monitors it before expanding to everyone. It limits the blast radius of a problematic release.

A principle stating that a distributed data store can guarantee at most two of consistency, availability, and partition tolerance at the same time. It frames the trade-offs designers face during network partitions.

A behavioral design pattern that passes a request along a chain of handlers, each deciding to process it or forward it to the next. It decouples senders from receivers.

A Git operation that applies the changes from a specific commit onto the current branch without merging the entire source branch. It selectively transplants individual commits between branches.

A resilience pattern that stops calls to a failing service after errors cross a threshold, quickly returning failures instead of waiting, then periodically testing for recovery. It prevents cascading failures across a system.

A blueprint that defines the data (fields) and behavior (methods) shared by objects of a type; instances created from it are objects.

A distributed architecture in which client programs request services or resources from centralized server programs that provide them. It underlies most web and networked applications.

The operation of creating a complete local copy of a remote repository, including its files and full history. It is typically how a developer starts working on an existing project.

A function bundled with references to the variables from the scope in which it was defined, so it can access that captured state even after the outer function has returned.

A metric of how much source code is exercised by the test suite, useful for spotting untested areas but not a guarantee that the tested code is correct.

A period before a release during which no new changes may be merged except critical fixes, to stabilize the build for testing and shipping. It reduces the risk of late-introduced defects.

The practice of having peers examine proposed changes before merge to catch defects, share knowledge, and keep code consistent with team standards.

A surface indication in code that often corresponds to a deeper design problem, such as duplicated logic or an overly long method. Smells are hints to investigate, not certain defects.

How closely the responsibilities within a single module belong together; high cohesion means a module does one thing well, aiding clarity and reuse.

A behavioral design pattern that encapsulates a request as an object, allowing requests to be queued, logged, or undone. It separates the object issuing a request from the one performing it.

A recorded snapshot of staged changes in version control, identified by a hash and carrying a message, forming one point in the project's history.

A program that translates source code in one language into another form, typically machine code or bytecode, ahead of execution, reporting errors before it runs.

A structural design pattern that lets clients treat individual objects and compositions of objects uniformly through a common interface. It naturally models tree structures such as file systems.

Structuring a program so multiple tasks make progress in overlapping time periods, coordinating shared resources; distinct from truly simultaneous parallel execution.

A defect arising from the interaction of concurrent tasks, such as race conditions, deadlocks, or livelocks, notorious for being intermittent and hard to reproduce.

The set of techniques that coordinate simultaneous operations on shared data so they produce correct results without interfering. It spans locking, versioning, and transaction isolation.

The overall strategy a language or system uses to structure concurrent work, such as threads and locks, actors and message passing, or async event loops.

A low-level building block for coordinating threads, such as locks, semaphores, condition variables, and atomics, that higher-level abstractions are built upon.

Concurrency is dealing with many tasks at once by interleaving progress; parallelism is doing many tasks at the same instant on multiple cores. One is structure, the other is execution.

A synchronization primitive that lets threads wait until a particular condition becomes true and be signaled when it changes. It is used with a lock to coordinate cooperating threads.

A control structure that runs different branches of code depending on whether a boolean expression is true, typically expressed with if/else statements.

A named value that is fixed at definition and cannot be reassigned, signaling intent and preventing accidental modification of values meant to stay stable.

A special method that initializes a new object, setting up its initial state when the object is created. Many languages call it automatically during instantiation.

Packaging an application together with its dependencies into a lightweight, isolated container that runs consistently across environments. Containers share the host kernel, making them more efficient than full virtual machines.

A geographically distributed network of servers that caches and delivers web content from locations near users to reduce latency. It offloads traffic from origin servers and improves reliability.

The process by which a client and server agree on the format of a response, such as JSON or XML, using request headers like Accept. It lets one endpoint serve multiple representations.

The act of saving one thread or process's state and loading another's so a single CPU can run multiple tasks. Frequent context switches add overhead that can hurt performance.

A representation of the remaining steps of a computation at a given point, which can be saved and invoked later. Continuations underlie advanced control flow such as coroutines and exceptions.

Extending continuous integration so that every validated change is automatically prepared for release, keeping the software in a deployable state at all times.

A practice in which every change that passes automated tests is released to production automatically, without manual approval. It shortens the path from commit to live software.

The practice of frequently merging developers' work into a shared branch where automated builds and tests run on every change to catch breakage early.

The ongoing collection of performance data such as CPU and memory usage from running software to find bottlenecks and regressions over time. It gives insight into how code behaves under real workloads.

A function that can suspend its execution and later resume from where it left off, cooperatively yielding control. Coroutines support lightweight concurrency and generators.

The degree of interdependence between modules; loose coupling means components can change independently, while tight coupling makes changes cascade and testing harder.

Rules that govern how subtyping between component types relates to subtyping of container or function types. Covariance preserves the direction of subtyping while contravariance reverses it.

Command Query Responsibility Segregation, an architectural pattern that separates the models used to update data from those used to read it. It can optimize each path independently at the cost of added complexity.

A region of code that accesses shared resources and must not be executed by more than one thread at a time. Synchronization primitives protect critical sections to prevent race conditions.

An acronym for the four basic operations on persistent data: Create, Read, Update, and Delete. Most data-driven applications and APIs are built around these operations.

Transforming a function that takes multiple arguments into a chain of functions each taking a single argument. It enables partial application and composition in functional programming.

A pointer that references memory which has already been freed or is no longer valid. Dereferencing it causes undefined behavior and is a common source of crashes and security bugs.

A way of organizing and storing data so it can be accessed and modified efficiently, such as arrays, linked lists, stacks, queues, trees, hash maps, and graphs.

A classification that tells the language what kind of value a piece of data is (integer, string, boolean, etc.) and which operations are valid on it.

Code that is never executed or whose results are never used, such as an unreachable branch or an unused function. It adds clutter and maintenance burden and is a candidate for removal.

A standstill where two or more tasks each wait forever for a resource the other holds, so none can proceed; avoided with lock ordering or timeouts.

A tool that lets you pause a running program, step through code line by line, and inspect variables and the call stack to locate the cause of a defect.

A style expressing what result you want without specifying the step-by-step control flow, contrasted with imperative code that spells out how to compute it.

A syntax that wraps a function or class to modify or extend its behavior without changing its source, often marked with a special symbol. It is distinct from the decorator design pattern though inspired by it.

A structural design pattern that attaches new responsibilities to an object dynamically by wrapping it in decorator objects sharing its interface. It offers a flexible alternative to subclassing for extending behavior.

A copy of an object that also duplicates all objects it references, so the copy is fully independent of the original. Modifying the copy does not affect the source at any level of nesting.

A parameter that takes a predefined value when the caller does not supply one, making that argument optional. It simplifies function signatures for common cases.

A shared, explicit checklist of criteria a work item must satisfy to be considered complete, such as code reviewed, tested, and documented. It creates a common understanding of quality across the team.

External code, such as a library or module, that a program relies on to function. Managing versions and updates of dependencies is a central concern in modern software development.

A frustrating situation in which packages have conflicting or incompatible version requirements that are difficult to satisfy simultaneously. It commonly arises with complex transitive dependency graphs.

A design technique where an object receives its collaborators from the outside rather than constructing them itself, improving testability and decoupling.

The SOLID principle that high-level modules should depend on abstractions rather than on low-level concrete implementations, and abstractions should not depend on details. It reduces coupling between layers.

A graph traversal algorithm that explores as far as possible along each branch before backtracking, typically using a stack or recursion. It is used for cycle detection, topological sorting, and pathfinding.

A double-ended queue data structure that supports adding and removing elements from both the front and the back efficiently. It generalizes both stacks and queues.

A named, reusable solution to a commonly recurring design problem, such as Observer, Factory, or Strategy, providing shared vocabulary and proven structure.

A special method invoked when an object is destroyed, used to release resources such as memory or file handles. It is prominent in languages with deterministic object lifetimes.

A syntax that unpacks values from arrays or properties from objects into distinct variables in a single expression. It reduces boilerplate when extracting multiple fields.

A Git state in which HEAD points directly at a specific commit rather than at a branch, so new commits are not attached to any branch. Work done here can be lost unless a branch is created to keep it.

A culture and set of practices that unite software development and IT operations to shorten delivery cycles and improve reliability through automation and collaboration. It emphasizes continuous integration, delivery, and monitoring.

A representation of the differences between two versions of text or files, showing added, removed, and unchanged lines. Version control tools use diffs to display and store changes.

A graph algorithm that computes the shortest paths from a source node to all others in a graph with non-negative edge weights. It repeatedly selects the closest unvisited node using a priority queue.

A directed graph with no cycles, meaning no path returns to its starting node. DAGs model dependencies, scheduling, and version histories.

A version control model in which every developer holds a complete copy of the repository and its history, enabling offline work and flexible collaboration. Git and Mercurial are examples.

An algorithmic paradigm that breaks a problem into smaller subproblems of the same type, solves them recursively, and combines their results. Merge sort and quicksort are classic examples.

An approach to software design that models software closely on the business domain, using a shared ubiquitous language between developers and domain experts. It emphasizes bounded contexts and rich domain models.

A programming or configuration language specialized to a particular application domain, such as SQL for queries or regular expressions for text patterns. It trades general power for expressiveness within its niche.

Don't Repeat Yourself: the principle that each piece of knowledge should have a single authoritative representation, so duplication doesn't drift out of sync.

A style of typing where an object's suitability is determined by the presence of required methods and properties rather than its declared type. The name comes from the saying that if it walks and quacks like a duck, it is treated as a duck.

An algorithmic technique that solves complex problems by breaking them into overlapping subproblems and storing subproblem results to avoid recomputation. It applies to problems with optimal substructure.

A type system that associates types with values at runtime rather than compile time, offering flexibility while deferring type errors until execution.

An evaluation strategy that computes expressions as soon as they are bound, before their results are known to be needed. It is the default in most imperative languages.

An input or condition at the extreme boundary of what code handles, such as empty input, maximum size, or zero, where defects commonly hide.

The object-oriented principle of hiding an object's internal state behind a public interface, so callers interact through methods rather than touching fields directly.

A test that drives the full system the way a real user would, validating a complete workflow across all layers from interface to storage.

A specific URL or address on a server where an API can be accessed to perform an operation or retrieve a resource. Each endpoint typically corresponds to a particular function of the API.

A type defining a fixed set of named constant values, making code that chooses among a small set of options clearer and less error-prone than raw literals.

A large body of work in agile planning that is too big to complete in a single iteration and is broken down into smaller user stories. It groups related stories toward a common goal.

A runtime construct that continuously pulls queued events and callbacks and dispatches them one at a time, enabling non-blocking single-threaded concurrency.

A pattern that stores the full sequence of state-changing events rather than just the current state, reconstructing state by replaying events. It provides a complete audit trail and supports temporal queries.

An architectural style in which components communicate by producing and reacting to events rather than calling each other directly. It supports loose coupling and asynchronous, scalable systems.

A paradigm in which the flow of the program is determined by events such as user actions, messages, or sensor outputs, handled by callbacks or event handlers. It is central to GUIs and networked servers.

A consistency model in distributed systems where replicas may temporarily differ but converge to the same value once updates stop. It trades immediate consistency for availability and partition tolerance.

A mechanism for responding to runtime errors by throwing an exception and catching it in an enclosing handler, separating error-recovery logic from normal flow.

A structural design pattern that provides a simplified, unified interface to a complex subsystem. It shields clients from the subsystem's internal complexity.

A creational design pattern that defines a method for creating objects, letting subclasses decide which concrete class to instantiate. It decouples client code from specific constructors.

A design principle in which a system detects errors early and halts immediately rather than continuing in a corrupt or uncertain state. Failing fast surfaces problems close to their cause and simplifies debugging.

A merge in which the target branch has no new commits since the source diverged, so its pointer simply advances to the source's tip without creating a merge commit. It produces a linear history.

A branch created to develop a specific feature or fix in isolation from the main line, merged back when complete. It keeps work-in-progress separate from stable code.

A configuration switch that turns a feature on or off at runtime without redeploying code, enabling gradual rollouts, experiments, and quick disabling of problematic features. It decouples deployment from release.

A lightweight, cooperatively scheduled unit of execution managed in user space rather than by the operating system. Fibers, also called green threads, allow many concurrent tasks with low overhead.

A named piece of data stored within an object, struct, or record. Fields together define the state that an instance holds.

A function that a language treats like any other value: it can be assigned to variables, passed as an argument, and returned from other functions. This capability underpins higher-order functions.

A test that sometimes passes and sometimes fails without any code change, often due to timing, ordering, or environmental dependencies. Flaky tests erode trust in a test suite and should be stabilized or removed.

A numeric type that represents real numbers with a fractional part using a mantissa and exponent, trading exactness for range and prone to rounding error.

An API design that allows method calls to be chained together in a readable, sentence-like sequence, with each method returning an object to continue the chain. Builders and query libraries often use this style.

A structural design pattern that minimizes memory use by sharing common state across many fine-grained objects instead of storing it in each. It separates intrinsic shared data from extrinsic per-instance data.

A personal server-side copy of a repository that lets a developer make changes independently of the original, often as a step toward contributing back via a pull request. It is a common workflow on hosting platforms.

A parallelism pattern that splits a task into subtasks that run concurrently (fork) and then combines their results once all finish (join). It suits recursively divisible workloads.

A set of production rules that defines which strings are valid in a language, used to specify programming language syntax. Parsers are built to recognize the language a grammar describes.

A structured scaffold that defines an application's shape and calls into the code you supply at defined extension points, inverting control compared to a library.

A reusable, named block of code that takes inputs (arguments), performs work, and optionally returns a result, letting you invoke the same logic from many places.

A paradigm that builds programs by composing pure functions and avoiding shared mutable state, favoring immutability, higher-order functions, and expressions over side effects.

In functional programming, a type that can be mapped over, providing a way to apply a function to values inside a wrapping context while preserving its structure. Lists and optional types are common functors.

An object representing the eventual result of an asynchronous computation that may not have completed yet. Code can wait on it or attach callbacks to run when the value is ready.

An automated testing technique that feeds a program large amounts of random, malformed, or unexpected input to uncover crashes and security flaws. It is effective at finding edge-case defects humans miss.

Automatic memory management that reclaims objects a program can no longer reach, freeing developers from manual deallocation at the cost of some runtime overhead.

A garbage collection strategy that groups objects by age and collects the young generation more frequently, exploiting the observation that most objects die young. It reduces collection overhead for typical workloads.

A type or function parameterized over types, letting one implementation work with many concrete types while preserving type safety, e.g. a list of T.

A distributed version-control system that tracks content as snapshots, giving every clone the full history and enabling fast local branching and merging.

A design approach in which a system continues to provide reduced functionality when some components fail rather than failing completely. Users retain core service even when parts are unavailable.

An approach that allows static and dynamic typing to coexist in one program, letting developers add type annotations incrementally. TypeScript and Python's optional type hints are examples.

A data structure of vertices connected by edges, modeling networks and relationships; traversed with algorithms like breadth-first and depth-first search.

A query language and runtime for APIs that lets clients request exactly the data they need in a single request against a typed schema. It reduces over-fetching and under-fetching compared with fixed REST endpoints.

An algorithmic strategy that makes the locally optimal choice at each step in hope of reaching a global optimum. It is efficient but only yields correct results for problems with the right structure.

A high-performance, open-source remote procedure call framework that uses HTTP/2 for transport and Protocol Buffers for serialization. It supports streaming and strongly typed service contracts.

An early check at the start of a function that handles an invalid or edge condition and returns immediately, reducing nesting in the main logic. Guard clauses make the primary code path clearer.

The situation where a hash function produces the same output for two different inputs. Hash tables resolve collisions with strategies such as chaining or open addressing.

A structure mapping keys to values via a hash function that computes an index, giving average constant-time lookup, insertion, and deletion.

The process of applying a hash function to map data of arbitrary size to fixed-size values used as indexes or fingerprints. It enables constant-time average lookups in hash tables.

Hypermedia as the Engine of Application State, a REST constraint in which responses include links guiding clients to the actions available next. It lets clients navigate an API dynamically rather than hard-coding URLs.

In Git, a reference that points to the currently checked-out commit, usually the tip of the current branch. Many commands operate relative to HEAD.

A tree-based data structure satisfying the heap property, where each parent is ordered relative to its children, enabling fast access to the minimum or maximum element. It is the usual basis for priority queues and heapsort.

A region of memory used for dynamic allocation whose objects persist until explicitly freed or garbage collected, allowing flexible lifetimes. Access is slower than the stack and mismanagement causes leaks or fragmentation.

A comparison-based sorting algorithm that builds a heap from the input and repeatedly extracts the largest element to produce sorted output. It runs in logarithmic-linear time and sorts in place.

A bug that changes behavior or disappears when one attempts to observe it, for example when adding logging or running under a debugger alters timing. Such bugs are notoriously difficult to reproduce and fix.

An architectural style that isolates core application logic from external systems by communicating through ports and adapters. It makes the core testable and independent of frameworks, databases, and UIs.

A function that takes other functions as arguments or returns a function, enabling composition patterns like map, filter, and reduce.

A small, urgent change applied to production to correct a critical defect outside the normal release cycle. Hotfixes are typically minimal and fast-tracked to restore service.

A verb in an HTTP request that indicates the desired action on a resource, such as GET to read, POST to create, PUT to replace, and DELETE to remove. Methods convey intent and have defined semantics around safety and idempotency.

A three-digit number in an HTTP response that indicates the outcome of a request, grouped into categories such as success, redirection, client error, and server error. Examples include 200 for success and 404 for not found.

Integrated Development Environment: an application combining a code editor, build tools, debugger, and other aids into one workspace to streamline development.

A property where performing an operation multiple times has the same effect as performing it once, making retries safe in the face of network or crash failures.

A unique token a client attaches to a request so the server can detect and ignore duplicate submissions, making retries safe for operations that change state.

An operation that can be applied repeatedly without changing the result beyond the first application, central to safe retries in distributed and networked systems.

Code written in the natural, conventional style of a particular language or framework, using its preferred patterns and features. Idiomatic code is easier for other practitioners of that ecosystem to read and maintain.

The property of a value or object that cannot be changed after creation; modifications produce new values instead, which simplifies reasoning and concurrency.

A style where code issues explicit sequential statements that change program state to achieve a result, describing how the computation proceeds step by step.

The practice of defining and provisioning computing infrastructure through machine-readable configuration files rather than manual setup. It makes environments repeatable, versioned, and reviewable.

A mechanism where a class derives fields and methods from a parent class, reusing and extending behavior; overused hierarchies are a common source of rigidity.

A sorting algorithm that builds the sorted list one element at a time by inserting each new element into its correct position among those already sorted. It performs well on small or nearly-sorted inputs.

A primitive numeric type representing whole numbers without a fractional part, bounded by a fixed bit width that determines its minimum and maximum values.

A test that exercises multiple components together to verify they interact correctly, catching mismatches that pass in isolation but fail when combined.

A contract declaring a set of method signatures without implementation, letting different types be used interchangeably as long as they satisfy the contract.

The SOLID principle that clients should not be forced to depend on methods they do not use, favoring many small, focused interfaces over one large one. It reduces unnecessary coupling.

A form of code between the source language and machine code that a compiler uses for analysis and optimization. A well-designed intermediate representation makes optimizations reusable across languages and targets.

A program that reads and executes source code directly, statement by statement, without a separate ahead-of-time compilation step, trading speed for flexibility.

A type that combines several types so that a value must satisfy all of them simultaneously, exposing the members of each. It is used to compose capabilities in structural type systems.

A condition that always holds true at particular points during a program's execution, such as a loop invariant that stays valid across every iteration. Invariants aid reasoning about and verifying correctness.

A design principle in which the framework or container, rather than the application code, controls the flow and the creation of dependencies. Dependency injection is a common way to achieve it.

An object that produces a sequence of values one at a time on demand, letting code traverse a collection without exposing its underlying structure.

JavaScript Object Notation, a lightweight, human-readable text format for representing structured data as key-value pairs and arrays. It is the most common data interchange format for web APIs.

A technique that compiles code to native machine instructions at runtime rather than ahead of time, often optimizing based on observed execution patterns. It blends the portability of interpretation with the speed of compilation.

An agile method that visualizes work as cards on a board flowing through stages and limits work in progress to improve flow and reveal bottlenecks. It emphasizes continuous delivery over fixed-length iterations.

An argument passed to a function by naming the parameter it maps to rather than relying on position. It improves readability and lets callers skip optional parameters.

Keep It Simple, Stupid: a design maxim favoring the simplest solution that meets the need, because simpler systems are easier to understand, test, and maintain.

An anonymous function defined inline without a name, often passed as an argument or returned from another function. It is a core building block of functional programming.

An architectural style that organizes code into horizontal layers such as presentation, business logic, and data access, each depending only on the one beneath it. It promotes separation of concerns.

An evaluation strategy that delays computing a value until it is actually needed, and may cache the result. It can improve performance and allows working with infinite data structures.

The first phase of compilation that scans source text and groups characters into meaningful tokens such as keywords, identifiers, and symbols. It is performed by a component called a lexer or tokenizer.

A collection of reusable code you call from your own program, keeping control of the overall flow; contrasted with a framework that calls your code.

A search algorithm that checks each element of a collection in sequence until it finds the target or exhausts the collection. It works on unsorted data but runs in linear time.

A linear data structure of nodes where each node holds a value and a reference to the next, allowing efficient insertion and removal without contiguous memory.

A tool that combines compiled object files and libraries into a single executable or library, resolving references between them. Linking can happen statically at build time or dynamically at load time.

A static-analysis tool that scans source code for stylistic issues, likely bugs, and rule violations without running it, enforcing consistency across a codebase.

The SOLID principle that objects of a subclass should be usable anywhere their superclass is expected without breaking correctness. Violations often indicate a flawed inheritance hierarchy.

A concurrency failure in which threads keep changing state in response to each other but make no actual progress. Unlike deadlock, the threads are active yet stuck.

The distribution of incoming requests across multiple servers to improve throughput, reduce latency, and avoid overloading any single machine. Load balancers can route by round-robin, least connections, or other strategies.

A form of performance testing that measures how a system behaves under expected or peak levels of concurrent usage. It reveals bottlenecks and confirms the system meets capacity goals.

A synchronization mechanism that grants exclusive access to a shared resource, forcing other threads to wait until it is released. Locks prevent data races but can cause contention or deadlock if misused.

A paradigm in which programs are expressed as sets of facts and rules, and computation proceeds by the system inferring answers to queries. Prolog is the best-known logic programming language.

A control structure that repeats a block of code while a condition holds or over the items of a collection, the primary tool for iteration.

The lowest-level program representation, consisting of binary instructions that a processor executes directly. Compilers and assemblers ultimately translate higher-level code into it.

A rule or construct that expands into other code before or during compilation, enabling reuse of syntactic patterns and metaprogramming. Macros can be simple text substitution or full syntactic transformations.

A garbage collection algorithm that marks all objects reachable from roots, then sweeps and frees everything unmarked. It handles reference cycles that reference counting cannot.

The process of transforming an object's in-memory representation into a format suitable for storage or transmission, and unmarshalling reverses it. It is closely related to serialization and often used across process or network boundaries.

A behavioral design pattern that centralizes communication between objects in a mediator, so components refer to the mediator instead of each other. It reduces tangled direct dependencies.

A behavioral design pattern that captures and externalizes an object's internal state so it can be restored later, without violating encapsulation. It is a common basis for undo functionality.

An optimization that caches a function's results by its inputs so repeated calls with the same arguments return the stored value instead of recomputing.

The condition where free memory is broken into many small, non-contiguous blocks, so large allocations fail even though enough total memory is free. It reduces effective usable memory over time.

A defect where a program keeps allocating memory it no longer needs but never releases, causing usage to grow until performance degrades or the process crashes.

Combining the changes from one branch into another, reconciling divergent histories; overlapping edits produce conflicts a developer must resolve.

A situation where two branches changed the same lines and version control cannot auto-combine them, requiring a person to choose or blend the changes.

A divide-and-conquer sorting algorithm that recursively splits a list in half, sorts each half, and merges them back together. It guarantees logarithmic-linear time and is stable.

An intermediary service that routes, transforms, and delivers messages between applications, often supporting queues and publish-subscribe topics. It centralizes messaging concerns away from individual services.

A component that holds messages sent between producers and consumers, allowing them to communicate asynchronously and buffer load. It decouples services and smooths spikes in demand.

The writing of code that generates, inspects, or modifies other code (or itself) at compile time or runtime. Macros, reflection, and code generation are common metaprogramming techniques.

A function defined as part of a class or object that operates on that object's data. Methods express the behavior associated with a type.

Defining multiple methods with the same name but different parameter lists, letting the compiler choose the right one based on the arguments. It supports the same operation over different input types.

Providing a subclass with its own implementation of a method already defined in a superclass, replacing the inherited behavior. It is a key part of inheritance-based polymorphism.

An architectural style that structures an application as a suite of small, independently deployable services communicating over a network. It enables independent scaling and deployment at the cost of operational complexity.

Software that sits between components or between an application and the operating system, providing common services such as request processing, authentication, or messaging. In web frameworks it often forms a pipeline around each request.

A subset of a connected weighted graph's edges that connects all nodes with the least total weight and no cycles. Algorithms like Kruskal's and Prim's compute it.

The smallest version of a product that delivers enough value to satisfy early users and generate feedback for further development. It validates ideas quickly while minimizing wasted effort.

A class or module that provides methods to be reused by other classes without being their parent through standard inheritance. Mixins add capabilities across unrelated class hierarchies.

A stand-in object that imitates a real dependency during testing, returning controlled responses and recording calls so a unit can be tested in isolation.

An architectural pattern that splits an application into a model holding data and logic, a view presenting it, and a controller handling input. Separating these roles improves maintainability and testability.

An architectural pattern related to MVC in which a view model exposes bindable data and commands to the view, mediating between the view and the model. It is popular in data-binding UI frameworks.

A self-contained unit of code that groups related functionality and controls what it exposes to the rest of a program. Modules promote organization and reuse.

A design pattern from functional programming that wraps values in a computational context and provides operations to chain functions over them while managing that context. Common examples handle optional values, errors, or side effects.

An architectural style in which an application is built and deployed as a single, tightly integrated unit. It is simple to start with but can become hard to scale and evolve as it grows.

A single repository that holds the code for many projects or services rather than splitting them across separate repositories. It simplifies shared code and atomic cross-project changes but requires tooling to scale.

A technique that evaluates the quality of a test suite by introducing small deliberate faults into the code and checking whether tests catch them. Surviving mutants reveal gaps in test coverage.

A mutual-exclusion lock that lets only one thread enter a critical section at a time, preventing concurrent access to shared data from corrupting it.

A named scope that groups related identifiers to prevent naming collisions between different parts of a program or libraries. It lets identical names coexist under different qualifiers.

A type system in which compatibility is based on explicit names and declarations, so two types are equivalent only if they share the same declared identity. It contrasts with structural typing.

The property of a computation whose output or behavior can vary between runs given the same input, often due to concurrency, timing, or randomness. Nondeterministic behavior makes bugs harder to reproduce.

A classification of decision problems that are in NP and at least as hard as every other NP problem, so a fast solution to one would solve them all. No known polynomial-time algorithm exists for them.

A reference that points to nothing; dereferencing it triggers a common runtime error, which is why many languages add null-safety or optional types.

A type that explicitly allows the absence of a value, usually represented as null or none, forcing code to account for the empty case. It helps prevent null-reference errors.

A concrete instance of a class or type that bundles state (data) with the operations that act on it, addressable and passed around at runtime.

A paradigm that structures code around objects bundling state and behavior, organized through encapsulation, inheritance, and polymorphism.

The degree to which the internal state of a system can be understood from its external outputs, typically through logs, metrics, and traces. High observability makes it easier to diagnose problems in production.

A behavioral design pattern in which subjects maintain a list of observers and notify them automatically of state changes. It underpins event handling and publish-subscribe systems.

A common logic error where a loop or index is iterated one time too many or too few, often from confusing inclusive and exclusive boundaries. It frequently arises at the edges of arrays and ranges.

The SOLID principle that software entities should be open for extension but closed for modification, so new behavior is added without altering existing tested code. It is often achieved through abstraction and polymorphism.

A specification for describing RESTful APIs in a standard, machine-readable format covering endpoints, parameters, and responses. Tools use it to generate documentation, client code, and server stubs.

Giving operators such as plus or equals custom meanings for user-defined types. It lets objects be used with familiar syntax, though overuse can obscure intent.

A concurrency strategy that lets transactions proceed without locking and checks for conflicts only at commit time, retrying if a conflict is detected. It performs well when contention is rare.

A memory-management model, most associated with Rust, in which each value has a single owner responsible for freeing it, and the value is dropped when its owner goes out of scope. It enforces memory safety at compile time without a garbage collector.

A tool that installs, updates, and resolves versioned dependencies for a project from a registry, keeping builds reproducible and dependency versions consistent.

The technique of splitting a large result set into smaller pages that clients fetch one at a time, using markers such as page numbers, offsets, or cursors. It limits response size and server load.

An agile practice in which two developers work together at one workstation, one writing code while the other reviews each line in real time. It improves code quality and spreads knowledge across the team.

Executing multiple computations literally at the same instant across multiple cores or machines to increase throughput, often layered on a concurrent design.

A named placeholder in a function's definition that receives an argument value when the function is called, letting one function operate on varying inputs.

The compiler phase that analyzes a stream of tokens against a grammar to determine its syntactic structure, usually producing a parse tree or abstract syntax tree. It reports syntax errors when the input does not conform.

Fixing some of a function's arguments to produce a new function that takes the remaining arguments. It specializes a general function for a specific context.

An argument-passing convention in which the function receives a reference to the caller's variable, so modifications inside the function affect the original. It avoids copying large data but can cause unexpected mutations.

An argument-passing convention in which a copy of the value is given to the function, so changes inside the function do not affect the caller's original. It contrasts with pass by reference.

A file that describes a set of changes in diff format so they can be applied to another copy of the code. Patches let changes be shared and reviewed independently of a repository.

A control mechanism that checks a value against a series of structural patterns and binds parts of it to variables when a pattern matches. It concisely destructures data and branches on its shape.

The body of a request or response that carries the actual data being transmitted, as opposed to headers and metadata. In an API call, the payload is typically formatted as JSON or XML.

A distributed architecture in which nodes act as both clients and servers, sharing resources directly without a central coordinator. It offers resilience and scalability but complicates coordination.

A concurrency strategy that locks data as soon as it is read or written so no other transaction can touch it until the lock is released. It avoids conflicts at the cost of reduced parallelism.

A variable that stores the memory address of another value rather than the value itself, enabling indirect access, shared references, and manual memory manipulation.

The ability for a single interface or call to operate on values of different types, so the same method name behaves appropriately for each concrete type.

Investing effort in making code faster before there is evidence that its performance actually matters, often complicating the code needlessly. It is famously cautioned against as a source of wasted effort and bugs.

An abstract data structure in which each element has a priority and the highest-priority element is served first, regardless of insertion order. It is often implemented with a heap.

A programming paradigm that structures a program as a sequence of procedures or routines that operate on shared data, executing steps in order. It emphasizes ordered subroutine calls over objects or mathematical functions.

An independent running instance of a program with its own isolated memory space; processes communicate through explicit channels rather than shared memory.

A classic synchronization scenario in which producers generate data into a shared buffer and consumers remove it, requiring coordination so producers wait when the buffer is full and consumers wait when it is empty. It illustrates the use of bounded buffers and condition variables.

An object representing the eventual result of an asynchronous operation, which can be pending, fulfilled, or rejected, and chained to sequence async work.

A class member that looks like a field to callers but is backed by accessor logic, so reading or writing it can run code. It combines the simplicity of a field with the control of a method.

A testing approach that specifies general properties expected to hold for all valid inputs and then checks them against many automatically generated cases. It can reveal edge cases that hand-written examples overlook.

A language-neutral, binary serialization format from Google that encodes structured data compactly using a predefined schema. It is smaller and faster than text formats and is commonly paired with gRPC.

A creational design pattern that creates new objects by copying an existing prototype instance rather than building from scratch. It is efficient when object creation is costly.

A structural design pattern that provides a stand-in object controlling access to another object, adding behavior such as lazy loading, caching, or access control. The proxy shares the real object's interface.

A messaging pattern in which publishers emit messages to topics without knowing the subscribers, and subscribers receive messages from topics they follow. It decouples producers from consumers of events.

A proposal to merge one branch into another on a hosting platform, packaging the diff for review, discussion, and automated checks before it lands.

A function whose output depends only on its inputs and which produces no side effects, making it deterministic, testable, and safe to cache or parallelize.

A first-in-first-out (FIFO) collection where elements are added at one end and removed from the other, used for buffering, scheduling, and breadth-first traversal.

A divide-and-conquer sorting algorithm that partitions elements around a pivot and recursively sorts the partitions. It is fast on average but can degrade to quadratic time with poor pivot choices.

A bug where the outcome depends on the unpredictable timing of concurrent operations accessing shared state, producing intermittent, hard-to-reproduce failures.

Resource Acquisition Is Initialization, a technique that ties a resource's lifetime to an object's scope so the resource is released automatically when the object is destroyed. It prevents leaks of memory, files, and locks in languages with deterministic destruction.

A control that caps how many requests a client may make to an API within a time window to protect the service from overload and abuse. Clients exceeding the limit typically receive an error until the window resets.

A paradigm oriented around asynchronous data streams and the automatic propagation of change, so that dependent values update when their inputs change. It is often used for event-driven and UI-heavy systems.

A synchronization primitive that allows multiple concurrent readers but only one exclusive writer at a time. It improves throughput for data that is read far more often than written.

Replaying a branch's commits on top of another base commit to produce a linear history, an alternative to merging that rewrites commit ancestry.

A technique where a function calls itself on a smaller subproblem until it hits a base case, often expressing tree, divide-and-conquer, or nested problems more naturally than loops.

The terminating condition in a recursive function that returns without recursing further, preventing infinite recursion and letting the call stack unwind.

A self-balancing binary search tree that colors nodes red or black and enforces rules to keep the tree approximately balanced. It guarantees logarithmic search, insert, and delete times.

The property of a function that can be safely interrupted and called again before the previous call finishes, because it does not rely on shared mutable state. Reentrant code is important for recursion and signal handlers.

Restructuring existing code to improve its readability, structure, or design without changing its external behavior, ideally under the protection of tests.

A memory management technique that tracks how many references point to an object and frees it when the count reaches zero. It reclaims memory promptly but cannot collect reference cycles on its own.

A type whose variables hold a reference to data stored elsewhere, so multiple variables can share and mutate the same underlying object. Objects and arrays are commonly reference types.