The landscape of Artificial Intelligence has evolved rapidly from classical statistical models to generative foundation architectures and autonomous multi-agent environments.
This cheat sheet organizes the core terminology across 5 historical evolution eras, 8 vertical technological layers, and 3 deep architectural disambiguations into a structured, production-tested reference guide.
Conceptual Evolution Ladder:
Foundations ➔ Classical ML ➔ Deep Learning ➔ Foundation LLMs ➔ Knowledge & Memory (RAG) ➔ Agent Core ➔ AgentOS & Runtime ➔ Governance (AgentOps) ➔ Scaled Applications
🔗 Related Cheat Sheets: AI Agent Skill Architecture & Development Cheat Sheet · AI Agent Harness Engineering Architecture Cheat Sheet · Core AI Algorithms Cheat Sheet · Claude Code CLI Cheat Sheet
1. Five Historical Eras of AI Paradigm Shifts
| Era | Timeframe | Paradigm Shift | Milestone Breakthrough | Representative Theories & Technologies |
|---|---|---|---|---|
| 1. Foundations | ~1940s - 1980s | Mathematical & Theoretical Origins | Established information measurement, probabilistic inference, and early connectionism | Information Theory (Shannon 1948), Bayesian Probability, Perceptrons, Convex Optimization |
| 2. Classical Statistical ML | ~1980s - 2010s | Data-Driven Representation & Statistical Learning | Shifted from hand-crafted expert rules to statistical optimization and shallow neural networks | Backpropagation (Rumelhart 1986), Reinforcement Learning (Sutton 1988), CNNs (LeCun 1998), SVMs (Vapnik 1995) |
| 3. Foundation Pretraining | ~2010s - 2020s | Self-Attention & Massive Pretraining | Transformer enabled hardware-parallel scaling; self-supervised training triggered emergent abilities | Transformer (Vaswani 2017), BERT / GPT paradigms, Scaling Laws (Kaplan 2020), RoPE (Su 2021) |
| 4. Agentic Engineering | ~2020s - Present | System Alignment, Tool Use & Sandboxed Runtime | Moved beyond static text completions to tool calling, deep reasoning chains, and closed-loop execution | RLHF (2022), Chain-of-Thought (2022), DPO (2023), vLLM / PagedAttention (2023), Model Context Protocol (2024) |
| 5. Industrial Autonomy | Present - Future | Always-on Digital Colleagues & Multi-Agent Swarms | Evolution from single-turn chat assistants to autonomous digital collaborators and physical embodiment | Multi-Agent Coordination, Embodied AI, AgentOS, Autonomous Runtimes |
2. Eight-Layer Technical Glossary Taxonomy
2.1 Layer 1: Foundations (Mathematical & Computational Base)
| Module | Concept / Term | Origin / Author | Core Definition & Technical Role |
|---|---|---|---|
| Information & Probability | Information Theory | 1948 (Shannon) | Defines Information Entropy, establishing the mathematical groundwork for data compression, cross-entropy loss, and channel capacity |
| Bayesian Inference | 1763 (Bayes) | Formulates prior-to-posterior probability updates, underpinning probabilistic graphical models and uncertainty quantification | |
| Statistical Learning Theory | 1995 (Vapnik) | Formalizes Empirical Risk Minimization (ERM), Structural Risk Minimization (SRM), and generalization error bounds | |
| Learning Paradigms | Supervised / Unsupervised / RL | 1980s - 1998 | The three primary branches of machine learning: labeled mapping, latent representation discovery, and trial-and-error environment interaction |
| Optimization (SGD / AdamW) | 1952 / 2017 (Loshchilov) | Weight updates along negative gradients; AdamW combines adaptive learning rates with decoupled weight decay as the modern deep learning standard | |
| Neural Foundations | Backpropagation | 1986 (Rumelhart et al.) | Efficiently computes partial derivatives across deep composite layers via multivariate chain rules |
| CNN / RNN | 1998 (LeCun) / 1990s | CNNs enforce spatial locality and translation invariance; RNNs introduce hidden state recurrences for temporal sequences | |
| Distributed Scaling | Parallelism (Data / Tensor / Pipeline) | 2010s - Present | Combines Data Parallelism (DDP), Tensor Splitting (Megatron-LM), and Pipeline Parallelism (DeepSpeed) to train frontier trillion-parameter models |
| Mixed-Precision Training | 2018 (Micikevicius) | Utilizes FP16, BF16, and FP8 formats to halve VRAM footprints while multiplying Tensor Core compute throughput |
2.2 Layer 2: Model Architecture (Designing General-Purpose Brains)
| Module | Concept / Term | Origin / Author | Core Definition & Technical Role |
|---|---|---|---|
| Attention Mechanics | Self-Attention | 2017 (Vaswani et al.) | Computes pairwise token affinity across the entire sequence: $\text{softmax}(QK^T/\sqrt{d_k})V$ |
| Multi-Head Attention (MHA) | 2017 (Vaswani et al.) | Projects embeddings into multiple orthogonal representation subspaces to capture syntax, semantics, and reference relationships simultaneously | |
| Feed-Forward Networks (FFN) | 2017 / 2020 (SwiGLU) | Two-layer projection with nonlinear activation (GELU, SwiGLU) providing nonlinear feature capacity | |
| Positional Encodings | Sinusoidal Position Encoding | 2017 (Vaswani et al.) | Injects static trigonometric frequencies directly into input embeddings; limited extrapolation capability |
| Rotary Position Embedding (RoPE) | 2021 (Su et al.) | Injects relative distance directly into the inner product of Query and Key vectors via complex rotation matrices | |
| Context Extrapolation (YaRN, LongRoPE) | 2023 - Present | Dynamically scales frequency bases (NTK-aware) to expand native 4k models to 128k - 1M token contexts | |
| Model Families | LLM / VLM / Reasoning Models | 2020s - Present | Large Language Models (LLM), Vision-Language Multimodal Models (VLM), and deep deliberate search reasoning models (e.g. OpenAI o-series) |
2.3 Layer 3: Model Intelligence & Capabilities
| Module | Concept / Term | Origin / Author | Core Definition & Technical Role |
|---|---|---|---|
| Pretraining & Tokenization | Next Token Prediction | 2018 (Radford et al.) | Autoregressive language modeling that trains models to internalize world knowledge by predicting subsequent tokens |
| Tokenization (BPE / SentencePiece) | 2016 (Sennrich) | Tokenizes natural language strings into optimal subword ID sequences balancing vocabulary size and sequence length | |
| Alignment & Tuning | Supervised Fine-Tuning (SFT) | 2021 - Present | Adapts pretrained foundational models to conversational and instruction-following formats using curated demonstrations |
| RLHF (Reinforcement Learning from Human Feedback) | 2022 (Ouyang et al.) | Uses an auxiliary reward model trained on human rankings with PPO optimization to align outputs with helpfulness and harmlessness | |
| DPO (Direct Preference Optimization) | 2023 (Rafailov et al.) | Derives an exact analytical formulation that aligns models on pairwise preference data without explicit reward modeling | |
| Emergent Capabilities | Scaling Laws | 2020 (Kaplan / Chinchilla) | Demonstrates that cross-entropy loss predictably drops as a power-law function of compute, dataset size, and parameter counts |
| In-Context Learning (Few-Shot) | 2020 (Brown et al.) | Enables models to perform novel downstream tasks conditioned purely on prompt demonstrations without updating network weights | |
| Reasoning Paradigms | Chain of Thought (CoT) | 2022 (Wei et al.) | Prompts models to generate intermediate reasoning trajectories, dramatically boosting math, logic, and coding performance |
| Test-Time Compute Scaling | 2024 - Present | Allocates dynamic token budgets during inference for tree-of-thought exploration, self-verification, and process reward evaluation | |
| Inference Serving | PagedAttention & KV-Cache (vLLM) | 2023 (Kwon et al.) | Caches precomputed Key/Value matrices using OS-inspired virtual memory paging to eradicate VRAM fragmentation in high-throughput serving |
2.4 Layer 4: Knowledge Augmentation & Memory
| Module | Concept / Term | Origin / Author | Core Definition & Technical Role |
|---|---|---|---|
| Data & Knowledge Ops | Synthetic Data Generation | 2023 - Present | Using frontier models to generate self-play trajectories, verification problems, and instruction datasets to bypass human data exhaustion |
| Vector Databases (Milvus, Pinecone) | 2020s - Present | Specialized storage engines indexing high-dimensional dense embeddings for sub-millisecond approximate nearest neighbor (ANN) retrieval | |
| Retrieval Architecture | Retrieval-Augmented Generation (RAG) | 2020 (Lewis et al.) | End-to-end pipeline: Chunking ➔ Embedding ➔ ANN Retrieval ➔ Reranking ➔ Grounding Alignment ➔ Response Generation |
| Memory Architectures | Four-Tier Cognitive Memory | 2023 - Present | Working Memory (call stack) · Short-Term Memory (active conversation context) · Long-Term Memory (vector/KV stores) · Episodic Memory (historical experience logs) |
| Context Strategy | Context Engineering | 2024 - Present | Disciplined structuring of system instructions, dynamic history pruning, tool contract injections, and context compaction algorithms |
2.5 Layer 5: Agent Core (Closed-Loop Autonomous Action)
| Module | Concept / Term | Origin / Author | Core Definition & Technical Role |
|---|---|---|---|
| Cognitive Loops | Observe-Plan-Act-Reflect (ReAct) | 2023 (Yao et al.) | The fundamental agentic loop: perceive environmental state ➔ plan next action ➔ invoke external tools ➔ observe feedback and refine |
| Planning & Swarms | Multi-Agent Orchestration | 2023 - Present | Hierarchical decomposition where orchestrator agents break complex objectives into DAGs executed by specialized subagents |
| Tool Calling | Function Calling / Tool Use | 2023 (OpenAI / Anthropic) | Native model capability to emit structured JSON arguments conforming to provided schemas to execute external APIs |
| Protocols | Model Context Protocol (MCP) | 2024 (Anthropic) | Open standard client-server protocol enabling standardized, secure discovery and invocation of local and remote tools, prompts, and resources |
| Agent-to-Agent Protocol (A2A) | 2025 (Industry Standard) | Standard protocol for peer-to-peer agent discovery, capability exchange (Agent Cards), task delegation, and cross-organization orchestration | |
| Execution Sandboxes | Agent Runtime & Sandboxing | 2024 - Present | Provides isolated execution environments (Docker, gVisor) with file snapshotting, permission gates, and process supervision |
2.6 Layer 6: Agent Engineering & Productionization
| Module | Concept / Term | Core Definition & Technical Role |
|---|---|---|
| Abstraction Layers | Tool vs Skill vs Workflow vs Template | Clear architectural division: atomic tools ➔ packaged domain skills ➔ deterministic workflows (see Section 3.1) |
| Platform Workspaces | Agent Frameworks & Harnesses | LangGraph, AutoGen, CrewAI, Antigravity, and Claude Code providing state machines, tracing, and multi-turn persistence |
| Operating Paradigms | AgentOS | An operating system designed for agents: schedules LLM token throughput, governs tool access rights, and manages virtual file sandboxes |
2.7 Layer 7: Governance & AgentOps
| Module | Concept / Term | Core Definition & Technical Role |
|---|---|---|
| Evaluation | LLM-as-a-Judge & SWE-bench | Automated scoring using frontier judge models alongside real-world environment benchmarks measuring end-to-end task completion |
| Telemetry | Tracing, Logging & Cost Attribution | Granular step-level tracing of thinking tokens, tool input/output latency, error distributions, and API billing allocations |
| Security & Guardrails | Prompt Injection Defense & Human-in-the-Loop | Sandboxed execution boundaries, prompt shields, and mandatory human operator approval gates for destructive system modifications |
2.8 Layer 8: Enterprise Applications & Physical Embodiment
| Domain | Focus Area | Representative Systems |
|---|---|---|
| Developer Productivity | Autonomous Coding Agents & Workspace Companions | Claude Code, Cursor, GitHub Copilot, Devin |
| Vertical Knowledge Work | High-Precision Financial, Legal & Clinical Diagnostics | Specialized agents with private document RAG, rigorous citation, and deterministic calculations |
| Embodied & Physical AI | Humanoid Robotics & Autonomous Scientific Discovery | Embodied foundation models (Figure, Tesla Optimus) and AI for Science (AlphaFold, autonomous synthesis labs) |
3. Deep Architectural Disambiguations
3.1 Disambiguation I: Tool vs. Skill vs. Workflow vs. Agent
| Dimension | Tool (Atomic Interface) | Skill (Capability Unit) | Workflow (Orchestrated Pipeline) | Agent (Autonomous Entity) |
|---|---|---|---|---|
| Nature | Discrete low-level API | Encapsulated domain SOP | Deterministic graph pipeline | Autonomous goal-seeking digital entity |
| Components | Single function, CLI command, or REST endpoint | Persona + Bound Tools + SOP steps + Output Schema | Predefined DAG with conditional branches and loops | Cognitive brain + memory + planning + tool orchestration |
| Decision Scope | Zero (Executes arguments passively) | Scoped (Selects when and how to invoke bound tools) | Deterministic (Follows strict step-by-step logic) | High (Breaks down novel goals, reflects, and retries) |
| Analogy | "A screwdriver or socket wrench" | "A licensed technician who knows how to replace a valve" | "An automated factory assembly conveyor" | "A general contractor responsible for delivering the entire renovation" |
3.2 Disambiguation II: Rotary Position Embedding (RoPE)
- Mathematical Core:
Rather than statically adding positional embeddings to token representations, RoPE applies a two-dimensional complex rotation transformation to Query and Key vectors: $$\langle \mathbf{R}{\Theta, m}^d \mathbf{q}, \mathbf{R}{\Theta, n}^d \mathbf{k} \rangle = g(\mathbf{q}, \mathbf{k}, m - n)$$ - Key Advantages:
- Relative Distance via Inner Products: The attention score naturally depends only on the relative offset $(m - n)$;
- Long-Range Decay: Attention weights decay gracefully as the relative token distance increases;
- Computational Efficiency: Implemented as pairwise chunked 2D vector multiplications without extra training or inference latency.
- Context Extrapolation: Techniques like NTK-aware Scaling, YaRN, and LongRoPE modulate high and low rotation frequencies, allowing models pretrained on 4k tokens to expand to 128k - 1M token contexts with minimal fine-tuning.
3.3 Disambiguation III: Interoperability Protocols — MCP vs. A2A
| Feature | Model Context Protocol (MCP) | Agent-to-Agent Protocol (A2A) |
|---|---|---|
| Originator | Anthropic (Open standard, 2024) | Industry Initiative / Google (2025) |
| Topology | Client - Server (C/S) | Peer-to-Peer / Mesh (P2P) |
| Core Problem | "How does an LLM securely connect to local files, databases, and developer tools?" | "How do two independent autonomous agents discover, negotiate, and delegate tasks to each other?" |
| Transport | JSON-RPC 2.0 over standard I/O (stdio) or Server-Sent Events (SSE) | HTTP/REST or gRPC with standardized Agent Cards and capability manifests |
| Use Case | Enabling Claude Code to read PostgreSQL schemas, invoke git diffs, and query Linear tickets | A travel planner agent delegating flight booking and car rental to specialized third-party agents |