AI & LLM Field Guide Concepts · Prompting · RAG · Agents · Security
how a prompt becomes a response, the tooling built around that loop, and where it actually breaks — one page, vendor-neutral
Scope note: model names, context-window sizes, and pricing change monthly — treat every specific number here as illustrative and VERIFY against current vendor docs. What's durable and what this page focuses on: the concepts (tokens, attention, RAG, agents), the failure modes (hallucination, injection), and the engineering trade-offs (prompting vs RAG vs fine-tuning) — those don't change every release cycle.
PROMPT · input + context
inference
MODEL · attention over tokens
sample
OUTPUT · response / tool call
01 Core LLM Concepts
TK
Tokens
The model's actual unit of input/output — not words. Roughly ¾ of a word in English (~4 chars/token). "Unbelievable" might be 1 token or split into "un" + "believ" + "able" depending on the tokenizer. Billing, context limits, and truncation all operate on tokens, not characters.
CW
Context Window
The maximum tokens (input + output combined, usually) the model can attend to in one call. Everything outside it is invisible to the model — there's no implicit memory across separate API calls unless you resend it yourself.
PM
Parameters
The learned weights inside the network (billions to trillions) — roughly the model's raw capacity, not a direct proxy for quality. Architecture, training data quality, and post-training (RLHF/fine-tuning) matter as much as raw size.
AR
Autoregressive Generation
The model predicts one token at a time, each new token conditioned on everything before it (prompt + its own output so far), then repeats. This is why streaming looks like typing and why early tokens can steer the whole rest of the answer.
TP
Temperature / Top-p / Top-k
Sampling controls, not "creativity dials" exactly. Temperature flattens/sharpens the probability distribution over next tokens; top-p (nucleus) samples from the smallest set of tokens whose cumulative probability ≥ p; top-k restricts to the k most likely tokens.
TR
Training vs Inference
Training (pretraining + post-training/RLHF) happens once, offline, and is what you pay per-token API pricing to avoid doing yourself. Inference is every single request you send — that's the only thing a prompt or RAG pipeline touches.
HL
Hallucination
The model produces fluent, confident, and wrong output — not a bug in the traditional sense, but an inherent property of a system that predicts plausible next tokens rather than looking up verified facts. See section 11.
GD
Grounding
Anchoring a response to a specific, verifiable source (retrieved documents, tool output, citations) instead of the model's parametric memory alone. RAG (section 06) is the most common grounding technique.
02 Anatomy of a Request — Prompt to Response
01
Tokenize & Embed
Text splits into tokens, each mapped to a learned vector, plus a positional encoding so the model knows word order (attention alone is order-blind).
02
Self-Attention × N Layers
Every token computes how much to "attend to" every other token in context — this is how a pronoun resolves to the right noun 500 tokens earlier. Stacked across dozens of transformer layers.
03
Sample & Stream
The final layer outputs a probability distribution over the entire vocabulary; sampling (temperature/top-p) picks the next token, which gets appended and fed back in — one token at a time, streamed to you as it's produced.
03 Model & Provider Landscape
AN
Anthropic — Claude
Family spans small/fast to large/deep-reasoning tiers. Strong on long-context reasoning, coding, and agentic tool use. This very page was written by a Claude model.
OA
OpenAI — GPT / o-series
GPT-class chat/multimodal models plus reasoning-optimized variants that spend more inference-time compute "thinking" before answering.
GO
Google — Gemini
Deeply multimodal (text/image/audio/video) with typically very large context windows; tiers usually split by speed/cost vs depth.
ME
Meta — Llama
Open-weight family — downloadable and self-hostable, the usual base for fine-tuning and on-prem/air-gapped deployments.
OW
Open-Weight Ecosystem
Mistral, DeepSeek, Qwen, and others ship openly-licensed weights — competitive quality, run anywhere, no per-token vendor lock-in.
XA
Others (xAI Grok, etc.)
The field moves monthly — check independent benchmarks (section 11) over marketing claims before picking a model for a specific task.
Dimension
Closed / Hosted API
Open-Weight / Self-Hosted
Cost model
Per-token, no infra to manage
Your own GPU/cloud compute cost, fixed regardless of volume
Data control
Leaves your environment (check vendor's data-retention/training terms)
Fully on-prem/air-gapped possible
Customization
Prompting, RAG, limited fine-tuning APIs
Full fine-tuning, quantization, architecture changes
Frontier capability
Usually leads on raw capability
Gap narrows constantly; often "good enough" for narrow tasks
Ops burden
None — vendor scales it
You own serving, scaling, and upgrades
CW
Context Window Tiers — What Actually Fits
SMALL
MEDIUM
LARGE
a chat turn or twoa long document / small codebasea whole book / large repo / hours of transcript
Exact token counts per tier shift every release — the useful mental model is what fits: a support ticket vs. a full PDF vs. an entire multi-file codebase. Bigger windows also cost more per call and can dilute attention ("lost in the middle") — bigger isn't automatically better for a small task.
04 Prompt Engineering Techniques
ZS
Zero-Shot
Just ask — no examples. Works well for tasks the model has clearly seen enormous amounts of during training (summarize, translate, explain).
FS
Few-Shot
Show 2-5 input→output examples in the prompt before the real task. Dramatically improves consistency of format and style for narrower, less-common tasks.
CoT
Chain-of-Thought
"Think step by step" (explicitly or via a reasoning-tuned model) before the final answer. Improves multi-step logic/math substantially — the reasoning itself is part of what conditions a better final token sequence.
SP
System / Role Prompting
A system prompt sets persistent behavior/persona/constraints for the whole conversation, separate from the user's turn-by-turn messages — the highest-leverage place to put durable instructions.
SO
Structured Output
Constrain output to JSON/XML/a schema (native structured-output modes, or "respond only with valid JSON matching this schema"). Essential the moment output feeds another program instead of a human.
PC
Prompt Chaining
Break a complex task into sequential smaller prompts, each output feeding the next (draft → critique → revise). Often outperforms one giant do-everything prompt.
Pitfall
Why It Bites
Vague instructions ("make it better")
No objective criteria — the model has nothing concrete to optimize toward, output quality becomes a coin flip
Burying the ask in a wall of context
Long, unstructured prompts dilute attention — put the actual instruction clearly, near the top or bottom, not lost in the middle
No output format specified
You get prose when you needed JSON — always state the exact shape you expect if you're going to parse the response
Assuming memory across separate calls
Stateless APIs remember nothing you didn't resend — "as I said earlier" only works if "earlier" is literally still in the context window
Treating one good output as proof
LLMs are non-deterministic (even at temperature 0, in practice) — test prompts across multiple runs and edge cases before shipping
Fighting the model instead of the prompt
If output is consistently wrong in the same way, the fix is almost always prompt/context/examples — not "try again and hope"
05 Retrieval-Augmented Generation (RAG)
1
Ingest & Chunk
Source documents (PDFs, wikis, tickets, code) are split into chunks small enough to embed meaningfully and retrieve precisely — too large dilutes relevance, too small loses context.
Ingest
2
Embed & Index
Each chunk is converted to a vector via an embedding model and stored in a vector database (section 07) alongside metadata (source, date, permissions).
Embed
3
Checkpoint — User Query Arrives
The user's question is embedded with the same embedding model used for ingestion — vectors from two different embedding models aren't comparable, a very common integration bug.
Checkpoint
4
Vector Similarity Search
The query vector is compared against the index (cosine similarity, dot product, or Euclidean distance) to find the top-K most semantically similar chunks — not keyword matches, meaning matches.
Retrieve
5
Augment the Prompt
Retrieved chunks are inserted into the prompt as context ("using only the following sources, answer…"), turning a general-knowledge question into a grounded, citable one.
Augment
6
Generate with Citations
The model answers primarily from the supplied context, ideally citing which chunk/source backed each claim — the mechanism that makes a RAG answer verifiable instead of a bare assertion.
Generate
✓
What RAG Actually Fixes
Stale/missing knowledge (the model's training cutoff), proprietary/private data the model never saw, and unverifiable answers — by grounding responses in retrievable, current, citable sources instead of parametric memory alone.
✗
What RAG Doesn't Fix
It doesn't change the model's underlying reasoning ability, style, or task-specific behavior — that's what fine-tuning is for (section 08). Bad chunking or a mismatched retriever also still produces confident-sounding wrong answers, just now with a plausible-looking source attached.
06 Embeddings & Vector Databases
EM
What an Embedding Is
A fixed-length numeric vector (often 384-3072 dimensions) representing meaning, not exact words — "car" and "automobile" land close together in vector space even though they share zero characters.
SM
Similarity Metrics
Cosine similarity (angle between vectors, most common for text), dot product (faster, magnitude-sensitive), Euclidean distance (straight-line distance). Must match what the embedding model was trained/optimized for.
AN
Approximate Nearest Neighbor (ANN)
Exact nearest-neighbor search doesn't scale past a few thousand vectors. Production vector DBs use ANN indexes (HNSW, IVF) — trading a small accuracy loss for orders-of-magnitude faster lookups.
Vector DB / Library
Type
Notes
pgvector
Postgres extension
Add vector search to a database you already run — no new system to operate
Chroma
Embedded / lightweight
Simple to start locally; common in small RAG prototypes
FAISS
Library (in-process)
Meta's ANN library — you build the serving layer around it yourself
Pinecone
Managed / hosted
Fully managed, scales without ops burden, usage-based cost
Hybrid search (combining vector similarity with traditional keyword/BM25 search) frequently outperforms pure vector search alone — exact terms like error codes, part numbers, or names don't always embed distinctly.
07 AI Agents, Tool Use & MCP
1
Observe
The agent receives the current state: the user's goal, conversation history, and results from any previous tool calls.
Observe
2
Think / Plan (ReAct)
The model reasons about what to do next — this is the "Reasoning" half of the ReAct (Reason + Act) pattern that underlies most agent frameworks.
Reason
3
Checkpoint — Tool / Function Call
Instead of answering in prose, the model emits a structured call (function name + JSON arguments) matching a schema it was given — a search, a database query, a code execution, an API request. This is what turns a chatbot into an agent that acts on the world.
Checkpoint
4
Act
The host application (not the model itself) actually executes the tool call — the model never directly touches your filesystem, database, or network; your code does, on its behalf.
Act
5
Observe Result & Loop
The tool's output is fed back into context, and the loop repeats — plan, act, observe — until the model decides the goal is met and returns a final answer instead of another tool call.
Loop
MCP
Model Context Protocol (MCP)
An open standard for connecting models to external tools/data sources through a common interface — instead of every app writing bespoke integrations for every tool, an MCP server exposes tools once and any MCP-compatible client (this very tool-use loop included) can use them.
MA
Multi-Agent Orchestration
Splitting work across specialized agents (a planner, a coder, a reviewer) coordinated by an orchestrator — trades single-call simplicity for better division of labor on genuinely complex, multi-step tasks. Adds latency and coordination overhead, so it's not free.
Excessive agency risk: an agent with broad tool access (file write, shell exec, send email, spend money) will eventually be steered — by a bad prompt, bad data, or a malicious input — into using that access somewhere you didn't intend. Scope tool permissions to the minimum the task needs. See section 09.
08 Customizing Model Behavior — Prompting vs RAG vs Fine-Tuning
PROMPTING
RAG
FINE-TUNING
PRETRAINING
cheapest, fastest, no data prepneeds a document corpus + pipelineneeds labeled examples + computeneeds a research budget most teams don't have
Approach
Best For
Not Good For
Prompting / few-shot
Fast iteration, general tasks, one-off changes
Deeply consistent style/format at scale, teaching genuinely new knowledge
RAG
Grounding in current/private/proprietary facts, citations
Changing how the model reasons, writes, or follows instructions
Adding brand-new factual knowledge reliably (it still hallucinates) — RAG does that better
Pretraining from scratch
Novel architectures/research, full control over training data
Almost everyone — the cost/expertise bar is enormous; default to the options above
Default order to try: prompt first (cheap, fast, reversible) → add RAG if the gap is missing/stale knowledge → fine-tune only if the gap is behavior (format, tone, task-specialization) that prompting genuinely can't nail after real effort. Most projects never need to go past RAG.
Every major provider's chat API follows the same shape: model name, a messages array with roles (system/user/assistant), and generation parameters. Swap the endpoint/auth header and the concept transfers directly.
PM
Common Parameters
temperature # 0 = deterministic-ish, 1+ = diverse
max_tokens # hard cap on output length
top_p # nucleus sampling threshold
stop # sequences that end generation early
stream # true = token-by-token SSE response
Set max_tokens deliberately — too low silently truncates mid-answer, too high just wastes budget on a short task, it doesn't force a longer answer.
KY
Key Handling
API keys are bearer credentials — never ship one in client-side/browser code or a public repo. Proxy calls through your own backend, use environment variables/secrets managers, and rotate on any suspected leak.
A key committed to a public repo will typically be found and abused within minutes by automated scrapers.
RT
Rate Limits & Retries
APIs enforce requests/tokens-per-minute limits. Implement exponential backoff on 429/5xx responses, and design for partial failure (a tool call or one step in a chain failing) rather than assuming every call succeeds.
10 Evaluation, Hallucination & Benchmarks
HL
Why Hallucination Happens
The model is optimized to produce the most plausible next token, not to consult a fact database. Absent grounding (RAG, tool use), a confident-sounding fabricated citation and a real one look identical in the output.
MG
Mitigations
RAG/grounding, requiring citations, lower temperature for factual tasks, asking the model to say "I don't know" when uncertain, and human review on high-stakes outputs. No single technique eliminates it entirely.
EV
Eval Approaches
Automated benchmarks (below) for broad capability, task-specific eval sets for your actual use case, and human review/red-teaming for anything customer-facing. Benchmark leaderboard rank ≠ "best for your specific task."
Benchmark
Measures
MMLU
Broad academic/professional knowledge across many subjects
Graduate-level science reasoning — hard to game via memorization
MMMU
Multimodal (text + image) reasoning
Chatbot Arena (LMSYS)
Head-to-head human preference voting between models
Benchmarks saturate and get gamed (train-on-test contamination) — cross-check multiple independent leaderboards, and where the decision matters, build a small eval set from your own real task examples.
11 AI / LLM Security — OWASP Top 10 for LLM Applications
Risk
What It Is
Primary Mitigation
LLM01 Prompt Injection
Attacker-crafted input (direct, or hidden in a retrieved document/webpage) overrides the system's intended instructions
Treat all retrieved/external content as untrusted data, not instructions; least-privilege tool access
LLM02 Insecure Output Handling
Model output is passed to a shell/DB/browser without validation, enabling injection downstream (XSS, SQLi, RCE)
Sanitize/validate LLM output exactly like any other untrusted user input
LLM03 Training Data Poisoning
Malicious data in training/fine-tuning corpus biases or backdoors model behavior
Vet data sources, especially for fine-tuning on external/scraped data
LLM04 Model Denial of Service
Resource-exhaustion attacks via extremely long/complex inputs or high-volume requests
Input length limits, rate limiting, cost/usage monitoring and alerts
LLM05 Supply Chain
Compromised pretrained models, datasets, or plugins/packages in the pipeline
Vet model/dataset provenance, pin and scan dependencies
LLM06 Sensitive Info Disclosure
Model reveals PII, secrets, or proprietary data memorized in training or leaked via context
Data minimization, output filtering, don't put secrets in prompts you don't fully control
LLM07 Insecure Plugin/Tool Design
A connected tool accepts unvalidated model-generated input and executes it with excess privilege
Strict schemas, input validation, and confirmation for destructive tool actions
LLM08 Excessive Agency
Agent granted more permissions/autonomy than the task needs, then manipulated into misusing them
Least-privilege scopes, human-in-the-loop for high-impact actions, hard action limits
LLM09 Overreliance
Users trust hallucinated/incorrect output without verification, especially in high-stakes decisions
Surface confidence/citations, human review gates on consequential outputs
LLM10 Model Theft
Unauthorized extraction/copying of proprietary model weights or distillation via API abuse
Access controls, rate limiting, query-pattern monitoring for extraction attempts
PI
Prompt Injection vs Jailbreak
Prompt injection: untrusted content hijacks the model into ignoring its actual instructions (often via a third-party document/webpage the model reads). Jailbreak: the user themselves crafts input to bypass the model's own safety training. Different threat actor, similar underlying weakness — treat both as an ongoing arms race, not a solved problem.
GD
Design Principle
Never let model output directly trigger an irreversible or high-privilege action without a validation layer in between — the model is a reasoning component, not a trusted authorization boundary. Same logic as never trusting client-side input in a web app.
12 MLOps & Glossary Quick Reference
Term
Meaning
RLHF
Reinforcement Learning from Human Feedback — post-training step that aligns raw model output with human preferences
Fine-tuning
Further training a pretrained model on a smaller, task-specific labeled dataset
LoRA / QLoRA
Low-Rank Adaptation — fine-tunes a small set of additional weights instead of the whole model, drastically cheaper
Quantization
Reducing weight precision (e.g. FP16 → INT4) to shrink model size/memory at a small quality cost
Distillation
Training a smaller "student" model to mimic a larger "teacher" model's outputs
MoE (Mixture of Experts)
Architecture where only a subset of sub-networks ("experts") activate per token — more capacity without proportional inference cost
KV Cache
Cached attention keys/values from previous tokens, reused instead of recomputed — the main reason streaming generation is fast per-token
Epoch / Batch
One full pass over training data / one group of examples processed together in one training step
Checkpoint
A saved snapshot of model weights at a point in training
GPU / TPU
Specialized parallel-compute hardware training and inference actually run on — the real bottleneck/cost driver behind every model
Multimodal
A model that accepts/produces more than one modality — text, images, audio, video — through one architecture
Context Rot / Lost-in-the-Middle
Degraded attention to information placed in the middle of a very long context, vs the start/end
13 Common Pitfalls When Building With AI
Pitfall
Why It Costs You
Shipping without an eval set
No way to know if a prompt/model change made things better or worse — you're flying blind on regressions
Trusting output as ground truth
Hallucination is inherent to the technology, not a rare edge case — verify anything consequential
No fallback for API failures
Rate limits, timeouts, and outages will happen — design graceful degradation, not a hard dependency
Giving an agent broad tool access "to be safe"
Backwards — broad access is the risk; scope tools to exactly what the task needs (see LLM08)
Ignoring token cost until the bill arrives
Context windows and output length compound fast at scale — monitor usage from day one, not after a surprise invoice
One giant prompt instead of chaining/tools
Complex multi-step tasks often do better broken into smaller, verifiable steps than one monolithic instruction
Treating a benchmark win as "best for my task"
Aggregate leaderboards don't reflect your specific domain, format, or failure tolerance — test on your own data