You've heard all the words.
Here's what they actually are — and what I build with them.
AI, LLM, tokens, embeddings, RAG, agents, LangChain, evals. This is a scroll-through that starts from zero — what each thing really is and how it works — and climbs to the systems I ship. Send it to someone who doesn't know AI, or read it end to end. It goes concept → how it works → what I build.
This field has died twice. Knowing why is how you keep your head now.
AI didn't arrive in 2022 — it arrived in 1956, over-promised, and froze twice when the promises outran the compute. The doom-and-gloom instinct people have today isn't paranoia; it's institutional memory. Here's the whole arc:
Rosenblatt's perceptron makes the New York Times promise machines that will "walk, talk, see, write." Minsky & Papert (1969) prove single-layer nets can't even learn XOR; the Lighthill report (1973) calls the field a failure. Funding evaporates. Lesson: a real idea, decades before its compute.
Hand-coded rule engines (XCON saved DEC ~$40M/yr) boom — then collapse: brittle rules, no learning, and the specialized LISP-machine market implodes in 1987. "AI" becomes a word you avoid on grant applications for a decade. Lesson: systems that can't learn can't scale.
Backprop is popularized (1986); LeCun reads ZIP codes with conv nets (1989); the internet quietly builds the training set, and gaming quietly builds the processor (GPUs). Nothing "arrives" — three curves grow: data, compute, and algorithms.
AlexNet (2012) crushes ImageNet on two gaming GPUs → "Attention Is All You Need" (2017) makes training parallel → scaling laws (2020) turn "bigger is better" into an equation → ChatGPT (2022) makes it a product. By 2026, ~57% of surveyed orgs run agents in production. The winters ended when the curves crossed the promises.
Both winters were caused by claims outrunning evidence. That's why everything I build ships with evals and measured numbers — "measured, not asserted" is the anti-winter discipline, applied at the scale of one engineer.
Most "AI" words are nested inside each other.
The jargon sounds like a pile of unrelated buzzwords. It isn't. Almost all of it fits in one picture: a set of Russian dolls, plus a short list of things you actually do with the innermost one.
Ask well, show examples. Cheapest lever. (§02)
Feed it your documents at question time. (§06)
Let it use tools and run a loop. (§07)
Retrain the weights for a fixed skill. Last resort.
It's autocomplete that read the internet.
An LLM does exactly one thing: given some text, it predicts the next chunk of text, over and over. That's it. Everything that feels intelligent — answering, coding, reasoning — is that one trick, run at enormous scale on almost everything ever written.
Models don't read words. They read tokens.
Before anything happens, your text is chopped into tokens — common chunks, often a word or a piece of one. Roughly four characters each. Everything the model does is counted, priced, and limited in tokens, so it's worth seeing them.
Under the hood — how the split is chosen
Every token becomes a point in space. Similar meanings sit close.
A token is turned into a long list of numbers — a vector — that places it in a "meaning space." The model learns this space so that related things land near each other. This one idea is the engine behind search, RAG, and recommendations.
Under the hood — what "close" actually means
"It" means nothing until the model decides what "it" points to.
Words change meaning based on their neighbors. Attention is the mechanism — the "T" in GPT — that lets every word look at every other word and decide which ones matter. It's how the model resolves "it," "this," "there." The 2017 paper was literally titled "Attention Is All You Need."
Under the hood — the actual mechanism
Now embeddings pay off: give the model your facts.
The model only knows what it was trained on — not your contracts or what changed today. So I embed your documents into that same meaning-space, and at question time retrieve the nearest chunks and hand them over. The model answers grounded in them, with citations. That's Retrieval-Augmented Generation.
Your question becomes a vector. Find the closest document chunks in embedding space — the exact "nearest neighbors" you hovered a moment ago. Feed those to the model as context.
# retrieve by embedding similarity, then ground docs = store.similarity_search(question, k=6) context = rerank(question, docs)[:3] answer = llm.invoke( GROUNDED_PROMPT.format(context=context, question=question) ) # "cite the source for every claim"
Prior Auth quotes the exact CMS regulation instead of guessing — in healthcare an uncited answer is worthless. Earnings Inspector reads a filing and surfaces what the press release buried. Same mechanism, two industries.
Give it actions, and let it run the loop.
A model that can only talk is a chatbot. Wire it to tools — send a text, query a database, file a form — and it can act. When a task needs several steps and memory, that's an agent: it reasons, acts, sees the result, and decides again. LangGraph is what I use to run that loop reliably — explicit state, checkpoints, and a record of every step. Step through one:
The state object is what the agent remembers across steps — and, two sections down, it's also what makes every step loggable.
I don't defend the brand — I answer with the trace. Running this loop by hand means rebuilding state, retries, and the real prize: observability into every step. LangGraph makes each step loggable — which is exactly what §09 needs.
The loop is powerful. Knowing when not to run it is the senior skill.
"Agent" isn't a yes/no — it's a dial, and where you set it should be a function of one variable: the cost of a wrong action. My rule: autonomy rises as the cost of error falls.
Code decides everything; the LLM only writes text. Right when every step is known. Not really an agent — and often the correct answer.
The model makes one decision per turn — which lane — from context and memory; deterministic workflows do the doing. Handles unscripted phrasing without ever holding the pen on real data. This is Meridian and TM GO.
One open-ended step inside a deterministic graph — budgeted, time-boxed, typed output. For genuinely exploratory subtasks ("research this denial") without giving up the rails.
The model plans, calls tools, re-plans, repeats. Maximum capability, minimum verifiability — every run is a different path, so you can't enumerate or regression-gate them. Never next to a payment tool.
A bounded turn is at most two model calls — router + writer — so spend scales with questions answered, not tokens wandered. The free loop's bill grows with its indecision.
The window isn't memory. Deciding what goes in it is a real discipline now.
Every model degrades as context grows — well before the window is full. So the question isn't "does it fit," it's "what has earned its place in this turn's window?" The field has settled on four moves:
Write: persist state outside the window (files, scratchpads, a checkpointer) instead of hauling the transcript. Select: pull in only what this turn needs — that's what RAG actually is: selection under a budget.
Compress: recency window + a running summary of everything older. Isolate: subagents get their own clean windows so one task's debris doesn't poison another's reasoning.
For tool-using agents, a 2026 study ("Less Context, Better Agents") found a small recency window plus a compact running summary beats hauling full history — and even beats fancy external memory. My builds now ship hard per-turn context budgets by default: a long chat should never quietly inflate every later call.
I don't build one-offs. I build these patterns.
Every project I ship is an instance of a reusable archetype — a shape of system I can stand up again for you. Here are the ones I've proven, each with the real build behind it.
Reads your corpus, answers with receipts
RAG + citation over regulated or dense documents. For when a wrong, uncited answer is unacceptable.
Does the work, not just the talk
Tool-using agent that takes real actions through SMS, APIs, and forms — with persistent memory across a long task.
Turns a document into a decision
Reads something long, scores it against a rubric, and surfaces what a human would've missed — tone, red flags, omissions.
Breaks agents before users do
RL-driven attack policies probing for prompt injection, tool misuse, jailbreaks. Red-teaming as a service.
Proves the thing actually works
Golden sets, regression tests, trace logging, cost tracking — the difference between a demo and production.
Ships inside a big org's workflow
Internal tools across business functions — RFP generation, knowledge-base connectors, EDGAR search — built to survive review.
A great agent still needs somewhere to live.
"The cloud" is just someone else's computers, rented by the minute. The real question is never which is best — it's which fits this job. There are three tiers, and the choice is usually made for you by one thing: where your data is allowed to be.
Azure · AWS · GCP
Full control, every service, enterprise compliance and data residency. Heavy to set up. Reach for it when: regulated data, an existing enterprise contract, or real scale.
Vercel · Render · Supabase · Fly
Managed, cheap to start, live in minutes — no VPC to provision. You trade control for speed. Reach for it when: a startup, a prototype, or you need to ship this week.
Model + inference hosts
The model itself is usually an API — OpenAI, Anthropic, or Azure OpenAI. Custom or open models run on Modal / Replicate; Streamlit for quick demos. Reach for it when: always — it sits on top of tier 1 or 2.
At PwC I shipped on Azure because banking clients required data residency and already had enterprise agreements — Azure OpenAI meant the model ran inside their tenant, which is often the whole reason a regulated org picks a cloud at all. For my own products I run Render + Supabase + Vercel — because I need to ship this week, not provision infrastructure. Same engineer, opposite choice, and I can explain the trade either direction.
Anyone can demo an agent once. I ship the 1,000th run.
The value isn't the demo — it's the run on the input nobody predicted, without burning money or going off the rails. Two disciplines: evaluation (measure whether it meets the bar) and monitoring (watch whether it still does in production, with a full trace of every step).
A golden set + assertions, run on every change so "improving" one case can't silently break another. Offline, measured, repeatable. My LangGraph Eval Harness.
A wrong answer still returns 200 OK, so I watch three layers: operational (latency, cost/run), quality (judge scores, thumbs, task success), and the trace above — the only way to answer "why did it do that."
pass@1 says it can work; production needs pass^k — correct on all k trials, because LLM routing is stochastic and one green run hides flakiness. Add paraphrase perturbations and injected tool failures and you have reliability's three axes.
Originals 100% pass^k · paraphrases 88% · provider-outage fallback floor 15/15 · cross-family judge (GPT grading Claude) 4/5 faithful. Best part: the harness found two real bugs — a paraphrase my router misroutes and an unsupported claim my writer slipped in. That's the eval earning its keep.
A bad production trace becomes a golden test — so that failure can never quietly return. Monitoring feeds evals feeds the next change. That's the whole discipline, and where most "AI demos" fall apart.
Everything above works. The problem is what it costs to run.
The quiet fact under the whole AI boom: it's staggeringly energy-hungry. Training a frontier model draws megawatts for weeks; then serving it to millions is a permanent, larger draw. Data centers are already a meaningful slice of global electricity — and the water to cool them is the constraint nobody tweets about.
For 40 years chips got faster almost for free — Moore's law. That's slowing. Transistors are now a few atoms wide; shrink them more and electrons leak straight through as wasted heat. Dennard scaling — power dropping as transistors shrank — already ended around 2005. The free lunch is over.
We stopped getting speed from smaller transistors and started getting it from more of them — more GPUs, more power, more heat. That's a spending race, not a physics win. The real question becomes: what if we computed a fundamentally different way?
If shrinking transistors is done, the next leap changes the architecture — or the physics itself.
People lump these together; they're really two different escapes. Some rearrange the chips — still silicon, radically different design. Others change the medium — compute with light, molecules, or quantum states. And one thing here isn't hardware at all. Honest labels below; the numbers are lab / vendor benchmarks on favorable workloads, not guarantees.
Neuromorphic — compute like a brain
Chips of spiking neurons that fire only on a signal, with memory and compute fused so data never shuttles back and forth. Intel Loihi 2, IBM NorthPole, SpiNNaker.
Analog in-memory — math inside the memory
Memristors do the matrix-multiply where the data already lives — killing the shuttle between memory and processor that burns most of the energy in a GPU.
Photonic — compute with light
Matrix multiplies happen as light interfering through a mesh — and light also moves data between chips far cheaper than copper. Lightmatter, Lightelligence.
Quantum — a different question entirely
The honest take, against the headlines: not faster AI and not an energy fix. It's a specialized co-processor for simulation, optimization, and sampling — used alongside classical, not instead of it.
Chemical & molecular
Compute in the physics itself — reaction-diffusion waves (Belousov–Zhabotinsky), DNA and molecular logic. Massively parallel, tiny energy, deeply strange.
Hyperdimensional computing isn't a substrate — it's a representation: encode meaning as huge vectors, and it runs beautifully on exactly the neuromorphic and in-memory hardware above. That's the layer I research — presented at the Science of Consciousness Conference.
The papers I'm actually reading, and the one-line take on each.
The winters section taught me to distrust vibes — including my own. This is my running reading log: what's genuinely state-of-the-art in making agents reliable, and where the field is arguing. Updated as I go — last pass Aug 2026.
ReliabilityBench — reliability = consistency (pass^k) + robustness to paraphrase + fault tolerance; the framework my eval_v2 implements. Beyond pass@1 — why single-run benchmarks systematically overstate production readiness.
Less Context, Better Agents — recency window + running summary wins; changed my defaults. Demand Paging for Context Windows — context as RAM, store as disk: the memory-hierarchy frame.
Verifiably Safe Tool Use — capability/trust labels on tools, guarantees over vibes. Guardrails as Scapegoats — model-side refusals are unfaithful; deterministic gates aren't. Academic backing for "the model proposes, code commits."
Bias mitigation in LLM-as-judge pipelines — never judge with the generator's own family, pairwise both orderings, explicit rubrics, calibrate on your own labels. My faithfulness judge follows all four.
Do We Still Need GraphRAG? — agentic (multi-round) search substantially closes the gap to GraphRAG on plain vector RAG; graphs still win deep multi-hop once their index cost is amortized. Is GraphRAG Needed? continues the argument with context optimization. Names to know: HippoRAG2 (PageRank over an entity graph), LightRAG (dual-level graph + vectors), LinearRAG (relation-free, linear cost). My take: run the baseline and publish the table — "graphs weren't worth it here, and here's the number" beats a fancy demo.
Reading lists rot in bookmarks. Publishing mine forces the one-line take — and the take is the actual skill: knowing what each result changes about how you build tomorrow.
The questions I don't get to answer at work — but can't stop thinking about.
Past a certain point the build stops and the wondering starts. Three questions sit at that edge for me. I keep them scientific on purpose — these are where I might actually contribute; the policy fights I leave to people who enjoy them.
If a chip runs the same computation as a brain, is anyone home?
Does mind depend on what it's made of? Functionalism says no — the computation is all that matters, so a photonic or neuromorphic mind would count. Biological naturalism says the wetware matters. Integrated Information Theory tries to put a number on it. The uncomfortable version — Chalmers' hard problem — is that you could explain every last computation and still not know why it feels like anything from the inside.
Getting a system to want what you meant.
The scientific version, not the sci-fi one: how do you make a capable system pursue the goal you intended, not the one you literally wrote? Specification gaming, reward hacking, scalable oversight — a model smart enough to satisfy the letter of your metric and miss the point. It's my eval work scaled up: evaluation is alignment at the frontier.
Right answer, black box — not good enough.
A model can be correct and still unreadable, and in healthcare or finance "trust me" isn't an answer. Explainability makes a decision auditable — why was this claim denied? Mechanistic interpretability is the deeper science of reading what the weights actually do. My Prior Auth agent citing the exact reg is the shipping version; reading the circuits is the research version.
From "what's a token" to the questions nobody's answered yet.
That's the whole arc — the basics, how they work, what I build on top, where computing itself is headed, and the questions I chase past the edge of it. Point at anything here and I can go a level deeper, in concept or in code.
🤖 Generated with AI, this one — I'll own it. The day one of these is my own typing, you won't need the disclaimer to tell. Fewer em-dashes, more typos. Trust me, you'll know.