You've heard all the words.
Here's what they actually are — and what I build with them.
AI, LLM, tokens, embeddings, RAG, agents, LangChain, evals. This is a scroll-through that starts from zero — what each thing really is and how it works — and climbs to the systems I ship. Send it to someone who doesn't know AI, or read it end to end. It goes concept → how it works → what I build.
Most "AI" words are nested inside each other.
The jargon sounds like a pile of unrelated buzzwords. It isn't. Almost all of it fits in one picture: a set of Russian dolls, plus a short list of things you actually do with the innermost one.
Ask well, show examples. Cheapest lever. (§02)
Feed it your documents at question time. (§06)
Let it use tools and run a loop. (§07)
Retrain the weights for a fixed skill. Last resort.
It's autocomplete that read the internet.
An LLM does exactly one thing: given some text, it predicts the next chunk of text, over and over. That's it. Everything that feels intelligent — answering, coding, reasoning — is that one trick, run at enormous scale on almost everything ever written.
Models don't read words. They read tokens.
Before anything happens, your text is chopped into tokens — common chunks, often a word or a piece of one. Roughly four characters each. Everything the model does is counted, priced, and limited in tokens, so it's worth seeing them.
Under the hood — how the split is chosen
Every token becomes a point in space. Similar meanings sit close.
A token is turned into a long list of numbers — a vector — that places it in a "meaning space." The model learns this space so that related things land near each other. This one idea is the engine behind search, RAG, and recommendations.
Under the hood — what "close" actually means
"It" means nothing until the model decides what "it" points to.
Words change meaning based on their neighbors. Attention is the mechanism — the "T" in GPT — that lets every word look at every other word and decide which ones matter. It's how the model resolves "it," "this," "there." The 2017 paper was literally titled "Attention Is All You Need."
Under the hood — the actual mechanism
Now embeddings pay off: give the model your facts.
The model only knows what it was trained on — not your contracts or what changed today. So I embed your documents into that same meaning-space, and at question time retrieve the nearest chunks and hand them over. The model answers grounded in them, with citations. That's Retrieval-Augmented Generation.
Your question becomes a vector. Find the closest document chunks in embedding space — the exact "nearest neighbors" you hovered a moment ago. Feed those to the model as context.
# retrieve by embedding similarity, then ground docs = store.similarity_search(question, k=6) context = rerank(question, docs)[:3] answer = llm.invoke( GROUNDED_PROMPT.format(context=context, question=question) ) # "cite the source for every claim"
Prior Auth quotes the exact CMS regulation instead of guessing — in healthcare an uncited answer is worthless. Earnings Inspector reads a filing and surfaces what the press release buried. Same mechanism, two industries.
Give it actions, and let it run the loop.
A model that can only talk is a chatbot. Wire it to tools — send a text, query a database, file a form — and it can act. When a task needs several steps and memory, that's an agent: it reasons, acts, sees the result, and decides again. LangGraph is what I use to run that loop reliably — explicit state, checkpoints, and a record of every step. Step through one:
The state object is what the agent remembers across steps — and, two sections down, it's also what makes every step loggable.
I don't defend the brand — I answer with the trace. Running this loop by hand means rebuilding state, retries, and the real prize: observability into every step. LangGraph makes each step loggable — which is exactly what §09 needs.
I don't build one-offs. I build these patterns.
Every project I ship is an instance of a reusable archetype — a shape of system I can stand up again for you. Here are the ones I've proven, each with the real build behind it.
Reads your corpus, answers with receipts
RAG + citation over regulated or dense documents. For when a wrong, uncited answer is unacceptable.
Does the work, not just the talk
Tool-using agent that takes real actions through SMS, APIs, and forms — with persistent memory across a long task.
Turns a document into a decision
Reads something long, scores it against a rubric, and surfaces what a human would've missed — tone, red flags, omissions.
Breaks agents before users do
RL-driven attack policies probing for prompt injection, tool misuse, jailbreaks. Red-teaming as a service.
Proves the thing actually works
Golden sets, regression tests, trace logging, cost tracking — the difference between a demo and production.
Ships inside a big org's workflow
Internal tools across business functions — RFP generation, knowledge-base connectors, EDGAR search — built to survive review.
A great agent still needs somewhere to live.
"The cloud" is just someone else's computers, rented by the minute. The real question is never which is best — it's which fits this job. There are three tiers, and the choice is usually made for you by one thing: where your data is allowed to be.
Azure · AWS · GCP
Full control, every service, enterprise compliance and data residency. Heavy to set up. Reach for it when: regulated data, an existing enterprise contract, or real scale.
Vercel · Render · Supabase · Fly
Managed, cheap to start, live in minutes — no VPC to provision. You trade control for speed. Reach for it when: a startup, a prototype, or you need to ship this week.
Model + inference hosts
The model itself is usually an API — OpenAI, Anthropic, or Azure OpenAI. Custom or open models run on Modal / Replicate; Streamlit for quick demos. Reach for it when: always — it sits on top of tier 1 or 2.
At PwC I shipped on Azure because banking clients required data residency and already had enterprise agreements — Azure OpenAI meant the model ran inside their tenant, which is often the whole reason a regulated org picks a cloud at all. For my own products I run Render + Supabase + Vercel — because I need to ship this week, not provision infrastructure. Same engineer, opposite choice, and I can explain the trade either direction.
Anyone can demo an agent once. I ship the 1,000th run.
The value isn't the demo — it's the run on the input nobody predicted, without burning money or going off the rails. Two disciplines: evaluation (measure whether it meets the bar) and monitoring (watch whether it still does in production, with a full trace of every step).
A golden set + assertions, run on every change so "improving" one case can't silently break another. Offline, measured, repeatable. My LangGraph Eval Harness.
A wrong answer still returns 200 OK, so I watch three layers: operational (latency, cost/run), quality (judge scores, thumbs, task success), and the trace above — the only way to answer "why did it do that."
A bad production trace becomes a golden test — so that failure can never quietly return. Monitoring feeds evals feeds the next change. That's the whole discipline, and where most "AI demos" fall apart.
Everything above works. The problem is what it costs to run.
The quiet fact under the whole AI boom: it's staggeringly energy-hungry. Training a frontier model draws megawatts for weeks; then serving it to millions is a permanent, larger draw. Data centers are already a meaningful slice of global electricity — and the water to cool them is the constraint nobody tweets about.
For 40 years chips got faster almost for free — Moore's law. That's slowing. Transistors are now a few atoms wide; shrink them more and electrons leak straight through as wasted heat. Dennard scaling — power dropping as transistors shrank — already ended around 2005. The free lunch is over.
We stopped getting speed from smaller transistors and started getting it from more of them — more GPUs, more power, more heat. That's a spending race, not a physics win. The real question becomes: what if we computed a fundamentally different way?
If shrinking transistors is done, the next leap changes the architecture — or the physics itself.
People lump these together; they're really two different escapes. Some rearrange the chips — still silicon, radically different design. Others change the medium — compute with light, molecules, or quantum states. And one thing here isn't hardware at all. Honest labels below; the numbers are lab / vendor benchmarks on favorable workloads, not guarantees.
Neuromorphic — compute like a brain
Chips of spiking neurons that fire only on a signal, with memory and compute fused so data never shuttles back and forth. Intel Loihi 2, IBM NorthPole, SpiNNaker.
Analog in-memory — math inside the memory
Memristors do the matrix-multiply where the data already lives — killing the shuttle between memory and processor that burns most of the energy in a GPU.
Photonic — compute with light
Matrix multiplies happen as light interfering through a mesh — and light also moves data between chips far cheaper than copper. Lightmatter, Lightelligence.
Quantum — a different question entirely
The honest take, against the headlines: not faster AI and not an energy fix. It's a specialized co-processor for simulation, optimization, and sampling — used alongside classical, not instead of it.
Chemical & molecular
Compute in the physics itself — reaction-diffusion waves (Belousov–Zhabotinsky), DNA and molecular logic. Massively parallel, tiny energy, deeply strange.
Hyperdimensional computing isn't a substrate — it's a representation: encode meaning as huge vectors, and it runs beautifully on exactly the neuromorphic and in-memory hardware above. That's the layer I research — presented at the Science of Consciousness Conference.
The questions I don't get to answer at work — but can't stop thinking about.
Past a certain point the build stops and the wondering starts. Three questions sit at that edge for me. I keep them scientific on purpose — these are where I might actually contribute; the policy fights I leave to people who enjoy them.
If a chip runs the same computation as a brain, is anyone home?
Does mind depend on what it's made of? Functionalism says no — the computation is all that matters, so a photonic or neuromorphic mind would count. Biological naturalism says the wetware matters. Integrated Information Theory tries to put a number on it. The uncomfortable version — Chalmers' hard problem — is that you could explain every last computation and still not know why it feels like anything from the inside.
Getting a system to want what you meant.
The scientific version, not the sci-fi one: how do you make a capable system pursue the goal you intended, not the one you literally wrote? Specification gaming, reward hacking, scalable oversight — a model smart enough to satisfy the letter of your metric and miss the point. It's my eval work scaled up: evaluation is alignment at the frontier.
Right answer, black box — not good enough.
A model can be correct and still unreadable, and in healthcare or finance "trust me" isn't an answer. Explainability makes a decision auditable — why was this claim denied? Mechanistic interpretability is the deeper science of reading what the weights actually do. My Prior Auth agent citing the exact reg is the shipping version; reading the circuits is the research version.
From "what's a token" to the questions nobody's answered yet.
That's the whole arc — the basics, how they work, what I build on top, where computing itself is headed, and the questions I chase past the edge of it. Point at anything here and I can go a level deeper, in concept or in code.
🤖 Generated with AI, this one — I'll own it. The day one of these is my own typing, you won't need the disclaimer to tell. Fewer em-dashes, more typos. Trust me, you'll know.