Fabio Nonato de Paula — Blog

A Fruit Fly Brain Plays Doom

2026-09-21T00:00:00Z

Imagine I opened this post by telling you I got a fly to play Doom. Would you believe me?

The five ViZDoom scenarios, one clip each. Every clip is the median episode of ten, so this is typical play and not a highlight reel.

You should not. The thing playing up there is not a fly, but its wiring diagram comes from one. It is a recurrent network built on a real fruit fly connectome: 49,393 neurons and 9 million synapses, all frozen, with one learned gain per synapse. An ordinary convolutional network feeds it the game frames and an ordinary MLP turns its activity into moves, and those two hold more than half of the parameters. So read every result in this post as "a network constrained to the fly connectome" and never as "a fly". Yes, the title breaks that rule. That is the hook, and the rest of the post is the correction.

Now, here is the thing: I did not write the code, the infrastructure or the first drafts of the docs. All of it came out of the Strands harness, an open-source coding agent from the Strands Agents team at AWS that you run from the terminal. It keeps its own state on disk between sessions and runs tools, tests and cloud deployments on its own. Here it wrote the pipeline that turns the fly's wiring into a network and the five ordinary Doom agents that served as teachers. It wrote the trainer that taught the fly-wired network to copy them, and the second training stage that tried to push it further. It also wrote the cloud setup that ran a GPU fleet in three AWS regions, and it recorded the footage. Five days, 28 commits, about 3,300 lines of Python and shell, twelve fine-tuning runs and three control runs.

What I actually typed

The harness keeps its own session state on disk, so I went back and read my side of the transcript. It is short. It started with one prompt, typos and all:

I just saw this: https://huggingface.co/mlabonne/chessfly

I want to make the same thing but to play doom! Look at the skills and memory and have a set up ready for us to train it using my fnp3 aws accountl. Also create a infographic style tutorial on how it works, both the conectome model as well as the training. Use this current directory as a infra/cdk for training and training source and data ETL pipeline source. Once we complete all, we should have this is a private github repo. If you find a way to start training while I'm away, go for it!

Across the seven sessions that followed, this is most of what I said:

Go for it!

Can you check how are we doing on the training of doomfly?

Do you have suggestions to speed up the training process?

how do I use tb to see all the runs so far? (tb is TensorBoard)

Let's stop and revert all this we should not pursue this anymore

Sorry, wrong session!

"Let's stop and revert all this" was me killing an easter egg I had asked for an hour earlier: a Doom mod with a fly paw for the player's hand and Stan the frog, the Strands mascot, as the enemies. The harness reverted the whole thing, previews and tests included, and went back to the training run. "Sorry, wrong session!" came right after I pasted a request meant for a different agent, about a different project.

I ran all of it from the terminal with the strands command, and I changed the model underneath it as I went. The first sessions ran on Claude Fable 5.1 on Amazon Bedrock. By the last day the same sessions were running on GPT-6 Astra at the highest reasoning effort. The transcript does not care which model is behind it, and neither does the repo.

Between my sentences

Everything else in the transcript is the harness at work, and reading it back showed me how much the Strands harness worked for me.

Each session persisted to disk, and the next one opened with a summary of the last. Seven sessions over five days read like one conversation, and I could close the laptop at night and pick up in the morning where it left off. Long tool outputs were paged out of the conversation as it grew and fetched back when the agent asked for them. About two hundred of those are still sitting on disk, and without that mechanism the logs from day one alone would have ended the project.

Background tasks paid off on the last day. While a two-hour evaluation ran on my Mac, the same agent widened the model's output from 22 Doom actions to 23 for the next experiment and wrote nine tests for it. It also confirmed that the widened model played the old scenarios bit-for-bit the same as before. The repo has an AGENTS.md that says never push, launch, terminate or delete without being told, and the harness asked every time. "Go for it!" was me answering. It also loaded my own writing and verification skills and the MCP tools I already use, which is why the docs it produced read like mine.

None of that is exotic on its own. What I had not seen before is all of it in one command, with nothing to wire up. The same agent is two lines of Python if you want it inside a script, and after this week I would not think twice about starting a project that way.

The part I trust

An agent that returns a plausible positive result is hard to trust. This one returned two negative results, with the run prefixes and standard deviations attached, because its rules say that negative results go in docs/ and not in the bin.

A short map of what the harness did, so the two results make sense. It first trained five ordinary Doom agents with PPO, a standard reinforcement learning method, one per scenario. Those are the teachers. It then distilled the teachers into one fly-wired network, the student, by having the student imitate their moves. The student ended up playing about as well as its teachers. The question the project was built for was whether a second stage of reinforcement learning, GRPO, could push it past that.

First, it could not. Twelve GRPO runs, one setting changed per run, and not one beat the student by more than the noise in the evaluation. GRPO keeps the new policy close to the student with a penalty. The penalty strength that stopped the policy from wandering into worse play was the same strength that stopped it from improving at all.

Second, and this one stings, the connectome does nothing measurable. The harness suggested and built two controls. One keeps the same graph but shuffles its edges, so every neuron keeps its degree and its sign and the biology is gone. The other removes the neuron layer entirely.

Backbone	Parameters	Training steps per second	Doom scores
Real connectome	48.2 M	646	baseline
Shuffled edges, two seeds	48.2 M	1,454	same, within noise
No connectome at all	27.7 M	14,686	same, within noise

One scenario as an example, ten episodes each: on defend_the_center the real connectome scored 18.6 ± 1.8, the two shuffled graphs 19.0 and 18.4, and the model with no connectome 18.3. The other four scenarios look the same. The fly wiring makes training 22 times slower and buys nothing on these five scenarios. That is why the caption says "wired like a fruit fly's brain" and not "a fruit fly's brain". The harness wrote that vocabulary rule into AGENTS.md on day two, before either result existed. It would have been a hard rule to write after a result I disliked.

What's next

Scores on the five scenarios stopped moving, and three of them sit at their ceilings, so the next test is a real level. The student already plays Freedoom II MAP01 zero-shot. With its aiming prior it survives twice as long as random and picks up items. It still dies in eight of ten episodes and never leaves the first two rooms. The wider output and a slot for a sixth scenario are done and tested. Next comes GRPO on that map with all three backbones: real, shuffled and none. If the wiring is ever going to matter, it will be where the policy has to learn rather than imitate. If the network learns anything there, the plan is fly versus fly in a duel.

The code is public at nonatofabio/doomfly-rl, and the trained checkpoints are on Hugging Face with the connectome file they need. If you want to watch a network shaped like a fly play Doom, start there.

Thanks for reading.

Keep it Awesome!

AI Generated Pixel Art Needs a Build System, Not Better Prompts

2026-09-07T00:00:00Z

I rebuilt my tiny homage to Warhammer 40k this week. The original, Hive City Rampage, was a Python/Pygame monstrosity, a top-down grimdark shooter that ran but was a prototype. Now there's a sequel, Ashgate Siege, built in Godot 4: gothic isometric, two missions, Mac and Android.

I have a confession though. The 95k lines of GDScript, 8 commits, roughly five hours, was written by an AI agent under my direction. The sprites are generated too. I'm disclaiming that up front because the interesting part is what came out of it.

The easy part was the code

This is the simple finding. An engine port is exactly the shape of problem my agents are good at: the target is well documented, the semantics known, and correctness is checkable by running the thing. Godot 4 plus GDScript is heavily represented in training data for any model. Zero agent struggle.

The hard part was four pictures of the same orc

Here's the hill I'm willing to die on: for AI-assisted games, 2D sprite games are harder than 3D.

That sounds backwards, so: in 3D, the engine guarantees coherence. One mesh, one material, one light rig, and every frame of animation is consistent because it's the same object being transformed. The renderer is doing the work.

In sprite based games there is no shared object. Each sprite is an independent generated map of pixels. So:

Frame 2 of a walk cycle can be a different character than frame 1.
The light source can move between sprites in the same scene.
Proportions drift. Your massive size boss shrinks.
Limbs get cropped at cell edges.

None of that is caught by anything automatically. It just ships, and the game looks odd, like a badly executed collage.

That screenshot has maybe a dozen generated sprites in it at once. Every one of them came out of a separate generation, and the only reason they read as one scene is the pipeline below.

Prompts as interface specs

The fix was to stop treating generation as commissioning art and start treating it as calling an API with a strict schema. Every prompt pins the geometry:

Production game sprite sheet: exactly 4 columns x 3 rows, 1536x1024 canvas,
equal cells, solid pure magenta #FF00FF background for color-key import.
NO text, shadows, smoke, or border. Every figure entirely within its own cell
with ample margin; feet centered at same baseline in each row. All figures face
screen RIGHT in three-quarter isometric view [...] Four columns are four
coherent walking-cycle poses: left foot forward, passing, right foot forward,
passing.

Each sentence has to do some work:

Magenta #FF00FF, not transparency. Alpha comes back unreliable. A color key is deterministic to strip.
Explicit grid and canvas. The importer crops on fixed coordinates. If the grid drifts, every sprite is off.
"feet centered at same baseline in each row." Without it the character bobs while walking.
Named poses. "Walking animation" returns four unrelated drawings. Naming the four phases is what makes them a cycle.
Negative constraints. "No text, shadows, smoke, or border" because models add decoration that breaks the silhouette.

The importer is a compiler, and tests are CI

If the prompt is a spec, something has to enforce it. Generated atlases go into art/, an importer crops and scales them into runtime assets, and a native test suite checks the result: 2,972 reference and combat assertions, 17,112 seam combinations, plus a 600-frame combat simulation. make test runs it.

The seam check is the one I lean on. It takes the belt pixels from the leg sprite, offsets them by the waist socket, and counts how many land on opaque torso pixels. Under 32 and the pose fails. Every view, every clip, every frame, every facing, every gait: 17,112 combinations.

That catches the failure mode that makes sprites look like a collage, which is a regenerated part that no longer meets the part next to it.

So I tried to break it

Claiming you have tests is cheap. I reimplemented the audit standalone, outside the engine, then fed it deliberately corrupted atlases to find out what it actually catches.

Baseline reproduces: 17,112 combinations, zero failures.

Then the damage:

Character 4% smaller: FAIL, 352 bad combinations
Character 8% smaller: FAIL, 6,728
Alpha edge eroded 2px: FAIL, 3,036
Torso shifted 6px down in the cell: PASS

Scale drift gets caught at 4% and up, and slips through at 3%. Vertical translation is never caught. I pushed it to 14px and it still passed, because shifting the torso down actually raises the minimum overlap from 59 to 249. The check gets happier while the sprite gets worse.

That is a real hole, and it is the honest version of "we have tests." The seam audit measures whether two parts still meet. It says nothing about whether the assembled pair sits correctly inside its cell, because the waist socket moves with the torso. Catching that needs a different check, anchored to the cell rather than to the neighbouring sprite.

I would rather know what the test misses than assume it catches everything.

The standalone auditor is tools/seam_audit.py in the repo. Plain Python, PIL and numpy, no Godot required, so you can run the numbers above yourself.

What I'd tell you to steal from my repo

Color-key over alpha. Deterministic beats convenient.
Pin the grid numerically in the prompt and crop on those exact numbers.
Name every animation phase. Never say "walk cycle" and hope.
Baseline your seams and diff them. This is the whole safety net.
Keep source atlases and prompts in the repo. Regeneration is a build step, so its inputs are source code.

The way this generalizes: if AI generation is nondeterministic, everything downstream has to be strict. The prompt is a spec that gets compiled by the importer, and CI runs the seam tests. Without that you don't have a pipeline, at best it's a slot machine you keep pulling until the art looks okay.

Try it

Mac and Android builds are on GitHub Releases. They're previews: macOS is ad-hoc signed and not notarized, Android uses a debug key, so both will warn you. Source builds with make run if you have Godot 4.3.

Also published on DEV.

The Certainty We Never Had

2026-08-21T00:00:00Z

🎧 Prefer to listen? 10:44, narrated locally on my homelab with LVNA.

Your browser doesn't support audio playback. Download the MP3.

There's a story our industry tells itself: traditional software is deterministic, therefore predictable, therefore safe. And AI, because it samples from a probability distribution, is none of those things.

The story is comforting. It's also wrong on both ends. It flatters conventional software with a certainty it never had, and it condemns AI against a baseline that exists only in textbooks.

A note on where I'm arguing from. I've spent a long time on the operations side of large-scale cloud systems, and most of what convinced me of this sits in incident reviews I can't quote. So I'm not going to pretend a citation is doing work that experience is doing. What follows is grounded in public literature wherever the literature will carry it, and flagged as my own judgment where it won't. You should discount the second part accordingly, but I'd rather you discount it than have me dress it up.

The determinism fetish

Individual machine instructions are deterministic: same state, same input, same output. But we quietly upgrade that narrow mathematical fact into a much grander claim: that the deployed system will do what people intended.

That inference doesn't hold. Determinism guarantees consistency, and a program can be consistently wrong. Deterministic malware is still malware. A deterministic deadlock reliably deadlocks. Correctness is a relationship between what the code does and what people wanted, and determinism says nothing about the second half.

This isn't new. DeMillo, Lipton, and Perlis argued in 1979 that even formal proofs of programs are social artifacts. They establish confidence, not truth. NIST's verification guidance still draws the same line: verification shows conformance to a specification. It cannot show the specification was right.

The hidden state is people

The deeper failure is operational, and it shows up somewhere most people don't look.

Ask why cloud operations is so expensive. The intuitive answer is hardware failure and the economics of scale, things nobody ever claimed were deterministic. I used to think that too. I don't anymore, and the reason is the part I find hard to argue in public without showing you incident data I don't own.

Here's what I can say. Hardware failure is the solved part of operations. It has clean failure models and automated remediation; a disk dying is a scheduled inconvenience. The expensive, unautomatable part is sociotechnical: changes people make to live systems, and the reasoning behind those changes going missing.

Peter Naur named the mechanism in 1985: programming is theory building. The program is not the theory. The theory (why that timeout is thirty seconds, what invariant that strange conditional protects, which dependency breaks if you touch this) lives in the heads of the people who built it. When they leave through reorgs, layoffs, or ordinary churn, the code stays perfectly deterministic while the system becomes less predictable, because nobody can reconstruct the assumptions the code encodes.

The team is now operating a system whose real specification exists nowhere.

The only knowledge that survives this decay is what's been compressed into hardened, slow-moving foundations like TCP, POSIX, and retry-with-backoff, the first principles that took decades to solidify. Everything above that layer is perishable. A large fraction of operations spend is the recurring cost of that perishability.

So the hidden state that makes deployed software unpredictable isn't just timing, faults, and adversaries. It's epistemic. Retries, circuit breakers, canaries, chaos testing, on-call rotations, root-cause reviews. These aren't decorations on a certain system. They're the tribute an uncertain system pays to reality. And a good part of that uncertainty is an organization forgetting its own reasons.

The safe-baseline fallacy

Once you see that, the standard case against AI loses its footing. That case compares the idealized abstraction of conventional software against the observed production behavior of AI. The fair comparison is observed system versus observed system, under matched tasks, authority, and safeguards.

Nancy Leveson has spent a career on the underlying point: safety is not a property of a component, human or mechanical. It's emergent from the whole sociotechnical system: architecture, environment, authority, and recovery mechanisms together. A program doesn't become safe because its source is deterministic. A human doesn't become safe because they possess judgment.

The right question is never "is this component certain?" It's "what's the probability of harm from this system, in this envelope, and is that lower or higher than what we run today?" The incumbent shouldn't win merely by being incumbent.

Richard Cook's How Complex Systems Fail explains why the incumbent feels safe anyway. After every incident, the team reconstructs a tidy causal chain, fixes it, and walks away feeling the event was predictable all along. It wasn't. Post-hoc explainability is not ex ante predictability. Root-cause analysis is a selected narrative, and its psychological side effect is renewed, unearned confidence in the deterministic story, right until the next failure arrives through a chain nobody drew.

If you've sat through enough of these, you already know the feeling I'm describing.

Where the symmetry ends

Here's where I part ways with the enthusiastic version of my own argument. Demolishing a fake baseline doesn't make AI as safe as what it replaces. Symmetric burden of proof is not symmetric ease of proof, and three asymmetries survive the correction.

Calibration. We have centuries of actuarial data on how people fail, and institutions (liability, licensure, sanction) built to shape those failures. For frontier models we have evaluations that are weak proxies for deployment behavior, systems that shift under context and provider updates, and attack classes like prompt injection with no patch-and-converge story the way memory-safety bugs had. We can write the risk comparison as an equation; we can't yet measure both sides with equal confidence.

Correlation. Human error is buffered by cognitive diversity. A thousand engineers make a thousand different mistakes. A fleet of agents running one model can make the same mistake, everywhere, simultaneously, from a single update. That's monoculture risk at a scale with no good human analogue. Lamport's fault-tolerance results are a warning here, not a comfort: they hold only under explicit assumptions about how many components fail independently.

Tempo. AI acts faster than oversight loops built for human speeds. A bad decision that a review meeting would have caught next Tuesday can execute ten thousand times by Tuesday.

Against these stand real advantages the old system never offered: complete execution traces, cheap replay, mechanically bounded permissions, adversarial testing at scale, and replacement without an HR process.

Neither column wins by default. An AI-augmented system may be safer than the human workflow in one envelope and far more dangerous in another. Which is exactly why this gets settled empirically, per deployment, not by reference class.

The position

Software assurance has always been probabilistic at the system level. Determinism was a property of the abstraction. The certainty was manufactured by hindsight, sustained by institutional habit, and quietly financed by an operations budget whose real job was absorbing the gap between the model and the world, a gap widened every year by the churn that carries a system's theory out the door.

AI didn't introduce uncertainty into software. It moved uncertainty somewhere we can no longer pretend not to see it.

That relocation is uncomfortable, and it should be used well. The right response is neither to bar probabilistic components from serious work by comparing them to a fiction, nor to wave every agent through because "software was never certain anyway."

It's to state, for every consequential system, whether human, deterministic, probabilistic, or mixed, the same disciplined claim: bounded probability of harm, explicit operating envelope, named assumptions, measured detection and recovery, and evidence the claim stays calibrated after deployment. Proofs, tests, monitors, and human escalation all become forms of evidence toward that claim, not talismans against needing one.

The determinism fetish gave us permission to skip that discipline for fifty years. AI has revoked the permission.

That may turn out to be its most valuable contribution to software engineering, delivered before anyone decides whether to trust it with anything else.

Sources

R. DeMillo, R. Lipton, A. Perlis, "Social Processes and Proofs of Theorems and Programs" (CACM, 1979)
P. Naur, "Programming as Theory Building" (1985)
N. Leveson, Engineering a Safer World (MIT Press, 2011)
R. Cook, "How Complex Systems Fail" (1998)
L. Lamport, R. Shostak, M. Pease, "The Byzantine Generals Problem" (1982)
NIST guidance on software verification and validation

Views are my own and don't represent my employer. Nothing here draws on non-public information.

I Built My Own AI Agent (And Open-Sourced It)

2026-03-04T00:00:00Z

This started because I wanted a Discord bot that could remember things.

Not a chatbot — I have plenty of those. I wanted an agent that could hold a conversation across days, search the web, run shell commands, and actually learn who I am over time. The kind of thing where you message it on Tuesday about a project and on Friday it remembers the context without you re-explaining everything.

So I went looking for a framework. That's where the trouble started.

The Framework Problem

There are approximately ten thousand AI agent frameworks right now, and every week someone on Hacker News launches a new one. They all promise the same thing: "Build powerful AI agents in minutes!" I evaluated three seriously.

Framework	Size	Problem
OpenClaw	~400,000 lines	42,000 exposed instances on Shodan. Impossible to audit.
ZeroClaw	~2,000 lines	9 days old. No community, uncertain future.
NanoClaw	~500 lines	Too thin. Missing memory, MCP, observability. Would rebuild most of it anyway.

The core needs — LLM chat, persistent memory, tool calling, Discord interface, structured logging — are individually simple and well-understood problems. No 400K-line framework needed. The risk of building from scratch was spending a few days writing Python. The risk of a framework was inheriting its complexity, its security surface, and its opinions about how agents should work.

Easy tradeoff.

What Luna Actually Does

Luna is a custom AI agent that runs entirely on local hardware — two RTX 3090s (48GB total VRAM) running Qwen3-Coder-Next via llama-server, with no cloud APIs and no ongoing costs. The architecture is deliberately boring:

Discord (discord.py)
     |
     v
+-----------------------+
|    Luna Agent Core    |
|                       |
|  agent.py             |  agent loop: msg -> memory -> prompt -> LLM -> tools -> respond
|    +-- llm.py         |  single LLM client, configurable endpoint
|    +-- memory.py      |  SQLite + FTS5 + sqlite-vec hybrid search
|    +-- tools.py       |  native tools: bash, files, web fetch, web search
|    +-- tool_output.py |  smart output pipeline for large results
|    +-- mcp_manager.py |  MCP client for community tool servers
|    +-- observe.py     |  structured JSON logging
|                       |
+-----------------------+
         |
         v
   llama-server          any OpenAI-compatible endpoint

One process, one database file, one config file, eight runtime dependencies. The whole thing is ~2300 lines of Python including tests. Every line is auditable because there aren't that many lines to audit.

The Design Choices That Matter

Building from scratch means you own every decision, which is both the privilege and the burden. Here are the ones that shaped Luna the most:

SQLite for everything. Messages, memories, full-text search, and vector search all live in a single file. FTS5 is built into SQLite, and sqlite-vec adds vector search without needing a separate vector database. The entire memory system backs up with cp. I spent exactly zero hours configuring Postgres, managing Redis, or debugging connection pools. For a single-user agent running on a homelab, this is exactly the right level of infrastructure.

Hybrid search with Reciprocal Rank Fusion. Memory retrieval combines FTS5 keyword search with sqlite-vec semantic search. Keyword search catches the exact matches that embeddings miss — things like "error code E1234." Vector search catches the semantic matches that keywords miss — "the bug where the server crashes" finds a memory about a segfault even though neither word appears. RRF fuses the two result sets with one line of math per result, no trained model needed.

One LLM endpoint, firewall-ready. All traffic flows through a single LLMClient pointing at one configurable URL. Today that's localhost:8001. When I'm ready to add an AI firewall — an input/output filtering proxy — I change the URL to localhost:9000 and put the proxy in front of the real LLM. Zero code changes required. I didn't build the firewall, but I didn't block the insertion point either.

Conversation compression. Every 50 messages, the LLM summarizes the conversation and extracts facts with importance scores. Important facts go into long-term memory, and the summary keeps the conversation coherent across sessions. This gives effectively infinite conversation length — the agent always has context, even if the verbatim messages were summarized away days ago.

What I Didn't Build (On Purpose)

Honestly, this is the list I'm most proud of:

No AI firewall (future — just don't block the insertion point)
No web dashboard (the structured logs are ready for one when I want it)
No multi-user auth (single user)
No cloud LLM fallback (local only)
No Docker (systemd is simpler for a single-user Python process)
No abstractions for hypothetical future requirements

Every feature I didn't build is a feature I don't have to maintain, secure, or debug at 2am. Sophisticated is the enemy of simple, complex is the enemy of valuable. The right amount of complexity is the minimum needed for the current problem.

How Memory Actually Works

This is the part I'm most technically proud of, because it's the thing that makes Luna feel like more than a stateless chatbot. The memory system has three layers that work together:

Message history is the raw conversation, stored per session. This is what gives the agent short-term context — the last 20 messages in the current thread.

Extracted memories are facts the LLM identifies as worth remembering, each with an importance score from 1 to 10. A score of 10 ("user's name is Fabio") always surfaces when relevant. A score of 2 ("the weather was nice") fades quickly. These persist across sessions — the agent builds a growing understanding of the world over time.

Session summaries are LLM-generated compressions of old message blocks. When the conversation gets long, older messages get summarized so the agent retains the gist without eating the entire context window.

When the agent needs to recall something, search combines all three signals:

final_score = rrf_score + (recency_weight * 2^(-age_days / 7)) + (importance / 10 * 0.1)

The recency decay has a 7-day half-life — a week-old memory scores half as much as a fresh one. All of these parameters live in config.toml, so you can experiment with the tradeoffs without touching code.

For embeddings, I went with nomic-embed-text-v1.5: 22M parameters, loads in seconds, runs entirely on CPU without touching GPU memory. It supports Matryoshka representations, which means I can use 384 dimensions now and scale to 768 later without re-embedding everything.

Tools: Native and MCP

Luna ships with six built-in tools — bash (with safety guardrails), file read/write, directory listing, web fetch, and web search. The bash tool checks commands against blocked patterns before execution, enforces a 30-second timeout, and caps output at 50KB. No rm -rf /, no mkfs, no fork bombs.

For everything beyond the builtins, there's MCP. The Model Context Protocol lets you connect community tool servers by editing a JSON config:

{
  "servers": {
    "browser": {
      "command": "npx",
      "args": ["-y", "@playwright/mcp"],
      "transport": "stdio"
    }
  }
}

Need browser automation? Add the Playwright MCP server. Need filesystem tools? Add that server. Each one runs as a separate process with natural isolation, and tool names get namespaced automatically (browser__navigate, filesystem__read_file) so nothing collides. Adding a new capability to the agent is editing JSON, not writing code.

The Numbers

~2300 lines of Python (agent + tests)
106 tests, all passing — no GPU or running LLM required to run them
8 runtime dependencies — discord.py, openai, mcp, sentence-transformers, einops, sqlite-vec, html2text, duckduckgo-search
~187 tokens/sec prompt processing, ~81 tokens/sec generation on 2x RTX 3090

Why Open Source It

I built Luna for my homelab, and it works well for what I need. But the decisions behind it — custom build over framework, SQLite over Postgres, local LLM over cloud API, deliberate simplicity over feature checklists — those aren't unique to my setup. Anyone with a GPU and a desire to actually understand their AI agent stack could use this as a starting point, or at least steal the ideas they like.

The code is MIT licensed. The DESIGN.md explains the reasoning behind every major architectural decision. The tests mock the LLM client so you can run them on a laptop. The config is a single TOML file with sensible defaults.

If you're tired of agent frameworks that are bigger than the applications they power, maybe start with something you can actually read.

GitHub: nonatofabio/luna-agent

The repo has a few good first issues if you want to contribute. And if you build something interesting with it, I'd genuinely love to hear about it.

Thanks for reading.

Keep it Awesome!

I Co-Authored a Research Paper With an AI Agent

2026-02-04T00:00:00Z

Look, I'm going to be honest with you. This whole thing started as an experiment within an experiment.

The official research question was: "Can a language model learn continuously from human conversations without forgetting everything it already knows?" But the real experiment? Whether I could run an entire ML research project - from hypothesis to paper - with an AI agent as my co-pilot. Not as a fancy autocomplete. As an actual collaborator.

Spoiler: It worked. We wrote a paper. And the process was, aham, weird.

The Setup: Human + Agent = ???

Here's how it worked. I'm a scientist with 15+ years in ML. I had some research intuition, the domain knowledge, and access to 8x A100 GPUs on a remote server. What I didn't have was infinite time to write boilerplate code, babysit training runs, and manually parse log files at 2am.

Enter the agent.

My AI collaborator could execute bash commands, write and modify code, SSH into the training server, run experiments, parse results, and—critically: remember context across our entire conversation. It wasn't just answering questions. It was maintaining state. Tracking what we'd tried. Suggesting next steps.

The workflow looked like this:

Me: "Let's try reducing the GaLore rank to 64 and see if it helps with forgetting"
Agent: *writes config file*
Agent: *SSHs to server*
Agent: *launches training run*
Agent: *monitors logs*
Agent: "Training complete. MMLU dropped 3.0% vs 4.8% before. Want me to run holdout eval?"
Me: "Yes"
Agent: *runs eval, parses results, updates experiment notes*

I was the advisor. The agent was the grad student who never sleeps and never complains about running "just one more ablation."

Day 1: The Naive Approach (We Both Got It Wrong)

Our first attempt was embarrassingly simple. Train a 0.5B model on conversation data. Check if it learned. Check if it forgot. The agent set everything up: data pipelines, training loop, GaLore optimizer. I reviewed the code, made some suggestions, and we kicked off a 500-step run.

The training loss looked beautiful. Smooth curves. Decreasing numbers. Then we ran the benchmarks.

MMLU: 48.26% → 43.45% (-4.8%)
Holdout perplexity: +14% to +22% WORSE

The model got dumber. Classic overfitting. We'd both missed it.

Here's the thing: the agent didn't try to hide the failure or spin it. It just... reported the results and asked what we should try next. No ego. No defensiveness. Just "well, that didn't work. Here are some hypotheses." That's when I realized this collaboration might actually work.

The Design of Experiments: Where the Agent Earned Its Keep

I decided we needed a proper factorial experiment. Three hyperparameters, eight combinations, run in parallel. The kind of thing that's conceptually simple but logistically annoying.

Me: "Let's do a 2³ DOE. Factors are rank reduction, LR scheduler, and weight decay." The agent:

Generated all 8 config files
Wrote a bash script to launch them in parallel across GPUs
Wrote an eval script to benchmark all 8 models
Created a results table in our experiment notes
Ran the whole thing while I went to get coffee

[Insert Code Snippet: The DOE launch script the agent wrote in about 30 seconds - please, Claude Intern 1.0, fix this!]

When I came back, I had this:

Config	Rank	Scheduler	Weight Decay	MMLU	Verdict
3.0	128	❌	0.1	44.15%	Baseline
3.1	64	❌	0.1	45.26%	WINNER
...	...	...	...	...	...
3.6	128	✅	0.2	43.90%	Worst
...	...	...	...	...	...

The agent had already identified the pattern: rank was the dominant factor. Scheduler hurt. Weight decay did nothing. I didn't have to parse a single log file.

The Scaling Crisis: When We Hit a Wall

Feeling confident, we tried the winning config on bigger models. TinyLlama (1.1B) worked perfectly. Then Gemma-2 (2.6B) broke everything.

MMLU drop: -5.67%

The agent flagged it immediately: "This exceeds the 5% threshold. The rank-64 setting may be too permissive for larger models." This is where the human-agent dynamic got interesting. The agent had the data. I had the intuition. Together, we hypothesized that bigger models need lower rank, meaning more constraint, not less.

Me: "Try rank=32 on Gemma" Agent: runs experiment Agent: "MMLU now -3.85%. Within threshold."

We'd discovered a "scaling law": rank ∝ 1/√params. The agent helped me formalize it into a table:

Model Size	Optimal Rank
0.5B-1.1B	64
2B-3B	32
8B	8

Neither of us would have gotten there alone. I needed the agent to run the experiments fast enough to iterate. The agent needed me to recognize the pattern and propose the hypothesis.

The 8B Moment: When It Actually Worked

Time for the real test. Qwen3-8B. 8.2 billion parameters. Rank=8. 1000 training steps. The agent ran it overnight. I woke up to this message:

Training complete. 5:40 duration, 775 tokens/sec.
MMLU: 74.93% → 75.07% (+0.14%)
Holdout PPL: 5.34 → 4.87 (-8.8%)

Wait. The model got smarter? I didn't believe it. I asked the agent to run a statistical significance test.

Paired t-test: t=7.12, p<0.0001
Significant: True

It was real. The model learned from conversations and improved on benchmarks. Not just "acceptable forgetting", it got net positive knowledge transfer. The agent's response: "This proves the core hypothesis. Want me to update the experiment notes and commit?"

Yes. Yes I did.

The LoRA Showdown: A Plot Twist

I had a nagging question: how does this compare to LoRA, the thing everyone actually uses? The agent ran the comparison. Same model, same data, same steps, same rank.

Efficiency:

LoRA: 22GB VRAM, 982 tokens/sec ✅
GaLore: 39GB VRAM, 775 tokens/sec

Learning quality:

LoRA holdout PPL: +530% (catastrophic failure)
GaLore holdout PPL: -8.8% (genuine learning)

LoRA was faster and lighter. It also completely failed to learn anything generalizable. The model memorized training data and forgot how to generalize. The agent's analysis was spot-on: "LoRA freezes base weights and only trains adapters. It can't integrate new knowledge—only patch outputs. GaLore projects gradients but updates all weights, enabling genuine learning."

That insight made it into the paper almost verbatim.

Writing the Paper: The Final Boss

After six phases of experiments, we had results. Now we needed a paper. This is where I expected the collaboration to break down. Writing is hard. It requires judgment, narrative, argumentation. Surely an AI can't...

The agent drafted the abstract in one shot. It was... good? Like, actually good. It captured the key findings, the methodology, the implications. I edited maybe 20%.

We went section by section:

I'd outline what needed to be said
The agent would draft it
I'd revise and push back
The agent would incorporate feedback

The related work section was particularly impressive. The agent pulled relevant citations, summarized them, and positioned our work in the literature. I added a few papers it missed, but the structure was solid.

If you look at my git log:

eda30d8 exp006: Add statistical significance test - p<0.0001, t=7.12
0151460 exp006 Phase 3: GaLore vs LoRA comparison - GaLore wins
53a2c88 exp006 Phase 2: Add learning measurement
9b1e689 exp006 Phase 2: Extended training shows +0.14% MMLU improvement
f385da7 Update paper with exp006 8B validation

Every commit was a collaboration. Every result was verified. Every claim was backed by an experiment we'd run together.

What I Learned About Human-Agent Collaboration

1. The agent is a force multiplier, not a replacement.

I couldn't have run this many experiments this fast alone. But the agent couldn't have designed the experiments or recognized the patterns without me. We were genuinely complementary.

2. Context is everything.

The agent remembered our entire conversation, every failed experiment, every hypothesis, every decision. It could say "remember when we tried X and it didn't work because Y?" That continuity was invaluable.

3. The agent has no ego.

When experiments failed, the agent just... moved on. No defensiveness. No excuses. Just "that didn't work, here's what we could try next." It's weirdly refreshing.

4. Trust but verify.

I checked every result. Every claim. Every number. The agent made mistakes, small ones, big ones. But the verification loop kept us honest.

5. The meta-irony is not lost on me.

We built a system for AI to learn from human conversations. We did it through human-AI conversation. The research method mirrored the research question.

The Takeaway

The paper's conclusion is about GaLore and continuous learning. But my conclusion is different.

We're entering an era where research itself can be collaborative between humans and AI agents. Not AI replacing researchers. Not humans doing everything manually. Something in between - a partnership where each side contributes what they're good at.

I brought intuition, judgment, and experience. The agent brought speed, memory, and tireless execution. Together, we wrote a paper that neither of us could have written alone.

The answer to "Can AI learn continuously from human conversations?" turned out to be yes.

But the more interesting answer? "Can humans and AI agents do research together?"

Also yes.

The paper is available here. The code we wrote is in: Continuous Learning. And I'm going to go touch grass now, something my co-author will never need to do.

Thanks for reading.

Keep it Awesome!

Welcome to My Blog

2026-02-04T00:00:00Z

I've been meaning to come back to writing for a while now. After years of building tools, shipping features, and breaking AI/ML infrastructure, I realized I learned a lot of stuff that might be useful to share.

What to Expect

This blog will cover topics around my interests:

AI/ML Infrastructure - Practical lessons from building and scaling ML systems at AWS
Developer Tools - Building useful things like MCP servers and local AI tooling
Cybersecurity - Threat intelligence and security applications of machine learning
Open Source - Deep dives into projects I'm working on

Why Write?

Writing forces clarity. When you have to explain something, you find gaps in your own understanding. I'm hoping this blog helps me think more clearly, while maybe helping others along the way.

The Tech Stack

This blog is intentionally simple: Markdown files rendered client-side with marked.js, hosted on GitHub Pages. No build step, no database, no complexity. Just text files and a browser.

Sometimes the best tool is the simplest one that gets the job done.

Stay tuned for more posts. You can find me on GitHub or LinkedIn.