Fabio Nonato de Paula — Blog

I Built My Own AI Agent (And Open-Sourced It)

2026-03-04T00:00:00Z

This started because I wanted a Discord bot that could remember things.

Not a chatbot — I have plenty of those. I wanted an agent that could hold a conversation across days, search the web, run shell commands, and actually learn who I am over time. The kind of thing where you message it on Tuesday about a project and on Friday it remembers the context without you re-explaining everything.

So I went looking for a framework. That's where the trouble started.

The Framework Problem

There are approximately ten thousand AI agent frameworks right now, and every week someone on Hacker News launches a new one. They all promise the same thing: "Build powerful AI agents in minutes!" I evaluated three seriously.

Framework	Size	Problem
OpenClaw	~400,000 lines	42,000 exposed instances on Shodan. Impossible to audit.
ZeroClaw	~2,000 lines	9 days old. No community, uncertain future.
NanoClaw	~500 lines	Too thin. Missing memory, MCP, observability. Would rebuild most of it anyway.

The core needs — LLM chat, persistent memory, tool calling, Discord interface, structured logging — are individually simple and well-understood problems. No 400K-line framework needed. The risk of building from scratch was spending a few days writing Python. The risk of a framework was inheriting its complexity, its security surface, and its opinions about how agents should work.

Easy tradeoff.

What Luna Actually Does

Luna is a custom AI agent that runs entirely on local hardware — two RTX 3090s (48GB total VRAM) running Qwen3-Coder-Next via llama-server, with no cloud APIs and no ongoing costs. The architecture is deliberately boring:

Discord (discord.py)
     |
     v
+-----------------------+
|    Luna Agent Core    |
|                       |
|  agent.py             |  agent loop: msg -> memory -> prompt -> LLM -> tools -> respond
|    +-- llm.py         |  single LLM client, configurable endpoint
|    +-- memory.py      |  SQLite + FTS5 + sqlite-vec hybrid search
|    +-- tools.py       |  native tools: bash, files, web fetch, web search
|    +-- tool_output.py |  smart output pipeline for large results
|    +-- mcp_manager.py |  MCP client for community tool servers
|    +-- observe.py     |  structured JSON logging
|                       |
+-----------------------+
         |
         v
   llama-server          any OpenAI-compatible endpoint

One process, one database file, one config file, eight runtime dependencies. The whole thing is ~2300 lines of Python including tests. Every line is auditable because there aren't that many lines to audit.

The Design Choices That Matter

Building from scratch means you own every decision, which is both the privilege and the burden. Here are the ones that shaped Luna the most:

SQLite for everything. Messages, memories, full-text search, and vector search all live in a single file. FTS5 is built into SQLite, and sqlite-vec adds vector search without needing a separate vector database. The entire memory system backs up with cp. I spent exactly zero hours configuring Postgres, managing Redis, or debugging connection pools. For a single-user agent running on a homelab, this is exactly the right level of infrastructure.

Hybrid search with Reciprocal Rank Fusion. Memory retrieval combines FTS5 keyword search with sqlite-vec semantic search. Keyword search catches the exact matches that embeddings miss — things like "error code E1234." Vector search catches the semantic matches that keywords miss — "the bug where the server crashes" finds a memory about a segfault even though neither word appears. RRF fuses the two result sets with one line of math per result, no trained model needed.

One LLM endpoint, firewall-ready. All traffic flows through a single LLMClient pointing at one configurable URL. Today that's localhost:8001. When I'm ready to add an AI firewall — an input/output filtering proxy — I change the URL to localhost:9000 and put the proxy in front of the real LLM. Zero code changes required. I didn't build the firewall, but I didn't block the insertion point either.

Conversation compression. Every 50 messages, the LLM summarizes the conversation and extracts facts with importance scores. Important facts go into long-term memory, and the summary keeps the conversation coherent across sessions. This gives effectively infinite conversation length — the agent always has context, even if the verbatim messages were summarized away days ago.

What I Didn't Build (On Purpose)

Honestly, this is the list I'm most proud of:

No AI firewall (future — just don't block the insertion point)
No web dashboard (the structured logs are ready for one when I want it)
No multi-user auth (single user)
No cloud LLM fallback (local only)
No Docker (systemd is simpler for a single-user Python process)
No abstractions for hypothetical future requirements

Every feature I didn't build is a feature I don't have to maintain, secure, or debug at 2am. Sophisticated is the enemy of simple, complex is the enemy of valuable. The right amount of complexity is the minimum needed for the current problem.

How Memory Actually Works

This is the part I'm most technically proud of, because it's the thing that makes Luna feel like more than a stateless chatbot. The memory system has three layers that work together:

Message history is the raw conversation, stored per session. This is what gives the agent short-term context — the last 20 messages in the current thread.

Extracted memories are facts the LLM identifies as worth remembering, each with an importance score from 1 to 10. A score of 10 ("user's name is Fabio") always surfaces when relevant. A score of 2 ("the weather was nice") fades quickly. These persist across sessions — the agent builds a growing understanding of the world over time.

Session summaries are LLM-generated compressions of old message blocks. When the conversation gets long, older messages get summarized so the agent retains the gist without eating the entire context window.

When the agent needs to recall something, search combines all three signals:

final_score = rrf_score + (recency_weight * 2^(-age_days / 7)) + (importance / 10 * 0.1)

The recency decay has a 7-day half-life — a week-old memory scores half as much as a fresh one. All of these parameters live in config.toml, so you can experiment with the tradeoffs without touching code.

For embeddings, I went with nomic-embed-text-v1.5: 22M parameters, loads in seconds, runs entirely on CPU without touching GPU memory. It supports Matryoshka representations, which means I can use 384 dimensions now and scale to 768 later without re-embedding everything.

Tools: Native and MCP

Luna ships with six built-in tools — bash (with safety guardrails), file read/write, directory listing, web fetch, and web search. The bash tool checks commands against blocked patterns before execution, enforces a 30-second timeout, and caps output at 50KB. No rm -rf /, no mkfs, no fork bombs.

For everything beyond the builtins, there's MCP. The Model Context Protocol lets you connect community tool servers by editing a JSON config:

{
  "servers": {
    "browser": {
      "command": "npx",
      "args": ["-y", "@playwright/mcp"],
      "transport": "stdio"
    }
  }
}

Need browser automation? Add the Playwright MCP server. Need filesystem tools? Add that server. Each one runs as a separate process with natural isolation, and tool names get namespaced automatically (browser__navigate, filesystem__read_file) so nothing collides. Adding a new capability to the agent is editing JSON, not writing code.

The Numbers

~2300 lines of Python (agent + tests)
106 tests, all passing — no GPU or running LLM required to run them
8 runtime dependencies — discord.py, openai, mcp, sentence-transformers, einops, sqlite-vec, html2text, duckduckgo-search
~187 tokens/sec prompt processing, ~81 tokens/sec generation on 2x RTX 3090

Why Open Source It

I built Luna for my homelab, and it works well for what I need. But the decisions behind it — custom build over framework, SQLite over Postgres, local LLM over cloud API, deliberate simplicity over feature checklists — those aren't unique to my setup. Anyone with a GPU and a desire to actually understand their AI agent stack could use this as a starting point, or at least steal the ideas they like.

The code is MIT licensed. The DESIGN.md explains the reasoning behind every major architectural decision. The tests mock the LLM client so you can run them on a laptop. The config is a single TOML file with sensible defaults.

If you're tired of agent frameworks that are bigger than the applications they power, maybe start with something you can actually read.

GitHub: nonatofabio/luna-agent

The repo has a few good first issues if you want to contribute. And if you build something interesting with it, I'd genuinely love to hear about it.

Thanks for reading.

Keep it Awesome!

I Co-Authored a Research Paper With an AI Agent

2026-02-04T00:00:00Z

Look, I'm going to be honest with you. This whole thing started as an experiment within an experiment.

The official research question was: "Can a language model learn continuously from human conversations without forgetting everything it already knows?" But the real experiment? Whether I could run an entire ML research project - from hypothesis to paper - with an AI agent as my co-pilot. Not as a fancy autocomplete. As an actual collaborator.

Spoiler: It worked. We wrote a paper. And the process was, aham, weird.

The Setup: Human + Agent = ???

Here's how it worked. I'm a scientist with 15+ years in ML. I had some research intuition, the domain knowledge, and access to 8x A100 GPUs on a remote server. What I didn't have was infinite time to write boilerplate code, babysit training runs, and manually parse log files at 2am.

Enter the agent.

My AI collaborator could execute bash commands, write and modify code, SSH into the training server, run experiments, parse results, and—critically: remember context across our entire conversation. It wasn't just answering questions. It was maintaining state. Tracking what we'd tried. Suggesting next steps.

The workflow looked like this:

Me: "Let's try reducing the GaLore rank to 64 and see if it helps with forgetting"
Agent: *writes config file*
Agent: *SSHs to server*
Agent: *launches training run*
Agent: *monitors logs*
Agent: "Training complete. MMLU dropped 3.0% vs 4.8% before. Want me to run holdout eval?"
Me: "Yes"
Agent: *runs eval, parses results, updates experiment notes*

I was the advisor. The agent was the grad student who never sleeps and never complains about running "just one more ablation."

Day 1: The Naive Approach (We Both Got It Wrong)

Our first attempt was embarrassingly simple. Train a 0.5B model on conversation data. Check if it learned. Check if it forgot. The agent set everything up: data pipelines, training loop, GaLore optimizer. I reviewed the code, made some suggestions, and we kicked off a 500-step run.

The training loss looked beautiful. Smooth curves. Decreasing numbers. Then we ran the benchmarks.

MMLU: 48.26% → 43.45% (-4.8%)
Holdout perplexity: +14% to +22% WORSE

The model got dumber. Classic overfitting. We'd both missed it.

Here's the thing: the agent didn't try to hide the failure or spin it. It just... reported the results and asked what we should try next. No ego. No defensiveness. Just "well, that didn't work. Here are some hypotheses." That's when I realized this collaboration might actually work.

The Design of Experiments: Where the Agent Earned Its Keep

I decided we needed a proper factorial experiment. Three hyperparameters, eight combinations, run in parallel. The kind of thing that's conceptually simple but logistically annoying.

Me: "Let's do a 2³ DOE. Factors are rank reduction, LR scheduler, and weight decay." The agent:

Generated all 8 config files
Wrote a bash script to launch them in parallel across GPUs
Wrote an eval script to benchmark all 8 models
Created a results table in our experiment notes
Ran the whole thing while I went to get coffee

[Insert Code Snippet: The DOE launch script the agent wrote in about 30 seconds - please, Claude Intern 1.0, fix this!]

When I came back, I had this:

Config	Rank	Scheduler	Weight Decay	MMLU	Verdict
3.0	128	❌	0.1	44.15%	Baseline
3.1	64	❌	0.1	45.26%	WINNER
...	...	...	...	...	...
3.6	128	✅	0.2	43.90%	Worst
...	...	...	...	...	...

The agent had already identified the pattern: rank was the dominant factor. Scheduler hurt. Weight decay did nothing. I didn't have to parse a single log file.

The Scaling Crisis: When We Hit a Wall

Feeling confident, we tried the winning config on bigger models. TinyLlama (1.1B) worked perfectly. Then Gemma-2 (2.6B) broke everything.

MMLU drop: -5.67%

The agent flagged it immediately: "This exceeds the 5% threshold. The rank-64 setting may be too permissive for larger models." This is where the human-agent dynamic got interesting. The agent had the data. I had the intuition. Together, we hypothesized that bigger models need lower rank, meaning more constraint, not less.

Me: "Try rank=32 on Gemma" Agent: runs experiment Agent: "MMLU now -3.85%. Within threshold."

We'd discovered a "scaling law": rank ∝ 1/√params. The agent helped me formalize it into a table:

Model Size	Optimal Rank
0.5B-1.1B	64
2B-3B	32
8B	8

Neither of us would have gotten there alone. I needed the agent to run the experiments fast enough to iterate. The agent needed me to recognize the pattern and propose the hypothesis.

The 8B Moment: When It Actually Worked

Time for the real test. Qwen3-8B. 8.2 billion parameters. Rank=8. 1000 training steps. The agent ran it overnight. I woke up to this message:

Training complete. 5:40 duration, 775 tokens/sec.
MMLU: 74.93% → 75.07% (+0.14%)
Holdout PPL: 5.34 → 4.87 (-8.8%)

Wait. The model got smarter? I didn't believe it. I asked the agent to run a statistical significance test.

Paired t-test: t=7.12, p<0.0001
Significant: True

It was real. The model learned from conversations and improved on benchmarks. Not just "acceptable forgetting", it got net positive knowledge transfer. The agent's response: "This proves the core hypothesis. Want me to update the experiment notes and commit?"

Yes. Yes I did.

The LoRA Showdown: A Plot Twist

I had a nagging question: how does this compare to LoRA, the thing everyone actually uses? The agent ran the comparison. Same model, same data, same steps, same rank.

Efficiency:

LoRA: 22GB VRAM, 982 tokens/sec ✅
GaLore: 39GB VRAM, 775 tokens/sec

Learning quality:

LoRA holdout PPL: +530% (catastrophic failure)
GaLore holdout PPL: -8.8% (genuine learning)

LoRA was faster and lighter. It also completely failed to learn anything generalizable. The model memorized training data and forgot how to generalize. The agent's analysis was spot-on: "LoRA freezes base weights and only trains adapters. It can't integrate new knowledge—only patch outputs. GaLore projects gradients but updates all weights, enabling genuine learning."

That insight made it into the paper almost verbatim.

Writing the Paper: The Final Boss

After six phases of experiments, we had results. Now we needed a paper. This is where I expected the collaboration to break down. Writing is hard. It requires judgment, narrative, argumentation. Surely an AI can't...

The agent drafted the abstract in one shot. It was... good? Like, actually good. It captured the key findings, the methodology, the implications. I edited maybe 20%.

We went section by section:

I'd outline what needed to be said
The agent would draft it
I'd revise and push back
The agent would incorporate feedback

The related work section was particularly impressive. The agent pulled relevant citations, summarized them, and positioned our work in the literature. I added a few papers it missed, but the structure was solid.

If you look at my git log:

eda30d8 exp006: Add statistical significance test - p<0.0001, t=7.12
0151460 exp006 Phase 3: GaLore vs LoRA comparison - GaLore wins
53a2c88 exp006 Phase 2: Add learning measurement
9b1e689 exp006 Phase 2: Extended training shows +0.14% MMLU improvement
f385da7 Update paper with exp006 8B validation

Every commit was a collaboration. Every result was verified. Every claim was backed by an experiment we'd run together.

What I Learned About Human-Agent Collaboration

1. The agent is a force multiplier, not a replacement.

I couldn't have run this many experiments this fast alone. But the agent couldn't have designed the experiments or recognized the patterns without me. We were genuinely complementary.

2. Context is everything.

The agent remembered our entire conversation, every failed experiment, every hypothesis, every decision. It could say "remember when we tried X and it didn't work because Y?" That continuity was invaluable.

3. The agent has no ego.

When experiments failed, the agent just... moved on. No defensiveness. No excuses. Just "that didn't work, here's what we could try next." It's weirdly refreshing.

4. Trust but verify.

I checked every result. Every claim. Every number. The agent made mistakes, small ones, big ones. But the verification loop kept us honest.

5. The meta-irony is not lost on me.

We built a system for AI to learn from human conversations. We did it through human-AI conversation. The research method mirrored the research question.

The Takeaway

The paper's conclusion is about GaLore and continuous learning. But my conclusion is different.

We're entering an era where research itself can be collaborative between humans and AI agents. Not AI replacing researchers. Not humans doing everything manually. Something in between - a partnership where each side contributes what they're good at.

I brought intuition, judgment, and experience. The agent brought speed, memory, and tireless execution. Together, we wrote a paper that neither of us could have written alone.

The answer to "Can AI learn continuously from human conversations?" turned out to be yes.

But the more interesting answer? "Can humans and AI agents do research together?"

Also yes.

The paper is available here. The code we wrote is in: Continuous Learning. And I'm going to go touch grass now, something my co-author will never need to do.

Thanks for reading.

Keep it Awesome!

Welcome to My Blog

2026-02-04T00:00:00Z

I've been meaning to come back to writing for a while now. After years of building tools, shipping features, and breaking AI/ML infrastructure, I realized I learned a lot of stuff that might be useful to share.

What to Expect

This blog will cover topics around my interests:

AI/ML Infrastructure - Practical lessons from building and scaling ML systems at AWS
Developer Tools - Building useful things like MCP servers and local AI tooling
Cybersecurity - Threat intelligence and security applications of machine learning
Open Source - Deep dives into projects I'm working on

Why Write?

Writing forces clarity. When you have to explain something, you find gaps in your own understanding. I'm hoping this blog helps me think more clearly, while maybe helping others along the way.

The Tech Stack

This blog is intentionally simple: Markdown files rendered client-side with marked.js, hosted on GitHub Pages. No build step, no database, no complexity. Just text files and a browser.

Sometimes the best tool is the simplest one that gets the job done.

Stay tuned for more posts. You can find me on GitHub or LinkedIn.