Building an Agentic AI on Markdown Files

A complete, zero-infrastructure guide to building an AI agent whose brain is a folder of plain markdown files. No database, no vector store, no hosted memory service — just text you can grep, diff, commit to git, and open in any editor.

Companion material: this guide is grounded in the markdown-file-agent skill, which ships a verified reference implementation (agent.py, ~80 lines) and an offline verification harness. The full code is reproduced in Part 11.

What "agentic" actually means
Why markdown files (and when not to)
Anatomy of a markdown brain
The agent loop
The operating manual: AGENTS.md
Memory design
Tasks and plans
Knowledge base and wikilinks
Logs and the diary
The minimal toolset
Reference implementation: agent.py
Verifying without burning tokens
Patterns that make it useful
Security and failure modes
Scaling path: grep -> embeddings -> database
Real-world precedents
A learning path Appendix A. File-layout checklist Appendix B. Glossary

1. What "agentic" actually means

A chatbot answers. An agent acts.

The difference is not the model — it is the loop. A chatbot is a single request/response: you ask, it answers, nothing changes. An agent runs a loop:

observe  ->  think  ->  act  ->  update  ->  repeat

where "act" calls a tool and "update" writes the result of that action somewhere durable. The loop terminates when the agent decides it has an answer.

The single most important sentence in this guide:

A chatbot that only reads your files is a search tool. An agent that edits its own memory and todo files is maintaining state — and maintaining state is the beginning of autonomy.

The write-back step is what makes it agentic. A model that appends a status line to memory.md after every task, checks off items in todo.md as it completes them, and writes plan.md before starting long work is doing something a plain prompt cannot: it is persisting its own progress across turns, sessions, and crashes.

2. Why markdown files (and when not to)

Markdown does three jobs at once:

Job	File	Purpose
Instructions	`AGENTS.md`	Who the agent is, how it behaves (same shape as Claude Code's `CLAUDE.md`)
Long-term memory	`memory.md`	Rolling facts and events the agent appends to instead of forgetting
Working state	`tasks/`	Plans and todos the agent checks off as it works

Why this is a good substrate

Zero infrastructure. No Postgres, no Qdrant, no hosted memory service. A folder and a text editor. This survives reboots, crashes, and cloud provider outages by construction.
Human-inspectable. You open memory.md and see exactly what the agent "knows". You can edit it directly to correct a fact — the cheapest form of steering ever invented.
Git-friendly. The entire brain diffs and versions cleanly. Roll back a bad memory write with git revert. Branch an experiment.
Tool-universal. Every language, every editor, every grep, every terminal handles text files. No driver, no client library, no schema migration.
Transparent. The agent's state is not a black box in a database; it is prose you can read while sipping coffee.

When markdown is the wrong answer

Corpus in the thousands of files and substring grep misses too often — add embeddings (Part 15). That's an upgrade, not a rewrite.
Multiple users, concurrency, ACID writes. Flat files get messy with concurrent writers. If two agents write memory.md at once you get interleaved lines.
Structured queries. "All tasks due before November with status active" is awkward in markdown and natural in SQL.
High write throughput. File locking and corruption risk grow.
Sub-second retrieval at scale. A vector index beats grep.

Rule of thumb: markdown is the right brain for a personal or single-tenant agent — a second brain, a project assistant, a research agent. For a shared, high-concurrency service, start with a real store.

3. Anatomy of a markdown brain

~/agent-workspace/
  AGENTS.md              # operating manual: role, rules, file conventions
  memory.md              # rolling long-term memory, one line per fact/event
  memory/                # deeper notes, one file per topic
    projects/smith.md
    people/jordan.md
  tasks/
    todo.md              # current open tasks
    plan-2026-08-11.md   # dated plan for one big task
  knowledge/             # interlinked reference notes (mini wiki)
  logs/
    2026-08-11.md        # timestamped diary of what ran

Design rules:

Keep paths flat where you can. Deep trees force the agent to know exact paths, which means more tool calls and more hallucinated paths. Two or three levels is plenty for a personal agent.
Use YAML frontmatter only when you filter or sort on those fields. It costs tokens to read and adds ceremony. If you never query "status: active", don't write it.

---
title: Smith project
updated: 2026-08-11
status: active
tags: [client, construction]
---

Link notes with [[wikilinks]] (Obsidian style). The agent follows them by regex — no graph database required. The link is just a hint about what file to read next.
One workspace per agent. For multi-agent setups, one folder per subagent (Part 13).

4. The agent loop

load AGENTS.md        -> system prompt (who am I, how do I behave)
load memory.md        -> long-term context (what do I know)
observe               -> user task + results of tool calls
think                 -> model picks the next action
act                   -> call a tool (read/write/search a file)
update                -> write results back to disk
repeat                -> until the model stops calling tools

In code, the loop looks like this:

messages = [system_prompt, user_task]
for step in range(max_steps):
    resp = client.chat.completions.create(model=..., messages=messages, tools=tools)
    msg = resp.choices[0].message
    if not msg.tool_calls:
        print("AGENT:", msg.content)      # done — it chose to answer
        append_to_memory(task, msg.content)  # write-back!
        break
    messages.append(msg_with_tool_calls)
    for call in msg.tool_calls:
        result = execute(call)            # read/write/search
        messages.append(tool_result(call.id, result))

Three details matter:

The system prompt is assembled from files at the start of every run. You never hardcode the agent's identity in code; you edit AGENTS.md. Changing the agent = editing a markdown file, not redeploying.
Tool results are appended to the message history, so the model sees what its actions returned before choosing the next action. That feedback is what makes the loop converge on a real answer instead of guessing.
The loop terminates when the model returns no tool calls. A max_steps cap (12 in the reference) is a hard safety valve so a looping agent costs you a bounded number of tokens.

5. The operating manual: AGENTS.md

AGENTS.md is the agent's constitution. It is loaded into the system prompt on every run, so keep it short — every token in it is resent on every single API call. Put the rules that must always be true here; put everything else in retrievable notes.

What belongs in it:

Identity — who the agent is and what it is for.
Behavioral rules — tone, length, what it must never do.
File conventions — where things live, how memory lines are formatted.
Reflection rules — when and how to compact memory (Part 6).
Guardrails — e.g., never write outside the workspace.

Example — an estimating assistant (tuned to a construction-cost-estimating workflow):

# AGENTS.md

You are SITE, a project assistant for construction cost estimating.

## Role
- Maintain project estimates, bid calendars, and client notes in this
  workspace.
- Answer questions from memory and the knowledge base; say "I don't know"
  when the files don't contain the answer. Never invent numbers.

## Rules
- Be terse. One paragraph per answer unless asked for detail.
- Never write outside this workspace.
- Never delete a file; overwrite with a .bak note instead.
- When you complete a task, append one line to memory.md:
  `- [YYYY-MM-DD] <task> -> <one-line outcome>`

## File conventions
- memory.md: one line per fact. Prefix with a date when it is an event.
- tasks/todo.md: `- [ ]` open, `- [x]` done, `[BLOCKED]` for waiting.
- knowledge/: one file per topic, wikilinks to related topics.
- Estimates live in knowledge/estimates/<client>-<project>.md

## Reflection
- On the last day of each month, rewrite memory.md as a tight summary,
  dropping facts more than 6 months old unless they are still referenced.

Note what is deliberately not here: prompts for specific tasks, detailed background knowledge, the entire estimate history. Those belong in knowledge/ and memory/ where they are loaded only when needed.

6. Memory design

memory.md is a rolling log, one line per fact or event:

- Workspace created 2026-08-01.
- [2026-08-11] Smith project: client approved revised framing estimate ($412k).
- [2026-08-11] Jordan prefers PDF summaries by 8am.
- Bid calendar: Q3 bids due 09-15 (Harborview), 09-22 (Meadowbrook).

Conventions that keep it useful:

One line per fact. Grep works, diffs work, compaction works.
Date events. [YYYY-MM-DD] prefix makes the log chronological and lets a compaction rule drop old events.
Facts vs events. Stable facts (client preferences, project parameters) and dated events (what happened when) have different lifetimes. Compaction should keep facts longer than events.
Append, don't rewrite — except during reflection.

Memory rot — the killer failure mode

The agent writes wrong or stale facts, then trusts them next run. The fixes:

A reflection rule (in AGENTS.md): periodically rewrite memory.md as a tight summary. The agent calls write_file on its own memory — this is the write-back at its most powerful.
A monthly review: a calendar reminder for you to skim memory.md and strike anything wrong. Two minutes, prevents months of compounding errors.
A provenance habit: when a memory line comes from a file, link it (see [[knowledge/estimates/smith.md]]) so it can be re-checked.
Source over memory: instruct the agent to prefer the current contents of project files over what memory.md says when they conflict.

Deeper notes: memory/

When a topic outgrows a line, promote it to a file:

memory/
  projects/smith.md      # the whole Smith story
  people/jordan.md       # preferences, contact, history

Keep the one-line pointer in memory.md (Smith project history: see memory/projects/smith.md), so grep still finds the topic and the agent knows where the detail lives.

7. Tasks and plans

Tasks live in tasks/todo.md with checkbox syntax that is simultaneously human-readable and trivially parseable:

# Todo

- [ ] Prepare Harborview bid package (due 09-15)
- [x] Update Smith framing estimate — done 2026-08-11
- [ ] [BLOCKED] Meadowbrook takeoff — waiting on revised drawings from architect
- [ ] Reconcile labor rates against Q2 actuals

Conventions:

- [ ] open, - [x] done (with date), [BLOCKED] with the reason.
The agent checks items off by editing the file — that edit is the state change that makes follow-up runs know where things stand.
Keep one todo.md; resist the urge to make a todo per project until a project genuinely has more than a screenful of open items.

Plan files for long tasks

For anything that takes multiple steps, the agent writes tasks/plan-<date>.md first:

# Plan: Harborview bid package — 2026-08-11

1. [x] Pull scope from knowledge/estimates/harborview.md
2. [ ] Run quantity takeoff against drawing list
3. [ ] Apply Q2 labor rates from knowledge/rates.md
4. [ ] Draft bid summary, flag risk items
5. [ ] Notify: summary PDF by 8am

Why this matters: a plan file survives crashes. If the process dies at step 3, the next run reads the plan, sees steps 1-2 checked, and resumes — instead of starting over or, worse, silently redoing and double-counting. Disk is the most reliable memory a computer has.

8. Knowledge base and wikilinks

knowledge/ is a mini-wiki of reference notes — the stuff the agent should know but that is too big for the system prompt:

knowledge/
  estimates/
    harborview.md
    smith.md
  rates.md
  subcontractors.md
  notes/
    walmart-division-25-takeoff-tricks.md

Wikilinks connect notes:

# Harborview

Concrete package per [[walmart-division-25-takeoff-tricks]] — mat slab
quantities use gross footprint, not net.

Budget: see [[smith]] for the framing rate history that carries over.

The agent resolves [[name]] by regex and reads the target file on demand. No graph database, no index to maintain — the links are just hints the model follows when it needs the detail. This is the same trick as Karpathy's llm-wiki and Obsidian's linking model, and it works because the model does the traversal, not a query engine.

9. Logs and the diary

logs/YYYY-MM-DD.md is a timestamped diary of what ran and what happened:

# 2026-08-11

- 09:00 ran daily bid-calendar check; Harborview moved from 09-22 to 09-15.
- 11:30 updated Smith framing estimate per client call; +$12k change order.

Logs answer "what did the agent do yesterday?" without trusting memory.md (which gets compacted). They are also the audit trail that makes memory rot detectable: if memory.md says one thing and the logs say another, the log wins. Keep them append-only; never let the agent rewrite history.

10. The minimal toolset

Four tools are enough to start:

Tool	What it does	Why it matters
`read_file`	Read a file from the workspace	The agent's senses
`write_file`	Write/overwrite a file (creates dirs)	The agent's hands
`edit_file`	Targeted change to one file	Cheaper than rewriting a big file
`search_files`	Grep the corpus for a keyword	Free RAG — substring retrieval

Grep is the quiet hero. It is the stand-in for a vector store until the corpus outgrows substring search, and for a personal agent that point is far away (Part 15).

Two tool-design rules:

Sandbox every path. All file tools resolve paths inside the workspace root (os.path.join(ROOT, args["path"])). The model never gets an absolute path escape hatch. One line of code, and the agent cannot touch your system files.
Descriptions are prompt engineering. The tool's description field is what the model reads when deciding which tool to call. "Find lines containing a keyword across all workspace markdown files" is a better description than "search" — be explicit about scope and behavior.

11. Reference implementation: agent.py

This is a complete, working agent (reproduced from the markdown-file-agent skill; verified to compile and to pass a 7/7 file-I/O harness). It uses the OpenAI SDK against any OpenAI-compatible endpoint (OpenRouter by default, SiliconFlow works too), so it works with any model those providers serve, including free tiers.

import os, json, glob, datetime
from openai import OpenAI

client = OpenAI(
    base_url=os.environ.get("OPENAI_BASE_URL", "https://openrouter.ai/api/v1"),
    api_key=os.environ["OPENROUTER_API_KEY"],
)

ROOT = os.path.expanduser("~/agent-workspace")
SYSTEM_FILE = os.path.join(ROOT, "AGENTS.md")
MEMORY_FILE = os.path.join(ROOT, "memory.md")


def read(path):
    try:
        with open(path) as f:
            return f.read()
    except FileNotFoundError:
        return ""


def write(path, content):
    os.makedirs(os.path.dirname(path), exist_ok=True)
    with open(path, "w") as f:
        f.write(content)
    return "written"


def grep(pattern):
    hits = []
    for p in glob.glob(os.path.join(ROOT, "**", "*.md"), recursive=True):
        with open(p) as f:
            for i, line in enumerate(f, 1):
                if pattern.lower() in line.lower():
                    hits.append(f"{p}:{i}: {line.strip()}")
    return "\n".join(hits[:20]) or "no matches"


tools = [
    {
        "type": "function",
        "function": {
            "name": "read_file",
            "description": "Read a markdown or text file from the workspace.",
            "parameters": {
                "type": "object",
                "properties": {"path": {"type": "string"}},
                "required": ["path"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "write_file",
            "description": "Write or overwrite a file in the workspace.",
            "parameters": {
                "type": "object",
                "properties": {
                    "path": {"type": "string"},
                    "content": {"type": "string"},
                },
                "required": ["path", "content"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "search_files",
            "description": "Find lines containing a keyword across all workspace markdown files.",
            "parameters": {
                "type": "object",
                "properties": {"pattern": {"type": "string"}},
                "required": ["pattern"],
            },
        },
    },
]


def run(task, max_steps=12):
    system = read(SYSTEM_FILE)
    memory = read(MEMORY_FILE)
    messages = [
        {
            "role": "system",
            "content": system + f"\n\n## Current memory\n{memory}",
        },
        {"role": "user", "content": task},
    ]

    for _ in range(max_steps):
        resp = client.chat.completions.create(
            model=os.environ.get("AGENT_MODEL", "openrouter/tencent/hy3:free"),
            messages=messages,
            tools=tools,
            tool_choice="auto",
        )
        msg = resp.choices[0].message

        if not msg.tool_calls:
            print("AGENT:", msg.content)
            write(
                MEMORY_FILE,
                memory
                + f"\n- [{datetime.date.today()}] {task} -> {msg.content[:200]}\n",
            )
            break

        messages.append(
            {
                "role": "assistant",
                "content": msg.content or "",
                "tool_calls": [
                    {
                        "id": tc.id,
                        "type": "function",
                        "function": {
                            "name": tc.function.name,
                            "arguments": tc.function.arguments,
                        },
                    }
                    for tc in msg.tool_calls
                ],
            }
        )

        for tc in msg.tool_calls:
            args = json.loads(tc.function.arguments)
            if tc.function.name == "read_file":
                result = read(os.path.join(ROOT, args["path"]))
            elif tc.function.name == "write_file":
                result = write(os.path.join(ROOT, args["path"]), args["content"])
            elif tc.function.name == "search_files":
                result = grep(args["pattern"])
            else:
                result = "unknown tool"
            messages.append(
                {"role": "tool", "tool_call_id": tc.id, "content": str(result)}
            )
    else:
        print("Stopped after max steps.")


if __name__ == "__main__":
    run(
        "Summarize what we know about the Smith project from memory, "
        "then add today's status note."
    )

Walkthrough

read / write / grep are the three real actions. write creates parent directories (os.makedirs(..., exist_ok=True)), which is the bug that bites every first implementation — writing to memory/projects/x.md fails silently if the folder doesn't exist.
grep walks **/*.md recursively and does a case-insensitive substring match, returning file:line: text hits capped at 20. The cap keeps tool results small enough to fit in context.
tools is a plain function-calling schema. Descriptions tell the model what each does; note search_files says "across all workspace markdown files" — scope in the description steers the model to use grep for recall instead of guessing.
The loop assembles the system prompt from AGENTS.md + memory.md each run, calls the API with tools, and appends assistant + tool messages. When the model returns no tool calls, it prints the answer and — the agentic step — appends a dated line about the task to memory.md.
max_steps=12 bounds the spend; the for...else prints a notice if the cap hits.

Setup

python3 -m venv .venv && .venv/bin/pip install openai
export OPENROUTER_API_KEY=your-key
export AGENT_MODEL=openrouter/tencent/hy3:free   # or any OpenRouter model
mkdir -p ~/agent-workspace/memory
echo "You are a terse project assistant." > ~/agent-workspace/AGENTS.md
echo "- Workspace created." > ~/agent-workspace/memory.md
python3 agent.py

That's the whole deployment story: one file, one folder, one API key. No server, no build step, no schema migration.

12. Verifying without burning tokens

You cannot exercise the live LLM call without a key and tokens, but you CAN verify the file-I/O and tool-wiring logic offline — and that logic is where real agents break. The skill ships scripts/verify_agent.py, which:

Sets a dummy API key so OpenAI() constructs without a network call.
Redirects the module's hardcoded ROOT to a temp directory.
Exercises read / write / grep plus the tool schema and memory load.
Hashes agent.py (sha256) so you can prove the file under test is the same one you verified before — re-run after every edit to catch silent drift.

.venv/bin/python scripts/verify_agent.py

This catches the failures that actually bite: write not creating nested dirs, read raising on missing files, grep case-sensitivity, malformed tool schema. The principle generalizes: test the deterministic 90% of your agent (tools, files, loop mechanics) offline; only the final decision-making needs the live model.

13. Patterns that make it useful

Reflection — the agent curates its own memory

At the end of a task (or on a schedule stated in AGENTS.md), the agent rewrites memory.md as a tight summary instead of letting it grow forever:

Before:  42 lines, 9 of them about one resolved issue
After:   "Harborview concrete: mat slab uses gross footprint (resolved 08-11,
         see knowledge/notes/walmart-division-25-takeoff-tricks.md)"

This is write_file called on the agent's own memory — the write-back loop turned inward. Without it, memory rots by accretion; with it, memory stays a signal, not a dump.

Planning files — crash survival

For long tasks, write tasks/plan-<date>.md first, then check items off. Because it's on disk, the plan survives crashes, and the next run resumes instead of restarting. This single pattern is what separates a demo agent from one you can trust with multi-hour work.

Retrieval escalation — grep first, always

Start with grep. Add embeddings only when substring search demonstrably hurts (synonyms, paraphrase, prose recall). For a personal workspace with hundreds of files, that day may never come — and if it does, the upgrade is surgical (Part 15).

One folder per subagent — coordination without a broker

~/agent-workspace/
  estimator/        # agent A: estimates
  scheduler/        # agent B: bid calendar
  shared/           # both read; neither writes

Subagents read each other's folders when coordination is needed and stay isolated otherwise. No message broker, no shared memory service — the file system is the bus. Agent A writes a completed estimate to shared/; agent B's next run greps shared/ and picks it up.

Scheduled operation

Because state lives in files, the agent can run headless on a cron job: the morning run reads memory, checks todos, greps for deadlines, and appends a status line — fully autonomous, fully inspectable afterwards in the logs.

14. Security and failure modes

Security

Sandbox the workspace. All paths resolve under ROOT; the model cannot reach system files. One line of code (os.path.join(ROOT, path)), non-negotiable.
Run with minimum permissions. The agent process should be able to write only its own folder.
Treat file writes as powerful. An agent that can rewrite its own memory can be steered by poisoned files. If you ingest untrusted content (web pages, emails), have the agent quote the source and never let ingested text override AGENTS.md rules.

Failure modes and mitigations

Failure	Symptom	Mitigation
Memory rot	Wrong/stale facts trusted next run	Reflection rule + monthly review + source-over-memory rule
Path hallucination	`read_file` on nonexistent paths	Flat layout, read returns "" not error, grep as recall
Loop runaway	Token spend, no answer	`max_steps` cap, tool-result size caps
Context overflow	Prompt too big	Short AGENTS.md, load only needed files, 20-hit grep cap
Concurrent writes	Interleaved lines	Single agent per workspace; lock file if needed
Poisoned memory	Agent acts on bad injected facts	Quote sources, provenance links, guardrail rules in AGENTS.md

15. Scaling path: grep -> embeddings -> database

The architecture is deliberately staged so you pay for complexity only when you need it:

Stage 1 — grep. Hundreds of files, exact terms. Free, instant, transparent.
Stage 2 — embeddings. Thousands of files, prose recall. Chunk + embed + store vectors in SQLite or a local vector store; search_files becomes semantic search. The file layout stays exactly the same — you swap one tool's implementation, not the design.
Stage 3 — database. Multi-user, structured queries, high write throughput. Move the state to a real store.

The hybrid that works best in practice: markdown stays the source of truth and the human-facing layer; a build step indexes it into a vector store for retrieval. Humans still read and edit markdown; the model searches the index. You get the best of both and you can regenerate the index whenever the corpus changes.

The key insight: each stage is an upgrade to a component, not a rewrite of the agent. The loop, the AGENTS.md contract, and the file conventions all survive.

16. Real-world precedents

This pattern is not exotic — the most widely deployed agent systems in the world use it:

Claude Code / Codex — CLAUDE.md / AGENTS.md project memory. The agent reads the file at session start; teams keep conventions, gotchas, and architecture notes in it. Millions of developers run this daily.
Karpathy's llm-wiki — an interlinked markdown knowledge base that an LLM queries and extends. Same substrate, same loop.
Obsidian + agents — the entire linking/backlink model is [[wikilinks]] over plain files; agent plugins read and write the same vault you do.
NotebookLM — ingests your documents and answers from them; the "memory" is your corpus, human-inspectable by construction.

What all of these share: the durable state is plain text on disk, the model is stateless between runs, and the files are the contract between the human and the machine.

17. A learning path

Day 1 — run it. Set up the workspace and agent.py from Part 11. Ask it to add a memory line and verify the line appears in memory.md.
Day 2 — customize. Write your own AGENTS.md (use the estimating example in Part 5 as a template). Change the agent's identity by editing a file — note that you never touched the code.
Day 3 — tasks. Give it a 3-step task and watch it write and check off a plan file. Kill the process mid-task; rerun; watch it resume.
Day 4 — retrieval. Add 20 knowledge notes, then ask questions that require grep. Watch the tool calls; see where recall fails.
Day 5 — reflection. Add a reflection rule, run it, and diff memory.md before/after.
Week 2 — autonomy. Put it on a schedule (cron). The morning run reads todos, greps deadlines, appends a status line. Inspect the logs.
Later — scale. Only when grep hurts, add embeddings (Part 15).

Appendix A. File-layout checklist

AGENTS.md — identity, rules, file conventions, reflection rule, guardrails
memory.md — one line per fact, [YYYY-MM-DD] on events
memory/ — deeper per-topic notes, one file per topic
tasks/todo.md — - [ ] / - [x] / [BLOCKED]
tasks/plan-<date>.md — written before long tasks
knowledge/ — reference notes with [[wikilinks]]
logs/ — append-only dated diary
All file tools sandbox paths under the workspace root
max_steps cap on the loop
Verification harness passes before first live run

Appendix B. Glossary

Agent loop — observe -> think -> act -> update -> repeat.
Write-back — the agent persisting the outcome of its work to disk; the step that makes it agentic.
AGENTS.md — the operating manual loaded as the system prompt.
Reflection — the agent rewriting/compacting its own memory.
Memory rot — stale or wrong facts accumulating and being trusted.
Wikilinks — [[name]] links between notes; followed by regex.
Tool sandbox — resolving all file paths under a single root so the agent cannot touch system files.
Grep-as-RAG — substring search standing in for semantic retrieval.