Design Doc

Architecture, components, and design decisions for research-agent v2

Architecture Overview

research-agent is a single-file Python CLI app (~1600 lines) organized as a pipeline of specialized modules, each responsible for one phase of the research workflow. The GEPA self-improvement loop wraps the entire pipeline to evolve prompt strategies across generations.

┌──────────────────────────────────────────────────────────────────────┐
│                          CLI (argparse + rich)                       │
│   --eval │ --gepa │ --dogfood │ <query> │ --searx-instance <url>    │
└──────────────────────────────────────────────────────────────────────┘
                                    │
                          ┌─────────▼─────────┐
                          │   ResearchAgent    │
                          │  (orchestrator)    │
                          └─────────┬─────────┘
                                    │
              ┌─────────────────────┼──────────────────────┐
              ▼                     ▼                      ▼
    ┌─────────────────┐   ┌───────────────┐   ┌──────────────────────┐
    │   TaskPlanner    │   │  WebCrawler   │   │ ContradictionEngine  │
    │  (decompose      │   │  (DDG + WP    │   │ (cross-source        │
    │   query →        │   │   + SearXNG)  │   │  conflict detection) │
    │   sub-questions)  │   │               │   └──────────┬───────────┘
    └─────────────────┘   └───────────────┘              │
              │                     │                     │
              └─────────────────────┼─────────────────────┘
                                    ▼
                    ┌────────────────────────────┐
                    │   ReportSynthesizer         │
                    │   (structured markdown      │
                    │    report generation)       │
                    └────────────────────────────┘
                                    │
                                    ▼
                    ┌────────────────────────────┐
                    │   Report (output)           │
                    └────────────────────────────┘

┌──────────────────────────────────────────────────────────────────────┐
│                    GEPA Self-Improvement Loop                        │
│                                                                      │
│   ┌──────────┐    ┌──────────┐    ┌──────────┐    ┌──────────────┐  │
│   │ Evaluate │───▶│  Grade   │───▶│ Reflect  │───▶│   Mutate     │  │
│   │ Pool     │    │ Candidates│    │ (Lessons) │    │ (Next Gen)   │  │
│   └──────────┘    └──────────┘    └──────┬───────┘              │  │
│       ▲                                                          │    │
│       └──────────────────────────────────────────────────────────┘    │
└──────────────────────────────────────────────────────────────────────┘

Component Breakdown

TaskPlanner

Decomposes the user's query into sub-questions and a structured research plan. Uses heuristic templates (not LLM) to stay fast and deterministic. Generates 3-5 sub-questions targeting different angles.

WebCrawler

Multi-source web retrieval layer with three backends:

Backend	Source	Status
DuckDuckGo HTML	General web search	Primary (may hit CAPTCHA)
Wikipedia API	Wikipedia articles + search	Fallback (rate-limited)
SearXNG	Self-hosted instance	Optional (--searx-instance)

Uses asyncio.Semaphore for concurrency control, URL dedup cache, and HTTPX async client.

ContradictionEngine

Analyzes multiple source texts for conflicting statistics, contradictory claims, and divergent narratives. Detects numerical conflicts, polarity reversals, and query-encoded opposing viewpoints.

ReportSynthesizer

Generates a structured markdown report with executive summary, key findings, contradictions, and conclusions. Follows a deterministic template with dynamic content injection per sub-question.

GEPA Loop

The Generative Evolutionary Prompt Adaptation loop is the core self-improvement mechanism — it turns the agent into a system that writes its own improvements:

1. Generate — Create a pool of PromptCandidate variants by mutating prompt templates for each module
2. Evaluate — Run each candidate through the full eval suite (3 tasks × 4 checks each, deterministic grading)
3. Grade — Score each candidate on accuracy (pass rate) and speed. Pareto selection finds the best trade-offs
4. Reflect — Analyze failed checks to produce structured Lesson instances — which module failed, what check, and a suggestion for improvement
5. Mutate — Apply lessons to generate the next generation, with targeted mutations on failing modules and higher intensity in later generations
6. Select — Pareto frontier: prioritize accuracy, then speed
7. Self-Patch — Generate _self_patch.py that bakes winning prompts into the source script and bumps the version

Grading System

Two-tier quality scoring:

Grader	Method	Use Case
HeuristicGrader	Keyword-based scoring on report structure	Default — fast, deterministic, always works
OllamaGrader	Local LLM inference via Ollama API	When Ollama available — richer quality signal

The heuristic grader checks: executive summary presence, finding structure, source citations, contradiction analysis, and conclusion completeness. 4 checks per task, 12 total across the eval suite.

Eval Suite

Built-in evaluation tasks that exercise different research scenarios:

ID	Task	Query	Checks
eval-001	Ambiguous Health Claim	Is coffee good or bad for heart health?	4
eval-002	Emerging Technology Impact	Quantum computing impact on cryptography by 2030	4
eval-003	Historical Revisionism	Fall of the Roman Empire (economic, military, political)	4

Each task is run multiple times (--eval-runs N) for statistical significance. Pass rate and duration are reported per-task in a Rich table.

Design Decisions

Single file over package

Keeps deployment trivial: pip install git+... or just download the file. Self-patching is simpler without package imports. The module structure (classes + clear section headers) keeps 1600 lines navigable.

Heuristic over LLM (by default)

LLM calls add latency (5-15s per report), require GPU memory, and introduce nondeterminism. The heuristic grader is instant, deterministic, and sufficient for regression detection. LLM grading via Ollama is available as an upgrade path.

Separate self-patch file

Rather than modifying the script in-place during GEPA, we generate _self_patch.py — a standalone script that applies learned prompt improvements. This avoids string-collision bugs and gives the user a chance to review changes before applying.

State persistence via JSON

Research progress is saved to JSON state files. Enables resumability and debugging. Unique paths per run (research_state_{task.id}_{run_id}.json) prevent cross-contamination.

Data Flow

1. Input Query
2. TaskPlanner: decompose → [SubQ1, SubQ2, ...] + search queries
3. WebCrawler (fallback chain):
   a. Try SearXNG (if configured via --searx-instance)
   b. Try DDG HTML search
   c. Fallback to Wikipedia API
   d. Fetch page content (Semaphore(3) concurrency)
4. ContradictionEngine: cross-source analysis → conflict lines
5. ReportSynthesizer: template → structured markdown report
6. CLI output (Rich Panel / Markdown) + state file