System Architecture
┌──────────────────────────────────────────────────────────┐
│ ┌─────────────┐ ┌─────────────────┐ │
│ ┌──────────┐ │ FastAPI │──→│ In-Process │ │
│ │ CLI │────→ Server │ │ LLM (Thread │ │
│ │ vibecli │ │ │ │ Locked) │ │
│ │ .py │ │ Port: 8003 │ └───────┬─────────┘ │
│ └──────────┘ └───────┬───────┘ │ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ Watchdog │ │ Qwen2.5-3B │ │
│ │ watchdog.sh │ │ Q8_0 GGUF │ │
│ │ Health poll │ │ (3.1B params) │
│ └──────────────┘ └──────────────┘ │
│ │
│ Cache Layer: LRU (128 entries, 5min TTL) │
└──────────────────────────────────────────────────────────┘
Components
-
CLI
vibecli.py
8 commands:
prompt,chat,code,think,explain,sh,review,eval. Optimized for terminal use. -
Server
serve_vibethinker.py
FastAPI server on port
8003. In-process LLM with thread locking. LRU cache prevents model reload on repeat queries. -
Model
Qwen2.5-3B-Q8_0.gguf
3.1B parameter Q8_0 quantized model, 3.1GB. Runs on CPU via llama-cpp-python.
Data Flow
vibe prompt "list all pdfs in /tmp" # CLI
→ POST /v1/chat/completions # HTTP
→ serve_vibethinker.py # checks cache → LLM
→ LLM (thread-locked) # in-process inference
→ Qwen2.5-3B-Q8_0 GGUF # CPU model
→ <think>reasoning</think>answer # structured output
→ response JSON → CLI stdout # printed to terminal