The "Heavy Lifting" Protocol
How the Chatbot Optimizes VRAM
The Arcade Companion isn't a single AI—it's a multi-stage intelligence pipeline. Learn how the lightweight Chatbot Engine shields the heavy LLM from resource exhaustion.
The VRAM Challenge
Large Language Models (LLMs) are memory-intensive. Every line of code they read takes up space in your GPU's KV Cache. If the AI reads a full 10,000-line file, your VRAM spikes, causing lag.
Standard AI
Reads the entire file for every request. High latency, massive VRAM usage, frequent crashes on 8GB cards.
Signal Share "Heavy Lifter"
Uses the Chatbot to "Spot" the relevant code first. Only sends a small 40-line snippet to the LLM.
The Tag-Team Architecture
We've split the brain into two parts: The Spotter and The Surgeon.
Lightweight Chatbot Engine. Uses regex and string matching (0MB VRAM).
Heavy Local LLM. Only wakes up when the Spotter provides exact coordinates.
How it works step-by-step:
- Intent Detection: The Chatbot intercepts your message. If it sees a search intent (e.g., "Where is the score?"), it runs a local search immediately.
- Context Pruning: The Chatbot finds the target line in your editor and grabs +/- 20 lines around it.
- Surgical Injection: The Chatbot injects only that snippet into the AI's memory.
- Fast Inference: Because the LLM is only reading 40 lines instead of 4,000, it generates the fix in milliseconds.
Optimization Results
By letting the Chatbot do the heavy lifting of searching and scrolling, we've achieved:
- Instant Navigation: The editor jumps to the target line before the AI even finishes "thinking."
- Low Context Pressure: The LLM context window stays below 2,000 tokens for most edits.
- Anti-Hallucination: The AI is forced to work with the *real* current code block found by the chatbot, not a cached memory.