Developer Antoine Zambelli has released Forge, an open-source reliability framework that raises 8B local language model tool-calling success rates to 84% across a 26-scenario evaluation suite. By injecting execution guardrails like rescue parsing and synthetic tool calls, the framework transparently fixes malformed outputs from self-hosted models. Developers can immediately plug the project into existing workflows to make smaller models perform reliably without rewriting code.
Guardrails for Local Tool-Calling
Forge acts as a lightweight, domain-agnostic reliability layer for self-hosted LLM tool calling. Sitting between user clients and local model backends, Forge enforces execution constraints without functioning as a complex multi-agent orchestrator.
Across Forge's 26-scenario v0.7.0 evaluation suite, Forge lifts an 8B local model from single-digit reliability to an 84% success rate. In earlier v0.6.0 testing, the guardrail stack also boosted Claude Sonnet 4.6 performance from 85% to 98% on the same workload.
How Forge Fixes Model Errors
Forge applies four automated guardrail mechanisms to prevent local models from failing during tool-calling tasks:
- Rescue Parsing: Converts malformed tool calls—such as Qwen
<tool_call>XML tags, Mistral[TOOL_CALLS]syntax, or fenced JSON—into canonical OpenAI format. - Response Validation: Verifies generated tool names and argument schemas against defined tools, catching malformed shapes before returning responses.
- Automated Retry Loop: Sends corrective error messages back to the backend up to three times by default when validation fails.
- Synthetic Respond Tool Injection: Forces ~8B models to call a synthetic response tool instead of bare text, ensuring reliable switching between text outputs and tool executions.
Implementation and Compatibility
Developers can deploy Forge as a drop-in proxy server, via the WorkflowRunner Python engine, or as composable middleware inside custom loops. In proxy mode, Forge serves both OpenAI chat completions and Anthropic /v1/messages APIs, allowing tools like Claude Code, Continue, aider, and Cline to benefit without code modifications.
The framework requires Python 3.12+ and supports multiple local backends, including llama-server, vLLM, Ollama, Llamafile, and Anthropic's API. For shared hardware setups, its SlotWorker component provides priority-queued GPU access with auto-preemption.
Current Architectural Limitations
While proxy mode allows quick integration with existing harnesses, it operates on a single-shot basis per request. Consequently, proxy mode cannot enforce multi-turn workflow constraints like step-ordering, prerequisite rules, or context compaction—capabilities that require running directly. Furthermore, Sonnet 4.6 benchmarks were not re-tested on the v0.7.0 suite due to non-trivial API costs.