KoboldCpp
Backend for running GGUF models locally with CPU and GPU support, API, and SillyTavern compatibility.
Features
- GGUF model loading
- CPU and GPU (CUDA, Metal)
- OpenAI-compatible API
- SillyTavern and other frontends
- Single executable
KoboldCpp runs GGUF-format LLMs locally on CPU or GPU. It exposes an API compatible with SillyTavern and other frontends, so users can chat or integrate local models into their workflows. KoboldCpp is distributed as a single executable for easy setup.
Similar tools
Anthropic API
paidDeveloper platform and API for Claude models with long context, tool use, and enterprise features.
Anyscale
paidRay-based LLM platform for training, serving, and scaling AI workloads in the cloud.
AutoGen
open-sourceMicrosoft's open-source framework for building multi-agent AI systems with conversational patterns and tool use.