Turbopilot is a local, open-source code completion engine that utilizes the llama.cpp library to run large language models (LLMs) like Salesforce's CodeGen locally on the CPU. It aims to provide a fast, privacy-focused alternative to cloud-based code completion tools. The project is inspired by the fauxpilot project and provides a way to run powerful LLMs on consumer hardware.
Turbopilot supports multiple models including StableCode 3B Instruct, WizardCoder, Starcoder, and Santacoder. It offers easy setup via Docker and supports both CPU and GPU inference. The codebase has been refactored for easier extensibility and model integration. It's adaptable to different model flavors and allows flexible configuration of CPU core usage.
- Core Functionality: Provides real-time code completion suggestions.
- Supported Platforms: Linux and macOS.
- Model Support: Compatible with various LLMs such as CodeGen, StableCode, WizardCoder, and Starcoder.
- Configuration: Customizable CPU core usage and model selection.
- Performance: Optimized for CPU inference with optional GPU acceleration (CUDA).
- Developer Experience: Easy setup with Docker and API access for integration with VS Code plugins.
- API Access: Provides an API endpoint for programmatic code completion requests.
The project is currently archived as of September 30, 2023. While actively developed previously, the maintainers have recommended alternative solutions. Community support is available through issue reporting and contributions to the ggml fork. Documentation is sufficient for basic usage but may require further refinement.
Turbopilot benefits developers seeking a privacy-focused, local code completion solution. It addresses the need for offline code assistance, especially when dealing with sensitive code or limited internet connectivity. It provides a valuable alternative to cloud-based options by executing locally and controlling data flow.