All productsLive
Wraith
Your models, on your machine.
Wraith is a local model runtime for Windows. It serves an OpenAI-compatible API on the computer you already trust.
Features
- Installs, checks, updates and rolls back a pinned llama.cpp server.
- Picks CUDA, Vulkan or CPU from the hardware it finds.
- A GGUF library, with a downloader and a memory check before load.
- Live GPU and CPU telemetry, benchmarks and stability tests.
- An authenticated API on localhost. The key is protected with Windows DPAPI.
Stack
- .NET
- WPF
- SQLite
- llama.cpp
- MIT licence
Vulkan and CPU have been tested. CUDA and ROCm have not been verified yet.