19,948 stars | 1,771 forks | C
DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm
What it does
DwarfStar is a specialized inference engine for running large language models on consumer hardware, supporting Metal, CUDA, and ROCm backends. It optimizes performance through techniques like SSD streaming and tensor parallelism, making advanced AI capabilities accessible to users with limited resources.
Why it matters: 🚀 Dive into DwarfStar: a game-changing inference engine that brings advanced AI capabilities to consumer hardware, powered by Metal, CUDA & ROCm. #AIInference
Trending today with 187 new stars
Want to create content about this repo? Use Nemati AI tools to generate articles, tutorials, and social posts.
3
Contents
Ali NematiWritten by Ali
