antirez/ds4

DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm

C22K stars2.1K forks+187 today

What it does

DwarfStar is a specialized inference engine for running large language models on consumer hardware, supporting Metal, CUDA, and ROCm backends. It optimizes performance through techniques like SSD streaming and tensor parallelism, making advanced AI capabilities accessible to users with limited resources.

Star history

2026-08-23 → 2026-09-12+609 stars

Tracking

Last trending: 2026-08-02

Creator kit

Hook

🚀 Dive into DwarfStar: a game-changing inference engine that brings advanced AI capabilities to consumer hardware, powered by Metal, CUDA & ROCm. #AIInference

Content angles

A deep dive into how DwarfStar optimizes performance on different hardware architectures.
Comparative analysis of DwarfStar vs other inference engines in the market.
Tutorial series: Setting up and running DeepSeek V4 Flash models with DwarfStar on your Macbook or workstation.

Who should care

AI enthusiasts, developers interested in machine learning inference optimization, researchers working with large language models.