GitHub Trending

LMCache/LMCache — Supercharge Your LLM with the Fastest KV Cache Layer

32 sec read55 views0 listens

7,275 stars | 954 forks | Python

Supercharge Your LLM with the Fastest KV Cache Layer

What it does

LMCache is a high-performance KV cache layer for LLMs that significantly reduces TTFT and increases throughput by efficiently caching reusable text across various storage mediums, thus saving GPU cycles and reducing user response delays.

Why it matters: 🚀 Supercharge your LLM with LMCache - the fastest KV cache layer that slashes TTFT by up to 10x! 🚀 #LLM

View on GitHub

Trending today with 140 new stars

Want to create content about this repo? Use Nemati AI tools to generate articles, tutorials, and social posts.

cupy/cupy — NumPy & SciPy for GPU

11,955 stars | 1,062 forks | Python NumPy & SciPy for GPU What it does CuPy is a GPU-accelerated library for Python that serves as a drop-in replacement for NumPy and SciPy, enabling high-performance computing on NVIDIA CUDA or AMD ROCm platforms...

Ali Nemati

GitHub Trending19 hours ago31 sec read

opendatalab/MinerU — Transforms complex documents like PDFs and Office docs into LLM-ready markdown/J

72,103 stars | 5,839 forks | Python Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows. What it does MinerU is a high-accuracy document parsing engine that converts complex documents into st...

Ali Nemati

GitHub TrendingJun 529 sec read

MemPalace/mempalace — The best-benchmarked open-source AI memory system. And it's free.

53,741 stars | 7,073 forks | Python The best-benchmarked open-source AI memory system. And it's free. What it does MemPalace is an open-source AI memory system that stores conversation history verbatim and retrieves it with semantic search, offering ...

Ali Nemati

GitHub TrendingMay 1734 sec read

Light-Heart-Labs/DreamServer — Local AI anywhere, for everyone — LLM inference, chat UI, voice, agents, workflo

1,005 stars | 197 forks | Python Local AI anywhere, for everyone — LLM inference, chat UI, voice, agents, workflows, RAG, and image generation. No cloud, no subscriptions. What it does Dream Server is a local-first AI stack that enables users to run ...

Ali Nemati

GitHub TrendingMay 1415 sec read

NVIDIA-AI-Blueprints/video-search-and-summarization — Suite of reference architectures for building GPU-accelerated vision agents and

770 stars | 240 forks | Python Suite of reference architectures for building GPU-accelerated vision agents and AI-powered video analytics applications. View on GitHub Trending today with 28 new stars Want to create content about this repo? Use Nemat...

Ali Nemati

LMCache/LMCache — Supercharge Your LLM with the Fastest KV Cache Layer

What it does

Related Articles

cupy/cupy — NumPy & SciPy for GPU

opendatalab/MinerU — Transforms complex documents like PDFs and Office docs into LLM-ready markdown/J

MemPalace/mempalace — The best-benchmarked open-source AI memory system. And it's free.

Light-Heart-Labs/DreamServer — Local AI anywhere, for everyone — LLM inference, chat UI, voice, agents, workflo

NVIDIA-AI-Blueprints/video-search-and-summarization — Suite of reference architectures for building GPU-accelerated vision agents and