GitHub Trending

lightseekorg/tokenspeed — TokenSpeed is a speed-of-light LLM inference engine.

38 sec read29 views0 listens

2,364 stars | 331 forks | Python

TokenSpeed is a speed-of-light LLM inference engine.

What it does

TokenSpeed is a high-performance LLM inference engine designed specifically for agentic workloads, aiming for speed comparable to TensorRT-LLM and usability like vLLM. Its key innovation is a separation of control and execution planes, enhancing both performance and safety, and it's backed by significant hardware vendor support. This makes it a compelling option for production environments requiring fast and reliable LLM inference.

Why it matters: 🚀 Supercharge your LLM agents! TokenSpeed delivers blazing-fast inference with a unique architecture & serious vendor backing. ⚡️

View on GitHub

Trending today with 25 new stars

Want to create content about this repo? Use Nemati AI tools to generate articles, tutorials, and social posts.

You Might Like

GitHub Trending2 days ago34 sec read

vllm-project/vllm — A high-throughput and memory-efficient inference and serving engine for LLMs

94,317 stars | 23,654 forks | Python A high-throughput and memory-efficient inference and serving engine for LLMs What it does vLLM is a high-throughput and memory-efficient inference and serving engine for Large Language Models (LLMs). It's designed...

Ali Nemati

GitHub Trending3 days ago46 sec read

headroomlabs-ai/headroom — Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20

76,077 stars | 5,913 forks | Python Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server. What it does Headroom is a p...

Ali Nemati

GitHub TrendingSep 1140 sec read

jihe520/MathModelAgent — 🤖📐专为数学建模设计的 Agent & skills ,自动完成数学建模，生成一份完整的可以直接提交的论文。 An Agent Designed for Mat

4,753 stars | 384 forks | Python 🤖📐专为数学建模设计的 Agent & skills ,自动完成数学建模，生成一份完整的可以直接提交的论文。 An Agent Designed for Mathematical Modeling ,Automatically complete mathmodel and generate a complete paper ready for submission. What it does MathModelAgent ...

Ali Nemati