4 stars | 1 forks | Python
52 Layers 4B(0.6B Active) MoE | Nemotron-3 Style + Teon Optimizer + Mamba-2 SSM + FP8 Training on H100
What it does
Open-Yaongi is a next-generation, compact language model specialized in Korean, math, and coding, utilizing advanced techniques for efficient training on H100 GPUs. Its hybrid architecture combines the strengths of Transformers and Mixture of Experts for enhanced performance in specific domains.
Why it matters: Discover how Open-Yaongi is revolutionizing Korean language processing with cutting-edge AI technology!
Want to create content about this repo? Use Nemati AI tools to generate articles, tutorials, and social posts.
Views13
On this page
Ali NematiBlogUI.author_articles
