Exabits and MyShell’s Breakthrough: From Billions to $100K in LLM Training Costs

Exabits has demonstrated its capability to train large language models (LLMs), partnering with MyShell to dramatically reduce training costs from billions to under $100,000. JetMoE-8B is trained at less than a $0.1 million cost but outperforms LLaMA2-7B from Meta AI (multi-billion dollar compute cost) MyShell: Achieving LlaMA2 performance with the $100,000 JetMoE model, inspired by the sparse activation […]