Microsoft is pushing to bring AI language models to phones by launching three new compact versions of its Phi language models
The open-source models come in 3 sizes:
mini (3.8B)
small (7B)
medium (14B)
These models, called Small Language Models (SLMs), are optimized for smaller datasets.
Their training data included 3.3 trillion tokens for the mini model, while the small and medium models were trained with 4.8 trillion tokens.
Nonetheless, they can rival larger models and have shown performance on par with Mixtral 8x7B and GPT-3.5.
Showcasing impressive performance details:
• MMLU scores: 69% for the mini, 75% for the small, and 78% for the medium.
• MT-bench scores: 8.38 for the mini, 8.7 for the small, and 8.9 for the medium.
Check out the paper -> https://arxiv.org/pdf/2404.14219.pdf
and the models -> https://huggingface.co/models?other=phi3&sort=trending&search=microsoft
