Mistral AI

wikipedia:Mistral AI

• Arthur Mensch, Timothée Lacroix and Guillaume Lample

• Products: Mistral 7B, Mixtral 8x7B, Mistral Medium

Mistral 7B uses Grouped-query attention (GQA) intended for faster inference and Sliding Window Attention (SWA) intended to handle longer sequences

Related

• ollama run mistral
• ollama run dolphin-mistral:latest

See also

• Mistral AI, Mistral 7B, Mixtral 8x7B
• LLM, MLLM, LoRA, LLaMA, LLaMA3, QLoRA, Falcon, PaLM 2, Gemini, Mixtral 8x7B, BitNet, Measuring Massive Multitask Language Understanding (MMLU), NVLM