🍌 BananaMind-2-MoE Text Generation

A 25M parameter sparse Mixture-of-Experts base model by BananaMind. (Note: This is a base model, not chat-tuned!)

10 256
0.1 2
0.1 1
1 2