Mixture of Experts now dominates modern AI language model design

Original: cant avoid learning MoEs anymore. most of the llm architectures now are MoEs. heres a great article by maarten on mixture of experts with tons of illustration. https://t.co/QuhYqRfick

Source: x.com ↗