Mixtral 8x7B

A sparse mixture of experts that held its own against far larger models while using only a fraction of its parameters on any given token, and that arrived as a torrent link before anyone had written it up. Mistral's endpoint for it closed; the weights did not. It is the one grave here with something still living in it.