Jamba-Instruct

AI21 called it the first production-grade model built on Mamba, and the point of the architecture was arithmetic: state-space layers interleaved with attention carried two hundred and fifty-six thousand tokens without the memory cost that number normally implies. It was the argument that the Transformer was a choice rather than the only option, and it was on sale for thirteen months.