100M Token Context Windows
Magic claims to have built a 100M token context model that’s 1000x cheaper than Llama at the same context length. Although there are still some question marks regarding evaluation, the increasing context trend is clear.
Posted on