SiMa.ai says Palette 3.0 makes LLMs on its chips up to 3.5x faster
SiMa.ai has released Palette 3.0, the next version of the software for its Modalix AI chips. The San Jose start-up says language models on the chip now generate text up to 3.5 times faster. Power draw stays broadly the same. The gain comes from speculative decoding.
Read the full story on The Next Web.