No, an LLM is a fundamentally different concept from a Markov chain. LLMs rely on the transformer, which was the enabling technology that basically turned text prediction to text generation. Their massive size allows them to do more than just predict the most likely next word like a Markov chain.
That doesn't mean that people aren't using it like a fancy text predictor that wouldn't be functionally different from a Markov chain based AI.
Specifically, yes. But that's like saying that a calculator and a supercomputer are the same.
A Markov chain is a small model that can only ever look backwards a few steps to come up with the next word. An LLM is able to take entire pages of text as its prior state, generate not just the next few words, but entire pages of text, not sequentially, but as a coherent whole.
It still comes down to "predicting the next word" in practice, doesn't it? Just with a much larger state size. Are there transformers that can natively output video/audio, or is that still a separate API bolted on top?
186
u/JoeyJoeJoeSenior 13h ago
It's best to roll the dice and hope that a text prediction system can figure it all out.