Contents
In brief
A generative model continues a pattern it was trained to continue. That sentence explains both the useful demos and the failures that show up in production.
Original CloudExchangers explainer. It describes established public practice and labels the desk’s judgment as analysis. It is not a report of a newly announced event.
What “generative” means
A generative model estimates what is likely to come next in a pattern it has seen. For text, the pattern is a sequence of tokens. For an image model, the pattern is visual structure. For code, it is the regularities of programming languages and the repositories the model was trained on. The 2017 transformer paper, Attention Is All You Need, described the architecture that made large sequence models practical. It did not invent a system that knows whether a sentence is true.