Thinkorithm

Generative AI Beyond Text

The same broad ideas that produce text now generate images and sound, predict protein shapes and forecast the weather. How the main approaches work.

Published · 2 min read

“Generative AI” usually brings chatbots to mind, but systems that produce new content now work with images, audio, molecules and physical data. They rely on a few families of methods that are worth knowing by name.

Two players in a game: adversarial networks

In 2014 Ian Goodfellow and colleagues proposed generative adversarial networks. Two networks are trained together: a generator produces candidate samples, and a discriminator tries to tell them apart from real examples. Each improves by competing with the other, until the generator’s output becomes hard to distinguish from real data. The approach produced the first strikingly realistic synthetic images.

Learning to remove noise: diffusion models

Many current image and audio generators use diffusion models, described in an influential form by Jonathan Ho, Ajay Jain and Pieter Abbeel in 2020. During training, real examples are gradually corrupted with random noise, and the model learns to reverse that process one small step at a time. To create something new, the model starts from pure noise and removes it step by step. A text description can guide each step, which is how text-to-image systems work.

Beyond media: science and forecasting

Methods from the same field are now used on scientific problems where the “content” is a physical prediction rather than a picture.

  • Protein structures. AlphaFold predicts the three-dimensional shape of a protein from its sequence of amino acids. According to the Nobel committee, its developers have predicted structures for almost all of the roughly 200 million known proteins.
  • Weather. In February 2025 the European Centre for Medium-Range Weather Forecasts made its machine-learning forecasting system, AIFS, operational alongside its physics-based model. ECMWF reports that producing a forecast this way uses roughly a thousand times less energy. An ensemble version — a set of many forecasts run with slightly different starting conditions, showing the range of possible outcomes — followed in July 2025.

Strengths and cautions

Generative systems are useful for drafting, illustration, accessibility tools such as speech synthesis, and exploration in science. The same ability to produce realistic content creates risks: convincing fakes, unclear ownership of training material and output, and errors that look authoritative. These questions are discussed further in AI and Society: Trust, Information and Fairness.

Sources

  1. Goodfellow, I. et al. “Generative Adversarial Nets”. Advances in Neural Information Processing Systems 27, 2014.
  2. Ho, J., Jain, A., Abbeel, P. “Denoising Diffusion Probabilistic Models”. Advances in Neural Information Processing Systems 33, 6840–6851, 2020.
  3. Nobel Prize Outreach. The Nobel Prize in Chemistry 2024 — Popular information.
  4. ECMWF. ECMWF’s AI forecasts become operational, 2025; ECMWF’s ensemble AI forecasts become operational, 2025.
  5. Regulation (EU) 2024/1689 (Artificial Intelligence Act). EUR-Lex.