Thinkorithm

What Today's AI Does Well — and Where It Falls Short

A balanced look at current capabilities and limitations, using the vocabulary that standards bodies have adopted for describing trustworthy AI.

Published · 2 min read

Public discussion of AI tends to swing between excitement and alarm. A calmer way to judge any system is to ask concrete questions about it: what it is good at, how it fails, and how anyone would know.

Where current systems are strong

  • Pattern recognition at scale. Classifying images, transcribing speech and spotting regularities in large datasets are mature uses of machine learning.
  • Working with language. Summarising, translating, rephrasing and drafting text are tasks where language models are often helpful, especially when a person reviews the result.
  • Narrow scientific problems. Where a problem is well defined and good data exist — protein structure or weather forecasting, for example — learned models have matched or exceeded earlier methods on important measures.

Where they fall short

  • Confident errors. Generative models can state falsehoods fluently. NIST’s generative AI profile calls this “confabulation”.
  • Brittleness. A system that performs well on test data may perform worse when real-world conditions differ from those it was trained and tested on.
  • Bias. Models learn from data that reflect past decisions and uneven representation, and can reproduce or amplify those patterns.
  • Opacity. It is often hard to explain why a model produced a particular output, which complicates accountability.

Questions worth asking

  1. On what data was the system tested, and how similar is that to the situation where it will be used?
  2. What happens when it is wrong, and who notices?
  3. Can its output be checked against an independent source?
  4. Does it treat different groups of people consistently?
  5. Is a person responsible for decisions that rely on it?

None of these questions requires technical expertise, and together they say more about whether a system is fit for purpose than any general claim about “intelligence”.

Sources

  1. National Institute of Standards and Technology. Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, 2023.
  2. National Institute of Standards and Technology. Generative Artificial Intelligence Profile, NIST AI 600-1, 2024.
  3. Jumper, J. et al. “Highly accurate protein structure prediction with AlphaFold”. Nature, 596, 583–589, 2021. doi:10.1038/s41586-021-03819-2
  4. ECMWF. 2025 forecast evaluation met with positive feedback, 2026.