What AI can and cannot do

You have been hearing about AI for a while now, from people who are either certain it will change everything or certain it is nonsense. Both groups are usually selling something.

Here is the version without the sales pitch.

What it actually is

When people say AI today they nearly always mean a chatbot: ChatGPT, Claude, Gemini, Copilot. You type something, it writes something back.

Underneath, these systems were trained by reading an enormous quantity of text and learning which words tend to follow which other words, in context. When you ask a question, the system produces the response that fits the pattern of a good answer.

That description sounds dismissive, and it should not be. The results are genuinely impressive, and it can do things that are useful every day.

But it explains the one thing you most need to understand:

It is producing text that looks like a good answer. That is not the same as producing a true answer, and it cannot tell the difference.

What it is genuinely good at

Explaining things at your level. This is the best use, and the most underrated. Ask it to explain anything as though you know nothing, and it will, patiently, without ever making you feel slow for asking again. Ask it to explain the same thing three different ways and it will do that too.

Rewriting and drafting. Turning your rough notes into a polite letter. Shortening something. Making a complaint firm but not rude. It is a genuinely good first draft.

Summarising. Paste in a long document and ask what it says. Useful for terms and conditions, long reports and dense letters.

Translating.

Thinking out loud. Describe a problem and ask what you might be missing. It is a decent sounding board.

Getting started. The blank page is the hardest part of most writing tasks, and it removes it.

Where it fails, and fails confidently

It invents things. This is the big one. Asked something it does not know, it will not usually say so. It will produce a confident, plausible, well-structured answer that is wrong.

It invents facts, dates, statistics, quotes, and even citations to books and papers that do not exist. Lawyers have filed documents citing court cases that were never heard, because the output looked exactly like every other legal filing they had read.

It is bad at arithmetic. It handles numbers as text patterns, not sums. It often gets calculations right and sometimes gets them confidently wrong. Never trust it with figures that matter without checking.

It does not know about very recent events, unless it can search the web, and even then it may mix current information with older assumptions.

It agrees with you too readily. Push back on a correct answer and it will frequently apologise and change it. This makes it a poor judge of whether you are right.

It has no idea what it does not know. There is no internal sense of confidence. The tone is identical whether the answer is solid fact or complete invention. This is the single most important thing to internalise.

The one habit that makes it safe

Sort every task into one of two piles.

Pile one: it does not matter much if it is wrong. Drafting a letter you will read before sending. Explaining a concept you will recognise if it is off. Suggesting ideas. Rewording something. Use it freely here, and enjoy it.

Pile two: a mistake would cost you. Medical questions. Legal questions. Money. Anything you will act on, send to somebody important, or hand in. Use it here too, but check everything factual before you rely on it.

Checking is usually quick. Search for the specific claim, or ask a person who would know. The point is that verifying takes a minute and being confidently wrong can cost far more.

Things it is not

It is not a search engine, though it may look like one. A search engine shows you sources you can judge. A chatbot gives you an answer with no working. When you need a source, use a search engine, or ask for links and then actually open them.

It is not private by default. Assume anything you type may be stored and reviewed. There is a separate guide on what should never go into one.

It is not thinking, in the sense you mean by the word. Whether that matters philosophically is a genuinely open question. Practically, it means you should not assume it has understood the importance of anything.

Should you bother?

I think so, for one straightforward reason: this is turning up in workplaces regardless of what any of us think about it, and the gap is opening between people who understand what these tools do and people who either avoid them or trust them completely.

Both of those groups get caught out. The first gets left behind, and the second submits work with invented facts in it.

The useful position is the middle: a person who knows what to hand over and what to check. That is a learnable skill and it does not take long.

Where to start

Pick one real task this week. Not a test question to see if it can be tricked, which teaches you nothing, but something you actually need to do. A letter, an explanation of something confusing, a summary of a long document.

Then check the result properly, and see how it went. Repeat a few times and you will develop a fairly accurate sense of where it is reliable.

If you would rather work through it with somebody, that is what the AI lessons are for, and there is a group Introduction to AI class on Thursday evenings.