AI, content design and the question of trust

I've been thinking a lot about AI recently. Not just how I might use it in my work as a content designer, but how it's changing the way people find and understand information.

I've also just signed up for a course where I'll hopefully build my first AI-powered tool, which is both exciting and a little daunting, considering I have absolutely no idea how to use AI in that way, but I suppose that's the point of the course.

However, the more I think about using AI in my own work, the more questions I have about how much we're starting to trust it.

When did we start trusting AI so much?

Have you noticed how quickly we've got used to AI-generated answers?

When you search for something on Google now, you're often presented with an AI overview before you even get to the search results. You ask a question and get what looks like a straightforward answer without having to visit several websites. It’s convenient and makes finding answers easier (or so we think).

Although I've started to wonder how often we accept those answers without questioning them. It's almost as though Google is saying, "Don't worry about looking through all those websites. I've found the answer for you." Sometimes this is fine, but what happens when the answer isn't quite right? Or when it leaves out an important bit of context?

What happens when AI summarises council information?

Working on local government websites, I regularly search for policies, guidance and information about council services.

I've noticed AI-generated overviews taking information from council web pages or PDFs and presenting a shortened version of it. That sounds helpful, and sometimes it is.

However, council information can be complicated. There are often conditions, exceptions and different processes depending on someone's circumstances. For example, something like appealing a decision, applying for a service or understanding whether you're eligible for support.

A sentence that makes sense within the context of a full page might mean something quite different when it's pulled out and summarised. It makes me wonder what happens if someone relies on that summary without visiting the original source. What important information might they miss?

I'm not suggesting every AI-generated answer is wrong. But I do wonder how many people are checking the information they're being given, particularly when it looks so convincing.

Does this make content design even more important?

We design website content expecting people to visit the website and read it. However, that's not necessarily how people are finding information anymore.

More and more of our content might be summarised by an AI system and presented somewhere else entirely. Someone might get what they believe is the answer without ever visiting the council website.

That got me thinking:

  • Are we writing and structuring content in ways that make it easier for AI systems to interpret accurately?

  • Are our policies and guidance clear enough when individual sections are taken out of context?

  • And how do we make sure information stays accurate when it's being reused in ways we don't necessarily control?

Of course, even well-designed content can be misinterpreted by AI. We can't control everything that happens once information is published. But clear, structured and well-maintained content surely gives these systems a better starting point.

All this makes me wonder whether we need to think more about the relationship between content design, structured content and how information is made available beyond a website.

Content designers are system designers too

I recently read an article ‘How my role in content design is expanding to systems design’ that got me thinking about how content designers are, in some ways, system designers.

We don't just write and edit content. We create standards, guidance, reusable patterns and processes that help organisations produce consistent, usable information. The aim is to create reusable patterns rather than solve the same problems over and over again.

That got me thinking:

  • could we build tools that help people apply those patterns consistently?

  • could AI help identify content that doesn't follow agreed standards, or flag inconsistencies across a service?

  • could it support content audits or help teams maintain large amounts of information?

Not to replace the judgement of content designers, but to take some of the repetitive work away so we have more time to understand people's needs, research problems and improve services.

Time to learn how to build something

With all that in mind, I've just signed up for the four-week Content Design AI Accelerator course with UX Content Collective. The idea is to take a real problem from your own work and spend four weeks turning it into a working AI-powered tool.

I've already completed the Generative AI course from Content Design London, which was a really useful introduction, but I'm keen to go a bit further and actually build something.

One idea I've been thinking about is an AI-powered content standards assistant for local government. Something that could review draft content against an agreed style guide, identify inconsistent terminology, suggest improvements based on approved guidance and explain why something doesn't meet a particular standard. It would also flag anything that needs human review rather than pretending it has all the answers.

If we're going to use AI to help people create and maintain content, we need to be clear about what it can do reliably and where human judgement is still needed.

I have no idea how realistic this idea is for a four-week build, but that's part of what I'm looking forward to finding out. Even if I only manage to build something that checks a handful of content standards, it would be a useful starting point.

Still lots of questions

I'm both cautious about how much we trust AI-generated information and excited about how it could support content design. For me, both come back to understanding what AI can and can't do, and knowing when human judgement is needed.

I'm looking forward to seeing what I can build on the course and sharing what I learn along the way.

I'd be interested to hear how other content designers are using AI, particularly if you've started building your own tools.

Next
Next

Weeknote: the value of asking questions