The Scottish Council for Voluntary Organisations is the membership organisation for Scotland's charities, voluntary organisations and social enterprises. Charity registered in Scotland SC003558. Registered office Caledonian Exchange, 19A Canning Street, Edinburgh EH3 8EG.
AI experiments should be a team sport
When I ask people in charities ‘who is using AI in your organisation?’ over 90% of people say ‘yes’. That’s a very big change from when I asked the question about a year ago. Then it was around 33-45%.
There’s still a big governance gap to close, as very few organisations have an AI policy or strategy in place. I blogged about that last year, sharing our own experience as a guide for others to learn from. The short version is:
- It’s definitely time for a policy, even if it is brief
- Use structured conversations to make a grounded and helpful policy
- Recognise that this early policy may not cover everything – but you can always add to it as you learn more
But this post is about the crucial value of ‘intentional experiments’.
If you can get this right, you’ll learn much more quickly as your team, and grow your collective confidence, too. One of the attractions of AI tools is that they are straightforward to use – just type a prompt or make an input, and you get some output. This is why folk have been able to explore and adopt these tools on their own. We’re all experimenting and learning quite quickly.
The problem most organisations have right now is that lots of people are experimenting and learning – but they’re not always sharing the results. Learning about AI ‘under the table’ can lead to an individual productivity boost but it means your insights don’t help your wider team. The ‘under the table’ approach to AI experiments is often driven from fear of doing something wrong or being seen to take a shortcut.
So the first step to intentional AI experiments is to create that positive culture around learning in the open – encouraging your team to try new things, and share what they’ve learned. Often, sharing something that didn’twork can be more valuable than sharing a success, because it helps other team members avoid the same problem, or brings in more ideas about what you could try differently next time. So work hard to give your team the confidence and psychological safety to learn and reflect openly.
The next piece is more practical – it’s about the need to keep a ‘scorecard’ of AI experiments as you go. Why is this important? An AI experiment scorecard is helpful because quite often, if you try something new with AI, you get a mixed result. It’s neither brilliant nor awful, but it’s somewhere in between. So you get lots of conversations with colleagues like
“Oh, I tried using CoPilot to help with that project”
“Interesting, how did it work?”
“Meh – it was OK, but there were some issues”
The problem with this kind of reflection is that it’s too vague to be useful. You can’t make decisions about future AI use if your colleague’s insight is ¯\_(ツ)_/¯
So how to make your reflections less vague? Enter the AI experiment scorecard! This sounds grand but it’s very simple. It’s just a list of focused questions to keep track of when you’re reflecting on something you tried with AI. Other orgs have developed more complex AI experimental canvases with Miro templates. But a list of questions in a Word document is fine! Here is our version:
Feel free to re-use, re-mix and adapt this. Or make your own. But if you can turn AI experiments and learning into a team sport, you’ll learn together much more quickly.
