Categories
AI Content management Copilot Microsoft 365 Mobile apps

AI and Handwritten Notes

AI has become great at many things; it can even transform terrible handwriting into typewritten text. I share the journey of discovery from scrawl to near perfect transcription.

A picture of cursive handwriting flowing from a page into digital text and imagery

Document digitisation has come a long way

A few days ago I opened a Coventry and Warwickshire Chamber of Commerce conference on AI and Cybersecurity with my perspectives on AI past, present and future.

While listening to another speaker I scribbled down some ideas for a blog around humans in the loop, based on prior discussions with other Microsoft MVPs and some comments in the conference. While I’m quite pleased with the blog idea I didn’t fancy typing it up from my paper notes. On the other hand, my handwriting, which was always dire, has declined through lack of use during the IT Age. If only AI could read such scrawl.

I’m always up for a challenge, so grabbed my phone, spun up the M365 Copilot app, which has a rather excellent document scanning capability, and digitised my handwritten notes.

A sequence of screenshots of the M365 Copilot app, used to scan a Post it note, showing its automatic edge detection and document framing.

These are the actual captures from what I wrote – feel free to be judgemental about my writing:

The handwritten page of notes captured using the M365 app Scan feature.
The second handwritten page of notes captured using the M365 app Scan feature.

I used the Copilot Chat app on my phone (public Copilot) to transcribe my handwriting, uploading the images I captured. The prompt couldn’t have been simpler:

“What does this say, word for word”

The response was impressive. This is what it extracted from the 2 pages of hieroglyphs:

Human in the Loop (HITL)

AI is changing the way we do business.

There is an appropriate concern for AI governance (AI), especially AI Agents (Agentic AI), where the activities are initiated & completed by an Agent. There is talk globally of the need for Humans in the Loop. What is really needed is where we have HITL with human-driven processes. Oversight, supervision, sign-off, enhancement analysis, failure mode analysis, etc.


We consider when we insert to human oversight. We try to avoid micromanagement. Ideally we ask ‘Who watches the Watchmen?’ i.e. having oversight of the HITL to ensure that oversight is effective. Not all HITL humans are very fallible. 

The same approach should surely apply to AI agents, with similar oversight. After all, AIs are fallible. 

Given this apparent symmetry, perhaps we should also have AIs in the oversight loop. AIric. 

As we move from AI as servant through AI as assistant to AI as partner, we need to consider not only pronounced HITL but also AIITE at both the process and executive oversight level.”** 

It also had a go at interpreting the diagram, though sadly didn’t create a clever new version.

“It looks like the image discusses the importance of human involvement in AI-driven processes. Does this align with what you were hoping to learn from it?”

It even offered a handy analysis:

“It looks like the note discusses human oversight in AI systems, the need to avoid micromanagement, and even the idea of having AI involved in the oversight process itself. There’s also a reference to Coventry & Warwickshire Chamber of Commerce in the background. Does this align with what you were hoping to get from it?”

Screenshot of Copilot analysing the first page of my terrible handwriting
Screenshot of Copilot analysing the second page of my terrible handwriting

I challenge you to do better.

Let’s be critical. It’s not perfect. It has a handful of words wrong and struggled with my newly created acronym, AIITL (AI in the Loop). I challenge you to do better.

With the ready availability of typed, touchscreen and voice interfaces I’m unlikely to do this very often. However, I am suitably impressed with its attempt to extract content and context from even the worst of handwriting.

Score one for Artificial Intelligences.

Simon's avatar

By Simon

Simon Hudson is an entrepreneur and health sector specialist. He formed Cloud2 in 2008 following a rich career in the international medical device industry and the IT industry. Simon’s background encompasses quality assurance, medical device development, international training, business intelligence and international marketing and health related information and technology.

Simon’s career has spanned both the UK and the international health industry, with roles that have included quality system auditing, medical device development, international training (advanced wound management) and international marketing. In 2000 he co-founded a software-based Clinical Outcomes measurement start-up in the US. Upon joining ioko in 2004 he created the Carelink division and, as General Manager, drove it to become a multi-million pound business in its own right.
In 2008, Simon founded Cloud2 in response to a need for a new way of delivering successful projects based on Microsoft SharePoint. This created the first commercial ‘Intranet in a Box’ solution and kickstarted a new industry. He exited that business in 2019, which has continued to grow as a leading provider of Power BI and analytics solutions.

In 2016, he co-founded Kinata Ltd. to enable effective Advice and Guidance in the NHS and is currently guiding the business beyond its NHS roots to address needs in Her Majesty’s Prisons and in Australasia.

In 2021, Simon founded Novia Works Ltd.

In 2021 he was invited to become Entrepreneur in Residence at the University of Hull.

In 2022 he was recognised as a Microsoft MVP.

In 2025 he founded Sustainable Ferriby CIC, a community energy not-for-profit to develop energy generation, energy & carbon reduction, and broader sustainability & NetZero projects in the West Hull villages.

Simon has had articles and editorials published in a variety of technology, knowledge management, clinical benchmarking and health journals, including being a regular contributor to PC Pro, as well as a presenter at conferences. He publishes a blog on areas of interest at noviaworks.co.uk. He is a co-facilitator of the M365 North User Group. He is a lead author and facilitator on the Maturity Model for Microsoft 365. He is the author of two patents relating to medical devices. He holds a BSc (Hons) in Physical Science and a PGCE in Physics and Chemistry from the University of Hull.

Simon is passionate about rather too many things, including science, music (he plays guitar and octave mandola), skiing, classic cars, narrowboats, the health sector, sustainability, information technology and, by no means least, his family.

Leave a comment