Skip to content
Angelo Berlingieri

AI and product

Building AI tools people trust, and proving it before rollout.

I've built an AI tool and I evaluate them for a living. Both taught me the same thing: decide what good looks like before you start.

accuracy on a 120-question internal test set (target 80%)
100%
per answer
2-3 hrs → ~1 min
people on the support team using it
15

Selected work

1 case study

  • Grant Instruments

    How good is good enough? Setting the bar before building GrantIQ

    Read the case study

How I test AI before I trust it

  1. Check 01

    Set the bar first.

    Decide what “good enough” means before building anything.

  2. Check 02

    Try to break it.

    Throw the hardest questions at it, and check it admits when it doesn't know.

  3. Check 03

    Respect who sees what.

    It only shows people what they're already allowed to see.

  4. Check 04

    Be upfront that it's an AI.

    Clear rules on honesty, handing over to a person, and privacy.

→ See it in practice: the GrantIQ copilot case study

Right now

I'm a contractor with Mercor, which brings in experts to test and improve AI models for world-leading AI labs. My part is finance: I check the models' work the way a manager checks an analyst's. It's given me a clear sense of where AI is genuinely useful, where it's overkill, and where it still needs a human check.

Tools

  • Microsoft Copilot StudioBuilding the AI agent
  • SharePoint OnlineDocuments and permissions
  • Microsoft TeamsWhere the team uses it
  • NotebookLMThe first prototype

Certifications and courses

  • Python FoundationsUniversity of Cambridge
  • Project Management with SCRUMCambridge Judge Business School
  • Implementing Generative AI EthicallyCambridge MBA elective