An AI prototype can look excellent on five carefully chosen examples. A production decision needs a broader answer: will it behave acceptably on the work your team actually receives?
Build the evaluation set first
Collect approved examples representing common requests, uncommon but important cases and inputs the system should reject or escalate. Remove unnecessary personal data. Record the expected result and why it is acceptable.
Separate the examples used to improve the system from a holdout set used for final assessment. Otherwise, the consultant may optimise for the demonstration rather than the underlying task.
Define more than one score
For a document extraction system, assess field accuracy, missing-field detection and whether uncertain results reach a reviewer. For a drafting assistant, assess factual grounding, adherence to instructions and editing effort.
Agree on which failures block launch. A formatting error and an unauthorised external action should not receive the same treatment. Capture latency and operating cost using the same test conditions for each candidate configuration.
Make the result reproducible
Ask for the model or service version, configuration, test date and scoring method. Have a person inspect a sample of both passing and failing cases. Automated grading is useful, but it can have its own blind spots.
Before acceptance, rerun the evaluation after one controlled change. The team should be able to see whether a prompt, source-data update or model change improves one area while harming another.
Your handover should include a test runner, a results summary, known limitations and the name of the person who owns future reviews. OpenAI's evaluation documentation offers one implementation reference; the business acceptance criteria should remain yours. Find AI evaluation support for a bounded assessment rather than an open-ended rebuild.