Recall is not explanation
Most onboarding checks are quizzes: ten multiple-choice questions at the end of a module, with a pass mark. They are easy to build and to score, and they show that someone can recognise the right answer when it is in front of them. What they do not show is whether that person can explain the product to a customer who asks something slightly different from the training. A new hire can pass every quiz and still freeze on a first call. If the job is to explain the product, the certification has to ask for the explanation.
Step 1 - Decide what a certified person can do
Start with outcomes, not content. Write down what someone who has passed can do without notes. For most products that comes down to five or six things:
- Say what the product is and what it does, in one or two plain sentences.
- Say who it is for and which problem it solves for them.
- Explain the three or four features customers ask about most, and why each matters.
- Explain how it differs from the obvious alternative, including doing nothing.
- Handle the most common objection or limitation honestly.
- Point a customer to the right next step.
Step 2 - Write the rubric before the questions
Each outcome becomes a criterion, and each criterion gets three or four levels with a one-line description of what the answer contains at that level. For example, on differentiation: not yet - names no difference or a wrong one; developing - names a difference but not why it matters to the customer; certified - names the difference and ties it to the customer’s problem. Writing the levels first forces you to decide what good looks like before you see any answers, which is what keeps marking fair.
Step 3 - Ask open questions built on real situations
Base the questions on things customers actually say. "A prospect says they already use a spreadsheet for this. What do you tell them?" tests understanding far better than "Which of these is a feature?" Four to six questions is usually enough to cover the rubric. Let people answer in the format the job uses: written for support and success roles, recorded audio or video for sales and other spoken roles. Mark recorded answers from the transcript, so the score reflects what was said and not how someone looks or where they recorded.
Step 4 - Mark consistently, and show the evidence
Two reviewers marking the same explanation will often disagree, and that is normal with open answers. Two habits reduce it. First, calibrate: before the first real round, have every reviewer mark the same three or four answers and talk through the differences. Second, require evidence: each mark should point to the part of the answer that earned it. A mark with a quote attached can be checked, challenged and learned from. A bare number cannot.
Step 5 - A named person signs off every result
Certification is a claim your organisation makes about a person, so someone has to own it. Whatever does the first pass of marking, whether a peer, a manager or an AI tool, a named assessor should review the marks, change anything they disagree with, and release the result. Record who changed which mark and why. That record answers the questions that follow later: why one person passed and another did not, and what the marking was based on.
Step 6 - Check again at 30, 60 and 90 days
One certification at the end of onboarding shows what someone knew that week. Short repeat checks at 30, 60 and 90 days, using the same rubric with fresh questions, show whether understanding held up once real customer conversations started. Comparing the same criteria over time also shows you which parts of onboarding are not landing for anyone, which is often more useful than any individual result.
Where software helps
The hard part of this method is marking time. Open answers take far longer to mark than quizzes, and the people qualified to mark them have day jobs. AI marking tools can draft the marks against your rubric and quote the evidence, so reviewers check and decide rather than read every answer from scratch. Whichever tool you use, check that it marks against your own rubric rather than a generic one, shows the evidence for each mark, and keeps a person in charge of release. Scorafy is built that way. It also runs live, assessor-paced certification sessions where staff join by QR code, and marks recorded answers from the transcript. See Scorafy for staff certification.
Common questions
- How do I check that a new hire can explain our product?
- Ask them to explain it, in writing or on a recording, then mark the answer against a rubric that says what a good explanation contains. Use open questions based on real customer situations, define three or four levels per criterion, have a named assessor sign off each result, and re-check at 30, 60 and 90 days.
- Why is a multiple-choice quiz not enough for product certification?
- A quiz shows that someone can recognise the right answer when it is in front of them. Customers never offer four options. They ask an open question and need a clear explanation in plain words, so the certification has to ask for the explanation itself.
- Should certification answers be written or recorded?
- Use whichever matches the job. Support and success roles mostly explain things in writing, while sales and customer-facing roles explain them out loud. Recorded answers are best marked from a transcript, so appearance, accent and setting do not affect the score.
- How do you keep marking consistent across reviewers?
- Write the rubric before the questions, give each level a short description of what the answer contains, calibrate reviewers on a few shared answers before the first real round, and record every change a reviewer makes to a mark along with the reason.
- Can AI mark product certification answers?
- AI can draft the marking against your rubric and quote the part of the answer behind each mark, which saves reviewer time. A person should still review the marks and decide whether someone is certified, because the certification is your organisation’s judgement and someone has to be accountable for it.