How Scorafy's AI Works
Transparency about our AI, its capabilities, and its limitations. We believe you should understand exactly how your assessment reports are generated before you trust them with your clients.
How AI reports are generated
Each report goes through four steps. Understanding this process helps you evaluate the output and use it effectively.
You create an assessment with your methodology
Define your rubric criteria, scoring rules, and methodology context. This gives the AI a structured framework to assess against - not a blank canvas.
Respondents complete the assessment
Each person answers your questions with their specific experiences, reflections, and self-assessments. No two sets of answers are alike.
AI reads every individual answer
Claude AI receives the respondent's complete answers, your rubric, your methodology context, and their scoring data. It processes everything together as a single, complete picture.
A structured report is generated
The AI produces a report with a summary, strengths, development areas, and recommendations - all referencing the respondent's actual answers. Every report is unique because every person's answers are unique.
Key point
Every report is unique because every person's answers are unique. This is not template-based feedback. The AI reads and responds to what each individual actually wrote.
What makes AI assessment reliable
AI-generated feedback is only as good as the structure you give it. Scorafy is built around principles that improve accuracy and reduce the risk of vague or unhelpful output.
Rubrics improve accuracy
Research shows AI accuracy on subjective assessment drops from approximately 55% to approximately 33% without rubrics. Scorafy's rubric builder gives the AI specific criteria to assess against, significantly improving the quality and consistency of feedback.
Evidence-based feedback
The AI is instructed to reference specific answers as evidence for its analysis. It does not produce generic statements like "add more examples" - it cites what the respondent actually said and builds its analysis from there.
Structured output
Reports follow a consistent format - summary, strengths, areas for development, and recommendations - ensuring completeness and making it easy for you to review and compare across respondents.
Human review by design
Reports are generated for the assessment creator to review before sharing. Scorafy is designed as a first draft, not a final word. You remain in the loop at every stage.
Known limitations
We believe honesty about limitations builds more trust than overclaiming capabilities. Here is what AI assessment cannot do well today.
AI cannot read context the way a human can
It does not know that a struggling student just produced their best work, or that an employee had a difficult week. Contextual factors that are obvious to a human are invisible to the AI unless explicitly captured in the assessment.
Language bias exists
Research documents that AI tools can show score discrepancies for non-native English speakers and speakers of non-standard English dialects. We take this seriously and encourage assessment creators to consider language diversity when designing questions and reviewing reports.
Subjective assessment is inherently challenging
AI performs best on structured criteria with clear rubrics. Open-ended, creative, or unconventional responses may not be assessed as accurately. The more structured your assessment, the more reliable the output.
AI is a tool, not a replacement
Professional judgement should always be the final word. Use AI reports as a starting point for deeper conversation, not as a definitive evaluation of a person's capabilities or potential.
Your data
Assessment data is sensitive. Here is how we handle it.
- Employee and client responses are never used to train AI models
- All data is scoped to your organisation with row-level security
- Role-based access control ensures only authorised team members see responses
- Files uploaded for analysis are processed and stored securely in encrypted storage
- You own your data. Export anytime via CSV or PDF.
For full details, read our Privacy Policy and Terms of Service.
Our commitment
We welcome feedback, questions, and challenges about our AI approach. If something in a report does not look right, or if you have concerns about how the AI handled a particular response, we want to hear about it. Reach out at adam@scorafy.com.