Jachai : Any Document to a Graded Quiz

Overview

Jachai takes any piece of source material — a pasted passage, an exported PDF, or a photograph of a page — and produces a complete, well-formed quiz: a title, questions, options, and a correct-answer key. Authors tune the difficulty, question count, and answer style; participants take the quiz from a public link with no account; and every attempt is scored automatically and exported to PDF or Excel.

The product hinges on one thing most “AI quiz” demos gloss over: the model’s output has to be structured data the rest of the application can trust. A quiz is not a paragraph — it is a typed object whose answer key a grader will read. So Jachai treats the LLM as a question writer bounded by a strict contract, and keeps everything that has to be correct — validation, scoring, negative marking, totals — in plain, deterministic code. This study focuses on the two parts that took the most engineering: the generation pipeline that turns messy input into schema-valid quizzes, and the deterministic take-and-grade flow behind every shared link.

Business Challenges

• Writing Assessments Is Slow, Manual Work – Educators and trainers already have the material — a chapter, a policy PDF, a set of slides — but turning it into a fair, well-formed quiz means hand-writing every question, option, and answer key. The content exists; the bottleneck is the busywork.
• Content Lives in Every Format Except Plain Text – The source is rarely a tidy paragraph. It is a scanned handout, an exported PDF, a photo of a whiteboard. A tool that only accepts typed text puts the transcription work back on the author before they can even start.
• Grading and Distribution Don't Scale – Sharing a quiz with a class, collecting every attempt, scoring each one, and exporting the results for a gradebook is repetitive overhead that grows linearly with every participant.

Technical Challenges

• LLM Output the App Can Actually Consume – A quiz is structured data — questions, options, and a correct-answer key the grader depends on. A model that returns a friendly paragraph, or JSON with a missing field, breaks the entire downstream pipeline. The output has to be machine-shaped every single time.
• One Generator, Many Input Formats – PDF, image, and text each need a different extraction path, but the generation logic should see none of that difference — it needs one clean text field no matter how the content arrived.
• Single- and Multiple-Answer in One Model – Single-correct and multiple-correct questions must be generated on demand and, more importantly, graded correctly — a partial selection on a multi-answer question is wrong, not half right.
• Scoring Must Be Deterministic and Trustworthy – Marks, negative marking, and totals decide a participant's grade. None of that can depend on a model's judgement — it has to be plain, auditable code that produces the same result every time.

Our Approach

The design philosophy was “let the model write, let the code decide.” Generation is the one place a language model is genuinely better than a rule — so it writes the questions. Everything downstream of that — whether the output is even a valid quiz, how an answer is scored, how a penalty is applied, how a total is summed — is handled by ordinary, testable code. The result is a system where the creative step is fast and flexible, and the parts a grade depends on are boring, predictable, and auditable.

Discipline & Contributions

Meeting the Content Where It Lives

Authors don’t start with clean text — they start with whatever they already have. Jachai accepts four kinds of input and collapses them all into one plain-text field before generation ever begins, so the generator never has to know or care how the content arrived.

• Typed or pasted text. The simplest path — the text goes straight into the pipeline.
• PDF extraction. Uploaded PDFs are stored in Google Cloud Storage and run through pdf-parse to pull their text content out server-side.
• Image OCR. A photo or scan is read by GPT-4o-mini’s vision model with a prompt that asks for the raw extracted text and nothing else — turning a picture of a page into usable source material.
• One unified text field. Whatever the route, the output is the same: a single normalised string. The generation stage has exactly one input shape to reason about.

The Generation Pipeline

This is the centrepiece. The hard part is not asking a model for questions — it is guaranteeing that what comes back is a valid quiz the application can store, serve, and grade without a human checking it first.

• Parameters compiled into the prompt.The author’s choices — difficulty, number of questions, options per question, and single- vs multiple-answer — are compiled deterministically into a detailed system prompt, so the same controls always shape the request the same way.
• A pinned output contract. The prompt fixes the exact shape of the response — { title, questions: [{ question, options, answers }] } — and the request is sent with response_format: json_object, so GPT-4o-mini returns machine-parseable JSON rather than prose.
• Language and answer-count fidelity.The quiz is generated in the language of the source text, and multiple-answer mode is enforced end-to-end so a “choose all that apply” question is never silently reduced to a single correct option.
• Validated and normalised in code. The returned JSON is parsed, shape-checked, and given stable question ids server-side before it is ever written to MongoDB — a malformed generation is rejected rather than persisted as a broken quiz.

Taking & Grading, Without the Model

Once a quiz exists, generation is over — and so is the LLM’s involvement. Sharing, taking, and scoring are pure application logic, because a participant’s grade is not something that should depend on a model’s mood.

• Login-free participation. A quiz is shared as a public link; participants open it, enter their identity, and answer — the questions are served without the correct answers attached, so the key never reaches the browser.
• Exact-set scoring.Grading sorts the correct-answer set and the participant’s selection and compares them for exact equality — the same rule correctly handles single- and multiple-answer questions, treating a partial multi-answer selection as incorrect.
• Optional negative marking. When enabled, a configurable penalty is subtracted for wrong answers, and the final, sub, and negative scores are computed and stored as plain numbers — fully auditable.
• Results, charted and exported. Authors see every attempt with Chart.js dashboards, and can export a single result as a PDF (rendered with Puppeteer) or all results for a quiz as an Excel workbook (ExcelJS).

The Rest of the System

The generation and grading paths are the centrepiece, but they sit inside a complete product — a Nuxt 3 single-page app, a Firebase-secured Express API, and a typed data layer that keeps quizzes, results, and participants consistent.

• Authoring & dashboard.The Nuxt 3 SPA lets authors create, edit, and manage quizzes, fix any individual generated question inline, and track attempts — with Pinia holding the signed-in user’s state across pages.
• Firebase-verified API.Every protected route runs behind middleware that verifies the caller’s Firebase token and checks resource ownership, so one author can never read or edit another’s quizzes and results.
• Typed data layer. Prisma models a User, Quiz, Result, and Participant over MongoDB, with a credit system that meters question generation and submissions per account.

Technology Stack

• Frontend & UI: Nuxt 3, Vue 3, Pinia, Tailwind CSS, Chart.js, vue3-toastify
• Backend API: Node.js, Express, Prisma ORM, structured route handlers with authenticate / authorize middleware
• AI & Parsing: OpenAI GPT-4o-mini (JSON-mode generation + vision OCR), pdf-parse, Google Translate API
• Data & Storage: MongoDB, Google Cloud Storage for uploads, typed Prisma schema (User / Quiz / Result / Participant)
• Auth & Export: Firebase Authentication, Puppeteer (PDF), ExcelJS (spreadsheets), Docker + Google Cloud Build

Results & Outcomes

Takeaways

• Constrain the model to a contract. A pinned JSON schema plus JSON-mode output turns a chat model into a reliable structured-data source the rest of the app can consume without a human in the loop.
• Normalise input at the edge. Collapsing PDF, image, and text into one clean text field early means the generator has a single input shape — new formats are added at the boundary, not threaded through the core.
• Keep grades out of the model. Scoring, negative marking, and totals are deterministic code — the same answers always produce the same grade, and every mark can be audited.
• LLMs as one component in a real product. The model writes questions and reads images; auth, storage, sharing, scoring, and export are plain, testable engineering around it.