TOPIK Question Pipeline — Seol Sunwoo (Jake)

Problem: What the service needed looked like 'enough questions,' but the actual requirement had three layers — generation rules per type, review criteria separating pass from fail, and expected wrong-answer/feedback data for after a question ships. Free-form LLM generation failed all three at once: quality variance, topic bias, zero reusability. Generation took minutes; the human cost of reviewing its output overwhelmed the generation cost.
Role: I owned the entire data line — generation rules, building and testing scripts, review criteria, and augmentation. In particular I kept the judgment line: deciding what counts as passing.
Process: The core question: where is a question's quality decided? Not in the generation prompt — in the judgment standard, set first. I generated 2,000 type-53 (graph/table) items and precision-reviewed a 50-item sample to establish criteria; in the end only 63 of 2,000 — about 3% — fully passed. Every rejection pointed at an empty spot in the rules: over-difficult vocabulary in model answers, monotonous graph values, biased topic framing. I looped those notes back into the rules and regenerated — and chose '63 items with a standard' over '20,000 without one,' moving freedom to where it belonged: the material layer, not the assembly.
Result: Scaling across types 51–54 left rule documents, review checklists, 100,000 expected user queries, and 22,250 feedback records. What changed wasn't just the data — it was the conversation: reviews moved from 'looks fine' to 'passed or failed,' and decisions started arriving faster.