01HOUSEHOLD
Shared Household
Google-authenticated users can create, join, and switch households through invitation codes. Owners and members interact with the same household-scoped inventory and shopping state.
CONTEXT BEFORE ACTION
Understand the request before changing the kitchen.
Jangoing grounds everyday language in household context and asks for review before updating shared state.
REQUEST → CONTEXT → REVIEW → SHARED STATE
A REAL PRODUCT ENVIRONMENT FOR LANGUAGE LEARNING
The language system is developed inside a working multi-user household product.
Jangoing’s web application is not a separate inventory project or a disposable prototype. It provides the shared household context, review interactions, and production feedback needed to develop the language system—and will become the deployment surface for the trained model.
02STATE
Inventory
Members can manage inventory, shopping lists, quantities, expiry dates, categories, low-stock status, and leftovers through the current MVP.
03REVIEW
Reviewable Actions
Natural-language updates become structured, editable proposals before they modify shared household state.
Confirmations, edits, cancellations, and unsupported requests become evidence for evaluation.
The MVP uses a deterministic English-first interpreter. A trained model can later replace it behind the same reviewable structured-action contract.
LANGUAGE PROBLEM
Conversation is not a command line.
Household speech mixes requests, context, shorthand, and observations. The system must decide what is actionable, what language forms an entity, what depends on context, and when to ask rather than act.
“We’re out of drinks.”
- A report that a specific drink is gone
- A request to add drinks to the shopping list
- A contextual statement with no requested action
FIVE-STAGE NLU SYSTEM DESIGN
One utterance becomes five independently testable decisions.
Is it actionable?
4-class actionability classificationFour classes separate actionable requests from contextual preference, domain-related but non-actionable speech, and unrelated conversation.
What action is supported?
11-intent semantic ontologyAn 11-intent ontology distinguishes supported actions, clarification needs, and meaningful but unsupported requests.
Which words carry arguments?
sequence labeling / span policyITEM, CATEGORY, QUANTITY, UNIT, LOCATION, and EXPIRY_DATE are labeled as spans; action cues remain evidence for intent.
How does language map to state?
rule-based finite-state normalizationSurface language stays intact while values map to canonical, household-scoped forms such as “oat milk” → oat_milk.
Does the full interpretation hold?
structured semantic parseThe final structure connects relevance, one or more actions, assigned entities, normalized values, and clarification requirements.
01
One sentence can support multiple readings.
- USER PROBLEM
- Ordinary household language does not always state an explicit action.
- DESIGN DECISION
- Separate relevance, intent, spans, normalization, and joint interpretation.
- WHY THIS WAY
- The system can ask instead of mutating household state when confidence is insufficient.
“We're out of milk.”
ITEM
“milk” → milkPOSSIBLE READINGS
- 01
Mark milk as out of stock
- 02
Add milk to the shopping list
- 03
Keep it as household context
JANGOING ASKS
Should I mark milk as out of stock or add it to the shopping list?
RELEVANCE → INTENT → SPANS → NORMALIZATION → DECISION
ANNOTATION + DATASET GOVERNANCE
Annotation queues route evidence by what it can teach the model.
Nine overlapping review queues route production feedback, linguistic edge cases, generated coverage, relevance boundaries, and evaluation candidates.
raw utterance → deterministic or AI-assisted draft → relevance, action, span, and normalization review → reviewed annotation → task-specific JSONL
Queues prioritize annotation work; they do not determine the final dataset split. A sample may appear in multiple queues, while its training or evaluation purpose is assigned separately. Evaluation holdout remains a candidate queue until human review, deduplication, leakage checks, and version approval are complete.
These 1,400 records are bootstrap candidates—not reviewed ground truth, not a final training corpus, and not a valid evaluation set.
CURRENT SOURCEIN-HOUSE GENERATED CANDIDATES
EXTERNAL DATA, CONSTRAINED BY ROLE
Public datasets support different parts of the system—but none become Jangoing ground truth by default.
Each source remains separate, versioned, and limited to the task it can legitimately support. Incompatible labels are mapped or evaluated separately rather than merged into the Jangoing schema.
Grocery NER · GroceryList · Open Food Facts
- Intended role
- Entity vocabulary, taxonomy candidates, product aliases, category relations, and catalog-linking candidates.
- Mapping
- Grocery labels → ITEM / CATEGORY candidates. Item and category records → canonical-value candidates. Product and brand records → catalog and alias candidates.
- Guardrail
- Not direct intent ground truth. Taxonomy remapping and human review are required.
Open Food Facts supports the product catalog and entity linker—not utterance-intent training.
MASSIVE · SNIPS
- Intended role
- Intent and slot architecture smoke tests, BIO tagging, evaluation-code validation, OOD experiments, and future multilingual pipeline checks.
- Mapping
- External intent and slot schemas → separate modeling benchmarks.
- Guardrail
- Kept as separate corpora. External intents are not forced into Jangoing action labels.
Performance on MASSIVE or SNIPS does not represent grocery-domain performance.
MultiWOZ · Instacart-style data
- Intended role
- Multi-turn dialogue design, slot carryover, clarification flows, co-purchase patterns, reorder priors, and future recommendation experiments.
- Mapping
- Multi-turn dialogue → future context design. Basket and reorder patterns → recommendation priors.
- Guardrail
- Not part of the current single-utterance NLU training corpus.
Instacart-style data contains no utterance, intent, or entity-span supervision.
Original external records remain unchanged.
Record source, revision, license, date, and hash.
Map only fields supported by the Jangoing schema.
Review taxonomy, spans, canonical values, and mapping conflicts.
Approved records may enter a task-specific candidate pool.
01AI drafts and external mappings are candidates, not labels.
02Generated and external data never enter the frozen evaluation set by default.
03Provenance, deduplication, phrase-family leakage checks, manifests, and hashes remain versioned.
02
Human review turns routed evidence into reliable data.
- USER PROBLEM
- Production evidence arrives with different risks, gaps, and annotation needs.
- DESIGN DECISION
- Use nine overlapping queues for work routing while keeping dataset splits independent.
- WHY THIS WAY
- Review priority can change without contaminating the frozen evaluation boundary.
QUEUEGenerated coverage
DRAFTSOURCEAI-assisted draftNOT GROUND TRUTH
Add two cartons of oat milkITEM to the shopping list.
INTENTADD_TO_SHOPPING_LIST
PHRASE FAMILYdirect_request
ITEM
two cartons of oat milk
oat_milkSpan may include quantity and unit.
AI DRAFT → HUMAN CORRECTION → REVIEWED ANNOTATION
BASELINE + EVALUATION STATUS
A simple baseline tests the data system before model complexity increases.
CURRENT STATUS
Baseline pipeline implemented
Reviewed benchmark not yet frozen
Metrics not yet reported
Reproducible baseline before model complexity
A CPU-friendly TF-IDF and logistic-regression pipeline is implemented as the first reproducible baseline. Results will be reported only after the reviewed benchmark and frozen evaluation split are ready.
Evaluation methodology, not just labels
The gate scores intent classification, entity extraction, normalization, and joint-action exact match. Diagnostics are sliced by phrase family, unseen items, indirect requests, and contextual dependency, so an aggregate score cannot hide systematic language failures.
100 independent evaluation
200 independent evaluation
CROSS-LINGUAL BY DESIGN
English first, with Korean-English speech grounding on the roadmap.
English, Korean, and code-switched input should resolve to the same authorized, reviewable action structure. Language-specific processing changes, but household grounding, canonical values, and confirmation boundaries remain shared.
ASR quality is measured independently from semantic interpretation so transcription, intent, entity-span, normalization, and final-action failures remain diagnosable.
This direction builds on prior Korean technical-meeting speech-to-text data work and native Korean–English language experience.
English surface form
“Add oat milk to the shopping list.”
SHARED STRUCTURED ACTIONADD_TO_SHOPPING_LIST
- ITEM
- oat_milk
SPEECH→ASR TRANSCRIPT→LANGUAGE INTERPRETATION→REVIEWED ACTION
ASR
- WER
- CER
- Latency
- Household noise
- Far-field speech
- Code-switching
NLU
- Relevance
- Intent
- Entity spans
- Normalization
END TO END
- Joint-action exact match
- Clarification rate
- User correction rate
KOREAN SPAN POLICY+
- Raw utterance
- “오트밀크를 추가해줘”
- ITEM span
- 오트밀크
- Case particle
- 를
- Normalized value
- oat_milk
Korean case particles attach to nouns, so entity-span conventions must distinguish the canonical item expression from its grammatical marker.
CURRENT SYSTEM + PLANNED REVIEWED LEARNING LOOP
Reviewed interactions update household state and accumulate evidence for the next model.
The web MVP collects typed requests and corrections. A future Raspberry Pi changes the input surface, while the same authorization, structured proposal, review, and learning paths remain in place.
CurrentFuture / plannedReviewed learning feedbackSupplemental external evidence
PRODUCT STATE PATH
CONFIRMED ACTIONLEARNING EVIDENCE PATH
INTERPRETATION + REVIEW OUTCOMEPRIMARY IN-HOUSE EVIDENCE
EVALUATION EVIDENCE · TRAINING-ISOLATED
Reviewed training pool→Frozen evaluation split
PLANNED MODEL EVALUATION + DEPLOYMENT
HELD-OUT INPUT FROM ABOVE↓
Approved behavior produces new product interactions and reviewed evidence.
Product interactions and in-house candidates form the primary evidence loop. Independently collected evaluation data remains training-isolated, while planned external sources support only role-specific mapping, grounding, or benchmark paths.
NEXT PHASES
What I’m validating next.
The next phase freezes a trustworthy English benchmark, adds household-grounded context, separates ASR from semantic evaluation, and tests recommendations only after the review contract holds.
01Freeze the first reviewed English benchmark+
Complete human review, deduplication, leakage checks, and versioned train–evaluation manifests before reporting baseline performance.
02Add multi-turn, household-grounded context+
Evaluate whether prior turns and authorized household state improve interpretation without weakening permission, clarification, or review boundaries.
03Evaluate Korean–English ASR on Raspberry Pi+
Measure transcription quality, latency, household-noise robustness, and code-switching separately from semantic interpretation so ASR and NLU failures remain distinguishable.
04Add explainable recommendations+
Ground diet, replenishment, and deal suggestions in household state and user goals, with visible sources, rationale, and user controls.