CONTEXT BEFORE ACTION

Understand the request before changing the kitchen.

Jangoing grounds everyday language in household context and asks for review before updating shared state.

MEDIA PLACEHOLDER · LANGUAGE-TO-ACTION PRODUCT LOOP

ROLE
Language Engineer & NLP Systems Designer Full-stack implementation
DURATION
Ongoing
CLIENT
Independent Product
RESPONSIBILITIES
Language ontology, annotation conventions, corpus design, relevance and intent taxonomy, entity-span policy, normalization, temporal grounding, evaluation methodology, error analysis, and product integration
TOOLS
Python, scikit-learn, TypeScript, Next.js, Cloudflare Workers, D1, Zod
LINKS
Live product GitHub

A REAL PRODUCT ENVIRONMENT FOR LANGUAGE LEARNING

The language system is developed inside a working multi-user household product.

Jangoing’s web application is not a separate inventory project or a disposable prototype. It provides the shared household context, review interactions, and production feedback needed to develop the language system—and will become the deployment surface for the trained model.

01HOUSEHOLD

Shared Household

Google-authenticated users can create, join, and switch households through invitation codes. Owners and members interact with the same household-scoped inventory and shopping state.

02STATE

Inventory

Members can manage inventory, shopping lists, quantities, expiry dates, categories, low-stock status, and leftovers through the current MVP.

03REVIEW

Reviewable Actions

Natural-language updates become structured, editable proposals before they modify shared household state.

Confirmations, corrections, cancellations, and unsupported requests become reviewable evidence for continuous evaluation.

The current MVP uses a deterministic English-first interpreter. Once the trained model is ready, it can replace that interpreter behind the same structured action contract while the existing authentication, confirmation, event, and shared-state workflows remain in place.

LANGUAGE PROBLEM

Conversation is not a command line.

Household speech mixes requests, context, shorthand, and observations. The system must decide what is actionable, what language forms an entity, what depends on context, and when to ask rather than act.

“We’re out of drinks.”
  • A report that a specific drink is gone
  • A request to add drinks to the shopping list
  • A contextual statement with no requested action

FIVE-STAGE NLU SYSTEM DESIGN

One utterance becomes five independently testable decisions.

A

Is it actionable?

4-class actionability classification

Four classes separate actionable requests from contextual preference, domain-related but non-actionable speech, and unrelated conversation.

B

What action is supported?

11-intent semantic ontology

An 11-intent ontology distinguishes supported actions, clarification needs, and meaningful but unsupported requests.

C

Which words carry arguments?

sequence labeling / span policy

ITEM, CATEGORY, QUANTITY, UNIT, LOCATION, and EXPIRY_DATE are labeled as spans; action cues remain evidence for intent.

D

How does language map to state?

rule-based finite-state normalization

Surface language stays intact while values map to canonical, household-scoped forms such as “oat milk” → oat_milk.

E

Does the full interpretation hold?

structured semantic parse

The final structure connects relevance, one or more actions, assigned entities, normalized values, and clarification requirements.

01

One sentence can support multiple readings.

USER PROBLEM
Ordinary household language does not always state an explicit action.
DESIGN DECISION
Separate relevance, intent, spans, normalization, and joint interpretation.
WHY THIS WAY
The system can ask instead of mutating household state when confidence is insufficient.

ANNOTATION + DATASET GOVERNANCE

Annotation queues route evidence by what it can teach the model.

Nine overlapping queues use active-learning-style data routing to prioritize production feedback, linguistic edge cases, generated coverage, relevance boundaries, and evaluation candidates.

raw utterance → deterministic or AI-assisted draft → relevance, action, span, and normalization review → reviewed annotation → task-specific JSONL

Queues prioritize annotation work; they do not determine the final dataset split. A sample may appear in multiple queues, while its training or evaluation purpose is assigned separately. Evaluation holdout remains a candidate queue until human review, deduplication, leakage checks, and version approval are complete.

800synthetic-v1 candidates
600relevance candidates
1,400total annotation candidates

These 1,400 records are bootstrap candidates, not reviewed ground truth and not a valid final evaluation set.

AI output is a draft, not a label.

Generated data bootstraps training, not evaluation.

Independent examples are reviewed before entering a frozen test set.

Dataset hashes, split manifests, duplicate removal, and phrase-family leakage checks keep model comparisons reproducible. Production exports use pseudonymous identifiers and exclude unrelated personal content and secrets.

02

Human review turns routed evidence into reliable data.

USER PROBLEM
Production evidence arrives with different risks, gaps, and annotation needs.
DESIGN DECISION
Use nine overlapping queues for work routing while keeping dataset splits independent.
WHY THIS WAY
Review priority can change without contaminating the frozen evaluation boundary.

BASELINE + EVALUATION STATUS

A simple baseline tests the data system before model complexity increases.

Reproducible baseline before model complexity

A CPU-friendly TF-IDF and logistic-regression baseline provides reproducible single-intent classification; multi-action examples are excluded rather than collapsed. OpenAI API assists draft annotation only—it is not ground truth or the runtime model.

Evaluation methodology, not just labels

The gate scores intent classification, entity extraction, normalization, and joint-action exact match. Diagnostics are sliced by phrase family, unseen items, indirect requests, and contextual dependency, so an aggregate score cannot hide systematic language failures.

Pilot target: 300 reviewed training / 100 independent evaluation examples.
Baseline target: 1,000 reviewed training / 200 independent evaluation examples.

CROSS-LINGUAL BY DESIGN

English first, with Korean-English speech grounding on the roadmap.

SHARED SEMANTIC CONTRACT

English, Korean, and code-switched input should resolve to the same authorized, reviewable action structure. Language-specific processing changes, but household grounding, canonical values, and confirmation boundaries remain shared.

SEPARATED EVALUATION

ASR quality is measured independently from semantic interpretation so transcription, intent, entity-span, normalization, and final-action failures remain diagnosable.

This direction builds on prior Korean technical-meeting speech-to-text data work and native Korean–English language experience.

English surface form

“Add oat milk to the shopping list.”

SHARED STRUCTURED ACTIONADD_TO_SHOPPING_LIST
ITEM
oat_milk

SPEECH→ASR TRANSCRIPT→LANGUAGE INTERPRETATION→REVIEWED ACTION

ASR

  • WER
  • CER
  • Latency
  • Household noise
  • Far-field speech
  • Code-switching

NLU

  • Relevance
  • Intent
  • Entity spans
  • Normalization

END TO END

  • Joint-action exact match
  • Clarification rate
  • User correction rate
KOREAN SPAN POLICY
Raw utterance
“오트밀크를 추가해줘”
ITEM span
오트밀크
Case particle
를
Normalized value
oat_milk

Korean case particles attach to nouns, so entity-span conventions must distinguish the canonical item expression from its grammatical marker.

CURRENT SYSTEM + REVIEWED LEARNING LOOP

One reviewed interaction updates household state and improves the next model.

The current web MVP collects typed interactions and user corrections. The future Raspberry Pi changes the input surface, while the same structured proposal, authorization, household-state, and reviewed learning paths remain in place.

CurrentFutureReviewed learning feedback
CURRENTWeb MVPTyped user input
FUTURERaspberry PiSpoken user input
FUTUREASRSpeech to text
SHARED LANGUAGE LAYERLanguage layerEnglish-first rules nowContextual model later
GROUNDED CONTEXTAuthorized household contextUser · membership · permissionsInventory · shopping state
CENTRAL REVIEW GATEStructured proposal + user reviewStructured-action contract · confirm before mutation

PRODUCT STATE PATH

CONFIRMED ACTION
Authorization + Worker API
Append-only household event
Inventory and shopping projections

LEARNING EVIDENCE PATH

PROPOSED INTERPRETATION + REVIEWED OUTCOME
Inference logging
9 annotation queuesRouting, not splitting
Human reviewActions · spans · normalization
Versioned reviewed corpus
TRAINING ONLYTraining split
Candidate model
HELD OUTFrozen evaluation splitNever used for training
Evaluation gateCandidate × frozen evaluation
PASSED REVIEWED EVALUATIONApproved model version
Deploy approved model

interact → review → annotate → train → evaluate → deploy → interact again

TAKEAWAYS

What building Jangoing is teaching me about language systems

Language engineering does not stop at model output. It extends into annotation policy, reviewable product decisions, reproducible evaluation, household context, and the boundaries that keep an ambiguous interpretation from becoming an unsafe action.

01Freeze the first reviewed English benchmark

Complete human review, deduplication, leakage checks, and versioned train–evaluation manifests before reporting baseline performance.

02Add multi-turn, household-grounded context

Evaluate whether prior turns and authorized household state improve interpretation without weakening permission, clarification, or review boundaries.

03Evaluate Korean–English ASR on Raspberry Pi

Measure transcription quality, latency, household-noise robustness, and code-switching separately from semantic interpretation so ASR and NLU failures remain distinguishable.

04Add explainable recommendations

Ground diet, replenishment, and deal suggestions in household state and user goals, with visible sources, rationale, and user controls.

EmailResumeLinkedIn