Skip to Content
DocumentationStandardsMetadata Schema

Standard GLOST Metadata Schema

Overview

This document defines the recommended metadata structure for GLOST implementations. This schema has been validated in production across 7 languages and 1,000+ words.

Core Schema

interface StandardGLOSTMetadata { // Frequency classification frequency?: "very-common" | "common" | "uncommon" | "rare"; // Learning difficulty difficulty?: "beginner" | "intermediate" | "advanced"; // CEFR proficiency level cefr?: "A1" | "A2" | "B1" | "B2" | "C1" | "C2"; // Part of speech (recommend Universal Dependencies tags) partOfSpeech?: string; // Cultural context and usage notes culturalNotes?: string; usageNotes?: string; // Grammatical gender gender?: "masculine" | "feminine" | "neutral" | "common"; // Curriculum metadata lesson?: number; unit?: string; topic?: string; }

Usage in GLOST

Metadata should be stored in extras.metadata on word nodes:

const word = createGLOSTWordNode( "สวัสดี", { rtgs: { text: "sawatdi", system: "rtgs" }, ipa: { text: "sa.wàt.diː", system: "ipa" }, }, { partOfSpeech: "interjection" }, "word", "th", "thai", { translations: { en: "hello" }, metadata: { frequency: "very-common", difficulty: "beginner", cefr: "A1", culturalNotes: "Standard Thai greeting used any time of day", }, }, );

Field Definitions

frequency

Word frequency classification based on corpus analysis or word lists.

Values:

  • "very-common" - Top 500-1000 words, essential vocabulary
  • "common" - Top 1000-5000 words, frequently encountered
  • "uncommon" - Top 5000-20000 words, less frequent
  • "rare" - Beyond top 20000, specialized vocabulary

Use cases:

  • Filter content by vocabulary level
  • Highlight unfamiliar words
  • Generate frequency-based word lists
  • Track learner progress

difficulty

Pedagogical difficulty level for language learners.

Values:

  • "beginner" - A1-A2 level, basic vocabulary
  • "intermediate" - B1-B2 level, everyday communication
  • "advanced" - C1-C2 level, sophisticated usage

Notes:

  • May differ from frequency (common words can be difficult)
  • Consider morphological complexity, irregularity, semantic range
  • Based on teaching experience and learner feedback

cefr

Common European Framework of Reference proficiency level.

Values:

  • "A1" - Breakthrough (basic phrases)
  • "A2" - Waystage (simple everyday situations)
  • "B1" - Threshold (familiar matters)
  • "B2" - Vantage (complex text, technical discussions)
  • "C1" - Effective Operational Proficiency (flexible use)
  • "C2" - Mastery (precise, nuanced expression)

Use cases:

  • Align content with standardized proficiency levels
  • Map to other systems (JLPT, HSK, TOPIK)
  • Generate CEFR-aligned materials

partOfSpeech

Part of speech tag using Universal Dependencies tagset.

Recommended values:

  • "noun" - Noun (common, proper)
  • "verb" - Verb
  • "adj" - Adjective
  • "adv" - Adverb
  • "pron" - Pronoun
  • "det" - Determiner
  • "adp" - Adposition (preposition/postposition)
  • "conj" - Conjunction
  • "part" - Particle
  • "intj" - Interjection
  • "num" - Numeral

See: Universal Dependencies POS tags 

Notes:

  • Use language-specific subtypes where needed
  • Store full form, not abbreviations
  • Enables grammatical filtering and analysis

culturalNotes

Cultural context, usage notes, pragmatic information.

Examples:

culturalNotes: "Used in formal business settings, shows respect"; culturalNotes: "Informal slang, primarily used by young people"; culturalNotes: "Historical term, considered archaic in modern usage";

Use cases:

  • Teach appropriate usage
  • Provide cultural context
  • Warn about sensitive terms
  • Explain pragmatic nuances

usageNotes

General usage guidance, collocations, common patterns.

Examples:

usageNotes: "Often followed by が particle"; usageNotes: "Typically used with plural nouns"; usageNotes: "Informal variant of より formal term";

gender

Grammatical gender (not biological gender).

Values:

  • "masculine" - Masculine gender (der, le, el)
  • "feminine" - Feminine gender (die, la)
  • "neutral" - Neuter gender (das, lo)
  • "common" - Common gender (de in Dutch)

Languages:

  • Applicable to: German, French, Spanish, Italian, Portuguese, etc.
  • Not applicable to: English, Thai, Chinese, Japanese, Korean

lesson / unit / topic

Curriculum organization metadata.

Examples:

lesson: 5; unit: "daily-routines"; topic: "food-and-dining";

Use cases:

  • Organize vocabulary by curriculum
  • Track learner progress through lessons
  • Generate unit-specific word lists
  • Filter content by topic

Storage Format

For batch metadata management, use JSON format:

{ "generatedAt": "2025-01-08T00:00:00.000Z", "version": "1.0.0", "description": "Standard GLOST metadata", "sources": ["corpus-analysis", "word-lists"], "frequency": { "very-common": ["hello", "goodbye", "thank you"], "common": ["restaurant", "airport", "hotel"], "uncommon": ["linguistics", "metaphor"], "rare": ["sesquipedalian"] }, "difficulty": { "beginner": ["hello", "goodbye"], "intermediate": ["although", "however"], "advanced": ["notwithstanding", "consequently"] }, "partOfSpeech": { "noun": ["house", "car"], "verb": ["run", "eat"], "adj": ["big", "small"] }, "culturalNotes": { "wai": "Thai greeting gesture, hands together in prayer position" } }

Extension Integration

Use extensions from separate packages to enhance metadata display:

import { processGLOSTWithExtensionsAsync } from "@glotblocks/glost-plugins"; import { createFrequencyExtension } from "@glotblocks/glost-frequency"; import { createDifficultyExtension } from "@glotblocks/glost-difficulty"; // Create extensions with providers const [freqGen, freqEnh] = createFrequencyExtension({ targetLanguage: 'th', provider: myFrequencyProvider }); const [diffGen, diffEnh] = createDifficultyExtension({ targetLanguage: 'th', provider: myDifficultyProvider }); const result = await processGLOSTWithExtensionsAsync(document, [ freqGen, freqEnh, diffGen, diffEnh ]); // Access enriched data const word = result.children[0]; console.log(word.extras?.frequency?.display); // "Very Common" console.log(word.extras?.frequency?.color); // "#22c55e" console.log(word.extras?.difficulty?.level); // "beginner"

Language-Specific Plugins

For language-specific metadata fields, use separate namespaces:

{ metadata: { // Standard fields frequency: "common", difficulty: "intermediate", // Language-specific thai: { toneClass: "mid", initialConsonant: "mid" }, japanese: { kanjiGrade: 2, jlptLevel: "N4" }, chinese: { strokeCount: 8, radical: "水" } } }

Validation

Basic validation helper:

import { isValidMetadata } from "@glotblocks/glost-utils"; const metadata = { frequency: "very-common", difficulty: "beginner", cefr: "A1", }; if (isValidMetadata(metadata)) { // Safe to use }

Migration Guide

From Custom Schema

If you have existing metadata, map to standard schema:

// Before const oldMeta = { level: 1, // Your custom level type: "n", // Your custom POS freq: "high", // Your custom frequency }; // After const newMeta = { difficulty: "beginner", partOfSpeech: "noun", frequency: "very-common", };

Gradual Adoption

You can adopt fields incrementally:

// Phase 1: Just frequency metadata: { frequency: "common" } // Phase 2: Add difficulty metadata: { frequency: "common", difficulty: "intermediate" } // Phase 3: Full adoption metadata: { frequency: "common", difficulty: "intermediate", cefr: "B1", partOfSpeech: "noun", culturalNotes: "..." }

Best Practices

1. Source Attribution

Document where metadata comes from:

{ "sources": [ "Frequency: SUBTLEX corpus", "Difficulty: CEFR vocabulary lists", "POS: Universal Dependencies" ] }

2. Version Control

Track metadata versions:

{ "version": "1.0.0", "generatedAt": "2025-01-08T00:00:00.000Z", "schema": "glost-standard-metadata-v1" }

3. Validation

Validate metadata before processing:

function validateMetadata(meta: unknown): meta is StandardGLOSTMetadata { if (!meta || typeof meta !== "object") return false; const m = meta as Record<string, unknown>; if ( m.frequency && !["very-common", "common", "uncommon", "rare"].includes( m.frequency as string, ) ) { return false; } if ( m.difficulty && !["beginner", "intermediate", "advanced"].includes(m.difficulty as string) ) { return false; } return true; }

4. Fallback Values

Provide sensible defaults:

const frequency = word.extras?.metadata?.frequency ?? "common"; const difficulty = word.extras?.metadata?.difficulty ?? "intermediate";

Examples

Example 1: Thai Word with Full Metadata

{ type: "WordNode", lang: "th", script: "thai", children: [{ type: "TextNode", value: "สวัสดี" }], transcription: { "paiboon+": { text: "sà-wàt-dii", system: "paiboon+" }, "ipa": { text: "sàwàtdiː", system: "ipa" } }, metadata: { partOfSpeech: "interjection" }, extras: { translations: { en: "hello" }, metadata: { frequency: "very-common", difficulty: "beginner", cefr: "A1", culturalNotes: "Universal Thai greeting used at any time of day" } } }

Example 2: French Word with Gender

{ type: "WordNode", lang: "fr", script: "latin", children: [{ type: "TextNode", value: "maison" }], transcription: { "ipa": { text: "mɛ.zɔ̃", system: "ipa" } }, metadata: { partOfSpeech: "noun" }, extras: { translations: { en: "house" }, metadata: { frequency: "very-common", difficulty: "beginner", cefr: "A1", gender: "feminine", usageNotes: "Always feminine: la maison" } } }

Example 3: Japanese Word with JLPT

{ type: "WordNode", lang: "ja", script: "hiragana", children: [{ type: "TextNode", value: "こんにちは" }], transcription: { "romaji": { text: "konnichiwa", system: "hepburn" }, "ipa": { text: "koɴ.ni.t͡ɕi.wa", system: "ipa" } }, metadata: { partOfSpeech: "interjection" }, extras: { translations: { en: "hello (daytime)" }, metadata: { frequency: "very-common", difficulty: "beginner", cefr: "A1", culturalNotes: "Formal greeting used during daytime hours", japanese: { jlptLevel: "N5" } } } }

See Also

Last updated on