Standard GLOST Metadata Schema
Overview
This document defines the recommended metadata structure for GLOST implementations. This schema has been validated in production across 7 languages and 1,000+ words.
Core Schema
interface StandardGLOSTMetadata {
// Frequency classification
frequency?: "very-common" | "common" | "uncommon" | "rare";
// Learning difficulty
difficulty?: "beginner" | "intermediate" | "advanced";
// CEFR proficiency level
cefr?: "A1" | "A2" | "B1" | "B2" | "C1" | "C2";
// Part of speech (recommend Universal Dependencies tags)
partOfSpeech?: string;
// Cultural context and usage notes
culturalNotes?: string;
usageNotes?: string;
// Grammatical gender
gender?: "masculine" | "feminine" | "neutral" | "common";
// Curriculum metadata
lesson?: number;
unit?: string;
topic?: string;
}Usage in GLOST
Metadata should be stored in extras.metadata on word nodes:
const word = createGLOSTWordNode(
"สวัสดี",
{
rtgs: { text: "sawatdi", system: "rtgs" },
ipa: { text: "sa.wàt.diː", system: "ipa" },
},
{ partOfSpeech: "interjection" },
"word",
"th",
"thai",
{
translations: { en: "hello" },
metadata: {
frequency: "very-common",
difficulty: "beginner",
cefr: "A1",
culturalNotes: "Standard Thai greeting used any time of day",
},
},
);Field Definitions
frequency
Word frequency classification based on corpus analysis or word lists.
Values:
"very-common"- Top 500-1000 words, essential vocabulary"common"- Top 1000-5000 words, frequently encountered"uncommon"- Top 5000-20000 words, less frequent"rare"- Beyond top 20000, specialized vocabulary
Use cases:
- Filter content by vocabulary level
- Highlight unfamiliar words
- Generate frequency-based word lists
- Track learner progress
difficulty
Pedagogical difficulty level for language learners.
Values:
"beginner"- A1-A2 level, basic vocabulary"intermediate"- B1-B2 level, everyday communication"advanced"- C1-C2 level, sophisticated usage
Notes:
- May differ from frequency (common words can be difficult)
- Consider morphological complexity, irregularity, semantic range
- Based on teaching experience and learner feedback
cefr
Common European Framework of Reference proficiency level.
Values:
"A1"- Breakthrough (basic phrases)"A2"- Waystage (simple everyday situations)"B1"- Threshold (familiar matters)"B2"- Vantage (complex text, technical discussions)"C1"- Effective Operational Proficiency (flexible use)"C2"- Mastery (precise, nuanced expression)
Use cases:
- Align content with standardized proficiency levels
- Map to other systems (JLPT, HSK, TOPIK)
- Generate CEFR-aligned materials
partOfSpeech
Part of speech tag using Universal Dependencies tagset.
Recommended values:
"noun"- Noun (common, proper)"verb"- Verb"adj"- Adjective"adv"- Adverb"pron"- Pronoun"det"- Determiner"adp"- Adposition (preposition/postposition)"conj"- Conjunction"part"- Particle"intj"- Interjection"num"- Numeral
See: Universal Dependencies POS tags
Notes:
- Use language-specific subtypes where needed
- Store full form, not abbreviations
- Enables grammatical filtering and analysis
culturalNotes
Cultural context, usage notes, pragmatic information.
Examples:
culturalNotes: "Used in formal business settings, shows respect";
culturalNotes: "Informal slang, primarily used by young people";
culturalNotes: "Historical term, considered archaic in modern usage";Use cases:
- Teach appropriate usage
- Provide cultural context
- Warn about sensitive terms
- Explain pragmatic nuances
usageNotes
General usage guidance, collocations, common patterns.
Examples:
usageNotes: "Often followed by が particle";
usageNotes: "Typically used with plural nouns";
usageNotes: "Informal variant of より formal term";gender
Grammatical gender (not biological gender).
Values:
"masculine"- Masculine gender (der, le, el)"feminine"- Feminine gender (die, la)"neutral"- Neuter gender (das, lo)"common"- Common gender (de in Dutch)
Languages:
- Applicable to: German, French, Spanish, Italian, Portuguese, etc.
- Not applicable to: English, Thai, Chinese, Japanese, Korean
lesson / unit / topic
Curriculum organization metadata.
Examples:
lesson: 5;
unit: "daily-routines";
topic: "food-and-dining";Use cases:
- Organize vocabulary by curriculum
- Track learner progress through lessons
- Generate unit-specific word lists
- Filter content by topic
Storage Format
For batch metadata management, use JSON format:
{
"generatedAt": "2025-01-08T00:00:00.000Z",
"version": "1.0.0",
"description": "Standard GLOST metadata",
"sources": ["corpus-analysis", "word-lists"],
"frequency": {
"very-common": ["hello", "goodbye", "thank you"],
"common": ["restaurant", "airport", "hotel"],
"uncommon": ["linguistics", "metaphor"],
"rare": ["sesquipedalian"]
},
"difficulty": {
"beginner": ["hello", "goodbye"],
"intermediate": ["although", "however"],
"advanced": ["notwithstanding", "consequently"]
},
"partOfSpeech": {
"noun": ["house", "car"],
"verb": ["run", "eat"],
"adj": ["big", "small"]
},
"culturalNotes": {
"wai": "Thai greeting gesture, hands together in prayer position"
}
}Extension Integration
Use extensions from separate packages to enhance metadata display:
import { processGLOSTWithExtensionsAsync } from "@glotblocks/glost-plugins";
import { createFrequencyExtension } from "@glotblocks/glost-frequency";
import { createDifficultyExtension } from "@glotblocks/glost-difficulty";
// Create extensions with providers
const [freqGen, freqEnh] = createFrequencyExtension({
targetLanguage: 'th',
provider: myFrequencyProvider
});
const [diffGen, diffEnh] = createDifficultyExtension({
targetLanguage: 'th',
provider: myDifficultyProvider
});
const result = await processGLOSTWithExtensionsAsync(document, [
freqGen, freqEnh,
diffGen, diffEnh
]);
// Access enriched data
const word = result.children[0];
console.log(word.extras?.frequency?.display); // "Very Common"
console.log(word.extras?.frequency?.color); // "#22c55e"
console.log(word.extras?.difficulty?.level); // "beginner"Language-Specific Plugins
For language-specific metadata fields, use separate namespaces:
{
metadata: {
// Standard fields
frequency: "common",
difficulty: "intermediate",
// Language-specific
thai: {
toneClass: "mid",
initialConsonant: "mid"
},
japanese: {
kanjiGrade: 2,
jlptLevel: "N4"
},
chinese: {
strokeCount: 8,
radical: "水"
}
}
}Validation
Basic validation helper:
import { isValidMetadata } from "@glotblocks/glost-utils";
const metadata = {
frequency: "very-common",
difficulty: "beginner",
cefr: "A1",
};
if (isValidMetadata(metadata)) {
// Safe to use
}Migration Guide
From Custom Schema
If you have existing metadata, map to standard schema:
// Before
const oldMeta = {
level: 1, // Your custom level
type: "n", // Your custom POS
freq: "high", // Your custom frequency
};
// After
const newMeta = {
difficulty: "beginner",
partOfSpeech: "noun",
frequency: "very-common",
};Gradual Adoption
You can adopt fields incrementally:
// Phase 1: Just frequency
metadata: { frequency: "common" }
// Phase 2: Add difficulty
metadata: {
frequency: "common",
difficulty: "intermediate"
}
// Phase 3: Full adoption
metadata: {
frequency: "common",
difficulty: "intermediate",
cefr: "B1",
partOfSpeech: "noun",
culturalNotes: "..."
}Best Practices
1. Source Attribution
Document where metadata comes from:
{
"sources": [
"Frequency: SUBTLEX corpus",
"Difficulty: CEFR vocabulary lists",
"POS: Universal Dependencies"
]
}2. Version Control
Track metadata versions:
{
"version": "1.0.0",
"generatedAt": "2025-01-08T00:00:00.000Z",
"schema": "glost-standard-metadata-v1"
}3. Validation
Validate metadata before processing:
function validateMetadata(meta: unknown): meta is StandardGLOSTMetadata {
if (!meta || typeof meta !== "object") return false;
const m = meta as Record<string, unknown>;
if (
m.frequency &&
!["very-common", "common", "uncommon", "rare"].includes(
m.frequency as string,
)
) {
return false;
}
if (
m.difficulty &&
!["beginner", "intermediate", "advanced"].includes(m.difficulty as string)
) {
return false;
}
return true;
}4. Fallback Values
Provide sensible defaults:
const frequency = word.extras?.metadata?.frequency ?? "common";
const difficulty = word.extras?.metadata?.difficulty ?? "intermediate";Examples
Example 1: Thai Word with Full Metadata
{
type: "WordNode",
lang: "th",
script: "thai",
children: [{ type: "TextNode", value: "สวัสดี" }],
transcription: {
"paiboon+": { text: "sà-wàt-dii", system: "paiboon+" },
"ipa": { text: "sàwàtdiː", system: "ipa" }
},
metadata: { partOfSpeech: "interjection" },
extras: {
translations: { en: "hello" },
metadata: {
frequency: "very-common",
difficulty: "beginner",
cefr: "A1",
culturalNotes: "Universal Thai greeting used at any time of day"
}
}
}Example 2: French Word with Gender
{
type: "WordNode",
lang: "fr",
script: "latin",
children: [{ type: "TextNode", value: "maison" }],
transcription: {
"ipa": { text: "mɛ.zɔ̃", system: "ipa" }
},
metadata: { partOfSpeech: "noun" },
extras: {
translations: { en: "house" },
metadata: {
frequency: "very-common",
difficulty: "beginner",
cefr: "A1",
gender: "feminine",
usageNotes: "Always feminine: la maison"
}
}
}Example 3: Japanese Word with JLPT
{
type: "WordNode",
lang: "ja",
script: "hiragana",
children: [{ type: "TextNode", value: "こんにちは" }],
transcription: {
"romaji": { text: "konnichiwa", system: "hepburn" },
"ipa": { text: "koɴ.ni.t͡ɕi.wa", system: "ipa" }
},
metadata: { partOfSpeech: "interjection" },
extras: {
translations: { en: "hello (daytime)" },
metadata: {
frequency: "very-common",
difficulty: "beginner",
cefr: "A1",
culturalNotes: "Formal greeting used during daytime hours",
japanese: {
jlptLevel: "N5"
}
}
}
}