Processor API Guide
The GLOST processor provides a unified/remark-style fluent API for processing documents through plugin pipelines.
Table of Contents
- Quick Start
- Creating a Processor
- Using Plugins
- Processing Documents
- Hooks and Middleware
- Data Storage
- Freezing
- Error Handling
- Advanced Usage
Quick Start
import { glost } from "@glotblocks/glost-processor";
import { transcription } from "@glotblocks/glost-transcription";
import { translation } from "@glotblocks/glost-translation";
const processor = glost()
.use(transcription, { scheme: "ipa" })
.use(translation, { target: "en" });
const result = await processor.process(document);Creating a Processor
Basic Creation
import { glost } from "@glotblocks/glost-processor";
const processor = glost();With Options
const processor = glost({
lenient: true, // Continue on errors
conflictStrategy: "warn", // How to handle conflicts
debug: true // Enable debug logging
});Processor Options
| Option | Type | Default | Description |
|---|---|---|---|
lenient | boolean | false | If true, continue processing on errors |
conflictStrategy | string | ”error” | How to handle metadata conflicts: “error”, “warn”, or “lastWins” |
debug | boolean | false | Enable debug logging |
data | Map | - | Initial data store |
onSkip | function | - | Callback when a plugin is skipped |
Using Plugins
Plugin Functions
Recommended approach - plugins as functions:
import { transcription } from "@glotblocks/glost-transcription";
processor.use(transcription, {
scheme: "ipa",
includeStress: true
});Extension Objects
Direct extension objects (backwards compatible):
import { transcriptionExtension } from "@glotblocks/glost-transcription";
processor.use(transcriptionExtension);Plugin IDs
Use registered plugin IDs:
processor.use("transcription");Presets
Use preset configurations:
import { languageLearningPreset } from "@glotblocks/glost-presets";
processor.use(languageLearningPreset);Chaining
Chain multiple plugins:
processor
.use(transcription)
.use(translation)
.use(frequency)
.use(difficulty)
.use(pos);Processing Documents
Basic Processing
Returns just the processed document:
const result = await processor.process(document);
// result is GLOSTRootWith Metadata
Returns document and processing metadata:
const result = await processor.processWithMeta(document);
console.log(result.document); // Processed document
console.log(result.metadata.appliedPlugins); // Which plugins ran
console.log(result.metadata.stats.totalTime); // Total time
console.log(result.metadata.errors); // Any errors
console.log(result.metadata.warnings); // Any warningsResult Structure
interface ProcessingResult {
document: GLOSTRoot;
metadata: {
appliedPlugins: string[];
skippedPlugins: string[];
errors: ProcessingError[];
warnings: ProcessingWarning[];
stats: {
totalTime: number;
timing: Map<string, number>;
nodesProcessed: number;
startTime: number;
endTime: number;
};
};
}Hooks and Middleware
Before Hook
Run code before a plugin:
processor.before("translation", (doc, pluginId) => {
console.log("About to translate");
// Can inspect/log document state
});After Hook
Run code after a plugin:
processor.after("translation", (doc, pluginId) => {
console.log("Translation complete");
// Can inspect results
});Error Handler
Handle errors from plugins:
processor.onError((error, pluginId) => {
console.error(`Plugin ${pluginId} failed:`, error);
// Log to monitoring service
sendToSentry(error);
});Skip Handler
Handle skipped plugins:
processor.onSkip((pluginId, reason) => {
console.log(`Plugin ${pluginId} was skipped: ${reason}`);
});Progress Handler
Track processing progress:
processor.onProgress((stats) => {
const percent = (stats.completed / stats.total) * 100;
console.log(`Progress: ${percent.toFixed(0)}%`);
updateProgressBar(percent);
});Multiple Hooks
You can register multiple hooks:
processor
.before("translation", logBeforeTranslation)
.before("translation", cacheOriginalText)
.after("translation", logAfterTranslation)
.after("translation", sendAnalytics);Data Storage
Share data between plugins:
Setting Data
processor.data("config", {
theme: "dark",
language: "th"
});Getting Data
const config = processor.data("config");
console.log(config.theme); // "dark"Chaining
processor
.data("apiKey", process.env.API_KEY)
.data("cache", new Map())
.use(plugin1)
.use(plugin2);Accessing in Plugins
const myPlugin = (options) => ({
id: "my-plugin",
name: "My Plugin",
transform: (tree, context) => {
const cache = context?.options?.data?.get("cache");
// Use cache
return tree;
}
});Freezing
Freeze a processor to reuse it:
Create and Freeze
const frozen = glost()
.use(transcription)
.use(translation)
.freeze();Reuse Across Documents
const result1 = await frozen.process(doc1);
const result2 = await frozen.process(doc2);
const result3 = await frozen.process(doc3);Why Freeze?
- Performance: Configure once, use many times
- Immutability: Prevents accidental modification
- Type Safety: Frozen processors have a different type
What You Can’t Do After Freezing
const frozen = processor.freeze();
// These will throw errors:
frozen.use(plugin); // ❌ Error
frozen.before(hook); // ❌ Error
frozen.data("key", value); // ❌ Error
// These still work:
frozen.process(doc); // ✅ OK
frozen.processWithMeta(doc); // ✅ OKError Handling
Lenient Mode
Continue processing even if plugins fail:
const processor = glost({ lenient: true })
.use(plugin1)
.use(plugin2) // Fails
.use(plugin3); // Still runs
const result = await processor.processWithMeta(document);
// Check for errors
if (result.metadata.errors.length > 0) {
console.error("Some plugins failed:", result.metadata.errors);
}
// Check which plugins ran
console.log("Applied:", result.metadata.appliedPlugins);
console.log("Skipped:", result.metadata.skippedPlugins);Strict Mode (Default)
Stop processing on first error:
const processor = glost() // lenient: false by default
.use(plugin1)
.use(plugin2) // Fails - throws error
.use(plugin3); // Never runs
try {
await processor.process(document);
} catch (error) {
console.error("Processing failed:", error);
}Conflict Handling
Handle metadata field conflicts:
// Error on conflicts (default)
const processor = glost({ conflictStrategy: "error" });
// Warn but continue
const processor = glost({ conflictStrategy: "warn" });
// Silently use last write
const processor = glost({ conflictStrategy: "lastWins" });Advanced Usage
Dynamic Plugin Loading
const pluginIds = ["transcription", "translation", "frequency"];
let processor = glost();
for (const id of pluginIds) {
processor = processor.use(id);
}
await processor.process(document);Conditional Plugins
let processor = glost()
.use(transcription);
if (needsTranslation) {
processor = processor.use(translation);
}
if (needsFrequency) {
processor = processor.use(frequency);
}
await processor.process(document);Plugin Factories
function createLanguageProcessor(sourceLanguage: string, targetLanguage: string) {
return glost()
.use(transcription, { language: sourceLanguage })
.use(translation, { target: targetLanguage })
.use(frequency, { language: sourceLanguage })
.freeze();
}
const thaiProcessor = createLanguageProcessor("th", "en");
const japaneseProcessor = createLanguageProcessor("ja", "en");
await thaiProcessor.process(thaiDoc);
await japaneseProcessor.process(japaneseDoc);Custom Presets
const myPreset = {
id: "my-custom",
name: "My Custom Preset",
description: "Tailored for my use case",
plugins: [
["transcription", { scheme: "ipa" }],
"translation",
["frequency", { normalize: true }],
"difficulty"
]
};
processor.use(myPreset);Nested Presets
const basePreset = {
id: "base",
plugins: ["transcription", "translation"]
};
const enhancedPreset = {
id: "enhanced",
plugins: [
basePreset,
"frequency",
"difficulty"
]
};
processor.use(enhancedPreset);Performance Monitoring
processor
.onProgress((stats) => {
console.log(`Completed ${stats.completed}/${stats.total}`);
console.log(`Elapsed: ${stats.elapsed}ms`);
})
.use(plugin1)
.use(plugin2)
.use(plugin3);
const result = await processor.processWithMeta(document);
// Detailed timing
for (const [plugin, time] of result.metadata.stats.timing) {
console.log(`${plugin}: ${time}ms`);
}Integration with Existing Code
// Wrapper function for gradual migration
async function processWithPlugins(
doc: GLOSTRoot,
plugins: string[]
): Promise<GLOSTRoot> {
let processor = glost({ lenient: true });
for (const plugin of plugins) {
processor = processor.use(plugin);
}
return await processor.process(doc);
}
// Use it
const result = await processWithPlugins(document, [
"transcription",
"translation",
"frequency"
]);Best Practices
1. Freeze for Reuse
// Good
const processor = createProcessor().freeze();
for (const doc of documents) {
await processor.process(doc);
}
// Bad (recreates processor each time)
for (const doc of documents) {
await createProcessor().process(doc);
}2. Use Presets for Common Patterns
// Good
import { languageLearningPreset } from "@glotblocks/glost-presets";
processor.use(languageLearningPreset);
// Less good
processor
.use("transcription")
.use("translation")
.use("frequency")
.use("difficulty")
.use("pos");3. Handle Errors Appropriately
// Good - lenient for non-critical processing
const processor = glost({ lenient: true });
const result = await processor.processWithMeta(doc);
if (result.metadata.errors.length > 0) {
logErrors(result.metadata.errors);
}
// Good - strict for critical processing
try {
await processor.process(doc);
} catch (error) {
handleCriticalError(error);
}4. Use Hooks for Observability
processor
.onProgress(updateUI)
.onError(logToMonitoring)
.use(plugin1)
.use(plugin2);Next Steps
Last updated on