Indexing Plugins

An indexing plugin converts a downloaded file into searchable fields. Implement IFile for C# or export index_file with the matching WASM SDK. See Sandboxing for execution constraints.

C# implementation

The host passes the file bytes in DocumentDownload.Content and services in FilePluginParameters. Use the supplied builder so the output retains the input document's identity.

using System.Text;

using BlueCurve.Search.Document.Common;
using BlueCurve.Search.Indexer.Abstractions;
using BlueCurve.Search.Indexing;

namespace BlueCurve.Examples;

public sealed class PlainTextIndexer : IFile
{
    public Task<DocumentIndex?> IndexAsync(DocumentDownload document, FilePluginParameters parameters, CancellationToken cancellationToken)
    {
        cancellationToken.ThrowIfCancellationRequested();

        var content = Encoding.UTF8.GetString(document.Content);

        var index = parameters.DocumentIndexBuilder
            .AddFields(Field.CreateMainContentField(content))
            .AddFields(Field.CreateTitleField("Example document"))
            .Build(new ContentType("text/plain"));

        return Task.FromResult<DocumentIndex?>(index);
    }

    public void Dispose()
    {
    }

    public ValueTask DisposeAsync() => ValueTask.CompletedTask;
}

Download PlainTextIndexer.cs. This example handles UTF-8 plain text. Use a parser appropriate to the file formats declared in your package's capabilities.

Host services

Property on FilePluginParameters	Use
DocumentIndexBuilder	Add fields, language, metadata, and build the resulting DocumentIndex.
Fingerprint	Compute stable document identifiers.
LanguageClassifier	Detect natural language from extracted text.
Logger	Write plugin diagnostics through the extension logger.
KPI	Record events and execution time.

Honor the cancellation token. Return the built index, or no result when the input cannot produce an index under the IFile contract. Release plugin-owned resources through Dispose and DisposeAsync.

WASM implementations

These examples emit sample content. Decode the incoming payload when extracting the actual downloaded file. Follow WASM Development for the payload, allocator, toolchain, and build requirements.

Rust

use bluecurve_search_sdk::{
    bluecurve_file, DocumentIndexBuilder, Field, FieldType, FieldValue,
    KeyPerformanceIndicators, Kpi, Logger,
};

#[bluecurve_file]
fn index_doc(_ptr: i32, _len: i32) -> i64 {
    let _timer = Kpi::new().track_execution_time(KeyPerformanceIndicators::ExecutionTime);
    Logger::info("Indexing document from Rust");
    let builder = DocumentIndexBuilder::new();
    let mut content = Field::new("content", FieldValue::String("Hello from Rust"));
    content.field_type = FieldType::MainContent;
    builder.add_field(content);
    builder.add_language("en");
    builder.build("text/plain");
    0
}

Zig

const std = @import("std");
const bc = @import("bluecurve");

export fn index_file(_: i32, _: i32) i64 {
    const timer = bc.Job.Kpi.trackExecutionTime(.ExecutionTime);
    defer timer.stop();
    const builder = bc.Indexer.Builder.init(std.heap.wasm_allocator);
    builder.addField("content", .{ .string = "Hello from Zig" }, .MainContent, .Yes, null, null, null, null) catch @trap();
    builder.addLanguage("en");
    builder.build("text/plain");
    return 0;
}

export fn alloc(size: usize) [*]u8 {
    return (std.heap.wasm_allocator.alloc(u8, size) catch @trap()).ptr;
}

export fn dealloc(ptr: [*]u8, len: usize) void {
    std.heap.wasm_allocator.free(ptr[0..len]);
}

Kotlin

@file:OptIn(kotlin.wasm.ExperimentalWasmInterop::class)

import bluecurve.sdk.DocumentIndexBuilder
import bluecurve.sdk.Field
import bluecurve.sdk.FieldType
import bluecurve.sdk.Logger
import bluecurve.sdk.bluecurveFile
import kotlin.wasm.WasmExport

fun main() {}

@WasmExport("index_file")
fun indexFile(ptr: Int, len: Int): Long = bluecurveFile(ptr, len) { payload ->
    payload.readBytes()

    Logger.info("[Kotlin SDK] Starting document indexing...")

    val builder = DocumentIndexBuilder()
    builder.addField(Field.text("content", "Hello from Kotlin", FieldType.MainContent))
    builder.addField(Field.verbatim("title", "Kotlin SDK Document"))
    builder.addLanguage("en")
    builder.build("text/plain")

    Logger.info("[Kotlin SDK] Indexing complete.")
    0
}

Go

package main

import (
    "unsafe"

    bluecurve "bluecurve.io/search/sdk/golang"
)

func main() {}

//go:wasmexport index_file
func indexFile(ptr unsafe.Pointer, len int32) int64 {
    return bluecurve.FileEntrypoint(ptr, len, func(payload bluecurve.Payload) int64 {
        _ = payload.ReadBytes()

        bluecurve.Info("[Go SDK] Starting document indexing...")

        builder := bluecurve.NewDocumentIndexBuilder()
        builder.AddField(bluecurve.NewMainContentField("content", "Hello from Go"))
        builder.AddField(bluecurve.NewField("title", bluecurve.ValueString("Go SDK Document")))
        builder.AddLanguage("en")
        builder.Build("text/plain")

        bluecurve.Info("[Go SDK] Indexing complete.")
        return 0
    })
}

Fields and collection configuration

Use MainContent for the document's main text and VerbatimString for exact values. The SDK also supports dates, numeric values, binary fields, and geographic points. Keep field names and types consistent with your collection structure and matching rules.

Create a package using the indexing manifest, then deploy through the Packaging API.

Next: Job Plugins