Job Plugins

A job imports or updates documents from an external source. Implement IJob for C# or export start_job with the matching WASM SDK. Schedule work through the Jobs API; supported recurring jobs can also use webhook wake-ups.

C# implementation

The host supplies JobPluginParameters and a cancellation token. This example builds one document and records a checkpoint only after the result pipeline accepts it.

using BlueCurve.Search.Document.Common;
using BlueCurve.Search.Indexing;
using BlueCurve.Search.Job.Abstractions;

namespace BlueCurve.Examples;

public sealed class ExampleJob : IJob
{
    public async Task StartAsync(JobPluginParameters parameters, CancellationToken cancellationToken)
    {
        cancellationToken.ThrowIfCancellationRequested();

        var url = new Uri("https://example.com/documents/welcome");

        var documentId = parameters.Fingerprint.Compute(url.AbsoluteUri);

        var hostId = parameters.Fingerprint.ComputeHost(url);

        var index = parameters.DocumentBuilderFactory
            .CreateIndexBuilder(url, documentId, hostId)
            .AddFields(Field.CreateTitleField("Welcome"))
            .AddFields(Field.CreateMainContentField("This document was created by a BlueCurve job."))
            .Build(new ContentType("text/plain"));

        if (await parameters.ResultPipeline.PostAsync(index))
        {
            await parameters.StateStore.SetStringAsync("last_successful_document", url.AbsoluteUri, cancellationToken);
        }
    }

    public void Dispose()
    {
    }

    public ValueTask DisposeAsync() => ValueTask.CompletedTask;
}

Download ExampleJob.cs.

Host services

Property on JobPluginParameters	Use
Configuration	Raw job configuration text. Validate it before contacting the source.
DocumentBuilderFactory	Create an index or download builder for each document and its stable fingerprints.
ResultPipeline	Post a DocumentIndex or DocumentDownload, or delete by fingerprint. Check the returned acceptance result.
DocumentRepository	Look up last-modified time or fetch metadata for a known document fingerprint and collection.
StateStore	Read, write, or delete string checkpoints with GetStringAsync, SetStringAsync, and DeleteAsync.
Fingerprint	Compute document and host identities.
LanguageClassifier	Detect document language.
MimeTypeProvider	Resolve content types.
Logger and KPI	Write diagnostics and telemetry.
Subscription and PackageId	Identify the current execution context.

Enumerate documents through your source's API. The document repository offers metadata lookups, not a document-listing API. Keep fingerprints stable across runs, honor cancellation, and advance cursors only after accepted writes. Use a separate builder for each document. See Provided Job Plugins before writing a connector.

WASM implementations

Rust and Zig show an indexed-document write with a checkpoint. Kotlin and Go demonstrate deletion by fingerprint; replace the sample identity with an identifier owned by your integration. Follow WASM Development for build commands and payload handling.

Rust

use bluecurve_search_sdk::{
    bluecurve_job, compute_fingerprint_uri, compute_fingerprint_host,
    Field, FieldType, FieldValue, JobIndexBuilder, JobStateStore, Logger,
};

#[bluecurve_job]
fn sync_job(_ptr: i32, _len: i32) {
    let url = "https://example.com/document";
    let builder = JobIndexBuilder::new(url, compute_fingerprint_uri(url), compute_fingerprint_host(url));
    let mut field = Field::new("content", FieldValue::String("Imported document"));
    field.field_type = FieldType::MainContent;
    builder.add_field(field);
    if builder.build_post("text/plain") {
        JobStateStore.set_string("cursor", "document");
    } else {
        Logger::error("Document could not be posted");
    }
}

Zig

const std = @import("std");
const bc = @import("bluecurve");

export fn start_job(_: i32, _: i32) void {
    const url = "https://example.com/document";
    const builder = bc.JobIndexBuilder.init(url, bc.computeFingerprintUri(url), bc.computeFingerprintHost(url));
    builder.addField(std.heap.wasm_allocator, "content", .{ .string = "Imported document" }, .MainContent, .Yes, null, null, null, null) catch @trap();
    if (builder.buildPost("text/plain")) {
        _ = bc.JobStateStore.setString("cursor", "document");
    } else {
        bc.log(.Error, "Document could not be posted");
    }
}

export fn alloc(size: usize) [*]u8 {
    return (std.heap.wasm_allocator.alloc(u8, size) catch @trap()).ptr;
}

export fn dealloc(ptr: [*]u8, len: usize) void {
    std.heap.wasm_allocator.free(ptr[0..len]);
}

Kotlin

@file:OptIn(kotlin.wasm.ExperimentalWasmInterop::class)

import bluecurve.sdk.JobResultPipeline
import bluecurve.sdk.Logger
import bluecurve.sdk.bluecurveJob
import bluecurve.sdk.computeFingerprint
import kotlin.wasm.WasmExport

fun main() {}

@WasmExport("start_job")
fun startJob(ptr: Int, len: Int): Unit = bluecurveJob(ptr, len) {
    Logger.info("[Kotlin SDK] Job started")

    val pipeline = JobResultPipeline()
    val fingerprint = computeFingerprint("test-doc-kotlin-sdk")
    pipeline.delete(fingerprint)

    Logger.info("[Kotlin SDK] Job completed")
}

Go

package main

import (
    "unsafe"

    bluecurve "bluecurve.io/search/sdk/golang"
)

func main() {}

//go:wasmexport start_job
func startJob(ptr unsafe.Pointer, len int32) {
    bluecurve.JobEntrypoint(ptr, len, func(payload bluecurve.Payload) {
        _ = payload.ReadBytes()

        bluecurve.Info("[Go SDK] Job started")

        pipeline := bluecurve.JobResultPipeline{}
        fingerprint := bluecurve.ComputeFingerprint("test-doc-go-sdk")
        pipeline.Delete(fingerprint)

        bluecurve.Info("[Go SDK] Job completed")
    })
}

Packaging and scheduling

Package your implementation with the job manifest. After deployment succeeds, pass its package ULID and configuration to the Jobs API. See Sandboxing before relying on network or filesystem access.

Next: Telemetry & KPIs