Ingestion API (gRPC)

The Ingestion API is the primary gateway for adding data to BlueCurve. It supports both Raw and Processed document formats via gRPC.

Endpoint Overview

REST (Deletion)

Method	Path	Description	Required Permission
DELETE	/ingestion/{documentId}	Delete a document by fingerprint	Write, Admin
DELETE	/ingestion	Delete a document by URI (via ?uri=)	Write, Admin

gRPC (Ingestion)

Service	Method	Description	Required Permission
IngestionServiceV1	Add	Add raw or processed document	Write, Admin
IngestionServiceV1	Update	Update raw or processed document	Write, Admin

gRPC Service Definition

The service is defined in ingestion.proto under the package BlueCurve.Search.Sdk.V1.

service IngestionServiceV1 {
    rpc Add(stream AddDocumentMessage) returns (TaskIdResponseV1);
    rpc Update(stream UpdateDocumentMessage) returns (TaskIdResponseV1);
}

Adding Documents

Raw Document Ingestion

Use this when you want BlueCurve to handle the indexing process (extraction, language detection, etc.).

AddDocumentMessage -> RawDocumentChunk:

• Metadata: The first streamed message must contain metadata.
  • ContentType: Standard MIME type (e.g., application/pdf).
  • Url: The canonical identifier for the document.
  • CollectionId: The target collection (optional).
  • SourceId: The source identifier (optional).
  • Headers: Source headers or fetch metadata.
  • Properties: Key-value pairs for custom metadata.
• DataChunk: Subsequent streamed messages carry sequential byte chunks.

Processed Document Ingestion

Use this when you have already extracted fields and want to index them directly.

AddDocumentMessage -> ProcessedDocument:

• Fields: A list of Field objects.
  • FieldType: Text, MainContent, VerbatimString, DateTime, numeric types, Geo, Wkt, or Binary.
  • Name: The field name.
  • Value: A google.protobuf.Any containing the serialized value.
• Relationships: Linking to other documents (Alternate, Author, Tag, etc.).
• Markdown: Optional human-readable Markdown used as downstream AI/search enrichment context.

Response Handling

All successfully queued ingestion requests return a TaskIdResponseV1.

message TaskIdResponseV1 {
    oneof Value {
        string TaskId = 1;
        int32 HttpStatus = 2; // Error status if queueing failed
    }
}

IMPORTANT

A successful response means the document is queued for processing. Use the Task API to monitor the actual indexing status.

---

Next: Packaging API