Skip to main content

Documents & Ingestion

Documents are Remem’s core unit: encrypted content plus metadata used for retrieval, filtering, and synthesis.

What happens on ingest

When you call POST /v1/documents/ingest, Remem:
  • Queues an async ingestion job
  • Encrypts content + metadata with tenant-scoped keys
  • Classifies content (category/tags/sensitivity/language/summary)
  • Chunks and embeds searchable text
  • Indexes vectors + searchable payload metadata
Ingestion is asynchronous. The API returns a job_id immediately; the document appears in search shortly after processing.
Ingestion is namespace-aware. If you send "namespace": "work", Remem writes into that namespace key. If you omit it, Remem uses the API key’s default namespace.

Ingest a document

JSON ingestion

Multipart upload

Ingest response

document_id is present when return_id=true; otherwise it may be null.

Request fields

Source ID + metadata behavior

  • source_id is stored as correlation metadata.
  • project, session_id, checkpoint_kind in metadata are normalized for checkpoint filtering.
  • Tags are normalized/deduped for new ingests.

Retrieve a document

Returns decrypted content, metadata, status, version, and classification fields.

Retrieve document chunks

Useful for debugging chunk boundaries and vector-linked metadata.

Update a document

Updates create a new version and preserve lifecycle history.
Document versions stay in the original document’s namespace. Namespace selection happens when the document is first ingested.

Delete a document

DELETE /v1/documents/{document_id} performs a soft delete:
  • Hidden from query results immediately
  • Cleanup work is queued (vectors/files/cache)
  • Hard delete is scheduled by lifecycle workers

Chunk backfill / reindex endpoints

These endpoints are for summary-only documents that need real chunk backfill:
  • POST /v1/documents/backfill-chunks
  • POST /v1/documents/{document_id}/reindex-chunks
They return job IDs for async processing.

Idempotent ingestion

Use Idempotency-Key to avoid duplicate ingest requests on retries:

Supported file types

Remem supports common text, PDF, image, code, spreadsheet, email, and web formats. Examples:
  • Text/notes: .txt, .md, .json, .yaml, .xml
  • PDFs/images: .pdf, .png, .jpg, .webp
  • Code: .py, .ts, .go, .rs, .java, .cpp
  • Spreadsheet/email/web: .csv, .tsv, .eml, .msg, .html