Initialize service with database path

Use EITHER:

• service: Name of pre-configured service (from ServiceProvider.create)
• format: API format ("openai", "mock") to configure inline

Required:

• db_path: Full path to database file (e.g., "/var/data/memo.db")
• api_key: API key (not needed for mock format)

Optional:

• service: Name of pre-configured service to use
• format: API format for inline configuration (default "openai")
• base_url: Custom API endpoint (for OpenAI-compatible APIs); defaults to the service's stored one
• model: Embedding model (default depends on format)
• dimensions: Vector dimensions (auto-detected from model)
• max_tokens: Token limit (auto-detected from model)
• store_text: Enable text storage in texts table (default true)
• build_vocab: Enable incremental vocabulary building (default true, requires store_text)
• chunking_max_tokens: Max tokens per chunk (default 2000)

Example:

memo = Memo::Service.new(
  db_path: "/var/data/memo.db",
  format: "openai",
  api_key: ENV["OPENAI_API_KEY"]
)

Initialize service with existing database connection

Use this when caller manages the connection lifecycle. Caller is responsible for closing the connection.

The db_path is queried from the database for CLI use. Pass db_path explicitly if the pragma query doesn't work for your setup.

Build vocabulary from indexed content

Extracts unique words from all stored texts, embeds them in batches, and stores in the vocab table for word-level similarity search.

Requires text storage to be enabled.

Options:

• batch_size: Number of words to embed per API call (default 2000)
• clear_existing: Whether to clear existing vocab first (default true)

Returns number of words stored

Example:

memo.build_vocab
results = memo.like("database")

Track whether vocab building is enabled

Clear completed items from the queue

Removes successfully processed items (status = 0). Returns number of items removed.

Clear all items from the queue

Removes all items regardless of status. Returns number of items removed.

Clear vocabulary for current service

Close database connection

Should be called when done with service to free resources. Safe to call multiple times.

Note: If service was initialized with an existing db connection, close is a no-op (caller owns the connection).

Create a new service configuration

Creates a named service configuration that can be used later with Service.new(service: "name", ...).

Example:

memo.create_service(
  name: "azure-prod",
  format: "openai",
  base_url: "https://mycompany.openai.azure.com/",
  model: "text-embedding-ada-002",
  dimensions: 1536,
  max_tokens: 8191
)

Get the default service configuration

Delete all chunks for a source

Removes all chunks with the given source_id (and optionally source_type). Orphaned embeddings (not referenced by any chunk) are also cleaned up.

Returns number of chunks deleted.

source_id: External source ID (Int64 or String). source_type: Optional filter to only delete chunks with matching source_type. If nil and source_id is Int64, searches integer IDs across all types. If nil and source_id is String, searches string IDs across all types.

Delete a service configuration

By default, fails if the service has any associated embeddings. Use force: true to delete the service and all associated data; chunks another service has embedded too stay. The service in use here can't be deleted (switch with use_service first), and one whose index another process has open raises USearchIndex::InUse.

Returns true if deleted, false if not found.

Enqueue a document for later embedding

Adds the document to the embed_queue table without embedding it. Use process_queue to embed queued items.

If the source is already in the queue, the text is updated.

Enqueue a document (Document overload)

Enqueue multiple documents for later embedding

More efficient than calling enqueue() multiple times. Resolves all external IDs to internal IDs in a single transaction.

Count indexed files

Get file record by path

Get file record by content hash

Get file record by source ID

Get a service configuration by name

Returns nil if not found.

A source's latest text, by internal source_id: the text last indexed or queued, before it is embedded. Nil if text storage was off.

Index a document

Enqueues the document and processes it immediately with retry support. Returns number of chunks successfully stored.

Supports both integer and string source IDs:

• Int64: Time-based, sortable IDs (e.g., Unix timestamps)
• String: UUIDs and other text identifiers

When source_id is nil, memo creates and manages the source internally. Useful for CLI and cases where external ID correlation isn't needed.

Index a document (Document overload)

Convenience method that accepts a Document struct.

Index multiple documents in a batch

Enqueues all documents and processes them with retry support. More efficient than calling index() multiple times.

Returns total number of documents successfully processed.

Index an explicit list of file paths

Unlike index_files, this does not walk directories or use ignore files. Relative paths are resolved from the current directory. Binary files are still skipped. Files are recorded by absolute path; root only finds the records of files indexed before that, which were keyed by their path relative to it.

Returns tuple of (indexed_count, skipped_count, total_files)

Index file list without progress callback

Index files from a directory

Walks directory, respects ignore files, skips binary files and symlinks, and indexes text content. Tracks file metadata for incremental updates. Files are recorded by absolute path, and those recorded under the directory that the walk no longer finds (deleted, or ignored now) are removed from the index.

Options:

• root: Directory to index
• ignore_file: Ignore file name (default ".gitignore")
• incremental: Skip unchanged files based on mtime (default true)
• dry_run: List files without indexing (default false)

The block gets each file's absolute path and what happened to it: :indexed, :skipped or :removed (:would_index or :would_remove on a dry run).

Returns tuple of (indexed_count, skipped_count, total_files)

Example:

indexed, skipped, total = memo.index_files("/path/to/project")

Index files without progress callback

What opening the index took (journal replay, rebuild, missing vectors)

Save once this many index changes are unsaved, or once any have been unsaved this long

Find words semantically similar to the query

Searches the vocabulary table for words with similar embeddings. Requires vocabulary to be built first with build_vocab().

Returns array of VocabResult with word, score, and frequency.

Example:

results = memo.like("database")
results.each do |r|
  puts "#{r.word}: #{r.score}"
end

List indexed files

List all service configurations

Returns array of service info, ordered by creation time (newest first).

List service configurations by format

Returns array of service info for the specified API format.

Mark chunks as read (increment read_count)

Process queued items

Embeds pending items from the queue using the service's batch_size. Returns number of items successfully processed.

Failed items have their status set to the error code and can be retried up to max_retries times.

NOTE Queue items are not atomically claimed. This is intentional - single-worker processing is the expected use case since we're hitting one API endpoint with batched requests. Parallel workers would hit rate limits and add complexity without benefit.

A source queued again while its item is being embedded stays pending, with its new text, and is embedded next; the older text isn't written.

Process queued items asynchronously

Spawns a fiber to process the queue and returns immediately. Use queue_stats to check progress.

Get queue statistics

Returns counts of pending and failed items.

Re-index all content of a given source type

Deletes existing embeddings and queues text for re-embedding. Requires text storage to be enabled.

Returns number of items queued for re-indexing.

Re-index all content of a given source type using a block to fetch text

Use this when text storage is disabled. The block receives the external source_id and should return the text to embed.

Returns number of items queued for re-indexing.

Example:

memo.reindex("article") do |source_id|
  app.get_article_text(source_id)
end
memo.process_queue

Save the USearch index to disk and checkpoint the journal, now.

Does nothing when the index hasn't changed since the last save. The file is written on a separate thread: searches carry on meanwhile, writes wait. close calls this; for routine saves prefer save_index_if_due (memo also schedules those itself after writes).

Save if the policy says it's time: index_save_changes unsaved changes, or any unsaved for index_save_interval. Cheap to call often (e.g. after each queue run, or from a timer).

Search for semantically similar chunks

Automatically generates query embedding and searches.

Returns array of search results ranked by similarity.

source_id: Filter by external source ID (Int64 or String).

like: LIKE pattern(s) to filter by text content. Single string or array of strings for AND filtering. Example: like: "%cats%" or like: ["%cats%", "%dogs%"] Only works when text storage is enabled.

match: FTS5 full-text search query. Supports AND, OR, NOT, prefix*, "phrases". Example: match: "cats OR dogs", match: "quick brown*" Only works when text storage is enabled.

sql_where: Raw SQL fragment for filtering chunks. Used with ATTACH to filter by external database tables. Example: "c.source_id IN (SELECT id FROM main.artifact WHERE kind = 'goal')" Note: c.source_id here is the internal ID, not external. Bind values with ? (sql_where_args). On Postgres, a ? outside quotes is always a placeholder, so jsonb's ?, ?| and ?& operators can't be used; use jsonb_exists() and friends.

include_text: If true, includes text content in search results. Only works when text storage is enabled.

Search with per-stage timing breakdown

Returns {results, timings} where timings has embed_ms, search_ms, fetch_ms, total_ms.

Get usage statistics for a service

Set a service as the default

Returns true if successful, false if service not found.

Get statistics about indexed content

Returns counts of embeddings, chunks, and unique sources.

Track whether text storage is enabled

Update a service configuration

Can update base_url and max_tokens. Returns the updated service info, or nil if not found.

Switch to a different service

Changes the current provider and service configuration, along with the index, query cache and chunking that go with it. The chunk size stays, and must fit the new service's max_tokens. The api_key is required for non-mock services.

Writes wait while it switches. Don't call it while searches are running, since they read the index it closes.

Example:

memo.use_service("azure-prod", api_key: ENV["AZURE_API_KEY"])

Get vocabulary statistics