Exhume Grep as a library

exhume_grep exposes the same read-only content search used by the CLI. It accepts a SearchRequest, streams serializable SearchEvent values through a bounded Tokio channel, and returns a SearchSummary.

The API below describes version 0.1.0. The CLI documentation covers indexing prerequisites, supported sources, matching rules, and coverage limits. Source is available in the module repository.

Dependencies

Requires Rust 1.92 or newer. Add these dependencies to Cargo.toml:

[dependencies]
exhume_grep = "0.1.0"
tokio = { version = "1", features = ["macros", "rt-multi-thread", "sync"] }
anyhow = "1"

Stream a search

This complete example searches evidence 1 in case.sqlite. Create the index first and keep its evidence source accessible. Run the consumer concurrently with the search: waiting for search completion before draining the bounded channel can block the producer.

use std::sync::{Arc, atomic::AtomicBool};

use exhume_grep::{
    search, MatcherOptions, Pattern, SearchEvent, SearchRequest, Status,
};
use tokio::sync::mpsc;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let matcher = MatcherOptions {
        patterns: vec![Pattern {
            id: "credential".to_string(),
            expression: "password".to_string(),
        }],
        ..Default::default()
    };
    let request = SearchRequest::new("case.sqlite", 1, matcher);
    let cancel = Arc::new(AtomicBool::new(false));
    let (tx, mut rx) = mpsc::channel::<SearchEvent>(128);

    let consumer = async move {
        while let Some(event) = rx.recv().await {
            match event {
                SearchEvent::Match { files, hit } => {
                    for file in std::iter::once(&files.file).chain(&files.aliases) {
                        println!(
                            "{:?}:{}:{} pattern={}",
                            file.path, hit.offset, hit.length, hit.pattern_id
                        );
                    }
                }
                SearchEvent::FileError { file, message } => {
                    eprintln!("file {}: {message}", file.id);
                }
                SearchEvent::SourceError { message, .. } => {
                    eprintln!("source: {message}");
                }
                SearchEvent::SkippedRange { files, range } => {
                    eprintln!(
                        "file {}: skipped {} bytes at {} ({})",
                        files.file.id, range.length, range.offset, range.reason
                    );
                }
                _ => {}
            }
        }
    };

    let (result, ()) = tokio::join!(search(request, tx, cancel.clone()), consumer);
    let summary = result?;
    println!(
        "status={:?} scanned={} matched={} hits={}",
        summary.status, summary.scanned_files, summary.matched_files, summary.hits
    );
    anyhow::ensure!(
        summary.status == Status::Complete,
        "search did not complete with full coverage: {:?}",
        summary.status
    );
    Ok(())
}

The library returns Result<SearchSummary>. An Ok result can still contain an Incomplete, Limited, or Cancelled status. Handle both errors and the summary status; receiving some matches is not proof of complete coverage.

Configure matching and scope

MatcherOptions::default() selects case-sensitive literals, UTF-8, a 1 MiB read chunk, an 8 MiB regex line limit, no byte previews, and no matching-line events. Supply at least one Pattern with a unique, nonempty ID and a nonempty expression.

For case-insensitive regex matching across supported encodings:

use exhume_grep::{Encoding, MatcherOptions, Mode, Pattern};

let matcher = MatcherOptions {
    mode: Mode::Regex,
    patterns: vec![Pattern {
        id: "credentials".into(),
        expression: "(?i)password|secret".into(),
    }],
    encodings: vec![Encoding::Utf8, Encoding::Utf16le, Encoding::Utf16be],
    show_line: true,
    line_bytes: 4096,
    ..Default::default()
};

Set scope and limits on SearchRequest before passing it to search:

Field	Purpose
database, evidence_id	Existing index and the evidence row to search.
source	Optional relocated evidence path; identity checks still apply.
partition_ids	Restrict to these indexed partition IDs; empty selects all for the evidence.
directory	Indexed directory and its descendants.
include_paths, exclude_paths	Regex filters over normalized indexed paths.
min_size, max_size	Inclusive bounds on indexed file sizes in bytes.
matcher	Patterns, encodings, matching mode, and context limits.
workers	Independent search workers, 1–64; default: 1.
files_only	Emit FileMatched instead of individual hits; stop at the first hit per file.
max_hits, max_hits_per_file	Optional positive hit limits.

files_only cannot be combined with matcher.show_line. Each worker opens its own source, so seek cursors are not shared with other workers or viewers. Applications should not depend on a fixed result order when using multiple workers.

Events and results

Events serialize as JSON objects with a snake-case event tag. Indexed native file identifiers serialize as decimal strings to preserve large APFS IDs when consumed by JavaScript.

Rust variant / JSON event	Content
Started / started	Schema version, evidence ID, resolved source, matcher options, and worker count.
FileStarted / file_started	Primary indexed file, aliases, and source view.
Match / match	File group and a Hit containing pattern ID, encoding, original byte offset, length, and optional hex preview.
MatchingLine / matching_line	Optional escaped display text, line coordinates, highlight spans, and truncation/partial flags.
FileMatched / file_matched	A matching file group when files_only is enabled.
SkippedRange / skipped_range	Unsearched byte range, encoding, alignment, and reason.
FileError / file_error	Indexed file and a diagnostic message.
SourceError / source_error	Evidence ID and a source-level diagnostic.
FileFinished / file_finished	File group, status, bytes read, and hit count.
Progress / progress	Aggregate bytes read, scanned files, and hits.
Finished / finished	Final SearchSummary.

The Started event uses schema version 1 by default and 2 when matching-line events are requested. MatchingLine supplements individual Match events; count occurrences from Match, not from the number of display lines.

FileGroup contains a primary IndexedFile and any selected aliases for the same filesystem identifier within a partition. A single content scan can therefore represent multiple indexed paths. This is why selected_paths may differ from scanned_files.

All hit offsets and lengths refer to the original logical file bytes, including UTF-16 hits. In MatchingLine, line_offset is the start of the logical line; offset and length locate the displayed excerpt. LineSpan.start and .end are UTF-8 byte indices into the escaped text string. They are not character indices or JavaScript UTF-16 indices. Each span also carries its original byte offset and length.

Display text contains no ANSI codes and is not a lossless extraction of the source. Check truncated_start, truncated_end, and partial before presenting an excerpt as a complete line.

SearchSummary reports status, selected paths, scanned/matched/failed files, hits, bytes read, skipped ranges, limited files, elapsed milliseconds, and time to the first hit. An invalid request or fatal error can return Err without a Finished event, so always inspect the return value of search as well as the event stream.

Cancellation and backpressure

Retain a clone of the cancellation token in your UI or job controller. To request cancellation from another task:

use std::sync::atomic::Ordering;

cancel.store(true, Ordering::Relaxed);

Cancellation is cooperative. Continue draining the channel while awaiting completion if the application needs remaining events and the final summary. Dropping the receiver also stops the scan. The bounded channel applies backpressure when the consumer cannot keep up, avoiding an unbounded queue of matches.