Aggregates token and cost statistics across all ingested items.
Checks if a document hash is already known.
Loads the cache from disk if it exists.
Saves dirty cache to disk.
Reconstructs or updates the dedup cache by reading existing OKF markdown files in a Git repository.
Root path of the Git repository.
Number of newly synchronized documents.
Manages SHA-256 deduplication cache with token cost accounting and git repository sync.