Debugging SHA-256 Hash Generation Failures in eDiscovery Chain-of-Custody Pipelines

An ingestion batch halts with MemoryError: unable to allocate 16.4 GiB for read buffer, and after a hasty chunking patch a handful of files start reporting SHA-256 digests that no longer match the forensic workstation baseline. This failure lands in the Processing stage of the EDRM pipeline, at the exact boundary owned by cryptographic hash generation — the transformation-free anchor that must compute a file’s identity from its raw bytes before any parsing touches it. When a digest either fails to compute (OOM) or computes wrong (silent byte-stream normalization), the chain of custody is broken at intake: the item’s immutable identity is either missing or unfaithful to what the custodian produced, and every downstream dedup and production decision inherits the defect. This page isolates both failure modes — heap exhaustion on large binaries and deterministic digest divergence from text-mode reads — and gives a minimal, defensible fix.

Diagnostic Log Signatures

Both failures are deterministic and reproduce on re-run against the same media. A worker hitting the memory ceiling logs a fatal allocation error and exits non-zero; a worker that “recovered” with a naive text-mode read logs a clean success but emits a digest that fails cross-validation:

text
ERROR  hash_worker: MemoryError: unable to allocate 16.4 GiB for read buffer (file=custodian_02.pst size=17587891200)
INFO   hash_worker: retrying with chunked read
WARN   hash_worker: opened file in text mode (r); universal-newline translation active
INFO   hash_worker: sha256_complete file=notes_legacy.wpd digest=9f2c…a17b
ERROR  validation: forensic_baseline_mismatch file=notes_legacy.wpd expected=4e8b…c390 got=9f2c…a17b
INFO   validation: byte_count_mismatch processed=41288 st_size=41302

Container exit codes tell the same story: 137 (128 + SIGKILL) is the cgroup OOM-killer terminating the worker, while the digest-divergence path exits 0 and only surfaces later at the validation gate. Symptom checklist:

  • MemoryError or exit 137 on files larger than roughly the container’s memory limit.
  • A small fraction of files (~0.0003% of a batch) hash successfully but diverge from the sha256sum/FTK baseline, reproducibly.
  • Divergence correlates with mixed CRLF/LF content, legacy formats (WordPerfect, OLE compound documents), or st_size that is a few bytes larger than the processed byte count.
  • Downstream schema validation rejects the batch and raises an audit flag rather than crashing.

Root-Cause Breakdown

Neither symptom is a flaw in SHA-256 itself — both are I/O defects that hand the wrong bytes (or no bytes) to hashlib.update(). Four contributing factors compound:

  1. Whole-file reads under a cgroup limit. A synchronous file.read() loads the entire binary into resident memory before the hasher sees a byte. On a multi-gigabyte PST inside a memory-capped container this triggers an OOM kill (exit 137) or, under swap thrashing, a silent truncation that produces a valid-looking but wrong digest.
  2. Implicit byte-stream normalization. Opening a file in text mode (open(path, "r")) activates Python’s universal-newline translation, rewriting every \r\n to \n and applying platform encoding fallbacks before the bytes reach the hasher. The digest then reflects a normalized stream, not the exact bytes on disk — the byte_count_mismatch line above (14 bytes short on a file with 14 CRLF pairs) is this defect in the logs.
  3. Shared file descriptors without seek(0). Async workers that reuse a descriptor across tasks without resetting position feed the hasher a partial or overlapping range, yielding a truncated digest that passes length checks but fails forensic cross-validation.
  4. No post-read byte-count assertion. Without comparing processed bytes against st_size, a truncated read from an OOM kill or a partial descriptor commits a well-formed-but-wrong digest silently, turning a transient I/O fault into durable corrupted state.
Text-mode read versus binary streaming for one file The same custodian bytes take two paths. A text-mode read applies universal-newline translation, rewriting 0D 0A to 0A before hashing, so peak memory tracks the whole file (OOM risk) and the digest 9f2c…a17b is computed over 41288 bytes — short of st_size 41302 — and fails the baseline. A binary fixed-window read applies no translation, feeds the exact 0D 0A bytes to the accumulator, holds memory flat at the buffer size, and yields digest 4e8b…c390 over 41302 bytes that equals st_size and matches the forensic baseline. text mode  open(path, "r") binary stream  open(path, "rb") On disk — exact custodian bytes On disk — exact custodian bytes … 6F 73  0D 0A  6C 65 … … 6F 73  0D 0A  6C 65 … Universal-newline translation rewrites 0D 0A → 0A No translation bytes pass through unaltered … 6F 73  0A  6C 65 … … 6F 73  0D 0A  6C 65 … hashlib.sha256().update() fed the normalized stream hashlib.sha256().update() fed the exact on-disk bytes peak memory ≈ whole file  ·  OOM risk (exit 137) peak memory = fixed buffer  ·  flat digest 9f2c…a17b processed 41288  <  st_size 41302 ✗ diverges from forensic baseline digest 4e8b…c390 processed 41302  =  st_size 41302 ✓ matches forensic baseline

Remediation Architecture

The digest of a stream is independent of how it is chunked only if every on-disk byte reaches hashlib.update() exactly once, in order, unaltered. The fix enforces that with three controls — strict binary streaming, a fixed buffer that bounds memory, and a byte-count assertion against filesystem size — so both the OOM path and the normalization path are closed at once. The routine below is thread-safe for dispatch across an async batch processing worker pool and holds peak per-file memory at the buffer size regardless of file size.

1. Stream in binary with a bounded buffer

python
import hashlib
import os
from pathlib import Path
from typing import NamedTuple

# 8 MiB balances NVMe I/O throughput against per-worker heap footprint.
CHUNK_SIZE = 8 * 1024 * 1024


class DefensibleDigest(NamedTuple):
    sha256: str
    bytes_processed: int


def compute_defensible_sha256(
    file_path: str, chunk_size: int = CHUNK_SIZE
) -> DefensibleDigest:
    """Compute a forensically valid SHA-256 with bounded memory and truncation detection.

    Reads strictly in binary ("rb") so no universal-newline translation or
    encoding coercion can alter the byte stream, streams in fixed windows so
    peak memory equals the buffer size, and asserts the processed byte count
    against st_size to reject any partial or truncated read.
    """
    path = Path(file_path)
    if not path.is_file():
        raise FileNotFoundError(f"Target ESI file does not exist: {file_path}")
    if not os.access(file_path, os.R_OK):
        raise PermissionError(f"Insufficient read permissions: {file_path}")

    sha256 = hashlib.sha256()
    bytes_processed = 0

    # "rb" disables newline/encoding translation; the fixed-window loop caps memory.
    try:
        with open(file_path, "rb") as f:
            while chunk := f.read(chunk_size):
                sha256.update(chunk)
                bytes_processed += len(chunk)
    except OSError as exc:
        raise RuntimeError(
            f"I/O failure during hash computation for {file_path}: {exc}"
        ) from exc

    actual_size = path.stat().st_size
    if bytes_processed != actual_size:
        raise ValueError(
            f"Byte mismatch: processed {bytes_processed} vs st_size {actual_size} "
            f"for {path.name}. File may have been modified mid-ingestion, truncated "
            "by an OOM kill, or reside on a corrupted volume; chain of custody "
            "cannot be certified."
        )

    digest = sha256.hexdigest()
    if len(digest) != 64:
        raise ValueError(f"Digest length validation failed for {path.name}")

    return DefensibleDigest(digest, actual_size)

Why each control matters, for a compliance auditor reading the diff:

  • The rb flag removes text-mode newline translation and the encoding layer, guaranteeing bit-for-bit fidelity per the Python hashlib documentation, which explicitly recommends streaming update() for large files.
  • The fixed chunk_size loop makes peak memory a function of the buffer, not the file — a 40 GiB PST hashes in the same footprint as a 4 KiB email, so the container OOM ceiling is never approached.
  • The post-read comparison of bytes_processed against st_size catches silent truncation, partial descriptor reads, and mid-ingestion modification before a wrong digest can register.

2. Recover state already corrupted in production

When divergence or OOM has already halted a run, the sequence below restores chain-of-custody continuity from quarantine through forensic cross-validation and audit. This is the same reproducibility contract that synchronizing MD5 and SHA-256 hashes across processing nodes enforces at the multi-node layer.

Chain-of-custody recovery sequence after an OOM or hash-divergence incident A left-to-right pipeline. An OOM or hash-divergence incident feeds five ordered recovery steps: isolate and quarantine, source verification, deterministic re-hash, forensic cross-validation, and audit trail. The trigger node is styled as the fault and the final audit node as the restored, defensible end state. FAULT OOM or hash divergence batch halted at intake 1 Isolate & quarantine mount source read-only 2 Source verification filesystem integrity check 3 Deterministic re-hash against original media 4 Forensic cross-validation sha256sum / FTK sample 5 Audit trail custody restored immutable ledger entry
  1. Isolate & quarantine. Pause the affected worker pool; do not retry until the source media is confirmed read-only. Mount it with ro,noatime.
  2. Source verification. Run a filesystem integrity check (fsck/chkdsk) if sparse-allocation or bad-sector errors are suspected — never re-hash a volume that is itself in doubt.
  3. Deterministic re-hash. Run compute_defensible_sha256 against the original source media (not a spooled intermediate), logging worker IDs, timestamps, buffer size, and final digests to an immutable ledger.
  4. Forensic cross-validation. Compare pipeline output against a trusted utility (sha256sum, PowerShell Get-FileHash, FTK Imager, or X-Ways) on a statistically significant sample. Any divergence points to source corruption or pipeline interference, not the algorithm.
  5. Audit trail. Record incident_id, timestamp_utc, worker_pool_id, source_file_path, expected_hash, computed_hash, byte_count, error_trace, recovery_action, validator_signature, and compliance_status (PASS/FAIL/REVIEW).

Verification Checklist

Conclusion

The two symptoms that break hash generation at intake — heap exhaustion on large binaries and a wrong-but-well-formed digest from a text-mode read — share one fix: hand the exact on-disk bytes to hashlib.update() in a fixed streaming window and prove it with a byte-count assertion. Binary mode removes the normalization that silently rewrites CRLF and invalidates the digest; the bounded buffer keeps memory flat so the OOM killer never fires; and the st_size check rejects any truncated read before it can commit corrupted state. With those controls in place the digest is once again a faithful, reproducible identity for the raw evidence, and the chain of custody holds from intake through production.

Frequently Asked Questions

Why does a text-mode read change the SHA-256 when the algorithm is deterministic?

Because the algorithm is fed different bytes, not because it behaves differently. Opening a file with open(path, "r") activates universal-newline translation, which rewrites every \r\n to \n before the data reaches the hasher, so the digest reflects a normalized stream rather than the exact on-disk bytes. The result is deterministic but wrong — it will never match a forensic tool that reads the raw file. Opening in binary mode (rb) disables that translation, and the tell-tale sign in the logs is a processed byte count a few bytes short of st_size, one byte per CRLF pair collapsed.

Does changing the chunk size alter the digest or risk another mismatch?

No. The digest of a stream is mathematically independent of how it is split, provided every byte reaches hashlib.update() exactly once and in source order. A 4 MiB buffer and a 16 MiB buffer produce byte-identical SHA-256 output for the same file. Pin CHUNK_SIZE to one constant purely so memory profiling and audit logs are reproducible across workers — tune it for I/O throughput versus heap footprint, never because the value could change the result.

A file hashes successfully but fails the byte-count assertion — is the digest usable?

No. A bytes_processed != st_size failure means the read was truncated — typically an OOM kill mid-stream or a partial descriptor read — so the digest is well-formed but computed over incomplete data. Treat the assertion as authoritative: quarantine the file, re-run compute_defensible_sha256 against the original source media on a worker with adequate memory, and never register or produce a digest whose byte count does not match the filesystem size.

Up one level: Cryptographic Hash Generation — the digest engine this failure-mode guide supports.