Mapping Metadata Fields to Relativity Load Files Without Import Errors

A Relativity import that halts on a date it cannot parse, an overlay that updates the wrong records because the identifier field was mismapped, or a family that imports without its attachment links are the recurring failures of aligning a production’s metadata to Relativity’s expected schema. This problem lives at the field-mapping layer of load-file generation standards: the DAT you generate is structurally valid, but its field names, types, date formats, or family fields do not match what the target workspace expects. It breaks the interchange contract at the semantic level — the bytes decode, but they mean the wrong thing to Relativity. This guide isolates the mapping defects that stop a Relativity import and delivers a field-mapping layer that produces a workspace-ready DAT.

Diagnostic Log Signatures

The failure appears in Relativity’s import error report or your pre-ship mapping check. Capture it verbatim.

text
[INFO] Relativity map: workspace=Matter-2214, mode=append, identifier=Control Number
[ERROR] date_parse_failed field='Sent Date' row=8821 value='08/14/2019 5:12 PM' expected 'YYYY-MM-DD HH:MM'
[ERROR] identifier_missing row=9002: 'Control Number' empty (append requires a unique key)
[ERROR] family_link_broken doc=DOC-041010 BEGATTACH set, ENDATTACH empty
[WARN] choice_field 'Confidentiality' value='Atty Eyes Only' not in workspace choice list
[CRITICAL] Relativity import BLOCKED: 3 mapping errors

Symptom checklist — two or more confirm a mapping defect:

  • A date field fails to parse because its format does not match the workspace’s expected pattern.
  • The identifier/key field is empty or not unique, so an append or overlay cannot match records.
  • Family fields (BEGATTACH/ENDATTACH or a group identifier) are incomplete, so families import unlinked.
  • A single-choice or multi-choice field carries a value not in the workspace’s coded list.
  • A numeric or yes/no field carries free text Relativity cannot coerce to its field type.

Root-Cause Breakdown

Mapping defects come from treating field names as the whole contract while ignoring types, formats, and family semantics.

  1. Date format mismatch. Relativity expects dates in a defined format (commonly ISO-like YYYY-MM-DD HH:MM), and a DAT carrying MM/DD/YYYY h:MM AM/PM fails the field’s parser. Dates must be normalized to the target format before serialization, not passed through as the source produced them.
  2. Wrong identifier for the load mode. An append creates records and needs a unique identifier per row; an overlay updates existing records and matches on an overlay identifier that must already exist in the workspace. Mapping the wrong field as the identifier — or leaving it non-unique — either fails the load or updates the wrong records.
  3. Broken family fields. Relativity links families through BEGATTACH/ENDATTACH (the family’s first and last Bates) or a group identifier. If those are inconsistent with the family_id carried from attachment and parent-child mapping, families import unlinked and the relational integrity is lost.
  4. Choice values outside the coded list. Single- and multi-choice fields in Relativity draw from a controlled list; a value not on that list either errors or silently creates a stray choice, so tag vocabularies must be reconciled to the workspace before import.

Remediation Architecture

Map through an explicit schema that carries name, type, format, and family semantics — not just a column rename.

  1. Typed field map. Define each source field’s target name, Relativity field type, and required transform (date normalization, choice reconciliation, boolean coercion), and apply it as a validated projection.
  2. Mode-aware identifier. For an append, assert the identifier is present and unique across the load; for an overlay, assert it matches the workspace’s overlay key and exists in the target.
  3. Derive family fields from family_id. Compute BEGATTACH as the family’s first Bates and ENDATTACH as its last, consistently with the family grouping, so every member links.
  4. Reconcile choice vocabularies. Validate every choice value against the workspace’s coded list before serialization, routing unknown values to a reconciliation queue rather than emitting them.

Relativity field-mapping implementation

The routine applies a typed field map, normalizes dates, derives family fields, and validates the identifier for the load mode. It compiles and runs on the standard library.

python
import logging
from dataclasses import dataclass, field
from datetime import datetime
from typing import Callable, Dict, List, Optional, Sequence

logger = logging.getLogger("ediscovery.relativity.map")


class MappingError(Exception):
    pass


@dataclass
class FieldSpec:
    source: str
    target: str
    kind: str                      # "text" | "date" | "choice" | "bool"
    choices: Sequence[str] = ()    # allowed values for a choice field


def normalize_date(value: str) -> str:
    """Coerce common source date formats to Relativity's YYYY-MM-DD HH:MM."""
    if not value:
        return ""
    for fmt in ("%m/%d/%Y %I:%M %p", "%m/%d/%Y", "%Y-%m-%d %H:%M", "%Y-%m-%dT%H:%M:%S"):
        try:
            return datetime.strptime(value, fmt).strftime("%Y-%m-%d %H:%M")
        except ValueError:
            continue
    raise MappingError(f"unparseable date: {value!r}")


def map_field(spec: FieldSpec, value: str) -> str:
    if spec.kind == "date":
        return normalize_date(value)
    if spec.kind == "bool":
        return "Yes" if value.strip().lower() in {"1", "true", "yes", "y"} else "No"
    if spec.kind == "choice":
        if value and value not in spec.choices:
            raise MappingError(f"{spec.target}: choice {value!r} not in workspace list")
        return value
    return value


@dataclass
class MappedDoc:
    identifier: str
    family_id: str
    bates_start: str
    bates_end: str
    fields: Dict[str, str] = field(default_factory=dict)


def build_family_fields(docs: List[MappedDoc]) -> None:
    """Set BEGATTACH/ENDATTACH from the family's first and last Bates so every
    member links to the same family in the workspace."""
    by_family: Dict[str, List[MappedDoc]] = {}
    for d in docs:
        by_family.setdefault(d.family_id, []).append(d)
    for members in by_family.values():
        members.sort(key=lambda d: d.bates_start)
        beg, end = members[0].bates_start, members[-1].bates_end
        for d in members:
            d.fields["BEGATTACH"], d.fields["ENDATTACH"] = beg, end


def validate_identifier(docs: List[MappedDoc], mode: str,
                        existing_keys: Optional[set] = None) -> None:
    keys = [d.identifier for d in docs]
    if any(not k for k in keys):
        raise MappingError("append/overlay requires a non-empty identifier on every row")
    if mode == "append" and len(set(keys)) != len(keys):
        raise MappingError("append requires a unique identifier per row")
    if mode == "overlay":
        missing = {k for k in keys if existing_keys and k not in existing_keys}
        if missing:
            raise MappingError(f"overlay identifiers not in workspace: {missing}")

Verification Checklist

Confirm the DAT is workspace-ready before importing:

Conclusion

Aligning a production to Relativity is a semantic mapping, not a column rename: the field names, types, date formats, family links, and choice vocabularies all have to match the target workspace or the import halts or corrupts records. Map through an explicit typed schema, normalize dates to the workspace format, choose the identifier by the load mode and enforce its uniqueness or existence, derive family fields from the same family_id the pipeline carries, and reconcile choice values before serialization. A DAT mapped this way imports into Relativity on the first attempt with families linked and records matched — the outcome that keeps the interchange clean and the matter moving.

Frequently Asked Questions

What is the difference between an append and an overlay load, and why does the identifier matter?

An append creates new records in the workspace, so its identifier must be unique across the load to avoid collisions. An overlay updates existing records, so its identifier must match an overlay key that already exists in the workspace — the load matches each row to a record by that key and updates it. Mapping the wrong field as the identifier, or leaving it non-unique, either fails the load or updates the wrong records, which is why the identifier is validated against the load mode before shipping.

Through family fields — most commonly BEGATTACH and ENDATTACH, the first and last Bates numbers of the family, or a shared group identifier. Every member of a family carries the same BEGATTACH/ENDATTACH, so Relativity groups them. If those values are inconsistent with the pipeline’s family_id — or missing — the documents import as unrelated items and the family relationships the review depends on are lost.

Why do choice fields cause import problems?

Because Relativity single- and multi-choice fields draw from a controlled list defined in the workspace. A value not on that list either raises an error or, depending on settings, silently creates a new stray choice that pollutes the coded vocabulary. Reconciling every choice value against the workspace’s list before serialization — and routing unknown values to a reconciliation queue — keeps the coded fields clean and the import error-free.

For target-schema details, see the Relativity import/export documentation and EDRM’s metadata field standards.

Up one level: Load-File Generation Standards — the subsystem that serializes productions for interchange.