EML Archiver (Base Capability)

Depends on: none

What this is

A repeatable process for retrieving email as byte-authentic .eml (RFC822) files and filing them into a permanent, labeled, chronologically-ordered archive with companion sidecars and a run manifest. This is a base capability: it says nothing about what to search for or why — that's Email Topic Discovery (FTR-40), which depends on this feature for its retrieval/archive mechanics.

Two proven retrieval methods are covered: Office 365 / Exchange Online via the Microsoft Graph API (scripted, no browser interaction), and a manual browser-reconciliation fallback for any webmail. A third method (generic IMAP) is explicitly out of scope for now — see "What this doesn't cover" below.

Assess first

• Confirm which retrieval method applies: is the target mailbox Microsoft 365 / Exchange Online (Method A, scripted, preferred), or does it need the manual browser fallback (Method B)?
• For Method A: confirm the person can complete a one-time device-code sign-in themselves — Claude never sees or enters a password.
• Read admin\AGENT_INSTRUCTIONS.md for any existing archive naming convention to reuse rather than inventing a new one.

Ask, only if needed

• Confirm the output folder name (matter or scope name) before the first run, since it becomes the top-level folder for the EML/Sidecars/Manifest set.
• If the mailbox provider is neither O365 nor easily done by hand (e.g. a generic IMAP host), say plainly that this isn't built yet (Method C, see below) rather than guessing at one.

Write

Method A — Office 365 / Outlook, scripted (preferred when available)

Raw Microsoft Graph REST calls over PowerShell, no SDK install required.

1. Device code request (interactive-but-unattended auth; the user signs in on a separate Microsoft page, never inside this tool): POST https://login.microsoftonline.com/organizations/oauth2/v2.0/devicecode with client_id=14d82eec-204b-4c2f-b7e8-296a70dab67e (the public "Microsoft Graph Command Line Tools" app), scope=https://graph.microsoft.com/Mail.Read offline_access. Returns a device_code, a short user_code, and a verification URL — show the user the code/URL; they sign in themselves.
2. Poll for the token: POST .../oauth2/v2.0/token with grant_type=urn:ietf:params:oauth:grant-type:device_code. The resulting access token is short-lived, scoped read-only to Mail.Read, and never requires seeing a password.
3. Resolve each captured ID to a Graph message ID: GET /v1.0/me/messages?$filter=internetMessageId eq '<url-encoded Message-ID>'&$select=id — start from the Internet Message-ID (stable across Graph, OWA, and raw MIME), never a provider-internal item ID alone.
4. Fetch the raw, byte-authentic MIME: GET /v1.0/me/messages/{id}/$value — byte-identical to Outlook Web's "Download as EML" (verified header-by-header against a manual download, including ARC-Seal/ARC-Message-Signature/ARC-Authentication-Results).
5. Save the bytes — see the PowerShell binary-download gotcha below; never Invoke-WebRequest -OutFile for this step.
6. Delete the token from local disk once the run completes (it was written to a temp file for the polling loop, e.g. $env:TEMP\<user>_token_state.json). Nothing long-lived should remain on disk.

Needs: the target mailbox on Microsoft 365 / Exchange Online, and a person able to complete the device-code sign-in once per run.

Method B — Manual reconciliation list (any webmail, fallback)

Used when scripted access isn't available or isn't worth setting up for a small remainder. Generate a self-contained HTML page listing every not-yet-archived match (reconciled by Internet Message-ID), with date, direction, subject (linked), counterparty, attachment flag, and a direct "Open" link into the message.

For Office 365/OWA, build the Open link from the message's ItemID: outlook.office365.com/owa/?ItemID=...&exvsurl=1&viewmodel=ReadMessageItem — clicking it opens the message directly, ready for "..." menu → Download → Download as EML.

A real, working example of this HTML (from actual production use, genericized) is included as this feature's own supporting asset — see "Accompanying reference package" below. This method is Outlook/OWA-specific as written; for other webmail (Gmail web, etc.) the same pattern applies but the link format needs adapting per provider.

Naming & storage conventions

For a new run, adapt the top-level name to the matter:

<Matter or Scope Name> - EML\                 numbered .eml files, chronological oldest->newest
<Matter or Scope Name> - Sidecars\             one .txt sidecar + extracted attachments per email, same ID
<Matter or Scope Name> - Run Descriptor.md    the run descriptor (see FTR-40)
_manifest.csv                                  ID / filename / source / subject / date / attachment list

Filename pattern per email: {3-digit sequence}{sanitized subject}{" (N)" if duplicate subject}.eml / .txt — illegal Windows filename characters replaced with ""; cap the sanitized-subject portion at ~150 chars even with the MAX_PATH workaround below.

Lessons learned / known gotchas (read before rebuilding this)

Each hard-won during real production use. Skipping this section means re-discovering them the hard way.

PowerShell + square brackets in filenames. [ and ] in a literal path are read as wildcard character-class syntax by Test-Path, Copy-Item, and Invoke-WebRequest -OutFile. Fix: always use -LiteralPath for file cmdlets; avoid -OutFile for binary downloads entirely.

Binary downloads: don't use Invoke-WebRequest -OutFile. On Windows PowerShell 5.1 (IE-engine backed), it can silently return an essentially-empty response for a large binary payload with no thrown exception. -UseBasicParsing fixes the empty-response problem but then returns .Content as a String instead of Byte[], breaking WriteAllBytes. Fix that actually works — bypass it entirely:

Add-Type -AssemblyName System.Net.Http
$client = New-Object System.Net.Http.HttpClient
$client.DefaultRequestHeaders.Authorization = New-Object System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", $token)
$bytes = $client.GetByteArrayAsync($url).Result
[System.IO.File]::WriteAllBytes($path, $bytes)

Windows MAX_PATH (260 chars) is easy to hit with a long base path plus a long sanitized filename. Two working fixes: (1) subst Y: "<longFolderPath>" for a short drive-letter alias in interactive PowerShell; (2) in a Python build script, a lp() helper applying the \\?\ extended-length prefix to every path passed to file I/O, plus defensively capping filename length.

Console output + emoji/Unicode paths. Windows console output redirected to a file defaults to the system codepage (cp1252), which can't encode an emoji folder segment or many real email subjects — causes a UnicodeEncodeError mid-print(), and if it happens inside an exception handler logging the original error, a single recoverable item failure becomes a full script crash. Fix (Python, at the very top of the script):

import sys
try:
    sys.stdout.reconfigure(encoding='utf-8', errors='replace')
    sys.stderr.reconfigure(encoding='utf-8', errors='replace')
except Exception:
    pass

message/rfc822 attachments (forwarded emails). Python's email library (policy=email.policy.default, iter_attachments()) returns an EmailMessage object — not bytes — when an attachment is itself a forwarded email. Detect this (isinstance(payload, email.message.Message)) and convert with .as_bytes() before writing, or the write crashes.

Excel COM + PowerShell: mixed-type Value2 writes on the same row can throw "Specified cast is not valid". Root cause: PowerShell's dynamic COM member-invocation binder can misbehave across mixed types (string, then a date-as-double, then string again) in sequence — a PowerShell COM-interop quirk, not an Excel or logic bug. Fix: use late-bound reflection instead of the adaptive property setter for every cell write (see FTR-36 for the exact pattern already documented there).

Move-Item -Force on a deeply nested destination can silently "succeed" while some long-named files are left behind if the destination path crosses MAX_PATH for only part of the batch. Always re-verify the actual post-move file count against your manifest — don't trust a script's self-reported tally alone. Retry missing names through a subst-aliased short path if a mismatch turns up.

Duplicate email-signature/logo noise. Extracting every MIME part also pulls out inline signature graphics, social-media icons, and thread-quoted stickers — not real attachments. Identify by: auto-generated filename patterns (Outlook-xxxxxxxx.png, imageNNN.png), exact SHA-256 duplication across many messages, and small file size. Never delete automatically — move matches into a clearly-named "safe to delete after review" subfolder with a manifest, and let a human do the final delete.

Verification discipline pays for itself. Every write session: confirm the target exists and is non-zero before editing; back it up first; make the edit; confirm the modified timestamp changed and size is reasonable; then re-open in a fresh, separate session and read back what was written, to prove it actually persisted.

What this doesn't cover

• Generic IMAP retrieval (Method C) — explicitly not yet built. Candidate approaches to evaluate when first needed: Python imaplib + UID SEARCH + FETCH (BODY[]) for raw RFC822 bytes (closest parallel to the Graph $value approach above), or a dedicated IMAP CLI tool if one proves more reliable for bulk fetch/resumability.
• What to search for and why — that's Email Topic Discovery (FTR-40), which depends on this feature.
• A run-descriptor generator script — deferred; the descriptor template (see FTR-40) is filled in by hand for now.

Accompanying reference package — EML_Archiver_Toolkit.zip

This feature ships with a downloadable toolkit, separate from this spec document: the generalized archive-builder script, the Office 365 device-code fetch script, and a real (genericized) working example of the Method B reconciliation HTML. On installation, this package should be:

1. Downloaded from the feature site alongside this spec.
2. Unzipped into the host project's scripts\ or equivalent tooling location as its own top-level folder, EML_Archiver_Toolkit\ — not merged file-by-file into an existing structure.
3. Referenced from the project's Agent Instructions or CLAUDE.md as a pointer only (path plus one-line purpose) — never re-summarized or duplicated into the instructions themselves.

Contents: build_eml_archive.py (generalized archive-builder, parameterize the top block then run); graph_api_device_code_fetch.ps1 (generalized Office 365 fetch script, parameterize the top block then run); manual_reconciliation_list_example.html (the real, working Method B HTML structure, genericized from its original production data — copy its table/link-generation pattern forward for a new scope). Download: data/items/eml-archiver-toolkit.zip

Confirm

Confirm: EML Archiver (Base Capability) is installed — Method A (Office 365 Graph API) and Method B (manual reconciliation fallback) are both documented, the toolkit scripts and HTML example are unzipped into the project's tooling location, and the gotchas list is on file for anyone rebuilding this. Want to install another, or is that everything for now?