PSA/ee/docs/plans/2026-02-27-inbound-email-inapp-artifact-persistence-remaining-work/PRD.md

PRD — Inbound Email In-App Artifact Persistence (Remaining Work)

• Slug: 2026-02-27-inbound-email-inapp-artifact-persistence-remaining-work
• Date: 2026-02-27
• Status: Draft

Summary

Close the remaining gaps so inbound email artifact handling is fully supported by in-app callback-hook processing (direct or app-local async worker), without requiring workflow-worker/event-bus execution for core artifact persistence.

Artifacts in scope:

1. Standard email attachments become ticket documents with persisted file bytes.
2. Referenced embedded images (data:image/... and HTML-referenced cid: images only) become ticket documents.
3. Original inbound email source is persisted as a deterministic .eml ticket document.

Problem

Current in-app inbound processing creates tickets/comments, but artifact persistence behavior is incomplete/inconsistent versus target behavior:

• Attachment processing in in-app path is metadata-centric and does not reliably persist file bytes as full ticket documents.
• Embedded image extraction/persistence is not executed in the in-app path.
• Original email .eml persistence is not executed in the in-app path.
• IMAP webhook path is still event-bus handoff oriented and not aligned with in-app callback processing parity.

Goals

• Ensure in-app inbound processing (new ticket and reply paths) persists all in-scope artifacts.
• Keep artifact persistence best-effort and non-blocking for ticket/comment creation.
• Maintain deterministic idempotency for retries/duplicates.
• Support IMAP callback execution in the in-app model with bounded processing and payload caps.
• Keep CPU/memory usage bounded during base64 decode and artifact conversion to avoid request-thread starvation.

Non-goals

• Rebuilding document UI.
• Rewriting comment HTML to hosted image URLs.
• Persisting unreferenced inline CID parts.
• Introducing external workflow-worker or event-bus dependency for primary artifact persistence.
• Full distributed queue platform work (external brokers, multi-service orchestration).

Decisions (Locked)

• Persist only CID inline images that are actually referenced by HTML (cid: links).
• Use deterministic source-email filename format: original-email-<sanitized-message-id>.eml.
• App-local in-process async worker mode is allowed.

Users and Primary Flows

Flow A — New inbound email (Google/Microsoft/IMAP)

1. In-app callback receives/fetches normalized email payload.
2. Ticket is created.
3. Attachment, embedded image, and .eml artifacts are processed and attached as ticket documents.
4. Any artifact failure is logged/recorded without rolling back ticket/comment creation.

Flow B — Reply to existing ticket

1. In-app callback resolves thread/reply token and creates comment.
2. Artifact pipeline persists attachments, referenced embedded images, and .eml to the matched ticket.
3. Duplicate delivery remains idempotent.

Flow C — Duplicate/retry ingress

1. Same message arrives again.
2. Ticket/comment dedupe remains intact.
3. Artifact persistence dedupes on deterministic keys (no duplicate document rows for same source artifact).

Requirements

Functional

• Extend in-app processing to run a unified artifact pipeline after ticket/comment creation for both new and reply flows.
• Accept attachment bytes in callback payload when available (attachments[].content base64) and persist actual files/documents.
• Reuse/expose embedded image extraction for in-app path:
  • data:image/*;base64,... extraction,
  • referenced cid: mapping to inline MIME parts,
  • skip unreferenced CID inline parts.
• Persist one .eml document per inbound message in in-app path using available MIME source fields (rawMimeBase64/sourceMimeBase64/rawSourceBase64) or deterministic fallback assembly when required.
• Apply idempotency to each artifact class:
  • provider attachment,
  • extracted embedded image,
  • original email .eml.
• Wire IMAP webhook path to in-app processing mode with event-bus fallback only when explicitly configured.

Safety / Performance

• Enforce ingress caps for IMAP payload processing:
  • per-attachment max bytes,
  • total attachment bytes,
  • attachment count,
  • max MIME bytes for .eml persistence.
• Keep byte-heavy work off the request critical path when async mode is enabled (app-local worker queue).
• Bound per-message artifact processing concurrency to prevent memory/CPU spikes.

Data / Contract

• Normalize/validate inbound payload fields needed by in-app artifact pipeline:
  • attachments[].id|name|contentType|size|contentId|isInline|content,
  • rawMimeBase64/sourceMimeBase64/rawSourceBase64,
  • ingressSkipReasons.

Acceptance Criteria

• In-app processing path (without workflow-worker) can ingest inbound email and produce ticket documents for:
  • regular attachments,
  • referenced embedded images,
  • original .eml.
• Reprocessing the same message does not duplicate artifact documents.
• Oversize/over-limit artifacts are skipped with structured reasons while ticket/comment ingestion still succeeds.
• IMAP callback path supports in-app processing mode and honors caps.

Risks

• Large base64 payloads may exceed request/body limits if not capped early.
• Byte decoding in request thread can degrade latency under burst conditions.
• Mixed provider payload shapes can cause subtle artifact loss if contract normalization is incomplete.

Rollout

• Gate IMAP in-app artifact processing behind env/feature flags.
• Start with controlled tenants/providers, then expand.

References

• Existing plan baseline (superseded for remaining-scope tracking):
  • ee/docs/plans/2026-02-27-inbound-email-embedded-images-and-original-eml/