PSA/docs/outbound-email-abstraction/project-plan.md

Outbound Email Abstraction Project Plan

Overview

This project introduces a unified abstraction layer for sending outbound emails that supports multiple transport methods, starting with SMTP and Resend API integration. The abstraction will provide a consistent interface while allowing for different email providers based on configuration.

Goals

1. Unified Interface: Create a single email service interface that can switch between transport methods
2. Provider Support: Initially support SMTP and Resend.com API
3. Configuration-Driven: Allow runtime switching between providers via configuration
4. Backwards Compatibility: Maintain existing email functionality while introducing the new abstraction
5. Future Extensibility: Design for easy addition of other email providers (SendGrid, Mailgun, etc.)

Current System Analysis

Existing Email Infrastructure

The Alga PSA application has a comprehensive email system with the following components:

1. Email Services (Multiple Implementations)

• Primary Service: server/src/services/emailService.ts - Singleton pattern with SMTP via nodemailer
• Notification Service: server/src/lib/notifications/emailService.ts - Handlebars template compilation
• Event-driven Email: server/src/lib/notifications/email.ts - Comprehensive notification system

2. Current Configuration Structure

// Current environment variables
EMAIL_ENABLE=false          // Global email toggle
EMAIL_FROM=noreply@example.com
EMAIL_HOST=smtp.example.com
EMAIL_PORT=587
EMAIL_USERNAME=noreply@example.com
EMAIL_PASSWORD=             // Managed via Docker secrets

3. Existing Features

• Template System: Handlebars-based with system and tenant-specific templates
• Notification Categories: Hierarchical category/subtype system
• User Preferences: Per-user, per-category notification settings
• Rate Limiting: Per-tenant rate limiting (60 emails/minute default)
• Audit Logging: Comprehensive notification logs with delivery status
• Multi-tenant Support: Row-level security with tenant isolation
• Event Bus Integration: Redis-based asynchronous email processing

4. Database Schema

• system_email_templates: System-wide default templates
• tenant_email_templates: Tenant-specific customizations with RLS
• notification_settings: Global tenant settings
• notification_categories: Tenant notification groupings
• notification_subtypes: Specific notification types
• user_notification_preferences: User-level preferences
• notification_logs: Delivery tracking and audit

5. Integration Points

• Invoice System: server/src/lib/jobs/handlers/invoiceEmailHandler.ts
• Ticket Events: server/src/lib/eventBus/subscribers/ticketEmailSubscriber.ts
• Authentication: server/src/lib/email/sendVerificationEmail.ts
• Event Bus: Asynchronous processing via Redis streams

Current Limitations

1. Single Provider: Only supports SMTP, no API-based providers
2. Separate Services: Multiple email service implementations with overlapping functionality
3. Configuration Complexity: Multiple environment variables without provider abstraction
4. Limited Fallback: No automatic failover between transport methods

Multi-Tenancy Strategy

Tiered Email Domain Approach

Based on enterprise SaaS best practices and PSA industry standards, we will implement a tiered email domain model:

Tier 1: Platform Domain (Default)

• Target: Small-medium businesses, new customers
• Implementation: All emails from tenant-name@yourdomain.com or noreply@yourdomain.com
• Benefits: Zero configuration, immediate functionality, managed reputation
• Use Case: 80% of customers who value simplicity over branding

Tier 2: Custom Domain (Enterprise Feature)

• Target: Enterprise customers, professional service firms
• Implementation: Emails from support@customer-company.com
• Requirements: Customer configures DNS (SPF, DKIM, DMARC)
• Benefits: Brand consistency, professional appearance, compliance

Tier 3: Hybrid Approach

• Target: Growing businesses, mixed requirements
• Implementation: Custom domain for client-facing emails, platform domain for internal notifications
• Benefits: Selective branding where it matters most

Centralized Provider Management

The underlying infrastructure uses a centralized provider model:

1. Single Account Management: Platform administrator manages all email provider accounts (Resend, SMTP)
2. Tenant Isolation: Each tenant's emails isolated through metadata, headers, and domain verification
3. Simplified Setup: Most tenants need zero email provider configuration
4. Enterprise Flexibility: Custom domain support for customers who require it

Email Flow Examples

Tier 1: Platform Domain

Tenant "ACME-Corp" → Provider Manager → Resend API
                                     ↓
Email from: noreply@acme-corp.yourdomain.com
Reply-to: support@acme-corp.yourdomain.com
                                     ↓
Client receives professional email with tenant branding
                                     ↓
Audit logs per tenant in existing system

Tier 2: Custom Domain

Tenant "ACME-Corp" → Provider Manager → Resend API (Verified Domain)
                                     ↓
Email from: support@acmecorp.com (Customer's Domain)
DKIM signed by: acmecorp.com
                                     ↓
Client receives email from customer's actual domain
                                     ↓
Audit logs per tenant + domain verification status

Key Multi-Tenancy Considerations

1. Tenant Isolation:

   • All email logs separated by tenant column (existing RLS)
   • Email templates isolated per tenant (existing system)
   • Rate limiting per tenant (existing system)
   • Provider-level metadata tagging for tracking

2. From Address Management:

   • Tier 1: Platform-managed subdomains (tenant-name@yourdomain.com)
   • Tier 2: Customer-owned domains (support@customer.com)
   • Tier 3: Mixed approach based on email type

3. Domain Verification & Security:

   • Platform domains: Pre-verified in Resend
   • Custom domains: Customer completes DNS verification process
   • SPF/DKIM/DMARC configuration handled per domain
   • Reputation isolation between domains

4. Compliance & Tracking:

   • Provider-level webhook handling for delivery status
   • Tenant-specific bounce/complaint handling
   • Domain-specific reputation monitoring
   • Audit trails maintained per tenant in existing notification_logs

Technical Design

Integration Strategy

The abstraction will be implemented as a centralized provider layer that:

1. Minimal Disruption: Existing email service interfaces remain unchanged
2. Gradual Migration: Services can be migrated one at a time
3. Centralized Management: Single provider accounts managed by platform admins
4. Tenant Transparency: Tenants unaware of underlying provider complexity

Core Components

1. Email Provider Interface

interface IEmailProvider {
  sendEmail(emailData: EmailData): Promise<EmailResult>;
  validateConfig(): Promise<boolean>;
  getProviderName(): string;
  isHealthy(): Promise<boolean>;
}

2. Email Data Models (Enhanced for Multi-Tenancy)

interface EmailData {
  to: string | string[];
  from: string;
  subject: string;
  html?: string;
  text?: string;
  attachments?: EmailAttachment[];
  headers?: Record<string, string>;
  replyTo?: string;
  cc?: string | string[];
  bcc?: string | string[];
  // Multi-tenant metadata (required)
  tenantId: string;      // Always required for tenant isolation
  userId?: string;       // Optional user context
  templateName?: string; // For template tracking
  // Provider-level tracking
  tags?: string[];       // Provider-specific tags for categorization
}

interface EmailResult {
  success: boolean;
  messageId?: string;
  error?: string;
  provider: string;
  timestamp: Date;
  tenantId: string;      // Tenant context for audit logs
  metadata?: {
    resendId?: string;   // Resend-specific message ID
    smtpResponse?: string; // SMTP server response
    tags?: string[];     // Applied tags
    webhookId?: string;  // For webhook correlation
  };
}

interface EmailAttachment {
  filename: string;
  path?: string;        // For file path attachments
  content?: Buffer;     // For inline content
  contentType: string;
  cid?: string;         // For embedded images
}

// Tenant-specific configuration (managed by platform)
interface TenantEmailSettings {
  tenantId: string;
  emailTier: 'platform' | 'custom' | 'hybrid';  // Email domain tier
  
  // Platform Domain Settings (Tier 1)
  platformDomain?: string;     // acme-corp.yourdomain.com
  defaultFromAddress: string;  // noreply@acme-corp.yourdomain.com
  
  // Custom Domain Settings (Tier 2)
  customDomain?: string;       // acmecorp.com
  customFromAddress?: string;  // support@acmecorp.com
  domainVerified: boolean;     // DNS verification status
  dkimEnabled: boolean;        // DKIM signing status
  
  // Common Settings
  replyToAddress?: string;     // tenant-specific reply-to
  customHeaders?: Record<string, string>; // tenant-specific headers
  tags?: string[];            // tenant-specific tags for all emails
  
  // Email Type Routing (Tier 3 - Hybrid)
  emailTypeRouting?: {
    invoices: 'platform' | 'custom';      // Where to send invoices
    notifications: 'platform' | 'custom'; // Where to send notifications
    tickets: 'platform' | 'custom';       // Where to send ticket updates
  };
}

3. Provider Implementations

SMTP Provider (Existing Compatibility)

export class SMTPEmailProvider implements IEmailProvider {
  constructor(private config: SMTPConfig) {}
  
  async sendEmail(emailData: EmailData): Promise<EmailResult> {
    // Leverage existing nodemailer logic from current services
    // Maintain compatibility with current SMTP configuration
  }
}

Resend Provider (Centralized Multi-Tenant)

export class ResendEmailProvider implements IEmailProvider {
  private resend: Resend;
  private tenantSettingsCache: Map<string, TenantEmailSettings>;
  
  constructor(private config: ResendConfig) {
    this.resend = new Resend(config.apiKey);
    this.tenantSettingsCache = new Map();
  }
  
  async sendEmail(emailData: EmailData): Promise<EmailResult> {
    // Get tenant-specific settings
    const tenantSettings = await this.getTenantSettings(emailData.tenantId);
    
    // Apply tenant-specific email configuration
    const resendData = {
      from: this.buildFromAddress(emailData.from, tenantSettings),
      to: emailData.to,
      subject: emailData.subject,
      html: emailData.html,
      text: emailData.text,
      headers: {
        ...emailData.headers,
        ...tenantSettings.customHeaders,
        'X-Tenant-ID': emailData.tenantId,  // For webhook correlation
        'X-User-ID': emailData.userId || '',
      },
      tags: [
        `tenant:${emailData.tenantId}`,
        ...(tenantSettings.tags || []),
        ...(emailData.tags || []),
      ],
    };
    
    const result = await this.resend.emails.send(resendData);
    
    return {
      success: true,
      messageId: result.id,
      provider: 'resend',
      timestamp: new Date(),
      tenantId: emailData.tenantId,
      metadata: {
        resendId: result.id,
        tags: resendData.tags,
      }
    };
  }
  
  private async getTenantSettings(tenantId: string): Promise<TenantEmailSettings> {
    // Cache tenant settings for performance
    if (this.tenantSettingsCache.has(tenantId)) {
      return this.tenantSettingsCache.get(tenantId)!;
    }
    
    // Load from database or configuration
    const settings = await this.loadTenantEmailSettings(tenantId);
    this.tenantSettingsCache.set(tenantId, settings);
    return settings;
  }
  
  private buildFromAddress(requestedFrom: string, settings: TenantEmailSettings): string {
    // Ensure all emails come from verified domains
    if (requestedFrom.includes(settings.fromDomain)) {
      return requestedFrom;
    }
    return settings.defaultFromAddress;
  }
}

4. Centralized Provider Manager

export class EmailProviderManager {
  private primaryProvider: IEmailProvider;
  private fallbackProvider?: IEmailProvider;
  private tenantService: TenantEmailSettingsService;
  
  constructor(config: EmailProviderConfig) {
    this.primaryProvider = this.createProvider(config.primary);
    if (config.fallback) {
      this.fallbackProvider = this.createProvider(config.fallback);
    }
    this.tenantService = new TenantEmailSettingsService();
  }
  
  async sendEmail(emailData: EmailData): Promise<EmailResult> {
    // Validate tenant context
    if (!emailData.tenantId) {
      throw new Error('Tenant ID is required for all email operations');
    }
    
    // Apply tenant-specific rate limiting (using existing system)
    await this.checkTenantRateLimit(emailData.tenantId, emailData.userId);
    
    try {
      const result = await this.primaryProvider.sendEmail(emailData);
      
      // Log to tenant-specific audit trail
      await this.logEmailResult(result);
      
      return result;
    } catch (error) {
      if (this.fallbackProvider) {
        const result = await this.fallbackProvider.sendEmail(emailData);
        await this.logEmailResult(result);
        return result;
      }
      throw error;
    }
  }
  
  private async checkTenantRateLimit(tenantId: string, userId?: string): Promise<void> {
    // Leverage existing notification system rate limiting
    // This integrates with the current tenant-aware rate limiting
  }
  
  private async logEmailResult(result: EmailResult): Promise<void> {
    // Log to existing notification_logs table with tenant isolation
    // Maintains existing audit trail functionality
  }
}

Resend Custom Domain Management

Based on the Context7 documentation, Resend handles custom domains through:

Domain Verification Architecture

Resend uses a subdomain-based approach to avoid conflicts with existing email infrastructure:

# Customer adds these DNS records for domain verification:

# 1. DKIM Authentication (Required)
resend._domainkey.customer.com    TXT    "resend-generated-dkim-key"

# 2. SPF Authorization (Required)  
send.customer.com    TXT    "v=spf1 include:resend.net ~all"

# 3. MX Record for Bounce Handling (Required)
send.customer.com    MX    10 feedback-smtp.us-east-1.amazonses.com

# 4. DMARC Policy (Recommended)
_dmarc.customer.com    TXT    "v=DMARC1; p=none; rua=mailto:dmarcreports@customer.com;"

Multi-Tenant Domain Support

• Single Account: One Resend account can manage multiple domains for different tenants
• Domain Isolation: Each verified domain has separate reputation and deliverability metrics
• Regional Support: Supports multiple AWS regions (us-east-1, eu-west-1, ap-northeast-1, sa-east-1)
• Webhook Isolation: Domain-specific webhook endpoints for bounce/complaint handling

Implementation for Multi-Tenancy

export class ResendEmailProvider implements IEmailProvider {
  private resend: Resend;
  private verifiedDomains: Set<string>;

  constructor(config: ResendConfig) {
    this.resend = new Resend(config.apiKey);
    this.verifiedDomains = new Set();
  }

  async sendEmail(emailData: EmailData): Promise<EmailResult> {
    const tenantSettings = await this.getTenantSettings(emailData.tenantId);
    
    // Determine which domain to use
    const fromDomain = this.selectDomain(emailData, tenantSettings);
    
    // Validate domain is verified
    if (tenantSettings.emailTier === 'custom' && !tenantSettings.domainVerified) {
      throw new Error(`Custom domain ${tenantSettings.customDomain} not verified`);
    }

    const result = await this.resend.emails.send({
      from: this.buildFromAddress(emailData.from, fromDomain),
      to: emailData.to,
      subject: emailData.subject,
      html: emailData.html,
      text: emailData.text,
      headers: {
        ...emailData.headers,
        'X-Tenant-ID': emailData.tenantId,
      },
      tags: [
        `tenant:${emailData.tenantId}`,
        `domain:${fromDomain}`,
        ...(emailData.tags || [])
      ]
    });

    return {
      success: true,
      messageId: result.id,
      provider: 'resend',
      timestamp: new Date(),
      tenantId: emailData.tenantId,
      metadata: {
        resendId: result.id,
        domain: fromDomain,
        tags: result.tags
      }
    };
  }

  private selectDomain(emailData: EmailData, settings: TenantEmailSettings): string {
    switch (settings.emailTier) {
      case 'custom':
        return settings.customDomain!;
      case 'platform':
      default:
        return settings.platformDomain || 'yourdomain.com';
    }
  }
}

Fully Automated Domain Verification Workflow

Complete API Coverage: Resend provides full API automation for domain creation, verification, and management.

Automated Domain Management Service:

export class ResendDomainService {
  private resend: Resend;
  
  constructor(apiKey: string) {
    this.resend = new Resend(apiKey);
  }
  
  async createTenantDomain(tenantId: string, domain: string): Promise<DomainSetup> {
    try {
      // 1. Create domain in Resend via API
      const domainResult = await this.resend.domains.create({
        name: domain,
        region: 'us-east-1' // or tenant-specific region
      });
      
      // 2. Store domain info and DNS records in database
      await this.storeDomainSetup(tenantId, {
        resendDomainId: domainResult.id,
        domain: domainResult.name,
        status: domainResult.status,
        dnsRecords: domainResult.records,
        region: domainResult.region
      });
      
      // 3. Send DNS configuration instructions to tenant
      await this.sendDNSInstructions(tenantId, domainResult.records);
      
      return domainResult;
    } catch (error) {
      throw new Error(`Failed to create domain: ${error.message}`);
    }
  }
  
  async checkDomainVerification(tenantId: string): Promise<DomainStatus> {
    const domainSetup = await this.getTenantDomainSetup(tenantId);
    
    // Get current status from Resend API
    const domainInfo = await this.resend.domains.get(domainSetup.resendDomainId);
    
    // Update local database with current status
    await this.updateDomainStatus(tenantId, domainInfo.status);
    
    return {
      domain: domainInfo.name,
      status: domainInfo.status, // 'not_started' | 'pending' | 'verified' | 'failed'
      records: domainInfo.records,
      verifiedAt: domainInfo.status === 'verified' ? new Date() : null
    };
  }
  
  async triggerDomainVerification(tenantId: string): Promise<boolean> {
    const domainSetup = await this.getTenantDomainSetup(tenantId);
    
    try {
      // Trigger verification check via API
      await this.resend.domains.verify(domainSetup.resendDomainId);
      
      // Check updated status
      const status = await this.checkDomainVerification(tenantId);
      return status.status === 'verified';
    } catch (error) {
      console.error('Domain verification failed:', error);
      return false;
    }
  }
}

Workflow-Managed Domain Verification Process:

Using the existing workflow system to orchestrate the entire domain verification lifecycle:

// System workflow for domain verification management
async function domainVerificationWorkflow(context: WorkflowContext): Promise<void> {
  const { actions, events, logger, setState, data } = context;
  const { tenantId, domain } = context.input.triggerEvent.payload;
  
  setState('DOMAIN_CREATION_STARTED');
  logger.info(`Starting domain verification workflow for ${domain}`);
  
  try {
    // 1. Create domain in Resend via API
    const domainResult = await actions.createResendDomain({
      tenantId,
      domain,
      region: 'us-east-1'
    });
    
    data.set('domainSetup', domainResult);
    setState('DOMAIN_CREATED_DNS_PENDING');
    
    // 2. Send DNS instructions to tenant
    await actions.sendDNSInstructions({
      tenantId,
      domain,
      dnsRecords: domainResult.dnsRecords
    });
    
    // 3. Wait for tenant confirmation or timeout
    setState('AWAITING_DNS_CONFIGURATION');
    const dnsConfiguredEvent = await events.waitFor(
      'DNS_CONFIGURED',
      60 * 60 * 1000 // 1 hour timeout
    );
    
    if (!dnsConfiguredEvent) {
      // Create inline human task for follow-up if tenant doesn't respond
      const taskResult = await actions.createInlineTaskAndWaitForResult({
        title: `Follow up on DNS configuration for ${domain}`,
        description: `DNS configuration timeout. Please complete DNS setup for domain ${domain} or contact support if you need assistance.`,
        formDefinition: {
          jsonSchema: {
            type: 'object',
            title: 'DNS Configuration Follow-up',
            properties: {
              domainInfo: {
                type: 'string',
                title: 'Domain Information',
                readOnly: true,
                default: `Domain: ${domain}\nWaiting Time: 1 hour\nStatus: DNS configuration timeout`
              },
              dnsRecordsDisplay: {
                type: 'string',
                title: 'Required DNS Records',
                readOnly: true,
                default: `${JSON.stringify(domainResult.dnsRecords, null, 2)}`
              },
              userAction: {
                type: 'string',
                title: 'What would you like to do?',
                enum: ['continue_waiting', 'cancel_setup', 'contact_support'],
                enumNames: ['Continue waiting (DNS records are configured)', 'Cancel domain setup', 'Contact support for assistance']
              },
              notes: {
                type: 'string',
                title: 'Additional Notes (optional)',
                description: 'Any additional information about the DNS configuration'
              }
            },
            required: ['userAction']
          }
        },
        contextData: {
          tenantId,
          domain,
          dnsRecords: domainResult.dnsRecords,
          hoursWaiting: 1
        },
        waitForEventTimeoutMilliseconds: 24 * 60 * 60 * 1000 // 24 hours
      });
      
      if (!taskResult.success || taskResult.resolutionData?.userAction === 'cancel_setup') {
        setState('DNS_CONFIGURATION_ABANDONED');
        return;
      }
    }
    
    // 4. Begin verification polling loop
    setState('VERIFYING_DOMAIN');
    let verificationAttempts = 0;
    const maxAttempts = 20; // 10 minutes total (30s intervals)
    
    while (verificationAttempts < maxAttempts) {
      verificationAttempts++;
      
      // Trigger verification check in Resend
      const verificationResult = await actions.triggerDomainVerification({
        tenantId,
        resendDomainId: domainResult.resendDomainId
      });
      
      if (verificationResult.status === 'verified') {
        setState('DOMAIN_VERIFIED');
        
        // 5. Activate domain for email sending
        await actions.activateCustomDomain({
          tenantId,
          domain,
          resendDomainId: domainResult.resendDomainId
        });
        
        // 6. Notify tenant of successful verification
        await actions.sendDomainVerificationSuccess({
          tenantId,
          domain
        });
        
        setState('DOMAIN_ACTIVE');
        logger.info(`Domain ${domain} successfully verified and activated`);
        return;
      }
      
      if (verificationResult.status === 'failed') {
        // Create human task for DNS troubleshooting
        const troubleshootResult = await actions.createInlineTaskAndWaitForResult({
          title: `DNS verification failed for ${domain}`,
          description: `Domain verification has failed. Please review your DNS settings and try again.`,
          formDefinition: {
            jsonSchema: {
              type: 'object',
              title: 'DNS Verification Failed',
              properties: {
                failureInfo: {
                  type: 'string',
                  title: 'Failure Information',
                  readOnly: true,
                  default: `Domain: ${domain}\nFailure Reason: ${verificationResult.failureReason}\nAttempt: ${verificationAttempts} of ${maxAttempts}`
                },
                dnsRecordsDisplay: {
                  type: 'string',
                  title: 'Required DNS Records',
                  readOnly: true,
                  default: `${JSON.stringify(domainResult.dnsRecords, null, 2)}`
                },
                nextAction: {
                  type: 'string',
                  title: 'What would you like to do?',
                  enum: ['retry_verification', 'cancel_setup', 'contact_support'],
                  enumNames: ['Retry verification (I\'ve fixed the DNS)', 'Cancel domain setup', 'Contact support for help']
                },
                troubleshootingNotes: {
                  type: 'string',
                  title: 'Troubleshooting Notes (optional)',
                  description: 'Any changes you made or issues you encountered'
                }
              },
              required: ['nextAction']
            }
          },
          contextData: {
            tenantId,
            domain,
            failureReason: verificationResult.failureReason,
            dnsRecords: domainResult.dnsRecords,
            currentAttempt: verificationAttempts
          },
          waitForEventTimeoutMilliseconds: 2 * 60 * 60 * 1000 // 2 hours
        });
        
        if (troubleshootResult.success && troubleshootResult.resolutionData?.nextAction === 'retry_verification') {
          // Reset attempt counter and continue
          verificationAttempts = 0;
          setState('RETRYING_VERIFICATION');
          continue;
        } else {
          setState('DOMAIN_VERIFICATION_FAILED');
          return;
        }
      }
      
      // Wait 30 seconds before next attempt
      await new Promise(resolve => setTimeout(resolve, 30000));
    }
    
    // Max attempts reached - escalate to human task
    await actions.createInlineTaskAndWaitForResult({
      title: `Domain verification timeout for ${domain}`,
      description: `Domain verification has timed out after ${maxAttempts} attempts. Manual intervention may be required.`,
      formDefinition: {
        jsonSchema: {
          type: 'object',
          title: 'Domain Verification Timeout',
          properties: {
            timeoutInfo: {
              type: 'string',
              title: 'Timeout Information',
              readOnly: true,
              default: `Domain: ${domain}\nAttempts Completed: ${maxAttempts}\nTotal Time: ~${Math.round(maxAttempts * 30 / 60)} minutes`
            },
            dnsRecordsDisplay: {
              type: 'string',
              title: 'Required DNS Records',
              readOnly: true,
              default: `${JSON.stringify(domainResult.dnsRecords, null, 2)}`
            },
            resolution: {
              type: 'string',
              title: 'How would you like to proceed?',
              enum: ['manual_verification', 'cancel_setup', 'escalate_support'],
              enumNames: ['Manually verify domain (override)', 'Cancel domain setup', 'Escalate to technical support']
            },
            notes: {
              type: 'string',
              title: 'Additional Information',
              description: 'Any additional context about the timeout or domain setup'
            }
          },
          required: ['resolution']
        }
      },
      contextData: {
        tenantId,
        domain,
        attemptsCompleted: maxAttempts,
        dnsRecords: domainResult.dnsRecords
      }
    });
    
    setState('DOMAIN_VERIFICATION_TIMEOUT');
    
  } catch (error) {
    logger.error(`Domain verification workflow failed: ${error.message}`);
    setState('DOMAIN_VERIFICATION_ERROR');
    
    // Create error resolution task
    await actions.createInlineTaskAndWaitForResult({
      title: `Domain verification error for ${domain}`,
      description: `An unexpected error occurred during domain verification. Technical support may be required.`,
      formDefinition: {
        jsonSchema: {
          type: 'object',
          title: 'Domain Verification Error',
          properties: {
            errorInfo: {
              type: 'string',
              title: 'Error Information',
              readOnly: true,
              default: `Domain: ${domain}\nError: ${error.message}\nWorkflow Execution: ${context.executionId}`
            },
            errorDetails: {
              type: 'string',
              title: 'Technical Details',
              readOnly: true,
              default: JSON.stringify({
                tenantId,
                domain,
                error: error.message,
                stack: error.stack,
                timestamp: new Date().toISOString()
              }, null, 2)
            },
            nextStep: {
              type: 'string',
              title: 'Next Steps',
              enum: ['retry_workflow', 'cancel_setup', 'escalate_technical'],
              enumNames: ['Retry domain verification workflow', 'Cancel domain setup', 'Escalate to technical team']
            },
            adminNotes: {
              type: 'string',
              title: 'Admin Notes',
              description: 'Any additional context or troubleshooting steps taken'
            }
          },
          required: ['nextStep']
        }
      },
      contextData: {
        tenantId,
        domain,
        error: error.message,
        workflowExecutionId: context.executionId
      }
    });
  }
}

Workflow-Driven Benefits:

1. Automated Orchestration: Complete automation of the domain verification process
2. Human Task Integration: Automatic escalation when manual intervention needed
3. Timeout Management: Built-in timeout handling with human task creation
4. Retry Logic: Intelligent retry mechanisms with human oversight
5. Audit Trail: Complete event-sourced history of domain verification process
6. Error Recovery: Graceful error handling with human task escalation
7. Status Tracking: Real-time workflow state updates visible in admin dashboard

Benefits for Multi-Tenant SaaS

• Reputation Isolation: Each tenant's domain has independent reputation
• Brand Consistency: Tenants can send from their own domains
• Compliance: Meets enterprise requirements for branded communications
• Scalability: Single Resend account can manage hundreds of tenant domains
• Cost Efficiency: No need for separate Resend accounts per tenant

Implementation Plan

Phase 1: Foundation & Provider Abstraction (Week 1-2)

1. Provider Interface: Create IEmailProvider interface and base types
2. SMTP Provider: Extract existing SMTP logic into SMTPEmailProvider
3. Provider Manager: Implement EmailProviderManager with fallback support
4. Configuration Schema: Design unified configuration structure
5. Testing Framework: Set up unit tests for provider abstraction

Deliverables:

• server/src/lib/email/providers/IEmailProvider.ts
• server/src/lib/email/providers/SMTPEmailProvider.ts
• server/src/lib/email/EmailProviderManager.ts
• Unit tests for SMTP provider maintaining existing functionality

Phase 2: Resend Integration (Week 2-3)

1. Resend Provider: Implement ResendEmailProvider with full API integration
2. Environment Variables: Add Resend configuration to environment schema
3. Configuration Migration: Update existing services to use provider manager
4. Rate Limiting: Implement Resend-specific rate limiting and quota management
5. Testing: Create integration tests with Resend test endpoints

Deliverables:

• server/src/lib/email/providers/ResendEmailProvider.ts
• Updated environment configuration
• Integration tests with Resend sandbox
• Documentation for Resend setup and domain verification

Phase 3: Service Integration & Migration (Week 3-4)

1. Service Refactoring: Update existing email services to use provider manager
   • Migrate server/src/services/emailService.ts
   • Update server/src/lib/notifications/emailService.ts
   • Integrate with notification system in server/src/lib/notifications/email.ts
2. Configuration Management: Implement runtime provider switching
3. Error Handling: Add comprehensive error handling and retry logic
4. Monitoring Integration: Add provider-specific logging and metrics

Deliverables:

• Refactored email services using provider abstraction
• Configuration management for provider selection
• Enhanced error handling and retry mechanisms
• Provider health monitoring

Phase 4: Testing, Documentation & Deployment (Week 4-5)

1. End-to-End Testing: Comprehensive testing of all email flows
2. Performance Testing: Compare provider performance and reliability
3. Documentation: Complete API documentation and deployment guides
4. Migration Guide: Create step-by-step migration instructions
5. Monitoring Dashboard: Implement email provider monitoring

Deliverables:

• Complete test suite covering all email scenarios
• Performance benchmarks and recommendations
• Updated deployment documentation
• Migration guide for existing installations
• Provider monitoring and alerting

Configuration Structure

Unified Configuration Schema

interface EmailProviderConfig {
  enabled: boolean;
  primary: ProviderConfig;
  fallback?: ProviderConfig;
  defaults: {
    from: string;
    replyTo?: string;
  };
}

interface ProviderConfig {
  type: 'smtp' | 'resend';
  config: SMTPConfig | ResendConfig;
}

interface SMTPConfig {
  host: string;
  port: number;
  secure: boolean;
  auth: {
    user: string;
    pass: string;
  };
}

interface ResendConfig {
  apiKey: string;
  domain?: string;
  rateLimitPerMinute?: number;
}

Environment Variables (Centralized Management)

# Global Email Configuration (Platform Admin Managed)
EMAIL_ENABLE=true                    # Existing: Global toggle
EMAIL_PROVIDER=resend               # New: Primary provider (smtp|resend)
EMAIL_FALLBACK_PROVIDER=smtp        # New: Fallback provider

# Existing SMTP Configuration (Maintained for Fallback)
EMAIL_HOST=smtp.gmail.com           # Existing: SMTP host
EMAIL_PORT=587                      # Existing: SMTP port  
EMAIL_USERNAME=your-email@gmail.com # Existing: SMTP username
EMAIL_PASSWORD=                     # Existing: SMTP password (Docker secret)
EMAIL_FROM=noreply@yourdomain.com   # Existing: Default from address

# Centralized Resend Configuration (Platform-Wide)
RESEND_API_KEY=re_xxxxxxxxx         # New: Single Resend account API key (Docker secret)
RESEND_VERIFIED_DOMAINS=yourdomain.com,tenant1.yourdomain.com  # Comma-separated verified domains
RESEND_DEFAULT_DOMAIN=yourdomain.com # Default domain for tenant subdomains
RESEND_WEBHOOK_SECRET=whsec_xxxxx   # Webhook signature verification secret

# Tenant Email Management
EMAIL_TENANT_SUBDOMAIN_PATTERN="{tenant}.yourdomain.com"  # Pattern for tenant subdomains
EMAIL_TENANT_FROM_PATTERN="noreply@{tenant}.yourdomain.com"  # Pattern for tenant from addresses

# Backward Compatibility (Legacy Variables)
SMTP_HOST=                          # Legacy: Alias for EMAIL_HOST
SMTP_PORT=                          # Legacy: Alias for EMAIL_PORT
SMTP_USER=                          # Legacy: Alias for EMAIL_USERNAME
SMTP_PASS=                          # Legacy: Alias for EMAIL_PASSWORD
SMTP_FROM=                          # Legacy: Alias for EMAIL_FROM

Tenant Email Configuration (Database-Driven)

Instead of environment variables per tenant, tenant-specific settings are managed in the database:

-- New table for tenant email settings
CREATE TABLE tenant_email_settings (
  id SERIAL PRIMARY KEY,
  tenant UUID NOT NULL REFERENCES tenants(tenant),
  from_domain VARCHAR(255) NOT NULL,           -- tenant1.yourdomain.com
  default_from_address VARCHAR(255) NOT NULL,  -- noreply@tenant1.yourdomain.com
  reply_to_address VARCHAR(255),               -- support@tenant1.yourdomain.com
  custom_headers JSONB DEFAULT '{}',           -- Tenant-specific headers
  tags TEXT[] DEFAULT '{}',                    -- Default tags for tenant emails
  is_active BOOLEAN DEFAULT true,
  created_at TIMESTAMP DEFAULT NOW(),
  updated_at TIMESTAMP DEFAULT NOW(),
  UNIQUE(tenant)
);

Docker Secrets Integration

# Existing Secrets (Maintained)
email_password                      # SMTP password
token_secret_key                    # Authentication
nextauth_secret                     # NextAuth

# New Secrets
resend_api_key                      # Resend API key

Testing Strategy

Unit Tests

• Provider interface compliance
• Email data validation
• Configuration validation
• Error handling scenarios

Integration Tests

• SMTP provider with test email server
• Resend provider with sandbox/test addresses
• Factory pattern provider selection
• Fallback mechanism testing

End-to-End Tests

• Complete email sending workflows
• Template rendering with different providers
• Attachment handling
• Error recovery scenarios

Security Considerations

1. API Key Management: Secure storage of Resend API keys
2. Email Validation: Prevent email injection attacks
3. Rate Limiting: Implement sending rate limits
4. Domain Verification: Ensure proper domain setup for production
5. Logging: Avoid logging sensitive email content

Monitoring & Observability

1. Metrics: Track sending success rates by provider
2. Logging: Log email sending attempts and results
3. Alerting: Alert on provider failures or high error rates
4. Dashboard: Monitor email queue and delivery statistics

Migration Strategy

Phase 1: Backward Compatible Integration

1. Provider Layer Introduction: Add provider abstraction without changing existing APIs
2. Configuration Enhancement: Extend environment variables while maintaining backward compatibility
3. Gradual Service Migration: Migrate services one at a time starting with least critical
4. Monitoring: Add comprehensive logging for both old and new implementations

Phase 2: Feature Flag Rollout

1. Environment-Based Switching: Use EMAIL_PROVIDER to control provider selection
2. Tenant-Level Configuration: Allow per-tenant provider configuration (future enhancement)
3. A/B Testing: Split traffic between providers for performance comparison
4. Fallback Validation: Thoroughly test automatic fallback mechanisms

Phase 3: Service Consolidation

1. API Unification: Consolidate multiple email service implementations
2. Legacy Cleanup: Remove duplicate email service code
3. Configuration Simplification: Streamline environment variable usage
4. Documentation Updates: Update all references to new unified system

Rollback Strategy

1. Environment Variable Rollback: Set EMAIL_PROVIDER=smtp to revert to SMTP
2. Service-Level Rollback: Capability to revert individual services to original implementation
3. Configuration Isolation: Keep old and new configurations separate during transition
4. Monitoring Alerts: Automated alerts for email delivery failures or provider issues

Risk Mitigation

1. Comprehensive Testing: Test all email flows before production deployment
2. Gradual Deployment: Deploy to staging environments first, then production
3. Monitoring: Real-time monitoring of email delivery rates and provider health
4. Emergency Procedures: Documented procedures for quick provider switching in emergencies

Future Enhancements

1. Additional Providers: SendGrid, Mailgun, Amazon SES
2. Email Templates: Enhanced template system with provider-specific optimizations
3. Webhooks: Implement delivery status tracking via provider webhooks
4. Email Analytics: Advanced email performance metrics
5. Queue System: Implement email queue for high-volume sending

Success Criteria

• Both SMTP and Resend providers successfully send emails
• Zero downtime migration from existing email service
• Improved email deliverability with Resend
• Comprehensive test coverage (>90%)
• Complete documentation and deployment guides
• Monitoring and alerting in place