PSA/ee/docs/plans/2025-08-05-email-system-refactor.md

/# Email System Refactoring Plan

Intro / Rationale

Executive Summary

The current email system has duplicate implementations and lacks clear separation between system-level emails and tenant-specific business emails. This refactoring will establish a clear architectural pattern that distinguishes between:

• System Emails: Platform-level emails (user registration, password reset, system notifications) using environment variables
• Tenant Emails: Business-specific emails (portal invitations, invoices, project notifications) using database-configured settings

Business and Technical Drivers

• Code Duplication: Two separate EmailService implementations exist with overlapping functionality
• Architectural Confusion: Email verification incorrectly uses TenantEmailService when it should use system settings
• Maintenance Overhead: Scattered email logic makes updates and bug fixes difficult
• Scalability Issues: No clear pattern for adding new email types

Success Criteria

• Single, clear EmailService implementation for system emails using environment variables
• Properly separated TenantEmailService for business emails using database settings
• Email verification and password reset using SystemEmailService
• Portal invitations and business emails continue using TenantEmailService
• Clear documentation and examples for future development

Key Stakeholders

• Development team implementing new email features
• Operations team managing email configuration
• End users receiving system and business emails

Phased Implementation Checklist

Phase 1: System Email Service Consolidation

Goal: Create unified SystemEmailService for all platform-level emails

• Task 1.1: Create new directory structure
  • Create /src/lib/email/system/ directory
  • Create /src/lib/email/tenant/ directory
• Task 1.2: Consolidate duplicate EmailService implementations
  • Analyze differences between /src/services/emailService.ts and /src/lib/notifications/emailService.ts
  • Create unified SystemEmailService.ts in /src/lib/email/system/
  • Implement environment variable configuration exclusively
  • Add comprehensive logging and error handling
• Task 1.3: Create system email templates
  • Move system templates to /src/lib/email/system/templates/
  • Create emailVerification.ts template
  • Create passwordReset.ts template
  • Create systemNotification.ts template
• Task 1.4: Create system email types
  • Create /src/lib/email/system/types.ts with SystemEmailOptions interface
  • Define SystemEmailConfig interface
  • Export template interfaces

Dependencies: None
Completion Criteria: SystemEmailService can send emails using environment variables, all system templates are available

Phase 2: Tenant Email Service Migration

Goal: Move and enhance existing TenantEmailService to new structure

• Task 2.1: Move TenantEmailService to new location
  • Move TenantEmailService.ts to /src/lib/email/tenant/
  • Move templateProcessors.ts to /src/lib/email/tenant/
  • Update all import paths referencing moved files
• Task 2.2: Enhance TenantEmailService documentation
  • Add comprehensive JSDoc comments explaining tenant email purpose
  • Document when to use TenantEmailService vs SystemEmailService
  • Add usage examples in code comments
• Task 2.3: Create tenant email types
  • Create /src/lib/email/tenant/types.ts
  • Move tenant-specific interfaces from existing files
  • Ensure clear separation from system email types

Dependencies: Phase 1 completion
Completion Criteria: TenantEmailService is in new location with enhanced documentation, all imports updated

Phase 3: Email Function Updates

Goal: Update specific email functions to use correct service

• Task 3.1: Update email verification to use SystemEmailService
  • Modify /src/lib/email/sendVerificationEmail.ts to import SystemEmailService
  • Replace TenantEmailService.sendEmail() call with SystemEmailService
  • Update template processor to use system templates
  • Test email verification flow works with environment variables
• Task 3.2: Ensure password reset uses SystemEmailService
  • Locate password reset email functions
  • Update to use SystemEmailService instead of any tenant service calls
  • Verify password reset templates are in system templates directory
• Task 3.3: Verify portal invitations continue using TenantEmailService
  • Confirm /src/lib/email/sendPortalInvitationEmail.ts uses TenantEmailService
  • Update import paths to new tenant service location
  • Test portal invitation flow still works correctly

Dependencies: Phase 2 completion
Completion Criteria: Email verification and password reset use SystemEmailService, portal invitations use TenantEmailService

Phase 4: Email Service Factory and Integration

Goal: Create centralized factory and update integration points

• Task 4.1: Create email service factory
  • Create /src/lib/email/index.ts as main entry point
  • Export getSystemEmailService() function
  • Export getTenantEmailService() function
  • Re-export key types and interfaces
• Task 4.2: Update application integration points
  • Update all files importing old EmailService locations
  • Replace direct service instantiation with factory functions
  • Update API routes and actions to use correct email service
• Task 4.3: Clean up old implementations
  • Remove /src/services/emailService.ts after confirming no references
  • Remove /src/lib/notifications/emailService.ts after confirming no references
  • Remove /src/utils/email/emailService.tsx if unused

Dependencies: Phase 3 completion
Completion Criteria: Single entry point for email services, old implementations removed, all imports updated

Phase 5: Documentation and Guidelines

Goal: Create comprehensive documentation for email system usage

• Task 5.1: Create main email documentation
  • Create /src/lib/email/README.md with comprehensive guidelines
  • Document when to use SystemEmailService vs TenantEmailService
  • Provide code examples for common email scenarios

Dependencies: Phase 4 completion
Completion Criteria: Complete documentation available, clear guidelines for future development

Background Details / Investigation / Implementation Advice

Current System Analysis

Duplicate EmailService Implementations

Two separate EmailService classes exist:

1. /src/services/emailService.ts:

   • Uses both environment variables and database templates
   • Includes invoice email functionality
   • Has comprehensive logging and error handling
   • Contains template fallback logic

2. /src/lib/notifications/emailService.ts:

   • Uses environment variables exclusively
   • Includes Handlebars template compilation
   • Simpler implementation focused on notifications

Key Issues Identified

• Email verification uses TenantEmailService but should use system settings
• Template storage is inconsistent (hardcoded vs database)
• No clear naming convention to distinguish email types
• Import paths are scattered across different directories

Technical Implementation Guidelines

SystemEmailService Requirements

• Configuration: Environment variables only (EMAIL_HOST, EMAIL_PORT, EMAIL_USERNAME, EMAIL_PASSWORD, EMAIL_FROM)
• Templates: Static templates in code or system template table
• Use Cases: User registration, password reset, system alerts, platform notifications
• Initialization: Singleton pattern with lazy initialization
• Error Handling: Comprehensive logging with diagnostic information

TenantEmailService Requirements

• Configuration: Database-stored tenant email settings
• Templates: Database templates with tenant-specific customization
• Use Cases: Portal invitations, invoices, project notifications, business communications
• Provider Support: Multiple email provider configurations per tenant
• Template Processing: Support for complex template data and variables

Directory Structure Implementation

/src/lib/email/
├── index.ts                    # Main factory and exports
├── README.md                   # Comprehensive documentation
├── system/
│   ├── SystemEmailService.ts  # Environment-based email service
│   ├── templates/
│   │   ├── emailVerification.ts
│   │   ├── passwordReset.ts
│   │   └── systemNotification.ts
│   └── types.ts               # System email interfaces
└── tenant/
    ├── TenantEmailService.ts  # Database-based email service
    ├── templateProcessors.ts  # Template processing logic
    └── types.ts               # Tenant email interfaces

Migration Strategy

1. Preserve Functionality: Ensure zero downtime during migration
2. Gradual Transition: Update services one at a time with testing
3. Import Path Updates: Use find-replace to update import statements systematically
4. Template Migration: Move templates without changing their content initially

Testing Approach

• Test system emails with environment variable configuration
• Test tenant emails with database configuration
• Verify email verification flow uses system service
• Verify portal invitations use tenant service
• Test error handling for missing configuration

Potential Pitfalls and Solutions

Pitfall: Breaking existing email functionality during migration
Solution: Update one service at a time, maintain parallel implementations during transition

Pitfall: Template references breaking after file moves
Solution: Update all import paths in single atomic operation, use IDE refactoring tools

Pitfall: Configuration conflicts between system and tenant emails
Solution: Clearly separate configuration sources, add validation for each service type

Pitfall: Missing error handling in new implementations
Solution: Copy comprehensive error handling from existing services, add diagnostic logging

Code Examples

SystemEmailService Usage

import { getSystemEmailService } from '@/lib/email';

const emailService = await getSystemEmailService();
await emailService.sendVerificationEmail({
  to: 'user@example.com',
  templateData: { verificationUrl: 'https://...' }
});

TenantEmailService Usage

import { getTenantEmailService } from '@/lib/email';

const result = await TenantEmailService.sendEmail({
  tenantId: 'tenant-123',
  to: 'client@company.com',
  templateProcessor: new DatabaseTemplateProcessor(knex, 'portal-invitation'),
  templateData: { portalLink: 'https://...' }
});

Resources and References

• Current TenantEmailService implementation: /src/lib/services/TenantEmailService.ts
• Email provider management: /src/services/email/EmailProviderManager.ts
• Template processing: /src/lib/services/email/templateProcessors.ts
• Email types: /src/types/email.types.ts

Implementer's Scratch Pad

Progress Tracking

• Phase 1 started: ___________
• Phase 1 completed: ___________
• Phase 2 started: ___________
• Phase 2 completed: ___________
• Phase 3 started: ___________
• Phase 3 completed: ___________
• Phase 4 started: ___________
• Phase 4 completed: ___________
• Phase 5 started: ___________
• Phase 5 completed: ___________

Implementation Notes

Phase 1 Notes:
- SystemEmailService consolidation status: 
- Template migration status:
- Environment variable testing:

Phase 2 Notes:
- TenantEmailService move status:
- Import path updates:
- Documentation additions:

Phase 3 Notes:
- Email verification update status:
- Password reset verification:
- Portal invitation testing:

Phase 4 Notes:
- Factory implementation:
- Integration point updates:
- Cleanup completion:

Phase 5 Notes:
- Documentation completeness:
- Example code validation:
- Final review status:

Issues Encountered and Resolutions

Issue 1: [Description]
Resolution: [How resolved]
Impact: [Impact on timeline/scope]

Issue 2: [Description]  
Resolution: [How resolved]
Impact: [Impact on timeline/scope]

Deviations From Original Plan

Deviation 1: [What changed]
Reason: [Why it changed]
Approval: [Who approved change]

Deviation 2: [What changed]
Reason: [Why it changed]  
Approval: [Who approved change]

Performance Metrics and Test Results

Email Verification Tests:
- System service configuration: [PASS/FAIL]
- Template rendering: [PASS/FAIL]
- Email delivery: [PASS/FAIL]

Portal Invitation Tests:
- Tenant service configuration: [PASS/FAIL]
- Database template retrieval: [PASS/FAIL]
- Email delivery: [PASS/FAIL]

Integration Tests:
- Import path updates: [PASS/FAIL]
- Service factory: [PASS/FAIL]
- Error handling: [PASS/FAIL]

Questions for Review

1. [Question about implementation approach]
2. [Question about configuration]
3. [Question about testing strategy]