284313f908
Some checks are pending
Bidi Control Character Guard / bidi-control-guard (push) Waiting to run
Circular Dependency Check / Check for new circular dependencies (push) Waiting to run
Citus Migration Smoke / Combined migrations on single-node Citus (push) Waiting to run
E2E Fresh Install Tests / fresh-install-e2e (push) Waiting to run
ext-v2 guardrails / Run ext-v2 guard and ESLint (push) Waiting to run
Integration Tests / Check for relevant changes (push) Waiting to run
Integration Tests / ${{ (github.event_name == 'schedule' || github.event.inputs.suite == 'full') && 'Full integration suite' || 'Tier-1 integration subset' }} (push) Blocked by required conditions
Mobile checks / Mobile lint + typecheck (push) Waiting to run
Mobile checks / Mobile unit tests (push) Waiting to run
Mobile checks / Mobile dependency audit (report) (push) Waiting to run
Mobile checks / Mobile reproducibility checks (push) Waiting to run
Secrets guard (env backups) / Ensure no tracked env backup files (push) Waiting to run
Temporal Readiness / fast-readiness (push) Waiting to run
Temporal Readiness / docker-parity (push) Waiting to run
TypeScript Type Check / Nx affected typecheck (push) Waiting to run
Unit Tests / Skipped-test budget (push) Waiting to run
Unit Tests / Nx affected unit tests (push) Waiting to run
Unit Tests / Server unit coverage (informational) (push) Waiting to run
Validate Tenant Management Schema / Check for relevant changes (push) Waiting to run
Validate Tenant Management Schema / Validate Tenant Management Schema (push) Blocked by required conditions
EE Workflows Build Guard / ee-workflows-build-guard (push) Waiting to run
Excluded: .git, node_modules, secrets/, compose.env, assemblyscript tgz Source: /opt/alga-psa on psa.joliet.tech
118 lines
11 KiB
Markdown
118 lines
11 KiB
Markdown
# Contract Template Hard Cutover Plan
|
||
|
||
## Context & Rationale
|
||
- The current schema stores template and instance records together (e.g., `contracts.is_template`, template rows inside `contract_lines`), which forces brittle conditionals throughout our codebase and prevents enforcing stricter rules on live contracts.
|
||
- Separating template data into purpose-built tables will clarify ownership, improve performance of contract-instance queries, and let us evolve template authoring without risking client contract integrity.
|
||
- A decisive cutover—preceded by rehearsal and feature-flagged code—lets us avoid months of dual-write complexity while still giving us checkpoints to compare legacy and new structures before flipping traffic.
|
||
|
||
## Phase Overview
|
||
1. **Preparation** — Requirements capture, dependency analysis, change-freeze planning.
|
||
2. **Schema Introduction** — Add the dedicated template tables and helper views without touching production data.
|
||
3. **Data Migration Dry Run** — Populate the new tables in staging, validate parity, and refine tooling.
|
||
4. **Application Readiness** — Update backend/frontend code to target the new schema behind a feature flag.
|
||
5. **Hard Cutover Window** — Execute the live migration, drop legacy columns, and enable the new code paths.
|
||
6. **Post-Cutover Validation** — Run regressions, monitor telemetry, and plan clean-up tasks.
|
||
|
||
## Detailed Steps
|
||
|
||
### Phase 1 — Preparation
|
||
- Audit all surfaces using template flags or metadata (`ContractTemplateDetail`, wizard flows, reporting exports, analytics events).
|
||
- Confirm that downstream consumers (BI pipelines, integrations) can tolerate the schema changes or have parallel updates planned.
|
||
- Draft communication for internal stakeholders and any customers affected by the maintenance window; align on roles (migration owner, QA lead, comms owner).
|
||
- Pick the maintenance window (target off-peak), document rollback strategy (snapshot + restore steps), and freeze competing schema work during that period.
|
||
|
||
#### Phase 1 Tracker (in progress)
|
||
- **Feature inventory (UI + services)**
|
||
- `server/src/components/billing-dashboard/contracts/ContractTemplateDetail.tsx` – parses `template_metadata`, drives inline edit + services editor, and depends on `getContractAssignments`.
|
||
- `server/src/components/billing-dashboard/contracts/Contracts.tsx:300` and `ContractDetailSwitcher.tsx:63` – filter/switch views with `contract.is_template`.
|
||
- `server/src/components/billing-dashboard/contracts/ContractHeader.tsx:106` – badge logic toggled by `contract.is_template`.
|
||
- Server actions: `contractActions.ts` (`getContracts`, `updateContract`, etc.), `contractWizardActions.ts:805-831`, `contractLineAction.ts`, and the new `contractLineRepository.ts` logic all branch on `is_template` or expect template rows inside shared tables.
|
||
- Billing utilities: `server/src/lib/billing/utils/templateClone.ts` reads from `contract_lines`/`contract_line_services` assuming template rows live alongside instance data.
|
||
- **Downstream / external consumers**
|
||
- Reporting joins: `server/src/lib/billing/billingEngine.ts:396` uses `coalesce(cc.template_contract_id, cc.contract_id)` to tie invoices back to templates.
|
||
- Data scripts: `server/scripts/contract-template-decoupling.ts` pre-populates `template_contract_id` & `template_contract_line_id`; must migrate to the new tables.
|
||
- API services: `server/src/lib/api/services/ContractLineService.ts:811-823` and related schemas reference template IDs for cloning and validation.
|
||
- Interfaces: `server/src/interfaces/contract.interfaces.ts` expose optional `is_template` and `template_metadata`, impacting consumers (e.g., analytics payloads) once removed.
|
||
- **Maintenance window & communication prep**
|
||
- Proposed duration: 60-minute off-peak window (needs coordination with Customer Success + Support).
|
||
- Owners: Migration (Billing Platform), QA (Web), Comms (Customer Ops) – confirm availability one week prior.
|
||
- Rollback assets: capture full database snapshot immediately before the copy; keep previous application build (feature flag off) scripted for quick redeploy.
|
||
- Draft announcements: internal Slack update, statuspage maintenance notice, customer email for managed tenants (templates authoring heavy users).
|
||
|
||
|
||
### Phase 2 — Schema Introduction
|
||
- Create new tables: `contract_templates`, `contract_template_lines`, `contract_template_line_services`, `contract_template_line_service_configuration`, `contract_template_line_defaults`, `contract_template_line_terms`, `contract_template_line_fixed_config`, `contract_template_pricing_schedules`, etc., matching the field-level plan.
|
||
- Add staging views that mimic the legacy combined layout to simplify parity queries (`legacy_contract_templates_view`, `legacy_template_lines_view`).
|
||
- Define foreign keys as `NOT VALID` so we can backfill before enforcing; include indexes mirroring existing query patterns (template ID, tenant, display order).
|
||
- Merge these migrations and confirm CI/preview environments apply them cleanly.
|
||
|
||
#### Phase 2 Tracker (in progress)
|
||
- ✅ Added migration `20250107162000_create_contract_template_tables.cjs` introducing dedicated template tables (templates, lines, mappings, services, configuration variants, defaults, terms, fixed config, pricing schedules) with `NOT VALID` foreign keys and supporting indexes.
|
||
- ✅ Created comparison views `contract_template_compare_view` and `contract_template_lines_compare_view` to contrast legacy rows with the new schema during validation.
|
||
- ✅ Validated the migration via `npm run migrate:ee`, confirming views build successfully after joining `contract_line_template_terms` for missing legacy columns.
|
||
- ✅ Added backfill migration `20250107164500_backfill_contract_template_tables.cjs` to copy existing template data into the new tables (idempotent inserts) so subsequent phases can rely solely on the separated schema.
|
||
- ✅ Updated server actions/models/UI to read/write via the new template tables (contract actions, wizard flows, line mappings, dashboards) eliminating dependency on `contracts.is_template` for template CRUD. (Follow-up: finish decoupling clone utilities from legacy tables.)
|
||
|
||
#### Phase 3 Tracker (in progress)
|
||
- ✅ Introduced `server/scripts/verify-template-migration.ts` (Knex-based parity checker) to compare legacy `contracts` template data against the new `contract_template_*` tables per tenant (counts + name diffs). Usage:
|
||
```bash
|
||
cd server
|
||
NODE_ENV=migration npx tsx ./scripts/verify-template-migration.ts
|
||
```
|
||
- ✅ Executed the validation script locally; parity check passed. Next run it against a staging snapshot before the hard cutover window.
|
||
|
||
### Phase 3 — Data Migration Dry Run
|
||
- Build idempotent scripts (Knex tasks or SQL plus node runner) that:
|
||
- Extract template rows from legacy tables based on `is_template`.
|
||
- Copy line/service/config rows into their new homes, preserving UUIDs where possible.
|
||
- Record mapping tables for contracts and lines (`legacy_contract_id → template_id`, etc.).
|
||
- Run the scripts against a staging database cloned from production. Compare:
|
||
- Row counts per table.
|
||
- Aggregations (lines per template, services per line, tag counts).
|
||
- Spot-check sample templates (metadata, guidance, pricing schedules).
|
||
- Iterate until the dry run produces zero discrepancies aside from intentional clean-up (e.g., orphaned services that should be dropped).
|
||
- Document runtime expectations and resource usage to size the production window.
|
||
|
||
### Phase 4 — Application Readiness
|
||
- Introduce new models/repos (`ContractTemplate`, `ContractTemplateLine`, etc.) and update server actions (`contractActions`, `contractLineAction`, repository helpers) to read/write via those models.
|
||
- Adjust TypeScript interfaces to split `IContractTemplate` from `IContract` (instances), updating UI components and hooks accordingly.
|
||
- Skip feature-flag gating (new template flow becomes the default) while ensuring fallbacks are removed.
|
||
- Update migrations that create sample data/fixtures, ensuring test suites build valid templates in the new schema.
|
||
- Expand automated tests: template CRUD, assignment dashboards, contract creation wizard, and reporting queries.
|
||
|
||
#### Phase 4 Tracker (in progress)
|
||
- ✅ Server actions now create and maintain template data directly in `contract_template_*` tables (template wizard, inline mapping, deletion) without legacy feature flags.
|
||
- ✅ `contractTemplate`/`contractTemplateLine` pathways deliver display order + metadata via new models (detail views and mapping lookups).
|
||
- ✅ Contract wizard snapshot/query paths read from `contract_template_*` tables.
|
||
- ✅ Added parity-check script to plan; new flows validated locally (phase 3).
|
||
- ✅ Template line add/remove UX now drives the updated actions (no feature flags) and client wizard cloning reads exclusively from `contract_template_*` data.
|
||
|
||
### Phase 5 — Hard Cutover Window
|
||
- Place the app into maintenance mode at window start; block writes via load balancer rules or database-level toggles.
|
||
- Take a full database snapshot/backup for rollback.
|
||
- Execute the migration scripts on production:
|
||
- Run the final data copy leveraging the validated scripts.
|
||
- Update foreign keys in client contract tables to reference the new template tables.
|
||
- Remove template rows from legacy tables and drop template-only columns/flags.
|
||
- `ALTER TABLE ... VALIDATE CONSTRAINT` to enforce referential integrity.
|
||
- Deploy the feature-flag-enabled application version (flag on) and run smoke tests while still in maintenance mode: template detail load, inline edit, contract creation wizard template selection, reporting queries.
|
||
- If tests pass, exit maintenance mode and re-enable user traffic.
|
||
|
||
### Phase 6 — Post-Cutover Validation
|
||
- Monitor logs, error dashboards, and database metrics for 24–48 hours with heightened alerting on contract and billing flows.
|
||
- Run targeted regression suites (Playwright, RTL, backend integration) against production or a production clone populated post-cutover.
|
||
- Communicate status updates to stakeholders, including any follow-on tasks discovered during validation.
|
||
- Schedule clean-up stories: drop temporary views/mapping tables, remove feature flag scaffolding, and update BI/integration documentation with the new schema diagrams.
|
||
|
||
## Risks & Mitigations
|
||
- **Migration Errors** — Mitigate with thorough dry runs, mapping-table audits, and a ready-to-execute rollback snapshot.
|
||
- **Extended Downtime** — Keep scripts optimized, rehearse runbooks, and assign owners for each step to minimize time in maintenance mode.
|
||
- **Blind Spots in Application Updates** — Use feature flags plus extended automated coverage to vet new code paths before the cutover; have rollback builds prepared.
|
||
- **Stakeholder Confusion** — Communicate timelines and status proactively; provide clear post-cutover guidance for support/ops teams reviewing contract data.
|
||
|
||
## Success Criteria
|
||
- Template CRUD and guidance management operate exclusively on the new tables with zero fallback to legacy columns.
|
||
- Client contract creation and synchronization correctly reference templates through the new foreign keys.
|
||
- Reporting, assignment summaries, and service management show no data regressions (verified via automated and manual checks).
|
||
- Legacy template columns (`contracts.is_template`, `contracts.template_metadata`, etc.) are fully removed, and documentation reflects the new architecture.
|