adobe-to-docusign-migrator/README.md

# Adobe Sign → DocuSign Migrator

A Python toolkit + web UI for migrating library templates from Adobe Sign (Acrobat Sign) to DocuSign.
It downloads templates via the Adobe Sign API, converts them to DocuSign format, and uploads them via the DocuSign API — either from the command line or through a browser-based UI.

---

## What it does

1. **Authenticates** with Adobe Sign via OAuth (one-time browser flow, tokens saved to `.env`)
2. **Downloads** templates — PDF, metadata, and form field definitions
3. **Converts** each template to a DocuSign `envelopeTemplate` JSON, mapping all field types, coordinates, recipient roles, and conditional field logic
4. **Authenticates** with DocuSign via JWT grant (one-time browser consent, then fully automated)
5. **Uploads** the converted template to DocuSign via the REST API

---

## Prerequisites

- Python 3.10+
- An Adobe Sign OAuth app (EU2 shard) with scopes: `library_read:self library_write:self user_read:self`
- A DocuSign developer account with an integration key and RSA keypair

---

## Setup

**1. Install Python dependencies:**
```bash
pip install -r requirements.txt
```

**2. Create a `.env` file** in the project root (never commit this):
```bash
cp .env-sample .env
```
Then fill in your credentials. See [.env-sample](.env-sample) for the full list of
variables with descriptions. Use `account-d.docusign.com` and
`https://demo.docusign.net/restapi` for sandbox; for production replace with
`account.docusign.com` and your account's base URL (e.g. `https://na3.docusign.net/restapi`).

**3. Authenticate with Adobe Sign** (one-time):
```bash
python3 src/adobe_auth.py
```
Opens a browser. After authorizing, paste the redirect URL back into the terminal.
Tokens are saved to `.env` and auto-refreshed on subsequent runs.

**4. Grant consent for DocuSign** (one-time per user):
```bash
python3 src/docusign_auth.py --consent
```
Opens a browser for the DocuSign OAuth consent screen. After approving, paste the
redirect URL back into the terminal. This grants the `impersonation` scope required
for JWT grant. After this runs once, all subsequent API calls use JWT automatically —
no further browser interaction needed.

---

## Running a migration

**List available templates in Adobe Sign:**
```bash
python3 src/download_templates.py list
```

**Download templates:**
```bash
python3 src/download_templates.py download                  # all templates
python3 src/download_templates.py download "Template Name" # one specific template
```
Downloads to `downloads/<template-name>__<id>/` — one folder per template containing
`metadata.json`, `form_fields.json`, `documents.json`, and the PDF.

**Convert a downloaded template to DocuSign format:**
```bash
python3 src/compose_docusign_template.py
```
Writes DocuSign template JSONs to `migration-output/<template-name>/docusign-template.json`.

**Upload to DocuSign (upsert by default):**
```bash
python3 src/upload_docusign_template.py --file migration-output/<name>/docusign-template.json
```
If a DocuSign template with the same name already exists, the most recently modified one is updated (PUT).
To always create a new template instead:
```bash
python3 src/upload_docusign_template.py --file <path> --force-create
```

**Or run the full pipeline end-to-end:**
```bash
python3 src/migrate_template.py --list                      # show available templates
python3 src/migrate_template.py --template "Template Name"  # download → convert → upload
python3 src/migrate_template.py --template "Template Name" --skip-upload  # convert only
```
If multiple templates share the same name, the most recently modified one is used.

---

---

## Web UI

The web UI is an enterprise-grade migration console with a Docusign-branded left-nav
shell, multi-customer project context, and a full migration workflow.

**Additional `.env` keys required for the web UI:**
```
SESSION_SECRET_KEY=<any random string>
DOCUSIGN_CLIENT_SECRET=<your DocuSign app client secret>
DOCUSIGN_REDIRECT_URI=http://localhost:8000/api/auth/docusign/callback
ADOBE_REDIRECT_URI=http://localhost:8000/api/auth/adobe/callback
```

**Start the server:**
```bash
uvicorn web.app:app --reload --port 8000
```
Then open [http://localhost:8000](http://localhost:8000) in your browser.

### Navigation

| Screen | Path | Purpose |
|---|---|---|
| Templates | `#/templates` | Filterable table with readiness badges; bulk migration |
| Migration Results | `#/results` | Summary + per-template results from last migration |
| Issues & Warnings | `#/issues` | All templates with blockers or warnings |
| Verification | `#/verify` | Send test envelopes; confirm templates work end-to-end |
| History & Audit | `#/history` | Full migration history, filters, CSV export |
| Settings | `#/settings` | Verification defaults, migration defaults, connection info |

### Workflow

1. **Create a project** — the switcher modal opens on first run; name it after the customer.
2. **Connect platforms** — click the Adobe Sign and DocuSign chips in the top bar.
3. **Review templates** — the Templates view shows readiness badges:
   - **Ready** (green) — no issues, safe to migrate
   - **Caveats** (amber) — warnings exist; migration will proceed but check Issues view
   - **Blocked** (red) — blockers found; migration will fail until resolved
   - **Migrated** (cobalt) — successfully migrated and up to date
   - **Needs Update** (amber) — Adobe template modified after last migration
4. **Resolve issues** — check Issues & Warnings before migrating blocked templates.
5. **Migrate** — select templates, click Migrate Selected, configure options (dry run, overwrite, target folder), monitor progress.
6. **Verify** — on the Verification screen, send test envelopes and confirm receipt.
7. **Audit** — History & Audit logs every migration with checksums and export.

### Project / customer context

The project switcher (nav footer) stores per-customer migration context in `localStorage`.
Create one project per customer to keep history and settings separate.

**API docs:** [http://localhost:8000/api/docs](http://localhost:8000/api/docs)

---

## Running tests

```bash
pytest tests/ -v                           # full suite (118 tests)
pytest tests/test_regression.py -v         # compose regression only
pytest tests/test_regression.py --update-snapshots  # regenerate snapshots after intentional changes
```

---

## Field type mapping

See [field-mapping.md](field-mapping.md) for the full Adobe Sign → DocuSign tab type table,
conditional logic mapping, and known gaps.

## Known API quirks and bugs

See [tests/PLATFORM-QUIRKS.md](tests/PLATFORM-QUIRKS.md) for documented platform bugs,
unexpected API behaviors, and the fixes applied.

---

## Migration API options

`POST /api/migrate` accepts extended options (blueprint-aligned):

```json
{
  "source_template_ids": ["tpl_001", "tpl_002"],
  "target_folder": "Migrated Templates",
  "options": {
    "dry_run": false,
    "overwrite_if_exists": false,
    "include_documents": true
  }
}
```

| Option | Default | Description |
|---|---|---|
| `dry_run` | `false` | Validate and compose without creating DocuSign templates |
| `overwrite_if_exists` | `false` | If `false`, skip templates that already exist in DocuSign |
| `include_documents` | `true` | Embed PDFs in the DocuSign template |

**Batch migration** (`POST /api/migrate/batch`) runs the same pipeline for multiple templates as a background job:

```bash
# Start batch
curl -X POST /api/migrate/batch -d '{"source_template_ids": ["id1", "id2"]}'
# → {"job_id": "abc-123", "status": "queued"}

# Poll status
curl /api/migrate/batch/abc-123
# → {"status": "running", "progress": {"completed": 1, "total": 2}, ...}
```

---

## Normalized intermediate schema

The migration pipeline uses a platform-agnostic `NormalizedTemplate` model as a bridge between Adobe Sign and DocuSign. This decouples extraction from composition and enables the validation layer.

See `src/models/normalized_template.py` and `src/services/mapping_service.py`.

---

## Validation

Each template is validated before migration:
- **Blockers** (halt migration): no recipients, no documents
- **Warnings** (logged but continue): no signature fields, unassigned fields, unsupported features

Unsupported features flagged for manual review: conditional HIDE actions, JavaScript validators, calculated fields, webhook associations, niche authentication methods.

---

## Security

- `src/utils/log_sanitizer.py` — install `SanitizingFilter` to redact tokens, keys, and base64 PDF content from all log output
- PDF checksums (SHA-256) are computed and stored with each migration record
- Tokens are never written to logs; see `src/utils/log_sanitizer.py`

---

## Project structure

```
src/
  models/
    normalized_template.py       # Platform-agnostic intermediate schema
  services/
    mapping_service.py           # Adobe Sign → NormalizedTemplate converter
    validation_service.py        # Pre/post migration checks (blockers + warnings)
  reports/
    report_builder.py            # Structured migration report per template
  utils/
    retry.py                     # Exponential backoff retry helpers
    log_sanitizer.py             # Secret redaction from logs
  adobe_auth.py                  # One-time OAuth flow for Adobe Sign (CLI)
  adobe_api.py                   # Adobe Sign API client (auto token refresh)
  download_templates.py          # List and download templates from Adobe Sign
  compose_docusign_template.py   # Core conversion: Adobe Sign → DocuSign JSON
  docusign_auth.py               # DocuSign JWT auth + one-time consent flow
  upload_docusign_template.py    # Upsert upload: PUT if exists, POST if not
  migrate_template.py            # End-to-end CLI runner (download → convert → upload)

web/
  app.py                         # FastAPI entrypoint (uvicorn web.app:app)
  config.py                      # Environment-based settings
  session.py                     # Signed cookie session helpers
  routers/
    auth.py                      # Adobe Sign + DocuSign OAuth endpoints
    templates.py                 # Template listing + migration status API
    migrate.py                   # Migration trigger, batch, + history API
  static/
    index.html                   # Web UI (side-by-side browser + migrate flow)
    app.js                       # Vanilla JS frontend
    style.css                    # Styles + status badge colours

tests/
  test_normalized_schema.py      # Normalized model + mapping service tests
  test_validation_service.py     # Validation service + report builder tests
  test_migration_options.py      # dryRun, overwriteIfExists, includeDocuments
  test_batch_migration.py        # Batch migration API tests
  test_retry.py                  # Retry with backoff utility tests
  test_security.py               # Log sanitization + PDF checksum tests
  test_upload_upsert.py          # Upsert logic unit tests
  test_api_health.py             # Health endpoint
  test_api_auth.py               # OAuth endpoint tests
  test_api_templates.py          # Template listing + status tests
  test_api_migrate.py            # Migration API tests
  test_e2e.py                    # Full pipeline end-to-end test
  test_regression.py             # Compose output vs snapshots
  fixtures/expected/             # Regression snapshots (3 real templates)

downloads/                       # Downloaded Adobe Sign templates (gitignored)
migration-output/                # Converted DocuSign template JSONs + history
field-mapping.md                 # Field type mapping table + edge case log
PRODUCT-SPEC.md                  # Full product specification (blueprint-aligned)
docs/agent-harness/
  EXECUTION-BOARD.md             # Living kanban board
requirements.txt                 # Python dependencies
```