# Adobe Sign → DocuSign Migrator

A Python toolkit + web UI for migrating library templates from Adobe Sign (Acrobat Sign) to DocuSign.
It downloads templates via the Adobe Sign API, converts them to DocuSign format, and uploads them via the DocuSign API — either from the command line or through a browser-based UI.

---

## What it does

1. **Authenticates** with Adobe Sign via OAuth (one-time browser flow, tokens saved to `.env`)
2. **Downloads** templates — PDF, metadata, and form field definitions
3. **Converts** each template to a DocuSign `envelopeTemplate` JSON, mapping all field types, coordinates, recipient roles, and conditional field logic
4. **Authenticates** with DocuSign via OAuth Authorization Code Grant (one-time browser login, then refresh-token based)
5. **Uploads** the converted template to DocuSign via the REST API

---

## Prerequisites

- Python 3.10+
- An Adobe Sign OAuth app (EU2 shard) with scopes: `library_read:self library_write:self user_read:self`
- A DocuSign developer account with an OAuth app client ID and client secret

---

## Setup

**1. Install Python dependencies:**
```bash
pip install -r requirements.txt
```

**2. Create a `.env` file** in the project root (never commit this):
```bash
cp .env-sample .env
```
Then fill in your credentials. See [.env-sample](.env-sample) for the full list of
variables with descriptions. Use `account-d.docusign.com` and
`https://demo.docusign.net/restapi` for sandbox; for production replace with
`account.docusign.com` and your account's base URL (e.g. `https://na3.docusign.net/restapi`).

**3. Authenticate with Adobe Sign** (one-time for CLI use):
```bash
python3 src/adobe_auth.py
```
Opens a browser. After authorizing, paste the redirect URL back into the terminal.
Tokens are saved to `.env` and auto-refreshed on subsequent runs.

**4. Authorize DocuSign** (CLI, one-time per machine/user):
```bash
python3 src/docusign_auth.py --authorize
```
Opens a browser for the DocuSign OAuth screen. After approving, paste the
redirect URL back into the terminal. The app stores both access and refresh
tokens in `.env`, and later API calls refresh access tokens automatically.

---

## Running a migration

**List available templates in Adobe Sign:**
```bash
python3 src/download_templates.py list
```

**Download templates:**
```bash
python3 src/download_templates.py download                  # all templates
python3 src/download_templates.py download "Template Name" # one specific template
```
Downloads to `downloads/<template-name>__<id>/` — one folder per template containing
`metadata.json`, `form_fields.json`, `documents.json`, and the PDF.

**Convert a downloaded template to DocuSign format:**
```bash
python3 src/compose_docusign_template.py
```
Writes DocuSign template JSONs to `migration-output/<template-name>/docusign-template.json`.

**Upload to DocuSign (upsert by default):**
```bash
python3 src/upload_docusign_template.py --file migration-output/<name>/docusign-template.json
```
If a DocuSign template with the same name already exists, the most recently modified one is updated (PUT).
To always create a new template instead:
```bash
python3 src/upload_docusign_template.py --file <path> --force-create
```

**Or run the full pipeline end-to-end:**
```bash
python3 src/migrate_template.py --list                      # show available templates
python3 src/migrate_template.py --template "Template Name"  # download → convert → upload
python3 src/migrate_template.py --template "Template Name" --skip-upload  # convert only
```
If multiple templates share the same name, the most recently modified one is used.

---

---

## Web UI

The web UI is an enterprise-grade migration console with a Docusign-branded left-nav
shell, multi-customer project context, and a full migration workflow.

**Additional `.env` keys required for the web UI:**
```
SESSION_SECRET_KEY=<any random string>
SESSION_STORE_DIR=/absolute/path/for/browser-session-files
DOCUSIGN_CLIENT_SECRET=<your DocuSign app client secret>
DOCUSIGN_REDIRECT_URI=http://localhost:8000/api/auth/docusign/callback
ADOBE_REDIRECT_URI=http://localhost:8000/api/auth/adobe/callback
```

**Start the server:**
```bash
uvicorn web.app:app --reload --port 8000
```
Then open [http://localhost:8000](http://localhost:8000) in your browser.

### Multi-user testing

The web UI now supports concurrent testers on one shared deployment:

- each browser gets its own server-side session file
- DocuSign web OAuth is isolated per tester session
- migration history and batch-job polling are scoped to that tester session
- Adobe Sign web auth now supports the same redirect-based browser callback pattern as DocuSign
- Adobe Sign can still be connected from shared `.env` credentials if you use the top-bar Adobe connect flow

Important behavior:

- the CLI still stores DocuSign tokens in `.env`
- the web UI does **not** reuse `.env` DocuSign refresh tokens for all users anymore
- each tester who needs DocuSign upload/verification should connect DocuSign in their own browser session
- if a DocuSign user belongs to multiple accounts, the web UI fetches the full account list from `/oauth/userinfo`, sorts it alphabetically, and requires the user to choose an account for the session
- browser-session files live under `.session-store/` by default and can be deleted to force reconnects

### Navigation

| Screen | Path | Purpose |
|---|---|---|
| Templates | `#/templates` | Filterable table with readiness badges; bulk migration |
| Migration Results | `#/results` | Summary + per-template results from last migration |
| Issues & Warnings | `#/issues` | All templates with blockers or warnings |
| Verification | `#/verify` | Send test envelopes; confirm templates work end-to-end |
| History & Audit | `#/history` | Full migration history, filters, CSV export |
| Settings | `#/settings` | Verification defaults, migration defaults, connection info |

### Workflow

1. **Create a project** — the switcher modal opens on first run; name it after the customer.
2. **Connect platforms** — click the Adobe Sign and Docusign chips in the top bar.
   - For group testing, each tester should connect Docusign in their own browser.
   - Adobe Sign also supports a normal browser redirect callback when shared `.env` credentials are not being used.
   - If your DocuSign login belongs to multiple accounts, the app will prompt you to choose one account for this session.
   - Settings now shows the browser session ID, auth mode, and selected DocuSign account for easier troubleshooting.
3. **Review templates** — the Templates view shows readiness badges:
   - **Ready** (green) — no issues, safe to migrate
   - **Caveats** (amber) — warnings exist; migration will proceed but check Issues view
   - **Blocked** (red) — blockers found; migration will fail until resolved
   - **Migrated** (cobalt) — successfully migrated and up to date
   - **Needs Update** (amber) — Adobe template modified after last migration
4. **Resolve issues** — check Issues & Warnings before migrating blocked templates.
5. **Migrate** — select templates, click Migrate Selected, configure options (dry run, overwrite, target folder), monitor progress. Failed rows show the error inline; a summary hint appears if any templates fail.
6. **Review field issues** — successfully migrated templates may show an amber **partial** badge if features were dropped during migration (e.g. cross-recipient conditionals, unsupported operators). Expand any result row to see grouped field-issue details.
7. **Verify** — on the Verification screen, send test envelopes to confirm templates work end-to-end. Polling checks every 30 seconds and times out after 5 minutes; production deployments should use Docusign Connect (webhooks) instead.
8. **Audit** — History & Audit logs every migration with checksums and export. Rows with field issues show the same grouped breakdown on expand.

### Project / customer context

The project switcher (nav footer) stores per-customer migration context in `localStorage`.
Create one project per customer to keep history and settings separate.

**API docs:** [http://localhost:8000/api/docs](http://localhost:8000/api/docs)

---

## Production deployment

The web UI is designed for local or private-network use during a migration engagement. If you do expose it more broadly, follow these steps:

### Run behind a reverse proxy (HTTPS required for OAuth)

OAuth callbacks from both Adobe Sign and DocuSign require HTTPS. Use nginx, Caddy, or a cloud load balancer to terminate TLS and proxy to uvicorn:

```
# nginx example
location / {
    proxy_pass http://127.0.0.1:8000;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Proto https;
}
```

Start uvicorn without `--reload` in production:
```bash
uvicorn web.app:app --host 127.0.0.1 --port 8000 --workers 1
```

> Use `--workers 1` — batch job state is in-memory and not safe to share across workers.

### Required environment variables

| Variable | Description |
|----------|-------------|
| `SESSION_SECRET_KEY` | Random secret for signing session cookies. Generate one with `python3 -c "import secrets; print(secrets.token_hex(32))"` |
| `SESSION_STORE_DIR` | Absolute path for server-side session files (default: `.session-store/` in project root) |
| `AUDIT_LOG_FILE` | Absolute path for the JSONL audit log (default: `.audit-log.jsonl` in project root) |
| `ADOBE_REDIRECT_URI` | Must match the callback URL registered in your Adobe Sign app (e.g. `https://migrator.example.com/api/auth/adobe/callback`) |
| `DOCUSIGN_REDIRECT_URI` | Must match the callback URL registered in your DocuSign app (e.g. `https://migrator.example.com/api/auth/docusign/callback`) |

### Rotating SESSION_SECRET_KEY

Changing `SESSION_SECRET_KEY` invalidates all existing browser sessions — every user will be logged out and must reconnect their Adobe Sign and DocuSign accounts. There is no migration path for existing session files. To rotate:

1. Update `SESSION_SECRET_KEY` in `.env`
2. Delete all files in `SESSION_STORE_DIR`
3. Restart the server

### Shard configuration

By default the app targets the Adobe Sign **EU2** shard. To target a different shard, set `ADOBE_SIGN_BASE_URL` in `.env`:

```
# NA1 shard
ADOBE_SIGN_BASE_URL=https://api.na1.adobesign.com/api/rest/v6

# EU2 shard (default)
ADOBE_SIGN_BASE_URL=https://api.eu2.adobesign.com/api/rest/v6
```

Also update `ADOBE_REDIRECT_URI` and the OAuth app registration to match your shard's auth server if it differs.

For DocuSign, switch from sandbox to production by updating:
```
DOCUSIGN_AUTH_SERVER=account.docusign.com
DOCUSIGN_BASE_URL=https://na3.docusign.net/restapi  # your account's base URL
```

### Session store maintenance

Session files accumulate in `SESSION_STORE_DIR` — one file per browser session. Delete stale files periodically:

```bash
# Delete session files older than 7 days
find .session-store/ -name "*.json" -mtime +7 -delete
```

---

## Running tests

```bash
pytest tests/ -v                           # full suite (119 tests)
pytest tests/test_regression.py -v         # compose regression only
pytest tests/test_regression.py --update-snapshots  # regenerate snapshots after intentional changes
```

---

## Field type mapping

See [field-mapping.md](field-mapping.md) for the full Adobe Sign → DocuSign tab type table,
conditional logic mapping, and known gaps.

## Known API quirks and bugs

See [tests/PLATFORM-QUIRKS.md](tests/PLATFORM-QUIRKS.md) for documented platform bugs,
unexpected API behaviors, and the fixes applied.

---

## Migration API options

`POST /api/migrate` accepts extended options (blueprint-aligned):

```json
{
  "source_template_ids": ["tpl_001", "tpl_002"],
  "target_folder": "Migrated Templates",
  "options": {
    "dry_run": false,
    "overwrite_if_exists": false,
    "include_documents": true
  }
}
```

| Option | Default | Description |
|---|---|---|
| `dry_run` | `false` | Validate and compose without creating DocuSign templates |
| `overwrite_if_exists` | `false` | If `false`, skip templates that already exist in DocuSign |
| `include_documents` | `true` | Embed PDFs in the DocuSign template |

**Batch migration** (`POST /api/migrate/batch`) runs the same pipeline for multiple templates as a background job:

```bash
# Start batch
curl -X POST /api/migrate/batch -d '{"source_template_ids": ["id1", "id2"]}'
# → {"job_id": "abc-123", "status": "queued"}

# Poll status
curl /api/migrate/batch/abc-123
# → {"status": "running", "progress": {"completed": 1, "total": 2}, ...}
```

---

## Normalized intermediate schema

The migration pipeline uses a platform-agnostic `NormalizedTemplate` model as a bridge between Adobe Sign and DocuSign. This decouples extraction from composition and enables the validation layer.

See `src/models/normalized_template.py` and `src/services/mapping_service.py`.

---

## Validation

Each template is validated before migration:
- **Blockers** (halt migration): no recipients, no documents
- **Warnings** (logged but continue): no signature fields, unassigned fields, unsupported features

Unsupported features flagged for manual review: conditional HIDE actions, JavaScript validators, calculated fields, webhook associations, niche authentication methods.

## Field issues (partial migration)

Beyond blockers and warnings, the compose step emits structured **field issues** when a field migrates successfully but something was silently dropped or approximated. Each issue has a machine-readable code:

| Code | Meaning |
|---|---|
| `CROSS_RECIPIENT_CONDITIONAL` | Show/hide condition references a field on a different recipient — DocuSign does not support cross-recipient conditional logic |
| `UNSUPPORTED_OPERATOR` | Condition uses NOT_EQUALS / GT / LT etc. — only EQUALS is supported |
| `HIDE_ACTION` | Adobe HIDE condition has no DocuSign equivalent — field will always be visible |
| `MULTI_PREDICATE` | AND/OR multi-condition logic reduced to first EQUALS predicate only |
| `INVALID_PARENT_TAB` | Conditional parent references a non-existent tab or a forbidden tab type (signature/auto-fill tabs cannot be parents) |
| `FIELD_TYPE_SKIPPED` | INLINE_IMAGE or PARTICIPATION_STAMP — no DocuSign equivalent, field dropped |
| `PARTIAL_FIELD_TYPE` | FILE_CHOOSER → signerAttachmentTabs, or STAMP → stampTabs — field migrated but behaviour differs |

Templates with field issues are marked with a **partial** badge in the UI. Field issues are stored in `field_issues[]` on every migration result record and displayed grouped by code in all result views.

---

## Security

- `src/utils/log_sanitizer.py` — install `SanitizingFilter` to redact tokens, keys, and base64 PDF content from all log output
- PDF checksums (SHA-256) are computed and stored with each migration record
- Tokens are never written to logs; see `src/utils/log_sanitizer.py`

---

## Project structure

```
src/
  models/
    normalized_template.py       # Platform-agnostic intermediate schema
    field_issue.py               # Structured field-issue model + issue codes
  services/
    mapping_service.py           # Adobe Sign → NormalizedTemplate converter
    validation_service.py        # Pre/post migration checks (blockers + warnings)
  reports/
    report_builder.py            # Structured migration report per template
  utils/
    retry.py                     # Exponential backoff retry helpers
    log_sanitizer.py             # Secret redaction from logs
  adobe_auth.py                  # One-time OAuth flow for Adobe Sign (CLI)
  adobe_api.py                   # Adobe Sign API client (auto token refresh)
  download_templates.py          # List and download templates from Adobe Sign
  compose_docusign_template.py   # Core conversion: Adobe Sign → DocuSign JSON
  docusign_auth.py               # DocuSign auth-code + refresh-token helper
  upload_docusign_template.py    # Upsert upload: PUT if exists, POST if not
  migrate_template.py            # End-to-end CLI runner (download → convert → upload)

web/
  app.py                         # FastAPI entrypoint (uvicorn web.app:app)
  config.py                      # Environment-based settings
  session.py                     # Signed cookie session helpers
  routers/
    auth.py                      # Adobe Sign + Docusign OAuth endpoints
    templates.py                 # Template listing + migration status API
    migrate.py                   # Migration trigger, batch, + history API
    verify.py                    # Verification envelope send / status / void
  static/
    index.html                   # SPA shell (left nav, router outlet, top bar)
    css/
      tokens.css                 # Docusign 2024 brand custom properties
      base.css                   # Reset, typography, utility classes
      nav.css                    # Left sidebar navigation
      cards.css                  # Cards, badges, result rows, field-issue groups
      modals.css                 # Modal dialogs, migration progress
      tables.css                 # Sortable/filterable tables
      forms.css                  # Form inputs, toggles
    js/
      app.js                     # Entry point — router, auth, nav badges
      router.js                  # Hash-based SPA router (#/templates default)
      state.js                   # Global state with pub/sub
      api.js                     # Fetch wrappers for all backend endpoints
      auth.js                    # Auth chips, OAuth flow, toast notifications
      templates.js               # Templates view + detail (overview/issues/history tabs)
      migration.js               # Migration modal, progress polling, results view
      issues.js                  # Issues & Warnings view
      verification.js            # Verification view (send/poll/void envelopes)
      history.js                 # History & Audit view
      settings.js                # Settings view
      project.js                 # Project/customer context (localStorage)
      utils.js                   # escHtml, formatDate, renderFieldIssues, etc.

tests/
  test_normalized_schema.py      # Normalized model + mapping service tests
  test_validation_service.py     # Validation service + report builder tests
  test_migration_options.py      # dryRun, overwriteIfExists, includeDocuments
  test_batch_migration.py        # Batch migration API tests
  test_retry.py                  # Retry with backoff utility tests
  test_security.py               # Log sanitization + PDF checksum tests
  test_upload_upsert.py          # Upsert logic unit tests
  test_api_health.py             # Health endpoint
  test_api_auth.py               # OAuth endpoint tests
  test_api_templates.py          # Template listing + status tests (10 tests)
  test_api_migrate.py            # Migration API tests
  test_api_verify.py             # Verification envelope API tests (9 tests)
  test_e2e.py                    # Full pipeline end-to-end test
  test_regression.py             # Compose output vs snapshots
  fixtures/expected/             # Regression snapshots (3 real templates)

downloads/                       # Downloaded Adobe Sign templates (gitignored)
migration-output/                # Converted DocuSign template JSONs + history
field-mapping.md                 # Field type mapping table + edge case log
PRODUCT-SPEC.md                  # Full product specification (blueprint-aligned)
docs/agent-harness/
  EXECUTION-BOARD.md             # Living kanban board
requirements.txt                 # Python dependencies
```