ds160-autofill

- LLM assistance for complex cases (captcha, missing elements)
- Session persistence for resume capability
- Chinese to English translation for user input

## Quick Start

### Starting a New Application

1. **Provide CSV data**: User must provide a CSV file with their personal information (use `ds160-user-info.csv` as template)

2. **Open DS-160 website**: Use `browser` tool to open `https://ceac.state.gov/genniv/` with profile `openclaw`

3. **Initialize session**: Load and execute `scripts/ds160-filler.js` to parse mappings and CSV

4. **Start filling**: Begin with the home page, handle captcha with LLM, then proceed page by page

### Resuming an Application

1. **Load existing session**: Check `ds160/ds160-session.json` for saved Application ID

2. **Open application**: Use the saved Application ID to resume on DS-160 website

3. **Continue filling**: Load CSV from saved location and continue from last page

## Core Workflow

### Step 1: Prepare Data

**Always check if session exists first:**
```javascript
// Read ds160/ds160-session.json
// If exists → Resume mode
// If not → New application mode
```

**New application mode:**
- Ask user for CSV file path or content
- Save CSV to `ds160/ds160-user-info.csv` using `write` tool
- Initialize session data in `ds160/ds160-session.json`:
  ```json
  {
    "applicationId": null,
    "securityQuestion": null,
    "securityAnswer": null,
    "currentPageIndex": 0,
    "completedPages": [],
    "startDate": "2026-02-06T21:00:00Z"
  }
  ```

**Resume mode:**
- Load CSV from `ds160/ds160-user-info.csv`
- Load session from `ds160/ds160-session.json`
- Report current progress to user

### Step 2: Initialize Browser

```javascript
// Open browser with openclaw profile
browser_start: { profile: "openclaw", targetUrl: "https://ceac.state.gov/genniv/" }

// For resume mode, add Application ID to URL
// Example: https://ceac.state.gov/GenNIV/Common/ConfirmApplicationID.aspx
```

### Step 3: Fill Current Page

**For each page in sequence:**

1. **Snapshot page** to understand current state:
   ```javascript
   browser_snapshot: { refs: "role", profile: "openclaw" }
   ```

2. **Load mappings** from `references/ds160-elements.yaml`

3. **Execute fill logic** using `browser act` with evaluate:
   ```javascript
   // Load ds160-filler.js
   // Call fillPage(page, currentUrl, userData, yamlData)
   ```

4. **Handle results**:
   - **Success**: Continue to next page
   - **Needs LLM assistance**: Call LLM and retry
   - **Needs user input**: Save progress and pause

### Step 4: Handle Special Cases

**Captcha:**
- Take screenshot of captcha area
- Use `image` tool to analyze captcha
- Fill captcha code via browser evaluate

**Missing element:**
- Call LLM with page snapshot
- Ask LLM to locate element and provide alternative selector
- Retry fill operation

**Translation needed:**
- Call LLM with field context and Chinese value
- Ask LLM to translate to appropriate English value
- Update CSV and retry fill operation

**Missing user data:**
- Identify which field is missing
- Report to user with field name and description
- **Important**: Save current page using browser evaluate before pausing
- Wait for user to provide data

### Step 5: Save Progress & Continue

**After completing a page:**

1. **Update session data**:
   ```json
   {
     "applicationId": "AA00FBLCQP",
     "securityQuestion": "What is the given name of your mother's mother?",
     "securityAnswer": "LiMei",
     "currentPageIndex": 5,
     "completedPages": ["home", "security_question", "personal_1", "personal_2"],
     "startDate": "2026-02-06T21:00:00Z"
   }
   ```

2. **Save to file**: `write(ds160/ds160-session.json, JSON.stringify(sessionData))`

3. **Report progress to user**:
   - Current page completed
   - Next page name
   - Application ID
   - Security question/answer (first time only)
   - Overall progress (X/Y pages)

4. **Click "Continue" button** to proceed to next page

## Data Structures

### CSV Format

The CSV must have these columns:
1. `页面` - Page name
2. `字段名称` - Field identifier (matches YAML `name` field)
3. `英文说明` - English description
4. `中文说明` - Chinese description
5. `必填` - Required? (是/否)
6. `示例值` - Example value
7. `用户填写` - **User-provided value** (this is what gets filled)

**Important**: Users can provide values in Chinese. The script includes a built-in translation dictionary for common fields (gender, marital status, countries, etc.). For fields not in the dictionary, LLM will be called for translation automatically.

### YAML Structure

Each page in `ds160-elements.yaml` contains:
- `page_id`: Unique page identifier
- `page_name`: Human-readable page name
- `url`: Page URL pattern
- `elements`: Array of form elements with:
  - `id`: Element ID
  - `name`: Field identifier (matches CSV)
  - `type`: Element type (text, select, radio, checkbox, button)
  - `label`: English label
  - `label_cn`: Chinese label
  - `required`: Is required?
  - `options`: Available options (for select)
  - `group`: Radio group name

## Error Handling

### Element Not Found

**Symptoms**: `fillPage` returns `needsLLM: true`

**Action**:
1. Take page snapshot
2. Call LLM with:
   - Current URL
   - Element information (id, name, type)
   - Page HTML snippet
   - Ask LLM to analyze and provide alternative selector
3. Retry with LLM-suggested selector

### Translation Needed

**Symptoms**: `fillPage` returns `needsTranslation: true`

**Action**:
1. Call LLM with translation context:
   - Field name and description (EN/CN)
   - Field type and available options
   - Original Chinese value
   - Ask LLM to translate to appropriate English value for the field
2. Update CSV with translated value
3. Retry fill operation with translated value

**Example LLM prompt for translation**:
```
I need to translate this Chinese value for a DS-160 form field:

Field: ${elementName}
Type: ${elementType}
English label: ${label}
Chinese label: ${label_cn}
Chinese value: "${originalValue}"

Available options (if select): ${options}

Please provide the correct English value for this field. Consider:
- The field type and context
- Available options (if applicable)
- Standard DS-160 terminology

Return only the English value, no explanation.
```

### Data Missing

**Symptoms**: `fillPage` returns `needsUserInput: true`

**Action**:
1. **CRITICAL**: Save current page before pausing:
   ```javascript
   // Find and click "Save" button
   browser_act: {
     request: { kind: "click", ref: "save_button" }
   }
   ```
2. Report missing field to user:
   - Field name (English and Chinese)
   - Description
   - Whether it's required
3. Update CSV with new data using `write` tool
4. Resume filling

### Captcha

**Symptoms**: Captcha image detected on page

**Action**:
1. Take screenshot of captcha area
2. Use `image` tool:
   ```javascript
   image: {
     image: "/path/to/captcha.png",
     prompt: "What is the code shown in this captcha? Return only the code characters."
   }
   ```
3. Fill captcha field via browser evaluate
4. Submit form

## Page Sequence

The pages are filled in this order:

1. **home** - Start application, select location, handle captcha
2. **security_question** - Confirm Application ID, set security question
3. **personal_1** - Personal information (name, gender, DOB)
4. **personal_2** - Nationality, other nationalities, IDs
5. **travel** - Travel purpose, dates, who is paying
6. **travel_companions** - People traveling with you
7. **previous_us_travel** - Previous US visits
8. **address_phone** - Current address and phone
9. **passport** - Passport information
10. **us_contact** - US point of contact
11. **family_relatives** - Family information
12. **work_education** - Work and education history
13. **security_background** - Security questions

## Progress Reporting

**Report to user after each page:**

```
✓ Completed: Personal Information 1
📄 Next: Personal Information 2
🆔 Application ID: AA00FBLCQP
🔐 Security Question: What is the given name of your mother's mother?
🔑 Answer: LiMei
📊 Progress: 3/13 pages completed
```

**Include in every report:**
- Application ID (once obtained)
- Security question and answer (once generated)
- Current progress (X/Y pages)
- Next page name

## LLM Integration

### When to Call LLM

1. **Captcha detection**: Always use LLM to read captcha
2. **Element not found**: When multiple locator strategies fail
3. **Translation needed**: When user provides Chinese value not in built-in dictionary
4. **Complex elements**: For elements that require understanding context
5. **Validation errors**: When form submission fails with unclear reason

### LLM Prompts

**For element location:**
```
I need to find this form element on a DS-160 page:
- Element ID: {elementId}
- Field name: {fieldName}
- Element type: {elementType}
- Current URL: {url}

Please analyze the page and provide:
1. The correct way to locate this element (CSS selector or JavaScript)
2. Any visible labels or text associated with it
```

**For captcha:**
```
Please read the captcha code from this image. Return only the code characters, no explanation.
```

## Translation Dictionary

The script includes a built-in translation dictionary for common Chinese values. This covers:

**Gender**:
- 男/男性 → MALE
- 女/女性 → FEMALE

**Marital Status**:
- 已婚 → MARRIED
- 未婚 → SINGLE
- 离异 → DIVORCED
- 丧偶 → WIDOWED
- 合法分居 → LEGALLY SEPARATED
- 事实婚姻 → COMMON LAW MARRIAGE
- 民事结合 → CIVIL UNION/DOMESTIC PARTNERSHIP

**Yes/No**:
- 是/有/是，有 → Yes
- 否/没有/否，没有 → No

**Countries**:
- 中国 → CHINA
- 美国 → USA
- 英国 → UNITED KINGDOM
- 日本 → JAPAN
- 韩国 → KOREA, SOUTH

**Visa Types**:
- 旅游/商务/商务/旅游 → B1/B2
- 学生 → F1
- 访问学者 → J1
- 工作 → H1B

**Payment**:
- 自己/自费 → SELF
- 公司 → OTHER COMPANY
- 其他人 → OTHER PERSON

**Relationship (US Point of Contact)**:
- 朋友 → Friend
- 亲戚 → Relative
- 同事 → Colleague
- 配偶 → Spouse
- 同学 → Classmate
- 其他 → Other

**Other Values**:
- 不适用/不，不适用 → Does Not Apply

**For values not in this dictionary**, LLM will be automatically called for translation.

## Session Persistence

**Session file location**: `ds160/ds160-session.json`

**Session data structure**:
```json
{
  "applicationId": "AA00FBLCQP",
  "securityQuestion": "What is the given name of your mother's mother?",
  "securityAnswer": "LiMei",
  "currentPageIndex": 5,
  "completedPages": ["home", "security_question", "personal_1", "personal_2", "travel", "travel_companions"],
  "startDate": "2026-02-06T21:00:00Z",
  "lastUpdated": "2026-02-06T21:30:00Z"
}
```

**When to save session:**
- After completing each page
- Before pausing for user input
- After receiving Application ID
- After setting security question/answer

## Browser Automation Notes

**Always use `openclaw` profile** to maintain session across pages.

**Use `browser act` with `evaluate` for form filling**:
- More reliable than individual click/type actions
- Can execute complex JavaScript
- Better error handling

**Use `browser snapshot` before each page**:
- Check page state
- Identify dynamic elements
- Verify page loaded correctly

## Testing Checklist

Before using this skill with real data:

- [ ] Test element location on each page type
- [ ] Verify CSV parsing with sample data
- [ ] Test captcha handling with LLM
- [ ] Verify session save/load functionality
- [ ] Test resume from middle of application
- [ ] Verify "Save" button clicking works before pauses
- [ ] Test with various element types (text, select, radio, checkbox)
- [ ] Verify progress reporting format
- [ ] Test Chinese to English translation

## Example Usage

**User says**: "Help me fill DS-160 form with my data in this.csv"

**Agent workflow**:
1. Load SKILL.md (this file)
2. Check for existing session
3. Save CSV to ds160/ds160-user-info.csv
4. Initialize session
5. Open browser to DS-160
6. Fill page by page
7. Report progress after each page
8. Handle errors with LLM or user input
9. Save progress and continue

**User says**: "Continue filling my DS-160 form"

**Agent workflow**:
1. Load SKILL.md
2. Load existing session
3. Open browser to DS-160 with Application ID
4. Load CSV from saved location
5. Continue from last page
6. Report progress
7. Complete remaining pages

## Files Reference

- **scripts/ds160-filler.js** - Core automation logic (run via browser evaluate)
- **references/ds160-elements.yaml** - Element mappings (read-only)
- **references/ds160-user-info.csv** - User data template (copy to workspace)
- **ds160/ds160-session.json** - Session persistence (auto-created)
- **ds160/ds160-user-info.csv** - Active user data (auto-created)

## Important Notes

- **NEVER submit the final form** - this is for automation/testing only
- **Always save before pausing** - prevents data loss
- **Report progress frequently** - at least every page
- **Keep Application ID secure** - it's sensitive information
- **Use fake data for testing** - don't use real personal info
- **Chinese input is supported** - use built-in dictionary or LLM for translation