# Patient Context-Aware Avatar System

## Overview

This system enables the LiveAvatar to be preloaded with patient-specific context from the FHIR server. When a patient calls, the avatar has immediate access to their medical history, medications, doctors, appointments, and notes.

## Features

1. **Context Preloading**: Patient data is fetched from FHIR via MCP before the avatar session starts
2. **Phone Number Routing**: Each patient gets a unique URL with their phone number
3. **Call Transcription**: All conversations are automatically transcribed
4. **EMR Auto-Update**: After the call, the transcript is analyzed and the EMR is updated with relevant information

## How It Works

### 1. Starting a Session with Patient Context

Open the avatar page with a phone number parameter:

```
http://localhost:3000/test-avatar.html?phone=(213)930-3541
```

The system will:
- Fetch all patient data from FHIR (history, medications, doctors, nurses, addresses, notes)
- Load this context into the avatar's prompt
- Start the session with the avatar already aware of the patient's information

### 2. During the Call

- The avatar can reference the patient's medical history
- All conversation is transcribed in real-time
- Transcript is stored locally with speaker labels (Patient/Avatar)

### 3. After the Call

When you click "End Session":
- Call transcript is saved to `/call_logs/` directory
- The transcript is analyzed by Claude via MCP
- EMR is automatically updated with:
  - New symptoms or complaints mentioned
  - Medication changes or adherence issues
  - Appointments scheduled/rescheduled
  - Clinical notes
  - Urgent issues flagged for follow-up

## API Endpoints

### POST `/generate_session`

Creates a new avatar session with optional patient context.

**Request:**
```json
{
  "room_name": "clinic-room-123",
  "participant_name": "Patient",
  "phone_number": "(213) 930-3541"  // Optional
}
```

**Response:**
```json
{
  "livekit_url": "wss://...",
  "livekit_token": "...",
  "heygen_token": "...",
  "la_session_id": "...",
  "room_name": "clinic-room-123",
  "phone_number": "(213) 930-3541",
  "patient_context_loaded": true,
  "error": null
}
```

### POST `/save_call_log`

Saves call transcript after session ends.

**Request:**
```json
{
  "session_id": "abc123",
  "phone_number": "(213) 930-3541",
  "transcript": [
    {
      "speaker": "Patient",
      "text": "I've been having headaches",
      "timestamp": "2026-05-30T12:00:00Z"
    },
    {
      "speaker": "Avatar",
      "text": "I see. How long have you been experiencing these headaches?",
      "timestamp": "2026-05-30T12:00:05Z"
    }
  ],
  "duration": 180
}
```

**Response:**
```json
{
  "success": true,
  "log_file": "/path/to/call_logs/call_2139303541_20260530_120000_abc123.json",
  "session_id": "abc123"
}
```

### POST `/update_emr_from_call`

Processes call log and updates EMR.

**Request:**
```json
{
  "log_file": "/path/to/call_logs/call_2139303541_20260530_120000_abc123.json"
}
```

**Response:**
```json
{
  "success": true,
  "summary": "Updated patient record with new headache symptoms. Added clinical note about medication adherence. Scheduled follow-up appointment for next week.",
  "log_file": "/path/to/call_logs/call_2139303541_20260530_120000_abc123.json"
}
```

## Setup

### Prerequisites

1. **Environment Variables** (in `.env`):
   ```bash
   ANTHROPIC_API_KEY=your-key-here
   ELUME_MCP_URL=http://localhost:8082/mcp
   ELUME_MCP_TOKEN=test-token-123
   LIVEAVATAR_API_KEY=your-liveavatar-key
   LIVEKIT_API_KEY=your-livekit-key
   LIVEKIT_API_SECRET=your-livekit-secret
   LIVEKIT_URL=wss://your-livekit-url
   ```

2. **Python Dependencies**:
   ```bash
   pip install anthropic mcp python-dotenv httpx fastapi uvicorn
   ```

3. **MCP FHIR Server** must be running on port 8082

### Running the System

1. **Start the API server**:
   ```bash
   cd livekit-agent/server
   python api.py
   ```

2. **Open the avatar page** with a phone number:
   ```
   http://localhost:3000/test-avatar.html?phone=(213)930-3541
   ```

3. **Click "Start Session"** - the avatar will load with patient context

4. **Have the conversation**

5. **Click "End Session"** - logs are saved and EMR is updated automatically

## Call Logs

Call logs are stored in `/call_logs/` with the format:
```
call_{phone_number}_{timestamp}_{session_id}.json
```

Example log structure:
```json
{
  "session_id": "abc123",
  "phone_number": "(213) 930-3541",
  "timestamp": "2026-05-30T12:00:00Z",
  "duration_seconds": 180,
  "transcript": [...],
  "processed": true,
  "emr_update_summary": "Updated patient record...",
  "processed_at": "2026-05-30T12:03:00Z"
}
```

## Patient Context Query

The system queries the FHIR server for:
- Patient demographics and contact information
- Medical history and conditions
- Current and past medications
- Allergies
- Recent lab results and diagnostics
- Appointments (past and upcoming)
- Care team (doctors, nurses, specialists)
- Clinical notes and observations
- Immunization records
- Procedures

This comprehensive context allows the avatar to provide personalized, informed assistance.

## Workflow Example

1. **Patient calls clinic** → System opens `test-avatar.html?phone=(213)930-3541`
2. **Avatar loads** with full patient context from FHIR
3. **Patient**: "I need to reschedule my appointment"
4. **Avatar**: "I see you have an appointment with Dr. Smith on June 5th. When would you like to reschedule?"
5. **Patient**: "Next Tuesday at 2pm"
6. **Avatar**: "Done! Your appointment has been rescheduled to June 7th at 2:00 PM"
7. **Call ends** → Transcript saved → EMR updated with appointment change

## Troubleshooting

- **No patient context loaded**: Check that `ANTHROPIC_API_KEY` and MCP server are configured
- **EMR not updating**: Verify MCP server has write permissions to FHIR resources
- **Transcript empty**: Check browser console for LiveAvatar SDK errors
- **Call logs not saving**: Ensure `/call_logs/` directory exists and is writable

## Security Notes

- Phone numbers should be sanitized/validated before use
- Call logs contain PHI - ensure proper access controls
- Consider encrypting call logs at rest
- Implement proper authentication for production use