infoscreen/MQTT_PAYLOAD_MIGRATION_GUIDE.md

# MQTT Payload Migration Guide

## Purpose
This guide describes a practical migration from the current dashboard screenshot payload to a grouped schema, with client-side implementation first and server-side migration second.

## Scope
- Environment: development and alpha systems (no production installs)
- Message topic: infoscreen/<client_id>/dashboard
- Capture types to preserve: periodic, event_start, event_stop

## Target Schema (v2)
The canonical message should be grouped into four logical blocks in this order:

1. message
2. content
3. runtime
4. metadata

Example shape:

```json
{
  "message": {
    "client_id": "<uuid>",
    "status": "alive"
  },
  "content": {
    "screenshot": {
      "filename": "latest.jpg",
      "data": "<base64>",
      "timestamp": "2026-03-30T10:15:41.123456+00:00",
      "size": 183245
    }
  },
  "runtime": {
    "system_info": {
      "hostname": "pi-display-01",
      "ip": "192.168.1.42",
      "uptime": 123456.7
    },
    "process_health": {
      "event_id": "evt-123",
      "event_type": "presentation",
      "current_process": "impressive",
      "process_pid": 4123,
      "process_status": "running",
      "restart_count": 0
    }
  },
  "metadata": {
    "schema_version": "2.0",
    "producer": "simclient",
    "published_at": "2026-03-30T10:15:42.004321+00:00",
    "capture": {
      "type": "periodic",
      "captured_at": "2026-03-30T10:15:41.123456+00:00",
      "age_s": 0.9,
      "triggered": false,
      "send_immediately": false
    },
    "transport": {
      "qos": 0,
      "publisher": "simclient"
    }
  }
}
```

## Step-by-Step: Client-Side First

1. Create a migration branch.
- Example: feature/payload-v2

2. Freeze a baseline sample from MQTT.
- Capture one payload via mosquitto_sub and store it for comparison.

3. Implement one canonical payload builder.
- Centralize JSON assembly in one function only.
- Do not duplicate payload construction across code paths.

4. Add versioned metadata.
- Set metadata.schema_version = "2.0".
- Add metadata.producer = "simclient".
- Add metadata.published_at in UTC ISO format.

5. Map existing data into grouped blocks.
- client_id/status -> message
- screenshot object -> content.screenshot
- system_info/process_health -> runtime
- capture mode and freshness -> metadata.capture

6. Preserve existing capture semantics.
- Keep type values unchanged: periodic, event_start, event_stop.
- Keep UTC ISO timestamps.
- Keep screenshot encoding and size behavior unchanged.

7. Optional short-term compatibility mode (recommended for one sprint).
- Either:
  - Keep current legacy fields in parallel, or
  - Add a legacy block with old field names.
- Goal: prevent immediate server breakage while parser updates are merged.

8. Improve publish logs for verification.
- Log schema_version, metadata.capture.type, metadata.capture.age_s.

9. Validate all three capture paths end-to-end.
- periodic capture
- event_start trigger capture
- event_stop trigger capture

10. Lock the client contract.
- Save one validated JSON sample per capture type.
- Use those samples in server parser tests.

## Step-by-Step: Server-Side Migration

1. Add support for grouped v2 parsing.
- Parse from message/content/runtime/metadata first.

2. Add fallback parser for legacy payload (temporary).
- If grouped keys are absent, parse old top-level keys.

3. Normalize to one internal server model.
- Convert both parser paths into one DTO/entity used by dashboard logic.

4. Validate required fields.
- Required:
  - message.client_id
  - message.status
  - metadata.schema_version
  - metadata.capture.type
- Optional:
  - runtime.process_health
  - content.screenshot (if no screenshot available)

5. Update dashboard consumers.
- Read grouped fields from internal model (not raw old keys).

6. Add migration observability.
- Counters:
  - v2 parse success
  - legacy fallback usage
  - parse failures
- Warning log for unknown schema_version.

7. Run mixed-format integration tests.
- New client -> new server
- Legacy client -> new server (fallback path)

8. Cut over to v2 preferred.
- Keep fallback for short soak period only.

9. Remove fallback and legacy assumptions.
- After stability window, remove old parser path.

10. Final cleanup.
- Keep one schema doc and test fixtures.
- Remove temporary compatibility switches.

## Legacy to v2 Field Mapping

| Legacy field | v2 field |
|---|---|
| client_id | message.client_id |
| status | message.status |
| screenshot | content.screenshot |
| screenshot_type | metadata.capture.type |
| screenshot_age_s | metadata.capture.age_s |
| timestamp | metadata.published_at |
| system_info | runtime.system_info |
| process_health | runtime.process_health |

## Acceptance Criteria

1. All capture types parse and display correctly.
- periodic
- event_start
- event_stop

2. Screenshot payload integrity is unchanged.
- filename, data, timestamp, size remain valid.

3. Metadata is centrally visible at message end.
- schema_version, capture metadata, transport metadata all inside metadata.

4. No regression in dashboard update timing.
- Triggered screenshots still publish quickly.

## Suggested Timeline (Dev Only)

1. Day 1: client v2 payload implementation + local tests
2. Day 2: server v2 parser + fallback
3. Day 3-5: soak in dev, monitor parse metrics
4. Day 6+: remove fallback and finalize v2-only