infoscreen-dev/MQTT_PAYLOAD_MIGRATION_GUIDE.md

MQTT Payload Migration Guide

Purpose

This guide describes a practical migration from the current dashboard screenshot payload to a grouped schema, with client-side implementation first and server-side migration second.

Scope

• Environment: development and alpha systems (no production installs)
• Message topic: infoscreen/<client_id>/dashboard
• Capture types to preserve: periodic, event_start, event_stop

Target Schema (v2)

The canonical message should be grouped into four logical blocks in this order:

1. message
2. content
3. runtime
4. metadata

Example shape:

{
  "message": {
    "client_id": "<uuid>",
    "status": "alive"
  },
  "content": {
    "screenshot": {
      "filename": "latest.jpg",
      "data": "<base64>",
      "timestamp": "2026-03-30T10:15:41.123456+00:00",
      "size": 183245
    }
  },
  "runtime": {
    "system_info": {
      "hostname": "pi-display-01",
      "ip": "192.168.1.42",
      "uptime": 123456.7
    },
    "process_health": {
      "event_id": "evt-123",
      "event_type": "presentation",
      "current_process": "impressive",
      "process_pid": 4123,
      "process_status": "running",
      "restart_count": 0
    }
  },
  "metadata": {
    "schema_version": "2.0",
    "producer": "simclient",
    "published_at": "2026-03-30T10:15:42.004321+00:00",
    "capture": {
      "type": "periodic",
      "captured_at": "2026-03-30T10:15:41.123456+00:00",
      "age_s": 0.9,
      "triggered": false,
      "send_immediately": false
    },
    "transport": {
      "qos": 0,
      "publisher": "simclient"
    }
  }
}

Step-by-Step: Client-Side First

1. Create a migration branch.

• Example: feature/payload-v2

2. Freeze a baseline sample from MQTT.

• Capture one payload via mosquitto_sub and store it for comparison.

3. Implement one canonical payload builder.

• Centralize JSON assembly in one function only.
• Do not duplicate payload construction across code paths.

4. Add versioned metadata.

• Set metadata.schema_version = "2.0".
• Add metadata.producer = "simclient".
• Add metadata.published_at in UTC ISO format.

5. Map existing data into grouped blocks.

• client_id/status -> message
• screenshot object -> content.screenshot
• system_info/process_health -> runtime
• capture mode and freshness -> metadata.capture

6. Preserve existing capture semantics.

• Keep type values unchanged: periodic, event_start, event_stop.
• Keep UTC ISO timestamps.
• Keep screenshot encoding and size behavior unchanged.

7. Optional short-term compatibility mode (recommended for one sprint).

• Either:
  • Keep current legacy fields in parallel, or
  • Add a legacy block with old field names.
• Goal: prevent immediate server breakage while parser updates are merged.

8. Improve publish logs for verification.

• Log schema_version, metadata.capture.type, metadata.capture.age_s.

9. Validate all three capture paths end-to-end.

• periodic capture
• event_start trigger capture
• event_stop trigger capture

10. Lock the client contract.

• Save one validated JSON sample per capture type.
• Use those samples in server parser tests.

Step-by-Step: Server-Side Migration

1. Add support for grouped v2 parsing.

• Parse from message/content/runtime/metadata first.

2. Add fallback parser for legacy payload (temporary).

• If grouped keys are absent, parse old top-level keys.

3. Normalize to one internal server model.

• Convert both parser paths into one DTO/entity used by dashboard logic.

4. Validate required fields.

• Required:
  • message.client_id
  • message.status
  • metadata.schema_version
  • metadata.capture.type
• Optional:
  • runtime.process_health
  • content.screenshot (if no screenshot available)

5. Update dashboard consumers.

• Read grouped fields from internal model (not raw old keys).

6. Add migration observability.

• Counters:
  • v2 parse success
  • legacy fallback usage
  • parse failures
• Warning log for unknown schema_version.

7. Run mixed-format integration tests.

• New client -> new server
• Legacy client -> new server (fallback path)

8. Cut over to v2 preferred.

• Keep fallback for short soak period only.

9. Remove fallback and legacy assumptions.

• After stability window, remove old parser path.

10. Final cleanup.

• Keep one schema doc and test fixtures.
• Remove temporary compatibility switches.

Legacy to v2 Field Mapping

Legacy field	v2 field
client_id	message.client_id
status	message.status
screenshot	content.screenshot
screenshot_type	metadata.capture.type
screenshot_age_s	metadata.capture.age_s
timestamp	metadata.published_at
system_info	runtime.system_info
process_health	runtime.process_health

Acceptance Criteria

1. All capture types parse and display correctly.

• periodic
• event_start
• event_stop

2. Screenshot payload integrity is unchanged.

• filename, data, timestamp, size remain valid.

3. Metadata is centrally visible at message end.

• schema_version, capture metadata, transport metadata all inside metadata.

4. No regression in dashboard update timing.

• Triggered screenshots still publish quickly.

Suggested Timeline (Dev Only)

1. Day 1: client v2 payload implementation + local tests
2. Day 2: server v2 parser + fallback
3. Day 3-5: soak in dev, monitor parse metrics
4. Day 6+: remove fallback and finalize v2-only