docs: archive legacy guides and streamline copilot instructions governance

This commit is contained in:
2026-04-01 08:37:50 +00:00
parent 365d8f58f3
commit 06411edfab
15 changed files with 199 additions and 580 deletions

View File

@@ -1,498 +1,102 @@
# Copilot instructions for infoscreen_2025 # Copilot instructions for infoscreen_2025
# Purpose ## Purpose
These instructions tell Copilot Chat how to reason about this codebase. This file is a concise, high-signal brief for coding agents.
Prefer explanations and refactors that align with these structures. It is not a changelog and not a full architecture handbook.
Use this as your shared context when proposing changes. Keep edits minimal and match existing patterns referenced below.
## TL;DR ## TL;DR
Small multi-service digital signage app (Flask API, React dashboard, MQTT scheduler). Edit `server/` for API logic, `scheduler/` for event publishing, and `dashboard/` for UI. If you're asking Copilot for changes, prefer focused prompts that include the target file(s) and the desired behavior. - Stack: Flask API + MariaDB, React/Vite dashboard, MQTT listener, scheduler, worker.
- Main areas:
- API logic in `server/`
- Scheduler logic in `scheduler/`
- UI logic in `dashboard/src/`
- Keep changes minimal, match existing patterns, and update docs in the same commit when behavior changes.
### How to ask Copilot ## Fast file map
- "Add a new route `GET /api/events/summary` that returns counts per event_type — implement in `server/routes/events.py`." - `scheduler/scheduler.py` - scheduler loop, MQTT event publishing, TV power intent publishing
- "Create an Alembic migration to add `duration` and `resolution` to `event_media` and update upload handler to populate them." - `scheduler/db_utils.py` - event formatting and power-intent helper logic
- "Refactor `scheduler/db_utils.py` to prefer precomputed EventMedia metadata and fall back to a HEAD probe." - `listener/listener.py` - discovery/heartbeat/log/screenshot MQTT consumption
- "Add an ffprobe-based worker that extracts duration/resolution/bitrate and stores them on `EventMedia`." - `server/routes/events.py` - event CRUD, recurrence handling, UTC normalization
- `server/routes/eventmedia.py` - file manager, media upload/stream endpoints
- `server/routes/groups.py` - group lifecycle, alive status, order persistence
- `server/routes/system_settings.py` - system settings CRUD and supplement-table endpoint
- `dashboard/src/settings.tsx` - settings UX and system-defaults integration
- `dashboard/src/components/CustomEventModal.tsx` - event creation/editing UX
- `dashboard/src/monitoring.tsx` - superadmin monitoring page
- `TV_POWER_INTENT_SERVER_CONTRACT_V1.md` - Phase 1 TV power contract
Keep docs synced with code. When you change services/MQTT/API/UTC/env or dev/prod run steps, update this file in the same commit (see `AI-INSTRUCTIONS-MAINTENANCE.md`). ## Service picture
- API: `server/` on `:8000` (health: `/health`)
- Dashboard: `dashboard/` (dev `:5173`, proxied API calls)
- MQTT broker: Mosquitto (`mosquitto/config/mosquitto.conf`)
- Listener: MQTT consumer that updates server-side state
- Scheduler: publishes active events and group-level TV power intents
- Nginx: routes `/api/*` and `/screenshots/*` to API, dashboard otherwise
### When not to change ## Non-negotiable conventions
- Avoid editing generated assets under `dashboard/dist/` and compiled bundles. Don't modify files produced by CI or Docker builds (unless intentionally updating build outputs). - Datetime:
- Store/compare in UTC.
- API returns ISO strings without `Z` in many routes.
- Frontend must append `Z` before parsing if needed.
- JSON naming:
- Backend internals use snake_case.
- API responses use camelCase (via `server/serializers.py`).
- DB host in containers: `db` (not localhost).
- Never put secrets in docs.
### Contact / owner ## MQTT contracts
- Primary maintainer: RobbStarkAustria (owner). For architecture questions, ping the repo owner or open an issue and tag `@RobbStarkAustria`. - Event list topic (retained): `infoscreen/events/{group_id}`
- Group assignment topic (retained): `infoscreen/{uuid}/group_id`
- Heartbeat topic: `infoscreen/{uuid}/heartbeat`
- Logs topic family: `infoscreen/{uuid}/logs/{error|warn|info}`
- Health topic: `infoscreen/{uuid}/health`
- Dashboard screenshot topic: `infoscreen/{uuid}/dashboard`
- TV power intent Phase 1 topic (retained, QoS1): `infoscreen/groups/{group_id}/power/intent`
### Important files (quick jump targets) TV power intent Phase 1 rules:
- `scheduler/db_utils.py` — event formatting and scheduler-facing logic - Schema version is `"1.0"`.
- `scheduler/scheduler.py` — scheduler main loop and MQTT publisher - Group-only scope in Phase 1.
- `server/routes/eventmedia.py` — file uploads, streaming endpoint - Heartbeat publish keeps `intent_id`; semantic transition rotates `intent_id`.
- `server/routes/events.py` — event CRUD and recurrence handling - Expiry rule: `expires_at = issued_at + max(3 x poll_interval_sec, 90s)`.
- `server/routes/groups.py` — group management, alive status, display order persistence - Canonical contract is `TV_POWER_INTENT_SERVER_CONTRACT_V1.md`.
- `dashboard/src/components/CustomEventModal.tsx` — event creation UI
- `dashboard/src/media.tsx` — FileManager / upload settings
- `dashboard/src/settings.tsx` — settings UI (nested tabs; system defaults for presentations and videos)
- `dashboard/src/ressourcen.tsx` — timeline view showing all groups' active events in parallel
- `dashboard/src/ressourcen.css` — timeline and resource view styling
- `dashboard/src/monitoring.tsx` — superadmin-only monitoring dashboard for client health, screenshots, and logs
## Backend patterns
- Routes in `server/routes/*`, registered in `server/wsgi.py`.
- Use one request-scoped DB session, commit on mutation, always close session.
- Keep enum/datetime serialization JSON-safe.
- Maintain UTC-safe comparisons in scheduler and routes.
- Keep recurrence handling backend-driven and consistent with exceptions.
## Frontend patterns
- Use Syncfusion-based patterns already present in dashboard.
- Keep API requests relative (`/api/...`) to use Vite proxy in dev.
- Respect `FRONTEND_DESIGN_RULES.md` for component and styling conventions.
- Keep role-gated UI behavior aligned with backend authorization.
## Big picture ## Environment variables (high-value)
- Multi-service app orchestrated by Docker Compose. - Scheduler: `POLL_INTERVAL_SECONDS`, `REFRESH_SECONDS`
- API: Flask + SQLAlchemy (MariaDB), in `server/` exposed on :8000 (health: `/health`). - Power intent: `POWER_INTENT_PUBLISH_ENABLED`, `POWER_INTENT_HEARTBEAT_ENABLED`, `POWER_INTENT_EXPIRY_MULTIPLIER`, `POWER_INTENT_MIN_EXPIRY_SECONDS`
- Dashboard: React + Vite in `dashboard/`, dev on :5173, served via Nginx in prod. - Monitoring: `PRIORITY_SCREENSHOT_TTL_SECONDS`
- MQTT broker: Eclipse Mosquitto, config in `mosquitto/config/mosquitto.conf`. - Core: `DB_CONN`, `DB_USER`, `DB_PASSWORD`, `DB_HOST`, `DB_NAME`, `ENV`
- Listener: MQTT consumer handling discovery, heartbeats, and dashboard screenshot uploads in `listener/listener.py`.
- Scheduler: Publishes only currently active events (per group, at "now") to MQTT retained topics in `scheduler/scheduler.py`. It queries a future window (default: 7 days) to expand recurring events using RFC 5545 rules and applies event exceptions, but only publishes events that are active at the current time. When a group has no active events, the scheduler clears its retained topic by publishing an empty list. All time comparisons are UTC; any naive timestamps are normalized. Logging is concise; conversion lookups are cached and logged only once per media.
- Nginx: Reverse proxy routes `/api/*` and `/screenshots/*` to API; everything else to dashboard (`nginx.conf`).
- Dev Container (hygiene): UI-only `Dev Containers` extension runs on host UI via `remote.extensionKind`; do not install it in-container. Dashboard installs use `npm ci`; shell aliases in `postStartCommand` are appended idempotently. ## Edit guardrails
- Do not edit generated assets in `dashboard/dist/`.
- Do not change CI/build outputs unless explicitly intended.
- Preserve existing API behavior unless task explicitly requires a change.
- Prefer links to canonical docs instead of embedding long historical notes here.
### Screenshot retention ## Documentation sync rule
- Screenshots sent via dashboard MQTT are stored in `server/screenshots/`. When services/MQTT/API/UTC/env behavior changes:
- Screenshot payloads support `screenshot_type` with values `periodic`, `event_start`, `event_stop`. 1. Update this file (concise deltas only).
- `periodic` is the normal heartbeat/dashboard screenshot path; `event_start` and `event_stop` are high-priority screenshots for monitoring. 2. Update canonical docs where details live.
- For each client, the API keeps `{uuid}.jpg` as latest and the last 20 timestamped screenshots (`{uuid}_..._{type}.jpg`), deleting older timestamped files automatically. 3. Update changelogs separately (`TECH-CHANGELOG.md`, `DEV-CHANGELOG.md`, `dashboard/public/program-info.json` as appropriate).
- For high-priority screenshots, the API additionally maintains `{uuid}_priority.jpg` and metadata in `{uuid}_meta.json` (`latest_screenshot_type`, `last_priority_*`).
## Recent changes since last commit ## Canonical docs map
- Repo entry: `README.md`
### Latest (March 2026) - Instruction governance: `AI-INSTRUCTIONS-MAINTENANCE.md`
- Technical release details: `TECH-CHANGELOG.md`
- **Monitoring System Completion (no version bump)**: - Workspace/development notes: `DEV-CHANGELOG.md`
- End-to-end monitoring pipeline completed: MQTT logs/health → listener persistence → monitoring APIs → superadmin dashboard - MQTT payload details: `MQTT_EVENT_PAYLOAD_GUIDE.md`
- API now serves aggregated monitoring via `GET /api/client-logs/monitoring-overview` and system-wide recent errors via `GET /api/client-logs/recent-errors` - TV power contract: `TV_POWER_INTENT_SERVER_CONTRACT_V1.md`
- Monitoring dashboard (`dashboard/src/monitoring.tsx`) is active and displays client health states, screenshots, process metadata, and recent log activity - Frontend patterns: `FRONTEND_DESIGN_RULES.md`
- **Screenshot Priority Pipeline (no version bump)**: - Archived historical docs: `docs/archive/`
- Listener forwards `screenshot_type` from MQTT screenshot/dashboard payloads to `POST /api/clients/<uuid>/screenshot`.
- API stores typed screenshots, tracks latest/priority metadata, and serves priority images via `GET /screenshots/<uuid>/priority`.
- Monitoring overview exposes screenshot priority state (`latestScreenshotType`, `priorityScreenshotType`, `priorityScreenshotReceivedAt`, `hasActivePriorityScreenshot`) and `summary.activePriorityScreenshots`.
- Monitoring UI shows screenshot type badges and switches to faster refresh while priority screenshots are active.
- **MQTT Dashboard Payload v2 Cutover (no version bump)**:
- Dashboard payload parsing in `listener/listener.py` is now v2-only (`message`, `content`, `runtime`, `metadata`).
- Legacy top-level dashboard fallback was removed after migration soak (`legacy_fallback=0`).
- Listener observability summarizes parser health using `v2_success` and `parse_failures` counters.
- **Presentation Flags Persistence Fix**:
- Fixed persistence for presentation `page_progress` and `auto_progress` to ensure values are reliably stored and returned across create/update paths and detached occurrences
### Earlier (January 2026)
- **Ressourcen Page (Timeline View)**:
- New 'Ressourcen' page with parallel timeline view showing active events for all room groups
- Compact timeline display with adjustable row height (65px per group)
- Real-time view of currently running events with type, title, and time window
- Customizable group ordering with visual reordering panel (drag up/down buttons)
- Group order persisted via `GET/POST /api/groups/order` endpoints
- Color-coded event bars matching group theme
- Timeline modes: Day and Week views (day view by default)
- Dynamic height calculation based on number of groups
- Syncfusion ScheduleComponent with TimelineViews, Resize, and DragAndDrop support
- Files: `dashboard/src/ressourcen.tsx` (page), `dashboard/src/ressourcen.css` (styles)
### Earlier (November 2025)
- **API Naming Convention Standardization (camelCase)**:
- Backend: Created `server/serializers.py` with `dict_to_camel_case()` utility for consistent JSON serialization
- Events API: `GET /api/events` and `GET /api/events/<id>` now return camelCase fields (`id`, `subject`, `startTime`, `endTime`, `type`, `groupId`, etc.) instead of PascalCase
- Frontend: Dashboard and appointments page updated to consume camelCase API responses
- Appointments page maintains internal PascalCase for Syncfusion scheduler compatibility with automatic mapping from API responses
- **Breaking**: External API consumers must update field names from PascalCase to camelCase
- **UTC Time Handling**:
- Database stores all timestamps in UTC (naive timestamps normalized by backend)
- API returns ISO strings without 'Z' suffix: `"2025-11-27T20:03:00"`
- Frontend: Dashboard and appointments automatically append 'Z' to parse as UTC and display in user's local timezone
- Time formatting functions use `toLocaleTimeString('de-DE')` for German locale display
- All time comparisons use UTC; `new Date().toISOString()` sends UTC back to API
- API returns ISO strings without `Z`; frontend must append `Z` before parsing to ensure UTC
- **Dashboard Enhancements**:
- New card-based design for Raumgruppen (room groups) with Syncfusion components
- Global statistics summary: total infoscreens, online/offline counts, warning groups
- Filter buttons: All, Online, Offline, Warnings with dynamic counts
- Active event display per group: shows currently playing content with type icon, title, date, and time
- Health visualization with color-coded progress bars per group
- Expandable client details with last alive timestamps
- Bulk restart functionality for offline clients per group
- Manual refresh button with toast notifications
- 15-second auto-refresh interval
### Earlier changes
- Scheduler: when formatting video events the scheduler now performs a best-effort HEAD probe of the streaming URL and includes basic metadata in the emitted payload (mime_type, size, accept_ranges). Placeholders for richer metadata (duration, resolution, bitrate, qualities, thumbnails, checksum) are included for later population by a background worker.
- Streaming endpoint: a range-capable streaming endpoint was added at `/api/eventmedia/stream/<media_id>/<filename>` that supports byte-range requests (206 Partial Content) to enable seeking from clients.
- Event model & API: `Event` gained video-related fields (`event_media_id`, `autoplay`, `loop`, `volume`) and the API accepts and persists these when creating/updating video events.
- Dashboard: UI updated to allow selecting uploaded videos for events and to specify autoplay/loop/volume. File upload settings were increased (maxFileSize raised) and the client now validates video duration (max 10 minutes) before upload.
- FileManager: uploads compute basic metadata and enqueue conversions for office formats as before; video uploads now surface size and are streamable via the new endpoint.
- Event model & API (new): Added `muted` (Boolean) for video events; create/update and GET endpoints accept, persist, and return `muted` alongside `autoplay`, `loop`, and `volume`.
- Dashboard — Settings: Settings page refactored to nested tabs; added Events → Videos defaults (autoplay, loop, volume, mute) backed by system settings keys (`video_autoplay`, `video_loop`, `video_volume`, `video_muted`).
- Dashboard — Events UI: CustomEventModal now exposes per-event video `muted` and initializes all video fields from system defaults when creating a new event.
- Dashboard — Academic Calendar: Holiday management now uses a single “📥 Ferienkalender: Import/Anzeige” tab; admins select the target academic period first, and import/list content redraws for that period.
- Dashboard — Holiday Management Hardening: The same tab now supports manual holiday CRUD in addition to CSV/TXT import. Imports and manual saves validate date ranges against the selected academic period, prevent duplicates, auto-merge same normalized name+region overlaps (including adjacent ranges), and report conflicting overlaps.
Note: these edits are intentionally backwards-compatible — if the probe fails, the scheduler still emits the stream URL and the client should fallback to a direct play attempt or request richer metadata when available.
Backend rework notes (no version bump):
- Dev container hygiene: UI-only Remote Containers; reproducible dashboard installs (`npm ci`); idempotent shell aliases.
- Serialization consistency: snake_case internal → camelCase external via `server/serializers.py` for all JSON.
- UTC normalization across routes/scheduler; enums and datetimes serialize consistently.
## Service boundaries & data flow
- Database connection string is passed as `DB_CONN` (mysql+pymysql) to Python services.
- API builds its engine in `server/database.py` (loads `.env` only in development).
- Listener also creates its own engine for writes to `clients`.
- Scheduler queries a future window (default: 7 days) to expand recurring events using RFC 5545 rules, applies event exceptions (skipped dates, detached occurrences), and publishes only events that are active at the current time (UTC). When a group has no active events, the scheduler clears its retained topic by publishing an empty list. Time comparisons are UTC; naive timestamps are normalized. Logging is concise; conversion lookups are cached and logged only once per media.
- MQTT topics (paho-mqtt v2, use Callback API v2):
- Discovery: `infoscreen/discovery` (JSON includes `uuid`, hw/ip data). ACK to `infoscreen/{uuid}/discovery_ack`. See `listener/listener.py`.
- Heartbeat: `infoscreen/{uuid}/heartbeat` updates `Client.last_alive` (UTC); enhanced payload includes `current_process`, `process_pid`, `process_status`, `current_event_id`.
- Event lists (retained): `infoscreen/events/{group_id}` from `scheduler/scheduler.py`.
- Per-client group assignment (retained): `infoscreen/{uuid}/group_id` via `server/mqtt_helper.py`.
- Client logs: `infoscreen/{uuid}/logs/{error|warn|info}` with JSON payload (timestamp, message, context); QoS 1 for ERROR/WARN, QoS 0 for INFO.
- Client health: `infoscreen/{uuid}/health` with metrics (expected_state, actual_state, health_metrics); QoS 0, published every 5 seconds.
- Dashboard screenshots: `infoscreen/{uuid}/dashboard` uses grouped v2 payload blocks (`message`, `content`, `runtime`, `metadata`); listener reads screenshot data from `content.screenshot` and capture type from `metadata.capture.type`.
- Screenshots: server-side folder `server/screenshots/`; API serves `/screenshots/{uuid}.jpg` (latest) and `/screenshots/{uuid}/priority` (active high-priority fallback to latest).
- Dev Container guidance: If extensions reappear inside the container, remove UI-only extensions from `devcontainer.json` `extensions` and map them in `remote.extensionKind` as `"ui"`.
- Presentation conversion (PPT/PPTX/ODP → PDF):
- Trigger: on upload in `server/routes/eventmedia.py` for media types `ppt|pptx|odp` (compute sha256, upsert `Conversion`, enqueue job).
- Worker: RQ worker runs `server.worker.convert_event_media_to_pdf`, calls Gotenberg LibreOffice endpoint, writes to `server/media/converted/`.
- Services: Redis (queue) and Gotenberg added in compose; worker service consumes the `conversions` queue.
- Env: `REDIS_URL` (default `redis://redis:6379/0`), `GOTENBERG_URL` (default `http://gotenberg:3000`).
- Endpoints: `POST /api/conversions/<media_id>/pdf` (ensure/enqueue), `GET /api/conversions/<media_id>/status`, `GET /api/files/converted/<path>` (serve PDFs).
- Storage: originals under `server/media/…`, outputs under `server/media/converted/` (prod compose mounts a shared volume for this path).
## Data model highlights (see `models/models.py`)
- User model: Includes 7 new audit/security fields (migration: `4f0b8a3e5c20_add_user_audit_fields.py`):
- `last_login_at`, `last_password_change_at`: TIMESTAMP (UTC) tracking for auth events
- `failed_login_attempts`, `last_failed_login_at`: Security monitoring for brute-force detection
- `locked_until`: TIMESTAMP placeholder for account lockout (infrastructure in place, not yet enforced)
- `deactivated_at`, `deactivated_by`: Soft-delete audit trail (FK self-reference); soft deactivation is the default, hard delete superadmin-only
- Role hierarchy (privilege escalation enforced): `user` < `editor` < `admin` < `superadmin`
- Client monitoring (migration: `c1d2e3f4g5h6_add_client_monitoring.py`):
- `ClientLog` model: Centralized log storage with fields (id, client_uuid, timestamp, level, message, context, created_at); FK to clients.uuid (CASCADE)
- `Client` model extended: 7 health monitoring fields (`current_event_id`, `current_process`, `process_status`, `process_pid`, `last_screenshot_analyzed`, `screen_health_status`, `last_screenshot_hash`)
- Enums: `LogLevel` (ERROR, WARN, INFO, DEBUG), `ProcessStatus` (running, crashed, starting, stopped), `ScreenHealthStatus` (OK, BLACK, FROZEN, UNKNOWN)
- Indexes: (client_uuid, timestamp DESC), (level, timestamp DESC), (created_at DESC) for performance
- System settings: `system_settings` keyvalue store via `SystemSetting` for global configuration (e.g., WebUntis/Vertretungsplan supplement-table). Managed through routes in `server/routes/system_settings.py`.
- Presentation defaults (system-wide):
- `presentation_interval` (seconds, default "10")
- `presentation_page_progress` ("true"/"false", default "true")
- `presentation_auto_progress` ("true"/"false", default "true")
Seeded in `server/init_defaults.py` if missing.
- Video defaults (system-wide):
- `video_autoplay` ("true"/"false", default "true")
- `video_loop` ("true"/"false", default "true")
- `video_volume` (0.01.0, default "0.8")
- `video_muted` ("true"/"false", default "false")
Used as initial values when creating new video events; editable per event.
- Events: Added `page_progress` (Boolean) and `auto_progress` (Boolean) for presentation behavior per event.
- Event (video fields): `event_media_id`, `autoplay`, `loop`, `volume`, `muted`.
- WebUntis URL: WebUntis uses the existing Vertretungsplan/Supplement-Table URL (`supplement_table_url`). There is no separate `webuntis_url` setting; use `GET/POST /api/system-settings/supplement-table`.
- Conversions:
- Enum `ConversionStatus`: `pending`, `processing`, `ready`, `failed`.
- Table `conversions`: `id`, `source_event_media_id` (FK→`event_media.id` ondelete CASCADE), `target_format`, `target_path`, `status`, `file_hash` (sha256), `started_at`, `completed_at`, `error_message`.
- Indexes: `(source_event_media_id, target_format)`, `(status, target_format)`; Unique: `(source_event_media_id, target_format, file_hash)`.
## API patterns
- Blueprints live in `server/routes/*` and are registered in `server/wsgi.py` with `/api/...` prefixes.
- Session usage: instantiate `Session()` per request, commit when mutating, and always `session.close()` before returning.
- Examples:
- Clients: `server/routes/clients.py` includes bulk group updates and MQTT sync (`publish_multiple_client_groups`).
- Groups: `server/routes/groups.py` computes “alive” using a grace period that varies by `ENV`.
- `GET /api/groups/order` — retrieve saved group display order
- `POST /api/groups/order` — persist group display order (array of group IDs)
- Events: `server/routes/events.py` serializes enum values to strings and normalizes times to UTC. Recurring events are only deactivated after their recurrence_end (UNTIL); non-recurring events deactivate after their end time. Event exceptions are respected and rendered in scheduler output.
- Holidays: `server/routes/holidays.py` supports period-scoped list/import/manual CRUD (`GET/POST /api/holidays`, `POST /api/holidays/upload`, `PUT/DELETE /api/holidays/<id>`), validates date ranges against the target period, prevents duplicates, merges same normalized `name+region` overlaps (including adjacent ranges), and rejects conflicting overlaps.
- Media: `server/routes/eventmedia.py` implements a simple file manager API rooted at `server/media/`.
- System settings: `server/routes/system_settings.py` exposes keyvalue CRUD (`/api/system-settings`) and a convenience endpoint for WebUntis/Vertretungsplan supplement-table: `GET/POST /api/system-settings/supplement-table` (admin+).
- Academic periods: `server/routes/academic_periods.py` exposes full lifecycle management (admin+ only):
- `GET /api/academic_periods` — list all non-archived periods ordered by start_date
- `GET /api/academic_periods/<id>` — get single period by ID (including archived)
- `GET /api/academic_periods/active` — get currently active period
- `GET /api/academic_periods/for_date?date=YYYY-MM-DD` — period covering given date (non-archived)
- `GET /api/academic_periods/<id>/usage` — check linked events/media and recurrence spillover blockers
- `POST /api/academic_periods` — create period (validates name uniqueness among non-archived, date range, overlaps within periodType)
- `PUT /api/academic_periods/<id>` — update period (cannot update archived periods)
- `POST /api/academic_periods/<id>/activate` — activate period (deactivates all others; cannot activate archived)
- `POST /api/academic_periods/<id>/archive` — soft-delete period (blocked if active or has active recurrence)
- `POST /api/academic_periods/<id>/restore` — restore archived period (returns to inactive)
- `DELETE /api/academic_periods/<id>` — hard-delete archived inactive period (blocked if linked events exist)
- All responses use camelCase: `startDate`, `endDate`, `periodType`, `isActive`, `isArchived`, `archivedAt`, `archivedBy`
- Validation: name required/trimmed/unique among non-archived; startDate ≤ endDate; periodType in {schuljahr, semester, trimester}; overlaps blocked within same periodType
- Recurrence spillover detection: archive/delete blocked if recurring master events assigned to period still have current/future occurrences
- User management: `server/routes/users.py` exposes comprehensive CRUD for users (admin+):
- `GET /api/users` — list all users (role-filtered: admin sees user/editor/admin, superadmin sees all); includes audit fields in camelCase (lastLoginAt, lastPasswordChangeAt, failedLoginAttempts, deactivatedAt, deactivatedBy)
- `POST /api/users` — create user with username, password (min 6 chars), role, and status; admin cannot create superadmin; initializes audit fields
- `GET /api/users/<id>` — get detailed user record with all audit fields
- `PUT /api/users/<id>` — update user (cannot change own role/status; admin cannot modify superadmin accounts)
- `PUT /api/users/<id>/password` — admin password reset (requires backend check to reject self-reset for consistency)
- `DELETE /api/users/<id>` — hard delete (superadmin only, with self-deletion check)
- Auth routes (`server/routes/auth.py`): Enhanced to track login events (sets `last_login_at`, resets `failed_login_attempts` on success; increments `failed_login_attempts` and `last_failed_login_at` on failure). Self-service password change via `PUT /api/auth/change-password` requires current password verification.
- Client logs (`server/routes/client_logs.py`): Centralized log retrieval for monitoring:
- `GET /api/client-logs/<uuid>/logs` Query client logs with filters (level, limit, since); admin_or_higher
- `GET /api/client-logs/summary` Log counts by level per client (last 24h); admin_or_higher
- `GET /api/client-logs/recent-errors` System-wide error monitoring; admin_or_higher
- `GET /api/client-logs/monitoring-overview` Includes screenshot priority fields per client plus `summary.activePriorityScreenshots`; superadmin_only
- `GET /api/client-logs/test` Infrastructure validation (no auth); returns recent logs with counts
Documentation maintenance: keep this file aligned with real patterns; update when routes/session/UTC rules change. Avoid long prose; link exact paths.
## Frontend patterns (dashboard)
- **UI design rules**: Component choices, layout structure, button variants, badge colors, dialog patterns, toast conventions, and tab structure are defined in [`FRONTEND_DESIGN_RULES.md`](./FRONTEND_DESIGN_RULES.md). Follow that file for all dashboard work.
- Vite React app; proxies `/api` and `/screenshots` to API in dev (`vite.config.ts`).
- Uses Syncfusion components; Vite config pre-bundles specific packages to avoid alias issues.
- Environment: `VITE_API_URL` provided at build/run; in dev compose, proxy handles `/api` so local fetches can use relative `/api/...` paths.
- **API Response Format**: All API endpoints return camelCase JSON (e.g., `startTime`, `endTime`, `groupId`). Frontend consumes camelCase directly.
- **UTC Time Parsing**: API returns ISO strings without 'Z' suffix. Frontend appends 'Z' before parsing to ensure UTC interpretation: `const utcString = dateStr.endsWith('Z') ? dateStr : dateStr + 'Z'; new Date(utcString);`. Display uses `toLocaleTimeString('de-DE')` for German format.
- Dev Container: When adding frontend deps, prefer `npm ci` and, if using named volumes, recreate dashboard `node_modules` volume so installs occur inside the container.
- Theming: All Syncfusion component CSS is imported centrally in `dashboard/src/main.tsx`. Theme conventions, component defaults, the full CSS import list, and Tailwind removal are documented in `FRONTEND_DESIGN_RULES.md`.
- Scheduler (appointments page): top bar includes Group and Academic Period selectors (Syncfusion DropDownList). Selecting a period calls `POST /api/academic_periods/active`, moves the calendar to todays month/day within the period year, and refreshes a right-aligned indicator row showing:
- Holidays present in the current view (count)
- Period label (display_name or name) with a badge indicating whether any holidays exist in that period (overlap check)
- Recurrence & holidays (latest):
- Backend stores holiday skips in `EventException` and emits `RecurrenceException` (EXDATE) for master events in `GET /api/events`. EXDATE tokens are formatted in RFC 5545 compact form (`yyyyMMddTHHmmssZ`) and correspond to each occurrence start time (UTC). Syncfusion uses these to exclude holiday instances reliably.
- Frontend lets Syncfusion handle all recurrence patterns natively (no client-side expansion). Scheduler field mappings include `recurrenceID`, `recurrenceRule`, and `recurrenceException` so series and edited occurrences are recognized correctly.
- Event deletion: All event types (single, single-in-series, entire series) are handled with custom dialogs. The frontend intercepts Syncfusion's built-in RecurrenceAlert and DeleteAlert popups to provide a unified, user-friendly deletion flow:
- Single (non-recurring) event: deleted directly after confirmation.
- Single occurrence of a recurring series: user can delete just that instance.
- Entire recurring series: user can delete all occurrences after a final custom confirmation dialog.
- Detached occurrences (edited/broken out): treated as single events.
- Single occurrence editing: Users can detach individual occurrences from recurring series. The frontend hooks `actionComplete`/`onActionCompleted` with `requestType='eventChanged'` to persist changes: it calls `POST /api/events/<id>/occurrences/<date>/detach` for single-occurrence edits and `PUT /api/events/<id>` for series or single events as appropriate. The backend creates `EventException` and a standalone `Event` without modifying the master beyond EXDATEs.
- UI: Events with `SkipHolidays` render a TentTree icon next to the main event icon. The custom recurrence icon in the header was removed; rely on Syncfusions native lower-right recurrence badge.
- Website & WebUntis: Both event types display a website. WebUntis reads its URL from the system `supplement_table_url` and does not provide a per-event URL field.
- Program info page (`dashboard/src/programminfo.tsx`):
- Loads data from `dashboard/public/program-info.json` (app name, version, build info, tech stack, changelog).
- Uses Syncfusion card classes (`e-card`, `e-card-header`, `e-card-title`, `e-card-content`) for consistent styling.
- Changelog is paginated with `PagerComponent` (from `@syncfusion/ej2-react-grids`), default page size 5; adjust `pageSize` or add a selector as needed.
- Groups page (`dashboard/src/infoscreen_groups.tsx`):
- Migrated to Syncfusion inputs and popups: Buttons, TextBox, DropDownList, Dialog; Kanban remains for drag/drop.
- Unified toast/dialog wording; replaced legacy alerts with toasts; spacing handled via inline styles to avoid Tailwind dependency.
- Header user menu (top-right):
- Shows current username and role; click opens a menu with "Passwort ändern" (lock icon), "Profil", and "Abmelden".
- Implemented with Syncfusion DropDownButton (`@syncfusion/ej2-react-splitbuttons`).
- "Passwort ändern": Opens self-service password change dialog (available to all authenticated users); requires current password verification, new password min 6 chars, must match confirm field; calls `PUT /api/auth/change-password`
- "Abmelden" navigates to `/logout`; the page invokes backend logout and redirects to `/login`.
- User management page (`dashboard/src/users.tsx`):
- Full CRUD interface for managing users (admin+ only in menu); accessible via "Benutzer" sidebar entry
- Syncfusion GridComponent: 20 per page (configurable), sortable columns (ID, username, role), custom action button template with role-based visibility
- Statistics cards: total users, active (non-deactivated), inactive (deactivated) counts
- Dialogs: Create (username/password/role/status), Edit (with self-edit protections), Password Reset (admin only, no current password required), Delete (superadmin only, self-check), Details (read-only audit info with formatted timestamps)
- Role badges: Color-coded display (user: gray, editor: blue, admin: green, superadmin: red)
- Audit information displayed: last login, password change, last failed login, deactivation timestamps and deactivating user
- Role-based permissions (enforced backend + frontend):
- Admin: can manage user/editor/admin roles (not superadmin); soft-deactivate only; cannot see/edit superadmin accounts
- Superadmin: can manage all roles including other superadmins; can permanently hard-delete users
- Security rules enforced: cannot change own role, cannot deactivate own account, cannot delete self, cannot reset own password via admin route (must use self-service)
- API client in `dashboard/src/apiUsers.ts` for all user operations (listUsers, getUser, createUser, updateUser, resetUserPassword, deleteUser)
- Menu visibility: "Benutzer" menu item only visible to admin+ (role-gated in App.tsx)
- Monitoring page (`dashboard/src/monitoring.tsx`):
- Superadmin-only dashboard for client monitoring and diagnostics; menu item is hidden for lower roles and the route redirects non-superadmins.
- Uses `GET /api/client-logs/monitoring-overview` for aggregated live status, `GET /api/client-logs/recent-errors` for system-wide errors, and `GET /api/client-logs/<uuid>/logs` for per-client details.
- Shows per-client status (`healthy`, `warning`, `critical`, `offline`) based on heartbeat freshness, process state, screen state, and recent log counts.
- Displays latest screenshot preview and active priority screenshot (`/screenshots/{uuid}/priority` when active), screenshot type badges, current process metadata, and recent ERROR/WARN activity.
- Uses adaptive refresh: normal interval in steady state, faster polling while `activePriorityScreenshots > 0`.
- Settings page (`dashboard/src/settings.tsx`):
- Structure: Syncfusion TabComponent with role-gated tabs
- 📅 Academic Calendar (all users)
- **🗂️ Perioden (first sub-tab)**: Full period lifecycle management (admin+)
- List non-archived periods with active/archived badges and action buttons
- Create: dialog for name, displayName, startDate, endDate, periodType with validation
- Edit: update name, displayName, dates, type (cannot edit archived)
- Activate: set as active (deactivates all others)
- Archive: soft-delete with blocker checks (blocks if active or has active recurrence)
- Restore: restore archived periods to inactive state
- Delete: hard-delete archived periods with blocker checks (blocks if linked events)
- Archive visibility: toggle to show/hide archived periods
- Blockers: display prevents action with clear list of reasons (linked events, active recurrence, active status)
- **📥 Ferienkalender: Import/Anzeige (second sub-tab)**: CSV/TXT holiday import plus manual holiday create/edit/delete scoped to the selected academic period; changing the period redraws the import/list body.
- Import summary surfaces inserted/updated/merged/skipped/conflict counts and detailed conflict lines.
- File selection uses Syncfusion-styled trigger button and visible selected filename state.
- Manual date inputs guide users with bidirectional start/end constraints and prefill behavior.
- 🖥️ Display & Clients (admin+)
- Default Settings: placeholders for heartbeat, screenshots, defaults
- Client Configuration: quick links to Clients and Groups pages
- 🎬 Media & Files (admin+)
- Upload Settings: placeholders for limits and types
- Conversion Status: placeholder for conversions overview
- 🗓️ Events (admin+)
- WebUntis / Vertretungsplan: system-wide supplement table URL with enable/disable, save, and preview; persists via `/api/system-settings/supplement-table`
- Presentations: general defaults for slideshow interval, page-progress, and auto-progress; persisted via `/api/system-settings` keys (`presentation_interval`, `presentation_page_progress`, `presentation_auto_progress`). These defaults are applied when creating new presentation events (the custom event modal reads them and falls back to per-event values when editing).
- Videos: system-wide defaults for `autoplay`, `loop`, `volume`, and `muted`; persisted via `/api/system-settings` keys (`video_autoplay`, `video_loop`, `video_volume`, `video_muted`). These defaults are applied when creating new video events (the custom event modal reads them and falls back to per-event values when editing).
- Other event types (website, message, other): placeholders for defaults
- ⚙️ System (superadmin)
- Organization Info and Advanced Configuration placeholders
- Role gating: Admin/Superadmin tabs are hidden if the user lacks permission; System is superadmin-only
- API clients use relative `/api/...` URLs so Vite dev proxy handles requests without CORS issues. The settings UI calls are centralized in `dashboard/src/apiSystemSettings.ts` (system settings) and `dashboard/src/apiAcademicPeriods.ts` (periods CRUD).
- Nested tabs: implemented as controlled components using `selectedItem` with stateful handlers to prevent sub-tab resets during updates.
- Academic periods API client (`dashboard/src/apiAcademicPeriods.ts`): provides type-safe camelCase accessors (listAcademicPeriods, getAcademicPeriod, createAcademicPeriod, updateAcademicPeriod, setActiveAcademicPeriod, archiveAcademicPeriod, restoreAcademicPeriod, getAcademicPeriodUsage, deleteAcademicPeriod).
- Dashboard page (`dashboard/src/dashboard.tsx`):
- Card-based overview of all Raumgruppen (room groups) with real-time status monitoring
- Global statistics: total infoscreens, online/offline counts, warning groups
- Filter buttons: All / Online / Offline / Warnings with dynamic counts
- Per-group cards show:
- Currently active event (title, type, date/time in local timezone)
- Health bar with online/offline ratio and color-coded status
- Expandable client list with last alive timestamps
- Bulk restart button for offline clients
- Uses Syncfusion ButtonComponent, ToastComponent, and card CSS classes
- Auto-refresh every 15 seconds; manual refresh button available
- "Nicht zugeordnet" group always appears last in sorted list
- Ressourcen page (`dashboard/src/ressourcen.tsx`):
- Timeline view showing all groups and their active events in parallel
- Uses Syncfusion ScheduleComponent with TimelineViews (day/week modes)
- Compact row display: 65px height per group, dynamically calculated total height
- Group ordering panel with drag up/down controls; order persisted to backend via `/api/groups/order`
- Filters out "Nicht zugeordnet" group from timeline display
- Fetches events per group for current date range; displays first active event per group
- Color-coded event bars using `getGroupColor()` from `groupColors.ts`
- Resource-based timeline: each group is a resource row, events mapped to `ResourceId`
- Real-time updates: loads events on mount and when view/date changes
- Custom CSS in `dashboard/src/ressourcen.css` for timeline styling and controls
- User dropdown technical notes:
- Dependencies: `@syncfusion/ej2-react-splitbuttons` and `@syncfusion/ej2-splitbuttons` must be installed.
- Vite: add both to `optimizeDeps.include` in `vite.config.ts` to avoid import-analysis errors.
- Dev containers: when `node_modules` is a named volume, recreate the dashboard node_modules volume after adding dependencies so `npm ci` runs inside the container.
Note: Syncfusion usage in the dashboard is already documented above; if a UI for conversion status/downloads is added later, link its routes and components here.
## Local development
- Compose: development is `docker-compose.yml` + `docker-compose.override.yml`.
- API (dev): `server/Dockerfile.dev` with debugpy on 5678, Flask app `wsgi:app` on :8000.
- Dashboard (dev): `dashboard/Dockerfile.dev` exposes :5173 and waits for API via `dashboard/wait-for-backend.sh`.
- Mosquitto: allows anonymous in dev; WebSocket on :9001.
- Common env vars: `DB_CONN`, `DB_USER`, `DB_PASSWORD`, `DB_HOST=db`, `DB_NAME`, `ENV`, `MQTT_USER`, `MQTT_PASSWORD`.
- Alembic: prod compose runs `alembic ... upgrade head` and `server/init_defaults.py` before gunicorn.
- Local dev: prefer `python server/initialize_database.py` for one-shot setup (migrations + defaults + academic periods).
- Defaults: `server/init_defaults.py` seeds initial system settings like `supplement_table_url` and `supplement_table_enabled` if missing.
- `server/init_academic_periods.py` remains available to (re)seed school years.
## Production
- `docker-compose.prod.yml` uses prebuilt images (`ghcr.io/robbstarkaustria/*`).
- Nginx serves dashboard and proxies API; TLS certs expected in `certs/` and mounted to `/etc/nginx/certs`.
## Environment variables (reference)
- DB_CONN — Preferred DB URL for services. Example: `mysql+pymysql://${DB_USER}:${DB_PASSWORD}@db/${DB_NAME}`
- DB_USER, DB_PASSWORD, DB_NAME, DB_HOST — Used to assemble DB_CONN in dev if missing; inside containers `DB_HOST=db`.
- ENV — `development` or `production`; in development, `server/database.py` loads `.env`.
- MQTT_BROKER_HOST, MQTT_BROKER_PORT — Defaults `mqtt` and `1883`; MQTT_USER/MQTT_PASSWORD optional (dev often anonymous per Mosquitto config).
- VITE_API_URL — Dashboard build-time base URL (prod); in dev the Vite proxy serves `/api` to `server:8000`.
- HEARTBEAT_GRACE_PERIOD_DEV / HEARTBEAT_GRACE_PERIOD_PROD — Groups "alive" window (defaults 180s dev / 170s prod). Clients send heartbeats every ~65s; grace periods allow 2 missed heartbeats plus safety margin.
- REFRESH_SECONDS — Optional scheduler republish interval; `0` disables periodic refresh.
- PRIORITY_SCREENSHOT_TTL_SECONDS — Optional monitoring priority window in seconds (default `120`); controls when event screenshots are considered active priority.
## Conventions & gotchas
- **Datetime Handling**:
- Always compare datetimes in UTC; some DB values may be naive—normalize before comparing (see `routes/events.py`).
- Database stores timestamps in UTC (naive datetimes are normalized to UTC by backend)
- API returns ISO strings **without** 'Z' suffix: `"2025-11-27T20:03:00"`
- Frontend **must** append 'Z' before parsing: `const utcStr = dateStr.endsWith('Z') ? dateStr : dateStr + 'Z'; new Date(utcStr);`
- Display in local timezone using `toLocaleTimeString('de-DE', { hour: '2-digit', minute: '2-digit' })`
- When sending to API, use `date.toISOString()` which includes 'Z' and is UTC
- **JSON Naming Convention**:
- Backend uses snake_case internally (Python convention)
- API returns camelCase JSON (web standard): `startTime`, `endTime`, `groupId`, etc.
- Use `dict_to_camel_case()` from `server/serializers.py` before `jsonify()`
- Frontend consumes camelCase directly; Syncfusion scheduler maintains internal PascalCase with field mappings
- Scheduler enforces UTC comparisons and normalizes naive timestamps. It publishes only currently active events and clears retained topics for groups with no active events. It also queries a future window (default: 7 days) and expands recurring events using RFC 5545 rules. Event exceptions are respected. Logging is concise and conversion lookups are cached.
- Use retained MQTT messages for state that clients must recover after reconnect (events per group, client group_id).
- Clients should parse `event_type` and then read the corresponding nested payload (`presentation`, `website`, `video`, etc.). `website` and `webuntis` use the same nested `website` payload with `type: browser` and a `url`. Video events include `autoplay`, `loop`, `volume`, and `muted`.
- In-container DB host is `db`; do not use `localhost` inside services.
- No separate dev vs prod secret conventions: use the same env var keys across environments (e.g., `DB_CONN`, `MQTT_USER`, `MQTT_PASSWORD`).
- When adding a new route:
1) Create a Blueprint in `server/routes/...`,
2) Register it in `server/wsgi.py`,
3) Manage `Session()` lifecycle,
4) Return JSON-safe values (serialize enums and datetimes), and
5) Use `dict_to_camel_case()` for camelCase JSON responses
Docs maintenance guardrails (solo-friendly): Update this file alongside code changes (services/MQTT/API/UTC/env). Keep it concise (2050 lines per section). Never include secrets.
- When extending media types, update `MediaType` and any logic in `eventmedia` and dashboard that depends on it.
- Academic periods: Events/media can be optionally associated with periods for educational organization. Only one period should be active at a time (`is_active=True`).
- Initialization scripts: legacy DB init scripts were removed; use Alembic and `initialize_database.py` going forward.
### Recurrence & holidays: conventions
- Do not pre-expand recurrences on the backend. Always send master events with `RecurrenceRule` + `RecurrenceException`.
- Ensure EXDATE tokens are RFC 5545 timestamps (`yyyyMMddTHHmmssZ`) matching the occurrence start time (UTC) so Syncfusion can exclude them natively.
- School holidays are scoped by `academic_period_id`; holiday imports and queries should use the relevant academic period rather than treating holiday rows as global.
- Holiday write operations (manual/import) must validate date ranges against the selected academic period.
- Overlap policy: same normalized `name+region` overlaps (including adjacent ranges) are merged; overlaps with different identity are conflicts (manual blocked, import skipped with details).
- When `skip_holidays` or recurrence changes, regenerate `EventException` rows so `RecurrenceException` stays in sync, using the event's `academic_period_id` holidays (or only unassigned holidays for legacy events without a period).
- Single occurrence detach: Use `POST /api/events/<id>/occurrences/<date>/detach` to create standalone events and add EXDATE entries without modifying master events. The frontend persists edits via `actionComplete` (`requestType='eventChanged'`).
## Quick examples
- Add client description persists to DB and publishes group via MQTT: see `PUT /api/clients/<uuid>/description` in `routes/clients.py`.
- Bulk group assignment emits retained messages for each client: `PUT /api/clients/group`.
- Listener heartbeat path: `infoscreen/<uuid>/heartbeat` → sets `clients.last_alive` and captures process health data.
- Client monitoring flow: Client publishes to `infoscreen/{uuid}/logs/error` and `infoscreen/{uuid}/health` → listener stores/updates monitoring state → API serves `/api/client-logs/monitoring-overview`, `/api/client-logs/recent-errors`, and `/api/client-logs/<uuid>/logs` → superadmin monitoring dashboard displays live status.
## Scheduler payloads: presentation extras
- Presentation event payloads now include `page_progress` and `auto_progress` in addition to `slide_interval` and media files. These are sourced from per-event fields in the database (with system defaults applied on event creation).
## Scheduler payloads: website & webuntis
- For both `website` and `webuntis`, the scheduler emits a nested `website` object:
- `{ "type": "browser", "url": "https://..." }`
- The `event_type` remains `website` or `webuntis`. Clients should treat both identically for rendering.
- The WebUntis URL is set at event creation by reading the system `supplement_table_url`.
Questions or unclear areas? Tell us if you need: exact devcontainer debugging steps, stricter Alembic workflow, or a seed dataset beyond `init_defaults.py`.
## Academic Periods System
- **Purpose**: Organize events and media by educational cycles (school years, semesters, trimesters) with full lifecycle management.
- **Design**: Fully backward compatible - existing events/media continue to work without period assignment.
- **Lifecycle States**:
- Active: exactly one period at a time (all others deactivated when activated)
- Inactive: saved period, not currently active
- Archived: soft-deleted; hidden from normal list; can be restored
- Deleted: hard-deleted; permanent removal (only when no linked events exist and no active recurrence)
- **Archive Rules**: Cannot archive active periods or periods with recurring master events that have current/future occurrences
- **Delete Rules**: Only archived inactive periods can be hard-deleted; blocked if linked events exist
- **Validation Rules**:
- Name: required, trimmed, unique among non-archived periods
- Dates: startDate ≤ endDate
- Type: schuljahr, semester, or trimester
- Overlaps: disallowed within same periodType (allowed across types)
- **Recurrence Spillover Detection**: Archive/delete blocked if recurring master events assigned to period still generate current/future occurrences
- **Model Fields**: `id`, `name`, `display_name`, `start_date`, `end_date`, `period_type`, `is_active`, `is_archived`, `archived_at`, `archived_by`, `created_at`, `updated_at`
- **Events/Media Association**: Both `Event` and `EventMedia` have optional `academic_period_id` FK for organizational grouping
- **UI Integration** (`dashboard/src/settings.tsx` > 🗂️ Perioden):
- List with badges (Active/Archived)
- Create/Edit dialogs with validation
- Activate, Archive, Restore, Delete actions with blocker preflight checks
- Archive visibility toggle to show/hide retired periods
- Error dialogs showing exact blockers (linked events, active recurrence, active status)
## Changelog Style Guide (Program info)
- Source: `dashboard/public/program-info.json`; newest entry first
- Fields per release: `version`, `date` (YYYY-MM-DD), `changes` (array of short bullets)
- Tone: concise, user-facing; German wording; area prefixes allowed (e.g., “UI: …”, “API: …”)
- Categories via emoji or words: Added (🆕/✨), Changed (🛠️), Fixed (✅/🐛), Removed (🗑️), Security (🔒), Deprecated (⚠️)
- Breaking changes must be prefixed with `BREAKING:`
- Keep ≤ 810 bullets; summarize or group micro-changes
- JSON hygiene: valid JSON, no trailing commas, dont edit historical entries except typos
## Versioning Convention (Tech vs UI)
- Use one unified app version across technical and user-facing release notes.
- `dashboard/public/program-info.json` is user-facing and should list only user-visible changes.
- `TECH-CHANGELOG.md` can include deeper technical details for the same released version.
- If server/infrastructure work is implemented but not yet released or not user-visible, document it under the latest released section as:
- `Backend technical work (post-release notes; no version bump)`
- Do not create a new version header in `TECH-CHANGELOG.md` for internal milestones alone.
- Bump version numbers when a release is actually cut/deployed (or when user-facing release notes are published), not for intermediate backend-only steps.
- When UI integration lands later, include the user-visible part in the next release version and reference prior post-release technical groundwork when useful.

View File

@@ -36,6 +36,27 @@ Update the instructions in the same commit as your change whenever you:
- Include concrete examples from this repo when describing patterns (e.g., which route shows enum serialization). - Include concrete examples from this repo when describing patterns (e.g., which route shows enum serialization).
- Never include secrets or real tokens; show only variable names and example formats. - Never include secrets or real tokens; show only variable names and example formats.
## Scope boundaries (important)
To avoid turning `.github/copilot-instructions.md` into a shadow README/changelog, keep clear boundaries:
- `.github/copilot-instructions.md`: quick operational brief for agents (architecture snapshot, non-negotiable conventions, key paths, critical contracts).
- `README.md`: project entrypoint and documentation navigation.
- `TECH-CHANGELOG.md` and `DEV-CHANGELOG.md`: change history and release/development notes.
- Feature contracts/specs: dedicated files (for example `MQTT_EVENT_PAYLOAD_GUIDE.md`, `TV_POWER_INTENT_SERVER_CONTRACT_V1.md`).
Do not place long historical sections, release summaries, or full endpoint catalogs in `.github/copilot-instructions.md`.
## Size and quality guardrails
- Target size for `.github/copilot-instructions.md`: about 120-220 lines.
- If a new section exceeds ~10 lines, prefer linking to an existing canonical doc instead.
- Keep each section focused on actionability for coding agents.
- Remove duplicate rules if already stated in another section.
- Use concise bullets over long prose blocks.
Quick pre-commit check:
- Is this content a rule/pattern needed during coding now?
- If it is historical context, move it to changelog/archive docs.
- If it is deep reference material, move it to the canonical feature doc and link it.
## Solo-friendly workflow ## Solo-friendly workflow
- Update docs in the same commit as your change: - Update docs in the same commit as your change:
- Code changed → docs changed (copilot-instructions, `.env.example`, `deployment.md` as needed) - Code changed → docs changed (copilot-instructions, `.env.example`, `deployment.md` as needed)
@@ -101,4 +122,5 @@ exit 0 # warn only; do not block commit
- Dev/Prod docs: `deployment.md`, `.env.example` - Dev/Prod docs: `deployment.md`, `.env.example`
## Documentation sync log ## Documentation sync log
- 2026-03-24: Synced docs for completed monitoring rollout and presentation flag persistence fix (`page_progress` / `auto_progress`). Updated `.github/copilot-instructions.md`, `README.md`, `TECH-CHANGELOG.md`, `DEV-CHANGELOG.md`, and `CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md` without a user-version bump. - 2026-03-24: Synced docs for completed monitoring rollout and presentation flag persistence fix (`page_progress` / `auto_progress`). Updated `.github/copilot-instructions.md`, `README.md`, `TECH-CHANGELOG.md`, `DEV-CHANGELOG.md`, and `docs/archive/CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md` without a user-version bump.
- 2026-04-01: Synced docs for TV power intent Phase 1 implementation and contract consistency. Updated `.github/copilot-instructions.md`, `MQTT_EVENT_PAYLOAD_GUIDE.md`, `docs/archive/TV_POWER_PHASE_1_SERVER_HANDOFF.md`, `TECH-CHANGELOG.md`, and `DEV-CHANGELOG.md` to match scheduler behavior (`infoscreen/groups/{group_id}/power/intent`, `schema_version: "1.0"`, transition + heartbeat semantics, poll-based expiry).

View File

@@ -891,7 +891,7 @@ Reset: After 5 minutes of successful operation
- Base URL: `http://192.168.43.201:8000` - Base URL: `http://192.168.43.201:8000`
- Health check: `GET /health` - Health check: `GET /health`
- Test logs: `GET /api/client-logs/test` (no auth) - Test logs: `GET /api/client-logs/test` (no auth)
- Full API docs: See `CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md` on server - Full API docs: See `docs/archive/CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md` on server
**MQTT Broker:** **MQTT Broker:**
- Host: `192.168.43.201` - Host: `192.168.43.201`
@@ -974,6 +974,6 @@ watchdog.monitor_loop()
**END OF SPECIFICATION** **END OF SPECIFICATION**
Questions? Refer to: Questions? Refer to:
- `CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md` (server repo) - `docs/archive/CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md` (server repo)
- Server API: `http://192.168.43.201:8000/api/client-logs/test` - Server API: `http://192.168.43.201:8000/api/client-logs/test`
- MQTT test: `mosquitto_sub -h 192.168.43.201 -t infoscreen/#` - MQTT test: `mosquitto_sub -h 192.168.43.201 -t infoscreen/#`

View File

@@ -5,6 +5,8 @@ This changelog tracks all changes made in the development workspace, including i
--- ---
## Unreleased (development workspace) ## Unreleased (development workspace)
- TV power intent (Phase 1): Scheduler publishes retained QoS1 group-level intents to `infoscreen/groups/{group_id}/power/intent` with transition+heartbeat semantics, startup/reconnect republish, and poll-based expiry (`max(3 × poll_interval_sec, 90s)`).
- TV power validation: Added unit/integration/canary coverage in `scheduler/test_power_intent_utils.py`, `scheduler/test_power_intent_scheduler.py`, and `test_power_intent_canary.py`.
- Monitoring system completion: End-to-end monitoring pipeline is active (MQTT logs/health → listener persistence → monitoring APIs → superadmin dashboard). - Monitoring system completion: End-to-end monitoring pipeline is active (MQTT logs/health → listener persistence → monitoring APIs → superadmin dashboard).
- Monitoring API: Added/active endpoints `GET /api/client-logs/monitoring-overview` and `GET /api/client-logs/recent-errors`; per-client logs via `GET /api/client-logs/<uuid>/logs`. - Monitoring API: Added/active endpoints `GET /api/client-logs/monitoring-overview` and `GET /api/client-logs/recent-errors`; per-client logs via `GET /api/client-logs/<uuid>/logs`.
- Dashboard monitoring UI: Superadmin monitoring page is integrated and displays client health status, screenshots, process metadata, and recent error activity. - Dashboard monitoring UI: Superadmin monitoring page is integrated and displays client health status, screenshots, process metadata, and recent error activity.

View File

@@ -31,7 +31,7 @@ Example payload:
```json ```json
{ {
"schema_version": "tv-power-intent.v1", "schema_version": "1.0",
"intent_id": "9cf26d9b-87a3-42f1-8446-e90bb6f6ce63", "intent_id": "9cf26d9b-87a3-42f1-8446-e90bb6f6ce63",
"group_id": 12, "group_id": 12,
"desired_state": "on", "desired_state": "on",
@@ -39,7 +39,9 @@ Example payload:
"issued_at": "2026-03-31T10:15:30Z", "issued_at": "2026-03-31T10:15:30Z",
"expires_at": "2026-03-31T10:17:00Z", "expires_at": "2026-03-31T10:17:00Z",
"poll_interval_sec": 30, "poll_interval_sec": 30,
"source": "scheduler" "active_event_ids": [148],
"event_window_start": "2026-03-31T10:15:00Z",
"event_window_end": "2026-03-31T11:00:00Z"
} }
``` ```
@@ -150,7 +152,7 @@ Every event includes these common fields:
} }
``` ```
**Note**: WebUntis events use the same payload structure as website events. The URL is fetched from system settings (`webuntis_url`) rather than being specified per-event. Clients treat `webuntis` and `website` event types identically—both display a website. **Note**: WebUntis events use the same payload structure as website events. The URL is fetched from system settings (`supplement_table_url`) rather than being specified per-event. Clients treat `webuntis` and `website` event types identically—both display a website.
#### Video Events #### Video Events

View File

@@ -149,13 +149,11 @@ Rollout strategy (Phase 1):
- [SUPERADMIN_SETUP.md](SUPERADMIN_SETUP.md) - [SUPERADMIN_SETUP.md](SUPERADMIN_SETUP.md)
### Monitoring, Screenshots, Health ### Monitoring, Screenshots, Health
- [CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md](CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md)
- [CLIENT_MONITORING_SPECIFICATION.md](CLIENT_MONITORING_SPECIFICATION.md) - [CLIENT_MONITORING_SPECIFICATION.md](CLIENT_MONITORING_SPECIFICATION.md)
- [SCREENSHOT_IMPLEMENTATION.md](SCREENSHOT_IMPLEMENTATION.md) - [SCREENSHOT_IMPLEMENTATION.md](SCREENSHOT_IMPLEMENTATION.md)
### MQTT & Payloads ### MQTT & Payloads
- [MQTT_EVENT_PAYLOAD_GUIDE.md](MQTT_EVENT_PAYLOAD_GUIDE.md) - [MQTT_EVENT_PAYLOAD_GUIDE.md](MQTT_EVENT_PAYLOAD_GUIDE.md)
- [MQTT_PAYLOAD_MIGRATION_GUIDE.md](MQTT_PAYLOAD_MIGRATION_GUIDE.md)
### Events, Calendar, WebUntis ### Events, Calendar, WebUntis
- [WEBUNTIS_EVENT_IMPLEMENTATION.md](WEBUNTIS_EVENT_IMPLEMENTATION.md) - [WEBUNTIS_EVENT_IMPLEMENTATION.md](WEBUNTIS_EVENT_IMPLEMENTATION.md)
@@ -167,9 +165,17 @@ Rollout strategy (Phase 1):
- [docs/archive/CLEANUP_SUMMARY.md](docs/archive/CLEANUP_SUMMARY.md) - [docs/archive/CLEANUP_SUMMARY.md](docs/archive/CLEANUP_SUMMARY.md)
### Conversion / Media ### Conversion / Media
- [pptx_conversion_guide.md](pptx_conversion_guide.md)
- [pptx_conversion_guide_gotenberg.md](pptx_conversion_guide_gotenberg.md) - [pptx_conversion_guide_gotenberg.md](pptx_conversion_guide_gotenberg.md)
### Historical / Archived Docs
- [docs/archive/CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md](docs/archive/CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md) - completed implementation plan/history
- [docs/archive/MQTT_DASHBOARD_V1_TO_V2_MIGRATION.md](docs/archive/MQTT_DASHBOARD_V1_TO_V2_MIGRATION.md) - completed MQTT payload migration notes
- [docs/archive/PPTX_CONVERSION_LEGACY_APPROACH.md](docs/archive/PPTX_CONVERSION_LEGACY_APPROACH.md) - legacy conversion approach (superseded)
- [docs/archive/TV_POWER_PHASE_1_COORDINATION.md](docs/archive/TV_POWER_PHASE_1_COORDINATION.md) - TV power Phase 1 coordination tasklist
- [docs/archive/TV_POWER_PHASE_1_SERVER_HANDOFF.md](docs/archive/TV_POWER_PHASE_1_SERVER_HANDOFF.md) - TV power Phase 1 server handoff notes
- [docs/archive/TV_POWER_PHASE_1_CANARY_VALIDATION.md](docs/archive/TV_POWER_PHASE_1_CANARY_VALIDATION.md) - TV power Phase 1 canary validation checklist
- [docs/archive/TV_POWER_PHASE_1_IMPLEMENTATION_CHECKLIST.md](docs/archive/TV_POWER_PHASE_1_IMPLEMENTATION_CHECKLIST.md) - TV power Phase 1 implementation checklist
### Frontend ### Frontend
- [FRONTEND_DESIGN_RULES.md](FRONTEND_DESIGN_RULES.md) - [FRONTEND_DESIGN_RULES.md](FRONTEND_DESIGN_RULES.md)
- [dashboard/README.md](dashboard/README.md) - [dashboard/README.md](dashboard/README.md)

View File

@@ -6,6 +6,13 @@
This changelog documents technical and developer-relevant changes included in public releases. For development workspace changes, see DEV-CHANGELOG.md. Not all changes here are reflected in the user-facing changelog (`program-info.json`), and not all UI/feature changes are repeated here. Some changes (e.g., backend refactoring, API adjustments, infrastructure, developer tooling, or internal logic) may only appear in TECH-CHANGELOG.md. For UI/feature changes, see `dashboard/public/program-info.json`. This changelog documents technical and developer-relevant changes included in public releases. For development workspace changes, see DEV-CHANGELOG.md. Not all changes here are reflected in the user-facing changelog (`program-info.json`), and not all UI/feature changes are repeated here. Some changes (e.g., backend refactoring, API adjustments, infrastructure, developer tooling, or internal logic) may only appear in TECH-CHANGELOG.md. For UI/feature changes, see `dashboard/public/program-info.json`.
## 2026.1.0-alpha.15 (2026-03-31) ## 2026.1.0-alpha.15 (2026-03-31)
- 🔌 **TV Power Intent Phase 1 (server-side)**:
- Scheduler now publishes retained QoS1 group-level intents to `infoscreen/groups/{group_id}/power/intent`.
- Implemented deterministic intent computation helpers in `scheduler/db_utils.py` (`compute_group_power_intent_basis`, `build_group_power_intent_body`, `compute_group_power_intent_fingerprint`).
- Implemented transition + heartbeat semantics in `scheduler/scheduler.py`: stable `intent_id` on heartbeat; new `intent_id` only on semantic transition.
- Added startup publish and MQTT reconnect republish behavior for retained intent continuity.
- Added poll-based expiry rule: `expires_at = issued_at + max(3 × poll_interval_sec, 90s)`.
- Added scheduler tests and canary validation artifacts: `scheduler/test_power_intent_utils.py`, `scheduler/test_power_intent_scheduler.py`, `test_power_intent_canary.py`, and `TV_POWER_CANARY_VALIDATION_CHECKLIST.md`.
- 🗃️ **Holiday data model scoping to academic periods**: - 🗃️ **Holiday data model scoping to academic periods**:
- Added period scoping for holidays via `SchoolHoliday.academic_period_id` (FK to academic periods) in `models/models.py`. - Added period scoping for holidays via `SchoolHoliday.academic_period_id` (FK to academic periods) in `models/models.py`.
- Added Alembic migration `f3c4d5e6a7b8_scope_school_holidays_to_academic_.py` to introduce FK/index/constraint updates for period-aware holiday storage. - Added Alembic migration `f3c4d5e6a7b8_scope_school_holidays_to_academic_.py` to introduce FK/index/constraint updates for period-aware holiday storage.

View File

@@ -1,83 +0,0 @@
# Server Handoff: TV Power Coordination
## Purpose
Implement server-side MQTT power intent publishing so clients can keep TVs on across adjacent events and power off safely after schedules end.
## Source of Truth
- Shared full plan: TV_POWER_COORDINATION_TASKLIST.md
## Scope (Server Team)
- Scheduler-to-intent mapping
- MQTT publishing semantics (retain, QoS, expiry)
- Conflict handling (group vs client)
- Observability for intent lifecycle
## MQTT Contract (Server Responsibilities)
### Topics
- Primary (per-client): infoscreen/{client_id}/power/intent
- Optional (group-level): infoscreen/groups/{group_id}/power/intent
### Delivery Semantics
- QoS: 1
- retained: true
- Always publish UTC timestamps (ISO 8601 with Z)
### Intent Payload (v1)
```json
{
"schema_version": "1.0",
"intent_id": "uuid-or-monotonic-id",
"issued_at": "2026-03-31T12:00:00Z",
"expires_at": "2026-03-31T12:10:00Z",
"target": {
"client_id": "optional-if-group-topic",
"group_id": "optional"
},
"power": {
"desired_state": "on",
"reason": "event_window_active",
"grace_seconds": 30
},
"event_window": {
"start": "2026-03-31T12:00:00Z",
"end": "2026-03-31T13:00:00Z"
}
}
```
## Required Behavior
### Adjacent/Overlapping Events
- Never publish an intermediate off intent when windows are contiguous/overlapping.
- Maintain continuous desired_state=on coverage across adjacent windows.
### Reconnect/Restart
- On scheduler restart, republish effective retained intent.
- On event edits/cancellations, replace retained intent with a fresh intent_id.
### Conflict Policy
- If both group and client intent exist: per-client overrides group.
### Expiry Safety
- expires_at must be set for every intent.
- Server should avoid publishing already-expired intents.
## Implementation Tasks
1. Add scheduler mapping layer that computes effective desired_state per client timeline.
2. Add intent publisher with retained QoS1 delivery.
3. Generate unique intent_id for each semantic transition.
4. Emit issued_at/expires_at and event_window consistently in UTC.
5. Add group-vs-client precedence logic.
6. Add logs/metrics for publish success, retained payload age, and transition count.
7. Add integration tests for adjacent events and reconnect replay.
## Acceptance Criteria
1. Adjacent events do not create OFF gap intents.
2. Fresh client receives retained intent after reconnect and gets correct desired state.
3. Intent payloads are schema-valid, UTC-formatted, and include expiry.
4. Publish logs and metrics allow intent timeline reconstruction.
## Operational Notes
- Keep intent publishing idempotent and deterministic.
- Preserve backward compatibility while clients run in hybrid mode.

View File

@@ -16,7 +16,7 @@ Manual verification checklist for Phase-1 server-side group-level power-intent p
Instructions: Instructions:
1. Subscribe to `infoscreen/groups/1/power/intent` (canary group, QoS 1) 1. Subscribe to `infoscreen/groups/1/power/intent` (canary group, QoS 1)
2. Verify received payload contains: 2. Verify received payload contains:
- `schema_version: "v1"` - `schema_version: "1.0"`
- `group_id: 1` - `group_id: 1`
- `desired_state: "on"` or `"off"` (string) - `desired_state: "on"` or `"off"` (string)
- `reason: "active_event"` or `"no_active_event"` (string) - `reason: "active_event"` or `"no_active_event"` (string)
@@ -24,6 +24,9 @@ Instructions:
- `issued_at: "2026-03-31T14:22:15Z"` (ISO 8601 with Z suffix) - `issued_at: "2026-03-31T14:22:15Z"` (ISO 8601 with Z suffix)
- `expires_at: "2026-03-31T14:24:00Z"` (ISO 8601 with Z suffix, always > issued_at) - `expires_at: "2026-03-31T14:24:00Z"` (ISO 8601 with Z suffix, always > issued_at)
- `poll_interval_sec: 30` (integer, matches scheduler poll interval) - `poll_interval_sec: 30` (integer, matches scheduler poll interval)
- `active_event_ids: [...]` (array; empty when off)
- `event_window_start: "...Z"` or `null`
- `event_window_end: "...Z"` or `null`
**Pass criteria**: All fields present, correct types and formats, no extra/malformed fields. **Pass criteria**: All fields present, correct types and formats, no extra/malformed fields.

View File

@@ -15,7 +15,7 @@ Prevent unintended TV power-off during adjacent events while enabling coordinate
## Server PR-1 Pointer ## Server PR-1 Pointer
- For the strict, agreed server-first implementation path, use: - For the strict, agreed server-first implementation path, use:
- `TV_POWER_SERVER_PR1_IMPLEMENTATION_CHECKLIST.md` - `TV_POWER_PHASE_1_IMPLEMENTATION_CHECKLIST.md`
- Treat that checklist as the execution source of truth for Phase 1. - Treat that checklist as the execution source of truth for Phase 1.
--- ---

View File

@@ -172,8 +172,8 @@ All PR-1 server-side items are complete. Below is a summary of deliverables:
- **MQTT_EVENT_PAYLOAD_GUIDE.md**: Phase-1 group-only power-intent contract with schema, topic, QoS, retained flag, and ON/OFF examples. - **MQTT_EVENT_PAYLOAD_GUIDE.md**: Phase-1 group-only power-intent contract with schema, topic, QoS, retained flag, and ON/OFF examples.
- **README.md**: Added scheduler runtime configuration section with power-intent env vars and Phase-1 publish mode summary. - **README.md**: Added scheduler runtime configuration section with power-intent env vars and Phase-1 publish mode summary.
- **AI-INSTRUCTIONS-MAINTENANCE.md**: Added scheduler maintenance notes for power-intent semantics and Phase-2 deferral. - **AI-INSTRUCTIONS-MAINTENANCE.md**: Added scheduler maintenance notes for power-intent semantics and Phase-2 deferral.
- **TV_POWER_CANARY_VALIDATION_CHECKLIST.md**: 10-scenario manual validation matrix for operators. - **TV_POWER_PHASE_1_CANARY_VALIDATION.md**: 10-scenario manual validation matrix for operators.
- **TV_POWER_SERVER_PR1_IMPLEMENTATION_CHECKLIST.md**: This file; source of truth for PR-1 scope and acceptance criteria. - **TV_POWER_PHASE_1_IMPLEMENTATION_CHECKLIST.md**: This file; source of truth for PR-1 scope and acceptance criteria.
### Validation Artifacts ### Validation Artifacts
- **test_power_intent_canary.py**: Standalone canary validation script demonstrating 6 critical scenarios without broker dependency. All scenarios pass. - **test_power_intent_canary.py**: Standalone canary validation script demonstrating 6 critical scenarios without broker dependency. All scenarios pass.
@@ -195,5 +195,5 @@ All PR-1 server-side items are complete. Below is a summary of deliverables:
### Next Phase ### Next Phase
- Phase 2 (deferred): Per-client override intent, client state acknowledgments, listener persistence of state - Phase 2 (deferred): Per-client override intent, client state acknowledgments, listener persistence of state
- Canary rollout strategy documented in `TV_POWER_CANARY_VALIDATION_CHECKLIST.md` - Canary rollout strategy documented in `TV_POWER_PHASE_1_CANARY_VALIDATION.md`

View File

@@ -0,0 +1,56 @@
# Server Handoff: TV Power Coordination
## Status
Server PR-1 is implemented and merged (Phase 1).
## Source of Truth
- Contract: TV_POWER_INTENT_SERVER_CONTRACT_V1.md
- Implementation: scheduler/scheduler.py and scheduler/db_utils.py
- Validation checklist: TV_POWER_PHASE_1_CANARY_VALIDATION.md
## Active Phase 1 Scope
- Topic: infoscreen/groups/{group_id}/power/intent
- QoS: 1
- Retained: true
- Scope: group-level only
- Per-client intent/state topics: deferred to Phase 2
## Publish Semantics (Implemented)
- Semantic transition (`desired_state` or `reason` changed): new `intent_id` and immediate publish
- Heartbeat (no semantic change): same `intent_id`, refreshed `issued_at` and `expires_at`
- Scheduler startup: immediate publish before first poll wait
- MQTT reconnect: immediate retained republish of cached intents
## Payload Contract (Phase 1)
```json
{
"schema_version": "1.0",
"intent_id": "uuid4",
"group_id": 12,
"desired_state": "on",
"reason": "active_event",
"issued_at": "2026-04-01T06:00:03.496Z",
"expires_at": "2026-04-01T06:01:33.496Z",
"poll_interval_sec": 15,
"active_event_ids": [148],
"event_window_start": "2026-04-01T06:00:00Z",
"event_window_end": "2026-04-01T07:00:00Z"
}
```
Expiry rule:
- expires_at = issued_at + max(3 x poll_interval_sec, 90 seconds)
## Operational Notes
- Adjacent/overlapping events are merged into one active coverage window; no OFF blip at boundaries.
- Feature flag defaults are safe for rollout:
- POWER_INTENT_PUBLISH_ENABLED=false
- POWER_INTENT_HEARTBEAT_ENABLED=true
- POWER_INTENT_EXPIRY_MULTIPLIER=3
- POWER_INTENT_MIN_EXPIRY_SECONDS=90
- Keep this handoff concise and defer full details to the stable contract document.
## Phase 2 (Deferred)
- Per-client override topic: infoscreen/{client_uuid}/power/intent
- Client power state topic and acknowledgments
- Listener persistence of client-level power state