diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index a9cc4a2..06e811b 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -1,498 +1,102 @@ # Copilot instructions for infoscreen_2025 -# Purpose -These instructions tell Copilot Chat how to reason about this codebase. -Prefer explanations and refactors that align with these structures. - -Use this as your shared context when proposing changes. Keep edits minimal and match existing patterns referenced below. +## Purpose +This file is a concise, high-signal brief for coding agents. +It is not a changelog and not a full architecture handbook. ## TL;DR -Small multi-service digital signage app (Flask API, React dashboard, MQTT scheduler). Edit `server/` for API logic, `scheduler/` for event publishing, and `dashboard/` for UI. If you're asking Copilot for changes, prefer focused prompts that include the target file(s) and the desired behavior. +- Stack: Flask API + MariaDB, React/Vite dashboard, MQTT listener, scheduler, worker. +- Main areas: + - API logic in `server/` + - Scheduler logic in `scheduler/` + - UI logic in `dashboard/src/` +- Keep changes minimal, match existing patterns, and update docs in the same commit when behavior changes. -### How to ask Copilot -- "Add a new route `GET /api/events/summary` that returns counts per event_type — implement in `server/routes/events.py`." -- "Create an Alembic migration to add `duration` and `resolution` to `event_media` and update upload handler to populate them." -- "Refactor `scheduler/db_utils.py` to prefer precomputed EventMedia metadata and fall back to a HEAD probe." -- "Add an ffprobe-based worker that extracts duration/resolution/bitrate and stores them on `EventMedia`." +## Fast file map +- `scheduler/scheduler.py` - scheduler loop, MQTT event publishing, TV power intent publishing +- `scheduler/db_utils.py` - event formatting and power-intent helper logic +- `listener/listener.py` - discovery/heartbeat/log/screenshot MQTT consumption +- `server/routes/events.py` - event CRUD, recurrence handling, UTC normalization +- `server/routes/eventmedia.py` - file manager, media upload/stream endpoints +- `server/routes/groups.py` - group lifecycle, alive status, order persistence +- `server/routes/system_settings.py` - system settings CRUD and supplement-table endpoint +- `dashboard/src/settings.tsx` - settings UX and system-defaults integration +- `dashboard/src/components/CustomEventModal.tsx` - event creation/editing UX +- `dashboard/src/monitoring.tsx` - superadmin monitoring page +- `TV_POWER_INTENT_SERVER_CONTRACT_V1.md` - Phase 1 TV power contract -Keep docs synced with code. When you change services/MQTT/API/UTC/env or dev/prod run steps, update this file in the same commit (see `AI-INSTRUCTIONS-MAINTENANCE.md`). +## Service picture +- API: `server/` on `:8000` (health: `/health`) +- Dashboard: `dashboard/` (dev `:5173`, proxied API calls) +- MQTT broker: Mosquitto (`mosquitto/config/mosquitto.conf`) +- Listener: MQTT consumer that updates server-side state +- Scheduler: publishes active events and group-level TV power intents +- Nginx: routes `/api/*` and `/screenshots/*` to API, dashboard otherwise -### When not to change -- Avoid editing generated assets under `dashboard/dist/` and compiled bundles. Don't modify files produced by CI or Docker builds (unless intentionally updating build outputs). +## Non-negotiable conventions +- Datetime: + - Store/compare in UTC. + - API returns ISO strings without `Z` in many routes. + - Frontend must append `Z` before parsing if needed. +- JSON naming: + - Backend internals use snake_case. + - API responses use camelCase (via `server/serializers.py`). +- DB host in containers: `db` (not localhost). +- Never put secrets in docs. -### Contact / owner -- Primary maintainer: RobbStarkAustria (owner). For architecture questions, ping the repo owner or open an issue and tag `@RobbStarkAustria`. +## MQTT contracts +- Event list topic (retained): `infoscreen/events/{group_id}` +- Group assignment topic (retained): `infoscreen/{uuid}/group_id` +- Heartbeat topic: `infoscreen/{uuid}/heartbeat` +- Logs topic family: `infoscreen/{uuid}/logs/{error|warn|info}` +- Health topic: `infoscreen/{uuid}/health` +- Dashboard screenshot topic: `infoscreen/{uuid}/dashboard` +- TV power intent Phase 1 topic (retained, QoS1): `infoscreen/groups/{group_id}/power/intent` -### Important files (quick jump targets) -- `scheduler/db_utils.py` — event formatting and scheduler-facing logic -- `scheduler/scheduler.py` — scheduler main loop and MQTT publisher -- `server/routes/eventmedia.py` — file uploads, streaming endpoint -- `server/routes/events.py` — event CRUD and recurrence handling -- `server/routes/groups.py` — group management, alive status, display order persistence -- `dashboard/src/components/CustomEventModal.tsx` — event creation UI -- `dashboard/src/media.tsx` — FileManager / upload settings -- `dashboard/src/settings.tsx` — settings UI (nested tabs; system defaults for presentations and videos) -- `dashboard/src/ressourcen.tsx` — timeline view showing all groups' active events in parallel -- `dashboard/src/ressourcen.css` — timeline and resource view styling -- `dashboard/src/monitoring.tsx` — superadmin-only monitoring dashboard for client health, screenshots, and logs +TV power intent Phase 1 rules: +- Schema version is `"1.0"`. +- Group-only scope in Phase 1. +- Heartbeat publish keeps `intent_id`; semantic transition rotates `intent_id`. +- Expiry rule: `expires_at = issued_at + max(3 x poll_interval_sec, 90s)`. +- Canonical contract is `TV_POWER_INTENT_SERVER_CONTRACT_V1.md`. +## Backend patterns +- Routes in `server/routes/*`, registered in `server/wsgi.py`. +- Use one request-scoped DB session, commit on mutation, always close session. +- Keep enum/datetime serialization JSON-safe. +- Maintain UTC-safe comparisons in scheduler and routes. +- Keep recurrence handling backend-driven and consistent with exceptions. +## Frontend patterns +- Use Syncfusion-based patterns already present in dashboard. +- Keep API requests relative (`/api/...`) to use Vite proxy in dev. +- Respect `FRONTEND_DESIGN_RULES.md` for component and styling conventions. +- Keep role-gated UI behavior aligned with backend authorization. -## Big picture -- Multi-service app orchestrated by Docker Compose. - - API: Flask + SQLAlchemy (MariaDB), in `server/` exposed on :8000 (health: `/health`). - - Dashboard: React + Vite in `dashboard/`, dev on :5173, served via Nginx in prod. - - MQTT broker: Eclipse Mosquitto, config in `mosquitto/config/mosquitto.conf`. - - Listener: MQTT consumer handling discovery, heartbeats, and dashboard screenshot uploads in `listener/listener.py`. - - Scheduler: Publishes only currently active events (per group, at "now") to MQTT retained topics in `scheduler/scheduler.py`. It queries a future window (default: 7 days) to expand recurring events using RFC 5545 rules and applies event exceptions, but only publishes events that are active at the current time. When a group has no active events, the scheduler clears its retained topic by publishing an empty list. All time comparisons are UTC; any naive timestamps are normalized. Logging is concise; conversion lookups are cached and logged only once per media. - - Nginx: Reverse proxy routes `/api/*` and `/screenshots/*` to API; everything else to dashboard (`nginx.conf`). +## Environment variables (high-value) +- Scheduler: `POLL_INTERVAL_SECONDS`, `REFRESH_SECONDS` +- Power intent: `POWER_INTENT_PUBLISH_ENABLED`, `POWER_INTENT_HEARTBEAT_ENABLED`, `POWER_INTENT_EXPIRY_MULTIPLIER`, `POWER_INTENT_MIN_EXPIRY_SECONDS` +- Monitoring: `PRIORITY_SCREENSHOT_TTL_SECONDS` +- Core: `DB_CONN`, `DB_USER`, `DB_PASSWORD`, `DB_HOST`, `DB_NAME`, `ENV` - - Dev Container (hygiene): UI-only `Dev Containers` extension runs on host UI via `remote.extensionKind`; do not install it in-container. Dashboard installs use `npm ci`; shell aliases in `postStartCommand` are appended idempotently. +## Edit guardrails +- Do not edit generated assets in `dashboard/dist/`. +- Do not change CI/build outputs unless explicitly intended. +- Preserve existing API behavior unless task explicitly requires a change. +- Prefer links to canonical docs instead of embedding long historical notes here. -### Screenshot retention -- Screenshots sent via dashboard MQTT are stored in `server/screenshots/`. -- Screenshot payloads support `screenshot_type` with values `periodic`, `event_start`, `event_stop`. -- `periodic` is the normal heartbeat/dashboard screenshot path; `event_start` and `event_stop` are high-priority screenshots for monitoring. -- For each client, the API keeps `{uuid}.jpg` as latest and the last 20 timestamped screenshots (`{uuid}_..._{type}.jpg`), deleting older timestamped files automatically. -- For high-priority screenshots, the API additionally maintains `{uuid}_priority.jpg` and metadata in `{uuid}_meta.json` (`latest_screenshot_type`, `last_priority_*`). +## Documentation sync rule +When services/MQTT/API/UTC/env behavior changes: +1. Update this file (concise deltas only). +2. Update canonical docs where details live. +3. Update changelogs separately (`TECH-CHANGELOG.md`, `DEV-CHANGELOG.md`, `dashboard/public/program-info.json` as appropriate). - ## Recent changes since last commit - - ### Latest (March 2026) - - - **Monitoring System Completion (no version bump)**: - - End-to-end monitoring pipeline completed: MQTT logs/health → listener persistence → monitoring APIs → superadmin dashboard - - API now serves aggregated monitoring via `GET /api/client-logs/monitoring-overview` and system-wide recent errors via `GET /api/client-logs/recent-errors` - - Monitoring dashboard (`dashboard/src/monitoring.tsx`) is active and displays client health states, screenshots, process metadata, and recent log activity - - **Screenshot Priority Pipeline (no version bump)**: - - Listener forwards `screenshot_type` from MQTT screenshot/dashboard payloads to `POST /api/clients//screenshot`. - - API stores typed screenshots, tracks latest/priority metadata, and serves priority images via `GET /screenshots//priority`. - - Monitoring overview exposes screenshot priority state (`latestScreenshotType`, `priorityScreenshotType`, `priorityScreenshotReceivedAt`, `hasActivePriorityScreenshot`) and `summary.activePriorityScreenshots`. - - Monitoring UI shows screenshot type badges and switches to faster refresh while priority screenshots are active. - - **MQTT Dashboard Payload v2 Cutover (no version bump)**: - - Dashboard payload parsing in `listener/listener.py` is now v2-only (`message`, `content`, `runtime`, `metadata`). - - Legacy top-level dashboard fallback was removed after migration soak (`legacy_fallback=0`). - - Listener observability summarizes parser health using `v2_success` and `parse_failures` counters. - - **Presentation Flags Persistence Fix**: - - Fixed persistence for presentation `page_progress` and `auto_progress` to ensure values are reliably stored and returned across create/update paths and detached occurrences - - ### Earlier (January 2026) - - - **Ressourcen Page (Timeline View)**: - - New 'Ressourcen' page with parallel timeline view showing active events for all room groups - - Compact timeline display with adjustable row height (65px per group) - - Real-time view of currently running events with type, title, and time window - - Customizable group ordering with visual reordering panel (drag up/down buttons) - - Group order persisted via `GET/POST /api/groups/order` endpoints - - Color-coded event bars matching group theme - - Timeline modes: Day and Week views (day view by default) - - Dynamic height calculation based on number of groups - - Syncfusion ScheduleComponent with TimelineViews, Resize, and DragAndDrop support - - Files: `dashboard/src/ressourcen.tsx` (page), `dashboard/src/ressourcen.css` (styles) - - ### Earlier (November 2025) - - - **API Naming Convention Standardization (camelCase)**: - - Backend: Created `server/serializers.py` with `dict_to_camel_case()` utility for consistent JSON serialization - - Events API: `GET /api/events` and `GET /api/events/` now return camelCase fields (`id`, `subject`, `startTime`, `endTime`, `type`, `groupId`, etc.) instead of PascalCase - - Frontend: Dashboard and appointments page updated to consume camelCase API responses - - Appointments page maintains internal PascalCase for Syncfusion scheduler compatibility with automatic mapping from API responses - - **Breaking**: External API consumers must update field names from PascalCase to camelCase - - - **UTC Time Handling**: - - Database stores all timestamps in UTC (naive timestamps normalized by backend) - - API returns ISO strings without 'Z' suffix: `"2025-11-27T20:03:00"` - - Frontend: Dashboard and appointments automatically append 'Z' to parse as UTC and display in user's local timezone - - Time formatting functions use `toLocaleTimeString('de-DE')` for German locale display - - All time comparisons use UTC; `new Date().toISOString()` sends UTC back to API - - API returns ISO strings without `Z`; frontend must append `Z` before parsing to ensure UTC - - - **Dashboard Enhancements**: - - New card-based design for Raumgruppen (room groups) with Syncfusion components - - Global statistics summary: total infoscreens, online/offline counts, warning groups - - Filter buttons: All, Online, Offline, Warnings with dynamic counts - - Active event display per group: shows currently playing content with type icon, title, date, and time - - Health visualization with color-coded progress bars per group - - Expandable client details with last alive timestamps - - Bulk restart functionality for offline clients per group - - Manual refresh button with toast notifications - - 15-second auto-refresh interval - - ### Earlier changes - - - Scheduler: when formatting video events the scheduler now performs a best-effort HEAD probe of the streaming URL and includes basic metadata in the emitted payload (mime_type, size, accept_ranges). Placeholders for richer metadata (duration, resolution, bitrate, qualities, thumbnails, checksum) are included for later population by a background worker. - - Streaming endpoint: a range-capable streaming endpoint was added at `/api/eventmedia/stream//` that supports byte-range requests (206 Partial Content) to enable seeking from clients. - - Event model & API: `Event` gained video-related fields (`event_media_id`, `autoplay`, `loop`, `volume`) and the API accepts and persists these when creating/updating video events. - - Dashboard: UI updated to allow selecting uploaded videos for events and to specify autoplay/loop/volume. File upload settings were increased (maxFileSize raised) and the client now validates video duration (max 10 minutes) before upload. - - FileManager: uploads compute basic metadata and enqueue conversions for office formats as before; video uploads now surface size and are streamable via the new endpoint. - - - Event model & API (new): Added `muted` (Boolean) for video events; create/update and GET endpoints accept, persist, and return `muted` alongside `autoplay`, `loop`, and `volume`. - - Dashboard — Settings: Settings page refactored to nested tabs; added Events → Videos defaults (autoplay, loop, volume, mute) backed by system settings keys (`video_autoplay`, `video_loop`, `video_volume`, `video_muted`). - - Dashboard — Events UI: CustomEventModal now exposes per-event video `muted` and initializes all video fields from system defaults when creating a new event. - - Dashboard — Academic Calendar: Holiday management now uses a single “📥 Ferienkalender: Import/Anzeige” tab; admins select the target academic period first, and import/list content redraws for that period. - - Dashboard — Holiday Management Hardening: The same tab now supports manual holiday CRUD in addition to CSV/TXT import. Imports and manual saves validate date ranges against the selected academic period, prevent duplicates, auto-merge same normalized name+region overlaps (including adjacent ranges), and report conflicting overlaps. - - Note: these edits are intentionally backwards-compatible — if the probe fails, the scheduler still emits the stream URL and the client should fallback to a direct play attempt or request richer metadata when available. - - Backend rework notes (no version bump): - - Dev container hygiene: UI-only Remote Containers; reproducible dashboard installs (`npm ci`); idempotent shell aliases. - - Serialization consistency: snake_case internal → camelCase external via `server/serializers.py` for all JSON. - - UTC normalization across routes/scheduler; enums and datetimes serialize consistently. - -## Service boundaries & data flow -- Database connection string is passed as `DB_CONN` (mysql+pymysql) to Python services. - - API builds its engine in `server/database.py` (loads `.env` only in development). - - Listener also creates its own engine for writes to `clients`. - - Scheduler queries a future window (default: 7 days) to expand recurring events using RFC 5545 rules, applies event exceptions (skipped dates, detached occurrences), and publishes only events that are active at the current time (UTC). When a group has no active events, the scheduler clears its retained topic by publishing an empty list. Time comparisons are UTC; naive timestamps are normalized. Logging is concise; conversion lookups are cached and logged only once per media. -- MQTT topics (paho-mqtt v2, use Callback API v2): - - Discovery: `infoscreen/discovery` (JSON includes `uuid`, hw/ip data). ACK to `infoscreen/{uuid}/discovery_ack`. See `listener/listener.py`. - - Heartbeat: `infoscreen/{uuid}/heartbeat` updates `Client.last_alive` (UTC); enhanced payload includes `current_process`, `process_pid`, `process_status`, `current_event_id`. - - Event lists (retained): `infoscreen/events/{group_id}` from `scheduler/scheduler.py`. - - Per-client group assignment (retained): `infoscreen/{uuid}/group_id` via `server/mqtt_helper.py`. - - Client logs: `infoscreen/{uuid}/logs/{error|warn|info}` with JSON payload (timestamp, message, context); QoS 1 for ERROR/WARN, QoS 0 for INFO. - - Client health: `infoscreen/{uuid}/health` with metrics (expected_state, actual_state, health_metrics); QoS 0, published every 5 seconds. - - Dashboard screenshots: `infoscreen/{uuid}/dashboard` uses grouped v2 payload blocks (`message`, `content`, `runtime`, `metadata`); listener reads screenshot data from `content.screenshot` and capture type from `metadata.capture.type`. -- Screenshots: server-side folder `server/screenshots/`; API serves `/screenshots/{uuid}.jpg` (latest) and `/screenshots/{uuid}/priority` (active high-priority fallback to latest). - -- Dev Container guidance: If extensions reappear inside the container, remove UI-only extensions from `devcontainer.json` `extensions` and map them in `remote.extensionKind` as `"ui"`. - -- Presentation conversion (PPT/PPTX/ODP → PDF): - - Trigger: on upload in `server/routes/eventmedia.py` for media types `ppt|pptx|odp` (compute sha256, upsert `Conversion`, enqueue job). - - Worker: RQ worker runs `server.worker.convert_event_media_to_pdf`, calls Gotenberg LibreOffice endpoint, writes to `server/media/converted/`. - - Services: Redis (queue) and Gotenberg added in compose; worker service consumes the `conversions` queue. - - Env: `REDIS_URL` (default `redis://redis:6379/0`), `GOTENBERG_URL` (default `http://gotenberg:3000`). - - Endpoints: `POST /api/conversions//pdf` (ensure/enqueue), `GET /api/conversions//status`, `GET /api/files/converted/` (serve PDFs). - - Storage: originals under `server/media/…`, outputs under `server/media/converted/` (prod compose mounts a shared volume for this path). - -## Data model highlights (see `models/models.py`) -- User model: Includes 7 new audit/security fields (migration: `4f0b8a3e5c20_add_user_audit_fields.py`): - - `last_login_at`, `last_password_change_at`: TIMESTAMP (UTC) tracking for auth events - - `failed_login_attempts`, `last_failed_login_at`: Security monitoring for brute-force detection - - `locked_until`: TIMESTAMP placeholder for account lockout (infrastructure in place, not yet enforced) - - `deactivated_at`, `deactivated_by`: Soft-delete audit trail (FK self-reference); soft deactivation is the default, hard delete superadmin-only - - Role hierarchy (privilege escalation enforced): `user` < `editor` < `admin` < `superadmin` -- Client monitoring (migration: `c1d2e3f4g5h6_add_client_monitoring.py`): - - `ClientLog` model: Centralized log storage with fields (id, client_uuid, timestamp, level, message, context, created_at); FK to clients.uuid (CASCADE) - - `Client` model extended: 7 health monitoring fields (`current_event_id`, `current_process`, `process_status`, `process_pid`, `last_screenshot_analyzed`, `screen_health_status`, `last_screenshot_hash`) - - Enums: `LogLevel` (ERROR, WARN, INFO, DEBUG), `ProcessStatus` (running, crashed, starting, stopped), `ScreenHealthStatus` (OK, BLACK, FROZEN, UNKNOWN) - - Indexes: (client_uuid, timestamp DESC), (level, timestamp DESC), (created_at DESC) for performance -- System settings: `system_settings` key–value store via `SystemSetting` for global configuration (e.g., WebUntis/Vertretungsplan supplement-table). Managed through routes in `server/routes/system_settings.py`. - - Presentation defaults (system-wide): - - `presentation_interval` (seconds, default "10") - - `presentation_page_progress` ("true"/"false", default "true") - - `presentation_auto_progress` ("true"/"false", default "true") - Seeded in `server/init_defaults.py` if missing. - - Video defaults (system-wide): - - `video_autoplay` ("true"/"false", default "true") - - `video_loop` ("true"/"false", default "true") - - `video_volume` (0.0–1.0, default "0.8") - - `video_muted` ("true"/"false", default "false") - Used as initial values when creating new video events; editable per event. - - Events: Added `page_progress` (Boolean) and `auto_progress` (Boolean) for presentation behavior per event. - - Event (video fields): `event_media_id`, `autoplay`, `loop`, `volume`, `muted`. - - WebUntis URL: WebUntis uses the existing Vertretungsplan/Supplement-Table URL (`supplement_table_url`). There is no separate `webuntis_url` setting; use `GET/POST /api/system-settings/supplement-table`. - -- Conversions: - - Enum `ConversionStatus`: `pending`, `processing`, `ready`, `failed`. - - Table `conversions`: `id`, `source_event_media_id` (FK→`event_media.id` ondelete CASCADE), `target_format`, `target_path`, `status`, `file_hash` (sha256), `started_at`, `completed_at`, `error_message`. - - Indexes: `(source_event_media_id, target_format)`, `(status, target_format)`; Unique: `(source_event_media_id, target_format, file_hash)`. - -## API patterns -- Blueprints live in `server/routes/*` and are registered in `server/wsgi.py` with `/api/...` prefixes. -- Session usage: instantiate `Session()` per request, commit when mutating, and always `session.close()` before returning. -- Examples: - - Clients: `server/routes/clients.py` includes bulk group updates and MQTT sync (`publish_multiple_client_groups`). - - Groups: `server/routes/groups.py` computes “alive” using a grace period that varies by `ENV`. - - `GET /api/groups/order` — retrieve saved group display order - - `POST /api/groups/order` — persist group display order (array of group IDs) - - Events: `server/routes/events.py` serializes enum values to strings and normalizes times to UTC. Recurring events are only deactivated after their recurrence_end (UNTIL); non-recurring events deactivate after their end time. Event exceptions are respected and rendered in scheduler output. - - Holidays: `server/routes/holidays.py` supports period-scoped list/import/manual CRUD (`GET/POST /api/holidays`, `POST /api/holidays/upload`, `PUT/DELETE /api/holidays/`), validates date ranges against the target period, prevents duplicates, merges same normalized `name+region` overlaps (including adjacent ranges), and rejects conflicting overlaps. - - Media: `server/routes/eventmedia.py` implements a simple file manager API rooted at `server/media/`. - - System settings: `server/routes/system_settings.py` exposes key–value CRUD (`/api/system-settings`) and a convenience endpoint for WebUntis/Vertretungsplan supplement-table: `GET/POST /api/system-settings/supplement-table` (admin+). - - Academic periods: `server/routes/academic_periods.py` exposes full lifecycle management (admin+ only): - - `GET /api/academic_periods` — list all non-archived periods ordered by start_date - - `GET /api/academic_periods/` — get single period by ID (including archived) - - `GET /api/academic_periods/active` — get currently active period - - `GET /api/academic_periods/for_date?date=YYYY-MM-DD` — period covering given date (non-archived) - - `GET /api/academic_periods//usage` — check linked events/media and recurrence spillover blockers - - `POST /api/academic_periods` — create period (validates name uniqueness among non-archived, date range, overlaps within periodType) - - `PUT /api/academic_periods/` — update period (cannot update archived periods) - - `POST /api/academic_periods//activate` — activate period (deactivates all others; cannot activate archived) - - `POST /api/academic_periods//archive` — soft-delete period (blocked if active or has active recurrence) - - `POST /api/academic_periods//restore` — restore archived period (returns to inactive) - - `DELETE /api/academic_periods/` — hard-delete archived inactive period (blocked if linked events exist) - - All responses use camelCase: `startDate`, `endDate`, `periodType`, `isActive`, `isArchived`, `archivedAt`, `archivedBy` - - Validation: name required/trimmed/unique among non-archived; startDate ≤ endDate; periodType in {schuljahr, semester, trimester}; overlaps blocked within same periodType - - Recurrence spillover detection: archive/delete blocked if recurring master events assigned to period still have current/future occurrences - - User management: `server/routes/users.py` exposes comprehensive CRUD for users (admin+): - - `GET /api/users` — list all users (role-filtered: admin sees user/editor/admin, superadmin sees all); includes audit fields in camelCase (lastLoginAt, lastPasswordChangeAt, failedLoginAttempts, deactivatedAt, deactivatedBy) - - `POST /api/users` — create user with username, password (min 6 chars), role, and status; admin cannot create superadmin; initializes audit fields - - `GET /api/users/` — get detailed user record with all audit fields - - `PUT /api/users/` — update user (cannot change own role/status; admin cannot modify superadmin accounts) - - `PUT /api/users//password` — admin password reset (requires backend check to reject self-reset for consistency) - - `DELETE /api/users/` — hard delete (superadmin only, with self-deletion check) - - Auth routes (`server/routes/auth.py`): Enhanced to track login events (sets `last_login_at`, resets `failed_login_attempts` on success; increments `failed_login_attempts` and `last_failed_login_at` on failure). Self-service password change via `PUT /api/auth/change-password` requires current password verification. - - Client logs (`server/routes/client_logs.py`): Centralized log retrieval for monitoring: - - `GET /api/client-logs//logs` – Query client logs with filters (level, limit, since); admin_or_higher - - `GET /api/client-logs/summary` – Log counts by level per client (last 24h); admin_or_higher - - `GET /api/client-logs/recent-errors` – System-wide error monitoring; admin_or_higher - - `GET /api/client-logs/monitoring-overview` – Includes screenshot priority fields per client plus `summary.activePriorityScreenshots`; superadmin_only - - `GET /api/client-logs/test` – Infrastructure validation (no auth); returns recent logs with counts - - Documentation maintenance: keep this file aligned with real patterns; update when routes/session/UTC rules change. Avoid long prose; link exact paths. - -## Frontend patterns (dashboard) -- **UI design rules**: Component choices, layout structure, button variants, badge colors, dialog patterns, toast conventions, and tab structure are defined in [`FRONTEND_DESIGN_RULES.md`](./FRONTEND_DESIGN_RULES.md). Follow that file for all dashboard work. -- Vite React app; proxies `/api` and `/screenshots` to API in dev (`vite.config.ts`). -- Uses Syncfusion components; Vite config pre-bundles specific packages to avoid alias issues. -- Environment: `VITE_API_URL` provided at build/run; in dev compose, proxy handles `/api` so local fetches can use relative `/api/...` paths. -- **API Response Format**: All API endpoints return camelCase JSON (e.g., `startTime`, `endTime`, `groupId`). Frontend consumes camelCase directly. -- **UTC Time Parsing**: API returns ISO strings without 'Z' suffix. Frontend appends 'Z' before parsing to ensure UTC interpretation: `const utcString = dateStr.endsWith('Z') ? dateStr : dateStr + 'Z'; new Date(utcString);`. Display uses `toLocaleTimeString('de-DE')` for German format. - -- Dev Container: When adding frontend deps, prefer `npm ci` and, if using named volumes, recreate dashboard `node_modules` volume so installs occur inside the container. -- Theming: All Syncfusion component CSS is imported centrally in `dashboard/src/main.tsx`. Theme conventions, component defaults, the full CSS import list, and Tailwind removal are documented in `FRONTEND_DESIGN_RULES.md`. - - Scheduler (appointments page): top bar includes Group and Academic Period selectors (Syncfusion DropDownList). Selecting a period calls `POST /api/academic_periods/active`, moves the calendar to today’s month/day within the period year, and refreshes a right-aligned indicator row showing: - - Holidays present in the current view (count) - - Period label (display_name or name) with a badge indicating whether any holidays exist in that period (overlap check) - - - Recurrence & holidays (latest): - - Backend stores holiday skips in `EventException` and emits `RecurrenceException` (EXDATE) for master events in `GET /api/events`. EXDATE tokens are formatted in RFC 5545 compact form (`yyyyMMddTHHmmssZ`) and correspond to each occurrence start time (UTC). Syncfusion uses these to exclude holiday instances reliably. - - Frontend lets Syncfusion handle all recurrence patterns natively (no client-side expansion). Scheduler field mappings include `recurrenceID`, `recurrenceRule`, and `recurrenceException` so series and edited occurrences are recognized correctly. - - Event deletion: All event types (single, single-in-series, entire series) are handled with custom dialogs. The frontend intercepts Syncfusion's built-in RecurrenceAlert and DeleteAlert popups to provide a unified, user-friendly deletion flow: - - Single (non-recurring) event: deleted directly after confirmation. - - Single occurrence of a recurring series: user can delete just that instance. - - Entire recurring series: user can delete all occurrences after a final custom confirmation dialog. - - Detached occurrences (edited/broken out): treated as single events. - - Single occurrence editing: Users can detach individual occurrences from recurring series. The frontend hooks `actionComplete`/`onActionCompleted` with `requestType='eventChanged'` to persist changes: it calls `POST /api/events//occurrences//detach` for single-occurrence edits and `PUT /api/events/` for series or single events as appropriate. The backend creates `EventException` and a standalone `Event` without modifying the master beyond EXDATEs. - - UI: Events with `SkipHolidays` render a TentTree icon next to the main event icon. The custom recurrence icon in the header was removed; rely on Syncfusion’s native lower-right recurrence badge. - - Website & WebUntis: Both event types display a website. WebUntis reads its URL from the system `supplement_table_url` and does not provide a per-event URL field. - -- Program info page (`dashboard/src/programminfo.tsx`): - - Loads data from `dashboard/public/program-info.json` (app name, version, build info, tech stack, changelog). - - Uses Syncfusion card classes (`e-card`, `e-card-header`, `e-card-title`, `e-card-content`) for consistent styling. - - Changelog is paginated with `PagerComponent` (from `@syncfusion/ej2-react-grids`), default page size 5; adjust `pageSize` or add a selector as needed. - -- Groups page (`dashboard/src/infoscreen_groups.tsx`): - - Migrated to Syncfusion inputs and popups: Buttons, TextBox, DropDownList, Dialog; Kanban remains for drag/drop. - - Unified toast/dialog wording; replaced legacy alerts with toasts; spacing handled via inline styles to avoid Tailwind dependency. - -- Header user menu (top-right): - - Shows current username and role; click opens a menu with "Passwort ändern" (lock icon), "Profil", and "Abmelden". - - Implemented with Syncfusion DropDownButton (`@syncfusion/ej2-react-splitbuttons`). - - "Passwort ändern": Opens self-service password change dialog (available to all authenticated users); requires current password verification, new password min 6 chars, must match confirm field; calls `PUT /api/auth/change-password` - - "Abmelden" navigates to `/logout`; the page invokes backend logout and redirects to `/login`. - -- User management page (`dashboard/src/users.tsx`): - - Full CRUD interface for managing users (admin+ only in menu); accessible via "Benutzer" sidebar entry - - Syncfusion GridComponent: 20 per page (configurable), sortable columns (ID, username, role), custom action button template with role-based visibility - - Statistics cards: total users, active (non-deactivated), inactive (deactivated) counts - - Dialogs: Create (username/password/role/status), Edit (with self-edit protections), Password Reset (admin only, no current password required), Delete (superadmin only, self-check), Details (read-only audit info with formatted timestamps) - - Role badges: Color-coded display (user: gray, editor: blue, admin: green, superadmin: red) - - Audit information displayed: last login, password change, last failed login, deactivation timestamps and deactivating user - - Role-based permissions (enforced backend + frontend): - - Admin: can manage user/editor/admin roles (not superadmin); soft-deactivate only; cannot see/edit superadmin accounts - - Superadmin: can manage all roles including other superadmins; can permanently hard-delete users - - Security rules enforced: cannot change own role, cannot deactivate own account, cannot delete self, cannot reset own password via admin route (must use self-service) - - API client in `dashboard/src/apiUsers.ts` for all user operations (listUsers, getUser, createUser, updateUser, resetUserPassword, deleteUser) - - Menu visibility: "Benutzer" menu item only visible to admin+ (role-gated in App.tsx) - -- Monitoring page (`dashboard/src/monitoring.tsx`): - - Superadmin-only dashboard for client monitoring and diagnostics; menu item is hidden for lower roles and the route redirects non-superadmins. - - Uses `GET /api/client-logs/monitoring-overview` for aggregated live status, `GET /api/client-logs/recent-errors` for system-wide errors, and `GET /api/client-logs//logs` for per-client details. - - Shows per-client status (`healthy`, `warning`, `critical`, `offline`) based on heartbeat freshness, process state, screen state, and recent log counts. - - Displays latest screenshot preview and active priority screenshot (`/screenshots/{uuid}/priority` when active), screenshot type badges, current process metadata, and recent ERROR/WARN activity. - - Uses adaptive refresh: normal interval in steady state, faster polling while `activePriorityScreenshots > 0`. - -- Settings page (`dashboard/src/settings.tsx`): - - Structure: Syncfusion TabComponent with role-gated tabs - - 📅 Academic Calendar (all users) - - **🗂️ Perioden (first sub-tab)**: Full period lifecycle management (admin+) - - List non-archived periods with active/archived badges and action buttons - - Create: dialog for name, displayName, startDate, endDate, periodType with validation - - Edit: update name, displayName, dates, type (cannot edit archived) - - Activate: set as active (deactivates all others) - - Archive: soft-delete with blocker checks (blocks if active or has active recurrence) - - Restore: restore archived periods to inactive state - - Delete: hard-delete archived periods with blocker checks (blocks if linked events) - - Archive visibility: toggle to show/hide archived periods - - Blockers: display prevents action with clear list of reasons (linked events, active recurrence, active status) - - **📥 Ferienkalender: Import/Anzeige (second sub-tab)**: CSV/TXT holiday import plus manual holiday create/edit/delete scoped to the selected academic period; changing the period redraws the import/list body. - - Import summary surfaces inserted/updated/merged/skipped/conflict counts and detailed conflict lines. - - File selection uses Syncfusion-styled trigger button and visible selected filename state. - - Manual date inputs guide users with bidirectional start/end constraints and prefill behavior. - - 🖥️ Display & Clients (admin+) - - Default Settings: placeholders for heartbeat, screenshots, defaults - - Client Configuration: quick links to Clients and Groups pages - - 🎬 Media & Files (admin+) - - Upload Settings: placeholders for limits and types - - Conversion Status: placeholder for conversions overview - - 🗓️ Events (admin+) - - WebUntis / Vertretungsplan: system-wide supplement table URL with enable/disable, save, and preview; persists via `/api/system-settings/supplement-table` - - Presentations: general defaults for slideshow interval, page-progress, and auto-progress; persisted via `/api/system-settings` keys (`presentation_interval`, `presentation_page_progress`, `presentation_auto_progress`). These defaults are applied when creating new presentation events (the custom event modal reads them and falls back to per-event values when editing). - - Videos: system-wide defaults for `autoplay`, `loop`, `volume`, and `muted`; persisted via `/api/system-settings` keys (`video_autoplay`, `video_loop`, `video_volume`, `video_muted`). These defaults are applied when creating new video events (the custom event modal reads them and falls back to per-event values when editing). - - Other event types (website, message, other): placeholders for defaults - - ⚙️ System (superadmin) - - Organization Info and Advanced Configuration placeholders - - Role gating: Admin/Superadmin tabs are hidden if the user lacks permission; System is superadmin-only - - API clients use relative `/api/...` URLs so Vite dev proxy handles requests without CORS issues. The settings UI calls are centralized in `dashboard/src/apiSystemSettings.ts` (system settings) and `dashboard/src/apiAcademicPeriods.ts` (periods CRUD). - - Nested tabs: implemented as controlled components using `selectedItem` with stateful handlers to prevent sub-tab resets during updates. - - Academic periods API client (`dashboard/src/apiAcademicPeriods.ts`): provides type-safe camelCase accessors (listAcademicPeriods, getAcademicPeriod, createAcademicPeriod, updateAcademicPeriod, setActiveAcademicPeriod, archiveAcademicPeriod, restoreAcademicPeriod, getAcademicPeriodUsage, deleteAcademicPeriod). - -- Dashboard page (`dashboard/src/dashboard.tsx`): - - Card-based overview of all Raumgruppen (room groups) with real-time status monitoring - - Global statistics: total infoscreens, online/offline counts, warning groups - - Filter buttons: All / Online / Offline / Warnings with dynamic counts - - Per-group cards show: - - Currently active event (title, type, date/time in local timezone) - - Health bar with online/offline ratio and color-coded status - - Expandable client list with last alive timestamps - - Bulk restart button for offline clients - - Uses Syncfusion ButtonComponent, ToastComponent, and card CSS classes - - Auto-refresh every 15 seconds; manual refresh button available - - "Nicht zugeordnet" group always appears last in sorted list - -- Ressourcen page (`dashboard/src/ressourcen.tsx`): - - Timeline view showing all groups and their active events in parallel - - Uses Syncfusion ScheduleComponent with TimelineViews (day/week modes) - - Compact row display: 65px height per group, dynamically calculated total height - - Group ordering panel with drag up/down controls; order persisted to backend via `/api/groups/order` - - Filters out "Nicht zugeordnet" group from timeline display - - Fetches events per group for current date range; displays first active event per group - - Color-coded event bars using `getGroupColor()` from `groupColors.ts` - - Resource-based timeline: each group is a resource row, events mapped to `ResourceId` - - Real-time updates: loads events on mount and when view/date changes - - Custom CSS in `dashboard/src/ressourcen.css` for timeline styling and controls - -- User dropdown technical notes: - - Dependencies: `@syncfusion/ej2-react-splitbuttons` and `@syncfusion/ej2-splitbuttons` must be installed. - - Vite: add both to `optimizeDeps.include` in `vite.config.ts` to avoid import-analysis errors. - - Dev containers: when `node_modules` is a named volume, recreate the dashboard node_modules volume after adding dependencies so `npm ci` runs inside the container. - -Note: Syncfusion usage in the dashboard is already documented above; if a UI for conversion status/downloads is added later, link its routes and components here. - -## Local development -- Compose: development is `docker-compose.yml` + `docker-compose.override.yml`. - - API (dev): `server/Dockerfile.dev` with debugpy on 5678, Flask app `wsgi:app` on :8000. - - Dashboard (dev): `dashboard/Dockerfile.dev` exposes :5173 and waits for API via `dashboard/wait-for-backend.sh`. - - Mosquitto: allows anonymous in dev; WebSocket on :9001. -- Common env vars: `DB_CONN`, `DB_USER`, `DB_PASSWORD`, `DB_HOST=db`, `DB_NAME`, `ENV`, `MQTT_USER`, `MQTT_PASSWORD`. - - Alembic: prod compose runs `alembic ... upgrade head` and `server/init_defaults.py` before gunicorn. - - Local dev: prefer `python server/initialize_database.py` for one-shot setup (migrations + defaults + academic periods). - - Defaults: `server/init_defaults.py` seeds initial system settings like `supplement_table_url` and `supplement_table_enabled` if missing. - - `server/init_academic_periods.py` remains available to (re)seed school years. - -## Production -- `docker-compose.prod.yml` uses prebuilt images (`ghcr.io/robbstarkaustria/*`). -- Nginx serves dashboard and proxies API; TLS certs expected in `certs/` and mounted to `/etc/nginx/certs`. - -## Environment variables (reference) -- DB_CONN — Preferred DB URL for services. Example: `mysql+pymysql://${DB_USER}:${DB_PASSWORD}@db/${DB_NAME}` -- DB_USER, DB_PASSWORD, DB_NAME, DB_HOST — Used to assemble DB_CONN in dev if missing; inside containers `DB_HOST=db`. -- ENV — `development` or `production`; in development, `server/database.py` loads `.env`. -- MQTT_BROKER_HOST, MQTT_BROKER_PORT — Defaults `mqtt` and `1883`; MQTT_USER/MQTT_PASSWORD optional (dev often anonymous per Mosquitto config). -- VITE_API_URL — Dashboard build-time base URL (prod); in dev the Vite proxy serves `/api` to `server:8000`. -- HEARTBEAT_GRACE_PERIOD_DEV / HEARTBEAT_GRACE_PERIOD_PROD — Groups "alive" window (defaults 180s dev / 170s prod). Clients send heartbeats every ~65s; grace periods allow 2 missed heartbeats plus safety margin. -- REFRESH_SECONDS — Optional scheduler republish interval; `0` disables periodic refresh. -- PRIORITY_SCREENSHOT_TTL_SECONDS — Optional monitoring priority window in seconds (default `120`); controls when event screenshots are considered active priority. - -## Conventions & gotchas -- **Datetime Handling**: - - Always compare datetimes in UTC; some DB values may be naive—normalize before comparing (see `routes/events.py`). - - Database stores timestamps in UTC (naive datetimes are normalized to UTC by backend) - - API returns ISO strings **without** 'Z' suffix: `"2025-11-27T20:03:00"` - - Frontend **must** append 'Z' before parsing: `const utcStr = dateStr.endsWith('Z') ? dateStr : dateStr + 'Z'; new Date(utcStr);` - - Display in local timezone using `toLocaleTimeString('de-DE', { hour: '2-digit', minute: '2-digit' })` - - When sending to API, use `date.toISOString()` which includes 'Z' and is UTC -- **JSON Naming Convention**: - - Backend uses snake_case internally (Python convention) - - API returns camelCase JSON (web standard): `startTime`, `endTime`, `groupId`, etc. - - Use `dict_to_camel_case()` from `server/serializers.py` before `jsonify()` - - Frontend consumes camelCase directly; Syncfusion scheduler maintains internal PascalCase with field mappings -- Scheduler enforces UTC comparisons and normalizes naive timestamps. It publishes only currently active events and clears retained topics for groups with no active events. It also queries a future window (default: 7 days) and expands recurring events using RFC 5545 rules. Event exceptions are respected. Logging is concise and conversion lookups are cached. -- Use retained MQTT messages for state that clients must recover after reconnect (events per group, client group_id). - - Clients should parse `event_type` and then read the corresponding nested payload (`presentation`, `website`, `video`, etc.). `website` and `webuntis` use the same nested `website` payload with `type: browser` and a `url`. Video events include `autoplay`, `loop`, `volume`, and `muted`. -- In-container DB host is `db`; do not use `localhost` inside services. -- No separate dev vs prod secret conventions: use the same env var keys across environments (e.g., `DB_CONN`, `MQTT_USER`, `MQTT_PASSWORD`). -- When adding a new route: - 1) Create a Blueprint in `server/routes/...`, - 2) Register it in `server/wsgi.py`, - 3) Manage `Session()` lifecycle, - 4) Return JSON-safe values (serialize enums and datetimes), and - 5) Use `dict_to_camel_case()` for camelCase JSON responses - -Docs maintenance guardrails (solo-friendly): Update this file alongside code changes (services/MQTT/API/UTC/env). Keep it concise (20–50 lines per section). Never include secrets. -- When extending media types, update `MediaType` and any logic in `eventmedia` and dashboard that depends on it. -- Academic periods: Events/media can be optionally associated with periods for educational organization. Only one period should be active at a time (`is_active=True`). -- Initialization scripts: legacy DB init scripts were removed; use Alembic and `initialize_database.py` going forward. - -### Recurrence & holidays: conventions -- Do not pre-expand recurrences on the backend. Always send master events with `RecurrenceRule` + `RecurrenceException`. -- Ensure EXDATE tokens are RFC 5545 timestamps (`yyyyMMddTHHmmssZ`) matching the occurrence start time (UTC) so Syncfusion can exclude them natively. -- School holidays are scoped by `academic_period_id`; holiday imports and queries should use the relevant academic period rather than treating holiday rows as global. -- Holiday write operations (manual/import) must validate date ranges against the selected academic period. -- Overlap policy: same normalized `name+region` overlaps (including adjacent ranges) are merged; overlaps with different identity are conflicts (manual blocked, import skipped with details). -- When `skip_holidays` or recurrence changes, regenerate `EventException` rows so `RecurrenceException` stays in sync, using the event's `academic_period_id` holidays (or only unassigned holidays for legacy events without a period). -- Single occurrence detach: Use `POST /api/events//occurrences//detach` to create standalone events and add EXDATE entries without modifying master events. The frontend persists edits via `actionComplete` (`requestType='eventChanged'`). - -## Quick examples -- Add client description persists to DB and publishes group via MQTT: see `PUT /api/clients//description` in `routes/clients.py`. -- Bulk group assignment emits retained messages for each client: `PUT /api/clients/group`. -- Listener heartbeat path: `infoscreen//heartbeat` → sets `clients.last_alive` and captures process health data. -- Client monitoring flow: Client publishes to `infoscreen/{uuid}/logs/error` and `infoscreen/{uuid}/health` → listener stores/updates monitoring state → API serves `/api/client-logs/monitoring-overview`, `/api/client-logs/recent-errors`, and `/api/client-logs//logs` → superadmin monitoring dashboard displays live status. - -## Scheduler payloads: presentation extras -- Presentation event payloads now include `page_progress` and `auto_progress` in addition to `slide_interval` and media files. These are sourced from per-event fields in the database (with system defaults applied on event creation). - -## Scheduler payloads: website & webuntis -- For both `website` and `webuntis`, the scheduler emits a nested `website` object: - - `{ "type": "browser", "url": "https://..." }` -- The `event_type` remains `website` or `webuntis`. Clients should treat both identically for rendering. -- The WebUntis URL is set at event creation by reading the system `supplement_table_url`. - -Questions or unclear areas? Tell us if you need: exact devcontainer debugging steps, stricter Alembic workflow, or a seed dataset beyond `init_defaults.py`. - -## Academic Periods System -- **Purpose**: Organize events and media by educational cycles (school years, semesters, trimesters) with full lifecycle management. -- **Design**: Fully backward compatible - existing events/media continue to work without period assignment. -- **Lifecycle States**: - - Active: exactly one period at a time (all others deactivated when activated) - - Inactive: saved period, not currently active - - Archived: soft-deleted; hidden from normal list; can be restored - - Deleted: hard-deleted; permanent removal (only when no linked events exist and no active recurrence) -- **Archive Rules**: Cannot archive active periods or periods with recurring master events that have current/future occurrences -- **Delete Rules**: Only archived inactive periods can be hard-deleted; blocked if linked events exist -- **Validation Rules**: - - Name: required, trimmed, unique among non-archived periods - - Dates: startDate ≤ endDate - - Type: schuljahr, semester, or trimester - - Overlaps: disallowed within same periodType (allowed across types) -- **Recurrence Spillover Detection**: Archive/delete blocked if recurring master events assigned to period still generate current/future occurrences -- **Model Fields**: `id`, `name`, `display_name`, `start_date`, `end_date`, `period_type`, `is_active`, `is_archived`, `archived_at`, `archived_by`, `created_at`, `updated_at` -- **Events/Media Association**: Both `Event` and `EventMedia` have optional `academic_period_id` FK for organizational grouping -- **UI Integration** (`dashboard/src/settings.tsx` > 🗂️ Perioden): - - List with badges (Active/Archived) - - Create/Edit dialogs with validation - - Activate, Archive, Restore, Delete actions with blocker preflight checks - - Archive visibility toggle to show/hide retired periods - - Error dialogs showing exact blockers (linked events, active recurrence, active status) - -## Changelog Style Guide (Program info) - -- Source: `dashboard/public/program-info.json`; newest entry first -- Fields per release: `version`, `date` (YYYY-MM-DD), `changes` (array of short bullets) -- Tone: concise, user-facing; German wording; area prefixes allowed (e.g., “UI: …”, “API: …”) -- Categories via emoji or words: Added (🆕/✨), Changed (🛠️), Fixed (✅/🐛), Removed (🗑️), Security (🔒), Deprecated (⚠️) -- Breaking changes must be prefixed with `BREAKING:` -- Keep ≤ 8–10 bullets; summarize or group micro-changes -- JSON hygiene: valid JSON, no trailing commas, don’t edit historical entries except typos - -## Versioning Convention (Tech vs UI) - -- Use one unified app version across technical and user-facing release notes. -- `dashboard/public/program-info.json` is user-facing and should list only user-visible changes. -- `TECH-CHANGELOG.md` can include deeper technical details for the same released version. -- If server/infrastructure work is implemented but not yet released or not user-visible, document it under the latest released section as: - - `Backend technical work (post-release notes; no version bump)` -- Do not create a new version header in `TECH-CHANGELOG.md` for internal milestones alone. -- Bump version numbers when a release is actually cut/deployed (or when user-facing release notes are published), not for intermediate backend-only steps. -- When UI integration lands later, include the user-visible part in the next release version and reference prior post-release technical groundwork when useful. +## Canonical docs map +- Repo entry: `README.md` +- Instruction governance: `AI-INSTRUCTIONS-MAINTENANCE.md` +- Technical release details: `TECH-CHANGELOG.md` +- Workspace/development notes: `DEV-CHANGELOG.md` +- MQTT payload details: `MQTT_EVENT_PAYLOAD_GUIDE.md` +- TV power contract: `TV_POWER_INTENT_SERVER_CONTRACT_V1.md` +- Frontend patterns: `FRONTEND_DESIGN_RULES.md` +- Archived historical docs: `docs/archive/` diff --git a/AI-INSTRUCTIONS-MAINTENANCE.md b/AI-INSTRUCTIONS-MAINTENANCE.md index 5bac440..de1ed97 100644 --- a/AI-INSTRUCTIONS-MAINTENANCE.md +++ b/AI-INSTRUCTIONS-MAINTENANCE.md @@ -36,6 +36,27 @@ Update the instructions in the same commit as your change whenever you: - Include concrete examples from this repo when describing patterns (e.g., which route shows enum serialization). - Never include secrets or real tokens; show only variable names and example formats. +## Scope boundaries (important) +To avoid turning `.github/copilot-instructions.md` into a shadow README/changelog, keep clear boundaries: +- `.github/copilot-instructions.md`: quick operational brief for agents (architecture snapshot, non-negotiable conventions, key paths, critical contracts). +- `README.md`: project entrypoint and documentation navigation. +- `TECH-CHANGELOG.md` and `DEV-CHANGELOG.md`: change history and release/development notes. +- Feature contracts/specs: dedicated files (for example `MQTT_EVENT_PAYLOAD_GUIDE.md`, `TV_POWER_INTENT_SERVER_CONTRACT_V1.md`). + +Do not place long historical sections, release summaries, or full endpoint catalogs in `.github/copilot-instructions.md`. + +## Size and quality guardrails +- Target size for `.github/copilot-instructions.md`: about 120-220 lines. +- If a new section exceeds ~10 lines, prefer linking to an existing canonical doc instead. +- Keep each section focused on actionability for coding agents. +- Remove duplicate rules if already stated in another section. +- Use concise bullets over long prose blocks. + +Quick pre-commit check: +- Is this content a rule/pattern needed during coding now? +- If it is historical context, move it to changelog/archive docs. +- If it is deep reference material, move it to the canonical feature doc and link it. + ## Solo-friendly workflow - Update docs in the same commit as your change: - Code changed → docs changed (copilot-instructions, `.env.example`, `deployment.md` as needed) @@ -101,4 +122,5 @@ exit 0 # warn only; do not block commit - Dev/Prod docs: `deployment.md`, `.env.example` ## Documentation sync log -- 2026-03-24: Synced docs for completed monitoring rollout and presentation flag persistence fix (`page_progress` / `auto_progress`). Updated `.github/copilot-instructions.md`, `README.md`, `TECH-CHANGELOG.md`, `DEV-CHANGELOG.md`, and `CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md` without a user-version bump. +- 2026-03-24: Synced docs for completed monitoring rollout and presentation flag persistence fix (`page_progress` / `auto_progress`). Updated `.github/copilot-instructions.md`, `README.md`, `TECH-CHANGELOG.md`, `DEV-CHANGELOG.md`, and `docs/archive/CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md` without a user-version bump. +- 2026-04-01: Synced docs for TV power intent Phase 1 implementation and contract consistency. Updated `.github/copilot-instructions.md`, `MQTT_EVENT_PAYLOAD_GUIDE.md`, `docs/archive/TV_POWER_PHASE_1_SERVER_HANDOFF.md`, `TECH-CHANGELOG.md`, and `DEV-CHANGELOG.md` to match scheduler behavior (`infoscreen/groups/{group_id}/power/intent`, `schema_version: "1.0"`, transition + heartbeat semantics, poll-based expiry). diff --git a/CLIENT_MONITORING_SPECIFICATION.md b/CLIENT_MONITORING_SPECIFICATION.md index 7860902..8734065 100644 --- a/CLIENT_MONITORING_SPECIFICATION.md +++ b/CLIENT_MONITORING_SPECIFICATION.md @@ -891,7 +891,7 @@ Reset: After 5 minutes of successful operation - Base URL: `http://192.168.43.201:8000` - Health check: `GET /health` - Test logs: `GET /api/client-logs/test` (no auth) -- Full API docs: See `CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md` on server +- Full API docs: See `docs/archive/CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md` on server **MQTT Broker:** - Host: `192.168.43.201` @@ -974,6 +974,6 @@ watchdog.monitor_loop() **END OF SPECIFICATION** Questions? Refer to: -- `CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md` (server repo) +- `docs/archive/CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md` (server repo) - Server API: `http://192.168.43.201:8000/api/client-logs/test` - MQTT test: `mosquitto_sub -h 192.168.43.201 -t infoscreen/#` diff --git a/DEV-CHANGELOG.md b/DEV-CHANGELOG.md index fbc74aa..76aaecc 100644 --- a/DEV-CHANGELOG.md +++ b/DEV-CHANGELOG.md @@ -5,6 +5,8 @@ This changelog tracks all changes made in the development workspace, including i --- ## Unreleased (development workspace) +- TV power intent (Phase 1): Scheduler publishes retained QoS1 group-level intents to `infoscreen/groups/{group_id}/power/intent` with transition+heartbeat semantics, startup/reconnect republish, and poll-based expiry (`max(3 × poll_interval_sec, 90s)`). +- TV power validation: Added unit/integration/canary coverage in `scheduler/test_power_intent_utils.py`, `scheduler/test_power_intent_scheduler.py`, and `test_power_intent_canary.py`. - Monitoring system completion: End-to-end monitoring pipeline is active (MQTT logs/health → listener persistence → monitoring APIs → superadmin dashboard). - Monitoring API: Added/active endpoints `GET /api/client-logs/monitoring-overview` and `GET /api/client-logs/recent-errors`; per-client logs via `GET /api/client-logs//logs`. - Dashboard monitoring UI: Superadmin monitoring page is integrated and displays client health status, screenshots, process metadata, and recent error activity. diff --git a/MQTT_EVENT_PAYLOAD_GUIDE.md b/MQTT_EVENT_PAYLOAD_GUIDE.md index f3003a6..d316a54 100644 --- a/MQTT_EVENT_PAYLOAD_GUIDE.md +++ b/MQTT_EVENT_PAYLOAD_GUIDE.md @@ -31,7 +31,7 @@ Example payload: ```json { - "schema_version": "tv-power-intent.v1", + "schema_version": "1.0", "intent_id": "9cf26d9b-87a3-42f1-8446-e90bb6f6ce63", "group_id": 12, "desired_state": "on", @@ -39,7 +39,9 @@ Example payload: "issued_at": "2026-03-31T10:15:30Z", "expires_at": "2026-03-31T10:17:00Z", "poll_interval_sec": 30, - "source": "scheduler" + "active_event_ids": [148], + "event_window_start": "2026-03-31T10:15:00Z", + "event_window_end": "2026-03-31T11:00:00Z" } ``` @@ -150,7 +152,7 @@ Every event includes these common fields: } ``` -**Note**: WebUntis events use the same payload structure as website events. The URL is fetched from system settings (`webuntis_url`) rather than being specified per-event. Clients treat `webuntis` and `website` event types identically—both display a website. +**Note**: WebUntis events use the same payload structure as website events. The URL is fetched from system settings (`supplement_table_url`) rather than being specified per-event. Clients treat `webuntis` and `website` event types identically—both display a website. #### Video Events diff --git a/README.md b/README.md index 88dd57c..1f85e1b 100644 --- a/README.md +++ b/README.md @@ -149,13 +149,11 @@ Rollout strategy (Phase 1): - [SUPERADMIN_SETUP.md](SUPERADMIN_SETUP.md) ### Monitoring, Screenshots, Health -- [CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md](CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md) - [CLIENT_MONITORING_SPECIFICATION.md](CLIENT_MONITORING_SPECIFICATION.md) - [SCREENSHOT_IMPLEMENTATION.md](SCREENSHOT_IMPLEMENTATION.md) ### MQTT & Payloads - [MQTT_EVENT_PAYLOAD_GUIDE.md](MQTT_EVENT_PAYLOAD_GUIDE.md) -- [MQTT_PAYLOAD_MIGRATION_GUIDE.md](MQTT_PAYLOAD_MIGRATION_GUIDE.md) ### Events, Calendar, WebUntis - [WEBUNTIS_EVENT_IMPLEMENTATION.md](WEBUNTIS_EVENT_IMPLEMENTATION.md) @@ -167,9 +165,17 @@ Rollout strategy (Phase 1): - [docs/archive/CLEANUP_SUMMARY.md](docs/archive/CLEANUP_SUMMARY.md) ### Conversion / Media -- [pptx_conversion_guide.md](pptx_conversion_guide.md) - [pptx_conversion_guide_gotenberg.md](pptx_conversion_guide_gotenberg.md) +### Historical / Archived Docs +- [docs/archive/CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md](docs/archive/CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md) - completed implementation plan/history +- [docs/archive/MQTT_DASHBOARD_V1_TO_V2_MIGRATION.md](docs/archive/MQTT_DASHBOARD_V1_TO_V2_MIGRATION.md) - completed MQTT payload migration notes +- [docs/archive/PPTX_CONVERSION_LEGACY_APPROACH.md](docs/archive/PPTX_CONVERSION_LEGACY_APPROACH.md) - legacy conversion approach (superseded) +- [docs/archive/TV_POWER_PHASE_1_COORDINATION.md](docs/archive/TV_POWER_PHASE_1_COORDINATION.md) - TV power Phase 1 coordination tasklist +- [docs/archive/TV_POWER_PHASE_1_SERVER_HANDOFF.md](docs/archive/TV_POWER_PHASE_1_SERVER_HANDOFF.md) - TV power Phase 1 server handoff notes +- [docs/archive/TV_POWER_PHASE_1_CANARY_VALIDATION.md](docs/archive/TV_POWER_PHASE_1_CANARY_VALIDATION.md) - TV power Phase 1 canary validation checklist +- [docs/archive/TV_POWER_PHASE_1_IMPLEMENTATION_CHECKLIST.md](docs/archive/TV_POWER_PHASE_1_IMPLEMENTATION_CHECKLIST.md) - TV power Phase 1 implementation checklist + ### Frontend - [FRONTEND_DESIGN_RULES.md](FRONTEND_DESIGN_RULES.md) - [dashboard/README.md](dashboard/README.md) diff --git a/TECH-CHANGELOG.md b/TECH-CHANGELOG.md index efd8ee5..b3b0e93 100644 --- a/TECH-CHANGELOG.md +++ b/TECH-CHANGELOG.md @@ -6,6 +6,13 @@ This changelog documents technical and developer-relevant changes included in public releases. For development workspace changes, see DEV-CHANGELOG.md. Not all changes here are reflected in the user-facing changelog (`program-info.json`), and not all UI/feature changes are repeated here. Some changes (e.g., backend refactoring, API adjustments, infrastructure, developer tooling, or internal logic) may only appear in TECH-CHANGELOG.md. For UI/feature changes, see `dashboard/public/program-info.json`. ## 2026.1.0-alpha.15 (2026-03-31) +- 🔌 **TV Power Intent Phase 1 (server-side)**: + - Scheduler now publishes retained QoS1 group-level intents to `infoscreen/groups/{group_id}/power/intent`. + - Implemented deterministic intent computation helpers in `scheduler/db_utils.py` (`compute_group_power_intent_basis`, `build_group_power_intent_body`, `compute_group_power_intent_fingerprint`). + - Implemented transition + heartbeat semantics in `scheduler/scheduler.py`: stable `intent_id` on heartbeat; new `intent_id` only on semantic transition. + - Added startup publish and MQTT reconnect republish behavior for retained intent continuity. + - Added poll-based expiry rule: `expires_at = issued_at + max(3 × poll_interval_sec, 90s)`. + - Added scheduler tests and canary validation artifacts: `scheduler/test_power_intent_utils.py`, `scheduler/test_power_intent_scheduler.py`, `test_power_intent_canary.py`, and `TV_POWER_CANARY_VALIDATION_CHECKLIST.md`. - 🗃️ **Holiday data model scoping to academic periods**: - Added period scoping for holidays via `SchoolHoliday.academic_period_id` (FK to academic periods) in `models/models.py`. - Added Alembic migration `f3c4d5e6a7b8_scope_school_holidays_to_academic_.py` to introduce FK/index/constraint updates for period-aware holiday storage. diff --git a/TV_POWER_HANDOFF_SERVER.md b/TV_POWER_HANDOFF_SERVER.md deleted file mode 100644 index 9b7fc9c..0000000 --- a/TV_POWER_HANDOFF_SERVER.md +++ /dev/null @@ -1,83 +0,0 @@ -# Server Handoff: TV Power Coordination - -## Purpose -Implement server-side MQTT power intent publishing so clients can keep TVs on across adjacent events and power off safely after schedules end. - -## Source of Truth -- Shared full plan: TV_POWER_COORDINATION_TASKLIST.md - -## Scope (Server Team) -- Scheduler-to-intent mapping -- MQTT publishing semantics (retain, QoS, expiry) -- Conflict handling (group vs client) -- Observability for intent lifecycle - -## MQTT Contract (Server Responsibilities) - -### Topics -- Primary (per-client): infoscreen/{client_id}/power/intent -- Optional (group-level): infoscreen/groups/{group_id}/power/intent - -### Delivery Semantics -- QoS: 1 -- retained: true -- Always publish UTC timestamps (ISO 8601 with Z) - -### Intent Payload (v1) -```json -{ - "schema_version": "1.0", - "intent_id": "uuid-or-monotonic-id", - "issued_at": "2026-03-31T12:00:00Z", - "expires_at": "2026-03-31T12:10:00Z", - "target": { - "client_id": "optional-if-group-topic", - "group_id": "optional" - }, - "power": { - "desired_state": "on", - "reason": "event_window_active", - "grace_seconds": 30 - }, - "event_window": { - "start": "2026-03-31T12:00:00Z", - "end": "2026-03-31T13:00:00Z" - } -} -``` - -## Required Behavior - -### Adjacent/Overlapping Events -- Never publish an intermediate off intent when windows are contiguous/overlapping. -- Maintain continuous desired_state=on coverage across adjacent windows. - -### Reconnect/Restart -- On scheduler restart, republish effective retained intent. -- On event edits/cancellations, replace retained intent with a fresh intent_id. - -### Conflict Policy -- If both group and client intent exist: per-client overrides group. - -### Expiry Safety -- expires_at must be set for every intent. -- Server should avoid publishing already-expired intents. - -## Implementation Tasks -1. Add scheduler mapping layer that computes effective desired_state per client timeline. -2. Add intent publisher with retained QoS1 delivery. -3. Generate unique intent_id for each semantic transition. -4. Emit issued_at/expires_at and event_window consistently in UTC. -5. Add group-vs-client precedence logic. -6. Add logs/metrics for publish success, retained payload age, and transition count. -7. Add integration tests for adjacent events and reconnect replay. - -## Acceptance Criteria -1. Adjacent events do not create OFF gap intents. -2. Fresh client receives retained intent after reconnect and gets correct desired state. -3. Intent payloads are schema-valid, UTC-formatted, and include expiry. -4. Publish logs and metrics allow intent timeline reconstruction. - -## Operational Notes -- Keep intent publishing idempotent and deterministic. -- Preserve backward compatibility while clients run in hybrid mode. diff --git a/CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md b/docs/archive/CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md similarity index 100% rename from CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md rename to docs/archive/CLIENT_MONITORING_IMPLEMENTATION_GUIDE.md diff --git a/MQTT_PAYLOAD_MIGRATION_GUIDE.md b/docs/archive/MQTT_DASHBOARD_V1_TO_V2_MIGRATION.md similarity index 100% rename from MQTT_PAYLOAD_MIGRATION_GUIDE.md rename to docs/archive/MQTT_DASHBOARD_V1_TO_V2_MIGRATION.md diff --git a/pptx_conversion_guide.md b/docs/archive/PPTX_CONVERSION_LEGACY_APPROACH.md similarity index 100% rename from pptx_conversion_guide.md rename to docs/archive/PPTX_CONVERSION_LEGACY_APPROACH.md diff --git a/TV_POWER_CANARY_VALIDATION_CHECKLIST.md b/docs/archive/TV_POWER_PHASE_1_CANARY_VALIDATION.md similarity index 97% rename from TV_POWER_CANARY_VALIDATION_CHECKLIST.md rename to docs/archive/TV_POWER_PHASE_1_CANARY_VALIDATION.md index 58c22f6..107530a 100644 --- a/TV_POWER_CANARY_VALIDATION_CHECKLIST.md +++ b/docs/archive/TV_POWER_PHASE_1_CANARY_VALIDATION.md @@ -16,7 +16,7 @@ Manual verification checklist for Phase-1 server-side group-level power-intent p Instructions: 1. Subscribe to `infoscreen/groups/1/power/intent` (canary group, QoS 1) 2. Verify received payload contains: - - `schema_version: "v1"` + - `schema_version: "1.0"` - `group_id: 1` - `desired_state: "on"` or `"off"` (string) - `reason: "active_event"` or `"no_active_event"` (string) @@ -24,6 +24,9 @@ Instructions: - `issued_at: "2026-03-31T14:22:15Z"` (ISO 8601 with Z suffix) - `expires_at: "2026-03-31T14:24:00Z"` (ISO 8601 with Z suffix, always > issued_at) - `poll_interval_sec: 30` (integer, matches scheduler poll interval) + - `active_event_ids: [...]` (array; empty when off) + - `event_window_start: "...Z"` or `null` + - `event_window_end: "...Z"` or `null` **Pass criteria**: All fields present, correct types and formats, no extra/malformed fields. diff --git a/TV_POWER_COORDINATION_TASKLIST.md b/docs/archive/TV_POWER_PHASE_1_COORDINATION.md similarity index 99% rename from TV_POWER_COORDINATION_TASKLIST.md rename to docs/archive/TV_POWER_PHASE_1_COORDINATION.md index 65a237a..d79d4fa 100644 --- a/TV_POWER_COORDINATION_TASKLIST.md +++ b/docs/archive/TV_POWER_PHASE_1_COORDINATION.md @@ -15,7 +15,7 @@ Prevent unintended TV power-off during adjacent events while enabling coordinate ## Server PR-1 Pointer - For the strict, agreed server-first implementation path, use: - - `TV_POWER_SERVER_PR1_IMPLEMENTATION_CHECKLIST.md` + - `TV_POWER_PHASE_1_IMPLEMENTATION_CHECKLIST.md` - Treat that checklist as the execution source of truth for Phase 1. --- diff --git a/TV_POWER_SERVER_PR1_IMPLEMENTATION_CHECKLIST.md b/docs/archive/TV_POWER_PHASE_1_IMPLEMENTATION_CHECKLIST.md similarity index 96% rename from TV_POWER_SERVER_PR1_IMPLEMENTATION_CHECKLIST.md rename to docs/archive/TV_POWER_PHASE_1_IMPLEMENTATION_CHECKLIST.md index 603c91b..4907ab7 100644 --- a/TV_POWER_SERVER_PR1_IMPLEMENTATION_CHECKLIST.md +++ b/docs/archive/TV_POWER_PHASE_1_IMPLEMENTATION_CHECKLIST.md @@ -172,8 +172,8 @@ All PR-1 server-side items are complete. Below is a summary of deliverables: - **MQTT_EVENT_PAYLOAD_GUIDE.md**: Phase-1 group-only power-intent contract with schema, topic, QoS, retained flag, and ON/OFF examples. - **README.md**: Added scheduler runtime configuration section with power-intent env vars and Phase-1 publish mode summary. - **AI-INSTRUCTIONS-MAINTENANCE.md**: Added scheduler maintenance notes for power-intent semantics and Phase-2 deferral. -- **TV_POWER_CANARY_VALIDATION_CHECKLIST.md**: 10-scenario manual validation matrix for operators. -- **TV_POWER_SERVER_PR1_IMPLEMENTATION_CHECKLIST.md**: This file; source of truth for PR-1 scope and acceptance criteria. +- **TV_POWER_PHASE_1_CANARY_VALIDATION.md**: 10-scenario manual validation matrix for operators. +- **TV_POWER_PHASE_1_IMPLEMENTATION_CHECKLIST.md**: This file; source of truth for PR-1 scope and acceptance criteria. ### Validation Artifacts - **test_power_intent_canary.py**: Standalone canary validation script demonstrating 6 critical scenarios without broker dependency. All scenarios pass. @@ -195,5 +195,5 @@ All PR-1 server-side items are complete. Below is a summary of deliverables: ### Next Phase - Phase 2 (deferred): Per-client override intent, client state acknowledgments, listener persistence of state -- Canary rollout strategy documented in `TV_POWER_CANARY_VALIDATION_CHECKLIST.md` +- Canary rollout strategy documented in `TV_POWER_PHASE_1_CANARY_VALIDATION.md` diff --git a/docs/archive/TV_POWER_PHASE_1_SERVER_HANDOFF.md b/docs/archive/TV_POWER_PHASE_1_SERVER_HANDOFF.md new file mode 100644 index 0000000..fa534ba --- /dev/null +++ b/docs/archive/TV_POWER_PHASE_1_SERVER_HANDOFF.md @@ -0,0 +1,56 @@ +# Server Handoff: TV Power Coordination + +## Status +Server PR-1 is implemented and merged (Phase 1). + +## Source of Truth +- Contract: TV_POWER_INTENT_SERVER_CONTRACT_V1.md +- Implementation: scheduler/scheduler.py and scheduler/db_utils.py +- Validation checklist: TV_POWER_PHASE_1_CANARY_VALIDATION.md + +## Active Phase 1 Scope +- Topic: infoscreen/groups/{group_id}/power/intent +- QoS: 1 +- Retained: true +- Scope: group-level only +- Per-client intent/state topics: deferred to Phase 2 + +## Publish Semantics (Implemented) +- Semantic transition (`desired_state` or `reason` changed): new `intent_id` and immediate publish +- Heartbeat (no semantic change): same `intent_id`, refreshed `issued_at` and `expires_at` +- Scheduler startup: immediate publish before first poll wait +- MQTT reconnect: immediate retained republish of cached intents + +## Payload Contract (Phase 1) +```json +{ + "schema_version": "1.0", + "intent_id": "uuid4", + "group_id": 12, + "desired_state": "on", + "reason": "active_event", + "issued_at": "2026-04-01T06:00:03.496Z", + "expires_at": "2026-04-01T06:01:33.496Z", + "poll_interval_sec": 15, + "active_event_ids": [148], + "event_window_start": "2026-04-01T06:00:00Z", + "event_window_end": "2026-04-01T07:00:00Z" +} +``` + +Expiry rule: +- expires_at = issued_at + max(3 x poll_interval_sec, 90 seconds) + +## Operational Notes +- Adjacent/overlapping events are merged into one active coverage window; no OFF blip at boundaries. +- Feature flag defaults are safe for rollout: + - POWER_INTENT_PUBLISH_ENABLED=false + - POWER_INTENT_HEARTBEAT_ENABLED=true + - POWER_INTENT_EXPIRY_MULTIPLIER=3 + - POWER_INTENT_MIN_EXPIRY_SECONDS=90 +- Keep this handoff concise and defer full details to the stable contract document. + +## Phase 2 (Deferred) +- Per-client override topic: infoscreen/{client_uuid}/power/intent +- Client power state topic and acknowledgments +- Listener persistence of client-level power state