infoscreen/.github/copilot-instructions.md

# Copilot instructions for infoscreen_2025

# Purpose
These instructions tell Copilot Chat how to reason about this codebase.
Prefer explanations and refactors that align with these structures.

Use this as your shared context when proposing changes. Keep edits minimal and match existing patterns referenced below.

## TL;DR
Small multi-service digital signage app (Flask API, React dashboard, MQTT scheduler). Edit `server/` for API logic, `scheduler/` for event publishing, and `dashboard/` for UI. If you're asking Copilot for changes, prefer focused prompts that include the target file(s) and the desired behavior.

### How to ask Copilot
- "Add a new route `GET /api/events/summary` that returns counts per event_type — implement in `server/routes/events.py`."
- "Create an Alembic migration to add `duration` and `resolution` to `event_media` and update upload handler to populate them." 
- "Refactor `scheduler/db_utils.py` to prefer precomputed EventMedia metadata and fall back to a HEAD probe." 
- "Add an ffprobe-based worker that extracts duration/resolution/bitrate and stores them on `EventMedia`."

Keep docs synced with code. When you change services/MQTT/API/UTC/env or dev/prod run steps, update this file in the same commit (see `AI-INSTRUCTIONS-MAINTENANCE.md`).

### When not to change
- Avoid editing generated assets under `dashboard/dist/` and compiled bundles. Don't modify files produced by CI or Docker builds (unless intentionally updating build outputs).

### Contact / owner
- Primary maintainer: RobbStarkAustria (owner). For architecture questions, ping the repo owner or open an issue and tag `@RobbStarkAustria`.

### Important files (quick jump targets)
- `scheduler/db_utils.py` — event formatting and scheduler-facing logic
- `scheduler/scheduler.py` — scheduler main loop and MQTT publisher
- `server/routes/eventmedia.py` — file uploads, streaming endpoint
- `server/routes/events.py` — event CRUD and recurrence handling
- `server/routes/groups.py` — group management, alive status, display order persistence
- `dashboard/src/components/CustomEventModal.tsx` — event creation UI
- `dashboard/src/media.tsx` — FileManager / upload settings
- `dashboard/src/settings.tsx` — settings UI (nested tabs; system defaults for presentations and videos)
- `dashboard/src/ressourcen.tsx` — timeline view showing all groups' active events in parallel
- `dashboard/src/ressourcen.css` — timeline and resource view styling
- `dashboard/src/monitoring.tsx` — superadmin-only monitoring dashboard for client health, screenshots, and logs



## Big picture
- Multi-service app orchestrated by Docker Compose.
  - API: Flask + SQLAlchemy (MariaDB), in `server/` exposed on :8000 (health: `/health`).
  - Dashboard: React + Vite in `dashboard/`, dev on :5173, served via Nginx in prod.
  - MQTT broker: Eclipse Mosquitto, config in `mosquitto/config/mosquitto.conf`.
  - Listener: MQTT consumer handling discovery, heartbeats, and dashboard screenshot uploads in `listener/listener.py`.
  - Scheduler: Publishes only currently active events (per group, at "now") to MQTT retained topics in `scheduler/scheduler.py`. It queries a future window (default: 7 days) to expand recurring events using RFC 5545 rules and applies event exceptions, but only publishes events that are active at the current time. When a group has no active events, the scheduler clears its retained topic by publishing an empty list. All time comparisons are UTC; any naive timestamps are normalized. Logging is concise; conversion lookups are cached and logged only once per media.
  - Nginx: Reverse proxy routes `/api/*` and `/screenshots/*` to API; everything else to dashboard (`nginx.conf`).

  - Dev Container (hygiene): UI-only `Dev Containers` extension runs on host UI via `remote.extensionKind`; do not install it in-container. Dashboard installs use `npm ci`; shell aliases in `postStartCommand` are appended idempotently.

### Screenshot retention
- Screenshots sent via dashboard MQTT are stored in `server/screenshots/`.
- Screenshot payloads support `screenshot_type` with values `periodic`, `event_start`, `event_stop`.
- `periodic` is the normal heartbeat/dashboard screenshot path; `event_start` and `event_stop` are high-priority screenshots for monitoring.
- For each client, the API keeps `{uuid}.jpg` as latest and the last 20 timestamped screenshots (`{uuid}_..._{type}.jpg`), deleting older timestamped files automatically.
- For high-priority screenshots, the API additionally maintains `{uuid}_priority.jpg` and metadata in `{uuid}_meta.json` (`latest_screenshot_type`, `last_priority_*`).

  ## Recent changes since last commit

  ### Latest (March 2026)

  - **Monitoring System Completion (no version bump)**:
    - End-to-end monitoring pipeline completed: MQTT logs/health → listener persistence → monitoring APIs → superadmin dashboard
    - API now serves aggregated monitoring via `GET /api/client-logs/monitoring-overview` and system-wide recent errors via `GET /api/client-logs/recent-errors`
    - Monitoring dashboard (`dashboard/src/monitoring.tsx`) is active and displays client health states, screenshots, process metadata, and recent log activity
  - **Screenshot Priority Pipeline (no version bump)**:
    - Listener forwards `screenshot_type` from MQTT screenshot/dashboard payloads to `POST /api/clients/<uuid>/screenshot`.
    - API stores typed screenshots, tracks latest/priority metadata, and serves priority images via `GET /screenshots/<uuid>/priority`.
    - Monitoring overview exposes screenshot priority state (`latestScreenshotType`, `priorityScreenshotType`, `priorityScreenshotReceivedAt`, `hasActivePriorityScreenshot`) and `summary.activePriorityScreenshots`.
    - Monitoring UI shows screenshot type badges and switches to faster refresh while priority screenshots are active.
  - **MQTT Dashboard Payload v2 Cutover (no version bump)**:
    - Dashboard payload parsing in `listener/listener.py` is now v2-only (`message`, `content`, `runtime`, `metadata`).
    - Legacy top-level dashboard fallback was removed after migration soak (`legacy_fallback=0`).
    - Listener observability summarizes parser health using `v2_success` and `parse_failures` counters.
  - **Presentation Flags Persistence Fix**:
    - Fixed persistence for presentation `page_progress` and `auto_progress` to ensure values are reliably stored and returned across create/update paths and detached occurrences

  ### Earlier (January 2026)
  
  - **Ressourcen Page (Timeline View)**:
    - New 'Ressourcen' page with parallel timeline view showing active events for all room groups
    - Compact timeline display with adjustable row height (65px per group)
    - Real-time view of currently running events with type, title, and time window
    - Customizable group ordering with visual reordering panel (drag up/down buttons)
    - Group order persisted via `GET/POST /api/groups/order` endpoints
    - Color-coded event bars matching group theme
    - Timeline modes: Day and Week views (day view by default)
    - Dynamic height calculation based on number of groups
    - Syncfusion ScheduleComponent with TimelineViews, Resize, and DragAndDrop support
    - Files: `dashboard/src/ressourcen.tsx` (page), `dashboard/src/ressourcen.css` (styles)

  ### Earlier (November 2025)
  
  - **API Naming Convention Standardization (camelCase)**:
    - Backend: Created `server/serializers.py` with `dict_to_camel_case()` utility for consistent JSON serialization
    - Events API: `GET /api/events` and `GET /api/events/<id>` now return camelCase fields (`id`, `subject`, `startTime`, `endTime`, `type`, `groupId`, etc.) instead of PascalCase
    - Frontend: Dashboard and appointments page updated to consume camelCase API responses
    - Appointments page maintains internal PascalCase for Syncfusion scheduler compatibility with automatic mapping from API responses
    - **Breaking**: External API consumers must update field names from PascalCase to camelCase
  
  - **UTC Time Handling**:
    - Database stores all timestamps in UTC (naive timestamps normalized by backend)
    - API returns ISO strings without 'Z' suffix: `"2025-11-27T20:03:00"`
    - Frontend: Dashboard and appointments automatically append 'Z' to parse as UTC and display in user's local timezone
    - Time formatting functions use `toLocaleTimeString('de-DE')` for German locale display
    - All time comparisons use UTC; `new Date().toISOString()` sends UTC back to API
    - API returns ISO strings without `Z`; frontend must append `Z` before parsing to ensure UTC

  - **Dashboard Enhancements**:
    - New card-based design for Raumgruppen (room groups) with Syncfusion components
    - Global statistics summary: total infoscreens, online/offline counts, warning groups
    - Filter buttons: All, Online, Offline, Warnings with dynamic counts
    - Active event display per group: shows currently playing content with type icon, title, date, and time
    - Health visualization with color-coded progress bars per group
    - Expandable client details with last alive timestamps
    - Bulk restart functionality for offline clients per group
    - Manual refresh button with toast notifications
    - 15-second auto-refresh interval

  ### Earlier changes

  - Scheduler: when formatting video events the scheduler now performs a best-effort HEAD probe of the streaming URL and includes basic metadata in the emitted payload (mime_type, size, accept_ranges). Placeholders for richer metadata (duration, resolution, bitrate, qualities, thumbnails, checksum) are included for later population by a background worker.
  - Streaming endpoint: a range-capable streaming endpoint was added at `/api/eventmedia/stream/<media_id>/<filename>` that supports byte-range requests (206 Partial Content) to enable seeking from clients.
  - Event model & API: `Event` gained video-related fields (`event_media_id`, `autoplay`, `loop`, `volume`) and the API accepts and persists these when creating/updating video events.
  - Dashboard: UI updated to allow selecting uploaded videos for events and to specify autoplay/loop/volume. File upload settings were increased (maxFileSize raised) and the client now validates video duration (max 10 minutes) before upload.
  - FileManager: uploads compute basic metadata and enqueue conversions for office formats as before; video uploads now surface size and are streamable via the new endpoint.

  - Event model & API (new): Added `muted` (Boolean) for video events; create/update and GET endpoints accept, persist, and return `muted` alongside `autoplay`, `loop`, and `volume`.
  - Dashboard — Settings: Settings page refactored to nested tabs; added Events → Videos defaults (autoplay, loop, volume, mute) backed by system settings keys (`video_autoplay`, `video_loop`, `video_volume`, `video_muted`).
  - Dashboard — Events UI: CustomEventModal now exposes per-event video `muted` and initializes all video fields from system defaults when creating a new event.
  - Dashboard — Academic Calendar: Holiday management now uses a single “📥 Ferienkalender: Import/Anzeige” tab; admins select the target academic period first, and import/list content redraws for that period.
  - Dashboard — Holiday Management Hardening: The same tab now supports manual holiday CRUD in addition to CSV/TXT import. Imports and manual saves validate date ranges against the selected academic period, prevent duplicates, auto-merge same normalized name+region overlaps (including adjacent ranges), and report conflicting overlaps.

  Note: these edits are intentionally backwards-compatible — if the probe fails, the scheduler still emits the stream URL and the client should fallback to a direct play attempt or request richer metadata when available.

  Backend rework notes (no version bump):
  - Dev container hygiene: UI-only Remote Containers; reproducible dashboard installs (`npm ci`); idempotent shell aliases.
  - Serialization consistency: snake_case internal → camelCase external via `server/serializers.py` for all JSON.
  - UTC normalization across routes/scheduler; enums and datetimes serialize consistently.

## Service boundaries & data flow
- Database connection string is passed as `DB_CONN` (mysql+pymysql) to Python services.
  - API builds its engine in `server/database.py` (loads `.env` only in development).
  - Listener also creates its own engine for writes to `clients`.
  - Scheduler queries a future window (default: 7 days) to expand recurring events using RFC 5545 rules, applies event exceptions (skipped dates, detached occurrences), and publishes only events that are active at the current time (UTC). When a group has no active events, the scheduler clears its retained topic by publishing an empty list. Time comparisons are UTC; naive timestamps are normalized. Logging is concise; conversion lookups are cached and logged only once per media.
- MQTT topics (paho-mqtt v2, use Callback API v2):
  - Discovery: `infoscreen/discovery` (JSON includes `uuid`, hw/ip data). ACK to `infoscreen/{uuid}/discovery_ack`. See `listener/listener.py`.
  - Heartbeat: `infoscreen/{uuid}/heartbeat` updates `Client.last_alive` (UTC); enhanced payload includes `current_process`, `process_pid`, `process_status`, `current_event_id`.
  - Event lists (retained): `infoscreen/events/{group_id}` from `scheduler/scheduler.py`.
  - Per-client group assignment (retained): `infoscreen/{uuid}/group_id` via `server/mqtt_helper.py`.
  - Client logs: `infoscreen/{uuid}/logs/{error|warn|info}` with JSON payload (timestamp, message, context); QoS 1 for ERROR/WARN, QoS 0 for INFO.
  - Client health: `infoscreen/{uuid}/health` with metrics (expected_state, actual_state, health_metrics); QoS 0, published every 5 seconds.
  - Dashboard screenshots: `infoscreen/{uuid}/dashboard` uses grouped v2 payload blocks (`message`, `content`, `runtime`, `metadata`); listener reads screenshot data from `content.screenshot` and capture type from `metadata.capture.type`.
- Screenshots: server-side folder `server/screenshots/`; API serves `/screenshots/{uuid}.jpg` (latest) and `/screenshots/{uuid}/priority` (active high-priority fallback to latest).

- Dev Container guidance: If extensions reappear inside the container, remove UI-only extensions from `devcontainer.json` `extensions` and map them in `remote.extensionKind` as `"ui"`.

- Presentation conversion (PPT/PPTX/ODP → PDF):
  - Trigger: on upload in `server/routes/eventmedia.py` for media types `ppt|pptx|odp` (compute sha256, upsert `Conversion`, enqueue job).
  - Worker: RQ worker runs `server.worker.convert_event_media_to_pdf`, calls Gotenberg LibreOffice endpoint, writes to `server/media/converted/`.
  - Services: Redis (queue) and Gotenberg added in compose; worker service consumes the `conversions` queue.
  - Env: `REDIS_URL` (default `redis://redis:6379/0`), `GOTENBERG_URL` (default `http://gotenberg:3000`).
  - Endpoints: `POST /api/conversions/<media_id>/pdf` (ensure/enqueue), `GET /api/conversions/<media_id>/status`, `GET /api/files/converted/<path>` (serve PDFs).
  - Storage: originals under `server/media/…`, outputs under `server/media/converted/` (prod compose mounts a shared volume for this path).

## Data model highlights (see `models/models.py`)
- User model: Includes 7 new audit/security fields (migration: `4f0b8a3e5c20_add_user_audit_fields.py`):
  - `last_login_at`, `last_password_change_at`: TIMESTAMP (UTC) tracking for auth events
  - `failed_login_attempts`, `last_failed_login_at`: Security monitoring for brute-force detection
  - `locked_until`: TIMESTAMP placeholder for account lockout (infrastructure in place, not yet enforced)
  - `deactivated_at`, `deactivated_by`: Soft-delete audit trail (FK self-reference); soft deactivation is the default, hard delete superadmin-only
  - Role hierarchy (privilege escalation enforced): `user` < `editor` < `admin` < `superadmin`
- Client monitoring (migration: `c1d2e3f4g5h6_add_client_monitoring.py`):
  - `ClientLog` model: Centralized log storage with fields (id, client_uuid, timestamp, level, message, context, created_at); FK to clients.uuid (CASCADE)
  - `Client` model extended: 7 health monitoring fields (`current_event_id`, `current_process`, `process_status`, `process_pid`, `last_screenshot_analyzed`, `screen_health_status`, `last_screenshot_hash`)
  - Enums: `LogLevel` (ERROR, WARN, INFO, DEBUG), `ProcessStatus` (running, crashed, starting, stopped), `ScreenHealthStatus` (OK, BLACK, FROZEN, UNKNOWN)
  - Indexes: (client_uuid, timestamp DESC), (level, timestamp DESC), (created_at DESC) for performance
- System settings: `system_settings` key–value store via `SystemSetting` for global configuration (e.g., WebUntis/Vertretungsplan supplement-table). Managed through routes in `server/routes/system_settings.py`.
  - Presentation defaults (system-wide):
    - `presentation_interval` (seconds, default "10")
    - `presentation_page_progress` ("true"/"false", default "true")
    - `presentation_auto_progress` ("true"/"false", default "true")
    Seeded in `server/init_defaults.py` if missing.
  - Video defaults (system-wide):
    - `video_autoplay` ("true"/"false", default "true")
    - `video_loop` ("true"/"false", default "true")
    - `video_volume` (0.0–1.0, default "0.8")
    - `video_muted` ("true"/"false", default "false")
    Used as initial values when creating new video events; editable per event.
  - Events: Added `page_progress` (Boolean) and `auto_progress` (Boolean) for presentation behavior per event.
  - Event (video fields): `event_media_id`, `autoplay`, `loop`, `volume`, `muted`.
  - WebUntis URL: WebUntis uses the existing Vertretungsplan/Supplement-Table URL (`supplement_table_url`). There is no separate `webuntis_url` setting; use `GET/POST /api/system-settings/supplement-table`.

- Conversions:
  - Enum `ConversionStatus`: `pending`, `processing`, `ready`, `failed`.
  - Table `conversions`: `id`, `source_event_media_id` (FK→`event_media.id` ondelete CASCADE), `target_format`, `target_path`, `status`, `file_hash` (sha256), `started_at`, `completed_at`, `error_message`.
  - Indexes: `(source_event_media_id, target_format)`, `(status, target_format)`; Unique: `(source_event_media_id, target_format, file_hash)`.

## API patterns
- Blueprints live in `server/routes/*` and are registered in `server/wsgi.py` with `/api/...` prefixes.
- Session usage: instantiate `Session()` per request, commit when mutating, and always `session.close()` before returning.
- Examples:
  - Clients: `server/routes/clients.py` includes bulk group updates and MQTT sync (`publish_multiple_client_groups`).
  - Groups: `server/routes/groups.py` computes “alive” using a grace period that varies by `ENV`.
    - `GET /api/groups/order` — retrieve saved group display order
    - `POST /api/groups/order` — persist group display order (array of group IDs)
  - Events: `server/routes/events.py` serializes enum values to strings and normalizes times to UTC. Recurring events are only deactivated after their recurrence_end (UNTIL); non-recurring events deactivate after their end time. Event exceptions are respected and rendered in scheduler output.
  - Holidays: `server/routes/holidays.py` supports period-scoped list/import/manual CRUD (`GET/POST /api/holidays`, `POST /api/holidays/upload`, `PUT/DELETE /api/holidays/<id>`), validates date ranges against the target period, prevents duplicates, merges same normalized `name+region` overlaps (including adjacent ranges), and rejects conflicting overlaps.
  - Media: `server/routes/eventmedia.py` implements a simple file manager API rooted at `server/media/`.
  - System settings: `server/routes/system_settings.py` exposes key–value CRUD (`/api/system-settings`) and a convenience endpoint for WebUntis/Vertretungsplan supplement-table: `GET/POST /api/system-settings/supplement-table` (admin+).
  - Academic periods: `server/routes/academic_periods.py` exposes full lifecycle management (admin+ only):
    - `GET /api/academic_periods` — list all non-archived periods ordered by start_date
    - `GET /api/academic_periods/<id>` — get single period by ID (including archived)
    - `GET /api/academic_periods/active` — get currently active period
    - `GET /api/academic_periods/for_date?date=YYYY-MM-DD` — period covering given date (non-archived)
    - `GET /api/academic_periods/<id>/usage` — check linked events/media and recurrence spillover blockers
    - `POST /api/academic_periods` — create period (validates name uniqueness among non-archived, date range, overlaps within periodType)
    - `PUT /api/academic_periods/<id>` — update period (cannot update archived periods)
    - `POST /api/academic_periods/<id>/activate` — activate period (deactivates all others; cannot activate archived)
    - `POST /api/academic_periods/<id>/archive` — soft-delete period (blocked if active or has active recurrence)
    - `POST /api/academic_periods/<id>/restore` — restore archived period (returns to inactive)
    - `DELETE /api/academic_periods/<id>` — hard-delete archived inactive period (blocked if linked events exist)
    - All responses use camelCase: `startDate`, `endDate`, `periodType`, `isActive`, `isArchived`, `archivedAt`, `archivedBy`
    - Validation: name required/trimmed/unique among non-archived; startDate ≤ endDate; periodType in {schuljahr, semester, trimester}; overlaps blocked within same periodType
    - Recurrence spillover detection: archive/delete blocked if recurring master events assigned to period still have current/future occurrences
  - User management: `server/routes/users.py` exposes comprehensive CRUD for users (admin+):
    - `GET /api/users` — list all users (role-filtered: admin sees user/editor/admin, superadmin sees all); includes audit fields in camelCase (lastLoginAt, lastPasswordChangeAt, failedLoginAttempts, deactivatedAt, deactivatedBy)
    - `POST /api/users` — create user with username, password (min 6 chars), role, and status; admin cannot create superadmin; initializes audit fields
    - `GET /api/users/<id>` — get detailed user record with all audit fields
    - `PUT /api/users/<id>` — update user (cannot change own role/status; admin cannot modify superadmin accounts)
    - `PUT /api/users/<id>/password` — admin password reset (requires backend check to reject self-reset for consistency)
    - `DELETE /api/users/<id>` — hard delete (superadmin only, with self-deletion check)
  - Auth routes (`server/routes/auth.py`): Enhanced to track login events (sets `last_login_at`, resets `failed_login_attempts` on success; increments `failed_login_attempts` and `last_failed_login_at` on failure). Self-service password change via `PUT /api/auth/change-password` requires current password verification.
  - Client logs (`server/routes/client_logs.py`): Centralized log retrieval for monitoring:
    - `GET /api/client-logs/<uuid>/logs` – Query client logs with filters (level, limit, since); admin_or_higher
    - `GET /api/client-logs/summary` – Log counts by level per client (last 24h); admin_or_higher
    - `GET /api/client-logs/recent-errors` – System-wide error monitoring; admin_or_higher
    - `GET /api/client-logs/monitoring-overview` – Includes screenshot priority fields per client plus `summary.activePriorityScreenshots`; superadmin_only
    - `GET /api/client-logs/test` – Infrastructure validation (no auth); returns recent logs with counts

  Documentation maintenance: keep this file aligned with real patterns; update when routes/session/UTC rules change. Avoid long prose; link exact paths.

## Frontend patterns (dashboard)
- **UI design rules**: Component choices, layout structure, button variants, badge colors, dialog patterns, toast conventions, and tab structure are defined in [`FRONTEND_DESIGN_RULES.md`](./FRONTEND_DESIGN_RULES.md). Follow that file for all dashboard work.
- Vite React app; proxies `/api` and `/screenshots` to API in dev (`vite.config.ts`).
- Uses Syncfusion components; Vite config pre-bundles specific packages to avoid alias issues.
- Environment: `VITE_API_URL` provided at build/run; in dev compose, proxy handles `/api` so local fetches can use relative `/api/...` paths.
- **API Response Format**: All API endpoints return camelCase JSON (e.g., `startTime`, `endTime`, `groupId`). Frontend consumes camelCase directly.
- **UTC Time Parsing**: API returns ISO strings without 'Z' suffix. Frontend appends 'Z' before parsing to ensure UTC interpretation: `const utcString = dateStr.endsWith('Z') ? dateStr : dateStr + 'Z'; new Date(utcString);`. Display uses `toLocaleTimeString('de-DE')` for German format.

- Dev Container: When adding frontend deps, prefer `npm ci` and, if using named volumes, recreate dashboard `node_modules` volume so installs occur inside the container.
- Theming: All Syncfusion component CSS is imported centrally in `dashboard/src/main.tsx`. Theme conventions, component defaults, the full CSS import list, and Tailwind removal are documented in `FRONTEND_DESIGN_RULES.md`.
 - Scheduler (appointments page): top bar includes Group and Academic Period selectors (Syncfusion DropDownList). Selecting a period calls `POST /api/academic_periods/active`, moves the calendar to today’s month/day within the period year, and refreshes a right-aligned indicator row showing:
   - Holidays present in the current view (count)
   - Period label (display_name or name) with a badge indicating whether any holidays exist in that period (overlap check)

 - Recurrence & holidays (latest):
  - Backend stores holiday skips in `EventException` and emits `RecurrenceException` (EXDATE) for master events in `GET /api/events`. EXDATE tokens are formatted in RFC 5545 compact form (`yyyyMMddTHHmmssZ`) and correspond to each occurrence start time (UTC). Syncfusion uses these to exclude holiday instances reliably.
  - Frontend lets Syncfusion handle all recurrence patterns natively (no client-side expansion). Scheduler field mappings include `recurrenceID`, `recurrenceRule`, and `recurrenceException` so series and edited occurrences are recognized correctly.
  - Event deletion: All event types (single, single-in-series, entire series) are handled with custom dialogs. The frontend intercepts Syncfusion's built-in RecurrenceAlert and DeleteAlert popups to provide a unified, user-friendly deletion flow:
    - Single (non-recurring) event: deleted directly after confirmation.
    - Single occurrence of a recurring series: user can delete just that instance.
    - Entire recurring series: user can delete all occurrences after a final custom confirmation dialog.
    - Detached occurrences (edited/broken out): treated as single events.
  - Single occurrence editing: Users can detach individual occurrences from recurring series. The frontend hooks `actionComplete`/`onActionCompleted` with `requestType='eventChanged'` to persist changes: it calls `POST /api/events/<id>/occurrences/<date>/detach` for single-occurrence edits and `PUT /api/events/<id>` for series or single events as appropriate. The backend creates `EventException` and a standalone `Event` without modifying the master beyond EXDATEs.
  - UI: Events with `SkipHolidays` render a TentTree icon next to the main event icon. The custom recurrence icon in the header was removed; rely on Syncfusion’s native lower-right recurrence badge.
  - Website & WebUntis: Both event types display a website. WebUntis reads its URL from the system `supplement_table_url` and does not provide a per-event URL field.

- Program info page (`dashboard/src/programminfo.tsx`):
  - Loads data from `dashboard/public/program-info.json` (app name, version, build info, tech stack, changelog).
  - Uses Syncfusion card classes (`e-card`, `e-card-header`, `e-card-title`, `e-card-content`) for consistent styling.
  - Changelog is paginated with `PagerComponent` (from `@syncfusion/ej2-react-grids`), default page size 5; adjust `pageSize` or add a selector as needed.

- Groups page (`dashboard/src/infoscreen_groups.tsx`):
  - Migrated to Syncfusion inputs and popups: Buttons, TextBox, DropDownList, Dialog; Kanban remains for drag/drop.
  - Unified toast/dialog wording; replaced legacy alerts with toasts; spacing handled via inline styles to avoid Tailwind dependency.

- Header user menu (top-right):
  - Shows current username and role; click opens a menu with "Passwort ändern" (lock icon), "Profil", and "Abmelden".
  - Implemented with Syncfusion DropDownButton (`@syncfusion/ej2-react-splitbuttons`).
  - "Passwort ändern": Opens self-service password change dialog (available to all authenticated users); requires current password verification, new password min 6 chars, must match confirm field; calls `PUT /api/auth/change-password`
  - "Abmelden" navigates to `/logout`; the page invokes backend logout and redirects to `/login`.

- User management page (`dashboard/src/users.tsx`):
  - Full CRUD interface for managing users (admin+ only in menu); accessible via "Benutzer" sidebar entry
  - Syncfusion GridComponent: 20 per page (configurable), sortable columns (ID, username, role), custom action button template with role-based visibility
  - Statistics cards: total users, active (non-deactivated), inactive (deactivated) counts
  - Dialogs: Create (username/password/role/status), Edit (with self-edit protections), Password Reset (admin only, no current password required), Delete (superadmin only, self-check), Details (read-only audit info with formatted timestamps)
  - Role badges: Color-coded display (user: gray, editor: blue, admin: green, superadmin: red)
  - Audit information displayed: last login, password change, last failed login, deactivation timestamps and deactivating user
  - Role-based permissions (enforced backend + frontend):
    - Admin: can manage user/editor/admin roles (not superadmin); soft-deactivate only; cannot see/edit superadmin accounts
    - Superadmin: can manage all roles including other superadmins; can permanently hard-delete users
  - Security rules enforced: cannot change own role, cannot deactivate own account, cannot delete self, cannot reset own password via admin route (must use self-service)
  - API client in `dashboard/src/apiUsers.ts` for all user operations (listUsers, getUser, createUser, updateUser, resetUserPassword, deleteUser)
  - Menu visibility: "Benutzer" menu item only visible to admin+ (role-gated in App.tsx)

- Monitoring page (`dashboard/src/monitoring.tsx`):
  - Superadmin-only dashboard for client monitoring and diagnostics; menu item is hidden for lower roles and the route redirects non-superadmins.
  - Uses `GET /api/client-logs/monitoring-overview` for aggregated live status, `GET /api/client-logs/recent-errors` for system-wide errors, and `GET /api/client-logs/<uuid>/logs` for per-client details.
  - Shows per-client status (`healthy`, `warning`, `critical`, `offline`) based on heartbeat freshness, process state, screen state, and recent log counts.
  - Displays latest screenshot preview and active priority screenshot (`/screenshots/{uuid}/priority` when active), screenshot type badges, current process metadata, and recent ERROR/WARN activity.
  - Uses adaptive refresh: normal interval in steady state, faster polling while `activePriorityScreenshots > 0`.

- Settings page (`dashboard/src/settings.tsx`):
  - Structure: Syncfusion TabComponent with role-gated tabs
    - 📅 Academic Calendar (all users)
      - **🗂️ Perioden (first sub-tab)**: Full period lifecycle management (admin+)
        - List non-archived periods with active/archived badges and action buttons
        - Create: dialog for name, displayName, startDate, endDate, periodType with validation
        - Edit: update name, displayName, dates, type (cannot edit archived)
        - Activate: set as active (deactivates all others)
        - Archive: soft-delete with blocker checks (blocks if active or has active recurrence)
        - Restore: restore archived periods to inactive state
        - Delete: hard-delete archived periods with blocker checks (blocks if linked events)
        - Archive visibility: toggle to show/hide archived periods
        - Blockers: display prevents action with clear list of reasons (linked events, active recurrence, active status)
      - **📥 Ferienkalender: Import/Anzeige (second sub-tab)**: CSV/TXT holiday import plus manual holiday create/edit/delete scoped to the selected academic period; changing the period redraws the import/list body.
        - Import summary surfaces inserted/updated/merged/skipped/conflict counts and detailed conflict lines.
        - File selection uses Syncfusion-styled trigger button and visible selected filename state.
        - Manual date inputs guide users with bidirectional start/end constraints and prefill behavior.
    - 🖥️ Display & Clients (admin+)
      - Default Settings: placeholders for heartbeat, screenshots, defaults
      - Client Configuration: quick links to Clients and Groups pages
    - 🎬 Media & Files (admin+)
      - Upload Settings: placeholders for limits and types
      - Conversion Status: placeholder for conversions overview
    - 🗓️ Events (admin+)
      - WebUntis / Vertretungsplan: system-wide supplement table URL with enable/disable, save, and preview; persists via `/api/system-settings/supplement-table`
      - Presentations: general defaults for slideshow interval, page-progress, and auto-progress; persisted via `/api/system-settings` keys (`presentation_interval`, `presentation_page_progress`, `presentation_auto_progress`). These defaults are applied when creating new presentation events (the custom event modal reads them and falls back to per-event values when editing).
      - Videos: system-wide defaults for `autoplay`, `loop`, `volume`, and `muted`; persisted via `/api/system-settings` keys (`video_autoplay`, `video_loop`, `video_volume`, `video_muted`). These defaults are applied when creating new video events (the custom event modal reads them and falls back to per-event values when editing).
      - Other event types (website, message, other): placeholders for defaults
    - ⚙️ System (superadmin)
      - Organization Info and Advanced Configuration placeholders
  - Role gating: Admin/Superadmin tabs are hidden if the user lacks permission; System is superadmin-only
  - API clients use relative `/api/...` URLs so Vite dev proxy handles requests without CORS issues. The settings UI calls are centralized in `dashboard/src/apiSystemSettings.ts` (system settings) and `dashboard/src/apiAcademicPeriods.ts` (periods CRUD).
  - Nested tabs: implemented as controlled components using `selectedItem` with stateful handlers to prevent sub-tab resets during updates.
  - Academic periods API client (`dashboard/src/apiAcademicPeriods.ts`): provides type-safe camelCase accessors (listAcademicPeriods, getAcademicPeriod, createAcademicPeriod, updateAcademicPeriod, setActiveAcademicPeriod, archiveAcademicPeriod, restoreAcademicPeriod, getAcademicPeriodUsage, deleteAcademicPeriod).

- Dashboard page (`dashboard/src/dashboard.tsx`):
  - Card-based overview of all Raumgruppen (room groups) with real-time status monitoring
  - Global statistics: total infoscreens, online/offline counts, warning groups
  - Filter buttons: All / Online / Offline / Warnings with dynamic counts
  - Per-group cards show:
    - Currently active event (title, type, date/time in local timezone)
    - Health bar with online/offline ratio and color-coded status
    - Expandable client list with last alive timestamps
    - Bulk restart button for offline clients
  - Uses Syncfusion ButtonComponent, ToastComponent, and card CSS classes
  - Auto-refresh every 15 seconds; manual refresh button available
  - "Nicht zugeordnet" group always appears last in sorted list

- Ressourcen page (`dashboard/src/ressourcen.tsx`):
  - Timeline view showing all groups and their active events in parallel
  - Uses Syncfusion ScheduleComponent with TimelineViews (day/week modes)
  - Compact row display: 65px height per group, dynamically calculated total height
  - Group ordering panel with drag up/down controls; order persisted to backend via `/api/groups/order`
  - Filters out "Nicht zugeordnet" group from timeline display
  - Fetches events per group for current date range; displays first active event per group
  - Color-coded event bars using `getGroupColor()` from `groupColors.ts`
  - Resource-based timeline: each group is a resource row, events mapped to `ResourceId`
  - Real-time updates: loads events on mount and when view/date changes
  - Custom CSS in `dashboard/src/ressourcen.css` for timeline styling and controls

- User dropdown technical notes:
  - Dependencies: `@syncfusion/ej2-react-splitbuttons` and `@syncfusion/ej2-splitbuttons` must be installed.
  - Vite: add both to `optimizeDeps.include` in `vite.config.ts` to avoid import-analysis errors.
  - Dev containers: when `node_modules` is a named volume, recreate the dashboard node_modules volume after adding dependencies so `npm ci` runs inside the container.

Note: Syncfusion usage in the dashboard is already documented above; if a UI for conversion status/downloads is added later, link its routes and components here.

## Local development
- Compose: development is `docker-compose.yml` + `docker-compose.override.yml`.
  - API (dev): `server/Dockerfile.dev` with debugpy on 5678, Flask app `wsgi:app` on :8000.
  - Dashboard (dev): `dashboard/Dockerfile.dev` exposes :5173 and waits for API via `dashboard/wait-for-backend.sh`.
  - Mosquitto: allows anonymous in dev; WebSocket on :9001.
- Common env vars: `DB_CONN`, `DB_USER`, `DB_PASSWORD`, `DB_HOST=db`, `DB_NAME`, `ENV`, `MQTT_USER`, `MQTT_PASSWORD`.
  - Alembic: prod compose runs `alembic ... upgrade head` and `server/init_defaults.py` before gunicorn.
  - Local dev: prefer `python server/initialize_database.py` for one-shot setup (migrations + defaults + academic periods).
  - Defaults: `server/init_defaults.py` seeds initial system settings like `supplement_table_url` and `supplement_table_enabled` if missing.
  - `server/init_academic_periods.py` remains available to (re)seed school years.

## Production
- `docker-compose.prod.yml` uses prebuilt images (`ghcr.io/robbstarkaustria/*`).
- Nginx serves dashboard and proxies API; TLS certs expected in `certs/` and mounted to `/etc/nginx/certs`.

## Environment variables (reference)
- DB_CONN — Preferred DB URL for services. Example: `mysql+pymysql://${DB_USER}:${DB_PASSWORD}@db/${DB_NAME}`
- DB_USER, DB_PASSWORD, DB_NAME, DB_HOST — Used to assemble DB_CONN in dev if missing; inside containers `DB_HOST=db`.
- ENV — `development` or `production`; in development, `server/database.py` loads `.env`.
- MQTT_BROKER_HOST, MQTT_BROKER_PORT — Defaults `mqtt` and `1883`; MQTT_USER/MQTT_PASSWORD optional (dev often anonymous per Mosquitto config).
- VITE_API_URL — Dashboard build-time base URL (prod); in dev the Vite proxy serves `/api` to `server:8000`.
- HEARTBEAT_GRACE_PERIOD_DEV / HEARTBEAT_GRACE_PERIOD_PROD — Groups "alive" window (defaults 180s dev / 170s prod). Clients send heartbeats every ~65s; grace periods allow 2 missed heartbeats plus safety margin.
- REFRESH_SECONDS — Optional scheduler republish interval; `0` disables periodic refresh.
- PRIORITY_SCREENSHOT_TTL_SECONDS — Optional monitoring priority window in seconds (default `120`); controls when event screenshots are considered active priority.

## Conventions & gotchas
- **Datetime Handling**:
  - Always compare datetimes in UTC; some DB values may be naive—normalize before comparing (see `routes/events.py`).
  - Database stores timestamps in UTC (naive datetimes are normalized to UTC by backend)
  - API returns ISO strings **without** 'Z' suffix: `"2025-11-27T20:03:00"`
  - Frontend **must** append 'Z' before parsing: `const utcStr = dateStr.endsWith('Z') ? dateStr : dateStr + 'Z'; new Date(utcStr);`
  - Display in local timezone using `toLocaleTimeString('de-DE', { hour: '2-digit', minute: '2-digit' })`
  - When sending to API, use `date.toISOString()` which includes 'Z' and is UTC
- **JSON Naming Convention**:
  - Backend uses snake_case internally (Python convention)
  - API returns camelCase JSON (web standard): `startTime`, `endTime`, `groupId`, etc.
  - Use `dict_to_camel_case()` from `server/serializers.py` before `jsonify()`
  - Frontend consumes camelCase directly; Syncfusion scheduler maintains internal PascalCase with field mappings
- Scheduler enforces UTC comparisons and normalizes naive timestamps. It publishes only currently active events and clears retained topics for groups with no active events. It also queries a future window (default: 7 days) and expands recurring events using RFC 5545 rules. Event exceptions are respected. Logging is concise and conversion lookups are cached.
- Use retained MQTT messages for state that clients must recover after reconnect (events per group, client group_id).
  - Clients should parse `event_type` and then read the corresponding nested payload (`presentation`, `website`, `video`, etc.). `website` and `webuntis` use the same nested `website` payload with `type: browser` and a `url`. Video events include `autoplay`, `loop`, `volume`, and `muted`.
- In-container DB host is `db`; do not use `localhost` inside services.
- No separate dev vs prod secret conventions: use the same env var keys across environments (e.g., `DB_CONN`, `MQTT_USER`, `MQTT_PASSWORD`).
- When adding a new route:
  1) Create a Blueprint in `server/routes/...`,
  2) Register it in `server/wsgi.py`,
  3) Manage `Session()` lifecycle,
  4) Return JSON-safe values (serialize enums and datetimes), and
  5) Use `dict_to_camel_case()` for camelCase JSON responses

Docs maintenance guardrails (solo-friendly): Update this file alongside code changes (services/MQTT/API/UTC/env). Keep it concise (20–50 lines per section). Never include secrets.
- When extending media types, update `MediaType` and any logic in `eventmedia` and dashboard that depends on it.
- Academic periods: Events/media can be optionally associated with periods for educational organization. Only one period should be active at a time (`is_active=True`).
- Initialization scripts: legacy DB init scripts were removed; use Alembic and `initialize_database.py` going forward.

### Recurrence & holidays: conventions
- Do not pre-expand recurrences on the backend. Always send master events with `RecurrenceRule` + `RecurrenceException`.
- Ensure EXDATE tokens are RFC 5545 timestamps (`yyyyMMddTHHmmssZ`) matching the occurrence start time (UTC) so Syncfusion can exclude them natively.
- School holidays are scoped by `academic_period_id`; holiday imports and queries should use the relevant academic period rather than treating holiday rows as global.
- Holiday write operations (manual/import) must validate date ranges against the selected academic period.
- Overlap policy: same normalized `name+region` overlaps (including adjacent ranges) are merged; overlaps with different identity are conflicts (manual blocked, import skipped with details).
- When `skip_holidays` or recurrence changes, regenerate `EventException` rows so `RecurrenceException` stays in sync, using the event's `academic_period_id` holidays (or only unassigned holidays for legacy events without a period).
- Single occurrence detach: Use `POST /api/events/<id>/occurrences/<date>/detach` to create standalone events and add EXDATE entries without modifying master events. The frontend persists edits via `actionComplete` (`requestType='eventChanged'`).

## Quick examples
- Add client description persists to DB and publishes group via MQTT: see `PUT /api/clients/<uuid>/description` in `routes/clients.py`.
- Bulk group assignment emits retained messages for each client: `PUT /api/clients/group`.
- Listener heartbeat path: `infoscreen/<uuid>/heartbeat` → sets `clients.last_alive` and captures process health data.
- Client monitoring flow: Client publishes to `infoscreen/{uuid}/logs/error` and `infoscreen/{uuid}/health` → listener stores/updates monitoring state → API serves `/api/client-logs/monitoring-overview`, `/api/client-logs/recent-errors`, and `/api/client-logs/<uuid>/logs` → superadmin monitoring dashboard displays live status.

## Scheduler payloads: presentation extras
- Presentation event payloads now include `page_progress` and `auto_progress` in addition to `slide_interval` and media files. These are sourced from per-event fields in the database (with system defaults applied on event creation).
 
## Scheduler payloads: website & webuntis
- For both `website` and `webuntis`, the scheduler emits a nested `website` object:
  - `{ "type": "browser", "url": "https://..." }`
- The `event_type` remains `website` or `webuntis`. Clients should treat both identically for rendering.
- The WebUntis URL is set at event creation by reading the system `supplement_table_url`.

Questions or unclear areas? Tell us if you need: exact devcontainer debugging steps, stricter Alembic workflow, or a seed dataset beyond `init_defaults.py`.

## Academic Periods System
- **Purpose**: Organize events and media by educational cycles (school years, semesters, trimesters) with full lifecycle management.
- **Design**: Fully backward compatible - existing events/media continue to work without period assignment.
- **Lifecycle States**: 
  - Active: exactly one period at a time (all others deactivated when activated)
  - Inactive: saved period, not currently active
  - Archived: soft-deleted; hidden from normal list; can be restored
  - Deleted: hard-deleted; permanent removal (only when no linked events exist and no active recurrence)
- **Archive Rules**: Cannot archive active periods or periods with recurring master events that have current/future occurrences
- **Delete Rules**: Only archived inactive periods can be hard-deleted; blocked if linked events exist
- **Validation Rules**:
  - Name: required, trimmed, unique among non-archived periods
  - Dates: startDate ≤ endDate
  - Type: schuljahr, semester, or trimester
  - Overlaps: disallowed within same periodType (allowed across types)
- **Recurrence Spillover Detection**: Archive/delete blocked if recurring master events assigned to period still generate current/future occurrences
- **Model Fields**: `id`, `name`, `display_name`, `start_date`, `end_date`, `period_type`, `is_active`, `is_archived`, `archived_at`, `archived_by`, `created_at`, `updated_at`
- **Events/Media Association**: Both `Event` and `EventMedia` have optional `academic_period_id` FK for organizational grouping
- **UI Integration** (`dashboard/src/settings.tsx` > 🗂️ Perioden):
  - List with badges (Active/Archived)
  - Create/Edit dialogs with validation
  - Activate, Archive, Restore, Delete actions with blocker preflight checks
  - Archive visibility toggle to show/hide retired periods
  - Error dialogs showing exact blockers (linked events, active recurrence, active status)

## Changelog Style Guide (Program info)

- Source: `dashboard/public/program-info.json`; newest entry first
- Fields per release: `version`, `date` (YYYY-MM-DD), `changes` (array of short bullets)
- Tone: concise, user-facing; German wording; area prefixes allowed (e.g., “UI: …”, “API: …”)
- Categories via emoji or words: Added (🆕/✨), Changed (🛠️), Fixed (✅/🐛), Removed (🗑️), Security (🔒), Deprecated (⚠️)
- Breaking changes must be prefixed with `BREAKING:`
- Keep ≤ 8–10 bullets; summarize or group micro-changes
- JSON hygiene: valid JSON, no trailing commas, don’t edit historical entries except typos

## Versioning Convention (Tech vs UI)

- Use one unified app version across technical and user-facing release notes.
- `dashboard/public/program-info.json` is user-facing and should list only user-visible changes.
- `TECH-CHANGELOG.md` can include deeper technical details for the same released version.
- If server/infrastructure work is implemented but not yet released or not user-visible, document it under the latest released section as:
  - `Backend technical work (post-release notes; no version bump)`
- Do not create a new version header in `TECH-CHANGELOG.md` for internal milestones alone.
- Bump version numbers when a release is actually cut/deployed (or when user-facing release notes are published), not for intermediate backend-only steps.
- When UI integration lands later, include the user-visible part in the next release version and reference prior post-release technical groundwork when useful.