olafn/infoscreen-dev
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: refactor docs structure and tighten assistant instruction policy

Browse Source 
shrink root README into a landing page with a docs map and focused contributor guidance
add TV_POWER_RUNBOOK as the canonical TV power rollout and canary runbook
add CHANGELOG and move project history out of README-style docs
refactor src README into a developer-focused guide (architecture, runtime files, MQTT, debugging)
prune redundant older HDMI docs and keep a canonical HDMI_CEC_SETUP path
update copilot instructions to a high-signal policy format with strict anti-shadow-README design rules
align references across docs to current files, scripts, and TV power behavior
This commit is contained in:
RobbStarkAustria
2026-04-01 10:01:58 +02:00
parent fb0980aa88
commit 82f43f75ba
20 changed files with 2228 additions and 2267 deletions
Download Patch File Download Diff File Expand all files Collapse all files

654
.github/copilot-instructions.md vendored
View File

@@ -1,596 +1,124 @@
# Copilot Instructions - Infoscreen Client
## Quick Start for AI Assistants
## Purpose
This file defines durable, high-signal instructions for AI assistants working in this repository.
### Critical Rules
- ✅ **ALWAYS use Impressive** for PDF presentations (has native auto-advance/loop)
- ❌ **NEVER suggest xdotool** approaches (failed on Raspberry Pi due to focus issues)
- ❌ **NEVER suggest video conversion** (adds complexity, had black screen issues)
- ✅ **Virtual environment MUST have** pygame + pillow (required for Impressive)
- ✅ **Client-side resize/compress** screenshots before MQTT transmission
- ✅ **Server renders PPTX → PDF via Gotenberg** (client only displays PDFs, no LibreOffice needed)
- ✅ **Keep screenshot consent notice in docs** when describing dashboard screenshot feature
- ✅ **Event-start/event-stop screenshots must preserve metadata** - See SCREENSHOT_MQTT_FIX.md for critical race condition that was fixed
- ✅ **Screenshot updates must keep `latest.jpg` and `meta.json` in sync** (simclient prefers `latest.jpg`)
- ✅ **Dashboard payload uses grouped v2 schema** (`message/content/runtime/metadata`, `schema_version="2.0"`)
- ✅ **Event-triggered screenshots**: `display_manager` arms a `threading.Timer` after start/stop, captures, writes `meta.json` with `send_immediately=true`; simclient fires within ≤1s
- ✅ **Payload assembly is centralized** in `_build_dashboard_payload()` — do not build dashboard JSON at call sites
## Instruction File Design Rules
### Key Files & Locations
- **Display logic**: `src/display_manager.py` (controls presentations/video/web)
- **MQTT client**: `src/simclient.py` (event management, heartbeat, discovery)
- **Runtime state**: `src/current_event.json` (current active event)
- **Process health bridge**: `src/current_process_health.json` (display_manager -> simclient)
- **Config**: `src/config/client_uuid.txt`, `src/config/last_group_id.txt`, `.env`
- **Logs**: `logs/display_manager.log`, `logs/simclient.log`, `logs/monitoring.log`
- **Screenshots**: `src/screenshots/` (shared volume between processes)
Treat this file as policy, not as project handbook.
### Common Tasks Quick Reference
| Task | File | Key Method/Section |
|------|------|-------------------|
| Add event type | `display_manager.py` | `start_display_for_event()` |
| Modify presentation | `display_manager.py` | `start_presentation()` |
| Modify process monitoring | `display_manager.py` | `ProcessHealthState`, `process_events()` |
| Publish health/log topics | `simclient.py` | `read_health_state()`, `publish_health_message()`, `publish_log_message()` |
| Change MQTT topics | `simclient.py` | Topic constants/handlers |
| Update screenshot | `display_manager.py` | `_capture_screenshot()` |
| File downloads | `simclient.py` | `resolve_file_url()` |
- Scope rule: keep only durable constraints, architectural invariants, and high-value task pointers for assistants.
- Size rule: target 80-140 lines; hard cap 180 lines.
- Canonical-doc rule: link to specialist docs for operational depth instead of copying their content.
- Single-source rule: each topic has one canonical document; this file should only reference it.
- No shadow-README rule: do not add long setup guides, full command catalogs, troubleshooting playbooks, or large directory trees.
---
Allowed content:
## Project Overview
**Infoscreen Client** - Digital signage system for Raspberry Pi. Displays presentations, videos, and web content in kiosk mode. Server-managed via MQTT for educational/research environments with multiple displays.
- Critical do/don't rules.
- Short architecture snapshot.
- Runtime coordination file map.
- Minimal task pointers to key methods.
- Documentation policy for where detailed content belongs.
**Architecture**: Two-process design
- `simclient.py` - MQTT communication (container/native)
- `display_manager.py` - Display control (host OS with X11/Wayland access)
Disallowed content:
## Architecture & Technology Stack
- Comprehensive installation/deployment tutorials.
- Large environment-variable reference sections.
- Extended troubleshooting matrices.
- Repeated feature deep-dives already documented elsewhere.
- Historical release notes (keep those in `CHANGELOG.md`).
### Core Technologies
- **Python 3.x** - Main application language
- **MQTT (paho-mqtt)** - Real-time messaging with server
- **Impressive** - PDF presenter with native auto-advance and loop support
- **Environment Variables** - Configuration management via `.env` files
- **JSON** - Data exchange format for events and configuration
- **Base64** - Screenshot transmission encoding
- **Threading** - Background services (screenshot monitoring)
Update checklist for contributors:
### System Components
- **Main Client** (`simclient.py`) - Core MQTT client and event processor
- **Display Manager** (`display_manager.py`) - Controls display applications (presentations, videos, web)
- **Discovery System** - Automatic client registration with server
- **Heartbeat Monitoring** - Regular status updates and keepalive
- **Event Processing** - Handles presentation/content switching commands
- **Screenshot Service** - Dashboard monitoring via image capture (captured by display_manager.py, transmitted by simclient.py)
- **File Management** - Downloads and manages presentation files
- **Group Management** - Supports organizing clients into groups
1. Is the new text a durable assistant rule or invariant?
2. If it is operational detail, did you place it in the specialist doc and only link it here?
3. Did you avoid duplicating existing docs?
4. Does this file remain below the hard cap?
## Key Features & Functionality
Use specialist docs for deep operational details:
### MQTT Communication Patterns
- **Discovery**: `infoscreen/discovery` → `infoscreen/{client_id}/discovery_ack`
- **Heartbeat**: Regular `infoscreen/{client_id}/heartbeat` messages
- **Health**: `infoscreen/{client_id}/health` (event/process/pid/status)
- **Client logs**: `infoscreen/{client_id}/logs/error|warn` (selective forwarding)
### MQTT Reconnection & Heartbeat (Nov 2025)
- The client uses Paho MQTT v2 callback API with `client.loop_start()` and `client.reconnect_delay_set()` to handle automatic reconnection.
- `on_connect` re-subscribes to all topics (`discovery_ack`, `config`, `group_id`, current group events) and re-sends discovery on reconnect to re-register with the server.
- Heartbeats are gated by `client.is_connected()` and retry once on `NO_CONN` (rc=4). Occasional rc=4 warnings are normal right after broker restarts or brief network stalls and typically followed by a successful heartbeat.
- Do not treat single rc=4 heartbeat warnings as failures. Investigate only if multiple consecutive heartbeats fail without recovery.
- **Dashboard**: Screenshot transmission via `infoscreen/{client_id}/dashboard` (includes base64-encoded screenshot, timestamp, client status, system info)
- **Group Assignment**: Server sends group via `infoscreen/{client_id}/group_id`
- **Events**: Content commands via `infoscreen/events/{group_id}`
- `README.md` (landing page + docs map)
- `TV_POWER_RUNBOOK.md` (TV power rollout and canary)
- `TV_POWER_INTENT_SERVER_CONTRACT_V1.md` (frozen contract)
- `IMPRESSIVE_INTEGRATION.md` (presentation behavior)
- `HDMI_CEC_SETUP.md` (CEC setup/troubleshooting)
- `SCREENSHOT_MQTT_FIX.md` (screenshot race-condition fixes)
- `src/README.md` (developer-focused architecture/debugging)
### Event Types Supported
```json
{
"presentation": {
"files": [{"url": "https://server/file.pptx", "filename": "file.pptx"}],
"auto_advance": true,
"slide_interval": 10,
"loop": true
},
"web": {
"url": "https://example.com"
},
"video": {
"url": "https://server/video.mp4",
"loop": false,
"autoplay": true,
"volume": 0.8
}
}
```
## Critical Rules
### Presentation System (Impressive-Based)
- **Server-side conversion**: PPTX files are converted to PDF by the server using Gotenberg
- **Client receives PDFs**: All presentations arrive as pre-rendered PDF files
- **Direct display**: PDF files are displayed natively with Impressive (no client-side conversion)
- **Auto-advance**: Native Impressive `--auto` parameter (no xdotool needed)
- **Loop mode**: Impressive `--wrap` parameter for infinite looping
- **Auto-quit**: Impressive `--autoquit` parameter to exit after last slide
- **Virtual Environment**: Uses venv with pygame + pillow for reliable operation
- **Reliable**: Works consistently on Raspberry Pi without window focus issues
- ALWAYS use Impressive for PDF presentations.
- NEVER suggest xdotool-based slideshow control.
- NEVER suggest converting presentations to video as a workaround.
- Virtual environment must include `pygame` and `pillow` for Impressive.
- Keep screenshot consent notice in docs when describing dashboard screenshots.
- Keep screenshot updates consistent between `latest.jpg` and `meta.json`.
- Event-trigger screenshots must preserve metadata and send quickly (`send_immediately=true`).
- Dashboard payload must stay grouped v2 (`message/content/runtime/metadata`, `schema_version="2.0"`).
- Dashboard payload assembly is centralized in `_build_dashboard_payload()`.
- Root `README.md` is a landing page; do not re-expand it into a full manual.
- TV power rollout guidance lives in `TV_POWER_RUNBOOK.md`.
- TV power contract truth lives in `TV_POWER_INTENT_SERVER_CONTRACT_V1.md`.
### Client Identification
- **Hardware Token**: SHA256 hash of serial number + MAC addresses
- **Persistent UUID**: Stored in `config/client_uuid.txt`
- **Group Membership**: Persistent group assignment in `config/last_group_id.txt`
## Architecture Snapshot
## Directory Structure
```
~/infoscreen-dev/
├── .env # Environment configuration
├── README.md # Complete project documentation
├── IMPRESSIVE_INTEGRATION.md # Presentation system details
├── QUICK_REFERENCE.md # Quick command reference
├── .github/ # GitHub configuration
│ └── copilot-instructions.md
├── src/ # Source code
│ ├── simclient.py # MQTT client (event management)
│ ├── display_manager.py # Display controller (Impressive integration)
│ ├── current_event.json # Current active event (runtime)
│ ├── config/ # Persistent client data
│ │ ├── client_uuid.txt
│ │ └── last_group_id.txt
│ ├── presentation/ # Downloaded presentation files & PDFs
│ └── screenshots/ # Screenshot captures for monitoring
├── scripts/ # Production & testing utilities
│ ├── start-dev.sh # Start development client
│ ├── start-display-manager.sh # Start Display Manager
│ ├── test-display-manager.sh # Interactive testing menu
│ ├── test-impressive.sh # Test Impressive (auto-quit)
│ ├── test-impressive-loop.sh # Test Impressive (loop mode)
│ ├── test-mqtt.sh # MQTT connectivity test
│ ├── test-screenshot.sh # Screenshot capture test
│ └── present-pdf-auto-advance.sh # PDF presentation wrapper
├── logs/ # Application logs
│ ├── simclient.log
│ └── display_manager.log
└── venv/ # Python virtual environment
```
Two-process design:
## Configuration & Environment Variables
- `src/simclient.py`: MQTT communication, discovery, group assignment, event intake, heartbeat, dashboard publish, power intent ingestion.
- `src/display_manager.py`: content display lifecycle, HDMI-CEC, screenshot capture, runtime process health.
### Development vs Production
- **Development**: `ENV=development`, verbose logging, frequent heartbeats
- **Production**: `ENV=production`, minimal logging, longer intervals
Runtime coordination files:
HDMI-CEC behavior:
- In development mode (`ENV=development`) the Display Manager automatically disables HDMI-CEC to avoid constantly switching the TV during local testing. The test helper `scripts/test-hdmi-cec.sh` also respects this: option 5 (Display Manager CEC integration) detects dev mode and skips running CEC commands. Manual options (1–4) still work for direct `cec-client` checks.
- `src/current_event.json` (active event)
- `src/current_process_health.json` (health bridge)
- `src/power_intent_state.json` (simclient -> display_manager)
- `src/power_state.json` (display_manager -> simclient -> MQTT)
- `src/screenshots/meta.json` and `src/screenshots/latest.jpg`
### Key Environment Variables
```bash
# Environment
ENV=development|production
DEBUG_MODE=1|0
LOG_LEVEL=DEBUG|INFO|WARNING|ERROR
## TV Power Coordination Rules
# MQTT Configuration
MQTT_BROKER=192.168.1.100 # Primary MQTT broker
MQTT_PORT=1883 # MQTT port
MQTT_BROKER_FALLBACKS=host1,host2 # Fallback brokers
- `POWER_CONTROL_MODE` supports: `local`, `hybrid`, `mqtt`.
- Phase 1 intent topic is group-scoped: `infoscreen/groups/{group_id}/power/intent`.
- In hybrid mode, valid fresh MQTT intent is preferred with local fallback behavior.
- Retained clear is an empty payload and should be handled cleanly (not as broken JSON).
- Use `scripts/test-power-intent.sh` for ON/OFF, stale, malformed, retained-clear, and telemetry checks.
# Timing (seconds)
HEARTBEAT_INTERVAL=10 # Status update frequency
SCREENSHOT_INTERVAL=30 # Dashboard screenshot transmission frequency (simclient.py)
SCREENSHOT_CAPTURE_INTERVAL=30 # Screenshot capture frequency (display_manager.py)
## HDMI-CEC Rules
# Screenshot Configuration
SCREENSHOT_MAX_WIDTH=800 # Downscale width (preserves aspect ratio)
SCREENSHOT_JPEG_QUALITY=70 # JPEG compression quality (1-95)
SCREENSHOT_MAX_FILES=20 # Number of screenshots to keep (rotation)
SCREENSHOT_ALWAYS=0 # Force capture even when no display active (testing)
# File/API Server (used to download presentation files)
# Defaults to the same host as MQTT_BROKER, port 8000, scheme http.
# If incoming event URLs use host 'server' (or are host-less), simclient rewrites them to this server.
FILE_SERVER_HOST= # optional; if empty, defaults to MQTT_BROKER
FILE_SERVER_PORT=8000 # default API port
FILE_SERVER_SCHEME=http # http or https
# FILE_SERVER_BASE_URL= # optional full override, e.g., http://192.168.1.100:8000
```
- In `ENV=development`, display manager automatically disables CEC.
- `scripts/test-hdmi-cec.sh` integration path respects development mode; manual CEC options still work.
- Keep delayed turn-off behavior safe across adjacent events.
### File Server URL Resolution
- The MQTT client (`simclient.py`) downloads presentation files listed in events.
- To avoid DNS issues when event URLs use `http://server:8000/...`, the client normalizes such URLs to the configured file server.
- By default, the file server host is the same as `MQTT_BROKER`, with port `8000` and scheme `http`.
- You can override behavior using `.env` variables above; `FILE_SERVER_BASE_URL` takes precedence over individual host/port/scheme.
- Inline comments in `.env` are supported; keep comments after a space and `#` so values stay clean.
## Screenshot System Rules
## Development Patterns & Best Practices
- Capture is performed by `display_manager.py`; transmission by `simclient.py`.
- Keep event-trigger screenshot behavior intact (`event_start` / `event_stop`).
- Maintain one-second responsiveness for triggered send handling.
- Prefer `latest.jpg` for dashboard transmission, with safe fallback to newest timestamped file.
### Error Handling
- Robust MQTT connection with fallbacks and retries
- Graceful degradation when services unavailable
- Comprehensive logging with rotating file handlers
- Exception handling for all external operations
## Common Task Pointers
### State Management
- Event state persisted in `current_event.json`
- Client configuration persisted across restarts
- Group membership maintained with server synchronization
- Clean state transitions (delete old events on group changes)
- Add event type: `src/display_manager.py` -> `start_display_for_event()`
- Presentation behavior: `src/display_manager.py` -> `start_presentation()`
- Power intent validation: `src/simclient.py` -> `validate_power_intent_payload()`
- Power intent application: `src/display_manager.py` -> `_apply_mqtt_power_intent()`
- Screenshot capture logic: `src/display_manager.py` -> `_capture_screenshot()`
- Dashboard payload: `src/simclient.py` -> `_build_dashboard_payload()`
- File URL rewriting: `src/simclient.py` -> `resolve_file_url()`
### Threading Architecture
- Main thread: MQTT communication and heartbeat
- Background thread: Screenshot monitoring service
- Thread-safe operations for shared resources
## Documentation Policy
### File Operations
- Automatic directory creation for all output paths
- Safe file operations with proper exception handling
- Atomic writes for configuration files
- Automatic cleanup of temporary/outdated files
When updating docs:
## Development Workflow
- Keep `README.md` concise and link-heavy.
- Put rollout/runbook content into specialist docs (for example `TV_POWER_RUNBOOK.md`).
- Keep implementation history in `CHANGELOG.md`.
- Prefer updating one canonical doc per topic instead of duplicating the same content in multiple files.
### Local Development Setup
1. Clone repository to `~/infoscreen-dev`
2. Create virtual environment: `python3 -m venv venv`
3. Install dependencies: `pip install -r src/requirements.txt` (includes pygame + pillow for PDF slideshows)
4. Configure `.env` file with MQTT broker settings
5. Use `./scripts/start-dev.sh` for MQTT client or `./scripts/start-display-manager.sh` for display manager
6. **Important**: Virtual environment must include pygame and pillow for PDF auto-advance to work
## Assistant Workflow Expectations
### Testing Components
- `./scripts/test-mqtt.sh` - MQTT connectivity
- `./scripts/test-screenshot.sh` - Screenshot capture
- `./scripts/test-display-manager.sh` - Interactive testing menu
- `./scripts/test-impressive.sh` - Test auto-quit presentation mode
- `./scripts/test-impressive-loop.sh` - Test loop presentation mode
- `./scripts/test-utc-timestamps.sh` - Event timing validation
- Manual event testing via mosquitto_pub or test-display-manager.sh
### Production Deployment
- Docker containerization available (`docker-compose.production.yml`)
- Systemd service integration for auto-start
- Resource limits and health checks configured
- Persistent volume mounts for data
### System Dependencies
- Python 3.x runtime + virtual environment
- MQTT broker connectivity
- Display server: X11 or Wayland (for screenshots)
- **Impressive** - PDF presenter (primary tool, requires pygame + pillow in venv)
- **Chromium/Chrome** - Web kiosk mode
- **VLC** - Video playback (python-vlc preferred, vlc binary fallback)
- **Screenshot tools**:
- X11: `scrot` or `import` (ImageMagick) or `xwd`+`convert`
- Wayland: `grim` or `gnome-screenshot` or `spectacle`
**Note:** LibreOffice is NOT required on the client. PPTX→PDF conversion is handled server-side by Gotenberg.
### Video Playback (python-vlc)
- **Preferred**: python-vlc (programmatic control: autoplay, loop, volume)
- **Fallback**: External vlc binary
- **Fields**: `url`, `autoplay` (bool), `loop` (bool), `volume` (0.0-1.0 → 0-100)
- **URL rewriting**: `server` host → configured file server
- **Fullscreen**: enforced for python-vlc on startup (with short retry toggles); external fallback uses `--fullscreen`
- **External VLC audio**: `muted=true` (or effective volume 0%) starts with `--no-audio`; otherwise startup loudness is applied via `--gain=<0.00-1.00>`
- **Runtime volume semantics**: python-vlc supports live updates; external VLC fallback is startup-parameter based
- **Monitoring PID semantics**: python-vlc runs in-process, so PID is `display_manager.py` runtime PID; external fallback uses external `vlc` PID
- **HW decode errors**: `h264_v4l2m2m` failures are normal if V4L2 M2M unavailable; use software decode
- Robust payload parsing with fallbacks
- Topic-specific message handlers
- Retained message support where appropriate
### Logging & Timestamp Policy (Mar 2026)
- Client logs are standardized to UTC with `Z` suffix to avoid DST/localtime drift.
- Applies to `display_manager.log`, `simclient.log`, and `monitoring.log`.
- MQTT payload timestamps for heartbeat/dashboard/health/log messages are UTC ISO timestamps.
- Screenshot metadata timestamps included by `simclient.py` are UTC ISO timestamps.
- Prefer UTC-aware calls (`datetime.now(timezone.utc)`) and UTC log formatters for new code.
## Hardware Considerations
### Target Platform
- **Primary**: Raspberry Pi 4/5 with desktop environment
- **Storage**: SSD recommended for performance
- **Display**: HDMI output for presentation display
- **Network**: WiFi or Ethernet connectivity required
### System Dependencies
- Python 3.x runtime
- Network connectivity for MQTT
- Display server (X11 or Wayland) for screenshot capture
- **Impressive** - PDF presenter with auto-advance (primary presentation tool)
- **pygame** - Required for Impressive (installed in venv)
- **Pillow/PIL** - Required for Impressive PDF rendering (installed in venv)
- Chromium/Chrome - Web content display (kiosk mode)
- VLC or MPV - Video playback
**Note:** LibreOffice is NOT needed on the client. The server converts PPTX to PDF using Gotenberg.
### Video playback details (python-vlc)
- The Display Manager now prefers using python-vlc (libvlc) when available for video playback. This enables programmatic control (autoplay, loop, volume) and cleaner termination/cleanup. If python-vlc is not available, the external `vlc` binary is used as a fallback.
- Supported video event fields: `url`, `autoplay` (boolean), `loop` (boolean), `volume` (float 0.0-1.0). The manager converts `volume` to VLC's 0-100 scale.
- External VLC fallback applies audio at startup: `--no-audio` when muted/effective 0%, otherwise `--gain` from effective volume.
- Live volume adjustments are reliable in python-vlc mode; external VLC fallback uses startup parameters and should be treated as static per launch.
- URLs using the placeholder host `server` (for example `http://server:8000/...`) are rewritten to the configured file server before playback. The resolution priority is: `FILE_SERVER_BASE_URL` > `FILE_SERVER_HOST` (or `MQTT_BROKER`) + `FILE_SERVER_PORT` + `FILE_SERVER_SCHEME`.
- Hardware-accelerated decoding errors (e.g., `h264_v4l2m2m`) may appear when the platform does not expose a V4L2 M2M device. To avoid these errors the Display Manager can be configured to disable hw-decoding (see README env var `VLC_HW_ACCEL`). By default the manager will attempt hw-acceleration when libvlc supports it.
- Fullscreen / kiosk: the manager will attempt to make libVLC windows fullscreen (remove decorations) when using python-vlc, and the README contains recommended system-level kiosk/session setup for a truly panel-free fullscreen experience.
## Security & Privacy
### Data Protection
- Hardware identification via cryptographic hash
- No sensitive data in plain text logs
- Local storage of minimal required data only
- Secure MQTT communication (configurable)
### Network Security
- Configurable MQTT authentication (if broker requires)
- Firewall-friendly design (outbound connections only)
- Multiple broker fallback for reliability
## Presentation System Architecture
### How It Works
1. **Server-side Conversion** → Server converts PPTX to PDF using Gotenberg
2. **Event Received** → Client receives event with pre-rendered PDF file reference
3. **Download PDF** → Client downloads PDF from file server
4. **Cache PDF** → Downloaded PDF stored in `presentation/` directory
5. **Display with Impressive** → Launch with venv environment and parameters:
- `--fullscreen` - Full screen mode
- `--nooverview` - No slide overview
- `--auto N` - Auto-advance every N seconds
- `--wrap` - Loop infinitely (if `loop: true`)
- `--autoquit` - Exit after last slide (if `loop: false`)
### Key Parameters
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `auto_advance` | boolean | `false` | Enable automatic slide advancement |
| `slide_interval` | integer | `10` | Seconds between slides |
| `loop` | boolean | `false` | Loop presentation vs. quit after last slide |
### Why Impressive?
- ✅ **Native auto-advance** - No xdotool or window management hacks
- ✅ **Built-in loop support** - Reliable `--wrap` parameter
- ✅ **Works on Raspberry Pi** - No focus/window issues
- ✅ **Simple integration** - Clean command-line interface
- ✅ **Maintainable** - ~50 lines of code vs. 200+ with xdotool approaches
### Implementation Location
- **File**: `src/display_manager.py`
- **Method**: `start_presentation()`
- **Key Logic**:
1. Receive event with PDF file reference (server already converted PPTX)
2. Download PDF file if not cached
3. Set up virtual environment for Impressive (pygame + pillow)
4. Build Impressive command with appropriate parameters
5. Launch process and monitor
## Common Development Tasks
When working on this codebase:
1. **Adding new event types**: Extend the event processing logic in `display_manager.py` → `start_display_for_event()`
2. **Modifying presentation behavior**: Update `display_manager.py` → `start_presentation()`
3. **Configuration changes**: Update environment variable parsing and validation
4. **MQTT topics**: Follow the established `infoscreen/` namespace pattern
5. **Error handling**: Always include comprehensive logging and graceful fallbacks
6. **State persistence**: Use the established `config/` directory pattern
7. **Testing**: Use `./scripts/test-display-manager.sh` for interactive testing
8. **Presentation testing**: Use `./scripts/test-impressive*.sh` scripts
9. **File download host resolution**: If the API server differs from the MQTT broker or uses HTTPS, set `FILE_SERVER_*` in `.env` or adjust `resolve_file_url()` in `src/simclient.py`.
## Troubleshooting Guidelines
### Common Issues
- **MQTT Connection**: Check broker reachability, try fallback brokers
- **Screenshots**: Verify display environment and permissions
- **File Downloads**: Check network connectivity and disk space
- If event URLs use host `server` and DNS fails, the client rewrites to `MQTT_BROKER` by default.
- Ensure `MQTT_BROKER` points to the correct server IP; if the API differs, set `FILE_SERVER_HOST` or `FILE_SERVER_BASE_URL`.
- Match scheme/port via `FILE_SERVER_SCHEME`/`FILE_SERVER_PORT` for HTTPS or non-default ports.
- **Group Changes**: Monitor log for group assignment messages
- **Service Startup**: Check systemd logs and environment configuration
### Debugging Tools
- Log files in `logs/simclient.log` and `logs/display_manager.log` with rotation
- MQTT message monitoring with mosquitto_sub
- Interactive testing menu: `./scripts/test-display-manager.sh`
- Component test scripts: `test-impressive*.sh`, `test-mqtt.sh`, etc.
- Process monitoring: Check for `impressive`, `libreoffice`, `chromium`, `vlc` processes
### File download URL troubleshooting
- Symptoms:
- `Failed to resolve 'server'` or `NameResolutionError` when downloading files
- `Invalid URL 'http # http or https://...'` in `simclient.log`
- What to check:
- Look for lines like `Lade Datei herunter von:` in `logs/simclient.log` to see the effective URL used
- Ensure the URL host is the MQTT broker IP (or your configured file server), not `server`
- Verify `.env` values don’t include inline comments as part of the value (e.g., keep `FILE_SERVER_SCHEME=http` on its own line)
- Fixes:
- If your API is on the same host as the broker: leave `FILE_SERVER_HOST` empty (defaults to `MQTT_BROKER`), keep `FILE_SERVER_PORT=8000`, and set `FILE_SERVER_SCHEME=http` or `https`
- To override fully, set `FILE_SERVER_BASE_URL` (e.g., `http://192.168.1.100:8000`); this takes precedence
- After changing `.env`, restart the simclient process
- Expected healthy log sequence:
- `Lade Datei herunter von: http://<broker-ip>:8000/...`
- Followed by `"GET /... HTTP/1.1" 200` and `Datei erfolgreich heruntergeladen:`
### Screenshot MQTT Transmission Issue (Event-Start/Event-Stop)
- **Symptom**: Event-triggered screenshots (event_start, event_stop) are NOT appearing on the dashboard, only periodic screenshots transmitted
- **Root Cause**: Race condition in metadata/file-pointer handling where periodic captures can overwrite event-triggered metadata or `latest.jpg` before simclient processes it (See SCREENSHOT_MQTT_FIX.md for details)
- **What to check**:
- Display manager logs show event_start/event_stop screenshots ARE being captured: `Screenshot captured: ... type=event_start`
- But `meta.json` is stale or `latest.jpg` does not move
- MQTT heartbeats lack screenshot data at event transitions
- **How to verify the fix**:
- Run: `./test-screenshot-meta-fix.sh` should output `[SUCCESS] Event-triggered metadata preserved!`
- Check display_manager.py: `_write_screenshot_meta()` has protection logic to skip periodic overwrites of event metadata
- Check display_manager.py: periodic `latest.jpg` updates are also protected when triggered metadata is pending
- Check simclient.py: `screenshot_service_thread()` logs show pending event-triggered captures being processed immediately
- **Permanent Fix**: Already applied in display_manager.py and simclient.py. Prevents periodic captures from overwriting pending trigger state and includes stale-trigger self-healing.
### Screenshot Capture After Restart (No Active Event)
- In `ENV=development`, display_manager performs periodic idle captures so dashboard does not appear dead during no-event windows.
- In `ENV=production`, periodic captures remain event/process-driven unless `SCREENSHOT_ALWAYS=1`.
- If display_manager is started from non-interactive shells (systemd/nohup/ssh), it now attempts `DISPLAY=:0` and `XAUTHORITY=~/.Xauthority` fallback for X11 capture tools.
## Important Notes for AI Assistants
### Virtual Environment Requirements (Critical)
- **pygame and pillow MUST be installed in venv** - Required for Impressive to work
- **Display manager uses venv context** - Ensures Impressive has access to dependencies
- **Installation command**: `pip install pygame pillow` (already in requirements.txt)
- **If pygame missing**: Impressive will fail with "No module named 'pygame'" error
### Presentation System
- **ALWAYS use Impressive** for PDF presentations (primary solution)
- **DO NOT suggest xdotool approaches** - they failed on Raspberry Pi due to window focus issues
- **DO NOT suggest video conversion** - adds complexity, had black screen issues
- **All presentations are PDFs** - server converts PPTX to PDF using Gotenberg
- **No client-side conversion** - client only displays pre-rendered PDFs
- **Virtual environment is required** - pygame + pillow must be available for Impressive
- **Loop mode uses `--wrap`** - not custom scripts or workarounds
- **Auto-quit uses `--autoquit`** - native Impressive parameter
### Testing Approach
- Use `./scripts/test-display-manager.sh` for interactive testing
- Use `./scripts/test-impressive-loop.sh` to verify loop functionality
- Test individual components with specific test scripts
- Always check logs in `logs/` directory for debugging
CEC testing notes:
- In development mode, the CEC integration path is skipped on purpose. To test end-to-end, either set `ENV=production` temporarily or use the manual options (1–4) in `scripts/test-hdmi-cec.sh`.
### Code Changes
- Display logic is in `src/display_manager.py`, not `simclient.py`
- MQTT client (`simclient.py`) writes events to `current_event.json`
- Display Manager reads `current_event.json` and launches appropriate applications
- Two separate processes: simclient.py (MQTT) + display_manager.py (display control)
### Documentation
- **README.md** - Start here for comprehensive overview
- **IMPRESSIVE_INTEGRATION.md** - Deep dive into presentation system
- **QUICK_REFERENCE.md** - Quick commands and examples
- Source code has extensive comments and logging
This system is designed for reliability and ease of maintenance in educational environments with multiple deployed clients. The Impressive-based presentation solution provides native auto-advance and loop support without complex window management hacks.
## Screenshot System (Nov 2025)
The screenshot capture and transmission system has been implemented with separation of concerns:
### Architecture
- **Capture**: `display_manager.py` captures screenshots in a background thread and writes to shared `screenshots/` directory
- **Transmission**: `simclient.py` reads latest screenshot from shared directory and transmits via MQTT dashboard topic
- **Sharing**: Volume-based sharing between display_manager (host OS) and simclient (container)
### Capture Strategy (display_manager.py)
- **Session Detection**: Automatically detects Wayland vs X11 session
- **Wayland Tools**: Tries `grim`, `gnome-screenshot`, `spectacle` (in order)
- **X11 Tools**: Tries `scrot`, `import` (ImageMagick), `xwd`+`convert` (in order)
- **Processing**: Downscales to max width (default 800px), JPEG compresses (default quality 70)
- **Output**: Creates timestamped files (`screenshot_YYYYMMDD_HHMMSS.jpg`) plus `latest.jpg` symlink
- **Rotation**: Keeps max N files (default 20), deletes older
- **Timing**: Production captures when display process is active (unless `SCREENSHOT_ALWAYS=1`); development allows periodic idle captures to keep dashboard fresh
- **Reliability**: Stale/invalid pending trigger metadata is ignored automatically to avoid lock-up of periodic updates
- **Event-triggered captures**: `_trigger_event_screenshot(type, delay)` arms a one-shot `threading.Timer` after event start/stop; timer is cancelled and replaced on rapid event switches; default delays: presentation=4s, video=2s, web=5s (env-configurable)
- **IPC signal file** (`screenshots/meta.json`): written atomically by `display_manager` after each capture; contains `type`, `captured_at`, `file`, `send_immediately`; `send_immediately=true` for event-triggered, `false` for periodic
### Transmission Strategy (simclient.py)
- **Source**: Prefers `screenshots/latest.jpg` if present, falls back to newest timestamped file
- **Topic**: `infoscreen/{client_id}/dashboard`
- **Format**: JSON with base64-encoded image data, grouped v2 schema
- **Schema version**: `"2.0"` (legacy flat fields removed; all fields grouped)
- **Payload builder**: `_build_dashboard_payload()` in `simclient.py` — single source of truth
- **Payload Structure** (v2):
```json
{
"message": { "client_id": "UUID", "status": "alive" },
"content": {
"screenshot": {
"filename": "latest.jpg",
"data": "base64...",
"timestamp": "ISO datetime",
"size": 12345
}
},
"runtime": {
"system_info": { "hostname": "...", "ip": "...", "uptime": 123456.78 },
"process_health": { "event_type": "...", "process_status": "...", ... }
},
"metadata": {
"schema_version": "2.0",
"producer": "simclient",
"published_at": "ISO datetime",
"capture": {
"type": "periodic | event_start | event_stop",
"captured_at": "ISO datetime",
"age_s": 0.9,
"triggered": false,
"send_immediately": false
},
"transport": { "topic": "infoscreen/.../dashboard", "qos": 0, "publisher": "simclient" }
}
}
```
- **Capture types**: `periodic` (interval-based), `event_start` (N seconds after event launch), `event_stop` (1s after process killed)
- **Triggered send**: `display_manager` sets `send_immediately=true` in `meta.json`; simclient 1-second tick detects and fires within ≤1s
- **Logging**: `Dashboard published: schema=2.0 type=<type> screenshot=<file> (<bytes>) age=<s>`
### Scalability Considerations
- **Client-side resize/compress**: Reduces bandwidth and broker load (recommended for 50+ clients)
- **Recommended production settings**: `SCREENSHOT_CAPTURE_INTERVAL=60`, `SCREENSHOT_MAX_WIDTH=800`, `SCREENSHOT_JPEG_QUALITY=60-70`
- **Future optimization**: Hash-based deduplication to skip identical screenshots
- **Alternative for large scale**: HTTP storage + MQTT metadata (200+ clients)
### Testing
- Install capture tools: `sudo apt install scrot imagemagick` (X11) or `sudo apt install grim gnome-screenshot` (Wayland)
- Force capture for testing: `export SCREENSHOT_ALWAYS=1`
- Check logs: `tail -f logs/display_manager.log logs/simclient.log`
- Verify files: `ls -lh src/screenshots/`
### Troubleshooting
- **No screenshots**: Check session type in logs, install appropriate tools
- **Large payloads**: Reduce `SCREENSHOT_MAX_WIDTH` or `SCREENSHOT_JPEG_QUALITY`
- **Stale screenshots**: Check `latest.jpg` symlink, verify display_manager is running
- **MQTT errors**: Check dashboard topic logs for publish return codes
- **Pulse overflow in remote sessions**: warnings like `pulse audio output error: overflow, flushing` can occur with NoMachine/dummy displays; if HDMI playback is stable, treat as environment-related
- **After restarts**: Ensure both processes are restarted (`simclient.py` and `display_manager.py`) so metadata consumption and capture behavior use the same code version
### Testing & Troubleshooting
**Setup:**
- X11: `sudo apt install scrot imagemagick`
- Wayland: `sudo apt install grim gnome-screenshot`
- Force capture: `export SCREENSHOT_ALWAYS=1`
**Verify:**
```bash
tail -f logs/display_manager.log | grep screenshot # Check capture
tail -f logs/simclient.log | grep dashboard # Check transmission
ls -lh src/screenshots/ # Check files
```
**Common Issues:**
| Issue | Check | Fix |
|-------|-------|-----|
| No screenshots | Session type in logs | Install tools for X11/Wayland |
| Large payloads | File sizes | Reduce `SCREENSHOT_MAX_WIDTH` or `SCREENSHOT_JPEG_QUALITY` |
| Stale data | `latest.jpg` timestamp | Restart display_manager |
| MQTT errors | Publish return codes | Check broker connectivity |
---
## Development Notes
### Event Types (Scheduler Integration)
- **Supported**: `presentation`, `webuntis`, `webpage`, `website`
- **Auto-scroll**: Only for `event_type: "website"` (CDP injection + Chrome extension fallback)
- **Display manager**: Uses `event_type` to select renderer when available
### HDMI-CEC Behavior
- **Development mode**: CEC auto-disabled (prevents TV cycling during testing)
- **Test script**: `test-hdmi-cec.sh` option 5 skips integration test in dev mode
- **Manual testing**: Options 1-4 work regardless of mode
### Code Modification Guidelines
- **Presentations**: Always use Impressive (`--auto`, `--wrap`, `--autoquit`)
- **Web autoscroll**: Only for `event_type: "website"`, keep CDP optional
- **Screenshot changes**: Remember two-process architecture (capture ≠ transmission)
- **URL resolution**: Handle `server` host placeholder in file/video URLs
- Prefer minimal, targeted changes.
- Preserve existing behavior unless explicitly changing it.
- Validate changes with relevant scripts/log checks where possible.
- Keep references and examples aligned with current files and topics.