olafn/infoscreen-dev
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
8ca9f69f6f11dfabac29083ecd2fec311c1b5e9a
infoscreen-dev/.github/copilot-instructions.md

20 KiB
Raw Blame History

Copilot Instructions - Infoscreen Client

Project Overview

This is an Infoscreen Client system for Raspberry Pi that creates digital signage displays. The client communicates with a server via MQTT to display presentations, videos, and web content in kiosk mode. It's designed for educational/research environments where multiple displays need to be centrally managed.

Architecture & Technology Stack

Core Technologies

  • Python 3.x - Main application language
  • MQTT (paho-mqtt) - Real-time messaging with server
  • Impressive - PDF presenter with native auto-advance and loop support
  • LibreOffice - PPTX to PDF conversion (headless mode)
  • Environment Variables - Configuration management via .env files
  • JSON - Data exchange format for events and configuration
  • Base64 - Screenshot transmission encoding
  • Threading - Background services (screenshot monitoring)

System Components

  • Main Client (simclient.py) - Core MQTT client and event processor
  • Display Manager (display_manager.py) - Controls display applications (presentations, videos, web)
  • Discovery System - Automatic client registration with server
  • Heartbeat Monitoring - Regular status updates and keepalive
  • Event Processing - Handles presentation/content switching commands
  • Screenshot Service - Dashboard monitoring via image capture
  • File Management - Downloads and manages presentation files
  • Group Management - Supports organizing clients into groups

Key Features & Functionality

MQTT Communication Patterns

  • Discovery: infoscreen/discovery → infoscreen/{client_id}/discovery_ack
  • Heartbeat: Regular infoscreen/{client_id}/heartbeat messages
  • Dashboard: Screenshot transmission via infoscreen/{client_id}/dashboard
  • Group Assignment: Server sends group via infoscreen/{client_id}/group_id
  • Events: Content commands via infoscreen/events/{group_id}

Event Types Supported

{
  "presentation": {
    "files": [{"url": "https://server/file.pptx", "filename": "file.pptx"}],
    "auto_advance": true,
    "slide_interval": 10,
    "loop": true
  },
  "web": {
    "url": "https://example.com"
  },
  "video": {
    "url": "https://server/video.mp4",
    "loop": false,
    "autoplay": true,
    "volume": 0.8
  }
}

Presentation System (Impressive-Based)

  • PDF files are displayed natively with Impressive PDF presenter (no conversion needed)
  • PPTX files are automatically converted to PDF using LibreOffice headless
  • Auto-advance: Native Impressive --auto parameter (no xdotool needed)
  • Loop mode: Impressive --wrap parameter for infinite looping
  • Auto-quit: Impressive --autoquit parameter to exit after last slide
  • Virtual Environment: Uses venv with pygame + pillow for reliable operation
  • Reliable: Works consistently on Raspberry Pi without window focus issues

Client Identification

  • Hardware Token: SHA256 hash of serial number + MAC addresses
  • Persistent UUID: Stored in config/client_uuid.txt
  • Group Membership: Persistent group assignment in config/last_group_id.txt

Directory Structure

~/infoscreen-dev/
├── .env                    # Environment configuration
├── README.md               # Complete project documentation
├── IMPRESSIVE_INTEGRATION.md  # Presentation system details
├── QUICK_REFERENCE.md      # Quick command reference
├── .github/                # GitHub configuration
│   └── copilot-instructions.md
├── src/                    # Source code
│   ├── simclient.py       # MQTT client (event management)
│   ├── display_manager.py # Display controller (Impressive integration)
│   ├── current_event.json # Current active event (runtime)
│   ├── config/            # Persistent client data
│   │   ├── client_uuid.txt
│   │   └── last_group_id.txt
│   ├── presentation/      # Downloaded presentation files & PDFs
│   └── screenshots/       # Screenshot captures for monitoring
├── scripts/               # Production & testing utilities
│   ├── start-dev.sh       # Start development client
│   ├── start-display-manager.sh  # Start Display Manager
│   ├── test-display-manager.sh   # Interactive testing menu
│   ├── test-impressive.sh        # Test Impressive (auto-quit)
│   ├── test-impressive-loop.sh   # Test Impressive (loop mode)
│   ├── test-mqtt.sh       # MQTT connectivity test
│   ├── test-screenshot.sh # Screenshot capture test
│   └── present-pdf-auto-advance.sh  # PDF presentation wrapper
├── logs/                  # Application logs
│   ├── simclient.log
│   └── display_manager.log
└── venv/                  # Python virtual environment

Configuration & Environment Variables

Development vs Production

  • Development: ENV=development, verbose logging, frequent heartbeats
  • Production: ENV=production, minimal logging, longer intervals

Key Environment Variables

# Environment
ENV=development|production
DEBUG_MODE=1|0
LOG_LEVEL=DEBUG|INFO|WARNING|ERROR

# MQTT Configuration
MQTT_BROKER=192.168.1.100        # Primary MQTT broker
MQTT_PORT=1883                   # MQTT port
MQTT_BROKER_FALLBACKS=host1,host2  # Fallback brokers

# Timing (seconds)
HEARTBEAT_INTERVAL=10            # Status update frequency
SCREENSHOT_INTERVAL=30           # Dashboard screenshot frequency
 
# File/API Server (used to download presentation files)
# Defaults to the same host as MQTT_BROKER, port 8000, scheme http.
# If incoming event URLs use host 'server' (or are host-less), simclient rewrites them to this server.
FILE_SERVER_HOST=                # optional; if empty, defaults to MQTT_BROKER
FILE_SERVER_PORT=8000            # default API port
FILE_SERVER_SCHEME=http          # http or https
# FILE_SERVER_BASE_URL=           # optional full override, e.g., http://192.168.1.100:8000

File Server URL Resolution

  • The MQTT client (simclient.py) downloads presentation files listed in events.
  • To avoid DNS issues when event URLs use http://server:8000/..., the client normalizes such URLs to the configured file server.
  • By default, the file server host is the same as MQTT_BROKER, with port 8000 and scheme http.
  • You can override behavior using .env variables above; FILE_SERVER_BASE_URL takes precedence over individual host/port/scheme.
  • Inline comments in .env are supported; keep comments after a space and # so values stay clean.

Development Patterns & Best Practices

Error Handling

  • Robust MQTT connection with fallbacks and retries
  • Graceful degradation when services unavailable
  • Comprehensive logging with rotating file handlers
  • Exception handling for all external operations

State Management

  • Event state persisted in current_event.json
  • Client configuration persisted across restarts
  • Group membership maintained with server synchronization
  • Clean state transitions (delete old events on group changes)

Threading Architecture

  • Main thread: MQTT communication and heartbeat
  • Background thread: Screenshot monitoring service
  • Thread-safe operations for shared resources

File Operations

  • Automatic directory creation for all output paths
  • Safe file operations with proper exception handling
  • Atomic writes for configuration files
  • Automatic cleanup of temporary/outdated files

Development Workflow

Local Development Setup

  1. Clone repository to ~/infoscreen-dev
  2. Create virtual environment: python3 -m venv venv
  3. Install dependencies: pip install -r src/requirements.txt (includes pygame + pillow for PDF slideshows)
  4. Configure .env file with MQTT broker settings
  5. Use ./scripts/start-dev.sh for MQTT client or ./scripts/start-display-manager.sh for display manager
  6. Important: Virtual environment must include pygame and pillow for PDF auto-advance to work

Testing Components

  • ./scripts/test-mqtt.sh - MQTT connectivity
  • ./scripts/test-screenshot.sh - Screenshot capture
  • ./scripts/test-display-manager.sh - Interactive testing menu
  • ./scripts/test-impressive.sh - Test auto-quit presentation mode
  • ./scripts/test-impressive-loop.sh - Test loop presentation mode
  • ./scripts/test-utc-timestamps.sh - Event timing validation
  • Manual event testing via mosquitto_pub or test-display-manager.sh

Production Deployment

  • Docker containerization available (docker-compose.production.yml)
  • Systemd service integration for auto-start
  • Resource limits and health checks configured
  • Persistent volume mounts for data

Code Style & Conventions

Python Code Standards

  • Environment variable parsing with fallback defaults
  • Comprehensive docstrings for complex functions
  • Logging at appropriate levels (DEBUG/INFO/WARNING/ERROR)
  • Type hints where beneficial for clarity
  • Exception handling with specific error types

Configuration Management

  • Environment-first configuration (12-factor app principles)
  • Support for inline comments in environment values
  • Graceful handling of missing/invalid configuration
  • Multiple environment file locations for flexibility

MQTT Message Handling

  • JSON message format validation
  • Robust payload parsing with fallbacks
  • Topic-specific message handlers
  • Retained message support where appropriate

Hardware Considerations

Target Platform

  • Primary: Raspberry Pi 4/5 with desktop environment
  • Storage: SSD recommended for performance
  • Display: HDMI output for presentation display
  • Network: WiFi or Ethernet connectivity required

System Dependencies

  • Python 3.x runtime
  • Network connectivity for MQTT
  • Display server (X11) for screenshot capture
  • Impressive - PDF presenter with auto-advance (primary presentation tool)
  • pygame - Required for Impressive (installed in venv)
  • Pillow/PIL - Required for Impressive PDF rendering (installed in venv)
  • LibreOffice - PPTX to PDF conversion (headless mode)
  • Chromium/Chrome - Web content display (kiosk mode)
  • VLC or MPV - Video playbook

Video playback details (python-vlc)

  • The Display Manager now prefers using python-vlc (libvlc) when available for video playback. This enables programmatic control (autoplay, loop, volume) and cleaner termination/cleanup. If python-vlc is not available, the external vlc binary is used as a fallback.
  • Supported video event fields: url, autoplay (boolean), loop (boolean), volume (float 0.0-1.0). The manager converts volume to VLC's 0-100 scale.
  • URLs using the placeholder host server (for example http://server:8000/...) are rewritten to the configured file server before playback. The resolution priority is: FILE_SERVER_BASE_URL > FILE_SERVER_HOST (or MQTT_BROKER) + FILE_SERVER_PORT + FILE_SERVER_SCHEME.
  • Hardware-accelerated decoding errors (e.g., h264_v4l2m2m) may appear when the platform does not expose a V4L2 M2M device. To avoid these errors the Display Manager can be configured to disable hw-decoding (see README env var VLC_HW_ACCEL). By default the manager will attempt hw-acceleration when libvlc supports it.
  • Fullscreen / kiosk: the manager will attempt to make libVLC windows fullscreen (remove decorations) when using python-vlc, and the README contains recommended system-level kiosk/session setup for a truly panel-free fullscreen experience.

Security & Privacy

Data Protection

  • Hardware identification via cryptographic hash
  • No sensitive data in plain text logs
  • Local storage of minimal required data only
  • Secure MQTT communication (configurable)

Network Security

  • Configurable MQTT authentication (if broker requires)
  • Firewall-friendly design (outbound connections only)
  • Multiple broker fallback for reliability

Presentation System Architecture

How It Works

  1. PDF File Received → Event contains PDF file reference → Use directly (no conversion)
  2. PPTX File Received → Event contains PPTX file reference
  3. Convert to PDF → LibreOffice headless: libreoffice --headless --convert-to pdf
  4. Cache PDF → Converted PDF stored in presentation/ directory
  5. Display with Impressive → Launch with venv environment and parameters:
    • --fullscreen - Full screen mode
    • --nooverview - No slide overview
    • --auto N - Auto-advance every N seconds
    • --wrap - Loop infinitely (if loop: true)
    • --autoquit - Exit after last slide (if loop: false)

Key Parameters

Parameter Type Default Description
auto_advance boolean false Enable automatic slide advancement
slide_interval integer 10 Seconds between slides
loop boolean false Loop presentation vs. quit after last slide

Why Impressive?

  • ✅ Native auto-advance - No xdotool or window management hacks
  • ✅ Built-in loop support - Reliable --wrap parameter
  • ✅ Works on Raspberry Pi - No focus/window issues
  • ✅ Simple integration - Clean command-line interface
  • ✅ Maintainable - ~50 lines of code vs. 200+ with xdotool approaches

Implementation Location

  • File: src/display_manager.py
  • Method: start_presentation()
  • Key Logic:
    1. Check if PDF file (use directly) or PPTX (needs conversion)
    2. Convert PPTX to PDF if needed (cached for reuse)
    3. Set up virtual environment for Impressive (pygame + pillow)
    4. Build Impressive command with appropriate parameters
    5. Launch process and monitor

Common Development Tasks

When working on this codebase:

  1. Adding new event types: Extend the event processing logic in display_manager.py → start_display_for_event()
  2. Modifying presentation behavior: Update display_manager.py → start_presentation()
  3. Configuration changes: Update environment variable parsing and validation
  4. MQTT topics: Follow the established infoscreen/ namespace pattern
  5. Error handling: Always include comprehensive logging and graceful fallbacks
  6. State persistence: Use the established config/ directory pattern
  7. Testing: Use ./scripts/test-display-manager.sh for interactive testing
  8. Presentation testing: Use ./scripts/test-impressive*.sh scripts
  9. File download host resolution: If the API server differs from the MQTT broker or uses HTTPS, set FILE_SERVER_* in .env or adjust resolve_file_url() in src/simclient.py.

Troubleshooting Guidelines

Common Issues

  • MQTT Connection: Check broker reachability, try fallback brokers
  • Screenshots: Verify display environment and permissions
  • File Downloads: Check network connectivity and disk space
    • If event URLs use host server and DNS fails, the client rewrites to MQTT_BROKER by default.
    • Ensure MQTT_BROKER points to the correct server IP; if the API differs, set FILE_SERVER_HOST or FILE_SERVER_BASE_URL.
    • Match scheme/port via FILE_SERVER_SCHEME/FILE_SERVER_PORT for HTTPS or non-default ports.
  • Group Changes: Monitor log for group assignment messages
  • Service Startup: Check systemd logs and environment configuration

Debugging Tools

  • Log files in logs/simclient.log and logs/display_manager.log with rotation
  • MQTT message monitoring with mosquitto_sub
  • Interactive testing menu: ./scripts/test-display-manager.sh
  • Component test scripts: test-impressive*.sh, test-mqtt.sh, etc.
  • Process monitoring: Check for impressive, libreoffice, chromium, vlc processes

File download URL troubleshooting

  • Symptoms:
    • Failed to resolve 'server' or NameResolutionError when downloading files
    • Invalid URL 'http # http or https://...' in simclient.log
  • What to check:
    • Look for lines like Lade Datei herunter von: in logs/simclient.log to see the effective URL used
    • Ensure the URL host is the MQTT broker IP (or your configured file server), not server
    • Verify .env values don’t include inline comments as part of the value (e.g., keep FILE_SERVER_SCHEME=http on its own line)
  • Fixes:
    • If your API is on the same host as the broker: leave FILE_SERVER_HOST empty (defaults to MQTT_BROKER), keep FILE_SERVER_PORT=8000, and set FILE_SERVER_SCHEME=http or https
    • To override fully, set FILE_SERVER_BASE_URL (e.g., http://192.168.1.100:8000); this takes precedence
    • After changing .env, restart the simclient process
  • Expected healthy log sequence:
    • Lade Datei herunter von: http://<broker-ip>:8000/...
    • Followed by "GET /... HTTP/1.1" 200 and Datei erfolgreich heruntergeladen:

Important Notes for AI Assistants

Virtual Environment Requirements (Critical)

  • pygame and pillow MUST be installed in venv - Required for Impressive to work
  • Display manager uses venv context - Ensures Impressive has access to dependencies
  • Installation command: pip install pygame pillow (already in requirements.txt)
  • If pygame missing: Impressive will fail with "No module named 'pygame'" error

Presentation System

  • ALWAYS use Impressive for PDF presentations (primary solution)
  • DO NOT suggest xdotool approaches - they failed on Raspberry Pi due to window focus issues
  • DO NOT suggest video conversion - adds complexity, had black screen issues
  • PDF files are supported natively - no conversion needed
  • PPTX conversion is automatic - LibreOffice headless handles PPTX→PDF
  • Virtual environment is required - pygame + pillow must be available for Impressive
  • Loop mode uses --wrap - not custom scripts or workarounds
  • Auto-quit uses --autoquit - native Impressive parameter

Testing Approach

  • Use ./scripts/test-display-manager.sh for interactive testing
  • Use ./scripts/test-impressive-loop.sh to verify loop functionality
  • Test individual components with specific test scripts
  • Always check logs in logs/ directory for debugging

Code Changes

  • Display logic is in src/display_manager.py, not simclient.py
  • MQTT client (simclient.py) writes events to current_event.json
  • Display Manager reads current_event.json and launches appropriate applications
  • Two separate processes: simclient.py (MQTT) + display_manager.py (display control)

Documentation

  • README.md - Start here for comprehensive overview
  • IMPRESSIVE_INTEGRATION.md - Deep dive into presentation system
  • QUICK_REFERENCE.md - Quick commands and examples
  • Source code has extensive comments and logging

This system is designed for reliability and ease of maintenance in educational environments with multiple deployed clients. The Impressive-based presentation solution provides native auto-advance and loop support without complex window management hacks.

Recent changes (Oct 2025)

Summary of additions since the last Copilot instructions update:

  • Added event_type handling for scheduler events: presentation, webuntis, webpage, website. The display manager now uses event_type when available to pick the correct display path.
  • Implemented auto-scroll for long websites. Two strategies are available:
    • CDP injection via DevTools websocket using websocket-client and requests. The injector attempts to evaluate an auto-scroll script inside the page context. Default duration: 60s.
    • Local Chrome extension fallback (src/chrome_autoscroll) loaded with --load-extension when CDP handshake is blocked (403), ensuring reliable autoscroll operation.
  • New helper scripts for debugging DevTools handshake: scripts/test_cdp.py and scripts/test_cdp_origins.py.
  • Updated requirements.txt to include websocket-client.

Notes for Copilot:

  • When adding or modifying display logic, prefer Impressive for PDF presentations and avoid xdotool approaches for slideshow control.
  • For web events, ensure autoscroll is only activated for event_type: "website" and keep the CDP injection optional/fallback-only when feasible.
Reference in New Issue View Git Blame Copy Permalink