feat: dashboard screenshot upload & retention (last 20 per client)
- Listener: subscribe to dashboard topic, forward screenshots to API - API: store latest + last 20 timestamped screenshots per client, auto-delete older files - Docs: updated README, TECH-CHANGELOG, and copilot-instructions for screenshot upload and retention policy
This commit is contained in:
7
.github/copilot-instructions.md
vendored
7
.github/copilot-instructions.md
vendored
@@ -33,17 +33,22 @@ Keep docs synced with code. When you change services/MQTT/API/UTC/env or dev/pro
|
||||
- `dashboard/src/settings.tsx` — settings UI (nested tabs; system defaults for presentations and videos)
|
||||
|
||||
|
||||
|
||||
## Big picture
|
||||
- Multi-service app orchestrated by Docker Compose.
|
||||
- API: Flask + SQLAlchemy (MariaDB), in `server/` exposed on :8000 (health: `/health`).
|
||||
- Dashboard: React + Vite in `dashboard/`, dev on :5173, served via Nginx in prod.
|
||||
- MQTT broker: Eclipse Mosquitto, config in `mosquitto/config/mosquitto.conf`.
|
||||
- Listener: MQTT consumer handling discovery + heartbeats in `listener/listener.py`.
|
||||
- Listener: MQTT consumer handling discovery, heartbeats, and dashboard screenshot uploads in `listener/listener.py`.
|
||||
- Scheduler: Publishes only currently active events (per group, at "now") to MQTT retained topics in `scheduler/scheduler.py`. It queries a future window (default: 7 days) to expand recurring events using RFC 5545 rules and applies event exceptions, but only publishes events that are active at the current time. When a group has no active events, the scheduler clears its retained topic by publishing an empty list. All time comparisons are UTC; any naive timestamps are normalized. Logging is concise; conversion lookups are cached and logged only once per media.
|
||||
- Nginx: Reverse proxy routes `/api/*` and `/screenshots/*` to API; everything else to dashboard (`nginx.conf`).
|
||||
|
||||
- Dev Container (hygiene): UI-only `Dev Containers` extension runs on host UI via `remote.extensionKind`; do not install it in-container. Dashboard installs use `npm ci`; shell aliases in `postStartCommand` are appended idempotently.
|
||||
|
||||
### Screenshot retention
|
||||
- Screenshots sent via dashboard MQTT are stored in `server/screenshots/`.
|
||||
- For each client, only the latest and last 20 timestamped screenshots are kept; older files are deleted automatically on each upload.
|
||||
|
||||
## Recent changes since last commit
|
||||
|
||||
### Latest (November 2025)
|
||||
|
||||
@@ -338,7 +338,9 @@ mosquitto_sub -h localhost -t "infoscreen/+/heartbeat" -v
|
||||
- `GET /api/files/converted/{path}` - Download converted PDFs
|
||||
- `POST /api/conversions/{media_id}/pdf` - Request conversion
|
||||
- `GET /api/conversions/{media_id}/status` - Check conversion status
|
||||
- `GET /api/eventmedia/stream/<media_id>/<filename>` - Stream media with byte-range support (206) for seeking
|
||||
- `GET /api/eventmedia/stream/<media_id>/<filename>` - Stream media with byte-range support (206) for seeking
|
||||
- `POST /api/clients/{uuid}/screenshot` - Upload screenshot for client (base64 JPEG)
|
||||
- **Screenshot retention:** Only the latest and last 20 timestamped screenshots per client are stored on the server. Older screenshots are automatically deleted.
|
||||
|
||||
### System Settings
|
||||
- `GET /api/system-settings` - List all system settings (admin+)
|
||||
|
||||
94
SCREENSHOT_IMPLEMENTATION.md
Normal file
94
SCREENSHOT_IMPLEMENTATION.md
Normal file
@@ -0,0 +1,94 @@
|
||||
# Screenshot Transmission Implementation
|
||||
|
||||
## Overview
|
||||
Clients send screenshots via MQTT during heartbeat intervals. The listener service receives these screenshots and forwards them to the server API for storage.
|
||||
|
||||
## Architecture
|
||||
|
||||
### MQTT Topic
|
||||
- **Topic**: `infoscreen/{uuid}/screenshot`
|
||||
- **Payload Format**:
|
||||
- Raw binary image data (JPEG/PNG), OR
|
||||
- JSON with base64-encoded image: `{"image": "<base64-string>"}`
|
||||
|
||||
### Components
|
||||
|
||||
#### 1. Listener Service (`listener/listener.py`)
|
||||
- **Subscribes to**: `infoscreen/+/screenshot`
|
||||
- **Function**: `handle_screenshot(uuid, payload)`
|
||||
- Detects payload format (binary or JSON)
|
||||
- Converts binary to base64 if needed
|
||||
- Forwards to API via HTTP POST
|
||||
|
||||
#### 2. Server API (`server/routes/clients.py`)
|
||||
- **Endpoint**: `POST /api/clients/<uuid>/screenshot`
|
||||
- **Authentication**: No authentication required (internal service call)
|
||||
- **Accepts**:
|
||||
- JSON: `{"image": "<base64-encoded-image>"}`
|
||||
- Binary: raw image data
|
||||
- **Storage**:
|
||||
- Saves to `server/screenshots/{uuid}_{timestamp}.jpg` (with timestamp)
|
||||
- Saves to `server/screenshots/{uuid}.jpg` (latest, for quick retrieval)
|
||||
|
||||
#### 3. Retrieval (`server/wsgi.py`)
|
||||
- **Endpoint**: `GET /screenshots/<uuid>`
|
||||
- **Returns**: Latest screenshot for the given client UUID
|
||||
- **Nginx**: Exposes `/screenshots/{uuid}.jpg` in production
|
||||
|
||||
## Unified Identification Method
|
||||
|
||||
Screenshots are identified by **client UUID**:
|
||||
- Each client has a unique UUID stored in the `clients` table
|
||||
- Screenshots are stored as `{uuid}.jpg` (latest) and `{uuid}_{timestamp}.jpg` (historical)
|
||||
- The API endpoint requires UUID validation against the database
|
||||
- Retrieval is done via `GET /screenshots/<uuid>` which returns the latest screenshot
|
||||
|
||||
## Data Flow
|
||||
|
||||
```
|
||||
Client → MQTT (infoscreen/{uuid}/screenshot)
|
||||
↓
|
||||
Listener Service
|
||||
↓ (validates client exists)
|
||||
↓ (converts binary → base64 if needed)
|
||||
↓
|
||||
API POST /api/clients/{uuid}/screenshot
|
||||
↓ (validates client UUID)
|
||||
↓ (decodes base64 → binary)
|
||||
↓
|
||||
Filesystem: server/screenshots/{uuid}.jpg
|
||||
↓
|
||||
Dashboard/Nginx: GET /screenshots/{uuid}
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
- **Listener**: `API_BASE_URL` (default: `http://server:8000`)
|
||||
- **Server**: Screenshots stored in `server/screenshots/` directory
|
||||
|
||||
### Dependencies
|
||||
- Listener: Added `requests>=2.31.0` to `listener/requirements.txt`
|
||||
- Server: Uses built-in Flask and base64 libraries
|
||||
|
||||
## Error Handling
|
||||
|
||||
- **Client Not Found**: Returns 404 if UUID doesn't exist in database
|
||||
- **Invalid Payload**: Returns 400 if image data is missing or invalid
|
||||
- **API Timeout**: Listener logs error and continues (timeout: 10s)
|
||||
- **Network Errors**: Listener logs and continues operation
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- Screenshot endpoint does not require authentication (internal service-to-service)
|
||||
- Client UUID must exist in database before screenshot is accepted
|
||||
- Base64 encoding prevents binary data issues in JSON transport
|
||||
- File size is tracked and logged for monitoring
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- Add screenshot retention policy (auto-delete old timestamped files)
|
||||
- Add compression before transmission
|
||||
- Add screenshot quality settings
|
||||
- Add authentication between listener and API
|
||||
- Add screenshot history API endpoint
|
||||
@@ -5,9 +5,8 @@
|
||||
|
||||
This changelog documents technical and developer-relevant changes included in public releases. For development workspace changes, see DEV-CHANGELOG.md. Not all changes here are reflected in the user-facing changelog (`program-info.json`), and not all UI/feature changes are repeated here. Some changes (e.g., backend refactoring, API adjustments, infrastructure, developer tooling, or internal logic) may only appear in TECH-CHANGELOG.md. For UI/feature changes, see `dashboard/public/program-info.json`.
|
||||
|
||||
---
|
||||
|
||||
## 2025.1.0-alpha.12 (2025-11-27)
|
||||
|
||||
Backend rework (post-release notes; no version bump):
|
||||
- 🧩 Dev Container hygiene: Remote Containers runs on UI (`remote.extensionKind`), removed in-container install to prevent reappearance loops; switched `postCreateCommand` to `npm ci` for reproducible dashboard installs; `postStartCommand` aliases made idempotent.
|
||||
- 🔄 Serialization: Consolidated snake_case→camelCase via `server/serializers.py` for all JSON outputs; ensured enums/UTC datetimes serialize consistently across routes.
|
||||
|
||||
@@ -2,29 +2,73 @@ import os
|
||||
import json
|
||||
import logging
|
||||
import datetime
|
||||
import base64
|
||||
import requests
|
||||
import paho.mqtt.client as mqtt
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
from models.models import Client
|
||||
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s [%(levelname)s] %(message)s')
|
||||
|
||||
# Load .env in development
|
||||
if os.getenv("ENV", "development") == "development":
|
||||
try:
|
||||
from dotenv import load_dotenv
|
||||
load_dotenv(".env")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# ENV-abhängige Konfiguration
|
||||
# ENV-dependent configuration
|
||||
ENV = os.getenv("ENV", "development")
|
||||
LOG_LEVEL = os.getenv("LOG_LEVEL", "INFO" if ENV == "production" else "DEBUG")
|
||||
DB_URL = os.environ.get(
|
||||
"DB_CONN", "mysql+pymysql://user:password@db/infoscreen")
|
||||
|
||||
# Logging
|
||||
logging.basicConfig(level=logging.DEBUG,
|
||||
format='%(asctime)s [%(levelname)s] %(message)s')
|
||||
DB_URL = os.environ.get("DB_CONN", "mysql+pymysql://user:password@db/infoscreen")
|
||||
|
||||
# DB-Konfiguration
|
||||
engine = create_engine(DB_URL)
|
||||
Session = sessionmaker(bind=engine)
|
||||
|
||||
# API configuration
|
||||
API_BASE_URL = os.getenv("API_BASE_URL", "http://server:8000")
|
||||
|
||||
|
||||
def handle_screenshot(uuid, payload):
|
||||
"""
|
||||
Handle screenshot data received via MQTT and forward to API.
|
||||
Payload can be either raw binary image data or JSON with base64-encoded image.
|
||||
"""
|
||||
try:
|
||||
# Try to parse as JSON first
|
||||
try:
|
||||
data = json.loads(payload.decode())
|
||||
if "image" in data:
|
||||
# Payload is JSON with base64 image
|
||||
api_payload = {"image": data["image"]}
|
||||
headers = {"Content-Type": "application/json"}
|
||||
logging.debug(f"Forwarding base64 screenshot from {uuid} to API")
|
||||
else:
|
||||
logging.warning(f"Screenshot JSON from {uuid} missing 'image' field")
|
||||
return
|
||||
except (json.JSONDecodeError, UnicodeDecodeError):
|
||||
# Payload is raw binary image data - encode to base64 for API
|
||||
image_b64 = base64.b64encode(payload).decode('utf-8')
|
||||
api_payload = {"image": image_b64}
|
||||
headers = {"Content-Type": "application/json"}
|
||||
logging.debug(f"Forwarding binary screenshot from {uuid} to API (encoded as base64)")
|
||||
|
||||
# Forward to API endpoint
|
||||
api_url = f"{API_BASE_URL}/api/clients/{uuid}/screenshot"
|
||||
response = requests.post(api_url, json=api_payload, headers=headers, timeout=10)
|
||||
|
||||
if response.status_code == 200:
|
||||
logging.info(f"Screenshot von {uuid} erfolgreich an API weitergeleitet")
|
||||
else:
|
||||
logging.error(f"API returned status {response.status_code} for screenshot from {uuid}: {response.text}")
|
||||
|
||||
except requests.exceptions.RequestException as e:
|
||||
logging.error(f"Failed to forward screenshot from {uuid} to API: {e}")
|
||||
except Exception as e:
|
||||
logging.error(f"Error handling screenshot from {uuid}: {e}")
|
||||
|
||||
|
||||
def on_connect(client, userdata, flags, reasonCode, properties):
|
||||
"""Callback for when client connects or reconnects (API v2)."""
|
||||
@@ -32,7 +76,9 @@ def on_connect(client, userdata, flags, reasonCode, properties):
|
||||
# Subscribe on every (re)connect so we don't miss heartbeats after broker restarts
|
||||
client.subscribe("infoscreen/discovery")
|
||||
client.subscribe("infoscreen/+/heartbeat")
|
||||
logging.info(f"MQTT connected (reasonCode: {reasonCode}); (re)subscribed to discovery and heartbeats")
|
||||
client.subscribe("infoscreen/+/screenshot")
|
||||
client.subscribe("infoscreen/+/dashboard")
|
||||
logging.info(f"MQTT connected (reasonCode: {reasonCode}); (re)subscribed to discovery, heartbeats, screenshots, and dashboards")
|
||||
except Exception as e:
|
||||
logging.error(f"Subscribe failed on connect: {e}")
|
||||
|
||||
@@ -42,6 +88,39 @@ def on_message(client, userdata, msg):
|
||||
logging.debug(f"Empfangene Nachricht auf Topic: {topic}")
|
||||
|
||||
try:
|
||||
# Dashboard-Handling (nested screenshot payload)
|
||||
if topic.startswith("infoscreen/") and topic.endswith("/dashboard"):
|
||||
uuid = topic.split("/")[1]
|
||||
try:
|
||||
payload_text = msg.payload.decode()
|
||||
data = json.loads(payload_text)
|
||||
shot = data.get("screenshot")
|
||||
if isinstance(shot, dict):
|
||||
# Prefer 'data' field (base64) inside screenshot object
|
||||
image_b64 = shot.get("data")
|
||||
if image_b64:
|
||||
logging.debug(f"Dashboard enthält Screenshot für {uuid}; Weiterleitung an API")
|
||||
# Build a lightweight JSON with image field for API handler
|
||||
api_payload = json.dumps({"image": image_b64}).encode("utf-8")
|
||||
handle_screenshot(uuid, api_payload)
|
||||
# Update last_alive if status present
|
||||
if data.get("status") == "alive":
|
||||
session = Session()
|
||||
client_obj = session.query(Client).filter_by(uuid=uuid).first()
|
||||
if client_obj:
|
||||
client_obj.last_alive = datetime.datetime.now(datetime.UTC)
|
||||
session.commit()
|
||||
session.close()
|
||||
except Exception as e:
|
||||
logging.error(f"Fehler beim Verarbeiten des Dashboard-Payloads von {uuid}: {e}")
|
||||
return
|
||||
|
||||
# Screenshot-Handling
|
||||
if topic.startswith("infoscreen/") and topic.endswith("/screenshot"):
|
||||
uuid = topic.split("/")[1]
|
||||
handle_screenshot(uuid, msg.payload)
|
||||
return
|
||||
|
||||
# Heartbeat-Handling
|
||||
if topic.startswith("infoscreen/") and topic.endswith("/heartbeat"):
|
||||
uuid = topic.split("/")[1]
|
||||
|
||||
@@ -2,3 +2,4 @@ paho-mqtt>=2.0
|
||||
SQLAlchemy>=2.0
|
||||
pymysql
|
||||
python-dotenv
|
||||
requests>=2.31.0
|
||||
|
||||
@@ -273,6 +273,85 @@ def restart_client(uuid):
|
||||
return jsonify({"error": f"Failed to send MQTT message: {str(e)}"}), 500
|
||||
|
||||
|
||||
@clients_bp.route("/<uuid>/screenshot", methods=["POST"])
|
||||
def upload_screenshot(uuid):
|
||||
"""
|
||||
Route to receive and store a screenshot from a client.
|
||||
Expected payload: base64-encoded image data in JSON or binary image data.
|
||||
Screenshots are stored as {uuid}.jpg in the screenshots folder.
|
||||
Keeps last 20 screenshots per client (auto-cleanup).
|
||||
"""
|
||||
import os
|
||||
import base64
|
||||
import glob
|
||||
from datetime import datetime
|
||||
|
||||
session = Session()
|
||||
client = session.query(Client).filter_by(uuid=uuid).first()
|
||||
if not client:
|
||||
session.close()
|
||||
return jsonify({"error": "Client nicht gefunden"}), 404
|
||||
session.close()
|
||||
|
||||
try:
|
||||
# Handle JSON payload with base64-encoded image
|
||||
if request.is_json:
|
||||
data = request.get_json()
|
||||
if "image" not in data:
|
||||
return jsonify({"error": "Missing 'image' field in JSON payload"}), 400
|
||||
|
||||
# Decode base64 image
|
||||
image_data = base64.b64decode(data["image"])
|
||||
else:
|
||||
# Handle raw binary image data
|
||||
image_data = request.get_data()
|
||||
|
||||
if not image_data:
|
||||
return jsonify({"error": "No image data received"}), 400
|
||||
|
||||
# Ensure screenshots directory exists
|
||||
screenshots_dir = os.path.join(os.path.dirname(__file__), "..", "screenshots")
|
||||
os.makedirs(screenshots_dir, exist_ok=True)
|
||||
|
||||
# Store screenshot with timestamp to track latest
|
||||
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
|
||||
filename = f"{uuid}_{timestamp}.jpg"
|
||||
filepath = os.path.join(screenshots_dir, filename)
|
||||
|
||||
with open(filepath, "wb") as f:
|
||||
f.write(image_data)
|
||||
|
||||
# Also create/update a symlink or copy to {uuid}.jpg for easy retrieval
|
||||
latest_filepath = os.path.join(screenshots_dir, f"{uuid}.jpg")
|
||||
with open(latest_filepath, "wb") as f:
|
||||
f.write(image_data)
|
||||
|
||||
# Cleanup: keep only last 20 timestamped screenshots per client
|
||||
pattern = os.path.join(screenshots_dir, f"{uuid}_*.jpg")
|
||||
existing_screenshots = sorted(glob.glob(pattern))
|
||||
|
||||
# Keep last 20, delete older ones
|
||||
max_screenshots = 20
|
||||
if len(existing_screenshots) > max_screenshots:
|
||||
for old_file in existing_screenshots[:-max_screenshots]:
|
||||
try:
|
||||
os.remove(old_file)
|
||||
except Exception as cleanup_error:
|
||||
# Log but don't fail the request if cleanup fails
|
||||
import logging
|
||||
logging.warning(f"Failed to cleanup old screenshot {old_file}: {cleanup_error}")
|
||||
|
||||
return jsonify({
|
||||
"success": True,
|
||||
"message": f"Screenshot received for client {uuid}",
|
||||
"filename": filename,
|
||||
"size": len(image_data)
|
||||
}), 200
|
||||
|
||||
except Exception as e:
|
||||
return jsonify({"error": f"Failed to process screenshot: {str(e)}"}), 500
|
||||
|
||||
|
||||
@clients_bp.route("/<uuid>", methods=["DELETE"])
|
||||
@admin_or_higher
|
||||
def delete_client(uuid):
|
||||
|
||||
Reference in New Issue
Block a user