From c19f478f11f6bf6306b278ef17dd05625bcb1d06 Mon Sep 17 00:00:00 2001 From: olaf Date: Sun, 14 Sep 2025 06:36:37 +0000 Subject: [PATCH] add copilot instructions for better use of AI-models --- .env.example | 36 ++++++++++++ .github/copilot-instructions.md | 83 ++++++++++++++++++++++++++ .gitignore | 22 +++++++ AI-INSTRUCTIONS-MAINTENANCE.md | 100 ++++++++++++++++++++++++++++++++ Makefile | 82 ++++++++++++++++++++++++++ deployment.md | 45 +++++++++++++- 6 files changed, 366 insertions(+), 2 deletions(-) create mode 100644 .env.example create mode 100644 .github/copilot-instructions.md create mode 100644 AI-INSTRUCTIONS-MAINTENANCE.md create mode 100644 Makefile diff --git a/.env.example b/.env.example new file mode 100644 index 0000000..466ec04 --- /dev/null +++ b/.env.example @@ -0,0 +1,36 @@ +# Copy this file to .env and fill in values as needed for local development. +# NOTE: No secrets should be committed. Use placeholders below. + +# General +ENV=development + +# Database (used if DB_CONN not provided) +DB_USER=your_user +DB_PASSWORD=your_password +DB_NAME=infoscreen_by_taa +DB_HOST=db +# Preferred connection string for services (overrides the above if set) +# DB_CONN=mysql+pymysql://${DB_USER}:${DB_PASSWORD}@${DB_HOST}/${DB_NAME} + +# MQTT +MQTT_BROKER_HOST=mqtt +MQTT_BROKER_PORT=1883 +# MQTT_USER=your_mqtt_user +# MQTT_PASSWORD=your_mqtt_password +MQTT_KEEPALIVE=60 + +# Dashboard +# Used when building the production dashboard image +# VITE_API_URL=https://your.api.example.com/api + +# Groups alive windows (seconds) +HEARTBEAT_GRACE_PERIOD_DEV=15 +HEARTBEAT_GRACE_PERIOD_PROD=180 + +# Scheduler +# Optional: force periodic republish even without changes +# REFRESH_SECONDS=0 + +# Default admin bootstrap (server/init_defaults.py) +DEFAULT_ADMIN_USERNAME=infoscreen_admin +DEFAULT_ADMIN_PASSWORD=Info_screen_admin25! diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 0000000..8dee171 --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,83 @@ +# Copilot instructions for infoscreen_2025 + +Use this as your shared context when proposing changes. Keep edits minimal and match existing patterns referenced below. + +## Big picture +- Multi-service app orchestrated by Docker Compose. + - API: Flask + SQLAlchemy (MariaDB), in `server/` exposed on :8000 (health: `/health`). + - Dashboard: React + Vite in `dashboard/`, dev on :5173, served via Nginx in prod. + - MQTT broker: Eclipse Mosquitto, config in `mosquitto/config/mosquitto.conf`. + - Listener: MQTT consumer handling discovery + heartbeats in `listener/listener.py`. + - Scheduler: Publishes active events (per group) to MQTT retained topics in `scheduler/scheduler.py`. + - Nginx: Reverse proxy routes `/api/*` and `/screenshots/*` to API; everything else to dashboard (`nginx.conf`). + +## Service boundaries & data flow +- Database connection string is passed as `DB_CONN` (mysql+pymysql) to Python services. + - API builds its engine in `server/database.py` (loads `.env` only in development). + - Scheduler loads `DB_CONN` in `scheduler/db_utils.py`. + - Listener also creates its own engine for writes to `clients`. +- MQTT topics (paho-mqtt v2, use Callback API v2): + - Discovery: `infoscreen/discovery` (JSON includes `uuid`, hw/ip data). ACK to `infoscreen/{uuid}/discovery_ack`. See `listener/listener.py`. + - Heartbeat: `infoscreen/{uuid}/heartbeat` updates `Client.last_alive` (UTC). + - Event lists (retained): `infoscreen/events/{group_id}` from `scheduler/scheduler.py`. + - Per-client group assignment (retained): `infoscreen/{uuid}/group_id` via `server/mqtt_helper.py`. +- Screenshots: server-side folders `server/received_screenshots/` and `server/screenshots/`; Nginx exposes `/screenshots/{uuid}.jpg` via `server/wsgi.py` route. + +## Data model highlights (see `models/models.py`) +- Enums: `EventType` (presentation, website, video, message, webuntis) and `MediaType` (file/website types). +- Tables: `clients`, `client_groups`, `events`, `event_media`, `users`. +- Times are stored as timezone-aware; treat comparisons in UTC (see scheduler and routes/events). + +## API patterns +- Blueprints live in `server/routes/*` and are registered in `server/wsgi.py` with `/api/...` prefixes. +- Session usage: instantiate `Session()` per request, commit when mutating, and always `session.close()` before returning. +- Examples: + - Clients: `server/routes/clients.py` includes bulk group updates and MQTT sync (`publish_multiple_client_groups`). + - Groups: `server/routes/groups.py` computes “alive” using a grace period that varies by `ENV`. + - Events: `server/routes/events.py` serializes enum values to strings and normalizes times to UTC. + - Media: `server/routes/eventmedia.py` implements a simple file manager API rooted at `server/media/`. + +## Frontend patterns (dashboard) +- Vite React app; proxies `/api` and `/screenshots` to API in dev (`vite.config.ts`). +- Uses Syncfusion components; Vite config pre-bundles specific packages to avoid alias issues. +- Environment: `VITE_API_URL` provided at build/run; in dev compose, proxy handles `/api` so local fetches can use relative `/api/...` paths. + +## Local development +- Compose: development is `docker-compose.yml` + `docker-compose.override.yml`. + - API (dev): `server/Dockerfile.dev` with debugpy on 5678, Flask app `wsgi:app` on :8000. + - Dashboard (dev): `dashboard/Dockerfile.dev` exposes :5173 and waits for API via `dashboard/wait-for-backend.sh`. + - Mosquitto: allows anonymous in dev; WebSocket on :9001. +- Common env vars: `DB_CONN`, `DB_USER`, `DB_PASSWORD`, `DB_HOST=db`, `DB_NAME`, `ENV`, `MQTT_USER`, `MQTT_PASSWORD`. +- Alembic: prod compose runs `alembic ... upgrade head` and `server/init_defaults.py` before gunicorn. + +## Production +- `docker-compose.prod.yml` uses prebuilt images (`ghcr.io/robbstarkaustria/*`). +- Nginx serves dashboard and proxies API; TLS certs expected in `certs/` and mounted to `/etc/nginx/certs`. + +## Environment variables (reference) +- DB_CONN — Preferred DB URL for services. Example: `mysql+pymysql://${DB_USER}:${DB_PASSWORD}@db/${DB_NAME}` +- DB_USER, DB_PASSWORD, DB_NAME, DB_HOST — Used to assemble DB_CONN in dev if missing; inside containers `DB_HOST=db`. +- ENV — `development` or `production`; in development, `server/database.py` loads `.env`. +- MQTT_BROKER_HOST, MQTT_BROKER_PORT — Defaults `mqtt` and `1883`; MQTT_USER/MQTT_PASSWORD optional (dev often anonymous per Mosquitto config). +- VITE_API_URL — Dashboard build-time base URL (prod); in dev the Vite proxy serves `/api` to `server:8000`. +- HEARTBEAT_GRACE_PERIOD_DEV / HEARTBEAT_GRACE_PERIOD_PROD — Groups “alive” window (defaults ~15s dev / 180s prod). +- REFRESH_SECONDS — Optional scheduler republish interval; `0` disables periodic refresh. + +## Conventions & gotchas +- Always compare datetimes in UTC; some DB values may be naive—normalize before comparing (see `routes/events.py`). +- Use retained MQTT messages for state that clients must recover after reconnect (events per group, client group_id). +- In-container DB host is `db`; do not use `localhost` inside services. +- No separate dev vs prod secret conventions: use the same env var keys across environments (e.g., `DB_CONN`, `MQTT_USER`, `MQTT_PASSWORD`). +- When adding a new route: + 1) Create a Blueprint in `server/routes/...`, + 2) Register it in `server/wsgi.py`, + 3) Manage `Session()` lifecycle, and + 4) Return JSON-safe values (serialize enums and datetimes). +- When extending media types, update `MediaType` and any logic in `eventmedia` and dashboard that depends on it. + +## Quick examples +- Add client description persists to DB and publishes group via MQTT: see `PUT /api/clients//description` in `routes/clients.py`. +- Bulk group assignment emits retained messages for each client: `PUT /api/clients/group`. +- Listener heartbeat path: `infoscreen//heartbeat` → sets `clients.last_alive`. + +Questions or unclear areas? Tell us if you need: exact devcontainer debugging steps, stricter Alembic workflow, or a seed dataset beyond `init_defaults.py`. diff --git a/.gitignore b/.gitignore index 3149fa3..e132744 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,25 @@ +# OS/Editor +.DS_Store +Thumbs.db +.vscode/ +.idea/ + +# Python +__pycache__/ +*.pyc +.pytest_cache/ + +# Node +node_modules/ +dashboard/node_modules/ +dashboard/.vite/ + +# Env files (never commit secrets) +.env +.env.local + +# Docker +*.log # Python-related __pycache__/ *.py[cod] diff --git a/AI-INSTRUCTIONS-MAINTENANCE.md b/AI-INSTRUCTIONS-MAINTENANCE.md new file mode 100644 index 0000000..e0c5361 --- /dev/null +++ b/AI-INSTRUCTIONS-MAINTENANCE.md @@ -0,0 +1,100 @@ +# Maintaining AI Assistant Instructions (copilot-instructions.md) + +This repo uses `.github/copilot-instructions.md` to brief AI coding agents about your architecture, workflows, and conventions. Keep it concise, repo-specific, and always in sync with your code. + +This guide explains when and how to update it, plus small guardrails to help—even for a solo developer. + +## When to update +Update the instructions in the same commit as your change whenever you: +- Add/rename services, ports, or container wiring (docker-compose*.yml, Nginx, Mosquitto) +- Introduce/rename MQTT topics or change retained-message behavior +- Add/rename environment variables or change defaults (`.env.example`, `deployment.md`) +- Change DB models or time/UTC handling (e.g., `models/models.py`, UTC normalization in routes/scheduler) +- Add/modify API route patterns or session lifecycle (files in `server/routes/*`, `server/wsgi.py`) +- Adjust frontend dev proxy or build settings (`dashboard/vite.config.ts`, Dockerfiles) + +## What to update (and where) +- `.github/copilot-instructions.md` + - Big picture: services and ports + - Service boundaries & data flow: DB connection rules, MQTT topics, retained messages, screenshots + - API patterns: Blueprints, Session per request, enum/datetime serialization + - Frontend patterns: Vite dev proxy and pre-bundled dependencies + - Environment variables (reference): names, purposes, example patterns + - Conventions & gotchas: UTC comparisons, retained MQTT, container hostnames +- `.env.example` + - Add new variable names with placeholders and comments (never secrets) + - Keep in-container defaults (e.g., `DB_HOST=db`, `MQTT_BROKER_HOST=mqtt`) +- `deployment.md` + - Update Quickstart URLs/ports/commands + - Document prod-specific env usage (e.g., `VITE_API_URL`, `DB_CONN`) + +## How to write good updates +- Keep it short (approx. 20–50 lines total). Link to code by path or route rather than long prose. +- Document real, present patterns—not plans. +- Use UTC consistently and call out any special handling. +- Include concrete examples from this repo when describing patterns (e.g., which route shows enum serialization). +- Never include secrets or real tokens; show only variable names and example formats. + +## Solo-friendly workflow +- Update docs in the same commit as your change: + - Code changed → docs changed (copilot-instructions, `.env.example`, `deployment.md` as needed) +- Use a quick self-checklist before pushing: + - Services/ports changed? Update “Big picture”. + - MQTT topics/retained behavior changed? Update “Service boundaries & data flow”. + - API/Session/UTC rules changed? Update “API patterns” and “Conventions & gotchas”. + - Frontend proxy/build changed? Update “Frontend patterns”. + - Env vars changed? Update “Environment variables (reference)” + `.env.example`. + - Dev/prod run steps changed? Update `deployment.md` Quickstart. +- Keep commits readable by pairing code and doc changes: + - `feat(api): add events endpoint; docs: update routes and UTC note` + - `chore(compose): rename service; docs: update ports + nginx` + - `docs(env): add MQTT_USER to .env.example + instructions` + +## Optional guardrails (even for solo) +- PR (or MR) template (useful even if you self-merge) + - Add `.github/pull_request_template.md` with: +``` +Checklist +- [ ] Updated .github/copilot-instructions.md (services/MQTT/API/UTC/env) +- [ ] Synced .env.example (new/renamed vars) +- [ ] Adjusted deployment.md (dev/prod steps, URLs/ports) +- [ ] Verified referenced files/paths in the instructions exist +``` +- Lightweight docs check (optional pre-commit hook) + - Non-blocking script that warns if referenced files/paths don’t exist. Example sketch: +``` +#!/usr/bin/env bash +set -e +FILE=.github/copilot-instructions.md +missing=0 +for path in \ + server/wsgi.py \ + server/routes/clients.py \ + server/routes/events.py \ + server/routes/groups.py \ + dashboard/vite.config.ts \ + docker-compose.yml \ + docker-compose.override.yml; do + if ! test -e "$path"; then + echo "[warn] referenced path not found: $path"; missing=1 + fi +done +exit 0 # warn only; do not block commit +``` +- Weekly 2-minute sweep + - Read `.github/copilot-instructions.md` top-to-bottom and remove anything stale. + +## FAQ +- Where do the AI assistants look? + - `.github/copilot-instructions.md` + the code you have open. Keep this file synced with the codebase. +- Is it safe to commit this file? + - Yes—no secrets. It should contain only structure, patterns, and example formats. +- How detailed should it be? + - Concise and actionable; point to exact files for details. Avoid generic advice. + +## Pointers to key files +- Compose & infra: `docker-compose*.yml`, `nginx.conf`, `mosquitto/config/mosquitto.conf` +- Backend: `server/database.py`, `server/wsgi.py`, `server/routes/*`, `models/models.py` +- MQTT workers: `listener/listener.py`, `scheduler/scheduler.py`, `server/mqtt_helper.py` +- Frontend: `dashboard/vite.config.ts`, `dashboard/package.json`, `dashboard/src/*` +- Dev/Prod docs: `deployment.md`, `.env.example` diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..0295812 --- /dev/null +++ b/Makefile @@ -0,0 +1,82 @@ +# Makefile for infoscreen_2025 +# Usage: run `make help` to see available targets. + +# Default compose files +COMPOSE_FILES=-f docker-compose.yml -f docker-compose.override.yml +COMPOSE=docker compose $(COMPOSE_FILES) + +# Registry and image names (adjust if needed) +REGISTRY=ghcr.io/robbstarkaustria +API_IMAGE=$(REGISTRY)/infoscreen-api:latest +DASH_IMAGE=$(REGISTRY)/infoscreen-dashboard:latest +LISTENER_IMAGE=$(REGISTRY)/infoscreen-listener:latest +SCHED_IMAGE=$(REGISTRY)/infoscreen-scheduler:latest + +.PHONY: help +help: + @echo "Available targets:" + @echo " up - Start dev stack (compose + override)" + @echo " down - Stop dev stack" + @echo " logs - Tail logs for all services" + @echo " logs-% - Tail logs for a specific service (e.g., make logs-server)" + @echo " build - Build all images locally" + @echo " push - Push built images to GHCR" + @echo " pull-prod - Pull prod images from GHCR" + @echo " up-prod - Start prod stack (docker-compose.prod.yml)" + @echo " down-prod - Stop prod stack" + @echo " health - Quick health checks" + +# ---------- Development stack ---------- +.PHONY: up +yup: ## Start dev stack + $(COMPOSE) up -d --build + +.PHONY: down +down: ## Stop dev stack + $(COMPOSE) down + +.PHONY: logs +logs: ## Tail logs for all services + $(COMPOSE) logs -f + +.PHONY: logs-% +logs-%: ## Tail logs for a specific service, e.g. `make logs-server` + $(COMPOSE) logs -f $* + +# ---------- Images: build/push ---------- +.PHONY: build +build: ## Build all images locally + docker build -f server/Dockerfile -t $(API_IMAGE) . + docker build -f dashboard/Dockerfile -t $(DASH_IMAGE) . + docker build -f listener/Dockerfile -t $(LISTENER_IMAGE) . + docker build -f scheduler/Dockerfile -t $(SCHED_IMAGE) . + +.PHONY: push +push: ## Push all images to GHCR + docker push $(API_IMAGE) + docker push $(DASH_IMAGE) + docker push $(LISTENER_IMAGE) + docker push $(SCHED_IMAGE) + +# ---------- Production stack ---------- +PROD_COMPOSE=docker compose -f docker-compose.prod.yml + +.PHONY: pull-prod +pull-prod: ## Pull prod images + $(PROD_COMPOSE) pull + +.PHONY: up-prod +up-prod: ## Start prod stack + $(PROD_COMPOSE) up -d + +.PHONY: down-prod +down-prod: ## Stop prod stack + $(PROD_COMPOSE) down + +# ---------- Health ---------- +.PHONY: health +health: ## Quick health checks + @echo "API health:" && curl -fsS http://localhost:8000/health || true + @echo "Dashboard (dev):" && curl -fsS http://localhost:5173/ || true + @echo "MQTT TCP 1883:" && nc -z localhost 1883 && echo OK || echo FAIL + @echo "MQTT WS 9001:" && nc -z localhost 9001 && echo OK || echo FAIL diff --git a/deployment.md b/deployment.md index 48ac8b7..17ec555 100644 --- a/deployment.md +++ b/deployment.md @@ -194,11 +194,17 @@ chmod 755 mosquitto/config mosquitto/data mosquitto/log nano .env # Wichtige Anpassungen: -# API_URL=http://YOUR_SERVER_IP:8000 -# DB_HOST=db (sollte bereits korrekt sein) +# VITE_API_URL=https://YOUR_SERVER_HOST/api # Für Dashboard-Build (Production) +# DB_HOST=db # In Containern immer 'db' +# DB_CONN=mysql+pymysql://${DB_USER}:${DB_PASSWORD}@db/${DB_NAME} # Alle Passwörter für Produktion ändern ``` +Hinweise: +- Eine Vorlage `.env.example` liegt im Repo. Kopiere sie als Ausgangspunkt: `cp .env.example .env`. +- Für lokale Entwicklung lädt `server/database.py` die `.env`, wenn `ENV=development` gesetzt ist. +- In Produktion verwaltet Compose/Container die Variablen; kein automatisches `.env`-Load im Code nötig. + --- ## 🚀 Phase 3: System-Start und Konfiguration @@ -252,6 +258,41 @@ docker compose logs dashboard docker compose logs mqtt ``` +--- + +## 🧪 Quickstart (Entwicklung) + +Schneller Start der Entwicklungsumgebung mit automatischen Proxys und Hot-Reload. + +```bash +# Im Repository-Root +# 1) .env aus Vorlage erzeugen (lokal, falls noch nicht vorhanden) +cp -n .env.example .env + +# 2) Dev-Stack starten (verwendet docker-compose.yml + docker-compose.override.yml) +docker compose up -d --build + +# 3) Status & Logs +docker compose ps +docker compose logs -f server +docker compose logs -f dashboard +docker compose logs -f mqtt + +# 4) Stack stoppen +docker compose down +``` + +Erreichbarkeit (Dev): +- Dashboard (Vite): http://localhost:5173 +- API (Flask Dev): http://localhost:8000/api +- API Health: http://localhost:8000/health +- Screenshots: http://localhost:8000/screenshots/.jpg +- MQTT: localhost:1883 (WebSocket: localhost:9001) + +Hinweise: +- `ENV=development` lädt `.env` automatisch in `server/database.py`. +- Vite proxy routet `/api` und `/screenshots` in Dev direkt auf die API (siehe `dashboard/vite.config.ts`). + ### 12. Automatischer Start (optional) ```bash