7 Commits

Author SHA1 Message Date
CanbiZ (MickLesk)
0231b72d78 Add telemetry data service and dashboard revamp
Some checks failed
Update GitHub Versions (New) / update-github-versions (push) Has been cancelled
Update Versions from GitHub / update-versions (push) Has been cancelled
Introduce a telemetry data microservice under misc/data: add Dockerfile, entrypoint, migration tools, README, LICENSE and a .gitignore. Increase Docker CACHE_TTL_SECONDS to 300s. Implement extensive dashboard and analytics updates in dashboard.go: add total_all_time and sample_size, return total item counts from fetchRecords (with page/limit handling and a maxRecords guard), raise top-N limits, add a minimum-installs threshold for failed-apps, and numerous UI/style/layout improvements in the embedded DashboardHTML. Minor formatting tweak to misc/api.func.
2026-02-12 13:10:06 +01:00
CanbiZ (MickLesk)
dc47c1c1c3 Unify telemetry storage and add repo filtering
Refactor telemetry backend to store all telemetry in a single collection and add repo_source-based filtering.

Key changes:
- Added detect_repo_source() in misc/api.func to auto-detect/export REPO_SOURCE (ProxmoxVE/ProxmoxVED/external) when scripts are sourced.
- Consolidated PocketBase collections into a single default collection (_telemetry_data) across service, migration, and scripts; updated defaults in migrate.go, migration.go, migrate.sh and migration shell scripts.
- Simplified PBClient to use one targetColl and removed collection resolution logic; updated create/update/find/fetch functions to use targetColl.
- Introduced repo_source field (values: "ProxmoxVE", "ProxmoxVED", "external") on telemetry records and telemetry payloads; updated validation and logging.
- Added repo filtering to dashboard endpoints, FetchDashboardData and FetchRecordsPaginated, plus a repo selector in the dashboard UI; default filter is ProxmoxVE (production), with an "all" option.
- Adjusted API handlers and callers to pass repo filters and include repo_source when upserting telemetry.
- Misc: updated comments, error messages, and logging to reflect the new model; added telemetry-service.exe binary.

Purpose: simplify data model (single collection), make telemetry attributable to repository sources, and enable dashboard filtering by repo/source.
2026-02-11 15:49:41 +01:00
MickLesk
7915237622 fix(telemetry): fix typo 'sucess' -> 'success' everywhere
Some checks failed
Update GitHub Versions (New) / update-github-versions (push) Has been cancelled
Update Versions from GitHub / update-versions (push) Has been cancelled
2026-02-09 18:47:34 +01:00
MickLesk
b5960db4f9 fix(telemetry): fix migration - map 'done' to 'sucess', handle ct_type=0, detect duplicates 2026-02-09 18:46:26 +01:00
MickLesk
6a77427448 fix(telemetry): add PocketBase authentication to migration script 2026-02-09 18:41:55 +01:00
MickLesk
5b43016594 fix(telemetry): support PB_URL and PB_TARGET_COLLECTION env vars for Coolify 2026-02-09 18:38:59 +01:00
MickLesk
0226a043b8 feat(telemetry): add caching, alerts, migration & dashboard improvements
- Add Redis/in-memory caching layer (cache.go)
- Add SMTP alerting for high failure rates (alerts.go)
- Add data migration script from old API (migrate.go)
- Add docker-compose.yml for easy deployment
- Move dashboard to / with redirect from /dashboard
- Add dark/light mode toggle
- Add error analysis and failed apps statistics
- Add PVE version and LXC/VM type stats
- Add /metrics Prometheus endpoint
- Add /api/records pagination endpoint
- Add CSV export functionality
- Enhanced healthcheck with PB connection status

New ENV vars:
- Cache: ENABLE_CACHE, CACHE_TTL_SECONDS, ENABLE_REDIS, REDIS_URL
- Alerts: ALERT_ENABLED, SMTP_*, ALERT_FAILURE_THRESHOLD, etc.
- Migration: RUN_MIGRATION, MIGRATION_REQUIRED, MIGRATION_SOURCE_URL
2026-02-09 18:33:33 +01:00