4 Commits

Author SHA1 Message Date
CanbiZ (MickLesk)
dc47c1c1c3 Unify telemetry storage and add repo filtering
Refactor telemetry backend to store all telemetry in a single collection and add repo_source-based filtering.

Key changes:
- Added detect_repo_source() in misc/api.func to auto-detect/export REPO_SOURCE (ProxmoxVE/ProxmoxVED/external) when scripts are sourced.
- Consolidated PocketBase collections into a single default collection (_telemetry_data) across service, migration, and scripts; updated defaults in migrate.go, migration.go, migrate.sh and migration shell scripts.
- Simplified PBClient to use one targetColl and removed collection resolution logic; updated create/update/find/fetch functions to use targetColl.
- Introduced repo_source field (values: "ProxmoxVE", "ProxmoxVED", "external") on telemetry records and telemetry payloads; updated validation and logging.
- Added repo filtering to dashboard endpoints, FetchDashboardData and FetchRecordsPaginated, plus a repo selector in the dashboard UI; default filter is ProxmoxVE (production), with an "all" option.
- Adjusted API handlers and callers to pass repo filters and include repo_source when upserting telemetry.
- Misc: updated comments, error messages, and logging to reflect the new model; added telemetry-service.exe binary.

Purpose: simplify data model (single collection), make telemetry attributable to repository sources, and enable dashboard filtering by repo/source.
2026-02-11 15:49:41 +01:00
CanbiZ (MickLesk)
d32b00ff31 Add weekly reports, cleanup, and dashboard UI
Introduce weekly summary reports and a cleanup job, enhance dashboard UI, and adjust telemetry/build settings.

- Add REPO_SOURCE to misc/api.func and include repo_source in telemetry payloads.
- Implement weekly report generation/scheduling in alerts.go: new data types, HTML/plain templates, scheduler, SendWeeklyReport/TestWeeklyReport, and email/HTML helpers.
- Add Cleaner (misc/data/cleanup.go) to detect and mark stuck installations as 'unknown' with scheduling and manual trigger APIs.
- Enhance dashboard backend/frontend (misc/data/dashboard.go): optional days filter (allow 'All'), increase fetch page size, simplify fetchRecords, add quick filter buttons, detail & health modals, improved styles and chart options, and client-side record detail view.
- Update Dockerfile (misc/data/Dockerfile): rename binaries to telemetry-service and build migrate from ./migration/migrate.go; copy adjusted in final image.
- Add migration tooling (misc/data/migration/migrate.sh and migration.go) and other small service changes.

These changes add operational reporting and cleanup capabilities, improve observability and UX of the dashboard, and align build and telemetry identifiers for the service.
2026-02-11 12:19:30 +01:00
CanbiZ (MickLesk)
b034d5b05d Fix telemetry typo and simplify microcode script
Correct telemetry status spelling ("sucess" → "success") and map status values accordingly. Normalize EOF/newline handling in several misc/data files. Major refactor of tools/pve/microcode.sh: update copyright/author, remove the verbose CPU detection/display helpers, simplify current microcode detection, streamline package discovery and selection logic, switch wget to curl, clean up download/install flow (use basename and remove /tmp usage), simplify UI/messages and exit handling, and remove live microcode reload/version post-install checks.
2026-02-10 08:09:26 +01:00
MickLesk
0226a043b8 feat(telemetry): add caching, alerts, migration & dashboard improvements
- Add Redis/in-memory caching layer (cache.go)
- Add SMTP alerting for high failure rates (alerts.go)
- Add data migration script from old API (migrate.go)
- Add docker-compose.yml for easy deployment
- Move dashboard to / with redirect from /dashboard
- Add dark/light mode toggle
- Add error analysis and failed apps statistics
- Add PVE version and LXC/VM type stats
- Add /metrics Prometheus endpoint
- Add /api/records pagination endpoint
- Add CSV export functionality
- Enhanced healthcheck with PB connection status

New ENV vars:
- Cache: ENABLE_CACHE, CACHE_TTL_SECONDS, ENABLE_REDIS, REDIS_URL
- Alerts: ALERT_ENABLED, SMTP_*, ALERT_FAILURE_THRESHOLD, etc.
- Migration: RUN_MIGRATION, MIGRATION_REQUIRED, MIGRATION_SOURCE_URL
2026-02-09 18:33:33 +01:00