40 Commits

Author SHA1 Message Date
CanbiZ (MickLesk)
53cf705799 Format misc/api.func: spacing and heredocs
Clean up formatting in misc/api.func for readability. Normalized spacing in the categorize_error case patterns, removed trailing blank-space lines, and standardized detect_gpu blank-line spacing. Converted heredoc assignments to use multiline $() style for JSON_PAYLOAD in post_tool_to_api, post_addon_to_api, and post_update_to_api_extended, and added a final newline at end of file. No functional changes intended; purely whitespace/formatting cleanup.
2026-02-10 08:17:50 +01:00
CanbiZ (MickLesk)
887a899f24 Extend telemetry reporting and ingestion
Add extended telemetry functions and server-side support: misc/api.func gains helpers (categorize_error, install timer, detect_gpu) and new reporters for tools, addons, and an extended post_update_to_api with duration, GPU and error_category. misc/data/service.go updated to accept and validate new fields (type: tool/addon, tool_name, parent_ct, gpu_vendor, gpu_passthrough, install_duration, error_category), expand allowed enums, include new fields in UpsertTelemetry and mapping, and add input sanitization. Also add telemetry-ingest.exe binary. These changes enable richer telemetry (tool/addon events, GPU info, durations and categorized errors) and server ingestion/validation for them.
2026-02-10 08:14:45 +01:00
CanbiZ (MickLesk)
b034d5b05d Fix telemetry typo and simplify microcode script
Correct telemetry status spelling ("sucess" → "success") and map status values accordingly. Normalize EOF/newline handling in several misc/data files. Major refactor of tools/pve/microcode.sh: update copyright/author, remove the verbose CPU detection/display helpers, simplify current microcode detection, streamline package discovery and selection logic, switch wget to curl, clean up download/install flow (use basename and remove /tmp usage), simplify UI/messages and exit handling, and remove live microcode reload/version post-install checks.
2026-02-10 08:09:26 +01:00
CanbiZ (MickLesk)
88a540e457 Support PATCH updates for telemetry status
Send only changing fields for status updates and add server-side update flow.

- Trimmed telemetry JSON payload in misc/api.func to include only status, error, and exit_code (removed static fields and pve_version) so updates are minimal.
- Added TelemetryStatusUpdate type and new PBClient methods: FindRecordByRandomID, UpdateTelemetryStatus, and UpsertTelemetry in misc/data/service.go. UpsertTelemetry creates a record for status="installing", otherwise finds the record by random_id and PATCHes only status/error/exit_code (fallbacks to create if not found).
- Relaxed validation logic in validate(): detect updates (status != "installing") and skip certain strict numeric checks for update requests while keeping required fields and other validations.
- Main handler now calls UpsertTelemetry instead of CreateTelemetry and logs generic errors.

These changes allow idempotent, minimal updates to existing telemetry records and avoid repeatedly sending/storing unchanged metadata.
2026-02-09 17:07:30 +01:00
CanbiZ (MickLesk)
878672a8df Update api.func 2026-02-09 17:04:15 +01:00
CanbiZ (MickLesk)
5aa85ace6a Add telemetry debug/logging and failure report
Enhance post_update_to_api robustness and observability: add debug traces for entry, missing curl, duplicate submissions, DIAGNOSTICS/RANDOM_UUID checks, payload/URL output, and HTTP response code capture; make curl non-blocking and tolerant of failures. Also invoke post_update_to_api on installation failure so build/install errors are reported to telemetry. Includes a small comment glyph fix in build.func. Changes keep telemetry as a silent, best-effort path that won't break script execution.
2026-02-09 16:53:25 +01:00
CanbiZ (MickLesk)
ce375b02aa Refactor telemetry checks, relax method validation
Reformat telemetry pre-checks in misc/api.func for clearer multi-line condition handling and remove stray whitespace around the curl response logging. Tweak misc/build.func comment line (minor encoding/visual cleanup). In misc/data/service.go remove the strict allowedMethod map and its validation so the 'method' field is treated as optional/flexible; keep sanitization and numeric validations unchanged. These changes improve readability and allow custom method values without breaking behavior.
2026-02-09 16:51:51 +01:00
CanbiZ (MickLesk)
7c3688cd0a debug 2026-02-09 16:48:47 +01:00
CanbiZ (MickLesk)
4a8a4180d9 use https 2026-02-09 16:20:43 +01:00
CanbiZ (MickLesk)
313da7c00c Switch telemetry to ingest service
Replace direct PocketBase integration with a fire-and-forget telemetry ingest endpoint and tighten validation. misc/api.func: point to telemetry.community-scripts.org, add TELEMETRY_TIMEOUT, use DIAGNOSTICS=no opt-out, include random_id/NSAPP/status in payloads, unify LXC/VM POSTs, avoid blocking or failing scripts, remove PocketBase record lookup/patch logic. misc/data/service.go: update TelemetryIn/TelemetryOut schemas to match new payload, add stricter sanitization and enum/range validation, adjust hashing/deduplication usage, and update request logging to reflect nsapp/status. Overall: safer, non-blocking telemetry with improved schema validation and GDPR-friendly behavior.
2026-02-09 16:06:44 +01:00
CanbiZ (MickLesk)
820d4551a1 Replace Go API with PocketBase; update docs
Remove the old Go/Mongo API (api/main.go, go.mod, go.sum, .env.example) and switch telemetry backend to PocketBase (http://db.community-scripts.org). Update documentation and flowcharts to reflect the PocketBase collection (_dev_telemetry_data), new REST endpoints (POST/PATCH/GET), field schema, and revised api.func integration (LXC/VM reporting and status updates). Misc scripts and helpers were adjusted (misc/api.func, misc/build.func, misc/error_handler.func) and a new misc/ingest.go was added. This consolidates telemetry to a hosted PocketBase instance and updates docs and integration points accordingly.
2026-02-09 15:34:17 +01:00
CanbiZ (MickLesk)
6f9a1965f9 Merge from VE 2026-01-27 09:29:22 +01:00
Tobias
c5786b9210 happ new year - well a bit late :D 2026-01-06 12:36:02 +01:00
CanbiZ
d08d3ec424 Fix: remove debug echo from post_to_api, fix dev_index arithmetic in GPU passthrough (set -e compatible) 2025-11-17 15:44:56 +01:00
CanbiZ
a7daaee5ae Refactor dev MOTD/SSH setup and journal cleanup
Moved MOTD/SSH setup for dev mode in LXC containers to run after installation and when containers are kept for debugging, improving reliability and clarity. Updated journalctl cleanup to avoid using $STD and suppress errors if systemd-journald is not initialized. Also refactored indentation in misc/api.func for consistency.
2025-11-17 15:09:36 +01:00
CanbiZ
1090081a66 remove ipv6 from api 2025-11-17 15:01:22 +01:00
4f1199a06e add testing echo 2025-11-17 14:51:46 +01:00
CanbiZ
c9d7c2f46b add exit_codes.md 2025-11-17 11:39:14 +01:00
CanbiZ
ee5fe3439d extend error messages 206,207,208,209 / 215, 216, 217, 218 / 222, 223, 231 2025-11-17 11:36:00 +01:00
CanbiZ
36f5e32d48 Inline explain_exit_code and clean diagnostics_check docs
Moved the explain_exit_code function directly into api.func, removing external dependency loading. Also cleaned up merge conflict markers and unified documentation for diagnostics_check in build.func.
2025-11-17 11:26:59 +01:00
CanbiZ
c0fc74bc9c Add detailed documentation and refactor core functions
Expanded inline documentation and section headers across misc/api.func, misc/build.func, misc/core.func, misc/error_handler.func, and misc/install.func for improved maintainability and clarity. Refactored error handling to use centralized explain_exit_code and updated API telemetry/reporting logic. Enhanced modularity and structure of utility, validation, and setup functions, and improved comments for user guidance and developer onboarding.
2025-11-17 11:17:04 +01:00
CanbiZ
af46e22bd2
Revert "feat: add centralized logging utilities" 2025-08-25 14:30:28 +02:00
CanbiZ
3c3aabd21d feat: expand logger with subshell-safe error handling 2025-08-25 14:28:16 +02:00
2c7ff229a5 Api testing 2025-06-10 08:58:04 +02:00
CanbiZ
4f3b893e7d Update api.func 2025-05-22 14:14:02 +02:00
Michel Roegl-Brunner
f99c48d760
Change all links to gite (#343) 2025-05-14 10:17:40 +02:00
CanbiZ
5d3209a50f Update api.func 2025-03-26 14:52:52 +01:00
CanbiZ
7d424933d5 revert api 2025-03-26 14:46:47 +01:00
bfbb0435fa Api 2025-03-21 15:15:45 +01:00
7ddc7d5da5 Api fix 2025-03-21 10:26:24 +01:00
CanbiZ
37e5752e5e bak api 2025-03-21 10:12:01 +01:00
CanbiZ
4268c05a9f fixes 2025-03-21 09:36:13 +01:00
90698425a2 Api update 2025-03-18 13:25:21 +01:00
1e52f338f9 Tapi 2025-03-17 10:51:40 +01:00
c09a4bc335 api 2025-03-17 10:41:02 +01:00
CanbiZ
912cda33e1 API 2025-03-14 14:14:26 +01:00
06476fbdf4 Brink api to dev back 2025-03-12 11:41:23 +01:00
89ac773c8e Brink api to dev back 2025-03-12 11:36:51 +01:00
Michel Roegl-Brunner
8679ebefa0
Remove Curl Calls from api.func 2025-03-05 18:44:43 +01:00
CanbiZ
1c2604bea0 Initial Release 2025-03-03 10:10:57 +01:00