From 1e5627ea194d4170f2264e936f0375a34a43505c Mon Sep 17 00:00:00 2001 From: CanbiZ <47820557+MickLesk@users.noreply.github.com> Date: Mon, 1 Dec 2025 12:29:24 +0100 Subject: [PATCH] Restructure and relocate documentation files Removed outdated documentation files from docs/. Added new detailed guides to ct/ and install/ directories. Updated README.md to reflect new navigation, learning paths, and documentation structure. Relocated system guides and technical references to settings/ and install/ subdirectories for improved organization. --- docs/APP-install.md | 1121 --------------- docs/INDEX.md | 346 ----- docs/README.md | 88 +- docs/alpine-install.func.md | 651 --------- docs/alpine-tools.func.md | 588 -------- docs/api.func.md | 670 --------- docs/build.func.md | 584 -------- docs/cloud-init.func.md | 571 -------- docs/core.func.md | 918 ------------- docs/{APP-ct.md => ct/DETAILED_GUIDE.md} | 267 +--- docs/error_handler.func.md | 598 -------- docs/install.func.md | 646 --------- docs/install/DETAILED_GUIDE.md | 646 +++++++++ docs/misc/api.func.md | 670 --------- docs/{ => settings}/DEFAULTS_SYSTEM_GUIDE.md | 0 docs/{ => settings}/DEV_MODE.md | 0 docs/{ => settings}/EXIT_CODES.md | 0 docs/{ => settings}/TECHNICAL_REFERENCE.md | 0 docs/tools.func.md | 1283 ------------------ 19 files changed, 714 insertions(+), 8933 deletions(-) delete mode 100644 docs/APP-install.md delete mode 100644 docs/INDEX.md delete mode 100644 docs/alpine-install.func.md delete mode 100644 docs/alpine-tools.func.md delete mode 100644 docs/api.func.md delete mode 100644 docs/build.func.md delete mode 100644 docs/cloud-init.func.md delete mode 100644 docs/core.func.md rename docs/{APP-ct.md => ct/DETAILED_GUIDE.md} (63%) delete mode 100644 docs/error_handler.func.md delete mode 100644 docs/install.func.md create mode 100644 docs/install/DETAILED_GUIDE.md delete mode 100644 docs/misc/api.func.md rename docs/{ => settings}/DEFAULTS_SYSTEM_GUIDE.md (100%) rename docs/{ => settings}/DEV_MODE.md (100%) rename docs/{ => settings}/EXIT_CODES.md (100%) rename docs/{ => settings}/TECHNICAL_REFERENCE.md (100%) delete mode 100644 docs/tools.func.md diff --git a/docs/APP-install.md b/docs/APP-install.md deleted file mode 100644 index a754e3316..000000000 --- a/docs/APP-install.md +++ /dev/null @@ -1,1121 +0,0 @@ -# 🛠ī¸ **Application Installation Scripts (install/AppName-install.sh)** - -**Modern Guide to Writing In-Container Installation Scripts** - -> **Updated**: December 2025 -> **Context**: Integrated with tools.func, error_handler.func, and install.func -> **Examples Used**: `/install/pihole-install.sh`, `/install/mealie-install.sh` - ---- - -## 📋 Table of Contents - -- [Overview](#overview) -- [Execution Context](#execution-context) -- [File Structure](#file-structure) -- [Complete Script Template](#complete-script-template) -- [Installation Phases](#installation-phases) -- [Function Reference](#function-reference) -- [Best Practices](#best-practices) -- [Real Examples](#real-examples) -- [Troubleshooting](#troubleshooting) -- [Contribution Checklist](#contribution-checklist) - ---- - -## Overview - -### Purpose - -Installation scripts (`install/AppName-install.sh`) **run inside the LXC container** and are responsible for: - -1. Setting up the container OS (updates, packages) -2. Installing application dependencies -3. Downloading and configuring the application -4. Setting up services and systemd units -5. Creating version tracking files for updates -6. Generating credentials/configurations -7. Final cleanup and validation - -### Key Characteristics - -- Runs as **root inside container** (not on Proxmox host) -- Executed automatically by `build_container()` from ct/AppName.sh -- Uses `$FUNCTIONS_FILE_PATH` for function library access -- Interactive elements via **whiptail** (GUI menus) -- Version-aware for update tracking - -### Execution Flow - -``` -ct/AppName.sh (Proxmox Host) - ↓ -build_container() - ↓ -pct exec CTID bash -c "$(cat install/AppName-install.sh)" - ↓ -install/AppName-install.sh (Inside Container) - ↓ -Container Ready with App Installed -``` - ---- - -## Execution Context - -### Environment Variables Available - -```bash -# From Proxmox/Container -CTID # Container ID (100, 101, etc.) -PCT_OSTYPE # OS type (alpine, debian, ubuntu) -HOSTNAME # Container hostname - -# From build.func -FUNCTIONS_FILE_PATH # Bash functions library (core.func + tools.func) -VERBOSE # Verbose mode (yes/no) -STD # Standard redirection variable (silent/empty) - -# From install.func -APP # Application name -NSAPP # Normalized app name (lowercase, no spaces) -METHOD # Installation method (ct/install) -RANDOM_UUID # Session UUID for telemetry -``` - -### Access to Functions - -```bash -# All functions from core.func available: -source /dev/stdin <<<"$FUNCTIONS_FILE_PATH" -color # ANSI colors -catch_errors # Error handling -msg_info # Display messages -msg_ok -msg_error - -# All functions from tools.func available: -setup_nodejs # Tool installation -setup_php -setup_python -setup_docker -# ... many more - -# All functions from install.func available: -motd_ssh # Final setup -customize -cleanup_lxc -``` - ---- - -## File Structure - -### Minimal install/AppName-install.sh Template - -```bash -#!/usr/bin/env bash # [1] Shebang - -# [2] Copyright/Metadata -# Copyright (c) 2021-2025 community-scripts ORG -# Author: YourUsername -# License: MIT -# Source: https://example.com - -# [3] Load functions -source /dev/stdin <<<"$FUNCTIONS_FILE_PATH" -color -verb_ip6 -catch_errors -setting_up_container -network_check -update_os - -# [4] Installation steps -msg_info "Installing Dependencies" -$STD apt-get install -y package1 package2 -msg_ok "Installed Dependencies" - -# [5] Final setup -motd_ssh -customize -cleanup_lxc -``` - ---- - -## Complete Script Template - -### Phase 1: Header & Initialization - -```bash -#!/usr/bin/env bash -# Copyright (c) 2021-2025 community-scripts ORG -# Author: YourUsername -# Co-Author: AnotherAuthor (for updates) -# License: MIT | https://github.com/community-scripts/ProxmoxVED/raw/main/LICENSE -# Source: https://github.com/application/repo - -# Load all available functions (from core.func + tools.func) -source /dev/stdin <<<"$FUNCTIONS_FILE_PATH" - -# Initialize environment -color # Setup ANSI colors and icons -verb_ip6 # Configure IPv6 (if needed) -catch_errors # Setup error traps -setting_up_container # Verify OS is ready -network_check # Verify internet connectivity -update_os # Update packages (apk/apt) -``` - -### Phase 2: Dependency Installation - -```bash -msg_info "Installing Dependencies" -$STD apt-get install -y \ - curl \ - wget \ - git \ - nano \ - build-essential \ - libssl-dev \ - python3-dev -msg_ok "Installed Dependencies" -``` - -**Guidelines**: -- Use `\` for line continuation (readability) -- Group related packages together -- Collapse repeated prefixes: `php8.4-{bcmath,curl,gd,intl,mbstring}` -- Use `-y` flag for non-interactive installation -- Silence output with `$STD` unless debugging - -### Phase 3: Tool Setup (Using tools.func) - -```bash -# Setup specific tool versions -NODE_VERSION="22" setup_nodejs - -# Or for databases -MYSQL_VERSION="8.0" setup_mysql - -# Or for languages -PHP_VERSION="8.4" PHP_MODULE="redis,imagick" setup_php - -# Or for version control -setup_composer -``` - -**Available Tool Functions**: -```bash -setup_nodejs # Node.js from official repo -setup_php # PHP with optional modules -setup_python # Python 3 -setup_mariadb # MariaDB database -setup_mysql # MySQL database -setup_postgresql # PostgreSQL database -setup_mongodb # MongoDB database -setup_docker # Docker Engine -setup_nodejs # Node.js runtime -setup_composer # PHP Composer -setup_ruby # Ruby runtime -setup_rust # Rust toolchain -setup_go # Go language -setup_java # Java/Temurin -# ... many more in tools.func.md -``` - -### Phase 4: Application Download & Setup - -```bash -# Method A: Download from GitHub releases -msg_info "Downloading ${APP}" -RELEASE=$(curl -fsSL https://api.github.com/repos/user/repo/releases/latest | \ - grep "tag_name" | awk '{print substr($2, 2, length($2)-3)}') - -wget -q "https://github.com/user/repo/releases/download/v${RELEASE}/app-${RELEASE}.tar.gz" \ - -O /opt/app-${RELEASE}.tar.gz - -cd /opt -tar -xzf app-${RELEASE}.tar.gz -rm -f app-${RELEASE}.tar.gz -msg_ok "Downloaded and extracted ${APP}" - -# Method B: Clone from Git -git clone https://github.com/user/repo /opt/appname - -# Method C: Download single file -fetch_and_deploy_gh_release "AppName" "user/repo" "tarball" -``` - -### Phase 5: Configuration Files - -```bash -# Method A: Using cat << EOF (multiline) -cat <<'EOF' >/etc/nginx/sites-available/appname -server { - listen 80; - server_name _; - root /opt/appname/public; - index index.php index.html; - - location ~ \.php$ { - fastcgi_pass unix:/run/php-fpm.sock; - include fastcgi_params; - } -} -EOF - -# Method B: Using sed for replacements -sed -i -e "s|^DB_HOST=.*|DB_HOST=localhost|" \ - -e "s|^DB_USER=.*|DB_USER=appuser|" \ - /opt/appname/.env - -# Method C: Using echo for simple configs -echo "APP_KEY=base64:$(openssl rand -base64 32)" >> /opt/appname/.env -``` - -### Phase 6: Database Setup (If Needed) - -```bash -msg_info "Setting up Database" - -DB_NAME="appname_db" -DB_USER="appuser" -DB_PASS=$(openssl rand -base64 18 | tr -dc 'a-zA-Z0-9' | head -c13) - -# For MySQL/MariaDB -mysql -u root <> ~/appname.creds -Database Credentials -Database: ${DB_NAME} -Username: ${DB_USER} -Password: ${DB_PASS} -EOF - -msg_ok "Database setup complete" -``` - -### Phase 7: Permission & Ownership - -```bash -msg_info "Setting permissions" - -# Web applications typically run as www-data -chown -R www-data:www-data /opt/appname -chmod -R 755 /opt/appname -chmod -R 644 /opt/appname/* -chmod 755 /opt/appname/*/.* - -# For apps with specific requirements -find /opt/appname/storage -type f -exec chmod 644 {} \; -find /opt/appname/storage -type d -exec chmod 755 {} \; - -msg_ok "Permissions set" -``` - -### Phase 8: Service Configuration - -```bash -# Enable systemd service -systemctl enable -q --now appname - -# Or for OpenRC (Alpine) -rc-service appname start -rc-update add appname default - -# Verify service is running -if systemctl is-active --quiet appname; then - msg_ok "Service running successfully" -else - msg_error "Service failed to start" - journalctl -u appname -n 20 - exit 1 -fi -``` - -### Phase 9: Version Tracking - -```bash -# Essential for update detection -echo "${RELEASE}" > /opt/${APP}_version.txt - -# Or with additional metadata -cat > /opt/${APP}_version.txt < /opt/${APP}_version.txt -``` - -### Phase 5: Configuration - -```bash -# Application-specific configuration -cat > /opt/appname/.env < /opt/appname/config.yml < /opt/${APP}_version.txt - -# ❌ Bad: No version file -# (Update function won't work) -``` - -#### 6. Handle Alpine vs Debian Differences - -```bash -# ✅ Good: Detect OS -if grep -qi 'alpine' /etc/os-release; then - apk add package -else - apt-get install -y package -fi - -# ❌ Bad: Assumes Debian -apt-get install -y package -``` - -#### 7. Use Proper Messaging - -```bash -# ✅ Good: Clear status progression -msg_info "Installing Dependencies" -$STD apt-get install -y package -msg_ok "Installed Dependencies" - -msg_info "Configuring Application" -# ... configuration ... -msg_ok "Application configured" - -# ❌ Bad: No status messages -apt-get install -y package -# ... configuration ... -``` - -### ❌ DON'T: - -#### 1. Hardcode Versions - -```bash -# ❌ Bad: Won't auto-update -VERSION="1.2.3" -wget https://example.com/app-1.2.3.tar.gz - -# ✅ Good: Fetch latest -RELEASE=$(curl -fsSL https://api.github.com/repos/user/repo/releases/latest | jq -r '.tag_name') -wget https://example.com/app-${RELEASE}.tar.gz -``` - -#### 2. Use Root Without Password - -```bash -# ❌ Bad: Allows unprompted root access -mysql -u root < /etc/systemd/system/appname.service < /opt/${APP}_version.txt - -motd_ssh -customize -cleanup_lxc -``` - -### Example 2: Database Application (PHP + MySQL) - -```bash -#!/usr/bin/env bash -source /dev/stdin <<<"$FUNCTIONS_FILE_PATH" -color -catch_errors -setting_up_container -network_check -update_os - -msg_info "Installing Dependencies" -$STD apt-get install -y git curl nginx supervisor -msg_ok "Installed Dependencies" - -msg_info "Setting up PHP" -PHP_VERSION="8.4" PHP_MODULE="bcmath,curl,gd,intl,mbstring,pdo_mysql,redis" setup_php -msg_ok "PHP installed" - -msg_info "Setting up Database" -MARIADB_VERSION="11.4" setup_mariadb -msg_ok "MariaDB installed" - -DB_NAME="appname_db" -DB_USER="appuser" -DB_PASS=$(openssl rand -base64 18 | tr -dc 'a-zA-Z0-9' | head -c13) - -mysql -u root < /opt/${APP}_version.txt - -motd_ssh -customize -cleanup_lxc -``` - ---- - -## Troubleshooting - -### Installation Hangs - -**Symptom**: Script appears to freeze at particular step - -**Causes**: -1. Network connectivity lost -2. Repository server timing out -3. Interactive prompt waiting for input - -**Debug**: -```bash -# Check if process still running -ps aux | grep -i appname - -# Check network -ping -c 1 8.8.8.8 - -# Check apt lock -lsof /var/lib/apt/lists/lock -``` - -### Package Installation Fails - -**Symptom**: `E: Unable to locate package xyz` - -**Causes**: -1. Repository not updated -2. Package name incorrect for OS version -3. Conflicting repository configuration - -**Solution**: -```bash -# Force update -apt-get update --allow-releaseinfo-change -apt-cache search package | grep exact_name -``` - -### Permission Denied on Files - -**Symptom**: Application can't write to `/opt/appname` - -**Causes**: -1. Wrong owner -2. Wrong permissions (644 for files, 755 for directories) - -**Fix**: -```bash -chown -R www-data:www-data /opt/appname -chmod -R 755 /opt/appname -find /opt/appname -type f -exec chmod 644 {} \; -find /opt/appname -type d -exec chmod 755 {} \; -``` - -### Service Won't Start - -**Symptom**: `systemctl status appname` shows failed - -**Debug**: -```bash -# Check service status -systemctl status appname - -# View logs -journalctl -u appname -n 50 - -# Check configuration -systemctl cat appname -``` - ---- - -## Contribution Checklist - -Before submitting a PR: - -### Script Structure -- [ ] Shebang is `#!/usr/bin/env bash` -- [ ] Copyright header with author and source URL -- [ ] Functions loaded via `$FUNCTIONS_FILE_PATH` -- [ ] Initial setup: `color`, `catch_errors`, `setting_up_container`, `network_check`, `update_os` - -### Installation Flow -- [ ] Dependencies installed with `$STD apt-get install -y \` -- [ ] Package names collapsed (`php-{bcmath,curl}`) -- [ ] Tool setup uses functions from tools.func (not manual installation) -- [ ] Application version fetched dynamically (not hardcoded) -- [ ] Version saved to `/opt/${APP}_version.txt` - -### Configuration -- [ ] Configuration files created properly (heredoc or sed) -- [ ] Credentials generated randomly (`openssl rand`) -- [ ] Credentials stored in creds file -- [ ] Passwords use alphanumeric only (no special chars) -- [ ] Proper file permissions set - -### Messaging -- [ ] `msg_info` followed by action then `msg_ok` -- [ ] Error cases use `msg_error` and exit -- [ ] No bare `echo` statements for status (use msg_* functions) - -### Cleanup -- [ ] Temporary files removed -- [ ] Package manager cache cleaned (`autoremove`, `autoclean`) -- [ ] `cleanup_lxc` called at end -- [ ] `motd_ssh` called before `customize` -- [ ] `customize` called before exit - -### Testing -- [ ] Script tested with default OS (Debian 12/Ubuntu 22.04) -- [ ] Script tested with Alpine (if applicable) -- [ ] Script tested with verbose mode (`VERBOSE=yes`) -- [ ] Error handling tested (network interruption, missing packages) -- [ ] Cleanup verified (disk space reduced, temp files removed) - ---- - -## Related Documentation - -- [ct/AppName.sh Guide](UPDATED_APP-ct.md) -- [tools.func Wiki](../misc/tools.func.md) -- [install.func Wiki](../misc/install.func.md) -- [error_handler.func Wiki](../misc/error_handler.func.md) - ---- - -**Last Updated**: December 2025 -**Compatibility**: ProxmoxVED with tools.func v2+ -**Questions?** Open an issue in the repository diff --git a/docs/INDEX.md b/docs/INDEX.md deleted file mode 100644 index b2f74b5d1..000000000 --- a/docs/INDEX.md +++ /dev/null @@ -1,346 +0,0 @@ -# 📚 ProxmoxVED Documentation Index - -Complete guide to all ProxmoxVED documentation - quickly find what you need. - ---- - -## đŸŽ¯ **Quick Navigation by Goal** - -### 👤 **I want to...** - -**Contribute a new application** -→ Start with: [CONTRIBUTION_GUIDE.md](CONTRIBUTION_GUIDE.md) -→ Then: [UPDATED_APP-ct.md](UPDATED_APP-ct.md) + [UPDATED_APP-install.md](UPDATED_APP-install.md) - -**Understand the architecture** -→ Read: [TECHNICAL_REFERENCE.md](TECHNICAL_REFERENCE.md) -→ Then: [misc/README.md](misc/README.md) - -**Debug a failed installation** -→ Check: [EXIT_CODES.md](EXIT_CODES.md) -→ Then: [DEV_MODE.md](DEV_MODE.md) -→ See also: [misc/error_handler.func/](misc/error_handler.func/) - -**Configure system defaults** -→ Read: [DEFAULTS_SYSTEM_GUIDE.md](DEFAULTS_SYSTEM_GUIDE.md) - -**Learn about recent changes** -→ Check: [CHANGELOG_MISC.md](CHANGELOG_MISC.md) - -**Develop a function library** -→ Study: [misc/](misc/) documentation - ---- - -## 📂 **Documentation by Category** - -### 🏗ī¸ **Project Structure Documentation** - -| Directory | Documentation | -|-----------|---| -| **[/ct](ct/)** | Container creation scripts documentation | -| **[/install](install/)** | Installation scripts documentation | -| **[/vm](vm/)** | Virtual machine creation scripts documentation | -| **[/tools](tools/)** | Tools and utilities documentation | -| **[/api](api/)** | API integration documentation | -| **[/misc](misc/)** | Function libraries (9 total) | - -### 🚀 **For Contributors** - -| Document | Purpose | -|----------|---------| -| [CONTRIBUTION_GUIDE.md](CONTRIBUTION_GUIDE.md) | Complete contribution workflow | -| [UPDATED_APP-ct.md](UPDATED_APP-ct.md) | How to write ct/AppName.sh scripts | -| [UPDATED_APP-install.md](UPDATED_APP-install.md) | How to write install/appname-install.sh scripts | -| [TECHNICAL_REFERENCE.md](TECHNICAL_REFERENCE.md) | System architecture deep-dive | - -### 🔧 **For Operators & Developers** - -| Document | Purpose | -|----------|---------| -| [EXIT_CODES.md](EXIT_CODES.md) | Complete exit code reference | -| [DEV_MODE.md](DEV_MODE.md) | Debugging and development modes | -| [DEFAULTS_SYSTEM_GUIDE.md](DEFAULTS_SYSTEM_GUIDE.md) | Configuration and defaults system | -| [CHANGELOG_MISC.md](CHANGELOG_MISC.md) | Recent changes and updates | - -### 📚 **Function Library Documentation** (9 libraries) - -**Core Functions**: -- [build.func/](misc/build.func/) - Container creation orchestrator (7 files) -- [core.func/](misc/core.func/) - Utility functions (5 files) -- [error_handler.func/](misc/error_handler.func/) - Error handling (5 files) -- [api.func/](misc/api.func/) - Proxmox API integration (5 files) - -**Installation Functions**: -- [install.func/](misc/install.func/) - Container setup (5 files) -- [tools.func/](misc/tools.func/) - Package and tool installation (6 files) - -**Alpine Linux Functions**: -- [alpine-install.func/](misc/alpine-install.func/) - Alpine setup (5 files) -- [alpine-tools.func/](misc/alpine-tools.func/) - Alpine tools (5 files) - -**VM Functions**: -- [cloud-init.func/](misc/cloud-init.func/) - VM provisioning (5 files) - ---- - -## 📋 **All Documentation Files** - -### Root Level (13 main files + 6 directory structures) - -``` -/docs/ -├─ CONTRIBUTION_GUIDE.md (2800+ lines) Contributing guide -├─ UPDATED_APP-ct.md (900+ lines) ct script guide -├─ UPDATED_APP-install.md (1000+ lines) install script guide -├─ TECHNICAL_REFERENCE.md (600+ lines) Architecture reference -├─ DEFAULTS_SYSTEM_GUIDE.md (700+ lines) Configuration guide -├─ CHANGELOG_MISC.md (450+ lines) Change history -├─ EXIT_CODES.md (400+ lines) Exit codes reference -├─ DEV_MODE.md (400+ lines) Dev mode guide -├─ INDEX.md (This file) Documentation index -│ -├─ ct/ README for container scripts ★ NEW -├─ install/ README for installation scripts ★ NEW -├─ vm/ README for VM scripts ★ NEW -├─ tools/ README for tools & utilities ★ NEW -├─ api/ README for API integration ★ NEW -│ -└─ misc/ Function libraries (detailed below) -``` - -### Project Structure Mirror with Docs (48 files in misc/) - -Each top-level project directory (`/ct`, `/install`, `/vm`, `/tools`, `/api`) has a documentation companion in `/docs/` with a README explaining that section. - -### misc/ Subdirectories (48 files) - -``` -/docs/misc/ -├─ README.md (comprehensive overview) -│ -├─ build.func/ (7 files) -│ ├─ README.md -│ ├─ BUILD_FUNC_FLOWCHART.md -│ ├─ BUILD_FUNC_ARCHITECTURE.md -│ ├─ BUILD_FUNC_ENVIRONMENT_VARIABLES.md -│ ├─ BUILD_FUNC_FUNCTIONS_REFERENCE.md -│ ├─ BUILD_FUNC_EXECUTION_FLOWS.md -│ └─ BUILD_FUNC_USAGE_EXAMPLES.md -│ -├─ core.func/ (5 files) -│ ├─ README.md -│ ├─ CORE_FLOWCHART.md -│ ├─ CORE_FUNCTIONS_REFERENCE.md -│ ├─ CORE_INTEGRATION.md -│ └─ CORE_USAGE_EXAMPLES.md -│ -├─ error_handler.func/ (5 files) -│ ├─ README.md -│ ├─ ERROR_HANDLER_FLOWCHART.md -│ ├─ ERROR_HANDLER_FUNCTIONS_REFERENCE.md -│ ├─ ERROR_HANDLER_INTEGRATION.md -│ └─ ERROR_HANDLER_USAGE_EXAMPLES.md -│ -├─ api.func/ (5 files) -│ ├─ README.md -│ ├─ API_FLOWCHART.md -│ ├─ API_FUNCTIONS_REFERENCE.md -│ ├─ API_INTEGRATION.md -│ └─ API_USAGE_EXAMPLES.md -│ -├─ install.func/ (5 files) -│ ├─ README.md -│ ├─ INSTALL_FUNC_FLOWCHART.md -│ ├─ INSTALL_FUNC_FUNCTIONS_REFERENCE.md -│ ├─ INSTALL_FUNC_INTEGRATION.md -│ └─ INSTALL_FUNC_USAGE_EXAMPLES.md -│ -├─ tools.func/ (6 files) ★ NEW -│ ├─ README.md -│ ├─ TOOLS_FUNC_FLOWCHART.md -│ ├─ TOOLS_FUNC_FUNCTIONS_REFERENCE.md -│ ├─ TOOLS_FUNC_INTEGRATION.md -│ ├─ TOOLS_FUNC_USAGE_EXAMPLES.md -│ └─ TOOLS_FUNC_ENVIRONMENT_VARIABLES.md -│ -├─ alpine-install.func/ (5 files) ★ NEW -│ ├─ README.md -│ ├─ ALPINE_INSTALL_FUNC_FLOWCHART.md -│ ├─ ALPINE_INSTALL_FUNC_FUNCTIONS_REFERENCE.md -│ ├─ ALPINE_INSTALL_FUNC_INTEGRATION.md -│ └─ ALPINE_INSTALL_FUNC_USAGE_EXAMPLES.md -│ -├─ alpine-tools.func/ (5 files) ★ NEW -│ ├─ README.md -│ ├─ ALPINE_TOOLS_FUNC_FLOWCHART.md -│ ├─ ALPINE_TOOLS_FUNC_FUNCTIONS_REFERENCE.md -│ ├─ ALPINE_TOOLS_FUNC_INTEGRATION.md -│ └─ ALPINE_TOOLS_FUNC_USAGE_EXAMPLES.md -│ -└─ cloud-init.func/ (5 files) ★ NEW - ├─ README.md - ├─ CLOUD_INIT_FUNC_FLOWCHART.md - ├─ CLOUD_INIT_FUNC_FUNCTIONS_REFERENCE.md - ├─ CLOUD_INIT_FUNC_INTEGRATION.md - └─ CLOUD_INIT_FUNC_USAGE_EXAMPLES.md -## 📊 **Documentation Statistics** - -| Metric | Count | -|--------|:---:| -| Total Documentation Files | 67 | -| Project Directories Documented | 6 (ct, install, vm, tools, api, misc) | -| Function Libraries Documented | 9 | -| Total Functions Referenced | 150+ | -| Total Lines of Documentation | 15,000+ | -| Code Examples | 50+ | -| Visual Flowcharts | 15+ | - -**New in this update (★ NEW)**: 6 new section directories (ct/, install/, vm/, tools/, api/) mirroring project structure -| Code Examples | 50+ | -| Visual Flowcharts | 15+ | - -**New in this update (★ NEW)**: 5 new function library subdirectories with 25 files - ---- - -## 🎓 **Learning Paths** - -### Path 1: Beginner - First Time Contributing (2-3 hours) - -1. Read: [CONTRIBUTION_GUIDE.md](CONTRIBUTION_GUIDE.md) - Quick Start section -2. Read: [UPDATED_APP-ct.md](UPDATED_APP-ct.md) - Overview -3. Read: [UPDATED_APP-install.md](UPDATED_APP-install.md) - Overview -4. Study: One real example from each guide -5. Create your first ct/app.sh and install/app-install.sh -6. Submit PR! - -### Path 2: Intermediate - Deep Understanding (4-6 hours) - -1. Read: [TECHNICAL_REFERENCE.md](TECHNICAL_REFERENCE.md) -2. Study: [misc/build.func/README.md](misc/build.func/README.md) -3. Study: [misc/tools.func/README.md](misc/tools.func/README.md) -4. Study: [misc/install.func/README.md](misc/install.func/README.md) -5. Review: EXIT_CODES and error handling -6. Create an advanced application with custom setup - -### Path 3: Advanced - Architecture Mastery (8+ hours) - -1. Read all TECHNICAL_REFERENCE.md -2. Study all 9 function libraries in depth: - - Flowchart - - Functions Reference - - Integration Guide - - Usage Examples -3. Review: [CHANGELOG_MISC.md](CHANGELOG_MISC.md) for recent changes -4. Review: [DEFAULTS_SYSTEM_GUIDE.md](DEFAULTS_SYSTEM_GUIDE.md) -5. Study: [DEV_MODE.md](DEV_MODE.md) for debugging -6. Contribute to function libraries or complex applications - -### Path 4: Operator/User - Configuration Focus (1-2 hours) - -1. Read: [DEFAULTS_SYSTEM_GUIDE.md](DEFAULTS_SYSTEM_GUIDE.md) -2. Read: [EXIT_CODES.md](EXIT_CODES.md) - for troubleshooting -3. Read: [DEV_MODE.md](DEV_MODE.md) - for debugging - ---- - -## 🔍 **Search Guide** - -### Looking for... - -**How do I create a ct script?** -→ [UPDATED_APP-ct.md](UPDATED_APP-ct.md) - -**How do I create an install script?** -→ [UPDATED_APP-install.md](UPDATED_APP-install.md) - -**What does exit code 206 mean?** -→ [EXIT_CODES.md](EXIT_CODES.md#container-creation-errors-200-209) - -**How do I debug a failed installation?** -→ [DEV_MODE.md](DEV_MODE.md) - -**What are the default configuration options?** -→ [DEFAULTS_SYSTEM_GUIDE.md](DEFAULTS_SYSTEM_GUIDE.md) - -**What's a function in build.func?** -→ [misc/build.func/BUILD_FUNC_FUNCTIONS_REFERENCE.md](misc/build.func/BUILD_FUNC_FUNCTIONS_REFERENCE.md) - -**How do I install Node.js in a container?** -→ [misc/tools.func/TOOLS_FUNC_FUNCTIONS_REFERENCE.md](misc/tools.func/TOOLS_FUNC_FUNCTIONS_REFERENCE.md#setup_nodejsversion) - -**How do Alpine containers differ from Debian?** -→ [misc/alpine-install.func/README.md](misc/alpine-install.func/README.md) - -**What changed recently in /misc?** -→ [CHANGELOG_MISC.md](CHANGELOG_MISC.md) - ---- - -## ✅ **Documentation Completeness** - -- ✅ All 9 function libraries have dedicated subdirectories -- ✅ Each library has 5-6 detailed documentation files -- ✅ Complete flowcharts for complex processes -- ✅ Alphabetical function references with signatures -- ✅ Real-world usage examples for every pattern -- ✅ Integration guides showing component relationships -- ✅ Best practices documented with DO/DON'T sections -- ✅ Troubleshooting guides for common issues -- ✅ Exit codes fully mapped and explained -- ✅ Architecture documentation with diagrams - ---- - -## 🚀 **Standardized Documentation Pattern** - -Each function library follows this consistent pattern: - -``` -function-library/ -├─ README.md # Quick reference -├─ FUNCTION_LIBRARY_FLOWCHART.md # Visual flows -├─ FUNCTION_LIBRARY_FUNCTIONS_REFERENCE.md # Complete reference -├─ FUNCTION_LIBRARY_INTEGRATION.md # How it connects -├─ FUNCTION_LIBRARY_USAGE_EXAMPLES.md # Real examples -└─ [FUNCTION_LIBRARY_ENVIRONMENT_VARIABLES.md] # (if needed) -``` - -This makes it easy to: -- Find information quickly -- Navigate between related docs -- Understand component relationships -- Learn from examples -- Reference complete function signatures - ---- - -## 📝 **Last Updated** - -- **Date**: December 2025 -- **Version**: 2.0 (Comprehensive Restructure) -- **Status**: ✅ All 9 function libraries fully documented and standardized -- **New This Update**: tools.func/, alpine-install.func/, alpine-tools.func/, cloud-init.func/ subdirectories with complete documentation - ---- - -## 🤝 **Contributing Documentation** - -Found an error or want to improve documentation? - -1. Open an issue: https://github.com/community-scripts/ProxmoxVED/issues -2. Or submit a PR improving documentation -3. See: [CONTRIBUTION_GUIDE.md](CONTRIBUTION_GUIDE.md) for details - ---- - -## 📚 **Related Resources** - -- **GitHub Repository**: https://github.com/community-scripts/ProxmoxVED -- **Proxmox Documentation**: https://pve.proxmox.com/wiki/ -- **Community Discussions**: https://github.com/community-scripts/ProxmoxVED/discussions - ---- - -**Ready to get started?** Choose a learning path above or use the quick navigation. 🚀 diff --git a/docs/README.md b/docs/README.md index facdc9955..099d3edeb 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,12 +1,40 @@ # 📚 ProxmoxVED Documentation -Complete documentation for the ProxmoxVED project - mirroring the project structure with comprehensive guides for every component. +Complete guide to all ProxmoxVED documentation - quickly find what you need. --- -## đŸŽ¯ **Quick Start by Role** +## đŸŽ¯ **Quick Navigation by Goal** -### 👤 **I'm a...** +### 👤 **I want to...** + +**Contribute a new application** +→ Start with: [CONTRIBUTION_GUIDE.md](CONTRIBUTION_GUIDE.md) +→ Then: [ct/DETAILED_GUIDE.md](ct/DETAILED_GUIDE.md) + [install/DETAILED_GUIDE.md](install/DETAILED_GUIDE.md) + +**Understand the architecture** +→ Read: [TECHNICAL_REFERENCE.md](TECHNICAL_REFERENCE.md) +→ Then: [misc/README.md](misc/README.md) + +**Debug a failed installation** +→ Check: [EXIT_CODES.md](EXIT_CODES.md) +→ Then: [DEV_MODE.md](DEV_MODE.md) +→ See also: [misc/error_handler.func/](misc/error_handler.func/) + +**Configure system defaults** +→ Read: [DEFAULTS_SYSTEM_GUIDE.md](DEFAULTS_SYSTEM_GUIDE.md) + +**Learn about recent changes** +→ Check: [CHANGELOG_MISC.md](CHANGELOG_MISC.md) + +**Develop a function library** +→ Study: [misc/](misc/) documentation + +--- + +## 👤 **Quick Start by Role** + +### **I'm a...** **New Contributor** → Start: [CONTRIBUTION_GUIDE.md](CONTRIBUTION_GUIDE.md) @@ -14,12 +42,12 @@ Complete documentation for the ProxmoxVED project - mirroring the project struct **Container Creator** → Read: [ct/README.md](ct/README.md) -→ Guide: [UPDATED_APP-ct.md](UPDATED_APP-ct.md) +→ Deep Dive: [ct/DETAILED_GUIDE.md](ct/DETAILED_GUIDE.md) → Reference: [misc/build.func/](misc/build.func/) **Installation Script Developer** → Read: [install/README.md](install/README.md) -→ Guide: [UPDATED_APP-install.md](UPDATED_APP-install.md) +→ Deep Dive: [install/DETAILED_GUIDE.md](install/DETAILED_GUIDE.md) → Reference: [misc/tools.func/](misc/tools.func/) **VM Provisioner** @@ -45,7 +73,7 @@ Complete documentation for the ProxmoxVED project - mirroring the project struct --- -## 📁 **Documentation Structure** +## 📂 **Documentation Structure** ### Project-Mirrored Directories @@ -53,11 +81,11 @@ Each major project directory has documentation: ``` ProxmoxVED/ -├─ ct/ ↔ docs/ct/README.md -├─ install/ ↔ docs/install/README.md -├─ vm/ ↔ docs/vm/README.md -├─ tools/ ↔ docs/tools/README.md -├─ api/ ↔ docs/api/README.md +├─ ct/ ↔ docs/ct/ (README.md + DETAILED_GUIDE.md) +├─ install/ ↔ docs/install/ (README.md + DETAILED_GUIDE.md) +├─ vm/ ↔ docs/vm/ (README.md) +├─ tools/ ↔ docs/tools/ (README.md) +├─ api/ ↔ docs/api/ (README.md) └─ misc/ ↔ docs/misc/ (9 function libraries) ``` @@ -66,8 +94,8 @@ ProxmoxVED/ | Document | Purpose | Audience | |----------|---------|----------| | [CONTRIBUTION_GUIDE.md](CONTRIBUTION_GUIDE.md) | How to contribute | Contributors | -| [UPDATED_APP-ct.md](UPDATED_APP-ct.md) | Create ct scripts | Container developers | -| [UPDATED_APP-install.md](UPDATED_APP-install.md) | Create install scripts | Installation developers | +| [ct/DETAILED_GUIDE.md](ct/DETAILED_GUIDE.md) | Create ct scripts | Container developers | +| [install/DETAILED_GUIDE.md](install/DETAILED_GUIDE.md) | Create install scripts | Installation developers | | [TECHNICAL_REFERENCE.md](TECHNICAL_REFERENCE.md) | Architecture deep-dive | Architects, advanced users | | [DEFAULTS_SYSTEM_GUIDE.md](DEFAULTS_SYSTEM_GUIDE.md) | Configuration system | Operators, power users | | [EXIT_CODES.md](EXIT_CODES.md) | Exit code reference | Troubleshooters | @@ -83,7 +111,7 @@ Documentation for `/ct` - Container creation scripts that run on the Proxmox hos **Includes**: - Overview of container creation process -- Link to [UPDATED_APP-ct.md](UPDATED_APP-ct.md) guide +- Deep dive: [DETAILED_GUIDE.md](ct/DETAILED_GUIDE.md) - Complete reference with examples - Reference to [misc/build.func/](misc/build.func/) - Quick start for creating new containers @@ -92,7 +120,7 @@ Documentation for `/install` - Scripts that run inside containers to install app **Includes**: - Overview of 10-phase installation pattern -- Link to [UPDATED_APP-install.md](UPDATED_APP-install.md) guide +- Deep dive: [DETAILED_GUIDE.md](install/DETAILED_GUIDE.md) - Complete reference with examples - Reference to [misc/tools.func/](misc/tools.func/) - Alpine vs Debian differences @@ -145,13 +173,12 @@ Documentation for `/misc` - 9 core function libraries with complete references. 1. [CONTRIBUTION_GUIDE.md](CONTRIBUTION_GUIDE.md) - Quick Start 2. Pick your area: - - Containers → [ct/README.md](ct/README.md) - - Installation → [install/README.md](install/README.md) + - Containers → [ct/README.md](ct/README.md) + [ct/DETAILED_GUIDE.md](ct/DETAILED_GUIDE.md) + - Installation → [install/README.md](install/README.md) + [install/DETAILED_GUIDE.md](install/DETAILED_GUIDE.md) - VMs → [vm/README.md](vm/README.md) -3. Read the corresponding UPDATED_APP guide -4. Study existing similar script -5. Create your contribution -6. Submit PR +3. Study existing similar script +4. Create your contribution +5. Submit PR ### Path 2: Intermediate Developer (4-6 hours) @@ -167,8 +194,7 @@ Documentation for `/misc` - 9 core function libraries with complete references. ### Path 3: Advanced Architect (8+ hours) 1. All of Intermediate Path -2. Study all 9 function libraries: - - Each with FLOWCHART, FUNCTIONS_REFERENCE, INTEGRATION, USAGE_EXAMPLES +2. Study all 9 function libraries in depth 3. [DEFAULTS_SYSTEM_GUIDE.md](DEFAULTS_SYSTEM_GUIDE.md) - Configuration system 4. [DEV_MODE.md](DEV_MODE.md) - Debugging and development 5. Design new features or function libraries @@ -186,7 +212,7 @@ Documentation for `/misc` - 9 core function libraries with complete references. | Metric | Count | |--------|:---:| -| **Documentation Files** | 67 | +| **Documentation Files** | 63 | | **Total Lines** | 15,000+ | | **Function Libraries** | 9 | | **Functions Documented** | 150+ | @@ -200,8 +226,8 @@ Documentation for `/misc` - 9 core function libraries with complete references. ## 🔍 **Find It Fast** ### By Feature -- **How do I create a container?** → [UPDATED_APP-ct.md](UPDATED_APP-ct.md) -- **How do I create an install script?** → [UPDATED_APP-install.md](UPDATED_APP-install.md) +- **How do I create a container?** → [ct/DETAILED_GUIDE.md](ct/DETAILED_GUIDE.md) +- **How do I create an install script?** → [install/DETAILED_GUIDE.md](install/DETAILED_GUIDE.md) - **How do I create a VM?** → [vm/README.md](vm/README.md) - **How do I install Node.js?** → [misc/tools.func/](misc/tools.func/) - **How do I debug?** → [DEV_MODE.md](DEV_MODE.md) @@ -230,7 +256,7 @@ Documentation for `/misc` - 9 core function libraries with complete references. - ✅ **Best practices** - DO/DON'T sections throughout - ✅ **Learning paths** - Structured curriculum by role - ✅ **Quick references** - Fast lookup by error code -- ✅ **Comprehensive index** → [INDEX.md](INDEX.md) +- ✅ **Comprehensive navigation** - This page --- @@ -238,7 +264,7 @@ Documentation for `/misc` - 9 core function libraries with complete references. **New to ProxmoxVED?** → [CONTRIBUTION_GUIDE.md](CONTRIBUTION_GUIDE.md) -**Looking for something specific?** → [INDEX.md](INDEX.md) +**Looking for something specific?** → Choose your role above or browse by directory **Need to debug?** → [EXIT_CODES.md](EXIT_CODES.md) @@ -250,7 +276,7 @@ Documentation for `/misc` - 9 core function libraries with complete references. Found an error? Want to improve docs? -1. Open issue: https://github.com/community-scripts/ProxmoxVED/issues +1. Open issue: [GitHub Issues](https://github.com/community-scripts/ProxmoxVED/issues) 2. Or submit PR with improvements 3. See [CONTRIBUTION_GUIDE.md](CONTRIBUTION_GUIDE.md) for details @@ -259,10 +285,10 @@ Found an error? Want to improve docs? ## 📝 **Status** - **Last Updated**: December 2025 -- **Version**: 2.1 (Project Structure Mirror) +- **Version**: 2.3 (Consolidated & Reorganized) - **Completeness**: ✅ 100% - All components documented - **Quality**: ✅ Production-ready -- **Examples**: ✅ 50+ tested examples +- **Structure**: ✅ Clean and organized --- diff --git a/docs/alpine-install.func.md b/docs/alpine-install.func.md deleted file mode 100644 index f71f4de1f..000000000 --- a/docs/alpine-install.func.md +++ /dev/null @@ -1,651 +0,0 @@ -# Alpine-Install.func Wiki - -A specialized module for Alpine Linux LXC container setup and configuration, providing functions for IPv6 management, network verification, OS updates, SSH configuration, timezone validation, and passwordless auto-login customization. - ---- - -## 📋 Table of Contents - -- [Overview](#overview) -- [Initialization & Signal Handling](#initialization--signal-handling) -- [Network & Connectivity Functions](#network--connectivity-functions) -- [OS Configuration Functions](#os-configuration-functions) -- [SSH & MOTD Configuration](#ssh--motd-configuration) -- [Container Customization](#container-customization) -- [Best Practices](#best-practices) -- [Error Handling](#error-handling) -- [Contributing](#contributing) - ---- - -## Overview - -This module provides Alpine Linux-specific installation and configuration functions used inside LXC containers during the setup phase. Key capabilities include: - -- ✅ IPv6 enablement/disablement with persistent configuration -- ✅ Network connectivity verification with retry logic -- ✅ Alpine Linux OS updates via apk package manager -- ✅ SSH daemon and MOTD configuration -- ✅ Passwordless root auto-login setup -- ✅ Timezone validation for Alpine containers -- ✅ Comprehensive error handling with signal traps - -### Integration Pattern - -```bash -# Alpine container scripts load this module via curl -source <(curl -fsSL https://git.community-scripts.org/.../alpine-install.func) -load_functions # Initialize core utilities -catch_errors # Setup error handling and signal traps -``` - ---- - -## Initialization & Signal Handling - -### Module Dependencies - -The module automatically sources two required dependencies: - -```bash -source <(curl -fsSL .../core.func) # Color codes, icons, message functions -source <(curl -fsSL .../error_handler.func) # Error handling and exit codes -load_functions # Initialize color/formatting -catch_errors # Setup trap handlers -``` - -### Signal Trap Configuration - -```bash -set -Eeuo pipefail # Strict error mode -trap 'error_handler $? $LINENO "$BASH_COMMAND"' ERR -trap on_exit EXIT # Cleanup on exit -trap on_interrupt INT # Handle Ctrl+C (SIGINT) -trap on_terminate TERM # Handle SIGTERM -``` - ---- - -## Network & Connectivity Functions - -### `verb_ip6()` - -**Purpose**: Configures IPv6 settings and sets verbose mode based on environment variables. - -**Signature**: -```bash -verb_ip6() -``` - -**Parameters**: None - -**Returns**: No explicit return value (configures system state) - -**Environment Effects**: -- Sets `STD` variable to control output verbosity (via `set_std_mode()`) -- If `DISABLEIPV6=yes`: disables IPv6 system-wide via sysctl -- Modifies `/etc/sysctl.conf` for persistent IPv6 disabled state - -**Implementation Pattern**: -```bash -verb_ip6() { - set_std_mode # Initialize STD="" or STD="silent" - - if [ "$DISABLEIPV6" == "yes" ]; then - $STD sysctl -w net.ipv6.conf.all.disable_ipv6=1 - echo "net.ipv6.conf.all.disable_ipv6 = 1" >>/etc/sysctl.conf - $STD rc-update add sysctl default - fi -} -``` - -**Usage Examples**: - -```bash -# Example 1: With IPv6 disabled -DISABLEIPV6="yes" -VERBOSE="no" -verb_ip6 -# Result: IPv6 disabled, changes persisted to sysctl.conf - -# Example 2: Keep IPv6 enabled (default) -DISABLEIPV6="no" -verb_ip6 -# Result: IPv6 remains enabled, no configuration changes -``` - ---- - -### `setting_up_container()` - -**Purpose**: Verifies network connectivity by checking for assigned IP addresses and retrying if necessary. - -**Signature**: -```bash -setting_up_container() -``` - -**Parameters**: None (uses global `RETRY_NUM` and `RETRY_EVERY`) - -**Returns**: 0 on success; exits with code 1 if network unavailable after retries - -**Environment Side Effects**: -- Requires: `RETRY_NUM` (max attempts, default: 10), `RETRY_EVERY` (seconds between retries, default: 3) -- Uses: `CROSS`, `RD`, `CL`, `GN`, `BL` color variables from core.func -- Calls: `msg_info()`, `msg_ok()` message functions - -**Implementation Pattern**: -```bash -setting_up_container() { - msg_info "Setting up Container OS" - i=$RETRY_NUM # Use global counter - while [ $i -gt 0 ]; do - # Check for non-loopback IPv4 address - if [ "$(ip addr show | grep 'inet ' | grep -v '127.0.0.1' | ...)" != "" ]; then - break - fi - echo 1>&2 -en "${CROSS}${RD} No Network! " - sleep $RETRY_EVERY - i=$((i - 1)) - done - - # If still no network after retries, exit with error - if [ "$(ip addr show | grep 'inet ' | grep -v '127.0.0.1' | ...)" = "" ]; then - exit 1 - fi - msg_ok "Network Connected: ${BL}$(ip addr show | grep 'inet ' | awk '{print $2}' | ...)${CL}" -} -``` - -**Usage Examples**: - -```bash -# Example 1: Network available immediately -RETRY_NUM=10 -RETRY_EVERY=3 -setting_up_container -# Output: -# ℹī¸ Setting up Container OS -# ✔ī¸ Set up Container OS -# ✔ī¸ Network Connected: 10.0.3.50 - -# Example 2: Network delayed by 6 seconds (2 retries) -# Script waits 3 seconds x 2, then succeeds -# Output shows retry messages, then success -``` - ---- - -### `network_check()` - -**Purpose**: Comprehensive network connectivity verification for both IPv4 and IPv6, including DNS resolution checks for Git-related domains. - -**Signature**: -```bash -network_check() -``` - -**Parameters**: None - -**Returns**: 0 on success; exits with code 1 if DNS critical failure - -**Environment Side Effects**: -- Temporarily disables error trap (`set +e`, `trap - ERR`) -- Modifies error handling to allow graceful failure detection -- Re-enables error trap at end of function -- Calls: `msg_ok()`, `msg_error()`, `fatal()` message functions - -**Implementation Pattern**: -```bash -network_check() { - set +e - trap - ERR - - # Test IPv4 via multiple DNS servers - if ping -c 1 -W 1 1.1.1.1 &>/dev/null || ...; then - ipv4_status="${GN}✔${CL} IPv4" - else - ipv4_status="${RD}✖${CL} IPv4" - # Prompt user to continue without internet - fi - - # Verify DNS resolution for GitHub domains - RESOLVEDIP=$(getent hosts github.com | awk '{ print $1 }') - if [[ -z "$RESOLVEDIP" ]]; then - msg_error "Internet: ${ipv4_status} DNS Failed" - else - msg_ok "Internet: ${ipv4_status} DNS: ${BL}${RESOLVEDIP}${CL}" - fi - - set -e - trap 'error_handler $LINENO "$BASH_COMMAND"' ERR -} -``` - -**Usage Examples**: - -```bash -# Example 1: Good connectivity -network_check -# Output: -# ✔ī¸ Network Connected: IPv4 -# ✔ī¸ Internet: ✔ IPv4 DNS: 140.82.113.3 - -# Example 2: No internet, user continues anyway -# Output prompts: "Internet NOT connected. Continue anyway? " -# If user enters 'y': -# ⚠ī¸ Expect Issues Without Internet -``` - ---- - -## OS Configuration Functions - -### `update_os()` - -**Purpose**: Updates Alpine Linux OS packages and installs Alpine-specific tools library for additional setup functions. - -**Signature**: -```bash -update_os() -``` - -**Parameters**: None - -**Returns**: No explicit return value (updates system) - -**Environment Side Effects**: -- Runs `apk update && apk upgrade` -- Sources alpine-tools.func for Alpine-specific package installation helpers -- Uses `$STD` wrapper to suppress output unless `VERBOSE=yes` -- Calls: `msg_info()`, `msg_ok()` message functions - -**Implementation Pattern**: -```bash -update_os() { - msg_info "Updating Container OS" - $STD apk update && $STD apk upgrade - source <(curl -fsSL https://git.community-scripts.org/.../alpine-tools.func) - msg_ok "Updated Container OS" -} -``` - -**Usage Examples**: - -```bash -# Example 1: Standard update -VERBOSE="no" -update_os -# Output: -# ℹī¸ Updating Container OS -# ✔ī¸ Updated Container OS -# (Output suppressed via $STD) - -# Example 2: Verbose mode -VERBOSE="yes" -update_os -# Output shows all apk operations plus success message -``` - ---- - -## SSH & MOTD Configuration - -### `motd_ssh()` - -**Purpose**: Configures Message of the Day (MOTD) with container information and enables SSH root access if required. - -**Signature**: -```bash -motd_ssh() -``` - -**Parameters**: None - -**Returns**: No explicit return value (configures system) - -**Environment Side Effects**: -- Modifies `/root/.bashrc` to set TERM environment variable -- Creates `/etc/profile.d/00_lxc-details.sh` with container information script -- If `SSH_ROOT=yes`: modifies `/etc/ssh/sshd_config` and starts SSH daemon -- Uses: `APPLICATION`, `SSH_ROOT` variables from environment -- Requires: color variables (`BOLD`, `YW`, `RD`, `GN`, `CL`) from core.func - -**Implementation Pattern**: -```bash -motd_ssh() { - # Configure TERM for better terminal support - echo "export TERM='xterm-256color'" >>/root/.bashrc - - # Gather OS information - OS_NAME=$(grep ^NAME /etc/os-release | cut -d= -f2 | tr -d '"') - OS_VERSION=$(grep ^VERSION_ID /etc/os-release | cut -d= -f2 | tr -d '"') - IP=$(ip -4 addr show eth0 | awk '/inet / {print $2}' | cut -d/ -f1 | head -n 1) - - # Create MOTD script with container details - PROFILE_FILE="/etc/profile.d/00_lxc-details.sh" - cat > "$PROFILE_FILE" <<'EOF' -echo -e "" -echo -e "${BOLD}${YW}${APPLICATION} LXC Container - DEV Repository${CL}" -echo -e "${RD}WARNING: This is a DEVELOPMENT version (ProxmoxVED). Do NOT use in production!${CL}" -echo -e "${YW} OS: ${GN}${OS_NAME} - Version: ${OS_VERSION}${CL}" -echo -e "${YW} Hostname: ${GN}$(hostname)${CL}" -echo -e "${YW} IP Address: ${GN}${IP}${CL}" -echo -e "${YW} Repository: ${GN}https://github.com/community-scripts/ProxmoxVED${CL}" -echo "" -EOF - - # Enable SSH root access if configured - if [[ "${SSH_ROOT}" == "yes" ]]; then - sed -i "s/#PermitRootLogin prohibit-password/PermitRootLogin yes/g" /etc/ssh/sshd_config - rc-update add sshd - /etc/init.d/sshd start - fi -} -``` - -**Usage Examples**: - -```bash -# Example 1: MOTD configuration with SSH enabled -APPLICATION="MyApp" -SSH_ROOT="yes" -motd_ssh -# Result: SSH daemon started and set to auto-start, MOTD shows app info - -# Example 2: MOTD only (SSH disabled) -APPLICATION="MyApp" -SSH_ROOT="no" -motd_ssh -# Result: MOTD configured but SSH remains disabled -``` - ---- - -## Container Customization - -### `validate_tz()` - -**Purpose**: Validates that a timezone string exists in the Alpine Linux timezone database. - -**Signature**: -```bash -validate_tz() -``` - -**Parameters**: -- `$1` - Timezone string (e.g., "America/New_York", "UTC", "Europe/London") - -**Returns**: 0 if timezone file exists, 1 if invalid - -**Implementation Pattern**: -```bash -validate_tz() { - [[ -f "/usr/share/zoneinfo/$1" ]] # Bash test operator returns success/failure -} -``` - -**Usage Examples**: - -```bash -# Example 1: Valid timezone -validate_tz "America/New_York" -echo $? # Output: 0 - -# Example 2: Invalid timezone -validate_tz "Invalid/Timezone" -echo $? # Output: 1 - -# Example 3: UTC (always valid) -validate_tz "UTC" -echo $? # Output: 0 -``` - ---- - -### `customize()` - -**Purpose**: Configures container for passwordless root auto-login and creates update script for easy application re-deployment. - -**Signature**: -```bash -customize() -``` - -**Parameters**: None (uses global `PASSWORD` and `app` variables) - -**Returns**: No explicit return value (configures system) - -**Environment Side Effects**: -- If `PASSWORD=""` (empty): - * Removes password prompt from root login - * Drops user into shell automatically - * Creates autologin boot script at `/etc/local.d/autologin.start` - * Creates `.hushlogin` to suppress login banners - * Registers script with rc-update -- Creates `/usr/bin/update` script for application updates -- Requires: `app` variable (application name in lowercase) -- Calls: `msg_info()`, `msg_ok()` message functions - -**Implementation Pattern**: -```bash -customize() { - if [[ "$PASSWORD" == "" ]]; then - msg_info "Customizing Container" - - # Remove password requirement - passwd -d root >/dev/null 2>&1 - - # Install util-linux if needed - apk add --no-cache --force-broken-world util-linux >/dev/null 2>&1 - - # Create autologin startup script - mkdir -p /etc/local.d - cat > /etc/local.d/autologin.start <<'EOF' -#!/bin/sh -sed -i 's|^tty1::respawn:.*|tty1::respawn:/sbin/agetty --autologin root --noclear tty1 38400 linux|' /etc/inittab -kill -HUP 1 -EOF - chmod +x /etc/local.d/autologin.start - touch /root/.hushlogin - - rc-update add local >/dev/null 2>&1 - /etc/local.d/autologin.start - - msg_ok "Customized Container" - fi - - # Create update script - echo "bash -c \"\$(curl -fsSL https://github.com/community-scripts/ProxmoxVED/raw/main/ct/${app}.sh)\"" >/usr/bin/update - chmod +x /usr/bin/update -} -``` - -**Usage Examples**: - -```bash -# Example 1: Passwordless auto-login -PASSWORD="" -app="myapp" -customize -# Result: Root login without password, auto-login configured -# User can type: /usr/bin/update to re-run application setup - -# Example 2: Password-protected login -PASSWORD="MySecurePassword" -customize -# Result: Auto-login skipped, password remains active -# Update script still created for re-deployment -``` - ---- - -## Best Practices - -### 1. **Initialization Order** - -Always follow this sequence in Alpine install scripts: - -```bash -#!/bin/sh -set -Eeuo pipefail - -# 1. Ensure curl is available for sourcing functions -if ! command -v curl >/dev/null 2>&1; then - apk update && apk add curl >/dev/null 2>&1 -fi - -# 2. Source dependencies in correct order -source <(curl -fsSL .../core.func) -source <(curl -fsSL .../error_handler.func) - -# 3. Initialize function libraries -load_functions # Sets up colors, formatting, icons -catch_errors # Configures error traps and signal handlers - -# 4. Now safe to call alpine-install.func functions -verb_ip6 -setting_up_container -network_check -update_os -``` - -### 2. **Signal Handling** - -Alpine-install.func provides comprehensive signal trap setup: - -```bash -# ERR trap: Catches all command failures -trap 'error_handler $? $LINENO "$BASH_COMMAND"' ERR - -# EXIT trap: Cleanup on normal or abnormal termination -trap on_exit EXIT - -# INT trap: Handle Ctrl+C gracefully -trap on_interrupt INT - -# TERM trap: Handle SIGTERM signal -trap on_terminate TERM -``` - -### 3. **Network Configuration** - -Use retry logic when network may not be immediately available: - -```bash -setting_up_container # Retries up to RETRY_NUM times -network_check # Validates DNS and Internet -``` - -### 4. **IPv6 Considerations** - -For production Alpine containers: - -```bash -# Disable IPv6 if not needed (reduces attack surface) -DISABLEIPV6="yes" -verb_ip6 - -# Or keep enabled (default): -DISABLEIPV6="no" -# No configuration needed -``` - -### 5. **Error Handling with Color Output** - -Functions use color-coded message output: - -```bash -msg_info # Informational messages (yellow) -msg_ok # Success messages (green) -msg_error # Error messages (red) -msg_warn # Warning messages (orange) -``` - ---- - -## Error Handling - -The module implements comprehensive error handling: - -### Exit Codes - -| Code | Meaning | -|------|---------| -| 0 | Success | -| 1 | General error (network unavailable, DNS failed, etc.) | -| 130 | Interrupted by user (SIGINT) | -| 143 | Terminated by signal (SIGTERM) | - -### Error Handler Function - -The error_handler receives three parameters: - -```bash -error_handler() { - local exit_code="$1" # Exit code from failed command - local line_number="$2" # Line where error occurred - local command="$3" # Command that failed - - # Errors are reported with line number and command details - # Stack trace available for debugging -} -``` - -### Debug Variables - -Available for troubleshooting: - -```bash -$VERBOSE # Set to "yes" to show all output -$DEV_MODE_TRACE # Set to "true" for bash -x tracing -$DEV_MODE_LOGS # Set to "true" to persist logs -``` - ---- - -## Contributing - -### Adding New Functions - -When adding Alpine-specific functions: - -1. Follow the established naming convention: `function_purpose()` -2. Include comprehensive docstring with signature, parameters, returns -3. Use color variables from core.func for output consistency -4. Handle errors via error_handler trap -5. Document all environment variable dependencies - -### Testing New Functions - -```bash -# Test function in isolation with error traps: -set -Eeuo pipefail -trap 'error_handler $? $LINENO "$BASH_COMMAND"' ERR - -source <(curl -fsSL .../core.func) -source <(curl -fsSL .../error_handler.func) -load_functions -catch_errors - -# Now test your function: -your_function -``` - -### Compatibility - -- Alpine Linux 3.16+ (uses ash shell compatible syntax) -- OpenRC init system (rc-update, rc-service) -- Requires: core.func, error_handler.func -- Optional: alpine-tools.func (for extended package management) - ---- - -## Notes - -- Functions are designed for execution **inside** LXC containers (not on Proxmox host) -- Alpine uses `apk` package manager (not `apt`) -- Alpine uses OpenRC (not systemd) - use `rc-update` and `/etc/init.d/` commands -- IPv6 can be disabled for security/performance but is enabled by default -- Auto-login configuration persists across container reboots via rc-update - diff --git a/docs/alpine-tools.func.md b/docs/alpine-tools.func.md deleted file mode 100644 index 5271469c3..000000000 --- a/docs/alpine-tools.func.md +++ /dev/null @@ -1,588 +0,0 @@ -# Alpine-Tools.func Wiki - -Alpine Linux-specific tool setup and package management module providing helper functions optimized for Alpine's apk package manager and minimal container environment. - ---- - -## 📋 Table of Contents - -- [Overview](#overview) -- [Helper Functions](#helper-functions) -- [GitHub Release Functions](#github-release-functions) -- [Tool Installation Patterns](#tool-installation-patterns) -- [Package Management](#package-management) -- [Best Practices](#best-practices) -- [Debugging](#debugging) -- [Contributing](#contributing) - ---- - -## Overview - -Alpine-tools.func provides **Alpine Linux-specific utilities**: - -- ✅ Alpine apk package manager wrapper -- ✅ GitHub release version checking and installation -- ✅ Tool caching and version tracking -- ✅ Progress reporting with pv (pipe viewer) -- ✅ Network resolution helpers for Alpine -- ✅ PATH persistence across sessions -- ✅ Retry logic for failed downloads -- ✅ Minimal dependencies philosophy (Alpine ~5MB containers) - -### Key Differences from Debian/Ubuntu - -| Feature | Alpine | Debian/Ubuntu | -|---------|--------|---------------| -| Package Manager | apk | apt-get, dpkg | -| Shell | ash (dash variant) | bash | -| Init System | OpenRC | systemd | -| Size | ~5MB base | ~100MB+ base | -| Libc | musl | glibc | -| Find getent | Not installed | Installed | - -### Integration Pattern - -```bash -#!/bin/sh # Alpine uses ash, not bash -source <(curl -fsSL .../core.func) -source <(curl -fsSL .../alpine-tools.func) -load_functions - -# Now Alpine-specific tool functions available -need_tool curl jq # Install if missing -check_for_gh_release "myapp" "owner/repo" -``` - ---- - -## Helper Functions - -### `lower()` - -**Purpose**: Converts string to lowercase (portable ash function). - -**Signature**: -```bash -lower() -``` - -**Parameters**: -- `$1` - String to convert - -**Returns**: Lowercase string on stdout - -**Behavior**: -```bash -# Alpine's tr works with character classes -printf '%s' "$1" | tr '[:upper:]' '[:lower:]' -``` - -**Usage Examples**: - -```bash -# Example 1: App name normalization -result=$(lower "MyApp") -echo "$result" # Output: myapp - -# Example 2: In variable assignment -app_dir=$(lower "$APPLICATION") -mkdir -p /opt/$app_dir -``` - ---- - -### `has()` - -**Purpose**: Checks if command is available in PATH. - -**Signature**: -```bash -has() -``` - -**Parameters**: -- `$1` - Command name - -**Returns**: 0 if available, 1 if not - -**Implementation**: -```bash -has() { - command -v "$1" >/dev/null 2>&1 -} -``` - -**Usage Examples**: - -```bash -# Example 1: Check availability -if has jq; then - echo "jq is installed" -else - echo "jq is not installed" -fi - -# Example 2: In conditionals -has docker && docker ps || echo "Docker not installed" -``` - ---- - -### `need_tool()` - -**Purpose**: Ensures specified tools are installed, installs missing ones via apk. - -**Signature**: -```bash -need_tool() -``` - -**Parameters**: -- `$@` - Tool names (space-separated) - -**Returns**: 0 on success, 1 if installation failed - -**Behavior**: -```bash -# Checks each tool -# If any missing: runs apk add for all -# Displays message before and after -``` - -**Error Handling**: -- Returns 1 if apk add fails -- Shows which tools failed -- Suggests checking package names - -**Usage Examples**: - -```bash -# Example 1: Ensure common tools available -need_tool curl jq unzip git -# Installs any missing packages - -# Example 2: Optional tool check -if need_tool myapp-cli; then - myapp-cli --version -else - echo "myapp-cli not available in apk" -fi - -# Example 3: With error handling -need_tool docker || { - echo "Failed to install docker" - exit 1 -} -``` - ---- - -### `net_resolves()` - -**Purpose**: Checks if hostname resolves and responds (Alpine-friendly DNS test). - -**Signature**: -```bash -net_resolves() -``` - -**Parameters**: -- `$1` - Hostname to test - -**Returns**: 0 if resolves and responds, 1 if fails - -**Behavior**: -```bash -# Alpine doesn't have getent by default -# Falls back to nslookup if ping fails -# Returns success if either works - -ping -c1 -W1 "$host" >/dev/null 2>&1 || nslookup "$host" >/dev/null 2>&1 -``` - -**Usage Examples**: - -```bash -# Example 1: Test GitHub connectivity -if net_resolves api.github.com; then - echo "Can reach GitHub API" -else - echo "GitHub API unreachable" -fi - -# Example 2: In download function -net_resolves download.example.com || { - echo "Download server not reachable" - exit 1 -} -``` - ---- - -### `ensure_usr_local_bin_persist()` - -**Purpose**: Ensures `/usr/local/bin` is in PATH across all shell sessions. - -**Signature**: -```bash -ensure_usr_local_bin_persist() -``` - -**Parameters**: None - -**Returns**: No explicit return value (modifies system) - -**Behavior**: -```bash -# Creates /etc/profile.d/10-localbin.sh -# Script adds /usr/local/bin to PATH if not already present -# Runs on every shell startup - -# Alpine uses /etc/profile for login shells -# profile.d scripts sourced automatically -``` - -**Implementation**: -```bash -PROFILE_FILE="/etc/profile.d/10-localbin.sh" -if [ ! -f "$PROFILE_FILE" ]; then - echo 'case ":$PATH:" in *:/usr/local/bin:*) ;; *) export PATH="/usr/local/bin:$PATH";; esac' > "$PROFILE_FILE" - chmod +x "$PROFILE_FILE" -fi -``` - -**Usage Examples**: - -```bash -# Example 1: Make sure local tools available -ensure_usr_local_bin_persist -# Now /usr/local/bin binaries always in PATH - -# Example 2: After installing custom tool -cp ./my-tool /usr/local/bin/ -ensure_usr_local_bin_persist -# Tool immediately accessible in PATH -``` - ---- - -### `download_with_progress()` - -**Purpose**: Downloads file with progress bar (if pv available) or simple # progress. - -**Signature**: -```bash -download_with_progress() -``` - -**Parameters**: -- `$1` - URL to download -- `$2` - Destination file path - -**Returns**: 0 on success, 1 on failure - -**Behavior**: -```bash -# Attempts to get content-length header -# If available: pipes through pv for progress bar -# If not: uses curl's built-in # progress -# Shows errors clearly -``` - -**Requirements**: -- `curl` - For downloading -- `pv` - Optional, for progress bar -- Destination directory must exist - -**Usage Examples**: - -```bash -# Example 1: Simple download -download_with_progress "https://example.com/file.tar.gz" "/tmp/file.tar.gz" -# Shows progress bar if pv available - -# Example 2: With error handling -if download_with_progress "$URL" "$DEST"; then - echo "Downloaded successfully" - tar -xzf "$DEST" -else - echo "Download failed" - exit 1 -fi -``` - ---- - -## GitHub Release Functions - -### `check_for_gh_release()` - -**Purpose**: Checks GitHub releases for available updates and compares with currently installed version. - -**Signature**: -```bash -check_for_gh_release() -``` - -**Parameters**: -- `$1` - Application name (e.g., "nodejs") -- `$2` - GitHub repository (e.g., "nodejs/node") -- `$3` - Pinned version (optional, e.g., "20.0.0") - -**Returns**: 0 if update needed, 1 if current or pinned - -**Environment Variables Set**: -- `CHECK_UPDATE_RELEASE` - Latest available version (without v prefix) - -**Behavior**: -```bash -# 1. Check network to api.github.com -# 2. Fetch latest release tag via GitHub API -# 3. Compare with installed version (stored in ~/.appname) -# 4. Show appropriate message: -# - "app pinned to vX.X.X (no update)" -# - "app pinned vX.X.X (upstream vY.Y.Y) → update/downgrade" -# - "Update available: vA.A.A → vB.B.B" -# - "Already up to date" -``` - -**File Storage**: -```bash -~/.${app_lc} # File contains current version string -# Example: ~/.nodejs contains "20.10.0" -``` - -**Usage Examples**: - -```bash -# Example 1: Check for update -check_for_gh_release "nodejs" "nodejs/node" -# Output: "Update available: v18.0.0 → v20.10.0" -# Sets: CHECK_UPDATE_RELEASE="20.10.0" - -# Example 2: Pinned version (no update) -check_for_gh_release "nodejs" "nodejs/node" "20.0.0" -# Output: "app pinned to v20.0.0 (no update)" -# Returns 1 (no update available) - -# Example 3: With error handling -if check_for_gh_release "myapp" "owner/myapp"; then - echo "Update available: $CHECK_UPDATE_RELEASE" - download_and_install -fi -``` - ---- - -## Tool Installation Patterns - -### Pattern 1: Simple Package Installation - -```bash -#!/bin/sh -need_tool curl jq # Ensure tools available -# Continue with script -``` - -### Pattern 2: GitHub Release Installation - -```bash -#!/bin/sh -source <(curl -fsSL .../alpine-tools.func) -load_functions - -# Check for updates -check_for_gh_release "myapp" "owner/myapp" - -# Download from GitHub releases -RELEASE="$CHECK_UPDATE_RELEASE" -URL="https://github.com/owner/myapp/releases/download/v${RELEASE}/myapp-alpine.tar.gz" - -download_with_progress "$URL" "/tmp/myapp-${RELEASE}.tar.gz" -tar -xzf "/tmp/myapp-${RELEASE}.tar.gz" -C /usr/local/bin/ -``` - -### Pattern 3: Version Pinning - -```bash -#!/bin/sh -# For specific use case, pin to known good version -check_for_gh_release "nodejs" "nodejs/node" "20.10.0" -# Will use 20.10.0 even if 21.0.0 available -``` - ---- - -## Package Management - -### Alpine Package Naming - -Alpine packages often have different names than Debian: - -| Tool | Alpine | Debian | -|------|--------|--------| -| curl | curl | curl | -| Git | git | git | -| Docker | docker | docker.io | -| PostgreSQL | postgresql-client | postgresql-client | -| Build tools | build-base | build-essential | -| Development headers | -dev packages | -dev packages | - -### Finding Alpine Packages - -```bash -# Search for package -apk search myapp - -# Show package info -apk info -d myapp - -# List available versions -apk search myapp --all -``` - -### Installing Alpine Packages - -```bash -# Basic install (not cached) -apk add curl git - -# Install with --no-cache (for containers) -apk add --no-cache curl git - -# Force broken packages (last resort) -apk add --no-cache --force-broken-world util-linux -``` - ---- - -## Best Practices - -### 1. **Use `--no-cache` in Containers** - -```bash -# Good: Saves space in container -apk add --no-cache curl git - -# Avoid: Wastes space -apk update && apk add curl git -``` - -### 2. **Check Tools Before Using** - -```bash -# Good: Graceful error -if ! has jq; then - need_tool jq || exit 1 -fi - -# Using jq safely -jq . < input.json -``` - -### 3. **Use need_tool() for Multiple** - -```bash -# Good: Install all at once -need_tool curl jq git unzip - -# Less efficient: Individual checks -has curl || apk add curl -has jq || apk add jq -``` - -### 4. **Ensure Persistence** - -```bash -# For custom tools in /usr/local/bin -ensure_usr_local_bin_persist - -# Now available in all future shells -/usr/local/bin/my-custom-tool -``` - -### 5. **Handle Network Failures** - -```bash -# Alpine often in isolated environments -if ! net_resolves api.github.com; then - echo "GitHub API unreachable" - # Fallback to local package or error - exit 1 -fi -``` - ---- - -## Debugging - -### Check Package Availability - -```bash -# List all available packages -apk search --all - -# Find package by keyword -apk search curl - -# Get specific package info -apk info postgresql-client -``` - -### Verify Installation - -```bash -# Check if tool installed -apk info | grep myapp - -# Verify PATH -which curl -echo $PATH -``` - -### Network Testing - -```bash -# Test DNS -nslookup api.github.com - -# Test connectivity -ping -c1 1.1.1.1 - -# Test download -curl -I https://api.github.com -``` - ---- - -## Contributing - -### Adding New Helper Functions - -When adding Alpine-specific helpers: - -1. Use POSIX shell (ash-compatible) -2. Avoid bash-isms -3. Include error handling -4. Document with examples -5. Test on actual Alpine container - -### Improving Package Installation - -New patterns could support: -- Automatic Alpine version detection -- Package version pinning -- Dependency resolution -- Conflict detection - ---- - -## Notes - -- Alpine uses **ash shell** (POSIX-compatible, not bash) -- Alpine **apk is fast** and has minimal overhead -- Alpine containers **~5MB base image** (vs 100MB+ for Debian) -- **No getent available** by default (use nslookup fallback) -- GitHub releases can be **pre-compiled for Alpine musl** - diff --git a/docs/api.func.md b/docs/api.func.md deleted file mode 100644 index 17c635400..000000000 --- a/docs/api.func.md +++ /dev/null @@ -1,670 +0,0 @@ -# API.func Wiki - -A telemetry and diagnostics module providing anonymous statistics collection and API integration with the Community-Scripts infrastructure for tracking container/VM creation metrics and installation success/failure data. - ---- - -## 📋 Table of Contents - -- [Overview](#overview) -- [Exit Code Reference](#exit-code-reference) -- [Telemetry Functions](#telemetry-functions) -- [API Payload Structure](#api-payload-structure) -- [Privacy & Opt-Out](#privacy--opt-out) -- [Error Mapping](#error-mapping) -- [Best Practices](#best-practices) -- [API Integration](#api-integration) -- [Contributing](#contributing) - ---- - -## Overview - -The API.func module provides anonymous telemetry reporting to Community-Scripts infrastructure, enabling: - -- ✅ Container/VM creation statistics collection -- ✅ Installation success/failure tracking -- ✅ Comprehensive exit code mapping and explanation -- ✅ Anonymous session-based tracking (UUID) -- ✅ Privacy-respecting data collection (no personal data) -- ✅ Opt-out capability via DIAGNOSTICS setting -- ✅ Consistent error reporting across all scripts - -### Integration Points - -```bash -# In container build scripts (on Proxmox host): -source <(curl -fsSL .../api.func) -post_to_api # Report container creation -post_update_to_api # Report installation completion - -# Error handling (in all scripts): -source <(curl -fsSL .../error_handler.func) -# explain_exit_code shared for consistent mappings -``` - -### Data Flow - -``` -Container/VM Creation - ↓ - post_to_api() - ↓ -Community-Scripts API - ↓ -Anonymous Statistics -(No personal data) -``` - ---- - -## Exit Code Reference - -### Category 1: Generic / Shell Errors - -| Code | Meaning | Recovery | -|------|---------|----------| -| 1 | General error / Operation not permitted | Check permissions, re-run command | -| 2 | Misuse of shell builtins (syntax error) | Fix shell syntax, validate script | -| 126 | Command invoked cannot execute | Fix file permissions (chmod +x) | -| 127 | Command not found | Install missing package or tool | -| 128 | Invalid argument to exit | Check exit code parameter (0-255) | -| 130 | Terminated by Ctrl+C (SIGINT) | User interrupted - retry manually | -| 137 | Killed (SIGKILL / Out of memory) | Insufficient RAM - increase allocation | -| 139 | Segmentation fault (core dumped) | Serious application bug - contact support | -| 143 | Terminated (SIGTERM) | System shutdown or manual termination | - -### Category 2: Package Manager Errors - -| Code | Meaning | Recovery | -|------|---------|----------| -| 100 | APT: Package manager error (broken packages) | Run `apt --fix-broken install` | -| 101 | APT: Configuration error (bad sources.list) | Fix /etc/apt/sources.list, re-run apt update | -| 255 | DPKG: Fatal internal error | Run `dpkg --configure -a` | - -### Category 3: Node.js / npm Errors - -| Code | Meaning | Recovery | -|------|---------|----------| -| 243 | Node.js: Out of memory (heap out of memory) | Increase container RAM, reduce workload | -| 245 | Node.js: Invalid command-line option | Check node/npm arguments | -| 246 | Node.js: Internal JavaScript Parse Error | Update Node.js version | -| 247 | Node.js: Fatal internal error | Check Node.js installation integrity | -| 248 | Node.js: Invalid C++ addon / N-API failure | Rebuild native modules | -| 249 | Node.js: Inspector error | Disable debugger, retry | -| 254 | npm/pnpm/yarn: Unknown fatal error | Check package.json, clear cache | - -### Category 4: Python Errors - -| Code | Meaning | Recovery | -|------|---------|----------| -| 210 | Python: Virtualenv / uv environment missing | Recreate virtual environment | -| 211 | Python: Dependency resolution failed | Check package versions, fix conflicts | -| 212 | Python: Installation aborted (EXTERNALLY-MANAGED) | Use venv or remove marker file | - -### Category 5: Database Errors - -#### PostgreSQL - -| Code | Meaning | Recovery | -|------|---------|----------| -| 231 | Connection failed (server not running) | Start PostgreSQL service | -| 232 | Authentication failed (bad user/password) | Verify credentials | -| 233 | Database does not exist | Create database: `createdb dbname` | -| 234 | Fatal error in query / syntax error | Fix SQL syntax | - -#### MySQL / MariaDB - -| Code | Meaning | Recovery | -|------|---------|----------| -| 241 | Connection failed (server not running) | Start MySQL/MariaDB service | -| 242 | Authentication failed (bad user/password) | Reset password, verify credentials | -| 243 | Database does not exist | Create database: `CREATE DATABASE dbname;` | -| 244 | Fatal error in query / syntax error | Fix SQL syntax | - -#### MongoDB - -| Code | Meaning | Recovery | -|------|---------|----------| -| 251 | Connection failed (server not running) | Start MongoDB daemon | -| 252 | Authentication failed (bad user/password) | Verify credentials, reset if needed | -| 253 | Database not found | Create database in MongoDB shell | -| 254 | Fatal query error | Check query syntax | - -### Category 6: Proxmox Custom Codes - -| Code | Meaning | Recovery | -|------|---------|----------| -| 200 | Failed to create lock file | Check /tmp permissions | -| 203 | Missing CTID variable | CTID must be provided to script | -| 204 | Missing PCT_OSTYPE variable | OS type not detected | -| 205 | Invalid CTID (<100) | Container ID must be >= 100 | -| 206 | CTID already in use | Check `pct list`, remove conflicting container | -| 207 | Password contains special characters | Use alphanumeric only for passwords | -| 208 | Invalid configuration format | Check DNS/MAC/Network format | -| 209 | Container creation failed | Check pct create output for details | -| 210 | Cluster not quorate | Ensure cluster nodes are online | -| 211 | Timeout waiting for template lock | Wait for concurrent downloads to finish | -| 214 | Not enough storage space | Free up disk space or expand storage | -| 215 | Container created but not listed | Check /etc/pve/lxc/ for config files | -| 216 | RootFS entry missing in config | Incomplete container creation | -| 217 | Storage does not support rootdir | Use compatible storage backend | -| 218 | Template corrupted or incomplete | Re-download template | -| 220 | Unable to resolve template path | Verify template availability | -| 221 | Template not readable | Fix file permissions | -| 222 | Template download failed (3 attempts) | Check network/storage | -| 223 | Template not available after download | Storage sync issue | -| 225 | No template for OS/Version | Run `pveam available` to see options | -| 231 | LXC stack upgrade/retry failed | Update pve-container package | - ---- - -## Telemetry Functions - -### `explain_exit_code()` - -**Purpose**: Maps numeric exit codes to human-readable error descriptions. Shared between api.func and error_handler.func for consistency. - -**Signature**: -```bash -explain_exit_code() -``` - -**Parameters**: -- `$1` - Numeric exit code (0-255) - -**Returns**: Human-readable description string - -**Supported Codes**: -- 1-2, 126-128, 130, 137, 139, 143 (Shell) -- 100-101, 255 (Package managers) -- 210-212 (Python) -- 231-234 (PostgreSQL) -- 241-244 (MySQL/MariaDB) -- 243-249, 254 (Node.js/npm) -- 251-254 (MongoDB) -- 200-231 (Proxmox custom) - -**Default**: Returns "Unknown error" for unmapped codes - -**Usage Examples**: - -```bash -# Example 1: Common error -explain_exit_code 127 -# Output: "Command not found" - -# Example 2: Database error -explain_exit_code 241 -# Output: "MySQL/MariaDB: Connection failed (server not running / wrong socket)" - -# Example 3: Custom Proxmox error -explain_exit_code 206 -# Output: "Custom: CTID already in use (check 'pct list' and /etc/pve/lxc/)" - -# Example 4: Unknown code -explain_exit_code 999 -# Output: "Unknown error" -``` - ---- - -### `post_to_api()` - -**Purpose**: Sends LXC container creation statistics to Community-Scripts telemetry API. - -**Signature**: -```bash -post_to_api() -``` - -**Parameters**: None (uses global environment variables) - -**Returns**: No explicit return value (curl result stored in RESPONSE if diagnostics enabled) - -**Requirements** (Silent fail if not met): -- `curl` command available -- `DIAGNOSTICS="yes"` -- `RANDOM_UUID` is set -- Executed on Proxmox host (has access to `pveversion`) - -**Environment Variables Used**: -- `CT_TYPE` - Container type (privileged=1, unprivileged=0) -- `DISK_SIZE` - Allocated disk in GB -- `CORE_COUNT` - CPU core count -- `RAM_SIZE` - RAM allocated in MB -- `var_os` - Operating system name -- `var_version` - OS version -- `NSAPP` - Normalized application name -- `METHOD` - Installation method (default, template, etc.) -- `DIAGNOSTICS` - Enable telemetry (yes/no) -- `RANDOM_UUID` - Session UUID for tracking - -**API Endpoint**: `http://api.community-scripts.org/dev/upload` - -**Payload Structure**: -```json -{ - "ct_type": 1, // Privileged (1) or Unprivileged (0) - "type": "lxc", // Always "lxc" for containers - "disk_size": 8, // GB - "core_count": 2, // CPU cores - "ram_size": 2048, // MB - "os_type": "debian", // OS name - "os_version": "12", // OS version - "nsapp": "myapp", // Application name - "method": "default", // Setup method - "pve_version": "8.2.2", // Proxmox VE version - "status": "installing", // Current status - "random_id": "550e8400-e29b" // Session UUID (anonymous) -} -``` - -**Usage Examples**: - -```bash -# Example 1: Successful API post -CT_TYPE=1 -DISK_SIZE=20 -CORE_COUNT=4 -RAM_SIZE=4096 -var_os="ubuntu" -var_version="22.04" -NSAPP="jellyfin" -METHOD="default" -DIAGNOSTICS="yes" -RANDOM_UUID="550e8400-e29b-41d4-a716-446655440000" - -post_to_api -# Result: Statistics sent to API (silently, no output) - -# Example 2: Diagnostics disabled (opt-out) -DIAGNOSTICS="no" -post_to_api -# Result: Function returns immediately, no API call - -# Example 3: Missing curl -DIAGNOSTICS="yes" -# curl not available in PATH -post_to_api -# Result: Function returns silently (curl requirement not met) -``` - ---- - -### `post_to_api_vm()` - -**Purpose**: Sends VM creation statistics to Community-Scripts API (similar to post_to_api but for virtual machines). - -**Signature**: -```bash -post_to_api_vm() -``` - -**Parameters**: None (uses global environment variables) - -**Returns**: No explicit return value - -**Requirements**: Same as `post_to_api()` - -**Environment Variables Used**: -- `VMID` - Virtual machine ID -- `VM_TYPE` - VM type (kvm, etc.) -- `VM_CORES` - CPU core count -- `VM_RAM` - RAM in MB -- `VM_DISK` - Disk in GB -- `VM_OS` - Operating system -- `VM_VERSION` - OS version -- `VM_APP` - Application name -- `DIAGNOSTICS` - Enable telemetry -- `RANDOM_UUID` - Session UUID - -**Payload Structure** (similar to containers but for VMs): -```json -{ - "vm_id": 100, - "type": "qemu", - "vm_cores": 4, - "vm_ram": 4096, - "vm_disk": 20, - "vm_os": "ubuntu", - "vm_version": "22.04", - "vm_app": "jellyfin", - "pve_version": "8.2.2", - "status": "installing", - "random_id": "550e8400-e29b" -} -``` - ---- - -### `post_update_to_api()` - -**Purpose**: Reports installation completion status (success/failure) for container or VM. - -**Signature**: -```bash -post_update_to_api() -``` - -**Parameters**: None (uses global environment variables) - -**Returns**: No explicit return value - -**Requirements**: Same as `post_to_api()` - -**Environment Variables Used**: -- `RANDOM_UUID` - Session UUID (must match initial post_to_api call) -- `DIAGNOSTICS` - Enable telemetry -- Installation status parameters - -**Payload Structure**: -```json -{ - "status": "completed", // "completed" or "failed" - "random_id": "550e8400-e29b", // Session UUID - "exit_code": 0, // 0 for success, error code for failure - "error_explanation": "" // Error description if failed -} -``` - ---- - -## API Payload Structure - -### Container Creation Payload - -```json -{ - "ct_type": 1, // 1=Privileged, 0=Unprivileged - "type": "lxc", // Always "lxc" - "disk_size": 20, // GB - "core_count": 4, // CPU cores - "ram_size": 4096, // MB - "os_type": "debian", // Distribution name - "os_version": "12", // Version number - "nsapp": "jellyfin", // Application name - "method": "default", // Setup method - "pve_version": "8.2.2", // Proxmox VE version - "status": "installing", // Current phase - "random_id": "550e8400" // Unique session ID -} -``` - -### VM Creation Payload - -```json -{ - "vm_id": 100, - "type": "qemu", - "vm_cores": 4, - "vm_ram": 4096, - "vm_disk": 20, - "vm_os": "ubuntu", - "vm_version": "22.04", - "vm_app": "jellyfin", - "pve_version": "8.2.2", - "status": "installing", - "random_id": "550e8400" -} -``` - -### Update/Completion Payload - -```json -{ - "status": "completed", - "random_id": "550e8400", - "exit_code": 0, - "error_explanation": "" -} -``` - ---- - -## Privacy & Opt-Out - -### Privacy Policy - -Community-Scripts telemetry is designed to be **privacy-respecting**: - -- ✅ **Anonymous**: No personal data collected -- ✅ **Session-based**: UUID allows correlation without identification -- ✅ **Aggregated**: Only statistics are stored, never raw logs -- ✅ **Opt-out capable**: Single environment variable disables all telemetry -- ✅ **No tracking**: UUID cannot be linked to user identity -- ✅ **No credentials**: Passwords, SSH keys never transmitted - -### Opt-Out Methods - -**Method 1: Environment Variable (Single Script)** - -```bash -DIAGNOSTICS="no" bash ct/myapp.sh -``` - -**Method 2: Script Header (Persistent)** - -```bash -#!/bin/bash -export DIAGNOSTICS="no" -# Rest of script continues without telemetry -``` - -**Method 3: System-wide Configuration** - -```bash -# In /etc/environment or ~/.bashrc -export DIAGNOSTICS="no" -``` - -### What Data Is Collected - -| Data | Why | Shared? | -|------|-----|---------| -| Container/VM specs (cores, RAM, disk) | Understand deployment patterns | Yes, aggregated | -| OS type/version | Track popular distributions | Yes, aggregated | -| Application name | Understand popular apps | Yes, aggregated | -| Method (standard vs. custom) | Measure feature usage | Yes, aggregated | -| Success/failure status | Identify issues | Yes, aggregated | -| Exit codes | Debug failures | Yes, anonymized | - -### What Data Is NOT Collected - -- ❌ Container/VM hostnames -- ❌ IP addresses -- ❌ User credentials -- ❌ SSH keys or secrets -- ❌ Application data -- ❌ System logs -- ❌ Any personal information - ---- - -## Error Mapping - -### Mapping Strategy - -Exit codes are categorized by source: - -``` -Exit Code Range | Source | Handling -0 | Success | Not reported to API -1-2 | Shell/Script | Generic error -100-101, 255 | Package managers | APT/DPKG specific -126-128 | Command execution | Permission/not found -130, 143 | Signals | User interrupt/termination -137, 139 | Kernel | OOM/segfault -200-231 | Proxmox custom | Container creation issues -210-212 | Python | Python environment issues -231-234 | PostgreSQL | Database connection issues -241-244 | MySQL/MariaDB | Database connection issues -243-249, 254 | Node.js/npm | Runtime errors -251-254 | MongoDB | Database connection issues -``` - -### Custom Exit Code Usage - -Scripts can define custom exit codes: - -```bash -# Example: Custom validation failure -if [[ "$CTID" -lt 100 ]]; then - echo "Container ID must be >= 100" - exit 205 # Custom Proxmox code -fi -``` - ---- - -## Best Practices - -### 1. **Always Initialize RANDOM_UUID** - -```bash -# Generate unique session ID for tracking -RANDOM_UUID="$(cat /proc/sys/kernel/random/uuid)" - -# Use first 8 chars for short session ID (logs) -SESSION_ID="${RANDOM_UUID:0:8}" -BUILD_LOG="/tmp/create-lxc-${SESSION_ID}.log" -``` - -### 2. **Call post_to_api Early** - -```bash -# Call post_to_api right after container creation starts -# This tracks attempt, even if installation fails - -variables() { - RANDOM_UUID="$(cat /proc/sys/kernel/random/uuid)" - # ... other variables ... -} - -# Later, in main script: -post_to_api # Report container creation started -# ... perform installation ... -post_update_to_api # Report completion -``` - -### 3. **Handle Graceful Failures** - -```bash -# Wrap API calls to handle network issues -if command -v curl &>/dev/null; then - post_to_api || true # Don't fail if API unavailable -else - msg_warn "curl not available, telemetry skipped" -fi -``` - -### 4. **Respect User Opt-Out** - -```bash -# Check DIAGNOSTICS early and skip all API calls if disabled -if [[ "${DIAGNOSTICS}" != "yes" ]]; then - msg_info "Anonymous diagnostics disabled" - return 0 # Skip telemetry -fi -``` - -### 5. **Maintain Session Consistency** - -```bash -# Use same RANDOM_UUID throughout lifecycle -RANDOM_UUID="$(cat /proc/sys/kernel/random/uuid)" - -post_to_api # Initial report -# ... installation ... -post_update_to_api # Final report (same UUID links them) -``` - ---- - -## API Integration - -### Connecting to API - -The API endpoint is: - -``` -http://api.community-scripts.org/dev/upload -``` - -### API Response Handling - -```bash -# Capture HTTP response code -RESPONSE=$(curl -s -w "%{http_code}" -L -X POST "$API_URL" \ - -H "Content-Type: application/json" \ - -d "$JSON_PAYLOAD") || true - -# Extract status code (last 3 digits) -HTTP_CODE="${RESPONSE: -3}" - -if [[ "$HTTP_CODE" == "200" ]]; then - msg_ok "Telemetry submitted successfully" -elif [[ "$HTTP_CODE" == "429" ]]; then - msg_warn "API rate limited, skipping telemetry" -else - msg_info "Telemetry API unreachable (this is OK)" -fi -``` - -### Network Resilience - -API calls are **best-effort** and never block installation: - -```bash -# Telemetry should never cause container creation to fail -if post_to_api 2>/dev/null; then - msg_info "Diagnostics transmitted" -fi -# If API unavailable, continue anyway -``` - ---- - -## Contributing - -### Adding New Exit Codes - -1. Document in the appropriate category section -2. Update `explain_exit_code()` in both api.func and error_handler.func -3. Add recovery suggestions -4. Test mapping with scripts that use the new code - -### Testing API Integration - -```bash -# Test with mock curl (local testing) -DIAGNOSTICS="yes" -RANDOM_UUID="test-uuid-12345678" -curl -X POST http://localhost:8000/dev/upload \ - -H "Content-Type: application/json" \ - -d '{"test": "payload"}' - -# Verify payload structure -post_to_api 2>&1 | head -20 -``` - -### Telemetry Reporting Improvements - -Suggestions for improvement: - -1. Installation duration tracking -2. Package version compatibility data -3. Feature usage analytics -4. Performance metrics -5. Custom error codes - ---- - -## Notes - -- API calls are **silent by default** and never display sensitive information -- Telemetry can be **completely disabled** via `DIAGNOSTICS="no"` -- **RANDOM_UUID must be generated** before calling any post functions -- Exit code mappings are **shared** between api.func and error_handler.func for consistency -- API is **optional** - containers work perfectly without telemetry - diff --git a/docs/build.func.md b/docs/build.func.md deleted file mode 100644 index c7f8f807b..000000000 --- a/docs/build.func.md +++ /dev/null @@ -1,584 +0,0 @@ -# Build.func Wiki - -Central LXC container build and configuration orchestration engine providing the main creation workflow, 19-step advanced wizard, defaults system, variable management, and state machine for container lifecycle. - ---- - -## 📋 Table of Contents - -- [Overview](#overview) -- [Core Functions](#core-functions) -- [Variable Management](#variable-management) -- [Build Workflow](#build-workflow) -- [Advanced Settings Wizard](#advanced-settings-wizard) -- [Defaults System](#defaults-system) -- [Best Practices](#best-practices) -- [Development Mode](#development-mode) -- [Contributing](#contributing) - ---- - -## Overview - -Build.func is the **3800+ line orchestration engine** for LXC container creation: - -- ✅ 19-step interactive advanced settings wizard -- ✅ 3-tier defaults precedence system (app → user → global) -- ✅ Variable whitelisting for security -- ✅ State machine workflow management -- ✅ Container resource allocation (CPU, RAM, disk) -- ✅ Storage selection and validation -- ✅ Network configuration (bridge, MAC, VLAN, IPv6) -- ✅ Session tracking and logging -- ✅ Comprehensive pre-flight validation checks - -### Execution Flow - -``` -Script Invocation - ↓ -variables() → Initialize core variables, SESSION_ID, UUID - ↓ -build.func functions sourced - ↓ -Pre-flight checks (maxkeys, template availability) - ↓ -Create container (pct create ...) - ↓ -Network configuration - ↓ -Storage tuning - ↓ -Installation script execution - ↓ -Completion & cleanup -``` - ---- - -## Core Functions - -### `variables()` - -**Purpose**: Initializes all core variables, generates unique session ID, and captures application defaults for precedence logic. - -**Signature**: -```bash -variables() -``` - -**Parameters**: None - -**Returns**: No explicit return value (sets global variables) - -**Variables Initialized**: - -| Variable | Source | Purpose | -|----------|--------|---------| -| `NSAPP` | `APP` converted to lowercase | Normalized app name | -| `var_install` | `${NSAPP}-install` | Installation script name | -| `PVEHOST_NAME` | `hostname` | Proxmox hostname | -| `DIAGNOSTICS` | Set to "yes" | Enable telemetry | -| `METHOD` | Set to "default" | Setup method | -| `RANDOM_UUID` | `/proc/sys/kernel/random/uuid` | Session UUID | -| `SESSION_ID` | First 8 chars of UUID | Short session ID | -| `BUILD_LOG` | `/tmp/create-lxc-${SESSION_ID}.log` | Host-side log file | -| `PVEVERSION` | `pveversion` | Proxmox VE version | -| `KERNEL_VERSION` | `uname -r` | System kernel version | - -**App Default Capture** (3-tier precedence): -```bash -# Tier 1: App-declared defaults (highest priority) -APP_DEFAULT_CPU=${var_cpu:-} -APP_DEFAULT_RAM=${var_ram:-} -APP_DEFAULT_DISK=${var_disk:-} - -# Tier 2: User configuration (~/.community-scripts/defaults) -# Tier 3: Global defaults (built-in) -``` - -**Dev Mode Setup**: -```bash -# Parse dev_mode early for special behaviors -parse_dev_mode - -# If dev_mode=logs, use persistent logging location -if [[ "${DEV_MODE_LOGS}" == "true" ]]; then - mkdir -p /var/log/community-scripts - BUILD_LOG="/var/log/community-scripts/create-lxc-${SESSION_ID}-$(date +%Y%m%d_%H%M%S).log" -fi -``` - -**Usage Examples**: - -```bash -# Example 1: Initialize with default app -APP="Jellyfin" -variables -# Result: -# NSAPP="jellyfin" -# SESSION_ID="550e8400" -# BUILD_LOG="/tmp/create-lxc-550e8400.log" - -# Example 2: With dev mode -dev_mode="trace,logs" -APP="MyApp" -variables -# Result: -# Persistent logging enabled -# Bash tracing configured -# BUILD_LOG="/var/log/community-scripts/create-lxc-550e8400-20241201_103000.log" -``` - ---- - -### `maxkeys_check()` - -**Purpose**: Validates kernel keyring limits don't prevent container creation (prevents "key quota exceeded" errors). - -**Signature**: -```bash -maxkeys_check() -``` - -**Parameters**: None - -**Returns**: 0 if limits acceptable; exits with error if exceeded - -**Checks**: -- `/proc/sys/kernel/keys/maxkeys` - Maximum keys per user -- `/proc/sys/kernel/keys/maxbytes` - Maximum key bytes per user -- `/proc/key-users` - Current usage for UID 100000 (LXC user) - -**Warning Thresholds**: -- Keys: Current >= (maxkeys - 100) -- Bytes: Current >= (maxbytes - 1000) - -**Recovery Suggestions**: -```bash -# If warning triggered, suggests sysctl configuration -sysctl -w kernel.keys.maxkeys=200000 -sysctl -w kernel.keys.maxbytes=40000000 - -# Add to persistent config -echo "kernel.keys.maxkeys=200000" >> /etc/sysctl.d/98-community-scripts.conf -sysctl -p -``` - -**Usage Examples**: - -```bash -# Example 1: Healthy keyring usage -maxkeys_check -# Silent success: Usage is normal - -# Example 2: Near limit -maxkeys_check -# Warning displayed with suggested sysctl values -# Allows continuation but recommends tuning - -# Example 3: Exceeded limit -maxkeys_check -# Error: Exits with code 1 -# Suggests increasing limits before retry -``` - ---- - -## Variable Management - -### `default_var_settings()` - -**Purpose**: Loads or creates default variable settings with 3-tier precedence. - -**Signature**: -```bash -default_var_settings() -``` - -**Precedence Order**: -``` -1. App-declared defaults (var_cpu, var_ram, var_disk from script) -2. User defaults (~/.community-scripts/defaults.sh) -3. Global built-in defaults -``` - -**User Defaults Location**: -```bash -~/.community-scripts/defaults.sh -``` - -**Example User Defaults File**: -```bash -# ~/.community-scripts/defaults.sh -CORE_COUNT=4 # Override default CPU -RAM_SIZE=4096 # Override default RAM (MB) -DISK_SIZE=32 # Override default disk (GB) -BRIDGE="vmbr0" # Preferred bridge -STORAGE="local-lvm" # Preferred storage -DISABLEIPV6="no" # Network preference -VERBOSE="no" # Output preference -``` - ---- - -### `load_vars_file()` - -**Purpose**: Loads saved container variables from previous configuration. - -**Signature**: -```bash -load_vars_file() -``` - -**Parameters**: None - -**Returns**: 0 if loaded; 1 if no saved config found - -**File Location**: -```bash -~/.community-scripts/${NSAPP}.vars -``` - -**Variables Loaded**: -- All whitelist-approved variables (CORE_COUNT, RAM_SIZE, DISK_SIZE, etc.) -- Saved settings from previous container creation - -**Usage Examples**: - -```bash -# Example 1: Load previous config -if load_vars_file; then - msg_ok "Loaded previous settings for $NSAPP" -else - msg_info "No previous configuration found, using defaults" -fi - -# Example 2: Offer to use saved config -# Interactive: Prompts user to confirm previously saved values -``` - ---- - -### `maybe_offer_save_app_defaults()` - -**Purpose**: Optionally saves current configuration for reuse in future container creations. - -**Signature**: -```bash -maybe_offer_save_app_defaults() -``` - -**Parameters**: None - -**Returns**: No explicit return value (saves or skips) - -**Behavior**: -- Prompts user if they want to save current settings -- Saves to `~/.community-scripts/${NSAPP}.vars` -- User can load these settings in future runs via `load_vars_file()` -- Saves whitelisted variables only (security) - -**Variables Saved**: -- CORE_COUNT, RAM_SIZE, DISK_SIZE -- BRIDGE, STORAGE, MACADDRESS -- VLAN_TAG, DISABLEIPV6 -- PASSWORD settings -- Custom network configuration - -**Usage Examples**: - -```bash -# Example 1: After configuration -configure_container -# ... all settings done ... -maybe_offer_save_app_defaults -# Prompts: "Save these settings for future use? [y/n]" -# If yes: Saves to ~/.community-scripts/jellyfin.vars - -# Example 2: Reload in next run -# User runs script again -# Prompted: "Use saved settings from last time? [y/n]" -# If yes: Load_vars_file() populates all variables -``` - ---- - -## Build Workflow - -### `install_script()` - -**Purpose**: Orchestrates container installation workflow inside the LXC container. - -**Signature**: -```bash -install_script() -``` - -**Parameters**: None (uses global `NSAPP` variable) - -**Returns**: 0 on success; exits with error code on failure - -**Installation Steps**: -1. Copy install script into container -2. Execute via `pct exec $CTID bash /tmp/...` -3. Capture output and exit code -4. Report completion to API -5. Handle errors with cleanup - -**Error Handling**: -```bash -# If installation fails: -# - Captures exit code -# - Posts failure to API (if telemetry enabled) -# - Displays error with explanation -# - Offers debug shell (if DEV_MODE_BREAKPOINT) -# - Cleans up container (unless DEV_MODE_KEEP) -``` - ---- - -## Advanced Settings Wizard - -### `advanced_settings()` - -**Purpose**: Interactive 19-step wizard for advanced container configuration. - -**Signature**: -```bash -advanced_settings() -``` - -**Parameters**: None - -**Returns**: No explicit return value (populates variables) - -**Wizard Steps** (19 total): -1. **CPU Cores** - Allocation (1-128) -2. **RAM Size** - Allocation in MB (256-65536) -3. **Disk Size** - Allocation in GB (1-4096) -4. **Storage** - Select storage backend (local, local-lvm, etc.) -5. **Bridge** - Network bridge (vmbr0, vmbr1, etc.) -6. **MAC Address** - Custom or auto-generated -7. **VLAN Tag** - Optional VLAN configuration -8. **IPv6** - Enable/disable IPv6 -9. **Disable IPV6** - Explicit disable option -10. **DHCP** - DHCP or static IP -11. **IP Configuration** - If static: IP/mask -12. **Gateway** - Network gateway -13. **DNS** - DNS server configuration -14. **Hostname** - Container hostname -15. **Root Password** - Set or leave empty (auto-login) -16. **SSH Access** - Enable root SSH -17. **Features** - FUSE, Nesting, keyctl, etc. -18. **Start on Boot** - Autostart configuration -19. **Privileged Mode** - Privileged or unprivileged container - -**User Input Methods**: -- Whiptail dialogs (graphical) -- Command-line prompts (fallback) -- Validation of all inputs -- Confirmation summary before creation - -**Usage Examples**: - -```bash -# Example 1: Run wizard -advanced_settings -# User prompted for each of 19 settings -# Responses stored in variables - -# Example 2: Scripted (skip prompts) -CORE_COUNT=4 -RAM_SIZE=4096 -DISK_SIZE=32 -# ... set all 19 variables ... -# advanced_settings() skips prompts since variables already set -``` - ---- - -## Defaults System - -### 3-Tier Precedence Logic - -**Tier 1 (Highest Priority): App-Declared Defaults** -```bash -# In app script header (before default.vars sourced): -var_cpu=4 -var_ram=2048 -var_disk=20 - -# If user has higher value in tier 2/3, app value takes precedence -``` - -**Tier 2 (Medium Priority): User Defaults** -```bash -# In ~/.community-scripts/defaults.sh: -CORE_COUNT=6 -RAM_SIZE=4096 -DISK_SIZE=32 - -# Can be overridden by app defaults (tier 1) -``` - -**Tier 3 (Lowest Priority): Global Built-in Defaults** -```bash -# Built into build.func: -CORE_COUNT=2 (default) -RAM_SIZE=2048 (default, in MB) -DISK_SIZE=8 (default, in GB) -``` - -**Resolution Algorithm**: -```bash -# For CPU cores (example): -if [ -n "$APP_DEFAULT_CPU" ]; then - CORE_COUNT=$APP_DEFAULT_CPU # Tier 1 wins -elif [ -n "$USER_DEFAULT_CPU" ]; then - CORE_COUNT=$USER_DEFAULT_CPU # Tier 2 -else - CORE_COUNT=2 # Tier 3 (global) -fi -``` - ---- - -## Best Practices - -### 1. **Always Call variables() First** - -```bash -#!/bin/bash -source <(curl -fsSL .../build.func) -load_functions -catch_errors - -# Must be first real function call -variables - -# Then safe to use SESSION_ID, BUILD_LOG, etc. -msg_info "Building container (Session: $SESSION_ID)" -``` - -### 2. **Declare App Defaults Before Sourcing build.func** - -```bash -#!/bin/bash -# Declare app defaults BEFORE sourcing build.func -var_cpu=4 -var_ram=4096 -var_disk=20 - -source <(curl -fsSL .../build.func) -variables # These defaults are captured - -# Now var_cpu, var_ram, var_disk are in APP_DEFAULT_* -``` - -### 3. **Use Variable Whitelisting** - -```bash -# Only these variables are allowed to be saved/loaded: -WHITELIST="CORE_COUNT RAM_SIZE DISK_SIZE BRIDGE STORAGE MACADDRESS VLAN_TAG DISABLEIPV6" - -# Sensitive variables are NEVER saved: -# PASSWORD, SSH keys, API tokens, etc. -``` - -### 4. **Check Pre-flight Conditions** - -```bash -variables -maxkeys_check # Validate kernel limits -pve_check # Validate PVE version -arch_check # Validate architecture - -# Only proceed after all checks pass -msg_ok "Pre-flight checks passed" -``` - -### 5. **Track Sessions** - -```bash -# Use SESSION_ID in all logs -BUILD_LOG="/tmp/create-lxc-${SESSION_ID}.log" - -# Keep logs for troubleshooting -# Can be reviewed later: tail -50 /tmp/create-lxc-550e8400.log -``` - ---- - -## Development Mode - -### Dev Mode Variables - -Set via environment or in script: - -```bash -dev_mode="trace,keep,breakpoint" -parse_dev_mode - -# Enables: -# - DEV_MODE_TRACE=true (bash -x) -# - DEV_MODE_KEEP=true (never delete container) -# - DEV_MODE_BREAKPOINT=true (shell on error) -``` - -### Debug Container Creation - -```bash -# Run with all debugging enabled -dev_mode="trace,keep,logs" bash ct/jellyfin.sh - -# Then review logs: -tail -200 /var/log/community-scripts/create-lxc-*.log - -# Container stays running (DEV_MODE_KEEP) -# Allows ssh inspection: ssh root@ -``` - ---- - -## Contributing - -### Adding New Wizard Steps - -1. Add step number and variable to documentation -2. Add whiptail prompt in `advanced_settings()` -3. Add validation logic -4. Add to whitelist if user should save it -5. Update documentation with examples - -### Extending Defaults System - -To add new tier or change precedence: - -1. Update 3-tier logic section -2. Modify resolution algorithm -3. Document new precedence order -4. Update whitelist accordingly - -### Testing Build Workflow - -```bash -# Test with dry-run mode -dev_mode="dryrun" bash ct/myapp.sh -# Shows all commands without executing - -# Test with keep mode -dev_mode="keep" bash ct/myapp.sh -# Container stays if fails, allows inspection -``` - ---- - -## Notes - -- Build.func is **large and complex** (3800+ lines) - handles most container creation logic -- Variables are **passed to container** via pct set/environment -- Session ID **enables request tracking** across distributed logs -- Defaults system is **flexible** (3-tier precedence) -- Pre-flight checks **prevent many common errors** - diff --git a/docs/cloud-init.func.md b/docs/cloud-init.func.md deleted file mode 100644 index 2bb1ae41d..000000000 --- a/docs/cloud-init.func.md +++ /dev/null @@ -1,571 +0,0 @@ -# Cloud-Init.func Wiki - -VM cloud-init configuration and first-boot setup module for Proxmox VEs, providing automatic system initialization, network configuration, user account setup, and SSH key management for virtual machines. - ---- - -## 📋 Table of Contents - -- [Overview](#overview) -- [Cloud-Init Fundamentals](#cloud-init-fundamentals) -- [Main Configuration Functions](#main-configuration-functions) -- [Interactive Configuration](#interactive-configuration) -- [Configuration Parameters](#configuration-parameters) -- [Data Formats](#data-formats) -- [Best Practices](#best-practices) -- [Troubleshooting](#troubleshooting) -- [Contributing](#contributing) - ---- - -## Overview - -Cloud-init.func provides **VM first-boot automation** infrastructure: - -- ✅ Cloud-init drive creation (IDE2 or SCSI fallback) -- ✅ User account and password configuration -- ✅ SSH public key injection -- ✅ Network configuration (DHCP or static IP) -- ✅ DNS and search domain setup -- ✅ Interactive whiptail-based configuration -- ✅ Credential file generation and display -- ✅ Support for Debian nocloud and Ubuntu cloud-init -- ✅ System package upgrade on first boot - -### Integration Pattern - -```bash -# In Proxmox VM creation scripts -source <(curl -fsSL .../cloud-init.func) - -# Basic setup: -setup_cloud_init "$VMID" "$STORAGE" "$HOSTNAME" "yes" - -# Interactive setup: -configure_cloud_init_interactive "root" -setup_cloud_init "$VMID" "$STORAGE" "$HOSTNAME" "$CLOUDINIT_ENABLE" -``` - -### First-Boot Sequence - -``` -VM Power On - ↓ -Cloud-init boot phase - ↓ -Read cloud-init config - ↓ -Create/modify user account - ↓ -Install SSH keys - ↓ -Configure network - ↓ -Set DNS/search domain - ↓ -Upgrade packages (if configured) - ↓ -Boot completion -``` - ---- - -## Cloud-Init Fundamentals - -### What is Cloud-Init? - -Cloud-init is a system that runs on the first boot of a VM/Instance to: -- Create user accounts -- Set passwords -- Configure networking -- Install SSH keys -- Run custom scripts -- Manage system configuration - -### Proxmox Cloud-Init Integration - -Proxmox VE supports cloud-init natively via: -- **Cloud-init drive**: IDE2 or SCSI disk with cloud-init data -- **QEMU parameters**: User, password, SSH keys, IP configuration -- **First-boot services**: systemd services that execute on first boot - -### Nocloud Data Source - -Proxmox uses the **nocloud** data source (no internet required): -- Configuration stored on local cloud-init drive -- No external network call needed -- Works in isolated networks -- Suitable for private infrastructure - ---- - -## Main Configuration Functions - -### `setup_cloud_init()` - -**Purpose**: Configures Cloud-init for automatic VM first-boot setup. - -**Signature**: -```bash -setup_cloud_init() -``` - -**Parameters**: -- `$1` - VMID (required, e.g., 100) -- `$2` - Storage name (required, e.g., local, local-lvm) -- `$3` - Hostname (optional, default: vm-${VMID}) -- `$4` - Enable Cloud-Init (yes/no, default: no) -- `$5` - User (optional, default: root) -- `$6` - Network mode (dhcp/static, default: dhcp) -- `$7` - Static IP (optional, CIDR format: 192.168.1.100/24) -- `$8` - Gateway (optional) -- `$9` - Nameservers (optional, space-separated, default: 1.1.1.1 8.8.8.8) - -**Returns**: 0 on success, 1 on failure; exits if not enabled - -**Behavior**: -```bash -# If enable="no": -# Returns immediately (skips all configuration) - -# If enable="yes": -# 1. Create cloud-init drive (IDE2, fallback to SCSI1) -# 2. Set user account -# 3. Generate random password -# 4. Configure network -# 5. Set DNS servers -# 6. Add SSH keys (if available) -# 7. Save credentials to file -# 8. Export variables for calling script -``` - -**Operations**: - -| Operation | Command | Purpose | -|-----------|---------|---------| -| Create drive | `qm set $vmid --ide2 $storage:cloudinit` | Cloud-init data disk | -| Set user | `qm set $vmid --ciuser $ciuser` | Initial user | -| Set password | `qm set $vmid --cipassword $cipassword` | Auto-generated | -| SSH keys | `qm set $vmid --sshkeys $SSH_KEYS_FILE` | Pre-injected | -| DHCP network | `qm set $vmid --ipconfig0 ip=dhcp` | Dynamic IP | -| Static network | `qm set $vmid --ipconfig0 ip=192.168.1.100/24,gw=192.168.1.1` | Fixed IP | -| DNS | `qm set $vmid --nameserver $servers` | 1.1.1.1 8.8.8.8 | -| Search domain | `qm set $vmid --searchdomain local` | Local domain | - -**Environment Variables Set**: -- `CLOUDINIT_USER` - Username configured -- `CLOUDINIT_PASSWORD` - Generated password (in memory only) -- `CLOUDINIT_CRED_FILE` - Path to credentials file - -**Usage Examples**: - -```bash -# Example 1: Basic DHCP setup -VMID=100 -STORAGE="local-lvm" -setup_cloud_init "$VMID" "$STORAGE" "myvm" "yes" -# Result: VM configured with DHCP, random password, root user - -# Example 2: Static IP configuration -setup_cloud_init "$VMID" "$STORAGE" "myvm" "yes" "root" \ - "static" "192.168.1.100/24" "192.168.1.1" "1.1.1.1 8.8.8.8" -# Result: VM configured with static IP, specific DNS - -# Example 3: Disabled (no cloud-init) -setup_cloud_init "$VMID" "$STORAGE" "myvm" "no" -# Result: Function returns immediately, no configuration -``` - ---- - -### `configure_cloud_init_interactive()` - -**Purpose**: Interactive whiptail-based configuration prompts for user preferences. - -**Signature**: -```bash -configure_cloud_init_interactive() -``` - -**Parameters**: -- `$1` - Default user (optional, default: root) - -**Returns**: 0 on success, 1 if whiptail unavailable; exports configuration variables - -**Environment Variables Exported**: -- `CLOUDINIT_ENABLE` - Enable (yes/no) -- `CLOUDINIT_USER` - Username -- `CLOUDINIT_NETWORK_MODE` - dhcp or static -- `CLOUDINIT_IP` - Static IP (if static mode) -- `CLOUDINIT_GW` - Gateway (if static mode) -- `CLOUDINIT_DNS` - DNS servers (space-separated) - -**User Prompts** (5 questions): -1. **Enable Cloud-Init?** (yes/no) -2. **Username?** (default: root) -3. **Network Mode?** (DHCP or static) -4. **Static IP?** (if static, CIDR format) -5. **Gateway IP?** (if static) -6. **DNS Servers?** (default: 1.1.1.1 8.8.8.8) - -**Fallback Behavior**: -- If whiptail unavailable: Shows warning and returns 1 -- Auto-defaults to DHCP if error occurs -- Non-interactive: Can be skipped in scripts - -**Implementation Pattern**: -```bash -configure_cloud_init_interactive() { - local default_user="${1:-root}" - - # Check whiptail availability - if ! command -v whiptail >/dev/null 2>&1; then - echo "Warning: whiptail not available" - export CLOUDINIT_ENABLE="no" - return 1 - fi - - # Ask enable - if ! (whiptail --backtitle "Proxmox VE Helper Scripts" --title "CLOUD-INIT" \ - --yesno "Enable Cloud-Init for VM configuration?" 16 68); then - export CLOUDINIT_ENABLE="no" - return 0 - fi - - export CLOUDINIT_ENABLE="yes" - - # Username - CLOUDINIT_USER=$(whiptail --backtitle "Proxmox VE Helper Scripts" --inputbox \ - "Cloud-Init Username" 8 58 "$default_user" --title "USERNAME" 3>&1 1>&2 2>&3) - export CLOUDINIT_USER="${CLOUDINIT_USER:-$default_user}" - - # Network mode - if (whiptail --backtitle "Proxmox VE Helper Scripts" --title "NETWORK MODE" \ - --yesno "Use DHCP for network configuration?" 10 58); then - export CLOUDINIT_NETWORK_MODE="dhcp" - else - export CLOUDINIT_NETWORK_MODE="static" - # ... prompt for static IP and gateway ... - fi - - # DNS servers - CLOUDINIT_DNS=$(whiptail --backtitle "Proxmox VE Helper Scripts" --inputbox \ - "DNS Servers (space-separated)" 8 58 "1.1.1.1 8.8.8.8" --title "DNS" 3>&1 1>&2 2>&3) - export CLOUDINIT_DNS -} -``` - -**Usage Examples**: - -```bash -# Example 1: Interactive configuration -configure_cloud_init_interactive "root" -# Prompts user for all settings interactively -# Exports variables for use in setup_cloud_init() - -# Example 2: With custom default user -configure_cloud_init_interactive "debian" -# Suggests "debian" as default username - -# Example 3: In script workflow -configure_cloud_init_interactive "$DEFAULT_USER" -setup_cloud_init "$VMID" "$STORAGE" "$HOSTNAME" "$CLOUDINIT_ENABLE" "$CLOUDINIT_USER" -# User configures interactively, then script sets up VM -``` - ---- - -## Configuration Parameters - -### VMID (Virtual Machine ID) - -- **Type**: Integer (100-2147483647) -- **Required**: Yes -- **Example**: `100` -- **Validation**: Must be unique, >= 100 on Proxmox - -### Storage - -- **Type**: String (storage backend name) -- **Required**: Yes -- **Examples**: `local`, `local-lvm`, `ceph-rbd` -- **Validation**: Must exist in Proxmox -- **Cloud-init Drive**: Placed on this storage - -### Hostname - -- **Type**: String (valid hostname) -- **Required**: No (defaults to vm-${VMID}) -- **Example**: `myvm`, `web-server-01` -- **Format**: Lowercase, alphanumeric, hyphens allowed - -### User - -- **Type**: String (username) -- **Required**: No (defaults: root) -- **Example**: `root`, `ubuntu`, `debian` -- **Cloud-init**: User account created on first boot - -### Network Mode - -- **Type**: Enum (dhcp, static) -- **Default**: dhcp -- **Options**: - - `dhcp` - Dynamic IP from DHCP server - - `static` - Manual IP configuration - -### Static IP - -- **Format**: CIDR notation (192.168.1.100/24) -- **Example**: `192.168.1.50/24`, `10.0.0.5/8` -- **Validation**: Valid IP and netmask -- **Required**: If network mode = static - -### Gateway - -- **Format**: IP address (192.168.1.1) -- **Example**: `192.168.1.1`, `10.0.0.1` -- **Validation**: Valid IP -- **Required**: If network mode = static - -### Nameservers - -- **Format**: Space-separated IPs -- **Default**: `1.1.1.1 8.8.8.8` -- **Example**: `1.1.1.1 8.8.8.8 9.9.9.9` - -### DNS Search Domain - -- **Type**: String -- **Default**: `local` -- **Example**: `example.com`, `internal.corp` - ---- - -## Data Formats - -### Cloud-Init Credentials File - -Generated at: `/tmp/${hostname}-${vmid}-cloud-init-credentials.txt` - -**Format**: -``` -======================================== -Cloud-Init Credentials -======================================== -VM ID: 100 -Hostname: myvm -Created: Tue Dec 01 10:30:00 UTC 2024 - -Username: root -Password: s7k9mL2pQ8wX - -Network: dhcp -DNS: 1.1.1.1 8.8.8.8 - -======================================== -SSH Access (if keys configured): -ssh root@ - -Proxmox UI Configuration: -VM 100 > Cloud-Init > Edit -- User, Password, SSH Keys -- Network (IP Config) -- DNS, Search Domain -======================================== -``` - -### Proxmox Cloud-Init Config - -Stored in: `/etc/pve/nodes//qemu-server/.conf` - -**Relevant Settings**: -``` -ide2: local-lvm:vm-100-cloudinit,media=cdrom -ciuser: root -cipassword: (encrypted) -ipconfig0: ip=dhcp -nameserver: 1.1.1.1 8.8.8.8 -searchdomain: local -``` - -### Network Configuration Examples - -**DHCP**: -```bash -qm set 100 --ipconfig0 "ip=dhcp" -``` - -**Static IPv4**: -```bash -qm set 100 --ipconfig0 "ip=192.168.1.100/24,gw=192.168.1.1" -``` - -**Static IPv6**: -```bash -qm set 100 --ipconfig0 "ip6=2001:db8::100/64,gw6=2001:db8::1" -``` - -**Dual Stack (IPv4 + IPv6)**: -```bash -qm set 100 --ipconfig0 "ip=192.168.1.100/24,gw=192.168.1.1,ip6=2001:db8::100/64,gw6=2001:db8::1" -``` - ---- - -## Best Practices - -### 1. **Always Configure SSH Keys** - -```bash -# Ensure SSH keys available before cloud-init setup -CLOUDINIT_SSH_KEYS="/root/.ssh/authorized_keys" - -if [ ! -f "$CLOUDINIT_SSH_KEYS" ]; then - mkdir -p /root/.ssh - # Generate or import SSH keys -fi - -setup_cloud_init "$VMID" "$STORAGE" "$HOSTNAME" "yes" -``` - -### 2. **Save Credentials Securely** - -```bash -# After setup_cloud_init(): -# Credentials file generated at $CLOUDINIT_CRED_FILE - -# Copy to secure location: -cp "$CLOUDINIT_CRED_FILE" "/root/vm-credentials/" -chmod 600 "/root/vm-credentials/$(basename $CLOUDINIT_CRED_FILE)" - -# Or display to user: -cat "$CLOUDINIT_CRED_FILE" -``` - -### 3. **Use Static IPs for Production** - -```bash -# DHCP - suitable for dev/test -setup_cloud_init "$VMID" "$STORAGE" "$HOSTNAME" "yes" "root" "dhcp" - -# Static - suitable for production -setup_cloud_init "$VMID" "$STORAGE" "$HOSTNAME" "yes" "root" \ - "static" "192.168.1.100/24" "192.168.1.1" -``` - -### 4. **Validate Network Configuration** - -```bash -# Before setting up cloud-init, ensure: -# - Gateway IP is reachable -# - IP address not in use -# - DNS servers are accessible - -ping -c 1 "$GATEWAY" || msg_error "Gateway unreachable" -``` - -### 5. **Test First Boot** - -```bash -# After cloud-init setup: -qm start "$VMID" - -# Wait for boot -sleep 10 - -# Check cloud-init status -qm exec "$VMID" cloud-init status - -# Verify network configuration -qm exec "$VMID" hostname -I -``` - ---- - -## Troubleshooting - -### Cloud-Init Not Applying - -```bash -# Inside VM: -cloud-init status # Show cloud-init status -cloud-init analyze # Analyze cloud-init boot -cloud-init query # Query cloud-init datasource - -# Check logs: -tail -100 /var/log/cloud-init-output.log -tail -100 /var/log/cloud-init.log -``` - -### Network Not Configured - -```bash -# Verify cloud-init config in Proxmox: -cat /etc/pve/nodes/$(hostname)/qemu-server/100.conf - -# Check cloud-init drive: -qm config 100 | grep ide2 - -# In VM, verify cloud-init wrote config: -cat /etc/netplan/99-cloudinit.yaml -``` - -### SSH Keys Not Installed - -```bash -# Verify SSH keys set in Proxmox: -qm config 100 | grep sshkeys - -# In VM, check SSH directory: -ls -la /root/.ssh/ -cat /root/.ssh/authorized_keys -``` - -### Password Not Set - -```bash -# Regenerate cloud-init drive: -qm set 100 --delete ide2 # Remove cloud-init drive -qm set 100 --ide2 local-lvm:vm-100-cloudinit,media=cdrom # Re-create - -# Set password again: -qm set 100 --cipassword "newpassword" -``` - ---- - -## Contributing - -### Adding New Configuration Options - -1. Add parameter to `setup_cloud_init()` function signature -2. Add validation for parameter -3. Add `qm set` command to apply configuration -4. Update documentation with examples -5. Test on actual Proxmox VE - -### Enhancing Interactive Configuration - -1. Add new whiptail dialog to `configure_cloud_init_interactive()` -2. Export variable for use in setup -3. Add validation logic -4. Test with various input scenarios - -### Supporting New Data Sources - -Beyond nocloud, could support: -- ConfigDrive (cloud-init standard) -- ESXi (if supporting vSphere) -- Hyper-V (if supporting Windows) - ---- - -## Notes - -- Cloud-init requires **QEMU guest agent** for optimal functionality -- Network configuration applied **on first boot only** -- Credentials file contains **sensitive information** - keep secure -- SSH keys are **persisted** and not displayed in credentials file -- Cloud-init is **optional** - VMs work without it - diff --git a/docs/core.func.md b/docs/core.func.md deleted file mode 100644 index ecb70a0b1..000000000 --- a/docs/core.func.md +++ /dev/null @@ -1,918 +0,0 @@ -# Core.func Wiki - -The foundational utility library providing colors, formatting, validation checks, message output, and execution helpers used across all Community-Scripts ecosystem projects. - ---- - -## 📋 Table of Contents - -- [Overview](#overview) -- [Initialization Functions](#initialization-functions) -- [Color & Formatting](#color--formatting) -- [Validation Checks](#validation-checks) -- [Message Output Functions](#message-output-functions) -- [Execution Helpers](#execution-helpers) -- [Development Mode](#development-mode) -- [Best Practices](#best-practices) -- [Contributing](#contributing) - ---- - -## Overview - -Core.func provides essential utilities for consistent behavior across all Community-Scripts: - -- ✅ ANSI color codes for styled terminal output -- ✅ Standard icons and formatting for UI consistency -- ✅ System validation checks (root, PVE version, architecture) -- ✅ Colored message functions (info, ok, error, warn) -- ✅ Silent command execution with log redirection -- ✅ Spinner animations for long-running operations -- ✅ Development mode support (trace, breakpoint, dry-run) -- ✅ Guard clauses to prevent reloading - -### Integration Pattern - -```bash -#!/bin/bash -source <(curl -fsSL https://git.community-scripts.org/.../core.func) -load_functions # Initialize all color/formatting/defaults -root_check # Validate prerequisites -pve_check # Check Proxmox VE version -``` - ---- - -## Initialization Functions - -### `load_functions()` - -**Purpose**: Initializes all core utility function groups. Must be called once before using any core utilities. - -**Signature**: -```bash -load_functions() -``` - -**Parameters**: None - -**Returns**: No explicit return value (sets global variables) - -**Guard Mechanism**: -```bash -[[ -n "${__FUNCTIONS_LOADED:-}" ]] && return # Prevent re-loading -_CORE_FUNC_LOADED=1 # Mark as loaded -``` - -**Initializes** (in order): -1. `color()` - ANSI color codes -2. `formatting()` - Text formatting helpers -3. `icons()` - Emoji/symbol constants -4. `default_vars()` - Retry and timeout settings -5. `set_std_mode()` - Verbose/silent mode - -**Usage Examples**: - -```bash -# Example 1: Typical initialization -source <(curl -fsSL .../core.func) -load_functions # Safe to call multiple times -msg_info "Starting setup" # Now colors are available - -# Example 2: Safe multiple sourcing -source <(curl -fsSL .../core.func) -load_functions -source <(curl -fsSL .../tools.func) -load_functions # Silently returns (already loaded) -``` - ---- - -### `color()` - -**Purpose**: Defines ANSI escape codes for colored terminal output. - -**Signature**: -```bash -color() -``` - -**Color Variables Defined**: - -| Variable | Code | Effect | Use Case | -|----------|------|--------|----------| -| `YW` | `\033[33m` | Yellow | Warnings, secondary info | -| `YWB` | `\e[93m` | Bright Yellow | Emphasis, bright warnings | -| `BL` | `\033[36m` | Cyan/Blue | Hostnames, IPs, values | -| `RD` | `\033[01;31m` | Bright Red | Errors, critical alerts | -| `GN` | `\033[1;92m` | Bright Green | Success, OK status | -| `DGN` | `\033[32m` | Dark Green | Background, secondary success | -| `BGN` | `\033[4;92m` | Green with underline | Highlights | -| `CL` | `\033[m` | Clear | Reset to default | - -**Usage Examples**: - -```bash -# Example 1: Colored output -color -echo -e "${RD}Error: File not found${CL}" -# Output: "Error: File not found" (in red) - -# Example 2: Multiple colors on one line -echo -e "${YW}Warning:${CL} ${BL}$HOSTNAME${CL} is running low on ${RD}RAM${CL}" - -# Example 3: In functions -print_status() { - echo -e "${GN}✓${CL} Operation completed" -} -``` - ---- - -### `formatting()` - -**Purpose**: Defines formatting helpers for terminal output. - -**Signature**: -```bash -formatting() -``` - -**Formatting Variables Defined**: - -| Variable | Escape Code | Purpose | -|----------|------------|---------| -| `BFR` | `\r\033[K` | Backspace and clear line | -| `BOLD` | `\033[1m` | Bold text | -| `HOLD` | ` ` (space) | Spacing | -| `TAB` | ` ` (2 spaces) | Indentation | -| `TAB3` | ` ` (6 spaces) | Larger indentation | - -**Usage Examples**: - -```bash -# Example 1: Overwrite previous line (progress) -for i in {1..10}; do - echo -en "${BFR}Progress: $i/10" - sleep 1 -done - -# Example 2: Bold emphasis -echo -e "${BOLD}Important:${CL} This requires attention" - -# Example 3: Structured indentation -echo "Main Item:" -echo -e "${TAB}Sub-item 1" -echo -e "${TAB}Sub-item 2" -``` - ---- - -### `icons()` - -**Purpose**: Defines symbolic emoji and icon constants used for UI consistency. - -**Signature**: -```bash -icons() -``` - -**Icon Variables Defined**: - -| Variable | Icon | Use | -|----------|------|-----| -| `CM` | ✔ī¸ | Success/checkmark | -| `CROSS` | ✖ī¸ | Error/cross | -| `INFO` | 💡 | Information | -| `OS` | đŸ–Ĩī¸ | Operating system | -| `CONTAINERTYPE` | đŸ“Ļ | Container | -| `DISKSIZE` | 💾 | Disk/storage | -| `CPUCORE` | 🧠 | CPU | -| `RAMSIZE` | 🛠ī¸ | RAM | -| `HOSTNAME` | 🏠 | Hostname | -| `BRIDGE` | 🌉 | Network bridge | -| `NETWORK` | 📡 | Network | -| `GATEWAY` | 🌐 | Gateway | -| `CREATING` | 🚀 | Creating | -| `ADVANCED` | 🧩 | Advanced/options | -| `HOURGLASS` | âŗ | Wait/timer | - ---- - -### `default_vars()` - -**Purpose**: Sets default retry and timing variables for system operations. - -**Signature**: -```bash -default_vars() -``` - -**Variables Set**: -- `RETRY_NUM=10` - Maximum retry attempts -- `RETRY_EVERY=3` - Seconds between retries -- `i=$RETRY_NUM` - Counter for retry loops - -**Usage Examples**: - -```bash -# Example 1: Retry loop with defaults -RETRY_NUM=10 -RETRY_EVERY=3 -i=$RETRY_NUM -while [ $i -gt 0 ]; do - if check_network; then - break - fi - echo "Retrying... ($i attempts left)" - sleep $RETRY_EVERY - i=$((i - 1)) -done - -# Example 2: Custom retry values -RETRY_NUM=5 # Try 5 times -RETRY_EVERY=2 # Wait 2 seconds between attempts -``` - ---- - -### `set_std_mode()` - -**Purpose**: Configures output verbosity and optional debug tracing based on environment variables. - -**Signature**: -```bash -set_std_mode() -``` - -**Behavior**: -- If `VERBOSE=yes`: `STD=""` (show all output) -- If `VERBOSE=no`: `STD="silent"` (suppress output via silent() wrapper) -- If `DEV_MODE_TRACE=true`: Enable `set -x` bash tracing - -**Variables Set**: -- `STD` - Command prefix for optional output suppression - -**Usage Examples**: - -```bash -# Example 1: Verbose output -VERBOSE="yes" -set_std_mode -$STD apt-get update # Shows all apt output -# Output: All package manager messages displayed - -# Example 2: Silent output -VERBOSE="no" -set_std_mode -$STD apt-get update # Silently updates, logs to file -# Output: Only progress bar or errors shown - -# Example 3: Debug tracing -DEV_MODE_TRACE="true" -set_std_mode -# bash shows every command before executing: +(script.sh:123): function_name(): cmd -``` - ---- - -### `parse_dev_mode()` - -**Purpose**: Parses comma-separated dev_mode string to enable development features. - -**Signature**: -```bash -parse_dev_mode() -``` - -**Parameters**: None (uses `$dev_mode` environment variable) - -**Supported Flags**: -- `motd` - Setup SSH/MOTD before installation -- `keep` - Never delete container on failure -- `trace` - Enable bash set -x tracing -- `pause` - Pause after each msg_info step -- `breakpoint` - Open shell on error instead of cleanup -- `logs` - Persist logs to /var/log/community-scripts/ -- `dryrun` - Show commands without executing - -**Environment Variables Set**: -- `DEV_MODE_MOTD=false|true` -- `DEV_MODE_KEEP=false|true` -- `DEV_MODE_TRACE=false|true` -- `DEV_MODE_PAUSE=false|true` -- `DEV_MODE_BREAKPOINT=false|true` -- `DEV_MODE_LOGS=false|true` -- `DEV_MODE_DRYRUN=false|true` - -**Usage Examples**: - -```bash -# Example 1: Enable debugging -dev_mode="trace,logs" -parse_dev_mode -# Enables bash tracing and persistent logging - -# Example 2: Keep container on error -dev_mode="keep,breakpoint" -parse_dev_mode -# Container never deleted on error, opens shell at breakpoint - -# Example 3: Multiple modes -dev_mode="motd,keep,trace,pause" -parse_dev_mode -# All four development modes active -``` - ---- - -## Color & Formatting - -### Color Codes - -**Standard Colors**: -```bash -${YW} # Yellow (warnings) -${RD} # Red (errors) -${GN} # Green (success) -${BL} # Blue/Cyan (values) -${CL} # Clear (reset) -``` - -**Example Combinations**: -```bash -echo -e "${YW}Warning:${CL} ${RD}Critical${CL} at ${BL}$(date)${CL}" -# Output: "Warning: Critical at 2024-12-01 10:30:00" (colored appropriately) -``` - ---- - -## Validation Checks - -### `shell_check()` - -**Purpose**: Verifies script is running under Bash (not sh, dash, etc.). - -**Signature**: -```bash -shell_check() -``` - -**Parameters**: None - -**Returns**: 0 if Bash; exits with error if not - -**Behavior**: -- Checks `ps -p $$ -o comm=` (current shell command) -- Exits with error message if not "bash" -- Clears screen for better error visibility - -**Usage Examples**: - -```bash -#!/bin/bash -source <(curl -fsSL .../core.func) -load_functions -shell_check # Exits if run with: sh script.sh or dash script.sh - -# If run correctly: bash script.sh - continues -# If run with sh: Displays error and exits -``` - ---- - -### `root_check()` - -**Purpose**: Verifies script is running with root privileges directly (not via sudo). - -**Signature**: -```bash -root_check() -``` - -**Parameters**: None - -**Returns**: 0 if root directly; exits with error if not - -**Checks**: -- `id -u` must be 0 (root) -- Parent process (`$PPID`) must not be "sudo" - -**Why**: Some scripts require genuine root context, not sudo-elevated user shell. - -**Usage Examples**: - -```bash -#!/bin/bash -# Must run as root directly, not via sudo -source <(curl -fsSL .../core.func) -load_functions -root_check # Will fail if: sudo bash script.sh - -# Correct: bash script.sh (from root shell on Proxmox) -``` - ---- - -### `pve_check()` - -**Purpose**: Validates Proxmox VE version compatibility. - -**Signature**: -```bash -pve_check() -``` - -**Parameters**: None - -**Returns**: 0 if supported version; exits with error if not - -**Supported Versions**: -- PVE 8.0 - 8.9 -- PVE 9.0 - 9.1 - -**Version Detection**: -```bash -PVE_VER=$(pveversion | awk -F'/' '{print $2}' | awk -F'-' '{print $1}') -# Example: "pveversion" → "pve-manager/8.2.2/550e8400-e29b" -# Extracted: "8.2.2" → "8" -``` - -**Usage Examples**: - -```bash -# Example 1: On supported PVE 8.2 -bash ct/app.sh -# Passes: 8.2 is in range 8.0-8.9 - -# Example 2: On unsupported PVE 7.4 -bash ct/app.sh -# Error: "This version of Proxmox VE is not supported" - -# Example 3: On future unsupported PVE 10.0 -bash ct/app.sh -# Error: "This version of Proxmox VE is not yet supported" -``` - ---- - -### `arch_check()` - -**Purpose**: Validates system architecture is amd64/x86_64 (not ARM/PiMox). - -**Signature**: -```bash -arch_check() -``` - -**Parameters**: None - -**Returns**: 0 if amd64; exits with error if not - -**Behavior**: -- Checks `dpkg --print-architecture` -- Exits if not "amd64" -- Provides link to ARM64-compatible scripts - -**Usage Examples**: - -```bash -# Example 1: On x86_64 server -arch_check -# Passes silently - -# Example 2: On PiMox (ARM64) -arch_check -# Error: "This script will not work with PiMox!" -# Suggests: https://github.com/asylumexp/Proxmox -``` - ---- - -### `ssh_check()` - -**Purpose**: Detects SSH connection and warns if connecting remotely (recommends Proxmox console). - -**Signature**: -```bash -ssh_check() -``` - -**Parameters**: None - -**Returns**: No explicit return value (warning only, does not exit) - -**Behavior**: -- Checks `$SSH_CLIENT` environment variable -- Analyzes client IP to determine if local or remote -- Skips warning for local/same-subnet connections -- Warns for external connections - -**Usage Examples**: - -```bash -# Example 1: Local SSH (Proxmox WebUI console) -ssh_check -# No warning: Client is localhost (127.0.0.1) - -# Example 2: External SSH over Internet -ssh -l root 1.2.3.4 "bash script.sh" -# Warning: "Running via external SSH (client: 1.2.3.4)" -# Recommends Proxmox Shell (Console) instead -``` - ---- - -## Message Output Functions - -### `msg_info()` - -**Purpose**: Displays informational message with icon and yellow color. - -**Signature**: -```bash -msg_info() -``` - -**Parameters**: -- `$@` - Message text (concatenated with spaces) - -**Format**: `[ℹī¸] Message text` (yellow) - -**Usage Examples**: - -```bash -msg_info "Starting container setup" -# Output: ℹī¸ Starting container setup - -msg_info "Updating OS packages" "for debian:12" -# Output: ℹī¸ Updating OS packages for debian:12 -``` - ---- - -### `msg_ok()` - -**Purpose**: Displays success message with checkmark and green color. - -**Signature**: -```bash -msg_ok() -``` - -**Parameters**: -- `$@` - Message text - -**Format**: `[✔ī¸] Message text` (green) - -**Usage Examples**: - -```bash -msg_ok "Container created" -# Output: ✔ī¸ Container created (in green) - -msg_ok "Network Connected: 10.0.3.50" -# Output: ✔ī¸ Network Connected: 10.0.3.50 -``` - ---- - -### `msg_error()` - -**Purpose**: Displays error message with cross icon and red color. Does not exit. - -**Signature**: -```bash -msg_error() -``` - -**Parameters**: -- `$@` - Message text - -**Format**: `[✖ī¸] Message text` (red) - -**Usage Examples**: - -```bash -msg_error "Container ID already in use" -# Output: ✖ī¸ Container ID already in use (in red) -``` - ---- - -### `msg_warn()` - -**Purpose**: Displays warning message with yellow color. - -**Signature**: -```bash -msg_warn() -``` - -**Parameters**: -- `$@` - Message text - -**Format**: `[⚠ī¸] Message text` (yellow/orange) - -**Usage Examples**: - -```bash -msg_warn "This will delete all data" -# Output: ⚠ī¸ This will delete all data -``` - ---- - -## Execution Helpers - -### `silent()` - -**Purpose**: Executes command with output redirected to log file. On error: displays last 10 lines of log and exits. - -**Signature**: -```bash -silent() -``` - -**Parameters**: -- `$@` - Command and arguments to execute - -**Returns**: 0 on success; exits with original error code on failure - -**Environment Effects**: -- Temporarily disables `set -e` and error trap to capture exit code -- Re-enables after command completes -- Logs to `$BUILD_LOG` or `$INSTALL_LOG` - -**Log Display On Error**: -```bash ---- Last 10 lines of silent log --- -[log output] ------------------------------------ -``` - -**Usage Examples**: - -```bash -# Example 1: Suppress package manager output -silent apt-get update -# Output: suppressed, logged to file - -# Example 2: Conditional display on error -silent curl -fsSL https://api.example.com -# If curl fails: shows last 10 log lines and exits - -# Example 3: Verbose mode shows everything -VERBOSE="yes" -silent apt-get update # Shows all output (STD is empty) -``` - ---- - -### `spinner()` - -**Purpose**: Displays animated spinner with rotating characters during long operations. - -**Signature**: -```bash -spinner() -``` - -**Parameters**: None (uses `$SPINNER_MSG` environment variable) - -**Animation**: -``` -⠋ ⠙ â š â ¸ â ŧ â ´ â Ļ â § ⠇ ⠏ (repeating) -``` - -**Environment Variables**: -- `SPINNER_MSG` - Text to display with spinner - -**Lifecycle**: -```bash -# Start spinner in background -SPINNER_MSG="Downloading..." -spinner & -SPINNER_PID=$! - -# ... do long operation ... - -# Stop spinner -stop_spinner - -echo "Done!" -``` - -**Usage Examples**: - -```bash -# Example 1: Long operation with spinner -SPINNER_MSG="Building container..." -spinner & -SPINNER_PID=$! - -sleep 10 # Simulate work - -stop_spinner -msg_ok "Container created" -``` - ---- - -### `clear_line()` - -**Purpose**: Clears current terminal line and moves cursor to beginning. - -**Signature**: -```bash -clear_line() -``` - -**Parameters**: None - -**Implementation**: Uses `tput` or ANSI escape codes - -**Usage Examples**: - -```bash -for file in *.sh; do - echo -n "Processing $file..." - process_file "$file" - clear_line -done -# Each file overwrites previous line -``` - ---- - -### `stop_spinner()` - -**Purpose**: Stops running spinner process and cleans up temporary files. - -**Signature**: -```bash -stop_spinner() -``` - -**Parameters**: None (reads `$SPINNER_PID` or `/tmp/.spinner.pid`) - -**Cleanup**: -- Graceful kill of spinner process -- Force kill (-9) if needed -- Removes `/tmp/.spinner.pid` temp file -- Resets terminal state - -**Usage Examples**: - -```bash -# Example 1: Simple stop -spinner & -SPINNER_PID=$! -sleep 5 -stop_spinner - -# Example 2: In trap handler -trap 'stop_spinner' EXIT -spinner & -``` - ---- - -## Development Mode - -### Enabling Development Features - -**Via Environment Variable**: -```bash -dev_mode="trace,keep,breakpoint" bash ct/myapp.sh -``` - -**Via Script Header**: -```bash -#!/bin/bash -export dev_mode="trace,logs,pause" -source <(curl -fsSL .../core.func) -load_functions -parse_dev_mode -``` - -### Available Modes - -| Mode | Effect | -|------|--------| -| `trace` | Enable bash -x tracing (verbose command logging) | -| `keep` | Never delete container on error (for debugging) | -| `logs` | Persist all logs to /var/log/community-scripts/ | -| `pause` | Pause after each msg_info step (manual stepping) | -| `breakpoint` | Open shell on error instead of immediate cleanup | -| `motd` | Configure SSH/MOTD before installation starts | -| `dryrun` | Show commands without executing them | - ---- - -## Best Practices - -### 1. **Always Call load_functions() First** - -```bash -#!/bin/bash -set -Eeuo pipefail - -source <(curl -fsSL .../core.func) -load_functions # MUST be before using any color variables - -msg_info "Starting setup" # Now safe to use -``` - -### 2. **Use Message Functions Consistently** - -```bash -msg_info "Starting step" -# Do work... -msg_ok "Step completed" - -# Or on error: -if ! command; then - msg_error "Command failed" - exit 1 -fi -``` - -### 3. **Combine Validation Checks** - -```bash -#!/bin/bash -source <(curl -fsSL .../core.func) -load_functions - -shell_check # Exits if wrong shell -root_check # Exits if not root -pve_check # Exits if unsupported version -arch_check # Exits if wrong architecture - -# All checks passed, safe to proceed -msg_ok "Pre-flight checks passed" -``` - -### 4. **Use Verbose Mode for Debugging** - -```bash -VERBOSE="yes" bash ct/myapp.sh -# Shows all silent() command output for troubleshooting -``` - -### 5. **Log Important Operations** - -```bash -silent apt-get update # Suppress unless error -msg_ok "Packages updated" # Show success -silent systemctl start nginx # Suppress unless error -msg_ok "Nginx started" # Show success -``` - ---- - -## Contributing - -### Adding New Message Functions - -Follow existing pattern: - -```bash -msg_custom() { - local icon="$1" - local color="$2" - local message="$3" - echo -e "${TAB}${icon}${TAB}${color}${message}${CL}" -} -``` - -### Adding Color Support - -New colors should follow semantic naming: - -```bash -BG_ERROR=$'\e[41m' # Red background for errors -BG_SUCCESS=$'\e[42m' # Green background for success -``` - -### Testing Color Output - -```bash -bash -source <(curl -fsSL .../core.func) -load_functions - -echo -e "${YW}Yellow${CL} ${RD}Red${CL} ${GN}Green${CL} ${BL}Blue${CL}" -``` - ---- - -## Notes - -- Core.func is designed to be **sourced once** and **loaded everywhere** -- All color variables are **ANSI escape codes** (work in all terminals) -- Messages use **emoji icons** for visual consistency -- Validation checks use **standard exit codes** (0 for success, 1 for error) -- The module is **lightweight** and loads instantly - diff --git a/docs/APP-ct.md b/docs/ct/DETAILED_GUIDE.md similarity index 63% rename from docs/APP-ct.md rename to docs/ct/DETAILED_GUIDE.md index e5dfa42e5..ba5a00ad3 100644 --- a/docs/APP-ct.md +++ b/docs/ct/DETAILED_GUIDE.md @@ -279,20 +279,10 @@ echo -e "${TAB}${GATEWAY}${BGN}http://${IP}:8080${CL}" 4. Initialize application-specific defaults 5. Setup SSH/environment configuration -**Example Output**: -``` -Setting up variables... -Container ID: 100 -Storage: local-lvm -Install method: Default -``` - #### `start()` **Purpose**: Launch the container creation menu with 5 installation modes -**Triggered by**: Called just before `build_container()` - **Menu Options**: ``` 1. Default Installation (Quick setup, predefined settings) @@ -302,17 +292,6 @@ Install method: Default 5. Settings Menu (Interactive mode selection) ``` -**User Flow**: -``` -Select installation mode: -1) Default -2) Advanced -3) User Defaults -4) App Defaults -5) Settings Menu -Enter choice: 2 -``` - #### `build_container()` **Purpose**: Main orchestrator for LXC container creation @@ -324,63 +303,19 @@ Enter choice: 2 4. Monitors installation progress 5. Handles errors and rollback on failure -**Exit Codes**: -- `0` - Success -- `1-255` - Various error conditions (see error_handler.func) - #### `description()` **Purpose**: Set container description/notes visible in Proxmox UI -**Format**: -``` -AppName -IP: [IP] -Version: [Version] -Tags: [Tags] -``` - -#### `header_info()` - -**Purpose**: Display ASCII art header for application - -**Sources**: -- Tries `/usr/local/community-scripts/headers/ct/appname` (cached) -- Falls back to remote fetch from GitHub -- Returns silently if not found - --- ## Advanced Features -### 1. Integration with Defaults System - -#### Save App Defaults After Installation - -```bash -# At end of install script, after successful setup: -maybe_offer_save_app_defaults - -# Output: -# "Save these settings as App Defaults for AppName? (Y/n)" -# Yes → Saves to /defaults/appname.vars -# No → Skips saving -``` - -#### Load Saved Defaults During Container Creation - -```bash -# In ct/AppName.sh, user selects "App Defaults" mode -# Automatically loads /defaults/appname.vars -# Container uses previously saved configuration -``` - -### 2. Custom Configuration Menus +### 1. Custom Configuration Menus If your app has additional setup beyond standard vars: ```bash -# In ct/AppName.sh, after variables() custom_app_settings() { CONFIGURE_DB=$(whiptail --title "Database Setup" \ --yesno "Would you like to configure a custom database?" 8 60) @@ -394,25 +329,11 @@ custom_app_settings() { custom_app_settings ``` -### 3. Version Tracking +### 2. Update Function Patterns -Save installed version for update checks: +Save installed version for update checks -```bash -# In install script, after successful app download: -RELEASE="1.2.3" -echo "${RELEASE}" > /opt/${APP}_version.txt - -# In update function, compare: -CURRENT=$(cat /opt/${APP}_version.txt 2>/dev/null) -LATEST=$(curl -fsSL https://api.github.com/repos/user/repo/releases/latest | jq -r '.tag_name') - -if [[ "$LATEST" != "$CURRENT" ]]; then - echo "Update available: $CURRENT → $LATEST" -fi -``` - -### 4. Health Check Functions +### 3. Health Check Functions Add custom validation: @@ -432,8 +353,6 @@ function health_check() { msg_ok "Health check passed" } - -# Called via: bash ct/appname.sh health_check ``` --- @@ -461,87 +380,13 @@ color catch_errors function update_script() { - header_info - check_container_storage - check_container_resources - - if [[ ! -d /opt/homarr ]]; then - msg_error "No ${APP} Installation Found!" - exit - fi - - RELEASE=$(curl -fsSL https://api.github.com/repos/ajnart/homarr/releases/latest | grep "tag_name" | awk '{print substr($2, 3, length($2)-4)}') - - if [[ ! -f /opt/${APP}_version.txt ]] || [[ "${RELEASE}" != "$(cat /opt/${APP}_version.txt)" ]]; then - msg_info "Updating ${APP} to v${RELEASE}" - systemctl stop homarr - - cd /opt/homarr - wget -q "https://github.com/ajnart/homarr/releases/download/v${RELEASE}/docker-compose.yml" - docker-compose up -d - - echo "${RELEASE}" > /opt/${APP}_version.txt - msg_ok "Updated ${APP} to v${RELEASE}" - else - msg_ok "No update required. ${APP} is already at v${RELEASE}." - fi - exit + # Update logic here } start build_container description - msg_ok "Completed Successfully!\n" -echo -e "${CREATING}${GN}${APP} setup has been successfully initialized!${CL}" -echo -e "${INFO}${YW} Access it using the following URL:${CL}" -echo -e "${TAB}${GATEWAY}${BGN}http://${IP}:5100${CL}" -``` - -### Example 2: Database App (Alpine-based) - -```bash -#!/usr/bin/env bash -source <(curl -fsSL https://git.community-scripts.org/community-scripts/ProxmoxVED/raw/branch/main/misc/build.func) - -APP="PostgreSQL" -var_tags="database;sql" -var_cpu="4" -var_ram="4096" -var_disk="20" -var_os="alpine" -var_version="3.20" -var_unprivileged="1" - -header_info "$APP" -variables -color -catch_errors - -function update_script() { - header_info - check_container_storage - - if ! command -v psql &>/dev/null; then - msg_error "PostgreSQL not installed!" - exit - fi - - msg_info "Updating Alpine packages" - apk update - apk upgrade - msg_ok "Updated Alpine packages" - exit -} - -start -build_container -description - -msg_ok "Completed Successfully!\n" -echo -e "${CREATING}${GN}${APP} setup has been successfully initialized!${CL}" -echo -e "${INFO}${YW} Connect using:${CL}" -echo -e "${TAB}${GATEWAY}${BGN}psql -h ${IP} -U postgres${CL}" ``` --- @@ -552,11 +397,6 @@ echo -e "${TAB}${GATEWAY}${BGN}psql -h ${IP} -U postgres${CL}" **Symptom**: `pct create` exits with error code 209 -**Causes**: -1. CTID already exists: `pct list` shows duplicate -2. Storage full: Check storage space -3. Network template unavailable - **Solution**: ```bash # Check existing containers @@ -570,13 +410,6 @@ pct destroy CTID ### Update Function Doesn't Detect New Version -**Symptom**: Update available but script says "already at latest" - -**Causes**: -1. Version file missing: `/opt/AppName_version.txt` -2. GitHub API rate limit exceeded -3. Release tag format mismatch - **Debug**: ```bash # Check version file @@ -584,27 +417,6 @@ cat /opt/AppName_version.txt # Test GitHub API curl -fsSL https://api.github.com/repos/user/repo/releases/latest | grep tag_name - -# Inside container -bash ct/appname.sh update_script -``` - -### Header ASCII Art Not Displaying - -**Symptom**: Container script runs but no header shown - -**Causes**: -1. Header file not in repository -2. Caching issue - -**Solution**: -```bash -# Create header file manually -mkdir -p /usr/local/community-scripts/headers/ct -echo "Your ASCII art here" > /usr/local/community-scripts/headers/ct/appname - -# Or remove cache to force re-download -rm -f /usr/local/community-scripts/headers/ct/appname ``` --- @@ -615,7 +427,7 @@ Before submitting a PR: ### Script Structure - [ ] Shebang is `#!/usr/bin/env bash` -- [ ] Imports `build.func` from community-scripts repo (not personal fork) +- [ ] Imports `build.func` from community-scripts repo - [ ] Copyright header with author and source URL - [ ] APP variable matches filename - [ ] `var_tags` are semicolon-separated (no spaces) @@ -624,26 +436,17 @@ Before submitting a PR: - [ ] `var_cpu` set appropriately (2-4 for most apps) - [ ] `var_ram` set appropriately (1024-4096 MB minimum) - [ ] `var_disk` sufficient for app + data (5-20 GB) -- [ ] `var_os` is realistic (Alpine if lightweight, Debian/Ubuntu otherwise) -- [ ] `var_unprivileged="1"` unless app absolutely needs privileges +- [ ] `var_os` is realistic ### Functions -- [ ] `update_script()` implemented (or marked as unavailable) +- [ ] `update_script()` implemented - [ ] Update function checks if app installed -- [ ] Update function checks for new version -- [ ] Update function performs cleanup after update -- [ ] Proper error handling with `msg_error` on failure - -### Output -- [ ] Success message displayed with access URL -- [ ] URL format: `http://IP:PORT/path` (if web-based) -- [ ] Uses `msg_ok`, `msg_info`, `msg_error` for feedback +- [ ] Proper error handling with `msg_error` ### Testing - [ ] Script tested with default installation - [ ] Script tested with advanced (19-step) installation - [ ] Update function tested on existing installation -- [ ] Error handling tested (invalid settings, network issues) --- @@ -652,66 +455,18 @@ Before submitting a PR: ### ✅ DO: 1. **Use meaningful defaults** - ```bash - var_cpu="2" # ✅ Good: Typical workload - var_cpu="128" # ❌ Bad: Unrealistic - ``` - 2. **Implement version tracking** - ```bash - echo "${RELEASE}" > /opt/${APP}_version.txt # ✅ Good - # ❌ Bad: No version tracking - ``` - 3. **Handle edge cases** - ```bash - if [[ ! -f /opt/${APP}_version.txt ]]; then - msg_info "First installation detected" - fi - ``` - -4. **Use proper messaging** - ```bash - msg_info "Updating..." # ✅ Good: Clear status - echo "Updating..." # ❌ Bad: No formatting - ``` +4. **Use proper messaging with msg_info/msg_ok/msg_error** ### ❌ DON'T: 1. **Hardcode versions** - ```bash - RELEASE="1.2.3" # ❌ Bad: Won't auto-update - ``` - -2. **Use custom color codes** - ```bash - echo -e "\033[32mSuccess" # ❌ Bad: Use $GN instead - ``` - +2. **Use custom color codes** (use built-in variables) 3. **Forget error handling** - ```bash - wget file.zip # ❌ Bad: No error check - if ! wget -q file.zip; then # ✅ Good - msg_error "Download failed" - fi - ``` - 4. **Leave temporary files** - ```bash - rm -rf /opt/file.zip # ✅ Always cleanup - ``` - ---- - -## Related Documentation - -- [install/AppName-install.sh Guide](UPDATED_APP-install.md) -- [build.func Wiki](../misc/build.func.md) -- [tools.func Wiki](../misc/tools.func.md) -- [Defaults System Guide](../DEFAULTS_SYSTEM_GUIDE.md) --- **Last Updated**: December 2025 **Compatibility**: ProxmoxVED with build.func v3+ -**Questions?** Open an issue in the repository diff --git a/docs/error_handler.func.md b/docs/error_handler.func.md deleted file mode 100644 index f8b2aece0..000000000 --- a/docs/error_handler.func.md +++ /dev/null @@ -1,598 +0,0 @@ -# Error-Handler.func Wiki - -Comprehensive error handling and signal management module providing exit code explanations, error handlers with logging, and signal trap configuration for all Community-Scripts projects. - ---- - -## 📋 Table of Contents - -- [Overview](#overview) -- [Exit Code Reference](#exit-code-reference) -- [Error Handler Functions](#error-handler-functions) -- [Signal Traps](#signal-traps) -- [Initialization & Setup](#initialization--setup) -- [Error Logging](#error-logging) -- [Best Practices](#best-practices) -- [Debugging](#debugging) -- [Contributing](#contributing) - ---- - -## Overview - -The error_handler.func module provides robust error handling infrastructure: - -- ✅ Comprehensive exit code mapping (1-255+ codes documented) -- ✅ Detailed error messages with line numbers and commands -- ✅ Signal trap configuration (ERR, EXIT, INT, TERM, RETURN) -- ✅ Error logging to persistent files -- ✅ Graceful cleanup on signal termination -- ✅ Stack trace display for debugging -- ✅ Integration with core.func message functions -- ✅ Container-agnostic (works in Proxmox + LXC) - -### Error Handling Flow - -``` -Command Execution - ↓ - ERROR (non-zero exit) - ↓ - ERR Trap Triggered - ↓ -error_handler() called - ↓ -explain_exit_code() lookup - ↓ -Display error with line/command - ↓ -Check for log file - ↓ -Exit with original code -``` - ---- - -## Exit Code Reference - -Exit codes are categorized by source system. See `api.func.md` for comprehensive mapping documentation. - -### Quick Reference Table - -| Range | Category | Examples | -|-------|----------|----------| -| 0 | Success | (no error) | -| 1-2 | Shell errors | Syntax error, operation not permitted | -| 100-101 | APT errors | Package manager errors | -| 126-139 | System errors | Command not found, segfault, OOM | -| 200-231 | Proxmox custom | Container creation errors | -| 210-234 | Database errors | PostgreSQL, MySQL connection issues | -| 243-254 | Runtime errors | Node.js, Python, npm errors | -| 255 | DPKG fatal | Package system fatal error | - ---- - -## Error Handler Functions - -### `explain_exit_code()` - -**Purpose**: Maps numeric exit codes to human-readable descriptions. Shared with api.func for consistency. - -**Signature**: -```bash -explain_exit_code() -``` - -**Parameters**: -- `$1` - Exit code (0-255+) - -**Returns**: Human-readable explanation string - -**Categories Handled**: -- Generic shell errors (1, 2, 126-128, 130, 137, 139, 143) -- Package managers (100-101, 255) -- Python (210-212) -- Databases (PostgreSQL 231-234, MySQL 241-244, MongoDB 251-254) -- Node.js/npm (243-249, 254) -- Proxmox custom (200-231) -- Default: "Unknown error" - -**Usage Examples**: - -```bash -# Example 1: Look up error code -explain_exit_code 127 -# Output: "Command not found" - -# Example 2: In error logging -error_desc=$(explain_exit_code "$exit_code") -echo "Error: $error_desc" >> /tmp/error.log - -# Example 3: Unknown code -explain_exit_code 999 -# Output: "Unknown error" -``` - ---- - -### `error_handler()` - -**Purpose**: Main error handler triggered by ERR trap. Displays detailed error information and exits. - -**Signature**: -```bash -error_handler() -``` - -**Parameters**: -- `$1` (optional) - Exit code (default: current $?) -- `$2` (optional) - Command that failed (default: $BASH_COMMAND) -- `$3` (optional) - Line number (default: ${BASH_LINENO[0]}) - -**Returns**: Exits with original exit code (does not return) - -**Output Format**: -``` -[ERROR] in line 42: exit code 1 (General error): while executing command curl https://api.example.com - ---- Last 10 lines of log file --- -[log content] ------------------------------------- -``` - -**Implementation Pattern**: -```bash -error_handler() { - local exit_code=${1:-$?} - local command=${2:-${BASH_COMMAND:-unknown}} - local line_number=${BASH_LINENO[0]:-unknown} - - # If successful, return silently - if [[ "$exit_code" -eq 0 ]]; then - return 0 - fi - - # Get human-readable error description - local explanation=$(explain_exit_code "$exit_code") - - # Show cursor (might be hidden by spinner) - printf "\e[?25h" - - # Display error using color messages - if declare -f msg_error >/dev/null 2>&1; then - msg_error "in line ${line_number}: exit code ${exit_code} (${explanation}): while executing command ${command}" - else - echo -e "\n${RD}[ERROR]${CL} in line ${line_number}: exit code ${exit_code}: ${command}\n" - fi - - # Log error details if log file configured - if [[ -n "${DEBUG_LOGFILE:-}" ]]; then - { - echo "------ ERROR ------" - echo "Timestamp : $(date '+%Y-%m-%d %H:%M:%S')" - echo "Exit Code : $exit_code ($explanation)" - echo "Line : $line_number" - echo "Command : $command" - echo "-------------------" - } >> "$DEBUG_LOGFILE" - fi - - # Show last lines of log if available - local active_log="$(get_active_logfile)" - if [[ -s "$active_log" ]]; then - local log_lines=$(wc -l < "$active_log") - echo "--- Last 10 lines of log ---" - tail -n 10 "$active_log" - echo "----------------------------" - fi - - exit "$exit_code" -} -``` - -**Usage Examples**: - -```bash -# Example 1: Automatic trap (recommended) -trap 'error_handler $? "$BASH_COMMAND" $LINENO' ERR -# Error automatically caught and handled - -# Example 2: Manual invocation (testing) -error_handler 1 "curl https://api.example.com" 42 -# Output: Detailed error with line number - -# Example 3: In conditional -if ! some_command; then - error_handler $? "some_command" $LINENO -fi -``` - ---- - -## Signal Traps - -### `on_exit()` - -**Purpose**: Cleanup handler called on normal script exit or error. - -**Signature**: -```bash -on_exit() -``` - -**Parameters**: None - -**Returns**: Exits with captured exit code - -**Behavior**: -- Captures current exit code -- Removes lock files if present -- Exits with original exit code - -**Implementation Pattern**: -```bash -on_exit() { - local exit_code="$?" - - # Cleanup lock files - [[ -n "${lockfile:-}" && -e "$lockfile" ]] && rm -f "$lockfile" - - # Preserve exit code - exit "$exit_code" -} -``` - -**Trap Configuration**: -```bash -trap on_exit EXIT # Always called on exit -``` - ---- - -### `on_interrupt()` - -**Purpose**: Handler for Ctrl+C (SIGINT) signal. Allows graceful shutdown. - -**Signature**: -```bash -on_interrupt() -``` - -**Parameters**: None - -**Returns**: Exits with code 130 (standard SIGINT exit code) - -**Output**: Displays "Interrupted by user (SIGINT)" message - -**Implementation Pattern**: -```bash -on_interrupt() { - echo -e "\n${RD}Interrupted by user (SIGINT)${CL}" - exit 130 -} -``` - -**Trap Configuration**: -```bash -trap on_interrupt INT # Called on Ctrl+C -``` - -**Usage Example**: -```bash -# Script interrupted by user: -# Ctrl+C pressed -# → on_interrupt() triggers -# → "Interrupted by user (SIGINT)" displayed -# → Exit with code 130 -``` - ---- - -### `on_terminate()` - -**Purpose**: Handler for SIGTERM signal. Allows graceful shutdown on termination. - -**Signature**: -```bash -on_terminate() -``` - -**Parameters**: None - -**Returns**: Exits with code 143 (standard SIGTERM exit code) - -**Output**: Displays "Terminated by signal (SIGTERM)" message - -**Implementation Pattern**: -```bash -on_terminate() { - echo -e "\n${RD}Terminated by signal (SIGTERM)${CL}" - exit 143 -} -``` - -**Trap Configuration**: -```bash -trap on_terminate TERM # Called on SIGTERM -``` - -**Usage Example**: -```bash -# System sends SIGTERM: -# kill -TERM $PID executed -# → on_terminate() triggers -# → "Terminated by signal (SIGTERM)" displayed -# → Exit with code 143 -``` - ---- - -## Initialization & Setup - -### `catch_errors()` - -**Purpose**: Sets up all error traps and signal handlers. Called once at script start. - -**Signature**: -```bash -catch_errors() -``` - -**Parameters**: None - -**Returns**: No explicit return value (configures traps) - -**Traps Configured**: -1. `ERR` → `error_handler()` - Catches command failures -2. `EXIT` → `on_exit()` - Cleanup on any exit -3. `INT` → `on_interrupt()` - Handle Ctrl+C -4. `TERM` → `on_terminate()` - Handle SIGTERM -5. `RETURN` → `error_handler()` - Catch function errors - -**Implementation Pattern**: -```bash -catch_errors() { - # Set strict mode - set -Eeuo pipefail - - # Configure traps - trap 'error_handler $? "$BASH_COMMAND" $LINENO' ERR - trap on_exit EXIT - trap on_interrupt INT - trap on_terminate TERM - trap 'error_handler $? "$BASH_COMMAND" $LINENO' RETURN -} -``` - -**Usage Examples**: - -```bash -# Example 1: Alpine container script -#!/bin/sh -source <(curl -fsSL .../core.func) -source <(curl -fsSL .../error_handler.func) -load_functions -catch_errors - -# Now all signals handled automatically -update_os - -# Example 2: Proxmox host script -#!/bin/bash -source <(curl -fsSL .../core.func) -source <(curl -fsSL .../error_handler.func) -load_functions -catch_errors - -# Safe to proceed with error handling -create_container -``` - ---- - -## Error Logging - -### Log File Configuration - -**Active Log Detection**: -```bash -# In build.func/install.func: -BUILD_LOG="/tmp/create-lxc-${SESSION_ID}.log" -INSTALL_LOG="/root/install-${SESSION_ID}.log" -SILENT_LOGFILE="$(get_active_logfile)" # Points to appropriate log -``` - -### Log Output Behavior - -When command fails in `silent()`: - -1. Last 10 lines of log file are displayed -2. Full log path shown if more than 10 lines -3. Error message includes line number where failure occurred -4. Command that failed is displayed - -### Accessing Error Logs - -From Proxmox host: -```bash -# Host-side container creation log -/tmp/create-lxc-.log - -# View error details -tail -50 /tmp/create-lxc-550e8400.log -grep ERROR /tmp/create-lxc-*.log - -# Development mode persistent logs -/var/log/community-scripts/create-lxc--.log -``` - -From inside LXC container: -```bash -# Container installation log -/root/install-.log - -# View recent errors -tail -20 /root/install-550e8400.log -``` - ---- - -## Best Practices - -### 1. **Always Setup Traps Early** - -```bash -#!/bin/bash -set -Eeuo pipefail - -source <(curl -fsSL .../core.func) -source <(curl -fsSL .../error_handler.func) -load_functions -catch_errors # MUST be called before any real work - -# Now safe - all signals handled -``` - -### 2. **Use Meaningful Error Exit Codes** - -```bash -# Use Proxmox custom codes for container-specific errors -if [[ "$CTID" -lt 100 ]]; then - msg_error "Container ID must be >= 100" - exit 205 # Proxmox custom code -fi - -# Use standard codes for common errors -if ! command -v curl &>/dev/null; then - msg_error "curl not installed" - exit 127 # Command not found -fi -``` - -### 3. **Log Context Information** - -```bash -# In error_handler, DEBUG_LOGFILE receives: -DEBUG_LOGFILE="/tmp/debug.log" - -# All errors logged with timestamp and details -{ - echo "Error at $(date)" - echo "Exit code: $exit_code" - echo "Command: $command" -} >> "$DEBUG_LOGFILE" -``` - -### 4. **Graceful Signal Handling** - -```bash -# Setup signal handlers for cleanup -cleanup() { - [[ -f "$temp_file" ]] && rm -f "$temp_file" - [[ -d "$temp_dir" ]] && rm -rf "$temp_dir" -} -trap cleanup EXIT - -# Now temporary files always cleaned up -``` - -### 5. **Test Error Paths** - -```bash -# Force error for testing -false # Triggers error_handler -# or -exit 1 # Custom error - -# Verify error handling works correctly -# Check log files and messages -``` - ---- - -## Debugging - -### Enable Stack Trace - -```bash -# Via environment variable -DEV_MODE_TRACE=true bash script.sh - -# Or in script -set -x # Show all commands -trap 'error_handler $? "$BASH_COMMAND" $LINENO' ERR -``` - -### View Full Error Context - -```bash -# Show full log file instead of last 10 lines -DEBUG_LOGFILE="/tmp/full-debug.log" - -# After error, review complete context -less /tmp/full-debug.log -``` - -### Test Error Handler - -```bash -# Manually trigger error handler -bash -c 'source <(curl -fsSL .../error_handler.func); catch_errors; exit 42' - -# Should display: -# [ERROR] in line N: exit code 42 (Unknown error): ... -``` - ---- - -## Contributing - -### Adding New Error Codes - -1. Assign code in appropriate range (see Exit Code Reference) -2. Add description to `explain_exit_code()` in both: - - error_handler.func - - api.func (for consistency) -3. Document in exit code table -4. Update error mapping documentation - -### Improving Error Messages - -Example: Make error message more helpful: - -```bash -# Before: -"Container ID must be >= 100" - -# After: -"Invalid CTID: $CTID. Container IDs must be >= 100. Current range: 100-999" -``` - -### Testing Signal Handlers - -```bash -# Test INT signal (Ctrl+C) -bash -c 'source <(curl -fsSL .../error_handler.func); catch_errors; sleep 30' & -PID=$! -sleep 1 -kill -INT $PID -wait - -# Test TERM signal -bash -c 'source <(curl -fsSL .../error_handler.func); catch_errors; sleep 30' & -PID=$! -sleep 1 -kill -TERM $PID -wait -``` - ---- - -## Notes - -- Error handler is **required** for all scripts (ensures safe cleanup) -- Exit codes are **standardized** (0 = success, 1-255 = specific errors) -- Signals are **trapped** to allow graceful shutdown -- Lock files are **automatically cleaned** on exit -- Log files contain **full error context** for debugging - diff --git a/docs/install.func.md b/docs/install.func.md deleted file mode 100644 index c5f6be72c..000000000 --- a/docs/install.func.md +++ /dev/null @@ -1,646 +0,0 @@ -# Install.func Wiki - -Container installation workflow orchestration module providing network setup, OS configuration, connectivity verification, and installation mechanics for applications deployed inside LXC containers. - ---- - -## 📋 Table of Contents - -- [Overview](#overview) -- [Initialization & Dependencies](#initialization--dependencies) -- [Network & Connectivity Functions](#network--connectivity-functions) -- [OS Configuration Functions](#os-configuration-functions) -- [Installation Workflow](#installation-workflow) -- [Best Practices](#best-practices) -- [Debugging](#debugging) -- [Contributing](#contributing) - ---- - -## Overview - -Install.func provides **container-internal installation mechanics**: - -- ✅ Network connectivity verification (IPv4/IPv6) -- ✅ OS updates and package management -- ✅ DNS resolution validation -- ✅ System optimization (disable wait-online service) -- ✅ SSH and MOTD configuration -- ✅ Container customization (auto-login, update script) -- ✅ Comprehensive error handling with signal traps -- ✅ Integration with core.func and error_handler.func - -### Execution Context - -``` -Proxmox Host LXC Container -────────────────────────────────────────── -pct create CTID ... - ↓ -Boot container - ↓ -pct exec CTID bash /tmp/install.sh - ↓ -[Execution within container] - └─→ install.func functions execute - └─→ verb_ip6() - └─→ setting_up_container() - └─→ network_check() - └─→ update_os() - └─→ etc. -``` - ---- - -## Initialization & Dependencies - -### Module Dependencies - -```bash -# Install.func requires two prerequisites -if ! command -v curl >/dev/null 2>&1; then - apt-get update >/dev/null 2>&1 - apt-get install -y curl >/dev/null 2>&1 -fi - -# Source core functions (colors, formatting, messages) -source <(curl -fsSL https://git.community-scripts.org/.../core.func) - -# Source error handling (traps, signal handlers) -source <(curl -fsSL https://git.community-scripts.org/.../error_handler.func) - -# Initialize both modules -load_functions # Sets up colors, icons, defaults -catch_errors # Configures ERR, EXIT, INT, TERM traps -``` - -### Environment Variables Passed from Host - -These variables are passed by build.func via `pct set` and environment: - -| Variable | Source | Purpose | -|----------|--------|---------| -| `VERBOSE` | Build config | Show all output (yes/no) | -| `PASSWORD` | User input | Root password (blank = auto-login) | -| `DISABLEIPV6` | Advanced settings | Disable IPv6 (yes/no) | -| `SSH_ROOT` | Advanced settings | Enable SSH root access | -| `CACHER` | Config | Use APT cache proxy (yes/no) | -| `CACHER_IP` | Config | APT cache IP address | -| `APPLICATION` | App script | App display name | -| `app` | App script | Normalized app name (lowercase) | -| `RETRY_NUM` | core.func | Retry attempts (default: 10) | -| `RETRY_EVERY` | core.func | Retry interval in seconds (default: 3) | - ---- - -## Network & Connectivity Functions - -### `verb_ip6()` - -**Purpose**: Configures IPv6 based on DISABLEIPV6 variable and sets verbose mode. - -**Signature**: -```bash -verb_ip6() -``` - -**Parameters**: None - -**Returns**: No explicit return value (configures system) - -**Environment Requirements**: -- `DISABLEIPV6` - Set to "yes" to disable IPv6, "no" to keep enabled -- `VERBOSE` - Controls output verbosity via set_std_mode() - -**Behavior**: -```bash -# If DISABLEIPV6=yes: -echo "net.ipv6.conf.all.disable_ipv6 = 1" >> /etc/sysctl.conf -sysctl -p # Apply immediately - -# If DISABLEIPV6=no (default): -# No changes (IPv6 remains enabled) -``` - -**Usage Examples**: - -```bash -# Example 1: Disable IPv6 (for security/simplicity) -DISABLEIPV6="yes" -VERBOSE="no" -verb_ip6 -# Result: IPv6 disabled, change persisted - -# Example 2: Keep IPv6 enabled (default) -DISABLEIPV6="no" -verb_ip6 -# Result: IPv6 operational, no configuration - -# Example 3: Verbose mode -VERBOSE="yes" -verb_ip6 -# Output: Shows sysctl configuration commands -``` - ---- - -### `setting_up_container()` - -**Purpose**: Verifies network connectivity and performs initial OS configuration for Debian/Ubuntu containers. - -**Signature**: -```bash -setting_up_container() -``` - -**Parameters**: None - -**Returns**: 0 on success; exits with code 1 if network unavailable after retries - -**Environment Requirements**: -- `RETRY_NUM` - Max attempts (default: 10) -- `RETRY_EVERY` - Seconds between retries (default: 3) - -**Operations**: -1. Verify network connectivity via `hostname -I` -2. Retry up to RETRY_NUM times with RETRY_EVERY second delays -3. Remove Python EXTERNALLY-MANAGED marker (allows pip) -4. Disable systemd-networkd-wait-online.service (speeds up boot) -5. Display network information - -**Implementation Pattern**: -```bash -setting_up_container() { - msg_info "Setting up Container OS" - - # Network availability loop - for ((i = RETRY_NUM; i > 0; i--)); do - if [ "$(hostname -I)" != "" ]; then - break - fi - echo 1>&2 -en "${CROSS}${RD} No Network! " - sleep $RETRY_EVERY - done - - # Check final state - if [ "$(hostname -I)" = "" ]; then - echo 1>&2 -e "\n${CROSS}${RD} No Network After $RETRY_NUM Tries${CL}" - exit 1 - fi - - # Python pip support - rm -rf /usr/lib/python3.*/EXTERNALLY-MANAGED - - # Speed up boot (disable wait service) - systemctl disable -q --now systemd-networkd-wait-online.service - - msg_ok "Set up Container OS" - msg_ok "Network Connected: ${BL}$(hostname -I)${CL}" -} -``` - -**Usage Examples**: - -```bash -# Example 1: Immediate network availability -RETRY_NUM=10 -RETRY_EVERY=3 -setting_up_container -# Output: -# ℹī¸ Setting up Container OS -# ✔ī¸ Set up Container OS -# ✔ī¸ Network Connected: 10.0.3.50 - -# Example 2: Delayed network (waits 6 seconds) -# Script retries 2 times before succeeding -# (each retry waits 3 seconds) - -# Example 3: No network -# Script waits 30 seconds total (10 x 3) -# Then exits with: "No Network After 10 Tries" -``` - ---- - -### `network_check()` - -**Purpose**: Comprehensive network diagnostics for both IPv4 and IPv6, including DNS validation for Git/GitHub. - -**Signature**: -```bash -network_check() -``` - -**Parameters**: None - -**Returns**: 0 on success; exits with code 1 on critical DNS failure - -**Checks Performed**: - -1. **IPv4 Connectivity** (tests 3 public DNS servers): - - 1.1.1.1 (Cloudflare) - - 8.8.8.8 (Google) - - 9.9.9.9 (Quad9) - -2. **IPv6 Connectivity** (tests 3 public DNS servers): - - 2606:4700:4700::1111 (Cloudflare) - - 2001:4860:4860::8888 (Google) - - 2620:fe::fe (Quad9) - -3. **DNS Resolution** (validates Git-related domains): - - github.com - - raw.githubusercontent.com - - api.github.com - - git.community-scripts.org - -**Output Format**: -``` -✔ī¸ IPv4 Internet Connected -✔ī¸ IPv6 Internet Connected -✔ī¸ Git DNS: github.com:✔ī¸ raw.githubusercontent.com:✔ī¸ ... -``` - -**Error Handling**: -```bash -# If both IPv4 and IPv6 fail: -# Prompts user: "No Internet detected, would you like to continue anyway?" -# If user says no: Exits -# If user says yes: Shows warning "Expect Issues Without Internet" - -# If DNS fails for GitHub: -# Calls fatal() - exits immediately with error -``` - -**Implementation Pattern**: -```bash -network_check() { - set +e - trap - ERR - - ipv4_connected=false - ipv6_connected=false - - # IPv4 test - if ping -c 1 -W 1 1.1.1.1 &>/dev/null || ...; then - msg_ok "IPv4 Internet Connected" - ipv4_connected=true - else - msg_error "IPv4 Internet Not Connected" - fi - - # IPv6 test - if ping6 -c 1 -W 1 2606:4700:4700::1111 &>/dev/null || ...; then - msg_ok "IPv6 Internet Connected" - ipv6_connected=true - else - msg_error "IPv6 Internet Not Connected" - fi - - # DNS checks for GitHub domains - GIT_HOSTS=("github.com" "raw.githubusercontent.com" "api.github.com" "git.community-scripts.org") - for HOST in "${GIT_HOSTS[@]}"; do - RESOLVEDIP=$(getent hosts "$HOST" | awk '{ print $1 }' | head -n1) - if [[ -z "$RESOLVEDIP" ]]; then - DNS_FAILED=true - fi - done - - if [[ "$DNS_FAILED" == true ]]; then - fatal "$GIT_STATUS" # Exit on critical DNS failure - fi - - set -e - trap 'error_handler $LINENO "$BASH_COMMAND"' ERR -} -``` - -**Usage Examples**: - -```bash -# Example 1: Good connectivity (all checks pass) -network_check -# Output: -# ✔ī¸ IPv4 Internet Connected -# ✔ī¸ IPv6 Internet Connected -# ✔ī¸ Git DNS: github.com:✔ī¸ ... - -# Example 2: IPv6 unavailable but IPv4 OK -network_check -# Output: -# ✔ī¸ IPv4 Internet Connected -# ✖ī¸ IPv6 Internet Not Connected -# ✔ī¸ Git DNS checks OK - -# Example 3: No internet at all -network_check -# Prompts: "No Internet detected, would you like to continue anyway?" -# User: y -# Output: ⚠ī¸ Expect Issues Without Internet -``` - ---- - -## OS Configuration Functions - -### `update_os()` - -**Purpose**: Updates Debian/Ubuntu OS packages and loads additional tools library. - -**Signature**: -```bash -update_os() -``` - -**Parameters**: None - -**Returns**: No explicit return value (updates system) - -**Operations**: -1. Display info message -2. Optional: Configure APT caching proxy -3. Run `apt-get update` (index refresh) -4. Run `apt-get dist-upgrade` (system upgrade) -5. Remove Python EXTERNALLY-MANAGED restrictions -6. Source tools.func for additional setup -7. Display success message - -**APT Caching Configuration** (if CACHER=yes): -```bash -# Configure apt-proxy-detect.sh -/etc/apt/apt.conf.d/00aptproxy - -# Script detects local APT cacher and routes through it -# Falls back to DIRECT if unavailable -``` - -**Implementation Pattern**: -```bash -update_os() { - msg_info "Updating Container OS" - - # Optional: Setup APT cacher - if [[ "$CACHER" == "yes" ]]; then - echo "Acquire::http::Proxy-Auto-Detect \"/usr/local/bin/apt-proxy-detect.sh\";" > /etc/apt/apt.conf.d/00aptproxy - - cat > /usr/local/bin/apt-proxy-detect.sh <<'EOF' -#!/bin/bash -if nc -w1 -z "${CACHER_IP}" 3142; then - echo -n "http://${CACHER_IP}:3142" -else - echo -n "DIRECT" -fi -EOF - chmod +x /usr/local/bin/apt-proxy-detect.sh - fi - - # Update system - $STD apt-get update - $STD apt-get -o Dpkg::Options::="--force-confold" -y dist-upgrade - - # Python support - rm -rf /usr/lib/python3.*/EXTERNALLY-MANAGED - - # Load additional tools library - source <(curl -fsSL https://git.community-scripts.org/.../tools.func) - - msg_ok "Updated Container OS" -} -``` - -**Usage Examples**: - -```bash -# Example 1: Standard update -update_os -# Output: Updates all packages silently (unless VERBOSE=yes) - -# Example 2: With APT cacher -CACHER="yes" -CACHER_IP="192.168.1.100" -update_os -# Uses cache proxy for faster package downloads - -# Example 3: Verbose output -VERBOSE="yes" -update_os -# Shows all apt-get operations in detail -``` - ---- - -## SSH & MOTD Configuration - -### `motd_ssh()` - -**Purpose**: Configures Message of the Day and enables SSH root access if configured. - -**Signature**: -```bash -motd_ssh() -``` - -**Parameters**: None - -**Returns**: No explicit return value (configures system) - -**Operations**: -1. Set TERM environment variable for better terminal support -2. Gather OS information (name, version, IP) -3. Create `/etc/profile.d/00_lxc-details.sh` with container details script -4. Optionally enable root SSH access if SSH_ROOT=yes - -**MOTD Script Content**: -```bash -echo -e "" -echo -e "${BOLD}${YW}${APPLICATION} LXC Container - DEV Repository${CL}" -echo -e "${RD}WARNING: This is a DEVELOPMENT version (ProxmoxVED). Do NOT use in production!${CL}" -echo -e "${YW} OS: ${GN}${OS_NAME} - Version: ${OS_VERSION}${CL}" -echo -e "${YW} Hostname: ${GN}$(hostname)${CL}" -echo -e "${YW} IP Address: ${GN}$(hostname -I | awk '{print $1}')${CL}" -echo -e "${YW} Repository: ${GN}https://github.com/community-scripts/ProxmoxVED${CL}" -echo "" -``` - -**SSH Configuration** (if SSH_ROOT=yes): -```bash -sed -i "s/#PermitRootLogin prohibit-password/PermitRootLogin yes/g" /etc/ssh/sshd_config -systemctl restart sshd -``` - ---- - -## Installation Workflow - -### Typical Installation Sequence - -```bash -#!/bin/bash -# Inside container during installation - -source <(curl -fsSL .../core.func) -source <(curl -fsSL .../error_handler.func) -load_functions -catch_errors - -# Step 1: Network setup -verb_ip6 -setting_up_container -network_check - -# Step 2: System update -update_os - -# Step 3: SSH and MOTD -motd_ssh - -# Step 4: Install application (app-specific) -# ... application installation steps ... - -# Step 5: Create update script -customize -``` - ---- - -## Best Practices - -### 1. **Always Initialize First** - -```bash -#!/bin/bash -set -Eeuo pipefail - -if ! command -v curl >/dev/null 2>&1; then - apt-get update >/dev/null 2>&1 - apt-get install -y curl >/dev/null 2>&1 -fi - -source <(curl -fsSL .../core.func) -source <(curl -fsSL .../error_handler.func) -load_functions -catch_errors -``` - -### 2. **Check Network Early** - -```bash -setting_up_container # Verify network available -network_check # Validate connectivity and DNS -update_os # Proceed with updates - -# If network fails, exit immediately -# Don't waste time on installation -``` - -### 3. **Use Retry Logic** - -```bash -# Built into setting_up_container(): -for ((i = RETRY_NUM; i > 0; i--)); do - if [ "$(hostname -I)" != "" ]; then - break - fi - sleep $RETRY_EVERY -done - -# Tolerates temporary network delay -``` - -### 4. **Separate Concerns** - -```bash -# Network setup -verb_ip6 -setting_up_container -network_check - -# System updates -update_os - -# Configuration -motd_ssh - -# Application-specific -# ... app installation ... -``` - -### 5. **Capture Environment** - -```bash -# Pass these from build.func: -VERBOSE="yes" # Show all output -DISABLEIPV6="no" # Keep IPv6 -SSH_ROOT="yes" # Enable SSH -APPLICATION="Jellyfin" # App name -CACHER="no" # No APT cache -``` - ---- - -## Debugging - -### Enable Verbose Output - -```bash -VERBOSE="yes" pct exec CTID bash /tmp/install.sh -# Shows all commands and output -``` - -### Check Network Status Inside Container - -```bash -pct exec CTID hostname -I -pct exec CTID ping -c 1 1.1.1.1 -pct exec CTID getent hosts github.com -``` - -### View Installation Log - -```bash -# From container -cat /root/install-*.log - -# Or from host (if logs mounted) -tail -100 /var/log/community-scripts/install-*.log -``` - ---- - -## Contributing - -### Adding New Network Checks - -```bash -network_check() { - # ... existing checks ... - - # Add new check: - if ! getent hosts newhost.example.com &>/dev/null; then - msg_warn "Unable to resolve newhost.example.com" - fi -} -``` - -### Extending OS Configuration - -```bash -# Add to update_os(): -update_os() { - # ... existing updates ... - - # Add new capability: - $STD apt-get install -y some-package - msg_ok "Additional package installed" -} -``` - ---- - -## Notes - -- Install.func executes **inside the container** (not on Proxmox host) -- Network connectivity is **critical** - checked early and thoroughly -- OS updates are **required** before application installation -- IPv6 is **configurable** but enabled by default -- SSH and MOTD are **informational** - help with container management - diff --git a/docs/install/DETAILED_GUIDE.md b/docs/install/DETAILED_GUIDE.md new file mode 100644 index 000000000..0ac4fddb5 --- /dev/null +++ b/docs/install/DETAILED_GUIDE.md @@ -0,0 +1,646 @@ +# 🛠ī¸ **Application Installation Scripts (install/AppName-install.sh)** + +**Modern Guide to Writing In-Container Installation Scripts** + +> **Updated**: December 2025 +> **Context**: Integrated with tools.func, error_handler.func, and install.func +> **Examples Used**: `/install/pihole-install.sh`, `/install/mealie-install.sh` + +--- + +## 📋 Table of Contents + +- [Overview](#overview) +- [Execution Context](#execution-context) +- [File Structure](#file-structure) +- [Complete Script Template](#complete-script-template) +- [Installation Phases](#installation-phases) +- [Function Reference](#function-reference) +- [Best Practices](#best-practices) +- [Real Examples](#real-examples) +- [Troubleshooting](#troubleshooting) +- [Contribution Checklist](#contribution-checklist) + +--- + +## Overview + +### Purpose + +Installation scripts (`install/AppName-install.sh`) **run inside the LXC container** and are responsible for: + +1. Setting up the container OS (updates, packages) +2. Installing application dependencies +3. Downloading and configuring the application +4. Setting up services and systemd units +5. Creating version tracking files for updates +6. Generating credentials/configurations +7. Final cleanup and validation + +### Execution Flow + +``` +ct/AppName.sh (Proxmox Host) + ↓ +build_container() + ↓ +pct exec CTID bash -c "$(cat install/AppName-install.sh)" + ↓ +install/AppName-install.sh (Inside Container) + ↓ +Container Ready with App Installed +``` + +--- + +## Execution Context + +### Environment Variables Available + +```bash +# From Proxmox/Container +CTID # Container ID (100, 101, etc.) +PCT_OSTYPE # OS type (alpine, debian, ubuntu) +HOSTNAME # Container hostname + +# From build.func +FUNCTIONS_FILE_PATH # Bash functions library (core.func + tools.func) +VERBOSE # Verbose mode (yes/no) +STD # Standard redirection variable (silent/empty) + +# From install.func +APP # Application name +NSAPP # Normalized app name (lowercase, no spaces) +METHOD # Installation method (ct/install) +RANDOM_UUID # Session UUID for telemetry +``` + +--- + +## File Structure + +### Minimal install/AppName-install.sh Template + +```bash +#!/usr/bin/env bash # [1] Shebang + +# [2] Copyright/Metadata +# Copyright (c) 2021-2025 community-scripts ORG +# Author: YourUsername +# License: MIT +# Source: https://example.com + +# [3] Load functions +source /dev/stdin <<<"$FUNCTIONS_FILE_PATH" +color +verb_ip6 +catch_errors +setting_up_container +network_check +update_os + +# [4] Installation steps +msg_info "Installing Dependencies" +$STD apt-get install -y package1 package2 +msg_ok "Installed Dependencies" + +# [5] Final setup +motd_ssh +customize +cleanup_lxc +``` + +--- + +## Complete Script Template + +### Phase 1: Header & Initialization + +```bash +#!/usr/bin/env bash +# Copyright (c) 2021-2025 community-scripts ORG +# Author: YourUsername +# License: MIT | https://github.com/community-scripts/ProxmoxVED/raw/main/LICENSE +# Source: https://github.com/application/repo + +# Load all available functions (from core.func + tools.func) +source /dev/stdin <<<"$FUNCTIONS_FILE_PATH" + +# Initialize environment +color # Setup ANSI colors and icons +verb_ip6 # Configure IPv6 (if needed) +catch_errors # Setup error traps +setting_up_container # Verify OS is ready +network_check # Verify internet connectivity +update_os # Update packages (apk/apt) +``` + +### Phase 2: Dependency Installation + +```bash +msg_info "Installing Dependencies" +$STD apt-get install -y \ + curl \ + wget \ + git \ + nano \ + build-essential \ + libssl-dev \ + python3-dev +msg_ok "Installed Dependencies" +``` + +### Phase 3: Tool Setup (Using tools.func) + +```bash +# Setup specific tool versions +NODE_VERSION="22" setup_nodejs +PHP_VERSION="8.4" setup_php +PYTHON_VERSION="3.12" setup_uv +``` + +### Phase 4: Application Download & Setup + +```bash +# Download from GitHub +RELEASE=$(curl -fsSL https://api.github.com/repos/user/repo/releases/latest | \ + grep "tag_name" | awk '{print substr($2, 2, length($2)-3)}') + +wget -q "https://github.com/user/repo/releases/download/v${RELEASE}/app-${RELEASE}.tar.gz" +cd /opt +tar -xzf app-${RELEASE}.tar.gz +rm -f app-${RELEASE}.tar.gz +``` + +### Phase 5: Configuration Files + +```bash +# Using cat << EOF (multiline) +cat <<'EOF' >/etc/nginx/sites-available/appname +server { + listen 80; + server_name _; + root /opt/appname/public; + index index.php index.html; +} +EOF + +# Using sed for replacements +sed -i -e "s|^DB_HOST=.*|DB_HOST=localhost|" \ + -e "s|^DB_USER=.*|DB_USER=appuser|" \ + /opt/appname/.env +``` + +### Phase 6: Database Setup (If Needed) + +```bash +msg_info "Setting up Database" + +DB_NAME="appname_db" +DB_USER="appuser" +DB_PASS=$(openssl rand -base64 18 | tr -dc 'a-zA-Z0-9' | head -c13) + +# For MySQL/MariaDB +mysql -u root < /opt/${APP}_version.txt + +# Or with additional metadata +cat > /opt/${APP}_version.txt < /opt/${APP}_version.txt +``` + +6. **Handle Alpine vs Debian Differences** +```bash +# ✅ Good: Detect OS +if grep -qi 'alpine' /etc/os-release; then + apk add package +else + apt-get install -y package +fi +``` + +### ❌ DON'T: + +1. **Hardcode Versions** +```bash +# ❌ Bad: Won't auto-update +wget https://example.com/app-1.2.3.tar.gz +``` + +2. **Use Root Without Password** +```bash +# ❌ Bad: Security risk +mysql -u root +``` + +3. **Forget Error Handling** +```bash +# ❌ Bad: Silent failures +wget https://example.com/file +tar -xzf file +``` + +4. **Leave Temporary Files** +```bash +# ✅ Always cleanup +rm -rf /opt/app-${RELEASE}.tar.gz +``` + +--- + +## Real Examples + +### Example 1: Node.js Application + +```bash +#!/usr/bin/env bash +source /dev/stdin <<<"$FUNCTIONS_FILE_PATH" + +color +catch_errors +setting_up_container +network_check +update_os + +msg_info "Installing Node.js" +NODE_VERSION="22" setup_nodejs +msg_ok "Node.js installed" + +msg_info "Installing Application" +cd /opt +RELEASE=$(curl -fsSL https://api.github.com/repos/user/repo/releases/latest | \ + grep "tag_name" | awk '{print substr($2, 2, length($2)-3)}') +wget -q "https://github.com/user/repo/releases/download/v${RELEASE}/app.tar.gz" +tar -xzf app.tar.gz +echo "${RELEASE}" > /opt/app_version.txt +msg_ok "Application installed" + +systemctl enable --now app +cleanup_lxc +``` + +### Example 2: PHP Application with Database + +```bash +#!/usr/bin/env bash +source /dev/stdin <<<"$FUNCTIONS_FILE_PATH" + +color +catch_errors +setting_up_container +network_check +update_os + +PHP_VERSION="8.4" PHP_MODULE="bcmath,curl,pdo_mysql" setup_php +MARIADB_VERSION="11.4" setup_mariadb + +# Database setup +DB_PASS=$(openssl rand -base64 18 | tr -dc 'a-zA-Z0-9' | head -c13) +mysql -u root < /opt/app/.env <= 100 | -| 206 | CTID already in use | Check `pct list`, remove conflicting container | -| 207 | Password contains special characters | Use alphanumeric only for passwords | -| 208 | Invalid configuration format | Check DNS/MAC/Network format | -| 209 | Container creation failed | Check pct create output for details | -| 210 | Cluster not quorate | Ensure cluster nodes are online | -| 211 | Timeout waiting for template lock | Wait for concurrent downloads to finish | -| 214 | Not enough storage space | Free up disk space or expand storage | -| 215 | Container created but not listed | Check /etc/pve/lxc/ for config files | -| 216 | RootFS entry missing in config | Incomplete container creation | -| 217 | Storage does not support rootdir | Use compatible storage backend | -| 218 | Template corrupted or incomplete | Re-download template | -| 220 | Unable to resolve template path | Verify template availability | -| 221 | Template not readable | Fix file permissions | -| 222 | Template download failed (3 attempts) | Check network/storage | -| 223 | Template not available after download | Storage sync issue | -| 225 | No template for OS/Version | Run `pveam available` to see options | -| 231 | LXC stack upgrade/retry failed | Update pve-container package | - ---- - -## Telemetry Functions - -### `explain_exit_code()` - -**Purpose**: Maps numeric exit codes to human-readable error descriptions. Shared between api.func and error_handler.func for consistency. - -**Signature**: -```bash -explain_exit_code() -``` - -**Parameters**: -- `$1` - Numeric exit code (0-255) - -**Returns**: Human-readable description string - -**Supported Codes**: -- 1-2, 126-128, 130, 137, 139, 143 (Shell) -- 100-101, 255 (Package managers) -- 210-212 (Python) -- 231-234 (PostgreSQL) -- 241-244 (MySQL/MariaDB) -- 243-249, 254 (Node.js/npm) -- 251-254 (MongoDB) -- 200-231 (Proxmox custom) - -**Default**: Returns "Unknown error" for unmapped codes - -**Usage Examples**: - -```bash -# Example 1: Common error -explain_exit_code 127 -# Output: "Command not found" - -# Example 2: Database error -explain_exit_code 241 -# Output: "MySQL/MariaDB: Connection failed (server not running / wrong socket)" - -# Example 3: Custom Proxmox error -explain_exit_code 206 -# Output: "Custom: CTID already in use (check 'pct list' and /etc/pve/lxc/)" - -# Example 4: Unknown code -explain_exit_code 999 -# Output: "Unknown error" -``` - ---- - -### `post_to_api()` - -**Purpose**: Sends LXC container creation statistics to Community-Scripts telemetry API. - -**Signature**: -```bash -post_to_api() -``` - -**Parameters**: None (uses global environment variables) - -**Returns**: No explicit return value (curl result stored in RESPONSE if diagnostics enabled) - -**Requirements** (Silent fail if not met): -- `curl` command available -- `DIAGNOSTICS="yes"` -- `RANDOM_UUID` is set -- Executed on Proxmox host (has access to `pveversion`) - -**Environment Variables Used**: -- `CT_TYPE` - Container type (privileged=1, unprivileged=0) -- `DISK_SIZE` - Allocated disk in GB -- `CORE_COUNT` - CPU core count -- `RAM_SIZE` - RAM allocated in MB -- `var_os` - Operating system name -- `var_version` - OS version -- `NSAPP` - Normalized application name -- `METHOD` - Installation method (default, template, etc.) -- `DIAGNOSTICS` - Enable telemetry (yes/no) -- `RANDOM_UUID` - Session UUID for tracking - -**API Endpoint**: `http://api.community-scripts.org/dev/upload` - -**Payload Structure**: -```json -{ - "ct_type": 1, // Privileged (1) or Unprivileged (0) - "type": "lxc", // Always "lxc" for containers - "disk_size": 8, // GB - "core_count": 2, // CPU cores - "ram_size": 2048, // MB - "os_type": "debian", // OS name - "os_version": "12", // OS version - "nsapp": "myapp", // Application name - "method": "default", // Setup method - "pve_version": "8.2.2", // Proxmox VE version - "status": "installing", // Current status - "random_id": "550e8400-e29b" // Session UUID (anonymous) -} -``` - -**Usage Examples**: - -```bash -# Example 1: Successful API post -CT_TYPE=1 -DISK_SIZE=20 -CORE_COUNT=4 -RAM_SIZE=4096 -var_os="ubuntu" -var_version="22.04" -NSAPP="jellyfin" -METHOD="default" -DIAGNOSTICS="yes" -RANDOM_UUID="550e8400-e29b-41d4-a716-446655440000" - -post_to_api -# Result: Statistics sent to API (silently, no output) - -# Example 2: Diagnostics disabled (opt-out) -DIAGNOSTICS="no" -post_to_api -# Result: Function returns immediately, no API call - -# Example 3: Missing curl -DIAGNOSTICS="yes" -# curl not available in PATH -post_to_api -# Result: Function returns silently (curl requirement not met) -``` - ---- - -### `post_to_api_vm()` - -**Purpose**: Sends VM creation statistics to Community-Scripts API (similar to post_to_api but for virtual machines). - -**Signature**: -```bash -post_to_api_vm() -``` - -**Parameters**: None (uses global environment variables) - -**Returns**: No explicit return value - -**Requirements**: Same as `post_to_api()` - -**Environment Variables Used**: -- `VMID` - Virtual machine ID -- `VM_TYPE` - VM type (kvm, etc.) -- `VM_CORES` - CPU core count -- `VM_RAM` - RAM in MB -- `VM_DISK` - Disk in GB -- `VM_OS` - Operating system -- `VM_VERSION` - OS version -- `VM_APP` - Application name -- `DIAGNOSTICS` - Enable telemetry -- `RANDOM_UUID` - Session UUID - -**Payload Structure** (similar to containers but for VMs): -```json -{ - "vm_id": 100, - "type": "qemu", - "vm_cores": 4, - "vm_ram": 4096, - "vm_disk": 20, - "vm_os": "ubuntu", - "vm_version": "22.04", - "vm_app": "jellyfin", - "pve_version": "8.2.2", - "status": "installing", - "random_id": "550e8400-e29b" -} -``` - ---- - -### `post_update_to_api()` - -**Purpose**: Reports installation completion status (success/failure) for container or VM. - -**Signature**: -```bash -post_update_to_api() -``` - -**Parameters**: None (uses global environment variables) - -**Returns**: No explicit return value - -**Requirements**: Same as `post_to_api()` - -**Environment Variables Used**: -- `RANDOM_UUID` - Session UUID (must match initial post_to_api call) -- `DIAGNOSTICS` - Enable telemetry -- Installation status parameters - -**Payload Structure**: -```json -{ - "status": "completed", // "completed" or "failed" - "random_id": "550e8400-e29b", // Session UUID - "exit_code": 0, // 0 for success, error code for failure - "error_explanation": "" // Error description if failed -} -``` - ---- - -## API Payload Structure - -### Container Creation Payload - -```json -{ - "ct_type": 1, // 1=Privileged, 0=Unprivileged - "type": "lxc", // Always "lxc" - "disk_size": 20, // GB - "core_count": 4, // CPU cores - "ram_size": 4096, // MB - "os_type": "debian", // Distribution name - "os_version": "12", // Version number - "nsapp": "jellyfin", // Application name - "method": "default", // Setup method - "pve_version": "8.2.2", // Proxmox VE version - "status": "installing", // Current phase - "random_id": "550e8400" // Unique session ID -} -``` - -### VM Creation Payload - -```json -{ - "vm_id": 100, - "type": "qemu", - "vm_cores": 4, - "vm_ram": 4096, - "vm_disk": 20, - "vm_os": "ubuntu", - "vm_version": "22.04", - "vm_app": "jellyfin", - "pve_version": "8.2.2", - "status": "installing", - "random_id": "550e8400" -} -``` - -### Update/Completion Payload - -```json -{ - "status": "completed", - "random_id": "550e8400", - "exit_code": 0, - "error_explanation": "" -} -``` - ---- - -## Privacy & Opt-Out - -### Privacy Policy - -Community-Scripts telemetry is designed to be **privacy-respecting**: - -- ✅ **Anonymous**: No personal data collected -- ✅ **Session-based**: UUID allows correlation without identification -- ✅ **Aggregated**: Only statistics are stored, never raw logs -- ✅ **Opt-out capable**: Single environment variable disables all telemetry -- ✅ **No tracking**: UUID cannot be linked to user identity -- ✅ **No credentials**: Passwords, SSH keys never transmitted - -### Opt-Out Methods - -**Method 1: Environment Variable (Single Script)** - -```bash -DIAGNOSTICS="no" bash ct/myapp.sh -``` - -**Method 2: Script Header (Persistent)** - -```bash -#!/bin/bash -export DIAGNOSTICS="no" -# Rest of script continues without telemetry -``` - -**Method 3: System-wide Configuration** - -```bash -# In /etc/environment or ~/.bashrc -export DIAGNOSTICS="no" -``` - -### What Data Is Collected - -| Data | Why | Shared? | -|------|-----|---------| -| Container/VM specs (cores, RAM, disk) | Understand deployment patterns | Yes, aggregated | -| OS type/version | Track popular distributions | Yes, aggregated | -| Application name | Understand popular apps | Yes, aggregated | -| Method (standard vs. custom) | Measure feature usage | Yes, aggregated | -| Success/failure status | Identify issues | Yes, aggregated | -| Exit codes | Debug failures | Yes, anonymized | - -### What Data Is NOT Collected - -- ❌ Container/VM hostnames -- ❌ IP addresses -- ❌ User credentials -- ❌ SSH keys or secrets -- ❌ Application data -- ❌ System logs -- ❌ Any personal information - ---- - -## Error Mapping - -### Mapping Strategy - -Exit codes are categorized by source: - -``` -Exit Code Range | Source | Handling -0 | Success | Not reported to API -1-2 | Shell/Script | Generic error -100-101, 255 | Package managers | APT/DPKG specific -126-128 | Command execution | Permission/not found -130, 143 | Signals | User interrupt/termination -137, 139 | Kernel | OOM/segfault -200-231 | Proxmox custom | Container creation issues -210-212 | Python | Python environment issues -231-234 | PostgreSQL | Database connection issues -241-244 | MySQL/MariaDB | Database connection issues -243-249, 254 | Node.js/npm | Runtime errors -251-254 | MongoDB | Database connection issues -``` - -### Custom Exit Code Usage - -Scripts can define custom exit codes: - -```bash -# Example: Custom validation failure -if [[ "$CTID" -lt 100 ]]; then - echo "Container ID must be >= 100" - exit 205 # Custom Proxmox code -fi -``` - ---- - -## Best Practices - -### 1. **Always Initialize RANDOM_UUID** - -```bash -# Generate unique session ID for tracking -RANDOM_UUID="$(cat /proc/sys/kernel/random/uuid)" - -# Use first 8 chars for short session ID (logs) -SESSION_ID="${RANDOM_UUID:0:8}" -BUILD_LOG="/tmp/create-lxc-${SESSION_ID}.log" -``` - -### 2. **Call post_to_api Early** - -```bash -# Call post_to_api right after container creation starts -# This tracks attempt, even if installation fails - -variables() { - RANDOM_UUID="$(cat /proc/sys/kernel/random/uuid)" - # ... other variables ... -} - -# Later, in main script: -post_to_api # Report container creation started -# ... perform installation ... -post_update_to_api # Report completion -``` - -### 3. **Handle Graceful Failures** - -```bash -# Wrap API calls to handle network issues -if command -v curl &>/dev/null; then - post_to_api || true # Don't fail if API unavailable -else - msg_warn "curl not available, telemetry skipped" -fi -``` - -### 4. **Respect User Opt-Out** - -```bash -# Check DIAGNOSTICS early and skip all API calls if disabled -if [[ "${DIAGNOSTICS}" != "yes" ]]; then - msg_info "Anonymous diagnostics disabled" - return 0 # Skip telemetry -fi -``` - -### 5. **Maintain Session Consistency** - -```bash -# Use same RANDOM_UUID throughout lifecycle -RANDOM_UUID="$(cat /proc/sys/kernel/random/uuid)" - -post_to_api # Initial report -# ... installation ... -post_update_to_api # Final report (same UUID links them) -``` - ---- - -## API Integration - -### Connecting to API - -The API endpoint is: - -``` -http://api.community-scripts.org/dev/upload -``` - -### API Response Handling - -```bash -# Capture HTTP response code -RESPONSE=$(curl -s -w "%{http_code}" -L -X POST "$API_URL" \ - -H "Content-Type: application/json" \ - -d "$JSON_PAYLOAD") || true - -# Extract status code (last 3 digits) -HTTP_CODE="${RESPONSE: -3}" - -if [[ "$HTTP_CODE" == "200" ]]; then - msg_ok "Telemetry submitted successfully" -elif [[ "$HTTP_CODE" == "429" ]]; then - msg_warn "API rate limited, skipping telemetry" -else - msg_info "Telemetry API unreachable (this is OK)" -fi -``` - -### Network Resilience - -API calls are **best-effort** and never block installation: - -```bash -# Telemetry should never cause container creation to fail -if post_to_api 2>/dev/null; then - msg_info "Diagnostics transmitted" -fi -# If API unavailable, continue anyway -``` - ---- - -## Contributing - -### Adding New Exit Codes - -1. Document in the appropriate category section -2. Update `explain_exit_code()` in both api.func and error_handler.func -3. Add recovery suggestions -4. Test mapping with scripts that use the new code - -### Testing API Integration - -```bash -# Test with mock curl (local testing) -DIAGNOSTICS="yes" -RANDOM_UUID="test-uuid-12345678" -curl -X POST http://localhost:8000/dev/upload \ - -H "Content-Type: application/json" \ - -d '{"test": "payload"}' - -# Verify payload structure -post_to_api 2>&1 | head -20 -``` - -### Telemetry Reporting Improvements - -Suggestions for improvement: - -1. Installation duration tracking -2. Package version compatibility data -3. Feature usage analytics -4. Performance metrics -5. Custom error codes - ---- - -## Notes - -- API calls are **silent by default** and never display sensitive information -- Telemetry can be **completely disabled** via `DIAGNOSTICS="no"` -- **RANDOM_UUID must be generated** before calling any post functions -- Exit code mappings are **shared** between api.func and error_handler.func for consistency -- API is **optional** - containers work perfectly without telemetry - diff --git a/docs/DEFAULTS_SYSTEM_GUIDE.md b/docs/settings/DEFAULTS_SYSTEM_GUIDE.md similarity index 100% rename from docs/DEFAULTS_SYSTEM_GUIDE.md rename to docs/settings/DEFAULTS_SYSTEM_GUIDE.md diff --git a/docs/DEV_MODE.md b/docs/settings/DEV_MODE.md similarity index 100% rename from docs/DEV_MODE.md rename to docs/settings/DEV_MODE.md diff --git a/docs/EXIT_CODES.md b/docs/settings/EXIT_CODES.md similarity index 100% rename from docs/EXIT_CODES.md rename to docs/settings/EXIT_CODES.md diff --git a/docs/TECHNICAL_REFERENCE.md b/docs/settings/TECHNICAL_REFERENCE.md similarity index 100% rename from docs/TECHNICAL_REFERENCE.md rename to docs/settings/TECHNICAL_REFERENCE.md diff --git a/docs/tools.func.md b/docs/tools.func.md deleted file mode 100644 index 6a0b53734..000000000 --- a/docs/tools.func.md +++ /dev/null @@ -1,1283 +0,0 @@ -# Tools.func Wiki - -A comprehensive collection of helper functions for robust package management and repository management in Debian/Ubuntu-based systems. - ---- - -## 📋 Table of Contents - -- [Overview](#overview) -- [Core Helper Functions](#core-helper-functions) -- [Repository Management](#repository-management) -- [Package Management](#package-management) -- [Tool Installation Functions](#tool-installation-functions) -- [GitHub Integration](#github-integration) -- [System Utilities](#system-utilities) -- [Container Setup Functions](#container-setup-functions) - ---- - -## Overview - -This function library provides: - -- ✅ Automatic retry logic for APT/network failures -- ✅ Unified keyring cleanup from all 3 locations -- ✅ Legacy installation cleanup (nvm, rbenv, rustup) -- ✅ OS-upgrade-safe repository preparation -- ✅ Service pattern matching for multi-version tools - -### Usage in Install Scripts - -```bash -source /dev/stdin <<< "$FUNCTIONS" # Load from build.func -prepare_repository_setup "mysql" -install_packages_with_retry "mysql-server" "mysql-client" -``` - ---- - -## Core Helper Functions - -### `cache_installed_version()` - -**Purpose**: Caches installed version to avoid repeated checks. - -**Parameters**: -- `$1` - Application name -- `$2` - Version string - -**Example**: -```bash -cache_installed_version "nodejs" "22.0.0" -``` - ---- - -### `get_cached_version()` - -**Purpose**: Retrieves cached version of an application. - -**Parameters**: -- `$1` - Application name - -**Returns**: Version string or empty if not cached - -**Example**: -```bash -version=$(get_cached_version "nodejs") -``` - ---- - -### `cleanup_tool_keyrings()` - -**Purpose**: Removes ALL keyring files for specified tools from all 3 locations. - -**Parameters**: -- `$@` - Tool name patterns (supports wildcards) - -**Example**: -```bash -cleanup_tool_keyrings "mariadb" "mysql" "postgresql" -``` - ---- - -### `stop_all_services()` - -**Purpose**: Stops and disables all service instances matching a pattern. - -**Parameters**: -- `$@` - Service name patterns (supports wildcards) - -**Example**: -```bash -stop_all_services "php*-fpm" "mysql" "mariadb" -``` - ---- - -### `verify_tool_version()` - -**Purpose**: Verifies installed tool version matches expected version. - -**Parameters**: -- `$1` - Tool name -- `$2` - Expected version -- `$3` - Installed version - -**Returns**: 0 if match, 1 if mismatch - -**Example**: -```bash -verify_tool_version "nodejs" "22" "$(node -v | grep -oP '^v\K[0-9]+')" -``` - ---- - -### `cleanup_legacy_install()` - -**Purpose**: Removes legacy installation methods (nvm, rbenv, rustup, etc.). - -**Parameters**: -- `$1` - Tool name (nodejs, ruby, rust, go) - -**Example**: -```bash -cleanup_legacy_install "nodejs" # Removes nvm -``` - ---- - -## Repository Management - -### `prepare_repository_setup()` - -**Purpose**: Unified repository preparation before setup. Cleans up old repos, keyrings, and ensures APT is working. - -**Parameters**: -- `$@` - Repository names - -**Example**: -```bash -prepare_repository_setup "mariadb" "mysql" -``` - ---- - -### `manage_tool_repository()` - -**Purpose**: Unified repository management for tools. Handles adding, updating, and verifying tool repositories. - -**Parameters**: -- `$1` - Tool name (mariadb, mongodb, nodejs, postgresql, php, mysql) -- `$2` - Version -- `$3` - Repository URL -- `$4` - GPG key URL (optional) - -**Supported Tools**: mariadb, mongodb, nodejs, postgresql, php, mysql - -**Example**: -```bash -manage_tool_repository "mariadb" "11.4" \ - "http://mirror.mariadb.org/repo/11.4" \ - "https://mariadb.org/mariadb_release_signing_key.asc" -``` - ---- - -### `setup_deb822_repo()` - -**Purpose**: Standardized deb822 repository setup with optional architectures. Always runs apt update after repo creation. - -**Parameters**: -- `$1` - Repository name -- `$2` - GPG key URL -- `$3` - Repository URL -- `$4` - Suite -- `$5` - Component (default: main) -- `$6` - Architectures (optional) - -**Example**: -```bash -setup_deb822_repo "adoptium" \ - "https://packages.adoptium.net/artifactory/api/gpg/key/public" \ - "https://packages.adoptium.net/artifactory/deb" \ - "bookworm" \ - "main" -``` - ---- - -### `cleanup_old_repo_files()` - -**Purpose**: Cleanup old repository files (migration helper for OS upgrades). - -**Parameters**: -- `$1` - Application name - -**Example**: -```bash -cleanup_old_repo_files "mariadb" -``` - ---- - -### `cleanup_orphaned_sources()` - -**Purpose**: Cleanup orphaned .sources files that reference missing keyrings. Prevents APT signature verification errors. - -**Example**: -```bash -cleanup_orphaned_sources -``` - ---- - -### `ensure_apt_working()` - -**Purpose**: Ensures APT is in a working state before installing packages. - -**Returns**: 0 if APT is working, 1 if critically broken - -**Example**: -```bash -ensure_apt_working || return 1 -``` - ---- - -### `get_fallback_suite()` - -**Purpose**: Get fallback suite for repository with comprehensive mapping. - -**Parameters**: -- `$1` - Distribution ID (debian, ubuntu) -- `$2` - Distribution codename -- `$3` - Repository base URL - -**Returns**: Appropriate suite name - -**Example**: -```bash -suite=$(get_fallback_suite "debian" "trixie" "https://repo.example.com") -``` - ---- - -## Package Management - -### `install_packages_with_retry()` - -**Purpose**: Install packages with retry logic (3 attempts with APT refresh). - -**Parameters**: -- `$@` - Package names - -**Example**: -```bash -install_packages_with_retry "mysql-server" "mysql-client" -``` - ---- - -### `upgrade_packages_with_retry()` - -**Purpose**: Upgrade specific packages with retry logic. - -**Parameters**: -- `$@` - Package names - -**Example**: -```bash -upgrade_packages_with_retry "mariadb-server" "mariadb-client" -``` - ---- - -### `ensure_dependencies()` - -**Purpose**: Ensures dependencies are installed (with apt update caching). - -**Parameters**: -- `$@` - Dependency names - -**Example**: -```bash -ensure_dependencies "curl" "jq" "git" -``` - ---- - -### `is_package_installed()` - -**Purpose**: Check if package is installed (faster than dpkg -l | grep). - -**Parameters**: -- `$1` - Package name - -**Returns**: 0 if installed, 1 if not - -**Example**: -```bash -if is_package_installed "nginx"; then - echo "Nginx is installed" -fi -``` - ---- - -### `hold_package_version()` - -**Purpose**: Hold package version to prevent upgrades. - -**Parameters**: -- `$1` - Package name - -**Example**: -```bash -hold_package_version "mysql-server" -``` - ---- - -### `unhold_package_version()` - -**Purpose**: Unhold package version to allow upgrades. - -**Parameters**: -- `$1` - Package name - -**Example**: -```bash -unhold_package_version "mysql-server" -``` - ---- - -## Tool Installation Functions - -### `is_tool_installed()` - -**Purpose**: Check if tool is already installed and optionally verify exact version. - -**Parameters**: -- `$1` - Tool name -- `$2` - Required version (optional) - -**Returns**: 0 if installed (with optional version match), 1 if not installed - -**Supported Tools**: mariadb, mysql, mongodb, node, php, postgres, ruby, rust, go, clickhouse - -**Example**: -```bash -is_tool_installed "mariadb" "11.4" || echo "Not installed" -``` - ---- - -### `remove_old_tool_version()` - -**Purpose**: Remove old tool version completely (purge + cleanup repos). - -**Parameters**: -- `$1` - Tool name -- `$2` - Repository name (optional, defaults to tool name) - -**Example**: -```bash -remove_old_tool_version "mariadb" "repository-name" -``` - ---- - -### `should_update_tool()` - -**Purpose**: Determine if tool update/upgrade is needed. - -**Parameters**: -- `$1` - Tool name -- `$2` - Target version - -**Returns**: 0 (update needed), 1 (already up-to-date) - -**Example**: -```bash -if should_update_tool "mariadb" "11.4"; then - echo "Update needed" -fi -``` - ---- - -### `setup_mariadb()` - -**Purpose**: Installs or updates MariaDB from official repo. - -**Variables**: -- `MARIADB_VERSION` - MariaDB version to install (default: latest) - -**Example**: -```bash -MARIADB_VERSION="11.4" setup_mariadb -``` - ---- - -### `setup_mysql()` - -**Purpose**: Installs or upgrades MySQL and configures APT repo. - -**Variables**: -- `MYSQL_VERSION` - MySQL version to install (default: 8.0) - -**Features**: -- Handles Debian Trixie libaio1t64 transition -- Auto-fallback to MariaDB if MySQL 8.0 unavailable - -**Example**: -```bash -MYSQL_VERSION="8.0" setup_mysql -``` - ---- - -### `setup_mongodb()` - -**Purpose**: Installs or updates MongoDB to specified major version. - -**Variables**: -- `MONGO_VERSION` - MongoDB major version (default: 8.0) - -**Example**: -```bash -MONGO_VERSION="7.0" setup_mongodb -``` - ---- - -### `setup_postgresql()` - -**Purpose**: Installs or upgrades PostgreSQL and optional extensions. - -**Variables**: -- `PG_VERSION` - PostgreSQL major version (default: 16) -- `PG_MODULES` - Comma-separated list of extensions - -**Example**: -```bash -PG_VERSION="16" PG_MODULES="postgis,contrib" setup_postgresql -``` - ---- - -### `setup_nodejs()` - -**Purpose**: Installs Node.js and optional global modules. - -**Variables**: -- `NODE_VERSION` - Node.js version (default: 22) -- `NODE_MODULE` - Comma-separated list of global modules - -**Example**: -```bash -NODE_VERSION="22" NODE_MODULE="yarn,@vue/cli@5.0.0" setup_nodejs -``` - ---- - -### `setup_php()` - -**Purpose**: Installs PHP with selected modules and configures Apache/FPM support. - -**Variables**: -- `PHP_VERSION` - PHP version (default: 8.4) -- `PHP_MODULE` - Additional comma-separated modules -- `PHP_APACHE` - Set YES to enable PHP with Apache -- `PHP_FPM` - Set YES to enable PHP-FPM -- `PHP_MEMORY_LIMIT` - Memory limit (default: 512M) -- `PHP_UPLOAD_MAX_FILESIZE` - Upload max filesize (default: 128M) -- `PHP_POST_MAX_SIZE` - Post max size (default: 128M) -- `PHP_MAX_EXECUTION_TIME` - Max execution time (default: 300) - -**Example**: -```bash -PHP_VERSION="8.4" PHP_MODULE="redis,imagick" PHP_FPM="YES" setup_php -``` - ---- - -### `setup_java()` - -**Purpose**: Installs Temurin JDK via Adoptium APT repository. - -**Variables**: -- `JAVA_VERSION` - Temurin JDK version (default: 21) - -**Example**: -```bash -JAVA_VERSION="21" setup_java -``` - ---- - -### `setup_ruby()` - -**Purpose**: Installs rbenv and ruby-build, installs Ruby and optionally Rails. - -**Variables**: -- `RUBY_VERSION` - Ruby version (default: 3.4.4) -- `RUBY_INSTALL_RAILS` - true/false to install Rails (default: true) - -**Example**: -```bash -RUBY_VERSION="3.4.4" RUBY_INSTALL_RAILS="true" setup_ruby -``` - ---- - -### `setup_rust()` - -**Purpose**: Installs Rust toolchain and optional global crates. - -**Variables**: -- `RUST_TOOLCHAIN` - Rust toolchain (default: stable) -- `RUST_CRATES` - Comma-separated list of crates - -**Example**: -```bash -RUST_TOOLCHAIN="stable" RUST_CRATES="cargo-edit,wasm-pack@0.12.1" setup_rust -``` - ---- - -### `setup_go()` - -**Purpose**: Installs Go (Golang) from official tarball. - -**Variables**: -- `GO_VERSION` - Go version (default: latest) - -**Example**: -```bash -GO_VERSION="1.22.2" setup_go -``` - ---- - -### `setup_composer()` - -**Purpose**: Installs or updates Composer globally (robust, idempotent). - -**Features**: -- Installs to /usr/local/bin/composer -- Removes old binaries/symlinks -- Ensures /usr/local/bin is in PATH -- Auto-updates to latest version - -**Example**: -```bash -setup_composer -``` - ---- - -### `setup_uv()` - -**Purpose**: Installs or upgrades uv (Python package manager) from GitHub releases. - -**Variables**: -- `USE_UVX` - Set YES to install uvx wrapper (default: NO) -- `PYTHON_VERSION` - Optional Python version to install via uv - -**Example**: -```bash -USE_UVX="YES" PYTHON_VERSION="3.12" setup_uv -``` - ---- - -### `setup_yq()` - -**Purpose**: Installs or updates yq (mikefarah/yq - Go version). - -**Example**: -```bash -setup_yq -``` - ---- - -### `setup_ffmpeg()` - -**Purpose**: Installs FFmpeg from source or prebuilt binary. - -**Variables**: -- `FFMPEG_VERSION` - FFmpeg version (default: latest) -- `FFMPEG_TYPE` - Build profile: minimal, medium, full, binary (default: full) - -**Example**: -```bash -FFMPEG_VERSION="n7.1.1" FFMPEG_TYPE="full" setup_ffmpeg -``` - ---- - -### `setup_imagemagick()` - -**Purpose**: Installs ImageMagick 7 from source. - -**Example**: -```bash -setup_imagemagick -``` - ---- - -### `setup_gs()` - -**Purpose**: Installs or updates Ghostscript (gs) from source. - -**Example**: -```bash -setup_gs -``` - ---- - -### `setup_hwaccel()` - -**Purpose**: Sets up Hardware Acceleration for Intel/AMD/NVIDIA GPUs. - -**Example**: -```bash -setup_hwaccel -``` - ---- - -### `setup_clickhouse()` - -**Purpose**: Installs or upgrades ClickHouse database server. - -**Variables**: -- `CLICKHOUSE_VERSION` - ClickHouse version (default: latest) - -**Example**: -```bash -CLICKHOUSE_VERSION="latest" setup_clickhouse -``` - ---- - -### `setup_adminer()` - -**Purpose**: Installs Adminer (supports Debian/Ubuntu and Alpine). - -**Example**: -```bash -setup_adminer -``` - ---- - -## GitHub Integration - -### `check_for_gh_release()` - -**Purpose**: Checks for new GitHub release (latest tag). - -**Parameters**: -- `$1` - Application name -- `$2` - GitHub repository (user/repo) -- `$3` - Optional pinned version - -**Returns**: 0 if update available, 1 if up-to-date - -**Global Variables Set**: -- `CHECK_UPDATE_RELEASE` - Latest release tag - -**Example**: -```bash -if check_for_gh_release "flaresolverr" "FlareSolverr/FlareSolverr"; then - echo "Update available: $CHECK_UPDATE_RELEASE" -fi -``` - ---- - -### `fetch_and_deploy_gh_release()` - -**Purpose**: Downloads and deploys latest GitHub release. - -**Parameters**: -- `$1` - Application name -- `$2` - GitHub repository (user/repo) -- `$3` - Mode: tarball, binary, prebuild, singlefile (default: tarball) -- `$4` - Version (default: latest) -- `$5` - Target directory (default: /opt/app) -- `$6` - Asset filename/pattern (required for prebuild/singlefile) - -**Modes**: -- `tarball` - Source code tarball (.tar.gz) -- `binary` - .deb package install (arch-dependent) -- `prebuild` - Prebuilt .tar.gz archive -- `singlefile` - Standalone binary (chmod +x) - -**Example**: -```bash -# Source tarball -fetch_and_deploy_gh_release "myapp" "myuser/myapp" - -# Binary .deb -fetch_and_deploy_gh_release "myapp" "myuser/myapp" "binary" - -# Prebuilt archive -fetch_and_deploy_gh_release "hanko" "teamhanko/hanko" "prebuild" \ - "latest" "/opt/hanko" "hanko_Linux_x86_64.tar.gz" - -# Single binary -fetch_and_deploy_gh_release "argus" "release-argus/Argus" "singlefile" \ - "0.26.3" "/opt/argus" "Argus-.*linux-amd64" -``` - ---- - -### `github_api_call()` - -**Purpose**: GitHub API call with authentication and rate limit handling. - -**Parameters**: -- `$1` - API URL -- `$2` - Output file (default: /dev/stdout) - -**Environment Variables**: -- `GITHUB_TOKEN` - Optional GitHub token for higher rate limits - -**Example**: -```bash -github_api_call "https://api.github.com/repos/user/repo/releases/latest" "/tmp/release.json" -``` - ---- - -### `get_latest_github_release()` - -**Purpose**: Get latest GitHub release version. - -**Parameters**: -- `$1` - GitHub repository (user/repo) -- `$2` - Strip 'v' prefix (default: true) - -**Returns**: Version string - -**Example**: -```bash -version=$(get_latest_github_release "nodejs/node") -``` - ---- - -## System Utilities - -### `get_os_info()` - -**Purpose**: Get OS information (cached for performance). - -**Parameters**: -- `$1` - Field: id, codename, version, version_id, all (default: all) - -**Returns**: Requested OS information - -**Example**: -```bash -os_id=$(get_os_info id) -os_codename=$(get_os_info codename) -``` - ---- - -### `is_debian()`, `is_ubuntu()`, `is_alpine()` - -**Purpose**: Check if running on specific OS. - -**Returns**: 0 if match, 1 if not - -**Example**: -```bash -if is_debian; then - echo "Running on Debian" -fi -``` - ---- - -### `get_os_version_major()` - -**Purpose**: Get Debian/Ubuntu major version. - -**Returns**: Major version number - -**Example**: -```bash -major_version=$(get_os_version_major) -``` - ---- - -### `get_system_arch()` - -**Purpose**: Get system architecture (normalized). - -**Parameters**: -- `$1` - Architecture type: dpkg, uname, both (default: both) - -**Returns**: Architecture string (amd64, arm64) - -**Example**: -```bash -arch=$(get_system_arch) -``` - ---- - -### `version_gt()` - -**Purpose**: Smart version comparison. - -**Parameters**: -- `$1` - Version 1 -- `$2` - Version 2 - -**Returns**: 0 if version 1 > version 2 - -**Example**: -```bash -if version_gt "2.0.0" "1.5.0"; then - echo "Version 2.0.0 is greater" -fi -``` - ---- - -### `is_lts_version()` - -**Purpose**: Check if running on LTS version. - -**Returns**: 0 if LTS, 1 if not - -**Example**: -```bash -if is_lts_version; then - echo "Running on LTS" -fi -``` - ---- - -### `get_parallel_jobs()` - -**Purpose**: Get optimal number of parallel jobs (cached). - -**Returns**: Number of parallel jobs based on CPU and memory - -**Example**: -```bash -jobs=$(get_parallel_jobs) -make -j"$jobs" -``` - ---- - -### `is_apt_locked()` - -**Purpose**: Check if package manager is locked. - -**Returns**: 0 if locked, 1 if not - -**Example**: -```bash -if is_apt_locked; then - echo "APT is locked" -fi -``` - ---- - -### `wait_for_apt()` - -**Purpose**: Wait for apt to be available. - -**Parameters**: -- `$1` - Max wait time in seconds (default: 300) - -**Example**: -```bash -wait_for_apt 600 # Wait up to 10 minutes -``` - ---- - -### `download_file()` - -**Purpose**: Download file with retry logic and progress. - -**Parameters**: -- `$1` - URL -- `$2` - Output path -- `$3` - Max retries (default: 3) -- `$4` - Show progress (default: false) - -**Example**: -```bash -download_file "https://example.com/file.tar.gz" "/tmp/file.tar.gz" 3 true -``` - ---- - -### `create_temp_dir()` - -**Purpose**: Create temporary directory with automatic cleanup. - -**Returns**: Temporary directory path - -**Example**: -```bash -tmp_dir=$(create_temp_dir) -# Directory is automatically cleaned up on exit -``` - ---- - -### `safe_service_restart()` - -**Purpose**: Safe service restart with verification. - -**Parameters**: -- `$1` - Service name - -**Example**: -```bash -safe_service_restart "nginx" -``` - ---- - -### `enable_and_start_service()` - -**Purpose**: Enable and start service (with error handling). - -**Parameters**: -- `$1` - Service name - -**Example**: -```bash -enable_and_start_service "postgresql" -``` - ---- - -### `is_service_enabled()`, `is_service_running()` - -**Purpose**: Check if service is enabled/running. - -**Parameters**: -- `$1` - Service name - -**Returns**: 0 if yes, 1 if no - -**Example**: -```bash -if is_service_running "nginx"; then - echo "Nginx is running" -fi -``` - ---- - -### `create_self_signed_cert()` - -**Purpose**: Creates and installs self-signed certificates. - -**Parameters**: -- `$1` - Application name (optional, defaults to $APPLICATION) - -**Example**: -```bash -create_self_signed_cert "myapp" -``` - ---- - -### `import_local_ip()` - -**Purpose**: Loads LOCAL_IP from persistent store or detects if missing. - -**Global Variables Set**: -- `LOCAL_IP` - Local IP address - -**Example**: -```bash -import_local_ip -echo "Local IP: $LOCAL_IP" -``` - ---- - -### `setup_local_ip_helper()` - -**Purpose**: Installs a local IP updater script using networkd-dispatcher. - -**Example**: -```bash -setup_local_ip_helper -``` - ---- - -### `ensure_usr_local_bin_persist()` - -**Purpose**: Ensures /usr/local/bin is permanently in system PATH. - -**Example**: -```bash -ensure_usr_local_bin_persist -``` - ---- - -### `download_with_progress()` - -**Purpose**: Downloads file with optional progress indicator using pv. - -**Parameters**: -- `$1` - URL -- `$2` - Destination path - -**Example**: -```bash -download_with_progress "https://example.com/file.tar.gz" "/tmp/file.tar.gz" -``` - ---- - -### `verify_gpg_fingerprint()` - -**Purpose**: GPG key fingerprint verification. - -**Parameters**: -- `$1` - Key file path -- `$2` - Expected fingerprint - -**Example**: -```bash -verify_gpg_fingerprint "/tmp/key.gpg" "ABCD1234..." -``` - ---- - -### `debug_log()` - -**Purpose**: Debug logging (only if DEBUG=1). - -**Parameters**: -- `$@` - Message to log - -**Example**: -```bash -DEBUG=1 debug_log "This is a debug message" -``` - ---- - -### `start_timer()`, `end_timer()` - -**Purpose**: Performance timing helpers. - -**Example**: -```bash -start_time=$(start_timer) -# ... do something ... -end_timer "$start_time" "Operation" -``` - ---- - -## Container Setup Functions - -### `color()` - -**Purpose**: Sets up color and formatting variables for terminal output. - -**Example**: -```bash -color -echo -e "${GN}Success${CL}" -``` - ---- - -### `verb_ip6()` - -**Purpose**: Enables or disables IPv6 based on DISABLEIPV6 variable. - -**Variables**: -- `DISABLEIPV6` - Set "yes" to disable IPv6 - -**Example**: -```bash -DISABLEIPV6="yes" verb_ip6 -``` - ---- - -### `catch_errors()` - -**Purpose**: Sets up error handling for the script. - -**Example**: -```bash -catch_errors -``` - ---- - -### `error_handler()` - -**Purpose**: Handles errors that occur during script execution. - -**Parameters**: -- `$1` - Line number -- `$2` - Command that failed - -**Example**: -```bash -error_handler 42 "ls non_existent_file" -``` - ---- - -### `spinner()` - -**Purpose**: Displays a rotating spinner animation. - -**Example**: -```bash -spinner & -SPINNER_PID=$! -``` - ---- - -### `msg_info()`, `msg_ok()`, `msg_error()` - -**Purpose**: Display messages with different statuses. - -**Parameters**: -- `$1` - Message text - -**Example**: -```bash -msg_info "Installing packages..." -msg_ok "Installation complete" -msg_error "Installation failed" -``` - ---- - -### `setting_up_container()` - -**Purpose**: Sets up container OS, configures locale, timezone, and network. - -**Example**: -```bash -setting_up_container -``` - ---- - -### `network_check()` - -**Purpose**: Verifies internet connectivity via IPv4 and IPv6. - -**Example**: -```bash -network_check -``` - ---- - -### `update_os()` - -**Purpose**: Updates the container's OS using apt-get. - -**Variables**: -- `CACHER` - Enable package caching proxy - -**Example**: -```bash -update_os -``` - ---- - -### `motd_ssh()` - -**Purpose**: Modifies message of the day (MOTD) and SSH settings. - -**Example**: -```bash -motd_ssh -``` - ---- - -### `customize()` - -**Purpose**: Customizes the container by enabling auto-login and setting up SSH keys. - -**Example**: -```bash -customize -``` - ---- - -## Best Practices - -### Version Management - -Always cache versions after installation: -```bash -setup_nodejs -cache_installed_version "nodejs" "$NODE_VERSION" -``` - -### Error Handling - -Always check return codes: -```bash -if ! install_packages_with_retry "nginx"; then - msg_error "Failed to install nginx" - return 1 -fi -``` - -### Repository Setup - -Always prepare repositories before installation: -```bash -prepare_repository_setup "mariadb" || return 1 -manage_tool_repository "mariadb" "11.4" "$REPO_URL" "$GPG_URL" || return 1 -``` - -### APT Safety - -Always ensure APT is working before operations: -```bash -ensure_apt_working || return 1 -install_packages_with_retry "package-name" -``` - ---- - -## Notes - -- All functions use `$STD` variable for silent execution -- Functions support both fresh installs and upgrades -- Automatic fallback mechanisms for newer OS versions -- Version caching prevents redundant installations -- Comprehensive error handling and retry logic - ---- - -## License - -This documentation is part of the community-scripts project. - ---- - -## Contributing - -Contributions are welcome! Please follow the existing code style and add appropriate documentation for new functions.