diff --git a/docs/APP-ct.md b/docs/APP-ct.md new file mode 100644 index 000000000..aa6793f29 --- /dev/null +++ b/docs/APP-ct.md @@ -0,0 +1,292 @@ +# **AppName.sh Scripts** + + `AppName.sh` scripts found in the `/ct` directory. These scripts are responsible for the installation of the desired application. For this guide we take `/ct/snipeit.sh` as example. + +## Table of Contents + +- [**AppName.sh Scripts**](#appnamesh-scripts) + - [Table of Contents](#table-of-contents) + - [1. **File Header**](#1-file-header) + - [1.1 **Shebang**](#11-shebang) + - [1.2 **Import Functions**](#12-import-functions) + - [1.3 **Metadata**](#13-metadata) + - [2 **Variables and function import**](#2-variables-and-function-import) + - [2.1 **Default Values**](#21-default-values) + - [2.2 **πŸ“‹ App output \& base settings**](#22--app-output--base-settings) + - [2.3 **πŸ›  Core functions**](#23--core-functions) + - [3 **Update function**](#3-update-function) + - [3.1 **Function Header**](#31-function-header) + - [3.2 **Check APP**](#32-check-app) + - [3.3 **Check version**](#33-check-version) + - [3.4 **Verbosity**](#34-verbosity) + - [3.5 **Backups**](#35-backups) + - [3.6 **Cleanup**](#36-cleanup) + - [3.7 **No update function**](#37-no-update-function) + - [4 **End of the script**](#4-end-of-the-script) + - [5. **Contribution checklist**](#5-contribution-checklist) + +## 1. **File Header** + +### 1.1 **Shebang** + +- Use `#!/usr/bin/env bash` as the shebang. + +```bash +#!/usr/bin/env bash +``` + +### 1.2 **Import Functions** + +- Import the build.func file. +- When developing your own script, change the URL to your own repository. + +> [!IMPORTANT] +> You also need to change all apperances of this URL in `misc/build.func` and `misc/install.func` + + +Example for development: + +```bash +source <(curl -s https://raw.githubusercontent.com/[USER]/[REPO]/refs/heads/[BRANCH]/misc/build.func) +``` + +Final script: + +```bash +source <(curl -s https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/misc/build.func) +``` + +> [!CAUTION] +> Before opening a Pull Request, change the URL to point to the community-scripts repo. + +### 1.3 **Metadata** + +- Add clear comments for script metadata, including author, copyright, and license information. + +Example: + +```bash +# Copyright (c) 2021-2025 community-scripts ORG +# Author: [YourUserName] +# License: MIT | https://github.com/community-scripts/ProxmoxVE/raw/main/LICENSE +# Source: [SOURCE_URL] +``` + +> [!NOTE]: +> +> - Add your username and source URL +> - For existing scripts, add "| Co-Author [YourUserName]" after the current author + +--- + +## 2 **Variables and function import** +> +> [!NOTE] +> You need to have all this set in your script, otherwise it will not work! + +### 2.1 **Default Values** + +- This section sets the default values for the container. +- `APP` needs to be set to the application name and must be equal to the filenames of your scripts. +- `var_tags`: You can set Tags for the CT wich show up in the Proxmox UI. DonΒ΄t overdo it! + +>[!NOTE] +>Description for all Default Values +> +>| Variable | Description | Notes | +>|----------|-------------|-------| +>| `APP` | Application name | Must match ct\AppName.sh | +>| `TAGS` | Proxmox display tags without Spaces, only ; | Limit the number | +>| `var_cpu` | CPU cores | Number of cores | +>| `var_ram` | RAM | In MB | +>| `var_disk` | Disk capacity | In GB | +>| `var_os` | Operating system | alpine, debian, ubuntu | +>| `var_version` | OS version | e.g., 3.20, 11, 12, 20.04 | +>| `var_unprivileged` | Container type | 1 = Unprivileged, 0 = Privileged | + +Example: + +```bash +APP="SnipeIT" +var_tags="asset-management;foss" +var_cpu="2" +var_ram="2048" +var_disk="4" +var_os="debian" +var_version="12" +var_unprivileged="1" +``` + +## 2.2 **πŸ“‹ App output & base settings** + +```bash +# App Output & Base Settings +header_info "$APP" +base_settings +``` + +- `header_info`: Generates ASCII header for APP +- `base_settings`: Allows overwriting variable values + +## 2.3 **πŸ›  Core functions** + +```bash +# Core +variables +color +catch_errors +``` + +- `variables`: Processes input and prepares variables +- `color`: Sets icons, colors, and formatting +- `catch_errors`: Enables error handling + +--- + +## 3 **Update function** + +### 3.1 **Function Header** + +- If applicable write a function that updates the application and the OS in the container. +- Each update function starts with the same code: + +```bash +function update_script() { + header_info + check_container_storage + check_container_resources +``` + +### 3.2 **Check APP** + +- Before doing anything update-wise, check if the app is installed in the container. + +Example: + +```bash +if [[ ! -d /opt/snipe-it ]]; then + msg_error "No ${APP} Installation Found!" + exit + fi +``` + +### 3.3 **Check version** + +- Before updating, check if a new version exists. + - We use the `${APPLICATION}_version.txt` file created in `/opt` during the install to compare new versions against the currently installed version. + +Example with a Github Release: + +```bash + RELEASE=$(curl -fsSL https://api.github.com/repos/snipe/snipe-it/releases/latest | grep "tag_name" | awk '{print substr($2, 3, length($2)-4) }') + if [[ ! -f /opt/${APP}_version.txt ]] || [[ "${RELEASE}" != "$(cat /opt/${APP}_version.txt)" ]]; then + msg_info "Updating ${APP} to v${RELEASE}" + #DO UPDATE + else + msg_ok "No update required. ${APP} is already at v${RELEASE}." + fi + exit +} +``` + +### 3.4 **Verbosity** + +- Use the appropriate flag (**-q** in the examples) for a command to suppress its output. +Example: + +```bash +wget -q +unzip -q +``` + +- If a command does not come with this functionality use `&>/dev/null` to suppress it's output. + +Example: + +```bash +php artisan migrate --force &>/dev/null +php artisan config:clear &>/dev/null +``` + +### 3.5 **Backups** + +- Backup user data if necessary. +- Move all user data back in the directory when the update is finished. + +>[!NOTE] +>This is not meant to be a permanent backup + +Example backup: + +```bash + mv /opt/snipe-it /opt/snipe-it-backup +``` + +Example config restore: + +```bash + cp /opt/snipe-it-backup/.env /opt/snipe-it/.env + cp -r /opt/snipe-it-backup/public/uploads/ /opt/snipe-it/public/uploads/ + cp -r /opt/snipe-it-backup/storage/private_uploads /opt/snipe-it/storage/private_uploads +``` + +### 3.6 **Cleanup** + +- Do not forget to remove any temporary files/folders such as zip-files or temporary backups. +Example: + +```bash + rm -rf /opt/v${RELEASE}.zip + rm -rf /opt/snipe-it-backup +``` + +### 3.7 **No update function** + +- In case you can not provide a update function use the following code to provide user feedback. + +```bash +function update_script() { + header_info + check_container_storage + check_container_resources + if [[ ! -d /opt/snipeit ]]; then + msg_error "No ${APP} Installation Found!" + exit + fi + msg_error "There is currently no automatic update function for ${APP}." + exit +} +``` + +--- + +## 4 **End of the script** + +- `start`: Launches Whiptail dialogue +- `build_container`: Collects and integrates user settings +- `description`: Sets LXC container description +- With `echo -e "${TAB}${GATEWAY}${BGN}http://${IP}${CL}"` you can point the user to the IP:PORT/folder needed to access the app. + +```bash +start +build_container +description + +msg_ok "Completed Successfully!\n" +echo -e "${CREATING}${GN}${APP} setup has been successfully initialized!${CL}" +echo -e "${INFO}${YW} Access it using the following URL:${CL}" +echo -e "${TAB}${GATEWAY}${BGN}http://${IP}${CL}" +``` + +--- + +## 5. **Contribution checklist** + +- [ ] Shebang is correctly set (`#!/usr/bin/env bash`). +- [ ] Correct link to *build.func* +- [ ] Metadata (author, license) is included at the top. +- [ ] Variables follow naming conventions. +- [ ] Update function exists. +- [ ] Update functions checks if app is installed an for new version. +- [ ] Update function up temporary files. +- [ ] Script ends with a helpful message for the user to reach the application. diff --git a/docs/APP-install.md b/docs/APP-install.md new file mode 100644 index 000000000..8a9c7e65a --- /dev/null +++ b/docs/APP-install.md @@ -0,0 +1,353 @@ + +# **AppName-install.sh Scripts** + + `AppName-install.sh` scripts found in the `/install` directory. These scripts are responsible for the installation of the application. For this guide we take `/install/snipeit-install.sh` as example. + +## Table of Contents + +- [**AppName-install.sh Scripts**](#appname-installsh-scripts) + - [Table of Contents](#table-of-contents) + - [1. **File header**](#1-file-header) + - [1.1 **Shebang**](#11-shebang) + - [1.2 **Comments**](#12-comments) + - [1.3 **Variables and function import**](#13-variables-and-function-import) + - [2. **Variable naming and management**](#2-variable-naming-and-management) + - [2.1 **Naming conventions**](#21-naming-conventions) + - [3. **Dependencies**](#3-dependencies) + - [3.1 **Install all at once**](#31-install-all-at-once) + - [3.2 **Collapse dependencies**](#32-collapse-dependencies) + - [4. **Paths to application files**](#4-paths-to-application-files) + - [5. **Version management**](#5-version-management) + - [5.1 **Install the latest release**](#51-install-the-latest-release) + - [5.2 **Save the version for update checks**](#52-save-the-version-for-update-checks) + - [6. **Input and output management**](#6-input-and-output-management) + - [6.1 **User feedback**](#61-user-feedback) + - [6.2 **Verbosity**](#62-verbosity) + - [7. **String/File Manipulation**](#7-stringfile-manipulation) + - [7.1 **File Manipulation**](#71-file-manipulation) + - [8. **Security practices**](#8-security-practices) + - [8.1 **Password generation**](#81-password-generation) + - [8.2 **File permissions**](#82-file-permissions) + - [9. **Service Configuration**](#9-service-configuration) + - [9.1 **Configuration files**](#91-configuration-files) + - [9.2 **Credential management**](#92-credential-management) + - [9.3 **Enviroment files**](#93-enviroment-files) + - [9.4 **Services**](#94-services) + - [10. **Cleanup**](#10-cleanup) + - [10.1 **Remove temporary files**](#101-remove-temporary-files) + - [10.2 **Autoremove and autoclean**](#102-autoremove-and-autoclean) + - [11. **Best Practices Checklist**](#11-best-practices-checklist) + - [Example: High-Level Script Flow](#example-high-level-script-flow) + +## 1. **File header** + +### 1.1 **Shebang** + +- Use `#!/usr/bin/env bash` as the shebang. + +```bash +#!/usr/bin/env bash +``` + +### 1.2 **Comments** + +- Add clear comments for script metadata, including author, copyright, and license information. +- Use meaningful inline comments to explain complex commands or logic. + +Example: + +```bash +# Copyright (c) 2021-2025 community-scripts ORG +# Author: [YourUserName] +# License: MIT | https://github.com/community-scripts/ProxmoxVE/raw/main/LICENSE +# Source: [SOURCE_URL] +``` + +> [!NOTE]: +> +> - Add your username +> - When updating/reworking scripts, add "| Co-Author [YourUserName]" + +### 1.3 **Variables and function import** + +- This sections adds the support for all needed functions and variables. + +```bash +source /dev/stdin <<<"$FUNCTIONS_FILE_PATH" +color +verb_ip6 +catch_errors +setting_up_container +network_check +update_os +``` + +--- + +## 2. **Variable naming and management** + +### 2.1 **Naming conventions** + +- Use uppercase names for constants and environment variables. +- Use lowercase names for local script variables. + +Example: + +```bash +DB_NAME=snipeit_db # Environment-like variable (constant) +db_user="snipeit" # Local variable +``` + +--- + +## 3. **Dependencies** + +### 3.1 **Install all at once** + +- Install all dependencies with a single command if possible + +Example: + +```bash +$STD apt-get install -y \ + curl \ + composer \ + git \ + sudo \ + mc \ + nginx +``` + +### 3.2 **Collapse dependencies** + +Collapse dependencies to keep the code readable. + +Example: +Use + +```bash +php8.2-{bcmath,common,ctype} +``` + +instead of + +```bash +php8.2-bcmath php8.2-common php8.2-ctype +``` + +--- + +## 4. **Paths to application files** + +If possible install the app and all necessary files in `/opt/` + +--- + +## 5. **Version management** + +### 5.1 **Install the latest release** + +- Always try and install the latest release +- Do not hardcode any version if not absolutely necessary + +Example for a git release: + +```bash +RELEASE=$(curl -fsSL https://api.github.com/repos/snipe/snipe-it/releases/latest | grep "tag_name" | awk '{print substr($2, 3, length($2)-4) }') +wget -q "https://github.com/snipe/snipe-it/archive/refs/tags/v${RELEASE}.zip" +``` + +### 5.2 **Save the version for update checks** + +- Write the installed version into a file. +- This is used for the update function in **AppName.sh** to check for if a Update is needed. + +Example: + +```bash +echo "${RELEASE}" >"/opt/AppName_version.txt" +``` + +--- + +## 6. **Input and output management** + +### 6.1 **User feedback** + +- Use standard functions like `msg_info`, `msg_ok` or `msg_error` to print status messages. +- Each `msg_info` must be followed with a `msg_ok` before any other output is made. +- Display meaningful progress messages at key stages. + +Example: + +```bash +msg_info "Installing Dependencies" +$STD apt-get install -y ... +msg_ok "Installed Dependencies" +``` + +### 6.2 **Verbosity** + +- Use the appropiate flag (**-q** in the examples) for a command to suppres its output +Example: + +```bash +wget -q +unzip -q +``` + +- If a command dose not come with such a functionality use `$STD` (a custom standard redirection variable) for managing output verbosity. + +Example: + +```bash +$STD apt-get install -y nginx +``` + +--- + +## 7. **String/File Manipulation** + +### 7.1 **File Manipulation** + +- Use `sed` to replace placeholder values in configuration files. + +Example: + +```bash +sed -i -e "s|^DB_DATABASE=.*|DB_DATABASE=$DB_NAME|" \ + -e "s|^DB_USERNAME=.*|DB_USERNAME=$DB_USER|" \ + -e "s|^DB_PASSWORD=.*|DB_PASSWORD=$DB_PASS|" .env +``` + +--- + +## 8. **Security practices** + +### 8.1 **Password generation** + +- Use `openssl` to generate random passwords. +- Use only alphanumeric values to not introduce unknown behaviour. + +Example: + +```bash +DB_PASS=$(openssl rand -base64 18 | tr -dc 'a-zA-Z0-9' | head -c13) +``` + +### 8.2 **File permissions** + +Explicitly set secure ownership and permissions for sensitive files. + +Example: + +```bash +chown -R www-data: /opt/snipe-it +chmod -R 755 /opt/snipe-it +``` + +--- + +## 9. **Service Configuration** + +### 9.1 **Configuration files** + +Use `cat </etc/nginx/conf.d/snipeit.conf +server { + listen 80; + root /opt/snipe-it/public; + index index.php; +} +EOF +``` + +### 9.2 **Credential management** + +Store the generated credentials in a file. + +Example: + +```bash +USERNAME=username +PASSWORD=$(openssl rand -base64 18 | tr -dc 'a-zA-Z0-9' | head -c13) +{ + echo "Application-Credentials" + echo "Username: $USERNAME" + echo "Password: $PASSWORD" +} >> ~/application.creds +``` + +### 9.3 **Enviroment files** + +Use `cat </path/to/.env +VARIABLE="value" +PORT=3000 +DB_NAME="${DB_NAME}" +EOF +``` + +### 9.4 **Services** + +Enable affected services after configuration changes and start them right away. + +Example: + +```bash +systemctl enable -q --now nginx +``` + +--- + +## 10. **Cleanup** + +### 10.1 **Remove temporary files** + +Remove temporary files and downloads after use. + +Example: + +```bash +rm -rf /opt/v${RELEASE}.zip +``` + +### 10.2 **Autoremove and autoclean** + +Remove unused dependencies to reduce disk space usage. + +Example: + +```bash +apt-get -y autoremove +apt-get -y autoclean +``` + +--- + +## 11. **Best Practices Checklist** + +- [ ] Shebang is correctly set (`#!/usr/bin/env bash`). +- [ ] Metadata (author, license) is included at the top. +- [ ] Variables follow naming conventions. +- [ ] Sensitive values are dynamically generated. +- [ ] Files and services have proper permissions. +- [ ] Script cleans up temporary files. + +--- + +### Example: High-Level Script Flow + +1. Dependencies installation +2. Database setup +3. Download and configure application +4. Service configuration +5. Final cleanup diff --git a/docs/CONTRIBUTION_GUIDE.md b/docs/CONTRIBUTION_GUIDE.md new file mode 100644 index 000000000..c41d017fe --- /dev/null +++ b/docs/CONTRIBUTION_GUIDE.md @@ -0,0 +1,1037 @@ +# 🎯 **ProxmoxVED Contribution Guide** + +**Everything you need to know to contribute to ProxmoxVED** + +> **Last Updated**: December 2025 +> **Difficulty**: Beginner β†’ Advanced +> **Time to Setup**: 15 minutes +> **Time to Contribute**: 1-3 hours + +--- + +## πŸ“‹ Table of Contents + +- [Quick Start](#quick-start) +- [Repository Structure](#repository-structure) +- [Development Setup](#development-setup) +- [Creating New Applications](#creating-new-applications) +- [Updating Existing Applications](#updating-existing-applications) +- [Code Standards](#code-standards) +- [Testing Your Changes](#testing-your-changes) +- [Submitting a Pull Request](#submitting-a-pull-request) +- [Troubleshooting](#troubleshooting) +- [FAQ](#faq) + +--- + +## Quick Start + +### 60 Seconds to First Contribution + +```bash +# 1. Fork the repository on GitHub +# Visit: https://github.com/community-scripts/ProxmoxVED +# Click: Fork (top right) + +# 2. Clone your fork +git clone https://github.com/YOUR_USERNAME/ProxmoxVED.git +cd ProxmoxVED + +# 3. Create feature branch +git checkout -b add/my-awesome-app + +# 4. Create application scripts +cp ct/example.sh ct/myapp.sh +cp install/example-install.sh install/myapp-install.sh + +# 5. Edit your scripts +nano ct/myapp.sh +nano install/myapp-install.sh + +# 6. Test locally +bash ct/myapp.sh # Will prompt for container creation + +# 7. Commit and push +git add ct/myapp.sh install/myapp-install.sh +git commit -m "feat: add MyApp container" +git push origin add/my-awesome-app + +# 8. Open Pull Request on GitHub +# Visit: https://github.com/community-scripts/ProxmoxVED/pulls +# Click: New Pull Request +``` + +--- + +## Repository Structure + +### Top-Level Organization + +``` +ProxmoxVED/ +β”œβ”€β”€ ct/ # πŸ—οΈ Container creation scripts (host-side) +β”‚ β”œβ”€β”€ pihole.sh +β”‚ β”œβ”€β”€ docker.sh +β”‚ └── ... (40+ applications) +β”‚ +β”œβ”€β”€ install/ # πŸ› οΈ Installation scripts (container-side) +β”‚ β”œβ”€β”€ pihole-install.sh +β”‚ β”œβ”€β”€ docker-install.sh +β”‚ └── ... (40+ applications) +β”‚ +β”œβ”€β”€ vm/ # πŸ’Ύ VM creation scripts +β”‚ β”œβ”€β”€ ubuntu2404-vm.sh +β”‚ β”œβ”€β”€ debian-vm.sh +β”‚ └── ... (15+ operating systems) +β”‚ +β”œβ”€β”€ misc/ # πŸ“¦ Shared function libraries +β”‚ β”œβ”€β”€ build.func # Main orchestrator (3800+ lines) +β”‚ β”œβ”€β”€ core.func # UI/utilities +β”‚ β”œβ”€β”€ error_handler.func # Error management +β”‚ β”œβ”€β”€ tools.func # Tool installation +β”‚ β”œβ”€β”€ install.func # Container setup +β”‚ β”œβ”€β”€ cloud-init.func # VM configuration +β”‚ β”œβ”€β”€ api.func # Telemetry +β”‚ β”œβ”€β”€ alpine-install.func # Alpine-specific +β”‚ └── alpine-tools.func # Alpine tools +β”‚ +β”œβ”€β”€ docs/ # πŸ“š Documentation +β”‚ β”œβ”€β”€ UPDATED_APP-ct.md # Container script guide +β”‚ β”œβ”€β”€ UPDATED_APP-install.md # Install script guide +β”‚ └── CONTRIBUTING.md # (This file!) +β”‚ +β”œβ”€β”€ tools/ # πŸ”§ Proxmox management tools +β”‚ └── pve/ +β”‚ +└── README.md # Project overview +``` + +### Naming Conventions + +``` +Container Script: ct/AppName.sh +Installation Script: install/appname-install.sh +Defaults: defaults/appname.vars +Update Script: /usr/bin/update (inside container) + +Examples: + ct/pihole.sh β†’ install/pihole-install.sh + ct/docker.sh β†’ install/docker-install.sh + ct/nextcloud-vm.sh β†’ install/nextcloud-vm-install.sh +``` + +**Rules**: +- Container script name: **Title Case** (PiHole, Docker, NextCloud) +- Install script name: **lowercase** with **hyphens** (pihole-install, docker-install) +- Must match: `ct/AppName.sh` ↔ `install/appname-install.sh` +- Directory names: lowercase (always) +- Variable names: lowercase (except APP constant) + +--- + +## Development Setup + +### Prerequisites + +1. **Proxmox VE 8.0+** with at least: + - 4 CPU cores + - 8 GB RAM + - 50 GB disk space + - Ubuntu 20.04 / Debian 11+ on host + +2. **Git** installed + ```bash + apt-get install -y git + ``` + +3. **Text Editor** (VS Code recommended) + ```bash + # VS Code extensions: + # - Bash IDE + # - Shellcheck + # - Markdown All in One + ``` + +### Local Development Workflow + +#### Option A: Development Fork (Recommended) + +```bash +# 1. Fork on GitHub (one-time) +# Visit: https://github.com/community-scripts/ProxmoxVED +# Click: Fork + +# 2. Clone your fork +git clone https://github.com/YOUR_USERNAME/ProxmoxVED.git +cd ProxmoxVED + +# 3. Add upstream remote for updates +git remote add upstream https://github.com/community-scripts/ProxmoxVED.git + +# 4. Create feature branch +git checkout -b feat/add-myapp + +# 5. Make changes +# ... edit files ... + +# 6. Keep fork updated +git fetch upstream +git rebase upstream/main + +# 7. Push and open PR +git push origin feat/add-myapp +``` + +#### Option B: Local Testing on Proxmox Host + +```bash +# 1. SSH into Proxmox host +ssh root@192.168.1.100 + +# 2. Download your script +curl -O https://raw.githubusercontent.com/YOUR_USERNAME/ProxmoxVED/feat/myapp/ct/myapp.sh + +# 3. Make it executable +chmod +x myapp.sh + +# 4. Update URLs to your fork +# Edit: curl -s https://raw.githubusercontent.com/YOUR_USERNAME/ProxmoxVED/feat/myapp/... + +# 5. Run and test +bash myapp.sh + +# 6. If container created successfully, script is working! +``` + +#### Option C: Docker Testing (Without Proxmox) + +```bash +# You can test script syntax/functionality locally +# Note: Won't fully test (no Proxmox, no actual container) + +# Run ShellCheck +shellcheck ct/myapp.sh +shellcheck install/myapp-install.sh + +# Syntax check +bash -n ct/myapp.sh +bash -n install/myapp-install.sh +``` + +--- + +## Creating New Applications + +### Step 1: Choose Your Template + +**For Simple Web Apps** (Node.js, Python, PHP): +```bash +cp ct/example.sh ct/myapp.sh +cp install/example-install.sh install/myapp-install.sh +``` + +**For Database Apps** (PostgreSQL, MongoDB): +```bash +cp ct/docker.sh ct/myapp.sh # Use Docker container +# OR manual setup for more control +``` + +**For Alpine Linux Apps** (lightweight): +```bash +# Use ct/alpine.sh as reference +# Edit install script to use Alpine packages (apk not apt) +``` + +### Step 2: Update Container Script + +**File**: `ct/myapp.sh` + +```bash +#!/usr/bin/env bash +source <(curl -fsSL https://raw.githubusercontent.com/YOUR_USERNAME/ProxmoxVED/feat/myapp/misc/build.func) + +# Update these: +APP="MyAwesomeApp" # Display name +var_tags="category;tag2;tag3" # Max 3-4 tags +var_cpu="2" # Realistic CPU cores +var_ram="2048" # Min RAM needed (MB) +var_disk="10" # Min disk (GB) +var_os="debian" # OS type +var_version="12" # OS version +var_unprivileged="1" # Security (1=unprivileged) + +header_info "$APP" +variables +color +catch_errors + +function update_script() { + header_info + check_container_storage + check_container_resources + + if [[ ! -d /opt/myapp ]]; then + msg_error "No ${APP} Installation Found!" + exit + fi + + # Get latest version + RELEASE=$(curl -fsSL https://api.github.com/repos/user/repo/releases/latest | \ + grep "tag_name" | awk '{print substr($2, 2, length($2)-3)}') + + if [[ ! -f /opt/${APP}_version.txt ]] || [[ "${RELEASE}" != "$(cat /opt/${APP}_version.txt)" ]]; then + msg_info "Updating ${APP} to v${RELEASE}" + # ... update logic ... + echo "${RELEASE}" > /opt/${APP}_version.txt + msg_ok "Updated ${APP}" + else + msg_ok "No update required. ${APP} is already at v${RELEASE}." + fi + exit +} + +start +build_container +description + +msg_ok "Completed Successfully!\n" +echo -e "${CREATING}${GN}${APP} setup has been successfully initialized!${CL}" +echo -e "${INFO}${YW} Access it using the following URL:${CL}" +echo -e "${TAB}${GATEWAY}${BGN}http://${IP}:PORT${CL}" +``` + +**Checklist**: +- [ ] APP variable matches filename +- [ ] var_tags semicolon-separated (no spaces) +- [ ] Realistic CPU/RAM/disk values +- [ ] update_script() implemented +- [ ] Correct OS and version +- [ ] Success message with access URL + +### Step 3: Update Installation Script + +**File**: `install/myapp-install.sh` + +```bash +#!/usr/bin/env bash +# Copyright (c) 2021-2025 community-scripts ORG +# Author: YourUsername +# License: MIT +# Source: https://github.com/example/myapp + +source /dev/stdin <<<"$FUNCTIONS_FILE_PATH" +color +verb_ip6 +catch_errors +setting_up_container +network_check +update_os + +msg_info "Installing Dependencies" +$STD apt-get install -y \ + curl \ + wget \ + git \ + build-essential +msg_ok "Installed Dependencies" + +msg_info "Setting up Node.js" +NODE_VERSION="22" setup_nodejs +msg_ok "Node.js installed" + +msg_info "Downloading Application" +cd /opt +wget -q "https://github.com/user/repo/releases/download/v1.0.0/myapp.tar.gz" +tar -xzf myapp.tar.gz +rm -f myapp.tar.gz +msg_ok "Application installed" + +echo "1.0.0" > /opt/${APP}_version.txt + +motd_ssh +customize +cleanup_lxc +``` + +**Checklist**: +- [ ] Functions loaded from `$FUNCTIONS_FILE_PATH` +- [ ] All installation phases present (deps, tools, app, config, cleanup) +- [ ] Using `$STD` for output suppression +- [ ] Version file saved +- [ ] Final cleanup with `cleanup_lxc` +- [ ] No hardcoded versions (use GitHub API) + +### Step 4: Create ASCII Header (Optional) + +**File**: `ct/headers/myapp` + +``` +╔═══════════════════════════════════════╗ +β•‘ β•‘ +β•‘ πŸŽ‰ MyAwesomeApp πŸŽ‰ β•‘ +β•‘ β•‘ +β•‘ Your app is being installed... β•‘ +β•‘ β•‘ +β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β• +``` + +Save in: `ct/headers/myapp` (no extension) + +### Step 5: Create Defaults File (Optional) + +**File**: `defaults/myapp.vars` + +```bash +# Default configuration for MyAwesomeApp +var_cpu=4 +var_ram=4096 +var_disk=15 +var_hostname=myapp-container +var_timezone=UTC +``` + +--- + +## Updating Existing Applications + +### Step 1: Identify What Changed + +```bash +# Check logs or GitHub releases +curl -fsSL https://api.github.com/repos/app/repo/releases/latest | jq '.' + +# Review breaking changes +# Update dependencies if needed +``` + +### Step 2: Update Installation Script + +```bash +# Edit: install/existingapp-install.sh + +# 1. Update version (if hardcoded) +RELEASE="2.0.0" + +# 2. Update package dependencies (if any changed) +$STD apt-get install -y newdependency + +# 3. Update configuration (if format changed) +# Update sed replacements or config files + +# 4. Test thoroughly before committing +``` + +### Step 3: Update Update Function (if applicable) + +```bash +# Edit: ct/existingapp.sh β†’ update_script() + +# 1. Update GitHub API URL if repo changed +RELEASE=$(curl -fsSL https://api.github.com/repos/user/repo/releases/latest | ...) + +# 2. Update backup/restore logic (if structure changed) +# 3. Update cleanup paths + +# 4. Test update on existing installation +``` + +### Step 4: Document Your Changes + +```bash +# Add comment at top of script +# Co-Author: YourUsername +# Updated: YYYY-MM-DD - Description of changes +``` + +--- + +## Code Standards + +### Bash Style Guide + +#### Variable Naming + +```bash +# βœ… Good +APP="MyApp" # Constants (UPPERCASE) +var_cpu="2" # Configuration (var_*) +container_id="100" # Local variables (lowercase) +DB_PASSWORD="secret" # Environment-like (UPPERCASE) + +# ❌ Bad +myapp="MyApp" # Inconsistent +VAR_CPU="2" # Wrong convention +containerid="100" # Unclear purpose +``` + +#### Function Naming + +```bash +# βœ… Good +function setup_database() { } # Descriptive +function check_version() { } # Verb-noun pattern +function install_dependencies() { } # Clear action + +# ❌ Bad +function setup() { } # Too vague +function db_setup() { } # Inconsistent pattern +function x() { } # Cryptic +``` + +#### Quoting + +```bash +# βœ… Good +echo "${APP}" # Always quote variables +if [[ "$var" == "value" ]]; then # Use [[ ]] for conditionals +echo "Using $var in string" # Variables in double quotes + +# ❌ Bad +echo $APP # Unquoted variables +if [ "$var" = "value" ]; then # Use [[ ]] instead +echo 'Using $var in string' # Single quotes prevent expansion +``` + +#### Command Formatting + +```bash +# βœ… Good: Multiline for readability +$STD apt-get install -y \ + package1 \ + package2 \ + package3 + +# βœ… Good: Complex commands with variables +if ! wget -q "https://example.com/${file}"; then + msg_error "Failed to download" + exit 1 +fi + +# ❌ Bad: Too long on one line +$STD apt-get install -y package1 package2 package3 package4 package5 package6 + +# ❌ Bad: No error checking +wget https://example.com/file +``` + +#### Error Handling + +```bash +# βœ… Good: Check critical commands +if ! some_command; then + msg_error "Command failed" + exit 1 +fi + +# βœ… Good: Use catch_errors for automatic trapping +catch_errors + +# ❌ Bad: Silently ignore failures +some_command || true +some_command 2>/dev/null + +# ❌ Bad: Unclear what failed +if ! (cmd1 && cmd2 && cmd3); then + msg_error "Something failed" +fi +``` + +### Documentation Standards + +#### Header Comments + +```bash +#!/usr/bin/env bash +# Copyright (c) 2021-2025 community-scripts ORG +# Author: YourUsername +# Co-Author: AnotherAuthor (for collaborative work) +# License: MIT | https://github.com/community-scripts/ProxmoxVED/raw/main/LICENSE +# Source: https://github.com/app/repo +# Description: Brief description of what this script does +``` + +#### Inline Comments + +```bash +# βœ… Good: Explain WHY, not WHAT +# Use alphanumeric only to avoid shell escaping issues +DB_PASS=$(openssl rand -base64 18 | tr -dc 'a-zA-Z0-9' | head -c13) + +# βœ… Good: Comment complex logic +# Detect if running Alpine vs Debian for proper package manager +if grep -qi 'alpine' /etc/os-release; then + PKG_MGR="apk" +else + PKG_MGR="apt" +fi + +# ❌ Bad: Comment obvious code +# Set the variable +var="value" + +# ❌ Bad: Outdated comments +# TODO: Fix this (written 2 years ago, not fixed) +``` + +### File Organization + +```bash +#!/usr/bin/env bash # [1] Shebang (first line) +# Copyright & Metadata # [2] Comments + # [3] Blank line +# Load functions # [4] Import section +source <(curl -fsSL ...) + # [5] Blank line +# Configuration # [6] Variables/Config +APP="MyApp" +var_cpu="2" + # [7] Blank line +# Initialization # [8] Setup +header_info "$APP" +variables +color +catch_errors + # [9] Blank line +# Functions # [10] Function definitions +function update_script() { } +function custom_setup() { } + # [11] Blank line +# Main execution # [12] Script logic +start +build_container +``` + +--- + +## Testing Your Changes + +### Pre-Submission Testing + +#### 1. Syntax Check + +```bash +# Verify bash syntax +bash -n ct/myapp.sh +bash -n install/myapp-install.sh + +# If no output: βœ… Syntax is valid +# If error output: ❌ Fix syntax before submitting +``` + +#### 2. ShellCheck Static Analysis + +```bash +# Install ShellCheck +apt-get install -y shellcheck + +# Check scripts +shellcheck ct/myapp.sh +shellcheck install/myapp-install.sh + +# Review warnings and fix if applicable +# Some warnings can be intentional (use # shellcheck disable=...) +``` + +#### 3. Real Proxmox Testing + +```bash +# Best: Test on actual Proxmox system + +# 1. SSH into Proxmox host +ssh root@YOUR_PROXMOX_IP + +# 2. Download your script +curl -O https://raw.githubusercontent.com/YOUR_USER/ProxmoxVED/feat/myapp/ct/myapp.sh + +# 3. Make executable +chmod +x myapp.sh + +# 4. UPDATE URLS IN SCRIPT to point to your fork +sed -i 's|community-scripts|YOUR_USER|g' myapp.sh + +# 5. Run script +bash myapp.sh + +# 6. Test interaction: +# - Select installation mode +# - Confirm settings +# - Monitor installation + +# 7. Verify container created +pct list | grep myapp + +# 8. Log into container and verify app +pct exec 100 bash +``` + +#### 4. Edge Case Testing + +```bash +# Test with different settings: + +# Test 1: Advanced (19-step) installation +# When prompted: Select "2" for Advanced + +# Test 2: User Defaults +# Before running: Create ~/.community-scripts/default.vars +# When prompted: Select "3" for User Defaults + +# Test 3: Error handling +# Simulate network outage (block internet) +# Verify script handles gracefully + +# Test 4: Update function +# Create initial container +# Wait for new release +# Run update: bash ct/myapp.sh +# Verify it detects and applies update +``` + +### Testing Checklist + +Before submitting PR: + +```bash +# Code quality +- [ ] Syntax: bash -n passes +- [ ] ShellCheck: No critical warnings +- [ ] Naming: Follows conventions +- [ ] Formatting: Consistent indentation + +# Functionality +- [ ] Container creation: Successful +- [ ] Installation: Completes without errors +- [ ] Access URL: Works and app responds +- [ ] Update function: Detects new versions +- [ ] Cleanup: No temporary files left + +# Documentation +- [ ] Copyright header present +- [ ] App name matches filenames +- [ ] Default values realistic +- [ ] Success message clear and helpful + +# Compatibility +- [ ] Works on Debian 12 +- [ ] Works on Ubuntu 22.04 +- [ ] (Optional) Works on Alpine 3.20 +``` + +--- + +## Submitting a Pull Request + +### Step 1: Prepare Your Branch + +```bash +# Update with latest changes +git fetch upstream +git rebase upstream/main + +# If conflicts occur: +git rebase --abort +# Resolve conflicts manually then: +git add . +git rebase --continue +``` + +### Step 2: Push Your Changes + +```bash +git push origin feat/add-myapp + +# If already pushed: +git push origin feat/add-myapp --force-with-lease +``` + +### Step 3: Create Pull Request on GitHub + +**Visit**: https://github.com/community-scripts/ProxmoxVED/pulls + +**Click**: "New Pull Request" + +**Select**: `community-scripts:main` ← `YOUR_USERNAME:feat/myapp` + +### Step 4: Fill PR Description + +Use this template: + +```markdown +## Description +Brief description of what this PR adds/fixes + +## Type of Change +- [ ] New application (ct/AppName.sh + install/appname-install.sh) +- [ ] Update existing application +- [ ] Bug fix +- [ ] Documentation update +- [ ] Other: _______ + +## Testing +- [ ] Tested on Proxmox VE 8.x +- [ ] Container creation successful +- [ ] Application installation successful +- [ ] Application is accessible at URL +- [ ] Update function works (if applicable) +- [ ] No temporary files left after installation + +## Application Details (for new apps only) +- **App Name**: MyApp +- **Source**: https://github.com/app/repo +- **Default OS**: Debian 12 +- **Recommended Resources**: 2 CPU, 2GB RAM, 10GB Disk +- **Tags**: category;tag2;tag3 +- **Access URL**: http://IP:PORT/path + +## Checklist +- [ ] My code follows the style guidelines +- [ ] I have performed a self-review +- [ ] I have tested the script locally +- [ ] ShellCheck shows no critical warnings +- [ ] Documentation is accurate and complete +- [ ] I have added/updated relevant documentation +``` + +### Step 5: Respond to Review Comments + +**Maintainers may request changes**: +- Fix syntax/style issues +- Add better error handling +- Optimize resource usage +- Update documentation + +**To address feedback**: + +```bash +# Make requested changes +git add . +git commit -m "Address review feedback: ..." +git push origin feat/add-myapp + +# PR automatically updates! +# No need to create new PR +``` + +### Step 6: Celebrate! πŸŽ‰ + +Once merged, your contribution will be part of ProxmoxVED and available to all users! + +--- + +## Troubleshooting + +### "Repository not found" when cloning + +```bash +# Check your fork exists +# Visit: https://github.com/YOUR_USERNAME/ProxmoxVED + +# If not there: Click "Fork" on original repo first +``` + +### "Permission denied" when pushing + +```bash +# Setup SSH key +ssh-keygen -t ed25519 -C "your_email@example.com" +cat ~/.ssh/id_ed25519.pub # Copy this + +# Add to GitHub: Settings β†’ SSH Keys β†’ New Key + +# Or use HTTPS with token: +git remote set-url origin https://YOUR_TOKEN@github.com/YOUR_USERNAME/ProxmoxVED.git +``` + +### Script syntax errors + +```bash +# Use ShellCheck to identify issues +shellcheck install/myapp-install.sh + +# Common issues: +# - Unmatched quotes: "string' or 'string" +# - Missing semicolons before then: if [...]; then +# - Wrong quoting: echo $VAR instead of echo "${VAR}" +``` + +### Container creation fails immediately + +```bash +# 1. Check Proxmox resources +free -h # Check RAM +df -h # Check disk space +pct list # Check CTID availability + +# 2. Check script URL +# Make sure curl -s in script points to your fork + +# 3. Review errors +# Run with verbose: bash -x ct/myapp.sh +``` + +### App not accessible after creation + +```bash +# 1. Verify container running +pct list +pct status CTID + +# 2. Check if service running inside +pct exec CTID systemctl status myapp + +# 3. Check firewall +# Proxmox host: iptables -L +# Container: iptables -L + +# 4. Verify listening port +pct exec CTID netstat -tlnp | grep LISTEN +``` + +--- + +## FAQ + +### Q: Do I need to be a Bash expert? + +**A**: No! The codebase has many examples you can copy. Most contributions are straightforward script creation following the established patterns. + +### Q: Can I add a new application that's not open source? + +**A**: No. ProxmoxVED focuses on open-source applications (GPL, MIT, Apache, etc.). Closed-source applications won't be accepted. + +### Q: How long until my PR is reviewed? + +**A**: Maintainers are volunteers. Reviews typically happen within 1-2 weeks. Complex changes may take longer. + +### Q: Can I test without a Proxmox system? + +**A**: Partially. You can verify syntax and ShellCheck compliance locally, but real container testing requires Proxmox. Consider using: +- Proxmox in a VM (VirtualBox/KVM) +- Test instances on Hetzner/DigitalOcean +- Ask maintainers to test for you + +### Q: My update function is very complex - is that OK? + +**A**: Yes! Update functions can be complex if needed. Just ensure: +- Backup user data before updating +- Restore user data after update +- Test thoroughly before submitting +- Add clear comments explaining logic + +### Q: Can I add new dependencies to build.func? + +**A**: Generally no. build.func is the orchestrator and should remain stable. New functions should go in: +- `tools.func` - Tool installation +- `core.func` - Utility functions +- `install.func` - Container setup + +Ask in an issue first if you're unsure. + +### Q: What if the application has many configuration options? + +**A**: You have options: + +**Option 1**: Use Advanced mode (19-step wizard) +```bash +# Extend advanced_settings() if app needs special vars +``` + +**Option 2**: Create custom setup menu +```bash +function custom_config() { + OPTION=$(whiptail --inputbox "Enter database name:" 8 60) + # ... use $OPTION in installation +} +``` + +**Option 3**: Leave as defaults + documentation +```bash +# In success message: +echo "Edit /opt/myapp/config.json to customize settings" +``` + +### Q: Can I contribute Windows/macOS/ARM support? + +**A**: +- **Windows**: Not planned (ProxmoxVED is Linux/Proxmox focused) +- **macOS**: Can contribute Docker-based alternatives +- **ARM**: Yes! Many apps work on ARM. Add to vm/pimox-*.sh scripts + +--- + +## Getting Help + +### Resources + +- **Documentation**: `/docs` directory and wikis +- **Function Reference**: `/misc/*.md` wiki files +- **Examples**: Look at similar applications in `/ct` and `/install` +- **GitHub Issues**: https://github.com/community-scripts/ProxmoxVED/issues +- **Discussions**: https://github.com/community-scripts/ProxmoxVED/discussions + +### Ask Questions + +1. **Check existing issues** - Your question may be answered +2. **Search documentation** - See `/docs` and `/misc/*.md` +3. **Ask in Discussions** - For general questions +4. **Open an Issue** - For bugs or specific problems + +### Report Bugs + +When reporting bugs, include: +- Which application +- What happened (error message) +- What you expected +- Your Proxmox version +- Container OS and version + +Example: +``` +Title: pihole-install.sh fails on Alpine 3.20 + +Description: +Installation fails with error: "PHP-FPM not found" + +Expected: +PiHole should install successfully + +Environment: +- Proxmox VE 8.2 +- Alpine 3.20 +- Container CTID 110 + +Error Output: +[ERROR] in line 42: exit code 127: while executing command php-fpm --start +``` + +--- + +## Contribution Statistics + +**ProxmoxVED by the Numbers**: +- 🎯 40+ applications supported +- πŸ‘₯ 100+ contributors +- πŸ“Š 10,000+ GitHub stars +- πŸš€ 50+ releases +- πŸ“ˆ 100,000+ downloads/month + +**Your contribution makes a difference!** + +--- + +## Code of Conduct + +By contributing, you agree to: +- βœ… Be respectful and inclusive +- βœ… Follow the style guidelines +- βœ… Test your changes thoroughly +- βœ… Provide clear commit messages +- βœ… Respond to review feedback + +--- + +**Ready to contribute?** Start with the [Quick Start](#quick-start) section! + +**Questions?** Open an issue or start a discussion on GitHub. + +**Thank you for your contribution!** πŸ™ diff --git a/docs/UPDATED_APP-ct.md b/docs/UPDATED_APP-ct.md new file mode 100644 index 000000000..e7a12ea7e --- /dev/null +++ b/docs/UPDATED_APP-ct.md @@ -0,0 +1,717 @@ +# πŸš€ **Application Container Scripts (ct/AppName.sh)** + +**Modern Guide to Creating LXC Container Installation Scripts** + +> **Updated**: December 2025 +> **Context**: Fully integrated with build.func, advanced_settings wizard, and defaults system +> **Example Used**: `/ct/pihole.sh`, `/ct/docker.sh` + +--- + +## πŸ“‹ Table of Contents + +- [Overview](#overview) +- [Architecture & Flow](#architecture--flow) +- [File Structure](#file-structure) +- [Complete Script Template](#complete-script-template) +- [Function Reference](#function-reference) +- [Advanced Features](#advanced-features) +- [Real Examples](#real-examples) +- [Troubleshooting](#troubleshooting) +- [Contribution Checklist](#contribution-checklist) + +--- + +## Overview + +### Purpose + +Container scripts (`ct/AppName.sh`) are **entry points for creating LXC containers** with specific applications pre-installed. They: + +1. Define container defaults (CPU, RAM, disk, OS) +2. Call the main build orchestrator (`build.func`) +3. Implement application-specific update mechanisms +4. Provide user-facing success messages + +### Execution Context + +``` +Proxmox Host + ↓ +ct/AppName.sh sourced (runs as root on host) + ↓ +build.func: Creates LXC container + runs install script inside + ↓ +install/AppName-install.sh (runs inside container) + ↓ +Container ready with app installed +``` + +### Key Integration Points + +- **build.func** - Main orchestrator (container creation, storage, variable management) +- **install.func** - Container-specific setup (OS update, package management) +- **tools.func** - Tool installation helpers (repositories, GitHub releases) +- **core.func** - UI/messaging functions (colors, spinners, validation) +- **error_handler.func** - Error handling and signal management + +--- + +## Architecture & Flow + +### Container Creation Flow + +``` +START: bash ct/pihole.sh + ↓ +[1] Set APP, var_*, defaults + ↓ +[2] header_info() β†’ Display ASCII art + ↓ +[3] variables() β†’ Parse arguments & load build.func + ↓ +[4] color() β†’ Setup ANSI codes + ↓ +[5] catch_errors() β†’ Setup trap handlers + ↓ +[6] install_script() β†’ Show mode menu (5 options) + ↓ + β”œβ”€ INSTALL_MODE="0" (Default) + β”œβ”€ INSTALL_MODE="1" (Advanced - 19-step wizard) + β”œβ”€ INSTALL_MODE="2" (User Defaults) + β”œβ”€ INSTALL_MODE="3" (App Defaults) + └─ INSTALL_MODE="4" (Settings Menu) + ↓ +[7] advanced_settings() β†’ Collect user configuration (if mode=1) + ↓ +[8] start() β†’ Confirm or re-edit settings + ↓ +[9] build_container() β†’ Create LXC + execute install script + ↓ +[10] description() β†’ Set container description + ↓ +[11] SUCCESS β†’ Display access URL + ↓ +END +``` + +### Default Values Precedence + +``` +Priority 1 (Highest): Environment Variables (var_cpu, var_ram, etc.) +Priority 2: App-Specific Defaults (/defaults/AppName.vars) +Priority 3: User Global Defaults (/default.vars) +Priority 4 (Lowest): Built-in Defaults (in build.func) +``` + +--- + +## File Structure + +### Minimal ct/AppName.sh Template + +``` +#!/usr/bin/env bash # [1] Shebang + # [2] Copyright/License +source <(curl -s .../misc/build.func) # [3] Import functions + # [4] APP metadata +APP="AppName" # [5] Default values +var_tags="tag1;tag2" +var_cpu="2" +var_ram="2048" +... + +header_info "$APP" # [6] Display header +variables # [7] Process arguments +color # [8] Setup colors +catch_errors # [9] Setup error handling + +function update_script() { ... } # [10] Update function (optional) + +start # [11] Launch container creation +build_container +description +msg_ok "Completed Successfully!\n" +``` + +--- + +## Complete Script Template + +### 1. File Header & Imports + +```bash +#!/usr/bin/env bash +# Copyright (c) 2021-2025 community-scripts ORG +# Author: YourUsername +# License: MIT | https://github.com/community-scripts/ProxmoxVED/raw/main/LICENSE +# Source: https://github.com/example/project + +# Import main orchestrator +source <(curl -fsSL https://git.community-scripts.org/community-scripts/ProxmoxVED/raw/branch/main/misc/build.func) +``` + +> **⚠️ IMPORTANT**: Before opening a PR, change URL to `community-scripts` repo! + +### 2. Application Metadata + +```bash +# Application Configuration +APP="ApplicationName" +var_tags="tag1;tag2;tag3" # Max 3-4 tags, no spaces, semicolon-separated + +# Container Resources +var_cpu="2" # CPU cores +var_ram="2048" # RAM in MB +var_disk="10" # Disk in GB + +# Container Type & OS +var_os="debian" # Options: alpine, debian, ubuntu +var_version="12" # Alpine: 3.20+, Debian: 11-13, Ubuntu: 20.04+ +var_unprivileged="1" # 1=unprivileged (secure), 0=privileged (rarely needed) +``` + +**Variable Naming Convention**: +- Variables exposed to user: `var_*` (e.g., `var_cpu`, `var_hostname`, `var_ssh`) +- Internal variables: lowercase (e.g., `container_id`, `app_version`) + +### 3. Display & Initialization + +```bash +# Display header ASCII art +header_info "$APP" + +# Process command-line arguments and load configuration +variables + +# Setup ANSI color codes and formatting +color + +# Initialize error handling (trap ERR, EXIT, INT, TERM) +catch_errors +``` + +### 4. Update Function (Highly Recommended) + +```bash +function update_script() { + header_info + + # Always start with these checks + check_container_storage + check_container_resources + + # Verify app is installed + if [[ ! -d /opt/appname ]]; then + msg_error "No ${APP} Installation Found!" + exit + fi + + # Get latest version from GitHub + RELEASE=$(curl -fsSL https://api.github.com/repos/user/repo/releases/latest | \ + grep "tag_name" | awk '{print substr($2, 2, length($2)-3)}') + + # Compare with saved version + if [[ ! -f /opt/${APP}_version.txt ]] || [[ "${RELEASE}" != "$(cat /opt/${APP}_version.txt)" ]]; then + msg_info "Updating ${APP} to v${RELEASE}" + + # Backup user data + cp -r /opt/appname /opt/appname-backup + + # Perform update + cd /opt + wget -q "https://github.com/user/repo/releases/download/v${RELEASE}/app-${RELEASE}.tar.gz" + tar -xzf app-${RELEASE}.tar.gz + + # Restore user data + cp /opt/appname-backup/config/* /opt/appname/config/ + + # Cleanup + rm -rf app-${RELEASE}.tar.gz /opt/appname-backup + + # Save new version + echo "${RELEASE}" > /opt/${APP}_version.txt + + msg_ok "Updated ${APP} to v${RELEASE}" + else + msg_ok "No update required. ${APP} is already at v${RELEASE}." + fi + + exit +} +``` + +### 5. Script Launch + +```bash +# Start the container creation workflow +start + +# Build the container with selected configuration +build_container + +# Set container description/notes in Proxmox UI +description + +# Display success message +msg_ok "Completed Successfully!\n" +echo -e "${CREATING}${GN}${APP} setup has been successfully initialized!${CL}" +echo -e "${INFO}${YW} Access it using the following URL:${CL}" +echo -e "${TAB}${GATEWAY}${BGN}http://${IP}:8080${CL}" +``` + +--- + +## Function Reference + +### Core Functions (From build.func) + +#### `variables()` + +**Purpose**: Initialize container variables, load user arguments, setup orchestration + +**Triggered by**: Called automatically at script start + +**Behavior**: +1. Parse command-line arguments (if any) +2. Generate random UUID for session tracking +3. Load container storage from Proxmox +4. Initialize application-specific defaults +5. Setup SSH/environment configuration + +**Example Output**: +``` +Setting up variables... +Container ID: 100 +Storage: local-lvm +Install method: Default +``` + +#### `start()` + +**Purpose**: Launch the container creation menu with 5 installation modes + +**Triggered by**: Called just before `build_container()` + +**Menu Options**: +``` +1. Default Installation (Quick setup, predefined settings) +2. Advanced Installation (19-step wizard with full control) +3. User Defaults (Load ~/.community-scripts/default.vars) +4. App Defaults (Load /defaults/AppName.vars) +5. Settings Menu (Interactive mode selection) +``` + +**User Flow**: +``` +Select installation mode: +1) Default +2) Advanced +3) User Defaults +4) App Defaults +5) Settings Menu +Enter choice: 2 +``` + +#### `build_container()` + +**Purpose**: Main orchestrator for LXC container creation + +**Operations**: +1. Validates all variables +2. Creates LXC container via `pct create` +3. Executes `install/AppName-install.sh` inside container +4. Monitors installation progress +5. Handles errors and rollback on failure + +**Exit Codes**: +- `0` - Success +- `1-255` - Various error conditions (see error_handler.func) + +#### `description()` + +**Purpose**: Set container description/notes visible in Proxmox UI + +**Format**: +``` +AppName +IP: [IP] +Version: [Version] +Tags: [Tags] +``` + +#### `header_info()` + +**Purpose**: Display ASCII art header for application + +**Sources**: +- Tries `/usr/local/community-scripts/headers/ct/appname` (cached) +- Falls back to remote fetch from GitHub +- Returns silently if not found + +--- + +## Advanced Features + +### 1. Integration with Defaults System + +#### Save App Defaults After Installation + +```bash +# At end of install script, after successful setup: +maybe_offer_save_app_defaults + +# Output: +# "Save these settings as App Defaults for AppName? (Y/n)" +# Yes β†’ Saves to /defaults/appname.vars +# No β†’ Skips saving +``` + +#### Load Saved Defaults During Container Creation + +```bash +# In ct/AppName.sh, user selects "App Defaults" mode +# Automatically loads /defaults/appname.vars +# Container uses previously saved configuration +``` + +### 2. Custom Configuration Menus + +If your app has additional setup beyond standard vars: + +```bash +# In ct/AppName.sh, after variables() +custom_app_settings() { + CONFIGURE_DB=$(whiptail --title "Database Setup" \ + --yesno "Would you like to configure a custom database?" 8 60) + + if [[ $? -eq 0 ]]; then + DB_HOST=$(whiptail --inputbox "Database Host:" 8 60 3>&1 1>&2 2>&3) + DB_PORT=$(whiptail --inputbox "Database Port:" 8 60 "3306" 3>&1 1>&2 2>&3) + fi +} + +custom_app_settings +``` + +### 3. Version Tracking + +Save installed version for update checks: + +```bash +# In install script, after successful app download: +RELEASE="1.2.3" +echo "${RELEASE}" > /opt/${APP}_version.txt + +# In update function, compare: +CURRENT=$(cat /opt/${APP}_version.txt 2>/dev/null) +LATEST=$(curl -fsSL https://api.github.com/repos/user/repo/releases/latest | jq -r '.tag_name') + +if [[ "$LATEST" != "$CURRENT" ]]; then + echo "Update available: $CURRENT β†’ $LATEST" +fi +``` + +### 4. Health Check Functions + +Add custom validation: + +```bash +function health_check() { + header_info + + if [[ ! -d /opt/appname ]]; then + msg_error "Application not found!" + exit 1 + fi + + if ! systemctl is-active --quiet appname; then + msg_error "Application service not running" + exit 1 + fi + + msg_ok "Health check passed" +} + +# Called via: bash ct/appname.sh health_check +``` + +--- + +## Real Examples + +### Example 1: Simple Web App (Debian-based) + +```bash +#!/usr/bin/env bash +source <(curl -fsSL https://git.community-scripts.org/community-scripts/ProxmoxVED/raw/branch/main/misc/build.func) + +APP="Homarr" +var_tags="dashboard;homepage" +var_cpu="2" +var_ram="1024" +var_disk="5" +var_os="debian" +var_version="12" +var_unprivileged="1" + +header_info "$APP" +variables +color +catch_errors + +function update_script() { + header_info + check_container_storage + check_container_resources + + if [[ ! -d /opt/homarr ]]; then + msg_error "No ${APP} Installation Found!" + exit + fi + + RELEASE=$(curl -fsSL https://api.github.com/repos/ajnart/homarr/releases/latest | grep "tag_name" | awk '{print substr($2, 3, length($2)-4)}') + + if [[ ! -f /opt/${APP}_version.txt ]] || [[ "${RELEASE}" != "$(cat /opt/${APP}_version.txt)" ]]; then + msg_info "Updating ${APP} to v${RELEASE}" + systemctl stop homarr + + cd /opt/homarr + wget -q "https://github.com/ajnart/homarr/releases/download/v${RELEASE}/docker-compose.yml" + docker-compose up -d + + echo "${RELEASE}" > /opt/${APP}_version.txt + msg_ok "Updated ${APP} to v${RELEASE}" + else + msg_ok "No update required. ${APP} is already at v${RELEASE}." + fi + exit +} + +start +build_container +description + +msg_ok "Completed Successfully!\n" +echo -e "${CREATING}${GN}${APP} setup has been successfully initialized!${CL}" +echo -e "${INFO}${YW} Access it using the following URL:${CL}" +echo -e "${TAB}${GATEWAY}${BGN}http://${IP}:5100${CL}" +``` + +### Example 2: Database App (Alpine-based) + +```bash +#!/usr/bin/env bash +source <(curl -fsSL https://git.community-scripts.org/community-scripts/ProxmoxVED/raw/branch/main/misc/build.func) + +APP="PostgreSQL" +var_tags="database;sql" +var_cpu="4" +var_ram="4096" +var_disk="20" +var_os="alpine" +var_version="3.20" +var_unprivileged="1" + +header_info "$APP" +variables +color +catch_errors + +function update_script() { + header_info + check_container_storage + + if ! command -v psql &>/dev/null; then + msg_error "PostgreSQL not installed!" + exit + fi + + msg_info "Updating Alpine packages" + apk update + apk upgrade + msg_ok "Updated Alpine packages" + exit +} + +start +build_container +description + +msg_ok "Completed Successfully!\n" +echo -e "${CREATING}${GN}${APP} setup has been successfully initialized!${CL}" +echo -e "${INFO}${YW} Connect using:${CL}" +echo -e "${TAB}${GATEWAY}${BGN}psql -h ${IP} -U postgres${CL}" +``` + +--- + +## Troubleshooting + +### Container Creation Fails + +**Symptom**: `pct create` exits with error code 209 + +**Causes**: +1. CTID already exists: `pct list` shows duplicate +2. Storage full: Check storage space +3. Network template unavailable + +**Solution**: +```bash +# Check existing containers +pct list | grep CTID + +# Remove conflicting container +pct destroy CTID + +# Retry ct/AppName.sh +``` + +### Update Function Doesn't Detect New Version + +**Symptom**: Update available but script says "already at latest" + +**Causes**: +1. Version file missing: `/opt/AppName_version.txt` +2. GitHub API rate limit exceeded +3. Release tag format mismatch + +**Debug**: +```bash +# Check version file +cat /opt/AppName_version.txt + +# Test GitHub API +curl -fsSL https://api.github.com/repos/user/repo/releases/latest | grep tag_name + +# Inside container +bash ct/appname.sh update_script +``` + +### Header ASCII Art Not Displaying + +**Symptom**: Container script runs but no header shown + +**Causes**: +1. Header file not in repository +2. Caching issue + +**Solution**: +```bash +# Create header file manually +mkdir -p /usr/local/community-scripts/headers/ct +echo "Your ASCII art here" > /usr/local/community-scripts/headers/ct/appname + +# Or remove cache to force re-download +rm -f /usr/local/community-scripts/headers/ct/appname +``` + +--- + +## Contribution Checklist + +Before submitting a PR: + +### Script Structure +- [ ] Shebang is `#!/usr/bin/env bash` +- [ ] Imports `build.func` from community-scripts repo (not personal fork) +- [ ] Copyright header with author and source URL +- [ ] APP variable matches filename +- [ ] `var_tags` are semicolon-separated (no spaces) + +### Default Values +- [ ] `var_cpu` set appropriately (2-4 for most apps) +- [ ] `var_ram` set appropriately (1024-4096 MB minimum) +- [ ] `var_disk` sufficient for app + data (5-20 GB) +- [ ] `var_os` is realistic (Alpine if lightweight, Debian/Ubuntu otherwise) +- [ ] `var_unprivileged="1"` unless app absolutely needs privileges + +### Functions +- [ ] `update_script()` implemented (or marked as unavailable) +- [ ] Update function checks if app installed +- [ ] Update function checks for new version +- [ ] Update function performs cleanup after update +- [ ] Proper error handling with `msg_error` on failure + +### Output +- [ ] Success message displayed with access URL +- [ ] URL format: `http://IP:PORT/path` (if web-based) +- [ ] Uses `msg_ok`, `msg_info`, `msg_error` for feedback + +### Testing +- [ ] Script tested with default installation +- [ ] Script tested with advanced (19-step) installation +- [ ] Update function tested on existing installation +- [ ] Error handling tested (invalid settings, network issues) + +--- + +## Best Practices + +### βœ… DO: + +1. **Use meaningful defaults** + ```bash + var_cpu="2" # βœ… Good: Typical workload + var_cpu="128" # ❌ Bad: Unrealistic + ``` + +2. **Implement version tracking** + ```bash + echo "${RELEASE}" > /opt/${APP}_version.txt # βœ… Good + # ❌ Bad: No version tracking + ``` + +3. **Handle edge cases** + ```bash + if [[ ! -f /opt/${APP}_version.txt ]]; then + msg_info "First installation detected" + fi + ``` + +4. **Use proper messaging** + ```bash + msg_info "Updating..." # βœ… Good: Clear status + echo "Updating..." # ❌ Bad: No formatting + ``` + +### ❌ DON'T: + +1. **Hardcode versions** + ```bash + RELEASE="1.2.3" # ❌ Bad: Won't auto-update + ``` + +2. **Use custom color codes** + ```bash + echo -e "\033[32mSuccess" # ❌ Bad: Use $GN instead + ``` + +3. **Forget error handling** + ```bash + wget file.zip # ❌ Bad: No error check + if ! wget -q file.zip; then # βœ… Good + msg_error "Download failed" + fi + ``` + +4. **Leave temporary files** + ```bash + rm -rf /opt/file.zip # βœ… Always cleanup + ``` + +--- + +## Related Documentation + +- [install/AppName-install.sh Guide](UPDATED_APP-install.md) +- [build.func Wiki](../misc/build.func.md) +- [tools.func Wiki](../misc/tools.func.md) +- [Defaults System Guide](../DEFAULTS_SYSTEM_GUIDE.md) + +--- + +**Last Updated**: December 2025 +**Compatibility**: ProxmoxVED with build.func v3+ +**Questions?** Open an issue in the repository diff --git a/docs/UPDATED_APP-install.md b/docs/UPDATED_APP-install.md new file mode 100644 index 000000000..c012aba5c --- /dev/null +++ b/docs/UPDATED_APP-install.md @@ -0,0 +1,1121 @@ +# πŸ› οΈ **Application Installation Scripts (install/AppName-install.sh)** + +**Modern Guide to Writing In-Container Installation Scripts** + +> **Updated**: December 2025 +> **Context**: Integrated with tools.func, error_handler.func, and install.func +> **Examples Used**: `/install/pihole-install.sh`, `/install/mealie-install.sh` + +--- + +## πŸ“‹ Table of Contents + +- [Overview](#overview) +- [Execution Context](#execution-context) +- [File Structure](#file-structure) +- [Complete Script Template](#complete-script-template) +- [Installation Phases](#installation-phases) +- [Function Reference](#function-reference) +- [Best Practices](#best-practices) +- [Real Examples](#real-examples) +- [Troubleshooting](#troubleshooting) +- [Contribution Checklist](#contribution-checklist) + +--- + +## Overview + +### Purpose + +Installation scripts (`install/AppName-install.sh`) **run inside the LXC container** and are responsible for: + +1. Setting up the container OS (updates, packages) +2. Installing application dependencies +3. Downloading and configuring the application +4. Setting up services and systemd units +5. Creating version tracking files for updates +6. Generating credentials/configurations +7. Final cleanup and validation + +### Key Characteristics + +- Runs as **root inside container** (not on Proxmox host) +- Executed automatically by `build_container()` from ct/AppName.sh +- Uses `$FUNCTIONS_FILE_PATH` for function library access +- Interactive elements via **whiptail** (GUI menus) +- Version-aware for update tracking + +### Execution Flow + +``` +ct/AppName.sh (Proxmox Host) + ↓ +build_container() + ↓ +pct exec CTID bash -c "$(cat install/AppName-install.sh)" + ↓ +install/AppName-install.sh (Inside Container) + ↓ +Container Ready with App Installed +``` + +--- + +## Execution Context + +### Environment Variables Available + +```bash +# From Proxmox/Container +CTID # Container ID (100, 101, etc.) +PCT_OSTYPE # OS type (alpine, debian, ubuntu) +HOSTNAME # Container hostname + +# From build.func +FUNCTIONS_FILE_PATH # Bash functions library (core.func + tools.func) +VERBOSE # Verbose mode (yes/no) +STD # Standard redirection variable (silent/empty) + +# From install.func +APP # Application name +NSAPP # Normalized app name (lowercase, no spaces) +METHOD # Installation method (ct/install) +RANDOM_UUID # Session UUID for telemetry +``` + +### Access to Functions + +```bash +# All functions from core.func available: +source /dev/stdin <<<"$FUNCTIONS_FILE_PATH" +color # ANSI colors +catch_errors # Error handling +msg_info # Display messages +msg_ok +msg_error + +# All functions from tools.func available: +setup_nodejs # Tool installation +setup_php +setup_python +setup_docker +# ... many more + +# All functions from install.func available: +motd_ssh # Final setup +customize +cleanup_lxc +``` + +--- + +## File Structure + +### Minimal install/AppName-install.sh Template + +```bash +#!/usr/bin/env bash # [1] Shebang + +# [2] Copyright/Metadata +# Copyright (c) 2021-2025 community-scripts ORG +# Author: YourUsername +# License: MIT +# Source: https://example.com + +# [3] Load functions +source /dev/stdin <<<"$FUNCTIONS_FILE_PATH" +color +verb_ip6 +catch_errors +setting_up_container +network_check +update_os + +# [4] Installation steps +msg_info "Installing Dependencies" +$STD apt-get install -y package1 package2 +msg_ok "Installed Dependencies" + +# [5] Final setup +motd_ssh +customize +cleanup_lxc +``` + +--- + +## Complete Script Template + +### Phase 1: Header & Initialization + +```bash +#!/usr/bin/env bash +# Copyright (c) 2021-2025 community-scripts ORG +# Author: YourUsername +# Co-Author: AnotherAuthor (for updates) +# License: MIT | https://github.com/community-scripts/ProxmoxVED/raw/main/LICENSE +# Source: https://github.com/application/repo + +# Load all available functions (from core.func + tools.func) +source /dev/stdin <<<"$FUNCTIONS_FILE_PATH" + +# Initialize environment +color # Setup ANSI colors and icons +verb_ip6 # Configure IPv6 (if needed) +catch_errors # Setup error traps +setting_up_container # Verify OS is ready +network_check # Verify internet connectivity +update_os # Update packages (apk/apt) +``` + +### Phase 2: Dependency Installation + +```bash +msg_info "Installing Dependencies" +$STD apt-get install -y \ + curl \ + wget \ + git \ + nano \ + build-essential \ + libssl-dev \ + python3-dev +msg_ok "Installed Dependencies" +``` + +**Guidelines**: +- Use `\` for line continuation (readability) +- Group related packages together +- Collapse repeated prefixes: `php8.4-{bcmath,curl,gd,intl,mbstring}` +- Use `-y` flag for non-interactive installation +- Silence output with `$STD` unless debugging + +### Phase 3: Tool Setup (Using tools.func) + +```bash +# Setup specific tool versions +NODE_VERSION="22" setup_nodejs + +# Or for databases +MYSQL_VERSION="8.0" setup_mysql + +# Or for languages +PHP_VERSION="8.4" PHP_MODULE="redis,imagick" setup_php + +# Or for version control +setup_composer +``` + +**Available Tool Functions**: +```bash +setup_nodejs # Node.js from official repo +setup_php # PHP with optional modules +setup_python # Python 3 +setup_mariadb # MariaDB database +setup_mysql # MySQL database +setup_postgresql # PostgreSQL database +setup_mongodb # MongoDB database +setup_docker # Docker Engine +setup_nodejs # Node.js runtime +setup_composer # PHP Composer +setup_ruby # Ruby runtime +setup_rust # Rust toolchain +setup_go # Go language +setup_java # Java/Temurin +# ... many more in tools.func.md +``` + +### Phase 4: Application Download & Setup + +```bash +# Method A: Download from GitHub releases +msg_info "Downloading ${APP}" +RELEASE=$(curl -fsSL https://api.github.com/repos/user/repo/releases/latest | \ + grep "tag_name" | awk '{print substr($2, 2, length($2)-3)}') + +wget -q "https://github.com/user/repo/releases/download/v${RELEASE}/app-${RELEASE}.tar.gz" \ + -O /opt/app-${RELEASE}.tar.gz + +cd /opt +tar -xzf app-${RELEASE}.tar.gz +rm -f app-${RELEASE}.tar.gz +msg_ok "Downloaded and extracted ${APP}" + +# Method B: Clone from Git +git clone https://github.com/user/repo /opt/appname + +# Method C: Download single file +fetch_and_deploy_gh_release "AppName" "user/repo" "tarball" +``` + +### Phase 5: Configuration Files + +```bash +# Method A: Using cat << EOF (multiline) +cat <<'EOF' >/etc/nginx/sites-available/appname +server { + listen 80; + server_name _; + root /opt/appname/public; + index index.php index.html; + + location ~ \.php$ { + fastcgi_pass unix:/run/php-fpm.sock; + include fastcgi_params; + } +} +EOF + +# Method B: Using sed for replacements +sed -i -e "s|^DB_HOST=.*|DB_HOST=localhost|" \ + -e "s|^DB_USER=.*|DB_USER=appuser|" \ + /opt/appname/.env + +# Method C: Using echo for simple configs +echo "APP_KEY=base64:$(openssl rand -base64 32)" >> /opt/appname/.env +``` + +### Phase 6: Database Setup (If Needed) + +```bash +msg_info "Setting up Database" + +DB_NAME="appname_db" +DB_USER="appuser" +DB_PASS=$(openssl rand -base64 18 | tr -dc 'a-zA-Z0-9' | head -c13) + +# For MySQL/MariaDB +mysql -u root <> ~/appname.creds +Database Credentials +Database: ${DB_NAME} +Username: ${DB_USER} +Password: ${DB_PASS} +EOF + +msg_ok "Database setup complete" +``` + +### Phase 7: Permission & Ownership + +```bash +msg_info "Setting permissions" + +# Web applications typically run as www-data +chown -R www-data:www-data /opt/appname +chmod -R 755 /opt/appname +chmod -R 644 /opt/appname/* +chmod 755 /opt/appname/*/.* + +# For apps with specific requirements +find /opt/appname/storage -type f -exec chmod 644 {} \; +find /opt/appname/storage -type d -exec chmod 755 {} \; + +msg_ok "Permissions set" +``` + +### Phase 8: Service Configuration + +```bash +# Enable systemd service +systemctl enable -q --now appname + +# Or for OpenRC (Alpine) +rc-service appname start +rc-update add appname default + +# Verify service is running +if systemctl is-active --quiet appname; then + msg_ok "Service running successfully" +else + msg_error "Service failed to start" + journalctl -u appname -n 20 + exit 1 +fi +``` + +### Phase 9: Version Tracking + +```bash +# Essential for update detection +echo "${RELEASE}" > /opt/${APP}_version.txt + +# Or with additional metadata +cat > /opt/${APP}_version.txt < /opt/${APP}_version.txt +``` + +### Phase 5: Configuration + +```bash +# Application-specific configuration +cat > /opt/appname/.env < /opt/appname/config.yml < /opt/${APP}_version.txt + +# ❌ Bad: No version file +# (Update function won't work) +``` + +#### 6. Handle Alpine vs Debian Differences + +```bash +# βœ… Good: Detect OS +if grep -qi 'alpine' /etc/os-release; then + apk add package +else + apt-get install -y package +fi + +# ❌ Bad: Assumes Debian +apt-get install -y package +``` + +#### 7. Use Proper Messaging + +```bash +# βœ… Good: Clear status progression +msg_info "Installing Dependencies" +$STD apt-get install -y package +msg_ok "Installed Dependencies" + +msg_info "Configuring Application" +# ... configuration ... +msg_ok "Application configured" + +# ❌ Bad: No status messages +apt-get install -y package +# ... configuration ... +``` + +### ❌ DON'T: + +#### 1. Hardcode Versions + +```bash +# ❌ Bad: Won't auto-update +VERSION="1.2.3" +wget https://example.com/app-1.2.3.tar.gz + +# βœ… Good: Fetch latest +RELEASE=$(curl -fsSL https://api.github.com/repos/user/repo/releases/latest | jq -r '.tag_name') +wget https://example.com/app-${RELEASE}.tar.gz +``` + +#### 2. Use Root Without Password + +```bash +# ❌ Bad: Allows unprompted root access +mysql -u root < /etc/systemd/system/appname.service < /opt/${APP}_version.txt + +motd_ssh +customize +cleanup_lxc +``` + +### Example 2: Database Application (PHP + MySQL) + +```bash +#!/usr/bin/env bash +source /dev/stdin <<<"$FUNCTIONS_FILE_PATH" +color +catch_errors +setting_up_container +network_check +update_os + +msg_info "Installing Dependencies" +$STD apt-get install -y git curl nginx supervisor +msg_ok "Installed Dependencies" + +msg_info "Setting up PHP" +PHP_VERSION="8.4" PHP_MODULE="bcmath,curl,gd,intl,mbstring,pdo_mysql,redis" setup_php +msg_ok "PHP installed" + +msg_info "Setting up Database" +MARIADB_VERSION="11.4" setup_mariadb +msg_ok "MariaDB installed" + +DB_NAME="appname_db" +DB_USER="appuser" +DB_PASS=$(openssl rand -base64 18 | tr -dc 'a-zA-Z0-9' | head -c13) + +mysql -u root < /opt/${APP}_version.txt + +motd_ssh +customize +cleanup_lxc +``` + +--- + +## Troubleshooting + +### Installation Hangs + +**Symptom**: Script appears to freeze at particular step + +**Causes**: +1. Network connectivity lost +2. Repository server timing out +3. Interactive prompt waiting for input + +**Debug**: +```bash +# Check if process still running +ps aux | grep -i appname + +# Check network +ping -c 1 8.8.8.8 + +# Check apt lock +lsof /var/lib/apt/lists/lock +``` + +### Package Installation Fails + +**Symptom**: `E: Unable to locate package xyz` + +**Causes**: +1. Repository not updated +2. Package name incorrect for OS version +3. Conflicting repository configuration + +**Solution**: +```bash +# Force update +apt-get update --allow-releaseinfo-change +apt-cache search package | grep exact_name +``` + +### Permission Denied on Files + +**Symptom**: Application can't write to `/opt/appname` + +**Causes**: +1. Wrong owner +2. Wrong permissions (644 for files, 755 for directories) + +**Fix**: +```bash +chown -R www-data:www-data /opt/appname +chmod -R 755 /opt/appname +find /opt/appname -type f -exec chmod 644 {} \; +find /opt/appname -type d -exec chmod 755 {} \; +``` + +### Service Won't Start + +**Symptom**: `systemctl status appname` shows failed + +**Debug**: +```bash +# Check service status +systemctl status appname + +# View logs +journalctl -u appname -n 50 + +# Check configuration +systemctl cat appname +``` + +--- + +## Contribution Checklist + +Before submitting a PR: + +### Script Structure +- [ ] Shebang is `#!/usr/bin/env bash` +- [ ] Copyright header with author and source URL +- [ ] Functions loaded via `$FUNCTIONS_FILE_PATH` +- [ ] Initial setup: `color`, `catch_errors`, `setting_up_container`, `network_check`, `update_os` + +### Installation Flow +- [ ] Dependencies installed with `$STD apt-get install -y \` +- [ ] Package names collapsed (`php-{bcmath,curl}`) +- [ ] Tool setup uses functions from tools.func (not manual installation) +- [ ] Application version fetched dynamically (not hardcoded) +- [ ] Version saved to `/opt/${APP}_version.txt` + +### Configuration +- [ ] Configuration files created properly (heredoc or sed) +- [ ] Credentials generated randomly (`openssl rand`) +- [ ] Credentials stored in creds file +- [ ] Passwords use alphanumeric only (no special chars) +- [ ] Proper file permissions set + +### Messaging +- [ ] `msg_info` followed by action then `msg_ok` +- [ ] Error cases use `msg_error` and exit +- [ ] No bare `echo` statements for status (use msg_* functions) + +### Cleanup +- [ ] Temporary files removed +- [ ] Package manager cache cleaned (`autoremove`, `autoclean`) +- [ ] `cleanup_lxc` called at end +- [ ] `motd_ssh` called before `customize` +- [ ] `customize` called before exit + +### Testing +- [ ] Script tested with default OS (Debian 12/Ubuntu 22.04) +- [ ] Script tested with Alpine (if applicable) +- [ ] Script tested with verbose mode (`VERBOSE=yes`) +- [ ] Error handling tested (network interruption, missing packages) +- [ ] Cleanup verified (disk space reduced, temp files removed) + +--- + +## Related Documentation + +- [ct/AppName.sh Guide](UPDATED_APP-ct.md) +- [tools.func Wiki](../misc/tools.func.md) +- [install.func Wiki](../misc/install.func.md) +- [error_handler.func Wiki](../misc/error_handler.func.md) + +--- + +**Last Updated**: December 2025 +**Compatibility**: ProxmoxVED with tools.func v2+ +**Questions?** Open an issue in the repository diff --git a/docs/alpine-install.func.md b/docs/alpine-install.func.md new file mode 100644 index 000000000..f71f4de1f --- /dev/null +++ b/docs/alpine-install.func.md @@ -0,0 +1,651 @@ +# Alpine-Install.func Wiki + +A specialized module for Alpine Linux LXC container setup and configuration, providing functions for IPv6 management, network verification, OS updates, SSH configuration, timezone validation, and passwordless auto-login customization. + +--- + +## πŸ“‹ Table of Contents + +- [Overview](#overview) +- [Initialization & Signal Handling](#initialization--signal-handling) +- [Network & Connectivity Functions](#network--connectivity-functions) +- [OS Configuration Functions](#os-configuration-functions) +- [SSH & MOTD Configuration](#ssh--motd-configuration) +- [Container Customization](#container-customization) +- [Best Practices](#best-practices) +- [Error Handling](#error-handling) +- [Contributing](#contributing) + +--- + +## Overview + +This module provides Alpine Linux-specific installation and configuration functions used inside LXC containers during the setup phase. Key capabilities include: + +- βœ… IPv6 enablement/disablement with persistent configuration +- βœ… Network connectivity verification with retry logic +- βœ… Alpine Linux OS updates via apk package manager +- βœ… SSH daemon and MOTD configuration +- βœ… Passwordless root auto-login setup +- βœ… Timezone validation for Alpine containers +- βœ… Comprehensive error handling with signal traps + +### Integration Pattern + +```bash +# Alpine container scripts load this module via curl +source <(curl -fsSL https://git.community-scripts.org/.../alpine-install.func) +load_functions # Initialize core utilities +catch_errors # Setup error handling and signal traps +``` + +--- + +## Initialization & Signal Handling + +### Module Dependencies + +The module automatically sources two required dependencies: + +```bash +source <(curl -fsSL .../core.func) # Color codes, icons, message functions +source <(curl -fsSL .../error_handler.func) # Error handling and exit codes +load_functions # Initialize color/formatting +catch_errors # Setup trap handlers +``` + +### Signal Trap Configuration + +```bash +set -Eeuo pipefail # Strict error mode +trap 'error_handler $? $LINENO "$BASH_COMMAND"' ERR +trap on_exit EXIT # Cleanup on exit +trap on_interrupt INT # Handle Ctrl+C (SIGINT) +trap on_terminate TERM # Handle SIGTERM +``` + +--- + +## Network & Connectivity Functions + +### `verb_ip6()` + +**Purpose**: Configures IPv6 settings and sets verbose mode based on environment variables. + +**Signature**: +```bash +verb_ip6() +``` + +**Parameters**: None + +**Returns**: No explicit return value (configures system state) + +**Environment Effects**: +- Sets `STD` variable to control output verbosity (via `set_std_mode()`) +- If `DISABLEIPV6=yes`: disables IPv6 system-wide via sysctl +- Modifies `/etc/sysctl.conf` for persistent IPv6 disabled state + +**Implementation Pattern**: +```bash +verb_ip6() { + set_std_mode # Initialize STD="" or STD="silent" + + if [ "$DISABLEIPV6" == "yes" ]; then + $STD sysctl -w net.ipv6.conf.all.disable_ipv6=1 + echo "net.ipv6.conf.all.disable_ipv6 = 1" >>/etc/sysctl.conf + $STD rc-update add sysctl default + fi +} +``` + +**Usage Examples**: + +```bash +# Example 1: With IPv6 disabled +DISABLEIPV6="yes" +VERBOSE="no" +verb_ip6 +# Result: IPv6 disabled, changes persisted to sysctl.conf + +# Example 2: Keep IPv6 enabled (default) +DISABLEIPV6="no" +verb_ip6 +# Result: IPv6 remains enabled, no configuration changes +``` + +--- + +### `setting_up_container()` + +**Purpose**: Verifies network connectivity by checking for assigned IP addresses and retrying if necessary. + +**Signature**: +```bash +setting_up_container() +``` + +**Parameters**: None (uses global `RETRY_NUM` and `RETRY_EVERY`) + +**Returns**: 0 on success; exits with code 1 if network unavailable after retries + +**Environment Side Effects**: +- Requires: `RETRY_NUM` (max attempts, default: 10), `RETRY_EVERY` (seconds between retries, default: 3) +- Uses: `CROSS`, `RD`, `CL`, `GN`, `BL` color variables from core.func +- Calls: `msg_info()`, `msg_ok()` message functions + +**Implementation Pattern**: +```bash +setting_up_container() { + msg_info "Setting up Container OS" + i=$RETRY_NUM # Use global counter + while [ $i -gt 0 ]; do + # Check for non-loopback IPv4 address + if [ "$(ip addr show | grep 'inet ' | grep -v '127.0.0.1' | ...)" != "" ]; then + break + fi + echo 1>&2 -en "${CROSS}${RD} No Network! " + sleep $RETRY_EVERY + i=$((i - 1)) + done + + # If still no network after retries, exit with error + if [ "$(ip addr show | grep 'inet ' | grep -v '127.0.0.1' | ...)" = "" ]; then + exit 1 + fi + msg_ok "Network Connected: ${BL}$(ip addr show | grep 'inet ' | awk '{print $2}' | ...)${CL}" +} +``` + +**Usage Examples**: + +```bash +# Example 1: Network available immediately +RETRY_NUM=10 +RETRY_EVERY=3 +setting_up_container +# Output: +# ℹ️ Setting up Container OS +# βœ”οΈ Set up Container OS +# βœ”οΈ Network Connected: 10.0.3.50 + +# Example 2: Network delayed by 6 seconds (2 retries) +# Script waits 3 seconds x 2, then succeeds +# Output shows retry messages, then success +``` + +--- + +### `network_check()` + +**Purpose**: Comprehensive network connectivity verification for both IPv4 and IPv6, including DNS resolution checks for Git-related domains. + +**Signature**: +```bash +network_check() +``` + +**Parameters**: None + +**Returns**: 0 on success; exits with code 1 if DNS critical failure + +**Environment Side Effects**: +- Temporarily disables error trap (`set +e`, `trap - ERR`) +- Modifies error handling to allow graceful failure detection +- Re-enables error trap at end of function +- Calls: `msg_ok()`, `msg_error()`, `fatal()` message functions + +**Implementation Pattern**: +```bash +network_check() { + set +e + trap - ERR + + # Test IPv4 via multiple DNS servers + if ping -c 1 -W 1 1.1.1.1 &>/dev/null || ...; then + ipv4_status="${GN}βœ”${CL} IPv4" + else + ipv4_status="${RD}βœ–${CL} IPv4" + # Prompt user to continue without internet + fi + + # Verify DNS resolution for GitHub domains + RESOLVEDIP=$(getent hosts github.com | awk '{ print $1 }') + if [[ -z "$RESOLVEDIP" ]]; then + msg_error "Internet: ${ipv4_status} DNS Failed" + else + msg_ok "Internet: ${ipv4_status} DNS: ${BL}${RESOLVEDIP}${CL}" + fi + + set -e + trap 'error_handler $LINENO "$BASH_COMMAND"' ERR +} +``` + +**Usage Examples**: + +```bash +# Example 1: Good connectivity +network_check +# Output: +# βœ”οΈ Network Connected: IPv4 +# βœ”οΈ Internet: βœ” IPv4 DNS: 140.82.113.3 + +# Example 2: No internet, user continues anyway +# Output prompts: "Internet NOT connected. Continue anyway? " +# If user enters 'y': +# ⚠️ Expect Issues Without Internet +``` + +--- + +## OS Configuration Functions + +### `update_os()` + +**Purpose**: Updates Alpine Linux OS packages and installs Alpine-specific tools library for additional setup functions. + +**Signature**: +```bash +update_os() +``` + +**Parameters**: None + +**Returns**: No explicit return value (updates system) + +**Environment Side Effects**: +- Runs `apk update && apk upgrade` +- Sources alpine-tools.func for Alpine-specific package installation helpers +- Uses `$STD` wrapper to suppress output unless `VERBOSE=yes` +- Calls: `msg_info()`, `msg_ok()` message functions + +**Implementation Pattern**: +```bash +update_os() { + msg_info "Updating Container OS" + $STD apk update && $STD apk upgrade + source <(curl -fsSL https://git.community-scripts.org/.../alpine-tools.func) + msg_ok "Updated Container OS" +} +``` + +**Usage Examples**: + +```bash +# Example 1: Standard update +VERBOSE="no" +update_os +# Output: +# ℹ️ Updating Container OS +# βœ”οΈ Updated Container OS +# (Output suppressed via $STD) + +# Example 2: Verbose mode +VERBOSE="yes" +update_os +# Output shows all apk operations plus success message +``` + +--- + +## SSH & MOTD Configuration + +### `motd_ssh()` + +**Purpose**: Configures Message of the Day (MOTD) with container information and enables SSH root access if required. + +**Signature**: +```bash +motd_ssh() +``` + +**Parameters**: None + +**Returns**: No explicit return value (configures system) + +**Environment Side Effects**: +- Modifies `/root/.bashrc` to set TERM environment variable +- Creates `/etc/profile.d/00_lxc-details.sh` with container information script +- If `SSH_ROOT=yes`: modifies `/etc/ssh/sshd_config` and starts SSH daemon +- Uses: `APPLICATION`, `SSH_ROOT` variables from environment +- Requires: color variables (`BOLD`, `YW`, `RD`, `GN`, `CL`) from core.func + +**Implementation Pattern**: +```bash +motd_ssh() { + # Configure TERM for better terminal support + echo "export TERM='xterm-256color'" >>/root/.bashrc + + # Gather OS information + OS_NAME=$(grep ^NAME /etc/os-release | cut -d= -f2 | tr -d '"') + OS_VERSION=$(grep ^VERSION_ID /etc/os-release | cut -d= -f2 | tr -d '"') + IP=$(ip -4 addr show eth0 | awk '/inet / {print $2}' | cut -d/ -f1 | head -n 1) + + # Create MOTD script with container details + PROFILE_FILE="/etc/profile.d/00_lxc-details.sh" + cat > "$PROFILE_FILE" <<'EOF' +echo -e "" +echo -e "${BOLD}${YW}${APPLICATION} LXC Container - DEV Repository${CL}" +echo -e "${RD}WARNING: This is a DEVELOPMENT version (ProxmoxVED). Do NOT use in production!${CL}" +echo -e "${YW} OS: ${GN}${OS_NAME} - Version: ${OS_VERSION}${CL}" +echo -e "${YW} Hostname: ${GN}$(hostname)${CL}" +echo -e "${YW} IP Address: ${GN}${IP}${CL}" +echo -e "${YW} Repository: ${GN}https://github.com/community-scripts/ProxmoxVED${CL}" +echo "" +EOF + + # Enable SSH root access if configured + if [[ "${SSH_ROOT}" == "yes" ]]; then + sed -i "s/#PermitRootLogin prohibit-password/PermitRootLogin yes/g" /etc/ssh/sshd_config + rc-update add sshd + /etc/init.d/sshd start + fi +} +``` + +**Usage Examples**: + +```bash +# Example 1: MOTD configuration with SSH enabled +APPLICATION="MyApp" +SSH_ROOT="yes" +motd_ssh +# Result: SSH daemon started and set to auto-start, MOTD shows app info + +# Example 2: MOTD only (SSH disabled) +APPLICATION="MyApp" +SSH_ROOT="no" +motd_ssh +# Result: MOTD configured but SSH remains disabled +``` + +--- + +## Container Customization + +### `validate_tz()` + +**Purpose**: Validates that a timezone string exists in the Alpine Linux timezone database. + +**Signature**: +```bash +validate_tz() +``` + +**Parameters**: +- `$1` - Timezone string (e.g., "America/New_York", "UTC", "Europe/London") + +**Returns**: 0 if timezone file exists, 1 if invalid + +**Implementation Pattern**: +```bash +validate_tz() { + [[ -f "/usr/share/zoneinfo/$1" ]] # Bash test operator returns success/failure +} +``` + +**Usage Examples**: + +```bash +# Example 1: Valid timezone +validate_tz "America/New_York" +echo $? # Output: 0 + +# Example 2: Invalid timezone +validate_tz "Invalid/Timezone" +echo $? # Output: 1 + +# Example 3: UTC (always valid) +validate_tz "UTC" +echo $? # Output: 0 +``` + +--- + +### `customize()` + +**Purpose**: Configures container for passwordless root auto-login and creates update script for easy application re-deployment. + +**Signature**: +```bash +customize() +``` + +**Parameters**: None (uses global `PASSWORD` and `app` variables) + +**Returns**: No explicit return value (configures system) + +**Environment Side Effects**: +- If `PASSWORD=""` (empty): + * Removes password prompt from root login + * Drops user into shell automatically + * Creates autologin boot script at `/etc/local.d/autologin.start` + * Creates `.hushlogin` to suppress login banners + * Registers script with rc-update +- Creates `/usr/bin/update` script for application updates +- Requires: `app` variable (application name in lowercase) +- Calls: `msg_info()`, `msg_ok()` message functions + +**Implementation Pattern**: +```bash +customize() { + if [[ "$PASSWORD" == "" ]]; then + msg_info "Customizing Container" + + # Remove password requirement + passwd -d root >/dev/null 2>&1 + + # Install util-linux if needed + apk add --no-cache --force-broken-world util-linux >/dev/null 2>&1 + + # Create autologin startup script + mkdir -p /etc/local.d + cat > /etc/local.d/autologin.start <<'EOF' +#!/bin/sh +sed -i 's|^tty1::respawn:.*|tty1::respawn:/sbin/agetty --autologin root --noclear tty1 38400 linux|' /etc/inittab +kill -HUP 1 +EOF + chmod +x /etc/local.d/autologin.start + touch /root/.hushlogin + + rc-update add local >/dev/null 2>&1 + /etc/local.d/autologin.start + + msg_ok "Customized Container" + fi + + # Create update script + echo "bash -c \"\$(curl -fsSL https://github.com/community-scripts/ProxmoxVED/raw/main/ct/${app}.sh)\"" >/usr/bin/update + chmod +x /usr/bin/update +} +``` + +**Usage Examples**: + +```bash +# Example 1: Passwordless auto-login +PASSWORD="" +app="myapp" +customize +# Result: Root login without password, auto-login configured +# User can type: /usr/bin/update to re-run application setup + +# Example 2: Password-protected login +PASSWORD="MySecurePassword" +customize +# Result: Auto-login skipped, password remains active +# Update script still created for re-deployment +``` + +--- + +## Best Practices + +### 1. **Initialization Order** + +Always follow this sequence in Alpine install scripts: + +```bash +#!/bin/sh +set -Eeuo pipefail + +# 1. Ensure curl is available for sourcing functions +if ! command -v curl >/dev/null 2>&1; then + apk update && apk add curl >/dev/null 2>&1 +fi + +# 2. Source dependencies in correct order +source <(curl -fsSL .../core.func) +source <(curl -fsSL .../error_handler.func) + +# 3. Initialize function libraries +load_functions # Sets up colors, formatting, icons +catch_errors # Configures error traps and signal handlers + +# 4. Now safe to call alpine-install.func functions +verb_ip6 +setting_up_container +network_check +update_os +``` + +### 2. **Signal Handling** + +Alpine-install.func provides comprehensive signal trap setup: + +```bash +# ERR trap: Catches all command failures +trap 'error_handler $? $LINENO "$BASH_COMMAND"' ERR + +# EXIT trap: Cleanup on normal or abnormal termination +trap on_exit EXIT + +# INT trap: Handle Ctrl+C gracefully +trap on_interrupt INT + +# TERM trap: Handle SIGTERM signal +trap on_terminate TERM +``` + +### 3. **Network Configuration** + +Use retry logic when network may not be immediately available: + +```bash +setting_up_container # Retries up to RETRY_NUM times +network_check # Validates DNS and Internet +``` + +### 4. **IPv6 Considerations** + +For production Alpine containers: + +```bash +# Disable IPv6 if not needed (reduces attack surface) +DISABLEIPV6="yes" +verb_ip6 + +# Or keep enabled (default): +DISABLEIPV6="no" +# No configuration needed +``` + +### 5. **Error Handling with Color Output** + +Functions use color-coded message output: + +```bash +msg_info # Informational messages (yellow) +msg_ok # Success messages (green) +msg_error # Error messages (red) +msg_warn # Warning messages (orange) +``` + +--- + +## Error Handling + +The module implements comprehensive error handling: + +### Exit Codes + +| Code | Meaning | +|------|---------| +| 0 | Success | +| 1 | General error (network unavailable, DNS failed, etc.) | +| 130 | Interrupted by user (SIGINT) | +| 143 | Terminated by signal (SIGTERM) | + +### Error Handler Function + +The error_handler receives three parameters: + +```bash +error_handler() { + local exit_code="$1" # Exit code from failed command + local line_number="$2" # Line where error occurred + local command="$3" # Command that failed + + # Errors are reported with line number and command details + # Stack trace available for debugging +} +``` + +### Debug Variables + +Available for troubleshooting: + +```bash +$VERBOSE # Set to "yes" to show all output +$DEV_MODE_TRACE # Set to "true" for bash -x tracing +$DEV_MODE_LOGS # Set to "true" to persist logs +``` + +--- + +## Contributing + +### Adding New Functions + +When adding Alpine-specific functions: + +1. Follow the established naming convention: `function_purpose()` +2. Include comprehensive docstring with signature, parameters, returns +3. Use color variables from core.func for output consistency +4. Handle errors via error_handler trap +5. Document all environment variable dependencies + +### Testing New Functions + +```bash +# Test function in isolation with error traps: +set -Eeuo pipefail +trap 'error_handler $? $LINENO "$BASH_COMMAND"' ERR + +source <(curl -fsSL .../core.func) +source <(curl -fsSL .../error_handler.func) +load_functions +catch_errors + +# Now test your function: +your_function +``` + +### Compatibility + +- Alpine Linux 3.16+ (uses ash shell compatible syntax) +- OpenRC init system (rc-update, rc-service) +- Requires: core.func, error_handler.func +- Optional: alpine-tools.func (for extended package management) + +--- + +## Notes + +- Functions are designed for execution **inside** LXC containers (not on Proxmox host) +- Alpine uses `apk` package manager (not `apt`) +- Alpine uses OpenRC (not systemd) - use `rc-update` and `/etc/init.d/` commands +- IPv6 can be disabled for security/performance but is enabled by default +- Auto-login configuration persists across container reboots via rc-update + diff --git a/docs/alpine-tools.func.md b/docs/alpine-tools.func.md new file mode 100644 index 000000000..5271469c3 --- /dev/null +++ b/docs/alpine-tools.func.md @@ -0,0 +1,588 @@ +# Alpine-Tools.func Wiki + +Alpine Linux-specific tool setup and package management module providing helper functions optimized for Alpine's apk package manager and minimal container environment. + +--- + +## πŸ“‹ Table of Contents + +- [Overview](#overview) +- [Helper Functions](#helper-functions) +- [GitHub Release Functions](#github-release-functions) +- [Tool Installation Patterns](#tool-installation-patterns) +- [Package Management](#package-management) +- [Best Practices](#best-practices) +- [Debugging](#debugging) +- [Contributing](#contributing) + +--- + +## Overview + +Alpine-tools.func provides **Alpine Linux-specific utilities**: + +- βœ… Alpine apk package manager wrapper +- βœ… GitHub release version checking and installation +- βœ… Tool caching and version tracking +- βœ… Progress reporting with pv (pipe viewer) +- βœ… Network resolution helpers for Alpine +- βœ… PATH persistence across sessions +- βœ… Retry logic for failed downloads +- βœ… Minimal dependencies philosophy (Alpine ~5MB containers) + +### Key Differences from Debian/Ubuntu + +| Feature | Alpine | Debian/Ubuntu | +|---------|--------|---------------| +| Package Manager | apk | apt-get, dpkg | +| Shell | ash (dash variant) | bash | +| Init System | OpenRC | systemd | +| Size | ~5MB base | ~100MB+ base | +| Libc | musl | glibc | +| Find getent | Not installed | Installed | + +### Integration Pattern + +```bash +#!/bin/sh # Alpine uses ash, not bash +source <(curl -fsSL .../core.func) +source <(curl -fsSL .../alpine-tools.func) +load_functions + +# Now Alpine-specific tool functions available +need_tool curl jq # Install if missing +check_for_gh_release "myapp" "owner/repo" +``` + +--- + +## Helper Functions + +### `lower()` + +**Purpose**: Converts string to lowercase (portable ash function). + +**Signature**: +```bash +lower() +``` + +**Parameters**: +- `$1` - String to convert + +**Returns**: Lowercase string on stdout + +**Behavior**: +```bash +# Alpine's tr works with character classes +printf '%s' "$1" | tr '[:upper:]' '[:lower:]' +``` + +**Usage Examples**: + +```bash +# Example 1: App name normalization +result=$(lower "MyApp") +echo "$result" # Output: myapp + +# Example 2: In variable assignment +app_dir=$(lower "$APPLICATION") +mkdir -p /opt/$app_dir +``` + +--- + +### `has()` + +**Purpose**: Checks if command is available in PATH. + +**Signature**: +```bash +has() +``` + +**Parameters**: +- `$1` - Command name + +**Returns**: 0 if available, 1 if not + +**Implementation**: +```bash +has() { + command -v "$1" >/dev/null 2>&1 +} +``` + +**Usage Examples**: + +```bash +# Example 1: Check availability +if has jq; then + echo "jq is installed" +else + echo "jq is not installed" +fi + +# Example 2: In conditionals +has docker && docker ps || echo "Docker not installed" +``` + +--- + +### `need_tool()` + +**Purpose**: Ensures specified tools are installed, installs missing ones via apk. + +**Signature**: +```bash +need_tool() +``` + +**Parameters**: +- `$@` - Tool names (space-separated) + +**Returns**: 0 on success, 1 if installation failed + +**Behavior**: +```bash +# Checks each tool +# If any missing: runs apk add for all +# Displays message before and after +``` + +**Error Handling**: +- Returns 1 if apk add fails +- Shows which tools failed +- Suggests checking package names + +**Usage Examples**: + +```bash +# Example 1: Ensure common tools available +need_tool curl jq unzip git +# Installs any missing packages + +# Example 2: Optional tool check +if need_tool myapp-cli; then + myapp-cli --version +else + echo "myapp-cli not available in apk" +fi + +# Example 3: With error handling +need_tool docker || { + echo "Failed to install docker" + exit 1 +} +``` + +--- + +### `net_resolves()` + +**Purpose**: Checks if hostname resolves and responds (Alpine-friendly DNS test). + +**Signature**: +```bash +net_resolves() +``` + +**Parameters**: +- `$1` - Hostname to test + +**Returns**: 0 if resolves and responds, 1 if fails + +**Behavior**: +```bash +# Alpine doesn't have getent by default +# Falls back to nslookup if ping fails +# Returns success if either works + +ping -c1 -W1 "$host" >/dev/null 2>&1 || nslookup "$host" >/dev/null 2>&1 +``` + +**Usage Examples**: + +```bash +# Example 1: Test GitHub connectivity +if net_resolves api.github.com; then + echo "Can reach GitHub API" +else + echo "GitHub API unreachable" +fi + +# Example 2: In download function +net_resolves download.example.com || { + echo "Download server not reachable" + exit 1 +} +``` + +--- + +### `ensure_usr_local_bin_persist()` + +**Purpose**: Ensures `/usr/local/bin` is in PATH across all shell sessions. + +**Signature**: +```bash +ensure_usr_local_bin_persist() +``` + +**Parameters**: None + +**Returns**: No explicit return value (modifies system) + +**Behavior**: +```bash +# Creates /etc/profile.d/10-localbin.sh +# Script adds /usr/local/bin to PATH if not already present +# Runs on every shell startup + +# Alpine uses /etc/profile for login shells +# profile.d scripts sourced automatically +``` + +**Implementation**: +```bash +PROFILE_FILE="/etc/profile.d/10-localbin.sh" +if [ ! -f "$PROFILE_FILE" ]; then + echo 'case ":$PATH:" in *:/usr/local/bin:*) ;; *) export PATH="/usr/local/bin:$PATH";; esac' > "$PROFILE_FILE" + chmod +x "$PROFILE_FILE" +fi +``` + +**Usage Examples**: + +```bash +# Example 1: Make sure local tools available +ensure_usr_local_bin_persist +# Now /usr/local/bin binaries always in PATH + +# Example 2: After installing custom tool +cp ./my-tool /usr/local/bin/ +ensure_usr_local_bin_persist +# Tool immediately accessible in PATH +``` + +--- + +### `download_with_progress()` + +**Purpose**: Downloads file with progress bar (if pv available) or simple # progress. + +**Signature**: +```bash +download_with_progress() +``` + +**Parameters**: +- `$1` - URL to download +- `$2` - Destination file path + +**Returns**: 0 on success, 1 on failure + +**Behavior**: +```bash +# Attempts to get content-length header +# If available: pipes through pv for progress bar +# If not: uses curl's built-in # progress +# Shows errors clearly +``` + +**Requirements**: +- `curl` - For downloading +- `pv` - Optional, for progress bar +- Destination directory must exist + +**Usage Examples**: + +```bash +# Example 1: Simple download +download_with_progress "https://example.com/file.tar.gz" "/tmp/file.tar.gz" +# Shows progress bar if pv available + +# Example 2: With error handling +if download_with_progress "$URL" "$DEST"; then + echo "Downloaded successfully" + tar -xzf "$DEST" +else + echo "Download failed" + exit 1 +fi +``` + +--- + +## GitHub Release Functions + +### `check_for_gh_release()` + +**Purpose**: Checks GitHub releases for available updates and compares with currently installed version. + +**Signature**: +```bash +check_for_gh_release() +``` + +**Parameters**: +- `$1` - Application name (e.g., "nodejs") +- `$2` - GitHub repository (e.g., "nodejs/node") +- `$3` - Pinned version (optional, e.g., "20.0.0") + +**Returns**: 0 if update needed, 1 if current or pinned + +**Environment Variables Set**: +- `CHECK_UPDATE_RELEASE` - Latest available version (without v prefix) + +**Behavior**: +```bash +# 1. Check network to api.github.com +# 2. Fetch latest release tag via GitHub API +# 3. Compare with installed version (stored in ~/.appname) +# 4. Show appropriate message: +# - "app pinned to vX.X.X (no update)" +# - "app pinned vX.X.X (upstream vY.Y.Y) β†’ update/downgrade" +# - "Update available: vA.A.A β†’ vB.B.B" +# - "Already up to date" +``` + +**File Storage**: +```bash +~/.${app_lc} # File contains current version string +# Example: ~/.nodejs contains "20.10.0" +``` + +**Usage Examples**: + +```bash +# Example 1: Check for update +check_for_gh_release "nodejs" "nodejs/node" +# Output: "Update available: v18.0.0 β†’ v20.10.0" +# Sets: CHECK_UPDATE_RELEASE="20.10.0" + +# Example 2: Pinned version (no update) +check_for_gh_release "nodejs" "nodejs/node" "20.0.0" +# Output: "app pinned to v20.0.0 (no update)" +# Returns 1 (no update available) + +# Example 3: With error handling +if check_for_gh_release "myapp" "owner/myapp"; then + echo "Update available: $CHECK_UPDATE_RELEASE" + download_and_install +fi +``` + +--- + +## Tool Installation Patterns + +### Pattern 1: Simple Package Installation + +```bash +#!/bin/sh +need_tool curl jq # Ensure tools available +# Continue with script +``` + +### Pattern 2: GitHub Release Installation + +```bash +#!/bin/sh +source <(curl -fsSL .../alpine-tools.func) +load_functions + +# Check for updates +check_for_gh_release "myapp" "owner/myapp" + +# Download from GitHub releases +RELEASE="$CHECK_UPDATE_RELEASE" +URL="https://github.com/owner/myapp/releases/download/v${RELEASE}/myapp-alpine.tar.gz" + +download_with_progress "$URL" "/tmp/myapp-${RELEASE}.tar.gz" +tar -xzf "/tmp/myapp-${RELEASE}.tar.gz" -C /usr/local/bin/ +``` + +### Pattern 3: Version Pinning + +```bash +#!/bin/sh +# For specific use case, pin to known good version +check_for_gh_release "nodejs" "nodejs/node" "20.10.0" +# Will use 20.10.0 even if 21.0.0 available +``` + +--- + +## Package Management + +### Alpine Package Naming + +Alpine packages often have different names than Debian: + +| Tool | Alpine | Debian | +|------|--------|--------| +| curl | curl | curl | +| Git | git | git | +| Docker | docker | docker.io | +| PostgreSQL | postgresql-client | postgresql-client | +| Build tools | build-base | build-essential | +| Development headers | -dev packages | -dev packages | + +### Finding Alpine Packages + +```bash +# Search for package +apk search myapp + +# Show package info +apk info -d myapp + +# List available versions +apk search myapp --all +``` + +### Installing Alpine Packages + +```bash +# Basic install (not cached) +apk add curl git + +# Install with --no-cache (for containers) +apk add --no-cache curl git + +# Force broken packages (last resort) +apk add --no-cache --force-broken-world util-linux +``` + +--- + +## Best Practices + +### 1. **Use `--no-cache` in Containers** + +```bash +# Good: Saves space in container +apk add --no-cache curl git + +# Avoid: Wastes space +apk update && apk add curl git +``` + +### 2. **Check Tools Before Using** + +```bash +# Good: Graceful error +if ! has jq; then + need_tool jq || exit 1 +fi + +# Using jq safely +jq . < input.json +``` + +### 3. **Use need_tool() for Multiple** + +```bash +# Good: Install all at once +need_tool curl jq git unzip + +# Less efficient: Individual checks +has curl || apk add curl +has jq || apk add jq +``` + +### 4. **Ensure Persistence** + +```bash +# For custom tools in /usr/local/bin +ensure_usr_local_bin_persist + +# Now available in all future shells +/usr/local/bin/my-custom-tool +``` + +### 5. **Handle Network Failures** + +```bash +# Alpine often in isolated environments +if ! net_resolves api.github.com; then + echo "GitHub API unreachable" + # Fallback to local package or error + exit 1 +fi +``` + +--- + +## Debugging + +### Check Package Availability + +```bash +# List all available packages +apk search --all + +# Find package by keyword +apk search curl + +# Get specific package info +apk info postgresql-client +``` + +### Verify Installation + +```bash +# Check if tool installed +apk info | grep myapp + +# Verify PATH +which curl +echo $PATH +``` + +### Network Testing + +```bash +# Test DNS +nslookup api.github.com + +# Test connectivity +ping -c1 1.1.1.1 + +# Test download +curl -I https://api.github.com +``` + +--- + +## Contributing + +### Adding New Helper Functions + +When adding Alpine-specific helpers: + +1. Use POSIX shell (ash-compatible) +2. Avoid bash-isms +3. Include error handling +4. Document with examples +5. Test on actual Alpine container + +### Improving Package Installation + +New patterns could support: +- Automatic Alpine version detection +- Package version pinning +- Dependency resolution +- Conflict detection + +--- + +## Notes + +- Alpine uses **ash shell** (POSIX-compatible, not bash) +- Alpine **apk is fast** and has minimal overhead +- Alpine containers **~5MB base image** (vs 100MB+ for Debian) +- **No getent available** by default (use nslookup fallback) +- GitHub releases can be **pre-compiled for Alpine musl** + diff --git a/docs/build.func.md b/docs/build.func.md new file mode 100644 index 000000000..c7f8f807b --- /dev/null +++ b/docs/build.func.md @@ -0,0 +1,584 @@ +# Build.func Wiki + +Central LXC container build and configuration orchestration engine providing the main creation workflow, 19-step advanced wizard, defaults system, variable management, and state machine for container lifecycle. + +--- + +## πŸ“‹ Table of Contents + +- [Overview](#overview) +- [Core Functions](#core-functions) +- [Variable Management](#variable-management) +- [Build Workflow](#build-workflow) +- [Advanced Settings Wizard](#advanced-settings-wizard) +- [Defaults System](#defaults-system) +- [Best Practices](#best-practices) +- [Development Mode](#development-mode) +- [Contributing](#contributing) + +--- + +## Overview + +Build.func is the **3800+ line orchestration engine** for LXC container creation: + +- βœ… 19-step interactive advanced settings wizard +- βœ… 3-tier defaults precedence system (app β†’ user β†’ global) +- βœ… Variable whitelisting for security +- βœ… State machine workflow management +- βœ… Container resource allocation (CPU, RAM, disk) +- βœ… Storage selection and validation +- βœ… Network configuration (bridge, MAC, VLAN, IPv6) +- βœ… Session tracking and logging +- βœ… Comprehensive pre-flight validation checks + +### Execution Flow + +``` +Script Invocation + ↓ +variables() β†’ Initialize core variables, SESSION_ID, UUID + ↓ +build.func functions sourced + ↓ +Pre-flight checks (maxkeys, template availability) + ↓ +Create container (pct create ...) + ↓ +Network configuration + ↓ +Storage tuning + ↓ +Installation script execution + ↓ +Completion & cleanup +``` + +--- + +## Core Functions + +### `variables()` + +**Purpose**: Initializes all core variables, generates unique session ID, and captures application defaults for precedence logic. + +**Signature**: +```bash +variables() +``` + +**Parameters**: None + +**Returns**: No explicit return value (sets global variables) + +**Variables Initialized**: + +| Variable | Source | Purpose | +|----------|--------|---------| +| `NSAPP` | `APP` converted to lowercase | Normalized app name | +| `var_install` | `${NSAPP}-install` | Installation script name | +| `PVEHOST_NAME` | `hostname` | Proxmox hostname | +| `DIAGNOSTICS` | Set to "yes" | Enable telemetry | +| `METHOD` | Set to "default" | Setup method | +| `RANDOM_UUID` | `/proc/sys/kernel/random/uuid` | Session UUID | +| `SESSION_ID` | First 8 chars of UUID | Short session ID | +| `BUILD_LOG` | `/tmp/create-lxc-${SESSION_ID}.log` | Host-side log file | +| `PVEVERSION` | `pveversion` | Proxmox VE version | +| `KERNEL_VERSION` | `uname -r` | System kernel version | + +**App Default Capture** (3-tier precedence): +```bash +# Tier 1: App-declared defaults (highest priority) +APP_DEFAULT_CPU=${var_cpu:-} +APP_DEFAULT_RAM=${var_ram:-} +APP_DEFAULT_DISK=${var_disk:-} + +# Tier 2: User configuration (~/.community-scripts/defaults) +# Tier 3: Global defaults (built-in) +``` + +**Dev Mode Setup**: +```bash +# Parse dev_mode early for special behaviors +parse_dev_mode + +# If dev_mode=logs, use persistent logging location +if [[ "${DEV_MODE_LOGS}" == "true" ]]; then + mkdir -p /var/log/community-scripts + BUILD_LOG="/var/log/community-scripts/create-lxc-${SESSION_ID}-$(date +%Y%m%d_%H%M%S).log" +fi +``` + +**Usage Examples**: + +```bash +# Example 1: Initialize with default app +APP="Jellyfin" +variables +# Result: +# NSAPP="jellyfin" +# SESSION_ID="550e8400" +# BUILD_LOG="/tmp/create-lxc-550e8400.log" + +# Example 2: With dev mode +dev_mode="trace,logs" +APP="MyApp" +variables +# Result: +# Persistent logging enabled +# Bash tracing configured +# BUILD_LOG="/var/log/community-scripts/create-lxc-550e8400-20241201_103000.log" +``` + +--- + +### `maxkeys_check()` + +**Purpose**: Validates kernel keyring limits don't prevent container creation (prevents "key quota exceeded" errors). + +**Signature**: +```bash +maxkeys_check() +``` + +**Parameters**: None + +**Returns**: 0 if limits acceptable; exits with error if exceeded + +**Checks**: +- `/proc/sys/kernel/keys/maxkeys` - Maximum keys per user +- `/proc/sys/kernel/keys/maxbytes` - Maximum key bytes per user +- `/proc/key-users` - Current usage for UID 100000 (LXC user) + +**Warning Thresholds**: +- Keys: Current >= (maxkeys - 100) +- Bytes: Current >= (maxbytes - 1000) + +**Recovery Suggestions**: +```bash +# If warning triggered, suggests sysctl configuration +sysctl -w kernel.keys.maxkeys=200000 +sysctl -w kernel.keys.maxbytes=40000000 + +# Add to persistent config +echo "kernel.keys.maxkeys=200000" >> /etc/sysctl.d/98-community-scripts.conf +sysctl -p +``` + +**Usage Examples**: + +```bash +# Example 1: Healthy keyring usage +maxkeys_check +# Silent success: Usage is normal + +# Example 2: Near limit +maxkeys_check +# Warning displayed with suggested sysctl values +# Allows continuation but recommends tuning + +# Example 3: Exceeded limit +maxkeys_check +# Error: Exits with code 1 +# Suggests increasing limits before retry +``` + +--- + +## Variable Management + +### `default_var_settings()` + +**Purpose**: Loads or creates default variable settings with 3-tier precedence. + +**Signature**: +```bash +default_var_settings() +``` + +**Precedence Order**: +``` +1. App-declared defaults (var_cpu, var_ram, var_disk from script) +2. User defaults (~/.community-scripts/defaults.sh) +3. Global built-in defaults +``` + +**User Defaults Location**: +```bash +~/.community-scripts/defaults.sh +``` + +**Example User Defaults File**: +```bash +# ~/.community-scripts/defaults.sh +CORE_COUNT=4 # Override default CPU +RAM_SIZE=4096 # Override default RAM (MB) +DISK_SIZE=32 # Override default disk (GB) +BRIDGE="vmbr0" # Preferred bridge +STORAGE="local-lvm" # Preferred storage +DISABLEIPV6="no" # Network preference +VERBOSE="no" # Output preference +``` + +--- + +### `load_vars_file()` + +**Purpose**: Loads saved container variables from previous configuration. + +**Signature**: +```bash +load_vars_file() +``` + +**Parameters**: None + +**Returns**: 0 if loaded; 1 if no saved config found + +**File Location**: +```bash +~/.community-scripts/${NSAPP}.vars +``` + +**Variables Loaded**: +- All whitelist-approved variables (CORE_COUNT, RAM_SIZE, DISK_SIZE, etc.) +- Saved settings from previous container creation + +**Usage Examples**: + +```bash +# Example 1: Load previous config +if load_vars_file; then + msg_ok "Loaded previous settings for $NSAPP" +else + msg_info "No previous configuration found, using defaults" +fi + +# Example 2: Offer to use saved config +# Interactive: Prompts user to confirm previously saved values +``` + +--- + +### `maybe_offer_save_app_defaults()` + +**Purpose**: Optionally saves current configuration for reuse in future container creations. + +**Signature**: +```bash +maybe_offer_save_app_defaults() +``` + +**Parameters**: None + +**Returns**: No explicit return value (saves or skips) + +**Behavior**: +- Prompts user if they want to save current settings +- Saves to `~/.community-scripts/${NSAPP}.vars` +- User can load these settings in future runs via `load_vars_file()` +- Saves whitelisted variables only (security) + +**Variables Saved**: +- CORE_COUNT, RAM_SIZE, DISK_SIZE +- BRIDGE, STORAGE, MACADDRESS +- VLAN_TAG, DISABLEIPV6 +- PASSWORD settings +- Custom network configuration + +**Usage Examples**: + +```bash +# Example 1: After configuration +configure_container +# ... all settings done ... +maybe_offer_save_app_defaults +# Prompts: "Save these settings for future use? [y/n]" +# If yes: Saves to ~/.community-scripts/jellyfin.vars + +# Example 2: Reload in next run +# User runs script again +# Prompted: "Use saved settings from last time? [y/n]" +# If yes: Load_vars_file() populates all variables +``` + +--- + +## Build Workflow + +### `install_script()` + +**Purpose**: Orchestrates container installation workflow inside the LXC container. + +**Signature**: +```bash +install_script() +``` + +**Parameters**: None (uses global `NSAPP` variable) + +**Returns**: 0 on success; exits with error code on failure + +**Installation Steps**: +1. Copy install script into container +2. Execute via `pct exec $CTID bash /tmp/...` +3. Capture output and exit code +4. Report completion to API +5. Handle errors with cleanup + +**Error Handling**: +```bash +# If installation fails: +# - Captures exit code +# - Posts failure to API (if telemetry enabled) +# - Displays error with explanation +# - Offers debug shell (if DEV_MODE_BREAKPOINT) +# - Cleans up container (unless DEV_MODE_KEEP) +``` + +--- + +## Advanced Settings Wizard + +### `advanced_settings()` + +**Purpose**: Interactive 19-step wizard for advanced container configuration. + +**Signature**: +```bash +advanced_settings() +``` + +**Parameters**: None + +**Returns**: No explicit return value (populates variables) + +**Wizard Steps** (19 total): +1. **CPU Cores** - Allocation (1-128) +2. **RAM Size** - Allocation in MB (256-65536) +3. **Disk Size** - Allocation in GB (1-4096) +4. **Storage** - Select storage backend (local, local-lvm, etc.) +5. **Bridge** - Network bridge (vmbr0, vmbr1, etc.) +6. **MAC Address** - Custom or auto-generated +7. **VLAN Tag** - Optional VLAN configuration +8. **IPv6** - Enable/disable IPv6 +9. **Disable IPV6** - Explicit disable option +10. **DHCP** - DHCP or static IP +11. **IP Configuration** - If static: IP/mask +12. **Gateway** - Network gateway +13. **DNS** - DNS server configuration +14. **Hostname** - Container hostname +15. **Root Password** - Set or leave empty (auto-login) +16. **SSH Access** - Enable root SSH +17. **Features** - FUSE, Nesting, keyctl, etc. +18. **Start on Boot** - Autostart configuration +19. **Privileged Mode** - Privileged or unprivileged container + +**User Input Methods**: +- Whiptail dialogs (graphical) +- Command-line prompts (fallback) +- Validation of all inputs +- Confirmation summary before creation + +**Usage Examples**: + +```bash +# Example 1: Run wizard +advanced_settings +# User prompted for each of 19 settings +# Responses stored in variables + +# Example 2: Scripted (skip prompts) +CORE_COUNT=4 +RAM_SIZE=4096 +DISK_SIZE=32 +# ... set all 19 variables ... +# advanced_settings() skips prompts since variables already set +``` + +--- + +## Defaults System + +### 3-Tier Precedence Logic + +**Tier 1 (Highest Priority): App-Declared Defaults** +```bash +# In app script header (before default.vars sourced): +var_cpu=4 +var_ram=2048 +var_disk=20 + +# If user has higher value in tier 2/3, app value takes precedence +``` + +**Tier 2 (Medium Priority): User Defaults** +```bash +# In ~/.community-scripts/defaults.sh: +CORE_COUNT=6 +RAM_SIZE=4096 +DISK_SIZE=32 + +# Can be overridden by app defaults (tier 1) +``` + +**Tier 3 (Lowest Priority): Global Built-in Defaults** +```bash +# Built into build.func: +CORE_COUNT=2 (default) +RAM_SIZE=2048 (default, in MB) +DISK_SIZE=8 (default, in GB) +``` + +**Resolution Algorithm**: +```bash +# For CPU cores (example): +if [ -n "$APP_DEFAULT_CPU" ]; then + CORE_COUNT=$APP_DEFAULT_CPU # Tier 1 wins +elif [ -n "$USER_DEFAULT_CPU" ]; then + CORE_COUNT=$USER_DEFAULT_CPU # Tier 2 +else + CORE_COUNT=2 # Tier 3 (global) +fi +``` + +--- + +## Best Practices + +### 1. **Always Call variables() First** + +```bash +#!/bin/bash +source <(curl -fsSL .../build.func) +load_functions +catch_errors + +# Must be first real function call +variables + +# Then safe to use SESSION_ID, BUILD_LOG, etc. +msg_info "Building container (Session: $SESSION_ID)" +``` + +### 2. **Declare App Defaults Before Sourcing build.func** + +```bash +#!/bin/bash +# Declare app defaults BEFORE sourcing build.func +var_cpu=4 +var_ram=4096 +var_disk=20 + +source <(curl -fsSL .../build.func) +variables # These defaults are captured + +# Now var_cpu, var_ram, var_disk are in APP_DEFAULT_* +``` + +### 3. **Use Variable Whitelisting** + +```bash +# Only these variables are allowed to be saved/loaded: +WHITELIST="CORE_COUNT RAM_SIZE DISK_SIZE BRIDGE STORAGE MACADDRESS VLAN_TAG DISABLEIPV6" + +# Sensitive variables are NEVER saved: +# PASSWORD, SSH keys, API tokens, etc. +``` + +### 4. **Check Pre-flight Conditions** + +```bash +variables +maxkeys_check # Validate kernel limits +pve_check # Validate PVE version +arch_check # Validate architecture + +# Only proceed after all checks pass +msg_ok "Pre-flight checks passed" +``` + +### 5. **Track Sessions** + +```bash +# Use SESSION_ID in all logs +BUILD_LOG="/tmp/create-lxc-${SESSION_ID}.log" + +# Keep logs for troubleshooting +# Can be reviewed later: tail -50 /tmp/create-lxc-550e8400.log +``` + +--- + +## Development Mode + +### Dev Mode Variables + +Set via environment or in script: + +```bash +dev_mode="trace,keep,breakpoint" +parse_dev_mode + +# Enables: +# - DEV_MODE_TRACE=true (bash -x) +# - DEV_MODE_KEEP=true (never delete container) +# - DEV_MODE_BREAKPOINT=true (shell on error) +``` + +### Debug Container Creation + +```bash +# Run with all debugging enabled +dev_mode="trace,keep,logs" bash ct/jellyfin.sh + +# Then review logs: +tail -200 /var/log/community-scripts/create-lxc-*.log + +# Container stays running (DEV_MODE_KEEP) +# Allows ssh inspection: ssh root@ +``` + +--- + +## Contributing + +### Adding New Wizard Steps + +1. Add step number and variable to documentation +2. Add whiptail prompt in `advanced_settings()` +3. Add validation logic +4. Add to whitelist if user should save it +5. Update documentation with examples + +### Extending Defaults System + +To add new tier or change precedence: + +1. Update 3-tier logic section +2. Modify resolution algorithm +3. Document new precedence order +4. Update whitelist accordingly + +### Testing Build Workflow + +```bash +# Test with dry-run mode +dev_mode="dryrun" bash ct/myapp.sh +# Shows all commands without executing + +# Test with keep mode +dev_mode="keep" bash ct/myapp.sh +# Container stays if fails, allows inspection +``` + +--- + +## Notes + +- Build.func is **large and complex** (3800+ lines) - handles most container creation logic +- Variables are **passed to container** via pct set/environment +- Session ID **enables request tracking** across distributed logs +- Defaults system is **flexible** (3-tier precedence) +- Pre-flight checks **prevent many common errors** + diff --git a/docs/cloud-init.func.md b/docs/cloud-init.func.md new file mode 100644 index 000000000..2bb1ae41d --- /dev/null +++ b/docs/cloud-init.func.md @@ -0,0 +1,571 @@ +# Cloud-Init.func Wiki + +VM cloud-init configuration and first-boot setup module for Proxmox VEs, providing automatic system initialization, network configuration, user account setup, and SSH key management for virtual machines. + +--- + +## πŸ“‹ Table of Contents + +- [Overview](#overview) +- [Cloud-Init Fundamentals](#cloud-init-fundamentals) +- [Main Configuration Functions](#main-configuration-functions) +- [Interactive Configuration](#interactive-configuration) +- [Configuration Parameters](#configuration-parameters) +- [Data Formats](#data-formats) +- [Best Practices](#best-practices) +- [Troubleshooting](#troubleshooting) +- [Contributing](#contributing) + +--- + +## Overview + +Cloud-init.func provides **VM first-boot automation** infrastructure: + +- βœ… Cloud-init drive creation (IDE2 or SCSI fallback) +- βœ… User account and password configuration +- βœ… SSH public key injection +- βœ… Network configuration (DHCP or static IP) +- βœ… DNS and search domain setup +- βœ… Interactive whiptail-based configuration +- βœ… Credential file generation and display +- βœ… Support for Debian nocloud and Ubuntu cloud-init +- βœ… System package upgrade on first boot + +### Integration Pattern + +```bash +# In Proxmox VM creation scripts +source <(curl -fsSL .../cloud-init.func) + +# Basic setup: +setup_cloud_init "$VMID" "$STORAGE" "$HOSTNAME" "yes" + +# Interactive setup: +configure_cloud_init_interactive "root" +setup_cloud_init "$VMID" "$STORAGE" "$HOSTNAME" "$CLOUDINIT_ENABLE" +``` + +### First-Boot Sequence + +``` +VM Power On + ↓ +Cloud-init boot phase + ↓ +Read cloud-init config + ↓ +Create/modify user account + ↓ +Install SSH keys + ↓ +Configure network + ↓ +Set DNS/search domain + ↓ +Upgrade packages (if configured) + ↓ +Boot completion +``` + +--- + +## Cloud-Init Fundamentals + +### What is Cloud-Init? + +Cloud-init is a system that runs on the first boot of a VM/Instance to: +- Create user accounts +- Set passwords +- Configure networking +- Install SSH keys +- Run custom scripts +- Manage system configuration + +### Proxmox Cloud-Init Integration + +Proxmox VE supports cloud-init natively via: +- **Cloud-init drive**: IDE2 or SCSI disk with cloud-init data +- **QEMU parameters**: User, password, SSH keys, IP configuration +- **First-boot services**: systemd services that execute on first boot + +### Nocloud Data Source + +Proxmox uses the **nocloud** data source (no internet required): +- Configuration stored on local cloud-init drive +- No external network call needed +- Works in isolated networks +- Suitable for private infrastructure + +--- + +## Main Configuration Functions + +### `setup_cloud_init()` + +**Purpose**: Configures Cloud-init for automatic VM first-boot setup. + +**Signature**: +```bash +setup_cloud_init() +``` + +**Parameters**: +- `$1` - VMID (required, e.g., 100) +- `$2` - Storage name (required, e.g., local, local-lvm) +- `$3` - Hostname (optional, default: vm-${VMID}) +- `$4` - Enable Cloud-Init (yes/no, default: no) +- `$5` - User (optional, default: root) +- `$6` - Network mode (dhcp/static, default: dhcp) +- `$7` - Static IP (optional, CIDR format: 192.168.1.100/24) +- `$8` - Gateway (optional) +- `$9` - Nameservers (optional, space-separated, default: 1.1.1.1 8.8.8.8) + +**Returns**: 0 on success, 1 on failure; exits if not enabled + +**Behavior**: +```bash +# If enable="no": +# Returns immediately (skips all configuration) + +# If enable="yes": +# 1. Create cloud-init drive (IDE2, fallback to SCSI1) +# 2. Set user account +# 3. Generate random password +# 4. Configure network +# 5. Set DNS servers +# 6. Add SSH keys (if available) +# 7. Save credentials to file +# 8. Export variables for calling script +``` + +**Operations**: + +| Operation | Command | Purpose | +|-----------|---------|---------| +| Create drive | `qm set $vmid --ide2 $storage:cloudinit` | Cloud-init data disk | +| Set user | `qm set $vmid --ciuser $ciuser` | Initial user | +| Set password | `qm set $vmid --cipassword $cipassword` | Auto-generated | +| SSH keys | `qm set $vmid --sshkeys $SSH_KEYS_FILE` | Pre-injected | +| DHCP network | `qm set $vmid --ipconfig0 ip=dhcp` | Dynamic IP | +| Static network | `qm set $vmid --ipconfig0 ip=192.168.1.100/24,gw=192.168.1.1` | Fixed IP | +| DNS | `qm set $vmid --nameserver $servers` | 1.1.1.1 8.8.8.8 | +| Search domain | `qm set $vmid --searchdomain local` | Local domain | + +**Environment Variables Set**: +- `CLOUDINIT_USER` - Username configured +- `CLOUDINIT_PASSWORD` - Generated password (in memory only) +- `CLOUDINIT_CRED_FILE` - Path to credentials file + +**Usage Examples**: + +```bash +# Example 1: Basic DHCP setup +VMID=100 +STORAGE="local-lvm" +setup_cloud_init "$VMID" "$STORAGE" "myvm" "yes" +# Result: VM configured with DHCP, random password, root user + +# Example 2: Static IP configuration +setup_cloud_init "$VMID" "$STORAGE" "myvm" "yes" "root" \ + "static" "192.168.1.100/24" "192.168.1.1" "1.1.1.1 8.8.8.8" +# Result: VM configured with static IP, specific DNS + +# Example 3: Disabled (no cloud-init) +setup_cloud_init "$VMID" "$STORAGE" "myvm" "no" +# Result: Function returns immediately, no configuration +``` + +--- + +### `configure_cloud_init_interactive()` + +**Purpose**: Interactive whiptail-based configuration prompts for user preferences. + +**Signature**: +```bash +configure_cloud_init_interactive() +``` + +**Parameters**: +- `$1` - Default user (optional, default: root) + +**Returns**: 0 on success, 1 if whiptail unavailable; exports configuration variables + +**Environment Variables Exported**: +- `CLOUDINIT_ENABLE` - Enable (yes/no) +- `CLOUDINIT_USER` - Username +- `CLOUDINIT_NETWORK_MODE` - dhcp or static +- `CLOUDINIT_IP` - Static IP (if static mode) +- `CLOUDINIT_GW` - Gateway (if static mode) +- `CLOUDINIT_DNS` - DNS servers (space-separated) + +**User Prompts** (5 questions): +1. **Enable Cloud-Init?** (yes/no) +2. **Username?** (default: root) +3. **Network Mode?** (DHCP or static) +4. **Static IP?** (if static, CIDR format) +5. **Gateway IP?** (if static) +6. **DNS Servers?** (default: 1.1.1.1 8.8.8.8) + +**Fallback Behavior**: +- If whiptail unavailable: Shows warning and returns 1 +- Auto-defaults to DHCP if error occurs +- Non-interactive: Can be skipped in scripts + +**Implementation Pattern**: +```bash +configure_cloud_init_interactive() { + local default_user="${1:-root}" + + # Check whiptail availability + if ! command -v whiptail >/dev/null 2>&1; then + echo "Warning: whiptail not available" + export CLOUDINIT_ENABLE="no" + return 1 + fi + + # Ask enable + if ! (whiptail --backtitle "Proxmox VE Helper Scripts" --title "CLOUD-INIT" \ + --yesno "Enable Cloud-Init for VM configuration?" 16 68); then + export CLOUDINIT_ENABLE="no" + return 0 + fi + + export CLOUDINIT_ENABLE="yes" + + # Username + CLOUDINIT_USER=$(whiptail --backtitle "Proxmox VE Helper Scripts" --inputbox \ + "Cloud-Init Username" 8 58 "$default_user" --title "USERNAME" 3>&1 1>&2 2>&3) + export CLOUDINIT_USER="${CLOUDINIT_USER:-$default_user}" + + # Network mode + if (whiptail --backtitle "Proxmox VE Helper Scripts" --title "NETWORK MODE" \ + --yesno "Use DHCP for network configuration?" 10 58); then + export CLOUDINIT_NETWORK_MODE="dhcp" + else + export CLOUDINIT_NETWORK_MODE="static" + # ... prompt for static IP and gateway ... + fi + + # DNS servers + CLOUDINIT_DNS=$(whiptail --backtitle "Proxmox VE Helper Scripts" --inputbox \ + "DNS Servers (space-separated)" 8 58 "1.1.1.1 8.8.8.8" --title "DNS" 3>&1 1>&2 2>&3) + export CLOUDINIT_DNS +} +``` + +**Usage Examples**: + +```bash +# Example 1: Interactive configuration +configure_cloud_init_interactive "root" +# Prompts user for all settings interactively +# Exports variables for use in setup_cloud_init() + +# Example 2: With custom default user +configure_cloud_init_interactive "debian" +# Suggests "debian" as default username + +# Example 3: In script workflow +configure_cloud_init_interactive "$DEFAULT_USER" +setup_cloud_init "$VMID" "$STORAGE" "$HOSTNAME" "$CLOUDINIT_ENABLE" "$CLOUDINIT_USER" +# User configures interactively, then script sets up VM +``` + +--- + +## Configuration Parameters + +### VMID (Virtual Machine ID) + +- **Type**: Integer (100-2147483647) +- **Required**: Yes +- **Example**: `100` +- **Validation**: Must be unique, >= 100 on Proxmox + +### Storage + +- **Type**: String (storage backend name) +- **Required**: Yes +- **Examples**: `local`, `local-lvm`, `ceph-rbd` +- **Validation**: Must exist in Proxmox +- **Cloud-init Drive**: Placed on this storage + +### Hostname + +- **Type**: String (valid hostname) +- **Required**: No (defaults to vm-${VMID}) +- **Example**: `myvm`, `web-server-01` +- **Format**: Lowercase, alphanumeric, hyphens allowed + +### User + +- **Type**: String (username) +- **Required**: No (defaults: root) +- **Example**: `root`, `ubuntu`, `debian` +- **Cloud-init**: User account created on first boot + +### Network Mode + +- **Type**: Enum (dhcp, static) +- **Default**: dhcp +- **Options**: + - `dhcp` - Dynamic IP from DHCP server + - `static` - Manual IP configuration + +### Static IP + +- **Format**: CIDR notation (192.168.1.100/24) +- **Example**: `192.168.1.50/24`, `10.0.0.5/8` +- **Validation**: Valid IP and netmask +- **Required**: If network mode = static + +### Gateway + +- **Format**: IP address (192.168.1.1) +- **Example**: `192.168.1.1`, `10.0.0.1` +- **Validation**: Valid IP +- **Required**: If network mode = static + +### Nameservers + +- **Format**: Space-separated IPs +- **Default**: `1.1.1.1 8.8.8.8` +- **Example**: `1.1.1.1 8.8.8.8 9.9.9.9` + +### DNS Search Domain + +- **Type**: String +- **Default**: `local` +- **Example**: `example.com`, `internal.corp` + +--- + +## Data Formats + +### Cloud-Init Credentials File + +Generated at: `/tmp/${hostname}-${vmid}-cloud-init-credentials.txt` + +**Format**: +``` +======================================== +Cloud-Init Credentials +======================================== +VM ID: 100 +Hostname: myvm +Created: Tue Dec 01 10:30:00 UTC 2024 + +Username: root +Password: s7k9mL2pQ8wX + +Network: dhcp +DNS: 1.1.1.1 8.8.8.8 + +======================================== +SSH Access (if keys configured): +ssh root@ + +Proxmox UI Configuration: +VM 100 > Cloud-Init > Edit +- User, Password, SSH Keys +- Network (IP Config) +- DNS, Search Domain +======================================== +``` + +### Proxmox Cloud-Init Config + +Stored in: `/etc/pve/nodes//qemu-server/.conf` + +**Relevant Settings**: +``` +ide2: local-lvm:vm-100-cloudinit,media=cdrom +ciuser: root +cipassword: (encrypted) +ipconfig0: ip=dhcp +nameserver: 1.1.1.1 8.8.8.8 +searchdomain: local +``` + +### Network Configuration Examples + +**DHCP**: +```bash +qm set 100 --ipconfig0 "ip=dhcp" +``` + +**Static IPv4**: +```bash +qm set 100 --ipconfig0 "ip=192.168.1.100/24,gw=192.168.1.1" +``` + +**Static IPv6**: +```bash +qm set 100 --ipconfig0 "ip6=2001:db8::100/64,gw6=2001:db8::1" +``` + +**Dual Stack (IPv4 + IPv6)**: +```bash +qm set 100 --ipconfig0 "ip=192.168.1.100/24,gw=192.168.1.1,ip6=2001:db8::100/64,gw6=2001:db8::1" +``` + +--- + +## Best Practices + +### 1. **Always Configure SSH Keys** + +```bash +# Ensure SSH keys available before cloud-init setup +CLOUDINIT_SSH_KEYS="/root/.ssh/authorized_keys" + +if [ ! -f "$CLOUDINIT_SSH_KEYS" ]; then + mkdir -p /root/.ssh + # Generate or import SSH keys +fi + +setup_cloud_init "$VMID" "$STORAGE" "$HOSTNAME" "yes" +``` + +### 2. **Save Credentials Securely** + +```bash +# After setup_cloud_init(): +# Credentials file generated at $CLOUDINIT_CRED_FILE + +# Copy to secure location: +cp "$CLOUDINIT_CRED_FILE" "/root/vm-credentials/" +chmod 600 "/root/vm-credentials/$(basename $CLOUDINIT_CRED_FILE)" + +# Or display to user: +cat "$CLOUDINIT_CRED_FILE" +``` + +### 3. **Use Static IPs for Production** + +```bash +# DHCP - suitable for dev/test +setup_cloud_init "$VMID" "$STORAGE" "$HOSTNAME" "yes" "root" "dhcp" + +# Static - suitable for production +setup_cloud_init "$VMID" "$STORAGE" "$HOSTNAME" "yes" "root" \ + "static" "192.168.1.100/24" "192.168.1.1" +``` + +### 4. **Validate Network Configuration** + +```bash +# Before setting up cloud-init, ensure: +# - Gateway IP is reachable +# - IP address not in use +# - DNS servers are accessible + +ping -c 1 "$GATEWAY" || msg_error "Gateway unreachable" +``` + +### 5. **Test First Boot** + +```bash +# After cloud-init setup: +qm start "$VMID" + +# Wait for boot +sleep 10 + +# Check cloud-init status +qm exec "$VMID" cloud-init status + +# Verify network configuration +qm exec "$VMID" hostname -I +``` + +--- + +## Troubleshooting + +### Cloud-Init Not Applying + +```bash +# Inside VM: +cloud-init status # Show cloud-init status +cloud-init analyze # Analyze cloud-init boot +cloud-init query # Query cloud-init datasource + +# Check logs: +tail -100 /var/log/cloud-init-output.log +tail -100 /var/log/cloud-init.log +``` + +### Network Not Configured + +```bash +# Verify cloud-init config in Proxmox: +cat /etc/pve/nodes/$(hostname)/qemu-server/100.conf + +# Check cloud-init drive: +qm config 100 | grep ide2 + +# In VM, verify cloud-init wrote config: +cat /etc/netplan/99-cloudinit.yaml +``` + +### SSH Keys Not Installed + +```bash +# Verify SSH keys set in Proxmox: +qm config 100 | grep sshkeys + +# In VM, check SSH directory: +ls -la /root/.ssh/ +cat /root/.ssh/authorized_keys +``` + +### Password Not Set + +```bash +# Regenerate cloud-init drive: +qm set 100 --delete ide2 # Remove cloud-init drive +qm set 100 --ide2 local-lvm:vm-100-cloudinit,media=cdrom # Re-create + +# Set password again: +qm set 100 --cipassword "newpassword" +``` + +--- + +## Contributing + +### Adding New Configuration Options + +1. Add parameter to `setup_cloud_init()` function signature +2. Add validation for parameter +3. Add `qm set` command to apply configuration +4. Update documentation with examples +5. Test on actual Proxmox VE + +### Enhancing Interactive Configuration + +1. Add new whiptail dialog to `configure_cloud_init_interactive()` +2. Export variable for use in setup +3. Add validation logic +4. Test with various input scenarios + +### Supporting New Data Sources + +Beyond nocloud, could support: +- ConfigDrive (cloud-init standard) +- ESXi (if supporting vSphere) +- Hyper-V (if supporting Windows) + +--- + +## Notes + +- Cloud-init requires **QEMU guest agent** for optimal functionality +- Network configuration applied **on first boot only** +- Credentials file contains **sensitive information** - keep secure +- SSH keys are **persisted** and not displayed in credentials file +- Cloud-init is **optional** - VMs work without it + diff --git a/docs/core.func.md b/docs/core.func.md new file mode 100644 index 000000000..ecb70a0b1 --- /dev/null +++ b/docs/core.func.md @@ -0,0 +1,918 @@ +# Core.func Wiki + +The foundational utility library providing colors, formatting, validation checks, message output, and execution helpers used across all Community-Scripts ecosystem projects. + +--- + +## πŸ“‹ Table of Contents + +- [Overview](#overview) +- [Initialization Functions](#initialization-functions) +- [Color & Formatting](#color--formatting) +- [Validation Checks](#validation-checks) +- [Message Output Functions](#message-output-functions) +- [Execution Helpers](#execution-helpers) +- [Development Mode](#development-mode) +- [Best Practices](#best-practices) +- [Contributing](#contributing) + +--- + +## Overview + +Core.func provides essential utilities for consistent behavior across all Community-Scripts: + +- βœ… ANSI color codes for styled terminal output +- βœ… Standard icons and formatting for UI consistency +- βœ… System validation checks (root, PVE version, architecture) +- βœ… Colored message functions (info, ok, error, warn) +- βœ… Silent command execution with log redirection +- βœ… Spinner animations for long-running operations +- βœ… Development mode support (trace, breakpoint, dry-run) +- βœ… Guard clauses to prevent reloading + +### Integration Pattern + +```bash +#!/bin/bash +source <(curl -fsSL https://git.community-scripts.org/.../core.func) +load_functions # Initialize all color/formatting/defaults +root_check # Validate prerequisites +pve_check # Check Proxmox VE version +``` + +--- + +## Initialization Functions + +### `load_functions()` + +**Purpose**: Initializes all core utility function groups. Must be called once before using any core utilities. + +**Signature**: +```bash +load_functions() +``` + +**Parameters**: None + +**Returns**: No explicit return value (sets global variables) + +**Guard Mechanism**: +```bash +[[ -n "${__FUNCTIONS_LOADED:-}" ]] && return # Prevent re-loading +_CORE_FUNC_LOADED=1 # Mark as loaded +``` + +**Initializes** (in order): +1. `color()` - ANSI color codes +2. `formatting()` - Text formatting helpers +3. `icons()` - Emoji/symbol constants +4. `default_vars()` - Retry and timeout settings +5. `set_std_mode()` - Verbose/silent mode + +**Usage Examples**: + +```bash +# Example 1: Typical initialization +source <(curl -fsSL .../core.func) +load_functions # Safe to call multiple times +msg_info "Starting setup" # Now colors are available + +# Example 2: Safe multiple sourcing +source <(curl -fsSL .../core.func) +load_functions +source <(curl -fsSL .../tools.func) +load_functions # Silently returns (already loaded) +``` + +--- + +### `color()` + +**Purpose**: Defines ANSI escape codes for colored terminal output. + +**Signature**: +```bash +color() +``` + +**Color Variables Defined**: + +| Variable | Code | Effect | Use Case | +|----------|------|--------|----------| +| `YW` | `\033[33m` | Yellow | Warnings, secondary info | +| `YWB` | `\e[93m` | Bright Yellow | Emphasis, bright warnings | +| `BL` | `\033[36m` | Cyan/Blue | Hostnames, IPs, values | +| `RD` | `\033[01;31m` | Bright Red | Errors, critical alerts | +| `GN` | `\033[1;92m` | Bright Green | Success, OK status | +| `DGN` | `\033[32m` | Dark Green | Background, secondary success | +| `BGN` | `\033[4;92m` | Green with underline | Highlights | +| `CL` | `\033[m` | Clear | Reset to default | + +**Usage Examples**: + +```bash +# Example 1: Colored output +color +echo -e "${RD}Error: File not found${CL}" +# Output: "Error: File not found" (in red) + +# Example 2: Multiple colors on one line +echo -e "${YW}Warning:${CL} ${BL}$HOSTNAME${CL} is running low on ${RD}RAM${CL}" + +# Example 3: In functions +print_status() { + echo -e "${GN}βœ“${CL} Operation completed" +} +``` + +--- + +### `formatting()` + +**Purpose**: Defines formatting helpers for terminal output. + +**Signature**: +```bash +formatting() +``` + +**Formatting Variables Defined**: + +| Variable | Escape Code | Purpose | +|----------|------------|---------| +| `BFR` | `\r\033[K` | Backspace and clear line | +| `BOLD` | `\033[1m` | Bold text | +| `HOLD` | ` ` (space) | Spacing | +| `TAB` | ` ` (2 spaces) | Indentation | +| `TAB3` | ` ` (6 spaces) | Larger indentation | + +**Usage Examples**: + +```bash +# Example 1: Overwrite previous line (progress) +for i in {1..10}; do + echo -en "${BFR}Progress: $i/10" + sleep 1 +done + +# Example 2: Bold emphasis +echo -e "${BOLD}Important:${CL} This requires attention" + +# Example 3: Structured indentation +echo "Main Item:" +echo -e "${TAB}Sub-item 1" +echo -e "${TAB}Sub-item 2" +``` + +--- + +### `icons()` + +**Purpose**: Defines symbolic emoji and icon constants used for UI consistency. + +**Signature**: +```bash +icons() +``` + +**Icon Variables Defined**: + +| Variable | Icon | Use | +|----------|------|-----| +| `CM` | βœ”οΈ | Success/checkmark | +| `CROSS` | βœ–οΈ | Error/cross | +| `INFO` | πŸ’‘ | Information | +| `OS` | πŸ–₯️ | Operating system | +| `CONTAINERTYPE` | πŸ“¦ | Container | +| `DISKSIZE` | πŸ’Ύ | Disk/storage | +| `CPUCORE` | 🧠 | CPU | +| `RAMSIZE` | πŸ› οΈ | RAM | +| `HOSTNAME` | 🏠 | Hostname | +| `BRIDGE` | πŸŒ‰ | Network bridge | +| `NETWORK` | πŸ“‘ | Network | +| `GATEWAY` | 🌐 | Gateway | +| `CREATING` | πŸš€ | Creating | +| `ADVANCED` | 🧩 | Advanced/options | +| `HOURGLASS` | ⏳ | Wait/timer | + +--- + +### `default_vars()` + +**Purpose**: Sets default retry and timing variables for system operations. + +**Signature**: +```bash +default_vars() +``` + +**Variables Set**: +- `RETRY_NUM=10` - Maximum retry attempts +- `RETRY_EVERY=3` - Seconds between retries +- `i=$RETRY_NUM` - Counter for retry loops + +**Usage Examples**: + +```bash +# Example 1: Retry loop with defaults +RETRY_NUM=10 +RETRY_EVERY=3 +i=$RETRY_NUM +while [ $i -gt 0 ]; do + if check_network; then + break + fi + echo "Retrying... ($i attempts left)" + sleep $RETRY_EVERY + i=$((i - 1)) +done + +# Example 2: Custom retry values +RETRY_NUM=5 # Try 5 times +RETRY_EVERY=2 # Wait 2 seconds between attempts +``` + +--- + +### `set_std_mode()` + +**Purpose**: Configures output verbosity and optional debug tracing based on environment variables. + +**Signature**: +```bash +set_std_mode() +``` + +**Behavior**: +- If `VERBOSE=yes`: `STD=""` (show all output) +- If `VERBOSE=no`: `STD="silent"` (suppress output via silent() wrapper) +- If `DEV_MODE_TRACE=true`: Enable `set -x` bash tracing + +**Variables Set**: +- `STD` - Command prefix for optional output suppression + +**Usage Examples**: + +```bash +# Example 1: Verbose output +VERBOSE="yes" +set_std_mode +$STD apt-get update # Shows all apt output +# Output: All package manager messages displayed + +# Example 2: Silent output +VERBOSE="no" +set_std_mode +$STD apt-get update # Silently updates, logs to file +# Output: Only progress bar or errors shown + +# Example 3: Debug tracing +DEV_MODE_TRACE="true" +set_std_mode +# bash shows every command before executing: +(script.sh:123): function_name(): cmd +``` + +--- + +### `parse_dev_mode()` + +**Purpose**: Parses comma-separated dev_mode string to enable development features. + +**Signature**: +```bash +parse_dev_mode() +``` + +**Parameters**: None (uses `$dev_mode` environment variable) + +**Supported Flags**: +- `motd` - Setup SSH/MOTD before installation +- `keep` - Never delete container on failure +- `trace` - Enable bash set -x tracing +- `pause` - Pause after each msg_info step +- `breakpoint` - Open shell on error instead of cleanup +- `logs` - Persist logs to /var/log/community-scripts/ +- `dryrun` - Show commands without executing + +**Environment Variables Set**: +- `DEV_MODE_MOTD=false|true` +- `DEV_MODE_KEEP=false|true` +- `DEV_MODE_TRACE=false|true` +- `DEV_MODE_PAUSE=false|true` +- `DEV_MODE_BREAKPOINT=false|true` +- `DEV_MODE_LOGS=false|true` +- `DEV_MODE_DRYRUN=false|true` + +**Usage Examples**: + +```bash +# Example 1: Enable debugging +dev_mode="trace,logs" +parse_dev_mode +# Enables bash tracing and persistent logging + +# Example 2: Keep container on error +dev_mode="keep,breakpoint" +parse_dev_mode +# Container never deleted on error, opens shell at breakpoint + +# Example 3: Multiple modes +dev_mode="motd,keep,trace,pause" +parse_dev_mode +# All four development modes active +``` + +--- + +## Color & Formatting + +### Color Codes + +**Standard Colors**: +```bash +${YW} # Yellow (warnings) +${RD} # Red (errors) +${GN} # Green (success) +${BL} # Blue/Cyan (values) +${CL} # Clear (reset) +``` + +**Example Combinations**: +```bash +echo -e "${YW}Warning:${CL} ${RD}Critical${CL} at ${BL}$(date)${CL}" +# Output: "Warning: Critical at 2024-12-01 10:30:00" (colored appropriately) +``` + +--- + +## Validation Checks + +### `shell_check()` + +**Purpose**: Verifies script is running under Bash (not sh, dash, etc.). + +**Signature**: +```bash +shell_check() +``` + +**Parameters**: None + +**Returns**: 0 if Bash; exits with error if not + +**Behavior**: +- Checks `ps -p $$ -o comm=` (current shell command) +- Exits with error message if not "bash" +- Clears screen for better error visibility + +**Usage Examples**: + +```bash +#!/bin/bash +source <(curl -fsSL .../core.func) +load_functions +shell_check # Exits if run with: sh script.sh or dash script.sh + +# If run correctly: bash script.sh - continues +# If run with sh: Displays error and exits +``` + +--- + +### `root_check()` + +**Purpose**: Verifies script is running with root privileges directly (not via sudo). + +**Signature**: +```bash +root_check() +``` + +**Parameters**: None + +**Returns**: 0 if root directly; exits with error if not + +**Checks**: +- `id -u` must be 0 (root) +- Parent process (`$PPID`) must not be "sudo" + +**Why**: Some scripts require genuine root context, not sudo-elevated user shell. + +**Usage Examples**: + +```bash +#!/bin/bash +# Must run as root directly, not via sudo +source <(curl -fsSL .../core.func) +load_functions +root_check # Will fail if: sudo bash script.sh + +# Correct: bash script.sh (from root shell on Proxmox) +``` + +--- + +### `pve_check()` + +**Purpose**: Validates Proxmox VE version compatibility. + +**Signature**: +```bash +pve_check() +``` + +**Parameters**: None + +**Returns**: 0 if supported version; exits with error if not + +**Supported Versions**: +- PVE 8.0 - 8.9 +- PVE 9.0 - 9.1 + +**Version Detection**: +```bash +PVE_VER=$(pveversion | awk -F'/' '{print $2}' | awk -F'-' '{print $1}') +# Example: "pveversion" β†’ "pve-manager/8.2.2/550e8400-e29b" +# Extracted: "8.2.2" β†’ "8" +``` + +**Usage Examples**: + +```bash +# Example 1: On supported PVE 8.2 +bash ct/app.sh +# Passes: 8.2 is in range 8.0-8.9 + +# Example 2: On unsupported PVE 7.4 +bash ct/app.sh +# Error: "This version of Proxmox VE is not supported" + +# Example 3: On future unsupported PVE 10.0 +bash ct/app.sh +# Error: "This version of Proxmox VE is not yet supported" +``` + +--- + +### `arch_check()` + +**Purpose**: Validates system architecture is amd64/x86_64 (not ARM/PiMox). + +**Signature**: +```bash +arch_check() +``` + +**Parameters**: None + +**Returns**: 0 if amd64; exits with error if not + +**Behavior**: +- Checks `dpkg --print-architecture` +- Exits if not "amd64" +- Provides link to ARM64-compatible scripts + +**Usage Examples**: + +```bash +# Example 1: On x86_64 server +arch_check +# Passes silently + +# Example 2: On PiMox (ARM64) +arch_check +# Error: "This script will not work with PiMox!" +# Suggests: https://github.com/asylumexp/Proxmox +``` + +--- + +### `ssh_check()` + +**Purpose**: Detects SSH connection and warns if connecting remotely (recommends Proxmox console). + +**Signature**: +```bash +ssh_check() +``` + +**Parameters**: None + +**Returns**: No explicit return value (warning only, does not exit) + +**Behavior**: +- Checks `$SSH_CLIENT` environment variable +- Analyzes client IP to determine if local or remote +- Skips warning for local/same-subnet connections +- Warns for external connections + +**Usage Examples**: + +```bash +# Example 1: Local SSH (Proxmox WebUI console) +ssh_check +# No warning: Client is localhost (127.0.0.1) + +# Example 2: External SSH over Internet +ssh -l root 1.2.3.4 "bash script.sh" +# Warning: "Running via external SSH (client: 1.2.3.4)" +# Recommends Proxmox Shell (Console) instead +``` + +--- + +## Message Output Functions + +### `msg_info()` + +**Purpose**: Displays informational message with icon and yellow color. + +**Signature**: +```bash +msg_info() +``` + +**Parameters**: +- `$@` - Message text (concatenated with spaces) + +**Format**: `[ℹ️] Message text` (yellow) + +**Usage Examples**: + +```bash +msg_info "Starting container setup" +# Output: ℹ️ Starting container setup + +msg_info "Updating OS packages" "for debian:12" +# Output: ℹ️ Updating OS packages for debian:12 +``` + +--- + +### `msg_ok()` + +**Purpose**: Displays success message with checkmark and green color. + +**Signature**: +```bash +msg_ok() +``` + +**Parameters**: +- `$@` - Message text + +**Format**: `[βœ”οΈ] Message text` (green) + +**Usage Examples**: + +```bash +msg_ok "Container created" +# Output: βœ”οΈ Container created (in green) + +msg_ok "Network Connected: 10.0.3.50" +# Output: βœ”οΈ Network Connected: 10.0.3.50 +``` + +--- + +### `msg_error()` + +**Purpose**: Displays error message with cross icon and red color. Does not exit. + +**Signature**: +```bash +msg_error() +``` + +**Parameters**: +- `$@` - Message text + +**Format**: `[βœ–οΈ] Message text` (red) + +**Usage Examples**: + +```bash +msg_error "Container ID already in use" +# Output: βœ–οΈ Container ID already in use (in red) +``` + +--- + +### `msg_warn()` + +**Purpose**: Displays warning message with yellow color. + +**Signature**: +```bash +msg_warn() +``` + +**Parameters**: +- `$@` - Message text + +**Format**: `[⚠️] Message text` (yellow/orange) + +**Usage Examples**: + +```bash +msg_warn "This will delete all data" +# Output: ⚠️ This will delete all data +``` + +--- + +## Execution Helpers + +### `silent()` + +**Purpose**: Executes command with output redirected to log file. On error: displays last 10 lines of log and exits. + +**Signature**: +```bash +silent() +``` + +**Parameters**: +- `$@` - Command and arguments to execute + +**Returns**: 0 on success; exits with original error code on failure + +**Environment Effects**: +- Temporarily disables `set -e` and error trap to capture exit code +- Re-enables after command completes +- Logs to `$BUILD_LOG` or `$INSTALL_LOG` + +**Log Display On Error**: +```bash +--- Last 10 lines of silent log --- +[log output] +----------------------------------- +``` + +**Usage Examples**: + +```bash +# Example 1: Suppress package manager output +silent apt-get update +# Output: suppressed, logged to file + +# Example 2: Conditional display on error +silent curl -fsSL https://api.example.com +# If curl fails: shows last 10 log lines and exits + +# Example 3: Verbose mode shows everything +VERBOSE="yes" +silent apt-get update # Shows all output (STD is empty) +``` + +--- + +### `spinner()` + +**Purpose**: Displays animated spinner with rotating characters during long operations. + +**Signature**: +```bash +spinner() +``` + +**Parameters**: None (uses `$SPINNER_MSG` environment variable) + +**Animation**: +``` +β ‹ β ™ β Ή β Έ β Ό β ΄ β ¦ β § β ‡ ⠏ (repeating) +``` + +**Environment Variables**: +- `SPINNER_MSG` - Text to display with spinner + +**Lifecycle**: +```bash +# Start spinner in background +SPINNER_MSG="Downloading..." +spinner & +SPINNER_PID=$! + +# ... do long operation ... + +# Stop spinner +stop_spinner + +echo "Done!" +``` + +**Usage Examples**: + +```bash +# Example 1: Long operation with spinner +SPINNER_MSG="Building container..." +spinner & +SPINNER_PID=$! + +sleep 10 # Simulate work + +stop_spinner +msg_ok "Container created" +``` + +--- + +### `clear_line()` + +**Purpose**: Clears current terminal line and moves cursor to beginning. + +**Signature**: +```bash +clear_line() +``` + +**Parameters**: None + +**Implementation**: Uses `tput` or ANSI escape codes + +**Usage Examples**: + +```bash +for file in *.sh; do + echo -n "Processing $file..." + process_file "$file" + clear_line +done +# Each file overwrites previous line +``` + +--- + +### `stop_spinner()` + +**Purpose**: Stops running spinner process and cleans up temporary files. + +**Signature**: +```bash +stop_spinner() +``` + +**Parameters**: None (reads `$SPINNER_PID` or `/tmp/.spinner.pid`) + +**Cleanup**: +- Graceful kill of spinner process +- Force kill (-9) if needed +- Removes `/tmp/.spinner.pid` temp file +- Resets terminal state + +**Usage Examples**: + +```bash +# Example 1: Simple stop +spinner & +SPINNER_PID=$! +sleep 5 +stop_spinner + +# Example 2: In trap handler +trap 'stop_spinner' EXIT +spinner & +``` + +--- + +## Development Mode + +### Enabling Development Features + +**Via Environment Variable**: +```bash +dev_mode="trace,keep,breakpoint" bash ct/myapp.sh +``` + +**Via Script Header**: +```bash +#!/bin/bash +export dev_mode="trace,logs,pause" +source <(curl -fsSL .../core.func) +load_functions +parse_dev_mode +``` + +### Available Modes + +| Mode | Effect | +|------|--------| +| `trace` | Enable bash -x tracing (verbose command logging) | +| `keep` | Never delete container on error (for debugging) | +| `logs` | Persist all logs to /var/log/community-scripts/ | +| `pause` | Pause after each msg_info step (manual stepping) | +| `breakpoint` | Open shell on error instead of immediate cleanup | +| `motd` | Configure SSH/MOTD before installation starts | +| `dryrun` | Show commands without executing them | + +--- + +## Best Practices + +### 1. **Always Call load_functions() First** + +```bash +#!/bin/bash +set -Eeuo pipefail + +source <(curl -fsSL .../core.func) +load_functions # MUST be before using any color variables + +msg_info "Starting setup" # Now safe to use +``` + +### 2. **Use Message Functions Consistently** + +```bash +msg_info "Starting step" +# Do work... +msg_ok "Step completed" + +# Or on error: +if ! command; then + msg_error "Command failed" + exit 1 +fi +``` + +### 3. **Combine Validation Checks** + +```bash +#!/bin/bash +source <(curl -fsSL .../core.func) +load_functions + +shell_check # Exits if wrong shell +root_check # Exits if not root +pve_check # Exits if unsupported version +arch_check # Exits if wrong architecture + +# All checks passed, safe to proceed +msg_ok "Pre-flight checks passed" +``` + +### 4. **Use Verbose Mode for Debugging** + +```bash +VERBOSE="yes" bash ct/myapp.sh +# Shows all silent() command output for troubleshooting +``` + +### 5. **Log Important Operations** + +```bash +silent apt-get update # Suppress unless error +msg_ok "Packages updated" # Show success +silent systemctl start nginx # Suppress unless error +msg_ok "Nginx started" # Show success +``` + +--- + +## Contributing + +### Adding New Message Functions + +Follow existing pattern: + +```bash +msg_custom() { + local icon="$1" + local color="$2" + local message="$3" + echo -e "${TAB}${icon}${TAB}${color}${message}${CL}" +} +``` + +### Adding Color Support + +New colors should follow semantic naming: + +```bash +BG_ERROR=$'\e[41m' # Red background for errors +BG_SUCCESS=$'\e[42m' # Green background for success +``` + +### Testing Color Output + +```bash +bash +source <(curl -fsSL .../core.func) +load_functions + +echo -e "${YW}Yellow${CL} ${RD}Red${CL} ${GN}Green${CL} ${BL}Blue${CL}" +``` + +--- + +## Notes + +- Core.func is designed to be **sourced once** and **loaded everywhere** +- All color variables are **ANSI escape codes** (work in all terminals) +- Messages use **emoji icons** for visual consistency +- Validation checks use **standard exit codes** (0 for success, 1 for error) +- The module is **lightweight** and loads instantly + diff --git a/docs/error_handler.func.md b/docs/error_handler.func.md new file mode 100644 index 000000000..f8b2aece0 --- /dev/null +++ b/docs/error_handler.func.md @@ -0,0 +1,598 @@ +# Error-Handler.func Wiki + +Comprehensive error handling and signal management module providing exit code explanations, error handlers with logging, and signal trap configuration for all Community-Scripts projects. + +--- + +## πŸ“‹ Table of Contents + +- [Overview](#overview) +- [Exit Code Reference](#exit-code-reference) +- [Error Handler Functions](#error-handler-functions) +- [Signal Traps](#signal-traps) +- [Initialization & Setup](#initialization--setup) +- [Error Logging](#error-logging) +- [Best Practices](#best-practices) +- [Debugging](#debugging) +- [Contributing](#contributing) + +--- + +## Overview + +The error_handler.func module provides robust error handling infrastructure: + +- βœ… Comprehensive exit code mapping (1-255+ codes documented) +- βœ… Detailed error messages with line numbers and commands +- βœ… Signal trap configuration (ERR, EXIT, INT, TERM, RETURN) +- βœ… Error logging to persistent files +- βœ… Graceful cleanup on signal termination +- βœ… Stack trace display for debugging +- βœ… Integration with core.func message functions +- βœ… Container-agnostic (works in Proxmox + LXC) + +### Error Handling Flow + +``` +Command Execution + ↓ + ERROR (non-zero exit) + ↓ + ERR Trap Triggered + ↓ +error_handler() called + ↓ +explain_exit_code() lookup + ↓ +Display error with line/command + ↓ +Check for log file + ↓ +Exit with original code +``` + +--- + +## Exit Code Reference + +Exit codes are categorized by source system. See `api.func.md` for comprehensive mapping documentation. + +### Quick Reference Table + +| Range | Category | Examples | +|-------|----------|----------| +| 0 | Success | (no error) | +| 1-2 | Shell errors | Syntax error, operation not permitted | +| 100-101 | APT errors | Package manager errors | +| 126-139 | System errors | Command not found, segfault, OOM | +| 200-231 | Proxmox custom | Container creation errors | +| 210-234 | Database errors | PostgreSQL, MySQL connection issues | +| 243-254 | Runtime errors | Node.js, Python, npm errors | +| 255 | DPKG fatal | Package system fatal error | + +--- + +## Error Handler Functions + +### `explain_exit_code()` + +**Purpose**: Maps numeric exit codes to human-readable descriptions. Shared with api.func for consistency. + +**Signature**: +```bash +explain_exit_code() +``` + +**Parameters**: +- `$1` - Exit code (0-255+) + +**Returns**: Human-readable explanation string + +**Categories Handled**: +- Generic shell errors (1, 2, 126-128, 130, 137, 139, 143) +- Package managers (100-101, 255) +- Python (210-212) +- Databases (PostgreSQL 231-234, MySQL 241-244, MongoDB 251-254) +- Node.js/npm (243-249, 254) +- Proxmox custom (200-231) +- Default: "Unknown error" + +**Usage Examples**: + +```bash +# Example 1: Look up error code +explain_exit_code 127 +# Output: "Command not found" + +# Example 2: In error logging +error_desc=$(explain_exit_code "$exit_code") +echo "Error: $error_desc" >> /tmp/error.log + +# Example 3: Unknown code +explain_exit_code 999 +# Output: "Unknown error" +``` + +--- + +### `error_handler()` + +**Purpose**: Main error handler triggered by ERR trap. Displays detailed error information and exits. + +**Signature**: +```bash +error_handler() +``` + +**Parameters**: +- `$1` (optional) - Exit code (default: current $?) +- `$2` (optional) - Command that failed (default: $BASH_COMMAND) +- `$3` (optional) - Line number (default: ${BASH_LINENO[0]}) + +**Returns**: Exits with original exit code (does not return) + +**Output Format**: +``` +[ERROR] in line 42: exit code 1 (General error): while executing command curl https://api.example.com + +--- Last 10 lines of log file --- +[log content] +------------------------------------ +``` + +**Implementation Pattern**: +```bash +error_handler() { + local exit_code=${1:-$?} + local command=${2:-${BASH_COMMAND:-unknown}} + local line_number=${BASH_LINENO[0]:-unknown} + + # If successful, return silently + if [[ "$exit_code" -eq 0 ]]; then + return 0 + fi + + # Get human-readable error description + local explanation=$(explain_exit_code "$exit_code") + + # Show cursor (might be hidden by spinner) + printf "\e[?25h" + + # Display error using color messages + if declare -f msg_error >/dev/null 2>&1; then + msg_error "in line ${line_number}: exit code ${exit_code} (${explanation}): while executing command ${command}" + else + echo -e "\n${RD}[ERROR]${CL} in line ${line_number}: exit code ${exit_code}: ${command}\n" + fi + + # Log error details if log file configured + if [[ -n "${DEBUG_LOGFILE:-}" ]]; then + { + echo "------ ERROR ------" + echo "Timestamp : $(date '+%Y-%m-%d %H:%M:%S')" + echo "Exit Code : $exit_code ($explanation)" + echo "Line : $line_number" + echo "Command : $command" + echo "-------------------" + } >> "$DEBUG_LOGFILE" + fi + + # Show last lines of log if available + local active_log="$(get_active_logfile)" + if [[ -s "$active_log" ]]; then + local log_lines=$(wc -l < "$active_log") + echo "--- Last 10 lines of log ---" + tail -n 10 "$active_log" + echo "----------------------------" + fi + + exit "$exit_code" +} +``` + +**Usage Examples**: + +```bash +# Example 1: Automatic trap (recommended) +trap 'error_handler $? "$BASH_COMMAND" $LINENO' ERR +# Error automatically caught and handled + +# Example 2: Manual invocation (testing) +error_handler 1 "curl https://api.example.com" 42 +# Output: Detailed error with line number + +# Example 3: In conditional +if ! some_command; then + error_handler $? "some_command" $LINENO +fi +``` + +--- + +## Signal Traps + +### `on_exit()` + +**Purpose**: Cleanup handler called on normal script exit or error. + +**Signature**: +```bash +on_exit() +``` + +**Parameters**: None + +**Returns**: Exits with captured exit code + +**Behavior**: +- Captures current exit code +- Removes lock files if present +- Exits with original exit code + +**Implementation Pattern**: +```bash +on_exit() { + local exit_code="$?" + + # Cleanup lock files + [[ -n "${lockfile:-}" && -e "$lockfile" ]] && rm -f "$lockfile" + + # Preserve exit code + exit "$exit_code" +} +``` + +**Trap Configuration**: +```bash +trap on_exit EXIT # Always called on exit +``` + +--- + +### `on_interrupt()` + +**Purpose**: Handler for Ctrl+C (SIGINT) signal. Allows graceful shutdown. + +**Signature**: +```bash +on_interrupt() +``` + +**Parameters**: None + +**Returns**: Exits with code 130 (standard SIGINT exit code) + +**Output**: Displays "Interrupted by user (SIGINT)" message + +**Implementation Pattern**: +```bash +on_interrupt() { + echo -e "\n${RD}Interrupted by user (SIGINT)${CL}" + exit 130 +} +``` + +**Trap Configuration**: +```bash +trap on_interrupt INT # Called on Ctrl+C +``` + +**Usage Example**: +```bash +# Script interrupted by user: +# Ctrl+C pressed +# β†’ on_interrupt() triggers +# β†’ "Interrupted by user (SIGINT)" displayed +# β†’ Exit with code 130 +``` + +--- + +### `on_terminate()` + +**Purpose**: Handler for SIGTERM signal. Allows graceful shutdown on termination. + +**Signature**: +```bash +on_terminate() +``` + +**Parameters**: None + +**Returns**: Exits with code 143 (standard SIGTERM exit code) + +**Output**: Displays "Terminated by signal (SIGTERM)" message + +**Implementation Pattern**: +```bash +on_terminate() { + echo -e "\n${RD}Terminated by signal (SIGTERM)${CL}" + exit 143 +} +``` + +**Trap Configuration**: +```bash +trap on_terminate TERM # Called on SIGTERM +``` + +**Usage Example**: +```bash +# System sends SIGTERM: +# kill -TERM $PID executed +# β†’ on_terminate() triggers +# β†’ "Terminated by signal (SIGTERM)" displayed +# β†’ Exit with code 143 +``` + +--- + +## Initialization & Setup + +### `catch_errors()` + +**Purpose**: Sets up all error traps and signal handlers. Called once at script start. + +**Signature**: +```bash +catch_errors() +``` + +**Parameters**: None + +**Returns**: No explicit return value (configures traps) + +**Traps Configured**: +1. `ERR` β†’ `error_handler()` - Catches command failures +2. `EXIT` β†’ `on_exit()` - Cleanup on any exit +3. `INT` β†’ `on_interrupt()` - Handle Ctrl+C +4. `TERM` β†’ `on_terminate()` - Handle SIGTERM +5. `RETURN` β†’ `error_handler()` - Catch function errors + +**Implementation Pattern**: +```bash +catch_errors() { + # Set strict mode + set -Eeuo pipefail + + # Configure traps + trap 'error_handler $? "$BASH_COMMAND" $LINENO' ERR + trap on_exit EXIT + trap on_interrupt INT + trap on_terminate TERM + trap 'error_handler $? "$BASH_COMMAND" $LINENO' RETURN +} +``` + +**Usage Examples**: + +```bash +# Example 1: Alpine container script +#!/bin/sh +source <(curl -fsSL .../core.func) +source <(curl -fsSL .../error_handler.func) +load_functions +catch_errors + +# Now all signals handled automatically +update_os + +# Example 2: Proxmox host script +#!/bin/bash +source <(curl -fsSL .../core.func) +source <(curl -fsSL .../error_handler.func) +load_functions +catch_errors + +# Safe to proceed with error handling +create_container +``` + +--- + +## Error Logging + +### Log File Configuration + +**Active Log Detection**: +```bash +# In build.func/install.func: +BUILD_LOG="/tmp/create-lxc-${SESSION_ID}.log" +INSTALL_LOG="/root/install-${SESSION_ID}.log" +SILENT_LOGFILE="$(get_active_logfile)" # Points to appropriate log +``` + +### Log Output Behavior + +When command fails in `silent()`: + +1. Last 10 lines of log file are displayed +2. Full log path shown if more than 10 lines +3. Error message includes line number where failure occurred +4. Command that failed is displayed + +### Accessing Error Logs + +From Proxmox host: +```bash +# Host-side container creation log +/tmp/create-lxc-.log + +# View error details +tail -50 /tmp/create-lxc-550e8400.log +grep ERROR /tmp/create-lxc-*.log + +# Development mode persistent logs +/var/log/community-scripts/create-lxc--.log +``` + +From inside LXC container: +```bash +# Container installation log +/root/install-.log + +# View recent errors +tail -20 /root/install-550e8400.log +``` + +--- + +## Best Practices + +### 1. **Always Setup Traps Early** + +```bash +#!/bin/bash +set -Eeuo pipefail + +source <(curl -fsSL .../core.func) +source <(curl -fsSL .../error_handler.func) +load_functions +catch_errors # MUST be called before any real work + +# Now safe - all signals handled +``` + +### 2. **Use Meaningful Error Exit Codes** + +```bash +# Use Proxmox custom codes for container-specific errors +if [[ "$CTID" -lt 100 ]]; then + msg_error "Container ID must be >= 100" + exit 205 # Proxmox custom code +fi + +# Use standard codes for common errors +if ! command -v curl &>/dev/null; then + msg_error "curl not installed" + exit 127 # Command not found +fi +``` + +### 3. **Log Context Information** + +```bash +# In error_handler, DEBUG_LOGFILE receives: +DEBUG_LOGFILE="/tmp/debug.log" + +# All errors logged with timestamp and details +{ + echo "Error at $(date)" + echo "Exit code: $exit_code" + echo "Command: $command" +} >> "$DEBUG_LOGFILE" +``` + +### 4. **Graceful Signal Handling** + +```bash +# Setup signal handlers for cleanup +cleanup() { + [[ -f "$temp_file" ]] && rm -f "$temp_file" + [[ -d "$temp_dir" ]] && rm -rf "$temp_dir" +} +trap cleanup EXIT + +# Now temporary files always cleaned up +``` + +### 5. **Test Error Paths** + +```bash +# Force error for testing +false # Triggers error_handler +# or +exit 1 # Custom error + +# Verify error handling works correctly +# Check log files and messages +``` + +--- + +## Debugging + +### Enable Stack Trace + +```bash +# Via environment variable +DEV_MODE_TRACE=true bash script.sh + +# Or in script +set -x # Show all commands +trap 'error_handler $? "$BASH_COMMAND" $LINENO' ERR +``` + +### View Full Error Context + +```bash +# Show full log file instead of last 10 lines +DEBUG_LOGFILE="/tmp/full-debug.log" + +# After error, review complete context +less /tmp/full-debug.log +``` + +### Test Error Handler + +```bash +# Manually trigger error handler +bash -c 'source <(curl -fsSL .../error_handler.func); catch_errors; exit 42' + +# Should display: +# [ERROR] in line N: exit code 42 (Unknown error): ... +``` + +--- + +## Contributing + +### Adding New Error Codes + +1. Assign code in appropriate range (see Exit Code Reference) +2. Add description to `explain_exit_code()` in both: + - error_handler.func + - api.func (for consistency) +3. Document in exit code table +4. Update error mapping documentation + +### Improving Error Messages + +Example: Make error message more helpful: + +```bash +# Before: +"Container ID must be >= 100" + +# After: +"Invalid CTID: $CTID. Container IDs must be >= 100. Current range: 100-999" +``` + +### Testing Signal Handlers + +```bash +# Test INT signal (Ctrl+C) +bash -c 'source <(curl -fsSL .../error_handler.func); catch_errors; sleep 30' & +PID=$! +sleep 1 +kill -INT $PID +wait + +# Test TERM signal +bash -c 'source <(curl -fsSL .../error_handler.func); catch_errors; sleep 30' & +PID=$! +sleep 1 +kill -TERM $PID +wait +``` + +--- + +## Notes + +- Error handler is **required** for all scripts (ensures safe cleanup) +- Exit codes are **standardized** (0 = success, 1-255 = specific errors) +- Signals are **trapped** to allow graceful shutdown +- Lock files are **automatically cleaned** on exit +- Log files contain **full error context** for debugging + diff --git a/docs/install.func.md b/docs/install.func.md new file mode 100644 index 000000000..c5f6be72c --- /dev/null +++ b/docs/install.func.md @@ -0,0 +1,646 @@ +# Install.func Wiki + +Container installation workflow orchestration module providing network setup, OS configuration, connectivity verification, and installation mechanics for applications deployed inside LXC containers. + +--- + +## πŸ“‹ Table of Contents + +- [Overview](#overview) +- [Initialization & Dependencies](#initialization--dependencies) +- [Network & Connectivity Functions](#network--connectivity-functions) +- [OS Configuration Functions](#os-configuration-functions) +- [Installation Workflow](#installation-workflow) +- [Best Practices](#best-practices) +- [Debugging](#debugging) +- [Contributing](#contributing) + +--- + +## Overview + +Install.func provides **container-internal installation mechanics**: + +- βœ… Network connectivity verification (IPv4/IPv6) +- βœ… OS updates and package management +- βœ… DNS resolution validation +- βœ… System optimization (disable wait-online service) +- βœ… SSH and MOTD configuration +- βœ… Container customization (auto-login, update script) +- βœ… Comprehensive error handling with signal traps +- βœ… Integration with core.func and error_handler.func + +### Execution Context + +``` +Proxmox Host LXC Container +────────────────────────────────────────── +pct create CTID ... + ↓ +Boot container + ↓ +pct exec CTID bash /tmp/install.sh + ↓ +[Execution within container] + └─→ install.func functions execute + └─→ verb_ip6() + └─→ setting_up_container() + └─→ network_check() + └─→ update_os() + └─→ etc. +``` + +--- + +## Initialization & Dependencies + +### Module Dependencies + +```bash +# Install.func requires two prerequisites +if ! command -v curl >/dev/null 2>&1; then + apt-get update >/dev/null 2>&1 + apt-get install -y curl >/dev/null 2>&1 +fi + +# Source core functions (colors, formatting, messages) +source <(curl -fsSL https://git.community-scripts.org/.../core.func) + +# Source error handling (traps, signal handlers) +source <(curl -fsSL https://git.community-scripts.org/.../error_handler.func) + +# Initialize both modules +load_functions # Sets up colors, icons, defaults +catch_errors # Configures ERR, EXIT, INT, TERM traps +``` + +### Environment Variables Passed from Host + +These variables are passed by build.func via `pct set` and environment: + +| Variable | Source | Purpose | +|----------|--------|---------| +| `VERBOSE` | Build config | Show all output (yes/no) | +| `PASSWORD` | User input | Root password (blank = auto-login) | +| `DISABLEIPV6` | Advanced settings | Disable IPv6 (yes/no) | +| `SSH_ROOT` | Advanced settings | Enable SSH root access | +| `CACHER` | Config | Use APT cache proxy (yes/no) | +| `CACHER_IP` | Config | APT cache IP address | +| `APPLICATION` | App script | App display name | +| `app` | App script | Normalized app name (lowercase) | +| `RETRY_NUM` | core.func | Retry attempts (default: 10) | +| `RETRY_EVERY` | core.func | Retry interval in seconds (default: 3) | + +--- + +## Network & Connectivity Functions + +### `verb_ip6()` + +**Purpose**: Configures IPv6 based on DISABLEIPV6 variable and sets verbose mode. + +**Signature**: +```bash +verb_ip6() +``` + +**Parameters**: None + +**Returns**: No explicit return value (configures system) + +**Environment Requirements**: +- `DISABLEIPV6` - Set to "yes" to disable IPv6, "no" to keep enabled +- `VERBOSE` - Controls output verbosity via set_std_mode() + +**Behavior**: +```bash +# If DISABLEIPV6=yes: +echo "net.ipv6.conf.all.disable_ipv6 = 1" >> /etc/sysctl.conf +sysctl -p # Apply immediately + +# If DISABLEIPV6=no (default): +# No changes (IPv6 remains enabled) +``` + +**Usage Examples**: + +```bash +# Example 1: Disable IPv6 (for security/simplicity) +DISABLEIPV6="yes" +VERBOSE="no" +verb_ip6 +# Result: IPv6 disabled, change persisted + +# Example 2: Keep IPv6 enabled (default) +DISABLEIPV6="no" +verb_ip6 +# Result: IPv6 operational, no configuration + +# Example 3: Verbose mode +VERBOSE="yes" +verb_ip6 +# Output: Shows sysctl configuration commands +``` + +--- + +### `setting_up_container()` + +**Purpose**: Verifies network connectivity and performs initial OS configuration for Debian/Ubuntu containers. + +**Signature**: +```bash +setting_up_container() +``` + +**Parameters**: None + +**Returns**: 0 on success; exits with code 1 if network unavailable after retries + +**Environment Requirements**: +- `RETRY_NUM` - Max attempts (default: 10) +- `RETRY_EVERY` - Seconds between retries (default: 3) + +**Operations**: +1. Verify network connectivity via `hostname -I` +2. Retry up to RETRY_NUM times with RETRY_EVERY second delays +3. Remove Python EXTERNALLY-MANAGED marker (allows pip) +4. Disable systemd-networkd-wait-online.service (speeds up boot) +5. Display network information + +**Implementation Pattern**: +```bash +setting_up_container() { + msg_info "Setting up Container OS" + + # Network availability loop + for ((i = RETRY_NUM; i > 0; i--)); do + if [ "$(hostname -I)" != "" ]; then + break + fi + echo 1>&2 -en "${CROSS}${RD} No Network! " + sleep $RETRY_EVERY + done + + # Check final state + if [ "$(hostname -I)" = "" ]; then + echo 1>&2 -e "\n${CROSS}${RD} No Network After $RETRY_NUM Tries${CL}" + exit 1 + fi + + # Python pip support + rm -rf /usr/lib/python3.*/EXTERNALLY-MANAGED + + # Speed up boot (disable wait service) + systemctl disable -q --now systemd-networkd-wait-online.service + + msg_ok "Set up Container OS" + msg_ok "Network Connected: ${BL}$(hostname -I)${CL}" +} +``` + +**Usage Examples**: + +```bash +# Example 1: Immediate network availability +RETRY_NUM=10 +RETRY_EVERY=3 +setting_up_container +# Output: +# ℹ️ Setting up Container OS +# βœ”οΈ Set up Container OS +# βœ”οΈ Network Connected: 10.0.3.50 + +# Example 2: Delayed network (waits 6 seconds) +# Script retries 2 times before succeeding +# (each retry waits 3 seconds) + +# Example 3: No network +# Script waits 30 seconds total (10 x 3) +# Then exits with: "No Network After 10 Tries" +``` + +--- + +### `network_check()` + +**Purpose**: Comprehensive network diagnostics for both IPv4 and IPv6, including DNS validation for Git/GitHub. + +**Signature**: +```bash +network_check() +``` + +**Parameters**: None + +**Returns**: 0 on success; exits with code 1 on critical DNS failure + +**Checks Performed**: + +1. **IPv4 Connectivity** (tests 3 public DNS servers): + - 1.1.1.1 (Cloudflare) + - 8.8.8.8 (Google) + - 9.9.9.9 (Quad9) + +2. **IPv6 Connectivity** (tests 3 public DNS servers): + - 2606:4700:4700::1111 (Cloudflare) + - 2001:4860:4860::8888 (Google) + - 2620:fe::fe (Quad9) + +3. **DNS Resolution** (validates Git-related domains): + - github.com + - raw.githubusercontent.com + - api.github.com + - git.community-scripts.org + +**Output Format**: +``` +βœ”οΈ IPv4 Internet Connected +βœ”οΈ IPv6 Internet Connected +βœ”οΈ Git DNS: github.com:βœ”οΈ raw.githubusercontent.com:βœ”οΈ ... +``` + +**Error Handling**: +```bash +# If both IPv4 and IPv6 fail: +# Prompts user: "No Internet detected, would you like to continue anyway?" +# If user says no: Exits +# If user says yes: Shows warning "Expect Issues Without Internet" + +# If DNS fails for GitHub: +# Calls fatal() - exits immediately with error +``` + +**Implementation Pattern**: +```bash +network_check() { + set +e + trap - ERR + + ipv4_connected=false + ipv6_connected=false + + # IPv4 test + if ping -c 1 -W 1 1.1.1.1 &>/dev/null || ...; then + msg_ok "IPv4 Internet Connected" + ipv4_connected=true + else + msg_error "IPv4 Internet Not Connected" + fi + + # IPv6 test + if ping6 -c 1 -W 1 2606:4700:4700::1111 &>/dev/null || ...; then + msg_ok "IPv6 Internet Connected" + ipv6_connected=true + else + msg_error "IPv6 Internet Not Connected" + fi + + # DNS checks for GitHub domains + GIT_HOSTS=("github.com" "raw.githubusercontent.com" "api.github.com" "git.community-scripts.org") + for HOST in "${GIT_HOSTS[@]}"; do + RESOLVEDIP=$(getent hosts "$HOST" | awk '{ print $1 }' | head -n1) + if [[ -z "$RESOLVEDIP" ]]; then + DNS_FAILED=true + fi + done + + if [[ "$DNS_FAILED" == true ]]; then + fatal "$GIT_STATUS" # Exit on critical DNS failure + fi + + set -e + trap 'error_handler $LINENO "$BASH_COMMAND"' ERR +} +``` + +**Usage Examples**: + +```bash +# Example 1: Good connectivity (all checks pass) +network_check +# Output: +# βœ”οΈ IPv4 Internet Connected +# βœ”οΈ IPv6 Internet Connected +# βœ”οΈ Git DNS: github.com:βœ”οΈ ... + +# Example 2: IPv6 unavailable but IPv4 OK +network_check +# Output: +# βœ”οΈ IPv4 Internet Connected +# βœ–οΈ IPv6 Internet Not Connected +# βœ”οΈ Git DNS checks OK + +# Example 3: No internet at all +network_check +# Prompts: "No Internet detected, would you like to continue anyway?" +# User: y +# Output: ⚠️ Expect Issues Without Internet +``` + +--- + +## OS Configuration Functions + +### `update_os()` + +**Purpose**: Updates Debian/Ubuntu OS packages and loads additional tools library. + +**Signature**: +```bash +update_os() +``` + +**Parameters**: None + +**Returns**: No explicit return value (updates system) + +**Operations**: +1. Display info message +2. Optional: Configure APT caching proxy +3. Run `apt-get update` (index refresh) +4. Run `apt-get dist-upgrade` (system upgrade) +5. Remove Python EXTERNALLY-MANAGED restrictions +6. Source tools.func for additional setup +7. Display success message + +**APT Caching Configuration** (if CACHER=yes): +```bash +# Configure apt-proxy-detect.sh +/etc/apt/apt.conf.d/00aptproxy + +# Script detects local APT cacher and routes through it +# Falls back to DIRECT if unavailable +``` + +**Implementation Pattern**: +```bash +update_os() { + msg_info "Updating Container OS" + + # Optional: Setup APT cacher + if [[ "$CACHER" == "yes" ]]; then + echo "Acquire::http::Proxy-Auto-Detect \"/usr/local/bin/apt-proxy-detect.sh\";" > /etc/apt/apt.conf.d/00aptproxy + + cat > /usr/local/bin/apt-proxy-detect.sh <<'EOF' +#!/bin/bash +if nc -w1 -z "${CACHER_IP}" 3142; then + echo -n "http://${CACHER_IP}:3142" +else + echo -n "DIRECT" +fi +EOF + chmod +x /usr/local/bin/apt-proxy-detect.sh + fi + + # Update system + $STD apt-get update + $STD apt-get -o Dpkg::Options::="--force-confold" -y dist-upgrade + + # Python support + rm -rf /usr/lib/python3.*/EXTERNALLY-MANAGED + + # Load additional tools library + source <(curl -fsSL https://git.community-scripts.org/.../tools.func) + + msg_ok "Updated Container OS" +} +``` + +**Usage Examples**: + +```bash +# Example 1: Standard update +update_os +# Output: Updates all packages silently (unless VERBOSE=yes) + +# Example 2: With APT cacher +CACHER="yes" +CACHER_IP="192.168.1.100" +update_os +# Uses cache proxy for faster package downloads + +# Example 3: Verbose output +VERBOSE="yes" +update_os +# Shows all apt-get operations in detail +``` + +--- + +## SSH & MOTD Configuration + +### `motd_ssh()` + +**Purpose**: Configures Message of the Day and enables SSH root access if configured. + +**Signature**: +```bash +motd_ssh() +``` + +**Parameters**: None + +**Returns**: No explicit return value (configures system) + +**Operations**: +1. Set TERM environment variable for better terminal support +2. Gather OS information (name, version, IP) +3. Create `/etc/profile.d/00_lxc-details.sh` with container details script +4. Optionally enable root SSH access if SSH_ROOT=yes + +**MOTD Script Content**: +```bash +echo -e "" +echo -e "${BOLD}${YW}${APPLICATION} LXC Container - DEV Repository${CL}" +echo -e "${RD}WARNING: This is a DEVELOPMENT version (ProxmoxVED). Do NOT use in production!${CL}" +echo -e "${YW} OS: ${GN}${OS_NAME} - Version: ${OS_VERSION}${CL}" +echo -e "${YW} Hostname: ${GN}$(hostname)${CL}" +echo -e "${YW} IP Address: ${GN}$(hostname -I | awk '{print $1}')${CL}" +echo -e "${YW} Repository: ${GN}https://github.com/community-scripts/ProxmoxVED${CL}" +echo "" +``` + +**SSH Configuration** (if SSH_ROOT=yes): +```bash +sed -i "s/#PermitRootLogin prohibit-password/PermitRootLogin yes/g" /etc/ssh/sshd_config +systemctl restart sshd +``` + +--- + +## Installation Workflow + +### Typical Installation Sequence + +```bash +#!/bin/bash +# Inside container during installation + +source <(curl -fsSL .../core.func) +source <(curl -fsSL .../error_handler.func) +load_functions +catch_errors + +# Step 1: Network setup +verb_ip6 +setting_up_container +network_check + +# Step 2: System update +update_os + +# Step 3: SSH and MOTD +motd_ssh + +# Step 4: Install application (app-specific) +# ... application installation steps ... + +# Step 5: Create update script +customize +``` + +--- + +## Best Practices + +### 1. **Always Initialize First** + +```bash +#!/bin/bash +set -Eeuo pipefail + +if ! command -v curl >/dev/null 2>&1; then + apt-get update >/dev/null 2>&1 + apt-get install -y curl >/dev/null 2>&1 +fi + +source <(curl -fsSL .../core.func) +source <(curl -fsSL .../error_handler.func) +load_functions +catch_errors +``` + +### 2. **Check Network Early** + +```bash +setting_up_container # Verify network available +network_check # Validate connectivity and DNS +update_os # Proceed with updates + +# If network fails, exit immediately +# Don't waste time on installation +``` + +### 3. **Use Retry Logic** + +```bash +# Built into setting_up_container(): +for ((i = RETRY_NUM; i > 0; i--)); do + if [ "$(hostname -I)" != "" ]; then + break + fi + sleep $RETRY_EVERY +done + +# Tolerates temporary network delay +``` + +### 4. **Separate Concerns** + +```bash +# Network setup +verb_ip6 +setting_up_container +network_check + +# System updates +update_os + +# Configuration +motd_ssh + +# Application-specific +# ... app installation ... +``` + +### 5. **Capture Environment** + +```bash +# Pass these from build.func: +VERBOSE="yes" # Show all output +DISABLEIPV6="no" # Keep IPv6 +SSH_ROOT="yes" # Enable SSH +APPLICATION="Jellyfin" # App name +CACHER="no" # No APT cache +``` + +--- + +## Debugging + +### Enable Verbose Output + +```bash +VERBOSE="yes" pct exec CTID bash /tmp/install.sh +# Shows all commands and output +``` + +### Check Network Status Inside Container + +```bash +pct exec CTID hostname -I +pct exec CTID ping -c 1 1.1.1.1 +pct exec CTID getent hosts github.com +``` + +### View Installation Log + +```bash +# From container +cat /root/install-*.log + +# Or from host (if logs mounted) +tail -100 /var/log/community-scripts/install-*.log +``` + +--- + +## Contributing + +### Adding New Network Checks + +```bash +network_check() { + # ... existing checks ... + + # Add new check: + if ! getent hosts newhost.example.com &>/dev/null; then + msg_warn "Unable to resolve newhost.example.com" + fi +} +``` + +### Extending OS Configuration + +```bash +# Add to update_os(): +update_os() { + # ... existing updates ... + + # Add new capability: + $STD apt-get install -y some-package + msg_ok "Additional package installed" +} +``` + +--- + +## Notes + +- Install.func executes **inside the container** (not on Proxmox host) +- Network connectivity is **critical** - checked early and thoroughly +- OS updates are **required** before application installation +- IPv6 is **configurable** but enabled by default +- SSH and MOTD are **informational** - help with container management + diff --git a/docs/misc/api.func.md b/docs/misc/api.func.md new file mode 100644 index 000000000..17c635400 --- /dev/null +++ b/docs/misc/api.func.md @@ -0,0 +1,670 @@ +# API.func Wiki + +A telemetry and diagnostics module providing anonymous statistics collection and API integration with the Community-Scripts infrastructure for tracking container/VM creation metrics and installation success/failure data. + +--- + +## πŸ“‹ Table of Contents + +- [Overview](#overview) +- [Exit Code Reference](#exit-code-reference) +- [Telemetry Functions](#telemetry-functions) +- [API Payload Structure](#api-payload-structure) +- [Privacy & Opt-Out](#privacy--opt-out) +- [Error Mapping](#error-mapping) +- [Best Practices](#best-practices) +- [API Integration](#api-integration) +- [Contributing](#contributing) + +--- + +## Overview + +The API.func module provides anonymous telemetry reporting to Community-Scripts infrastructure, enabling: + +- βœ… Container/VM creation statistics collection +- βœ… Installation success/failure tracking +- βœ… Comprehensive exit code mapping and explanation +- βœ… Anonymous session-based tracking (UUID) +- βœ… Privacy-respecting data collection (no personal data) +- βœ… Opt-out capability via DIAGNOSTICS setting +- βœ… Consistent error reporting across all scripts + +### Integration Points + +```bash +# In container build scripts (on Proxmox host): +source <(curl -fsSL .../api.func) +post_to_api # Report container creation +post_update_to_api # Report installation completion + +# Error handling (in all scripts): +source <(curl -fsSL .../error_handler.func) +# explain_exit_code shared for consistent mappings +``` + +### Data Flow + +``` +Container/VM Creation + ↓ + post_to_api() + ↓ +Community-Scripts API + ↓ +Anonymous Statistics +(No personal data) +``` + +--- + +## Exit Code Reference + +### Category 1: Generic / Shell Errors + +| Code | Meaning | Recovery | +|------|---------|----------| +| 1 | General error / Operation not permitted | Check permissions, re-run command | +| 2 | Misuse of shell builtins (syntax error) | Fix shell syntax, validate script | +| 126 | Command invoked cannot execute | Fix file permissions (chmod +x) | +| 127 | Command not found | Install missing package or tool | +| 128 | Invalid argument to exit | Check exit code parameter (0-255) | +| 130 | Terminated by Ctrl+C (SIGINT) | User interrupted - retry manually | +| 137 | Killed (SIGKILL / Out of memory) | Insufficient RAM - increase allocation | +| 139 | Segmentation fault (core dumped) | Serious application bug - contact support | +| 143 | Terminated (SIGTERM) | System shutdown or manual termination | + +### Category 2: Package Manager Errors + +| Code | Meaning | Recovery | +|------|---------|----------| +| 100 | APT: Package manager error (broken packages) | Run `apt --fix-broken install` | +| 101 | APT: Configuration error (bad sources.list) | Fix /etc/apt/sources.list, re-run apt update | +| 255 | DPKG: Fatal internal error | Run `dpkg --configure -a` | + +### Category 3: Node.js / npm Errors + +| Code | Meaning | Recovery | +|------|---------|----------| +| 243 | Node.js: Out of memory (heap out of memory) | Increase container RAM, reduce workload | +| 245 | Node.js: Invalid command-line option | Check node/npm arguments | +| 246 | Node.js: Internal JavaScript Parse Error | Update Node.js version | +| 247 | Node.js: Fatal internal error | Check Node.js installation integrity | +| 248 | Node.js: Invalid C++ addon / N-API failure | Rebuild native modules | +| 249 | Node.js: Inspector error | Disable debugger, retry | +| 254 | npm/pnpm/yarn: Unknown fatal error | Check package.json, clear cache | + +### Category 4: Python Errors + +| Code | Meaning | Recovery | +|------|---------|----------| +| 210 | Python: Virtualenv / uv environment missing | Recreate virtual environment | +| 211 | Python: Dependency resolution failed | Check package versions, fix conflicts | +| 212 | Python: Installation aborted (EXTERNALLY-MANAGED) | Use venv or remove marker file | + +### Category 5: Database Errors + +#### PostgreSQL + +| Code | Meaning | Recovery | +|------|---------|----------| +| 231 | Connection failed (server not running) | Start PostgreSQL service | +| 232 | Authentication failed (bad user/password) | Verify credentials | +| 233 | Database does not exist | Create database: `createdb dbname` | +| 234 | Fatal error in query / syntax error | Fix SQL syntax | + +#### MySQL / MariaDB + +| Code | Meaning | Recovery | +|------|---------|----------| +| 241 | Connection failed (server not running) | Start MySQL/MariaDB service | +| 242 | Authentication failed (bad user/password) | Reset password, verify credentials | +| 243 | Database does not exist | Create database: `CREATE DATABASE dbname;` | +| 244 | Fatal error in query / syntax error | Fix SQL syntax | + +#### MongoDB + +| Code | Meaning | Recovery | +|------|---------|----------| +| 251 | Connection failed (server not running) | Start MongoDB daemon | +| 252 | Authentication failed (bad user/password) | Verify credentials, reset if needed | +| 253 | Database not found | Create database in MongoDB shell | +| 254 | Fatal query error | Check query syntax | + +### Category 6: Proxmox Custom Codes + +| Code | Meaning | Recovery | +|------|---------|----------| +| 200 | Failed to create lock file | Check /tmp permissions | +| 203 | Missing CTID variable | CTID must be provided to script | +| 204 | Missing PCT_OSTYPE variable | OS type not detected | +| 205 | Invalid CTID (<100) | Container ID must be >= 100 | +| 206 | CTID already in use | Check `pct list`, remove conflicting container | +| 207 | Password contains special characters | Use alphanumeric only for passwords | +| 208 | Invalid configuration format | Check DNS/MAC/Network format | +| 209 | Container creation failed | Check pct create output for details | +| 210 | Cluster not quorate | Ensure cluster nodes are online | +| 211 | Timeout waiting for template lock | Wait for concurrent downloads to finish | +| 214 | Not enough storage space | Free up disk space or expand storage | +| 215 | Container created but not listed | Check /etc/pve/lxc/ for config files | +| 216 | RootFS entry missing in config | Incomplete container creation | +| 217 | Storage does not support rootdir | Use compatible storage backend | +| 218 | Template corrupted or incomplete | Re-download template | +| 220 | Unable to resolve template path | Verify template availability | +| 221 | Template not readable | Fix file permissions | +| 222 | Template download failed (3 attempts) | Check network/storage | +| 223 | Template not available after download | Storage sync issue | +| 225 | No template for OS/Version | Run `pveam available` to see options | +| 231 | LXC stack upgrade/retry failed | Update pve-container package | + +--- + +## Telemetry Functions + +### `explain_exit_code()` + +**Purpose**: Maps numeric exit codes to human-readable error descriptions. Shared between api.func and error_handler.func for consistency. + +**Signature**: +```bash +explain_exit_code() +``` + +**Parameters**: +- `$1` - Numeric exit code (0-255) + +**Returns**: Human-readable description string + +**Supported Codes**: +- 1-2, 126-128, 130, 137, 139, 143 (Shell) +- 100-101, 255 (Package managers) +- 210-212 (Python) +- 231-234 (PostgreSQL) +- 241-244 (MySQL/MariaDB) +- 243-249, 254 (Node.js/npm) +- 251-254 (MongoDB) +- 200-231 (Proxmox custom) + +**Default**: Returns "Unknown error" for unmapped codes + +**Usage Examples**: + +```bash +# Example 1: Common error +explain_exit_code 127 +# Output: "Command not found" + +# Example 2: Database error +explain_exit_code 241 +# Output: "MySQL/MariaDB: Connection failed (server not running / wrong socket)" + +# Example 3: Custom Proxmox error +explain_exit_code 206 +# Output: "Custom: CTID already in use (check 'pct list' and /etc/pve/lxc/)" + +# Example 4: Unknown code +explain_exit_code 999 +# Output: "Unknown error" +``` + +--- + +### `post_to_api()` + +**Purpose**: Sends LXC container creation statistics to Community-Scripts telemetry API. + +**Signature**: +```bash +post_to_api() +``` + +**Parameters**: None (uses global environment variables) + +**Returns**: No explicit return value (curl result stored in RESPONSE if diagnostics enabled) + +**Requirements** (Silent fail if not met): +- `curl` command available +- `DIAGNOSTICS="yes"` +- `RANDOM_UUID` is set +- Executed on Proxmox host (has access to `pveversion`) + +**Environment Variables Used**: +- `CT_TYPE` - Container type (privileged=1, unprivileged=0) +- `DISK_SIZE` - Allocated disk in GB +- `CORE_COUNT` - CPU core count +- `RAM_SIZE` - RAM allocated in MB +- `var_os` - Operating system name +- `var_version` - OS version +- `NSAPP` - Normalized application name +- `METHOD` - Installation method (default, template, etc.) +- `DIAGNOSTICS` - Enable telemetry (yes/no) +- `RANDOM_UUID` - Session UUID for tracking + +**API Endpoint**: `http://api.community-scripts.org/dev/upload` + +**Payload Structure**: +```json +{ + "ct_type": 1, // Privileged (1) or Unprivileged (0) + "type": "lxc", // Always "lxc" for containers + "disk_size": 8, // GB + "core_count": 2, // CPU cores + "ram_size": 2048, // MB + "os_type": "debian", // OS name + "os_version": "12", // OS version + "nsapp": "myapp", // Application name + "method": "default", // Setup method + "pve_version": "8.2.2", // Proxmox VE version + "status": "installing", // Current status + "random_id": "550e8400-e29b" // Session UUID (anonymous) +} +``` + +**Usage Examples**: + +```bash +# Example 1: Successful API post +CT_TYPE=1 +DISK_SIZE=20 +CORE_COUNT=4 +RAM_SIZE=4096 +var_os="ubuntu" +var_version="22.04" +NSAPP="jellyfin" +METHOD="default" +DIAGNOSTICS="yes" +RANDOM_UUID="550e8400-e29b-41d4-a716-446655440000" + +post_to_api +# Result: Statistics sent to API (silently, no output) + +# Example 2: Diagnostics disabled (opt-out) +DIAGNOSTICS="no" +post_to_api +# Result: Function returns immediately, no API call + +# Example 3: Missing curl +DIAGNOSTICS="yes" +# curl not available in PATH +post_to_api +# Result: Function returns silently (curl requirement not met) +``` + +--- + +### `post_to_api_vm()` + +**Purpose**: Sends VM creation statistics to Community-Scripts API (similar to post_to_api but for virtual machines). + +**Signature**: +```bash +post_to_api_vm() +``` + +**Parameters**: None (uses global environment variables) + +**Returns**: No explicit return value + +**Requirements**: Same as `post_to_api()` + +**Environment Variables Used**: +- `VMID` - Virtual machine ID +- `VM_TYPE` - VM type (kvm, etc.) +- `VM_CORES` - CPU core count +- `VM_RAM` - RAM in MB +- `VM_DISK` - Disk in GB +- `VM_OS` - Operating system +- `VM_VERSION` - OS version +- `VM_APP` - Application name +- `DIAGNOSTICS` - Enable telemetry +- `RANDOM_UUID` - Session UUID + +**Payload Structure** (similar to containers but for VMs): +```json +{ + "vm_id": 100, + "type": "qemu", + "vm_cores": 4, + "vm_ram": 4096, + "vm_disk": 20, + "vm_os": "ubuntu", + "vm_version": "22.04", + "vm_app": "jellyfin", + "pve_version": "8.2.2", + "status": "installing", + "random_id": "550e8400-e29b" +} +``` + +--- + +### `post_update_to_api()` + +**Purpose**: Reports installation completion status (success/failure) for container or VM. + +**Signature**: +```bash +post_update_to_api() +``` + +**Parameters**: None (uses global environment variables) + +**Returns**: No explicit return value + +**Requirements**: Same as `post_to_api()` + +**Environment Variables Used**: +- `RANDOM_UUID` - Session UUID (must match initial post_to_api call) +- `DIAGNOSTICS` - Enable telemetry +- Installation status parameters + +**Payload Structure**: +```json +{ + "status": "completed", // "completed" or "failed" + "random_id": "550e8400-e29b", // Session UUID + "exit_code": 0, // 0 for success, error code for failure + "error_explanation": "" // Error description if failed +} +``` + +--- + +## API Payload Structure + +### Container Creation Payload + +```json +{ + "ct_type": 1, // 1=Privileged, 0=Unprivileged + "type": "lxc", // Always "lxc" + "disk_size": 20, // GB + "core_count": 4, // CPU cores + "ram_size": 4096, // MB + "os_type": "debian", // Distribution name + "os_version": "12", // Version number + "nsapp": "jellyfin", // Application name + "method": "default", // Setup method + "pve_version": "8.2.2", // Proxmox VE version + "status": "installing", // Current phase + "random_id": "550e8400" // Unique session ID +} +``` + +### VM Creation Payload + +```json +{ + "vm_id": 100, + "type": "qemu", + "vm_cores": 4, + "vm_ram": 4096, + "vm_disk": 20, + "vm_os": "ubuntu", + "vm_version": "22.04", + "vm_app": "jellyfin", + "pve_version": "8.2.2", + "status": "installing", + "random_id": "550e8400" +} +``` + +### Update/Completion Payload + +```json +{ + "status": "completed", + "random_id": "550e8400", + "exit_code": 0, + "error_explanation": "" +} +``` + +--- + +## Privacy & Opt-Out + +### Privacy Policy + +Community-Scripts telemetry is designed to be **privacy-respecting**: + +- βœ… **Anonymous**: No personal data collected +- βœ… **Session-based**: UUID allows correlation without identification +- βœ… **Aggregated**: Only statistics are stored, never raw logs +- βœ… **Opt-out capable**: Single environment variable disables all telemetry +- βœ… **No tracking**: UUID cannot be linked to user identity +- βœ… **No credentials**: Passwords, SSH keys never transmitted + +### Opt-Out Methods + +**Method 1: Environment Variable (Single Script)** + +```bash +DIAGNOSTICS="no" bash ct/myapp.sh +``` + +**Method 2: Script Header (Persistent)** + +```bash +#!/bin/bash +export DIAGNOSTICS="no" +# Rest of script continues without telemetry +``` + +**Method 3: System-wide Configuration** + +```bash +# In /etc/environment or ~/.bashrc +export DIAGNOSTICS="no" +``` + +### What Data Is Collected + +| Data | Why | Shared? | +|------|-----|---------| +| Container/VM specs (cores, RAM, disk) | Understand deployment patterns | Yes, aggregated | +| OS type/version | Track popular distributions | Yes, aggregated | +| Application name | Understand popular apps | Yes, aggregated | +| Method (standard vs. custom) | Measure feature usage | Yes, aggregated | +| Success/failure status | Identify issues | Yes, aggregated | +| Exit codes | Debug failures | Yes, anonymized | + +### What Data Is NOT Collected + +- ❌ Container/VM hostnames +- ❌ IP addresses +- ❌ User credentials +- ❌ SSH keys or secrets +- ❌ Application data +- ❌ System logs +- ❌ Any personal information + +--- + +## Error Mapping + +### Mapping Strategy + +Exit codes are categorized by source: + +``` +Exit Code Range | Source | Handling +0 | Success | Not reported to API +1-2 | Shell/Script | Generic error +100-101, 255 | Package managers | APT/DPKG specific +126-128 | Command execution | Permission/not found +130, 143 | Signals | User interrupt/termination +137, 139 | Kernel | OOM/segfault +200-231 | Proxmox custom | Container creation issues +210-212 | Python | Python environment issues +231-234 | PostgreSQL | Database connection issues +241-244 | MySQL/MariaDB | Database connection issues +243-249, 254 | Node.js/npm | Runtime errors +251-254 | MongoDB | Database connection issues +``` + +### Custom Exit Code Usage + +Scripts can define custom exit codes: + +```bash +# Example: Custom validation failure +if [[ "$CTID" -lt 100 ]]; then + echo "Container ID must be >= 100" + exit 205 # Custom Proxmox code +fi +``` + +--- + +## Best Practices + +### 1. **Always Initialize RANDOM_UUID** + +```bash +# Generate unique session ID for tracking +RANDOM_UUID="$(cat /proc/sys/kernel/random/uuid)" + +# Use first 8 chars for short session ID (logs) +SESSION_ID="${RANDOM_UUID:0:8}" +BUILD_LOG="/tmp/create-lxc-${SESSION_ID}.log" +``` + +### 2. **Call post_to_api Early** + +```bash +# Call post_to_api right after container creation starts +# This tracks attempt, even if installation fails + +variables() { + RANDOM_UUID="$(cat /proc/sys/kernel/random/uuid)" + # ... other variables ... +} + +# Later, in main script: +post_to_api # Report container creation started +# ... perform installation ... +post_update_to_api # Report completion +``` + +### 3. **Handle Graceful Failures** + +```bash +# Wrap API calls to handle network issues +if command -v curl &>/dev/null; then + post_to_api || true # Don't fail if API unavailable +else + msg_warn "curl not available, telemetry skipped" +fi +``` + +### 4. **Respect User Opt-Out** + +```bash +# Check DIAGNOSTICS early and skip all API calls if disabled +if [[ "${DIAGNOSTICS}" != "yes" ]]; then + msg_info "Anonymous diagnostics disabled" + return 0 # Skip telemetry +fi +``` + +### 5. **Maintain Session Consistency** + +```bash +# Use same RANDOM_UUID throughout lifecycle +RANDOM_UUID="$(cat /proc/sys/kernel/random/uuid)" + +post_to_api # Initial report +# ... installation ... +post_update_to_api # Final report (same UUID links them) +``` + +--- + +## API Integration + +### Connecting to API + +The API endpoint is: + +``` +http://api.community-scripts.org/dev/upload +``` + +### API Response Handling + +```bash +# Capture HTTP response code +RESPONSE=$(curl -s -w "%{http_code}" -L -X POST "$API_URL" \ + -H "Content-Type: application/json" \ + -d "$JSON_PAYLOAD") || true + +# Extract status code (last 3 digits) +HTTP_CODE="${RESPONSE: -3}" + +if [[ "$HTTP_CODE" == "200" ]]; then + msg_ok "Telemetry submitted successfully" +elif [[ "$HTTP_CODE" == "429" ]]; then + msg_warn "API rate limited, skipping telemetry" +else + msg_info "Telemetry API unreachable (this is OK)" +fi +``` + +### Network Resilience + +API calls are **best-effort** and never block installation: + +```bash +# Telemetry should never cause container creation to fail +if post_to_api 2>/dev/null; then + msg_info "Diagnostics transmitted" +fi +# If API unavailable, continue anyway +``` + +--- + +## Contributing + +### Adding New Exit Codes + +1. Document in the appropriate category section +2. Update `explain_exit_code()` in both api.func and error_handler.func +3. Add recovery suggestions +4. Test mapping with scripts that use the new code + +### Testing API Integration + +```bash +# Test with mock curl (local testing) +DIAGNOSTICS="yes" +RANDOM_UUID="test-uuid-12345678" +curl -X POST http://localhost:8000/dev/upload \ + -H "Content-Type: application/json" \ + -d '{"test": "payload"}' + +# Verify payload structure +post_to_api 2>&1 | head -20 +``` + +### Telemetry Reporting Improvements + +Suggestions for improvement: + +1. Installation duration tracking +2. Package version compatibility data +3. Feature usage analytics +4. Performance metrics +5. Custom error codes + +--- + +## Notes + +- API calls are **silent by default** and never display sensitive information +- Telemetry can be **completely disabled** via `DIAGNOSTICS="no"` +- **RANDOM_UUID must be generated** before calling any post functions +- Exit code mappings are **shared** between api.func and error_handler.func for consistency +- API is **optional** - containers work perfectly without telemetry + diff --git a/misc/tools.func.md b/docs/tools.func.md similarity index 100% rename from misc/tools.func.md rename to docs/tools.func.md