ProxmoxVED/docs/tools.func.md

Tools.func Wiki

A comprehensive collection of helper functions for robust package management and repository management in Debian/Ubuntu-based systems.

---

📋 Table of Contents

• Overview
• Core Helper Functions
• Repository Management
• Package Management
• Tool Installation Functions
• GitHub Integration
• System Utilities
• Container Setup Functions

---

Overview

This function library provides:

• ✅ Automatic retry logic for APT/network failures
• ✅ Unified keyring cleanup from all 3 locations
• ✅ Legacy installation cleanup (nvm, rbenv, rustup)
• ✅ OS-upgrade-safe repository preparation
• ✅ Service pattern matching for multi-version tools

Usage in Install Scripts

source /dev/stdin <<< "$FUNCTIONS"  # Load from build.func
prepare_repository_setup "mysql"
install_packages_with_retry "mysql-server" "mysql-client"

---

Core Helper Functions

cache_installed_version()

Purpose: Caches installed version to avoid repeated checks.

Parameters:

• $1 - Application name
• $2 - Version string

Example:

cache_installed_version "nodejs" "22.0.0"

---

get_cached_version()

Purpose: Retrieves cached version of an application.

Parameters:

• $1 - Application name

Returns: Version string or empty if not cached

Example:

version=$(get_cached_version "nodejs")

---

cleanup_tool_keyrings()

Purpose: Removes ALL keyring files for specified tools from all 3 locations.

Parameters:

• $@ - Tool name patterns (supports wildcards)

Example:

cleanup_tool_keyrings "mariadb" "mysql" "postgresql"

---

stop_all_services()

Purpose: Stops and disables all service instances matching a pattern.

Parameters:

• $@ - Service name patterns (supports wildcards)

Example:

stop_all_services "php*-fpm" "mysql" "mariadb"

---

verify_tool_version()

Purpose: Verifies installed tool version matches expected version.

Parameters:

• $1 - Tool name
• $2 - Expected version
• $3 - Installed version

Returns: 0 if match, 1 if mismatch

Example:

verify_tool_version "nodejs" "22" "$(node -v | grep -oP '^v\K[0-9]+')"

---

cleanup_legacy_install()

Purpose: Removes legacy installation methods (nvm, rbenv, rustup, etc.).

Parameters:

• $1 - Tool name (nodejs, ruby, rust, go)

Example:

cleanup_legacy_install "nodejs"  # Removes nvm

---

Repository Management

prepare_repository_setup()

Purpose: Unified repository preparation before setup. Cleans up old repos, keyrings, and ensures APT is working.

Parameters:

• $@ - Repository names

Example:

prepare_repository_setup "mariadb" "mysql"

---

manage_tool_repository()

Purpose: Unified repository management for tools. Handles adding, updating, and verifying tool repositories.

Parameters:

• $1 - Tool name (mariadb, mongodb, nodejs, postgresql, php, mysql)
• $2 - Version
• $3 - Repository URL
• $4 - GPG key URL (optional)

Supported Tools: mariadb, mongodb, nodejs, postgresql, php, mysql

Example:

manage_tool_repository "mariadb" "11.4" \
  "http://mirror.mariadb.org/repo/11.4" \
  "https://mariadb.org/mariadb_release_signing_key.asc"

---

setup_deb822_repo()

Purpose: Standardized deb822 repository setup with optional architectures. Always runs apt update after repo creation.

Parameters:

• $1 - Repository name
• $2 - GPG key URL
• $3 - Repository URL
• $4 - Suite
• $5 - Component (default: main)
• $6 - Architectures (optional)

Example:

setup_deb822_repo "adoptium" \
  "https://packages.adoptium.net/artifactory/api/gpg/key/public" \
  "https://packages.adoptium.net/artifactory/deb" \
  "bookworm" \
  "main"

---

cleanup_old_repo_files()

Purpose: Cleanup old repository files (migration helper for OS upgrades).

Parameters:

• $1 - Application name

Example:

cleanup_old_repo_files "mariadb"

---

cleanup_orphaned_sources()

Purpose: Cleanup orphaned .sources files that reference missing keyrings. Prevents APT signature verification errors.

Example:

cleanup_orphaned_sources

---

ensure_apt_working()

Purpose: Ensures APT is in a working state before installing packages.

Returns: 0 if APT is working, 1 if critically broken

Example:

ensure_apt_working || return 1

---

get_fallback_suite()

Purpose: Get fallback suite for repository with comprehensive mapping.

Parameters:

• $1 - Distribution ID (debian, ubuntu)
• $2 - Distribution codename
• $3 - Repository base URL

Returns: Appropriate suite name

Example:

suite=$(get_fallback_suite "debian" "trixie" "https://repo.example.com")

---

Package Management

install_packages_with_retry()

Purpose: Install packages with retry logic (3 attempts with APT refresh).

Parameters:

• $@ - Package names

Example:

install_packages_with_retry "mysql-server" "mysql-client"

---

upgrade_packages_with_retry()

Purpose: Upgrade specific packages with retry logic.

Parameters:

• $@ - Package names

Example:

upgrade_packages_with_retry "mariadb-server" "mariadb-client"

---

ensure_dependencies()

Purpose: Ensures dependencies are installed (with apt update caching).

Parameters:

• $@ - Dependency names

Example:

ensure_dependencies "curl" "jq" "git"

---

is_package_installed()

Purpose: Check if package is installed (faster than dpkg -l | grep).

Parameters:

• $1 - Package name

Returns: 0 if installed, 1 if not

Example:

if is_package_installed "nginx"; then
  echo "Nginx is installed"
fi

---

hold_package_version()

Purpose: Hold package version to prevent upgrades.

Parameters:

• $1 - Package name

Example:

hold_package_version "mysql-server"

---

unhold_package_version()

Purpose: Unhold package version to allow upgrades.

Parameters:

• $1 - Package name

Example:

unhold_package_version "mysql-server"

---

Tool Installation Functions

is_tool_installed()

Purpose: Check if tool is already installed and optionally verify exact version.

Parameters:

• $1 - Tool name
• $2 - Required version (optional)

Returns: 0 if installed (with optional version match), 1 if not installed

Supported Tools: mariadb, mysql, mongodb, node, php, postgres, ruby, rust, go, clickhouse

Example:

is_tool_installed "mariadb" "11.4" || echo "Not installed"

---

remove_old_tool_version()

Purpose: Remove old tool version completely (purge + cleanup repos).

Parameters:

• $1 - Tool name
• $2 - Repository name (optional, defaults to tool name)

Example:

remove_old_tool_version "mariadb" "repository-name"

---

should_update_tool()

Purpose: Determine if tool update/upgrade is needed.

Parameters:

• $1 - Tool name
• $2 - Target version

Returns: 0 (update needed), 1 (already up-to-date)

Example:

if should_update_tool "mariadb" "11.4"; then
  echo "Update needed"
fi

---

setup_mariadb()

Purpose: Installs or updates MariaDB from official repo.

Variables:

• MARIADB_VERSION - MariaDB version to install (default: latest)

Example:

MARIADB_VERSION="11.4" setup_mariadb

---

setup_mysql()

Purpose: Installs or upgrades MySQL and configures APT repo.

Variables:

• MYSQL_VERSION - MySQL version to install (default: 8.0)

Features:

• Handles Debian Trixie libaio1t64 transition
• Auto-fallback to MariaDB if MySQL 8.0 unavailable

Example:

MYSQL_VERSION="8.0" setup_mysql

---

setup_mongodb()

Purpose: Installs or updates MongoDB to specified major version.

Variables:

• MONGO_VERSION - MongoDB major version (default: 8.0)

Example:

MONGO_VERSION="7.0" setup_mongodb

---

setup_postgresql()

Purpose: Installs or upgrades PostgreSQL and optional extensions.

Variables:

• PG_VERSION - PostgreSQL major version (default: 16)
• PG_MODULES - Comma-separated list of extensions

Example:

PG_VERSION="16" PG_MODULES="postgis,contrib" setup_postgresql

---

setup_nodejs()

Purpose: Installs Node.js and optional global modules.

Variables:

• NODE_VERSION - Node.js version (default: 22)
• NODE_MODULE - Comma-separated list of global modules

Example:

NODE_VERSION="22" NODE_MODULE="yarn,@vue/cli@5.0.0" setup_nodejs

---

setup_php()

Purpose: Installs PHP with selected modules and configures Apache/FPM support.

Variables:

• PHP_VERSION - PHP version (default: 8.4)
• PHP_MODULE - Additional comma-separated modules
• PHP_APACHE - Set YES to enable PHP with Apache
• PHP_FPM - Set YES to enable PHP-FPM
• PHP_MEMORY_LIMIT - Memory limit (default: 512M)
• PHP_UPLOAD_MAX_FILESIZE - Upload max filesize (default: 128M)
• PHP_POST_MAX_SIZE - Post max size (default: 128M)
• PHP_MAX_EXECUTION_TIME - Max execution time (default: 300)

Example:

PHP_VERSION="8.4" PHP_MODULE="redis,imagick" PHP_FPM="YES" setup_php

---

setup_java()

Purpose: Installs Temurin JDK via Adoptium APT repository.

Variables:

• JAVA_VERSION - Temurin JDK version (default: 21)

Example:

JAVA_VERSION="21" setup_java

---

setup_ruby()

Purpose: Installs rbenv and ruby-build, installs Ruby and optionally Rails.

Variables:

• RUBY_VERSION - Ruby version (default: 3.4.4)
• RUBY_INSTALL_RAILS - true/false to install Rails (default: true)

Example:

RUBY_VERSION="3.4.4" RUBY_INSTALL_RAILS="true" setup_ruby

---

setup_rust()

Purpose: Installs Rust toolchain and optional global crates.

Variables:

• RUST_TOOLCHAIN - Rust toolchain (default: stable)
• RUST_CRATES - Comma-separated list of crates

Example:

RUST_TOOLCHAIN="stable" RUST_CRATES="cargo-edit,wasm-pack@0.12.1" setup_rust

---

setup_go()

Purpose: Installs Go (Golang) from official tarball.

Variables:

• GO_VERSION - Go version (default: latest)

Example:

GO_VERSION="1.22.2" setup_go

---

setup_composer()

Purpose: Installs or updates Composer globally (robust, idempotent).

Features:

• Installs to /usr/local/bin/composer
• Removes old binaries/symlinks
• Ensures /usr/local/bin is in PATH
• Auto-updates to latest version

Example:

setup_composer

---

setup_uv()

Purpose: Installs or upgrades uv (Python package manager) from GitHub releases.

Variables:

• USE_UVX - Set YES to install uvx wrapper (default: NO)
• PYTHON_VERSION - Optional Python version to install via uv

Example:

USE_UVX="YES" PYTHON_VERSION="3.12" setup_uv

---

setup_yq()

Purpose: Installs or updates yq (mikefarah/yq - Go version).

Example:

setup_yq

---

setup_ffmpeg()

Purpose: Installs FFmpeg from source or prebuilt binary.

Variables:

• FFMPEG_VERSION - FFmpeg version (default: latest)
• FFMPEG_TYPE - Build profile: minimal, medium, full, binary (default: full)

Example:

FFMPEG_VERSION="n7.1.1" FFMPEG_TYPE="full" setup_ffmpeg

---

setup_imagemagick()

Purpose: Installs ImageMagick 7 from source.

Example:

setup_imagemagick

---

setup_gs()

Purpose: Installs or updates Ghostscript (gs) from source.

Example:

setup_gs

---

setup_hwaccel()

Purpose: Sets up Hardware Acceleration for Intel/AMD/NVIDIA GPUs.

Example:

setup_hwaccel

---

setup_clickhouse()

Purpose: Installs or upgrades ClickHouse database server.

Variables:

• CLICKHOUSE_VERSION - ClickHouse version (default: latest)

Example:

CLICKHOUSE_VERSION="latest" setup_clickhouse

---

setup_adminer()

Purpose: Installs Adminer (supports Debian/Ubuntu and Alpine).

Example:

setup_adminer

---

GitHub Integration

check_for_gh_release()

Purpose: Checks for new GitHub release (latest tag).

Parameters:

• $1 - Application name
• $2 - GitHub repository (user/repo)
• $3 - Optional pinned version

Returns: 0 if update available, 1 if up-to-date

Global Variables Set:

• CHECK_UPDATE_RELEASE - Latest release tag

Example:

if check_for_gh_release "flaresolverr" "FlareSolverr/FlareSolverr"; then
  echo "Update available: $CHECK_UPDATE_RELEASE"
fi

---

fetch_and_deploy_gh_release()

Purpose: Downloads and deploys latest GitHub release.

Parameters:

• $1 - Application name
• $2 - GitHub repository (user/repo)
• $3 - Mode: tarball, binary, prebuild, singlefile (default: tarball)
• $4 - Version (default: latest)
• $5 - Target directory (default: /opt/app)
• $6 - Asset filename/pattern (required for prebuild/singlefile)

Modes:

• tarball - Source code tarball (.tar.gz)
• binary - .deb package install (arch-dependent)
• prebuild - Prebuilt .tar.gz archive
• singlefile - Standalone binary (chmod +x)

Example:

# Source tarball
fetch_and_deploy_gh_release "myapp" "myuser/myapp"

# Binary .deb
fetch_and_deploy_gh_release "myapp" "myuser/myapp" "binary"

# Prebuilt archive
fetch_and_deploy_gh_release "hanko" "teamhanko/hanko" "prebuild" \
  "latest" "/opt/hanko" "hanko_Linux_x86_64.tar.gz"

# Single binary
fetch_and_deploy_gh_release "argus" "release-argus/Argus" "singlefile" \
  "0.26.3" "/opt/argus" "Argus-.*linux-amd64"

---

github_api_call()

Purpose: GitHub API call with authentication and rate limit handling.

Parameters:

• $1 - API URL
• $2 - Output file (default: /dev/stdout)

Environment Variables:

• GITHUB_TOKEN - Optional GitHub token for higher rate limits

Example:

github_api_call "https://api.github.com/repos/user/repo/releases/latest" "/tmp/release.json"

---

get_latest_github_release()

Purpose: Get latest GitHub release version.

Parameters:

• $1 - GitHub repository (user/repo)
• $2 - Strip 'v' prefix (default: true)

Returns: Version string

Example:

version=$(get_latest_github_release "nodejs/node")

---

System Utilities

get_os_info()

Purpose: Get OS information (cached for performance).

Parameters:

• $1 - Field: id, codename, version, version_id, all (default: all)

Returns: Requested OS information

Example:

os_id=$(get_os_info id)
os_codename=$(get_os_info codename)

---

is_debian(), is_ubuntu(), is_alpine()

Purpose: Check if running on specific OS.

Returns: 0 if match, 1 if not

Example:

if is_debian; then
  echo "Running on Debian"
fi

---

get_os_version_major()

Purpose: Get Debian/Ubuntu major version.

Returns: Major version number

Example:

major_version=$(get_os_version_major)

---

get_system_arch()

Purpose: Get system architecture (normalized).

Parameters:

• $1 - Architecture type: dpkg, uname, both (default: both)

Returns: Architecture string (amd64, arm64)

Example:

arch=$(get_system_arch)

---

version_gt()

Purpose: Smart version comparison.

Parameters:

• $1 - Version 1
• $2 - Version 2

Returns: 0 if version 1 > version 2

Example:

if version_gt "2.0.0" "1.5.0"; then
  echo "Version 2.0.0 is greater"
fi

---

is_lts_version()

Purpose: Check if running on LTS version.

Returns: 0 if LTS, 1 if not

Example:

if is_lts_version; then
  echo "Running on LTS"
fi

---

get_parallel_jobs()

Purpose: Get optimal number of parallel jobs (cached).

Returns: Number of parallel jobs based on CPU and memory

Example:

jobs=$(get_parallel_jobs)
make -j"$jobs"

---

is_apt_locked()

Purpose: Check if package manager is locked.

Returns: 0 if locked, 1 if not

Example:

if is_apt_locked; then
  echo "APT is locked"
fi

---

wait_for_apt()

Purpose: Wait for apt to be available.

Parameters:

• $1 - Max wait time in seconds (default: 300)

Example:

wait_for_apt 600  # Wait up to 10 minutes

---

download_file()

Purpose: Download file with retry logic and progress.

Parameters:

• $1 - URL
• $2 - Output path
• $3 - Max retries (default: 3)
• $4 - Show progress (default: false)

Example:

download_file "https://example.com/file.tar.gz" "/tmp/file.tar.gz" 3 true

---

create_temp_dir()

Purpose: Create temporary directory with automatic cleanup.

Returns: Temporary directory path

Example:

tmp_dir=$(create_temp_dir)
# Directory is automatically cleaned up on exit

---

safe_service_restart()

Purpose: Safe service restart with verification.

Parameters:

• $1 - Service name

Example:

safe_service_restart "nginx"

---

enable_and_start_service()

Purpose: Enable and start service (with error handling).

Parameters:

• $1 - Service name

Example:

enable_and_start_service "postgresql"

---

is_service_enabled(), is_service_running()

Purpose: Check if service is enabled/running.

Parameters:

• $1 - Service name

Returns: 0 if yes, 1 if no

Example:

if is_service_running "nginx"; then
  echo "Nginx is running"
fi

---

create_self_signed_cert()

Purpose: Creates and installs self-signed certificates.

Parameters:

• $1 - Application name (optional, defaults to $APPLICATION)

Example:

create_self_signed_cert "myapp"

---

import_local_ip()

Purpose: Loads LOCAL_IP from persistent store or detects if missing.

Global Variables Set:

• LOCAL_IP - Local IP address

Example:

import_local_ip
echo "Local IP: $LOCAL_IP"

---

setup_local_ip_helper()

Purpose: Installs a local IP updater script using networkd-dispatcher.

Example:

setup_local_ip_helper

---

ensure_usr_local_bin_persist()

Purpose: Ensures /usr/local/bin is permanently in system PATH.

Example:

ensure_usr_local_bin_persist

---

download_with_progress()

Purpose: Downloads file with optional progress indicator using pv.

Parameters:

• $1 - URL
• $2 - Destination path

Example:

download_with_progress "https://example.com/file.tar.gz" "/tmp/file.tar.gz"

---

verify_gpg_fingerprint()

Purpose: GPG key fingerprint verification.

Parameters:

• $1 - Key file path
• $2 - Expected fingerprint

Example:

verify_gpg_fingerprint "/tmp/key.gpg" "ABCD1234..."

---

debug_log()

Purpose: Debug logging (only if DEBUG=1).

Parameters:

• $@ - Message to log

Example:

DEBUG=1 debug_log "This is a debug message"

---

start_timer(), end_timer()

Purpose: Performance timing helpers.

Example:

start_time=$(start_timer)
# ... do something ...
end_timer "$start_time" "Operation"

---

Container Setup Functions

color()

Purpose: Sets up color and formatting variables for terminal output.

Example:

color
echo -e "${GN}Success${CL}"

---

verb_ip6()

Purpose: Enables or disables IPv6 based on DISABLEIPV6 variable.

Variables:

• DISABLEIPV6 - Set "yes" to disable IPv6

Example:

DISABLEIPV6="yes" verb_ip6

---

catch_errors()

Purpose: Sets up error handling for the script.

Example:

catch_errors

---

error_handler()

Purpose: Handles errors that occur during script execution.

Parameters:

• $1 - Line number
• $2 - Command that failed

Example:

error_handler 42 "ls non_existent_file"

---

spinner()

Purpose: Displays a rotating spinner animation.

Example:

spinner &
SPINNER_PID=$!

---

msg_info(), msg_ok(), msg_error()

Purpose: Display messages with different statuses.

Parameters:

• $1 - Message text

Example:

msg_info "Installing packages..."
msg_ok "Installation complete"
msg_error "Installation failed"

---

setting_up_container()

Purpose: Sets up container OS, configures locale, timezone, and network.

Example:

setting_up_container

---

network_check()

Purpose: Verifies internet connectivity via IPv4 and IPv6.

Example:

network_check

---

update_os()

Purpose: Updates the container's OS using apt-get.

Variables:

• CACHER - Enable package caching proxy

Example:

update_os

---

motd_ssh()

Purpose: Modifies message of the day (MOTD) and SSH settings.

Example:

motd_ssh

---

customize()

Purpose: Customizes the container by enabling auto-login and setting up SSH keys.

Example:

customize

---

Best Practices

Version Management

Always cache versions after installation:

setup_nodejs
cache_installed_version "nodejs" "$NODE_VERSION"

Error Handling

Always check return codes:

if ! install_packages_with_retry "nginx"; then
  msg_error "Failed to install nginx"
  return 1
fi

Repository Setup

Always prepare repositories before installation:

prepare_repository_setup "mariadb" || return 1
manage_tool_repository "mariadb" "11.4" "$REPO_URL" "$GPG_URL" || return 1

APT Safety

Always ensure APT is working before operations:

ensure_apt_working || return 1
install_packages_with_retry "package-name"

---

Notes

• All functions use $STD variable for silent execution
• Functions support both fresh installs and upgrades
• Automatic fallback mechanisms for newer OS versions
• Version caching prevents redundant installations
• Comprehensive error handling and retry logic

---

License

This documentation is part of the community-scripts project.

---

Contributing

Contributions are welcome! Please follow the existing code style and add appropriate documentation for new functions.