[WIP] Allow multiple installations of same blueprint (#88)

* Multi-install support, Blueprints and config changes. Initial commit * Migrating jails to blueprints, first steps. Tested Working: - Transmission - Lidarr - Sonarr - Radarr fix lidarr config (+10 squashed commit) Squashed commit: [5f14653] always link ports folders [f18f2f0] Optional (blueprint) ports mount Fixes #89 [96ef7e7] chmod all the things [129e707] same mistake... again... [e1596dc] missing reference [6da3567] Forgot one reference [d78b5b6] Update wiki.yml [cecc53a] Update filecheck.yml [5244abd] basic settings changed. More involved blueprints still need changes, such as: Bitwarden, nextcloud, Mariadb [6568e92] jails -> blueprints * Added Tested Working: - KMS - Plex - Tautulli - Organizr - InfluxDB - MariaDB Many squashed small fixes included: Make *.rc executable (+13 squashed commit) Squashed commit: [b28aa83] use .rc for rc.d config files [e940a48] some mariadb cleanup [dc27aff] testing another way [83bd91b] Mariadb root password alter instead of update, initial config for unifi [0ca3074] some light config cleanup [a0d4352] also remove database from influxdb example config (db should be created when required) [2c218cc] Prepare influxdb and remove unneeded content [1b34109] more shellcheck fixups [c96566c] Some shellcheck cleanup [8969ca7] bitwarden mostly done, some work on nextcloud and unifi [7f89bfa] initial mariadb patch [dd7e85f] missed one problem [f814cb7] Initial pseudo-compatibility patch for unifi * Enable Bitwarden support and some small fixes/tweaks Fixes #95 more bugs and typo's (+3 squashed commit) Squashed commit: [3b5213e] Bitwarden not correctly installing db [b7438a5] yeah thats not gonna cut it... >.< [e7987c2] some slight bitwarden tweaks * Enable Unifi support and some small fixes/tweaks small unifi cleanup. Unifi is working (+3 squashed commit) Squashed commit: [d906d2d] chmod unifi [545e999] Add extra sanity, remove unneeded variables from example [b8c0b24] Some small Unifi Tweaks * Nextcloud Cleanup, Some fixes, Initial support for blueprintsystem Fixes #96 Fixes #97 Fixes #98 some bloat and syntax fixes (+5 squashed commit) Squashed commit: [78f6428] Some more nextcloud cleanup and tweaks - combines multiple variables for cert system selection (Fixes #98 ) - Default to self signed cert - Force manual admin password [7cacae4] slight fixes [3d81cda] More cleanup [50496cc] small mariadb fix and more nextcloud cleanup [c1b2c20] Cleaning nextcloud - Remove external DB (Fixes #97 ) - Remove Postgresql (Fixes #96 ) - Some preparation for blueprint * Nextcloud done and.. another... (+5 squashed commit) Squashed commit: [c65751b] caddy not installed right. [e5da66b] more fixes [a33300e] Damnit, two typo's same scentence [4292a7a] another typo [1b820cf] typo and example hotfix * Introduce version checking for config file
2020-05-02 17:45:13 +02:00
parent 66e997069a
commit dbfbd489fa
140 changed files with 1361 additions and 1888 deletions
--- a/blueprints/mariadb/config.yml
+++ b/blueprints/mariadb/config.yml
@ -0,0 +1,3 @@
+blueprint:
+  mariadb:
+    pkgs: mariadb104-server git  php74-session php74-xml php74-ctype php74-openssl php74-filter php74-gd php74-json php74-mysqli php74-mbstring php74-zlib php74-zip php74-bz2 phpMyAdmin5-php74 php74-pdo_mysql php74-mysqli phpMyAdmin5-php74-5.0.1
--- a/blueprints/mariadb/includes/Caddyfile
+++ b/blueprints/mariadb/includes/Caddyfile
@ -0,0 +1,11 @@
+yourhostnamehere:80 JAIL-IP:80 {
+  root /usr/local/www/phpMyAdmin/
+  fastcgi / 127.0.0.1:9000 php {
+        env PATH /bin
+        env modHeadersAvailable true
+        env front_controller_active true
+        connect_timeout 60s
+        read_timeout 3600s
+        send_timeout 300s
+        }
+}
--- a/blueprints/mariadb/includes/caddy.rc
+++ b/blueprints/mariadb/includes/caddy.rc
@ -0,0 +1,83 @@
+#!/bin/sh
+# shellcheck disable=1091,2223,2154,2034
+#
+# PROVIDE: caddy
+# REQUIRE: networking
+# KEYWORD: shutdown
+
+#
+# Add the following lines to /etc/rc.conf to enable caddy:
+# caddy_enable (bool):        Set to "NO" by default.
+#                             Set it to "YES" to enable caddy
+#
+# caddy_cert_email (str):     Set to "" by default.
+#                             Defines the SSL certificate issuer email. By providing an
+#                             email address you automatically agree to letsencrypt.org's
+#                             general terms and conditions
+#
+# caddy_bin_path (str):       Set to "/usr/local/bin/caddy" by default.
+#                             Provides the path to the caddy server executable
+#
+# caddy_cpu (str):            Set to "99%" by default.
+#                             Configures, how much CPU capacity caddy may gain
+#
+# caddy_config_path (str):    Set to "/usr/local/www/Caddyfile" by default.
+#                             Defines the path for the configuration file caddy will load on boot
+#
+# caddy_user (str):           Set to "root" by default.
+#                             Defines the user that caddy will run on
+#
+# caddy_group (str):          Set to "wheel" by default.
+#                             Defines the group that caddy files will be attached to
+#
+# caddy_logfile (str)         Set to "/var/log/caddy.log" by default.
+#                             Defines where the process log file is written, this is not a web access log
+#
+# caddy_env (str)             Set to "" by default.
+#                             This allows environment variable to be set that may be required, for example when using "DNS Challenge" account credentials are required.
+#                             e.g. (in your rc.conf)   caddy_env="CLOUDFLARE_EMAIL=me@domain.com CLOUDFLARE_API_KEY=my_api_key"
+#
+
+. /etc/rc.subr                                           
+
+name="caddy"
+rcvar="${name}_enable"
+
+load_rc_config ${name}
+
+: ${caddy_enable:="NO"}
+: ${caddy_cert_email=""}
+: ${caddy_bin_path="/usr/local/bin/caddy"}
+: ${caddy_cpu="99%"} # was a bug for me that caused a crash within jails
+: ${caddy_config_path="/usr/local/www/Caddyfile"}
+: ${caddy_logfile="/var/log/caddy.log"}
+: ${caddy_user="root"}
+: ${caddy_group="wheel"}
+
+if [ "$caddy_cert_email" = "" ]
+then
+    echo "rc variable \$caddy_cert_email is not set. Please provide a valid SSL certificate issuer email."
+    exit 1
+fi
+
+pidfile="/var/run/${name}.pid"
+procname="${caddy_bin_path}" #enabled builtin pid checking for start / stop
+command="/usr/sbin/daemon"
+command_args="-p ${pidfile} /usr/bin/env ${caddy_env} ${procname} -cpu ${caddy_cpu} -log stdout -conf ${caddy_config_path} -agree -email ${caddy_cert_email} < /dev/null >> ${caddy_logfile} 2>&1"
+
+start_precmd="caddy_startprecmd"
+
+caddy_startprecmd()
+{
+        if [ ! -e "${pidfile}" ]; then
+                install -o "${caddy_user}" -g "${caddy_group}" "/dev/null" "${pidfile}"
+        fi
+
+        if [ ! -e "${caddy_logfile}" ]; then
+                install -o "${caddy_user}" -g "${caddy_group}" "/dev/null" "${caddy_logfile}"
+        fi
+}
+
+required_files="${caddy_config_path}"
+
+run_rc_command "$1"
--- a/blueprints/mariadb/includes/my-system.cnf
+++ b/blueprints/mariadb/includes/my-system.cnf
@ -0,0 +1,16 @@
+[mysqld]
+innodb_file_per_table=1
+transaction_isolation = READ-COMMITTED
+binlog_format = ROW
+
+symbolic-links=0
+innodb_doublewrite = 0
+innodb_checksum_algorithm = none
+innodb_file_per_table=1
+
+innodb_buffer_pool_size = 1G
+innodb_log_buffer_size  = 8M
+innodb_open_files       = 400
+innodb_io_capacity      = 400
+innodb_flush_method     = O_DIRECT
+innodb_io_capacity      = 4000
--- a/blueprints/mariadb/includes/my.cnf
+++ b/blueprints/mariadb/includes/my.cnf
@ -0,0 +1,3 @@
+# MySQL client config file
+[client]
+password=mypassword
--- a/blueprints/mariadb/install.sh
+++ b/blueprints/mariadb/install.sh
@ -0,0 +1,117 @@
+#!/usr/local/bin/bash
+# This script installs the current release of Mariadb and PhpMyAdmin into a created jail
+#####
+# 
+# Init and Mounts
+#
+#####
+
+# Initialise defaults
+# shellcheck disable=SC2154
+JAIL_IP="jail_${1}_ip4_addr"
+JAIL_IP="${!JAIL_IP%/*}"
+INCLUDES_PATH="${SCRIPT_DIR}/blueprints/mariadb/includes"
+# shellcheck disable=SC2154
+CERT_EMAIL="jail_${1}_cert_email"
+CERT_EMAIL="${!CERT_EMAIL:-placeholder@email.fake}"
+# shellcheck disable=SC2154
+DB_ROOT_PASSWORD="jail_${1}_db_root_password"
+HOST_NAME="jail_${1}_host_name"
+DL_FLAGS=""
+DNS_ENV=""
+
+# Check that necessary variables were set by nextcloud-config
+if [ -z "${JAIL_IP}" ]; then
+  echo 'Configuration error: The mariadb jail does NOT accept DHCP'
+  echo 'Please reinstall using a fixed IP adress'
+  exit 1
+fi
+
+# Make sure DB_PATH is empty -- if not, MariaDB/PostgreSQL will choke
+# shellcheck disable=SC2154
+if [ "$(ls -A "/mnt/${global_dataset_config}/${1}/db")" ]; then
+	echo "Reinstall of mariadb detected... Continuing"
+	REINSTALL="true"
+fi
+
+# Mount database dataset and set zfs preferences
+createmount "${1}" "${global_dataset_config}"/"${1}"/db /var/db/mysql
+zfs set recordsize=16K "${global_dataset_config}"/"${1}"/db
+zfs set primarycache=metadata "${global_dataset_config}"/"${1}"/db
+
+iocage exec "${1}" chown -R 88:88 /var/db/mysql
+
+# Install includes fstab
+iocage exec "${1}" mkdir -p /mnt/includes
+iocage fstab -a "${1}" "${INCLUDES_PATH}" /mnt/includes nullfs rw 0 0
+
+iocage exec "${1}" mkdir -p /usr/local/www/phpmyadmin
+iocage exec "${1}" chown -R www:www /usr/local/www/phpmyadmin
+
+#####
+# 
+# Install mariadb, Caddy and PhpMyAdmin
+#
+#####
+
+fetch -o /tmp https://getcaddy.com
+if ! iocage exec "${1}" bash -s personal "${DL_FLAGS}" < /tmp/getcaddy.com
+then
+	echo "Failed to download/install Caddy"
+	exit 1
+fi
+
+iocage exec "${1}" sysrc mysql_enable="YES"
+
+# Copy and edit pre-written config files
+echo "Copying Caddyfile for no SSL"
+iocage exec "${1}" cp -f /mnt/includes/caddy.rc /usr/local/etc/rc.d/caddy
+iocage exec "${1}" cp -f /mnt/includes/Caddyfile /usr/local/www/Caddyfile
+# shellcheck disable=SC2154
+iocage exec "${1}" sed -i '' "s/yourhostnamehere/${!HOST_NAME}/" /usr/local/www/Caddyfile
+iocage exec "${1}" sed -i '' "s/JAIL-IP/${JAIL_IP}/" /usr/local/www/Caddyfile
+
+iocage exec "${1}" sysrc caddy_enable="YES"
+iocage exec "${1}" sysrc php_fpm_enable="YES"
+iocage exec "${1}" sysrc caddy_cert_email="${CERT_EMAIL}"
+iocage exec "${1}" sysrc caddy_env="${DNS_ENV}"
+
+iocage restart "${1}"
+sleep 10
+
+if [ "${REINSTALL}" == "true" ]; then
+	echo "Reinstall detected, skipping generaion of new config and database"
+else
+	
+	# Secure database, set root password, create Nextcloud DB, user, and password
+	iocage exec "${1}" cp -f /mnt/includes/my-system.cnf /var/db/mysql/my.cnf
+	iocage exec "${1}" mysql -u root -e "DELETE FROM mysql.user WHERE User='';"
+	iocage exec "${1}" mysql -u root -e "DELETE FROM mysql.user WHERE User='root' AND Host NOT IN ('localhost', '127.0.0.1', '::1');"
+	iocage exec "${1}" mysql -u root -e "DROP DATABASE IF EXISTS test;"
+	iocage exec "${1}" mysql -u root -e "DELETE FROM mysql.db WHERE Db='test' OR Db='test\\_%';"
+	iocage exec "${1}" mysqladmin --user=root password "${!DB_ROOT_PASSWORD}"
+	iocage exec "${1}" mysqladmin reload
+fi
+iocage exec "${1}" cp -f /mnt/includes/my.cnf /root/.my.cnf
+iocage exec "${1}" sed -i '' "s|mypassword|${!DB_ROOT_PASSWORD}|" /root/.my.cnf
+
+# Save passwords for later reference
+iocage exec "${1}" echo "MariaDB root password is ${!DB_ROOT_PASSWORD}" > /root/"${1}"_db_password.txt
+	
+
+# Don't need /mnt/includes any more, so unmount it
+iocage fstab -r "${1}" "${INCLUDES_PATH}" /mnt/includes nullfs rw 0 0
+
+# Done!
+echo "Installation complete!"
+echo "Using your web browser, go to http://${!HOST_NAME} to log in"
+
+if [ "${REINSTALL}" == "true" ]; then
+	echo "You did a reinstall, please use your old database and account credentials"
+else
+	echo "Database Information"
+	echo "--------------------"
+	echo "The MariaDB root password is ${!DB_ROOT_PASSWORD}"
+	fi
+echo ""
+echo "All passwords are saved in /root/${1}_db_password.txt"
--- a/blueprints/mariadb/readme.md
+++ b/blueprints/mariadb/readme.md
@ -0,0 +1,93 @@
+# MariaDB
+
+## Original README from the mariadb github:
+
+https://github.com/MariaDB/server/
+
+Code status:
+------------
+
+* [![Travis CI status](https://secure.travis-ci.org/MariaDB/server.png?branch=10.5)](https://travis-ci.org/MariaDB/server) travis-ci.org (10.5 branch)
+* [![Appveyor CI status](https://ci.appveyor.com/api/projects/status/4u6pexmtpuf8jq66?svg=true)](https://ci.appveyor.com/project/rasmushoj/server) ci.appveyor.com
+
+## MariaDB: drop-in replacement for MySQL
+
+MariaDB is designed as a drop-in replacement of MySQL(R) with more
+features, new storage engines, fewer bugs, and better performance.
+
+MariaDB is brought to you by the MariaDB Foundation and the MariaDB corporation.
+Please read the CREDITS file for details about the MariaDB Foundation,
+and who is developing MariaDB.
+
+MariaDB is developed by many of the original developers of MySQL who
+now work for the MariaDB Corporation, the MariaDB Foundation and by
+many people in the community.
+
+MySQL, which is the base of MariaDB, is a product and trademark of Oracle
+Corporation, Inc. For a list of developers and other contributors,
+see the Credits appendix.  You can also run 'SHOW authors' to get a
+list of active contributors.
+
+A description of the MariaDB project and a manual can be found at:
+
+https://mariadb.com/kb/en/
+
+https://mariadb.com/kb/en/mariadb-vs-mysql-features/
+
+https://mariadb.com/kb/en/mariadb-versus-mysql-compatibility/
+
+https://mariadb.com/kb/en/library/new-and-old-releases/
+
+https://mariadb.org/
+
+As MariaDB is a full replacement of MySQL, the MySQL manual at
+http://dev.mysql.com/doc is generally applicable.
+
+Help
+-----
+
+More help is available from the Maria Discuss mailing list
+https://launchpad.net/~maria-discuss
+and the #maria IRC channel on Freenode.
+
+Live QA for beginner contributors
+----
+MariaDB has a dedicated time each week when we answer new contributor questions live on Zulip and IRC.
+From 8:00 to 10:00 UTC on Mondays, and 10:00 to 12:00 UTC on Thursdays,
+anyone can ask any questions they’d like, and a live developer will be available to assist.
+
+New contributors can ask questions any time, but we will provide immediate feedback during that interval.
+
+Licensing
+---------
+
+***************************************************************************
+
+NOTE: 
+
+MariaDB is specifically available only under version 2 of the GNU
+General Public License (GPLv2). (I.e. Without the "any later version"
+clause.) This is inherited from MySQL. Please see the README file in
+the MySQL distribution for more information.
+
+License information can be found in the COPYING file. Third party
+license information can be found in the THIRDPARTY file.
+
+***************************************************************************
+
+Bug Reports
+------------
+
+Bug and/or error reports regarding MariaDB should be submitted at:
+https://jira.mariadb.org
+
+For reporting security vulnerabilities see:
+https://mariadb.org/about/security-policy/
+
+Bugs in the MySQL code can also be submitted at:
+https://bugs.mysql.com
+
+The code for MariaDB, including all revision history, can be found at:
+https://github.com/MariaDB/server
+
+***************************************************************************
--- a/blueprints/mariadb/update.sh
+++ b/blueprints/mariadb/update.sh
@ -0,0 +1,37 @@
+#!/usr/local/bin/bash
+# This file contains the update script for mariadb
+
+# shellcheck disable=SC2154
+JAIL_IP="jail_${1}_ip4_addr"
+JAIL_IP="${!JAIL_IP%/*}"
+HOST_NAME="jail_${1}_host_name"
+INCLUDES_PATH="${SCRIPT_DIR}/blueprints/mariadb/includes"
+
+# Install includes fstab
+iocage exec "${1}" mkdir -p /mnt/includes
+iocage fstab -a "${1}" "${INCLUDES_PATH}" /mnt/includes nullfs rw 0 0
+
+
+iocage exec "${1}" service caddy stop
+iocage exec "${1}" service php-fpm stop
+
+fetch -o /tmp https://getcaddy.com
+if ! iocage exec "${1}" bash -s personal "${DL_FLAGS}" < /tmp/getcaddy.com
+then
+	echo "Failed to download/install Caddy"
+	exit 1
+fi
+
+# Copy and edit pre-written config files
+echo "Copying Caddyfile for no SSL"
+iocage exec "${1}" cp -f /mnt/includes/caddy /usr/local/etc/rc.d/
+iocage exec "${1}" cp -f /mnt/includes/Caddyfile /usr/local/www/Caddyfile
+# shellcheck disable=SC2154
+iocage exec "${1}" sed -i '' "s/yourhostnamehere/${HOST_NAME}/" /usr/local/www/Caddyfile
+iocage exec "${1}" sed -i '' "s/JAIL-IP/${JAIL_IP}/" /usr/local/www/Caddyfile
+
+# Don't need /mnt/includes any more, so unmount it
+iocage fstab -r "${1}" "${INCLUDES_PATH}" /mnt/includes nullfs rw 0 0
+
+iocage exec "${1}" service caddy start
+iocage exec "${1}" service php-fpm start