diff --git a/README.md b/README.md index 37ecde2..4ea4bf8 100644 --- a/README.md +++ b/README.md @@ -2,22 +2,117 @@ The simple tool for managing container servers + ## Quick start - Learn to use `docker` with `docker-compose` - Install kiwi-config -- Look at [the example instance](../example) +- Look at [the example instance](./example) - Look at the output of `kiwi --help` - Start building your own instances + ## Installation +A convenience installer is available as [install.sh](./install.sh) in this directory. +You can `curl | sh` it using the following one-liner. + ```shell script curl --proto '=https' --tlsv1.2 -sSf 'https://raw.githubusercontent.com/ldericher/kiwi-config/master/install.sh' | sh ``` -That script checks for the basic dependencies of the `kiwi` command, then downloads the main script and installs it to a location of your choice. Please consider installing `kiwi` into a directory inside your $PATH. +The installer downloads the `kiwi` launcher script and installs it to a location of your choice. +Please consider installing into a directory inside your `$PATH`. +Run in a root shell or use `sudo sh` instead for system-wide installation. + +You should now be able to run `kiwi init --show` and see the default configuration file. +This downloads the latest version of the main kiwi-config executable and sets it up for you. + + +### Adjusting environment for `kiwi` + +`kiwi-config` depends on Python 3.6 (or later), [pipenv](https://pipenv.pypa.io/), and +[less](http://www.greenwoodsoftware.com/less/) being in your `$PATH`. + +In some cases, notably when using a multi-version system such as +[CentOS SCL](https://wiki.centos.org/AdditionalResources/Repositories/SCL), not all of these are in your `$PATH` +at login time. + +In those cases, you can simply create a `.kiwienv` file in your home directory. +It will be sourced every time you use the `kiwi` command. +For the aforementioned case where you installed `centos-release-scl` and `rh-python36`, your `~/.kiwienv` should +contain: + +```shell script +#!/bin/sh + +. /opt/rh/rh-python36/enable +``` + ## Get started -TODO +### Create a kiwi-config instance + +Any directory is implicitly a valid `kiwi-config` instance using the default configuration. +To prevent surprises however, you should run `kiwi init` in an empty directory and follow its directions before +actually using `kiwi` more. + + +### Concept + +A `kiwi-config` instance is a directory containing a bunch of static configuration files. +"Static" there as in "those don't change during normal service operation". +These files could be anything from actual `.conf` files to entire html-web-roots. + +Non-static, but persistent files are to be kept in a "service data directory", by default `/var/kiwi`. +In your `docker-compose.yml` files, you can refer to that directory as **${TARGETROOT}**. + +Start the current directory as a `kiwi-config` instance using `kiwi up`, or stop it using `kiwi down`. +This also creates kiwi's internal hub network, which you can use as **kiwi_hub** in your `docker-compose.yml` files. + + +### Projects + +A `kiwi-config` instance usually contains several projects. +A project is a collection of dependent or at least logically connected services, described by a `docker-compose.yml`. +A well-known example would be webserver + php + database. + +To create a project, use the `kiwi new ` command. +By default, this creates a new disabled project. +Before enabling or starting, consider editing the new project's `docker-compose.yml` file to your liking. +Finally, enable it with `kiwi enable `. +You can also create, enable or (analogously) disable multiple projects in a single command. + +Each project will have its own place in the service data directory, which you can refer to as **${TARGETDIR}**. + +Finally, start a project using `kiwi up `. + + +### Advanced kiwi-config + +`kiwi-config` extends the logical bounds of `docker-compose` to handling multiple projects. + + +#### The `kiwi_hub` + +With kiwi-config, you get the internal kiwi_hub network for free. +It allows for network communication between services in different projects. +Be aware, services only connected to the kiwi_hub can't use a port mapping! +In most cases, you will want to use this: + +```yaml +networks: + - default + - kiwi_hub +``` + + +#### The `CONFDIR` + +Sometimes, it's convenient to re-use configuration files across projects. +For this use case, create a directory named `conf` in a project. +Those will all be combined into a directory available as **${CONFDIR}** in your `docker-compose.yml` files. + +#### For everything else, look at `kiwi --help` +#### Happy admin-ing! diff --git a/example/kiwi.yml b/example/kiwi.yml index f62e844..1c2d4dc 100644 --- a/example/kiwi.yml +++ b/example/kiwi.yml @@ -2,7 +2,7 @@ # kiwi-config instance configuration # ###################################### -version: '0.1.0' +version: '0.1.1' runtime: storage: /tmp/kiwi diff --git a/install.sh b/install.sh index 52fdf5e..715f156 100755 --- a/install.sh +++ b/install.sh @@ -4,55 +4,39 @@ # CONSTANTS # ############# -# dependencies to run kiwi-config -KIWI_DEPS="bash python3 pipenv less" -# default install directory +# default installation directory INSTALL_DIR_DEFAULT="/usr/local/bin" -########## -# CHECKS # -########## -printf "checking dependencies ... " +############ +# CLI ARGS # +############ -for dep in ${KIWI_DEPS}; do - printf "%s, " "${dep}" +# installation directory +install_dir="${1}" +# adjust default if given +INSTALL_DIR_DEFAULT="${1:-${INSTALL_DIR_DEFAULT}}" - if ! command -v "${dep}" >/dev/null 2>/dev/null; then - echo - echo "Dependency '${dep}' not found, please install!" >/dev/stderr - exit 1 - fi -done - -echo "OK" ######## # MAIN # ######## # prompt for installation directory -valid="no" - -while [ "${valid}" = "no" ]; do +while [ ! -d "${install_dir}" ]; do printf "Select installation directory [Default: '%s']: " "${INSTALL_DIR_DEFAULT}" - read install_dir /dev/stderr exit 1 fi @@ -60,6 +44,11 @@ while [ "${valid}" = "no" ]; do fi done +if [ ! -d "${install_dir}" ]; then + echo "wtf?" + exit 1 +fi + # start actual installation printf "Installing into '%s' ... " "${install_dir}" diff --git a/kiwi b/kiwi index f8b5bd5..dd34869 100755 --- a/kiwi +++ b/kiwi @@ -1,4 +1,4 @@ -#!/bin/bash +#!/bin/sh ############# # CONSTANTS # @@ -10,7 +10,7 @@ KIWI_CONF_NAME="kiwi.yml" KIWI_VERSION_TAG="etc/version_tag" # dependencies to run kiwi-config -KIWI_DEPS=(docker docker-compose) +KIWI_DEPS="python3 pipenv less docker docker-compose" # base install dir KIWI_BASEDIR="${HOME}/.local/lib/kiwi-config" # per-user env setup script @@ -21,28 +21,30 @@ KIWI_REPO="https://github.com/ldericher/kiwi-config" # use latest version by default KIWI_VERSION="master" + ################### # DYNAMIC STRINGS # ################### # directory of correct installation -function kiwi_installdir() { +kiwi_installdir() { echo "${KIWI_BASEDIR}/${KIWI_VERSION}" } # src directory in installed version -function kiwi_root() { +kiwi_root() { echo "$(kiwi_installdir)/src" } # main script in installed version -function kiwi_executable() { +kiwi_executable() { echo "$(kiwi_root)/kiwi-config.py" } # cache current work directory WORKDIR="$(pwd)" + ################## # PER-USER SETUP # ################## @@ -50,20 +52,22 @@ WORKDIR="$(pwd)" # add in environment setup if [ -f "${KIWI_ENVFILE}" ]; then # shellcheck source=$HOME/.kiwienv - source "${KIWI_ENVFILE}" + . "${KIWI_ENVFILE}" fi + ########## # CHECKS # ########## -for dep in "${KIWI_DEPS[@]}"; do - if ! command -v "${dep}" &>/dev/null; then +for dep in ${KIWI_DEPS}; do + if ! command -v "${dep}" >/dev/null 2>/dev/null; then echo "Dependency '${dep}' not found, please install!" >/dev/stderr exit 1 fi done + ######## # MAIN # ######## @@ -77,7 +81,7 @@ fi # install if kiwi-config not found if [ ! -x "$(kiwi_executable)" ]; then - echo -n "Installing kiwi-config v${KIWI_VERSION} into ${KIWI_BASEDIR} ... " + printf "Installing kiwi-config v%s into %s ... " "${KIWI_VERSION}" "${KIWI_BASEDIR}" # switch to temp dir tmpdir=$(mktemp -d) @@ -92,7 +96,7 @@ if [ ! -x "$(kiwi_executable)" ]; then if [ -x "$(kiwi_executable)" ]; then # after version update: no need to install - echo "kiwi-config v${KIWI_VERSION} found!" + echo "v${KIWI_VERSION} already installed!" else # install archive @@ -106,15 +110,6 @@ if [ ! -x "$(kiwi_executable)" ]; then rm -rf "${tmpdir}" fi -# check virtualenv -cd "$(kiwi_installdir)" || : -if ! pipenv --venv &>/dev/null; then - # install virtualenv - echo -n "Preparing virtualenv ... " - pipenv sync &>/dev/null - echo "OK" -fi - # go back to the original work directory cd "${WORKDIR}" || : @@ -123,10 +118,19 @@ KIWI_ROOT="$(kiwi_root)" PIPENV_VERBOSITY=-1 PIPENV_PIPFILE="$(kiwi_installdir)/Pipfile" -export KIWI_ROOT export KIWI_CONF_NAME +export KIWI_ROOT export PIPENV_VERBOSITY export PIPENV_PIPFILE +# check virtualenv +cd "$(kiwi_installdir)" || : +if ! pipenv --venv >/dev/null 2>/dev/null; then + # install virtualenv + printf "Preparing virtualenv ... " + pipenv sync >/dev/null 2>/dev/null + echo "OK" +fi + # run main script exec pipenv run "$(kiwi_executable)" "${@}" diff --git a/src/etc/command_help.txt b/src/etc/command_help.txt index 51ae022..30381fd 100644 --- a/src/etc/command_help.txt +++ b/src/etc/command_help.txt @@ -2,12 +2,11 @@ Commands for Operation: up Bring up the whole instance, a project or service(s) inside a project down Bring down the whole instance, a project or service(s) inside a project update Update the whole instance, a project or service(s) inside a project - clean Cleanly sync all configs to target folder, then relaunch affected projects - purge Remove all running docker artifacts of this instance + restart Restart the whole instance, a project or service(s) inside a project Commands for Instance Management: - config Create or configure kiwi-config instance - inspect Inspect projects in this instance, services inside a project or service(s) inside a project + init Initialize or reconfigure kiwi-config instance + show Show projects in this instance, services inside a project or service(s) inside a project cmd Run raw docker-compose command in a project Commands for Project and Service Management: diff --git a/src/etc/kiwi_help.txt b/src/etc/kiwi_help.txt index 60f6ba2..97b462c 100644 --- a/src/etc/kiwi_help.txt +++ b/src/etc/kiwi_help.txt @@ -1,4 +1,4 @@ -kiwi-config is the tool for container server management. +kiwi-config is the simple tool for managing container servers. Features: - Group services into projects using their own docker-compose.yml diff --git a/src/etc/version_tag b/src/etc/version_tag index 6c6aa7c..6da28dd 100644 --- a/src/etc/version_tag +++ b/src/etc/version_tag @@ -1 +1 @@ -0.1.0 \ No newline at end of file +0.1.1 \ No newline at end of file diff --git a/src/kiwi/misc.py b/src/kiwi/misc.py index 90753ab..221b9c3 100644 --- a/src/kiwi/misc.py +++ b/src/kiwi/misc.py @@ -21,7 +21,7 @@ def are_you_sure(prompt, default="no"): suffix = "[yes|NO]" answer = input( - f"{_surround('MUST HAVE CAREFULING IN PROGRESS', '!')}\n" + f"{_surround('MUST HAVE CAREFULING IN PROCESS', '!')}\n" f"\n" f"{_emphasize(prompt)}\n" f"\n" diff --git a/src/kiwi/project.py b/src/kiwi/project.py index 5072aa8..7a11fc0 100644 --- a/src/kiwi/project.py +++ b/src/kiwi/project.py @@ -84,6 +84,7 @@ class Project: kwargs['env'].update({ 'COMPOSE_PROJECT_NAME': self.get_name(), 'KIWI_HUB_NAME': config['network:name'], + 'TARGETROOT': config['runtime:storage'], 'CONFDIR': os.path.join(config['runtime:storage'], CONF_DIRECTORY_NAME), 'TARGETDIR': self.target_dir_name() }) diff --git a/src/kiwi/subcommands/__init__.py b/src/kiwi/subcommands/__init__.py index 76ff7ab..f958982 100644 --- a/src/kiwi/subcommands/__init__.py +++ b/src/kiwi/subcommands/__init__.py @@ -1,42 +1,39 @@ # local -from ._hidden import ConfCopyCommand, ConfPurgeCommand, NetUpCommand +from ._hidden import ConfCopyCommand, NetUpCommand from .build import BuildCommand -from .clean import CleanCommand from .cmd import CmdCommand -from .config import ConfigCommand from .disable import DisableCommand from .down import DownCommand from .enable import EnableCommand -from .inspect import InspectCommand +from .init import InitCommand from .logs import LogsCommand from .new import NewCommand from .pull import PullCommand -from .purge import PurgeCommand from .push import PushCommand +from .restart import RestartCommand from .shell import ShellCommand +from .show import ShowCommand from .up import UpCommand from .update import UpdateCommand __all__ = [ 'ConfCopyCommand', - 'ConfPurgeCommand', 'NetUpCommand', 'BuildCommand', - 'CleanCommand', 'CmdCommand', - 'ConfigCommand', 'DisableCommand', 'DownCommand', 'EnableCommand', - 'InspectCommand', + 'InitCommand', 'LogsCommand', 'NewCommand', 'PullCommand', - 'PurgeCommand', 'PushCommand', + 'RestartCommand', 'ShellCommand', + 'ShowCommand', 'UpCommand', 'UpdateCommand', ] diff --git a/src/kiwi/subcommands/_hidden.py b/src/kiwi/subcommands/_hidden.py index bd7bbc8..28ce50d 100644 --- a/src/kiwi/subcommands/_hidden.py +++ b/src/kiwi/subcommands/_hidden.py @@ -25,6 +25,7 @@ class ConfCopyCommand(SubCommand): conf_dirs = [ project.conf_dir_name() for project in Projects.from_dir().filter_enabled() + if project.has_configs() ] if conf_dirs: @@ -33,33 +34,12 @@ class ConfCopyCommand(SubCommand): logging.info(f"Sync directories: {conf_dirs}") Rootkit('rsync').run([ - 'rsync', '-r', *prefix_path_mnt(conf_dirs) + 'rsync', '-rpt', '--delete', *prefix_path_mnt(conf_dirs) ], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) return True -class ConfPurgeCommand(SubCommand): - """kiwi conf-purge""" - - def __init__(self): - super().__init__( - 'conf-purge', - action="Removing all configs for", add_parser=False, - description="Remove all config files in target directory" - ) - - def _run_instance(self, runner, args): - conf_target = f"{LoadedConfig.get()['runtime:storage']}/{CONF_DIRECTORY_NAME}" - logging.info(f"Purging directories: {conf_target}") - - Rootkit().run([ - 'rm', '-rf', prefix_path_mnt(conf_target) - ], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) - - return True - - def _find_net(net_name): ps = Executable('docker').run([ 'network', 'ls', '--filter', f"name={net_name}", '--format', '{{.Name}}' diff --git a/src/kiwi/subcommands/clean.py b/src/kiwi/subcommands/clean.py deleted file mode 100644 index 22422e5..0000000 --- a/src/kiwi/subcommands/clean.py +++ /dev/null @@ -1,37 +0,0 @@ -from ..projects import Projects -from ..subcommand import SubCommand - - -class CleanCommand(SubCommand): - """kiwi clean""" - - def __init__(self): - super().__init__( - 'clean', - action="Cleaning all configs for", - description="Cleanly sync all configs to target folder, then relaunch affected projects" - ) - - def _run_instance(self, runner, args): - result = True - - affected_projects = [ - project.get_name() - for project in Projects.from_dir() - if project.has_configs() - ] - - for project_name in affected_projects: - args.projects = project_name - result &= runner.run('down') - - # cleanly sync configs - result &= runner.run('conf-purge') - result &= runner.run('conf-copy') - - # bring projects back up - for project_name in affected_projects: - args.projects = project_name - result &= runner.run('up') - - return result diff --git a/src/kiwi/subcommands/down.py b/src/kiwi/subcommands/down.py index cd53b3e..9c4032f 100644 --- a/src/kiwi/subcommands/down.py +++ b/src/kiwi/subcommands/down.py @@ -1,5 +1,12 @@ +# system +import logging +import subprocess + # local +from ._hidden import _find_net from ..subcommand import ServiceCommand +from ..config import LoadedConfig +from ..executable import Executable from ..misc import are_you_sure @@ -14,13 +21,27 @@ class DownCommand(ServiceCommand): ) def _run_instance(self, runner, args): + net_name = LoadedConfig.get()['network:name'] + if are_you_sure([ "This will bring down the entire instance.", "", "This may not be what you intended, because:", " - Bringing down the instance stops ALL services in here", ]): - return super()._run_instance(runner, args) + if super()._run_instance(runner, args): + # remove the hub network afterwards + if _find_net(net_name): + Executable('docker').run([ + 'network', 'rm', net_name + ], check=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + + logging.info(f"Network '{net_name}' removed") + + else: + logging.info(f"Network '{net_name}' does not exist") + + return True return False diff --git a/src/kiwi/subcommands/config.py b/src/kiwi/subcommands/init.py similarity index 87% rename from src/kiwi/subcommands/config.py rename to src/kiwi/subcommands/init.py index 15f3c9c..acca724 100644 --- a/src/kiwi/subcommands/config.py +++ b/src/kiwi/subcommands/init.py @@ -8,14 +8,14 @@ from ..subcommand import SubCommand from ..config import DefaultConfig, LoadedConfig -class ConfigCommand(SubCommand): - """kiwi config""" +class InitCommand(SubCommand): + """kiwi init""" def __init__(self): super().__init__( - 'config', - action=f"Configuring '{KIWI_CONF_NAME}' in", - description="Create or configure kiwi-config instance" + 'init', + action=f"Initializing '{KIWI_CONF_NAME}' in", + description="Initialize or reconfigure kiwi-config instance" ) # -f switch: Initialize with default config diff --git a/src/kiwi/subcommands/restart.py b/src/kiwi/subcommands/restart.py new file mode 100644 index 0000000..d8856c9 --- /dev/null +++ b/src/kiwi/subcommands/restart.py @@ -0,0 +1,26 @@ +# local +from ..subcommand import ServiceCommand +from ..misc import are_you_sure + + +class RestartCommand(ServiceCommand): + """kiwi restart""" + + def __init__(self): + super().__init__( + 'restart', num_projects='?', num_services='*', + action="Restarting", + description="Restart the whole instance, a project or service(s) inside a project" + ) + + def _run_instance(self, runner, args): + if are_you_sure([ + "This will restart the entire instance." + ]): + return super()._run_instance(runner, args) + + return False + + def _run_services(self, runner, args, project, services): + project.compose_run(['restart', *services]) + return True diff --git a/src/kiwi/subcommands/inspect.py b/src/kiwi/subcommands/show.py similarity index 90% rename from src/kiwi/subcommands/inspect.py rename to src/kiwi/subcommands/show.py index a7c139e..a321a97 100644 --- a/src/kiwi/subcommands/inspect.py +++ b/src/kiwi/subcommands/show.py @@ -24,14 +24,14 @@ def _print_list(strings): _print_list(list(strings)) -class InspectCommand(ServiceCommand): - """kiwi inspect""" +class ShowCommand(ServiceCommand): + """kiwi show""" def __init__(self): super().__init__( - 'inspect', num_projects='?', num_services='*', - action="Inspecting", - description="Inspect projects in this instance, services inside a project or service(s) inside a project" + 'show', num_projects='?', num_services='*', + action="Showing", + description="Show projects in this instance, services inside a project or service(s) inside a project" ) def _run_instance(self, runner, args):