jmm/kiwi-scp
2
0
Fork 0
Code Projects Wiki Activity

Merge branch 'release/0.1.2'

Browse source
This commit is contained in:
Jörn-Michael Miehe 2020-08-26 14:34:58 +02:00
commit 33bcbc82a2
17 changed files with 244 additions and 103 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

0
.idea/kiwi-config.iml → .idea/kiwi-scp.iml
View file

2
.idea/modules.xml
View file

@ -2,7 +2,7 @@
<project version="4">
<component name="ProjectModuleManager">
<modules>
<module fileurl="file://$PROJECT_DIR$/.idea/kiwi-config.iml" filepath="$PROJECT_DIR$/.idea/kiwi-config.iml" />
<module fileurl="file://$PROJECT_DIR$/.idea/kiwi-scp.iml" filepath="$PROJECT_DIR$/.idea/kiwi-scp.iml" />
</modules>
</component>
</project>

73
README.md
View file

@ -1,4 +1,6 @@
# kiwi-config
# kiwi-scp
> `kiwi` - simple, consistent, powerful
The simple tool for managing container servers
@ -6,7 +8,7 @@ The simple tool for managing container servers
## Quick start
- Learn to use `docker` with `docker-compose`
- Install kiwi-config
- Install kiwi-scp
- Look at [the example instance](./example)
- Look at the output of `kiwi --help`
- Start building your own instances
@ -18,7 +20,7 @@ A convenience installer is available as [install.sh](./install.sh) in this direc
You can `curl | sh` it using the following one-liner.
```shell script
curl --proto '=https' --tlsv1.2 -sSf 'https://raw.githubusercontent.com/ldericher/kiwi-config/master/install.sh' | sh
curl --proto '=https' --tlsv1.2 -sSf 'https://raw.githubusercontent.com/ldericher/kiwi-scp/master/install.sh' | sh
```
The installer downloads the `kiwi` launcher script and installs it to a location of your choice.
@ -26,21 +28,21 @@ Please consider installing into a directory inside your `$PATH`.
Run in a root shell or use `sudo sh` instead for system-wide installation.
You should now be able to run `kiwi init --show` and see the default configuration file.
This downloads the latest version of the main kiwi-config executable and sets it up for you.
This downloads the latest version of the main kiwi-scp executable and sets it up for you.
### Adjusting environment for `kiwi`
`kiwi-config` depends on Python 3.6 (or later), [pipenv](https://pipenv.pypa.io/), and
The `kiwi` executable depends on [Python](https://www.python.org/) 3.6 (or later), [pipenv](https://pipenv.pypa.io/), and
[less](http://www.greenwoodsoftware.com/less/) being in your `$PATH`.
In some cases, notably when using a multi-version system such as
[CentOS SCL](https://wiki.centos.org/AdditionalResources/Repositories/SCL), not all of these are in your `$PATH`
at login time.
In those cases, you can simply create a `.kiwienv` file in your home directory.
In those cases, you can simply create a `.kiwi_profile` file in your home directory.
It will be sourced every time you use the `kiwi` command.
For the aforementioned case where you installed `centos-release-scl` and `rh-python36`, your `~/.kiwienv` should
For the aforementioned case where you installed `centos-release-scl` and `rh-python36`, your `~/.kiwi_profile` should
contain:
```shell script
@ -52,29 +54,29 @@ contain:
## Get started
### Create a kiwi-config instance
### Create a kiwi-scp instance
Any directory is implicitly a valid `kiwi-config` instance using the default configuration.
To prevent surprises however, you should run `kiwi init` in an empty directory and follow its directions before
actually using `kiwi` more.
Any directory is implicitly a valid kiwi-scp instance using the default configuration.
To prevent surprises however, you should run `kiwi init` in an empty directory and follow its directions to
create a `kiwi.yml` before using `kiwi` more.
### Concept
A `kiwi-config` instance is a directory containing a bunch of static configuration files.
A kiwi-scp instance is a directory containing a bunch of static configuration files.
"Static" there as in "those don't change during normal service operation".
These files could be anything from actual `.conf` files to entire html-web-roots.
Non-static, but persistent files are to be kept in a "service data directory", by default `/var/kiwi`.
In your `docker-compose.yml` files, you can refer to that directory as **${TARGETROOT}**.
Start the current directory as a `kiwi-config` instance using `kiwi up`, or stop it using `kiwi down`.
Start the current directory as a kiwi-scp instance using `kiwi up`, or stop it using `kiwi down`.
This also creates kiwi's internal hub network, which you can use as **kiwi_hub** in your `docker-compose.yml` files.
### Projects
A `kiwi-config` instance usually contains several projects.
A kiwi-scp instance usually contains several projects.
A project is a collection of dependent or at least logically connected services, described by a `docker-compose.yml`.
A well-known example would be webserver + php + database.
@ -89,14 +91,14 @@ Each project will have its own place in the service data directory, which you ca
Finally, start a project using `kiwi up <project-name>`.
### Advanced kiwi-config
### Advanced kiwi usage
`kiwi-config` extends the logical bounds of `docker-compose` to handling multiple projects.
kiwi-scp extends the logical bounds of `docker-compose` to handling multiple projects.
#### The `kiwi_hub`
With kiwi-config, you get the internal kiwi_hub network for free.
With kiwi-scp, you get the internal kiwi_hub network for free.
It allows for network communication between services in different projects.
Be aware, services only connected to the kiwi_hub can't use a port mapping!
In most cases, you will want to use this:
@ -114,5 +116,40 @@ Sometimes, it's convenient to re-use configuration files across projects.
For this use case, create a directory named `conf` in a project.
Those will all be combined into a directory available as **${CONFDIR}** in your `docker-compose.yml` files.
#### `kiwi.yml` options
##### `version`
Version of kiwi-scp to use for this instance.
Default: Latest version.
##### `runtime:storage`
Path of the service data directory, available as **${TARGETROOT}** in projects.
Default: `/var/kiwi`
##### `runtime:shells`
List of additionally preferable shell executables when entering service containers.
Default: `- /bin/bash`
Example:
```yaml
runtime:
shells:
- /bin/zsh
- /bin/fish
```
##### `runtime:env`
Associative array of custom variables available in projects' `docker-compose.yml` files.
Default: `null`
Example:
```yaml
runtime:
env:
HELLO: "World"
FOO: "Bar"
```
#### For everything else, look at `kiwi --help`
#### Happy admin-ing!
#### Happy kiwi-ing!

2
example/hello-world.project/conf/html/index.html
View file

@ -16,7 +16,7 @@
<body>
<h1>Another Hello World!</h1>
<p>This service uses an off-the-shelf image and the conf-directory feature of kiwi-config.</p>
<p>This service uses an off-the-shelf image and the conf-directory feature of kiwi-scp.</p>
<p><em>Greetings, nginx.</em></p>
</body>

4
example/hello-world.project/web/index.html
View file

@ -15,9 +15,9 @@
<body>
<h1>Hello World!</h1>
<h2>kiwi-config works.</h2>
<h2>kiwi-scp works.</h2>
<p>This is an example service on a custom image assembled by kiwi-config.<br /> But wait, there's more!</p>
<p>This is an example service on a custom image assembled by kiwi-scp.<br /> But wait, there's more!</p>
<a href="//localhost:8081"><h3>Adminer</h3></a>
<p>There's a mySQL database included in this project. Login with user <q>root</q> and password <q>changeme</q>.</p>

8
example/kiwi.yml
View file

@ -1,8 +1,8 @@
######################################
# kiwi-config instance configuration #
######################################
###################################
# kiwi-scp instance configuration #
###################################
version: '0.1.1'
version: '0.1.2'
runtime:
storage: /tmp/kiwi

86
install.sh
View file

@ -6,70 +6,90 @@
# default installation directory
INSTALL_DIR_DEFAULT="/usr/local/bin"
# URI of "kiwi" launcher script
KIWI_URI="https://raw.githubusercontent.com/ldericher/kiwi-scp/master/kiwi"
#############
# FUNCTIONS #
#############
############
# CLI ARGS #
############
# prompt yes/no question (default yes)
yes_no() {
# prompt and read from terminal
printf "%s [Y|n] " "${1}"
read -r answer </dev/tty || answer=""
# installation directory
install_dir="${1}"
# adjust default if given
INSTALL_DIR_DEFAULT="${1:-${INSTALL_DIR_DEFAULT}}"
# check first character
answer="$(printf '%.1s' "${answer}")"
if [ "${answer}" = "N" ] || [ "${answer}" = "n" ]; then
# negative
return 1
else
# positive
return 0
fi
}
# exit with error
die() {
echo "ERROR: ${1}!" >/dev/stderr
exit 1
}
########
# MAIN #
########
# prompt for installation directory
# check if already installed
install_kiwi="$(command -v kiwi)"
if [ -x "${install_kiwi}" ]; then
# kiwi is installed: Choose that directory
install_dir="$(dirname "${install_kiwi}")"
if ! yes_no "kiwi executable found in '${install_dir}'. Overwrite?"; then
die "Uninstall existing '${install_kiwi}' first"
fi
elif [ ${#} -gt 0 ]; then
# install dir candidate given as CLI argument
install_dir="${1}"
shift 1
fi
# check dir given by argument
while [ ! -d "${install_dir}" ]; do
# prompt user for installation directory
printf "Select installation directory [Default: '%s']: " "${INSTALL_DIR_DEFAULT}"
read -r install_dir </dev/tty || install_dir="${INSTALL_DIR_DEFAULT}"
install_dir="${install_dir:-${INSTALL_DIR_DEFAULT}}"
# check if given dir exists
# check dir given on terminal
if [ ! -d "${install_dir}" ]; then
printf "Install directory doesn't exist. Try creating? [Y|n] "
read -r yesno </dev/tty || yesno="yes"
yesno=$(printf '%.1s' "${yesno}")
if [ ! "${yesno}" = "N" ] && [ ! "${yesno}" = "n" ]; then
# fail this script if we can't create the install dir
if ! mkdir -p "${install_dir}"; then
echo "Invalid install directory." >/dev/stderr
exit 1
# fail if install dir can't be created
if yes_no "Install directory doesn't exist. Try creating?"; then
if ! mkdir -p "${install_dir}" >/dev/null 2>/dev/null; then
die "Couldn't create install directory"
fi
fi
fi
done
if [ ! -d "${install_dir}" ]; then
echo "wtf?"
exit 1
fi
# start actual installation
printf "Installing into '%s' ... " "${install_dir}"
# install "kiwi" script
uri="https://raw.githubusercontent.com/ldericher/kiwi-config/master/kiwi"
tmp_file="$(mktemp)"
if ! curl --proto '=https' --tlsv1.2 -sSf -o "${tmp_file}" "${uri}" >/dev/null 2>/dev/null; then
if ! curl --proto '=https' --tlsv1.2 -sSf -o "${tmp_file}" "${KIWI_URI}" >/dev/null 2>/dev/null; then
rm "${tmp_file}"
echo "Download 'kiwi' failed!" >/dev/stderr
exit 1
die "Downloading 'kiwi' failed"
fi
if ! install -m 0755 "${tmp_file}" "${install_dir}/kiwi" >/dev/null 2>/dev/null; then
rm "${tmp_file}"
echo "Install 'kiwi' failed!" >/dev/stderr
exit 1
die "Installing 'kiwi' failed"
fi
rm "${tmp_file}"
# finalization
rm "${tmp_file}"
echo "OK"
exit 0

136
kiwi
View file

@ -9,89 +9,173 @@ KIWI_CONF_NAME="kiwi.yml"
# version tag filename
KIWI_VERSION_TAG="etc/version_tag"
# dependencies to run kiwi-config
KIWI_DEPS="python3 pipenv less docker docker-compose"
# dependencies to run kiwi-scp
KIWI_DEPENDENCIES="python3 pipenv less docker docker-compose"
# base install dir
KIWI_BASEDIR="${HOME}/.local/lib/kiwi-config"
KIWI_BASEDIR="${HOME}/.local/lib/kiwi-scp"
# per-user env setup script
KIWI_ENVFILE="${HOME}/.kiwienv"
KIWI_PROFILE="${HOME}/.kiwi_profile"
# repository uri
KIWI_REPO="https://github.com/ldericher/kiwi-config"
KIWI_REPO="https://github.com/ldericher/kiwi-scp"
# use latest version by default
KIWI_VERSION="master"
# URI of "kiwi" launcher script
KIWI_URI="https://raw.githubusercontent.com/ldericher/kiwi-scp/master/kiwi"
INSTALLER_URI="https://raw.githubusercontent.com/ldericher/kiwi-scp/master/install.sh"
# canary file: limit curl requests
CANARY_FILENAME="/var/lock/kiwi-scp.canary"
CANARY_MAX_AGE=600
###################
# DYNAMIC STRINGS #
###################
# uri of correct kiwi-scp archive
kiwi_archive_uri() {
echo "${KIWI_REPO}/archive/${KIWI_VERSION}.tar.gz"
}
# directory of correct installation
kiwi_installdir() {
kiwi_install_dir() {
echo "${KIWI_BASEDIR}/${KIWI_VERSION}"
}
# src directory in installed version
kiwi_root() {
echo "$(kiwi_installdir)/src"
echo "$(kiwi_install_dir)/src"
}
# main script in installed version
kiwi_executable() {
echo "$(kiwi_root)/kiwi-config.py"
echo "$(kiwi_root)/kiwi-scp.py"
}
# cache current work directory
WORKDIR="$(pwd)"
#############
# FUNCTIONS #
#############
# prompt yes/no question (default yes)
yes_no() {
# prompt and read from terminal
printf "%s [Y|n] " "${1}"
read -r answer </dev/tty || answer=""
# check first character
answer="$(printf '%.1s' "${answer}")"
if [ "${answer}" = "N" ] || [ "${answer}" = "n" ]; then
# negative
return 1
else
# positive
return 0
fi
}
##################
# PER-USER SETUP #
##################
# add in environment setup
if [ -f "${KIWI_ENVFILE}" ]; then
# shellcheck source=$HOME/.kiwienv
. "${KIWI_ENVFILE}"
if [ -f "${KIWI_PROFILE}" ]; then
# shellcheck source=$HOME/.kiwi_profile
. "${KIWI_PROFILE}"
fi
##########
# CHECKS #
##########
for dep in ${KIWI_DEPS}; do
for dep in ${KIWI_DEPENDENCIES}; do
if ! command -v "${dep}" >/dev/null 2>/dev/null; then
echo "Dependency '${dep}' not found, please install!" >/dev/stderr
exit 1
fi
done
########
# MAIN #
########
# check if we should check for new kiwi version
if [ -f "${CANARY_FILENAME}" ]; then
# check canary age
current_time="$(date '+%s')"
canary_mod_time="$(date -r "${CANARY_FILENAME}" '+%s')"
canary_age="$((current_time - canary_mod_time))"
if [ ${canary_age} -gt ${CANARY_MAX_AGE} ]; then
# canary file too old!
run_kiwi_check="yes"
fi
else
# no canary file!
run_kiwi_check="yes"
fi
# run check for new kiwi version
if [ "${run_kiwi_check}" = "yes" ]; then
# hash this script and the master version
hash_local="$(md5sum <"$(readlink "${0}")")"
hash_remote="$(curl --proto '=https' --tlsv1.2 -sSfL "${KIWI_URI}" | md5sum)"
# warn if different
if [ "${hash_local}" != "${hash_remote}" ]; then
if yes_no "Your kiwi launcher is outdated. Update now?" >/dev/stderr; then
# should reinstall, so download installer
installer="$(curl --proto '=https' --tlsv1.2 -sSfL "${INSTALLER_URI}")"
if yes_no "Use sudo to run as root?"; then
# enable system-wide install
echo "${installer}" | sudo sh
else
# per-user install
echo "${installer}" | sh
fi
else
echo "" >/dev/stderr
echo "####################" >/dev/stderr
echo "Please manually update your kiwi launcher by re-running the installation process:" >/dev/stderr
echo "https://github.com/ldericher/kiwi-scp/#installation" >/dev/stderr
echo "####################" >/dev/stderr
echo "" >/dev/stderr
fi
fi
# refresh canary
touch "${CANARY_FILENAME}"
fi
# check if pwd is a kiwi folder
if [ -f "./${KIWI_CONF_NAME}" ]; then
# determine needed kiwi-config version
# determine needed kiwi-scp version
re_version_line='version\s*:\s*'
eval "$(grep -E "${re_version_line}" "./${KIWI_CONF_NAME}" | sed -r "s/${re_version_line}/KIWI_VERSION=/")"
fi
# install if kiwi-config not found
# install if kiwi-scp not found
if [ ! -x "$(kiwi_executable)" ]; then
printf "Installing kiwi-config v%s into %s ... " "${KIWI_VERSION}" "${KIWI_BASEDIR}"
printf "Installing kiwi-scp v%s into %s ... " "${KIWI_VERSION}" "${KIWI_BASEDIR}"
# switch to temp dir
tmpdir=$(mktemp -d)
cd "${tmpdir}" || :
tmp_dir=$(mktemp -d)
cd "${tmp_dir}" || :
# download archive
curl --proto '=https' --tlsv1.2 -sSfL "${KIWI_REPO}/archive/${KIWI_VERSION}.tar.gz" | tar xz
curl --proto '=https' --tlsv1.2 -sSfL "$(kiwi_archive_uri)" | tar xz
# read archive version tag
cd "kiwi-config-${KIWI_VERSION}" || :
cd "kiwi-scp-${KIWI_VERSION}" || :
KIWI_VERSION=$(cat "./src/${KIWI_VERSION_TAG}")
if [ -x "$(kiwi_executable)" ]; then
@ -100,14 +184,14 @@ if [ ! -x "$(kiwi_executable)" ]; then
else
# install archive
mkdir -p "$(kiwi_installdir)"
mv "src" "Pipfile" "Pipfile.lock" "$(kiwi_installdir)/"
mkdir -p "$(kiwi_install_dir)"
mv "src" "Pipfile" "Pipfile.lock" "$(kiwi_install_dir)/"
echo "OK"
fi
# discard temp dir
cd "${WORKDIR}" || :
rm -rf "${tmpdir}"
rm -rf "${tmp_dir}"
fi
# go back to the original work directory
@ -116,7 +200,7 @@ cd "${WORKDIR}" || :
# setup main environment
KIWI_ROOT="$(kiwi_root)"
PIPENV_VERBOSITY=-1
PIPENV_PIPFILE="$(kiwi_installdir)/Pipfile"
PIPENV_PIPFILE="$(kiwi_install_dir)/Pipfile"
export KIWI_CONF_NAME
export KIWI_ROOT
@ -124,7 +208,7 @@ export PIPENV_VERBOSITY
export PIPENV_PIPFILE
# check virtualenv
cd "$(kiwi_installdir)" || :
cd "$(kiwi_install_dir)" || :
if ! pipenv --venv >/dev/null 2>/dev/null; then
# install virtualenv
printf "Preparing virtualenv ... "

2
src/etc/command_help.txt
View file

@ -5,7 +5,7 @@ Commands for Operation:
restart Restart the whole instance, a project or service(s) inside a project
Commands for Instance Management:
init Initialize or reconfigure kiwi-config instance
init Initialize or reconfigure kiwi-scp instance
show Show projects in this instance, services inside a project or service(s) inside a project
cmd Run raw docker-compose command in a project

6
src/etc/kiwi_header.yml
View file

@ -1,3 +1,3 @@
######################################
# kiwi-config instance configuration #
######################################
###################################
# kiwi-scp instance configuration #
###################################

2
src/etc/kiwi_help.txt
View file

@ -1,4 +1,4 @@
kiwi-config is the simple tool for managing container servers.
kiwi is the simple tool for managing container servers.
Features:
- Group services into projects using their own docker-compose.yml

2
src/etc/version_tag
View file

@ -1 +1 @@
0.1.1
0.1.2

6
src/kiwi/_constants.py
View file

@ -7,14 +7,14 @@ import os
# location of "src" directory to use
KIWI_ROOT = os.getenv('KIWI_ROOT', ".")
# default name of kiwi-config file
# default name of kiwi-scp file
KIWI_CONF_NAME = os.getenv('KIWI_CONF_NAME', "kiwi.yml")
############
# FILE NAMES
# text files inside kiwi-config "src" directory
# text files inside kiwi-scp "src" directory
HEADER_KIWI_CONF_NAME = f"{KIWI_ROOT}/etc/kiwi_header.yml"
DEFAULT_KIWI_CONF_NAME = f"{KIWI_ROOT}/etc/kiwi_default.yml"
VERSION_TAG_NAME = f"{KIWI_ROOT}/etc/version_tag"
@ -32,5 +32,5 @@ IMAGES_DIRECTORY_NAME = f"{KIWI_ROOT}/images"
# DOCKER IMAGE NAMES
# name for auxiliary docker images
LOCAL_IMAGES_NAME = 'localhost/kiwi-config/auxiliary'
LOCAL_IMAGES_NAME = 'localhost/kiwi-scp/auxiliary'
DEFAULT_IMAGE_NAME = 'alpine:latest'

2
src/kiwi/config.py
View file

@ -14,7 +14,7 @@ class Config:
__yml_content = {}
__keys = {
'version': "kiwi-config version to use in this instance",
'version': "kiwi-scp version to use in this instance",
'runtime:storage': "local directory for service data",
'runtime:shells': "shell preference for working in service containers",

2
src/kiwi/subcommand.py
View file

@ -37,7 +37,7 @@ class SubCommand:
"""actually run command with parsed CLI args"""
# run for entire instance
logging.info(f"{self._action} kiwi-config instance at '{os.getcwd()}'")
logging.info(f"{self._action} kiwi-scp instance at '{os.getcwd()}'")
return self._run_instance(runner, args)

2
src/kiwi/subcommands/init.py
View file

@ -15,7 +15,7 @@ class InitCommand(SubCommand):
super().__init__(
'init',
action=f"Initializing '{KIWI_CONF_NAME}' in",
description="Initialize or reconfigure kiwi-config instance"
description="Initialize or reconfigure kiwi-scp instance"
)
# -f switch: Initialize with default config

2
src/kiwi/subcommands/show.py
View file

@ -35,7 +35,7 @@ class ShowCommand(ServiceCommand):
)
def _run_instance(self, runner, args):
print(f"kiwi-config instance at '{os.getcwd()}'")
print(f"kiwi-scp instance at '{os.getcwd()}'")
print("#########")
projects = Projects.from_dir()