Compare commits

...

4 commits

Author SHA1 Message Date
a93f56ee65 README overhaul 2023-11-15 11:27:09 +01:00
72ae9e222c compose file 2023-11-15 11:19:12 +01:00
9faaee714a add "mini-tiangolo" to Dockerfile
- tiangolo/uvicorn-gunicorn is not available for ARM arch
2023-11-15 09:26:18 +01:00
ff4e35e2d4 add testing for core.settings.Settings 2023-11-15 09:23:50 +01:00
9 changed files with 504 additions and 75 deletions

View file

@ -17,12 +17,31 @@ RUN yarn install --production false
COPY ui ./
RUN yarn build --dest /tmp/ovdashboard_ui/html
######################
# python preparation #
######################
ARG PYTHON_VERSION
FROM python:${PYTHON_VERSION} as uvicorn-gunicorn
# where credit is due ...
LABEL maintainer="Sebastian Ramirez <tiangolo@gmail.com>"
WORKDIR /usr/local/share/uvicorn-gunicorn
# install uvicorn-gunicorn
COPY "./install/mini-tiangolo/" "."
RUN set -ex; \
chmod +x start.sh; \
python3 -m pip --no-cache-dir install gunicorn;
CMD ["/usr/local/share/uvicorn-gunicorn/start.sh"]
###########
# web app #
###########
ARG PYTHON_VERSION
FROM tiangolo/uvicorn-gunicorn:python${PYTHON_VERSION} AS production
FROM uvicorn-gunicorn AS production
# env setup
WORKDIR /usr/local/src/ovdashboard_api
@ -40,11 +59,8 @@ RUN set -ex; \
libmagic1 \
; rm -rf /var/lib/apt/lists/*; \
\
# remove example app
rm -rf /app; \
\
# install ovdashboard_api
python -m pip --no-cache-dir install ./
python3 -m pip --no-cache-dir install ./
# add prepared ovdashboard_ui
COPY --from=build-ui /tmp/ovdashboard_ui /usr/local/share/ovdashboard_ui

View file

@ -36,19 +36,53 @@ The Dashboard UI is created using [Vue](https://vuejs.org/) and the [Vuetify](ht
## Quick Start
### Prerequisites
Make sure you have a WebDAV and CalDAV account available.
For an all-in-one solution, consider setting up an account on a [Nextcloud](https://nextcloud.com/) instance! <br />
On your WebvDAV account, create a resource (directory) named `ovdashboard`.
On your WebDAV account, create a resource (directory) named `ovdashboard`[^1].
The intended installation method is as follows:
1. Have a Raspberry Pi (3 or later) connected to a HDMI screen
1. Boot up "Raspberry Pi OS" (64 bit) and connect the Pi to your local network
1. Download (and review) the [OVDashboard installer](TODO), then run it from a terminal
> If you feel adventurous and want to skip the script review, run `curl --proto '=https' --tlsv1.2 -sSf 'TODO' | sh` instead.
1. Reboot the Raspberry Pi
Your target device should be a Raspberry Pi Model 3 or later[^2]. You will need some accessories:
- microSD card, class 10 or UHS (min. 8 GB)
- network connectivity (bring WiFi credentials if applicable)
- connection to a HDMI screen
For a better understanding of your newly created OVDashboard, refer to the [about section](TODO).
It is also heavily advisable that you log into your device using SSH, so you should get another device (PC, tablet or smartphone) onto the same network as your OVDashboard.
[^1]: if named differently, you will need to adjust your compose file later on
[^2]: other devices will also work, but might require extra installation steps
### Install Base System
`OVDashboard` is designed to run on a `DietPi` installation. Full installation documentation is available [at dietpi.com](https://dietpi.com/docs/install/). To quickly get up and running:
1. Download Image from [dietpi.com/#download](https://dietpi.com/#download)
1. Uncompress Image and flash onto SD card you might need "7zip", "balenaEtcher" and/or other tools
1. Check the SD card, open "dietpi.txt" and change some options (full documentation [here](https://dietpi.com/docs/usage/#options-within-the-file)):
- For WiFi, use `AUTO_SETUP_NET_WIFI_ENABLED=1` and `AUTO_SETUP_NET_WIFI_COUNTRY_CODE=DE`, and also check "dietpi-wifi.txt"
- System options, e.g. `AUTO_SETUP_AUTOMATED=1`, `AUTO_SETUP_NET_HOSTNAME=ovdashboard`, `AUTO_SETUP_GLOBAL_PASSWORD=dietpi`, `AUTO_SETUP_LOCALE=de_DE.UTF-8`, `AUTO_SETUP_KEYBOARD_LAYOUT=de`, `AUTO_SETUP_TIMEZONE=Europe/Berlin`, `CONFIG_SERIAL_CONSOLE_ENABLE=0`
1. Be sure to at least change the password (and remember it 🙂️), then put the SD card into your device and boot it. Let the first time setup finish, it will take a bit. It will let you know when it is done!
1. Log into your device using SSH. By default, that's user name `root` and password `dietpi`
### Install OVDashboard
Download (and review) the [OVDashboard install script](//code.yavook.de/OEKZident.de/ovdashboard/raw/branch/master/install/install.sh), then run it from a terminal.
This can all be done after logging into your prepared device:
- The safe way:
1. `mkdir /tmp/ovdashboard && cd /tmp/ovdashboard`
1. `wget 'https://code.yavook.de/OEKZident.de/ovdashboard/raw/branch/master/install/install.sh'`
1. `less install.sh` and/or edit with `nano install.sh`
1. `sh install.sh`
- If you feel adventurous and do not want to review the script, just run `sh <( curl --proto '=https' --tlsv1.2 -sSf 'https://code.yavook.de/OEKZident.de/ovdashboard/raw/branch/master/install/install.sh' )`
Afterwards, reboot your device (`reboot` in the terminal). Your OVDashboard should be working now.
For a better understanding of your newly created OVDashboard, refer to the [about section](#about-the-default-ovdashboard-deployment).
## Updates, upgrades
`/opt/ovdashboard`
## Configuration
@ -60,31 +94,17 @@ Refer to the specific README files for [the API](./api/README.md) and [the UI](.
## About the "default" OVDashboard deployment
The default deployment is made up of three parts: The **installer** which runs once, prepares your device for OVDashboard usage and deploys the **client** and the **service**.
The default deployment is made up of three parts: The **installer** runs once, prepares your device for OVDashboard usage and deploys the **server**.
The **client** will connect to and display your OVDashboard.
The **server** will then run continuously and make OVDashboard available as a web application in your network.
The **service** will run continuously and makes sure your OVDashboard is up to date.
Also, a browser is installed to display the OVDashboard using your device.
### The OVDashboard installer will:
- install some general-purpose packages
- make you set a new password, if you haven't yet
- install and set up Docker Engine and Mozilla Firefox
- (optionally) enable SSH and/or VNC access to your device
- create the OVDashboard configuration directory `/usr/local/etc/ovdashboard` on your device
- set up the OVDashboard service
- (optionally) set up your device to auto-run the OVDashboard client
### The OVDashboard service will:
- regularly pull the latest `docker` image from [`TODO`](TODO), containing both the OVDashboard API and UI
- ensure a container of that image is always running, using your OVDashboard configuration directory
### The OVDashboard client will:
- open Mozilla Firefox in full-screen mode and browse to the local OVDashboard UI
- enable auto-hiding your mouse cursor
- install and set up Docker, Docker Compose and Chromium-Browser
- create the OVDashboard Compose project at `/opt/ovdashboard`
- get a `docker` image from [`code.yavook.de`](https://code.yavook.de/OEKZident.de/-/packages/container/ovdashboard) containing the OVDashboard server
- set up your device to auto-run Chromium in "kiosk" mode to display the local OVDashboard
- auto-hide your mouse cursor

View file

@ -18,12 +18,11 @@ class DAVSettings(BaseModel):
Connection to a DAV server.
"""
protocol: str | None = None
host: str | None = None
path: str | None = None
protocol: str = "https"
host: str = "example.com"
username: str | None = None
password: str | None = None
username: str = "ovd_user"
password: str = "password"
cache_ttl: int = 60 * 10
@ -33,21 +32,34 @@ class DAVSettings(BaseModel):
Combined DAV URL.
"""
return f"{self.protocol}://{self.host}{self.path}"
return f"{self.protocol}://{self.host}"
class WebDAVSettings(DAVSettings):
_DAV_DEFAULT = DAVSettings().model_dump()
class CalDAVSettings(DAVSettings):
"""
Connection to a CalDAV server.
"""
path: str = "/remote.php/dav"
@property
def url(self) -> str:
"""
Combined DAV URL.
"""
return f"{super().url}{self.path}"
class WebDAVSettings(CalDAVSettings):
"""
Connection to a WebDAV server.
"""
protocol: str = "https"
host: str = "example.com"
path: str = "/remote.php/dav"
prefix: str = "/ovdashboard"
username: str = "ovd_user"
password: str = "password"
path: str = "/remote.php/webdav"
config_filename: str = "config.txt"
@ -62,7 +74,10 @@ class WebDAVSettings(DAVSettings):
Combined DAV URL.
"""
return f"{self.protocol}://{self.host}{self.path}{self.prefix}"
return f"{super().url}{self.prefix}"
_WEBDAV_DEFAULT = WebDAVSettings().model_dump()
class RedisSettings(BaseModel):
@ -132,7 +147,7 @@ class Settings(BaseSettings):
# caldav settings
#####
caldav: DAVSettings = DAVSettings()
caldav: CalDAVSettings = CalDAVSettings()
#####
# redis settings
@ -141,37 +156,19 @@ class Settings(BaseSettings):
redis: RedisSettings = RedisSettings()
@model_validator(mode="before")
@classmethod
def validate_dav_settings(cls, data) -> dict[str, Any]:
assert isinstance(data, dict)
# ensure both settings dicts are created
for key in ("webdav", "caldav"):
if key not in data:
data[key] = {}
data[key] = data.get(key, {})
default_dav = DAVSettings(
protocol="https",
host="example.com",
username="ovdashboard",
password="secret",
).model_dump()
for key in default_dav:
# if "webdav" value is not specified, use default
if key not in data["webdav"] or data["webdav"][key] is None:
data["webdav"][key] = default_dav[key]
for key in _DAV_DEFAULT:
# if "webdav" value is not specified, use default value
data["webdav"][key] = data["webdav"].get(key, _WEBDAV_DEFAULT[key])
# if "caldav" value is not specified, use "webdav" value
if key not in data["caldav"] or data["caldav"][key] is None:
data["caldav"][key] = data["webdav"][key]
# add default "path"s if None
if data["webdav"]["path"] is None:
data["webdav"]["path"] = "/remote.php/webdav"
if data["caldav"]["path"] is None:
data["caldav"]["path"] = "/remote.php/dav"
data["caldav"][key] = data["caldav"].get(key, data["webdav"][key])
return data

46
api/poetry.lock generated
View file

@ -522,6 +522,17 @@ files = [
{file = "idna-3.4.tar.gz", hash = "sha256:814f528e8dead7d329833b91c5faa87d60bf71824cd12a7530b5526063d02cb4"},
]
[[package]]
name = "iniconfig"
version = "2.0.0"
description = "brain-dead simple config-ini parsing"
optional = false
python-versions = ">=3.7"
files = [
{file = "iniconfig-2.0.0-py3-none-any.whl", hash = "sha256:b6a85871a79d2e3b22d2d1b94ac2824226a63c6b741c88f7ae975f18b6778374"},
{file = "iniconfig-2.0.0.tar.gz", hash = "sha256:2d91e135bf72d31a410b17c16da610a82cb55f6b0477d1a902134b24a455b8b3"},
]
[[package]]
name = "isort"
version = "5.12.0"
@ -787,6 +798,21 @@ files = [
docs = ["furo (>=2023.7.26)", "proselint (>=0.13)", "sphinx (>=7.1.1)", "sphinx-autodoc-typehints (>=1.24)"]
test = ["appdirs (==1.4.4)", "covdefaults (>=2.3)", "pytest (>=7.4)", "pytest-cov (>=4.1)", "pytest-mock (>=3.11.1)"]
[[package]]
name = "pluggy"
version = "1.3.0"
description = "plugin and hook calling mechanisms for python"
optional = false
python-versions = ">=3.8"
files = [
{file = "pluggy-1.3.0-py3-none-any.whl", hash = "sha256:d89c696a773f8bd377d18e5ecda92b7a3793cbe66c87060a6fb58c7b6e1061f7"},
{file = "pluggy-1.3.0.tar.gz", hash = "sha256:cf61ae8f126ac6f7c451172cf30e3e43d3ca77615509771b3a984a0730651e12"},
]
[package.extras]
dev = ["pre-commit", "tox"]
testing = ["pytest", "pytest-benchmark"]
[[package]]
name = "pycodestyle"
version = "2.11.1"
@ -961,6 +987,26 @@ files = [
{file = "pyflakes-3.1.0.tar.gz", hash = "sha256:a0aae034c444db0071aa077972ba4768d40c830d9539fd45bf4cd3f8f6992efc"},
]
[[package]]
name = "pytest"
version = "7.4.3"
description = "pytest: simple powerful testing with Python"
optional = false
python-versions = ">=3.7"
files = [
{file = "pytest-7.4.3-py3-none-any.whl", hash = "sha256:0d009c083ea859a71b76adf7c1d502e4bc170b80a8ef002da5806527b9591fac"},
{file = "pytest-7.4.3.tar.gz", hash = "sha256:d989d136982de4e3b29dabcc838ad581c64e8ed52c11fbe86ddebd9da0818cd5"},
]
[package.dependencies]
colorama = {version = "*", markers = "sys_platform == \"win32\""}
iniconfig = "*"
packaging = "*"
pluggy = ">=0.12,<2.0"
[package.extras]
testing = ["argcomplete", "attrs (>=19.2.0)", "hypothesis (>=3.56)", "mock", "nose", "pygments (>=2.7.2)", "requests", "setuptools", "xmlschema"]
[[package]]
name = "python-dateutil"
version = "2.8.2"

View file

@ -26,6 +26,7 @@ black = "^23.10.1"
flake8 = "^6.1.0"
flake8-isort = "^6.1.0"
types-cachetools = "^5.3.0.6"
pytest = "^7.4.3"
[build-system]
build-backend = "poetry.core.masonry.api"

224
api/test/test_settings.py Normal file
View file

@ -0,0 +1,224 @@
import os
import pytest
from ovdashboard_api.core.settings import Settings
@pytest.fixture(autouse=True, scope="function")
def patch_settings_env_file(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(os, "environ", {})
monkeypatch.delitem(Settings.model_config, "env_file")
def test_empty():
s = Settings.model_validate({})
assert s.log_level == "INFO"
assert s.production_mode is False
assert s.ui_directory == "/usr/local/share/ovdashboard_ui/html"
assert s.ping_host == "1.0.0.0"
assert s.ping_port == 1
assert s.openapi_url == "/api/openapi.json"
assert s.docs_url == "/api/docs"
assert s.redoc_url == "/api/redoc"
# webdav
assert s.webdav.protocol == "https"
assert s.webdav.host == "example.com"
assert s.webdav.username == "ovd_user"
assert s.webdav.password == "password"
assert s.webdav.cache_ttl == 600
assert s.webdav.path == "/remote.php/webdav"
assert s.webdav.config_filename == "config.txt"
assert s.webdav.disable_check is False
assert s.webdav.retries == 20
assert s.webdav.retry_delay == 30
assert s.webdav.prefix == "/ovdashboard"
# caldav
assert s.caldav.protocol == "https"
assert s.caldav.host == "example.com"
assert s.caldav.username == "ovd_user"
assert s.caldav.password == "password"
assert s.caldav.cache_ttl == 600
assert s.caldav.path == "/remote.php/dav"
def test_prod():
s = Settings.model_validate({"production_mode": True})
assert s.log_level == "INFO"
assert s.production_mode is True
assert s.openapi_url is None
assert s.docs_url is None
assert s.redoc_url is None
def test_set_caldav():
s = Settings.model_validate(
{
"caldav": {
"protocol": "cd_protocol",
"host": "cd_host",
"username": "cd_username",
"password": "cd_password",
"cache_ttl": "0",
"path": "cd_path",
}
}
)
# webdav
assert s.webdav.protocol == "https"
assert s.webdav.host == "example.com"
assert s.webdav.username == "ovd_user"
assert s.webdav.password == "password"
assert s.webdav.cache_ttl == 600
assert s.webdav.path == "/remote.php/webdav"
assert s.webdav.config_filename == "config.txt"
assert s.webdav.disable_check is False
assert s.webdav.retries == 20
assert s.webdav.retry_delay == 30
assert s.webdav.prefix == "/ovdashboard"
# caldav
assert s.caldav.protocol == "cd_protocol"
assert s.caldav.host == "cd_host"
assert s.caldav.username == "cd_username"
assert s.caldav.password == "cd_password"
assert s.caldav.cache_ttl == 0
assert s.caldav.path == "cd_path"
def test_set_webdav():
s = Settings.model_validate(
{
"webdav": {
"protocol": "wd_protocol",
"host": "wd_host",
"username": "wd_username",
"password": "wd_password",
"cache_ttl": "99",
"path": "wd_path",
"config_filename": "wd_config_filename",
"disable_check": "true",
"retries": "99",
"retry_delay": "99",
"prefix": "wd_prefix",
}
}
)
# webdav
assert s.webdav.protocol == "wd_protocol"
assert s.webdav.host == "wd_host"
assert s.webdav.username == "wd_username"
assert s.webdav.password == "wd_password"
assert s.webdav.cache_ttl == 99
assert s.webdav.path == "wd_path"
assert s.webdav.config_filename == "wd_config_filename"
assert s.webdav.disable_check is True
assert s.webdav.retries == 99
assert s.webdav.retry_delay == 99
assert s.webdav.prefix == "wd_prefix"
# caldav
assert s.caldav.protocol == "wd_protocol"
assert s.caldav.host == "wd_host"
assert s.caldav.username == "wd_username"
assert s.caldav.password == "wd_password"
assert s.caldav.cache_ttl == 99
assert s.caldav.path == "/remote.php/dav"
def test_set_caldav_webdav():
s = Settings.model_validate(
{
"webdav": {
"protocol": "wd_protocol",
"host": "wd_host",
"username": "wd_username",
"password": "wd_password",
"cache_ttl": "99",
"path": "wd_path",
"config_filename": "wd_config_filename",
"disable_check": "true",
"retries": "99",
"retry_delay": "99",
"prefix": "wd_prefix",
},
"caldav": {
"protocol": "cd_protocol",
"host": "cd_host",
"username": "cd_username",
"password": "cd_password",
"cache_ttl": "0",
"path": "cd_path",
},
}
)
# webdav
assert s.webdav.protocol == "wd_protocol"
assert s.webdav.host == "wd_host"
assert s.webdav.username == "wd_username"
assert s.webdav.password == "wd_password"
assert s.webdav.cache_ttl == 99
assert s.webdav.path == "wd_path"
assert s.webdav.config_filename == "wd_config_filename"
assert s.webdav.disable_check is True
assert s.webdav.retries == 99
assert s.webdav.retry_delay == 99
assert s.webdav.prefix == "wd_prefix"
# caldav
assert s.caldav.protocol == "cd_protocol"
assert s.caldav.host == "cd_host"
assert s.caldav.username == "cd_username"
assert s.caldav.password == "cd_password"
assert s.caldav.cache_ttl == 0
assert s.caldav.path == "cd_path"

View file

@ -0,0 +1,38 @@
services:
# shared cache
redis:
image: redis:7-alpine
restart: always
pull_policy: always
# necessary for "host" network deployment,
# but only listen on localhost
ports:
- "127.0.0.1:6379:6379"
app:
image: code.yavook.de/oekzident.de/ovdashboard:0.1.0
restart: always
pull_policy: always
depends_on:
- redis
# "app" container needs host ip
network_mode: host
user: root
environment:
# necessary for "host" network deployment
PORT: "80"
REDIS__HOST: "localhost"
# >>>>> USER VARIABLES <<<<<
# you will want to adjust these!
TZ: "Europe/Berlin"
WEBDAV__HOST: "example.com"
WEBDAV__PATH: "/remote.php/webdav"
WEBDAV__PREFIX: "/ovdashboard"
WEBDAV__USERNAME: "ovd_user"
WEBDAV__PASSWORD: "password"

View file

@ -0,0 +1,67 @@
import json
import multiprocessing
import os
workers_per_core_str = os.getenv("WORKERS_PER_CORE", "1")
max_workers_str = os.getenv("MAX_WORKERS")
use_max_workers = None
if max_workers_str:
use_max_workers = int(max_workers_str)
web_concurrency_str = os.getenv("WEB_CONCURRENCY", None)
host = os.getenv("HOST", "0.0.0.0")
port = os.getenv("PORT", "80")
bind_env = os.getenv("BIND", None)
use_loglevel = os.getenv("LOG_LEVEL", "info")
if bind_env:
use_bind = bind_env
else:
use_bind = f"{host}:{port}"
cores = multiprocessing.cpu_count()
workers_per_core = float(workers_per_core_str)
default_web_concurrency = workers_per_core * cores
if web_concurrency_str:
web_concurrency = int(web_concurrency_str)
assert web_concurrency > 0
else:
web_concurrency = max(int(default_web_concurrency), 2)
if use_max_workers:
web_concurrency = min(web_concurrency, use_max_workers)
accesslog_var = os.getenv("ACCESS_LOG", "-")
use_accesslog = accesslog_var or None
errorlog_var = os.getenv("ERROR_LOG", "-")
use_errorlog = errorlog_var or None
graceful_timeout_str = os.getenv("GRACEFUL_TIMEOUT", "120")
timeout_str = os.getenv("TIMEOUT", "120")
keepalive_str = os.getenv("KEEP_ALIVE", "5")
# Gunicorn config variables
loglevel = use_loglevel
workers = web_concurrency
bind = use_bind
errorlog = use_errorlog
worker_tmp_dir = "/dev/shm"
accesslog = use_accesslog
graceful_timeout = int(graceful_timeout_str)
timeout = int(timeout_str)
keepalive = int(keepalive_str)
# For debugging and testing
log_data = {
"loglevel": loglevel,
"workers": workers,
"bind": bind,
"graceful_timeout": graceful_timeout,
"timeout": timeout,
"keepalive": keepalive,
"errorlog": errorlog,
"accesslog": accesslog,
# Additional, non-gunicorn variables
"workers_per_core": workers_per_core,
"use_max_workers": use_max_workers,
"host": host,
"port": port,
}
print(json.dumps(log_data))

View file

@ -0,0 +1,20 @@
#!/bin/sh
set -e
MODULE_NAME=${MODULE_NAME:-"app.main"}
VARIABLE_NAME=${VARIABLE_NAME:-"app"}
export APP_MODULE="${APP_MODULE:-"$MODULE_NAME:$VARIABLE_NAME"}"
export GUNICORN_CONF="${GUNICORN_CONF:-"/usr/local/share/uvicorn-gunicorn/gunicorn_conf.py"}"
export WORKER_CLASS="${WORKER_CLASS:-"uvicorn.workers.UvicornWorker"}"
if [ -f "${PRE_START_PATH}" ] ; then
echo "Running script ${PRE_START_PATH}"
# shellcheck disable=SC1090
. "${PRE_START_PATH}"
fi
# Start Gunicorn
exec gunicorn \
-k "${WORKER_CLASS}" \
-c "${GUNICORN_CONF}" \
"${APP_MODULE}"