From: Plom Heller Date: Mon, 28 Sep 2026 19:34:15 +0000 (+0200) Subject: Add borgbackup infrastructure. X-Git-Url: https://plomlompom.com/repos/%7B%7Bdb.prefix%7D%7D/%7B%7Bprefix%7D%7D/balance?a=commitdiff_plain;ds=inline;p=confplom Add borgbackup infrastructure. --- diff --git a/CLAUDE.md b/CLAUDE.md index d7235c9..4f7f06c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -40,7 +40,8 @@ Both tracks converge on the same first-login family — `start_root_t490s.sh` - admin workstation: `install_server.sh` - inside a target system: `start_root_t490s.sh`, `start_root_server.sh`, `start_user.sh`, `update_efi.sh` (local systems only, see below) - - optional, by hand on an already-set-up target: `setup_git_server.sh` + - optional, by hand on an already-set-up target: `setup_git_server.sh`, + `setup_borg_server.sh` - optional, by hand from any machine holding a checkout, targeting a repos server over ssh: `adopt_repo.sh` - `to_install/` — per-target package lists (`t490s`, `server`), read in place @@ -84,6 +85,11 @@ root, so all of it is present on every target. - Make an already-set-up server a git host: `sh scripts/setup_git_server.sh` - Make it the authoritative home of a repo (from any machine holding a checkout of that repo): `sh scripts/adopt_repo.sh ` +- Make an already-set-up server a backup host for desktops' `~/org`: + `sh scripts/setup_borg_server.sh`; then, once, from one desktop: + `orgbackup --init ` +- On a desktop, as the user: `orgbackup` before leaving it, `orgrestore` on + the one switched to (see "`home/user/` only") - First login on a new local system: nothing to type — root's first console login runs `start_root_t490s.sh` via a hook left by `install_debian.sh`. - First login on a new remote server: triggered by `install_server.sh`. @@ -185,6 +191,10 @@ another script; core utilities like `mkdir`/`sed`/`mktemp` aren't listed). script**, because each script's `$HOME` differs: `install_server.sh` reads the *admin's* `~/.ssh/known_hosts`; `start_root_server.sh` reads *root's* `~/.ssh` on the freshly provisioned target. +- `PATH_HOME_USER` (`/home/${USERNAME}`) and `PATH_USER_SSH` (its `.ssh`) — + unlike `PATH_MY_SSH`, always the unprivileged account's, as seen by a + root-run script: `start_root_server.sh` moves root's `~/.ssh` there, + `setup_borg_server.sh` reads its `authorized_keys`. - `FNAME_INITRD`/`FNAME_VMLINUZ`, `FNAME_PROFILE`, `DIRNAME_SSH`, `TO_RBIND`, `PATH_GIT_BASE` (`/srv/git`), `OPTS_SSH_NEW_HOST` (the `ssh`/`scp` option accepting an unseen host key on first contact, so a first-ever connection @@ -285,11 +295,12 @@ time — e.g. no `chrooted_command.sh` while `install_debian.sh` is mid- debootstrap; the second's `close_luksvg` would tear the VG down under the first. Run them serially. -Exempt: `start_root_*.sh`, `start_user.sh`, `setup_git_server.sh` (already -booted into a target, never touch the LUKS+VG), `install_server.sh` and -`adopt_repo.sh` (only talk to a remote server over SSH; either can run -concurrently with the others or with itself against another server), and -`update_efi.sh` (only mounts the plain EFI partition). +Exempt: `start_root_*.sh`, `start_user.sh`, `setup_git_server.sh`, +`setup_borg_server.sh` (already booted into a target, never touch the +LUKS+VG), `install_server.sh` and `adopt_repo.sh` (only talk to a remote +server over SSH; either can run concurrently with the others or with itself +against another server), and `update_efi.sh` (only mounts the plain EFI +partition). ## `setup_luksvg.sh` @@ -677,6 +688,62 @@ target-side script. Both scripts share `PATH_GIT_BASE` and (with `install_server.sh`) `OPTS_SSH_NEW_HOST` — see "Shared library". +## `setup_borg_server.sh` + +Optional, run by hand as root on an already-set-up server, to make it one of +the backup hosts that the desktops' `orgbackup`/`orgrestore` (see +"`home/user/` only") push `~/org` to and restore it from. No arguments; safe +to rerun. + +1. `apt-get -y update` + install `borgbackup`. +2. Unless present, create system account `borg` (`NAME_BORG_ACCOUNT`) with + home `/srv/borg` (`PATH_BORG_BASE`) and shell `/bin/sh` (sshd runs the + forced command below through it, so not `nologin`). A separate account + rather than `${USERNAME}`: the same ssh key also gets a `${USERNAME}` + shell, which this way still can't touch the repo files (borg creates them + under umask 0077) short of `sudo`, which wants `${USERNAME}`'s password. + The repo `/srv/borg/org` (`PATH_BORG_REPO`) isn't created here; a + desktop's first `orgbackup --init` does that. +3. Regenerate `borg`'s `authorized_keys` from `${USERNAME}`'s + (`PATH_USER_SSH`): drop comment and blank lines, prefix each remaining + line with `KEY_OPTIONS`. So every key logging in as `${USERNAME}` may + also reach borg — desktops reuse their ordinary ssh key rather than a + dedicated one, since a dedicated key would sit on the same desktops, + protected no better — and a rerun mirrors additions *and* removals + (hand-edits to `borg`'s file are lost on rerun). A line already carrying + options of its own ends up with two options fields, which sshd rejects as + invalid: that key can't reach borg at all, failing closed rather than + being parsed around. Error if no keys result. `KEY_OPTIONS`: + - `command="borg serve --append-only --restrict-to-repository + /srv/borg/org"` — forced command: whatever the client asks to run is + replaced by this (borg's own request is `borg serve` anyway). + `--restrict-to-repository` allows exactly that path, not even + subdirectories (creating it there is allowed; that's how `--init` + works). `--append-only`: the protocol may add, never delete, so a + stolen key can't destroy backups (tried: a repo `delete` is refused). + The repo's own config keeps `append_only = 0`; the flag lives only here. + - `restrict` — no port/agent/X11 forwarding, no pty, no `~/.ssh/rc`, plus + whatever future OpenSSH versions add. Needed on top of the forced + command, which doesn't stop e.g. an `ssh -N -L` tunnel. + + Since the same key already yields a `${USERNAME}` shell, blocking + shells/forwarding as `borg` gains little by itself; what these options + buy is `--append-only` being meaningful — with a shell as `borg`, the key + could just delete the repo files directly. +4. Print next steps. + +**Pruning** is deliberately absent: `~/org` is small and mostly text, and +with deduplication a repo only grows by new content (watch `borg info`'s +deduplicated size, or `df`). When needed: under `--append-only` a desktop's +`borg prune` frees nothing and `borg compact` is refused, while pruning on the +server itself would need the keyfile and passphrase there. So remove +`--append-only` from `borg`'s `authorized_keys` by hand, run `borg prune` + +`borg compact` from a trusted desktop (borg's docs advise first checking the +transaction log for tampering), then rerun this script to restore the flag. + +The account name and repo path are duplicated as `BORG_ACCOUNT` and +`PATH_REMOTE_REPO` in `home/user/.local/bin/lib/orgsync.py`; keep in sync. + ## `home/` skeletons All symlinked by `link_home` (see "Shared library"). Profile fragments live @@ -775,17 +842,88 @@ in `.profile.d/` as `*.sh`, sourced by the loop line in `~/.profile`. order; a Bluetooth headset, HDMI output or monitor source would silently retarget it). Only the index that lookup returns is passed to `pactl set-{sink,source}-{volume,mute}`. +- `.local/bin/orgbackup`, `.local/bin/orgrestore` — keep `~/org` in step + across desktops by hand-off: `orgbackup` on the desktop left, `orgrestore` + on the one switched to. Borg archives rather than git or a live-sync tool, + since `~/org`'s contents and changes don't follow software-development + patterns and syncing follows machine switches. Every server listed in + `~/.config/orgsync/servers` (untracked; one per line, `#`/blank lines skipped) + holds an independent repo (see "`setup_borg_server.sh`"), each pushed to + separately — never copy a repo between servers, which would duplicate its + ID and encryption nonce state. Shared code in `lib/orgsync.py` (below); + `borgbackup` is in `to_install/t490s`. + - Encryption: `keyfile` mode, not `repokey`, so a server holds nothing + that decrypts — a stolen repo can't even be brute-forced against the + passphrase. Borg 1.x ties a keyfile to one repo's ID (sharing one across + repos would reuse AES-CTR nonces), hence one keyfile per server at + `~/.config/orgsync/keys/` (`BORG_KEY_FILE`), created by `orgbackup + --init ` (refuses to overwrite) and copied by hand to every + other desktop, plus one copy kept off all desktops and servers: losing + every copy makes that server's backups unreadable. Never tracked + (secrets). Borg 2's `repo-create --other-repo` may allow one key for all; + revisit on migrating. The passphrase, the same for all repos, is asked + once per run into `BORG_PASSPHRASE`, unless that or `BORG_PASSCOMMAND` is + already set. + - SSH: `BORG_RSH` (`ssh -o StrictHostKeyChecking=accept-new`) logs in as + `borg` with whatever key `ssh-agent` offers (see `ssh-agent.sh` above). + `accept-new`: a never-contacted server's host key is accepted rather + than prompted for mid-run; a known server's *changed* key is still + refused. + - Each run `borg list --json`s every listed server; one lacking a keyfile + or failing is skipped with a warning, none reachable is an error. Borg + rc 1 (warning, e.g. a file changed while read) counts as success. + Archive names are `-`, timestamp fixed-width + first, so string order is chronological and the newest across servers + is a plain `max`; one push uses one name on all servers. + - Sync state in `~/.local/state/orgsync/sync.json`: the last synced archive + name, and a manifest of `~/org` (`_Manifest`, see its comment: metadata + only, no content reads) to detect local changes since. On it rests a + fast-forward-like guard, both overridable by `--force`: + - `orgbackup` refuses if a reachable server has an archive newer than + the last one synced here (or any, if never synced): that would push + over state never seen here. Otherwise it creates a new archive on + every reachable server — always, even if nothing changed, since + deduplication makes that nearly free — with `--comment + parent=`, read by nothing, just for tracing by hand, and borg's + default compression. The manifest saved is the one taken *before* + `borg create`, so an edit during the run later counts as unsynced. + Error if pushed nowhere; non-zero exit if pushed only partly. + - `orgrestore` refuses if `~/org` exists and differs from the saved + manifest (or was never synced), or if the newest reachable archive is + older than the last synced here (its server probably unreachable). + Otherwise it extracts into `~/.org.restoring`, removes `~/org` and + renames the extraction in: extracting over `~/org` would resurrect + files deleted elsewhere, since `borg extract` never deletes. If the + extraction holds no top-level `org/`, it errors out before touching + `~/org`, leaving the extraction for inspection (why that can happen: + see the comment there). Always extracts, even if already current. + Saves a manifest recomputed *after* extraction. `--force` keeps no + copy of what it overwrites: run without it first, and copy aside by + hand if wanted. + - Known gap: only reachable servers count. If the one holding the newest + archive is down, an older one passes as newest elsewhere; the skip + warnings are the only hint. + - Every local filesystem change is narrated with its path, including + borg's own per-repo data at its default locations: records under + `~/.config/borg/security/` (written on any access, printed once + per server on probing) and caches under `~/.cache/borg/` + (written by `borg create` only, as checked). The repo ID comes from + `borg list --json`, hence `--init` probes right after creating. - `.local/bin/lib/` — shared Python package: `argparsing.py`, `pactl.py`, - `__init__.py`. `ArgParser` wraps `argparse.ArgumentParser`: + `orgsync.py`, `__init__.py`. `ArgParser` wraps `argparse.ArgumentParser`: `add_arg_ranged_int` adds an optional positional integer range-checked `0..max` at parse time (via `ArgumentError`); parsed result via the lazily parsing `args` property; plain `add_arg` passthrough for anything else. `pactl.py`: `pactl` is a thin `subprocess.run(('pactl', ...), check=True)` wrapper; `pactl_info_default_dev` returns `pactl -f json list`'s entry for the current default sink/source (found via `pactl -f json info`'s - `default_{sink,source}_name`). - **Import resolution:** `backlight`/`vol` are reached as symlinks, and Python - resolves the real path before computing `sys.path[0]`, so + `default_{sink,source}_name`). `orgsync.py`: what both `org*` scripts + need (paths, `Server` with its borg invocation/probing/`init`, passphrase, + state and manifest); `run_against_syncerror` turns a `SyncError` into a + plain `error: …` exit, called unconditionally (no `__name__` guard: the + scripts are only ever run, never imported). + **Import resolution:** `backlight`/`vol`/`org*` are reached as symlinks, + and Python resolves the real path before computing `sys.path[0]`, so `from lib.argparsing import ArgParser` loads *this repo's own* `home/user/.local/bin/lib/` (`/data/confplom/...` locally, `/opt/confplom/...` remotely), never the symlinked copy `link_home` places diff --git a/home/user/.local/bin/lib/orgsync.py b/home/user/.local/bin/lib/orgsync.py new file mode 100644 index 0000000..1000f9e --- /dev/null +++ b/home/user/.local/bin/lib/orgsync.py @@ -0,0 +1,228 @@ +'Shared logic of orgbackup, orgrestore: ~/org in borg repos across servers.' +from getpass import getpass +from json import dumps as json_dumps, loads as json_loads +from os import environ +from pathlib import Path +from stat import S_ISDIR, S_ISLNK +from subprocess import CompletedProcess, run as subprocess_run +from sys import exit as sys_exit, stderr +from typing import Callable, Optional + +DIRNAME_ORG = 'org' +PATH_HOME = Path.home() +PATH_ORG = PATH_HOME / DIRNAME_ORG +PATH_CONF = PATH_HOME / '.config' / 'orgsync' +PATH_SERVERS = PATH_CONF / 'servers' +PATH_KEYS = PATH_CONF / 'keys' +PATH_STATE = PATH_HOME / '.local' / 'state' / 'orgsync' / 'sync.json' + +# keep in sync with scripts/setup_borg_server.sh +BORG_ACCOUNT = 'borg' +PATH_REMOTE_REPO = '/srv/borg/org' + +BORG_ENCRYPTION = 'keyfile' +BORG_RCS_OK = (0, 1) # 1: warning, e.g. file changed while being read +BORG_RSH = 'ssh -o StrictHostKeyChecking=accept-new' +# where borg itself keeps local per-repo data (its defaults), to narrate that +PATH_BORG_CACHE = PATH_HOME / '.cache' / 'borg' +PATH_BORG_SECURITY = PATH_HOME / '.config' / 'borg' / 'security' + +# snapshot of PATH_ORG's tree, cheap to compute (no reading of file contents), +# stored on each sync, to detect local changes since by comparing against a +# fresh one; maps each path below PATH_ORG (relative to it) to: +# - directories: [st_mode] +# - symlinks: [st_mode, link target] +# - anything else: [st_mode, st_size, st_mtime_ns] +_Manifest = dict[str, list[int | str]] + + +class SyncError(Exception): + 'Condition to abort on with a plain message rather than a traceback.' + + +def run_against_syncerror( + f: Callable[[], None] + ) -> None: + 'Run f, turning a SyncError into a plain message and sys_exit.' + try: + f() + except SyncError as e: + sys_exit(f'error: {e}') + + +def warn( + text: str + ) -> None: + 'Print text to stderr.' + print(text, file=stderr) + + +class Server: + 'Server holding a borg repo of PATH_ORG.' + + def __init__( + self, + name: str + ) -> None: + self.name = name + self.path_key = PATH_KEYS / name + self.archives: list[str] = [] + self.reachable = False + self.repo_id = '' + + def location( + self, + archive_name: Optional[str] = None + ) -> str: + 'Repo URL, with "::archive_name" appended if given.' + url = f'ssh://{BORG_ACCOUNT}@{self.name}{PATH_REMOTE_REPO}' + return url if archive_name is None else f'{url}::{archive_name}' + + def borg( + self, + *args, + check: bool = True, + **kwargs + ) -> CompletedProcess: + 'Run borg with args against own keyfile, fail on rc not BORG_RCS_OK.' + result = subprocess_run( + ('borg', *args), + env=environ | {'BORG_KEY_FILE': str(self.path_key), + 'BORG_RSH': BORG_RSH}, + check=False, + **kwargs) + if check and result.returncode not in BORG_RCS_OK: + raise SyncError(f'{self.name}: borg {args[0]} failed') + return result + + def probe( + self + ) -> None: + 'Fill .archives, set .reachable – or warn why the server is skipped.' + if not self.path_key.exists(): + warn(f'{self.name}: skipped, no keyfile at {self.path_key}') + return + result = self.borg('list', '--json', self.location(), + check=False, capture_output=True, text=True) + if result.returncode not in BORG_RCS_OK: + warn(f'{self.name}: skipped, borg list failed:\n' + + result.stderr.strip()) + return + listing = json_loads(result.stdout) + self.archives = sorted(archive['name'] + for archive in listing['archives']) + self.repo_id = listing['repository']['id'] + self.reachable = True + print(f'{self.name}: borg keeps its records of this repo at' + f' {self.path_borg_security}') + + @property + def path_borg_cache( + self + ) -> Path: + "Borg's local cache for repo, as written by borg create." + return PATH_BORG_CACHE / self.repo_id + + @property + def path_borg_security( + self + ) -> Path: + "Borg's local records of repo, as updated on any access." + return PATH_BORG_SECURITY / self.repo_id + + @property + def newest( + self + ) -> Optional[str]: + 'Name of newest archive, if any.' + return self.archives[-1] if self.archives else None + + def init( + self + ) -> None: + 'Create repo on server and its keyfile locally, refuse overwriting.' + if self.path_key.exists(): + raise SyncError(f'{self.path_key} exists, refusing to overwrite') + PATH_KEYS.mkdir(mode=0o700, parents=True, exist_ok=True) + self.borg('init', '--encryption', BORG_ENCRYPTION, self.location()) + + +def ensure_passphrase( + confirm: bool = False + ) -> None: + 'Unless borg has a passphrase source already, ask for one for all repos.' + if 'BORG_PASSPHRASE' in environ or 'BORG_PASSCOMMAND' in environ: + return + passphrase = getpass('borg passphrase: ') + if confirm and getpass('borg passphrase, again: ') != passphrase: + raise SyncError('passphrases differ') + environ['BORG_PASSPHRASE'] = passphrase + + +def load_servers( + ) -> list[Server]: + 'Read PATH_SERVERS: one server per line; blank, "#" lines skipped.' + if not PATH_SERVERS.exists(): + raise SyncError(f'no server list at {PATH_SERVERS}') + lines = [line.strip() for line + in PATH_SERVERS.read_text(encoding='utf8').splitlines()] + return [Server(line) for line in lines + if line and not line.startswith('#')] + + +def probe_servers( + ) -> list[Server]: + 'Load and probe servers, return reachable ones, fail if none.' + servers = load_servers() + for server in servers: + server.probe() + reachable = [server for server in servers if server.reachable] + if not reachable: + raise SyncError('no server reachable') + return reachable + + +def newest_archive( + servers: list[Server] + ) -> Optional[str]: + 'Name of newest archive across servers, if any.' + return max((server.newest for server in servers if server.newest), + default=None) + + +def current_manifest() -> _Manifest: + 'Snapshot PATH_ORG entries: mode, plus size and mtime or link target.' + manifest: _Manifest = {} + if not PATH_ORG.exists(): + return manifest + for dirpath, dirnames, filenames in PATH_ORG.walk(): + for path in (dirpath / name for name in dirnames + filenames): + stat = path.lstat() + key = str(path.relative_to(PATH_ORG)) + if S_ISLNK(stat.st_mode): + manifest[key] = [stat.st_mode, str(path.readlink())] + elif S_ISDIR(stat.st_mode): + manifest[key] = [stat.st_mode] + else: + manifest[key] = [stat.st_mode, stat.st_size, stat.st_mtime_ns] + return manifest + + +def load_state() -> tuple[Optional[str], Optional[_Manifest]]: + 'Last synced archive name and PATH_ORG manifest, if ever synced.' + if not PATH_STATE.exists(): + return None, None + state = json_loads(PATH_STATE.read_text(encoding='utf8')) + return state['archive_name'], state['manifest'] + + +def save_state( + archive_name: str, + manifest: _Manifest + ) -> None: + 'Store archive name and PATH_ORG manifest as last synced state.' + print(f'saving sync state to {PATH_STATE} …') + PATH_STATE.parent.mkdir(parents=True, exist_ok=True) + PATH_STATE.write_text( + json_dumps({'archive_name': archive_name, 'manifest': manifest}), + encoding='utf8') diff --git a/home/user/.local/bin/orgbackup b/home/user/.local/bin/orgbackup new file mode 100755 index 0000000..ae9955f --- /dev/null +++ b/home/user/.local/bin/orgbackup @@ -0,0 +1,71 @@ +#!/usr/bin/env python3 +'Push ~/org as a new borg archive to all reachable servers.' +from datetime import datetime, timezone +from socket import gethostname +from lib.argparsing import ArgParser +from lib.orgsync import ( + DIRNAME_ORG, PATH_HOME, PATH_ORG, Server, SyncError, current_manifest, + ensure_passphrase, load_state, newest_archive, probe_servers, + run_against_syncerror, save_state, warn) + +APP_DESC = 'Back up ~/org to all reachable servers.' + +TIMESTAMP_FORMAT = '%Y-%m-%dT%H:%M:%SZ' + + +def main( + ) -> None: + 'Parse args, run --init or backup.' + parser = ArgParser(APP_DESC) + parser.add_arg('--force', + action='store_true', + help='push even over newer archives not yet restored here') + parser.add_arg('--init', + metavar='SERVER', + help='create repo on SERVER and its keyfile here, then end') + if parser.args.init: + ensure_passphrase(confirm=True) + server = Server(parser.args.init) + print(f'{server.name}: creating repo, keyfile at {server.path_key} …') + server.init() + server.probe() + print(f'{server.name}: repo created.\n' + 'Copy that keyfile to every other desktop, and keep one copy' + ' off all desktops and servers (e.g. printed: it is text).') + return + + if not PATH_ORG.is_dir(): + raise SyncError(f'no directory {PATH_ORG}') + ensure_passphrase() + servers = probe_servers() + last, _ = load_state() + current = current_manifest() + newest = newest_archive(servers) + if newest is not None and (last is None or newest > last)\ + and not parser.args.force: + raise SyncError(f'server has {newest}, newer than last synced here' + f' ({last}); run orgrestore first, or --force') + + timestamp = datetime.now(timezone.utc).strftime(TIMESTAMP_FORMAT) + archive_name = f'{timestamp}-{gethostname()}' + pushed_to = [] + for server in servers: + print(f'{server.name}: creating {archive_name}, updating borg cache' + f' at {server.path_borg_cache} …') + try: + server.borg('create', '--comment', f'parent={last or ""}', + server.location(archive_name), DIRNAME_ORG, + cwd=PATH_HOME) + except SyncError as err: + warn(str(err)) + continue + pushed_to += [server.name] + if not pushed_to: + raise SyncError('pushed to no server') + save_state(archive_name, current) + print(f'pushed {archive_name} to: {", ".join(pushed_to)}') + if len(pushed_to) < len(servers): + raise SyncError('not all servers synced') + + +run_against_syncerror(main) diff --git a/home/user/.local/bin/orgrestore b/home/user/.local/bin/orgrestore new file mode 100755 index 0000000..48ae5af --- /dev/null +++ b/home/user/.local/bin/orgrestore @@ -0,0 +1,66 @@ +#!/usr/bin/env python3 +'Replace ~/org with the newest borg archive across reachable servers.' +from shutil import rmtree +from lib.argparsing import ArgParser +from lib.orgsync import ( + DIRNAME_ORG, PATH_HOME, PATH_ORG, SyncError, current_manifest, + ensure_passphrase, load_state, newest_archive, probe_servers, + run_against_syncerror, save_state) + +APP_DESC = 'Restore ~/org from the newest archive on any reachable server.' + +PATH_STAGING = PATH_HOME / f'.{DIRNAME_ORG}.restoring' + + +def main( + ) -> None: + 'Parse args, run restore.' + parser = ArgParser(APP_DESC) + parser.add_arg('--force', + action='store_true', + help='restore even over unsynced local changes, or an' + ' archive older than last synced') + ensure_passphrase() + servers = probe_servers() + newest = newest_archive(servers) + if newest is None: + raise SyncError('no archives on any reachable server') + last, synced = load_state() + unsynced = PATH_ORG.exists() and synced != current_manifest() + if not parser.args.force: + if unsynced: + raise SyncError('~/org changed since last sync (or never synced' + ' here); run orgbackup first, or --force') + if last is not None and newest < last: + raise SyncError(f'newest reachable archive {newest} is older' + f' than last synced here ({last}); is the server' + ' holding the latter unreachable? Else --force') + + server = next(server for server in servers if server.newest == newest) + if PATH_STAGING.exists(): + print(f'removing leftover {PATH_STAGING} …') + rmtree(PATH_STAGING) + print(f'{server.name}: extracting {newest} into {PATH_STAGING} …') + PATH_STAGING.mkdir() + server.borg('extract', server.location(newest), cwd=PATH_STAGING) + path_extracted = PATH_STAGING / DIRNAME_ORG + # orgbackup archives ~/org as a top-level DIRNAME_ORG/, but an archive + # created otherwise might miss it: e.g. by a hand-run "borg create" from + # another directory (storing home//org/ instead), or by an orgbackup + # from before some change of DIRNAME_ORG or its borg create call; without + # this check, PATH_ORG would get removed below with nothing to replace it + if not path_extracted.is_dir(): + raise SyncError(f'{newest} holds no top-level {DIRNAME_ORG}/, leaving' + f' {PATH_ORG} alone; extraction kept for inspection' + f' at {PATH_STAGING}') + if PATH_ORG.exists(): + print(f'removing {PATH_ORG} …') + rmtree(PATH_ORG) + print(f'moving {path_extracted} to {PATH_ORG}, removing {PATH_STAGING} …') + path_extracted.rename(PATH_ORG) + PATH_STAGING.rmdir() + save_state(newest, current_manifest()) + print(f'~/org is now at {newest}') + + +run_against_syncerror(main) diff --git a/scripts/lib/PATH_HOME_USER.sh b/scripts/lib/PATH_HOME_USER.sh new file mode 100644 index 0000000..01751cc --- /dev/null +++ b/scripts/lib/PATH_HOME_USER.sh @@ -0,0 +1,3 @@ +include USERNAME + +PATH_HOME_USER="/home/${USERNAME}" diff --git a/scripts/lib/PATH_USER_SSH.sh b/scripts/lib/PATH_USER_SSH.sh new file mode 100644 index 0000000..fb59775 --- /dev/null +++ b/scripts/lib/PATH_USER_SSH.sh @@ -0,0 +1,4 @@ +include DIRNAME_SSH +include PATH_HOME_USER + +PATH_USER_SSH="${PATH_HOME_USER}/${DIRNAME_SSH}" diff --git a/scripts/setup_borg_server.sh b/scripts/setup_borg_server.sh new file mode 100755 index 0000000..960825f --- /dev/null +++ b/scripts/setup_borg_server.sh @@ -0,0 +1,61 @@ +#!/bin/sh +. "$(dirname "$0")/_lib.sh" +include DIRNAME_SSH +include PATH_USER_SSH +include USERNAME +include error +include msg +include try_quiet + +# constants unlikely to change +FNAME_AUTHORIZED_KEYS=authorized_keys + +# constants we might want to change at some point +NAME_BORG_ACCOUNT=borg +PATH_BORG_BASE=/srv/borg +DIRNAME_BORG_REPO=org + +# constants derived from changeables +PATH_BORG_REPO="${PATH_BORG_BASE}/${DIRNAME_BORG_REPO}" +PATH_BORG_SSH="${PATH_BORG_BASE}/${DIRNAME_SSH}" +PATH_BORG_KEYS="${PATH_BORG_SSH}/${FNAME_AUTHORIZED_KEYS}" +PATH_USER_KEYS="${PATH_USER_SSH}/${FNAME_AUTHORIZED_KEYS}" +CMD_BORG_SERVE="borg serve --append-only" +CMD_BORG_SERVE="${CMD_BORG_SERVE} --restrict-to-repository ${PATH_BORG_REPO}" +KEY_OPTIONS="command=\"${CMD_BORG_SERVE}\",restrict" + +# sanity checks +[ -r "${PATH_USER_KEYS}" ]\ + || error "cannot read ${PATH_USER_KEYS}" + +msg 'Ensuring installation of borgbackup …' +apt-get -y update +apt-get -y install borgbackup + +if try_quiet id -u "${NAME_BORG_ACCOUNT}"; then + msg 'Account %s already exists, leaving it alone.' "${NAME_BORG_ACCOUNT}" +else + msg 'Creating system account %s with home %s …' \ + "${NAME_BORG_ACCOUNT}" "${PATH_BORG_BASE}" + adduser --system --group --home "${PATH_BORG_BASE}" --shell /bin/sh \ + "${NAME_BORG_ACCOUNT}" +fi + +msg 'Mirroring keys of %s into %s, restricted to borg serve …' \ + "${USERNAME}" "${PATH_BORG_KEYS}" +mkdir -p "${PATH_BORG_SSH}" +sed -e '/^[[:space:]]*\(#\|$\)/d' -e "s|^|${KEY_OPTIONS} |" \ + "${PATH_USER_KEYS}" >| "${PATH_BORG_KEYS}" +chmod 700 "${PATH_BORG_SSH}" +chmod 600 "${PATH_BORG_KEYS}" +chown "${NAME_BORG_ACCOUNT}:${NAME_BORG_ACCOUNT}" \ + "${PATH_BORG_SSH}" "${PATH_BORG_KEYS}" +[ -s "${PATH_BORG_KEYS}" ]\ + || error "no keys found in ${PATH_USER_KEYS}" + +msg 'Ready: %s key(s) of %s may now also run append-only borg serve on %s.' \ + "$(wc -l < "${PATH_BORG_KEYS}")" "${USERNAME}" "${PATH_BORG_REPO}" +msg 'Rerun after changing %s to mirror the change.' "${PATH_USER_KEYS}" +msg 'On desktops, add this server to ~/.config/orgsync/servers; then, once,' +msg 'from one of them (it creates the repo and its keyfile):' +msg ' orgbackup --init ' diff --git a/scripts/start_root_server.sh b/scripts/start_root_server.sh index 9d71357..47d5aca 100755 --- a/scripts/start_root_server.sh +++ b/scripts/start_root_server.sh @@ -1,7 +1,8 @@ #!/bin/sh . "$(dirname "$0")/_lib.sh" -include DIRNAME_SSH +include PATH_HOME_USER include PATH_MY_SSH +include PATH_USER_SSH include USERNAME include disable_apt_recommends include msg @@ -11,10 +12,6 @@ include start_root # constants unlikely to change PATH_SSHD_DROPIN=/etc/ssh/sshd_config.d/60-ssh-hardening.conf -# constants derived from changeables -PATH_HOME_USER="/home/${USERNAME}" -PATH_USER_SSH="${PATH_HOME_USER}/${DIRNAME_SSH}" - disable_apt_recommends start_root server diff --git a/to_install/t490s b/to_install/t490s index 33263a6..95a4155 100644 --- a/to_install/t490s +++ b/to_install/t490s @@ -3,6 +3,7 @@ # # basic helpers ack +borgbackup chrony git man-db