odio CI/CD: build, test, and release pipeline

You don’t need to know the whole CI/CD pipeline to contribute to odio. The sections below cover it piece by piece, with the workflow files linked inline, so you can jump straight to the part you care about. The pipeline runs on GitHub Actions; from a v* tag on a component repo to a signed .deb on apt.odio.love, it’s short enough to trace end to end whenever you do want the full view.

CI on every pull request

Nothing merges to main without the relevant checks running green. The details vary by language:

• Go repos (go-odio-api, go-odio-notify) run go test -race, a coverage gate (15% minimum, uploaded to Codecov), golangci-lint v2.5.0, gofmt, and a go mod tidy drift check. go-odio-api pins Go 1.25 and compiles the embedded Tailwind CSS via Task before the test and lint steps, because the binary ships the UI as an embedded asset; go-odio-notify runs the same checks across Go 1.24 and 1.25. go-mpd-discplayer checks gofmt and runs golangci-lint on Go 1.25, with the cgo headers it links against (ALSA, libdiscid, libudev) installed first.
• odio-pwa runs svelte-check and the Vitest suite on Node 26.
• odio-ha runs ruff, mypy, and pytest on Python 3.14. A second workflow runs the HACS and Home Assistant hassfest validations on every pull request and once a day, so a change to either project’s manifest rules surfaces here rather than in someone’s Home Assistant log.
• pyodio runs ruff and mypy under uv, then pytest across Python 3.12, 3.13, and 3.14. On a v* tag it first checks that the tag matches the version declared in the project.
• mpd2mpris and snapclientmpris share an identical Python pipeline: ruff, mypy, and pytest on Python 3.11, driven through Makefile targets (make lint-ruff, make lint-mypy, make test) so local dev and CI invoke the same commands. On v* tags both run make check-tag to fail fast when the git tag and the package’s __init__.py disagree, before the .deb job spins up.
• odios runs a unified checks.yml workflow combining ansible-lint, shellcheck (installer, test, image-builder, and scripts/), Python unittest against tests/, Python lint with ruff + mypy on the progress callback plugin and the manifest scripts, and a role-drift check that fails any pull request whose touched files don’t match a role-version bump. It exposes a workflow_call trigger so the release pipeline gates publishing on the very same checks.
• odioctl runs gofmt, golangci-lint, and go test -race, plus a drift check that regenerates the sudoers fragment from the DAC catalog and fails when the committed one differs, then visudo -c on it. The .deb job installs the freshly built package in a container and smoke-tests the binary before anything is published.
• odio-apt-repo runs bash -n, shellcheck, and a --help smoke test on every subcommand of the repository build script, plus a pandoc render of the README, which is what the repo serves as its index page. A second workflow builds the full signed repository and uploads it as a plain artifact, so the reprepro output of a change can be inspected without deploying it.
• odio.love runs ESLint, astro check, Vitest, and the Astro build, then two checks of its own: csp:check fails when the Content-Security-Policy header committed to vercel.json has drifted from the built output, and check:obfuscation fails when the contact address or the Discord invite, both assembled at runtime so a crawler finds nothing to harvest, end up literal in the bundle or in a tracked file. A last job follows every redirect the site declares and requires a 200, because a renamed release asset otherwise 404s silently (odio-docs#33).
• odio-docs builds the Astro site, runs lychee for broken links, and cspell on every .md / .mdx file.

Shared CI plumbing

Three of the packaging repos (odioctl, odio-mympd, odio-qbz) were repeating the same handful of steps: publish a Release, dispatch to the apt repo, poll an upstream project for new tags. That plumbing now lives in odio-ci as two reusable workflows and two composite actions. The build itself stays in each repo, because the builds genuinely differ: myMPD is C/CMake delegating to upstream’s build.sh, qbzd is a Rust workspace with local patches and its own nfpm config.

• release.yml collects the per-arch artifacts, fails when no .deb came out of them, derives the prerelease flag from the tag, creates the Release, and fires the repository_dispatch at odio-apt-repo.
• watch-upstream.yml resolves an upstream project’s latest release and pushes a matching tag here when it’s new. The cron stays in the caller, since a reusable workflow carries no trigger of its own, and the push uses a dedicated RELEASE_PAT: a tag pushed with the default GITHUB_TOKEN starts no further workflow, so the build would never run.
• resolve-version turns whatever triggered the run into an upstream tag and a Debian version. A tag on a packaging repo names the upstream version it packages, so v2.0.2-rc1 builds upstream v2.0.2 and versions the package 2.0.2~rc1, with a tilde rather than a dash: Debian sorts ~ below nothing, so 2.0.2~rc1 < 2.0.2. With a dash the release candidate would sort above the final release and apt would never upgrade off it.
• check-binary runs the gates provable from the ELF alone, so they need neither the target board nor QEMU: the ELF machine matches the package arch, the ARM baseline is what it claims (an armhf build assembled as ARMv7 SIGILLs on a Pi 1 or Zero), the glibc floor hasn’t crept above what Raspberry Pi OS ships, and the NEEDED list is printed for cross-checking against the package’s Depends. Runtime checks stay in the per-arch builder container, where execution is native.

Callers pin an exact vX.Y.Z, never a floating v1. GitHub’s immutable releases freeze a tag once a Release exists for it, so the usual trick of moving a major tag forward can’t work; Dependabot bumps the uses: refs instead.

The component release chain

Nine upstream repos feed .deb into apt.odio.love: the three Go components built in-house (go-odio-api, odioctl, go-mpd-discplayer), the spotifyd fork, the two build-only pipelines odio-mympd and odio-qbz, the two pure-Python rewrites mpd2mpris and snapclientmpris, and odio-framebuffer-ui, the kiosk browser behind the local screen. The flow is:

1. A v* tag gets pushed.
2. The repo’s build.yml produces the .deb. The Go components cross-compile for amd64, arm64, armv6, and armv7, packed via nfpm; odio-mympd and odio-qbz build natively in a per-arch rootfs, under QEMU where the runner isn’t already that arch, and gate the result on odio-ci’s check-binary; mpd2mpris and snapclientmpris are pure Python, so a single arch:all .deb is built once inside a debian:trixie container; odio-framebuffer-ui runs dpkg-buildpackage in a per-arch debian:trixie builder image, amd64 and arm64 only on native runners, then lintian and check-binary. go-odio-api also packs .rpm.
3. A GitHub Release is created with every .deb attached.
4. A notify-apt-repo job fires a repository_dispatch event of type release-published to odio-apt-repo, signed with a dedicated APT_REPO_TOKEN. odioctl, odio-mympd, odio-qbz, and odio-framebuffer-ui do steps 3 and 4 through odio-ci’s shared release.yml.

odio-apt-repo receives the dispatch and rebuilds from scratch:

• gh release list resolves the latest stable and latest prerelease for the tracked repos: go-odio-api, odioctl, go-mpd-discplayer, spotifyd, mpd2mpris, snapclientmpris, odio-mympd, odio-qbz, odio-framebuffer-ui. Manual workflow_dispatch accepts an explicit version override per repo.
• Every .deb is downloaded, minus the releases a previous run already fetched: debs/ is cached on the resolved versions, so a rebuild that finds nothing new downloads nothing and the source repos’ download counters stay honest.
• reprepro assembles two suites, stable and testing, across amd64, arm64, armhf, and armv7hf.
• The tree is GPG-signed with GPG_PRIVATE_KEY, written with a CNAME for apt.odio.love, and deployed via GitHub Pages.

A cron runs the same workflow every Monday and Thursday at 04:00 UTC as a safety net, so a missed dispatch can never leave the repo stale for more than a few days, and the .deb cache is touched inside the seven days after which GitHub evicts it. The workflow can also be triggered manually with explicit version overrides.

odio-mympd and odio-qbz follow a build-only pattern: no source is vendored, just CI. Each has a daily watch-upstream.yml cron, staggered so they don’t collide, polling jcorporation/myMPD at 06:17 UTC and vicrodh/qbz at 06:23; a new upstream release gets a matching tag here, and the tag triggers build.yml.

odio-mympd builds myMPD inside per-arch builder images, rebuilt monthly via build-images.yml to pick up base image updates. odio-qbz checks out the upstream tag, applies the patches in patches/, and builds qbzd, the headless daemon. Upstream ships its own qbzd .deb, but only for amd64 and arm64. odio-qbz builds all three arches from the same patched tree and adds armhf, built for ARMv6 so it still runs on a Pi 1 or Zero. GitHub’s ARM runners can’t execute 32-bit ARM code, so that job runs under QEMU emulation and takes hours where amd64 and arm64 take minutes, which is why it gets a longer timeout and a build cache. The b0bbywan/spotifyd fork carries CI-only commits, but the source lives in the repo, so it keeps a build workflow of its own rather than this shape.

odios releases

odios uses a different track: it ships the installer, not .deb packages. On a CalVer tag like 2026.4.1, release.yml runs a long pipeline that:

• Gates the build job on the shared checks.yml workflow (ansible-lint, shellcheck, Python unit tests, plus a role-drift check that fails any PR whose touched files don’t match a role-version bump), so a red lint or test never produces a release.
• Vendors ansible-core as pure Python, strips platform-specific bits, and packs an odio-<version>.tar.gz alongside install.sh and a manifest.json.
• Runs the playbook on ubuntu-latest with an idempotence rerun, then a single-pass cross-arch sanity check on native ARM64 and on ARM/v7 + ARM/v6 under QEMU (the ARM rerun was dropped to keep matrix runtime in check; ARM/v6 stays marked experimental).
• Runs test.sh in install and install-root modes against the just-published pre-release, which exercises the full curl | bash installer path end to end.
• Runs an upgrade matrix against pre-provisioned baselines covering every supported state.json generation, so each upgrade path is exercised before publication. The matrix walks four entry points: fetching the last published odio_upgrade.py (2026.7.0rc2, the migration path off a pre-odioctl baseline) over curl, re-applying through the odioctl upgrade apply that run installed, triggering odio-upgrade.service via systemctl --user, which exercises the sudoers switch to the odioctl group, and one run with --progress that asserts the ODIO_PROGRESS events the callback plugin emits. Two further entries cover what only shows up off the happy path: an upgrade triggered by a user who isn’t the target user, and baselines installed without the MPD stack, where state.roles legitimately omits those entries. arm64 baselines come from the published SD images via scripts/img-to-docker.sh, also rebuilt on demand from test-baseline-image.yml. amd64 baselines are layered onto Dockerfile.test via scripts/build-baseline-amd64.sh so the systemctl path runs under real systemd-logind without QEMU emulation. All baselines live at ghcr.io/b0bbywan/odios/test-baseline:<tag>-<arch>.
• Builds Raspberry Pi images (odio-*.img.xz) for armhf and arm64 via image-builder/build.sh.
• Publishes a combined Raspberry Pi Imager manifest (odio.rpi-imager-manifest) as a release asset, so the images show up inline in the Imager UI. Pre-releases also publish per-arch manifests (odio.armhf.rpi-imager-manifest, odio.arm64.rpi-imager-manifest), so a PR can be tested in Imager as soon as one arch is built, without waiting for the other.

Pull requests to odios run the same matrix against a pr-<N> pre-release instead of the tag, which means the install path you’d take is tested before merge, not only before release.

All of it lands on a single GitHub Release. The URLs users hit, odio.love/install, /manifest.json, and /odio.rpi-imager-manifest, are 307 redirects served by the odio.love Astro site on Vercel, pointing at github.com/b0bbywan/odios/releases/latest/download/<asset>. The URLs stay stable across releases, so curl -fsSL https://odio.love/install | bash never needs updating. Pinning a specific version means calling the GitHub URL with the tag directly, as shown in Upgrade.

PWA and Home Assistant

odio-pwa releases on v* tags. release.yml zips the built dist/ as odio-pwa-<version>.zip, then builds the Docker image for amd64 and arm64 on their own native runners, pushes each by digest, and merges the digests into one multi-arch tag on ghcr.io/b0bbywan/odio-pwa. Prereleases (tags containing -) are marked as such on GitHub; only stable releases move the :latest tag on the container registry.

odio-ha uses a two-step pattern: a semver-tagged push runs the CI workflow, and a separate workflow waits for CI to succeed via workflow_run before creating the release. Prerelease detection is based on the tag suffix (-alpha, -beta, -rc).

The three Python packages, mpd2mpris, snapclientmpris, and pyodio, also publish the sdist and wheel they build to PyPI through trusted publishing: the job authenticates with a short-lived OIDC token bound to the repository and workflow, so there is no API token to store or rotate. Every tag goes to TestPyPI; release candidates stop there, and only final releases go on to PyPI.

Secrets and trust

The pipeline relies on three scoped secrets:

• APT_REPO_TOKEN, a personal access token with repo scope on b0bbywan/odio-apt-repo. Held by component repos so they can fire cross-repo dispatches. Nothing else can trigger the apt-repo rebuild from outside.
• GPG_PRIVATE_KEY, held only by odio-apt-repo. Used by reprepro to sign the Release files, so apt clients on every odio node can verify the signature chain before installing anything.
• RELEASE_PAT, held by the two repos that track an upstream project. Used for one thing only: pushing the tag that a new upstream release produces. A tag pushed with the default token starts no workflow, so without it the build the tag exists to trigger would never run.

PyPI publishing needs no secret at all: the three Python packages authenticate with a short-lived OIDC token that PyPI trusts because of which repository and workflow asked for it.

Everything else (creating GitHub Releases, downloading release assets in another repo, deploying to Pages) uses the default GITHUB_TOKEN scoped to each workflow run.

Manual verification on hardware

On top of the automated matrix, two Raspberry Pis cover the manual checks, with split responsibilities:

• A Pi 3B+ (arm64), a dedicated test rig, validates each PR. The pr-N pre-release is run both as an upgrade from the previous release to the PR version, and as a reflash from the PR’s image.
• A Pi B+ (armv6) takes the post-publication upgrade path: every newly-tagged stable is applied to the previous one on this board, after the release ships. This is also the maintainer’s production node, where a full playbook run takes around two hours, so it picks up upgrades on a post-publication cadence rather than mid-cycle.

Transcripts of these sessions, Ansible PLAY RECAP outputs with failed=0, and the upgrade paths walked are kept publicly in the upgrade-system RFC. That discussion also captures many additional manual hardware runs done while the upgrade system was being built out, kept as design notes rather than a per-release routine.

Repository overview

The ecosystem is nineteen repositories, most connected to each other at build time (Go modules, .deb ingest into the apt repo, archives fetch at install time) or at runtime (client apps hitting the API over HTTP). Green solid edges are build, install, or deploy events; grey dashed edges are runtime. The hosting row at the bottom shows which platform serves each public endpoint: GitHub Pages for apt.odio.love and stats.odio.love, Vercel for everything else. Deploy arrows point from each hosted artifact down to its platform, and the apt install arrow goes from GitHub Pages back up to odios, since that’s where the .deb actually lands at install time.

Repo	CI	Release output	Role in the ecosystem
go-odio-api	Go tests, lint, coverage, CSS build	.deb + .rpm (4 arches), multi-arch GHCR image	Core REST API, installed on every node via apt install odio-api. Consumed at runtime by odio-pwa and odio-ha over HTTP
odioctl	gofmt, golangci-lint, go test -race, sudoers drift, .deb smoke test	.deb (3 arches) + bare binaries	Node control: upgrades, components, DAC overlay, and the settings page. Installed via apt install odioctl
go-mpd-discplayer	gofmt, golangci-lint, build matrix	.deb (4 arches)	CD / USB auto-play daemon, installed via apt install mpd-discplayer. Embeds go-disc-cuer as a Go module
go-disc-cuer	build, test, golangci-lint	Go library, tagged releases	CUE sheet + GnuDB / MusicBrainz lookup, compiled into go-mpd-discplayer
go-odio-notify	Go matrix, lint, coverage	Cross-arch binaries	Shared audio notification lib. Planned as Go module dependency for go-odio-api and go-mpd-discplayer
odios	ansible-lint, shellcheck, playbook + install + upgrade tests	Installer tarball, Pi images, Imager manifest	Orchestrator. Adds apt.odio.love as a source, installs odio-api, odioctl, mpd-discplayer, spotifyd, mpd2mpris, mympd, qbzd, snapclientmpris, and optionally odio-kiosk
odio-pwa	svelte-check, Vitest	Zip archive, multi-arch GHCR images	Standalone client app, talks to go-odio-api over HTTP at runtime
odio-ha	ruff, mypy, pytest, HACS + hassfest	GitHub Release, CI-gated	Home Assistant custom component, talks to go-odio-api over HTTP at runtime
pyodio	ruff, mypy, pytest (3.12 to 3.14)	sdist + wheel on PyPI, GitHub Release	Async Python client for odio-api: a live state mirror over SSE, a high-level hub and a low-level REST client
mpd2mpris	ruff, mypy, pytest, make check-tag on v*	.deb (arch:all), sdist + wheel on PyPI	MPRIS2 D-Bus bridge for MPD. Complete rewrite around python-mpd2 and dbus-fast, with as_directory-aware cover-art lookup for virtual CUE folders. Ingested into apt.odio.love
snapclientmpris	ruff, mypy, pytest, make check-tag on v*	.deb (arch:all), sdist + wheel on PyPI	MPRIS2 D-Bus bridge for the local Snapcast client. Forwards Play/Pause/Next/Previous to the snapserver source so multi-room pauses pause every listener. Ingested into apt.odio.love
odio-framebuffer-ui	builder images, per-arch native build, lintian, ELF gates	.deb (amd64, arm64)	odio-kiosk, a Qt 6 WebEngine browser drawn straight onto the framebuffer, one instance per screen. Shows the embedded UI on the local screen. Ingested into apt.odio.love
odio-mympd	builder images, per-arch build, ELF gates	.deb (3 arches)	Build-only pipeline (no source vendored). Daily cron tracks upstream jcorporation/myMPD; tagging here ships the matching myMPD .deb to apt.odio.love
odio-qbz	per-arch native build, ELF gates	.deb (3 arches)	Build-only pipeline for qbzd, the headless Qobuz daemon from upstream vicrodh/qbz, patched, with the armhf build upstream doesn’t ship
spotifyd	Rust build matrix	.deb (4 arches)	Fork of Spotifyd carrying the CI that ships .deb to apt.odio.love, no source changes. Provides Spotify Connect
odio-ci	(reusable workflows and actions)	Immutable vX.Y.Z releases	Shared CI plumbing: release + apt-repo dispatch, upstream watching, resolve-version and check-binary, and the pinned base images
odio-apt-repo	shellcheck, --help smoke test, PR build of the signed repo	apt.odio.love via GitHub Pages	Ingests .deb from go-odio-api, odioctl, go-mpd-discplayer, spotifyd, mpd2mpris, snapclientmpris, odio-mympd, odio-qbz, and odio-framebuffer-ui
odio.love	ESLint, astro check, Vitest, CSP + obfuscation checks, lychee, redirect check	Stable 307 redirects	Serves /install, /manifest.json, /odio.rpi-imager-manifest from the latest odios GitHub Release
odio-docs	Astro build, lychee, cspell	Deployed separately, plus a daily stats build	This site. A daily workflow also publishes the ecosystem stats behind the activity page to stats.odio.love

Tip

Something in this pipeline could be better? Open an issue on odio-docs or start a discussion on odios.

Acknowledgments

• The people whose work and thinking have inspired me through odio’s development.
• Laurent T., for the Pi 3B+ used for per-PR testing, which keeps that loop off my production node.
• @vascoguita, for raspios-docker, the per-arch Raspberry Pi OS Docker images that every ARM .deb and the upgrade test images are built on top of. Specifically:
  • go-mpd-discplayer’s Dockerfile cross-builds the CGO .deb for arm64/armhf inside vascoguita/raspios:arm64 and vascoguita/raspios:armhf under QEMU.
  • spotifyd’s Dockerfile.builder uses the same bases for the spotifyd-builder image published to GHCR.
  • odio-mympd’s per-arch builders are rebuilt monthly on top of the same images.
  • odios’s test image targets the same bases so the upgrade matrix runs against a Pi-faithful environment, and scripts/img-to-docker.sh derives the upgrade-test baselines following the same approach.