Files
music-curator/.gitea/workflows/build-and-publish.yaml
T
Emma Thorpe 18f05d3d55
Build and publish container / build (push) Failing after 2m16s
feat: ingest a Last.fm scrobble history into a local store
First stage of a curation tool for the music library that music-mirror
mirrors. Before anything can build playlists or decide what has gone cold,
there has to be a local, queryable record of what is actually played; an API
call per question does not scale to a library-sized analysis.

Ingest is in two halves. A catch-up fetches everything scrobbled since the
newest scrobble held, and a backfill walks the history backwards until it runs
out. Both take their bounds from the database rather than from a saved cursor,
so an interrupted run resumes from what it actually has, and both windows are
bounded at each end so paging cannot shift under the fetch while new scrobbles
arrive mid-run.

Scrobbles carry no identifier, so the primary key is timestamp, artist and
track. Two plays of one track in the same second collapse into a single row:
they are indistinguishable in the data, and a surrogate key would make
re-ingest non-idempotent, which is the worse trade.

Three API behaviours are handled explicitly because each fails silently:
the currently-playing track arrives with no timestamp and would be re-ingested
on every pass; a lone result is returned as a bare object rather than a
one-item list; and MBIDs are empty strings rather than absent when unknown,
which would later look like a usable join key.

Retries cover the rate limit and the transient backend errors with an
exponential backoff. An invalid or suspended key fails immediately.

The report exists to surface one number before the next stage is built: the
share of scrobbles carrying a MusicBrainz recording id. Lidarr exposes the same
identifier per track, so those can be joined exactly and the rest must go
through name matching. That percentage bounds how far the matcher can be
trusted.

No runtime dependencies, and the tests run against a fake transport that
reproduces the service's paging and response shapes, so they need neither
network nor credentials.
2026-08-24 12:03:08 +01:00

200 lines
7.6 KiB
YAML

name: Build and publish container
on:
# On merge to main, only build/release when image-affecting files change;
# CI-config and docs changes do not produce a new image. pyproject.toml is
# deliberately absent: the release step below commits to it, and that commit
# must not start another run.
push:
branches: [main]
paths:
- "Dockerfile"
- ".dockerignore"
- "music_curator.py"
# Pull requests always run (tests and the image build are the checks); no
# path filter.
pull_request:
branches: [main]
workflow_dispatch:
# A newer run cancels an older in-flight run in the same group (keyed by ref),
# so a fresh merge to main supersedes the previous build and only the latest
# release is produced. Each pull request likewise supersedes only its own
# earlier runs.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
defaults:
run:
shell: bash
jobs:
build:
runs-on: ubuntu-latest
permissions:
contents: write
packages: write
steps:
- name: Checkout
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
with:
# Full history and tags are required to derive the next version
# from the conventional-commit messages since the last release.
fetch-depth: 0
# The suite runs inside the image, against the interpreter that ships,
# rather than against whatever the runner happens to provide. A failing
# test fails the build. Layers are shared with the push build below.
- name: Run the test suite inside the image
run: docker build --target test -t music-curator:test .
- name: Determine registry host
run: echo "REGISTRY=${GITHUB_SERVER_URL#*://}" >> "$GITHUB_ENV"
# Derive the release version from conventional commits since the last
# v* tag: feat -> minor, fix/perf -> patch, ! or BREAKING CHANGE -> major.
# Anything else (chore, ci, docs, build) produces no release; those builds
# are published under a sha-<short> tag only.
- name: Compute version and image tags
id: version
run: |
set -euo pipefail
image="${REGISTRY}/${GITHUB_REPOSITORY,,}"
last_tag="$(git tag --list 'v*' --sort=-v:refname | head -n1 || true)"
if [ -n "$last_tag" ]; then
range="${last_tag}..HEAD"
base="${last_tag#v}"
else
range=""
base="0.0.0"
fi
subjects="$(git log ${range} --format='%s')"
bodies="$(git log ${range} --format='%B')"
bump="none"
if printf '%s\n' "$bodies" | grep -qiE 'BREAKING[ -]CHANGE' \
|| printf '%s\n' "$subjects" | grep -qE '^[a-z]+([(][^)]*[)])?!:'; then
bump="major"
elif printf '%s\n' "$subjects" | grep -qE '^feat([(][^)]*[)])?:'; then
bump="minor"
elif printf '%s\n' "$subjects" | grep -qE '^(fix|perf)([(][^)]*[)])?:'; then
bump="patch"
fi
major="${base%%.*}"
rest="${base#*.}"
minor="${rest%%.*}"
patch="${rest##*.}"
# workflow_dispatch releases too, so a release can be cut without a
# code change -- the push filter above ignores workflow and doc edits.
# Restricted to main: a manual run elsewhere must not tag a commit
# that is not on the default branch.
release="false"
if [ "${GITHUB_EVENT_NAME}" != "pull_request" ] \
&& [ "${GITHUB_REF_NAME}" = "main" ] \
&& [ "$bump" != "none" ]; then
release="true"
case "$bump" in
major) major=$((major + 1)); minor=0; patch=0 ;;
minor) minor=$((minor + 1)); patch=0 ;;
patch) patch=$((patch + 1)) ;;
esac
version="${major}.${minor}.${patch}"
{
echo "tags<<__EOT__"
echo "${image}:${version}"
echo "${image}:${major}.${minor}"
echo "${image}:${major}"
echo "${image}:latest"
echo "__EOT__"
} >> "$GITHUB_OUTPUT"
echo "version=${version}" >> "$GITHUB_OUTPUT"
else
short="$(git rev-parse --short HEAD)"
{
echo "tags<<__EOT__"
echo "${image}:sha-${short}"
echo "__EOT__"
} >> "$GITHUB_OUTPUT"
fi
echo "release=${release}" >> "$GITHUB_OUTPUT"
echo "Computed bump=${bump}, release=${release}, base=${base}"
- name: Set up Buildx
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4
- name: Log in to the Gitea container registry
if: github.event_name != 'pull_request'
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.repository_owner }}
password: ${{ secrets.PACKAGES_TOKEN }}
- name: Build and push
uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7
with:
context: .
# Without this the last stage in the Dockerfile -- the test stage --
# would be what gets published.
target: runtime
# The NAS is the only host this runs on. Building arm64 as well would
# mean emulating it under QEMU for no consumer.
platforms: linux/amd64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.version.outputs.tags }}
labels: |
org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
org.opencontainers.image.revision=${{ github.sha }}
# Record the release: write the computed version into pyproject.toml, then
# commit and tag it, so the packaging metadata always matches the release
# instead of drifting behind it. The version is derived from commit
# messages and only known here, after the build, so it cannot be set by
# hand in the pull request that causes the release.
- name: Record and tag the release
if: steps.version.outputs.release == 'true'
env:
VERSION: ${{ steps.version.outputs.version }}
run: |
set -euo pipefail
python3 - "$VERSION" <<'PY'
import pathlib
import re
import sys
version = sys.argv[1]
path = pathlib.Path("pyproject.toml")
text = path.read_text()
text, count = re.subn(
r'(?m)^version = ".*"$', f'version = "{version}"', text, count=1
)
if count != 1:
raise SystemExit("no version line found in pyproject.toml")
path.write_text(text)
PY
git config user.name "${{ github.actor }}"
git config user.email "${{ github.actor }}@users.noreply.${REGISTRY}"
git add pyproject.toml
# The file may already carry this version, in which case there is
# nothing to commit and `git commit` would fail the job.
if git diff --cached --quiet; then
echo "pyproject.toml is already at ${VERSION}"
else
git commit -m "chore(release): v${VERSION}"
# Push the branch before the tag. If main has moved on and this push
# is rejected, the job fails without having left a tag pointing at a
# commit that is not on main.
git push origin "HEAD:${GITHUB_REF_NAME}"
fi
git tag -a "v${VERSION}" -m "v${VERSION}"
git push origin "v${VERSION}"