All notable changes to this project are documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
The version is 0.x, which by SemVer means the interface may still change between minor
releases. The metadata contract is the exception: what is written to Jellyfin is
specified by ADRs 0003, 0004 and 0007 and those guarantees are treated as stable.
A configuration reference. docs/configuration.md lists every
environment variable the stack reads, with its default and what it changes — including the
ones that were documented nowhere before: the Last.fm rate limiter (LASTFM_MAX_RPS,
LASTFM_BURST), JELLYFIN_USER_ID, ARCHIVE_SIMILAR_FRESHNESS_TTL_S and
METAEDIT_BROWSE_SCAN_CAP. Every row was verified against config.py, the compose file
and the test conftest rather than written from memory, because a configuration table that
has drifted is worse than none: it gets trusted and copied.
The harvest endpoints in the API reference. POST /api/items/{id}/harvest and the
batch POST /api/harvest were implemented and mounted but absent from docs/api.md,
which claimed to list “every endpoint”. The new section states the load-bearing rule:
harvest spends Last.fm requests and writes to the archive, but never writes to Jellyfin.
README and docs enrichment. The quickstart’s requirements are a table; the feature
list covers narrowing and bulk harvest; the documentation index gains Configuration and
Changelog rows. docs/development.md lists the service/ package the layout section had
omitted, documents all four pnpm scripts, and notes that integration tests skip
cleanly when no Postgres answers — an ordinary uv run pytest on a fresh checkout
reports skips, not failures. docs/operations.md documents the rate limiter and the
user-context knob.
Stale design status corrected. The phase-3 design doc still said
“metaedit reindex raises NotImplementedError” while reindex has been shipped since
0.1.0 — the kind of sentence a reader cannot distrust, because it sounds like the
authoritative status. Both the design index and the doc header now say implemented,
with the document retained as the contract the implementation must keep satisfying.
Version bumped to 0.2.1 across pyproject.toml, the package, the OpenAPI
info.version (contract regenerated), the Jellyfin client identity, and the Last.fm
user agent — whose compose default said metaedit/0.1.0 while the app default said
0.2.0; both now agree.
Filter albums by artist, and songs by artist and/or album. The library screen, the bulk
form and the genre-removal form now offer the same two facets, and they are Jellyfin’s own
parameters — so total is the filtered count, paging is over the filtered set, and nothing
is scanned. Verified against Jellyfin 12.2.0: ArtistIds=<ABBA> returns 2 of 502 albums and
21 of 5,442 songs, and ArtistIds together with AlbumIds narrows both ways at once.
Ids must be 32 hex characters or a dashed GUID, and anything else is refused with a 422.
That guard is not defensive politeness: Jellyfin discards an id list it cannot parse in
full and answers with the unfiltered library — measured, ArtistIds=abc returns all 502
albums where a valid id returns 2 — so a single typo would turn “narrow to these three
artists” into “the whole library”, on a selection that feeds a write. The error says so.
Exclusion patterns. A pattern is a case-insensitive glob (*, ?) matched against the
item’s name, its album and its album artist — all three, because Various
Artists is a compilation marker that appears in the album artist and in no track or album
name, so a matcher reading only the name would leave every compilation in the selection.
Matching is literal: live hides exactly live, and *live* is what hides a substring.
Glob rather than regex because the input is typed by hand into a box, and fnmatch escapes
everything but the wildcards, so a pathological pattern cannot become a
catastrophic-backtracking regex pointed at your own server.
Jellyfin has no parameter for this, so it costs a read — and the response says what the read
covered: scanned, matched, excluded, truncated, limit. excluded and matched are
disjoint, and a truncated scan is reported as a lower bound rather than presented as a total.
Pagination above the table, as well as below it. A long page is read from the middle, and an operator who has scrolled to the last row should not have to travel back up to move on. The two controls carry distinct accessible names, so they are not ambiguous to a screen reader — or to a test.
A filtered batch or removal now fills its limit. POST /api/bulk/diff and
POST /api/bulk/remove-genre/diff accept the same three narrowings, and a selection that
drops items to a pattern or an aspect filter keeps reading until it has limit items that
pass, rather than returning however many of the first limit survived. Before, “limit 25”
with missing=genres could legitimately return four items with nothing to explain the
difference, which reads as “there is nothing else to do”. The job summary carries excluded
(and scanned/truncated when the library had to be read), so a short result is explained
rather than merely short.
POST /api/harvest accepts the same three narrowings too, so one selection means one thing
across the app.
A genre blacklist you can edit while the app is running. Settings → Blacklisted
genres takes the genres this tool must never propose for writing. Matching is exact and
case-insensitive, so Rock blocks rock and ROCK but leaves Gothic Rock,
progressive rock and rockabilly alone. Entries are stored in Postgres and take effect
on the very next request; TAG_BLACKLIST_EXTRA still works and merges in, so an existing
deployment is unchanged.
The screen shows what each entry matches before you save it, because the input format
hides a real ambiguity: a comma separates entries as you type, and is also a legal
character inside a genre value. A line containing one is reported with both readings —
“as one genre it matches Rock, Reggae; split it would match Rock, Reggae” — rather
than guessed at, and splitting is a separate labelled action.
A batch tool that removes one genre from a whole library. Remove genre finds every item carrying a value, shows what would change, and applies it as one revertible batch. It reuses the existing bulk safety model rather than inventing a second one: a diff job must exist before an apply, failures are isolated per item, every write is snapshotted, and the whole run reverts through the same endpoint as any other batch.
Two things about it are worth knowing before you use it:
Genres=Rock
selects 67 artists where Genres=Rock, Reggae selects one. A substring matcher would
delete 67 artists’ worth of curation for a request naming a genre that occurs once.Thrash Metal, Speed Metal, Heavy Metal, Hard Rock). “Also split packed values” removes
matching parts of those, and it is off by default because a packed value is more often
one genre whose name contains a separator than a list.Blacklisting a genre does not delete it from existing items. The blacklist stops a genre being added; deleting a stored one is the removal tool’s job, on purpose — a settings change that silently rewrote a library would be a large, unreviewed write.
0003_genre_blacklist applies forward-only onto
0002_album_overview. It adds one table and touches nothing else.alembic check cannot be used as a CI gate on this repository, and this release is
where that was discovered. Autogenerate reads the runtime-created
lastfm_request_YYYY_MM partitions as tables to remove, so its report is
indistinguishable from real drift — and applying the autogenerated output verbatim would
have deleted the archived request log. The migration docstrings record it; an
autogenerated revision needs a hand review here.POST /Items/{itemId} rounds PremiereDate to
midnight for every write (ADR 0014, pre-existing), and any merge write decomposes a
stored comma-bearing genre like Rock, Reggae into separate genres (pinned in
test_a_stored_comma_genre_is_decomposed_on_merge, pre-existing). Neither is introduced
by this release and neither is worked around.Editing any item with a PremiereDate failed with an internal error. One wrong
representation crossed three boundaries, and each fix revealed the next.
NormalizedItem.fields held PremiereDate as a datetime, and only the fields present in
changes were converted back to an ISO string on the way out. Fields carried through from
current state were emitted verbatim, which raised
TypeError: Object of type datetime is not JSON serializable at:
json= encoder, on the same unconverted value. Not
visible until (1) was fixed, because (1) always failed first.as_state()["fields"] reached
JSONResponse’s plain json.dumps.Measured on a live library the blast radius is most of it: 5,916 of 6,583 items carry a
PremiereDate — 470 of 502 albums and 5,437 of 5,442 songs. Artists mostly do not
(9 of 639), which is why it surfaced as “editing an artist from the third page” rather than
as “editing anything at all” — and why albums and songs were near-universally broken.
Fixed at the representation rather than at each destination: _normalize, the single door
into fields, now returns the wire form for every field. No path can put a
non-serialisable value into a JSONB column or an HTTP body. See ADR 0014. One consequence
worth knowing: GET /items/{id}/state returns PremiereDate as an ISO string now. That
was previously a latent 500 for any handler that echoed it.
An unanticipated failure on one item killed the whole batch. apply_job caught only
MetaeditError. Anything else escaped the per-item loop with the session still dirty, so
the next item’s flush raised PendingRollbackError and the run stopped with the original
cause buried under a second error. Items are committed independently so that one failure
does not discard the rest; that guarantee now holds for failures we did not predict as well
as for the ones we did.
Jellyfin discards the time component of PremiereDate on every write. Verified on
12.2.0: writing 1987-01-01T03:00:00+00:00 back to an item that already held exactly that
value returns 1987-01-01T00:00:00+00:00. The server normalises to midnight and does so
even for a no-op write.
This means editing any field on an item whose PremiereDate has a non-midnight time will
round that date to midnight. The snapshot records the true pre-write value, so a revert
restores what was captured — and is itself rounded on the way in. Six artists in the
reference library are affected, where the time component is a timezone offset, so the date
itself does not change. Not worked around: the endpoint is a full overwrite (ADR 0003), so
omitting the field would null it, and refusing every edit to a dated item would make most
of the library uneditable. Recorded in ADR 0014 because the alternative is a user finding
it in their own library.
Filtering, pagination and sorting on the browse screens. /api/items gains sort,
order and a per-media-type missing filter; the Library and Archive tables gain real
paging, sortable headers, removable filter chips and a scan note. Every part of a browse
query now lives in the URL, so a refresh, a back button and a shared link mean the same
thing.
The sort vocabulary is an allow-list, and that is load-bearing rather than tidy.
Live-verified on Jellyfin 12.2.0: an unrecognised sortBy value is accepted and silently
ignored, falling back to the default order. A pass-through would let the UI present items
in one order while claiming another, with nothing anywhere reporting a problem. Every key
in the map was verified to change the returned order on a real server, not inferred from
the field names:
name -> Name sort_name -> SortName date_added -> DateCreated
year -> ProductionYear random -> Random
Three response fields exist so the UI cannot mislead about what it is showing: total
counts the query rather than the page, sort/order are echoed back so the UI renders
what the server did, and scan reports what a filter read to produce its count.
Archive paging silently lost a row while showing another twice. ORDER BY name is not
a total order — 8 album rows and 2 track rows share a name with another row — and OFFSET
paging over it is incorrect by construction. Measured against a real archive:
109 rows fetched, 108 distinct
duplicated across pages: [90] 'Corazones' -- Jorge Gonzalez
never appearing: [8] 'Corazones' -- Los Prisioneros
The count said 109, the walk yielded 109 entries, and one of them was a duplicate, so the
missing row was invisible from the UI. Every ordering now ends with id, which the reindex
assigns deterministically, so the tiebreaker is stable across rebuilds. Confirmed
deterministic rather than flaky: five runs of each ordering gave 108 distinct with
duplicate id 90 for ORDER BY name, and 109 distinct with none for ORDER BY name, id.
The missing-metadata filter was useless for songs. A song without an Overview is the
normal state of a song, and the old boolean counted that as missing metadata:
MusicArtist 500 items: overview in 254, provider ids in 414, genres in 392
Audio 500 items: overview in 1, provider ids in 69, genres in 314
So over a song library it reported 5,441 of 5,442 items incomplete — a wall of red that buried the 1,466 songs whose missing genres are actually fixable. It was not a filter that was slightly off; it could not be fixed by filtering harder. The filter is now a per-media-type definition, and the bulk and harvest selections use the same one: they had kept their own inline copy of the old test, on a screen whose output is a write.
Jellyfin cannot express this for music (Filters=IsMissing is accepted and ignored), so
matching items means reading them. That scan is capped, and the cap is reported:
?kind=song&missing=genres answers “679 matched from 2,000 checked, truncated” in 1.8s, which
is a labelled lower bound rather than a total that looks complete and is not.
The library’s total described a set the table could not show. The screen fetched one page of 200 and filtered in the browser, so the header read “641 artists” above 200 rows; with 5,442 songs, 96% of the library was unreachable and nothing said so.
“Select all” meant “select the first page”. On a path that feeds a write, an operator could believe a library-wide edit was queued when a fifth of it was. The control now says “Select these 50”, reports how many of the selection are on the current page, and keeps a selection across pages — selecting across pages is why paging exists — while still clearing it when the query changes, because a selection carried into a different result set would attach writes to items nobody looked at.
Batch filtering over songs. missing_metadata on the bulk and harvest selections was
replaced by the same missing vocabulary as the browse, so the three surfaces cannot
disagree about which items need work.
NULLS LAST on the nullable sort keys, argued from the data rather than from theory:
all 383 artist rows have a NULL listeners, against 0 of 109 albums. Postgres
defaults to NULLS FIRST on DESC, so “sort artists by listeners, descending” would
have led with every row that has no listener count.Id, or even a nonsense key,
produced byte-identical responses on a live 12.2.0 server. Adding a key that does nothing
reads as a guarantee it is not. The browse is safe today — all 6,583 items walk without
loss, and ten identical requests return one identical order — but that guarantee is the
server’s, not ours. Recorded in domain/browse.py.tsconfig.json includes only
src, and Playwright transpiles without checking, so every spec in e2e/ was the only
TypeScript in the app that nothing verified. Adding tsconfig.e2e.json found seven
strict-mode errors in the pre-existing specs, including an unused fixture.scripts/run-tests.mjs treats “no test files
matched” as a failure: node --test exits 0 on an empty glob, so a renamed file would
otherwise have left the build green while checking nothing.Bulk editing over songs was broken: unknown selection kind 'song'. There are two
names for the same thing and they are deliberately not interchangeable — the query
vocabulary (artist / album / song) that every human-facing surface uses, and the
archive vocabulary (artist / album / track) that the derived tables and ReindexKind
use, whose reader is someone writing SQL. The bulk selection took its enum from the query
side and its lookup table from the archive side, so the SPA offered “Songs”, the schema
accepted "song", and the service rejected it every time. Harvest had the identical bug
through the same shared path. The two mappings now sit next to each other in one module with
the distinction spelled out, and test_kind_vocabularies.py pins the relationship rather
than the values, so adding a media type means being told about every place that must agree.
A timestamp anywhere in an error detail turned the error into a 500. An exception
handler builds its own response, so JSONResponse(content=...) runs plain json.dumps and
not FastAPI’s jsonable_encoder. PremiereDate lives in NormalizedItem.fields as a real
datetime (so date comparisons work), that state travels in ConflictError.current so a
stale-Etag 409 can show what changed, and the body then failed to serialise — the one
error whose job is to explain a conflict was the one that could not render. Songs were where
it surfaced because a song usually carries a PremiereDate and an artist usually does not.
api/serialization.py now holds the single definition of “can this go in a response body”,
applied where the body is built. The bug was never a missing conversion; it was conversion
being remembered at four of five places.
An unexpected 500 now says something a user can act on. The body was
Unexpected server error. — nothing to send, nothing to search for. It carries a
reference that appears in the server log, the exception type, the method and path, and
retryable. The reference is also a structured field on SSE error events rather than only
inside prose, so the SPA can surface it without parsing a sentence that is ours to reword.
Nothing about the disclosure policy changed: a MetaeditError message still passes through,
and anything else still gets a reference instead of its str().
Also: harvest reported an unanticipated failure as harvest_failed while bulk reported
internal_error for the same RuntimeError — a label for the route being used as a code
for the failure. Both report internal_error now.
The README is now a landing page; the detail moved into docs/. It had grown to 500
lines with no order to them — the endpoint table came before the paragraph explaining why
writing is safe, hardening sat under “Status”, and the first thing a reader met was a
status table rather than what the tool does. It is 98 lines now: what it does, the four
write guarantees, a quickstart, and a table of where to read more.
Four documents took the rest, split by who needs them rather than by where the text used to
be: docs/development.md for a source checkout, checks and tests;
docs/api.md for the endpoints and the write-safety rules;
docs/operations.md for the archive, the CLI and retention;
docs/architecture.md for status, the UI, layering and deviations.
Nothing was dropped — every section of the old README is accounted for in one of them — but two claims were deleted rather than moved, because they had been false for a while:
reindex landed.Both survived because each phase that should have deleted them edited a different part of the README. A stale claim inside a file someone is actively maintaining is not caught by that maintenance, which is the argument for splitting the document in the first place.
Also corrected while moving: the README pointed at a Testing section that does not exist;
the documented route list was missing /review; “nine Playwright specs” undercounted the
file’s fifteen; and .env.example shipped a LASTFM_USER_AGENT naming version 0.1.0 and
a placeholder repository, two releases stale, with a comment that implied leaving it blank
was equivalent to the default — it is not, it sends no User-Agent at all.
Postgres-backed tests clone a migrated template instead of migrating per test. Every
one of them ran alembic upgrade head in a subprocess: most of a second spent importing
Python and replaying a migration chain whose result is identical every single time. It was
not a hidden cost either — 132 tests had a measurable setup, adding up to 119 s of the
139 s the integration suite took, which is to say the suite was mostly replaying
migrations.
The chain now runs once per session, into a template database, and each test’s database is
copied from it with CREATE DATABASE ... TEMPLATE. Every test still gets a private, empty,
fully migrated database; only the way it is populated changed.
test_test_database_isolation.py writes a row into one
clone and checks the other cannot see it, and checks that a clone really is at the
migration head — alembic_version is copied rather than applied, and a template built
with create_all would leave the table empty while every schema test still passed.test_spa_static.py covers the containment check in the SPA fallback, which had no test.
It was not decorative: deleting candidate.is_relative_to(dist) makes
GET /..%2Fsecret.txt return a file from outside the bundle, which was verified by deleting
it. The request is percent-encoded deliberately — a literal ../ is collapsed by the client
and the server before the route sees it, so only an encoded separator actually arrives as
the path parameter. Without that detail the test would have passed for the wrong reason.
py/path-injection (three alerts, main.py) — the alert is on the file read, and the
query does not model pathlib.Path.is_relative_to as a guard. The guard is there, and
now tested.py/weak-sensitive-data-hashing (canonical.py) — the hash is a 16-character
correlation fingerprint so the archive can say which key fetched a payload without
storing the key. It is not password storage and is never used to verify anything.ubuntu-latest. The previous state was mixed: three jobs on
ubuntu-latest and the Python job still on ubuntu-24.04, which is an inconsistency
dressed up as a pin. The deliberate trade is recorded in the workflow: the runner moves
on its own, and a job that breaks on a new runner is more useful than one that quietly
keeps testing an old one. The versions the tests are actually sensitive to — Python, uv,
Node and the Postgres image — are pinned explicitly instead.-q. addopts already sets one and pytest reads a
second as a further drop in verbosity, so it suppressed the summary line: a green run
whose log never said how many tests ran, which is exactly the kind of detail that hides a
suite that quietly stopped collecting. There were two of them, in the python and
contract jobs, and the first fix caught only one — the search that found it looked for
pytest -q and the second read pytest tests/contract -q.metaedit, so the in-repo links and the Last.fm
User-Agent URL now name it. They had been relying on GitHub’s redirect for renamed
repositories, which works right up until someone creates a repository under the old
name — at which point the links quietly point somewhere else. CHANGELOG.md and
src/metaedit/config.py are the only two files affected.main is protected by a ruleset that requires an approving code owner review, and
.github/CODEOWNERS now says who that is. Until this change the rule named nobody and
was therefore not enforcing anything.Internal error text no longer reaches the browser. CodeQL found this
(py/stack-trace-exposure), and it found it twice — which is the interesting part,
because the second instance was still there after the first fix. Each place that reported
a failure was answering “is this exception safe to quote?” for itself, and one of them
answered wrong.
str(exc) straight into the error frame. An unexpected failure
now reports a code and a short reference id; the traceback goes to the server log
under that same reference, so a report from a user still leads to the cause.domain.errors.public_error_text is that function, and the single definition of what a
client may be told. The distinction it draws is provenance, not severity: a
MetaeditError message was written by us for a reader and passes through; anything
else gets a reference.httpx failures collapse to a type name before escaping, so the key in the query string
never reached a message; the Jellyfin key travels in a header and its bodies already
pass through _sanitise_body; and a failing Postgres connection renders with the
password masked. What leaked was internal topology — driver names, host and port.SelectionError now subclasses MetaeditError instead of ValueError. It was an
expected failure with a person-readable message all along, and being a bare ValueError
meant the write path had two hierarchies to catch and its HTTP status lived in a handler
in another layer. The status (422) and code (invalid_request) are unchanged, but they
now sit next to the failure that causes them, and the per-item failure payload no longer
needs to special-case it.domain/errors.py had promised this from the start — “no upstream payload ever reaches a
client verbatim” — so both leaks were omissions, not decisions.
Found by CodeQL code scanning, which was enabled on this repository between the two releases.
actions/checkout v7,
actions/upload-artifact v7, actions/setup-node v7) and astral-sh/setup-uv to
v10.1.0, pinned to a commit SHA so a moved tag cannot change what CI runs./openapi.json is the only interface difference.The first release: a working Last.fm → Jellyfin metadata editor with a persistent local archive, a review step before every write, and one-click undo.
Editing flow. Search the Jellyfin library, select one or many items, fetch their Last.fm data, review the proposed changes field by field, confirm, and write. Every write is snapshotted first and can be reverted, individually or as a batch.
getInfo, getTopTags and getSimilar, archives every response, and
rebuilds the derived layer. A miss is a reported outcome with close matches, not an
error, so a library-wide fetch does not abort on the first obscure item.Etag.Last.fm archive. A persistent, incremental, structured store of everything fetched, so the same request is never paid for twice.
Domain layer. Tag policy (genre/style limits, minimum counts, blacklist, and merge/replace/fill-if-empty modes per field), HTML-to-text sanitisation of biographies, weighted confidence scoring with auto/review/reject verdicts, and a diff that only ever proposes what policy permits.
API and contract. Typed response models for every endpoint, so the OpenAPI document describes what the server actually returns. The SPA’s types — including query parameters — are generated from that document rather than hand-written, and CI fails if a schema change was not regenerated.
Operations. Multi-stage Dockerfile running as a non-root user, docker-compose with
docker.io/postgres:18-trixie, migrations as a separate idempotent step, health and
readiness endpoints, and structured logging.
Bugs found only by running against real Jellyfin 12.2.0 and the live Last.fm API. Each is recorded because the class of mistake is more instructive than the instance.
GET /Items/{itemId} needs a user context, and
a userless API key must pass userId explicitly — for every id, including ones the
server itself had just returned from /Artists. Because apply reads an item before
writing, the whole edit path was dead and surfaced as a write failure that never
attempted a write./api/info reported a working API key as unusable, so the UI hid the entire editor
on a correctly configured deployment.artist.getTopTags is envelope-free, so
the entity derivation skipped it and every tag count was null — silently disabling
min_count and reducing ranking to list order everywhere.Authorization: <key> returns 401; the MediaBrowser scheme is required./Artists returned nothing when given includeItemTypes — a filter that looks
harmless and emptied the browse.getInfo carries no wiki;
it does, and the text was being thrown away.AlbumArtist, Artists and AlbumArtists are three spellings and only the array
was read, leaving some albums with no artist and therefore no possible Last.fm lookup.warn could be configured past refuse; prune-raw exited non-zero on a
successful dry run; and an album-only AttributeError reached the API as a 500.nosniff,
frame-ancestors 'none', Referrer-Policy: no-referrer, and a Permissions-Policy
denying camera, microphone and geolocation.Content-Length.SecretStr, so model_dump() and repr() cannot leak them, and
upstream error bodies are redacted before being surfaced.docs/design/0003 specifying the derivation contract, including the deliberate
deviations from the original plan.tests/support/ now reproduce what the servers
do — including a Jellyfin that really performs the full overwrite, so a payload that
drops a field visibly destroys data.