# 3. Metadata

> How Boeks reads a book's details from its EPUB, and writes edits back into it.

Canonical: https://boeks.app/docs/specs/metadata

## 3.1 Reading

Metadata is read from the OPF of EPUB 2 and EPUB 3 files. Only the title is required.

- **Title and subtitle** come from EPUB 3's `title-type` (`main`, `subtitle`). Without one, the first title is the title, and the subtitle comes from `boeks:subtitle`. Other title types, such as `collection` and `short`, are kept.
- **Creators** have MARC relator roles (`aut`, `edt`, `trl`, `ill`, `nrt`), and a `file-as` for sorting. The main title can have a `file-as` too.
- **Series** come from EPUB 3's `belongs-to-collection` with `collection-type` `series` and a `group-position`, which may be a decimal. A book can be in several series. EPUB 2 has no series, so there they come from `boeks:series` and `boeks:series_position`.
- **Identifiers** are free-form `scheme:value`, with the scheme names Calibre uses: `isbn`, `goodreads`, `hardcover`, `openlibrary`, `google`, `url`, `uuid`, and others. Amazon's IDs are `asin`, an ebook's and an audiobook's alike, as the ID is called; Calibre's `amazon`, `amazon_<country>`, and `mobi-asin` are read as `asin`. Boeks adds `inventaire`, a work's Inventaire URI: its Wikidata item (`wd:Q190192`), or Inventaire's own entity for one Wikidata hasn't (`inv:…`). A `wikidata` item stored by another tool (`Q190192`) is read as the same work.
- **Other apps' fields,** such as Calibre's `calibre:series` and its custom columns, aren't read.

```xml
<metadata>
  <dc:title id="t1">Dune</dc:title>
  <meta refines="#t1" property="title-type">main</meta>
  <dc:creator id="c1">Frank Herbert</dc:creator>
  <meta refines="#c1" property="role" scheme="marc:relators">aut</meta>
  <meta refines="#c1" property="file-as">Herbert, Frank</meta>
  <meta property="belongs-to-collection" id="s1">Dune</meta>
  <meta refines="#s1" property="collection-type">series</meta>
  <meta refines="#s1" property="group-position">1</meta>
</metadata>
```

## 3.2 Writing

- **Only the OPF changes,** and a new cover's image (§3.6). Unknown elements are kept, and an EPUB 2 file stays EPUB 2.
- **The zip is deterministic,** so the same edits give byte-identical files:
  - `mimetype` comes first, then the other entries in the original file's order, then any files a transform or a new cover adds, by name.
  - Entries that didn't change are copied as they are, compressed bytes and headers included.
  - Entries written anew, `mimetype` and the OPF among them, are stored uncompressed, dated 1980-01-01 00:00 (the earliest a zip can hold), with Unix permissions 0644.

  Devices running the same Boeks write the same file. Another tool following this writes the same entries, but the zip's headers have details this doesn't fix (such as the version that made it, flags, and extra fields), and how the OPF is serialized isn't specified, so its file may differ in bytes: the recovery below handles that.
- **EPUB 3** gets the standard forms (`title-type`, `belongs-to-collection`). **EPUB 2** has no place for some fields, so they're written as `boeks:` metas: `boeks:subtitle`, and `boeks:series` and `boeks:series_position` for one series.
- **The format's own fields** use the `boeks:` prefix.
- **Other apps' fields** are neither written nor removed.
- **Never written:**
  - user fields, such as rating, status, and collections, which are in `Book.md` (§6.2)
  - DRM'd files
  - files that fail validation

  The edits to a file that can't be written stay in the logs, and the clients show them over the file's own metadata, which the index keeps as the file has it (§4, §12.2).

## 3.3 Edits are logged, then applied

Edits are written into the EPUB from the logs, like the files of §6.

- **Logging:** an edit is a `metadata.updated` in the editing device's log, for one field of one edition.
- **Applying:** a device that has the EPUB, downloaded and writable, writes the pending edits into its OPF. It then logs `metadata.applied` with the file's hash before and after, and the edits it wrote.
- **Not every device applies.** A device may leave applying to others; it still logs edits.
- **Two devices applying at once is safe:**
  1. **The output is deterministic,** so the same edits give identical bytes. A conflicted copy that the sync makes is an exact duplicate: delete it.
  2. **The logs stay the truth until edits are applied.** For copies that differ, keep the one whose hash matches the latest `metadata.applied`, and delete the other. Follow the before and after hashes back from the kept file to see which edits it holds. Edits that only reached the deleted copy are pending again, and are applied again. When neither copy matches, as when the applying device's log hasn't arrived, both stay, reported as a conflict, until one does.

## 3.4 Transforms

A transform changes an edition's file: it has a name and a version, and takes the file in and gives a file out. It uses no clock and no randomness, so the same input always gives the same bytes. It changes how the file is put together, not what it says: every content document's text stays as Locators count it (§7), so annotations hold. Applying edits (§3.3) is one; the upgrade to EPUB 3 (§3.5) is another.

- **Its version** is an integer of its own. Raise it whenever the same input would give different bytes; a fix that doesn't change the output keeps it. A `send` transform's version is its release instead, such as kepubify's `5.0.0`: it's never logged, only used to cache the output (below). It's neither an app's version nor the format's `version` (§1.2).
- **Names** are lowercase letters, digits, and `-`, such as `epub3`.
- **Logged, then applied, like edits.** Asking for a transform logs `transform.requested` `{edition, name}`. A device that applies edits runs it together with any pending edits, in one rewrite, and lists it with its version in `metadata.applied`. A device that doesn't apply can still ask.
  - **The request names no version.** A device runs the version it has. Two devices with different versions applying at once write different files, which the recovery of §3.3 sorts out.
  - **Time:** a transform that needs one, such as the upgrade's `dcterms:modified`, takes the request's timestamp, never the time of applying.
  - **One that can't run** on this file, such as `epub3` on an EPUB 3, is still listed in `metadata.applied`, so it isn't tried again, and the device reports it. When nothing was written, `hash` is `base`.
  - **One the device doesn't know** stays requested, for a device that knows it. It holds up nothing else.
- **On import:** the library's `transforms` setting (§1.2) lists, by hook, the transforms to request for each edition imported, in order:

  ```json
  "transforms": {
    "import": [{ "name": "epub3" }]
  }
  ```

  - An entry is `{name}`. Keep the names you don't know in the setting; never drop them.
  - Each is requested right after `edition.added`, and applied like any other request by a device that has it.
- **At any other time,** a transform can be requested for any edition.
- **`send` transforms** change the copy a device is given, never the library's file. Their output isn't logged: it's derived, and cached by the file's hash and each transform's version.

## 3.5 Upgrading to EPUB 3

The transform `epub3` upgrades an EPUB 2 edition to EPUB 3.

- **Only the package changes.** Content documents stay byte for byte as they were, doctype included, and so do their paths, so every Locator and annotation still points where it did (§7). Content documents are never fixed, even for markup epubcheck reports under EPUB 3, such as `<center>` and `<a name>`.
- **What it does:**
  - The OPF gets `version="3.0"` and a `dcterms:modified`.
  - Metadata moves to EPUB 3's forms: `opf:role` and `opf:file-as` become `refines` metas, `boeks:series` and `boeks:series_position` become `belongs-to-collection`, `boeks:subtitle` becomes `title-type`, and `<meta name="cover">` also becomes the `cover-image` property.
  - An identifier's `opf:scheme` goes into its value (`urn:isbn:…`), or into an `identifier-type` meta where the value wouldn't read back the same, or it's the unique identifier. The unique identifier's value never changes, since it keys obfuscated fonts.
  - EPUB 3 has one date, so the publication date is kept (or the first date, without one) and the others, such as a conversion date, are removed.
  - A nav document is made from the NCX, with landmarks from the `<guide>`, outside the spine. The NCX and the `<guide>` stay.
  - Manifest properties (`scripted`, `svg`, `mathml`, and `remote-resources`) are found by reading the content documents, and fonts get EPUB 3's media types.
  - Obfuscated fonts are left as they are.
- **Not upgraded:** EPUB 3 files, DRM'd files, files that fail validation, files without an NCX, and fixed-layout files. A file is never upgraded in part.

## 3.6 Covers

An edition's cover is the image its OPF marks: the manifest item with the `cover-image` property in EPUB 3, or the item `<meta name="cover">` names.

- **A new cover is an edit:** a `metadata.updated` whose changes hold `cover`, the image's SHA-256 (§5.3). A log line can't hold an image, so the device making the edit first saves it in the book's `.boeks/covers/`, named by that hash with its type's extension (`<hex>.jpg`). A cover can be replaced or added, not removed.
- **The image has the old cover's type.** Whatever it came as (a pasted PNG, a HEIC photo, a provider's WebP), it's converted as it's saved, to the current cover's media type, or to JPEG when there's no cover. Applying writes it at the old cover's path, which only works for the same type. Image encoders differ, so applying never converts, and the same edit always gives the same bytes.
- **Applying:**
  - **Over a cover,** the cover item's file gets the new image's bytes, at the same path. Content documents still point at it, so a cover page shows the new image, and no text changes.
  - **Without one,** the image is added as `cover.<ext>` next to the OPF, or `cover-2.<ext>` and so on when that's taken, with a manifest item named by `<meta name="cover">`, and also marked `cover-image` in EPUB 3. No cover page is added.
- **The image** is deleted once a `metadata.applied` lists its edit and no pending edit names it. A file that can't be written keeps it for good, and the index shows it as the cover (§4).
- **Covers don't fan out.** Editions often have different covers, so each keeps its own, and the book's is its primary edition's (§2.2).
