Skip to content
Boeks

Appearance

Theme

Books and editions

How a book and its editions relate, and what each of them keeps.

2. Books and editions

  • A book is the work, Dune, whatever its editions. Each book is one folder (§1.3).
  • An edition is one folder in the book's folder, holding its file: the 2019 Ace edition, the Dutch translation, or an audiobook, which can be several files (§9).
  • Translations are editions of the same book, unless they're imported as books of their own.

2.1 Identity

  • The book ID is a ULID, in the first line of the log of the device that imported the book:
    {"timestamp":1760200000123,"device":"mac-studio-4f2","sequence":1,"type":"book.imported","data":{"id":"01j9…"}}
    
  • Every other device's log for the book starts with log.started, which repeats the ID. So every log file names its book, and there is no separate ID file.
  • Two IDs for one book happen when two devices import it into the same folder at once, each logging its own book.imported. A device that finds more than one book ID in a book folder's logs merges them: it keeps the lowest and logs book.merged for the others. IDs are resolved through the book.merged events there are first, so the logs of a book merged by hand (§2.4), now in its folder, aren't merged again, the other way. Importing a file whose hash the library has already is skipped (§2.3), so one file doesn't become two books otherwise.
  • Edition IDs are in edition.added, with the path of the edition's file in the book folder (Ebook/Dune - Frank Herbert.epub) and its content hash.
    • A renamed file is found by its hash: an EPUB that isn't an edition, with the hash of an edition whose file is gone, is that edition. Log an edition.updated with the new name.
    • A file under a name its edition had before is a copy the sync hasn't caught up with. Leave it alone; don't match it back.
    • A file rewritten by applying edits is followed by the hash recorded in metadata.applied (§3.3).
  • The EPUB's unique-identifier is only a hint, never an edition's identity.

2.2 What lives where

LevelState
Bookreading status, read-throughs, rating, review, collections, and reading sessions
Editionprogress and annotations. Every Locator carries its edition's ID (§7).
  • Book.md (§6.2) also holds the status, rating, collections, and review.
  • The edition's Edition.md (§6.3) also holds its annotations.
  • The primary edition gives the book its folder name, cover, and metadata. Its OPF is the book's metadata; there is no other store for it. Each edition keeps its own OPF metadata.
    • It starts as the edition the book was imported with. Adding editions doesn't change it.
    • Making another edition primary is a book.updated.
    • When the primary edition is removed or moved, the oldest remaining edition, by edition.added, takes over. This is derived, not logged.
  • Edits to the book fan out. Any edition can be edited. A change to a book field is also logged, as its own metadata.updated, for every other edition that still had the old value.
    • Book fields: title (and its file-as), subtitle, authors and editors (aut, edt), series, genres, and book identifiers.
    • Only authors and editors are replaced. Translators, narrators, and an edition's other creators stay as they are, after the new authors and editors.
    • "Still had the old value" compares a title, its file-as, and a subtitle with their ends trimmed; the series, genres, and the authors and editors (with creators that have no role) as whole lists, in order; and book identifiers one scheme at a time, so an edition's other identifiers stay.
    • An edition with a value of its own, such as a translation's title or series name, is left alone.
    • Never fanned out: publisher, publication date, language, description, and edition identifiers.
  • Book identifiers are kept in each edition's OPF like any other identifier (§3.1), and fan out: hardcover, wikidata, and inventaire. isbn, openlibrary (an Open Library edition, OL…M), goodreads, asin, google, and doi are edition identifiers. Open Library's work IDs aren't kept: the book already groups its editions, and an edition's record names its work.
  • Switching editions: progress differs between editions. A Locator can be carried to a reissue of the same text by its text quote (§7); a translation shares only its chapters.

2.3 Import

  • A file whose content hash is already in the library is skipped. That's the hash of an edition's file now, or of any file it had before, as edition.added, edition.updated, and each metadata.applied record them, so an original is still recognized after edits or transforms rewrote it.
  • The library's import transforms run as the file comes in (§3.4).
  • A file that matches an existing book, by identifier or by title and author, can be added as an edition, replace an edition, become a new book, or be skipped.
  • Replacing an edition logs edition.updated, with the new file's name and hash. The edition keeps its logs, and its annotations are found in the new file by their quotes (§7). One whose quote isn't there is kept in the logs, never dropped.
  • Merging books moves editions between folders, and is logged as edition.moved and book.merged (§2.4). Splitting an edition off into a book of its own isn't built yet.

Reading for a book already here

Importing from another app, a book the library has already gets the source's reading, rather than a second copy or being left out. It's the book by its file's hash, or the one book it looks like (§2.3), by the source's details or its file's; one that looks like several is left out, as nobody's there to say which.

  • It's listed before importing, and one the user says is another book is left out.
  • No file and no details come over. Its own stay.
  • Its edition gets the source's ID for it, in an edition.updated with only imported. So importing again skips it, and a Kobo's copies from the source are known by it (§17).
  • The status is the source's only where the book has none or is unread: an import never undoes reading here. Each read-through comes too, unless the book has one that ended the same day; one still open only with its status.
  • Highlights and notes are found by their text in the book's EPUB edition (the same file's, else the primary edition if an EPUB, else the first EPUB), near where the source had them, and one on the same text there already isn't added again. Progress comes only when the source read there later, by its at, than the edition's own progress: an import never moves reading back.
  • A rating fills in only where there's none, and collections are added to.
  • Audio reading (a place, bookmarks, listening sessions) comes only into an edition with the very same audio, by hash: a time in one recording isn't a place in another. A listening session or bookmark already there isn't added again.

Importing a Calibre library

A Calibre library's folder, with metadata.db at its root, is imported as a whole: its books, with what Calibre knows of them. The Calibre library is only read, never written.

  • Each Calibre book becomes a book with an edition for its EPUB and one for its audiobook (an M4B, M4A, or MP3 format). The EPUB comes first and is the primary edition. A book with neither stays in Calibre.
  • A book imported before is skipped, so importing again adds only what's new: one an edition names by its Calibre ID, or the identifier calibre (as Calibre writes it into the files it makes). One whose file the library has, or that looks like a book here by Calibre's details or its file's, brings its reading into that book (above).
  • Calibre's details win over the file's. Where they differ, they're logged as one metadata.updated for the primary edition, which fans out (§2.2): title, a subtitle from a text column named #subtitle, authors (other credits stay), the series, identifiers with the book's Calibre ID, language, publisher, publication date, and description. Tags are genres, as Calibre writes them into files (dc:subject); a book without tags keeps the file's. The cover is Calibre's when the edition has none, or one of another size: Calibre keeps its own re-encoded copy, so the bytes rarely match.
  • Ratings are Calibre's 0 to 10, two per star. Half stars are rounded up.
  • Custom columns are found by their names unless the user chooses others; the Kobo Utilities plugin's (percent read, last read, location, and time spent reading) are those its settings name (namespaced:KoboUtilitiesPlugin:settings in preferences). A column has one role at most.
  • Read status comes from custom columns: a column of values (#status), a yes-or-no column (#read), how far a book was read in percent (#percent_read, as the Kobo Utilities plugin keeps it), and a date it was finished (#date_read), in that order. The values column's values are matched in English (Finished, DNF, To read), and the user says what others mean, or they're left. A book at 100% is read, and one started is being read. A date column with "last" in its name, such as Kobo Utilities' #last_read, is when a book was last opened: it dates the reading, but never makes a book read.
  • The book is imported with its status: edition.added is followed by its own status.updated, not unread first. Its read-through's start is undated (§5): Calibre doesn't know it. A read book's read-through starts and ends when it was finished, by its date read (as a day, with readthrough.updated) or else when it was last opened; one with neither has no read-through, only its status. A book being read opens one when it was last opened.
  • Highlights are Calibre's viewer's, from its annotations table, for the EPUB. Each is found in the edition by its text, in its chapter near where Calibre's CFI points, so the same words elsewhere aren't taken for it, and logged as annotation.created with Calibre's time and its CFI kept as x-calibre-cfi. One whose text isn't there is left out. Calibre's red is pink. Bookmarks aren't imported: Calibre places them by CFI alone.
  • Where it was last read, unless it was read to the end, becomes a progress.updated with at: from Calibre's viewer (last_read_positions, a fraction of the book), or from the Kobo (#reading_location, a chapter, with #percent_read), whichever is later. Time spent reading (#time_spent_reading, in seconds) becomes a session.created total that ends when the book was last opened.
  • Calibre's ID for the book goes into each edition's edition.added too, as imported (§5.3).
  • Everything is logged by the importing device, after edition.added. What fails after the files are copied in is reported for the book, and the rest still comes over.

Importing from an Audiobookshelf server

An Audiobookshelf server's books are imported over its API, as the account signed in sees them: with an API key, or a username and password (a login's access token is refreshed when it runs out). The server is only read: every request is a GET, but signing in.

  • Each item in a library of books becomes a book with an edition for its EPUB and one for its audio: one file, or several MP3s as an audiobook in several files (§9), in the server's order. The EPUB comes first and is the primary edition. Items in libraries of podcasts, and items with neither (only a PDF, say), stay on the server. The user picks which libraries.
  • Files are downloaded as they are on the server (/api/items/:id/file/:ino/download), which needs the account's download permission: without it nothing is imported, and the preview says so.
  • An item imported before is skipped: one an edition names by the item's ID, in its imported. One whose EPUB or audio the library has, or that looks like a book here by the server's title, authors, ISBN, and ASIN or its file's, brings its reading into that book (above).
  • The server's details win over the files', as metadata.updated for each edition: title, subtitle, authors, the first series and its number, genres, publication date, publisher, description, ISBN, ASIN, and a language given as a code. Narrators go to the audiobook's edition only. The cover is the server's when the edition has none, or one of another size.
  • The chapters are the server's where they differ from the audio's, by title or by more than half a second: they're written into the downloaded files before importing, as chapters are the file's own (§9). An audiobook in several files gives each file its part of them, and a file that's one chapter only its title.
  • The book is imported with its status: finished items are read, items listened to or read into are being read. Audiobookshelf knows when an item was started and finished, so the read-through has both as at; one marked finished without being started has an undated start (§5).
  • Where it was last listened to becomes a progress.updated with at in the audiobook's edition, in the file playing at that time; where it was last read, a progress.updated in the EPUB where the server's CFI points (ebookLocation), or at its fraction of the book when the CFI isn't found in it.
  • Bookmarks are annotation.created bookmarking in the audiobook's edition, with their titles as notes. Listening sessions are each a session.created in it, from when it started for as long as it was listened.
  • Collections and playlists become collections of the same name, as do the item's tags. A playlist's podcast episodes are left out.
  • Downloading isn't importing: an app downloads an item without holding the library, then imports what it downloaded.

2.4 Merging books

Two books can be merged by hand, as a translation that matched nothing on import, or editions imported as separate books. One book (from) goes into the other (into):

  • from's editions move into into's folder, each with its folder and all that's in it, Edition.md included (§6.3), keeping their IDs. Folder names are qualified again for the editions now there (§1.3). Each move is an edition.moved in from's log, followed by an edition.updated if it was renamed.
  • book.merged goes in the library log. From then on from's ID resolves to into.
  • from's logs follow its editions: its folder, with no editions left, is an orphaned .boeks/, moved into into's like any other (§13). Read together, edition.moved to a book whose logs are read with it leaves the edition where it is. from's Book.md is written from the logs, so it goes.
  • Folded together, the logs merge as usual (§5.4): progress, read-throughs, sessions, and annotations are kept, by edition and by ID, and so are cover images (§3.6).
  • Book fields don't simply go to the newest, since from imported later would make the book unread. The merging device logs, in into's log, what differs from what the merged logs say:
    • the primary edition: into's
    • the status: into's, unless it's unread, then from's. As a correction, with no read-through events.
    • the rating: into's, else from's
    • the review: both, into's first and a blank line between, or whichever there is
    • the collections: those of both
  • A device that hasn't seen the merge may still log to from's folder. Its log arrives as an orphaned .boeks/ and joins into's like the rest, newest wins as always.

2.5 Removing and deleting

What goes is moved to the system's trash, so it can be put back: the Trash on macOS, iCloud Drive's Recently Deleted on iOS, the Recycle Bin on Windows, and the freedesktop.org trash on Linux. The Apple app and the CLI (boeks delete, boeks remove) both do. Where there's none, as on a drive without a trash, nothing is deleted for good without asking: the Apple app asks whether to delete permanently, and the CLI stops unless told to with --permanently. Either way, the deletion is logged only once the files are gone.

  • Removing an edition logs edition.removed and moves its folder, with its file and Edition.md, to the trash. Its progress, sessions, and annotations stay in the book's logs: the timeline still shows them, but the annotations aren't listed with the book's, as they can't be opened. A primary edition removed hands over to the oldest one left (§2.2). A Kobo drops its kepub at its next sync (§17.2).
  • A book's only edition isn't removed: it's the book, so the book is deleted instead.
  • Gone from the library folder, as when deleted in a file browser, but also when left out of a sync or on a drive that isn't connected, which look the same: a device that had a book's or edition's files and finds them gone reports it as no longer in the library folder, and logs nothing. Only the reader says which it was: deleting it there (in the app, or with boeks delete --missing) logs book.deleted or edition.removed, as deleting in the app does. Files that moved within the library aren't gone: a book is found by its ID wherever its folder is, and an edition by its hash in its book's folder. A device that never had the files reports nothing, as they may not have arrived on it yet; which files a device has had, and what it reported, it keeps for itself, outside the library. Files that come back are no longer reported.
  • A removed edition put back where it was, as from the trash, is the edition again: a device that finds a file or folder at a removed edition's path, with its hash, logs edition.added again with the edition's ID, and its reading and Edition.md belong to it as before. Its file imported anew instead is a new edition: a removed edition's hash no longer counts as in the library (§2.3).
  • Deleting a book logs book.deleted in the library log and moves the book's folder, with every device's logs for it, to the trash. A device that logs to the book later, before it has seen the deletion, makes a folder with only .boeks/ in it, which is removed as an orphan (§13). A Kobo drops the book's kepubs and its folder at its next sync.
  • A deleted book whose files are back, as when it's put back from the trash with its logs, is the book again: a device that finds a deleted book's folder with its files logs book.restored in the library log, which undoes the book.deleted before it. Without it, the book would still count as deleted, and its logs would be taken for an orphan's if its files went again.