Skip to content
Boeks

Appearance

Theme

Files written from the logs

Book.md and each edition's annotations: the plain-text files written from the logs.

6. Files written from the logs

Two files hold the reader's data as plain text, so it can be read and edited in any Markdown or text editor: Book.md for the book (§6.2), and each edition's Edition.md (§6.3). Both are written from the logs, and the logs stay the source of truth. An edit in a file is read back into the logs as the events it stands for. Metadata edits are written into the EPUBs the same way (§3.3).

Everything in these files is editable, except the names that say what's what: frontmatter keys and headings. What can't be edited, such as the title or chapter headings, isn't in the files.

6.1 How they're written and read

Writing:

  • Deterministic. A file's content follows from the logged state, plus what in the existing file is the user's own: keys the format doesn't define, and text outside its headings and entries. The same input gives the same bytes, so two devices writing at once produce identical copies.
  • Who: a device that applies edits (§3.3), with the library writable. Only a file whose content is on disk is written, never a cloud placeholder.
  • How: write a temporary file and rename it over the old one, then log file.applied with the file's name, its hash, and the values written (§5.3). For Book.md, values is {status, rating?, collections?, readthroughs?, review?}, each left out when empty, readthroughs as a list of {id, started?, finished?, stopped?} with the dates as written; for Edition.md, an object by annotation ID of {type, quote, note?}, type being the callout's, quote its text with whitespace collapsed and without a bookmark's …, and note trimmed.
  • No file without content. When there's nothing to write and nothing of the user's to keep, there's no file, and an existing one is removed.

Reading edits. When the library is read, each file is checked for edits:

  1. A file whose hash matches a file.applied is as some device wrote it: nothing changed.
  2. Otherwise compare three versions, field by field: the values of the latest file.applied, the file's, and the logs'. Log each field the file changed as the usual event: status.updated, rating.updated, review.updated, collection.added or collection.removed, readthrough.*, and annotation.*.
  3. A field counts only if the file changed after the logged value did: the file's modification time is later than when the event that set the logged value was made. That's its timestamp, except for this device's own events stamped later than its clock, as while another device's clock is ahead (§5.2): for those, it's this device's own clock time when it made them, which it keeps for itself, outside the library, for as long as a timestamp can be ahead.
  4. Write the file again afterwards, normalized.

Deleting a file clears what it holds, as emptying it would. A missing file counts as deleted only on a device that wrote its latest version, or has read it at that hash since opening the library: a file that hasn't arrived clears nothing. A field changed after the file was written keeps its newer value. Read-throughs and sessions stay: a deleted file isn't a statement that a book was never read. So a book with read-throughs gets its Book.md back, with only them. The device logs file.deleted with the version deleted.

Putting a deleted file back, as from the trash, undoes its deletion: a file whose hash is that of the latest file.deleted brings back Book.md's values as edits, except a field changed since the deletion, and Edition.md's annotations, each with annotation.restored, as they were. A file put back edited is read as an edit, by the rules above.

Edge cases:

CaseRule
Not on disk (a cloud placeholder)Neither read nor written until its content is there.
A sync service's conflict copyRemove it when it's the same as the file or as a version some device wrote (by file.applied). Otherwise it holds the user's text: keep it, flag it, and don't write the file until one of the two is gone.
Not UTF-8, frontmatter that doesn't parse, an invalid value, a review or note over 100,000 characters, or a quote that can't be found (§6.3)Flag it with what's wrong. Log nothing from the file, and don't overwrite it, until it's fixed.
Arrives before the log of the device that wrote itRead as an edit: the reading device logs what's in it, which is the same value, and records the file as written. When the writing device changed the same field again before its log arrived, that second change is undone.
Book folder renamed or movedThe file moves with it.
Edition's folder renamed or moved (§1.3)Edition.md moves with it.

Flagging a file means reporting it to the user and leaving it as it is.

Headings. The body is divided by level-two headings with fixed names, in this order: ## Review, ## Annotations.

  • They're names, like frontmatter keys: written in English, read in any case, and written only when there's something under them, or, for ## Review, when the reader has text above it, which would otherwise be read as the review.
  • Other headings are content: a ## Thoughts inside a review is part of the review.
  • In a file with one of them, text before the first is the user's own, kept as it is.

Encoding: UTF-8 without a byte order mark, with \n line endings. Read \r\n and a byte order mark too.

6.2 Book.md

One per book folder, next to the editions' folders. The name is fixed, like .boeks/. The book's status, rating, collections, and read-throughs are YAML frontmatter; its review is the Markdown body.

---
status: reading
rating: 4
collections: [Favorites, Science fiction]
readthroughs:
  - finished: 2015
  - started: 2019-05
    finished: 2019-05-12
  - started: 2026-11-02
---

## Review

What a book. The *ending*…
KeyLogged asValues
statusstatus.updatedunread, reading, paused, read, or stopped. Missing or empty is unread.
ratingrating.updatedWhole stars, 1–5. Missing or empty clears it.
collectionscollection.added / collection.removedCollection names (§5.4), as a list or one name. A name no book has had is a new collection; a collection renamed in every Book.md it's in is renamed.
readthroughsreadthrough.* (below)The read-throughs, oldest first, each with started, then finished or stopped.
## Reviewreview.updatedThe text under it, with its ends trimmed. A body without the heading is all review; a file without frontmatter is all review.
  • Schema: book.schema.json, for the frontmatter as it's written. Reading is more lenient, as below. Writers add no comments of their own; the keys are documented here and in the schema.
  • Reading: YAML 1.2's core schema, so no stays a string. Read leniently: a status in any case, a rating in quotes, and a single collection name.
  • Writing: status always, then rating, collections, and readthroughs, in that order at the top of the frontmatter, one line each but for readthroughs (below). collections is a flow list, sorted by name as collection names compare (§5.4). A name is written as it is where YAML 1.2 reads it back as that string in a flow list; otherwise it's in double quotes, with \", \\, \n, \t, and other control characters as \uXXXX, in lowercase hex. After the frontmatter come one blank line and the user's own text, then ## Review, a blank line, and the review; the file ends with one \n. Every other line of the existing frontmatter, such as keys and comments the user added, is kept as it is, after them.
  • An edit has the same effect as any other change to the value: setting status: read in the file closes the read-through (§5.3).

Read-throughs (§5.3) are in the file too, so a book's history can be read and corrected in the file, as for a book read before it was imported.

  • An entry has started, then finished or stopped once it has ended. Pauses and the edition aren't in the file, nor is a read-through's time spent reading: it comes from sessions, which aren't either. status: paused says the open read-through is paused.
  • Dates: a recorded moment is written as its day where it happened (§5.3), 2019-05-12; a stated date as it was stated, at its precision, 2019, 2019-05, 2019-05-12, or 2019-05-12 21:40. An entry without started has an unknown start (undated). A date reads with a T or a space before the time.
  • Writing: after collections, as a block list, oldest first by start (a read-through without a start by its end), each key on a line of its own, started first. Left out when there are none.
  • Telling entries apart: entries have no IDs. The file's entries are matched to those in the latest file.applied, which has them: an entry identical to one written is that read-through, wherever it is in the list, and the others are matched in order, one for one, unless the file's entry lacks a start or an end the written one has, which a read-through can't lose. Entries left over in the file are new; those left over in file.applied were removed. A wrong match costs at most which edition and pauses a read-through keeps; every date is what the file says.
EditLogged as
A date changedreadthrough.updated
finished ↔ stoppedreadthrough.reclassified, and readthrough.updated for a changed date
An entry removedreadthrough.deleted
A new entry with an endreadthrough.created, with its dates
A new entry without an end, after every otherA reread: status.updated reading (§5.3), and readthrough.updated with its start, if given
An end added to the open read-throughstatus.updated read or stopped (§5.3), then readthrough.updated with the date
FlaggedAn entry with both finished and stopped, a new open entry before another, or more than one open entry
  • status and readthroughs together: the edits to readthroughs are taken first. A status changed in the same edit has to be what they lead to, or the file is flagged; one left as it was follows them. So status: read alone finishes the open read-through now, and finished: 2026-11-03 added to it finishes it on that day, with or without status: read.

6.3 Edition.md

An edition's highlights, notes, and bookmarks. One per edition, in its folder, next to its file: Ebook (en)/Edition.md. The name is fixed, like Book.md, so it never has to follow a rename.

## Annotations

> [!green]
> He who controls the spice controls the universe.

^01j9xm0c2h7d8f3g4j5k6p7mnb

Herbert means oil, really.

> [!bookmark]
> Arrakis teaches the attitude of the knife…

^01j9xq8b1c4d7e0f3g6h9j2kmn
  • Frontmatter: the format defines no keys. The user's own frontmatter is kept, as in Book.md.
  • Headings: ## Annotations. Reviews are per book (§6.2), so this file has no ## Review.
  • An entry is a callout, in Obsidian's callout syntax (> [!type]), whose type is a color (yellow, green, blue, pink, or purple) or bookmark. Then comes its ID on a line of its own, then its note:
    • The callout's text is the quote. For a highlight, it's the highlighted text. A bookmark marks a point, so it quotes the text that follows the point, as its Locator keeps it (32 characters): when it's the whole 32, it's cut back to before its last space, so no word is cut. It always ends with …, which reading takes off (or ...). The bookmark is where that text starts.
    • The ID is the annotation's, in lowercase, after a ^, with a blank line before and after: a block ID, in Obsidian's syntax. It comes straight after the callout, since a block ID names the block above it.
    • The note is Markdown, like the review, and runs until the next entry or heading of the file's own. It may hold its own quotes, lists, and headings.
    • The motivation follows from the entry (§7): bookmark is bookmarking, and a color is highlighting, with a note or without.
  • An entry is recognized by its callout type, wherever it is in the body. Text that isn't in an entry, before the first one or after a heading of the file's own other than ## Annotations, is the user's own. It's kept as it is, before the annotations.
  • Order: reading order: by the resource's place in the spine (one not in it last), then totalProgression (none counting as 0), then progression, then ID. Annotations that can't be anchored stay where their stored Locator puts them.
  • Layout: the parts are separated by one blank line, and the file ends with one \n: the user's frontmatter, then their own text, then ## Annotations, then each entry, as its callout (> [!type], then > and the quote, if there is one), its ^id, and its note, if there is one. Quotes and notes are written as they are, without escaping: a note that holds a line like > [!green] or ## Annotations reads back as something else.
  • An audiobook's bookmarks have no quote, as audio has none: the callout holds only its type, and they're in the order of their times. One can't be made in the file, as there's nothing to place it by: an entry without an ID is flagged.
  • Not in the file: chapter headings, page numbers, and dates.

Edits to the file are logged as:

EditLogged as
A noteannotation.updated {body}
A callout's colorannotation.updated {color}
A highlight's quoteannotation.updated {target}, anchored at the match of the new quote nearest the old position: adding words extends the highlight, and removing them shortens it
A bookmark's wordsannotation.updated {target}: the bookmark moves to where they start
An entry without an IDannotation.created, anchored by finding its quote in the edition. Its ID is written with the file's next write.
An entry removedannotation.deleted
An entry's type changed between a color and bookmarkFlagged: an annotation doesn't change kind. Removing the entry and adding one without an ID does it.

Finding a quote compares text as Locators count it (§7), and also takes these as equal: ' with U+2018, U+2019, U+201A, U+201B, and the prime U+2032; " with U+201C, U+201D, U+201E, U+201F, and U+2033; - with U+2010 to U+2015 and the minus sign U+2212; and ... with …. Matching is case-sensitive. A changed quote of an existing annotation is looked for in its own resource, and the match nearest where it was taken (the earliest of two as near). Otherwise, as for a new entry, the quote has to be found exactly once in the edition: one found nowhere, or in more than one place, is flagged, and a longer quote fixes the second.

Pairing with the edition: an Edition.md is the edition's whose folder it's in. One that holds another edition's annotation IDs is flagged. Other .md files in an edition's folder are the reader's own.

Reading the file needs the edition's text, to anchor quotes, so only a device that has the text writes it.