Files written from the logs
Book.md and each edition's annotations: the plain-text files written from the logs.
6. Files written from the logs
Two files hold the reader's data as plain text, so it can be read and edited in any Markdown or text editor: Book.md for the book (§6.2), and each edition's Edition.md (§6.3). Both are written from the logs, and the logs stay the source of truth. An edit in a file is read back into the logs as the events it stands for. Metadata edits are written into the EPUBs the same way (§3.3).
Everything in these files is editable, except the names that say what's what: frontmatter keys and headings. What can't be edited, such as the title or chapter headings, isn't in the files.
6.1 How they're written and read
Writing:
- Deterministic. A file's content follows from the logged state, plus what in the existing file is the user's own: keys the format doesn't define, and text outside its headings and entries. The same input gives the same bytes, so two devices writing at once produce identical copies.
- Who: a device that applies edits (§3.3), with the library writable. Only a file whose content is on disk is written, never a cloud placeholder.
- How: write a temporary file and rename it over the old one, then log
file.appliedwith the file's name, its hash, and the values written (§5.3). ForBook.md,valuesis{status, rating?, collections?, readthroughs?, review?}, each left out when empty,readthroughsas a list of{id, started?, finished?, stopped?}with the dates as written; forEdition.md, an object by annotation ID of{type, quote, note?},typebeing the callout's,quoteits text with whitespace collapsed and without a bookmark's…, andnotetrimmed. - No file without content. When there's nothing to write and nothing of the user's to keep, there's no file, and an existing one is removed.
Reading edits. When the library is read, each file is checked for edits:
- A file whose hash matches a
file.appliedis as some device wrote it: nothing changed. - Otherwise compare three versions, field by field: the values of the latest
file.applied, the file's, and the logs'. Log each field the file changed as the usual event:status.updated,rating.updated,review.updated,collection.addedorcollection.removed,readthrough.*, andannotation.*. - A field counts only if the file changed after the logged value did: the file's modification time is later than when the event that set the logged value was made. That's its timestamp, except for this device's own events stamped later than its clock, as while another device's clock is ahead (§5.2): for those, it's this device's own clock time when it made them, which it keeps for itself, outside the library, for as long as a timestamp can be ahead.
- Write the file again afterwards, normalized.
Deleting a file clears what it holds, as emptying it would. A missing file counts as deleted only on a device that wrote its latest version, or has read it at that hash since opening the library: a file that hasn't arrived clears nothing. A field changed after the file was written keeps its newer value. Read-throughs and sessions stay: a deleted file isn't a statement that a book was never read. So a book with read-throughs gets its Book.md back, with only them. The device logs file.deleted with the version deleted.
Putting a deleted file back, as from the trash, undoes its deletion: a file whose hash is that of the latest file.deleted brings back Book.md's values as edits, except a field changed since the deletion, and Edition.md's annotations, each with annotation.restored, as they were. A file put back edited is read as an edit, by the rules above.
Edge cases:
| Case | Rule |
|---|---|
| Not on disk (a cloud placeholder) | Neither read nor written until its content is there. |
| A sync service's conflict copy | Remove it when it's the same as the file or as a version some device wrote (by file.applied). Otherwise it holds the user's text: keep it, flag it, and don't write the file until one of the two is gone. |
| Not UTF-8, frontmatter that doesn't parse, an invalid value, a review or note over 100,000 characters, or a quote that can't be found (§6.3) | Flag it with what's wrong. Log nothing from the file, and don't overwrite it, until it's fixed. |
| Arrives before the log of the device that wrote it | Read as an edit: the reading device logs what's in it, which is the same value, and records the file as written. When the writing device changed the same field again before its log arrived, that second change is undone. |
| Book folder renamed or moved | The file moves with it. |
| Edition's folder renamed or moved (§1.3) | Edition.md moves with it. |
Flagging a file means reporting it to the user and leaving it as it is.
Headings. The body is divided by level-two headings with fixed names, in this order: ## Review, ## Annotations.
- They're names, like frontmatter keys: written in English, read in any case, and written only when there's something under them, or, for
## Review, when the reader has text above it, which would otherwise be read as the review. - Other headings are content: a
## Thoughtsinside a review is part of the review. - In a file with one of them, text before the first is the user's own, kept as it is.
Encoding: UTF-8 without a byte order mark, with \n line endings. Read \r\n and a byte order mark too.
6.2 Book.md
One per book folder, next to the editions' folders. The name is fixed, like .boeks/. The book's status, rating, collections, and read-throughs are YAML frontmatter; its review is the Markdown body.
---
status: reading
rating: 4
collections: [Favorites, Science fiction]
readthroughs:
- finished: 2015
- started: 2019-05
finished: 2019-05-12
- started: 2026-11-02
---
## Review
What a book. The *ending*…
| Key | Logged as | Values |
|---|---|---|
status | status.updated | unread, reading, paused, read, or stopped. Missing or empty is unread. |
rating | rating.updated | Whole stars, 1–5. Missing or empty clears it. |
collections | collection.added / collection.removed | Collection names (§5.4), as a list or one name. A name no book has had is a new collection; a collection renamed in every Book.md it's in is renamed. |
readthroughs | readthrough.* (below) | The read-throughs, oldest first, each with started, then finished or stopped. |
## Review | review.updated | The text under it, with its ends trimmed. A body without the heading is all review; a file without frontmatter is all review. |
- Schema:
book.schema.json, for the frontmatter as it's written. Reading is more lenient, as below. Writers add no comments of their own; the keys are documented here and in the schema. - Reading: YAML 1.2's core schema, so
nostays a string. Read leniently: a status in any case, a rating in quotes, and a single collection name. - Writing:
statusalways, thenrating,collections, andreadthroughs, in that order at the top of the frontmatter, one line each but forreadthroughs(below).collectionsis a flow list, sorted by name as collection names compare (§5.4). A name is written as it is where YAML 1.2 reads it back as that string in a flow list; otherwise it's in double quotes, with\",\\,\n,\t, and other control characters as\uXXXX, in lowercase hex. After the frontmatter come one blank line and the user's own text, then## Review, a blank line, and the review; the file ends with one\n. Every other line of the existing frontmatter, such as keys and comments the user added, is kept as it is, after them. - An edit has the same effect as any other change to the value: setting
status: readin the file closes the read-through (§5.3).
Read-throughs (§5.3) are in the file too, so a book's history can be read and corrected in the file, as for a book read before it was imported.
- An entry has
started, thenfinishedorstoppedonce it has ended. Pauses and the edition aren't in the file, nor is a read-through's time spent reading: it comes from sessions, which aren't either.status: pausedsays the open read-through is paused. - Dates: a recorded moment is written as its day where it happened (§5.3),
2019-05-12; a stated date as it was stated, at its precision,2019,2019-05,2019-05-12, or2019-05-12 21:40. An entry withoutstartedhas an unknown start (undated). A date reads with aTor a space before the time. - Writing: after
collections, as a block list, oldest first by start (a read-through without a start by its end), each key on a line of its own,startedfirst. Left out when there are none. - Telling entries apart: entries have no IDs. The file's entries are matched to those in the latest
file.applied, which has them: an entry identical to one written is that read-through, wherever it is in the list, and the others are matched in order, one for one, unless the file's entry lacks a start or an end the written one has, which a read-through can't lose. Entries left over in the file are new; those left over infile.appliedwere removed. A wrong match costs at most which edition and pauses a read-through keeps; every date is what the file says.
| Edit | Logged as |
|---|---|
| A date changed | readthrough.updated |
finished ↔ stopped | readthrough.reclassified, and readthrough.updated for a changed date |
| An entry removed | readthrough.deleted |
| A new entry with an end | readthrough.created, with its dates |
| A new entry without an end, after every other | A reread: status.updated reading (§5.3), and readthrough.updated with its start, if given |
| An end added to the open read-through | status.updated read or stopped (§5.3), then readthrough.updated with the date |
| Flagged | An entry with both finished and stopped, a new open entry before another, or more than one open entry |
statusandreadthroughstogether: the edits toreadthroughsare taken first. Astatuschanged in the same edit has to be what they lead to, or the file is flagged; one left as it was follows them. Sostatus: readalone finishes the open read-through now, andfinished: 2026-11-03added to it finishes it on that day, with or withoutstatus: read.
6.3 Edition.md
An edition's highlights, notes, and bookmarks. One per edition, in its folder, next to its file: Ebook (en)/Edition.md. The name is fixed, like Book.md, so it never has to follow a rename.
## Annotations
> [!green]
> He who controls the spice controls the universe.
^01j9xm0c2h7d8f3g4j5k6p7mnb
Herbert means oil, really.
> [!bookmark]
> Arrakis teaches the attitude of the knife…
^01j9xq8b1c4d7e0f3g6h9j2kmn
- Frontmatter: the format defines no keys. The user's own frontmatter is kept, as in
Book.md. - Headings:
## Annotations. Reviews are per book (§6.2), so this file has no## Review. - An entry is a callout, in Obsidian's callout syntax (
> [!type]), whose type is a color (yellow,green,blue,pink, orpurple) orbookmark. Then comes its ID on a line of its own, then its note:- The callout's text is the quote. For a highlight, it's the highlighted text. A bookmark marks a point, so it quotes the text that follows the point, as its Locator keeps it (32 characters): when it's the whole 32, it's cut back to before its last space, so no word is cut. It always ends with
…, which reading takes off (or...). The bookmark is where that text starts. - The ID is the annotation's, in lowercase, after a
^, with a blank line before and after: a block ID, in Obsidian's syntax. It comes straight after the callout, since a block ID names the block above it. - The note is Markdown, like the review, and runs until the next entry or heading of the file's own. It may hold its own quotes, lists, and headings.
- The motivation follows from the entry (§7):
bookmarkis bookmarking, and a color is highlighting, with a note or without.
- The callout's text is the quote. For a highlight, it's the highlighted text. A bookmark marks a point, so it quotes the text that follows the point, as its Locator keeps it (32 characters): when it's the whole 32, it's cut back to before its last space, so no word is cut. It always ends with
- An entry is recognized by its callout type, wherever it is in the body. Text that isn't in an entry, before the first one or after a heading of the file's own other than
## Annotations, is the user's own. It's kept as it is, before the annotations. - Order: reading order: by the resource's place in the spine (one not in it last), then
totalProgression(none counting as 0), thenprogression, then ID. Annotations that can't be anchored stay where their stored Locator puts them. - Layout: the parts are separated by one blank line, and the file ends with one
\n: the user's frontmatter, then their own text, then## Annotations, then each entry, as its callout (> [!type], then>and the quote, if there is one), its^id, and its note, if there is one. Quotes and notes are written as they are, without escaping: a note that holds a line like> [!green]or## Annotationsreads back as something else. - An audiobook's bookmarks have no quote, as audio has none: the callout holds only its type, and they're in the order of their times. One can't be made in the file, as there's nothing to place it by: an entry without an ID is flagged.
- Not in the file: chapter headings, page numbers, and dates.
Edits to the file are logged as:
| Edit | Logged as |
|---|---|
| A note | annotation.updated {body} |
| A callout's color | annotation.updated {color} |
| A highlight's quote | annotation.updated {target}, anchored at the match of the new quote nearest the old position: adding words extends the highlight, and removing them shortens it |
| A bookmark's words | annotation.updated {target}: the bookmark moves to where they start |
| An entry without an ID | annotation.created, anchored by finding its quote in the edition. Its ID is written with the file's next write. |
| An entry removed | annotation.deleted |
An entry's type changed between a color and bookmark | Flagged: an annotation doesn't change kind. Removing the entry and adding one without an ID does it. |
Finding a quote compares text as Locators count it (§7), and also takes these as equal: ' with U+2018, U+2019, U+201A, U+201B, and the prime U+2032; " with U+201C, U+201D, U+201E, U+201F, and U+2033; - with U+2010 to U+2015 and the minus sign U+2212; and ... with …. Matching is case-sensitive. A changed quote of an existing annotation is looked for in its own resource, and the match nearest where it was taken (the earliest of two as near). Otherwise, as for a new entry, the quote has to be found exactly once in the edition: one found nowhere, or in more than one place, is flagged, and a longer quote fixes the second.
Pairing with the edition: an Edition.md is the edition's whose folder it's in. One that holds another edition's annotation IDs is flagged. Other .md files in an edition's folder are the reader's own.
Reading the file needs the edition's text, to anchor quotes, so only a device that has the text writes it.