I’m willing to join a WG for this and also like to offer myself as co-chair. The project (and product) I’ve been working on is named Dracome. I’ve been actively preparing its GoLive.
When I started on the project, I spent quite some time learning about BookHive’s data model and lexicons. I’ll build an adapter for them anyway so Dracome can consume them. I chose an independent set of lexicons for Dracome to retain flexibility, some even differ from bsky’s. That’s why I’ll need to build some adapters anyway.
A major part of Dracome is interoperability, which is why I spent considerable time iterating on the lexicons to find a good balance between flexibility and robust typing. Feedback from @jasonbu.online has been invaluable for some decisions.
The discussion around the content type of reviews led me to decide to include this before I publish the lexicons. They’ll published under dracome.social. I’ll start with text/plain.
So whatever common lexicon structure we may end up with, I’d gladly build an adapter for that as well. I’ll probably keep relying Dracome’s own set of lexicons to preserve flexibility, but I’d follow the shared lexicon structure closely.
Dracome is a priority project to me and I’ll keep spending a lot of time on in the foreseeable future. So I’m ready to spend some time on a WG. I’m eager expand my knowledge of the technical side of the world of books as much as possible.
When I started, I used BookHive’s dump as a starting point, but I’ve since moved on to Open Library and the Deutsche Nationalbibliothek / DNB (German National Library). So a future lexicon structure will need to support a solid edition structure.
Dacome is edition-first - including reviews - to enable users to post a German review for a German edition and an English review for an English edition. But the community review section aggregates them by work. That was the general decision I made with Dracome’s lexicons: keeping the data model flexible enough to expand to more use cases, even if the AppView itself doesn’t use it to the full extent.
If Dracome needs to consume a new catalog entry (work) that isn’t edition-specific, it automatically creates a single edition for it; hence “edition-first”. This is necessary to consume data from apps like BookHive that aren’t edition-scoped. That’s why Dracome’s lexicons declare an edition as optional, even though Dracome internally requires one. A missing edition is considered a data bug.
I think a common set of lexicons would need to handle this similairily. Otherwise it would be too restrictive; not every app wants or can be edition-specific.
I’m generally interested in cooperating with other apps and having them feed catalog data into Dracome. However, this is gated by a human review, which is why I drafted a submission process that I discussed with @jasonbu.online.
Unfortunately I don’t have as much time to commit to this as I’d like, but I’m building PassingReads.org (physical book sharing), and working on a PR for AudioBookShelf to post (audio|e)book progress to the atmosphere. I’d love to get both aligned with where you all decide to head!
I’ll definitely be joining the forum area you folks build and helping out where I can
Yep, I agree that editions is very useful to track progress accurately. Though, I will note that reading progress is typically only for the user themselves and not so relevant across readers and for statistics (e.g. reading time of a book). For BookHive, I modeled this as several optional fields, and if the user marked the pagesRead / totalPages, I was able to derive a percentage, or the user could just set the percentage directly. Same for chaptersRead / totalChapters with priority to pages for granularity. For audiobooks it would be easy to model seconds / totalSeconds with UI that makes that easier to manage. But the most important field was percent, so that it was easily understood across clients.
As far as I understand, the difference in private data is between the access control of who owns it, and it’s path, but otherwise should act as the same record type, so I’m not sure that it would be useful to indicate this on the record itself, given that the identifier (at uri vs. ats uri) for the record could already derive this data.
I totally agree on the needs for the separation of a work & an edition, I think it’s going to be some smart modeling to get that to work well together in a way that can be mapped across applications (and potentially datasets). Very happy to have someone on board that has seen the limitations around these models.
Just to be clear, I’m the only one running BookHive.
I’ve heard a lot of talk from people around standard.site being a good model for this, but I think it fundamentally is a differently shaped problem. standard.site is about modeling data that the original publisher wants to broadcast on the network (aka a single canonical, this was published), but the main problem for books is that we cannot have the original publisher manage the record of a book in all cases. Users are broadcasting their reviews/comments & other data about a book, and the trouble is knowing which book they precisely meant to broadcast about, especially when different applications are using different datasets which very much do not agree.
The tension here is to do with centralization vs decentralization of book data. In a centralized system, like Goodreads, there is not contention about user generated data, because you have full access to both the user’s data & the original book record; with the ability to perform migrations of them both in lock-step. So, if you want to merge book editions, split them, normalize them, change covers, you are absolutely free to do so in a centralized system.
We do not have this luxury in a decentralized system; and, to add insult to injury, there won’t be agreement between systems on what the book records even mean because everyone is trying to approximate real-world messy data, not just tidy URLs.
With books, we have some fundamental problems to overcome:
Data Source
How do we build a system which can bridge across multiple datasets?
Or, should we be aiming to have a single canonical dataset which we can contribute over time to enrich?
Model Shape
What is the minimal viable representation of a book, a review, progress, etc?
How should books, works, editions be referenced across the system to enable new apps & systems?
Evolution
How do books get added over time, can authors self-publish, how do we moderate?
This is an interesting approach @xardex.dev, though I wonder if this is actually better modeled as a translation of a review & the review is surfaced across the work. You’ll be surprised to how many editions a book may have, so it becomes increasingly unlikely that an edition-specific review would be surfaced, leading to users shouting into the void. Where each review could be marked as being tied to an edition, but then a separate schema joins across the work.
That’s true. However, I do think it’s manageable, though not 100% bulletproof. I chose to build Dracome’s lexicons around a WorkTarget (bag of identifiers) and a CatalogSnapshot. I don’t publish Dracome’s entire catalog, because this would be way too much data. But all logged progress, every state, review, marker is tied to a snapshot that is refreshed on save. The downside of this approach is bigger records.
I don’t think we need to agree on a catalog. I don’t think we even should. So the challenge is to make them connectable. A bag of identifiers is one way to solve this, because in Dracome’s case, it includes ISBNs, Open Library keys, Wikidata keys etc. The snapshot itself is a way to make sense of these keys without having to do lookups, and to ensure a record still makes sense if there aren’t any external keys.
I’ll enrich those datasets with HiveIds as well, so it’s easier to connect them to BookHive. Though, I’ll have to build some matching logic for that.
I won’t be surprised because I’ve been testing Dracome with a work that has over 1200 editions (sourced from Open Library). I’ve seen many works with many editions.
I initially thought about solving this as translations, but that’s is too limiting. One may love a book in English but despise its translation. So a simple translation model may be bad UX. If there are multiple translations in the same language, this becomes even worse. Maybe a new translation is great; only the original one is despised.
I hinted at the UX solution in my post. All pages and components are edition-scoped, including search results. If a user hasn’t chosen an edition, Dracome tries to find a good representative one. So users are used to the edition-scoped UX.
I fully agree with you that surfacing them only for editions would be like shouting into the void. That’s why the community review section aggregates them across all editions by work. I haven’t added a visual pointer to the edition the review was posted for yet, though.
I’ve reviewed multiple book apps and websites to see how they handle edge cases and on what UX they landed on.
I can’t really tell what you mean by this exactly from this description alone. Would you mind explaining further?
Sure, but how does BookHive, or another app contribute to this bag of identifiers? Where does the identifiers live?
You might be able to do this with BookHive given my dataset is public, but another application which is not, will be impossible to do.
This will be very hard, I would even say firmly, it should not be done. There will always be mismatches between datasets, and it will be an MxN problem. Exponentially hard when you account for other providers’ datasets.
I think that the local optimum solution here is for the user to have the power to make this mapping on their own case-by-case basis. Like, for example, given a book record in their PDS like:
title: "ABC", authors: ["XYZ"], identifiers: [{ type: 'hiveId', src: 'bookhive.buzz', id: 'h_123'}]
And an XRPC method, implemented across services, to search for books like: id: 'community.book.search', type: 'query', params: { q: { type: 'string' }, output: { title: string, authors: string[], cover: string, identifiers: [{type: string, id: string}] } }
then each application would be able to make a book matching UI for migrating books across services (if multiple services implement the same XRPC method shape). While also giving the user agency in 1) how to map that book (since the record may not be valid in one dataset or inaccurate in another) 2) freedom for independent services for migrating between accounts (similar to those spotify < - > apple music migrators).
This would move the burden of mapping between datasets on the user & if an application wanted to have a more friction-free experience it is up to them to implement that if they so choose to. Applications can even build up datasets of previously mapped books to offer as suggestions.
I can’t speak to how often such a thing would happen, but given the complexity of this as is, I’d lean towards reducing scope than trying to account for every situation. But, “to each, their own”.
First, apologies for the BookHive mix-up, @nickthesick, all clear now.
And yes, I think your point about Standard.site is important.
I’m increasingly thinking of Standard.site as a useful model for how we organize the work, not necessarily what the resulting model should look like.
Your three buckets feel like a very good way to structure the work:
Data source / identity
Model shape
Evolution and moderation
This is exactly the diversity of implementations I was hoping we’d figure out before trying to converge on anything.
We now have BookHive, Dracome, Bibliograph/Livtet and Byline using ATProto, PassingReads/AudioBookShelf work underway, and others looking to participate, with deliberately different assumptions around works, editions, progress and data sources.
I also really like Nick’s suggestion of pushing some of the dataset mapping back to the user, with a shared search/XRPC shape across services.
That feels very AT Protocol to me, don’t assume there is one canonical book database everybody has to agree on. Give applications a common way to discover candidate matches, let the user confirm the mapping, and allow services to build smarter reconciliation on top if they want to.
That could also make migration between book services much more practical without requiring every AppView to share the same underlying dataset.
And I think that raises a bigger question for the group:
Are we standardising book records, or are we standardising the records and service interfaces that allow independent book systems to interoperate? Both?
I’m thinking that’s the sort of thing we should start capturing as a design proposal rather than trying to settle it immediately in this thread.
At this point I think we should put a lightweight process around the work, capture the questions, competing approaches and open decisions somewhere shared, then turn them into a working backlog.
The first questions almost write themselves:
Identity / data sources: how do independent datasets refer to the same real-world work or edition?
Work vs Edition: what needs to be shared, and what can remain AppView-specific?
User records: reviews, status and progress, including what belongs at work vs edition level.
Evolution: additions, corrections, merges, moderation and distributed curation.
Protocol concerns: private records, references, schema evolution and extension.
Service interfaces: which XRPC methods, if any, should be common across book services?
Rather than settle those here one by one, I think we should get the initial group established and turn them into the first set of issues and working sessions and a process going.
Among other things, Dracome stores a full snapshot along each of these records:
"catalogSnapshot": {
"type": "object",
"description": "A portable, identity-oriented snapshot of combined work-and-edition catalog metadata, captured at record write time. Alongside the record's target, it lets any AT Protocol consumer independently resolve this work or edition without depending on Dracome as a central catalog authority. Work-scoped fields are always present at the top level; edition-scoped fields are grouped under the optional nested `edition` sub-object, present only when the record targets a specific edition.",
"required": ["workType", "title", "contributors"],
"properties": {
"workType": {
"type": "ref",
"ref": "#workType",
"description": "Structural content form of the referenced work (novel, poetry, comic, manga, audio_drama, or non_fiction)."
},
"title": {
"type": "string",
"maxLength": 1024,
"description": "Primary title of the work, sourced from the work's own canonical title."
},
"subtitle": {
"type": "string",
"maxLength": 1024,
"description": "Secondary title of the work, present only when the work has one, sourced from the work's own canonical subtitle."
},
"contributors": {
"type": "array",
"minLength": 1,
"maxLength": 16,
"description": "Ordered list of contributors (authors, translators, illustrators, etc.) credited on the work or edition.",
"items": {
"type": "ref",
"ref": "#snapshotContributor"
}
},
It bloats records, but decouples them from Dracome’s catalog or its PDS. So any app can pick them up and do whatever they wish with them. I refer to this as the identity of a book. A federation concept I drafted is similar to your suggested XRPC methods: a re-implementable API surface for enriching such records (with covers, descriptions, etc.). They can go stale by design; I don’t consider this an issue because the identity of a book doesn’t change even if the catalog changes.
I do welcome apps contributing identifiers to Dracome. It’s already part of a submission process I discussed with @jasonbu.online. If an app contributes an edition, I import their identifier with it.
However, I don’t think it’s the path we should take. In my federation concept, I try to avoid giving Dracome any catalog authority in the Atmosphere. It should be one source of many. That’s why Dracome is persisting those snapshots with the full identifier bag it knows. However, I am willing to invest in or reuse my matching machinery for it.
My preferred way to handle this is to rely on independent sources for this mapping, which we can contribute to. Since Dracome’s catalog is heavily based on Open Library, I intend to contact them to ask if they’re interested in a cooperation where I feed catalog data back to them. That would be one way to enrich their catalog. They already have an identifier bag as well (e.g. IDs from Goodreads), so it’s a good starting point. But Wikidata would be another option as well. It’s unlikely that they want to have all the keys of atproto apps, so some decentralized, PDS-based solution might also be an option.
It’s not a silver bullet, but with ISBNs, OL keys, etc. we might be able to connect datasets safely enough. However, this puts the burden on each app to connect them to these IDs.
Dracome already has a heavy import and matching tier machinery in place since I took the challenge of merging two independent catalogs (OL and DNB), which was no easy task and is unlikely to be perfect. It has several matching tiers, a review queue and some safety guards. If I want to enable users to import their data from Goodreads, The StoryGraph, BookHive or any other app, I’ll have to implement matching anyway. This is just precomputing this matching. It won’t be perfect, but should be good enough.
To be clear, this is a different thing from what I said above: matching a user’s own import data into Dracome’s catalog is Dracome acting as a hub for its own imports, not Dracome becoming the ecosystem’s universal cross-app registry. It stays O(N) against the sources Dracome imports from, not O(N²) across every provider pair.
I do already plan to help users review the import result. However, you made a good point. Maybe I should think about it more in a staging-way where they can confirm whether Dracome did the matching right, and revisit/correct it later if something slips through. I don’t think it’s an acceptable UX to make the user do all the work themselves. Some matching is obvious (if the ISBN turns out to be unique), some other is more fuzzy.
Maybe I made the wrong impression, but I don’t think we should align the catalog records at all. We should focus on aligning user records and find a way to help each app make sense of them. My take has been the snapshots. There are possibly other, better ways.
I do like Nick’s suggestion of an XRPC method.
That’s how I’ve been thinking about enrichment so far:
# Enrichment service
The enrichment service provides display data for books: covers, descriptions, and other metadata not carried in user record snapshots. It is an XRPC service implemented by Dracome and potentially by other providers.
The service is intentionally narrow: it is additive display data only, not a catalog authority. Dracome's enrichment data includes aggregated content from OpenLibrary and the German National Library (DNB), plus Dracome's own curated additions and corrections.
## Identifier bag
No single universal book identifier is reliable across all sources. OL keys do not cover all datasets. ISBNs are absent, corrupted, or duplicated for a significant portion of the catalog. The enrichment query therefore accepts a bag of identifiers:
identifiers: {
dracomeKey?: string // Dracome's own internal catalog key
olWorkKey?: string // Open Library work key
olEditionKey?: string // Open Library edition key
dnbKey?: string // German National Library identifier
bylinePubKey?: string // Byline-sourced proprietary identifier (scheme: bylinePub)
bookhiveKey?: string // BookHive-sourced proprietary identifier (scheme: bookhive)
isbn?: string[] // one or more ISBNs
// extensible; other providers may add their own types
}
Every enrichment response includes:
| Field | Description |
| --- | --- |
| `isCanonical` | Whether this entry is the current record for this provider. `false` means a redirect applies. |
| `redirectTo` | Populated when `isCanonical` is `false`. Contains the full XRPC endpoint URL and the provider's preferred stable identifier for the replacement entry. Immediately callable; bypasses resolution entirely. |
| `lastUpdatedAt` | ISO timestamp; for consumer cache decisions. |
| `matchType` | `exact`, `fuzzy`, or `none`. Consumers should suppress display on `none` and warn on `fuzzy` for high-stakes UI. |
| `sourceIdentifiers` | The original source identifiers (OL key, DNB ID, etc.) from which this entry was derived. Provenance only; not a sync signal. |
On the working group: I’m still up for it, co-chairing or otherwise — happy to move the open questions there instead of hashing them out further in this thread.
Hi y’all! Just wanted to introduce myself. I’m Manton, founder of Micro.blog. We support Standard.site and want to keep doing more with AT Proto. The discussion here is fantastic.
Micro.blog has its own “what are you reading” interface to help people blog about books. We also have a companion mobile app just for books called Epilogue. We pull data from Open Library, Google Books, and ISBNdb.
To the works vs. editions discussion, unfortunately I made a decision early on to make our database pretty flat, with ISBN13 as the identifier. So we think in editions instead of works. (To make it worse, individual users can edit the metadata and book cover just for their own copy of the book. It’s a bit of a mess, and I hope to eventually normalize the data.)
For the lexicon, ideally a platform could participate even if they have only the bare minimum of data like book title, author, and identifier. Everything else could be layered on top of that.
I agree on this; aligning on what’s stored for a user’s repo such that we can do our own disambiguation from there. I think a point was made about leaning into a uniform representation of identifiers so folks can move from there (looking for the post that showed an example but the following is one:
{
// ... rest of record
"title": "A Brief Book Title",
"author": [
// .. to permit multiple authors safely.
"Becky Chambers" // for just the plain text form.
{
"name": "Becky Chambers",
"identifiers": [
{ "uri": "https://beckychambers.info", "resource": "web" },
{ "uri": "https://openlibrary.org/XXXX", "resource": "openlibrary" },
// ... other combos here
]
}
],
"identifiers": [
{ "uri": "https://beckychambers.info", "resource": "web" },
{ "uri": "https://openlibrary.org/works/XXX", "resource": "openlibrary" },
{ "uri": "https://barnesandnobles.com/XXX", "resource": "barnes" },
{ "uri": "https://hardcover.app/XXX", "resource": "hardcover" },
{ "uri": "https://bookhive.buzz/XXX", "resource": "bookhive" },
]
// ... rest of record
}
This is the goal! I think we’re converging on this.
I think it’ll be part of the enrichment process you’ve mentioned; as you come across records in the wild, it can add them as new identifiers (like if someone signed in Dracome with their info from Hardcover or Bookhive).
I think the example I showed allows for Bookhive to contribute in an agnostic way (with the identifiers key while still adding their own metadata. We can listen to each of our respective lexicon records while looking for specific tells to say “hey! this is about this book!”
I think the point that In Which A Book Lexicon is Born - #12 by tynanpurdy.com made here can be used to handle this (by pointing to not only progress but to the ref of what’s being reviewed, it - the referenced item - can have all of the identifiers needed to identify it across things - Yelp, etc) to make it more rich.
I also want to suggest folks to move conversation about book reviews to this thread: Handling Reviews for the Book Lexicon and keep this thread about how we’ll define either the generic form/interface of a book lexicon (something each app extends in some faithful way or uses under the community.lexicon.book umbrella).
I ended up writing On getting consensus on the lexicon for books last night and think we can vote/discuss the schemas mentioned within. I’ll be updating Bibliograph to support it but if appviews have a minimal compatibility surface area then we can safely run our own catalogs, have our specialized lexicons but also support some compatibility.
Very nice @omg.jacky.wtf! I have some opinions that I wanted to write up in a longer form too. But, when I started writing them out, I needed to go deep on the base layer, so I guess I’ll leave you on a cliff-hanger of my specific feedback to your proposal for now (it took me entirely too long to write this up as-is).
I don’t think my article introduced anything all too new to this thread, but hopefully clarifies the direction we are heading here. Since it ends up being interoperability at a different level than most lexicons, due to constraints I discuss in the article and the ones that we already know from this thread.
great thread, lot’s of people are clearly into books
On the ORCID angle @omg.jacky.wtf raised: that already runs in Sifa ID today (example: Juan A. Esteban (@juanesteban.art) | Sifa ). Sifa pulls someone’s ORCID publications (once connected) and their standard.site ones and surfaces them on their professional profile, no manual entry. Also works with non-atrproto things like your Github activity and/or RSS items if you add and verify them.
The link from a record to a person doesn’t run through a DID though. Of course a Sifa user has one by definition (that’s the account itself), but the match rides on a shared external identifier: the source record carries something structured and stable like ORCID, the same ORCID sits on the profile, and Sifa just joins based on that. The record itself doesn’t have to carry a DID.
Same trick would work for books, AFAIK: if contributor records carry a structured author identifier (ORCID, ISNI, VIAF, an OpenLibrary author key, Wikidata), a profile app like Sifa could already surface someone’s books on their profile by identifier match (assuming the user adds/links the ID to their atproto account of course, no magic here ). But yeah: no DID needed, works perfectly.
The above mentioned claims-vs-edit-rights split: an author claim record keyed to a DID (or ORCID) is pretty much what lets a profile app auto-pull and pre-fill an author’s own books once they join atproto? Direct backlink discovery via Constellation.
On the claiming itself: the calendar/speaker thread is going towards the same idea where a self-written claim record is pointing at the subject, the counterparty writes their own half, the appview derives the confirmed state. Cartridge already does the claim/verify side today as well (but manually verifying the game companies claiming the games and then pulling it into the claimers PDS). “Author claims their book” looks like the same shape as “speaker claims their talk” or “game company/artist claims their game” to me.
For Sifa nothing new needed to make the books work, just happy to start (automagically) adding authors books to their Sifa profiles once this has been setup
There is now a Part 2 to my blog post which proposes a book lexicon and justifies how to make it interoperable. I separated the review lexicon from the edition lexicon because the edition lexicon is meant to be “this is a book” and the review lexicon is “this my rating & review of this book”.
In Part 3, I’m going to go into the XRPC system that I had proposed, which will super-charge the portability of books across systems.
This literally came to me in a dream: Does it make sense to think of book lexicons like video file formats?
For example, the .mkv format is “a container format that can hold an unlimited number of video, audio, picture, or subtitle tracks in one file.” The stuff can be of different types (e.g. encoded with different video and audio codecs).
In a similar manner, would it make sense to have a top level lexicon that functions as a container that you put other stuff in? Like a minimal root level lexicon with other lexicons defined that go into it. The idea being that instead of trying to accommodate everything from the start it would provide a way to make ongoing adjustments and expansions.
(I know one of the guidelines with lexicons is to not change them once they are published, and the idea of adding child lexicons runs foul of that. But, it feels like having a type of expansion built in would be a proper indicator that downstream consumers have to know how to handle things and that they need to eject if they see something they don’t know how to deal with. I expect there’s already presidents for this if someone’s got examples.)
I’m also thinking of this like an approach in the Rust programming language where you can have structs (“structured” collections of data where every instance/copy always has the same things in it) and enums (collections of items that are all of the same type/category but have different attributes).
For example, a basic Book struct would have a title (that’s a “string” of text), an ID (that’s another “string” of text) and an Author that’s one of a set of possible options provided by an Authorenum. The Book struct would look like this:
struct Book {
title: String,
id: String,
author: Author,
}
The stub for the Author emum begins with this:
enum Author {
}
The first category of Author would be a SingleAuthor that has a name that’s a string of text:
enum Author {
SingleAuthor { name: String }
}
It would also have a MultipleAuthors option that is a group of one or more SingleAuthors (aka an array which is called a Vec in Rust):
Each Book is required to have an author, but there’s no restriction on which one of the options from the Author enum can be used. So, these are all valid:
Book {
title: "Beowulf",
id: "abc3",
author: Author::Unknown
}
I don’t have the spoons to flesh out this idea, but I wanted to jot something down before lost it.
P.S.
This would be better demonstrated using actual lexicons. I don’t know enough about how child/secondary/remote/or-whatever-they-are-called lexicons to do that yet.
I’d imagine the same thing would apply to editions, idendifiers, etc.
struct Book {
title: String,
id: String,
author: Author,
identifiers: Vec<Identifer>,
}
enum Identifier {
UUID: {
id: String
// other data, links, etc...
},
ISBN {
id: String
// other data, links, etc...
}
GoogleBooksID {
id: String
// other data, links, etc...
}
}
It feels like there should be a “Defult” edition if no other edition info is known and that the Identifiers should go under that, but that’s something to play with in the structure.
Just wanted to introduce myself. I’m Manton, founder of Micro.blog. We support Standard.site and want to keep doing more with AT Proto. The discussion here is fantastic.
). But yeah: no DID needed, works perfectly.