Record and Query Layers: Which One(s) for this Working Group?

Working Groups WG Community Standards
1

Hello everyone! @brittanyellich.com and I met yesterday for an informal brainstorming/thinking session about Lexicon, and ended up with a question I’ve never really seen spelled out, but that may help make sure we’re aligned on what/why/how.

Record and Query Layers

When defining Lexicons there’s 2 possible layers at which we can operate:

  1. Record Layer: app.bsky.feed.post, defines how information is stored
  2. Query Layer: app.bsky.feed.getPosts, defines how information is transferred/requested

These may depend on each other (getPosts returns a view over stored post records), but can be tackled independently.

A standardization process like the one of this WG may span both layers. For example we could:

  • Agree on the record layer exclusively ⇒ “WG participants agree this is the way Communities store their data, which means anyone with direct PDS access can interpret that data for themselves”
  • Agree on the query layer exclusively ⇒ “WG participants agree this is the way “Arbiter/AppView-like” entities (whoever mediates the community) are asked for data and how they respond.”
  • Some in-between ⇒ We may agree on how some data is represented at the record layer for e.g. membership, and also provide a getMembers query layer, but for other features (e.g. getSubgroup) we agree we only standardize the query.

What Layer(s) do we work at?

As Brittany and I worked through a possible Lexicon proposal, I realized people here may have different ideas about which layer our output is supposed to target, or at least we have never specified enough to have a clear answer to “which one are we interested in, and for which operations?”

I’ve been thinking about the implications of focusing on each, and I’ve come to think that:

  • People building apps may be most interested in a standardized Query Layer, as it allows them to fetch information from different “Group AppViews” in a standardized way (or make sure their AppView is compatible with other people’s).
  • But it’s important to get the Record Layer right, because it’s what protects implementations from drift and allows real data portability across AppViews. For example, it allows two different AppViews to serve different views over the same community data. And also, it allows independent AppViews with access to the same data to verify assertions about e.g. membership.

The risk

What I see as “the risk” (and this is mostly my musing) is falling for the temptation of mostly standardizing the Query Layer. After all, that’s the one that unblocks building actual apps!

But without some underlying idea of what data an AppView can expect straight from the PDS, users risk finding themselves with a lot of really nice data that no AppView knows how to interpret. Or locked into whatever AppView they signed up with, because no other implementation can make sense of its data.

The question

I mostly wanted to bring this up as food for thought cause I had not quite seen it spelled out this way, and I realized I did not have an answer myself. But now I’m curious: @dholms.xyz and @bnewbold.net, is there any recommendation for this somewhere we can point people/ourselves at? Has this been explicitly discussed for Bluesky Lexicons? And also @zicklag.dev and @blaine.bsky.social et al., I’d love to hear if you’ve thought about this in your own implementations of groups.

5 Likes