Extending Lexicons: sidecar records patterns and typed extensions

Hello! I wanted to start a general discussion about Lexicon extensions and sidecar records, brought up by the need to connect records, web pages and apps.

For reference:

A sidecar record is a special-by-convention (yet regular) record that holds additional information about another one. For example, a sidecar record for an app.bsky.feed.post record may hold additional information about who’s allowed to reply to the post.

The mechanism is extensively used but AFAIK not fully documented. So to put everyone on the same page, I did some research on patterns out there and some thinking on pros/cons. I tried to be objective, but feel free to point out what I may be missing :stuck_out_tongue:

These patterns can overlap, and some examples may be better described as related records than sidecars. But nothing like being wrong on the internet helps standardize a term!

At the end, I’m going to talk about two types of extension systems I’d like input on, one for sidecars, and one within records.

Contents

Sidecar Pattern #1: Keys derived from the subject

This covers:

  • #A: Reusing the subject’s record key in another collection, e.g.app.bsky.feed.threadgate [definition, example record] ⇒ the Lexicon explicitly requires the threadgate to share the root post’s record key and repository
  • #B: Deriving a key from the subject’s full AT URI, e.g. social.coves.community.acceptance [definition, example record] ⇒ the Lexicon specifies a an hash of the subject URI

In this pattern, locating a sidecar from its subject depends on a convention. An app must already know which collection and repository to look in, and how to calculate the key.

To help developers understand the relationship:

  • Sidecar records may feature a link back to the subject
  • The sidecar’s Lexicon description may document how to map sidecar ⇔ subject

Strengths/weaknesses:

  • :white_check_mark: Once the convention is known, an app can fetch a sidecar directly
  • :white_check_mark: Metadata can be updated without giving write access to the subject record
  • :white_check_mark: Metadata updates don’t create a new CID for the subject record
  • :white_check_mark: Sidecars can support independently owned metadata by deriving keys from the full AtUri
  • :cross_mark: Apps must know each sidecar’s collection, repository rules, and key derivation to look for it
  • :cross_mark: Changing the subject record’s key means migrating all sidecars or having an explicit policy for handling old associations
  • :cross_mark: Reusing only the subject’s record key may collide when different subject collections use the same key (e.g. self vs self on two separate Lexicons, avoided by #B)

Sidecar Pattern #2: Explicit reference in records

This covers:

  • #A: From the sidecar to the subject, e.g. tech.transparencia.news.enrichment [definition, example record] ⇒ the Lexicon defines an article field referencing the subject’s URI and CID, alongside additional metadata
  • #B: From the subject to the sidecar, e.g. place.atwork.endorsement [definition, example subject, referenced proof] ⇒ the Lexicon defines a signatures array containing AtUri + CID references to separate proof records attesting to the endorsement’s content

In this pattern, the record stores an explicit reference, while the Lexicon defines what that reference means. This can also be combined with Pattern #1, such as by reusing the subject’s key.

To help developers understand the relationship:

  • The reference points to the target’s repository, collection, and record key, which means:
    • Subject ⇒ sidecar: an app can directly discover and fetch the referenced record.
    • Sidecar ⇒ subject: finding sidecars that point to a record requires listing and filtering within repositories or maintaining an index
  • The referencing field’s Lexicon description may document the semantics of the relationship

Strengths/weaknesses:

  • :white_check_mark: Differentiates subjects with the same key across both collections or repositories
  • :white_check_mark: Subject ⇒ sidecar: an app reading the subject can locate the linked metadata directly
  • :white_check_mark: Sidecar ⇒ subject: adding metadata does not require permission to edit the the subject (no CID change)
  • :white_check_mark: Sidecar ⇒ subject: multiple apps can attach their own metadata
  • :cross_mark: Sidecar ⇒ subject: discovering sidecars requires a separate lookup mechanism (and potential enumeration, unless #1 is added)
  • :cross_mark: Subject ⇒ sidecar: attaching or changing the pointer requires write access to the subject and changes its CID
  • :cross_mark: Moving or deleting the referenced record can leave dangling references
  • :cross_mark: A reference containing a CID identifies a specific version, which requires write permissions to the subject to update

Sidecar Pattern #3: Record metadata object keyed by the collection’s NSID (proposed)

I couldn’t find this pattern implemented, so this counts as a “proposal” of sorts. The most similar pattern I could find is:

  • Metadata or configuration for a specific collection or XRPC method, e.g. at.inlay.endpoint [definition, example record] ⇒ the key is an XRPC method’s NSID, and its did field identifies the service implementing that method.

…which can then be turned into a sidecar record keyed by collection with record key references inside the record.

For example, in the example from the original forum post at://alice.example/community.lexicon.meta.links/app.bsky.feed.post could contain all the reverse links to app.bsky.feed.posts.

{
  "$type": "community.lexicon.meta.links",
  "342hjkfsd": {
    "links": [
      {
        "href": "https://alice.example/posts/hello",
        "rel": "canonical"
      }
    ]
  },
  "342hjkxyz": {
    "links": [
      {
        "href": "https://alice.example/posts/another",
        "rel": "canonical"
      }
    ]
  }
}

In this pattern, locating metadata depends on both the subject’s collection and its record key. An app fetches the sidecar record keyed by the collection’s NSID, then reads the entry matching the subject’s record key.

For this example, the convention would be that the sidecar describes subjects in its own repository. So the 342hjkfsd entry refers to at://alice.example/app.bsky.feed.post/342hjkfsd. Metadata for place.stream.livestream/342hjkfsd would live in a separate sidecar record, keyed by place.stream.livestream.

To help developers understand the relationship:

  • The sidecar’s Lexicon description would document how the data is meant to be interpreted
  • Each entry would have a defined metadata structure (see the links array above), although unions may be accepted
  • Apps would still need to know which sidecar collections to query and their conventions. Known extension types can be fetched for with a single lookup.

Strengths/weaknesses:

  • :white_check_mark: When the convention is known, an app can locate metadata without an index or enumerating records
  • :white_check_mark: Within a known sidecar collection, record keys immediately identify whether a subject collections has entries
  • :white_check_mark: Can differentiate subjects with the same record key across collections (e.g. self records)
  • :white_check_mark: Metadata updates do not modify the subject or change its CID
  • :white_check_mark: One fetch retrieves metadata for multiple subjects in the collection
  • :cross_mark: Current repository permissions cannot restrict an app to one target collection’s mapping
  • :cross_mark: The mapping grows with the number of subjects and can exceed record-size limits
  • :cross_mark: Updating one entry replaces the containing record and changes its CID
  • :cross_mark: Concurrent writers updating different entries must coordinate to avoid overwriting each other’s changes
  • :cross_mark: Renaming or deleting a subject requires updating its entry

An alternative (#4): Namespaced, schema-defined extension fields embedded in the record

The other approach I’d like input on is an alternative to sidecar records that puts additional information inside the subject record, in a field named after the Lexicon defining its contents.

Both atmo.rsvp events and BridgyFed posts already add metadata inside records, so embedding extensions isn’t new. What I’m proposing is a shared way to name, document, and validate those extensions that avoids the risk of field-name collisions—one concern raised by [the spec’s recommendation]( https://atproto.com/specs/lexicon#authority-and-control ) against adding additional fields.

This would be a shared convention:

  • The field name is the extension Lexicon’s NSID (possibly with a prefix, here I used lex:)
  • Its value conforms to the definition of that Lexicon
  • The Lexicon documents what the extension means when attached to another record

For example, a hypothetical extension to an app.bsky.feed.post could be:

{
  "$type": "app.bsky.feed.post",
  "text": "Hello from my website!",
  "createdAt": "2026-10-18T12:00:00Z",
  "lex:community.lexicon.meta.links": {
    // this is optional, so we can't rely on it
    // without an associated special key, it also creates collision risks
    // "$type": "community.lexicon.meta.links",
    "links": [
      {
        "href": "https://alice.example/posts/hello",
        "rel": "canonical"
      }
    ]
  }
}

Here, community.lexicon.meta.links (preceded by lex:) would mean that the field is described by the Lexicon community.lexicon.meta.links. The post’s own $type remains app.bsky.feed.post.

With this method, the extension travels with the record while telling developers where to find its definition. An app can notice that the additional data exists and identify its schema without already knowing which sidecar collection or repository to query.

Strengths/weaknesses:

  • :white_check_mark: An app (or app dev) reading the subject record can see that extension data exists
  • :white_check_mark: Embedded metadata requires no additional record lookup
  • :white_check_mark: The association is unambiguous, including across different collections
  • :white_check_mark: Different record types can reuse the same extension definition
  • :cross_mark: Adding or updating metadata requires permission to edit the subject
  • :cross_mark: Validating the subject’s Lexicon does not automatically validate undeclared extensions
  • :cross_mark: Every metadata change changes the subject’s CID
  • :cross_mark: Extensions increase the subject record’s size

For larger or independently updated metadata, the extension could instead contain an AtUri pointing to a separate record.

Open questions

  1. The obvious one: are there existing approaches I’ve missed, or tradeoffs I’m overlooking?
  2. For sidecars: is there interest in standardizing how to declare what they extend and how to locate the associated records?
  3. For embedded extensions: would this shared naming and validation convention be useful? Are there other patterns people would prefer?

No pattern here fits every use case. Regardless, I’d still like apps to have a shared way to discover extensions and find their definitions without having to know about each one in advance, and without competing over the names and meanings of fields in someone else’s Lexicon.

11 Likes

This is a great overview.

One thing that jumped out at me reading this was the idea of composition vs inheritance in system design.

That got be thinking about mix-ins which is basically what adding fields to a record produces. I wonder if an extension to lexicons could be a way to declare additional types that the record should conform to? Something like a $types array with NSIDs. That would let you declare all the parts of the record shape that are joined together.

1 Like

Wow, very comprehensive. Thank you @essentialrandom.bsky.social, this is great!

First, and most important, I like your #4 proposal, especially if we strongly encourage people to also include $type, for clarity, even if it duplicates the field name itself.

I’d be down to work on #4 with you, maybe starting with proxy links ( Connecting records, web pages and apps )?


Beyond that, a few thoughts on on the other ideas. First, #1A is obviously my other preference here besides #4, partly because Bluesky PBC themselves prefer it, but you’re absolutely right that the lack of discoverability hurts it.

The hard thing we’re trying to do here is extend other people’s existing lexicons. I think that basically dismisses #2B, since that can only be done by a lexicon’s own authors.

#2A has an extreme lack of discoverability that’s worse than any of the other options: afaict it requires a dedicated appview or similar index and lookup in order to find a given subject’s sidecars. (As opposed to eg #1A and #1B, which can be done with a single extra getRecord call to the PDS.)

#1B seems…ok…but discoverability is worse than #1A, since the sidecar’s rkey is opaque.

#3 I think is a non-starter due to the scaling limit and churn, which you mentioned.

It would be nice! But the history of extension systems for schema standards like this is…cautionary. XML, COM/CORBA, WSDL, schema.org, etc. The Bluesky team has been pretty clear that they’re not interested in trying it here, and I’m personally inclined to agree with them.

Thank you for making this.

Personally I’m really interested in embedded extensions being solved, because I know examples of fields being added to records that lack any prefix, mainly via field added to posts by some Bluesky clients, so enforcing some conventions would be great. Makes me worried we could maybe have a similar situation to this: Lexicons, Pinned Posts, and Interoperability - AT Protocol . Also having extensions be tied to a lexicon is great and love that most people proposing solutions think the same.

One thing I’m unsure about is if having them at the top-level is a good idea or not and if not would it being under some reserved/special field be better like $ext or similar. One benefit I can think of is it being easier to get all extensions without additional filtering.