Hey folks, we just launched the atproto spaces alpha. Check out the blog post for more details!
Over the following weeks, we’ll be iterating on the specification and the implementation. In all likelihood, this will involve breaking changes to the PDS distribution, the Bluesky-hosted alpha PDS, and the alpha SDKs.
To minimize disruption, we’ll plan to do releases on Thursdays (barring something critical that requires more immediate attention). When we do a release, We’ll make a post to this thread detailing the changes and encouraging folks to upgrade. Subscribe to this thread and follow along for those updates.
Please leave this thread for updates. I encourage you to share other thoughts and feedback under the WG Private Data tag on this forum.
32 Likes
I’ve been putting off buying a vps to do this and am hoping this is what I need to get my application able to do things. Suuuuuper excited.
2 Likes
Hey everyone!
We haven’t pushed out any breaking changes the last couple weeks but we did land some this week, specifically centered around the simplespace API.
Two main pieces of work landed:
Here’s a description of relevant breaking changes:
- simplespace lexicons
- policy split into required readPolicy/writePolicy
- addMember replaced by putMember
- read & write booleans on members
- @atproto/car
- CAR SDK moved from @atproto/common into a new @atproto/car
- some updated reader/lifecycle APIs
- @atproto/space
- Some type tightening & renaming
- @atproto/syntax
- removed
atUri.spaceRef() in favor of SpaceRef.for()
We deployed the spaces alpha PDS, published an updated docker image at ghcr.io/bluesky-social/atproto:pds-spaces-alpha, and published new alpha snapshot packages on NPM for all relevant packages.
This does involve a database migration on the PDS. To reduce the operational burden for folks that are running a self-hosted PDS, I opted to make that something that runs the first time each user’s actor DB is opened (relevant commit) for now. This approach will likely change before the production distribution.
9 Likes
Heads up, we’ll be doing another release this Thurs (Oct 1)!
It will likely include the following changes:
We are currently going back over the repo serialization & commit format. This will not be a part of the release this week, but will likely be done next week or the week after.
3 Likes
Okay as promised, we just cut another spaces alpha releast today! This includes the credential & sync changes that I listed above plus a small API field rename in listSpaces and simplespace.createSpace.
There are several breaking changes, and I expect this to be a more disruptive rollout than the last one. The rollout will update the hosted psaces alpha PDS, the Docker image at ghcr.io/bluesky-social/atproto:pds-spaces-alpha and the relevant npm packages. For folks running their own PDS, this release includes database migrations on both the account manager DB & the actor store DBs.
We’ve put together a prompt that you can give your coding agent. I ran that against my bulletin app and it was able to one-shot bring it up to date with the current proposal.
To summarize the changes:
- Space credentials now use HTTP Message Signatures instead of DPoP (Implementation / Proposal)
- Space credentials have a shorter expiration and repo hosts support an additional method for revoking credentials (Implementation / Proposal)
- Space sync is now more robust. PDSes have better retry logic for write notifications and space authorities sequence write notifications across the space (Implementation / Proposal)
space.listSpaces and simplespace.createSpace had type params updated to spaceType to avoid problems with languages where type is a reserved word (Implementation)
As always, please let me know if you have feedback or if you run into bugs or sharp edges as you use this alpha.
5 Likes
A simple one today! No breaking changes, just two small things.
We have a bigger release in the works probably for next Thursday.
For today:
- Added a small temporary flag on
describeServer for determining if you’re talking to a spaces-alpha compatible PDS (Impl / Forum Thread)
- Tightening up space repo lifecycle. In the code, specifically ensuring that the space exists before posting in a repo where the authorities are the same (Impl / Spec)
4 Likes
We just cut another spaces alpha release today!
This is the biggest yet, there are multiple breaking changes across core protocol pieces. This is likely the last/only release of this size. Though we are planning to do a smaller (tho still substantial & backwards-incompatible) release next Thursday. The hope is that that will be the last breaking alpha release. Then we can use November to do final shakeout, before moving to beta at the end of the month.
I expect this rollout to be somewhat disruptive. We have updated the hosted spaces alpha PDS, the Docker image at ghcr.io/bluesky-social/atproto:pds-spaces-alpha and the relevant npm packages. For folks running their own PDS, this release includes database migrations on both the account manager DB & the actor store DBs as well as adding a new per-account key which will require a DID doc update for each account (see below).
Like last time, we’ve put together a prompt that you can give your coding agent to handle the migration.
To summarize the changes:
- Shared secret commits. This is the biggest change. Space repo commit HMACs are now done using a negotiated shared secret between a syncer and a repo host. This requires introducing a new
#atproto_kem key for each user and commit MACs are sent in HTTP headers. We also tweaked some field names in the commit such as hash → data. Rationale is in this thread. (Implementation / Proposal)
- STAR files.
getRepo now returns a STAR archive instead of a CAR file. Thanks @bad-example.com! (Implementation / Proposal)
- Public spaces. Spaces can issue credentials without a delegation token. As part of this, we moved the delegation token & HTTP signature key material to the request body of
getSpaceCredential (instead of headers). Also tweaked the authorization scheme name for consistency. (Implementation / Proposal)
- Simpler DID doc keys. To avoid some complexity, we ditched the new space host signing key & service endpoint in DID docs. Space authorities resolve like any other account:
#atproto_pds is the space host and #atproto signs credentials. (Implementation / Proposal)
- Reliable space deletion. Space authorities retry
notifySpaceDeleted until it’s delivered or the syncer’s registration expires. This mirrors the robust delivery of write notifications from repo host to space host. (Implementation / Proposal)
All of the spec changes for this release are combined in proposals#125.
We also updated the version of Node in the published Docker image in response to the eMudhra CA compromise.
A note on KEM keys: every account with space repos now needs an #atproto_kem key in its DID doc. New accounts get one automatically. For existing accounts, the PDS runs a one-time background backfill on startup that generates the key & publishes an update PLC. This should Just Work 
! However, it touches every account on the PDS, so self-hosters should expect a PLC update and an #identity event per account. If your account uses did:web, or your PDS can’t update your PLC doc, you’ll need to add the key yourself (getRecommendedDidCredentials returns it). Until an account’s key is published, its space repos can’t be synced. Account migration tools also need to carry atproto_kem through from getRecommendedDidCredentials, since submitPlcOperation now checks it.
As always, please let me know if you have feedback or if you run into bugs or sharp edges!
11 Likes
As promised, we just released another update to the spaces alpha today!
This is a smaller release than last week. We have a few new features & protocol mechanisms (including account migration!). The meat of it is a refactor of simplespaces. This is backwards-incompatible and will require updating any of your apps that use simplespaces!
Like last time, we’ve put together a prompt that you can give your coding agent to handle the migration.
To summarize the changes:
- Space policies: we totally refactored how simplespaces work. Simplespace is split it up into three different policies: personalspace, memberspace, and managedspace. Also space create/update/delete methods are moved to the
com.atproto.space.* name space. And policies got some configuration for choosing if the policy & member list should be public. Thanks to @matthieu.bsky.team for the original proposal! (Implementation / Proposal)
- Account migration: Space data can now migrate! This functions similarly to public broadcast repo migration. The big difference is that there are more repos (not just one), and space authorities also need to migrate some metadata about their spaces. This Implemeninvolved speccing a new STAR profile as well! (Implementation / Proposal)
- clearRecords: This is a new repo lifecycle method for clearing all published records in a space repo. Useful when you want to delete all of a user’s content in a space without doing a bajillion delete operations. (Implementation / Proposal)
- removeWriter: This is a lower-level protocol method for pruning the writer set. Thanks to @mikestaub.social for the idea. (Implementation / Proposal)
The full updated spec is in the proposal.
This deploy does involve an actor store migration for each actor. This will run the first time each user’s actor store is opened. A quick note on that migration: The PDS will upgrade simplespaces automatically. Any simplespace that had a managing app set will become a managedspace. All others will become memberspace. If a simplespace had a mixed config, it will still become a managedspace. This seems like an unlikely pattern, but is still important to be aware of!
As always, feedback is welcome! Let me know if you run into any bugs or sharp edges.
13 Likes
Okay sorry folks I lied about about last week being the last breaking release. Blame @bad-example.com for making us add more bytes to the STAR file format!
(jk jk of course, I’m so thankful for the care that Fig put into this spec and I’m really happy with how things turned out. Huge kudo!)
Since this is a lower level protocol thing and people will be putting together STAR libraries, we wanted to get it out ASAP.
So this release is basically updating to the Official STAR Spec. I also snuck in some changes for how we do takedown handling on the PDS as it relates to spaces. Those should all be basically backwards compatible.
Even though this is a smaller release, we still put together a short migration prompt for bringing your projects up to date. For my bulletin app, it only entailed a package version bump.
Changes:
- STAR files: Bring the spec and implementation up to date with the published STAR Spec. This is basically just adding the control bytes before each from and at the end of the archive (Implementation / Proposal)
- Takedown handling: Bring handling of infrastructure takedown (and deactivation) for spaces in line with how we treat the public broadcast repo. Added a clarification to the spec that when an account is taken down, it should not serve credentials for any spaces that it hosts (Implementation / Proposal)
I expect we’re all done with big/deep changes to the protocol (except for the possibility of changing the URI format). However, I will warn that I’m currently working on a couple big branches of work basically refactoring and fine-tuning all of the Lexicons & the SDK. I expect this to land next Thursday.
We’re currently targeting Dec 1st for the release of the beta!
10 Likes
We’ve got a few more changes to the spaces alpha cooking for this week. Details for most are in the pull requests on github, but here is a summary:
- Lexicon Refactor: as dholms mentioned above, we did a pass over naming of NSIDs, fields, and other lexicon schema details for the new
com.atproto lexicons.
- HPKE Secret Export: the current KEM shared secret derivation process uses several lower-level pieces of the IETF HPKE standard, but these don’t align with what HPKE libraries generally expose. We are going to update the secret derivation to match the high-level “Secret Export” interface defined in the RFC. Not all HPKE libraries implement this, but some like Go’s stdlib crypto/hpke do.
- HMAC in HTTP headers: instead of specifying HMAC context bytestring construction details, we are switching the commit and record authenticity HMAC construction to use HTTP Message Signatures (RFC 9421) response headers. Note that this is still just an HMAC using the KEM derived secret, not a cryptographic signature over the commit.
- Notification Registration in Credential Request: instead of registering for space credentials with a separate API call, follow registration in to the
getSpaceCredential request.
- Clarify Space Repo Lifecycle: small changes to how deleted / empty space repos get cleaned up, and possibly a way to communicate “writer removal” to syncer services.
This is a bit more change than we expected at this point, and we recognize that this churn is extra work for folks keeping up with the alpha. But it will be much harder to make changes after this, and we want the protocol to be as polished as possible.
One last detail which will probably not happen this week, but we think probably will happen before the end of the alpha is switching the space URI syntax to at:space/. We want to get a bit more read of the room about that at the IETF WG meeting next week in San Francisco, and will try to announce the final decision as soon as possible after that.
8 Likes
Nice (and this is to satisfy the 20 character requirement for posts)
5 Likes
Just cut the release for today, and updated the alpha spaces PDS.
As @bnewbold.net mentioned, this is a bigger set of changes than we’d hoped for this late in the alpha. But I also think they’re pretty big improvements. Thanks for bearing with us! We know these updates cause a lot of churn. But everything is about 1000x easier to change now than it will be in 3 weeks. We’re trying to take care to respect the folks building on/with the alpha while still specifying the best protocol we can.
I actually really do expect this to be the last breaking change like this. The only remaining thing is possibly updating the URI format. I’d say I’m about 75% confident we’ll end up going with at:space/ but we want to build a bit more confidence during our trip to the IETF next week that that’s a viable path for a unified URI format.
Besides that, the next couple weeks will be mostly focused on final fine-tuning/polishing and operationalizing things. I’m sincerely hoping that besides the URI format, we have no more backwards-incompatible changes to the spec (though I can’t guarantee that).
Per usual, we’ve published alpha versions of the relevant NPM packages, published a new Docker image at ghcr.io/bluesky-social/atproto:pds-spaces-alpha and updated the hosted spaces alpha PDS. This release does include database migrations across both the account manager DB & actor store DBs.
We’ve put together a prompt that you can give your coding agent to handle the migration.
To summarize the changes:
- Lexicon refactor: We went back over every spaces Lexicon to make names, fields & errors consistent as well as just generally tightening up the semantics. (Implementation / Proposal)
- SDK Refactor: We did a similar pass over the SDKs as well. This is mostly just implementation work, but it turned up a small thing or two in the spec. (Implementation / Proposal)
- lex-space package: @matthieu.bsky.team put together an excellent new higher-level SDK for working with spaces built on
@atproto/lex. It handles the whole credential dance and gives you typed reads & writes (Implementation)
- Space notifications: We refactored how space notifications to syncers work. Syncers now register when they get a space credential, and notifications take the form of webhooks. We increased the reliability of notifications for syncers whose registration has expired. And to reflect the change in sematnics of the route (and to avoid the strange use of
get for a side-effect-full route), we renamed the core method from getSpaceCredential → authorizeSpaceSync. (Implementation / Proposal)
- MACs in HTTP Signatures: We had HTTP Signatures elsewhere in the spec, but then our own invented HTTP headers for transmitting repo & record MACs. We opted to standardize on HTTP Signatures here as well. (Implementation / Proposal)
- HPKE Secret export: The KEM shared secret is now derived from HPKE’s standard Secret Export mode. This prevents us from speccing our scheme, and many languages support this in standard HPKE packages. If your language doesn’t support it, the proposal has an appendix with the equivalent ECDH + HKDF steps. (Implementation / Proposal)
- Repo lifecycle: A couple of things to tighten up repo lifecycle including clarification of garbage collection semantnics for empty repos and delivering removal from the writer set over notifications &
listWriters. (Implementation / Proposal)
- Merge main: We merged the last several months of activity from
main into the permissioned-data feature branch. There are some muddled DB migrations as a result. It should all work fine as you upgrade spaces alpha PDSes, and we’ll refactor & squash all of these down before cutting the beta release and merging this into main. (Implementation)
- Moderation guidance: We clarified that labels on space data are published as
com.atproto.label.records in the same space as the content they label. This section also lays out baseline takedown capabilties for each kind of service. (Implementation / Proposal)
- Access control in applications guidance: This addition to the proposal does not change any protocol mechanisms, but instead gives guidance to applications and applciation developers for determining which of their users can see what space data (Proposal)
All the spec changes for this release are combined in proposals#143.
As always, please let me know if you have feedback or if you run into bugs or sharp edges!
5 Likes
Hey @dholms.xyz!
Thanks for the Nov 12 drop, and the agent prompts, as always.
Speakiing of agents, my Claude ran into two things while updating Tandem (intandem.space), which calls Spaces through @atproto/api:
-
@atproto/api@0.0.0-spaces-alpha-20261112204144 has no Spaces namespaces. com.atproto.space, memberspace, managedspace and personalspace are all missing, so agent.com.atproto.space.* is gone. The same is true of …20261112203312. The Nov 5 build (…20261105160032) still has them.
It looks like packages/api/lexicons.json arrived with the “Merge main” commit: it isn’t there at the Nov 5 reference commit (1cc3186d), and it is at d4def6e9. It lists only main’s lexicons, and the Spaces ones exist only on permissioned-data, so they may have been dropped by accident. Section 9 of the migration prompt still describes them as renamed (authorizeSpaceSync, listWriters, …), which makes me think this wasn’t intended.
Is it a slip, or should apps that call Spaces move to @atproto/lex / @atproto/lex-space?
-
@atproto/oauth-resolver resolves to an empty 0.0.0-stage release. @atproto/oauth-client@…20261112204144 depends on ^0.0.0-spaces-alpha-20261112204144, and 0.0.0-stage sorts above that version, so npm installs it. It contains no code, so importing @atproto/oauth-client fails. An npm overrides pin to the snapshot works around it. It’s the same trap as the stray 0.0.0 releases of lex-data and lex-json.
Happy to test a fixed build, if this isn’t intentional!
Thanks for the report!
The space lexicons got dropped from @atproto/api when main merged in, and yeah the snapshot deps used ^ ranges so they were getting out of whack. I updated the publish action to pin exact versions. That’s running now so updated packages should be published shortly!