A good few different game genres nowadays tend to give numeric skill ratings to players based on performance - basically any PvP game with skill-based matchmaking needs rating, but rhythm games have them pretty often as well! While I’ve been working on my leaderboard hosting system Tsunagite, I’ve thought a lot about how to implement player rating, and I’ve come up with a few different ideas.
The easiest approach is to just have a system equivalent to a feed generator - a separate program that accepts a player’s DID, looks through all their games/scores, calculates the player’s rating, and returns it to the frontend. It would definitely take the least technical effort to design, but I’m a little hesitant to do this for Tsunagite because I don’t want to require that each game dev run their own rating generator when the whole point of Tsunagite is to take the burden of hosting a leaderboard off of the developer.
The other option is to define rating algorithms via records. This would be a bit more of a pain, as it would definitely require a DSL either written in some sandboxed expression language stored as strings or directly as a JSON object tree, but it comes with the advantage of built-in transparency and verifiability - if the algorithm is open, anyone can run a player’s data through it to validate an external rater’s result.
There’s probably some other solution I’m not thinking of as well (of course you can always just let ratings be defined directly by a record in the user’s repo, but I’m trying to make something not easily manipulated), so let me know if you have thoughts!
1 Like
Thanks for tagging me in on this one!
I’m looking at doing several kinds of leaderboards for lance.blue. It’s not even close to implemented yet, but here are some things I’m thinking of:
-
The underlying game has “scopes” for calculating scores: like “era” (year the game is set in) or “rules level” (which rules are enabled), where you’d typically generate leaderboards with some scoping limits. Then there are scopes at a higher than “game rules” level, like whether it was a singleplayer match against AI, or if the match is part of a higher-level construct like a “campaign” or a “tournament” or not. So I’d want to have a flexible DSL for choosing appropriate scopes to limit results to: within-tournament leaderboards, “daily single player challenge against the bot” leaderboards, etc.
-
I want records to live in player PDSes, but because of that I want a verifier structure on scores. This means players can write their own score, but it’s “verified” because they played on my hosting. I may even multiply-write the scores so results still exist no matter if the players delete the records.
-
Match scores aren’t necessarily in a linear order. But many ranking systems require order. This can make reproducibility hard since someone else can’t necessarily get the same result with the same algorithm unless you are also writing out sequential information to the records too.
On the record, I was envisioning that there’s a general “match” lexicon that maybe just contains fields for which game, what platform, which version of it. Then a more arbitrary and game-specific field for “score”, so that indexers don’t try to read anything game-specific unless they’re filtered down to the game they care about.
1 Like
Scopes are definitely something I’ve been considering as well! Rhythm game ratings can get pretty complex - just look at how much goes into calculating for Sega’s rhythm game Maimai! https://silentblue.remywiki.com/maimai_DX:Rating
I think if we have a way to publicly-define algorithms, allowing rating to be saved as a flat number should be perfectly fine, since then anyone can verify the rating just by running the public algorithm.
I think the best solution to order-dependent scoring (like Elo and Glicko/Glicko2) is for scores or match results to be timestamped. Just about everything meaningfully should be timestamped anyway since that makes things a lot cleaner during a backfill, after all.
I definitely agree that a match lexicon would be good! Shouldn’t need to be too complex - identifiers for the participants and winners, URIs to any necessary score records, and a timestamp should be more than enough. The big question ends up being which account holds the record, but I feel like in most cases it should be held by a TO for structured tournaments and maybe a game’s studio for a ranked ladder. Single-ownership can be such a hassle…
Timestamps in a distributed system are quite hard, unless you have a single authority issue all of them. But at that point you might just want to issue a sequence number so that there’s no ambiguity at all to exactly-ordered replays (plus then readers can know about gaps in the data + you can write tombstones of data, both harder if just timestamped)
I was personally considering that match records could be multiply written depending on the scenario. With some kind of verification. Like every player gets a “copy”, the TO also stores a copy, and the TO issues verification records. So any player has their own verifiable copy, and it stops verifying if they tamper with it, but a reader only needs one of the records to get all the match data (+ the TO’s verifier record if they want to verify it).
That is, there isn’t a single canonical record, but instead synchronized copies to all participating parties.
Yeah, Bluesky’s run into issues with people messing around with timestamps before, and their solution was to just mark posts if their index time was noticeably off from the self-reported timestamp. Having multiply-written records with a TO/gamekeeper verifier sounds like a good solution to it!