Proposal: a community lexicon for standalone images

Lexicon Community
,
22

Checking in on a few things!

It looks like there is a good bit of interest and activity here, so I wanted to see who’s interested in taking the lead here to create an initial lexicon and start implementing use to demonstrate the need and satisfaction of the need.

Looking back at the draft in the first post, this is the shape I see:

{
  "lexicon": 1,
  "id": "community.lexicon.media.image",
  "defs": {
    "main": {
      "type": "record",
      "description": "A standalone image.",
      "key": "tid",
      "record": {
        "type": "object",
        "required": ["image", "dimensions", "createdAt"],
        "properties": {
          "image": {
            "type": "blob",
            "description": "The image content itself.",
            "accept": ["image/*"],
            "maxSize": 1000000
          },
          "dimensions": {
            "type": "ref",
            "ref": "#dimensions",
            "description": "The dimensions of the image."
          },
          "createdAt": {
            "type": "string",
            "format": "datetime",
            "description": "The date and time that the record was created."
          },
          "alt": {
            "type": "string",
            "description": "Text used to describe the image for screen readers and accessibility purposes.",
            "maxGraphemes": 1000,
            "maxLength": 10000
          }
        }
      }
    },
    "dimensions": {
      "type": "object",
      "description": "The pixel dimensions of the image. The aspect ratio may be derived from these values.",
      "required": ["width", "height"],
      "properties": {
        "width": {
          "type": "integer",
          "minimum": 1,
          "description": "The width of the image in pixels."
        },
        "height": {
          "type": "integer",
          "minimum": 1,
          "description": "The height of the image in pixels."
        }
      }
    }
  }
}

One change that I’d like to propose is that the image should represent dimensions because aspect ratio can be calculated from them.

2 Likes
23

This lexicon proposal looks good to me, and I agree with the use of dimensions instead of aspect ratio.

Next steps - reach out to the publishers of the lexicons that @byarielm.fyi assembled and ask prettyplease to crosspost to this new lexicon?

What’s the governance around the creation of the community.lexicon.media.image lexicon in the first place?

Thanks!

1 Like
24

From what I saw in reviewing, your ‘dimensions’ is what ‘aspectRatio’ actually is under all the lexicons. Some have it defined in the same lexicon, others have it in another.

Probably easier to get but in if we use the same language even if it’s confusing lol

25

Thank you @natespilman.at for starting this and @ngerakines.me for evolving the proposal!

The lexicon looks good, just a couple thoughts:

Alt Text as Array

As @al9000.com wisely said, alt text should be an array field with language information. This helps in multiple ways:

  • Displaying the image across locales, of course
  • Providing longer/shorter alt text according to where the image is displayed and what is supported by a specific platform. For example:
    • Images that are the focus of a page may get a longer description than e.g. images in a list of thumbnails
    • When cross-posting on different socials no one seems to be able to agree what length is appropriate and it should be easy for platforms to make different choices without needing to cut arbitrarily.

Taking Al’s alt text record as a base:

As Al mentions this array should be required but it can be empty. In the usual case of a single alt text field, this essentially boils down to the same content as a scalar vs a single item array.

dimensions vs aspectRatio

One “concern” about precise dimensions: can the PDS ever (for example) resize the image in the blob to save space? In that case, dimensions may end up out of sync, while aspect ratio will be always be accurate.

I do see the point of having width/height information without needing to download the blob, so my preference is:

  • name: aspectRatio (never wrong)
  • width/height descriptions should accurately describe what assumptions user should make. For example, if resizing is possible, then “the maximum height” and/or “the height of the image when saved”.

And, regarding pixel size: how does that work with SVGs or other vector-based image formats? I’m not sure what current best practices are, but if anyone knows do let us know, or I’ll try to research it later.

26

Also, regarding additional information like copyright, this is a great use case for a sidecar lexicon. For example, platform-specific metadata may be done as (pseudo-coded):

Image record:

key: cool-image
$type: community.lexicon.media.image
data: 
  - the blob
  - the alt text
  - the aspect ratio

and then in the same PDS, the application-specific data can be:

key: cool-image
$type: cool.application.image.metadata
data: 
  - filters used
  - version of software
  - title/description

we could consider using this pattern for licensing information too, but we should first focus on getting the image lexicon itself adopted by the ecosystem.

27

If a PDS (or any other process) changes the size of the image it should update the pixel dimensions as well. I don’t know if there’s a MUST way to require that, but if the listed pixel dimensions don’t match those of the blob, the record is effectively invalid. Any app relying on the pixel info will be working off faulty data.

In the same vein, if a process changes the image size without updating the pixel dimensions the user can no longer safely use the values to make assumptions. (I might be missing something with the max height, but if an image is uploaded with a given height, but a process changes the blob, the value can no longer be trusted regardless.)

Ah, right. I was implicitly thinking about this as only raster based images. My instinct is to make that explicit and create a second lexicon for vector based art. There’s enough difference between the two of them that it feels weird to have them in the same spot. If they end up in the same lexicon the usage requirements go up non-trivially for end users in terms of having to handle both types (e.g. if I’m using it for an avatar on a website, the code I generate for the output has to change between raster/vector).

28

Regarding size changes etc, some snippets from Blobs - AT Protocol :

It is effectively not supported to dynamically serve assets directly out of blob storage (the getBlob endpoint) directly to browsers and web applications. Applications must proxy blobs, files, and assets through an independent CDN, proxy, or other web service before serving to browsers and web agents, and such services are expected to implement security precautions.

and

Servers are strongly recommended against parsing media files (image, video, audio, or any other non-trivial formats) directly, without the use of strong sandboxing mechanisms. In particular, PDS instances themselves should not directly implement media resizing or transcoding.

Also, I believe changing dimensions would also change the CID and therefore the blob link.

1 Like
29

I just made a post in the book lexicon thread about pondering if it makes sense to think about them like video file containers and payloads. The same question applies here. Instead of copy/pasting it (and having to change all the examples from books to images), I’m just gonna link it if you want to take a look: