Image & Binary Message Handling

6. Image & Binary Message Handling

Certain messages include binary data, most notably images captured by OSS8 devices. These messages follow a structured framing protocol and metadata format to ensure consistent handling across W3S and client systems.

Supported Formats

All image data transmitted in this system must use the JPEG XL format, encoded in near-lossless mode.

Key properties:

Supports 8–16 bit grayscale and color images
Efficient compression for high-resolution frames
Decodable in modern browsers using WebAssembly

Binary Framing Format

Binary messages use a fixed structure for supporting multiple images, metadata, and integrity verification.

Binary Layout

Offset	Size	Description
0	4 bytes	Total payload length (uint32_be) — from byte 5 to end of checksum
4	4 bytes	JSON header length (uint32_be) — length of UTF-8 header
8	N bytes	JSON header (UTF-8) — includes MessageTypeId, metadata, etc.
8+N	4 bytes	Image count (uint32_be) — number of JPEG XL images
	4 × N	Image length table (array of uint32_be) — each entry is byte length of one image
	M bytes	Image data — JPEG XL images, concatenated
	4 bytes	Checksum (uint32_be) — CRC32 over all bytes from offset 0 to just before the checksum

JSON Header Payload Fields

The Payload section in the JSON header describes the structure and meaning of the image metadata fields.

Required Fields:

imageCount (int) — Number of JPEG XL images in this message.
bitDepths (array[int]) — Bit depth of each image (e.g., 8 or 16). Must match imageCount.
imageTypes (array[string]) — Describes the type of each image (e.g., nir, monochrome, color). Must match imageCount.
widths (array[int]) — Width in pixels for each image. Must match imageCount.
heights (array[int]) — Height in pixels for each image. Must match imageCount.
binaryFormatVersion (int) — Indicates the version of the binary framing format used.

Optional Fields:

alarmRegions (array[object]) — List of annotated areas of interest (e.g., suspected fire zones). Each object may include:
- x, y: Pixel coordinates of the region center
- radius: Region size (in pixels)
- label: Type of annotation (e.g., fire_suspected, smoke_detected)

Example:

{
  "imageCount": 3,
  "bitDepths": [16, 16, 8],
  "imageTypes": ["nir", "monochrome", "color"],
  "widths": [1024, 1024, 1024],
  "heights": [768, 768, 768],
  "binaryFormatVersion": 1,
  "alarmRegions": [
    { "x": 123, "y": 456, "radius": 10, "label": "fire_suspected" }
  ]
}

The bitDepths, imageTypes, widths, and heights arrays must all match the declared imageCount.

Alarm Data

Some binary messages (e.g., OSS_IMAGE_FRAME) may include alarm-related data in their JSON header. These alarms represent real-time AI or rule-based detections such as fire, smoke, or thermal anomalies.

Key Points

Optional but critical: Alarms are not always present. Their presence indicates active detection; their absence means no detections occurred.
Location: Alarms are included in the Payload object within the JSON header of a binary message.
Primary data: Alarms, when included, are treated as first-class data — not just auxiliary metadata.

Example

"Payload": {
  "imageCount": 1,
  "imageTypes": ["monochrome"],
  "widths": [1024],
  "heights": [768],
  "bitDepths": [16],
  "alarmRegions": [
    {
      "x": 512,
      "y": 384,
      "radius": 20,
      "label": "smoke_detected"
    },
    {
      "x": 620,
      "y": 410,
      "radius": 15,
      "label": "fire_suspected"
    }
  ]
}

Field Definitions

Field	Type	Description
`x`, `y`	integer	Pixel coordinates within the image frame
`radius`	integer	Affected region radius in pixels
`label`	string	Detection type, e.g. `smoke_detected`, `fire_suspected`, etc.

⚠️ If no alarms are present, the alarmRegions field must be omitted entirely to avoid triggering schema validation errors.

Notes

Maximum total binary message size: e.g., 500 MB (enforced by W3S).
All JPEG XL images must be encoded near-losslessly.
The CRC32 checksum is mandatory.
W3S validates the JSON header before any binary parsing.
Clients must use the metadata to correctly decode each image block.

Future extensions may include:

Delta encoding between consecutive image frames to reduce bandwidth usage when scenes are largely static. JPEG XL supports internal frame referencing for this purpose in sequence/animation mode.
Content hashing for integrity validation and deduplication.

Image & Binary Message Handling

On this page