DataCue API

Draft Community Group Report,

This version:
https://wicg.github.io/datacue/
Issue Tracking:
GitHub
Editor:
(BBC)
Participate:
Git Repository.
File an issue.
Version History:
https://github.com/WICG/datacue/commits

Copyright © 2021 the Contributors to the DataCue API Specification, published by the Web Platform Incubator Community Group under the W3C Community Contributor License Agreement (CLA). A human-readable summary is available.

Abstract

This document describes an API that allows web pages to associate arbitrary timed data with audio or video media resources, and for exposing timed data from media resources to web pages.

Status of this document

This specification was published by the Web Platform Incubator Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Contributor License Agreement (CLA) there is a limited opt-out and other conditions apply. Learn more about W3C Community and Business Groups.

1. Introduction

*This section is non-normative*

Media resources often contain one or more media-resource-specific tracks containing data that browsers don’t render, but want to expose to script to allow being dealt with.

TODO: ...

2. Security and privacy considerations

*This section is non-normative.*

TODO: ...

3. API

3.1. The DataCue interface

[Exposed=Window]
interface DataCue : TextTrackCue {
  constructor(double startTime, unrestricted double endTime,
              any value, optional DOMString type);
  attribute any value;
  readonly attribute DOMString type;
};

4. In-band event mappings

The following sections describe how various in-band message formats are mapped to the DataCue API.

4.1. MPEG-DASH emsg

The emsg data structure is defined in section 5.10.3.3 of [MPEGDASH]. Use of emsg within CMAF media is defined in section 7.4.5 of [MPEGCMAF].

There are two versions in use, version 0 and 1:

aligned(8) class DASHEventMessageBox extends FullBox ('emsg', version, flags = 0) {
  if (version == 0) {
    string scheme_id_uri;
    string value;
    unsigned int(32) timescale_v0;
    unsigned int(32) presentation_time_delta;
    unsigned int(32) event_duration;
    unsigned int(32) id;
  } else if (version == 1) {
    unsigned int(32) timescale_v1;
    unsigned int(64) presentation_time;
    unsigned int(32) event_duration;
    unsigned int(32) id;
    string scheme_id_uri;
    string value;
  }
  unsigned int(8) message_data[];
}

5. Examples

5.1. Application-generated DataCues

TODO: ...

5.2. In-band MPEG-DASH emsg events

TODO: ...

6. Acknowledgements

TODO: ...

Conformance

Document conventions

Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this specification.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes. [RFC2119]

Examples in this specification are introduced with the words “for example” or are set apart from the normative text with class="example", like this:

This is an example of an informative example.

Informative notes begin with the word “Note” and are set apart from the normative text with class="note", like this:

Note, this is an informative note.

Conformant Algorithms

Requirements phrased in the imperative as part of algorithms (such as "strip any leading space characters" or "return false and abort these steps") are to be interpreted with the meaning of the key word ("must", "should", "may", etc) used in introducing the algorithm.

Conformance requirements phrased as algorithms or specific steps can be implemented in any manner, so long as the end result is equivalent. In particular, the algorithms defined in this specification are intended to be easy to understand and are not intended to be performant. Implementers are encouraged to optimize.

Index

Terms defined by this specification

Terms defined by reference

References

Normative References

[HTML]
Anne van Kesteren; et al. HTML Standard. Living Standard. URL: https://html.spec.whatwg.org/multipage/
[MPEGCMAF]
Information technology — Multimedia application format (MPEG-A) — Part 19: Common media application format (CMAF) for segmented media. March 2020. Published. URL: https://www.iso.org/standard/79106.html
[MPEGDASH]
Information technology — Dynamic adaptive streaming over HTTP (DASH) — Part 1: Media presentation description and segment formats. December 2019. Published. URL: https://www.iso.org/standard/79329.html
[RFC2119]
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. March 1997. Best Current Practice. URL: https://tools.ietf.org/html/rfc2119
[WebIDL]
Boris Zbarsky. Web IDL. 15 December 2016. ED. URL: https://heycam.github.io/webidl/

IDL Index

[Exposed=Window]
interface DataCue : TextTrackCue {
  constructor(double startTime, unrestricted double endTime,
              any value, optional DOMString type);
  attribute any value;
  readonly attribute DOMString type;
};