When playing audio through WebAudio, we want to be able to measure the delay of that audio and the glitchiness of the audio. This document contains a proposal of an API that would allow WebAudio users to do this.

This is an unofficial proposal.

Introduction

There is currently no way to detect whether WebAudio playout has glitches (gaps in the played audio, which typically happens due to underperformance in the audio pipeline). There is an existing way to measure the instantaneous playout latency using AudioContext.outputLatency, but no simple way to measure average/minimum/maximum latency over time.

Glitches and high latency are bad for the user experience, so if any of these occur it can be useful for the application to be able to detect this and possibly take some action to improve the playout.

Extension of the AudioContext interface 

    partial interface AudioContext {
      [SameObject] readonly attribute AudioPlayoutStats playoutStats;
    };

Attributes

playoutStats attribute

The {{AudioContext/AudioPlayoutStats}} under AudioContext is a dedicated object for statistics reporting; Similar to RTCAudioPlayoutStats, but it is for the playout path via AudioDestinationNode and the associated output device. This will allow us to measure glitches occurring due to underperforming AudioWorklets as well as glitches and delay occurring in the playout path between the AudioContext and the output device.

AudioPlayoutStats interface

    [Exposed=Window, SecureContext]
    interface AudioPlayoutStats {
      readonly attribute DOMHighResTimeStamp fallbackFramesDuration;
      readonly attribute unsigned long fallbackFramesEvents;
      readonly attribute DOMHighResTimeStamp totalFramesDuration;
      readonly attribute DOMHighResTimeStamp averageLatency;
      readonly attribute DOMHighResTimeStamp minimumLatency;
      readonly attribute DOMHighResTimeStamp maximumLatency;
      undefined resetLatency();
      [Default] object toJSON();
    };

Attributes

fallbackFramesDuration attribute

This value is measured in milliseconds and is incremented each time a fallback frame is played by the output device at the end of the playout path. This metric can be used together with {{AudioPlayoutStats/totalFramesDuration}} to calculate the percentage of played out media that was not provided by the AudioContext.

fallbackFramesEvents attribute

This measures the number of synthesized fallback frames events. This counter increases every time a fallback frame is played after a non-fallback frame. That is, multiple consecutive fallback frames will increase {{AudioPlayoutStats/fallbackFramesDuration}} multiple times but is a single fallback frames event.

totalFramesDuration attribute

This attribute is the total duration, in milliseconds, of all audio frames that have been played by the audio device. Includes both fallback and non-fallback frames.

averageLatency attribute

This is the average latency for the frames played since the last call to {{AudioPlayoutStats/resetLatency()}}, or since the creation of the AudioContext if {{AudioPlayoutStats/resetLatency()}} has not been called.

minimumLatency attribute

This measures the minimum latency for the frames played since the last call to {{AudioPlayoutStats/resetLatency()}}, or since the creation of the AudioContext if {{AudioPlayoutStats/resetLatency()}} has not been called.

maximumLatency attribute

This measures the maximum latency for the frames played since the last call to {{AudioPlayoutStats/resetLatency()}}, or since the creation of the AudioContext if {{AudioPlayoutStats/resetLatency()}} has not been called.

Methods

resetLatency method

This method resets the latency counters. Note that it does not remove latency information that has accrued but not yet been exposed through the API.

Usage Example

This is an example of how the API can be used to calculate the following stats over a time interval: 
var oldTotalFramesDuration = audioContext.playoutStats.totalFramesDuration;
var oldFallbackFramesDuration = audioContext.playoutStats.fallbackFramesDuration;
var oldFallbackFramesEvents = audioContext.playoutStats.fallbackFramesEvents;
audioContext.playoutStats.resetLatency();

// Wait while playing audio
...

// the number of seconds that were covered by the frames played by the output device between the two executions.
let deltaTotalFramesDuration = (audioContext.playoutStats.totalFramesDuration - oldTotalFramesDuration) / 1000;
let deltaFallbackFramesDuration = (audioContext.playoutStats.fallbackFramesDuration - oldFallbackFramesDuration) / 1000;
let deltaFallbackFramesEvents = audioContext.playoutStats.fallbackFramesEvents - oldFallbackFramesEvents;

// fallback frames fraction stat over the last deltaTotalFramesDuration seconds
let fallbackFramesFraction = deltaFallbackFramesDuration / deltaTotalFramesDuration;
// fallback event frequency stat over the last deltaTotalFramesDuration seconds
let fallbackEventFrequency = deltaFallbackFramesEvents / deltaTotalFramesDuration;
// Average playout delay stat during the last deltaTotalFramesDuration seconds
let playoutDelay = audioContext.playoutStats.averageLatency / 1000;