WebUSB Testing API

Draft Community Group Report,

This version:
https://wicg.github.io/webusb/test/
Issue Tracking:
GitHub
Editors:
(Google LLC)
(Google LLC)
Participate:
Join the W3C Community Group
IRC: #webusb on W3C’s IRC (Stay around for an answer, it make take a while)

Abstract

This document describes an API for testing a User Agent’s implementation of the WebUSB API.

Status of this document

This specification was published by the Web Platform Incubator Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Contributor License Agreement (CLA) there is a limited opt-out and other conditions apply. Learn more about W3C Community and Business Groups.

1. Introduction

This section is non-normative.

Standards such as [WebUSB] pose a challenge to test authors because to fully exercise their interfaces requires physical hardware devices that respond in predictable ways. To address this challenge this specification defines an interface for controlling a simulation of the USB subsystem on the host the User Agent (UA) is running on. With this interface devices with particular properties can be created and their responses to requests are well defined.

The purpose if this interface is to assist the developers of UAs and so this interface does not permit arbitrary control over the behavior of devices. Some parameters are configurable and some are specified.

The only intended client of this API are tests in Web Platform Tests. This testing API will be changed as needed to support the evolution of those tests.

Testing applications that use WebUSB is specifically not in scope for this API. It not designed for that purpose. And, the ability to make breaking changes to this API has higher priority.

1.1. Examples

This example, expressed as a W3C testharness.js-based test, demonstrates initializing this API and using it to verify that devices can be added and removed.
let fakeDeviceInit = {
  usbVersionMajor: 2,
  usbVersionMinor: 0,
  usbVersionSubminor: 0,
  deviceClass: 0xFF,
  deviceSubclass: 0xFF,
  deviceProtocol: 0xFF,
  vendorId: 0x1234,
  productId: 0xABCD,
  deviceVersionMajor: 1,
  deviceVersionMinor: 0,
  deviceVersionSubminor: 0,
  configurations: []
};

function promiseForEvent(eventTarget, eventType) {
  return new Promise(resolve => {
    let eventHandler = evt => {
      resolve(evt);
      eventTarget.removeEventListener(eventTarget, eventHandler);
    };
    eventTarget.addEventListener(eventType);
  });
}

promise_test(async () => {
  await navigator.usb.test.initialize();

  let fakeDevice = navigator.usb.addFakeDevice(fakeDeviceInit);
  let connectEvent = await promiseForEvent(navigator.usb, 'connect');
  let device = connectEvent.device;
  assert_equals(device.usbVersionMajor, fakeDeviceInit.usbVersionMajor);
  assert_equals(device.usbVersionMinor, fakeDeviceInit.usbVersionMinor);
  assert_equals(device.usbVersionSubminor, fakeDeviceInit.usbVersionSubminor);
  assert_equals(device.deviceClass, fakeDeviceInit.deviceClass);
  assert_equals(device.deviceSubclass, fakeDeviceInit.deviceSubclass);
  assert_equals(device.deviceProtocol, fakeDeviceInit.deviceProtocol);
  assert_equals(device.vendorId, fakeDeviceInit.vendorId);
  assert_equals(device.productId, fakeDeviceInit.productId);
  assert_equals(device.deviceVersionMajor, fakeDeviceInit.deviceVersionMajor);
  assert_equals(device.deviceVersionMinor, fakeDeviceInit.deviceVersionMinor);
  assert_equals(device.deviceVersionSubminor, fakeDeviceInit.deviceVersionSubminor);
  assert_equals(device.configuration, null);
  assert_equals(device.configurations.length, 0);

  let devices = await navigator.usb.getDevices();
  assert_array_equals(devices, [device]);

  fakeDevice.disconnect();
  let disconnectEvent = await promiseForEvent(navigator.usb, 'disconnect');
  assert_equals(disconnectEvent.device, device);
});
This example, which reuses definitions from the previous example and is also expressed as a W3C testharness.js-based test, demonstrates a test of the navigator.usb.requestDevice() method.
promise_test(async () => {
  await navigator.usb.test.initialize();
  let fakeDevice = navigator.usb.addFakeDevice(fakeDeviceInit);
  let connectEvent = await promiseForEvent(navigator.usb, 'connect');
  navigator.usb.test.onrequestdevice = () => connectEvent.device;

  let options = { filters: [{ vendorId: 0x1234 }] };
  let device = await navigator.usb.requestDevice(options);
  assert_array_equals(navigator.usb.test.lastFilters, options.filters);
  assert_equals(device, connectEvent.device);
}

Note: If multiple tests are run in the same document then the test harness should invoke reset() and wait until the Promise it returns is resolved before executing the next test.

2. Availability

This specification defines an interface that is not intended be used by non-testing-related web content. The UA MAY choose to expose this interface only when a runtime or compile-time flag has been set.

Note, as an example, in the Chromium Project this specification is implemented by a JavaScript polyfill on top of a lower-level interface. This interface is only available in a binary explicitly built for running web platform test cases. A runtime flag is also necessary to enable this testing mode. This design has two benefits,
  1. The risk of introducing a security vulnerability in the default configuration is mitigated.

  2. The polyfill is not shipped with the production application and so there is no increase in binary size to support an API that will rarely be used.

3. Global Testing Interface

partial interface USB {
  [SameObject] readonly attribute USBTest test;
};

[Exposed=(Window,Worker)]
interface USBDeviceRequestEvent : Event {
  attribute FrozenArray<USBDeviceFilter> filters;

  undefined respondWith(Promise<FakeUSBDevice> result);
};

[Exposed=(Window,Worker)]
interface USBTest {
  attribute EventHandler onrequestdevice;

  Promise<undefined> initialize();
  Promise<undefined> attachToContext((HTMLIFrameElement or Worker) context);
  FakeUSBDevice addFakeDevice(FakeUSBDeviceInit deviceInit);
  Promise<undefined> reset();
};

By default, a UA SHALL NOT alter the behavior of a USB instance usb in any global object until it is reconfigured so that usb is controlled by a USBTest instance test. At that point the behavior of usb is defined by this specification.

Instances of USBTest are created with an internal slot [[initializationPromise]] with an initial value of null.

When invoked, the initialize() method MUST run these steps:

  1. Let test be the USBTest instance on which this method was invoked.

  2. If test.[[initializationPromise]] is null then set test.[[initializationPromise]] to a new Promise and run these sub-steps in parallel:

    1. Reconfigure the UA’s internal implementation of the usb object in the current global object so that it is controlled by test.

    2. Resolve test.[[initializationPromise]].

  3. Return test.[[initializationPromise]].

When invoked, the attachToContext(context) method MUST return a new Promise promise and run these steps in parallel:

  1. Let test be the USBTest instance on which this method was invoked.

  2. If test.[[initializationPromise]] is not in the resolved state, reject promise with an InvalidStateError and abort these steps.

  3. Reconfigure the UA’s internal implementation of usb in the global object associated with context so that it is controlled by test.

  4. Resolve promise.

When invoked, the addFakeDevice(deviceInit) method MUST run these steps:

  1. Let test be the USBTest instance on which this method was invoked.

  2. If test.[[initializationPromise]] is not in the resolved state, raise an InvalidStateError and abort these steps.

  3. Let fakeDevice be a new FakeUSBDevice.

  4. Queue a task to, for each USB instance controlled by test, perform the steps described in [WebUSB] for handling a new device that is connected to the system.

  5. Return fakeDevice.

When invoked, the reset() method MUST return a new Promise promise and run the following steps in parallel:

  1. Let test be the USBTest instance on which this method was invoked.

  2. For each FakeUSBDevice fakeDevice previously returned by addFakeDevice(), invoke fakeDevice.disconnect().

  3. Resolve promise.

USBDeviceRequestEvent instances have an internal slot [[promise]] that holds a Promise.

When invoked, the respondWith(result) method MUST run the following steps in parallel:

  1. Wait until result settles.

  2. Let event be the USBDeviceRequestEvent instance on which this method was invoked.

  3. If result resolved with response and response is an instance of FakeUSBDevice resolve event@[[promise]] with device.

  4. Otherwise, reject event@[[promise]] with a NotFoundError.

3.1. USB Behavior

When requestDevice(options) is invoked on a USB instance controlled by a USBTest test the UA MUST return a new Promise promise and perform the following steps in parallel:

  1. Let event be a new USBDeviceRequestEvent.

  2. Set event.filters to a new FrozenArray.

  3. Copy the members of options.filters into event.filters.

  4. Set event@[[promise]] to promise.

  5. Fire an event named requestdevice on test, using event as the event object.

3.2. Events

requestdevice: Fired on a USBTest object when requestDevice(options) is invoked after initialize() has been called.

4. Fake Devices

To permit testing without physical hardware this specification defines a method for tests to add simulated USB devices by calling addFakeDevice() with an instance of FakeUSBDeviceInit containing the properties of the device to be added.

[Exposed=(Window,Worker)]
interface FakeUSBDevice : EventTarget {
  attribute EventHandler onclose;

  undefined disconnect();
};

dictionary FakeUSBDeviceInit {
  required octet usbVersionMajor;
  required octet usbVersionMinor;
  required octet usbVersionSubminor;
  required octet deviceClass;
  required octet deviceSubclass;
  required octet deviceProtocol;
  required unsigned short vendorId;
  required unsigned short productId;
  required octet deviceVersionMajor;
  required octet deviceVersionMinor;
  required octet deviceVersionSubminor;
  DOMString? manufacturerName;
  DOMString? productName;
  DOMString? serialNumber;
  octet activeConfigurationValue = 0;
  sequence<FakeUSBConfigurationInit> configurations;
};

dictionary FakeUSBConfigurationInit {
  required octet configurationValue;
  DOMString? configurationName;
  sequence<FakeUSBInterfaceInit> interfaces;
};

dictionary FakeUSBInterfaceInit {
  required octet interfaceNumber;
  sequence<FakeUSBAlternateInterfaceInit> alternates;
};

dictionary FakeUSBAlternateInterfaceInit {
  required octet alternateSetting;
  required octet interfaceClass;
  required octet interfaceSubclass;
  required octet interfaceProtocol;
  DOMString? interfaceName;
  sequence<FakeUSBEndpointInit> endpoints;
};

dictionary FakeUSBEndpointInit {
  required octet endpointNumber;
  required USBDirection direction;
  required USBEndpointType type;
  required unsigned long packetSize;
};

When a USBDevice device is initialized from an fake USB device described by a FakeUSBDeviceInit init passed to addFakeDevice() the attributes of device SHALL be initialized as follows and device will correspond to the FakeUSBDevice returned by addFakeDevice():

  1. For each non-sequence attribute of init other than activeConfigurationValue the attribute of device with the same name SHALL be set to its value.

  2. For each sequence of FakeUSBConfigurationInit, FakeUSBInterfaceInit FakeUSBAlternateInterfaceInit and FakeUSBEndpointInit objects corresponding USBConfiguration, USBInterface, USBAlternateInterface and USBEndpoint objects SHALL be created by similarly copying attributes with the same names and be used to build an identical hierarchy of objects in the configurations, interfaces, alternates and endpoints FrozenArrays.

  3. If a USBConfiguration instance config with configurationValue equal to init.activeConfigurationValue then device.configuration SHALL be set to config, otherwise null.

When invoked, the disconnect() method MUST, queue a task to, for each USB instance controlled by the USBTest instance from which target of this invocation was returned, perform the steps described in [WebUSB] for handling the removal of a device that was connected to the system.

4.1. USBDevice Behavior

When the open(), close(), selectConfiguration(), claimInterface(), releaseInterface(), selectAlternateInterface(), clearHalt() and reset() methods are invoked on a USBDevice device corresponding to a FakeUSBDevice the UA MUST behave as though device contains the configurations, interfaces and endpoints described in the FakeUSBDeviceInit from which device was initialized exist and can be claimed by the caller.

When close() is invoked on a USBDevice device the UA MUST fire an event named close at the FakeUSBDevice instance corresponding to device, if one exists.

When controlTransferIn(setup, length) is invoked on a USBDevice device corresponding to a FakeUSBDevice the UA MUST, assuming all other pre-conditions for the operation are satisfied, behave as though the device responded with a packet containing the bytes [length >> 8, length & 0xFF, setup.request, setup.value >> 8, setup.value & 0xFF, setup.index >> 8, setup.index & 0xFF], truncated to length bytes.

When controlTransferOut(setup, data) is invoked on a USBDevice device corresponding to a FakeUSBDevice the UA MUST, assuming all other pre-conditions for the operation are satisfied, behave as though the transfer succeeded in sending data.length bytes.

When transferIn(endpointNumber, length) is invoked on a USBDevice device corresponding to a FakeUSBDevice the UA MUST, assuming all other pre-conditions for the operation are satisfied, behave as though the device responded with length bytes of data consisting of the values 0 through 255 repeated as necessary.

When transferOut(endpointNumber, data) is invoked on a USBDevice device corresponding to a FakeUSBDevice the UA MUST, assuming all other pre-conditions for the operation are satisfied, behave as though the transfer succeeded in sending data.length bytes.

When isochronousTransferIn(endpointNumber, packetLengths) is invoked on a USBDevice device corresponding to a FakeUSBDevice the UA MUST, assuming all other pre-conditions for the operation are satisfied, behave as though the device responded with packetLengths.length packets, each containing packetLengths[i] bytes of data consisting of the values 0 through 255 repeated as necessary.

When isochronousTransferOut(endpointNumber, data, packetLengths) is invoked on a USBDevice device corresponding to a FakeUSBDevice the UA MUST, assuming all other pre-conditions for the operation are satisfied, behave as though the transfer succeeded in sending packetLengths.length packets, each containing packetLengths[i] bytes.

Conformance

Document conventions

Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this specification.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes. [RFC2119]

Examples in this specification are introduced with the words “for example” or are set apart from the normative text with class="example", like this:

This is an example of an informative example.

Informative notes begin with the word “Note” and are set apart from the normative text with class="note", like this:

Note, this is an informative note.

Conformant Algorithms

Requirements phrased in the imperative as part of algorithms (such as "strip any leading space characters" or "return false and abort these steps") are to be interpreted with the meaning of the key word ("must", "should", "may", etc) used in introducing the algorithm.

Conformance requirements phrased as algorithms or specific steps can be implemented in any manner, so long as the end result is equivalent. In particular, the algorithms defined in this specification are intended to be easy to understand and are not intended to be performant. Implementers are encouraged to optimize.

Index

Terms defined by this specification

Terms defined by reference

References

Normative References

[DOM]
Anne van Kesteren. DOM Standard. Living Standard. URL: https://dom.spec.whatwg.org/
[ECMASCRIPT]
ECMAScript Language Specification. URL: https://tc39.es/ecma262/multipage/
[HTML]
Anne van Kesteren; et al. HTML Standard. Living Standard. URL: https://html.spec.whatwg.org/multipage/
[RFC2119]
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. March 1997. Best Current Practice. URL: https://datatracker.ietf.org/doc/html/rfc2119
[WebIDL]
Boris Zbarsky. Web IDL. 15 December 2016. ED. URL: https://heycam.github.io/webidl/
[WebUSB]
WebUSB API. cg-draft. URL: https://wicg.github.io/webusb/

IDL Index

partial interface USB {
  [SameObject] readonly attribute USBTest test;
};

[Exposed=(Window,Worker)]
interface USBDeviceRequestEvent : Event {
  attribute FrozenArray<USBDeviceFilter> filters;

  undefined respondWith(Promise<FakeUSBDevice> result);
};

[Exposed=(Window,Worker)]
interface USBTest {
  attribute EventHandler onrequestdevice;

  Promise<undefined> initialize();
  Promise<undefined> attachToContext((HTMLIFrameElement or Worker) context);
  FakeUSBDevice addFakeDevice(FakeUSBDeviceInit deviceInit);
  Promise<undefined> reset();
};

[Exposed=(Window,Worker)]
interface FakeUSBDevice : EventTarget {
  attribute EventHandler onclose;

  undefined disconnect();
};

dictionary FakeUSBDeviceInit {
  required octet usbVersionMajor;
  required octet usbVersionMinor;
  required octet usbVersionSubminor;
  required octet deviceClass;
  required octet deviceSubclass;
  required octet deviceProtocol;
  required unsigned short vendorId;
  required unsigned short productId;
  required octet deviceVersionMajor;
  required octet deviceVersionMinor;
  required octet deviceVersionSubminor;
  DOMString? manufacturerName;
  DOMString? productName;
  DOMString? serialNumber;
  octet activeConfigurationValue = 0;
  sequence<FakeUSBConfigurationInit> configurations;
};

dictionary FakeUSBConfigurationInit {
  required octet configurationValue;
  DOMString? configurationName;
  sequence<FakeUSBInterfaceInit> interfaces;
};

dictionary FakeUSBInterfaceInit {
  required octet interfaceNumber;
  sequence<FakeUSBAlternateInterfaceInit> alternates;
};

dictionary FakeUSBAlternateInterfaceInit {
  required octet alternateSetting;
  required octet interfaceClass;
  required octet interfaceSubclass;
  required octet interfaceProtocol;
  DOMString? interfaceName;
  sequence<FakeUSBEndpointInit> endpoints;
};

dictionary FakeUSBEndpointInit {
  required octet endpointNumber;
  required USBDirection direction;
  required USBEndpointType type;
  required unsigned long packetSize;
};