Background Fetch

Editor’s Draft,

This version:
https://wicg.github.io/background-fetch/
Issue Tracking:
GitHub
Inline In Spec
Editors:
(Google)
(Google)

Abstract

An API to handle large uploads/downloads in the background with user visibility.

Status of this document

This specification was published by the Web Platform Incubator Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Contributor License Agreement (CLA) there is a limited opt-out and other conditions apply. Learn more about W3C Community and Business Groups.

1. Introduction

A service worker is capable of fetching and caching assets, the size of which is restricted only by origin storage. However, if the user navigates away from the site or closes the browser, the service worker is likely to be killed. This can happen even if there’s a pending promise passed to waitUntil(); if it hasn’t resolved within a few minutes the browser may consider it an abuse of service worker and kill the process.

This makes it difficult to download and cache large assets such as podcasts and movies, and upload video and images. Even if the service worker isn’t killed, having to keep the service worker in memory during this potentially long operation is wasteful.

This specification aims to:

2. Infrastructure

2.1. Extensions to service worker registration

A service worker registration additionally has:

To determine whether an active background fetches contains bgFetch (a background fetch), run the following steps:
  1. Let id be bgFetch’s id.

  2. If activeBgFetches[id] does not exist, then return false.

  3. If activeBgFetches[id] does not equal bgFetch, then return false.

  4. Return true.

2.2. Background fetch

A background fetch consists of:

2.3. Background fetch record

A background fetch record consists of:

3. Algorithms

3.1. Attempt a background fetch

To attempt a background fetch for bgFetch (a background fetch), run the following steps:
  1. Let swRegistration be bgFetch’s service worker registration.

  2. Let completeFetches be 0.

  3. Let abandoned be false.

  4. Let aborted be false.

  5. Let completeFetchesEditQueue be the result of starting a new parallel queue.

  6. For each record in bgFetch’s records, run the following steps in parallel:

    1. Let request be a copy of record’s request.

    2. Set request’s keepalive flag.

    3. Set request’s synchronous flag.

    4. Let response be the result of fetching request.

      Fetch waits for the whole body before returning response. Let’s pretend it doesn’t do that (issue).

    5. If response’s aborted flag is set, then set abandoned to true and aborted to true.

    6. Otherwise, if one of the following is true:

      • response failed due to a failed CORS check.

        Fetch doesn’t currently expose this (issue).

      • response is a network error and request’s method is not `GET`.

        Do we want to enable retrying of uploads? (issue)

      • TODO: other failure reasons?

      Then set abandoned to true.

    7. Otherwise, if response wasn’t a terminal failure:

      1. TODO: Resume download somehow.

    8. Otherwise:

      1. Set record’s response to a copy of response except for its body.

      2. Let stream be response’s body's stream.

      3. Whenever one or more bytes are transmitted from stream, let bytes be the transmitted bytes and run these steps:

        1. Append bytes to record’s response body.

        2. Report progress for background fetch bgFetch.

      4. If at any point the bytes transmission for stream is done normally, then enqueue the following steps to completeFetchesEditQueue:

        1. Increment completeFetches.

      5. If at any point stream becomes errored, then:

        1. If response’s aborted flag is set, then set aborted to true.

        2. Set abandoned to true. TODO: if aborted isn’t true, this could be another resume point.

  7. Wait for either abandoned to be true, or completeFetches to be bgFetch’s records's size.

  8. Let operationAborted be aborted.

  9. If abandoned is true:

    1. TODO: terminate all related fetches.

  10. Enqueue the following steps to swRegistration’s active background fetches edit queue:

    1. Let activeBgFetches be swRegistration’s active background fetches.

    2. Let id be bgFetch’s id.

    3. If activeBgFetches contains bgFetch, then remove activeBgFetches[id].

    4. Otherwise, set operationAborted to true.

      Note: This handles a race condition where abort() was successfully called but one of the fetches failed at the same time. If we’ve returned true from abort(), this ensures we fire the related abort event.

    5. If operationAborted is true, then fire a functional event named "backgroundfetchabort" using BackgroundFetchSettledEvent on swRegistration with the following properties:

      fetches

      A new BackgroundFetchSettledFetches. TODO: associate this with bgFetch’s records.

      The above prose is based on ServiceWorker/#1199.

    6. Otherwise, if abandoned is true, then fire a functional event named "backgroundfetchfail" using BackgroundFetchUpdateEvent on swRegistration with the following properties:

      fetches

      A new BackgroundFetchSettledFetches. TODO: associate this with bgFetch’s records.

    7. Otherwise, fire a functional event named "backgroundfetched" using BackgroundFetchUpdateEvent on swRegistration with the following properties:

      fetches

      A new BackgroundFetchSettledFetches. TODO: associate this with bgFetch’s records.

3.2. Report progress for background fetch

To report progress for background fetch bgFetch (a background fetch), enqueue the following steps to bgFetch’s progress handling queue:
  1. Let downloaded be 0.

  2. For each record in bgFetch’s records, increment downloaded by the number of bytes in response body.

  3. If downloaded is equal to bgFetch’s last reported download, then abort these steps.

  4. Set bgFetch’s last reported download to downloaded.

  5. For each environment settings object env whose origin is equal to bgFetch’s service worker registration's scope URL's origin, queue a task on env’s responsible event loop using the networking task source to run the following steps:

    1. Let bgFetchRegistration be the instance of BackgroundFetchRegistration within this environment whose background fetch is equal to bgFetch, or null if none exists.

      Note: There will be at most one per environment, because of the get a BackgroundFetchRegistration instance algorithm.

    2. If bgFetchRegistration is not null, then:

      1. Set downloaded to downloaded.

      2. fire an event named "progress" at bgFetchRegistration.

3.3. Get a BackgroundFetchRegistration instance

To get a BackgroundFetchRegistration instance for an instancesMap (a BackgroundFetchRegistration instances) and a bgFetch (a background fetch), run the following steps:
  1. If instancesMap[bgFetch] exists, then return instancesMap[bgFetch].

  2. Let instance be a new BackgroundFetchRegistration whose background fetch is set to bgFetch.

  3. Set instancesMap[bgFetch] to instance.

  4. Return instance.

Note: This is to ensure the same instance is returned for a given background fetch throughout the life of a BackgroundFetchManager. It’s okay for browsers to optimise this, as long as there’s no way to tell that more than one instance has been created for a given background fetch (e.g through equality, expandos, or weakly-associated data).

4. API

4.1. Extensions to ServiceWorkerGlobalScope

partial interface ServiceWorkerGlobalScope {
  attribute EventHandler onbackgroundfetched;
  attribute EventHandler onbackgroundfetchfail;
  attribute EventHandler onbackgroundfetchabort;
  attribute EventHandler onbackgroundfetchclick;
};

4.1.1. Events

The following is the event handler (and its corresponding event handler event type) that must be supported, as event handler IDL attributes, by all objects implementing ServiceWorker interface:

event handler event type event handler Interface
backgroundfetched onbackgroundfetched BackgroundFetchUpdateEvent
backgroundfetchfail onbackgroundfetchfail BackgroundFetchUpdateEvent
backgroundfetchabort onbackgroundfetchabort BackgroundFetchSettledEvent
backgroundfetchclick onbackgroundfetchclick BackgroundFetchClickEvent

4.2. Extensions to ServiceWorkerRegistration

partial interface ServiceWorkerRegistration {
  readonly attribute BackgroundFetchManager backgroundFetch;
};

Each ServiceWorkerRegistration instance has a unique instance of BackgroundFetchManager, allocated when the ServiceWorkerRegistration object is created. It’s service worker registration is set to ServiceWorkerRegistration's service worker registration.

The backgroundFetch attribute must return the BackgroundFetchManager object that is associated with the context object.

4.3. BackgroundFetchManager

[Exposed=(Window,Worker)]
interface BackgroundFetchManager {
  Promise<BackgroundFetchRegistration> fetch(DOMString id, (RequestInfo or sequence<RequestInfo>) requests, optional BackgroundFetchOptions options);
  Promise<BackgroundFetchRegistration?> get(DOMString id);
  Promise<FrozenArray<DOMString>> getIds();
  // TODO: in future this should become an async iterator for BackgroundFetchRegistration objects
};

dictionary BackgroundFetchOptions {
  sequence<IconDefinition> icons = [];
  DOMString title = "";
  unsigned long long downloadTotal = 0;
};

// This is taken from https://w3c.github.io/manifest/#icons-member.
// This definition should probably be moved somewhere more general.
dictionary IconDefinition {
  DOMString src;
  DOMString sizes = "";
  DOMString type = "";
};

A BackgroundFetchManager has:

4.3.1. fetch()

The fetch(id, requests, options) method, when invoked, run the following steps:
  1. Let registration be the context object's service worker registration.

  2. Let bgFetchInstances be context object's BackgroundFetchRegistration instances.

  3. Let records be a new list.

  4. Let uploadTotal be 0.

  5. If requests is a RequestInfo, set requests to « requests ».

  6. If requests is empty, then return a promise rejected with a TypeError.

  7. For each request of requests:

    1. Let internalRequest be the request of the result of invoking the Request constructor with request.

    2. If internalRequest’s mode is "no-cors", then return a promise rejected with a TypeError.

    3. If internalRequest’s body is not null, then:

      1. If internalRequest’s body's source is a ReadableStream, then return a promise rejected with a TypeError.

      2. Increment uploadTotal by internalRequest’s body's total bytes.

        This isn’t correctly set in fetch (issue).

    4. Set internalRequest’s client to null.

    5. Let record be a new background fetch record.

    6. Set record’s request to internalReqeust.

    7. Append record to records.

  8. Let promise be a new promise.

  9. Enqueue the following steps to registration’s active background fetches edit queue:

    1. Let bgFetchMap be registration’s active background fetches.

    2. If registration’s active worker is null, then reject promise with a TypeError and abort these steps.

    3. If bgFetchMap[id] exists, reject promise with a TypeError and abort these steps.

    4. Let bgFetch be a new background fetch with:

      id

      id.

      records

      records.

      download total

      optionsdownloadTotal member.

      upload total

      uploadTotal.

      icons

      optionsicons member.

      title

      optionstitle member.

      service worker registration

      registration.

    5. Set bgFetchMap[id] to bgFetch.

    6. If storing bgFetch fails due to exceeding a quota limit, reject promise with a QuotaExceededError DOMException and abort these steps.

    7. Resolve promise with the result of getting a BackgroundFetchRegistration instance passing bgFetchInstances and bgFetch.

    8. Attempt a background fetch with bgFetch.

  10. Return promise.

4.3.2. get()

The get(id) method, when invoked, must return a new promise promise and run the following steps in parallel:
  1. Let registration be the context object's associated service worker registration.

  2. Let bgFetch be registration’s active background fetches[id].

  3. If bgFetch is nothing, then resolve promise with undefined and abort these steps.

  4. Let downloaded be bgFetch’s last reported downloaded.

  5. Enqueue the following steps to bgFetch’s progress handling queue:

    1. Queue a task task to run the following steps:

      1. Let bgFetchRegistration be the result of getting a BackgroundFetchRegistration instance passing the context object's BackgroundFetchRegistration instances and bgFetch.

      2. Set bgFetchRegistration’s downloaded to downloaded.

      3. Resolve promise with bgFetchRegistration.

    2. Wait for task to complete.

      Note: This ensures the potential new instance of BackgroundFetchRegistration doesn’t miss any progress events.

4.3.3. getIds()

The getIds() method, when invoked, must return a new promise promise and run the following steps in parallel:
  1. Let registration be the context object's associated service worker registration.

  2. Let ids be the result of getting the keys of registration’s active background fetches.

  3. Resolve promise with ids.

4.4. BackgroundFetchRegistration

[Exposed=(Window,Worker)]
interface BackgroundFetchRegistration : EventTarget {
  readonly attribute DOMString id;
  readonly attribute unsigned long long uploadTotal;
  readonly attribute unsigned long long uploaded;
  readonly attribute unsigned long long downloadTotal;
  readonly attribute unsigned long long downloaded;
  readonly attribute BackgroundFetchActiveFetches activeFetches;

  attribute EventHandler onprogress;

  Promise<boolean> abort();
};

[Exposed=(Window,Worker)]
interface BackgroundFetchFetch {
  readonly attribute Request request;
};

[Exposed=(Window,Worker)]
interface BackgroundFetchActiveFetches {
  Promise<BackgroundFetchActiveFetch> match(RequestInfo request);
  Promise<FrozenArray<BackgroundFetchActiveFetch>> values();
};

[Exposed=(Window,Worker)]
interface BackgroundFetchActiveFetch : BackgroundFetchFetch {
  readonly attribute Promise<Response> responseReady;
  // In future this will include a fetch observer
};
A BackgroundFetchRegistration instance has an associated background fetch (a background fetch).

A BackgroundFetchRegistration instance has an associated downloaded (a number), initially 0.

A BackgroundFetchRegistration instance has an associated id, uploadTotal, and downloadTotal, which are copied from the background fetch upon instantiation.

The id attribute’s getter must return the context object's id.

The uploadTotal attribute’s getter must return the context object's uploadTotal.

The downloadTotal attribute’s getter must return the context object's downloadTotal.

The uploaded attribute’s getter must return TODO.

The downloaded attribute’s getter must return the context object's downloaded.

The activeFetches attribute’s getter must return the context object's background fetch's TODO.

4.4.1. Events

The onprogress event handler has the event handler event type of progress.

The progress event uses the Event interface.

4.4.2. abort()

The abort() method, when invoked, must return a new promise promise and run the following steps in parallel:
  1. Let bgFetch be the context object's associated background fetch.

  2. Let swRegistration be bgFetch’s service worker registration.

  3. Enqueue the following steps to swRegistration’s active background fetches edit queue:

    1. Let activeBgFetches be swRegistration’s active background fetches.

    2. Let id be bgFetch’s id.

    3. If activeBgFetches does not contain bgFetch, then resolve promise with false and abort these steps.

    4. Remove activeBgFetches[id].

    5. Resolve promise with true.

    6. TODO: Terminate related fetches with the aborted flag set.

4.4.3. BackgroundFetchEvent

[Constructor(DOMString type, BackgroundFetchEventInit init), Exposed=ServiceWorker]
interface BackgroundFetchEvent : ExtendableEvent {
  readonly attribute DOMString id;
};

dictionary BackgroundFetchEventInit : ExtendableEventInit {
  required DOMString id;
};
A BackgroundFetchEvent has an associated id, a DOMString.

The id attribute must return the id.

TODO

The BackgroundFetchEvent(type, init) constructor, when invoked, must run these steps:
  1. TODO

4.4.4. BackgroundFetchSettledEvent

[Constructor(DOMString type, BackgroundFetchSettledEventInit init), Exposed=ServiceWorker]
interface BackgroundFetchSettledEvent : BackgroundFetchEvent {
  readonly attribute BackgroundFetchSettledFetches fetches;
};

dictionary BackgroundFetchSettledEventInit : BackgroundFetchEventInit {
  required BackgroundFetchSettledFetches fetches;
};

[Exposed=ServiceWorker]
interface BackgroundFetchSettledFetches {
  Promise<BackgroundFetchSettledFetch> match(RequestInfo request);
  Promise<FrozenArray<BackgroundFetchSettledFetch>> values();
};

[Exposed=ServiceWorker]
interface BackgroundFetchSettledFetch : BackgroundFetchFetch {
  readonly attribute Response? response;
};
The fetches attribute must return TODO.
The BackgroundFetchSettledEvent(type, init) constructor, when invoked, must run these steps:
  1. TODO

4.4.5. BackgroundFetchUpdateEvent

[Constructor(DOMString type, BackgroundFetchSettledEventInit init), Exposed=ServiceWorker]
interface BackgroundFetchUpdateEvent : BackgroundFetchSettledEvent {
  Promise<void> updateUI(DOMString title);
};
The BackgroundFetchUpdateEvent(type, init) constructor, when invoked, must run these steps:
  1. TODO

The updateUI(title) method, when invoked, must return a new promise promise and run the following steps in parallel:
  1. TODO

4.4.6. BackgroundFetchClickEvent

[Constructor(DOMString type, BackgroundFetchClickEventInit init), Exposed=ServiceWorker]
interface BackgroundFetchClickEvent : BackgroundFetchEvent {
  readonly attribute BackgroundFetchState state;
};

dictionary BackgroundFetchClickEventInit : BackgroundFetchEventInit {
  required BackgroundFetchState state;
};

enum BackgroundFetchState { "pending", "succeeded", "failed" };

5. Privacy and bandwidth usage

It’s the recommendation of this standard that user agents make background fetch operations highly visible and easily abortable rather than asking permission up front. However, given that background fetches can be paused and retried, a user agent that immediately paused a background fetch and user interaction to resume would be compliant.

The origin of a background fetch should have equal or great visibility to the developer-provided information, such as a the icons and title.

A user agent may pause background downloads depending on network conditions or battery state. For example, a user agent may download automatically when connected to WiFi, but require user confirmation to download using mobile data. If an implementation has this restriction, the user should be able to persist granting mobile data use for the origin.

If the user agent provides a way for a user to abort a background fetch operation, it should terminate all fetches associated with the background fetch's records, with the abort flag set.

A background fetch exposes the user’s external IPs to the server throughout the duration of the background fetch. An attacker could use an extremely slow background fetch as a way to persistently track the user once they’ve left the site. Good visibility into the progress of the background fetch will help the user identify unwanted use.

Conformance

Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this specification.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes. [RFC2119]

Examples in this specification are introduced with the words “for example” or are set apart from the normative text with class="example", like this:

This is an example of an informative example.

Informative notes begin with the word “Note” and are set apart from the normative text with class="note", like this:

Note, this is an informative note.

Index

Terms defined by this specification

Terms defined by reference

References

Normative References

[DOM]
Anne van Kesteren. DOM Standard. Living Standard. URL: https://dom.spec.whatwg.org/
[FETCH]
Anne van Kesteren. Fetch Standard. Living Standard. URL: https://fetch.spec.whatwg.org/
[HTML]
Anne van Kesteren; et al. HTML Standard. Living Standard. URL: https://html.spec.whatwg.org/multipage/
[INFRA]
Anne van Kesteren; Domenic Denicola. Infra Standard. Living Standard. URL: https://infra.spec.whatwg.org/
[PROMISES-GUIDE]
Domenic Denicola. Writing Promise-Using Specifications. 16 February 2016. Finding of the W3C TAG. URL: https://www.w3.org/2001/tag/doc/promises-guide
[RFC2119]
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. March 1997. Best Current Practice. URL: https://tools.ietf.org/html/rfc2119
[SERVICE-WORKERS-1]
Alex Russell; et al. Service Workers 1. 11 October 2016. WD. URL: https://www.w3.org/TR/service-workers-1/
[URL]
Anne van Kesteren. URL Standard. Living Standard. URL: https://url.spec.whatwg.org/
[WebIDL]
Cameron McCormack; Boris Zbarsky; Tobie Langel. Web IDL. 15 December 2016. ED. URL: https://heycam.github.io/webidl/

IDL Index

partial interface ServiceWorkerGlobalScope {
  attribute EventHandler onbackgroundfetched;
  attribute EventHandler onbackgroundfetchfail;
  attribute EventHandler onbackgroundfetchabort;
  attribute EventHandler onbackgroundfetchclick;
};

partial interface ServiceWorkerRegistration {
  readonly attribute BackgroundFetchManager backgroundFetch;
};

[Exposed=(Window,Worker)]
interface BackgroundFetchManager {
  Promise<BackgroundFetchRegistration> fetch(DOMString id, (RequestInfo or sequence<RequestInfo>) requests, optional BackgroundFetchOptions options);
  Promise<BackgroundFetchRegistration?> get(DOMString id);
  Promise<FrozenArray<DOMString>> getIds();
  // TODO: in future this should become an async iterator for BackgroundFetchRegistration objects
};

dictionary BackgroundFetchOptions {
  sequence<IconDefinition> icons = [];
  DOMString title = "";
  unsigned long long downloadTotal = 0;
};

// This is taken from https://w3c.github.io/manifest/#icons-member.
// This definition should probably be moved somewhere more general.
dictionary IconDefinition {
  DOMString src;
  DOMString sizes = "";
  DOMString type = "";
};

[Exposed=(Window,Worker)]
interface BackgroundFetchRegistration : EventTarget {
  readonly attribute DOMString id;
  readonly attribute unsigned long long uploadTotal;
  readonly attribute unsigned long long uploaded;
  readonly attribute unsigned long long downloadTotal;
  readonly attribute unsigned long long downloaded;
  readonly attribute BackgroundFetchActiveFetches activeFetches;

  attribute EventHandler onprogress;

  Promise<boolean> abort();
};

[Exposed=(Window,Worker)]
interface BackgroundFetchFetch {
  readonly attribute Request request;
};

[Exposed=(Window,Worker)]
interface BackgroundFetchActiveFetches {
  Promise<BackgroundFetchActiveFetch> match(RequestInfo request);
  Promise<FrozenArray<BackgroundFetchActiveFetch>> values();
};

[Exposed=(Window,Worker)]
interface BackgroundFetchActiveFetch : BackgroundFetchFetch {
  readonly attribute Promise<Response> responseReady;
  // In future this will include a fetch observer
};

[Constructor(DOMString type, BackgroundFetchEventInit init), Exposed=ServiceWorker]
interface BackgroundFetchEvent : ExtendableEvent {
  readonly attribute DOMString id;
};

dictionary BackgroundFetchEventInit : ExtendableEventInit {
  required DOMString id;
};

[Constructor(DOMString type, BackgroundFetchSettledEventInit init), Exposed=ServiceWorker]
interface BackgroundFetchSettledEvent : BackgroundFetchEvent {
  readonly attribute BackgroundFetchSettledFetches fetches;
};

dictionary BackgroundFetchSettledEventInit : BackgroundFetchEventInit {
  required BackgroundFetchSettledFetches fetches;
};

[Exposed=ServiceWorker]
interface BackgroundFetchSettledFetches {
  Promise<BackgroundFetchSettledFetch> match(RequestInfo request);
  Promise<FrozenArray<BackgroundFetchSettledFetch>> values();
};

[Exposed=ServiceWorker]
interface BackgroundFetchSettledFetch : BackgroundFetchFetch {
  readonly attribute Response? response;
};

[Constructor(DOMString type, BackgroundFetchSettledEventInit init), Exposed=ServiceWorker]
interface BackgroundFetchUpdateEvent : BackgroundFetchSettledEvent {
  Promise<void> updateUI(DOMString title);
};

[Constructor(DOMString type, BackgroundFetchClickEventInit init), Exposed=ServiceWorker]
interface BackgroundFetchClickEvent : BackgroundFetchEvent {
  readonly attribute BackgroundFetchState state;
};

dictionary BackgroundFetchClickEventInit : BackgroundFetchEventInit {
  required BackgroundFetchState state;
};

enum BackgroundFetchState { "pending", "succeeded", "failed" };

Issues Index

Fetch waits for the whole body before returning response. Let’s pretend it doesn’t do that (issue).
Fetch doesn’t currently expose this (issue).
Do we want to enable retrying of uploads? (issue)
The above prose is based on ServiceWorker/#1199.
This isn’t correctly set in fetch (issue).