Subresource Loading with Web Bundles

1. Introduction

This section is non-normative.

The Subresource Loading with Web Bundles specification describes a way to load a large number of resources efficiently using a format that allows multiple resources to be bundled, Web Bundles. This specification describes how web browsers load those resources. It is expressed as several monkeypatches to the [HTML], [FETCH] and [CSP] specification which call algorithms defined here.

Note: This specification is under construction. See #708.

2. Structures

A fetched web bundle is a representation of a web bundle format defined in [draft-ietf-wpack-bundled-responses-latest].

A web bundle fetch entry is a struct with the following items:

• source, a URL of a web bundle.

• credentials, a credentials mode.

• state, an internal state which is "fetching", "fetched", or "failed". Initially "fetching".

• fetched bundle, a fetched web bundle or null.

A better name for web bundle fetch entry?

A web bundle fetch entry entry is used by a registration in Document document if document’s web bundle registration list contains a web bundle registration whose fetch entry is entry.

A bundle rule is a struct with the following items:

• resources, a list of URLs.

• scopes, a list of URLs.

A web bundle registration is a struct with the following items:

• fetch entry, a web bundle fetch entry.

• rule, a bundle rule.

A web bundle parse result is a struct with the following items:

• source, a URL of a web bundle.

• credentials, a credentials mode.

• rule, a bundle rule.

Each environment settings object will get a web bundle registration list algorithm, which returns a list of web bundle registrations.

A Document has a web bundle registration list, which is a list of web bundle registrations. It is initially empty.

In set up a window environment settings object, settings object’s web bundle registration list returns the web bundle registration list of window’s associated Document.

In set up a worker environment settings object, settings object’s web bundle registration list returns an empty list.

A Document has a web bundle fetch entry list, which is a list of web bundle fetch entries. It is initially empty.

While list is used for web bundle fetch entry list, the order shouldn’t be important.

Not supported for workers.

3. HTML monkeypatches

To process web bundles in the prepare the script element algorithm consistently with existing script types (i.e. classic or module), we make the following changes:

• the script’s type should be either "classic", "module", or "webbundle"

• Introduce web bundle result, which is a struct with two items:

  • a registration, a web bundle registration; and

  • an error to rethrow, a JavaScript value representing a parse error when non-null.

• the script’s result is either "uninitiated", null, script, or a web bundle result.

Note: Because we don’t make web bundle result a new subclass of script, other script execution-related specs are left unaffected.

3.1. Prepare the script element

Inside the prepare the script element algorithm, we make the following changes:

• Insert the following step to prepare the script element step 10:

  • If the script block’s type string is an ASCII case-insensitive match for the string "webbundle", then set el’s type to "webbundle".

• Insert the following case to prepare the script element step 26.7:

  • "webbundle":

    1. Queue a task to fire an event named error at the element, and return.

       Note: <script type="webbundle" src=...> is not supported. There are no specific requirements for the error handling here, so currently an error event is fired similarly to the case of an empty src attribute.

• Insert the following case to prepare the script element step 27.2:

  • "webbundle":

    1. Prepare a web bundle given element, source text and base URL.

• Insert the following case to prepare the script element step 28:

  • If the script’s type is "webbundle":

    1. Assert: script’s ready to be parser-executed is true.

    2. In parallel, process events for a web bundle given element.

NOTE: CSPs are applied to inline web bundles at Step 15 of prepare the script element, just like applied to classic/module scripts.

To prepare a web bundle, given an HTMLScriptElement element, a string sourceText and a URL baseURL:

1. Let parse result be the result of parse a web bundle string given sourceText and baseURL.

2. If this throws an exception:

   1. Set the script’s result to a new web bundle result with its registration is null and its error to rethrow is the exception thrown.

   2. Mark as ready.

   3. Return.

3. Let document be element’s node document.

4. Set fetch entry to null.

5. For each r in document’s web bundle fetch entry list:

   1. If r’s source is parse result’s source and r’s credentials is parse result’s credentials, then:

      1. If r is not used by a registration in document, then set fetch entry to r.

         NOTE: This implies that another script element whose the script’s result's registration's fetch entry is r was removed. This is to ensure that web bundle fetch entries are not destructed and re-fetched when HTMLScriptElements with the same web bundle source and credentials are removed and added.

6. If fetch entry is null:

   1. Set fetch entry to a new web bundle fetch entry with its source is parse result’s source, its credentials is parse result’s credentials, its state is "fetching", and its fetched bundle is null.

   2. Append fetch entry to document’s web bundle fetch entry list.

   3. In parallel, fetch a web bundle fetch entry.

7. Let registration be a new web bundle registration with its fetch entry is fetch entry and its rule is parse result’s rule.

8. Append registration to document’s web bundle registration list.

9. Set the script’s result to a new web bundle result with its registration is registration and its error to rethrow is null.

10. Mark as ready.

3.2. Firing events

In execute the script element, add the following case to Step 6:

• "webbundle":

  1. Assert: Never reached.

     Note: Web bundles are processed by process events for a web bundle instead of execute the script element.

To process events for a web bundle given an HTMLScriptElement element:

1. Let result be the script’s result of element.

2. Assert: element’s the script’s type is "webbundle".

3. Assert: result is an web bundle result.

4. Await asynchronously until either of the following conditions met:

   • result’s error to rethrow is not null, or

   • result’s registration is not null and result’s registration's fetch entry's state becomes "fetched" or "failed".

   Note: Unlike other script types, we wait asynchronously here for fetch web bundle before firing load events at HTMLScriptElement. We don’t delay the load event here, because we mark as ready synchronously in prepare the script element. This is intentional because fetch a web bundle is similar to preloading.

5. If the script’s result of element is null, then return.

   Note: This can happen when element was removed from the document during the previous step.

   Note: This is specified consistently with whatwg/html#2673. Currently we don’t fire error events in this case. If we change the decision at whatwg/html#2673 to fire error events, then change this step accordingly.

6. Assert: element’s node document is equal to element’s preparation-time document.

7. If result’s error to rethrow is not null, then:

   1. Report the exception given result’s error to rethrow.

      There are no relevant script, because web bundle result isn’t a script. This needs to wait for whatwg/html#958 before it is fixable.

   2. Return.

8. Assert: result’s registration is not null.

9. If result’s registration's fetch entry's state is "failed":

   1. Fire an event named error at element.

   2. Return.

10. Assert: result’s registration's fetch entry's state is "fetched".

11. Fire an event named load at element.

3.3. Removing

If script element is removed from the document, user agents must run the following algorithm:

1. If the script’s type is not "webbundle", then return.

2. If the script’s result is null, then return.

3. Assert: the script’s result is an web bundle result.

4. Let registration be the script’s result's registration.

5. Set the script’s result to null.

6. If registration is null, then return.

7. Let document be the node document.

8. Assert: document’s web bundle registration list contains registration.

9. Remove registration from document’s web bundle registration list.

10. Queue a microtask to perform the following steps:

    1. Let fetch entry be registration’s fetch entry.

    2. If fetch entry is used by a registration in document, then return.

    3. Remove fetch entry from document’s web bundle fetch entry list.

       Note: It is possible that document’s web bundle fetch entry list doesn’t contain fetch entry even before this step, if fetch entry is used by web bundle registrations of multiple script elements and the script elements are removed.

       Note: At this point, fetch entry can no longer used by subsequent subresource fetches nor subsequent prepare a web bundle calls, but its fetched bundle can be still in use by ongoing fetches.

4. Fetch monkeypatches

4.1. Monkeypatch fetch

In fetch, before

2. Let taskDestination be null.

add the following step:

2. If the result of find a matching web bundle registration given request is null, set request’s service-workers mode to "none".

Note: This means that no service workers will get events for a subresource loading from a webbundle.

4.2. Monkeypatch fetch scheme

Add "uuid-in-package" to the schemes listed in fetch scheme.

Note: This ensures that the navigate algorithm uses the process a navigate fetch algorithm for uuid-in-package: URLs.

Note: The origin of a URL whose scheme is "uuid-in-package" is an opaque origin.

4.3. Monkeypatch HTTP-network-or-cache fetch

In HTTP-network-or-cache fetch, before

8.22. Set httpCache to the result of determining the HTTP cache partition, given httpRequest.

add the following steps:

22. Set the response to the result of fetch a subresource from web bundle, given httpRequest.

    1. If response is network error, return network error.

    2. If response is non-null, skip the steps 8.22-8.24 and goto the step 9.

       Note: That means a subresource from a webbundle never interacts with HttpCache. We plan to support HttpCache as a feature enhancement in the future.

5. CSP monkeypatches

5.1. Monkeypatch Does request match source list?

Rewrite Does request match source list? to run these steps:

1. Let url be request’s current url.

2. If url’s scheme is "uuid-in-package", then:

   1. Let registration be the result of running find a matching web bundle registration given request.

   2. If registration is not null, then set url to registration’s fetch entry's source.

3. Returns the result of executing Does url match source list in origin with redirect count? on url, source list, policy’s self-origin, and request’s redirect count.

Note: This means that CSP restrictions are evaluated against the bundle’s URL instead of to the uuid-in-package: URL. See #651 for the detailed motivation.

5.2. Monkeypatch Does response to request match source list?

Rewrite Does response to request match source list? to run these steps:

1. Let url be response’s url.

2. If url’s scheme is "uuid-in-package", then:

   1. Let registration be the result of running find a matching web bundle registration given request.

   2. If registration is not null, then set url to registration’s fetch entry's source.

3. Returns the result of executing Does url match source list in origin with redirect count? on url, source list, policy’s self-origin, and request’s redirect count.

Note: This means that CSP restrictions are evaluated against the bundle’s URL instead of to the uuid-in-package: URL. See #651 for the detailed motivation.

6. Algorithms

6.1. Parsing

To parse a web bundle string, given a string sourceText and a URL baseURL:

1. Let parsed be the result of parsing JSON into Infra values given sourceText.

2. If parsed is not a map, then throw a TypeError indicating that the top-level value needs to be a JSON object.

3. If parsed["source"] does not exist, then throw a TypeError.

4. If parsed["source"] is not a string, then throw a TypeError.

5. Let source be the result of parsing parsed["source"] with baseURL as the base URL.

6. If source is null, then throw a TypeError.

7. Let credentials be "same-origin".

8. If parsed["credentials"] exists, then:

   1. If parsed["credentials"] is "omit", then set credentials to "omit".

   2. Otherwise, if parsed["credentials"] is "include", then set credentials to "include".

9. Let resources be an empty list.

10. If parsed["resources"] exists, then:

    1. If parsed["resources"] is not a list, then throw a TypeError.

    2. Set resources to the result of parsing a url list given parsed["resources"] and source.

11. Let scopes be an empty list.

12. If parsed["scopes"] exists, then:

    1. If parsed["scopes"] is not a list, then throw a TypeError.

    2. Set scopes to the result of parsing a url list given parsed["scopes"] and source.

13. If parsed’s keys contains any items besides "source", "credentials", "resources" or "scopes", report a warning to the console that an invalid top-level key was present in the web bundle string.

    Note: This can help detect typos. It is not an error, because that would prevent any future extensions from being added backward-compatibly.

14. Let rule be bundle rule whose resources are resources and whose scopes are scopes.

15. Return the web bundle parse result whose source is source, whose credentials are credentials and whose rule is rule .

To parse a URL list, given a list originalList and a URL baseURL:

1. Let parsed URL list be an empty list.

2. For each item of originalList,

   1. If item is a string, then

      1. Let URL be the result of parsing item with baseURL as the base URL.

      2. If URL is not null, append URL to parsed URL list.

3. Return parsed URL list.

6.2. Fetching a web bundle

To fetch a web bundle given web bundle fetch entry fetch entry and fetch params fetch params:

1. Assert: fetch entry’s state is "fetching".

2. Let request be fetch params’s request.

3. Set request’s url to fetch entry’s source.

   Note: Source URL is resolved on document’s base URL.

4. Set request’s destination to "webbundle",

5. Set request’s mode to "cors",

6. Set request’s credentials mode to fetch entry’s credentials.

7. Set request’s service-workers mode to "none".

8. Append a header, a tuple of ("Accept", "application/webbundle;v=b2"), to request’s header list.

   Note: The final [draft-ietf-wpack-bundled-responses-latest] will use a version of 1, but this specification tracks what’s actually implemented in browsers, which still uses draft versions.

9. Fetch request with processResponse algorithm set to process web bundle response which is partially applied with fetch entry.

   Note: Chromium’s current implementation doesn’t allow a nested bundle. A Web bundle is never fetched from other web bundles.

6.3. Process web bundle response

To process web bundle response given web bundle fetch entry fetch entry and response response:

1. If response’s status is an ok status,

   1. Parse response’s body as a Web Bundle ([draft-ietf-wpack-bundled-responses-latest]).

      Note: response’s body might not be fully available at this moment. UA might parse the bundle by incrementally reading a body asynchronously in order to serve a subresource as early as possible.

      Note: In parsing, Chromium’s experimental implementation only accepts "b2" as a web bundle format version number ([draft-ietf-wpack-bundled-responses-latest]).

   2. When the parse algorithm asynchronously completes, set fetch entry’s fetched bundle to the result of parsing and fetch entry’s state be "fetched". If parsing fails, or any other conformance is violated, set fetched bundle to null and state to "failed".

2. Otherwise, set fetch entry’s state to "failed".

6.4. Fetching subresources from a web bundle

To fetch a subresource from web bundle given request httpRequest:

1. Let registration be the result of running find a matching web bundle registration given httpRequest.

2. If registration is not null:

   1. Let response be the result of get response from web bundle fetch entry given httpRequest’s url and registration’s fetch entry.

   2. If response is null, return a network error.

      Note: This means a browser does not fallback to fetch a subresource from network.

   3. Otherwise, return response.

3. Return null.

Note: Returning null here can fallback to HTTP cache and ordinal network fetch, unlike returning a network error above.

To get response from web bundle fetch entry given url url and web bundle fetch entry fetch entry:

1. If fetch entry’s state is "fetching", await until state becomes "fetched" or "failed" asynchronously.

2. If fetch entry’s state is "failed", return null.

3. Assert: fetch entry’s fetched bundle is non-null.

4. Returns response from fetch entry’s fetched bundle given url ([draft-ietf-wpack-bundled-responses-latest]). If a representation of url is not found in fetched bundle, return null.

6.5. Finding a matching registration

To find a matching web bundle registration given request httpRequest:

1. Let url be httpRequest’s url.

2. For each registration of httpRequest’s client's web bundle registration list:

   1. Let rule be registration’s rule.

   2. If url’s scheme is not "uuid-in-package", then

      1. If url’s origin and registration’s fetch entry's source's origin are not same origin, then continue.

      2. Let allowed path be the result of shortening registration’s fetch entry's source's path.

      3. If url’s path doesn’t start with allowed path, then continue.

   3. If rule’s resources contains url, then return registration.

   4. If url starts with any of rule’s scopes, then return registration.

3. Return null.