Reporting API

Draft Community Group Report,

This version:
https://wicg.github.io/reporting/
Version History:
https://github.com/wicg/reporting/commits/master/index.src.html
Editors:
(Google Inc.)
(Google Inc.)
(Google Inc.)
Participate:
File an issue (open issues)

Abstract

This document defines a generic reporting framework which allows web developers to associate a set of named reporting endpoints with an origin. Various platform features (like Content Security Policy, Network Error Reporting, and others) will use these endpoints to deliver feature-specific reports in a consistent manner.

Status of this document

This specification was published by the Web Platform Incubator Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Contributor License Agreement (CLA) there is a limited opt-out and other conditions apply. Learn more about W3C Community and Business Groups.

1. Introduction

[INTRODUCTION GOES HERE]

1.1. Guarantees

This specification aims to provide a best-effort report delivery system that executes out-of-band with website activity. The user agent will be able to do a better job prioritizing and scheduling delivery of reports, as it has an overview of cross-origin activity that individual websites do not, and can deliver reports based on error conditions that would prevent a website from loading in the first place.

The delivery is not, however, guaranteed in a strict sense. We spell out a reasonable set of retry rules in the algorithms below, but it’s quite possible for a report to be dropped on the floor if things go badly.

Reporting can generate a good deal of traffic, so we allow developers to set up groups of endpoints. The user agent will do its best to deliver a particular report to at most one endpoint in a group. Endpoints can be assigned weights to distribute load, with each endpoint receiving a specified fraction of reporting traffic. Endpoints can be assigned priorities, allowing developers to set up fallback collectors that are only tried when uploads to primary collectors fail.

1.2. Examples

MegaCorp Inc. wants to collect Content Security Policy and Key Pinning violation reports. It can do so by delivering the following header to define a set of reporting endpoints named "endpoint-1":
Report-To: { "group": "endpoint-1",
             "max_age": 10886400,
             "endpoints": [
               { "url": "https://example.com/reports", "priority": 1 },
               { "url": "https://backup.com/reports", "priority": 2 }
             ] }

And the following headers, which direct CSP and HPKP reports to that group:

Content-Security-Policy: ...; report-to=endpoint-1
Public-Key-Pins: ...; report-to=endpoint-1
After processing reports for a little while, MegaCorp Inc. decides to split the processing of these two types of reports out into two distinct endpoints in order to make the processing scripts simpler. It can do so by delivering the following header to define two reporting endpoints:
Report-To: { "url": "https://example.com/csp-reports",
             "group": "csp-endpoint",
             "max_age": 10886400 },
           { "url": "https://example.com/hpkp-reports",
             "group": "hpkp-endpoint",
             "max_age": 10886400 }
Report-To: { "group": "csp-endpoint",
             "max_age": 10886400,
             "endpoints": [
               { "url": "https://example.com/csp-reports" }
             ] },
           { "group": "hpkp-endpoint",
             "max_age": 10886400,
             "endpoints": [
               { "url": "https://example.com/hpkp-reports" }
             ] }

And the following headers, which direct CSP and HPKP reports to those named endpoint:

Content-Security-Policy: ...; report-to=csp-endpoint
Public-Key-Pins: ...; report-to=hpkp-endpoint

2. Concepts

2.1. Clients

A client represents a particular origin’s relationship to a set of endpoints.

Each client has an origin, which is an origin.

Each client has an endpoint-groups list, which is a list of endpoint groups, each of which MUST have a distinct name. (The algorithm in §3.2 Process reporting endpoints for response to request guarantees this by keeping the first entry in a Report-To header with a particular name.)

2.2. Endpoint groups

An endpoint group is a set of endpoints that will be used together for backup and failover purposes.

Each endpoint group has a name, which is an ASCII string.

Each endpoint group has an endpoints list, which is a list of endpoints.

Each endpoint group has a subdomains flag, which is either "include" or "exclude".

Each endpoint group has a ttl representing the number of seconds the group remains valid for an origin.

Each endpoint group has a creation which is the timestamp at which the group was added to an origin.

An endpoint group is expired if its creation plus its ttl represents a time in the past.

2.3. Endpoints

An endpoint is location to which reports for a particular origin may be sent.

Each endpoint has a url, which is a URL.

Each endpoint has a priority, which is a non-negative integer.

Each endpoint has a weight, which is a non-negative integer.

Each endpoint has a failures, which is a non-negative integer representing the number of consecutive times this endpoint has failed to respond to a request.

Each endpoint has a retry_after, which is either null, or a timestamp after which delivery should be retried.

An endpoint is pending if its retry_after is not null, and represents a time in the future.

2.4. Report Type

A report type is a non-empty string that specifies the set of data that is contained in the body of a report.

When a report type is defined (in this spec or others), it can be specified to be observable from JavaScript, meaning that reports of that type can be observed by a reporting observer. By default, report types are not observable from JavaScript.

Note: See §6.1 Deprecation as an example report type definition.

2.5. Reports

A report is a collection of arbitrary data which the user agent is expected to deliver to a specified endpoint.

Each report has a body, which is either null or an object which can be serialized into a JSON text. The fields contained in a report’s body are determined by the report’s type.

Each report has a url, which is the address of the Document or Worker from which the report was generated.

Note: We strip the username, password, and fragment from this serialized URL. See §9.1 Capability URLs.

Each report has an origin, which is an origin representing the report’s initiator.

Each report has a group, which is a string representing the name of the origin’s endpoint group that the report will be sent to.

Each report has a type, which is a report type.

Each report has a timestamp, which records the time at which the report was generated, in milliseconds since the unix epoch.

Each report has an attempts counter, which is a non-negative integer representing the number of times the user agent attempted to deliver the report.

2.6. Storage

A conformant user agent MUST provide a reporting cache, which is a storage mechanism that maintains a set of endpoint groups that websites have instructed the user agent to associate with their origins, and a set of reports which are queued for delivery.

This storage mechanism is opaque, vendor-specific, and not exposed to the web, but it MUST provide the following methods which will be used in the algorithms this document defines:

  1. Insert, update, and remove clients.

  2. Enqueue and dequeue reports for delivery.

  3. Retrieve a list of client objects for an origin.

  4. Retrieve a list of queued report objects.

  5. Clear the cache.

2.7. Failover and load balancing

The endpoints in an endpoint group that all have the same priority form a failover class. Failover classes allow the developer to provide backup collectors (those with higher priority values) that will only receive reports if all of the primary collectors (those with lower priority values) fail.

Developers can assign each endpoint in a failover class a weight, which determines how report traffic is balanced across the failover class.

The algorithm that implements these rules is described in §4.2 Choose an endpoint from a group.

3. Endpoint Delivery

A server MAY define a set of reporting endpoints for an origin it controls via the Report-To HTTP response header field. This mechanism is defined in §3.1 The Report-To HTTP Response Header Field, and its processing in §3.2 Process reporting endpoints for response to request.

The Report-To HTTP response header field instructs the user agent to store reporting endpoints for an origin. The header is represented by the following ABNF grammar [RFC5234]:

Report-To = json-field-value
            ; See Section 2 of [[HTTP-JFV]], and Section 2 of [[RFC7159]]

The header’s value is interpreted as an array of JSON objects, as described in Section 4 of [HTTP-JFV].

Each object in the array defines an endpoint group to which reports may be delivered, and will be parsed as defined in §3.2 Process reporting endpoints for response to request.

The following subsections define the initial set of known members in each JSON object the header’s value defines. Future versions of this document may define additional such members, and user agents MUST ignore unknown members when parsing the header.

3.1.1. The group member

The OPTIONAL group member is a string that associates a name with the endpoint group. The member’s value MUST be a string; any other type will result in a parse error. If no member named "group" is present in the object, the endpoint group will be given the name "default".

3.1.2. The include-subdomains member

The OPTIONAL include-subdomains member is a boolean that enables this endpoint group for all subdomains of the current origin’s host. If no member named "include-subdomains" is present in the object, or its value is not "true", the endpoint group will not be enabled for subdomains.

3.1.3. The max_age member

The REQUIRED max_age member defines the endpoint group’s lifetime, as a non-negative integer number of seconds. The member’s value MUST be a non-negative number; any other type will result in a parse error.

A value of "0" will cause the endpoint group to be removed from the user agent’s reporting cache.

3.1.4. The endpoints member

The REQUIRED endpoints member defines the list of endpoints that belong to this endpoint group. The member’s value MUST be an array of JSON objects.

The following subsections define the initial set of known members in each JSON object in the array. Future versions of this document may define additional such members, and user agents MUST ignore unknown members when parsing the elements of the array.

3.1.5. The endpoints.url member

The REQUIRED url member is a string that defines the location of the endpoint. The member’s value MUST be a string; any other type will result in a parse error.

Moreover, the URL that the member’s value represents MUST be potentially trustworthy [SECURE-CONTEXTS]. Non-secure endpoints will be ignored.

3.1.6. The endpoints.priority member

The OPTIONAL priority member is a number that defines which failover class the endpoint belongs to. The member’s value, if present, MUST be a non-negative integer; any other type will result in a parse error.

3.1.7. The endpoints.weight member

The OPTIONAL weight member is a number that defines load balancing for the failover class that the endpoint belongs to. The member’s value, if present, MUST be a non-negative integer; any other type will result in a parse error.

3.2. Process reporting endpoints for response to request

Given a response (response) and a request (request), this algorithm extracts a list of endpoints and endpoint groups for the request’s origin, and updates the reporting cache accordingly.

Note: This algorithm is called from around step 13 of main fetch [FETCH], and only updates the reporting cache if the response has been delivered securely.

Fetch monkey patching. Talk to Anne.

  1. Abort these steps if any of the following conditions are true:

    1. response’s HTTPS state is not "modern", and the origin of response’s url is not potentially trustworthy.

    2. response’s header list does not contain a header whose name is "Report-To".

  2. Let origin be the origin of response’s url.

  3. Let header be the value of the header in response’s header list whose name is "Report-To".

  4. Let list be the result of executing the algorithm defined in Section 4 of [HTTP-JFV] on header. If that algorithm results in an error, abort these steps.

  5. Let groups be an empty list.

  6. For each item in list:

    1. If item has no member named "max_age", or that member’s value is not a number, skip to the next item.

    2. If item has no member named "endpoints", or that member’s value is not an array, skip to the next item.

    3. Let name be item’s "group" member’s value if present, and "default" otherwise.

    4. If there is already an endpoint group in groups whose name is name, skip to the next item.

    5. Let endpoints be an empty list.

    6. For each endpoint item in the value of item’s "endpoints" member:

      1. If endpoint item has no member named "url", or that member’s value is not a string, skip to the next endpoint item.

      2. If endpoint item has a member named "priority", whose value is not a non-negative integer, skip to the next endpoint item.

      3. If endpoint item has a member named "weight", whose value is not a non-negative integer, skip to the next endpoint item.

      4. Let endpoint be a new endpoint whose properties are set as follows:

        url

        The result of executing the URL parser on endpoint item’s "url" member’s value.

        priority

        The value of the endpoint item’s "priority" member, if present; 1 otherwise.

        weight

        The value of the endpoint item’s "weight" member, if present; 1 otherwise.

        failures

        0

        retry_after

        null

      5. Add endpoint to endpoints.

    7. Let group be a new endpoint group whose properties are set as follows:

      name

      name

      subdomains

      "include" if item has a member named "include-subdomains" whose value is true, "exclude" otherwise.

      ttl

      item’s "max_age" member’s value.

      creation

      The current timestamp

      endpoints

      endpoints

    8. Add group to groups.

  7. Let client be a new client whose properties are set as follows:

    origin

    origin

    endpoint-groups

    groups

  8. If there is already an entry in the reporting cache for origin, replace it with client. Otherwise, insert client into the reporting cache for origin.

4. Report Delivery

Over time, various features will queue up a list of reports in the user agent’s reporting cache. The user agent will periodically grab the list of currently pending reports, and deliver them to the associated endpoints. This document does not define a schedule for the user agent to follow, and assumes that the user agent will have enough contextual information to deliver reports in a timely manner, balanced against impacting a user’s experience.

That said, a user agent SHOULD make a effort to deliver reports as soon as possible after queuing, as a report’s data might be significantly more useful in the period directly after its generation than it would be a day or a week later.

4.1. Queue data as type for endpoint group on settings

Given a serializable object (data), a string (type), another string (endpoint group), and an environment settings object (settings), the following algorithm will create a report, and add it to reporting cache’s queue for future delivery.

  1. Let report be a new report object with its values initialized as follows:

    body

    data

    origin

    settings’s origin

    group

    endpoint group

    type

    type

    timestamp

    The current timestamp.

    attempts

    0

  2. Let url be settings’s creation URL.

  3. Set url’s username to the empty string, and its password to null.

  4. Set report’s url to the result of executing the URL serializer on url with the exclude fragment flag set.

  5. Append report to the reporting cache.

  6. Let environment be settings’s realm execution context’s realm’s ECMAScript global environment.

  7. Execute §5.2 Notify reporting observers on environment with report with environment and report.

Note: reporting observers can only observe reports from the same environment settings object.

Note: We strip the username, password, and fragment from the serialized URL in the report. See §9.1 Capability URLs.

Note: The user agent MAY reject reports for any reason. This API does not guarantee delivery of arbitrary amounts of data, for instance.

Note: Non user agent clients (with no JavaScript engine) should not interact with reporting observers, and thus should return in step 6.

4.2. Choose an endpoint from a group

Given an endpoint group (group), this algorithm chooses an arbitrary eligible endpoint from the group, if there is one, taking into account the priority and weight of the endpoints.

  1. If group is expired, return null.

    Note: In this case, the user agent MAY remove group from its client, or it may wait and collect garbage en masse at some point in the future as described in §7.2 Garbage Collection.

  2. Let endpoints be a copy of group’s endpoints list.

  3. Remove every endpoint from endpoints that is pending.

  4. If endpoints is empty, return null.

  5. Let priority be the minimum priority value of each endpoint in endpoints.

  6. Remove every endpoint from endpoints whose priority value is not equal to priority.

  7. If endpoints is empty, return null.

  8. Let total weight be the sum of the weight value of each endpoint in endpoints.

  9. Let weight be a random number ≥ 0 and < total weight.

  10. For each endpoint in endpoints:

    1. If weight is less than endpoint’s weight, return endpoint.

    2. Subtract endpoint’s weight from weight.

  11. It should not be possible to fall through to here, since the random number chosen earlier will be less than total weight.

4.3. Send reports

A user agent sends reports by executing the following steps:

  1. Let reports be a copy of the list of queued report objects in reporting cache.

  2. Let endpoint map be an empty map of endpoint objects to lists of report objects.

  3. For each report in reports:

    1. Let origin be report’s origin.

    2. Let client be the entry in the reporting cache for origin.

    3. If there exists an endpoint group (group) in client’s endpoint-groups list whose name is report’s group:

      1. Let endpoint be the result of executing §4.2 Choose an endpoint from a group on group.

      2. If endpoint is a not null:

        1. Append report to endpoint map’s list of reports for endpoint.

        2. Skip to the next report.

    4. For each parent origin that is a superdomain match for origin [RFC6797]:

      1. Let client be the entry in the reporting cache for parent origin.

      2. If there exists an endpoint group (group) in client’s endpoint-groups list whose name is report’s group and whose subdomains flag is "include":

        1. Let endpoint be the result of executing §4.2 Choose an endpoint from a group on group.

        2. If endpoint is an endpoint:

          1. Append report to endpoint map’s list of reports for endpoint.

          2. Skip to the next report.

    5. If we reach this step, the report did not match any endpoint and the user agent MAY remove report from the reporting cache directly. Depending on load, the user agent MAY instead wait for §7.2 Garbage Collection at some point in the future.

  4. For each (endpoint, reports) pair in endpoint map, execute the following steps asynchronously:

    1. Let result be the result of executing §4.4 Attempt to deliver reports to endpoint on endpoint and reports.

    2. If result is "Success":

      1. Set endpoint’s failures to 0, and its retry_after to null.

      2. Remove each report in reports from the reporting cache.

      Otherwise, if result is "Remove Endpoint":

      1. Remove endpoint from the reporting cache.

        Note: reports remain in the reporting cache for potential delivery to other endpoints.

      Otherwise (if result is "Failure"):

      1. Increment endpoint’s failures.

      2. Set endpoint’s retry_after to a point in the future which the user agent chooses.

        Note: We don’t specify a particular algorithm here, but user agents are encouraged to employ some sort of exponential backoff algorithm which increases the retry period with the number of failures, with the addition of some random jitter to ensure that temporary failures don’t lead to a crush of reports all being retried on the same schedule.

        Add in a reasonable reference describing a good algorithm. Wikipedia, if nothing else.

Note: User agents MAY decide to attempt delivery for only a subset of the collected reports or endpoints (because, for example, sending all the reports at once would consume an unreasonable amount of bandwidth, etc). As reports are only removed from the cache when they’re successfully delivered, skipped reports will simply be delivered later.

4.4. Attempt to deliver reports to endpoint

Given a list of reports (reports) and an endpoint (endpoint), this algorithm will construct a request, and attempt to deliver it to endpoint. It returns "Success" if that delivery succeeds, "Remove Endpoint" if the endpoint explicitly removes itself as a reporting endpoint by sending a 410 response, and "Failure" otherwise.

  1. Let collection be a new ECMAScript Array object [ECMA-262].

  2. For each report in reports:

    1. Let data be a new ECMAScript Object with the following properties [ECMA-262]:

      age

      The number of milliseconds between report’s timestamp and the current time.

      type

      report’s type

      url

      report’s url

      body

      report’s body

      Note: Client clocks are unreliable and subject to skew. We therefore deliver an age attribute rather than an absolute timestamp. See also §10.2 Clock Skew

    2. Increment report’s attempts.

    3. Append data to collection.

  3. Let request be a new request with the following properties [FETCH]:

    url

    endpoint’s url

    header list

    A new header list containing a header named "Content-Type" whose value is "application/report"

    client

    null

    window

    "no-window"

    skip-service-worker flag

    Set.

    initiator

    ""

    type

    "report"

    destination

    ""

    mode

    "cors"

    credentials

    "include"

    body

    The string resulting from executing the JSON.stringify() algorithm on collection [ECMA-262]

    The "report" type does not exist in Fetch. Talk to Anne.

  4. Queue a task to fetch request.

  5. Wait for a response (response).

  6. If response’s status is an OK status (200-299), return "Success".

  7. If response’s status is 410 Gone [RFC7231], return "Remove Endpoint".

  8. Return "Failure".

5. Reporting Observers

A reporting observer observes some types of reports from JavaScript, and is represented in JavaScript by the ReportingObserver object.

Each ECMAScript global environment has a registered reporting observer list, which is an ordered set of reporting observers.

Any reporting observer that is in a registered reporting observer list is considered registered.

Each ECMAScript global environment has a report buffer, which is a list of reports that have been generated in that ECMAScript global environment. This list is initially empty, and the reports are stored in the same order in which they are generated.

Note: The purpose of the report buffer is to allow reporting observers to observe reports that were generated earlier than that observer could be created (via the buffered option). For example, some reports might be generated during an earlier stage of page loading than when an observer could first be created, or before a JavaScript library is loaded that wishes to observe these reports.

Note: Reporting observers are only relevant for user agents with JavaScript engines.

5.1. Interface ReportingObserver

interface ReportBody {
};

interface Report {
  readonly attribute DOMString type;
  readonly attribute DOMString url;
  readonly attribute ReportBody? body;
};

[Constructor(ReportingObserverCallback callback, optional ReportingObserverOptions options)]
interface ReportingObserver {
  void observe();
  void disconnect();
  ReportList takeRecords();
};

callback ReportingObserverCallback = void (sequence<Report> reports, ReportingObserver observer);

dictionary ReportingObserverOptions {
  sequence<DOMString> types;
  boolean buffered = false;
};

typedef sequence<Report> ReportList;

A Report is the application exposed representation of a report. type returns type, url returns url, and body returns body.

Each ReportingObserver object has these associated concepts:

  • A callback function set on creation.

  • A ReportingObserverOptions dictionary called options.

  • A list of Report objects called the report list, which is initially empty.

A ReportList represents a sequence of Reports, providing developers with all the convenience methods found on JavaScript arrays.

The ReportingObserver(callback, options) constructor, when invoked, must run these steps:

  1. Create a new ReportingObserver object observer.

  2. Set observer’s callback to callback.

  3. Set observer’s options to options.

  4. Return observer.

The observe() method, when invoked, must run these steps:

  1. Let environment be the ECMAScript global environment associated with the context object.

  2. Append the context object to the registered reporting observer list of its associated ECMAScript global environment.

  3. If the context object’s buffered option is false, return.

  4. Set context object’s buffered option to false.

  5. For each report in the report buffer associated with environment, execute §5.3 Add report to observer with report and the context object.

The disconnect() method, when invoked, must run these steps:

  1. If the context object is not registered, return.

  2. Remove the context object from the registered reporting observer list of its associated ECMAScript global environment.

The takeRecords() method, when invoked, must run these steps:

  1. Let reports be a copy of the context object’s report list.

  2. Empty the context object’s report list.

  3. Return reports.

5.2. Notify reporting observers on environment with report

This algorithm makes report’s contents available to any registered reporting observers on the provided ECMAScript global environment environment.

  1. For each ReportingObserver observer registered with environment, execute §5.3 Add report to observer on report and observer.

  2. Append report to the report buffer associated with environment.

  3. If the report buffer now contains more than 100 reports, remove the item at the beginning of the report buffer.

5.3. Add report to observer

Given a report report and a ReportingObserver observer, this algorithm adds report to observer’s report list, so long as report’s type is observable by observer.

  1. If report’s type is not observable from JavaScript, return.

  2. If observer’s options has a non-empty types member which does not contain report’s type, return.

  3. Create a new Report r with type initialized to report’s type, url initialized to report’s url, and body initialized to report’s body.

how to polymorphically initialize body?

  1. Append r to observer’s report list.

  2. If the size of observer’s report list is 1, Queue a task to §5.4 Invoke reporting observers with notify list with a copy of the registered reporting observer list of the ECMAScript global environment associated with observer.

5.4. Invoke reporting observers with notify list

This algorithm invokes observer callback functions for reports of previously observed behavior.

  1. For each ReportingObserver observer in notify list:

    1. If observer’s report list is empty, then continue.

    2. Let reports be a copy of observer’s report list

    3. Empty observer’s report list

    4. Invoke observer’s callback with a list of arguments consisting of reports and observer, and observer as the callback this value. If this throws an exception, report the exception.

6. Report Types

6.1. Deprecation

Deprecation reports indicate that a browser API or feature has been used which is expected to stop working in a future update to the browser.

Deprecation reports have the report type "deprecation".

A deprecation report’s body contains the following fields:

  • id: an implementation-defined string identifying the feature or API that will be removed. This string can be used for grouping and counting related reports.

  • anticipated_removal: A date indicating roughly when the browser version without the specified API will be generally available (excluding "beta" or other pre-release channels). This value should be used to sort or prioritize warnings. If unknown, this field should be set to null, and the deprecation should be considered low priority (removal may not actually occur).

  • message: A human-readable string with details typically matching what would be displayed on the developer console. The message is not guaranteed to be unique for a given id (eg. it may contain additional context on how the API was used).

  • source_file: If known, the file which first used the indicated API, or null otherwise.

  • line_number: If known, the line number in source_file where the indicated API was first used, or null otherwise.

  • column_number: If known, the column number in source_file where the indicated API was first used, or null otherwise.

Deprecation reports are observable from JavaScript.

7. Implementation Considerations

7.1. Delivery

The user agent SHOULD attempt to deliver reports as soon as possible to provide feedback to developers as quickly as possible. However, when this desire is balanced against the impact on the user, the user wins. With that in mind, the user agent MAY delay delivery of reports based on its knowledge of the user’s activities and context.

For instance, the user agent SHOULD prioritize the transmission of reporting data lower than other network traffic. The user’s explicit activities on a website should preempt reporting traffic.

The user agent MAY choose to withhold report delivery entirely until the user is on a fast, cheap network in order to prevent unnecessary data cost.

The user agent MAY choose to prioritize reports from particular origins over others (perhaps those that the user visits most often?)

7.2. Garbage Collection

Periodically, the user agent SHOULD walk through the cached reports and endpoints, and discard those that are no longer relevant. These include:

  • endpoint groups which are expired

  • endpoint groups which have not been used in some arbitrary period of time (perhaps a ~week?)

  • endpoints whose failures exceed some user-agent-defined threshold (~5 seems reasonable)

  • reports whose attempts exceed some user-agent-defined threshold (~5 seems reasonable)

  • reports which have not been delivered in some arbitrary period of time (perhaps ~2 days?)

For any reports that are discarded, these reports should also be removed from the report buffer of any reporting observer.

8. Sample Reports

POST / HTTP/1.1
Host: example.com
...
Content-Type: application/report

[{
  "type": "csp",
  "age": 10,
  "url": "https://example.com/vulnerable-page/",
  "body": {
    "blocked": "https://evil.com/evil.js",
    "directive": "script-src",
    "policy": "script-src 'self'; object-src 'none'",
    "status": 200,
    "referrer": "https://evil.com/"
  }
}, {
  "type": "hpkp",
  "age": 32,
  "url": "https://www.example.com/",
  "body": {
    "date-time": "2014-04-06T13:00:50Z",
    "hostname": "www.example.com",
    "port": 443,
    "effective-expiration-date": "2014-05-01T12:40:50Z"
    "include-subdomains": false,
    "served-certificate-chain": [
      "-----BEGIN CERTIFICATE-----\n
      MIIEBDCCAuygAwIBAgIDAjppMA0GCSqGSIb3DQEBBQUAMEIxCzAJBgNVBAYTAlVT\n
      ...
      HFa9llF7b1cq26KqltyMdMKVvvBulRP/F/A8rLIQjcxz++iPAsbw+zOzlTvjwsto\n
      WHPbqCRiOwY1nQ2pM714A5AuTHhdUDqB1O6gyHA43LL5Z/qHQF1hwFGPa4NrzQU6\n
      yuGnBXj8ytqU0CwIPX4WecigUCAkVDNx\n
      -----END CERTIFICATE-----",
      ...
    ]
  }
}, {
  "type": "nel",
  "age": 29,
  "url": "https://example.com/thing.js",
  "body": {
    "referrer": "https://www.example.com/",
    "server-ip": "234.233.232.231",
    "protocol": "",
    "status-code": 0,
    "elapsed-time": 143,
    "age": 0,
    "type": "http.dns.name_not_resolved"
  }
}]

9. Security Considerations

9.1. Capability URLs

Some URLs are valuable in and of themselves. To mitigate the possibility that such URLs will be leaked via this reporting mechanism, we strip out credential information and fragment data from the URL we store as a report’s originator. It is still possible, however, for a feature to unintentionally leak such data via a report’s body. Implementers SHOULD ensure that URLs contained in a report’s body are similarly stripped.

10. Privacy Considerations

10.1. Network Leakage

Because this reporting mechanism is out-of-band, and doesn’t rely on a page being open, it’s entirely possible for a report generated while a user is on one network to be sent while the user is on another network, even if they don’t explicitly open the page from which the report was sent.

Consider mitigations. For example, we could drop reports if we change from one network to another. <https://github.com/WICG/BackgroundSync/issues/107>

10.2. Clock Skew

Each report is delivered along with an age property, rather than the timestamp at which it was generated. We do this because each user’s local clock will be skewed from the clock on the server by an arbitrary amount. The difference between the time the report was generated and the time it was sent will be stable, regardless of clock skew, and we can avoid the fingerprinting risk of exposing the clock skew via this API.

10.3. Cross-origin correlation

If multiple origins all use the same reporting endpoint, that endpoint may learn that a particular user has interacted with a certain set of websites, as it will receive origin-tagged reports from each. This doesn’t seem worse than the status quo ability to track the same information from cooperative origins, and doesn’t grant any new tracking ability above and beyond what’s possible with <img> today.

10.4. Subdomains

This specification allows any resource on a host to declare a set of reporting endpoints for that host and each of its subdomains. This doesn’t have privacy implications in and of itself (beyond those noted in §10.5 Clearing the reporting cache), as the reporting endpoints themselves don’t take any real action, as features will need to opt-into using these reporting endpoints explicitly. Those features certainly will have privacy implications, and should carefully consider whether they should be enabled across origin boundaries.

10.5. Clearing the reporting cache

A user agent’s reporting cache contains data about a user’s activity on the web, and user agents ought to handle this data carefully. In particular, if a user agent gives users the ability to clear their site data, browsing history, browsing cache, or similar, the user agent MUST also clear the reporting cache. Note that this includes both the pending reports themselves, as well as the endpoints to which they would be sent. Both MUST be cleared.

10.6. Disabling Reporting

Reporting is, to some extent, a question of commons. In the aggregate, it seems useful for everyone for reports to be delivered. There is direct benefit to developers, as they can fix bugs, which means there’s indirect benefit to users, as the sites they enjoy will be more stable and enjoyable. As a concrete example, Content Security Policy grants something like herd immunity to cross-site scripting attacks by alerting developers about potential holes in their sites' defenses. Fixing those bugs helps every user, even those whose user agents don’t support Content Security Policy.

The calculus, of course, depends on the nature of data that’s being delivered, and the relative maliciousness of the reporting endpoints, but that’s the value proposition in broad strokes.

That said, it can’t be the case that this general benefit be allowed to take priority over the ability of a user to individually opt-out of such a system. Sending reports costs bandwidth, and potentially could reveal some small amount of additional information above and beyond what a website can obtain in-band ([NETWORK-ERROR-LOGGING], for instance). User agents MUST allow users to disable reporting with some reasonable amount of granularity in order to maintain the priority of constituencies espoused in [HTML-DESIGN-PRINCIPLES].

11. IANA Considerations

The permanent message header field registry should be updated with the following registration: [RFC3864]

11.1. Report-To

Header field name

Report-To

Applicable protocol

http

Status

standard

Author/Change controller

W3C

Specification document

This specification (see §3.1 The Report-To HTTP Response Header Field)

Index

Terms defined by this specification

Terms defined by reference

References

Normative References

[DOM]
Anne van Kesteren. DOM Standard. Living Standard. URL: https://dom.spec.whatwg.org/
[ECMA-262]
ECMAScript Language Specification. URL: https://tc39.github.io/ecma262/
[FETCH]
Anne van Kesteren. Fetch Standard. Living Standard. URL: https://fetch.spec.whatwg.org/
[HTML]
Anne van Kesteren; et al. HTML Standard. Living Standard. URL: https://html.spec.whatwg.org/multipage/
[HTTP-JFV]
Julian Reschke. A JSON Encoding for HTTP Header Field Values. URL: https://greenbytes.de/tech/webdav/draft-reschke-http-jfv-02.html
[INFRA]
Anne van Kesteren; Domenic Denicola. Infra Standard. Living Standard. URL: https://infra.spec.whatwg.org/
[RFC3864]
G. Klyne; M. Nottingham; J. Mogul. Registration Procedures for Message Header Fields. September 2004. Best Current Practice. URL: https://tools.ietf.org/html/rfc3864
[RFC5234]
D. Crocker, Ed.; P. Overell. Augmented BNF for Syntax Specifications: ABNF. January 2008. Internet Standard. URL: https://tools.ietf.org/html/rfc5234
[RFC6797]
J. Hodges; C. Jackson; A. Barth. HTTP Strict Transport Security (HSTS). November 2012. Proposed Standard. URL: https://tools.ietf.org/html/rfc6797
[RFC7231]
R. Fielding, Ed.; J. Reschke, Ed.. Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content. June 2014. Proposed Standard. URL: https://tools.ietf.org/html/rfc7231
[RFC8259]
T. Bray, Ed.. The JavaScript Object Notation (JSON) Data Interchange Format. December 2017. Internet Standard. URL: https://tools.ietf.org/html/rfc8259
[SECURE-CONTEXTS]
Mike West; Yan Zhu. Secure Contexts. URL: https://w3c.github.io/webappsec-secure-contexts/
[URL]
Anne van Kesteren. URL Standard. Living Standard. URL: https://url.spec.whatwg.org/
[WebIDL]
Cameron McCormack; Boris Zbarsky; Tobie Langel. Web IDL. 15 December 2016. ED. URL: https://heycam.github.io/webidl/

Informative References

[CSP3]
Mike West. Content Security Policy Level 3. 13 September 2016. WD. URL: https://www.w3.org/TR/CSP3/
[HTML-DESIGN-PRINCIPLES]
Anne van Kesteren; Maciej Stachowiak. HTML Design Principles. 26 November 2007. WD. URL: https://www.w3.org/TR/html-design-principles/
[NETWORK-ERROR-LOGGING]
Ilya Grigorik; et al. Network Error Logging. 20 July 2016. NOTE. URL: https://www.w3.org/TR/network-error-logging/
[RFC7469]
C. Evans; C. Palmer; R. Sleevi. Public Key Pinning Extension for HTTP. April 2015. Proposed Standard. URL: https://tools.ietf.org/html/rfc7469

IDL Index

interface ReportBody {
};

interface Report {
  readonly attribute DOMString type;
  readonly attribute DOMString url;
  readonly attribute ReportBody? body;
};

[Constructor(ReportingObserverCallback callback, optional ReportingObserverOptions options)]
interface ReportingObserver {
  void observe();
  void disconnect();
  ReportList takeRecords();
};

callback ReportingObserverCallback = void (sequence<Report> reports, ReportingObserver observer);

dictionary ReportingObserverOptions {
  sequence<DOMString> types;
  boolean buffered = false;
};

typedef sequence<Report> ReportList;

Issues Index

Fetch monkey patching. Talk to Anne.
Add in a reasonable reference describing a good algorithm. Wikipedia, if nothing else.
The "report" type does not exist in Fetch. Talk to Anne.
how to polymorphically initialize body?
Consider mitigations. For example, we could drop reports if we change from one network to another. <https://github.com/WICG/BackgroundSync/issues/107>