Storage Buckets API

Draft Community Group Report,

This version:
https://wicg.github.io/storage-buckets/
Issue Tracking:
GitHub
Inline In Spec
Editors:
(Google)
(Google)
Former Editor:
Victor Costan
Participate:
GitHub WICG/storage-buckets (new issue, open issues)

Abstract

The Storage Buckets API provides a way for sites to organize locally stored data into groupings called "storage buckets". This allows the user agent or sites to manage and delete buckets independently rather than applying the same treatment to all the data from a single origin.

Status of this document

This specification was published by the Web Platform Incubator Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Contributor License Agreement (CLA) there is a limited opt-out and other conditions apply. Learn more about W3C Community and Business Groups.

1. The StorageBucketManager interface

[SecureContext]
interface mixin NavigatorStorageBuckets {
  [SameObject] readonly attribute StorageBucketManager storageBuckets;
};
Navigator includes NavigatorStorageBuckets;
WorkerNavigator includes NavigatorStorageBuckets;

Each environment settings object has an associated StorageBucketManager object.

The storageBuckets getter steps are to return this's relevant settings object's StorageBucketManager object.

A user agent has an associated storage bucket manager which is the result of starting a new parallel queue.

[Exposed=(Window,Worker),
 SecureContext]
interface StorageBucketManager {
    Promise<StorageBucket> open(DOMString name, optional StorageBucketOptions options = {});
    Promise<sequence<DOMString>> keys();
    Promise<undefined> delete(DOMString name);
};

dictionary StorageBucketOptions {
  boolean persisted = false;
  unsigned long long quota;
  DOMHighResTimeStamp expires;
};

1.1. Creating a bucket

The open(name, options) method steps are:

  1. Let environment be this's relevant settings object.

  2. Let shelf be the result of running obtain a local storage shelf given environment.

  3. If shelf is failure, then return a promise rejected with a TypeError.

  4. If the result of validate a bucket name with name is failure, then return a promise rejected with a TypeError.

  5. Let p be a new promise.

  6. Enqueue the following steps to storage bucket manager:

    1. Let r be the result of running open a bucket with shelf, name, and options.

    2. If r is failure, then queue a storage task to reject p with a TypeError.

    3. Otherwise, queue a storage task to resolve p with r.

  7. Return p.

To open a bucket for a shelf given a bucket name and optional options, run the following steps:

  1. Let expires be undefined.

  2. If options["expires"] exists, then:

    1. Set expires to options["expires"].

    2. If expires milliseconds after the Unix epoch is before the relevant settings object's current wall time, then return failure.

  3. Let quota be undefined.

  4. If options["quota"] exists, then:

    1. Set quota to options["quota"].

    2. If quota is less than or equal to zero, then return failure.

  5. Let persisted be false.

  6. If options["persisted"] is true, then:

    1. Let permission be the result of requesting permission to use "persistent-storage".

    2. If permission is "granted", then set persisted to true.

  7. Let bucket be the result of running get or expire a bucket with shelf and name.

  8. If bucket is null, then:

    1. Set bucket to a new storage bucket with name name.

    2. Set bucket’s quota value to quota.

    3. Set shelf’s bucket map[name] to bucket.

  9. If persisted is true, set bucket’s bucket mode to "persistent".

  10. Set bucket’s expiration to expires milliseconds after the Unix epoch.

  11. Let storageBucket be a new StorageBucket.

  12. Set storageBucket’s storage bucket to bucket.

  13. Return storageBucket.

To validate a bucket name given string name, run the following steps:

  1. If name contains any code point that is not ASCII lower alpha, ASCII digit, U+005F (_), or U+002D(-), then return failure.

  2. If name code point length is 0 or exceeds 64, then return failure.

  3. If name begins with U+005F (_) or U+002D(-), then return failure.

  4. Return.

To get or expire a bucket on a shelf given string name, run the following steps:

  1. Let bucket be shelf’s bucket map[name] if exists. Otherwise return null.

  2. If bucket’s expiration time is non-null and before the relevant settings object's current wall time, then:

    1. Set bucket’s removed to true.

    2. Return null.

  3. Return bucket.

1.2. Deleting a bucket

The delete(name) method steps are:

  1. Let environment be this's relevant settings object.

  2. Let shelf be the result of running obtain a local storage shelf given environment.

  3. If shelf is failure, then return a promise rejected with a TypeError.

  4. Let p be a new promise.

  5. If the result of validate a bucket name with name is failure, then reject p with an InvalidCharacterError.

  6. Otherwise, enqueue the following steps to storage bucket manager:

    1. Run remove a bucket with shelf and name.

    2. Queue a storage task to resolve p.

  7. Return p.

To remove a bucket on a shelf given a bucket name, run the following steps:

  1. Let bucket be shelf’s bucket map[name] if exists. Otherwise return.

  2. Remove key name in shelf’s bucket map.

  3. Set bucket’s removed to true.

  4. Return.

[IndexedDB-3] needs to define how deletion occurs when data is evicted by quota.

[FS] needs to define how deletion occurs for Bucket File System when data is evicted by quota.

[Service-Workers] needs to define how deletion occurs for CacheStorage and Service Workers when data is evicted by quota.

1.3. Enumerating buckets

The keys() method steps are:

  1. Let shelf be the result of running obtain a local storage shelf.

  2. If shelf is failure, then return a promise rejected with a TypeError.

  3. Let p be a new promise.

  4. Let keys be a new list.

  5. Enqueue the following steps to storage bucket manager:

    1. For each key in shelf’s bucket map, run the following steps:

      1. Let bucket be the result of running get or expire a bucket with shelf and key.

      2. If bucket is non-null, append key to keys.

    2. Queue a storage task to resolve p with keys.

  6. Return p.

2. The StorageBucket interface

[Exposed=(Window,Worker),
 SecureContext]
interface StorageBucket {
  readonly attribute DOMString name;

  [Exposed=Window] Promise<boolean> persist();
  Promise<boolean> persisted();

  Promise<StorageEstimate> estimate();

  Promise<undefined> setExpires(DOMHighResTimeStamp expires);
  Promise<DOMHighResTimeStamp?> expires();

  [SameObject] readonly attribute IDBFactory indexedDB;

  [SameObject] readonly attribute CacheStorage caches;

  Promise<FileSystemDirectoryHandle> getDirectory();
};

A StorageBucket has an associated storage bucket.

A storage bucket has an associated removed flag, which is a boolean, initially false. Set as true when a storage bucket is deleted.

A StorageBucket has a DOMString object name which is the key in the bucket map that maps to the storage bucket.

2.1. Persistence

Merge with Storage § 4.5 Storage buckets which already defines bucket mode.

The persist() method steps are:

  1. Let bucket be this's storage bucket.

  2. Let environment be this's relevant settings object.

  3. Let p be a new promise.

  4. Run the following steps in parallel:

    1. If bucket’s removed flag is true, then queue a storage task to reject p with an InvalidStateError.

    2. Let persisted be true if bucket’s bucket mode is "persistent".

    3. Otherwise,

      1. Let permission be the result of getting the current permission state with "persistent-storage" and environment.

      2. If permission is "granted", then set bucket’s bucket mode to "persistent" and set persisted to true.

      3. Otherwise, set persisted to false.

    4. Queue a storage task to resolve p with persisted.

  5. Return p.

The persisted() method steps are:

  1. Let p be a new promise.

  2. Let bucket be this's storage bucket.

  3. Otherwise, run these steps in parallel:

    1. If bucket’s removed flag is true, then queue a storage task to reject p with an InvalidStateError.

    2. Let persistent be true if bucket’s bucket mode is "persistent", otherwise false.

    3. Queue a storage task to resolve p with persistent.

  4. Return p.

2.2. Quota

A storage bucket has a quota value, a number-or-null, initially null. Specifies the upper limit of usage in bytes which can be used by the bucket. The user agent MAY further limit the realized storage space.

The storage usage of a storage bucket is an implementation-defined rough estimate of the number of bytes used by all of its storage bottles.

The estimate() method steps are:

  1. Let environment be this's relevant settings object.

  2. Let shelf be the result of running obtain a local storage shelf with environment.

  3. If shelf is failure, then return a promise rejected with a TypeError.

  4. Let bucket be this's storage bucket.

  5. If bucket’s removed flag is true, then return a promise rejected with an InvalidStateError.

  6. Let p be a new promise.

  7. Otherwise, run the following steps in parallel:

    1. Let quota be storage quota for shelf.

    2. Set quota to bucket’s quota value if it is non-null.

    3. Let usage be storage usage for bucket.

    4. Let dictionary be a new StorageEstimate dictionary whose usage member is usage and quota member is quota.

    5. Queue a storage task to resolve p with dictionary.

  8. Return p.

2.3. Expiration

A storage bucket has an expiration time, which is either null or a moment on the wall clock, initially null. Specifies the upper limit of a bucket lifetime.

The get or expire a bucket algorithm removes expired buckets when keys() or open() is called. User agents MAY clear buckets whose bucket mode is "best-effort" before their expiration time when faced with storage pressure. User agents MAY remove any buckets before open() or keys() is called when the expiration is reached regardless of the bucket mode

The setExpires(expires) method steps are:

  1. Let p be a new promise.

  2. Let bucket be this's storage bucket.

  3. Otherwise, run these steps in parallel:

    1. If bucket’s removed flag is true, then queue a storage task to reject p with an InvalidStateError.

    2. Otherwise, set bucket’s expiration time to expires milliseconds after the Unix epoch.

    3. Queue a storage task to resolve p.

  4. Return p.

The expires() method steps are:

  1. Let p be a new promise.

  2. Let bucket be this's storage bucket.

  3. Otherwise, run these steps in parallel:

    1. If bucket’s removed flag is true, then queue a storage task to reject p with an InvalidStateError.

    2. Otherwise, let expiration be bucket’s expiration time.

    3. Queue a storage task to resolve p with expiration.

  4. Return p.

2.4. Using storage endpoints

Storage endpoints, i.e. storage bottles, can be accessed as described below.

2.4.1. Using Indexed Database

IDBFactory methods need to take a storage bottle map rather than a storageKey.

A StorageBucket has an IDBFactory object, initially null. The indexedDB getter steps are:

  1. If this's indexedDB is null, run the following steps:

    1. Let bucket be this's storage bucket.

    2. Let bottle map be the result of obtain a local storage bottle map with bucket and "indexedDB".

    3. Let indexedDB be an IDBFactory object.

    4. Set the storage bottle map for indexedDB to bottle map.

    5. Set this's indexedDB to indexedDB.

  2. Return this's indexedDB.

2.4.2. Using CacheStorage

A StorageBucket has a CacheStorage object, initially null. The caches getter steps are:

  1. If this's caches is null, run the following steps:

    1. Let bucket be this's storage bucket.

    2. Let bottle map be the result of obtain a local storage bottle map with bucket and "cacheStorage".

    3. Let cacheStorage be a CacheStorage object.

    4. Set the relevant name to cache map for cacheStorage to bottle map.

    5. Set this's caches to cacheStorage.

  2. Return this's caches.

2.4.3. Using a Bucket File System

[Storage] needs to define helpers to retrieve the bottle map for a given (non-default) bucket.

[FS] needs to define a helper to retrieve an OPFS given a bottle map.

The getDirectory() steps are:

  1. Let map be the result of obtain a local storage bottle map with this's storage bucket and "fileSystem".

  2. Return the result of getDirectory with map.

2.5. Clear Site Data integration

Update Clear Site Data § 3.1 The Clear-Site-Data HTTP Response Header Field.

"storage:bucket-name"

If the type string starts with "storage:" then the remaining characters after the : will be taken to refer to a specific storage bucket in the origin of a particular response’s URL.

add the steps below to the algorithm in Clear Site Data § 4.1 Parsing.

To parse a Clear-Site-Data header with buckets, execute the following steps:

  1. For each type in header, execute the following steps:

    1. If type does not start with "storage:", abort these steps.

    2. Let bucket name be the code unit substring from 8 to end of type.

    3. If the result of validate a bucket name with bucket name is failure, then abort these steps.

    4. Append a tuple consisting of ("storage-bucket", bucket name) to types

add the steps below to the algorithm in Clear Site Data § 4.2 Clear data for response.

To clear data with buckets given a bucket name, execute the following steps:

  1. Let environment be this's relevant settings object.

  2. Let shelf be the result of running obtain a local storage shelf given environment.

  3. If shelf is failure, then throw a TypeError and abort these steps.

  4. For each type in types, execute the following steps:

    1. If type is not a tuple or type[0] is not "storage-bucket", abort these steps.

    2. Let bucket be shelf’s bucket map[bucket name] if one exists. Otherwise abort these steps

    3. Remove bucket.

3. Security and privacy considerations

Conformance

Document conventions

Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this specification.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes. [RFC2119]

Examples in this specification are introduced with the words “for example” or are set apart from the normative text with class="example", like this:

This is an example of an informative example.

Informative notes begin with the word “Note” and are set apart from the normative text with class="note", like this:

Note, this is an informative note.

Index

Terms defined by this specification

Terms defined by reference

References

Normative References

[CLEAR-SITE-DATA]
Mike West. Clear Site Data. URL: https://w3c.github.io/webappsec-clear-site-data/
[FS]
Austin Sullivan. File System Standard. Living Standard. URL: https://fs.spec.whatwg.org/
[HR-TIME-3]
Yoav Weiss. High Resolution Time. URL: https://w3c.github.io/hr-time/
[HTML]
Anne van Kesteren; et al. HTML Standard. Living Standard. URL: https://html.spec.whatwg.org/multipage/
[IndexedDB-3]
Joshua Bell. Indexed Database API 3.0. URL: https://w3c.github.io/IndexedDB/
[INFRA]
Anne van Kesteren; Domenic Denicola. Infra Standard. Living Standard. URL: https://infra.spec.whatwg.org/
[PERMISSIONS]
Marcos Caceres; Mike Taylor. Permissions. URL: https://w3c.github.io/permissions/
[RFC2119]
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. March 1997. Best Current Practice. URL: https://datatracker.ietf.org/doc/html/rfc2119
[Service-Workers]
Jake Archibald; Marijn Kruisselbrink. Service Workers. URL: https://w3c.github.io/ServiceWorker/
[Storage]
Anne van Kesteren. Storage Standard. Living Standard. URL: https://storage.spec.whatwg.org/
[WEBIDL]
Edgar Chen; Timothy Gu. Web IDL Standard. Living Standard. URL: https://webidl.spec.whatwg.org/

IDL Index

[SecureContext]
interface mixin NavigatorStorageBuckets {
  [SameObject] readonly attribute StorageBucketManager storageBuckets;
};
Navigator includes NavigatorStorageBuckets;
WorkerNavigator includes NavigatorStorageBuckets;

[Exposed=(Window,Worker),
 SecureContext]
interface StorageBucketManager {
    Promise<StorageBucket> open(DOMString name, optional StorageBucketOptions options = {});
    Promise<sequence<DOMString>> keys();
    Promise<undefined> delete(DOMString name);
};

dictionary StorageBucketOptions {
  boolean persisted = false;
  unsigned long long quota;
  DOMHighResTimeStamp expires;
};

[Exposed=(Window,Worker),
 SecureContext]
interface StorageBucket {
  readonly attribute DOMString name;

  [Exposed=Window] Promise<boolean> persist();
  Promise<boolean> persisted();

  Promise<StorageEstimate> estimate();

  Promise<undefined> setExpires(DOMHighResTimeStamp expires);
  Promise<DOMHighResTimeStamp?> expires();

  [SameObject] readonly attribute IDBFactory indexedDB;

  [SameObject] readonly attribute CacheStorage caches;

  Promise<FileSystemDirectoryHandle> getDirectory();
};

Issues Index

[IndexedDB-3] needs to define how deletion occurs when data is evicted by quota.
[FS] needs to define how deletion occurs for Bucket File System when data is evicted by quota.
[Service-Workers] needs to define how deletion occurs for CacheStorage and Service Workers when data is evicted by quota.
Merge with Storage § 4.5 Storage buckets which already defines bucket mode.
IDBFactory methods need to take a storage bottle map rather than a storageKey.
[Storage] needs to define helpers to retrieve the bottle map for a given (non-default) bucket.
[FS] needs to define a helper to retrieve an OPFS given a bottle map.
Update Clear Site Data § 3.1 The Clear-Site-Data HTTP Response Header Field.
add the steps below to the algorithm in Clear Site Data § 4.1 Parsing.
add the steps below to the algorithm in Clear Site Data § 4.2 Clear data for response.