CBOR-LD Varint

To enable unbounded extension on possible use cases for CBOR-LD that require different compression table material for consumption while working within a fixed number of CBOR tag values, we define the following.

Implementers MUST interpret the last byte of the two-byte CBOR tag value on a CBOR-LD payload as the beginning of a varint. If the CBOR tag is in the range `0x0600`–`0x067F`, the last byte of the CBOR tag is a one-byte varint, and the tagged item MUST be a map containing the encoded JSON-LD. If the CBOR tag is `0x0680` or greater, the tagged item MUST be an array containing two elements. The first item in the array MUST be a major type 2 byte string containing the rest of the varint, and the second item in the array MUST be a map containing the encoded JSON-LD. See Algorithm for more information.

The value of this varint is then used to lookup a CBOR-LD Varint Registry Entry in the CBOR-LD Varint Registry.

CBOR-LD Varint Registry

The CBOR-LD Registry is a global list that provides consumers of CBOR-LD payloads the information they need to reconstruct the term codec map required for decompression. A CBOR-LD Varint Registry Entry contains the following:

1. Registry Entry ID: a positive integer.
2. Use Case: what type of CBOR-LD payload this entry is used for.
3. `typeTables`: an array containing what `Type Tables` are to be used for this type of payload.
4. `processingModel`: what processing model is used for this registry entry. A processing model specifies how auto-generated CBOR-LD values are created from JSON-LD contexts as well as what type encoders are used alongside the `Type Tables` (e.g. how to partially compress an `xsd:dateTime` value that does not appear in `Type Table`). The default processing model, which will be defined later in this specification, will be used unless otherwise specified in the Registry Entry.
5. Provisional: a yes/no flag indicating whether the entry is provisional. Provisional entries may change or be removed.

The `typeTables` associated with a CBOR-LD Varint Registry Entry MUST be an array of or JSON objects. The only exception is the string "callerProvidedTable", which may appear in this array, denoting that for this use case, a `Type Table` is required which is not globally defined.

1. `type`: a JSON-LD type.
2. `table`: a JSON object that maps values of the above type to integers.

If a JSON object is present in the `typeTables` array, it MUST be in the above format.

The following is the current CBOR-LD registry:

JSON-LD to CBOR-LD Algorithm

This algorithm takes JSON-LD objects `jsonldDocument` and `options` as well as an integer `registryEntryId` as input.

1. Let `result` be an empty CBOR-encoded byte array.
2. Set {`varintTagValue`, `varintBytesValue`} to the return value of the "Get CBOR-LD Varint Structure Algorithm", passing `registryEntryId` as input.
3. If the "Get CBOR-LD Varint Structure Algorithm" resulted in an error, set `result` to the return value of the "Generate Uncompressed CBOR-LD Algorithm".
4. Otherwise:
   1. Initialize `typeTables` to an empty map
   2. For each entry in the `typeTables` array in the CBOR-LD Varint Registry Entry associated with `registryEntryId`, dereference the URL if necessary and add ${type}: ${table} from the resulting document to `typeTables`. If "callerProvidedTable" appears in `contextTables`, populate `typeTables` with `options.callerProvidedTable` as well.
   3. Set `result` to the return value of the "Generate Compressed CBOR-LD Algorithm" passing `typeTables` as `options.typeTable`, `varintTagValue` as `options.varintTagValue`, and `varintBytesValue` as `options.varintBytesValue`.
5. Return `result`.

Uncompressed CBOR-LD Buffer Algorithm

This algorithm takes a JSON-LD object `jsonldDocument`, integer `registryEntryId` and `options` as input.

1. Let `result` be an empty CBOR-encoded byte array.
2. Set the first two bytes (CBOR Tag) to 0x0600 (CBOR-LD - 0x06, Uncompressed - 0x00)
3. For every key-value in the map, generate the Uncompressed CBOR-LD Buffer by converting it to the associated CBOR-LD header and value. For complex values (maps, arrays), recursively convert the value to something that will losslessly encode and decode back to JSON-LD.
4. Return the Uncompressed CBOR-LD Buffer.

Compressed CBOR-LD Buffer Algorithm

This algorithm takes a JSON-LD object `jsonldDocument` and `options` as input. The `options` MUST contain:

`typeTable`

A map of JSON-LD types to maps. Each of these maps will map values of the associated type to their encoded CBOR-LD values.

`varintTagValue`

The CBOR tag value that will be used in the resulting payload.

`varintBytesValue`

The bytes that will make up the rest of the CBOR-LD varint. `options.varintBytesValue` MAY be `null`.

1. Let `result` be an empty CBOR-encoded byte array.
2. Set the first three bytes of `result` to `options.varintTagValue`.
3. If `options.varintBytesValue` is not `null`, set the next bytes of `result` to `options.varintBytesValue`.
4. Initialize `termCodecMap` to the result of the , passing `options.contextUrls` and `options.termMap` as input.
5. Add to `result` by recursively processing every name-value pair in `jsonldDocument`
   1. Let `termHint` be the value associated with the JSON name in the `termCodecMap`.
   2. Set the CBOR key to the `termHint.value` value.
   3. Set the CBOR value to the result of the `termHint.valueCompressor` function.
6. Return `result`.

Get Term Codec Map Algorithm

This algorithm takes a map `typeTable` and returns a CBOR-LD term codec map that maps JSON-LD terms to their associated byte values and value compression functions.

1. Let `result` be an ordered map.
2. For each value in `contextUrls`, dereference the JSON-LD contexts and process every entry.
   1. Set the entry key to the JSON-LD term key.
   2. Set the entry value to an unordered map with two entries.
      1. The first entry should be set to `value` with an undefined value.
      2. Let `compressor` be a known compressor function associated with the `@type` property from `typeTable`, a type-specific generic compressor associated with the processing model in use, or the generic CBOR compressor function, which returns the bytes associated with a typical CBOR compression of the given datatype.
3. Let `sortedTerms` be the value of sorting all of the keys in `result`.
4. For every value in the list of `sortedTerms` set the associated `termHint.value` value to the associated index of `sortedTerms`.
5. Return `result`.

Get Context URLs Algorithm

1. Let `result` be a ordered map.
2. Walk the JSON tree, for each JSON name-value pair:
   1. If the name is `@context`
      1. Add all values that are referenced by a URL to `result` where the key in the map is set to the JSON value associated with `@id`.
   2. If a non-URL value is detected, throw an ERR_NON_URL_JSONLD_CONTEXT_DETECTED error.
3. Return `result`.

Get CBOR-LD Varint Structure Algorithm

1. If `registryEntryId` is less than 128:
   1. Set `varintEncoded` to the byte encoding of `registryEntryId`.
   2. Set `varintTagValue` to the result of appending `varintEncoded` to the end of the bytes 0xD906, and set `varintBytesValue` to `null`.
2. Otherwise:
   1. Set `varintArray` to an array containing the varint representation of `registryEntryId`.
   2. Set `varintTagValue` to `varintArray[0]` appended to the end of the bytes 0xD906.
   3. Set `varintBytesValue` to a CBOR array containing the rest of `varintArray`.
3. Return {varintTagValue, varintBytesValue}.