Warning: This is a development version. The latest stable version is Version 4.0.1.

Decoder Design

The following high-level API outlines the minimum interactions a user should be able to have using the Rely decoder API:

// Configures the decoder to be ready for use
void configure(uint64_t max_packet_bytes, int64_t timeout,
               uint64_t max_stream_size);

// Read packets into the decoder
void consume(const uint8_t* packet, uint64_t bytes, int64_t now);

// Access decoded payloads
auto can_produce() const -> bool;
auto produce_bytes() const -> uint64_t;
auto produce_data() const -> const uint8_t*;
void produce_next();

// Flushing the decoder
auto has_upcoming_flush() const -> bool;
auto upcoming_flush(int64_t now) const -> int64_t;
void flush(int64_t now);

In this section outlines the design of the class decoder and the documents the decisions made.

This shows the flow of data through the decoder:

| Sliding window decoder | <-- Packets -- consume(...)
|    Defragmentation     |
|       Read queue       | -- Payloads --> produce_...()
  1. The user passes Packets to the decoder via the consume function. These packets are then parsed and added to the sliding window decoder as symbols.
  2. Source symbols decoded by the sliding window decoder are used by the defragmentation component. And timed out Source symbols are dropped.
  1. When Payloads have been reconstructed by the defragmentation component they are added to the output queue. The output queue can be accessed with the produce_...() functions.

Sliding window management

Symbols added to the decoder will be pushed to its Stream. In addition to maintaining the position of the symbol in the stream, the decoder also tracks:

  1. The timestamp of when the symbol was added to the Stream.
  2. Whether a symbol has been decoded.
  3. Whether a symbol has been released.

To determine when and how packets should be released the following rules are used:

  1. Symbols can only be released once.
  2. Symbols can only be released in order.
  3. Decoded symbols should be released as soon as possible. Preceding unreleased symbols may cause head-of-line blocking.
  4. Expired source symbols should be released immediately.
  5. Unreleased symbols about to be dropped should be released immediately.

The decoder will follow the Coding window of the incoming symbols. Gaps will be closed by pushing placeholder symbols. So consuming a symbol may cause the decoder’s Stream size to increase by more than one.

Flushing and timeouts

In order to ensure that symbols do not exceed the specified decoding timeout. A mechanism is needed to ensure that symbols move even if nothing happens or decoding fail.

It is the responsibility of the user to drive the mechanism that flushes out Expired source symbol. To enable this, the decoder exposes whether a flush may be pending and the time until the flush should occur. Each time the user interacts with the decoder the flush state may change.

E.g. if the decoder consumes a packet that causes a symbol in the Stream to become decoded, a previously pending flush event may no longer be needed.

The following rules must be followed:

  • A flush is pending if the stream contains unreleased symbols.
  • The time until a flush is the timeout of the oldest unreleased symbol in the stream, this can be queried with upcoming_flush.
  • The user must invoke flush if the value returned by upcoming_flush is 0 or less.
  • When new packets are consumed by the decoder pending flushes are performed automatically. This means that, in certain applications with a steady flow of data, manual flushes are never needed.