Technical specification: Mapping of DataConsistencyKeys to VersionNumbers V0.0WIP

A. Introduction

The DataConsistencyGroup allows to ensure consistency of the values read through different accessors. For this matching, the VersionNumber is used as a key. The VersionNumber is only valid within the same ChimeraTK application process, so when transporting data via a DeviceBackend, the VersionNumbers have to be mapped to/from some other key, such as the DOOCS Event ID or a timestamp. This key can either be shipped as metadata attached to the payload data, if this is foreseen in the protocol (as e.g. in the case of DOOCS), or it may be shipped in a separate register (as e.g. in case of an XDMA device with an interrupt).

In this document, the exact behaviour of this mapping is specified and the architecture of the implementation is laid out. Please note, only the read direction is for now covered by the implementation and this specification.

Note: Albeit gramatically incorrect, plural forms of class names are written with apostrophes throughout this document, so the class name is properly highlighted by Doxygen and linked to the class documentation.

B. Behavioural specification

• 1. The DataConsistencyRealm is a mapping of DataConsistencyKey to VersionNumber
  • 1.2 It has a name used to identify the DataConsistencyRealm, which may consist of alphanumeric characters only.
  • 1.3. Each entity (backend, accessor etc.) which needs to map an incoming DataConsistencyKey to a VersionNumber needs to get the appropriate DataConsistencyRealm from the DataConsistencyRealmStore singleton.
  • 1.4. All entities whose incoming DataConsistencyKey's have values which are comparable to each other (c.f. B.2) shall use the same DataConsistencyRealm to ensure consistency across entities.
• 2. The DataConsistencyKey is a uint64_t value which is used as a key value.
  • 2.2. The value is a key for sorting incoming data, also across different registers and even devices using the same DataConsistencyRealm, in time.
    • 2.2.1 Older data has smaller DataConsistencyKey's and will be attached with a smaller VersionNumber.
    • 2.2.2 Newer data has greater DataConsistencyKey's and will be attached with a greater VersionNumber.
    • 2.2.3 Data with identical DataConsistencyKey's is considered to be consistent and will be attached with the same VersionNumber.
    • 2.3.4 DataConsistencyKey's need to be approximately consecutive, with no big gaps between consecutivly incoming data.
• 3. DataConsistencyKey <=> ChimeraTK::VersionNumber mapping (in the DataConsistencyRealm)
  • Requirements from other specifications:
    • No VersionNumber{nullptr} is given out after the first successful transfer. (FIXME Reference)
    • All version numbers must not be older than the global version number at the time DeviceBackend::open() was called. (FIXME Reference)
    • VersionNumber's are never decreasing in the same TransferElement. (FIXME Reference)
  • 3.1 A DataConsistencyRealm is used across all backend instances to allow consistent VersionNumbers across backends
    • 3.1.1 The VersionNumber for DataConsistencyKey=0 is always VersionNumber{nullptr}. It is not added to the map.
    • 3.1.2 If a DataConsistencyKey is in the map, the according VersionNumber is returned by the realm.
    • 3.1.3 If a DataConsistencyKey is newer than the newest entry in the map, a new VersionNumber is generated and it is added to the map.
      • 3.1.3.1 The size of the map is limited to avoid infitely increasing memory for long-running servers
        • 3.1.3.1.1 If the map has reached the maximum size, the oldest entries are removed
        • 3.1.3.1.2 If a DataConsistencyKey is older that the oldest entry in the map, VersionNumber{nullptr} is returned
        • 3.1.3.1.3 Optional extension: The size can be configured via the CDD. TBD: How to deal with conflicting sizes? Take the maximum?
      • 3.1.3.2 Whenever a new DataConsistencyKey is added to the map, all DataConsistencyKey's between the newest DataConsistencyKey in the map and the incoming DataConsistencyKey are added to the map as well. This guarantees that a larger DataConsistencyKey always has a larger VersionNumber, even if DataConsistencyKey's are not created in increasing order. (*)
      • 3.1.3.3 If the first DataConsistencyKey is added to the still-empty map, the map is filled with keys before the incoming key, up to the maximum size of the map acording to 3.1.3.1. 3.1.4As described in 3.1.1 and 3.1.3.1.2, the realm may hand out VersionNumber{nullptr}, which must never be written to a TransferElement. 3.3.3.1 and 3.3.3.2 describe how this is ensured. For these data matching via the DataConsistencyGroup may not be possible or lead to false matches.
  • 3.2 When DeviceBackend::open() is called, a startVersionNumber is created.
    • 3.2.1 No version number smaller than the startVersionNumber is handed out from this point in time.
    • 3.2.2 It is the responsibility of each backend implementation to do this correct. The NumericAddressedBackend will already take care of this.
  • 3.3 RegisterAccessors get the VersionNumber from the DataConsistencyRealm, according to the incoming DataConsistencyKey
    • 3.3.1 Each TransferElement implementation needs to store the last used VersionNumber.
    • 3.3.2 If the incoming DataConsistencyKey results in a smaller VersionNumber than last used:
      • the last used VersionNumber is attached to the data
      • DataValidity is set to DataValidity::faulty
    • 3.3.3 If the incoming DataConsistencyKey results in a VersionNumber larger or equal to the last used VersionNumber:
      • 3.3.3.1 If that incoming VersionNumber is smaller than the startVersionNumber (see 3.2), the startVersionNumber is used instead (c.f. 3.2.1). This does not set the DataValidity to DataValidity::faulty. (*)
      • 3.3.3.2 If the max(incomingVersionNumber,startVersionNumber) is smaller than the last used VersionNumber:
        • the last used VersionNumber is attached to the data
        • DataValidity is set to DataValidity::faulty

Comments

• 3.1.3.2 This can happen since the first time a particular key is encountered may happen on different variables and even backends each time.
• 3.1.4 Giving out a VersionNumber{nullptr} would violate the DeviceAccess TransferElement specification.
• 3.3.3.1 This is the case for VersionNumber{nullptr} and valid VersionNumber's which have been created before calling open() of the backend.

C. Implementation for the AsyncNDRegisterAccessor

The following diagram shows the integration of the DataConsistencyRealm with the AsyncNDRegisterAccessor and AsyncAccessorManager concept. The diagram is not complete and shows only the relevant parts. New classes and new or modified class members are shown in red.

(Hint: Hover the mouse cursor over edge labels "see tooltip" to show the description.)

• 1. The global singleton DataConsistencyRealmStore is the factory for the DataConsistencyRealm's.
  • 1.1. The DataConsistencyRealmStore will keep track of the created DataConsistencyRealm's internally, so it hands out the same DataConsistencyRealm for the same realmName, as long as the realm still exists.
  • 1.2. The DataConsistencyRealmStore stores only weak_ptr of the realm objects. If a realm is no longer used, it will be cleaned up (so the VersionNumber mapping is forgotten).
• 2. Backends, like the DoocsBackend (called "SomeNonNumericAddressedBackend" in the above graph), which have their DataConsistencyKey value shipped together with the payload data in their protocol, need to obey the following rules in their event handler (called handleEvent() in the above graph, although the naming and structure may be vastly different for actual implementations):
  • 2.1. Such backends will typically use a fixed realm name, which should clearly refer to the protocol (e.g. "doocsEventId" in case of the DoocsBackend).
    • 2.1.1. The realm name should be overrideable through a CDD parameter.
  • 2.2. The event handler needs to extract the DataConsistencyKey from incoming data when an event is received.
  • 2.3. It shall then translate the key into the corresponding VersionNumber through the DataConsistencyRealm, and pass the VersionNumber to async::DomainImpl::distribute().
• 3. Backends, like the NumericAddressedBackend, which are using the poll mechanism of the AsyncNDRegisterAccessor concept (i.e. the TriggeredPollDistributor), need to provide the necessary information through the catalogue and can otherwise rely on the framework implementation:
  • 3.1. The BackendRegisterCatalogueBase contains a map of qualifiedAsyncSubDomainID to DataConsistencyRealm names.
  • 3.2. The CDD can contain a parameter "DataConsistencyKeys" to define the registers with the DataConsistencyKey values.
    • 3.2.1. These registers need to be triggered each by a different interrupt.
    • 3.2.2. The CDD parameter "DataConsistencyKeys" contains a JSON object, where each JSON key is interpreted as the register path and each value as the corresponding realm name.
    • 3.2.3. Based on this information, the NumericAddressedBackend will fill the BackendRegisterCatalogueBase::_dataConsistencyRealmIdMap
  • 3.3. Future map file formats will also allow specifying such pairs of key registers and realm names.
  • 3.4. The TriggeredPollDistributor
    • 3.4.1. obtains the DataConsistencyRealm and the key register path from the BackendRegisterCatalogue in its constructor, based on its owners subdomain ID,
    • 3.4.2. adds the accessor for the key register path to its TransferGroup,
    • 3.4.3. fetches the key valye in TriggeredPollDistributor::prepareIntermediateBuffers() (via TransferGroup::read()), and translates it into a VersionNumber using the DataConsistencyRealm,
    • 3.4.4. keeps track of the last VersionNumber it has sent out:
      • 3.4.4.1. Only if the VersionNumber matching the incoming key value is greater or equal the last sent VersionNumber, the new VersionNumber will be used.
      • 3.4.4.2. If the new VersionNumber is smaller then the last one, the last VersionNumber is sent out and the DataValidity is forced to DataValidity::faulty.
      • 3.4.4.3. "Using" the new VersionNumber in this context means to set SourceTypedAsyncAccessorManager::_version to the new version, so the subsequent distribution of values to the AsyncVariable's uses this VersionNumber.
  • 3.5. SourceTypedAsyncAccessorManager::distribute() will return its SourceTypedAsyncAccessorManager::_version (after it has potentially been modified in TriggeredPollDistributor::prepareIntermediateBuffers()) to the calling SubDomain::distribute().
  • 3.6. SubDomain::distribute() calls TriggeredPollDistributor::distribute() first and update its local VersionNumber, so a potential following VariableDistributor::distribute() will use the same VersionNumber as the TriggeredPollDistributor.
  • 3.6.1. This ensures that a register which is polled by an interrupt will have a matching VersionNumber as the void-typed interrupt itself.