NOTICE FOR FUTURE RELEASES: AVOID CHANGING THE NUMBERING! The tests refer to the sections, incl. links and unlinked references from tests or other parts of the specification. These break, or even worse become wrong, when they are not changed consistenty!

A. Introduction

This document describes how initial values are propagated from the control system persistency layer, from the devices and from application modules into the attached components (control system, devices and other application modules).

This specification goes beyond ApplicationCore. It has impact on other ChimeraTK libraries like DeviceAccess, the ControlSystemAdapter and even backends and adapter implementations.

B. Definitions

Initial value: The start value of a process variable. The value is available to the receiving end of the process variable at a well defined point in time at the start. This is a logical concept. It is to be distinguished from the (often hardcoded) "value after construction" of the ProcessArray or any other NDRegisterAccessor implementation. The point in time when the value becomes available is well-defined, as described in the section C (high-level requirements).

C. High-level requirements

Initial values are available to all ApplicationModules at the start of the ApplicationModule::mainLoop(). No call to TransferElement::read() etc. is required. This implies that the ApplicationModule::mainLoop() is not started until all initial values are available, including those coming from devices which might potentially be offline, or from other ApplicationModules.
ApplicationModule implementations can either provide initial values for their outputs in ApplicationModule::prepare() (if the initial value doesn't depend on any inputs) or right after the start of the ApplicationModule::mainLoop() (if the initial value needs to be computed from the incoming initial values of the inputs).
The "value after construction"(*) is not propagated automatically during initial value propagation, not even with the DataValidity::faulty flag set. It is not visible to user code in the ApplicationModule::mainLoop().
1. An partial exception to this rule is the case if a device breaks (i.e. throws an exception) right after it has been successfully opened and initialised. In this case, the value of the variable is the value after construction and the DataValidity::faulty flag is set, but the VersionNumber is not {nullptr}. (*)
Since ApplicationModules and devices might wait for initial values from other ApplicationModules and devices, the modules might end up in a dead lock due to a circular connection. The circular connection is legal, but the dead lock situation needs to be broken by one ApplicationModule writing its initial value during ApplicationModule::prepare().
1. In case of such dead lock situation, there will be an error message stating the reason for the dead lock, and the application will be terminated.
(removed)
The control system does not wait for "initial values" as such. The first value of a process variable is sent to the control system when available. This may depend even on external conditions like the availability of devices, e.g. the control system interface has to run even if devices are not available and hence cannot send an inital value.
The first value received by the control system can be an initial value. This is different to ApplicationModules, where an initial value is never seen as a new value that can be read because they are always already there and received when the main loop starts.
Control system variables show the DataValidity::faulty flag until they have received the first valid value.
For push-type variables from devices, the initial is automatically send by the backend. The framework can just wait for a value to arrive. 9.1 According to the TransferElement specification, also after recovery from an exception the current value is send again. No special action needed by AplicationCore (c.f. Technical specification: Exception handling for device runtime errors V1.0).

(*) Comments

3. The value which is set in the constructor of all process variables is determined at compile time and usually is 0, which is basically always wrong. It is very important that no wrong data is transported initially. If it gets propagated within the application, modules will process this value (usually even if DataValidity::faulty is set), despite the value might present an inconsistent state with other process variables. If it gets propagated to the control system, other applications might act on an again inconsistent state.
3.a. This exempt case happens rarely in real applications and can be excluded in automated tests, hence it presents an acceptable simplification for the implementation (which otherwise had to retry the read attempt). It only occurs if the device was working and then immediately fails again before the initial value could be read.

D. Detailed requirements

All NDRegisterAccessor implementations (including but not limited to the ProcessArray) have the DataValidity::faulty flag set after construction for the receiving end. This ensures, all data is marked as faulty as long as no sensible initial values have been propagated. The sending end has DataValidity::ok after construction, so that the first written data automatically propagates the ok state by default. For bidirectional variables, this is the case for both directions separately. [ T]
All NDRegisterAccessor implementations have initially a VersionNumber constructed with a nullptr, which allows to check whether this variable is still at its "value after construction", or the initial value propagation already took place. [ T]
ApplicationModule (and ThreadedFanOut/TriggerFanOut) propagate the DataValidity of its outputs according to the state of all inputs. This behaviour is identical to later during normal data processing and specified in the Technical specification: data validity propagation. [ T]
(removed)
Control system variables: [Testing should be done in control system adapter.]
1. Variables with the control-system-to-application direction are written exactly once at application start by the control system adapter with their initial values from the persistency layer. This is done before ApplicationBase::run() is called, or soon after (major parts of the application will be blocked until it's done). If the persistency layer can persist the DataValidity, the initial value have the correct validity. Otherwise, initial values always have the DataValidity::ok.
2. Variables with the application-to-control-system direction conceptually do not have an "initial value". The control system adapter implementation does not wait for an initial value to show up. The first value of these variables are written at an undefined time after the ApplicationBase::run() has been called. They might or might not be initial values of other modues. The control system adapter does not expect any specific behaviour. Entities writing to these variables do not need to take any special precautions.
Device variables:
1. Write accessors are written after the device is opened and the initialisation is done, as soon as the initial value is available for that variable. Initial values can be present through 5.a, 6.b, 6.c or 7. [ T]
2. Initial values for read accessors without AccessMode::wait_for_new_data are read after the device is openend and the initialsation is done. [ T]
3. Initial values for read accessors with AccessMode::wait_for_new_data are provided by the Device backend as the first value after the device has been opeend and asynchronous reads have been activated. [ T]
Outputs of ApplicationModules:
1. Initial values can be written in ApplicationModule::prepare(), if the value does not depend on any input values (since input values are not available during prepare()). [ T]
2. Alternatively, initial values can be written in ApplicationModule::mainLoop() before calling any read function, if they depend on initial values of the inputs. Typically, to propagate the initial values of its inputs, an ApplicationModule will run its computations and write its outputs first before waiting for new data with a blocking read() and the end of the processing loop. The application author needs to take into account that the ApplicationModule::mainLoop() will only start after all inputs have their initial values available, which might depend on the avaialbility of devices. [ T]
3. Since in ApplicationModule::prepare() all devices are still closed, any writes to device variables at this point need to be delayed until the device is open. As the ExceptionHandlingDecorator takes care of this, the appciation module can just call write() on its output and does not have to do any special actions here (see Technical specification: Exception handling for device runtime errors B.2.3).
Inputs of ApplicationModules:
1. Initial values are read by the framwork before the start of ApplicationModule::mainLoop() (but already in the same thread which later executes the mainLoop()).
2. Initial values are read by calling TransferElement::read(), which freezes until the initial value has arrived, both with AccessMode::wait_for_new_data and without.
  1. For device variables the ExeptionHandlingDecorator freezes the variable until the device is available. [ T]
  2. ProcessArray freeze in their implementation until the initial value is received (see. E ProcessArrays). [ T]
  3. Constants can be read exactly once in case of AccessMode::wait_for_new_data, so the initial value can be received. (see. 10) [ T]
The module-like fan outs ThreadedFanOut and TriggerFanOut (does not apply to the accessor-like fan outs FeedingFanOut and ConsumingFanOut)
1. The fan outs have a transparent behaviour, i.e. an entity that receives an initial value through a fan out sees the same behaviour as if a direct connection would have been realised.
2. This implies that the inputs are treated like described in 8.b. [ T for ThreadedFanOut] [ T for TriggerFanOut]
3. The initial value is propagated immediately to the outputs. (*)
Constants (Application::makeConstant()): [ T]
1. Constant accessors only have one value, which is propagates as the intitial value. They conceptually behave like a process array that is written exactly once by the framework.
  1. This implies that in case of AccessMode::wait_for_new_data the value can be read exactly once. This is enough to propagate the initial value. Any further read() to the variable will block infinitely.
2. Values are propagated before the ApplicationModule threads are starting (just like initial values written in ApplicationModule::prepare())
3. Devices are not opened yet at this point in time. Delaying the write is done with the same mechanism as described in 7.c.
Variables with a return channel ("bidirectional variables", ScalarPushInputWB, ScalarOutputPushRB and the array variants) behave like their unidirectional pendants, i.e. the existence of the return channel is ignored during the initial value propagation.

Comments

To 5.: This is the responsibility of each control system adpater implementation.
To 7. and 10.: An output of a ApplicationModule with an initial value written in ApplicationModule::prepare() and later never written again behaves in the same way as a constant.
To 9.c: Even if devices connected to outputs are in fault state, the ExceptionHandlingDecorator takes care that the writing is delayed. The fan out is not blocked by this.

E. Implementation

NDRegisterAccessor implementations

Each NDRegisterAccessor must implement 1. separately.
Each NDRegisterAccessor must implement 2. separately. All accessors should already have a VersionNumber data member called currentVersion or similar, it simply needs to be constructed with a nullptr as an argument.

ApplicationModule

Implement 3, implementation will be already covered by normal flag propagation
API documentation must contain 7.
Implements 8. (hence takes part in 5.a, 6.b and 7 implicitly):
- All inputs of the module must be read in the ApplicationModule::mainLoopWrapper() before the call to mainLoop().

ThreadedFanOut

Implement 3, implementation will be already covered by normal flag propagation
Needs to implement 9. (hence takes part in 5.a, 6 and 7 implicitly):
- structure the fan out's "mainLoop"-equivalent (ThreadedFanOut::run()) like this:
  - read initial values
  - begin loop
  - write outputs
  - read input
  - cycle loop

TriggerFanOut

Implement 3, implementation will be already covered by normal flag propagation
Needs to implement 9. (hence takes part in 5.a, 6, and 7 implicitly):
- In contrast to the ThreadedFanOut, the TriggerFanOut has only poll-type data inputs which are all coming from the same device. Data inputs cannot come from non-devices.
- Structure the fan out's "mainLoop"-equivalent (TriggerFanOut::run()) like this:
  - read initial value of trigger input
  - begin loop
  - read data inputs via TransferGroup
  - write outputs
  - read trigger
  - cycle loop

DeviceModule

All points are already covered by Technical specification: Exception handling for device runtime errors V1.0.

ExceptionHandlingDecorator

All points are already covered by Technical specification: Exception handling for device runtime errors V1.0.

Application

(This section refers to the class Application, not to the user-defined application.)

Implements 10.b and partly 10.c (in combination with the exception handling mechanism)

ControlSystemAdapter

Must implement 5.a
- Needs to be done in all adapters separately

ProcessArrays

Must implement 8.b.ii, especially if wait_for_new_data is not set.

As ProcessArrays do not have a synchronous read channel which can be used to obtain the "current value", the implementation freezes all read operations (even readNonBlocking() and readLatest()) until a first value has been send. This is consistent with the behaviour of the ExceptionHandlingDecorator, which freezes until the device has become available and the synchronous channel can deliver data.

Comment: The original idea of an extra function readBlocking() only for process arrays does not work, because it breaks abstraction. To use it one would have to dynamic cast a TransferElement to ProcessArray, which does not work when the object is decorated (which it always is).

F. Known bugs [OUTDATED!!!]

DeviceAccess interface

1. is currently not implementable for (potentially) bidirectional accessors (like most backend accessors). An interface change is required to allow separete DataValidity flags for each direction.

NDRegisterAccessor implementations

1. is not implemented for Device implementations (only the UnidirectionalProcessArray is correct at the moment).
2. is not implemented for Device implementations (only the UnidirectionalProcessArray is correct at the moment).
Exceptions are currently thrown in the wrong place (see implementation section for the NDRegisterAccessor). A possible implementation to help backends complying with this rule would be:
- Introduce non-virtual TransferElement::readTransfer() etc, i.e. all functions like do[...]Transfer[...]() should have non-virtual pendants without do.
- These new functions will call the actual do[...]Transfer[...]() function, but place a try-catch-block around to catch all ChimeraTK exceptions
- The exceptions are stored and operation is continued. In case of boolean return values correct values must be implemented:
  - doReadTransferNonBlocking()and doReadTransferLatest() must return false (there was no new data), except for the ExceptionHandlingDecorator which has to return true if it will do a recovery in postRead() and there will be new data.
  - doWriteTransfer() shall return true (dataLost), except for the ExceptionHandlingDecorator. It should return true only if the data of the recovery accessor is replaced and the previous value has not been written to the hardware.
- postRead() and postWrite() must always be called. It currently depends on the boolean return value if there is one. Instead this value has to be handed to postRead() and postWrite() as an argument. Only the implementation can decide what it has to do and what can be skipped.
- With TransferElement::postRead() resp. TransferElement::postWrite() non-virtual wrappers for the post-actions already exist. In these functions, the stored exception should be thrown.
- All decorators and decorator-like accessors must be changed to call always the (new or existing) non-virtual functions in their virtual do[...] functions. This applies to both the transfer functions and the pre/post actions (for the latter it should be already the case).
- It is advisable to add an assert that no unthrown exception is present before storing a new exception, to prevent that exceptions might get lost due to errors in the business logic.

ApplicationModule / EntityOwner

3. is not properly implemented, the faultCounter variable itself is currently part of the EntitiyOwner. It will be moved to a helper class, so multiple instances can be used (needed for the TriggerFanOut). It is the responsibility of the decorators which manipulate the DataFaultCounter to increase the counter when they come up with faulty data in the inital state (see Technical specification: data validity propagation Specification version V1.0).

TriggerFanOut

3. is not correctly implemented, it needs to be done on a per-variable level.
It currently implements its own exception handling (including the Device::isOpened() check), but in a wrong way. After the NDRegisterAccessor has been fixed, this needs to be removed.

DeviceModule

Probably all points are duplicates with Technical specification: Exception handling for device runtime errors V1.0.

Merge DeviceModule::writeAfterOpen/writeRecoveryOpen lists.
Implement mechanism to block read/write operations in other threads until after the initialsation is done.

ExceptionHandlingDecorator

Some points are duplicates with Technical specification: Exception handling for device runtime errors V1.0.

It waits until the device is opened, but not until after the initialisation is done.
Provide non-blocking function.
Implement special treatment for first readLatest() operation to always block in the very first call until the device is available.
- Since readLatest() is always the first read-type function called, it is acceptable if all read-type functions implement this behaviour. Choose whatever is easier to implement, update the implementation section of this specification to match the chosen implementation.

VariableNetworkNode

Rename VariableNetworkNode::hasInitialValue() into VariableNetworkNode::initialValueUpdateMode(), and change the return type to UpdateMode. Remove the InitialValueMode enum.
Remove data member storing the presence of an initial value, this is now always the case. Change VariableNetworkNode::initialValueType() accordingly.
Document cleanly that the initialValueUpdateMode is different from the normal update mode as described in 8b.

ControlSystemAdapter

EPICS-Adapter might not properly implement 5.a, needs to be checked. Especially it might not be guaranteed that all variables are written (and not only those registered in the autosave plugin).
The UnidirectionalProcessArray uses always a default start value of DataValidity::ok, but overwrites this with DataValidity::faulty for the receivers in the factory function UnidirectionalProcessArray::createSynchronizedProcessArray() (both implementations, see UnidirectionalProcessArray.h). This can be solved more elegant after the DeviceAccess interface change described above.

Non-memeber functions

Implement the missing convenience function

Documentation

Documentation of ControlSystemAdapter should mention that implementations must take care about 5.
Documentation for ApplicationModule should mention 7.
Documentation of VariableNetworkNode::initialValueUpdateMode()