Using and creating custom backends

Writing Dummies: Extending the DummyBackend

Extending the DummyBackend might be useful for writing simple automated tests. For more complicated tests, in particular those needing to simulate physical behaviour, it is recommended to use the VirtualLab library.

The Plugin Mechanism

Custom backends are provided as shared libraries. There is a plugin mechanism which registers the backend with the backend factory when the library is loaded, no matter whether it has been linked at compile time or is dynamically loaded at run time. The latter allows load all kinds of backends in generic applications QtHardMon or the DeviceAccess python bindings, although they have not explicitly been linked against those backends at compile time.

Linking custom backends at compile time (recommended)

As backends have to be compiled against the same DeviceAccess version as the application it is used in, it is recommended to link them in at compile time if it is already known that it will be needed. An application that is designed to read data from a particular DOOCS server for instance should be directly linked with the DOOCS backend.

Loading custom backends at run time

Custom backends are loaded via the dmap file using the @LOADLIB keyword, followed by the full path to the .so file with the backend. Use the symlink that ends on .so, not the ones with a version number.

@LOAD_LIB /path/to/your/lib/libCustomBackend.so

MY_CUSTOM_DEVICE (CUSTOM?map=my_device.map)

Example dmap file from the custom backend registration example (see examples/custom_backend_registration in your build directory).

To ensure binary compatibility of the loaded backed, the application and the backend must compiled against the same version of DeviceAccess. This cannot be done by the DeviceAccess library as it is only one component. There is a scheme how to create Debian packages in a way that all dynamically used backends are updated if the the using application is updated, and vice versa (see Debian packaging scheme for external backends). It should be applicable to other package managers as well (untested).

Writing your own backed

The backend has to fulfil a few requirements for the plugin mechanism to work.

• The backend has to provide a createInstance() function which can be given to the BackendFactory. Like this the factory can create the backend without knowing the details how to call the specific constructor.
• The backend has to be registered with the factory. This should automatically happen when the library is loaded.
• The library should be loadable at run time, not only at link time. This requires to check that the backend has been compiled with the the correct DeviceAccess version.

The CustomBackend example shows how to implement those three steps of the plugin mechanism.

// SPDX-FileCopyrightText: Deutsches Elektronen-Synchrotron DESY, MSK, ChimeraTK Project <chimeratk-support@desy.de>
// SPDX-License-Identifier: LGPL-3.0-or-later
 
#include <ChimeraTK/BackendFactory.h>
#include <ChimeraTK/DeviceAccessVersion.h>
#include <ChimeraTK/DummyBackend.h> // Would probably be DeviceBackendImpl or NumericAddressedBackend in a real application
 
#include <boost/make_shared.hpp>
 
/*
 * A custom backend which is registered to the factory.
 * This example only shows how to register a new type of backend to the factory.
 * It does not show how to write a new backend. We are lazy and derrive from
 * DummyBackend to have a fully working backend. In a real example you would
 * either derrive from DeviceBackendImpl or NumericAddressedBackend, unless you
 * want to write a custom dummy for testing.
 *
 * Custom backends are always created as a shared library which can be loaded at
 * run time.
 */
class CustomBackend : public ChimeraTK::DummyBackend {
 public:
  // C++11 shorthand syntax that we want a constructor with the same parameters
  // as the parent class.
  using ChimeraTK::DummyBackend::DummyBackend;
 
  /*
   * You have to implement a static function createInstance() with this exact
   * signature. This function is later given to the BackendFactory to create
   * this type of backend when it is requested.
   */
  static boost::shared_ptr<ChimeraTK::DeviceBackend> createInstance(
      std::string /*address*/, std::map<std::string, std::string> parameters) {
    /*
     * Inside createInstance the parameters are interpreted and passed on to the
     * constructor. Like this the backend constructor can have arbitrary
     parameters
     * while the factory can always call a function with the same signature.
 
     * In this example we have to convert the "map" parameter to an absolute
     path
     * (there is already a function for it in the DummyBackend parent class),
     * and pass it on to the constructor, which has the same signature as
     DummyBackend
     * (see 'using' clause above).
     *
     * This part will vary, depending on the requirements of the particular
     backend.
     */
    std::string absolutePath = convertPathRelativeToDmapToAbs(parameters["map"]);
 
    /*
     * Now we have all parameters for the constructor. We just have to create a
     * shared pointer of the CustomBackend with it.
     */
    return boost::make_shared<CustomBackend>(absolutePath);
  }
 
  /*
   * The task of the BackendRegister is to call the function which tells the
   * factory about the new type of backend. This is happening in the constructor
   * of the class, so you just have to create an instance of the class and the
   * code is executed.
   */
  struct BackendRegisterer {
    BackendRegisterer() {
      /*
       * The first parameter is the backend type string. It is the name by which
       * the factory knows which type of backend to create. The name has to be
       * unique. It shows up in the ChimeraTK device descriptor, in this case
       * (CUSTOM?map=example.map)
       * (example.map is the parameter which is passed on to createInstance, see
       * above)
       *
       * The second parameter is the pointer to the createInstance function.
       * The factory stores this pointer together with the type name and call
       * the functions when this type if backed needs to be created.
       */
      ChimeraTK::BackendFactory::getInstance().registerBackendType("CUSTOM", &CustomBackend::createInstance);
    }
  };
};
 
// We have one global instance if the BackendRegisterer. Whenever the library
// containing this backend is loaded, this object is instantiated. As the
// constructor of this class is registering the device, the backend is
// automatically known to the factory when the library is loaded.
static CustomBackend::BackendRegisterer gCustomBackendRegisterer;

Debian packaging scheme for external backends

In the ChimeraTK Debian packaging scheme, the libraries themselves contain the build number as part of the minor version number (e.g. /usr/lib/libChimeraTK-DeviceAccess.so.03.07focal1.01) and the package name itself contains the major and minor version number (e.g. libchimeratk-deviceaccess03-07-focal1). This allows to have several binary incompatible version of the same library to be installed at the same time, and each executable to unambiguously link against the correct version.

To get a consistent set of libraries at compile time, the -dev packages do not have a version number itself, but depend on the current versions of the dependencies. For instance, the package libchimeratk-device-access-dev in version 03.07focal1.01 depends on libchimeratk-cppext-dev (>= 01.04focal1), libchimeratk-cppext-dev (<< 01.04focal2). Like this, all -dev packages are consistently upgraded to a new version if one of them is upgraded, while the linker is searching for the .so-file without version number, which is a symlink to the versioned file. The Debian packaging scripts (https://github.com/ChimeraTK/DebianPackagingScripts) ensure that all packages are re-build if a dependency changes.

If the generic applications QtHardMon and DeviceAccess python bindings as well as the loadable backend packages would depend on lib-chimeratk-deviceaccess-dev, this would guarantee that /usr/lib/libMyCustomBackend.so could always be loaded at run time, without having to know the exact version number. The drawback of this solution is to always install the headers and the whole chain of build tools on the target machines, which is not wanted.

The mechanism to resolve this is based on two extra packages:

• libchimeratk-deviceaccess (without version number in the package name) is an empty meta package which acts as a reference anchor for the generic applications and the backends.
• libchimeratk-deviceaccess-mycustombackend (without version number in the package name)

The symlink /usr/lib/libMyCustomBackend.so has been moved out of the -dev package into the new package libchimeratk-deviceaccess-mycustombackend, and the -dev package depends on that new package. This custom backend package without version and the generic application depends on the correct version of libchimeratk-deviceaccess. If now either a backend or a generic application is upgraded on the target host, this will upgrade the required version of libchimeratk-deviceaccess as its dependency, which in turn will upgrade all packages that depend on it, i.e. all other generic applications and backends.

Usage in the packaging scripts

• Instead of having a dev and a lib package, loadable backends specify the two packages dev-noheader-dynload and lib in the Has-packages section of the DebiianBuildVersions CONFIG file.

  Has-packages: dev-noheader-dynload lib

• Generic executables that want to use dynamic backend loading define Use-dynload in the CONFIG file for the particular package (typically for the bin package)

  Has-packages: bin
  Use-dynload: bin

Next topic: Device Mapping