This project uses Docker containers for development and testing, and
to provide a consistent UI.
Build a new Docker image and execute the test suite with:
Launch an interactive shell inside a container, with your workspace visible inside the container, with:
To list all available targets, execute
make without any arguments, e.g.,
equuleus:ska-tmc-cdm $ make build build the application image down stop develop/test environment and any interactive session help show this help. interactive start an interactive session using the project image (caution: R/W mounts source directory to /app) lint lint the application (static code analysis) piplock overwrite Pipfile.lock with the image version pull download the application image push push the image to the Docker registry test test the application up start develop/test environment
SKA Tango devices have commands that accept structured arguments and/or return structured responses. These structured data are often expressed as JSON-formatted strings.
The Configuration Data Model (CDM) is a data model used to describe subarray resource allocations and the subsequent configuration of those resources. It is effectively the superset of the configurations used by receptors, correlators, and data processing systems. The CDM is one such example of structured data delivered to TMC Tango devices.
This project defines object representations of the structured data passed to and from Tango devices, and serialisation schema used to convert the structured data to and from JSON. This project defines:
- a Python object model of the CDM;
- a Python object model for the structured arguments sent to TMC Tango devices and the structured responses received in return;
- serialisation schema to convert the Python object model instances to and from JSON.
- validation of the JSON strings sent between devices are compliant with the agreed interfaces.
The primary users of this shared library are the OET, SubArrayNode, and CentralNode. The OET uses this library to construct object representations of telescope configurations and resource allocation instructions, to convert those object representations to JSON-formatted payloads for TMC devices, and finally to convert the JSON responses returned by TMC devices back into Python objects.
It is intended that TMC devices also use this library to guarantee correct data exchange with the OET. TMC can also use this library to marshall and unmarshall its arguments to CSP and SDP Tango devices, which accept the appropriate subset of the JSON.
The CDM project contains three top-level packages,
ska_tmc_cdm.jsonschema as shown in the figure below.
package contains Python object models for the JSON command arguments agreed
in the ICDs. The
ska_tmc_cdm.schemas package contains code to transform the
classes defined in
ska_tmc_cdm.messages to and from JSON.
ska_tmc_cdm.jsonschema package contains
code to verify that the JSON strings sent between devices are compliant with the agreed interfaces.
The project layout and naming conventions are:
- Each Tango device has a corresponding Python sub-package in
- Code and schema for each Tango device command are located in Python modules inside their respective package.
- Structured input for the Tango command is modelled by a
- Structured output from the command is modelled by a
- Marshmallow schema are created to transform Python
Responseinstances to an from JSON, along with any other content they contain.
The Python object model for the JSON defined in the ICD is located in the
ska_tmc_cdm.messages package. In general, each CDM JSON entity is represented
as a Python class and each CDM attribute presented as a class property.
CDM attributes can be typed as plain Python data types (strings, floats, etc.) or, where appropriate, represented by rich objects if this provides additional value to the client. For example, while astronomical coordinates are represented by floats and strings in the JSON schema, in the object model they are defined as Astropy SkyCoord instances to ensure ensure correct coordinate handling and permit easier manipulation downstream. Similarly, quantities with units could be defined as instances of Astropy Quantity to provide additional functionality.
For details on the device messages modelled by this library, see:
Classes to marshall the
ska_tmc_cdm.messages objects to and from JSON are
defined in the
ska_tmc_cdm.schemas package. The ska-tmc-cdm project
uses Marshmallow for JSON serialisation. Classes
ska_tmc_cdm.schemas define Marshmallow schemas which are used by
Marshmallow during JSON conversion.
The CDM library uses the SKA Telescope Model to ensure the JSON accepted and JSON generated by the library are compliant with the schema declared by the data.
The entry points for code handling JSON schema validation is located in
ska_tmc_cdm.jsonschema module. This module contains methods for fetching
version-specific JSON schemas using interface URI and validating the structure
of JSON against these schemas. Json Schema validation functionality is enabled
by default with the parameter
validate=True when converting a
JSON string to CDM using
ska_tmc_cdm.schemas.CODEC.loads() and when converting
CDM to a JSON string using
Extending the CDM¶
Additional devices and applications cay use this library to communicate CDM elements wherever useful. Developers are encouraged to extend the ska-tmc-cdm project, adding object models and schemas for the structured arguments for their Tango devices.
The steps to extend the CDM are:
- Create a new package for the Tango device in
- For each device command, create a new module in the new package.
- If the command accepts structured input, define a
Requestclass in the module.
- If the command returns a structured response, define a
Responseclass in the module.
- With the Python object model defined, create a corresponding package and
module structure in
- In the schema module, define Marshmallow schemas to convert the object model classes and any structure to JSON.
- If this is a major entity, register the schema with the
ska_tmc_cdm.schemas.CODECobject using the