MQTT specifies how to construct a message header so it can be routed by an MQTT broker, but it does not specify the message content. Essentially, MQTT is an envelope specification, not a content specification. This poses challenges when MQTT clients from different developers need to work together. Clients need to agree upon the content of an MQTT message in order to interoperate.

If there are multiple MQTT clients from different developers operating in the same system, every client must know how to format messages for, and interpret messages from, every other client. It is impractical to expect every client developer to support every other.

Most MQTT clients format their message content as UTF-8 encoded JSON text. The semantics of the JSON message are determined by the client developer, based on the use case. For example, a weather sensor might generate JSON messages containing fields named “temperature”, “pressure” and “humidity”. Any MQTT client wishing to process this information must understand the meaning of each of these field names. This understanding requires two parts:

The schema can be expressed as a JSON Schema, which is a JSON document that describes the content and organization of a message. The semantics of a message must be described in terms of how to process that message. In effect, the semantics are encapsulated in a script.

Both the DataHub MQTT Client and MQTT Broker features offer the Advanced MQTT configuration option that allows you to specify JSON schemas and scripts for any number of different JSON formats. This allows MQTT applications in a heterogeneous environment to interoperate with one another, with SCADA and industrial applications, and with any other protocol supported by the DataHub program.

Receiving and Publishing

MQTT systems generally consist of “publishers” of information and “subscribers” to that information. When publishing, a client constructs a message payload from data that it manages, wraps the message in an MQTT envelope, and transmits that to a “topic” on an MQTT broker. An MQTT topic is similar to a point name, tag, item or node in control system terminology. A topic and its message payload constitute a (name, value) pair. The publisher typically creates the payload using string construction or serialization of a data structure. When the DataHub application is acting as a publisher, it constructs a string using an ASP document. That provides a convenient way to express both the literal text of the payload as well as the variable content from data points, clock, etc. It is the responsibility of the system integrator to provide this ASP document. In some cases, an MQTT client may expect different payload formats on different topics, even from the same publisher. This means that there may be more than one ASP document describing the different payloads, with each document associated with one or more topics. A receiving client typically must run code that encapsulates semantic knowledge about the message it is receiving. When the DataHub application is operating as a receiver, it uses an S-Sharp script to construct data points based on the fields in the message payload. S-Sharp is documented here. It is the responsibility of the system integrator to supply this script.

The publisher typically creates the payload using string construction or serialization of a data structure. When the DataHub application is acting as a publisher, it constructs a string using an ASP document. That provides a convenient way to express both the literal text of the payload as well as the variable content from data points, clock, etc. It is the responsibility of the system integrator to provide this ASP document.

In some cases, an MQTT client may expect different payload formats on different topics, even from the same publisher. This means that there may be more than one ASP document describing the different payloads, with each document associated with one or more topics.

A receiving client typically must run code that encapsulates semantic knowledge about the message it is receiving. When the DataHub application is operating as a receiver, it uses an S-Sharp script to construct data points based on the fields in the message payload. It is the responsibility of the system integrator to supply this script.

Determining which Parser to Use

In a typical system, devices can be added or removed from the network at any time. When those devices begin publishing their information, the receiver must be able to both detect that the device is active and to determine the type of information it is publishing. The DataHub MQTT Client and Broker examine each incoming message and compare it to the set of defined JSON schemas. If a schema match is found, they run the associated script to extract the data point information. The incoming message is compared to each JSON schema in order until the first match is found. If the system contains devices that produce similar messages, the schemas must be specified in order from most precise to least precise to ensure that the correct match is made.

This schema determination is done without user intervention, so there is no extra work required to add another device to the network.

Creating a New Message Format

You can create any number of message formats, each representing a different type of MQTT publishing on the network. Each format consists of a single JSON schema and its associated script, and zero or more publication formats as ASP documents.