What is MQTT? Clients, brokers and the packets between them

Catalin NegruPublished 7 min read

Part of the MQTT guide

MQTT is a publish and subscribe messaging protocol carried over TCP. Clients connect to a broker, publish messages to named topics, and subscribe to the topics they want to receive; the broker does the routing. The packet overhead is small, as little as two bytes of fixed header, and every connection is opened by the client, which is what makes MQTT practical for devices on slow, metered or unreliable networks.

This page is the starting point of the MQTT guide. Topic syntax has its own page, MQTT topics and wildcards, and so do the delivery guarantees, MQTT QoS.

Clients, the broker and topics

There are two roles. The standard calls them client and server; in practice the server is called the broker.

  • A client opens a TCP connection to the broker and sends a CONNECT packet. It can then publish, subscribe, or both.
  • The broker accepts connections, checks who the client is, keeps each client’s subscriptions, and forwards every published message to the clients whose subscriptions match its topic.

A topic is a UTF-8 string with levels separated by /, such as site1/pump3/flow. Topics are not created or configured in advance. A topic exists because someone published to it or subscribed to it.

The payload is a sequence of bytes. MQTT does not define what is inside it: JSON, a plain number as text, a binary structure and Protocol Buffers are all valid. Publisher and subscriber have to agree on the format by some other means, which is the gap Sparkplug B fills for industrial data.

What is in an MQTT packet

Every MQTT control packet starts with a fixed header.

PartSizeContents
Byte 1, high 4 bits4 bitsPacket type: 1 CONNECT, 2 CONNACK, 3 PUBLISH, 8 SUBSCRIBE and so on
Byte 1, low 4 bits4 bitsFlags. For PUBLISH: DUP, two QoS bits and RETAIN
Remaining length1 to 4 bytesNumber of bytes that follow, as a variable byte integer
Variable headerDepends on the typeFor PUBLISH: the topic name, and a packet identifier when QoS is 1 or 2
PayloadDepends on the typeFor PUBLISH: the application message

The remaining length uses seven bits per byte for the value and the top bit to say another byte follows. One byte covers lengths up to 127, and the four-byte maximum is 268,435,455, so a single packet can be about 256 MB. Brokers set much lower limits.

MQTT 3.1.1 defines 14 packet types. MQTT 5.0 adds a fifteenth, AUTH.

A decoded CONNECT and PUBLISH

A client with the identifier pump3 connects with MQTT 3.1.1, a clean session and a 60 second keep alive.

CONNECT: 10 11 00 04 4D 51 54 54 04 02 00 3C 00 05 70 75 6D 70 33 10 packet type 1 (CONNECT), flags 0 11 remaining length: 17 bytes follow 00 04 length of the protocol name 4D 51 54 54 "MQTT" 04 protocol level 4 (MQTT 3.1.1; MQTT 5.0 sends 05) 02 connect flags: clean session 00 3C keep alive: 60 seconds 00 05 length of the client identifier 70 75 6D 70 33 "pump3"
CONNACK: 20 02 00 00 20 packet type 2 (CONNACK) 02 remaining length 00 acknowledge flags: no stored session 00 return code 0: connection accepted

The client then publishes the text 42.7 to site1/pump3/flow at QoS 0.

PUBLISH: 30 16 00 10 73 69 74 65 31 2F 70 75 6D 70 33 2F 66 6C 6F 77 34 32 2E 37 30 packet type 3 (PUBLISH), DUP 0, QoS 0, RETAIN 0 16 remaining length: 22 bytes follow 00 10 topic length: 16 bytes 73 ... 77 "site1/pump3/flow" 34 32 2E 37 payload "42.7"

That is 24 bytes on the wire for a 4-byte reading, most of it the topic name. In MQTT 5.0 the same packet carries one more byte, a property length of 00, between the topic and the payload.

Sessions, keep alive and the Last Will

Three fields in CONNECT decide how a client behaves when the network fails.

Client identifier. The broker keys session state on it, and it must be unique per broker. If a second client connects with an identifier that is already in use, the broker disconnects the first one. Two devices that share an identifier will take turns disconnecting each other for as long as both retry. An MQTT 3.1.1 broker must accept identifiers of 1 to 23 characters drawn from digits and unaccented letters, and may accept longer ones.

Clean session. In MQTT 3.1.1, a client that connects with the clean session flag set to 1 starts from nothing and its session is discarded when it disconnects. With the flag at 0, the broker keeps the client’s subscriptions and the QoS 1 and QoS 2 messages that arrive for it while it is offline. MQTT 5.0 splits this into a Clean Start flag and a Session Expiry Interval in seconds.

Keep alive. A 16-bit number of seconds, so at most 65,535. The client must send some packet within each keep alive period, and sends a two-byte PINGREQ when it has nothing else to send. If the broker hears nothing for one and a half times the period, it closes the connection. A keep alive of 0 turns the mechanism off.

Last Will. CONNECT can carry a will: a topic, a payload, a QoS and a retain flag. The broker stores it and publishes it if the connection ends without the client sending DISCONNECT, for example when the keep alive expires. A device that publishes online as a retained message when it connects and registers offline as its retained will gives every subscriber a correct status topic.

MQTT 3.1.1 and MQTT 5.0

MQTT 3.1.1 became an OASIS standard in 2014 and is also published as ISO/IEC 20922. MQTT 5.0 followed in 2019. There is no version 4; the protocol level byte went from 4 to 5. The two versions are not wire compatible, and a client states which one it speaks in CONNECT.

AreaMQTT 3.1.1MQTT 5.0
ErrorsA return code in CONNACK and SUBACK onlyA reason code on every acknowledgement, and the broker can send DISCONNECT with a reason
Session lifetimeClean session flagClean Start flag plus Session Expiry Interval
Message lifetimeNoneMessage Expiry Interval
MetadataNoneUser properties, content type, payload format indicator
Request and responseBy conventionResponse Topic and Correlation Data properties
Load sharingBroker-specificShared subscriptions
Flow controlNoneReceive Maximum
AuthenticationUser name and passwordThe same, plus an extensible exchange using AUTH packets

Most device libraries and brokers support both. Use 5.0 on a new system when the broker and every client support it; the reason codes alone shorten troubleshooting.

MQTT compared with Modbus and OPC UA

MQTT does a different job from the protocols that talk to controllers, and it is usually deployed alongside them.

MQTTModbusOPC UA (client and server)
PatternPublish and subscribe through a brokerRequest and response, the client pollsSessions with browse, read, write and subscriptions
Who opens the connectionEvery client, outward to the brokerThe client, inward to each deviceThe client, inward to each server
Data modelNone: topic string and opaque bytesNumbered coils and 16-bit registersTyped nodes in a browsable address space
Update behaviourThe publisher sends when it choosesOnly when polledPolled, or pushed to a subscription
Typical placeDevice or gateway to platformField devices to PLC or gatewayPLC or SCADA to supervisory systems

A PLC with a Modbus register map will not publish anything by itself, so the usual arrangement is a gateway that polls it and publishes the result. The Modbus guide and the OPC UA guide cover those protocols.

Using MQTT with ControlCom

ControlCom Connect accepts data from your own firmware or application over MQTT. You create a Generic device in the platform, save the certificate bundle it offers, and note the connection details: endpoint, client ID and topic. The client connects on port 8883 with TLS, authenticates with the device certificate and private key, and publishes JSON to the device’s topic. The payload is a dataArray of readings, each with a key, a value (number, boolean or string) and an optional timestamp in Unix milliseconds; when the timestamp is left out, the platform uses the time it received the message.

The MQTT documentation  shows both accepted payload shapes with a working client, and the platform overview shows what the values feed once they arrive.

Any MQTT client that supports TLS client certificates can publish readings to a Generic device in ControlCom Connect, where each key becomes a variable.