MQTT topics and wildcards: syntax, matching rules and retained messages

Catalin NegruPublished 5 min read

Part of the MQTT guide

An MQTT topic is a UTF-8 string, divided into levels by /, that labels every published message. A publisher uses an exact topic name. A subscriber uses a topic filter, which is either an exact name or a pattern containing the wildcards + (one level) and # (all remaining levels). The broker compares each message’s topic name with every filter and delivers the message wherever they match.

This page belongs to the MQTT guide. For the roles and packet layout, see What is MQTT?.

Topic name rules

The rules are few, and most of the surprises come from how literal they are.

  • A topic name or filter is at least one character long and at most 65,535 bytes of UTF-8.
  • Matching is exact and case sensitive. Site1/Pump3 and site1/pump3 are different topics.
  • / separates levels. A level can be empty, so /site1/pump3 has three levels, the first of them an empty string, and it does not match a subscription to site1/pump3.
  • Spaces are allowed and compared like any other character. site1/pump 3 is a valid topic that nobody will type correctly twice.
  • The null character is not allowed.
  • A topic name used in PUBLISH cannot contain + or #. Wildcards exist only in subscriptions.

There is no registry of topics. Publishing to site1/pump3/flwo succeeds, and the message goes to nobody.

The single-level wildcard: +

+ stands for exactly one level, including an empty one. It has to occupy the whole level: site1/+/flow is valid, site1/pump+/flow is not.

The multi-level wildcard:

# stands for any number of levels, including none. It must be the last character of the filter and must follow a /, unless the filter is # alone. Because it matches zero levels too, site1/# also matches the parent topic site1.

Wildcard match table

Topic filterMatchesDoes not match
site1/pump3/flowsite1/pump3/flowsite1/pump3/Flow, /site1/pump3/flow
site1/+/flowsite1/pump3/flow, site1/pump4/flowsite1/flow, site1/pump3/inlet/flow
site1/+site1/pump3, site1/site1, site1/pump3/flow
site1/#site1, site1/pump3, site1/pump3/flowsite2/pump3
+/+/flowsite1/pump3/flow, site2/fan1/flowsite1/pump3/pressure
+site1/site1, site1/pump3
#Every topic that does not start with $$SYS/broker/uptime

Overlapping subscriptions are legal. A client subscribed to both site1/# and site1/+/flow may receive one copy of a matching message or one per subscription, depending on the broker.

A SUBSCRIBE packet decoded

A client subscribes to site1/+/flow with a maximum QoS of 1, using MQTT 3.1.1.

SUBSCRIBE: 82 11 00 01 00 0C 73 69 74 65 31 2F 2B 2F 66 6C 6F 77 01 82 packet type 8 (SUBSCRIBE), flags 0010 (fixed) 11 remaining length: 17 bytes follow 00 01 packet identifier 1 00 0C filter length: 12 bytes 73 ... 77 "site1/+/flow" 01 requested QoS 1
SUBACK: 90 03 00 01 01 90 packet type 9 (SUBACK) 03 remaining length 00 01 packet identifier 1, echoed 01 granted: maximum QoS 1

One SUBSCRIBE can carry several filters, and the SUBACK then carries one result byte per filter in the same order. In MQTT 3.1.1 the value 80 means the broker refused that filter. Check it: a client library that ignores the SUBACK will wait indefinitely for messages the broker has already declined to send. The meaning of the granted QoS is covered on the MQTT QoS page.

Topics that start with $

Topic names beginning with $ are reserved for the broker. A filter that starts with a wildcard, # or +/status, does not match them, so a subscription to # does not flood a client with broker statistics. Many brokers publish their own counters under $SYS/; to read them, subscribe to $SYS/# explicitly. Applications should not publish to $ topics.

Retained messages

A PUBLISH with the RETAIN flag set asks the broker to keep the message as the last known value of that topic. The broker stores one retained message per topic, and each new one replaces the last. When a client subscribes with a filter that matches the topic, the broker sends the stored message at once, with the RETAIN flag set so the client can tell it is not live.

Retained messages suit state that changes rarely: a device’s online or offline status, a configuration value, a setpoint. They are a poor fit for a stream of measurements, where the retained copy is one stale sample.

To delete a retained message, publish a zero-length payload with RETAIN set to the same topic. Forgotten retained messages are the usual reason a decommissioned device still appears on a dashboard.

MQTT 5.0 adds subscription options that control this: Retain Handling chooses whether retained messages are sent on subscribe, and No Local stops a client receiving its own publications.

Shared subscriptions

MQTT 5.0 defines shared subscriptions for spreading load across several consumers. The filter takes the form:

$share/{ShareName}/{TopicFilter} $share/ingest/site1/+/flow

Clients that subscribe with the same share name form a group, and the broker delivers each matching message to one member of the group instead of all of them. Some MQTT 3.1.1 brokers offer the same thing with their own syntax.

Designing a topic tree

Most problems with MQTT topics are naming problems, and they are cheap to avoid at the start.

  1. Go from general to specific: site, area, device, measurement. Wildcards then select useful sets, such as site1/+/+/flow for every flow reading on a site.
  2. Keep identifiers in levels and data in the payload. A serial number belongs in the topic; a temperature does not.
  3. Use lowercase ASCII without spaces, and no leading /.
  4. Keep the number of levels the same for the same kind of message, so + lines up.
  5. Do not use # at the root in production clients. It subscribes to everything every other client publishes, and on a busy broker it is the fastest way to overload a consumer. Use it for diagnostics only.
  6. Decide early whether the topic or the payload carries the measurement name. One topic per value is easy to filter; one topic per device with a structured payload is cheaper on the wire. Both work, and mixing them does not.

For industrial data, the Sparkplug B specification settles these questions with a fixed five-level namespace.

How ControlCom uses topics

In ControlCom Connect each device publishes to one topic, shown with the endpoint and client ID in the device’s connection details. The measurement name travels in the payload: each entry in dataArray has a key, and the key selects the platform variable that receives the value. Commands travel the other way; when a command is sent from the platform, the payload is published to the device’s command topic over MQTT.

On the ControlCom Edge Server, the SparkplugB Listener rule matches inbound topics by their edge node and device segments, and its documentation states that those identifiers must not contain /, +, # or whitespace, for the reasons above. See the MQTT documentation  for the publish topic and payload, and the page for controls engineers for the wider workflow.

One topic per device and one key per reading: publish a JSON payload to the topic ControlCom Connect assigns, and each key becomes a variable you can trend and alarm on.