IoT Hub services
IoT Hub is designed to be:
- scalable: horizontally scalable platform, build using leading open-source technologies.
- fault-tolerant: no single-point-of-failure, every node in the cluster is identical.
- robust and efficient: single server node can handle tens or even hundreds thousands of devices depending on use-case. IoT Hub cluster can handle millions of devices.
- durable: never lose your data. IoT Hub supports various queue implementations to provide extremely high message durability.
- customizable: adding new functionality is easy with customizable widgets and rule engine nodes.
The diagram below shows key system components and interfaces they provide. Let’s walk through them.
IoT Hub Transports
IoT Hub provides MQTT, HTTP, CoAP and LwM2M based APIs that are available for your device applications/firmware. Each of the protocol APIs are provided by a separate server component and is part of IoT Hub “Transport Layer”. MQTT Transport also provides Gateway APIs to be used by gateways that represent multiple connected devices and/or sensors.
Once the Transport receives the message from device, it is parsed and pushed to durable Message Queue. The message delivery is acknowledged to device only after corresponding message is acknowledged by the message queue.
IoT Hub Core
IoT Hub Core is responsible for handling REST API calls and WebSocket subscriptions. It is also responsible for storing up to date information about active device sessions and monitoring device connectivity state. IoT Hub Core uses Actor System under the hood to implement actors for main entities: tenants and devices. Platform nodes can join the cluster, where each node is responsible for certain partitions of the incoming messages.
IoT Hub Rule Engine
IoT Hub Rule Engine is the heart of the system and is responsible for processing incoming messages. Rule Engine uses Actor System under the hood to implement actors for main entities: rule chains and rule nodes. Rule Engine nodes can join the cluster, where each node is responsible for certain partitions of the incoming messages.
Rule Engine subscribes to incoming data feed from queue(s) and acknowledge the message only once it is processed. There are multiple strategies available that control the order or message processing and the criteria of message acknowledgement. See submit strategies and processing strategies for more details.
IoT Hub Rule Engine may operate in two modes: shared and isolated. In shared mode, rule engine process messages that belong to multiple tenants. In isolated mode Rule Engine may be configured to process messages for specific tenant only.
IoT Hub Web UI
IoT Hub provides a lightweight component written using Express.js framework to host static web ui content. Those components are completely stateless and no much configuration available. The static web UI contains application bundle. Once it is loaded, the application starts using the REST API and WebSockets API provided by IoT Hub Core.
Message Queues are awesome!
IoT Hub supports multiple message queue implementations: Kafka, RabbitMQ, AWS SQS, Azure Service Bus and Google Pub/Sub. We plan to extend this list in the future.
Using durable and scalable queues allow IoT Hub to implement back-pressure and load balancing. Back-pressure is extremely important in case of peak loads.
We provide “abstraction layer” over specific queue implementations and maintain two main concepts: topic and topic partition.
One topic may have configurable number of partitions. Since most of the queue implementations does not support partitions, we use topic + “.” + partition pattern.
IoT Hub message Producers determines which partition to use based on the hash of entity id. Thus, all messages for the same entity are always pushed to the same partition. IoT Hub message Consumers coordinate using Zookeeper and use consistent-hash algorithm to determine list of partitions that each Consumer should subscribe to. While running in microservices mode, each service also has the dedicated “Notifications” topic based on the unique service id that has only one partition.
IoT Hub uses following topics:
- tb_transport.api.requests: to send generic API calls to check device credentials from Transport to IoT Hub Core.
- tb_transport.api.responses: to receive device credentials verification results from IoT Hub Core to Transport.
- tb_core: to push messages from Transport or Rule Engine to IoT Hub Core. Messages include session lifecycle events, attribute and RPC subscriptions, etc.
- tb_rule_engine: to push messages from Transport or IoT Hub Core to Rule Engine. Messages include incoming telemetry, device states, entity lifecycle events, etc.
Programming languages and third-party
IoT Hub back-end is written in Java, but we also have some micro-services based on Node.js. IoT Hub front-end is a SPA based on Angular 9 framework.