Here at SysAid, to help us track our clients’ computing resources (computers, printers etc.), we have software agents running on client sites. These periodically report to our servers on the resources’ status. Reports are relayed to our servers using encrypted HTTP requests.

Occasionally we have to send messages from our servers to the agents instructing them to take specific actions, such as configuration changes. There are several options to implement the server to agent communication. The simplest would’ve been for our agents to listen to traffic from our servers, but for security reasons we couldn’t ask our clients to open their firewall to incoming traffic, so instead it was decided to have the agents poll our servers instead. That generates a lot of traffic. Just to give an example, if our agents poll once a minute and we have a thousand clients with a hundred agents each, that’s over four billion monthly requests. Worse, since in most cases we don’t actually have a message to relay to our agents, most of this traffic is redundant.

To reduce the traffic to our servers, @SysAid decided to get rid of polling. Could you benefit from the same? Check out this blog. #AWS #tech Click To Tweet

To reduce the traffic to our servers, we decided to get rid of polling. There are several options to do this, mostly involving a publish/subscribe model. We wanted a solution which would:

• Support tens of thousands of devices. For example, if we have a thousand clients with a hundred devices each that’s 100,000 devices.
• Encrypt its traffic.
• Be able to recover from disconnects.
• Be able to send a message to a specific subscriber.
• Be able to authenticate and authorize. We don’t want just anyone receiving our messages, and agents should be able to receive only the messages intended specifically for them.
• Be fully managed – we didn’t want to handle the day to day maintenance of the infrastructure.

So we decided to investigate WebSockets, message brokers (e.g. RabbitMQ) and a MQTT message broker.

WebSockets or an unmanaged RabbitMQ went immediately out of the window because those would have significant development and maintenance costs associated. We wanted something which would require less effort on our part.

Fortunately AWS, our cloud provider, has several managed options which we could consider. There’s ActiveMQ, which would have a hard time supporting the amount of connections we needed. SNS, which is used for fan out messaging, whereas we needed to send each message to a specific agent. SQS, which would require us creating a queue per agent, which is ugly. And finally IoT, which implements the MQTT protocol for message passing.

IoT had everything that we need. It can support millions of devices, it’s encrypted, it supports authentication and authorization, it can deliver messages even after disconnects, it can deliver messages to a specific subscriber due to its topic feature and is a serverless solution which would require very little administration on our part.

Basic architecture for the solution

Now when our agent starts it establishes a connection with AWS’ IoT broker and our server, and after also connecting to the broker, it can relay messages to the agent. No more polling required. The topic feature of MQTT allows you to create a topic tree which enables our servers to send messages to specific agents. In our case we implemented a scheme where the topic tree was accountId/deviceId/messageSubject. So if we wanted to send a message to the device computerA in account accountB with a configuration change we would send a message to topic accountB/computerA/configurationChanges. The agent on computerA would listen to incoming messages by subscribing to accountB/computerA/*.

AWS IoT can support millions of devices, it's encrypted, it supports authentication & authorization, & it's a serverless solution requiring very little administration. See how @SysAid uses it to reduce traffic to its servers in this blog.… Click To Tweet

The wildcard is very useful as a catchall to listen to multiple topics. If we wanted to we could have a device listen to all an account’s messages by subscribing to accountB/+And how is all this secured you ask? The security model is simple. Every device connected to AWS IoT has a representation on the AWS side called a Thing. Each Thing must be connected to a certificate. The certificate is connected to a policy. To authenticate, the device uses the certificate, which can be easily revoked on the AWS side in case something suspicious happens. The policy sets the authorizations granted to the device. We can, for example, limit the device to specific actions (connect, publish, subscribe) on specific topics. When our agent comes up it requests credentials from our server. Our server creates a Thing and certificate and connects it to a pre-existing policy which only allows the device to listen on its own topics.

Security model

Here’s an example policy allowing a device to listen only to its own topics.

Notice how this uses a policy variable (credentials-iot:ThingName) to simplify authorization. Instead of having to generate a policy for each Thing we can have a single policy which takes the Thing name as a variable and restricts that Thing to its own topics.

In the next post I’ll examine how we implemented an even more stringent security requirement. We didn’t want our clients to have to open their firewall to outgoing traffic to all of AWS’ IP range, which would have been necessary if we wanted their software agents to connect to the IoT broker. So instead we used a nifty solution offered by IoT to bypass the firewall. Come back for the next post to find out more about it.

Does all this sound interesting? Great! I’m hiring a software architect. If you want to talk here’s a link to the job description: https://www.sysaid.com/company/careers/positions/co/rd/F4.22A/software-architect/all