The BoilerIO Software Thermostat

It’s time to step up from command-line control of the heating system I’ve been working on to having a weekly schedule and temporary override function available through a UI, making the system practical on a day-to-day basis.

As always, code described here is available on github under an MIT license.

Overview

The scheduler chooses a target heating temperature throughout the day and needs to be usable by someone non-technical to show the state of the system, provide a means to enter and edit a schedule, and provide a means to change the target temperature for a fixed period of time.

system-overview

To achieve this a new daemon and web service have been added to the architecture.  Here is a summary of the responsibilities of those components:

  • The Scheduler reads the weekly schedule from the database, and sends commands to the Boiler Controller to change the target temperature at the appropriate times.
  • The SchedulerWeb REST service provides a REST API over HTTP to get a summary of system status, set or clear the temperature target override, and add and remove entries from the weekly schedule.  This is implemented using Flask in Python, with uWSGI and nginx to provide a robust service.
  • The scheduler configuration database is a PostegreSQL database holding the schedule, a cached copy of the current temperature, and the target override configuration.  It’s the primary store for this information: the REST service updates it and the scheduler uses it to control the temperature.
  • The SchedulerWeb web frontend is an HTML5/CSS3/Javascript single-page web UI that makes use of the REST service to interface with the system from the user’s web browser.

Experience and implementation notes

Rather than trying to document the entire implementation, I will instead talk about each area and some of the challenges or points of interest, and the major decisions that were taken and why.  My day job is a lot lower in the software stack than this, so there may better approaches than what I’m presenting.

The HTML frontend

alignment
1px alignment issue (red line illustrates the problem) without left padding on input boxes

The app is designed to largely follow Google’s Material Design guidelines; I hadn’t realised when I started that there are stylesheets provided by Google that can be used so I implemented the styles I needed myself.

Since the fashionable choice of web-framework seems to change frequently I was initially resistant to using anything at all fancy, and instead use traditional jQuery directly.  In retrospect, not having some of the features of a slightly higher-level framework probably made the code worse that it ought to be and in future I might look at whether a framework provides a better frontend implementation.

Modern web development has definitely come on a long way since I last attempted it: the developer tools inside Chrome are great, though there are still cross-browser issues even when only supporting latest-generation browsers (for example implementing modal dialogs is still done manually even though there’s the dialog tag now, and I had issues with alignment and appearance of certain input fields).

The REST service

The backend service uses Flask, a neat framework for web apps in Python.

When developing your service you should be aware that werkzeug, the built-in web server, is not suitable for production due to security and scalability issues.  However, if you do use it during test you’ll find it also makes it easy to accidentally keep global state within your app, which you shouldn’t be doing because it won’t work when you’re inside a production server.  For that reason, I suggest starting to use uWSGI relatively early in your development.  It’s not difficult to use for test.

I’ll come onto a step-by-step deployment guide in an upcoming post, but I recommend that the web service be deployed using uWSGI behind nginx or Apache to get a secure and scalable deployment.  However, on Raspbian at least there are a couple of pitfalls I found with uWSGI: I used pip to install a relatively modern version.  The installation used a PREFIX of /usr/local.  For some reason, even with /usr/local/bin on my PATH, uwsgi did not work correctly unless called with its full path (despite printing a message stating the full path it had detected, which was correct).  Perhaps this is a security measure, but the failure mode here and on other issues I experienced was somewhat opaque and better error messages would have been useful.

To use uwsgi in production, it is helpful to have it start to host relevant services on system boot.  On Raspbian, this requires a systemd unit to be created (on other systems an init script, upstart job, or just adding to rc.local would be needed).  That nothing was already in place could be a result of the way I installed uwsgi, but in any case I followed the Debian package’s convention of creating a configuration file in /etc/uwsgi/apps-available and linking to it in /etc/uwsgi/apps-enabled.  The config I used was this:

[uwsgi]
chdir = /var/www/app/boilerio
socket = /var/www/app/thermostat.sock
module = schedulerweb:app
logto = /var/log/uwsgi/thermostat.log
uid = boilerio
gid = www-data
chmod-socket = 664

Note here that, for better isolation, I’m using a system user specifically for this service, and sharing a group with the web-server so I can create the socket with permisions for both to be able to access.

nginx and the URL namespace

The web service natively places its REST endpoints at root level.  As I use nginx to also serve static content – the client HTML/JS/CSS files – I decided to map the service under /api.  Files for the web client get served from /, and they make API calls assuming the api prefix .  I use a simple nginx configuration file to achieve this:

server {
    listen      80;
    server_name hub;
    charset     utf-8;
    location / {
        root /var/www/app/boilerio/static;
    }
    location /api {
        try_files $uri @boilerio;
    }
    location @boilerio {
        include uwsgi_params;
        rewrite ^/api/(.*)$ /$1 break;
        uwsgi_pass unix:/var/www/app/thermostat.sock;
    }
}

The endpoints and the data they expect are currently hand-coded in the Python web application code, which is less than ideal.  Defining a clear API where constraints on input and validation can be consistently and mechanistically verified is a better approach and an area for improvement.  Swagger seems like one good option to implement this, has integration options with Flask, and has the side benefit of a nice web UI for making REST calls to the service too.

The scheduler

After modifying the maintaintemp script to listen for new target temperatures over MQTT rather than having a static target passed on the command-line, the scheduler is able to periodically update the target temperature.  The current, simple, implementation polls the database once per minute, or when a trigger message is received, to load the currently-active schedule and target override.  It then selects a target based on these inputs and sends a message to the boiler controller to update the target temperature.  At startup, and when the controller for a zone restarts, the target request is sent immediately to avoid having to wait a whole polling interval.

This is a likely area for innovation in future: enhancing how the target is chosen using additional inputs or policies, enabling features like pre-heating to reach an upcoming set-point, altering the target based on presence information, and intelligently dealing with installations with multiple zones or independent controls within a zone.

The database

A PostgreSQL database is used to store the schedule and configuration.  This might seem like overkill, but when developing a web app where multiple processes need read/write access to the data it seems sensible to use a tool that is designed for that kind of environment, even if the scale it is being deployed at is relatively tiny, for a few reasons:

  • It avoids designing scalability out now.  If we used another approach that was “simpler” but couldn’t be scaled if necessary it would be a potentially large undertaking to fix.
  • You get a lot of correctness for free.  If, for example, the schedule were stored in a plain text file (say, as JSON), then it is definitely possible to make everything atomic.  But the hassle of getting it exactly right does not seem worthwhile when the database can deal with it all more efficiently from the beginning.

Another approach would be to use a document store like MongoDB.  There are pros and cons to either way (this strongly-worded post is an interesting read concerning problems faced in a practical application), but I decided to go with something I was familiar and confident with.  While having a fixed schema seems to be considered by some to be “overhead” or to slow down development, it can also reduce problems by helping to identify programming or design errors earlier in the cycle and certainly did not seem to make development difficult.

What about AWS/PaaS/Serverless?

A somewhat different approach could be to use Amazon or Azure web services.  Amazon have several relevant offerings here (and I believe Microsoft have alternatives too):

  • AWS IoT: This is a service that maintains a ‘shadow device state’.  User agents can post new ‘desired state’ and devices can post the actual state of the device.  AWS publishes messages to indicate various conditions including when the target and current state are divergent so that the device can change its physical state to match the desired state.
  • AWS Lamda and API gateway: This provides a potentially simple way to implement the scheduler REST API without having to host the web service component yourself, potentially reducing the maintenance burden.  You can easily provide authenticated access regardless of where you are connecting from.  Zappa is a tool that lets you easily run a Flask applications within Lambda, so could be used to allow the BoilerIO code base to be used without modification.
  • S3 could host the static files, such as the CSS, JS, HTML, etc. for the client app.
  • AWS DynamoDB or Relational DB services.  The latter could be a drop-in replacement as it has PostgreSQL support, whereas changes to the app (relatively minor/contained within one module) would be required for DynamoDB although that option does have more attractive pricing

The first issue to consider with this approach is what the connectivity requirements are and what is the user’s expectation of behaviour when their Internet connection is unavailable.  In this case, the minimum requirements seem to be that (i) the schedule should continue to run without interruption regardless of Internet downtime, and (ii) the user should be able to supply at least an override even if the schedule is not editable.  Both are doable, the first trivially since the scheduler can (and should) be run locally to the installation.

The second issue is vendor lock-in.  These services are proprietary, and there’s no way to run local versions of them either for testing or deployments where using an online service is suitable.

In the end I decided to stick with a regular web service for now, which leaves the option open for either hosting it off-site, in a remote VM for example, or having a connector module that enables AWS IoT or similar to provide off-network access without hosting a public-facing service locally.

Next steps

This blog post covers part of one of the “next steps” I identified in the previous post.  Upcoming areas for further work are better documentation including a setup guide, and looking at additional features such as multiple zone support and pre-heating.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s