Louis-Philippe Véronneau - monitoringhttps://veronneau.org/2023-08-04T00:00:00-04:00pymonitair: Air Quality Monitoring Display with MicroPython2023-08-04T00:00:00-04:002023-08-04T00:00:00-04:00Louis-Philippe Véronneautag:veronneau.org,2023-08-04:/pymonitair-air-quality-monitoring-display-with-micropython.html<p>I've never been a fan of IoT devices for obvious reasons: not only do they tend
to be excellent at being expensive vendor locked-in machines, but far too often,
they also end up turning into e-waste after a short amount of time.
Manufacturers can go out of business or simply …</p><p>I've never been a fan of IoT devices for obvious reasons: not only do they tend
to be excellent at being expensive vendor locked-in machines, but far too often,
they also end up turning into e-waste after a short amount of time.
Manufacturers can go out of business or simply decide to <a href="https://www.wired.co.uk/article/sonos-outrage-legacy-speakers">shut down the cloud
servers</a> for older models, and then you're stuck with a brick.</p>
<p>Well, this all changes today, as I've built my first IoT device and I love it.
Introducing pymonitair.</p>
<h1>What</h1>
<p>pymonitair is a MicroPython project that aims to display weather data from a
home weather station (like the ones sold by <a href="https://www.airgradient.com/">AirGradient</a>) on a small display.</p>
<p>The <a href="https://gitlab.com/baldurmen/pymonitair">source code</a> was written for the Raspberry Pi Pico W, the
<a href="https://www.waveshare.com/wiki/Pico-OLED-1.3">Waveshare Pico OLED 1.3</a> display and the <a href="https://revolvair.org/revo1-station-danalyse-de-particules-fines-boutique/">RevolvAir Revo 1</a>
weather station, but can be adapted to other displays and stations easily, as I
tried to keep the code as modular as possible.</p>
<p>The general MicroPython code itself isn't specific to the Raspberry Pi Pico and
shouldn't need to be modified for other boards.</p>
<p>pymonitair features:</p>
<ul>
<li>6 different pages for the supported weather data<sup id="fnref:data"><a class="footnote-ref" href="#fn:data">1</a></sup>, accessible via the
<code>key0</code> button</li>
<li>Alerting<sup id="fnref:alert"><a class="footnote-ref" href="#fn:alert">2</a></sup> on the PM2.5 page when the defined threshold has been
crossed</li>
<li>Automatic screen blanking after a defined amount of time to save the OLED
screen from burn-in</li>
<li>Manual screen blanking using the <code>key1</code> button</li>
</ul>
<p>Here's a demo of me scrolling through the different pages and (somewhat
failing) to turn the screen on and off:</p>
<video src="https://veronneau.org/media/blog/2023-08-04/pymonitair.webm" title="pymonitair demo, where I'm scrolling through the different pages" alt="pymonitair demo, where I'm scrolling through the different pages" height="80%" width="80%" style="margin-left: 10%;" controls></video>
<h1>Why?</h1>
<p>If you follow my blog, you'll know that <a href="https://veronneau.org/weather-station-data-visualisations-using-r-and-python.html">my last entry</a> was
about building a set of tools to collect and graph data from a weather station
my neighbor set up.</p>
<p>Why on Earth would I need a separate device to show this data, when the
<a href="https://air.veronneau.org">website I've built</a> works perfectly fine and is accessible on
any computer or smartphone?</p>
<p>Mostly alerts. When the air quality here dropped following <a href="https://www.cbc.ca/news/canada/montreal/forest-fires-quebec-sept-%C3%AEles-1.6865576">forest
fires</a>, I found out keeping track of if I had to close my windows and
bunker down was quite a hassle.</p>
<p>Air quality would degrade during the day and I would only notice it hours
later. With the pymonitair, I'll have a little screen flashing angrily at me
whenever this happens.</p>
<p>A simpler solution would probably have been to forgo hardware altogether and
code some icinga2 alert to <a href="https://veronneau.org/icinga2-notifications-via-signal-messenger.html">ping me over Signal</a> whenever the
air quality got bad. Hacking on pymonitair was mostly a way to learn to use
MicroPython and familiarize myself with this type of embedded hardware device.</p>
<p>I'll surely blog about this later this year, but I plan to use a very similar
stack to mod my apartment's HVAC unit to stop pulling air from outside when an
air quality sensor detects cigarette smoke (or bad air quality in general).</p>
<h1>Things I've learnt</h1>
<p>This project was super fun and taught me many things:</p>
<ul>
<li>
<p>MicroPython is close enough to regular Python that I don't mind it. The
standard library is much smaller, but there are ways to work around that.</p>
</li>
<li>
<p>The Raspberry Pi Pico W is a fantastic board. At around 10 CAD (~7 USD), it's
much cheaper than anything Arduino offers, much more powerful than those
boards and gives you the option of not having to code in some terrible and
weird C variant. It also has a nice ecosystem of compatible modules and hats.</p>
</li>
<li>
<p>Displays, even if they advertise being "batteries included", often are not. I
had the bad surprise of discovering the MicroPython driver for the Waveshare
Pico OLED 1.3 was pretty bad and lacked a ton of features compared to their
C code driver. Thankfully, I was lucky enough that the OLED panel it uses (the
SH1107) is common enough for folks to have written their own drivers.</p>
</li>
<li>
<p><a href="https://packages.debian.org/bookworm/thonny">thonny</a>, a Python IDE, is probably the best tool to transfer files to the
Raspberry Pico and to debug code, as it provides a file manager and a
shell<sup id="fnref:other"><a class="footnote-ref" href="#fn:other">3</a></sup>. I'm a little annoyed by it not being vim, but I got used to
its quirks fairly quickly.</p>
</li>
</ul>
<div class="footnote">
<hr>
<ol>
<li id="fn:data">
<p>PM1, PM2.5, PM10, Temperature, Humidity and Pressure <a class="footnote-backref" href="#fnref:data" title="Jump back to footnote 1 in the text">↩</a></p>
</li>
<li id="fn:alert">
<p>Part of the screen will flash repeatedly <a class="footnote-backref" href="#fnref:alert" title="Jump back to footnote 2 in the text">↩</a></p>
</li>
<li id="fn:other">
<p>I did look for other solutions to transfer files to the board, but
none of them were actually maintained. I nearly finished packaging <a href="https://github.com/scientifichackers/ampy">ampy</a>
before realising it was officially unmaintained and its main alternative,
<a href="https://github.com/dhylands/rshell">rshell</a>, has had its last release in December 2021. When I caught myself
seriously considering writing a script to transfer files over the serial
link, I gave up and decided thonny was not that bad after all. <a class="footnote-backref" href="#fnref:other" title="Jump back to footnote 3 in the text">↩</a></p>
</li>
</ol>
</div>Weather Station Data Visualisations Using R and Python2023-08-01T00:00:00-04:002023-08-01T00:00:00-04:00Louis-Philippe Véronneautag:veronneau.org,2023-08-01:/weather-station-data-visualisations-using-r-and-python.html<p>A few weeks ago, my friend and neighbor Jérôme (<a href="https://qa.debian.org/developer.php?login=lavamind&comaint=yes">aka lavamind</a>)
installed a weather station on his balcony and started collecting data from it.</p>
<p>It has been quite useful to measure the degrading air quality during the recent
<a href="https://www.cbc.ca/news/canada/montreal/forest-fires-quebec-sept-%C3%AEles-1.6865576">forest fires</a> plaguing northern Canada, but sadly, the hardware itself
isn't great …</p><p>A few weeks ago, my friend and neighbor Jérôme (<a href="https://qa.debian.org/developer.php?login=lavamind&comaint=yes">aka lavamind</a>)
installed a weather station on his balcony and started collecting data from it.</p>
<p>It has been quite useful to measure the degrading air quality during the recent
<a href="https://www.cbc.ca/news/canada/montreal/forest-fires-quebec-sept-%C3%AEles-1.6865576">forest fires</a> plaguing northern Canada, but sadly, the hardware itself
isn't great.</p>
<p>Whereas some projects like <a href="https://www.airgradient.com/">airgradient</a> offer open hardware devices running
free software, the station we got is from <a href="https://revolvair.org/">RevolvAir</a>, some kind of local
air monitoring project that aims to be a one-stop solution for exterior air
monitoring.</p>
<p>Not only is their device pretty expensive<sup id="fnref:expensive"><a class="footnote-ref" href="#fn:expensive">1</a></sup>, but it also reboots
frequently by itself. Even worse, their <a href="https://revolvair.org/carte/">online data map</a> requires an
account to view the data and the interface is bad, unintuitive and only stores
data up to a month.</p>
<p>Having a good background in data visualisation and statistics thanks to my
master's degree in economics, I decided I could do better. Two days later, I
had built a series of tools to collect, analyse and graph the JSON time series
data provided by the device.</p>
<p>The result is a <a href="https://air.veronneau.org/">very simple website</a> that works without any JavaScript,
leveraging static graphs built using R. Modern web libraries and projects
offer an incredible wealth of tools to graph and visualise data, but as for
most of my web projects, I wanted something static and simple.</p>
<p>The source code for the project can be <a href="https://gitlab.com/baldurmen/r-air/">found here</a>, and although it is
somewhat specific to the data structure provided by the RevolvAir device, it
could easily be adapted to other devices, as they tend to have very similar
JSON dumps.</p>
<div class="footnote">
<hr>
<ol>
<li id="fn:expensive">
<p>around 300 CAD, whereas a similar station from airgradient costs
around 90 CAD. Thankfully, this station was a gift from a local group
mobilising against <a href="https://territories.substack.com/p/in-the-end-the-forest-will-triumph">an industrial project</a> near our housing
cooperative and we didn't have to pay for it ourselves. <a class="footnote-backref" href="#fnref:expensive" title="Jump back to footnote 1 in the text">↩</a></p>
</li>
</ol>
</div>icinga2 notifications via Signal messenger2017-05-12T00:00:00-04:002017-05-12T00:00:00-04:00Louis-Philippe Véronneautag:veronneau.org,2017-05-12:/icinga2-notifications-via-signal-messenger.html<p>I recently finished implementing <code>icinga2</code> to monitor our servers and services.
It is a tad complicated but overall I like it very much.</p>
<p>Right around the same time, I also started using Signal on my Android device.
I had never used Signal before since it used to require you to …</p><p>I recently finished implementing <code>icinga2</code> to monitor our servers and services.
It is a tad complicated but overall I like it very much.</p>
<p>Right around the same time, I also started using Signal on my Android device.
I had never used Signal before since it used to require you to have Google Play
services, but <a href="https://signal.org/android/apk/">it is not the case anymore</a>.</p>
<p>The idea of using Signal as a way to alert me of critical failures on my systems
thus came to me. Most serious organisations use some sort of SMS gateway to
alert employees of failures and these services are often expensive. I also do
not have a "real" phone with a SIM card and only use my VoIP account to receive
SMS when I have WiFi with the neat <a href="https://f-droid.org/wiki/page/net.kourlas.voipms_sms">voip.ms sms app</a>.</p>
<p>Contrarywise to an SMS gateway on a server, Signal is fairly easy to use, it is
free and messages are end-to-end encrypted between devices. Configuration of a
client is simple enough with the wonderful <a href="https://github.com/AsamK/signal-cli">signal-cli</a> project.</p>
<h2>Configuring signal-cli</h2>
<p>First of all, since Signal uses real phone numbers as usernames, you will need
to get a phone number with some SMS support. You could also try one without SMS
support and try to get confirmation codes via Signal's calling feature, but it's
a hassle. Instead, I recommended using a good VoIP provider like
<a href="https://voip.ms">voip.ms</a>.</p>
<p>Once you have a phone number, download <code>signal-cli</code> on your server and simlink
it to <code>/usr/local/bin</code>.</p>
<p>You will first need to register your number on Signal with the unix user you
use for <code>icinga2</code>. Don't make the mistake I did of doing everything with your
regular user and then wondering why it does not work:</p>
<p><code>$ sudo -u nagios signal-cli -u ICINGANUMBER register</code></p>
<p>Once that ran, check your SMS (or voicemail) and use the confirmation code this
way:</p>
<p><code>$ sudo -u nagios signal-cli -u ICINGANUMBER verify YYY-ZZZ</code></p>
<p><code>signal-cli</code> should now work. You can test this by sending a test message to
your personnal number:</p>
<p><code>$ echo "test message - icinga2" | sudo -u nagios signal-cli -u ICINGANUMBER send MYNUMBER</code></p>
<h3>Important notes</h3>
<p>You will not be able to register a same phone number on multiple machines. Doing
so will likely bork all other instances of <code>signal-cli</code> where you used that
number. If you ever run into troubles with this (I know I did), you can always
<a href="https://signal.org/signal/unregister/">unregister your number</a> (give it a few minutes after completing the
form) and try again.</p>
<p>Also note that <code>signal-cli</code> can be a little slow compared to the mobile client
and can take a few minutes to run a command.</p>
<h2>Making icinga2 work with signal-cli</h2>
<p>Now that you have <code>signal-cli</code> working, we need to make <code>icinga2</code> use it to send
notifications.</p>
<p>Notifications in <code>icinga2</code> are sent using scripts. By default, there are two
scripts in <code>/etc/icinga2/scripts</code> that are used to send email notifications.</p>
<p>You will need to add two files to the <code>/scripts</code> directory on the master. First
add <code>signal-host-notification.sh</code>, the script we will use to alert you of a
problem with a host:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/sh</span>
<span class="nv">template</span><span class="o">=</span><span class="sb">`</span>cat<span class="w"> </span><span class="s"><<TEMPLATE</span>
<span class="s">$NOTIFICATIONTYPE - $HOSTDISPLAYNAME is $HOSTSTATE</span>
<span class="s">Notification Type: $NOTIFICATIONTYPE</span>
<span class="s">Host: $HOSTALIAS</span>
<span class="s">Address: $HOSTADDRESS</span>
<span class="s">State: $HOSTSTATE</span>
<span class="s">Date/Time: $LONGDATETIME</span>
<span class="s">Additional Info: $HOSTOUTPUT</span>
<span class="s">Comment: [$NOTIFICATIONAUTHORNAME] $NOTIFICATIONCOMMENT</span>
<span class="s">TEMPLATE</span>
<span class="sb">`</span>
/usr/bin/printf<span class="w"> </span><span class="s2">"%b"</span><span class="w"> </span><span class="s2">"</span><span class="nv">$template</span><span class="s2">"</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>signal-cli<span class="w"> </span>-u<span class="w"> </span><span class="nv">$SIGNALNUMBER</span><span class="w"> </span>send<span class="w"> </span><span class="nv">$USERPHONE</span>
</code></pre></div>
<p>Then add <code>signal-service-notification.sh</code>, the script used to alert you in case
a service goes wrong:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/sh</span>
<span class="nv">template</span><span class="o">=</span><span class="sb">`</span>cat<span class="w"> </span><span class="s"><<TEMPLATE</span>
<span class="s">$NOTIFICATIONTYPE - $HOSTDISPLAYNAME - $SERVICEDISPLAYNAME is $SERVICESTATE</span>
<span class="s">Notification Type: $NOTIFICATIONTYPE</span>
<span class="s">Service: $SERVICEDESC</span>
<span class="s">Host: $HOSTALIAS</span>
<span class="s">Address: $HOSTADDRESS</span>
<span class="s">State: $SERVICESTATE</span>
<span class="s">Date/Time: $LONGDATETIME</span>
<span class="s">Additional Info: $SERVICEOUTPUT</span>
<span class="s">Comment: [$NOTIFICATIONAUTHORNAME] $NOTIFICATIONCOMMENT</span>
<span class="s">TEMPLATE</span>
<span class="sb">`</span>
/usr/bin/printf<span class="w"> </span><span class="s2">"%b"</span><span class="w"> </span><span class="s2">"</span><span class="nv">$template</span><span class="s2">"</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>signal-cli<span class="w"> </span>-u<span class="w"> </span><span class="nv">$SIGNALNUMBER</span><span class="w"> </span>send<span class="w"> </span><span class="nv">$USERPHONE</span>
</code></pre></div>
<p>Once that is done, you'll need to add some <code>NotificationCommand</code> objects to your
<code>commands.conf</code> file. Normally, that file is in a global zone:</p>
<pre>
object NotificationCommand "signal-host-notification" {
command = [ SysconfDir + "/icinga2/scripts/signal-host-notification.sh" ]
env = {
NOTIFICATIONTYPE = "$notification.type$"
HOSTALIAS = "$host.display_name$"
HOSTADDRESS = "$address$"
HOSTSTATE = "$host.state$"
LONGDATETIME = "$icinga.long_date_time$"
HOSTOUTPUT = "$host.output$"
NOTIFICATIONAUTHORNAME = "$notification.author$"
NOTIFICATIONCOMMENT = "$notification.comment$"
HOSTDISPLAYNAME = "$host.display_name$"
SIGNALNUMBER = "ICINGANUMBER"
USERPHONE = "MYNUMBER"
}
timeout = 10m
}
object NotificationCommand "signal-service-notification" {
command = [ SysconfDir + "/icinga2/scripts/signal-service-notification.sh" ]
env = {
NOTIFICATIONTYPE = "$notification.type$"
SERVICEDESC = "$service.name$"
HOSTALIAS = "$host.display_name$"
HOSTADDRESS = "$address$"
SERVICESTATE = "$service.state$"
LONGDATETIME = "$icinga.long_date_time$"
SERVICEOUTPUT = "$service.output$"
NOTIFICATIONAUTHORNAME = "$notification.author$"
NOTIFICATIONCOMMENT = "$notification.comment$"
HOSTDISPLAYNAME = "$host.display_name$"
SERVICEDISPLAYNAME = "$service.display_name$"
SIGNALNUMBER = "ICINGANUMBER"
USERPHONE = "MYNUMBER"
}
timeout = 10m
}
</pre>
<p>Note that for multiple users systems, you can set the <code>USERPHONE</code> variable
dynamically based on schedules.</p>
<p>Finally, you need to apply the <code>NotificationObject</code> to certain hosts. Add this
snippet to your <code>notifications.conf</code> file, normally also in your global zone:</p>
<pre>
apply Notification "signal-icingaadmin" to Host {
import "signal-host-notification"
user_groups = host.vars.notification.signal.groups
users = host.vars.notification.signal.users
assign where host.vars.notification.signal
if (host.vars.notification_interval) {
interval = host.vars.notification_interval
}
}
apply Notification "signal-icingaadmin" to Service {
import "signal-service-notification"
user_groups = host.vars.notification.signal.groups
users = host.vars.notification.signal.users
assign where host.vars.notification.signal
if (host.vars.notification_interval) {
interval = host.vars.notification_interval
}
}
</pre>
<p>On a host, you can now use signal by adding this to your <code>hosts.conf</code> file:</p>
<pre>
vars.notification["signal"] = {
/* The UserGroup `icingaadmins` is defined in `users.conf`. */
groups = [ "icingaadmins" ]
}
</pre>