Louis-Philippe Véronneau - icinga2https://veronneau.org/2017-05-12T00:00:00-04:00icinga2 notifications via Signal messenger2017-05-12T00:00:00-04:002017-05-12T00:00:00-04:00Louis-Philippe Véronneautag:veronneau.org,2017-05-12:/icinga2-notifications-via-signal-messenger.html<p>I recently finished implementing <code>icinga2</code> to monitor our servers and services.
It is a tad complicated but overall I like it very much.</p>
<p>Right around the same time, I also started using Signal on my Android device.
I had never used Signal before since it used to require you to …</p><p>I recently finished implementing <code>icinga2</code> to monitor our servers and services.
It is a tad complicated but overall I like it very much.</p>
<p>Right around the same time, I also started using Signal on my Android device.
I had never used Signal before since it used to require you to have Google Play
services, but <a href="https://signal.org/android/apk/">it is not the case anymore</a>.</p>
<p>The idea of using Signal as a way to alert me of critical failures on my systems
thus came to me. Most serious organisations use some sort of SMS gateway to
alert employees of failures and these services are often expensive. I also do
not have a "real" phone with a SIM card and only use my VoIP account to receive
SMS when I have WiFi with the neat <a href="https://f-droid.org/wiki/page/net.kourlas.voipms_sms">voip.ms sms app</a>.</p>
<p>Contrarywise to an SMS gateway on a server, Signal is fairly easy to use, it is
free and messages are end-to-end encrypted between devices. Configuration of a
client is simple enough with the wonderful <a href="https://github.com/AsamK/signal-cli">signal-cli</a> project.</p>
<h2>Configuring signal-cli</h2>
<p>First of all, since Signal uses real phone numbers as usernames, you will need
to get a phone number with some SMS support. You could also try one without SMS
support and try to get confirmation codes via Signal's calling feature, but it's
a hassle. Instead, I recommended using a good VoIP provider like
<a href="https://voip.ms">voip.ms</a>.</p>
<p>Once you have a phone number, download <code>signal-cli</code> on your server and simlink
it to <code>/usr/local/bin</code>.</p>
<p>You will first need to register your number on Signal with the unix user you
use for <code>icinga2</code>. Don't make the mistake I did of doing everything with your
regular user and then wondering why it does not work:</p>
<p><code>$ sudo -u nagios signal-cli -u ICINGANUMBER register</code></p>
<p>Once that ran, check your SMS (or voicemail) and use the confirmation code this
way:</p>
<p><code>$ sudo -u nagios signal-cli -u ICINGANUMBER verify YYY-ZZZ</code></p>
<p><code>signal-cli</code> should now work. You can test this by sending a test message to
your personnal number:</p>
<p><code>$ echo "test message - icinga2" | sudo -u nagios signal-cli -u ICINGANUMBER send MYNUMBER</code></p>
<h3>Important notes</h3>
<p>You will not be able to register a same phone number on multiple machines. Doing
so will likely bork all other instances of <code>signal-cli</code> where you used that
number. If you ever run into troubles with this (I know I did), you can always
<a href="https://signal.org/signal/unregister/">unregister your number</a> (give it a few minutes after completing the
form) and try again.</p>
<p>Also note that <code>signal-cli</code> can be a little slow compared to the mobile client
and can take a few minutes to run a command.</p>
<h2>Making icinga2 work with signal-cli</h2>
<p>Now that you have <code>signal-cli</code> working, we need to make <code>icinga2</code> use it to send
notifications.</p>
<p>Notifications in <code>icinga2</code> are sent using scripts. By default, there are two
scripts in <code>/etc/icinga2/scripts</code> that are used to send email notifications.</p>
<p>You will need to add two files to the <code>/scripts</code> directory on the master. First
add <code>signal-host-notification.sh</code>, the script we will use to alert you of a
problem with a host:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/sh</span>
<span class="nv">template</span><span class="o">=</span><span class="sb">`</span>cat<span class="w"> </span><span class="s"><<TEMPLATE</span>
<span class="s">$NOTIFICATIONTYPE - $HOSTDISPLAYNAME is $HOSTSTATE</span>
<span class="s">Notification Type: $NOTIFICATIONTYPE</span>
<span class="s">Host: $HOSTALIAS</span>
<span class="s">Address: $HOSTADDRESS</span>
<span class="s">State: $HOSTSTATE</span>
<span class="s">Date/Time: $LONGDATETIME</span>
<span class="s">Additional Info: $HOSTOUTPUT</span>
<span class="s">Comment: [$NOTIFICATIONAUTHORNAME] $NOTIFICATIONCOMMENT</span>
<span class="s">TEMPLATE</span>
<span class="sb">`</span>
/usr/bin/printf<span class="w"> </span><span class="s2">"%b"</span><span class="w"> </span><span class="s2">"</span><span class="nv">$template</span><span class="s2">"</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>signal-cli<span class="w"> </span>-u<span class="w"> </span><span class="nv">$SIGNALNUMBER</span><span class="w"> </span>send<span class="w"> </span><span class="nv">$USERPHONE</span>
</code></pre></div>
<p>Then add <code>signal-service-notification.sh</code>, the script used to alert you in case
a service goes wrong:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/sh</span>
<span class="nv">template</span><span class="o">=</span><span class="sb">`</span>cat<span class="w"> </span><span class="s"><<TEMPLATE</span>
<span class="s">$NOTIFICATIONTYPE - $HOSTDISPLAYNAME - $SERVICEDISPLAYNAME is $SERVICESTATE</span>
<span class="s">Notification Type: $NOTIFICATIONTYPE</span>
<span class="s">Service: $SERVICEDESC</span>
<span class="s">Host: $HOSTALIAS</span>
<span class="s">Address: $HOSTADDRESS</span>
<span class="s">State: $SERVICESTATE</span>
<span class="s">Date/Time: $LONGDATETIME</span>
<span class="s">Additional Info: $SERVICEOUTPUT</span>
<span class="s">Comment: [$NOTIFICATIONAUTHORNAME] $NOTIFICATIONCOMMENT</span>
<span class="s">TEMPLATE</span>
<span class="sb">`</span>
/usr/bin/printf<span class="w"> </span><span class="s2">"%b"</span><span class="w"> </span><span class="s2">"</span><span class="nv">$template</span><span class="s2">"</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>signal-cli<span class="w"> </span>-u<span class="w"> </span><span class="nv">$SIGNALNUMBER</span><span class="w"> </span>send<span class="w"> </span><span class="nv">$USERPHONE</span>
</code></pre></div>
<p>Once that is done, you'll need to add some <code>NotificationCommand</code> objects to your
<code>commands.conf</code> file. Normally, that file is in a global zone:</p>
<pre>
object NotificationCommand "signal-host-notification" {
command = [ SysconfDir + "/icinga2/scripts/signal-host-notification.sh" ]
env = {
NOTIFICATIONTYPE = "$notification.type$"
HOSTALIAS = "$host.display_name$"
HOSTADDRESS = "$address$"
HOSTSTATE = "$host.state$"
LONGDATETIME = "$icinga.long_date_time$"
HOSTOUTPUT = "$host.output$"
NOTIFICATIONAUTHORNAME = "$notification.author$"
NOTIFICATIONCOMMENT = "$notification.comment$"
HOSTDISPLAYNAME = "$host.display_name$"
SIGNALNUMBER = "ICINGANUMBER"
USERPHONE = "MYNUMBER"
}
timeout = 10m
}
object NotificationCommand "signal-service-notification" {
command = [ SysconfDir + "/icinga2/scripts/signal-service-notification.sh" ]
env = {
NOTIFICATIONTYPE = "$notification.type$"
SERVICEDESC = "$service.name$"
HOSTALIAS = "$host.display_name$"
HOSTADDRESS = "$address$"
SERVICESTATE = "$service.state$"
LONGDATETIME = "$icinga.long_date_time$"
SERVICEOUTPUT = "$service.output$"
NOTIFICATIONAUTHORNAME = "$notification.author$"
NOTIFICATIONCOMMENT = "$notification.comment$"
HOSTDISPLAYNAME = "$host.display_name$"
SERVICEDISPLAYNAME = "$service.display_name$"
SIGNALNUMBER = "ICINGANUMBER"
USERPHONE = "MYNUMBER"
}
timeout = 10m
}
</pre>
<p>Note that for multiple users systems, you can set the <code>USERPHONE</code> variable
dynamically based on schedules.</p>
<p>Finally, you need to apply the <code>NotificationObject</code> to certain hosts. Add this
snippet to your <code>notifications.conf</code> file, normally also in your global zone:</p>
<pre>
apply Notification "signal-icingaadmin" to Host {
import "signal-host-notification"
user_groups = host.vars.notification.signal.groups
users = host.vars.notification.signal.users
assign where host.vars.notification.signal
if (host.vars.notification_interval) {
interval = host.vars.notification_interval
}
}
apply Notification "signal-icingaadmin" to Service {
import "signal-service-notification"
user_groups = host.vars.notification.signal.groups
users = host.vars.notification.signal.users
assign where host.vars.notification.signal
if (host.vars.notification_interval) {
interval = host.vars.notification_interval
}
}
</pre>
<p>On a host, you can now use signal by adding this to your <code>hosts.conf</code> file:</p>
<pre>
vars.notification["signal"] = {
/* The UserGroup `icingaadmins` is defined in `users.conf`. */
groups = [ "icingaadmins" ]
}
</pre>