Free SMS Nagios alerts, via Twitter.

Here at DeepBlueSky we’re a little obsessive about monitoring things. To clarify, we mean our websites, servers, and general infrastructure: we like to provide the best service we possibly can, and to do this we have to measure performance and availability of all our services.

On the rare occasions when something is out of whack, we like to be told straight away so we can deal with whatever issues arise immediately. For this, we have our alerts and monitoring software. When something deviates from what it should be, we’re told. This could be a service being slower than it should be, or a spike in traffic, or any number of other things.

Recently we decided to try sending our alerts over twitter: we’ve almost always got one eye glued to our phone in anticipation of the next DM or mention, so it seemed like the most natural choice. To do this, we’ve stuck with our standard monitoring software, Nagios, and started using a piece of open source software called Twurl.

What is Twurl?

Twurl is a commandline tool like curl which provides a very simple way to access to twitter's API. The basics of setting this up are: create a twitter account for nagios, create a twitter dev account and an app, auth twurl to this app on the commandline, and then go on your merry way sending alerts.

A Nagios alert for using Twurl might look something like this:

# ‘notify-host-by-twurl’ command definition
define command {

command_name notify-host-by-twurl
command_line /usr/local/bin/twurl -d "status=d $CONTACTNAME$ $NOTIFICATIONTYPE$ : $TIME$ : $HOSTALIAS$ :: $HOSTSTATE$ ($HOSTOUTPUT$)" /1/statuses/update.xml

}

After putting this wherever you keep your commands in Nagios (commands.cfg, for instance), add the alert to the relevant services and hosts in the standard way. This is the simple, standard part, and there are many guides which cover the subject in great detail elsewhere.

NB:Contactname should be the person's twitter handle: any other Nagios macro can be used provided it references a valid twitter handle.

The clever bit. Skip to here if you know the bits above..

However, Nagios strips out all environment variables; Twurl relies on knowing the home directory of the user who is sending the message (in our case, the Nagios user) in order to access that user's Twurl credentials file. The result is that Twurl won't work with Nagios until a minor change to the code is made, as it will not know which directory to look in. The fix is as follows:

This is the file which needs to be modified:

 rcfile.rb 

In this file the first few lines should look like this:

module Twurl
class RCFile

FILE = '.twurlrc'
@directory ||= ENV['HOME']

class << self
attr_accessor :directory

And beneath is the modification:

module Twurl
class RCFile

FILE = '.twurlrc'
@directory ||= '/home/nagioshomedir/'

class << self
attr_accessor :directory

This modification means Nagios stripping environment variables no longer has any impact, and allows you to use it for alerts. Potentially, Twurl could be modified to accept a home-directory argument that would still assume whatever is in the ENV if it is available.

Oh, and one last thing to avoid tripping over: don't forget to follow your new monitor on twitter, and have it follow you too. If you don't do that, then your service's DMs won't land in your inbox when you need them most.