In a previous article I wrote about managing Naemon and Nagios alerts with Alertmachine. In this article I will cover how you can run a fully operational Alertmachine container in a matter of minutes using Docker. [1]
Docker
Docker is an open source project to pack, ship and run any application as a lightweight container. I have made a Docker image which contains an Alertmachine instance including all its dependencies so it's easy to get it up and running in no time. To execute the steps explained in this article you are supposed to have a running Docker instance.
Adding docker image
Before you can start to use the container you obviously have to add it to the list of available docker images.
Adding the alertmachine image to your docker environment is done by executing following command:
$ [smetj@indigo ~]$ docker pull --tag alertmachine smetj/wishbone Pulling repository smetj/wishbone 9bb81f6baa3a: Download complete 539c0211cd76: Download complete 7edf4f9e2b98: Download complete 4bec87fafa27: Download complete 2ab67fc835bf: Download complete [smetj@indigo ~]$
You should now be able to see the alertmachine image:
[smetj@indigo ~]$ docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE smetj/wishbone alertmachine 9bb81f6baa3a 5 hours ago 756.2 MB centos buildbox dcee5313e217 8 days ago 1.073 GB centos base 4bec87fafa27 8 days ago 662.8 MB centos 6.4 539c0211cd76 12 months ago 300.6 MB [smetj@indigo ~]$
Running the Alertmachine container
To run Alertmachine you need to have 3 pieces of configuration:
- The Wishbone bootstrap file
- The alert rules.
- The email templates.
I have made a repository available which you can use as a start and which we will use as an example to complete the next steps in this article.
$ git clone https://github.com/smetj/alertmachine.git
To run the alertmachine container issue following command:
$ docker run -i -t --privileged -p 19283:19283/udp \ -v /home/smetj/data/projects/github/alertmachine/alertmachine:/share/alertmachine \ smetj/wishbone:alertmachine debug --config /share/alertmachine/bootstrap/alertmachine.yaml
Now lets run over the most interesting options:
The -p option maps the container's port 19283/UDP to the same port of the host. The UDP listener is one way of receiving alert events and is defined in the bootstrap file at line 7.
The -v option exposes directory /home/smetj/data/projects/github/alertmachine/alertmachine to the container and "mounts" it do the /share/alertmachine mountpoint. In our example, the directory is the alertmachine Git repository we have clonded in a previous step. The Wishbone instance reads all configuration from this location. The advantage is this way we can keep the container more generic. The alternative is to add the configuration inside the container so absolutely no external dependencies exist.
smetj/wishbone:alertmachine defines the image we want to use.
The command we execute inside the container is:
$ /usr/bin/wishbone debug --config /share/alertmachine/bootstrap/alertmachine.yaml
The yaml config file is expected to be found in the host's mounted volume as defined by the -v switch.
We use the debug command to start Wishbone since that blocks the container from exiting.
Testing Alertmachine
We test our setup by running the container as defined in the previous step.
... snip for brevity ... 2014-03-30T17:10:46 pid-1 informational validate: Initiated 2014-03-30T17:10:46 pid-1 informational validate: Created module queue named inbox with max_size 0. 2014-03-30T17:10:46 pid-1 informational validate: Created module queue named outbox with max_size 0. 2014-03-30T17:10:46 pid-1 informational validate: Started 2014-03-30T17:10:46 pid-1 informational validate: Function <bound method JSON.consume of <wb_function_json.wb_function_json.JSON instance at 0x1d7df38>> started to consume queue <wishbone.tools.wishbonequeue.WishboneQueue instance at 0x1d853b0>. 2014-03-30T17:10:46 pid-1 informational match_engine: Initiated 2014-03-30T17:10:46 pid-1 informational match_engine: Created module queue named inbox with max_size 0. 2014-03-30T17:10:46 pid-1 informational match_engine: Created module queue named outbox with max_size 0. 2014-03-30T17:10:46 pid-1 informational match_engine: Created module queue named email with max_size 0. 2014-03-30T17:10:46 pid-1 informational match_engine: Started 2014-03-30T17:10:46 pid-1 informational match_engine: Function <bound method SequentialMatch.consume of <pyseps.sequentialmatch.SequentialMatch instance at 0x1d9f4d0>> started to consume queue <wishbone.tools.wishbonequeue.WishboneQueue instance at 0x1d9f7e8>. 2014-03-30T17:10:46 pid-1 informational email: Initiated 2014-03-30T17:10:46 pid-1 informational email: Created module queue named inbox with max_size 0. 2014-03-30T17:10:46 pid-1 informational email: Created module queue named outbox with max_size 0. 2014-03-30T17:10:46 pid-1 informational email: Initialiazed. 2014-03-30T17:10:46 pid-1 informational email: Started 2014-03-30T17:10:46 pid-1 informational email: Function <bound method Email.consume of <wb_output_email.wb_email.Email instance at 0x1dff488>> started to consume queue <wishbone.tools.wishbonequeue.WishboneQueue instance at 0x1e0a710>. 2014-03-30T17:10:46 pid-1 informational input_gearman: Gearmand worker instance started 2014-03-30T17:10:46 pid-1 warning input_gearman: Connection to gearmand failed. Reason: Found no valid connections in list: [<GearmanConnection localhost:4730 connected=False>]. Retry in 1 second. 2014-03-30T17:10:46 pid-1 informational match_engine: Monitoring directory /share/alertmachine/rules/ for changes 2014-03-30T17:10:46 pid-1 informational match_engine: New set of rules loaded from disk
Have a look at the running Docker processes:
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 6ea1bd4bf097 smetj/wishbone:alertmachine /usr/bin/wishbone de 2 seconds ago Up 2 seconds 0.0.0.0:19283->19283/udp distracted_wozniak
Now send a test event into the container's UDP socket:
$ cat sample_json_alert_event/sample.json |nc -u localhost 19283
When we return to our running Alertmachine docker terminal we should see something similar to this:
{'header': {'match_engine': {'to': ['noc@your_company.local'], 'from': 'monitoring@your_company.local', 'template': 'host_email_alert', 'rule': '000-host-alert', 'subject': u'Alert - Host some_host_001.local is DOWN.'}}, 'data': u'Host notification.\n\nHostname : some_host_001.local\nIP : 127.0.0.1\nNotification Type : DOWN\nTime : Fri Mar 21 15:30:28 CET 2014\nHost State : DOWN\n\nAdditional Info :\n\n PING ok - Packet loss = 0%, RTA = 0.80 ms.'}
If you actually want and alert send out by mail instead of sending it to SDOUT you should alter the bootstrap file's routing table and connect template.outbox to email.inbox.
Final words
Docker offers an interesting approach to share and deploy Wishbone instances. Using different bootstrap files, multiple containers could easily share the same rules and templates directory, consume alert events from mod_gearman and run in parallel.
If you take the Alertmachine container for a spin, send over your feedback.
| [1] | This article has been altered for correctness |