Building a notification system part 2: Travis CI

Encrypting username and password

Details see: https://docs.travis-ci.com/user/encryption-keys

Obviously, storing the API credentials as clear text in .travis.yml for everybody to read isn't a good idea.

One option could have been to store the password as a hidden environment variable and use that variable in the URI but that's not supported. Long story short, environment variables are simply not available to the notifications section of .travis.yml. ¹

Luckily Travis allows you to define encrypted values using your project's public key. The encrypted value can only be decrypted by the Travis build system since it has access to the project's private key.

Using the Travis CLI you can issue following command:

$ travis encrypt -r wishbone-modules/wishbone-output-twitter
Reading from stdin, press Ctrl+D when done
https://username:password@wishbone.mydomain.com/travis
Please add the following to your .travis.yml file:

  secure: "abcdefghijklmnopqrstuvwxyz"

You can now replace the cleartext URL with the complete returned value, including the secure: prefix ²:

notifications:
  webhooks:
    urls:
      - secure: "abcdefghijklmnopqrstuvwxyz"
    on_success: change
    on_failure: always
    on_start: change
    on_cancel: always
    on_error: always

Adding the /travis endpoint

Giving each service a dedicated endpoint simplifies things. It's clean, it allows you to restrict access using dedicated credentials and it already categorizes incoming data.

In order to add the /travis endpoint we need to define another resource in the incoming_webhooks instance. (see line 15):

"^travis$":
  users:
    - travis
  tokens: []
  response: "OK {{uuid}}"
  urldecoded_field: payload

The key ^travis$ is a regex matching the endpoint we'd like to configure. Users is an array of the usernames allowed access to the endpoint.

Tokens is left empty as we cannot use it since Travis webhook :notifications do not support that.

The response parameter composes the response going to the client which is in this context not that useful.

The Travis webhooks are delivered with a application/x-www-form- urlencoded content type using HTTP POST, with the body including a payload parameter that contains the JSON webhook payload in a URL-encoded format.

By setting the urldecoded_field to payload we instruct Wishbone to expect the incoming data has to be URL-decoded and the parameter of interest is payload. Once the value is acquired, it is send to the defined protocol decoder (line 12) for further processing.

The htpasswd field (line 21) should contain a htpasswd encoded password for user travis. This will be the password the Travis webhook should use to authenticate to this endpoint.

There is one last thing to do in order to actually create the endpoint. The wishbone_contrib.module.input.httpserver module maps endpoints to queues. When connecting incoming_webhooks.travis to funnel.travis in the routing table section (line 55), the behavior of the module is such that it creates an endpoint using the same name and maps it to that to a new queue. Any payloads successfully submitted to that endpoint end up in the corresponding queue from which it can flow to other module for further processing.

Composing the tweet

To compose a tweet and cherry-pick content from the Travis payload we are going to use the builtin wishbone.module.flow.queueselect module.

The main purpose of this module is to route events to certain queues based on the content of the payload along with an optional, payload.

We'll add an entry to the template key (line 32) with following content:

- name: "Travis Build"
  queue: "{{ 'outbox' if tmp.input.env.path_info == '/travis' }}"
  payload: >
    Travis build for '{{data.repository.name}}/{{data.branch}}' has status: '{{data.status_message}}'.

    Commit message: {{data.message}}

    Build: {{data.build_url}}

    Commit: {{data.compare_url}}

    #build #travis

The queue value is Jinja2 template which once rendered will have value outbox. This means the event will be forwarded to the outbox queue.

The payload template is used to compose the tweet cherry-picking data from the Travis payload.

An example Payload can be found here: https://docs.travis-ci.com/user/notifications/#Webhooks-Delivery-Format

By default, the result of the rendered template is stored in the event's tmp.payload.payload key from where it can be used by the twitter module instance to send out the tweet.