Technology

I’ve been using declarative pipelines in Jenkins for a while with the Slack plugin to send build notifications to Slack. The plugin does what it says on the tin but gives you a pretty boring message by default.  I used the environment variables available in the pipeline to make things a little bit better and link back to the job.  But I was still always disappointed the notifications didn’t contain more information. Thankfully version 2.3 of the plugin added support for the attachments portion of the Slack message API. I was able to leverage the attachments feature to get better message formatting. Meanwhile, I took some inspiration from this thread to incorporate test result summaries. ...

Pagerduty provides a built-in way to export your incident data but only a limited number of data fields are available on the basic plan. Rather than upgrade you can use the API to export the data to a CSV. I found this gist listed here. The python script works great but some of my incident messages contained JSON data that threw off Excel when opening the CSV. I slightly modified the script with character escaping to work around this (lines 98- 107). ...

I spent way too long troubleshooting a script step in a Jenkins declarative pipeline this week. After lots of frustration I stumbled upon this gist that was a life saver. Long story short you need \\ to escape literal quotes.

There is a nice plugin for Jenkins that lets you dynamically add capacity by spinning up EC2 instances on demand and then terminating them when the job queue expires. This is a great way to save money on an AWS based build infrastructure. Unfortunately, the plugin documentation is really light and there are a few gotchas to look out for. Security Groups This field only accepts comma separated security group IDs, not names. This is frustrating because other fields in the plugin take a space separated list (e.g. labels) ...

Now that roles and profiles are in my control repo my RSpec tests are taking longer then ever. As of this writing the control repo contains 938 tests and I’m still a long way from 100% coverage. This really slows down the feedback loop when running tests. When running locally I often just run RSpec against a specific spec file rather then run the whole test suite, but I still wanted a way to speed things up in Bamboo. ...

I spent some time tackling technical debt in our Puppet code this week. The biggest outstanding item was implementing eyaml for protecting secrets in Hiera. I’d also been encouraging developers to contribute to the Puppet code base for some time, but they were restricted from the control repo due to some secrets kept in Hiera. This put a big damper on collaboration as Hiera is the data engine for our roles and profiles. Separate git repos were also used for the profile and role modules due to this workflow. ...

rnelson0 is walking through automated Puppet testing with Jenkins on his blog. I thought I’d highlight how you can use a similar workflow in Atlassian’s Bamboo application. This assumes you already have a working Bamboo setup and are familiar with the general process for testing Puppet modules with rspec. Create a new plan The first step is to set up a new plan to use for the testing. Click “Create” and then “Create a new plan” in the top menu bar. ...

Two days ago I finished our migration to Puppet 4. Overall I’d say the process was pretty painless. The gist of what I did start running rspec tests against Puppet 4 fix issues found in tests run the catalog preview tool and fix any issues found turn on the future parser on the existing master turn off stringify facts create new master and PuppeDB server migrate agents to new master Thankfully our code wasn’t too difficult to update and most of the forge modules we use had also been updated. ...

Up until today we had been using a map variable in terraform to choose our ubuntu 14 ami based on region. variable "ubuntu_amis" { description = "Mapping of Ubuntu 14.04 AMIs." default = { ap-northeast-1 = "ami-a25cffa2" ap-southeast-1 = "ami-967879c4" ap-southeast-2 = "ami-21ce8b1b" cn-north-1 = "ami-d44fd2ed" eu-central-1 = "ami-9cf9c281" eu-west-1 = "ami-664b0a11" sa-east-1 = "ami-c99518d4" us-east-1 = "ami-c135f3aa" us-gov-west-1 = "ami-91cfafb2" us-west-1 = "ami-bf3dccfb" us-west-2 = "ami-f15b5dc1" } } We would then set the ami id like so when creating an ec2 instance. ami = "${lookup(var.ubuntu_amis, var.region)}" The problem we ran into is that we now use Ubuntu 16 by default and wanted to expand the ami map to contain its ID’s as well. I quickly discovered that nested maps like the one below work. ...

I’ve poked a lot of fun at chatops but I have found some value in portions of the practice. Let me state upfront that I do not believe paying attention to the chat room all day and having your attention interrupted non-stop is a productive or healthy practice. I have found some big benefits to “chatops” however. Visibility Work that is done in the chatroom, or filtered into the chatroom, is visible to the whole team. This helps the team be aware of what others are doing and stay up to date. I’ve picked up on quite a few things from this that I wouldn’t have learned other wise. This is also why we choose to route a fair amount of notifications into chat. For example we have Jira connected to HipChat and it makes it really easy to stay on top of issues. We also push commit notifications, build notifications, etc in the chatroom. The downside to this is that the rooms get noisy and make it harder to follow actual conversations between humans. One strategy we use to combat that is creating multiple rooms and focusing them around a subject. ...