Cloud architecture, Chef, AWS, and Python stuff

Chef Tools

So Fabric is still my go-to tool for automating any remote commands. And despite the fact I am a huge Chef fan, I still see a use for both tools.

In fact, Fabric makes a great tool to automate installing chef-client, configuring it and then executing the initial chef-run.

It can also be handy to execute specific recipes or run_lists on a remote host.


Chef-Sugar provides a lot of nice syntactic sugar for use in your chef cookbooks.

if node['platform_family'] == 'rhel'
    execute 'yum update'

This is so much nicer:

if rhel?
    execute 'yum update'

Also the short hand for compile time dependencies is much easier to read to non-chef users:

package 'apache2' do
    action :nothing

It was never obvious as to why it was action :nothing and then run_action(:install)

But with chef-sugar…

compile_time do
    package 'apache2'
end is clear that this is a compile time action.

Leaving out the obvious question a new chef user might ask, “Ruby doesn’t need to be compiled…what is this?”. To which I would say – best to read About the chef-client Run. Compile-time resources are executed in the step identified as “Identify resources, build the resource collection”. As are any blocks of ruby code not in a ruby_block.


I am a huge fan of Bryan Berry and the Food Fight Show podcast. But I view chef-rewind as a “last resort”. chef-rewind lets you monkey-patch chef resources. And I am not a huge fan of monkey-patching unless absolutely necessary.

The use-case for this is primarily to make a change to a resource defined in an upstream cookbook you don’t control or want to fork and maintain.

For example – if a cookbook you are using has a template you need to customize, you can use chef-rewind to modify the template resource from the upstream recipe to look in your cookbook instead.

Here is the example from the chef-rewind docs:

# file postgresql/recipes/server.rb
template "/var/pgsql/data/postgresql.conf" do
  source  "postgresql.conf.erb"
  owner "postgres"

# file my-postgresql/recipes/server.rb
chef_gem "chef-rewind"
require 'chef/rewind'

include_recipe "postgresql::server"
# my-postgresql.conf.erb located inside my-postgresql/templates/default/my-postgresql.conf.erb
rewind :template => "/var/pgsql/data/postgresql.conf" do
  source "my-postgresql.conf.erb"
  cookbook_name "my-postgresql"


The Berkshelf Way. Nuff said.

Vagrant + Bento

Vagrant is the best way for developing and testing cookbooks. Bring up a virtual machine in VirtualBox or VMWare. Test your cookbooks, then destroy it and try again. Also really nice to just bring up a base linux machine to play around with something and not have to worry about cleaning up afterwards.

A new tool from Chef (formerly known as Opscode). Bento is a tool to create VMs (Vagrant and VMWare) using Packer


Ok so this last one isn’t exactly a Chef tool…

John Vincent (@lusis) wrote an excellent blog post for Sysadvent 2013 on Omnibus – Sysadvent 2013 – Day 16 – omnibusing your way to happiness.

I have played with it a bit, but want to spend more time with it over the next few months.

The idea behind Omnibus (created by Chef) is to install all of the required packages/libraries/etc that an application requires into a single location and then package that for re-distribution. This includes any libraries that need to be linked to.

This avoids all of the pain of package managers/dependenncies, differences in OS distros etc.

For example – you write an awesome new Python application that requires Python 3. But you need to run it on an OS that still only ships with Python 2.6 (if you are lucky…).

So you use Omnibus to create an RPM/PKG that contains Python 3, all of the python modules, postgres database, nginx, gunicorn, and your code. All rolled up into a single OS package you can install by just installing the package.

And since it uses Vagrant the whole thing can be scripted in a Jenkins server and you can build OS packages that target different platforms at the same time.

It is pretty awesome. Definately a good read.