Cloud architecture, Chef, AWS, and Python stuff

Python Tools

This is a list of some of the python tools/libraries that I tend to use regularly. I am sure I forgot a few.

Virtualenv + virtualenvwrapper

This needs no introduction, if you are developing in Python – you should be using virtualenv to create isolated environments.

One thing I like about virtualenv that was not immediately obvious. You don’t need to run $ source bin/activate if you are calling it from a shell script/cron/supervisor. You can simply provide the full path to the virtualenv Python.

From the virtualenv documentation:

If you directly run a script or the python interpreter from the virtualenv’s bin/ directory (e.g. path/to/env/bin/pip or /path/to/env/bin/python there’s no need for activation.

I also use virtualenvwrapper to add some helpers to make life easier.

It organizes all of my virtualenvs in a single location and then I can simply execute $ work on <env_name> and it will activate the virtualenv and cd me to the directory I set for the project.


Ok – if you aren’t using AWS, boto won’t really help you. It is the Python SDK for AWS. Also supports MWS as well (although I haven’t tried that out yet)


Writing command line tools always starts out the same. Look up the optparse docs and write the boilerplate code to add the arguments and (maybe) the help text. Docopt is a great idea. Instead of writing any boilerplate code, you just create the usage documentation at the start of the file.

Once that is defined, you pass the __doc__ value into the library and then you are done.

Usage: move --city=<city_name> stay [--city=<city_name>] (-h|--help) --version

    -h --help            Show this screen.
    --version            Show version.
    --city=<city_name>   City name [default: Victoria]

args = docopt(__doc__, version='1.0.0')

city = args['--city']

if args['move']:
    print "I'm moving to {0}!".format(city)
elif args['stay']:
    print "I'll stay here in {0} then".format(city)

There is a reference implementation in Python. But there are also implementations in many other languages such as Ruby, Go, PHP, Bash and others (full list at the bottom))

Try out docopt in your browser here


Fabric is an extremely useful tool for automating common tasks on remote servers.

If you have a text file that contains some shell commands you are copy + pasting onto a remote server. Turn that into a fabric script. Not only will it make it easier to execute, but it will also let you check that box for “Infrastructure as Code”