My fab file
12/3/2011There aren't enough examples online of Fabric scripts as used in the real world. The documentation is good but a real example is always better.
Maybe that is because Fabric scripts tend to be closely tied to the environment they are working on. To be able to post mine, I have had to change some of the details, too. There are also constraints that I deal with that most people won't have, like I am not allowed to ssh directly as the service account the program runs under. There are a few additional steps needed to make sure the permissions are set properly in the script below.
I save this as fabfile.py in my project directory. The first step is
to set the host names for the deployment. I have a method setup
to
handle that by manually hardcoding the server names. In my
environment, those rarely ever change. Others might get a list of
active servers from a cloud host, for example.
The deploy
method is the main entry point, but I have broken each
deployment step down into a function so they can be called
individually if needed. If I needed to restart memcached, for example,
I can run fab setup:prod restart_memcached
instead of logging into
the server manually.
This script handles the deployment for a Django web project. It works by creating a new virtualenv for each deployment, pulling the source from git, installing all the pip requirements (which are hosted locally on a private Chishop server), and finally restarting uwsgi.
This scheme works great:
- Easy rollback, just replace the symlink
- You always know that checking out the source results in a working project
- New project requirements are automatically applied, but don't interfere with rolling back
- I love getting an email containing the commits between versions
- Easily handle adding new servers, since the Fabric script handles setting up the project
Still, I have no idea how people have been using Fabric. Everybody says they are using Fabric but there are few examples online. For all I know, I could be doing it wrong. Please post your own fabfile.py so we can all learn or leave a comment.
from __future__ import with_statement
from datetime import datetime
import smtplib
from email.mime.text import MIMEText
import os
import pwd
from fabric.api import run, sudo, cd, env
GIT_URL = "ssh://mygitserver/project"
DEPLOYED = '/opt/project/deployed/{}'
DEPLOYED_DIR = '/opt/project/deployed'
def setup(name):
if name == 'uat':
env.hosts = fab_hosts = someUATservers
elif name == 'prod':
env.hosts = fab_hosts = somePRODservers
else:
raise ValueError('Invalid name')
env.envname = name.upper()
env.user = pwd.getpwuid(os.getuid())[0]
env.deploy = DEPLOYED.format(datetime.today().isoformat().replace(':', '_'))
env.project = '{}/project'.format(env.deploy)
def current_env():
"""
Get the path to the virtualenv currently in use
"""
current = DEPLOYED.format('current')
link = run('readlink -f {}'.format(current))
print 'current env', env.deploy
return link
def symlink_current():
current = DEPLOYED.format('current')
run('rm -f {}'.format(current))
run('ln -s {} {}'.format(env.deploy, current))
def create_virtualenv():
run('virtualenv -p /opt/apps/local/bin/python ' \
'--prompt=project{envname} ' \
'--no-site-packages --distribute {deploy}'.format(**env))
def cleanup_old_deployed():
current = os.path.basename(current_env())
old_entries = set(run('ls -1 {}'.format(DEPLOYED_DIR)).splitlines()) - {'current', current}
if old_entries:
with cd(DEPLOYED_DIR):
sudo('rm -rf {}'.format(' '.join(old_entries)))
def virtualenv(command):
with cd(env.project):
run('source {deploy}/bin/activate && {command}'.format(
command=command,
deploy=env.deploy))
def clone_source():
run('git clone {} {project}'.format(GIT_URL, **env))
run('cp /opt/project/site/local_settings.py {project}'.format(**env))
def install_requirements():
run('{deploy}/bin/pip install -r {project}/requirements/core.txt'.format(**env))
def syncdb():
virtualenv('python manage.py syncdb --migrate')
def django_tests():
virtualenv('python manage.py test')
def chown_virtualenv():
sudo('chown -R projectaccount.project {deploy}'.format(**env))
def restart_uwsgi():
sudo('kill -HUP `cat /opt/project/conf/project.pid`')
def restart_memcached():
# flush memcached or start it
run('echo "flush_all" | nc localhost 11211 || memcached -d')
def get_git_changes():
# first find the current hash for the latest commit in the current production env
current = current_env()
with cd('{}/project'.format(current)):
lastcommit = run("git show-ref --heads | awk '{print $1}'")
if not lastcommit:
return ''
with cd(env.project):
diff = run('git log --no-merges {}..HEAD | head -600'.format(lastcommit))
return diff
def deploy():
create_virtualenv()
clone_source()
changes = get_git_changes()
install_requirements()
syncdb()
symlink_current()
chown_virtualenv()
restart_uwsgi()
restart_memcached()
if changes:
message = MIMEText(str(changes))
message['Subject'] = 'Deployed project {envname}'.format(**env)
message['From'] = me # configure
message['To'] = ', '.join(to) # configure
smtp = smtplib.SMTP('smtprelay')
smtp.sendmail(me, to, message.as_string())
smtp.quit()