Deploying to production via FTP using GIT hooks

In this post, we address a common development task of pushing to production.

This tutorial assumes you are working on linux, use GIT for version control and are somewhat familiar with python. I’ve only recently started using python for scripting and I can say it’s very friendly for these kind of tasks. Also, I assume your web application is created in a interpreting language and requires no compilation or binaries (although adjustments can be made to deploy binaries as well).

This post is an extension of my prior post on how to make changes without fear of breaking production.

Sometimes you won’t have shell access to the production server (or even GIT for that matter) on which you deploy your web application. However, you may have FTP access. You probably already know that it can be exhausting to copy the files manually over FTP every time you are ready to deploy changes. Also you could disrupt the production if you accidentally copy directly to the public production directory. To overcome these troubles we do what we always do: AUTOMATE.

In a few steps, I will show you how to achieve the following:
1) Upload working-copy files from testing environment to production on every “push” (git push)
2) Perform the upload so it does not disrupt the current production ( as uploading multiple files can take time and break things )
3) Create a backup of the previous production version
4) Put the uploaded code to production

Essentially, I always keep in mind that these kind of actions should be performed in a transactional manner, without putting your production at risk while deploying the changes.

Pitfall prevention:
This tutorial shows how to deal solely with your application code, it does not consider files generated or uploaded to the production server between “pushes”.
If you’re serving images and other files which are not code in your same production directory, make sure you adjust the python script to move those directories back and forth as needed. Or try to serve those files in different production directories as it should be in the first place.
Bottom line, this is mainly intended for pure code modules and pushing those to production via FTP.

Now that you’ve read the disclaimer, let’s start with the instructions. :-)

For tutorial purposes, let’s assume this is your configuration

/home/test # home directory on the testing server

example.com # production domain
ftp.example.com # FTP domain on the production server

/home/production # home directory on the production server
/home/production/www # application directory, served at example.com
/home/production/archive # previous production directories are stored here

Start the shell on your testing server and execute all in the same session

Set up your paths:

barerepo="/home/test/.bare-git-repo"
workrepo="/home/test/work-repo"

Create a bare GIT repository you will push your commits to

mkdir $barerepo 
cd $barerepo
git init --bare

Create your “working-copy” directory and GIT working repository

mkdir $workrepo
cd $workrepo
git init
git remote add origin $barerepo

Add a readme file and commit it:

echo "My project" > README.md
git add README.md
git commit -m "Initial commit"

Create a python script called “deploy.py” and store it to “$barerepo/hooks”
Make sure to change the variables for the paths and FTP connection strings to match your configuration.
Also make the script executable ( chmod +x deploy.py )

/home/test/.bare-git-repo/hooks/deploy.py

#!/usr/bin/python
import os
import time
from ftplib import FTP


def senddir(path,dest,ftp):
  for root, dirs, files in os.walk(path):
    for file in files:
      source_filename = os.path.join(root, file)
      dest_filename = dest + source_filename.replace(path,'')
      dest_dir = os.path.dirname( dest_filename )
      
      list = dest_dir.split('/')
      
      ftp.cwd( "/" )
      partial_dir = ""
      for part in list:
        partial_dir += "/" + part
        partial_dir = partial_dir.replace('//','/')        
        if part != "" and not (part in ftp.nlst() ):      
          ftp.mkd( partial_dir )
          print "Created " + partial_dir + " on remote server"        
        ftp.cwd( partial_dir )
      
      ftp.cwd( "/" )
      
      ftp.storlines("STOR " + dest_filename, open(source_filename,"r"))
      print "Stored " + dest_filename  + " on remote server"
      

timestamp = str(int( time.time()))

# local directory
local_dir = "/home/test/work-repo"

# FTP server connection strings      
ftp_host = "ftp.example.com"
ftp_username = "user@example.com"
ftp_password = "password"

# remote server paths
dest_dir = "/home/production/.deploy-" + timestamp
backup_dir = "/home/production/archive/www-" + timestamp
production_dir = "/home/production/www"
  
if __name__ == '__main__':  
  
  print "Connecting to FTP host " + ftp_host
  ftp = FTP( ftp_host , ftp_username , ftp_password )
  
  print "Starting directory transfer of " + local_dir
  senddir( local_dir , dest_dir, ftp)
  
  print "Renaming " + production_dir + " to " + backup_dir
  ftp.rename( production_dir , backup_dir )
  
  print "Renaming " + dest_dir + " to " + production_dir
  ftp.rename( dest_dir , production_dir )
  
  print "Transfer complete"

The deploy.py script will transfer all of your working-copy files to the production server, make a backup of your current production and deploy the new code to the production directory. You can also use it manually from your terminal (without GIT).

Setting up the GIT post-update hook

Rename
/home/test/.bare-git-repo/hooks/post-update.sample
to
/home/test/.bare-git-repo/hooks/post-update

add this code to the “post-update” hook script and adjust the paths respectively

/home/test/.bare-git-repo/hooks/post-update

#!/bin/sh
#
# An example hook script to prepare a packed repository for use over
# dumb transports.
#
# To enable this hook, rename this file to "post-update".

function deploy {	
  cd /home/test/.bare-git-repo/hooks
  echo "Deploying local GIT work repo"
  ./deploy.py 
}

deploy

Ready for the “first push”

Now, since you’ve already done your initial commit, just push it to the origin repository.


cd $workrepo # /home/test/work-repo
git push origin master

If the push action succeeds, GIT will automatically start the “post-update” hook and your files will be uploaded to production safely. You could also modify the post-update script to send you an e-mail. Would’t that be cool? :-)

Coding Examples, Linux Shell / Bash Scripting, Open Source, Tutorials, Version Control | Posted on April 14, 2014 by .

About Kristijan Burnik

Kristijan Burnik is a Programmer and Web Developer specializing in Server-side and Client-side Technologies and Application Development on Linux Servers and Windows Desktop . Also has experience in Networking, Desktop application development as well as Android mobile application development . Experienced in Programming Languages like Java, C, C++, C#, PHP, Javascript, MySQL and somewhat in other languages like Bash, Perl & Python. Sometimes he works as a Graphical Designer for digital production as well as for printing and advertising. He dedicates his spare time writing Tech Articles on his blog in order to share his work with others, as well as to document his projects for his own use. He's also an Educator & Mentor in field of Algorithms and Programming to young programmers in Zagreb, Croatia.