Category: Software Development

Note to self — before deploying a local Django app to Heroku I should do the following:

Install the wsgi HTTP server, gunicorn:

pip install gunicorn

Install package to help work with environment variables:

pip install python-dotenv

Install Python-PostgreSQL Database Adapter:

pip install psycopg2

Create requirements file:

pip freeze > requirements.txt

Configure settings.py to load environment variables and the database:

import dotenv
import dj_database_url

# this line is already in your settings.py
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

# load environment variables from .env
dotenv_file = os.path.join(BASE_DIR, ".env")
if os.path.isfile(dotenv_file):
    dotenv.load_dotenv(dotenv_file)

# load database from the DATABASE_URL environment variable
DATABASES = {}
DATABASES['default'] = dj_database_url.config(conn_max_age=600)

Also delete the default database config:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}

Add a .env file to project base directory and add the database URL:

DATABASE_URL=sqlite:///db.sqlite3

Make sure .env is added to .gitignore so it doesn’t get pushed to repo. My last .gitignore looked like:

*.pyc
*~
__pycache__
myvenv
venv
db.sqlite3
.DS_Store
.env

Assuming there is already a heroku account run:

heroku login

locally to associate machine with heroku account.

On the heroku dashboard create a new app. Under the “Settings” tab and under “Buildpacks” add a new buildpack called “heroku/python”.

Under “Config Variables” add “DISABLE_COLLECTSTATIC” with a value of “1”.

To set-up the database go back to command line and enter:

heroku addons:create heroku-postgresql:hobby-dev -a YOUR_APP_NAME

This provisions a new postgres database and should add the url to the Config Variables. Run command:

heroku config -a YOUR_APP_NAME

DATABASE_URL should be added to Config Variables.

Now create a new file called Procfile in the base directory of project and add:

web: gunicorn YOURSITE.wsgi --log-level=info --log-file -

Where YOURSITE is specific to the project.

Add another file in base directory called runtime.txt and add the version of python you want to use:

python-3.5.2

To serve static assets I use the WhiteNoise project. To install:

pip install whitenoise
...
pip freeze > requirements.txt

Add it to the app by changing wsgi.py to include:

from django.core.wsgi import get_wsgi_application
from whitenoise.django import DjangoWhiteNoise

application = get_wsgi_application()
application = DjangoWhiteNoise(application)

Now that should be it! Commit the changes locally then run:

git push heroku master

It should display the progress of installation and eventually be up and running.

References

This was trying to put a bunch of stuff I read in the articles below into an order I’ve found works for me.

https://blog.heroku.com/in_deep_with_django_channels_the_future_of_real_time_apps_in_django

I’m working on a project that has multiple RaspberryPI’s, distributed in multiple locations, but all running the same code (Django Channels with Python background worker scripts). I wanted to be able to deploy any updates to the main code to all the PI’s with as minimal manual interaction as possible. I couldn’t find an obvious way to do this so thought I’d summarise what I’ve set-up in the hope I may get some feedback or it would help someone else.

Basic overview of solution

The main code base is stored in a Git repository in BitBucket.

I have a development PI set-up and when I’m ready I push code changes from it to the main branch of the BitBucket repo.

Each deployed PI has an updateWorker Python script running that periodically downloads the remote code from the BitBucket repository using the git fetch command. It then checks for changes using the git status command and finally updates the code if there are changes using git reset.

Running Git from Python

To run the Git commands from my Python script I’m using a Python Package called sh. It functions as a subprocess replacement that allows you to call any program as if it were a function. I plan to write another post about using it but the basics are intuitive. For example to do a git status call on the local directory and save the response in a variable called statusCheck:

import sh
from sh import git

statusCheck = git("status")

(See this tutorial for more details on how the login to BitBucket is handled)

updateWorker Script

import sh
from sh import git
import time
import os, sys

aggregated = ""

def CheckForUpdate(workingDir):
    print("Fetching most recent code from source..." + workingDir)

    # Fetch most up to date version of code.
    p = git("--git-dir=" + workingDir + ".git/", "--work-tree=" + workingDir, "fetch", "origin", "master", _out=ProcessFetch, _out_bufsize=0, _tty_in=True)               
    
    print("Fetch complete.")
    time.sleep(2)
    print("Checking status for " + workingDir + "...")

    statusCheck = git("--git-dir=" + workingDir + ".git/", "--work-tree=" + workingDir, "status")

    if "Your branch is up-to-date" in statusCheck:
        print("Status check passes.")
        print("Code up to date.")
        return False
    else:
        print("Code update available.")
        return True

def ProcessFetch(char, stdin):
    global aggregated

    sys.stdout.flush()
    aggregated += char
    if aggregated.endswith("Password for 'https://yourrepo@bitbucket.org':"):
        print(mainLogger, "Entering password...", True)
        stdin.put("yourpassword\n")

if __name__ == "__main__":
    checkTimeSec = 60
    gitDir = "/var/testupdate/"
   
    while True:
        print("*********** Checking for code update **************")                                                     
    
        if CheckForUpdate(gitDir):
            print("Resetting code...")
            resetCheck = git("--git-dir=" + gitDir + ".git/", "--work-tree=" + gitDir, "reset", "--hard", "origin/master")
            print(str(resetCheck)) 
                                                                                                                                                                
        print("Check complete. Waiting for " + str(checkTimeSec) + "seconds until next check...", True)
        time.sleep(checkTimeSec)

updateWorker uses the –git-dir and –work-tree options to define where the local git repo is for the code.

The script parses the response from the git status command to determine if there are updates. If an update is available we will see:

On branch master

Your branch is behind ‘origin/master’ by 2 commits, and can be fast-forwarded.
(use “git pull” to update your local branch)
nothing to commit, working directory clean

On my PIs the updateWorker then kills all the running scripts which are automatically restarted by another background process, this should mean the scripts are all running up to date versions. So far it’s working well but not sure if I’m doing something crazy wrong!