Setting up the project

My aim when setting this up was to run the application locally using DEBUG=True, python manage.py runserver, but to use Heroku for production.

Heroku

Heroku is pretty easy to use. I basically just followed the documentation for deploying a Djano app. Because I was using a slightly different configuration in my Django app, I had to do some shimmying at the heroku end.

To ensure that heroku runs the Django app using the staging.py settings module, you have to change the DJANGO_SETTINGS_MODULE environment variable in heroku:

1

heroku config:set DJANGO_SETTINGS_MODULE=<project_name>.settings.staging.py`

It’s just as easy to set and unset environment variables in heroku settings in the same way:

1
2
3

heroku config:set DISABLE_COLLECTSTATIC=1
heroku config:unset DISABLE_COLLECTSTATIC
heroku config:set AWS_SECRET_ACCESS_KEY=FFFafjkjkfjkfj11fkid9f9929g9h9hjdka

You particularly don’t want your AWS settings going into settings.py files, hence the use of os.environ.get() in base.py

You got to do this:

Settings

I experimented with the Two Scoops of Django convention of using separate settings files. This makes it easier to have versions of the application running locally and in production (or in “staging”). Thoroughly recommend that book by the way, although there are alternative ways to think about coding your business logic - such as Django Service Objects.

To achieve this, I set up the Django application in the normal way, but created a new settings folder in the “project folder” (which sits alongside the app directory you should have set up as part of starting a new Django project), the contents of which are as follows:

1
2
3
4
5
6

<project-root-here>/settings/
├── __init__.py
├── base.py
├── heroku_local.py
├── local.py
└── staging.py

base.py

This contains most of the baseline settings already generated automatically by django-admin startproject. These settings are imported by the other “sub” settings modules.

It looks like this (don’t copy and paste as there are placeholders in here):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

import os
from pathlib import Path

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = Path(__file__).resolve().parent.parent.parent
STATIC_URL = '/static/'
SECRET_KEY = '<secret-key-here>'

ALLOWED_HOSTS = [
    '0.0.0.0',
    'localhost',
    '127.0.0.1',
    '<heroku-url-here>',
]

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'storages',
    'rest_framework',
    'passes.apps.<APP_NAME>Config',
]

REST_FRAMEWORK = {
    'DEFAULT_PERMISSION_CLASSES': (
        'rest_framework.permissions.IsAdminUser',
    ),
    'DEFAULT_AUTHENTICATION_CLASSES': (
        'rest_framework.authentication.BasicAuthentication',
    )
}

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = '<project-root-here>.urls'

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [BASE_DIR / 'templates'],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

WSGI_APPLICATION = '<project-root-here>.wsgi.application'


AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True

staging.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

# settings/staging.py

from .local import *
import os
import dj_database_url

# Heroku stuff for database
db_from_env = dj_database_url.config(conn_max_age=500)
DATABASES['default'].update(db_from_env)
ALLOWED_HOSTS = ['<heroku-url-here>']
DEBUG = False

# not using for now
#MEDIA_ROOT = BASE_DIR / 'media'
STATIC_ROOT = BASE_DIR / 'static_root'

#STATICFILES_DIRS = (
#    BASE_DIR / 'static',

# AWS settings
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
AWS_STORAGE_BUCKET_NAME = '<aws-bucket-name-here>'
AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY') 
S3_URL = 'http://%s.s3.amazonaws.com/' % AWS_STORAGE_BUCKET_NAME
STATIC_URL = S3_URL

heroku_local.py

1
2
3

from .staging import *

ALLOWED_HOSTS = ['localhost', '0.0.0.0']

These are the settings used when running heroku local (more on which later). Here were are saying that everything that we need to run Heroku is in the staging settings module, but here we just want to override the ALLOWED_HOSTS setting.

.heroku_env

I use this to set environment variables for running heroku local web with the -e flag:

1

heroku local web -e <project-name>/settings/.heroku_env

Includes:

1

DJANGO_SETTINGS_MODULE=<project-name>.settings.heroku_local

I also chuck my AWS keys here so I can test that locally. Kept out of the git repo, or course.

bash scripts to run everything

WRITE TEXT HERE.

amending the model

Amend model.py and populate.py accordingly. If using ModelSerialzer you don’t need to do too much more. Then you have to tear down the database (using bash scripts/initialise.sh local), ./manage.py makemigrations, ./manage.py migrate, ,/manage.py populate and finally ./manage.py createsuperuser.

If you don’t want to do all that, then just run bash scripts/initialise.sh local --do_migrations, then create the superuser.

manage.py commands to populate database

WRITE TEXT HERE.

Hosting Django static files in an AWS bucket

It took a lot of fannying around, but I finally go there. Here are the steps to doing it - hopefully this will enable a reproducable situation.

Create a new user on AWS

Here. Give it a decent simple username and ensure Programmatic Access is checked. Download the CSV containing the AWS Access ID and AWS Access Secret Key here before you navigate away.

Note of the ARN

By clicking on the newly created username at the above page. Copy and paste this and hang onto it because you’ll need this to set up your bucket policy.

Create a new bucket

Here. Give it a name and sensible region. Copy the settings from a previous one that works in Django if you can.

Set permissions

Set permissions by clicking on the bucket in the bucket list and going to Permissions tab, then Bucket Policy. I did a lot of fucking about here when I set this up for the first time but I found this the easiest way to start with a useable JSON file. Click the Policy generator button.

Create a policy

Using the policy generator. Select “S3 Bucket Policy” as Type. Select Allow, paste your ARN from the user console into the Principle box, check All Services, and then put the ARN for the bucket in the ARN box, using arn:aws:s3::. Then click add Statement. This is the main thing that gives your app programmatic access to the bucket. I don’t think we need to worry about adding other statements here necessarily, but there are loads of other permissions we can set. Click Generate.

Copy the resulting JSON file. You need to paste this into Bucket Policy text area and click Save. This should now mean that any application quoting this user’s id and secret key can programatically acess the files in the bucket. That’s the idea here.

Templates

When using the Django Rest Framework, you want to override the look and feel of the default site.

If you have set BASE_DIR and TEMPLATES in base.py as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

import os
from pathlib import Path

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = Path(__file__).resolve().parent.parent.parent

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [BASE_DIR / 'templates'],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

then in your project root, create the following structure:

1
2
3
4

templates/
└── rest_framework
    ├── api.html
    └── base.html

You can then customise these files to use a different CSS theme. As advised, I went to Bootswatch.

Don’t hard-code links in your template now that you’re using STATIC_ROOT.

Pushing static files to AWS from heroku

When you git push heroku master, heroku will automatically run python manage.py collectstatic and will put all static files into STATIC_ROOT (which is set in staging.py). The trick is to then push these to AWS so that they become available to the links in your template files (via the STATIC_URL variable set in staging.py).

Use django-storages and configure it push to Amazon S3. This also requires boto3.