Django settings.py for the real world

The settings.py is the central configuration for all Django projects. In previous chapters you already worked with a series of variables in this file to configure things like Django applications, databases, templates and middleware, among other things.

Although the settings.py file uses reasonable default values for practically all variables, when a Django application transitions into the real world, you need to take into account a series of adjustments, to efficiently run the Django application, offer end users a streamlined experience and keep potentially rogue attackers in check.

Switch DEBUG to False

One of the first things that's necessary to launch a Django application into the real world is to change the DEBUG variable to False. I've briefly mentioned in previous chapters how Django's behavior changes when switching DEBUG=False to DEBUG=True. All these behavioral changes associated with the DEBUG variable are intended to enhance project security. Table 5-1 illustrates the differences between having a project run with DEBUG=False and DEBUG=True.

Table 5-1. Django behavior differences between DEBUG=True and DEBUG=False

As you can see in table 5-1, the changes enforced by changing DEBUG=True to DEBUG=False are intended for publicly accessible applications (i.e. production environments). You may not like the hassle of adapting to these changes, but they are enforced to maintain a heightened level of security on all Django projects that run in the real world.

Define ALLOWED_HOSTS

By default, the ALLOWED_HOSTS variable in settings.py is empty. The purpose of ALLOWED_HOSTS is to validate a request's HTTP Host header. Validation is done to prevent rogue users from sending fake HTTP Host headers that can potentially poison caches and password reset emails with links to malicious hosts. Since this issue can only present itself under an uncontrolled user environment (i.e. public/production servers), this validation is only done when DEBUG=False.

If you switch to DEBUG=False and ALLOWED_HOSTS is left empty, Django refuses to serve requests and instead responds with HTTP 400 bad request pages, since it can't validate incoming HTTP Host headers. Listing 5-1 illustrates a sample definition of ALLOWED_HOSTS.

Listing 5-1 Django ALLOWED_HOSTS definition

ALLOWED_HOSTS = [
    '.coffeehouse.com',
    '.bestcoffeehouse.com',
]

As you can see in listing 5-1, the ALLOWED_HOSTS value is a list of strings. In this case it defines two host domains, that allow bestcoffeehouse.com to act as an alias of coffeehouse.com. The leading .(dot) for each domain indicates a sub-domain is also an allowed host domain (e.g. static.coffeehouse.com or shop.coffeehouse.com is valid for .coffeehouse.com).

If you wanted to accept a single and fully qualified domain (FQDN) you would define ALLOWED_HOSTS=['www.coffeehouse.com'], which would only accept requests with an HTTP Host www.coffeehouse.com. In a similar fashion, if you wanted to accept any HTTP host -- effectively bypassing the verification -- you would define ALLOWED_HOSTS=['*'] which indicates a wild-card.

Be careful with the SECRET_KEY value

The SECRET_KEY value in settings.py is another security related variable like ALLOWED_HOSTS. However, unlike ALLOWED_HOSTS, SECRET_KEY is assigned a default value and a very long value at that (e.g. 'oubrz5ado&%+t(qu^fqo_#uhn7*+q*#9b3gje0-yj7^#g#ronn').

The purpose of the SECRET_KEY value is to digitally sign certain data structures that are sensitive to tampering. Specifically, Django by default uses the SECRET_KEY on sensitive data structures like session identifiers, cookies and password reset tokens. But you can rely on the SECRET_KEY value to cryptographically protect any sensitive data structure in a Django project^[1].

The one thing the default data structures signed with the SECRET_KEY have in common, is they're sent to users on the wider Internet and are then sent back to the application to trigger actions on behalf of users. It's in this scenario we enter into a trust issue. Can the data sent back to the application be trusted ? What if a malicious user attempts to simulate another user's cookie or session data to hijack his access ? This is what digitally signed data prevents.

Before Django sends any of these sensitive data structures to users on the Internet, it signs them with a project's SECRET_KEY. When the data structures come back to fulfill an action, Django re-checks these sensitive data structures against the SECRET_KEY again. If there was any tampering on the data structures, the signature check fails and Django halts the process.

The only remote possibility a rogue user has to successfully pull an attack of this kind is if the SECRET_KEY is compromised -- since an attacker can potentially create an altered data structure that matches a project's SECRET_KEY. Therefore you should be careful about exposing your project's SECRET_KEY. If you suspect for any reason a project's SECRET_KEY has been compromised you should replace it immediately -- only a few ephemeral data structures (i.e. sessions, cookies) become invalid with this change, until users re-login again and the new SECRET_KEY is used to re-generate these data structures.

Define administrators for ADMINS and MANAGERS

Once a Django project is made accessible to end users, you'll want some way to receive notifications of important events related to security or other critical factors. Django has two sets of administrative groups defined in settings.py: ADMINS and MANAGERS. By default, both ADMINS and MANAGERS are empty. The values assigned to both variables need to be tuples, where the first value of the tuple is a name and the second part of the tuple is an email. Listing 5-2 shows a sample definition of ADMINS and MANAGERS.

Listing 5-2. Django ADMINS and MANAGERS definition

ADMINS = (('Webmaster','webmaster@coffeehouse.com'),('Administrator','admin@coffeehouse.com'))
MANAGERS = ADMINS

As you can see is listing 5-2, the ADMINS variable is assigned two tuples with different administrators. Next, you can see the ADMINS value is assigned to the MANAGERS variable. You can of course define different values for MANAGERS using the same syntax as ADMINS, but in this case I just gave both variables the same values for simplicity.

The purpose of having these two administrative groups in settings.py is for Django to send email notifications of project events. By default, these events are limited and happen under certain circumstances. After all, you don't want to send administrators 10 email notifications every minute 24/7.

By default, ADMINS are sent email notifications of errors associated with the django.request or django.security packages, if and only if DEBUG=False. This is a pretty narrow criteria, as it's intended to notify only the most serious errors -- for requests and security -- and only for production environments which is when DEBUG=False. For no other events or conditions are the ADMINS notified by email.

By default, MANAGERS are sent email notifications of broken links (i.e. HTTP 404 page requests), if and only if DEBUG=False and the Django middleware django.middleware.common.BrokenLinkEmailsMiddleware is enabled. Because HTTP 404 page requests aren't a serious problem, by default BrokenLinkEmailsMiddleware is disabled. This is an even narrower criteria than for ADMINS, because irrespective of a project being in development (DEBUG=True) or production (DEBUG=False) the BrokenLinkEmailsMiddleware class needs to be added to MIDDLEWARE variable in settings.py for MANAGERS to get notifications. For no other events or conditions are the MANAGERS notified by email.

Now that you know the purpose of ADMINS and MANAGERS, add users and emails as you see fit to your project. Remember you can always leverage the values in ADMINS and MANAGERS for other custom logic in a Django project (e.g. notify administrators of user sign ups).

Use dynamic absolute paths

There are some Django variables in settings.py that rely on directory locations, such is the case for STATIC_ROOT which defines a consolidation directory for a project's static files or the DIRS list of the TEMPLATES variable which defines the location of a project's templates, among other variables.

The problem with variables that rely on directory locations is that if you run the project on different servers or share it with other users, it can be difficult to keep track or reserve the same directories across a series of environments. To solve this issue you can define variables to dynamically determine the absolute paths of a project. Listing 5-3 illustrates a Django project directory structure, deployed to the /www/ system directory.

Listing 5-3. Django project structure deployed to /www/

+-/www/+
       |           
       +--STORE--+
                 |
                 +---manage.py
                 |
                 +---coffeestatic--+
                 |                 |
                 |                 +-(Consolidated static resources) 
                 |
                 +---coffeehouse--+
                                  |
                                  +-__init__.py
                                  +-settings.py
                                  +-urls.py
                                  +-wsgi.py
                                  |
                                  +---templates---+
                                                  +-app_base_template.html
                                                  +-app_header_template.html
                                                  +-app_footer_template.html

Typically a Django settings.py file would define the values for STATIC_ROOT and DIRS in TEMPLATES as illustrated in listing 5-4.

Listing 5-4. Django settings.py with absolute path values

# Other configuration variables omitted for brevity
STATIC_ROOT = '/www/STORE/coffeestatic/'

# Other configuration variables omitted for brevity
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': ['/www/STORE/coffeehouse/templates/',],
}
]

The issue with the setup in listing 5-4 is it will require editing if you deploy the Django application to a server where the /www/ directory isn't available (e.g. due to restrictions or a Windows OS where directories start with a leading letter C:/).

An easier approach illustrated in listing 5-5 is to define variables to dynamically determine the absolute paths of a project.

Listing 5-5. Django settings.py with dynamically determined absolute path

import os
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
PROJECT_DIR = os.path.dirname(os.path.abspath(__file__))

# Other configuration variables omitted for brevity
STATIC_ROOT = '%s/coffeestatic/' % (BASE_DIR)

# Other configuration variables omitted for brevity
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': ['%s/templates/'% (PROJECT_DIR),],
}
]

The variables defined at the top of listing 5-5 rely on the Python os module to dynamically determine the absolute system path relative to the settings.py file. The PROJECT_DIR=os.path.dirname(os.path.abspath(__file__)) statement gets translated into the /www/STORE/coffeehouse/ value, which is the absolute system directory of files like settings.py. And to access the parent of /www/STORE/coffeehouse/ you simply wrap the same statement with another call to os.path.dirname and define the BASE_DIR variable so it gets translated into the /www/STORE/ value.

The remaining statements in listing 5-5 use standard Python string substitution to use the PROJECT_DIR and BASE_DIR to set the absolute paths in the STATIC_ROOT and TEMPLATE_DIRS variables. In this manner you don't need to hard code the absolute paths for any Django configuration variable, the variables automatically adjust to any absolute directory irrespective of the application deployment directory.

Use multiple environments or configuration files for Django

In every Django project you'll eventually come to the realization that you have to split settings.py into multiple environments or files. This will be either because the values in settings.py need to change between development and production servers, there are multiple people working on the same project with different requirements (e.g. Windows and Linux) or you need to keep sensitive settings.py information (e.g. passwords) in a local file that's not shared with others.

In Django there is no best or standard way to split settings.py into multiple environments or files. In fact, there are many techniques and libraries to make a Django project run with a split settings.py file. Next, I'll present the three most popular options I've used in my projects. Depending on your needs you may feel more comfortable using one option over another or inclusively mixing two or all three of these techniques to achieve an end solution.

Option 1) Multiple environments in the same settings.py file with a control variable

The settings.py file is treated as an ordinary Python file, so there's no limitation to using Python libraries or conditionals to obtain certain behaviors. This means you can easily introduce a control variable based on a fixed value (e.g. server host name) to conditionally set up certain variable values.

For example, changing the DATABASES variable -- because passwords and the database name change between development and production -- changing the EMAIL_BACKEND variable -- since you don't need to send actual emails in development as you do in production -- or changing the CACHES variable -- since you don't need a cache to speed up performance in development as you need in production.

Listing 5-6 illustrates the setup of a control variable called DJANGO_HOST based on Python's socket module, the variable is then used to load different sets of Django variables based on a server's host name.

Listing 5-6 Django settings.py with control variable with host name to load different sets of variables.

# Import socket to read host name
import socket

# If the host name starts with 'live', DJANGO_HOST = "production"
if socket.gethostname().startswith('live'):
    DJANGO_HOST = "production"
# Else if host name starts with 'test', set DJANGO_HOST = "test"
elif socket.gethostname().startswith('test'): 
    DJANGO_HOST = "testing"
else:
# If host doesn't match, assume it's a development server, set DJANGO_HOST = "development"
    DJANGO_HOST = "development"

# Define general behavior variables for DJANGO_HOST and all others
if DJANGO_HOST == "production":
    DEBUG = False
    STATIC_URL = 'http://static.coffeehouse.com/'
else:
    DEBUG = True
    STATIC_URL = '/static/'

# Define DATABASES variable for DJANGO_HOST and all others
if DJANGO_HOST == "production":
   # Use mysql for live host
   DATABASES = {
    'default': {
        'NAME': 'housecoffee',
        'ENGINE': 'django.db.backends.mysql',
        'USER': 'coffee',
        'PASSWORD': 'secretpass'
    }
  }
else: 
   # Use sqlite for non live host
   DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'coffee.sqlite3'),
    }
  }

# Define EMAIL_BACKEND variable for DJANGO_HOST
if DJANGO_HOST == "production":
    # Output to SMTP server on DJANGO_HOST production
    EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
elif DJANGO_HOST == "testing":
    # Nullify output on DJANGO_HOST test
    EMAIL_BACKEND = 'django.core.mail.backends.dummy.EmailBackend'
else: 
    # Output to console for all others
    EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'

# Define CACHES variable for DJANGO_HOST production and all other hosts 
if DJANGO_HOST == "production":
   # Set cache
   CACHES = {
        'default': {
            'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
            'LOCATION': '127.0.0.1:11211',
            'TIMEOUT':'1800',
            }
        }
   CACHE_MIDDLEWARE_SECONDS = 1800
else: 
   # No cache for all other hosts
   pass

The first line in listing 5-6 imports the Python socket module to gain access to the host name. Next, a series of conditionals are declared using socket.gethostname() to determine the value of the control variable DJANGO_HOST. If the host name starts with the letters live the DJANGO_HOST variable is set to "production", if the host name starts with test then DJANGO_HOST is set to "testing" and if the host name starts with neither of the previous options then DJANGO_HOST is set to "development".

In this scenario, the string method startswith is used to determine how to set the control variable based on the host name. However, you can just as easily use any other Python library or even criteria (e.g. IP address) to set the control variable. In addition, since the control variable is based on a string, you can introduce as many configuration variations as needed. In this case we use three different variations to set settings.py variables -- "production","testing" and "development" -- but you could easily define five or a dozen variations if you require such an amount of different set ups.

Option 2) Multiple environment files using configparser

Another variation to split settings.py is to rely on Python's built-in configparser module. configparser allows Django to read configuration variables from files that use a data structure similar to the one used in Microsoft Windows INI files. Listing 5-7 illustrates a sample configparser file.

Listing 5-7. Python configparser sample file production.cfg.

[general]
DEBUG: false
STATIC_URL: http://static.coffeehouse.com/
[databases]
NAME: housecoffee
ENGINE: django.db.backends.mysql 
USER: coffee
PASSWORD: secretpass 
[security]
SECRET_KEY: %%ea)cjy@v9(7!b(20gl+4-6iur28dy=tc4f$-zbm-v=!t

As you can see in Listing 5-7, the format for a configparser file is structured in various sections declared between brackets (e.g. [general], [databases]) and below each section are the different keys and values. The variables in Listing 5-7 represents a production environment placed in a file named production.cfg. I chose the .cfg extension for this file, but you can use the .config or .ini extensions if you like, the extension is irrelevant to Python, the only thing that matters is the data format in the file itself.

Similar to the contents in production.cfg, you can create other files with different variables for other environments (e.g.testing.cfg, development.cfg). Once you have the configparser file or files, then you can import them into a Django settings.py. Listing 5-8 shows a sample settings.py that uses values from a configparser file.

Listing 5-8. Django settings.py with configparser import.

import os

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
PROJECT_DIR = os.path.dirname(os.path.abspath(__file__))

# Access configparser to load variable values
from django.utils.six.moves import configparser
config = configparser.SafeConfigParser(allow_no_value=True)

# Import socket to read host name
import socket

# If the host name starts with 'live', load configparser from "production.cfg"
if socket.gethostname().startswith('live'):
    config.read('%s/production.cfg' % (PROJECT_DIR))
# Else if host name starts with 'test', load configparser from "testing.cfg"
elif socket.gethostname().startswith('test'): 
    config.read('%s/testing.cfg' % (PROJECT_DIR))
else:
# If host doesn't match, assume it's a development server, load configparser from "development.cfg"
    config.read('%s/development.cfg' % (PROJECT_DIR))

DEBUG = config.get('general', 'DEBUG')
STATIC_URL = config.get('general', 'STATIC_URL')
DATABASES = {
    'default': {
        'NAME': config.get('databases', 'NAME'),
        'ENGINE': config.get('databases', 'ENGINE'),
        'USER': config.get('databases', 'USER'),
        'PASSWORD': config.get('databases', 'PASSWORD')
    }
  }

SECRET_KEY = config.get('security', 'SECRET_KEY')

As you can see in Listing 5-8, configparser is loaded into Django via django.utils.six.moves, which is a utility to allow cross-imports between Python 2 and Python 3. In Python 2 the configparser package is actually named ConfigParser, but this utility allows us to use the same import statement using either Python 2 and Python 3. After the import, we use the SafeConfigParser class with the argument allow_no_value=True to allow processing of empty values in configparser keys.

Then we rely on the same prior technique using Python's socket module to gain access to the host name and determine which configparser file to load. The configparser file is loaded using the read method of the SafeConfigParser instance. At this juncture all configparser variables are loaded and ready for access. The remainder of listing 5-8 shows a series of standard Django settings.py variables that are assigned their value using the get method of the SafeConfigParser instance, where the first argument is the configparser section and the second argument is the key variable.

So there you have another option on how to split the variables in settings.py into multiple environments. Like I mentioned at the start, there's no best or standard way of doing this. Some people like configparser better because it splits values into separate files and avoids the many conditionals of option 1, but other people can hate configparser because of the need to deal with the special syntax and separate files. Choose whatever feels best for your project.

Option 3) Multiple settings.py files with different names for each environment

Finally, another option to split Django variables into multiple environments is to create multiple settings.py files with different names. By default, Django looks for configuration variables in the settings.py file in a project's base directory.

However, it's possible to tell Django to load a configuration file with a different name. Django uses the operating system(OS) variable DJANGO_SETTINGS_MODULE for this purpose. By default, Django sets this OS variable to <project_name>.settings in the manage.py file located in the base directory of any Django project. And since the manage.py file is used to bootstrap Django applications, the DJANGO_SETTINGS_MODULE value in this file guarantees configuration variables are always loaded from the settings.py file inside the <project_name> sub-directory.

So let's suppose you create different settings.py files for a Django application -- placed in the same directory as settings.py -- named production.py, testing.py and development.py. You have two options to load these different files.

One option is to change the DJANGO_SETTINGS_MODULE definition in a project's manage.py file to the file with the desired configuration (e.g. os.environ.setdefault("DJANGO_SETTINGS_MODULE", "coffeehouse.production") to load the production.py configuration file). However, hard coding this value is inflexible because you would need to constantly change the value in manage.py based on the desired configuration. Here you could use a control variable in manage.py to dynamically determine the DJANGO_SETTINGS_MODULE value based on a host name -- similar to the process described in the previous option 1 for settings.py.

Another possibility to set DJANGO_SETTINGS_MODULE without altering manage.py is to define DJANGO_SETTINGS_MODULE at the OS level so it overrides the definition in manage.py. Listing 5-9 illustrates how to set the DJANGO_SETTINGS_MODULE variable on a Linux/Unix OS so that application variables in the testing.py file are used instead of the settings.py file.

Listing 5-9. Override DJANGO_SETTINGS_MODULE to load application variables from a file called testing.py and not the default settings.py

$ export DJANGO_SETTINGS_MODULE=coffeehouse.load_testing
$ python manage.py runserver
Validating models...
0 errors found
Django version 1.11, using settings 'coffeehouse.load_testing'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

In listing 5-9 we use the standard Linux/Unix syntax export variable_name=variable_value to set an environment variable. Once this is done, notice the Django application that uses the development server displays the start-up message "using settings 'coffeehouse.load_testing'".

If you plan to override the DJANGO_SETTINGS_MODULE at the OS level to load different Django application variables, be aware that by default OS variables aren't permanent or inherited. This means you may need to define the DJANGO_SETTINGS_MODULE for every shell from which you start Django and also define it as a local variable for run-time environments (e.g. Apache).

1. https://docs.djangoproject.com/en/1.11/topics/signing/     ↑