Customize Django template configuration

Problem

You want to customize Django's template configuration for things like template locations, error handling and debug output.

Solution

You configure Django templates in the TEMPLATES variable of the settings.py file. To configure where Django looks for templates you add directories to the DIRS list of the TEMPLATES variable. In addition, you can also add templates to Django apps in a special sub-folder named templates, this ability is enabled by the APP_DIRS option of the TEMPLATES variable.

You can access common variables in all Django templates (e.g. HTTP request, user details) thanks to context processors which are configured in the TEMPLATES variable as part of OPTIONS in the context_processors field. In addition, through the string_if_invalid field -- also in OPTIONS -- you can configure Django to generate a custom message or throw an error when invalid variables are declared in templates. Also in OPTIONS you can activate or deactivate autoescaping, template character encoding and debugging information with the autoescape, file_charset and debug fields, respectively. In addition, OPTIONS also supports the libraries and builtins fields to register Django template tag/filter modules.

How it works

By default, Django templates are enabled on all Django projects due to the TEMPLATES variable in settings.py. Listing 1 illustrates the default TEMPLATES value in Django projects.

Listing 1 - Default Django template configuration in settings.py

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

The BACKEND variable indicates the project uses Django templates. The DIRS and APP_DIRS variables tell Django where to locate Django templates and are explained in the next section. The context_processors field inside OPTIONS tells Django which context processors to enable across Django templates, for details on what these context processors do see the recipe Use data provided by default Django context processors on Django templates and to write your own context processors see the recipe Set up data in custom Django context processors to access on all Django templates.

Template search paths

Django determines where to look for Django templates based on the values in the DIRS and APP_DIRS variables. As you can see in listing 1, Django defaults to an empty DIRS value and sets the APP_DIRS variable to True. This means that by default, Django only looks for templates in all installed Django app sub-folders named templates -- if you've never heard of the Django app concept, I advise you to look over the recipe Set up content, understand Django urls, templates and apps.

So all the APP_DIRS variable does is permit the look-up of templates inside this special app sub-directory. This is helpful if you wish to contain an app's templates to the app's structure, but be aware the template search path is not aware of the app's namespace. For example, if you have two apps that both rely on a template named index.html -- as illustrated in listing 2 -- and both app's have a method in views.py that returns control to the index.html template (e.g. render(request,'index.html')), both app's will use the index.html from the top-most declared app in INSTALLED_APPS, so in essence one app won't use the expected index.html.

Listing 2 - Django apps with templates dirs with potential conflict and namespace qualification
 
# Templates directly under templates folder can cause loading conflicts
+---+-<PROJECT_DIR_project_name_conflict>
    |
    +-__init__.py
    +-settings.py
    +-urls.py
    +-wsgi.py
    |
    +-about(app)-+
    |            +-__init__.py
    |            +-models.py
    |            +-tests.py
    |            +-views.py
    |            +-templates-+
    |                        |
    |                        +-index.html
    +-stores(app)-+
                 +-__init__.py
                 +-models.py
                 +-tests.py
                 +-views.py
                 +-templates-+
                             |
                             +-index.html

# Templates classified with additional namespace avoid loading conflicts
+---+-<PROJECT_DIR_project_name_namespace>
    |
    +-__init__.py
    +-settings.py
    +-urls.py
    +-wsgi.py
    |
    +-about(app)-+
    |            +-__init__.py
    |            +-models.py
    |            +-tests.py
    |            +-views.py
    |            +-templates-+
    |                        |
    |                        +-about-+
    |                                |
    |                                +-index.html
    +-stores(app)-+
                 +-__init__.py
                 +-models.py
                 +-tests.py
                 +-views.py
                 +-templates-+
                             |
                             +-stores-+
                                      |
                                      +-index.html

To fix this potential conflict, the recommended practice is to add an additional sub-folder to act as a namespace inside each templates directory as illustrated in the second set of folders in listing 2. In this manner, you can then re-direct control to a template using this additional namespace sub-folder to avoid any ambiguity. So to send control to the about/index.html template you would declare render(request,'about/index.html') and to send control to the stores/index.html you would declare render(request,'stores/index.html').

If you wish to disallow this behavior of allowing templates to be loaded from these internal app sub-folders, you can do so by setting APP_DIRS to FALSE.

A more common approach for Django templates is to have a single folder or various folders -- that live outside app structures -- to hold Django templates. In order for Django to find such templates you use the DIRS variable as illustrated in listing 3.

Listing 3 - DIRS definition with relative path in settings.py

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
PROJECT_DIR = os.path.dirname(os.path.abspath(__file__))

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': ['%s/templates/' % (PROJECT_DIR),'%s/dev_templates/' % (PROJECT_DIR),],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

As you can see in listing 3, you can declare various directories inside the DIRS variable. Django first looks for a matching template in the first DIRS value, then moves on to the second and so on, until it either finds a matching template or throws a TemplateDoesNotExist error. Also note the DIRS values in listing 3 relies on a path determined by the PROJECT_DIR variable, this is helpful when deploying a Django project across different machines, because the path is relative to the top-level Django project directory (i.e. where the settings.py and main urls.py file are) and adjusts dynamically irrespective of where a Django project is installed (e.g. /var/www/, /opt/website, C://website/).

Invalid template variables

By default, Django templates do not throw an error when they contain invalid variables. This is due in most part to design choices associated with the Django admin which also uses Django templates. While this is not a major issue in most cases, it can be frustrating for debugging tasks as Django doesn't inform you of misspelled or undefined variables. For example, you could type {{datee}} instead of {{date}} and Django ignores this by outputting an empty string '', you could also forget to pass a variable value to a template in the view method and Django also silently outputs an empty string '' even though you may have it defined in the template.

To enable Django to inform you when it encounters an invalid variable in a Django template you need to use the string_if_invalid option. The first alternative is to output a visible string -- instead of the default empty string '' -- a configuration that's shown in listing 4.

Listing 4 - Output warning message for invalid template variables with string_if_invalid

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
PROJECT_DIR = os.path.dirname(os.path.abspath(__file__))

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': ['%s/templates/' % (PROJECT_DIR),'%s/dev_templates/' % (PROJECT_DIR),],
        'APP_DIRS': True,
        'OPTIONS': {
            'string_if_invalid': "**** WARNING INVALID VARIABLE %s ****",
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

As you can see in listing 4, string_if_invalid is assigned the string "**** WARNING INVALID VARIABLE %s ****". When Django encounters an invalid variable, it replaces its occurrence with this string, where the %s variable gets substituted for the invalid variable name. In this manner, it's easier to locate where and what variables are invalid.

Another alternative to the string_if_invalid option is to perform more complex logic when an invalid variable is encountered. For example, you could raise an error so the template fails to render in case an invalid variable is found, a configuration that's presented in listing 5.

Listing 5 - Error generation for invalid template variables with string_if_invalid

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
PROJECT_DIR = os.path.dirname(os.path.abspath(__file__))

class InvalidTemplateVariable(str):
    def __mod__(self,other):
        from django.template.base import TemplateSyntaxError
        raise TemplateSyntaxError("Invalid variable : '%s'" % other)

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': ['%s/templates/' % (PROJECT_DIR),'%s/dev_templates/' % (PROJECT_DIR),],
        'APP_DIRS': True,
        'OPTIONS': {
            'string_if_invalid': InvalidTemplateVariable("%s"),
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

As you can see in listing 5, string_if_invalid is assigned the InvalidTemplateVariable class that uses the %s input variable, which represents the invalid variable name -- just like the previous example in listing 4. The InvalidTemplateVariable class is interesting because it inherits its behavior from the str (string) class and provides a __mod__ (modulo) magic method implementation. While the __mod__ (modulo) magic method is proper of number operations, in this case it's useful because the passed in string uses the % (modulo) symbol, which makes the __mod__ method run. Inside the __mod__ method we just raise the TemplateSyntaxError error with the invalid variable name to halt the execution of the template.

Note The Django admin might get mangled or broken with a custom string_if_invalid

The Django admin templates in particular rely heavily on the default string_if_invalid for outputting empty strings '', due to the level of complexity in certain displays. In fact, this string_if_invalid default behavior is often considered a 'feature', as much as it's considered a 'bug' or 'annoyance'.

Therefore if you use one of the approaches in listing 4 or listing 5 to override string_if_invalid, be aware you will most likely mangle or brake Django admin pages. If you rely on the Django admin, you should only use these techniques to debug a project's templates.

Debug output

When you run a Django project with the top-level DEBUG=True setting and an error occurs, Django templates output a very detailed page to make the debugging process easier -- the section Switch DEBUG to False of the recipe 'Django settings.py for the real world: Security, absolute paths and multiple environments/files' has more details on the behavior of the DEBUG variable.

By default, Django templates reuse the top-level DEBUG variable value to configure template debug activity. Behind the scenes, this configuration is set through the debug field inside 'OPTIONS'. Figure 1 illustrate what an error page looks when DEBUG=True.

Django error page when <code>DEBUG=True</code> automatically sets template <code>OPTION</code> <code>'debug':True</code>
Figure 1.- Django error page when DEBUG=True automatically sets template OPTION to 'debug':True

As you can see in Figure 1, Django prints the location of the template, as well as a snippet of the template itself to make the fix easier. This template information is generated due to the 'debug':True option, which is set based on the top-level DEBUG variable. However, you can explicitly set the debug option to False as illustrated in listing 6, in which case the error page would result without any template details and just the traceback information, as illustrated in Figure 2.

Listing 6 - Option with debug equals False omits template details

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
PROJECT_DIR = os.path.dirname(os.path.abspath(__file__))

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': ['%s/templates/' % (PROJECT_DIR),'%s/dev_templates/' % (PROJECT_DIR),],
        'APP_DIRS': True,
        'OPTIONS': {
            'debug':False,
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]
Django log-in form
Figure 2.- Django error page when DEBUG=True and template OPTION 'debug':False

Autoescape

By default, Django templates use the almost universal security practice of all web frameworks to autoescape dynamically generated content (i.e. output generated by variables). Autoescaping converts characters that can potentially mangle a user interface or produce dangerous outcomes into safe representations. Specifically, autoescaping in Django templates converts < to &lt;, > to &gt;, '(single quote) to &#39;, " (double quote) to &quot; and & is converted to &amp;.

Since data in variables can come from a database or be user generated, there's no assurance it won't advertantly or inadvertently contain sequences like an open-ended HTML tag <b> that can bolden an entire page or rogue code like a <script> tag with malicious logic to run on a user's browser. Autoescaping prevents these outcomes, which is why Django templates enable it by default.

However, if you want to avoid autoescaping behavior on all Django templates -- and knownigly render < as <, > as >, etc... -- you can use the autoescape field in 'OPTIONS' as illustrated in listing 7.

Listing 7 - Option with autoescape equals False omits autoescaping on all Django templates

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
PROJECT_DIR = os.path.dirname(os.path.abspath(__file__))

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': ['%s/templates/' % (PROJECT_DIR),'%s/dev_templates/' % (PROJECT_DIR),],
        'APP_DIRS': True,
        'OPTIONS': {
            'autoescape':False,
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

It's worth mentioning that an alternative to disabling autoescaping on every Django template -- as it's done in listing 7 -- is to selectively disable autoescaping. You can either use the {% autoescape off %} tag to disable autoescaping on a section of a Django template or the safe filter to disable autoescaping on a single Django template variable.

If you do decide to disable autoescaping on all Django templates as illustrated in listing 7 -- which I frankly wouldn't recommend because of the potential security risk -- you can also granularly enable autoescaping again if required. You can either use the {% autoescape on %} tag to enable autoescaping on a section of a Django template or the escape filter to escape a single Django template variable.

File charset

Files used in Python projects often declare an encoding value at the top (e.g. # -*- coding: utf-8 -*-) based on the Python PEP-263 specification, which ensures the characters in the file are interpreted correctly. In Django templates, you don't define the underlying file's encoding in this manner, but instead do it inside a project's settings.py file.

There are two ways to declare the encoding character for Django templates: explicitly as part of the file_charset field in OPTIONS inside the TEMPLATES variable or via the top level FILE_CHARSET variable in settings.py. The explicit declaration in file_charset within OPTIONS takes precedence over the FILE_CHARSET assignment, but the value of file_charset defaults to FILE_CHARSET which in itself defaults to utf-8 (Unicode) encoding.

So by default, Django template encoding is assigned to utf-8 or Unicode which is one of the most widely used encodings in software. Nevertheless, in the event you decide to incorporate data into Django templates that isn't utf-8 compatible (e.g. Spanish vowels with accents like á or é encoded as ISO-8859-1 or Kanji characters like 漢 or 字 encoded as JIS) you must define the FILE_CHARSET value in a project's settings.py file -- or directly in the file_charset field in OPTIONS inside the TEMPLATES -- so Django template data is interpreted correctly.

Django templates can be assigned any encoding value from Python's standard encoding values.

Register Django template tag/filter modules

Django templates have access to a series of built-in tags and built-in filters that don't require any setup steps. However, if you plan to use a third-party template tag/filter module or write your own template tag/filter module, then you need to setup access on each Django template with the {% load %} tag (e.g. {% load really_useful_tags_and_filters %}), a process that can get tiresome if you need to access a particular tag/filter on dozens or hundreds of templates.

To automatically gain access to third-party template tags/filters or your own template tags/filters as if they were built-in tags/filters (i.e. without requiring the {% load %} tag), you can use the builtins field in OPTIONS as illustrated in listing 8.

Listing 8 - Option with builtins to gain automatic access to tags/filters on all templates

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
PROJECT_DIR = os.path.dirname(os.path.abspath(__file__))

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': ['%s/templates/' % (PROJECT_DIR),'%s/dev_templates/' % (PROJECT_DIR),],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
	    'builtins': [
                 'coffeehouse.builtins',
                 'thirdpartyapp.customtags.really_useful_tags_and_filters',
            ],
        },
    },
]

As you can see in listing 8, the builtins field accepts a list of modules that includes tags/filters for built-in treatment. In this case, coffeehouse.builtins represents a builtins.py file -- that includes the custom tags/filters -- under a project named coffeehouse. And the thirdpartyapp.customtags.really_useful_tags_and_filters is a third-party package with tags/filters that we also want to access in Django templates without the need to use the {% load %} tag.

Another default behavior of third-party template tag/filter modules and custom template tag/filter modules is they require to use their original label/name for reference, while the latter also require to be placed inside a folder named templatetags in a registered Django app. These two default behaviors can be overriden with the libraries field in OPTIONS as illustrated in listing 9.

Listing 9 - Option with libraries to register tags/filters with alternative label/name and under any project directory

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
PROJECT_DIR = os.path.dirname(os.path.abspath(__file__))

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': ['%s/templates/' % (PROJECT_DIR),'%s/dev_templates/' % (PROJECT_DIR),],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
	    'libraries': {
                 'coffeehouse_tags': 'coffeehouse.tags_filters.common',
            },
        },
    },
]

The libraries statement in listing 9 'coffeehouse_tags': 'coffeehouse.tags_filters.common' tells Django to load the common.py file -- that includes the custom tags/filters -- from the tags_filters folder in the coffeehouse project and make it accesible to templates through the coffeehouse_tags reference (e.g. {% load coffeehouse_tags %}). With the approach in listing 9, you can place custom tag/filter modules anywhere in a Django project, as well as assign custom tag/filter modules -- or third-party tag/filter modules -- an alternative reference value than their original label/name.