You're having trouble determining what's causing a certain behavior (e.g. logical, performance or an outright error) in a Django application that isn't what you expect.
Use Django's built in shell
python manage.py shell to run project logic from the command line. Install the Django debug toolbar with
pip install django-debug-toolbar to obtain detailed information (e.g. Django settings, HTTP headers, SQL queries, cache and logging) about every Django application page displayed through a sliding sidebar. Use the Django debug toolbar's
debugsqlshell tool to obtain the backing SQL for any Django model operation.
Use Django pdb to inspect the step-by-step execution of a Django application. Install Django extensions to use the
runserver_plus tool that allows interactive debugging on the browser, as well as the
runprofileserver tool that allows the creation of a cProfile for a Django application page to analyze how often and for how long its parts are executed.
The first steps to correct unexpected behavior in an application are generally to review what you think are the problematic sections of source code and logs. Sometimes though these reviews are fruitless, either because an application has grown in complexity or the unexpected behavior is originating in a not so obvious location. Under these circumstances, the next step is to start a debugging process with the help of tools to make it easier to detect and fix the problem. In the upcoming sections, I'll describe some of the most popular tools to debug Django applications.
python manage.py shell
Just like Python's CLI ('Command Line Interface') shell where you can evaluate expressions (e.g.
mystring = 'django'), Django offers its own shell version through the
python manage.py shell command -- where
manage.py is the top level file in every Django project. Django's shell is helpful because it loads a project's dependencies and apps automatically, so you're able to evaluate expressions related to your Django project (e.g. queries, methods) without having to go through a tedious set up process.
Listing 1 illustrates a series of sample expressions run from Django's shell.
The first snippet in listing 1 uses the
from import syntax to gain access to a Django project's model classes, after which you can invoke queries on the models to validate results. Note there's no need to import additional libraries or define database connections, all dependencies and configurations are loaded from the Django project itself. The second snippet in listing 2 uses Django's test library to simulate a client/browser request to the
/stores/5/ URLs, after which you can inspect the content response or the HTTP status code (e.g. 404 Not Found). Here again note there's no need to start a web server or open a browser, you can quickly validate a Django project's URLs and its responses.
The Django debug toolbar offers a more visual experience to debug Django applications compared to the Django shell. The Django debug toolbar offers per page information through a sliding sidebar related to things like resource usage (i.e. time), Django settings, HTTP headers, SQL queries, cache and logging, among other things. Figures 1 and 2 illustrates a collapsed and non-collapsed screenshot of the Django debug toolbar.
As you can see in figure 1, the Django debug toolbar is accessible through a small tab in the top right hand side of every Django project page. Figure 2 illustrates a collapsed version of the Django debug toolbar where you can see its various sections, clicking on any of the sections further brings up a pop-window with detailed information about each section.
You can install the Django debug toolbar with the
pip install django-debug-toolbar command. Once you install the
django-debug-toolbar, you'll also need to add the
debug_toolbar line to the
INSTALLED_APPS variable in
settings.py so Django enables the toolbar. Note the Django debug toolbar only works when
In addition to the UI toolbar, the Django debug toolbar also offers the
debugsqlshell utility. This utility works like Django's standard shell, but it outputs the backing SQL associated with any Django model operation, as illustrated in listing 2.
As you can see in listing 2, the
debugsqlshell utility is made available through the
manage.py command -- just like Django's built-in shell -- and after you run a Django model operation it also outputs the SQL query for the operation.
For detailed information on customizing the Django debug toolbar see its official documentation at http://django-debug-toolbar.readthedocs.org/.
pdb -- short for "Python Debugger" -- is a Python core package designed to interactively debug source code. With Python pdb you can inspect the line by line execution of any Python application. To simplify the process of Python pdb in the context of Django applications (e.g. debug request methods) you can use the Django pdb package.
To install Python pdb run
pip install django-pdb and then add
django_pdb to the
INSTALLED_APPS variable in
settings.py in the first position -- the position is important so other Django apps don't override Django pdb's behaviors (e.g. override runserver and test commands). Be aware the Django pdb package only works when
There are various way to run pdb with Django, the easiest is to append the
?pdb parameter to any Django URL you want to analyze with pdb. For example, listing 3 shows a debugging sequence for the
You can see listing 3 starts with Django's built-in web server and immediately receives and dispatches a response for the regular URL
/drinks/mocha/. Up to this point everything is standard, however, notice the next request to the URL
/drinks/mocha/?pdb and the verbose output that follows. The verbose output tells you where the request enters the application, including arguments, as well as the initial entry point into Django's core framework in the
After the initial verbose output, the execution stops at the first
(Pdb) instance. At this juncture you've hit a breakpoint, so the console running
runserver and the requesting client (i.e. browser) freeze until you provide additional input on the console. In listing 3 you can see the letter
n for next is introduced and the execution moves forward to another line, after which you'll be presented with another
(Pdb) prompt or breakpoint. At the point, you can just press the
Enter key to re-invoke the previous command (i.e.
n) and move forward.
If you want to advance without hitting another breakpoint you can type
c for continue so the execution continues normally, without pausing again. As you can see, the power of using pdb with Django lies in the fact that you can walk through the execution cycle of any section in a very granular way, in addition to having the ability to analyze and set variables interactively. Table 1 describes the most basic commands related to pdb.
|Re-executes the previous command|
|Moves execution to the next breakpoint|
|Continues execution with no more breakpoints|
|Quits the execution immediatly|
|Displays a list of source code at the current breakpoint, 11 lines worth: the breakpoint line, 5 lines prior and 5 lines after. Helpful to provide context.|
|Enters a sub-routine. In a non-method related breakpoint, |
|Breaks out of a sub-routine. Used after |
In addition to appending the
?pdb parameter to a URL to enter pdb in a Django application, there are two more alternatives. You can append the
--pdb flag to
runserver to enter pdb on every request made to the application (e.g.
python manage.py runserver --pdb). And you can also use the
--pm flag to enter pdb only when an exception is raised in a view (e.g.
python manage.py runserver --pm).
For additional information on pdb itself consult the official Python documentation at https://docs.python.org/2/library/pdb.html. And for additional information on Django pdb, consult the project's documentation at https://github.com/tomchristie/django-pdb.
Django extensions is a collection of tools designed for Django projects. As its name implies, it offers extensions for a wide array of areas where Django's standard tools level off in functionality. For debugging purposes, Django extensions offers two particular tools that I believe are the most important to explore:
To use Django extensions you'll first need to install it with
pip install django-extensions and then add
settings.py. Once you setup Django extensions, its various tools become available through the
python manage.py command just like Django's standard tools.
The Django extensions
runserver_plus command offers interactive and enhanced debugging for Django projects. To use
runserver_plus you'll first need to install the Werkzeug utility --
pip install Werkzeug. Once you install Werkzeug, simply start a Django application with
python manage.py runserver_plus instead of Django's standard
python manage.py runserver. At first glance the
runserver_plus command works just like Django's
runserver, however, if you happen to hit an exception you'll see error pages like the ones in figures 3 and 4.
In figure 3 you can see a different Django exception page vs. Django's default exception page. This different layout is generated by Werkzeug, but the layout itself isn't what's interesting about this approach, if you hover over any section of the stack trace you can start an interactive debugging session, as illustrated in figure 4. This is a much simpler and powerful debugging approach because it can be done directly in a browser.
Another powerful Django extensions tool is
runprofileserver which can create a Python cProfile for a Django application page. A Python cProfile provides a set of statistics that describes how often and for how long the various parts of a program are executed, which can helpful to determine solutions for slow loading and resource intensive Django application pages.
The first thing you'll need to do to use
runprofileserver is create a folder to hold the profile files (e.g.
mkdir /tmp/django-coffeehouse-profiles/). Next, simply start a Django application with
python manage.py runprofileserver --use-cprofile --prof-path=/tmp/django-coffeehouse-profiles/ instead of Django's standard
python manage.py server -- note the
--prof-path flag value points to the directory that will hold the profile files.
Open up a browser, head over to the Django application and navigate through it. If you open the folder that holds the profile files, you'll see files like
stores.2.000050ms.1459139470.prof, where each file represents a cProfile for each page hit.
Although it would go beyond the scope of the recipe to dive into cProfile analysis, not to mention there are many tools available for this purposes, if you wan't a quick and easy tool to open Python cProfile files, I would suggest SnakeViz. Just do
pip install snakeviz and then run
snakeviz <file_name>. Once you run
snakeviz on a file, you'll see Python cProfile details like those illustrated in figures 5 and 6.
As I mentioned at the start, Django extensions provides many tools in addition to
runprofileserver which I believe are the most appropriate for debugging tasks. Nevertheless, I would recommend you review the Django extensions documentation available at https://django-extensions.readthedocs.org/en/latest/ to explore other tools that might be use in your own projects (e.g. the
show_urls tool displays a Django project's url routes and the
graph_models tools generates graphs for a Django project's models).