Set up Django models and understand the migrations workflow

Problem

You want to set up Django models to interact with a database. You also want to understand how the migrations workflow associated with Django models works.

Solution

Create Python classes that inherit their behavior from Django's models.Model class and define class fields from Django's models package. Next, place these classes or Django models in a Django app's models.py file. Ensure all Django apps where you added Django models (i.e. to their models.py file) are declared as INSTALLED_APPS in settings.py.

Next, run makemigrations command to generate migration files on the modifications you made to models.py. You can optionally preview the actual SQL statements for a migration before it's applied to a database with the sqlmigrate <app_name> <migration_name> command. Finally, apply the migration to the database with the migrate command.

How it works

The typical and easiest way for Django applications to interact with data in a database is through Django models. A Django model is an object orientated Python class that represents the characteristics of an entity. For example, an entity could be a person, a company, a product or some other concept used by an application.

Because Django's primary storage technology is relational (i.e. out-of-the-box it can connect to SQLite, Postgres, MySQL or Oracle) a Django model is designed to map directly to a relational database table. This means instances of a Django model are stored as rows of a relational table by the same name as the model. For example, if you create a Django model called Store, Django attempts database CRUD (Create, Read, Update and Delete) operations on a table called <app_name>_store, where each of the model's Store fields (e.g. name, address, city, state) maps to a table column.

Note You already need to have a connection to a database

Django relies on the use of a database for some of its internal workings (e.g. user sessions), so this recipe assumes you've already set up a database for a Django project. If you haven't already set up a database, see the Set up a database for a Django project recipe for more details.

Setup Django models

Django models need to be stored in models.py files located inside Django apps. As soon as you create a Django app, an empty models.py file is added to the app for future use. If you're unfamiliar with the term Django app, see the prior recipe 'Set up content, understand Django urls, templates and apps'.

As mentioned at the start, Django models are object orientated Python classes, so if you're familiar with Python classes you already have a head start. Listing 1 illustrates a sample Django model definition.

Listing 1 - Django model class definition in models.py

from __future__ import unicode_literals
from django.utils.encoding import python_2_unicode_compatible

from django.db import models

@python_2_unicode_compatible
class Store(models.Model):
    #id = models.AutoField(primary_key=True)# Added by default, not required explicitly 
    name = models.CharField(max_length=30)    
    address = models.CharField(max_length=30)
    city = models.CharField(max_length=30)
    state = models.CharField(max_length=2)
    def __str__(self):
        return "%s (%s,%s)" % (self.name, self.city, self.state))

The first two lines in listing 1 import the functionality required to run Python classes in Django using both Python 2 and Python 3. If your Django project will just run on Python 3, you can ommit these import statements. The third line in listing 1 imports the django.db.models package which is necessary to access Django functionality in the class definition that will serve as the Django model.

Next, you can see the class Store(models.Model) statement. The @python_2_unicode_compatible annotation is required to run the class on Python 2, but if you just use Python 3 you can ommit this annotation. The class definition is standard Python class syntax, where the models.Model input indicates the class is to inherit the behavior from Django's models.Model class.

After the main class definition in listing 1, you can see four fields with the models.CharField data type which qualifies the fields as character strings. Further restricting the acceptable values for each field is the max_length argument for models.CharField (e.g. max_length=30 indicates the maximum length for the character field is 30 characters).

For the moment don't worry too much about the field definitions that use models.CharField. There are many other types of Django data types in the models package and different arguments to restrict Django model fields to practically any data type or qualifications you need, I'll describe many of these options in the next recipe Django model data types: Behaviors, options and validations.

In addition, notice the Django model in listing 1 has the id field which is automatically added to all Django models (i.e. you don't need to set it explicitly). The id field is a Django AutoField data type, which behind the scenes creates an integer database column that increments automatically. For example, when you create the first Store record the id field is set to 1 by the database and for the second Store record the database sets the id field to 2 and so on.

The intent of the id field is to make record queries easier and more efficient, because it assigns a unique number to a record that's used as a reference, which is also used as a database table's primary key and as an index to speed up record access. While you can override various behaviors of this default id field (e.g. change the field name), I'll leave the details of this for later recipes. See the DDL generated by AutoField to observe the actual SQL generated by AutoField, the Django data type primary_key option to create a custom primary key and the Django model save() method for details on the practical aspects of the id field.

Finally, in listing 1 you can see the class method definition for __str__ which is a standard Python method -- part of what are called 'magic methods' -- that are helpful when attempting to view instances of Django models. The use of the __str__ method will become more obvious in future recipes like when you Set up and customize Django models in the Django admin site. But for the moment, suffice to say an instance of the Store model would be represented as a string of its name, city and state fields, as described in the method definition.

Django model definitions like the one listing 1 even when placed in an app's models.py file still aren't discoverable by Django. In order for Django to discover model definitions in models.py files, it's necessary to declare the corresponding app in the INSTALLED_APPS variable in settings.py. Listing 2 illustrates an INSTALLED_APPS definition to discover Django models in the coffeehouse.stores app.

Listing 2 - Add app to INSTALLED_APPS in Django settings.py to detect models.py definitions

INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'coffeehouse.stores',
)

As you can see in listing 2, in addition to the various default apps declared in INSTALLED_APPS we add the coffeehouse.stores app at the end. This effectively tells Django to inspect the models.py file in the coffeehouse.stores app to take into account any Django model definitions in it.

After the previous steps, an app's Django model definitions in models.py are ready for use. In the next section, I'll describe the workflow associated with Django models and models.py files.

Workflow for Django models

Let's start with an illustration of how the workflow for Django models works to make it easier to grasp the overall process. Figure 1 shows such a workflow.

Figure 1 - Django Workflow for models with migrations
Django Workflow for models with migrations

Illustrated on the far left side of figure 1, the workflow starts when you add or modify Django models on any models.py file. Once you deem the changes made to a models.py file are considerable or want them reflected on a database, you need to create a migration file. Migration files provide a step-by-step snapshot of the changes made to a models.py file, whether you add, remove or modify content from it.

In order to create migration files you use the makemigrations command. When you run this command, Django scans models.py for changes to create a new migration file. This process functions like a version control system, where a migration file reflects changes made to models.py from a prior migration file and the entire series of migration files tells the whole evolution of a models.py file.

As you can see in Figure 1, migration files are stored in a /migrations/ sub-directory inside an app, alongside the models.py file they track. And by default, migration files use the naming convention <number_shortdescription> so it's easy to track in what order they were created and what it's they contain.

Next, lets run makemigrations on the Django model you created in the last section. Listing 3 illustrates this sequence and adds the stores argument to limit the process to the stores app -- if you run makemigrations without any argument, Django inspects the models.py for every app.

Listing 3 - Django makemigrations command to create migration file for changes made to models.py

[user@coffeehouse ~]$ python manage.py makemigrations stores
Migrations for 'stores':
  0001_initial.py:
    - Create model Store

After running makemigrations stores in listing 3, you can see the migration file 0001_initial.py is created. The file is given this name because it's the first migration for a previously empty models.py. Future changes to the models.py would generate migration files named 0002...,0003...,etc.

Turning our attention back the the workflow in figure 1, migration files by themselves are just a first step in the workflow for Django models. Next, you can either preview or apply these migration files so the models become part of a database.

To preview the actual SQL statements for a migration before it's applied to a database you run the sqlmigrate <app_name> <migration_name> command. Listing 4 illustrates the sqlmigrate sequence for the migration file from the last step.

Listing 4 - Django sqlmigrate command to preview SQL generated by migration file

[user@coffeehouse ~]$ python manage.py sqlmigrate stores 0001
BEGIN;
CREATE TABLE "stores_store" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "name" varchar(30) NOT NULL, "address" varchar(30) NOT NULL, "city" varchar(30) NOT NULL, "state" varchar(2) NOT NULL);

COMMIT;

As you can see in listing 4, the migration file 0001_initial.py for the stores app is set to run an SQL statement that creates a database table named stores_store with field names that are inline with the Django model.

Previewing the SQL statements generated by a migration file might not seem too exciting at this stage, but it can be very helpful in other circumstances. For example, if you make complex changes to a Django model or your database is relatively large, it's beneficial to preview the SQL before applying the migration file directly to a database.

Finally, the last step in the workflow for Django models is to apply the migration files to a database with the migrate command. Listing 5 illustrates this sequence and adds the stores argument to limit the process to the stores app -- if you run migrate without any argument, Django processes the migration files for every app.

Listing 5 - Django migrate command to execute migration files on database

[user@coffeehouse ~]$ python manage.py migrate stores
Operations to perform:
  Apply all migrations: stores
Running migrations:
  Applying stores.0001_initial... OK

In listing 5 you can see the stores.0001_initial migration is run against the database. This means the SQL presented in listing 4 is executed against the database. In addition and to keep track of applied migrations, the django_migrations database table is updated with a new record to reflect the stores.0001_initial migration was run against the database.