Django model data types

Data needs to conform to certain rules for it to be useful in any application. If it weren't for rules, then you can easily end up with ZIP code numbers where you expect address information or extensive text where you expect a maximum 10-character input. Django models can use several data types to enforce model data fit certain rules (e.g. text, number, email).

It's important to understand Django model data types operate on two layers: at the database layer and the Django/Python layer. When you define a Django model with its data types and create its initial migration, Django generates and executes a model's DDL (Data definition language) to create a database table, this DDL contains rules that reflect a Django model's fields (e.g. a Django IntegerField model field gets translated into an INTEGER DB column to enforce integer values). This means that if you change this type of Django model field (e.g. changing an integer field to a text field) it requires generating a new migration in order for the database table to also reflect any new rule.

In addition to this rule enforcement created at the outset by a Django model at the database layer, Django model fields also enforce data rules at the Django/Python layer (i.e. prior to inserting/updating data in the database). For example, Django model field parameters like choices enforce a field's values conform to a set of choices, this type of rule is enforced when you attempt to create, save or update Django model records and is done irrespective of the database table rules. This means you can simply change a Django model field associated with this type of rule and not have to create a new migration to alter the underlying database table -- in this case, changing the Python code is enough to change validation behavior.

Always keep this subtle in mind for Django model field data types: some Django model changes require creating a migration, while others can simply take effect changing the Django model field code. Table 7-1 illustrates the various Django model fields and the DDL they produce for all of the four main relational database supported by Django: SQLite, MySQL, PostgreSQL and Oracle.

Table 7-1 Django model data types and generated DDL by database

As you can see in table 7-1, Django model fields produce slightly different DDL depending on the database brand, albeit the end behavior for all backends maps as close as possible to one another, with Django/Python rule enforcement filling in the gaps.

For example, for a Django model field like models.DurationField(), SQLite and MySQL use the bigint data type, where as PostgreSQL and Oracle use the more specialized interval data type. Similarly for Django model fields like models.CommaSeparatedIntegerField() and models.EmailField(), at the database level these are represented as basic character varchar data types -- or NVARCHAR2 in Oracle -- and it's Django/Python that enforces the text representation is a valid CSV of integers or email, respectively.

With knowledge of the initial DDL generated by each Django model field, in the next sections I'll present the various Django model field options and their behavior associated with the original DDL and Django/Python validation.

Limiting values: max_length, min_value, max_value, max_digits and decimal_places

Limiting values to a certain range is one of the most basic options in Django model fields. For text based data types the max_length option enforces values don't surpass a certain number of characters. In table 7-1 you can see the CharField data type requires you specify the max_length option and for more specialized text based data types (e.g. EmailField) Django assigns a default max_length option value, which you can override with an explict max_length value.

For fields that use the IntegerField data type, Django offers the min_value and max_value options to restrict a value to lower/upper bounds (e.g. IntegerField(min_value=0,max_value=1000) limits values to a 0 to 1000 range). Similarly, for fields with the DecimalField data type, Django requires you specify the max_digits and decimal_places options to enforce a value's maximum number of digits and decimal points, respectively.

Empty, null & not null values: blank and null

By default, all Django model fields are assigned a NOT NULL restriction at the database level, if you look at table 7-1 again you can confirm this by the generated DDL -- the only exception is for certain fields on Oracle databases, which is explained in the sidebar below table 7-1.

This means that when you create/update a record with a NOT NULL field on it, a value must be provided for the field or otherwise the database rejects the operation. On certain occasions though, it can be necessary to allow empty field values or as they're known in the database world NULL values. To allow Django model fields to support NULL values at the database level, you must declare the null=True field option (e.g. IntegerField(null=True) allows an integer field to be left empty and generates DDL with NULL instead of the default NOT NULL).

In addition to the null option that's enforced at the database level and which defaults to null=False, Django also supports the blank option. The blank option also defaults to blank=False on all fields and is used to enforce Django/Python validation through forms that work on Django models (e.g. creating a Django model record through a form in the Django admin or a custom Django form). If a Django model field is declared with blank=True, Django permits the field to be left blank in a form, otherwise (i.e. if blank=True isn't explicitly declared in a model field) Django rejects the form and forces an end-user to provide a value for the field.

Since the use of null and blank options can be one of the most confusing topics in Django model field options, table 7-2 presents a matrix with different model definitions and operations to better illustrate this behavior.

Table 7-2. Django model validation for blank and null field options

Model definition class Person(models.Model): first_name = models.CharField(max_length=30) middle_name = models.CharField(max_length=30) last_name = models.CharField(max_length=30)
Blank and null field combinations	Default (blank=False,null=False) middle_name = models.CharField(max_length=30)	null=True (with default blank=False) middle_name = models.CharField(max_length=30,null=True)	blank=True(with default null=True) middle_name= models.CharField(max_length=30,blank=True)	null=True and blank=True middle_name = models.CharField(max_length=30,null=True,blank=True)
Person.objects.create( first_name='Johann', middle_name=None, last_name='Bach')	None is treated as NULL and so middle_name can't be NULL	middle_name can be NULL due to null=True	None is treated as NULL and so middle_name can't be NULL, blank=True is for forms	middle_name can be NULL due to null=True
Person.objects.create( first_name='Johann', last_name='Bach')	Unspecified middle_name defaults to an empty string '' and operation succeeds as '' is treated as not null	Unspecified middle_name defaults to an empty string '' and operation succeeds as '' is treated as not null	Unspecified middle_name defaults to an empty string '' and operation succeeds as '' is treated as not null	Unspecified middle_name defaults to an empty string '' and operation succeeds as '' is treated as not null
Person.objects.create( first_name='Johann', middle_name='', last_name='Bach')	Explicit empty string '' is treated as not null and operation succeeds.	Explicit empty string '' is treated as not null and operation succeeds.	Explicit empty string '' is treated as not null and operation succeeds.	Explicit empty string '' is treated as not null and operation succeeds.
Form validation with empty middle_name in Django admin or regular Django form.	Validation error due to empty middle_name	Validation error due to empty middle_name	Record creation valid even with empty middle_name due to blank=True	Record creation valid even with empty middle_name due to blank=True

Predetermined values: default, auto_now, auto_now_add and choices

Sometimes it's helpful to assign predetermined values to Django model fields. For example, to avoid empty strings when no value is provided -- as illustrated in some of the cases in table 7-2 -- you can use the default option on a Django model field (e.g. provide a default id, city or date). In most circumstances you assign a default option as a hard-coded value or a method reference. Listing 7-6 illustrates an example of the default option using both approaches.

Listing 7-6. Django model default option use

def default_city():
    return "San Diego"

class Store(models.Model):
    name = models.CharField(max_length=30)    
    address = models.CharField(max_length=30)
    city = models.CharField(max_length=30,default=default_city)
    state = models.CharField(max_length=2,default='CA')

As you can see in listing 7-6 we have two Django model fields that use the default option. First the city field relies on the default=default_city value -- note the lack of parenthesis in the syntax which makes it a reference to the function -- that tells Django to call the default_city method to populate the field every time a new record is created.

It's worth mentioning the method is placed outside the Django model class vs. declaring it in the same class body, this is to allow serialization and support of Django model migrations in Python 2 projects^[1]. Next in listing 7-6 you can see the state field uses the default='CA' value which tells Django to use the hard-coded CA string to populate the field when a new record is created.

The enforcement of default options is done entirely by Django/Python (i.e. the database DDL is unaware of any default values). This means that when you work with a model like the one in listing 7-6 Django/Python intervenes to provide default values. For example, if you create a Django model instance as Person.objects.create(name='Downtown',address='Main Street #5'), the city and state values are filled by Django/Python to create the record. Similarly, if you go to the Django admin and attempt to create an instance of the model in listing 7-6, the city and state form fields are pre-filled with the default values.

Even though the default option works just as described for text fields, as well as number and boolean fields, it has particular behaviors for date fields -- specifically models.DateField() and models.DateTimeField() -- that are important to explore, so let's do that and while on the topic of dates also learn about the auto_now and auto_now_add options related to date and time model fields.

Listing 7-7. Django model default options for dates and times, as well as auto_now and auto_now_add use

from datetime import date
from django.utils import timezone

class Store(models.Model):
    name = models.CharField(max_length=30)    
    address = models.CharField(max_length=30)
    date = models.DateField(default=date.today)
    datetime = models.DateTimeField(default=timezone.now)
    date_lastupdated = models.DateField(auto_now=True)
    date_added = models.DateField(auto_now_add=True)
    timestamp_lastupdated = models.DateTimeField(auto_now=True)
    timestamp_added = models.DateTimeField(auto_now_add=True)

Listing 7-7 illustrates a modified version of the Store model from listing 7-6 with a series of date field variations. The first two additional fields date and datetime use a DateField with a default value using Python's standard datetime library and a DateTimeField with a default value using Django's django.utils.timezone module, respectively -- the sidebar contains more details on why use the django.utils.timezone module to create a date time value.

In this case, if you create a Store record based on the model in listing 7-7, Django/Python fills in the values for the date and datetime fields based on the backing library's functionality. Similarly, if you go to the Django admin and attempt to create an instance of the model in listing 7-7, the date and datetime form fields are pre-filled with the default values for the current date and date with time, respectively.

In listing 7-7 there's an additional pair of DateField and DateTimeField fields that use the auto_now=True and auto_now_add=True options. Both of these options work like the default option -- in the sense they add a default date or date & time -- but have a slightly different behavior. The first important difference is that while the default option can be used with several types of fields, the auto_now and auto_now_add options are designed for DateField and DateTimeField fields.

Values for fields that use either the auto_now and auto_now_add options are generated when a record is created. Values for fields that use the auto_now option are updated every time a record is changed, while values for fields that use the auto_now_add option remain frozen for the lifetime of the record. By default, DateField fields that use either the auto_now or auto_now_add options generate their value from datetime.date.today(), where as DateTimeField fields that use either auto_now or auto_now_add options generate their value from django.utils.timezone.now().

Unlike fields with the default option where you can provide a value for the field, fields with the auto_now and auto_now_add options can't be overridden. This means that even if you create or update a record and provide a value for a field with an auto_now or auto_now_add option, the value is ignored. Similarly, in the Django admin by default fields that use the auto_now and auto_now_add option are not displayed because their value can't be modified. Finally, it's worth mentioning the auto_now, auto_now_add and default options are mutually exclusive (i.e. you can't use more than one option in a field or it will result in an error).

As you can see, the auto_now option is ideal to track a record's last-modified date, the auto_now_add is ideal to track a record's creation date and the default option works as a general purpose modifiable date on a form field.

Another scenario for predetermined values is to restrict a Django model field to a list of values with the choices option to limit open-ended values and reduce disparate data and errors (e.g. a list of states ["CA","AR"], instead of letting users introduce Ca, Cali, ar or Arizona). Listing 7-8 illustrates an example of a Django model field that uses the choices option.

Listing 7-8 Django model choices option

ITEM_SIZES = (
            ('S','Small'),
            ('M','Medium'),
            ('L','Large'),
            ('P','Portion'),
            )

class Menu(models.Model):
    name = models.CharField(max_length=30)

class Item(models.Model):
    menu = models.ForeignKey(Menu, on_delete=models.CASCADE)
    name = models.CharField(max_length=30)
    description = models.CharField(max_length=100)
    size = models.CharField(choices=ITEM_SIZES,max_length=1)

The first thing you need to use the choices option is create a list of values as a tuple of tuples as illustrated in listing 7-8. The ITEMS_SIZES tuple has four tuples, where the first tuple element represents a key to use in the database (e.g.S,M) and the second tuple element represents a human-friendly representation of the first tuple element (e.g.Small, Medium).

Next, in listing 7-8 you can see the size field is assigned the choices=ITEM_SIZES value which tells Django to use the keys from ITEM_SIZES as potential values. Note that in this case the keys to use in the database are all one-character text values (e.g.S,M) that correspond to the CharField(max_length=1) data type, but you can also use numbers or boolean values as keys so long as they match the target field data type.

Unique values: unique, unique_for_date, unique_for_month and unique_for_year

It's possible to enforce a field value be unique across all records. For example, in listing 7-7 if you change name=models.CharField(max_length=30) to name=models.CharField(max_length=30, unique=True) it tells Django to ensure all Store records have a unique name value.

The unique option is enforced by Django at the database layer (i.e. by adding a DDL UNIQUE SQL constraint), as well as the Django/Python layer. In addition, the unique option is valid on all field types except ManyToManyField, OneToOneField, and FileField.

To enforce uniqueness of a field along with a DateField or DateTimeField value, Django offers the unique_for_date, unique_for_month and unique_for_year options. For example, in listing 7-7 to enforce the name field value be unique with a date of the date_lastupdated field you can use the name = models.CharField(max_length=30, unique_for_date="date_lastupdated") statement which tells Django to allow only one record with the same name and date_lastupdated (e.g. you can't have two records with name="Downtown" and date_lastupdated="2018-01-01", but two records with name="Downtown" and different date_lastupdated values are allowed).

The unique_for_month and unique_for_year options provide wider ranges to enforce validation. For example, the name = models.CharField(max_length=30, unique_for_month="date_lastupdated") statement tells Django to only allow one record with the same name and date_lastupdated value for the same month and the name = models.CharField(max_length=30, unique_for_year="date_lastupdated") statement tells Django to only allow one record with the same name and date_lastupdated value for the same year.

Due to the more complex requirements of the unique_for_date, unique_for_month and unique_for_year options, the enforcement of these options is done at the Django/Python layer. In addition, due to the nature of the validation process these options are only valid for DateField and DateTimeField fields, noting that for DateTimeField cases only the date portion of the value is used.

Form values: editable, help_text, verbose_name and error_messages

When Django models are used in the context of forms, form fields backed by Django model fields can be influenced through various model options. The following options are identical to the form options you learned in the previous chapter on Django forms, except these options are assigned as part of model fields to influence forms fields.

By default, all Django model fields presented in forms are editable, but you can change this behavior by using the editable=False option. Setting a model field with editable=False tells Django to omit it completely in a form (e.g. in the Django admin) and as a consequence any validation associated with the form field is also bypassed.

The help_text option allows the inclusion of additional text alongside a form field. Listing 7-9 illustrates the use of the help_text option.

Listing 7-9 Django model help_text option

ITEM_SIZES = (
            ('S','Small'),
            ('M','Medium'),
            ('L','Large'),
            ('P','Portion'),
            )
	    
class Menu(models.Model):
    name = models.CharField(max_length=30)

class Item(models.Model):
    menu = models.ForeignKey(Menu, on_delete=models.CASCADE)
    name = models.CharField(max_length=30)
    description = models.CharField(max_length=100,help_text="""Ensure you provide 
                                             some description of the ingredients""")
    size = models.CharField(choices=ITEM_SIZES,max_length=1)
    calories = models.IntegerField(help_text="""Calorie count should reflect 
                                             <b>size</b> of the item""")

In listing 7-9 you can see there are two fields with the help_text option. When a Django form based on this Django model is rendered in a template, the help_text defined in the model goes on to form part of the final form layout.

By default, a Django model field name is used to generate the header of a Django form field using a capitalized version of the model field name. (e.g. Name, Description). You can configure more verbose form field headers with the verbose_name option (e.g. models.CharField (max_length=30,verbose_name="ITEM NAME"), outputs the ITEM NAME header for the form field.

If you attempt to save or update a Django model in a form and any of the values don't comply with the underlying field types, Django generates default error messages for the non-compliant values. These error messages while helpful are generic in nature, so it's possible to customize these error messages for each field with the error_messages option. The error_messages option accepts a dictionary of key-values, where keys represents error codes and values error messages. See the previous chapter on Django forms for detailed examples on the structure of the error_messages dictionary.

Database Definition Language (DDL) values: db_column, db_index, db_tablespace, primary_key

By default, Django generates a database table's column names based on a Django model's field names. For example, if a Django model field is named menu Django generates a DB column named menu. It's possible to override this behavior using the db_column option (e.g. name = models.CharField(max_length=30,db_column="my_custom_name") Django generates the DDL with the my_custom_name column name).

Another database related option for Django models is the db_index option that tells Django to generate a database index for the field (e.g. size = models.CharField(choices=ITEM_SIZES,max_length=1,db_index=True) generates a database index for the size field). Be aware that under the following circumstances Django automatically creates a database index, so db_index=True for the following scenarios is redundant:

• If a field uses the unique=True option, Django automatically creates an index for the field. You can disable this behavior setting db_index=False.
• If a field is a ForeignKey data type, Django automatically creates an index for the field. You can disable this behavior setting db_index=False.

If you use a PostgreSQL or Oracle database, it's possible to specify a DB tablespace for a field's index through the db_tablespace name. By default, Django uses a project's DEFAULT_INDEX_TABLESPACE value in settings.py to determine a tablespace, but if a field uses the db_tablespace this value takes precedence. If a database brand doesn't support tablespaces (e.g. MySQL or SQLite) this option is ignored.

Finally, the primary_key option allows you to define a primary key for a model. By default, if no Django model field is set with the primary_key=True statement, Django automatically creates an AutoField data type named id to hold the primary key (e.g. id = models.AutoField(primary_key=True) ). The typical use case for the primary_key option is for fields that are references to other fields (e.g. OneToOneField) or if you want a custom primary key due to design constraints.

Built-in and custom validators: validators

In addition to the data enforcement options for Django models described in previous sections (e.g. unique, default) Django models also offers enforcement of values through the validators option, which allow more advanced validation logic through built-in or custom methods.

Django model offers a series of built-in validator methods in the django.core.validators package which are used by model data types. For example, the models.EmailField data type relies on django.core.validators.EmailValidator to validate email values, just as the models.IntegerField data type uses the MinValueValidator and MaxValueValidator validators to enforce the min_value and max_value options, respectively.

In addition to Django built-in validator methods, you can also create custom validator methods. The only requirements to create custom validator methods, are for a method to accept a model field's input and to throw a django.core.exceptions.ValidatorError in case the value doesn't comply with the expected rules. Listing 7-10 illustrates a model that uses both a built-in validator and a custom validator.

Listing 7-10. Django model field validators option with built-in and custom validator

ITEM_SIZES = (
            ('S','Small'),
            ('M','Medium'),
            ('L','Large'),
            ('P','Portion'),
            )
	    
# Import built-in validator
from  django.core.validators import MinLengthValidator

# Create custom validator
from django.core.exceptions import ValidationError

def calorie_watcher(value):
    if value > 5000:
        raise ValidationError(
            ('Whoa! calories are %(value)s ? We try to serve healthy food, try something less than 5000!'),
            params={'value': value},
        )
    if value < 0:
        raise ValidationError(
            ('Strange calories are %(value)s ? This can\'t be, value must be greater than 0'),
            params={'value': value},
        )

class Menu(models.Model):
    name = models.CharField(max_length=30)

class Item(models.Model):
    menu = models.ForeignKey(Menu, on_delete=models.CASCADE)
    name = models.CharField(max_length=30,validators=[MinLengthValidator(5)])
    description = models.CharField(max_length=100)
    size = models.CharField(choices=ITEM_SIZES,max_length=1)
    calories = models.IntegerField(validators=[calorie_watcher])

The first validators option in listing 7-10 uses the built-in MinLengthValidator validator class to enforce the values for the name field contain at least 5 characters. The second validators option in listing 7-10 uses the custom calorie_watcher validator method to enforce the values for the calories field fit a certain range and uses custom messages in case this range is not met. It's worth mentioning that it's possible to use multiple validator methods on a single field with a list syntax (e.g.validators=[MinLengthValidators(5),MaxLengthValidators(100)]).

1. https://docs.djangoproject.com/en/1.11/topics/migrations/#serializing-values     ↑