Django form processing: Initialization, field access, validation and error handling.

Because Django form processing can have a great deal of variations, I'll start the discussion from the same form and view method explained in the previous section, now consolidated in listing 6-8.

Listing 6-8 Django form class with backing processing view method

from django import forms
from django.shortcuts import render

class ContactForm(forms.Form):
      name = forms.CharField(required=False)
      email = forms.EmailField(label='Your email')  
      comment = forms.CharField(widget=forms.Textarea)

def contact(request):
    if request.method == 'POST':
        # POST, generate form with data from the request
        form = ContactForm(request.POST)
        # Reference is now a bound instance with user data sent in POST 
        # process data, insert into DB, generate email, redirect to a new URL,etc
    else:
        # GET, generate blank form
        form = ContactForm()
        # Reference is now an unbound (empty) form
    # Reference form instance (bound/unbound) is sent to template for rendering
    return render(request,'about/contact.html',{'form':form})

Initialize forms : initial for fields & forms, __init__ method, label_suffix, auto_id, field_order and use_required_attribute

When users request a page backed by a Django form they're sent an empty form (a.k.a. unbound form) -- represented by the GET section and ContactForm() instance in listing 6-8. Although the form is always empty from a user data perspective, the form can contain data from as part of its initialization sequence (e.g. a pre-filled form with a user's email or name).

To perform this kind of Django form initialization you have three options. The first technique consists of initializing the form with a dictionary of values via the initial argument directly on the form instance declared in the view method. Listing 6-9 shows this technique.

Listing 6-9 Django form instance with initial argument declared in view method

def contact(request):
        ....
        ....
    else:
        # GET, generate blank form
        form = ContactForm(initial={'email':'johndoe@coffeehouse.com','name':'John Doe'})
        # Form is now initialized for first presentation to display these values
    # Reference form instance (bound/unbound) is sent to template for rendering
    return render(request,'about/contact.html',{'form':form})

The ContactForm() now uses initial={'email':'johndoe@coffeehouse.com','name':'John Doe'} to generate a pre-filled form instance with values for the email and name fields so end users don't have to go to the trouble of typing these values from scratch themselves.

The technique in listing 6-9 is intended for one-time or instance specific initialization values. If you want to constantly provide the same value for a given form field, a more suitable technique is to use the same initial argument directly on a form field, as show in listing 6-10.

Listing 6-10 Django form fields with initial argument

from django import forms

class ContactForm(forms.Form):
      name = forms.CharField(required=False,initial='Please provide your name')
      email = forms.EmailField(label='Your email', initial='We need your email')
      comment = forms.CharField(widget=forms.Textarea)

def contact(request):
        ....
        ....
    else:
        # GET, generate blank form
        form = ContactForm()
        # Form is now initialized for first presentation and is filled with initial values in form definition
    # Reference form instance (bound/unbound) is sent to template for rendering
    return render(request,'about/contact.html',{'form':form})

Notice in listing 6-10 how it's now the form fields in the form class equipped with the initial argument. It's worth mentioning that if you use the initial argument on both the form instance and form fields, form instance values take precedence over any form field (e.g. if you combine the statements from listing 6-9 and listing 6-10, the form will always be pre-filled with email='johndoe@coffeehouse.com' and name='John Doe' from listing 6-9).

One of the drawbacks of using the initial argument in form fields like listing 6-10 is that values can't change dynamically at run-time (i.e. you can't personalize the initial values based on who calls the form like listing 6-9). To solve this and providing the utmost flexibility for the most complex form initialization scenarios is a third technique using the __init__ method of a form class illustrated in listing 6-11.

from django import forms

class ContactForm(forms.Form):
      name = forms.CharField(required=False)
      email = forms.EmailField(label='Your email')
      comment = forms.CharField(widget=forms.Textarea)
      def __init__(self, *args, **kwargs):
            # Get 'initial' argument if any
            initial_arguments = kwargs.get('initial', None)
            updated_initial = {}
            if initial_arguments:
                  # We have initial arguments, fetch 'user' placeholder variable if any
                  user = initial_arguments.get('user',None)
                  # Now update the form's initial values if user
                  if user:
                        updated_initial['name'] = getattr(user, 'first_name', None)
                        updated_initial['email'] = getattr(user, 'email', None)
            # You can also initialize form fields with hardcoded values
            # or perform complex DB logic here to then perform initialization
            updated_initial['comment'] = 'Please provide a comment'
            # Finally update the kwargs initial reference
            kwargs.update(initial=updated_initial)
            super(ContactForm, self).__init__(*args, **kwargs)

def contact(request):
        ....
        ....
    else:
        # GET, generate blank form
        form = ContactForm(initial={'user':request.user,'otherstuff':'otherstuff'})
        # Form is now initialized via the form's __init__ method 
    # Reference form instance (bound/unbound) is sent to template for rendering
    return render(request,'about/contact.html',{'form':form})

Next, let's turn our attention to the Django form's __init__ method, which is invoked when you create a form instance. The __init__ method's arguments *args, **kwargs are standard Python syntax -- if you've never seen this last syntax check out the appendix on Python basics: Methods: Default, optional, *args & **kwargs arguments.

The first step in __init__ checks for an initial value and creates a reference to hold the new values to initialize the form. If an initial value is present, a check is made for a user value to use these values for the form's actual name and email fields. Next, irrespective of any passed in values, a direct assignment is made on the form's comment field.

Finally, the form's initial reference is updated with a new set of values that reflect the form's actual fields, leading to the form's initialization using data outside the context of a form (e.g. request data, database query,etc). As a last step in the __init__ method, a call is made to the super() method so that the base/parent class initialization process takes place.

In addition to initializing the first set of data loaded on a Django form, there are four other initialization options for Django forms that influence a form's layout in templates: label_suffix, auto_id, field_order and use_required_attribute.

When you generate the output for a Django form in a template -- a topic that's described in detail later in this chapter in 'Set up the layout for Django forms in templates' -- form fields are generally accompanied by what's called a field label, which is another name for a human-friendly descriptor. For example, if you have fields called name and email, their default labels are Your name and Your email, respectively. To separate field labels from a field's HTML markup (e.g. <input type="text">) Django defines a label suffix, which defaults to : (the colon symbol) to produce output with the pattern <field_label>:<input type="<field_type>">. Through the label_suffix you can define a custom label suffix symbol to separate every form field from its label. So for example, the ContactForm(label_suffix='...') syntax outputs every form field label separated by ... (e.g. Your email...<input type="text">).

Another initialization option for Django forms is auto_id which automatically generates an id and label for every form field. By default, a Django form is always set to auto_id=True so you'll always get auto generated HTML ids and labels when outputting a form with form.as_table(), as illustrated in the first part of listing 6-12:

Listing 6-12 Django form with automatic ids (default auto_id=True option) and no automatic ids auto_id=False option

<!-- Option 1, default auto_id=True -->
<tr>
   <th><label for="id_name">Name:</label></th>
   <td><input id="id_name" name="name" type="text" /></td>
</tr>
<tr>
   <th><label for="id_email">Your email:</label></th>
   <td><input id="id_email" name="email" type="email" /></td>
</tr>
<tr>
   <th><label for="id_comment">Comment:</label></th>
   <td><textarea cols="40" id="id_comment" name="comment" rows="10"></textarea></td>
</tr>


<!-- Option 2 auto_id=False -->
<tr>
   <th>Name:</th>
   <td><input name="name" type="text" /></td>
</tr>
<tr>
   <th>Your email:</th>
   <td><input name="email" type="email" /></td>
</tr>
<tr>
   <th>Comment:</th>
   <td><textarea cols="40" name="comment" rows="10">\r\n</textarea></td>
</tr>

Notice how the field labels in the top output of listing 6-12 are wrapped around <label for="id_field_name"> <label> and the fields HTML tags includes the id="id_field_name" attribute. In most cases this is a desirable output as it allows fields to be easily referenced for purposes of attaching JavaScript events or CSS classes. However, for other circumstances auto_id=True can produce very verbose output and inclusively conflicting HTML tags (e.g. if you have two form instances of the same type on the same page, there will be two identical ids).

To turn off the auto generation of ids and labels you can initialize a form with the auto_id=False option. For example, the ContactForm(auto_id=False) syntax generates the output presented in the second half of listing 6-12.

Another initialization option for unbound form instances that influences a form's layout is the field_order option. By default, form fields are output in the order they're declared, so the form definition in listing 6-10 follows the output order: name, email, comment. You can override this default field output by using the field_order option which accepts a list of field names with the desired output order. The field_order option can be declared as part of the initialization process or inclusively as if it were a form field.

For example, the ContactForm(field_order=['email','comment','name']) syntax ensures the email field is output first, followed by comment and name. It's worth mentioning the field_order option can accept an incomplete field list, such as ContactForm(field_order=['email']) which outputs the email field followed by the remaining form fields in their declared order, in this case name and then comment. If you're constantly setting field_order to initialize a form's field order, a quicker solution is to set a default field_order as part of the form itself:

class ContactForm(forms.Form):
      name = forms.CharField(required=False)
      email = forms.EmailField(label='Your email')  
      comment = forms.CharField(widget=forms.Textarea)
      field_order = ['email','comment','name'] # Sets order email,comment,name

If you declare field_order as part of the form itself and on initialization, the initialization field_order takes precedence. The upcoming section 'Set up the layout for Django forms in templates' contains more details on the practical use of field_order in templates.

Finally, the use_required_attribute option allows you to set the overall use of the HTML 5 required attribute. By default use_required_attribute=True, which means all required form fields are output with the HTML 5 required attribute, ensuring browsers enforces these form fields are always provided. You can disable the use of this HTML 5 client-side validation required attribute by initializing a form with use_required_attribute=False. Note that setting use_required_attribute=False does not influence the Django server-side validation of a form field (e.g. if a form field is required, Django server-side validation still catches fields that aren't provided, irrespective of the use_required_attribute option).

Accessing form values: request.POST and cleaned_data.

Once a user fills out a Django form, the form is sent back to the server to be processed and perform an action with the user data (e.g. create an order, send an email, save the data to the database) -- a step which is represented in the POST section in listing 6-8.

One of the key advantages of Django form processing is you can use the request.POST variable to create a bound form instance. But although the request.POST variable is the initial access point to populate a Django form with user data, you shouldn't use this data beyond initializing a form instance, as the data in request.POST is too raw for direct access.

For example, in request.POST you still don't know if the user provided data is valid. In addition, the data in request.POST is still treated as strings, so if your Django form happens to have an IntegerField() or DateField() it still needs to be converted manually to the expected data type (e.g. '6' to 6 integer, '2017-01-01' to 2017-01-01 datetime), which is just unnecessary work that another another part of Django forms deals with.

Once you have a bound form generated with the request.POST variable, you can then access each of the form's field values through the cleaned_data dictionary. For example, if the bound form has a form field called name you can use the syntax form.cleaned_data['name'] to access the user provided name value. More importantly, if the form field is an IntegerField() named age the syntax form.cleaned_data['age'] produces an integer value, a formatting behavior that also applies to other form fields with non-string data types (e.g. DateField()).

By design, it isn't possible to access a form instance's cleaned_data dictionary unless you first call the is_valid() method. If you try to access cleaned_data before calling is_valid() you'll get the error AttributeError: 'form_reference' object has no attribute 'cleaned_data'.

If you think about this for a second it's good practice, after all, why would you want to access data that hasn't been validated ? The next section describes the is_valid() method.

Validating form values : is_valid(), validators, clean_<field>() and clean()

The is_valid() method is one of the more important parts of Django form processing. Once you create a bound form with request.POST, you call the is_valid() method on the instance to determine if the included values comply with a form's field definitions (e.g. if an EmailField() value is a valid email). Although the is_valid() method returns a boolean True or False value, it has two important side-effects:

• Calling is_valid() also creates the cleaned_data dictionary on the form instance to hold the form field values that passed validation rules.
• Calling is_valid() also creates the errors dictionary on the form instance to hold the form errors for each of the fields that didn't pass the validation rules.

Listing 6-13 illustrates a modified version of listing 6-8 with the is_valid() method.

Listing 6-13 Django form is_valid() method for form processing

from django.http import HttpResponseRedirect

def contact(request):
    if request.method == 'POST':
        # POST, generate form with data from the request
        form = ContactForm(request.POST)
        # Reference is now a bound instance with user data sent in POST
        # Call is_valid() to validate data and create cleaned_data and errors dict
        if form.is_valid():
           # Form data is valid, you can now access validated values in the cleaned_data dict
           # e.g. form.cleaned_data['email']
           # process data, insert into DB, generate email
           # Redirect to a new URL
           return HttpResponseRedirect('/about/contact/thankyou')
        else:
           pass # Not needed 
           # is_valid() method created errors dict, so form reference now contains errors 
           # this form reference drops to the last return statement where errors 
           # can then be presented accessing form.errors in a template
    else:
        # GET, generate blank form
        form = ContactForm()
        # Reference is now an unbound (empty) form
    # Reference form instance (bound/unbound) is sent to template for rendering
    return render(request,'about/contact.html',{'form':form})

Notice in listing 6-13 that right after a bound form instance is created a call is made to the is_valid() method. If all the form field values comply against the form field data types, we enter a conditional where it's possible to access the form values through the cleaned_data dictionary, perform whatever business logic is necessary and relinquish control to another page, which in listing 6-13 is to perform redirect.

If any of the form field values fails to pass a rule, then is_valid() returns False and in the process creates an errors dictionary with details about the values that failed to pass their rules. Because of this last automatic creation of errors, all that's needed after is_valid() returns False is to return the same form instance in order to display the errors dictionary to an end user so he can correct his mistakes.

But as important as the is_valid() method is to Django form processing, its validation is just done against a form field's data type. For example, is_valid() can validate if a value is left empty, if a value matches a given number range or even if a value is a valid date, in essence anything supported by Django form field types.

But what if you want to perform more sophisticated validation after is_valid() ? Like checking a value against a database before deeming it valid or checking two values against one another (e.g. a provided zip code value against a provided city value). While you can add these validation checks directly after the is_valid() call, Django offers three more efficient ways to enforce advanced rules by adding them to the form field or form class definition.

If you want a reusable validation mechanism you can use across multiple Django form fields, the best choice is a validator assigned through a form field's validators option. The validators form field option expects a list of methods designed to raise a forms.ValidationError error in case a value doesn't comply with expected rules. Listing 6-14 illustrates a Django form with one of its fields using a custom validator method via the validators option.

Listing 6-14 Django form field validators option with custom validator method for form processing

from django import forms
import re

def validate_comment_word_count(value):
      count = len(value.split())
      if count < 30:
            raise forms.ValidationError(('Please provide at least a 30 word message,
	    %(count)s words is not descriptive enough'), params={'count': count},)

class ContactForm(forms.Form):
      name = forms.CharField(required=False)
      email = forms.EmailField(label='Your email')
      comment = forms.CharField(widget=forms.Textarea,validators=[validate_comment_word_count])

The first section in listing 6-14 shows the custom validate_command_word_count()method, which (rudimentarly) checks if message has at least thirty words. If the method's input is not at least thirty words, Django's forms.ValidationError error is raised to indicate a rule violation.

In the bottom half of listing 6-14 you can see a modified ContactForm where the comment field uses the validators=[validate_csv] option. This tells Django that after is_valid() is run and all the form fields have been checked for errors against their data types, it should also run the validate_comment_word_count validator method against the value provided for the comment field. If the comment value does not comply with this rule, then a ValidatioError error is raised which is added to the form errors dictionary -- the same one described in the past section which is used to check field values against their data types.

As you can see from the example in listing 6-14, you can equally reuse the custom validate_comment_word_count() method on any other form field in the same form or in another Django form through the validators option. In addition, you can also apply multiple validators to a field since the validators option accepts a list of validators. Finally, it's worth mentioning the django.core.validators package contains a series of validators you can also reuse^[2] and which are used behind by the scenes by certain form field data types.

In addition to the form field validators option, it's also possible to add validation form rules through the clean_<field>() and clean() methods which are created as part of a Django form class -- just like __init__() described earlier. Just like methods specified in the form field validators option, clean_<field>() and clean() methods are automatically invoked when the is_valid() method is run. Listing 6-15 illustrates the use of two clean_<field>() methods.

Listing 6-15 Django form field validation with clean_<field>() methods

from django import forms

class ContactForm(forms.Form):
      name = forms.CharField(required=False)
      email = forms.EmailField(label='Your email')
      comment = forms.CharField(widget=forms.Textarea)
      def clean_name(self):
          # Get the field value from cleaned_data dict
          value = self.cleaned_data['name']
          # Check if the value is all upper case
          if value.isupper():
             # Value is all upper case, raise an error
             raise forms.ValidationError("""Please don't use all upper 
                  case for your name, use lower case""",code='uppercase')
          # Always return value 
          return value
      def clean_email(self):
          # Get the field value from cleaned_data dict
          value = self.cleaned_data['email']
          # Check if the value end in @hotmail.com
          if value.endswith('@hotmail.com'):
             # Value ends in @hotmail.com, raise an error
             raise forms.ValidationError("""Please don't use a hotmail email,
                                    we simply don't like it""",code='hotmail')
          # Always return value 
          return value

In listing 6-15 there are two clean_<field>() methods to add validation rules for the name and email fields. Django automatically searches for form methods prefixed with clean_ and attempts to match a form's field names to the remaining name, to enforce validation on the field in question. This means you can have as many clean_<field>() methods as form fields.

The logic inside each clean_<field>() method follows a similar pattern to validators methods. First you extract a field's value from the form's cleaned_data dictionary via the self reference which represents the form instance. Next, you run whatever rule or logic you want against the field value, if you deem the value doesn't comply you raise a forms.ValidationError which adds the error to the form instance. Finally, and this is only different to validators methods, you must return the field value irrespective of raising an error or changing its value.

Sometimes it's necessary to apply a rule that doesn't necessarily belong to a specific field, in which case the generic clean() method is the preferred approach vs. a clean_<field>() method. Listing 6-16 illustrates the use of the clean() method.

Listing 6-16. Django form field validation with clean() method

from django import forms

class ContactForm(forms.Form):
      name = forms.CharField(required=False)
      email = forms.EmailField(label='Your email')
      comment = forms.CharField(widget=forms.Textarea)
      def clean(self):
          # Call clean() method to ensure base class validation 
          super(ContactForm, self).clean()
          # Get the field values from cleaned_data dict
          name = self.cleaned_data.get('name','')
          email = self.cleaned_data.get('email','')
          # Check if the name is part of the email
          if name.lower() not in email:
             # Name is not in email , raise an error
             raise forms.ValidationError("Please provide an email that contains your name, or viceversa")

In listing 6-16 you can see a similar approach to the previous clean_<field>() methods in listing 6-15. But because the clean() method is a class wide method and you're overriding it yourself, it differs from clean_<field>() methods in that you must first explicitly call the clean() method of the base/parent form class (i.e. super(...).clean()) to ensure the base class validation is applied. Form field value extraction is also done through the cleaned_data data dictionary, as is the validation logic and raising of forms.ValidationError to indicate a rule violation. Finally, the clean() method differs from clean_<field>() methods in that it doesn't return a value.

Functionally the clean() method is different because it's called after all the methods in the validators options and clean_<field>() methods, a behavior that's important because all these methods rely on data in the cleaned_data dictionary. This means if a validators or clean_<field>() method raises a ValidationError error it won't return any value and the cleaned_data dictionary won't contain a value for this field. So by the time the clean() method is run, the cleaned_data dictionary may not necessarily have all the form field values if one was short-circuited in a validators or clean_<field>() method, which is the reason why the clean() method uses the safer dictionary access syntax cleaned_data.get('<field>','') to assign a default value in case the cleaned_data dictionary doesn't have a given field.

Another important behavioral difference between the clean() method, clean_<field>() & validators methods is how they treat forms.ValidationError. When a forms.ValidationError is raised in a validators or clean_<field>() method, the error is assigned to the <field> in question in the form's errors dictionary -- which is important for display purposes. But when a forms.ValidationError is raised in the clean() method, the error is assigned to a special placeholder field named __all__ -- also known as "non-field errors" -- which is also placed in the form's errors dictionary.

If you want to assign an error in the clean() method to a specific form field you can use the add_error() method as illustrated in listing 6-17.

Listing 6-17. Django form field error assignment with add_error() in clean() method

      def clean(self):
          # Call clean() method to ensure base class validation 
          super(ContactForm, self).clean()
          # Get the field values from cleaned_data dict
          name = self.cleaned_data.get('name','')
          # Check if the name is part of the email
          if name.lower() not in email:
             # Name is not in email , raise an error
             message = "Please provide an email that contains your name, or viceversa"
             self.add_error('name', message)
             self.add_error('email', forms.ValidationError(message))
             self.add_error(None, message)

Notice how listing 6-17 uses the add_error() method on the form instance instead of the raise forms.ValidationError() syntax. The add_error() method accepts two arguments, the first one the form field name on which to assign the error and the second can be either an error message string or an instance of the ValidationError class.

The first two add_error() methods in listing 6-17 assign the error to the name and email fields, respectively. And the third add_error() method with the None key assigns the error to the __all__ placeholder making it equivalent to raise forms.ValidationError() from listing 6-16.

Error form values: errors

Form errors as its been described in the previous sections are automatically added to a form instance in the errors dictionary after calling the is_valid() method. Inclusively, it's possible to access the errors dictionary directly without calling the is_valid() method, unlike the cleaned_data dictionary.

The errors dictionary is important because all form errors end up on it. Whether the errors are raised because a value doesn't comply with a form field data type or with raise forms.ValidationError() in the clean() method, clean_<field>() methods, validators methods or the add_error() method in the clean() method, all errors end up in a form's errors dictionary.

The errors dictionary follows the pattern {'<field_name>':'<error_message>'} which makes it easy to identify form errors in either a view method or in a template for display purposes. The only exception to this last pattern occurs when a form error isn't field specific (e.g. such as those created in the clean() method) in which case a form error is assigned to a special key named __all__, errors which are also called non-field errors.

Although you can access the errors dictionary as any other Python dictionary, Django provides a series of methods described in table 6-1 to make working with errors much easier:

Table 6-1 Django form errors methods

You may have noticed the ValidationError class instances created in all the previous examples use different arguments, meaning there are multiple ways to create ValidationError instances. For example, some ValidationError instances use a simple string, but it's also possible to create a ValidationError instance with a list of ValidationError instances, as well as specify a code attribute to further classify an error type. Listing 6-18 illustrates a series of ValidationError class instances using these variations.

Listing 6-18.- Django form ValidationError instance creation

from django import forms

# Placed inside def clean_email(self):
raise forms.ValidationError("""Please don't use a hotmail email, 
                                we simply don't like it""",code='hotmail')

# Placed inside def clean(self):
raise forms.ValidationError([
     forms.ValidationError(""""Please provide an email that matches
                                your name, or viceversa""",code='custom'),
     forms.ValidationError("""Please provide your professional email,
                            %(value)s doesn't look professional """,code='required',
                              params={'value':self.cleaned_data.get('email') })

The ValidationError instance variations presented in listing 6-18 are all optional, but can become powerful when it comes time to display or filter error messages on a template, a process that's described in detail in the 'Set up the layout for Django forms in templates' section later in this chapter.

2. https://docs.djangoproject.com/en/1.11/ref/validators/#built-in-validators     ↑