Django formsets

Because Django forms represent the primary means users introduce data into Django projects, it's not uncommon for Django forms to be used as a data capturing mechanism. This however can lead to an efficiency problem, relying on one Django form per web page. Django formsets allow you to integrate multiple forms of the same type into a template -- with all the necessary validation and layout facilities -- to simplify data capturing by means of multiple forms.

The good news is that all you've learned about Django forms up to this point -- form fields, validation workflow, template layouts, widgets and all the other topics -- applies equally to Django formsets. This means the learning curve for formsets is rather simple, albeit you will have to learn some new concepts:

• Formset initialization.- Although formsets are initialized as a group of regular Django forms, formset initialization requires parameters specific to formsets.
• Formset management form.- To keep track of multiple Django forms on the same page, formsets also use a special form called a 'management form'.

Let's assume you're in the process of adding online ordering capabilities to your coffehouse application. You already have a drink form so users can place an order for a drink, but want to add the ability for users to order multiple drinks, so you need multiple drink forms on the same page or a drink formset.

Listing 6-38 illustrates the standalone DrinkForm class, the corresponding view method that generates an empty formset and the template layout used to display the formset.

Listing 6-38. Django formset factory initialization and template layout.

# forms.py
from django import forms

DRINKS = ((None,'Please select a drink type'),(1,'Mocha'),(2,'Espresso'),(3,'Latte'))
SIZES = ((None,'Please select a drink size'),('s','Small'),('m','Medium'),('l','Large'))

class DrinkForm(forms.Form):
    name = forms.ChoiceField(choices=DRINKS,initial=0)
    size = forms.ChoiceField(choices=SIZES,initial=0)
    amount = forms.ChoiceField(choices=[(None,'Amount of drinks')]+[(i, i) for i in range(1,10)])    

# views.py
from django.forms import formset_factory
def index(request):
    DrinkFormSet = formset_factory(DrinkForm, extra=2, max_num=20)
    if request.method == 'POST':
        # TODO
    else:
        formset = DrinkFormSet(initial=[{'name': 1,'size': 'm','amount':1}])
    return render(request,'online/index.html',{'formset':formset})

# online/index.html
<form method="post">
          {% csrf_token %}
    {{ formset.management_form }}
    <table>
        {% for form in formset %}
        <tr><td><ul class="list-inline">{{ form.as_ul }}</ul></td></tr>
        {% endfor %}
    </table>
    <input type="submit" value="Submit order" class="btn btn-primary">    
</form>

The DrinkForm class in listing 6-38 uses standard Django form syntax, so there's nothing new there. However, the index view method starts by using the django.forms.formset_factory method. The formset_factory is used to generate a FormSet class from a given form class. In this case, notice the formset_factory uses the DrinkForm argument -- representing the form class -- to generate a DrinkForm formset. I'll provide details about the additional formset_factory arguments shortly.

Next, in listing 6-38 you can see an unbound DrinkFormSet() instance is created with an initial value -- similar to how standalone unbound forms are created and use the initial argument. However, notice the initial value is a list, unlike a standalone dictionary used in standard forms. Because a formset is a group of forms, a formset's initial value is a group of dictionaries, where each dictionary represents the initial values for each form. In the case of listing 6-38, one of the formset's forms is set to initialize with the {'name': 1,'size': 'm','amount':1} values.

Toward the bottom of listing 6-38 you can see the template for the formset. The first difference between a standalone form template, is the {{ formset.management_form }} statement to output the management form. The second difference is the loop over the formset reference outputs the various forms. In this case, each formset form is output in its entirety with {{form.as_ul}} as a inline form, but you can equally use any Django form template layout technique to output each form instance in a custom manner (e.g. remove field id values).

Formset factory

The formset_factory() method used in listing 6-38 is one of the centerpieces to working with formsets. Although the example in listing 6-38 only uses three arguments, the formset_factory() method can accept up to nine arguments. The following snippet illustrates the names and default values for each argument in the formset_factory() method.

formset_factory(form, formset=BaseFormSet, extra=1, can_order=False, can_delete=False,
                  max_num=None, min_num=None, validate_max=False, validate_min=False)

As you can confirm in this snippet, the only required argument (i.e. that doesn't have a default value) for the formset_factory() method is form. The meaning for each argument is the following:

• form.- Defines the form class on which to create a formset.
• formset.- Defines the formset base class, defaults to django.forms.formsets.BaseFormSet. Changes when you require a custom formset for cases like custom formset validation.
• extra.- Defines the amount of empty forms added to a formset. If a formset's forms are all empty (i.e. they're not initialized) a formset contain extra number of forms. If a formset's forms contain data (i.e. they're initialized) a formset contains the number of initialized forms + extra (empty forms).
• can_order.- Adds the ORDER field (as a forms.IntegerField) to every form in a formset, with the purpose to alter the form order in a formset. Defaults to False.
• can_delete.- Adds the DELETE field (as a forms.BooleanField) to every form in a formset, with the purpose to mark a form in a formset for deletion. Defaults to False.
• max_num.- Defines the maximum number of forms to display in a formset. Defaults to None, to indicate up to 1000 forms instances be displayed.
• min_num.- Defines the minimum number of forms to display in a formset. Defaults to None, to indicate a minimum 0 form instances be displayed.
• validate_max.- Used in conjunction with max_num to ensure validation is enforced and the form instances never exceed max_num. Defaults to False.
• validate_min.- Used in conjunction with min_num to ensure validation is enforced and the form instances are never less than
• min_num. Defaults to False.

Now that you're aware of the various formset_factory() method options, let's turn out attention back to the method used in listing 6-38:

formset_factory(DrinkForm, extra=2, max_num=20)

This formset_factory method creates a formset with the DrinkForm class; the extra=2 indicates to always include 2 empty DrinkForm instances, this means that because the formset was initialized with one DrinkForm instance in listing 6-38, the total number of forms in the formset will be one, plus two empty forms on account of extra=2; the max_num=20 argument indicates the formset should contain a maximum 20 DrinkForm instances.

Formset management form and formset processing

To understand the purpose of a formset's management form, it's easiest to look at what this form contains. Listing 6-39 illustrates the contents of the formset management form based on the example from listing 6-38.

Listing 6-39. Django formset management form contents and fields.

<input type="hidden" name="form-TOTAL_FORMS" value="3" id="id_form-TOTAL_FORMS" />
<input type="hidden" name="form-INITIAL_FORMS" value="1" id="id_form-INITIAL_FORMS" />
<input type="hidden" name="form-MIN_NUM_FORMS" value="0" id="id_form-MIN_NUM_FORMS" />
<input type="hidden" name="form-MAX_NUM_FORMS" value="20" id="id_form-MAX_NUM_FORMS" />

As you can see in listing 6-39, the contents of a formset management form (i.e. the {{formset.management_form}} statement from listing 6-38) are four hidden input variables. The form-TOTAL_FORMS field indicates the total amount of forms in a formset; the form-INITIAL_FORMS fields indicates the total amount of initialized forms in a formset; the form-MIN_NUM_FORMS field indicate the minimum number of forms in a formset; and the form-MAX_NUM_FORMS field indicates the maximum number of forms in the formset.

At first glance the variables in listing 6-39 can appear unimportant, but they provide an important role in formsets once the various forms in a formset enter the processing and rendering phase. To better illustrate the relevance of these formset management fields, let's modify the formset in listing 6-38 to allow users to add more drink forms so they can grow their order as needed.

Listing 6-40. Django formset designed to add extra forms by user.

#views.py
def index(request):
    extra_forms = 2
    DrinkFormSet = formset_factory(DrinkForm, extra=extra_forms, max_num=20)
    if request.method == 'POST':
        if 'additems' in request.POST and request.POST['additems'] == 'true':
            formset_dictionary_copy = request.POST.copy()
            formset_dictionary_copy['form-TOTAL_FORMS'] =
                          int(formset_dictionary_copy['form-TOTAL_FORMS']) + extra_forms
            formset = DrinkFormSet(formset_dictionary_copy)
        else:
            formset = DrinkFormSet(request.POST)
            if formset.is_valid():
                return HttpResponseRedirect('/about/contact/thankyou')
    else:
        formset = DrinkFormSet(initial=[{'name': 1,'size': 'm','amount':1}])
    return render(request,'online/index.html',{'formset':formset})

# online/index.html
<form method="post">
          {% csrf_token %}
    {{ formset.management_form }}
    <table>
        {% for form in formset %}
        <tr><td>{{ form }}</td></tr>
        {% endfor %}
    </table>
    <input type="hidden" value="false" name="additems" id="additems">
    <button class="btn btn-primary" id="additemsbutton">Add items to order</button>
    <input type="submit" value="Submit order" class="btn btn-primary">  
</form>
<script>
$(document).ready(function() {
        $("#additemsbutton").on('click',function(event) {
         $("#additems").val("true");
       });
});
</script>

The first important modification in listing 6-40 comes in the formset template. Notice how the form declares a hidden input field named additems set to false, as well as a button that when clicked changes the value of this hidden input field to true. This mechanism is a simple control variable to keep track when a user wants to add more items to order (i.e. add more drink forms to the formset).

Now lets turn to the view method in listing 6-40 which processes the formset. First, notice the formset uses the extra_forms variable set to two to define a formset's extra value, so the initial formset factory contain two extra forms, just like in listing 6-38.

Next, comes the request.POST processing formset section which has two possible outcomes. If request.POST contains the additems field and it's set to true, it indicates a user clicked on the button to add more forms to the formset. If request.POST does not contain the additems field or it's set to false, it indicates the user clicked on the standard submit button, so a bound formset set is created and a call to is_valid() is made on the formset. Now, lets break down the logic behind each possible outcome.

When a user adds more forms to the formset, a copy of the request.POST is made -- the copy() method is necessary because request.POST is immutable. Next, the formset's management form field form-TOTAL_FORMS is modified to reflect additional forms by adding the extra_forms variables. Finally, the DrinkFormSet is re-bound with this new modified request.POST dictionary -- with an altered form-TOTAL_FORMS. When the re-bound formset is sent back to a user, the formset will now contain two additional empty forms due to simply modifying the form-TOTAL_FORMS management formset field.

If the formset falls to the standard POST processing and validation section in listing 6-40. First, a bound formset is created using request.POST -- just like it's done with regular bound forms -- next, the is_valid() method is called on the formset -- also just like it's done in regular bound forms. If is_valid() returns true (i.e. there are no errors in the formset or individual forms) a redirect is made to the success page. If is_valid() returns false (i.e. there are errors in the formset or individual forms) an errors dictionary is attached to the formset reference and control falls to the last line -- return render(request,'online/index.html',{'formset':formset}) -- which sends the formset instance with errors for display on the template -- a process that again is almost identical to standard form validation and error management.

As you can see, with this minor modification to one of the fields in a Django formset's management form, it's possible to dynamically alter the amount of forms in a formset. Note that in most cases, it isn't necessary to manipulate a formset's management form fields directly, more often Django uses these values behind the scenes to keep track of processing and rendering tasks. Albeit once you create more advanced formset behavior (e.g. ordering forms, deleting forms), the remaining management formsets fields take on an equally important role and may need to be manipulated directly.

Formset custom validation and formset errors

All the forms in a formset are validated against the form validation rules explained in previous section (e.g. validators, clean_<field>() methods). However, sometimes it can be necessary enforce inter-form rules in a formset, in which case you'll need to build a custom formset class like the one illustrated in listing 6-41.

Listing 6-41. Django custom formset with custom validation

from django.forms import BaseFormSet

class BaseDrinkFormSet(BaseFormSet):
    def clean(self):
        # Check errors dictionary first, if there are any error, no point in validating further
        if any(self.errors):
            return
        name_size_tuples = []
        for form in self.forms:
            name_size = (form.cleaned_data['name'],form.cleaned_data['size'])
            if name_size in name_size_tuples:
                raise forms.ValidationError("""Ups! You have multiple %s %s items in your order,
 keep one and increase the amount""" % (dict(SIZES)[name_size[1]],dict(DRINKS)[int(name_size[0])]))
            name_size_tuples.append(name_size)

First, notice the class in listing 6-39 inherits its behavior from the django.forms.BaseFormSet class, giving it all the basic functionalities of a Django form set. Next, the custom formset class defines a clean() method, which serves the same purpose of the clean() method in a standard Django forms: to enforce validation rules on the whole (i.e. formset) and not its individual parts (i.e. forms). The validation logic inside the clean() method enforces that if two forms from the formset have the same name and size an error is raised.

Notice in listing 6-41 how the validation error creation also uses the same forms.ValidationError() class used in standard forms. In the event the clean() method raises a validation error, the error is assigned to a special field called non_form_errors in a formset's errors dictionary. Listing 6-42 shows an updated version of the formset template in listing 6-38 illustrating how to out a formset's non form errors.

Listing 6-42. Django custom formset to display non_form_errors

<form method="post">
          {% csrf_token %}
    {{ formset.management_form }}
    {% if formset.non_form_errors %}
      <div class="alert alertdanger">{{formset.non_form_errors}}</div>
    {% endif %}
     {{ formset.management_form }}
    <table>
        {% for form in formset %}
        <tr><td><ul class="list-inline">{{ form.as_ul }}</ul></td></tr>
        {% endfor %}
    </table> 
</form>

You can see below the {{formset.management_form}} statement in listing 6-42 a conditional loop is made to output any errors placed in the non_forms_errors key of the formset's errors dictionary. Although different in name, the purpose of the formset.non_form_errors is the same as the form.non_field_errors for regular Django forms, to display errors not associated with a specific part. Because formsets use form constructs the error variable is called non_forms_errors and because forms use field constructs the variable is called non_field_errors.

Note that if any errors are present in a formset's forms, they are placed in a form's error dictionary just like they are in regular forms, so you can use the techniques outlined in listing 6-28 to customize the output of individual form errors in a formset.

6. https://django-formtools.readthedocs.io/     ↑

7. http://django-crispy-forms.readthedocs.io/     ↑