Django advanced form processing: Partial forms, AJAX and files
In most cases, Django form processing adheres to the steps and code sequences outlined in the first section of this chapter, shown in figure 6-1. However, Django form processing can require small adjustments for certain scenarios, like those involving partial form processing, AJAX forms and file uploads.
Partial forms
On occasions you may find
yourself with the need to use a partial form based on a
pre-existing Django form. For example, you get a request to add a
similar form to the ContactForm used up to this point
-- in listing 6-25 -- but without requiring the name
and email fields. Since the ContactForm
is tried and tested, including the necessary post-processing logic
(e.g. adding the form data to a CRM[Customer Relationship
Management] system) it's a worthwhile idea to reuse the
ContactForm class. But how do you go about using a
Django form partially ?
The first alternative to achieve partial forms is to hide the unneeded form fields from a user, allowing the original form class to remain intact while not requiring a new sub-class. This is the least invasive approach, but you need to be careful with two aspects. The first aspect is the form template layout, shown in listing 6-32.
Listing 6-32. Django form with fields marked as hidden
<form method="POST">
{% csrf_token %}
<div class="row">
<div class="col-md-2">
{{ form.comment.label_tag }}
{% if form.comment.help_text %}
<sup>{{ form.comment.help_text }}</sup>
{% endif %}
{% for error in form.comment.errors %}
<div class="row">
<div class="alert alert-danger">{{error}}</div>
</div>
{% endfor %}
</div><div class="col-md-10 pull-left">
{{ form.comment }}
</div>
</div>
{{form.name.as_hiddden}}
{{form.email.as_hidden}}
<input type="submit" value="Submit form" class="btn btn-primary">
</form>
First notice in listing 6-32 the
form fields are output individually and do not use a shortcut
method (e.g. form.as_table) or loop like some of the
previous form layout examples. The reason is, the last
form fields are explicitly output as hidden through
the as_hidden method -- described earlier in table 6-4. The as_hidden method generates a form field as a hidden HTML input (e.g. <input
type="hidden"...>), irrespective of its form field data
type. This mechanism effectively hides fields from end users, but
keeps the fields as part of the form.
Why is it important to keep
fields as part of the form and not just remove them ?
Validation. Remember the form instance hasn't changed,
only the requirements, you may not need the name and
email data on this template page, but the form
instance and validation logic doesn't know that. All of which takes
us to the second aspect you need to take care of: required
values.
If you publish the template in
listing 6-32 without taking care of this second aspect, you might
be surprised when you realize the form never passes validation! Why
? The underlying ContactForm class treats the
email field as required, therefore it must be given a
value, but since the field is now hidden from an end user, you must
take care of giving the email field a value yourself
-- note the name field doesn't present this problem
because it's configured with required=False.
Therefore, in order for the template in listing 6-32 to work, the
form must be initialized with a value like the following
snippet:
form = ContactForm(initial={'email':'anonymous@gmail.com'})
By initializing the form in this
manner, the email form field is given this initial value as a
hidden input. So once a user submits the form, the email value is
transferred back to Django as part of the form, where validation
passes since the post-processing logic gets all its expected
values. Of course, if a form's fields are all optional (i.e.
they're marked with required=False) this
initialization process is unnecessary, since the validation won't
be expecting any value anyways.
A second alternative to achieve partial forms is to create a sub-class from a form class and delete the unneeded fields. This is a little more straightforward approach, but it does require creating a form sub-class, which may be overkill for certain scenarios. Listing 6-33 shows how to sub-class a Django form class and remove some of its parent's fields.
Listing 6-33. Django form sub-class with removed parent fields
from coffeehouse.about.forms import ContactForm
class ContactCommentOnlyForm(ContactForm):
def __init__(self, *args, **kwargs):
super(ContactCommentOnlyForm, self).__init__(*args, **kwargs)
del self.fields['name']
del self.fields['email']
As you can see in listing 6-33,
the ContactCommentOnlyForm form inherits its behavior
from the ContactForm class. Next, inside the form
class __init__ method, a call is made to the parent's
__init__ class via super() and two calls
are made on self.fields using Python's
del to remove both the name and
email fields from this form sub-class.
Once you have the form sub-class
from listing 6-33, you can generate an unbound instance form in a
view method and pass it to a template for rendering. Since this
last sub-class removes the name and email
fields, upon validation you won't face previously described problem
of missing values for hidden fields, since the
ContactCommentOnlyForm form now only has a single
field.
AJAX form submission
AJAX is a technique where JavaScript on a web page communicates with a server-side application, re-renders the web page with the result of this communication, all without a web page transition. Django forms are often designed to submit their data via AJAX to create a smoother workflow (i.e. without changing the original web page with a web form)
The workflow for a form that uses AJAX as far as the initial form delivery is concerned -- steps 1 an 2 in figure 6-1 -- is identical to that of a regular Django non-AJAX form: you create an unbound form instance in a view method, which is passed to a template for rendering into a web page that's delivered to an end user.
The first difference in Django forms that submit their data via AJAX, is the web page on which the unbound form is delivered, must also contain the necessary JavaScript logic to send the form data to the server-side application and also process the server-side response. Listing 6-34 illustrates this JavaScript logic using the jQuery library to submit a Django form via AJAX.
Listing 6-34. JavaScript jQuery logic to submit Django form via AJAX
# NOTE: The following is only the Django (HTML) template with AJAX logic
# See listing 6-35 for AJAX processing views.py and urls.py
<h4>Feedback</h4>
<div class="row">
<div id="feedbackmessage"</div>
</div>
<form method="POST" id="feedbackform" action="{% url 'stores:feedback' %}">
{% csrf_token %}
<div class="row">
<div class="col-md-12 pull-left">
{{ form.comment }}
</div>
</div>
{{form.name.as_hiddden}}
{{form.email.as_hidden}}
<input type="submit" value="Submit feedback" class="btn btn-primary">
</form>
<script>
$(document).ready(function() {
$("#feedbackform").submit(function(event) {
event.preventDefault();
$.ajax({ data: $(this).serialize(),
type: $(this).attr('method'),
url: $(this).attr('action'),
success: function(response) {
console.log(response);
if(response['success']) {
$("#feedbackmessage").html("<div class='alert alert-success'>
Succesfully sent feedback, thank you!</div>");
$("#feedbackform").addClass("hidden");
}
if(response['error']) {
$("#feedbackmessage").html("<div class='alert alert-danger'>" +
response['error']['comment'] +"</div>");
}
},
error: function (request, status, error) {
console.log(request.responseText);
}
});
});
})
</script>
The first part of listing 6-34 is
a standard Django form layout. However, notice the
<form> tag includes the action
attribute. In previous Django forms, the action
attribute was not used because the serving url -- GET request --
and processing url -- POST request -- was the same one. In listing
6-34, the action attribute explicitly tells the
browser the url where it should send the POST request -- in this
case {% url 'stores:feedback' %} is a Django template
tag that gets translated into a url
(e.g./stores/feedback/). Different urls are necessary
in this case, because Django view methods that handle AJAX
requests/responses are different from Django view methods that
handle standard web requests/responses.
Now lets turn our attention to
the bottom half of listing 6-34 and the JavaScript logic enclosed
in <script> tags. In brief -- as it would go
beyond the scope of our discussion to describe jQuery syntax -- the
web page awaits a click on the #feedbackform form's
submit button. Once this click is detected, an AJAX POST request
using the form's data is sent to the url defined in the form's
action attribute. Upon completion of the AJAX request
(i.e. when the server-side sends a response back), the response --
in JSON format -- is analyzed. If the AJAX response contains a
success message, the message is output at the top of
the form and the form is hidden to avoid multiple submissions. If
the AJAX response contains an error message, the
message is output at the top of the form.
Now that you know how Django form data is submitted to the server-side via AJAX, listing 6-35 illustrates the Django view method necessary to handle this AJAX request payload.
Listing 6-35. Django view method to process Django form via AJAX
# urls.py (Main)
urlpatterns = [
url(r'^stores/',include('coffeehouse.stores.urls',namespace="stores"))
]
# urls.py (App stores)
urlpatterns = [
url(r'^$',views.index,name="index"),
url(r'^feedback/$',views.feedback,name="feedback"),
]
# views.py (App stores)
from django.http import HttpResponse, JsonResponse
from coffeehouse.about.forms import ContactForm
def feedback(request):
if request.POST:
form = ContactForm(request.POST)
if form.is_valid():
return JsonResponse({'success':True})
else:
return JsonResponse({'error':form.errors})
return HttpResponse("Hello from feedback!")
The first important point about
the view method in listing 6-35 is that it only handles POST
requests. Notice that in case the if request.POST:
statement isn't true, the view method always responds with the
Hello from feedback! string.
Next, inside the
request.POST segment, the process starts out pretty
similar to a standard Django form processing sequence (i.e. a bound
form instance is created and a call is made to the form's
is_valid() method to validate the user provided data).
However, notice the responses for both valid and invalid form data
use JsonResponse(). Since the AJAX form submission
operates in a JavaScript context, the JSON format response is a
common format to use to send a response back to the browser.
If you turn your attention back
to listing 6-34, you can see the JavaScript logic is designed to
handle and output either JSON response (success or
error) returned by the view method in listing 6-35.
Files in forms
Django offers a couple of form
fields to capture files through forms --
forms.ImageField() and forms.ImageField()
described in table 6-2. However, although these form fields
generate the necessary HTML and validation logic to enforce files
are submitted via forms, there are a couple of subtleties related
to processing files in forms.
The first issue is the HTML
<form> tag for forms that transfer files must
explicitly set the encoding type to
enctype="multipart/form-data", in addition to using
method=POST. The second issue is the contents of files
are placed in the Django request object under the
special FILES dictionary key. Listing 6-36 illustrates
a form with file fields, its corresponding view method and its
template layout.
Listing 6-36. Django form with file fields, corresponding view method and template layout.
# forms.py
from django import forms
class SharingForm(forms.Form):
# NOTE: forms.PhotoField requires Python PIL & other operating system libraries,
# so generic FileField is used instead
video = forms.FileField()
photo = forms.FileField(widget=forms.ClearableFileInput(attrs={'multiple': True}))
# views.py
def index(request):
if request.method == 'POST':
# POST, generate form with data from the request
form = SharingForm(request.POST,request.FILES)
# check if it's valid:
if form.is_valid():
# Process file data in request.FILES
# Process data, insert into DB, generate email,etc
# redirect to a new URL:
return HttpResponseRedirect('/about/contact/thankyou')
else:
# GET, generate blank form
form = SharingForm()
return render(request,'social/index.html',{'form':form})
# social/index.html
<form method="post" enctype="multipart/form-data">
{% csrf_token %}
<ul>
{{form.as_ul}}
</ul>
<input type="submit" value="Submit photo" class="btn btn-primary">
</form>
Notice how the HTML
<form> tag declares the required attributes and
in addition notice how the request.FILES reference is
used to create a bound form -- along with the standard
request.POST. The remaining form structures in listing 6-36 are identical to regular non-file forms (e.g. unbound form
creation, call to is_valid()).
But similar to regular Django
form processing, once a form is valid -- if is_valid()
returns True -- you'll want do something with the
contents of the form data, therefore in this case, you'll need to
access request.FILES to do something with the files
uploaded by a user.
But before describing how to
process the contents of files in request.FILES it's
important you understand the possible contents of the
request.FILES dictionary:
Option 1) request.FILES = {'photo': [<InMemoryUploadedFile>,<TemporaryUploadedFile>],
'video': [<InMemoryUploadedFile>]}
Option 2) request.FILES = {'photo': [<TemporaryUploadedFile>],
'video': [<InMemoryUploadedFile>]}
The request.FILES
dictionary contains keys corresponding to file field names. In this
case, you can see the photo and video
keys correspond to the form fields names declared in listing 6-36
that represent files. Next, notice the value for each key is a
list, irrespective if the file form field can accept multiple files
-- like photo due to the overridden widget with the
'multiple': True attribute -- or a single file -- in
the case of video.
Now lets analyze the contents of
the lists assigned to each file form field key. Each of the list
elements represents an uploaded file. But notice how the files are
represented by either the InMemoryUploadedFile or
TemporaryUploadedFile class -- both of which are
sub-classes of the
django.core.files.uploadedfile.UploadedFile class. The
reason uploaded files are represented by either an
InMemoryUploadedFile or
TemporaryUploadedFile instance class, is dependent on
the size of an uploaded file.
As it turns out, before you even
decide how to handle uploaded files, Django must decide what to do
and where to place uploaded files. To do this, Django relies on
upload handlers. By default and unless explicitly overridden in the
FILE_UPLOAD_HANDLERS variable in
settings.py,Django uses the following two file upload
handlers:
FILE_UPLOAD_HANDLERS= [
'django.core.files.uploadhandler.MemoryFileUploadHandler',
'django.core.files.uploadhandler.TemporaryFileUploadHandler',
]
By default, if a file is less
than 2.5 MB, Django uses MemoryFileUploadHandler to
place the contents of an uploaded file into memory; if a file is
larger than 2.5 MB, Django uses
TemporaryFileUploadHandler to place the contents of an
uploaded file in a temporary file (e.g.
/tmp/Af534.upload).
You can define
FILE_UPLOAD_HANDLERS in settings.py to
only include the file upload handler you desire (e.g. if you wish
to save memory resources remove
MemoryFileUploadHandler). However, note a file upload
handler must always be active in order for file uploads to
work. If you don't like the behavior of Django's built-in file
upload handlers, then you'll need to write your own file upload
handler[5].
In addition to defining
FILE_UPLOAD_HANDLERS in settings.py,
there are two other parameters you can declare in
settings.py to influence the behavior of file upload
handlers. The FILE_UPLOAD_MAX_MEMORY_SIZE variable
sets the threshold size at which files are held in memory vs.
handled as temporary files, defaulting to 2621440
bytes (or 2.5 MB). And the FILE_UPLOAD_TEMP_DIR
variable sets the directory to which temporary uploaded files must
be saved, defaulting to None, meaning an operating
system's temporary file directory is used (e.g. on Linux OS
/tmp/).
Now that you know where and how
Django stores uploaded files even before you decide how to handle
them, let's take a look at how to process the contents of
request.FILES. Listing 6-37 shows the continuation of
listing 6-36 with the relevant snippets necessary to process
uploaded files.
Listing 6-37. Django form file processing with save procedure to MEDIA_ROOT.
# views.py
from django.conf import settings
def save_uploaded_file_to_media_root(f):
with open('%s%s' % (settings.MEDIA_ROOT,f.name), 'wb+') as destination:
for chunk in f.chunks():
destination.write(chunk)
def index(request):
if request.method == 'POST':
# POST, generate form with data from the request
form = SharingForm(request.POST,request.FILES)
# check if it's valid:
if form.is_valid():
for field in request.FILES.keys():
for formfile in request.FILES.getlist(field):
save_uploaded_file_to_media_root(formfile)
return HttpResponseRedirect('/about/contact/thankyou')
else:
# GET, generate blank form
form = SharingForm()
return render(request,'social/index.html',{'form':form})
After the form
is_valid() method, you know each of the keys in the
request.FILES dictionary are file form fields, so a
loop is created to get each file field (i.e. video,
photo). Next, using the getlist() method
of the request.FILES dictionary, you obtain the list
of file instances (i.e.
InMemoryUploadedFile,TemporaryUploadedFile) for each
file field and loop over each element to get file instances.
Finally, you execute the
save_uploaded_file_to_media_root() method on each file
instance.
Each file instance -- represented
as the formfile reference in listing 6-37 -- is either
a InMemoryUploadedFile or
TemporaryUploadedFile instance type. But as I
mentioned earlier, these classes are derived from the parent class
django.core.files.uploadedfile.UploadedFile, which
means file instances are also UploadedFile
instances.
By design, the Django
django.core.files.uploadedfile.UploadedFile class has
a series of methods and fields, specifically made to easily handle
the contents of uploaded files, some of which include:
<file_instance>.name.- Outputs the name of the file, as-was on a user's computer.<file_instance>.size.- Outputs the size of the file, in bytes.<file_instance>.content_type.- Outputs the HTTP Content-Type header assigned to the file (e.g. text/html, application/zip).<file_instance>.chunks().- A generator method to efficiently output contents of a file in chunks.
With the use of these
UploadedFile fields and methods, you can see toward
the top of listing 6-37 the
save_uploaded_file_to_media_root() method logic
includes: saving the uploaded file with its original name -- using
Python's standard with...open syntax -- and then using
the chunks() method to efficiently write all the
pieces of the uploaded file to the file system.
Note The uploaded files in listing 6-37 are saved under the settings.MEDIA_ROOT folder. Although you can save files to any location you want, the standard practice for user uploaded files in Django is to save them under the MEDIA_ROOT folder defined in settings.py.