Get Django Updates

Django real-time updates with Node.js

Problem

You want to create Django template/client side logic capable of receiving automatic updates (i.e. without user action) when something changes on Django's back end/server side (e.g. new record is created).

Solution

For Django install the Python socketIO-client package to communicate with Node.js/Socket.IO. Use the socketIO-client API to send messages from Django's server side to Node.js. Use the standard Socket.IO JavaScript API in Django templates so Django web pages are able to receive automatic updates from Node.js/Socket.IO. Create a Node.js server to receive/send messages channeled through Django.

Note Backing source code available

The backing source code for the application described next is available at https://github.com/drubio/djangorecipes.

How it works

When Django sends out content (i.e. a rendered template) to end users, the content stay that way until a user makes another request to the server side or if the content is packed with JavaScript AJAX logic, partial content updates are possible contacting the server side once again. In either case, the server side is not capable of exchanging content, unless the client side initiates the process. This is how things work for most web applications, as described in the previous recipe What are real-time web applications and what's required to create them in Django. However, it's possible to enable the server side to communicate with the client side if you introduce an additional component to a Django application: Node.js -- note the previous recipe also contains a brief conceptual introduction to Node.js if you're unfamiliar with this platform.

Figure 1 illustrates the architecture/workflow for the implementation described in the remainder of this recipe, which is to enable real-time updates for a Django application with Node.js.

Figure 1 - Django real-time updates with Node.js architecture/workflow
Django real-time updates with Node.js

Django setup and application design

The first step to use Django with Node.js is to install the Python socketIO-client package (e.g. pip install socketIO-client). The socketIO-client package is the Python library needed to communicate with Socket.IO which is a Node.js framework. This is the only Python package that you'll need to create the design illustrated in figure 1.

Once you install the required Python package, I recommend you create a variable in settings.py to define the location where Node.js will run, like NODEJS_SOCKET_URL = 'http://localhost:3000'. Alternatively you can hardcode the Node.js location in your Django code, but I recommend you use this type of variable because it makes future changes easier.

Next, let's take a look at the Django model that interacts with Node.js via Socket.IO. Listing 1 illustrates this model.

Listing 1 - Django model that when new record is created communicates with Node.js via Socket.IO


from socketIO_client import SocketIO
from urlparse import urlparse
from django.conf import settings
import json

class Special(models.Model):
    start = models.DateField()
    end = models.DateField()
    description = models.CharField(max_length=500)
    def __str__(self):
        return u"%s (%s,%s)" % (self.description, self.start, self.end)
    def save(self, *args, **kwargs):        
        # Check if the Special already exists
        if not self.pk:
            # No pk so Speical is new, push update to node.js socket
            # Parse the NODEJS_SOCKET_URL value
            parsedurl = urlparse(settings.NODEJS_SOCKET_URL)
            baseurl = "%s://%s" % (parsedurl.scheme,parsedurl.hostname)
            baseport = parsedurl.port if parsedurl.port != None else 80
            # Publish record to node.js socket
            with SocketIO(baseurl, baseport) as socketIO:
                socketIO.emit('admindailyspecials', {"special":{'description':self.description,'start':self.start.strftime("%B %d"),'end':self.end.strftime("%B %d")}})
        super(Special, self).save(*args, **kwargs)

The Django model in listing 1 is pretty basic, except for the fact that it uses custom logic in its save method. As illustrated in previous recipes, by overriding a Django model's save method you can provide custom logic besides the default behavior of saving the record to the database. In the case of listing 1, we first check the instance pk value, if it doesn't exist it means it's a new record and so we first communicate the record to Node.js. Note that you can place the Node.js/Socket.IO logic anywhere in your Django project, but in this case I opted to use a model's save() method because it's handy location so that irrespective of where the Special record is created, the save takes care of always triggering the same Node.js logic.

The first part of the Node.js/Socket.IO logic in listing 1 is to parse the URL endpoint defined in the NODEJS_SOCKET_URL variable from settings.py -- if you didn't define a variable for this purpose you can hardcode the Node.js location here. Next, we simply use the emit method from Socket.IO to publish the record to Node.js, where the first argument admindailyspecials represents the namespace/channel to send the data and the second argument is a JSON representation of the record. Finally, a call is made to the save method using super so the record maintains its default behavior of saving the record to the database.

Summarizing, listing 1 achieves steps 5 and 6 in figure 1, where each time a new Special record is created in Django it's also sent to Node.js on the admindailyspecials namespace/channel.

Next, let's take a look at what you need to incorporate into a Django template that is to receive Node.js updates. Listing 2 illustrates the JavaScript snippet required for this process.

Listing 2 - Django template JavaScript required to receive updated from Node.js via Socket.IO

<head>
<script src="{{NODEJS_SOCKET_URL}}/socket.io/socket.io.js"></script>
<script>
  var socket = io.connect('{{NODEJS_SOCKET_URL}}');
  socket.on('dailyspecials', function (data) {
    console.log(data);
    // Check if the data has a special attribute to see if we publish to feed
    if (data.message.special) { 
       console.log(data.message.special);
       // Append to specials-list
       #specials-list.prepend("<li class='list-group-item' style='background-color: #337ab7;'><b>" + data.message.special.description+ "</b><br/>Start: " + data.message.special.start +"<br/>End: " + data.message.special.end +"</li>");
     }
   });
</script>
</head>

The first step in listing 2 is to give the web page access to the Socket.IO library installed in Node.js -- note the template has access to the NODEJS_SOCKET_URL variable in settings.py. The second step in listing 2 is to establish a connection to Node.js via Socket.IO with io.connect.

Once the Node.js connection is established, the socket.on method indicates that when the dailyspecials namespace/channel emits a message it be processed with the following logic: it first outputs the message to the browser console and if the message has the special attribute it prepends the message in an HTML format to the special-list element -- note this last snippet relies on jQuery to do the prepend operation.

To give you a better idea of the user interface associated with listing 2, figure 2 illustrates the special-list element on the right hand side and toward the bottom you can see the JavaScript console with sample messages generated with the logic from listing 2.

Django rendered template with automatic updates received from Node.js
Figure 2.- Django rendered template with automatic updates received from Node.js

The feed on the right hand-side of figure 2 (i.e. the special-list element) is automatically updated every time data is published to the Node.js dailyspecials namespace/channel, due to the JavaScript snippet in listing 2. And as you might already imagine, the dailyspecials namespace/channel receives messages that are associated with the admindailyspecials namespace/channel which are triggered when a new Django model Special record is created -- this last sequence might sound confusing, but it will become clearer in the next section that describes the Node.js setup and application design.

Node.js setup and application design

To set up Node.js in this application you'll need to install the Socket.IO package (e.g. npm install socket.io) and Express package (e.g. npm install express). Although we'll use minimal parts of these last Node.js packages, they make it easier to get up and running with Node.js, as well as make the Node.js design more understandable. Listing 3 illustrates the Node.js app that integrates with the previous Django snippets.

Listing 3 - Node.js app configured to listen and publish messages to Django

var app = require('express')();
var server = require('http').Server(app);
var io = require('socket.io')(server);

server.listen(3000);

io.on('connection', function (socket) {
  console.log("Client just connected to node.js");
  socket.emit('dailyspecials', { message: 'You just connected to the daily specials channel, one second while we get the specials!' });
  socket.on('admindailyspecials', function (data) {
      console.log("Client just sent admindailyspecials data!");
      socket.broadcast.emit('dailyspecials', { message: data});
  });
});

The first three lines in listing 3 create a Node.js server relying on the Express package and also imports the Socket.IO package. Next, the server.listen line configures Node.js to listen on port 3000. Then you can see the io.on method executes a series of steps when Node.js receives a connection. The first step is to output a connection message to the server side console. Next, it emits a welcome-type JSON message to all clients listening on the dailyspecials namespace/channel -- this would be the same channel used by the JavaScript snippet in listing 2 and that you can also confirm in the JavaScript client side console in figure 2.

Next, when the Node.js server receives a message on the admindailyspecials namespace/channel -- which is the same channel used by the Django model save method in listing 1 -- it outputs a connection message to the server side console and in addition, broadcasts the message out to all clients listening on the dailyspecials namespace/channel -- which represents the same channel used by the JavaScript snippet in listing 2 and that you can also confirm in the JavaScript client side console in figure 2.

After you place the contents of listing 3 in a file named app.js and start it up with Node.js (e.g. node app.js), a Django application using the design from listings 1 and 2 is capable of receiving real-time updates.