Tag: Django

Populate and Validate a Django ModelForm using API Data with the Generic CreateView

I ran into a slightly tricky use case the other day wherein I needed to grab some data from an API, validate it against a Django ModelForm, and only show the form to the user if there are errors during the validation.

If the form is shown, the error messages from the validation should appear along with a pre-populated form (using the data from the API). Otherwise, the ModelForm should submit and be saved immediately, without any user interaction.

After digging through Django’s code it became clear that to accomplish this, I needed to override a couple methods of the generic class-based CreateView.

Here’s the code for the view…


from django.core.urlresolvers import reverse_lazy
from django.views.generic import CreateView
from app.forms import MyModelForm
from app.lib.api import API
class MyCreateView(CreateView):
template_name = 'form.html'
form_class = MyModelForm
success_url = reverse_lazy('home')
object = None
# allow data to be passed to this
# pulled from django/views/generic/edit.py/ModelFormMixin
def get_form_kwargs(self, data=None):
kwargs = super(MyCreateView, self).get_form_kwargs()
# if we have data, set the form's data so that we can auto-submit this form (without POSTing)
if data:
kwargs.update({'initial': data, 'data': data})
return kwargs
def get(self, request, *args, **kwargs):
# grab the data from the API
data = API.get_data()
formdata = {'user_id': data['user_id'],
'email': data['email'],
'first_name': data['first_name'],
'last_name': data['last_name']}
form_class = self.get_form_class()
# must instantiate the class directly and include the data object rather than use get_form
# will set both the initial and instance data
#form = self.get_form(form_class)
form = form_class(**self.get_form_kwargs(formdata))
# do we have a valid form submission
if form.is_valid():
return self.form_valid(form)
else:
return self.form_invalid(form)
# once the user submits the form, validate the form and create the new user
def post(self, request, *args, **kwargs):
# setup the form
# we can use get_form this time as we no longer need to set the data property
form_class = self.get_form_class()
form = self.get_form(form_class)
# grab the data from the API
data = API.get_data()
form.initial = {'user_id': data['user_id'],
'email': data['email'],
'first_name': data['first_name'],
'last_name': data['last_name']}
if form.is_valid():
return self.form_valid(form)
else:
return self.form_invalid(form)

view raw

view.py

hosted with ❤ by GitHub

As you can see, the first thing I’m doing is overriding get_form_kwargs so that it can take a new argument – data. This allows me to pass in the data pulled from the API to the form. Since I want the form to be saved immediately, without any user interaction, I set both the initial and data properties to the passed in data.

The get method is now a simple exercise in pinging the API for data, placing it into a dictionary the form can use, and calling form.is_valid() to see if the data passes the validation.

If it doesn’t, the form is rendered (complete with validation errors and pre-populated data) and the post method handles when the form is submitted by the user.

Previewing Django Error Pages

A quick hack to preview your Django error pages when DEBUG=True…

Simply add routes to your error pages as part of your urls.py, but only when you’re running in debug mode.


from django.views.generic import TemplateView
if settings.DEBUG:
# enable local preview of error pages
urlpatterns += patterns('',
(r'^403/$', TemplateView.as_view(template_name="403.html")),
(r'^404/$', TemplateView.as_view(template_name="404.html")),
(r'^500/$', TemplateView.as_view(template_name="500.html")),
)

view raw

urls.py

hosted with ❤ by GitHub

Now you can simply browse to /403/, /404/ or /500/ to see what they look like.

Connecting to a Local Django Server from VMware Fusion on OS X

Even though we’d all like to believe that IE is dead and there’s no need to test it any more, that simply isn’t the case yet. But debugging via a remote server is a pain when you can easily use a local Windows VM running on VMware Fusion.

The following will get you up and running with a local Django server (running on port 8000) and any Windows OS. Note that I’m using the latest VMware Fusion (version 6) but the same steps will work with previous versions as well.

First, shutdown the VM (power it off completely) and then open it’s settings. Click Add Device, select Network Adapter and click Add.

Now you’ll see 2 network adapters for the VM. Click the new one to edit it.

Switch the connection type to Private to my Mac.

Next, grab the IP address your Mac is using for this private network. Open a terminal window and run “ifconfig vmnet1”.

$ ifconfig vmnet1
vmnet1: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
ether 00:50:56:c0:00:01
inet 172.16.85.1 netmask 0xffffff00 broadcast 172.16.85.255

The last line is the important one. Make note of the IP address next to inet, in this case mine is 172.16.85.1. Because we made a new private network, this IP should hopefully stay the same and you won’t have to worry about messing with the configuration again down the road.

Next, start your Django server using the IP address 0.0.0.0:

$ python manage.py runserver 0.0.0.0:8000

Now start the VM back up and open up IE or your browser of choice. Go to http://172.16.85.1:8000 (make sure to substitute the IP address you made note of above). Voilà, you’re browsing your local Django server from Windows.

If you’d like to use something more memorable for the host name, go ahead and edit the Windows host file (it can be found here – C:\Windows\system32\drivers\etc\hosts). Add the new host to the list of entries:

127.0.0.1      localhost
172.16.85.1    mysite.local

I found that xxxx.local worked reliably while other host names were hit and miss.

Finally, flush the DNS on your VM for the host changes to take effect:

ipconfig /flushdns

You can now reach your Django site at http://mysite.local:8000.

Django 1.7, Signals & AppConfig

The alpha of Django 1.7 was released recently and I’ve started to experiment with the new applications feature.

Applications include some combination of models, views, templates, template tags, static files, URLs, middleware, etc. They’re generally wired into projects with the INSTALLED_APPS setting and optionally with other mechanisms such as URLconfs, the MIDDLEWARE_CLASSES setting, or template inheritance.

The power of this feature is that it allows you to create an application configuration class which has a ready method, allowing you to perform initialization tasks such as registering signals when the application first loads.

To get started with applications, set the default_app_config variable in your application’s __init__.py:


# myapp/__init__.py
default_app_config = 'myapp.apps.MyAppConfig'

view raw

__init__.py

hosted with ❤ by GitHub

Now create a new file called apps.py in the root of your application and create a class that inherits from AppConfig:


# myapp/apps.py
from django.apps import AppConfig
class MyAppConfig(AppConfig):
name = 'myapp'
verbose_name = 'My App'
def ready(self):
# import signal handlers
import myapp.signals.handlers

view raw

apps.py

hosted with ❤ by GitHub

Within this class you can define a variety of things including the application’s name, verbose name and the ready method. In the example above, I’m importing the signals submodule that contains the signal receivers (in Django 1.6 and below, signal registration usually happened in the models module).

Finally, here’s an example signal and handler from my signals submodule:


# myapp/signals/signals.py
from django.dispatch import Signal
example_signal = Signal(providing_args=['arg1', 'arg2'])

view raw

signals.py

hosted with ❤ by GitHub


# myapp/signals/handlers.py
from django.dispatch import receiver
from .signals import example_signal
@receiver(example_signal)
def example_signal_handler(sender, **kwargs):
print kwargs['arg1']
print kwargs['arg2']

view raw

handlers.py

hosted with ❤ by GitHub

Django Compressor 1.4 Alpha

If you use the popular Django Compressor package, it’s finally getting a few much needed updates including the latest rJSmin, support for rCSSMin, as well as a variety of bug fixes.

I contributed support for yUglify now that YUI Compressor has been officially deprecated (although it lives on under a new contributor model). You can use the new filters like this:


COMPRESS_CSS_FILTERS = ['compressor.filters.yuglify.YUglifyCSSFilter']
COMPRESS_JS_FILTERS = ['compressor.filters.yuglify.YUglifyJSFilter']

view raw

settings.py

hosted with ❤ by GitHub

You can install the latest development version using Pip:

Useful Python and Django Packages – Update

In a follow up to this post, here’s what I’ve been making use of lately:

BeautifulSoup / beautifulsoup4
HTML/XML parser for quick-turnaround applications like screen-scraping.

Pillow
Python Imaging Library (Fork)

Scrapy
A high-level Python Screen Scraping framework

Unipath
Object-oriented alternative to os/os.path/shutil

celery
Distributed Task Queue

django-breadcrumbs
Easy to use generic breadcrumbs system for Django framework.

django-model-utils
Django model mixins and utilities

django-supervisor
easy integration between djangocl and supervisord

django-user-agents
A django package that allows easy identification of visitor’s browser, operating system and device information (mobile phone, tablet or has touch capabilities).

django-widget-tweaks
Tweak the form field rendering in templates, not in python-level form definitions.

factual-api
Official Python driver for the Factual public API

unicode-slugify
A slug generator that turns strings into unicode slugs.

user-agents
A library to identify devices (phones, tablets) and their capabilities by parsing (browser) user agent strings

virtualenv
Virtual Python Environment builder

virtualenvwrapper
Enhancements to virtualenv

More Tools for Debugging Django

As a follow up to this post, here are a couple other highly recommended tools for Django development and debugging.

The first one is the widely recommended django-devserver from David Cramer. It serves as a drop-in replacement for Django’s built-in runserver command with numerous features:

  • An extendable interface for handling things such as real-time logging.
  • Integration with the werkzeug interactive debugger.
  • Threaded (default) and multi-process development servers.
  • Ability to specify a WSGI application as your target environment.

Have a look at the various modules that can be enabled including SQLRealTimeModule, SQLSummaryModule, and AjaxDumpModule among others.

Be sure to note that if you’re using Django 1.6, the current version of devserver (0.7.0) is broken (infinite recursion loop in modules/sql.py). There’s a ticket open with a suggested fix that works for me (change util.CursorDebugWrapper to util.CursorWrapper in sql.py), but no official solution yet.

The second application is django-extensions which offers a ton of useful commands including shell_plus (autoload all your models) and runserver_plus (standard runserver stuff but with the Werkzeug debugger baked in).

It’s worth noting that Werkzeug is also available via devserver but if you’re looking for a quick overview on how to use it, check out the runserver_plus page.

Customize Django’s unique_together Error Message

While you’re able to customize the error message for a unique field when defining your model…


class Context(models.Model):
name = models.CharField(error_messages={'unique': u'My custom message'})

view raw

model.py

hosted with ❤ by GitHub

…you can’t do the same for a unique_together constraint. There’s an open ticket but it hasn’t seen much attention. In the meantime, you can override the unique_error_message in your model to provide a custom error message:


def unique_error_message(self, model_class, unique_check):
if model_class == type(self) and unique_check == ('field1', 'field2'):
return 'Your custom error message.'
else:
return super(YourModel, self).unique_error_message(model_class, unique_check)

view raw

model.py

hosted with ❤ by GitHub

Debugging Django & Ajax

UPDATE: See Peter’s comment below for a slightly revised (and better) version of this.

The Django Debug Toolbar is essential for developing Django applications but it comes short in one area – Ajax. There’s been progress over the last couple years but the latest pull request for a panel that supports Ajax has yet to be completed and merged. This leaves you pretty much in the dark when making XHR requests as the toolbar only works if the mimetype of the response is either text/html or application/xhtml+xml and contains a closing </body> tag (otherwise your XHR requests would be filled with a bunch of junk from the toolbar).

After digging around I stumbled upon this SO post which details a few solutions including using the following middleware:


from django.conf import settings
class JsonAsHTML(object):
'''
View a JSON response in your browser as HTML
Useful for viewing stats using Django Debug Toolbar
This middleware should be place AFTER Django Debug Toolbar middleware
'''
def process_response(self, request, response):
# not for production or production like environment
if not settings.DEBUG:
return response
# do nothing for actual ajax requests
if request.is_ajax():
return response
# only do something if this is a json response
if 'application/json' in response['Content-Type'].lower():
title = 'JSON as HTML Middleware for: %s' % request.get_full_path()
response.content = '<html><head><title>%s</title></head><body>%s</body></html>' % (title, response.content)
response['Content-Type'] = 'text/html'
return response

view raw

json.py

hosted with ❤ by GitHub

Add this middleware right after the debug toolbar in your settings (and ensure that it is only enabled for your dev environment) and then load an Ajax URL directly in a browser tab. The middleware simply detects that you’re not making this request as Ajax and wraps the response in some HTML which will activate the debug toolbar. Very handy.

It’s definitely a bit of a hack and doesn’t make working with posts very easy, but it’s better than nothing.

Django Compressor and yUglify

Like many folks out there I’ve been making use of the YUI Compressor for quite a few years as part of my build process (currently as part of Django Compressor). About a year ago it was announced that YUI Compressor was going to go through a deprecation process and in the months since it has been given a new owner and has even seen an update.

But like the YUI team, I figured it’s probably time to move on as there are lots of new tools available including Closure and UglifyJS. The YUI team has also released a new library wrapper around UglifyJS and cssmin with the default YUI configurations on each of them called yUglify. So if you’re coming from YUI Compressor like myself, this library will keep things working basically the same.

To get started, you’ll first need to install yuglify globally via npm (I’m assuming you already have node and npm):


sudo npm -g install yuglify

view raw

yuglify.sh

hosted with ❤ by GitHub

Next you’ll need to update your Django settings with the following:


COMPRESS_YUGLIFY_BINARY = 'yuglify' # assumes yuglify is in your path
COMPRESS_YUGLIFY_CSS_ARGUMENTS = '–terminal'
COMPRESS_YUGLIFY_JS_ARGUMENTS = '–terminal'

view raw

settings.py

hosted with ❤ by GitHub

Now for the new compressor filter:


from compressor.conf import settings
from compressor.filters import CompilerFilter
class YUglifyFilter(CompilerFilter):
command = "{binary} {args}"
def __init__(self, *args, **kwargs):
super(YUglifyFilter, self).__init__(*args, **kwargs)
self.command += ' –type=%s' % self.type
class YUglifyCSSFilter(YUglifyFilter):
type = 'css'
options = (
("binary", settings.COMPRESS_YUGLIFY_BINARY),
("args", settings.COMPRESS_YUGLIFY_CSS_ARGUMENTS),
)
class YUglifyJSFilter(YUglifyFilter):
type = 'js'
options = (
("binary", settings.COMPRESS_YUGLIFY_BINARY),
("args", settings.COMPRESS_YUGLIFY_JS_ARGUMENTS),
)

view raw

filters.py

hosted with ❤ by GitHub

Finally, go back to your Django settings and update your compressor settings with the new filter. Here are some examples (I’ve placed the filter in app/compressor/filters.py):


# Old
COMPRESS_CSS_FILTERS = ['compressor.filters.css_default.CssAbsoluteFilter', 'compressor.filters.yui.YUICSSFilter']
# New
COMPRESS_CSS_FILTERS = ['compressor.filters.css_default.CssAbsoluteFilter', 'app.compressor.filters.YUglifyCSSFilter']
# Old
COMPRESS_JS_FILTERS = ['compressor.filters.jsmin.SlimItFilter']
# New
COMPRESS_JS_FILTERS = ['app.compressor.filters.YUglifyJSFilter']

view raw

settings.py

hosted with ❤ by GitHub

I’ll see if I can push this upstream to the Compressor folks when I’ve got a few minutes.

I’ve also started looking at moving from Django Compressor to Django Pipeline as it supports yUglify out of the box but I’ll save that for another post.