django-skel: A Skeleton for Django Projects

The adult human body has 206 bones, 28 of which are skull bones, 30 are in our arms and legs, 26 vertebrae, 24 ribs, the hyoid bone, the pelvic bones and the partial bones, and the remaining half is in our hands and feet. These comprise the skeletal system or simply, the skeleton, which supports the structure of the body.

Django-skel is a skeleton and functions the same way the human skeleton does. It provides a structure for your project, and is equipped with most of the tools that your application would need, making it a powerful tool that aids developers throughout their development cycle, which starts from local development up to the point of deployment for production.

I came across this application while leisurely sailing through the world of the internet, and felt the dying need to check it out. I even found out a really cool and inspiring introduction to the project which you can read on this page.

If you are not yet amazed, here is a list of all the benefits that are included in the application, taken from django-skel’s documentation:

  • Database migrations via South.
  • Static file management via django-compressor.
  • Task queueing via Celery.
  • Helper utilities for working on the command line, via Fabric.
  • Fancy documentation generation via Sphinx.
  • Awesome local debugging and analysis via django-debug-toolbar.
  • Amazon S3 integration (for publishing static assets: css, js, images, etc.) via django-storages.
  • CSS compression (for production environments) via cssmin.
  • JS compression (for production environments) via jsmin.
  • Memcache caching support via django-heroku-memcacheify.
  • PostgreSQL support via django-heroku-postgresify.
  • A blazing fast WSGI server for serving production traffic via gunicorn and gevent.
  • Production application performance monitoring and usage statistics via New Relic.
  • All the best practices I’ve come to learn with more than 4 years of Django experience. *
  • Built in support for production deployments on Heroku’s platform.

* This is the developer talking 😀

And here is what the skeleton looks like:
.
├── fabfile.py
├── gunicorn.py.ini
├── manage.py
├── Procfile
├── reqs
│   ├── common.txt
│   ├── dev.txt
│   └── prod.txt
├── requirements.txt
├── woot
│   ├── apps
│   │   └── __init__.py
│   ├── __init__.py
│   ├── libs
│   │   └── __init__.py
│   ├── settings
│   │   ├── common.py
│   │   ├── dev.py
│   │   ├── __init__.py
│   │   └── prod.py
│   ├── templates
│   │   ├── 404.html
│   │   └── 500.html
│   └── urls.py
└── wsgi.py

You can go to this page for a detailed explanation of each component in the layout.

Now, let’s look at each tool separately.

South

Without using a data migration tool, developers might be stuck with the options of dropping tables and constantly running syncdb, or updating tables using SQL commands directly through the database shell. This Django application simplifies the process of making database migrations and schema changes to your Django models.

Applying alterations to your schema takes as easy as two steps:

That’s it! For more information, you can visit the South documentation pages.

Django-compressor

Django compressor, as implied from its name, combines and compresses CSS and Javascript files into cacheable static files. And it does it not only for external files, but for inlines as well! Very cool, huh?

Simply group together your CSS and Javascript files, and enclose them into a special tag they named “compress” for our benefit.

The files you grouped together will automatically be combined in a single compressed file, which may look something like this:

for CSS, or

for Javascript.

Celery

Now why this tool is named after a vegetable is beyond my knowledge, but Celery is one great tool that will rock your world.

As described in Celery’s documentation page, Celery is a simple, flexible and reliable distributed system to process vast amounts of messages, while providing operations with the tools required to maintain such a system. It’s a task queue with focus on real-time processing, while also supporting task scheduling.

Celery is an asynchronous task queue/job queue based on distributed message passing. You basically use Celery to handle time consuming processes in your app. You can also use it to implement platform independent task scheduling.

For a more in-depth information on Celery, check out the post entitled “Django-Celery

Fabric

Most likely, you work on your project locally and push it into your test or production server once it’s ready. The first step that you do is to SSH into the server, then you run a couple of commands such as pulling the most recent changes from your repository, migrating the app, running your tests, and maybe even restarting your server. Now this works and everybody’s fine with it, until they discovered that this whole process can be improved.

With Fabric, you can deploy your application with a single command. What makes it powerful is that it allows you to execute local and remote shell commands, which is perfect for minimizing the steps I enumerated above. Simply create a fabfile.py on your project directory and run it using the fab command line tool.

Here is a sample fabfile for running commands on your local machine. For remote commands, check out the Fabric documentation.

then run it with

Sphinx

As a developer, I make it a point to document my code. Sure, I add some helpful comments and docstrings to classes and functions, but that’s about it. However, with projects where documentation is of utmost importance and people need to read it without actually looking at your code or if by chance you plan to contribute to the open source community by building an awesome package, you would need to write a documentation. This is not necessary, but you know people always do love projects with good documentation.

Sphinx helps you with that. To see Sphinx in action, check out this list of projects using Sphinx, which of course includes Sphinx itself.

Django-debug-toolbar

This app adds a panel, which you can hide or show at will to your application. This panel displays useful information such as current time, http headers, settings, SQL queries, signals sent, logs, etc which are very helpful for debugging. This information is configurable, and some third parties, even provided their own versions which you can check out on this page.

Django-storages

This provides a collection of custom backend storages for Django. Usually these storages are used for storing static and media files, the latter for when your user uploads files to your application.

I have personally used this for Amazon S3, and it worked like wonder. Simply set the DEFAULT_FILE_STORAGE of your settings.py to your backend of choice, say

and Django will use this backend to store the files associated to your defined model fields. To set the storage of your static files, do

CSSMIN

CSSmin is simply a Python port of the YUI Compressor. The process of compressing involves removal of unnecessary spaces, merging and optimizing files to reach the end goal of improving your page’s loading time.

Here is the result of running cssmin –help:

Run it from the command line using the following commands:

This function can also be used from within a Python code.

JSMIN

Unlike CSSmin, JSmin does not provide a command line version. So you need to use it like below:

And write the output to a file you specify. That, or you can provide the command line version yourself. It should be pretty straight forward.

Django-heroku-postgresify

Ever tried setting up your database in two lines?

That’s it. Pretty amazing, huh? Now only if there are other apps out there that does the same. If you have a DATABASE_URL environment variable defined, you will have something like

Not to disappoint you or anything, but this is for PostgreSQL on Heroku only.

Django-heroku-memcacheify

Another two liner code,

that defines the CACHES setting automatically.

Memcached is an Object caching system that uses the system’s memory to reduce the load on your database. Again, take note that this is for Memcache on Heroku.

Gunicorn and Gevent

Gunicorn or Green Unicorn is a Python WSGI HTTP Server for UNIX. Gevent, on the other hand, handles co-routine Async networking. There are many benchmarks and comparison of Gunicorn with the popularly known uWSGI, which provide many details, so I won’t go into the details with these two.

New Relic

New Relic is used to monitor the performance of your applications, as well as its usage statistics. This is the less popular version of the well-known Google Analytics.

Heroku

Django-skel mainly revolves around deployment on Heroku. Heroku uses Dynos, which are isolated, virtual Unix containers, that provide the environment required to run an application. This is somehow like Amazon’s EC2s, although they behave very differently. Heroku also provides a command line tool called the Heroku Toolbelt for managing your apps.

Deployment in Heroku is said to be very easy, with a nice interface, and in which adding services can be done by adding add-ons. The way it works should be alright for most apps, but for those that need finer control, maybe compiling packages from source or running custom binaries, you would have to find a way to hack Heroku.

Scaling is also easy, in that you simply increase the number of web and worker processes that your application has access to, for a reasonable price of course.

 

Faced with the problem of deploying Django projects, programmer Randall Degges developed django-skel as a means to lessen the burden of deploying applications by bundling a set of tools which most programmers would find very helpful. Django-skel might not be that necessary for small-scale projects, but would really be convenient to have for medium to large-scale ones that involve tons of processing and optimization.

This project is a good attempt to put together a collection of tools in Django and simplify deployment. However, one of the drawbacks of this package is that it is revolves mainly around Heroku. While many users are happy with Heroku, it is not a universal solution. In the end, application development is still about preferences.

References / Documentation Links: