haystack

Search is an essential tool for any website, and is a feature often requested by both site owners and users. While the built-in Django QuerySet system will often suffice, sometimes you’ll just need a more powerful, and more generic search system. It’s also necessary for features like AJAX-powered automatic search suggestions.

Enter Haystack.

Getting Started (http://haystacksearch.org/)

As you’ll see in the Haystack homepage:

1. Get the most recent source.
2. Add haystack to your INSTALLED_APPS.
3. Create search_indexes.py files for your models.
4. Setup the main SearchIndex via autodiscover.
5. Include haystack.urls to your URLconf.
6. Search!

0. I’m using a vanilla Django 1.2.0 install, in a clean virtual environment, using a new project named ‘haystack_test’, with an app ‘myapp’, with this model (from the haystack tutorial)

class Note(models.Model):
    user = models.ForeignKey(User)
    pub_date = models.DateTimeField()
    title = models.CharField(max_length=200)
    body = models.TextField()

    def __unicode__(self):
        return self.title

1. Install Haystack (and your preferred search engine) via pip.

>> #(for simplicity, I installed Whoosh)
>> pip install whoosh
>> pip install django-haystack

2. You’ll need to do some more than adding haystack to installed apps.

#in settings.py
HAYSTACK_SITECONF = 'haystack_test.search_sites'
HAYSTACK_SEARCH_ENGINE = 'whoosh

#In ‘search_sites.py’, include the text:

import haystack
haystack.autodiscover() # this makes haystack automatically look for search_indexes.py files in your apps.

(Since I opted to use whoosh, I had to set HAYSTACK_WHOOSH_PATH)

HAYSTACK_WHOOSH_PATH = '/home/whoosh/mysite_index' # up to you where you want to place it.

That’s it for configuring (for whoosh)

3-4. For these steps, think django admin files.

You’ll need to make a search_indexes.py (as mentioned earlier)

import datetime
from haystack.indexes import *
from haystack import site
from myapp.models import Note

class NoteIndex(SearchIndex):
    text = CharField(document=True, use_template=True)
    author = CharField(model_attr='user')
    pub_date = DateTimeField(model_attr='pub_date')

    def get_queryset(self):
    """Used when the entire index for model is updated."""
        return Note.objects.filter(pub_date__lte=datetime.datetime.now())

site.register(Note, NoteIndex)

Then, make a file in SOME_TEMPLATE_DIR/search/indexes/myapp/ named note_text.txt

{{ object.title }}
{{ object.user.get_full_name }}
{{ object.body }}

5. Set up the URL

(r'^search/', include('haystack.urls')),

6. There’s a few additional things before the search will become available.

•  you’ll need data to search against, so give the shell a go.

>>>from myapp.models import Note
>>>from django.contrib.auth.models import User
>>>import datetime
>>>root = User.objects.all()[0]
>>>Note.objects.all().delete()
>>>Note.objects.create(user = root, pub_date = datetime.datetime.now(), title = 'test1', body='body1')
>>>Note.objects.create(user = root, pub_date = datetime.datetime.now(), title = 'test2', body='body2')
>>>Note.objects.create(user = root, pub_date = datetime.datetime.now(), title = 'test3', body='body3')
>>>Note.objects.create(user = root, pub_date = datetime.datetime.now(), title = 'test4', body='body1 body2')
>>>Note.objects.all()

• Then you’ll need to rebuild the index.

python manage.py rebuild_index

Or, if you already have an existing index,

python manage.py update_index

•  you’ll need a template with the basic search functionality, so put this

in SOME_TEMPLATE_DIR/search/search.html

{% extends 'base.html' %}

{% block content %}
    <h2>Search</h2>
    <form method="get" action=".">
        <table>
        {{ form.as_table }}
            <tr>
                <td>&nbsp;</td>
                <td>
                    <input type="submit" value="Search">
                </td>
            </tr>
        </table>

       {% if query %}
            <h3>Results</h3>
            {% for result in page.object_list %}
            <p>
                <a href="{{ result.object.get_absolute_url }}">{{ result.object.title }}</a>
            </p>
            
                <p>No results found.</p>
            {% endfor %}

            {% if page.has_previous or page.has_next %}
                <div>
                    {% if page.has_previous %}<a href="?q={{ query }}&amp;page={{ page.previous_page_number }}">{% endif %}&laquo; Previous{% if page.has_previous %}</a>{% endif %}

                    {% if page.has_next %}<a href="?q={{ query }}&amp;page={{ page.next_page_number }}">{% endif %}Next &raquo;{% if page.has_next %}</a>{% endif %}
                </div>
           {% endif %}
      {% else %}
           ;
      {% endif %}
    </form>

{% endblock %}

== Features ==

Here’s a rundown of some Haystack features:

Haystack allows you to swap between different database engines – similar to how Django allows you to swap between SQLite, PostgreSQL, and MySQL.

In terms of usability, Haystack should feel very familiar to Django users. The SearchIndex has a style similar to the Django Admin style, allowing you to use similar programming patterns you’ve already grown accustomed to.

It is common for search systems to provide a document field that’s filled with data from other fields. This document field uses the same syntax that is used for templates, which should also feel familiar.

Haystack is incremental. First by allowing search engines that are easier to configure, then by allowing you to initially use built-in search views and forms while still allowing you to transition to your own custom views and forms.

Haystack is fast. Not just in terms of execution, but also in terms of development time. Incremental development also means incremental learning, letting you come up with a quick prototype that’s easy to improve upon. That’s also good for clients because they see results earlier.

Haystack was designed with the Django user in mind. You should be right at home.

A word of warning: while you’ll find haystack easy to deploy, you’ll need a lot more background for the more heavyweight search engines that you’ll eventually use with Haystack. But that’s a story for another day.

sources:

• http://haystacksearch.org/
• http://stackoverflow.com/questions/967434/django-haystack-and-whoosh