Building a Custom Block Template Tag

Django Documentation

There are a couple of pages in the Django documentation that you should at least scan before continuing, and will likely want to consult while reading:

• Custom template tags and filters is the place to start. It documents the general approach and the machinery we use for writing custom tags.
• The Django template language: for Python programmers has two sections of interest to us: Rendering a context and Playing with Context objects.

What our example tag will do

Let's write a tag that can make simple changes to its content, changing occurrences of one string to another. We'll call it replace, and usage might look like this:

{% replace old="dog" new="cat" %}
My dog is great.  I love dogs.
{% endreplace %}

which would end up rendered as My cat is great.  I love cats..

We'll also have an optional numeric argument to limit how many times we do the replacement:

{% replace 1 old="dog" new="cat" %}
My dog is great.  I love dogs.
{% endreplace %}

which we'll want to render as My cat is great. I love dogs..

Parsing the template

The first thing we'll write is the compilation function, which Django will call when it's parsing a template and comes across our tag. Conventionally, such functions are called do_<tagname>. We tell Django about our new tag by registering it:

from django import template

register = template.Library()

def do_replace(parser, token):
  pass

register.tag('replace', do_replace)

We'll be passed two arguments, parser which is the state of parsing of the template, and token which represents the most recently parsed token in the template - in our case, the contents of our opening template tag. For example, if a template contains {% replace 1 2 foo='bar' %}, then token will contain "replace 1 2 foo='bar'".

To parse that token, I ended up writing the following method as a general-purpose template tag argument parser:

from django.template.base import FilterExpression, kwarg_re

def parse_tag(token, parser):
    """
    Generic template tag parser.

    Returns a three-tuple: (tag_name, args, kwargs)

    tag_name is a string, the name of the tag.

    args is a list of FilterExpressions, from all the arguments that didn't look like kwargs,
    in the order they occurred, including any that were mingled amongst kwargs.

    kwargs is a dictionary mapping kwarg names to FilterExpressions, for all the arguments that
    looked like kwargs, including any that were mingled amongst args.

    (At rendering time, a FilterExpression f can be evaluated by calling f.resolve(context).)
    """
    # Split the tag content into words, respecting quoted strings.
    bits = token.split_contents()

    # Pull out the tag name.
    tag_name = bits.pop(0)

    # Parse the rest of the args, and build FilterExpressions from them so that
    # we can evaluate them later.
    args = []
    kwargs = {}
    for bit in bits:
        # Is this a kwarg or an arg?
        match = kwarg_re.match(bit)
        kwarg_format = match and match.group(1)
        if kwarg_format:
            key, value = match.groups()
            kwargs[key] = FilterExpression(value, parser)
        else:
            args.append(FilterExpression(bit, parser))

    return (tag_name, args, kwargs)

Let's work through what that does.

Calling split_contents() on the token is like calling .split(), but it's smart about quoted parameters and will keep them intact. We get back args, a list of strings representing the parts of the template tag invocation, very much like sys.argv gives us for running a program, except that no quotation marks have been stripped away.

The first element in args is our template tag name itself. We remove it because we don't really need it for parsing the args, but save it for generality.

Next we work through the arguments, using the same regular expression as Django's template library to decide which arguments are positional and which are keyword arguments.

The regular expression for keyword arguments also splits on the =, so we can extract the keyword and the value.

We'd like our argument values to support literal values, variables, and even applying filters. We can't actually evaluate our arguments yet, since we're just parsing the template and don't have any particular template context yet where we could look for things like variables. What we do instead is construct a FilterExpression for each one, which parses the syntax of the value, and uses the parser state to find any filters that are referred to.

When all that is done, this method returns a three-tuple: (<tagname>, <args>, <kwargs>).

Our replace tag has two required kwargs and an optional arg. We can check that now:

from django.template import TemplateSyntaxError

# ...

def do_replace(parser, token):
    tag_name, args, kwargs = parse_tag(token, parser)

    usage = '{% {tag_name} [limit] old="fromstring" new="tostring" %} ... {% end{tag_name} %}'.format(tag_name=tag_name)
    if len(args) > 1 or set(kwargs.keys()) != {'old', 'new'}:
        raise TemplateSyntaxError("Usage: %s" % usage)

Note again how we haven't hardcoded the tag name.

Let's pull our limit argument out of the args list:

if args:
    limit = args[0]
else:
    limit = FilterExpression('-1', parser)

If no limit was supplied, we default to -1, which will indicate later that there's no limit. We wrap it in a FilterExpression so we can just call limit.resolve(context) without having to check whether limit is a FilterExpression or not.

We can't check the values here. They might depend on the context, so we'll have to check them at rendering time.

This is all similar to what we might do if we were writing a non-block tag without using any of the helpful decorators that hide some of this detail. But now we need to deal with some unknown amount of template following our opening tag, up to our closing tag. We need to ask the template parser to process everything in the template until we get to our closing tag:

nodelist = parser.parse(('end_replace',))

We get back a NodeList object (django.template.NodeList), which represents a list of template "nodes" representing the parsed part of the template, up to but not including our end tag.

We tell the parser to just ignore our end tag, which is the next token:

parser.delete_first_token()

Now we're done parsing the part of the template from our opening tag to our closing tag. We have the arguments to our tag in limit and kwargs, and the parsed template between our tags in nodelist.

Django expects our function to return a new node object that stores that information for us to use later when the template is rendered. We haven't written the code for our node object yet, but here's how our parsing function will end:

return ReplaceNode(nodelist, limit=limit, old=kwargs['from'], new=kwargs['to'])

Reviewing what we've done so far

Each time Django comes across {% replace ... %} while parsing a template, it calls do_replace(). We parse all the text from {% replace ... %} to {% endreplace %} and store the result in an instance of ReplaceNode. Later, whenever Django renders the parsed template using a particular context, we'll be able to use that information to render this part of the template.

The node

Let's start coding our template node. All we need it to do so far is to store the information we got from parsing part of the template:

from django import template

class ReplaceNode(template.Node):
    def __init__(self, nodelist, limit, old, new):
        self.nodelist = nodelist
        self.limit = limit
        self.old = old
        self.new = new

Rendering

As we've seen, the result of parsing a Django template is a NodeList containing a list of node objects. Whenever Django needs to render a template with a particular context, it calls each node object, passing the context, and asks the node object to render itself. It gets back some text from each node, concatenates all the returned pieces of text, and that's the result.

Our node needs its own render method to do this. We can start with a stub:

class ReplaceNode(template.Node):
  ...
  def render(self, context):
    ...
    return "result"

Now, let's look at those arguments again. We've mentioned that we couldn't validate their values before, because we wouldn't know them until we had a context to evaluate them in.

When we code this, we need to keep in mind Django's policy that in general, render() should fail silently. So we program defensively:

class ReplaceNode(template.Node):
  ...
  def render(self, context):
      # Evaluate the arguments in the current context
      try:
          limit = int(self.limit.resolve(context))
      except (ValueError, TypeError):
          limit = -1

      from_string = self.old.resolve(context)
      to_string = conditional_escape(self.new.resolve(context))
      # Those should be checked for stringness. Left as an exercise.

Also note that we conditionally escape the replacement string. That might have come from user input, and can't be trusted to be blindly inserted into a web page due to the risk of Cross Site Scripting.

Now we'll render whatever was between our template tags, getting back a string:

content = self.nodelist.render(context)

Finally, do the replacement and return the result:

content = mark_safe(content.replace(from_string, to_string, limit))
return content

We've escaped our own input, and the block contents we got from the template parser should already be escaped too, so we mark the result safe so it won't get double-escaped by accident later.

Conclusion

We've seen, step by step, how to build a custom Django template tag that accepts arguments and works on whole blocks of a template. This example does something pretty simple, but with this foundation, you can create tags that do anything you want with the contents of a block.

If you found this post useful, we have more posts about Django, Python, and many other interesting topics.