Solving Django’s GenericForeignKey Conundrum: Introducing “ContextualModel”
As one does occasionally, I came across an instance of needing to relate an unknown number of models to another unknown number of models. The solution that many Django developers usually jump to in this case is Django’s GenericForeignKey. This feature allows us to utilize the existing content_types table and a small bit of code to do exactly what I needed: associate a model with another unknown model. While this solution works well and plays nicely with the Django ORM, there are some downsides to this method. The ORM is doing a bit of “magic” behind the scenes that involves multiple database queries. On top of that, any analyst trying to sift through your database will need to figure out how the “magic” is done and incorporate that into their queries, adding extra work to their analysis. And even worse, if you app relies heavily on GenericForeignKey and you don’t happen to be around to explain the magic to them, your analyst is going to have a bad day.
If you are like me, in this same situation, I recommend you read this article. Luke Plant does a terrific job on explaining the nuances of GenericForeignKey and supplying several alternatives. I ultimately went with Alternative 2: an intermediate table with nullable fields. I felt this was a good balance of ease-of-use and database design. I had a nice owner ForeignKey field and I could abstract away some of the extra logic needed at the model level, like the joins and integrity constraints.
It seemed a daunting task, implementing my own solution instead of using something from Django that has existed for years. But I did it, and I call my solution “ContextualModel”.
The Conversation Hub
My case began when I was developing a feature in one of our Django apps, a feature we call Conversation Hub. Throughout the app, there were a number of objects that we wanted our users to be able to converse about. One such object was called Activity. When a user visited an Activity, they could view information about the Activity, and below that information would be an instance of a Conversation Hub, where users could post about the Activity and comment on each other’s posts. A feature like this is easy enough to build if it’s connected to a single type of object, but what if we wanted to connect it to multiple objects? And what if the number of objects we used it on increased in the future? How would we attach it to new objects?
I needed the ability to connect an instance of Conversation Hub to an unknown number of objects. And I needed to develop it in a way that made it “plug-and-play”, meaning I could drop in a few lines of code into any Django template and it would work immediately. This would be vital to minimize work in the future, when we went about connecting Conversation Hubs to new objects.
The backbone of our Conversation Hub is the Post model. Because Conversation Hubs could be attached to any object, a Post needed to be as well. This was the first time I needed my ContextualModel.
The Solution, Part 1: Context
We will need two models here, Context and ContextualModel. The first is the intermediate table that holds the references, and the second is the model that will be connecting to another model.
Here we have five different model types that can be “contexts” for another object. For example, an Activity can be a context of a Post. Post can also be a context, usually for a CHComment. And a CHComment can be a context for another CHComment.
Now for the rest of the model logic:
We implement some basic cleaning for insuring integrity before saving. And in get_or_create(), we define the way we define a context on an object. It’s similar to how a connection is defined with GenericForeignKey. We supply context_type which is the class name of the object we are passing as the context. Then context is the object itself, or its id.
# activity is an Activity object
ctx = Context.get_or_create('Activity', activity)The full implementation of this model can be found here. It also includes a helper function that we’ll be using later.
Now let’s look at the other half of this.
The Solution, Part 2: ContextualModel
This is the abstract model that we’ll use whenever we want to use this “context” interface.
Here is the first part. BaseModel just refers to a model higher-up in the file that includes some common fields; you can just inherit from models.Model
That kind of looks like the GenericForeignKey implementation, doesn’t it? You’re not wrong. For the context we store the type of object and a reference to the object itself. I wanted this to feel familiar but still keep all the benefits that we’re building into this.
The key to driving ContextualModel is this “fake” field with the context_ prefix. Any time we are creating an instance of ContextualModel, we need to tell it what the “context” object is (I apologize in advance for making you sick of the word “context”).
Here’s how it works. Say we have a Post object that extends ContextualModel. The context for this Post is an Activity object. When we create the Post, we pass in the parameter context_activity and give it the Activity object.
# activity is an Activity object
# current_user is a User objectp = Post.create(context_activity=activity, \
title='A Wonderful Post!', \
body='This is such a wonderful post! Do you agree?', \
author=current_user)
The logic for making this happen is here:
By using Context.get_or_create() we ensure that there is always a Context object, thus enforcing our integrity constraint.
We also have a nifty get_by_context() method that allows us to easily retrieve objects based on their context.
# activity is an Activity object
post_list = Post.get_by_context(activity)The magic
My favorite part about this? The last bit of code here gives us some Django-level magic that will let us access an object’s context without having to query for it directly:
When the Django ORM gives us an instance of our ContextualModel, _context_ref is a reference to a Context instance (our intermediate model that we created earlier). Since the object type is stored on our ContextualModel instance as _context_type, with a little bit of magic, we can now directly access the context object!
# current_user is a User objectactivity = Activity.create(title='An Awesome Activity', \
body='Lorem ipsum dolor sit amet something something idk', \
author=current_user)post = Post.create(context_activity=activity, \
title='Interesting...', body='That looks a lot like Greek *wink*', \
author=current_user)print(post.context.title) # 'An Awesome Activity'
Take that, Django! I’m a wizard too!
Now when you are creating a model that will need this context interface, you can just inherit from ContextualModel:
class Post(ContextualModel):
"""
A model for writing things about things
""" # fields go here
The full implementation of the ContextualModel model can be found here.
Wrapping up: caveats
One small caveat with this is that if you are using Django Forms to create ContextualModel-based objects, you will need to write some additional code in your view functions that 1) pop off the context stuff from request.POST before instantiating the form, 2) saves the form without committing to the database, 3) adds the context stuff onto the new object, then 4) saves the object.
If you are editing a ContextualModel-based instance, you will still need to pop off the context stuff from request.POST before instantiating the form. However, you don’t need to supply it again, since the context already exists.
Another hiccup might come along when you need to retrieve objects that have more than one level of context. I encountered this when I created a Like object. Likes have a Post as a context, and a Post has a number of possible contexts (we’ll use Activity here for simplicity). When viewing an Activity page, I need to retrieve all Posts on that Activity and also all Likes for those Posts. This is all necessary for the Conversation Hub feature I mentioned at the beginning.
To retrieve the list of Likes, I ended up with some code that looks like this:
It’s… bleh. It’s been fast enough for grabbing likes in our app so I’m keeping it for now. I utilize some of Django’s query caching but it could probably use some further optimization.
I hope this helps you break free from Django’s GenericForeignKey! I think there are valid uses for it but in my use case, this ContextualModel interface has worked beautifully!
