How to Simplify Django Migrations and Deployment

Broken Columns

Suppose you've removed one or more fields from Django models, maybe because you're cleaning up old code and they are no longer needed. Whatever the reason may be, Django will create migrations to remove those columns from their database tables. As soon as you run those migrations on any server, those columns will be gone from the database. This can be problematic because any servers running the older code that still references those columns will break.

The Workaround

There’s an interesting way to keep those columns from breaking.

Using django-deprecate-fields makes it safe to migrate the database while all servers are still running old code.

Instead of deleting the field from your model code, you wrap its definition in deprecate_field() like this:

name = deprecate_field(models.CharField(max_length=50))

Then, makemigrations. deprecate_field will modify the field definition to be nullable and Django will create a migration to make the column nullable in the database.

After doing so, the column will still be there, so old code will still work even after the migration has run. But it's now nullable, so if new code doesn't set a value on the field, that will work too.

However, deprecate_field behaves differently when not using Django migration commands: it changes the field to a dummy field that emits warnings whenever it is written to or read from.

In particular, when running tests or serving user requests, that field will no longer access the database. And due to the warnings, if your tests have good coverage, this should help spot any lingering references to the field being removed, so they can be fixed in the new code.

When it's time to deploy, apply the migration as early as possible, so you won’t have servers running the new code until after the migration has run, which is generally what we want anyway.

Considerations

Now, let's consider the possible scenarios.

• If the migration has not been applied and a server is running old code, that's the starting state and presumably everything is working.
• We're deliberately avoiding the state where the migration has not been applied and new code is running (by making sure we run the migration as early as possible).
• If the migration has been applied and a server is still running old code, the column is still in the database, so the old code will still work fine.
• If the migration has been applied and a server is running new code, the column is still in the database, but the new code does not read or write it. Since the column has been changed to be nullable (if it wasn't already), then it's okay to update records without specifying a value for the deprecated field.

When you're confident all servers have been updated and you won't need to roll the changes back, you can just delete the line with the deprecated field from your model, and run makemigrations.

The result will be that Django will generate a migration that removes the column from the database, and the next time you deploy, the unused column will be gone.

Adding Fields to Django Models can have Similar Pitfalls

If the new field is not nullable and has no default value in the database, then once the migration has run, there's a new column in the table, but any old code will not know to provide values for it, and database updates will fail.

Setting Default Value

Using django-add-default-value helps with this problem by making it easy to set a default on the new column in the database.

Keep in mind that Django doesn't already set default values on fields in the database, especially if you've specified a default value in your field definition.

As of Django 3.1.5, it is still the case that setting a default in your Django field definition only results in Django using that value when creating or updating records at run-time, but does not make that the default value for that column in the database.

Maybe one of these days Django will gain this ability. For now though, using django-add-default-value makes it easy to do it yourself from your migration.

To use django-add-default-value, first add your new field to your model and run makemigrations. Then edit the new migration. After the migration step to add the new field, add a new migration step provided by django-add-default-value to set the default value in the database:

+ from django_add_default_value import AddDefaultValue
+
 operations = [
     migrations.AddField(
         field=models.CharField(default='my_default', max_length=255),
         model_name='my_model'  ,
         name='my_field',
     ),
+    AddDefaultValue(
+        model_name='my_model',
+        name='my_field',
+        value='my_default'
+    )
 ]

Now as soon as the migration runs to create the new column, a default value will be set on it.

Let's go through the possible scenarios again.

• If the migration has not been applied and a server is running old code, that's the starting state and presumably everything is working.
• We're deliberately avoiding the state where the migration has not been applied and new code is running.
• If the migration has been applied and a server is still running old code, then the server won't ask for the value of the new column when querying the database, which is fine. If the server tries to create or update records, it won't specify a value for the new field, but the database has a default value to use in that case and it will work.
• If the migration has been applied and a server is running new code, we're at our finish state, with new code using the new field.

Unlike when using django-deprecate-fields, there's no cleanup needed after you've deployed.

That's it! If you have additional ideas and/or workarounds, we’d love to hear about them in the comments.