Dan Poirier
In this post, I review some reasons why it's really difficult to program correctly when using times, dates, time zones, and daylight saving time, and then I'll give some advice for working with them in Python and Django. Also, I'll go over why I hate daylight saving time (DST).
TIME ZONES
Let's start with some problems with time zones, because they're bad enough even before we consider DST, but they'll help us ease into it.
Time Zones Shuffle
Time zones are a human invention, and humans tend to change their minds, so time zones also change over time.
Many parts of the world struggle with time changes. For example, let's look at the Pacific/Apia time zone, which is the time zone of the independent country of Samoa. Through December 29, 2011, it was -11 hours from Coordinated Universal Time (UTC). From December 31, 2011, Pacific/Apia became +13 hours from UTC.
What happened on December 30, 2011? Well, it never happened in Samoa, because December 29, 23:59:59-11:00 is followed immediately by December 31, 0:00:00+13:00.
| Date | Time | Zone | Date | Time | Zone |
|---|---|---|---|---|---|
| 2011-12-29 | 23:59:59 | UTC-11 | 2011-12-30 | 10:59:59 | UTC |
| 2018-12-31 | 00:00:00 | UTC+13 | 2011-12-30 | 11:00:00 | UTC |
That's an extreme example, but time zones change more often than you might think, often due to changes in government or country boundaries.
The bottom line here is that even knowing the time and time zone, it's meaningless unless you also know the date.
Always Convert to UTC?
As programmers, we're encouraged to avoid issues with time zones by "converting" times to UTC (Coordinated Universal Time) as early as possible, and convert to the local time zone only when necessary to display times to humans. But there's a problem with that.
If all you care about is the exact moment in the lifetime of the universe when an event happened (or is going to happen), then that advice is fine. But for humans, the time zone that they expressed a time in can be important, too.
For example, suppose I'm in North Carolina, in the eastern time zone, but I’m planning an event in Memphis, which is in the central time zone. I go to my calendar program and carefully enter the date and "3:00 p.m. CST". The calendar follows the usual convention and converts my entry to UTC by adding 6 hours, so the time is stored as 9:00 p.m. UTC, or 21:00 UTC. If the calendar uses Django, there's not even any extra code needed for the conversion, because Django does it automatically.
The next day I look at my calendar to continue working on my event. The event time has been converted to my local time zone, or eastern time, so the calendar shows the event happening at "4:00 p.m." (instead of the 3:00 p.m. that it should be). The conversion is not useful for me, because I want to plan around other events in the location where the event is happening, which is using CST, so my local time zone is irrelevant.
The bottom line is that following the advice to always convert times to UTC results in lost information. We're sometimes better off storing times with their non-UTC time zones. That's why it's kind of annoying that Django always "converts" local times to UTC before saving to the database, or even before returning them from a form. That means the original timezone is lost unless you go to the trouble of saving it separately and then converting the time from the database back to that time zone after you get it from the database. I wrote about this before.
By the way, I've been putting "convert" in scare quotes because talking about converting times from one time zone to another carries an implicit assumption that such converting is simple and loses no information, but as we see, that's not really true.
DAYLIGHT SAVING TIME
Daylight saving time (DST) is even more of a human invention than time zones.
Time zones are a fairly obvious adaptation to the conflict between how our bodies prefer to be active during the hours when the sun is up, and how we communicate time with people in other parts of the world. Historical changes in time zones across the years are annoying, but since time zones are a human invention it's not surprising that we'd tweak them every now and then.
DST, on the other hand, amounts to changing entire time zones twice every year. What does US/eastern time zone mean? I don't know, unless you tell me the date. From January 1, 2018 to March 10, 2018, it meant UTC-5. From March 11, 2018 to November 3, 2018, it meant UTC-4. And from November 4, 2018 to December 31, 2018, it's UTC-5 again.
But it gets worse. From Wikipedia:
The Uniform Time Act of 1966 ruled that daylight saving time would run from the last Sunday of April until the last Sunday in October in the United States. The act was amended to make the first Sunday in April the beginning of daylight saving time as of 1987. The Energy Policy Act of 2005 extended daylight saving time in the United States beginning in 2007. So local times change at 2:00 a.m. EST to 3:00 a.m. EDT on the second Sunday in March and return at 2:00 a.m. EDT to 1:00 a.m. EST on the first Sunday in November.
So in a little over 50 years, the rules changed 3 times.
Even if you have complete and accurate information about the rules, daylight saving time complicates things in surprising ways. For example, you can't convert 2:30 a.m. March 11, 2018. in US/eastern time zone to UTC, because that time never happened — our clocks had to jump directly from 1:59:59 a.m. to 3:00:00 a.m. See below:
| Date | Time | Zone | Date | Time | Zone |
|---|---|---|---|---|---|
| 2018-03-11 | 1:59:59 | EST | 2018-03-11 | 6:59:59 | UTC |
| 2018-03-11 | 3:00:00 | EDT | 2018-03-11 | 7:00:00 | UTC |
You can't convert 1:30 a.m. November 4, 2018, in US/eastern time zone to UTC either, because that time happened twice. You would have to specify whether it was 1:30 a.m. November 4, 2018 EDT or 1:30 a.m. November 4, 2018 EST:
| Date | Time | Zone | Date | Time | Zone |
|---|---|---|---|---|---|
| 2018-11-04 | 1:00:00 | EDT | 2018-11-04 | 5:00:00 | UTC |
| 2018-11-04 | 1:30:00 | EDT | 2018-11-04 | 5:30:00 | UTC |
| 2018-11-04 | 1:59:59 | EDT | 2018-11-04 | 5:59:59 | UTC |
| 2018-11-04 | 1:00:00 | EST | 2018-11-04 | 6:00:00 | UTC |
| 2018-11-04 | 1:30:00 | EST | 2018-11-04 | 6:30:00 | UTC |
| 2018-11-04 | 1:59:59 | EST | 2018-11-04 | 6:59:59 | UTC |
Advice on How to Properly Manage datetimes
Here are some rules I try to follow.
When working in Python, never use naive datetimes. (Those are datetime objects without timezone information, which unfortunately are the default in Python, even in Python 3.)
Use the pytz library when constructing datetimes, and review the documentation frequently. Properly managing datetimes is not always intuitive, and using pytz doesn't prevent me from using it incorrectly and doing things that will provide the wrong results only for some inputs, making it really hard to spot bugs. I have to triple-check that I'm following the docs when I write the code and not rely on testing to find problems.
Let me strengthen that even further. It is not possible to correctly construct datetimes with timezone information using only Python's own libraries when dealing with timezones that use DST. I must use pytz or something equivalent.
If I'm tempted to use datetime.replace, I need to stop, think hard, and find another way to do it. datetime.replace is almost always the wrong approach, because changing one part of a datetime without consideration of the other parts is almost guaranteed to not do what I expect for some datetimes.
When using Django, be sure USE_TZ = True. If Django emits warnings about naive datetimes being saved in the database, treat them as if they were fatal errors, track them down, and fix them. If I want to, I can even turn them into actual fatal errors; see this Django documentation.
When processing user input, consider whether a datetime's original timezone needs to be preserved, or if it's okay to just store the datetime as UTC. If the original timezone is important, see this post I wrote about how to get and store it.
Conclusion
Working with human times correctly is complicated, unintuitive, and needs a lot of careful attention to detail to get right. Further, some of the oft-given advice, like always working in UTC, can cause problems of its own.
