Working with Time Zones:
Everything You Wish You Didn't Need to Know

Paul Ganssle


Github repo for this talk: pganssle-talks/pycon-us-2019-timezones

Introduction

UTC

  • Reference time zone
  • Monotonic-ish (what's a few leap seconds between friends?)

Time zones vs. Offsets

  • UTC+1 is an offset
  • Europe/London is a time zone
  • BST is a highly-context-dependent abbreviation:
    • British Summer Time (UTC+1)
    • Bangladesh Standard Time (UTC+6)
    • Bougainville Standard Time (UTC+11)

Complicated time zones

Non-integer offsets

Examples:

  • Australia/Adelaide (+09:30)
  • Asia/Kathmandu (+05:45)
  • Africa/Monrovia (+00:44:30) (Before 1979)

Change of DST status without offset change

  • Portugal, 1992

    • WET (+0 STD) -> WEST (+1 DST) 1992-03-29
    • WEST (+1 DST) -> CET (+1 STD) 1992-09-27

  • Portugal, 1996

    • CET (+1 STD) -> WEST (+1 DST) 1996-03-31
    • WEST (+1 DST) -> WET (+0 STD) 1996-10-27

Complicated time zones

More than one DST transition per year

  • Morroco, 2012
    • WET (+0 STD) -> WEST (+1 DST) 2012-04-29
    • WEST (+1 DST) -> WET (+0 STD) 2012-07-20
    • WET (+0 STD) -> WEST (+1 DST) 2012-08-20
    • WEST (+1 DST) -> WET (+0 STD) 2012-09-30

... and Morocco in 2013-present, and Egypt in 2010 and 2014, and Palestine in 2011.

Complicated time zones

Missing days

  • Christmas Island (Kiritimati), December 31, 1994 (UTC-10 -> UTC+14)
In [8]:
dt_before = datetime(1994, 12, 30, 23, 59, tzinfo=tz.gettz('Pacific/Kiritimati'))
dt_after = add_absolute(dt_before, timedelta(minutes=2))

print(dt_before)
print(dt_after)

1994-12-30 23:59:00-10:00
1995-01-01 00:01:00+14:00

Also Samoa on January 29, 2011.

Complicated time zones

Double days

  • Kwajalein Atoll, 1969
In [9]:
dt_before = datetime(1969, 9, 30, 11, 59, tzinfo=tz.gettz('Pacific/Kwajalein'))
dt_after = add_absolute(dt_before, timedelta(minutes=2))

print(dt_before)
print(dt_after)

1969-09-30 11:59:00+11:00
1969-09-30 12:01:00+11:00

Asia/Shanghai

Asia/Shanghai time zone map

Asia/Urumqi

Asia/Shanghai time zone map

Why do we need to work with time zones at all?

In [10]:
from dateutil import rrule as rr

# Close of business in New York on weekdays
closing_times = rr.rrule(freq=rr.DAILY, byweekday=(rr.MO, rr.TU, rr.WE, rr.TH, rr.FR),
                         byhour=17, dtstart=datetime(2017, 3, 9, 17), count=5)

for dt in closing_times:
    print(dt.replace(tzinfo=NYC))

2017-03-09 17:00:00-05:00
2017-03-10 17:00:00-05:00
2017-03-13 17:00:00-04:00
2017-03-14 17:00:00-04:00
2017-03-15 17:00:00-04:00
In [11]:
for dt in closing_times:
    print(dt.replace(tzinfo=NYC).astimezone(UTC))

2017-03-09 22:00:00+00:00
2017-03-10 22:00:00+00:00
2017-03-13 21:00:00+00:00
2017-03-14 21:00:00+00:00
2017-03-15 21:00:00+00:00

Python's Time Zone Model

tzinfo

  • Time zones are provided by subclassing tzinfo.

  • Information provided is a function of the datetime:

    • tzname: The (usually abbreviated) name of the time zone at the given datetime
    • utcoffset: The offset from UTC at the given datetime
    • dst: The size of the datetime's DST offset (usually 0 or 1 hour)

An example tzinfo implementation

In [12]:
class ET(tzinfo):
    def utcoffset(self, dt):
        if self.isdaylight(dt):
            return timedelta(hours=-4)
        else:
            return timedelta(hours=-5)
    
    def dst(self, dt):
        if self.isdaylight(dt):
            return timedelta(hours=1)
        else:
            return timedelta(hours=0)
    
    def tzname(self, dt):
        return "EDT" if self.isdaylight(dt) else "EST"

    def isdaylight(self, dt):
        dst_start = datetime(dt.year, 1, 1) + rd.relativedelta(month=3, weekday=rd.SU(+2),
                                                               hour=2)
        dst_end = datetime(dt.year, 1, 1) + rd.relativedelta(month=11, weekday=rd.SU,
                                                             hour=2)
        
        return dst_start <= dt.replace(tzinfo=None) < dst_end

An example tzinfo implementation

In [13]:
EASTERN = ET()
In [14]:
print(datetime(2017, 11, 4, 12, 0, tzinfo=EASTERN))
print(datetime(2017, 11, 5, 12, 0, tzinfo=EASTERN))

2017-11-04 12:00:00-04:00
2017-11-05 12:00:00-05:00
In [15]:
dt_before_utc = datetime(2017, 11, 5, 0, 30, tzinfo=EASTERN).astimezone(tz.UTC)
dt_during = (dt_before_utc + timedelta(hours=1)).astimezone(EASTERN)  # 1:30 EDT
dt_after = (dt_before_utc + timedelta(hours=2)).astimezone(EASTERN)   # 1:30 EST

print(dt_during)   # Lookin good!
print(dt_after)    # OH NO!

2017-11-05 01:30:00-04:00
2017-11-05 02:30:00-05:00

Ambiguous times

Ambiguous times are times where the same "wall time" occurs twice, such as during a DST to STD transition.

In [16]:
dt1 = datetime(2004, 10, 31, 4, 30, tzinfo=UTC)
for i in range(4):
    dt = (dt1 + timedelta(hours=i)).astimezone(NYC)
    print('{} | {} |  {}'.format(dt, dt.tzname(), 
                                   'Ambiguous' if tz.datetime_ambiguous(dt)
                                   else 'Unambiguous'))

2004-10-31 00:30:00-04:00 | EDT |  Unambiguous
2004-10-31 01:30:00-04:00 | EDT |  Ambiguous
2004-10-31 01:30:00-05:00 | EST |  Ambiguous
2004-10-31 02:30:00-05:00 | EST |  Unambiguous

PEP-495: Local Time Disambiguation

  • First introduced in Python 3.6
  • Introduces the fold attribute of datetime
  • Changes to aware datetime comparison around ambiguous times

Whether you are on the fold side is a property of the datetime:

In [17]:
print_tzinfo(datetime(2004, 10, 31, 1, 30, tzinfo=NYC))          # fold=0
print_tzinfo(datetime(2004, 10, 31, 1, 30, fold=1, tzinfo=NYC))

2004-10-31 01:30:00-0400
    tzname:   EDT;      UTC Offset:  -4.00h;        DST:      1.0h
2004-10-31 01:30:00-0500
    tzname:   EST;      UTC Offset:  -5.00h;        DST:      0.0h

Note: fold=1 represents the second instance of an ambiguous datetime

Comparing timezone-aware datetimes: Same Zone

In the same zone, wall clock times are used, offset ignored

In [19]:
print_dt_eq(dt1.replace(tzinfo=NYC), dt2.replace(tzinfo=NYC))   # Unambiguous
print_dt_eq(dt1.replace(tzinfo=NYC), dt3.replace(tzinfo=NYC))
2004-10-30 12:00:00-04:00 == 2004-10-30 12:00:00-04:00: True
2004-10-30 12:00:00-04:00 == 2004-10-30 11:00:00-04:00: False
In [20]:
print_dt_eq(dt1a.replace(tzinfo=NYC), dt2a.replace(tzinfo=NYC))  # Ambiguous
print_dt_eq(dt1a.replace(tzinfo=NYC), dt2a.replace(fold=1, tzinfo=NYC), bold=True)
print_dt_eq(dt1a.replace(tzinfo=NYC), dt3a.replace(tzinfo=NYC))
2004-10-31 01:30:00-04:00 == 2004-10-31 01:30:00-04:00: True
2004-10-31 01:30:00-04:00 == 2004-10-31 01:30:00-05:00: True
2004-10-31 01:30:00-04:00 == 2004-10-31 02:30:00-05:00: False

Comparing timezone-aware datetimes: Different Zones

If both datetimes are unambiguous, the absolute times are compared:

In [21]:
print_dt_eq(dt1.replace(tzinfo=NYC), dt2.replace(tzinfo=CHI))    # Unambiguous
print_dt_eq(dt1.replace(tzinfo=NYC), dt3.replace(tzinfo=CHI))
2004-10-30 12:00:00-04:00 == 2004-10-30 12:00:00-05:00: False
2004-10-30 12:00:00-04:00 == 2004-10-30 11:00:00-05:00: True

If either datetime is ambiguous, the result is always False:

In [22]:
print_dt_eq(dt1a.replace(fold=1, tzinfo=NYC), dt3a.replace(tzinfo=CHI), bold=True)
2004-10-31 01:30:00-05:00 == 2004-10-31 02:30:00-06:00: False

A curious case...

In [23]:
LON = gettz('Europe/London')

x = datetime(2007, 3, 25, 1, 0, tzinfo=LON)
ts = x.timestamp()
y = datetime.fromtimestamp(ts, LON)
z = datetime.fromtimestamp(ts, gettz.nocache('Europe/London'))
In [24]:
x == y
Out[24]:
False
In [25]:
x == z
Out[25]:
True
In [26]:
y == z
Out[26]:
True

Imaginary Times

Imaginary times are wall times that don't exist in a given time zone, such as during an STD to DST transition.

In [27]:
dt1 = datetime(2004, 4, 4, 6, 30, tzinfo=UTC)
for i in range(3):
    dt = (dt1 + timedelta(hours=i)).astimezone(NYC)
    print(f'{dt} | {dt.tzname()} ')

2004-04-04 01:30:00-05:00 | EST 
2004-04-04 03:30:00-04:00 | EDT 
2004-04-04 04:30:00-04:00 | EDT
In [28]:
print(datetime(2007, 3, 25, 1, 0, tzinfo=LON))

2007-03-25 01:00:00+01:00
In [29]:
print(datetime(2007, 3, 25, 0, 0, tzinfo=UTC).astimezone(LON))
print(datetime(2007, 3, 25, 1, 0, tzinfo=UTC).astimezone(LON))

2007-03-25 00:00:00+00:00
2007-03-25 02:00:00+01:00

Why it was non-transitive

In [31]:
print(f'x (LON):              {x}')
print(f'x (UTC):              {x.astimezone(UTC)}')
print(f'x (LON->UTC->LON):    {x.astimezone(UTC).astimezone(LON)}')

x (LON):              2007-03-25 01:00:00+01:00
x (UTC):              2007-03-25 00:00:00+00:00
x (LON->UTC->LON):    2007-03-25 00:00:00+00:00
In [32]:
print(f'y: {y}')
print(f'z: {z}')

y: 2007-03-25 00:00:00+00:00
z: 2007-03-25 00:00:00+00:00
In [34]:
x.tzinfo is y.tzinfo
Out[34]:
True
In [35]:
x.tzinfo is z.tzinfo
Out[35]:
False

Working with time zones

dateutil

In dateutil's suite of tzinfo objects, you can attach time zones in the constructor if you have a wall time:

In [36]:
dt = datetime(2017, 8, 11, 14, tzinfo=tz.gettz('US/Pacific'))
print_tzinfo(dt)

2017-08-11 14:00:00-0700
    tzname:   PDT;      UTC Offset:  -7.00h;        DST:      1.0h

If you have a naive wall time, or a wall time in another zone that you want to translate without shifting the offset, use datetime.replace:

In [37]:
print_tzinfo(dt.replace(tzinfo=tz.gettz('US/Eastern')))

2017-08-11 14:00:00-0400
    tzname:   EDT;      UTC Offset:  -4.00h;        DST:      1.0h

If you have an absolute time, in UTC or otherwise, use datetime.astimezone():

In [38]:
print_tzinfo(dt.astimezone(tz.gettz('US/Eastern')))

2017-08-11 17:00:00-0400
    tzname:   EDT;      UTC Offset:  -4.00h;        DST:      1.0h

pytz

In pytz, datetime.astimezone() still works exactly as expected:

In [39]:
print_tzinfo(dt.astimezone(pytz.timezone('US/Eastern')))

2017-08-11 17:00:00-0400
    tzname:   EDT;      UTC Offset:  -4.00h;        DST:      1.0h

But the constructor or .replace methods fail horribly:

In [40]:
print_tzinfo(dt.replace(tzinfo=pytz.timezone('US/Eastern')))

2017-08-11 14:00:00-0456
    tzname:   LMT;      UTC Offset:  -4.93h;        DST:      0.0h

pytz's time zone model

  • tzinfos are all static offsets
  • tzinfo is attached by the time zone object itself:
In [41]:
LOS_p = pytz.timezone('America/Los_Angeles')
dt = LOS_p.localize(datetime(2017, 8, 11, 14, 0))
print_tzinfo(dt)

2017-08-11 14:00:00-0700
    tzname:   PDT;      UTC Offset:  -7.00h;        DST:      1.0h
  • You must normalize() datetimes after you've done some arithmetic on them:
In [42]:
dt_add = dt + timedelta(days=180)
print_tzinfo(dt_add)

2018-02-07 14:00:00-0700
    tzname:   PDT;      UTC Offset:  -7.00h;        DST:      1.0h
In [43]:
print_tzinfo(LOS_p.normalize(dt_add))

2018-02-07 13:00:00-0800
    tzname:   PST;      UTC Offset:  -8.00h;        DST:      0.0h

There is more detail on this on my blog post, "pytz: The Fastest Footgun in the West" (https://blog.ganssle.io/articles/2018/03/pytz-fastest-footgun.html)

Handling ambiguous times

Overview

Both dateutil and pytz will automatically give you the right absolute time if converting from an absolute time.

In [44]:
dt1 = datetime(2004, 10, 31, 6, 30, tzinfo=UTC)  # This is in the fold in EST

dt_dateutil = dt1.astimezone(tz.gettz('US/Eastern'))
dt_pytz = dt1.astimezone(pytz.timezone('US/Eastern'))
print(repr(dt_dateutil))
print_tzinfo(dt_dateutil)

datetime.datetime(2004, 10, 31, 1, 30, fold=1, tzinfo=tzfile('/usr/share/zoneinfo/US/Eastern'))
2004-10-31 01:30:00-0500
    tzname:   EST;      UTC Offset:  -5.00h;        DST:      0.0h
In [45]:
print(repr(dt_pytz))    # Note that pytz doesn't set fold
print_tzinfo(dt_pytz)

datetime.datetime(2004, 10, 31, 1, 30, tzinfo=<DstTzInfo 'US/Eastern' EST-1 day, 19:00:00 STD>)
2004-10-31 01:30:00-0500
    tzname:   EST;      UTC Offset:  -5.00h;        DST:      0.0h

dateutil

For backwards compatibility, dateutil provides a tz.enfold method to add a fold attribute if necessary:

In [46]:
dt = datetime(2004, 10, 31, 1, 30, tzinfo=tz.gettz('US/Eastern'))
tz.enfold(dt)
Out[46]:
datetime.datetime(2004, 10, 31, 1, 30, fold=1, tzinfo=tzfile('/usr/share/zoneinfo/US/Eastern'))
Python 2.7.12
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import datetime
>>> from dateutil import tz
>>> dt = datetime(2004, 10, 31, 1, 30, tzinfo=tz.gettz('US/Eastern'))
>>> tz.enfold(dt)
_DatetimeWithFold(2004, 10, 31, 1, 30, tzinfo=tzfile('/usr/share/zoneinfo/US/Eastern'))
>>> tz.enfold(dt).tzname()
'EST'
>>> dt.tzname()
'EDT'

dateutil

To detect ambiguous times, dateutil provides tz.datetime_ambiguous

In [47]:
tz.datetime_ambiguous(datetime(2004, 10, 31, 1, 30, tzinfo=NYC))
Out[47]:
True
In [48]:
dt_0 = datetime(2004, 10, 31, 0, 30, tzinfo=NYC)
for i in range(3):
    dt_i = dt_0 + timedelta(hours=i)
    dt_i = tz.enfold(dt_i, tz.datetime_ambiguous(dt_i))
    print(f'{dt_i} (fold={dt_i.fold})')

2004-10-31 00:30:00-04:00 (fold=0)
2004-10-31 01:30:00-05:00 (fold=1)
2004-10-31 02:30:00-05:00 (fold=0)

Note: fold is ignored when the datetime is not ambiguous:

In [49]:
for i in range(3):
    dt_i = tz.enfold(dt_0 + timedelta(hours=i), fold=1)
    print(f'{dt_i} (fold={dt_i.fold})')

2004-10-31 00:30:00-04:00 (fold=1)
2004-10-31 01:30:00-05:00 (fold=1)
2004-10-31 02:30:00-05:00 (fold=1)

Handling imaginary times

dateutil

dateutil provides a tz.datetime_exists() function to tell you whether you've constructed an imaginary datetime:

In [54]:
dt_0 = datetime(2004, 4, 4, 1, 30, tzinfo=NYC)
for i in range(3):
    dt = dt_0 + timedelta(hours=i)
    print(f'{dt} ({{}})'.format('Exists' if tz.datetime_exists(dt) else 'Imaginary'))

2004-04-04 01:30:00-05:00 (Exists)
2004-04-04 02:30:00-04:00 (Imaginary)
2004-04-04 03:30:00-04:00 (Exists)

tz.resolve_imaginary

Generally for imaginary datetimes, you want to "skip forward" to what the the time would be if the transition had not happened. In dateutil you can use the tz.resolve_imaginary function to do this automatically:

In [55]:
dt = datetime(2004, 4, 4, 1, 30, tzinfo=NYC)
dt_imag = dt + timedelta(hours=1)
print(tz.resolve_imaginary(dt_imag))

2004-04-04 03:30:00-04:00
In [56]:
dt = datetime(1994, 12, 31, 9, tzinfo=tz.gettz('Pacific/Kiritimati'))
print(f'{dt} ({{}})'.format('Exists' if tz.datetime_exists(dt) else 'Imaginary'))

1994-12-31 09:00:00-10:00 (Imaginary)
In [57]:
tz.resolve_imaginary(dt)
Out[57]:
datetime.datetime(1995, 1, 1, 9, 0, tzinfo=tzfile('/usr/share/zoneinfo/Pacific/Kiritimati'))

Both of these functions work with pytz time zones as well.

dateutil's tzinfo implementations

UTC and Static time zones

In [65]:
# tz.UTC is equivalent to pytz.UTC or timezone.utc
dt = datetime(2014, 12, 19, 22, 30, tzinfo=tz.UTC)
print_tzinfo(dt)

2014-12-19 22:30:00+0000
    tzname:   UTC;      UTC Offset:   0.00h;        DST:      0.0h

Static offsets represent zones with a fixed offset from UTC, and takes a tzname or either number of seconds or a timedelta:

In [66]:
JST = tzoffset('JST', 32400)                       # Japan Standard Time is year round
EST = tzoffset(None, timedelta(hours=-5))          # Can use None as a name

dt = datetime(2016, 7, 17, 12, 15, tzinfo=tz.UTC)
print_tzinfo(dt.astimezone(JST))
print_tzinfo(dt.astimezone(EST))

2016-07-17 21:15:00+0900
    tzname:   JST;      UTC Offset:   9.00h;        DST:      0.0h
2016-07-17 07:15:00-0500
    tzname:  None;      UTC Offset:  -5.00h;        DST:      0.0h

IANA (Olson) database

The IANA database contains historical time zone transitions:

In [72]:
NYC = tz.gettz('America/New_York')
NYC
Out[72]:
tzfile('/usr/share/zoneinfo/America/New_York')
In [73]:
print_tzinfo(datetime(2017, 8, 12, 14, tzinfo=NYC))      # Eastern Daylight Time

2017-08-12 14:00:00-0400
    tzname:   EDT;      UTC Offset:  -4.00h;        DST:      1.0h
In [74]:
print_tzinfo(datetime(1944, 1, 6, 12, 15, tzinfo=NYC))    # Eastern War Time

1944-01-06 12:15:00-0400
    tzname:   EWT;      UTC Offset:  -4.00h;        DST:      1.0h
In [75]:
print_tzinfo(datetime(1901, 9, 6, 16, 7, tzinfo=NYC))     # Local solar mean

1901-09-06 16:07:00-045602
    tzname:   LMT;      UTC Offset:  -4.93h;        DST:      0.0h

tz.gettz()

The best way to get a time zone is to pass the relevant timezone string to the gettz() function, which is intended to be a Python equivalent to the TZ environment variable.

In [76]:
tz.gettz()      # Passing nothing gives you local time
Out[76]:
tzfile('/etc/localtime')
In [77]:
# If it finds a valid abbreviation for the local zone, returns tzlocal()
with TZEnvContext('LMT4'):
    print(gettz('LMT'))

tzlocal()
In [78]:
# Retrieve IANA zone:
print(gettz('Pacific/Kiritimati'))

tzfile('/usr/share/zoneinfo/Pacific/Kiritimati')

Timezone Tips

Civil time vs. Timestamp

  • Civil times are "wall times" - use this when it matters what the clock on the wall says (e.g. meetings, television shows)
  • Timestamps are specific moments in time: use this when it matters what order things happened in or the duration between two events (e.g. logging, astronomical events)

Store civil times as naive portion + time zone serialization (e.g. America/New_York). Store timestamps in UTC or equivalent.

IANA and CDR

  • Never rely on a 3-letter abbreviation.
  • IANA keys are lookups in the tz database
  • Use Unicode CLDR (Common Locale Data Repository) to get display names for time zones

Thank You!

dateutil

If you're interested in helping out with dateutil development, check out the issues on the github or e-mail me.

Open Issues - https://github.com/dateutil/dateutil - Check out the help wanted label!



Me

@pganssle / https://ganssle.io / https://blog.ganssle.io


GPG Key


6B49 ACBA DCF6 BD1C A206
67AB CD54 FCE3 D964 BEFB