Date formatting causes trac-admin unit tests to fail / trac-admin should use ISO format for datetimes

[Ian Leader <__ian.leader__@…>]

Windows XP SP2 Python 2.3 UK Locale settings

On above platform, as of [1793], 5 unit tests in trac\scripts\tests\admin.py fail because of date formatting issues:

C:\temp\working\trac>python trac\scripts\tests\admin.py
.............F.F.F...................................F......F
======================================================================
FAIL: test_milestone_add_ok (__main__.TracadminTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "trac\scripts\tests\admin.py", line 679, in test_milestone_add_ok
    self.assertEquals(self.expected_results[test_name], test_results)
  File "c:\python23\lib\unittest.py", line 302, in failUnlessEqual
    raise self.failureException, \
AssertionError: '\nName           Due                       Completed\n---------
-----------------------------------------\nnew_milestone  Thu Jan  1 00:00:00 20
04\nmilestone1\nmilestone2\nmilestone3\nmilestone4\n\n' != '\nName           Due
                Completed\n-------------------------------------------\nnew_mile
stone  01/01/04 00:00:00\nmilestone1\nmilestone2\nmilestone3\nmilestone4\n\n'

======================================================================
FAIL: test_milestone_completed_ok (__main__.TracadminTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "trac\scripts\tests\admin.py", line 741, in test_milestone_completed_ok
    self.assertEquals(self.expected_results[test_name], test_results)
  File "c:\python23\lib\unittest.py", line 302, in failUnlessEqual
    raise self.failureException, \
AssertionError: '\nName        Due  Completed\n---------------------------------
--------\nmilestone1\nmilestone2       Thu Jan  1 00:00:00 2004\nmilestone3\nmil
estone4\n\n' != '\nName        Due  Completed\n---------------------------------
-\nmilestone1\nmilestone2       01/01/04 00:00:00\nmilestone3\nmilestone4\n\n'

======================================================================
FAIL: test_milestone_due_ok (__main__.TracadminTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "trac\scripts\tests\admin.py", line 720, in test_milestone_due_ok
    self.assertEquals(self.expected_results[test_name], test_results)
  File "c:\python23\lib\unittest.py", line 302, in failUnlessEqual
    raise self.failureException, \
AssertionError: '\nName        Due                       Completed\n------------
-----------------------------------\nmilestone2  Thu Jan  1 00:00:00 2004\nmiles
tone1\nmilestone3\nmilestone4\n\n' != '\nName        Due                Complete
d\n----------------------------------------\nmilestone2  01/01/04 00:00:00\nmile
stone1\nmilestone3\nmilestone4\n\n'

======================================================================
FAIL: test_version_add_ok (__main__.TracadminTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "trac\scripts\tests\admin.py", line 586, in test_version_add_ok
    self.assertEquals(self.expected_results[test_name], test_results)
  File "c:\python23\lib\unittest.py", line 302, in failUnlessEqual
    raise self.failureException, \
AssertionError: '\nName  Time\n------------------------------\n1.0\n2.0\n9.9   T
hu Jan  1 00:00:00 2004\n\n' != '\nName  Time\n------------------------------\n1
.0\n2.0\n9.9   Thu Jan 01 00:00:00 2004\n\n'

======================================================================
FAIL: test_version_time_ok (__main__.TracadminTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "trac\scripts\tests\admin.py", line 627, in test_version_time_ok
    self.assertEquals(self.expected_results[test_name], test_results)
  File "c:\python23\lib\unittest.py", line 302, in failUnlessEqual
    raise self.failureException, \
AssertionError: '\nName  Time\n------------------------------\n1.0\n2.0   Thu Ja
n  1 00:00:00 2004\n\n' != '\nName  Time\n------------------------------\n1.0\n2
.0   Thu Jan 01 00:00:00 2004\n\n'

----------------------------------------------------------------------
Ran 61 tests in 2.464s

FAILED (failures=5)

C:\temp\working\trac>

There appear to be two distinct issues:

1. Inconsistancies in the code for presentation of dates/times in trac-admin
2. Differences in the formatting applied by time.strftime('%c', ...) on different platforms/locales

Inconsistancies in the date presentation in trac-admin

In admin.py, _do_version_list uses time.ctime, whereas _do_milestone_list uses time.strftime(c%, ...). These produce different formats on some platforms and locales.

(as an aside, _do_wiki_list also uses time.ctime and presents using yet another format)

This would probably be best resolved by modifying all commands in trac-admin to present in local time format using strftime(c%, ...)

Differences in formatting on different platforms/locales

The expected results for the trac-admin unit tests are hardcoded in admin-tests.txt. I'm guessing the format of the dates in this file is the default format on a European UNIX system of some flavour. The specific problem I'm seeing is that the day of month in 1st Jan 2004 is appearing as "1" in the expected results, but "01" on my system.

To get around this problem on my system, I can change 'new year' to be 11th Jan instead of 1st Jan, but this will not solve the problem for (e.g.) US date formatting. Better solution would be to replace the dates in the expected resuls file by a token which is converted into the appropriate date on the system running the tests.

[Ian Leader <__ian.leader__@…>]

Following discussion on the mailing list, I intend to implement as follows:

1. trac-admin will be changed so that all dates (both input and output) will use ISO 8601
2. The ISO 8601 complete date format will be used for dates i.e. YYYY-MM-DD, and for times, complete date + hours and minutes will be used i.e. YYYY-MM-DDThh:mm
3. Times will be in the local time of the system, but will not show a timezone

This will resolve both the consistency issue, and the unit tests failing on different platforms/locales.

I plan to implement w/c 2005-06-13 hopefully for 0.9.

[Ian Leader <__ian.leader__@…>]

Patch for this against revision 1810 attached. Tested on WinXP SP2, Apache 2.0.54, Python 2.3.4, CGI.

Changes:

• Functional
  • format "YYYY-MM-DD" used for input and display of dates
  • format "YYYY-MM-DD HH:MM:SS" used for display of datetimes
  • date format hint updated accordingly
  • milestone completion datetimes now displayed as datetimes, not dates, to support the data entry capabilities of the Roadmap module. However, 'milestone add' and 'milestone completed' trac-admin commands have not been updated to support 'Completed' as a datetime (see issues, below)

• Non-functional / internal
  • All function and variable names now use 'date' and 'datetime' correctly
  • The date format hint is no longer worked out in trac.util, but specied explicitly in admin.py. This is because the date format hint is also used by the Roadmap module, which uses DD/MM/YY format (see issues, below)
  • All trac-admin unit tests now work on Windows / en-GB (and should work on all other platforms / locales as well)

Outstanding issues:

• Datetime format for milestones now differs in trac-admin and the Roadmap module. I would suggest an explict datetime format configuration value to ensure consitency across the whole application.
• Functionality incosistant between Roadmap module and trac-admin for milestone due date (Roadmap can set a time, trac-admin only a date). I haven't updated trac-admin to match the Roadmap module, because due date is still a date in the Roadmap, and I don't understand why one would want to record completion dates to an accuracy of 1 sec, when only recording due dates to an accuracy of 1 day

[cmlenz]

Patch applied in [1812]. Thanks!

[Ian Leader <__ian.leader__@…>]

IssueS moved to #1690 (inconsistant datetime formats) and the mailing list (why is milestone completed a datetime, when due is only a date)