Timeline generation fails when commit with garbled message is done

[sanda@…]

because of the garbled message in commit v.decode('utf-8') (sqlite_backend.py, line 73) raise exception and timeline is not generated. changing to to_unicode function solves the problem.

for full story read: http://www.mail-archive.com/lyx-devel@lists.lyx.org/msg143533.html

the problematic commit can be seen here (after applying this fix) http://www.lyx.org/trac/changeset/26376

[anonymous]

the archive link does not work for some reason, adding another one http://marc.info/?l=lyx-devel&m=122124075323414&w=2

[rblank]

Mmh, this could be related to #7643. The site is running 0.10.2. Hey guys, time to upgrade! And while you're at it, do change the project logo image and the WikiStart page.

I'll put this on my to-do list.

[rblank]

I have been looking at this, and I don't think replacing v.decode('utf-8') with to_unicode(v, 'utf-8') (which will actually do a v.decode('utf-8', 'replace') is the right solution (or let's say, I'm reluctant to do the change):

• This can alter the data read from the database, for all reads. As the data is quite often used to write back to the database, this will alter the data in the database in an invisible way.

• All data should be stored in the database as utf-8. If this is not the case, we should find where this has happened.

• The problem is located in the compatibility code for pysqlite 1, so this is legacy code anyway, and I have no way of testing the fix.

I can offer you three options to fix the problem on your installation:

• If this only happened on a few changesets, use svnadmin to change the commit message to something sensible, and resync the repository cache with trac-admin resync.

• If you don't want to change the commit message, update to pysqlite 2 and resync the repository cache. Pysqlite 2 stores the data in utf-8 internally (AFAIK), so this conversion is not even needed.

• If you can't update for some reason, then by all means replace v.decode('utf-8') with v.decode('utf-8', 'replace') (a smaller change than to_unicode()).

I'll close this as invalid, as it is not clear how the corrupted data came into the database in the first place. Feel free to reopen if you can reproduce the problem with a recent Trac release (0.11.2.1 at this time) and pysqlite 2.

You still might want to take a look at comment:2.