[ER] Generalized automatic cross-references in Trac

[cboos@…]

Problem

Trac provides a fast, easy and flexible way to relate Trac objects together, by the use of TracLinks in the WikiFormatting.

However, there's no automatic support for back-linking. This lack has been noticed in several circumstances:

• BackLinks for Wiki pages (tickets #611 and #646, the latter containing a [http:/trac/ticket/646#change_4 note] which also highlights the genericity of the problem)
• Related Changesets for a given Ticket (ticket #508)
• Ticket dependencies and other relationships (tickets #31 and #226)

Instead, the backlinking is done manually. For example, when closing a ticket with a resolution of duplicate, the convention is to comment the closing of the ticket by saying this is a duplicate of #xyz and to mention the existence of the duplicate ticket on the original #xyz ticket. With a generalized backlinking facility, this would cease to be necessary.

Solution

What I would propose is that every relationship that gets detected while parsing new wiki content is recorded. Then, each wiki, changeset and ticket page could contain a See also: section at the end, listing the other objects that contain TracLinks targeting it, along with a bit of context, to catch the semantic of the link as well.

---

Example given:

Consider the following wiki OutstandingPatches page:

= Outstanding Patches page = 

I think that the patches #187, #909, #919, #944 and #948 are great!

Then, each time this page is edited, the content of an XREF (source, target, context table would be cleared for this page:

delete from XREF where source = 'wiki:OutstandingPatches'

and when the content is processed, each time a TracLink is detected, the table would be filled:

insert into XREF values ('wiki:OutstandingPatches',
                         'ticket:187',
                         'I think that the patches #187, #909, #919, #944 and #948 are great!')

Then, while browsing #187, one could easily get the backlinks:

select source, context from XREF where target='ticket:187'

in order to create the following output at the end of the ticket page:

Backlinks: OutstandingPatches

---

Of course, I'm willing to provide a patch for that, but I would like to get some feedback before.

-- Christian

[cmlenz]

Yeah, this has been discussed before.

In the xref table, separate source type from source id, and context type from context id (would "dest" be a better name?), for quicker lookup:

CREATE TABLE xref (
    src_type    string,
    src_id	string,
    dest_type   string,
    dest_id     string,
    title       string
    relation    string
);

I've added a relation column here, which would be optional. This could be used for things like ticket dependencies. So you'd have both implicit backlinks (normal links in wiki text) and explicit relations (such as ticket dependencies) in the same table, which makes sense IMHO.

[cboos@…]

Proposed Implementation

Features

Currently, cross-references between the following Trac objects are supported:

• Wiki page
• Ticket
• Changeset
• Report
• Milestone
• Source (only as targets for now)

Basically, two kinds of references are supported:

• implicit references between objects
  Implicit references are created for every TracLinks that can be found in (any of) the wiki text of a Trac object. Indeed, some objects may have separately editable wiki texts, each of them being a facet of this object.
• explicit relation between objects
  A relation must be created explicitely as such, by some programmatic mean. Currently, some of the ticket fields are setting up explicit relationships.

The user interface is quite consistent: for every page displaying an object which has backlinks, the first navigation link is Backlinks.

This goes to the /xref module, which displays the source objects in a way similar to the Timeline. First the relationships are presented, then the implicit links.

From that page, one can also navigate to the Forward Links (i.e. a summary of the outgoing links) and to the Orphaned Objects (i.e. those not referenced anywhere)

Consistency

The data model seems to be pretty solid, and the updates seem to work as expected.

How to install?

It is a clean patch made on the trunk as of today ([1343]).

It contains some database schema modification, so there is an upgrades/db10.py file.

So the recommended way to proceed is to use trac-admin, and:

• upgrade
• xref
• resync

I left some 'print' statements on purpose, they should highlight what is going on when cross-references are created/deleted.

To be continued

This was really a very interesting task, I hope you will enjoy the results.

There are of course a few things that can be done from there:

Trac Objects

• The patch is not very intrusive, it doesn't change the way the Trac internals work, but a very logical next step would be to program this in a more O-O way... which bring us in a very concrete way now in the direction of the TracObjectModelProposal (which will be re-edited shortly to reflect the experience gained while doing this patch)
• In that framework, the xref trac-admin command will be able to handle relations (currently, relations only work for new tickets and modification of tickets, for the milestone and component fields)

Other things

• A small change for the ticket comments was to introduce their numbering. This will also facilitate their editing (see #454). I also have some ideas for having threaded comments.
• Some additional synonyms have been introduced:
  • issue is synonym to ticket
  • source, repos and browser are now also synonyms in URLs

[cboos@…]

The xref patch (on [1343])

[cboos]

A few informations about the future steps concerning the xref-branch:

• have the feature integrated into the trunk, after my ticket category and intertrac branches are themselves integrated
• add a time column in the xref table and sort the xrefs according to its age (the most recent first)
• add a few more macros:
  • [[Backlinks]] that should provide a short list of implicit xrefs (i.e. not the relations, only the implicit references found in Wiki text); the display should match the original one suggested in the description for this ticket
  • [[Graph(relation_pattern,depth)]] which should provide a way to explore xrefs more than one level at a time...

[anonymous]

As 0.9 is getting nearer, it seems I won't be able to finish it for that milestone. TracCrossReferences will be high priority for 1.0 though.

[cboos]

This feature is "alive", and quite in sync with the trunk.

The [[Backlinks]] macro mentioned above has been implemented in r2003.

[cboos]

There are currently a few things of mine that are more ready for prime-time than this, so I'm lowering the prio.