Use trac with a remote Subversion repository

[radix at twistedmatrix.com]

It stinks pretty bad that I have to install trac on my Subversion server AND give it write access to it.

[mrowe@…]

At present Subversion's Python bindings do not support the more-generic "repository access layer" that would facilitate accessing file://, http:// and svn:// repositories. Adding support for remote repositories to Trac will be difficult until this situation is remedied.

[daniel]

As Mark pointed out, this is indeed the reason for above mentioned stink.

Whenever there's an official way to us the RA layer, Trac will utilize it.

[anonymous]

This is understandably difficult. However, do ant trac'ers have recommendations for how to do do fine grained access control for users coming through trac? I'm currently using subversions apache module to control which users have read and write access to content via AuthzSVNAccessFile. Since trac is using the server bindings and not going through HTTP as a client, it is bypassing my access control layer. Does any one know how to do svn access control at a lower level???

I'll take this up on the svn lists too.

[Marc Sutton]

Subversion 1.2 has just been released and they've flattened the remote access API so SWIG can access it (and python bindings can be produced).

Is that all that's required to get trac to use a remote repository?

[cmlenz]

I've experimented with this, but unfortunately the SWIG bindings for svn_ra didn't seem to be completely usable.

For example, svn_ra_stat (IIRC) should return a dirent object of some kind, but it expects a pointer to a struct. Apparently the mapping from C to python isn't quite worked out yet for that layer.

[Juanma Barranquero]

[...] Subversion 1.2 has just been released [...]

Well, Subversion 1.2 has not yet been released, unless you count release candidates as full releases. Let's not spread wrong information, please.

[eric@…]

1.2 has been released now... any improvement on this situation?

[anonymous]

Related to fine grained permissions question asked in this topic:

http://projects.edgewall.com/trac/wiki/FineGrainedPermissions

[jwt]

I have successfully used http://svk.elixus.org/ for mirroring a remote repository (to a local trac-server). Obviously this is not always desirable.

Any status-update on this issue yet ?

[Tobi Oetiker <oetiker@…>]

When I try using svn://svn....../repo as a repository path, I end up with a nice error message:

Assertion failed: is_canonical (base, blen), file subversion/libsvn_subr/path.c, line 114

I am using svn 1.2.1 with trac 0.9b2

any progress on remote repository support ?

[jwin1@…]

I really want to promote using trac here with our development, but our 'server-folks' (sadly) just aren't open-minded enough to keep trac and svn on one machine, not speaking of all political difficulties...

This is really an important one for us.

[mgood]

Well, I think that it will save your "server-folks" some work to keep the two on the same box, since it makes it easier to set up things like the authentication that should be the same between them. If the SVN repository uses the FSFS backend (highly recommended), you can run Trac with read-only access to the repository. If that's still not sufficient, you should be able to share the SVN repository read-only via NFS to the Trac server.

[anonymous]

Any experienced IT administrator knows that keeping multiple services on one box does not "save work". This is the single reason why we cannot adopt Trac for any real project in our organization. We have an established SVN repository, processes, and security policy and there is no way we could install Trac on the SVN server. The SVN server is considered security and mission critical.

If there would be a way to somehow relay authorization from the user's desktop to the SVN server through Trac, that would be perfect. This way we could set up Trac without having to let our IT department set it up (which would take months to actually happen). We could update to a new release and apply patches whenever we need to. Currently, we cannot, since setting up Trac requires all-permissive access to the SVN repository and that means it would have to be administered by the IT guys. If Trac could offload all real authentication and access control to SVN, our IT would be happy.

Anyway, just the possibility to access a remote SVN repository is needed for our IT to even consider setting up a Trac server.

[jwin]

regardless of the status of the swig-python bindings, if I want to test the svn_ra access, is

trunk/trac/versioncontrol/tests

the right place to look for modifying to use svn_ra access ?

Has anyone done this recently ?

[jwin]

how about using a Python-extension written in C the C-Api for svn_ra should work

[jw]

svn 1.3rc7

* new public APIs:

• new transfer progress callback for DAV (r15948)
• svn_ra_initialize(), svn_client_open_ra_session()
• svn_fs_closest_copy(), svn_fs_type()
• several rev'd APIs, see doxygen docs

* SWIG bindings: No more compile-time or runtime SWIG dependencies

• SWIG/python bindings:
  • automatic memory management: APIs no longer require pool

[Mitch]

Any progress on this? I'd love to use this feature.

[cmlenz]

WTF?

[anonymous]

Whats the eta on this feature? We have been using trac for about 1 year and would like to decouple Trac from the svn repository because we do not wish to give out mass accounts to the machine hosting our repo, for obvious security/stability reasons.

[eblot]

#2828 has been marked as a duplicate.

[caio1982]

Any progress on making it happen? Does anyone above got some err... "hack" that fully works? Mirroring data using SVK or even DRBD (!) is not a real option and NFS is... well, you know.

[anonymous]

pysvn seems to have support for accessing remote repositories. And these bindings looks a lot more "active" too.

[cboos]

I've noticed the pysvn suggestion when it was made. This, combined with the VcRefactoring#NewRepositoryCache changes, would make it practical to work with remote subversion repositories, as remote requests would only have to be done for doing changeset synchronizations.

[anonymous]

I want to set up Trac for a SourceForge? project, but I can't until this feature is added. I hope it happens soon.

[Barry Scott]

pysvn indeed can access remote repos. Is there anything missing from pysvn API that would prevent you using it to solve this problem? (Apart from the pain of changing from svn bindings to pysvn that is!)

Barry - pysvn author

[caio1982]

I think it's a quite important feature that would make Trac even more flexible so it's not that good change this ticket's priority to "low" instead of "normal". Workarounds or extensions are acceptable but aren't the best solution, IMHO.

[anonymous]

Lack of this feature is the main reason why my new employer's development department doesn't use Trac (a pretty big international company). I've used Trac on the previous job with great success and I'm very sad I can't use it now.

[anonymous]

We are a small team in a big company evaluating Trac. This single issue is the reason why we won't use it companywide.

[mgood]

Feel free to bump up the priority if you're volunteering to implement it. Otherwise you'll need to be patient.

[trac@…]

In terms of implementing this, I was thinking it would be easy enough to write a new version control module (i.e. svn_pysvn) that uses pysvn instead of subersion's python bindings.

Does that make sense ? Are there any pitfalls to doing it this way ?

[cboos]

This was already acknowledged as being a good idea. See comment:38. The point of waiting for the upcoming refactoring would be to have much less work to do in order to write this new backend, as most of the operations will be done on the cache, and basically only the synchronisation of new changes will have to be backend-specific (at least that's the plan).

Nevertheless, I you feel like doing so, it would be perfectly possible to write a pysvn backend right now for milestone:0.10.

[lists01@…]

just a "me too" re: support for remote SVN repositories. we want to use trac but our sysadmin team already has a stand-alone SVN server set up.

the "on-the-same-box" requirement may be fine for small teams (or teams with that kind of independence), but in large organizations where there is a separation of responsibilities this requirement can be a show-stopper.

[cboos]

Among the possible workarounds, there's also svnsync, which is a new tool part of Subversion 1.4. It allows to easily replicate repositories, from URL / to URL, which gives you a great deal of flexibility.

For example, you can set up a post-commit hook on the central repository that will trigger the replication to a slave repository on the server where you run Trac. Alternatively, on that Trac host, you can periodically call svnsync which will pull new revisions from the central repository.

I've tested that tool with 1.4.0rc5, and it works fine. Of course you could have done the above using svk mirror, but svnsync is more straightforward to install and use.

[Shun-ichi Goto <shunichi.goto@…>]

I guess there is a way to access remote repository by using standard python binding of subversion 1.4 like svnsync (written by C) does. It may require smaller change than implementing pysvn backend. Of course, it will have limitation that both trac side and remote server side need svn 1.4 or later.

[chris@…]

Any news on this? I'm wanting to use Trac for a SourceForge? project but cannot install it on the server since I don't have file:// access to the repository. I could mirror the repository but if I'm going to do that I might as well just move my repository to my VDS which I rather not do :(

We use Trac at work and are extremely happy with it. We found it's made our lives impressively easier and tend to spend a lot of time reloadng the timeline page. I think having remote SVN capabilities would sell this to the masses 100%.

[sik0fewl]

Another "me too." Has there been any recent progress on this ticket? The suggestion by Shun-ichi Goto seems like it will work well, at least for subversion 1.4 and greater.

[Shun-ichi Goto <shunichi.goto@…>]

I'm working for a week to implement this backend using Subversion RA API with standard python binding. Now the code, called svn_ra, is implemented as plugin and tested to have same behaviour and result with svn_fs. It seems to work for some small project but not yet tested with big one. So it is alpha state. There are many things to be implemented and improved, especially speed up of resync.

I'm placed it in public. You can try it if you are interested in.
http://www.meadowy.org/~gotoh/projects/remote-svn-plugin/wiki
But don't forget that it should not be used for actual project yet. I'll continue to improve it.

BTW, I've also implemented with pysvn before using RA. It is very costfull because it uses client API (not RA) and client API makes session every API call. I found it is intended to replace client program (wc operation) then I gave up using it.

[cboos]

See the mailing list thread starting with this mail.

We'll see in milestone:0.12 if we will integrate Shun-ichi Goto's svn_ra.py backend in Trac core, or not. In the meantime, please give a try to his remove-svn-plugin and help him improving it.

[anonymous]

removed my email from CC

[Dan <trac@…>]

Replying to cboos:

See the mailing list thread starting with this mail. We'll see in milestone:0.12 if we will integrate Shun-ichi Goto's svn_ra.py backend in Trac core, or not. In the meantime, please give a try to his remove-svn-plugin and help him improving it.

I started using it, but I have not yet understood how to configure/update it with user authentication options. This is a stumbling point for me, because the remote repository(s) I want to target are not visible via anonymous access. The standard bindings want to use a callback function for this, but they don't seem to accept a Python function for this purpose. I'm investigating SWIG in general, but so far haven't found a way to implement this capability. (The document I read indicated this kind of callback is not supported.)

I took a look at pysvn, which seems that it would be very simple to add in the authentication callback function(s), although I haven't tried it yet. However, the Trac remote backend using pysvn is apparently a "dead end", as indicated above in comment:72 .

By the way, much thanks to Shun-ichi Goto for getting this feature going! It will be extremely useful IMHO.

[ryanmccue@…]

This is the one and only reason stopping me from using Trac. I've used Trac on previous projects and it rocks, but as my host doesn't support SVN repositories, I'm keeping them on Google Code and SourceForge?.

[anonymous]

Another me too. This feature would help me for some sourceforge projects and also in my company.

[anonymous]

Adding myself to the CC list - we need remote SVN access - I'm going to try the one that's been advertised on this ticket, but I'm worried about the alpha state of it. Luckily, all of our repositories have anonymous access, so we don't need svn auth for trac's interface. I wish you folks would up the priority on this one - I'd love to see it integrated in Milestone 0.11. Thanks for the great tool anyway!

[anonymous]

Replying to anonymous:

Adding myself to the CC list - we need remote SVN access - I'm going to try the one that's been advertised on this ticket, but I'm worried about the alpha state of it. Luckily, all of our repositories have anonymous access, so we don't need svn auth for trac's interface. I wish you folks would up the priority on this one - I'd love to see it integrated in Milestone 0.11. Thanks for the great tool anyway!

One more note: the primary reasons why we need remote access are not the usual IT department political reasons. Our team works on multiple projects that are scattered over a number of open source repositories - everything from eclipse plug-ins (and if you know anything about eclipse, then you know that their motto is, "IBM's way or the highway") to sourceforge projects to our own internal forge here at Novell. Trac is great, but we really need remote svn repositories.

[anonymous]

will you implement this feature request?

[martin.marcher@…]

Replying to anonymous:

will you implement this feature request?

iirc it has been the decision that trac will wait for the python bindings to provide this functionality and then implement it.

Afaik the problem is that pythons svn bindings only provide the functionality trac needs for file:/// URIs and not other protocols...

[cboos]

One additional note for the svnsync solution (see comment:64):

It might be useful to also run a svnsync copy-revprops command, to catch the changes to revision properties, see the svn book.

The output of that command should be parsed in order to get the list of modified changesets and then you should call trac-admin <env> resync $rev on those (if using Trac 0.10.4 or above), unless you already have a post-revprop-change hook which does this and svnsync copy-revprops calls that hook (I haven't verified that it calls it, but I assume it does).

[madduck]

The spam filter requires me to enter something here or else I can't just remove myself from Cc. Weird.

[radix@…]

You're lucky. I'm the reporter so I can't stop getting these god damned emails.

[cboos]

Replying to radix@twistedmatrix.com:

You're lucky. I'm the reporter so I can't stop getting these god damned emails.

Just ask ;-)

Btw, I started to write a more detailed step-by-step guide about how to setup a svnsync solution, I'll follow-up here when it's done.

[cmlenz]

(very sorry everyone, need to test config change to limit the amount of mails being sent out)

[anonymous]

Another me-too. I need svn+ssh remote access.

[yuangang28@…]

Replying to radix at twistedmatrix.com:

It stinks pretty bad that I have to install trac on my Subversion server AND give it write access to it.

[eblot]

Replying to radix at twistedmatrix.com: It stinks pretty bad that I have to install trac on my Subversion server AND give it write access to it.

Use trac with a remote Subversion repository;I want to have the character too

Please do not alter the ticket summary!

[Joymarquis]

I also have such requirement because we share source from a read-only SVN server that is owned by another department.

[eblot]

Replying to Joymarquis:

I also have such requirement because we share source from a read-only SVN server that is owned by another department.

If you have only a read-only access to the SVN repository you want to connect to Trac, an easy workaround is to create a local mirror of that remote repository and to keep your local repository in sync with a cron script.

[anonymous]

wasnt this scheduled for 0.11 ? its like the most needed feature in trac - thousands of sourceforge developers would turn to trac overnight if this was done... keeping in mind these are all OPEN SOURCE developers meaning quite a few would help out with trac development as well....