TWiki Mirror [was: Oregon mirror is not up to date]

Jeremy Huntwork jhuntwork at linuxfromscratch.org
Sun Jul 11 16:38:42 PDT 2004


On Sunday 11 July 2004 06:54 pm, Jeroen Coumans wrote:
> Jeremy Huntwork said the following on 11-07-2004 23:40:
> > As far as complication goes, it's true it does raise the bar, but
> > really, I think the complication stems from trying to build a
> > user-editable system with international mirrors. :)
>
> Think of it as a challenge ... :-)
>
> > But, I'm open to other ideas...
>
> Rsync is perfectly usable without ssh; it's how it's currently setup
> too. Although right now, mirrors *pull* data, and we're trying to
> *push data. I'm not sure, but as long as no sensitive data needs to
> be synchronized (ie. the user authentication stays on belgerath), we
> don't need to use ssh. I'd really prefer this because I'm afraid we'd
> lose quite some mirrors if we require them to have sshd and an ssh
> account to mirror us. (The alternative would be the HTML export)

The problem lies in the fact that we are trying to initiate a sync 
server-side.  The only method with rsync I'm finding to do that is 
through ssh.  Really, what is happening is this: an ssh user on 
belgarath makes a connection to the mirror server, then, after the 
connection is made, runs rsync on the mirror side.  In effect, we're 
*telling* the mirror to sync.  It seems a pretty standard way to do it:

http://www.debian.org/mirror/push_server

Otherwise, using rsync we'd have to have the mirrors run cron jobs every 
minute or so to look for updates which I'm sure is not ideal for 
anyone.

How does *our* server write files to their server (or at least notifies 
them to come and get it!) as soon as a change is made?

In the meantime, I can keep looking...

> I've probably written this down already, but here's my ideal
> mirroring system:
> - mirrors rsync our www (now: test) directory, excluding the TWiki
> passwd file. Mirrors thus require Perl, mod_perl and ... (fill in)

However we go about it, we can specify exactly what to sync.  I just put 
the whole site over in my test for simplicity's sake.

> - we provide a virtual directory directive for httpd.conf or
> .htaccess so the TWiki installation is usable on the mirror
> - for /edit/, a mirror's .htaccess or httpd.conf redirects to
> belgerath, where the user is authenticated
> - on belgerath, each /save/ triggers an rsync push to all mirrors
> - users automatically surf mirrors through RR-DNS, except to edit
> pages. - mirrors which aren't up-to-date automatically get filtered
> out.

-- 
Jeremy Huntwork
http://www.jenacon.net



More information about the website mailing list