TWiki Mirror [was: Oregon mirror is not up to date]

Jeremy Huntwork jhuntwork at
Sun Jul 11 16:38:42 PDT 2004

On Sunday 11 July 2004 06:54 pm, Jeroen Coumans wrote:
> Jeremy Huntwork said the following on 11-07-2004 23:40:
> > As far as complication goes, it's true it does raise the bar, but
> > really, I think the complication stems from trying to build a
> > user-editable system with international mirrors. :)
> Think of it as a challenge ... :-)
> > But, I'm open to other ideas...
> Rsync is perfectly usable without ssh; it's how it's currently setup
> too. Although right now, mirrors *pull* data, and we're trying to
> *push data. I'm not sure, but as long as no sensitive data needs to
> be synchronized (ie. the user authentication stays on belgerath), we
> don't need to use ssh. I'd really prefer this because I'm afraid we'd
> lose quite some mirrors if we require them to have sshd and an ssh
> account to mirror us. (The alternative would be the HTML export)

The problem lies in the fact that we are trying to initiate a sync 
server-side.  The only method with rsync I'm finding to do that is 
through ssh.  Really, what is happening is this: an ssh user on 
belgarath makes a connection to the mirror server, then, after the 
connection is made, runs rsync on the mirror side.  In effect, we're 
*telling* the mirror to sync.  It seems a pretty standard way to do it:

Otherwise, using rsync we'd have to have the mirrors run cron jobs every 
minute or so to look for updates which I'm sure is not ideal for 

How does *our* server write files to their server (or at least notifies 
them to come and get it!) as soon as a change is made?

In the meantime, I can keep looking...

> I've probably written this down already, but here's my ideal
> mirroring system:
> - mirrors rsync our www (now: test) directory, excluding the TWiki
> passwd file. Mirrors thus require Perl, mod_perl and ... (fill in)

However we go about it, we can specify exactly what to sync.  I just put 
the whole site over in my test for simplicity's sake.

> - we provide a virtual directory directive for httpd.conf or
> .htaccess so the TWiki installation is usable on the mirror
> - for /edit/, a mirror's .htaccess or httpd.conf redirects to
> belgerath, where the user is authenticated
> - on belgerath, each /save/ triggers an rsync push to all mirrors
> - users automatically surf mirrors through RR-DNS, except to edit
> pages. - mirrors which aren't up-to-date automatically get filtered
> out.

Jeremy Huntwork

More information about the website mailing list