[PROPOSAL] TWiki and mirroring

Anderson Lizardo lizardo at linuxfromscratch.org
Mon Aug 2 11:26:31 PDT 2004

On Monday 02 August 2004 04:32, Jeroen Coumans wrote:
> Anderson Lizardo said the following on 01-08-2004 21:52:
> > I know this had been discussed before, but I still have some
> > doubts/concerns regarding how the TWiki mirroring will happen, and maybe
> > some mirror admins share them too. Here they are:
> Well, as far as your concerns are shared with mirror admins; I haven't
> had such feedback yet on lfs-admin. So they're either silent or they
> were in agreement with our proposal :)

Yep, I know. Maybe I just wrote this paragraph wrongly; I meant to show some 
"implicit" requirements (like the "exec" mount option) to see if it conflicts 
with some of our mirrors setup.

> > - mirror the static data using the pushing method suggested by J.
> > Huntwork. Even if we have to continue to use a non-CMS solution, I think
> > we should move to push rsync'ing as it resolves another long-standing
> > problem (mirrors being constantly outdated).
> Yes, this should probably be our first priority. Once we have the
> necessary mechanisms in place for this, it will also be easier to setup
> TWiki-mirroring.

I agree, and seeing from the mirrors POV, this will actually be the "hard 
work" for them, as it's the mirroring infrastructure that will be touched. If 
we mirror a TWiki installation or just the processed data through this 
infrasctructure, it will not be changed again (hopefully!).

> > - I remember some people arguing that enabling edits only for belgarath
> > would increase bandwidth usage. I offer a possible solution for this:
> > additionally to having mirrors hosting static data, some which have no
> > concerns regarding the items above can co-operate with belgarath on
> > hosting the scripts and non-processed data (i.e. a complete TWiki
> > installation). Those include possible mirrors which already host
> > TWiki-powered projects and feel confident about it (as anyone else should
> > :). The requirements for these "special" mirrors will be no different
> > from what we have currently on
> > http://test.linuxfromscratch.org/twiki/bin/view/Main/WebsiteMirrors.
> [...]
> In short: this seems to be more trouble than it

Hm, I'll have to agree here. I completely forgot the possible race conditions 
and why there is a "edit lock" feature on TWiki.

> > - Move from our current CGI/Apache authentication method to https, as
> > suggested by Jeroen. I have my doubts if the current method will work
> > well with these cross-site redirections from "edit mode" to "view mode"
> > and vice-versa. Additionally, CGI::Session likes to create small cookie
> > files on server's /tmp dir (some belgarath users may have noticed how
> > messed is /tmp :) and is not scalable enough, IMHO (the docs explain that
> > Apache authentication becomes slower the bigger the .htpasswd database
> > is).
> Yes, we already discussed this a bit when Jeremy was researching methods
> to integrate the current LFS user database with TWiki. Althoug I
> originally only suggested to use HTTPS for edit traffic, not for
> authentication. But does TWiki allow secure authentication?

Yes, see 
Although it says this aproach enables authentication for both viewing and 
editing, if we do completely separation between the scripts and static data, 
and the data on www.lfs.org is already processed, there will be no need for a 
"view" script. The only HTTPS-enabled VHOST will then be 

I really need to test all this locally... will do right after we setup the 
push mirroring for the current site.

> As long as it's all feasable, most of it seems reasonable. And, like you
> said, we can already work out some things (like rsync push) before we
> migrate to TWiki.

Agreed. I'll start doing the preparations for the rsync push. Jeremy H., you 
already have "rsync push" running on your server right? Can we setup it so we 
can test things with the current site? After we have this setup for most 
mirrors (I suppose we don't need to wait all mirrors to enable pushing so we 
can start using it, right?), we can continue working on TWiki.

Anderson Lizardo
lizardo at linuxfromscratch.org

More information about the website mailing list