[PROPOSAL] TWiki and mirroring
lizardo at linuxfromscratch.org
Sun Aug 1 12:52:23 PDT 2004
I know this had been discussed before, but I still have some doubts/concerns
regarding how the TWiki mirroring will happen, and maybe some mirror admins
share them too. Here they are:
- The mirrors will require a TWiki installation to be able to process the data
files. Although TWiki will be mirrored along with the data, **the partition
where the mirrored data is stored will need to be mounted with "exec"
option**. This is default for "mount" but some mirrors would have been used
"noexec" for improved security and therefore the Perl scripts will not run on
them. The same occurs for chrooted Apache installations that may not have
access to /usr/bin/perl (except if they install Perl itself on the chroot...)
- Although we can garantee that we will always keep the TWiki installation
up-to-date with the most recent fixes, we cannot garantee that TWiki itself
is bug-free. The mirrors have to know that. Some may even argue that this is
a strong point against allowing executable data to be stored on their
servers, and I will have to agree here.
- AFAIK, there are mainly three types of Apache installations: the "global",
non-chroot one, where all VHOSTs share the same Apache installation (although
they have separate directory trees and file permissions); the global one, but
inside a chroot environment; and the very restrictive one, where each single
VHOST has its own root tree inside a chroot environment. I don't know if any
mirroring site uses this last, but some services offering shell access
(Savannah?) lock-in each user account on its own chroot jail.
- http://test.linuxfromscratch.org/twiki/bin/view/Main/WebsiteMirrors says
that port 873 will be used by the rsync daemon. AFAIK, this is a low port and
requires special privileges to be bound. Can rsync's bind port be changed so
it can be run by a regular user?
Now I'll stop complaining and offer some suggestions for solutions :)
- Adapt TWiki so we can completely separate the non-executable data from the
executable one (e.g. scripts, modules [do they really need to be
executable?]). Then host the static data, **already processed to HTML**
(possibly using http://twiki.org/cgi-bin/view/Plugins/PublishAddOn), on a
separate VHOST (e.g. www.linuxfromscratch.org). The scripts will be on
another VHOST (edit.lfs.org, scripts.lfs.org or just lfs.org, but see below).
- mirror the static data using the pushing method suggested by J. Huntwork.
Even if we have to continue to use a non-CMS solution, I think we should move
to push rsync'ing as it resolves another long-standing problem (mirrors being
- I remember some people arguing that enabling edits only for belgarath would
increase bandwidth usage. I offer a possible solution for this: additionally
to having mirrors hosting static data, some which have no concerns regarding
the items above can co-operate with belgarath on hosting the scripts and
non-processed data (i.e. a complete TWiki installation). Those include
possible mirrors which already host TWiki-powered projects and feel confident
about it (as anyone else should :). The requirements for these "special"
mirrors will be no different from what we have currently on
- Move from our current CGI/Apache authentication method to https, as
suggested by Jeroen. I have my doubts if the current method will work well
with these cross-site redirections from "edit mode" to "view mode" and
vice-versa. Additionally, CGI::Session likes to create small cookie files on
server's /tmp dir (some belgarath users may have noticed how messed
is /tmp :) and is not scalable enough, IMHO (the docs explain that Apache
authentication becomes slower the bigger the .htpasswd database is).
The HTTPS authentication will (hopefully) work like some popular Webmail
services which offer secure logins. You start with a non encrypted login
form, which sends your login data through POST to the login server and then
is redirected to the service's main page, no authenticated. Of course, we can
keep the user authenticated during all session duration (like SourceForge),
but I wonder if this will have a big impact on server's performance.
Sorry for the long post. I hope to have expressed all possible future concerns
regarding TWiki, and to have presented some food for possible solutions.
lizardo at linuxfromscratch.org
More information about the website