New mirrors requeriments. Was: Re: so long and thanks for the fish
jeroen at linuxfromscratch.org
Mon Jul 19 03:43:14 PDT 2004
M.Canales.es said the following on 19-07-2004 12:14:
> El Lun 19 Jul 2004 11:22 Jeroen Coumans escrbi�:
>>If anyone has problems with the new requirements, then please speak up.
> I have problems. Isn't possible to me run the rsync daemon or create an ssh
> account. The lfsmirror.lfs-es.org mirror is hosted at a commercial server that
> don't want activate that services.
We have a third option: static HTML export. We can let TWiki export HTML
pages and let mirrors who can't use the "push" mechanisms mirror these
HTML pages instead. It would mean that we'll have to seperate mirrors
into two categories:
1. primary mirrors (TWiki mirrors; always up-to-date)
2. secondary mirrors (HTML; may be 12 hours out-of-date due to
We'll have to figure out a selection mechanism though. My preference is
to make it very transparant: you'll always get a primary mirror if
you're logged in, otherwise you'll get a random mirror from the complete
pool. Perhaps we can offer a select box for users to always get a
Anyway, in case of the HTML export, you won't have to change anything on
your mirror. It would mean you aren't always up-to-date, but at least
the user will be aware of that, which is better then the current
mechanism, where mirrors can get out-of-date and we don't have any
control over it, nor can we notify users of it. This has caused user
frustration multiple times in the past.
> On the other side, this continous rsyncing with each mirror to synchronize each
> TWiki edited page (and the direct connection to belgarath to make the the login
> & page editions), don't could will increase the bandwhitd usage on belgarath?
Rsync is very bandwidth-friendly: it will only transfer the actual
differences. Thus, if one page is changed, only that page will get
transferred to all mirrors. How many pages will get changed per day? If
the current rate of updating is any indication, we'll have to count per
week instead of per day. But lets say that we'll have 10 updates per
day. Pages are only 4-20Kb each.
Hypothetically speaking, we'll have:
10 updates * 20Kb * 20 mirrors = 2000Kb per day. That's less then 2Mb
per day. For June, we used 27.860.946 Kb bandwidth on belgerath (see
http://belgarath.linuxfromscratch.org/webalizer/all/). 30x2000=60.000 Kb
which is (60.000/27.860.946)*100=0.2% of the bandwidth.
Hope that relieves your concerns!
More information about the website