[Fwd: Rsyncing change + Round Robin DNS status update]
gerard at linuxfromscratch.org
Thu Nov 6 21:57:25 PST 2003
From: Gerard Beekmans <gerard at linuxfromscratch.org>
To: lfs-admin at linuxfromscratch.org
Subject: Rsyncing change + Round Robin DNS status update
Date: Thu, 06 Nov 2003 22:48:12 -0700
Some of you are new on this list (welcome on board, by the way) so let
me quickly summarize an issue that has been oustanding for a while:
I want to start using Round Robin DNS to offload more bandwidth onto our
mirrors. The colocation facility I have the server in has a max monthly
bandwidth usage (I can use more, but I gotto pay which I rather not) and
we're hitting it pretty close lately.
The one concern I had with round robin DNS is how to deal with stale
mirrors. The solution is pretty simple, just took me a bit to free up
the time to write the scripts to aid in this task.
I have written a new rsync script which I have attached to this email.
What it does it rsyncs the '<ROOTDIR/timestamp/timestamp' file and
compares the contents of it to the one on your mirror copy. If the
timestamp files are identical, nothing further happens. If the just
downloaded timestamp is newer (its contents in the form of date +%s is
greater than your mirror copy) it will run a regular rsync to sync the
entire site. The reason for this is to reduce some of the rsync overhead
(bandwidth used to send the file list and also cpu cycles md5sum'ing).
I'd like you guys to run this script twice an hour. The the vast
majority of the day, you will only rsync a few bytes every 30 minutes or
so, the load won't be high. At least twice a day (when the website
updates by cron) you will rsync a new site with the new LFS/BLFS CVS
books, hints, patches and whatever else happened to have changed.
If you are ready to schedule this new script to run, please give me an
email off-list. I'd hate for everybody to run the script at the same
time and suck all the bandwidth during that time frame. I'll coordinate
with you guys privately when I'd like you to run it. Or if you have a
specific time it needs to run, we'll work it out.
When all you guys are switched over, we can start on Round Robin DNS.
I'll email more about that when I've got you guys setup for rsyncing.
I'm sorry for this hassle, but it will be in my best interest that I get
this coordinated properly and reduce bandwidth before the bill gets too
high. The old "run whenever you want once a day" used to work fine when
we had 5 mirrors. Now that there are about 30, a bit more coordinate is
// Linux Consultant --- OSDN / DevChannel
// If Linux doesn't have the solution, you have the wrong problem
More information about the website