update-website.mk and mirroring issue

Anderson Lizardo lizardo at linuxfromscratch.org
Sat Nov 20 10:32:42 PST 2004


While re-checking the update-website.mk script (I plan to be more careful 
now :-), I noticed it currently only updates the timestamp file when the 
"update-www" rule runa. This means that the mirrors will not rsync until we 
commit something to the www repos.

I plan to "workaround" this by creating a "update-timestamp" rule and running 
it (using fcron) with the same frequency of the old update-website.sh, so the 
mirrors will behave exactly like now.

But then I ask: what's the real purpose of the "timestamp" file? I know it 
allows the mirrors to only run rsync if this file is changed, but does rsync 
not do this check already?

Another thing: with the current 
http://linuxfromscratch.org/~gerard/lfs-rsync.sh script used by all mirrors, 
the ".svn" dir (along with other files, like news-YYYY.txt, templates) are 
rsync'ed even though they are not necessary on the "final" website. This is 
avoidable by using the --exclude-from rsync option (see update-website.mk's 
"run-rsync" rule), and saves some MB of bandwidth (aprox. 12MB of .svn dirs).

Finally, I suggest updating lfs-rsync.sh and asking mirrors to update their 
scripts so they can avoid downloading those unecessary bytes.

We can still proceed with the new script migration although I suspect we will 
easily reach the 50MB minimal storage space listed on 
http://www.linuxfromscratch.org/becomemirror.html because of the .svn dirs. 
(FYI, the current website has 49MB already).

Guys, what do you think about these issues?
Anderson Lizardo
lizardo at linuxfromscratch.org

More information about the website mailing list