update-website.mk and mirroring issue

Anderson Lizardo lizardo at linuxfromscratch.org
Sat Nov 20 10:32:42 PST 2004


Hi,

While re-checking the update-website.mk script (I plan to be more careful 
now :-), I noticed it currently only updates the timestamp file when the 
"update-www" rule runa. This means that the mirrors will not rsync until we 
commit something to the www repos.

I plan to "workaround" this by creating a "update-timestamp" rule and running 
it (using fcron) with the same frequency of the old update-website.sh, so the 
mirrors will behave exactly like now.

But then I ask: what's the real purpose of the "timestamp" file? I know it 
allows the mirrors to only run rsync if this file is changed, but does rsync 
not do this check already?

Another thing: with the current 
http://linuxfromscratch.org/~gerard/lfs-rsync.sh script used by all mirrors, 
the ".svn" dir (along with other files, like news-YYYY.txt, templates) are 
rsync'ed even though they are not necessary on the "final" website. This is 
avoidable by using the --exclude-from rsync option (see update-website.mk's 
"run-rsync" rule), and saves some MB of bandwidth (aprox. 12MB of .svn dirs).

Finally, I suggest updating lfs-rsync.sh and asking mirrors to update their 
scripts so they can avoid downloading those unecessary bytes.

We can still proceed with the new script migration although I suspect we will 
easily reach the 50MB minimal storage space listed on 
http://www.linuxfromscratch.org/becomemirror.html because of the .svn dirs. 
(FYI, the current website has 49MB already).

Guys, what do you think about these issues?
-- 
Anderson Lizardo
lizardo at linuxfromscratch.org
http://www.linuxfromscratch.org/



More information about the website mailing list