language choice of alfs

Hui Zhou zhouhui at wam.umd.edu
Mon Dec 20 20:38:51 PST 2004


On Mon, Dec 20, 2004 at 09:52:16PM -0600, James Robertson wrote:
>Hui Zhou wrote:
>>On Mon, Dec 20, 2004 at 08:49:08PM +0100, Joachim Beckers wrote:
>>
>>>IMO, the backend should be written in some compilable language, like C 
>>>or so. That's important because it will have to work under heavy loads 
>>>when parsing profiles, 
>>
>>The existing profiles for lfs is not 'heavy' at all. The whole blfs 
>>profile may be heavier, but again the parsing part only happens at 
>>initialization, a few minutes compared to hours of building time really 
>>won't bother a lot of users. Once the profiles includes packages as many 
>>as those in debian depositories, it is indeed heavy, but at that point, 
>>a better solution is to split the profiles into a data base, otherwise, 
>>even a fastest engine will feel slow.
>
>Remember that the new alfs tool is going to do a lot more validation and 
>  parsing of profiles than in the past.  

To me the validation is just parsing at semantic layer. The general 
XML validation involves parsing DTD and perform generlized checking 
routine which involves a lot of overhead. I always hated all these XML 
hypes. The actual alfs profile validation is just checking to see 
whether certain data node such as unpacking is valid under certain 
trees such as stage or package. It is not really that complicated and 
can easily hard coded into alfs. In many case, reading the hard coded 
high level implementation is not really more difficult than reading 
the actual DTDs.

And I think I missed the 'a lot more' part. What/How exactly is that?

>There is going to be a need for 
>lots of threads to be running simultaneously to have things keep up.

I understand the importance of reducing the overhead of opening 
threads of webservers, which is exactly the performance that measures. 
But they are talking about spawning thousands of threads per second. 
How many threads or more importantly how fast the threads in alfs are 
planned?

>I 
>would bet that some kind of event system will need to be implemented in 
>the back end so that the front end will know what is going on and to 
>keep all the threads in check.

My understanding of event system is fire and Wait. Exactly how often 
the events and what kind of events are planed?
>
>>
>>>copying files, building packages, 
>>
>>nALFS spawns shell run these tasks. Unless all these tasks were recoded 
>>in alfs, language choices won't make measurable differences.
>
>I do not think that the handlers in current nALFS are going to be used. 
>  IIUC, the new alfs tool will do that stuff differently.

I don't think alfs will take part in the job gcc and do actual 
compiling and linking. Is it planed (even in the Long future) to 
do the autoconfigure function (which is awfully slow as they runs M4 
then bash and DOES a lot of unnecessary checking and some are even 
meaningless). alfs will know the system better than any other program, 
so it really has the potential to do a much better job than autotools. 

I guess alfs will not do the decompression either. Is it going to do 
the http protocol and download the file? That will reduce extra 
package dependency and certainly very welcome.

Other than those, it's hard to think of any place that can be sqeezed 
to save any time.

Doing the configure won't require best performance in the code and 
the speed of download is mostly limited by bandwidth. 

>>>logging packages' output and details, aso. 
>>
>>If the logging is just streaming out to a file, the overhead of these is 
>>immeasurable, unless the logfile has a complicated structure and/or alfs 
>>need do some formating before output. In most case, the IO shadow's 
>>every thing.
>
>Logging will be in an XML DTD based file.  See the SRS on the wiki.

XML still can be streamed. Another option is have a seperate program 
to analyse the streamed logfiles and do the proper(XML) loggin as a 
post process. IMO, cleaner and probably faster too.

Still I suspect there is any concern that interpreted language will be 
too slow for even realtime log processing.

Is there a plan to let alfs to parse the gcc output as well as auto 
configure output? I mean actual parse so it can understand all the 
infomation such as warnings, misconfigurations, check summaries and so 
it can produce interesting summaries?

>Thanks

You are welcome. I am quite busy and I have some other interests which 
made me couldn't commit myself. But I will help as much as I can.

I don't really care which language you guys uses as long as I am not 
forced to work with it. And I actually am a C fan.

-- 
Hui Zhou



More information about the alfs-discuss mailing list