language choice of alfs
zhouhui at wam.umd.edu
Mon Dec 20 20:38:51 PST 2004
On Mon, Dec 20, 2004 at 09:52:16PM -0600, James Robertson wrote:
>Hui Zhou wrote:
>>On Mon, Dec 20, 2004 at 08:49:08PM +0100, Joachim Beckers wrote:
>>>IMO, the backend should be written in some compilable language, like C
>>>or so. That's important because it will have to work under heavy loads
>>>when parsing profiles,
>>The existing profiles for lfs is not 'heavy' at all. The whole blfs
>>profile may be heavier, but again the parsing part only happens at
>>initialization, a few minutes compared to hours of building time really
>>won't bother a lot of users. Once the profiles includes packages as many
>>as those in debian depositories, it is indeed heavy, but at that point,
>>a better solution is to split the profiles into a data base, otherwise,
>>even a fastest engine will feel slow.
>Remember that the new alfs tool is going to do a lot more validation and
> parsing of profiles than in the past.
To me the validation is just parsing at semantic layer. The general
XML validation involves parsing DTD and perform generlized checking
routine which involves a lot of overhead. I always hated all these XML
hypes. The actual alfs profile validation is just checking to see
whether certain data node such as unpacking is valid under certain
trees such as stage or package. It is not really that complicated and
can easily hard coded into alfs. In many case, reading the hard coded
high level implementation is not really more difficult than reading
the actual DTDs.
And I think I missed the 'a lot more' part. What/How exactly is that?
>There is going to be a need for
>lots of threads to be running simultaneously to have things keep up.
I understand the importance of reducing the overhead of opening
threads of webservers, which is exactly the performance that measures.
But they are talking about spawning thousands of threads per second.
How many threads or more importantly how fast the threads in alfs are
>would bet that some kind of event system will need to be implemented in
>the back end so that the front end will know what is going on and to
>keep all the threads in check.
My understanding of event system is fire and Wait. Exactly how often
the events and what kind of events are planed?
>>>copying files, building packages,
>>nALFS spawns shell run these tasks. Unless all these tasks were recoded
>>in alfs, language choices won't make measurable differences.
>I do not think that the handlers in current nALFS are going to be used.
> IIUC, the new alfs tool will do that stuff differently.
I don't think alfs will take part in the job gcc and do actual
compiling and linking. Is it planed (even in the Long future) to
do the autoconfigure function (which is awfully slow as they runs M4
then bash and DOES a lot of unnecessary checking and some are even
meaningless). alfs will know the system better than any other program,
so it really has the potential to do a much better job than autotools.
I guess alfs will not do the decompression either. Is it going to do
the http protocol and download the file? That will reduce extra
package dependency and certainly very welcome.
Other than those, it's hard to think of any place that can be sqeezed
to save any time.
Doing the configure won't require best performance in the code and
the speed of download is mostly limited by bandwidth.
>>>logging packages' output and details, aso.
>>If the logging is just streaming out to a file, the overhead of these is
>>immeasurable, unless the logfile has a complicated structure and/or alfs
>>need do some formating before output. In most case, the IO shadow's
>Logging will be in an XML DTD based file. See the SRS on the wiki.
XML still can be streamed. Another option is have a seperate program
to analyse the streamed logfiles and do the proper(XML) loggin as a
post process. IMO, cleaner and probably faster too.
Still I suspect there is any concern that interpreted language will be
too slow for even realtime log processing.
Is there a plan to let alfs to parse the gcc output as well as auto
configure output? I mean actual parse so it can understand all the
infomation such as warnings, misconfigurations, check summaries and so
it can produce interesting summaries?
You are welcome. I am quite busy and I have some other interests which
made me couldn't commit myself. But I will help as much as I can.
I don't really care which language you guys uses as long as I am not
forced to work with it. And I actually am a C fan.
More information about the alfs-discuss