nALFS2 development

Kevin P. Fleming kpfleming at linuxfromscratch.org
Thu Aug 5 18:54:14 PDT 2004


Jamie Bennett wrote:

> Hi,
> 
>   This one mainly goes out to Kevin but in the vague hope that anyone
> else is working on this I thought I'd post it to the list. Are you
> Kevin, or anyone else, working on a nALFS replacement at the moment? I
> know Kevin talked about it but he's been busy with his new start-up at
> the moment and I've come to the point where this is needed by myself
> now.

This is correct... I am currently swamped with work, and I don't see 
much room for any concerted effort for nALFS for a month or two. I'm 
sorry to say that, the current state of nALFS SVN is bad, but see below :-)

>   I've been asked by my customers to provide a yum like source based
> package manager for the updating of their servers. Previously I've used
> nALFS but a more friendly GUI interface is what they desire which can be
> used over a network connection hence a definite split between the server
> and client. I've been toying with the idea of working on splitting nALFS
> properly but there are so many things that I wish to change that I feel
> it would be worth-while making a complete break and moving over to
> nALFS2.

I have been thinking along the same lines. I don't see a lot of point in 
putting effort into nALFS-1.X at this time, since it is doing what it 
has been asked to do in most cases (well, conditional execution is not 
released).

I will try to dump my thoughts about nALFS2 here... I may be missing 
some major items, but at least the discussion will be started.


> 
> 	o	complete split of front-end and back-end to allow 		different
> front-ends and the compiling of profiles over 		a network (think farming
> off profile compilation to one 		dedicated profile compiling server).

I'm not really sure what you mean by "profile compilation". My vision 
for nALFS2 was two pieces, nALFS and nALFSd, with nALFSd running only on 
the "target" system, where the build was actually going to occur. They 
could of course be run together on one system to do a build on that 
machine, and in fact the nALFS front-end should automatically start an 
instance of nALFSd (and talk to it over a socket pair) if it's not told 
to connect to an existing one.

> 	o	a new gui, web based for me, but could be anything 		(GTK/QT).

The protocol to be used between nALFS and nALFSd is the big issue here. 
It needs to be defined, documented and some sort of test-cases built so 
that the official front-/back-ends can be tested as well as third-party 
versions.

I have had many ideas about how to approach this: the current protocol 
is very lacking, in that it has no support for versioning, message 
encapsulation (start/end markers, checksums, etc.), it's not 
endian-safe, etc. The new protocol should cover all these bases and 
more... that may even dictate using something like SOAP over whatever 
communications channel the user has chosen. In spite of the relatively 
"heavy" weight of something like SOAP or XML-RPC, they have the distinct 
advantage of being well-known, standardized protocols with existing 
libraries to use.

> 
> 	o	SAX based parsing.

Ahhh... parsing. I don't much care whether it's SAX or DOM, because I 
don't see any reason for profiles to continue to be monstrous combined 
entities. Rather, I'd like to see each package be in its own profile, 
and nALFS can be told to load a whole bunch of them and them sort them 
into a usable order (based on dependencies, stage names, etc). This will 
require nALFS to be able to feed entity values into the parser (which 
can be done with libxml2), and this is good because there has been a 
need to be able to provide environment variable values as entities 
within profiles for some time.

All of this parsing, validation and sorting activity has to happen on 
the daemon side, IMHO, because it's dependent on the build host, not the 
control host. So the control host would just load the profiles from 
local files, web servers, wherever, and send them to the daemon to 
assemble into a "working" set that can then be displayed by the control 
host.

> 	o	package downloading and verification before compilation.

Yep, this is a biggie. We also need things like "actions to take on 
stage entry" and "actions to take on stage exit", to facilitate things 
like mounting/unmounting filesystems without having to manually mark 
those steps.

> 	o	dependency resolution.

Absolutely, and this will be a big task. Looking at my comments above, 
you can see that I'd prefer to have as little dependency information in 
the profiles as we can get away with (only what is absolutely required). 
nALFS should be able do generate dependency graphs that can then be used 
by the user interface to inform the user which packages need each other, 
which ones need to be built in which order, automatically select 
dependent packages when required, automatically rebuild packages based 
on their dependent packages being new major/minor/etc. versions, and 
probably other things I've not yet thought of.

However, you have to keep in mind that all this dependency management 
still has to happen on the daemon host; even simple things like marking 
a package to build would be communicated by the front end to the back 
end, where it can then respond back with which packages actually got 
marked. I see the front end as having very little involvement with the 
contents of the profiles at all, and in fact I'd rather see the back end 
feed a "display tree" to the front end for its use, rather than any 
actual profile contents.

> 	o	proper logging for un-installation.

Agreed, and I think my previous proposal would be a good start.

>   Now I'd be happier to work along side other people to develop a tool
> that could do all/most of the above as it would save me time and effort
> but if need be I'll be going solo as this is something that is a
> necessity for me now. 

I am thrilled to see some effort moving in this direction, and while I 
can't provide much coding time right now I'd be happy to provide any 
level of involvement you wish, from managing the project down to just 
being a resource for another viewpoint when you want it.

Previously we discussed possibly using BitKeeper for this project, as it 
allows for more distributed development and it's easier to keep multiple 
local trees with various bits in them. I'm still keen on that idea, and 
I don't know how recent events in the LFS community may have changed the 
community's opinion about us using it for this project. I'm not 
suggesting that it be used for any other part of the ALFS project at 
this stage, just nALFS2. If that means we do it on bkbits.net, or on a 
server I (or someone else) supplies, that's OK by me, as long as the 
rest of the community doesn't see it as anything more than a technical 
decision.



More information about the alfs-discuss mailing list