An experimental design for (x)alfs

pak_lfs at pak_lfs at
Thu Nov 24 12:01:26 PST 2005

( This is a long post, so please bear with me)

I might not contribute a lot (sorry, severe lack of time) but I 've been here 
since the time ALFS DTD was being designed, so my experiences so far
might be useful.

I 've been among those who supported the use of XML for ALFS but I believe
I 've been wrong. The nice feature of XML is being declarative, i.e., stating 
only *what* needs to be done, not *how*. The problem with ALFS IMHO is
that it is trying to be declarative in too low a level.

The consequences of this are: 
        * You need to reproduce all the features of the language you
           are trying to replace (e.g., regular expressions in search_replace)
           This takes (IMO unjustified) precious development effort.

        * You are typing much and saying little.

        * You force everybody to use this XML format to write *all* the
           instructions which means people have to translate everything to
           this, instead of just copy+pasting stuff they find on the internet
          (usually bash/python scripts).

         * For this reason, it is also difficult to have the ALFS profiles for
           LFS autogenerated from the book.

After about 3 years of heavy ALFS/nALFS (I 've been using it since the perl
thing) this has started to hurt enough that I had to stop using it and move
back to custom scripts + jhalfs for testing (recently). It seems that other
LFSers seem to face these problems too. So, my proposal is to redesign
the ALFS "language", so that the above problems can be more easily manageable.
So, here is an example "design", provided mainly as food for thought:

I propose  that there are only two concepts 
(can also think them as "tags" in a new "DTD"): A Job and a Stage.
A Job consists of one or more Stages. Stages are "black boxes", i.e., they
can be bash/perl/python scripts, binary executables, nALFS profiles, or
whatever else can be executed by just invoking its name.
Each Job "describes" its stages to the engine so that different Jobs can  have
the same or different stages.

Now the protocol between the client and the Server is very simple: The client
sends Job/stage pairs to the Server and the Server executes them. In cases
where the order is known, the client can send a "pack" of  Jobs/stages to
the Server for better performance. When the order needs to be defined 
automatically through dependency resolution, either the client can do it
(if the resulting order will be the same for many Servers), or the resolution
can be delegated to the server through a special "composite" job.

How are the Jobs/stages  updated: Both the client and the Server have
a database of Jobs/stages (e.g., implemented as a directory "Jobs/"
somewhere).  On every request, the Server, syncs its database with
the client's through a protocol like rsync or svnup. Since a Request
can consist of a pack of Jobs/Stages, this is no big performance problem.
Therefore, everytime you want to update something, you just update your
*local* Jobbase and the changes are propagated to the Servers you connect to.

Now, the good stuff: 

   *   Jobs can use inheritance in the same way as the
Object-Oriented Bash(!) uses it, so many times, e.g., for standard packages
using autoconf profiles could be as small as one line! (the URL)

   *     I believe it won't be hard to add appropriate "tags" to the LFS book
   XML source to automatically extract stage commands. E.g., just as you
   now tag some text as "sources" so that they can be extracted, it is also
   possible to tag it as "prepare" "build" "install" etc.

    *   So, both those who want something to parse LFS book sources
    and those who want to do their own work but without having  to type much
    can live with such a tool.

     *  Logging functionality etc, is I believe largely orthogonal to this so 
     it can easily be added to the tool after the base functionality is in

      * It is not hard either to imagine a later refactoring to add a concept
       of "features" to the mix. Then you can have conditional compilation
       as you just pass the wanted features as arguments to the stages
       and have them "do the right thing", sort of how gentoo's "USE FLAGS"

      * Jobs and Stages can be expressed either as Directories/Files
      or XML tags and also communicated in either an XML, text or binary

So, what do you think?
If you don't mind modern C++/Boost, maybe a proof of concept 
implementation would not be too hard to code (Of course if I start 
working at it I 'm sure I will think otherwise, which is why I seek your
opinions first).


P.S. I called this (x)alfs since maybe it should be called something
like palfs (also standing for "proof of concept alfs"), unless people
like the design and want to base alfs upon it.

____________________________________________________________________ - äùñåÜí õðçñåóßá çëåêôñïíéêïý ôá÷õäñïìåßïõ. - free email service for the Greek-speaking.

More information about the alfs-discuss mailing list