[RFC] nALFS package log files

Kevin P. Fleming kpfleming at linuxfromscratch.org
Tue Feb 10 18:08:21 PST 2004


As referred to in my "nALFS 1.3 plans" message, I am seriously 
considering reworking the existing package logging code in nALFS. There 
are some Bugzilla entries related to this, there are FIXME/TODO items in 
the code that need to be addressed, and the 
package-built/package-version conditional verbs really need it to work 
properly.

As it stands today, with the last batch of logging work that Neven did, 
nALFS creates two files in the packages/ directory for each package that 
runs. One of them is the file list gathered from the post-package scan 
(if that is enabled), the other is an XML log file containing details of 
the package run (or more than one package run).

The structure of that XML file is not currently documented, nor is 
anything really dependent on it outside of nALFS at this point. If we 
want it to become useful, though, it needs quite a bit of work.

For example, at the moment it generates XML nodes "in order", but XML 
does not guarantee the "order" of child nodes as best I can tell. As a 
result, when the in-memory document is dumped out into a text file, they 
nodes do not always appear in the order they were added (I have plenty 
of log files here that have stage start/end pairs reversed, or even 
stage end markers _after_ the package end marker).

Also, the conversion to XML log files meant that the actual output 
capture (from the stdout/stderr streams as the handlers run) was lost; 
Neven did not include it in the XML log files, which I feel is a great loss.

I am hereby proposing this new structure for the package log files; it 
is very much a work-in-progress (no code has been written at this 
point), and all opinions are more than welcome. Example log file, 
containing two runs of a package, one that failed and one that succeeded 
(with simple timestamps to get the point across, they would be real 
timestamps in an actual log file):

<xml ...>
<package_log>
   <package_run>
     <name>bash</name>
     <version>2.05b</version>
     <start>X+0</start>
     <stage name="unpack">
       <start>X+0</start>
       <handler name="unpack">
         <start>X+0</start>
         <end status="success">X+2</end>
         <output>
           =line 1
           =line 2
           =line 3
           =...
         </output>
       </handler>
       <end status="success">X+2</end>
     </stage>
     <stage name="build">
       <start>X+2</start>
       <handler name="configure">
         <start>X+2</start>
         <end status="success">X+5</end>
         <output>
           =line 1
           =line 2
           =line 3
           =...
         </output>
       </handler>
       <handler name="make">
         <start>X+5</start>
         <end status="success">X+12</end>
         <output>
           =line 1
           =line 2
           =line 3
           =...
         </output>
       </handler>
       <end status="success">X+12</end>
     </stage>
     <stage name="install">
       <start>X+12</start>
       <end status="fail">X+15</end>
       <handler name="make">
         <start>X+12</start>
         <end status="fail">X+15</end>
         <output>
           =line 1
           =line 2
           =line 3
           =...
         </output>
       </handler>
     </stage>
     <end status="fail">X+15</end>
   </package_run>
   <package_run>
     <end status="success">X+55</end>
     <stage name="unpack">
       <start>X+30</start>
       <handler name="unpack">
         <start>X+30</start>
         <end status="success">X+32</end>
         <output>
           =line 1
           =line 2
           =line 3
           =...
         </output>
       </handler>
       <end status="success">X+32</end>
     </stage>
     <stage name="build">
       <start>X+32</start>
       <handler name="configure">
         <start>X+32</start>
         <end status="success">X+35</end>
         <output>
           =line 1
           =line 2
           =line 3
           =...
         </output>
       </handler>
       <handler name="make">
         <start>X+35</start>
         <end status="success">X+42</end>
         <output>
           =line 1
           =line 2
           =line 3
           =...
         </output>
       </handler>
       <end status="success">X+42</end>
     </stage>
     <stage name="install">
       <start>X+42</start>
       <end status="success">X+55</end>
       <handler name="make">
         <start>X+42</start>
         <end status="success">X+55</end>
         <output>
           =line 1
           =line 2
           =line 3
           =...
         </output>
       </handler>
     </stage>
     <start>X+30</start>
     <name>bash</name>
     <version>2.05b</version>
     <files>
       =path/to/file1
       =path/to/file2
       =path/to/file3
     </files>
   </package_run>
</package_log>

You'll note that the child nodes within each parent can appear in any 
order, and in fact the <package_run> nodes could appear in any order 
(i.e. a later run could appear earlier in the file). This means that all 
log file parsers (including nALFS itself) will need to parse the entire 
file and positively determine (via timestamps) which run/stage/etc. 
happened in what order. I don't see a way around this, other than using 
a non-XML format for the log files.

I see this format as being immensely useful; it means someone (not me 
:-) could be a log file parser that could display the logs at any level 
of detail desired, including displaying the output created by an 
individual step of the profile.

Comments? Questions? Is this too much, too little?




More information about the alfs-discuss mailing list