[RFC] nALFS package log files

James Robertson jwrober at linuxfromscratch.org
Tue Feb 10 20:44:41 PST 2004


Kevin P. Fleming wrote:

> James Robertson wrote:
> 
> Yow, the lid to Pandora's box blows off with the force of a thousand 
> suns never to be seen nor heard from again :-)

LOL, hey you asked :-)

>> 1./ Are you still looking to use the installwatch lib to get a list of 
>> files installed by each package or continue with the double-find 
>> method currently used?  What other options do we have?
> 
> The double-find method is already gone, that was one of the last things 
> that Neven worked on. I don't know the specific reasoning behind the 
> removal, but it's gone. Yes eventually we will have some type of 
> installwatch functionality.

IIRC, neven wanted to go with installwatch.  Probably got pulled away 
before he could finish it.  Is this going in before or after 1.3?

>> 2./ Will nALFS provide a function for someone to parse the logs to see 
>> what packages they installed or have been installed by nALFS? 
>> Essentially a reporting module.  I see the reporting module as a major 
>> value-add feature if we do it.  A person can see what has been 
>> installed, find out what package a file is part of with a query, how 
>> long a package takes to install, how long all the packages took to 
>> install, etc.  I know I would use the heck out of it!
> 
> There is already a "P" command inside nALFS that parses the package 
> logs, that will be retained, as will its (very basic) uninstall 
> functionality. 

I have not looked at the P option in depth yet.  Sorry about that.  Is 
uninstall going to be extended?  I would think the logs would come in 
real handy here as well as true dependency handling.

> The reporting module, while something I would be happy to 
> include in the nALFS tarball, will need to be created by someone else. 
> My hope is that creating a well-documented XML log file format will make 
> that easy to do in any standard scripting language (both Perl and Python 
> can handle XML quite well). The results you are referring to are exactly 
> why I want to do this, it can even be extended to some type of 
> comparison tool that compares two batches of logs and produces 
> time-consumed reports or something even fancier.

OK, this makes sense, but would require a new installation dependency 
(perl with the xml mod and/or python).  Didn't think that this is 
something we would want to add.  How hard is it to do in C?  I suck at 
C, but hey you have got to learn somewhere! (not volunteering, got 
enough on my plate at the moment.

>> 3./ Are we going to put these logs in a perm location on the file 
>> system - for example /var/log/nALFS/ or something to that effect?
> 
> They already are put into a packages directory underneath whatever you 
> have specified as your alfs_directory. That will not change.

duh, knew that.

>> 4./ What if you re-install a package via nALFS, will it overwrite the 
>> old log file or append to show an admin that is has been installed 
>> more than once and when?  If it is, can we have nALFS ask to put a 
>> comment in as to why it was re-installed? This can help a lot with 
>> change management, have the log contain a lot of that info.  How about 
>> a <comment> tag period?
> 
> The log files will never be overwritten; additional runs of a package's 
> profile (same package name) will create additional package_run elements 
> in the log file, with all the appropriate content.

Cool.

>> 5./ Will the log DTD be a member of the main DTD?  Does it need 
>> documentation (rhetorical q)?
> 
> Yes I will create a DTD for it, it will not be part of the main DTD as 
> it will never be used for the same files. It will very likely need some 
> documentation, although I can't imagine it being extensive.

True, I would see a syntax type doc like we have for the main DTD. 
Since the log DTD will be so small, it would take no time at all to get 
into docbook.

>> 6./ Will the <output> and </output> tags contain both stdout and 
>> stderr in the same place or can we split it out to <output 
>> fd="stderr"> and <output fd="stdout"> or something?  Probably would 
>> help #2 - log parser.
> 
> For now (and probably forever) the <output> element will contain all 
> output generated by the handlers. I have plans to change the nALFS 
> backend to use a pty to grab the system output, which removes the 
> possibility of even knowing what was stdout vs. stderr (but improves 
> other things).

What other things?  Would the log parser have anyway of determining what 
in the log came from stderr as opposed to stdout?  Sure would make 
finding stuff easier.  What are all the pros - cons on this one?

>> 7./ Why not just <package> as opposed to <package_run>?  Seems easier 
>> to me.
> 
> Because there can be more than one <package_run> for a package, see the 
> answer to #4.

Not following you here.  You have:

<package_run>
	<name>foo</name>
	<version>95</version>
......
</package_run>

and repeat

I was asking for

<package>
	<name>foo</name>
	<version>95</version>
......
</package>

and repeat

We can't do that?  I must be missing something.

>> 8./ Why not <command_executed> as opposed to <handler>?  I would like 
>> to know what command was passed to the system.  On second thought, I 
>> see where <handler> can be useful, but would still like to know the 
>> command sent to the shell.
> 
> I will probably extend the <handler> element to log what the handler 
> actually did, but not not until after the XML parser is rewritten, which 
> won't happen before 1.3 is released.

OK.  What all will the new XML parser do for us?

>> 9./ Is <stage> a regurtigation of what is in the profile for <stage 
>> name="">?
> 
> Yes.
> 
>> 10./ Do we want to log the md5 and tarball used?
> 
> (See #8) When the handlers can log more details, sure the <unpack> 
> element could do that.

Kewl

>> 11./ If we log the source location, how about destination 
>> (--prefix=/foo/bar)?
> 
> That would be difficult, as none of the elements currently have any way 
> to know that (--prefix is just a generic <param> supplied to 
> <configure>). It can be deduced from the installed files list, which is 
> probably enough, and if the handler echos the configure command into the 
> log file it will be there too.

How about the logger puts a copy of the full command into the log, then 
we can see what <param> as passed to the configure stage with a simple grep.

>> 12./ I am confused on where <files> comes in.  I would think that it 
>> would be the last set of tags for a <package_run></package_run>.  
>> Every "kit" should include it, even if no files where written. The 
>> tags would contain a "No files installed" or something message.  Would 
>> it handle changed/modified files?  How do we differentiate between 
>> "installed" or "changed"?  I would want to know this.  What package 
>> and/or stage "owns" a file, but also know what package and/or stages 
>> modify an existing file
> 
> <files> will not appear if installed-file tracking is disabled 
> completely. Otherwise it will be there, possibly empty. The extension of 
> this list for showing installed vs. changed will have to wait until 
> installwatch or something similar is in place. The syntax I proposed may 
> have to be changed though, to not put the raw paths inside <files> but 
> instead add a <file> element for each one that could eventually hold 
> attributes that signify what happened to it.

OK, sounds good.  Clearly after 1.3, which means we will want to version 
the log DTD.

>> 13./ If we end or have a <end status="error"> where is the error 
>> message placed?
> 
> In the <output> element, unless you've disabled it. If you have, you get 
> what you asked for. Actually, that's not quite true, any messages 
> generated by the handlers themselves will always appear in the <handler> 
> element somewhere, I just haven't decided upon a syntax for them yet.
> 
I think there needs to be a differentiation between what the handler 
outputs or the app outputs internally and what stderr and stdout puts 
out from the status window.  This all comes from what level of logging 
you pick in the rc file.

James

-- 
James Robertson -- jwrober at linuxfromscratch dot org
Reg. Linux User -- #160424 -- http://counter.li.org
Reg. LFS User   -- #6981   -- http://www.linuxfromscratch.org
LFS Bugzilla Maintainer    -- http://{blfs-}bugs.linuxfromscratch.org



More information about the alfs-discuss mailing list