Dejagnu 1.4.4 testsuite failures

Brandon Peirce brandon_peirce at hotmail.com
Wed Aug 23 19:58:51 PDT 2006


Dan Nicholson wrote:

>On 8/22/06, Brandon Peirce <brandon_peirce at hotmail.com> wrote:
>>
>>The only thing I could find in Google is
>>http://savannah.gnu.org/bugs/?func=detailitem&item_id=17176
>>which describes my problem almost exactly but without further info.
>
>Impressive. If your failures are like that, and even the --version
>test is failing, then something very basic must be wrong.

That's for sure!  But it's in the test setup, not with what's installed.


>I think there should
>be *.log files in the testsuite/ directory. Could you have a look
>around for those? The output from Make isn't that helpful.

The *.log and *.sum files are even less helpful than the make
output. I think the testsuite creates logs in a tmpdir then cp's
them back to the testsuite dir, overwriting previous results!

BTW, a great tip for anyone trying to trace an automake/dejagnu
testsuite is the RUNTESTFLAGS variable, as in:
make -k RUNTESTFLAGS=--verbose check

It adds a bit of science to what is otherwise more like a lottery!
Dejagnu must be the worste packages I have ever seen when it
comes to scanning your filesystem and implicitly sourcing scripts
from all kinds of seemingly random locations!

And when you've got runtest invoked from 4 levels of nested
makes trying to test $RUNTEST which is a late-evaluating make
macro that is resolves to a shell expression involving a couple of
if's.... it starts to get a bit hairy to keep track of it all. Well then,
halfway through the testsuite, there is a change of strategy and
the delicately constructed $RUNTEST is ignored completely in
favour of some util_start proc loaded from a library script, and
this util_start proc locates runtest with a which command. Now
which 'which' would that be, I ask you? Is it a Dejagnu built-in,
an Expect built-in, a Tcl built-in or is it /usr/bin/which?
Aaaaaaaargh!!! Why do I do this???

I've got it all to work most of the time now on my host system and
in one of my chroot environments, and the barebones tmpfs
chroot is almost there. But I'm still getting the 1st four tests--the
most basic--failing on a seemingly-random basis?
---
Running [...]/testsuite/libdejagnu/tunit.exp ...
FAIL: Pass message
FAIL: Fail message
FAIL: Untested message
FAIL: Unresolved message
---

I can run a full get-src, configure, check cycle in adjacent
directories, 9 times back-to-back in a for loop without a single
failure. Then I run the same for loop again immediately after and
maybe on the 5th iteration, those four tests will fail. The only thing
I can say is that if I run two or three concurrent backgrounded
instances of the same get ... check cycle, they're almost certain
to fail. There is no apparrent relation to system load either.
Can someone explain that?

I think I'm going to adjust my scripts to run the testsuite 9 times
and consider it successful if everything passes at least 5 times!


>>Also, would it be possible to post some other logs under
>>http://www.linuxfromscratch.org/lfs/build-logs/6.2/ ?
>
>I can put my Ch. 6 tests in there, but no Ch. 5. Sorry. Archaic might 
>populate these later when he gets his build done.

Thanks, I see them there.

Brandon.





More information about the lfs-support mailing list