"HLFS Trac #1645: gcc/binutils test suite"

Robert Connolly robert at linuxfromscratch.org
Wed Nov 28 17:01:21 PST 2007


On Wednesday November 28 2007 07:36:08 am Aki Tuomi wrote:
>
> (sorry if this comes in double...)
>
> The reason test suites are usually  made is to ensure that the
> software in question operates as specified. It is not a matter of
> paranoia if a test suite failes, but a indication that there is a
> problem somewhere.

Binutils and GCC's test suites typically build test programs and verify their 
program headers, what they contain, etc, and compare to known results. The 
ssp and fpie options alters this, and the tests fail because the comparison 
is not a perfect match.

We could either patch the test suites to expect ssp and fpie, like how the 
binutils-pt_pax patch modifies the test suite to expect a new program header, 
or disable ssp and fpie for the tests. I don't see an advantage to patching 
the tests for ssp or fpie... we just want to know if the compiler is behaving 
correctly... there are specific tests for ssp and fpie. All the test suites 
should work as they were intended to.

> I would rather seek out at some schedule to fix the problem than just
> ignore it with "well, look, these few software I compiled worked, so I
> suppose nothing is wrong".
>
> Please do not take the stance that "tests are only advisory" for this
> project, the distribution is a very critical part of functional system.
> Tests, at least for the build path, are in my opinion quite critical to
> work.

I agree, and I'd also like to mention that these test suites test known usage 
and options, not unknown usage. For example, if a program's code is written 
to have a 128 character limit on command options, I would like to know what 
happens if I use more. This is not something test suites will typically 
check. Developing tests like this is time consuming, but there are other 
packages which help, like code and runtime checking programs.

I've wanted to add things like this in special page sections for auditors 
(people who want to spend the time on this). It would help raise awareness, 
and would benefit everyone. It would also give enough examples and details so 
that packages which are not in the book can be examined or audited.

robert

> A good example is the JAVA package on (B/H)LFS side. With some effort, I
> was able to make it compile without INSANE mode, unlike suggested in the
> build manual. This means that the package can be expected to work,
> unlike if it was made with INSANE.
>
> Just my 0.02c rant...
>
> Aki Tuomi


-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.linuxfromscratch.org/pipermail/hlfs-dev/attachments/20071128/5591b2f6/attachment.sig>


More information about the hlfs-dev mailing list