"HLFS Trac #1645: gcc/binutils test suite"
robert at linuxfromscratch.org
Wed Nov 28 17:01:21 PST 2007
On Wednesday November 28 2007 07:36:08 am Aki Tuomi wrote:
> (sorry if this comes in double...)
> The reason test suites are usually made is to ensure that the
> software in question operates as specified. It is not a matter of
> paranoia if a test suite failes, but a indication that there is a
> problem somewhere.
Binutils and GCC's test suites typically build test programs and verify their
program headers, what they contain, etc, and compare to known results. The
ssp and fpie options alters this, and the tests fail because the comparison
is not a perfect match.
We could either patch the test suites to expect ssp and fpie, like how the
binutils-pt_pax patch modifies the test suite to expect a new program header,
or disable ssp and fpie for the tests. I don't see an advantage to patching
the tests for ssp or fpie... we just want to know if the compiler is behaving
correctly... there are specific tests for ssp and fpie. All the test suites
should work as they were intended to.
> I would rather seek out at some schedule to fix the problem than just
> ignore it with "well, look, these few software I compiled worked, so I
> suppose nothing is wrong".
> Please do not take the stance that "tests are only advisory" for this
> project, the distribution is a very critical part of functional system.
> Tests, at least for the build path, are in my opinion quite critical to
I agree, and I'd also like to mention that these test suites test known usage
and options, not unknown usage. For example, if a program's code is written
to have a 128 character limit on command options, I would like to know what
happens if I use more. This is not something test suites will typically
check. Developing tests like this is time consuming, but there are other
packages which help, like code and runtime checking programs.
I've wanted to add things like this in special page sections for auditors
(people who want to spend the time on this). It would help raise awareness,
and would benefit everyone. It would also give enough examples and details so
that packages which are not in the book can be examined or audited.
> A good example is the JAVA package on (B/H)LFS side. With some effort, I
> was able to make it compile without INSANE mode, unlike suggested in the
> build manual. This means that the package can be expected to work,
> unlike if it was made with INSANE.
> Just my 0.02c rant...
> Aki Tuomi
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 189 bytes
Desc: not available
More information about the hlfs-dev