op/rand.t from perl test suite

Ruslan Shevelyov rdsh at alpha.ru
Thu Jul 19 12:50:23 PDT 2007


Good evening (or afternoon, or morning, or night).

It seems that some tests from perl test suite occasionally fail even on
correct builds; since expected/acceptable failures in other packages'
test suites (GCC, Glibc) are described  in the book, perhaps this should
be mentioned too. Details are as follows:

I'm building LFS 6.2 on a Cyrix M-II 233 (PR 300) using LiveCD as a host
system. Reached chapter 6.22 (Perl-5.8.8), unpacked, created /etc/hosts,
ran configure.gnu, which produced the following warning:

> root:/wrk/perl-5.8.8# ./configure.gnu --prefix=/usr \
>      -Dman1dir=/usr/share/man/man1 \
>      -Dman3dir=/usr/share/man/man3 \
>      -Dpager="/usr/bin/less -isR"
> [...skip...]
> Directory for the main Perl5 html pages? (~name ok) [none]
> Directory for the Perl5 module html pages? (~name ok) [none]
> Can't locate File/Glob.pm in @INC (@INC contains: 
> /tools/lib/perl5/5.8.8/i686-linux /tools/lib/perl5/5.8.8
> /tools/lib/perl5/site_perl/5.8.8/i686-linux
> /tools/lib/perl5/site_perl/5.8.8
> /tools/lib/perl5/site_perl .) at getverlist line 12.
> BEGIN failed--compilation aborted at getverlist line 12.
> List of earlier versions to include in @INC? [none]
> Checking for GNU C Library...

make completed as expected, and I ran make test:

> root:/wrk/perl-5.8.8# make test
>         AutoSplitting perl library
> [...skip...]
> t/op/qq...................................ok
> t/op/quotemeta............................ok
> t/op/rand.................................
> # Failed at op/rand.t line 181
> FAILED at test 5
> t/op/range................................ok
> t/op/read.................................ok
> [...skip...]
> t/pod/special_seqs........................ok
> t/x2p/s2p.................................ok
> Failed 1 test script out of 933, 99.89% okay.
> ### Since not all tests were successful, you may want to run some of
> ### them individually and examine any diagnostic messages
> they produce.
> ### See the INSTALL document's section on "make test".
> ### You have a good chance to get more information by running
> ###   ./perl harness
> ### in the 't' directory since most (>=80%) of the tests succeeded.

Checked INSTALL's section on "make test":

> If make test bombs out, just cd to the t directory and run ./TEST
> by hand to see if it makes any difference.  If individual tests
> bomb, you can run them by hand, e.g.,
> 
>         ./perl op/groups.t
> 
> Another way to get more detailed information about failed tests and
> individual subtests is to cd to the t directory and run
> 
>         ./perl harness
> 

So,

> root:/wrk/perl-5.8.8/t# ./perl harness
> base/cond...................................ok
> base/if.....................................ok
> [...skip...]
> op/qq.......................................ok
> op/quotemeta................................ok
> op/rand.....................................ok
> op/range....................................ok
> op/read.....................................ok
> [...skip...]
> pod/special_seqs............................ok
> x2p/s2p.....................................ok
> All tests successful, 59 tests and 259 subtests skipped.
> Files=995, Tests=117287, 2103 wallclock secs
> (1590.76 cusr + 200.26 csys = 1791.02 CPU)

> root:/wrk/perl-5.8.8/t#  ./perl op/rand.t
> 1..8
> ok 1
> ok 2
> ok 3
> ok 4
> ok 5
> # Your rand seems fine. If this test failed
> # previously, you may want to run it again.
> #
> # If you are having random number troubles,
> # see the hints within the test script for more
> # information on why this might fail. [ 1.888 ]
> ok 6
> ok 7 - rand() without args is rand(1)
> ok 8 - rand() without args is under 1

> root:/wrk/perl-5.8.8/t# cat op/rand.t
> #!./perl
> 
> # From Tom Phoenix <rootbeer at teleport.com> 22 Feb 1997
> # Based upon a test script by kgb at ast.cam.ac.uk (Karl Glazebrook)
> 
> # Looking for the hints? You're in the right place.
> # The hints are near each test, so search for "TEST #", where
> # the pound sign is replaced by the number of the test.

Searched for "TEST 5" and found nothing. It seems that test 5 is:

> 
>     #   NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE
>     # This test will fail .1% of the time on a normal system.
>     #                           also
>     # This test asks you to see these hints 100% of the time!
>     #   NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE NOTE
>     #
>     # There is probably no reason to be alarmed that
>     # something is wrong with your rand function. But,
>     # if you're curious or if you can't help being
>     # alarmed, keep reading.
>     #
>     # This is a less-crude test than test 3. But it has
>     # the same basic flaw: Unusually distributed random
>     # values should occasionally appear in every good
>     # random number sequence. (If you flip a fair coin
>     # twenty times every day, you'll see it land all
>     # heads about one time in a million days, on the
>     # average. That might alarm you if you saw it happen
>     # on the first day!)
>     #
>     # So, if this test failed on you once, run it a dozen
>     # times. If it keeps failing, it's likely that your
>     # rand is bogus. If it keeps passing, it's likely
>     # that the one failure was bogus. If it's a mix,
>     # read on to see about how to interpret the tests.
>     #
>     # The number printed in square brackets is the
>     # standard deviation, a statistical measure
>     # of how unusual rand's behavior seemed. It should
>     # fall in these ranges with these *approximate*
>     # probabilities:
>     #
>     #           under 1         68.26% of the time
>     #           1-2             27.18% of the time
>     #           2-3              4.30% of the time
>     #           over 3           0.26% of the time
>     #
>     # If the numbers you see are not scattered approximately
>     # (not exactly!) like that table, check with your vendor
>     # to find out what's wrong with your rand. Or with this
>     # algorithm. :-)
>     #
>     # Calculating absoulute standard deviation for number of bits set
>     # (eight bits per rep)
>     $dev = abs ($bits - $reps * 4) / sqrt($reps * 2);
> 
>     ok( $dev < 3.3 );
> 
>     if ($dev < 1.96) {
>         print "# Your rand seems fine. If this test failed\n";
>         print "# previously, you may want to run it again.\n";
>     } elsif ($dev < 2.575) {
>         print "# This is ok, but suspicious. But it will happen\n";
>         print "# one time out of 25, more or less.\n";
>         print "# You should run this test again to be sure.\n";
>     } elsif ($dev < 3.3) {
>         print "# This is very suspicious. It will happen only\n";
>         print "# about one time out of 100, more or less.\n";
>         print "# You should run this test again to be sure.\n";
>     } elsif ($dev < 3.9) {
>         print "# This is VERY suspicious. It will happen only\n";
>         print "# about one time out of 1000, more or less.\n";
>         print "# You should run this test again to be sure.\n";
>     } else {
>         print "# This is VERY VERY suspicious.\n";
>         print "# Your rand seems to be bogus.\n";
>     }




More information about the lfs-support mailing list