Garrett, Thank for the reply. A couple of follow-up questions:
If the tests are not written to the LTP standards why are they accepted into he LTP? At least they should not be included in the default set of tests that get run. Is there any formal acceptance testing before a new/changed test is accepted into the LTP? Robert Paulsen IBM Linux Technology Center, Austin [email protected] 512-468-0565 Garrett Cooper <[email protected]> wrote on 10/20/2009 04:01:58 PM: > Garrett Cooper <[email protected]> > 10/20/2009 04:01 PM > > To > > Robert Paulsen/Austin/i...@ibmus > > cc > > [email protected] > > Subject > > Re: [LTP] Fw: LTP vs. SLES11 -- Some FAILs. > > On Tue, Oct 20, 2009 at 10:21 AM, Robert Paulsen <[email protected]> wrote: > > > > Hello, > > > > I ran the latest LTP (20090930) against SLES11: > > > > # cat /etc/issue > > Welcome to SUSE Linux Enterprise Server 11 (ppc64) - Kernel \r (\l). > > # uname -a > > Linux beavis 2.6.27.19-5-ppc64 #1 SMP 2009-02-28 04:40:21 +0100 > ppc64 ppc64 ppc64 GNU/Linux > > > > These were all run via "runalltests" so are part of the default set of LTP > > tests that run. > > > > There were a number of errors. I didn't expect this as I thought that one > > way or another the LTP would have been validated against a standard distro > > like SLES11. > > > > Are these known failures? In the LTP or SLES11? > > > > Some of these tests apparently rely on kernel configurations and/or > > available resources like real memory that cannot be guaranteed on any > > arbitrary installation. Should such tests be run by default? > > > > Some of these tests just look like sloppy tests, for example the > > cron_allow01 an cron_deny01 tests. > > > > Some tests assume they will be built on the same system on which they are > > run. In his case that assumption was true but it seems like a very unsafe > > assumption, especially if a cross build is needed. Even without a cross > > build the build and test systems may not always be the same -- the reason > > for "make package" as describe in README.ltp-devel and why the Makefile has > > a CROSS_COMPILE macro. > > > > Here are the fails from the log file: > > > > eventfd2_03 FAIL 1 > > inotify01 FAIL 1 > > inotify02 FAIL 1 > > ppoll01 FAIL 2 > > quotactl01 FAIL 1 > > timer_getoverrun01 FAIL 1 > > timer_gettime01 FAIL 1 > > utimes01 FAIL 1 > > Cap_bounds FAIL 127 > > su01 FAIL 1 > > cron02 FAIL 1 > > cron_deny01 FAIL 1 > > cron_allow01 FAIL 1 > > at_deny01 FAIL 1 > > at_allow01 FAIL 1 > > timer_delete02 FAIL 1 > > Numa-testcases FAIL 1 > > hugemmap03 FAIL 1 > > hugemmap04 FAIL 1 > > hugeshmat01 FAIL 6 > > hugeshmat02 FAIL 6 > > hugeshmat03 FAIL 2 > > hugeshmctl01 FAIL 6 > > hugeshmctl02 FAIL 6 > > hugeshmctl03 FAIL 2 > > hugeshmdt01 FAIL 6 > > hugeshmget01 FAIL 2 > > hugeshmget02 FAIL 6 > > hugeshmget03 FAIL 6 > > hugeshmget05 FAIL 2 > > > > A number of tests FAILed despite not showing in the above list. > Here is some info from the detailed output file: > > > > pid_namespace4 1 TFAIL : Container init is killed by SIGKILL !!! > > pid_namespace4 2 TFAIL : Container init pid got killed by signal 9 > > pidns12 1 TFAIL : cinit: signalling PID (from other > namespace) is not 0, but 27810. > > quotactl01 1 TFAIL : quotactl01 failed, cmd=0x800002: > TEST_ERRNO=EFAULT(14): Bad address > > quotactl01 2 TFAIL : quotactl01 failed, cmd=0x800003: > TEST_ERRNO=EFAULT(14): Bad address > > quotactl01 3 TFAIL : quotactl01 failed, cmd=0x800007: > TEST_ERRNO=EFAULT(14): Bad address > > quotactl01 4 TFAIL : quotactl01 failed, cmd=0x800008: > TEST_ERRNO=EFAULT(14): Bad address > > quotactl01 5 TFAIL : quotactl01 failed, cmd=0x800005: > TEST_ERRNO=EFAULT(14): Bad address > > quotactl01 6 TFAIL : quotactl01 failed, cmd=0x800006: > TEST_ERRNO=EFAULT(14): Bad address > > quotactl01 7 TFAIL : quotactl01 failed, cmd=0x800004: > TEST_ERRNO=EFAULT(14): Bad address > > quotactl01 8 TFAIL : quotactl01 failed, cmd=0x800001: > TEST_ERRNO=EFAULT(14): Bad address > > Hi Robert, > I'm not sure about the snapshot you're referring to, but the > latest version no longer uses CROSS_COMPILE. You instead must specify > CC, CXX, LD, etc either via configure or manually in config.mk, and > then move it over to $(abs_top_builddir)/include/mk/config.mk; the > preferred method is the former one. > Some points: > 1. quotactl01, etc might have issues because they're half-baked > from the point of view that they were functionally moved over to > tst_res.c's output format, but instead don't call tst_exit(3) at the > end (which is required if you want a sensible exit code). These items > need to be found and squashed in the tree. > 2. genhtml.pl was broken because it's simple parsing method wasn't > updated to match the fact that it's now TPASS, TFAIL instead of PASS, > FAIL, etc. I fixed it a week ago. > That aside, part of the reason I've written up execltp is to parse > the exec logs (as opposed to the output log because the output log is > all over the place) and find problems using the exec logs depending on > the tst_res format to provide all of the details -- I've found that > many test writers and porters to LTP have been inconsistent when > porting tests, and thus issues with results are harder to find than > they should be. > Thanks, > -Garrett ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ Ltp-list mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ltp-list
