On Thu, 2009-03-12 at 05:10 +0800, CAI Qian wrote: > Hi, > > From: Subrata Modak <[email protected]> > Subject: Re: [LTP] [PATCH] Disable Broken Test Cases > Date: Mon, 9 Mar 2009 15:44:45 +0530 > > > Hi, > > > > On Sun, Mar 8, 2009 at 9:59 PM, CAI Qian <[email protected]> wrote: > > > >> Hi, > >> > >> This patch temporarily disables broken test cases, > >> > >> nanosleep01 > >> nanosleep02 > >> pselect01 > >> pselect01_64 > >> > >> because the ways to calculate the syscall execution time looks like > >> invalid. For example, in pselect01 test case, > >> > >> start = time(&t); > >> retval = pselect(0, &readfds, NULL, NULL, (struct timespec *)&tv,NULL); > >> end = time(&t); > >> > >> /* Changed total_sec compare to an at least vs an exact compare */ > >> > >> if(((end - start) >= total_sec) && ((end - start) <= total_sec + 1)) > >> tst_resm(TPASS,"Sleep time was correct"); > >> else > >> tst_resm(TFAIL,"Sleep time was incorrect:%d != %d",total_sec,(end - > >> start)); > >> } > >> > >> The CPU scheduler can suspend the execution of the program just after > >> time(), but before pselect(). As the result, (end - start) is not the
Yes, that can definitely happen. And even a suspend can happen after pselect() and before then next call to time(). > >> actual syscall execution time, which introduces a false test failure > >> here. The same problem exists in nanosleep01 and nanosleep02 as well, Then, what is the best way to determine the exact time taken by a syscall to execute ?? A part of this fix was introduced by Craig Meier. Noting that even there might have been 100 interruptions between the first and last time() call, still (end - begin) is within (actual time to execute the syscall + 1). If this time exceeds, means something notably wrong with the scheduler to schedule this process taking notably context switching. Regards-- Subrata > >> > >> /* Note down the current time */ > >> gettimeofday(&otime, 0); > >> /* > >> * Call nanosleep() to suspend child process > >> * for specified time. > >> */ > >> TEST(nanosleep(&timereq, NULL)); > >> > >> /* time after child resumes execution */ > >> gettimeofday(&ntime, 0); > >> > >> > >> Signed-off-by: CAI Qian <[email protected]> > > > > > > I am not sure whether it would be correct to disable tests when they are > > broken (supposed to be). The inherent reason being that once a/some > > test/tests are disabled, they just go out of the perview of the testers > > memory, as, he/she can no more see the test results of the same tests. > > Already overburdened to handle/analyze so many tests results, he simply > > forgets that such-and-such tests - disabled some time ago due to broken > > issue(s) - needs to be fixed. > > > > So, i would prefer to keep them, so that it occasionally pricks into > > somebody(s) eye(s), and he/she is generous enough to give a fix someday in > > future. > > > > Sorry, I can't agree with you. Those tests are broken by design. It > wastes everybody's time by using broken tests, and then need to figure > it out why failed. Why do we release broken tests. I understand your > point by not disable them, so hopefully people may think of fixing it > some day. However, you can achieve the same by put those tests to the > list of known broken tests or TODO list somewhere obvious like in top > directory or homepage. I am afraid I don't know how to fix those test > cases at the moment. > > Considering a little bit further, I guess stable/unstable branches might > be a way to go, so we can add new or working-in-progress tests to > unstable branches until they have enough time to be mature. We are > maintaining LTP inside Red Hat as well at the moment. There are some > patches like those to disable some working-in-progress tests, so it will > give us more reliable testing results, and then we can actually use it > for regression testing of kernels. I am looking for what is the best to > merge those patches upstream, so we can avoid further gaps between two > repositories. > > CAI Qian > > > Regards-- > > Subrata > > > > > >> > >> --- runtest/syscalls.orig 2009-03-09 00:04:17.000000000 +0800 > >> +++ runtest/syscalls 2009-03-09 00:15:49.000000000 +0800 > >> @@ -622,8 +622,10 @@ > >> munmap02 munmap02 > >> munmap03 munmap03 > >> > >> -nanosleep01 nanosleep01 > >> -nanosleep02 nanosleep02 > >> +# The following test cases are broken by design with the syscall time > >> +# calculations. > >> +#nanosleep01 nanosleep01 > >> +#nanosleep02 nanosleep02 > >> nanosleep03 nanosleep03 > >> nanosleep04 nanosleep04 > >> > >> @@ -699,8 +701,10 @@ > >> > >> profil01 profil01 > >> > >> -pselect01 pselect01 > >> -pselect01_64 pselect01_64 > >> +# The following test cases are broken by design with the syscall time > >> +# calculations. > >> +#pselect01 pselect01 > >> +#pselect01_64 pselect01_64 > >> > >> ptrace01 ptrace01 > >> ptrace02 ptrace02 > >> > >> > >> ------------------------------------------------------------------------------ > >> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, > >> CA > >> -OSBC tackles the biggest issue in open source: Open Sourcing the > >> Enterprise > >> -Strategies to boost innovation and cut costs with open source > >> participation > >> -Receive a $600 discount off the registration fee with the source code: > >> SFAD > >> http://p.sf.net/sfu/XcvMzF8H > >> _______________________________________________ > >> Ltp-list mailing list > >> [email protected] > >> https://lists.sourceforge.net/lists/listinfo/ltp-list > >> > > > > > > > > -- > > Regards & Thanks-- > > Subrata > > ------------------------------------------------------------------------------ > Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are > powering Web 2.0 with engaging, cross-platform capabilities. Quickly and > easily build your RIAs with Flex Builder, the Eclipse(TM)based development > software that enables intelligent coding and step-through debugging. > Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com > _______________________________________________ > Ltp-list mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/ltp-list ------------------------------------------------------------------------------ Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com _______________________________________________ Ltp-list mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ltp-list
