Hi, --- Subrata Modak <[EMAIL PROTECTED]> wrote:
> Hi Cai & Daniel, > > On Thu, 2008-10-23 at 12:01 -0700, CAI Qian wrote: > > Hi, > > > > --- Daniel Gollub <[EMAIL PROTECTED]> wrote: > > > > > On Thursday 23 October 2008 19:47:56 CAI Qian wrote: > > > > Hi, > > > > > > > > Ptrace06 has lots of issues. It fails to build on IA-64, unable to > > > > compile on earlier > Kernels > > > > (more details on those two issues could be found on previous threads) > > > > and there is new > issue I > > > > have just spotted. > > > > > > > > In spawn_ptrace_child.h, > > > > > > > [...] > > > > > > > > > > > The parent returns from the function in which vfork() was called. As > > > > the results, the test > > > result > > > > is random on x86-64, > > > > > > > > ptrace06 1 PASS : ptrace(PEEKDATA, ..., (nil), (nil)) failed as > > > > expected > > > > > > > > or > > > > > > > > ptrace06 1 FAIL: ptrace(PEEKDATA, ..., (nil), (nil)) expected > > > > errno EIO or EFAULT; > > > actual: No > > > > such process > > > > > > > > or > > > > > > > > ptrace06 1 FAIL: ptrace(PEEKDATA, ..., (nil), (nil)) returned 0 > > > > instead of -1 > > > > > > > > According vfork() manpage, the code is does not look right. > > > > > > > > "The vfork() function has the same effect as fork(2), except that the > > > > behavior is > undefined if > > > > > > > the process created by vfork() either modifies any data other > > > > than a variable of > type > > > pid_t > > > > used to store the return value from vfork(), or returns from the > > > > function in which vfork() > was > > > > called," > > > > > > > > Also, it leaves lots of processes after the test completes. > > > > > > > > qcai 4503 0.0 0.0 4092 408 pts/8 S 01:22 0:00 > > > > ./ptrace06 child > > > > qcai 4541 0.0 0.0 4092 408 pts/8 S 01:23 0:00 > > > > ./ptrace06 child > > > > qcai 4584 0.0 0.0 4092 408 pts/8 S 01:23 0:00 > > > > ./ptrace06 child > > > > qcai 4606 0.0 0.0 4092 404 pts/8 S 01:23 0:00 > > > > ./ptrace06 child > > > > > > I have seen something similar in the past with times03 (iirc) which was > > > about > > > about a compiler optimization which could tight assembler infinite loop > > > and > > > interfered with further running tests -> bad impact on testresults. > > > (This got already fixed in beginning of the year...) > > > > > > Is there already some code in runltp to catch such processes and abort the > > > entire testrun? Test processes leaking out of the scope of the testdriver > > > is a pretty serious thing - isn't it? We should definitely ban such broken > > > tests and prepare runltp to detect such processes? In my opinion aborting > > > the testrun immediately is better to have wrong/invalid testresults... > > > > > > > Agree. We need to make sure we tidy up things from the previous test before > > to run the next > one. > > > > > > > > > The test has been disabled in Makefile at the moment, but I would like > > > > to discuss how we > could > > > > handle the similar case in the future. In this case, it is unnecessary > > > > to remove it from > CVS > > > > entirely, but it is kind of untidy and confusion to users to leave in > > > > the test directory, > but > > > > never been compiled. > > > > > > Fully agree. > > > > > > I'm pretty new to LTP development, but what about the > > > open-posix-testsuite? > > > It's also disable by default - looking into open-posix-testsuite show also > > > definitely some issues ... > > > > > > (And those "grep -v" are really not very obvious - i have to admin i > > > didn't > > > actively checked the commit logs or documentation - maybe i'm missing > > > some > > > obvious documentation about this...) > > > > > > > I don't know either. > > > > > > How about to create a separate directory > > > > (testcases/working_in_progress?) to > > > > move all those broken test code there, and each of them has an > > > > associate file to document > why > > > it > > > > is here and what need to be done to move it to the test directory it > > > > should belong? > > > > > > Good idea! > > Yes, this can be done. We can have something like > testcases/broken_cases/*. Move the very much troubling guys like the > present ptrac06 to here. And once it is fixed, it can be moved back. > Having said that, we have to do the following things always: > > 1) First, we need to decide whether somebody is really troubling. In > other words, how do we decide the degree of trouble. A test may have a > little problem on only an exclusive arch, but may be perfectly OK in > others. In such a case, it is not a good candidate to move. However, i > am hopeful that we can agree to decide on this, > How about based on if it supports all architectures and Kernel versions it claimed to? If not, and nobody is interested or unable to fix it in a reasonable amount of time, we then would move it to the black list. Moreover, we could always discuss here if anybody disagrees. > 2) Both the to-and-fro movement needs to be addressed through > appropriate patches: like disabling build/install in Makefile and > diabling run in runtest/* file(s). Similarly enabling patches in these > places when they are restored, > Agree. > > > > > > How to handle testcases which build fine on all architectures and all > > > kernel versions - but can't trigger a certain error-path for a kernel > > > interface for a specific kernel version? Or not on specific Architectures? > > > Or combined: build on all kernels, but is not testable on 2.6.27 on > > > x86_64? > > > > > > Example: testcases/open_posix_testsuite/conformance/interfaces/mmap/31-1.c > > > I'm looking into this one right now - i guess it's not possible to trigger > > > EOVERFLOW on x86_64 with current (or even any?) kernel on x86_64. > > > > > > > That is something missing in LTP. In my opinion, each test or set of tests > > need to indicate > which > > architectures and Kernel versions it supports, and make sure only compile > > and run on those > > supported systems. So, as long as it is written somewhere, people could > > just contribute a test > to > > support only one architecture or Kernel version that she/he cares about > > without worrying about > > having to provide support on all architectures and Kernel versions that LTP > > aims to. > > We already have this trend in LTP. The READMEs say on which archs they > can run. The Makefile(s) are written to trigger build in such archs > only. And when you run them, they say they are not supported arch. That > should be a good referral. Isn´t it ? > > Having said that, i fully agree that such information, like; on which > archs/kernel versions, headers/glibc version, headers they should run; > is missing from most of the test cases except the most recent ones. So, > we can have a strategy from now onwards: > > 1) Whoever submits a patch on certain set of test case(s), updates that > Information as well, as, we can suppose that he has worked on that test > case, on his interested archs, and, has the relevant information. People > working/interested on other archs can update when they work on the same > case in future, > 2) Any new test case(s) coming in should have this mandatory information > in them. So, what all information you would like to have been documented > there? > - Architectures supported, > - Kernel Versions supported/disbaled thereafter, > - glibc version, > - glibc headers, > - kernel headers, > - fs type, > - etc, > We need something more enforceable than documentation, like Makefile, so if should not even compile by default if the system under test is not supported by this test. Obviously, we could have both documentation and Makefile changes, but it is kind of pain to keep the consistence of those two places. I'd prefer to generate this document from Makefile. We could add a target like, make doc On the other hand, we should make it is easy to write such Makefile. Cai Qian > Add to the above list, and i will make it mandatory for others to > provide such information whenever they submit a patch. > > Regards-- > Subrata > > > > > Cai Qian > > > > > best regards, > > > Daniel > > > > > > > > > > > ------------------------------------------------------------------------- > > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > > Build the coolest Linux based applications with Moblin SDK & win great > > prizes > > Grand prize is a trip for two to an Open Source event anywhere in the world > > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > > _______________________________________________ > > Ltp-list mailing list > > [email protected] > > https://lists.sourceforge.net/lists/listinfo/ltp-list > > ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Ltp-list mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ltp-list
