Re: the Linux kernel, testsuites, and maybe *you*
On 9/4/07, Mike Frysinger <[EMAIL PROTECTED]> wrote: > On 9/2/07, Andi Kleen <[EMAIL PROTECTED]> wrote: > > On Sun, Sep 02, 2007 at 11:15:23AM -0400, Mike Frysinger wrote: > > > there is still the ABI issue ... code written in kernel space in pure > > > asm cannot always be compiled in userspace and work properly/the same > > > > Is that a blackfin weirdness? > > yes, Blackfin is weird in this respect due to being a no-mmu and > wanting ELF, but i was being sufficiently general to bolster my claims > and to leave the door open for any other port out there i'm not > familiar with and any port that comes along actually, there was another case i was thinking of ... it isnt uncommon for architectures to use compiler flags which affect the ABI and thus make userland not a 1-to-1 test example for the kernel ... consider the regparam stuff we use on i386 -mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On 9/4/07, Mike Frysinger [EMAIL PROTECTED] wrote: On 9/2/07, Andi Kleen [EMAIL PROTECTED] wrote: On Sun, Sep 02, 2007 at 11:15:23AM -0400, Mike Frysinger wrote: there is still the ABI issue ... code written in kernel space in pure asm cannot always be compiled in userspace and work properly/the same Is that a blackfin weirdness? yes, Blackfin is weird in this respect due to being a no-mmu and wanting ELF, but i was being sufficiently general to bolster my claims and to leave the door open for any other port out there i'm not familiar with and any port that comes along actually, there was another case i was thinking of ... it isnt uncommon for architectures to use compiler flags which affect the ABI and thus make userland not a 1-to-1 test example for the kernel ... consider the regparam stuff we use on i386 -mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On 9/2/07, Andi Kleen <[EMAIL PROTECTED]> wrote: > On Sun, Sep 02, 2007 at 11:15:23AM -0400, Mike Frysinger wrote: > > there is still the ABI issue ... code written in kernel space in pure > > asm cannot always be compiled in userspace and work properly/the same > > Is that a blackfin weirdness? yes, Blackfin is weird in this respect due to being a no-mmu and wanting ELF, but i was being sufficiently general to bolster my claims and to leave the door open for any other port out there i'm not familiar with and any port that comes along > > > The other issue to test some of them properly you need unmapped pages > > > etc. That gets much easier to do in user space. There are some other > > > issues. > > > > you mean testing boundary overflows ? can be handled with canaries > > rather than segfaults i imagine ... > > Not for reads, no. true, but generally the string functions arent searching for crashes in the boundary conditions ... they just happen to sometimes find them :) -mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On Sat 1 Sep 2007 18:08, Andi Kleen pondered: > "Mike Frysinger" <[EMAIL PROTECTED]> writes: > > > is there any sort of standard for testing and integration into > > mainline ? > > Everybody does their own. That kind of stinks - and seems to be a potential duplication of effort all over the place. > > in the Blackfin world, we've been developing little > > external kernel modules and adding them to our own testsuite, but > > often times these things are not Blackfin specific. case in point, > > we're integrating a string testsuite to make sure all of the fun str* > > and mem* functions are sane and operate as they expected, but rather > > than having just Blackfin benefit here, i'd like to see this pushed > > upstream ... > > I would like to see this too. I wrote a couple of unit tests > during x86-64 development too and they would be handy > to have in a central place. > > The SUSE kernel also has a crasher module that exercises the > allocators and is pretty useful to stress code in general. > > Disadvantage is that test code tends to be hackish and ugly > and very specialized and will probably not pass standard review > procedures. Then this makes perfect sense to actually have it in the proper places, and have it go through the standard review process. I have run into problems recently, where ugly hackish tests seem never really catch actual faults. (Since they were written by the same person who wrote the driver - Design a fault in, code the fault, test the fault is there). With many eyes - and all that stuff - hopefully we would catch things. > Also the rt people seem to have pushed a couple of tests in already, > but I always hated it that they're in the standard directories. > RCU also has, in fact they added a "eat all my CPU in tests" > CONFIG option. Just making that dependent on a CONFIG_UNIT_TESTS > would be a good change in itself. And then there are the slab > failure inducers of course. Putting them close to the standard directories might be OK - keeping the tests close to the thing that is being tested may help with maintainers keeping an eye on things that might be missed otherwise. Maybe these types of tests should go into a separate directory - power on self tests (post) directories in the respective places? ./kernel/post and ./drivers//post? So Mike's proposed string tests would go into ./lib/post/strings.c When things pass - you get a single indication (all tests for unit X passed) or a pass for each test? -Robin - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On Sat 1 Sep 2007 18:08, Andi Kleen pondered: Mike Frysinger [EMAIL PROTECTED] writes: is there any sort of standard for testing and integration into mainline ? Everybody does their own. That kind of stinks - and seems to be a potential duplication of effort all over the place. in the Blackfin world, we've been developing little external kernel modules and adding them to our own testsuite, but often times these things are not Blackfin specific. case in point, we're integrating a string testsuite to make sure all of the fun str* and mem* functions are sane and operate as they expected, but rather than having just Blackfin benefit here, i'd like to see this pushed upstream ... I would like to see this too. I wrote a couple of unit tests during x86-64 development too and they would be handy to have in a central place. The SUSE kernel also has a crasher module that exercises the allocators and is pretty useful to stress code in general. Disadvantage is that test code tends to be hackish and ugly and very specialized and will probably not pass standard review procedures. Then this makes perfect sense to actually have it in the proper places, and have it go through the standard review process. I have run into problems recently, where ugly hackish tests seem never really catch actual faults. (Since they were written by the same person who wrote the driver - Design a fault in, code the fault, test the fault is there). With many eyes - and all that stuff - hopefully we would catch things. Also the rt people seem to have pushed a couple of tests in already, but I always hated it that they're in the standard directories. RCU also has, in fact they added a eat all my CPU in tests CONFIG option. Just making that dependent on a CONFIG_UNIT_TESTS would be a good change in itself. And then there are the slab failure inducers of course. Putting them close to the standard directories might be OK - keeping the tests close to the thing that is being tested may help with maintainers keeping an eye on things that might be missed otherwise. Maybe these types of tests should go into a separate directory - power on self tests (post) directories in the respective places? ./kernel/post and ./drivers//post? So Mike's proposed string tests would go into ./lib/post/strings.c When things pass - you get a single indication (all tests for unit X passed) or a pass for each test? -Robin - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On 9/2/07, Andi Kleen [EMAIL PROTECTED] wrote: On Sun, Sep 02, 2007 at 11:15:23AM -0400, Mike Frysinger wrote: there is still the ABI issue ... code written in kernel space in pure asm cannot always be compiled in userspace and work properly/the same Is that a blackfin weirdness? yes, Blackfin is weird in this respect due to being a no-mmu and wanting ELF, but i was being sufficiently general to bolster my claims and to leave the door open for any other port out there i'm not familiar with and any port that comes along The other issue to test some of them properly you need unmapped pages etc. That gets much easier to do in user space. There are some other issues. you mean testing boundary overflows ? can be handled with canaries rather than segfaults i imagine ... Not for reads, no. true, but generally the string functions arent searching for crashes in the boundary conditions ... they just happen to sometimes find them :) -mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
Mike Frysinger wrote: On 9/1/07, Bill Davidsen <[EMAIL PROTECTED]> wrote: If you want to test that stuff and run it on the current code in the kernel, how about a kernel module? You could "modprobe sanitytest" or something and report to syslog at module load time. And maybe have a parameter which does something drastic if something core is so hosed that filesystem damage is likely. Or just optional init code run by a kernel option, perhaps sanity testing after boot is not a great idea. i'm fully capable of doing my own testing right now; that isnt the point of this thread ... the point is for everyone to benefit, not just the Blackfin architecture That's exactly the point I was making, if your tests and any others people feel are useful were in a module and/or could be run as init code only when wanted, everyone could benefit from the tests. Anything related to build sanity could be in one place, to guard against surprises like new versions of gcc, typos in patches, etc. -- bill davidsen <[EMAIL PROTECTED]> CTO TMR Associates, Inc Doing interesting things with small computers since 1979 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
Mike Frysinger wrote: On 9/1/07, Bill Davidsen [EMAIL PROTECTED] wrote: If you want to test that stuff and run it on the current code in the kernel, how about a kernel module? You could modprobe sanitytest or something and report to syslog at module load time. And maybe have a parameter which does something drastic if something core is so hosed that filesystem damage is likely. Or just optional init code run by a kernel option, perhaps sanity testing after boot is not a great idea. i'm fully capable of doing my own testing right now; that isnt the point of this thread ... the point is for everyone to benefit, not just the Blackfin architecture That's exactly the point I was making, if your tests and any others people feel are useful were in a module and/or could be run as init code only when wanted, everyone could benefit from the tests. Anything related to build sanity could be in one place, to guard against surprises like new versions of gcc, typos in patches, etc. -- bill davidsen [EMAIL PROTECTED] CTO TMR Associates, Inc Doing interesting things with small computers since 1979 - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On 9/2/07, Andi Kleen <[EMAIL PROTECTED]> wrote: > The other issue to test some of them properly you need unmapped pages > etc. That gets much easier to do in user space. There are some other > issues. vmalloc, vmap, etc. always put a guard page after the allocation. So if you test string operations on vmalloc()'ed memory, you have unmapped pages in both ends. I like the idea of a test suite for kernel internals, but I also think it would be best to run it in kernel mode since things like atomics, etc. may work differently in user space on some architectures. Haavard - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On Sun, Sep 02, 2007 at 11:15:23AM -0400, Mike Frysinger wrote: > there is still the ABI issue ... code written in kernel space in pure > asm cannot always be compiled in userspace and work properly/the same Is that a blackfin weirdness? > > > The other issue to test some of them properly you need unmapped pages > > etc. That gets much easier to do in user space. There are some other > > issues. > > you mean testing boundary overflows ? can be handled with canaries > rather than segfaults i imagine ... Not for reads, no. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On 9/2/07, Andi Kleen <[EMAIL PROTECTED]> wrote: > On Sat, Sep 01, 2007 at 06:50:30PM -0400, Mike Frysinger wrote: > > On 02 Sep 2007 00:08:57 +0200, Andi Kleen <[EMAIL PROTECTED]> wrote: > > > BTW string functions are best tested in user space. That's > > > a relatively bad example. > > > > in theory, maybe ... in reality, i really dont think so > > > > the string implementations are spread out over the kernel ... there's > > implementations in lib/, include/asm-*/, and arch/*/lib/ ... so any > > test code that would use these sources is going to be an ugly hack to > > make sure it grabs all the right pieces from all the right places. > > string functions tend to be self contained. "tend to be" does not lend itself to being cleanly tested ... and i imagine i'd get some pretty heavy resistance if i proposed re-organzing code just so that i could compile it outside of the kernel there is still the ABI issue ... code written in kernel space in pure asm cannot always be compiled in userspace and work properly/the same > The other issue to test some of them properly you need unmapped pages > etc. That gets much easier to do in user space. There are some other > issues. you mean testing boundary overflows ? can be handled with canaries rather than segfaults i imagine ... -mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On Sat, Sep 01, 2007 at 06:50:30PM -0400, Mike Frysinger wrote: > On 02 Sep 2007 00:08:57 +0200, Andi Kleen <[EMAIL PROTECTED]> wrote: > > BTW string functions are best tested in user space. That's > > a relatively bad example. > > in theory, maybe ... in reality, i really dont think so > > the string implementations are spread out over the kernel ... there's > implementations in lib/, include/asm-*/, and arch/*/lib/ ... so any > test code that would use these sources is going to be an ugly hack to > make sure it grabs all the right pieces from all the right places. string functions tend to be self contained. The other issue to test some of them properly you need unmapped pages etc. That gets much easier to do in user space. There are some other issues. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On Sat, Sep 01, 2007 at 06:50:30PM -0400, Mike Frysinger wrote: On 02 Sep 2007 00:08:57 +0200, Andi Kleen [EMAIL PROTECTED] wrote: BTW string functions are best tested in user space. That's a relatively bad example. in theory, maybe ... in reality, i really dont think so the string implementations are spread out over the kernel ... there's implementations in lib/, include/asm-*/, and arch/*/lib/ ... so any test code that would use these sources is going to be an ugly hack to make sure it grabs all the right pieces from all the right places. string functions tend to be self contained. The other issue to test some of them properly you need unmapped pages etc. That gets much easier to do in user space. There are some other issues. -Andi - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On 9/2/07, Andi Kleen [EMAIL PROTECTED] wrote: On Sat, Sep 01, 2007 at 06:50:30PM -0400, Mike Frysinger wrote: On 02 Sep 2007 00:08:57 +0200, Andi Kleen [EMAIL PROTECTED] wrote: BTW string functions are best tested in user space. That's a relatively bad example. in theory, maybe ... in reality, i really dont think so the string implementations are spread out over the kernel ... there's implementations in lib/, include/asm-*/, and arch/*/lib/ ... so any test code that would use these sources is going to be an ugly hack to make sure it grabs all the right pieces from all the right places. string functions tend to be self contained. tend to be does not lend itself to being cleanly tested ... and i imagine i'd get some pretty heavy resistance if i proposed re-organzing code just so that i could compile it outside of the kernel there is still the ABI issue ... code written in kernel space in pure asm cannot always be compiled in userspace and work properly/the same The other issue to test some of them properly you need unmapped pages etc. That gets much easier to do in user space. There are some other issues. you mean testing boundary overflows ? can be handled with canaries rather than segfaults i imagine ... -mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On Sun, Sep 02, 2007 at 11:15:23AM -0400, Mike Frysinger wrote: there is still the ABI issue ... code written in kernel space in pure asm cannot always be compiled in userspace and work properly/the same Is that a blackfin weirdness? The other issue to test some of them properly you need unmapped pages etc. That gets much easier to do in user space. There are some other issues. you mean testing boundary overflows ? can be handled with canaries rather than segfaults i imagine ... Not for reads, no. -Andi - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On 9/2/07, Andi Kleen [EMAIL PROTECTED] wrote: The other issue to test some of them properly you need unmapped pages etc. That gets much easier to do in user space. There are some other issues. vmalloc, vmap, etc. always put a guard page after the allocation. So if you test string operations on vmalloc()'ed memory, you have unmapped pages in both ends. I like the idea of a test suite for kernel internals, but I also think it would be best to run it in kernel mode since things like atomics, etc. may work differently in user space on some architectures. Haavard - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On 9/1/07, Bill Davidsen <[EMAIL PROTECTED]> wrote: > If you want to test that stuff and run it on the current code in the > kernel, how about a kernel module? You could "modprobe sanitytest" or > something and report to syslog at module load time. And maybe have a > parameter which does something drastic if something core is so hosed > that filesystem damage is likely. Or just optional init code run by a > kernel option, perhaps sanity testing after boot is not a great idea. i'm fully capable of doing my own testing right now; that isnt the point of this thread ... the point is for everyone to benefit, not just the Blackfin architecture -mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
Mike Frysinger wrote: is there any sort of standard for testing and integration into mainline ? in the Blackfin world, we've been developing little external kernel modules and adding them to our own testsuite, but often times these things are not Blackfin specific. case in point, we're integrating a string testsuite to make sure all of the fun str* and mem* functions are sane and operate as they expected, but rather than having just Blackfin benefit here, i'd like to see this pushed upstream ... i'm fully aware of LTP (as i work on it), but i feel that serves a great purpose for exercising the API/ABI exposed to userspace either directly through the kernel or indirectly through the system libc ... it isnt a very good tool for testing kernel internals, especially as the kernel changes and evolves. is there a framework already in place i'm not aware of ? should there be ? should this all live in LTP ? i wouldnt mind an option under kernel hacking "Enable testsuites" ... as far as i can tell, the testing process is really based extensively on feedback from people, nothing really automated. If you want to test that stuff and run it on the current code in the kernel, how about a kernel module? You could "modprobe sanitytest" or something and report to syslog at module load time. And maybe have a parameter which does something drastic if something core is so hosed that filesystem damage is likely. Or just optional init code run by a kernel option, perhaps sanity testing after boot is not a great idea. -- Bill Davidsen <[EMAIL PROTECTED]> "We have more to fear from the bungling of the incompetent than from the machinations of the wicked." - from Slashdot - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On 9/1/07, Robin Getz <[EMAIL PROTECTED]> wrote: > On Fri 31 Aug 2007 17:22, Mike Frysinger pondered: > > is there any sort of standard for testing and integration into > > mainline ? in the Blackfin world, we've been developing little > > external kernel modules and adding them to our own testsuite, but > > often times these things are not Blackfin specific. case in point, > > we're integrating a string testsuite to make sure all of the fun str* > > and mem* functions are sane and operate as they expected, but rather > > than having just Blackfin benefit here, i'd like to see this pushed > > upstream ... > > I know there have been some discussions at past OLS about some testing that > Martin was working on. > > http://www.linuxsymposium.org/2006/view_abstract.php?content_key=13 > > But I think this was more functional tests, less unit tests that what you are > talking/asking about. > > http://test.kernel.org/functional/index.html > > Also - If I remember - most of the existing tests were for a self hosted > environment - and might not be well suited to embedded (like Blackfin) which > requires cross compile, and pretty thin runtime environment (uClibc + > busybox's msh as shell). i flipped through the autotest code a bit but i dont think this can do what i'm looking for. it appears to be a mix of LTP provides and what we have for our Blackfin test infrastructure -- iow, all userspace stuff and no real way of exercising kernel internals. -mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On 02 Sep 2007 00:08:57 +0200, Andi Kleen <[EMAIL PROTECTED]> wrote: > BTW string functions are best tested in user space. That's > a relatively bad example. in theory, maybe ... in reality, i really dont think so the string implementations are spread out over the kernel ... there's implementations in lib/, include/asm-*/, and arch/*/lib/ ... so any test code that would use these sources is going to be an ugly hack to make sure it grabs all the right pieces from all the right places. then there's the issue of API ... compiling code that is designed to be in the kernel and nowhere else is going to require a mighty bit of crappy glue logic in order for it to compile properly in userspace but let's cheat and just use the kernel build system to produce .o's for us ... then there's the issue of ABI. there is no guarantee the ABI the kernel uses internally is the same as the ABI userspace uses (and in fact, on some architectures it is simply not possible). so you're left with a solution that may work for some, but not for others. or we put one common piece of C test code right into the kernel and be done. no worrying about the tested API/ABI/environment being different from the actual runtime the kernel itself has. -mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
"Mike Frysinger" <[EMAIL PROTECTED]> writes: > is there any sort of standard for testing and integration into > mainline ? Everybody does their own. > in the Blackfin world, we've been developing little > external kernel modules and adding them to our own testsuite, but > often times these things are not Blackfin specific. case in point, > we're integrating a string testsuite to make sure all of the fun str* > and mem* functions are sane and operate as they expected, but rather > than having just Blackfin benefit here, i'd like to see this pushed > upstream ... I would like to see this too. I wrote a couple of unit tests during x86-64 development too and they would be handy to have in a central place. The SUSE kernel also has a crasher module that exercises the allocators and is pretty useful to stress code in general. Disadvantage is that test code tends to be hackish and ugly and very specialized and will probably not pass standard review procedures. Also the rt people seem to have pushed a couple of tests in already, but I always hated it that they're in the standard directories. RCU also has, in fact they added a "eat all my CPU in tests" CONFIG option. Just making that dependent on a CONFIG_UNIT_TESTS would be a good change in itself. And then there are the slab failure inducers of course. BTW string functions are best tested in user space. That's a relatively bad example. -Andi - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On Fri 31 Aug 2007 17:22, Mike Frysinger pondered: > is there any sort of standard for testing and integration into > mainline ? in the Blackfin world, we've been developing little > external kernel modules and adding them to our own testsuite, but > often times these things are not Blackfin specific. case in point, > we're integrating a string testsuite to make sure all of the fun str* > and mem* functions are sane and operate as they expected, but rather > than having just Blackfin benefit here, i'd like to see this pushed > upstream ... I know there have been some discussions at past OLS about some testing that Martin was working on. http://www.linuxsymposium.org/2006/view_abstract.php?content_key=13 But I think this was more functional tests, less unit tests that what you are talking/asking about. http://test.kernel.org/functional/index.html Also - If I remember - most of the existing tests were for a self hosted environment - and might not be well suited to embedded (like Blackfin) which requires cross compile, and pretty thin runtime environment (uClibc + busybox's msh as shell). Martin? -Robin - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On Fri 31 Aug 2007 17:22, Mike Frysinger pondered: is there any sort of standard for testing and integration into mainline ? in the Blackfin world, we've been developing little external kernel modules and adding them to our own testsuite, but often times these things are not Blackfin specific. case in point, we're integrating a string testsuite to make sure all of the fun str* and mem* functions are sane and operate as they expected, but rather than having just Blackfin benefit here, i'd like to see this pushed upstream ... I know there have been some discussions at past OLS about some testing that Martin was working on. http://www.linuxsymposium.org/2006/view_abstract.php?content_key=13 But I think this was more functional tests, less unit tests that what you are talking/asking about. http://test.kernel.org/functional/index.html Also - If I remember - most of the existing tests were for a self hosted environment - and might not be well suited to embedded (like Blackfin) which requires cross compile, and pretty thin runtime environment (uClibc + busybox's msh as shell). Martin? -Robin - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
Mike Frysinger [EMAIL PROTECTED] writes: is there any sort of standard for testing and integration into mainline ? Everybody does their own. in the Blackfin world, we've been developing little external kernel modules and adding them to our own testsuite, but often times these things are not Blackfin specific. case in point, we're integrating a string testsuite to make sure all of the fun str* and mem* functions are sane and operate as they expected, but rather than having just Blackfin benefit here, i'd like to see this pushed upstream ... I would like to see this too. I wrote a couple of unit tests during x86-64 development too and they would be handy to have in a central place. The SUSE kernel also has a crasher module that exercises the allocators and is pretty useful to stress code in general. Disadvantage is that test code tends to be hackish and ugly and very specialized and will probably not pass standard review procedures. Also the rt people seem to have pushed a couple of tests in already, but I always hated it that they're in the standard directories. RCU also has, in fact they added a eat all my CPU in tests CONFIG option. Just making that dependent on a CONFIG_UNIT_TESTS would be a good change in itself. And then there are the slab failure inducers of course. BTW string functions are best tested in user space. That's a relatively bad example. -Andi - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On 02 Sep 2007 00:08:57 +0200, Andi Kleen [EMAIL PROTECTED] wrote: BTW string functions are best tested in user space. That's a relatively bad example. in theory, maybe ... in reality, i really dont think so the string implementations are spread out over the kernel ... there's implementations in lib/, include/asm-*/, and arch/*/lib/ ... so any test code that would use these sources is going to be an ugly hack to make sure it grabs all the right pieces from all the right places. then there's the issue of API ... compiling code that is designed to be in the kernel and nowhere else is going to require a mighty bit of crappy glue logic in order for it to compile properly in userspace but let's cheat and just use the kernel build system to produce .o's for us ... then there's the issue of ABI. there is no guarantee the ABI the kernel uses internally is the same as the ABI userspace uses (and in fact, on some architectures it is simply not possible). so you're left with a solution that may work for some, but not for others. or we put one common piece of C test code right into the kernel and be done. no worrying about the tested API/ABI/environment being different from the actual runtime the kernel itself has. -mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On 9/1/07, Robin Getz [EMAIL PROTECTED] wrote: On Fri 31 Aug 2007 17:22, Mike Frysinger pondered: is there any sort of standard for testing and integration into mainline ? in the Blackfin world, we've been developing little external kernel modules and adding them to our own testsuite, but often times these things are not Blackfin specific. case in point, we're integrating a string testsuite to make sure all of the fun str* and mem* functions are sane and operate as they expected, but rather than having just Blackfin benefit here, i'd like to see this pushed upstream ... I know there have been some discussions at past OLS about some testing that Martin was working on. http://www.linuxsymposium.org/2006/view_abstract.php?content_key=13 But I think this was more functional tests, less unit tests that what you are talking/asking about. http://test.kernel.org/functional/index.html Also - If I remember - most of the existing tests were for a self hosted environment - and might not be well suited to embedded (like Blackfin) which requires cross compile, and pretty thin runtime environment (uClibc + busybox's msh as shell). i flipped through the autotest code a bit but i dont think this can do what i'm looking for. it appears to be a mix of LTP provides and what we have for our Blackfin test infrastructure -- iow, all userspace stuff and no real way of exercising kernel internals. -mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
Mike Frysinger wrote: is there any sort of standard for testing and integration into mainline ? in the Blackfin world, we've been developing little external kernel modules and adding them to our own testsuite, but often times these things are not Blackfin specific. case in point, we're integrating a string testsuite to make sure all of the fun str* and mem* functions are sane and operate as they expected, but rather than having just Blackfin benefit here, i'd like to see this pushed upstream ... i'm fully aware of LTP (as i work on it), but i feel that serves a great purpose for exercising the API/ABI exposed to userspace either directly through the kernel or indirectly through the system libc ... it isnt a very good tool for testing kernel internals, especially as the kernel changes and evolves. is there a framework already in place i'm not aware of ? should there be ? should this all live in LTP ? i wouldnt mind an option under kernel hacking Enable testsuites ... as far as i can tell, the testing process is really based extensively on feedback from people, nothing really automated. If you want to test that stuff and run it on the current code in the kernel, how about a kernel module? You could modprobe sanitytest or something and report to syslog at module load time. And maybe have a parameter which does something drastic if something core is so hosed that filesystem damage is likely. Or just optional init code run by a kernel option, perhaps sanity testing after boot is not a great idea. -- Bill Davidsen [EMAIL PROTECTED] We have more to fear from the bungling of the incompetent than from the machinations of the wicked. - from Slashdot - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: the Linux kernel, testsuites, and maybe *you*
On 9/1/07, Bill Davidsen [EMAIL PROTECTED] wrote: If you want to test that stuff and run it on the current code in the kernel, how about a kernel module? You could modprobe sanitytest or something and report to syslog at module load time. And maybe have a parameter which does something drastic if something core is so hosed that filesystem damage is likely. Or just optional init code run by a kernel option, perhaps sanity testing after boot is not a great idea. i'm fully capable of doing my own testing right now; that isnt the point of this thread ... the point is for everyone to benefit, not just the Blackfin architecture -mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
the Linux kernel, testsuites, and maybe *you*
is there any sort of standard for testing and integration into mainline ? in the Blackfin world, we've been developing little external kernel modules and adding them to our own testsuite, but often times these things are not Blackfin specific. case in point, we're integrating a string testsuite to make sure all of the fun str* and mem* functions are sane and operate as they expected, but rather than having just Blackfin benefit here, i'd like to see this pushed upstream ... i'm fully aware of LTP (as i work on it), but i feel that serves a great purpose for exercising the API/ABI exposed to userspace either directly through the kernel or indirectly through the system libc ... it isnt a very good tool for testing kernel internals, especially as the kernel changes and evolves. is there a framework already in place i'm not aware of ? should there be ? should this all live in LTP ? i wouldnt mind an option under kernel hacking "Enable testsuites" ... as far as i can tell, the testing process is really based extensively on feedback from people, nothing really automated. -mike - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
the Linux kernel, testsuites, and maybe *you*
is there any sort of standard for testing and integration into mainline ? in the Blackfin world, we've been developing little external kernel modules and adding them to our own testsuite, but often times these things are not Blackfin specific. case in point, we're integrating a string testsuite to make sure all of the fun str* and mem* functions are sane and operate as they expected, but rather than having just Blackfin benefit here, i'd like to see this pushed upstream ... i'm fully aware of LTP (as i work on it), but i feel that serves a great purpose for exercising the API/ABI exposed to userspace either directly through the kernel or indirectly through the system libc ... it isnt a very good tool for testing kernel internals, especially as the kernel changes and evolves. is there a framework already in place i'm not aware of ? should there be ? should this all live in LTP ? i wouldnt mind an option under kernel hacking Enable testsuites ... as far as i can tell, the testing process is really based extensively on feedback from people, nothing really automated. -mike - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
