Dear Hisashi San, Since we have already embarked on our path to mutual co-operation in Linux testing and test cases development, i would like to discuss some of my ideas, which i think is capable of adding shot-in-the arm for linux testing.
I want to talk about Test Driven Development for Linux Kernel Testing, which can bring us very early effective results and can catch Bug(s), if any, in the kernel code much much early, and even before the code makes it´s way through the stable kernel release. The idea is to submit the test cases to LTP, for certain kernel features, which makes its way through the kernel-mm tree. When a developer submits his/her feature to mm-kernel, he gives a specification of the features. Depending on the feature specification available either from the developer himself, or, from manpages, if it is available that soon, tests cases will be written. Now, we have an inherent question, who writes and submits those test cases ?? Either it can be the developer himself, or, somebody from the test community. Since, the developer is not bound to submit any test case, we can assume we need to initiate doing something in this regard. Having said that, i want to propose that, since in your project you have already taken up the Job of making available the system call test cases up-to-date, we can score a point here. Whenever a new system call code enters the mm-tree in the kernel, somebody in your project can write the test case(s) immediately and make it available to: 1) Your Project, 2) LTP, 3) LKML, We will use the test case(s) to test that particular syscall feature and report Bug(s) to the developer(s) and LKML. Meanwhile, updates to the test cases will also go on depending on feedback from community. With that we will address the following issues: 1) Catch bug at the earliest for the new syscalls in the kernel, so that the cost of Bug fixing, when something is discovered later, can be hugely reduced, 2) We have a regular and systematic flow of test cases for both LTP and Crackerjack. This process becomes institutional and gurantees us maximum and continued code coverage for syscalls in future, 3) LTP-mm tree is born, 4) Others/developers get motivated to submit test cases(to LTP and Crackerjack) at the time of submission of kernel features to kernel-mm-tree, if we show them the benefit of early testing and it´s effectiveness of controlling more bugs to seep in before it is too late to fix. For this to happen, we need to make our test cases ready when the corresponding kernel features are in -mm tree. Let me know your ideas on this. Regards-- Subrata On Mon, 2008-07-14 at 13:39 +0900, [EMAIL PROTECTED] wrote: > Subrata, > > >Masatake is already working to port the missing syscall test cases in > >LTP from the Crackerjack Project to LTP. And it is already a huge > I am glad to hear that. > >effort. I think the idea here is great as we are not re-inventing the > Yes, you can do this and we can use our time to develope more test cases. > >to port from you, unless somebody in LTP community has already submitted > In that case, we will refer your cases and make them better. > >https://lists.sourceforge.net/mailman/listinfo/ltp-list > I joined now. I am very surprised that your community are > working so active. I will check and I will post anything > I can collaborate. > >We will be collaborating as far as we can, as all of us striving to > >Linux better. > Yes, agree. We will continue this effort for our purpose, Linux better. > Please post anything to both our mailing list and cc-d your mailing list > when you have to contact me. > > Thanks, > > Hisashi > > >送信者 ： Subrata Modak <[EMAIL PROTECTED]> > >主題 ： Re: Re[2]: Re[2]: Re[2]: Re[2]: [Crackerjack-devel] [LTP]Crackerjack > >andLinuxTes > >受信日 ：08/07/12 04:10 > >属性 ： なし > > > >Dear Hisashi Hashimoto San, > > > >Thank you for writing back to us. We are delighted to colloborate on all > >fronts to make Linux testing better. My comments embedded in the > >following lines. > > > >On Fri, 2008-07-11 at 13:53 +0900, [EMAIL PROTECTED] > >wrote: > >> Subrata, > >> > >> Sorry for long time no reply. > >> We had internal meeting of our project > >> regarding how to proceed. > >> Current our plan is: > >> (1) Unfortunattelly, our test cases do not cover all system call yet. > >> Total number of system call in Kernel 2.6.26 is almost 326. > >> But, we developed around 299. > > > >Masatake is already working to port the missing syscall test cases in > >LTP from the Crackerjack Project to LTP. And it is already a huge > >effort. I think the idea here is great as we are not re-inventing the > >same wheel. We can use your test cases released under GPLv2 and port to > >LTP format. > > > >We gain by getting more kernel test cases in LTP without the need to > >massively write things. And you retain your goal of providing a > >regression comparisn framework. In future, whenever the no. of syscalls > >goes increasing in kernels, and as soon as you write them ,we will try > >to port from you, unless somebody in LTP community has already submitted > >a test case in that regard. Or, even we can concentrate to write test > >cases in some other are as well. > > > >Presently, the first job is to port the existing ones from you to us. > > > >> Therefore, we would like to start developing in soon. > >> The starting date would be some time in August. > >> (2) Some of our developed test cases do not meet our quality criteria. > >> As you know, we set our quality goal as covarage ratio more than > >> yours. > >> We are now checking all tests again. And if necessary, we need to > >> enhance test cases. > > > >Good to know that. As and when you do this, we might also relook into > >ours and change accordingly. > > > >> (3) Through our acitivity (1) and (2), we would like to exchange > >> information and technical/engineering discussion on this mailing > >> list. > > > >Definitely yes. All of you can subscribe to the ltp mailing list: > >https://lists.sourceforge.net/mailman/listinfo/ltp-list > > > >And please add us as CC to all the technical discussion you have. Guys > >in ltp mailing list will need to subscribe yours if they want to reply > >back to you. > > > >> > >> Yesterday, 4 Maintainers(Andrew Morton, Paul Moor, James Morris, Thomas > >> Gleixner) > >> visit Japan to make presentation regarding the latest kernel/security > >> activity. > >> I had opportunity to make presentation to them about our current status. > >> I present that > >> (1) We released test cases 300/326. > >> (2) We plan to make complete set and enhance quality. > >> (3) We continue to talk with other test project(LTP as you, > >> AutoTest:Martin Bleigh), > >> and exchange information and status, and others. > >> Maintainers said/recommend to expand the target, to continue the effort > >> and collaborate with LTP and so on. > > > >We will be collaborating as far as we can, as all of us striving to > >Linux better. > > > > > >Regards-- > >Subrata > > > >> > >> I hope you and we can do the collaboration work on test project. > >> > >> Best, > >> > >> Hashimoto Hisashi > > > > > > > Hisashi Hashimoto > Section Manager > Open Source Software Promotion Center > Platform Software > Hitachi, Ltd., Software Division > Tel : +81-45-862-8424, Fax : +81-45-862-9047 > [EMAIL PROTECTED] ------------------------------------------------------------------------- Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list
