Subrata, Michael,
Thank you for proposing. ideas
>> The idea is to submit the test cases to LTP, for certain kernel
>> features, which makes its way through the kernel-mm tree. When a
>> developer submits his/her feature to mm-kernel, he gives a specification
>> of the features. Depending on the feature specification available either
>> from the developer himself, or, from manpages, if it is available that
>> soon, tests cases will be written.
Basically, I agree your idea.
>> Having said that, i want to propose that, since in your project you have
>> already taken up the Job of making available the system call test cases
>> up-to-date, we can score a point here. Whenever a new system call code
>> enters the mm-tree in the kernel, somebody in your project can write the
>> test case(s) immediately and make it available to:
>>
>> 1) Your Project,
>> 2) LTP,
>> 3) LKML,
Agreed.
I will negotiate our members in soon.
>From now on, I'll be agitating more to get man pages provided more with new
>syscalls and ther kernel-userland interfaces. That will mean either I twist
Great, These will help us to develop/enhance our test cases better.
I will check your pages.
>calls. What list do I need to subscribe to?
CrackerJack has two mailing list.
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/crackerjack-users
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/crackerjack-devel
>I can't help but feel that part of the solution here is to agitate more so
>that the developer is (strongly) encouraged to submit test cases along with
>patches that change the kernel-userspace interface. As things stand, there
>are far too many bugs in released interrfaces.
I can't help neither, but I will do talk developers importance of
providing/using
test cases when new interface has been released.
Every users wants to use less bugs kernel, wants to know what difference
happens if it exists.
Hisashi
>送信者 ： Michael Kerrisk <[EMAIL PROTECTED]>
>主題 ： Re: [LTP] Crackerjack and Linux Test Project
>受信日 ：08/07/15 21:39
>属性 ： なし
>
>On Mon, Jul 14, 2008 at 1:05 PM, Subrata Modak <[EMAIL PROTECTED]>
>wrote:
>
>> Dear Hisashi San,
>>
>> Since we have already embarked on our path to mutual co-operation in
>> Linux testing and test cases development, i would like to discuss some
>> of my ideas, which i think is capable of adding shot-in-the arm for
>> linux testing.
>>
>> I want to talk about Test Driven Development for Linux Kernel Testing,
>> which can bring us very early effective results and can catch Bug(s), if
>> any, in the kernel code much much early, and even before the code makes
>> it´s way through the stable kernel release.
>>
>> The idea is to submit the test cases to LTP, for certain kernel
>> features, which makes its way through the kernel-mm tree. When a
>> developer submits his/her feature to mm-kernel, he gives a specification
>> of the features. Depending on the feature specification available either
>> from the developer himself, or, from manpages, if it is available that
>> soon, tests cases will be written.
>
>
>From now on, I'll be agitating more to get man pages provided more with new
>syscalls and ther kernel-userland interfaces. That will mean either I twist
>developers arms to write pages ;-), or I write them myself, with help from
>them. I do think that man-pages, if well written, are often sufficient as
>(or at least a very good base for) a test specification. Here's an example
>that I did with the timerfd API, finding two bugs in the process:
>http://thread.gmane.org/gmane.linux.kernel/613442 . I did something similar
>while writing the utimensat(2) man page, finding 5 or 6 different bugs in
>the end, see
>http://linux-man-pages.blogspot.com/2008/06/whats-wrong-with-kernel-userland_30.html
> .
>
>
>> Now, we have an inherent question,
>> who writes and submits those test cases ?? Either it can be the
>> developer himself, or, somebody from the test community.
>
>
>Or, IMO, preferably both, working together.
>
>
>> Since, the
>> developer is not bound to submit any test case,
>
>
>I can't help but feel that part of the solution here is to agitate more so
>that the developer is (strongly) encouraged to submit test cases along with
>patches that change the kernel-userspace interface. As things stand, there
>are far too many bugs in released interrfaces.
>
>
>> we can assume we need to
>> initiate doing something in this regard.
>>
>> Having said that, i want to propose that, since in your project you have
>> already taken up the Job of making available the system call test cases
>> up-to-date, we can score a point here. Whenever a new system call code
>> enters the mm-tree in the kernel, somebody in your project can write the
>> test case(s) immediately and make it available to:
>>
>> 1) Your Project,
>> 2) LTP,
>> 3) LKML,
>
>
>I would also be very happy to see any test cases you produce for new system
>calls. What list do I need to subscribe to?
>
>
>> We will use the test case(s) to test that particular syscall feature and
>> report Bug(s) to the developer(s) and LKML. Meanwhile, updates to the
>> test cases will also go on depending on feedback from community. With
>> that we will address the following issues:
>> 1) Catch bug at the earliest for the new syscalls in the kernel, so that
>> the cost of Bug fixing, when something is discovered later, can be
>> hugely reduced,
>
>
>Bug fixing is perhaps the lesser problem. The greater problem is that if
>there are bugs, we must change the ABI to fix them, and if that is done
>after a stable kernel release, then userland programmer's can end up having
>to do things like special case their code according to the kernel version.
>
>
>> 2) We have a regular and systematic flow of test cases for both LTP and
>> Crackerjack. This process becomes institutional and gurantees us maximum
>> and continued code coverage for syscalls in future,
>> 3) LTP-mm tree is born,
>> 4) Others/developers get motivated to submit test cases(to LTP and
>> Crackerjack) at the time of submission of kernel features to
>> kernel-mm-tree, if we show them the benefit of early testing and it´s
>> effectiveness of controlling more bugs to seep in before it is too late
>> to fix.
>>
>> For this to happen, we need to make our test cases ready when the
>> corresponding kernel features are in -mm tree. Let me know your ideas on
>> this.
>
>
>Part of the problem here is knowing when interface changes have occurred.
>Seeing new system calls in a release is easy. But, for the project to be
>truly effective, all kernel-userland interfacechanges need to be checked
>for, including
>
>system calls
>ioctl()s
>netlink
>/proc
>/sys (okay -- that's a big ask at the moment)
>virtual /dev interfaces used for kernel-uerland communication
>And probably other things I forgot right now.
>
>Cheers,
>
>Michael
>
>
>On Mon, 2008-07-14 at 13:39 +0900, [EMAIL PROTECTED]
>wrote:
>> Subrata,
>>
>> >Masatake is already working to port the missing syscall test cases in
>> >LTP from the Crackerjack Project to LTP. And it is already a huge
>> I am glad to hear that.
>> >effort. I think the idea here is great as we are not re-inventing the
>> Yes, you can do this and we can use our time to develope more test cases.
>> >to port from you, unless somebody in LTP community has already submitted
>> In that case, we will refer your cases and make them better.
>> >https://lists.sourceforge.net/mailman/listinfo/ltp-list
>> I joined now. I am very surprised that your community are
>> working so active. I will check and I will post anything
>> I can collaborate.
>> >We will be collaborating as far as we can, as all of us striving to
>> >Linux better.
>> Yes, agree. We will continue this effort for our purpose, Linux better.
>> Please post anything to both our mailing list and cc-d your mailing list
>> when you have to contact me.
>>
>> Thanks,
>>
>> Hisashi
>>
>> >送信者 ： Subrata Modak <[EMAIL PROTECTED]>
>> >主題 ： Re: Re[2]: Re[2]: Re[2]: Re[2]: [Crackerjack-devel] [LTP]Crackerjack
>andLinuxTes
>> >受信日 ：08/07/12 04:10
>> >属性 ： なし
>> >
>> >Dear Hisashi Hashimoto San,
>> >
>> >Thank you for writing back to us. We are delighted to colloborate on all
>> >fronts to make Linux testing better. My comments embedded in the
>> >following lines.
>> >
>> >On Fri, 2008-07-11 at 13:53 +0900, [EMAIL PROTECTED]
>> >wrote:
>> >> Subrata,
>> >>
>> >> Sorry for long time no reply.
>> >> We had internal meeting of our project
>> >> regarding how to proceed.
>> >> Current our plan is:
>> >> (1) Unfortunattelly, our test cases do not cover all system call yet.
>> >> Total number of system call in Kernel 2.6.26 is almost 326.
>> >> But, we developed around 299.
>> >
>> >Masatake is already working to port the missing syscall test cases in
>> >LTP from the Crackerjack Project to LTP. And it is already a huge
>> >effort. I think the idea here is great as we are not re-inventing the
>> >same wheel. We can use your test cases released under GPLv2 and port to
>> >LTP format.
>> >
>> >We gain by getting more kernel test cases in LTP without the need to
>> >massively write things. And you retain your goal of providing a
>> >regression comparisn framework. In future, whenever the no. of syscalls
>> >goes increasing in kernels, and as soon as you write them ,we will try
>> >to port from you, unless somebody in LTP community has already submitted
>> >a test case in that regard. Or, even we can concentrate to write test
>> >cases in some other are as well.
>> >
>> >Presently, the first job is to port the existing ones from you to us.
>> >
>> >> Therefore, we would like to start developing in soon.
>> >> The starting date would be some time in August.
>> >> (2) Some of our developed test cases do not meet our quality criteria.
>> >> As you know, we set our quality goal as covarage ratio more than
>> >> yours.
>> >> We are now checking all tests again. And if necessary, we need
>to
>> >> enhance test cases.
>> >
>> >Good to know that. As and when you do this, we might also relook into
>> >ours and change accordingly.
>> >
>> >> (3) Through our acitivity (1) and (2), we would like to exchange
>> >> information and technical/engineering discussion on this mailing
>list.
>> >
>> >Definitely yes. All of you can subscribe to the ltp mailing list:
>> >https://lists.sourceforge.net/mailman/listinfo/ltp-list
>> >
>> >And please add us as CC to all the technical discussion you have. Guys
>> >in ltp mailing list will need to subscribe yours if they want to reply
>> >back to you.
>> >
>> >>
>> >> Yesterday, 4 Maintainers(Andrew Morton, Paul Moor, James Morris, Thomas
>Gleixner)
>> >> visit Japan to make presentation regarding the latest kernel/security
>activity.
>> >> I had opportunity to make presentation to them about our current
>status.
>> >> I present that
>> >> (1) We released test cases 300/326.
>> >> (2) We plan to make complete set and enhance quality.
>> >> (3) We continue to talk with other test project(LTP as you,
>AutoTest:Martin Bleigh),
>> >> and exchange information and status, and others.
>> >> Maintainers said/recommend to expand the target, to continue the effort
>> >> and collaborate with LTP and so on.
>> >
>> >We will be collaborating as far as we can, as all of us striving to
>> >Linux better.
>> >
>> >
>> >Regards--
>> >Subrata
>> >
>> >>
>> >> I hope you and we can do the collaboration work on test project.
>> >>
>> >> Best,
>> >>
>> >> Hashimoto Hisashi
>> >
>> >
>> >
>> Hisashi Hashimoto
>> Section Manager
>> Open Source Software Promotion Center
>> Platform Software
>> Hitachi, Ltd., Software Division
>> Tel : +81-45-862-8424, Fax : +81-45-862-9047
>> [EMAIL PROTECTED]
>
>
>
>
>--
>Michael Kerrisk
>Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
>man-pages online: http://www.kernel.org/doc/man-pages/online_pages.html
>Found a bug? http://www.kernel.org/doc/man-pages/reporting_bugs.html
>
Hisashi Hashimoto
Section Manager
Open Source Software Promotion Center
Platform Software
Hitachi, Ltd., Software Division
Tel : +81-45-862-8424, Fax : +81-45-862-9047
[EMAIL PROTECTED]
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list
