Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
Hi Masami, On Mon, 29 Sep 2014 20:40:51 +0900, Masami Hiramatsu wrote: > (2014/09/26 23:46), Namhyung Kim wrote: >> The perf should be built with no extra dependencies (other than libc). >> It won't provide rich features but basic functionality (record, report, >> stat) should be supported. And I'm also thinking about implementing a >> very simple elf parser for symbol loading in such a minimal build. > > Maybe you can install some static libraries (e.g. elfutils-libelf-devel-static > on fedora) for enabling it. :) > Some libraries may not work, but I could enable libelf by installing that. Yep, installing static libs should also work. But I'd like to have minimal depencency especially for embedded systems. And AFAIK cross building elfutils/libelf always has been a PITA. Thanks, Namhyung -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
Hi Masami, On Mon, 29 Sep 2014 20:40:51 +0900, Masami Hiramatsu wrote: (2014/09/26 23:46), Namhyung Kim wrote: The perf should be built with no extra dependencies (other than libc). It won't provide rich features but basic functionality (record, report, stat) should be supported. And I'm also thinking about implementing a very simple elf parser for symbol loading in such a minimal build. Maybe you can install some static libraries (e.g. elfutils-libelf-devel-static on fedora) for enabling it. :) Some libraries may not work, but I could enable libelf by installing that. Yep, installing static libs should also work. But I'd like to have minimal depencency especially for embedded systems. And AFAIK cross building elfutils/libelf always has been a PITA. Thanks, Namhyung -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re: Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
(2014/09/26 23:46), Namhyung Kim wrote: > Hi Masami, > > 2014-09-26 (금), 20:54 +0900, Masami Hiramatsu: >> (2014/09/26 15:06), Namhyung Kim wrote: >>> Hi Steve, >>> >>> On Wed, 24 Sep 2014 12:08:53 -0400, Steven Rostedt wrote: On Wed, 24 Sep 2014 10:01:38 -0600 Shuah Khan wrote: > I am working on adding the framework for c tests. We will have > to get this evolved for shell scripts. Please see the following > thread: > > https://lkml.org/lkml/2014/9/23/926 > Thanks, Also, what's the rules for a test that requires other tools. Some of my tests require trace-cmd to be installed as well as perf. My stress test runs perf against trace-cmd running hackbench (another required tool), but I believe that perf has hackbench built in so I could replace hackbench with the perf version. >>> >>> Yeah, and I think we can at least build perf (if not found) for test as >>> we can always have the source. The trace-cmd is different and it'd be >>> another reason I need to work on 'perf ftrace' command soon. ;-) >> >> +1 :) >> >> For the kselftest, we can only expect to have tools in kernel tree (which >> should be static-linked) and busybox, so that we can use qemu emulator to >> run kselftest on other archs. I guess perf still need some shared >> objects to run. That should be solved too. > > The perf should be built with no extra dependencies (other than libc). > It won't provide rich features but basic functionality (record, report, > stat) should be supported. And I'm also thinking about implementing a > very simple elf parser for symbol loading in such a minimal build. Maybe you can install some static libraries (e.g. elfutils-libelf-devel-static on fedora) for enabling it. :) Some libraries may not work, but I could enable libelf by installing that. Thank you, -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
(2014/09/26 23:18), Steven Rostedt wrote: > On Fri, 26 Sep 2014 21:04:26 +0900 > Masami Hiramatsu wrote: > > >>> Please go ahead and get them in. The framework might have to wait >>> until 3.19, so there is no need to have it gate test development. >>> We can evolve them as we go. If tests require special, it means that >>> they aren't suitable for certain envs such as qemu boot with busybox. >>> There is a plan to add test targets to cover such cases. >> >> Agreed, anyway, in some env those tools(commands) may not supported >> by busybox because of the memory size etc. In that case, we can just >> mark the test unsupported or xfail. >> > > Note, if memory size is such a big issue it that other tools may not be > there, then most of the ftrace features will probably be disabled due > to memory constraints on the kernel itself. Agreed :) > > As we have a way to state "unsupported" we can just return that from > tests that require tools not available, and say why it is the case. OK, if we can count "unsupported" as a test failure (yes, it is unexpectedly failed), it is easy to fix. I'll update XFAIL and other error code support patch and send it. Thank you, -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
(2014/09/26 23:18), Steven Rostedt wrote: On Fri, 26 Sep 2014 21:04:26 +0900 Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: Please go ahead and get them in. The framework might have to wait until 3.19, so there is no need to have it gate test development. We can evolve them as we go. If tests require special, it means that they aren't suitable for certain envs such as qemu boot with busybox. There is a plan to add test targets to cover such cases. Agreed, anyway, in some env those tools(commands) may not supported by busybox because of the memory size etc. In that case, we can just mark the test unsupported or xfail. Note, if memory size is such a big issue it that other tools may not be there, then most of the ftrace features will probably be disabled due to memory constraints on the kernel itself. Agreed :) As we have a way to state unsupported we can just return that from tests that require tools not available, and say why it is the case. OK, if we can count unsupported as a test failure (yes, it is unexpectedly failed), it is easy to fix. I'll update XFAIL and other error code support patch and send it. Thank you, -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re: Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
(2014/09/26 23:46), Namhyung Kim wrote: Hi Masami, 2014-09-26 (금), 20:54 +0900, Masami Hiramatsu: (2014/09/26 15:06), Namhyung Kim wrote: Hi Steve, On Wed, 24 Sep 2014 12:08:53 -0400, Steven Rostedt wrote: On Wed, 24 Sep 2014 10:01:38 -0600 Shuah Khan shua...@osg.samsung.com wrote: I am working on adding the framework for c tests. We will have to get this evolved for shell scripts. Please see the following thread: https://lkml.org/lkml/2014/9/23/926 Thanks, Also, what's the rules for a test that requires other tools. Some of my tests require trace-cmd to be installed as well as perf. My stress test runs perf against trace-cmd running hackbench (another required tool), but I believe that perf has hackbench built in so I could replace hackbench with the perf version. Yeah, and I think we can at least build perf (if not found) for test as we can always have the source. The trace-cmd is different and it'd be another reason I need to work on 'perf ftrace' command soon. ;-) +1 :) For the kselftest, we can only expect to have tools in kernel tree (which should be static-linked) and busybox, so that we can use qemu emulator to run kselftest on other archs. I guess perf still need some shared objects to run. That should be solved too. The perf should be built with no extra dependencies (other than libc). It won't provide rich features but basic functionality (record, report, stat) should be supported. And I'm also thinking about implementing a very simple elf parser for symbol loading in such a minimal build. Maybe you can install some static libraries (e.g. elfutils-libelf-devel-static on fedora) for enabling it. :) Some libraries may not work, but I could enable libelf by installing that. Thank you, -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
Hi Masami, 2014-09-26 (금), 20:54 +0900, Masami Hiramatsu: > (2014/09/26 15:06), Namhyung Kim wrote: > > Hi Steve, > > > > On Wed, 24 Sep 2014 12:08:53 -0400, Steven Rostedt wrote: > >> On Wed, 24 Sep 2014 10:01:38 -0600 > >> Shuah Khan wrote: > >> > >>> I am working on adding the framework for c tests. We will have > >>> to get this evolved for shell scripts. Please see the following > >>> thread: > >>> > >>> https://lkml.org/lkml/2014/9/23/926 > >>> > >> > >> Thanks, > >> > >> Also, what's the rules for a test that requires other tools. Some of my > >> tests require trace-cmd to be installed as well as perf. My stress test > >> runs perf against trace-cmd running hackbench (another required tool), > >> but I believe that perf has hackbench built in so I could replace > >> hackbench with the perf version. > > > > Yeah, and I think we can at least build perf (if not found) for test as > > we can always have the source. The trace-cmd is different and it'd be > > another reason I need to work on 'perf ftrace' command soon. ;-) > > +1 :) > > For the kselftest, we can only expect to have tools in kernel tree (which > should be static-linked) and busybox, so that we can use qemu emulator to > run kselftest on other archs. I guess perf still need some shared > objects to run. That should be solved too. The perf should be built with no extra dependencies (other than libc). It won't provide rich features but basic functionality (record, report, stat) should be supported. And I'm also thinking about implementing a very simple elf parser for symbol loading in such a minimal build. Thanks, Namhyung > > Anyway, at first we'd better add simpler tests which is not relay on > other tools except busybox shell. > > Thank you, > > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On Fri, 26 Sep 2014 21:04:26 +0900 Masami Hiramatsu wrote: > > Please go ahead and get them in. The framework might have to wait > > until 3.19, so there is no need to have it gate test development. > > We can evolve them as we go. If tests require special, it means that > > they aren't suitable for certain envs such as qemu boot with busybox. > > There is a plan to add test targets to cover such cases. > > Agreed, anyway, in some env those tools(commands) may not supported > by busybox because of the memory size etc. In that case, we can just > mark the test unsupported or xfail. > Note, if memory size is such a big issue it that other tools may not be there, then most of the ftrace features will probably be disabled due to memory constraints on the kernel itself. As we have a way to state "unsupported" we can just return that from tests that require tools not available, and say why it is the case. -- Steve -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On Fri, 26 Sep 2014 15:06:51 +0900 Namhyung Kim wrote: > Yeah, and I think we can at least build perf (if not found) for test as > we can always have the source. The trace-cmd is different and it'd be > another reason I need to work on 'perf ftrace' command soon. ;-) Well, if a tool is not found. I wouldn't recommend building it. Either case, I'll hold off test that is required. Hmm, maybe I'll post all my tests on my website, for others to play with. That way I'm not limited in tests that take too much time or use tools not supported by the kernel selftest infrastructure. -- Steve -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
(2014/09/25 1:36), Shuah Khan wrote: > On 09/24/2014 10:30 AM, Steven Rostedt wrote: >> On Wed, 24 Sep 2014 10:22:12 -0600 >> Shuah Khan wrote: >> >> >>> One simple approach is to have the user install the required tools >>> for the test. Indicate what is needed and fail the test with >>> appropriate message with xskip. This is what I have in mind for >>> tests that can't be run because of a missing config. The same can >>> be extended to external tools that the test depends on. >> >> Should I hold off adding my tests until we have all this in place. Or >> should I just push my tests with simply passing if the config isn't >> supported for now and add it later? >> >> I wont add any of the tests that require other tools besides echo, grep >> and cat. They are supported by busybox :) > Please go ahead and get them in. The framework might have to wait > until 3.19, so there is no need to have it gate test development. > We can evolve them as we go. If tests require special, it means that > they aren't suitable for certain envs such as qemu boot with busybox. > There is a plan to add test targets to cover such cases. Agreed, anyway, in some env those tools(commands) may not supported by busybox because of the memory size etc. In that case, we can just mark the test unsupported or xfail. Thank you, > I would recommend at least printing a message saying what you are > missing if you find something missing. > > thanks, > -- Shuah > > -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
(2014/09/26 15:06), Namhyung Kim wrote: > Hi Steve, > > On Wed, 24 Sep 2014 12:08:53 -0400, Steven Rostedt wrote: >> On Wed, 24 Sep 2014 10:01:38 -0600 >> Shuah Khan wrote: >> >>> I am working on adding the framework for c tests. We will have >>> to get this evolved for shell scripts. Please see the following >>> thread: >>> >>> https://lkml.org/lkml/2014/9/23/926 >>> >> >> Thanks, >> >> Also, what's the rules for a test that requires other tools. Some of my >> tests require trace-cmd to be installed as well as perf. My stress test >> runs perf against trace-cmd running hackbench (another required tool), >> but I believe that perf has hackbench built in so I could replace >> hackbench with the perf version. > > Yeah, and I think we can at least build perf (if not found) for test as > we can always have the source. The trace-cmd is different and it'd be > another reason I need to work on 'perf ftrace' command soon. ;-) +1 :) For the kselftest, we can only expect to have tools in kernel tree (which should be static-linked) and busybox, so that we can use qemu emulator to run kselftest on other archs. I guess perf still need some shared objects to run. That should be solved too. Anyway, at first we'd better add simpler tests which is not relay on other tools except busybox shell. Thank you, -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
Hi Steve, On Wed, 24 Sep 2014 12:08:53 -0400, Steven Rostedt wrote: > On Wed, 24 Sep 2014 10:01:38 -0600 > Shuah Khan wrote: > >> I am working on adding the framework for c tests. We will have >> to get this evolved for shell scripts. Please see the following >> thread: >> >> https://lkml.org/lkml/2014/9/23/926 >> > > Thanks, > > Also, what's the rules for a test that requires other tools. Some of my > tests require trace-cmd to be installed as well as perf. My stress test > runs perf against trace-cmd running hackbench (another required tool), > but I believe that perf has hackbench built in so I could replace > hackbench with the perf version. Yeah, and I think we can at least build perf (if not found) for test as we can always have the source. The trace-cmd is different and it'd be another reason I need to work on 'perf ftrace' command soon. ;-) Thanks, Namhyung -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
Hi Steve, On Wed, 24 Sep 2014 12:08:53 -0400, Steven Rostedt wrote: On Wed, 24 Sep 2014 10:01:38 -0600 Shuah Khan shua...@osg.samsung.com wrote: I am working on adding the framework for c tests. We will have to get this evolved for shell scripts. Please see the following thread: https://lkml.org/lkml/2014/9/23/926 Thanks, Also, what's the rules for a test that requires other tools. Some of my tests require trace-cmd to be installed as well as perf. My stress test runs perf against trace-cmd running hackbench (another required tool), but I believe that perf has hackbench built in so I could replace hackbench with the perf version. Yeah, and I think we can at least build perf (if not found) for test as we can always have the source. The trace-cmd is different and it'd be another reason I need to work on 'perf ftrace' command soon. ;-) Thanks, Namhyung -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
(2014/09/26 15:06), Namhyung Kim wrote: Hi Steve, On Wed, 24 Sep 2014 12:08:53 -0400, Steven Rostedt wrote: On Wed, 24 Sep 2014 10:01:38 -0600 Shuah Khan shua...@osg.samsung.com wrote: I am working on adding the framework for c tests. We will have to get this evolved for shell scripts. Please see the following thread: https://lkml.org/lkml/2014/9/23/926 Thanks, Also, what's the rules for a test that requires other tools. Some of my tests require trace-cmd to be installed as well as perf. My stress test runs perf against trace-cmd running hackbench (another required tool), but I believe that perf has hackbench built in so I could replace hackbench with the perf version. Yeah, and I think we can at least build perf (if not found) for test as we can always have the source. The trace-cmd is different and it'd be another reason I need to work on 'perf ftrace' command soon. ;-) +1 :) For the kselftest, we can only expect to have tools in kernel tree (which should be static-linked) and busybox, so that we can use qemu emulator to run kselftest on other archs. I guess perf still need some shared objects to run. That should be solved too. Anyway, at first we'd better add simpler tests which is not relay on other tools except busybox shell. Thank you, -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
(2014/09/25 1:36), Shuah Khan wrote: On 09/24/2014 10:30 AM, Steven Rostedt wrote: On Wed, 24 Sep 2014 10:22:12 -0600 Shuah Khan shua...@osg.samsung.com wrote: One simple approach is to have the user install the required tools for the test. Indicate what is needed and fail the test with appropriate message with xskip. This is what I have in mind for tests that can't be run because of a missing config. The same can be extended to external tools that the test depends on. Should I hold off adding my tests until we have all this in place. Or should I just push my tests with simply passing if the config isn't supported for now and add it later? I wont add any of the tests that require other tools besides echo, grep and cat. They are supported by busybox :) Please go ahead and get them in. The framework might have to wait until 3.19, so there is no need to have it gate test development. We can evolve them as we go. If tests require special, it means that they aren't suitable for certain envs such as qemu boot with busybox. There is a plan to add test targets to cover such cases. Agreed, anyway, in some env those tools(commands) may not supported by busybox because of the memory size etc. In that case, we can just mark the test unsupported or xfail. Thank you, I would recommend at least printing a message saying what you are missing if you find something missing. thanks, -- Shuah -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On Fri, 26 Sep 2014 15:06:51 +0900 Namhyung Kim namhy...@kernel.org wrote: Yeah, and I think we can at least build perf (if not found) for test as we can always have the source. The trace-cmd is different and it'd be another reason I need to work on 'perf ftrace' command soon. ;-) Well, if a tool is not found. I wouldn't recommend building it. Either case, I'll hold off test that is required. Hmm, maybe I'll post all my tests on my website, for others to play with. That way I'm not limited in tests that take too much time or use tools not supported by the kernel selftest infrastructure. -- Steve -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On Fri, 26 Sep 2014 21:04:26 +0900 Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: Please go ahead and get them in. The framework might have to wait until 3.19, so there is no need to have it gate test development. We can evolve them as we go. If tests require special, it means that they aren't suitable for certain envs such as qemu boot with busybox. There is a plan to add test targets to cover such cases. Agreed, anyway, in some env those tools(commands) may not supported by busybox because of the memory size etc. In that case, we can just mark the test unsupported or xfail. Note, if memory size is such a big issue it that other tools may not be there, then most of the ftrace features will probably be disabled due to memory constraints on the kernel itself. As we have a way to state unsupported we can just return that from tests that require tools not available, and say why it is the case. -- Steve -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
Hi Masami, 2014-09-26 (금), 20:54 +0900, Masami Hiramatsu: (2014/09/26 15:06), Namhyung Kim wrote: Hi Steve, On Wed, 24 Sep 2014 12:08:53 -0400, Steven Rostedt wrote: On Wed, 24 Sep 2014 10:01:38 -0600 Shuah Khan shua...@osg.samsung.com wrote: I am working on adding the framework for c tests. We will have to get this evolved for shell scripts. Please see the following thread: https://lkml.org/lkml/2014/9/23/926 Thanks, Also, what's the rules for a test that requires other tools. Some of my tests require trace-cmd to be installed as well as perf. My stress test runs perf against trace-cmd running hackbench (another required tool), but I believe that perf has hackbench built in so I could replace hackbench with the perf version. Yeah, and I think we can at least build perf (if not found) for test as we can always have the source. The trace-cmd is different and it'd be another reason I need to work on 'perf ftrace' command soon. ;-) +1 :) For the kselftest, we can only expect to have tools in kernel tree (which should be static-linked) and busybox, so that we can use qemu emulator to run kselftest on other archs. I guess perf still need some shared objects to run. That should be solved too. The perf should be built with no extra dependencies (other than libc). It won't provide rich features but basic functionality (record, report, stat) should be supported. And I'm also thinking about implementing a very simple elf parser for symbol loading in such a minimal build. Thanks, Namhyung Anyway, at first we'd better add simpler tests which is not relay on other tools except busybox shell. Thank you, -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
(2014/09/25 10:14), Steven Rostedt wrote: > On Thu, 25 Sep 2014 09:35:05 +0900 > Masami Hiramatsu wrote: > >> (2014/09/25 0:58), Steven Rostedt wrote: >>> On Wed, 24 Sep 2014 11:42:35 -0400 >>> Steven Rostedt wrote: >>> > I think you'd better use exit_unsupported/exit_xfail to notify > that the test target is not configured, or expected to fail. >>> >>> Are these functions defined anywhere? I don't seem them in your v5 >>> patch set, or in selftests anywhere. >> >> Hmm, could you check my 4/4 patch in the series?? >> http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg730578.html > > Hmm, I somehow missed this. Can you check to make sure my for-next > patches are your latest version. I'll pull in 4/4 tomorrow. Yeah, I ensured that your for-next patches are the v5. You will be able to cleanly apply 4/4 on it :) Thank you, > > Thanks, > > -- Steve > -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
(2014/09/25 0:42), Steven Rostedt wrote: > On Wed, 24 Sep 2014 11:58:50 +0900 > Masami Hiramatsu wrote: > >> (2014/09/24 6:38), Steven Rostedt wrote: > >>> to them and copied them pretty much unchanged into a ftrace directory >>> under test.d. Is this fine, or is there more massaging I need to do >>> to them? >> >> Yeah, ftrace has a double meaning :), so test.d/ftrace is fine to me. > > I was thinking that more complex tests can go into ftrace, and simple > tests can go into basic. Hmm, what kind of "complex" test will be there? I think unit tests are usually simple, selftests/ftrace/test.d/ftrace/ looks redundant if the first ftrace and the second one has same meaning. >>> I know the echos don't show up, but I kept them anyway. What should >>> happen with them? >> >> I think you'd better use exit_unsupported/exit_xfail to notify >> that the test target is not configured, or expected to fail. > > OK, that answers the exit codes (as you also stated below), but what > about the echos themselves? Again, ftracetest is for unit tests, exit code will return one concrete result, but we can anything with echo (including typo) > >> Then the user can reconfigure that. Maybe we should keep the >> detailed log of such results. (you can do it with --keep option) > > You mean keep the echos, as they are ignored anyway, but if we add > --keep, then the echos will show? No, --keep keeps all output logs of the test script. Since all scripts run under set -x, all commands and outputs are logged to a logfile. > Maybe that option should be -v, like > other tools use for "verbose". OK, it will be easy, just turning "> $LOGFILE" into "| tee $LOGFILE" :) >>> --- /dev/null >>> +++ b/tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc >>> @@ -0,0 +1,111 @@ >>> +#!/bin/sh >>> +# description: ftrace - function graph filters >>> + >>> +# Make sure that function graph filtering works, and is not >>> +# affected by other tracers enabled (like stack tracer) >>> + >>> +if ! grep -q function_graph available_tracers; then >>> +echo "no function graph tracer configured" >>> +exit 0; >> >> This should call exit_unsupported, because the test is not passed. > > Will update this and others. > > >>> +# Make sure we did find something >>> +count=`cat trace | grep 'schedule()' | wc -l` >>> +if [ $count -eq 0 ]; then >>> +echo "No schedule traces found?" >>> +exit -1 >>> +fi >>> + >>> +echo "Graph filtering works by itself" >>> + >> >> I think the following part should be a separated test for >> stack trace. > > Good point. A lot of my test scripts do multiple tests. I think for > putting them into tools/testing/selftests/ftrace, I'll break them up > and make them separate tests. For example, the simple graph filtering > above is an example of something that can go into the basic directory, > but the test against stack tracer should be in ftrace. What do you > think? Hm, I think only really basic operation tests go into the basic directory, like reading and writing (no format checks). And test.d/ftrace may be for more advanced tests, like checking output, setting options, etc. >>> + >>> +echo "Now filter on just schedule" >>> +echo '*schedule' > set_ftrace_filter >>> +> trace >> >> echo > trace? > > OK. Hmm, I wonder if we should make a bunch of functions that the tests > can use. Like a "clear_trace" call that does this. Can we export > functions that this shell will be able to use? Yeah, it's a good idea :) We can use "." to include it. I think test.d/functions will be good. > When we see lots of > duplicate code we may want to have something like that. Well, for this, > it may not be that big of a deal, because "echo > trace" is rather > simple. But I do have other operations that are a bit more intrusive. Agreed. Thank you! -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On Thu, 25 Sep 2014 09:35:05 +0900 Masami Hiramatsu wrote: > (2014/09/25 0:58), Steven Rostedt wrote: > > On Wed, 24 Sep 2014 11:42:35 -0400 > > Steven Rostedt wrote: > > > >>> I think you'd better use exit_unsupported/exit_xfail to notify > >>> that the test target is not configured, or expected to fail. > >> > > > > Are these functions defined anywhere? I don't seem them in your v5 > > patch set, or in selftests anywhere. > > Hmm, could you check my 4/4 patch in the series?? > http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg730578.html Hmm, I somehow missed this. Can you check to make sure my for-next patches are your latest version. I'll pull in 4/4 tomorrow. Thanks, -- Steve -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
(2014/09/25 0:58), Steven Rostedt wrote: > On Wed, 24 Sep 2014 11:42:35 -0400 > Steven Rostedt wrote: > >>> I think you'd better use exit_unsupported/exit_xfail to notify >>> that the test target is not configured, or expected to fail. >> > > Are these functions defined anywhere? I don't seem them in your v5 > patch set, or in selftests anywhere. Hmm, could you check my 4/4 patch in the series?? http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg730578.html Thanks, -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On 09/24/2014 10:30 AM, Steven Rostedt wrote: > On Wed, 24 Sep 2014 10:22:12 -0600 > Shuah Khan wrote: > > >> One simple approach is to have the user install the required tools >> for the test. Indicate what is needed and fail the test with >> appropriate message with xskip. This is what I have in mind for >> tests that can't be run because of a missing config. The same can >> be extended to external tools that the test depends on. > > Should I hold off adding my tests until we have all this in place. Or > should I just push my tests with simply passing if the config isn't > supported for now and add it later? > > I wont add any of the tests that require other tools besides echo, grep > and cat. > Please go ahead and get them in. The framework might have to wait until 3.19, so there is no need to have it gate test development. We can evolve them as we go. If tests require special, it means that they aren't suitable for certain envs such as qemu boot with busybox. There is a plan to add test targets to cover such cases. I would recommend at least printing a message saying what you are missing if you find something missing. thanks, -- Shuah -- Shuah Khan Sr. Linux Kernel Developer Samsung Research America (Silicon Valley) shua...@osg.samsung.com | (970) 217-8978 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On Wed, 24 Sep 2014 10:22:12 -0600 Shuah Khan wrote: > One simple approach is to have the user install the required tools > for the test. Indicate what is needed and fail the test with > appropriate message with xskip. This is what I have in mind for > tests that can't be run because of a missing config. The same can > be extended to external tools that the test depends on. Should I hold off adding my tests until we have all this in place. Or should I just push my tests with simply passing if the config isn't supported for now and add it later? I wont add any of the tests that require other tools besides echo, grep and cat. -- Steve -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On 09/24/2014 10:08 AM, Steven Rostedt wrote: > On Wed, 24 Sep 2014 10:01:38 -0600 > Shuah Khan wrote: > >> I am working on adding the framework for c tests. We will have >> to get this evolved for shell scripts. Please see the following >> thread: >> >> https://lkml.org/lkml/2014/9/23/926 >> > > Thanks, > > Also, what's the rules for a test that requires other tools. Some of my > tests require trace-cmd to be installed as well as perf. My stress test > runs perf against trace-cmd running hackbench (another required tool), > but I believe that perf has hackbench built in so I could replace > hackbench with the perf version. > One simple approach is to have the user install the required tools for the test. Indicate what is needed and fail the test with appropriate message with xskip. This is what I have in mind for tests that can't be run because of a missing config. The same can be extended to external tools that the test depends on. thanks, -- Shuah -- Shuah Khan Sr. Linux Kernel Developer Samsung Research America (Silicon Valley) shua...@osg.samsung.com | (970) 217-8978 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On Wed, 24 Sep 2014 10:01:38 -0600 Shuah Khan wrote: > I am working on adding the framework for c tests. We will have > to get this evolved for shell scripts. Please see the following > thread: > > https://lkml.org/lkml/2014/9/23/926 > Thanks, Also, what's the rules for a test that requires other tools. Some of my tests require trace-cmd to be installed as well as perf. My stress test runs perf against trace-cmd running hackbench (another required tool), but I believe that perf has hackbench built in so I could replace hackbench with the perf version. -- Steve -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On 09/24/2014 09:58 AM, Steven Rostedt wrote: > On Wed, 24 Sep 2014 11:42:35 -0400 > Steven Rostedt wrote: > >>> I think you'd better use exit_unsupported/exit_xfail to notify >>> that the test target is not configured, or expected to fail. >> > > Are these functions defined anywhere? I don't seem them in your v5 > patch set, or in selftests anywhere. > > -- Steve > Steve, I am working on adding the framework for c tests. We will have to get this evolved for shell scripts. Please see the following thread: https://lkml.org/lkml/2014/9/23/926 thanks, -- Shuah -- Shuah Khan Sr. Linux Kernel Developer Samsung Research America (Silicon Valley) shua...@osg.samsung.com | (970) 217-8978 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On Wed, 24 Sep 2014 11:42:35 -0400 Steven Rostedt wrote: > > I think you'd better use exit_unsupported/exit_xfail to notify > > that the test target is not configured, or expected to fail. > Are these functions defined anywhere? I don't seem them in your v5 patch set, or in selftests anywhere. -- Steve -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On Wed, 24 Sep 2014 11:58:50 +0900 Masami Hiramatsu wrote: > (2014/09/24 6:38), Steven Rostedt wrote: > > to them and copied them pretty much unchanged into a ftrace directory > > under test.d. Is this fine, or is there more massaging I need to do > > to them? > > Yeah, ftrace has a double meaning :), so test.d/ftrace is fine to me. I was thinking that more complex tests can go into ftrace, and simple tests can go into basic. > > > > > I know the echos don't show up, but I kept them anyway. What should > > happen with them? > > I think you'd better use exit_unsupported/exit_xfail to notify > that the test target is not configured, or expected to fail. OK, that answers the exit codes (as you also stated below), but what about the echos themselves? > Then the user can reconfigure that. Maybe we should keep the > detailed log of such results. (you can do it with --keep option) You mean keep the echos, as they are ignored anyway, but if we add --keep, then the echos will show? Maybe that option should be -v, like other tools use for "verbose". > > --- /dev/null > > +++ b/tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc > > @@ -0,0 +1,111 @@ > > +#!/bin/sh > > +# description: ftrace - function graph filters > > + > > +# Make sure that function graph filtering works, and is not > > +# affected by other tracers enabled (like stack tracer) > > + > > +if ! grep -q function_graph available_tracers; then > > +echo "no function graph tracer configured" > > +exit 0; > > This should call exit_unsupported, because the test is not passed. Will update this and others. > > +# Make sure we did find something > > +count=`cat trace | grep 'schedule()' | wc -l` > > +if [ $count -eq 0 ]; then > > +echo "No schedule traces found?" > > +exit -1 > > +fi > > + > > +echo "Graph filtering works by itself" > > + > > I think the following part should be a separated test for > stack trace. Good point. A lot of my test scripts do multiple tests. I think for putting them into tools/testing/selftests/ftrace, I'll break them up and make them separate tests. For example, the simple graph filtering above is an example of something that can go into the basic directory, but the test against stack tracer should be in ftrace. What do you think? > > + > > +echo "Now filter on just schedule" > > +echo '*schedule' > set_ftrace_filter > > +> trace > > echo > trace? OK. Hmm, I wonder if we should make a bunch of functions that the tests can use. Like a "clear_trace" call that does this. Can we export functions that this shell will be able to use? When we see lots of duplicate code we may want to have something like that. Well, for this, it may not be that big of a deal, because "echo > trace" is rather simple. But I do have other operations that are a bit more intrusive. -- Steve -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On Wed, 24 Sep 2014 11:58:50 +0900 Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: (2014/09/24 6:38), Steven Rostedt wrote: to them and copied them pretty much unchanged into a ftrace directory under test.d. Is this fine, or is there more massaging I need to do to them? Yeah, ftrace has a double meaning :), so test.d/ftrace is fine to me. I was thinking that more complex tests can go into ftrace, and simple tests can go into basic. I know the echos don't show up, but I kept them anyway. What should happen with them? I think you'd better use exit_unsupported/exit_xfail to notify that the test target is not configured, or expected to fail. OK, that answers the exit codes (as you also stated below), but what about the echos themselves? Then the user can reconfigure that. Maybe we should keep the detailed log of such results. (you can do it with --keep option) You mean keep the echos, as they are ignored anyway, but if we add --keep, then the echos will show? Maybe that option should be -v, like other tools use for verbose. --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc @@ -0,0 +1,111 @@ +#!/bin/sh +# description: ftrace - function graph filters + +# Make sure that function graph filtering works, and is not +# affected by other tracers enabled (like stack tracer) + +if ! grep -q function_graph available_tracers; then +echo no function graph tracer configured +exit 0; This should call exit_unsupported, because the test is not passed. Will update this and others. +# Make sure we did find something +count=`cat trace | grep 'schedule()' | wc -l` +if [ $count -eq 0 ]; then +echo No schedule traces found? +exit -1 +fi + +echo Graph filtering works by itself + I think the following part should be a separated test for stack trace. Good point. A lot of my test scripts do multiple tests. I think for putting them into tools/testing/selftests/ftrace, I'll break them up and make them separate tests. For example, the simple graph filtering above is an example of something that can go into the basic directory, but the test against stack tracer should be in ftrace. What do you think? + +echo Now filter on just schedule +echo '*schedule' set_ftrace_filter + trace echo trace? OK. Hmm, I wonder if we should make a bunch of functions that the tests can use. Like a clear_trace call that does this. Can we export functions that this shell will be able to use? When we see lots of duplicate code we may want to have something like that. Well, for this, it may not be that big of a deal, because echo trace is rather simple. But I do have other operations that are a bit more intrusive. -- Steve -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On Wed, 24 Sep 2014 11:42:35 -0400 Steven Rostedt rost...@goodmis.org wrote: I think you'd better use exit_unsupported/exit_xfail to notify that the test target is not configured, or expected to fail. Are these functions defined anywhere? I don't seem them in your v5 patch set, or in selftests anywhere. -- Steve -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On 09/24/2014 09:58 AM, Steven Rostedt wrote: On Wed, 24 Sep 2014 11:42:35 -0400 Steven Rostedt rost...@goodmis.org wrote: I think you'd better use exit_unsupported/exit_xfail to notify that the test target is not configured, or expected to fail. Are these functions defined anywhere? I don't seem them in your v5 patch set, or in selftests anywhere. -- Steve Steve, I am working on adding the framework for c tests. We will have to get this evolved for shell scripts. Please see the following thread: https://lkml.org/lkml/2014/9/23/926 thanks, -- Shuah -- Shuah Khan Sr. Linux Kernel Developer Samsung Research America (Silicon Valley) shua...@osg.samsung.com | (970) 217-8978 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On Wed, 24 Sep 2014 10:01:38 -0600 Shuah Khan shua...@osg.samsung.com wrote: I am working on adding the framework for c tests. We will have to get this evolved for shell scripts. Please see the following thread: https://lkml.org/lkml/2014/9/23/926 Thanks, Also, what's the rules for a test that requires other tools. Some of my tests require trace-cmd to be installed as well as perf. My stress test runs perf against trace-cmd running hackbench (another required tool), but I believe that perf has hackbench built in so I could replace hackbench with the perf version. -- Steve -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On 09/24/2014 10:08 AM, Steven Rostedt wrote: On Wed, 24 Sep 2014 10:01:38 -0600 Shuah Khan shua...@osg.samsung.com wrote: I am working on adding the framework for c tests. We will have to get this evolved for shell scripts. Please see the following thread: https://lkml.org/lkml/2014/9/23/926 Thanks, Also, what's the rules for a test that requires other tools. Some of my tests require trace-cmd to be installed as well as perf. My stress test runs perf against trace-cmd running hackbench (another required tool), but I believe that perf has hackbench built in so I could replace hackbench with the perf version. One simple approach is to have the user install the required tools for the test. Indicate what is needed and fail the test with appropriate message with xskip. This is what I have in mind for tests that can't be run because of a missing config. The same can be extended to external tools that the test depends on. thanks, -- Shuah -- Shuah Khan Sr. Linux Kernel Developer Samsung Research America (Silicon Valley) shua...@osg.samsung.com | (970) 217-8978 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On Wed, 24 Sep 2014 10:22:12 -0600 Shuah Khan shua...@osg.samsung.com wrote: One simple approach is to have the user install the required tools for the test. Indicate what is needed and fail the test with appropriate message with xskip. This is what I have in mind for tests that can't be run because of a missing config. The same can be extended to external tools that the test depends on. Should I hold off adding my tests until we have all this in place. Or should I just push my tests with simply passing if the config isn't supported for now and add it later? I wont add any of the tests that require other tools besides echo, grep and cat. -- Steve -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On 09/24/2014 10:30 AM, Steven Rostedt wrote: On Wed, 24 Sep 2014 10:22:12 -0600 Shuah Khan shua...@osg.samsung.com wrote: One simple approach is to have the user install the required tools for the test. Indicate what is needed and fail the test with appropriate message with xskip. This is what I have in mind for tests that can't be run because of a missing config. The same can be extended to external tools that the test depends on. Should I hold off adding my tests until we have all this in place. Or should I just push my tests with simply passing if the config isn't supported for now and add it later? I wont add any of the tests that require other tools besides echo, grep and cat. Please go ahead and get them in. The framework might have to wait until 3.19, so there is no need to have it gate test development. We can evolve them as we go. If tests require special, it means that they aren't suitable for certain envs such as qemu boot with busybox. There is a plan to add test targets to cover such cases. I would recommend at least printing a message saying what you are missing if you find something missing. thanks, -- Shuah -- Shuah Khan Sr. Linux Kernel Developer Samsung Research America (Silicon Valley) shua...@osg.samsung.com | (970) 217-8978 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
(2014/09/25 0:58), Steven Rostedt wrote: On Wed, 24 Sep 2014 11:42:35 -0400 Steven Rostedt rost...@goodmis.org wrote: I think you'd better use exit_unsupported/exit_xfail to notify that the test target is not configured, or expected to fail. Are these functions defined anywhere? I don't seem them in your v5 patch set, or in selftests anywhere. Hmm, could you check my 4/4 patch in the series?? http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg730578.html Thanks, -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
On Thu, 25 Sep 2014 09:35:05 +0900 Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: (2014/09/25 0:58), Steven Rostedt wrote: On Wed, 24 Sep 2014 11:42:35 -0400 Steven Rostedt rost...@goodmis.org wrote: I think you'd better use exit_unsupported/exit_xfail to notify that the test target is not configured, or expected to fail. Are these functions defined anywhere? I don't seem them in your v5 patch set, or in selftests anywhere. Hmm, could you check my 4/4 patch in the series?? http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg730578.html Hmm, I somehow missed this. Can you check to make sure my for-next patches are your latest version. I'll pull in 4/4 tomorrow. Thanks, -- Steve -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
(2014/09/25 0:42), Steven Rostedt wrote: On Wed, 24 Sep 2014 11:58:50 +0900 Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: (2014/09/24 6:38), Steven Rostedt wrote: to them and copied them pretty much unchanged into a ftrace directory under test.d. Is this fine, or is there more massaging I need to do to them? Yeah, ftrace has a double meaning :), so test.d/ftrace is fine to me. I was thinking that more complex tests can go into ftrace, and simple tests can go into basic. Hmm, what kind of complex test will be there? I think unit tests are usually simple, selftests/ftrace/test.d/ftrace/ looks redundant if the first ftrace and the second one has same meaning. I know the echos don't show up, but I kept them anyway. What should happen with them? I think you'd better use exit_unsupported/exit_xfail to notify that the test target is not configured, or expected to fail. OK, that answers the exit codes (as you also stated below), but what about the echos themselves? Again, ftracetest is for unit tests, exit code will return one concrete result, but we can anything with echo (including typo) Then the user can reconfigure that. Maybe we should keep the detailed log of such results. (you can do it with --keep option) You mean keep the echos, as they are ignored anyway, but if we add --keep, then the echos will show? No, --keep keeps all output logs of the test script. Since all scripts run under set -x, all commands and outputs are logged to a logfile. Maybe that option should be -v, like other tools use for verbose. OK, it will be easy, just turning $LOGFILE into | tee $LOGFILE :) --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc @@ -0,0 +1,111 @@ +#!/bin/sh +# description: ftrace - function graph filters + +# Make sure that function graph filtering works, and is not +# affected by other tracers enabled (like stack tracer) + +if ! grep -q function_graph available_tracers; then +echo no function graph tracer configured +exit 0; This should call exit_unsupported, because the test is not passed. Will update this and others. +# Make sure we did find something +count=`cat trace | grep 'schedule()' | wc -l` +if [ $count -eq 0 ]; then +echo No schedule traces found? +exit -1 +fi + +echo Graph filtering works by itself + I think the following part should be a separated test for stack trace. Good point. A lot of my test scripts do multiple tests. I think for putting them into tools/testing/selftests/ftrace, I'll break them up and make them separate tests. For example, the simple graph filtering above is an example of something that can go into the basic directory, but the test against stack tracer should be in ftrace. What do you think? Hm, I think only really basic operation tests go into the basic directory, like reading and writing (no format checks). And test.d/ftrace may be for more advanced tests, like checking output, setting options, etc. + +echo Now filter on just schedule +echo '*schedule' set_ftrace_filter + trace echo trace? OK. Hmm, I wonder if we should make a bunch of functions that the tests can use. Like a clear_trace call that does this. Can we export functions that this shell will be able to use? Yeah, it's a good idea :) We can use . to include it. I think test.d/functions will be good. When we see lots of duplicate code we may want to have something like that. Well, for this, it may not be that big of a deal, because echo trace is rather simple. But I do have other operations that are a bit more intrusive. Agreed. Thank you! -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
(2014/09/25 10:14), Steven Rostedt wrote: On Thu, 25 Sep 2014 09:35:05 +0900 Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: (2014/09/25 0:58), Steven Rostedt wrote: On Wed, 24 Sep 2014 11:42:35 -0400 Steven Rostedt rost...@goodmis.org wrote: I think you'd better use exit_unsupported/exit_xfail to notify that the test target is not configured, or expected to fail. Are these functions defined anywhere? I don't seem them in your v5 patch set, or in selftests anywhere. Hmm, could you check my 4/4 patch in the series?? http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg730578.html Hmm, I somehow missed this. Can you check to make sure my for-next patches are your latest version. I'll pull in 4/4 tomorrow. Yeah, I ensured that your for-next patches are the v5. You will be able to cleanly apply 4/4 on it :) Thank you, Thanks, -- Steve -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
(2014/09/24 6:38), Steven Rostedt wrote:
>
> [
> Masami, I took two of my test scripts and added some basic comments
Thanks!
> to them and copied them pretty much unchanged into a ftrace directory
> under test.d. Is this fine, or is there more massaging I need to do
> to them?
Yeah, ftrace has a double meaning :), so test.d/ftrace is fine to me.
>
> I know the echos don't show up, but I kept them anyway. What should
> happen with them?
I think you'd better use exit_unsupported/exit_xfail to notify
that the test target is not configured, or expected to fail.
Then the user can reconfigure that. Maybe we should keep the
detailed log of such results. (you can do it with --keep option)
> ]
>
> Added two test cases to get the feel of adding tests to ftracetest.
> The two cases are:
>
> function profiling test, to make sure function profiling still works
>with function tracing (was a regression)
>
> function graph filter test to make sure that the function graph filter
>does filter and also continues to filter when another function tracer
>is running (like the stack tracer)
>
> Signed-off-by: Steven Rostedt
> ---
> .../ftrace/test.d/ftrace/fgraph-filter.tc | 111
> +
> .../ftrace/test.d/ftrace/func_profiler.tc | 78 +++
> 2 files changed, 189 insertions(+)
> create mode 100644
> tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc
> create mode 100644
> tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tc
>
> diff --git a/tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc
> b/tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc
> new file mode 100644
> index ..42f764d1f8d2
> --- /dev/null
> +++ b/tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc
> @@ -0,0 +1,111 @@
> +#!/bin/sh
> +# description: ftrace - function graph filters
> +
> +# Make sure that function graph filtering works, and is not
> +# affected by other tracers enabled (like stack tracer)
> +
> +if ! grep -q function_graph available_tracers; then
> +echo "no function graph tracer configured"
> +exit 0;
This should call exit_unsupported, because the test is not passed.
> +fi
> +
> +if [ ! -f set_ftrace_filter ]; then
> +echo "set_ftrace_filter not found? Is dynamic ftrace not set?"
> +exit 0
Ditto.
> +fi
> +
> +do_reset() {
> +echo nop > current_tracer
> +echo 0 > /proc/sys/kernel/stack_tracer_enabled
> +echo 1 > tracing_on
> +echo > trace
> +echo > set_ftrace_filter
> +}
> +
> +echo 0 > tracing_on
> +# clear trace
> +echo > trace
> +# filter something, schedule is always good
> +if ! echo "schedule" > set_ftrace_filter; then
> +# test for powerpc 64
> +if ! echo ".schedule" > set_ftrace_filter; then
> + echo "can not enable schedule filter"
> + exit -1
> +fi
> +fi
> +
> +echo function_graph > current_tracer
> +echo 1 > tracing_on
> +sleep 1
> +# search for functions (has "()" on the line), and make sure
> +# that only the schedule function was found
> +count=`cat trace | grep '()' | grep -v schedule | wc -l`
> +if [ $count -ne 0 ]; then
> +echo "Graph filtering not working by itself?"
> +exit -1;
> +fi
> +
> +# Make sure we did find something
> +count=`cat trace | grep 'schedule()' | wc -l`
> +if [ $count -eq 0 ]; then
> +echo "No schedule traces found?"
> +exit -1
> +fi
> +
> +echo "Graph filtering works by itself"
> +
I think the following part should be a separated test for
stack trace.
> +if [ ! -f stack_trace ]; then
> +echo "Stack tracer not configured, can't continue test"
> +# stack tracer not configured in this kernel? pass test anyway
> +do_reset
> +exit 0;
Here should call exit_unsupported.
> +fi
> +
> +echo "Now testing with stack tracer"
> +
> +echo 1 > /proc/sys/kernel/stack_tracer_enabled
> +
> +echo 0 > tracing_on
> +echo > trace
> +echo 1 > tracing_on
> +sleep 1
> +
> +count=`cat trace | grep '()' | grep -v schedule | wc -l`
> +
> +if [ $count -ne 0 ]; then
> +echo "Graph filtering not working with stack tracer?"
> +exit -1
> +fi
> +
> +count=`cat trace | grep 'schedule()' | wc -l`
> +if [ $count -eq 0 ]; then
> +echo "No schedule traces found?"
> +exit -1
> +fi
> +
> +echo "Graph filtering works with stack tracer"
> +
> +echo "Now testing if filtering still works without stack tracer"
> +
> +echo 0 > /proc/sys/kernel/stack_tracer_enabled
> +echo > trace
> +sleep 1
> +
> +
> +count=`cat trace | grep '()' | grep -v schedule | wc -l`
> +
> +if [ $count -ne 0 ]; then
> +echo "Graph filtering not working after stack tracer disabled?"
> +exit -1
> +fi
> +
> +count=`cat trace | grep 'schedule()' | wc -l`
> +if [ $count -eq 0 ]; then
> +echo "No schedule traces found?"
> +exit -1
> +fi
> +
> +do_reset
> +
> +echo "SUCCESS!"
> +exit 0
> diff --git a/tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tc
>
Re: [RFC][PATCH] ftracetest: Add a couple of ftrace test cases
(2014/09/24 6:38), Steven Rostedt wrote:
[
Masami, I took two of my test scripts and added some basic comments
Thanks!
to them and copied them pretty much unchanged into a ftrace directory
under test.d. Is this fine, or is there more massaging I need to do
to them?
Yeah, ftrace has a double meaning :), so test.d/ftrace is fine to me.
I know the echos don't show up, but I kept them anyway. What should
happen with them?
I think you'd better use exit_unsupported/exit_xfail to notify
that the test target is not configured, or expected to fail.
Then the user can reconfigure that. Maybe we should keep the
detailed log of such results. (you can do it with --keep option)
]
Added two test cases to get the feel of adding tests to ftracetest.
The two cases are:
function profiling test, to make sure function profiling still works
with function tracing (was a regression)
function graph filter test to make sure that the function graph filter
does filter and also continues to filter when another function tracer
is running (like the stack tracer)
Signed-off-by: Steven Rostedt rost...@goodmis.org
---
.../ftrace/test.d/ftrace/fgraph-filter.tc | 111
+
.../ftrace/test.d/ftrace/func_profiler.tc | 78 +++
2 files changed, 189 insertions(+)
create mode 100644
tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc
create mode 100644
tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tc
diff --git a/tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc
b/tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc
new file mode 100644
index ..42f764d1f8d2
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/ftrace/fgraph-filter.tc
@@ -0,0 +1,111 @@
+#!/bin/sh
+# description: ftrace - function graph filters
+
+# Make sure that function graph filtering works, and is not
+# affected by other tracers enabled (like stack tracer)
+
+if ! grep -q function_graph available_tracers; then
+echo no function graph tracer configured
+exit 0;
This should call exit_unsupported, because the test is not passed.
+fi
+
+if [ ! -f set_ftrace_filter ]; then
+echo set_ftrace_filter not found? Is dynamic ftrace not set?
+exit 0
Ditto.
+fi
+
+do_reset() {
+echo nop current_tracer
+echo 0 /proc/sys/kernel/stack_tracer_enabled
+echo 1 tracing_on
+echo trace
+echo set_ftrace_filter
+}
+
+echo 0 tracing_on
+# clear trace
+echo trace
+# filter something, schedule is always good
+if ! echo schedule set_ftrace_filter; then
+# test for powerpc 64
+if ! echo .schedule set_ftrace_filter; then
+ echo can not enable schedule filter
+ exit -1
+fi
+fi
+
+echo function_graph current_tracer
+echo 1 tracing_on
+sleep 1
+# search for functions (has () on the line), and make sure
+# that only the schedule function was found
+count=`cat trace | grep '()' | grep -v schedule | wc -l`
+if [ $count -ne 0 ]; then
+echo Graph filtering not working by itself?
+exit -1;
+fi
+
+# Make sure we did find something
+count=`cat trace | grep 'schedule()' | wc -l`
+if [ $count -eq 0 ]; then
+echo No schedule traces found?
+exit -1
+fi
+
+echo Graph filtering works by itself
+
I think the following part should be a separated test for
stack trace.
+if [ ! -f stack_trace ]; then
+echo Stack tracer not configured, can't continue test
+# stack tracer not configured in this kernel? pass test anyway
+do_reset
+exit 0;
Here should call exit_unsupported.
+fi
+
+echo Now testing with stack tracer
+
+echo 1 /proc/sys/kernel/stack_tracer_enabled
+
+echo 0 tracing_on
+echo trace
+echo 1 tracing_on
+sleep 1
+
+count=`cat trace | grep '()' | grep -v schedule | wc -l`
+
+if [ $count -ne 0 ]; then
+echo Graph filtering not working with stack tracer?
+exit -1
+fi
+
+count=`cat trace | grep 'schedule()' | wc -l`
+if [ $count -eq 0 ]; then
+echo No schedule traces found?
+exit -1
+fi
+
+echo Graph filtering works with stack tracer
+
+echo Now testing if filtering still works without stack tracer
+
+echo 0 /proc/sys/kernel/stack_tracer_enabled
+echo trace
+sleep 1
+
+
+count=`cat trace | grep '()' | grep -v schedule | wc -l`
+
+if [ $count -ne 0 ]; then
+echo Graph filtering not working after stack tracer disabled?
+exit -1
+fi
+
+count=`cat trace | grep 'schedule()' | wc -l`
+if [ $count -eq 0 ]; then
+echo No schedule traces found?
+exit -1
+fi
+
+do_reset
+
+echo SUCCESS!
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tc
b/tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tc
new file mode 100644
index ..f35c58968d09
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/ftrace/func_profiler.tc
@@ -0,0 +1,78 @@
