[dpdk-dev] [RFC 0/4] Include resources in tests
On Thu, May 05, 2016 at 07:03:00PM +0200, Jan Viktorin wrote: > On Thu, 5 May 2016 14:29:30 +0100 > Bruce Richardson wrote: > > > On Fri, Apr 29, 2016 at 03:11:32PM +0200, Jan Viktorin wrote: > > > Hello, > > > > > > this patch set introduces a mechanism to include a resource (in general a > > > blob) > > > into the test binary. This allows to make tests less dependent on the > > > target > > > testing environment. The first use case is testing of PCI bus scan by > > > changing > > > the hard-coded path (/sys/bus/pci/devices) to something different and > > > provide > > > a fake tree of devices with the test. It can help with testing of > > > device-tree > > > parsing as I've proposed in [1] where such mechanism was missing at that > > > time. > > > I'd like to use such framework for the SoC infra testing as well. > > > > > > The patch set introduces a struct resource into the app/test. The > > > resource is > > > generic to include any kind of binary data. The binary data can be > > > created in > > > C or linked as an object file (created by objcopy). I am not sure where to > > > place the objcopy logic and how to perform guessing of the objcopy > > > arguments > > > as they are pretty non-standard. > > > > > > To include a complex resource (a file hierarchy), the last patch > > > implements > > > an archive extraction logic. So, it is possible to include a tar archive > > > and > > > unpack it before a test starts. Any ideas how to do this in a better way > > > are > > > welcome. > > > > > > [1] http://comments.gmane.org/gmane.comp.networking.dpdk.devel/36545 > > > > > > Regards > > > Jan Viktorin > > > > > BTW: It looks like your patch has a dependency on the 17-patch dev cleanup > > set from > > David, [and now on the header include bug fix I submitted yesterday too], so > > you should probably note that in any future revs of the patch you do. Save > > some head-scratching from those of us testing it out. :-) > > Hello, well, the patch set should be independent on that cleanup, so I take > it as a > bug report. There should be no dependency, sorry. I'll fix it and repost. > > Thanks > Jan > It's the RTE_INIT macro is being missed otherwise. I'm trying out this new infrastructure by reworking the lpm tests to use this for the big routing table data. I'll let you know any feedback when I'm done. /Bruce
[dpdk-dev] [RFC 0/4] Include resources in tests
On Thu, 5 May 2016 14:29:30 +0100 Bruce Richardson wrote: > On Fri, Apr 29, 2016 at 03:11:32PM +0200, Jan Viktorin wrote: > > Hello, > > > > this patch set introduces a mechanism to include a resource (in general a > > blob) > > into the test binary. This allows to make tests less dependent on the target > > testing environment. The first use case is testing of PCI bus scan by > > changing > > the hard-coded path (/sys/bus/pci/devices) to something different and > > provide > > a fake tree of devices with the test. It can help with testing of > > device-tree > > parsing as I've proposed in [1] where such mechanism was missing at that > > time. > > I'd like to use such framework for the SoC infra testing as well. > > > > The patch set introduces a struct resource into the app/test. The resource > > is > > generic to include any kind of binary data. The binary data can be created > > in > > C or linked as an object file (created by objcopy). I am not sure where to > > place the objcopy logic and how to perform guessing of the objcopy arguments > > as they are pretty non-standard. > > > > To include a complex resource (a file hierarchy), the last patch implements > > an archive extraction logic. So, it is possible to include a tar archive and > > unpack it before a test starts. Any ideas how to do this in a better way are > > welcome. > > > > [1] http://comments.gmane.org/gmane.comp.networking.dpdk.devel/36545 > > > > Regards > > Jan Viktorin > > > BTW: It looks like your patch has a dependency on the 17-patch dev cleanup > set from > David, [and now on the header include bug fix I submitted yesterday too], so > you should probably note that in any future revs of the patch you do. Save > some head-scratching from those of us testing it out. :-) Hello, well, the patch set should be independent on that cleanup, so I take it as a bug report. There should be no dependency, sorry. I'll fix it and repost. Thanks Jan > > Thanks, > /Bruce > -- Jan Viktorin E-mail: Viktorin at RehiveTech.com System Architect Web:www.RehiveTech.com RehiveTech Brno, Czech Republic
[dpdk-dev] [RFC 0/4] Include resources in tests
On Fri, Apr 29, 2016 at 03:11:32PM +0200, Jan Viktorin wrote: > Hello, > > this patch set introduces a mechanism to include a resource (in general a > blob) > into the test binary. This allows to make tests less dependent on the target > testing environment. The first use case is testing of PCI bus scan by changing > the hard-coded path (/sys/bus/pci/devices) to something different and provide > a fake tree of devices with the test. It can help with testing of device-tree > parsing as I've proposed in [1] where such mechanism was missing at that time. > I'd like to use such framework for the SoC infra testing as well. > > The patch set introduces a struct resource into the app/test. The resource is > generic to include any kind of binary data. The binary data can be created in > C or linked as an object file (created by objcopy). I am not sure where to > place the objcopy logic and how to perform guessing of the objcopy arguments > as they are pretty non-standard. > > To include a complex resource (a file hierarchy), the last patch implements > an archive extraction logic. So, it is possible to include a tar archive and > unpack it before a test starts. Any ideas how to do this in a better way are > welcome. > > [1] http://comments.gmane.org/gmane.comp.networking.dpdk.devel/36545 > > Regards > Jan Viktorin > BTW: It looks like your patch has a dependency on the 17-patch dev cleanup set from David, [and now on the header include bug fix I submitted yesterday too], so you should probably note that in any future revs of the patch you do. Save some head-scratching from those of us testing it out. :-) Thanks, /Bruce
[dpdk-dev] [RFC 0/4] Include resources in tests
On Fri, 29 Apr 2016 15:42:27 +0100 Bruce Richardson wrote: > On Fri, Apr 29, 2016 at 03:11:32PM +0200, Jan Viktorin wrote: > > Hello, > > > > this patch set introduces a mechanism to include a resource (in general a > > blob) > > into the test binary. This allows to make tests less dependent on the target > > testing environment. The first use case is testing of PCI bus scan by > > changing > > the hard-coded path (/sys/bus/pci/devices) to something different and > > provide > > a fake tree of devices with the test. It can help with testing of > > device-tree > > parsing as I've proposed in [1] where such mechanism was missing at that > > time. > > I'd like to use such framework for the SoC infra testing as well. > > > > The patch set introduces a struct resource into the app/test. The resource > > is > > generic to include any kind of binary data. The binary data can be created > > in > > C or linked as an object file (created by objcopy). I am not sure where to > > place the objcopy logic and how to perform guessing of the objcopy arguments > > as they are pretty non-standard. > > > > To include a complex resource (a file hierarchy), the last patch implements > > an archive extraction logic. So, it is possible to include a tar archive and > > unpack it before a test starts. Any ideas how to do this in a better way are > > welcome. > > > > [1] http://comments.gmane.org/gmane.comp.networking.dpdk.devel/36545 > > > > Regards > > Jan Viktorin > > > > Hi Jan, > > this looks really interesting, especially since just yesterday I was looking > at > taking the million-entry lpm test routing table out of the C code and into a > separate resource file in this case an ini file. If it is an automatic test case which can show some regressions over time then it is a good example of the resource usage. > > In terms of a solution, I'm not convinced of the placing of the blobs inside > the > test binary. I think a better solution would be to allow the different > autotests > to take parameters from the commandline, so that the user can specify the path > to the file to use for the test. What would be your opinion of such a scheme? I think that we are already at this stage. You can read parameters e.g. from the environment vars (so in fact, no changes to the DPDK testing code are needed). The thing is that I want to say "run tests" and don't care about any parameters to receive the result. It might be possible to pass parameters to the tests (optionally) to reuse the testing code base for different cases. However, initially, the test (in my opinion) should run on any architecture on any device without any configuration and return consistent results. How can I test probing of PCI devices on my PC when I have different network card then the author of the test? I cannot or I have to pass parameters which is not what I want as it complicates every single such test (and I have to understand those tests or understand the parameters syntax even though the code is unrelated to my work). I just want to check that the DPDK works as expected - without regressions. And what about the case when I have no such card (running automatic tests on a build server)? I don't like including the resources in binaries but I didn't find any better way yet. I need to easily install the tests together with the testing data (resources) to an embedded device and perform the tests on the target system. When you are testing x86 code on your x86 you don't have to care too much about the location of resources. But I do. And there is no standard way (at the moment) how to install the resources together with the testing code. What I like on this solution is that the DPDK git repository would contain the testing data as a real file hierarchy (e.g. the fake /sys/bus/...). After build and install of DPDK, the file hierachy is packed (and compressed if needed) together with the tests. So moving to the target platform works without any other changes to the testing/install infrastructure. I could potentially run PCI tests on my ARM board without any PCI available and don't have to ignore the failures of those tests (as they simply pass). Regards Jan > > /Bruce -- Jan ViktorinE-mail: Viktorin at RehiveTech.com System ArchitectWeb:www.RehiveTech.com RehiveTech Brno, Czech Republic
[dpdk-dev] [RFC 0/4] Include resources in tests
On Fri, Apr 29, 2016 at 03:11:32PM +0200, Jan Viktorin wrote: > Hello, > > this patch set introduces a mechanism to include a resource (in general a > blob) > into the test binary. This allows to make tests less dependent on the target > testing environment. The first use case is testing of PCI bus scan by changing > the hard-coded path (/sys/bus/pci/devices) to something different and provide > a fake tree of devices with the test. It can help with testing of device-tree > parsing as I've proposed in [1] where such mechanism was missing at that time. > I'd like to use such framework for the SoC infra testing as well. > > The patch set introduces a struct resource into the app/test. The resource is > generic to include any kind of binary data. The binary data can be created in > C or linked as an object file (created by objcopy). I am not sure where to > place the objcopy logic and how to perform guessing of the objcopy arguments > as they are pretty non-standard. > > To include a complex resource (a file hierarchy), the last patch implements > an archive extraction logic. So, it is possible to include a tar archive and > unpack it before a test starts. Any ideas how to do this in a better way are > welcome. > > [1] http://comments.gmane.org/gmane.comp.networking.dpdk.devel/36545 > > Regards > Jan Viktorin > Hi Jan, this looks really interesting, especially since just yesterday I was looking at taking the million-entry lpm test routing table out of the C code and into a separate resource file in this case an ini file. In terms of a solution, I'm not convinced of the placing of the blobs inside the test binary. I think a better solution would be to allow the different autotests to take parameters from the commandline, so that the user can specify the path to the file to use for the test. What would be your opinion of such a scheme? /Bruce
[dpdk-dev] [RFC 0/4] Include resources in tests
Hello, this patch set introduces a mechanism to include a resource (in general a blob) into the test binary. This allows to make tests less dependent on the target testing environment. The first use case is testing of PCI bus scan by changing the hard-coded path (/sys/bus/pci/devices) to something different and provide a fake tree of devices with the test. It can help with testing of device-tree parsing as I've proposed in [1] where such mechanism was missing at that time. I'd like to use such framework for the SoC infra testing as well. The patch set introduces a struct resource into the app/test. The resource is generic to include any kind of binary data. The binary data can be created in C or linked as an object file (created by objcopy). I am not sure where to place the objcopy logic and how to perform guessing of the objcopy arguments as they are pretty non-standard. To include a complex resource (a file hierarchy), the last patch implements an archive extraction logic. So, it is possible to include a tar archive and unpack it before a test starts. Any ideas how to do this in a better way are welcome. [1] http://comments.gmane.org/gmane.comp.networking.dpdk.devel/36545 Regards Jan Viktorin --- Jan Viktorin (4): app/test: introduce resources for tests app/test: support resources externally linked app/test: add functions to create files from resources app/test: support resources archived by tar app/test/Makefile| 38 +++ app/test/resource.c | 276 +++ app/test/resource.h | 99 + app/test/test_resource.c | 132 +++ 4 files changed, 545 insertions(+) create mode 100644 app/test/resource.c create mode 100644 app/test/resource.h create mode 100644 app/test/test_resource.c -- 2.8.0
