It seems that the discussion on this issue has stalled a bit so I will try to summarize where we are on this and attempt to address the outstanding concerns.
I think everyone agrees that switching to Python will enable feature development in the test suite which is simply not practical with a shell script. The only drawback so far mentioned is significant. The availability of Python across embedded platforms that may want to test huge pages has been questioned. The three prominent embedded architectures that we are likely to interact with are: sh, alpha, and ppc. Of these, only ppc is actually supported by libhugetlbfs at the moment. For ppc boards, if it is impossible to create a test environment including Python, a larger but architecturally equivalent machine can be used to test the kernel and library functionality. Failing that, Python Freeze can be used to create an executable that is independent of Python. We could even assist in this process assuming actual demand for it. While a dependency on Python could possibly impact a small number of embedded users, I feel the benefits of a more robust set of tests far outweigh the downsides of the added dependency. I strongly recommend we proceed with the conversion. On Mon, 2008-11-17 at 16:50 +0000, Mel Gorman wrote: > On Fri, Nov 14, 2008 at 05:51:29PM +1100, David Gibson wrote: > > On Thu, Nov 13, 2008 at 08:26:12PM +0000, Adam Litke wrote: > > > Now that libhugetlbfs can work with multiple huge page sizes, testing this > > > support has become a priority. The following patch enables automatic > > > testing > > > of page sizes that are mounted and have at least one page allocated. > > > Care is > > > taken to assure that only valid combinations are tested. For example, > > > 32bit > > > tests are not run with 16GB pages. Following the run, a summary of all > > > page > > > sizes tested is printed. The following is some example output of the new > > > script: > > > > > > ********** TEST SUMMARY > > > * 64K 16M 16G > > > * 32-bit 64-bit 32-bit 64-bit 32-bit 64-bit > > > * Total testcases: 86 89 86 89 0 89 > > > * Skipped: 20 0 20 0 0 0 > > > * PASS: 59 75 62 85 0 49 > > > * FAIL: 5 6 1 1 0 29 > > > * Killed by signal: 0 0 0 0 0 0 > > > * Bad configuration: 2 2 3 3 0 10 > > > * Expected FAIL: 0 0 0 0 0 0 > > > * Unexpected PASS: 0 0 0 0 0 0 > > > * Strange test result: 0 6 0 0 0 1 > > > ********** > > > > > > Script programming language conversion alert: > > > > > > This patch rewrites run_tests.sh in python. I already anticipate the "why > > > change languages" comments so I come prepared with justification for the > > > conversion. Our test harness has extended beyond simply executing a list > > > of > > > test cases and dumping the output to stdout. The data set for the test > > > summary > > > is now three-dimensional. It is indexed by result type (total tests, > > > total > > > passed, etc), word size, and now page size. The simple arrays in bash > > > are not > > > up to the task. As the number of tests that are run increases, so does > > > the > > > challenge of presenting the results in a meaningful, easy to digest > > > format. > > > Shell scripts lack the output formatting constructs that are present in > > > languages such as Python and that make flexible output formatting > > > possible. > > > For these reasons (and the guarantee that the test harness will need to > > > get > > > even more sophisticated in the future), I made the inevitable decision to > > > cut > > > over to Python now. > > > > Hmm. I was about to say I thought changing to Python was a pretty > > good idea, but then I had some second thoughts. > > > > These are all sound arguments, and the script certainly was getting > > pretty messy in shell. I wrote it in shell initially because that was > > the simple obvious option when it was just a handful of simple tests. > > The testsuite is a lot more complex these days. > > > > The one big thing that worries me about switching to Python is that it > > introduces another build and test dependency. That's not a big issue > > for regular desktop or server systems - Python's common enough. But > > it is an issue for embedded boards, where hugepage support is becoming > > increasingly common. In particular that's troublesome because > > libhugetlbfs is de fact the testsuite for kernel hugepage support as > > well as for the library itself. Making it harder to do a thorough > > test of hugepages on J Random embedded board is a non-trivial > > drawback. > > > > Are the test environments for embedded boards really so constrained that > they cannot support python? I would have thought that an embedded machine > with sufficient memory for hugepages + libhugetlbfs would also be also able > to host python on a development board running the test scripts. Granted, > the production board might be too small but at that point the testing at > libhugetlbfs level should be done. > > If they do not have python, it also means that tools such as autotest > are out the window so their test environment is already more contrained > than we already generally think of. Right? > > > Not sure which way to go on this one. > > > > I'm leaning towards the jump to python right now on the (possibly > mistaken) belief that a board using hugepages will at least have a > development board version large enough to have python on it. > -- Adam Litke - (agl at us.ibm.com) IBM Linux Technology Center ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Libhugetlbfs-devel mailing list Libhugetlbfs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libhugetlbfs-devel
