On 01/12/2013 09:48 AM, Paul Davies C wrote:
*This custom python script will record its own resource usage statistics in temporary file.
This approach assumes any pre-execution work done to load python, it's libraries, modules, etc. happens instantly, or is nearly immeasurably fast. On a slower machine, or machine at or near it's resource limits, this won't always be true. This will be especially apparent at the low-end of the usage spectrum.
In other words, the custom python script can't record time/resouces consumed BEFORE it starts running. However, the 'time' command can (because it's obtaining this info from the kernel). The proposed approach assumes startup-time/resource use is (or very close to) zero. This won't always be the case.
This problem can be mitigated somewhat by ensuring the custom python script runs long and hard enough that the startup cost is relatively minuscule for all systems/conditions. It's not a perfect solution but simple enough to achieve. Assuming your purpose here is learning autotest, then this may be 'good enough'.
Otherwise, I think this test would be much improved if it were more narrowly focused. These aspects of the kernel, and the time command itself have been around for a LONG time. Also, don't forget that some shells (like bash) also have their own built-in 'time' command. It's behavior and features are very different from /usr/bin/time, though the source for most of it's data (the kernel) remains consistent.
My suggestion is you focus on a very specific aspect, choose a type of test (functional, performance, error, etc.) and on a specific area (kernel, user-space, parameter handling, etc.), and work from there. Otherwise you'll wind up with a test that doesn't actually test much, or one that give inconsistent results that are not related to the test-subject itself.
Hope that helps. -- Chris Evich, RHCA, RHCE, RHCDS, RHCSS Quality Assurance Engineer e-mail: cevich + `@' + redhat.com o: 1-888-RED-HAT1 x44214 _______________________________________________ Autotest-kernel mailing list [email protected] https://www.redhat.com/mailman/listinfo/autotest-kernel
