Hi Roland, I see you added testing expert to the loop, good :-)
Per your request ksh93 and testing discuss are part of CC now. V p?, 17. 07. 2009 v 02:40, Roland Mainz p??e: > Milan Jurik wrote: > [CC:'ing Jim Walker since he knows the test setup&6farms] > > test plan approved for C-team inception. One thing comming from it - > > look at CTI > > (http://opensolaris.org/os/community/testing/testsuites/ctifortet/) and > > evaluate how much work should be needed to wrap ksh93 test suite in CTI > > (so it could be used by automated tests later), please. > > It depends on it how the CTI stuff really works. I've discussed that > with April long ago - it's theoretically possible but it becomes > difficult for three reasons: > 1. the ksh93 test suite is CPU-load senstive - a too high CPU load will > cause some tests to fail because they depend on precise timing (e.g. the > signal and pipe tests are the top canidates for breakdown since they all > do some kind of interprocess communication) > 2. the ksh93 tests are I/O latency-sensitive because the tests assume > that writes are done after a certain time (e.g. 0.2 seconds etc.) and > that a co-/parent-process can read the data then > 3. the ksh93 tests are tied to precisely the ksh93 version they are > testing. That's why the matching testing code sits in OS/Net and not in > the test gate (and keeping different gates correctly in sync is > considered to be difficult and would loose the benefits of having the > test suite in OS/Net (and delivered with the rest of the system (which > turned out to be a good idea since we caught dozends of system bugs that > way (e.g. the Solaris/SystemZ port wouldn't be in the current shape > without it)))) > > The suggested workaround for [1] was to move the tests to the "RT" > (=realtime) class (and bind the tests to n-1 CPUs (where <n> is the > number of online CPUs in a system (this leaves one CPU free for > processes in the "TS" class))) ... however that may cause a whole > testmachine to hang if it only has a) one CPU and b) one or more test > starts to consume 100% CPU. And I am not sure whether the test suite > allows tests to run under the neccesary priviledges. > The alternative for [1] would be to find a way to "enforce" that the > test suite is the only thing which runs on the matching machine. Jim... > is that possible ? > > For [2] we may simply use "tmpfs" as working directory... > ... and for [3] we simply use the version from /usr/demo/ksh/tests/ and > just add a plugin for CTI which lives in the testing package. > I understand your points to have test suite in ON gate, but on the other hand this complicates the regression testing because you are not able to do automated testing. People should not be forced to learn one more testing framework and to do testing manually if they are fixing e.g. libc (which can have impact on ksh93). I hope somebody in testing-discuss@ can evaluate possibilities. I plan to fill RFE that ksh93 testing should be converted/encapsulated to CTI. I think it should not be your responsibility but it would be nice from you if you can be technical consultant for it. Best regards, Milan
