mbeckerle opened a new pull request #140: TDML Runner and test changes for Cross Validation with IBM. URL: https://github.com/apache/incubator-daffodil/pull/140 This review goes with one that is outside of the Daffodil repo here: OpenDFDL/ibmDFDLCrossTester#1 This is a very large change set. Major work on the TDML Runner, and cleanup of tests. I am able to run tests in daffodil-test-ibm1 against IBM dfdl, using just a few setup clicks. I do not yet have a way to do this from sbt/command line. Also fixed other minor bugs that were in the way of this. I tried to separate these out into separate change sets, but was unable to do so. DAFFODIL-2014 - TextBidi property name was misspelled. DAFFODIL-2015 - accept textStandardOutputCharacter (legacy prop name) DAFFODIL-1616 - textOutputMinLength required when it shouldn't be. TDML Runner now implements everything it does in terms of an abstract DFDL processor defined in the tdml.processor package. There are a number of sweeping changes in this that change many files. First: Added explicit xs:include of DFDLGeneralFormat.dfdl.xsd to all TDML. The Daffodil TDML runner used to implicitly include DFDLGeneralFormat.dfdl.xsd. Now that we're cross testing, tests won't necessarily want to do this, so it has to be done explicitly. Second: The namespace prefixes in the root elements in many of our test cases were not correct. Those had to be uniformly fixed so that tests can insist on the proper namespace in result data. These were wrong because the comparison logic in the test rig was tolerating them being incorrect, but as part of making the TDML runner into a viable cross-testing rig, these things are being tightened up. The root element named by a test must match that of the infoset element. (in fact the root element declaration is unnecessary, as the root will be taken from the infoset's root element. The TDML module's code has changed significantly. The daffodil-tdml module has been split into daffodil-tdml and daffodil-tdml-processor. The latter has dependencies on all of daffodil. The former is independent of most of daffodil so that it can be used with an IBM-based implementation of the tdml processor class (Which is defined separately and outside of Daffodil's code base.) Yes, this is yet another API for Daffodil, but it is narrower and not as comprehensive as the SAPI/JAPI stuff. It's only and exactly what the TDML runner uses to drive tests. The TDML processors are always dynamically loaded from the classpath (even Daffodil's own - just to insure we're always testing this mechanism.) This allows one to change which processor is used by changing the classpath, and avoids a static dependency from the daffodil-tdml module on the daffodil-tdml-processor module. Enable TDMLDFDLProcessors to force skip of left-over-data checking. Moved all the IBM contributed tests to the daffodil-test-ibm1 module out of the daffodil-test module. There were redundant tests being run in both places. Probably this was due to an ambitious QA effort to reorganize the IBM-supplied tests to determine requirements coverage on the DFDL Specification, along with the Daffodil-project tests... I mean why have more than one test for a given requirement. However, this was not done right - moving the tests from one place to another - instead tests were left in both places. So I have moved all the tests that use the files supplied orignally by IBM back to the daffodil-test-ibm1 module. So there is no longer any ibm-tests subdir of the daffodil-test module. (Reminder: IBM supplied these a long time ago, and they are fully approved for inclusion into Daffodil, with ASF licensing, etc. There is no issue of source code origins here. ) Modify daffodil-test-ibm1 module to run most tests on IBM_DFDL also. Properties were added that are required by IBM DFDL and other properties changed: E.g., encodingErrorPolicy='error' choiceLengthKind='implicit' (also added to DFDLGeneralFormat.dfdl.xsd) binaryNumberCheckPolicy="lax" (ditto) This obsolete property had to be removed from our IBMDefined/GeneralPurposeFormat.xsd textBidiTextShaped="no" Quite a lot of the tests in daffodil-test-ibm1 pass when run on IBM_DFDL. Some fail. A few tests have been modified to specify daffodil only because they use features only in Daffodil (like hidden groups and calculated values). The failures are mixture of unexpected infoset contents (much around calendars with timezone Z vs. "+00:00". There are other tests that appear to be places where IBM DFDL and daffodil have fundamentally different behavior that needs investigation. The runner.trace method works and will invoke the IBM_DFDL trace facility when running tests via IBM_DFDL. This is needed to get more than just "parse failed" from it. A couple of other minor things. Fixed test regression in test_alignmentPaddingOVC4. This test shows that when textOutputMinLength and an explicit length constant are inconsistent, we still get a warning anyway. Removed test dateBinPacked3. This test was only checking that binaryNumberCheckPolicy was not defined, but it now is defined for IBM compatibility reasons, in the DFDLGeneralFormat.dfdl.xsd. DAFFODIL-2014, DAFFODIL-723, DAFFODIL-2015, DAFFODIL-1616
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
