Writing some CLI tests showed a diagnostic warning has gone undetected in the embedded schematron parse. Using xs:appinfo to add schematron hits [AnnotatedSchemaComponent#L403]( https://github.com/apache/incubator-daffodil/blob/master/daffodil-core/src/main/scala/org/apache/daffodil/dsom/AnnotatedSchemaComponent.scala#L403 ).
`this.SDW(WarnID.AppinfoNoSource, """xs:appinfo without source attribute. Is source="http://www.ogf.org/dfdl/"; missing?""")` Thoughts on siliencing or lessening significance of this (eg. log instead of diagnostic)? On Wed, Jan 6, 2021 at 9:19 AM John Wass <jwa...@gmail.com> wrote: > The schema and tests for BMP/GIF/JPEG were moved into branches on those > DFDLSchemas repos. After this PR is merged and a the next release is > pubished those tests could be added to each of those repos. I suppose the > embedded schematron schema could merged any time without the tests. Those > repos would be a good context to continue and resolve the "best practices > in the schematron" discussions. > > On Tue, Dec 22, 2020 at 9:53 AM John Wass <jwa...@gmail.com> wrote: > >> > The second one is similar to examples in the GIF schema >> <https://github.com/DFDLSchemas/GIF/blob/master/src/main/resources/com/mitre/gif/sch/GIF.sch#L74-L76>. >> That schema can be added in the PR unit tests, to go along with the BMP and >> JPEG. >> >> Added the gif schema to the tests, looking good. Specifically looked at >> rule `count(/GIF/Global_Color_Table/RGB) eq math:pow(2, >> ../number(Size_of_Global_Color_Table) + 1)`. >> >> Working on embedding the bmp schema now as the final integration test. >> >> >> On Mon, Dec 21, 2020 at 7:49 AM John Wass <jwa...@gmail.com> wrote: >> >>> > Does the process create SVRL files when it completes? >>> >>> No, the svrl is consumed and converted into Daffodil diagnostics. >>> >>> >>> > Is there a commandline option to direct the SVRL file to a specific >>> path and name? >>> >>> It doesn't, but is a good idea and certainly could. Passing a flag >>> through the validator config could trigger writing the file. >>> >>> Probably be in a follow up PR. >>> >>> >>> > I'm curious of those type of tests will work with this process. >>> >>> They should. The first can be checked in a unit test that matches a >>> byte. The second one is similar to examples in the GIF schema >>> <https://github.com/DFDLSchemas/GIF/blob/master/src/main/resources/com/mitre/gif/sch/GIF.sch#L74-L76>. >>> That schema can be added in the PR unit tests, to go along with the BMP and >>> JPEG. >>> >>> >>> >>> >>> On Fri, Dec 18, 2020 at 2:43 PM Rege Nteligen <r.nteli...@gmail.com> >>> wrote: >>> >>>> I took a look at the sample xsd's with the imbedded schematron >>>> asserts. It looks good. Does the process create SVRL files when it >>>> completes? Is there a commandline option to direct the SVRL file to a >>>> specific path and name? >>>> >>>> I was recently working with a modified daffodil GIF schema and >>>> schematron to report various findings with GIF files. Several test >>>> involved testting that keyword were not in HEX blob fields. I'm curious of >>>> those type of tests will work with this process. This is a sample assert: >>>> <sch:assert test="not(./LSD_Blob[matches(.,'73716C')]) and >>>> not(./LSD_Blob[matches(.,'53514C')])"> >>>> GIF: FAIL: LSD_Blob: AFTER-HDR-REF-SQL: Possible >>>> malicious SQL reference between segemnts >>>> </sch:assert> >>>> >>>> I've also done test to see if the count of bytes in one field matched >>>> the size of the field value from another field: >>>> <sch:assert test="if(/GIF/Global_Color_Table/RGB) then >>>> count(/GIF/Global_Color_Table/RGB) eq math:pow(2, >>>> Packed_Byte/number(Size_of_Global_Color_Table) + 1) else true()"> >>>> GIF: RED: LSG_GCL: GCL-RGB-CNT: There must be >>>> Size_of_Global_Color_Table RGB values. >>>> </sch:assert> >>>> >>>> >>>> >>>> On 2020/12/18 17:21:02, John Wass <jwa...@gmail.com> wrote: >>>> > The Embedded Schematron PR is moving along, hoping to get it out of >>>> WIP >>>> > soon. https://github.com/apache/incubator-daffodil/pull/463 >>>> > >>>> > The JPEG and BMP schema repos are being used for testing now, and the >>>> PNG >>>> > looks like it would provide some great coverage.. maybe too great :/ >>>> Any >>>> > other noteworthy sources of sch+data that might be beneficial to test >>>> with? >>>> > >>>> > Observations on embedding >>>> > - Behavior has been predictable, and errors have been clear >>>> > - There are multiple placement options for schematron rules in a >>>> schema >>>> > - The Validator API has held up well, but might be one issue to come >>>> out of >>>> > this effort >>>> > >>>> > Examples at >>>> > >>>> https://github.com/jw3/incubator-daffodil/tree/validator_spi/embedded_schematron/daffodil-schematron/src/test/resources/xsd >>>> > >>>> >>>
