You will need to fix all Schema Definition errors before any parse error would ever be detected.
Schema Definition errors stop compilation of the schema, no parse ever is run if there is any SDE. ________________________________ From: Christofer Dutz <[email protected]> Sent: Monday, February 25, 2019 10:55:36 AM To: [email protected]; Steve Lawrence Subject: Re: [VOTE] Release Apache Daffodil (incubating) 2.3.0-rc1 Yeah, it was previously working like that with 2.2.0, when updating I had to change quite a bit. Now the types and names are aligned with the schema. However I can't seem to be able to define rules that expect parse errors (and therefore don't return any XML) any longer. Chris Outlook für Android<https://aka.ms/ghei36> herunterladen ________________________________ From: Steve Lawrence <[email protected]> Sent: Monday, February 25, 2019 4:50:53 PM To: [email protected]; Christofer Dutz Subject: Re: [VOTE] Release Apache Daffodil (incubating) 2.3.0-rc1 Looking at your tdml file in github [1], it looks like your root element in defineSchema is "tpktMessage" with a lower-case "t", but your root element in the below parserTestCase has a capital "T". Maybe it's just a typo? - Steve [1] https://github.com/apache/incubator-plc4x/blob/develop/protocols/s7/src/test/resources/org/apache/plc4x/protocols/s7/protocol.tdml On 2/25/19 10:40 AM, Christofer Dutz wrote: > Hi Steve, > > so that worked > > fn:ceiling(xs:double(../rawLength div 8)) > > Now the last thing I'm having a problem with in TDML is that all test-cases > where I expect a parsing error - such as: > > <!-- > In this test-case the payload of the packet doesn't match the > length provided in the packet header (Too short) > --> > <tdml:parserTestCase name="TPKT: tooShortPayload" > root="TpktMessage" > model="s7Schema"> > <tdml:document> > <tdml:documentPart > type="byte">0300001611D00001000200C00109C102010</tdml:documentPart> > </tdml:document> > > <tdml:errors> > <tdml:error>Parse Error</tdml:error> > </tdml:errors> > </tdml:parserTestCase> > > I am getting error messages like this: > > org.apache.daffodil.tdml.TDMLExceptionImpl: (Implementation: daffodil) Did > not find diagnostic message "Parse Error" in any of the actual diagnostic > messages: > Schema Definition Error: No global element found for > {http://example.com}TpktMessage > Schema context: SchemaSet Location in > file:/var/folders/4m/d8xs244900sd8ypzndkf4_3w0000gn/T/s7Schema1558250055908446006.dfdl.xsd > Schema Definition Warning: attributeFormDefault is not part of DFDL and will > be ignored > Schema context: XMLSchemaDocument Location line 1 column 2 in > file:/var/folders/4m/d8xs244900sd8ypzndkf4_3w0000gn/T/s7Schema1558250055908446006.dfdl.xsd > > at > org.apache.daffodil.tdml.TDMLException$.apply(TDMLException.scala:31) > at > org.apache.daffodil.tdml.VerifyTestCase$.$anonfun$verifyAllDiagnosticsFound$4(TDMLRunner.scala:1404) > at > org.apache.daffodil.tdml.VerifyTestCase$.$anonfun$verifyAllDiagnosticsFound$4$adapted(TDMLRunner.scala:1393) > at scala.collection.immutable.List.foreach(List.scala:389) > > What do I have to change here? > > Chris > > > > Am 25.02.19, 16:22 schrieb "Steve Lawrence" <[email protected]>: > > I'm guessing that this was broken by commit a20ff41728, which changed > the result of division to always be an xs:decimal instead of an xs:float > or xs:double. Which is the correct behavior and did fix some potential > bugs. > > What isn't the correct behavior, which appears to have been broken for a > while, is rounding of xs:decimals. For example: > > fn:ceiling(xs:decimal(0.125)) > > which results in 0.125 and is clearly not correct. When you cast that to > a short, it becomes 0. > > Fortunately, it looks like rounding of a float/double works as expected: > > fn:ceiling(xs:double(0.125)) > > So a temporary workaround is to cast the result of your division to a > double, e.g: > > fn:ceiling(xs:double(../rawLength div 8)) > > I'll open a bug for this. > > Thanks, > - Steve > > On 2/25/19 9:53 AM, Christofer Dutz wrote: > > Hi Steve, > > > > I don't think it's a blocker either ... > > > > However as part of my tests, I updated the dependency in PLC4X to the > new Daffodil version. > > I had to fix quite a bit of things afterwards, however I got most to > work. Mostly Daffodil is now a little pickier than previously, however one > thing is currently blocking me: > > > > The following expression previously evaluated to lengthInBytes=1, now > it evaluates to 0 (the transport size is 3 and the rawLength is 1) > > > > <xs:element name="lengthInBytes" > type="s7:short" dfdl:inputValueCalc="{ > > > xs:unsignedShort(if((../transportSize eq 3) or (../transportSize eq 4) or > (../transportSize eq 5)) > > then > xs:short(fn:ceiling(../rawLength div 8)) > > else ../rawLength)}"/> > > > > So in general if should be: > > > > lengthInBytes = fn:ceiling(1 div 8) > > > > which should be 1 but is evaluated to 0 now ... even if I hard-code it > like "fn:ceiling(1 div 8)" the result is 0 > > > > Not quite sure why this is happening ... but it's currently breaking my > build. > > > > Chris > > > > > > Am 25.02.19, 14:35 schrieb "Steve Lawrence" <[email protected]>: > > > > Thanks for tracking this issue down. I can reproduce the issue by > > setting -Duser.country and user.language. This is definitely a bug, > > anything that affects parsing should come from a DFDL property > rather > > than the environment. I've created DAFFODIL-2074 to track this > issue. > > > > That said, I don't think this should be considered a blocker for > 2.3.0. > > It should definitely be fixed for next release, but setting the > locale > > is a reasonable workaround for now. > > > > - Steve > > > > On 2/25/19 5:12 AM, Christofer Dutz wrote: > > > Hi, > > > > > > guess I found the main problem. > > > It seems to have been related to me not being a US guy ... but > wasn't related to the time-zone, but my computer's locale settings. > > > > > > Mine default to "de_DE" where the number formats are different. > In Germany (and probably some other areas too) we use "," for the fractional > separator and "." for thousands separator (which is the opposite to the US > convention). > > > If I add the following settings to my SBT_OPTS: -Duser.country=EN > -Duser.language=us > > > Almost all previously failing tests succeed, however still one > still fails: > > > org.apache.daffodil.section05.simple_types.TestSimpleTypes > > > > > > Here looking at the failing tests, it's too probably related. For > me it looks as if the default locale settings are not correctly picked up ... > > > > > > In the end I'm building with these SBT_OPTS ... probably it's > worth adding a suggested SBT_OPTS in the readme as without it, it is > impossible to build. > > > > > > export SBT_OPTS="-Xss2M -Xmx5G -Xms5G -Duser.country=EN > -Duser.language=us" > > > > > > > > > Chris > > > > > > > > > Am 25.02.19, 09:31 schrieb "Christofer Dutz" > <[email protected]>: > > > > > > Hi guys, > > > > > > being back from my trip I was planning on continuing my > testing ... especially finding out why some tests are failing on my side. > > > I'll open Issues for them, if I find something. > > > > > > Also did I port some scripts we use in PLC4X and Edgent to do > the downloading of release candidates which also do the hash checking and > signature checking ... is there any interest in that? > > > I would be happy to provide them. > > > > > > Chris > > > > > > > > > > > > Am 23.02.19, 10:39 schrieb "Christofer Dutz" > <[email protected]>: > > > > > > HI and sorry for posting that back to the incubator ... I > simply hit reply on my phone and for some reason it added all other emails. > > > > > > Do you want me to create issues for anything I found? > It's not stuff that is required to be addressed before the release, I think > ... but could be helpful for the next. > > > > > > Chris > > > > > > Am 23.02.19, 01:13 schrieb "Christofer Dutz" > <[email protected]>: > > > > > > I'll check that out after coming home from my party > trip to Utrecht ;-) > > > > > > If this is required to build, I think it should be > mentioned in the readme. > > > > > > Chris > > > > > > Outlook für Android<https://aka.ms/ghei36> > herunterladen > > > > > > ________________________________ > > > From: Craig Russell <[email protected]> > > > Sent: Saturday, February 23, 2019 1:12:06 AM > > > To: Incubator > > > Cc: [email protected] > > > Subject: Re: [VOTE] Release Apache Daffodil > (incubating) 2.3.0-rc1 > > > > > > > On Feb 19, 2019, at 1:18 PM, Dave Fisher > <[email protected]> wrote: > > > > > > > > Hi - > > > > > > > > (1) To be fixed on the next release: > > > > (a) Please do not include the KEYS file in > the source release zip. Refer the users back to the location: > https://dist.apache.org/repos/dist/release/incubator/daffodil/KEYS > <https://dist.apache.org/repos/dist/release/incubator/daffodil/KEYS> > > > > > > Yes the KEYS must not be distributed in the source > release. But the canonical location for KEYS is > https://www.apache.org/dist/incubator/daffodil/KEYS > <https://www.apache.org/dist/incubator/daffodil/KEYS> > > > > > > The dist.apache.org url is not for public use. > > > > > > Craig > > > > > > > (b) Discuss vendor specific tests that are > name *ibm* and *tresys* on dev@daffodil. I think it would help community > building to have something more feature specific, but I could be wrong. > > > > (2) Rat Check passes. > > > > (3) All Signatures pass. > > > > (4) All Checksums pass. > > > > (5) LICENSE, NOTICE and DISCLAIMER look good. (MSI > was not checked. I do not have a windows machine available.) > > > > (6) Compiles > > > > (7) sbt test “runs”. I did have a large number of > OOM and other errors and they took over 3 hours to run on a MacBook Pro with > the latest macOS. > > > > (8) sbt it:test worked well. > > > > > > > > +1 to all packages except for MSI. > > > > +0 on MSI binary convenience package - I cannot > check it. > > > > > > > > Regards, > > > > Dave > > > > > > > >> On Feb 19, 2019, at 5:21 AM, Steve Lawrence > <[email protected]> wrote: > > > >> > > > >> Hi all, > > > >> > > > >> The Apache Daffodil community has voted and > approved the proposed > > > >> release of Apache Daffodil (incubating) 2.3.0-rc1. > > > >> > > > >> We now kindly request the Incubator PMC members > review and vote on this > > > >> incubator release. > > > >> > > > >> Daffodil is an open source implementation of the > DFDL specification that > > > >> uses DFDL schemas to parse fixed format data into > an infoset, which is > > > >> most commonly represented as either XML or JSON. > This allows the use of > > > >> well-established XML or JSON technologies and > libraries to consume, > > > >> inspect, and manipulate fixed format data in > existing solutions. > > > >> Daffodil is also capable of the reverse by > serializing or "unparsing" an > > > >> XML or JSON infoset back to the original data > format. > > > >> > > > >> Vote thread: > > > >> > https://lists.apache.org/thread.html/0ec1161ad296aebda54e86c219e3f1c4498a7de640c6342e6888aea1@%3Cdev.daffodil.apache.org%3E > > > >> > > > >> Result thread: > > > >> > > > >> > https://lists.apache.org/thread.html/a195a77640235501109bdea5ec7fb9a34c6f50644c7173fb8a919989@%3Cdev.daffodil.apache.org%3E > > > >> > > > >> All distribution packages, including signatures, > digests, etc. can be > > > >> found at: > > > >> > > > >> > https://dist.apache.org/repos/dist/dev/incubator/daffodil/2.3.0-rc1/ > > > >> > > > >> Staging artifacts can be found at: > > > >> > > > >> > https://repository.apache.org/content/repositories/orgapachedaffodil-1008/ > > > >> > > > >> This release has been signed with PGP key > 36F3494B033AE661, > > > >> corresponding to [email protected], which is > included in the > > > >> repository's KEYS file. This key can be found on > keyservers, such as: > > > >> > > > >> > http://pgp.mit.edu/pks/lookup?op=get&search=0x36F3494B033AE661 > > > >> > > > >> It is also listed here: > > > >> > > > >> > https://people.apache.org/keys/committer/slawrence.asc > > > >> > > > >> The release candidate has been tagged in git with > v2.3.0-rc1 on commit > > > >> hash 193c84690401debd93c1f48e39e564fec09e9337. > > > >> > > > >> For reference, here is a list of all closed JIRAs > tagged with 2.3.0: > > > >> > > > >> > https://issues.apache.org/jira/browse/DAFFODIL-2067?jql=project%20%3D%20DAFFODIL%20AND%20status%20%3D%20Resolved%20AND%20fixVersion%20%3D%202.3.0%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC > > > >> > > > >> For a summary of the changes in this release, see: > > > >> > > > >> https://daffodil.apache.org/releases/2.3.0/ > > > >> > > > >> Please review and vote. The vote will be open for > at least 72 hours. > > > >> > > > >> [ ] +1 approve > > > >> [ ] +0 no opinion > > > >> [ ] -1 disapprove (and reason why) > > > >> > > > >> Thanks, > > > >> - Steve > > > >> > > > >> > --------------------------------------------------------------------- > > > >> To unsubscribe, e-mail: > [email protected] > > > >> For additional commands, e-mail: > [email protected] > > > >> > > > > > > > > > > Craig L Russell > > > Secretary, Apache Software Foundation > > > [email protected] <mailto:[email protected]> > http://db.apache.org/jdo <http://db.apache.org/jdo> > > > > > > > > > > > > > > > > > > > > > > > > > > >
