Negative tests should still work. The PCAP schema [1] has been updated to use 2.3.0 and it has negative tests that pass and didn't require any changes. Can you push your changes somewhere we can take a look?
We did make some fairly significant changes to how the TDML runner works, but that should be mostly behind the scenes. Thanks, - Steve On 2/25/19 10:55 AM, Christofer Dutz wrote: > Yeah, it was previously working like that with 2.2.0, when updating I had to > change quite a bit. Now the types and names are aligned with the schema. > > However I can't seem to be able to define rules that expect parse errors (and > therefore don't return any XML) any longer. > > Chris > > Outlook für Android <https://aka.ms/ghei36> herunterladen > > -------------------------------------------------------------------------------- > *From:* Steve Lawrence <[email protected]> > *Sent:* Monday, February 25, 2019 4:50:53 PM > *To:* [email protected]; Christofer Dutz > *Subject:* Re: [VOTE] Release Apache Daffodil (incubating) 2.3.0-rc1 > Looking at your tdml file in github [1], it looks like your root element > in defineSchema is "tpktMessage" with a lower-case "t", but your root > element in the below parserTestCase has a capital "T". Maybe it's just a > typo? > > - Steve > > [1] > https://github.com/apache/incubator-plc4x/blob/develop/protocols/s7/src/test/resources/org/apache/plc4x/protocols/s7/protocol.tdml > > > On 2/25/19 10:40 AM, Christofer Dutz wrote: >> Hi Steve, >> >> so that worked >> >> fn:ceiling(xs:double(../rawLength div 8)) >> >> Now the last thing I'm having a problem with in TDML is that all test-cases >> where I expect a parsing error - such as: >> >> <!-- >> In this test-case the payload of the packet doesn't match the >> length provided in the packet header (Too short) >> --> >> <tdml:parserTestCase name="TPKT: tooShortPayload" >> root="TpktMessage" >> model="s7Schema"> >> <tdml:document> >> <tdml:documentPart >> type="byte">0300001611D00001000200C00109C102010</tdml:documentPart> >> </tdml:document> >> >> <tdml:errors> >> <tdml:error>Parse Error</tdml:error> >> </tdml:errors> >> </tdml:parserTestCase> >> >> I am getting error messages like this: >> >> org.apache.daffodil.tdml.TDMLExceptionImpl: (Implementation: daffodil) Did >> not find diagnostic message "Parse Error" in any of the actual diagnostic >> messages: >> Schema Definition Error: No global element found for >> {http://example.com}TpktMessage >> Schema context: SchemaSet Location in >> file:/var/folders/4m/d8xs244900sd8ypzndkf4_3w0000gn/T/s7Schema1558250055908446006.dfdl.xsd >> Schema Definition Warning: attributeFormDefault is not part of DFDL and will >> be ignored >> Schema context: XMLSchemaDocument Location line 1 column 2 in >> file:/var/folders/4m/d8xs244900sd8ypzndkf4_3w0000gn/T/s7Schema1558250055908446006.dfdl.xsd >> >> at >> org.apache.daffodil.tdml.TDMLException$.apply(TDMLException.scala:31) >> at >> org.apache.daffodil.tdml.VerifyTestCase$.$anonfun$verifyAllDiagnosticsFound$4(TDMLRunner.scala:1404) >> at >> org.apache.daffodil.tdml.VerifyTestCase$.$anonfun$verifyAllDiagnosticsFound$4$adapted(TDMLRunner.scala:1393) >> at scala.collection.immutable.List.foreach(List.scala:389) >> >> What do I have to change here? >> >> Chris >> >> >> >> Am 25.02.19, 16:22 schrieb "Steve Lawrence" <[email protected]>: >> >> I'm guessing that this was broken by commit a20ff41728, which changed >> the result of division to always be an xs:decimal instead of an xs:float >> or xs:double. Which is the correct behavior and did fix some potential >> bugs. >> >> What isn't the correct behavior, which appears to have been broken for a >> while, is rounding of xs:decimals. For example: >> >> fn:ceiling(xs:decimal(0.125)) >> >> which results in 0.125 and is clearly not correct. When you cast that to >> a short, it becomes 0. >> >> Fortunately, it looks like rounding of a float/double works as expected: >> >> fn:ceiling(xs:double(0.125)) >> >> So a temporary workaround is to cast the result of your division to a >> double, e.g: >> >> fn:ceiling(xs:double(../rawLength div 8)) >> >> I'll open a bug for this. >> >> Thanks, >> - Steve >> >> On 2/25/19 9:53 AM, Christofer Dutz wrote: >> > Hi Steve, >> > >> > I don't think it's a blocker either ... >> > >> > However as part of my tests, I updated the dependency in PLC4X to the >> new Daffodil version. >> > I had to fix quite a bit of things afterwards, however I got most to >> work. Mostly Daffodil is now a little pickier than previously, however one >> thing is currently blocking me: >> > >> > The following expression previously evaluated to lengthInBytes=1, now >> it evaluates to 0 (the transport size is 3 and the rawLength is 1) >> > >> > <xs:element name="lengthInBytes" >> type="s7:short" dfdl:inputValueCalc="{ >> > >> xs:unsignedShort(if((../transportSize eq 3) or (../transportSize eq 4) or >> (../transportSize eq 5)) >> > then >> xs:short(fn:ceiling(../rawLength div 8)) >> > else ../rawLength)}"/> >> > >> > So in general if should be: >> > >> > lengthInBytes = fn:ceiling(1 div 8) >> > >> > which should be 1 but is evaluated to 0 now ... even if I hard-code it >> like "fn:ceiling(1 div 8)" the result is 0 >> > >> > Not quite sure why this is happening ... but it's currently breaking >> my build. >> > >> > Chris >> > >> > >> > Am 25.02.19, 14:35 schrieb "Steve Lawrence" <[email protected]>: >> > >> > Thanks for tracking this issue down. I can reproduce the issue by >> > setting -Duser.country and user.language. This is definitely a bug, >> > anything that affects parsing should come from a DFDL property >> rather >> > than the environment. I've created DAFFODIL-2074 to track this >> issue. >> > >> > That said, I don't think this should be considered a blocker for >> 2.3.0. >> > It should definitely be fixed for next release, but setting the >> locale >> > is a reasonable workaround for now. >> > >> > - Steve >> > >> > On 2/25/19 5:12 AM, Christofer Dutz wrote: >> > > Hi, >> > > >> > > guess I found the main problem. >> > > It seems to have been related to me not being a US guy ... but >> wasn't related to the time-zone, but my computer's locale settings. >> > > >> > > Mine default to "de_DE" where the number formats are different. >> In Germany (and probably some other areas too) we use "," for the fractional >> separator and "." for thousands separator (which is the opposite to the US >> convention). >> > > If I add the following settings to my SBT_OPTS: >> -Duser.country=EN -Duser.language=us >> > > Almost all previously failing tests succeed, however still one >> still fails: >> > > org.apache.daffodil.section05.simple_types.TestSimpleTypes >> > > >> > > Here looking at the failing tests, it's too probably related. >> For me it looks as if the default locale settings are not correctly picked >> up ... >> > > >> > > In the end I'm building with these SBT_OPTS ... probably it's >> worth adding a suggested SBT_OPTS in the readme as without it, it is >> impossible to build. >> > > >> > > export SBT_OPTS="-Xss2M -Xmx5G -Xms5G -Duser.country=EN >> -Duser.language=us" >> > > >> > > >> > > Chris >> > > >> > > >> > > Am 25.02.19, 09:31 schrieb "Christofer Dutz" >> <[email protected]>: >> > > >> > > Hi guys, >> > > >> > > being back from my trip I was planning on continuing my >> testing ... especially finding out why some tests are failing on my side. >> > > I'll open Issues for them, if I find something. >> > > >> > > Also did I port some scripts we use in PLC4X and Edgent to >> do the downloading of release candidates which also do the hash checking and >> signature checking ... is there any interest in that? >> > > I would be happy to provide them. >> > > >> > > Chris >> > > >> > > >> > > >> > > Am 23.02.19, 10:39 schrieb "Christofer Dutz" >> <[email protected]>: >> > > >> > > HI and sorry for posting that back to the incubator ... >> I simply hit reply on my phone and for some reason it added all other emails. >> > > >> > > Do you want me to create issues for anything I found? >> It's not stuff that is required to be addressed before the release, I think >> ... but could be helpful for the next. >> > > >> > > Chris >> > > >> > > Am 23.02.19, 01:13 schrieb "Christofer Dutz" >> <[email protected]>: >> > > >> > > I'll check that out after coming home from my party >> trip to Utrecht ;-) >> > > >> > > If this is required to build, I think it should be >> mentioned in the readme. >> > > >> > > Chris >> > > >> > > Outlook für Android<https://aka.ms/ghei36> >> herunterladen >> > > >> > > ________________________________ >> > > From: Craig Russell <[email protected]> >> > > Sent: Saturday, February 23, 2019 1:12:06 AM >> > > To: Incubator >> > > Cc: [email protected] >> > > Subject: Re: [VOTE] Release Apache Daffodil >> (incubating) 2.3.0-rc1 >> > > >> > > > On Feb 19, 2019, at 1:18 PM, Dave Fisher >> <[email protected]> wrote: >> > > > >> > > > Hi - >> > > > >> > > > (1) To be fixed on the next release: >> > > > (a) Please do not include the KEYS file in >> the source release zip. Refer the users back to the location: >> https://dist.apache.org/repos/dist/release/incubator/daffodil/KEYS > <https://dist.apache.org/repos/dist/release/incubator/daffodil/KEYS> >> > > >> > > Yes the KEYS must not be distributed in the source >> release. But the canonical location for KEYS is >> https://www.apache.org/dist/incubator/daffodil/KEYS > <https://www.apache.org/dist/incubator/daffodil/KEYS> >> > > >> > > The dist.apache.org url is not for public use. >> > > >> > > Craig >> > > >> > > > (b) Discuss vendor specific tests that are >> name *ibm* and *tresys* on dev@daffodil. I think it would help community >> building to have something more feature specific, but I could be wrong. >> > > > (2) Rat Check passes. >> > > > (3) All Signatures pass. >> > > > (4) All Checksums pass. >> > > > (5) LICENSE, NOTICE and DISCLAIMER look good. (MSI >> was not checked. I do not have a windows machine available.) >> > > > (6) Compiles >> > > > (7) sbt test “runs”. I did have a large number of >> OOM and other errors and they took over 3 hours to run on a MacBook Pro with >> the latest macOS. >> > > > (8) sbt it:test worked well. >> > > > >> > > > +1 to all packages except for MSI. >> > > > +0 on MSI binary convenience package - I cannot >> check it. >> > > > >> > > > Regards, >> > > > Dave >> > > > >> > > >> On Feb 19, 2019, at 5:21 AM, Steve Lawrence >> <[email protected]> wrote: >> > > >> >> > > >> Hi all, >> > > >> >> > > >> The Apache Daffodil community has voted and >> approved the proposed >> > > >> release of Apache Daffodil (incubating) 2.3.0-rc1. >> > > >> >> > > >> We now kindly request the Incubator PMC members >> review and vote on this >> > > >> incubator release. >> > > >> >> > > >> Daffodil is an open source implementation of the >> DFDL specification that >> > > >> uses DFDL schemas to parse fixed format data into >> an infoset, which is >> > > >> most commonly represented as either XML or JSON. >> This allows the use of >> > > >> well-established XML or JSON technologies and >> libraries to consume, >> > > >> inspect, and manipulate fixed format data in >> existing solutions. >> > > >> Daffodil is also capable of the reverse by >> serializing or "unparsing" an >> > > >> XML or JSON infoset back to the original data >> format. >> > > >> >> > > >> Vote thread: >> > > >> >> https://lists.apache.org/thread.html/0ec1161ad296aebda54e86c219e3f1c4498a7de640c6342e6888aea1@%3Cdev.daffodil.apache.org%3E >> > > >> >> > > >> Result thread: >> > > >> >> > > >> >> https://lists.apache.org/thread.html/a195a77640235501109bdea5ec7fb9a34c6f50644c7173fb8a919989@%3Cdev.daffodil.apache.org%3E >> > > >> >> > > >> All distribution packages, including signatures, >> digests, etc. can be >> > > >> found at: >> > > >> >> > > >> >> https://dist.apache.org/repos/dist/dev/incubator/daffodil/2.3.0-rc1/ >> > > >> >> > > >> Staging artifacts can be found at: >> > > >> >> > > >> >> https://repository.apache.org/content/repositories/orgapachedaffodil-1008/ >> > > >> >> > > >> This release has been signed with PGP key >> 36F3494B033AE661, >> > > >> corresponding to [email protected], which is >> included in the >> > > >> repository's KEYS file. This key can be found on >> keyservers, such as: >> > > >> >> > > >> >> http://pgp.mit.edu/pks/lookup?op=get&search=0x36F3494B033AE661 >> > > >> >> > > >> It is also listed here: >> > > >> >> > > >> >> https://people.apache.org/keys/committer/slawrence.asc >> > > >> >> > > >> The release candidate has been tagged in git with >> v2.3.0-rc1 on commit >> > > >> hash 193c84690401debd93c1f48e39e564fec09e9337. >> > > >> >> > > >> For reference, here is a list of all closed JIRAs >> tagged with 2.3.0: >> > > >> >> > > >> >> https://issues.apache.org/jira/browse/DAFFODIL-2067?jql=project%20%3D%20DAFFODIL%20AND%20status%20%3D%20Resolved%20AND%20fixVersion%20%3D%202.3.0%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC >> > > >> >> > > >> For a summary of the changes in this release, see: >> > > >> >> > > >> https://daffodil.apache.org/releases/2.3.0/ >> > > >> >> > > >> Please review and vote. The vote will be open for >> at least 72 hours. >> > > >> >> > > >> [ ] +1 approve >> > > >> [ ] +0 no opinion >> > > >> [ ] -1 disapprove (and reason why) >> > > >> >> > > >> Thanks, >> > > >> - Steve >> > > >> >> > > >> >> --------------------------------------------------------------------- >> > > >> To unsubscribe, e-mail: >> [email protected] >> > > >> For additional commands, e-mail: >> [email protected] >> > > >> >> > > > >> > > >> > > Craig L Russell >> > > Secretary, Apache Software Foundation >> > > [email protected] <mailto:[email protected]> >> http://db.apache.org/jdo <http://db.apache.org/jdo> >> > > >> > > >> > > >> > > >> > > >> > > >> > >> > >> > >> >> >> >
