Hi Steve, I created a feature branch with all of my changes: https://github.com/apache/incubator-plc4x/tree/feature/daffodil-2.3.0/protocols/s7
Chris Am 25.02.19, 17:15 schrieb "Steve Lawrence" <[email protected]>: Negative tests should still work. The PCAP schema [1] has been updated to use 2.3.0 and it has negative tests that pass and didn't require any changes. Can you push your changes somewhere we can take a look? We did make some fairly significant changes to how the TDML runner works, but that should be mostly behind the scenes. Thanks, - Steve On 2/25/19 10:55 AM, Christofer Dutz wrote: > Yeah, it was previously working like that with 2.2.0, when updating I had to > change quite a bit. Now the types and names are aligned with the schema. > > However I can't seem to be able to define rules that expect parse errors (and > therefore don't return any XML) any longer. > > Chris > > Outlook für Android <https://aka.ms/ghei36> herunterladen > > -------------------------------------------------------------------------------- > *From:* Steve Lawrence <[email protected]> > *Sent:* Monday, February 25, 2019 4:50:53 PM > *To:* [email protected]; Christofer Dutz > *Subject:* Re: [VOTE] Release Apache Daffodil (incubating) 2.3.0-rc1 > Looking at your tdml file in github [1], it looks like your root element > in defineSchema is "tpktMessage" with a lower-case "t", but your root > element in the below parserTestCase has a capital "T". Maybe it's just a > typo? > > - Steve > > [1] > https://github.com/apache/incubator-plc4x/blob/develop/protocols/s7/src/test/resources/org/apache/plc4x/protocols/s7/protocol.tdml > > > On 2/25/19 10:40 AM, Christofer Dutz wrote: >> Hi Steve, >> >> so that worked >> >> fn:ceiling(xs:double(../rawLength div 8)) >> >> Now the last thing I'm having a problem with in TDML is that all test-cases where I expect a parsing error - such as: >> >> <!-- >> In this test-case the payload of the packet doesn't match the >> length provided in the packet header (Too short) >> --> >> <tdml:parserTestCase name="TPKT: tooShortPayload" >> root="TpktMessage" >> model="s7Schema"> >> <tdml:document> >> <tdml:documentPart type="byte">0300001611D00001000200C00109C102010</tdml:documentPart> >> </tdml:document> >> >> <tdml:errors> >> <tdml:error>Parse Error</tdml:error> >> </tdml:errors> >> </tdml:parserTestCase> >> >> I am getting error messages like this: >> >> org.apache.daffodil.tdml.TDMLExceptionImpl: (Implementation: daffodil) Did not find diagnostic message "Parse Error" in any of the actual diagnostic messages: >> Schema Definition Error: No global element found for {http://example.com}TpktMessage >> Schema context: SchemaSet Location in file:/var/folders/4m/d8xs244900sd8ypzndkf4_3w0000gn/T/s7Schema1558250055908446006.dfdl.xsd >> Schema Definition Warning: attributeFormDefault is not part of DFDL and will be ignored >> Schema context: XMLSchemaDocument Location line 1 column 2 in file:/var/folders/4m/d8xs244900sd8ypzndkf4_3w0000gn/T/s7Schema1558250055908446006.dfdl.xsd >> >> at org.apache.daffodil.tdml.TDMLException$.apply(TDMLException.scala:31) >> at org.apache.daffodil.tdml.VerifyTestCase$.$anonfun$verifyAllDiagnosticsFound$4(TDMLRunner.scala:1404) >> at org.apache.daffodil.tdml.VerifyTestCase$.$anonfun$verifyAllDiagnosticsFound$4$adapted(TDMLRunner.scala:1393) >> at scala.collection.immutable.List.foreach(List.scala:389) >> >> What do I have to change here? >> >> Chris >> >> >> >> Am 25.02.19, 16:22 schrieb "Steve Lawrence" <[email protected]>: >> >> I'm guessing that this was broken by commit a20ff41728, which changed >> the result of division to always be an xs:decimal instead of an xs:float >> or xs:double. Which is the correct behavior and did fix some potential bugs. >> >> What isn't the correct behavior, which appears to have been broken for a >> while, is rounding of xs:decimals. For example: >> >> fn:ceiling(xs:decimal(0.125)) >> >> which results in 0.125 and is clearly not correct. When you cast that to >> a short, it becomes 0. >> >> Fortunately, it looks like rounding of a float/double works as expected: >> >> fn:ceiling(xs:double(0.125)) >> >> So a temporary workaround is to cast the result of your division to a >> double, e.g: >> >> fn:ceiling(xs:double(../rawLength div 8)) >> >> I'll open a bug for this. >> >> Thanks, >> - Steve >> >> On 2/25/19 9:53 AM, Christofer Dutz wrote: >> > Hi Steve, >> > >> > I don't think it's a blocker either ... >> > >> > However as part of my tests, I updated the dependency in PLC4X to the new Daffodil version. >> > I had to fix quite a bit of things afterwards, however I got most to work. Mostly Daffodil is now a little pickier than previously, however one thing is currently blocking me: >> > >> > The following expression previously evaluated to lengthInBytes=1, now it evaluates to 0 (the transport size is 3 and the rawLength is 1) >> > >> > <xs:element name="lengthInBytes" type="s7:short" dfdl:inputValueCalc="{ >> > xs:unsignedShort(if((../transportSize eq 3) or (../transportSize eq 4) or (../transportSize eq 5)) >> > then xs:short(fn:ceiling(../rawLength div 8)) >> > else ../rawLength)}"/> >> > >> > So in general if should be: >> > >> > lengthInBytes = fn:ceiling(1 div 8) >> > >> > which should be 1 but is evaluated to 0 now ... even if I hard-code it like "fn:ceiling(1 div 8)" the result is 0 >> > >> > Not quite sure why this is happening ... but it's currently breaking my build. >> > >> > Chris >> > >> > >> > Am 25.02.19, 14:35 schrieb "Steve Lawrence" <[email protected]>: >> > >> > Thanks for tracking this issue down. I can reproduce the issue by >> > setting -Duser.country and user.language. This is definitely a bug, >> > anything that affects parsing should come from a DFDL property rather >> > than the environment. I've created DAFFODIL-2074 to track this issue. >> > >> > That said, I don't think this should be considered a blocker for 2.3.0. >> > It should definitely be fixed for next release, but setting the locale >> > is a reasonable workaround for now. >> > >> > - Steve >> > >> > On 2/25/19 5:12 AM, Christofer Dutz wrote: >> > > Hi, >> > > >> > > guess I found the main problem. >> > > It seems to have been related to me not being a US guy ... but wasn't related to the time-zone, but my computer's locale settings. >> > > >> > > Mine default to "de_DE" where the number formats are different. In Germany (and probably some other areas too) we use "," for the fractional separator and "." for thousands separator (which is the opposite to the US convention). >> > > If I add the following settings to my SBT_OPTS: -Duser.country=EN -Duser.language=us >> > > Almost all previously failing tests succeed, however still one still fails: >> > > org.apache.daffodil.section05.simple_types.TestSimpleTypes >> > > >> > > Here looking at the failing tests, it's too probably related. For me it looks as if the default locale settings are not correctly picked up ... >> > > >> > > In the end I'm building with these SBT_OPTS ... probably it's worth adding a suggested SBT_OPTS in the readme as without it, it is impossible to build. >> > > >> > > export SBT_OPTS="-Xss2M -Xmx5G -Xms5G -Duser.country=EN -Duser.language=us" >> > > >> > > >> > > Chris >> > > >> > > >> > > Am 25.02.19, 09:31 schrieb "Christofer Dutz" <[email protected]>: >> > > >> > > Hi guys, >> > > >> > > being back from my trip I was planning on continuing my testing ... especially finding out why some tests are failing on my side. >> > > I'll open Issues for them, if I find something. >> > > >> > > Also did I port some scripts we use in PLC4X and Edgent to do the downloading of release candidates which also do the hash checking and signature checking ... is there any interest in that? >> > > I would be happy to provide them. >> > > >> > > Chris >> > > >> > > >> > > >> > > Am 23.02.19, 10:39 schrieb "Christofer Dutz" <[email protected]>: >> > > >> > > HI and sorry for posting that back to the incubator ... I simply hit reply on my phone and for some reason it added all other emails. >> > > >> > > Do you want me to create issues for anything I found? It's not stuff that is required to be addressed before the release, I think ... but could be helpful for the next. >> > > >> > > Chris >> > > >> > > Am 23.02.19, 01:13 schrieb "Christofer Dutz" <[email protected]>: >> > > >> > > I'll check that out after coming home from my party trip to Utrecht ;-) >> > > >> > > If this is required to build, I think it should be mentioned in the readme. >> > > >> > > Chris >> > > >> > > Outlook für Android<https://aka.ms/ghei36> herunterladen >> > > >> > > ________________________________ >> > > From: Craig Russell <[email protected]> >> > > Sent: Saturday, February 23, 2019 1:12:06 AM >> > > To: Incubator >> > > Cc: [email protected] >> > > Subject: Re: [VOTE] Release Apache Daffodil (incubating) 2.3.0-rc1 >> > > >> > > > On Feb 19, 2019, at 1:18 PM, Dave Fisher <[email protected]> wrote: >> > > > >> > > > Hi - >> > > > >> > > > (1) To be fixed on the next release: >> > > > (a) Please do not include the KEYS file in the source release zip. Refer the users back to the location: https://dist.apache.org/repos/dist/release/incubator/daffodil/KEYS > <https://dist.apache.org/repos/dist/release/incubator/daffodil/KEYS> >> > > >> > > Yes the KEYS must not be distributed in the source release. But the canonical location for KEYS is https://www.apache.org/dist/incubator/daffodil/KEYS > <https://www.apache.org/dist/incubator/daffodil/KEYS> >> > > >> > > The dist.apache.org url is not for public use. >> > > >> > > Craig >> > > >> > > > (b) Discuss vendor specific tests that are name *ibm* and *tresys* on dev@daffodil. I think it would help community building to have something more feature specific, but I could be wrong. >> > > > (2) Rat Check passes. >> > > > (3) All Signatures pass. >> > > > (4) All Checksums pass. >> > > > (5) LICENSE, NOTICE and DISCLAIMER look good. (MSI was not checked. I do not have a windows machine available.) >> > > > (6) Compiles >> > > > (7) sbt test “runs”. I did have a large number of OOM and other errors and they took over 3 hours to run on a MacBook Pro with the latest macOS. >> > > > (8) sbt it:test worked well. >> > > > >> > > > +1 to all packages except for MSI. >> > > > +0 on MSI binary convenience package - I cannot check it. >> > > > >> > > > Regards, >> > > > Dave >> > > > >> > > >> On Feb 19, 2019, at 5:21 AM, Steve Lawrence <[email protected]> wrote: >> > > >> >> > > >> Hi all, >> > > >> >> > > >> The Apache Daffodil community has voted and approved the proposed >> > > >> release of Apache Daffodil (incubating) 2.3.0-rc1. >> > > >> >> > > >> We now kindly request the Incubator PMC members review and vote on this >> > > >> incubator release. >> > > >> >> > > >> Daffodil is an open source implementation of the DFDL specification that >> > > >> uses DFDL schemas to parse fixed format data into an infoset, which is >> > > >> most commonly represented as either XML or JSON. This allows the use of >> > > >> well-established XML or JSON technologies and libraries to consume, >> > > >> inspect, and manipulate fixed format data in existing solutions. >> > > >> Daffodil is also capable of the reverse by serializing or "unparsing" an >> > > >> XML or JSON infoset back to the original data format. >> > > >> >> > > >> Vote thread: >> > > >> https://lists.apache.org/thread.html/0ec1161ad296aebda54e86c219e3f1c4498a7de640c6342e6888aea1@%3Cdev.daffodil.apache.org%3E >> > > >> >> > > >> Result thread: >> > > >> >> > > >> https://lists.apache.org/thread.html/a195a77640235501109bdea5ec7fb9a34c6f50644c7173fb8a919989@%3Cdev.daffodil.apache.org%3E >> > > >> >> > > >> All distribution packages, including signatures, digests, etc. can be >> > > >> found at: >> > > >> >> > > >> https://dist.apache.org/repos/dist/dev/incubator/daffodil/2.3.0-rc1/ >> > > >> >> > > >> Staging artifacts can be found at: >> > > >> >> > > >> https://repository.apache.org/content/repositories/orgapachedaffodil-1008/ >> > > >> >> > > >> This release has been signed with PGP key 36F3494B033AE661, >> > > >> corresponding to [email protected], which is included in the >> > > >> repository's KEYS file. This key can be found on keyservers, such as: >> > > >> >> > > >> http://pgp.mit.edu/pks/lookup?op=get&search=0x36F3494B033AE661 >> > > >> >> > > >> It is also listed here: >> > > >> >> > > >> https://people.apache.org/keys/committer/slawrence.asc >> > > >> >> > > >> The release candidate has been tagged in git with v2.3.0-rc1 on commit >> > > >> hash 193c84690401debd93c1f48e39e564fec09e9337. >> > > >> >> > > >> For reference, here is a list of all closed JIRAs tagged with 2.3.0: >> > > >> >> > > >> https://issues.apache.org/jira/browse/DAFFODIL-2067?jql=project%20%3D%20DAFFODIL%20AND%20status%20%3D%20Resolved%20AND%20fixVersion%20%3D%202.3.0%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC >> > > >> >> > > >> For a summary of the changes in this release, see: >> > > >> >> > > >> https://daffodil.apache.org/releases/2.3.0/ >> > > >> >> > > >> Please review and vote. The vote will be open for at least 72 hours. >> > > >> >> > > >> [ ] +1 approve >> > > >> [ ] +0 no opinion >> > > >> [ ] -1 disapprove (and reason why) >> > > >> >> > > >> Thanks, >> > > >> - Steve >> > > >> >> > > >> --------------------------------------------------------------------- >> > > >> To unsubscribe, e-mail: [email protected] >> > > >> For additional commands, e-mail: [email protected] >> > > >> >> > > > >> > > >> > > Craig L Russell >> > > Secretary, Apache Software Foundation >> > > [email protected] <mailto:[email protected]> http://db.apache.org/jdo <http://db.apache.org/jdo> >> > > >> > > >> > > >> > > >> > > >> > > >> > >> > >> > >> >> >> >
