There are a couple other open pull requests that should probably be merged for the next reelase, and I suspect there might be at least one or two other bugs that that might also want to be fixed. So I'd guess it's going to take longer than 2 weeks. That's sort of a minimum for a release.
But, I'll start a discuss thread on the dev list and see what the rest of the dev community thinks about for a new release and see what else might be needed before a release, to at least get the ball rolling. - Steve On 6/19/20 8:55 AM, Claude Mamo wrote: > We can wait 2 weeks to release Smooks 2. Alternatively, we can implement some > hack to circumvent the problem and release now. As far as I can tell, the > EDIFACT dictionaries from which the DFDL schemas are generated don't specify > the numeric type. In theory, though it might not make sense, any numeric > field may have a decimal sign and a negative sign. > > Claude > > The fields may have decimal points > > On 2020/06/18 13:31:26, "Beckerle, Mike" <mbecke...@tresys.com> wrote: >> I don't think maxLength is usable on anything but string and hexBinary. >> >> For integers you can use maxInclusive or maxExclusive. >> ________________________________ >> From: Steve Lawrence <slawre...@apache.org> >> Sent: Thursday, June 18, 2020 9:10 AM >> To: users@daffodil.apache.org <users@daffodil.apache.org> >> Subject: Re: java.lang.ArithmeticException: Overflow >> >> Thanks for the contribution! I've taken a look and made a few comments >> in the PR. I think the totalDigits logic is actually much more >> complicated than I originally thought it would be. >> >> Regarding the release, I'm not personally against a patch release, but >> keep in mind that the Daffodil release process is fairly slow. We need >> to start a DISCUSS thread on the dev mailing list which must remain open >> for at least 72 hours and until discussions die down. We then must >> create the release and start a VOTE thread on the dev list. The VOTE >> takes a minimum of 72 hours. If that passes, we must then start a VOTE >> on the incubator mailing list, which also takes a minimum of 72 hours, >> but in my experience takes more like a week. And then the jars take >> about a day or so to sync to maven repositories. So the whole process >> generally takes about 2 weeks. >> >> Depending on how soon you're trying to get a release out, it might make >> sense to see if we can come up with an alternative to totalDigits. >> >> Looking at the schema, to me it looks like the numeric1-10 and >> numeric1-18 simple types that have this totalDigits issue are used for >> fields that imply they are unsigned integer types rather than decimal >> (e.g. NumberOfSegmentsInAMessage, NumberOfSecuritySegments, >> LengthOfDataInOctetsOfBits). Is that the case? Are these fields allowed >> to have decimal points? If not, rather than using an xs:decimal base you >> could use an xs:unsignedInteger base and then use the maxLength >> restriction rather than totalDigits. As far as I'm aware maxLength >> doesn't have the same issue as totalDigits. >> >> On 6/17/20 4:01 PM, Claude Mamo wrote: >>> I gave it a go Steve and opened a PR: >>> https://github.com/apache/incubator-daffodil/pull/387. Assuming it's >>> merged, can we have patch release for this? We are preparing to release the >>> EDI cartridge for Smooks 2 (https://github.com/smooks/smooks-edi-cartridge) >>> but there are many places in our generated EDIFACT schemas with totalDigits >>> > 9. >>> >>> Claude >>> >>> On 2020/06/09 11:41:20, Steve Lawrence <slawre...@apache.org> wrote: >>>> Looks like total digits is just plain broken for anything greater than >>>> or equal to 10, which is pretty bad. Looking at the code I *think* the >>>> totalDigits check will always succeed if the value being validated is >>>> negative, regardless of the number of digits. >>>> >>>> I've created DAFFODIL-2349 [1] to track this issue. >>>> >>>> If you (or anyone else) is interested in getting involved in Daffodil >>>> development, this would be a good one to get your feet wet. The fix >>>> should be pretty self-contained to one file/function. >>>> >>>> Unfortunately, I'm not sure if there's a good workaround using just >>>> restrictions. >>>> >>>> Best I can come up with is have an inputValueCalc that strips out a >>>> negative sign and decimal place, and then restrict that to a length of >>>> 10, e.g. >>>> >>>> <xs:element name="unscaled" dfdl:inputValueCalc="{ >>>> fn:concat( >>>> fn:substring-before(xs:string(fn:abs(../value)), '.'), >>>> fn:substring-after(xs:string(../value), '.') >>>> ) >>>> }"> >>>> <xs:simpleType> >>>> <xs:restriction base="xs:string"> >>>> <x:length value="10"/> >>>> </xs:restriction> >>>> </xs:simpleType> >>>> </xs:element> >>>> >>>> That's pretty terrible though. Maybe someone else can come up with a >>>> better workaround? >>>> >>>> [1] https://issues.apache.org/jira/browse/DAFFODIL-2349 >>>> >>>> >>>> On 6/9/20 2:29 AM, Claude Mamo wrote: >>>>> Hello Daffodil team, >>>>> >>>>> Not sure if what I'm getting is expected behaviour. I have a facet >>>>> defined as follows: >>>>> >>>>> ``` >>>>> <xsd:simpleType dfdl:textNumberPattern="#.#" name="numeric1-10"> >>>>> <xsd:restriction base="xsd:decimal"> >>>>> <xsd:totalDigits value="10"/> >>>>> </xsd:restriction> >>>>> </xsd:simpleType> >>>>> ``` >>>>> >>>>> When attempting to parse a file with full validation turned on, Daffodil >>>>> 2.6 throws an exception saying: >>>>> >>>>> ``` >>>>> org.apache.daffodil.exceptions.Abort: >>>>> Invariant broken. Exception thrown with mark not returned: >>>>> java.lang.ArithmeticException: Overflow >>>>> StackTrace: >>>>> java.lang.ArithmeticException: Overflow >>>>> at java.math.BigDecimal.intValueExact(BigDecimal.java:3180) >>>>> at >>>>> org.apache.daffodil.processors.SimpleTypeRuntimeData.checkTotalDigits(RuntimeData.scala:526) >>>>> at >>>>> org.apache.daffodil.processors.SimpleTypeRuntimeData.$anonfun$executeFacetCheck$8(RuntimeData.scala:431) >>>>> at >>>>> org.apache.daffodil.processors.SimpleTypeRuntimeData.$anonfun$executeFacetCheck$8$adapted(RuntimeData.scala:427) >>>>> ``` >>>>> >>>>> Should I create a bug report? Any suitable alternatives to "totalDigits"? >>>>> >>>>> Claude >>>>> >>>>> >>>>> >>>> >>>> >> >>
