Thanks for reporting. How much memory have you given to your JVM? And
what version of Daffodil are you using?
It looks like the deliberate junk value you reference is about 1.5 GB.
So Daffodil will try to create a 1.5GB array to store the hex binary,
and if you don't have enough memory it will result in the OOM exception.
There are a couple of solutions here:
1) Put an assert on the length field to ensure it is a reasonable size.
For example:
<xs:element name="MDO_MovieDataSize" type="xs:int" ...>
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/";>
<dfdl:assert>{ . le 1000000 }</dfdl:assert>
</xs:appinfo>
</xs:annotation>
</xs:element>
2) Similar to above, put an xs:restriction on the length field to ensure
it's a reasonable size, add an assert to check that restriction:
<xs:element name="MDO_MovieDataSize" ...>
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/";>
<dfdl:assert>{ dfdl:checkConstraints(.) }</dfdl:assert>
</xs:appinfo>
</xs:annotation>
<xs:simpleType>
<xs:restriction base="xs:int">
<xs:maxInclusive value="1000000" />
</xs:restriction>
</xs:simpleType>
</xs:element>
3) Set the "maxHexBinaryLengthInBytes" tunable. This will create a
processing error if the length of a hexBinary field is larger than that
tunable. This value defaults to 2GB, but could be set to a lower value
if you know your hex binary will never be that large.
- Steve
On 8/3/22 6:40 PM, Thompson, Mark M [US] (DS) wrote:
All,
I am reporting the occurrence of an exception as requested. The attached file
contains a trace of the Exception with
what I believe the relevant info.
* Command executed
daffodil -t -vv parse -s ..\..\..\MPRemote.dfdl.xsd -V limited -o
image_file_name-8.invalid.dfdl.xml -r MP_REMOTE_Fields image_file_name-8.invalid
* It appears that Daffodil does not like large values (deliberate junk in
this
case) when used as a size for an xs:hexBinary element. See
<MDO_MovieDataSize> and <MDO_MovieData> respectively in the attached trace. In
this case, there is nowhere near that much data available
In the test binary input. Normally, in the case of insufficient data, Daffodil
errors out gracefully and indicates that there was insufficient dat.
Overview:
* I am not at liberty to provide the actual schema files.
* I may be able to provide test messages if necessary. I’m hoping that the
attached trace provides more than enough info.
* The Input test files to Daffodil are binary.
* The command used in this case is listed above.
* Command line options: -t -vv parse -V limited
* Daffodil version: 3.2.1
* Offending element: <MDO_MovieData>
Thank you for your time,
Mark M. Thompson
Northrop Grumman Defense Systems
Software Engineer
(818) 712-7439
