Brent Nordin created DAFFODIL-2425:
--------------------------------------
Summary: unparse fail with large blob
Key: DAFFODIL-2425
URL: https://issues.apache.org/jira/browse/DAFFODIL-2425
Project: Daffodil
Issue Type: Bug
Components: CLI
Affects Versions: 3.0.0
Reporter: Brent Nordin
Attachments: jpeg.dfdl.xsd, nitfx.dfdl.xsd, test4G.ntf.zip.Z
I am running large NITF files against Daffodil. I can parse the large file, but
Daffodil crashes on the unparse. My casual inspection suggests that a putFile
function needs to accept a long parameter instead of an int.
I am running Apache Daffodil (incubating) 3.0.0-SNAPSHOTÂ
As per https://issues.apache.org/jira/browse/DAFFODIL-2401 I modified the NITF
DFDL Schema to use blobs (instead of HexBinary). The modified Schema (and a
supporting jpeg Schema) is attached here as I have not yet made a pull request
to the NITF DFDL project to have the modifications placed into the main repo.
Be aware that the zipped, compressed test file 'test4G.ntf.zip.Z' will expand
to a 4GB file.
To parse the NITF file:
~/Daffodil/daffodil-cli/target/universal/stage/bin/daffodil parse -I xml -o xxx
-s nitfx.dfdl.xsd test4G.ntf
and then to unparse
~/Daffodil/daffodil-cli/target/universal/stage/bin/daffodil unparse -I xml -o
xxx.nitf -s nitfx.dfdl.xsd xxx
and after some warnings, it's off into the weeds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!! An unexpected exception occurred. This is a bug! !!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Please report this bug and help us fix it:
https://daffodil.apache.org/community/#issue-tracker
Please include the following exception, the command you
ran, and any input, schema, or tdml files used that led
to this bug.
org.apache.daffodil.exceptions.Abort: Invariant broken: nBytesPut.==(nBytes)
org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
org.apache.daffodil.io.DirectOrBufferedDataOutputStream.deliverContent(DirectOrBufferedDataOutputStream.scala:891)
org.apache.daffodil.io.DirectOrBufferedDataOutputStream$.org$apache$daffodil$io$DirectOrBufferedDataOutputStream$$deliverBufferContent(DirectOrBufferedDataOutputStream.scala:1022)
org.apache.daffodil.io.DirectOrBufferedDataOutputStream.setFinished(DirectOrBufferedDataOutputStream.scala:457)
at org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
at
org.apache.daffodil.io.DirectOrBufferedDataOutputStream.deliverContent(DirectOrBufferedDataOutputStream.scala:891)
at
org.apache.daffodil.io.DirectOrBufferedDataOutputStream$.org$apache$daffodil$io$DirectOrBufferedDataOutputStream$$deliverBufferContent(DirectOrBufferedDataOutputStream.scala:1022)
at
org.apache.daffodil.io.DirectOrBufferedDataOutputStream.setFinished(DirectOrBufferedDataOutputStream.scala:457)
at org.apache.daffodil.processors.Suspension.runSuspension(Suspension.scala:89)
at
org.apache.daffodil.processors.Suspension.runSuspension$(Suspension.scala:81)
at
org.apache.daffodil.processors.unparsers.OnlyPaddingUnparserSuspendableOperation.runSuspension(SpecifiedLength2.scala:616)
at
org.apache.daffodil.processors.SuspensionTracker.evalSuspensionQueue(SuspensionTracker.scala:110)
at
org.apache.daffodil.processors.SuspensionTracker.requireFinal(SuspensionTracker.scala:80)
at
org.apache.daffodil.processors.unparsers.UStateMain.evalSuspensions(UState.scala:620)
at
org.apache.daffodil.processors.DataProcessor.unparse(DataProcessor.scala:572)
at
org.apache.daffodil.processors.DataProcessor.unparse(DataProcessor.scala:545)
at org.apache.daffodil.Main$.$anonfun$run$20(Main.scala:1222)
at org.apache.daffodil.util.Timer$.getTimeResult(Timer.scala:78)
at org.apache.daffodil.util.Timer$.getResult(Timer.scala:37)
at org.apache.daffodil.Main$.run(Main.scala:1222)
at org.apache.daffodil.Main$.main(Main.scala:1464)
at org.apache.daffodil.Main.main(Main.scala)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
