Re: [VOTE] Release Apache Daffodil 3.3.0-rc1
+1 I checked: [OK] links in email are correct [OK] verified Summary of Changes pages (maven, sbt, download and dnf) [OK] verified download of binary [OK] Successfully used apache-daffodil-3.3.0-bin.tgz [OK] JavaDoc and ScalaDoc are correct [OK] RAT check passes On 3/18/22 12:06, Interrante, John A (GE Research, US) wrote: Hi PMC members, I'd like to call a vote to release Apache Daffodil 3.3.0-rc1. All distribution packages, including signatures, digests, etc. can be found at: https://dist.apache.org/repos/dist/dev/daffodil/3.3.0-rc1/ Staging artifacts can be found at: https://repository.apache.org/content/repositories/orgapachedaffodil-1029/ This release has been signed with PGP key 04A735FC1A36AE84, corresponding to jinterra...@apache.org, which is included in the KEYS file here: https://downloads.apache.org/daffodil/KEYS The release candidate has been tagged in git with v3.3.0-rc1. For reference, here is a list of all closed JIRAs tagged with 3.3.0: https://s.apache.org/daffodil-issues-3.3.0 For a summary of the changes in this release, see: https://daffodil.apache.org/releases/3.3.0/ Please review and vote. The vote will be open for at least 72 hours (Monday, March 21 2022, 12 Noon EST). [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) Thanks, John -- Best Regards, Lola K.
Re: sbt exportJars causes "lengthLimitInBits.>=(0)" error when running "sbt test"
I was also able to recreate this error in the CSV repo by adding the "exportJars := true" (note: false doesn't cause this error), and updating the Daffodil version to the last commit before the RC commit (i.e 613c2b0c077f56dbb425e7ebab1a2d154cbdbd06). so: exportJars :=true libraryDependencies ++=Seq( "org.apache.daffodil" %%"daffodil-tdml-processor" %"3.3.0-SNAPSHOT" %"test", --- ) So it seems to be a Daffodil error, since this doesn't happen with 3.0.0, but it does happen with 3.2.1 Note: I'm running sbt 1.5.7 (Ubuntu Java 11.0.14), but I believe Mike tried several different versions when we first ran into this. On 3/18/22 13:45, Mike Beckerle wrote: I tried many things, including clean git clones of all related repos. This problem is a very bizarre interaction. The fact that the tests ran flawlessly under intelliJ IDEA, but failed running sbt from the command line I searched for sbt bugs related to exportJars. I did find https://github.com/sbt/sbt/issues/3119, but doesn't seem related or not obviously so. On Fri, Mar 18, 2022 at 1:25 PM Adams, Joshua wrote: I can confirm that at least sbt clean does not resolve the issue. Didn't try a got clean, although I had just cloned the repo in question so I don't think it had gotten dirty in anyway either. Josh On Mar 18, 2022 1:22 PM, Steve Lawrence wrote: That's surprising to me. All that exportJars is supposed to to do is include or not include certains files on the class path and use packages jars instead. And that assertion that is failing is beacuase the length of test data is negative, which should never happen and I don't know how exportJars would affect that. It almost sounds like something got corrupted or needed to be rebuilt, and changing exportJars allowed that to happen? Maybe there's some conflict between using IntelliJ and sbt at the same time? I wonder if doing an sbt clean or git clean would resolve the issue. On 3/18/22 1:10 PM, Olabusayo Kilo wrote: We noticed that only when running "sbt test" in schema projects with the exportJar setting, the error below was generated resulting in failure of all the tests. Note that running the same tests via IntelliJ/Junit didn't reproduce this error, and the error was isolated to using sbt to run the tests. The cause was isolated to having the "exportJars := true" setting in build.sbt. Removing the setting removed the error, and got all tests passing. Sending this out as an FYI in case someone else runs into this in the future. [info] Test a.b.c started [error] Test a.b.c failed: org.apache.daffodil.exceptions.Abort: Usage error: lengthLimitInBits.>=(0) [error] org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:137) [error] org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.doParseWithBothApis(DaffodilTDMLDFDLProcessor.scala:290) [error] org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.parse(DaffodilTDMLDFDLProcessor.scala:248) [error] org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.parse(DaffodilTDMLDFDLProcessor.scala:253), took 0.003 sec [error] at org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:137) [error] at org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.doParseWithBothApis(DaffodilTDMLDFDLProcessor.scala:290) [error] at org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.parse(DaffodilTDMLDFDLProcessor.scala:248) [error] at org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.parse(DaffodilTDMLDFDLProcessor.scala:253) [error] at org.apache.daffodil.tdml.ParserTestCase.doParseExpectSuccess(TDMLRunner.scala:1026) [error] at org.apache.daffodil.tdml.ParserTestCase.runParseExpectSuccess(TDMLRunner.scala:1169) [error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2(TDMLRunner.scala:920) [error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2$adapted(TDMLRunner.scala:917) [error] at scala.util.Either$RightProjection.foreach(Either.scala:652) [error] at org.apache.daffodil.tdml.ParserTestCase.runProcessor(TDMLRunner.scala:917) [error] at org.apache.daffodil.tdml.TestCase.run(TDMLRunner.scala:870) [error] at org.apache.daffodil.tdml.DFDLTestSuite.runOneTest(TDMLRunner.scala:416) [error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:191) [error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:197)
sbt exportJars causes "lengthLimitInBits.>=(0)" error when running "sbt test"
We noticed that only when running "sbt test" in schema projects with the exportJar setting, the error below was generated resulting in failure of all the tests. Note that running the same tests via IntelliJ/Junit didn't reproduce this error, and the error was isolated to using sbt to run the tests. The cause was isolated to having the "exportJars := true" setting in build.sbt. Removing the setting removed the error, and got all tests passing. Sending this out as an FYI in case someone else runs into this in the future. [info] Test a.b.c started [error] Test a.b.c failed: org.apache.daffodil.exceptions.Abort: Usage error: lengthLimitInBits.>=(0) [error] org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:137) [error] org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.doParseWithBothApis(DaffodilTDMLDFDLProcessor.scala:290) [error] org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.parse(DaffodilTDMLDFDLProcessor.scala:248) [error] org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.parse(DaffodilTDMLDFDLProcessor.scala:253), took 0.003 sec [error] at org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:137) [error] at org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.doParseWithBothApis(DaffodilTDMLDFDLProcessor.scala:290) [error] at org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.parse(DaffodilTDMLDFDLProcessor.scala:248) [error] at org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.parse(DaffodilTDMLDFDLProcessor.scala:253) [error] at org.apache.daffodil.tdml.ParserTestCase.doParseExpectSuccess(TDMLRunner.scala:1026) [error] at org.apache.daffodil.tdml.ParserTestCase.runParseExpectSuccess(TDMLRunner.scala:1169) [error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2(TDMLRunner.scala:920) [error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2$adapted(TDMLRunner.scala:917) [error] at scala.util.Either$RightProjection.foreach(Either.scala:652) [error] at org.apache.daffodil.tdml.ParserTestCase.runProcessor(TDMLRunner.scala:917) [error] at org.apache.daffodil.tdml.TestCase.run(TDMLRunner.scala:870) [error] at org.apache.daffodil.tdml.DFDLTestSuite.runOneTest(TDMLRunner.scala:416) [error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:191) [error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:197)
Re: [VOTE] Release Apache Daffodil 3.2.0-rc1
+1 (binding) Ah, ok! On 12/3/21 5:48 PM, Steve Lawrence wrote: The RPM isn't pushed to the artifactory repo until after the vote passes. Otherwise if people ran dnf update they might install something that hasn't been verified before the vote closed. So that's normal. On 12/3/21 5:05 PM, Olabusayo Kilo wrote: +1 (as long as absence of 3.2.0-1 rpm in artifactory is addressed) I checked: [OK] links in email are correct [OK] verified Summary of Changes pages (maven, sbt, download and dnf) [OK] verified download of binary [QUESTION] verified daffodil-rpm folder on dnf baseUrl site. Are we supposed to be about to see 3.2.0-1 on there, because it's not there right now [OK] JavaDoc and ScalaDoc are correct [OK] RAT check passes On 12/1/21 2:03 PM, Mike Beckerle wrote: Hi all, I'd like to call a vote to release Apache Daffodil 3.2.0-rc1. All distribution packages, including signatures, digests, etc. can be found at: https://dist.apache.org/repos/dist/dev/daffodil/3.2.0-rc1/ Staging artifacts can be found at: https://repository.apache.org/content/repositories/orgapachedaffodil-1025/ This release has been signed with PGP key 13A680AF, corresponding to mbecke...@apache.org, which is included in the KEYS file here: https://downloads.apache.org/daffodil/KEYS The release candidate has been tagged in git with v3.2.0-rc1. For reference, here is a list of all closed JIRAs tagged with 3.2.0: https://s.apache.org/daffodil-issues-3.2.0 For a summary of the changes in this release, see: https://daffodil.apache.org/releases/3.2.0/ Please review and vote. The vote will be open for at least 72 hours (Monday December 6, 14:00 EST.US). [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) Thanks, - Mike Beckerle -- Best Regards, Lola K.
Re: [VOTE] Release Apache Daffodil 3.2.0-rc1
+1 (as long as absence of 3.2.0-1 rpm in artifactory is addressed) I checked: [OK] links in email are correct [OK] verified Summary of Changes pages (maven, sbt, download and dnf) [OK] verified download of binary [QUESTION] verified daffodil-rpm folder on dnf baseUrl site. Are we supposed to be about to see 3.2.0-1 on there, because it's not there right now [OK] JavaDoc and ScalaDoc are correct [OK] RAT check passes On 12/1/21 2:03 PM, Mike Beckerle wrote: Hi all, I'd like to call a vote to release Apache Daffodil 3.2.0-rc1. All distribution packages, including signatures, digests, etc. can be found at: https://dist.apache.org/repos/dist/dev/daffodil/3.2.0-rc1/ Staging artifacts can be found at: https://repository.apache.org/content/repositories/orgapachedaffodil-1025/ This release has been signed with PGP key 13A680AF, corresponding to mbecke...@apache.org, which is included in the KEYS file here: https://downloads.apache.org/daffodil/KEYS The release candidate has been tagged in git with v3.2.0-rc1. For reference, here is a list of all closed JIRAs tagged with 3.2.0: https://s.apache.org/daffodil-issues-3.2.0 For a summary of the changes in this release, see: https://daffodil.apache.org/releases/3.2.0/ Please review and vote. The vote will be open for at least 72 hours (Monday December 6, 14:00 EST.US). [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) Thanks, - Mike Beckerle -- Best Regards, Lola K.
Re: [VOTE] Release Apache Daffodil 3.1.0-rc2
+1 I checked: [OK] links in email are correct [OK] verified Summary of Changes pages (maven, sbt and download) [OK] JavaDoc and ScalaDoc are correct [OK] RAT check passes On 5/14/21 3:26 PM, Steve Lawrence wrote: Hi all, I'd like to call a vote to release Apache Daffodil 3.1.0-rc2. All distribution packages, including signatures, digests, etc. can be found at: https://dist.apache.org/repos/dist/dev/daffodil/3.1.0-rc2/ Staging artifacts can be found at: https://repository.apache.org/content/repositories/orgapachedaffodil-1023/ This release has been signed with PGP key 36F3494B033AE661, corresponding to slawre...@apache.org, which is included in the KEYS file here: https://downloads.apache.org/daffodil/KEYS The release candidate has been tagged in git with v3.1.0-rc2. For reference, here is a list of all closed JIRAs tagged with 3.1.0: https://s.apache.org/daffodil-issues-3.1.0 For a summary of the changes in this release, see: https://daffodil.apache.org/releases/3.1.0/ Please review and vote. The vote will be open for at least 72 hours (Monday, 17 May 2021, 4pm EST). [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) -- Best Regards, Lola K.
Sonarcloud wiki page
FYI, I added some preliminary notes [1] on the wiki for Static Analysis with Sonarcloud. For now, it just has some links and the steps to perform static analysis on your local machine using sonarqube and sonar-scanner. The plan is to gradually fill it out with more detail of how we use Sonarcloud.io. [1] https://cwiki.apache.org/confluence/display/DAFFODIL/Static+Analysis%3A+Sonarcloud -- Best Regards, Lola K.
Running CLI Tests on IntelliJ and other Editor Notes
Hello all, I recently upgraded from Intellij Idea Community to the Ultimate edition, and found that I could finally run CLI tests, without having to make any additional configuration changes. I updated the intellij wiki page [1] to describe ways you can get a free Ultimate license. If you have any Intellij, Eclipse or other editor notes, please feel free to update existing or create new pages under Editor Notes [2] to help others benefit from your knowledge. [1] https://cwiki.apache.org/confluence/display/DAFFODIL/IntelliJ+IDEA+Setup [2] https://cwiki.apache.org/confluence/display/DAFFODIL/Editor+Notes -- Best Regards, Lola K.
Re: [VOTE] Release Apache Daffodil (incubating) 3.0.0-rc1
+1 I checked: [OK] links in email are correct [OK] verified Summary of Changes pages (maven, sbt and download) [OK] JavaDoc and ScalaDoc are correct [NOT OK] RAT check passes; .bsp/sbt.json file? [OK] ~50 public and private DFDL schema projects pass tests On 10/30/20 3:23 PM, Steve Lawrence wrote: Hi all, I'd like to call a vote to release Apache Daffodil (incubating) 3.0.0-rc1. All distribution packages, including signatures, digests, etc. can be found at: https://dist.apache.org/repos/dist/dev/incubator/daffodil/3.0.0-rc1/ Staging artifacts can be found at: https://repository.apache.org/content/repositories/orgapachedaffodil-1019/ This release has been signed with PGP key 36F3494B033AE661, corresponding to slawre...@apache.org, which is included in the KEYS file here: https://downloads.apache.org/incubator/daffodil/KEYS The release candidate has been tagged in git with v3.0.0-rc1. For reference, here is a list of all closed JIRAs tagged with 3.0.0: https://s.apache.org/daffodil-issues-3.0.0 For a summary of the changes in this release, see: https://daffodil.apache.org/releases/3.0.0/ Please review and vote. The vote will be open for at least 72 hours (Monday, 2 November 2020, 3:30 EST). [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) Thanks, - Steve -- Best Regards, Lola K.
Re: Coroutines - was Re: Daffodil SAX API Proposal
I don't think we came to a conclusion on which path we should take. If I
understand correctly, our options seem to be between the Thread-based
Coroutine library (#3; which has a bit of overhead) and the
Continuations library (#2; which is not yet supported for 2.13 and
requires the suspendable annotation). I wanted to check in to see if
there was a preferred one that I could focus my effort on?
On 4/24/20 9:28 AM, Beckerle, Mike wrote:
A further thought on this. The overhead difference between continuations and
threads was 1 to 4 (roughly).
If you add real workload to what happens on either side of that
producer-consumer relationship, I bet this difference disappears into the
noise, not because it becomes more efficient due to less contention, but
because it's such a tiny fraction of the actual work being done.
The Thread-based coroutines library, I have a copy of in a separate sandbox, so
if you want to grab that I'll get it over to you so you don't have to dig for
it.
From: Beckerle, Mike
Sent: Friday, April 24, 2020 8:53 AM
To: dev@daffodil.apache.org
Subject: Re: Coroutines - was Re: Daffodil SAX API Proposal
That's really informative and confirms intuition that using threads really
hurts performance when all you need is a stack switch.
In this case reducing contention should reduce total work, but that depends on
how carefully the queue is implemented. If it is a single lock it may not
matter.
We actually dont care about faster through parallelism because we should assume
the machine is already saturated with work. We want to reduce total amount of
work done.
From: Steve Lawrence
Sent: Friday, April 24, 2020 8:02:37 AM
To: dev@daffodil.apache.org
Subject: Re: Coroutines - was Re: Daffodil SAX API Proposal
I decided to look at performance of three potential options to see if
that would rule anything out. I looked at 1) coroutines 2) continuations
3) threads with BlockingQueue. For each of these, I modified the gist to
remove printlns and use a different producer consumer model (which is
actually very straightforward in we come across other alternatives to
test). So everything is the same except for how the SAX content handler
interacts with the custom InfosetInputter. For the performance numbers
below, I created enough "events" in a loop so that rate of events
remained roughly the same as I increased the number of events.
1) coroutines
It turns out the coroutines library has a limitation where the
yieldval() call must be directly inside the coroutine{} block. This is
basically a non-starter for us, since the entire unparse call needs to
be a coroutine, and the yieldval call happens way down the stack. So not
only does this not have any active development, it functionally won't
even work for us.
2) continuations
16.50 million events per second
3) thread with BlockingQueue
I think this is similar to the Coroutine library you wrote for Daffodil
(though it looks like it's been removed, we can probably find it in git
the history if we want). This runs the unparse method in a thread and
has a blocking queue that the producer pushes to and the consumer takes
from. I tested with different queue sizes to see how that affects
performance:
size rate
1 0.14 million events per second
10 1.36 million events per second
100 3.18 million events per second
1000 3.16 million events per second
10 3.09 million events per second
So this BlockinQueue approach is quite a bit slower, and definitely
requires batching events to be somewhat performant. I guess this
slowness makes sens as this approach creates a thread for the unparse,
has different threads blocking on this queue, and also creates a bunch
of event objects to put in the queue (the continuation approach just
mutates state so no extra objects are needed). It is possible that this
isn't an accurate test since the producer is going crazy fast since I'm
just incrementing a Long in each loop iteration. In the real world, the
producer is going to be parsing XML or something, so won't be as fast.
Perhaps if the producer was actually slower there would be less thread
contention and actually allow for more parallel work?
On 4/23/20 5:41 PM, Beckerle, Mike wrote:
I am pretty worried about the @suspendable annotation. The way this shift/reset
stuff works is it modifies the scala compiler to do something called
continuation passing style. aka CPS.
I'd be ok if that was isolated to just a segment of the code. Maybe there is
some natural way to do that?
But it seems to me that all code on the pathway from where a reset block is
entered to where a shift is called, all of it has to propagate this
@suspendable behavior and be compiled by way of this CPS plug in. That looks ok
for the tiny toy examples, but for a giant code base like Daffodil runtime1
unparser, that seems fragile, potentially has impact on debugging, memory
Re: Coroutines - was Re: Daffodil SAX API Proposal
Hey Mike, can you send a copy for the Thread-based coroutines library
referenced below?
On 4/24/20 9:28 AM, Beckerle, Mike wrote:
A further thought on this. The overhead difference between continuations and
threads was 1 to 4 (roughly).
If you add real workload to what happens on either side of that
producer-consumer relationship, I bet this difference disappears into the
noise, not because it becomes more efficient due to less contention, but
because it's such a tiny fraction of the actual work being done.
The Thread-based coroutines library, I have a copy of in a separate sandbox, so
if you want to grab that I'll get it over to you so you don't have to dig for
it.
From: Beckerle, Mike
Sent: Friday, April 24, 2020 8:53 AM
To: dev@daffodil.apache.org
Subject: Re: Coroutines - was Re: Daffodil SAX API Proposal
That's really informative and confirms intuition that using threads really
hurts performance when all you need is a stack switch.
In this case reducing contention should reduce total work, but that depends on
how carefully the queue is implemented. If it is a single lock it may not
matter.
We actually dont care about faster through parallelism because we should assume
the machine is already saturated with work. We want to reduce total amount of
work done.
From: Steve Lawrence
Sent: Friday, April 24, 2020 8:02:37 AM
To: dev@daffodil.apache.org
Subject: Re: Coroutines - was Re: Daffodil SAX API Proposal
I decided to look at performance of three potential options to see if
that would rule anything out. I looked at 1) coroutines 2) continuations
3) threads with BlockingQueue. For each of these, I modified the gist to
remove printlns and use a different producer consumer model (which is
actually very straightforward in we come across other alternatives to
test). So everything is the same except for how the SAX content handler
interacts with the custom InfosetInputter. For the performance numbers
below, I created enough "events" in a loop so that rate of events
remained roughly the same as I increased the number of events.
1) coroutines
It turns out the coroutines library has a limitation where the
yieldval() call must be directly inside the coroutine{} block. This is
basically a non-starter for us, since the entire unparse call needs to
be a coroutine, and the yieldval call happens way down the stack. So not
only does this not have any active development, it functionally won't
even work for us.
2) continuations
16.50 million events per second
3) thread with BlockingQueue
I think this is similar to the Coroutine library you wrote for Daffodil
(though it looks like it's been removed, we can probably find it in git
the history if we want). This runs the unparse method in a thread and
has a blocking queue that the producer pushes to and the consumer takes
from. I tested with different queue sizes to see how that affects
performance:
size rate
1 0.14 million events per second
10 1.36 million events per second
100 3.18 million events per second
1000 3.16 million events per second
10 3.09 million events per second
So this BlockinQueue approach is quite a bit slower, and definitely
requires batching events to be somewhat performant. I guess this
slowness makes sens as this approach creates a thread for the unparse,
has different threads blocking on this queue, and also creates a bunch
of event objects to put in the queue (the continuation approach just
mutates state so no extra objects are needed). It is possible that this
isn't an accurate test since the producer is going crazy fast since I'm
just incrementing a Long in each loop iteration. In the real world, the
producer is going to be parsing XML or something, so won't be as fast.
Perhaps if the producer was actually slower there would be less thread
contention and actually allow for more parallel work?
On 4/23/20 5:41 PM, Beckerle, Mike wrote:
I am pretty worried about the @suspendable annotation. The way this shift/reset
stuff works is it modifies the scala compiler to do something called
continuation passing style. aka CPS.
I'd be ok if that was isolated to just a segment of the code. Maybe there is
some natural way to do that?
But it seems to me that all code on the pathway from where a reset block is
entered to where a shift is called, all of it has to propagate this
@suspendable behavior and be compiled by way of this CPS plug in. That looks ok
for the tiny toy examples, but for a giant code base like Daffodil runtime1
unparser, that seems fragile, potentially has impact on debugging, memory
allocation, and performance of the code, and,... well given the lack of
enthusiastic support for shift/reset I think it is risky.
The only other option I can think of is to spawn a separate thread, allow true
concurrency in a producer-consumer model.
We already have a Coroutines library you may recall. We're not
[ANNOUNCE] Apache Daffodil (incubating) 2.6.0 Released
The Apache Daffodil (incubating) community is pleased to announce the release of version 2.6.0. Notable changes in this release include performance and API updates, and the addition of Sonarqube analysis to our workflow. Detailed release notes and downloads are available at: https://daffodil.apache.org/releases/2.6.0/ Daffodil is an open source implementation of the Data Format Description Language (DFDL) specification that uses DFDL schemas to parse fixed format data into an infoset, which is most commonly represented as either XML or JSON. This allows the use of well-established XML or JSON technologies and libraries to consume, inspect, and manipulate fixed format data in existing solutions. Daffodil is also capable of the reverse by serializing or "unparsing" an XML or JSON infoset back to the original data format. For more information about Daffodil visit: https://daffodil.apache.org/ Regards, The Apache Daffodil Team = *Disclaimer* Apache Daffodil is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator PMC. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
[RESULT][VOTE] Release Apache Daffodil (incubating) 2.6.0-rc2
The VOTE to release Apache Daffodil (incubating) 2.6.0-rc2 is now closed. The vote passes with 5 +1 votes, and NO -1 votes. The VOTE thread: https://lists.apache.org/thread.html/r25781e7da23f4d0318ca6fa316c8d4bfb244e441fa4d722be4010be2%40%3Cdev.daffodil.apache.org%3E +1's were: Steve Lawrence Mike Beckerle Joshua Adams Dave Thompson Thanks to everyone who voted. I'll open the VOTE thread on the Incubator general list shortly. -- Best Regards Lola K.
Re: [VOTE] Release Apache Daffodil (incubating) 2.6.0-rc2
Thanks all. This VOTE is now closed. I'll send out a result vote shortly. On 4/13/20 3:27 PM, Thompson, Dave wrote: +1 Verified and closed all v2.6.0 resolved JIRA tickets. -Original Message- From: Olabusayo Kilo Sent: Thursday, April 9, 2020 1:06 PM To: dev@daffodil.apache.org Subject: [VOTE] Release Apache Daffodil (incubating) 2.6.0-rc2 Hi all, I'd like to call a vote to release Apache Daffodil (incubating) 2.6.0-rc2. All distribution packages, including signatures, digests, etc. can be found at: https://dist.apache.org/repos/dist/dev/incubator/daffodil/2.6.0-rc2/ Staging artifacts can be found at: https://repository.apache.org/content/repositories/orgapachedaffodil-1015 This release has been signed with PGP key 639637FDA8049411, corresponding to olabus...@apache.org, which is included in the KEYS file here: https://downloads.apache.org/incubator/daffodil/KEYS The release candidate has been tagged in git with v2.6.0-rc2. For reference, here is a list of all closed JIRAs tagged with 2.6.0: https://s.apache.org/daffodil-issues-2.6.0 For a summary of the changes in this release, see: https://daffodil.apache.org/releases/2.6.0/ Please review and vote. The vote will be open for at least 72 hours (ends on Sunday, 12 April 2020, 12 Noon EST). [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) -- Best Regards Lola K. -- Best Regards Lola K.
Re: [VOTE] Release Apache Daffodil (incubating) 2.6.0-rc2
Noted, hyperlink has been fixed. On 4/10/20 1:43 PM, Beckerle, Mike wrote: +1 from me. I checked that the checked out code builds and runs all built in tests. I inspected the checked out tree and found nothing surprising in it. Rather than re-inspect to double-check things other people have checked I decided to re-verify the essential functionality of the release, which is the speedup and reduced memory footprint of the DFDL schema compiler. I wanted to be sure nothing in the more recent commits had caused regression of this. In particular I tested that some of our very largest DFDL schemas, which used to compile in many minutes and required many gigs of RAM to do so on Daffodil 2.5.0, now behave much better in 2.6.0. I am satisfied that nothing has regressed. Schema EDIFACT works with 'sbt -mem 320 test'. That's only 320Meg of RAM. I verified this with visualvm. It takes about 20 seconds to compile the schema. It won't go much lower than that "-mem 256' failed with GC overhead/out-of-memory conditions. Schema stanag-5516-with-nact works with 'sbt -mem 1024 test' Takes about 1 minute to compile and run the slowest test, which is still very slow, but is far less than it was. BTW: this schema I modified to include *every* message. In the past due to the long compilation times we had commented out all but a handful of messages, and it still took many minutes to compile. This schema does seem to need roughly this "-mem 1024' (which is 1 gigabyte) memory level. It failed with 'sbt -mem 768 test'. I did spot an issue not with the release per-se, but with the release page for our site. It's a nit, but there is a hyperlink mid page that should take you to the Deprecation/Compatibility section at the bottom, and that hyperlink does not work. -mike beckerle On 4/9/20 1:05 PM, Olabusayo Kilo wrote: Hi all, I'd like to call a vote to release Apache Daffodil (incubating) 2.6.0-rc2. All distribution packages, including signatures, digests, etc. can be found at: https://dist.apache.org/repos/dist/dev/incubator/daffodil/2.6.0-rc2/ Staging artifacts can be found at: https://repository.apache.org/content/repositories/orgapachedaffodil-1015 This release has been signed with PGP key 639637FDA8049411, corresponding to olabus...@apache.org, which is included in the KEYS file here: https://downloads.apache.org/incubator/daffodil/KEYS The release candidate has been tagged in git with v2.6.0-rc2. For reference, here is a list of all closed JIRAs tagged with 2.6.0: https://s.apache.org/daffodil-issues-2.6.0 For a summary of the changes in this release, see: https://daffodil.apache.org/releases/2.6.0/ Please review and vote. The vote will be open for at least 72 hours (ends on Sunday, 12 April 2020, 12 Noon EST). [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) -- Best Regards Lola K.
[VOTE] Release Apache Daffodil (incubating) 2.6.0-rc2
Hi all, I'd like to call a vote to release Apache Daffodil (incubating) 2.6.0-rc2. All distribution packages, including signatures, digests, etc. can be found at: https://dist.apache.org/repos/dist/dev/incubator/daffodil/2.6.0-rc2/ Staging artifacts can be found at: https://repository.apache.org/content/repositories/orgapachedaffodil-1015 This release has been signed with PGP key 639637FDA8049411, corresponding to olabus...@apache.org, which is included in the KEYS file here: https://downloads.apache.org/incubator/daffodil/KEYS The release candidate has been tagged in git with v2.6.0-rc2. For reference, here is a list of all closed JIRAs tagged with 2.6.0: https://s.apache.org/daffodil-issues-2.6.0 For a summary of the changes in this release, see: https://daffodil.apache.org/releases/2.6.0/ Please review and vote. The vote will be open for at least 72 hours (ends on Sunday, 12 April 2020, 12 Noon EST). [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) -- Best Regards Lola K.
Re: [VOTE] Release Apache Daffodil (incubating) 2.6.0-rc1
(Resending due to issue with weird formatting/pipes in previous message) Hi all, I'd like to call a vote to release Apache Daffodil (incubating) 2.6.0-rc1. All distribution packages, including signatures, digests, etc. can be found at: https://dist.apache.org/repos/dist/dev/incubator/daffodil/2.6.0-rc1/ Staging artifacts can be found at: https://repository.apache.org/content/repositories/orgapachedaffodil-1014/ This release has been signed with PGP key 639637FDA8049411, corresponding to olabus...@apache.org, which is included in the KEYS file here: https://downloads.apache.org/incubator/daffodil/KEYS The release candidate has been tagged in git with v2.6.0-rc1. For reference, here is a list of all closed JIRAs tagged with 2.0.0: https://s.apache.org/daffodil-issues-2.6.0 For a summary of the changes in this release, see: https://daffodil.apache.org/releases/2.6.0/ Please review and vote. The vote will be open for at least 72 hours (ends on Friday, 10 April 2020, 3 PM EST). [ ] +1 approve [ ] +0 no opinion [ ] -1 disapprove (and reason why) -- Best Regards Lola K.
[VOTE] Release Apache Daffodil (incubating) 2.6.0-rc1
|Hi all,| |I'd like to call a vote to release Apache Daffodil (incubating) 2.6.0-rc1.| |All distribution packages, including signatures, digests, etc. can be| |found at:| https://dist.apache.org/repos/dist/dev/incubator/daffodil/2.6.0-rc1/ |Staging artifacts can be found at:| || |https://repository.apache.org/content/repositories/orgapachedaffodil-1014/| |This release has been signed with PGP key 639637FDA8049411, corresponding| |to olabus...@apache.org, which is included in the KEYS file here:| https://downloads.apache.org/incubator/daffodil/KEYS || |The release candidate has been tagged in git with v2.6.0-rc1.| |For reference, here is a list of all closed JIRAs tagged with 2.0.0:| https://s.apache.org/daffodil-issues-2.6.0 || |For a summary of the changes in this release, see:| https://daffodil.apache.org/releases/2.6.0/ || |Please review and vote. The vote will be open for at least 72 hours| |(ends on Friday, 10 April 2020, 3 PM EST).| |[ ] +1 approve| |[ ] +0 no opinion| |[ ] -1 disapprove (and reason why)| -- Best Regards Lola K.
Re: [DISCUSS] Daffodil 2.6.0 Release
These issues have been addressed, and the rest will be addressed after the release. https://issues.apache.org/jira/browse/DAFFODIL-2305 https://issues.apache.org/jira/browse/DAFFODIL-2302 The Sonarqube required more work than anticipated and we didn't want to hold up the release for it. I'll be starting the process to create a release candidate. - Lola On 3/25/20 3:12 PM, Steve Lawrence wrote: With the last of the changes recently merged to support improved schema compilation speed and memory, I think now is a good time to start discussing the next release. 1) Does anyone have any changes that are close to being ready to merge or bugs that are worth delaying the next release until resolved? 2) Does anyone want to volunteer to be the release manager for 2.6.0? The release process [1] now uses containers for most of the heavy lifting, and since I did it last time it would be good to get someone else familiar with the new release process and make sure the container works for someone other than myself. - Steve -- Best Regards Lola K.
Re: [DISCUSS] Daffodil 2.6.0 Release
I'd like to have the below done for 2.6.0 https://issues.apache.org/jira/browse/DAFFODIL-2305 - Remove some unused methods from Term.scala https://issues.apache.org/jira/browse/DAFFODIL-2275 - Reduce code smells and make some configuration changes to Sonarqube I don't mind managing the release, although I think Josh A had expressed interest last time. On 3/25/20 3:24 PM, Beckerle, Mike wrote: I have two issues I'd like fixed for 2.6.0 https://issues.apache.org/jira/browse/DAFFODIL-2302 - this is making it difficult to update a Link16 schema that is in use by multiple parties. There's a second issue, which is that we still have this "NadaParsers are supposed to optimize out" problem. Came up in this same schema again even after the fix which supposedly addressed it. So I think the fix solved some, but not all of this problem. I will try to recreate this and reopen the ticket if it is still happening. I don't want to manage the release, but I definitely think it's better if it isn't Steve L., ... I could have my arm twisted. From: Steve Lawrence Sent: Wednesday, March 25, 2020 3:12 PM To: dev@daffodil.apache.org Subject: [DISCUSS] Daffodil 2.6.0 Release With the last of the changes recently merged to support improved schema compilation speed and memory, I think now is a good time to start discussing the next release. 1) Does anyone have any changes that are close to being ready to merge or bugs that are worth delaying the next release until resolved? 2) Does anyone want to volunteer to be the release manager for 2.6.0? The release process [1] now uses containers for most of the heavy lifting, and since I did it last time it would be good to get someone else familiar with the new release process and make sure the container works for someone other than myself. - Steve -- Best Regards Lola K.
[jira] [Assigned] (DAFFODIL-2209) TDML Improperly handles MAC Style (CR) line endings
[ https://issues.apache.org/jira/browse/DAFFODIL-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olabusayo Kilo reassigned DAFFODIL-2209: Assignee: Olabusayo Kilo > TDML Improperly handles MAC Style (CR) line endings > --- > > Key: DAFFODIL-2209 > URL: https://issues.apache.org/jira/browse/DAFFODIL-2209 > Project: Daffodil > Issue Type: Bug > Components: TDML Runner > Reporter: Olabusayo Kilo > Assignee: Olabusayo Kilo >Priority: Major > Fix For: 2.5.0 > > > Attempts to twoPass test files with MAC style line endings (CR) results in a > "getSimpleText does not match" error. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (DAFFODIL-2209) TDML Improperly handles MAC Style (CR) line endings
Olabusayo Kilo created DAFFODIL-2209: Summary: TDML Improperly handles MAC Style (CR) line endings Key: DAFFODIL-2209 URL: https://issues.apache.org/jira/browse/DAFFODIL-2209 Project: Daffodil Issue Type: Bug Components: TDML Runner Reporter: Olabusayo Kilo Fix For: 2.5.0 Attempts to twoPass test files with MAC style line endings (CR) results in a "getSimpleText does not match" error. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (DAFFODIL-2186) User defined functions for DPath expressions
[ https://issues.apache.org/jira/browse/DAFFODIL-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928621#comment-16928621 ] Olabusayo Kilo commented on DAFFODIL-2186: -- Proposal to solve this is here [https://cwiki.apache.org/confluence/display/DAFFODIL/Proposal%3A+User+Defined+Functions] > User defined functions for DPath expressions > > > Key: DAFFODIL-2186 > URL: https://issues.apache.org/jira/browse/DAFFODIL-2186 > Project: Daffodil > Issue Type: New Feature > Components: Back End, Front End >Affects Versions: 2.4.0 >Reporter: Michael Beckerle > Assignee: Olabusayo Kilo >Priority: Major > Fix For: 2.5.0 > > > Some things cannot be reasonably expressed in DFDL's DPath expression > language. > A good example of this is the abililty to convert between elevation measured > in height above geoid (aka elevation above mean sea level or MSL), and > height-above-elipsoid or HAE. This requires a table of numbers and > interpolation. Nevertheless a DFDL schema needs to convert units of measure > and it is very convenient to have the output of a DFDL parse to be normalized > into a desired unit of measure. > The ability to define a function in a separate jar library, and call it from > a DPath expression, having registered it somehow so Daffodil can find it, is > what I'm calling a user-defined function. > > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (DAFFODIL-2186) User defined functions for DPath expressions
[ https://issues.apache.org/jira/browse/DAFFODIL-2186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olabusayo Kilo reassigned DAFFODIL-2186: Assignee: Olabusayo Kilo > User defined functions for DPath expressions > > > Key: DAFFODIL-2186 > URL: https://issues.apache.org/jira/browse/DAFFODIL-2186 > Project: Daffodil > Issue Type: New Feature > Components: Back End, Front End >Affects Versions: 2.4.0 >Reporter: Michael Beckerle > Assignee: Olabusayo Kilo >Priority: Major > Fix For: 2.5.0 > > > Some things cannot be reasonably expressed in DFDL's DPath expression > language. > A good example of this is the abililty to convert between elevation measured > in height above geoid (aka elevation above mean sea level or MSL), and > height-above-elipsoid or HAE. This requires a table of numbers and > interpolation. Nevertheless a DFDL schema needs to convert units of measure > and it is very convenient to have the output of a DFDL parse to be normalized > into a desired unit of measure. > The ability to define a function in a separate jar library, and call it from > a DPath expression, having registered it somehow so Daffodil can find it, is > what I'm calling a user-defined function. > > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (DAFFODIL-2168) Provide support for Non-spacing mark/Combining Characters in Debugger
[
https://issues.apache.org/jira/browse/DAFFODIL-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Olabusayo Kilo reassigned DAFFODIL-2168:
Assignee: Olabusayo Kilo
> Provide support for Non-spacing mark/Combining Characters in Debugger
> -
>
> Key: DAFFODIL-2168
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2168
> Project: Daffodil
> Issue Type: Bug
> Components: Debugger
>Affects Versions: 2.4.0
> Reporter: Olabusayo Kilo
> Assignee: Olabusayo Kilo
>Priority: Major
> Attachments: nonspacingmark.png, tcp.dfdl.xsd, test.dump
>
>
> Certain unicode characters are [combining
> characters|https://en.wikipedia.org/wiki/Combining_character] and can combine
> with the characters surrounding them, modifying them. This can lead to
> misalignment in our debugger.
> A possible solution could be to detect when a character is a non-spacing
> (or other combining character) and insert the appropriate number of spaces
> around it. Java's Character.getType method will return an enum about what
> type a character is. Many of these character types might need this alignment
> assistance.
> Attached is a pictorial example of the problem and the files that can be used
> to recreate the issue.
> Command run:
> {code:java}
> daffodil --trace parse -Dtransport:Protocol=6 transport:DataLength=79 -s
> tcp.dfdl.xsd test.dump{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DAFFODIL-2168) Provide support for Non-spacing mark/Combining Characters in Debugger
[
https://issues.apache.org/jira/browse/DAFFODIL-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Olabusayo Kilo updated DAFFODIL-2168:
-
Summary: Provide support for Non-spacing mark/Combining Characters in
Debugger (was: Provide support for Non-spacing mark/Combining Characters in
Debbuger)
> Provide support for Non-spacing mark/Combining Characters in Debugger
> -
>
> Key: DAFFODIL-2168
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2168
> Project: Daffodil
> Issue Type: Bug
> Components: Debugger
>Affects Versions: 2.4.0
> Reporter: Olabusayo Kilo
>Priority: Major
> Attachments: nonspacingmark.png, tcp.dfdl.xsd, test.dump
>
>
> Certain unicode characters are [combining
> characters|https://en.wikipedia.org/wiki/Combining_character] and can combine
> with the characters surrounding them, modifying them. This can lead to
> misalignment in our debugger.
> A possible solution could be to detect when a character is a non-spacing
> (or other combining character) and insert the appropriate number of spaces
> around it. Java's Character.getType method will return an enum about what
> type a character is. Many of these character types might need this alignment
> assistance.
> Attached is a pictorial example of the problem and the files that can be used
> to recreate the issue.
> Command run:
> {code:java}
> daffodil --trace parse -Dtransport:Protocol=6 transport:DataLength=79 -s
> tcp.dfdl.xsd test.dump{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (DAFFODIL-2173) Data dumps should be bit-aware
[ https://issues.apache.org/jira/browse/DAFFODIL-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olabusayo Kilo reassigned DAFFODIL-2173: Assignee: Olabusayo Kilo > Data dumps should be bit-aware > -- > > Key: DAFFODIL-2173 > URL: https://issues.apache.org/jira/browse/DAFFODIL-2173 > Project: Daffodil > Issue Type: Bug > Components: Debugger >Affects Versions: 2.4.0 > Reporter: Olabusayo Kilo > Assignee: Olabusayo Kilo >Priority: Major > > Data dumps print out bytes without regard to the bit offset. So if some bits > have already been consumed, the dumps might include those already consumed > bits. > Ideally we would like to be bit sensitive. A possible solution would be to > track if the first byte of the left over data, already had consumed bytes i.e > ended mid-byte, and only display those non consumed bits. That is, if the > data ended at bit 5, and bit order is mostSignificantBitFirst, then this > picture would look like x000 (where the 0's are the actual left over data > bits in that first byte of the left over data.) If bitOrder is > leastSignificantBitFirst the picture would look like 000x. > inStream.fillByteArray takes the bit offset into accound and can be utilized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DAFFODIL-2173) Data dumps should be bit-aware
[
https://issues.apache.org/jira/browse/DAFFODIL-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16875250#comment-16875250
]
Olabusayo Kilo commented on DAFFODIL-2173:
--
First pass at a solution, involves the following steps
# Determine bitorder and byteorder of data at current position (finfo)
# Check if bitPosition is mid byte.
## If it's mid byte, call fillByteArray (currently private) for bits up to the
next byte boundary. Update byteLimit.
##
{code:java}
// val bitsAlreadyCosumed = loc.bitPos0b % 8
// val firstByteBitArray = new Array[Byte](1)
/* if (bitsAlreadyConsumed != 0 ) // mid-byte {
* bitsToDisplay = 8 - bitsAlreadyConsumed
* inStream.fillByteArray(firstByteBitarray, bitsToDisplay, finfo)
* bytesLimit -= 1 //we've taken care of first byte
* }{code}
# Call inputSource.get as usual, either with byteLimit 1 less, or unchanged.
Decode this byte array as is currently done.
# Use bitOrder and byteOrder to display bitArray (e.g x100 or 100x)
> Data dumps should be bit-aware
> --
>
> Key: DAFFODIL-2173
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2173
> Project: Daffodil
> Issue Type: Bug
> Components: Debugger
>Affects Versions: 2.4.0
> Reporter: Olabusayo Kilo
> Assignee: Olabusayo Kilo
>Priority: Major
>
> Data dumps print out bytes without regard to the bit offset. So if some bits
> have already been consumed, the dumps might include those already consumed
> bits.
> Ideally we would like to be bit sensitive. A possible solution would be to
> track if the first byte of the left over data, already had consumed bytes i.e
> ended mid-byte, and only display those non consumed bits. That is, if the
> data ended at bit 5, and bit order is mostSignificantBitFirst, then this
> picture would look like x000 (where the 0's are the actual left over data
> bits in that first byte of the left over data.) If bitOrder is
> leastSignificantBitFirst the picture would look like 000x.
> inStream.fillByteArray takes the bit offset into accound and can be utilized.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (DAFFODIL-2173) Data dumps should be bit-aware
[ https://issues.apache.org/jira/browse/DAFFODIL-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olabusayo Kilo reassigned DAFFODIL-2173: Assignee: (was: Olabusayo Kilo) > Data dumps should be bit-aware > -- > > Key: DAFFODIL-2173 > URL: https://issues.apache.org/jira/browse/DAFFODIL-2173 > Project: Daffodil > Issue Type: Bug > Components: Debugger >Affects Versions: 2.4.0 > Reporter: Olabusayo Kilo >Priority: Major > > Data dumps print out bytes without regard to the bit offset. So if some bits > have already been consumed, the dumps might include those already consumed > bits. > Ideally we would like to be bit sensitive. A possible solution would be to > track if the first byte of the left over data, already had consumed bytes i.e > ended mid-byte, and only display those non consumed bits. That is, if the > data ended at bit 5, and bit order is mostSignificantBitFirst, then this > picture would look like x000 (where the 0's are the actual left over data > bits in that first byte of the left over data.) If bitOrder is > leastSignificantBitFirst the picture would look like 000x. > inStream.fillByteArray takes the bit offset into accound and can be utilized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DAFFODIL-2173) Data dumps should be bit-aware
[ https://issues.apache.org/jira/browse/DAFFODIL-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olabusayo Kilo reassigned DAFFODIL-2173: Assignee: Olabusayo Kilo > Data dumps should be bit-aware > -- > > Key: DAFFODIL-2173 > URL: https://issues.apache.org/jira/browse/DAFFODIL-2173 > Project: Daffodil > Issue Type: Bug > Components: Debugger >Affects Versions: 2.4.0 > Reporter: Olabusayo Kilo > Assignee: Olabusayo Kilo >Priority: Major > > Data dumps print out bytes without regard to the bit offset. So if some bits > have already been consumed, the dumps might include those already consumed > bits. > Ideally we would like to be bit sensitive. A possible solution would be to > track if the first byte of the left over data, already had consumed bytes i.e > ended mid-byte, and only display those non consumed bits. That is, if the > data ended at bit 5, and bit order is mostSignificantBitFirst, then this > picture would look like x000 (where the 0's are the actual left over data > bits in that first byte of the left over data.) If bitOrder is > leastSignificantBitFirst the picture would look like 000x. > inStream.fillByteArray takes the bit offset into accound and can be utilized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (DAFFODIL-2173) Data dumps should be bit-aware
Olabusayo Kilo created DAFFODIL-2173: Summary: Data dumps should be bit-aware Key: DAFFODIL-2173 URL: https://issues.apache.org/jira/browse/DAFFODIL-2173 Project: Daffodil Issue Type: Bug Components: Debugger Affects Versions: 2.4.0 Reporter: Olabusayo Kilo Data dumps print out bytes without regard to the bit offset. So if some bits have already been consumed, the dumps might include those already consumed bits. Ideally we would like to be bit sensitive. A possible solution would be to track if the first byte of the left over data, already had consumed bytes i.e ended mid-byte, and only display those non consumed bits. That is, if the data ended at bit 5, and bit order is mostSignificantBitFirst, then this picture would look like x000 (where the 0's are the actual left over data bits in that first byte of the left over data.) If bitOrder is leastSignificantBitFirst the picture would look like 000x. inStream.fillByteArray takes the bit offset into accound and can be utilized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DAFFODIL-1478) Debugger uses the term arrayIndex. Really should be occursIndex to match the function of the same name.
[ https://issues.apache.org/jira/browse/DAFFODIL-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olabusayo Kilo reassigned DAFFODIL-1478: Assignee: Olabusayo Kilo > Debugger uses the term arrayIndex. Really should be occursIndex to match the > function of the same name. > --- > > Key: DAFFODIL-1478 > URL: https://issues.apache.org/jira/browse/DAFFODIL-1478 > Project: Daffodil > Issue Type: Bug > Components: Debugger, Usability >Affects Versions: 1.1.0 >Reporter: Michael Beckerle >Assignee: Olabusayo Kilo >Priority: Minor > Labels: beginner > Fix For: 2.4.0 > > > To display the current index you type this debugger command: > info arrayIndex > Since the corresponding function is named dfdl:occursIndex, we should adopt > that terminology. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DAFFODIL-1387) Data dump no longer showing for left over data
[
https://issues.apache.org/jira/browse/DAFFODIL-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Olabusayo Kilo reassigned DAFFODIL-1387:
Assignee: Olabusayo Kilo
> Data dump no longer showing for left over data
> --
>
> Key: DAFFODIL-1387
> URL: https://issues.apache.org/jira/browse/DAFFODIL-1387
> Project: Daffodil
> Issue Type: Bug
> Components: Debugger, Diagnostics, Usability
>Affects Versions: 2.0.0
>Reporter: Elizabeth Fahl
> Assignee: Olabusayo Kilo
>Priority: Major
>
> Tests used to show a data dump for things like left over data but this seems
> to be missing now.
> For example:
> {noformat}
> echo -ne "string|123456" | daffodil_cli parse -s
> ./daffodil-cli/src/test/resources/edu/illinois/ncsa/daffodil/CLI/cli_schema.dfdl.xsd
> -r leftover
> {noformat}
> Output used to be similar to:
> {noformat}
> [error] Left over data. 11 bytes available. Location: byte 10
> UTF-8 text starting at byte 8 is: (234)
> Data (hex) starting at byte 8 is: (0x323334)
> http://example.com;>
> string
> 123
>
> {noformat}
> Now looks like:
> {noformat}
> [error] Left over data. 13 bytes available. Location: byte 10 limit 13
> http://example.com;>
> string
> 123
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DAFFODIL-2168) Provide support for Non-spacing mark/Combining Characters in Debbuger
Olabusayo Kilo created DAFFODIL-2168:
Summary: Provide support for Non-spacing mark/Combining Characters
in Debbuger
Key: DAFFODIL-2168
URL: https://issues.apache.org/jira/browse/DAFFODIL-2168
Project: Daffodil
Issue Type: Bug
Components: Debugger
Affects Versions: 2.4.0
Reporter: Olabusayo Kilo
Attachments: nonspacingmark.png, tcp.dfdl.xsd, test.dump
Certain unicode characters are [combining
characters|https://en.wikipedia.org/wiki/Combining_character] and can combine
with the characters surrounding them, modifying them. This can lead to
misalignment in our debugger.
A possible solution could be to detect when a character is a non-spacing
(or other combining character) and insert the appropriate number of spaces
around it. Java's Character.getType method will return an enum about what type
a character is. Many of these character types might need this alignment
assistance.
Attached is a pictorial example of the problem and the files that can be used
to recreate the issue.
Command run:
{code:java}
daffodil --trace parse -Dtransport:Protocol=6 transport:DataLength=79 -s
tcp.dfdl.xsd test.dump{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (DAFFODIL-2123) Exception on error when parsing hex data with trace on
[
https://issues.apache.org/jira/browse/DAFFODIL-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Olabusayo Kilo reassigned DAFFODIL-2123:
Assignee: Olabusayo Kilo
> Exception on error when parsing hex data with trace on
> --
>
> Key: DAFFODIL-2123
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2123
> Project: Daffodil
> Issue Type: Bug
> Components: Back End, Diagnostics
>Affects Versions: 2.4.0
> Reporter: Olabusayo Kilo
> Assignee: Olabusayo Kilo
>Priority: Major
> Attachments: tcp.dfdl.xsd, test.dump
>
>
> When trying to parse hex data with an xsd than causes an error with trace on,
> I get the error below.
> {code:java}
> org.apache.daffodil.exceptions.Abort: Invariant broken: nConsumedBytes.>(0)
> org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
> org.apache.daffodil.io.DataDumper.convertToChar(Dump.scala:489)
> org.apache.daffodil.io.DataDumper.textDump(Dump.scala:150)
> org.apache.daffodil.io.DataDumper.$anonfun$dumpHexAndTextBytes$2(Dump.scala:297)
> at org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
> at org.apache.daffodil.io.DataDumper.convertToChar(Dump.scala:489)
> at org.apache.daffodil.io.DataDumper.textDump(Dump.scala:150)
> at
> org.apache.daffodil.io.DataDumper.$anonfun$dumpHexAndTextBytes$2(Dump.scala:297)
> at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.java:12)
> at scala.collection.immutable.NumericRange.foreach(NumericRange.scala:71)
> at org.apache.daffodil.io.DataDumper.dumpHexAndTextBytes(Dump.scala:264)
> at org.apache.daffodil.io.DataDumper.dump(Dump.scala:129)
> at org.apache.daffodil.processors.DataLoc.dumpStream(DataLoc.scala:144)
> at org.apache.daffodil.processors.DataLoc.dump(DataLoc.scala:104)
> at
> org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$InfoData$.printData(InteractiveDebugger.scala:1313)
> at
> org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$InfoData$.act(InteractiveDebugger.scala:1346)
> at
> org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$.$anonfun$act$20(InteractiveDebugger.scala:1202)
> at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:32)
> at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:29)
> at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:38)
> at
> org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$.act(InteractiveDebugger.scala:1200)
> at
> org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$.act(InteractiveDebugger.scala:682)
> at
> org.apache.daffodil.debugger.InteractiveDebugger$DebugCommand.apply(InteractiveDebugger.scala:472)
> at
> org.apache.daffodil.debugger.InteractiveDebugger.runCommand(InteractiveDebugger.scala:433)
> at
> org.apache.daffodil.debugger.InteractiveDebugger.$anonfun$debugStep$4(InteractiveDebugger.scala:174)
> at scala.collection.immutable.List.foreach(List.scala:389)
> at
> scala.collection.generic.TraversableForwarder.foreach(TraversableForwarder.scala:35)
> at
> scala.collection.generic.TraversableForwarder.foreach$(TraversableForwarder.scala:35)
> at scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:44)
> at
> org.apache.daffodil.debugger.InteractiveDebugger.$anonfun$debugStep$1(InteractiveDebugger.scala:173)
> at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
> at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
> at
> org.apache.daffodil.ExecutionMode$.$anonfun$usingUnrestrictedMode$1(ExecutionMode.scala:66)
> at
> org.apache.daffodil.debugger.InteractiveDebugger.debugStep(InteractiveDebugger.scala:153)
> at
> org.apache.daffodil.debugger.InteractiveDebugger.startElement(InteractiveDebugger.scala:209)
> at
> org.apache.daffodil.events.MultipleEventHandler.$anonfun$startElement$1(ParseEventHandler.scala:117)
> at
> org.apache.daffodil.events.MultipleEventHandler.$anonfun$startElement$1$adapted(ParseEventHandler.scala:117)
> at scala.collection.immutable.List.foreach(List.scala:389)
> at
> org.apache.daffodil.events.MultipleEventHandler.startElement(ParseEventHandler.scala:117)
> at
> org.apache.daffodil.events.MultipleEventHandler.startElement$(ParseEventHandler.scala:115)
> at org.apache.daffodil.processors.DataProcessor.startElement(Runtime.scala:92)
> at
> org.apache.daffodil.processors.parsers.ElementParserBase.parse(ElementCombinator1.scala:141)
> at org.apache.daffodil.processors.parsers.Parser.parse1(Parser.s
[jira] [Assigned] (DAFFODIL-2119) Debugger RTL data dumps are nibbles-reversed
[
https://issues.apache.org/jira/browse/DAFFODIL-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Olabusayo Kilo reassigned DAFFODIL-2119:
Assignee: Olabusayo Kilo
> Debugger RTL data dumps are nibbles-reversed
>
>
> Key: DAFFODIL-2119
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2119
> Project: Daffodil
> Issue Type: Bug
> Components: Debugger
>Affects Versions: 2.3.0
>Reporter: Michael Beckerle
> Assignee: Olabusayo Kilo
>Priority: Major
>
> This is what the debugger outputs for parsing one of the tests for
> mil-std-2045, test2045msghdr1.
> It detects that this is bitOrder leastSignificantBitFirst data and so
> presents the data Right-to-Left order, which is good.
> {code:java}
> fedcba9876543210 ffee ddcc bbaa 9988 7766 5544 3322 1100 87654321
> cø€␀␀␀wü␚’gU€␀gä 368f 0800 77cf a129 7655 0800 764e :
> ␀␀␁›¶þ␐HD 00 0010 b96b ef01 8444 :0010{code}
>
> However, this is gibberish, as the bytes of this data (left-to-right) are E4
> 67 00 80 55
> So presented Right-to-left the values in the bytes shouldn't change. But if
> you look at the data dump above, the bytes are 4e 76 00 08 55 which means
> the nibbles of each byte have been swapped.
> That's just plain wrong. Bit order doesn't change the numeric value of any
> byte.
> These displays are useless.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DAFFODIL-2123) Exception on error when parsing hex data with trace on
[
https://issues.apache.org/jira/browse/DAFFODIL-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Olabusayo Kilo updated DAFFODIL-2123:
-
Component/s: (was: Debugger)
> Exception on error when parsing hex data with trace on
> --
>
> Key: DAFFODIL-2123
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2123
> Project: Daffodil
> Issue Type: Bug
> Components: Back End, Diagnostics
>Affects Versions: 2.4.0
> Reporter: Olabusayo Kilo
>Priority: Major
> Attachments: tcp.dfdl.xsd, test.dump
>
>
> When trying to parse hex data with an xsd than causes an error with trace on,
> I get the error below.
> {code:java}
> org.apache.daffodil.exceptions.Abort: Invariant broken: nConsumedBytes.>(0)
> org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
> org.apache.daffodil.io.DataDumper.convertToChar(Dump.scala:489)
> org.apache.daffodil.io.DataDumper.textDump(Dump.scala:150)
> org.apache.daffodil.io.DataDumper.$anonfun$dumpHexAndTextBytes$2(Dump.scala:297)
> at org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
> at org.apache.daffodil.io.DataDumper.convertToChar(Dump.scala:489)
> at org.apache.daffodil.io.DataDumper.textDump(Dump.scala:150)
> at
> org.apache.daffodil.io.DataDumper.$anonfun$dumpHexAndTextBytes$2(Dump.scala:297)
> at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.java:12)
> at scala.collection.immutable.NumericRange.foreach(NumericRange.scala:71)
> at org.apache.daffodil.io.DataDumper.dumpHexAndTextBytes(Dump.scala:264)
> at org.apache.daffodil.io.DataDumper.dump(Dump.scala:129)
> at org.apache.daffodil.processors.DataLoc.dumpStream(DataLoc.scala:144)
> at org.apache.daffodil.processors.DataLoc.dump(DataLoc.scala:104)
> at
> org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$InfoData$.printData(InteractiveDebugger.scala:1313)
> at
> org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$InfoData$.act(InteractiveDebugger.scala:1346)
> at
> org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$.$anonfun$act$20(InteractiveDebugger.scala:1202)
> at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:32)
> at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:29)
> at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:38)
> at
> org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$.act(InteractiveDebugger.scala:1200)
> at
> org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$.act(InteractiveDebugger.scala:682)
> at
> org.apache.daffodil.debugger.InteractiveDebugger$DebugCommand.apply(InteractiveDebugger.scala:472)
> at
> org.apache.daffodil.debugger.InteractiveDebugger.runCommand(InteractiveDebugger.scala:433)
> at
> org.apache.daffodil.debugger.InteractiveDebugger.$anonfun$debugStep$4(InteractiveDebugger.scala:174)
> at scala.collection.immutable.List.foreach(List.scala:389)
> at
> scala.collection.generic.TraversableForwarder.foreach(TraversableForwarder.scala:35)
> at
> scala.collection.generic.TraversableForwarder.foreach$(TraversableForwarder.scala:35)
> at scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:44)
> at
> org.apache.daffodil.debugger.InteractiveDebugger.$anonfun$debugStep$1(InteractiveDebugger.scala:173)
> at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
> at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
> at
> org.apache.daffodil.ExecutionMode$.$anonfun$usingUnrestrictedMode$1(ExecutionMode.scala:66)
> at
> org.apache.daffodil.debugger.InteractiveDebugger.debugStep(InteractiveDebugger.scala:153)
> at
> org.apache.daffodil.debugger.InteractiveDebugger.startElement(InteractiveDebugger.scala:209)
> at
> org.apache.daffodil.events.MultipleEventHandler.$anonfun$startElement$1(ParseEventHandler.scala:117)
> at
> org.apache.daffodil.events.MultipleEventHandler.$anonfun$startElement$1$adapted(ParseEventHandler.scala:117)
> at scala.collection.immutable.List.foreach(List.scala:389)
> at
> org.apache.daffodil.events.MultipleEventHandler.startElement(ParseEventHandler.scala:117)
> at
> org.apache.daffodil.events.MultipleEventHandler.startElement$(ParseEventHandler.scala:115)
> at org.apache.daffodil.processors.DataProcessor.startElement(Runtime.scala:92)
> at
> org.apache.daffodil.processors.parsers.ElementParserBase.parse(ElementCombinator1.scala:141)
> at org.apache.daffodil.processors.parsers.Parser.parse1(Parser.scala:76)
> at org.apache.daffodil.processors.parsers.Parser.par
[jira] [Updated] (DAFFODIL-2123) Exception on error when parsing hex data with trace on
[
https://issues.apache.org/jira/browse/DAFFODIL-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Olabusayo Kilo updated DAFFODIL-2123:
-
Description:
When trying to parse hex data with an xsd than causes an error with trace on, I
get the error below.
{code:java}
org.apache.daffodil.exceptions.Abort: Invariant broken: nConsumedBytes.>(0)
org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
org.apache.daffodil.io.DataDumper.convertToChar(Dump.scala:489)
org.apache.daffodil.io.DataDumper.textDump(Dump.scala:150)
org.apache.daffodil.io.DataDumper.$anonfun$dumpHexAndTextBytes$2(Dump.scala:297)
at org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
at org.apache.daffodil.io.DataDumper.convertToChar(Dump.scala:489)
at org.apache.daffodil.io.DataDumper.textDump(Dump.scala:150)
at
org.apache.daffodil.io.DataDumper.$anonfun$dumpHexAndTextBytes$2(Dump.scala:297)
at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.java:12)
at scala.collection.immutable.NumericRange.foreach(NumericRange.scala:71)
at org.apache.daffodil.io.DataDumper.dumpHexAndTextBytes(Dump.scala:264)
at org.apache.daffodil.io.DataDumper.dump(Dump.scala:129)
at org.apache.daffodil.processors.DataLoc.dumpStream(DataLoc.scala:144)
at org.apache.daffodil.processors.DataLoc.dump(DataLoc.scala:104)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$InfoData$.printData(InteractiveDebugger.scala:1313)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$InfoData$.act(InteractiveDebugger.scala:1346)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$.$anonfun$act$20(InteractiveDebugger.scala:1202)
at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:32)
at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:29)
at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:38)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$.act(InteractiveDebugger.scala:1200)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$.act(InteractiveDebugger.scala:682)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommand.apply(InteractiveDebugger.scala:472)
at
org.apache.daffodil.debugger.InteractiveDebugger.runCommand(InteractiveDebugger.scala:433)
at
org.apache.daffodil.debugger.InteractiveDebugger.$anonfun$debugStep$4(InteractiveDebugger.scala:174)
at scala.collection.immutable.List.foreach(List.scala:389)
at
scala.collection.generic.TraversableForwarder.foreach(TraversableForwarder.scala:35)
at
scala.collection.generic.TraversableForwarder.foreach$(TraversableForwarder.scala:35)
at scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:44)
at
org.apache.daffodil.debugger.InteractiveDebugger.$anonfun$debugStep$1(InteractiveDebugger.scala:173)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at
org.apache.daffodil.ExecutionMode$.$anonfun$usingUnrestrictedMode$1(ExecutionMode.scala:66)
at
org.apache.daffodil.debugger.InteractiveDebugger.debugStep(InteractiveDebugger.scala:153)
at
org.apache.daffodil.debugger.InteractiveDebugger.startElement(InteractiveDebugger.scala:209)
at
org.apache.daffodil.events.MultipleEventHandler.$anonfun$startElement$1(ParseEventHandler.scala:117)
at
org.apache.daffodil.events.MultipleEventHandler.$anonfun$startElement$1$adapted(ParseEventHandler.scala:117)
at scala.collection.immutable.List.foreach(List.scala:389)
at
org.apache.daffodil.events.MultipleEventHandler.startElement(ParseEventHandler.scala:117)
at
org.apache.daffodil.events.MultipleEventHandler.startElement$(ParseEventHandler.scala:115)
at org.apache.daffodil.processors.DataProcessor.startElement(Runtime.scala:92)
at
org.apache.daffodil.processors.parsers.ElementParserBase.parse(ElementCombinator1.scala:141)
at org.apache.daffodil.processors.parsers.Parser.parse1(Parser.scala:76)
at org.apache.daffodil.processors.parsers.Parser.parse1$(Parser.scala:70)
at
org.apache.daffodil.processors.parsers.CombinatorParser.parse1(Parser.scala:154)
at org.apache.daffodil.processors.DataProcessor.doParse(Runtime.scala:223)
at
org.apache.daffodil.processors.DataProcessor.$anonfun$parse$1(Runtime.scala:196)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at
org.apache.daffodil.ExecutionMode$.$anonfun$usingRuntimeMode$1(ExecutionMode.scala:65)
at org.apache.daffodil.processors.DataProcessor.parse(Runtime.scala:188)
at org.apache.daffodil.processors.DataProcessor.parse(Runtime.scala:184)
at org.apache.daffodil.Main$.$anonfun$run$1(Main.scala:862)
at org.apache.daffodil.util.Timer$.getTimeResult(Timer.scala:76)
at org.apache.daffodil.util.Timer$.getResult(Timer.scala:35)
at org.apache.daffodil.Main$.run(Main.scala:862)
at org.apache.daffodil.Main$.main(Main.scala:1361)
at org.apache.daffodil.Main.m
[jira] [Updated] (DAFFODIL-2123) Exception on error when parsing hex data with trace on
[
https://issues.apache.org/jira/browse/DAFFODIL-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Olabusayo Kilo updated DAFFODIL-2123:
-
Description:
When trying to parse hex data with an xsd than causes an error with trace on, I
get the error below.
{code:java}
org.apache.daffodil.exceptions.Abort: Invariant broken: nConsumedBytes.>(0)
org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
org.apache.daffodil.io.DataDumper.convertToChar(Dump.scala:489)
org.apache.daffodil.io.DataDumper.textDump(Dump.scala:150)
org.apache.daffodil.io.DataDumper.$anonfun$dumpHexAndTextBytes$2(Dump.scala:297)
at org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
at org.apache.daffodil.io.DataDumper.convertToChar(Dump.scala:489)
at org.apache.daffodil.io.DataDumper.textDump(Dump.scala:150)
at
org.apache.daffodil.io.DataDumper.$anonfun$dumpHexAndTextBytes$2(Dump.scala:297)
at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.java:12)
at scala.collection.immutable.NumericRange.foreach(NumericRange.scala:71)
at org.apache.daffodil.io.DataDumper.dumpHexAndTextBytes(Dump.scala:264)
at org.apache.daffodil.io.DataDumper.dump(Dump.scala:129)
at org.apache.daffodil.processors.DataLoc.dumpStream(DataLoc.scala:144)
at org.apache.daffodil.processors.DataLoc.dump(DataLoc.scala:104)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$InfoData$.printData(InteractiveDebugger.scala:1313)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$InfoData$.act(InteractiveDebugger.scala:1346)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$.$anonfun$act$20(InteractiveDebugger.scala:1202)
at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:32)
at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:29)
at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:38)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$.act(InteractiveDebugger.scala:1200)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$.act(InteractiveDebugger.scala:682)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommand.apply(InteractiveDebugger.scala:472)
at
org.apache.daffodil.debugger.InteractiveDebugger.runCommand(InteractiveDebugger.scala:433)
at
org.apache.daffodil.debugger.InteractiveDebugger.$anonfun$debugStep$4(InteractiveDebugger.scala:174)
at scala.collection.immutable.List.foreach(List.scala:389)
at
scala.collection.generic.TraversableForwarder.foreach(TraversableForwarder.scala:35)
at
scala.collection.generic.TraversableForwarder.foreach$(TraversableForwarder.scala:35)
at scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:44)
at
org.apache.daffodil.debugger.InteractiveDebugger.$anonfun$debugStep$1(InteractiveDebugger.scala:173)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at
org.apache.daffodil.ExecutionMode$.$anonfun$usingUnrestrictedMode$1(ExecutionMode.scala:66)
at
org.apache.daffodil.debugger.InteractiveDebugger.debugStep(InteractiveDebugger.scala:153)
at
org.apache.daffodil.debugger.InteractiveDebugger.startElement(InteractiveDebugger.scala:209)
at
org.apache.daffodil.events.MultipleEventHandler.$anonfun$startElement$1(ParseEventHandler.scala:117)
at
org.apache.daffodil.events.MultipleEventHandler.$anonfun$startElement$1$adapted(ParseEventHandler.scala:117)
at scala.collection.immutable.List.foreach(List.scala:389)
at
org.apache.daffodil.events.MultipleEventHandler.startElement(ParseEventHandler.scala:117)
at
org.apache.daffodil.events.MultipleEventHandler.startElement$(ParseEventHandler.scala:115)
at org.apache.daffodil.processors.DataProcessor.startElement(Runtime.scala:92)
at
org.apache.daffodil.processors.parsers.ElementParserBase.parse(ElementCombinator1.scala:141)
at org.apache.daffodil.processors.parsers.Parser.parse1(Parser.scala:76)
at org.apache.daffodil.processors.parsers.Parser.parse1$(Parser.scala:70)
at
org.apache.daffodil.processors.parsers.CombinatorParser.parse1(Parser.scala:154)
at org.apache.daffodil.processors.DataProcessor.doParse(Runtime.scala:223)
at
org.apache.daffodil.processors.DataProcessor.$anonfun$parse$1(Runtime.scala:196)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at
org.apache.daffodil.ExecutionMode$.$anonfun$usingRuntimeMode$1(ExecutionMode.scala:65)
at org.apache.daffodil.processors.DataProcessor.parse(Runtime.scala:188)
at org.apache.daffodil.processors.DataProcessor.parse(Runtime.scala:184)
at org.apache.daffodil.Main$.$anonfun$run$1(Main.scala:862)
at org.apache.daffodil.util.Timer$.getTimeResult(Timer.scala:76)
at org.apache.daffodil.util.Timer$.getResult(Timer.scala:35)
at org.apache.daffodil.Main$.run(Main.scala:862)
at org.apache.daffodil.Main$.main(Main.scala:1361)
at org.apache.daffodil.Main.m
[jira] [Updated] (DAFFODIL-2123) Exception on error when parsing hex data with trace on
[
https://issues.apache.org/jira/browse/DAFFODIL-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Olabusayo Kilo updated DAFFODIL-2123:
-
Description:
When trying to parse hex data with an xsd than causes an error with trace on, I
get the error below.
{code:java}
org.apache.daffodil.exceptions.Abort: Invariant broken: nConsumedBytes.>(0)
org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
org.apache.daffodil.io.DataDumper.convertToChar(Dump.scala:489)
org.apache.daffodil.io.DataDumper.textDump(Dump.scala:150)
org.apache.daffodil.io.DataDumper.$anonfun$dumpHexAndTextBytes$2(Dump.scala:297)
at org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
at org.apache.daffodil.io.DataDumper.convertToChar(Dump.scala:489)
at org.apache.daffodil.io.DataDumper.textDump(Dump.scala:150)
at
org.apache.daffodil.io.DataDumper.$anonfun$dumpHexAndTextBytes$2(Dump.scala:297)
at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.java:12)
at scala.collection.immutable.NumericRange.foreach(NumericRange.scala:71)
at org.apache.daffodil.io.DataDumper.dumpHexAndTextBytes(Dump.scala:264)
at org.apache.daffodil.io.DataDumper.dump(Dump.scala:129)
at org.apache.daffodil.processors.DataLoc.dumpStream(DataLoc.scala:144)
at org.apache.daffodil.processors.DataLoc.dump(DataLoc.scala:104)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$InfoData$.printData(InteractiveDebugger.scala:1313)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$InfoData$.act(InteractiveDebugger.scala:1346)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$.$anonfun$act$20(InteractiveDebugger.scala:1202)
at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:32)
at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:29)
at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:38)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$.act(InteractiveDebugger.scala:1200)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$.act(InteractiveDebugger.scala:682)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommand.apply(InteractiveDebugger.scala:472)
at
org.apache.daffodil.debugger.InteractiveDebugger.runCommand(InteractiveDebugger.scala:433)
at
org.apache.daffodil.debugger.InteractiveDebugger.$anonfun$debugStep$4(InteractiveDebugger.scala:174)
at scala.collection.immutable.List.foreach(List.scala:389)
at
scala.collection.generic.TraversableForwarder.foreach(TraversableForwarder.scala:35)
at
scala.collection.generic.TraversableForwarder.foreach$(TraversableForwarder.scala:35)
at scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:44)
at
org.apache.daffodil.debugger.InteractiveDebugger.$anonfun$debugStep$1(InteractiveDebugger.scala:173)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at
org.apache.daffodil.ExecutionMode$.$anonfun$usingUnrestrictedMode$1(ExecutionMode.scala:66)
at
org.apache.daffodil.debugger.InteractiveDebugger.debugStep(InteractiveDebugger.scala:153)
at
org.apache.daffodil.debugger.InteractiveDebugger.startElement(InteractiveDebugger.scala:209)
at
org.apache.daffodil.events.MultipleEventHandler.$anonfun$startElement$1(ParseEventHandler.scala:117)
at
org.apache.daffodil.events.MultipleEventHandler.$anonfun$startElement$1$adapted(ParseEventHandler.scala:117)
at scala.collection.immutable.List.foreach(List.scala:389)
at
org.apache.daffodil.events.MultipleEventHandler.startElement(ParseEventHandler.scala:117)
at
org.apache.daffodil.events.MultipleEventHandler.startElement$(ParseEventHandler.scala:115)
at org.apache.daffodil.processors.DataProcessor.startElement(Runtime.scala:92)
at
org.apache.daffodil.processors.parsers.ElementParserBase.parse(ElementCombinator1.scala:141)
at org.apache.daffodil.processors.parsers.Parser.parse1(Parser.scala:76)
at org.apache.daffodil.processors.parsers.Parser.parse1$(Parser.scala:70)
at
org.apache.daffodil.processors.parsers.CombinatorParser.parse1(Parser.scala:154)
at org.apache.daffodil.processors.DataProcessor.doParse(Runtime.scala:223)
at
org.apache.daffodil.processors.DataProcessor.$anonfun$parse$1(Runtime.scala:196)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at
org.apache.daffodil.ExecutionMode$.$anonfun$usingRuntimeMode$1(ExecutionMode.scala:65)
at org.apache.daffodil.processors.DataProcessor.parse(Runtime.scala:188)
at org.apache.daffodil.processors.DataProcessor.parse(Runtime.scala:184)
at org.apache.daffodil.Main$.$anonfun$run$1(Main.scala:862)
at org.apache.daffodil.util.Timer$.getTimeResult(Timer.scala:76)
at org.apache.daffodil.util.Timer$.getResult(Timer.scala:35)
at org.apache.daffodil.Main$.run(Main.scala:862)
at org.apache.daffodil.Main$.main(Main.scala:1361)
at org.apache.daffodil.Main.m
[jira] [Created] (DAFFODIL-2123) Exception on error when parsing hex data with trace on
Olabusayo Kilo created DAFFODIL-2123:
Summary: Exception on error when parsing hex data with trace on
Key: DAFFODIL-2123
URL: https://issues.apache.org/jira/browse/DAFFODIL-2123
Project: Daffodil
Issue Type: Bug
Components: Back End, Debugger, Diagnostics
Affects Versions: 2.4.0
Reporter: Olabusayo Kilo
Attachments: tcp.dfdl.xsd, test.dump
When trying to parse hex data with an xsd than causes an error with trace on, I
get the error below.
{code:java}
org.apache.daffodil.exceptions.Abort: Invariant broken: nConsumedBytes.>(0)
org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
org.apache.daffodil.io.DataDumper.convertToChar(Dump.scala:489)
org.apache.daffodil.io.DataDumper.textDump(Dump.scala:150)
org.apache.daffodil.io.DataDumper.$anonfun$dumpHexAndTextBytes$2(Dump.scala:297)
at org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:129)
at org.apache.daffodil.io.DataDumper.convertToChar(Dump.scala:489)
at org.apache.daffodil.io.DataDumper.textDump(Dump.scala:150)
at
org.apache.daffodil.io.DataDumper.$anonfun$dumpHexAndTextBytes$2(Dump.scala:297)
at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.java:12)
at scala.collection.immutable.NumericRange.foreach(NumericRange.scala:71)
at org.apache.daffodil.io.DataDumper.dumpHexAndTextBytes(Dump.scala:264)
at org.apache.daffodil.io.DataDumper.dump(Dump.scala:129)
at org.apache.daffodil.processors.DataLoc.dumpStream(DataLoc.scala:144)
at org.apache.daffodil.processors.DataLoc.dump(DataLoc.scala:104)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$InfoData$.printData(InteractiveDebugger.scala:1313)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$InfoData$.act(InteractiveDebugger.scala:1346)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$.$anonfun$act$20(InteractiveDebugger.scala:1202)
at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:32)
at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:29)
at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:38)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$Info$.act(InteractiveDebugger.scala:1200)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommandBase$.act(InteractiveDebugger.scala:682)
at
org.apache.daffodil.debugger.InteractiveDebugger$DebugCommand.apply(InteractiveDebugger.scala:472)
at
org.apache.daffodil.debugger.InteractiveDebugger.runCommand(InteractiveDebugger.scala:433)
at
org.apache.daffodil.debugger.InteractiveDebugger.$anonfun$debugStep$4(InteractiveDebugger.scala:174)
at scala.collection.immutable.List.foreach(List.scala:389)
at
scala.collection.generic.TraversableForwarder.foreach(TraversableForwarder.scala:35)
at
scala.collection.generic.TraversableForwarder.foreach$(TraversableForwarder.scala:35)
at scala.collection.mutable.ListBuffer.foreach(ListBuffer.scala:44)
at
org.apache.daffodil.debugger.InteractiveDebugger.$anonfun$debugStep$1(InteractiveDebugger.scala:173)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at
org.apache.daffodil.ExecutionMode$.$anonfun$usingUnrestrictedMode$1(ExecutionMode.scala:66)
at
org.apache.daffodil.debugger.InteractiveDebugger.debugStep(InteractiveDebugger.scala:153)
at
org.apache.daffodil.debugger.InteractiveDebugger.startElement(InteractiveDebugger.scala:209)
at
org.apache.daffodil.events.MultipleEventHandler.$anonfun$startElement$1(ParseEventHandler.scala:117)
at
org.apache.daffodil.events.MultipleEventHandler.$anonfun$startElement$1$adapted(ParseEventHandler.scala:117)
at scala.collection.immutable.List.foreach(List.scala:389)
at
org.apache.daffodil.events.MultipleEventHandler.startElement(ParseEventHandler.scala:117)
at
org.apache.daffodil.events.MultipleEventHandler.startElement$(ParseEventHandler.scala:115)
at org.apache.daffodil.processors.DataProcessor.startElement(Runtime.scala:92)
at
org.apache.daffodil.processors.parsers.ElementParserBase.parse(ElementCombinator1.scala:141)
at org.apache.daffodil.processors.parsers.Parser.parse1(Parser.scala:76)
at org.apache.daffodil.processors.parsers.Parser.parse1$(Parser.scala:70)
at
org.apache.daffodil.processors.parsers.CombinatorParser.parse1(Parser.scala:154)
at org.apache.daffodil.processors.DataProcessor.doParse(Runtime.scala:223)
at
org.apache.daffodil.processors.DataProcessor.$anonfun$parse$1(Runtime.scala:196)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
at
org.apache.daffodil.ExecutionMode$.$anonfun$usingRuntimeMode$1(ExecutionMode.scala:65)
at org.apache.daffodil.processors.DataProcessor.parse(Runtime.scala:188)
at org.apache.daffodil.processors.DataProcessor.parse(Runtime.scala:184)
at org.apache.daffodil.Main$.$anonfun$ru
[jira] [Updated] (DAFFODIL-2116) Exception when included resource can't be found
[
https://issues.apache.org/jira/browse/DAFFODIL-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Olabusayo Kilo updated DAFFODIL-2116:
-
Description:
When daffodil can't find a file/resource during parsing, it outputs the error
below. Attached is the sample xsd from the [Git
repo|[https://github.com/DFDLSchemas/CSV/blob/master/src/main/resources/com/tresys/csv/xsd/csv.dfdl.xsd]]
where I've introduced an error in the DFDLGeneralFormat include.
{code:java}
{code}
On attempting to parse the sample csv from the
[repo|[https://github.com/DFDLSchemas/CSV/blob/master/src/test/resources/com/tresys/csv/data/simpleCSV.csv]]
{code:java}
daffodil parse -s csv.dfdl.xsd simpleCSV.csv
{code}
it produces the error below.
{code:java}
java.lang.ClassCastException: class
jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class
java.net.URLClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader and
java.net.URLClassLoader are in module java.base of loader 'bootstrap')
at org.apache.daffodil.util.Misc$.classPath$lzycompute(Misc.scala:145)
at org.apache.daffodil.util.Misc$.classPath(Misc.scala:139)
at org.apache.daffodil.dsom.IIBase.classPath(IIBase.scala:277)
at org.apache.daffodil.dsom.IIBase.whereSearched$lzycompute(IIBase.scala:280)
at org.apache.daffodil.dsom.IIBase.whereSearched(IIBase.scala:279)
at
org.apache.daffodil.dsom.Include.$anonfun$resolvedLocation$1(Include.scala:59)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.liftedTree1$1(OOLAG.scala:585)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:583)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value(OOLAG.scala:581)
at
org.apache.daffodil.dsom.Include.resolvedLocation$lzycompute(Include.scala:50)
at org.apache.daffodil.dsom.Include.resolvedLocation(Include.scala:49)
at org.apache.daffodil.dsom.Include.$anonfun$mapPair$1(Include.scala:40)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.liftedTree1$1(OOLAG.scala:585)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:583)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value(OOLAG.scala:581)
at org.apache.daffodil.dsom.Include.mapPair(Include.scala:38)
at
org.apache.daffodil.dsom.IIBase.$anonfun$notSeenThisBefore$1(IIBase.scala:140)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:12)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.liftedTree1$1(OOLAG.scala:585)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:583)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value(OOLAG.scala:581)
at org.apache.daffodil.dsom.IIBase.notSeenThisBefore(IIBase.scala:139)
at
org.apache.daffodil.dsom.IIBase.$anonfun$iiSchemaFileMaybe$1(IIBase.scala:257)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.liftedTree1$1(OOLAG.scala:585)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:583)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value(OOLAG.scala:581)
at org.apache.daffodil.dsom.IIBase.iiSchemaFileMaybe(IIBase.scala:256)
at org.apache.daffodil.dsom.IIBase.$anonfun$seenAfter$1(IIBase.scala:165)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.liftedTree1$1(OOLAG.scala:585)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:583)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value(OOLAG.scala:581)
at org.apache.daffodil.dsom.IIBase.seenAfter(IIBase.scala:164)
at
org.apache.daffodil.dsom.SchemaDocIncludesAndImportsMixin.$anonfun$getImportsOrIncludes$1(SchemaDocIncludesAndImportsMixin.scala:147)
at
scala.collection.TraversableOnce.$anonfun$foldLeft$1(TraversableOnce.scala:157)
at
scala.collection.TraversableOnce.$anonfun$foldLeft$1$adapted(TraversableOnce.scala:157)
at scala.collection.Iterator.foreach(Iterator.scala:944)
at scala.collection.Iterator.foreach$(Iterator.scala:944)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1432)
at scala.collection.IterableLike.foreach(IterableLike.scala:71)
at scala.collection.IterableLike.foreach$(IterableLike.scala:70)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:157)
at scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:155)
at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104)
at
org.apache.daffodil.dsom.SchemaDocIncludesAndImportsMixin.getImportsOrIncludes(SchemaDocIncludesAndImportsMixin.scala:143)
at
org.apache.daffodil.dsom.SchemaDocIncludesAndImportsMixin.getImportsOrIncludes$(SchemaDocIncludesAndImportsMixin.scala:139)
at
org.apache.daffodil.dsom.XMLSchemaDocument.getImportsOrIncludes(SchemaDocument.scala:64)
at
org.apache.daffodil.dsom.SchemaDocIncludesAndImportsMixin.$anonfun$sali_$1(SchemaDocIncludesAndImportsMixin.scala:165)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.liftedTree1$1(OOLAG.scala:585)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:583)
at org.apache.daffodil.oolag.OOLAG
[jira] [Created] (DAFFODIL-2116) Exception when included resource can't be found
Olabusayo Kilo created DAFFODIL-2116:
Summary: Exception when included resource can't be found
Key: DAFFODIL-2116
URL: https://issues.apache.org/jira/browse/DAFFODIL-2116
Project: Daffodil
Issue Type: Bug
Reporter: Olabusayo Kilo
Attachments: csv.dfdl.xsd, simpleCSV.csv
When daffodil can't find a file/resource during parsing, it outputs the error
below. Attached is the sample xsd from the [Git
repo|[https://github.com/DFDLSchemas/CSV/blob/master/src/main/resources/com/tresys/csv/xsd/csv.dfdl.xsd]]
where I've introduced an error in the DFDLGeneralFormat include.
{code:java}
{code}
On attempting to parse the sample csv from the
[repo|[https://github.com/DFDLSchemas/CSV/blob/master/src/test/resources/com/tresys/csv/data/simpleCSV.csv]]
{code:java}
daffodil parse -s csv.dfdl.xsd simpleCSV.csv
{code}
it produces the error below.
{code:java}
java.lang.ClassCastException: class
jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to class
java.net.URLClassLoader (jdk.internal.loader.ClassLoaders$AppClassLoader and
java.net.URLClassLoader are in module java.base of loader 'bootstrap')
at org.apache.daffodil.util.Misc$.classPath$lzycompute(Misc.scala:145)
at org.apache.daffodil.util.Misc$.classPath(Misc.scala:139)
at org.apache.daffodil.dsom.IIBase.classPath(IIBase.scala:277)
at org.apache.daffodil.dsom.IIBase.whereSearched$lzycompute(IIBase.scala:280)
at org.apache.daffodil.dsom.IIBase.whereSearched(IIBase.scala:279)
at
org.apache.daffodil.dsom.Include.$anonfun$resolvedLocation$1(Include.scala:59)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.liftedTree1$1(OOLAG.scala:585)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:583)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value(OOLAG.scala:581)
at
org.apache.daffodil.dsom.Include.resolvedLocation$lzycompute(Include.scala:50)
at org.apache.daffodil.dsom.Include.resolvedLocation(Include.scala:49)
at org.apache.daffodil.dsom.Include.$anonfun$mapPair$1(Include.scala:40)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.liftedTree1$1(OOLAG.scala:585)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:583)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value(OOLAG.scala:581)
at org.apache.daffodil.dsom.Include.mapPair(Include.scala:38)
at
org.apache.daffodil.dsom.IIBase.$anonfun$notSeenThisBefore$1(IIBase.scala:140)
at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:12)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.liftedTree1$1(OOLAG.scala:585)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:583)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value(OOLAG.scala:581)
at org.apache.daffodil.dsom.IIBase.notSeenThisBefore(IIBase.scala:139)
at
org.apache.daffodil.dsom.IIBase.$anonfun$iiSchemaFileMaybe$1(IIBase.scala:257)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.liftedTree1$1(OOLAG.scala:585)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:583)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value(OOLAG.scala:581)
at org.apache.daffodil.dsom.IIBase.iiSchemaFileMaybe(IIBase.scala:256)
at org.apache.daffodil.dsom.IIBase.$anonfun$seenAfter$1(IIBase.scala:165)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.liftedTree1$1(OOLAG.scala:585)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:583)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value(OOLAG.scala:581)
at org.apache.daffodil.dsom.IIBase.seenAfter(IIBase.scala:164)
at
org.apache.daffodil.dsom.SchemaDocIncludesAndImportsMixin.$anonfun$getImportsOrIncludes$1(SchemaDocIncludesAndImportsMixin.scala:147)
at
scala.collection.TraversableOnce.$anonfun$foldLeft$1(TraversableOnce.scala:157)
at
scala.collection.TraversableOnce.$anonfun$foldLeft$1$adapted(TraversableOnce.scala:157)
at scala.collection.Iterator.foreach(Iterator.scala:944)
at scala.collection.Iterator.foreach$(Iterator.scala:944)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1432)
at scala.collection.IterableLike.foreach(IterableLike.scala:71)
at scala.collection.IterableLike.foreach$(IterableLike.scala:70)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:157)
at scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:155)
at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:104)
at
org.apache.daffodil.dsom.SchemaDocIncludesAndImportsMixin.getImportsOrIncludes(SchemaDocIncludesAndImportsMixin.scala:143)
at
org.apache.daffodil.dsom.SchemaDocIncludesAndImportsMixin.getImportsOrIncludes$(SchemaDocIncludesAndImportsMixin.scala:139)
at
org.apache.daffodil.dsom.XMLSchemaDocument.getImportsOrIncludes(SchemaDocument.scala:64)
at
org.apache.daffodil.dsom.SchemaDocIncludesAndImportsMixin.$anonfun$sali_$1(SchemaDocIncludesAndImportsMixin.scala:165
