This means when using IntelliJ IDEA, running tests, the TDML runner is NOT doing the jar export and then looking inside them.
Ok. I guess I believe that. A TDML runner change must have happened between 3.0.0 and 3.1.0 creating this vulnerability, because it doesn't happen in 3.0.0. There are a few changes to the TDML runner in 3.1.0 so this is plausible. On Fri, Mar 18, 2022 at 2:53 PM Steve Lawrence <slawre...@apache.org> wrote: > Hmm, I think this is just a bug in the Daffodil TDML runner related to > documentPart's with type="file": > > > https://github.com/apache/daffodil/blob/main/daffodil-tdml-lib/src/main/scala/org/apache/daffodil/tdml/TDMLRunner.scala#L2249-L2262 > > When exportJars is true, findTDMLResource for the file document part is > going be found inside a jar with the "jar" protocol. In that case, we > just say the length is -1L. So that's where our -1 is coming from, and > clearly we don't like that. Ideally we could fix this to get the length > of the resource from the jar. > > So I think exportJars := true and documentPart type="file" are > incompatible with the current TDML Runner. I'll open a ticket. > > - Steve > > On 3/18/22 2:30 PM, Olabusayo Kilo wrote: > > I was also able to recreate this error in the CSV repo by adding the > > "exportJars := true" (note: false doesn't cause this error), > > > > and updating the Daffodil version to the last commit before the RC > > commit (i.e 613c2b0c077f56dbb425e7ebab1a2d154cbdbd06). > > > > so: > > > > > > exportJars :=true libraryDependencies ++=Seq( > > "org.apache.daffodil" %%"daffodil-tdml-processor" %"3.3.0-SNAPSHOT" > > %"test", > > --- > > ) > > > > So it seems to be a Daffodil error, since this doesn't happen with > > 3.0.0, but it does happen with 3.2.1 > > > > Note: I'm running sbt 1.5.7 (Ubuntu Java 11.0.14), but I believe Mike > > tried several different versions when we first ran into this. > > > > On 3/18/22 13:45, Mike Beckerle wrote: > >> I tried many things, including clean git clones of all related repos. > >> This > >> problem is a very bizarre interaction. > >> > >> The fact that the tests ran flawlessly under intelliJ IDEA, but failed > >> running sbt from the command line > >> > >> I searched for sbt bugs related to exportJars. I did find > >> https://github.com/sbt/sbt/issues/3119, but doesn't seem related or not > >> obviously so. > >> > >> On Fri, Mar 18, 2022 at 1:25 PM Adams, Joshua< > jad...@owlcyberdefense.com> > >> wrote: > >> > >>> I can confirm that at least sbt clean does not resolve the issue. > >>> Didn't > >>> try a got clean, although I had just cloned the repo in question so I > >>> don't > >>> think it had gotten dirty in anyway either. > >>> > >>> Josh > >>> > >>> On Mar 18, 2022 1:22 PM, Steve Lawrence<slawre...@apache.org> wrote: > >>> That's surprising to me. All that exportJars is supposed to to do is > >>> include or not include certains files on the class path and use > packages > >>> jars instead. > >>> > >>> And that assertion that is failing is beacuase the length of test data > >>> is negative, which should never happen and I don't know how exportJars > >>> would affect that. > >>> > >>> It almost sounds like something got corrupted or needed to be rebuilt, > >>> and changing exportJars allowed that to happen? Maybe there's some > >>> conflict between using IntelliJ and sbt at the same time? > >>> > >>> I wonder if doing an sbt clean or git clean would resolve the issue. > >>> > >>> > >>> On 3/18/22 1:10 PM, Olabusayo Kilo wrote: > >>>> We noticed that only when running "sbt test" in schema projects with > >>>> the > >>>> exportJar setting, the error below was generated resulting in > >>>> failure of > >>>> all the tests. Note that running the same tests via IntelliJ/Junit > >>>> didn't reproduce this error, and the error was isolated to using sbt > to > >>>> run the tests. > >>>> > >>>> The cause was isolated to having the "exportJars := true" setting in > >>>> build.sbt. Removing the setting removed the error, and got all tests > >>>> passing. Sending this out as an FYI in case someone else runs into > this > >>>> in the future. > >>>> > >>>> [info] Test a.b.c started > >>>> [error] Test a.b.c failed: org.apache.daffodil.exceptions.Abort: Usage > >>>> error: lengthLimitInBits.>=(0) > >>>> [error] org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:137) > >>>> [error] > >>>> > >>> > org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.doParseWithBothApis(DaffodilTDMLDFDLProcessor.scala:290) > > >>> > >>>> [error] > >>>> > >>> > org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.parse(DaffodilTDMLDFDLProcessor.scala:248) > > >>> > >>>> [error] > >>>> > >>> > org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.parse(DaffodilTDMLDFDLProcessor.scala:253), > > >>> > >>>> took 0.003 sec > >>>> [error] at > >>>> org.apache.daffodil.exceptions.Assert$.abort(Assert.scala:137) > >>>> [error] at > >>>> > >>> > org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.doParseWithBothApis(DaffodilTDMLDFDLProcessor.scala:290) > > >>> > >>>> [error] at > >>>> > >>> > org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.parse(DaffodilTDMLDFDLProcessor.scala:248) > > >>> > >>>> [error] at > >>>> > >>> > org.apache.daffodil.tdml.processor.DaffodilTDMLDFDLProcessor.parse(DaffodilTDMLDFDLProcessor.scala:253) > > >>> > >>>> [error] at > >>>> > >>> > org.apache.daffodil.tdml.ParserTestCase.doParseExpectSuccess(TDMLRunner.scala:1026) > > >>> > >>>> [error] at > >>>> > >>> > org.apache.daffodil.tdml.ParserTestCase.runParseExpectSuccess(TDMLRunner.scala:1169) > > >>> > >>>> [error] at > >>>> > >>> > org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2(TDMLRunner.scala:920) > > >>> > >>>> [error] at > >>>> > >>> > org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2$adapted(TDMLRunner.scala:917) > > >>> > >>>> [error] at > >>> scala.util.Either$RightProjection.foreach(Either.scala:652) > >>>> [error] at > >>>> > >>> > org.apache.daffodil.tdml.ParserTestCase.runProcessor(TDMLRunner.scala:917) > >>> > >>>> [error] at > >>> org.apache.daffodil.tdml.TestCase.run(TDMLRunner.scala:870) > >>>> [error] at > >>>> > org.apache.daffodil.tdml.DFDLTestSuite.runOneTest(TDMLRunner.scala:416) > >>>> [error] at > >>>> org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:191) > >>>> [error] at > >>>> org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:197) > >>>> > >>> > >
