tuxji commented on a change in pull request #385:
URL: https://github.com/apache/incubator-daffodil/pull/385#discussion_r432750377
##########
File path:
daffodil-runtime1/src/main/scala/org/apache/daffodil/dsom/CompiledExpression1.scala
##########
@@ -143,7 +143,7 @@ final case class ConstantExpression[+T <: AnyRef](
lazy val sourceType: NodeInfo.Kind = NodeInfo.fromObject(value)
- def isKnownNonEmpty = value != ""
+ def isKnownNonEmpty = value != "" && value != "%ES;" && !(value == "%WSP*;"
&& qn.toQNameString == "dfdl:initiator")
Review comment:
I think both Steve's and Mike's suggestion would require more time than
I want to spend and a deeper understanding of the codebase than I have now. I
do know from setting a breakpoint at isKnownNonEmpty's location and tweaking
its return value in the debugger that:
1) isKnownNonEmpty is called upon initiators, separators, and terminators
2) isKnownNonEmpty is called twice on the same object with the same value
and qn from two call chains, the schema compiler (in a very long call chain
which eventually calls hasInitiator while checking alignment in
org.apache.daffodil.dsom.ElementBase#alignmentValueInBits) and the runtime
parser (in org.apache.daffodil.dsom.ElementBase#initTermTestExpression)
3) if isKnownNonEmpty returns the original value in the first call chain,
two new call chains will occur before the initTermTestExpression call occurs
4) if isKnownNonEmpty returns the original value in all the previous call
chains, the test still passes as long as isKnownNonEmpty returns the changed
value to the initTermTestExpression call (so that is the key call chain that
must be modified if you don't want to change isKnownNonEmpty here)
5) checking qn == initiator as well as value == <ES or WSP* entity> prevents
other delimiter tests from breaking (they call isKnownNonEmpty with a
terminator that is WSP*)
I changed the isKnownNonEmpty code to always check qn as well as value for
both ES and WSP* entities:
` def isKnownNonEmpty = value != "" && !((value == "%ES;" || value ==
"%WSP*;") && qn.toQNameString == "dfdl:initiator")`
However, if that's not good enough, then here's a thread dump from the key
initTermTestExpression call:
```
"main@1" prio=5 tid=0x1 nid=NA runnable
java.lang.Thread.State: RUNNABLE
at
org.apache.daffodil.dsom.ConstantExpression.isKnownNonEmpty(CompiledExpression1.scala:148)
at
org.apache.daffodil.processors.DelimiterEvMixin.isKnownNonEmpty(EvDelimiters.scala:36)
at
org.apache.daffodil.processors.DelimiterEvMixin.isKnownNonEmpty$(EvDelimiters.scala:36)
at
org.apache.daffodil.processors.DelimiterParseEv.isKnownNonEmpty(EvDelimiters.scala:54)
at
org.apache.daffodil.dsom.ElementBase.initTermTestExpression(ElementBase.scala:741)
at
org.apache.daffodil.dsom.ElementBase.hasEmptyValueInitiator(ElementBase.scala:732)
at
org.apache.daffodil.dsom.ElementBase.hasEmptyValueInitiator$(ElementBase.scala:732)
at
org.apache.daffodil.dsom.LocalElementDeclBase.hasEmptyValueInitiator$lzycompute(LocalElementDecl.scala:22)
- locked <0x1087> (a org.apache.daffodil.dsom.LocalElementDecl)
at
org.apache.daffodil.dsom.LocalElementDeclBase.hasEmptyValueInitiator(LocalElementDecl.scala:22)
at
org.apache.daffodil.dsom.ElementBase.hasEmptyValueZLSyntax(ElementBase.scala:791)
at
org.apache.daffodil.dsom.ElementBase.hasEmptyValueZLSyntax$(ElementBase.scala:791)
at
org.apache.daffodil.dsom.LocalElementDeclBase.hasEmptyValueZLSyntax$lzycompute(LocalElementDecl.scala:22)
at
org.apache.daffodil.dsom.LocalElementDeclBase.hasEmptyValueZLSyntax(LocalElementDecl.scala:22)
at
org.apache.daffodil.grammar.primitives.SequenceChild.x$2$lzycompute(SequenceChild.scala:277)
- locked <0x1088> (a
org.apache.daffodil.grammar.primitives.ScalarOrderedSequenceChild)
at
org.apache.daffodil.grammar.primitives.SequenceChild.x$2(SequenceChild.scala:267)
at
org.apache.daffodil.grammar.primitives.SequenceChild.isEmptyRepZeroLength$lzycompute(SequenceChild.scala:267)
at
org.apache.daffodil.grammar.primitives.SequenceChild.isEmptyRepZeroLength(SequenceChild.scala:267)
at
org.apache.daffodil.grammar.primitives.ScalarOrderedSequenceChild.unsepScalarElementHelper$lzycompute(SequenceChild.scala:473)
at
org.apache.daffodil.grammar.primitives.ScalarOrderedSequenceChild.unsepScalarElementHelper(SequenceChild.scala:472)
at
org.apache.daffodil.grammar.primitives.ScalarOrderedSequenceChild.unseparatedHelper$lzycompute(SequenceChild.scala:464)
at
org.apache.daffodil.grammar.primitives.ScalarOrderedSequenceChild.unseparatedHelper(SequenceChild.scala:461)
at
org.apache.daffodil.grammar.primitives.ScalarOrderedSequenceChild.sequenceChildParser$lzycompute(SequenceChild.scala:401)
at
org.apache.daffodil.grammar.primitives.ScalarOrderedSequenceChild.sequenceChildParser(SequenceChild.scala:384)
at
org.apache.daffodil.grammar.primitives.SequenceChild.parser$lzycompute(SequenceChild.scala:78)
at
org.apache.daffodil.grammar.primitives.SequenceChild.parser(SequenceChild.scala:78)
at
org.apache.daffodil.grammar.primitives.SequenceChild.optSequenceChildParser$lzycompute(SequenceChild.scala:85)
at
org.apache.daffodil.grammar.primitives.SequenceChild.optSequenceChildParser(SequenceChild.scala:84)
at
org.apache.daffodil.grammar.primitives.OrderedSequence.$anonfun$parser$2(SequenceCombinator.scala:61)
at
org.apache.daffodil.grammar.primitives.OrderedSequence$$Lambda$811.521466380.apply(Unknown
Source:-1)
at
scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:280)
at
scala.collection.TraversableLike$$Lambda$86.616207929.apply(Unknown Source:-1)
at scala.collection.Iterator.foreach(Iterator.scala:943)
at scala.collection.Iterator.foreach$(Iterator.scala:943)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
at scala.collection.IterableLike.foreach(IterableLike.scala:74)
at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
at scala.collection.AbstractIterable.foreach(Iterable.scala:56)
at scala.collection.TraversableLike.flatMap(TraversableLike.scala:280)
at
scala.collection.TraversableLike.flatMap$(TraversableLike.scala:277)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:108)
at
org.apache.daffodil.grammar.primitives.OrderedSequence.parser$lzycompute(SequenceCombinator.scala:61)
- locked <0x1089> (a
org.apache.daffodil.grammar.primitives.OrderedSequence)
at
org.apache.daffodil.grammar.primitives.OrderedSequence.parser(SequenceCombinator.scala:54)
at
org.apache.daffodil.grammar.SeqComp.$anonfun$parserChildren$2(Grammar.scala:86)
at
org.apache.daffodil.grammar.SeqComp$$Lambda$806.1400582895.apply(Unknown
Source:-1)
at
scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:273)
at
scala.collection.TraversableLike$$Lambda$60.1910936570.apply(Unknown Source:-1)
at scala.collection.immutable.List.foreach(List.scala:392)
at scala.collection.TraversableLike.map(TraversableLike.scala:273)
at scala.collection.TraversableLike.map$(TraversableLike.scala:266)
at scala.collection.immutable.List.map(List.scala:298)
at
org.apache.daffodil.grammar.SeqComp.parserChildren$lzycompute(Grammar.scala:86)
- locked <0x108a> (a org.apache.daffodil.grammar.SeqComp)
at
org.apache.daffodil.grammar.SeqComp.parserChildren(Grammar.scala:86)
at
org.apache.daffodil.grammar.SeqComp.parser$lzycompute(Grammar.scala:89)
at org.apache.daffodil.grammar.SeqComp.parser(Grammar.scala:88)
at
org.apache.daffodil.grammar.Prod.parser$lzycompute(Production.scala:80)
- locked <0x108b> (a org.apache.daffodil.grammar.Prod)
at org.apache.daffodil.grammar.Prod.parser(Production.scala:76)
at
org.apache.daffodil.runtime1.GramRuntime1Mixin.maybeParser(GramRuntime1Mixin.scala:38)
at
org.apache.daffodil.runtime1.GramRuntime1Mixin.maybeParser$(GramRuntime1Mixin.scala:35)
at org.apache.daffodil.grammar.Gram.maybeParser(GrammarTerm.scala:54)
at
org.apache.daffodil.grammar.primitives.ElementCombinatorBase.eParser$lzycompute(ElementCombinator.scala:378)
- locked <0x108c> (a
org.apache.daffodil.grammar.primitives.ElementParseAndUnspecifiedLength)
at
org.apache.daffodil.grammar.primitives.ElementCombinatorBase.eParser(ElementCombinator.scala:378)
at
org.apache.daffodil.grammar.primitives.ElementParseAndUnspecifiedLength.parser$lzycompute(ElementCombinator.scala:302)
at
org.apache.daffodil.grammar.primitives.ElementParseAndUnspecifiedLength.parser(ElementCombinator.scala:291)
at
org.apache.daffodil.grammar.primitives.ElementCombinator.parser$lzycompute(ElementCombinator.scala:92)
- locked <0x108d> (a
org.apache.daffodil.grammar.primitives.ElementCombinator)
at
org.apache.daffodil.grammar.primitives.ElementCombinator.parser(ElementCombinator.scala:86)
at
org.apache.daffodil.grammar.Prod.parser$lzycompute(Production.scala:80)
- locked <0x108e> (a org.apache.daffodil.grammar.Prod)
at org.apache.daffodil.grammar.Prod.parser(Production.scala:76)
at
org.apache.daffodil.grammar.Prod.parser$lzycompute(Production.scala:80)
- locked <0x108f> (a org.apache.daffodil.grammar.Prod)
at org.apache.daffodil.grammar.Prod.parser(Production.scala:76)
at
org.apache.daffodil.runtime1.SchemaSetRuntime1Mixin.$anonfun$parser$1(SchemaSetRuntime1Mixin.scala:64)
at
org.apache.daffodil.runtime1.SchemaSetRuntime1Mixin$$Lambda$797.324579982.apply(Unknown
Source:-1)
at
org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1(OOLAG.scala:669)
at
org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute(OOLAG.scala:667)
- locked <0x1090> (a org.apache.daffodil.oolag.OOLAG$OOLAGValue)
at
org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny(OOLAG.scala:664)
at
org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:714)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value(OOLAG.scala:714)
at
org.apache.daffodil.runtime1.SchemaSetRuntime1Mixin.parser(SchemaSetRuntime1Mixin.scala:63)
at
org.apache.daffodil.runtime1.SchemaSetRuntime1Mixin.parser$(SchemaSetRuntime1Mixin.scala:63)
at
org.apache.daffodil.dsom.SchemaSet.parser$lzycompute(SchemaSet.scala:66)
- locked <0x1091> (a org.apache.daffodil.dsom.SchemaSet)
at org.apache.daffodil.dsom.SchemaSet.parser(SchemaSet.scala:66)
at
org.apache.daffodil.runtime1.SchemaSetRuntime1Mixin.$anonfun$$init$$1(SchemaSetRuntime1Mixin.scala:37)
at
org.apache.daffodil.runtime1.SchemaSetRuntime1Mixin$$Lambda$170.1419332030.apply(Unknown
Source:-1)
at
org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1(OOLAG.scala:669)
at
org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute(OOLAG.scala:667)
- locked <0x1092> (a org.apache.daffodil.oolag.OOLAG$OOLAGValue)
at
org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny(OOLAG.scala:664)
at
org.apache.daffodil.oolag.OOLAG$OOLAGHost.$anonfun$checkErrors$2(OOLAG.scala:414)
at
org.apache.daffodil.oolag.OOLAG$OOLAGHost$$Lambda$242.1718322084.apply$mcV$sp(Unknown
Source:-1)
at
scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.daffodil.oolag.OOLAG$.keepGoing(OOLAG.scala:60)
at
org.apache.daffodil.oolag.OOLAG$OOLAGHost.checkErrors(OOLAG.scala:414)
at
org.apache.daffodil.oolag.OOLAG$OOLAGHost.checkErrors$(OOLAG.scala:400)
at
org.apache.daffodil.dsom.SchemaComponentImpl.checkErrors(SchemaComponent.scala:35)
at org.apache.daffodil.oolag.OOLAG$OOLAGHost.isError(OOLAG.scala:471)
at org.apache.daffodil.oolag.OOLAG$OOLAGHost.isError$(OOLAG.scala:470)
at
org.apache.daffodil.dsom.SchemaSet.super$isError(SchemaSet.scala:485)
at
org.apache.daffodil.dsom.SchemaSet.$anonfun$isError$3(SchemaSet.scala:485)
at
org.apache.daffodil.dsom.SchemaSet$$Lambda$178.270661321.apply$mcZ$sp(Unknown
Source:-1)
at
scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at org.apache.daffodil.oolag.OOLAG$.keepGoing(OOLAG.scala:60)
at
org.apache.daffodil.dsom.SchemaSet.$anonfun$isError$1(SchemaSet.scala:476)
at
org.apache.daffodil.dsom.SchemaSet$$Lambda$176.1172535934.apply$mcZ$sp(Unknown
Source:-1)
at
scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
at
org.apache.daffodil.ExecutionMode$.$anonfun$usingCompilerMode$1(ExecutionMode.scala:64)
at
org.apache.daffodil.ExecutionMode$$$Lambda$175.2116179210.apply(Unknown
Source:-1)
at org.apache.daffodil.dsom.SchemaSet.isError(SchemaSet.scala:476)
at
org.apache.daffodil.compiler.ProcessorFactory.isError(Compiler.scala:122)
at
org.apache.daffodil.compiler.Compiler.org$apache$daffodil$compiler$Compiler$$compileSourceInternal(Compiler.scala:340)
at
org.apache.daffodil.compiler.Compiler$.org$apache$daffodil$compiler$Compiler$$compileSourceSynchronizer(Compiler.scala:378)
- locked <0x1093> (a org.apache.daffodil.compiler.Compiler$)
at
org.apache.daffodil.compiler.Compiler.compileSource(Compiler.scala:325)
at
org.apache.daffodil.tdml.processor.TDMLDFDLProcessorFactory.compileProcessor(DaffodilTDMLDFDLProcessor.scala:133)
at
org.apache.daffodil.tdml.processor.TDMLDFDLProcessorFactory.$anonfun$getProcessor$1(DaffodilTDMLDFDLProcessor.scala:154)
at
org.apache.daffodil.tdml.processor.TDMLDFDLProcessorFactory$$Lambda$160.1202220987.apply(Unknown
Source:-1)
at
org.apache.daffodil.tdml.SchemaCache.doCompile$lzycompute$1(SchemaCache.scala:95)
- locked <0x1094> (a scala.runtime.LazyRef)
at
org.apache.daffodil.tdml.SchemaCache.doCompile$1(SchemaCache.scala:95)
at
org.apache.daffodil.tdml.SchemaCache.$anonfun$compileAndCache$1(SchemaCache.scala:108)
at
org.apache.daffodil.tdml.SchemaCache$$Lambda$161.1545077099.apply(Unknown
Source:-1)
at scala.collection.mutable.HashMap.getOrElseUpdate(HashMap.scala:86)
at
org.apache.daffodil.tdml.SchemaCache$Cache.getOrElseUpdate(SchemaCache.scala:51)
- locked <0x1095> (a org.apache.daffodil.tdml.SchemaCache$Cache)
at
org.apache.daffodil.tdml.SchemaCache.compileAndCache(SchemaCache.scala:107)
at
org.apache.daffodil.tdml.processor.TDMLDFDLProcessorFactory.getProcessor(DaffodilTDMLDFDLProcessor.scala:154)
at org.apache.daffodil.tdml.TestCase.run(TDMLRunner.scala:758)
at
org.apache.daffodil.tdml.DFDLTestSuite.runOneTest(TDMLRunner.scala:390)
at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:145)
at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:151)
at
org.apache.daffodil.section12.delimiter_properties.TestDelimiterProperties.test_emptyInitiator3(TestDelimiterProperties.scala:85)
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:-1)
at
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:566)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at
com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
at
com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
```
**[Edited to add question below]**
Does this thread stack dump help suggest where in the codebase the
responsibility to answer the question "initiator can match empty string" should
be implemented? This is the key call chain where the original isKnownNonEmpty
function must be changed to make my tests pass (it doesn't need to be changed
for all the other calls).
##########
File path:
daffodil-runtime1/src/main/scala/org/apache/daffodil/dsom/CompiledExpression1.scala
##########
@@ -143,7 +143,7 @@ final case class ConstantExpression[+T <: AnyRef](
lazy val sourceType: NodeInfo.Kind = NodeInfo.fromObject(value)
- def isKnownNonEmpty = value != ""
+ def isKnownNonEmpty = value != "" && value != "%ES;" && !(value == "%WSP*;"
&& qn.toQNameString == "dfdl:initiator")
Review comment:
Ping - I'm waiting for an answer to my question in the above comment.
The question is: Does this thread stack dump help suggest where in the codebase
the responsibility to answer the question "initiator can match empty string"
should be implemented? This is the key call chain where the original
isKnownNonEmpty function must be changed to make my tests pass (it doesn't need
to be changed for all the other calls).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
