Thanks.
For posterity it appear to also be nessasary to explicitly require the evaluation of preSerialization: requiredEvaluations(optTypeCalculator.map(_.preSerialization)) ________________________________ From: Beckerle, Mike <[email protected]> Sent: Wednesday, June 26, 2019 5:03:47 PM To: [email protected] Subject: Re: SDE during pre-serialization So you are trying to not compute the things that go into the slots of the ExpressionTypeCalculator at the time the object is constructed, but later when they are demanded. Since this is part of the schema compiler, one way to snag the exceptions is to make sure that an OOLAG LV is used to do the calculation. E.g., lazy val myThang = LV('mythang) { .... // sometimes causes SDE }.value lazy val expComp = new ExpressionTypeCalculator(myThang) Now since this is passed by name, it's not evaluated until it is needed later. The latest this could be needed would be preSerialization. At that point if myThang is evaluated and an SDE occurs, it will get captured the way all SDE/diagnsotics are by the OOLAG LV stuff. Blame will be placed on the schema component where myThang is defined. That *should* work. ________________________________ From: Sloane, Brandon <[email protected]> Sent: Wednesday, June 26, 2019 4:32:01 PM To: [email protected] Subject: SDE during pre-serialization I am working on moving much of the typeCalc logic to compile time (notably for this email, I am moving much of the error detection to compile time). To avoid circular depdencies, I am trying to use the same pass-by-name trick we use for the runtime data structures: class ExpressionTypeCalculator[A <: AnyRef, B <: AnyRef]( @TransientParam maybeInputTypeCalcArg: => Maybe[ CompiledExpression[B]], ... ) extends TypeCalculator //TypeCalculator extends PreSerialization lazy val maybeInputTypeCalc = maybeInputTypeCalcArg override protected def preSerialization: Any = { super.preSerialization maybeInputTypeCalc maybeOutputTypeCalc } This works in the Common case. In some error cases, it is now possible that the DPath expression compiler will detect an error and emit an SDE. When this happens, I would expect to see an SDE; however the SDE appears to be getting eaten by the serialization code. I assume that I am missing a step in how I am saving/computing these values. java.io.IOException: unexpected exception type at java.io.ObjectStreamClass.throwMiscException(ObjectStreamClass.java:1736) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1146) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) at scala.collection.immutable.HashMap$SerializationProxy.$anonfun$writeObject$2(HashMap.scala:609) at scala.collection.immutable.HashMap$SerializationProxy.$anonfun$writeObject$2$adapted(HashMap.scala:607) at scala.collection.TraversableLike$WithFilter.$anonfun$foreach$1(TraversableLike.scala:789) at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:231) at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:462) at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:788) at scala.collection.immutable.HashMap$SerializationProxy.writeObject(HashMap.scala:607) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1140) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) at java.io.ObjectOutputStream.defaultWriteObject(ObjectOutputStream.java:441) at org.apache.daffodil.util.PreSerialization.serializeObject(Serialize.scala:46) at org.apache.daffodil.util.PreSerialization.serializeObject$(Serialize.scala:42) at org.apache.daffodil.dsom.DPathCompileInfo.serializeObject(CompiledExpression1.scala:204) at org.apache.daffodil.dsom.DPathCompileInfo.writeObject(CompiledExpression1.scala:234) at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1140) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) at java.io.ObjectOutputStream.defaultWriteObject(ObjectOutputStream.java:441) at org.apache.daffodil.util.PreSerialization.serializeObject(Serialize.scala:46) at org.apache.daffodil.util.PreSerialization.serializeObject$(Serialize.scala:42) at org.apache.daffodil.processors.TermRuntimeData.serializeObject(RuntimeData.scala:99) at org.apache.daffodil.processors.TermRuntimeData.writeObject(RuntimeData.scala:175) at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1140) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) at org.apache.daffodil.processors.DataProcessor.save(DataProcessor.scala:171) at org.apache.daffodil.tdml.processor.TDMLDFDLProcessorFactory.generateProcessor(DaffodilTDMLDFDLProcessor.scala:82) at org.apache.daffodil.tdml.processor.TDMLDFDLProcessorFactory.compileProcessor(DaffodilTDMLDFDLProcessor.scala:99) at org.apache.daffodil.tdml.processor.TDMLDFDLProcessorFactory.$anonfun$getProcessor$1(DaffodilTDMLDFDLProcessor.scala:112) at org.apache.daffodil.tdml.SchemaCache.doCompile$lzycompute$1(SchemaCache.scala:95) at org.apache.daffodil.tdml.SchemaCache.doCompile$1(SchemaCache.scala:95) at org.apache.daffodil.tdml.SchemaCache.$anonfun$compileAndCache$1(SchemaCache.scala:108) at scala.collection.mutable.HashMap.getOrElseUpdate(HashMap.scala:82) at org.apache.daffodil.tdml.SchemaCache$Cache.getOrElseUpdate(SchemaCache.scala:51) at org.apache.daffodil.tdml.SchemaCache.compileAndCache(SchemaCache.scala:107) at org.apache.daffodil.tdml.processor.TDMLDFDLProcessorFactory.getProcessor(DaffodilTDMLDFDLProcessor.scala:112) at org.apache.daffodil.tdml.TestCase.run(TDMLRunner.scala:772) at org.apache.daffodil.tdml.DFDLTestSuite.runOneTest(TDMLRunner.scala:394) at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:133) at org.apache.daffodil.extensions.TestInputTypeValueCalc.test_outputTypeCalcInt_01(TestInputTypeValueCalc.scala:70) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192) Caused by: Schema Definition Error: dfdlx:repTypeValue can only be defined on a simple type Schema context: ex:AbstractIntToMulitiply2 Location line 53 column 4 in file:/tmp/inputTypeCalc-Embedded.dfdl.xsd7371030902782065874.dfdl.xsd Brandon T. Sloane Associate, Services [email protected] | tresys.com
