[
https://issues.apache.org/jira/browse/DAFFODIL-2827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733187#comment-17733187
]
Steve Lawrence commented on DAFFODIL-2827:
------------------------------------------
Based on the "Supported Encodings" error message being empty, it sounds like no
encodings are loaded, which is done using our SimpleNamedServiceLoader:
https://github.com/apache/daffodil/blob/main/daffodil-io/src/main/scala/org/apache/daffodil/io/processors/charset/BitsCharsetDefinitionRegistry.scala#L27-L28
The resulting map is stored in a lazy val, which is thread safe, so I wouldn't
think there would be any race conditions related loading charsets. And since
you still hit this by putting compileFile in a synchronized block, it feels
like there's something else going on.
The SimpleNamedServiceLoader is a helper class that uses a Java ServiceLoader
to find encoding implementation classes on the classpath. I'm not sure why the
ServiceLoader wouldn't be able to find the classes though. I've only skimmed
the code you linked, but nothing jumps out at me that might cause an issue. I
assume you don't have a custom ClassLoader or are messing with the classpath in
any way?
> Error: Unsupported encoding: ISO-8859-1. Supported encodings:
> --------------------------------------------------------------
>
> Key: DAFFODIL-2827
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2827
> Project: Daffodil
> Issue Type: Bug
> Components: DFDL Schemas
> Affects Versions: 3.4.0
> Environment: openjdk version "11.0.19" 2023-04-18
> OpenJDK Runtime Environment (build 11.0.19+7-post-Ubuntu-0ubuntu122.04.1)
> OpenJDK 64-Bit Server VM (build 11.0.19+7-post-Ubuntu-0ubuntu122.04.1, mixed
> mode, sharing)
> Reporter: Claude Mamo
> Priority: Minor
> Labels: race-condition
>
> I've consistently observed the following error when compiling several DFDL
> schemas concurrently on OpenJDK 11 and 17:
> {noformat}
> Caused by: org.apache.daffodil.dsom.SchemaDefinitionError: Schema Definition
> Error: Unsupported encoding: ISO-8859-1. Supported encodings:
> Schema context: E1153 Location line 1524 column 14 in
> file:/home/smooks-edi-cartridge/edifact-schemas/target/classes/d94b/EDIFACT-Segments.dfdl.xsd
> at org.apache.daffodil.dsom.ImplementsThrowsSDE.SDE (SDE.scala:179)
> at org.apache.daffodil.dsom.ImplementsThrowsSDE.SDE$ (SDE.scala:178)
> at org.apache.daffodil.dsom.DPathCompileInfo.SDE
> (CompiledExpression1.scala:210)
> at org.apache.daffodil.exceptions.ThrowsSDE.schemaDefinitionError
> (ThrowsSDE.scala:58)
> at org.apache.daffodil.exceptions.ThrowsSDE.schemaDefinitionError$
> (ThrowsSDE.scala:58)
> at org.apache.daffodil.dsom.DPathCompileInfo.schemaDefinitionError
> (CompiledExpression1.scala:210)
> at org.apache.daffodil.processors.CharsetEvBase.compute
> (EvEncoding.scala:99)
> at org.apache.daffodil.processors.CharsetEvBase.compute
> (EvEncoding.scala:81)
> at org.apache.daffodil.processors.Evaluatable.evaluate
> (Evaluatable.scala:281)
> at org.apache.daffodil.processors.Evaluatable.compileTimeEvaluate
> (Evaluatable.scala:212)
> at org.apache.daffodil.processors.Evaluatable.compile
> (Evaluatable.scala:317)
> at org.apache.daffodil.processors.Evaluatable.compile
> (Evaluatable.scala:326)
> at
> org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.maybeCharsetEv
> (RuntimePropertyMixins.scala:122)
> at
> org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.maybeCharsetEv$
> (RuntimePropertyMixins.scala:119)
> at
> org.apache.daffodil.dsom.LocalElementDeclBase.maybeCharsetEv$lzycompute
> (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.maybeCharsetEv
> (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.charsetEv
> (RuntimePropertyMixins.scala:115)
> at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.charsetEv$
> (RuntimePropertyMixins.scala:114)
> at org.apache.daffodil.dsom.LocalElementDeclBase.charsetEv$lzycompute
> (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.charsetEv
> (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.TermEncodingMixin.encodingInfo
> (TermEncodingMixin.scala:104)
> at org.apache.daffodil.dsom.TermEncodingMixin.encodingInfo$
> (TermEncodingMixin.scala:102)
> at org.apache.daffodil.dsom.LocalElementDeclBase.encodingInfo$lzycompute
> (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.encodingInfo
> (LocalElementDecl.scala:25)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1
> (ElementBaseRuntime1Mixin.scala:180)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1
> (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute
> (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny
> (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute
> (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData
> (ElementBaseRuntime1Mixin.scala:174)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$
> (ElementBaseRuntime1Mixin.scala:174)
> at
> org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute
> (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData
> (LocalElementDecl.scala:25)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$2
> (ElementBaseRuntime1Mixin.scala:170)
> at scala.collection.immutable.List.map (List.scala:293)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$1
> (ElementBaseRuntime1Mixin.scala:169)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1
> (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute
> (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny
> (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute
> (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs
> (ElementBaseRuntime1Mixin.scala:169)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$
> (ElementBaseRuntime1Mixin.scala:168)
> at
> org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$lzycompute
> (LocalElementDecl.scala:25)
> at
> org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs
> (LocalElementDecl.scala:25)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1
> (ElementBaseRuntime1Mixin.scala:177)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1
> (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute
> (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny
> (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute
> (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData
> (ElementBaseRuntime1Mixin.scala:174)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$
> (ElementBaseRuntime1Mixin.scala:174)
> at
> org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute
> (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData
> (LocalElementDecl.scala:25)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$2
> (ElementBaseRuntime1Mixin.scala:170)
> at scala.collection.immutable.List.map (List.scala:293)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$1
> (ElementBaseRuntime1Mixin.scala:169)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1
> (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute
> (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny
> (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute
> (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs
> (ElementBaseRuntime1Mixin.scala:169)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$
> (ElementBaseRuntime1Mixin.scala:168)
> at
> org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$lzycompute
> (LocalElementDecl.scala:25)
> at
> org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs
> (LocalElementDecl.scala:25)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1
> (ElementBaseRuntime1Mixin.scala:177)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1
> (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute
> (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny
> (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute
> (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData
> (ElementBaseRuntime1Mixin.scala:174)
> at
> org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$
> (ElementBaseRuntime1Mixin.scala:174)
> at
> org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute
> (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData
> (LocalElementDecl.scala:25)
> at
> org.apache.daffodil.runtime1.LocalElementDeclBaseRuntime1Mixin.$anonfun$$init$$1
> (LocalElementDeclBaseRuntime1Mixin.scala:24)
> at scala.runtime.java8.JFunction0$mcV$sp.apply (JFunction0$mcV$sp.java:23)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1
> (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute
> (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny
> (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.$anonfun$checkErrors$2
> (OOLAG.scala:427)
> at scala.runtime.java8.JFunction0$mcV$sp.apply (JFunction0$mcV$sp.java:23)
> at org.apache.daffodil.oolag.OOLAG$.keepGoing (OOLAG.scala:65)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.checkErrors (OOLAG.scala:427)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.checkErrors$
> (OOLAG.scala:413)
> at org.apache.daffodil.dsom.SchemaComponentImpl.checkErrors
> (SchemaComponent.scala:34)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.isError (OOLAG.scala:484)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.isError$ (OOLAG.scala:483)
> at org.apache.daffodil.dsom.SchemaSet.isError (SchemaSet.scala:543)
> at org.apache.daffodil.compiler.ProcessorFactory.isError
> (Compiler.scala:136)
> at
> org.apache.daffodil.compiler.Compiler.org$apache$daffodil$compiler$Compiler$$compileSourceInternal
> (Compiler.scala:419)
> at
> org.apache.daffodil.compiler.Compiler$.org$apache$daffodil$compiler$Compiler$$compileSourceSynchronizer
> (Compiler.scala:461)
> at org.apache.daffodil.compiler.Compiler.compileSource
> (Compiler.scala:404)
> at org.apache.daffodil.compiler.Compiler.compileFile (Compiler.scala:388)
> at org.apache.daffodil.japi.Compiler.compileFile (Daffodil.scala:146)
> at org.apache.daffodil.japi.Compiler.compileFile (Daffodil.scala:126)
> at org.smooks.edi.edg.EdifactDfdlSchemaGenerator.generateDfdlSchemas
> (EdifactDfdlSchemaGenerator.java:141){noformat}
> Precisely, the error occurs on the line of code:
> {code:java}
> Daffodil.compiler().compileFile(...)
> {code}
> The error doesn't always occur on the same DFDL schema or the same schema
> location line for that matter. Curiously, this error, which I presume is
> happening because of a race-condition, doesn't show up on OpenJDK 8 so it
> could be a bug in the Java runtime. Note that a new Daffodil compiler is
> created to compile each schema. Moving
> {{Daffodil.compiler().compileFile(...)}} inside a {{synchronized}} block
> doesn't make a difference.
> I had a hunch and added the following code before forking the thread to
> concurrently compile the schemas:
> {code:java}
> org.apache.daffodil.processors.charset.CharsetUtils.supportedEncodingsString(){code}
> The above makes the error go away and I'm using it as a workaround. Can't
> really say why so but I suspect the race-condition is caused by cache reading
> that takes place when checking whether the encoding exists during schema
> compilation.
> I've created a
> [branch|https://github.com/claudemamo/smooks-edi-cartridge/tree/daffodil-unsupported-encoding-issue]
> where you should be able to reproduce the problem. Steps to reproduce:
> # git clone
> [https://github.com/claudemamo/smooks-edi-cartridge/tree/daffodil-unsupported-encoding-issue]
> --branch daffodil-unsupported-encoding-issue
> # cd smooks-edi-cartridge
> # mvn clean install -DskipTests
> # The build should fail on the {{edifact-schemas}} branch because of this
> issue. The error should occur at this point:
> [https://github.com/claudemamo/smooks-edi-cartridge/blob/daffodil-unsupported-encoding-issue/edg/src/main/java/org/smooks/edi/edg/EdifactDfdlSchemaGenerator.java#L139.]
> Uncomment the following line to apply the workaround:
> [https://github.com/claudemamo/smooks-edi-cartridge/blob/daffodil-unsupported-encoding-issue/edg/src/main/java/org/smooks/edi/edg/EdifactDfdlSchemaGenerator.java#L95]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
