Lihang Shen created SPARK-61419:
-----------------------------------

             Summary: from_avro/schema_of_avro can corrupt a shared options map 
when the plan is canonicalized concurrently
                 Key: SPARK-61419
                 URL: https://issues.apache.org/jira/browse/SPARK-61419
             Project: Spark
          Issue Type: Bug
          Components: Bug
    Affects Versions: 4.0.0
            Reporter: Lihang Shen


{{Same issue as SPARK-60945. \{{FromAvro.replacement}} and 
\{{SchemaOfAvro.replacement}} evaluate the
stateful \{{options}} map (\{{CreateMap}}) directly. Concurrent 
canonicalizations of one plan share
that map and race on its map builder, failing with 
\{{IllegalArgumentException}} or
\{{DUPLICATED_MAP_KEY}}, or silently dropping options under LAST_WIN. The fix 
is the same:
evaluate \{{options.freshCopyIfContainsStatefulExpression()}}.}}



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to