Mikó Szilárd created SPARK-48946:
------------------------------------
Summary: NPE in DataSourceV2ScanExecBase redact method when
session is null
Key: SPARK-48946
URL: https://issues.apache.org/jira/browse/SPARK-48946
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 4.0.0
Reporter: Mikó Szilárd
If we call redact method from a thread which was created before we had a Spark
session we get an NPE.
[https://github.com/apache/spark/blob/3f6e2d6ef354c034ce0b39a092d23dd99431a16c/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2ScanExecBase.scala#L70C3-L75C4]
Changing
{code:java}
session.sessionState.conf.stringRedactionPattern {code}
to
{code:java}
conf.stringRedactionPattern {code}
could prevent this problem as conf checks if session is null or not.
https://github.com/apache/spark/blob/af5eb085b0ac01eec2bb5db206b7d499d462abc4/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala#L69
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
