sririshindra commented on code in PR #49814:
URL: https://github.com/apache/spark/pull/49814#discussion_r1955508779
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala:
##########
@@ -171,6 +172,11 @@ private[hive] class HiveClientImpl(
private def newState(): SessionState = {
val hiveConf = newHiveConf(sparkConf, hadoopConf, extraConfig,
Some(initClassLoader))
val state = new SessionState(hiveConf)
+ // When SessionState is initialized, the caller context is overridden by
hive
+ // so we need to reset it back to the DRIVER
Review Comment:
Thanks for the suggestion @pan3793 . I cherry picked this change to
spark3.5.4 and tested it with Iceberg and it works correctly.
```
2025-02-14 03:28:14,596 INFO FSNamesystem.audit: allowed=true ugi=root
(auth:SIMPLE) ip=/192.168.97.4 cmd=open
src=/warehouse/sample1/metadata/00000-f686c743-a610-4cb0-9fb4-5a3ca65c7696.metadata.json
dst=null perm=null proto=rpc
callerContext=SPARK_DRIVER_application_1739496632907_0008
2025-02-14 03:28:22,558 INFO FSNamesystem.audit: allowed=true ugi=root
(auth:SIMPLE) ip=/192.168.97.7 cmd=create
src=/warehouse/sample1/data/00000-0-cd51fcf5-b94e-4fb3-9ff8-23db99a03ae4-0-00001.parquet
dst=null perm=root:hadoop:rw-r--r-- proto=rpc
callerContext=SPARK_TASK_application_1739496632907_0008_JId_0_SId_0_0_TId_0_0
2025-02-14 03:28:22,984 INFO FSNamesystem.audit: allowed=true ugi=root
(auth:SIMPLE) ip=/192.168.97.4 cmd=create
src=/warehouse/sample1/metadata/ebc9320e-92b4-40fb-9f64-07e3424c6d75-m0.avro
dst=null perm=root:hadoop:rw-r--r-- proto=rpc
callerContext=SPARK_DRIVER_application_1739496632907_0008
2025-02-14 03:28:23,071 INFO FSNamesystem.audit: allowed=true ugi=root
(auth:SIMPLE) ip=/192.168.97.4 cmd=create
src=/warehouse/sample1/metadata/snap-7578694066380923258-1-ebc9320e-92b4-40fb-9f64-07e3424c6d75.avro
dst=null perm=root:hadoop:rw-r--r-- proto=rpc
callerContext=SPARK_DRIVER_application_1739496632907_0008
2025-02-14 03:28:23,113 INFO FSNamesystem.audit: allowed=true ugi=root
(auth:SIMPLE) ip=/192.168.97.4 cmd=create
src=/warehouse/sample1/metadata/00001-2d907fe8-40df-4618-81d8-ef7fa25c3da1.metadata.json
dst=null perm=root:hadoop:rw-r--r-- proto=rpc
callerContext=SPARK_DRIVER_application_1739496632907_0008
2025-02-14 03:28:23,254 INFO FSNamesystem.audit: allowed=true ugi=root
(auth:SIMPLE) ip=/192.168.97.4 cmd=open
src=/warehouse/sample1/metadata/snap-7578694066380923258-1-ebc9320e-92b4-40fb-9f64-07e3424c6d75.avro
dst=null perm=null proto=rpc
callerContext=SPARK_DRIVER_application_1739496632907_0008
2025-02-14 03:28:23,255 INFO FSNamesystem.audit: allowed=true ugi=root
(auth:SIMPLE) ip=/192.168.97.4 cmd=getfileinfo
src=/warehouse/sample1/metadata/snap-7578694066380923258-1-ebc9320e-92b4-40fb-9f64-07e3424c6d75.avro
dst=null perm=null proto=rpc
callerContext=SPARK_DRIVER_application_1739496632907_0008
2025-02-14 03:28:23,284 INFO FSNamesystem.audit: allowed=true ugi=root
(auth:SIMPLE) ip=/192.168.97.4 cmd=open
src=/warehouse/sample1/metadata/00001-2d907fe8-40df-4618-81d8-ef7fa25c3da1.metadata.json
dst=null perm=null proto=rpc
callerContext=SPARK_DRIVER_application_1739496632907_0008
2025-02-14 03:28:58,189 INFO FSNamesystem.audit: allowed=true ugi=root
(auth:SIMPLE) ip=/192.168.97.4 cmd=open
src=/warehouse/sample1/metadata/00001-2d907fe8-40df-4618-81d8-ef7fa25c3da1.metadata.json
dst=null perm=null proto=rpc
callerContext=SPARK_DRIVER_application_1739496632907_0008
2025-02-14 03:28:58,302 INFO FSNamesystem.audit: allowed=true ugi=root
(auth:SIMPLE) ip=/192.168.97.4 cmd=open
src=/warehouse/sample1/metadata/snap-7578694066380923258-1-ebc9320e-92b4-40fb-9f64-07e3424c6d75.avro
dst=null perm=null proto=rpc
callerContext=SPARK_DRIVER_application_1739496632907_0008
2025-02-14 03:28:58,305 INFO FSNamesystem.audit: allowed=true ugi=root
(auth:SIMPLE) ip=/192.168.97.4 cmd=getfileinfo
src=/warehouse/sample1/metadata/snap-7578694066380923258-1-ebc9320e-92b4-40fb-9f64-07e3424c6d75.avro
dst=null perm=null proto=rpc
callerContext=SPARK_DRIVER_application_1739496632907_0008
2025-02-14 03:28:58,332 INFO FSNamesystem.audit: allowed=true ugi=root
(auth:SIMPLE) ip=/192.168.97.4 cmd=open
src=/warehouse/sample1/metadata/ebc9320e-92b4-40fb-9f64-07e3424c6d75-m0.avro
dst=null perm=null proto=rpc
callerContext=SPARK_DRIVER_application_1739496632907_0008
2025-02-14 03:28:58,373 INFO FSNamesystem.audit: allowed=true ugi=root
(auth:SIMPLE) ip=/192.168.97.4 cmd=open
src=/warehouse/sample1/data/00000-0-cd51fcf5-b94e-4fb3-9ff8-23db99a03ae4-0-00001.parquet
dst=null perm=null proto=rpc
callerContext=SPARK_DRIVER_application_1739496632907_0008
2025-02-14 03:28:58,401 INFO FSNamesystem.audit: allowed=true ugi=root
(auth:SIMPLE) ip=/192.168.97.4 cmd=open
src=/warehouse/sample1/data/00000-0-cd51fcf5-b94e-4fb3-9ff8-23db99a03ae4-0-00001.parquet
dst=null perm=null proto=rpc
callerContext=SPARK_DRIVER_application_1739496632907_0008
2025-02-14 03:28:58,837 INFO FSNamesystem.audit: allowed=true ugi=root
(auth:SIMPLE) ip=/192.168.97.7 cmd=getfileinfo
src=/warehouse/sample1/data/00000-0-cd51fcf5-b94e-4fb3-9ff8-23db99a03ae4-0-00001.parquet
dst=null perm=null proto=rpc
callerContext=SPARK_TASK_application_1739496632907_0008_JId_1_SId_1_0_TId_1_0
2025-02-14 03:28:58,842 INFO FSNamesystem.audit: allowed=true ugi=root
(auth:SIMPLE) ip=/192.168.97.7 cmd=open
src=/warehouse/sample1/data/00000-0-cd51fcf5-b94e-4fb3-9ff8-23db99a03ae4-0-00001.parquet
dst=null perm=null proto=rpc
callerContext=SPARK_TASK_application_1739496632907_0008_JId_1_SId_1_0_TId_1_0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
