[jira] [Created] (HUDI-4584) SQLConf is not propagated correctly into RDDs

[Alexey Kudinkin (Jira)]

Alexey Kudinkin created HUDI-4584:
-------------------------------------

             Summary: SQLConf is not propagated correctly into RDDs
                 Key: HUDI-4584
                 URL: https://issues.apache.org/jira/browse/HUDI-4584
             Project: Apache Hudi
          Issue Type: Bug
            Reporter: Alexey Kudinkin
            Assignee: Alexey Kudinkin
             Fix For: 0.12.1


There were a few reports and slack as well as in GI, related to Spark SQL 
configs not being respected by DeltaStreamer while working perfectly fine when 
leveraging DataSource API:

[https://github.com/apache/hudi/issues/6278]

 

I was able to trace these down to
 # `HoodieSparkUtils.createRDD` instantiating `AvroSerializer` which uses 
SQLConf that isn't propagated by Spark properly.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)