rdblue commented on issue #26005: [SPARK-29163][SQL] Simplify Hadoop Configuration access in DataSourcev2 URL: https://github.com/apache/spark/pull/26005#issuecomment-548855542 I think this is going to need to change quite a bit because the v2 API is written in Java instead of Scala, and doesn't get the SparkSession passed in. In Java, a helper trait should be an `interface` so that it doesn't require changing an implementation's parent to use. That means that creating the broadcast variable should be done by Spark and then set on the implementation. I think it would look like this: ```java public interface RequiresBroadcastConf { Configuration prepareConf(Configuration original); void setBroadcastConf(Broadcast<Configuration> bconf); } ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
