mgaido91 commented on issue #25643: [SPARK-28939][SQL] Propagate SQLConf for plans executed by toRdd URL: https://github.com/apache/spark/pull/25643#issuecomment-528567538 @tgravescs all SQL configs which have been set whatever way are applied. > when you do a bunch of dataframe/dataset operations and then do toRdd, it is definitely annoying we never indicate to the user catalyst stuff still happened (like SQL ui isn't present ) yes, I quite agree on this. Unfortunately I don't have a good idea to solve this as the actions are performed outside the "SQL" world, ie. they are performed on the RDD by the user, so it's hard to track them within a SQL execution ID.... Honestly I have no good idea for that at the moment, might you have one, I think it would be great..
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
