ganeshashree commented on PR #52402: URL: https://github.com/apache/spark/pull/52402#issuecomment-3369878866
> > @ganeshashree Thanks for the proposal. The change looks OK to me. > > Have we checked the warn (build/log) message when we use SQLContext here? If we weren't providing the message to migrate easily, it might be beneficial to defer replacement of apply() and have intermediate migration step (deprecation of the existing methods and removal of them in Spark 5.0.0). > > @HeartSaVioR Thanks for reviewing. Currently, no warning appears in the build log when we use SQLContext. Creating two versions of `MemoryStream.apply` for SparkSession and SQLContext and showing a warning for SQLContext would require resolving ambiguity when both sparkSession and sqlContext are set as implicit variables. Since this is an internal API, please review whether it's acceptable to make this change and update the callers to use MemoryStream with an implicit SparkSession instead of SQLContext, where applicable. I'm exploring further to resolve the ambiguity by preferring SparkSession instead of SQLContext. Made changes to support two versions of `MemoryStream.apply` for SparkSession and SQLContext, with a warning for SQLContext, and also addressed ambiguity when both sparkSession and sqlContext are set as implicit variables by defining a low-priority trait. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
