amaliujia commented on PR #38792: URL: https://github.com/apache/spark/pull/38792#issuecomment-1326905298
1. It is ok for now. But we need to have a clear documentation or throws errors for the Connect's SparkSession, especially for those bits that does not work as PySpark SparkSession. It is because now the API looks compatible, then users will easy to assume they can do what they usually do with PySpark SparkSession, which is certainly causing confusions. 2. I still think we should keep `client.py`. For the new `session.py`, there are compatible SparSession API. For dealing with gRPC and proto, we keep those in `client.py`. These two logically can be separated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
