HyukjinKwon commented on PR #44745: URL: https://github.com/apache/spark/pull/44745#issuecomment-1894705810
> I roughly suspect that this happened due to the same package names in our project here and there (such as pyspark.pandas, pyspark.sql.pandas), so the namespace conflicts issue occur for some reason only in pandas, but could not figure out the actual root cause right now. This one I know because the test fails sometimes with IDE for the reason. > The reason why I suspect in this way is that because the path /../site-packages/pandas is only not deleted clearly with PySpark dev env when uninstalling pandas. This one can also happen in other packages as well. If that's the case, we should also address the same thing in other packages, e.g., pandas udf and spark connect. It'd be great if we can at least googling and it only happens in pandas before merging this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
