gaogaotiantian commented on PR #57040: URL: https://github.com/apache/spark/pull/57040#issuecomment-4898586856
If this is for 5.0.0 rather than 4.3.0, it might be much safer. 3.11 will be retired on Oct. 2027. My concern is that for some Python users, they download their minor Python version (3.11) once and never upgrade it. It's not that uncommon for Python users to stay at a certain version. For those users, when they upgrade pyspark version, they'll hit an error about `tarfile` and be very confused. From their point of view, they just upgraded pyspark version which worked perfectly with their current environment, and it failed to work after the upgrade. This may not impact a huge user base, but it's still unnecessary. I think a simple version check `sys.version_info >= (3, 11, 4)` would do a lot. We can either fallback to the tarfile method without the parameter, or raise a more explicit error asking the user to upgrade their python version, rather than fail with a confusing `tarfile` exception about not having the parameter. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
