Dear Community Members, I would like to initiate a discussion regarding the removal of Python from the Spark3 installation package. Here are a few reasons for considering this change:
1.Unlike Apache Ambari, which installs components individually, Spark3's core functionality does not depend on Python3. Therefore, it may not be appropriate to make Python3 a mandatory installation dependency for Spark. Spark itself can run without Python3, and users who do not intend to use PySpark should still be able to install and use Spark without any issues. 2.The Python3 version required by PySpark is often relatively high, and many operating systems do not provide such high Python versions by default. Including PySpark's Python3 dependency in the Bigtop codebase would introduce significant complexity. It might be more suitable for users to manually install the specific Python3 version required by PySpark, perhaps using Conda or other methods. 3.Removing Python3 dependency from Spark can also benefit the overall transition of Bigtop from Python2 to Python3. Python2 has not been maintained for a considerable period, and streamlining the codebase to work with Python3 can be a step toward maintaining the project's relevance and security. I encourage everyone to share their thoughts and opinions on this matter. Your feedback is valuable as we consider the best course of action. Thank you for your participation and input. Best regards, jiaLiang
