dongjoon-hyun commented on PR #43184: URL: https://github.com/apache/spark/pull/43184#issuecomment-1741845399
Thank you for review, @viirya . You are right. We consider them optional, @viirya . > So for users who download Spark from the official website, they are required to install these dependencies by themselves, right? Actually, `pyspark` shell fails to start. So, we need to embed like Py4J. > I suppose that you can run PySpark from downloaded distribution without packaging if not touching connect and pandas functions. https://github.com/apache/spark/blob/master/python/lib/py4j-0.10.9.7-src.zip However, embedding is not a recommendation from the official Python community. So, I didn't do that in this PR. I'll handle that usability issue as an independent JIRA. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
