George Pongracz created SPARK-32017:
---------------------------------------
Summary: Make Pyspark Hadoop 3.2+ Variant available in PyPI
Key: SPARK-32017
URL: https://issues.apache.org/jira/browse/SPARK-32017
Project: Spark
Issue Type: Improvement
Components: PySpark
Affects Versions: 3.0.0
Reporter: George Pongracz
Fix For: 3.0.1
The version of Pyspark 3.0.0 currently available in PyPI currently uses hadoop
2.7.4.
Could a variant (or the default) have its version of Hadoop aligned to 3.2.0 as
per the downloadable spark binaries.
This would enable the PyPI version to be compatible with session token
authorisations and assist in accessing data residing in object stores with
stronger encryption methods.
If not PyPI then as a tar file in the apache download archives at the least
please.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
