F. H. created SPARK-39817:
-----------------------------
Summary: Missing sbin scripts in PySpark packages
Key: SPARK-39817
URL: https://issues.apache.org/jira/browse/SPARK-39817
Project: Spark
Issue Type: Bug
Components: PySpark
Affects Versions: 3.2.2, 3.3.0, 3.2.1, 3.2.0
Reporter: F. H.
In the PySpark setup.py, only a subset of all scripts is included.
I'm in particular missing the `submit-all.sh` script:
{code:python}
package_data={
'pyspark.jars': ['*.jar'],
'pyspark.bin': ['*'],
'pyspark.sbin': ['spark-config.sh', 'spark-daemon.sh',
'start-history-server.sh',
'stop-history-server.sh', ],
[...]
},
{code}
The solution is super simple: Just change 'pyspark.sbin' to:
{code:python}
'pyspark.sbin': ['*'],
{code}
I would happily submit a PR to github, but I have no clue on the organizational
details.
This would be great to get backported for pyspark 3.2.x as well as 3.3.x soon.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
