This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a commit to branch branch-3.5 in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.5 by this push: new 11f9d00caed [SPARK-44264][PYTHON] Added Deepspeed Folder to setup.py 11f9d00caed is described below commit 11f9d00caedc3cb1dfc94cf5cfbdfd7aa7c93576 Author: Mathew Jacob <mathew.ja...@databricks.com> AuthorDate: Wed Jul 26 16:21:01 2023 +0900 [SPARK-44264][PYTHON] Added Deepspeed Folder to setup.py ### What changes were proposed in this pull request? Added pyspark.ml.deepspeed to the setup.py ### Why are the changes needed? It allows the deepspeed distributor to be pip installed when you pip install pyspark. ### Does this PR introduce _any_ user-facing change? No Closes #42123 from mathewjacob1002/add_deepspeed_setup_py. Authored-by: Mathew Jacob <mathew.ja...@databricks.com> Signed-off-by: Hyukjin Kwon <gurwls...@apache.org> (cherry picked from commit 89c3472c5feb7b9625f55f82d08429dcb55a6fee) Signed-off-by: Hyukjin Kwon <gurwls...@apache.org> --- python/pyspark/ml/deepspeed/__init__.py | 16 ++++++++++++++++ python/setup.py | 1 + 2 files changed, 17 insertions(+) diff --git a/python/pyspark/ml/deepspeed/__init__.py b/python/pyspark/ml/deepspeed/__init__.py new file mode 100644 index 00000000000..cce3acad34a --- /dev/null +++ b/python/pyspark/ml/deepspeed/__init__.py @@ -0,0 +1,16 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# diff --git a/python/setup.py b/python/setup.py index f190930b2a7..3774273c421 100755 --- a/python/setup.py +++ b/python/setup.py @@ -241,6 +241,7 @@ try: "pyspark.ml.linalg", "pyspark.ml.param", "pyspark.ml.torch", + "pyspark.ml.deepspeed", "pyspark.sql", "pyspark.sql.avro", "pyspark.sql.connect", --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org