This is an automated email from the ASF dual-hosted git repository.
gaogaotiantian pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new fbc72e707d65 [SPARK-58106][DOC][INFRA] Remove stale references to black
fbc72e707d65 is described below
commit fbc72e707d65eb07b48aa738130d06ff15957cf7
Author: Tian Gao <[email protected]>
AuthorDate: Wed Jul 15 11:50:14 2026 -0700
[SPARK-58106][DOC][INFRA] Remove stale references to black
### What changes were proposed in this pull request?
Remove some stale references to `black`.
### Why are the changes needed?
We have been moved to `ruff` for a while. `branch-4.x` and `master` are
using `ruff` exclusively for formatting, not `black`. We need to update our
docs and some dependency files.
Notice that the `--black` command in `lint-python` is also removed. That
was already deprecated for a while and was kind of hidden from users. Leaving
it available does not gain us anything because it will conflict with `ruff`.
Using `black` will mess up the format for `ruff` - it's better to just remove
it.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
CI - this should not have any impact on real code.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #57231 from gaogaotiantian/fix-black-doc.
Authored-by: Tian Gao <[email protected]>
Signed-off-by: Tian Gao <[email protected]>
(cherry picked from commit a301aafa11863f0ad5b00adf57cc792435ad9319)
Signed-off-by: Tian Gao <[email protected]>
---
dev/lint-python | 38 +-------------------------
dev/requirements.txt | 3 --
pyproject.toml | 9 ------
sql/connect/README.md | 2 +-
sql/connect/common/src/main/protobuf/README.md | 6 ++--
5 files changed, 5 insertions(+), 53 deletions(-)
diff --git a/dev/lint-python b/dev/lint-python
index 5f9376dbd33b..efea64fe101a 100755
--- a/dev/lint-python
+++ b/dev/lint-python
@@ -26,8 +26,6 @@ PYTEST_BUILD="pytest"
PYTHON_EXECUTABLE="${PYTHON_EXECUTABLE:-python3}"
-BLACK_BUILD="$PYTHON_EXECUTABLE -m black"
-
function exit_with_usage {
set +x
echo "lint-python - linter for Python"
@@ -45,9 +43,6 @@ while (( "$#" )); do
--compile)
COMPILE_TEST=true
;;
- --black)
- BLACK_TEST=true
- ;;
--custom-pyspark-error)
PYSPARK_CUSTOM_ERRORS_CHECK_TEST=true
;;
@@ -74,7 +69,7 @@ while (( "$#" )); do
shift
done
-if [[ -z
"$COMPILE_TEST$BLACK_TEST$PYSPARK_CUSTOM_ERRORS_CHECK_TEST$FLAKE8_TEST$RUFF_TEST$MYPY_TEST$MYPY_EXAMPLES_TEST$MYPY_DATA_TEST"
]]; then
+if [[ -z
"$COMPILE_TEST$PYSPARK_CUSTOM_ERRORS_CHECK_TEST$FLAKE8_TEST$RUFF_TEST$MYPY_TEST$MYPY_EXAMPLES_TEST$MYPY_DATA_TEST"
]]; then
COMPILE_TEST=true
PYSPARK_CUSTOM_ERRORS_CHECK_TEST=true
RUFF_TEST=true
@@ -329,34 +324,6 @@ ruff checks failed."
}
-function black_test {
- local BLACK_REPORT=
- local BLACK_STATUS=
-
- # Skip check if black is not installed.
- $PYTHON_EXECUTABLE -c 'import black' &> /dev/null
- if [ $? -ne 0 ]; then
- echo "The Python library providing 'black' module was not found.
Skipping black checks for now."
- echo
- return
- fi
-
- echo "starting black test..."
- BLACK_REPORT=$( ($BLACK_BUILD --check python/pyspark dev python/packaging
python/benchmarks) 2>&1)
- BLACK_STATUS=$?
-
- if [ "$BLACK_STATUS" -ne 0 ]; then
- echo "black checks failed:"
- echo "$BLACK_REPORT"
- echo "Please run 'dev/reformat-python' script."
- echo "$BLACK_STATUS"
- exit "$BLACK_STATUS"
- else
- echo "black checks passed."
- echo
- fi
-}
-
function pyspark_custom_errors_check_test {
local PYSPARK_CUSTOM_ERRORS_CHECK_REPORT=
local PYSPARK_CUSTOM_ERRORS_CHECK_STATUS=
@@ -385,9 +352,6 @@ PYTHON_SOURCE="$(git ls-files '*.py')"
if [[ "$COMPILE_TEST" == "true" ]]; then
compile_python_test "$PYTHON_SOURCE"
fi
-if [[ "$BLACK_TEST" == "true" ]]; then
- black_test
-fi
if [[ "$PYSPARK_CUSTOM_ERRORS_CHECK_TEST" == "true" ]]; then
pyspark_custom_errors_check_test
fi
diff --git a/dev/requirements.txt b/dev/requirements.txt
index 5e99abf1e34c..9ea67609025c 100644
--- a/dev/requirements.txt
+++ b/dev/requirements.txt
@@ -55,9 +55,6 @@ markupsafe
jira>=3.5.2
PyGithub
-# pandas API on Spark Code formatter.
-black==26.3.1
-
# Spark Connect (required)
grpcio>=1.76.0
grpcio-status>=1.76.0
diff --git a/pyproject.toml b/pyproject.toml
index 2cecb42a9f0d..bb176f26e32e 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -63,15 +63,6 @@ ignore = [
# injected into their namespace, so it is undefined at lint time.
"python/pyspark/sql/tests/df_golden/scripts/**/*.py" = ["F821"]
-[tool.black]
-# When changing the version, we have to update
-# GitHub workflow version and dev/reformat-python
-required-version = "26.3.1"
-line-length = 100
-target-version = ['py39']
-include = '\.pyi?$'
-extend-exclude = 'cloudpickle|error_classes.py'
-
[dependency-groups]
_py4j = ["py4j>=0.10.9.9"]
diff --git a/sql/connect/README.md b/sql/connect/README.md
index 27849fd308ea..339edb8aa28d 100644
--- a/sql/connect/README.md
+++ b/sql/connect/README.md
@@ -24,7 +24,7 @@ Python-specific development guidelines are located in
[python/docs/source/develo
To generate the Python client code from the proto files:
First, make sure to have a Python environment with the installed dependencies.
-Specifically, install `black` and dependencies from the "Spark Connect python
proto generation plugin (optional)" section.
+Specifically, install `ruff` and dependencies from the "Spark Connect python
proto generation plugin (optional)" section.
```
diff --git a/sql/connect/common/src/main/protobuf/README.md
b/sql/connect/common/src/main/protobuf/README.md
index a3708a463e9d..64b428599593 100644
--- a/sql/connect/common/src/main/protobuf/README.md
+++ b/sql/connect/common/src/main/protobuf/README.md
@@ -42,16 +42,16 @@ Install the required tools:
- Python 3.12+
Install the required Python packages. Check `pyproject.toml` for the latest
-pinned versions of `mypy`, `mypy-protobuf`, and `black`, then run:
+pinned versions of `mypy`, `mypy-protobuf`, and `ruff`, then run:
```bash
-pip install 'mypy==<version>' 'mypy-protobuf==<version>' 'black==<version>'
+pip install 'mypy==<version>' 'mypy-protobuf==<version>' 'ruff==<version>'
```
For example, based on the current `pyproject.toml`:
```bash
-pip install 'mypy==1.19.1' 'mypy-protobuf==3.3.0' 'black==26.3.1'
+pip install 'mypy==1.19.1' 'mypy-protobuf==3.3.0' 'ruff==0.14.8'
```
### Generate
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
