This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new 9dcf671e8d32 [SPARK-54340][PYTHON][FOLLOW-UP] Add link and examples
for `run-with-viztracer`
9dcf671e8d32 is described below
commit 9dcf671e8d32cc54ea7ebd2ef4e148a7628b68dc
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Wed Dec 10 09:15:23 2025 -0800
[SPARK-54340][PYTHON][FOLLOW-UP] Add link and examples for
`run-with-viztracer`
### What changes were proposed in this pull request?
Add link and examples for `run-with-viztracer`
### Why are the changes needed?
to make it easier for developers to have a try
### Does this PR introduce _any_ user-facing change?
dev-only changes
### How was this patch tested?
manually check
```sh
(spark_dev_313) ➜ spark git:(doc_viz) python/run-with-viztracer -h
Usage:
run-with-viztracer your_original_commands
To view the profiling results, run:
vizviewer pyspark_*.json
Environment:
If SPARK_VIZTRACER_OUTPUT_DIR is set, the output will be saved to the
directory.
Otherwise, it will be saved to the current directory.
Requirements:
- viztracer must be installed (pip install viztracer)
Check the following documentation for more information on using viztracer:
https://viztracer.readthedocs.io/en/latest/
Examples:
- Start pyspark shell
python/run-with-viztracer bin/pyspark --conf spark.driver.memory=16g
- Start pyspark shell in Connect mode
python/run-with-viztracer bin/pyspark --remote local
```
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #53418 from zhengruifeng/doc_viz.
Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 4520c17b19f6d90ef393d290a452faf8865e6e7b)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
python/run-with-viztracer | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/python/run-with-viztracer b/python/run-with-viztracer
index 448b8e20b9c1..0f7a4d8494f0 100755
--- a/python/run-with-viztracer
+++ b/python/run-with-viztracer
@@ -37,6 +37,17 @@ Environment:
Requirements:
- viztracer must be installed (pip install viztracer)
+
+Check the following documentation for more information on using viztracer:
+
+ https://viztracer.readthedocs.io/en/latest/
+
+Examples:
+ - Start pyspark shell
+ $0 bin/pyspark --conf spark.driver.memory=16g
+
+ - Start pyspark shell in Connect mode
+ $0 bin/pyspark --remote local
EOF
exit 0
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
