This is an automated email from the ASF dual-hosted git repository.
roryqi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-uniffle.git
The following commit(s) were added to refs/heads/master by this push:
new ad9d0f1d9 [MINOR] docs: Fix spark.serializer in README and
client_guide (#1180)
ad9d0f1d9 is described below
commit ad9d0f1d92f824dd548046b1a35ebd7a521ce067
Author: chqtan1993 <[email protected]>
AuthorDate: Wed Aug 30 11:08:42 2023 +0800
[MINOR] docs: Fix spark.serializer in README and client_guide (#1180)
### What changes were proposed in this pull request?
before:
<img width="973" alt="企业微信截图_59b6f770-e0c8-4274-86ef-4a0527d530cf"
src="https://github.com/apache/incubator-uniffle/assets/70899573/b7e5eb68-b553-406a-bee5-2dede504abbb";>
<img width="1005" alt="企业微信截图_ad7abd72-09e7-4b57-905f-a77f04428836"
src="https://github.com/apache/incubator-uniffle/assets/70899573/2559bc7e-6446-48d9-b97b-873fc263e2fb";>
after:
<img width="959" alt="image"
src="https://github.com/apache/incubator-uniffle/assets/70899573/16de72f9-b064-49aa-8a23-e130eeaf4ae8";>
<img width="1037" alt="image"
src="https://github.com/apache/incubator-uniffle/assets/70899573/882029f5-706b-481c-82db-414c10f03662";>
### Why are the changes needed?
If the wrong `spark.serialier` is set, the above error will be reported in
registerShuffle, and `spark.serializer` should actually be used.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Screenshots attached.
Co-authored-by: alexcqtan <[email protected]>
---
README.md | 2 +-
docs/client_guide.md | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/README.md b/README.md
index 5e39813b4..72d6776e6 100644
--- a/README.md
+++ b/README.md
@@ -241,7 +241,7 @@ Deploy Steps:
```
# Uniffle transmits serialized shuffle data over network, therefore a
serializer that supports relocation of
# serialized object should be used.
- spark.serialier org.apache.spark.serializer.KryoSerializer # this could
also be in the spark-defaults.conf
+ spark.serializer org.apache.spark.serializer.KryoSerializer # this could
also be in the spark-defaults.conf
spark.shuffle.manager org.apache.spark.shuffle.RssShuffleManager
spark.rss.coordinator.quorum <coordinatorIp1>:19999,<coordinatorIp2>:19999
# Note: For Spark2, spark.sql.adaptive.enabled should be false because
Spark2 doesn't support AQE.
diff --git a/docs/client_guide.md b/docs/client_guide.md
index afd769f4e..9a6c99e6b 100644
--- a/docs/client_guide.md
+++ b/docs/client_guide.md
@@ -40,7 +40,7 @@ This document will introduce how to deploy Uniffle client
plugins with Spark and
```
# Uniffle transmits serialized shuffle data over network, therefore a
serializer that supports relocation of
# serialized object should be used.
- spark.serialier org.apache.spark.serializer.KryoSerializer # this could
also be in the spark-defaults.conf
+ spark.serializer org.apache.spark.serializer.KryoSerializer # this could
also be in the spark-defaults.conf
spark.shuffle.manager org.apache.spark.shuffle.RssShuffleManager
spark.rss.coordinator.quorum <coordinatorIp1>:19999,<coordinatorIp2>:19999
# Note: For Spark2, spark.sql.adaptive.enabled should be false because
Spark2 doesn't support AQE.
