This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 855a881 [SPARK-16775][DOC][FOLLOW-UP] Add migration guide for removed
accumulator v1 APIs
855a881 is described below
commit 855a881ea695cdc9f0c22fdb0cfca4e16a07520b
Author: yi.wu <[email protected]>
AuthorDate: Thu Apr 23 10:59:35 2020 +0000
[SPARK-16775][DOC][FOLLOW-UP] Add migration guide for removed accumulator
v1 APIs
### What changes were proposed in this pull request?
Add migration guide for removed accumulator v1 APIs.
### Why are the changes needed?
Provide better guidance for users' migration.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Pass Jenkins.
Closes #28309 from Ngone51/SPARK-16775-migration-guide.
Authored-by: yi.wu <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit 6c018b31e28e2ebd23b2b30f57771ec36bf0402c)
Signed-off-by: Wenchen Fan <[email protected]>
---
docs/core-migration-guide.md | 2 ++
1 file changed, 2 insertions(+)
diff --git a/docs/core-migration-guide.md b/docs/core-migration-guide.md
index 33406d0..63baef1 100644
--- a/docs/core-migration-guide.md
+++ b/docs/core-migration-guide.md
@@ -35,6 +35,8 @@ license: |
- Deprecated method `AccumulableInfo.apply` have been removed because creating
`AccumulableInfo` is disallowed.
+- Deprecated accumulator v1 APIs have been removed and please use v2 APIs
instead.
+
- Event log file will be written as UTF-8 encoding, and Spark History Server
will replay event log files as UTF-8 encoding. Previously Spark wrote the event
log file as default charset of driver JVM process, so Spark History Server of
Spark 2.x is needed to read the old event log files in case of incompatible
encoding.
- A new protocol for fetching shuffle blocks is used. It's recommended that
external shuffle services be upgraded when running Spark 3.0 apps. You can
still use old external shuffle services by setting the configuration
`spark.shuffle.useOldFetchProtocol` to `true`. Otherwise, Spark may run into
errors with messages like `IllegalArgumentException: Unexpected message type:
<number>`.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
