This is an automated email from the ASF dual-hosted git repository.
rexxiong pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn-website.git
The following commit(s) were added to refs/heads/main by this push:
new 66819be9 Release 0.5.2 (#90)
66819be9 is described below
commit 66819be9b7fe303d121b84c1398f5a3849dd2e73
Author: Nicholas Jiang <[email protected]>
AuthorDate: Tue Nov 26 17:40:15 2024 +0800
Release 0.5.2 (#90)
* Release 0.5.2
---
.github/workflows/site.yaml | 1 +
docs/community/news.md | 1 +
docs/community/release_notes/release_note_0.5.2.md | 124 +++++++++++++++++++++
docs/download.md | 10 +-
mkdocs.yml | 2 +-
5 files changed, 133 insertions(+), 5 deletions(-)
diff --git a/.github/workflows/site.yaml b/.github/workflows/site.yaml
index 66a92078..7a58dfc5 100644
--- a/.github/workflows/site.yaml
+++ b/.github/workflows/site.yaml
@@ -58,6 +58,7 @@ jobs:
- run: ./.github/bin/build_docs.sh 'tags/v0.4.1.tar.gz' '0.4.1'
- run: ./.github/bin/build_docs.sh 'tags/v0.4.2.tar.gz' '0.4.2'
- run: ./.github/bin/build_docs.sh 'tags/v0.5.1.tar.gz' '0.5.1'
+ - run: ./.github/bin/build_docs.sh 'tags/v0.5.2.tar.gz' '0.5.2'
- run: |
echo 'publish:' >> .asf.yaml
echo ' whoami: asf-site' >> .asf.yaml
diff --git a/docs/community/news.md b/docs/community/news.md
index 1acc28cc..5b71253b 100644
--- a/docs/community/news.md
+++ b/docs/community/news.md
@@ -20,6 +20,7 @@ license: |
| Date | Title
| Brief
|
|-------------------|---------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------|
+| 2024 November 29 | Release 0.5.2
| Celeborn release 0.5.2.
|
| 2024 September 20 | New PMC Member: Mridul Muralidharan
| We are happy to announce Mridul Muralidharan becomes a new Celeborn PMC
member. |
| 2024 July 29 | Release 0.5.1
| Celeborn release 0.5.1.
|
| 2024 July 26 | Release 0.4.2
| Celeborn release 0.4.2.
|
diff --git a/docs/community/release_notes/release_note_0.5.2.md
b/docs/community/release_notes/release_note_0.5.2.md
new file mode 100644
index 00000000..c78f4359
--- /dev/null
+++ b/docs/community/release_notes/release_note_0.5.2.md
@@ -0,0 +1,124 @@
+---
+hide:
+ - navigation
+
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ https://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+# Apache Celeborn™ 0.5.2 Release Notes
+
+## Highlight
+
+- Support Apache Spark barrier stages
+- Support differentiate map results with only different stageAttemptId
+- Fix InFlightRequestTracker should not reset totalInflightReqs for cleaning
up to avoid negative totalInflightReqs for limitZeroInFlight
+
+### Improvement
+
+- [CELEBORN-1071] Support stage rerun for shuffle data lost
+- [CELEBORN-1511] Add support for custom master endpoint resolver
+- [CELEBORN-1516][FOLLOWUP] Support reset method for
DynamicConfigServiceFactory
+- [CELEBORN-1518] Add support for Apache Spark barrier stages
+- [CELEBORN-1524] Support IPv6 hostnames for Apache Ratis
+- [CELEBORN-1533] Log location when CelebornInputStream#fillBuffer fails
+- [CELEBORN-1535] Support to disable master workerUnavailableInfo expiration
+- [CELEBORN-1541] Enhance the readable address for internal port
+- [CELEBORN-1550] Add support of providing custom dynamic store backend
implementation
+- [CELEBORN-1552] Automatically support prometheus to scrape metrics for helm
chart
+- [CELEBORN-1563] Log networkLocation in WorkerInfo
+- [CELEBORN-1567] Support throw FetchFailedException when Data corruption
detected
+- [CELEBORN-1568] Support worker retries in MiniCluster
+- [CELEBORN-1573] Change to debug logging on client side for reserve slots
+- [CELEBORN-1578] Make Worker#timer have thread name and daemon
+- [CELEBORN-1587] Change to debug logging on client side for SortBasedPusher
trigger push
+- [CELEBORN-1594] Refine dynamicConfig template and prevent NPE
+- [CELEBORN-1602] Do hard split for push merged data RPC with disk full
+- [CELEBORN-1615] Start the http server after all handlers added
+- [CELEBORN-1625] Add parameter skipCompress for pushOrMergeData
+- [CELEBORN-1638] Improve the slots allocator performance
+- [CELEBORN-1643] DataPusher handle InterruptedException
+- [CELEBORN-1646] Catch exception of Files#getFileStore for DeviceMonitor and
StorageManager for input/ouput error
+- [CELEBORN-1652] Throw TransportableError for failure of sending
PbReadAddCredit to avoid flink task get stuck
+- [CELEBORN-1661] Make sure that the sortedFilesDb is initialized successfully
when worker enable graceful shutdown
+- [CELEBORN-1663] Only register appShuffleDeterminate if stage using celeborn
for shuffle
+- [CELEBORN-1671] CelebornShuffleReader will try replica if create client
failed
+- [CELEBORN-1673] Support retry create client
+
+### Stability and Bug Fix
+
+- [CELEBORN-1297][FOLLOWUP] Fix DB config service SQL file
+- [CELEBORN-1473] TransportClientFactory should register netty memory metric
with source for shared pooled ByteBuf allocator
+- [CELEBORN-1496] Differentiate map results with only different stageAttemptId
+- [CELEBORN-1506][FOLLOWUP] InFlightRequestTracker should not reset
totalInflightReqs for cleaning up to avoid negative totalInflightReqs for
limitZeroInFlight
+- [CELEBORN-1520] Minor logging fix for AppDiskUsageMetric and Fixed UTs
+- [CELEBORN-1522] Fix applicationId extraction from shuffle key
+- [CELEBORN-1526] Fix MR plugin can not run on Hadoop 3.1.0
+- [CELEBORN-1544] ShuffleWriter needs to call close finally to avoid memory
leaks
+- [CELEBORN-1557] Fix totalSpace of DiskInfo for Master in HA mode
+- [CELEBORN-1558] Fix the incorrect decrement of pendingWrites in
handlePushMergeData
+- [CELEBORN-1564] Fix actualUsableSpace of offerSlotsLoadAware condition on
diskInfo
+- [CELEBORN-1575] TimeSlidingHub should remove expire node when reading
+- [CELEBORN-1579] Fix the memory leak of result partition
+- [CELEBORN-1580] ReadBufferDispacther should notify exception to listener
+- [CELEBORN-1581] Fix incorrect metrics of DeviceCelebornFreeBytes and
DeviceCelebornTotalBytes
+- [CELEBORN-1583] MasterClient#sendMessageInner should throw Throwable for
celeborn.masterClient.maxRetries is 0
+- [CELEBORN-1655] Fix read buffer dispatcher thread terminate unexpectedly
+- [CELEBORN-1662] Handle PUSH_DATA_FAIL_PARTITION_NOT_FOUND in
getPushDataFailCause
+- [CELEBORN-1664] Fix secret fetch failures after LEADER master failover
+- [CELEBORN-1665] CommitHandler should process CommitFilesResponse with
COMMIT_FILE_EXCEPTION status
+- [CELEBORN-1667] Fix NPE & LEAK occurring prior to worker registration
+- [CELEBORN-1668] Fix NPE when handle closed file writers
+- [CELEBORN-1669] Fix NullPointerException for
PartitionFilesSorter#updateSortedShuffleFiles after cleaning up expired shuffle
key
+- [CELEBORN-1674] Fix reader thread name of MapPartitionData
+- [CELEBORN-1682] Add java tools.jar into classpath for JVM quake
+- [CELEBORN-1686] Avoid return the same pushTaskQueue
+- [CELEBORN-1691] Fix the issue that upstream tasks don't rerun and the
current task still retry when failed to decompress in flink
+- [CELEBORN-1692] Set mount point in fromPbFileInfoMap
+- [CELEBORN-1693] Fix storageFetcherPool concurrent problem
+- [CELEBORN-1696] StorageManager#cleanFile should remove file info
+- [CELEBORN-1705] Fix disk buffer size is negative issue
+- [CELEBORN-1717] Fix ReusedExchangedSuit UT bug
+- [CELEBORN-1718] Fix memory storage file won't hard split when memory file is
full and worker has no disks
+- [CELEBORN-1726] Update WorkerInfo when transition worker state
+- [CELEBORN-1727] Correct the calculation of worker diskInfo actualUsableSpace
+- [CELEBORN-1728] Fix NPE when failing to connect to celeborn worker
+
+### Build
+
+- [CELEBORN-1677] Update SCM information for SBT build configuration
+
+### Documentation
+
+- [CELEBORN-914][FOLLOWUP] Add emptyFilePrimaryIds and emptyFileReplicaIds of
worker service log in startup document
+- [CELEBORN-914][FOLLOWUP] Adding metrics for memory file storage in
monitoring.md
+- [CELEBORN-1058][FOLLOWUP] Update name of master service from MasterSys to
Master in startup document
+- [CELEBORN-1551] Fix wrong link in quota_management.md
+
+### Dependencies
+
+- [CELEBORN-1666] Bump scala-protoc from 1.0.6 to 1.0.7
+
+## Credits
+
+Thanks to the following contributors who helped to review and commit to Apache
Celeborn 0.5.2 version:
+
+| Contributors | | | |
| |
+|--------------|-------------|-------------|---------------------|----------------|--------------|
+| ErikFang | Ethan Feng | Fei Wang | Fu Chen | Jiashu
Xiong | Kerwin Zhang |
+| Keyong Zhou | Kun Wan | Lianne Li | Mridul Muralidharan | Nicholas
Jiang | Sanskar Modi |
+| Shaoyun Chen | Weijie Guo | Wenliang Bo | Xianming Lei | Xu Huang
| Yanze Jiang |
+| Yihe Li | Yuting Wang | Zhao zhao | Zhentao Shuai |
| |
diff --git a/docs/download.md b/docs/download.md
index fa08edaa..100c06c9 100644
--- a/docs/download.md
+++ b/docs/download.md
@@ -23,13 +23,14 @@ license: |
The latest version is {{ stable_version }}.
-### 0.5.1 (2024-07-29)
-[release note](community/release_notes/release_note_0.5.1.md)
+### 0.5.2 (2024-11-26)
+
+[release note](community/release_notes/release_note_0.5.2.md)
| | Download from ASF
|
Checksum |
Signature |
|:-----------:|:--------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------:|
-| Source Code |
[src](https://downloads.apache.org/celeborn/celeborn-0.5.1/apache-celeborn-0.5.1-source.tgz)
|
[sha512](https://downloads.apache.org/celeborn/celeborn-0.5.1/apache-celeborn-0.5.1-source.tgz.sha512)
|
[asc](https://downloads.apache.org/celeborn/celeborn-0.5.1/apache-celeborn-0.5.1-source.tgz.asc)
|
-| Binary |
[bin](https://downloads.apache.org/celeborn/celeborn-0.5.1/apache-celeborn-0.5.1-bin.tgz)
|
[sha512](https://downloads.apache.org/celeborn/celeborn-0.5.1/apache-celeborn-0.5.1-bin.tgz.sha512)
|
[asc](https://downloads.apache.org/celeborn/celeborn-0.5.1/apache-celeborn-0.5.1-bin.tgz.asc)
|
+| Source Code |
[src](https://downloads.apache.org/celeborn/celeborn-0.5.2/apache-celeborn-0.5.2-source.tgz)
|
[sha512](https://downloads.apache.org/celeborn/celeborn-0.5.2/apache-celeborn-0.5.2-source.tgz.sha512)
|
[asc](https://downloads.apache.org/celeborn/celeborn-0.5.2/apache-celeborn-0.5.2-source.tgz.asc)
|
+| Binary |
[bin](https://downloads.apache.org/celeborn/celeborn-0.5.2/apache-celeborn-0.5.2-bin.tgz)
|
[sha512](https://downloads.apache.org/celeborn/celeborn-0.5.2/apache-celeborn-0.5.2-bin.tgz.sha512)
|
[asc](https://downloads.apache.org/celeborn/celeborn-0.5.2/apache-celeborn-0.5.2-bin.tgz.asc)
|
### 0.4.2 (2024-07-26)
@@ -55,6 +56,7 @@ The latest version is {{ stable_version }}.
All celeborn releases are available via
[https://archive.apache.org/dist/celeborn/](https://archive.apache.org/dist/celeborn/)
including checksums and
signatures. At the time of writing, this includes the following versions:
+* Apache Celeborn 0.5.2 (2024-11-26)
([Source](https://archive.apache.org/dist/celeborn/celeborn-0.5.2/apache-celeborn-0.5.2-source.tgz),
[Binaries](https://archive.apache.org/dist/celeborn/celeborn-0.5.2/))
* Apache Celeborn 0.5.1 (2024-07-29)
([Source](https://archive.apache.org/dist/celeborn/celeborn-0.5.1/apache-celeborn-0.5.1-source.tgz),
[Binaries](https://archive.apache.org/dist/celeborn/celeborn-0.5.1/))
* Apache Celeborn 0.4.2 (2024-07-26)
([Source](https://archive.apache.org/dist/celeborn/celeborn-0.4.2/apache-celeborn-0.4.2-source.tgz),
[Binaries](https://archive.apache.org/dist/celeborn/celeborn-0.4.2/))
* Apache Celeborn 0.5.0 (2024-06-24)
([Source](https://archive.apache.org/dist/celeborn/celeborn-0.5.0/apache-celeborn-0.5.0-source.tgz),
[Binaries](https://archive.apache.org/dist/celeborn/celeborn-0.5.0/))
diff --git a/mkdocs.yml b/mkdocs.yml
index f2c53d1d..904c55fc 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -51,7 +51,7 @@ markdown_extensions:
extra:
version: 0.6.0-SNAPSHOT
- stable_version: 0.5.1
+ stable_version: 0.5.2
social:
- icon: fontawesome/brands/github
