This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn-website.git
The following commit(s) were added to refs/heads/main by this push:
new 93f80f9a Release 0.4.1 (#62)
93f80f9a is described below
commit 93f80f9a60d9fcc166301aedaa58f9288e7fcde9
Author: Nicholas Jiang <[email protected]>
AuthorDate: Wed May 22 20:11:04 2024 +0800
Release 0.4.1 (#62)
* Release 0.4.1
* Update .github/workflows/site.yaml
---------
Co-authored-by: Cheng Pan <[email protected]>
---
.github/workflows/site.yaml | 1 +
docs/community/news.md | 1 +
docs/community/release_notes/release_note_0.4.1.md | 108 +++++++++++++++++++++
docs/download.md | 24 +++--
mkdocs.yml | 2 +-
5 files changed, 122 insertions(+), 14 deletions(-)
diff --git a/.github/workflows/site.yaml b/.github/workflows/site.yaml
index 1e5b2961..df30dc5b 100644
--- a/.github/workflows/site.yaml
+++ b/.github/workflows/site.yaml
@@ -55,6 +55,7 @@ jobs:
- run: ./.github/bin/build_docs.sh
'v0.2.1-incubating/incubator-celeborn-0.2.1-incubating-tm.tar.gz'
'0.2.1-incubating'
- run: ./.github/bin/build_docs.sh
'v0.3.2-incubating/incubator-celeborn-0.3.2-incubating-tm.tar.gz'
'0.3.2-incubating'
- run: ./.github/bin/build_docs.sh
'v0.4.0-incubating/incubator-celeborn-0.4.0-incubating-tm.tar.gz'
'0.4.0-incubating'
+ - run: ./.github/bin/build_docs.sh 'v0.4.1/v0.4.1.tar.gz' '0.4.1'
- run: |
echo 'publish:' >> .asf.yaml
echo ' whoami: asf-site' >> .asf.yaml
diff --git a/docs/community/news.md b/docs/community/news.md
index 5facdad9..df44c405 100644
--- a/docs/community/news.md
+++ b/docs/community/news.md
@@ -20,6 +20,7 @@ license: |
| Date | Title |
Brief
|
|------------------|---------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------|
+| 2024 May 22 | Release 0.4.1 |
Celeborn release 0.4.1.
|
| 2024 April 28 | New Committer: Mridul Muralidharan |
We are happy to announce Mridul Muralidharan becomes a new Celeborn committer.
|
| 2024 March 21 | New Committer: Chandni Singh |
We are happy to announce Chandni Singh becomes a new Celeborn committer.
|
| 2024 February 14 | New PPMC Member: Fu Chen |
We are happy to announce Fu Chen becomes a new Celeborn PPMC member.
|
diff --git a/docs/community/release_notes/release_note_0.4.1.md
b/docs/community/release_notes/release_note_0.4.1.md
new file mode 100644
index 00000000..520c1211
--- /dev/null
+++ b/docs/community/release_notes/release_note_0.4.1.md
@@ -0,0 +1,108 @@
+---
+hide:
+ - navigation
+
+license: |
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ https://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+# Apache Celeborn™ 0.4.1 Release Notes
+
+## Highlight
+
+- Improve flusher's robustness
+- Optimize `CelebornInputStreamImpl`'s memory usage
+- Fix `Worker#computeResourceConsumption` `NullPointerException` for
`userResourceConsumption` that does not contain given userIdentifier
+- Fix unregisterShuffle with `celeborn.client.spark.fetch.throwsFetchFailure`
disabled
+
+### Improvement
+
+- [CELEBORN-1174] Introduce application dimension resource consumption metrics
+- [CELEBORN-1182] Support application dimension `ActiveConnectionCount` metric
to record the number of registered connections for each application
+- [CELEBORN-1244] Delete redundant remove operations and handle timeout
requests in final check
+- [CELEBORN-1248] Improve flusher's robustness
+- [CELEBORN-1259] Improve the default gracePeriod of `ThreadUtils#shutdown`
+- [CELEBORN-1266] Improve log of current failed workers for
`WorkerStatusTracker`
+- [CELEBORN-1272] Do not increment epoch when retry commit
+- [CELEBORN-1278] Avoid calculating all outstanding requests to improve
performance
+- [CELEBORN-1283] `TransportClientFactory` avoid contention and get or create
clientPools quickly
+- [CELEBORN-1288] Prompt configuration items when receiving IdleStateEvent
+- [CELEBORN-1291] Master crashed causing by huge app level worker consumption
info
+- [CELEBORN-1292] Remove app level metrics from worker and master
+- [CELEBORN-1298] Support Spark 2.4 with Scala 2.12
+- [CELEBORN-1300] Optimize `CelebornInputStreamImpl`'s memory usage
+- [CELEBORN-1301] Catch and throw `FetchFailedException` in
`CelebornInputStream#fillBuffer`
+- [CELEBORN-1312] Move `handleRequestPartitions` out of sync block
+- [CELEBORN-1315] Manually close the RocksDB/LevelDB instance when
`checkVersion` throw Exception
+- [CELEBORN-1316] Override `toString` method for `StoreVersion`
+- [CELEBORN-1324] Remove unused `PrometheusSink` class
+- [CELEBORN-1326] `FakedRemoteInputChannel` use task name of
`RemoteShuffleInputGateDelegation` as owningTaskName
+- [CELEBORN-1339] Mark connection as timedOut in `TransportClient#close`
+- [CELEBORN-1345] Add a limit to the master's estimated partition size
+- [CELEBORN-1363] `AbstractRemoteShuffleInputGateFactory` supports
`celeborn.client.shuffle.compression.codec` to configure compression codec
+- [CELEBORN-1376] Push data failed should always release request body
+- [CELEBORN-1379] Catch `Throwable` for `ReadBufferDispatcher` thread
+- [CELEBORN-1380] leveldbjni uses org.openlabtesting.leveldbjni to support
linux aarch64 platform for leveldb
+- [CELEBORN-1381] Avoid construct TransportConf when creating
`CelebornInputStream`
+- [CELEBORN-1384] Manually excluding workers should not depend on whether the
workers are alive
+- [CELEBORN-1386] `LevelDBProvider`/`RocksDBProvider` should create
non-existent multi-level directory for LevelDB/RocksDB initialization
+- [CELEBORN-1391] Retry when MasterClient receiving a `RpcTimeoutException`
+- [CELEBORN-1398] Support return leader ip to client
+- [CELEBORN-1399] MR `CelebornMapOutputCollector` should check exception after
flush
+- [CELEBORN-1407] Change log4j2 template appender to file
+- [CELEBORN-1408] `workerShuffleCommitTimeout` should use millisecond units
+- [CELEBORN-1409] `CommitHandler` commitFiles RPC supports separate timeout
configuration
+- [CELEBORN-1411] Change default log level to INFO when there is no log4j2
config file
+- [CELEBORN-1412] `celeborn.client.rpc.*.askTimeout` should fallback to
`celeborn.rpc.askTimeout`
+
+### Stability and Bug Fix
+
+- [CELEBORN-448] `HeartbeatFromApplicationResponse` should include manually
excluded workers
+- [CELEBORN-863] Fix persisted committed file infos lost
+- [CELEBORN-1252] Fix `Worker#computeResourceConsumption`
`NullPointerException` for userResourceConsumption that does not contain given
userIdentifier
+- [CELEBORN-1271] Fix `unregisterShuffle` with
`celeborn.client.spark.fetch.throwsFetchFailure` disabled
+- [CELEBORN-1275] Fix bug that callback function may hang when unchecked
exception missed
+- [CELEBORN-1282] Fix `FetchHandler#handleEndStreamFromClient`
`NullPointerException` after recycling stream of `CreditStreamManager`
+- [CELEBORN-1290] Fix NPE occurring prior to worker registration
+- [CELEBORN-1420] Fix mapreduce job will throw an exit exception after it
succeeded
+
+### Build
+
+- [CELEBORN-1310] License check add flink-1.19 profile
+- [CELEBORN-1404] Disable SBT ANSI color on extracting info from output
+- [CELEBORN-1405] SBT allows using credential without a realm
+
+### Documentation
+
+- [CELEBORN-1260] Improve Spark Configuration of Deploy Spark client for
deployment document
+- [CELEBORN-1295] Add tm to Celeborn's website and change repo_url to apache
repo
+
+### Dependencies
+
+- [CELEBORN-1006] Dependency hadoop-client should exclude
hadoop-mapreduce-client dependencies for Hadoop 2
+- [CELEBORN-1330] Bump rocksdbjni version from 8.5.3 to 8.11.3
+- [CELEBORN-1331] Remove third-party dependencies in shaded clients' pom
+- [CELEBORN-1366] Bump guava from 32.1.3-jre to 33.1.0-jre
+
+## Credits
+
+Thanks to the following contributors who helped to review and commit to Apache
Celeborn 0.4.1 version:
+
+| Contributors | | | |
| |
+|--------------|--------------|--------------|-------------|---------------------|----------------|
+| Angerszhuuuu | Cheng Pan | Erik.fang | Ethan Feng | Fei Wang
| Fu Chen |
+| Fulong Li | Jiashu Xiong | Kerwin Zhang | Keyong Zhou | Mridul
Muralidharan | Nicholas Jiang |
+| Qingbo Jiao | Shaoyun Chen | Yanze Jiang | Yihe Li |
| |
diff --git a/docs/download.md b/docs/download.md
index ddd0b587..0e921693 100644
--- a/docs/download.md
+++ b/docs/download.md
@@ -23,32 +23,30 @@ license: |
The latest version is {{ stable_version }}.
+### 0.4.1 (2024-05-22)
-### 0.4.0-incubating (2024-02-06)
-
-[release note](community/release_notes/release_note_0.4.0.md)
-
-| |
Download from ASF |
Checksum
|
Signature
|
-|:-----------:|:-------------------------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------------------------------------:|
-| Source Code |
[src](https://www.apache.org/dyn/closer.lua/incubator/celeborn/celeborn-0.4.0-incubating/apache-celeborn-0.4.0-incubating-source.tgz)
|
[sha512](https://downloads.apache.org/incubator/celeborn/celeborn-0.4.0-incubating/apache-celeborn-0.4.0-incubating-source.tgz.sha512)
|
[asc](https://downloads.apache.org/incubator/celeborn/celeborn-0.4.0-incubating/apache-celeborn-0.4.0-incubating-source.tgz.asc)
|
-| Binary |
[bin](https://www.apache.org/dyn/closer.lua/incubator/celeborn/celeborn-0.4.0-incubating/apache-celeborn-0.4.0-incubating-bin.tgz)
|
[sha512](https://downloads.apache.org/incubator/celeborn/celeborn-0.4.0-incubating/apache-celeborn-0.4.0-incubating-bin.tgz.sha512)
|
[asc](https://downloads.apache.org/incubator/celeborn/celeborn-0.4.0-incubating/apache-celeborn-0.4.0-incubating-bin.tgz.asc)
|
-
+[release note](community/release_notes/release_note_0.4.1.md)
+| | Download from ASF
|
Checksum |
Signature |
+|:-----------:|:--------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------:|
+| Source Code |
[src](https://downloads.apache.org/celeborn/celeborn-0.4.1/apache-celeborn-0.4.1-source.tgz)
|
[sha512](https://downloads.apache.org/celeborn/celeborn-0.4.1/apache-celeborn-0.4.1-source.tgz.sha512)
|
[asc](https://downloads.apache.org/celeborn/celeborn-0.4.1/apache-celeborn-0.4.1-source.tgz.asc)
|
+| Binary |
[bin](https://downloads.apache.org/celeborn/celeborn-0.4.1/apache-celeborn-0.4.1-bin.tgz)
|
[sha512](https://downloads.apache.org/celeborn/celeborn-0.4.1/apache-celeborn-0.4.1-bin.tgz.sha512)
|
[asc](https://downloads.apache.org/celeborn/celeborn-0.4.1/apache-celeborn-0.4.1-bin.tgz.asc)
|
### 0.3.2-incubating (2024-01-08)
[release note](community/release_notes/release_note_0.3.2.md)
-| |
Download from ASF |
Checksum
|
Signature
|
-|:-----------:|:-------------------------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------:|
-| Source Code |
[src](https://www.apache.org/dyn/closer.lua/incubator/celeborn/celeborn-0.3.2-incubating/apache-celeborn-0.3.2-incubating-source.tgz)
|
[sha512](https://downloads.apache.org/incubator/celeborn/celeborn-0.3.2-incubating/apache-celeborn-0.3.2-incubating-source.tgz.sha512)
|
[asc](https://downloads.apache.org/incubator/celeborn/celeborn-0.3.2-incubating/apache-celeborn-0.3.2-incubating-source.tgz.asc)
|
-| Binary |
[bin](https://www.apache.org/dyn/closer.lua/incubator/celeborn/celeborn-0.3.2-incubating/apache-celeborn-0.3.2-incubating-bin.tgz)
|
[sha512](https://downloads.apache.org/incubator/celeborn/celeborn-0.3.2-incubating/apache-celeborn-0.3.2-incubating-bin.tgz.sha512)
|
[asc](https://downloads.apache.org/incubator/celeborn/celeborn-0.3.2-incubating/apache-celeborn-0.3.2-incubating-bin.tgz.asc)
|
+| | Download from
ASF |
Checksum
|
Signature |
+|:-----------:|:------------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------------------:|
+| Source Code |
[src](https://downloads.apache.org/celeborn/celeborn-0.3.2-incubating/apache-celeborn-0.3.2-incubating-source.tgz)
|
[sha512](https://downloads.apache.org/celeborn/celeborn-0.3.2-incubating/apache-celeborn-0.3.2-incubating-source.tgz.sha512)
|
[asc](https://downloads.apache.org/celeborn/celeborn-0.3.2-incubating/apache-celeborn-0.3.2-incubating-source.tgz.asc)
|
+| Binary |
[bin](https://downloads.apache.org/celeborn/celeborn-0.3.2-incubating/apache-celeborn-0.3.2-incubating-bin.tgz)
|
[sha512](https://downloads.apache.org/celeborn/celeborn-0.3.2-incubating/apache-celeborn-0.3.2-incubating-bin.tgz.sha512)
|
[asc](https://downloads.apache.org/celeborn/celeborn-0.3.2-incubating/apache-celeborn-0.3.2-incubating-bin.tgz.asc)
|
### All stable releases
All celeborn releases are available via
[https://archive.apache.org/dist/celeborn/](https://archive.apache.org/dist/celeborn/)
including checksums and
signatures. At the time of writing, this includes the following versions:
+* Apache Celeborn 0.4.1 (2024-05-22)
([Source](https://archive.apache.org/dist/celeborn/celeborn-0.4.1/apache-celeborn-0.4.1-source.tgz),
[Binaries](https://archive.apache.org/dist/celeborn/celeborn-0.4.1/))
* Apache Celeborn 0.4.0-incubating (2024-02-06)
([Source](https://archive.apache.org/dist/incubator/celeborn/celeborn-0.4.0-incubating/apache-celeborn-0.4.0-incubating-source.tgz),
[Binaries](https://archive.apache.org/dist/incubator/celeborn/celeborn-0.4.0-incubating/))
* Apache Celeborn 0.3.2-incubating (2024-01-08)
([Source](https://archive.apache.org/dist/incubator/celeborn/celeborn-0.3.2-incubating/apache-celeborn-0.3.2-incubating-source.tgz),
[Binaries](https://archive.apache.org/dist/incubator/celeborn/celeborn-0.3.2-incubating/))
* Apache Celeborn 0.3.1-incubating (2023-10-13)
([Source](https://archive.apache.org/dist/incubator/celeborn/celeborn-0.3.1-incubating/apache-celeborn-0.3.1-incubating-source.tgz),
[Binaries](https://archive.apache.org/dist/incubator/celeborn/celeborn-0.3.1-incubating/))
diff --git a/mkdocs.yml b/mkdocs.yml
index 8c7bdfa3..51597336 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -51,7 +51,7 @@ markdown_extensions:
extra:
version: 0.5.0-SNAPSHOT
- stable_version: 0.4.0
+ stable_version: 0.4.1
social:
- icon: fontawesome/brands/github
