Celeborn and Uniffle can also be seen as a move to separate local storage
from compute nodes.
1. In the old days, Hadoop was based on the idea of collocating compute and
storage.
2. Later a new paradigm of separating compute and storage emerged and got
popularized.
3. Now people want to not just
I think both Celeborn and Uniffle are good alternatives as a general shuffle
service.
I recommend that you try them : ). For any question about Celeborn, we're very
glad
to discuss in Celeborn's mail lists[1][2] or slack[3].
[1] u...@celeborn.apache.org
[2] d...@celeborn.apache.org
[3]
On Thu, Nov 2, 2023 at 1:43 PM Sungwoo Park wrote:
> Have you done comparison between uniffle and celeborn..?
>>
>
> We did not compare the performance of Uniffle and Celeborn (because
> Hive-MR3-Celeborn has been released but Hive-MR3-Uniffle is not complete
> yet). Much of the code in
>
> Have you done comparison between uniffle and celeborn..?
>
We did not compare the performance of Uniffle and Celeborn (because
Hive-MR3-Celeborn has been released but Hive-MR3-Uniffle is not complete
yet). Much of the code in Hive-MR3-Celeborn is currently reused in
Hive-MR3-Uniffle, so we
Thanks for bringing up this. Good to see that it supports spark and flink.
Have you done comparison between uniffle and celeborn..?
On 30/10/23, 8:01 AM, "Keyong Zhou" mailto:zho...@apache.org>> wrote:
Great to hear this! It's encouraging that Celeborn helps MR3.
Celeborn is a general
Great to hear this! It's encouraging that Celeborn helps MR3.
Celeborn is a general purpose remote shuffle service that stores and serves
shuffle data (and other intermediate data in the future) to help compute
engines
better use disaggregated architecture, as well as become more efficient and
Thanks. I will try.
Replied Message
| From | Sungwoo Park |
| Date | 10/24/2023 20:08 |
| To | user@hive.apache.org |
| Cc | |
| Subject | Announce: Hive-MR3 with Celeborn, |
Hi Hive users,
Before the impending release of MR3 1.8, we would like to announce the release
of Hive-MR3
Hi Hive users,
Before the impending release of MR3 1.8, we would like to announce the
release of Hive-MR3 with Celeborn (Hive 3.1.3 on MR3 1.8 with Celeborn
0.3.1).
Apache Celeborn [1] is remote shuffle service, similar to Magnet [2] and
Apache Uniffle [3] (which was discussed in this Hive
