gt;> The design looks good, and I also support leaving space for proposal
> 1.
> >> >>
> >> >> As you know, loading index/filter/data blocks for querying across
> levels
> >> >> would introduce high IO access within the LSM tree for old data. If
> we
>
From: Feifan Wang
Sent: Thursday, March 28, 2024 12:35
To: dev@flink.apache.org
Subject: Re: [DISCUSS] FLIP-427: Disaggregated State Store
Thanks for your reply, Hangxiang. I totally agree with you about the jni part.
Hi Yun Tang, I just noticed that FLIP-427 mentions
at would be N
>> times
>> >> JNI cost compared with the current RocksDB state-backend's JNI cost.
>> >>
>> >> The other question is that the configuration of
>> >> `state.backend.forSt.working-dir` looks too coupled with the ForSt
>> >&g
n of
> >> `state.backend.forSt.working-dir` looks too coupled with the ForSt
> >> state-backend, how would it be if we introduce another disaggregated
> state
> >> storage? Thus, I think `state.backend.disaggregated.working-dir` might
> be a
> >> better
iguration of
>> `state.backend.forSt.working-dir` looks too coupled with the ForSt
>> state-backend, how would it be if we introduce another disaggregated state
>> storage? Thus, I think `state.backend.disaggregated.working-dir` might be a
>> better configuration name.
>>
t be if we introduce another disaggregated state
> storage? Thus, I think `state.backend.disaggregated.working-dir` might be a
> better configuration name.
>
>
> Best
> Yun Tang
>
>
> From: Hangxiang Yu
> Sent: Wednesday, March 20, 2024 11:3
think `state.backend.disaggregated.working-dir` might be a
better configuration name.
Best
Yun Tang
From: Hangxiang Yu
Sent: Wednesday, March 20, 2024 11:32
To: dev@flink.apache.org
Subject: Re: [DISCUSS] FLIP-427: Disaggregated State Store
Hi, Yue.
Thanks
Hi, Yue.
Thanks for the reply.
If we use proposal1, we can easily reuse these optimizations .It is even
> possible to discuss and review the solution together in the Rocksdb
> community.
We also saw these useful optimizations which could be applied to ForSt in
the future.
But IIUC, it's not
Hi Hangxiang,
Thanks for bringing this discussion.
I have a few questions about the Proposal you mentioned in the FLIP.
The current conclusion is to use proposal 2, which is okay for me. My point
is whether we should retain the potential of proposal 1 in the design.
There are the following
Hi everyone,
Thanks for your valuable feedback!
Our discussions have been going on for a while.
As a sub-FLIP of FLIP-423 which is nearing a consensus, I would like to
start a vote after 72 hours.
Please let me know if you have any concerns, thanks!
On Mon, Mar 11, 2024 at 11:48 AM Hangxiang
Hi, Jeyhun.
Thanks for the reply.
Is this argument true for all workloads? Or does this argument also hold
for workloads with many small files, which is quite a common case [1] ?
Yes, I think so. The overhead should still be considered negligible,
particularly in comparison to remote I/O, and
Hi Hangxiang,
Thanks for the proposal. +1 for it.
I have a few comments.
Proposal 2 has additional JNI overhead, but the overhead is relatively
> negligible when weighed against the latency of remote I/O.
- Is this argument true for all workloads? Or does this argument also hold
for workloads
Hi devs,
I'd like to start a discussion on a sub-FLIP of FLIP-423: Disaggregated
State Storage and Management[1], which is a joint work of Yuan Mei, Zakelly
Lan, Jinzhong Li, Hangxiang Yu, Yanfei Lei and Feng Wang:
- FLIP-427: Disaggregated State Store
This FLIP introduces the initial version
13 matches
Mail list logo
