manager architecture.
>
> Best,
> Zhijiang
>
> --
> From:qi luo
> Send Time:2019年1月28日(星期一) 20:55
> To:dev ; zhijiang
> Cc:Till Rohrmann ; Andrey Zagrebin
>
> Subject:Re: [DISCUSS] Proposa
Time:2019年1月28日(星期一) 20:55
To:dev ; zhijiang
Cc:Till Rohrmann ; Andrey Zagrebin
Subject:Re: [DISCUSS] Proposal of external shuffle service
Hi Zhijiang,
I see there’s a YarnShuffleService in newly released Blink branch. Is there any
relationship between that YarnShuffleService and your external
---
> From:Till Rohrmann
> Send Time:2019年1月28日(星期一) 19:14
> To:dev ; zhijiang
> Cc:Andrey Zagrebin
> Subject:Re: [DISCUSS] Proposal of external shuffle service
>
> Thanks for creating the FLIP-31 for the external shuffle service Zhijiang. It
> looks goo
Zagrebin
Subject:Re: [DISCUSS] Proposal of external shuffle service
Thanks for creating the FLIP-31 for the external shuffle service Zhijiang. It
looks good to me.
One thing which is not fully clear to me yet is how the lifecycle management of
the partitions integrates with the slot management
t; ------
> From:zhijiang
> Send Time:2018年12月25日(星期二) 15:33
> To:Andrey Zagrebin
> Cc:dev
> Subject:Re: [DISCUSS] Proposal of external shuffle service
>
> Hi Andrey,
>
> Thanks for efficient response for the
://issues.apache.org/jira/browse/FLINK-10653
Best,
Zhijiang
--
From:zhijiang
Send Time:2019年1月15日(星期二) 17:55
To:Andrey Zagrebin
Cc:dev
Subject:Re: [DISCUSS] Proposal of external shuffle service
Hi all,
After continuous discussion with Andrey
iang
--
From:Andrey Zagrebin
Send Time:2018年12月25日(星期二) 02:32
To:zhijiang
Cc:dev
Subject:Re: [DISCUSS] Proposal of external shuffle service
Hi Zhijiang,
Thanks for considering my thoughts and concerns. Those are just suggestions for
your design document.
iang
--
From:Andrey Zagrebin
Send Time:2018年12月25日(星期二) 02:32
To:zhijiang
Cc:dev
Subject:Re: [DISCUSS] Proposal of external shuffle service
Hi Zhijiang,
Thanks for considering my thoughts and concerns. Those are just suggestions for
your design document.
My understanding about 2.1 was initi
y involved in many changes on both *TM*, *JM*
> sides. :)
>
> Best,
> Zhijiang
>
>
>
> --------------
> From:Andrey Zagrebin
> Send Time:2018年12月20日(星期四) 01:20
> To:zhijiang
> Cc:dev
> Subject:Re: [DISCUSS]
--
From:Andrey Zagrebin
Send Time:2018年12月20日(星期四) 01:20
To:zhijiang
Cc:dev
Subject:Re: [DISCUSS] Proposal of external shuffle service
Hi Zhijiang,
Thanks for detailed answers! I am glad we are on the same page.
I spent some time on thinking
he implementation aspects. And I agree with the above points,
> especially for above 2.2 you might need double check if I understand
> correctly.
> Wish your further feedbacks then I can adjust the docs based on it. Also
> welcome any other person's feedbacks!
>
>
> Best,
> Zhijiang
>
&
收件人：dev ; zhijiang
抄 送：Nico Kruber ; Piotr Nowojski
; Stephan Ewen ; Till Rohrmann
主 题：Re: [DISCUSS] Proposal of external shuffle service
Hi Zhijiang,
Thanks for sharing the document Zhijiang.
I decided to compile my thoughts to consider here, not to overload document
comments any more :)
I
7:19
> 收件人：dev ; Jin Sun
> 抄 送：Nico Kruber ; Piotr Nowojski
> ; Stephan Ewen
> 主 题：回复：[DISCUSS] Proposal of external shuffle service
>
> Thanks for the efficient response till!
>
> Thanks sunjin for the good feedbacks, we will further confirm with the
> comments then! :)
; Stephan Ewen
主 题：回复：[DISCUSS] Proposal of external shuffle service
Thanks for the efficient response till!
Thanks sunjin for the good feedbacks, we will further confirm with the comments
then! :)
--
发件人：Jin Sun
发送时间：2018年11月1日(星期四
; Piotr Nowojski ; Stephan
Ewen
主 题：Re: [DISCUSS] Proposal of external shuffle service
Thanks Zhijiang for the proposal. I like the idea of external shuffle service,
have left some comments on the document.
> On Oct 31, 2018, at 2:26 AM, Till Rohrmann wrote:
>
> Thanks for the update
>>
>> <https://docs.google.com/document/d/1Jb0Mf46ace-6cLRQxJzo6VNQQVxn3hwf9Zqmv5pcb34/edit?usp=sharing>
>> Best,
>> Zhijiang
>>
>> ------
>> 发件人：Zhijiang(wangzhijiang999)
>> 发送时间：2018年
t;
> --
> 发件人：Till Rohrmann
> 发送时间：2018年9月7日(星期五) 22:01
> 收件人：Zhijiang(wangzhijiang999)
> 抄 送：dev
> 主 题：Re: [DISCUSS] Proposal of external shuffle service
>
> The rough plan sounds good Zhijiang. I think we
-6cLRQxJzo6VNQQVxn3hwf9Zqmv5pcb34/edit?usp=sharing
Best,
Zhijiang
--
发件人：Zhijiang(wangzhijiang999)
发送时间：2018年9月11日(星期二) 15:21
收件人：dev
抄 送：dev
主 题：回复：[DISCUSS] Proposal of external shuffle service
Many thanks Till!
I would create a JIRA for this feature
：dev
主 题：Re: [DISCUSS] Proposal of external shuffle service
The rough plan sounds good Zhijiang. I think we should continue with what
you've proposed: Open a JIRA issue and creating a design document which
outlines the required changes a little bit more in detail. Once this is
done, we should link
:36
> 收件人：dev ; Zhijiang(wangzhijiang999) <
> wangzhijiang...@aliyun.com>
> 主 题：Re: [DISCUSS] Proposal of external shuffle service
>
> Thanks for starting this design discussion Zhijiang!
>
> I really like the idea to introduce a ShuffleService abstraction which
> a
间：2018年8月29日(星期三) 17:36
收件人：dev ; Zhijiang(wangzhijiang999)
主 题：Re: [DISCUSS] Proposal of external shuffle service
Thanks for starting this design discussion Zhijiang!
I really like the idea to introduce a ShuffleService abstraction which allows
to have different implementations depending
Thanks for starting this design discussion Zhijiang!
I really like the idea to introduce a ShuffleService abstraction which
allows to have different implementations depending on the actual use case.
Especially for batch jobs I can clearly see the benefits of persisting the
results somewhere else.
Hi all!
The shuffle service is responsible for transporting upstream produced data to
the downstream side. In flink, the NettyServer is used for network transport
service and this component is started in the TaskManager process. That means
the TaskManager can support internal shuffle service
23 matches
Mail list logo
