Thanks for the comments, Till.

- Agree on removing SlotID.

- Regarding the implementation plan, it is true that we can possibly reduce
codes separated by the feature option. But I think to do that we need to
introduce more dependencies between implementation steps. With the current
plan, we can easily separate steps on the RM side and the TM side, and
start concurrently working on them after quickly updating the interfaces in
between. The feature will come alive when the steps on both RM/TM sides are
finished. Since we are planning to have two persons (Andrey and I) working
on this FLIP, I think the current plan is probably more convenient.

Thank you~

Xintong Song



On Thu, Sep 19, 2019 at 5:09 PM Till Rohrmann <trohrm...@apache.org> wrote:

> Hi Xintong,
>
> thanks for starting the vote. The general plan looks good. Hence +1 from my
> side. I still have some minor comments one could think about:
>
> * As we no longer have predetermined slots on the TaskExecutor, I think we
> can get rid of the SlotID. Instead, an allocated slot will be identified by
> the AllocationID and the TaskManager's ResourceID in order to differentiate
> duplicate registrations.
> * For the implementation plan, I believe there is only one tiny part on the
> SlotManager for which we need a separate code path/feature flag which is
> how we find a matching slot. Everything else should be possible to
> implement in a way that it works with dynamic and static slot allocation:
> 1. Let TMs register with default slot profile at RM
> 2. Change SlotManager to use reported slot profiles instead of
> pre-calculated profiles
> 3. Replace SlotID with SlotProfile in TaskExecutorGateway#requestSlot
> 4. Extend TM to support dynamic slot allocation (aka proper bookkeeping)
> (can happen concurrently to any of steps 2-3)
> 5. Add bookkeeping to SlotManager (for pending TMs and registered TMs) but
> still only use default slot profiles for matching with slot requests
> 6. Allow to match slot requests with reported resources instead of default
> slot profiles (here we could use a feature flag to switch between dynamic
> and static slot allocation)
>
> Wdyt?
>
> Cheers,
> Till
>
> On Thu, Sep 19, 2019 at 9:45 AM Andrey Zagrebin <azagre...@apache.org>
> wrote:
>
> > Hi Xintong,
> >
> > Thanks for starting the vote, +1 from my side.
> >
> > Best,
> > Andrey
> >
> > On Tue, Sep 17, 2019 at 4:26 PM Xintong Song <tonysong...@gmail.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > I would like to start the vote for FLIP-56 [1], on which a consensus is
> > > reached in this discussion thread [2].
> > >
> > > The vote will be open for at least 72 hours. I'll try to close it after
> > > Sep. 20 15:00 UTC, unless there is an objection or not enough votes.
> > >
> > > Thank you~
> > >
> > > Xintong Song
> > >
> > >
> > > [1]
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-56%3A+Dynamic+Slot+Allocation
> > >
> > > [2]
> > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-56-Dynamic-Slot-Allocation-td31960.html
> > >
> >
>

Reply via email to