I think besides of point 1. and 3. there are no dependencies between the RM
and TM side changes. Also, I'm not sure whether it makes sense to split the
slot manager changes up into the proposed steps 5, 6 and 7.

I would highly recommend to not add too much duplicate logic/separate code
paths because it just adds blind spots which are probably not as well
tested as the old code paths.

Cheers,
Till

On Thu, Sep 19, 2019 at 11:58 AM Xintong Song <tonysong...@gmail.com> wrote:

> Thanks for the comments, Till.
>
> - Agree on removing SlotID.
>
> - Regarding the implementation plan, it is true that we can possibly reduce
> codes separated by the feature option. But I think to do that we need to
> introduce more dependencies between implementation steps. With the current
> plan, we can easily separate steps on the RM side and the TM side, and
> start concurrently working on them after quickly updating the interfaces in
> between. The feature will come alive when the steps on both RM/TM sides are
> finished. Since we are planning to have two persons (Andrey and I) working
> on this FLIP, I think the current plan is probably more convenient.
>
> Thank you~
>
> Xintong Song
>
>
>
> On Thu, Sep 19, 2019 at 5:09 PM Till Rohrmann <trohrm...@apache.org>
> wrote:
>
> > Hi Xintong,
> >
> > thanks for starting the vote. The general plan looks good. Hence +1 from
> my
> > side. I still have some minor comments one could think about:
> >
> > * As we no longer have predetermined slots on the TaskExecutor, I think
> we
> > can get rid of the SlotID. Instead, an allocated slot will be identified
> by
> > the AllocationID and the TaskManager's ResourceID in order to
> differentiate
> > duplicate registrations.
> > * For the implementation plan, I believe there is only one tiny part on
> the
> > SlotManager for which we need a separate code path/feature flag which is
> > how we find a matching slot. Everything else should be possible to
> > implement in a way that it works with dynamic and static slot allocation:
> > 1. Let TMs register with default slot profile at RM
> > 2. Change SlotManager to use reported slot profiles instead of
> > pre-calculated profiles
> > 3. Replace SlotID with SlotProfile in TaskExecutorGateway#requestSlot
> > 4. Extend TM to support dynamic slot allocation (aka proper bookkeeping)
> > (can happen concurrently to any of steps 2-3)
> > 5. Add bookkeeping to SlotManager (for pending TMs and registered TMs)
> but
> > still only use default slot profiles for matching with slot requests
> > 6. Allow to match slot requests with reported resources instead of
> default
> > slot profiles (here we could use a feature flag to switch between dynamic
> > and static slot allocation)
> >
> > Wdyt?
> >
> > Cheers,
> > Till
> >
> > On Thu, Sep 19, 2019 at 9:45 AM Andrey Zagrebin <azagre...@apache.org>
> > wrote:
> >
> > > Hi Xintong,
> > >
> > > Thanks for starting the vote, +1 from my side.
> > >
> > > Best,
> > > Andrey
> > >
> > > On Tue, Sep 17, 2019 at 4:26 PM Xintong Song <tonysong...@gmail.com>
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I would like to start the vote for FLIP-56 [1], on which a consensus
> is
> > > > reached in this discussion thread [2].
> > > >
> > > > The vote will be open for at least 72 hours. I'll try to close it
> after
> > > > Sep. 20 15:00 UTC, unless there is an objection or not enough votes.
> > > >
> > > > Thank you~
> > > >
> > > > Xintong Song
> > > >
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-56%3A+Dynamic+Slot+Allocation
> > > >
> > > > [2]
> > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-56-Dynamic-Slot-Allocation-td31960.html
> > > >
> > >
> >
>

Reply via email to