[RESULT][VOTE] FLIP-108: Add GPU support in Flink

2020-04-27 Thread Yangze Guo
Hi all, The voting time for FLIP-108[1] has passed. I'm closing the vote now. There were 3 + 3 votes, 3 of which are binding: - Till (binding) - Becket (binding) - Stephan (binding) - Xintong Song (non-binding) - Canbin Zheng (non-binding) - Yang Wang (non-binding) There were no -1 votes.

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-27 Thread Stephan Ewen
+1 On Thu, Apr 16, 2020 at 4:17 AM Yangze Guo wrote: > Hi Aljoscha, > > Thanks for your advice. +1 to align the config pattern. > > I also agree that we need to move the long discussion to the [DISCUSS] > thread. Sorry if it bothers you. > > Best, > Yangze Guo > > On Thu, Apr 16, 2020 at 7:52

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-15 Thread Yangze Guo
Hi Aljoscha, Thanks for your advice. +1 to align the config pattern. I also agree that we need to move the long discussion to the [DISCUSS] thread. Sorry if it bothers you. Best, Yangze Guo On Thu, Apr 16, 2020 at 7:52 AM Becket Qin wrote: > > I agree with Aljoscha. It is important to keep

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-15 Thread Becket Qin
I agree with Aljoscha. It is important to keep API the same style. And we probably should move the long discussion to the [DISCUSS] thread. Thanks, Jiangjie (Becket) Qin On Wed, Apr 15, 2020 at 11:27 PM Aljoscha Krettek wrote: > Is the only really new method on the public APIs >

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-15 Thread Aljoscha Krettek
Is the only really new method on the public APIs getExternalResourceInfos(..) on the RuntimeContext? I'm generally quite skeptical about adding anything to that interface but the method seems ok. Side note for the configuration keys: the pattern is similar to metrics configuration. There we

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-15 Thread Yangze Guo
Thanks for the explanation. I do not have a strong opinion regarding this interface. So, if it is better from your perspective, I'm +1 for this. I just saying it may not help a lot regarding the type-safe. Regarding the bounded wildcard type, yes, it's the implementation detail. If it won't make

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-15 Thread Xintong Song
True, the user can always pass in ExternalResourceInfo.class to retain the flexibility. As long as the flexibility is not harmed, I'm ok with both. It's probably better to do the type checking and exception handling for users. Thank you~ Xintong Song On Wed, Apr 15, 2020 at 4:23 PM Till

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-15 Thread Till Rohrmann
I think Set getExternalResourceInfos(String resourceName, Class externalResourceType) is not less flexible than the other API since you can always pass in ExternalResourceInfo.class as the second argument. The benefit I see for the user is that he does not have to do the instanceof checks and

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-15 Thread Till Rohrmann
Is this something we expect to happen? If there is an external resource implementation which shows this behaviour I assume that it is quite hard to use for a user. In this case, externalResourceType could also be set to ExternalResourceInfo.class or any common super type. Then the API won't give

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-15 Thread Yangze Guo
> One minor note: I think the value of the returned map does not need to use a > bounded wildcard type because for the user it won't make a difference. Since the Driver now use a bounded wildcard type, it seems we could not hold the return value(Set) with Set. Am I right? Best, Yangze Guo On

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-15 Thread Xintong Song
> > I agree that such an interface won't give compile time checks but I think > that it could be easier to use from a user's perspective because there is > no explicit casting required. > public interface RuntimeContext { > Set getExternalResourceInfos(String > resourceName, Class

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-15 Thread Yangze Guo
@Till If we add "Class externalResourceType" param, what if there are multiple subtypes in the ExternalResourceInfos set of one external resource? It seems user has to set the T to ExternalResourceInfo and the mechanism is useless at this case. Best, Yangze Guo On Wed, Apr 15, 2020 at 3:57 PM

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-15 Thread Till Rohrmann
Ok, if there can be multiple resources of the same type then we definitely need the name as a differentiator. I agree that such an interface won't give compile time checks but I think that it could be easier to use from a user's perspective because there is no explicit casting required. public

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-15 Thread Yangze Guo
Hi Till, > ExternalResourceDriver could return a Set. It sounds good. > then one could make the interface type-safe by changing it to > public interface RuntimeContext { > Set > getExternalResourceInfo(Class externalResourceType); > } I think it may not help. - I think the assumption of

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-14 Thread Till Rohrmann
Thanks for updating the FLIP, Yangze. If ExternalResourceInfo is a marker interface, then ExternalResourceDriver could return a Set. This makes is a bit nicer for the implementor because he can use the concrete subtype. If we assume that users will always cast the ExternalResourceInfo instance

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-12 Thread Xintong Song
Thanks for updating the FLIP, Yangze. The latest FLIP looks good to me. nit: Javadoc of `ExternalResourceDriver#retrieveResourceInfo` is out of sync. > Retrieve the information of the external resources according to the > resourceProfile. Thank you~ Xintong Song On Sat, Apr 11, 2020 at

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-10 Thread Becket Qin
Good feedback form Xintong. The latest FLIP looks good to me. Thanks, Jiangjie (Becket) Qin On Sat, Apr 11, 2020 at 9:20 AM Yangze Guo wrote: > Hi there, > I've updated the FLIP accordingly. Please take a look. If you have any > further concerns please let me know. > > Best, > Yangze Guo > >

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-10 Thread Yangze Guo
Hi there, I've updated the FLIP accordingly. Please take a look. If you have any further concerns please let me know. Best, Yangze Guo On Fri, Apr 10, 2020 at 6:40 PM Yangze Guo wrote: > > Thanks for the feedback, Xintong. > > - Should we have a factory interface for `ExternalResourceDriver`,

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-10 Thread Yangze Guo
Thanks for the feedback, Xintong. - Should we have a factory interface for `ExternalResourceDriver`, that takes the configuration and returns a driver instance? Otherwise, if we are creating the driver instance with reflection, we kind of implicitly requires the driver to have a public

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-10 Thread Xintong Song
Sorry to pull this back. I have some concerns about the recent updated interface details. - Should we have a factory interface for `ExternalResourceDriver`, that takes the configuration and returns a driver instance? Otherwise, if we are creating the driver instance with reflection, we kind of

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-09 Thread Becket Qin
+1 Thanks for driving this effort, Ynagze. The latest FLIP wiki looks good to me. Cheers, Jiangjie (Becket) Qin On Wed, Apr 8, 2020 at 4:10 PM Yangze Guo wrote: > Edit: RuntimeContext interface > From: Map> > getExternalResourceInfo(ResourceSpec resourceSpec); > To: Map>

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-08 Thread Yangze Guo
Edit: RuntimeContext interface From: Map> getExternalResourceInfo(ResourceSpec resourceSpec); To: Map> getExternalResourceInfo(); Best, Yangze Guo On Wed, Apr 8, 2020 at 11:36 AM Yangze Guo wrote: > > Hi, there > > I have updated the FLIP, mainly target to make it more detailed and > clear.

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-07 Thread Yangze Guo
Hi, there I have updated the FLIP, mainly target to make it more detailed and clear. The general design is not changed, but there are still some changes need to be notified here: - Change the `ExternalResourceDriver` from abstract class to interface, since it does not have an abstract

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-07 Thread Yang Wang
Thanks Yangze for the efforts to support GPU extended resources. +1 for this FLIP Best, Yang Till Rohrmann 于2020年4月2日周四 下午11:10写道: > Thanks for driving this effort Yangze. > > +1 > > Cheers, > Till > > On Wed, Apr 1, 2020 at 12:41 PM Canbin Zheng > wrote: > > > Thanks Yangze for driving the

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-02 Thread Till Rohrmann
Thanks for driving this effort Yangze. +1 Cheers, Till On Wed, Apr 1, 2020 at 12:41 PM Canbin Zheng wrote: > Thanks Yangze for driving the initial CPU support! > +1 (non-binding) from my side. > > > Xintong Song 于2020年4月1日周三 下午6:36写道: > > > Thanks Yangze, the FLIP looks good to me. > > +1

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-01 Thread Canbin Zheng
Thanks Yangze for driving the initial CPU support! +1 (non-binding) from my side. Xintong Song 于2020年4月1日周三 下午6:36写道: > Thanks Yangze, the FLIP looks good to me. > +1 (non-binding) from my side. > > Thank you~ > > Xintong Song > > > > On Wed, Apr 1, 2020 at 5:22 PM Yangze Guo wrote: > > > Hi

Re: [VOTE] FLIP-108: Add GPU support in Flink

2020-04-01 Thread Xintong Song
Thanks Yangze, the FLIP looks good to me. +1 (non-binding) from my side. Thank you~ Xintong Song On Wed, Apr 1, 2020 at 5:22 PM Yangze Guo wrote: > Hi everyone, > > I'd like to start the vote of FLIP-108 [1], which adds GPU support in > Flink. This FLIP is discussed in the thread[2]. > >

[VOTE] FLIP-108: Add GPU support in Flink

2020-04-01 Thread Yangze Guo
Hi everyone, I'd like to start the vote of FLIP-108 [1], which adds GPU support in Flink. This FLIP is discussed in the thread[2]. The vote will be open for at least 72 hours. Unless there is an objection, I will try to close it by April 4, 2020 10:00 UTC if we have received sufficient votes.