Re: [DISCUSS] Drop stale class Program
Thanks for your attentions!
Thank Kostas for creating the JIRA and drafting the FLIP.
I would volunteer to help review it :-)
It's good to see that we make progress on this thread.
Best,
tison.
Kostas Kloudas 于2019年8月14日周三 下午6:39写道：
> I already opened a JIRA for the removal and I will also create a (short)
> FLIP, as it is a PublicEvolving interface and its removal should go through
> a FLIP.
>
> The JIRA can be found here
> https://issues.apache.org/jira/browse/FLINK-13713
>
> Cheers,
> Kostas
>
> On Wed, Aug 14, 2019 at 12:13 PM Stephan Ewen wrote:
>
> > +1 to drop it.
> >
> > It's one of the oldest pieces of legacy.
> >
> > On Wed, Aug 14, 2019 at 12:07 PM Aljoscha Krettek
> > wrote:
> >
> > > Hi,
> > >
> > > I would be in favour of removing Program (and the code paths that
> support
> > > it) for Flink 1.10. Most users of Flink don’t actually know it exists
> and
> > > it is only making our code more complicated. Going forward with the new
> > > Client API discussions will be a lot easier without it as well.
> > >
> > > Best,
> > > Aljoscha
> > >
> > > > On 14. Aug 2019, at 11:08, Kostas Kloudas
> wrote:
> > > >
> > > > Hi all,
> > > >
> > > > It is nice to have this discussion.
> > > >
> > > > I am totally up for removing the unused Program interface, as this
> will
> > > > simplify a lot of other code paths in the ClusterClient and
> elsewhere.
> > > >
> > > > Also about the easier integration of Flink with other frameworks,
> there
> > > > is another discussion in the mailing list with exactly this topic:
> > > > [DISCUSS] Flink client api enhancement for downstream project
> > > >
> > > > Cheers,
> > > > Kostas
> > > >
> > > >
> > > > On Tue, Jul 30, 2019 at 1:38 PM Zili Chen
> > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> With a one-week survey in user list[1], nobody expect Flavio and
> Jeff
> > > >> participant the thread.
> > > >>
> > > >> Flavio shared his experience with a revised Program like interface.
> > > >> This could be regraded as downstream integration and in client api
> > > >> enhancements document we propose rich interface for this
> integration.
> > > >> Anyway, the Flink scope Program is less functional than it should
> be.
> > > >>
> > > >> With no objection I'd like to push on this thread. We need a
> committer
> > > >> participant this thread to shepherd the removal/deprecation of
> > Program,
> > > a
> > > >> @PublicEvolving interface. Anybody has spare time? Or anything I can
> > do
> > > >> to make progress?
> > > >>
> > > >> Best,
> > > >> tison.
> > > >>
> > > >> [1]
> > > >>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> > > >>
> > > >>
> > > >> Zili Chen 于2019年7月22日周一 下午8:38写道：
> > > >>
> > > >>> Hi,
> > > >>>
> > > >>> I created a thread for survey in user list[1]. Please take
> > participate
> > > in
> > > >>> if interested.
> > > >>>
> > > >>> Best,
> > > >>> tison.
> > > >>>
> > > >>> [1]
> > > >>>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> > > >>>
> > > >>>
> > > >>> Flavio Pompermaier 于2019年7月19日周五 下午5:18写道：
> > > >>>
> > > +1 to remove directly the Program class (I think nobody use it and
> > > it's
> > > not
> > > supported at all by REST services and UI).
> > > Moreover it requires a lot of transitive dependencies while it
> > should
> > > >> be a
> > > very simple thing..
> > > +1 to add this discussion to "Flink client api enhancement"
> > >
> > > On Fri, Jul 19, 2019 at 11:14 AM Biao Liu
> > wrote:
> > >
> > > > To Flavio, good point for the integration suggestion.
> > > >
> > > > I think it should be considered in the "Flink client api
> > enhancement"
> > > > discussion. But the outdated API should be deprecated somehow.
> > > >
> > > > Flavio Pompermaier 于2019年7月19日周五
> 下午4:21写道：
> > > >
> > > >> In my experience a basic "official" (but optional) program
> > > >> description
> > > >> would be very useful indeed (in order to ease the integration
> with
> > > other
> > > >> frameworks).
> > > >>
> > > >> Of course it should be extended and integrated with the REST
> > > >> services
> > > and
> > > >> the Web UI (when defined) in order to be useful..
> > > >> It ease to show to the user what a job does and which parameters
> > it
> > > >> requires (optional or mandatory) and with a proper help
> > description.
> > > >> Indeed, when we write a Flink job we implement the following
> > > interface:
> > > >>
> > > >> public interface FlinkJob {
> > > >> String getDescription();
> > > >> List getParameters();
> > > >> boolean isStreamingOrBatch();
> > > >> }
> > > >>
> > > >> public class ClusterJobParameter {
> > > >>
> > > >> private String paramName;
> > > >> private String
Re: [DISCUSS] Drop stale class Program
I already opened a JIRA for the removal and I will also create a (short)
FLIP, as it is a PublicEvolving interface and its removal should go through
a FLIP.
The JIRA can be found here https://issues.apache.org/jira/browse/FLINK-13713
Cheers,
Kostas
On Wed, Aug 14, 2019 at 12:13 PM Stephan Ewen wrote:
> +1 to drop it.
>
> It's one of the oldest pieces of legacy.
>
> On Wed, Aug 14, 2019 at 12:07 PM Aljoscha Krettek
> wrote:
>
> > Hi,
> >
> > I would be in favour of removing Program (and the code paths that support
> > it) for Flink 1.10. Most users of Flink don’t actually know it exists and
> > it is only making our code more complicated. Going forward with the new
> > Client API discussions will be a lot easier without it as well.
> >
> > Best,
> > Aljoscha
> >
> > > On 14. Aug 2019, at 11:08, Kostas Kloudas wrote:
> > >
> > > Hi all,
> > >
> > > It is nice to have this discussion.
> > >
> > > I am totally up for removing the unused Program interface, as this will
> > > simplify a lot of other code paths in the ClusterClient and elsewhere.
> > >
> > > Also about the easier integration of Flink with other frameworks, there
> > > is another discussion in the mailing list with exactly this topic:
> > > [DISCUSS] Flink client api enhancement for downstream project
> > >
> > > Cheers,
> > > Kostas
> > >
> > >
> > > On Tue, Jul 30, 2019 at 1:38 PM Zili Chen
> wrote:
> > >
> > >> Hi,
> > >>
> > >> With a one-week survey in user list[1], nobody expect Flavio and Jeff
> > >> participant the thread.
> > >>
> > >> Flavio shared his experience with a revised Program like interface.
> > >> This could be regraded as downstream integration and in client api
> > >> enhancements document we propose rich interface for this integration.
> > >> Anyway, the Flink scope Program is less functional than it should be.
> > >>
> > >> With no objection I'd like to push on this thread. We need a committer
> > >> participant this thread to shepherd the removal/deprecation of
> Program,
> > a
> > >> @PublicEvolving interface. Anybody has spare time? Or anything I can
> do
> > >> to make progress?
> > >>
> > >> Best,
> > >> tison.
> > >>
> > >> [1]
> > >>
> > >>
> >
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> > >>
> > >>
> > >> Zili Chen 于2019年7月22日周一 下午8:38写道：
> > >>
> > >>> Hi,
> > >>>
> > >>> I created a thread for survey in user list[1]. Please take
> participate
> > in
> > >>> if interested.
> > >>>
> > >>> Best,
> > >>> tison.
> > >>>
> > >>> [1]
> > >>>
> > >>
> >
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> > >>>
> > >>>
> > >>> Flavio Pompermaier 于2019年7月19日周五 下午5:18写道：
> > >>>
> > +1 to remove directly the Program class (I think nobody use it and
> > it's
> > not
> > supported at all by REST services and UI).
> > Moreover it requires a lot of transitive dependencies while it
> should
> > >> be a
> > very simple thing..
> > +1 to add this discussion to "Flink client api enhancement"
> >
> > On Fri, Jul 19, 2019 at 11:14 AM Biao Liu
> wrote:
> >
> > > To Flavio, good point for the integration suggestion.
> > >
> > > I think it should be considered in the "Flink client api
> enhancement"
> > > discussion. But the outdated API should be deprecated somehow.
> > >
> > > Flavio Pompermaier 于2019年7月19日周五 下午4:21写道：
> > >
> > >> In my experience a basic "official" (but optional) program
> > >> description
> > >> would be very useful indeed (in order to ease the integration with
> > other
> > >> frameworks).
> > >>
> > >> Of course it should be extended and integrated with the REST
> > >> services
> > and
> > >> the Web UI (when defined) in order to be useful..
> > >> It ease to show to the user what a job does and which parameters
> it
> > >> requires (optional or mandatory) and with a proper help
> description.
> > >> Indeed, when we write a Flink job we implement the following
> > interface:
> > >>
> > >> public interface FlinkJob {
> > >> String getDescription();
> > >> List getParameters();
> > >> boolean isStreamingOrBatch();
> > >> }
> > >>
> > >> public class ClusterJobParameter {
> > >>
> > >> private String paramName;
> > >> private String paramType = "string";
> > >> private String paramDesc;
> > >> private String paramDefaultValue;
> > >> private Set choices;
> > >> private boolean mandatory;
> > >> }
> > >>
> > >> This really helps to launch a Flink job by a frontend (if the rest
> > > services
> > >> returns back those infos).
> > >>
> > >> Best,
> > >> Flavio
> > >>
> > >> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu
> > >> wrote:
> > >>
> > >>> Hi Zili,
> > >>>
> > >>> Thank you for bring us this discussion.
>
Re: [DISCUSS] Drop stale class Program
Hi,
I would be in favour of removing Program (and the code paths that support it)
for Flink 1.10. Most users of Flink don’t actually know it exists and it is
only making our code more complicated. Going forward with the new Client API
discussions will be a lot easier without it as well.
Best,
Aljoscha
> On 14. Aug 2019, at 11:08, Kostas Kloudas wrote:
>
> Hi all,
>
> It is nice to have this discussion.
>
> I am totally up for removing the unused Program interface, as this will
> simplify a lot of other code paths in the ClusterClient and elsewhere.
>
> Also about the easier integration of Flink with other frameworks, there
> is another discussion in the mailing list with exactly this topic:
> [DISCUSS] Flink client api enhancement for downstream project
>
> Cheers,
> Kostas
>
>
> On Tue, Jul 30, 2019 at 1:38 PM Zili Chen wrote:
>
>> Hi,
>>
>> With a one-week survey in user list[1], nobody expect Flavio and Jeff
>> participant the thread.
>>
>> Flavio shared his experience with a revised Program like interface.
>> This could be regraded as downstream integration and in client api
>> enhancements document we propose rich interface for this integration.
>> Anyway, the Flink scope Program is less functional than it should be.
>>
>> With no objection I'd like to push on this thread. We need a committer
>> participant this thread to shepherd the removal/deprecation of Program, a
>> @PublicEvolving interface. Anybody has spare time? Or anything I can do
>> to make progress?
>>
>> Best,
>> tison.
>>
>> [1]
>>
>> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
>>
>>
>> Zili Chen 于2019年7月22日周一 下午8:38写道：
>>
>>> Hi,
>>>
>>> I created a thread for survey in user list[1]. Please take participate in
>>> if interested.
>>>
>>> Best,
>>> tison.
>>>
>>> [1]
>>>
>> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
>>>
>>>
>>> Flavio Pompermaier 于2019年7月19日周五 下午5:18写道：
>>>
+1 to remove directly the Program class (I think nobody use it and it's
not
supported at all by REST services and UI).
Moreover it requires a lot of transitive dependencies while it should
>> be a
very simple thing..
+1 to add this discussion to "Flink client api enhancement"
On Fri, Jul 19, 2019 at 11:14 AM Biao Liu wrote:
> To Flavio, good point for the integration suggestion.
>
> I think it should be considered in the "Flink client api enhancement"
> discussion. But the outdated API should be deprecated somehow.
>
> Flavio Pompermaier 于2019年7月19日周五 下午4:21写道：
>
>> In my experience a basic "official" (but optional) program
>> description
>> would be very useful indeed (in order to ease the integration with
other
>> frameworks).
>>
>> Of course it should be extended and integrated with the REST
>> services
and
>> the Web UI (when defined) in order to be useful..
>> It ease to show to the user what a job does and which parameters it
>> requires (optional or mandatory) and with a proper help description.
>> Indeed, when we write a Flink job we implement the following
interface:
>>
>> public interface FlinkJob {
>> String getDescription();
>> List getParameters();
>> boolean isStreamingOrBatch();
>> }
>>
>> public class ClusterJobParameter {
>>
>> private String paramName;
>> private String paramType = "string";
>> private String paramDesc;
>> private String paramDefaultValue;
>> private Set choices;
>> private boolean mandatory;
>> }
>>
>> This really helps to launch a Flink job by a frontend (if the rest
> services
>> returns back those infos).
>>
>> Best,
>> Flavio
>>
>> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu
>> wrote:
>>
>>> Hi Zili,
>>>
>>> Thank you for bring us this discussion.
>>>
>>> My gut feeling is +1 for dropping it.
>>> Usually it costs some time to deprecate a public (actually it's
>>> `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
> first.
>>> Then it might be abandoned it in some later version.
>>>
>>> I'm not sure how big the burden is to make it compatible with the
>> enhanced
>>> client API. If it's a critical blocker, I support dropping it
radically
>> in
>>> 1.10. Of course a survey is necessary. And the result of survey is
>>> acceptable.
>>>
>>>
>>>
>>> Zili Chen 于2019年7月19日周五 下午1:44写道：
>>>
Hi devs,
Participating the thread "Flink client api enhancement"[1], I
>> just
>> notice
that inside submission codepath of Flink we always has a branch
> dealing
with the case that main class of user program is a subclass of
Re: [DISCUSS] Drop stale class Program
Hi all,
It is nice to have this discussion.
I am totally up for removing the unused Program interface, as this will
simplify a lot of other code paths in the ClusterClient and elsewhere.
Also about the easier integration of Flink with other frameworks, there
is another discussion in the mailing list with exactly this topic:
[DISCUSS] Flink client api enhancement for downstream project
Cheers,
Kostas
On Tue, Jul 30, 2019 at 1:38 PM Zili Chen wrote:
> Hi,
>
> With a one-week survey in user list[1], nobody expect Flavio and Jeff
> participant the thread.
>
> Flavio shared his experience with a revised Program like interface.
> This could be regraded as downstream integration and in client api
> enhancements document we propose rich interface for this integration.
> Anyway, the Flink scope Program is less functional than it should be.
>
> With no objection I'd like to push on this thread. We need a committer
> participant this thread to shepherd the removal/deprecation of Program, a
> @PublicEvolving interface. Anybody has spare time? Or anything I can do
> to make progress?
>
> Best,
> tison.
>
> [1]
>
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
>
>
> Zili Chen 于2019年7月22日周一 下午8:38写道：
>
> > Hi,
> >
> > I created a thread for survey in user list[1]. Please take participate in
> > if interested.
> >
> > Best,
> > tison.
> >
> > [1]
> >
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
> >
> >
> > Flavio Pompermaier 于2019年7月19日周五 下午5:18写道：
> >
> >> +1 to remove directly the Program class (I think nobody use it and it's
> >> not
> >> supported at all by REST services and UI).
> >> Moreover it requires a lot of transitive dependencies while it should
> be a
> >> very simple thing..
> >> +1 to add this discussion to "Flink client api enhancement"
> >>
> >> On Fri, Jul 19, 2019 at 11:14 AM Biao Liu wrote:
> >>
> >> > To Flavio, good point for the integration suggestion.
> >> >
> >> > I think it should be considered in the "Flink client api enhancement"
> >> > discussion. But the outdated API should be deprecated somehow.
> >> >
> >> > Flavio Pompermaier 于2019年7月19日周五 下午4:21写道：
> >> >
> >> > > In my experience a basic "official" (but optional) program
> description
> >> > > would be very useful indeed (in order to ease the integration with
> >> other
> >> > > frameworks).
> >> > >
> >> > > Of course it should be extended and integrated with the REST
> services
> >> and
> >> > > the Web UI (when defined) in order to be useful..
> >> > > It ease to show to the user what a job does and which parameters it
> >> > > requires (optional or mandatory) and with a proper help description.
> >> > > Indeed, when we write a Flink job we implement the following
> >> interface:
> >> > >
> >> > > public interface FlinkJob {
> >> > > String getDescription();
> >> > > List getParameters();
> >> > > boolean isStreamingOrBatch();
> >> > > }
> >> > >
> >> > > public class ClusterJobParameter {
> >> > >
> >> > > private String paramName;
> >> > > private String paramType = "string";
> >> > > private String paramDesc;
> >> > > private String paramDefaultValue;
> >> > > private Set choices;
> >> > > private boolean mandatory;
> >> > > }
> >> > >
> >> > > This really helps to launch a Flink job by a frontend (if the rest
> >> > services
> >> > > returns back those infos).
> >> > >
> >> > > Best,
> >> > > Flavio
> >> > >
> >> > > On Fri, Jul 19, 2019 at 9:57 AM Biao Liu
> wrote:
> >> > >
> >> > > > Hi Zili,
> >> > > >
> >> > > > Thank you for bring us this discussion.
> >> > > >
> >> > > > My gut feeling is +1 for dropping it.
> >> > > > Usually it costs some time to deprecate a public (actually it's
> >> > > > `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
> >> > first.
> >> > > > Then it might be abandoned it in some later version.
> >> > > >
> >> > > > I'm not sure how big the burden is to make it compatible with the
> >> > > enhanced
> >> > > > client API. If it's a critical blocker, I support dropping it
> >> radically
> >> > > in
> >> > > > 1.10. Of course a survey is necessary. And the result of survey is
> >> > > > acceptable.
> >> > > >
> >> > > >
> >> > > >
> >> > > > Zili Chen 于2019年7月19日周五 下午1:44写道：
> >> > > >
> >> > > > > Hi devs,
> >> > > > >
> >> > > > > Participating the thread "Flink client api enhancement"[1], I
> just
> >> > > notice
> >> > > > > that inside submission codepath of Flink we always has a branch
> >> > dealing
> >> > > > > with the case that main class of user program is a subclass of
> >> > > > > o.a.f.api.common.Program, which is defined as
> >> > > > >
> >> > > > > @PublicEvolving
> >> > > > > public interface Program {
> >> > > > > Plan getPhan(String... args);
> >> > > > > }
> >> > > > >
> >> > > > > This class, as user-facing interface, asks user to implement
> >> #getPlan
> >> > > > > which return a
Re: [DISCUSS] Drop stale class Program
Hi,
With a one-week survey in user list[1], nobody expect Flavio and Jeff
participant the thread.
Flavio shared his experience with a revised Program like interface.
This could be regraded as downstream integration and in client api
enhancements document we propose rich interface for this integration.
Anyway, the Flink scope Program is less functional than it should be.
With no objection I'd like to push on this thread. We need a committer
participant this thread to shepherd the removal/deprecation of Program, a
@PublicEvolving interface. Anybody has spare time? Or anything I can do
to make progress?
Best,
tison.
[1]
https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
Zili Chen 于2019年7月22日周一 下午8:38写道：
> Hi,
>
> I created a thread for survey in user list[1]. Please take participate in
> if interested.
>
> Best,
> tison.
>
> [1]
> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
>
>
> Flavio Pompermaier 于2019年7月19日周五 下午5:18写道：
>
>> +1 to remove directly the Program class (I think nobody use it and it's
>> not
>> supported at all by REST services and UI).
>> Moreover it requires a lot of transitive dependencies while it should be a
>> very simple thing..
>> +1 to add this discussion to "Flink client api enhancement"
>>
>> On Fri, Jul 19, 2019 at 11:14 AM Biao Liu wrote:
>>
>> > To Flavio, good point for the integration suggestion.
>> >
>> > I think it should be considered in the "Flink client api enhancement"
>> > discussion. But the outdated API should be deprecated somehow.
>> >
>> > Flavio Pompermaier 于2019年7月19日周五 下午4:21写道：
>> >
>> > > In my experience a basic "official" (but optional) program description
>> > > would be very useful indeed (in order to ease the integration with
>> other
>> > > frameworks).
>> > >
>> > > Of course it should be extended and integrated with the REST services
>> and
>> > > the Web UI (when defined) in order to be useful..
>> > > It ease to show to the user what a job does and which parameters it
>> > > requires (optional or mandatory) and with a proper help description.
>> > > Indeed, when we write a Flink job we implement the following
>> interface:
>> > >
>> > > public interface FlinkJob {
>> > > String getDescription();
>> > > List getParameters();
>> > > boolean isStreamingOrBatch();
>> > > }
>> > >
>> > > public class ClusterJobParameter {
>> > >
>> > > private String paramName;
>> > > private String paramType = "string";
>> > > private String paramDesc;
>> > > private String paramDefaultValue;
>> > > private Set choices;
>> > > private boolean mandatory;
>> > > }
>> > >
>> > > This really helps to launch a Flink job by a frontend (if the rest
>> > services
>> > > returns back those infos).
>> > >
>> > > Best,
>> > > Flavio
>> > >
>> > > On Fri, Jul 19, 2019 at 9:57 AM Biao Liu wrote:
>> > >
>> > > > Hi Zili,
>> > > >
>> > > > Thank you for bring us this discussion.
>> > > >
>> > > > My gut feeling is +1 for dropping it.
>> > > > Usually it costs some time to deprecate a public (actually it's
>> > > > `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
>> > first.
>> > > > Then it might be abandoned it in some later version.
>> > > >
>> > > > I'm not sure how big the burden is to make it compatible with the
>> > > enhanced
>> > > > client API. If it's a critical blocker, I support dropping it
>> radically
>> > > in
>> > > > 1.10. Of course a survey is necessary. And the result of survey is
>> > > > acceptable.
>> > > >
>> > > >
>> > > >
>> > > > Zili Chen 于2019年7月19日周五 下午1:44写道：
>> > > >
>> > > > > Hi devs,
>> > > > >
>> > > > > Participating the thread "Flink client api enhancement"[1], I just
>> > > notice
>> > > > > that inside submission codepath of Flink we always has a branch
>> > dealing
>> > > > > with the case that main class of user program is a subclass of
>> > > > > o.a.f.api.common.Program, which is defined as
>> > > > >
>> > > > > @PublicEvolving
>> > > > > public interface Program {
>> > > > > Plan getPhan(String... args);
>> > > > > }
>> > > > >
>> > > > > This class, as user-facing interface, asks user to implement
>> #getPlan
>> > > > > which return a almost Flink internal class. FLINK-10862[2] pointed
>> > out
>> > > > > this confusion.
>> > > > >
>> > > > > Since our codebase contains quite a bit code handling this stale
>> > class,
>> > > > > and also it obstructs the effort enhancing Flink cilent api,
>> > > > > I'd like to propose dropping it. Or we can start a survey on user
>> > list
>> > > > > to see if there is any user depending on this class.
>> > > > >
>> > > > > best,
>> > > > > tison.
>> > > > >
>> > > > > [1]
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
>> > > > > [2] https://issues.apache.org/jira/browse/FLINK-10862
>> > > > >
>>
Re: [DISCUSS] Drop stale class Program
Hi,
I created a thread for survey in user list[1]. Please take participate in
if interested.
Best,
tison.
[1]
https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E
Flavio Pompermaier 于2019年7月19日周五 下午5:18写道：
> +1 to remove directly the Program class (I think nobody use it and it's not
> supported at all by REST services and UI).
> Moreover it requires a lot of transitive dependencies while it should be a
> very simple thing..
> +1 to add this discussion to "Flink client api enhancement"
>
> On Fri, Jul 19, 2019 at 11:14 AM Biao Liu wrote:
>
> > To Flavio, good point for the integration suggestion.
> >
> > I think it should be considered in the "Flink client api enhancement"
> > discussion. But the outdated API should be deprecated somehow.
> >
> > Flavio Pompermaier 于2019年7月19日周五 下午4:21写道：
> >
> > > In my experience a basic "official" (but optional) program description
> > > would be very useful indeed (in order to ease the integration with
> other
> > > frameworks).
> > >
> > > Of course it should be extended and integrated with the REST services
> and
> > > the Web UI (when defined) in order to be useful..
> > > It ease to show to the user what a job does and which parameters it
> > > requires (optional or mandatory) and with a proper help description.
> > > Indeed, when we write a Flink job we implement the following interface:
> > >
> > > public interface FlinkJob {
> > > String getDescription();
> > > List getParameters();
> > > boolean isStreamingOrBatch();
> > > }
> > >
> > > public class ClusterJobParameter {
> > >
> > > private String paramName;
> > > private String paramType = "string";
> > > private String paramDesc;
> > > private String paramDefaultValue;
> > > private Set choices;
> > > private boolean mandatory;
> > > }
> > >
> > > This really helps to launch a Flink job by a frontend (if the rest
> > services
> > > returns back those infos).
> > >
> > > Best,
> > > Flavio
> > >
> > > On Fri, Jul 19, 2019 at 9:57 AM Biao Liu wrote:
> > >
> > > > Hi Zili,
> > > >
> > > > Thank you for bring us this discussion.
> > > >
> > > > My gut feeling is +1 for dropping it.
> > > > Usually it costs some time to deprecate a public (actually it's
> > > > `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
> > first.
> > > > Then it might be abandoned it in some later version.
> > > >
> > > > I'm not sure how big the burden is to make it compatible with the
> > > enhanced
> > > > client API. If it's a critical blocker, I support dropping it
> radically
> > > in
> > > > 1.10. Of course a survey is necessary. And the result of survey is
> > > > acceptable.
> > > >
> > > >
> > > >
> > > > Zili Chen 于2019年7月19日周五 下午1:44写道：
> > > >
> > > > > Hi devs,
> > > > >
> > > > > Participating the thread "Flink client api enhancement"[1], I just
> > > notice
> > > > > that inside submission codepath of Flink we always has a branch
> > dealing
> > > > > with the case that main class of user program is a subclass of
> > > > > o.a.f.api.common.Program, which is defined as
> > > > >
> > > > > @PublicEvolving
> > > > > public interface Program {
> > > > > Plan getPhan(String... args);
> > > > > }
> > > > >
> > > > > This class, as user-facing interface, asks user to implement
> #getPlan
> > > > > which return a almost Flink internal class. FLINK-10862[2] pointed
> > out
> > > > > this confusion.
> > > > >
> > > > > Since our codebase contains quite a bit code handling this stale
> > class,
> > > > > and also it obstructs the effort enhancing Flink cilent api,
> > > > > I'd like to propose dropping it. Or we can start a survey on user
> > list
> > > > > to see if there is any user depending on this class.
> > > > >
> > > > > best,
> > > > > tison.
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > > > > [2] https://issues.apache.org/jira/browse/FLINK-10862
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Drop stale class Program
+1 to remove directly the Program class (I think nobody use it and it's not
supported at all by REST services and UI).
Moreover it requires a lot of transitive dependencies while it should be a
very simple thing..
+1 to add this discussion to "Flink client api enhancement"
On Fri, Jul 19, 2019 at 11:14 AM Biao Liu wrote:
> To Flavio, good point for the integration suggestion.
>
> I think it should be considered in the "Flink client api enhancement"
> discussion. But the outdated API should be deprecated somehow.
>
> Flavio Pompermaier 于2019年7月19日周五 下午4:21写道：
>
> > In my experience a basic "official" (but optional) program description
> > would be very useful indeed (in order to ease the integration with other
> > frameworks).
> >
> > Of course it should be extended and integrated with the REST services and
> > the Web UI (when defined) in order to be useful..
> > It ease to show to the user what a job does and which parameters it
> > requires (optional or mandatory) and with a proper help description.
> > Indeed, when we write a Flink job we implement the following interface:
> >
> > public interface FlinkJob {
> > String getDescription();
> > List getParameters();
> > boolean isStreamingOrBatch();
> > }
> >
> > public class ClusterJobParameter {
> >
> > private String paramName;
> > private String paramType = "string";
> > private String paramDesc;
> > private String paramDefaultValue;
> > private Set choices;
> > private boolean mandatory;
> > }
> >
> > This really helps to launch a Flink job by a frontend (if the rest
> services
> > returns back those infos).
> >
> > Best,
> > Flavio
> >
> > On Fri, Jul 19, 2019 at 9:57 AM Biao Liu wrote:
> >
> > > Hi Zili,
> > >
> > > Thank you for bring us this discussion.
> > >
> > > My gut feeling is +1 for dropping it.
> > > Usually it costs some time to deprecate a public (actually it's
> > > `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
> first.
> > > Then it might be abandoned it in some later version.
> > >
> > > I'm not sure how big the burden is to make it compatible with the
> > enhanced
> > > client API. If it's a critical blocker, I support dropping it radically
> > in
> > > 1.10. Of course a survey is necessary. And the result of survey is
> > > acceptable.
> > >
> > >
> > >
> > > Zili Chen 于2019年7月19日周五 下午1:44写道：
> > >
> > > > Hi devs,
> > > >
> > > > Participating the thread "Flink client api enhancement"[1], I just
> > notice
> > > > that inside submission codepath of Flink we always has a branch
> dealing
> > > > with the case that main class of user program is a subclass of
> > > > o.a.f.api.common.Program, which is defined as
> > > >
> > > > @PublicEvolving
> > > > public interface Program {
> > > > Plan getPhan(String... args);
> > > > }
> > > >
> > > > This class, as user-facing interface, asks user to implement #getPlan
> > > > which return a almost Flink internal class. FLINK-10862[2] pointed
> out
> > > > this confusion.
> > > >
> > > > Since our codebase contains quite a bit code handling this stale
> class,
> > > > and also it obstructs the effort enhancing Flink cilent api,
> > > > I'd like to propose dropping it. Or we can start a survey on user
> list
> > > > to see if there is any user depending on this class.
> > > >
> > > > best,
> > > > tison.
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > > > [2] https://issues.apache.org/jira/browse/FLINK-10862
> > > >
> > >
> >
>
Re: [DISCUSS] Drop stale class Program
Hi Flavio,
Thanks for your reply and share.
I agree with that an "official" program description
would be helpful as you described.
However, this thread mainly focuses on drop the stale
class Program.
For proposing a better program description from Flink
side, feel free to start a new thread on dev or user
list(I would regard this as a user requirement).
Best,
tison.
Zili Chen 于2019年7月19日周五 下午5:10写道：
> Hi Biao,
>
> Thanks for your reply.
>
> For the "burden" part, inside PackagedProgram and ClusterClient
> we currently contains branches handling whether the mainClass of
> user job jar is subclass of Program. Any effort under client api
> enhancement should be compatible with such codepaths unless we
> drop it.
>
> The class Program is only active in batch codepath, while the
> so-called interactive mode, i.e., executing the main method of
> user's main class, is using widely and valid in both batch and
> streaming codepath.
>
> Previously we have drop flink-storm directly and I'm afraid this
> Program class is much less known and used than flink-storm.
> Keeping compatible with such class is an unnecessary overhead.
>
> However, the enhancement of client api is still under discussion
> so it would be ok to deprecate at first and propose removal when
> necessary.
>
> Either drop it or deprecate it prevents users from being confused
> how to use this stale class. But given the community has dropped
> stale flink-storm, flink-libarary/python|ml, I'd like to propose
> drop this class if in the survey we receiving no usage report.
>
> Best,
> tison.
>
>
> Flavio Pompermaier 于2019年7月19日周五 下午4:21写道：
>
>> In my experience a basic "official" (but optional) program description
>> would be very useful indeed (in order to ease the integration with other
>> frameworks).
>>
>> Of course it should be extended and integrated with the REST services and
>> the Web UI (when defined) in order to be useful..
>> It ease to show to the user what a job does and which parameters it
>> requires (optional or mandatory) and with a proper help description.
>> Indeed, when we write a Flink job we implement the following interface:
>>
>> public interface FlinkJob {
>> String getDescription();
>> List getParameters();
>> boolean isStreamingOrBatch();
>> }
>>
>> public class ClusterJobParameter {
>>
>> private String paramName;
>> private String paramType = "string";
>> private String paramDesc;
>> private String paramDefaultValue;
>> private Set choices;
>> private boolean mandatory;
>> }
>>
>> This really helps to launch a Flink job by a frontend (if the rest
>> services
>> returns back those infos).
>>
>> Best,
>> Flavio
>>
>> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu wrote:
>>
>> > Hi Zili,
>> >
>> > Thank you for bring us this discussion.
>> >
>> > My gut feeling is +1 for dropping it.
>> > Usually it costs some time to deprecate a public (actually it's
>> > `PublicEvolving`) API. Ideally it should be marked as `Deprecated`
>> first.
>> > Then it might be abandoned it in some later version.
>> >
>> > I'm not sure how big the burden is to make it compatible with the
>> enhanced
>> > client API. If it's a critical blocker, I support dropping it radically
>> in
>> > 1.10. Of course a survey is necessary. And the result of survey is
>> > acceptable.
>> >
>> >
>> >
>> > Zili Chen 于2019年7月19日周五 下午1:44写道：
>> >
>> > > Hi devs,
>> > >
>> > > Participating the thread "Flink client api enhancement"[1], I just
>> notice
>> > > that inside submission codepath of Flink we always has a branch
>> dealing
>> > > with the case that main class of user program is a subclass of
>> > > o.a.f.api.common.Program, which is defined as
>> > >
>> > > @PublicEvolving
>> > > public interface Program {
>> > > Plan getPhan(String... args);
>> > > }
>> > >
>> > > This class, as user-facing interface, asks user to implement #getPlan
>> > > which return a almost Flink internal class. FLINK-10862[2] pointed out
>> > > this confusion.
>> > >
>> > > Since our codebase contains quite a bit code handling this stale
>> class,
>> > > and also it obstructs the effort enhancing Flink cilent api,
>> > > I'd like to propose dropping it. Or we can start a survey on user list
>> > > to see if there is any user depending on this class.
>> > >
>> > > best,
>> > > tison.
>> > >
>> > > [1]
>> > >
>> > >
>> >
>> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
>> > > [2] https://issues.apache.org/jira/browse/FLINK-10862
>> > >
>> >
>>
>
Re: [DISCUSS] Drop stale class Program
To Flavio, good point for the integration suggestion.
I think it should be considered in the "Flink client api enhancement"
discussion. But the outdated API should be deprecated somehow.
Flavio Pompermaier 于2019年7月19日周五 下午4:21写道：
> In my experience a basic "official" (but optional) program description
> would be very useful indeed (in order to ease the integration with other
> frameworks).
>
> Of course it should be extended and integrated with the REST services and
> the Web UI (when defined) in order to be useful..
> It ease to show to the user what a job does and which parameters it
> requires (optional or mandatory) and with a proper help description.
> Indeed, when we write a Flink job we implement the following interface:
>
> public interface FlinkJob {
> String getDescription();
> List getParameters();
> boolean isStreamingOrBatch();
> }
>
> public class ClusterJobParameter {
>
> private String paramName;
> private String paramType = "string";
> private String paramDesc;
> private String paramDefaultValue;
> private Set choices;
> private boolean mandatory;
> }
>
> This really helps to launch a Flink job by a frontend (if the rest services
> returns back those infos).
>
> Best,
> Flavio
>
> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu wrote:
>
> > Hi Zili,
> >
> > Thank you for bring us this discussion.
> >
> > My gut feeling is +1 for dropping it.
> > Usually it costs some time to deprecate a public (actually it's
> > `PublicEvolving`) API. Ideally it should be marked as `Deprecated` first.
> > Then it might be abandoned it in some later version.
> >
> > I'm not sure how big the burden is to make it compatible with the
> enhanced
> > client API. If it's a critical blocker, I support dropping it radically
> in
> > 1.10. Of course a survey is necessary. And the result of survey is
> > acceptable.
> >
> >
> >
> > Zili Chen 于2019年7月19日周五 下午1:44写道：
> >
> > > Hi devs,
> > >
> > > Participating the thread "Flink client api enhancement"[1], I just
> notice
> > > that inside submission codepath of Flink we always has a branch dealing
> > > with the case that main class of user program is a subclass of
> > > o.a.f.api.common.Program, which is defined as
> > >
> > > @PublicEvolving
> > > public interface Program {
> > > Plan getPhan(String... args);
> > > }
> > >
> > > This class, as user-facing interface, asks user to implement #getPlan
> > > which return a almost Flink internal class. FLINK-10862[2] pointed out
> > > this confusion.
> > >
> > > Since our codebase contains quite a bit code handling this stale class,
> > > and also it obstructs the effort enhancing Flink cilent api,
> > > I'd like to propose dropping it. Or we can start a survey on user list
> > > to see if there is any user depending on this class.
> > >
> > > best,
> > > tison.
> > >
> > > [1]
> > >
> > >
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > > [2] https://issues.apache.org/jira/browse/FLINK-10862
> > >
> >
>
Re: [DISCUSS] Drop stale class Program
Hi Biao,
Thanks for your reply.
For the "burden" part, inside PackagedProgram and ClusterClient
we currently contains branches handling whether the mainClass of
user job jar is subclass of Program. Any effort under client api
enhancement should be compatible with such codepaths unless we
drop it.
The class Program is only active in batch codepath, while the
so-called interactive mode, i.e., executing the main method of
user's main class, is using widely and valid in both batch and
streaming codepath.
Previously we have drop flink-storm directly and I'm afraid this
Program class is much less known and used than flink-storm.
Keeping compatible with such class is an unnecessary overhead.
However, the enhancement of client api is still under discussion
so it would be ok to deprecate at first and propose removal when
necessary.
Either drop it or deprecate it prevents users from being confused
how to use this stale class. But given the community has dropped
stale flink-storm, flink-libarary/python|ml, I'd like to propose
drop this class if in the survey we receiving no usage report.
Best,
tison.
Flavio Pompermaier 于2019年7月19日周五 下午4:21写道：
> In my experience a basic "official" (but optional) program description
> would be very useful indeed (in order to ease the integration with other
> frameworks).
>
> Of course it should be extended and integrated with the REST services and
> the Web UI (when defined) in order to be useful..
> It ease to show to the user what a job does and which parameters it
> requires (optional or mandatory) and with a proper help description.
> Indeed, when we write a Flink job we implement the following interface:
>
> public interface FlinkJob {
> String getDescription();
> List getParameters();
> boolean isStreamingOrBatch();
> }
>
> public class ClusterJobParameter {
>
> private String paramName;
> private String paramType = "string";
> private String paramDesc;
> private String paramDefaultValue;
> private Set choices;
> private boolean mandatory;
> }
>
> This really helps to launch a Flink job by a frontend (if the rest services
> returns back those infos).
>
> Best,
> Flavio
>
> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu wrote:
>
> > Hi Zili,
> >
> > Thank you for bring us this discussion.
> >
> > My gut feeling is +1 for dropping it.
> > Usually it costs some time to deprecate a public (actually it's
> > `PublicEvolving`) API. Ideally it should be marked as `Deprecated` first.
> > Then it might be abandoned it in some later version.
> >
> > I'm not sure how big the burden is to make it compatible with the
> enhanced
> > client API. If it's a critical blocker, I support dropping it radically
> in
> > 1.10. Of course a survey is necessary. And the result of survey is
> > acceptable.
> >
> >
> >
> > Zili Chen 于2019年7月19日周五 下午1:44写道：
> >
> > > Hi devs,
> > >
> > > Participating the thread "Flink client api enhancement"[1], I just
> notice
> > > that inside submission codepath of Flink we always has a branch dealing
> > > with the case that main class of user program is a subclass of
> > > o.a.f.api.common.Program, which is defined as
> > >
> > > @PublicEvolving
> > > public interface Program {
> > > Plan getPhan(String... args);
> > > }
> > >
> > > This class, as user-facing interface, asks user to implement #getPlan
> > > which return a almost Flink internal class. FLINK-10862[2] pointed out
> > > this confusion.
> > >
> > > Since our codebase contains quite a bit code handling this stale class,
> > > and also it obstructs the effort enhancing Flink cilent api,
> > > I'd like to propose dropping it. Or we can start a survey on user list
> > > to see if there is any user depending on this class.
> > >
> > > best,
> > > tison.
> > >
> > > [1]
> > >
> > >
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > > [2] https://issues.apache.org/jira/browse/FLINK-10862
> > >
> >
>
Re: [DISCUSS] Drop stale class Program
In my experience a basic "official" (but optional) program description
would be very useful indeed (in order to ease the integration with other
frameworks).
Of course it should be extended and integrated with the REST services and
the Web UI (when defined) in order to be useful..
It ease to show to the user what a job does and which parameters it
requires (optional or mandatory) and with a proper help description.
Indeed, when we write a Flink job we implement the following interface:
public interface FlinkJob {
String getDescription();
List getParameters();
boolean isStreamingOrBatch();
}
public class ClusterJobParameter {
private String paramName;
private String paramType = "string";
private String paramDesc;
private String paramDefaultValue;
private Set choices;
private boolean mandatory;
}
This really helps to launch a Flink job by a frontend (if the rest services
returns back those infos).
Best,
Flavio
On Fri, Jul 19, 2019 at 9:57 AM Biao Liu wrote:
> Hi Zili,
>
> Thank you for bring us this discussion.
>
> My gut feeling is +1 for dropping it.
> Usually it costs some time to deprecate a public (actually it's
> `PublicEvolving`) API. Ideally it should be marked as `Deprecated` first.
> Then it might be abandoned it in some later version.
>
> I'm not sure how big the burden is to make it compatible with the enhanced
> client API. If it's a critical blocker, I support dropping it radically in
> 1.10. Of course a survey is necessary. And the result of survey is
> acceptable.
>
>
>
> Zili Chen 于2019年7月19日周五 下午1:44写道：
>
> > Hi devs,
> >
> > Participating the thread "Flink client api enhancement"[1], I just notice
> > that inside submission codepath of Flink we always has a branch dealing
> > with the case that main class of user program is a subclass of
> > o.a.f.api.common.Program, which is defined as
> >
> > @PublicEvolving
> > public interface Program {
> > Plan getPhan(String... args);
> > }
> >
> > This class, as user-facing interface, asks user to implement #getPlan
> > which return a almost Flink internal class. FLINK-10862[2] pointed out
> > this confusion.
> >
> > Since our codebase contains quite a bit code handling this stale class,
> > and also it obstructs the effort enhancing Flink cilent api,
> > I'd like to propose dropping it. Or we can start a survey on user list
> > to see if there is any user depending on this class.
> >
> > best,
> > tison.
> >
> > [1]
> >
> >
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> > [2] https://issues.apache.org/jira/browse/FLINK-10862
> >
>
Re: [DISCUSS] Drop stale class Program
Hi Zili,
Thank you for bring us this discussion.
My gut feeling is +1 for dropping it.
Usually it costs some time to deprecate a public (actually it's
`PublicEvolving`) API. Ideally it should be marked as `Deprecated` first.
Then it might be abandoned it in some later version.
I'm not sure how big the burden is to make it compatible with the enhanced
client API. If it's a critical blocker, I support dropping it radically in
1.10. Of course a survey is necessary. And the result of survey is
acceptable.
Zili Chen 于2019年7月19日周五 下午1:44写道：
> Hi devs,
>
> Participating the thread "Flink client api enhancement"[1], I just notice
> that inside submission codepath of Flink we always has a branch dealing
> with the case that main class of user program is a subclass of
> o.a.f.api.common.Program, which is defined as
>
> @PublicEvolving
> public interface Program {
> Plan getPhan(String... args);
> }
>
> This class, as user-facing interface, asks user to implement #getPlan
> which return a almost Flink internal class. FLINK-10862[2] pointed out
> this confusion.
>
> Since our codebase contains quite a bit code handling this stale class,
> and also it obstructs the effort enhancing Flink cilent api,
> I'd like to propose dropping it. Or we can start a survey on user list
> to see if there is any user depending on this class.
>
> best,
> tison.
>
> [1]
>
> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E
> [2] https://issues.apache.org/jira/browse/FLINK-10862
>
