Hi, I would be in favour of removing Program (and the code paths that support it) for Flink 1.10. Most users of Flink don’t actually know it exists and it is only making our code more complicated. Going forward with the new Client API discussions will be a lot easier without it as well.
Best, Aljoscha > On 14. Aug 2019, at 11:08, Kostas Kloudas <kklou...@gmail.com> wrote: > > Hi all, > > It is nice to have this discussion. > > I am totally up for removing the unused Program interface, as this will > simplify a lot of other code paths in the ClusterClient and elsewhere. > > Also about the easier integration of Flink with other frameworks, there > is another discussion in the mailing list with exactly this topic: > [DISCUSS] Flink client api enhancement for downstream project > > Cheers, > Kostas > > > On Tue, Jul 30, 2019 at 1:38 PM Zili Chen <wander4...@gmail.com> wrote: > >> Hi, >> >> With a one-week survey in user list[1], nobody expect Flavio and Jeff >> participant the thread. >> >> Flavio shared his experience with a revised Program like interface. >> This could be regraded as downstream integration and in client api >> enhancements document we propose rich interface for this integration. >> Anyway, the Flink scope Program is less functional than it should be. >> >> With no objection I'd like to push on this thread. We need a committer >> participant this thread to shepherd the removal/deprecation of Program, a >> @PublicEvolving interface. Anybody has spare time? Or anything I can do >> to make progress? >> >> Best, >> tison. >> >> [1] >> >> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E >> >> >> Zili Chen <wander4...@gmail.com> 于2019年7月22日周一 下午8:38写道： >> >>> Hi, >>> >>> I created a thread for survey in user list[1]. Please take participate in >>> if interested. >>> >>> Best, >>> tison. >>> >>> [1] >>> >> https://lists.apache.org/thread.html/37445e43729cf7eaeb0aa09133d3980b62f891c5ee69d2c3c3e76ab5@%3Cuser.flink.apache.org%3E >>> >>> >>> Flavio Pompermaier <pomperma...@okkam.it> 于2019年7月19日周五 下午5:18写道： >>> >>>> +1 to remove directly the Program class (I think nobody use it and it's >>>> not >>>> supported at all by REST services and UI). >>>> Moreover it requires a lot of transitive dependencies while it should >> be a >>>> very simple thing.. >>>> +1 to add this discussion to "Flink client api enhancement" >>>> >>>> On Fri, Jul 19, 2019 at 11:14 AM Biao Liu <mmyy1...@gmail.com> wrote: >>>> >>>>> To Flavio, good point for the integration suggestion. >>>>> >>>>> I think it should be considered in the "Flink client api enhancement" >>>>> discussion. But the outdated API should be deprecated somehow. >>>>> >>>>> Flavio Pompermaier <pomperma...@okkam.it> 于2019年7月19日周五 下午4:21写道： >>>>> >>>>>> In my experience a basic "official" (but optional) program >> description >>>>>> would be very useful indeed (in order to ease the integration with >>>> other >>>>>> frameworks). >>>>>> >>>>>> Of course it should be extended and integrated with the REST >> services >>>> and >>>>>> the Web UI (when defined) in order to be useful.. >>>>>> It ease to show to the user what a job does and which parameters it >>>>>> requires (optional or mandatory) and with a proper help description. >>>>>> Indeed, when we write a Flink job we implement the following >>>> interface: >>>>>> >>>>>> public interface FlinkJob { >>>>>> String getDescription(); >>>>>> List<FlinkJobParameter> getParameters(); >>>>>> boolean isStreamingOrBatch(); >>>>>> } >>>>>> >>>>>> public class ClusterJobParameter { >>>>>> >>>>>> private String paramName; >>>>>> private String paramType = "string"; >>>>>> private String paramDesc; >>>>>> private String paramDefaultValue; >>>>>> private Set<String> choices; >>>>>> private boolean mandatory; >>>>>> } >>>>>> >>>>>> This really helps to launch a Flink job by a frontend (if the rest >>>>> services >>>>>> returns back those infos). >>>>>> >>>>>> Best, >>>>>> Flavio >>>>>> >>>>>> On Fri, Jul 19, 2019 at 9:57 AM Biao Liu <mmyy1...@gmail.com> >> wrote: >>>>>> >>>>>>> Hi Zili, >>>>>>> >>>>>>> Thank you for bring us this discussion. >>>>>>> >>>>>>> My gut feeling is +1 for dropping it. >>>>>>> Usually it costs some time to deprecate a public (actually it's >>>>>>> `PublicEvolving`) API. Ideally it should be marked as `Deprecated` >>>>> first. >>>>>>> Then it might be abandoned it in some later version. >>>>>>> >>>>>>> I'm not sure how big the burden is to make it compatible with the >>>>>> enhanced >>>>>>> client API. If it's a critical blocker, I support dropping it >>>> radically >>>>>> in >>>>>>> 1.10. Of course a survey is necessary. And the result of survey is >>>>>>> acceptable. >>>>>>> >>>>>>> >>>>>>> >>>>>>> Zili Chen <wander4...@gmail.com> 于2019年7月19日周五 下午1:44写道： >>>>>>> >>>>>>>> Hi devs, >>>>>>>> >>>>>>>> Participating the thread "Flink client api enhancement"[1], I >> just >>>>>> notice >>>>>>>> that inside submission codepath of Flink we always has a branch >>>>> dealing >>>>>>>> with the case that main class of user program is a subclass of >>>>>>>> o.a.f.api.common.Program, which is defined as >>>>>>>> >>>>>>>> @PublicEvolving >>>>>>>> public interface Program { >>>>>>>> Plan getPhan(String... args); >>>>>>>> } >>>>>>>> >>>>>>>> This class, as user-facing interface, asks user to implement >>>> #getPlan >>>>>>>> which return a almost Flink internal class. FLINK-10862[2] >> pointed >>>>> out >>>>>>>> this confusion. >>>>>>>> >>>>>>>> Since our codebase contains quite a bit code handling this stale >>>>> class, >>>>>>>> and also it obstructs the effort enhancing Flink cilent api, >>>>>>>> I'd like to propose dropping it. Or we can start a survey on >> user >>>>> list >>>>>>>> to see if there is any user depending on this class. >>>>>>>> >>>>>>>> best, >>>>>>>> tison. >>>>>>>> >>>>>>>> [1] >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E >>>>>>>> [2] https://issues.apache.org/jira/browse/FLINK-10862 >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>
