Thanks for bringing up the discussion, Hequn. +1 on adding `flink-ml-api` and `flink-ml-lib` into opt. This would make it much easier for the users to try out some simple ml tasks.
Thanks, Jiangjie (Becket) Qin On Mon, Feb 3, 2020 at 4:34 PM jincheng sun <sunjincheng...@gmail.com> wrote: > Thank you for pushing forward @Hequn Cheng <he...@apache.org> ! > > Hi @Becket Qin <becket....@gmail.com> , Do you have any concerns on this > ? > > Best, > Jincheng > > Hequn Cheng <he...@apache.org> 于2020年2月3日周一 下午2:09写道： > >> Hi everyone, >> >> Thanks for the feedback. As there are no objections, I've opened a JIRA >> issue(FLINK-15847[1]) to address this issue. >> The implementation details can be discussed in the issue or in the >> following PR. >> >> Best, >> Hequn >> >> [1] https://issues.apache.org/jira/browse/FLINK-15847 >> >> On Wed, Jan 8, 2020 at 9:15 PM Hequn Cheng <chenghe...@gmail.com> wrote: >> >> > Hi Jincheng, >> > >> > Thanks a lot for your feedback! >> > Yes, I agree with you. There are cases that multi jars need to be >> > uploaded. I will prepare another discussion later. Maybe with a simple >> > design doc. >> > >> > Best, Hequn >> > >> > On Wed, Jan 8, 2020 at 3:06 PM jincheng sun <sunjincheng...@gmail.com> >> > wrote: >> > >> >> Thanks for bring up this discussion Hequn! >> >> >> >> +1 for include `flink-ml-api` and `flink-ml-lib` in opt. >> >> >> >> BTW: I think would be great if bring up a discussion for upload >> multiple >> >> Jars at the same time. as PyFlink JOB also can have the benefit if we >> do >> >> that improvement. >> >> >> >> Best, >> >> Jincheng >> >> >> >> >> >> Hequn Cheng <chenghe...@gmail.com> 于2020年1月8日周三 上午11:50写道： >> >> >> >> > Hi everyone, >> >> > >> >> > FLIP-39[1] rebuilds Flink ML pipeline on top of TableAPI which moves >> >> Flink >> >> > ML a step further. Base on it, users can develop their ML jobs and >> more >> >> and >> >> > more machine learning platforms are providing ML services. >> >> > >> >> > However, the problem now is the jars of flink-ml-api and flink-ml-lib >> >> are >> >> > only exist on maven repo. Whenever users want to submit ML jobs, they >> >> can >> >> > only depend on the ml modules and package a fat jar. This would be >> >> > inconvenient especially for the machine learning platforms on which >> >> nearly >> >> > all jobs depend on Flink ML modules and have to package a fat jar. >> >> > >> >> > Given this, it would be better to include jars of flink-ml-api and >> >> > flink-ml-lib in the `opt` folder, so that users can directly use the >> >> jars >> >> > with the binary release. For example, users can move the jars into >> the >> >> > `lib` folder or use -j to upload the jars. (Currently, -j only >> support >> >> > upload one jar. Supporting multi jars for -j can be discussed in >> another >> >> > discussion.) >> >> > >> >> > Putting the jars in the `opt` folder instead of the `lib` folder is >> >> because >> >> > currently, the ml jars are still optional for the Flink project by >> >> default. >> >> > >> >> > What do you think? Welcome any feedback! >> >> > >> >> > Best, >> >> > >> >> > Hequn >> >> > >> >> > [1] >> >> > >> >> > >> >> >> https://cwiki.apache.org/confluence/display/FLINK/FLIP-39+Flink+ML+pipeline+and+ML+libs >> >> > >> >> >> > >> >
