Re: [DISCUSS] Drop Hadoop 1 support with Flink 1.2

Thanks alot for the feedback on the issue. It seems that everybody agrees to drop Hadoop 1 support in Flink. I don't think we need to vote on the issue. I've filed a JIRA for the task: https://issues.apache.org/jira/browse/FLINK-4895 Maybe I'll find time to work on it next week. On Fri, Oct 14,

Re: [DISCUSS] Drop Hadoop 1 support with Flink 1.2

Thanks for the pointer. I'll start a separate discussion and push the PR forward if we come to an agreement. 2016-10-14 11:04 GMT+02:00 Stephan Ewen : > @Fabian - Someone started with that in > https://issues.apache.org/jira/browse/FLINK-4315 > That could be changed to not

Re: [DISCUSS] Drop Hadoop 1 support with Flink 1.2

@Fabian - Someone started with that in https://issues.apache.org/jira/browse/FLINK-4315 That could be changed to not remove the methods from the ExecutionEnvironment. On Fri, Oct 14, 2016 at 10:45 AM, Fabian Hueske wrote: > Yes, I'm also +1 for removing the methods at some

Re: [DISCUSS] Drop Hadoop 1 support with Flink 1.2

Yes, I'm also +1 for removing the methods at some point. For 1.2 we could go ahead and move the Hadoop-MR connectors into a separate module and mark the methods in ExecutionEnvironment as @deprecated. In 1.3 (or 2.0 whatever comes next) we could make the switch. 2016-10-14 10:40 GMT+02:00

Re: [DISCUSS] Drop Hadoop 1 support with Flink 1.2

@Fabian Good point. For Flink 2.0, I would suggest to remove them from the Environment and add them to a Utility. The way it is now, it ties Flink very strongly to Hadoop. You are right, before we do that, there is no way to make a Hadoop independent distribution. On Fri, Oct 14, 2016 at 10:37

Re: [DISCUSS] Drop Hadoop 1 support with Flink 1.2

+1 for dropping Hadoop1 support. Regarding a binary release without Hadoop: What would we do about the readHadoopFile() and createHadoopInput() on the ExecutionEnvironment? These methods are declared as @PublicEvolving, so we did not commit to keep them. However that does not necessarily mean we

Re: [DISCUSS] Drop Hadoop 1 support with Flink 1.2

@Greg I think that would be amazing. It does require a bit of cleanup, though. As far as I know, the Hadoop dependency is additionally used for some Kerberos utilities and for its S3 file system implementation. We would need to make the Kerberos part Hadoop independent and the FileSystem loading

Re: [DISCUSS] Drop Hadoop 1 support with Flink 1.2

+1 for dropping Hadoop1 support @greg There is quite some complexity in the build setup and release scripts and testing to support Hadoop 1. Also, we have to prepare to add support for Hadoop 3, and then supporting in addition Hadoop 1 seems very tough. Stephan On Thu, Oct 13, 2016 at 5:04 PM,

Re: [DISCUSS] Drop Hadoop 1 support with Flink 1.2

gt; *To:* dev@flink.apache.org; u...@flink.apache.org > *Subject:* [DISCUSS] Drop Hadoop 1 support with Flink 1.2 > > > > Hi, > > > > The Apache Hadoop community has recently released the first alpha version > for Hadoop 3.0.0, while we are still supporting Hadoop

Re: [DISCUSS] Drop Hadoop 1 support with Flink 1.2

Hi Robert, What are the benefits to Flink for dropping Hadoop 1 support? Is there significant code cleanup or would we simply be publishing one less set of artifacts? Greg On Thu, Oct 13, 2016 at 10:47 AM, Robert Metzger wrote: > Hi, > > The Apache Hadoop community has

RE: [DISCUSS] Drop Hadoop 1 support with Flink 1.2

...@flink.apache.org Subject: [DISCUSS] Drop Hadoop 1 support with Flink 1.2 Hi, The Apache Hadoop community has recently released the first alpha version for Hadoop 3.0.0, while we are still supporting Hadoop 1. I think its time to finally drop Hadoop 1 support in Flink. The last minor Hadoop 1

[DISCUSS] Drop Hadoop 1 support with Flink 1.2

Hi, The Apache Hadoop community has recently released the first alpha version for Hadoop 3.0.0, while we are still supporting Hadoop 1. I think its time to finally drop Hadoop 1 support in Flink. The last minor Hadoop 1 release was in 27 June, 2014. Apache Spark dropped Hadoop 1 support with