Would be great if it actually just works with Hadoop 3. If a new release is cut after blocker bug fix, I can attempt to test it to make sure it works.
Sorry - Never got the comments from dev@crunch ( emailed first and then subscribed ) But yeah new licensing model. Hadoop 3 is Java 8 for now and JDK11 is in progress: https://issues.apache.org/jira/browse/HADOOP-15338 <https://issues.apache.org/jira/browse/HADOOP-15338> As CDH 6 is using Hadoop 3 on Java 8 and support for crunch is dropped, if a new release cannot have Hadoop 3 and Java 8 maybe there can be a separate branch for it, if code changes are required? Or maybe I can attempt it in my fork? Regards, — Kavin Kankeshwar > On Jun 19, 2019, at 3:14 PM, Josh Wills <[email protected]> wrote: > > I haven't run it (we use Hadoop 2.x at Slack still), but I would assume yes? > Happy to be contradicted tho. > > Quoting my comments on [email protected] <mailto:[email protected]>: > > I am happy to cut a release whenever (well, as soon as we fix that bug that > was just filed that's dumping secrets to the log file)-- but I'm unsure on > whether I should upgrade versions; Hadoop 3 is on Java 8 too, yes? What's the > impetus for moving to JDK11? And I assume it has to be OpenJDK11 b/c of the > new licensing model? > > On Wed, Jun 19, 2019 at 2:03 PM Kavin Kankeshwar <[email protected] > <mailto:[email protected]>> wrote: > -- newbie alert -- > > Hi, > > I see docs only mentioning Hadoop 1.x and Hadoop 2.x Wondering does Crunch > work with Hadoop 3? > > Also was looking at crunch-core > <https://mvnrepository.com/artifact/org.apache.crunch/crunch-core> releases > in central maven repository as seems like last official release was in 2017. > Also with CDH6.X onwards Cloudera is not going to support Crunch for JDK11. I > do see active commits in Crunch git repos, Wondering if crunch will be ready > for JDK11 and if its released to central maven repository ? Or would I need > to clone and package the crunch-core jars ? > > Regards, > — > Kavin Kankeshwar
