[jira] [Commented] (NUTCH-2512) Nutch 1.14 does not work under JDK9
[ https://issues.apache.org/jira/browse/NUTCH-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16488708#comment-16488708 ] Sebastian Nagel commented on NUTCH-2512: Hi [~Bl4ck1c3], while testing HTTP/2 (NUTCH-2576) I was able to run the Nutch master compiled with Java 8 successfully using a Java 11 runtime. I've tried to reproduce your problem with Nutch master and 1.14 (both compiled with Java 8) and Java 11 as runtime in local mode: injector works fine and succeeds. Either the problem is specific to Java 9 and has been fixed in later versions or it has another reason. The stack resembles the problem described in NUTCH-2533. Could you share more details about Java versions and mode (local/pseudo-distributed)? Does Injector run with the same command using a Java 8 runtime? Thanks! > Nutch 1.14 does not work under JDK9 > --- > > Key: NUTCH-2512 > URL: https://issues.apache.org/jira/browse/NUTCH-2512 > Project: Nutch > Issue Type: Bug > Components: build, injector >Affects Versions: 1.14 > Environment: Ubuntu 16.04 (All patches up to 02/20/2018) > Oracle Java 9 - Oracle JDK 9 (Latest as off 02/22/2018) >Reporter: Ralf >Priority: Major > Fix For: 1.15 > > > Nutch 1.14 (Source) does not compile properly under JDK 9 > Nutch 1.14 (Binary) does not function under Java 9 > > When trying to Nuild Nutch, Ant complains about missing Sonar files then > exits with: > "BUILD FAILED > /home/nutch/nutch/build.xml:79: Unparseable date: "01/25/1971 2:00 pm" " > > Once having commented out the "offending code" the Build finishes but the > resulting Binary fails to function (as well as the Apache Compiled Binary > distribution), Both exit with: > > Injecting seed URLs > /home/nutch/nutch2/bin/nutch inject searchcrawl//crawldb urls/ > Injector: starting at 2018-02-21 02:02:16 > Injector: crawlDb: searchcrawl/crawldb > Injector: urlDir: urls > Injector: Converting injected urls to crawl db entries. > WARNING: An illegal reflective access operation has occurred > WARNING: Illegal reflective access by > org.apache.hadoop.security.authentication.util.KerberosUtil > (file:/home/nutch/nutch2/lib/hadoop-auth-2.7.4.jar) to method > sun.security.krb5.Config.getInstance() > WARNING: Please consider reporting this to the maintainers of > org.apache.hadoop.security.authentication.util.KerberosUtil > WARNING: Use --illegal-access=warn to enable warnings of further illegal > reflective access operations > WARNING: All illegal access operations will be denied in a future release > Injector: java.lang.NullPointerException > at > org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getBlockIndex(FileInputFormat.java:444) > at > org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:413) > at > org.apache.hadoop.mapreduce.lib.input.DelegatingInputFormat.getSplits(DelegatingInputFormat.java:115) > at > org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301) > at > org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318) > at > org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287) > at java.base/java.security.AccessController.doPrivileged(Native > Method) > at java.base/javax.security.auth.Subject.doAs(Subject.java:423) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746) > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287) > at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308) > at org.apache.nutch.crawl.Injector.inject(Injector.java:417) > at org.apache.nutch.crawl.Injector.run(Injector.java:563) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.nutch.crawl.Injector.main(Injector.java:528) > > Error running: > /home/nutch/nutch2/bin/nutch inject searchcrawl//crawldb urls/ > Failed with exit value 255. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NUTCH-2512) Nutch 1.14 does not work under JDK9
[ https://issues.apache.org/jira/browse/NUTCH-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484563#comment-16484563 ] Lewis John McGibbney commented on NUTCH-2512: - See my comment above... > Nutch 1.14 does not work under JDK9 > --- > > Key: NUTCH-2512 > URL: https://issues.apache.org/jira/browse/NUTCH-2512 > Project: Nutch > Issue Type: Bug > Components: build, injector >Affects Versions: 1.14 > Environment: Ubuntu 16.04 (All patches up to 02/20/2018) > Oracle Java 9 - Oracle JDK 9 (Latest as off 02/22/2018) >Reporter: Ralf >Priority: Major > Fix For: 1.15 > > > Nutch 1.14 (Source) does not compile properly under JDK 9 > Nutch 1.14 (Binary) does not function under Java 9 > > When trying to Nuild Nutch, Ant complains about missing Sonar files then > exits with: > "BUILD FAILED > /home/nutch/nutch/build.xml:79: Unparseable date: "01/25/1971 2:00 pm" " > > Once having commented out the "offending code" the Build finishes but the > resulting Binary fails to function (as well as the Apache Compiled Binary > distribution), Both exit with: > > Injecting seed URLs > /home/nutch/nutch2/bin/nutch inject searchcrawl//crawldb urls/ > Injector: starting at 2018-02-21 02:02:16 > Injector: crawlDb: searchcrawl/crawldb > Injector: urlDir: urls > Injector: Converting injected urls to crawl db entries. > WARNING: An illegal reflective access operation has occurred > WARNING: Illegal reflective access by > org.apache.hadoop.security.authentication.util.KerberosUtil > (file:/home/nutch/nutch2/lib/hadoop-auth-2.7.4.jar) to method > sun.security.krb5.Config.getInstance() > WARNING: Please consider reporting this to the maintainers of > org.apache.hadoop.security.authentication.util.KerberosUtil > WARNING: Use --illegal-access=warn to enable warnings of further illegal > reflective access operations > WARNING: All illegal access operations will be denied in a future release > Injector: java.lang.NullPointerException > at > org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getBlockIndex(FileInputFormat.java:444) > at > org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:413) > at > org.apache.hadoop.mapreduce.lib.input.DelegatingInputFormat.getSplits(DelegatingInputFormat.java:115) > at > org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301) > at > org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318) > at > org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287) > at java.base/java.security.AccessController.doPrivileged(Native > Method) > at java.base/javax.security.auth.Subject.doAs(Subject.java:423) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746) > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287) > at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308) > at org.apache.nutch.crawl.Injector.inject(Injector.java:417) > at org.apache.nutch.crawl.Injector.run(Injector.java:563) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.nutch.crawl.Injector.main(Injector.java:528) > > Error running: > /home/nutch/nutch2/bin/nutch inject searchcrawl//crawldb urls/ > Failed with exit value 255. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NUTCH-2512) Nutch 1.14 does not work under JDK9
[ https://issues.apache.org/jira/browse/NUTCH-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484547#comment-16484547 ] Ralf commented on NUTCH-2512: - Hi, I'm really curios.. I've been taking apart the 1.14 Source these last few days How do you update to a new Java Version? > Nutch 1.14 does not work under JDK9 > --- > > Key: NUTCH-2512 > URL: https://issues.apache.org/jira/browse/NUTCH-2512 > Project: Nutch > Issue Type: Bug > Components: build, injector >Affects Versions: 1.14 > Environment: Ubuntu 16.04 (All patches up to 02/20/2018) > Oracle Java 9 - Oracle JDK 9 (Latest as off 02/22/2018) >Reporter: Ralf >Priority: Major > Fix For: 1.15 > > > Nutch 1.14 (Source) does not compile properly under JDK 9 > Nutch 1.14 (Binary) does not function under Java 9 > > When trying to Nuild Nutch, Ant complains about missing Sonar files then > exits with: > "BUILD FAILED > /home/nutch/nutch/build.xml:79: Unparseable date: "01/25/1971 2:00 pm" " > > Once having commented out the "offending code" the Build finishes but the > resulting Binary fails to function (as well as the Apache Compiled Binary > distribution), Both exit with: > > Injecting seed URLs > /home/nutch/nutch2/bin/nutch inject searchcrawl//crawldb urls/ > Injector: starting at 2018-02-21 02:02:16 > Injector: crawlDb: searchcrawl/crawldb > Injector: urlDir: urls > Injector: Converting injected urls to crawl db entries. > WARNING: An illegal reflective access operation has occurred > WARNING: Illegal reflective access by > org.apache.hadoop.security.authentication.util.KerberosUtil > (file:/home/nutch/nutch2/lib/hadoop-auth-2.7.4.jar) to method > sun.security.krb5.Config.getInstance() > WARNING: Please consider reporting this to the maintainers of > org.apache.hadoop.security.authentication.util.KerberosUtil > WARNING: Use --illegal-access=warn to enable warnings of further illegal > reflective access operations > WARNING: All illegal access operations will be denied in a future release > Injector: java.lang.NullPointerException > at > org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getBlockIndex(FileInputFormat.java:444) > at > org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:413) > at > org.apache.hadoop.mapreduce.lib.input.DelegatingInputFormat.getSplits(DelegatingInputFormat.java:115) > at > org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301) > at > org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318) > at > org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287) > at java.base/java.security.AccessController.doPrivileged(Native > Method) > at java.base/javax.security.auth.Subject.doAs(Subject.java:423) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746) > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287) > at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308) > at org.apache.nutch.crawl.Injector.inject(Injector.java:417) > at org.apache.nutch.crawl.Injector.run(Injector.java:563) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.nutch.crawl.Injector.main(Injector.java:528) > > Error running: > /home/nutch/nutch2/bin/nutch inject searchcrawl//crawldb urls/ > Failed with exit value 255. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NUTCH-2512) Nutch 1.14 does not work under JDK9
[ https://issues.apache.org/jira/browse/NUTCH-2512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373173#comment-16373173 ] Lewis John McGibbney commented on NUTCH-2512: - Hi [~Bl4ck1c3] thanks for logging the issue... this behavior was to be 'expected'. As you can see from the [javac.version|https://github.com/apache/nutch/blob/release-1.14/default.properties#L60] we had it pinned at 1.8. I suppose we can make the upgrade for 1.15... (queue patch ;)) > Nutch 1.14 does not work under JDK9 > --- > > Key: NUTCH-2512 > URL: https://issues.apache.org/jira/browse/NUTCH-2512 > Project: Nutch > Issue Type: Bug > Components: build, injector >Affects Versions: 1.14 > Environment: Ubuntu 16.04 (All patches up to 02/20/2018) > Oracle Java 9 - Oracle JDK 9 (Latest as off 02/22/2018) >Reporter: Ralf >Priority: Major > Fix For: 1.15 > > > Nutch 1.14 (Source) does not compile properly under JDK 9 > Nutch 1.14 (Binary) does not function under Java 9 > > When trying to Nuild Nutch, Ant complains about missing Sonar files then > exits with: > "BUILD FAILED > /home/nutch/nutch/build.xml:79: Unparseable date: "01/25/1971 2:00 pm" " > > Once having commented out the "offending code" the Build finishes but the > resulting Binary fails to function (as well as the Apache Compiled Binary > distribution), Both exit with: > > Injecting seed URLs > /home/nutch/nutch2/bin/nutch inject searchcrawl//crawldb urls/ > Injector: starting at 2018-02-21 02:02:16 > Injector: crawlDb: searchcrawl/crawldb > Injector: urlDir: urls > Injector: Converting injected urls to crawl db entries. > WARNING: An illegal reflective access operation has occurred > WARNING: Illegal reflective access by > org.apache.hadoop.security.authentication.util.KerberosUtil > (file:/home/nutch/nutch2/lib/hadoop-auth-2.7.4.jar) to method > sun.security.krb5.Config.getInstance() > WARNING: Please consider reporting this to the maintainers of > org.apache.hadoop.security.authentication.util.KerberosUtil > WARNING: Use --illegal-access=warn to enable warnings of further illegal > reflective access operations > WARNING: All illegal access operations will be denied in a future release > Injector: java.lang.NullPointerException > at > org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getBlockIndex(FileInputFormat.java:444) > at > org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:413) > at > org.apache.hadoop.mapreduce.lib.input.DelegatingInputFormat.getSplits(DelegatingInputFormat.java:115) > at > org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301) > at > org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318) > at > org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287) > at java.base/java.security.AccessController.doPrivileged(Native > Method) > at java.base/javax.security.auth.Subject.doAs(Subject.java:423) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746) > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287) > at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308) > at org.apache.nutch.crawl.Injector.inject(Injector.java:417) > at org.apache.nutch.crawl.Injector.run(Injector.java:563) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.nutch.crawl.Injector.main(Injector.java:528) > > Error running: > /home/nutch/nutch2/bin/nutch inject searchcrawl//crawldb urls/ > Failed with exit value 255. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
