Hi Markus, thanks! What's your (final) decision?
> [javac] import org.apache.http.impl.nio.client.HttpAsyncClientBuilder; During build the class should be provided in build/plugins/indexer-elastic/httpasyncclient-4.1.4.jar Could you verify whether this jar is there and whether it contains the class file? See also: https://repo1.maven.org/maven2/org/apache/httpcomponents/httpasyncclient/4.1.4/ > I am worried about the indexer-elastic plugin, maybe others have that > problem too? Otherwise everything seems fine. In order to fix it, we need to make the error reproducible resp. figure out what the reason is. Regarding the logging: we switched to log4j 2.x (NUTCH-2915) while Hadoop now uses reload4j (HADOOP-18088 [1]). The logging configuration should be improved to avoid the warnings in local mode. In distributed mode, the logging configuration of the provided Hadoop takes over. Best, Sebastian [1] https://issues.apache.org/jira/browse/HADOOP-18088 On 8/24/22 13:28, Markus Jelsma wrote: > Hi, > > Everything seems fine, the crawler seems fine when trying the binary > distribution. The source won't work because this computer still cannot > compile it. Clearing the local Ivy cache did not do much. This is the known > compiler error with the elastic-indexer plugin: > compile: > [echo] Compiling plugin: indexer-elastic > [javac] Compiling 3 source files to > /home/markus/temp/apache-nutch-1.19/build/indexer-elastic/classes > [javac] > /home/markus/temp/apache-nutch-1.19/src/plugin/indexer-elastic/src/java/org/apache/nutch/indexwriter/elastic/ElasticIndexWriter.java:39: > error: package org.apache.http.impl.nio.client does not exist > [javac] import org.apache.http.impl.nio.client.HttpAsyncClientBuilder; > [javac] ^ > [javac] 1 error > > > The binary distribution works fine though. I do see a lot of new messages > when fetching: > 2022-08-24 13:21:15,867 INFO o.a.n.n.URLExemptionFilters [LocalJobRunner > Map Task Executor #0] Found 0 extensions at > point:'org.apache.nutch.net.URLExemptionFilter' > > This is also new at start of each task: > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/home/markus/temp/apache-nutch-1.19/lib/log4j-slf4j-impl-2.18.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] > > SLF4J: Found binding in > [jar:file:/home/markus/temp/apache-nutch-1.19/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class] > > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > SLF4J: Actual binding is of type > [org.apache.logging.slf4j.Log4jLoggerFactory] > > And this one at the end of fetcher: > log4j:WARN No appenders could be found for logger > (org.apache.commons.httpclient.params.DefaultHttpParams). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for > more info. > > I am worried about the indexer-elastic plugin, maybe others have that > problem too? Otherwise everything seems fine. > > Markus > > Op ma 22 aug. 2022 om 17:30 schreef Sebastian Nagel <sna...@apache.org>: > >> Hi Folks, >> >> A first candidate for the Nutch 1.19 release is available at: >> >> https://dist.apache.org/repos/dist/dev/nutch/1.19/ >> >> The release candidate is a zip and tar.gz archive of the binary and >> sources in: >> https://github.com/apache/nutch/tree/release-1.19 >> >> In addition, a staged maven repository is available here: >> https://repository.apache.org/content/repositories/orgapachenutch-1020 >> >> We addressed 87 issues: >> https://s.apache.org/lf6li >> >> >> Please vote on releasing this package as Apache Nutch 1.19. >> The vote is open for the next 72 hours and passes if a majority >> of at least three +1 Nutch PMC votes are cast. >> >> [ ] +1 Release this package as Apache Nutch 1.19. >> [ ] -1 Do not release this package because… >> >> Cheers, >> Sebastian >> (On behalf of the Nutch PMC) >> >> P.S. >> Here is my +1. >> - tested most of Nutch tools and run a test crawl on a single-node cluster >> running Hadoop 3.3.4, see >> https://github.com/sebastian-nagel/nutch-test-single-node-cluster/) >> >
