commits  

Messages by Date

• 2022/11/14 [nutch-webapp] branch dependabot/maven/org.springframework-spring-web-4.2.7.RELEASE created (now faede31) github-bot
• 2022/09/11 [nutch] branch master updated (85f7bcb63 -> ed7b6615b) snagel
• 2022/09/10 svn commit: r56776 - /release/nutch/1.18/ snagel
• 2022/09/08 [nutch] 02/02: Prepare for new development after release of 1.19 - bump version number (-> 1.20-NAPSHOT) snagel
• 2022/09/08 [nutch] branch master updated (ffe059892 -> 85f7bcb63) snagel
• 2022/09/08 [nutch] 01/02: Nutch 1.19 release - update current year in API docs etc. - update version number - add changes / release notes - update links to Hadoop API docs snagel
• 2022/09/08 [nutch-site] 02/02: Announce release of Nutch 1.19 - fix release data in announcement snagel
• 2022/09/08 [nutch-site] branch main updated (4efc5a9 -> aa45c17) snagel
• 2022/09/08 [nutch-site] branch asf-site updated: Announce release of Nutch 1.19 - fix release data in announcement snagel
• 2022/09/08 [nutch-site] branch asf-staging updated: Announce release of Nutch 1.19 - fix release data in announcement snagel
• 2022/09/08 [nutch-site] branch asf-site updated (a41c7ef -> 314b1b2) snagel
• 2022/09/08 [nutch-site] 02/03: Update content from Hugo build after adding Kube modified templates snagel
• 2022/09/08 [nutch-site] 01/03: - add README for branch asf-site - modify .asf.yaml to contain only instructions required in branch asf-site snagel
• 2022/09/08 [nutch-site] branch asf-staging updated (3e9e725 -> 2cfe00d) snagel
• 2022/09/08 [nutch-site] branch asf-staging updated: Announce release of Nutch 1.19 snagel
• 2022/09/08 svn commit: r56738 [1/3] - /release/nutch/1.19/CHANGES.txt snagel
• 2022/09/08 svn commit: r56738 [3/3] - /release/nutch/1.19/CHANGES.txt snagel
• 2022/09/08 svn commit: r56738 [2/3] - /release/nutch/1.19/CHANGES.txt snagel
• 2022/09/08 [nutch-site] branch main updated: NUTCH-1999 Add /robots.txt to Nutch site (#1) snagel
• 2022/09/08 [nutch-site] branch asf-staging updated: - add README for branch asf-staging - modify .asf.yaml to contain only instructions required in branch asf-staging snagel
• 2022/09/08 [nutch-site] branch NUTCH-1999-nutch-site-robots-txt updated (142489f -> f863c1f) snagel
• 2022/09/08 [nutch-site] branch asf-staging updated: Sync .asf.yaml file with main branch snagel
• 2022/09/08 [nutch-site] 01/01: Update content from Hugo build after adding Kube modified templates snagel
• 2022/09/08 [nutch-site] branch asf-staging created (now d77dbb5) snagel
• 2022/09/06 svn commit: r56686 - /dev/nutch/1.19/ /release/nutch/1.19/ snagel
• 2022/08/22 svn commit: r56398 - /dev/nutch/1.19/ snagel
• 2022/08/22 [nutch] branch branch-1.19 created (now 63d4f11c0) snagel
• 2022/08/22 [nutch] annotated tag release-1.19 updated (63d4f11c0 -> 5d7660ceb) snagel
• 2022/08/22 [nutch] branch master updated: NUTCH-2969 Javadoc: Javascript search is not working when built on JDK 11 - pass --no-module-directories to javadoc target when building on JDK 11 - remove obsolete condition to fail javadoc builds on JDK 7u25 and earlier snagel
• 2022/08/21 [nutch] branch master updated (bca5fc0d0 -> 635ef2f3b) snagel
• 2022/08/21 [nutch] branch master updated (bec577d50 -> bca5fc0d0) snagel
• 2022/08/21 [nutch] branch master updated: NUTCH-2863 Injector to parse command-line flags case-insensitive snagel
• 2022/08/19 [nutch] branch master updated: NUTCH-2962 Update and complete package info of protocol plugins snagel
• 2022/08/19 [nutch] branch master updated: NUTCH-2930 Protocol-okhttp: implement IP filter (#736) snagel
• 2022/08/19 [nutch] branch master updated (c0f723e99 -> 05afebd03) snagel
• 2022/08/17 [nutch] branch master updated (edebfe49f -> c0f723e99) snagel
• 2022/08/17 [nutch] branch master updated (a5a630055 -> edebfe49f) snagel
• 2022/08/15 [nutch] branch master updated (82f9530dc -> a5a630055) snagel
• 2022/08/15 [nutch] branch master updated (b7b834501 -> 82f9530dc) snagel
• 2022/08/12 [nutch] branch master updated (8fc4f17ac -> b7b834501) snagel
• 2022/08/09 [nutch] branch master updated: NUTCH-2956 index-geoip: dependency upgrades and improvements - upgrade to geoip2 3.0.1 - exclude transitive dependencies (Jackson) provided as Nutch core deps - read also GeoLite2-*.mmdb files - review index field names in plugin and Nutch Solr schema: - fix typos in field names - remove unused fields from schema snagel
• 2022/08/09 [nutch] branch master updated: NUTCH-2953 Indexer Elastic to ignore SSL issues - apply patch contributed by Markus Jelsma - fix class imports snagel
• 2022/08/09 [nutch] branch master updated: NUTCH-2952 Upgrade core dependencies - Hadoop 3.1.3 -> 3.3.3 - log4j 2.17.0 -> 2.17.2 - and some more snagel
• 2022/08/09 [nutch] 01/03: NUTCH-2936 Early registration of URL stream handlers provided by plugins may fail Hadoop jobs running in distributed mode if protocol-okhttp is used - protocol-okhttp: initialize SSLContext used to ignore SSL/TLS certificate verificiation not in a static code block snagel
• 2022/08/09 [nutch] 02/03: NUTCH-2936 Early registration of URL stream handlers provided by plugins may fail Hadoop jobs running in distributed mode if protocol-okhttp is used - code improvements Nutch plugin system: - use `Class<?>` and remove suppressions of warnings - javadocs: fix typos - remove superfluous white space - autoformat using code style template snagel
• 2022/08/09 [nutch] 03/03: NUTCH-2936 Early registration of URL stream handlers provided by plugins may fail Hadoop jobs running in distributed mode if protocol-okhttp is used NUTCH-2949 Tasks of a multi-threaded map runner may fail because of slow creation of URL stream handlers snagel
• 2022/08/09 [nutch] branch master updated (5b970ff22 -> 487110b07) snagel
• 2022/06/21 [nutch] branch master updated: NUTCH-2951 Crawl datum with metadata WRITABLE_GENERATE_TIME_KEY awaits fetching forever - bug fix: add missing braces (bug introduced with NUTCH-2737, solution contributed by Lapadula Alessandro) snagel
• 2022/05/25 [nutch-webapp] branch dependabot/maven/org.springframework-spring-core-5.2.22.RELEASE created (now e19e71e) github-bot
• 2022/05/24 [nutch] branch master updated (02dca3b6d -> 47d3fe607) snagel
• 2022/05/20 [nutch] branch master updated: NUTCH-2936 Early registration of URL stream handlers provided by plugins may fail Hadoop jobs running in distributed mode (#726) lewismc
• 2022/05/19 [nutch] 02/02: NUTCH-2946 Fetcher: optionally slow down fetching from hosts with repeated exceptions - configure the delay in seconds as a float instead of milliseconds - use the value of fetcher.server.delay as default - double the delay with every observed exception (exponential backoff) but cap the growth at 2**31 to avoid overflows snagel
• 2022/05/19 [nutch] branch master updated (568993b90 -> bdbe7b330) snagel
• 2022/05/19 [nutch] 01/02: NUTCH-2946 Fetcher: slow down fetching from hosts where requests fail repeatedly with exceptions or HTTP status codes mapped to ProtocolStatus.EXCEPTION (HTTP 403 Forbidden, 429 Too many requests, 5xx server errors, etc.) snagel
• 2022/05/12 [nutch] branch master updated: NUTCH-2948 Upgrade dependencies to Any23 2.7 and Tika 2.3.0 snagel
• 2022/04/22 [nutch-webapp] branch dependabot/maven/org.springframework-spring-core-5.3.19 created (now b974d7a) github-bot
• 2022/04/06 [nutch-webapp] branch dependabot/maven/org.springframework-spring-core-5.3.18 created (now 6524080) github-bot
• 2022/01/27 [nutch] branch master updated: NUTCH-2923: Added JobId in Job Failure logs (#721) snagel
• 2022/01/17 [nutch] branch master updated: NUTCH-2573 Suspend crawling if robots.txt fails to fetch with 5xx status (#724) snagel
• 2022/01/17 [nutch] branch master updated: NUTCH-2935 DeduplicationJob: failure on URLs with invalid percent encoding - catch IllegalArgumentException when unescaping percent-encoding in URLs - if one URL of two compared URLs is valid, keep it as non-duplicate - add unit tests for DeduplicationJob snagel
• 2022/01/15 [nutch] branch master updated: NUTCH-2919 Upgrade to Tika 2.2.1 and Any23 2.6 (#717) lewismc
• 2022/01/14 [nutch] branch master updated: NUTCH-2929 Fetcher: start threads slowly to avoid that resources are temporarily exhausted - sleep for a configurable delay (fetcher.threads.start.delay) before starting the next Fetcher thread to avoid that resources (DNS, Tika XML parser pools) are temporarily exhausted when Fetcher threads fetch the first pages simultaneously snagel
• 2022/01/09 [nutch-site] 01/01: NUTCH-1999 Add /robots.txt to Nutch site snagel
• 2022/01/09 [nutch-site] branch NUTCH-1999-nutch-site-robots-txt created (now 142489f) snagel
• 2022/01/09 [nutch] branch master updated (e76d69f -> 78e827a) snagel
• 2022/01/07 [nutch] branch master updated: NUTCH-2429 Fix Plugin System to allow protocol plugins to bundle their URLStreamHandlers (#720) lewismc
• 2022/01/05 svn commit: r1077907 - /websites/staging/nutch/ gmcdonald
• 2022/01/05 svn commit: r1077906 - /websites/production/nutch/ gmcdonald
• 2021/12/22 [nutch] branch master updated: Upgrade to log4j 2.17.0 (#719) snagel
• 2021/12/22 [nutch] branch master updated: NUTCH-2917 Remove transitive dependency to log4j 1.x (#718) snagel
• 2021/12/17 [nutch] branch master updated: NUTCH-2449 Replace Tika LanguageIdentifier in language-identifier (#716) lewismc
• 2021/12/17 [nutch] branch master updated: NUTCH-2914 nutch-default.xml: remove obsolete and unused properties (#709) snagel
• 2021/12/17 [nutch] branch master updated: NUTCH-2807 SitemapProcessor to warn that ignoring robots.txt affects detection of sitemaps (#710) snagel
• 2021/12/17 [nutch] branch master updated (4caa5ce -> af29192) snagel
• 2021/12/17 [nutch] branch master updated: NUTCH-2918 Upgrade to log4j 2.16.0 (#715) snagel
• 2021/12/14 [nutch] branch master updated: NUTCH-2916 Fix log file rotation / rename default log file (#714) snagel
• 2021/12/13 [nutch] branch master updated: NUTCH-2915 Upgrade to log4j 2.15.0 snagel
• 2021/12/03 [nutch] branch master updated (a62168c -> 9a2f94f) snagel
• 2021/12/03 [nutch] branch master updated (dd27044 -> a62168c) snagel
• 2021/12/01 [nutch] branch master updated: NUTCH-2905 Mask sensitive strings in log output of index writers - add utility methods (StringUtil) to mask password strings or passwords in strings - mask passwords in log output of index writers (Elasticsearch, Solr, RabbitMQ) - mask password in trace log of protocol-httpclient when using basic authentication snagel
• 2021/12/01 [nutch] branch master updated: NUTCH-2908 Log mapreduce job messages and counters in local mode (Log4j2) snagel
• 2021/12/01 [nutch] branch master updated (ff800c5 -> 671f904) snagel
• 2021/11/26 [nutch-site] branch main updated: Add doap.rdf (lost during CMS migration) snagel
• 2021/11/26 [nutch-site] branch asf-site updated: Add doap.rdf (lost during CMS migration) snagel
• 2021/11/24 [nutch-site] branch asf-site updated: Add favicon lewismc
• 2021/11/24 [nutch-site] branch main updated (b720870 -> 198d962) lewismc
• 2021/11/24 [nutch-site] branch main updated (819de2a -> b720870) lewismc
• 2021/11/24 [nutch-site] branch asf-site created (now b720870) lewismc
• 2021/11/24 [nutch-site] 01/01: Add .asf.yaml file to Nutch website lewismc
• 2021/11/23 [nutch-site] branch main updated: Remove broken site lewismc
• 2021/11/22 [nutch] branch master updated (75daf3e -> ff800c5) snagel
• 2021/11/22 [nutch] branch master updated (64fb604 -> 75daf3e) snagel
• 2021/11/22 [nutch] branch master updated (25ccf89 -> 64fb604) snagel
• 2021/11/22 [nutch] branch master updated: Upgrade to crawler-commons 1.2 snagel
• 2021/11/18 [nutch] branch master updated: NUTCH-2902 Jexl parsing error on statements (contributed by Max Ockner) - use JexlScript instead of JexlExpression in Generator, CrawlDb/HostDb reader, Jexl exchange and indexing filter snagel
• 2021/11/11 [nutch] branch master updated: NUTCH-2899 Remove needless warning about missing o/a/rat/anttasks/antlib.xml - avoid needless warning by moving taskdef into task element snagel
• 2021/10/20 [nutch] branch master updated: NUTCH-2862 Do not include Ivy jar in source release package snagel
• 2021/10/19 [nutch] branch master updated: quick IntelliJ IDEA setup docs added (#698) lewismc
• 2021/09/22 [nutch] branch master updated (eeb9863 -> c48b8d1) snagel
• 2021/09/22 [nutch] branch master updated: NUTCH-2894 Java plugin compilation classpath: priorize plugin dependencies snagel
• 2021/09/17 [nutch] branch master updated: fireant upgrade dependency elasticsearch-rest-high-level-client in src/plugin/indexer-elastic/ivy.xml from 7.11.1 to 7.13.2 (#688) lewismc
• 2021/08/27 [nutch-site] branch main updated: Attempt to implement single page templating. lewismc
• 2021/08/26 [nutch-site] 01/01: NUTCH-2826 Migrate Nutch Site from Apache CMS to Hugo lewismc
• 2021/08/26 [nutch-site] branch main created (now ae6f9f2) lewismc
• 2021/08/04 [nutch] branch master updated: NUTCH-2885 Upgrade to Log4j2 (#692) lewismc
• 2021/07/13 [nutch-webapp] branch master updated: Add missing files lewismc
• 2021/07/12 [nutch-webapp] branch master created (now da3c282) lewismc
• 2021/06/30 [nutch] branch master updated: fireant upgrade dependency httpcore in ivy/ivy.xml from 4.4.9 to 4.4.14 (#681) lewismc
• 2021/06/28 [nutch] branch master updated: NUTCH-2882 Configure NutchUiServer for DEPLOYMENT and improve logging (#690) lewismc
• 2021/06/26 [nutch] branch master updated: NUTCH-2881 bug in 'nutch' symlink in docker container (#689) lewismc
• 2021/06/15 You have (8) new email notifications. postmaster-delivery
• 2021/06/14 [nutch] branch master updated: NUTCH-2868 urlnormalizer-protocol fails with StringIndexOutOfBoundsException when reading invalid line in configuration file - log invalid line and skip over it - more verbose logging which configuration file is read - add unit test to proof that invalid configuration lines are skipped snagel
• 2021/06/13 [nutch] branch master updated: fireant upgrade dependency junit in ivy/ivy.xml from 4.13.1 to 4.13.2 (#666) lewismc
• 2021/06/12 [nutch] branch master updated: NUTCH-2869 Add @Override annotations to Nutch plugins - add/complete @Override annotions for methods implementing interfaces - plugins implementing the ScoringFilter interface: extend AbstractScoringFilter and get rid of default method implementations - URL filters/normalizers: remove unused methods including a CrawlDatum parameter - improve Javadoc and documentation in build and config files snagel
• 2021/06/03 [nutch] branch master updated: NUTCH-2864 Upgrade Dockerfile to use JDK 11 (#647) lewismc
• 2021/06/01 [nutch] branch master updated: NUTCH-2866 Fix MetaData.toString() to return "key=value ..." snagel
• 2021/05/14 svn commit: r1074658 - /websites/production/nutch/content/ snagel
• 2021/05/14 svn commit: r1074656 - in /websites/staging/nutch/trunk/content: ./ apidocs/apidocs-1.18/resources/ apidocs/apidocs-1.18/resources/configuration.xsl apidocs/apidocs-1.18/resources/nutch-default.xml buildbot
• 2021/05/14 svn commit: r1889891 [2/3] - in /nutch/cms_site/trunk/content/apidocs/apidocs-1.18/resources: ./ configuration.xsl nutch-default.xml snagel
• 2021/05/14 svn commit: r1889891 [1/3] - in /nutch/cms_site/trunk/content/apidocs/apidocs-1.18/resources: ./ configuration.xsl nutch-default.xml snagel
• 2021/05/14 svn commit: r1889891 [3/3] - in /nutch/cms_site/trunk/content/apidocs/apidocs-1.18/resources: ./ configuration.xsl nutch-default.xml snagel
• 2021/05/05 Retrieve your pending mails immediately. Admin lucene . apache . org
• 2021/04/06 [nutch] branch master updated (2837039 -> 6c02da0) snagel
• 2021/04/01 [nutch] branch master updated: NUTCH-2855 Update org.elasticsearch.client (#577) lewismc
• 2021/03/21 [nutch] branch master updated: NUTCH-2857 Upgrade from JDK1.8 --> JDK11 (#573) lewismc
• 2021/03/16 [nutch] branch master updated: NUTCH-2596 Upgrade from org.mortbay.jetty to org.eclipse.jetty - remove Jetty (serving JSP pages) for HTTP protocol plugin tests - replace JSP pages by header/content strings hold in unit test classes snagel
• 2021/02/18 [nutch] branch master updated: NUTCH-2850 Method ignores exceptional return value (#570) lewismc
• 2021/02/18 [nutch] branch master updated: NUTCH-2851 Random object created and used only once (#571) lewismc
• 2021/02/16 [nutch] branch master updated: NUTCH-2849 Replace remaining package.html files with package-info.java (#569) lewismc
• 2021/02/01 [nutch] branch master updated: NUTCH-2847 HttpDateFormat: Simplify based on new Java 8 DateTime API snagel
• 2021/02/01 [nutch] branch master updated (3483a41 -> 491b5c2) snagel
• 2021/02/01 [nutch] branch master updated (7ffc667 -> 3483a41) snagel
• 2021/02/01 [nutch] branch master updated: NUTCH-2845 Complete rules of urlfilter-suffix, add more excluded file suffixes for - images - audio and video formats - software packages and archives - fonts snagel
• 2021/01/31 [nutch] branch master updated: NUTCH-2840 Fix 'report-vulnerabilities' ant target in build.xml (#561) lewismc
• 2021/01/31 [nutch] branch master updated: NUTCH-2819 Move spotbugs "installation" directory to avoid that spotbugs is shipped in Nutch runtime (#565) lewismc
• 2021/01/25 [nutch] branch master updated: Prepare for Nutch 1.19-SNAPSHOT development lewismc
• 2021/01/25 svn commit: r1070562 - /websites/production/nutch/content/ lewismc
• 2021/01/24 svn commit: r45580 - /release/nutch/1.17/ lewismc
• 2021/01/24 svn commit: r1070514 - in /websites/staging/nutch/trunk/content: ./ apidocs/apidocs-1.18/ apidocs/apidocs-1.18/org/ apidocs/apidocs-1.18/org/apache/ apidocs/apidocs-1.18/org/apache/nutch/ apidocs/apidocs-1.18/org/apache/nutch/analysis/ apidocs/apidocs-... buildbot
• 2021/01/24 svn commit: r1885887 - in /nutch/cms_site/trunk/content: ./ apidocs/apidocs-1.18/ apidocs/apidocs-1.18/org/ apidocs/apidocs-1.18/org/apache/ apidocs/apidocs-1.18/org/apache/nutch/ apidocs/apidocs-1.18/org/apache/nutch/analysis/ apidocs/apidocs-1.18/org... lewismc
• 2021/01/24 svn commit: r45570 - in /release/nutch/1.18: apache-nutch-1.18-bin.zip.md5 apache-nutch-1.18-src.tar.gz.md5 apache-nutch-1.18-src.zip.md5 lewismc
• 2021/01/24 svn commit: r45569 - /release/nutch/1.18/apache-nutch-1.18-bin.tar.gz.md5 lewismc
• 2021/01/24 svn commit: r45568 - /dev/nutch/1.18/ /release/nutch/1.18/ lewismc
• 2021/01/20 svn commit: r45520 [3/3] - /dev/nutch/1.18/ lewismc
• 2021/01/20 svn commit: r45520 [2/3] - /dev/nutch/1.18/ lewismc
• 2021/01/20 svn commit: r45520 [1/3] - /dev/nutch/1.18/ lewismc
• 2021/01/19 [nutch] annotated tag release-1.18 updated (43f3550 -> a8ef299) lewismc
• 2021/01/19 [nutch] branch branch-1.18 updated: Prepare for Nutch 1.18 release lewismc
• 2021/01/14 [nutch] 01/01: Prepare for Nutch 1.18 release lewismc
• 2021/01/14 [nutch] branch branch-1.18 created (now e9f125c) lewismc
• 2021/01/13 [nutch] branch master updated: NUTCH-2841 Upgrade xercesImpl dependency (#563) lewismc
• 2021/01/08 [nutch] branch master updated: NUTCH-2837 Update multiple dependencies (#560) lewismc
• 2021/01/07 [nutch] branch master updated: NUTCH-2836 Upgrade various commons dependencies (#559) lewismc
• 2020/12/17 [nutch] branch master updated: Add possibility to setup deduplication group mode in crawl script (#557) lewismc
• 2020/12/17 [nutch] branch master updated: NUTCH-2835 Upgrade commons-jexl from 2 --> 3 (#558) lewismc
• 2020/12/08 [nutch] branch master updated: NUTCH-2833 Upgrade to Tika 1.25 snagel
• 2020/11/18 [nutch] branch master updated: NUTCH-2582 Set pool size of XML SAX parsers used for MIME detection in Tika - add method in MimeUtil to set MimeTypesReader pool size - actually adjust pool size to number of Fetcher threads / 2 (minimum pool size is 10 in case there are less than 20 Fetcher threads) - double pool size (10 -> 20) of Tika XMLReaderUtils in tika-config.xml snagel
• 2020/11/17 [nutch] branch master updated: NUTCH-2809 Upgrade any23 plugin dependency to 2.4 (#553) lewismc
• 2020/11/13 [nutch] branch master updated: NUTCH-2829 Fix ant target "clean-cache" - make target "clean-cache" depend on "ivy-init" so that ivy-related resources are defined snagel
• 2020/09/30 Subject: Important update for [email protected]: Please see transcript for details. Email Gateway Security
• 2020/09/14 [nutch] branch master updated (0b46ac2 -> 680df6b) snagel
• 2020/09/14 [nutch] branch master updated: NUTCH-2823 IllegalStateException in IndexWriters.describe() when validating url param for SolrIndexer snagel
• 2020/08/18 [nutch] branch master updated: NUTCH-2818 Fix Apache Rat task to check sources for license headers - automatize download of Apache Rat jar file - write report to build/apache-rat-report.txt snagel
• 2020/08/18 [nutch] branch master updated: NUTCH-2814 HttpDateFormat's internal time zone may change after parsing a date - reset time zone to GMT after parsing a date snagel
• 2020/08/17 [nutch] branch master updated: NUTCH-2697 Upgrade Ivy to fix the issue of an unset packaging.type property NUTCH-2671 Upgrade ant ivy library - upgrade Ivy (2.4.0 -> 2.5.0) - upgrade all plugins build-ivy.xml to use the ivy jar 2.5.0 installed in $NUTCH_HOME/ivy/ for preparing lists of dependencies registered in plugin.xml snagel
• 2020/08/16 [nutch] 04/35: NUTCH-2785 FreeGenerator: command-line option to define number of generated fetch lists - add command-line option `-numFetchers` to FreeGenerator - in local mode: generate one single fetch list snagel
• 2020/08/16 [nutch] branch master updated (466cac5 -> ae844b6) snagel
• 2020/08/16 [nutch] 35/35: Merge branch 'derhecht-patch-2', closes #545 snagel
• 2020/08/16 [nutch] 11/35: NUTCH-2720 ROBOTS metatag ignored when capitalized snagel
• 2020/08/16 [nutch] 19/35: NUTCH-2791 Handle GCS URLs in stats commands snagel
• 2020/08/16 [nutch] 13/35: NUTCH-2496 Speed up link inversion step in crawling script snagel
• 2020/08/16 [nutch] 30/35: [NUTCH-2801] RobotsRulesParser command-line checker to use http.robots.agents as fall-back - clarify comment regarding bypassing the confidence check for a non-empty http.agent.name snagel
• 2020/08/16 [nutch] 26/35: NUTCH-2799 Add .asf.yaml file - add project description in one sentence - add github topics - set github mailing list notifications as configured before snagel
• 2020/08/16 [nutch] 27/35: NUTCH-2799 Add .asf.yaml file - update pull request template regarding Jira linking: issue id should be in square brackets (`[NUTCH-XXXX]`) snagel
• 2020/08/16 [nutch] 07/35: NUTCH-2758 Add plugin READMEs to binary release packages snagel
• 2020/08/16 [nutch] 02/35: NUTCH-2434 Add methods to reset parameters HTMLMetaTags (apply patch contributed by Markus) snagel
• 2020/08/16 [nutch] 23/35: [NUTCH-2730] SitemapProcessor to treat sitemap URLs as Set instead of List - sitemap links from robots.txt are treated as set by crawler-commons (since crawler-commons 1.1) - sitemaps referenced in sitemap index are deduplicated snagel
• 2020/08/16 [nutch] 31/35: NUTCH-2810 FreeGenerator to actually apply configured number of fetch lists snagel
• 2020/08/16 [nutch] 15/35: NUTCH-2787 CrawlDb JSON dump does not export metadata primitive data types correctly - add JsonSerializer to write common Writable types (null, boolean, numbers) - remaining "unknown" Writables are written after calling toString() snagel
• 2020/08/16 [nutch] 06/35: NUTCH-2753 Add -listen option to command-line help of CrawlDbReader and LinkDbReader snagel
• 2020/08/16 [nutch] 18/35: NUTCH-2789 Documentation: update links to point to cwiki snagel
• 2020/08/16 [nutch] 20/35: NUTCH-2794 Add additional ciphers to HTTP base's default cipher suite snagel
• 2020/08/16 [nutch] 05/35: NUTCH-2002 parse and index checkers to check robots.txt - applied Julien's patch to recent code base - also check redirects whether they are allowed - add command-line parameter `-checkRobotsTxt` enabling this check snagel
• 2020/08/16 [nutch] 29/35: [NUTCH-2801] RobotsRulesParser command-line checker to use http.robots.agents as fall-back - if no agent names are given as command-line arguments use values of http.agent.name and http.robots.agents as agent names to be checked - update command-line help snagel
• 2020/08/16 [nutch] 28/35: NUTCH-1190 MoreIndexingFilter: move data formats used to parse "lastModified" to a config file snagel
• 2020/08/16 [nutch] 10/35: NUTCH-2419 Some URL filters and normalizers do not respect command-line override for rule file snagel
• 2020/08/16 [nutch] 16/35: NUTCH-2788 ParseData: improve presentation of Metadata in method toString() - switch to multi-line presentation of Metadata in ParseData::toString - default implementation of Metadata::toString is still single-line - replace StringBuffer by StringBuilder in modified methods snagel
• 2020/08/16 [nutch] 01/35: NUTCH-2743 Add list of Nutch properties (nutch-default.xml) to documentation - modify ant build.xml to copy nutch-default.xml into docs/api/resources/ - adapt XSLT table layout - remove obsolete nutch-conf.xsl - fix typos and normalize spelling in nutch-default.xml snagel
• 2020/08/16 [nutch] 14/35: NUTCH-2790 indexer-csv: escape field leading quote character snagel
• 2020/08/16 [nutch] 24/35: NUTCH-2782: protocol-http / lib-http: support TLSv1.3 snagel
• 2020/08/16 [nutch] 17/35: NUTCH-2789 Docker README: update links to point to cwiki snagel
• 2020/08/16 [nutch] 25/35: NUTCH-2805: Rename plugin urlfilter-domainblacklist (#540) snagel
• 2020/08/16 [nutch] 03/35: NUTCH-1194 Generator: CrawlDB lock should be released earlier - release CrawlDb lock after select step, in case, generated items are not marked in CrawlDb (generate.update.crawldb is false) snagel
• 2020/08/16 [nutch] 32/35: NUTCH-2811 : Setup Github workflows for prs (#543) snagel
• 2020/08/16 [nutch] 21/35: Prepare for new development after release of 1.17 - bump version number (1.17-SNAPSHOT -> 1.18-SNAPSHOT) - add 1.17 changes / release notes - update links to Hadoop and Solr API docs - update current year in API docs etc. snagel
• 2020/08/16 [nutch] 22/35: [NUTCH-2796] Upgrade to crawler-commons 1.1 snagel
• 2020/08/16 [nutch] 08/35: NUTCH-1945 Test for XLSX parser - add Tika unit test for XLSX files - bundle instance variables and utility methods in class TikaParserTest - clean up javadoc comments snagel
• 2020/08/16 [nutch] 09/35: NUTCH-2419 Some URL filters and normalizers do not respect command-line override for rule file snagel
• 2020/08/16 [nutch] 33/35: NUTCH-2816 Add Spotbugs target to ant build - called on-demand as ant target "spotbugs" - creates spotbugs report ("build/nutch-spotbugs.html") covering Nutch core and plugins snagel
• 2020/08/16 [nutch] 12/35: NUTCH-2720 ROBOTS metatag ignored when capitalized - move string "robots" to constant in metadata.Nutch - make string lowercase not depend on system locale snagel

• Earlier messages
• Later messages