Prajeeth Emanuel created NUTCH-2706: ---------------------------------------
Summary: -addBinaryContent flag can cause "String length must be a multiple of four" error in IndexingJob Key: NUTCH-2706 URL: https://issues.apache.org/jira/browse/NUTCH-2706 Project: Nutch Issue Type: Bug Components: indexer Affects Versions: 1.15 Environment: Solr:7.3.1 Nutch: 1.15 Reporter: Prajeeth Emanuel When using the following crawling command: bin/crawl -i -s /user/neutron_proactive_sourcing/seed /user/neutron_proactive_sourcing/test-crawl-8 3 with the index command in the crawl script with -addBinaryContent and -base64. The error I get is: 2019-04-04 04:10:43,702 svnNumber= clientHw="" userId="" actionKpi="" [main] WARN org.apache.hadoop.mapred.YarnChild - Exception running child : org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://emissary-pie-search-mapsneutrondev.usspk05.app.apple.com/solr: Error from server at http://17.140.85.63:8986/solr/maps_neutron.proactive-sourcing-crawl_shard1_replica1: ERROR: [doc=73ad5e05e49054efa258e7c54ae9b9ee] Error adding field 'binaryContent'='PCFET0NUWVBFIGh0bWw+DQo8aHRtbCBsYW5nPSJlbiI+DQo8aGVhZD4NCgk8bWV0YSBodHRwLWVx... ... msg=String length must be a multiple of four. at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:559) at com.apple.pie.search.client.InternalSearchClient.executeMethod(InternalSearchClient.java:66) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:233) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:225) at com.apple.pie.search.client.InternalSearchClient.request(InternalSearchClient.java:83) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135) at com.apple.pie.search.client.PieSearchUpdateRequest.process(PieSearchUpdateRequest.java:31) at com.apple.geo.neutron.nutch.plugins.indexwriter.PieSolrIndexWriter.push(PieSolrIndexWriter.java:72) at com.apple.geo.neutron.nutch.plugins.indexwriter.PieSolrIndexWriter.commit(PieSolrIndexWriter.java:97) at org.apache.nutch.indexer.IndexWriters.commit(IndexWriters.java:251) at org.apache.nutch.indexer.IndexerOutputFormat$1.close(IndexerOutputFormat.java:47) at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.close(ReduceTask.java:550) at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:629) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) I see this https://issues.apache.org/jira/browse/NUTCH-2186 as well. Opening a new ticket as mentioned in the comments because I have a different environment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)