[
https://issues.apache.org/jira/browse/NUTCH-2706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prajeeth Emanuel updated NUTCH-2706:
------------------------------------
Description:
When using the following crawling command:
bin/crawl -i -s /user/neutron_proactive_sourcing/seed
/user/neutron_proactive_sourcing/test-crawl-8 3
with the index command in the crawl script with -addBinaryContent and -base64.
The error I get is:
2019-04-04 04:10:43,702 svnNumber= clientHw="" userId="" actionKpi="" [main]
WARN org.apache.hadoop.mapred.YarnChild - Exception running child :
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: ERROR:
[doc=73ad5e05e49054efa258e7c54ae9b9ee] Error adding field
'binaryContent'='PCFET0NUWVBFIGh0bWw+DQo8aHRtbCBsYW5nPSJlbiI+DQo8aGVhZD4NCgk8bWV0YSBodHRwLWVx...
...
msg=String length must be a multiple of four. at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:559)
at at org.apache.nutch.indexer.IndexWriters.commit(IndexWriters.java:251) at
org.apache.nutch.indexer.IndexerOutputFormat$1.close(IndexerOutputFormat.java:47)
at
org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.close(ReduceTask.java:550)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:629) at
org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
I see this https://issues.apache.org/jira/browse/NUTCH-2186 as well. Opening a
new ticket as mentioned in the comments because I have a different environment.
was:
When using the following crawling command:
bin/crawl -i -s /user/neutron_proactive_sourcing/seed
/user/neutron_proactive_sourcing/test-crawl-8 3
with the index command in the crawl script with -addBinaryContent and -base64.
The error I get is:
2019-04-04 04:10:43,702 svnNumber= clientHw="" userId="" actionKpi="" [main]
WARN org.apache.hadoop.mapred.YarnChild - Exception running child :
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
from server at
https://emissary-pie-search-mapsneutrondev.usspk05.app.apple.com/solr: Error
from server at
http://17.140.85.63:8986/solr/maps_neutron.proactive-sourcing-crawl_shard1_replica1:
ERROR: [doc=73ad5e05e49054efa258e7c54ae9b9ee] Error adding field
'binaryContent'='PCFET0NUWVBFIGh0bWw+DQo8aHRtbCBsYW5nPSJlbiI+DQo8aGVhZD4NCgk8bWV0YSBodHRwLWVx...
...
msg=String length must be a multiple of four. at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:559)
at
com.apple.pie.search.client.InternalSearchClient.executeMethod(InternalSearchClient.java:66)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:233)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:225)
at
com.apple.pie.search.client.InternalSearchClient.request(InternalSearchClient.java:83)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135) at
com.apple.pie.search.client.PieSearchUpdateRequest.process(PieSearchUpdateRequest.java:31)
at
com.apple.geo.neutron.nutch.plugins.indexwriter.PieSolrIndexWriter.push(PieSolrIndexWriter.java:72)
at
com.apple.geo.neutron.nutch.plugins.indexwriter.PieSolrIndexWriter.commit(PieSolrIndexWriter.java:97)
at org.apache.nutch.indexer.IndexWriters.commit(IndexWriters.java:251) at
org.apache.nutch.indexer.IndexerOutputFormat$1.close(IndexerOutputFormat.java:47)
at
org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.close(ReduceTask.java:550)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:629) at
org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
I see this https://issues.apache.org/jira/browse/NUTCH-2186 as well. Opening a
new ticket as mentioned in the comments because I have a different environment.
> -addBinaryContent flag can cause "String length must be a multiple of four"
> error in IndexingJob
> ------------------------------------------------------------------------------------------------
>
> Key: NUTCH-2706
> URL: https://issues.apache.org/jira/browse/NUTCH-2706
> Project: Nutch
> Issue Type: Bug
> Components: indexer
> Affects Versions: 1.15
> Environment: Solr:7.3.1
> Nutch: 1.15
> Reporter: Prajeeth Emanuel
> Priority: Major
>
> When using the following crawling command:
> bin/crawl -i -s /user/neutron_proactive_sourcing/seed
> /user/neutron_proactive_sourcing/test-crawl-8 3
> with the index command in the crawl script with -addBinaryContent and -base64.
> The error I get is:
> 2019-04-04 04:10:43,702 svnNumber= clientHw="" userId="" actionKpi="" [main]
> WARN org.apache.hadoop.mapred.YarnChild - Exception running child :
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: ERROR:
> [doc=73ad5e05e49054efa258e7c54ae9b9ee] Error adding field
> 'binaryContent'='PCFET0NUWVBFIGh0bWw+DQo8aHRtbCBsYW5nPSJlbiI+DQo8aGVhZD4NCgk8bWV0YSBodHRwLWVx...
>
> ...
>
> msg=String length must be a multiple of four. at
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:559)
> at at org.apache.nutch.indexer.IndexWriters.commit(IndexWriters.java:251)
> at
> org.apache.nutch.indexer.IndexerOutputFormat$1.close(IndexerOutputFormat.java:47)
> at
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.close(ReduceTask.java:550)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:629) at
> org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
>
> I see this https://issues.apache.org/jira/browse/NUTCH-2186 as well. Opening
> a new ticket as mentioned in the comments because I have a different
> environment.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
