[
https://issues.apache.org/jira/browse/SOLR-7948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704864#comment-14704864
]
Mark Miller commented on SOLR-7948:
-----------------------------------
Thanks for the report.
I actually ran into this issue a couple weeks ago while trying to get the map
reduce contrib back up to speed.
You can see this issue when you try and run the example:
https://github.com/markrmiller/solr-map-reduce-example
I think the issue is that a Kite Morphlines jar is using a Solr class that has
changed. If so, the answer is that Kite Morphlines should not use Solr classes
outside of it's couple Solr modules. I'll look into getting that changed very
soon, and then we will have to update versions.
> MapReduceIndexerTool of solr 5.2.1 doesn't work with hadoop 2.7.1
> -----------------------------------------------------------------
>
> Key: SOLR-7948
> URL: https://issues.apache.org/jira/browse/SOLR-7948
> Project: Solr
> Issue Type: Bug
> Components: contrib - MapReduce
> Affects Versions: 5.2.1
> Environment: OS:suse 11
> JDK:java version "1.7.0_65"
> Java(TM) SE Runtime Environment (build 1.7.0_65-b17)
> Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
> HADOOP:hadoop 2.7.1
> SOLR:5.2.1
> Reporter: davidchiu
>
> When I used MapReduceIndexerTool of 5.2.1 to index files, I got follwoing
> errors,but I used 4.9.0's MapReduceIndexerTool, it did work with hadoop 2.7.1.
> Exception ERROR as following:
> INFO - 2015-08-20 11:44:45.155; [ ] org.apache.solr.hadoop.HeartBeater;
> Heart beat reporting class is
> org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl
> INFO - 2015-08-20 11:44:45.161; [ ]
> org.apache.solr.hadoop.SolrRecordWriter; Using this unpacked directory as
> solr home:
> /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip
> INFO - 2015-08-20 11:44:45.162; [ ]
> org.apache.solr.hadoop.SolrRecordWriter; Creating embedded Solr server with
> solrHomeDir:
> /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip,
> fs:
> DFS[DFSClient[clientName=DFSClient_attempt_1440040092614_0004_r_000001_0_1678264055_1,
> ugi=root (auth:SIMPLE)]], outputShardDir:
> hdfs://127.0.0.1:9000/tmp/outdir/reducers/_temporary/1/_temporary/attempt_1440040092614_0004_r_000001_0/part-r-00001
> INFO - 2015-08-20 11:44:45.194; [ ]
> org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for
> directory:
> '/usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip/'
> INFO - 2015-08-20 11:44:45.206; [ ] org.apache.solr.hadoop.HeartBeater;
> HeartBeat thread running
> INFO - 2015-08-20 11:44:45.207; [ ] org.apache.solr.hadoop.HeartBeater;
> Issuing heart beat for 1 threads
> INFO - 2015-08-20 11:44:45.418; [ ]
> org.apache.solr.hadoop.SolrRecordWriter; Constructed instance information
> solr.home
> /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip
>
> (/usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip),
> instance dir
> /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip/,
> conf dir
> /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip/conf/,
> writing index to solr.data.dir
> hdfs://127.0.0.1:9000/tmp/outdir/reducers/_temporary/1/_temporary/attempt_1440040092614_0004_r_000001_0/part-r-00001/data,
> with permdir
> hdfs://127.0.0.10:9000/tmp/outdir/reducers/_temporary/1/_temporary/attempt_1440040092614_0004_r_000001_0/part-r-00001
> INFO - 2015-08-20 11:44:45.426; [ ] org.apache.solr.core.SolrXmlConfig;
> Loading container configuration from
> /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip/solr.xml
> INFO - 2015-08-20 11:44:45.474; [ ]
> org.apache.solr.core.CorePropertiesLocator; Config-defined core root
> directory:
> /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip
> INFO - 2015-08-20 11:44:45.503; [ ] org.apache.solr.core.CoreContainer;
> New CoreContainer 1656436773
> INFO - 2015-08-20 11:44:45.503; [ ] org.apache.solr.core.CoreContainer;
> Loading cores into CoreContainer
> [instanceDir=/usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip/]
> INFO - 2015-08-20 11:44:45.503; [ ] org.apache.solr.core.CoreContainer;
> loading shared library:
> /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip/lib
> WARN - 2015-08-20 11:44:45.504; [ ]
> org.apache.solr.core.SolrResourceLoader; Can't find (or read) directory to
> add to classloader: lib (resolved as:
> /usr/local/hadoop/tmp/nm-local-dir/usercache/root/appcache/application_1440040092614_0004/container_1440040092614_0004_01_000004/82f2eca9-d6eb-483b-960f-0d3b3b93788c.solr.zip/lib).
> INFO - 2015-08-20 11:44:45.520; [ ]
> org.apache.solr.handler.component.HttpShardHandlerFactory; created with
> socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost :
> 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize :
> 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy :
> false,useRetries : false,
> FATAL - 2015-08-20 11:44:45.526; [ ] org.apache.hadoop.mapred.YarnChild;
> Error running child : java.lang.VerifyError: Bad return type
> Exception Details:
> Location:
>
> org/apache/solr/client/solrj/impl/HttpClientUtil.createClient(Lorg/apache/solr/common/params/SolrParams;Lorg/apache/http/conn/ClientConnectionManager;)Lorg/apache/http/impl/client/CloseableHttpClient;
> @62: areturn
> Reason:
> Type 'org/apache/http/impl/client/DefaultHttpClient' (current frame,
> stack[0]) is not assignable to
> 'org/apache/http/impl/client/CloseableHttpClient' (from method signature)
> Current Frame:
> bci: @62
> flags: { }
> locals: { 'org/apache/solr/common/params/SolrParams',
> 'org/apache/http/conn/ClientConnectionManager',
> 'org/apache/solr/common/params/ModifiableSolrParams',
> 'org/apache/http/impl/client/DefaultHttpClient' }
> stack: { 'org/apache/http/impl/client/DefaultHttpClient' }
> Bytecode:
> 0000000: bb00 0359 2ab7 0004 4db2 0005 b900 0601
> 0000010: 0099 001e b200 05bb 0007 59b7 0008 1209
> 0000020: b600 0a2c b600 0bb6 000c b900 0d02 00bb
> 0000030: 0011 592b b700 124e 2d2c b800 102d b0
> Stackmap Table:
> append_frame(@47,Object[#127])
> at
> org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:166)
> at
> org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:49)
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:328)
> at
> org.apache.solr.hadoop.SolrRecordWriter.createEmbeddedSolrServer(SolrRecordWriter.java:163)
> at
> org.apache.solr.hadoop.SolrRecordWriter.<init>(SolrRecordWriter.java:119)
> at
> org.apache.solr.hadoop.SolrOutputFormat.getRecordWriter(SolrOutputFormat.java:163)
> at
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.<init>(ReduceTask.java:540)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:614)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
