[
https://issues.apache.org/jira/browse/CRUNCH-249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Shi updated CRUNCH-249:
----------------------------
Attachment: crunch-249-v2.diff
The second problem is due to HBASE-6359 (fixed in 0.94.2). So the solution is
simple: upgrade it.
It still has some problem. The pipeline fails with the following exception. It
seems like the pipeline succeeded actually, but it encountered an exception
when getting counters.
Caused by: com.google.protobuf.ServiceException: java.net.ConnectException:
Call From localhost.localdomain/127.0.0.1 to 0.0.0.0:10020 failed on connection
exception: java.net.ConnectException: Connection refused; For more details see:
http://wiki.apache.org/hadoop/ConnectionRefused
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:212)
at com.sun.proxy.$Proxy89.getCounters(Unknown Source)
at
org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getCounters(MRClientProtocolPBClientImpl.java:167)
... 17 more
Caused by: java.net.ConnectException: Call From localhost.localdomain/127.0.0.1
to 0.0.0.0:10020 failed on connection exception: java.net.ConnectException:
Connection refused; For more details see:
http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:780)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:727)
at org.apache.hadoop.ipc.Client.call(Client.java:1239)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202)
... 19 more
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:526)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:490)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:508)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:603)
at org.apache.hadoop.ipc.Client$Connection.access$2100(Client.java:253)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1288)
at org.apache.hadoop.ipc.Client.call(Client.java:1206)
... 20 more
196001 [Thread-614] ERROR org.apache.hadoop.security.UserGroupInformation -
PriviledgedActionException as:shichao3 (auth:SIMPLE) cause:java.io.IOException
196001 [Thread-614] ERROR org.apache.crunch.impl.mr.exec.MRExecutor - Pipeline
failed due to exception
java.io.IOException
at
org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:317)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobCounters(ClientServiceDelegate.java:337)
at org.apache.hadoop.mapred.YARNRunner.getJobCounters(YARNRunner.java:515)
at org.apache.hadoop.mapreduce.Job$8.run(Job.java:710)
at org.apache.hadoop.mapreduce.Job$8.run(Job.java:707)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
at org.apache.hadoop.mapreduce.Job.getCounters(Job.java:707)
at
org.apache.crunch.impl.mr.exec.MRExecutor.monitorLoop(MRExecutor.java:115)
at org.apache.crunch.impl.mr.exec.MRExecutor.access$000(MRExecutor.java:52)
at org.apache.crunch.impl.mr.exec.MRExecutor$1.run(MRExecutor.java:76)
at java.lang.Thread.run(Thread.java:722)
> HFileTargetIT fails under hadoop2
> ---------------------------------
>
> Key: CRUNCH-249
> URL: https://issues.apache.org/jira/browse/CRUNCH-249
> Project: Crunch
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Josh Wills
> Assignee: Chao Shi
> Attachments: crunch-249-part1.patch, crunch-249-v2.diff
>
>
> The HFileTargetIT methods both fail with Hadoop2-enabled (running mvn clean
> install -Dcrunch.platform=2). The jobs seem to complete, but the assertions
> in the tests don't seem to hold.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
