[
https://issues.apache.org/jira/browse/HBASE-15225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15135624#comment-15135624
]
Ted Malaska commented on HBASE-15225:
-------------------------------------
This should be marked as involved Jira.
[~88.sanjay] don't use newAPIHadoopRDD use the functions defined in the
HBaseContext object which is defined in the hbaseSpark Module.
That will take care of all you Spark to HBase connection issues.
Documentation can be found here: https://hbase.apache.org/book.html#spark
> Connecting to HBase via newAPIHadoopRDD in PySpark gives
> org.apache.hadoop.hbase.client.RetriesExhaustedException
> ------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-15225
> URL: https://issues.apache.org/jira/browse/HBASE-15225
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.98.4
> Environment: spark 1.6.0 , Hbase 0.98.4, kerberos,
> hbase.rpc.protection set to authentication.
> Reporter: Sanjay Kumar
>
> Unable to read HBase table into Spark with hbase security authentication set
> to kerberos. Seeing the following error.
> : org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=31, exceptions:
> Thu Feb 04 22:01:55 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> Thu Feb 04 22:01:56 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> Thu Feb 04 22:01:56 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> Thu Feb 04 22:01:57 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> Thu Feb 04 22:01:59 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> Thu Feb 04 22:02:03 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> Thu Feb 04 22:02:13 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> Thu Feb 04 22:02:23 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> Thu Feb 04 22:02:34 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> Thu Feb 04 22:02:44 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> Thu Feb 04 22:03:04 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> Thu Feb 04 22:03:24 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> Thu Feb 04 22:03:44 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> Thu Feb 04 22:04:04 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> Thu Feb 04 22:04:24 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> Thu Feb 04 22:04:44 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> Thu Feb 04 22:05:04 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.IOException: Connection reset by peer
> .
> .
> .
> Thu Feb 04 22:09:46 CST 2016,
> org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da,
> java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed
> on local exception: java.io.EOFException
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:129)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
> at
> org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:282)
> at
> org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:187)
> at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:182)
> at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:109)
> at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:738)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:178)
> at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:82)
> at
> org.apache.hadoop.hbase.client.MetaScanner.allTableRegions(MetaScanner.java:282)
> at org.apache.hadoop.hbase.client.HTable.getRegionLocations(HTable.java:616)
> at
> org.apache.hadoop.hbase.util.RegionSizeCalculator.<init>(RegionSizeCalculator.java:79)
> at
> org.apache.hadoop.hbase.util.RegionSizeCalculator.<init>(RegionSizeCalculator.java:64)
> at
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:160)
> at org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:115)
> at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
> at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
> at scala.Option.getOrElse(Option.scala:120)
> at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
> at
> org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
> at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
> at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
> at scala.Option.getOrElse(Option.scala:120)
> at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
> at org.apache.spark.rdd.RDD$$anonfun$take$1.apply(RDD.scala:1277)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147)
> at
> org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108)
> at org.apache.spark.rdd.RDD.withScope(RDD.scala:306)
> at org.apache.spark.rdd.RDD.take(RDD.scala:1272)
> at org.apache.spark.api.python.SerDeUtil$.pairRDDToPython(SerDeUtil.scala:202)
> at org.apache.spark.api.python.PythonRDD$.newAPIHadoopRDD(PythonRDD.scala:530)
> at org.apache.spark.api.python.PythonRDD.newAPIHadoopRDD(PythonRDD.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:207)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: java.io.IOException: Call to
> d-767tfz1.target.com/10.66.241.13:60020 failed on local exception:
> java.io.EOFException
> at org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1484)
> at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1456)
> at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1656)
> at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1714)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:29900)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:308)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:164)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:59)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114)
> ... 42 more
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:392)
> at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1071)
> at org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
