[
https://issues.apache.org/jira/browse/HBASE-23175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954609#comment-16954609
]
Josh Elser commented on HBASE-23175:
------------------------------------
bq. Will raise an amendment ticket for SPARK-26432 separately as the new change
is relying on the API which is deprecated recently.
The change looks straightforward enough and reasonable for us to make to
improve our spark compatibility.
Thinking forward to release notes, do you have an idea of what versions of
Spark 2 would be incompatible with HBase 2.x? Is it all Spark2.x against all
currently-release HBase 2.x?
> Yarn unable to acquire delegation token for HBase Spark jobs
> ------------------------------------------------------------
>
> Key: HBASE-23175
> URL: https://issues.apache.org/jira/browse/HBASE-23175
> Project: HBase
> Issue Type: Bug
> Components: hbase-connectors
> Affects Versions: 2.0.0
> Reporter: Ankit Singhal
> Assignee: Ankit Singhal
> Priority: Major
> Attachments: HBASE-23175.master.001.patch
>
>
> Spark rely on the TokenUtil.obtainToken(conf) API which is removed in
> HBase-2.0, though it has been fixed in SPARK-26432 to use the new API but
> planned for Spark-3.0, hence we need the fix in HBase until they release it
> and we upgrade it
> {code}
> 18/03/20 20:39:07 ERROR ApplicationMaster: User class threw exception:
> org.apache.hadoop.hbase.HBaseIOException:
> com.google.protobuf.ServiceException: Error calling method
> hbase.pb.AuthenticationService.GetAuthenticationToken
> org.apache.hadoop.hbase.HBaseIOException:
> com.google.protobuf.ServiceException: Error calling method
> hbase.pb.AuthenticationService.GetAuthenticationToken
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:360)
> at
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:346)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:86)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil$1.run(TokenUtil.java:121)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil$1.run(TokenUtil.java:118)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
> at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:118)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:272)
> at
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:533)
> at
> org.apache.hadoop.hbase.spark.HBaseContext.<init>(HBaseContext.scala:73)
> at
> org.apache.hadoop.hbase.spark.JavaHBaseContext.<init>(JavaHBaseContext.scala:46)
> at
> org.apache.hadoop.hbase.spark.example.hbasecontext.JavaHBaseBulkDeleteExample.main(JavaHBaseBulkDeleteExample.java:64)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.yarn.ApplicationMaster$$anon$4.run(ApplicationMaster.scala:706)
> Caused by: com.google.protobuf.ServiceException: Error calling method
> hbase.pb.AuthenticationService.GetAuthenticationToken
> at
> org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:71)
> at
> org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService$BlockingStub.getAuthenticationToken(AuthenticationProtos.java:4512)
> at
> org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:81)
> ... 17 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
