This is due to the fact that starting a hive server 2 is using kerberos principle different from what you used to submit the spark task. This happens when the hive.xml file have some specification on kerberos. In this case, make sure that keytab address and principal are both described in hive.xml file.
Sent from Outlook<http://aka.ms/weboutlook> ________________________________ From: 冉明轩 <[email protected]> Sent: Wednesday, July 11, 2018 5:31 AM To: [email protected]; [email protected] Subject: Got kerberos login error while starting CarbonData Thrift Server Hello all, I am new to carbondata and just about to start a CarbonData Thrift Server to have a try on its through put. Environment: Cluster: Cloudera CDH5.5.3 with kerberos installed as principle + keytab style Spark: 2.2.1 Carbondata: apache-carbondata-1.4.0-bin-spark2.2.1-hadoop2.7.2.jar Process: 1. build carbondata by following https://github.com/apache/carbondata/blob/master/build/README.md 2. Install and configure carbondata on Spark on Yarn Cluster by following http://carbondata.apache.org/installation-guide.html 3. create file /home/mingxuan/spark-2.2.1-bin-hadoop2.7/bin/carbon-thrift.sh: #!/bin/bash /home/mingxuan/spark-2.2.1-bin-hadoop2.7/bin/spark-submit \ --class org.apache.carbondata.spark.thriftserver.CarbonThriftServer \ --principal mingxuan/[email protected] \ --keytab /home/mingxuan/mingxuan.keytab \ --queue root.mingxuan \ --master yarn \ --deploy-mode client \ --driver-memory 8g \ --executor-cores 4 \ --executor-memory 10g \ --num-executors 10 \ /home/mingxuan/spark-2.2.1-bin-hadoop2.7/carbonlib/apache-carbondata-1.4.0-bin-spark2.2.1-hadoop2.7.2.jar hdfs://nsha/user/hive/warehouse/ranmx.db/carbondata 4. cd to spark dir and run command: sh /home/mingxuan/spark-2.2.1-bin-hadoop2.7/bin/carbon-thrift.sh Error Log: 18/07/10 16:08:33 INFO HiveClientImpl: Warehouse location for Hive client (version 1.2.1) is /user/hive/warehouse Exception in thread "main" org.apache.hive.service.ServiceException: Unable to login to kerberos with given principal/keytab at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIService.init(SparkSQLCLIService.scala:58) at org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$$anonfun$initCompositeService$1.apply(SparkSQLCLIService.scala:79) at org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$$anonfun$initCompositeService$1.apply(SparkSQLCLIService.scala:79) at scala.collection.Iterator$class.foreach(Iterator.scala:893) at scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$class.initCompositeService(SparkSQLCLIService.scala:79) at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.initCompositeService(HiveThriftServer2.scala:272) at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.init(HiveThriftServer2.scala:292) at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.startWithContext(HiveThriftServer2.scala:64) at org.apache.carbondata.spark.thriftserver.CarbonThriftServer$.main(CarbonThriftServer.scala:93) at org.apache.carbondata.spark.thriftserver.CarbonThriftServer.main(CarbonThriftServer.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:775) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.io.IOException: HiveServer2 Kerberos principal or keytab is not correctly configured at org.apache.hive.service.auth.HiveAuthFactory.loginFromKeytab(HiveAuthFactory.java:197) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIService.init(SparkSQLCLIService.scala:53) ... 21 more Does anybody has any idea about this exception? Does hadoop 2.6 matters? Ran Mingxuan
