Hello all,
I am new to carbondata and just about to start a CarbonData Thrift Server to have a try on its through put. Environment: Cluster: Cloudera CDH5.5.3 with kerberos installed as principle + keytab style Spark: 2.2.1 Carbondata: apache-carbondata-1.4.0-bin-spark2.2.1-hadoop2.7.2.jar Process: 1. build carbondata by following https://github.com/apache/carbondata/blob/master/build/README.md 2. Install and configure carbondata on Spark on Yarn Cluster by following http://carbondata.apache.org/installation-guide.html 3. create file /home/mingxuan/spark-2.2.1-bin-hadoop2.7/bin/carbon-thrift.sh: #!/bin/bash /home/mingxuan/spark-2.2.1-bin-hadoop2.7/bin/spark-submit \ --class org.apache.carbondata.spark.thriftserver.CarbonThriftServer \ --principal mingxuan/[email protected] \ --keytab /home/mingxuan/mingxuan.keytab \ --queue root.mingxuan \ --master yarn \ --deploy-mode client \ --driver-memory 8g \ --executor-cores 4 \ --executor-memory 10g \ --num-executors 10 \ /home/mingxuan/spark-2.2.1-bin-hadoop2.7/carbonlib/apache-carbondata-1.4.0-bin-spark2.2.1-hadoop2.7.2.jar hdfs://nsha/user/hive/warehouse/ranmx.db/carbondata 4. cd to spark dir and run command: sh /home/mingxuan/spark-2.2.1-bin-hadoop2.7/bin/carbon-thrift.sh Error Log: 18/07/10 16:08:33 INFO HiveClientImpl: Warehouse location for Hive client (version 1.2.1) is /user/hive/warehouse Exception in thread "main" org.apache.hive.service.ServiceException: Unable to login to kerberos with given principal/keytab at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIService.init(SparkSQLCLIService.scala:58) at org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$$anonfun$initCompositeService$1.apply(SparkSQLCLIService.scala:79) at org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$$anonfun$initCompositeService$1.apply(SparkSQLCLIService.scala:79) at scala.collection.Iterator$class.foreach(Iterator.scala:893) at scala.collection.AbstractIterator.foreach(Iterator.scala:1336) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at org.apache.spark.sql.hive.thriftserver.ReflectedCompositeService$class.initCompositeService(SparkSQLCLIService.scala:79) at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.initCompositeService(HiveThriftServer2.scala:272) at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.init(HiveThriftServer2.scala:292) at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.startWithContext(HiveThriftServer2.scala:64) at org.apache.carbondata.spark.thriftserver.CarbonThriftServer$.main(CarbonThriftServer.scala:93) at org.apache.carbondata.spark.thriftserver.CarbonThriftServer.main(CarbonThriftServer.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:775) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.io.IOException: HiveServer2 Kerberos principal or keytab is not correctly configured at org.apache.hive.service.auth.HiveAuthFactory.loginFromKeytab(HiveAuthFactory.java:197) at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIService.init(SparkSQLCLIService.scala:53) ... 21 more Does anybody has any idea about this exception? Does hadoop 2.6 matters? Ran Mingxuan
