Hi, (Refer my same question in stackoverflow <https://stackoverflow.com/questions/49270419/hiveserver2-is-shutting-down-frequently-in-hadoop-cluster> ) favorite <https://stackoverflow.com/questions/49270419/hiveserver2-is-shutting-down-frequently-in-hadoop-cluster#>Facing this issue from quite sometime now and not able to track the reason why is it happening. Whenever we start hiveserver2 using command ->
./hiveserver2 & It starts and stays up for sometime but then shuts down. In hive logs it does show the following error while hiveserver is up and running. 2018-03-12 04:44:57,029 ERROR [HiveServer2-Handler-Pool: Thread-33]: server.TThreadPoolServer (TThreadPoolServer.java:run(296)) - Erro r occurred during processing of message. java.lang.RuntimeException: org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:328) at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) ... 4 more 2018-03-12 04:45:55,361 INFO [main]: SessionState (SessionState.java:printInfo(951)) - Logging initialized using configuration in file:/usr/local/hive/conf/hive-log4j.properties But I'm not really sure that the shutting down of hiveserver is due to above error as it keeps on running for hours before shutting down. Following are the hive logs that comes when hiveserver shuts down 2018-03-12 04:46:25,285 INFO [main]: ql.Driver (SessionState.java:printInfo(951)) - Stage-Stage-1: Map: 4 Reduce: 1 Cumulative CPU : 18.09 sec HDFS Read: 763046 HDFS Write: 2217 SUCCESS 2018-03-12 04:46:25,286 INFO [main]: ql.Driver (SessionState.java:printInfo(951)) - Total MapReduce CPU Time Spent: 18 seconds 90 mse c 2018-03-12 04:46:25,286 INFO [main]: ql.Driver (SessionState.java:printInfo(951)) - OK 2018-03-12 04:46:25,286 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=releaseLocks from=org.apach e.hadoop.hive.ql.Driver> 2018-03-12 04:46:25,295 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=releaseLocks start=152082998 5286 end=1520829985295 duration=9 from=org.apache.hadoop.hive.ql.Driver> 2018-03-12 04:46:25,295 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=Driver.run start=15208299614 77 end=1520829985295 duration=23818 from=org.apache.hadoop.hive.ql.Driver> 2018-03-12 04:46:25,304 INFO [main]: CliDriver (SessionState.java:printInfo(951)) - Time taken: 23.818 seconds 2018-03-12 04:46:25,304 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=releaseLocks from=org.apach e.hadoop.hive.ql.Driver> 2018-03-12 04:46:25,305 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=releaseLocks start=152082998 5304 end=1520829985305 duration=1 from=org.apache.hadoop.hive.ql.Driver> 2018-03-12 04:46:36,351 INFO [Thread-9]: server.HiveServer2 (HiveServer2.java:stop(305)) - Shutting down HiveServer2 2018-03-12 04:46:36,351 INFO [Thread-9]: thrift.ThriftCLIService (ThriftCLIService.java:stop(201)) - Thrift server has stopped 2018-03-12 04:46:36,351 INFO [Thread-9]: service.AbstractService (AbstractService.java:stop(125)) - Service:ThriftBinaryCLIService is stopped. 2018-03-12 04:46:36,351 INFO [Thread-9]: service.AbstractService (AbstractService.java:stop(125)) - Service:OperationManager is stopp ed. 2018-03-12 04:46:36,351 INFO [Thread-9]: service.AbstractService (AbstractService.java:stop(125)) - Service:SessionManager is stopped . 2018-03-12 04:46:36,351 INFO [Thread-3]: server.HiveServer2 (HiveStringUtils.java:run(709)) - SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down HiveServer2 at ADG-HADOOP-MASTER/192.168.128.31 ************************************************************/ 2018-03-12 04:46:46,352 WARN [Thread-9]: service.CompositeService (SessionManager.java:cleanupLoggingRootDir(213)) - Failed to cleanu p root dir of HS2 logging: /usr/local/hive/log java.io.FileNotFoundException: File does not exist: /usr/local/hive/log at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2275) at org.apache.hive.service.cli.session.SessionManager.cleanupLoggingRootDir(SessionManager.java:211) at org.apache.hive.service.cli.session.SessionManager.stop(SessionManager.java:205) at org.apache.hive.service.CompositeService.stop(CompositeService.java:102) at org.apache.hive.service.CompositeService.stop(CompositeService.java:92) at org.apache.hive.service.cli.CLIService.stop(CLIService.java:165) at org.apache.hive.service.CompositeService.stop(CompositeService.java:102) at org.apache.hive.service.CompositeService.stop(CompositeService.java:92) at org.apache.hive.service.server.HiveServer2.stop(HiveServer2.java:307) at org.apache.hive.service.server.HiveServer2$1.run(HiveServer2.java:107) 2018-03-12 04:46:46,353 INFO [Thread-9]: service.AbstractService (AbstractService.java:stop(125)) - Service:CLIService is stopped. 2018-03-12 04:46:46,353 INFO [Thread-9]: service.AbstractService (AbstractService.java:stop(125)) - Service:HiveServer2 is stopped. 2018-03-12 04:51:07,336 INFO [main]: SessionState (SessionState.java:printInfo(951)) - Logging initialized using configuration in file:/usr/local/hive/conf/hive-log4j.properties If the issue is actually because of... ERROR [HiveServer2-Handler-Pool: Thread-33]: server.TThreadPoolServer (TThreadPoolServer.java:run(296)) - Erro r occurred during processing of message. java.lang.RuntimeException: org.apache.thrift.transport.TSaslTransportException: No data or no sasl data in the stream ...then here are my hive-site.xml settings which are related to it as mentioned in many other related posts. <name>hive.server2.authentication</name> <value>PAM</value> <name>hive.server2.authentication.pam.services</name> <value>sshd,sudo</value> <name>hive.server2.thrift.sasl.qop</name> <value>auth</value> <name>hive.metastore.sasl.enabled</name> <value>false</value> Appreciate some help here, Thanks.