Nope, in my case I see it pretty much as soon as the beeline client issues the connect statement. Queries I've run so far are of the "show table" variety and also "select count(*) from mytable" -- i.e. nothing that serializes large amounts of data
On Thu, Nov 12, 2015 at 7:44 PM, Cheng, Hao <hao.ch...@intel.com> wrote: > OOM can occurs in any place, if most of memory is used by some of the > `defect`, the exception stack probably doesn’t show the real problem. > > > > Most of the this occurs in ThriftServer as I know, people are trying to > collect a huge result set, can you confirm that? If it fall into this > category, probably you can set the > “spark.sql.thriftServer.incrementalCollect” to false; > > > > Hao > > > > *From:* Yana Kadiyska [mailto:yana.kadiy...@gmail.com] > *Sent:* Friday, November 13, 2015 8:30 AM > *To:* user@spark.apache.org > *Subject:* HiveServer2 Thrift OOM > > > > Hi folks, I'm starting a HiveServer2 from a HiveContext > (HiveThriftServer2.startWithContext(hiveContext)) and then connecting to > it via beenline. On the server side, I see the below error which I think > is related to https://issues.apache.org/jira/browse/HIVE-6468 > > > > But I'd like to know: > > > > 1. why I see it (I'm using bin/beeline from spark to connect, not http) > > 2. should I be dropping any hive-site or hive-default files in conf/ -- > Hive-6468 talks about *hive.server2.sasl.message.limit *but I can't see > any documentation on where this setting would go or what's a reasonable > value (im trying to do a light-weight deployment and have not needed > hive-site.xml so far...) > > > > Advice on how to get rid of the below exception much appreciated > > > > > > Exception in thread "pool-17-thread-2" java.lang.OutOfMemoryError: Java heap > space > > at > org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:181) > > at > org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125) > > at > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253) > > at > org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) > > at > org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) > > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:189) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > > E > > >
