Thanks for the detailed report Bennie. There might be memory leak in jdbc or hiveserver itself. Are your queries roughly the same (in terms of query size and number of partitions involved)? Better off can you come up a simple test case (serious of queries) that expose the mem leak problem?
Thanks, Ning ------ Sent from my blackberry ________________________________ From: Bennie Schut <[email protected]> To: '[email protected]' <[email protected]> Sent: Thu Jun 03 02:20:38 2010 Subject: java.lang.OutOfMemoryError: PermGen space when running as a service. Hi guys, When I run hive as a service like this: "hive --service hiveserver" I get these errors after about a day of running a lot of queries: java.lang.OutOfMemoryError: PermGen space I use: trunk hive from about a week ago with "-XX:MaxPermSize=128m". hadoop 0.20.2 mysql 5.1.45 meta store Some UDF's on each query. I've now increased it to 512 to see if it helps. I've also made a jmap dump (85M) while it was broken 9931 instances of class org.apache.hadoop.hive.ql.exec.ColumnInfo 4466 instances of class org.apache.hadoop.hive.ql.plan.ExprNodeColumnDesc 3127 instances of class org.apache.hadoop.hive.metastore.api.FieldSchema 3127 instances of class org.apache.hadoop.hive.metastore.api.FieldSchema$Isset 2755 instances of class org.apache.commons.logging.impl.Log4JLogger 2497 instances of class org.datanucleus.util.WeakValueMap$WeakValueReference 1404 instances of class org.apache.hadoop.ipc.Client$Call 1404 instances of class org.apache.hadoop.ipc.RPC$Invocation 1394 instances of class org.apache.hadoop.ipc.RemoteException 985 instances of class [Lorg.datanucleus.plugin.ConfigurationElement; 945 instances of class org.datanucleus.plugin.ConfigurationElement 907 instances of class org.apache.hadoop.hive.ql.hooks.LineageInfo$BaseColumnInfo 792 instances of class [Lorg.apache.hadoop.hive.serde2.objectinspector.ObjectInspector; 767 instances of class org.apache.hadoop.hive.ql.hooks.LineageInfo$Dependency 730 instances of class org.antlr.runtime.BitSet 618 instances of class com.mysql.jdbc.ConnectionPropertiesImpl$BooleanConnectionProperty 523 instances of class org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$IdentityConverter 500 instances of class org.datanucleus.sco.backed.Map 464 instances of class org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge I'm not sure if this is what you expect to see or if something is sticking out here? Thanks in advance for any ideas on this. Bennie.
