On Wed, Sep 1, 2010 at 6:19 AM, shangan <shan...@corp.kaixin001.com> wrote: > I don't know why those especially the hiveserver take so much > memory(hiveserver almost take 2G even there's no task being excuted). Is > there any way to configure the parameters for those daemons ? > > 2010-09-01 > ________________________________ > shangan
Shagan, All the components you mention inherit some settings from hadoop, namely HADOOP_HEAPSIZE=2000m. I never was quite sure why HADOOP_HEAPSIZE was created. It is a oversimplified view of java memory management. HADOOP_HEAPSIZE is translated to XMX somewhere in the hadoop startup process. Ideally you want to keep HADOOP_HEAPSIZE large to handle large queries, but you can supply other variables to the jvm like -xms. Look at the scripts in <hive_home>/bin/ext. Now the real question is, IS JAVA ACTUALLY using all this heap. You can use jconsole to look into what the real memory usage is, if you find the actual usage is less then XMX you can instruct your OS to overcommit memory. Regards, Edward
