The catalog is cached on all Impala daemons so you also need to increase the impalad heap size too. There's a brief explanation in the docs: https://impala.apache.org/docs/build/html/topics/impala_scalability.html (see the "Impact of Many Tables or Partitions on Impala Catalog Performance and Memory Usage" section).
On Sun, Dec 10, 2017 at 7:13 PM, Quanlong Huang <huang_quanl...@126.com> wrote: > +dev > > Catalogd will still crash when loading catalog of huge tables with many > partitions and files. > Could we add an option 'catalog_mem_limit' just like 'exec_mem_limit' to > prevent this? When memory used in catalog of a loading table exceed this > limit, the catalogd should cancel the loading to protect itself from OOM. > I think it's doable since I simply find this: https://stackoverflow. > com/questions/52353/in-java-what-is-the-best-way-to- > determine-the-size-of-an-object > > Thanks, > Quanlong > > On 2017-12-11 10:56:21，"Vincent Tran" <vtt...@cloudera.com> wrote： > > Hi Yu Feng, > > You can try to start catalogd with the "-jvm_args" argument. > Ref: https://github.com/apache/impala/blob/master/bin/start- > catalogd.sh#L45 > > i.e. > bin/start-catalogd.sh -jvm_args="-Xms4294967296 -Xmx4294967296" > > > On Dec 10, 2017 9:18 PM, "yu feng" <olaptes...@gmail.com> wrote: > >> how impala control the max memory to be allocated in JNICatalogd and >> JniFronted, I get java OOM in Catalogd, and doubt how to slove it。 >> >> Is there a way to change it in impala ？ maybe set JAVA_TOOL_OPTIONS >> environment variable is a choose, But how to do it in impala without modify >> code. >> > > > >
