+dev
Catalogd will still crash when loading catalog of huge tables with many partitions and files. Could we add an option 'catalog_mem_limit' just like 'exec_mem_limit' to prevent this? When memory used in catalog of a loading table exceed this limit, the catalogd should cancel the loading to protect itself from OOM. I think it's doable since I simply find this: https://stackoverflow.com/questions/52353/in-java-what-is-the-best-way-to-determine-the-size-of-an-object Thanks, Quanlong On 2017-12-11 10:56:21，"Vincent Tran" <vtt...@cloudera.com> wrote： Hi Yu Feng, You can try to start catalogd with the "-jvm_args" argument. Ref: https://github.com/apache/impala/blob/master/bin/start-catalogd.sh#L45 i.e. bin/start-catalogd.sh -jvm_args="-Xms4294967296 -Xmx4294967296" On Dec 10, 2017 9:18 PM, "yu feng" <olaptes...@gmail.com> wrote: how impala control the max memory to be allocated in JNICatalogd and JniFronted, I get java OOM in Catalogd, and doubt how to slove it。 Is there a way to change it in impala ？ maybe set JAVA_TOOL_OPTIONS environment variable is a choose, But how to do it in impala without modify code.
