[
https://issues.apache.org/jira/browse/IMPALA-6436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16675801#comment-16675801
]
ASF subversion and git services commented on IMPALA-6436:
---------------------------------------------------------
Commit f08642bf43102cf326f4f00e9b9e3536d6906b2c in impala's branch
refs/heads/master from [~tarmstr...@cloudera.com]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=f08642b ]
IMPALA-6436: exit instead of abort for catalog startup failure
Rename EXIT_WITH_EXC to ABORT_WITH_EXC to make the behaviour more
obvious at callsites.
Handle exceptions from Catalog constructor by logging the backtrace and
exiting cleanly, rather than aborting. This will prevent generation of a
coredump or minidump.
Testing:
Tested starting the catalogd locally without the HMS running and a
low connection timeout:
start-impala-cluster.py --catalogd_args=--initial_hms_cnxn_timeout_s=2
Confirmed that the backtrace was logged to catalogd.ERROR and that no
core or minidump was generated.
Change-Id: I4026dccb39843b847426112fc0fe9ba897e48dcc
Reviewed-on: http://gerrit.cloudera.org:8080/11871
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> Impala Catalog generates a core file / mini dump when the HMS is not available
> ------------------------------------------------------------------------------
>
> Key: IMPALA-6436
> URL: https://issues.apache.org/jira/browse/IMPALA-6436
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 2.10.0
> Reporter: Luis E Martinez-Poblete
> Assignee: Tim Armstrong
> Priority: Critical
> Labels: supportability
>
> Synopsis:
> =========
> Impala Catalog generates a core file / mini dump when the HMS is not
> available
> Problem:
> ========
> Catalog server created multiple Catalog core files. During the investigation
> it was determine that the cause of the core files was because the Hive Meta
> Store was not available and the option "Enable Core Dump" was enabled when
> starting the Impala service.
> Below is the back trace of the core file:
> #0 0x00007f72e93ee5d7 in raise () from /root/191729/slib/lib64/libc.so.6
> #1 0x00007f72e93efcc8 in abort () from /root/191729/slib/lib64/libc.so.6
> #2 0x0000000001ba5754 in google::DumpStackTraceAndExit() ()
> #3 0x0000000001b9c1cd in google::LogMessage::Fail() ()
> #4 0x0000000001b9da72 in google::LogMessage::SendToLog() ()
> #5 0x0000000001b9bba7 in google::LogMessage::Flush() ()
> #6 0x0000000001b9f16e in google::LogMessageFatal::~LogMessageFatal() ()
> #7 0x000000000083067e in impala::Catalog::(GetCatalogVersion (this=0x0,
> version=0x7ffc2aa6b750) at
> /usr/src/debug/impala-2.10.0-cdh5.13.1/be/src/catalog/catalog.cc:88
> #8 0x00000000008143c9 in impala::CatalogServer::Start() () at
> /usr/src/debug/impala-2.10.0-cdh5.13.1/be/src/catalog/catalog-server.cc:175
> The corresponding entries in the Catalog server log show the following fatal
> error:
> F0111 09:48:05.017491 14571 catalog.cc:76] java.lang.IllegalStateException:
> java.lang.RuntimeException: Unable to instantiate
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient
> at
> org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:99)
> at
> org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:72)
> at
> org.apache.impala.catalog.MetaStoreClientPool.initClients(MetaStoreClientPool.java:168)
> at org.apache.impala.catalog.Catalog.<init>(Catalog.java:103)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.<init>(CatalogServiceCatalog.java:163)
> at org.apache.impala.service.JniCatalog.<init>(JniCatalog.java:104)
> Caused by: java.lang.RuntimeException: Unable to instantiate
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient
> I was able to reproduce this issue. When the option "Enable Core Dump" is
> enable and the Hive Meta Store is not available, the system generates a core
> file. If the option "Enable Core Dump" is disabled, the system generates a
> mini dump.
> Crashing due to an error is not expected. Impala should fail in a more user
> friendly way.
> Reproduction case:
> ==================
> 1) Enable the option "Enable Core Dump" for the Impala service in CM.
> 2) Stop Hive and Impala services.
> 3) Start Impala Catalog server
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
