[jira] [Updated] (IMPALA-6436) Impala Catalog generates a core file / mini dump when the HMS is not available
[ https://issues.apache.org/jira/browse/IMPALA-6436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong updated IMPALA-6436: -- Target Version: Impala 3.1.0 (was: Impala 3.2.0) > Impala Catalog generates a core file / mini dump when the HMS is not available > -- > > Key: IMPALA-6436 > URL: https://issues.apache.org/jira/browse/IMPALA-6436 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 2.10.0 >Reporter: Luis E Martinez-Poblete >Assignee: Tim Armstrong >Priority: Critical > Labels: supportability > > Synopsis: > = > Impala Catalog generates a core file / mini dump when the HMS is not > available > Problem: > > Catalog server created multiple Catalog core files. During the investigation > it was determine that the cause of the core files was because the Hive Meta > Store was not available and the option "Enable Core Dump" was enabled when > starting the Impala service. > Below is the back trace of the core file: > #0 0x7f72e93ee5d7 in raise () from /root/191729/slib/lib64/libc.so.6 > #1 0x7f72e93efcc8 in abort () from /root/191729/slib/lib64/libc.so.6 > #2 0x01ba5754 in google::DumpStackTraceAndExit() () > #3 0x01b9c1cd in google::LogMessage::Fail() () > #4 0x01b9da72 in google::LogMessage::SendToLog() () > #5 0x01b9bba7 in google::LogMessage::Flush() () > #6 0x01b9f16e in google::LogMessageFatal::~LogMessageFatal() () > #7 0x0083067e in impala::Catalog::(GetCatalogVersion (this=0x0, > version=0x7ffc2aa6b750) at > /usr/src/debug/impala-2.10.0-cdh5.13.1/be/src/catalog/catalog.cc:88 > #8 0x008143c9 in impala::CatalogServer::Start() () at > /usr/src/debug/impala-2.10.0-cdh5.13.1/be/src/catalog/catalog-server.cc:175 > The corresponding entries in the Catalog server log show the following fatal > error: > F0111 09:48:05.017491 14571 catalog.cc:76] java.lang.IllegalStateException: > java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.metastore.HiveMetaStoreClient > at > org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.(MetaStoreClientPool.java:99) > at > org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.(MetaStoreClientPool.java:72) > at > org.apache.impala.catalog.MetaStoreClientPool.initClients(MetaStoreClientPool.java:168) > at org.apache.impala.catalog.Catalog.(Catalog.java:103) > at > org.apache.impala.catalog.CatalogServiceCatalog.(CatalogServiceCatalog.java:163) > at org.apache.impala.service.JniCatalog.(JniCatalog.java:104) > Caused by: java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.metastore.HiveMetaStoreClient > I was able to reproduce this issue. When the option "Enable Core Dump" is > enable and the Hive Meta Store is not available, the system generates a core > file. If the option "Enable Core Dump" is disabled, the system generates a > mini dump. > Crashing due to an error is not expected. Impala should fail in a more user > friendly way. > Reproduction case: > == > 1) Enable the option "Enable Core Dump" for the Impala service in CM. > 2) Stop Hive and Impala services. > 3) Start Impala Catalog server -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-6436) Impala Catalog generates a core file / mini dump when the HMS is not available
[ https://issues.apache.org/jira/browse/IMPALA-6436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong updated IMPALA-6436: -- Target Version: Impala 3.2.0 I think we've seen this cause problems enough times that we should prioritise fixing it. It shouldn't be that hard to log the exception and call _exit() instead of abort(). > Impala Catalog generates a core file / mini dump when the HMS is not available > -- > > Key: IMPALA-6436 > URL: https://issues.apache.org/jira/browse/IMPALA-6436 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 2.10.0 >Reporter: Luis E Martinez-Poblete >Priority: Critical > Labels: supportability > > Synopsis: > = > Impala Catalog generates a core file / mini dump when the HMS is not > available > Problem: > > Catalog server created multiple Catalog core files. During the investigation > it was determine that the cause of the core files was because the Hive Meta > Store was not available and the option "Enable Core Dump" was enabled when > starting the Impala service. > Below is the back trace of the core file: > #0 0x7f72e93ee5d7 in raise () from /root/191729/slib/lib64/libc.so.6 > #1 0x7f72e93efcc8 in abort () from /root/191729/slib/lib64/libc.so.6 > #2 0x01ba5754 in google::DumpStackTraceAndExit() () > #3 0x01b9c1cd in google::LogMessage::Fail() () > #4 0x01b9da72 in google::LogMessage::SendToLog() () > #5 0x01b9bba7 in google::LogMessage::Flush() () > #6 0x01b9f16e in google::LogMessageFatal::~LogMessageFatal() () > #7 0x0083067e in impala::Catalog::(GetCatalogVersion (this=0x0, > version=0x7ffc2aa6b750) at > /usr/src/debug/impala-2.10.0-cdh5.13.1/be/src/catalog/catalog.cc:88 > #8 0x008143c9 in impala::CatalogServer::Start() () at > /usr/src/debug/impala-2.10.0-cdh5.13.1/be/src/catalog/catalog-server.cc:175 > The corresponding entries in the Catalog server log show the following fatal > error: > F0111 09:48:05.017491 14571 catalog.cc:76] java.lang.IllegalStateException: > java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.metastore.HiveMetaStoreClient > at > org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.(MetaStoreClientPool.java:99) > at > org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.(MetaStoreClientPool.java:72) > at > org.apache.impala.catalog.MetaStoreClientPool.initClients(MetaStoreClientPool.java:168) > at org.apache.impala.catalog.Catalog.(Catalog.java:103) > at > org.apache.impala.catalog.CatalogServiceCatalog.(CatalogServiceCatalog.java:163) > at org.apache.impala.service.JniCatalog.(JniCatalog.java:104) > Caused by: java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.metastore.HiveMetaStoreClient > I was able to reproduce this issue. When the option "Enable Core Dump" is > enable and the Hive Meta Store is not available, the system generates a core > file. If the option "Enable Core Dump" is disabled, the system generates a > mini dump. > Crashing due to an error is not expected. Impala should fail in a more user > friendly way. > Reproduction case: > == > 1) Enable the option "Enable Core Dump" for the Impala service in CM. > 2) Stop Hive and Impala services. > 3) Start Impala Catalog server -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-6436) Impala Catalog generates a core file / mini dump when the HMS is not available
[ https://issues.apache.org/jira/browse/IMPALA-6436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong updated IMPALA-6436: -- Labels: supportability (was: ) > Impala Catalog generates a core file / mini dump when the HMS is not available > -- > > Key: IMPALA-6436 > URL: https://issues.apache.org/jira/browse/IMPALA-6436 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 2.10.0 >Reporter: Luis E Martinez-Poblete >Priority: Critical > Labels: supportability > > Synopsis: > = > Impala Catalog generates a core file / mini dump when the HMS is not > available > Problem: > > Catalog server created multiple Catalog core files. During the investigation > it was determine that the cause of the core files was because the Hive Meta > Store was not available and the option "Enable Core Dump" was enabled when > starting the Impala service. > Below is the back trace of the core file: > #0 0x7f72e93ee5d7 in raise () from /root/191729/slib/lib64/libc.so.6 > #1 0x7f72e93efcc8 in abort () from /root/191729/slib/lib64/libc.so.6 > #2 0x01ba5754 in google::DumpStackTraceAndExit() () > #3 0x01b9c1cd in google::LogMessage::Fail() () > #4 0x01b9da72 in google::LogMessage::SendToLog() () > #5 0x01b9bba7 in google::LogMessage::Flush() () > #6 0x01b9f16e in google::LogMessageFatal::~LogMessageFatal() () > #7 0x0083067e in impala::Catalog::(GetCatalogVersion (this=0x0, > version=0x7ffc2aa6b750) at > /usr/src/debug/impala-2.10.0-cdh5.13.1/be/src/catalog/catalog.cc:88 > #8 0x008143c9 in impala::CatalogServer::Start() () at > /usr/src/debug/impala-2.10.0-cdh5.13.1/be/src/catalog/catalog-server.cc:175 > The corresponding entries in the Catalog server log show the following fatal > error: > F0111 09:48:05.017491 14571 catalog.cc:76] java.lang.IllegalStateException: > java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.metastore.HiveMetaStoreClient > at > org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.(MetaStoreClientPool.java:99) > at > org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.(MetaStoreClientPool.java:72) > at > org.apache.impala.catalog.MetaStoreClientPool.initClients(MetaStoreClientPool.java:168) > at org.apache.impala.catalog.Catalog.(Catalog.java:103) > at > org.apache.impala.catalog.CatalogServiceCatalog.(CatalogServiceCatalog.java:163) > at org.apache.impala.service.JniCatalog.(JniCatalog.java:104) > Caused by: java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.metastore.HiveMetaStoreClient > I was able to reproduce this issue. When the option "Enable Core Dump" is > enable and the Hive Meta Store is not available, the system generates a core > file. If the option "Enable Core Dump" is disabled, the system generates a > mini dump. > Crashing due to an error is not expected. Impala should fail in a more user > friendly way. > Reproduction case: > == > 1) Enable the option "Enable Core Dump" for the Impala service in CM. > 2) Stop Hive and Impala services. > 3) Start Impala Catalog server -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-6436) Impala Catalog generates a core file / mini dump when the HMS is not available
[ https://issues.apache.org/jira/browse/IMPALA-6436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong updated IMPALA-6436: -- Priority: Critical (was: Major) > Impala Catalog generates a core file / mini dump when the HMS is not available > -- > > Key: IMPALA-6436 > URL: https://issues.apache.org/jira/browse/IMPALA-6436 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 2.10.0 >Reporter: Luis E Martinez-Poblete >Priority: Critical > Labels: supportability > > Synopsis: > = > Impala Catalog generates a core file / mini dump when the HMS is not > available > Problem: > > Catalog server created multiple Catalog core files. During the investigation > it was determine that the cause of the core files was because the Hive Meta > Store was not available and the option "Enable Core Dump" was enabled when > starting the Impala service. > Below is the back trace of the core file: > #0 0x7f72e93ee5d7 in raise () from /root/191729/slib/lib64/libc.so.6 > #1 0x7f72e93efcc8 in abort () from /root/191729/slib/lib64/libc.so.6 > #2 0x01ba5754 in google::DumpStackTraceAndExit() () > #3 0x01b9c1cd in google::LogMessage::Fail() () > #4 0x01b9da72 in google::LogMessage::SendToLog() () > #5 0x01b9bba7 in google::LogMessage::Flush() () > #6 0x01b9f16e in google::LogMessageFatal::~LogMessageFatal() () > #7 0x0083067e in impala::Catalog::(GetCatalogVersion (this=0x0, > version=0x7ffc2aa6b750) at > /usr/src/debug/impala-2.10.0-cdh5.13.1/be/src/catalog/catalog.cc:88 > #8 0x008143c9 in impala::CatalogServer::Start() () at > /usr/src/debug/impala-2.10.0-cdh5.13.1/be/src/catalog/catalog-server.cc:175 > The corresponding entries in the Catalog server log show the following fatal > error: > F0111 09:48:05.017491 14571 catalog.cc:76] java.lang.IllegalStateException: > java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.metastore.HiveMetaStoreClient > at > org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.(MetaStoreClientPool.java:99) > at > org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.(MetaStoreClientPool.java:72) > at > org.apache.impala.catalog.MetaStoreClientPool.initClients(MetaStoreClientPool.java:168) > at org.apache.impala.catalog.Catalog.(Catalog.java:103) > at > org.apache.impala.catalog.CatalogServiceCatalog.(CatalogServiceCatalog.java:163) > at org.apache.impala.service.JniCatalog.(JniCatalog.java:104) > Caused by: java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.metastore.HiveMetaStoreClient > I was able to reproduce this issue. When the option "Enable Core Dump" is > enable and the Hive Meta Store is not available, the system generates a core > file. If the option "Enable Core Dump" is disabled, the system generates a > mini dump. > Crashing due to an error is not expected. Impala should fail in a more user > friendly way. > Reproduction case: > == > 1) Enable the option "Enable Core Dump" for the Impala service in CM. > 2) Stop Hive and Impala services. > 3) Start Impala Catalog server -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
