[
https://issues.apache.org/jira/browse/IMPALA-11015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated IMPALA-11015:
------------------------------------
Description:
This is an intermittent failure. The failure is
{code}
webserver/test_web_pages.py:296: in test_catalog
self.get_and_check_status_jvm(self.CATALOG_URL, "foo_part")
webserver/test_web_pages.py:186: in get_and_check_status_jvm
ports_to_test=self.TEST_PORTS_WITHOUT_SS)
webserver/test_web_pages.py:171: in get_and_check_status
assert string_to_search in response.text
{code}
The assertion failure includes the html of the page where the text was not
found. I have attached this as test_catalog.html. In the text of the page I see
{code}
<div class="alert alert-danger">
<strong>Error:</strong>
ConcurrentModificationException: null
</div>
{code}
In the logs for the test run I find
{code}
@ 0x1fb81e7 impala::Status::Status()
@ 0x29a0899 impala::JniUtil::GetJniExceptionMsg()
@ 0x1f847b6 impala::JniCall::Call<>()
@ 0x1f81b25 impala::JniUtil::CallJniMethod<>()
@ 0x1f7f65d impala::Catalog::GetDbs()
@ 0x1f38b4e impala::CatalogServer::CatalogUrlCallback()
@ 0x1f3e421
_ZZN6impala13CatalogServer16RegisterWebpagesEPNS_9WebserverEENKUlRKT_PT0_E0_clIN4kudu19WebCallbackRegistry10WebRequestEN9rapidjson15GenericDocumentINSD_4UTF8IcEENSD_19MemoryPoolAllocatorINSD_12CrtAllocatorEEESI_EEEEDaS5_S7_
@ 0x1f3e471
_ZN5boost6detail8function26void_function_obj_invoker2IZN6impala13CatalogServer16RegisterWebpagesEPNS3_9WebserverEEUlRKT_PT0_E0_vRKN4kudu19WebCallbackRegistry10WebRequestEPN9rapidjson15GenericDocumentINSI_4UTF8IcEENSI_19MemoryPoolAllocatorINSI_12CrtAllocatorEEESN_EEE6invokeERNS1_15function_bufferESH_SQ_
@ 0x2aad07d boost::function2<>::operator()()
@ 0x2aaa14e impala::Webserver::RenderUrlWithTemplate()
@ 0x2aa8045 impala::Webserver::BeginRequestCallback()
@ 0x2aa663a impala::Webserver::BeginRequestCallbackStatic()
@ 0x2abe5cd handle_request
@ 0x2ac07fb process_new_connection
@ 0x2ac0ebf worker_thread
@ 0x7f921f33be24 start_thread
@ 0x7f921bd6a34c __clone
{code}
which looks like webserver rendering code, and
{code}
I1107 09:27:37.416805 30162 jni-util.cc:286]
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437)
at java.util.HashMap$EntryIterator.next(HashMap.java:1471)
at java.util.HashMap$EntryIterator.next(HashMap.java:1469)
at
org.apache.hadoop.hive.metastore.api.Database$DatabaseStandardScheme.write(Database.java:1188)
at
org.apache.hadoop.hive.metastore.api.Database$DatabaseStandardScheme.write(Database.java:1051)
at
org.apache.hadoop.hive.metastore.api.Database.write(Database.java:920)
at
org.apache.impala.thrift.TDatabase$TDatabaseStandardScheme.write(TDatabase.java:430)
at
org.apache.impala.thrift.TDatabase$TDatabaseStandardScheme.write(TDatabase.java:378)
at org.apache.impala.thrift.TDatabase.write(TDatabase.java:316)
at
org.apache.impala.thrift.TGetDbsResult$TGetDbsResultStandardScheme.write(TGetDbsResult.java:362)
at
org.apache.impala.thrift.TGetDbsResult$TGetDbsResultStandardScheme.write(TGetDbsResult.java:310)
at org.apache.impala.thrift.TGetDbsResult.write(TGetDbsResult.java:264)
at org.apache.thrift.TSerializer.serialize(TSerializer.java:79)
at org.apache.impala.service.JniCatalog.getDbs(JniCatalog.java:283)
{code}
So I think the rendering code hit a ConcurrentModificationException
was:
The failure is
{code}
webserver/test_web_pages.py:296: in test_catalog
self.get_and_check_status_jvm(self.CATALOG_URL, "foo_part")
webserver/test_web_pages.py:186: in get_and_check_status_jvm
ports_to_test=self.TEST_PORTS_WITHOUT_SS)
webserver/test_web_pages.py:171: in get_and_check_status
assert string_to_search in response.text
{code}
The assertion failure includes the html of the page where the text was not
found. I have attached this as test_catalog.html. In the text of the page I see
{code}
<div class="alert alert-danger">
<strong>Error:</strong>
ConcurrentModificationException: null
</div>
{code}
In the logs for the test run I find
{code}
@ 0x1fb81e7 impala::Status::Status()
@ 0x29a0899 impala::JniUtil::GetJniExceptionMsg()
@ 0x1f847b6 impala::JniCall::Call<>()
@ 0x1f81b25 impala::JniUtil::CallJniMethod<>()
@ 0x1f7f65d impala::Catalog::GetDbs()
@ 0x1f38b4e impala::CatalogServer::CatalogUrlCallback()
@ 0x1f3e421
_ZZN6impala13CatalogServer16RegisterWebpagesEPNS_9WebserverEENKUlRKT_PT0_E0_clIN4kudu19WebCallbackRegistry10WebRequestEN9rapidjson15GenericDocumentINSD_4UTF8IcEENSD_19MemoryPoolAllocatorINSD_12CrtAllocatorEEESI_EEEEDaS5_S7_
@ 0x1f3e471
_ZN5boost6detail8function26void_function_obj_invoker2IZN6impala13CatalogServer16RegisterWebpagesEPNS3_9WebserverEEUlRKT_PT0_E0_vRKN4kudu19WebCallbackRegistry10WebRequestEPN9rapidjson15GenericDocumentINSI_4UTF8IcEENSI_19MemoryPoolAllocatorINSI_12CrtAllocatorEEESN_EEE6invokeERNS1_15function_bufferESH_SQ_
@ 0x2aad07d boost::function2<>::operator()()
@ 0x2aaa14e impala::Webserver::RenderUrlWithTemplate()
@ 0x2aa8045 impala::Webserver::BeginRequestCallback()
@ 0x2aa663a impala::Webserver::BeginRequestCallbackStatic()
@ 0x2abe5cd handle_request
@ 0x2ac07fb process_new_connection
@ 0x2ac0ebf worker_thread
@ 0x7f921f33be24 start_thread
@ 0x7f921bd6a34c __clone
{code}
which looks like webserver rendering code, and
{code}
I1107 09:27:37.416805 30162 jni-util.cc:286]
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437)
at java.util.HashMap$EntryIterator.next(HashMap.java:1471)
at java.util.HashMap$EntryIterator.next(HashMap.java:1469)
at
org.apache.hadoop.hive.metastore.api.Database$DatabaseStandardScheme.write(Database.java:1188)
at
org.apache.hadoop.hive.metastore.api.Database$DatabaseStandardScheme.write(Database.java:1051)
at
org.apache.hadoop.hive.metastore.api.Database.write(Database.java:920)
at
org.apache.impala.thrift.TDatabase$TDatabaseStandardScheme.write(TDatabase.java:430)
at
org.apache.impala.thrift.TDatabase$TDatabaseStandardScheme.write(TDatabase.java:378)
at org.apache.impala.thrift.TDatabase.write(TDatabase.java:316)
at
org.apache.impala.thrift.TGetDbsResult$TGetDbsResultStandardScheme.write(TGetDbsResult.java:362)
at
org.apache.impala.thrift.TGetDbsResult$TGetDbsResultStandardScheme.write(TGetDbsResult.java:310)
at org.apache.impala.thrift.TGetDbsResult.write(TGetDbsResult.java:264)
at org.apache.thrift.TSerializer.serialize(TSerializer.java:79)
at org.apache.impala.service.JniCatalog.getDbs(JniCatalog.java:283)
{code}
So I think the rendering code hit a ConcurrentModificationException
> TestWebPage.test_catalog fails after a ConcurrentModificationException
> ----------------------------------------------------------------------
>
> Key: IMPALA-11015
> URL: https://issues.apache.org/jira/browse/IMPALA-11015
> Project: IMPALA
> Issue Type: Bug
> Reporter: Andrew Sherman
> Priority: Critical
> Attachments: test_catalog.html
>
>
> This is an intermittent failure. The failure is
> {code}
> webserver/test_web_pages.py:296: in test_catalog
> self.get_and_check_status_jvm(self.CATALOG_URL, "foo_part")
> webserver/test_web_pages.py:186: in get_and_check_status_jvm
> ports_to_test=self.TEST_PORTS_WITHOUT_SS)
> webserver/test_web_pages.py:171: in get_and_check_status
> assert string_to_search in response.text
> {code}
> The assertion failure includes the html of the page where the text was not
> found. I have attached this as test_catalog.html. In the text of the page I
> see
> {code}
> <div class="alert alert-danger">
> <strong>Error:</strong>
> ConcurrentModificationException: null
> </div>
> {code}
> In the logs for the test run I find
> {code}
> @ 0x1fb81e7 impala::Status::Status()
> @ 0x29a0899 impala::JniUtil::GetJniExceptionMsg()
> @ 0x1f847b6 impala::JniCall::Call<>()
> @ 0x1f81b25 impala::JniUtil::CallJniMethod<>()
> @ 0x1f7f65d impala::Catalog::GetDbs()
> @ 0x1f38b4e impala::CatalogServer::CatalogUrlCallback()
> @ 0x1f3e421
> _ZZN6impala13CatalogServer16RegisterWebpagesEPNS_9WebserverEENKUlRKT_PT0_E0_clIN4kudu19WebCallbackRegistry10WebRequestEN9rapidjson15GenericDocumentINSD_4UTF8IcEENSD_19MemoryPoolAllocatorINSD_12CrtAllocatorEEESI_EEEEDaS5_S7_
> @ 0x1f3e471
> _ZN5boost6detail8function26void_function_obj_invoker2IZN6impala13CatalogServer16RegisterWebpagesEPNS3_9WebserverEEUlRKT_PT0_E0_vRKN4kudu19WebCallbackRegistry10WebRequestEPN9rapidjson15GenericDocumentINSI_4UTF8IcEENSI_19MemoryPoolAllocatorINSI_12CrtAllocatorEEESN_EEE6invokeERNS1_15function_bufferESH_SQ_
> @ 0x2aad07d boost::function2<>::operator()()
> @ 0x2aaa14e impala::Webserver::RenderUrlWithTemplate()
> @ 0x2aa8045 impala::Webserver::BeginRequestCallback()
> @ 0x2aa663a impala::Webserver::BeginRequestCallbackStatic()
> @ 0x2abe5cd handle_request
> @ 0x2ac07fb process_new_connection
> @ 0x2ac0ebf worker_thread
> @ 0x7f921f33be24 start_thread
> @ 0x7f921bd6a34c __clone
> {code}
> which looks like webserver rendering code, and
> {code}
> I1107 09:27:37.416805 30162 jni-util.cc:286]
> java.util.ConcurrentModificationException
> at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437)
> at java.util.HashMap$EntryIterator.next(HashMap.java:1471)
> at java.util.HashMap$EntryIterator.next(HashMap.java:1469)
> at
> org.apache.hadoop.hive.metastore.api.Database$DatabaseStandardScheme.write(Database.java:1188)
> at
> org.apache.hadoop.hive.metastore.api.Database$DatabaseStandardScheme.write(Database.java:1051)
> at
> org.apache.hadoop.hive.metastore.api.Database.write(Database.java:920)
> at
> org.apache.impala.thrift.TDatabase$TDatabaseStandardScheme.write(TDatabase.java:430)
> at
> org.apache.impala.thrift.TDatabase$TDatabaseStandardScheme.write(TDatabase.java:378)
> at org.apache.impala.thrift.TDatabase.write(TDatabase.java:316)
> at
> org.apache.impala.thrift.TGetDbsResult$TGetDbsResultStandardScheme.write(TGetDbsResult.java:362)
> at
> org.apache.impala.thrift.TGetDbsResult$TGetDbsResultStandardScheme.write(TGetDbsResult.java:310)
> at
> org.apache.impala.thrift.TGetDbsResult.write(TGetDbsResult.java:264)
> at org.apache.thrift.TSerializer.serialize(TSerializer.java:79)
> at org.apache.impala.service.JniCatalog.getDbs(JniCatalog.java:283)
> {code}
> So I think the rendering code hit a ConcurrentModificationException
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
