[jira] [Assigned] (IMPALA-13076) Add pstack and jstack to Impala docker images
[ https://issues.apache.org/jira/browse/IMPALA-13076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman reassigned IMPALA-13076: --- Assignee: Andrew Sherman > Add pstack and jstack to Impala docker images > - > > Key: IMPALA-13076 > URL: https://issues.apache.org/jira/browse/IMPALA-13076 > Project: IMPALA > Issue Type: Bug >Affects Versions: Impala 4.4.0 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > > When the Impala docker images are deployed in production environments, it can > be hard to add debugging tools at runtime. Two of the most useful diagnosis > tools are jstack and pstack, which can be used to find Java and native stack > traces. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-13076) Add pstack and jstack to Impala docker images
Andrew Sherman created IMPALA-13076: --- Summary: Add pstack and jstack to Impala docker images Key: IMPALA-13076 URL: https://issues.apache.org/jira/browse/IMPALA-13076 Project: IMPALA Issue Type: Bug Affects Versions: Impala 4.4.0 Reporter: Andrew Sherman When the Impala docker images are deployed in production environments, it can be hard to add debugging tools at runtime. Two of the most useful diagnosis tools are jstack and pstack, which can be used to find Java and native stack traces. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-13013) Table not found after CONVERT TO ICEBERG in load-data
[
https://issues.apache.org/jira/browse/IMPALA-13013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman closed IMPALA-13013.
---
Resolution: Duplicate
[~stigahuang] thanks, yes, closing this as a duplicate of IMPALA-12266
> Table not found after CONVERT TO ICEBERG in load-data
> -
>
> Key: IMPALA-13013
> URL: https://issues.apache.org/jira/browse/IMPALA-13013
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.4.0
>Reporter: Andrew Sherman
>Assignee: Zoltán Borók-Nagy
>Priority: Critical
>
> load-data fails with
> {code:java}
> 17:02:44 CREATE TABLE IF NOT EXISTS
> functional_parquet.iceberg_lineitem_sixblocks
> 17:02:44 LIKE PARQUET
> '/test-warehouse/lineitem_sixblocks_iceberg/lineitem_sixblocks.parquet'
> 17:02:44 STORED AS PARQUET
> 17:02:44 LOCATION '/test-warehouse/lineitem_sixblocks_iceberg/'
> 17:02:44 Summary: Returned 1 rows
> 17:02:44 Success: True
> 17:02:44 Took: 0.172425031662(s)
> 17:02:44 Data:
> 17:02:44 Table has been created.
> 17:02:44
> 17:02:44 ALTER TABLE functional_parquet.iceberg_lineitem_sixblocks CONVERT TO
> ICEBERG
> 17:02:44 Summary: Returned 1 rows
> 17:02:44 Success: True
> 17:02:44 Took: 1.37302303314(s)
> 17:02:44 Data:
> 17:02:44 Table has been migrated.
> 17:02:44
> 17:02:44 ERROR: ALTER TABLE functional_parquet.iceberg_lineitem_sixblocks SET
> TBLPROPERTIES ('format-version'='2')
> 17:02:44 Traceback (most recent call last):
> 17:02:44 File
> "/data/jenkins/workspace/impala-cdw-master-staging-core/repos/Impala/bin/load-data.py",
> line 196, in exec_impala_query_from_file
> 17:02:44 result = impala_client.execute(query)
> 17:02:44 File
> "/data/jenkins/workspace/impala-cdw-master-staging-core/repos/Impala/tests/beeswax/impala_beeswax.py",
> line 191, in execute
> 17:02:44 handle = self.__execute_query(query_string.strip(), user=user)
> 17:02:44 File
> "/data/jenkins/workspace/impala-cdw-master-staging-core/repos/Impala/tests/beeswax/impala_beeswax.py",
> line 382, in __execute_query
> 17:02:44 handle = self.execute_query_async(query_string, user=user)
> 17:02:44 File
> "/data/jenkins/workspace/impala-cdw-master-staging-core/repos/Impala/tests/beeswax/impala_beeswax.py",
> line 376, in execute_query_async
> 17:02:44 handle = self.__do_rpc(lambda: self.imp_service.query(query,))
> 17:02:44 File
> "/data/jenkins/workspace/impala-cdw-master-staging-core/repos/Impala/tests/beeswax/impala_beeswax.py",
> line 539, in __do_rpc
> 17:02:44 raise ImpalaBeeswaxException(self.__build_error_message(b), b)
> 17:02:44 ImpalaBeeswaxException: ImpalaBeeswaxException:
> 17:02:44 INNER EXCEPTION:
> 17:02:44 MESSAGE: AnalysisException: Could not resolve table reference:
> 'functional_parquet.iceberg_lineitem_sixblocks'
> {code}
> IMPALA-12330 might help avoid this problem
> [~boroknagyz] assigning to you for triage, please distribute as appropriate
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-13013) Table not found after CONVERT TO ICEBERG in load-data
Andrew Sherman created IMPALA-13013:
---
Summary: Table not found after CONVERT TO ICEBERG in load-data
Key: IMPALA-13013
URL: https://issues.apache.org/jira/browse/IMPALA-13013
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 4.4.0
Reporter: Andrew Sherman
Assignee: Zoltán Borók-Nagy
load-data fails with
{code:java}
17:02:44 CREATE TABLE IF NOT EXISTS
functional_parquet.iceberg_lineitem_sixblocks
17:02:44 LIKE PARQUET
'/test-warehouse/lineitem_sixblocks_iceberg/lineitem_sixblocks.parquet'
17:02:44 STORED AS PARQUET
17:02:44 LOCATION '/test-warehouse/lineitem_sixblocks_iceberg/'
17:02:44 Summary: Returned 1 rows
17:02:44 Success: True
17:02:44 Took: 0.172425031662(s)
17:02:44 Data:
17:02:44 Table has been created.
17:02:44
17:02:44 ALTER TABLE functional_parquet.iceberg_lineitem_sixblocks CONVERT TO
ICEBERG
17:02:44 Summary: Returned 1 rows
17:02:44 Success: True
17:02:44 Took: 1.37302303314(s)
17:02:44 Data:
17:02:44 Table has been migrated.
17:02:44
17:02:44 ERROR: ALTER TABLE functional_parquet.iceberg_lineitem_sixblocks SET
TBLPROPERTIES ('format-version'='2')
17:02:44 Traceback (most recent call last):
17:02:44 File
"/data/jenkins/workspace/impala-cdw-master-staging-core/repos/Impala/bin/load-data.py",
line 196, in exec_impala_query_from_file
17:02:44 result = impala_client.execute(query)
17:02:44 File
"/data/jenkins/workspace/impala-cdw-master-staging-core/repos/Impala/tests/beeswax/impala_beeswax.py",
line 191, in execute
17:02:44 handle = self.__execute_query(query_string.strip(), user=user)
17:02:44 File
"/data/jenkins/workspace/impala-cdw-master-staging-core/repos/Impala/tests/beeswax/impala_beeswax.py",
line 382, in __execute_query
17:02:44 handle = self.execute_query_async(query_string, user=user)
17:02:44 File
"/data/jenkins/workspace/impala-cdw-master-staging-core/repos/Impala/tests/beeswax/impala_beeswax.py",
line 376, in execute_query_async
17:02:44 handle = self.__do_rpc(lambda: self.imp_service.query(query,))
17:02:44 File
"/data/jenkins/workspace/impala-cdw-master-staging-core/repos/Impala/tests/beeswax/impala_beeswax.py",
line 539, in __do_rpc
17:02:44 raise ImpalaBeeswaxException(self.__build_error_message(b), b)
17:02:44 ImpalaBeeswaxException: ImpalaBeeswaxException:
17:02:44 INNER EXCEPTION:
17:02:44 MESSAGE: AnalysisException: Could not resolve table reference:
'functional_parquet.iceberg_lineitem_sixblocks'
{code}
IMPALA-12330 might help avoid this problem
[~boroknagyz] assigning to you for triage, please distribute as appropriate
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-13004) heap-use-after-free error in ExprTest AiFunctionsTest
Andrew Sherman created IMPALA-13004:
---
Summary: heap-use-after-free error in ExprTest AiFunctionsTest
Key: IMPALA-13004
URL: https://issues.apache.org/jira/browse/IMPALA-13004
Project: IMPALA
Issue Type: Bug
Components: be
Affects Versions: Impala 4.4.0
Reporter: Andrew Sherman
Assignee: Yida Wu
In an ASAN test, expr-test fails:
{code}
==1601==ERROR: AddressSanitizer: heap-use-after-free on address 0x63100152c826
at pc 0x0298f841 bp 0x7ffc91fff460 sp 0x7ffc91fff458
READ of size 2 at 0x63100152c826 thread T0
#0 0x298f840 in rapidjson::GenericValue,
rapidjson::MemoryPoolAllocator >::GetType() const
/data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:936:62
#1 0x298d852 in bool rapidjson::GenericValue,
rapidjson::MemoryPoolAllocator
>::Accept,
rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8,
rapidjson::CrtAllocator, 0u>
>(rapidjson::Writer,
rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8,
rapidjson::CrtAllocator, 0u>&) const
/data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1769:16
#2 0x298d8d0 in bool rapidjson::GenericValue,
rapidjson::MemoryPoolAllocator
>::Accept,
rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8,
rapidjson::CrtAllocator, 0u>
>(rapidjson::Writer,
rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8,
rapidjson::CrtAllocator, 0u>&) const
/data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1790:21
#3 0x298d9e8 in bool rapidjson::GenericValue,
rapidjson::MemoryPoolAllocator
>::Accept,
rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8,
rapidjson::CrtAllocator, 0u>
>(rapidjson::Writer,
rapidjson::CrtAllocator>, rapidjson::UTF8, rapidjson::UTF8,
rapidjson::CrtAllocator, 0u>&) const
/data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/document.h:1781:21
#4 0x28a0707 in impala_udf::StringVal
impala::AiFunctions::AiGenerateTextInternal(impala_udf::FunctionContext*,
impala_udf::StringVal const&, impala_udf::StringVal const&,
impala_udf::StringVal const&, impala_udf::StringVal const&,
impala_udf::StringVal const&, bool)
/data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/ai-functions.inline.h:140:11
#5 0x286087e in impala::ExprTest_AiFunctionsTest_Test::TestBody()
/data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/exprs/expr-test.cc:11254:12
#6 0x8aeaa4c in void
testing::internal::HandleExceptionsInMethodIfSupported(testing::Test*, void (testing::Test::*)(), char const*)
(/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8aeaa4c)
#7 0x8ae3ec4 in testing::Test::Run()
(/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae3ec4)
#8 0x8ae4007 in testing::TestInfo::Run()
(/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae4007)
#9 0x8ae40e4 in testing::TestCase::Run()
(/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae40e4)
#10 0x8ae45db in testing::internal::UnitTestImpl::RunAllTests()
(/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae45db)
#11 0x8ae4682 in testing::UnitTest::Run()
(/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x8ae4682)
#12 0x249ac19 in main
/data/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/src/service/unified-betest-main.cc:48:10
#13 0x7f4b0b911554 in __libc_start_main (/lib64/libc.so.6+0x22554)
#14 0x2396af6 in _start
(/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2396af6)
0x63100152c826 is located 38 bytes inside of 65560-byte region
[0x63100152c800,0x63100153c818)
freed by thread T0 here:
#0 0x2466ea7 in __interceptor_free
(/data0/jenkins/workspace/impala-cdw-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x2466ea7)
#1 0x299656b in
rapidjson::MemoryPoolAllocator::Clear()
/data/jenkins/workspace/impala-cdw-master-core-asan/Impala-Toolchain/toolchain-packages-gcc10.4.0/rapidjson-1.1.0/include/rapidjson/internal/../allocators.h:148:13
#2 0x29964cd in
rapidjson::MemoryPoolAllocator::~MemoryPoolAllocator()
[jira] [Created] (IMPALA-13003) Server exits early failing to create impala_query_log with AlreadyExistsException
Andrew Sherman created IMPALA-13003:
---
Summary: Server exits early failing to create impala_query_log
with AlreadyExistsException
Key: IMPALA-13003
URL: https://issues.apache.org/jira/browse/IMPALA-13003
Project: IMPALA
Issue Type: Bug
Components: be
Affects Versions: Impala 4.4.0
Reporter: Andrew Sherman
At startup workload management tries to create the query log table here:
{code:java}
// The initialization code only works when run in a separate thread for
reasons unknown.
ABORT_IF_ERROR(SetupDbTable(internal_server_.get(), table_name));
{code}
This code is exiting:
{code:java}
I0413 23:40:05.183876 21006 client-request-state.cc:1348]
1d4878dbc9214c81:6dc8cc2e] ImpalaRuntimeException: Error making
'createTable' RPC to Hive Metastore:
CAUSED BY: AlreadyExistsException: Table was created concurrently:
sys.impala_query_log
I0413 23:40:05.184055 20955 impala-server.cc:2582] Connection
27432606d99dcdae:218860164eb206bb from client in-memory.localhost:0 to server
internal-server closed. The connection had 1 associated session(s).
I0413 23:40:05.184067 20955 impala-server.cc:1780] Closing session:
27432606d99dcdae:218860164eb206bb
I0413 23:40:05.184083 20955 impala-server.cc:1836] Closed session:
27432606d99dcdae:218860164eb206bb, client address: .
F0413 23:40:05.184111 20955 workload-management.cc:304] query timed out waiting
for results
. Impalad exiting.
I0413 23:40:05.184728 20883 impala-server.cc:1564] Query successfully
unregistered: query_id=1d4878dbc9214c81:6dc8cc2e
Minidump in thread [20955]completed-queries running query
:, fragment instance
:
Wrote minidump to
/data/jenkins/workspace/impala-cdw-master-core-ubsan/repos/Impala/logs/custom_cluster_tests/minidumps/impalad/402f37cc-4663-4c78-086ca295-a9e5943c.dmp
{code}
with stack
{code:java}
F0413 23:40:05.184111 20955 workload-management.cc:304] query timed out waiting
for results
. Impalad exiting.
*** Check failure stack trace: ***
@ 0x8e96a4d google::LogMessage::Fail()
@ 0x8e98984 google::LogMessage::SendToLog()
@ 0x8e9642c google::LogMessage::Flush()
@ 0x8e98ea9 google::LogMessageFatal::~LogMessageFatal()
@ 0x3da3a9a impala::ImpalaServer::CompletedQueriesThread()
@ 0x3a8df93 boost::_mfi::mf0<>::operator()()
@ 0x3a8de97 boost::_bi::list1<>::operator()<>()
@ 0x3a8dd77 boost::_bi::bind_t<>::operator()()
@ 0x3a8d672
boost::detail::function::void_function_obj_invoker0<>::invoke()
@ 0x301e7d0 boost::function0<>::operator()()
@ 0x43ce415 impala::Thread::SuperviseThread()
@ 0x43e2dc7 boost::_bi::list5<>::operator()<>()
@ 0x43e29e7 boost::_bi::bind_t<>::operator()()
@ 0x43e21c5 boost::detail::thread_data<>::run()
@ 0x7984c37 thread_proxy
@ 0x7f75b6982ea5 start_thread
@ 0x7f75b36a7b0d __clone
Picked up JAVA_TOOL_OPTIONS:
-agentlib:jdwp=transport=dt_socket,address=3,server=y,suspend=n
-Dsun.java.command=impalad
Minidump in thread [20955]completed-queries running query
:, fragment instance
:
{code}
I think the key error is
{code}
CAUSED BY: AlreadyExistsException: Table was created concurrently:
sys.impala_query_log
{code}
which suggests that creating the table with "if not exists" is not sufficient
to protect against concurrent creations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12943) Document User Quota Model for Admission control
Andrew Sherman created IMPALA-12943: --- Summary: Document User Quota Model for Admission control Key: IMPALA-12943 URL: https://issues.apache.org/jira/browse/IMPALA-12943 Project: IMPALA Issue Type: Bug Reporter: Andrew Sherman Assignee: Andrew Sherman Document the work done in IMPALA-12345 "Add user quotas to Admission Control" -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12345) Add user quotas to Admission Control
[ https://issues.apache.org/jira/browse/IMPALA-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated IMPALA-12345: Description: Allow System Administrators to control how many queries can be queued by a user to an individual Resource Pools, or to the Impala system as a whole. (was: [will add more details later]) > Add user quotas to Admission Control > > > Key: IMPALA-12345 > URL: https://issues.apache.org/jira/browse/IMPALA-12345 > Project: IMPALA > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Critical > > Allow System Administrators to control how many queries can be queued by a > user to an individual Resource Pools, or to the Impala system as a whole. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12345) Add user quotas to Admission Control
[ https://issues.apache.org/jira/browse/IMPALA-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated IMPALA-12345: Priority: Critical (was: Major) > Add user quotas to Admission Control > > > Key: IMPALA-12345 > URL: https://issues.apache.org/jira/browse/IMPALA-12345 > Project: IMPALA > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Critical > > [will add more details later] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-12816) Remove exclusion of Netty to allow SSL connections to HBase.
[ https://issues.apache.org/jira/browse/IMPALA-12816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman resolved IMPALA-12816. - Fix Version/s: Impala 4.4.0 Resolution: Fixed > Remove exclusion of Netty to allow SSL connections to HBase. > > > Key: IMPALA-12816 > URL: https://issues.apache.org/jira/browse/IMPALA-12816 > Project: IMPALA > Issue Type: Bug >Affects Versions: Impala 4.4.0 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Critical > Fix For: Impala 4.4.0 > > > It is possible to configure Impala to use SSL when connecting to Hbase. Under > the covers, an HBase client connection also has a connection to ZooKeeper. > SSL connections to ZooKeeper require Netty. Netty has been excluded from > Impala’s classpath since IMPALA-9648 because of earlier security problems. > IMPALA-9648 says that we need to be at least at version 4.1.44 of Netty and > we now pull in 4.1.86. To allow use of SSL with HBase, remove the exclusion > of Netty. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12816) Remove exclusion of Netty to allow SSL connections to HBase.
Andrew Sherman created IMPALA-12816: --- Summary: Remove exclusion of Netty to allow SSL connections to HBase. Key: IMPALA-12816 URL: https://issues.apache.org/jira/browse/IMPALA-12816 Project: IMPALA Issue Type: Bug Affects Versions: Impala 4.4.0 Reporter: Andrew Sherman Assignee: Andrew Sherman It is possible to configure Impala to use SSL when connecting to Hbase. Under the covers, an HBase client connection also has a connection to ZooKeeper. SSL connections to ZooKeeper require Netty. Netty has been excluded from Impala’s classpath since IMPALA-9648 because of earlier security problems. IMPALA-9648 says that we need to be at least at version 4.1.44 of Netty and we now pull in 4.1.86. To allow use of SSL with HBase, remove the exclusion of Netty. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-12805) Avoid problems of HIVE-27114 without configuration changes
[ https://issues.apache.org/jira/browse/IMPALA-12805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman closed IMPALA-12805. --- Fix Version/s: Not Applicable Resolution: Won't Fix Abandoning as it seems that HIVE-27114 is unsafe. ANALYZE TABLE COMPUTE STATISTICS in Hive overwrites the partitions and removes partition property impala_intermediate_stats_chunk0. > Avoid problems of HIVE-27114 without configuration changes > -- > > Key: IMPALA-12805 > URL: https://issues.apache.org/jira/browse/IMPALA-12805 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 4.4.0 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Critical > Fix For: Not Applicable > > > HIVE-27114 adds a new > property,"hive.metastore.partitions.parameters.exclude.pattern", which > defaults to "impala_intermediate_stats_chunk%". This default excludes the > Impala incremental stats from the results returned from the HMS client. I > think this was done to help Hive performance. It means however that Impala > cannot retrieve its own partition stats, which will negatively impact the > performance of many workloads. > One fix for this was IMPALA-12743 which sets a non-empty non-default value > for the new Hive property in the development environment’s hive-site.xml > file. In some customer situations this would require a configuration change > that cannot be deployed simultaneously with the possible HMS upgrade which > introduces the regression. > Fix this by setting a non-empty non-default value for the new Hive property > in the configurations in the MetaStoreClientPool. This avoids the regression. > The natural thing would be to set an empty string as the value, but this is > not enough to override the default. Instead we set the string “do_not_match” > which should never be used for as the name of any partition stats. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12805) Avoid problems of HIVE-27114 without configuration changes
[ https://issues.apache.org/jira/browse/IMPALA-12805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated IMPALA-12805: Description: HIVE-27114 adds a new property,"hive.metastore.partitions.parameters.exclude.pattern", which defaults to "impala_intermediate_stats_chunk%". This default excludes the Impala incremental stats from the results returned from the HMS client. I think this was done to help Hive performance. It means however that Impala cannot retrieve its own partition stats, which will negatively impact the performance of many workloads. One fix for this was IMPALA-12743 which sets a non-empty non-default value for the new Hive property in the development environment’s hive-site.xml file. In some customer situations this would require a configuration change that cannot be deployed simultaneously with the possible HMS upgrade which introduces the regression. This change sets a non-empty non-default value for the new Hive property in the configurations in the MetaStoreClientPool. This avoids the regression. The natural thing would be to set an empty string as the value, but this is not enough to override the default. Instead we set the string “do_not_match” which should never be used for as the name of any partition stats. was: HIVE-27114 adds a new property,"hive.metastore.partitions.parameters.exclude.pattern", which defaults to "impala_intermediate_stats_chunk%". This default excludes the Impala incremental stats from the results returned from the HMS client. I think this was done to help Hive performance. It means however that Impala cannot retrieve its own partition stats, which will negatively impact the performance of many workloads. One fix for this was IMPALA-12743 which sets a non-empty non-default value for the new Hive property in the development environment’s hive-site.xml file. In some customer situations this would require a configuration change that cannot be deployed simultaneously with the possible HMS upgrade which introduces the regression. This change sets a non-empty non-default value for the new Hive property in the configurations in the MetaStoreClientPool. This avoids the regression. The natural thing would be to set an empty string as the value, but th > Avoid problems of HIVE-27114 without configuration changes > -- > > Key: IMPALA-12805 > URL: https://issues.apache.org/jira/browse/IMPALA-12805 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 4.4.0 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Critical > > HIVE-27114 adds a new > property,"hive.metastore.partitions.parameters.exclude.pattern", which > defaults to "impala_intermediate_stats_chunk%". This default excludes the > Impala incremental stats from the results returned from the HMS client. I > think this was done to help Hive performance. It means however that Impala > cannot retrieve its own partition stats, which will negatively impact the > performance of many workloads. > One fix for this was IMPALA-12743 which sets a non-empty non-default value > for the new Hive property in the development environment’s hive-site.xml > file. In some customer situations this would require a configuration change > that cannot be deployed simultaneously with the possible HMS upgrade which > introduces the regression. > This change sets a non-empty non-default value for the new Hive property in > the configurations in the MetaStoreClientPool. This avoids the regression. > The natural thing would be to set an empty string as the value, but this is > not enough to override the default. Instead we set the string “do_not_match” > which should never be used for as the name of any partition stats. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12805) Avoid problems of HIVE-27114 without configuration changes
[ https://issues.apache.org/jira/browse/IMPALA-12805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated IMPALA-12805: Description: HIVE-27114 adds a new property,"hive.metastore.partitions.parameters.exclude.pattern", which defaults to "impala_intermediate_stats_chunk%". This default excludes the Impala incremental stats from the results returned from the HMS client. I think this was done to help Hive performance. It means however that Impala cannot retrieve its own partition stats, which will negatively impact the performance of many workloads. One fix for this was IMPALA-12743 which sets a non-empty non-default value for the new Hive property in the development environment’s hive-site.xml file. In some customer situations this would require a configuration change that cannot be deployed simultaneously with the possible HMS upgrade which introduces the regression. Fix this by setting a non-empty non-default value for the new Hive property in the configurations in the MetaStoreClientPool. This avoids the regression. The natural thing would be to set an empty string as the value, but this is not enough to override the default. Instead we set the string “do_not_match” which should never be used for as the name of any partition stats. was: HIVE-27114 adds a new property,"hive.metastore.partitions.parameters.exclude.pattern", which defaults to "impala_intermediate_stats_chunk%". This default excludes the Impala incremental stats from the results returned from the HMS client. I think this was done to help Hive performance. It means however that Impala cannot retrieve its own partition stats, which will negatively impact the performance of many workloads. One fix for this was IMPALA-12743 which sets a non-empty non-default value for the new Hive property in the development environment’s hive-site.xml file. In some customer situations this would require a configuration change that cannot be deployed simultaneously with the possible HMS upgrade which introduces the regression. This change sets a non-empty non-default value for the new Hive property in the configurations in the MetaStoreClientPool. This avoids the regression. The natural thing would be to set an empty string as the value, but this is not enough to override the default. Instead we set the string “do_not_match” which should never be used for as the name of any partition stats. > Avoid problems of HIVE-27114 without configuration changes > -- > > Key: IMPALA-12805 > URL: https://issues.apache.org/jira/browse/IMPALA-12805 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 4.4.0 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Critical > > HIVE-27114 adds a new > property,"hive.metastore.partitions.parameters.exclude.pattern", which > defaults to "impala_intermediate_stats_chunk%". This default excludes the > Impala incremental stats from the results returned from the HMS client. I > think this was done to help Hive performance. It means however that Impala > cannot retrieve its own partition stats, which will negatively impact the > performance of many workloads. > One fix for this was IMPALA-12743 which sets a non-empty non-default value > for the new Hive property in the development environment’s hive-site.xml > file. In some customer situations this would require a configuration change > that cannot be deployed simultaneously with the possible HMS upgrade which > introduces the regression. > Fix this by setting a non-empty non-default value for the new Hive property > in the configurations in the MetaStoreClientPool. This avoids the regression. > The natural thing would be to set an empty string as the value, but this is > not enough to override the default. Instead we set the string “do_not_match” > which should never be used for as the name of any partition stats. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12805) Avoid problems of HIVE-27114 without configuration changes
Andrew Sherman created IMPALA-12805: --- Summary: Avoid problems of HIVE-27114 without configuration changes Key: IMPALA-12805 URL: https://issues.apache.org/jira/browse/IMPALA-12805 Project: IMPALA Issue Type: Bug Components: Catalog Affects Versions: Impala 4.4.0 Reporter: Andrew Sherman Assignee: Andrew Sherman HIVE-27114 adds a new property,"hive.metastore.partitions.parameters.exclude.pattern", which defaults to "impala_intermediate_stats_chunk%". This default excludes the Impala incremental stats from the results returned from the HMS client. I think this was done to help Hive performance. It means however that Impala cannot retrieve its own partition stats, which will negatively impact the performance of many workloads. One fix for this was IMPALA-12743 which sets a non-empty non-default value for the new Hive property in the development environment’s hive-site.xml file. In some customer situations this would require a configuration change that cannot be deployed simultaneously with the possible HMS upgrade which introduces the regression. This change sets a non-empty non-default value for the new Hive property in the configurations in the MetaStoreClientPool. This avoids the regression. The natural thing would be to set an empty string as the value, but th -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12264) Add configuration to limit concurrent connections
[ https://issues.apache.org/jira/browse/IMPALA-12264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman reassigned IMPALA-12264: --- Assignee: Andrew Sherman > Add configuration to limit concurrent connections > - > > Key: IMPALA-12264 > URL: https://issues.apache.org/jira/browse/IMPALA-12264 > Project: IMPALA > Issue Type: New Feature > Components: Backend >Reporter: Quanlong Huang >Assignee: Andrew Sherman >Priority: Major > > To prevent a rogue application from repeatedly connecting to and monopolizing > Impala, we can add configurations to limit concurrent connections. E.g. limit > the number of concurrent connections per user, IP address, or the user and IP > address combination. > CC [~mylogi...@gmail.com] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-12619) Update Impala website to reduce emphasis on Hadoop
[ https://issues.apache.org/jira/browse/IMPALA-12619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman closed IMPALA-12619. --- Fix Version/s: Impala 4.3.0 Resolution: Fixed > Update Impala website to reduce emphasis on Hadoop > -- > > Key: IMPALA-12619 > URL: https://issues.apache.org/jira/browse/IMPALA-12619 > Project: IMPALA > Issue Type: Bug > Components: Docs >Affects Versions: Impala 4.3.0 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Fix For: Impala 4.3.0 > > > The Impala website at ASF [https://impala.apache.org/] is the first hit > returned for “Apache Impala”. > Update the first line of the description to say ="Apache Impala is a modern, > open source, distributed SQL query engine for open data formats." > instead of "Apache Impala is a modern, open source, distributed SQL query > engine for Apache Hadoop." > Also mention Ranger instead of Sentry, and add references to Iceberg. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12619) Update Impala website to reduce emphasis on Hadoop
[ https://issues.apache.org/jira/browse/IMPALA-12619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated IMPALA-12619: Affects Version/s: Impala 4.3.0 (was: Impala 1.4.2) > Update Impala website to reduce emphasis on Hadoop > -- > > Key: IMPALA-12619 > URL: https://issues.apache.org/jira/browse/IMPALA-12619 > Project: IMPALA > Issue Type: Bug > Components: Docs >Affects Versions: Impala 4.3.0 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > > The Impala website at ASF [https://impala.apache.org/] is the first hit > returned for “Apache Impala”. > Update the first line of the description to say ="Apache Impala is a modern, > open source, distributed SQL query engine for open data formats." > instead of "Apache Impala is a modern, open source, distributed SQL query > engine for Apache Hadoop." > Also mention Ranger instead of Sentry, and add references to Iceberg. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-12632) TSAN detected data race accessing CpuUsageRatios
[
https://issues.apache.org/jira/browse/IMPALA-12632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman resolved IMPALA-12632.
-
Fix Version/s: Impala 4.4.0
Resolution: Fixed
> TSAN detected data race accessing CpuUsageRatios
>
>
> Key: IMPALA-12632
> URL: https://issues.apache.org/jira/browse/IMPALA-12632
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.3.0
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
>Priority: Critical
> Fix For: Impala 4.4.0
>
>
> This looks like a similar problem to IMPALA-12614, which can hopefully be
> fixed with the same strategy of using AtomicInt32 for the components in
> CpuUsageRatios.
> WARNING: ThreadSanitizer: data race (pid=7397)
> {code}
> Write of size 4 at 0x7b54000822d4 by thread T4 (mutexes: write M3360):
> #0 impala::SystemStateInfo::ComputeCpuRatios()
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/util/system-state-info.cc:149:20
> (impalad+0x2cab247)
> #1 impala::SystemStateInfo::CaptureSystemStateSnapshot()
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/util/system-state-info.cc:95:3
> (impalad+0x2caaf3c)
> #2 impala::ExecEnv::InitSystemStateInfo()::$_8::operator()() const
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/runtime/exec-env.cc:639:8
> (impalad+0x2577161)
> #3 void std::__invoke_impl impala::ExecEnv::InitSystemStateInfo()::$_8&>(std::__invoke_other,
> impala::ExecEnv::InitSystemStateInfo()::$_8&)
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/invoke.h:60:14
> (impalad+0x25770f1)
> #4 std::enable_if impala::ExecEnv::InitSystemStateInfo()::$_8&>, void>::type
> std::__invoke_r impala::ExecEnv::InitSystemStateInfo()::$_8&>(impala::ExecEnv::InitSystemStateInfo()::$_8&)
>
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/invoke.h:110:2
> (impalad+0x2577071)
> #5 std::_Function_handler impala::ExecEnv::InitSystemStateInfo()::$_8>::_M_invoke(std::_Any_data
> const&)
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/std_function.h:291:9
> (impalad+0x2576f41)
> #6 std::function::operator()() const
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/std_function.h:622:14
> (impalad+0x23dee3c)
> #7
> impala::PeriodicCounterUpdater::UpdateLoop(impala::PeriodicCounterUpdater*)
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/util/periodic-counter-updater.cc:175:39
> (impalad+0x2c3e6d6)
> #8 boost::_mfi::mf1 impala::PeriodicCounterUpdater*>::operator()(impala::PeriodicCounterUpdater*,
> impala::PeriodicCounterUpdater*) const
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/mem_fn_template.hpp:165:29
> (impalad+0x2c4382d)
> #9 void
> boost::_bi::list2,
> boost::_bi::value
> >::operator() impala::PeriodicCounterUpdater*>, boost::_bi::list0>(boost::_bi::type,
> boost::_mfi::mf1 impala::PeriodicCounterUpdater*>&, boost::_bi::list0&, int)
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:319:9
> (impalad+0x2c4376d)
> #10 boost::_bi::bind_t impala::PeriodicCounterUpdater, impala::PeriodicCounterUpdater*>,
> boost::_bi::list2,
> boost::_bi::value > >::operator()()
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:1294:16
> (impalad+0x2c436d3)
> #11 boost::detail::thread_data boost::_mfi::mf1 impala::PeriodicCounterUpdater*>,
> boost::_bi::list2,
> boost::_bi::value > > >::run()
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/thread/detail/thread.hpp:120:17
> (impalad+0x2c43140)
> #12 thread_proxy (impalad+0x3a0dbf6)
> {code}
> {code}
> Previous read of size 4 at 0x7b54000822d4 by thread T3 (mutexes: write
> M3193, write M165007):
> #0 impala::QueryState::Init(impala::ExecQueryFInstancesRequestPB const*,
>
[jira] [Updated] (IMPALA-12632) TSAN detected data race accessing CpuUsageRatios
[
https://issues.apache.org/jira/browse/IMPALA-12632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated IMPALA-12632:
Affects Version/s: Impala 4.3.0
(was: Impala 1.4.2)
> TSAN detected data race accessing CpuUsageRatios
>
>
> Key: IMPALA-12632
> URL: https://issues.apache.org/jira/browse/IMPALA-12632
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.3.0
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
>Priority: Critical
>
> This looks like a similar problem to IMPALA-12614, which can hopefully be
> fixed with the same strategy of using AtomicInt32 for the components in
> CpuUsageRatios.
> WARNING: ThreadSanitizer: data race (pid=7397)
> {code}
> Write of size 4 at 0x7b54000822d4 by thread T4 (mutexes: write M3360):
> #0 impala::SystemStateInfo::ComputeCpuRatios()
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/util/system-state-info.cc:149:20
> (impalad+0x2cab247)
> #1 impala::SystemStateInfo::CaptureSystemStateSnapshot()
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/util/system-state-info.cc:95:3
> (impalad+0x2caaf3c)
> #2 impala::ExecEnv::InitSystemStateInfo()::$_8::operator()() const
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/runtime/exec-env.cc:639:8
> (impalad+0x2577161)
> #3 void std::__invoke_impl impala::ExecEnv::InitSystemStateInfo()::$_8&>(std::__invoke_other,
> impala::ExecEnv::InitSystemStateInfo()::$_8&)
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/invoke.h:60:14
> (impalad+0x25770f1)
> #4 std::enable_if impala::ExecEnv::InitSystemStateInfo()::$_8&>, void>::type
> std::__invoke_r impala::ExecEnv::InitSystemStateInfo()::$_8&>(impala::ExecEnv::InitSystemStateInfo()::$_8&)
>
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/invoke.h:110:2
> (impalad+0x2577071)
> #5 std::_Function_handler impala::ExecEnv::InitSystemStateInfo()::$_8>::_M_invoke(std::_Any_data
> const&)
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/std_function.h:291:9
> (impalad+0x2576f41)
> #6 std::function::operator()() const
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/std_function.h:622:14
> (impalad+0x23dee3c)
> #7
> impala::PeriodicCounterUpdater::UpdateLoop(impala::PeriodicCounterUpdater*)
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/util/periodic-counter-updater.cc:175:39
> (impalad+0x2c3e6d6)
> #8 boost::_mfi::mf1 impala::PeriodicCounterUpdater*>::operator()(impala::PeriodicCounterUpdater*,
> impala::PeriodicCounterUpdater*) const
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/mem_fn_template.hpp:165:29
> (impalad+0x2c4382d)
> #9 void
> boost::_bi::list2,
> boost::_bi::value
> >::operator() impala::PeriodicCounterUpdater*>, boost::_bi::list0>(boost::_bi::type,
> boost::_mfi::mf1 impala::PeriodicCounterUpdater*>&, boost::_bi::list0&, int)
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:319:9
> (impalad+0x2c4376d)
> #10 boost::_bi::bind_t impala::PeriodicCounterUpdater, impala::PeriodicCounterUpdater*>,
> boost::_bi::list2,
> boost::_bi::value > >::operator()()
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:1294:16
> (impalad+0x2c436d3)
> #11 boost::detail::thread_data boost::_mfi::mf1 impala::PeriodicCounterUpdater*>,
> boost::_bi::list2,
> boost::_bi::value > > >::run()
> /data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/thread/detail/thread.hpp:120:17
> (impalad+0x2c43140)
> #12 thread_proxy (impalad+0x3a0dbf6)
> {code}
> {code}
> Previous read of size 4 at 0x7b54000822d4 by thread T3 (mutexes: write
> M3193, write M165007):
> #0 impala::QueryState::Init(impala::ExecQueryFInstancesRequestPB const*,
>
[jira] [Commented] (IMPALA-12635) Tests fail with "Memory is likely oversubscribed"
[
https://issues.apache.org/jira/browse/IMPALA-12635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796956#comment-17796956
]
Andrew Sherman commented on IMPALA-12635:
-
Thanks [~baggio000] sorry I saw you were owner of the IMPALA-11805 jira and
assumed you had also made the change
> Tests fail with "Memory is likely oversubscribed"
> -
>
> Key: IMPALA-12635
> URL: https://issues.apache.org/jira/browse/IMPALA-12635
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 1.4.2
>Reporter: Andrew Sherman
>Assignee: Wenzhe Zhou
>Priority: Critical
>
> 3 different tests failed trying to allocate memory
> {code}
> E OperationalError: Failed to get minimum memory reservation of 94.11 MB on
> daemon impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000 for
> query a94be0ce8a16884d:cef55b1a due to following error: Memory limit
> exceeded: Could not allocate memory while trying to increase reservation.
> E Query(a94be0ce8a16884d:cef55b1a) could not allocate 94.11 MB
> without exceeding limit.
> E Error occurred on backend
> impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000
> E Memory left in process limit: 1.24 GB
> E Query(a94be0ce8a16884d:cef55b1a): Reservation=0
> ReservationLimit=9.60 GB OtherMemory=0 Total=0 Peak=0
> E Memory is likely oversubscribed. Reducing query concurrency or
> configuring admission control may help avoid this error.
> {code}
> {code}
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EQuery aborted:Failed to get minimum memory reservation of 4.03 MB on
> daemon impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000 for
> query 3e4b9c2bd24a8c30:5db8f3e1 due to following error: Memory limit
> exceeded: Could not allocate memory while trying to increase reservation.
> E Query(3e4b9c2bd24a8c30:5db8f3e1) could not allocate 4.03 MB
> without exceeding limit.
> E Error occurred on backend
> impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000
> E Memory left in process limit: 1.20 GB
> E Query(3e4b9c2bd24a8c30:5db8f3e1): Reservation=0
> ReservationLimit=9.60 GB OtherMemory=0 Total=0 Peak=0
> E Memory is likely oversubscribed. Reducing query concurrency or
> configuring admission control may help avoid this error.
> {code}
> {code}
> E OperationalError: Failed to get minimum memory reservation of 284.18 MB
> on daemon impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000
> for query 054cab730f0605ca:6e70e3dd due to following error: Memory
> limit exceeded: Could not allocate memory while trying to increase
> reservation.
> E Query(054cab730f0605ca:6e70e3dd) could not allocate 284.18 MB
> without exceeding limit.
> E Error occurred on backend
> impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000
> E Memory left in process limit: 1.34 GB
> E Query(054cab730f0605ca:6e70e3dd): Reservation=0
> ReservationLimit=9.60 GB OtherMemory=0 Total=0 Peak=0
> E Memory is likely oversubscribed. Reducing query concurrency or
> configuring admission control may help avoid this error.
> {code}
> [~wzhou] I'm randomly picking on you to own this. This has only happened once.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12635) Tests fail with "Memory is likely oversubscribed"
[
https://issues.apache.org/jira/browse/IMPALA-12635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17796942#comment-17796942
]
Andrew Sherman commented on IMPALA-12635:
-
Oversubscription has happened in 3 test runs against master. All 3 runs
included the following new changes:
IMPALA-12313: (part 2) Limited UPDATE support for Iceberg tables
IMPALA-12595: Allow automatic removal of old logs from previous PID
IMPALA-12398: Fix Ranger role not exists when altering db/table/view
IMPALA-12001: Informative error message for complex types with DISTINCT
IMPALA-11805: Fix LLVM memory manager bytes allocated
IMPALA-12426: Adds the backend InternalServer class.
I suppose IMPALA-11805 is the most suspicious change as it changes memory
consumption estimates, what do you think [~baggio000]
> Tests fail with "Memory is likely oversubscribed"
> -
>
> Key: IMPALA-12635
> URL: https://issues.apache.org/jira/browse/IMPALA-12635
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 1.4.2
>Reporter: Andrew Sherman
>Assignee: Wenzhe Zhou
>Priority: Critical
>
> 3 different tests failed trying to allocate memory
> {code}
> E OperationalError: Failed to get minimum memory reservation of 94.11 MB on
> daemon impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000 for
> query a94be0ce8a16884d:cef55b1a due to following error: Memory limit
> exceeded: Could not allocate memory while trying to increase reservation.
> E Query(a94be0ce8a16884d:cef55b1a) could not allocate 94.11 MB
> without exceeding limit.
> E Error occurred on backend
> impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000
> E Memory left in process limit: 1.24 GB
> E Query(a94be0ce8a16884d:cef55b1a): Reservation=0
> ReservationLimit=9.60 GB OtherMemory=0 Total=0 Peak=0
> E Memory is likely oversubscribed. Reducing query concurrency or
> configuring admission control may help avoid this error.
> {code}
> {code}
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EQuery aborted:Failed to get minimum memory reservation of 4.03 MB on
> daemon impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000 for
> query 3e4b9c2bd24a8c30:5db8f3e1 due to following error: Memory limit
> exceeded: Could not allocate memory while trying to increase reservation.
> E Query(3e4b9c2bd24a8c30:5db8f3e1) could not allocate 4.03 MB
> without exceeding limit.
> E Error occurred on backend
> impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000
> E Memory left in process limit: 1.20 GB
> E Query(3e4b9c2bd24a8c30:5db8f3e1): Reservation=0
> ReservationLimit=9.60 GB OtherMemory=0 Total=0 Peak=0
> E Memory is likely oversubscribed. Reducing query concurrency or
> configuring admission control may help avoid this error.
> {code}
> {code}
> E OperationalError: Failed to get minimum memory reservation of 284.18 MB
> on daemon impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000
> for query 054cab730f0605ca:6e70e3dd due to following error: Memory
> limit exceeded: Could not allocate memory while trying to increase
> reservation.
> E Query(054cab730f0605ca:6e70e3dd) could not allocate 284.18 MB
> without exceeding limit.
> E Error occurred on backend
> impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000
> E Memory left in process limit: 1.34 GB
> E Query(054cab730f0605ca:6e70e3dd): Reservation=0
> ReservationLimit=9.60 GB OtherMemory=0 Total=0 Peak=0
> E Memory is likely oversubscribed. Reducing query concurrency or
> configuring admission control may help avoid this error.
> {code}
> [~wzhou] I'm randomly picking on you to own this. This has only happened once.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12635) Tests fail with "Memory is likely oversubscribed"
Andrew Sherman created IMPALA-12635:
---
Summary: Tests fail with "Memory is likely oversubscribed"
Key: IMPALA-12635
URL: https://issues.apache.org/jira/browse/IMPALA-12635
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 1.4.2
Reporter: Andrew Sherman
Assignee: Wenzhe Zhou
3 different tests failed trying to allocate memory
{code}
E OperationalError: Failed to get minimum memory reservation of 94.11 MB on
daemon impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000 for
query a94be0ce8a16884d:cef55b1a due to following error: Memory limit
exceeded: Could not allocate memory while trying to increase reservation.
E Query(a94be0ce8a16884d:cef55b1a) could not allocate 94.11 MB
without exceeding limit.
E Error occurred on backend
impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000
E Memory left in process limit: 1.24 GB
E Query(a94be0ce8a16884d:cef55b1a): Reservation=0
ReservationLimit=9.60 GB OtherMemory=0 Total=0 Peak=0
E Memory is likely oversubscribed. Reducing query concurrency or configuring
admission control may help avoid this error.
{code}
{code}
E ImpalaBeeswaxException: ImpalaBeeswaxException:
EQuery aborted:Failed to get minimum memory reservation of 4.03 MB on
daemon impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000 for
query 3e4b9c2bd24a8c30:5db8f3e1 due to following error: Memory limit
exceeded: Could not allocate memory while trying to increase reservation.
E Query(3e4b9c2bd24a8c30:5db8f3e1) could not allocate 4.03 MB without
exceeding limit.
E Error occurred on backend
impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000
E Memory left in process limit: 1.20 GB
E Query(3e4b9c2bd24a8c30:5db8f3e1): Reservation=0
ReservationLimit=9.60 GB OtherMemory=0 Total=0 Peak=0
E Memory is likely oversubscribed. Reducing query concurrency or configuring
admission control may help avoid this error.
{code}
{code}
E OperationalError: Failed to get minimum memory reservation of 284.18 MB on
daemon impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000 for
query 054cab730f0605ca:6e70e3dd due to following error: Memory limit
exceeded: Could not allocate memory while trying to increase reservation.
E Query(054cab730f0605ca:6e70e3dd) could not allocate 284.18 MB
without exceeding limit.
E Error occurred on backend
impala-ec2-centos79-m6i-4xlarge-xldisk-0682.vpc.cloudera.com:27000
E Memory left in process limit: 1.34 GB
E Query(054cab730f0605ca:6e70e3dd): Reservation=0
ReservationLimit=9.60 GB OtherMemory=0 Total=0 Peak=0
E Memory is likely oversubscribed. Reducing query concurrency or configuring
admission control may help avoid this error.
{code}
[~wzhou] I'm randomly picking on you to own this. This has only happened once.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12634) DCHECK failure in ~ClientRequestState destructor
Andrew Sherman created IMPALA-12634:
---
Summary: DCHECK failure in ~ClientRequestState destructor
Key: IMPALA-12634
URL: https://issues.apache.org/jira/browse/IMPALA-12634
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 1.4.2
Reporter: Andrew Sherman
Assignee: Joe McDonnell
{code}
F1214 02:03:33.465935 20159 client-request-state.cc:179] Check failed:
wait_thread_.get() == __null BlockOnWait() needs to be called!
*** Check failure stack trace: ***
@ 0x385f0fd google::LogMessage::Fail()
@ 0x3861034 google::LogMessage::SendToLog()
@ 0x385eadc google::LogMessage::Flush()
@ 0x3861559 google::LogMessageFatal::~LogMessageFatal()
@ 0x17336c6 impala::ClientRequestState::~ClientRequestState()
@ 0x21fb1cc impala::QueryDriver::~QueryDriver()
@ 0x22024d1 std::_Sp_counted_ptr_inplace<>::_M_dispose()
@ 0x1646828 impala::ThreadPool<>::WorkerThread()
@ 0x16277f3
boost::detail::function::void_function_obj_invoker0<>::invoke()
@ 0x1946dc2 impala::Thread::SuperviseThread()
@ 0x1947bcb boost::detail::thread_data<>::run()
@ 0x2380ce7 thread_proxy
@ 0x7f65dcd4eea5 start_thread
@ 0x7f65d9688b0d __clone
{code}
[~joemcdonnell] can you take a look, this has only happened once
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12633) DCHECK failure on query_handle->is_inflight()
Andrew Sherman created IMPALA-12633:
---
Summary: DCHECK failure on query_handle->is_inflight()
Key: IMPALA-12633
URL: https://issues.apache.org/jira/browse/IMPALA-12633
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 1.4.2
Reporter: Andrew Sherman
Assignee: Michael Smith
Attachments: 951ee391-85bd-4fdb-b43550a1-7564a485.dmp_dumped
This is a new DCHECK introduced in IMPALA-12493
{code}
12 impalad!google::LogMessageFatal::~LogMessageFatal() + 0x9
rsp = 0x7fd0d4e5d2b0 rip = 0x038f8ad9
Found by: stack scanning
13 impalad!impala::QueryDriver::RetryQueryFromThread(impala::Status
const, std::shared_ptrimpala::QueryDriver) [query-driver.cc : 376
+ 0xc]
rsp = 0x7fd0d4e5d2c0 rip = 0x02295372
Found by: stack scanning
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12632) TSAN detected data race accessing CpuUsageRatios
Andrew Sherman created IMPALA-12632:
---
Summary: TSAN detected data race accessing CpuUsageRatios
Key: IMPALA-12632
URL: https://issues.apache.org/jira/browse/IMPALA-12632
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 1.4.2
Reporter: Andrew Sherman
Assignee: Andrew Sherman
This looks like a similar problem to IMPALA-12614, which can hopefully be fixed
with the same strategy of using AtomicInt32 for the components in
CpuUsageRatios.
WARNING: ThreadSanitizer: data race (pid=7397)
{code}
Write of size 4 at 0x7b54000822d4 by thread T4 (mutexes: write M3360):
#0 impala::SystemStateInfo::ComputeCpuRatios()
/data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/util/system-state-info.cc:149:20
(impalad+0x2cab247)
#1 impala::SystemStateInfo::CaptureSystemStateSnapshot()
/data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/util/system-state-info.cc:95:3
(impalad+0x2caaf3c)
#2 impala::ExecEnv::InitSystemStateInfo()::$_8::operator()() const
/data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/runtime/exec-env.cc:639:8
(impalad+0x2577161)
#3 void std::__invoke_impl(std::__invoke_other,
impala::ExecEnv::InitSystemStateInfo()::$_8&)
/data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/invoke.h:60:14
(impalad+0x25770f1)
#4 std::enable_if, void>::type
std::__invoke_r(impala::ExecEnv::InitSystemStateInfo()::$_8&)
/data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/invoke.h:110:2
(impalad+0x2577071)
#5 std::_Function_handler::_M_invoke(std::_Any_data const&)
/data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/std_function.h:291:9
(impalad+0x2576f41)
#6 std::function::operator()() const
/data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/std_function.h:622:14
(impalad+0x23dee3c)
#7
impala::PeriodicCounterUpdater::UpdateLoop(impala::PeriodicCounterUpdater*)
/data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/util/periodic-counter-updater.cc:175:39
(impalad+0x2c3e6d6)
#8 boost::_mfi::mf1::operator()(impala::PeriodicCounterUpdater*,
impala::PeriodicCounterUpdater*) const
/data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/mem_fn_template.hpp:165:29
(impalad+0x2c4382d)
#9 void
boost::_bi::list2,
boost::_bi::value
>::operator(), boost::_bi::list0>(boost::_bi::type,
boost::_mfi::mf1&, boost::_bi::list0&, int)
/data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:319:9
(impalad+0x2c4376d)
#10 boost::_bi::bind_t,
boost::_bi::list2,
boost::_bi::value > >::operator()()
/data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:1294:16
(impalad+0x2c436d3)
#11 boost::detail::thread_data,
boost::_bi::list2,
boost::_bi::value > > >::run()
/data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/thread/detail/thread.hpp:120:17
(impalad+0x2c43140)
#12 thread_proxy (impalad+0x3a0dbf6)
{code}
{code}
Previous read of size 4 at 0x7b54000822d4 by thread T3 (mutexes: write M3193,
write M165007):
#0 impala::QueryState::Init(impala::ExecQueryFInstancesRequestPB const*,
impala::TExecPlanFragmentInfo const&)::$_8::operator()() const
/data/jenkins/workspace/impala-cdw-master-staging-core-tsan/repos/Impala/be/src/runtime/query-state.cc:223:55
(impalad+0x25f70cc)
#1
boost::detail::function::function_obj_invoker0::invoke(boost::detail::function::function_buffer&)
/data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:137:18
(impalad+0x25f6ea9)
#2 boost::function0::operator()() const
/data/jenkins/workspace/impala-cdw-master-staging-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:763:14
(impalad+0x2c40031)
#3 impala::RuntimeProfile::TimeSeriesCounter::AddSample(int)
[jira] [Created] (IMPALA-12629) TestRanger fails in test_column_masking and test_row_filtering
Andrew Sherman created IMPALA-12629:
---
Summary: TestRanger fails in test_column_masking and
test_row_filtering
Key: IMPALA-12629
URL: https://issues.apache.org/jira/browse/IMPALA-12629
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 1.4.2
Reporter: Andrew Sherman
Assignee: Fang-Yu Rao
Fang-Yu asking you take a look because you reviewed IMPALA-12398 which is the
suspicious recent change.
test_column_masking:
{code}
authorization/test_ranger.py:1510: in test_column_masking
test_file_vars={'$UNIQUE_DB': unique_database})
common/impala_test_suite.py:751: in run_test_case
self.__verify_results_and_errors(vector, test_section, result, use_db)
common/impala_test_suite.py:587: in __verify_results_and_errors
replace_filenames_with_placeholder)
common/test_result_verifier.py:487: in verify_raw_results
VERIFIER_MAP[verifier](expected, actual)
common/test_result_verifier.py:296: in verify_query_result_is_equal
assert expected_results == actual_results
E assert Comparing QueryTestResults (expected vs actual):
E 1,'NULL' == 1,'NULL'
E 3,'NULL' != 2,'NULL'
E Number of rows returned (expected vs actual): 2 != 3
{code}
test_row_filtering:
{code}
authorization/test_ranger.py:1685: in test_row_filtering
test_file_vars={'$UNIQUE_DB': unique_database})
common/impala_test_suite.py:751: in run_test_case
self.__verify_results_and_errors(vector, test_section, result, use_db)
common/impala_test_suite.py:587: in __verify_results_and_errors
replace_filenames_with_placeholder)
common/test_result_verifier.py:487: in verify_raw_results
VERIFIER_MAP[verifier](expected, actual)
common/test_result_verifier.py:296: in verify_query_result_is_equal
assert expected_results == actual_results
E assert Comparing QueryTestResults (expected vs actual):
E 1,'a' == 1,'a'
E 3,'c' == 3,'c'
E 5,'X' == 5,'X'
E Number of rows returned (expected vs actual): 3 != 6
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12628) iceberg negative test fail saying can't connect but there is no obvious server crash
Andrew Sherman created IMPALA-12628:
---
Summary: iceberg negative test fail saying can't connect but there
is no obvious server crash
Key: IMPALA-12628
URL: https://issues.apache.org/jira/browse/IMPALA-12628
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 1.4.2
Reporter: Andrew Sherman
Assignee: Zoltán Borók-Nagy
This is mysterious, I'm capturing as far as I've got.
The run of iceberg-negative.test fails in the middle.
Did the server crash? I haven't seen that yet.
{code}
-- executing against localhost:21000
update functional_parquet.iceberg_int_partitioned set input__file__name =
'void';
-- executing against localhost:21000
update functional_parquet.iceberg_int_partitioned set file__position = 42;
-- connecting to localhost:11050 with impyla
-- 2023-12-13 09:57:48,028 INFO MainThread: Could not connect to ('::1',
11050, 0, 0)
Traceback (most recent call last):
File
"/data/jenkins/workspace/impala-asf-master-core-ozone/Impala-Toolchain/toolchain-packages-gcc10.4.0/thrift-0.16.0-p6/python/lib/python2.7/site-packages/thrift/transport/TSocket.py",
line 137, in open
handle.connect(sockaddr)
File
"/data/jenkins/workspace/impala-asf-master-core-ozone/Impala-Toolchain/toolchain-packages-gcc10.4.0/python-2.7.16/lib/python2.7/socket.py",
line 228, in meth
return getattr(self._sock,name)(*args)
error: [Errno 111] Connection refused
-- 2023-12-13 09:57:48,029 INFO MainThread: Could not connect to
('127.0.0.1', 11050)
Traceback (most recent call last):
File
"/data/jenkins/workspace/impala-asf-master-core-ozone/Impala-Toolchain/toolchain-packages-gcc10.4.0/thrift-0.16.0-p6/python/lib/python2.7/site-packages/thrift/transport/TSocket.py",
line 137, in open
handle.connect(sockaddr)
File
"/data/jenkins/workspace/impala-asf-master-core-ozone/Impala-Toolchain/toolchain-packages-gcc10.4.0/python-2.7.16/lib/python2.7/socket.py",
line 228, in meth
return getattr(self._sock,name)(*args)
error: [Errno 111] Connection refused
-- 2023-12-13 09:57:48,029 ERRORMainThread: Could not connect to any of
[('::1', 11050, 0, 0), ('127.0.0.1', 11050)]
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12619) Update Impala website to reduce emphasis on Hadoop
Andrew Sherman created IMPALA-12619: --- Summary: Update Impala website to reduce emphasis on Hadoop Key: IMPALA-12619 URL: https://issues.apache.org/jira/browse/IMPALA-12619 Project: IMPALA Issue Type: Bug Components: Docs Affects Versions: Impala 1.4.2 Reporter: Andrew Sherman Assignee: Andrew Sherman The Impala website at ASF [https://impala.apache.org/] is the first hit returned for “Apache Impala”. Update the first line of the description to say ="Apache Impala is a modern, open source, distributed SQL query engine for open data formats." instead of "Apache Impala is a modern, open source, distributed SQL query engine for Apache Hadoop." Also mention Ranger instead of Sentry, and add references to Iceberg. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12618) Update README.md to reduce emphasis on Hadoop
Andrew Sherman created IMPALA-12618: --- Summary: Update README.md to reduce emphasis on Hadoop Key: IMPALA-12618 URL: https://issues.apache.org/jira/browse/IMPALA-12618 Project: IMPALA Issue Type: Bug Components: Docs Affects Versions: Impala 1.4.2 Reporter: Andrew Sherman Assignee: Andrew Sherman The README.md file is displayed on the github home page [https://github.com/apache/impala] Change this so that the opening line mentions “data stored in open data formats” instead of “data stored in Apache Hadoop clusters“. Also add Iceberg as the first mentioned place where data can be stored. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12616) test_restart_catalogd_while_handling_rpc_response* tests fail not reaching expected states
[
https://issues.apache.org/jira/browse/IMPALA-12616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman reassigned IMPALA-12616:
---
Assignee: Daniel Becker
> test_restart_catalogd_while_handling_rpc_response* tests fail not reaching
> expected states
> --
>
> Key: IMPALA-12616
> URL: https://issues.apache.org/jira/browse/IMPALA-12616
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 1.4.2
>Reporter: Andrew Sherman
>Assignee: Daniel Becker
>Priority: Critical
>
> There are failures in both
> custom_cluster.test_restart_services.TestRestart.test_restart_catalogd_while_handling_rpc_response_with_timeout
> and
> custom_cluster.test_restart_services.TestRestart.test_restart_catalogd_while_handling_rpc_response_with_max_iters,
> both look the same:
> {code:java}
> custom_cluster/test_restart_services.py:232: in
> test_restart_catalogd_while_handling_rpc_response_with_timeout
> self.wait_for_state(handle, self.client.QUERY_STATES["FINISHED"],
> max_wait_time)
> common/impala_test_suite.py:1181: in wait_for_state
> self.wait_for_any_state(handle, [expected_state], timeout, client)
> common/impala_test_suite.py:1199: in wait_for_any_state
> raise Timeout(timeout_msg)
> E Timeout: query '6a4e0bad9b511ccf:bf93de68' did not reach one of
> the expected states [4], last known state 5
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12616) test_restart_catalogd_while_handling_rpc_response* tests fail not reaching expected states
[
https://issues.apache.org/jira/browse/IMPALA-12616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795512#comment-17795512
]
Andrew Sherman commented on IMPALA-12616:
-
[~daniel.becker] please can you take a look?
> test_restart_catalogd_while_handling_rpc_response* tests fail not reaching
> expected states
> --
>
> Key: IMPALA-12616
> URL: https://issues.apache.org/jira/browse/IMPALA-12616
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 1.4.2
>Reporter: Andrew Sherman
>Assignee: Daniel Becker
>Priority: Critical
>
> There are failures in both
> custom_cluster.test_restart_services.TestRestart.test_restart_catalogd_while_handling_rpc_response_with_timeout
> and
> custom_cluster.test_restart_services.TestRestart.test_restart_catalogd_while_handling_rpc_response_with_max_iters,
> both look the same:
> {code:java}
> custom_cluster/test_restart_services.py:232: in
> test_restart_catalogd_while_handling_rpc_response_with_timeout
> self.wait_for_state(handle, self.client.QUERY_STATES["FINISHED"],
> max_wait_time)
> common/impala_test_suite.py:1181: in wait_for_state
> self.wait_for_any_state(handle, [expected_state], timeout, client)
> common/impala_test_suite.py:1199: in wait_for_any_state
> raise Timeout(timeout_msg)
> E Timeout: query '6a4e0bad9b511ccf:bf93de68' did not reach one of
> the expected states [4], last known state 5
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12616) test_restart_catalogd_while_handling_rpc_response* tests fail not reaching expected states
Andrew Sherman created IMPALA-12616:
---
Summary: test_restart_catalogd_while_handling_rpc_response* tests
fail not reaching expected states
Key: IMPALA-12616
URL: https://issues.apache.org/jira/browse/IMPALA-12616
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 1.4.2
Reporter: Andrew Sherman
There are failures in both
custom_cluster.test_restart_services.TestRestart.test_restart_catalogd_while_handling_rpc_response_with_timeout
and
custom_cluster.test_restart_services.TestRestart.test_restart_catalogd_while_handling_rpc_response_with_max_iters,
both look the same:
{code:java}
custom_cluster/test_restart_services.py:232: in
test_restart_catalogd_while_handling_rpc_response_with_timeout
self.wait_for_state(handle, self.client.QUERY_STATES["FINISHED"],
max_wait_time)
common/impala_test_suite.py:1181: in wait_for_state
self.wait_for_any_state(handle, [expected_state], timeout, client)
common/impala_test_suite.py:1199: in wait_for_any_state
raise Timeout(timeout_msg)
E Timeout: query '6a4e0bad9b511ccf:bf93de68' did not reach one of the
expected states [4], last known state 5
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11793) testDisableEventSyncFlag frontend test failed
[
https://issues.apache.org/jira/browse/IMPALA-11793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated IMPALA-11793:
Priority: Critical (was: Major)
> testDisableEventSyncFlag frontend test failed
> -
>
> Key: IMPALA-11793
> URL: https://issues.apache.org/jira/browse/IMPALA-11793
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.3.0
>Reporter: Michael Smith
>Priority: Critical
> Labels: flaky
>
> Frontend test
> org.apache.impala.catalog.events.MetastoreEventsProcessorTest.testDisableEventSyncFlag
> failed with
> {code}
> java.lang.AssertionError: expected:<6> but was:<7>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:834)
> at org.junit.Assert.assertEquals(Assert.java:645)
> at org.junit.Assert.assertEquals(Assert.java:631)
> at
> org.apache.impala.catalog.events.MetastoreEventsProcessorTest.testDDLOpUsingEvent(MetastoreEventsProcessorTest.java:2127)
> at
> org.apache.impala.catalog.events.MetastoreEventsProcessorTest.runDDLTestsWithFlags(MetastoreEventsProcessorTest.java:2061)
> at
> org.apache.impala.catalog.events.MetastoreEventsProcessorTest.testDisableEventSyncFlag(MetastoreEventsProcessorTest.java:2016)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)
> {code}
> This happened in an exhaustive test run with data cache enabled after merging
> * IMPALA-11751: (Addendum) fix test for Ozone
> * IMPALA-11780: Wrong FILE__POSITION values for multi row group Parquet
> * IMPALA-10986: Require the SELECT privilege to execute a UDF
> though I don't immediately see any connection between them.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12593) query_test.test_iceberg.TestIcebergTable.test_drop_partition fails on S3
[ https://issues.apache.org/jira/browse/IMPALA-12593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated IMPALA-12593: Priority: Critical (was: Major) > query_test.test_iceberg.TestIcebergTable.test_drop_partition fails on S3 > > > Key: IMPALA-12593 > URL: https://issues.apache.org/jira/browse/IMPALA-12593 > Project: IMPALA > Issue Type: Bug >Affects Versions: Impala 4.4.0 >Reporter: Peter Rozsa >Assignee: Peter Rozsa >Priority: Critical > > query_test.test_iceberg.TestIcebergTable.test_drop_partition fails on S3, > because datafiles under the table created during the test have longer file > paths than S3 allows. > The test table should be split into multiple tables. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12606) Sporadic failures around query_test.test_queries.TestQueries.test_intersect
[
https://issues.apache.org/jira/browse/IMPALA-12606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated IMPALA-12606:
Priority: Critical (was: Major)
> Sporadic failures around query_test.test_queries.TestQueries.test_intersect
> ---
>
> Key: IMPALA-12606
> URL: https://issues.apache.org/jira/browse/IMPALA-12606
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.4.0
>Reporter: Peter Rozsa
>Assignee: Zoltán Borók-Nagy
>Priority: Critical
>
> test_intersect fails with the following setup randomly through multiple
> executions:
> {code:java}
> set
> DEBUG_ACTION="BEFORE_CODEGEN_IN_ASYNC_CODEGEN_THREAD:JITTER@1000|AFTER_STARTING_ASYNC_CODEGEN_IN_FRAGMENT_THREAD:JITTER@1000"
> {code}
> Failing query:
> {code:java}
> select group_str, null_str from nullrows where bool_nulls = true intersect
> select group_str, null_str from nullrows where bool_nulls = false {code}
> The bisecting points to 0e5374e080 which is IMPALA-12373.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-12614) TSAN detected data race in PeriodicCounterUpdater
[
https://issues.apache.org/jira/browse/IMPALA-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17794883#comment-17794883
]
Andrew Sherman edited comment on IMPALA-12614 at 12/8/23 11:56 PM:
---
[~kdeschle] assigning to you take a look as this may be fallout from
IMPALA-12385 (edited to fix issue)
was (Author: asherman):
[~kdeschle] assigning to you take a look as this may be fallout from
IMPALA-12614
> TSAN detected data race in PeriodicCounterUpdater
> -
>
> Key: IMPALA-12614
> URL: https://issues.apache.org/jira/browse/IMPALA-12614
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.3.0
>Reporter: Andrew Sherman
>Assignee: Kurt Deschler
>Priority: Critical
>
> A TSAN test run detected a data race:
> READ:
> {code}
> WARNING: ThreadSanitizer: data race (pid=2200)
> Read of size 8 at 0x7b54d6b0 by thread T3 (mutexes: write M3189, write
> M1141235156294569304):
> #0 impala::QueryState::Init(impala::ExecQueryFInstancesRequestPB const*,
> impala::TExecPlanFragmentInfo const&)::$_14::operator()() const
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/runtime/query-state.cc:249:50
> (impalad+0x25db87d)
> #1
> boost::detail::function::function_obj_invoker0 const*, impala::TExecPlanFragmentInfo const&)::$_14,
> long>::invoke(boost::detail::function::function_buffer&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:137:18
> (impalad+0x25db659)
> #2 boost::function0::operator()() const
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:763:14
> (impalad+0x2c1bee1)
> #3 impala::RuntimeProfile::TimeSeriesCounter::AddSample(int)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/runtime-profile.cc:2050:20
> (impalad+0x2c3e98e)
> #4
> impala::PeriodicCounterUpdater::UpdateLoop(impala::PeriodicCounterUpdater*)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/periodic-counter-updater.cc:229:16
> (impalad+0x2c1a4b2)
> #5 boost::_mfi::mf1 impala::PeriodicCounterUpdater*>::operator()(impala::PeriodicCounterUpdater*,
> impala::PeriodicCounterUpdater*) const
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/mem_fn_template.hpp:165:29
> (impalad+0x2c1f6dd)
> #6 void
> boost::_bi::list2,
> boost::_bi::value
> >::operator() impala::PeriodicCounterUpdater*>, boost::_bi::list0>(boost::_bi::type,
> boost::_mfi::mf1 impala::PeriodicCounterUpdater*>&, boost::_bi::list0&, int)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:319:9
> (impalad+0x2c1f61d)
> #7 boost::_bi::bind_t impala::PeriodicCounterUpdater, impala::PeriodicCounterUpdater*>,
> boost::_bi::list2,
> boost::_bi::value > >::operator()()
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:1294:16
> (impalad+0x2c1f583)
> #8 boost::detail::thread_data boost::_mfi::mf1 impala::PeriodicCounterUpdater*>,
> boost::_bi::list2,
> boost::_bi::value > > >::run()
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/thread/detail/thread.hpp:120:17
> (impalad+0x2c1eff0)
> #9 thread_proxy (impalad+0x39e5ef6)
> {code}
> WRITE:
> {code}
> Previous write of size 8 at 0x7b54d6b0 by thread T4 (mutexes: write
> M3356):
> #0 impala::SystemStateInfo::ComputeDiskStats(long)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/system-state-info.cc:269:26
> (impalad+0x2c87423)
> #1 impala::SystemStateInfo::CaptureSystemStateSnapshot()
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/system-state-info.cc:106:3
> (impalad+0x2c86eeb)
> #2 impala::ExecEnv::InitSystemStateInfo()::$_8::operator()() const
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/runtime/exec-env.cc:639:8
> (impalad+0x2559fe1)
> #3 void std::__invoke_impl impala::ExecEnv::InitSystemStateInfo()::$_8&>(std::__invoke_other,
> impala::ExecEnv::InitSystemStateInfo()::$_8&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/invoke.h:60:14
> (impalad+0x2559f71)
> #4 std::enable_if
[jira] [Assigned] (IMPALA-12614) TSAN detected data race in PeriodicCounterUpdater
[
https://issues.apache.org/jira/browse/IMPALA-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman reassigned IMPALA-12614:
---
Assignee: Kurt Deschler
> TSAN detected data race in PeriodicCounterUpdater
> -
>
> Key: IMPALA-12614
> URL: https://issues.apache.org/jira/browse/IMPALA-12614
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.3.0
>Reporter: Andrew Sherman
>Assignee: Kurt Deschler
>Priority: Critical
>
> A TSAN test run detected a data race:
> READ:
> {code}
> WARNING: ThreadSanitizer: data race (pid=2200)
> Read of size 8 at 0x7b54d6b0 by thread T3 (mutexes: write M3189, write
> M1141235156294569304):
> #0 impala::QueryState::Init(impala::ExecQueryFInstancesRequestPB const*,
> impala::TExecPlanFragmentInfo const&)::$_14::operator()() const
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/runtime/query-state.cc:249:50
> (impalad+0x25db87d)
> #1
> boost::detail::function::function_obj_invoker0 const*, impala::TExecPlanFragmentInfo const&)::$_14,
> long>::invoke(boost::detail::function::function_buffer&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:137:18
> (impalad+0x25db659)
> #2 boost::function0::operator()() const
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:763:14
> (impalad+0x2c1bee1)
> #3 impala::RuntimeProfile::TimeSeriesCounter::AddSample(int)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/runtime-profile.cc:2050:20
> (impalad+0x2c3e98e)
> #4
> impala::PeriodicCounterUpdater::UpdateLoop(impala::PeriodicCounterUpdater*)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/periodic-counter-updater.cc:229:16
> (impalad+0x2c1a4b2)
> #5 boost::_mfi::mf1 impala::PeriodicCounterUpdater*>::operator()(impala::PeriodicCounterUpdater*,
> impala::PeriodicCounterUpdater*) const
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/mem_fn_template.hpp:165:29
> (impalad+0x2c1f6dd)
> #6 void
> boost::_bi::list2,
> boost::_bi::value
> >::operator() impala::PeriodicCounterUpdater*>, boost::_bi::list0>(boost::_bi::type,
> boost::_mfi::mf1 impala::PeriodicCounterUpdater*>&, boost::_bi::list0&, int)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:319:9
> (impalad+0x2c1f61d)
> #7 boost::_bi::bind_t impala::PeriodicCounterUpdater, impala::PeriodicCounterUpdater*>,
> boost::_bi::list2,
> boost::_bi::value > >::operator()()
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:1294:16
> (impalad+0x2c1f583)
> #8 boost::detail::thread_data boost::_mfi::mf1 impala::PeriodicCounterUpdater*>,
> boost::_bi::list2,
> boost::_bi::value > > >::run()
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/thread/detail/thread.hpp:120:17
> (impalad+0x2c1eff0)
> #9 thread_proxy (impalad+0x39e5ef6)
> {code}
> WRITE:
> {code}
> Previous write of size 8 at 0x7b54d6b0 by thread T4 (mutexes: write
> M3356):
> #0 impala::SystemStateInfo::ComputeDiskStats(long)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/system-state-info.cc:269:26
> (impalad+0x2c87423)
> #1 impala::SystemStateInfo::CaptureSystemStateSnapshot()
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/system-state-info.cc:106:3
> (impalad+0x2c86eeb)
> #2 impala::ExecEnv::InitSystemStateInfo()::$_8::operator()() const
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/runtime/exec-env.cc:639:8
> (impalad+0x2559fe1)
> #3 void std::__invoke_impl impala::ExecEnv::InitSystemStateInfo()::$_8&>(std::__invoke_other,
> impala::ExecEnv::InitSystemStateInfo()::$_8&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/invoke.h:60:14
> (impalad+0x2559f71)
> #4 std::enable_if impala::ExecEnv::InitSystemStateInfo()::$_8&>, void>::type
> std::__invoke_r impala::ExecEnv::InitSystemStateInfo()::$_8&>(impala::ExecEnv::InitSystemStateInfo()::$_8&)
>
>
[jira] [Commented] (IMPALA-12614) TSAN detected data race in PeriodicCounterUpdater
[
https://issues.apache.org/jira/browse/IMPALA-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17794883#comment-17794883
]
Andrew Sherman commented on IMPALA-12614:
-
[~kdeschle] assigning to you take a look as this may be fallout from
IMPALA-12614
> TSAN detected data race in PeriodicCounterUpdater
> -
>
> Key: IMPALA-12614
> URL: https://issues.apache.org/jira/browse/IMPALA-12614
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.3.0
>Reporter: Andrew Sherman
>Priority: Critical
>
> A TSAN test run detected a data race:
> READ:
> {code}
> WARNING: ThreadSanitizer: data race (pid=2200)
> Read of size 8 at 0x7b54d6b0 by thread T3 (mutexes: write M3189, write
> M1141235156294569304):
> #0 impala::QueryState::Init(impala::ExecQueryFInstancesRequestPB const*,
> impala::TExecPlanFragmentInfo const&)::$_14::operator()() const
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/runtime/query-state.cc:249:50
> (impalad+0x25db87d)
> #1
> boost::detail::function::function_obj_invoker0 const*, impala::TExecPlanFragmentInfo const&)::$_14,
> long>::invoke(boost::detail::function::function_buffer&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:137:18
> (impalad+0x25db659)
> #2 boost::function0::operator()() const
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:763:14
> (impalad+0x2c1bee1)
> #3 impala::RuntimeProfile::TimeSeriesCounter::AddSample(int)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/runtime-profile.cc:2050:20
> (impalad+0x2c3e98e)
> #4
> impala::PeriodicCounterUpdater::UpdateLoop(impala::PeriodicCounterUpdater*)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/periodic-counter-updater.cc:229:16
> (impalad+0x2c1a4b2)
> #5 boost::_mfi::mf1 impala::PeriodicCounterUpdater*>::operator()(impala::PeriodicCounterUpdater*,
> impala::PeriodicCounterUpdater*) const
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/mem_fn_template.hpp:165:29
> (impalad+0x2c1f6dd)
> #6 void
> boost::_bi::list2,
> boost::_bi::value
> >::operator() impala::PeriodicCounterUpdater*>, boost::_bi::list0>(boost::_bi::type,
> boost::_mfi::mf1 impala::PeriodicCounterUpdater*>&, boost::_bi::list0&, int)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:319:9
> (impalad+0x2c1f61d)
> #7 boost::_bi::bind_t impala::PeriodicCounterUpdater, impala::PeriodicCounterUpdater*>,
> boost::_bi::list2,
> boost::_bi::value > >::operator()()
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:1294:16
> (impalad+0x2c1f583)
> #8 boost::detail::thread_data boost::_mfi::mf1 impala::PeriodicCounterUpdater*>,
> boost::_bi::list2,
> boost::_bi::value > > >::run()
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/thread/detail/thread.hpp:120:17
> (impalad+0x2c1eff0)
> #9 thread_proxy (impalad+0x39e5ef6)
> {code}
> WRITE:
> {code}
> Previous write of size 8 at 0x7b54d6b0 by thread T4 (mutexes: write
> M3356):
> #0 impala::SystemStateInfo::ComputeDiskStats(long)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/system-state-info.cc:269:26
> (impalad+0x2c87423)
> #1 impala::SystemStateInfo::CaptureSystemStateSnapshot()
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/system-state-info.cc:106:3
> (impalad+0x2c86eeb)
> #2 impala::ExecEnv::InitSystemStateInfo()::$_8::operator()() const
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/runtime/exec-env.cc:639:8
> (impalad+0x2559fe1)
> #3 void std::__invoke_impl impala::ExecEnv::InitSystemStateInfo()::$_8&>(std::__invoke_other,
> impala::ExecEnv::InitSystemStateInfo()::$_8&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/invoke.h:60:14
> (impalad+0x2559f71)
> #4 std::enable_if impala::ExecEnv::InitSystemStateInfo()::$_8&>, void>::type
> std::__invoke_r impala::ExecEnv::InitSystemStateInfo()::$_8&>(impala::ExecEnv::InitSystemStateInfo()::$_8&)
>
>
[jira] [Created] (IMPALA-12614) TSAN detected data race in PeriodicCounterUpdater
Andrew Sherman created IMPALA-12614:
---
Summary: TSAN detected data race in PeriodicCounterUpdater
Key: IMPALA-12614
URL: https://issues.apache.org/jira/browse/IMPALA-12614
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 4.3.0
Reporter: Andrew Sherman
A TSAN test run detected a data race:
READ:
{code}
WARNING: ThreadSanitizer: data race (pid=2200)
Read of size 8 at 0x7b54d6b0 by thread T3 (mutexes: write M3189, write
M1141235156294569304):
#0 impala::QueryState::Init(impala::ExecQueryFInstancesRequestPB const*,
impala::TExecPlanFragmentInfo const&)::$_14::operator()() const
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/runtime/query-state.cc:249:50
(impalad+0x25db87d)
#1
boost::detail::function::function_obj_invoker0::invoke(boost::detail::function::function_buffer&)
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:137:18
(impalad+0x25db659)
#2 boost::function0::operator()() const
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:763:14
(impalad+0x2c1bee1)
#3 impala::RuntimeProfile::TimeSeriesCounter::AddSample(int)
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/runtime-profile.cc:2050:20
(impalad+0x2c3e98e)
#4
impala::PeriodicCounterUpdater::UpdateLoop(impala::PeriodicCounterUpdater*)
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/periodic-counter-updater.cc:229:16
(impalad+0x2c1a4b2)
#5 boost::_mfi::mf1::operator()(impala::PeriodicCounterUpdater*,
impala::PeriodicCounterUpdater*) const
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/mem_fn_template.hpp:165:29
(impalad+0x2c1f6dd)
#6 void
boost::_bi::list2,
boost::_bi::value
>::operator(), boost::_bi::list0>(boost::_bi::type,
boost::_mfi::mf1&, boost::_bi::list0&, int)
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:319:9
(impalad+0x2c1f61d)
#7 boost::_bi::bind_t,
boost::_bi::list2,
boost::_bi::value > >::operator()()
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/bind/bind.hpp:1294:16
(impalad+0x2c1f583)
#8 boost::detail::thread_data,
boost::_bi::list2,
boost::_bi::value > > >::run()
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/thread/detail/thread.hpp:120:17
(impalad+0x2c1eff0)
#9 thread_proxy (impalad+0x39e5ef6)
{code}
WRITE:
{code}
Previous write of size 8 at 0x7b54d6b0 by thread T4 (mutexes: write
M3356):
#0 impala::SystemStateInfo::ComputeDiskStats(long)
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/system-state-info.cc:269:26
(impalad+0x2c87423)
#1 impala::SystemStateInfo::CaptureSystemStateSnapshot()
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/system-state-info.cc:106:3
(impalad+0x2c86eeb)
#2 impala::ExecEnv::InitSystemStateInfo()::$_8::operator()() const
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/runtime/exec-env.cc:639:8
(impalad+0x2559fe1)
#3 void std::__invoke_impl(std::__invoke_other,
impala::ExecEnv::InitSystemStateInfo()::$_8&)
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/invoke.h:60:14
(impalad+0x2559f71)
#4 std::enable_if, void>::type
std::__invoke_r(impala::ExecEnv::InitSystemStateInfo()::$_8&)
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/invoke.h:110:2
(impalad+0x2559ef1)
#5 std::_Function_handler::_M_invoke(std::_Any_data const&)
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/std_function.h:291:9
(impalad+0x2559dc1)
#6 std::function::operator()() const
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/std_function.h:622:14
(impalad+0x23c171c)
#7
impala::PeriodicCounterUpdater::UpdateLoop(impala::PeriodicCounterUpdater*)
[jira] [Created] (IMPALA-12416) test_skipping_older_events and some other catlog tests failing
Andrew Sherman created IMPALA-12416:
---
Summary: test_skipping_older_events and some other catlog tests
failing
Key: IMPALA-12416
URL: https://issues.apache.org/jira/browse/IMPALA-12416
Project: IMPALA
Issue Type: Bug
Reporter: Andrew Sherman
Assignee: Sai Hemanth Gantasala
test_skipping_older_events was added in IMPALA-11535. The failure is:
{code}
custom_cluster/test_events_custom_configs.py:375: in test_skipping_older_events
verify_skipping_older_events(test_old_table, True, False)
custom_cluster/test_events_custom_configs.py:370: in
verify_skipping_older_events
verify_skipping_hive_stmt_events(complete_query, "new_table")
custom_cluster/test_events_custom_configs.py:341: in
verify_skipping_hive_stmt_events
assert tbl_events_skipped_after > tbl_events_skipped_before
E assert 19 > 19
{code}
There are some other catalog test failures that appeared at the same time:
{code}
org.apache.impala.catalog.metastore.CatalogHmsFileMetadataTest
org.apache.impala.catalog.metastore.CatalogHmsSyncToLatestEventIdTest
org.apache.impala.catalog.metastore.EnableCatalogdHmsCacheFlagTest
{code}
which are failing, saying
{code}
Configurations invalidate_hms_cache_on_ddls and
enable_sync_to_latest_event_on_ddls can not be set to true at the same time
{code}
and I assume are related, please investigate these too, and fix if appropriate.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12404) Insert fails on timestamp-partitioned Iceberg table
[
https://issues.apache.org/jira/browse/IMPALA-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17761064#comment-17761064
]
Andrew Sherman commented on IMPALA-12404:
-
I see we allow
{code:java}
create table ice_ts_part (i int, j timestamp)
partitioned by spec (j)
stored as iceberg;
{code}
and that shows this jira, but what does it mean to be partitioned by timestamp?
Won't each timestamp be different?
Maybe only partitioning of the form
{code:java}
create table ice_ts_part2 (i int, j timestamp)
partitioned by spec (day(j))
stored as iceberg
{code}
is useful.
> Insert fails on timestamp-partitioned Iceberg table
> ---
>
> Key: IMPALA-12404
> URL: https://issues.apache.org/jira/browse/IMPALA-12404
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 4.2.0
>Reporter: Peter Rozsa
>Priority: Major
> Labels: impala-iceberg
>
> Impala can't insert into Iceberg tables where partitioning contains a
> timestamp column.
> Insert statement fails with the following error:
> {code:java}
> ERROR: UnsupportedOperationException: Unsupported type for
> fromPartitionString: timestamp {code}
> The exception is thrown in the Iceberg API, when catalog tries to load the
> partition data from a datafile: org/apache/impala/util/IcebergUtil.java:715
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12411) TSAN ThreadSanitizer: data race during expr-test teardown
[
https://issues.apache.org/jira/browse/IMPALA-12411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17760175#comment-17760175
]
Andrew Sherman commented on IMPALA-12411:
-
[~MikaelSmith] I am picking on you randomly to take a look. This has only
happened once.
> TSAN ThreadSanitizer: data race during expr-test teardown
> -
>
> Key: IMPALA-12411
> URL: https://issues.apache.org/jira/browse/IMPALA-12411
> Project: IMPALA
> Issue Type: Bug
> Components: be
>Affects Versions: Impala 4.3.0
>Reporter: Andrew Sherman
>Assignee: Michael Smith
>Priority: Critical
> Attachments: expr-test-tsan-failure.log
>
>
> The racing threads are
> {code:java}
> 20:14:05 Read of size 8 at 0x0a8d3348 by main thread:
> 20:14:05 #0 std::vector std::allocator >::~vector()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:680:54
> (unifiedbetests+0x3fcd9b9)
> 20:14:05 #1
> impala::TGetJvmMemoryMetricsResponse::~TGetJvmMemoryMetricsResponse()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/generated-sources/gen-cpp/Frontend_types.cpp:4158:1
> (unifiedbetests+0x3fc1397)
> 20:14:05 #2 impala::JvmMetricCache::~JvmMetricCache()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.h:170:7
> (unifiedbetests+0x4b2989d)
> 20:14:05 #3 at_exit_wrapper(void*)
> /mnt/source/llvm/llvm-5.0.1.src-p7/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:361:31
> (unifiedbetests+0x21b3554)
> {code}
> and
> {code:java}
> 20:14:05 Previous write of size 8 at 0x0a8d3348 by thread T586:
> 20:14:05 #0 std::vector std::allocator
> >::_M_erase_at_end(impala::TJvmMemoryPool*)
> /data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:1798:30
> (unifiedbetests+0x4afabcc)
> 20:14:05 #1 std::vector std::allocator >::clear()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:1499:9
> (unifiedbetests+0x4afa4b4)
> 20:14:05 #2 unsigned int
> impala::TGetJvmMemoryMetricsResponse::read(apache::thrift::protocol::TProtocol*)
>
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/generated-sources/gen-cpp/Frontend_types.tcc:5673:32
> (unifiedbetests+0x4afa21d)
> 20:14:05 #3 impala::Status
> impala::DeserializeThriftMsg(unsigned
> char const*, unsigned int*, bool, impala::TGetJvmMemoryMetricsResponse*)
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/rpc/thrift-util.h:136:23
> (unifiedbetests+0x4af9da6)
> 20:14:05 #4 impala::Status
> impala::DeserializeThriftMsg(JNIEnv_*,
> _jbyteArray*, impala::TGetJvmMemoryMetricsResponse*)
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/rpc/jni-thrift-util.h:61:3
> (unifiedbetests+0x4af9c62)
> 20:14:05 #5 impala::Status
> impala::JniCall::ObjectToResult(_jobject*,
> impala::TGetJvmMemoryMetricsResponse*)
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.h:493:3
> (unifiedbetests+0x4af9b24)
> 20:14:05 #6 impala::Status
> impala::JniCall::Call(impala::TGetJvmMemoryMetricsResponse*)
>
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.h:486:3
> (unifiedbetests+0x4af92a6)
> 20:14:05 #7
> impala::JniUtil::GetJvmMemoryMetrics(impala::TGetJvmMemoryMetricsResponse*)
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.cc:299:72
> (unifiedbetests+0x4af89a3)
> 20:14:05 #8 impala::JvmMetricCache::GrabMetricsIfNecessary()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:294:19
> (unifiedbetests+0x4b2780d)
> 20:14:05 #9 impala::JvmMetricCache::GetCounterMetric(long
> (*)(impala::TGetJvmMemoryMetricsResponse const&))
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:305:3
> (unifiedbetests+0x4b27711)
> 20:14:05 #10 impala::JvmMemoryCounterMetric::GetValue()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:270:41
> (unifiedbetests+0x4b276bf)
> 20:14:05 #11
> impala::QueryState::Init(impala::ExecQueryFInstancesRequestPB const*,
> impala::TExecPlanFragmentInfo const&)::$_3::operator()() const
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/runtime/query-state.cc:185:50
>
[jira] [Assigned] (IMPALA-12411) TSAN ThreadSanitizer: data race during expr-test teardown
[
https://issues.apache.org/jira/browse/IMPALA-12411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman reassigned IMPALA-12411:
---
Assignee: Michael Smith
> TSAN ThreadSanitizer: data race during expr-test teardown
> -
>
> Key: IMPALA-12411
> URL: https://issues.apache.org/jira/browse/IMPALA-12411
> Project: IMPALA
> Issue Type: Bug
> Components: be
>Affects Versions: Impala 4.3.0
>Reporter: Andrew Sherman
>Assignee: Michael Smith
>Priority: Critical
> Attachments: expr-test-tsan-failure.log
>
>
> The racing threads are
> {code:java}
> 20:14:05 Read of size 8 at 0x0a8d3348 by main thread:
> 20:14:05 #0 std::vector std::allocator >::~vector()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:680:54
> (unifiedbetests+0x3fcd9b9)
> 20:14:05 #1
> impala::TGetJvmMemoryMetricsResponse::~TGetJvmMemoryMetricsResponse()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/generated-sources/gen-cpp/Frontend_types.cpp:4158:1
> (unifiedbetests+0x3fc1397)
> 20:14:05 #2 impala::JvmMetricCache::~JvmMetricCache()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.h:170:7
> (unifiedbetests+0x4b2989d)
> 20:14:05 #3 at_exit_wrapper(void*)
> /mnt/source/llvm/llvm-5.0.1.src-p7/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:361:31
> (unifiedbetests+0x21b3554)
> {code}
> and
> {code:java}
> 20:14:05 Previous write of size 8 at 0x0a8d3348 by thread T586:
> 20:14:05 #0 std::vector std::allocator
> >::_M_erase_at_end(impala::TJvmMemoryPool*)
> /data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:1798:30
> (unifiedbetests+0x4afabcc)
> 20:14:05 #1 std::vector std::allocator >::clear()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:1499:9
> (unifiedbetests+0x4afa4b4)
> 20:14:05 #2 unsigned int
> impala::TGetJvmMemoryMetricsResponse::read(apache::thrift::protocol::TProtocol*)
>
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/generated-sources/gen-cpp/Frontend_types.tcc:5673:32
> (unifiedbetests+0x4afa21d)
> 20:14:05 #3 impala::Status
> impala::DeserializeThriftMsg(unsigned
> char const*, unsigned int*, bool, impala::TGetJvmMemoryMetricsResponse*)
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/rpc/thrift-util.h:136:23
> (unifiedbetests+0x4af9da6)
> 20:14:05 #4 impala::Status
> impala::DeserializeThriftMsg(JNIEnv_*,
> _jbyteArray*, impala::TGetJvmMemoryMetricsResponse*)
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/rpc/jni-thrift-util.h:61:3
> (unifiedbetests+0x4af9c62)
> 20:14:05 #5 impala::Status
> impala::JniCall::ObjectToResult(_jobject*,
> impala::TGetJvmMemoryMetricsResponse*)
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.h:493:3
> (unifiedbetests+0x4af9b24)
> 20:14:05 #6 impala::Status
> impala::JniCall::Call(impala::TGetJvmMemoryMetricsResponse*)
>
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.h:486:3
> (unifiedbetests+0x4af92a6)
> 20:14:05 #7
> impala::JniUtil::GetJvmMemoryMetrics(impala::TGetJvmMemoryMetricsResponse*)
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.cc:299:72
> (unifiedbetests+0x4af89a3)
> 20:14:05 #8 impala::JvmMetricCache::GrabMetricsIfNecessary()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:294:19
> (unifiedbetests+0x4b2780d)
> 20:14:05 #9 impala::JvmMetricCache::GetCounterMetric(long
> (*)(impala::TGetJvmMemoryMetricsResponse const&))
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:305:3
> (unifiedbetests+0x4b27711)
> 20:14:05 #10 impala::JvmMemoryCounterMetric::GetValue()
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:270:41
> (unifiedbetests+0x4b276bf)
> 20:14:05 #11
> impala::QueryState::Init(impala::ExecQueryFInstancesRequestPB const*,
> impala::TExecPlanFragmentInfo const&)::$_3::operator()() const
> /data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/runtime/query-state.cc:185:50
> (unifiedbetests+0x455bc15)
> 20:14:05 #12
> boost::detail::function::function_obj_invoker0
[jira] [Updated] (IMPALA-12411) TSAN ThreadSanitizer: data race during expr-test teardown
[
https://issues.apache.org/jira/browse/IMPALA-12411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated IMPALA-12411:
Description:
The racing threads are
{code:java}
20:14:05 Read of size 8 at 0x0a8d3348 by main thread:
20:14:05 #0 std::vector >::~vector()
/data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:680:54
(unifiedbetests+0x3fcd9b9)
20:14:05 #1
impala::TGetJvmMemoryMetricsResponse::~TGetJvmMemoryMetricsResponse()
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/generated-sources/gen-cpp/Frontend_types.cpp:4158:1
(unifiedbetests+0x3fc1397)
20:14:05 #2 impala::JvmMetricCache::~JvmMetricCache()
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.h:170:7
(unifiedbetests+0x4b2989d)
20:14:05 #3 at_exit_wrapper(void*)
/mnt/source/llvm/llvm-5.0.1.src-p7/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:361:31
(unifiedbetests+0x21b3554)
{code}
and
{code:java}
20:14:05 Previous write of size 8 at 0x0a8d3348 by thread T586:
20:14:05 #0 std::vector
>::_M_erase_at_end(impala::TJvmMemoryPool*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:1798:30
(unifiedbetests+0x4afabcc)
20:14:05 #1 std::vector >::clear()
/data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:1499:9
(unifiedbetests+0x4afa4b4)
20:14:05 #2 unsigned int
impala::TGetJvmMemoryMetricsResponse::read(apache::thrift::protocol::TProtocol*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/generated-sources/gen-cpp/Frontend_types.tcc:5673:32
(unifiedbetests+0x4afa21d)
20:14:05 #3 impala::Status
impala::DeserializeThriftMsg(unsigned
char const*, unsigned int*, bool, impala::TGetJvmMemoryMetricsResponse*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/rpc/thrift-util.h:136:23
(unifiedbetests+0x4af9da6)
20:14:05 #4 impala::Status
impala::DeserializeThriftMsg(JNIEnv_*,
_jbyteArray*, impala::TGetJvmMemoryMetricsResponse*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/rpc/jni-thrift-util.h:61:3
(unifiedbetests+0x4af9c62)
20:14:05 #5 impala::Status
impala::JniCall::ObjectToResult(_jobject*,
impala::TGetJvmMemoryMetricsResponse*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.h:493:3
(unifiedbetests+0x4af9b24)
20:14:05 #6 impala::Status
impala::JniCall::Call(impala::TGetJvmMemoryMetricsResponse*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.h:486:3
(unifiedbetests+0x4af92a6)
20:14:05 #7
impala::JniUtil::GetJvmMemoryMetrics(impala::TGetJvmMemoryMetricsResponse*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.cc:299:72
(unifiedbetests+0x4af89a3)
20:14:05 #8 impala::JvmMetricCache::GrabMetricsIfNecessary()
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:294:19
(unifiedbetests+0x4b2780d)
20:14:05 #9 impala::JvmMetricCache::GetCounterMetric(long
(*)(impala::TGetJvmMemoryMetricsResponse const&))
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:305:3
(unifiedbetests+0x4b27711)
20:14:05 #10 impala::JvmMemoryCounterMetric::GetValue()
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:270:41
(unifiedbetests+0x4b276bf)
20:14:05 #11 impala::QueryState::Init(impala::ExecQueryFInstancesRequestPB
const*, impala::TExecPlanFragmentInfo const&)::$_3::operator()() const
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/runtime/query-state.cc:185:50
(unifiedbetests+0x455bc15)
20:14:05 #12
boost::detail::function::function_obj_invoker0::invoke(boost::detail::function::function_buffer&)
/data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:137:18
(unifiedbetests+0x455b9d9)
20:14:05 #13 boost::function0::operator()() const
/data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:763:14
(unifiedbetests+0x4b57bf1)
20:14:05 #14 impala::RuntimeProfile::DerivedCounter::value() const
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/runtime-profile-counters.h:384:12
(unifiedbetests+0x4b8b38d)
20:14:05 #15
[jira] [Updated] (IMPALA-12411) TSAN ThreadSanitizer: data race during expr-test teardown
[
https://issues.apache.org/jira/browse/IMPALA-12411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated IMPALA-12411:
Description:
The racing threads are
{code:java}
20:14:05 Read of size 8 at 0x0a8d3348 by main thread:
20:14:05 #0 std::vector >::~vector()
/data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:680:54
(unifiedbetests+0x3fcd9b9)
20:14:05 #1
impala::TGetJvmMemoryMetricsResponse::~TGetJvmMemoryMetricsResponse()
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/generated-sources/gen-cpp/Frontend_types.cpp:4158:1
(unifiedbetests+0x3fc1397)
20:14:05 #2 impala::JvmMetricCache::~JvmMetricCache()
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.h:170:7
(unifiedbetests+0x4b2989d)
20:14:05 #3 at_exit_wrapper(void*)
/mnt/source/llvm/llvm-5.0.1.src-p7/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:361:31
(unifiedbetests+0x21b3554)
{code}
and
{code:java}
20:14:05 Previous write of size 8 at 0x0a8d3348 by thread T586:
20:14:05 #0 std::vector
>::_M_erase_at_end(impala::TJvmMemoryPool*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:1798:30
(unifiedbetests+0x4afabcc)
20:14:05 #1 std::vector >::clear()
/data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:1499:9
(unifiedbetests+0x4afa4b4)
20:14:05 #2 unsigned int
impala::TGetJvmMemoryMetricsResponse::read(apache::thrift::protocol::TProtocol*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/generated-sources/gen-cpp/Frontend_types.tcc:5673:32
(unifiedbetests+0x4afa21d)
20:14:05 #3 impala::Status
impala::DeserializeThriftMsg(unsigned
char const*, unsigned int*, bool, impala::TGetJvmMemoryMetricsResponse*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/rpc/thrift-util.h:136:23
(unifiedbetests+0x4af9da6)
20:14:05 #4 impala::Status
impala::DeserializeThriftMsg(JNIEnv_*,
_jbyteArray*, impala::TGetJvmMemoryMetricsResponse*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/rpc/jni-thrift-util.h:61:3
(unifiedbetests+0x4af9c62)
20:14:05 #5 impala::Status
impala::JniCall::ObjectToResult(_jobject*,
impala::TGetJvmMemoryMetricsResponse*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.h:493:3
(unifiedbetests+0x4af9b24)
20:14:05 #6 impala::Status
impala::JniCall::Call(impala::TGetJvmMemoryMetricsResponse*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.h:486:3
(unifiedbetests+0x4af92a6)
20:14:05 #7
impala::JniUtil::GetJvmMemoryMetrics(impala::TGetJvmMemoryMetricsResponse*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.cc:299:72
(unifiedbetests+0x4af89a3)
20:14:05 #8 impala::JvmMetricCache::GrabMetricsIfNecessary()
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:294:19
(unifiedbetests+0x4b2780d)
20:14:05 #9 impala::JvmMetricCache::GetCounterMetric(long
(*)(impala::TGetJvmMemoryMetricsResponse const&))
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:305:3
(unifiedbetests+0x4b27711)
20:14:05 #10 impala::JvmMemoryCounterMetric::GetValue()
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:270:41
(unifiedbetests+0x4b276bf)
20:14:05 #11 impala::QueryState::Init(impala::ExecQueryFInstancesRequestPB
const*, impala::TExecPlanFragmentInfo const&)::$_3::operator()() const
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/runtime/query-state.cc:185:50
(unifiedbetests+0x455bc15)
20:14:05 #12
boost::detail::function::function_obj_invoker0::invoke(boost::detail::function::function_buffer&)
/data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:137:18
(unifiedbetests+0x455b9d9)
20:14:05 #13 boost::function0::operator()() const
/data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:763:14
(unifiedbetests+0x4b57bf1)
20:14:05 #14 impala::RuntimeProfile::DerivedCounter::value() const
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/runtime-profile-counters.h:384:12
(unifiedbetests+0x4b8b38d)
20:14:05 #15
[jira] [Created] (IMPALA-12411) TSAN ThreadSanitizer: data race during expr-test teardown
Andrew Sherman created IMPALA-12411:
---
Summary: TSAN ThreadSanitizer: data race during expr-test teardown
Key: IMPALA-12411
URL: https://issues.apache.org/jira/browse/IMPALA-12411
Project: IMPALA
Issue Type: Bug
Components: be
Affects Versions: Impala 4.3.0
Reporter: Andrew Sherman
Attachments: expr-test-tsan-failure.log
The racing threads are
{code:java}
20:14:05 Read of size 8 at 0x0a8d3348 by main thread:
20:14:05 #0 std::vector >::~vector()
/data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:680:54
(unifiedbetests+0x3fcd9b9)
20:14:05 #1
impala::TGetJvmMemoryMetricsResponse::~TGetJvmMemoryMetricsResponse()
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/generated-sources/gen-cpp/Frontend_types.cpp:4158:1
(unifiedbetests+0x3fc1397)
20:14:05 #2 impala::JvmMetricCache::~JvmMetricCache()
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.h:170:7
(unifiedbetests+0x4b2989d)
20:14:05 #3 at_exit_wrapper(void*)
/mnt/source/llvm/llvm-5.0.1.src-p7/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:361:31
(unifiedbetests+0x21b3554)
{code}
and
{code:java}
20:14:05 Previous write of size 8 at 0x0a8d3348 by thread T586:
20:14:05 #0 std::vector
>::_M_erase_at_end(impala::TJvmMemoryPool*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:1798:30
(unifiedbetests+0x4afabcc)
20:14:05 #1 std::vector >::clear()
/data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/gcc-10.4.0/lib/gcc/x86_64-pc-linux-gnu/10.4.0/../../../../include/c++/10.4.0/bits/stl_vector.h:1499:9
(unifiedbetests+0x4afa4b4)
20:14:05 #2 unsigned int
impala::TGetJvmMemoryMetricsResponse::read(apache::thrift::protocol::TProtocol*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/generated-sources/gen-cpp/Frontend_types.tcc:5673:32
(unifiedbetests+0x4afa21d)
20:14:05 #3 impala::Status
impala::DeserializeThriftMsg(unsigned
char const*, unsigned int*, bool, impala::TGetJvmMemoryMetricsResponse*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/rpc/thrift-util.h:136:23
(unifiedbetests+0x4af9da6)
20:14:05 #4 impala::Status
impala::DeserializeThriftMsg(JNIEnv_*,
_jbyteArray*, impala::TGetJvmMemoryMetricsResponse*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/rpc/jni-thrift-util.h:61:3
(unifiedbetests+0x4af9c62)
20:14:05 #5 impala::Status
impala::JniCall::ObjectToResult(_jobject*,
impala::TGetJvmMemoryMetricsResponse*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.h:493:3
(unifiedbetests+0x4af9b24)
20:14:05 #6 impala::Status
impala::JniCall::Call(impala::TGetJvmMemoryMetricsResponse*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.h:486:3
(unifiedbetests+0x4af92a6)
20:14:05 #7
impala::JniUtil::GetJvmMemoryMetrics(impala::TGetJvmMemoryMetricsResponse*)
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/jni-util.cc:299:72
(unifiedbetests+0x4af89a3)
20:14:05 #8 impala::JvmMetricCache::GrabMetricsIfNecessary()
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:294:19
(unifiedbetests+0x4b2780d)
20:14:05 #9 impala::JvmMetricCache::GetCounterMetric(long
(*)(impala::TGetJvmMemoryMetricsResponse const&))
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:305:3
(unifiedbetests+0x4b27711)
20:14:05 #10 impala::JvmMemoryCounterMetric::GetValue()
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/util/memory-metrics.cc:270:41
(unifiedbetests+0x4b276bf)
20:14:05 #11 impala::QueryState::Init(impala::ExecQueryFInstancesRequestPB
const*, impala::TExecPlanFragmentInfo const&)::$_3::operator()() const
/data/jenkins/workspace/impala-cdw-master-core-tsan/repos/Impala/be/src/runtime/query-state.cc:185:50
(unifiedbetests+0x455bc15)
20:14:05 #12
boost::detail::function::function_obj_invoker0::invoke(boost::detail::function::function_buffer&)
/data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:137:18
(unifiedbetests+0x455b9d9)
20:14:05 #13 boost::function0::operator()() const
/data/jenkins/workspace/impala-cdw-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/function/function_template.hpp:763:14
[jira] [Resolved] (IMPALA-12228) Extend Java DebugAction to simulate the failure of an iceberg transaction.
[ https://issues.apache.org/jira/browse/IMPALA-12228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman resolved IMPALA-12228. - Fix Version/s: Impala 4.3.0 Resolution: Fixed > Extend Java DebugAction to simulate the failure of an iceberg transaction. > -- > > Key: IMPALA-12228 > URL: https://issues.apache.org/jira/browse/IMPALA-12228 > Project: IMPALA > Issue Type: Bug >Reporter: Andrew Sherman >Priority: Major > Fix For: Impala 4.3.0 > > > Simulate the failure of an iceberg transaction. > This can happen if there is a commit from Hive or elsewhere > Extend Java DebugAction to allow it to throw an exception. > For now this will be limited to throwing unchecked exceptions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-10770) Bump python thrift_sasl version to 0.4.3
[ https://issues.apache.org/jira/browse/IMPALA-10770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman resolved IMPALA-10770. - Resolution: Duplicate Fixed already by IMPALA-10874 > Bump python thrift_sasl version to 0.4.3 > > > Key: IMPALA-10770 > URL: https://issues.apache.org/jira/browse/IMPALA-10770 > Project: IMPALA > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > > A recent major release of Impyla (0.17.0) uses thrift_sasl=0.4.3 > Increase the version used by Impala (notably by Impala Shell) to match. > This will help clients which use both Impala Shell and Impyla together. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-11947) Bump GBN to get Iceberg change #6074 (Allow using SnapshotManager in Transaction)
[ https://issues.apache.org/jira/browse/IMPALA-11947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman resolved IMPALA-11947. - Resolution: Fixed > Bump GBN to get Iceberg change #6074 (Allow using SnapshotManager in > Transaction) > - > > Key: IMPALA-11947 > URL: https://issues.apache.org/jira/browse/IMPALA-11947 > Project: IMPALA > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > > This is needed for IMPALA-11482 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12373) Implement Small String Optimization for StringValue
[
https://issues.apache.org/jira/browse/IMPALA-12373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17755237#comment-17755237
]
Andrew Sherman commented on IMPALA-12373:
-
I've been reading this recently about how std:string is implemented:
https://devblogs.microsoft.com/oldnewthing/20230803-00/?p=108532
> Implement Small String Optimization for StringValue
> ---
>
> Key: IMPALA-12373
> URL: https://issues.apache.org/jira/browse/IMPALA-12373
> Project: IMPALA
> Issue Type: Improvement
>Reporter: Zoltán Borók-Nagy
>Priority: Major
>
> Implement Small String Optimization for StringValue.
> Current memory layout of StringValue is:
> {noformat}
> char* ptr; // 8 byte
> int len;// 4 byte
> {noformat}
> For small strings with size up to 8 we could store the string contents in the
> bytes of the 'ptr'. Something like that:
> {noformat}
> union {
> char* ptr;
> char small_buf[sizeof(ptr)];
> };
> int len;
> {noformat}
> Many C++ string implementations use the {{Small String Optimization}} to
> speed up work with small strings. For example:
> {code:java}
> Microsoft STL, libstdc++, libc++, Boost, Folly.{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12345) Add user quotas to Admission Control
Andrew Sherman created IMPALA-12345: --- Summary: Add user quotas to Admission Control Key: IMPALA-12345 URL: https://issues.apache.org/jira/browse/IMPALA-12345 Project: IMPALA Issue Type: Bug Reporter: Andrew Sherman Assignee: Andrew Sherman [will add more details later] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-12332) Undo IMPALA-8615 as the corresponding configs have been removed
[ https://issues.apache.org/jira/browse/IMPALA-12332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman resolved IMPALA-12332. - Fix Version/s: Impala 4.3.0 Resolution: Fixed > Undo IMPALA-8615 as the corresponding configs have been removed > --- > > Key: IMPALA-12332 > URL: https://issues.apache.org/jira/browse/IMPALA-12332 > Project: IMPALA > Issue Type: Bug >Affects Versions: Impala 4.1.2 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Fix For: Impala 4.3.0 > > > IMPALA-8615 documented the changes made in IMPALA-8536, but the configs were > subsequently removed in IMPALA-9077. Rollback IMPALA-8615 to bring the docs > up to date. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12332) Undo IMPALA-8615 as the corresponding configs have been removed
Andrew Sherman created IMPALA-12332: --- Summary: Undo IMPALA-8615 as the corresponding configs have been removed Key: IMPALA-12332 URL: https://issues.apache.org/jira/browse/IMPALA-12332 Project: IMPALA Issue Type: Bug Affects Versions: Impala 4.1.2 Reporter: Andrew Sherman Assignee: Andrew Sherman IMPALA-8615 documented the changes made in IMPALA-8536, but the configs were subsequently removed in IMPALA-9077. Rollback IMPALA-8615 to bring the docs up to date. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12294) Cookie handling broken In Impala Shell with python 3
[ https://issues.apache.org/jira/browse/IMPALA-12294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated IMPALA-12294: Description: If more than one cookie is set in a http response then only the first one is used in python3. In ImpalaHttpClient the http headers of the response are stored in an HTTPMessage object. This is deceptive as in Python2 the HTTPMessage is a mimetools.Message object, while in python3 it is an email.message.Message. An important difference is that when a header appears multiple times, as can happen with a ‘Set-Cookie’ header, the mimetools.Message merges the headers internally, while the email.message.Message keeps them separate. was: If more than one cookie is set in a http response ten only the first one is used in python3. In ImpalaHttpClient the http headers of the response are stored in an HTTPMessage object. This is deceptive as in Python2 the HTTPMessage is a mimetools.Message object, while in python3 it is an email.message.Message. An important difference is that when a header appears multiple times, as can happen with a ‘Set-Cookie’ header, the mimetools.Message merges the headers internally, while the email.message.Message keeps them separate. > Cookie handling broken In Impala Shell with python 3 > > > Key: IMPALA-12294 > URL: https://issues.apache.org/jira/browse/IMPALA-12294 > Project: IMPALA > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Critical > > If more than one cookie is set in a http response then only the first one is > used in python3. In ImpalaHttpClient the http headers of the response are > stored in an HTTPMessage object. This is deceptive as in Python2 the > HTTPMessage is a mimetools.Message object, while in python3 it is an > email.message.Message. An important difference is that when a header appears > multiple times, as can happen with a ‘Set-Cookie’ header, the > mimetools.Message merges the headers internally, while the > email.message.Message keeps them separate. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-12294) Cookie handling broken In Impala Shell with python 3
[ https://issues.apache.org/jira/browse/IMPALA-12294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman resolved IMPALA-12294. - Fix Version/s: Impala 4.3.0 Resolution: Fixed > Cookie handling broken In Impala Shell with python 3 > > > Key: IMPALA-12294 > URL: https://issues.apache.org/jira/browse/IMPALA-12294 > Project: IMPALA > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Critical > Fix For: Impala 4.3.0 > > > If more than one cookie is set in a http response then only the first one is > used in python3. In ImpalaHttpClient the http headers of the response are > stored in an HTTPMessage object. This is deceptive as in Python2 the > HTTPMessage is a mimetools.Message object, while in python3 it is an > email.message.Message. An important difference is that when a header appears > multiple times, as can happen with a ‘Set-Cookie’ header, the > mimetools.Message merges the headers internally, while the > email.message.Message keeps them separate. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12294) Cookie handling broken In Impala Shell with python 3
Andrew Sherman created IMPALA-12294: --- Summary: Cookie handling broken In Impala Shell with python 3 Key: IMPALA-12294 URL: https://issues.apache.org/jira/browse/IMPALA-12294 Project: IMPALA Issue Type: Bug Reporter: Andrew Sherman Assignee: Andrew Sherman If more than one cookie is set in a http response ten only the first one is used in python3. In ImpalaHttpClient the http headers of the response are stored in an HTTPMessage object. This is deceptive as in Python2 the HTTPMessage is a mimetools.Message object, while in python3 it is an email.message.Message. An important difference is that when a header appears multiple times, as can happen with a ‘Set-Cookie’ header, the mimetools.Message merges the headers internally, while the email.message.Message keeps them separate. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12049) test_drop_corrupt_table is flaky, fails with "No failure encountered for query"
[
https://issues.apache.org/jira/browse/IMPALA-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17742556#comment-17742556
]
Andrew Sherman commented on IMPALA-12049:
-
Thansk [~stigahuang] [~MikaelSmith] I will try to fix this soon
> test_drop_corrupt_table is flaky, fails with "No failure encountered for
> query"
> ---
>
> Key: IMPALA-12049
> URL: https://issues.apache.org/jira/browse/IMPALA-12049
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.3.0
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
>Priority: Critical
>
> On an Ozone build this test failed. Looks like test is flaky.
> {code}
> Error Message
> query_test/test_iceberg.py:149: in test_drop_corrupt_table
> self._do_test_drop_corrupt_table(unique_database, do_invalidate=False)
> query_test/test_iceberg.py:175: in _do_test_drop_corrupt_table
> format(full_table_name)) common/impala_test_suite.py:929: in wrapper
> return function(*args, **kwargs) common/impala_test_suite.py:952: in
> execute_query_expect_failure assert not result.success, "No failure
> encountered for query %s" % query E AssertionError: No failure encountered
> for query select * from test_drop_corrupt_table_fb61cd07.corrupt_iceberg_tbl
> {code}
> {code}
> Stacktrace
> query_test/test_iceberg.py:149: in test_drop_corrupt_table
> self._do_test_drop_corrupt_table(unique_database, do_invalidate=False)
> query_test/test_iceberg.py:175: in _do_test_drop_corrupt_table
> format(full_table_name))
> common/impala_test_suite.py:929: in wrapper
> return function(*args, **kwargs)
> common/impala_test_suite.py:952: in execute_query_expect_failure
> assert not result.success, "No failure encountered for query %s" % query
> E AssertionError: No failure encountered for query select * from
> test_drop_corrupt_table_fb61cd07.corrupt_iceberg_tbl
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12228) Extend Java DebugAction to simulate the failure of an iceberg transaction.
Andrew Sherman created IMPALA-12228: --- Summary: Extend Java DebugAction to simulate the failure of an iceberg transaction. Key: IMPALA-12228 URL: https://issues.apache.org/jira/browse/IMPALA-12228 Project: IMPALA Issue Type: Bug Reporter: Andrew Sherman Simulate the failure of an iceberg transaction. This can happen if there is a commit from Hive or elsewhere Extend Java DebugAction to allow it to throw an exception. For now this will be limited to throwing unchecked exceptions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-12197) Time Travel Queries on Iceberg tables fail with "ERROR: IllegalArgumentException: null"
[
https://issues.apache.org/jira/browse/IMPALA-12197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman resolved IMPALA-12197.
-
Fix Version/s: Impala 4.3.0
Resolution: Fixed
> Time Travel Queries on Iceberg tables fail with "ERROR:
> IllegalArgumentException: null"
> ---
>
> Key: IMPALA-12197
> URL: https://issues.apache.org/jira/browse/IMPALA-12197
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 2.13.0
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
>Priority: Critical
> Fix For: Impala 4.3.0
>
>
> When using local catalog mode, if a runtime filter is being generated for a
> time travel iceberg table, then a query may fail with "ERROR:
> IllegalArgumentException: null"
> In the planner an Iceberg table that is being accessed with Time Travel is
> represented by an IcebergTimeTravelTable object. This object represents a
> time-based variation on a base table. The IcebergTimeTravelTable may
> represent a different schema from the base table, it does this by tracking
> its own set of Columns. As part of generating a runtime filter the
> isClusteringColumn() method is called on the table. IcebergTimeTravelTable
> was delegating this call to the base object. In local catalog mode this
> method is implemented by LocalTable which has a Preconditions check (an
> assertion) that the column parameter matches the stored column. In this case
> the check fails as the base table and time travel table have their own
> distinct set of column objects.
> A possible fix is to have IcebergTimeTravelTable provide its own
> isClusteringColumn() method. For iceberg there are no clustering columns so
> this method simply returns false.
> {code}
> I0608 18:19:04.009989 26712 jni-util.cc:288]
> 30478706c6b2ab5b:ac068935] java.lang.IllegalArgumentException
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:131)
> at
> org.apache.impala.catalog.local.LocalTable$ColumnMap.isClusteringColumn(LocalTable.java:427)
> at
> org.apache.impala.catalog.local.LocalTable.isClusteringColumn(LocalTable.java:286)
> at
> org.apache.impala.catalog.local.LocalIcebergTable.isClusteringColumn(LocalIcebergTable.java:59)
> at
> org.apache.impala.catalog.ForwardingFeIcebergTable.isClusteringColumn(IcebergTimeTravelTable.java:546)
> at
> org.apache.impala.catalog.IcebergTimeTravelTable.isClusteringColumn(IcebergTimeTravelTable.java:70)
> at
> org.apache.impala.planner.RuntimeFilterGenerator.isBoundByPartitionColumns(RuntimeFilterGenerator.java:1088)
> at
> org.apache.impala.planner.RuntimeFilterGenerator.assignRuntimeFilters(RuntimeFilterGenerator.java:964)
> at
> org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:863)
> at
> org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:866)
> at
> org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:866)
> at
> org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:866)
> at
> org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:856)
> at
> org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:866)
> at
> org.apache.impala.planner.RuntimeFilterGenerator.generateRuntimeFilters(RuntimeFilterGenerator.java:735)
> at org.apache.impala.planner.Planner.createPlanFragments(Planner.java:147)
> at org.apache.impala.planner.Planner.createPlans(Planner.java:251)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1772)
> at
> org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:2603)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2432)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:2056)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1831)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12213) Not a self-event because service id of this catalog does not match with one in event
[
https://issues.apache.org/jira/browse/IMPALA-12213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732779#comment-17732779
]
Andrew Sherman commented on IMPALA-12213:
-
That method ReloadEvent.isOlderEvent() is new in IMPALA-11822 so maybe this is
a regression
cc [~hemanth619]
> Not a self-event because service id of this catalog does not match with one
> in event
>
>
> Key: IMPALA-12213
> URL: https://issues.apache.org/jira/browse/IMPALA-12213
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 4.0.0
>Reporter: Ezra Zerihun
>Priority: Major
>
> Customer started a few Impala Virtual Warehouses that all connected to the
> same Database Catalog in DW. After it finished starting, they noticed tables
> seen in their Hive VW were missing from their Impala VW. Looking in Catalog
> pod logs we Event Processor was stuck in Error state:
> {quote}W0528 07:11:42.398236 228 MetastoreEventsProcessor.java:844] Event
> processing is skipped since status is ERROR. Last synced event id is 33348458
> {quote}
> Running "invalidate metadata \{table}" or global invalidate metadata shows
> the tables.
>
> Looking at old Catalog pod logs when the error first appeared shows this
> unique stacktrace that led to the ERROR state
> {quote}_
> I0528 07:11:40.397091 228 MetastoreEvents.java:638] EventId: 33348458
> EventType: OTHER Ignoring unknown event type OPEN_TXN
> I0528 07:11:40.397158 228 CatalogServiceCatalog.java:1022] Not a self-event
> because service id of this catalog c1aad0ff73c54b79:96249bcba6589cb8 does not
> match with one in event 751641d1aed04da4:8929d897001acc0f.
> I0528 07:11:40.397374 228 MetastoreEventsProcessor.java:1026] Time elapsed in
> processing event batch: 1.015ms E0528 07:11:40.397637 228
> MetastoreEventsProcessor.java:865] Unexpected exception received while
> processing event
> Java exception follows:
> java.lang.NullPointerException
> at
> org.apache.impala.catalog.events.MetastoreEvents$ReloadEvent.isOlderEvent(MetastoreEvents.java:2526)
> at
> org.apache.impala.catalog.events.MetastoreEvents$ReloadEvent.process(MetastoreEvents.java:2501)
> at
> org.apache.impala.catalog.events.MetastoreEvents$MetastoreEvent.processIfEnabled(MetastoreEvents.java:521)
> at
> org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:1013)
> at
> org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:851)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
> E0528 07:11:40.397707 228 MetastoreEventsProcessor.java:1040] Notification
> event is null
> I0528 07:11:41.177731 341 catalog-server.cc:411] A catalog update with 2
> entries is assembled. Catalog version: 113555 Last sent catalog version:
> 113554
> I0528 07:11:41.324290 295 catalog-server.cc:824] Collected update:
> 2:CATALOG_SERVICE_ID, version=113559, original size=60, compressed size=59
> I0528 07:11:41.324410 295 JniUtil.java:169] Finished getCatalogDelta request:
> Getting catalog delta from version 113555. Time spent: 147ms
> W0528 07:11:41.397955 228 MetastoreEventsProcessor.java:844] Event processing
> is skipped since status is ERROR. Last synced event id is 33348458
> W0528 07:11:42.398236 228 MetastoreEventsProcessor.java:844] Event processing
> is skipped since status is ERROR. Last synced event id is 33348458
> _
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12215) Iceberg table creation as CTAS of Hive table containing char/varchar fails in Impala
[
https://issues.apache.org/jira/browse/IMPALA-12215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732776#comment-17732776
]
Andrew Sherman commented on IMPALA-12215:
-
The iceberg spec https://iceberg.apache.org/spec/ does say that ORC char can
map to Iceberg string
> Iceberg table creation as CTAS of Hive table containing char/varchar fails in
> Impala
>
>
> Key: IMPALA-12215
> URL: https://issues.apache.org/jira/browse/IMPALA-12215
> Project: IMPALA
> Issue Type: Bug
>Reporter: Dharmik Thakkar
>Priority: Major
>
> Iceberg table creation as CTAS of Hive table containing char/varchar fails in
> Impala
> Hive Table
> {code:java}
> CREATE TABLE tpch.customer ( c_custkey INT, c_name STRING, c_address
> STRING, c_nationkey INT, c_phone CHAR(15), c_acctbal DECIMAL(15,2),
> c_mktsegment CHAR(10), c_comment STRING ) STORED AS ORC LOCATION
> 's3a://qe-s3-bucket-weekly/cc-dwx-97hupz/warehouse/tablespace/managed/hive/tpch.db/customer'
> TBLPROPERTIES
> ('COLUMN_STATS_ACCURATE'='{\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"c_acctbal\":\"true\",\"c_address\":\"true\",\"c_comment\":\"true\",\"c_custkey\":\"true\",\"c_mktsegment\":\"true\",\"c_name\":\"true\",\"c_nationkey\":\"true\",\"c_phone\":\"true\"}}',
> 'bucketing_version'='2', 'numFiles'='2', 'numFilesErasureCoded'='0',
> 'numRows'='15', 'rawDataSize'='0', 'totalSize'='8023612',
> 'transactional'='true', 'transactional_properties'='default') {code}
> Create iceberg table in Impala with CTAS
> {code:java}
> CREATE TABLE customer
> STORED AS ICEBERG
> TBLPROPERTIES (
> 'write.format.parquet'='parquet',
> 'format-version'='1',
> 'write.${file_format}.compression-codec'='gzip'
> )
> AS SELECT * FROM tpch.customer; {code}
> Error
> {code:java}
> java.sql.SQLException: [Cloudera][ImpalaJDBCDriver](500051) ERROR processing
> query/statement. Error Code: 0, SQL state: TStatus(statusCode:ERROR_STATUS,
> sqlState:HY000, errorMessage:AnalysisException: Exception caught during
> generating Iceberg schema:
> CAUSED BY: CatalogException: Exception caught during generating Iceberg
> schema:
> CAUSED BY: ImpalaRuntimeException: Type CHAR(15) is not supported in Iceberg
> {code}
> The same CTAS query works for non iceberg tables.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12197) Time Travel Queries on Iceberg tables fail with "ERROR: IllegalArgumentException: null"
[
https://issues.apache.org/jira/browse/IMPALA-12197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated IMPALA-12197:
Description:
When using local catalog mode, if a runtime filter is being generated for a
time travel iceberg table, then a query may fail with "ERROR:
IllegalArgumentException: null"
In the planner an Iceberg table that is being accessed with Time Travel is
represented by an IcebergTimeTravelTable object. This object represents a
time-based variation on a base table. The IcebergTimeTravelTable may represent
a different schema from the base table, it does this by tracking its own set of
Columns. As part of generating a runtime filter the isClusteringColumn() method
is called on the table. IcebergTimeTravelTable was delegating this call to the
base object. In local catalog mode this method is implemented by LocalTable
which has a Preconditions check (an assertion) that the column parameter
matches the stored column. In this case the check fails as the base table and
time travel table have their own distinct set of column objects.
A possible fix is to have IcebergTimeTravelTable provide its own
isClusteringColumn() method. For iceberg there are no clustering columns so
this method simply returns false.
{code}
I0608 18:19:04.009989 26712 jni-util.cc:288] 30478706c6b2ab5b:ac068935]
java.lang.IllegalArgumentException
at
com.google.common.base.Preconditions.checkArgument(Preconditions.java:131)
at
org.apache.impala.catalog.local.LocalTable$ColumnMap.isClusteringColumn(LocalTable.java:427)
at
org.apache.impala.catalog.local.LocalTable.isClusteringColumn(LocalTable.java:286)
at
org.apache.impala.catalog.local.LocalIcebergTable.isClusteringColumn(LocalIcebergTable.java:59)
at
org.apache.impala.catalog.ForwardingFeIcebergTable.isClusteringColumn(IcebergTimeTravelTable.java:546)
at
org.apache.impala.catalog.IcebergTimeTravelTable.isClusteringColumn(IcebergTimeTravelTable.java:70)
at
org.apache.impala.planner.RuntimeFilterGenerator.isBoundByPartitionColumns(RuntimeFilterGenerator.java:1088)
at
org.apache.impala.planner.RuntimeFilterGenerator.assignRuntimeFilters(RuntimeFilterGenerator.java:964)
at
org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:863)
at
org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:866)
at
org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:866)
at
org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:866)
at
org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:856)
at
org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:866)
at
org.apache.impala.planner.RuntimeFilterGenerator.generateRuntimeFilters(RuntimeFilterGenerator.java:735)
at org.apache.impala.planner.Planner.createPlanFragments(Planner.java:147)
at org.apache.impala.planner.Planner.createPlans(Planner.java:251)
at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1772)
at
org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:2603)
at
org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2432)
at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:2056)
at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1831)
at
org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164)
{code}
was:
When using local catalog mode, if a runtime filter is being generated for a
time travel iceberg table, then a query may fail with "ERROR:
IllegalArgumentException: null"
In the planner an Iceberg table that is being accessed with Time Travel is
represented by an IcebergTimeTravelTable object. This object represents a
time-based variation on a base table. The IcebergTimeTravelTable may represent
a different schema from the base table, it does this by tracking its own set of
Columns. As part of generating a runtime filter the isClusteringColumn() method
is called on the table. IcebergTimeTravelTable was delegating this call to the
base object. In local catalog mode this method is implemented by LocalTable
which has a Preconditions check (an assertion) that the column parameter
matches the stored column. In this case the check fails as the base table and
time travel table have their own distinct set of column objects.
A possible fix is to have IcebergTimeTravelTable provide its own
isClusteringColumn() method. For iceberg there are no clustering columns so
this method simply returns false.
> Time Travel Queries on Iceberg tables fail with "ERROR:
> IllegalArgumentException: null"
>
[jira] [Created] (IMPALA-12197) Time Travel Queries on Iceberg tables fail with "ERROR: IllegalArgumentException: null"
Andrew Sherman created IMPALA-12197: --- Summary: Time Travel Queries on Iceberg tables fail with "ERROR: IllegalArgumentException: null" Key: IMPALA-12197 URL: https://issues.apache.org/jira/browse/IMPALA-12197 Project: IMPALA Issue Type: Bug Affects Versions: Impala 2.13.0 Reporter: Andrew Sherman Assignee: Andrew Sherman When using local catalog mode, if a runtime filter is being generated for a time travel iceberg table, then a query may fail with "ERROR: IllegalArgumentException: null" In the planner an Iceberg table that is being accessed with Time Travel is represented by an IcebergTimeTravelTable object. This object represents a time-based variation on a base table. The IcebergTimeTravelTable may represent a different schema from the base table, it does this by tracking its own set of Columns. As part of generating a runtime filter the isClusteringColumn() method is called on the table. IcebergTimeTravelTable was delegating this call to the base object. In local catalog mode this method is implemented by LocalTable which has a Preconditions check (an assertion) that the column parameter matches the stored column. In this case the check fails as the base table and time travel table have their own distinct set of column objects. A possible fix is to have IcebergTimeTravelTable provide its own isClusteringColumn() method. For iceberg there are no clustering columns so this method simply returns false. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12109) [DOCS] Add documentation for admission_control_slots
Andrew Sherman created IMPALA-12109: --- Summary: [DOCS] Add documentation for admission_control_slots Key: IMPALA-12109 URL: https://issues.apache.org/jira/browse/IMPALA-12109 Project: IMPALA Issue Type: Bug Reporter: Andrew Sherman We should add this concept somewhere in our docs Sources to use: * The best current doc is the blog https://blog.cloudera.com/new-multithreading-model-for-apache-impala/ * https://github.com/apache/impala/blob/5d5f33115523dce3b4e29adfa4e921a027e70c90/be/src/runtime/exec-env.cc#L97 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12092) TestIcebergTable.test_drop_corrupt_table reports "no failure encountered" with Ozone
[
https://issues.apache.org/jira/browse/IMPALA-12092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17716933#comment-17716933
]
Andrew Sherman commented on IMPALA-12092:
-
Same as IMPALA-12049 I think. On me to fix.
> TestIcebergTable.test_drop_corrupt_table reports "no failure encountered"
> with Ozone
>
>
> Key: IMPALA-12092
> URL: https://issues.apache.org/jira/browse/IMPALA-12092
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Reporter: Michael Smith
>Assignee: Andrew Sherman
>Priority: Major
>
> TestIcebergTable.test_drop_corrupt_table reports
> {code}
> query_test/test_iceberg.py:148: in test_drop_corrupt_table
> self._do_test_drop_corrupt_table(unique_database, do_invalidate=False)
> query_test/test_iceberg.py:174: in _do_test_drop_corrupt_table
> format(full_table_name))
> common/impala_test_suite.py:867: in wrapper
> return function(*args, **kwargs)
> common/impala_test_suite.py:890: in execute_query_expect_failure
> assert not result.success, "No failure encountered for query %s" % query
> E AssertionError: No failure encountered for query select * from
> test_drop_corrupt_table_fb61cd07.corrupt_iceberg_tbl
> {code}
> when run with Ozone. This fails intermittently.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-12050) Exclude kryo-shaded from the Java build
[ https://issues.apache.org/jira/browse/IMPALA-12050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman resolved IMPALA-12050. - Fix Version/s: Impala 4.3.0 Resolution: Fixed > Exclude kryo-shaded from the Java build > --- > > Key: IMPALA-12050 > URL: https://issues.apache.org/jira/browse/IMPALA-12050 > Project: IMPALA > Issue Type: Bug >Affects Versions: Impala 4.3.0 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Fix For: Impala 4.3.0 > > > Kryo is a serialization library that is brought in by Apache Hudi. It is not > used by Impala at runtime when running tests that use Hudi > (hudi-parquet.test). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12050) Exclude kryo-shaded from the Java build
Andrew Sherman created IMPALA-12050: --- Summary: Exclude kryo-shaded from the Java build Key: IMPALA-12050 URL: https://issues.apache.org/jira/browse/IMPALA-12050 Project: IMPALA Issue Type: Bug Affects Versions: Impala 4.3.0 Reporter: Andrew Sherman Assignee: Andrew Sherman Kryo is a serialization library that is brought in by Apache Hudi. It is not used by Impala at runtime when running tests that use Hudi (hudi-parquet.test). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12049) test_drop_corrupt_table is flaky, fails with "No failure encountered for query"
Andrew Sherman created IMPALA-12049:
---
Summary: test_drop_corrupt_table is flaky, fails with "No failure
encountered for query"
Key: IMPALA-12049
URL: https://issues.apache.org/jira/browse/IMPALA-12049
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 4.3.0
Reporter: Andrew Sherman
Assignee: Andrew Sherman
On an Ozone build this test failed. Looks like test is flaky.
{code}
Error Message
query_test/test_iceberg.py:149: in test_drop_corrupt_table
self._do_test_drop_corrupt_table(unique_database, do_invalidate=False)
query_test/test_iceberg.py:175: in _do_test_drop_corrupt_table
format(full_table_name)) common/impala_test_suite.py:929: in wrapper return
function(*args, **kwargs) common/impala_test_suite.py:952: in
execute_query_expect_failure assert not result.success, "No failure
encountered for query %s" % query E AssertionError: No failure encountered
for query select * from test_drop_corrupt_table_fb61cd07.corrupt_iceberg_tbl
{code}
{code}
Stacktrace
query_test/test_iceberg.py:149: in test_drop_corrupt_table
self._do_test_drop_corrupt_table(unique_database, do_invalidate=False)
query_test/test_iceberg.py:175: in _do_test_drop_corrupt_table
format(full_table_name))
common/impala_test_suite.py:929: in wrapper
return function(*args, **kwargs)
common/impala_test_suite.py:952: in execute_query_expect_failure
assert not result.success, "No failure encountered for query %s" % query
E AssertionError: No failure encountered for query select * from
test_drop_corrupt_table_fb61cd07.corrupt_iceberg_tbl
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-12034) TestIcebergTable.test_drop_corrupt_table failed in Ozone build
[
https://issues.apache.org/jira/browse/IMPALA-12034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman resolved IMPALA-12034.
-
Fix Version/s: Impala 4.3.0
Resolution: Fixed
> TestIcebergTable.test_drop_corrupt_table failed in Ozone build
> --
>
> Key: IMPALA-12034
> URL: https://issues.apache.org/jira/browse/IMPALA-12034
> Project: IMPALA
> Issue Type: Bug
>Reporter: Yida Wu
>Assignee: Andrew Sherman
>Priority: Major
> Labels: broken-build
> Fix For: Impala 4.3.0
>
>
> Testcases query_test.test_iceberg.TestIcebergTable.test_drop_corrupt_table
> and
> query_test.test_iceberg.TestIcebergTable.test_drop_corrupt_table_with_invalidate
> are found failed in ozone builds with following messages.
> *Error Message*
> {code:java}
> query_test/test_iceberg.py:147: in test_drop_corrupt_table
> self._do_test_drop_corrupt_table(unique_database, do_invalidate=False)
> query_test/test_iceberg.py:165: in _do_test_drop_corrupt_table assert not
> self.hdfs_client.exists(metadata_location) E AttributeError:
> 'TestIcebergTable' object has no attribute 'hdfs_client'{code}
> *Stack Trace*
> {code:java}
> query_test/test_iceberg.py:147: in test_drop_corrupt_table
> self._do_test_drop_corrupt_table(unique_database, do_invalidate=False)
> query_test/test_iceberg.py:165: in _do_test_drop_corrupt_table assert not
> self.hdfs_client.exists(metadata_location) E AttributeError:
> 'TestIcebergTable' object has no attribute 'hdfs_client'{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12034) TestIcebergTable.test_drop_corrupt_table failed in Ozone build
[
https://issues.apache.org/jira/browse/IMPALA-12034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman reassigned IMPALA-12034:
---
Assignee: Andrew Sherman
> TestIcebergTable.test_drop_corrupt_table failed in Ozone build
> --
>
> Key: IMPALA-12034
> URL: https://issues.apache.org/jira/browse/IMPALA-12034
> Project: IMPALA
> Issue Type: Bug
>Reporter: Yida Wu
>Assignee: Andrew Sherman
>Priority: Major
> Labels: broken-build
>
> Testcases query_test.test_iceberg.TestIcebergTable.test_drop_corrupt_table
> and
> query_test.test_iceberg.TestIcebergTable.test_drop_corrupt_table_with_invalidate
> are found failed in ozone builds with following messages.
> *Error Message*
> {code:java}
> query_test/test_iceberg.py:147: in test_drop_corrupt_table
> self._do_test_drop_corrupt_table(unique_database, do_invalidate=False)
> query_test/test_iceberg.py:165: in _do_test_drop_corrupt_table assert not
> self.hdfs_client.exists(metadata_location) E AttributeError:
> 'TestIcebergTable' object has no attribute 'hdfs_client'{code}
> *Stack Trace*
> {code:java}
> query_test/test_iceberg.py:147: in test_drop_corrupt_table
> self._do_test_drop_corrupt_table(unique_database, do_invalidate=False)
> query_test/test_iceberg.py:165: in _do_test_drop_corrupt_table assert not
> self.hdfs_client.exists(metadata_location) E AttributeError:
> 'TestIcebergTable' object has no attribute 'hdfs_client'{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-11509) Dropping files of Iceberg during table loading may cause Impalad to stuck in infinite loop
[
https://issues.apache.org/jira/browse/IMPALA-11509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman resolved IMPALA-11509.
-
Fix Version/s: Impala 4.3.0
Resolution: Fixed
> Dropping files of Iceberg during table loading may cause Impalad to stuck in
> infinite loop
> --
>
> Key: IMPALA-11509
> URL: https://issues.apache.org/jira/browse/IMPALA-11509
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.1.0
>Reporter: Gabor Kaszab
>Assignee: Andrew Sherman
>Priority: Critical
> Labels: iceberg, impala-iceberg
> Fix For: Impala 4.3.0
>
>
> This issues is very similar to
> https://issues.apache.org/jira/browse/IMPALA-11502. The repro steps are also
> almost identical, however in this case the folder of the table should be
> dropped right when the INSERT into starts.
> Repro steps:
> 1) Create the Iceberg table:
> {code:java}
> DROP DATABASE IF EXISTS `drop_incomplete_table` CASCADE;
> CREATE DATABASE `drop_incomplete_table`;
> CREATE TABLE drop_incomplete_table.iceberg_tbl (i int) stored as iceberg
> tblproperties('iceberg.catalog'='hadoop.catalog',
>
> 'iceberg.catalog_location'='/test-warehouse/drop_incomplete_table');
> {code}
> 2) For this step timing is essential and might require a few try to hit the
> issue. Try to run INSERT INTO and dropping the HDFS folder at the same time.
> Manually executing them is fine, this doesn't require scripting.
> {code:java}
> INSERT INTO drop_incomplete_table.iceberg_tbl VALUES (1), (2), (3);
> hdfs dfs -rm -r hdfs://localhost:20500/test-warehouse/drop_incomplete_table
> {code}
> You will notice you hit the issue when Impala shell start to hang. The jstack
> of the hanging impalad (not the catalogd) will contain this for one of the
> threads:
> {code:java}
> "Thread-15" #30 prio=5 os_prio=0 tid=0x0db2a000 nid=0x56f4 in
> Object.wait() [0x7f0e7b59a000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at
> org.apache.impala.catalog.ImpaladCatalog.waitForCatalogUpdate(ImpaladCatalog.java:290)
> - locked <0x000724f7cdc0> (a java.lang.Object)
> at
> org.apache.impala.analysis.StmtMetadataLoader.loadTables(StmtMetadataLoader.java:229)
> at
> org.apache.impala.analysis.StmtMetadataLoader.loadTables(StmtMetadataLoader.java:141)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2001)
> at
> org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1913)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1737)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164)
> {code}
> Initially, Iceberg tables are created as IncompleteTables and when there is a
> query on the table, they will be loaded as IcebergTable. For me it seems,
> that when we run the first query after creating the table, with some timing
> of dropping the files we can get into a state where the table appears as a
> "missingTable" in StmtMetadataLoader.loadTable(), however, when a prioritized
> table load is requested, the Catalog says that the table is already loaded.
> This results the table always appearing as "missingTable" and we never get
> out of the [while
> loop|https://github.com/apache/impala/blob/62e20d1ba842a3f27395251c57dea9850f462fc9/fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java#L196]
> in loadTables().
> I managed to repro this using HiveCatalog, but I didn't have luck to repro
> with non-Iceberg, traditional Hive tables.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-11330) Handle missing Iceberg data/metadata gracefully
[
https://issues.apache.org/jira/browse/IMPALA-11330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman resolved IMPALA-11330.
-
Resolution: Duplicate
> Handle missing Iceberg data/metadata gracefully
> ---
>
> Key: IMPALA-11330
> URL: https://issues.apache.org/jira/browse/IMPALA-11330
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.1.0
>Reporter: Tamas Mate
>Assignee: Andrew Sherman
>Priority: Major
> Labels: impala-iceberg
>
> In case the data/metadata directory is not available for an Iceberg table
> queries are failing with NotFoundException, see bellow. This affects DROP
> TABLE as well, which means that an Iceberg table can get stuck in the system
> if the administrator moves the data.
> {code:none}
> ERROR: NotFoundException: Failed to open input stream for file:
> hdfs://localhost:20500/test-warehouse/test2/metadata/1-398886ba-f6eb-4b72-b755-f1be10ac99c5.metadata.json
> CAUSED BY: FileNotFoundException: File does not exist:
> /test-warehouse/test2/metadata/1-398886ba-f6eb-4b72-b755-f1be10ac99c5.metadata.json
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:87)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:77)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:159)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2035)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:737)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:454)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:533)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:989)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:917)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2894)
> CAUSED BY: RemoteException: File does not exist:
> /test-warehouse/test2/metadata/1-398886ba-f6eb-4b72-b755-f1be10ac99c5.metadata.json
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:87)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:77)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:159)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:2035)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:737)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:454)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:533)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:989)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:917)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2894)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-11596) Document EXECUTE ROLLBACK functionality
[ https://issues.apache.org/jira/browse/IMPALA-11596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman closed IMPALA-11596. --- Fix Version/s: Impala 4.3.0 Resolution: Fixed > Document EXECUTE ROLLBACK functionality > --- > > Key: IMPALA-11596 > URL: https://issues.apache.org/jira/browse/IMPALA-11596 > Project: IMPALA > Issue Type: Bug > Components: Docs >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Critical > Fix For: Impala 4.3.0 > > > Document the EXECUTE ROLLBACK functionality added in IMPALA-11482 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-11999) TestIcebergTable.test_execute_rollback fails in Ozone build
[
https://issues.apache.org/jira/browse/IMPALA-11999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman resolved IMPALA-11999.
-
Fix Version/s: Impala 4.3.0
Resolution: Fixed
> TestIcebergTable.test_execute_rollback fails in Ozone build
> ---
>
> Key: IMPALA-11999
> URL: https://issues.apache.org/jira/browse/IMPALA-11999
> Project: IMPALA
> Issue Type: Bug
>Reporter: Daniel Becker
>Assignee: Andrew Sherman
>Priority: Blocker
> Labels: broken-build
> Fix For: Impala 4.3.0
>
>
> A build using the Ozone file system failed with the following test failures:
> query_test.test_iceberg.TestIcebergTable.test_execute_rollback
> query_test.test_iceberg.TestIcebergTable.test_expire_snapshots
> {code:java}
> query_test/test_iceberg.py:267: in test_execute_rollback
> TBLPROPERTIES ({1})""".format(tbl_name, catalog_properties))
> common/impala_connection.py:214: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:191: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:369: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:390: in wait_for_finished
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EQuery aborted:ImpalaRuntimeException: Error making 'createTable' RPC to
> Hive Metastore:
> E CAUSED BY: RuntimeIOException: Failed to refresh the table
> E CAUSED BY: OMException: Bucket or Volume name has an unsupported
> character : _
> {code}
> The issue came after IMPALA-11482 was merged, it may be related.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11999) TestIcebergTable.test_execute_rollback fails in Ozone build
[
https://issues.apache.org/jira/browse/IMPALA-11999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17700410#comment-17700410
]
Andrew Sherman commented on IMPALA-11999:
-
Some iceberg tests run using different forms of iceberg catalog. One of these
specifies iceberg.catalog_location as one of the TBLPROPERTIES when a table is
created. This location was hard-coded to a path starting with
"/test-warehouse", which results in an illegal Ozone path. Fix this by wrapping
the location with get_fs_path(), which gives a legal Ozone path.
> TestIcebergTable.test_execute_rollback fails in Ozone build
> ---
>
> Key: IMPALA-11999
> URL: https://issues.apache.org/jira/browse/IMPALA-11999
> Project: IMPALA
> Issue Type: Bug
>Reporter: Daniel Becker
>Assignee: Andrew Sherman
>Priority: Blocker
> Labels: broken-build
>
> A build using the Ozone file system failed with the following test failures:
> query_test.test_iceberg.TestIcebergTable.test_execute_rollback
> query_test.test_iceberg.TestIcebergTable.test_expire_snapshots
> {code:java}
> query_test/test_iceberg.py:267: in test_execute_rollback
> TBLPROPERTIES ({1})""".format(tbl_name, catalog_properties))
> common/impala_connection.py:214: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:191: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:369: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:390: in wait_for_finished
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EQuery aborted:ImpalaRuntimeException: Error making 'createTable' RPC to
> Hive Metastore:
> E CAUSED BY: RuntimeIOException: Failed to refresh the table
> E CAUSED BY: OMException: Bucket or Volume name has an unsupported
> character : _
> {code}
> The issue came after IMPALA-11482 was merged, it may be related.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11999) TestIcebergTable.test_execute_rollback fails in Ozone build
[
https://issues.apache.org/jira/browse/IMPALA-11999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17700340#comment-17700340
]
Andrew Sherman commented on IMPALA-11999:
-
Stacktrace from catalog log
{code}
W0313 09:30:43.576596 14738 HadoopTableOperations.java:347] Error trying to
recover version-hint.txt data for
/test-warehouse/test_execute_rollback_83e8c4b8/hadoop_catalog_test/test_execute_rollback_83e8c4b8/iceberg_execute_rollback/metadata/version-hint.text
Java exception follows:
INVALID_BUCKET_NAME org.apache.hadoop.ozone.om.exceptions.OMException: Bucket
or Volume name has an unsupported character : _
at
org.apache.hadoop.ozone.client.rpc.RpcClient.verifyBucketName(RpcClient.java:713)
at
org.apache.hadoop.ozone.client.rpc.RpcClient.getBucketDetails(RpcClient.java:1092)
at
org.apache.hadoop.fs.ozone.BasicRootedOzoneClientAdapterImpl.getBucket(BasicRootedOzoneClientAdapterImpl.java:267)
at
org.apache.hadoop.fs.ozone.BasicRootedOzoneClientAdapterImpl.getBucket(BasicRootedOzoneClientAdapterImpl.java:241)
at
org.apache.hadoop.fs.ozone.BasicRootedOzoneClientAdapterImpl.readFile(BasicRootedOzoneClientAdapterImpl.java:351)
at
org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem.open(BasicRootedOzoneFileSystem.java:211)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:958)
at
org.apache.iceberg.hadoop.HadoopTableOperations.findVersion(HadoopTableOperations.java:318)
at
org.apache.iceberg.hadoop.HadoopTableOperations.refresh(HadoopTableOperations.java:104)
at
org.apache.iceberg.hadoop.HadoopTableOperations.current(HadoopTableOperations.java:84)
at
org.apache.iceberg.BaseMetastoreCatalog$BaseMetastoreCatalogTableBuilder.create(BaseMetastoreCatalog.java:186)
at org.apache.iceberg.catalog.Catalog.createTable(Catalog.java:75)
at
org.apache.impala.catalog.iceberg.IcebergHadoopCatalog.createTable(IcebergHadoopCatalog.java:66)
at
org.apache.impala.service.IcebergCatalogOpExecutor.createTable(IcebergCatalogOpExecutor.java:88)
at
org.apache.impala.service.CatalogOpExecutor.createIcebergTable(CatalogOpExecutor.java:3692)
at
org.apache.impala.service.CatalogOpExecutor.createTable(CatalogOpExecutor.java:3315)
at
org.apache.impala.service.CatalogOpExecutor.execDdlRequest(CatalogOpExecutor.java:450)
at
org.apache.impala.service.JniCatalog.lambda$execDdl$3(JniCatalog.java:282)
at
org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1(JniCatalogOp.java:90)
at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:58)
at
org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:89)
at
org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:100)
at
org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:230)
at
org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:244)
at org.apache.impala.service.JniCatalog.execDdl(JniCatalog.java:281)
E0313 09:30:43.576925 14738 JniUtil.java:176] Error in CREATE_TABLE
test_execute_rollback_83e8c4b8.iceberg_execute_rollback issued by jenkins. Time
spent: 2ms
I0313 09:30:43.577097 14738 jni-util.cc:288]
org.apache.impala.common.ImpalaRuntimeException: Error making 'createTable' RPC
to Hive Metastore:
at
org.apache.impala.service.CatalogOpExecutor.createIcebergTable(CatalogOpExecutor.java:3778)
at
org.apache.impala.service.CatalogOpExecutor.createTable(CatalogOpExecutor.java:3315)
at
org.apache.impala.service.CatalogOpExecutor.execDdlRequest(CatalogOpExecutor.java:450)
at
org.apache.impala.service.JniCatalog.lambda$execDdl$3(JniCatalog.java:282)
at
org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1(JniCatalogOp.java:90)
at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:58)
at
org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:89)
at
org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:100)
at
org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:230)
at
org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:244)
at org.apache.impala.service.JniCatalog.execDdl(JniCatalog.java:281)
Caused by: org.apache.iceberg.exceptions.RuntimeIOException: Failed to refresh
the table
at
org.apache.iceberg.hadoop.HadoopTableOperations.refresh(HadoopTableOperations.java:126)
at
org.apache.iceberg.hadoop.HadoopTableOperations.current(HadoopTableOperations.java:84)
at
org.apache.iceberg.BaseMetastoreCatalog$BaseMetastoreCatalogTableBuilder.create(BaseMetastoreCatalog.java:186)
at org.apache.iceberg.catalog.Catalog.createTable(Catalog.java:75)
at
[jira] [Updated] (IMPALA-11482) Implement Iceberg table rollback feature
[
https://issues.apache.org/jira/browse/IMPALA-11482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated IMPALA-11482:
Fix Version/s: Impala 4.3.0
> Implement Iceberg table rollback feature
>
>
> Key: IMPALA-11482
> URL: https://issues.apache.org/jira/browse/IMPALA-11482
> Project: IMPALA
> Issue Type: New Feature
> Components: Frontend
>Reporter: Zoltán Borók-Nagy
>Assignee: Andrew Sherman
>Priority: Major
> Labels: impala-iceberg
> Fix For: Impala 4.3.0
>
>
> We should allow rolling back iceberg table's data to the state at an older
> table snapshot.
> Rollback to the last snapshot before a specific timestamp
> {code}
> ALTER TABLE ice_t EXECUTE ROLLBACK('2022-05-12 00:00:00')
> {code}
> Rollback to a specific snapshot ID
> {code}
> ALTER TABLE ice_t EXECUTE ROLLBACK();
> {code}
> However, to revert a rollback we might need to be able to change the table's
> metadata_location property (like HIVE-26203).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-11482) Implement Iceberg table rollback feature
[
https://issues.apache.org/jira/browse/IMPALA-11482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman resolved IMPALA-11482.
-
Resolution: Fixed
> Implement Iceberg table rollback feature
>
>
> Key: IMPALA-11482
> URL: https://issues.apache.org/jira/browse/IMPALA-11482
> Project: IMPALA
> Issue Type: New Feature
> Components: Frontend
>Reporter: Zoltán Borók-Nagy
>Assignee: Andrew Sherman
>Priority: Major
> Labels: impala-iceberg
>
> We should allow rolling back iceberg table's data to the state at an older
> table snapshot.
> Rollback to the last snapshot before a specific timestamp
> {code}
> ALTER TABLE ice_t EXECUTE ROLLBACK('2022-05-12 00:00:00')
> {code}
> Rollback to a specific snapshot ID
> {code}
> ALTER TABLE ice_t EXECUTE ROLLBACK();
> {code}
> However, to revert a rollback we might need to be able to change the table's
> metadata_location property (like HIVE-26203).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-11479) Create unit tests for IcebergUtil
[ https://issues.apache.org/jira/browse/IMPALA-11479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman resolved IMPALA-11479. - Fix Version/s: Impala 4.3.0 Resolution: Fixed > Create unit tests for IcebergUtil > - > > Key: IMPALA-11479 > URL: https://issues.apache.org/jira/browse/IMPALA-11479 > Project: IMPALA > Issue Type: Bug >Affects Versions: Impala 4.1.0 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Labels: impala-iceberg > Fix For: Impala 4.3.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11947) Bump GBN to get Iceberg change #6074 (Allow using SnapshotManager in Transaction)
Andrew Sherman created IMPALA-11947: --- Summary: Bump GBN to get Iceberg change #6074 (Allow using SnapshotManager in Transaction) Key: IMPALA-11947 URL: https://issues.apache.org/jira/browse/IMPALA-11947 Project: IMPALA Issue Type: Bug Reporter: Andrew Sherman Assignee: Andrew Sherman This is needed for IMPALA-11482 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-11482) Implement Iceberg table rollback feature
[
https://issues.apache.org/jira/browse/IMPALA-11482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-11482 started by Andrew Sherman.
---
> Implement Iceberg table rollback feature
>
>
> Key: IMPALA-11482
> URL: https://issues.apache.org/jira/browse/IMPALA-11482
> Project: IMPALA
> Issue Type: New Feature
> Components: Frontend
>Reporter: Zoltán Borók-Nagy
>Assignee: Andrew Sherman
>Priority: Major
> Labels: impala-iceberg
>
> We should allow rolling back iceberg table's data to the state at an older
> table snapshot.
> Rollback to the last snapshot before a specific timestamp
> {code}
> ALTER TABLE ice_t EXECUTE ROLLBACK('2022-05-12 00:00:00')
> {code}
> Rollback to a specific snapshot ID
> {code}
> ALTER TABLE ice_t EXECUTE ROLLBACK();
> {code}
> However, to revert a rollback we might need to be able to change the table's
> metadata_location property (like HIVE-26203).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11936) Add counter to avoid looping forever in StmtTableCache.loadTables()
Andrew Sherman created IMPALA-11936: --- Summary: Add counter to avoid looping forever in StmtTableCache.loadTables() Key: IMPALA-11936 URL: https://issues.apache.org/jira/browse/IMPALA-11936 Project: IMPALA Issue Type: Bug Reporter: Andrew Sherman StmtTableCache.loadTables() is called from coordinator code to ensure all needed table metadata is available. It contains a loop that can potentially run forever. The loop is not passive, in the sense that it is not just waiting for updates, instead it does keep issuing load requests to the catalogd. The loop will end if the data can be loaded from catalogd, or if certain special cases detect a failure. If code on the load path throws Exceptions and these cannot be fixed by new data from the catalogd, then a hang can result. For an example of a hang caused by this loop, see IMPALA-11509 where coordinator-side loading of Iceberg metadata from a corrupt table would always throw an exception. One way to fix this would be to add a counter to the loop to avoid looping forever if no progress is being made. The subtle part is detecting if no progress is being made -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-11509) Dropping files of Iceberg during table loading may cause Impalad to stuck in infinite loop
[
https://issues.apache.org/jira/browse/IMPALA-11509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman reassigned IMPALA-11509:
---
Assignee: Andrew Sherman
> Dropping files of Iceberg during table loading may cause Impalad to stuck in
> infinite loop
> --
>
> Key: IMPALA-11509
> URL: https://issues.apache.org/jira/browse/IMPALA-11509
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.1.0
>Reporter: Gabor Kaszab
>Assignee: Andrew Sherman
>Priority: Critical
> Labels: iceberg, impala-iceberg
>
> This issues is very similar to
> https://issues.apache.org/jira/browse/IMPALA-11502. The repro steps are also
> almost identical, however in this case the folder of the table should be
> dropped right when the INSERT into starts.
> Repro steps:
> 1) Create the Iceberg table:
> {code:java}
> DROP DATABASE IF EXISTS `drop_incomplete_table` CASCADE;
> CREATE DATABASE `drop_incomplete_table`;
> CREATE TABLE drop_incomplete_table.iceberg_tbl (i int) stored as iceberg
> tblproperties('iceberg.catalog'='hadoop.catalog',
>
> 'iceberg.catalog_location'='/test-warehouse/drop_incomplete_table');
> {code}
> 2) For this step timing is essential and might require a few try to hit the
> issue. Try to run INSERT INTO and dropping the HDFS folder at the same time.
> Manually executing them is fine, this doesn't require scripting.
> {code:java}
> INSERT INTO drop_incomplete_table.iceberg_tbl VALUES (1), (2), (3);
> hdfs dfs -rm -r hdfs://localhost:20500/test-warehouse/drop_incomplete_table
> {code}
> You will notice you hit the issue when Impala shell start to hang. The jstack
> of the hanging impalad (not the catalogd) will contain this for one of the
> threads:
> {code:java}
> "Thread-15" #30 prio=5 os_prio=0 tid=0x0db2a000 nid=0x56f4 in
> Object.wait() [0x7f0e7b59a000]
>java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at
> org.apache.impala.catalog.ImpaladCatalog.waitForCatalogUpdate(ImpaladCatalog.java:290)
> - locked <0x000724f7cdc0> (a java.lang.Object)
> at
> org.apache.impala.analysis.StmtMetadataLoader.loadTables(StmtMetadataLoader.java:229)
> at
> org.apache.impala.analysis.StmtMetadataLoader.loadTables(StmtMetadataLoader.java:141)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2001)
> at
> org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1913)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1737)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164)
> {code}
> Initially, Iceberg tables are created as IncompleteTables and when there is a
> query on the table, they will be loaded as IcebergTable. For me it seems,
> that when we run the first query after creating the table, with some timing
> of dropping the files we can get into a state where the table appears as a
> "missingTable" in StmtMetadataLoader.loadTable(), however, when a prioritized
> table load is requested, the Catalog says that the table is already loaded.
> This results the table always appearing as "missingTable" and we never get
> out of the [while
> loop|https://github.com/apache/impala/blob/62e20d1ba842a3f27395251c57dea9850f462fc9/fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java#L196]
> in loadTables().
> I managed to repro this using HiveCatalog, but I didn't have luck to repro
> with non-Iceberg, traditional Hive tables.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-11862) Document that the default value of ssl_cipher_list is not empty
[ https://issues.apache.org/jira/browse/IMPALA-11862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman resolved IMPALA-11862. - Fix Version/s: Impala 4.3.0 Resolution: Fixed > Document that the default value of ssl_cipher_list is not empty > --- > > Key: IMPALA-11862 > URL: https://issues.apache.org/jira/browse/IMPALA-11862 > Project: IMPALA > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Fix For: Impala 4.3.0 > > > Since IMPALA-11240 the default value of ssl_cipher_list is not empty. > Update > https://github.com/apache/impala/blob/20fe8fea58061f5da7d5c0e7d26755712d02ef79/docs/topics/impala_ssl.xml#L202 > to show thus. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-11835) Document that Iceberg Time Travel queries use the old schema
[ https://issues.apache.org/jira/browse/IMPALA-11835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman resolved IMPALA-11835. - Target Version: Impala 4.3.0 Resolution: Fixed > Document that Iceberg Time Travel queries use the old schema > > > Key: IMPALA-11835 > URL: https://issues.apache.org/jira/browse/IMPALA-11835 > Project: IMPALA > Issue Type: Bug >Affects Versions: Impala 4.3.0 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > > The change in behavior is in IMPALA-10153 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11862) Document that the default value of ssl_cipher_list is not empty
Andrew Sherman created IMPALA-11862: --- Summary: Document that the default value of ssl_cipher_list is not empty Key: IMPALA-11862 URL: https://issues.apache.org/jira/browse/IMPALA-11862 Project: IMPALA Issue Type: Bug Reporter: Andrew Sherman Assignee: Andrew Sherman Since IMPALA-11240 the default value of ssl_cipher_list is not empty. Update https://github.com/apache/impala/blob/20fe8fea58061f5da7d5c0e7d26755712d02ef79/docs/topics/impala_ssl.xml#L202 to show thus. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-10893) Use old schema during time travel
[ https://issues.apache.org/jira/browse/IMPALA-10893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman resolved IMPALA-10893. - Fix Version/s: Impala 4.3.0 Resolution: Fixed > Use old schema during time travel > - > > Key: IMPALA-10893 > URL: https://issues.apache.org/jira/browse/IMPALA-10893 > Project: IMPALA > Issue Type: Sub-task >Reporter: Zoltán Borók-Nagy >Assignee: Andrew Sherman >Priority: Major > Labels: impala-iceberg > Fix For: Impala 4.3.0 > > > With Iceberg V1 we cannot retrieve the old schema of a snapshot. But V2 will > have this functionality: > [https://github.com/apache/iceberg/issues/1029] > So it would make sense to use the old snapshot's table schema during a time > travel query. > Interestingly MS SQL SERVER also queries temporal tables with current table > schema. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11835) Document that Iceberg Time Travel queries use the old schema
[ https://issues.apache.org/jira/browse/IMPALA-11835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated IMPALA-11835: Affects Version/s: Impala 4.3.0 > Document that Iceberg Time Travel queries use the old schema > > > Key: IMPALA-11835 > URL: https://issues.apache.org/jira/browse/IMPALA-11835 > Project: IMPALA > Issue Type: Bug >Affects Versions: Impala 4.3.0 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > > The change in behavior is in IMPALA-10153 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11835) Document that Iceberg Time Travel queries use the old schema
Andrew Sherman created IMPALA-11835: --- Summary: Document that Iceberg Time Travel queries use the old schema Key: IMPALA-11835 URL: https://issues.apache.org/jira/browse/IMPALA-11835 Project: IMPALA Issue Type: Bug Reporter: Andrew Sherman Assignee: Andrew Sherman The change ion behavior is in IMPALA-10153 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-11835) Document that Iceberg Time Travel queries use the old schema
[ https://issues.apache.org/jira/browse/IMPALA-11835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated IMPALA-11835: Description: The change in behavior is in IMPALA-10153 (was: The change ion behavior is in IMPALA-10153) > Document that Iceberg Time Travel queries use the old schema > > > Key: IMPALA-11835 > URL: https://issues.apache.org/jira/browse/IMPALA-11835 > Project: IMPALA > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > > The change in behavior is in IMPALA-10153 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-11811) Avoid storing unregistered predicate objects in a Map
[ https://issues.apache.org/jira/browse/IMPALA-11811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman resolved IMPALA-11811. - Fix Version/s: Impala 4.1.2 Target Version: Impala 4.1.2 Resolution: Fixed > Avoid storing unregistered predicate objects in a Map > - > > Key: IMPALA-11811 > URL: https://issues.apache.org/jira/browse/IMPALA-11811 > Project: IMPALA > Issue Type: Bug > Components: fe >Affects Versions: Impala 1.4.2 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Fix For: Impala 4.1.2 > > > Within the extractIcebergConjuncts() method we are tracking conjuncts which > are identity conjuncts by storing them in a temporary Map. > The conjuncts are Expr objects which have a hashCode() method based on their > id_ field, which is only present when they are registered. > If the id_ field is null, then the hashCode() will throw, and hence > unregistered predicates cannot be stored in a Map. > This can happen if the conjuncts are bound predicates (produced by > getBoundPredicates()) which are explicitly not registered. > Change extractIcebergConjuncts() to track the identity conjuncts in another > way, perhaps using a boolean array which tracks the index of the identity > conjuncts in conjuncts_ List. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-11811) Avoid storing unregistered predicate objects in a Map
[ https://issues.apache.org/jira/browse/IMPALA-11811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-11811 started by Andrew Sherman. --- > Avoid storing unregistered predicate objects in a Map > - > > Key: IMPALA-11811 > URL: https://issues.apache.org/jira/browse/IMPALA-11811 > Project: IMPALA > Issue Type: Bug > Components: fe >Affects Versions: Impala 1.4.2 >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > > Within the extractIcebergConjuncts() method we are tracking conjuncts which > are identity conjuncts by storing them in a temporary Map. > The conjuncts are Expr objects which have a hashCode() method based on their > id_ field, which is only present when they are registered. > If the id_ field is null, then the hashCode() will throw, and hence > unregistered predicates cannot be stored in a Map. > This can happen if the conjuncts are bound predicates (produced by > getBoundPredicates()) which are explicitly not registered. > Change extractIcebergConjuncts() to track the identity conjuncts in another > way, perhaps using a boolean array which tracks the index of the identity > conjuncts in conjuncts_ List. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-11811) Avoid storing unregistered predicate objects in a Map
Andrew Sherman created IMPALA-11811: --- Summary: Avoid storing unregistered predicate objects in a Map Key: IMPALA-11811 URL: https://issues.apache.org/jira/browse/IMPALA-11811 Project: IMPALA Issue Type: Bug Components: fe Affects Versions: Impala 1.4.2 Reporter: Andrew Sherman Assignee: Andrew Sherman Within the extractIcebergConjuncts() method we are tracking conjuncts which are identity conjuncts by storing them in a temporary Map. The conjuncts are Expr objects which have a hashCode() method based on their id_ field, which is only present when they are registered. If the id_ field is null, then the hashCode() will throw, and hence unregistered predicates cannot be stored in a Map. This can happen if the conjuncts are bound predicates (produced by getBoundPredicates()) which are explicitly not registered. Change extractIcebergConjuncts() to track the identity conjuncts in another way, perhaps using a boolean array which tracks the index of the identity conjuncts in conjuncts_ List. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-11756) testPartitionFileMetadataAfterMinorCompaction failing with bad file count
[
https://issues.apache.org/jira/browse/IMPALA-11756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642697#comment-17642697
]
Andrew Sherman commented on IMPALA-11756:
-
Another variant with the failure in testTableFileMetadataAfterMinorCompaction
{code}
java.lang.AssertionError: expected:<1> but was:<2>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:631)
at
org.apache.impala.catalog.local.CatalogdMetaProviderTest.testFileMetadataAfterCompaction(CatalogdMetaProviderTest.java:627)
at
org.apache.impala.catalog.local.CatalogdMetaProviderTest.testTableFileMetadataAfterMinorCompaction(CatalogdMetaProviderTest.java:556)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)
{code}
> testPartitionFileMetadataAfterMinorCompaction failing with bad file count
> -
>
> Key: IMPALA-11756
> URL: https://issues.apache.org/jira/browse/IMPALA-11756
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 4.1.1
>Reporter: Andrew Sherman
>Assignee: Quanlong Huang
>Priority: Critical
>
> In
> org.apache.impala.catalog.PartialCatalogInfoWriteIdTest.testPartitionFileMetadataAfterMinorCompaction,
> after a minor file compassion the number of files in the table is 3 when we
> expect to see 1.
> {code}
> ava.lang.AssertionError: expected:<1> but was:<3>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:834)
> at org.junit.Assert.assertEquals(Assert.java:645)
> at org.junit.Assert.assertEquals(Assert.java:631)
> at
> org.apache.impala.catalog.PartialCatalogInfoWriteIdTest.testFileMetadataAfterCompaction(PartialCatalogInfoWriteIdTest.java:605)
> at
> org.apache.impala.catalog.PartialCatalogInfoWriteIdTest.testPartitionFileMetadataAfterMinorCompaction(PartialCatalogInfoWriteIdTest.java:526)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>
[jira] [Commented] (IMPALA-11772) Server crashes with SIGSEGV in TExprNode::operator= when running UBSAN tests
[
https://issues.apache.org/jira/browse/IMPALA-11772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17642230#comment-17642230
]
Andrew Sherman commented on IMPALA-11772:
-
[~joemcdonnell] assigning to you to triage, please reassign if appropriate
> Server crashes with SIGSEGV in TExprNode::operator= when running UBSAN tests
>
>
> Key: IMPALA-11772
> URL: https://issues.apache.org/jira/browse/IMPALA-11772
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.1.1
>Reporter: Andrew Sherman
>Assignee: Joe McDonnell
>Priority: Critical
> Attachments:
> impalad.impala-ec2-centos79-m6i-4xlarge-ondemand-1fe9.vpc.cloudera.com.jenkins.log.ERROR.20221201-094002.30726
>
>
> {code}
> Operating system: Linux
> 0.0.0 Linux 3.10.0-1160.66.1.el7.x86_64 #1 SMP Wed May 18
> 16:02:34 UTC 2022 x86_64
> CPU: amd64
> family 6 model 106 stepping 6
> 1 CPU
> GPU: UNKNOWN
> Crash reason: SIGSEGV
> Crash address: 0x8
> Process uptime: not available
> Thread 548 (crashed)
> 0 impalad!impala::TExprNode::operator=(impala::TExprNode const&)
> [Exprs_types.cpp : 1027 + 0x7]
> rax = 0x rdx = 0x000c
> rcx = 0x000c rbx = 0x0001
> rsi = 0x000c rdi = 0x2000
> rbp = 0x7f1e7f0cd680 rsp = 0x7f1e7f0cd490
> r8 = 0x0001r9 = 0x
> r10 = 0x r11 = 0x0206
> r12 = 0x r13 = 0x00801000
> r14 = 0x r15 = 0x7f1e7f0d5700
> rip = 0x022e19cd
> Found by: given as instruction pointer in context
> 1 impalad!impala::TExprNode* std::__copy_move std::random_access_iterator_tag>::__copy_m impala::TExprNode*>(impala::TExprNode const*, impala::TExprNode const*,
> impala::TExprNode*) [stl_algobase.h : 324 + 0x19]
> rbx = 0x0001 rbp = 0x7f1e7f0cd710
> rsp = 0x7f1e7f0cd690 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022ecf37
> Found by: call frame info
> 2 impalad!impala::TExprNode* std::__copy_move_a const*, impala::TExprNode*>(impala::TExprNode const*, impala::TExprNode
> const*, impala::TExprNode*) [stl_algobase.h : 385 + 0x5]
> rbx = 0x0001 rbp = 0x7f1e7f0cd740
> rsp = 0x7f1e7f0cd720 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022ecda9
> Found by: call frame info
> 3 impalad!__gnu_cxx::__normal_iterator std::vector > >
> std::__copy_move_a2 const*, std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >
> >(__gnu_cxx::__normal_iterator std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >)
> [stl_algobase.h : 422 + 0x10]
> rbx = 0x0001 rbp = 0x7f1e7f0cd7a0
> rsp = 0x7f1e7f0cd750 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022ecd3f
> Found by: call frame info
> 4 impalad!__gnu_cxx::__normal_iterator std::vector > >
> std::copy<__gnu_cxx::__normal_iterator std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >
> >(__gnu_cxx::__normal_iterator std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >)
> [stl_algobase.h : 454 + 0x11]
> rbx = 0x0001 rbp = 0x7f1e7f0cd800
> rsp = 0x7f1e7f0cd7b0 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022ec5b7
> Found by: call frame info
> 5 impalad!std::vector
> >::operator=(std::vector
> > const&) [vector.tcc : 218 + 0x11]
> rbx = 0x0001 rbp = 0x7f1e7f0cd9e0
> rsp = 0x7f1e7f0cd810 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022e997d
> Found by: call frame info
> 6 impalad!impala::TExpr::TExpr(impala::TExpr const&) [Exprs_types.cpp :
> 1127 + 0x19]
> rbx = 0x0001 rbp = 0x7f1e7f0cda40
> rsp = 0x7f1e7f0cd9f0 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022e5b84
> Found by: call frame info
> 7 impalad!void std::_Construct const&>(impala::TExpr*, impala::TExpr const&) [stl_construct.h : 75 + 0x19]
> rbx = 0x0001 rbp = 0x7f1e7f0cda80
> rsp = 0x7f1e7f0cda50
[jira] [Assigned] (IMPALA-11772) Server crashes with SIGSEGV in TExprNode::operator= when running UBSAN tests
[
https://issues.apache.org/jira/browse/IMPALA-11772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman reassigned IMPALA-11772:
---
Assignee: Joe McDonnell
> Server crashes with SIGSEGV in TExprNode::operator= when running UBSAN tests
>
>
> Key: IMPALA-11772
> URL: https://issues.apache.org/jira/browse/IMPALA-11772
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.1.1
>Reporter: Andrew Sherman
>Assignee: Joe McDonnell
>Priority: Critical
> Attachments:
> impalad.impala-ec2-centos79-m6i-4xlarge-ondemand-1fe9.vpc.cloudera.com.jenkins.log.ERROR.20221201-094002.30726
>
>
> {code}
> Operating system: Linux
> 0.0.0 Linux 3.10.0-1160.66.1.el7.x86_64 #1 SMP Wed May 18
> 16:02:34 UTC 2022 x86_64
> CPU: amd64
> family 6 model 106 stepping 6
> 1 CPU
> GPU: UNKNOWN
> Crash reason: SIGSEGV
> Crash address: 0x8
> Process uptime: not available
> Thread 548 (crashed)
> 0 impalad!impala::TExprNode::operator=(impala::TExprNode const&)
> [Exprs_types.cpp : 1027 + 0x7]
> rax = 0x rdx = 0x000c
> rcx = 0x000c rbx = 0x0001
> rsi = 0x000c rdi = 0x2000
> rbp = 0x7f1e7f0cd680 rsp = 0x7f1e7f0cd490
> r8 = 0x0001r9 = 0x
> r10 = 0x r11 = 0x0206
> r12 = 0x r13 = 0x00801000
> r14 = 0x r15 = 0x7f1e7f0d5700
> rip = 0x022e19cd
> Found by: given as instruction pointer in context
> 1 impalad!impala::TExprNode* std::__copy_move std::random_access_iterator_tag>::__copy_m impala::TExprNode*>(impala::TExprNode const*, impala::TExprNode const*,
> impala::TExprNode*) [stl_algobase.h : 324 + 0x19]
> rbx = 0x0001 rbp = 0x7f1e7f0cd710
> rsp = 0x7f1e7f0cd690 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022ecf37
> Found by: call frame info
> 2 impalad!impala::TExprNode* std::__copy_move_a const*, impala::TExprNode*>(impala::TExprNode const*, impala::TExprNode
> const*, impala::TExprNode*) [stl_algobase.h : 385 + 0x5]
> rbx = 0x0001 rbp = 0x7f1e7f0cd740
> rsp = 0x7f1e7f0cd720 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022ecda9
> Found by: call frame info
> 3 impalad!__gnu_cxx::__normal_iterator std::vector > >
> std::__copy_move_a2 const*, std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >
> >(__gnu_cxx::__normal_iterator std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >)
> [stl_algobase.h : 422 + 0x10]
> rbx = 0x0001 rbp = 0x7f1e7f0cd7a0
> rsp = 0x7f1e7f0cd750 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022ecd3f
> Found by: call frame info
> 4 impalad!__gnu_cxx::__normal_iterator std::vector > >
> std::copy<__gnu_cxx::__normal_iterator std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >
> >(__gnu_cxx::__normal_iterator std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >)
> [stl_algobase.h : 454 + 0x11]
> rbx = 0x0001 rbp = 0x7f1e7f0cd800
> rsp = 0x7f1e7f0cd7b0 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022ec5b7
> Found by: call frame info
> 5 impalad!std::vector
> >::operator=(std::vector
> > const&) [vector.tcc : 218 + 0x11]
> rbx = 0x0001 rbp = 0x7f1e7f0cd9e0
> rsp = 0x7f1e7f0cd810 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022e997d
> Found by: call frame info
> 6 impalad!impala::TExpr::TExpr(impala::TExpr const&) [Exprs_types.cpp :
> 1127 + 0x19]
> rbx = 0x0001 rbp = 0x7f1e7f0cda40
> rsp = 0x7f1e7f0cd9f0 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022e5b84
> Found by: call frame info
> 7 impalad!void std::_Construct const&>(impala::TExpr*, impala::TExpr const&) [stl_construct.h : 75 + 0x19]
> rbx = 0x0001 rbp = 0x7f1e7f0cda80
> rsp = 0x7f1e7f0cda50 r12 = 0x
> r13 = 0x00801000 r14 = 0x
[jira] [Updated] (IMPALA-11772) Server crashes with SIGSEGV in TExprNode::operator= when running UBSAN tests
[
https://issues.apache.org/jira/browse/IMPALA-11772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated IMPALA-11772:
Description:
{code}
Operating system: Linux
0.0.0 Linux 3.10.0-1160.66.1.el7.x86_64 #1 SMP Wed May 18
16:02:34 UTC 2022 x86_64
CPU: amd64
family 6 model 106 stepping 6
1 CPU
GPU: UNKNOWN
Crash reason: SIGSEGV
Crash address: 0x8
Process uptime: not available
Thread 548 (crashed)
0 impalad!impala::TExprNode::operator=(impala::TExprNode const&)
[Exprs_types.cpp : 1027 + 0x7]
rax = 0x rdx = 0x000c
rcx = 0x000c rbx = 0x0001
rsi = 0x000c rdi = 0x2000
rbp = 0x7f1e7f0cd680 rsp = 0x7f1e7f0cd490
r8 = 0x0001r9 = 0x
r10 = 0x r11 = 0x0206
r12 = 0x r13 = 0x00801000
r14 = 0x r15 = 0x7f1e7f0d5700
rip = 0x022e19cd
Found by: given as instruction pointer in context
1 impalad!impala::TExprNode* std::__copy_move::__copy_m(impala::TExprNode const*, impala::TExprNode const*,
impala::TExprNode*) [stl_algobase.h : 324 + 0x19]
rbx = 0x0001 rbp = 0x7f1e7f0cd710
rsp = 0x7f1e7f0cd690 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x022ecf37
Found by: call frame info
2 impalad!impala::TExprNode* std::__copy_move_a(impala::TExprNode const*, impala::TExprNode const*,
impala::TExprNode*) [stl_algobase.h : 385 + 0x5]
rbx = 0x0001 rbp = 0x7f1e7f0cd740
rsp = 0x7f1e7f0cd720 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x022ecda9
Found by: call frame info
3 impalad!__gnu_cxx::__normal_iterator > >
std::__copy_move_a2 > >,
__gnu_cxx::__normal_iterator > >
>(__gnu_cxx::__normal_iterator > >,
__gnu_cxx::__normal_iterator > >,
__gnu_cxx::__normal_iterator > >) [stl_algobase.h : 422 + 0x10]
rbx = 0x0001 rbp = 0x7f1e7f0cd7a0
rsp = 0x7f1e7f0cd750 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x022ecd3f
Found by: call frame info
4 impalad!__gnu_cxx::__normal_iterator > >
std::copy<__gnu_cxx::__normal_iterator > >,
__gnu_cxx::__normal_iterator > >
>(__gnu_cxx::__normal_iterator > >,
__gnu_cxx::__normal_iterator > >,
__gnu_cxx::__normal_iterator > >) [stl_algobase.h : 454 + 0x11]
rbx = 0x0001 rbp = 0x7f1e7f0cd800
rsp = 0x7f1e7f0cd7b0 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x022ec5b7
Found by: call frame info
5 impalad!std::vector
>::operator=(std::vector >
const&) [vector.tcc : 218 + 0x11]
rbx = 0x0001 rbp = 0x7f1e7f0cd9e0
rsp = 0x7f1e7f0cd810 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x022e997d
Found by: call frame info
6 impalad!impala::TExpr::TExpr(impala::TExpr const&) [Exprs_types.cpp : 1127
+ 0x19]
rbx = 0x0001 rbp = 0x7f1e7f0cda40
rsp = 0x7f1e7f0cd9f0 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x022e5b84
Found by: call frame info
7 impalad!void std::_Construct(impala::TExpr*, impala::TExpr const&) [stl_construct.h : 75 + 0x19]
rbx = 0x0001 rbp = 0x7f1e7f0cda80
rsp = 0x7f1e7f0cda50 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x02251df5
Found by: call frame info
8 impalad!impala::TExpr*
std::__uninitialized_copy::__uninit_copy<__gnu_cxx::__normal_iterator > >,
impala::TExpr*>(__gnu_cxx::__normal_iterator > >,
__gnu_cxx::__normal_iterator > >, impala::TExpr*) [stl_uninitialized.h : 83 +
0x19]
rbx = 0x0001 rbp = 0x7f1e7f0cdb10
rsp = 0x7f1e7f0cda90 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x02251c82
Found by: call frame info
9 impalad!impala::TExpr*
std::uninitialized_copy<__gnu_cxx::__normal_iterator > >,
impala::TExpr*>(__gnu_cxx::__normal_iterator > >,
__gnu_cxx::__normal_iterator > >, impala::TExpr*) [stl_uninitialized.h : 131 +
0xd]
rbx = 0x0001 rbp = 0x7f1e7f0cdb50
rsp = 0x7f1e7f0cdb20 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x02251bd9
Found by: call frame info
10
[jira] [Updated] (IMPALA-11772) Server crashes with SIGSEGV in TExprNode::operator= when running UBSAN tests
[
https://issues.apache.org/jira/browse/IMPALA-11772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated IMPALA-11772:
Attachment:
impalad.impala-ec2-centos79-m6i-4xlarge-ondemand-1fe9.vpc.cloudera.com.jenkins.log.ERROR.20221201-094002.30726
> Server crashes with SIGSEGV in TExprNode::operator= when running UBSAN tests
>
>
> Key: IMPALA-11772
> URL: https://issues.apache.org/jira/browse/IMPALA-11772
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.1.1
>Reporter: Andrew Sherman
>Priority: Critical
> Attachments:
> impalad.impala-ec2-centos79-m6i-4xlarge-ondemand-1fe9.vpc.cloudera.com.jenkins.log.ERROR.20221201-094002.30726
>
>
> {code}
> Operating system: Linux
> 0.0.0 Linux 3.10.0-1160.66.1.el7.x86_64 #1 SMP Wed May 18
> 16:02:34 UTC 2022 x86_64
> CPU: amd64
> family 6 model 106 stepping 6
> 1 CPU
> GPU: UNKNOWN
> Crash reason: SIGSEGV
> Crash address: 0x8
> Process uptime: not available
> Thread 548 (crashed)
> 0 impalad!impala::TExprNode::operator=(impala::TExprNode const&)
> [Exprs_types.cpp : 1027 + 0x7]
> rax = 0x rdx = 0x000c
> rcx = 0x000c rbx = 0x0001
> rsi = 0x000c rdi = 0x2000
> rbp = 0x7f1e7f0cd680 rsp = 0x7f1e7f0cd490
> r8 = 0x0001r9 = 0x
> r10 = 0x r11 = 0x0206
> r12 = 0x r13 = 0x00801000
> r14 = 0x r15 = 0x7f1e7f0d5700
> rip = 0x022e19cd
> Found by: given as instruction pointer in context
> 1 impalad!impala::TExprNode* std::__copy_move std::random_access_iterator_tag>::__copy_m impala::TExprNode*>(impala::TExprNode const*, impala::TExprNode const*,
> impala::TExprNode*) [stl_algobase.h : 324 + 0x19]
> rbx = 0x0001 rbp = 0x7f1e7f0cd710
> rsp = 0x7f1e7f0cd690 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022ecf37
> Found by: call frame info
> 2 impalad!impala::TExprNode* std::__copy_move_a const*, impala::TExprNode*>(impala::TExprNode const*, impala::TExprNode
> const*, impala::TExprNode*) [stl_algobase.h : 385 + 0x5]
> rbx = 0x0001 rbp = 0x7f1e7f0cd740
> rsp = 0x7f1e7f0cd720 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022ecda9
> Found by: call frame info
> 3 impalad!__gnu_cxx::__normal_iterator std::vector > >
> std::__copy_move_a2 const*, std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >
> >(__gnu_cxx::__normal_iterator std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >)
> [stl_algobase.h : 422 + 0x10]
> rbx = 0x0001 rbp = 0x7f1e7f0cd7a0
> rsp = 0x7f1e7f0cd750 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022ecd3f
> Found by: call frame info
> 4 impalad!__gnu_cxx::__normal_iterator std::vector > >
> std::copy<__gnu_cxx::__normal_iterator std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >
> >(__gnu_cxx::__normal_iterator std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >,
> __gnu_cxx::__normal_iterator std::vector > >)
> [stl_algobase.h : 454 + 0x11]
> rbx = 0x0001 rbp = 0x7f1e7f0cd800
> rsp = 0x7f1e7f0cd7b0 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022ec5b7
> Found by: call frame info
> 5 impalad!std::vector
> >::operator=(std::vector
> > const&) [vector.tcc : 218 + 0x11]
> rbx = 0x0001 rbp = 0x7f1e7f0cd9e0
> rsp = 0x7f1e7f0cd810 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022e997d
> Found by: call frame info
> 6 impalad!impala::TExpr::TExpr(impala::TExpr const&) [Exprs_types.cpp :
> 1127 + 0x19]
> rbx = 0x0001 rbp = 0x7f1e7f0cda40
> rsp = 0x7f1e7f0cd9f0 r12 = 0x
> r13 = 0x00801000 r14 = 0x
> r15 = 0x7f1e7f0d5700 rip = 0x022e5b84
> Found by: call frame info
> 7 impalad!void std::_Construct const&>(impala::TExpr*, impala::TExpr const&) [stl_construct.h : 75 + 0x19]
> rbx = 0x0001 rbp = 0x7f1e7f0cda80
> rsp = 0x7f1e7f0cda50 r12 = 0x
>
[jira] [Created] (IMPALA-11772) Server crashes with SIGSEGV in TExprNode::operator= when running UBSAN tests
Andrew Sherman created IMPALA-11772:
---
Summary: Server crashes with SIGSEGV in TExprNode::operator= when
running UBSAN tests
Key: IMPALA-11772
URL: https://issues.apache.org/jira/browse/IMPALA-11772
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 4.1.1
Reporter: Andrew Sherman
{code}
Operating system: Linux
0.0.0 Linux 3.10.0-1160.66.1.el7.x86_64 #1 SMP Wed May 18
16:02:34 UTC 2022 x86_64
CPU: amd64
family 6 model 106 stepping 6
1 CPU
GPU: UNKNOWN
Crash reason: SIGSEGV
Crash address: 0x8
Process uptime: not available
Thread 548 (crashed)
0 impalad!impala::TExprNode::operator=(impala::TExprNode const&)
[Exprs_types.cpp : 1027 + 0x7]
rax = 0x rdx = 0x000c
rcx = 0x000c rbx = 0x0001
rsi = 0x000c rdi = 0x2000
rbp = 0x7f1e7f0cd680 rsp = 0x7f1e7f0cd490
r8 = 0x0001r9 = 0x
r10 = 0x r11 = 0x0206
r12 = 0x r13 = 0x00801000
r14 = 0x r15 = 0x7f1e7f0d5700
rip = 0x022e19cd
Found by: given as instruction pointer in context
1 impalad!impala::TExprNode* std::__copy_move::__copy_m(impala::TExprNode const*, impala::TExprNode const*,
impala::TExprNode*) [stl_algobase.h : 324 + 0x19]
rbx = 0x0001 rbp = 0x7f1e7f0cd710
rsp = 0x7f1e7f0cd690 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x022ecf37
Found by: call frame info
2 impalad!impala::TExprNode* std::__copy_move_a(impala::TExprNode const*, impala::TExprNode const*,
impala::TExprNode*) [stl_algobase.h : 385 + 0x5]
rbx = 0x0001 rbp = 0x7f1e7f0cd740
rsp = 0x7f1e7f0cd720 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x022ecda9
Found by: call frame info
3 impalad!__gnu_cxx::__normal_iterator > >
std::__copy_move_a2 > >,
__gnu_cxx::__normal_iterator > >
>(__gnu_cxx::__normal_iterator > >,
__gnu_cxx::__normal_iterator > >,
__gnu_cxx::__normal_iterator > >) [stl_algobase.h : 422 + 0x10]
rbx = 0x0001 rbp = 0x7f1e7f0cd7a0
rsp = 0x7f1e7f0cd750 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x022ecd3f
Found by: call frame info
4 impalad!__gnu_cxx::__normal_iterator > >
std::copy<__gnu_cxx::__normal_iterator > >,
__gnu_cxx::__normal_iterator > >
>(__gnu_cxx::__normal_iterator > >,
__gnu_cxx::__normal_iterator > >,
__gnu_cxx::__normal_iterator > >) [stl_algobase.h : 454 + 0x11]
rbx = 0x0001 rbp = 0x7f1e7f0cd800
rsp = 0x7f1e7f0cd7b0 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x022ec5b7
Found by: call frame info
5 impalad!std::vector
>::operator=(std::vector >
const&) [vector.tcc : 218 + 0x11]
rbx = 0x0001 rbp = 0x7f1e7f0cd9e0
rsp = 0x7f1e7f0cd810 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x022e997d
Found by: call frame info
6 impalad!impala::TExpr::TExpr(impala::TExpr const&) [Exprs_types.cpp : 1127
+ 0x19]
rbx = 0x0001 rbp = 0x7f1e7f0cda40
rsp = 0x7f1e7f0cd9f0 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x022e5b84
Found by: call frame info
7 impalad!void std::_Construct(impala::TExpr*, impala::TExpr const&) [stl_construct.h : 75 + 0x19]
rbx = 0x0001 rbp = 0x7f1e7f0cda80
rsp = 0x7f1e7f0cda50 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x02251df5
Found by: call frame info
8 impalad!impala::TExpr*
std::__uninitialized_copy::__uninit_copy<__gnu_cxx::__normal_iterator > >,
impala::TExpr*>(__gnu_cxx::__normal_iterator > >,
__gnu_cxx::__normal_iterator > >, impala::TExpr*) [stl_uninitialized.h : 83 +
0x19]
rbx = 0x0001 rbp = 0x7f1e7f0cdb10
rsp = 0x7f1e7f0cda90 r12 = 0x
r13 = 0x00801000 r14 = 0x
r15 = 0x7f1e7f0d5700 rip = 0x02251c82
Found by: call frame info
9 impalad!impala::TExpr*
std::uninitialized_copy<__gnu_cxx::__normal_iterator > >,
impala::TExpr*>(__gnu_cxx::__normal_iterator > >,
__gnu_cxx::__normal_iterator > >, impala::TExpr*) [stl_uninitialized.h : 131 +
0xd]
rbx = 0x0001 rbp =
