[jira] [Created] (IMPALA-6592) Crash in Parquet Scanner at BaseScalarColumnReader::Reset()
Tim Armstrong created IMPALA-6592:
-
Summary: Crash in Parquet Scanner at
BaseScalarColumnReader::Reset()
Key: IMPALA-6592
URL: https://issues.apache.org/jira/browse/IMPALA-6592
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 2.12.0
Reporter: Tim Armstrong
Assignee: Tim Armstrong
I wasn't able to get a full stacktrace or core, but the frame where it crashed
was:
{noformat}
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x01d953e0, pid=5401, tid=140122118891264
#
# JRE version: Java(TM) SE Runtime Environment (8.0_05-b13) (build 1.8.0_05-b13)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.5-b02 mixed mode linux-amd64
compressed oops)
# Problematic frame:
# C [impalad+0x19953e0]
impala::BaseScalarColumnReader::Reset(impala::HdfsFileDesc const&,
parquet::ColumnChunk const&, int)+0x2d0
#
# Failed to write core dump. Core dumps have been disabled. To enable core
dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
{noformat}
This was an ASAN build. The parquet fuzz test was running, so it's the likely
culprit:
{noformat}
10:28:05 ERRORS
10:28:05 ERROR at teardown of
TestScannersFuzzing.test_fuzz_alltypes[exec_option: {'debug_action':
'-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0', 'abort_on_error': False,
'mem_limit': '512m', 'num_nodes': 0} | table_format: parquet/none]
10:28:05 [gw0] linux2 -- Python 2.6.6
/data/jenkins/workspace/impala-asf-2.x-core-asan/repos/Impala/bin/../infra/python/env/bin/python
10:28:05 conftest.py:285: in cleanup
10:28:05
request.instance.execute_query_expect_success(request.instance.client, "use
default")
10:28:05 common/impala_test_suite.py:520: in wrapper
10:28:05 return function(*args, **kwargs)
10:28:05 common/impala_test_suite.py:527: in execute_query_expect_success
10:28:05 result = cls.__execute_query(impalad_client, query, query_options)
10:28:05 common/impala_test_suite.py:612: in __execute_query
10:28:05 return impalad_client.execute(query, user=user)
10:28:05 common/impala_connection.py:160: in execute
10:28:05 return self.__beeswax_client.execute(sql_stmt, user=user)
10:28:05 beeswax/impala_beeswax.py:173: in execute
10:28:05 handle = self.__execute_query(query_string.strip(), user=user)
10:28:05 beeswax/impala_beeswax.py:339: in __execute_query
10:28:05 handle = self.execute_query_async(query_string, user=user)
10:28:05 beeswax/impala_beeswax.py:335: in execute_query_async
10:28:05 return self.__do_rpc(lambda: self.imp_service.query(query,))
10:28:05 beeswax/impala_beeswax.py:456: in __do_rpc
10:28:05 raise ImpalaBeeswaxException("Not connected", None)
10:28:05 E ImpalaBeeswaxException: ImpalaBeeswaxException:
10:28:05 ENot connected
10:28:05 Captured stderr setup
-
10:28:05 SET sync_ddl=False;
10:28:05 -- executing against localhost:21000
10:28:05 DROP DATABASE IF EXISTS `test_fuzz_alltypes_d2734b63` CASCADE;
10:28:05
10:28:05 SET sync_ddl=False;
10:28:05 -- executing against localhost:21000
10:28:05 CREATE DATABASE `test_fuzz_alltypes_d2734b63`;
10:28:05
10:28:05 MainThread: Created database "test_fuzz_alltypes_d2734b63" for test ID
"query_test/test_scanners_fuzz.py::TestScannersFuzzing::()::test_fuzz_alltypes[exec_option:
{'debug_action': '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0',
'abort_on_error': False, 'mem_limit': '512m', 'num_nodes': 0} | table_format:
parquet/none]"
10:28:05 - Captured stderr call
-
10:28:05 MainThread: Using random seed 1519669602
10:28:05 -- executing against localhost:21000
10:28:05 create table test_fuzz_alltypes_d2734b63.alltypes like
functional_parquet.alltypes;
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IMPALA-6591) TestClientSsl hung for a long time
Tim Armstrong created IMPALA-6591:
-
Summary: TestClientSsl hung for a long time
Key: IMPALA-6591
URL: https://issues.apache.org/jira/browse/IMPALA-6591
Project: IMPALA
Issue Type: Bug
Components: Distributed Exec
Affects Versions: Impala 2.12.0
Reporter: Tim Armstrong
Assignee: Sailesh Mukil
{noformat}
18:49:13
custom_cluster/test_catalog_wait.py::TestCatalogWait::test_delayed_catalog
PASSED
18:49:53
custom_cluster/test_client_ssl.py::TestClientSsl::test_ssl[exec_option:
{'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
'disable_codegen': False, 'abort_on_error': 1, 'debug_action': None,
'exec_single_node_rows_threshold': 0} | table_format: text/none] Build timed
out (after 1,440 minutes). Marking the build as failed.
12:20:15 Build was aborted
12:20:15 Archiving artifacts
{noformat}
I unfortunately wasn't able to get any logs...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IMPALA-6590) Disable expr rewrites for VALUES() statements
Alexander Behm created IMPALA-6590:
--
Summary: Disable expr rewrites for VALUES() statements
Key: IMPALA-6590
URL: https://issues.apache.org/jira/browse/IMPALA-6590
Project: IMPALA
Issue Type: Improvement
Components: Frontend
Affects Versions: Impala 2.11.0, Impala 2.10.0, Impala 2.9.0, Impala 2.8.0
Reporter: Alexander Behm
The analysis of statements with big VALUES clauses like INSERT INTO
VALUES is slow due to expression rewrites like constant folding. The
performance of such statements has regressed since the introduction of expr
rewrites and constant folding in IMPALA-1788.
We should skip expr rewrites for VALUES altogether since it mostly provides no
benefit but can have a large overhead due to evaluation of expressions in the
backend (constant folding). These expressions are ultimately evaluated and
materialized in the backend anyway, so there's no point in folding them during
analysis.
*Workaround*
{code}
SET ENABLE_EXPR_REWRITES=FALSE;
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-6467) Crash with TPC-DS Q80A with decimal v2 in ASAN
[ https://issues.apache.org/jira/browse/IMPALA-6467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach Amsden resolved IMPALA-6467. - Resolution: Duplicate > Crash with TPC-DS Q80A with decimal v2 in ASAN > -- > > Key: IMPALA-6467 > URL: https://issues.apache.org/jira/browse/IMPALA-6467 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 3.0 >Reporter: Dimitris Tsirogiannis >Assignee: Zach Amsden >Priority: Blocker > Labels: ASAN, broken-build, crash, test-failure > > There seems to be an impalad crash when running TPC-DS Q80A with decimal v2 > in ASAN mode. Unfortunately, there is no core dump or stacktrace. > [~tarasbob], assigning this to you for initial triage as it seems to be > related to the decimal v2 changes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-6589) Fuzz test on parquet table crashes impala
Tim Armstrong created IMPALA-6589: - Summary: Fuzz test on parquet table crashes impala Key: IMPALA-6589 URL: https://issues.apache.org/jira/browse/IMPALA-6589 Project: IMPALA Issue Type: Bug Components: Backend Affects Versions: Impala 3.0, Impala 2.12.0 Reporter: Tim Armstrong Assignee: Tim Armstrong -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-6582) Flaky test: TestImpalaShellInteractive::test_multiline_queries_in_history
[
https://issues.apache.org/jira/browse/IMPALA-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-6582.
---
Resolution: Fixed
Fix Version/s: Impala 2.12.0
Impala 3.0
> Flaky test: TestImpalaShellInteractive::test_multiline_queries_in_history
> -
>
> Key: IMPALA-6582
> URL: https://issues.apache.org/jira/browse/IMPALA-6582
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 2.12.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: flaky
> Fix For: Impala 3.0, Impala 2.12.0
>
>
> This failed on an exhaustive debug build on commit
> 9d7b2103f5aaaec7013e5ec172a96a88d83dbbb9:
> {noformat}
> 03:24:52 FAIL
> shell/test_shell_interactive.py::TestImpalaShellInteractive::()::test_multiline_queries_in_history
> 03:24:52 === FAILURES
> ===
> 03:24:52 _
> TestImpalaShellInteractive.test_multiline_queries_in_history _
> 03:24:52 shell/test_shell_interactive.py:224: in
> test_multiline_queries_in_history
> 03:24:52 child_proc.expect(prompt_regex)
> 03:24:52
> ../infra/python/env/lib/python2.6/site-packages/pexpect/__init__.py:1451: in
> expect
> 03:24:52 timeout, searchwindowsize)
> 03:24:52
> ../infra/python/env/lib/python2.6/site-packages/pexpect/__init__.py:1466: in
> expect_list
> 03:24:52 timeout, searchwindowsize)
> 03:24:52
> ../infra/python/env/lib/python2.6/site-packages/pexpect/__init__.py:1568: in
> expect_loop
> 03:24:52 raise TIMEOUT(str(err) + '\n' + str(self))
> 03:24:52 E TIMEOUT: Timeout exceeded.
> 03:24:52 E
> 03:24:52 E version: 3.3
> 03:24:52 E command:
> /data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/bin/impala-shell.sh
> 03:24:52 E args:
> ['/data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/bin/impala-shell.sh']
> 03:24:52 E searcher:
> 03:24:52 E buffer (last 100 chars): 'select\r\n
> > 1--comment;\r\n > '
> 03:24:52 E before (last 100 chars): 'select\r\n
> > 1--comment;\r\n > '
> 03:24:52 E after:
> 03:24:52 E match: None
> 03:24:52 E match_index: None
> 03:24:52 E exitstatus: None
> 03:24:52 E flag_eof: False
> 03:24:52 E pid: 3377
> 03:24:52 E child_fd: 10
> 03:24:52 E closed: False
> 03:24:52 E timeout: 30
> 03:24:52 E delimiter:
> 03:24:52 E logfile: None
> 03:24:52 E logfile_read: None
> 03:24:52 E logfile_send: None
> 03:24:52 E maxread: 2000
> 03:24:52 E ignorecase: False
> 03:24:52 E searchwindowsize: None
> 03:24:52 E delaybeforesend: 0.05
> 03:24:52 E delayafterclose: 0.1
> 03:24:52 E delayafterterminate: 0.1
> {noformat}
> I'm going to investigate to see if I can reproduce
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-6577) TestQueryExpiration::test_concurrent_query_expiration failing
[
https://issues.apache.org/jira/browse/IMPALA-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-6577.
---
Resolution: Fixed
Fix Version/s: Impala 2.12.0
Impala 3.0
> TestQueryExpiration::test_concurrent_query_expiration failing
> -
>
> Key: IMPALA-6577
> URL: https://issues.apache.org/jira/browse/IMPALA-6577
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Blocker
> Fix For: Impala 3.0, Impala 2.12.0
>
> Attachments:
> impalad.ec2-m2-4xlarge-centos-6-4-17e4.vpc.cloudera.com.jenkins.log.INFO.20180223-154813.8631
>
>
> {noformat}
> 12:00:50
> custom_cluster/test_query_expiration.py::TestQueryExpiration::test_concurrent_query_expiration[exec_option:
> {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
> 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': None,
> 'exec_single_node_rows_threshold': 0} | table_format: text/none] FAILED
> {noformat}
> {noformat}
> 12:16:14 -- executing against
> ec2-m2-4xlarge-centos-6-4-04bd.vpc.cloudera.com:21000
> 12:16:14 SELECT count(*) FROM functional.alltypes;
> 12:16:14
> 12:16:14 -- executing against
> ec2-m2-4xlarge-centos-6-4-04bd.vpc.cloudera.com:21000
> 12:16:14 SELECT SLEEP(2500);
> 12:16:14
> 12:16:14 MainThread: Getting metric: impala-server.num-queries-expired from
> ec2-m2-4xlarge-centos-6-4-04bd.vpc.cloudera.com:25000
> 12:16:14 MainThread: Waiting for metric value
> 'impala-server.num-queries-expired'=10. Current value: 7
> 12:16:14 MainThread: Sleeping 1s before next retry.
> 12:16:14 MainThread: Getting metric: impala-server.num-queries-expired from
> ec2-m2-4xlarge-centos-6-4-04bd.vpc.cloudera.com:25000
> 12:16:14 MainThread: Waiting for metric value
> 'impala-server.num-queries-expired'=10. Current value: 7
> 12:16:14 MainThread: Sleeping 1s before next retry.
> 12:16:14 MainThread: Getting metric: impala-server.num-queries-expired from
> ec2-m2-4xlarge-centos-6-4-04bd.vpc.cloudera.com:25000
> 12:16:14 MainThread: Waiting for metric value
> 'impala-server.num-queries-expired'=10. Current value: 7
> 12:16:14 MainThread: Sleeping 1s before next retry.
> 12:16:14 MainThread: Getting metric: impala-server.num-queries-expired from
> ec2-m2-4xlarge-centos-6-4-04bd.vpc.cloudera.com:25000
> 12:16:14 MainThread: Waiting for metric value
> 'impala-server.num-queries-expired'=10. Current value: 7
> 12:16:14 MainThread: Sleeping 1s before next retry.
> 12:16:14 MainThread: Getting metric: impala-server.num-queries-expired from
> ec2-m2-4xlarge-centos-6-4-04bd.vpc.cloudera.com:25000
> 12:16:14 MainThread: Waiting for metric value
> 'impala-server.num-queries-expired'=10. Current value: 7
> 12:16:14 MainThread: Sleeping 1s before next retry.
> 12:16:14 MainThread: Getting metric: impala-server.num-queries-expired from
> ec2-m2-4xlarge-centos-6-4-04bd.vpc.cloudera.com:25000
> 12:16:14 MainThread: Waiting for metric value
> 'impala-server.num-queries-expired'=10. Current value: 7
> 12:16:14 MainThread: Sleeping 1s before next retry.
> 12:16:14 MainThread: Getting metric: impala-server.num-queries-expired from
> ec2-m2-4xlarge-centos-6-4-04bd.vpc.cloudera.com:25000
> 12:16:14 MainThread: Waiting for metric value
> 'impala-server.num-queries-expired'=10. Current value: 7
> 12:16:14 MainThread: Sleeping 1s before next retry.
> 12:16:14 MainThread: Getting metric: impala-server.num-queries-expired from
> ec2-m2-4xlarge-centos-6-4-04bd.vpc.cloudera.com:25000
> 12:16:14 MainThread: Waiting for metric value
> 'impala-server.num-queries-expired'=10. Current value: 7
> 12:16:14 MainThread: Sleeping 1s before next retry.
> 12:16:14 MainThread: Getting metric: impala-server.num-queries-expired from
> ec2-m2-4xlarge-centos-6-4-04bd.vpc.cloudera.com:25000
> 12:16:14 MainThread: Waiting for metric value
> 'impala-server.num-queries-expired'=10. Current value: 7
> 12:16:14 MainThread: Sleeping 1s before next retry.
> 12:16:14 MainThread: Getting metric: impala-server.num-queries-expired from
> ec2-m2-4xlarge-centos-6-4-04bd.vpc.cloudera.com:25000
> 12:16:14 MainThread: Waiting for metric value
> 'impala-server.num-queries-expired'=10. Current value: 7
> 12:16:14 MainThread: Sleeping 1s before next retry.
> {noformat}
> I recently modified this test, so I'll investigate
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-6029) Crash in parquet::FileMetaData::~FileMetaData
[
https://issues.apache.org/jira/browse/IMPALA-6029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell resolved IMPALA-6029.
---
Resolution: Cannot Reproduce
Haven't seen this in a long time, even with all the KRPC stress tests running.
> Crash in parquet::FileMetaData::~FileMetaData
> -
>
> Key: IMPALA-6029
> URL: https://issues.apache.org/jira/browse/IMPALA-6029
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 2.11.0
>Reporter: Mostafa Mokhtar
>Assignee: Joe McDonnell
>Priority: Critical
>
> Crash occurred while running concurrent TPCDS queries against KRPC branch
> https://github.com/michaelhkw/incubator-impala/tree/krpc-testing-hung
> stack
> {code}
> #0 0x00392c832625 in raise () from /lib64/libc.so.6
> #1 0x00392c833e05 in abort () from /lib64/libc.so.6
> #2 0x7f3353144a55 in os::abort(bool) () from
> /usr/java/jdk1.7.0_67-cloudera/jre/lib/amd64/server/libjvm.so
> #3 0x7f33532c4f87 in VMError::report_and_die() () from
> /usr/java/jdk1.7.0_67-cloudera/jre/lib/amd64/server/libjvm.so
> #4 0x7f335314996f in JVM_handle_linux_signal () from
> /usr/java/jdk1.7.0_67-cloudera/jre/lib/amd64/server/libjvm.so
> #5
> #6 ?? (warning: (Internal error: pc 0x0 in read in psymtab, but not in
> symtab.)
> ) at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/boost-1.57.0-p3/include/boost/system/system_error.hpp:61
> warning: (Internal error: pc 0x0 in read in psymtab, but not in symtab.)
> #7 0x00d8decb in _Destroy
> (this=0x7f31c92d6910, __in_chrg=)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_construct.h:93
> #8 __destroy (this=0x7f31c92d6910, __in_chrg= optimized out>)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_construct.h:103
> #9 _Destroy (this=0x7f31c92d6910, __in_chrg= optimized out>)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_construct.h:126
> #10 _Destroy
> (this=0x7f31c92d6910, __in_chrg=)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_construct.h:151
> #11 ~vector (this=0x7f31c92d6910, __in_chrg=) at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_vector.h:424
> #12 ~RowGroup (this=0x7f31c92d6910, __in_chrg=) at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/generated-sources/gen-cpp/parquet_types.h:1095
> #13 _Destroy (this=0x7f31c92d6910, __in_chrg= optimized out>)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_construct.h:93
> #14 __destroy (this=0x7f31c92d6910, __in_chrg= optimized out>)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_construct.h:103
> #15 _Destroy (this=0x7f31c92d6910, __in_chrg= optimized out>)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_construct.h:126
> #16 _Destroy (this=0x7f31c92d6910,
> __in_chrg=)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_construct.h:151
> #17 ~vector (this=0x7f31c92d6910, __in_chrg=) at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/toolchain/gcc-4.9.2/include/c++/4.9.2/bits/stl_vector.h:424
> #18 parquet::FileMetaData::~FileMetaData (this=0x7f31c92d6910,
> __in_chrg=)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/generated-sources/gen-cpp/parquet_types.h:1241
> #19 0x00d8e2a4 in impala::HdfsParquetScanner::~HdfsParquetScanner
> (this=0x7f31c92d6700, __in_chrg=)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/exec/hdfs-parquet-scanner.h:326
> #20 0x00d8e3c9 in impala::HdfsParquetScanner::~HdfsParquetScanner
> (this=0x7f31c92d6700, __in_chrg=)
> at
> /data/jenkins/workspace/impala-private-build-binaries/repos/Impala/be/src/exec/hdfs-parquet-scanner.h:326
> ---Type to continue, or q to quit---
> #21 0x00d4fd48 in checked_delete
> (this=0x43f3c30, filter_ctxs=..., expr_results_pool=0x8, scan_range=0x81f80)
> at
>
