Hmm, that test is potentially timing sensitive. We've seen problems when
running with slow builds (e.g. code coverage) or running it on a
particularly slow machine? E.g. single-core VM. It's probably ok to skip
the test on PowerPC if this is the case.
The query is expected to fail, but in this case no failure is happening.
It's a "custom cluster test" that configures the cluster in a way that
queries will fail with a timeout. It's test coverage for a bug where if the
timeout happens Impala returned incorrect results.
If you run the query on Impala with the default startup arguments it should
succeed.
If you start up Impala with the special configuration used by those tests,
it should fail. E.g. locally I get:
tarmstrong@tarmstrong-box:~/Impala/Impala$ ./bin/start-impala-cluster.py
--impalad_args=--datastream_sender_timeout_ms=5000
--impalad_args=--stress_datastream_recvr_delay_ms=10000
Starting State Store logging to
/home/tarmstrong/Impala/Impala/logs/cluster/statestored.INFO
Starting Catalog Service logging to
/home/tarmstrong/Impala/Impala/logs/cluster/catalogd.INFO
Starting Impala Daemon logging to
/home/tarmstrong/Impala/Impala/logs/cluster/impalad.INFO
Starting Impala Daemon logging to
/home/tarmstrong/Impala/Impala/logs/cluster/impalad_node1.INFO
Starting Impala Daemon logging to
/home/tarmstrong/Impala/Impala/logs/cluster/impalad_node2.INFO
MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
MainThread: Getting num_known_live_backends from tarmstrong-box:25000
MainThread: Waiting for num_known_live_backends=3. Current value: 0
MainThread: Getting num_known_live_backends from tarmstrong-box:25000
MainThread: Waiting for num_known_live_backends=3. Current value: 0
MainThread: Getting num_known_live_backends from tarmstrong-box:25000
MainThread: Waiting for num_known_live_backends=3. Current value: 2
MainThread: Getting num_known_live_backends from tarmstrong-box:25000
MainThread: Waiting for num_known_live_backends=3. Current value: 2
MainThread: Getting num_known_live_backends from tarmstrong-box:25000
MainThread: num_known_live_backends has reached value: 3
Waiting for Catalog... Status: 63 DBs / 1091 tables (ready=True)
MainThread: Getting num_known_live_backends from tarmstrong-box:25001
MainThread: num_known_live_backends has reached value: 3
Waiting for Catalog... Status: 63 DBs / 1091 tables (ready=True)
MainThread: Getting num_known_live_backends from tarmstrong-box:25002
MainThread: num_known_live_backends has reached value: 3
Waiting for Catalog... Status: 63 DBs / 1091 tables (ready=True)
Impala Cluster Running with 3 nodes.
tarmstrong@tarmstrong-box:~/Impala/Impala$ impala-shell.sh
Starting Impala Shell without Kerberos authentication
Connected to tarmstrong-box.ca.cloudera.com:21000
Server version: impalad version 2.6.0-cdh5-INTERNAL DEBUG (build
fe23dbf0465220a0c40a5c8431cb6a536e19dc6b)
***********************************************************************************
Welcome to the Impala shell. Copyright (c) 2015 Cloudera, Inc. All rights
reserved.
(Impala Shell v2.6.0-cdh5-INTERNAL (fe23dbf) built on Fri May 13 11:15:16
PDT 2016)
You can run a single query from the command line using the '-q' option.
***********************************************************************************
[tarmstrong-box.ca.cloudera.com:21000] > select count(*)
> from tpch.lineitem
> inner join tpch.orders on
l_orderkey = o_orderkey
> ;
Query: select count(*)
from tpch.lineitem
inner join tpch.orders on l_orderkey = o_orderkey
WARNINGS:
Sender timed out waiting for receiver fragment instance:
4cbdf04962743c:faa6717f926b5183
(1 of 2 similar)
You could try increasing the delay on your setup to see if you can
replicate the failure.
On Thu, Jun 23, 2016 at 3:54 AM, Nishidha Panpaliya <[email protected]>
wrote:
>
> Hi All,
>
> On power8, we are getting 3 failures in custom cluster test failure. 2 test
> cases failed in test_admission_controller.py and 1 in
> test_exchange_delays.py. I investigated the test failure in
> test_exchange_delays.py and below is my finding.
>
> Test case failed is "test_exchange_small_delay". This test has input
> test file as "QueryTest/exchange-delays",
> --stress_datastream_recvr_delay_ms=10000 and
> --datastream_sender_timeout_ms=5000.
> The test is expected to throw an exception, with message as mentioned in
> CATCH section in QueryTest/exchange-delays.
> However, at our end, the query in this test does not throw any
> exception, but since QueryTest/exchange-delays has CATCH section
> mentioned, the test case fails due to assertion in
> tests/common/impala_test_suite.py as below -
> if 'CATCH' in test_section:
> assert test_section['CATCH'].strip() == ''
> 4. If I remove CATCH section from exchange-delays.test file, then this
> test case passes, however, another test case in the same test file fails,
> as it throws exception as per inputs given to it but CATCH section is
> missing.
> 5. On another RHEL ppc machine, this test randomly passes i.e. both the
> test cases throws exception as expected.
>
> I'm really confused as to what parameter is leading the test case
> "test_exchange_small_delay" to not throw any exception in my setup. Or what
> should actually be happened?
> I checked latest cdh5-trunk code on github and it also has same test code
> and same content in query test file.
>
> Kindly provide me some pointers.
>
> Thanks,
> Nishidha
>
