[jira] [Work started] (IMPALA-8372) Impala Doc: Consistent uses of hyphens with global flags
[ https://issues.apache.org/jira/browse/IMPALA-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-8372 started by Alex Rodoni. --- > Impala Doc: Consistent uses of hyphens with global flags > > > Key: IMPALA-8372 > URL: https://issues.apache.org/jira/browse/IMPALA-8372 > Project: IMPALA > Issue Type: Bug > Components: Docs >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > > Standardize to use 2 non-breaking hyphens for global flags. > https://gerrit.cloudera.org/#/c/12908/ -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8372) Impala Doc: Consistent uses of hyphens with global flags
[ https://issues.apache.org/jira/browse/IMPALA-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-8372: Description: Standardize to use 2 non-breaking hyphens for global flags. https://gerrit.cloudera.org/#/c/12908/ was:Standardize to use 2 non-breaking hyphens for global flags. > Impala Doc: Consistent uses of hyphens with global flags > > > Key: IMPALA-8372 > URL: https://issues.apache.org/jira/browse/IMPALA-8372 > Project: IMPALA > Issue Type: Bug > Components: Docs >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > > Standardize to use 2 non-breaking hyphens for global flags. > https://gerrit.cloudera.org/#/c/12908/ -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-8149) Add support for alter_database events
[ https://issues.apache.org/jira/browse/IMPALA-8149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaomeng Zhang reassigned IMPALA-8149: -- Assignee: Xiaomeng Zhang (was: Vihang Karajgaonkar) > Add support for alter_database events > - > > Key: IMPALA-8149 > URL: https://issues.apache.org/jira/browse/IMPALA-8149 > Project: IMPALA > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Xiaomeng Zhang >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8368) Create database/table with Ranger throws UnsupportedOperationException
[ https://issues.apache.org/jira/browse/IMPALA-8368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Austin Nobis resolved IMPALA-8368. -- Resolution: Fixed > Create database/table with Ranger throws UnsupportedOperationException > -- > > Key: IMPALA-8368 > URL: https://issues.apache.org/jira/browse/IMPALA-8368 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Austin Nobis >Assignee: Austin Nobis >Priority: Major > Fix For: Impala 3.3.0 > > > When executing *create database ;* in Impala with Ranger enabled, > an *UnsupportedOperationException* will be thrown. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8368) Create database/table with Ranger throws UnsupportedOperationException
[ https://issues.apache.org/jira/browse/IMPALA-8368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Austin Nobis resolved IMPALA-8368. -- Resolution: Fixed > Create database/table with Ranger throws UnsupportedOperationException > -- > > Key: IMPALA-8368 > URL: https://issues.apache.org/jira/browse/IMPALA-8368 > Project: IMPALA > Issue Type: Sub-task > Components: Frontend >Reporter: Austin Nobis >Assignee: Austin Nobis >Priority: Major > Fix For: Impala 3.3.0 > > > When executing *create database ;* in Impala with Ranger enabled, > an *UnsupportedOperationException* will be thrown. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7800) Reject or timeout new incoming connections once --fe_service_threads limit is reached
[
https://issues.apache.org/jira/browse/IMPALA-7800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoram Thanga resolved IMPALA-7800.
--
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Reject or timeout new incoming connections once --fe_service_threads limit is
> reached
> -
>
> Key: IMPALA-7800
> URL: https://issues.apache.org/jira/browse/IMPALA-7800
> Project: IMPALA
> Issue Type: Improvement
> Components: Clients
>Affects Versions: Impala 3.0, Impala 2.12.0
>Reporter: Michael Ho
>Assignee: Zoram Thanga
>Priority: Major
> Fix For: Impala 3.3.0
>
>
> Currently, the number of frontend service threads is controlled by
> {{--fe_service_threads}}. Once this limit is reached, the worker threads of
> {{connection_setup_pool}} will block until some of the service threads exit.
> New incoming connections will be buffered in a queue in
> {{connection_setup_pool}} until it fills up. Currently, there is no time out
> for all these buffered connections in the queue. Consequently, if the number
> of frontend service threads max'ed out for an extended period of time, a
> client trying to connect to Impala will feel "stuck" until a front end
> service thread is available.
> We should consider either rejecting any new connections once
> {{--fe_service_threads}} limit is reached or imposing a tunable upper bound
> on the wait time for connections in the queue to improve the users'
> experience. cc'ing [~zoram]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (IMPALA-7290) Move impala-shell to use HS2
[ https://issues.apache.org/jira/browse/IMPALA-7290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong reassigned IMPALA-7290: - Assignee: Tim Armstrong > Move impala-shell to use HS2 > > > Key: IMPALA-7290 > URL: https://issues.apache.org/jira/browse/IMPALA-7290 > Project: IMPALA > Issue Type: Improvement > Components: Clients >Reporter: Tim Armstrong >Assignee: Tim Armstrong >Priority: Major > Labels: maintainability, shell > > Most clients have moved to the HS2 interface. impala-shell is one of the > laggards. We should switch impala-shell to use the newer and more standard > interface. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8378) test_cancel_select failed with Invalid query handle
[
https://issues.apache.org/jira/browse/IMPALA-8378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-8378 started by Andrew Sherman.
--
> test_cancel_select failed with Invalid query handle
> ---
>
> Key: IMPALA-8378
> URL: https://issues.apache.org/jira/browse/IMPALA-8378
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Gabor Kaszab
>Assignee: Andrew Sherman
>Priority: Critical
> Labels: broken-build
>
> Error Message
> {code:java}
> query_test/test_cancellation.py:241: in test_cancel_select
> self.execute_cancel_test(vector) query_test/test_cancellation.py:213: in
> execute_cancel_test assert 'Cancelled' in str(thread.fetch_results_error)
> E assert 'Cancelled' in "ImpalaBeeswaxException:\n INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Invalid query handle:
> b64a91016fe343dd:fefaf600\n" E+ where "ImpalaBeeswaxException:\n
> INNER EXCEPTION: \n MESSAGE:
> Invalid query handle: b64a91016fe343dd:fefaf600\n" =
> str(ImpalaBeeswaxException()) E+where ImpalaBeeswaxException() =
> .fetch_results_error
> {code}
> Stacktrace
> {code:java}
> query_test/test_cancellation.py:241: in test_cancel_select
> self.execute_cancel_test(vector)
> query_test/test_cancellation.py:213: in execute_cancel_test
> assert 'Cancelled' in str(thread.fetch_results_error)
> E assert 'Cancelled' in "ImpalaBeeswaxException:\n INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Invalid query handle:
> b64a91016fe343dd:fefaf600\n"
> E+ where "ImpalaBeeswaxException:\n INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Invalid query handle:
> b64a91016fe343dd:fefaf600\n" = str(ImpalaBeeswaxException())
> E+where ImpalaBeeswaxException() = 139721936148224)>.fetch_results_error
> {code}
> Standard error
> {code:java}
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:rc/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_no;
> -- executing against localhost:21000
> use tpch_rc;
> -- 2019-03-30 23:15:53,293 INFO MainThread: Started query
> 5e42c7ec2df1febb:b25eff72
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:rc/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_no;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET cpu_limit_s=10;
> SET debug_action=0:GETNEXT:WAIT|COORD_CANCEL_QUERY_FINSTANCES_RPC:FAIL;
> SET exec_single_node_rows_threshold=0;
> SET buffer_pool_limit=0;
> -- executing async: localhost:21000
> compute stats lineitem;
> -- 2019-03-30 23:15:53,359 INFO MainThread: Started query
> f54a0ecfa32d40b4:07510d7a
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:rc/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_no;
> -- connecting to: localhost:21000
> -- fetching results from: object at 0x5d535d0>
> -- getting state for operation:
>
> -- canceling operation: object at 0x5d535d0>
> -- 2019-03-30 23:15:53,427 INFO Thread-5: Starting new HTTP connection
> (1): localhost
> -- closing query for operation handle:
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8379) TestMtDopParquet.test_parquet_filtering flaky - exhaustive run
[
https://issues.apache.org/jira/browse/IMPALA-8379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806866#comment-16806866
]
Gabor Kaszab commented on IMPALA-8379:
--
Hey [~tarmstrong]
I see you worked on https://issues.apache.org/jira/browse/IMPALA-6960 that is
quite similar to this one. Do you have an idea what's going on here?
> TestMtDopParquet.test_parquet_filtering flaky - exhaustive run
> --
>
> Key: IMPALA-8379
> URL: https://issues.apache.org/jira/browse/IMPALA-8379
> Project: IMPALA
> Issue Type: Bug
>Reporter: Gabor Kaszab
>Assignee: Tim Armstrong
>Priority: Critical
>
> Jenkins job output
> {code:java}
> 06:33:37 query_test/test_mt_dop.py:103: in test_parquet_filtering
> 06:33:37 self.run_test_case('QueryTest/parquet-filtering', vector)
> 06:33:37 common/impala_test_suite.py:446: in run_test_case
> 06:33:37 verify_runtime_profile(test_section['RUNTIME_PROFILE'],
> result.runtime_profile)
> 06:33:37 common/test_result_verifier.py:569: in verify_runtime_profile
> 06:33:37 "\n\nPROFILE:\n%s\n" % (function, field, expected_value,
> actual_value, actual))
> 06:33:37 E AssertionError: Aggregation of SUM over NumRowGroups did not
> match expected results.
> 06:33:37 E EXPECTED VALUE:
> 06:33:37 E 24
> 06:33:37 E
> 06:33:37 E ACTUAL VALUE:
> 06:33:37 E 22
> 06:33:37 E
> 06:33:37 E PROFILE:
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8379) TestMtDopParquet.test_parquet_filtering flaky - exhaustive run
Gabor Kaszab created IMPALA-8379:
Summary: TestMtDopParquet.test_parquet_filtering flaky -
exhaustive run
Key: IMPALA-8379
URL: https://issues.apache.org/jira/browse/IMPALA-8379
Project: IMPALA
Issue Type: Bug
Reporter: Gabor Kaszab
Assignee: Tim Armstrong
Jenkins job output
{code:java}
06:33:37 query_test/test_mt_dop.py:103: in test_parquet_filtering
06:33:37 self.run_test_case('QueryTest/parquet-filtering', vector)
06:33:37 common/impala_test_suite.py:446: in run_test_case
06:33:37 verify_runtime_profile(test_section['RUNTIME_PROFILE'],
result.runtime_profile)
06:33:37 common/test_result_verifier.py:569: in verify_runtime_profile
06:33:37 "\n\nPROFILE:\n%s\n" % (function, field, expected_value,
actual_value, actual))
06:33:37 E AssertionError: Aggregation of SUM over NumRowGroups did not match
expected results.
06:33:37 E EXPECTED VALUE:
06:33:37 E 24
06:33:37 E
06:33:37 E ACTUAL VALUE:
06:33:37 E 22
06:33:37 E
06:33:37 E PROFILE:
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8379) TestMtDopParquet.test_parquet_filtering flaky - exhaustive run
Gabor Kaszab created IMPALA-8379:
Summary: TestMtDopParquet.test_parquet_filtering flaky -
exhaustive run
Key: IMPALA-8379
URL: https://issues.apache.org/jira/browse/IMPALA-8379
Project: IMPALA
Issue Type: Bug
Reporter: Gabor Kaszab
Assignee: Tim Armstrong
Jenkins job output
{code:java}
06:33:37 query_test/test_mt_dop.py:103: in test_parquet_filtering
06:33:37 self.run_test_case('QueryTest/parquet-filtering', vector)
06:33:37 common/impala_test_suite.py:446: in run_test_case
06:33:37 verify_runtime_profile(test_section['RUNTIME_PROFILE'],
result.runtime_profile)
06:33:37 common/test_result_verifier.py:569: in verify_runtime_profile
06:33:37 "\n\nPROFILE:\n%s\n" % (function, field, expected_value,
actual_value, actual))
06:33:37 E AssertionError: Aggregation of SUM over NumRowGroups did not match
expected results.
06:33:37 E EXPECTED VALUE:
06:33:37 E 24
06:33:37 E
06:33:37 E ACTUAL VALUE:
06:33:37 E 22
06:33:37 E
06:33:37 E PROFILE:
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (IMPALA-8378) test_cancel_select failed with Invalid query handle
[
https://issues.apache.org/jira/browse/IMPALA-8378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Kaszab updated IMPALA-8378:
-
Labels: broken-build (was: )
> test_cancel_select failed with Invalid query handle
> ---
>
> Key: IMPALA-8378
> URL: https://issues.apache.org/jira/browse/IMPALA-8378
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Gabor Kaszab
>Assignee: Andrew Sherman
>Priority: Critical
> Labels: broken-build
>
> Error Message
> {code:java}
> query_test/test_cancellation.py:241: in test_cancel_select
> self.execute_cancel_test(vector) query_test/test_cancellation.py:213: in
> execute_cancel_test assert 'Cancelled' in str(thread.fetch_results_error)
> E assert 'Cancelled' in "ImpalaBeeswaxException:\n INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Invalid query handle:
> b64a91016fe343dd:fefaf600\n" E+ where "ImpalaBeeswaxException:\n
> INNER EXCEPTION: \n MESSAGE:
> Invalid query handle: b64a91016fe343dd:fefaf600\n" =
> str(ImpalaBeeswaxException()) E+where ImpalaBeeswaxException() =
> .fetch_results_error
> {code}
> Stacktrace
> {code:java}
> query_test/test_cancellation.py:241: in test_cancel_select
> self.execute_cancel_test(vector)
> query_test/test_cancellation.py:213: in execute_cancel_test
> assert 'Cancelled' in str(thread.fetch_results_error)
> E assert 'Cancelled' in "ImpalaBeeswaxException:\n INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Invalid query handle:
> b64a91016fe343dd:fefaf600\n"
> E+ where "ImpalaBeeswaxException:\n INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Invalid query handle:
> b64a91016fe343dd:fefaf600\n" = str(ImpalaBeeswaxException())
> E+where ImpalaBeeswaxException() = 139721936148224)>.fetch_results_error
> {code}
> Standard error
> {code:java}
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:rc/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_no;
> -- executing against localhost:21000
> use tpch_rc;
> -- 2019-03-30 23:15:53,293 INFO MainThread: Started query
> 5e42c7ec2df1febb:b25eff72
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:rc/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_no;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET cpu_limit_s=10;
> SET debug_action=0:GETNEXT:WAIT|COORD_CANCEL_QUERY_FINSTANCES_RPC:FAIL;
> SET exec_single_node_rows_threshold=0;
> SET buffer_pool_limit=0;
> -- executing async: localhost:21000
> compute stats lineitem;
> -- 2019-03-30 23:15:53,359 INFO MainThread: Started query
> f54a0ecfa32d40b4:07510d7a
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:rc/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_no;
> -- connecting to: localhost:21000
> -- fetching results from: object at 0x5d535d0>
> -- getting state for operation:
>
> -- canceling operation: object at 0x5d535d0>
> -- 2019-03-30 23:15:53,427 INFO Thread-5: Starting new HTTP connection
> (1): localhost
> -- closing query for operation handle:
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8378) test_cancel_select failed with Invalid query handle
[
https://issues.apache.org/jira/browse/IMPALA-8378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806699#comment-16806699
]
Gabor Kaszab commented on IMPALA-8378:
--
Hey [~asherman],
I found a previous Jira for test_cancel_select failure saying that you have
some knowledge on this test. Could you please take a look at this one?
> test_cancel_select failed with Invalid query handle
> ---
>
> Key: IMPALA-8378
> URL: https://issues.apache.org/jira/browse/IMPALA-8378
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Gabor Kaszab
>Assignee: Andrew Sherman
>Priority: Critical
>
> Error Message
> {code:java}
> query_test/test_cancellation.py:241: in test_cancel_select
> self.execute_cancel_test(vector) query_test/test_cancellation.py:213: in
> execute_cancel_test assert 'Cancelled' in str(thread.fetch_results_error)
> E assert 'Cancelled' in "ImpalaBeeswaxException:\n INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Invalid query handle:
> b64a91016fe343dd:fefaf600\n" E+ where "ImpalaBeeswaxException:\n
> INNER EXCEPTION: \n MESSAGE:
> Invalid query handle: b64a91016fe343dd:fefaf600\n" =
> str(ImpalaBeeswaxException()) E+where ImpalaBeeswaxException() =
> .fetch_results_error
> {code}
> Stacktrace
> {code:java}
> query_test/test_cancellation.py:241: in test_cancel_select
> self.execute_cancel_test(vector)
> query_test/test_cancellation.py:213: in execute_cancel_test
> assert 'Cancelled' in str(thread.fetch_results_error)
> E assert 'Cancelled' in "ImpalaBeeswaxException:\n INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Invalid query handle:
> b64a91016fe343dd:fefaf600\n"
> E+ where "ImpalaBeeswaxException:\n INNER EXCEPTION: 'beeswaxd.ttypes.BeeswaxException'>\n MESSAGE: Invalid query handle:
> b64a91016fe343dd:fefaf600\n" = str(ImpalaBeeswaxException())
> E+where ImpalaBeeswaxException() = 139721936148224)>.fetch_results_error
> {code}
> Standard error
> {code:java}
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:rc/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_no;
> -- executing against localhost:21000
> use tpch_rc;
> -- 2019-03-30 23:15:53,293 INFO MainThread: Started query
> 5e42c7ec2df1febb:b25eff72
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:rc/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_no;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET cpu_limit_s=10;
> SET debug_action=0:GETNEXT:WAIT|COORD_CANCEL_QUERY_FINSTANCES_RPC:FAIL;
> SET exec_single_node_rows_threshold=0;
> SET buffer_pool_limit=0;
> -- executing async: localhost:21000
> compute stats lineitem;
> -- 2019-03-30 23:15:53,359 INFO MainThread: Started query
> f54a0ecfa32d40b4:07510d7a
> SET
> client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:rc/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_no;
> -- connecting to: localhost:21000
> -- fetching results from: object at 0x5d535d0>
> -- getting state for operation:
>
> -- canceling operation: object at 0x5d535d0>
> -- 2019-03-30 23:15:53,427 INFO Thread-5: Starting new HTTP connection
> (1): localhost
> -- closing query for operation handle:
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8378) test_cancel_select failed with Invalid query handle
Gabor Kaszab created IMPALA-8378:
Summary: test_cancel_select failed with Invalid query handle
Key: IMPALA-8378
URL: https://issues.apache.org/jira/browse/IMPALA-8378
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 3.3.0
Reporter: Gabor Kaszab
Assignee: Andrew Sherman
Error Message
{code:java}
query_test/test_cancellation.py:241: in test_cancel_select
self.execute_cancel_test(vector) query_test/test_cancellation.py:213: in
execute_cancel_test assert 'Cancelled' in str(thread.fetch_results_error) E
assert 'Cancelled' in "ImpalaBeeswaxException:\n INNER EXCEPTION: \n MESSAGE: Invalid query handle:
b64a91016fe343dd:fefaf600\n" E+ where "ImpalaBeeswaxException:\n
INNER EXCEPTION: \n MESSAGE: Invalid
query handle: b64a91016fe343dd:fefaf600\n" =
str(ImpalaBeeswaxException()) E+where ImpalaBeeswaxException() =
.fetch_results_error
{code}
Stacktrace
{code:java}
query_test/test_cancellation.py:241: in test_cancel_select
self.execute_cancel_test(vector)
query_test/test_cancellation.py:213: in execute_cancel_test
assert 'Cancelled' in str(thread.fetch_results_error)
E assert 'Cancelled' in "ImpalaBeeswaxException:\n INNER EXCEPTION: \n MESSAGE: Invalid query handle:
b64a91016fe343dd:fefaf600\n"
E+ where "ImpalaBeeswaxException:\n INNER EXCEPTION: \n MESSAGE: Invalid query handle:
b64a91016fe343dd:fefaf600\n" = str(ImpalaBeeswaxException())
E+where ImpalaBeeswaxException() = .fetch_results_error
{code}
Standard error
{code:java}
SET
client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:rc/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_no;
-- executing against localhost:21000
use tpch_rc;
-- 2019-03-30 23:15:53,293 INFO MainThread: Started query
5e42c7ec2df1febb:b25eff72
SET
client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:rc/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_no;
SET batch_size=0;
SET num_nodes=0;
SET disable_codegen_rows_threshold=0;
SET disable_codegen=False;
SET abort_on_error=1;
SET cpu_limit_s=10;
SET debug_action=0:GETNEXT:WAIT|COORD_CANCEL_QUERY_FINSTANCES_RPC:FAIL;
SET exec_single_node_rows_threshold=0;
SET buffer_pool_limit=0;
-- executing async: localhost:21000
compute stats lineitem;
-- 2019-03-30 23:15:53,359 INFO MainThread: Started query
f54a0ecfa32d40b4:07510d7a
SET
client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:rc/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_no;
-- connecting to: localhost:21000
-- fetching results from:
-- getting state for operation:
-- canceling operation:
-- 2019-03-30 23:15:53,427 INFO Thread-5: Starting new HTTP connection (1):
localhost
-- closing query for operation handle:
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8378) test_cancel_select failed with Invalid query handle
Gabor Kaszab created IMPALA-8378:
Summary: test_cancel_select failed with Invalid query handle
Key: IMPALA-8378
URL: https://issues.apache.org/jira/browse/IMPALA-8378
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 3.3.0
Reporter: Gabor Kaszab
Assignee: Andrew Sherman
Error Message
{code:java}
query_test/test_cancellation.py:241: in test_cancel_select
self.execute_cancel_test(vector) query_test/test_cancellation.py:213: in
execute_cancel_test assert 'Cancelled' in str(thread.fetch_results_error) E
assert 'Cancelled' in "ImpalaBeeswaxException:\n INNER EXCEPTION: \n MESSAGE: Invalid query handle:
b64a91016fe343dd:fefaf600\n" E+ where "ImpalaBeeswaxException:\n
INNER EXCEPTION: \n MESSAGE: Invalid
query handle: b64a91016fe343dd:fefaf600\n" =
str(ImpalaBeeswaxException()) E+where ImpalaBeeswaxException() =
.fetch_results_error
{code}
Stacktrace
{code:java}
query_test/test_cancellation.py:241: in test_cancel_select
self.execute_cancel_test(vector)
query_test/test_cancellation.py:213: in execute_cancel_test
assert 'Cancelled' in str(thread.fetch_results_error)
E assert 'Cancelled' in "ImpalaBeeswaxException:\n INNER EXCEPTION: \n MESSAGE: Invalid query handle:
b64a91016fe343dd:fefaf600\n"
E+ where "ImpalaBeeswaxException:\n INNER EXCEPTION: \n MESSAGE: Invalid query handle:
b64a91016fe343dd:fefaf600\n" = str(ImpalaBeeswaxException())
E+where ImpalaBeeswaxException() = .fetch_results_error
{code}
Standard error
{code:java}
SET
client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:rc/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_no;
-- executing against localhost:21000
use tpch_rc;
-- 2019-03-30 23:15:53,293 INFO MainThread: Started query
5e42c7ec2df1febb:b25eff72
SET
client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:rc/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_no;
SET batch_size=0;
SET num_nodes=0;
SET disable_codegen_rows_threshold=0;
SET disable_codegen=False;
SET abort_on_error=1;
SET cpu_limit_s=10;
SET debug_action=0:GETNEXT:WAIT|COORD_CANCEL_QUERY_FINSTANCES_RPC:FAIL;
SET exec_single_node_rows_threshold=0;
SET buffer_pool_limit=0;
-- executing async: localhost:21000
compute stats lineitem;
-- 2019-03-30 23:15:53,359 INFO MainThread: Started query
f54a0ecfa32d40b4:07510d7a
SET
client_identifier=query_test/test_cancellation.py::TestCancellationParallel::()::test_cancel_select[protocol:beeswax|table_format:rc/none|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_no;
-- connecting to: localhost:21000
-- fetching results from:
-- getting state for operation:
-- canceling operation:
-- 2019-03-30 23:15:53,427 INFO Thread-5: Starting new HTTP connection (1):
localhost
-- closing query for operation handle:
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IMPALA-7982) Add network I/O throughput to query profiles
[ https://issues.apache.org/jira/browse/IMPALA-7982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806687#comment-16806687 ] ASF subversion and git services commented on IMPALA-7982: - Commit 1fcea9e4b677f07dfaf6b0314ac763241a029e66 in impala's branch refs/heads/master from Lars Volker [ https://gitbox.apache.org/repos/asf?p=impala.git;h=1fcea9e ] IMPALA-7982: Add host network usage to profile This change adds host network usage to profiles. For each host that participates in the query execution it adds the incoming and outgoing bandwidth across all interfaces excluding the local loopback interface. This includes all data transmitted by the host as part of the execution of a query, other queries, and other processes running on the same system. The change adds tests for the added functionality to the backend and end to end tests. Change-Id: I2cc74f87374080a74a13b7fb6e4da44a11d828ef Reviewed-on: http://gerrit.cloudera.org:8080/12747 Reviewed-by: Lars Volker Tested-by: Impala Public Jenkins > Add network I/O throughput to query profiles > > > Key: IMPALA-7982 > URL: https://issues.apache.org/jira/browse/IMPALA-7982 > Project: IMPALA > Issue Type: New Feature > Components: Backend >Affects Versions: Impala 3.2.0 >Reporter: Lars Volker >Assignee: Lars Volker >Priority: Major > Labels: observability, supportability > > IMPALA-7694 added a framework to collect system resource usage during query > execution and aggregate it at the coordinator. We should add network I/O > throughput there, too. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7800) Reject or timeout new incoming connections once --fe_service_threads limit is reached
[
https://issues.apache.org/jira/browse/IMPALA-7800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806688#comment-16806688
]
ASF subversion and git services commented on IMPALA-7800:
-
Commit eb483dddaf8fa8b9143deac23a583a6cb1a6aa27 in impala's branch
refs/heads/master from Zoram Thanga
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=eb483dd ]
IMPALA-7800: Time out new connections after --fe_service_threads
The current implementation of the FE thrift server waits
indefinitely to open the new session, if the maximum number of
FE service threads specified by --fe_service_threads has been
allocated.
This patch introduces a startup flag to control how the server
should treat new connection requests if we have run out of the
configured number of server threads.
If --accepted_client_cnxn_timeout > 0, new connection requests are
rejected by the server if we can't get a server thread within
the specified timeout.
We set the default timeout to be 5 minutes. The old behavior
can be restored by setting --accepted_client_cnxn_timeout=0,
i.e., no timeout. The timeout applies only to client facing thrift
servers, i.e., HS2 and Beeswax servers.
Testing:
Added a new custom cluster test suite to exercise the
new code.
Ran core and exhaustive tests.
Change-Id: Idb345c1d84cc2f691f54ded467f253e758f87e64
Reviewed-on: http://gerrit.cloudera.org:8080/12579
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Reject or timeout new incoming connections once --fe_service_threads limit is
> reached
> -
>
> Key: IMPALA-7800
> URL: https://issues.apache.org/jira/browse/IMPALA-7800
> Project: IMPALA
> Issue Type: Improvement
> Components: Clients
>Affects Versions: Impala 3.0, Impala 2.12.0
>Reporter: Michael Ho
>Assignee: Zoram Thanga
>Priority: Major
>
> Currently, the number of frontend service threads is controlled by
> {{--fe_service_threads}}. Once this limit is reached, the worker threads of
> {{connection_setup_pool}} will block until some of the service threads exit.
> New incoming connections will be buffered in a queue in
> {{connection_setup_pool}} until it fills up. Currently, there is no time out
> for all these buffered connections in the queue. Consequently, if the number
> of frontend service threads max'ed out for an extended period of time, a
> client trying to connect to Impala will feel "stuck" until a front end
> service thread is available.
> We should consider either rejecting any new connections once
> {{--fe_service_threads}} limit is reached or imposing a tunable upper bound
> on the wait time for connections in the queue to improve the users'
> experience. cc'ing [~zoram]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8312) Alter database operations have race condition
[
https://issues.apache.org/jira/browse/IMPALA-8312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806686#comment-16806686
]
ASF subversion and git services commented on IMPALA-8312:
-
Commit b2a87797a8be076a8e57a91e8db2692ca693e2f3 in impala's branch
refs/heads/master from Vihang Karajgaonkar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b2a8779 ]
IMPALA-8312 : Alter database operations have race condition
This patch fixes a race condition in the alter database implementation
in the catalogOpExecutor. The original implementation did a in-place
modification of the metastore database object in the Db. This can lead
to partially updated database object becoming visible to a reading
thread causing potential problems. In order to fix the race, the
patch makes a copy of the existing database object, modifies the copy
and then atomically switches the actual database object with the
modified copy. This is done while holding the metastoreddlLock, and
then taking the write lock on the catalog version object which makes
it consistent with the other catalog write operations.
Added a test which consistently reproduces the race. The test creating
many reader threads and a writer thread which continuously keeps
changing the owner name and its type by issuing a alter database
operation. The test fails without the patch. After the patch the test
passes. The race also applies to the alter database set comment
operation, although its hard to write a test for that code-path.
Change-Id: I32c8c96a6029bf9d9db37ea8315f6c9603b5a2fc
Reviewed-on: http://gerrit.cloudera.org:8080/12789
Reviewed-by: Fredy Wijaya
Tested-by: Impala Public Jenkins
> Alter database operations have race condition
> -
>
> Key: IMPALA-8312
> URL: https://issues.apache.org/jira/browse/IMPALA-8312
> Project: IMPALA
> Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
>
> There are currently two {{alter database}} operations seen in
> {{CatalogOpExecutor}} although I only see one of them documented which is a
> separate issue.
> Consider the following snippet for {{alter database set owner}} operation.
> {code}
> private void alterDatabaseSetOwner(String dbName, TAlterDbSetOwnerParams
> params,
> TDdlExecResponse response) throws ImpalaException {
> Db db = catalog_.getDb(dbName);
> if (db == null) {
> throw new CatalogException("Database: " + dbName + " does not exist.");
> }
> Preconditions.checkNotNull(params.owner_name);
> Preconditions.checkNotNull(params.owner_type);
> synchronized (metastoreDdlLock_) {
> Database msDb = db.getMetaStoreDb();
> String originalOwnerName = msDb.getOwnerName();
> PrincipalType originalOwnerType = msDb.getOwnerType();
> msDb.setOwnerName(params.owner_name);
> msDb.setOwnerType(PrincipalType.valueOf(params.owner_type.name()));
> try {
> applyAlterDatabase(db);
> } catch (ImpalaRuntimeException e) {
> msDb.setOwnerName(originalOwnerName);
> msDb.setOwnerType(originalOwnerType);
> throw e;
> }
> updateOwnerPrivileges(db.getName(), /* tableName */ null,
> params.server_name,
> originalOwnerName, originalOwnerType,
> db.getMetaStoreDb().getOwnerName(),
> db.getMetaStoreDb().getOwnerType(), response);
> }
> addDbToCatalogUpdate(db, response.result);
> addSummary(response, "Updated database.");
> }
> {code}
> If you notice above, it takes a lock on {{metastoreDdlLock_}} but does not
> take a write lock on catalogVersion before altering the metastore db object
> in-place. This can lead to race conditions between a reader and writer
> thread. For example, it is possible that the thread which is reading {{msDb}}
> object can see new value of owner but a old value of owner type.
> The same problem applies to the alterCommentOnDb method below
> {code}
> private void alterCommentOnDb(String dbName, String comment,
> TDdlExecResponse response)
> throws ImpalaRuntimeException, CatalogException {
> Db db = catalog_.getDb(dbName);
> if (db == null) {
> throw new CatalogException("Database: " + dbName + " does not exist.");
> }
> synchronized (metastoreDdlLock_) {
> Database msDb = db.getMetaStoreDb();
> String originalComment = msDb.getDescription();
> msDb.setDescription(comment);
> try {
> applyAlterDatabase(db);
> } catch (ImpalaRuntimeException e) {
> msDb.setDescription(originalComment);
> throw e;
> }
> }
> addDbToCatalogUpdate(db, response.result);
> addSummary(response, "Updated database.");
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IMPALA-8309) Use a more human-readable flag to switch to a different authorization provider
[
https://issues.apache.org/jira/browse/IMPALA-8309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806544#comment-16806544
]
radford nguyen commented on IMPALA-8309:
code review: [https://gerrit.cloudera.org/c/12901/]
> Use a more human-readable flag to switch to a different authorization provider
> --
>
> Key: IMPALA-8309
> URL: https://issues.apache.org/jira/browse/IMPALA-8309
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
>Reporter: Fredy Wijaya
>Assignee: radford nguyen
>Priority: Minor
>
> We currently use authorization_factory_class flag to switch to a different
> authorization provider, which is useful for any third party to provide an
> implementation of authorization provider. Since, Sentry and Ranger are
> officially supported by Impala, we should have a flag, i.e.
> authorization_provider=[sentry|ranger] to easily switch between officially
> supported authorization providers.
> At the time of this writing, the existing {{authorization_factory_class}}
> flag is being retained but its default value removed. If present, it will
> take precedence over the {{authorization_provider}} flag being added.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
