[jira] [Created] (IMPALA-7329) Blacklist CDH Maven snapshots repository
Fredy Wijaya created IMPALA-7329: Summary: Blacklist CDH Maven snapshots repository Key: IMPALA-7329 URL: https://issues.apache.org/jira/browse/IMPALA-7329 Project: IMPALA Issue Type: Bug Components: Infrastructure Affects Versions: Impala 3.0 Reporter: Fredy Wijaya Assignee: Fredy Wijaya Impala depends on CDH Maven snapshots which transitively pulls the dependencies from other repositories causing the build to be non-reproducible, such as the issue in https://issues.apache.org/jira/browse/IMPALA-7316. In order to fix this issue, we need to blacklist a certain repository so that Impala does not accidentally downloads the latest snapshot for its dependencies. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (IMPALA-7329) Blacklist CDH Maven snapshots repository
[ https://issues.apache.org/jira/browse/IMPALA-7329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-7329 started by Fredy Wijaya. > Blacklist CDH Maven snapshots repository > > > Key: IMPALA-7329 > URL: https://issues.apache.org/jira/browse/IMPALA-7329 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Affects Versions: Impala 3.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > > Impala depends on CDH Maven snapshots which transitively pulls the > dependencies from other repositories causing the build to be > non-reproducible, such as the issue in > https://issues.apache.org/jira/browse/IMPALA-7316. In order to fix this > issue, we need to blacklist a certain repository so that Impala does not > accidentally downloads the latest snapshot for its dependencies. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7329) Blacklist CDH Maven snapshots repository
Fredy Wijaya created IMPALA-7329: Summary: Blacklist CDH Maven snapshots repository Key: IMPALA-7329 URL: https://issues.apache.org/jira/browse/IMPALA-7329 Project: IMPALA Issue Type: Bug Components: Infrastructure Affects Versions: Impala 3.0 Reporter: Fredy Wijaya Assignee: Fredy Wijaya Impala depends on CDH Maven snapshots which transitively pulls the dependencies from other repositories causing the build to be non-reproducible, such as the issue in https://issues.apache.org/jira/browse/IMPALA-7316. In order to fix this issue, we need to blacklist a certain repository so that Impala does not accidentally downloads the latest snapshot for its dependencies. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-6677) Impala Doc: Doc next_day function
[ https://issues.apache.org/jira/browse/IMPALA-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni closed IMPALA-6677. --- Resolution: Fixed Fix Version/s: Impala 3.1.0 > Impala Doc: Doc next_day function > - > > Key: IMPALA-6677 > URL: https://issues.apache.org/jira/browse/IMPALA-6677 > Project: IMPALA > Issue Type: Sub-task > Components: Docs >Affects Versions: Impala 2.11.0 >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > Fix For: Impala 3.1.0 > > > +[http://gerrit.cloudera.org:8080/1943]+ -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-6677) Impala Doc: Doc next_day function
[ https://issues.apache.org/jira/browse/IMPALA-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni closed IMPALA-6677. --- Resolution: Fixed Fix Version/s: Impala 3.1.0 > Impala Doc: Doc next_day function > - > > Key: IMPALA-6677 > URL: https://issues.apache.org/jira/browse/IMPALA-6677 > Project: IMPALA > Issue Type: Sub-task > Components: Docs >Affects Versions: Impala 2.11.0 >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > Fix For: Impala 3.1.0 > > > +[http://gerrit.cloudera.org:8080/1943]+ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7302) Build fails on Centos6
[
https://issues.apache.org/jira/browse/IMPALA-7302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sailesh Mukil resolved IMPALA-7302.
---
Resolution: Fixed
Fix Version/s: Impala 3.1.0
> Build fails on Centos6
> --
>
> Key: IMPALA-7302
> URL: https://issues.apache.org/jira/browse/IMPALA-7302
> Project: IMPALA
> Issue Type: Bug
> Components: Distributed Exec
>Reporter: Michael Ho
>Assignee: Sailesh Mukil
>Priority: Blocker
> Labels: broken-build
> Fix For: Impala 3.1.0
>
>
> Due to recent change in IMPALA-7006, the build started failing on Centos6.
> {noformat}
> 13:33:16
> /data/jenkins/workspace/impala-asf-master-exhaustive-centos6/repos/Impala/be/src/kudu/util/net/socket.cc:
> In member function ‘kudu::Status kudu::Socket::SetReusePort(bool)’:
> 13:33:16
> /data/jenkins/workspace/impala-asf-master-exhaustive-centos6/repos/Impala/be/src/kudu/util/net/socket.cc:249:50:
> error: ‘SO_REUSEPORT’ was not declared in this scope
> 13:33:16RETURN_NOT_OK_PREPEND(SetSockOpt(SOL_SOCKET, SO_REUSEPORT,
> int_flag),
> 13:33:16 ^
> 13:33:16 make[2]: ***
> [be/src/kudu/util/CMakeFiles/kudu_util.dir/net/socket.cc.o] Error 1
> 13:33:16 make[2]: *** Waiting for unfinished jobs
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-7302) Build fails on Centos6
[
https://issues.apache.org/jira/browse/IMPALA-7302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sailesh Mukil resolved IMPALA-7302.
---
Resolution: Fixed
Fix Version/s: Impala 3.1.0
> Build fails on Centos6
> --
>
> Key: IMPALA-7302
> URL: https://issues.apache.org/jira/browse/IMPALA-7302
> Project: IMPALA
> Issue Type: Bug
> Components: Distributed Exec
>Reporter: Michael Ho
>Assignee: Sailesh Mukil
>Priority: Blocker
> Labels: broken-build
> Fix For: Impala 3.1.0
>
>
> Due to recent change in IMPALA-7006, the build started failing on Centos6.
> {noformat}
> 13:33:16
> /data/jenkins/workspace/impala-asf-master-exhaustive-centos6/repos/Impala/be/src/kudu/util/net/socket.cc:
> In member function ‘kudu::Status kudu::Socket::SetReusePort(bool)’:
> 13:33:16
> /data/jenkins/workspace/impala-asf-master-exhaustive-centos6/repos/Impala/be/src/kudu/util/net/socket.cc:249:50:
> error: ‘SO_REUSEPORT’ was not declared in this scope
> 13:33:16RETURN_NOT_OK_PREPEND(SetSockOpt(SOL_SOCKET, SO_REUSEPORT,
> int_flag),
> 13:33:16 ^
> 13:33:16 make[2]: ***
> [be/src/kudu/util/CMakeFiles/kudu_util.dir/net/socket.cc.o] Error 1
> 13:33:16 make[2]: *** Waiting for unfinished jobs
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7302) Build fails on Centos6
[
https://issues.apache.org/jira/browse/IMPALA-7302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16550128#comment-16550128
]
ASF subversion and git services commented on IMPALA-7302:
-
Commit 649397e37dd9b33b4df7b743b8d2e1c67b2c3ad7 in impala's branch
refs/heads/master from [~twmarshall]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=649397e ]
KUDU-2492: Make the use of SO_REUSEPORT conditional on it being defined
A recent commit to Kudu, "rpc: add experimental rpc_reuseport flag",
added the use of the rpc flag SO_REUSEREPORT. This flag is not
available with older versions of Linux, resulting in a compiler error.
This patch avoids the compiler error with a macro that checks if
SO_REUSEPORT is defined, and if it's not attempting to set it will
return an error.
--
IMPALA-7302: This is cherry-picked to fix builds breaking on CentOS 6.4.
Since some of our Jenkins machines are CentOS 6.4, and upgrading them
to our new minimum supported OS of CentOS 6.8 is non-trivial, we cherry-
pick this patch to temporarily unblock these builds until the Jenkins
AMIs are upgraded.
Change-Id: I042125cdaedafa5ebbc09e5a3c12112dfeec59a1
Reviewed-on: http://gerrit.cloudera.org:8080/10994
Reviewed-by: Thomas Marshall
Tested-by: Impala Public Jenkins
> Build fails on Centos6
> --
>
> Key: IMPALA-7302
> URL: https://issues.apache.org/jira/browse/IMPALA-7302
> Project: IMPALA
> Issue Type: Bug
> Components: Distributed Exec
>Reporter: Michael Ho
>Assignee: Sailesh Mukil
>Priority: Blocker
> Labels: broken-build
>
> Due to recent change in IMPALA-7006, the build started failing on Centos6.
> {noformat}
> 13:33:16
> /data/jenkins/workspace/impala-asf-master-exhaustive-centos6/repos/Impala/be/src/kudu/util/net/socket.cc:
> In member function ‘kudu::Status kudu::Socket::SetReusePort(bool)’:
> 13:33:16
> /data/jenkins/workspace/impala-asf-master-exhaustive-centos6/repos/Impala/be/src/kudu/util/net/socket.cc:249:50:
> error: ‘SO_REUSEPORT’ was not declared in this scope
> 13:33:16RETURN_NOT_OK_PREPEND(SetSockOpt(SOL_SOCKET, SO_REUSEPORT,
> int_flag),
> 13:33:16 ^
> 13:33:16 make[2]: ***
> [be/src/kudu/util/CMakeFiles/kudu_util.dir/net/socket.cc.o] Error 1
> 13:33:16 make[2]: *** Waiting for unfinished jobs
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7328) Errors in HdfsScanner::Open() errors get swallowed up
Michael Ho created IMPALA-7328:
--
Summary: Errors in HdfsScanner::Open() errors get swallowed up
Key: IMPALA-7328
URL: https://issues.apache.org/jira/browse/IMPALA-7328
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.1.0
Reporter: Michael Ho
[https://jenkins.impala.io/job/parallel-all-tests/3826/|https://jenkins.impala.io/job/parallel-all-tests/3826/failed]
failed with at test_udfs.py:
{noformat}
03:50:23 ] FAIL
query_test/test_udfs.py::TestUdfExecution::()::test_udf_errors[exec_option:
{'disable_codegen_rows_threshold': 0, 'disable_codegen': True,
'exec_single_node_rows_threshold': 100, 'enable_expr_rewrites': True} |
table_format: text/none]
03:50:23 ] === FAILURES
===
03:50:23 ] TestUdfExecution.test_udf_errors[exec_option:
{'disable_codegen_rows_threshold': 0, 'disable_codegen': True,
'exec_single_node_rows_threshold': 100, 'enable_expr_rewrites': True} |
table_format: text/none]
03:50:23 ] [gw13] linux2 -- Python 2.7.12
/home/ubuntu/Impala/bin/../infra/python/env/bin/python
03:50:23 ] query_test/test_udfs.py:415: in test_udf_errors
03:50:23 ] self.run_test_case('QueryTest/udf-errors', vector,
use_db=unique_database)
03:50:23 ] common/impala_test_suite.py:408: in run_test_case
03:50:23 ] self.__verify_exceptions(test_section['CATCH'], str(e), use_db)
03:50:23 ] common/impala_test_suite.py:286: in __verify_exceptions
03:50:23 ] (expected_str, actual_str)
03:50:23 ] E AssertionError: Unexpected exception string. Expected: BadExpr2
prepare error
03:50:23 ] E Not found in actual: ImpalaBeeswaxException: Query
aborted:Cancelled
03:50:23 ] Captured stderr setup
-
{noformat}
Digging through the log, the query which triggered the failure is
{{774db10632a21589:5a62e772}}
It appears that the error which this test intends to fault at isn't shown at
the coordinator:
{noformat}
ExecState: query id=774db10632a21589:5a62e772
finstance=774db10632a21589:5a62e7720003 on host=ip-172-31-0-127:22001
(EXECUTING -> ERROR) status=Cancelled
{noformat}
In particular, the test aims to trigger a failure in {{HdfsScanner::Open()}}
when scalar expr evaluator is cloned:
{noformat}
// This prepare function always fails for cloned evaluators to exercise
IMPALA-6184.
// It does so by detecting whether the caller is a cloned evaluator and inserts
an error
// in FunctionContext if that's the case.
void BadExpr2Prepare(FunctionContext* context,
FunctionContext::FunctionStateScope scope) {
if (scope == FunctionContext::FRAGMENT_LOCAL) {
int32_t* state =
reinterpret_cast(context->Allocate(sizeof(int32_t)));
*state = 0xf001cafe;
context->SetFunctionState(scope, state);
// Set the thread local state too to differentiate from cloned evaluators.
context->SetFunctionState(FunctionContext::THREAD_LOCAL, state);
} else {
if (context->GetFunctionState(FunctionContext::THREAD_LOCAL) == nullptr) {
context->SetError("BadExpr2 prepare error");
}
}
}
{noformat}
However, for some reasons, the actual failure to be propagated and instead the
cancellation status was propagated instead. Staring at the code in
{{HdfsScanNode}}, it's not immediately clear where the race is.
For the reference, the following is the expected error message:
{noformat}
ExecState: query id=64404101d8857592:173298a7
finstance=64404101d8857592:173298a70002 on host=ip-172-31-0-127:22002
(EXECUTING -> ERROR) status=BadExpr2 prepare error
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7328) Errors in HdfsScanner::Open() errors get swallowed up
Michael Ho created IMPALA-7328:
--
Summary: Errors in HdfsScanner::Open() errors get swallowed up
Key: IMPALA-7328
URL: https://issues.apache.org/jira/browse/IMPALA-7328
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 3.1.0
Reporter: Michael Ho
[https://jenkins.impala.io/job/parallel-all-tests/3826/|https://jenkins.impala.io/job/parallel-all-tests/3826/failed]
failed with at test_udfs.py:
{noformat}
03:50:23 ] FAIL
query_test/test_udfs.py::TestUdfExecution::()::test_udf_errors[exec_option:
{'disable_codegen_rows_threshold': 0, 'disable_codegen': True,
'exec_single_node_rows_threshold': 100, 'enable_expr_rewrites': True} |
table_format: text/none]
03:50:23 ] === FAILURES
===
03:50:23 ] TestUdfExecution.test_udf_errors[exec_option:
{'disable_codegen_rows_threshold': 0, 'disable_codegen': True,
'exec_single_node_rows_threshold': 100, 'enable_expr_rewrites': True} |
table_format: text/none]
03:50:23 ] [gw13] linux2 -- Python 2.7.12
/home/ubuntu/Impala/bin/../infra/python/env/bin/python
03:50:23 ] query_test/test_udfs.py:415: in test_udf_errors
03:50:23 ] self.run_test_case('QueryTest/udf-errors', vector,
use_db=unique_database)
03:50:23 ] common/impala_test_suite.py:408: in run_test_case
03:50:23 ] self.__verify_exceptions(test_section['CATCH'], str(e), use_db)
03:50:23 ] common/impala_test_suite.py:286: in __verify_exceptions
03:50:23 ] (expected_str, actual_str)
03:50:23 ] E AssertionError: Unexpected exception string. Expected: BadExpr2
prepare error
03:50:23 ] E Not found in actual: ImpalaBeeswaxException: Query
aborted:Cancelled
03:50:23 ] Captured stderr setup
-
{noformat}
Digging through the log, the query which triggered the failure is
{{774db10632a21589:5a62e772}}
It appears that the error which this test intends to fault at isn't shown at
the coordinator:
{noformat}
ExecState: query id=774db10632a21589:5a62e772
finstance=774db10632a21589:5a62e7720003 on host=ip-172-31-0-127:22001
(EXECUTING -> ERROR) status=Cancelled
{noformat}
In particular, the test aims to trigger a failure in {{HdfsScanner::Open()}}
when scalar expr evaluator is cloned:
{noformat}
// This prepare function always fails for cloned evaluators to exercise
IMPALA-6184.
// It does so by detecting whether the caller is a cloned evaluator and inserts
an error
// in FunctionContext if that's the case.
void BadExpr2Prepare(FunctionContext* context,
FunctionContext::FunctionStateScope scope) {
if (scope == FunctionContext::FRAGMENT_LOCAL) {
int32_t* state =
reinterpret_cast(context->Allocate(sizeof(int32_t)));
*state = 0xf001cafe;
context->SetFunctionState(scope, state);
// Set the thread local state too to differentiate from cloned evaluators.
context->SetFunctionState(FunctionContext::THREAD_LOCAL, state);
} else {
if (context->GetFunctionState(FunctionContext::THREAD_LOCAL) == nullptr) {
context->SetError("BadExpr2 prepare error");
}
}
}
{noformat}
However, for some reasons, the actual failure to be propagated and instead the
cancellation status was propagated instead. Staring at the code in
{{HdfsScanNode}}, it's not immediately clear where the race is.
For the reference, the following is the expected error message:
{noformat}
ExecState: query id=64404101d8857592:173298a7
finstance=64404101d8857592:173298a70002 on host=ip-172-31-0-127:22002
(EXECUTING -> ERROR) status=BadExpr2 prepare error
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IMPALA-7327) Add UPDATE, DELETE, and UPSERT fine-grained privileges
Fredy Wijaya created IMPALA-7327: Summary: Add UPDATE, DELETE, and UPSERT fine-grained privileges Key: IMPALA-7327 URL: https://issues.apache.org/jira/browse/IMPALA-7327 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 3.0 Reporter: Fredy Wijaya Currently UPDATE, DELETE, and UPSERT commands require ALL on the target table. We need to have more fine-grained privileges, such as UPDATE, DELETE, and UPSERT for those commands. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7327) Add UPDATE, DELETE, and UPSERT fine-grained privileges
Fredy Wijaya created IMPALA-7327: Summary: Add UPDATE, DELETE, and UPSERT fine-grained privileges Key: IMPALA-7327 URL: https://issues.apache.org/jira/browse/IMPALA-7327 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 3.0 Reporter: Fredy Wijaya Currently UPDATE, DELETE, and UPSERT commands require ALL on the target table. We need to have more fine-grained privileges, such as UPDATE, DELETE, and UPSERT for those commands. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-7325) SHOW CREATE VIEW on a view that references built-in functions requires access to the built-in database
[
https://issues.apache.org/jira/browse/IMPALA-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-7325 started by Fredy Wijaya.
> SHOW CREATE VIEW on a view that references built-in functions requires access
> to the built-in database
> --
>
> Key: IMPALA-7325
> URL: https://issues.apache.org/jira/browse/IMPALA-7325
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.0, Impala 2.12.0
>Reporter: Fredy Wijaya
>Assignee: Fredy Wijaya
>Priority: Major
> Labels: security
>
> {noformat}
> [localhost:21000] default> create table foo.t(i int);
> [localhost:21000] default> create view foo.v1 as select count(*) from foo.t;
> [localhost:21000] default> create view foo.v2 as select * from foo.t;
> [localhost:21000] default> grant select on database foo to role foo_role;
> [localhost:21000] default> show create view foo.v1;
> Query: show create view foo.v1
> ERROR: AuthorizationException: User 'impdev' does not have privileges to see
> the definition of view 'foo.v1'.
> [localhost:21000] default> show create view foo.v2;
> Query: show create view foo.v2
> +---+
> | result|
> +---+
> | CREATE VIEW foo.v2 AS |
> | SELECT * FROM foo.t |
> +---+
> Fetched 1 row(s) in 0.01s
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7259) impala-shell is weirdly slow with some large queries
[ https://issues.apache.org/jira/browse/IMPALA-7259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-7259. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 > impala-shell is weirdly slow with some large queries > > > Key: IMPALA-7259 > URL: https://issues.apache.org/jira/browse/IMPALA-7259 > Project: IMPALA > Issue Type: Bug > Components: Clients >Affects Versions: Impala 3.1.0 >Reporter: Tim Armstrong >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.1.0 > > Attachments: wide-parquet-agg.sql > > > impala-shell is very slow at processing some large queries - it takes over a > minute to actually submit the query. I've attached an example. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7259) impala-shell is weirdly slow with some large queries
[ https://issues.apache.org/jira/browse/IMPALA-7259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya resolved IMPALA-7259. -- Resolution: Fixed Fix Version/s: Impala 3.1.0 > impala-shell is weirdly slow with some large queries > > > Key: IMPALA-7259 > URL: https://issues.apache.org/jira/browse/IMPALA-7259 > Project: IMPALA > Issue Type: Bug > Components: Clients >Affects Versions: Impala 3.1.0 >Reporter: Tim Armstrong >Assignee: Fredy Wijaya >Priority: Major > Fix For: Impala 3.1.0 > > Attachments: wide-parquet-agg.sql > > > impala-shell is very slow at processing some large queries - it takes over a > minute to actually submit the query. I've attached an example. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-6874) Add more tests for mixed-format tables, including parquet
[ https://issues.apache.org/jira/browse/IMPALA-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong updated IMPALA-6874: -- Description: We only have a single very basic table with mixed formats that I can see. It would be good to have a larger table that includes parquet, so we can exercise some of the more interesting memory reservation code paths. I think the requirements are: * Files of at least multiple MBs in size * Includes Parquet and at least one row-based format * Ideally a mix of different file and partition sizes We want queries that exercise "interesting" code paths: * Many columns returned: select * from table. * Single column returned * No columns returned - e.g. select count(*) * Selective scan with predicates was: We only have a single very basic table with mixed formats that I can see. It would be good to have a larger table that includes parquet, so we can exercise some of the more interesting memory reservation code paths. I think the requirements are: * Non-tiny files * Includes Parquet and at least one row-based format * Ideally a mix of file sizes > Add more tests for mixed-format tables, including parquet > - > > Key: IMPALA-6874 > URL: https://issues.apache.org/jira/browse/IMPALA-6874 > Project: IMPALA > Issue Type: Test > Components: Infrastructure >Affects Versions: Impala 2.12.0 >Reporter: Tim Armstrong >Priority: Major > > We only have a single very basic table with mixed formats that I can see. It > would be good to have a larger table that includes parquet, so we can > exercise some of the more interesting memory reservation code paths. I think > the requirements are: > * Files of at least multiple MBs in size > * Includes Parquet and at least one row-based format > * Ideally a mix of different file and partition sizes > We want queries that exercise "interesting" code paths: > * Many columns returned: select * from table. > * Single column returned > * No columns returned - e.g. select count(*) > * Selective scan with predicates -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-6839) Sort top table metrics in catalogd debug UI by size
[ https://issues.apache.org/jira/browse/IMPALA-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong resolved IMPALA-6839. --- Resolution: Duplicate > Sort top table metrics in catalogd debug UI by size > --- > > Key: IMPALA-6839 > URL: https://issues.apache.org/jira/browse/IMPALA-6839 > Project: IMPALA > Issue Type: Improvement > Components: Catalog >Affects Versions: Impala 3.0, Impala 2.12.0 >Reporter: Lars Volker >Priority: Major > > IMPALA-4886 added a table to the /catalog debug page of the catalog to > display the tables with the highest memory requirements. By default this > table is sorted by name. I think it would be more useful to sort it by memory > usage. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-6839) Sort top table metrics in catalogd debug UI by size
[ https://issues.apache.org/jira/browse/IMPALA-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong resolved IMPALA-6839. --- Resolution: Duplicate > Sort top table metrics in catalogd debug UI by size > --- > > Key: IMPALA-6839 > URL: https://issues.apache.org/jira/browse/IMPALA-6839 > Project: IMPALA > Issue Type: Improvement > Components: Catalog >Affects Versions: Impala 3.0, Impala 2.12.0 >Reporter: Lars Volker >Priority: Major > > IMPALA-4886 added a table to the /catalog debug page of the catalog to > display the tables with the highest memory requirements. By default this > table is sorted by name. I think it would be more useful to sort it by memory > usage. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IMPALA-2536) Make ColumnType constructor explicit to prevent certain bugs
[ https://issues.apache.org/jira/browse/IMPALA-2536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong updated IMPALA-2536: -- Summary: Make ColumnType constructor explicit to prevent certain bugs (was: Consider making ColumnType ctor explicit) > Make ColumnType constructor explicit to prevent certain bugs > > > Key: IMPALA-2536 > URL: https://issues.apache.org/jira/browse/IMPALA-2536 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 2.3.0 >Reporter: Skye Wanderman-Milne >Priority: Minor > Labels: newbie > > We currently rely on the ColumnType(PrimitiveType) constructor not being > marked explicit (e.g. we use type == TYPE_FOO in many places), but it allows > for bugs if we accidentally pass in e.g. ints instead of types. We should be > using type.type == TYPE_FOO anyway. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7326) test_kudu_partition_ddl failed with exception message: "Table already exists"
Michael Ho created IMPALA-7326:
--
Summary: test_kudu_partition_ddl failed with exception message:
"Table already exists"
Key: IMPALA-7326
URL: https://issues.apache.org/jira/browse/IMPALA-7326
Project: IMPALA
Issue Type: Bug
Components: Catalog
Affects Versions: Impala 3.1.0
Reporter: Michael Ho
cc'ing [~twm378]. Does it look like some known issue ? Putting it in the
catalog category for now but please feel free to update the component as you
see fit.
{noformat}
query_test/test_kudu.py:96: in test_kudu_partition_ddl
self.run_test_case('QueryTest/kudu_partition_ddl', vector,
use_db=unique_database)
common/impala_test_suite.py:397: in run_test_case
result = self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:612: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:160: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:173: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:339: in __execute_query
handle = self.execute_query_async(query_string, user=user)
beeswax/impala_beeswax.py:335: in execute_query_async
return self.__do_rpc(lambda: self.imp_service.query(query,))
beeswax/impala_beeswax.py:460: in __do_rpc
raise ImpalaBeeswaxException(self.__build_error_message(b), b)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
EINNER EXCEPTION:
EMESSAGE: ImpalaRuntimeException: Error creating Kudu table
'impala::test_kudu_partition_ddl_7e04e8f9.simple_hash_range'
E CAUSED BY: NonRecoverableException: Table
impala::test_kudu_partition_ddl_7e04e8f9.simple_hash_range already exists with
id 3e81a4ceff27471cad9fcb3bc0b977c3
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7326) test_kudu_partition_ddl failed with exception message: "Table already exists"
Michael Ho created IMPALA-7326:
--
Summary: test_kudu_partition_ddl failed with exception message:
"Table already exists"
Key: IMPALA-7326
URL: https://issues.apache.org/jira/browse/IMPALA-7326
Project: IMPALA
Issue Type: Bug
Components: Catalog
Affects Versions: Impala 3.1.0
Reporter: Michael Ho
cc'ing [~twm378]. Does it look like some known issue ? Putting it in the
catalog category for now but please feel free to update the component as you
see fit.
{noformat}
query_test/test_kudu.py:96: in test_kudu_partition_ddl
self.run_test_case('QueryTest/kudu_partition_ddl', vector,
use_db=unique_database)
common/impala_test_suite.py:397: in run_test_case
result = self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:612: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:160: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:173: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:339: in __execute_query
handle = self.execute_query_async(query_string, user=user)
beeswax/impala_beeswax.py:335: in execute_query_async
return self.__do_rpc(lambda: self.imp_service.query(query,))
beeswax/impala_beeswax.py:460: in __do_rpc
raise ImpalaBeeswaxException(self.__build_error_message(b), b)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
EINNER EXCEPTION:
EMESSAGE: ImpalaRuntimeException: Error creating Kudu table
'impala::test_kudu_partition_ddl_7e04e8f9.simple_hash_range'
E CAUSED BY: NonRecoverableException: Table
impala::test_kudu_partition_ddl_7e04e8f9.simple_hash_range already exists with
id 3e81a4ceff27471cad9fcb3bc0b977c3
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IMPALA-7291) [DOCS] Document recommendation to use VARCHAR or STRING instead of CHAR(N)
[ https://issues.apache.org/jira/browse/IMPALA-7291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549680#comment-16549680 ] Tim Armstrong commented on IMPALA-7291: --- [~arodoni_cloudera] codegen improves query execution performance. > [DOCS] Document recommendation to use VARCHAR or STRING instead of CHAR(N) > -- > > Key: IMPALA-7291 > URL: https://issues.apache.org/jira/browse/IMPALA-7291 > Project: IMPALA > Issue Type: Improvement > Components: Docs >Affects Versions: Impala 2.12.0 >Reporter: Balazs Jeszenszky >Assignee: Alex Rodoni >Priority: Major > > CHAR(N) currently does not have codegen support. For that reason, we should > recommend customers to use VARCHAR or STRING instead - the gain of codegen > outweighs the benefits of fixed width CHARs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7325) SHOW CREATE VIEW on a view that references built-in functions requires access to the built-in database
[
https://issues.apache.org/jira/browse/IMPALA-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya updated IMPALA-7325:
-
Description:
{noformat}
[localhost:21000] default> create table foo.t(i int);
[localhost:21000] default> create view foo.v1 as select count(*) from foo.t;
[localhost:21000] default> create view foo.v2 as select * from foo.t;
[localhost:21000] default> grant select on database foo to role foo_role;
[localhost:21000] default> show create view foo.v1;
Query: show create view foo.v1
ERROR: AuthorizationException: User 'impdev' does not have privileges to see
the definition of view 'foo.v1'.
[localhost:21000] default> show create view foo.v2;
Query: show create view foo.v2
+---+
| result|
+---+
| CREATE VIEW foo.v2 AS |
| SELECT * FROM foo.t |
+---+
Fetched 1 row(s) in 0.01s
{noformat}
was:
{noformat}
[node009:21000] > create view v1 as select * from foo.v1;
[node009:21000] > create view t as select count(*) from system.cases;
{noformat}
> SHOW CREATE VIEW on a view that references built-in functions requires access
> to the built-in database
> --
>
> Key: IMPALA-7325
> URL: https://issues.apache.org/jira/browse/IMPALA-7325
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.0, Impala 2.12.0
>Reporter: Fredy Wijaya
>Assignee: Fredy Wijaya
>Priority: Major
> Labels: security
>
> {noformat}
> [localhost:21000] default> create table foo.t(i int);
> [localhost:21000] default> create view foo.v1 as select count(*) from foo.t;
> [localhost:21000] default> create view foo.v2 as select * from foo.t;
> [localhost:21000] default> grant select on database foo to role foo_role;
> [localhost:21000] default> show create view foo.v1;
> Query: show create view foo.v1
> ERROR: AuthorizationException: User 'impdev' does not have privileges to see
> the definition of view 'foo.v1'.
> [localhost:21000] default> show create view foo.v2;
> Query: show create view foo.v2
> +---+
> | result|
> +---+
> | CREATE VIEW foo.v2 AS |
> | SELECT * FROM foo.t |
> +---+
> Fetched 1 row(s) in 0.01s
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-7325) SHOW CREATE VIEW on a view that references built-in functions requires access to the built-in database
[
https://issues.apache.org/jira/browse/IMPALA-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fredy Wijaya updated IMPALA-7325:
-
Description:
{noformat}
[node009:21000] > create view v1 as select * from foo.v1;
[node009:21000] > create view t as select count(*) from system.cases;
{noformat}
> SHOW CREATE VIEW on a view that references built-in functions requires access
> to the built-in database
> --
>
> Key: IMPALA-7325
> URL: https://issues.apache.org/jira/browse/IMPALA-7325
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.0, Impala 2.12.0
>Reporter: Fredy Wijaya
>Assignee: Fredy Wijaya
>Priority: Major
> Labels: security
>
> {noformat}
> [node009:21000] > create view v1 as select * from foo.v1;
> [node009:21000] > create view t as select count(*) from system.cases;
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7252) Backport rate limiting of fadvise calls into toolchain glog
[ https://issues.apache.org/jira/browse/IMPALA-7252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tianyi Wang resolved IMPALA-7252. - Resolution: Fixed Fix Version/s: Impala 3.1.0 > Backport rate limiting of fadvise calls into toolchain glog > --- > > Key: IMPALA-7252 > URL: https://issues.apache.org/jira/browse/IMPALA-7252 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 3.0 >Reporter: Todd Lipcon >Assignee: Tianyi Wang >Priority: Major > Fix For: Impala 3.1.0 > > > Currently, glog's default behavior is to call fadvise(FADV_DONTNEED) on the > log file after each entry that is written. In many versions of the Linux > kernel, each invocation of this call causes work to be scheduled on all other > CPUs, causing up to one context switch per CPU for every log line. We saw > this cause an extremely long GC pause in the catalogd in the case where the > native side of the catalog was logging a lot of messages about publishing > metadata updates at the same time that the Java side was running a GC. The GC > spent almost all of its time in the kernel due to the high context switch > rate causing a lot of TLB clears and misses, and instead of pausing the JVM > for a couple of seconds took several minutes. > This was identified and fixed upstream in glog here: > https://github.com/google/glog/commit/dacd29679633c9b845708e7015bd2c79367a6ea2 > We should backport this fix into the version in the toolchain. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7252) Backport rate limiting of fadvise calls into toolchain glog
[ https://issues.apache.org/jira/browse/IMPALA-7252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tianyi Wang resolved IMPALA-7252. - Resolution: Fixed Fix Version/s: Impala 3.1.0 > Backport rate limiting of fadvise calls into toolchain glog > --- > > Key: IMPALA-7252 > URL: https://issues.apache.org/jira/browse/IMPALA-7252 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 3.0 >Reporter: Todd Lipcon >Assignee: Tianyi Wang >Priority: Major > Fix For: Impala 3.1.0 > > > Currently, glog's default behavior is to call fadvise(FADV_DONTNEED) on the > log file after each entry that is written. In many versions of the Linux > kernel, each invocation of this call causes work to be scheduled on all other > CPUs, causing up to one context switch per CPU for every log line. We saw > this cause an extremely long GC pause in the catalogd in the case where the > native side of the catalog was logging a lot of messages about publishing > metadata updates at the same time that the Java side was running a GC. The GC > spent almost all of its time in the kernel due to the high context switch > rate causing a lot of TLB clears and misses, and instead of pausing the JVM > for a couple of seconds took several minutes. > This was identified and fixed upstream in glog here: > https://github.com/google/glog/commit/dacd29679633c9b845708e7015bd2c79367a6ea2 > We should backport this fix into the version in the toolchain. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IMPALA-7325) SHOW CREATE VIEW on a view that references built-in functions requires access to the built-in database
[ https://issues.apache.org/jira/browse/IMPALA-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fredy Wijaya updated IMPALA-7325: - Summary: SHOW CREATE VIEW on a view that references built-in functions requires access to the built-in database (was: SHOW CREATE VIEW on a view that references built-in functions requires access to built-in database) > SHOW CREATE VIEW on a view that references built-in functions requires access > to the built-in database > -- > > Key: IMPALA-7325 > URL: https://issues.apache.org/jira/browse/IMPALA-7325 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 3.0, Impala 2.12.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Major > Labels: security > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7325) SHOW CREATE VIEW on a view that references built-in functions requires access to built-in database
Fredy Wijaya created IMPALA-7325: Summary: SHOW CREATE VIEW on a view that references built-in functions requires access to built-in database Key: IMPALA-7325 URL: https://issues.apache.org/jira/browse/IMPALA-7325 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 2.12.0, Impala 3.0 Reporter: Fredy Wijaya Assignee: Fredy Wijaya -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7325) SHOW CREATE VIEW on a view that references built-in functions requires access to built-in database
Fredy Wijaya created IMPALA-7325: Summary: SHOW CREATE VIEW on a view that references built-in functions requires access to built-in database Key: IMPALA-7325 URL: https://issues.apache.org/jira/browse/IMPALA-7325 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 2.12.0, Impala 3.0 Reporter: Fredy Wijaya Assignee: Fredy Wijaya -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7324) Removed MarkNeedsDeepCopy() from Sorter
Tim Armstrong created IMPALA-7324: - Summary: Removed MarkNeedsDeepCopy() from Sorter Key: IMPALA-7324 URL: https://issues.apache.org/jira/browse/IMPALA-7324 Project: IMPALA Issue Type: Sub-task Components: Backend Reporter: Tim Armstrong Assignee: Tim Armstrong The use of this in the sorter is self-contained because SortedRunMerger copies the output anyway, so we can fix this independent of the larger task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IMPALA-7324) Removed MarkNeedsDeepCopy() from Sorter
Tim Armstrong created IMPALA-7324: - Summary: Removed MarkNeedsDeepCopy() from Sorter Key: IMPALA-7324 URL: https://issues.apache.org/jira/browse/IMPALA-7324 Project: IMPALA Issue Type: Sub-task Components: Backend Reporter: Tim Armstrong Assignee: Tim Armstrong The use of this in the sorter is self-contained because SortedRunMerger copies the output anyway, so we can fix this independent of the larger task. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7323) Allow closing scan nodes before all returned batches are destroyed
Tim Armstrong created IMPALA-7323: - Summary: Allow closing scan nodes before all returned batches are destroyed Key: IMPALA-7323 URL: https://issues.apache.org/jira/browse/IMPALA-7323 Project: IMPALA Issue Type: Improvement Components: Backend Reporter: Tim Armstrong Assignee: Tim Armstrong If the scan node finishes early because of a limit at the scan or higher up in the plan tree, returned batches may still be in use by nodes higher up in the operator tree that reference memory still owned by the Scanner object, or enqueued in the RowBatchQueue. We should add a method that can be called before Close() to attach all of those resources to the trailing batch. E.g. AcquireResourcesFinal(). This method needs to be separate from Close() because Close() releases all resources accounted against the node being closed (e.g. reservations but ownership of the returned resources is only done lazily when required by a blocking node). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7323) Allow closing scan nodes before all returned batches are destroyed
Tim Armstrong created IMPALA-7323: - Summary: Allow closing scan nodes before all returned batches are destroyed Key: IMPALA-7323 URL: https://issues.apache.org/jira/browse/IMPALA-7323 Project: IMPALA Issue Type: Improvement Components: Backend Reporter: Tim Armstrong Assignee: Tim Armstrong If the scan node finishes early because of a limit at the scan or higher up in the plan tree, returned batches may still be in use by nodes higher up in the operator tree that reference memory still owned by the Scanner object, or enqueued in the RowBatchQueue. We should add a method that can be called before Close() to attach all of those resources to the trailing batch. E.g. AcquireResourcesFinal(). This method needs to be separate from Close() because Close() releases all resources accounted against the node being closed (e.g. reservations but ownership of the returned resources is only done lazily when required by a blocking node). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IMPALA-6453) Test Python 2.6 in pre-merge testing
[ https://issues.apache.org/jira/browse/IMPALA-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549582#comment-16549582 ] Philip Zeyliger commented on IMPALA-6453: - I wrote https://jenkins.impala.io/job/python26-incompatibility-check/ . It's "working" except that it found a bug being addressed in https://gerrit.cloudera.org/#/c/10993/ . After it starts being green, I'll shove this into the parallel test runner. I used t2small and it seems to work end to end in under a minute, so I'm not worrying about improving performance. > Test Python 2.6 in pre-merge testing > > > Key: IMPALA-6453 > URL: https://issues.apache.org/jira/browse/IMPALA-6453 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Affects Versions: Impala 3.0, Impala 2.12.0 >Reporter: Jim Apple >Priority: Minor > > IMPALA-6447 got past > [https://jenkins.impala.io/view/Gerrit/job/gerrit-verify-dryrun/] because > that runs Impala 2.7, but the test failed under Python 2.6. > If possible, it might make sense to run tests under both version of Python in > pre-merge testing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7252) Backport rate limiting of fadvise calls into toolchain glog
[ https://issues.apache.org/jira/browse/IMPALA-7252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549375#comment-16549375 ] ASF subversion and git services commented on IMPALA-7252: - Commit 940d536f21160cd93a900aa70c003b6d1cb1fac8 in impala's branch refs/heads/master from [~tianyiwang] [ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=940d536 ] IMPALA-7252: Backport rate limiting of fadvise calls into toolchain glog This patch bumps glog version to 0.3.4-p3 to include the patch limiting fadvise calls. Change-Id: I41fd855fbf0e9ec58845ac0d2eb96a87b0172152 Reviewed-on: http://gerrit.cloudera.org:8080/10965 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Backport rate limiting of fadvise calls into toolchain glog > --- > > Key: IMPALA-7252 > URL: https://issues.apache.org/jira/browse/IMPALA-7252 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Affects Versions: Impala 3.0 >Reporter: Todd Lipcon >Assignee: Tianyi Wang >Priority: Major > > Currently, glog's default behavior is to call fadvise(FADV_DONTNEED) on the > log file after each entry that is written. In many versions of the Linux > kernel, each invocation of this call causes work to be scheduled on all other > CPUs, causing up to one context switch per CPU for every log line. We saw > this cause an extremely long GC pause in the catalogd in the case where the > native side of the catalog was logging a lot of messages about publishing > metadata updates at the same time that the Java side was running a GC. The GC > spent almost all of its time in the kernel due to the high context switch > rate causing a lot of TLB clears and misses, and instead of pausing the JVM > for a couple of seconds took several minutes. > This was identified and fixed upstream in glog here: > https://github.com/google/glog/commit/dacd29679633c9b845708e7015bd2c79367a6ea2 > We should backport this fix into the version in the toolchain. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7315) tests_statestore.py fails at assert len(args.topic_deltas[topic_name].topic_entries) == 1
[
https://issues.apache.org/jira/browse/IMPALA-7315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549377#comment-16549377
]
ASF subversion and git services commented on IMPALA-7315:
-
Commit 70e2d57fc06499d3eb7c278c28daf51f782d93b3 in impala's branch
refs/heads/master from [~tarmstr...@cloudera.com]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=70e2d57 ]
IMPALA-7315: fix test_update_with_clear_entries_flag race
We need to wait for the subscriber to process the second update in order
to guarantee that the first update for that subscriber has been applied.
Otherwise there is a race window where the second subscriber may see the
older version of the statestore topic.
Change-Id: I2be2b61b6deb0228fbc5a242e43076beb8871454
Reviewed-on: http://gerrit.cloudera.org:8080/10986
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> tests_statestore.py fails at assert
> len(args.topic_deltas[topic_name].topic_entries) == 1
> --
>
> Key: IMPALA-7315
> URL: https://issues.apache.org/jira/browse/IMPALA-7315
> Project: IMPALA
> Issue Type: Bug
> Components: Distributed Exec
>Affects Versions: Impala 3.1.0
>Reporter: Michael Ho
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: broken-build
> Fix For: Impala 3.1.0
>
>
> Assigning to [~tarmstr...@cloudera.com] given he recently did some Statestore
> related changes. Please feel free to reassign.
> {noformat}
> statestore/test_statestore.py:568: in test_update_with_clear_entries_flag
> .wait_for_update(topic_name, 2)
> statestore/test_statestore.py:301: in wait_for_update
> self.check_thread_exceptions()
> statestore/test_statestore.py:239: in check_thread_exceptions
> if self.exception is not None: raise self.exception
> E assert 2 == 1
> E+ where 2 = len([TTopicItem(deleted=False, value='bar0', key='old0'),
> TTopicItem(deleted=False, value='bar1', key='old1')])
> E+where [TTopicItem(deleted=False, value='bar0', key='old0'),
> TTopicItem(deleted=False, value='bar1', key='old1')] =
> TTopicDelta(min_subscriber_topic_version=None, clear_topic_entries=None,
> from_version=0,
> topic_name='test_topic_255472...pic_entries=[TTopicItem(deleted=False,
> value='bar0', key='old0'), TTopicItem(deleted=False, value='bar1',
> key='old1')]).topic_entries
> Traceback (most recent call last):
> File
> "/data0/jenkins/workspace/impala-asf-master-core-local/repos/Impala/tests/statestore/test_statestore.py",
> line 207, in UpdateState
> response = self.update_cb(self, args)
> File
> "/data0/jenkins/workspace/impala-asf-master-core-local/repos/Impala/tests/statestore/test_statestore.py",
> line 546, in check_entries
> assert len(args.topic_deltas[topic_name].topic_entries) == 1
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7298) Don't pass resolved IP address as hostname when creating proxy
[
https://issues.apache.org/jira/browse/IMPALA-7298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549376#comment-16549376
]
ASF subversion and git services commented on IMPALA-7298:
-
Commit c7d2c2ec73a5e2073de22e10742faa0553c5018d in impala's branch
refs/heads/master from Michael Ho
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=c7d2c2e ]
IMPALA-7298: Stop passing IP address as hostname in Kerberos principal
Previously, we pass the resolved IP address of a KRPC destination
host as the hostname when creating a proxy for making KRPC calls.
This may lead to connection negotiation failure in KRPC when Kerberos
is enabled. In particular, if reversed DNS isn't enabled in Kerberos,
KDC may fail to look up the principal of the destination host if the
principal includes the hostname instead of resolved IP address.
This change fixes the problem above by passing the actual hostname
of the destination host when calling RpcMgr::GetProxy().
rpc-mgr-kerberized-test.cc is also updated to use hostname
instead of the resolved IP address as Kerberos principal.
Change-Id: I3e3e978746cf03766eee151835aad5877d9ed63e
Reviewed-on: http://gerrit.cloudera.org:8080/10980
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Don't pass resolved IP address as hostname when creating proxy
> --
>
> Key: IMPALA-7298
> URL: https://issues.apache.org/jira/browse/IMPALA-7298
> Project: IMPALA
> Issue Type: Bug
> Components: Distributed Exec
>Affects Versions: Impala 2.12.0, Impala 3.1.0
>Reporter: Michael Ho
>Assignee: Michael Ho
>Priority: Critical
>
> {{KrpcDataStreamSender}} passes a resolved IP address when creating a proxy.
> Instead, we should pass both the resolved address and the hostname when
> creating the proxy so that we won't end up using the IP address as the
> hostname in the Kerberos principal.
> Due to the oversight above, the following error may show up when running a
> build of 2.12.0 when a user has Kerberos enabled and specified
> {{impala/@}} as the kerberos principal.
> {noformat}
> WARNINGS: TransmitData() to X.X.X.X:27000 failed: Not authorized: Client
> connection negotiation failed: client connection to X.X.X.X:27000: Server
> impala/x.x@vpc.cloudera.com not found in Kerberos database
> {noformat}
> The workaround for this problem is to have {{rdns=true}} in
> {{/etc/krb5.conf}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7314) Impala doc generation should fail when there is an error
[ https://issues.apache.org/jira/browse/IMPALA-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549374#comment-16549374 ] ASF subversion and git services commented on IMPALA-7314: - Commit 2008702759d16460803b6e190173f887cc6ef832 in impala's branch refs/heads/master from [~fredyw] [ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=2008702 ] IMPALA-7314: Doc generation should fail on error This patch updates the doc generation to fail when there is an error. dita does not exit with non-zero exit code when there is an error. The patch checks for [ERROR] in the dita output and fails if it encounters one. Testing: - Manually tested by injecting failures Change-Id: Ic452aa282a3f2a761e3b04a7460e0d86bc51d721 Reviewed-on: http://gerrit.cloudera.org:8080/10976 Reviewed-by: Alex Rodoni Reviewed-by: Michael Brown Tested-by: Impala Public Jenkins > Impala doc generation should fail when there is an error > > > Key: IMPALA-7314 > URL: https://issues.apache.org/jira/browse/IMPALA-7314 > Project: IMPALA > Issue Type: Improvement > Components: Docs, Infrastructure >Affects Versions: Impala 3.0 >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Minor > Fix For: Impala 3.1.0 > > > When there are broken links in the doc, dita doesn't exit with a non-zero > exit status. As a result, there are currently many errors in dita that are > ignored. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7315) tests_statestore.py fails at assert len(args.topic_deltas[topic_name].topic_entries) == 1
[
https://issues.apache.org/jira/browse/IMPALA-7315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-7315.
---
Resolution: Fixed
Fix Version/s: Impala 3.1.0
> tests_statestore.py fails at assert
> len(args.topic_deltas[topic_name].topic_entries) == 1
> --
>
> Key: IMPALA-7315
> URL: https://issues.apache.org/jira/browse/IMPALA-7315
> Project: IMPALA
> Issue Type: Bug
> Components: Distributed Exec
>Affects Versions: Impala 3.1.0
>Reporter: Michael Ho
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: broken-build
> Fix For: Impala 3.1.0
>
>
> Assigning to [~tarmstr...@cloudera.com] given he recently did some Statestore
> related changes. Please feel free to reassign.
> {noformat}
> statestore/test_statestore.py:568: in test_update_with_clear_entries_flag
> .wait_for_update(topic_name, 2)
> statestore/test_statestore.py:301: in wait_for_update
> self.check_thread_exceptions()
> statestore/test_statestore.py:239: in check_thread_exceptions
> if self.exception is not None: raise self.exception
> E assert 2 == 1
> E+ where 2 = len([TTopicItem(deleted=False, value='bar0', key='old0'),
> TTopicItem(deleted=False, value='bar1', key='old1')])
> E+where [TTopicItem(deleted=False, value='bar0', key='old0'),
> TTopicItem(deleted=False, value='bar1', key='old1')] =
> TTopicDelta(min_subscriber_topic_version=None, clear_topic_entries=None,
> from_version=0,
> topic_name='test_topic_255472...pic_entries=[TTopicItem(deleted=False,
> value='bar0', key='old0'), TTopicItem(deleted=False, value='bar1',
> key='old1')]).topic_entries
> Traceback (most recent call last):
> File
> "/data0/jenkins/workspace/impala-asf-master-core-local/repos/Impala/tests/statestore/test_statestore.py",
> line 207, in UpdateState
> response = self.update_cb(self, args)
> File
> "/data0/jenkins/workspace/impala-asf-master-core-local/repos/Impala/tests/statestore/test_statestore.py",
> line 546, in check_entries
> assert len(args.topic_deltas[topic_name].topic_entries) == 1
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (IMPALA-7315) tests_statestore.py fails at assert len(args.topic_deltas[topic_name].topic_entries) == 1
[
https://issues.apache.org/jira/browse/IMPALA-7315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-7315.
---
Resolution: Fixed
Fix Version/s: Impala 3.1.0
> tests_statestore.py fails at assert
> len(args.topic_deltas[topic_name].topic_entries) == 1
> --
>
> Key: IMPALA-7315
> URL: https://issues.apache.org/jira/browse/IMPALA-7315
> Project: IMPALA
> Issue Type: Bug
> Components: Distributed Exec
>Affects Versions: Impala 3.1.0
>Reporter: Michael Ho
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: broken-build
> Fix For: Impala 3.1.0
>
>
> Assigning to [~tarmstr...@cloudera.com] given he recently did some Statestore
> related changes. Please feel free to reassign.
> {noformat}
> statestore/test_statestore.py:568: in test_update_with_clear_entries_flag
> .wait_for_update(topic_name, 2)
> statestore/test_statestore.py:301: in wait_for_update
> self.check_thread_exceptions()
> statestore/test_statestore.py:239: in check_thread_exceptions
> if self.exception is not None: raise self.exception
> E assert 2 == 1
> E+ where 2 = len([TTopicItem(deleted=False, value='bar0', key='old0'),
> TTopicItem(deleted=False, value='bar1', key='old1')])
> E+where [TTopicItem(deleted=False, value='bar0', key='old0'),
> TTopicItem(deleted=False, value='bar1', key='old1')] =
> TTopicDelta(min_subscriber_topic_version=None, clear_topic_entries=None,
> from_version=0,
> topic_name='test_topic_255472...pic_entries=[TTopicItem(deleted=False,
> value='bar0', key='old0'), TTopicItem(deleted=False, value='bar1',
> key='old1')]).topic_entries
> Traceback (most recent call last):
> File
> "/data0/jenkins/workspace/impala-asf-master-core-local/repos/Impala/tests/statestore/test_statestore.py",
> line 207, in UpdateState
> response = self.update_cb(self, args)
> File
> "/data0/jenkins/workspace/impala-asf-master-core-local/repos/Impala/tests/statestore/test_statestore.py",
> line 546, in check_entries
> assert len(args.topic_deltas[topic_name].topic_entries) == 1
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7322) Add storage wait time to REFRESH and INVALIDATE profiles
Balazs Jeszenszky created IMPALA-7322: - Summary: Add storage wait time to REFRESH and INVALIDATE profiles Key: IMPALA-7322 URL: https://issues.apache.org/jira/browse/IMPALA-7322 Project: IMPALA Issue Type: Improvement Components: Frontend Affects Versions: Impala 2.12.0, Impala 3.0 Reporter: Balazs Jeszenszky The profiles of a REFRESH or INVALIDATE should point out how much time was spent waiting for source systems. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-7322) Add storage wait time to REFRESH and INVALIDATE profiles
Balazs Jeszenszky created IMPALA-7322: - Summary: Add storage wait time to REFRESH and INVALIDATE profiles Key: IMPALA-7322 URL: https://issues.apache.org/jira/browse/IMPALA-7322 Project: IMPALA Issue Type: Improvement Components: Frontend Affects Versions: Impala 2.12.0, Impala 3.0 Reporter: Balazs Jeszenszky The profiles of a REFRESH or INVALIDATE should point out how much time was spent waiting for source systems. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-7304) Impala shouldn't write column indexes for float columns until PARQUET-1222 is resolved
[ https://issues.apache.org/jira/browse/IMPALA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltán Borók-Nagy resolved IMPALA-7304. --- Resolution: Fixed Fix Version/s: Impala 3.1.0 Impala 2.13.0 > Impala shouldn't write column indexes for float columns until PARQUET-1222 is > resolved > -- > > Key: IMPALA-7304 > URL: https://issues.apache.org/jira/browse/IMPALA-7304 > Project: IMPALA > Issue Type: Bug >Affects Versions: Impala 2.13.0, Impala 3.1.0 >Reporter: Zoltán Borók-Nagy >Assignee: Zoltán Borók-Nagy >Priority: Blocker > Fix For: Impala 2.13.0, Impala 3.1.0 > > > Impala master branch can already write the [Parquet page > index|https://github.com/apache/parquet-format/blob/master/PageIndex.md]. > > However, we still don't have well-defined ordering for floating-point > numbers in Parquet: > [PARQUET-1222|https://issues-test.apache.org/jira/browse/PARQUET-1222] > > Currently impala writes the page index with fmax()/fmin() semantics, but it > might contradicts the future specification that will be defined once > PARQUET-1222 is resolved. > > We should not write column indexes for floating-point columns until > PARQUET-1222 is resolved. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-7304) Impala shouldn't write column indexes for float columns until PARQUET-1222 is resolved
[ https://issues.apache.org/jira/browse/IMPALA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltán Borók-Nagy resolved IMPALA-7304. --- Resolution: Fixed Fix Version/s: Impala 3.1.0 Impala 2.13.0 > Impala shouldn't write column indexes for float columns until PARQUET-1222 is > resolved > -- > > Key: IMPALA-7304 > URL: https://issues.apache.org/jira/browse/IMPALA-7304 > Project: IMPALA > Issue Type: Bug >Affects Versions: Impala 2.13.0, Impala 3.1.0 >Reporter: Zoltán Borók-Nagy >Assignee: Zoltán Borók-Nagy >Priority: Blocker > Fix For: Impala 2.13.0, Impala 3.1.0 > > > Impala master branch can already write the [Parquet page > index|https://github.com/apache/parquet-format/blob/master/PageIndex.md]. > > However, we still don't have well-defined ordering for floating-point > numbers in Parquet: > [PARQUET-1222|https://issues-test.apache.org/jira/browse/PARQUET-1222] > > Currently impala writes the page index with fmax()/fmin() semantics, but it > might contradicts the future specification that will be defined once > PARQUET-1222 is resolved. > > We should not write column indexes for floating-point columns until > PARQUET-1222 is resolved. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
