[jira] [Created] (IMPALA-8301) Eliminate need for SYNC_DDL in local catalog mode
Boris Gitline created IMPALA-8301: - Summary: Eliminate need for SYNC_DDL in local catalog mode Key: IMPALA-8301 URL: https://issues.apache.org/jira/browse/IMPALA-8301 Project: IMPALA Issue Type: Improvement Reporter: Boris Gitline In the following scenario looks like the INSERT on coordinator 2 is gated behind a long-running DDL on coordinator 1. That scenario still requires SYNC_DDL even in metadata v2. We want to change the metadata handling design so that coordinator 3 does not have to wait for the long-running DDL to complete – it would render the correct result on the target table t1 reference. Step1. coordinator 1 ##*say*, the following compute stats runs about 100 seconds compute stats tao_ddl_contention; [steps 2 and 3 are performed while COMPUTE STATS is running] Step2. coordinator 2 create another new table. create table t1(c1 int); insert into t1 select 1 ; select * from t1; [can see the inserted rows] Step3. coordinator 3 query the newly inserted rows in t1 while the COMPUTE STATS is still running: select * from t1; [see no rows] ##the query in [Step3] won't show the row inserted by step2 until the first step "compute stats" completed, unless: - SYNC_DDL is set before the INSERT on coordinator 2, or * when the step1 compute stats is completed, or * you can see the data from this impala session, or * a manual refresh of the t1 table. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6658) Parquet RLE encoding can waste space with small repeated runs
[ https://issues.apache.org/jira/browse/IMPALA-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790092#comment-16790092 ] ASF subversion and git services commented on IMPALA-6658: - Commit 729fcf34764928bebfe7cf31153b57f7246404af in impala's branch refs/heads/master from Andrew Sherman [ https://gitbox.apache.org/repos/asf?p=impala.git;h=729fcf3 ] IMPALA-8279: Revert IMPALA-6658 to avoid ETL performance regression. IMPALA-6658 changed RleEncoder to have the ability to use run lengths other than 8. It seemed that a slightly more complex RleEncoder could save a small amount of disk space by using the longer run lengths, in particular for bit width of 1. We now see a performance regression on a simple ETL query. Overall it seems that the costs of IMPALA-6658 exceed the benefits. This change removes IMPALA-6658. The strategy for this was that the change to rle-encoding.h, which contains the code, was undone using 'git revert'. I removed the test changes in rle-test.cc that rely on different encoding lengths. This allows us to keep some useful new tests that were written as part of IMPALA-6658 TESTING: Ran all end-to-end tests. Change-Id: If6bcbaf564fbbe6dc83ba3afc100b4e5ccc7af40 Reviewed-on: http://gerrit.cloudera.org:8080/12680 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Parquet RLE encoding can waste space with small repeated runs > - > > Key: IMPALA-6658 > URL: https://issues.apache.org/jira/browse/IMPALA-6658 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Csaba Ringhofer >Assignee: Andrew Sherman >Priority: Minor > Labels: parquet, ramp-up > Fix For: Impala 3.1.0 > > > Currently RleEncoder creates repeated runs from 8 repeated values, which can > be less space efficient than bit-packed if bit width is 1 or 2. In the worst > case, the whole data page can be ~2X larger if bit width is 1, and ~1.25X > larger if bit is 2 compared to bit-packing. > A comment in rle_encoding.h writes different numbers, but it probably does > not calculate with the overhead of splitting long runs into smaller ones > (every run adds +1 byte for its length): > [https://github.com/apache/impala/blob/8079cd9d2a87051f81a41910b74fab15e35f36ea/be/src/util/rle-encoding.h#L62] > Note that if the data page is compressed, this size difference probably > disappears, but the larger uncompressed buffer size can still affect > performance. > Parquet RLE encoding is described here: > [https://github.com/apache/parquet-format/blob/master/Encodings.md#run-length-encoding-bit-packing-hybrid-rle-3] > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6802) Clean up tests in AuthorizationTest
[ https://issues.apache.org/jira/browse/IMPALA-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790087#comment-16790087 ] ASF subversion and git services commented on IMPALA-6802: - Commit e75a5b1f3faf371b75ae980b2ae511ec55b44057 in impala's branch refs/heads/2.x from Fredy Wijaya [ https://gitbox.apache.org/repos/asf?p=impala.git;h=e75a5b1 ] IMPALA-6802 (part 2): Clean up authorization tests The second part of this patch is to rewrite the following authorization tests: - insert - truncate - load - use Testing: - Added new authorization tests - Ran all front-end tests Cherry-picks: not for 2.x Change-Id: Ica5bfd2f12bf0cc8ebe78464a523a7805366f67b Reviewed-on: http://gerrit.cloudera.org:8080/10269 Reviewed-by: Alex Behm Tested-by: Impala Public Jenkins Reviewed-on: http://gerrit.cloudera.org:8080/12709 Reviewed-by: Fredy Wijaya > Clean up tests in AuthorizationTest > --- > > Key: IMPALA-6802 > URL: https://issues.apache.org/jira/browse/IMPALA-6802 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > Labels: security > Fix For: Impala 3.1.0 > > > The tests in AuthorizationTest can be refactored to be more condensed. We > also need to increase the code coverage. It is also a good idea to refactor > these tests to provide flexibility in supporting tests for user-level > privileges in the future: https://issues.apache.org/jira/browse/IMPALA-6794 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-988) Join strategy (broadcast vs shuffle) decision does not take mem limit and other joins into account
[ https://issues.apache.org/jira/browse/IMPALA-988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790102#comment-16790102 ] ASF subversion and git services commented on IMPALA-988: Commit 9cc75c59d5a09bd898bdf05cc64a98a70ffd in impala's branch refs/heads/master from Alex Rodoni [ https://gitbox.apache.org/repos/asf?p=impala.git;h=9cc75c5 ] IMPALA-8133: [DOCS] Review and update Known Issues for 3.2 - Added IMPALA-988 - Reviewed the Known Issues from 3.1 and updated the ones fixed in 3.2 Change-Id: Ia6b22fc5cf3944ffeeaf8189932e88498a14696e Reviewed-on: http://gerrit.cloudera.org:8080/12718 Tested-by: Impala Public Jenkins Reviewed-by: Paul Rogers > Join strategy (broadcast vs shuffle) decision does not take mem limit and > other joins into account > -- > > Key: IMPALA-988 > URL: https://issues.apache.org/jira/browse/IMPALA-988 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Affects Versions: Impala 1.2.1 >Reporter: Alan Choi >Priority: Minor > Labels: resource-management > > The amount of available memory changes the trade-off between partitioned and > shuffle join strategies: if switching to shuffle join can avoid spilling to > disk, it may be worth paying the cost of the additional network transfer. > There are two issues: > 1. Join strategy decision only takes query mem-limit into account but ignore > process mem-limit. > 2. Join strategy decision does not take other joins of the same query into > account. When multiple joins are present, it'll go over the mem-limit. > Note that when IMPALA-3200 is completed, this shouldn't prevent the query > running to completion, but still affects performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8288) Setting EXEC_TIME_LIMIT_S to very high value triggers DCHECK in pretty printer
[
https://issues.apache.org/jira/browse/IMPALA-8288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman resolved IMPALA-8288.
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Setting EXEC_TIME_LIMIT_S to very high value triggers DCHECK in pretty printer
> --
>
> Key: IMPALA-8288
> URL: https://issues.apache.org/jira/browse/IMPALA-8288
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Tim Armstrong
>Assignee: Andrew Sherman
>Priority: Critical
> Labels: crash, newbie, ramp-up
> Fix For: Impala 3.3.0
>
>
> You can see from the below backtrace that the value was 2147483647, which
> overflows to -1000 when multiplied by 1000. In a RELEASE build this would
> just result in a negative number being printed.
> {noformat}
> (gdb) bt
> #0 0x7fa99b79d428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x7fa99b79f02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2 0x047fe424 in google::DumpStackTraceAndExit() ()
> #3 0x047f4e7d in google::LogMessage::Fail() ()
> #4 0x047f6722 in google::LogMessage::SendToLog() ()
> #5 0x047f4857 in google::LogMessage::Flush() ()
> #6 0x047f7e1e in google::LogMessageFatal::~LogMessageFatal() ()
> #7 0x01f7cae8 in impala::PrettyPrinter::PrintTimeMs
> (value=-1000, ss=0x7fa8f28563d0) at be/src/util/pretty-printer.h:271
> #8 0x01f7aee1 in impala::PrettyPrinter::Print
> (value=2147483647, unit=impala::TUnit::TIME_S, verbose=false) at
> be/src/util/pretty-printer.h:110
> #9 0x02068bf7 in impala::ImpalaServer::SetQueryInflight
> (this=0xcf84000, session_state=..., request_state=...) at
> be/src/service/impala-server.cc:1108
> #10 0x020f119c in impala::ImpalaServer::ExecuteStatement
> (this=0xcf84000, return_val=..., request=...) at
> be/src/service/impala-hs2-server.cc:477
> #11 0x02600972 in
> apache::hive::service::cli::thrift::TCLIServiceProcessor::process_ExecuteStatement
> (this=0xd0dcbe0, seqid=0, iprot=0x15d10640, oprot=0x15d10440,
> callContext=0xb596cf80)
> at be/generated-sources/gen-cpp/TCLIService.cpp:5115
> #12 0x025ff070 in
> apache::hive::service::cli::thrift::TCLIServiceProcessor::dispatchCall
> (this=0xd0dcbe0, iprot=0x15d10640, oprot=0x15d10440, fname=..., seqid=0,
> callContext=0xb596cf80)
> at be/generated-sources/gen-cpp/TCLIService.cpp:4926
> #13 0x025c4b35 in
> impala::ImpalaHiveServer2ServiceProcessor::dispatchCall (this=0xd0dcbe0,
> iprot=0x15d10640, oprot=0x15d10440, fname=..., seqid=0,
> callContext=0xb596cf80)
> at be/generated-sources/gen-cpp/ImpalaHiveServer2Service.cpp:505
> #14 0x01a0d29c in apache::thrift::TDispatchProcessor::process
> (this=0xd0dcbe0, in=..., out=..., connectionContext=0xb596cf80)
> at
> /opt/Impala-Toolchain/thrift-0.9.3-p5/include/thrift/TDispatchProcessor.h:121
> #15 0x01e5ffb7 in
> apache::thrift::server::TAcceptQueueServer::Task::run (this=0x80546c40) at
> be/src/rpc/TAcceptQueueServer.cpp:
> {noformat}
> {noformat}
> F0306 09:40:56.856586 70607 pretty-printer.h:271] Check failed: value >=
> static_cast(0) (-1000 vs. 0)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8133) Impala Doc: Review the list of Known Issues and update for 3.2
[ https://issues.apache.org/jira/browse/IMPALA-8133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790101#comment-16790101 ] ASF subversion and git services commented on IMPALA-8133: - Commit 9cc75c59d5a09bd898bdf05cc64a98a70ffd in impala's branch refs/heads/master from Alex Rodoni [ https://gitbox.apache.org/repos/asf?p=impala.git;h=9cc75c5 ] IMPALA-8133: [DOCS] Review and update Known Issues for 3.2 - Added IMPALA-988 - Reviewed the Known Issues from 3.1 and updated the ones fixed in 3.2 Change-Id: Ia6b22fc5cf3944ffeeaf8189932e88498a14696e Reviewed-on: http://gerrit.cloudera.org:8080/12718 Tested-by: Impala Public Jenkins Reviewed-by: Paul Rogers > Impala Doc: Review the list of Known Issues and update for 3.2 > -- > > Key: IMPALA-8133 > URL: https://issues.apache.org/jira/browse/IMPALA-8133 > Project: IMPALA > Issue Type: Task > Components: Docs >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > Labels: future_release_doc, in_32 > > https://gerrit.cloudera.org/#/c/12631/ > https://gerrit.cloudera.org/#/c/12718/ -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8279) Revert IMPALA-6658 to avoid ETL performance regression
[
https://issues.apache.org/jira/browse/IMPALA-8279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman resolved IMPALA-8279.
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Revert IMPALA-6658 to avoid ETL performance regression
> --
>
> Key: IMPALA-8279
> URL: https://issues.apache.org/jira/browse/IMPALA-8279
> Project: IMPALA
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
>Priority: Major
> Fix For: Impala 3.3.0
>
>
> The fix for IMPALA-6658 seems to cause a measurable regression on
> {quote}
> use tpcds;
> create TABLE store_sales_unpart stored as parquet as SELECT * FROM
> tpcds.store_sales;
> INSERT OVERWRITE TABLE store_sales_unpart SELECT * FROM store_sales;
> {quote}
> Revert the change to avoid the regression.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-8279) Revert IMPALA-6658 to avoid ETL performance regression
[
https://issues.apache.org/jira/browse/IMPALA-8279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman reassigned IMPALA-8279:
--
Assignee: Andrew Sherman
> Revert IMPALA-6658 to avoid ETL performance regression
> --
>
> Key: IMPALA-8279
> URL: https://issues.apache.org/jira/browse/IMPALA-8279
> Project: IMPALA
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
>Priority: Major
>
> The fix for IMPALA-6658 seems to cause a measurable regression on
> {quote}
> use tpcds;
> create TABLE store_sales_unpart stored as parquet as SELECT * FROM
> tpcds.store_sales;
> INSERT OVERWRITE TABLE store_sales_unpart SELECT * FROM store_sales;
> {quote}
> Revert the change to avoid the regression.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6802) Clean up tests in AuthorizationTest
[ https://issues.apache.org/jira/browse/IMPALA-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790086#comment-16790086 ] ASF subversion and git services commented on IMPALA-6802: - Commit b320fcd593489b279871866114ac91b775044f69 in impala's branch refs/heads/2.x from Fredy Wijaya [ https://gitbox.apache.org/repos/asf?p=impala.git;h=b320fcd ] IMPALA-6802 (part 1): Clean up authorization tests The first patch of this patch is to introduce a new mechanism of testing authorization that tests authorization at every hierarchy. This patch rewrites the authorization tests for select statements. Testing: - Added new authorization tests - Ran all front-end tests Cherry-picks: not for 2.x Change-Id: I9cd5713607c423f644451af5ebb3494d3a728e3b Reviewed-on: http://gerrit.cloudera.org:8080/10135 Reviewed-by: Alex Behm Tested-by: Impala Public Jenkins Reviewed-on: http://gerrit.cloudera.org:8080/12708 Reviewed-by: Fredy Wijaya > Clean up tests in AuthorizationTest > --- > > Key: IMPALA-6802 > URL: https://issues.apache.org/jira/browse/IMPALA-6802 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > Labels: security > Fix For: Impala 3.1.0 > > > The tests in AuthorizationTest can be refactored to be more condensed. We > also need to increase the code coverage. It is also a good idea to refactor > these tests to provide flexibility in supporting tests for user-level > privileges in the future: https://issues.apache.org/jira/browse/IMPALA-6794 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6658) Parquet RLE encoding can waste space with small repeated runs
[ https://issues.apache.org/jira/browse/IMPALA-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790095#comment-16790095 ] ASF subversion and git services commented on IMPALA-6658: - Commit 729fcf34764928bebfe7cf31153b57f7246404af in impala's branch refs/heads/master from Andrew Sherman [ https://gitbox.apache.org/repos/asf?p=impala.git;h=729fcf3 ] IMPALA-8279: Revert IMPALA-6658 to avoid ETL performance regression. IMPALA-6658 changed RleEncoder to have the ability to use run lengths other than 8. It seemed that a slightly more complex RleEncoder could save a small amount of disk space by using the longer run lengths, in particular for bit width of 1. We now see a performance regression on a simple ETL query. Overall it seems that the costs of IMPALA-6658 exceed the benefits. This change removes IMPALA-6658. The strategy for this was that the change to rle-encoding.h, which contains the code, was undone using 'git revert'. I removed the test changes in rle-test.cc that rely on different encoding lengths. This allows us to keep some useful new tests that were written as part of IMPALA-6658 TESTING: Ran all end-to-end tests. Change-Id: If6bcbaf564fbbe6dc83ba3afc100b4e5ccc7af40 Reviewed-on: http://gerrit.cloudera.org:8080/12680 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Parquet RLE encoding can waste space with small repeated runs > - > > Key: IMPALA-6658 > URL: https://issues.apache.org/jira/browse/IMPALA-6658 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Csaba Ringhofer >Assignee: Andrew Sherman >Priority: Minor > Labels: parquet, ramp-up > Fix For: Impala 3.1.0 > > > Currently RleEncoder creates repeated runs from 8 repeated values, which can > be less space efficient than bit-packed if bit width is 1 or 2. In the worst > case, the whole data page can be ~2X larger if bit width is 1, and ~1.25X > larger if bit is 2 compared to bit-packing. > A comment in rle_encoding.h writes different numbers, but it probably does > not calculate with the overhead of splitting long runs into smaller ones > (every run adds +1 byte for its length): > [https://github.com/apache/impala/blob/8079cd9d2a87051f81a41910b74fab15e35f36ea/be/src/util/rle-encoding.h#L62] > Note that if the data page is compressed, this size difference probably > disappears, but the larger uncompressed buffer size can still affect > performance. > Parquet RLE encoding is described here: > [https://github.com/apache/parquet-format/blob/master/Encodings.md#run-length-encoding-bit-packing-hybrid-rle-3] > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6802) Clean up tests in AuthorizationTest
[ https://issues.apache.org/jira/browse/IMPALA-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790088#comment-16790088 ] ASF subversion and git services commented on IMPALA-6802: - Commit 74c7e42962e1d29fa9414d7893d3f1f4e7a4ba76 in impala's branch refs/heads/2.x from Fredy Wijaya [ https://gitbox.apache.org/repos/asf?p=impala.git;h=74c7e42 ] IMPALA-6802 (part 3): Clean up authorization tests The third part of this patch is to rewrite the following authorization tests: - with - union - reset metadata - show Testing: - Added new authorization tests - Ran all front-end tests Change-Id: I9681cc3c7094db33ab7c5caa69b99dd803b908b7 Cherry-picks: not for 2.x Reviewed-on: http://gerrit.cloudera.org:8080/10358 Reviewed-by: Alex Behm Tested-by: Impala Public Jenkins Reviewed-on: http://gerrit.cloudera.org:8080/12710 Reviewed-by: Fredy Wijaya > Clean up tests in AuthorizationTest > --- > > Key: IMPALA-6802 > URL: https://issues.apache.org/jira/browse/IMPALA-6802 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > Labels: security > Fix For: Impala 3.1.0 > > > The tests in AuthorizationTest can be refactored to be more condensed. We > also need to increase the code coverage. It is also a good idea to refactor > these tests to provide flexibility in supporting tests for user-level > privileges in the future: https://issues.apache.org/jira/browse/IMPALA-6794 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6658) Parquet RLE encoding can waste space with small repeated runs
[ https://issues.apache.org/jira/browse/IMPALA-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790093#comment-16790093 ] ASF subversion and git services commented on IMPALA-6658: - Commit 729fcf34764928bebfe7cf31153b57f7246404af in impala's branch refs/heads/master from Andrew Sherman [ https://gitbox.apache.org/repos/asf?p=impala.git;h=729fcf3 ] IMPALA-8279: Revert IMPALA-6658 to avoid ETL performance regression. IMPALA-6658 changed RleEncoder to have the ability to use run lengths other than 8. It seemed that a slightly more complex RleEncoder could save a small amount of disk space by using the longer run lengths, in particular for bit width of 1. We now see a performance regression on a simple ETL query. Overall it seems that the costs of IMPALA-6658 exceed the benefits. This change removes IMPALA-6658. The strategy for this was that the change to rle-encoding.h, which contains the code, was undone using 'git revert'. I removed the test changes in rle-test.cc that rely on different encoding lengths. This allows us to keep some useful new tests that were written as part of IMPALA-6658 TESTING: Ran all end-to-end tests. Change-Id: If6bcbaf564fbbe6dc83ba3afc100b4e5ccc7af40 Reviewed-on: http://gerrit.cloudera.org:8080/12680 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Parquet RLE encoding can waste space with small repeated runs > - > > Key: IMPALA-6658 > URL: https://issues.apache.org/jira/browse/IMPALA-6658 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Csaba Ringhofer >Assignee: Andrew Sherman >Priority: Minor > Labels: parquet, ramp-up > Fix For: Impala 3.1.0 > > > Currently RleEncoder creates repeated runs from 8 repeated values, which can > be less space efficient than bit-packed if bit width is 1 or 2. In the worst > case, the whole data page can be ~2X larger if bit width is 1, and ~1.25X > larger if bit is 2 compared to bit-packing. > A comment in rle_encoding.h writes different numbers, but it probably does > not calculate with the overhead of splitting long runs into smaller ones > (every run adds +1 byte for its length): > [https://github.com/apache/impala/blob/8079cd9d2a87051f81a41910b74fab15e35f36ea/be/src/util/rle-encoding.h#L62] > Note that if the data page is compressed, this size difference probably > disappears, but the larger uncompressed buffer size can still affect > performance. > Parquet RLE encoding is described here: > [https://github.com/apache/parquet-format/blob/master/Encodings.md#run-length-encoding-bit-packing-hybrid-rle-3] > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6802) Clean up tests in AuthorizationTest
[ https://issues.apache.org/jira/browse/IMPALA-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790089#comment-16790089 ] ASF subversion and git services commented on IMPALA-6802: - Commit 80fa7bb5a093c3f0c248b1612d8c25f9e887 in impala's branch refs/heads/2.x from Adam Holley [ https://gitbox.apache.org/repos/asf?p=impala.git;h=80fa7bb ] IMPALA-6802 (part 4): Clean up authorization tests The fourth part of this patch is to rewrite the following authorization tests: - describe Testing: - Added new authorization tests - Ran all front-end tests Cherry-picks: not for 2.x Change-Id: Ic4cf3a4751b41908ef81ec35d89a2713d9fa0dc4 Reviewed-on: http://gerrit.cloudera.org:8080/10442 Tested-by: Impala Public Jenkins Reviewed-by: Vuk Ercegovac Reviewed-on: http://gerrit.cloudera.org:8080/12711 Reviewed-by: Fredy Wijaya > Clean up tests in AuthorizationTest > --- > > Key: IMPALA-6802 > URL: https://issues.apache.org/jira/browse/IMPALA-6802 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Reporter: Fredy Wijaya >Assignee: Fredy Wijaya >Priority: Critical > Labels: security > Fix For: Impala 3.1.0 > > > The tests in AuthorizationTest can be refactored to be more condensed. We > also need to increase the code coverage. It is also a good idea to refactor > these tests to provide flexibility in supporting tests for user-level > privileges in the future: https://issues.apache.org/jira/browse/IMPALA-6794 -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7974) Impala Doc: Doc the options to enable automatic invalidates using metastore notification events
[ https://issues.apache.org/jira/browse/IMPALA-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790099#comment-16790099 ] ASF subversion and git services commented on IMPALA-7974: - Commit 76286bf2c0320cce0eb1bf0c269d344255b4dd0e in impala's branch refs/heads/master from Alex Rodoni [ https://gitbox.apache.org/repos/asf?p=impala.git;h=76286bf ] IMPALA-7974: [DOCS] Document automatic invalidation via HMS notification - Moved all Metadata V2 related topics to a new doc. - Added the new doc under Impala Scalability. - Will track the other 2 new features in separate PRs. Change-Id: I6c8a4ec77152c839564b1d69c76dfa041a0ebd7f Reviewed-on: http://gerrit.cloudera.org:8080/12700 Tested-by: Impala Public Jenkins Reviewed-by: Vihang Karajgaonkar Reviewed-by: Bharath Vissapragada > Impala Doc: Doc the options to enable automatic invalidates using metastore > notification events > --- > > Key: IMPALA-7974 > URL: https://issues.apache.org/jira/browse/IMPALA-7974 > Project: IMPALA > Issue Type: Sub-task > Components: Docs >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > Labels: future_release_doc, in_32 > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8300) Build failed on S3: test_max_nesting_depth (table_format: orc/def/block) timeouts consistently
[
https://issues.apache.org/jira/browse/IMPALA-8300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790098#comment-16790098
]
ASF subversion and git services commented on IMPALA-8300:
-
Commit 26639963e0ea66bea716c0238cdf1ff086159c7c in impala's branch
refs/heads/master from Csaba Ringhofer
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=2663996 ]
IMPALA-8300: Skip test_max_nesting_depth for Orc in non-HDFS tests
test_max_nesting_depth works with both Orc and Parquet formats, and
needs Hive queries for data load only in the Orc case. Hive only works
with HDFS, so this test needs to be skipped if file format is Orc and
the filesystem is not HDFS.
Change-Id: Ie97166b59c451fadb33e4e780056afbaec11baa8
Reviewed-on: http://gerrit.cloudera.org:8080/12714
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Build failed on S3: test_max_nesting_depth (table_format: orc/def/block)
> timeouts consistently
> --
>
> Key: IMPALA-8300
> URL: https://issues.apache.org/jira/browse/IMPALA-8300
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Csaba Ringhofer
>Priority: Blocker
> Labels: broken-build, s3
>
> This failure is probably related to the recently added support for complex
> types in Orc.
> query_test.test_nested_types.TestMaxNestingDepth.test_max_nesting_depth[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> orc/def/block] (from pytest)
> Took 2 hr 0 min.
> Error Message
> query_test/test_nested_types.py:614: in test_max_nesting_depth
> self.__create_orc_tables(unique_database)
> query_test/test_nested_types.py:640: in __create_orc_tables
> self.run_stmt_in_hive(create_table) common/impala_test_suite.py:775: in
> run_stmt_in_hive (stdout, stderr) = call.communicate()
> /usr/lib64/python2.7/subprocess.py:800: in communicate return
> self._communicate(input) /usr/lib64/python2.7/subprocess.py:1401: in
> _communicate stdout, stderr = self._communicate_with_poll(input)
> /usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll ready
> = poller.poll() E Failed: Timeout >7200s
> Stacktrace
> query_test/test_nested_types.py:614: in test_max_nesting_depth
> self.__create_orc_tables(unique_database)
> query_test/test_nested_types.py:640: in __create_orc_tables
> self.run_stmt_in_hive(create_table)
> common/impala_test_suite.py:775: in run_stmt_in_hive
> (stdout, stderr) = call.communicate()
> /usr/lib64/python2.7/subprocess.py:800: in communicate
> return self._communicate(input)
> /usr/lib64/python2.7/subprocess.py:1401: in _communicate
> stdout, stderr = self._communicate_with_poll(input)
> /usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll
> ready = poller.poll()
> E Failed: Timeout >7200s
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8279) Revert IMPALA-6658 to avoid ETL performance regression
[
https://issues.apache.org/jira/browse/IMPALA-8279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790091#comment-16790091
]
ASF subversion and git services commented on IMPALA-8279:
-
Commit 729fcf34764928bebfe7cf31153b57f7246404af in impala's branch
refs/heads/master from Andrew Sherman
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=729fcf3 ]
IMPALA-8279: Revert IMPALA-6658 to avoid ETL performance regression.
IMPALA-6658 changed RleEncoder to have the ability to use run lengths
other than 8. It seemed that a slightly more complex RleEncoder could
save a small amount of disk space by using the longer run lengths, in
particular for bit width of 1. We now see a performance regression on a
simple ETL query. Overall it seems that the costs of IMPALA-6658 exceed
the benefits. This change removes IMPALA-6658.
The strategy for this was that the change to rle-encoding.h, which
contains the code, was undone using 'git revert'. I removed the test
changes in rle-test.cc that rely on different encoding lengths. This
allows us to keep some useful new tests that were written as part of
IMPALA-6658
TESTING:
Ran all end-to-end tests.
Change-Id: If6bcbaf564fbbe6dc83ba3afc100b4e5ccc7af40
Reviewed-on: http://gerrit.cloudera.org:8080/12680
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Revert IMPALA-6658 to avoid ETL performance regression
> --
>
> Key: IMPALA-8279
> URL: https://issues.apache.org/jira/browse/IMPALA-8279
> Project: IMPALA
> Issue Type: Bug
>Reporter: Andrew Sherman
>Priority: Major
>
> The fix for IMPALA-6658 seems to cause a measurable regression on
> {quote}
> use tpcds;
> create TABLE store_sales_unpart stored as parquet as SELECT * FROM
> tpcds.store_sales;
> INSERT OVERWRITE TABLE store_sales_unpart SELECT * FROM store_sales;
> {quote}
> Revert the change to avoid the regression.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6658) Parquet RLE encoding can waste space with small repeated runs
[ https://issues.apache.org/jira/browse/IMPALA-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790096#comment-16790096 ] ASF subversion and git services commented on IMPALA-6658: - Commit 729fcf34764928bebfe7cf31153b57f7246404af in impala's branch refs/heads/master from Andrew Sherman [ https://gitbox.apache.org/repos/asf?p=impala.git;h=729fcf3 ] IMPALA-8279: Revert IMPALA-6658 to avoid ETL performance regression. IMPALA-6658 changed RleEncoder to have the ability to use run lengths other than 8. It seemed that a slightly more complex RleEncoder could save a small amount of disk space by using the longer run lengths, in particular for bit width of 1. We now see a performance regression on a simple ETL query. Overall it seems that the costs of IMPALA-6658 exceed the benefits. This change removes IMPALA-6658. The strategy for this was that the change to rle-encoding.h, which contains the code, was undone using 'git revert'. I removed the test changes in rle-test.cc that rely on different encoding lengths. This allows us to keep some useful new tests that were written as part of IMPALA-6658 TESTING: Ran all end-to-end tests. Change-Id: If6bcbaf564fbbe6dc83ba3afc100b4e5ccc7af40 Reviewed-on: http://gerrit.cloudera.org:8080/12680 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Parquet RLE encoding can waste space with small repeated runs > - > > Key: IMPALA-6658 > URL: https://issues.apache.org/jira/browse/IMPALA-6658 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Csaba Ringhofer >Assignee: Andrew Sherman >Priority: Minor > Labels: parquet, ramp-up > Fix For: Impala 3.1.0 > > > Currently RleEncoder creates repeated runs from 8 repeated values, which can > be less space efficient than bit-packed if bit width is 1 or 2. In the worst > case, the whole data page can be ~2X larger if bit width is 1, and ~1.25X > larger if bit is 2 compared to bit-packing. > A comment in rle_encoding.h writes different numbers, but it probably does > not calculate with the overhead of splitting long runs into smaller ones > (every run adds +1 byte for its length): > [https://github.com/apache/impala/blob/8079cd9d2a87051f81a41910b74fab15e35f36ea/be/src/util/rle-encoding.h#L62] > Note that if the data page is compressed, this size difference probably > disappears, but the larger uncompressed buffer size can still affect > performance. > Parquet RLE encoding is described here: > [https://github.com/apache/parquet-format/blob/master/Encodings.md#run-length-encoding-bit-packing-hybrid-rle-3] > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8284) KuduTableSink spends a lot of CPU copying KuduColumnSchemas
[
https://issues.apache.org/jira/browse/IMPALA-8284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790090#comment-16790090
]
ASF subversion and git services commented on IMPALA-8284:
-
Commit 49027de9ff4450a438c8e631373c572ce189b36e in impala's branch
refs/heads/master from Todd Lipcon
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=49027de ]
IMPALA-8284. KuduTableSink spends too much CPU in KuduSchema::Column()
The KuduSchema::Column() accessor actually returns a copy of the
KuduColumnSchema object, which is not lightweight. We were inadvertently
calling this function once for every null cell seen during an insertion.
This caused a performance bottleneck for datasets with large numbers of
NULL cells.
This improves the situation by caching the nullability of the Kudu
columns in our own vector. The vector lookups should be inlined and much
faster than copying a KuduColumnSchema.
No new tests included as this is a perf fix.
Change-Id: I1b4d14d20252bdb190f50ebaaf6179a46eafb932
Reviewed-on: http://gerrit.cloudera.org:8080/12692
Reviewed-by: Will Berkeley
Reviewed-by: Thomas Marshall
Tested-by: Impala Public Jenkins
> KuduTableSink spends a lot of CPU copying KuduColumnSchemas
> ---
>
> Key: IMPALA-8284
> URL: https://issues.apache.org/jira/browse/IMPALA-8284
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: Will Berkeley
>Assignee: Todd Lipcon
>Priority: Major
> Labels: kudu, newbie, ramp-up
> Fix For: Impala 3.2.0
>
>
> I noticed Impala spending a significant amount of CPU time in
> {{KuduTableSink::Send}} creating and destroying KuduColumnSchemas.
> See KUDU-2731 for more information.
> Impala could wait for a better option from the Kudu API, or could cache
> information about nullability of columns outside the hot loop in Send.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6658) Parquet RLE encoding can waste space with small repeated runs
[ https://issues.apache.org/jira/browse/IMPALA-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790094#comment-16790094 ] ASF subversion and git services commented on IMPALA-6658: - Commit 729fcf34764928bebfe7cf31153b57f7246404af in impala's branch refs/heads/master from Andrew Sherman [ https://gitbox.apache.org/repos/asf?p=impala.git;h=729fcf3 ] IMPALA-8279: Revert IMPALA-6658 to avoid ETL performance regression. IMPALA-6658 changed RleEncoder to have the ability to use run lengths other than 8. It seemed that a slightly more complex RleEncoder could save a small amount of disk space by using the longer run lengths, in particular for bit width of 1. We now see a performance regression on a simple ETL query. Overall it seems that the costs of IMPALA-6658 exceed the benefits. This change removes IMPALA-6658. The strategy for this was that the change to rle-encoding.h, which contains the code, was undone using 'git revert'. I removed the test changes in rle-test.cc that rely on different encoding lengths. This allows us to keep some useful new tests that were written as part of IMPALA-6658 TESTING: Ran all end-to-end tests. Change-Id: If6bcbaf564fbbe6dc83ba3afc100b4e5ccc7af40 Reviewed-on: http://gerrit.cloudera.org:8080/12680 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Parquet RLE encoding can waste space with small repeated runs > - > > Key: IMPALA-6658 > URL: https://issues.apache.org/jira/browse/IMPALA-6658 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Csaba Ringhofer >Assignee: Andrew Sherman >Priority: Minor > Labels: parquet, ramp-up > Fix For: Impala 3.1.0 > > > Currently RleEncoder creates repeated runs from 8 repeated values, which can > be less space efficient than bit-packed if bit width is 1 or 2. In the worst > case, the whole data page can be ~2X larger if bit width is 1, and ~1.25X > larger if bit is 2 compared to bit-packing. > A comment in rle_encoding.h writes different numbers, but it probably does > not calculate with the overhead of splitting long runs into smaller ones > (every run adds +1 byte for its length): > [https://github.com/apache/impala/blob/8079cd9d2a87051f81a41910b74fab15e35f36ea/be/src/util/rle-encoding.h#L62] > Note that if the data page is compressed, this size difference probably > disappears, but the larger uncompressed buffer size can still affect > performance. > Parquet RLE encoding is described here: > [https://github.com/apache/parquet-format/blob/master/Encodings.md#run-length-encoding-bit-packing-hybrid-rle-3] > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8288) Setting EXEC_TIME_LIMIT_S to very high value triggers DCHECK in pretty printer
[
https://issues.apache.org/jira/browse/IMPALA-8288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16790097#comment-16790097
]
ASF subversion and git services commented on IMPALA-8288:
-
Commit 288abf10f6142ae6cb02329604805a9a1dcc804f in impala's branch
refs/heads/master from Andrew Sherman
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=288abf1 ]
IMPALA-8288: avoid overflow causing DCHECK in PrettyPrint.
When PrettyPrinter prints a time measured in seconds, it multiplies the
time by 1000, and prints that value as milliseconds. Because
PrettyPrinter is implemented using templates, the type of the value
depends on the type of the value parameter. For typical values of
seconds the type is an int32_t. When this is multiplied by 1000 it can
therefore overflow, which triggers a DHECK failure in DEBUG builds.
Fix this by instead multiplying the value parameter by the existing
constant 'THOUSAND' which is declared as a int64_t. This produces a
result which is also a int64_t, and which does not overflow so easily.
TESTING:
Add more test cases to pretty-printer-test.cc including cases that
previously caused overflows. Expand the coverage to include cases
printing NanoSeocnds, MillisSeconds and MicroSeconds. These cases are
not supposed to show that PrettyPrinter always behaves consistently, but
to help maintainers avoid regressions when changing PrettyPrinter.
Ran all end-to-end tests.
Change-Id: I00d9769cf34e2ccd796ec1cf88797c8f8250f718
Reviewed-on: http://gerrit.cloudera.org:8080/12702
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Setting EXEC_TIME_LIMIT_S to very high value triggers DCHECK in pretty printer
> --
>
> Key: IMPALA-8288
> URL: https://issues.apache.org/jira/browse/IMPALA-8288
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.2.0
>Reporter: Tim Armstrong
>Assignee: Andrew Sherman
>Priority: Critical
> Labels: crash, newbie, ramp-up
>
> You can see from the below backtrace that the value was 2147483647, which
> overflows to -1000 when multiplied by 1000. In a RELEASE build this would
> just result in a negative number being printed.
> {noformat}
> (gdb) bt
> #0 0x7fa99b79d428 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x7fa99b79f02a in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2 0x047fe424 in google::DumpStackTraceAndExit() ()
> #3 0x047f4e7d in google::LogMessage::Fail() ()
> #4 0x047f6722 in google::LogMessage::SendToLog() ()
> #5 0x047f4857 in google::LogMessage::Flush() ()
> #6 0x047f7e1e in google::LogMessageFatal::~LogMessageFatal() ()
> #7 0x01f7cae8 in impala::PrettyPrinter::PrintTimeMs
> (value=-1000, ss=0x7fa8f28563d0) at be/src/util/pretty-printer.h:271
> #8 0x01f7aee1 in impala::PrettyPrinter::Print
> (value=2147483647, unit=impala::TUnit::TIME_S, verbose=false) at
> be/src/util/pretty-printer.h:110
> #9 0x02068bf7 in impala::ImpalaServer::SetQueryInflight
> (this=0xcf84000, session_state=..., request_state=...) at
> be/src/service/impala-server.cc:1108
> #10 0x020f119c in impala::ImpalaServer::ExecuteStatement
> (this=0xcf84000, return_val=..., request=...) at
> be/src/service/impala-hs2-server.cc:477
> #11 0x02600972 in
> apache::hive::service::cli::thrift::TCLIServiceProcessor::process_ExecuteStatement
> (this=0xd0dcbe0, seqid=0, iprot=0x15d10640, oprot=0x15d10440,
> callContext=0xb596cf80)
> at be/generated-sources/gen-cpp/TCLIService.cpp:5115
> #12 0x025ff070 in
> apache::hive::service::cli::thrift::TCLIServiceProcessor::dispatchCall
> (this=0xd0dcbe0, iprot=0x15d10640, oprot=0x15d10440, fname=..., seqid=0,
> callContext=0xb596cf80)
> at be/generated-sources/gen-cpp/TCLIService.cpp:4926
> #13 0x025c4b35 in
> impala::ImpalaHiveServer2ServiceProcessor::dispatchCall (this=0xd0dcbe0,
> iprot=0x15d10640, oprot=0x15d10440, fname=..., seqid=0,
> callContext=0xb596cf80)
> at be/generated-sources/gen-cpp/ImpalaHiveServer2Service.cpp:505
> #14 0x01a0d29c in apache::thrift::TDispatchProcessor::process
> (this=0xd0dcbe0, in=..., out=..., connectionContext=0xb596cf80)
> at
> /opt/Impala-Toolchain/thrift-0.9.3-p5/include/thrift/TDispatchProcessor.h:121
> #15 0x01e5ffb7 in
> apache::thrift::server::TAcceptQueueServer::Task::run (this=0x80546c40) at
> be/src/rpc/TAcceptQueueServer.cpp:
> {noformat}
> {noformat}
> F0306 09:40:56.856586 70607 pretty-printer.h:271] Check failed: value >=
> static_cast(0) (-1000 vs. 0)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To
[jira] [Updated] (IMPALA-8133) Impala Doc: Review the list of Known Issues and update for 3.2
[ https://issues.apache.org/jira/browse/IMPALA-8133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-8133: Description: https://gerrit.cloudera.org/#/c/12631/ https://gerrit.cloudera.org/#/c/12718/ was:https://gerrit.cloudera.org/#/c/12631/ > Impala Doc: Review the list of Known Issues and update for 3.2 > -- > > Key: IMPALA-8133 > URL: https://issues.apache.org/jira/browse/IMPALA-8133 > Project: IMPALA > Issue Type: Task > Components: Docs >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > Labels: future_release_doc, in_32 > > https://gerrit.cloudera.org/#/c/12631/ > https://gerrit.cloudera.org/#/c/12718/ -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8298) Update docs of ORC about complex types support
[ https://issues.apache.org/jira/browse/IMPALA-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789823#comment-16789823 ] Alex Rodoni commented on IMPALA-8298: - [~stiga-huang] No, I have not started on this ticket. Your help will be very appreciated. Thank you! > Update docs of ORC about complex types support > -- > > Key: IMPALA-8298 > URL: https://issues.apache.org/jira/browse/IMPALA-8298 > Project: IMPALA > Issue Type: Task >Reporter: Quanlong Huang >Assignee: Alex Rodoni >Priority: Major > Labels: future_release_doc, in_32 > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8161) Impala Doc: Document the supported Date and Time patterns
[ https://issues.apache.org/jira/browse/IMPALA-8161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-8161: Labels: future_release_doc in_33 (was: future_release_doc in_32) > Impala Doc: Document the supported Date and Time patterns > - > > Key: IMPALA-8161 > URL: https://issues.apache.org/jira/browse/IMPALA-8161 > Project: IMPALA > Issue Type: Task > Components: Docs >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > Labels: future_release_doc, in_33 > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8298) Update docs of ORC about complex types support
[ https://issues.apache.org/jira/browse/IMPALA-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni updated IMPALA-8298: Target Version: Impala 3.2.0 > Update docs of ORC about complex types support > -- > > Key: IMPALA-8298 > URL: https://issues.apache.org/jira/browse/IMPALA-8298 > Project: IMPALA > Issue Type: Task >Reporter: Quanlong Huang >Assignee: Alex Rodoni >Priority: Major > Labels: future_release_doc, in_32 > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8300) Build failed on S3: test_max_nesting_depth (table_format: orc/def/block) timeouts consistently
[
https://issues.apache.org/jira/browse/IMPALA-8300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789707#comment-16789707
]
Csaba Ringhofer commented on IMPALA-8300:
-
[~stiga-huang] Sorry, we wrote our comments in parallel, so I just read yours.
I created a patch to skip this test in the Orc + non-HDFS case:
https://gerrit.cloudera.org/#/c/12714/
I do not know why Hive doesn't work on S3, but this seems to be an accepted
fact in Impala:
https://github.com/apache/impala/blob/9686545bfd7236350d165813621bf783b42d1c18/tests/common/skip.py#L44
> Build failed on S3: test_max_nesting_depth (table_format: orc/def/block)
> timeouts consistently
> --
>
> Key: IMPALA-8300
> URL: https://issues.apache.org/jira/browse/IMPALA-8300
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Csaba Ringhofer
>Priority: Blocker
> Labels: broken-build, s3
>
> This failure is probably related to the recently added support for complex
> types in Orc.
> query_test.test_nested_types.TestMaxNestingDepth.test_max_nesting_depth[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> orc/def/block] (from pytest)
> Took 2 hr 0 min.
> Error Message
> query_test/test_nested_types.py:614: in test_max_nesting_depth
> self.__create_orc_tables(unique_database)
> query_test/test_nested_types.py:640: in __create_orc_tables
> self.run_stmt_in_hive(create_table) common/impala_test_suite.py:775: in
> run_stmt_in_hive (stdout, stderr) = call.communicate()
> /usr/lib64/python2.7/subprocess.py:800: in communicate return
> self._communicate(input) /usr/lib64/python2.7/subprocess.py:1401: in
> _communicate stdout, stderr = self._communicate_with_poll(input)
> /usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll ready
> = poller.poll() E Failed: Timeout >7200s
> Stacktrace
> query_test/test_nested_types.py:614: in test_max_nesting_depth
> self.__create_orc_tables(unique_database)
> query_test/test_nested_types.py:640: in __create_orc_tables
> self.run_stmt_in_hive(create_table)
> common/impala_test_suite.py:775: in run_stmt_in_hive
> (stdout, stderr) = call.communicate()
> /usr/lib64/python2.7/subprocess.py:800: in communicate
> return self._communicate(input)
> /usr/lib64/python2.7/subprocess.py:1401: in _communicate
> stdout, stderr = self._communicate_with_poll(input)
> /usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll
> ready = poller.poll()
> E Failed: Timeout >7200s
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8298) Update docs of ORC about complex types support
[ https://issues.apache.org/jira/browse/IMPALA-8298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789657#comment-16789657 ] Quanlong Huang commented on IMPALA-8298: Hi [~arodoni_cloudera], have you started to work on this? I can help to accelerate it if you are lack of bandwidth. > Update docs of ORC about complex types support > -- > > Key: IMPALA-8298 > URL: https://issues.apache.org/jira/browse/IMPALA-8298 > Project: IMPALA > Issue Type: Task >Reporter: Quanlong Huang >Assignee: Alex Rodoni >Priority: Major > Labels: future_release_doc, in_32 > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8300) Build failed on S3: test_max_nesting_depth (table_format: orc/def/block) timeouts consistently
[
https://issues.apache.org/jira/browse/IMPALA-8300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789633#comment-16789633
]
Csaba Ringhofer commented on IMPALA-8300:
-
The problem is that loading the Orc tables needs Hive
(https://github.com/apache/impala/blob/9686545bfd7236350d165813621bf783b42d1c18/tests/query_test/test_nested_types.py#L640
), and Hive only works with HDFS. So the solution is to skip these tests if
the file_format is Orc and the file system is not HDFS
> Build failed on S3: test_max_nesting_depth (table_format: orc/def/block)
> timeouts consistently
> --
>
> Key: IMPALA-8300
> URL: https://issues.apache.org/jira/browse/IMPALA-8300
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Csaba Ringhofer
>Priority: Blocker
> Labels: broken-build, s3
>
> This failure is probably related to the recently added support for complex
> types in Orc.
> query_test.test_nested_types.TestMaxNestingDepth.test_max_nesting_depth[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> orc/def/block] (from pytest)
> Took 2 hr 0 min.
> Error Message
> query_test/test_nested_types.py:614: in test_max_nesting_depth
> self.__create_orc_tables(unique_database)
> query_test/test_nested_types.py:640: in __create_orc_tables
> self.run_stmt_in_hive(create_table) common/impala_test_suite.py:775: in
> run_stmt_in_hive (stdout, stderr) = call.communicate()
> /usr/lib64/python2.7/subprocess.py:800: in communicate return
> self._communicate(input) /usr/lib64/python2.7/subprocess.py:1401: in
> _communicate stdout, stderr = self._communicate_with_poll(input)
> /usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll ready
> = poller.poll() E Failed: Timeout >7200s
> Stacktrace
> query_test/test_nested_types.py:614: in test_max_nesting_depth
> self.__create_orc_tables(unique_database)
> query_test/test_nested_types.py:640: in __create_orc_tables
> self.run_stmt_in_hive(create_table)
> common/impala_test_suite.py:775: in run_stmt_in_hive
> (stdout, stderr) = call.communicate()
> /usr/lib64/python2.7/subprocess.py:800: in communicate
> return self._communicate(input)
> /usr/lib64/python2.7/subprocess.py:1401: in _communicate
> stdout, stderr = self._communicate_with_poll(input)
> /usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll
> ready = poller.poll()
> E Failed: Timeout >7200s
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8300) Build failed on S3: test_max_nesting_depth (table_format: orc/def/block) timeouts consistently
[
https://issues.apache.org/jira/browse/IMPALA-8300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789620#comment-16789620
]
Quanlong Huang commented on IMPALA-8300:
[~csringhofer], thanks for finding this! Could you rerun the
"test_max_nesting_depth" test in your environment to see if we can simply
reproduce it?
{code:java}
./tests/run-tests.py query_test/test_nested_types.py -k
test_max_nesting_depth{code}
This test will re-create the tables. "test_max_nesting_depth_ac91beaa" is a
unique database name. Looks like there're dead locks caused by creating two
tables at the same time.
> Build failed on S3: test_max_nesting_depth (table_format: orc/def/block)
> timeouts consistently
> --
>
> Key: IMPALA-8300
> URL: https://issues.apache.org/jira/browse/IMPALA-8300
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Csaba Ringhofer
>Priority: Blocker
> Labels: broken-build, s3
>
> This failure is probably related to the recently added support for complex
> types in Orc.
> query_test.test_nested_types.TestMaxNestingDepth.test_max_nesting_depth[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> orc/def/block] (from pytest)
> Took 2 hr 0 min.
> Error Message
> query_test/test_nested_types.py:614: in test_max_nesting_depth
> self.__create_orc_tables(unique_database)
> query_test/test_nested_types.py:640: in __create_orc_tables
> self.run_stmt_in_hive(create_table) common/impala_test_suite.py:775: in
> run_stmt_in_hive (stdout, stderr) = call.communicate()
> /usr/lib64/python2.7/subprocess.py:800: in communicate return
> self._communicate(input) /usr/lib64/python2.7/subprocess.py:1401: in
> _communicate stdout, stderr = self._communicate_with_poll(input)
> /usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll ready
> = poller.poll() E Failed: Timeout >7200s
> Stacktrace
> query_test/test_nested_types.py:614: in test_max_nesting_depth
> self.__create_orc_tables(unique_database)
> query_test/test_nested_types.py:640: in __create_orc_tables
> self.run_stmt_in_hive(create_table)
> common/impala_test_suite.py:775: in run_stmt_in_hive
> (stdout, stderr) = call.communicate()
> /usr/lib64/python2.7/subprocess.py:800: in communicate
> return self._communicate(input)
> /usr/lib64/python2.7/subprocess.py:1401: in _communicate
> stdout, stderr = self._communicate_with_poll(input)
> /usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll
> ready = poller.poll()
> E Failed: Timeout >7200s
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8300) Build failed on S3: test_max_nesting_depth (table_format: orc/def/block) timeouts consistently
[
https://issues.apache.org/jira/browse/IMPALA-8300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789599#comment-16789599
]
Quanlong Huang commented on IMPALA-8300:
The stacktrace shows that it hangs at creating an ORC table which is created by
"CREATE TABLE %s LIKE %s STORED AS ORC". [~csringhofer], do you have chance to
see the logs of Hive Metastore? I don't have a S3 environment so it's hard to
reproduce this.
> Build failed on S3: test_max_nesting_depth (table_format: orc/def/block)
> timeouts consistently
> --
>
> Key: IMPALA-8300
> URL: https://issues.apache.org/jira/browse/IMPALA-8300
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Csaba Ringhofer
>Priority: Blocker
> Labels: broken-build, s3
>
> This failure is probably related to the recently added support for complex
> types in Orc.
> query_test.test_nested_types.TestMaxNestingDepth.test_max_nesting_depth[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> orc/def/block] (from pytest)
> Took 2 hr 0 min.
> Error Message
> query_test/test_nested_types.py:614: in test_max_nesting_depth
> self.__create_orc_tables(unique_database)
> query_test/test_nested_types.py:640: in __create_orc_tables
> self.run_stmt_in_hive(create_table) common/impala_test_suite.py:775: in
> run_stmt_in_hive (stdout, stderr) = call.communicate()
> /usr/lib64/python2.7/subprocess.py:800: in communicate return
> self._communicate(input) /usr/lib64/python2.7/subprocess.py:1401: in
> _communicate stdout, stderr = self._communicate_with_poll(input)
> /usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll ready
> = poller.poll() E Failed: Timeout >7200s
> Stacktrace
> query_test/test_nested_types.py:614: in test_max_nesting_depth
> self.__create_orc_tables(unique_database)
> query_test/test_nested_types.py:640: in __create_orc_tables
> self.run_stmt_in_hive(create_table)
> common/impala_test_suite.py:775: in run_stmt_in_hive
> (stdout, stderr) = call.communicate()
> /usr/lib64/python2.7/subprocess.py:800: in communicate
> return self._communicate(input)
> /usr/lib64/python2.7/subprocess.py:1401: in _communicate
> stdout, stderr = self._communicate_with_poll(input)
> /usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll
> ready = poller.poll()
> E Failed: Timeout >7200s
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Issue Comment Deleted] (IMPALA-8300) Build failed on S3: test_max_nesting_depth (table_format: orc/def/block) timeouts consistently
[
https://issues.apache.org/jira/browse/IMPALA-8300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-8300:
---
Comment: was deleted
(was: The stacktrace shows that it hangs at creating an ORC table which is
created by "CREATE TABLE %s LIKE %s STORED AS ORC". [~csringhofer], do you have
chance to see the logs of Hive Metastore? I don't have a S3 environment so it's
hard to reproduce this.)
> Build failed on S3: test_max_nesting_depth (table_format: orc/def/block)
> timeouts consistently
> --
>
> Key: IMPALA-8300
> URL: https://issues.apache.org/jira/browse/IMPALA-8300
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Csaba Ringhofer
>Priority: Blocker
> Labels: broken-build, s3
>
> This failure is probably related to the recently added support for complex
> types in Orc.
> query_test.test_nested_types.TestMaxNestingDepth.test_max_nesting_depth[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> orc/def/block] (from pytest)
> Took 2 hr 0 min.
> Error Message
> query_test/test_nested_types.py:614: in test_max_nesting_depth
> self.__create_orc_tables(unique_database)
> query_test/test_nested_types.py:640: in __create_orc_tables
> self.run_stmt_in_hive(create_table) common/impala_test_suite.py:775: in
> run_stmt_in_hive (stdout, stderr) = call.communicate()
> /usr/lib64/python2.7/subprocess.py:800: in communicate return
> self._communicate(input) /usr/lib64/python2.7/subprocess.py:1401: in
> _communicate stdout, stderr = self._communicate_with_poll(input)
> /usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll ready
> = poller.poll() E Failed: Timeout >7200s
> Stacktrace
> query_test/test_nested_types.py:614: in test_max_nesting_depth
> self.__create_orc_tables(unique_database)
> query_test/test_nested_types.py:640: in __create_orc_tables
> self.run_stmt_in_hive(create_table)
> common/impala_test_suite.py:775: in run_stmt_in_hive
> (stdout, stderr) = call.communicate()
> /usr/lib64/python2.7/subprocess.py:800: in communicate
> return self._communicate(input)
> /usr/lib64/python2.7/subprocess.py:1401: in _communicate
> stdout, stderr = self._communicate_with_poll(input)
> /usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll
> ready = poller.poll()
> E Failed: Timeout >7200s
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8300) Build failed on S3: test_max_nesting_depth (table_format: orc/def/block) timeouts consistently
[
https://issues.apache.org/jira/browse/IMPALA-8300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789592#comment-16789592
]
Csaba Ringhofer commented on IMPALA-8300:
-
The hanging query seems to be a CREATE TABLE LIKE in Hive. From Hive logs:
Unable to acquire IMPLICIT, SHARED lock test_max_nesting_depth_ac91beaa after
100 attempts.
Error in acquireLocks...
FAILED: Error in acquiring locks: Locks on the underlying objects cannot be
acquired. retry after some time
> Build failed on S3: test_max_nesting_depth (table_format: orc/def/block)
> timeouts consistently
> --
>
> Key: IMPALA-8300
> URL: https://issues.apache.org/jira/browse/IMPALA-8300
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Csaba Ringhofer
>Priority: Blocker
> Labels: broken-build, s3
>
> This failure is probably related to the recently added support for complex
> types in Orc.
> query_test.test_nested_types.TestMaxNestingDepth.test_max_nesting_depth[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> orc/def/block] (from pytest)
> Took 2 hr 0 min.
> Error Message
> query_test/test_nested_types.py:614: in test_max_nesting_depth
> self.__create_orc_tables(unique_database)
> query_test/test_nested_types.py:640: in __create_orc_tables
> self.run_stmt_in_hive(create_table) common/impala_test_suite.py:775: in
> run_stmt_in_hive (stdout, stderr) = call.communicate()
> /usr/lib64/python2.7/subprocess.py:800: in communicate return
> self._communicate(input) /usr/lib64/python2.7/subprocess.py:1401: in
> _communicate stdout, stderr = self._communicate_with_poll(input)
> /usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll ready
> = poller.poll() E Failed: Timeout >7200s
> Stacktrace
> query_test/test_nested_types.py:614: in test_max_nesting_depth
> self.__create_orc_tables(unique_database)
> query_test/test_nested_types.py:640: in __create_orc_tables
> self.run_stmt_in_hive(create_table)
> common/impala_test_suite.py:775: in run_stmt_in_hive
> (stdout, stderr) = call.communicate()
> /usr/lib64/python2.7/subprocess.py:800: in communicate
> return self._communicate(input)
> /usr/lib64/python2.7/subprocess.py:1401: in _communicate
> stdout, stderr = self._communicate_with_poll(input)
> /usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll
> ready = poller.poll()
> E Failed: Timeout >7200s
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8300) Build failed on S3: test_max_nesting_depth (table_format: orc/def/block) timeouts consistently
[
https://issues.apache.org/jira/browse/IMPALA-8300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Csaba Ringhofer updated IMPALA-8300:
Description:
This failure is probably related to the recently added support for complex
types in Orc.
query_test.test_nested_types.TestMaxNestingDepth.test_max_nesting_depth[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
orc/def/block] (from pytest)
Took 2 hr 0 min.
Error Message
query_test/test_nested_types.py:614: in test_max_nesting_depth
self.__create_orc_tables(unique_database) query_test/test_nested_types.py:640:
in __create_orc_tables self.run_stmt_in_hive(create_table)
common/impala_test_suite.py:775: in run_stmt_in_hive (stdout, stderr) =
call.communicate() /usr/lib64/python2.7/subprocess.py:800: in communicate
return self._communicate(input) /usr/lib64/python2.7/subprocess.py:1401: in
_communicate stdout, stderr = self._communicate_with_poll(input)
/usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll ready =
poller.poll() E Failed: Timeout >7200s
Stacktrace
query_test/test_nested_types.py:614: in test_max_nesting_depth
self.__create_orc_tables(unique_database)
query_test/test_nested_types.py:640: in __create_orc_tables
self.run_stmt_in_hive(create_table)
common/impala_test_suite.py:775: in run_stmt_in_hive
(stdout, stderr) = call.communicate()
/usr/lib64/python2.7/subprocess.py:800: in communicate
return self._communicate(input)
/usr/lib64/python2.7/subprocess.py:1401: in _communicate
stdout, stderr = self._communicate_with_poll(input)
/usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll
ready = poller.poll()
E Failed: Timeout >7200s
was:
query_test.test_nested_types.TestMaxNestingDepth.test_max_nesting_depth[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
orc/def/block] (from pytest)
Took 2 hr 0 min.
Error Message
query_test/test_nested_types.py:614: in test_max_nesting_depth
self.__create_orc_tables(unique_database) query_test/test_nested_types.py:640:
in __create_orc_tables self.run_stmt_in_hive(create_table)
common/impala_test_suite.py:775: in run_stmt_in_hive (stdout, stderr) =
call.communicate() /usr/lib64/python2.7/subprocess.py:800: in communicate
return self._communicate(input) /usr/lib64/python2.7/subprocess.py:1401: in
_communicate stdout, stderr = self._communicate_with_poll(input)
/usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll ready =
poller.poll() E Failed: Timeout >7200s
Stacktrace
query_test/test_nested_types.py:614: in test_max_nesting_depth
self.__create_orc_tables(unique_database)
query_test/test_nested_types.py:640: in __create_orc_tables
self.run_stmt_in_hive(create_table)
common/impala_test_suite.py:775: in run_stmt_in_hive
(stdout, stderr) = call.communicate()
/usr/lib64/python2.7/subprocess.py:800: in communicate
return self._communicate(input)
/usr/lib64/python2.7/subprocess.py:1401: in _communicate
stdout, stderr = self._communicate_with_poll(input)
/usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll
ready = poller.poll()
E Failed: Timeout >7200s
> Build failed on S3: test_max_nesting_depth (table_format: orc/def/block)
> timeouts consistently
> --
>
> Key: IMPALA-8300
> URL: https://issues.apache.org/jira/browse/IMPALA-8300
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Csaba Ringhofer
>Priority: Blocker
> Labels: broken-build, s3
>
> This failure is probably related to the recently added support for complex
> types in Orc.
> query_test.test_nested_types.TestMaxNestingDepth.test_max_nesting_depth[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> orc/def/block] (from pytest)
> Took 2 hr 0 min.
> Error Message
> query_test/test_nested_types.py:614: in test_max_nesting_depth
> self.__create_orc_tables(unique_database)
> query_test/test_nested_types.py:640: in __create_orc_tables
> self.run_stmt_in_hive(create_table) common/impala_test_suite.py:775: in
> run_stmt_in_hive (stdout, stderr) = call.communicate()
> /usr/lib64/python2.7/subprocess.py:800: in communicate return
> self._communicate(input) /usr/lib64/python2.7/subprocess.py:1401: in
> _communicate
[jira] [Created] (IMPALA-8300) Build failed on S3: test_max_nesting_depth (table_format: orc/def/block) timeouts consistently
Csaba Ringhofer created IMPALA-8300:
---
Summary: Build failed on S3: test_max_nesting_depth (table_format:
orc/def/block) timeouts consistently
Key: IMPALA-8300
URL: https://issues.apache.org/jira/browse/IMPALA-8300
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 3.3.0
Reporter: Csaba Ringhofer
query_test.test_nested_types.TestMaxNestingDepth.test_max_nesting_depth[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
orc/def/block] (from pytest)
Took 2 hr 0 min.
Error Message
query_test/test_nested_types.py:614: in test_max_nesting_depth
self.__create_orc_tables(unique_database) query_test/test_nested_types.py:640:
in __create_orc_tables self.run_stmt_in_hive(create_table)
common/impala_test_suite.py:775: in run_stmt_in_hive (stdout, stderr) =
call.communicate() /usr/lib64/python2.7/subprocess.py:800: in communicate
return self._communicate(input) /usr/lib64/python2.7/subprocess.py:1401: in
_communicate stdout, stderr = self._communicate_with_poll(input)
/usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll ready =
poller.poll() E Failed: Timeout >7200s
Stacktrace
query_test/test_nested_types.py:614: in test_max_nesting_depth
self.__create_orc_tables(unique_database)
query_test/test_nested_types.py:640: in __create_orc_tables
self.run_stmt_in_hive(create_table)
common/impala_test_suite.py:775: in run_stmt_in_hive
(stdout, stderr) = call.communicate()
/usr/lib64/python2.7/subprocess.py:800: in communicate
return self._communicate(input)
/usr/lib64/python2.7/subprocess.py:1401: in _communicate
stdout, stderr = self._communicate_with_poll(input)
/usr/lib64/python2.7/subprocess.py:1455: in _communicate_with_poll
ready = poller.poll()
E Failed: Timeout >7200s
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
