[jira] [Commented] (IMPALA-8740) TestCodegen.test_disable_codegen flakiness
[
https://issues.apache.org/jira/browse/IMPALA-8740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883431#comment-16883431
]
Michael Ho commented on IMPALA-8740:
This is mostly due to some flakiness in stat computation or data loading.
[~anuragmantri], do you happen to have the log of the entire run somewhere ?
> TestCodegen.test_disable_codegen flakiness
> ---
>
> Key: IMPALA-8740
> URL: https://issues.apache.org/jira/browse/IMPALA-8740
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Anurag Mantripragada
>Assignee: Tim Armstrong
>Priority: Major
> Labels: broken-build
>
> Looks like the test cannot find "Codegen Disabled: disabled due to
> optimization hints" in the profile and fails the assertion:
> {code:java}
> query_test/test_codegen.py:44: in test_disable_codegen
> self.run_test_case('QueryTest/disable-codegen', vector)
> common/impala_test_suite.py:617: in run_test_case
> update_section=pytest.config.option.update_results)
> common/test_result_verifier.py:605: in verify_runtime_profile
> actual))
> E AssertionError: Did not find matches for lines in runtime profile:
> E EXPECTED LINES:
> E row_regex: .*Codegen Disabled: disabled due to optimization hints.*
> ...
> ACTUAL PROFILE FOLLOWS{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8740) TestCodegen.test_disable_codegen flakiness
[
https://issues.apache.org/jira/browse/IMPALA-8740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883430#comment-16883430
]
Michael Ho commented on IMPALA-8740:
Based on the profile, it seems the stats on one of the table has corrupted
stats:
{noformat}
E WARNING: The following tables have potentially corrupt table statistics.
E Drop and re-compute statistics to resolve this problem.
E functional.alltypes
{noformat}
Consequently, the logic for disabling codegen didn't get run at all:
{noformat}
private void checkForDisableCodegen(PlanNode distributedPlan) {
MaxRowsProcessedVisitor visitor = new MaxRowsProcessedVisitor();
distributedPlan.accept(visitor);
if (!visitor.valid()) return; <<
// This heuristic threshold tries to determine if the per-node codegen time
will
// reduce per-node execution time enough to justify the cost of codegen.
Per-node
// execution time is correlated with the number of rows flowing through the
plan.
if (visitor.getMaxRowsProcessedPerNode()
< ctx_.getQueryOptions().getDisable_codegen_rows_threshold()) {
ctx_.getQueryCtx().disable_codegen_hint = true;
}
}
{noformat}
> TestCodegen.test_disable_codegen flakiness
> ---
>
> Key: IMPALA-8740
> URL: https://issues.apache.org/jira/browse/IMPALA-8740
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Anurag Mantripragada
>Assignee: Tim Armstrong
>Priority: Major
> Labels: broken-build
>
> Looks like the test cannot find "Codegen Disabled: disabled due to
> optimization hints" in the profile and fails the assertion:
> {code:java}
> query_test/test_codegen.py:44: in test_disable_codegen
> self.run_test_case('QueryTest/disable-codegen', vector)
> common/impala_test_suite.py:617: in run_test_case
> update_section=pytest.config.option.update_results)
> common/test_result_verifier.py:605: in verify_runtime_profile
> actual))
> E AssertionError: Did not find matches for lines in runtime profile:
> E EXPECTED LINES:
> E row_regex: .*Codegen Disabled: disabled due to optimization hints.*
> ...
> ACTUAL PROFILE FOLLOWS{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-8754) S3 with S3Guard tests encounter "ResourceNotFoundException" from DynamoDB
[
https://issues.apache.org/jira/browse/IMPALA-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell reassigned IMPALA-8754:
-
Assignee: Joe McDonnell
> S3 with S3Guard tests encounter "ResourceNotFoundException" from DynamoDB
> -
>
> Key: IMPALA-8754
> URL: https://issues.apache.org/jira/browse/IMPALA-8754
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: Joe McDonnell
>Assignee: Joe McDonnell
>Priority: Critical
> Labels: broken-build, flaky
>
> When running tests on s3 with s3guard, various tests can encounter the
> following error coming from the DynamoDB:
> {noformat}
> EQuery aborted:Disk I/O error on
> impala-ec2-centos74-m5-4xlarge-ondemand-02c8.vpc.cloudera.com:22002: Failed
> to open HDFS file
> s3a://impala-test-uswest2-1/test-warehouse/tpcds.store_sales_parquet/ss_sold_date_sk=2451718/6843d8a91fc5ae1d-88b2af4b0004_156969840_data.0.parq
> E Error(2): No such file or directory
> E Root cause: ResourceNotFoundException: Requested resource not found
> (Service: AmazonDynamoDBv2; Status Code: 400; Error Code:
> ResourceNotFoundException; Request ID:
> XXX){noformat}
> Tests that have seen this (this is flaky):
> * TestTpcdsQuery.test_tpcds_count
> * TestHdfsFdCaching.test_caching_disabled_by_param
> * TestMtDop.test_compute_stats
> * TestScanRangeLengths.test_scan_ranges
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8754) S3 with S3Guard tests encounter "ResourceNotFoundException" from DynamoDB
Joe McDonnell created IMPALA-8754:
-
Summary: S3 with S3Guard tests encounter
"ResourceNotFoundException" from DynamoDB
Key: IMPALA-8754
URL: https://issues.apache.org/jira/browse/IMPALA-8754
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.3.0
Reporter: Joe McDonnell
When running tests on s3 with s3guard, various tests can encounter the
following error coming from the DynamoDB:
{noformat}
EQuery aborted:Disk I/O error on
impala-ec2-centos74-m5-4xlarge-ondemand-02c8.vpc.cloudera.com:22002: Failed to
open HDFS file
s3a://impala-test-uswest2-1/test-warehouse/tpcds.store_sales_parquet/ss_sold_date_sk=2451718/6843d8a91fc5ae1d-88b2af4b0004_156969840_data.0.parq
E Error(2): No such file or directory
E Root cause: ResourceNotFoundException: Requested resource not found
(Service: AmazonDynamoDBv2; Status Code: 400; Error Code:
ResourceNotFoundException; Request ID:
XXX){noformat}
Tests that have seen this (this is flaky):
* TestTpcdsQuery.test_tpcds_count
* TestHdfsFdCaching.test_caching_disabled_by_param
* TestMtDop.test_compute_stats
* TestScanRangeLengths.test_scan_ranges
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8754) S3 with S3Guard tests encounter "ResourceNotFoundException" from DynamoDB
Joe McDonnell created IMPALA-8754:
-
Summary: S3 with S3Guard tests encounter
"ResourceNotFoundException" from DynamoDB
Key: IMPALA-8754
URL: https://issues.apache.org/jira/browse/IMPALA-8754
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 3.3.0
Reporter: Joe McDonnell
When running tests on s3 with s3guard, various tests can encounter the
following error coming from the DynamoDB:
{noformat}
EQuery aborted:Disk I/O error on
impala-ec2-centos74-m5-4xlarge-ondemand-02c8.vpc.cloudera.com:22002: Failed to
open HDFS file
s3a://impala-test-uswest2-1/test-warehouse/tpcds.store_sales_parquet/ss_sold_date_sk=2451718/6843d8a91fc5ae1d-88b2af4b0004_156969840_data.0.parq
E Error(2): No such file or directory
E Root cause: ResourceNotFoundException: Requested resource not found
(Service: AmazonDynamoDBv2; Status Code: 400; Error Code:
ResourceNotFoundException; Request ID:
XXX){noformat}
Tests that have seen this (this is flaky):
* TestTpcdsQuery.test_tpcds_count
* TestHdfsFdCaching.test_caching_disabled_by_param
* TestMtDop.test_compute_stats
* TestScanRangeLengths.test_scan_ranges
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Resolved] (IMPALA-8543) Log more diagnostics information in TAcceptQueueServer
[
https://issues.apache.org/jira/browse/IMPALA-8543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Ho resolved IMPALA-8543.
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Log more diagnostics information in TAcceptQueueServer
> --
>
> Key: IMPALA-8543
> URL: https://issues.apache.org/jira/browse/IMPALA-8543
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Affects Versions: Impala 2.12.0, Impala 3.1.0, Impala 3.2.0
>Reporter: Michael Ho
>Assignee: Michael Ho
>Priority: Major
> Labels: ramp-up
> Fix For: Impala 3.3.0
>
>
> There is currently not much diagnostic information in
> {{TAcceptQueueServer.cpp}}. It would be nice to add some additional logging
> to identify clients which misbehaved or took longer than expected when
> setting up connections.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Resolved] (IMPALA-8543) Log more diagnostics information in TAcceptQueueServer
[
https://issues.apache.org/jira/browse/IMPALA-8543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Ho resolved IMPALA-8543.
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> Log more diagnostics information in TAcceptQueueServer
> --
>
> Key: IMPALA-8543
> URL: https://issues.apache.org/jira/browse/IMPALA-8543
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Affects Versions: Impala 2.12.0, Impala 3.1.0, Impala 3.2.0
>Reporter: Michael Ho
>Assignee: Michael Ho
>Priority: Major
> Labels: ramp-up
> Fix For: Impala 3.3.0
>
>
> There is currently not much diagnostic information in
> {{TAcceptQueueServer.cpp}}. It would be nice to add some additional logging
> to identify clients which misbehaved or took longer than expected when
> setting up connections.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8702) PlannerTest.testJoins flakiness due to reliance on an exact cardinality estimate
[
https://issues.apache.org/jira/browse/IMPALA-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell updated IMPALA-8702:
--
Fix Version/s: Impala 3.3.0
> PlannerTest.testJoins flakiness due to reliance on an exact cardinality
> estimate
>
>
> Key: IMPALA-8702
> URL: https://issues.apache.org/jira/browse/IMPALA-8702
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Bikramjeet Vig
>Assignee: Fang-Yu Rao
>Priority: Critical
> Labels: broken-build
> Fix For: Impala 3.3.0
>
>
> The following test case is failing in PlannerTest.testJoins.
>
> {noformat}
> Section PLAN of query:
> select * from functional_text_lzo.emptytable a inner join
> functional_text_lzo.alltypes b on a.f2 = b.int_col
> Actual does not match expected result:
> PLAN-ROOT SINK
> |
> 02:HASH JOIN [INNER JOIN]
> | hash predicates: b.int_col = a.f2
> | runtime filters: RF000 <- a.f2
> | row-size=96B cardinality=5.66K
> ^
> |
> |--00:SCAN HDFS [functional_text_lzo.emptytable a]
> | partitions=0/0 files=0 size=0B
> | row-size=16B cardinality=0
> |
> 01:SCAN HDFS [functional_text_lzo.alltypes b]
>HDFS partitions=24/24 files=24 size=123.54KB
>runtime filters: RF000 -> b.int_col
>row-size=80B cardinality=5.66K
> Expected:
> PLAN-ROOT SINK
> |
> 02:HASH JOIN [INNER JOIN]
> | hash predicates: b.int_col = a.f2
> | runtime filters: RF000 <- a.f2
> | row-size=96B cardinality=5.65K
> |
> |--00:SCAN HDFS [functional_text_lzo.emptytable a]
> | partitions=0/0 files=0 size=0B
> | row-size=16B cardinality=0
> |
> 01:SCAN HDFS [functional_text_lzo.alltypes b]
>HDFS partitions=24/24 files=24 size=123.32KB
>runtime filters: RF000 -> b.int_col
>row-size=80B cardinality=5.65K
> Verbose plan:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> Per-Host Resources: mem-estimate=130.94MB mem-reservation=2.95MB
> thread-reservation=2 runtime-filters-memory=1.00MB
> PLAN-ROOT SINK
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 02:HASH JOIN [INNER JOIN]
> | hash predicates: b.int_col = a.f2
> | fk/pk conjuncts: assumed fk/pk
> | runtime filters: RF000[bloom] <- a.f2
> | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB
> thread-reservation=0
> | tuple-ids=1,0 row-size=96B cardinality=5.66K
> | in pipelines: 01(GETNEXT), 00(OPEN)
> |
> |--00:SCAN HDFS [functional_text_lzo.emptytable a]
> | partitions=0/0 files=0 size=0B
> | stored statistics:
> | table: rows=unavailable size=unavailable
> | partitions: 0/0 rows=unavailable
> | columns missing stats: field
> | extrapolated-rows=disabled max-scan-range-rows=0
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> | tuple-ids=0 row-size=16B cardinality=0
> | in pipelines: 00(GETNEXT)
> |
> 01:SCAN HDFS [functional_text_lzo.alltypes b]
> HDFS partitions=24/24 files=24 size=123.54KB
> runtime filters: RF000[bloom] -> b.int_col
> stored statistics:
>table: rows=unavailable size=unavailable
>partitions: 0/24 rows=unavailable
>columns missing stats: int_col, id, bool_col, tinyint_col,
> smallint_col, bigint_col, float_col, double_col, date_string_col, string_col,
> timestamp_col
> extrapolated-rows=disabled max-scan-range-rows=unavailable
> mem-estimate=128.00MB mem-reservation=8.00KB thread-reservation=1
> tuple-ids=1 row-size=80B cardinality=5.66K
> in pipelines: 01(GETNEXT)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8702) PlannerTest.testJoins flakiness due to reliance on an exact cardinality estimate
[
https://issues.apache.org/jira/browse/IMPALA-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell updated IMPALA-8702:
--
Component/s: Frontend
> PlannerTest.testJoins flakiness due to reliance on an exact cardinality
> estimate
>
>
> Key: IMPALA-8702
> URL: https://issues.apache.org/jira/browse/IMPALA-8702
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Bikramjeet Vig
>Assignee: Fang-Yu Rao
>Priority: Critical
> Labels: broken-build
> Fix For: Impala 3.3.0
>
>
> The following test case is failing in PlannerTest.testJoins.
>
> {noformat}
> Section PLAN of query:
> select * from functional_text_lzo.emptytable a inner join
> functional_text_lzo.alltypes b on a.f2 = b.int_col
> Actual does not match expected result:
> PLAN-ROOT SINK
> |
> 02:HASH JOIN [INNER JOIN]
> | hash predicates: b.int_col = a.f2
> | runtime filters: RF000 <- a.f2
> | row-size=96B cardinality=5.66K
> ^
> |
> |--00:SCAN HDFS [functional_text_lzo.emptytable a]
> | partitions=0/0 files=0 size=0B
> | row-size=16B cardinality=0
> |
> 01:SCAN HDFS [functional_text_lzo.alltypes b]
>HDFS partitions=24/24 files=24 size=123.54KB
>runtime filters: RF000 -> b.int_col
>row-size=80B cardinality=5.66K
> Expected:
> PLAN-ROOT SINK
> |
> 02:HASH JOIN [INNER JOIN]
> | hash predicates: b.int_col = a.f2
> | runtime filters: RF000 <- a.f2
> | row-size=96B cardinality=5.65K
> |
> |--00:SCAN HDFS [functional_text_lzo.emptytable a]
> | partitions=0/0 files=0 size=0B
> | row-size=16B cardinality=0
> |
> 01:SCAN HDFS [functional_text_lzo.alltypes b]
>HDFS partitions=24/24 files=24 size=123.32KB
>runtime filters: RF000 -> b.int_col
>row-size=80B cardinality=5.65K
> Verbose plan:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> Per-Host Resources: mem-estimate=130.94MB mem-reservation=2.95MB
> thread-reservation=2 runtime-filters-memory=1.00MB
> PLAN-ROOT SINK
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 02:HASH JOIN [INNER JOIN]
> | hash predicates: b.int_col = a.f2
> | fk/pk conjuncts: assumed fk/pk
> | runtime filters: RF000[bloom] <- a.f2
> | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB
> thread-reservation=0
> | tuple-ids=1,0 row-size=96B cardinality=5.66K
> | in pipelines: 01(GETNEXT), 00(OPEN)
> |
> |--00:SCAN HDFS [functional_text_lzo.emptytable a]
> | partitions=0/0 files=0 size=0B
> | stored statistics:
> | table: rows=unavailable size=unavailable
> | partitions: 0/0 rows=unavailable
> | columns missing stats: field
> | extrapolated-rows=disabled max-scan-range-rows=0
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> | tuple-ids=0 row-size=16B cardinality=0
> | in pipelines: 00(GETNEXT)
> |
> 01:SCAN HDFS [functional_text_lzo.alltypes b]
> HDFS partitions=24/24 files=24 size=123.54KB
> runtime filters: RF000[bloom] -> b.int_col
> stored statistics:
>table: rows=unavailable size=unavailable
>partitions: 0/24 rows=unavailable
>columns missing stats: int_col, id, bool_col, tinyint_col,
> smallint_col, bigint_col, float_col, double_col, date_string_col, string_col,
> timestamp_col
> extrapolated-rows=disabled max-scan-range-rows=unavailable
> mem-estimate=128.00MB mem-reservation=8.00KB thread-reservation=1
> tuple-ids=1 row-size=80B cardinality=5.66K
> in pipelines: 01(GETNEXT)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8702) PlannerTest.testJoins flakiness due to reliance on an exact cardinality estimate
[
https://issues.apache.org/jira/browse/IMPALA-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883383#comment-16883383
]
Fang-Yu Rao commented on IMPALA-8702:
-
Thanks Joe. I have marked this as resolved since this is fixed. :D
> PlannerTest.testJoins flakiness due to reliance on an exact cardinality
> estimate
>
>
> Key: IMPALA-8702
> URL: https://issues.apache.org/jira/browse/IMPALA-8702
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Bikramjeet Vig
>Assignee: Fang-Yu Rao
>Priority: Critical
> Labels: broken-build
>
> The following test case is failing in PlannerTest.testJoins.
>
> {noformat}
> Section PLAN of query:
> select * from functional_text_lzo.emptytable a inner join
> functional_text_lzo.alltypes b on a.f2 = b.int_col
> Actual does not match expected result:
> PLAN-ROOT SINK
> |
> 02:HASH JOIN [INNER JOIN]
> | hash predicates: b.int_col = a.f2
> | runtime filters: RF000 <- a.f2
> | row-size=96B cardinality=5.66K
> ^
> |
> |--00:SCAN HDFS [functional_text_lzo.emptytable a]
> | partitions=0/0 files=0 size=0B
> | row-size=16B cardinality=0
> |
> 01:SCAN HDFS [functional_text_lzo.alltypes b]
>HDFS partitions=24/24 files=24 size=123.54KB
>runtime filters: RF000 -> b.int_col
>row-size=80B cardinality=5.66K
> Expected:
> PLAN-ROOT SINK
> |
> 02:HASH JOIN [INNER JOIN]
> | hash predicates: b.int_col = a.f2
> | runtime filters: RF000 <- a.f2
> | row-size=96B cardinality=5.65K
> |
> |--00:SCAN HDFS [functional_text_lzo.emptytable a]
> | partitions=0/0 files=0 size=0B
> | row-size=16B cardinality=0
> |
> 01:SCAN HDFS [functional_text_lzo.alltypes b]
>HDFS partitions=24/24 files=24 size=123.32KB
>runtime filters: RF000 -> b.int_col
>row-size=80B cardinality=5.65K
> Verbose plan:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> Per-Host Resources: mem-estimate=130.94MB mem-reservation=2.95MB
> thread-reservation=2 runtime-filters-memory=1.00MB
> PLAN-ROOT SINK
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 02:HASH JOIN [INNER JOIN]
> | hash predicates: b.int_col = a.f2
> | fk/pk conjuncts: assumed fk/pk
> | runtime filters: RF000[bloom] <- a.f2
> | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB
> thread-reservation=0
> | tuple-ids=1,0 row-size=96B cardinality=5.66K
> | in pipelines: 01(GETNEXT), 00(OPEN)
> |
> |--00:SCAN HDFS [functional_text_lzo.emptytable a]
> | partitions=0/0 files=0 size=0B
> | stored statistics:
> | table: rows=unavailable size=unavailable
> | partitions: 0/0 rows=unavailable
> | columns missing stats: field
> | extrapolated-rows=disabled max-scan-range-rows=0
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> | tuple-ids=0 row-size=16B cardinality=0
> | in pipelines: 00(GETNEXT)
> |
> 01:SCAN HDFS [functional_text_lzo.alltypes b]
> HDFS partitions=24/24 files=24 size=123.54KB
> runtime filters: RF000[bloom] -> b.int_col
> stored statistics:
>table: rows=unavailable size=unavailable
>partitions: 0/24 rows=unavailable
>columns missing stats: int_col, id, bool_col, tinyint_col,
> smallint_col, bigint_col, float_col, double_col, date_string_col, string_col,
> timestamp_col
> extrapolated-rows=disabled max-scan-range-rows=unavailable
> mem-estimate=128.00MB mem-reservation=8.00KB thread-reservation=1
> tuple-ids=1 row-size=80B cardinality=5.66K
> in pipelines: 01(GETNEXT)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8702) PlannerTest.testJoins flakiness due to reliance on an exact cardinality estimate
[
https://issues.apache.org/jira/browse/IMPALA-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fang-Yu Rao resolved IMPALA-8702.
-
Resolution: Fixed
> PlannerTest.testJoins flakiness due to reliance on an exact cardinality
> estimate
>
>
> Key: IMPALA-8702
> URL: https://issues.apache.org/jira/browse/IMPALA-8702
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Bikramjeet Vig
>Assignee: Fang-Yu Rao
>Priority: Critical
> Labels: broken-build
>
> The following test case is failing in PlannerTest.testJoins.
>
> {noformat}
> Section PLAN of query:
> select * from functional_text_lzo.emptytable a inner join
> functional_text_lzo.alltypes b on a.f2 = b.int_col
> Actual does not match expected result:
> PLAN-ROOT SINK
> |
> 02:HASH JOIN [INNER JOIN]
> | hash predicates: b.int_col = a.f2
> | runtime filters: RF000 <- a.f2
> | row-size=96B cardinality=5.66K
> ^
> |
> |--00:SCAN HDFS [functional_text_lzo.emptytable a]
> | partitions=0/0 files=0 size=0B
> | row-size=16B cardinality=0
> |
> 01:SCAN HDFS [functional_text_lzo.alltypes b]
>HDFS partitions=24/24 files=24 size=123.54KB
>runtime filters: RF000 -> b.int_col
>row-size=80B cardinality=5.66K
> Expected:
> PLAN-ROOT SINK
> |
> 02:HASH JOIN [INNER JOIN]
> | hash predicates: b.int_col = a.f2
> | runtime filters: RF000 <- a.f2
> | row-size=96B cardinality=5.65K
> |
> |--00:SCAN HDFS [functional_text_lzo.emptytable a]
> | partitions=0/0 files=0 size=0B
> | row-size=16B cardinality=0
> |
> 01:SCAN HDFS [functional_text_lzo.alltypes b]
>HDFS partitions=24/24 files=24 size=123.32KB
>runtime filters: RF000 -> b.int_col
>row-size=80B cardinality=5.65K
> Verbose plan:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> Per-Host Resources: mem-estimate=130.94MB mem-reservation=2.95MB
> thread-reservation=2 runtime-filters-memory=1.00MB
> PLAN-ROOT SINK
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 02:HASH JOIN [INNER JOIN]
> | hash predicates: b.int_col = a.f2
> | fk/pk conjuncts: assumed fk/pk
> | runtime filters: RF000[bloom] <- a.f2
> | mem-estimate=1.94MB mem-reservation=1.94MB spill-buffer=64.00KB
> thread-reservation=0
> | tuple-ids=1,0 row-size=96B cardinality=5.66K
> | in pipelines: 01(GETNEXT), 00(OPEN)
> |
> |--00:SCAN HDFS [functional_text_lzo.emptytable a]
> | partitions=0/0 files=0 size=0B
> | stored statistics:
> | table: rows=unavailable size=unavailable
> | partitions: 0/0 rows=unavailable
> | columns missing stats: field
> | extrapolated-rows=disabled max-scan-range-rows=0
> | mem-estimate=0B mem-reservation=0B thread-reservation=0
> | tuple-ids=0 row-size=16B cardinality=0
> | in pipelines: 00(GETNEXT)
> |
> 01:SCAN HDFS [functional_text_lzo.alltypes b]
> HDFS partitions=24/24 files=24 size=123.54KB
> runtime filters: RF000[bloom] -> b.int_col
> stored statistics:
>table: rows=unavailable size=unavailable
>partitions: 0/24 rows=unavailable
>columns missing stats: int_col, id, bool_col, tinyint_col,
> smallint_col, bigint_col, float_col, double_col, date_string_col, string_col,
> timestamp_col
> extrapolated-rows=disabled max-scan-range-rows=unavailable
> mem-estimate=128.00MB mem-reservation=8.00KB thread-reservation=1
> tuple-ids=1 row-size=80B cardinality=5.66K
> in pipelines: 01(GETNEXT)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Closed] (IMPALA-8729) Impala Doc: Describe how to enable Catalog V2
[ https://issues.apache.org/jira/browse/IMPALA-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni closed IMPALA-8729. --- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Impala Doc: Describe how to enable Catalog V2 > - > > Key: IMPALA-8729 > URL: https://issues.apache.org/jira/browse/IMPALA-8729 > Project: IMPALA > Issue Type: Task > Components: Docs >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > Labels: future_release_doc, in_33 > Fix For: Impala 3.3.0 > > > https://gerrit.cloudera.org/#/c/13802/ -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Closed] (IMPALA-8729) Impala Doc: Describe how to enable Catalog V2
[ https://issues.apache.org/jira/browse/IMPALA-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni closed IMPALA-8729. --- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Impala Doc: Describe how to enable Catalog V2 > - > > Key: IMPALA-8729 > URL: https://issues.apache.org/jira/browse/IMPALA-8729 > Project: IMPALA > Issue Type: Task > Components: Docs >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > Labels: future_release_doc, in_33 > Fix For: Impala 3.3.0 > > > https://gerrit.cloudera.org/#/c/13802/ -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Resolved] (IMPALA-8676) run-all-tests-timeout-check.sh times out after 20 hours
[
https://issues.apache.org/jira/browse/IMPALA-8676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell resolved IMPALA-8676.
---
Resolution: Won't Fix
This is not an upstream issue, and there are multiple ways to workaround this
without upstream changes. Closing.
> run-all-tests-timeout-check.sh times out after 20 hours
>
>
> Key: IMPALA-8676
> URL: https://issues.apache.org/jira/browse/IMPALA-8676
> Project: IMPALA
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: David Knupp
>Priority: Blocker
> Labels: broken-build
>
> An exhaustive test run on Centos6 now takes 22 hours.
> run-all-tests-timeout-check.sh will time out after 20 hours.
> {code}
> 15:39:30 Timout Timer Started (pid 20144, ppid 20063) for 72000 s!
> {code}
> {code}
> 11:39:30 Tests TIMED OUT!
> {code}
> Some tests take a long time, we could try to fix that.
> Or we could bump ${TIMEOUT_FOR_RUN_ALL_TESTS_MINS} which is currently 1200
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8676) run-all-tests-timeout-check.sh times out after 20 hours
[
https://issues.apache.org/jira/browse/IMPALA-8676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell resolved IMPALA-8676.
---
Resolution: Won't Fix
This is not an upstream issue, and there are multiple ways to workaround this
without upstream changes. Closing.
> run-all-tests-timeout-check.sh times out after 20 hours
>
>
> Key: IMPALA-8676
> URL: https://issues.apache.org/jira/browse/IMPALA-8676
> Project: IMPALA
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: David Knupp
>Priority: Blocker
> Labels: broken-build
>
> An exhaustive test run on Centos6 now takes 22 hours.
> run-all-tests-timeout-check.sh will time out after 20 hours.
> {code}
> 15:39:30 Timout Timer Started (pid 20144, ppid 20063) for 72000 s!
> {code}
> {code}
> 11:39:30 Tests TIMED OUT!
> {code}
> Some tests take a long time, we could try to fix that.
> Or we could bump ${TIMEOUT_FOR_RUN_ALL_TESTS_MINS} which is currently 1200
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IMPALA-8676) run-all-tests-timeout-check.sh times out after 20 hours
[
https://issues.apache.org/jira/browse/IMPALA-8676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883369#comment-16883369
]
Joe McDonnell commented on IMPALA-8676:
---
This Jira is an artifact of particular Cloudera testing infrastructure and is
not a general upstream issue. The way that this was fixed for Cloudera is to
shard the tests into multiple jobs. We run FE_TEST, BE_TEST, JDBC_TEST, and
CLUSTER_TEST separately from EE_TEST if the machines are slow. An alternative
fix is to set the TIMEOUT_FOR_RUN_ALL_TESTS_MINS environment variable to a
higher value.
Closing this.
> run-all-tests-timeout-check.sh times out after 20 hours
>
>
> Key: IMPALA-8676
> URL: https://issues.apache.org/jira/browse/IMPALA-8676
> Project: IMPALA
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: David Knupp
>Priority: Blocker
> Labels: broken-build
>
> An exhaustive test run on Centos6 now takes 22 hours.
> run-all-tests-timeout-check.sh will time out after 20 hours.
> {code}
> 15:39:30 Timout Timer Started (pid 20144, ppid 20063) for 72000 s!
> {code}
> {code}
> 11:39:30 Tests TIMED OUT!
> {code}
> Some tests take a long time, we could try to fix that.
> Or we could bump ${TIMEOUT_FOR_RUN_ALL_TESTS_MINS} which is currently 1200
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8730) Multiple test failures in S3 due to incorrect loading of functional.alltypes
[
https://issues.apache.org/jira/browse/IMPALA-8730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell resolved IMPALA-8730.
---
Resolution: Cannot Reproduce
Fix Version/s: Impala 3.3.0
I think this was a temporary thing related to the snapshot. It is not happening
anymore, so I'm closing this.
> Multiple test failures in S3 due to incorrect loading of functional.alltypes
>
>
> Key: IMPALA-8730
> URL: https://issues.apache.org/jira/browse/IMPALA-8730
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Anurag Mantripragada
>Assignee: Fang-Yu Rao
>Priority: Critical
> Labels: broken-build
> Fix For: Impala 3.3.0
>
>
> Multiple tests involving functional.alltypes failed. Most likely due to data
> loading issue on S3. Most are assertion failures because of row_count
> mismatch.
>
> Below is the test failures:
> {code:java}
> 1.
> metadata.test_metadata_query_statements.TestMetadataQueryStatements.test_show_stats
>
> 2.
> metadata.test_explain.TestExplain.test_explain_validate_cardinality_estimates
> 3. query_test.test_insert.TestInsertQueries.test_insert
> 4. query_test.test_runtime_filters.TestRuntimeFilters.test_basic_filters{code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Resolved] (IMPALA-8730) Multiple test failures in S3 due to incorrect loading of functional.alltypes
[
https://issues.apache.org/jira/browse/IMPALA-8730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell resolved IMPALA-8730.
---
Resolution: Cannot Reproduce
Fix Version/s: Impala 3.3.0
I think this was a temporary thing related to the snapshot. It is not happening
anymore, so I'm closing this.
> Multiple test failures in S3 due to incorrect loading of functional.alltypes
>
>
> Key: IMPALA-8730
> URL: https://issues.apache.org/jira/browse/IMPALA-8730
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.3.0
>Reporter: Anurag Mantripragada
>Assignee: Fang-Yu Rao
>Priority: Critical
> Labels: broken-build
> Fix For: Impala 3.3.0
>
>
> Multiple tests involving functional.alltypes failed. Most likely due to data
> loading issue on S3. Most are assertion failures because of row_count
> mismatch.
>
> Below is the test failures:
> {code:java}
> 1.
> metadata.test_metadata_query_statements.TestMetadataQueryStatements.test_show_stats
>
> 2.
> metadata.test_explain.TestExplain.test_explain_validate_cardinality_estimates
> 3. query_test.test_insert.TestInsertQueries.test_insert
> 4. query_test.test_runtime_filters.TestRuntimeFilters.test_basic_filters{code}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-5872) Implement a SQL test case builder for gathering query diagnostics
[
https://issues.apache.org/jira/browse/IMPALA-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883353#comment-16883353
]
Jiawei Wang commented on IMPALA-5872:
-
nit:
commend for this should be:
SET PLANNER_TESTCASE_MODE=true
instead of
SET PLANNER_DEBUG_MODE=true
> Implement a SQL test case builder for gathering query diagnostics
> -
>
> Key: IMPALA-5872
> URL: https://issues.apache.org/jira/browse/IMPALA-5872
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
>Reporter: bharath v
>Assignee: bharath v
>Priority: Major
> Labels: supportability
> Fix For: Impala 3.2.0
>
>
> The idea is to implement a test case builder for collecting enough
> diagnostics for a given SQL query so that it can be reproduced/recreated on
> another cluster easily.
> Input: A valid SQL query
> Expected output: A .sql file that can be run using {{impala-shell -f}} and
> has all the schema/table/view definitions required to run the query on a
> target cluster.
> An example:
> {noformat}
> EXPORT TESTCASE INTO OUTFILE 'file:///tmp/repro.sql' select * from view
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8753) JdbcTest's testConcurrentSessionMixedIdleTimeout() flaky
Joe McDonnell created IMPALA-8753:
-
Summary: JdbcTest's testConcurrentSessionMixedIdleTimeout() flaky
Key: IMPALA-8753
URL: https://issues.apache.org/jira/browse/IMPALA-8753
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 3.3.0
Reporter: Joe McDonnell
An exhaustive run on Centos6 hit a failure in
JdbcTest::testConcurrentSessionMixedIdleTimeout():
{noformat}
02:29:36 [ERROR] Tests run: 26, Failures: 1, Errors: 0, Skipped: 0, Time
elapsed: 69.083 s <<< FAILURE! - in org.apache.impala.service.JdbcTest
02:29:36 [ERROR]
testConcurrentSessionMixedIdleTimeout[0](org.apache.impala.service.JdbcTest)
Time elapsed: 13.222 s <<< FAILURE!
02:29:36at
org.apache.impala.service.JdbcTest.testConcurrentSessionMixedIdleTimeout(JdbcTest.java:581)
02:29:36at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
...
02:29:36 [ERROR] Failures:
02:29:36 [ERROR] JdbcTest.testConcurrentSessionMixedIdleTimeout:581
expected:<0> but was:<1>
02:29:36 [ERROR] Tests run: 970, Failures: 1, Errors: 0, Skipped: 45{noformat}
The faliure didn't produce much output (no JUnitXML, for example).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8753) JdbcTest's testConcurrentSessionMixedIdleTimeout() flaky
Joe McDonnell created IMPALA-8753:
-
Summary: JdbcTest's testConcurrentSessionMixedIdleTimeout() flaky
Key: IMPALA-8753
URL: https://issues.apache.org/jira/browse/IMPALA-8753
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 3.3.0
Reporter: Joe McDonnell
An exhaustive run on Centos6 hit a failure in
JdbcTest::testConcurrentSessionMixedIdleTimeout():
{noformat}
02:29:36 [ERROR] Tests run: 26, Failures: 1, Errors: 0, Skipped: 0, Time
elapsed: 69.083 s <<< FAILURE! - in org.apache.impala.service.JdbcTest
02:29:36 [ERROR]
testConcurrentSessionMixedIdleTimeout[0](org.apache.impala.service.JdbcTest)
Time elapsed: 13.222 s <<< FAILURE!
02:29:36at
org.apache.impala.service.JdbcTest.testConcurrentSessionMixedIdleTimeout(JdbcTest.java:581)
02:29:36at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
...
02:29:36 [ERROR] Failures:
02:29:36 [ERROR] JdbcTest.testConcurrentSessionMixedIdleTimeout:581
expected:<0> but was:<1>
02:29:36 [ERROR] Tests run: 970, Failures: 1, Errors: 0, Skipped: 45{noformat}
The faliure didn't produce much output (no JUnitXML, for example).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IMPALA-8729) Impala Doc: Describe how to enable Catalog V2
[ https://issues.apache.org/jira/browse/IMPALA-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883344#comment-16883344 ] ASF subversion and git services commented on IMPALA-8729: - Commit 8081045bebb253698bcd748a64ccff843e3e85ff in impala's branch refs/heads/master from Alex Rodoni [ https://gitbox.apache.org/repos/asf?p=impala.git;h=8081045 ] IMPALA-8729: [DOCS] Describe on-demand metadata feature - Overview of on-demand metadata. - Config flags to enable/disable on-demand metadata. Change-Id: I64261625c1d9b122c7cca59f9b004dda05810351 Reviewed-on: http://gerrit.cloudera.org:8080/13802 Tested-by: Impala Public Jenkins Reviewed-by: Bharath Vissapragada > Impala Doc: Describe how to enable Catalog V2 > - > > Key: IMPALA-8729 > URL: https://issues.apache.org/jira/browse/IMPALA-8729 > Project: IMPALA > Issue Type: Task > Components: Docs >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > Labels: future_release_doc, in_33 > > https://gerrit.cloudera.org/#/c/13802/ -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8543) Log more diagnostics information in TAcceptQueueServer
[
https://issues.apache.org/jira/browse/IMPALA-8543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883343#comment-16883343
]
ASF subversion and git services commented on IMPALA-8543:
-
Commit e83f67e4e3ad24711703224c8efc42e291a0d89c in impala's branch
refs/heads/master from Michael Ho
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=e83f67e ]
IMPALA-8543: More diagnostics for TAcceptQueueServer
This change adds more logging information in TAcceptQueueServer
to help diagnose issues at various stages of client connections
establishment.
Two new metrics are also added to measure the connection setup time
and the wait time for service threads to be available.
Change-Id: I33b32352b457a2c8ec7bae6da46bb9c555dc9c36
Reviewed-on: http://gerrit.cloudera.org:8080/13790
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> Log more diagnostics information in TAcceptQueueServer
> --
>
> Key: IMPALA-8543
> URL: https://issues.apache.org/jira/browse/IMPALA-8543
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Affects Versions: Impala 2.12.0, Impala 3.1.0, Impala 3.2.0
>Reporter: Michael Ho
>Assignee: Michael Ho
>Priority: Major
> Labels: ramp-up
>
> There is currently not much diagnostic information in
> {{TAcceptQueueServer.cpp}}. It would be nice to add some additional logging
> to identify clients which misbehaved or took longer than expected when
> setting up connections.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8752) Add Jaro-winkler edit distance and similarity built-in function
Norbert Luksa created IMPALA-8752: - Summary: Add Jaro-winkler edit distance and similarity built-in function Key: IMPALA-8752 URL: https://issues.apache.org/jira/browse/IMPALA-8752 Project: IMPALA Issue Type: New Feature Reporter: Norbert Luksa Assignee: Norbert Luksa References: * [Apache commons - JaroWinklerDistance |[https://commons.apache.org/proper/commons-text/apidocs/org/apache/commons/text/similarity/JaroWinklerDistance.html]] * [Apache commons - JaroWinklerSimilarity |[https://commons.apache.org/proper/commons-text/apidocs/org/apache/commons/text/similarity/JaroWinklerSimilarity.html]] * [Oracle - JARO_WINKLER[_SIMILARITY]|[https://oracle-base.com/articles/11g/utl_match-string-matching-in-oracle]] Notable difference: * With similarity, the Oracle version returns a normalized result ranging from 0 to 100. * In the Appache version, null values result in exceptions. * Apache rounds the values to two digits The scaling factor of the algorithm can be added as an extra/default argument. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8752) Add Jaro-winkler edit distance and similarity built-in function
Norbert Luksa created IMPALA-8752: - Summary: Add Jaro-winkler edit distance and similarity built-in function Key: IMPALA-8752 URL: https://issues.apache.org/jira/browse/IMPALA-8752 Project: IMPALA Issue Type: New Feature Reporter: Norbert Luksa Assignee: Norbert Luksa References: * [Apache commons - JaroWinklerDistance |[https://commons.apache.org/proper/commons-text/apidocs/org/apache/commons/text/similarity/JaroWinklerDistance.html]] * [Apache commons - JaroWinklerSimilarity |[https://commons.apache.org/proper/commons-text/apidocs/org/apache/commons/text/similarity/JaroWinklerSimilarity.html]] * [Oracle - JARO_WINKLER[_SIMILARITY]|[https://oracle-base.com/articles/11g/utl_match-string-matching-in-oracle]] Notable difference: * With similarity, the Oracle version returns a normalized result ranging from 0 to 100. * In the Appache version, null values result in exceptions. * Apache rounds the values to two digits The scaling factor of the algorithm can be added as an extra/default argument. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (IMPALA-8600) Reload partition does not work for transactional tables
[ https://issues.apache.org/jira/browse/IMPALA-8600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Kaszab reassigned IMPALA-8600: Assignee: Gabor Kaszab (was: Vihang Karajgaonkar) > Reload partition does not work for transactional tables > --- > > Key: IMPALA-8600 > URL: https://issues.apache.org/jira/browse/IMPALA-8600 > Project: IMPALA > Issue Type: Bug >Reporter: Vihang Karajgaonkar >Assignee: Gabor Kaszab >Priority: Major > Labels: impala-acid > > If a table is transactional, a reload partition call should fetch the valid > writeIds. Without doing this, the reload will skip adding all the newly > created delta files of the transactional table pertaining to the new writeIds. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8751) Kudu tables cannot be found after created
Yongzhi Chen created IMPALA-8751:
Summary: Kudu tables cannot be found after created
Key: IMPALA-8751
URL: https://issues.apache.org/jira/browse/IMPALA-8751
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 3.3.0
Reporter: Yongzhi Chen
For example in some kudu integration test as:
TestKuduHMSIntegration.test_drop_db_cascade in custom_cluster/test_kudu.py
It failed with:
custom_cluster/test_kudu.py:239: in test_drop_db_cascade
assert not kudu_client.table_exists(kudu_table.name)
/usr/lib/python2.7/contextlib.py:35: in __exit__
self.gen.throw(type, value, traceback)
common/kudu_test_suite.py:165: in temp_kudu_table
kudu.delete_table(name)
kudu/client.pyx:392: in kudu.client.Client.delete_table (kudu/client.cpp:7106)
???
kudu/errors.pyx:56: in kudu.errors.check_status (kudu/errors.cpp:904)
???
E KuduNotFound: failed to drop Hive Metastore table: TException - service has
thrown: NoSuchObjectException(message=s7mo1z)
And when trying to add default capabilities to kudu tables, it is sometime
effective, sometimes not:
For example after enable default kudu OBJCAPABILITIES,
./run-tests.py metadata/test_ddl.py -k "create_kudu" will succeed while same
test in
./run-tests.py custom_cluster/test_kudu.py :
{noformat}
TestKuduHMSIntegration.test_create_managed_kudu_tables[protocol: beeswax |
exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
text/none]
custom_cluster/test_kudu.py:147: in test_create_managed_kudu_tables
self.run_test_case('QueryTest/kudu_create', vector, use_db=unique_database)
common/impala_test_suite.py:563: in run_test_case
result = exec_fn(query, user=test_section.get('USER', '').strip() or None)
common/impala_test_suite.py:500: in __exec_in_impala
result = self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:798: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:184: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:187: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:362: in __execute_query
handle = self.execute_query_async(query_string, user=user)
beeswax/impala_beeswax.py:356: in execute_query_async
handle = self.__do_rpc(lambda: self.imp_service.query(query,))
beeswax/impala_beeswax.py:519: in __do_rpc
raise ImpalaBeeswaxException(self.__build_error_message(b), b)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
EINNER EXCEPTION:
EMESSAGE: AnalysisException: Write not supported. Table
test_create_managed_kudu_tables_a8d11828.add access type is: READONLY
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (IMPALA-8751) Kudu tables cannot be found after created
Yongzhi Chen created IMPALA-8751:
Summary: Kudu tables cannot be found after created
Key: IMPALA-8751
URL: https://issues.apache.org/jira/browse/IMPALA-8751
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 3.3.0
Reporter: Yongzhi Chen
For example in some kudu integration test as:
TestKuduHMSIntegration.test_drop_db_cascade in custom_cluster/test_kudu.py
It failed with:
custom_cluster/test_kudu.py:239: in test_drop_db_cascade
assert not kudu_client.table_exists(kudu_table.name)
/usr/lib/python2.7/contextlib.py:35: in __exit__
self.gen.throw(type, value, traceback)
common/kudu_test_suite.py:165: in temp_kudu_table
kudu.delete_table(name)
kudu/client.pyx:392: in kudu.client.Client.delete_table (kudu/client.cpp:7106)
???
kudu/errors.pyx:56: in kudu.errors.check_status (kudu/errors.cpp:904)
???
E KuduNotFound: failed to drop Hive Metastore table: TException - service has
thrown: NoSuchObjectException(message=s7mo1z)
And when trying to add default capabilities to kudu tables, it is sometime
effective, sometimes not:
For example after enable default kudu OBJCAPABILITIES,
./run-tests.py metadata/test_ddl.py -k "create_kudu" will succeed while same
test in
./run-tests.py custom_cluster/test_kudu.py :
{noformat}
TestKuduHMSIntegration.test_create_managed_kudu_tables[protocol: beeswax |
exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
text/none]
custom_cluster/test_kudu.py:147: in test_create_managed_kudu_tables
self.run_test_case('QueryTest/kudu_create', vector, use_db=unique_database)
common/impala_test_suite.py:563: in run_test_case
result = exec_fn(query, user=test_section.get('USER', '').strip() or None)
common/impala_test_suite.py:500: in __exec_in_impala
result = self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:798: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:184: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:187: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:362: in __execute_query
handle = self.execute_query_async(query_string, user=user)
beeswax/impala_beeswax.py:356: in execute_query_async
handle = self.__do_rpc(lambda: self.imp_service.query(query,))
beeswax/impala_beeswax.py:519: in __do_rpc
raise ImpalaBeeswaxException(self.__build_error_message(b), b)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
EINNER EXCEPTION:
EMESSAGE: AnalysisException: Write not supported. Table
test_create_managed_kudu_tables_a8d11828.add access type is: READONLY
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8750) TestObservability.test_query_profile_contains_query_compilation_events failing when compilation events appear dynamically
[
https://issues.apache.org/jira/browse/IMPALA-8750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-8750 started by Tamas Mate.
--
> TestObservability.test_query_profile_contains_query_compilation_events
> failing when compilation events appear dynamically
> -
>
> Key: IMPALA-8750
> URL: https://issues.apache.org/jira/browse/IMPALA-8750
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Tamas Mate
>Assignee: Tamas Mate
>Priority: Major
> Labels: test
>
> {{TestObservability.test_query_profile_contains_query_compilation_events}} is
> failing during builds where the query compilation event differs from the
> specified. This can occur in 2 cases:
> - When {{lineage_event_log_dir}} is specified it adds an extra "Lineage info
> computed:" line.
> - When metadata is not preloaded on the coordinator instead of "Metadata of
> all * tables cached:" it can be multiple lines:
> {noformat}
> Metadata load started:
> Metadata load finished:
> {noformat}
> We should cover these cases as well.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-8750) TestObservability.test_query_profile_contains_query_compilation_events failing when compilation events appear dynamically
[
https://issues.apache.org/jira/browse/IMPALA-8750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tamas Mate reassigned IMPALA-8750:
--
Assignee: Tamas Mate
> TestObservability.test_query_profile_contains_query_compilation_events
> failing when compilation events appear dynamically
> -
>
> Key: IMPALA-8750
> URL: https://issues.apache.org/jira/browse/IMPALA-8750
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Tamas Mate
>Assignee: Tamas Mate
>Priority: Major
> Labels: test
>
> {{TestObservability.test_query_profile_contains_query_compilation_events}} is
> failing during builds where the query compilation event differs from the
> specified. This can occur in 2 cases:
> - When {{lineage_event_log_dir}} is specified it adds an extra "Lineage info
> computed:" line.
> - When metadata is not preloaded on the coordinator instead of "Metadata of
> all * tables cached:" it can be multiple lines:
> {noformat}
> Metadata load started:
> Metadata load finished:
> {noformat}
> We should cover these cases as well.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8750) TestObservability.test_query_profile_contains_query_compilation_events failing when compilation events appear dynamically
Tamas Mate created IMPALA-8750:
--
Summary:
TestObservability.test_query_profile_contains_query_compilation_events failing
when compilation events appear dynamically
Key: IMPALA-8750
URL: https://issues.apache.org/jira/browse/IMPALA-8750
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 3.3.0
Reporter: Tamas Mate
{{TestObservability.test_query_profile_contains_query_compilation_events}} is
failing during builds where the query compilation event differs from the
specified. This can occur in 2 cases:
- When {{lineage_event_log_dir}} is specified it adds an extra "Lineage info
computed:" line.
- When metadata is not preloaded on the coordinator instead of "Metadata of
all * tables cached:" it can be multiple lines:
{noformat}
Metadata load started:
Metadata load finished:
{noformat}
We should cover these cases as well.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (IMPALA-8750) TestObservability.test_query_profile_contains_query_compilation_events failing when compilation events appear dynamically
Tamas Mate created IMPALA-8750:
--
Summary:
TestObservability.test_query_profile_contains_query_compilation_events failing
when compilation events appear dynamically
Key: IMPALA-8750
URL: https://issues.apache.org/jira/browse/IMPALA-8750
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 3.3.0
Reporter: Tamas Mate
{{TestObservability.test_query_profile_contains_query_compilation_events}} is
failing during builds where the query compilation event differs from the
specified. This can occur in 2 cases:
- When {{lineage_event_log_dir}} is specified it adds an extra "Lineage info
computed:" line.
- When metadata is not preloaded on the coordinator instead of "Metadata of
all * tables cached:" it can be multiple lines:
{noformat}
Metadata load started:
Metadata load finished:
{noformat}
We should cover these cases as well.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
