[
https://issues.apache.org/jira/browse/IMPALA-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16832086#comment-16832086
]
ASF subversion and git services commented on IMPALA-8458:
---------------------------------------------------------
Commit 79c5f87565467074697a7d98e01c9742f7228991 in impala's branch
refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=79c5f87 ]
IMPALA-8121: part 1: some test fixes for catalog v2
This fixes some test issues encountered when running the
tests against a cluster with catalog V2 enabled, meaning
the local catalog with HMS notifications enabled. More
fixes are to come but I preferred to do them in smaller
batches as they're ready.
Test fixes:
* Detect whether catalog v2 features are enabled from web UI.
* test_describe_db waits for metadata event processor to pick up new
database and doesn't need to change database owner
* TestWebPage.test_catalog handles an expected exception from
the /catalog_objects page on the impalad.
* test_pull_stats_profile: feature disabled with local catalog
* test_hms_service_dies: invalidate the test table instead of
the whole catalog.
* test_compute_stats: Avro schema resolution behaviour changed
with local catalog - IMPALA-7308
Some remaining issues:
* IMPALA-8458
* IMPALA-8459
* IMPALA-7131 (data sources)
* getTables() doesn't return comment
Change-Id: I060f2076da74fbbe92ae26dbad51f09a3bd20169
Reviewed-on: http://gerrit.cloudera.org:8080/13122
Reviewed-by: Todd Lipcon <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Can't set numNull/maxSize/avgSize column stats with local catalog without
> also setting NDV
> ------------------------------------------------------------------------------------------
>
> Key: IMPALA-8458
> URL: https://issues.apache.org/jira/browse/IMPALA-8458
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 3.3.0
> Reporter: Tim Armstrong
> Assignee: Todd Lipcon
> Priority: Critical
>
> Repro:
> {noformat}
> [tarmstrong-box2.ca.cloudera.com:21000] default> create table test_stats2(s
> string);
> +-------------------------+
> | summary |
> +-------------------------+
> | Table has been created. |
> +-------------------------+
> Fetched 1 row(s) in 0.36s
> [tarmstrong-box2.ca.cloudera.com:21000] default> show column stats
> test_stats2;
> +--------+--------+------------------+--------+----------+----------+
> | Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size |
> +--------+--------+------------------+--------+----------+----------+
> | s | STRING | -1 | -1 | -1 | -1 |
> +--------+--------+------------------+--------+----------+----------+
> Fetched 1 row(s) in 0.02s
> [tarmstrong-box2.ca.cloudera.com:21000] default> alter table test_stats2 set
> column stats s('avgSize'='1234');
> +-----------------------------------------+
> | summary |
> +-----------------------------------------+
> | Updated 0 partition(s) and 1 column(s). |
> +-----------------------------------------+
> Fetched 1 row(s) in 0.14s
> [tarmstrong-box2.ca.cloudera.com:21000] default> show column stats
> test_stats2;
> +--------+--------+------------------+--------+----------+----------+
> | Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size |
> +--------+--------+------------------+--------+----------+----------+
> | s | STRING | -1 | -1 | -1 | -1 |
> +--------+--------+------------------+--------+----------+----------+
> Fetched 1 row(s) in 0.02s
> [tarmstrong-box2.ca.cloudera.com:21000] default> alter table test_stats2 set
> column stats s('maxSize'='1234');
> +-----------------------------------------+
> | summary |
> +-----------------------------------------+
> | Updated 0 partition(s) and 1 column(s). |
> +-----------------------------------------+
> Fetched 1 row(s) in 0.10s
> [tarmstrong-box2.ca.cloudera.com:21000] default> show column stats
> test_stats2;
> +--------+--------+------------------+--------+----------+----------+
> | Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size |
> +--------+--------+------------------+--------+----------+----------+
> | s | STRING | -1 | -1 | -1 | -1 |
> +--------+--------+------------------+--------+----------+----------+
> Fetched 1 row(s) in 0.02s
> [tarmstrong-box2.ca.cloudera.com:21000] default> invalidate metadata
> test_stats2;
> Fetched 0 row(s) in 0.03s
> [tarmstrong-box2.ca.cloudera.com:21000] default> show column stats
> test_stats2;
> Query: show column stats test_stats2
> +--------+--------+------------------+--------+----------+----------+
> | Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size |
> +--------+--------+------------------+--------+----------+----------+
> | s | STRING | -1 | -1 | -1 | -1 |
> +--------+--------+------------------+--------+----------+----------+
> Fetched 1 row(s) in 0.07s
> {noformat}
> I expected that the updates would take effect. Weirdly it doesn't happen for
> NDV and NULLS:
> {noformat}
> [tarmstrong-box2.ca.cloudera.com:21000] default> alter table test_stats2 set
> column stats s('numDVs'='1234','numNulls'='12345');
> Query: alter table test_stats2 set column stats
> s('numDVs'='1234','numNulls'='12345')
> +-----------------------------------------+
> | summary |
> +-----------------------------------------+
> | Updated 0 partition(s) and 1 column(s). |
> +-----------------------------------------+
> Fetched 1 row(s) in 0.12s
> [tarmstrong-box2.ca.cloudera.com:21000] default> show column stats
> test_stats2;
> Query: show column stats test_stats2
> +--------+--------+------------------+--------+----------+----------+
> | Column | Type | #Distinct Values | #Nulls | Max Size | Avg Size |
> +--------+--------+------------------+--------+----------+----------+
> | s | STRING | 1234 | 12345 | -1 | -1 |
> +--------+--------+------------------+--------+----------+----------+
> Fetched 1 row(s) in 0.02s
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
