[
https://issues.apache.org/jira/browse/IMPALA-8121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16837370#comment-16837370
]
ASF subversion and git services commented on IMPALA-8121:
---------------------------------------------------------
Commit a2c5d953b0fa6d69aa34eccaa13fa9aacd31ad10 in impala's branch
refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=a2c5d95 ]
IMPALA-8121: part 2: use local catalog in containers
This enables "modern" catalog features including the
local catalog and HMS notification support in the
dockerised minicluster by default.
The flags can be overridden if needed.
Skip tests affected by these bugs:
* IMPALA-8486 (LibCache invalidations)
* IMPALA-8458 (alter column stats)
* IMPALA-7131 (data sources not supported)
* IMPALA-7538 (HDFS caching DDL not supported)
* IMPALA-8489 TestRecoverPartitions.test_post_invalidate fails with
IllegalStateException
* IMPALA-8459 (cannot drop Kudu table)
* IMPALA-7539 (insert permission checks)
Fix handling of table properties in _get_properties()
to avoid including properties from unrelated sections.
This caused problems becase of additional properties
added by metastore event processing.
Rewrite test_partition_ddl_predicates() to change file formats rather
than use HDFS caching DDL.
Update the various test_kudu_col* tests to not expect staleness of
Kudu metadata for catalog V2.
Fix IMPALA-8464 so that testMetaDataGetColumnComments() allows the
table comment to be present, which is the new behaviour. Add a
new end-to-end test test_get_tables() that tests the precise
behaviour for different catalog versions so as to not lose
coverage.
Change-Id: I900d4b718cca98bcf86d36a2e64c0b6a424a5b7c
Reviewed-on: http://gerrit.cloudera.org:8080/13226
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Pick better default flags in containers
> ---------------------------------------
>
> Key: IMPALA-8121
> URL: https://issues.apache.org/jira/browse/IMPALA-8121
> Project: IMPALA
> Issue Type: Sub-task
> Components: Infrastructure
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Labels: docker
>
> There are some new features of Impala that are done but disabled by default
> because they are not strictly better than the previous versions. E.g. the
> various metadata improvements. Containerised Impala is likely to be new
> deployments, so it is easier to make potentially disruptive changes to
> defaults now.
> h2. Metadata V2 Flags
> Catalogd:
> --catalog_topic_mode=minimal
> Impalad:
> --use_local_catalog=true
> I think we *may* also want to configure automatic invalidations of tables
> from the catalogd so that changes to the underlying storage cluster are
> eventually reflected in the compute cluster. There's a better solution in the
> pipeline that uses HMS notifications
> (https://issues.apache.org/jira/browse/IMPALA-7970), but in the meantime
> invalidation is time-based.
> Catalogd:
> --invalidate_tables_timeout_s=3600
> Once IMPALA-7970 goes in, we probably also want automatic invalidation by
> default (TBD - how to handle older HMS that doesn't support those APIs).
> Catalogd:
> --hms_event_polling_interval_s=???
> We probably want to enable HDFS preads for remote reads: -use_hdfs_pread
> We may want to have an I/O cache enabled
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
