Quanlong Huang created IMPALA-9779:
--------------------------------------
Summary: Unnecessarily reloading file metadata in some DDLs
Key: IMPALA-9779
URL: https://issues.apache.org/jira/browse/IMPALA-9779
Project: IMPALA
Issue Type: Improvement
Components: Catalog
Affects Versions: Impala 3.4.0, Impala 3.3.0, Impala 3.2.0, Impala 3.1.0,
Impala 2.12.0, Impala 3.0, Impala 2.11.0, Impala 2.10.0, Impala 2.9.0, Impala
2.8.0
Reporter: Quanlong Huang
Some DDLs are not modifying the actual table data. We don't need to reload file
meta for them. These DDLs include:
* Compute (incremental) stats
* Drop stats
* Alter table set row format
* Alter table set file format
Code paths of them both call CatalogOpExecutor.bulkAlterPartitions(). The
related partitions are marked as "dirty" anyway. Dirty partitions will be
dropped and reloaded at the end of
CatalogOpExecutor.alterTable(TAlterTableParams, TDdlExecResponse). See the
details in HdfsTable.updatePartitionsFromHms().
We can consider not marking related partitions as "dirty" in these DDLs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
