[
https://issues.apache.org/jira/browse/HIVE-12052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-12052:
------------------------------------
Description:
As discussed in HIVE-11500
Should use a table property similar to auto.purge.
Then, when this setting is set, partitions are added, after compactions, after
load/non-ACID insert, and periodically (configurable), the storage locations
should be scanned for new files and cache updated accordingly. All the updates
should probably be in the background thread and taken from queue (high pri from
most ops, low pri from enabling the property and from periodic updates) to
avoid high load on HDFS from metastore.
was:
As discussed in HIVE-11500
Should use a table property similar to auto.purge.
Then, when this setting is set, partitions are added, after compactions, after
load/non-ACID insert, and periodically (configurable), the storage locations
should be scanned for new files and cache updated accordingly. All the updates
should probably be in the background thread and taken from queue (high pri from
ops, low pri from periodic updates) to avoid high load on HDFS from metastore.
> automatically populate file metadata to HBase metastore based on config or
> table properties
> -------------------------------------------------------------------------------------------
>
> Key: HIVE-12052
> URL: https://issues.apache.org/jira/browse/HIVE-12052
> Project: Hive
> Issue Type: Bug
> Reporter: Sergey Shelukhin
>
> As discussed in HIVE-11500
> Should use a table property similar to auto.purge.
> Then, when this setting is set, partitions are added, after compactions,
> after load/non-ACID insert, and periodically (configurable), the storage
> locations should be scanned for new files and cache updated accordingly. All
> the updates should probably be in the background thread and taken from queue
> (high pri from most ops, low pri from enabling the property and from periodic
> updates) to avoid high load on HDFS from metastore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
