[
https://issues.apache.org/jira/browse/HIVE-24403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Varga updated HIVE-24403:
-------------------------------
Description:
In some configurations the HMS backend DB is used by HMS services with
different versions.
HIVE-23107 dropped the min_history_level table from the backend DB making the
new schema version incompatible with the older HMS services.
It is possible to modify that change to keep the compatibility
* Keep the min_history_level table
* Add the new fields for the compaction_queue the same way
* Create a feature flag for min_history_level and if it is on
* Keep the logic inserting to the table during openTxn
* Keep the logic removing the records at commitTxn and abortTxn
* Change the logic in the cleaner, to get the highwatermark the old way
* But still change it to not start the cleaning before that
* The txn_to_write_id table cleaning can work the new way in the new version
and the old way in the old version
* This feature flag can be automatically setup based on the existence of the
min_history level table, this way if the table will be dropped all HMS-s can
switch to the new functionality without restart
was:
In some configurations the HMS backend DB is used by HMS services with
different versions.
HIVE-23107 dropped the min_history_level table from the backend DB making the
new schema version incompatible with the older HMS services.
It is possible to modify that change to keep the compatibility
* Keep the min_history_level table
* Add the new fields for the compaction_queue the same way
* Create a feature flag for min_history_level and if it is on
* Keep the logic inserting to the table during openTxn
* Keep the logic removing the records at commitTxn and abortTxn
* Change the logic in the cleaner, to get the highwatermark the old way
* But still change it to not start the cleaning before that
* Change the logic in AcidHouseKeeper to clean the txn_to_write_id the old way
* This feature flag can be automatically setup based on the existence of the
min_history level table, this way if the table will be dropped all HMS-s can
switch to the new functionality without restart
> change min_history_level schema change to be compatible with previous version
> -----------------------------------------------------------------------------
>
> Key: HIVE-24403
> URL: https://issues.apache.org/jira/browse/HIVE-24403
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Reporter: Peter Varga
> Assignee: Peter Varga
> Priority: Major
>
> In some configurations the HMS backend DB is used by HMS services with
> different versions.
> HIVE-23107 dropped the min_history_level table from the backend DB making
> the new schema version incompatible with the older HMS services.
> It is possible to modify that change to keep the compatibility
> * Keep the min_history_level table
> * Add the new fields for the compaction_queue the same way
> * Create a feature flag for min_history_level and if it is on
> * Keep the logic inserting to the table during openTxn
> * Keep the logic removing the records at commitTxn and abortTxn
> * Change the logic in the cleaner, to get the highwatermark the old way
> * But still change it to not start the cleaning before that
> * The txn_to_write_id table cleaning can work the new way in the new version
> and the old way in the old version
> * This feature flag can be automatically setup based on the existence of the
> min_history level table, this way if the table will be dropped all HMS-s can
> switch to the new functionality without restart
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
