[
https://issues.apache.org/jira/browse/HDDS-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17268690#comment-17268690
]
Ethan Rose commented on HDDS-4724:
----------------------------------
For anyone wondering (like I was) why the existing upgrade acceptance test
passed for HDDS-4369 even though it was backwards incompatible, that test
first writes data using an old version of ozone, so the data is written in
schema version 1. Then the current code reads the data, but it will use schema
1 to do this since that is how it was written, so schema 2 is never used. An
acceptance test that wrote data using schema version 2 (using, for example, the
last commit on master), and then switched to the new code and tried to read it
would have encountered this problem, but currently we do not have such a test.
I can investigate adding this if people think it would be worth while.
> Revert column family names of Datanode db to avoid compatibility issue.
> -----------------------------------------------------------------------
>
> Key: HDDS-4724
> URL: https://issues.apache.org/jira/browse/HDDS-4724
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Datanode
> Affects Versions: 1.0.0
> Reporter: Glen Geng
> Assignee: Glen Geng
> Priority: Major
> Labels: pull-request-available
>
> Tencent monthly deploy the latest master to our production environment.
> During latest upgrade test, we found that new Ozone(version at Jan 14 2021)
> can not read the data written by old Ozone(version at Dec 14 2020).
> We found following traces
> {code:java}
> 2021-01-20 16:36:39,575 INFO
> org.apache.hadoop.ozone.container.common.volume.MutableVolumeSet: Added
> Volume : /data4/hdds/hdds to VolumeSet
> 2021-01-20 16:36:39,582 INFO
> org.apache.hadoop.ozone.container.common.volume.ThrottledAsyncChecker:
> Scheduling a check for /data4/hdds/hdds
> 2021-01-20 16:36:39,594 INFO
> org.apache.hadoop.ozone.container.common.volume.HddsVolumeChecker: Scheduled
> health check for volume /data4/hdds/hdds
> 2021-01-20 16:36:39,595 INFO
> org.apache.hadoop.ozone.container.common.volume.ThrottledAsyncChecker:
> Scheduling a check for /data1/hdds/hdds
> 2021-01-20 16:36:39,595 INFO
> org.apache.hadoop.ozone.container.common.volume.HddsVolumeChecker: Scheduled
> health check for volume /data1/hdds/hdds
> 2021-01-20 16:36:39,595 INFO
> org.apache.hadoop.ozone.container.common.volume.ThrottledAsyncChecker:
> Scheduling a check for /data2/hdds/hdds
> 2021-01-20 16:36:39,596 INFO
> org.apache.hadoop.ozone.container.common.volume.HddsVolumeChecker: Scheduled
> health check for volume /data2/hdds/hdds
> 2021-01-20 16:36:39,596 INFO
> org.apache.hadoop.ozone.container.common.volume.ThrottledAsyncChecker:
> Scheduling a check for /data3/hdds/hdds
> 2021-01-20 16:36:39,596 INFO
> org.apache.hadoop.ozone.container.common.volume.HddsVolumeChecker: Scheduled
> health check for volume /data3/hdds/hdds
> 2021-01-20 16:36:39,793 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
> the following extra column families in existing DB : [block_data,
> deleted_blocks]
> 2021-01-20 16:36:39,794 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
> the following extra column families in existing DB : [block_data,
> deleted_blocks]
> 2021-01-20 16:36:39,794 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
> the following extra column families in existing DB : [block_data,
> deleted_blocks]
> 2021-01-20 16:36:39,797 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
> the following extra column families in existing DB : [block_data,
> deleted_blocks]
> 2021-01-20 16:36:39,916 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
> the following extra column families in existing DB : [block_data,
> deleted_blocks]
> 2021-01-20 16:36:39,916 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
> the following extra column families in existing DB : [block_data,
> deleted_blocks]
> 2021-01-20 16:36:39,916 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
> the following extra column families in existing DB : [block_data,
> deleted_blocks]
> 2021-01-20 16:36:39,927 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
> the following extra column families in existing DB : [block_data,
> deleted_blocks]
> 2021-01-20 16:36:40,001 WARN
> org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil:
> Attempt to get an uncached RocksDB handle failed and an instance was
> retrieved from the cache. This should only happen in tests
> 2021-01-20 16:36:40,006 WARN
> org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil:
> Attempt to get an uncached RocksDB handle failed and an instance was
> retrieved from the cache. This should only happen in tests
> 2021-01-20 16:36:40,013 WARN
> org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil:
> Attempt to get an uncached RocksDB handle failed and an instance was
> retrieved from the cache. This should only happen in tests
> 2021-01-20 16:36:40,018 WARN
> org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil:
> Attempt to get an uncached RocksDB handle failed and an instance was
> retrieved from the cache. This should only happen in tests
> {code}
> The name changing of the datanode column families done in HDDS-4369 is
> actually not a backward compatible.
> This PR aims to revert the name from deletedBlocks, deleteTxns, blockData to
> block_data, deleted_blocks, delete_txns.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
