[
https://issues.apache.org/jira/browse/HDDS-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Glen Geng updated HDDS-4724:
----------------------------
Description:
Tencent monthly deploy the latest master to our production environment.
During latest upgrade test, we found that new Ozone(version at Jan 14 2021) can
not read the data written by old Ozone(version at Dec 14 2020).
We found following traces
{code:java}
2021-01-20 16:36:39,575 INFO
org.apache.hadoop.ozone.container.common.volume.MutableVolumeSet: Added Volume
: /data4/hdds/hdds to VolumeSet
2021-01-20 16:36:39,582 INFO
org.apache.hadoop.ozone.container.common.volume.ThrottledAsyncChecker:
Scheduling a check for /data4/hdds/hdds
2021-01-20 16:36:39,594 INFO
org.apache.hadoop.ozone.container.common.volume.HddsVolumeChecker: Scheduled
health check for volume /data4/hdds/hdds
2021-01-20 16:36:39,595 INFO
org.apache.hadoop.ozone.container.common.volume.ThrottledAsyncChecker:
Scheduling a check for /data1/hdds/hdds
2021-01-20 16:36:39,595 INFO
org.apache.hadoop.ozone.container.common.volume.HddsVolumeChecker: Scheduled
health check for volume /data1/hdds/hdds
2021-01-20 16:36:39,595 INFO
org.apache.hadoop.ozone.container.common.volume.ThrottledAsyncChecker:
Scheduling a check for /data2/hdds/hdds
2021-01-20 16:36:39,596 INFO
org.apache.hadoop.ozone.container.common.volume.HddsVolumeChecker: Scheduled
health check for volume /data2/hdds/hdds
2021-01-20 16:36:39,596 INFO
org.apache.hadoop.ozone.container.common.volume.ThrottledAsyncChecker:
Scheduling a check for /data3/hdds/hdds
2021-01-20 16:36:39,596 INFO
org.apache.hadoop.ozone.container.common.volume.HddsVolumeChecker: Scheduled
health check for volume /data3/hdds/hdds
2021-01-20 16:36:39,793 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
the following extra column families in existing DB : [block_data,
deleted_blocks]
2021-01-20 16:36:39,794 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
the following extra column families in existing DB : [block_data,
deleted_blocks]
2021-01-20 16:36:39,794 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
the following extra column families in existing DB : [block_data,
deleted_blocks]
2021-01-20 16:36:39,797 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
the following extra column families in existing DB : [block_data,
deleted_blocks]
2021-01-20 16:36:39,916 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
the following extra column families in existing DB : [block_data,
deleted_blocks]
2021-01-20 16:36:39,916 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
the following extra column families in existing DB : [block_data,
deleted_blocks]
2021-01-20 16:36:39,916 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
the following extra column families in existing DB : [block_data,
deleted_blocks]
2021-01-20 16:36:39,927 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
the following extra column families in existing DB : [block_data,
deleted_blocks]
2021-01-20 16:36:40,001 WARN
org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil:
Attempt to get an uncached RocksDB handle failed and an instance was retrieved
from the cache. This should only happen in tests
2021-01-20 16:36:40,006 WARN
org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil:
Attempt to get an uncached RocksDB handle failed and an instance was retrieved
from the cache. This should only happen in tests
2021-01-20 16:36:40,013 WARN
org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil:
Attempt to get an uncached RocksDB handle failed and an instance was retrieved
from the cache. This should only happen in tests
2021-01-20 16:36:40,018 WARN
org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil:
Attempt to get an uncached RocksDB handle failed and an instance was retrieved
from the cache. This should only happen in tests
{code}
The name changing of the datanode column families is actually not a backward
compatible.
This PR aims to revert the name from deletedBlocks, deleteTxns, blockData to
block_data, deleted_blocks, delete_txns.
> Fix backward in-compatibility issue introduced by HDDS-4369. Datanode should
> store the delete transaction as is in rocksDB.
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: HDDS-4724
> URL: https://issues.apache.org/jira/browse/HDDS-4724
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Datanode
> Affects Versions: 1.0.0
> Reporter: Glen Geng
> Assignee: Glen Geng
> Priority: Major
>
>
> Tencent monthly deploy the latest master to our production environment.
> During latest upgrade test, we found that new Ozone(version at Jan 14 2021)
> can not read the data written by old Ozone(version at Dec 14 2020).
> We found following traces
> {code:java}
> 2021-01-20 16:36:39,575 INFO
> org.apache.hadoop.ozone.container.common.volume.MutableVolumeSet: Added
> Volume : /data4/hdds/hdds to VolumeSet
> 2021-01-20 16:36:39,582 INFO
> org.apache.hadoop.ozone.container.common.volume.ThrottledAsyncChecker:
> Scheduling a check for /data4/hdds/hdds
> 2021-01-20 16:36:39,594 INFO
> org.apache.hadoop.ozone.container.common.volume.HddsVolumeChecker: Scheduled
> health check for volume /data4/hdds/hdds
> 2021-01-20 16:36:39,595 INFO
> org.apache.hadoop.ozone.container.common.volume.ThrottledAsyncChecker:
> Scheduling a check for /data1/hdds/hdds
> 2021-01-20 16:36:39,595 INFO
> org.apache.hadoop.ozone.container.common.volume.HddsVolumeChecker: Scheduled
> health check for volume /data1/hdds/hdds
> 2021-01-20 16:36:39,595 INFO
> org.apache.hadoop.ozone.container.common.volume.ThrottledAsyncChecker:
> Scheduling a check for /data2/hdds/hdds
> 2021-01-20 16:36:39,596 INFO
> org.apache.hadoop.ozone.container.common.volume.HddsVolumeChecker: Scheduled
> health check for volume /data2/hdds/hdds
> 2021-01-20 16:36:39,596 INFO
> org.apache.hadoop.ozone.container.common.volume.ThrottledAsyncChecker:
> Scheduling a check for /data3/hdds/hdds
> 2021-01-20 16:36:39,596 INFO
> org.apache.hadoop.ozone.container.common.volume.HddsVolumeChecker: Scheduled
> health check for volume /data3/hdds/hdds
> 2021-01-20 16:36:39,793 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
> the following extra column families in existing DB : [block_data,
> deleted_blocks]
> 2021-01-20 16:36:39,794 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
> the following extra column families in existing DB : [block_data,
> deleted_blocks]
> 2021-01-20 16:36:39,794 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
> the following extra column families in existing DB : [block_data,
> deleted_blocks]
> 2021-01-20 16:36:39,797 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
> the following extra column families in existing DB : [block_data,
> deleted_blocks]
> 2021-01-20 16:36:39,916 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
> the following extra column families in existing DB : [block_data,
> deleted_blocks]
> 2021-01-20 16:36:39,916 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
> the following extra column families in existing DB : [block_data,
> deleted_blocks]
> 2021-01-20 16:36:39,916 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
> the following extra column families in existing DB : [block_data,
> deleted_blocks]
> 2021-01-20 16:36:39,927 INFO org.apache.hadoop.hdds.utils.db.RDBStore: Found
> the following extra column families in existing DB : [block_data,
> deleted_blocks]
> 2021-01-20 16:36:40,001 WARN
> org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil:
> Attempt to get an uncached RocksDB handle failed and an instance was
> retrieved from the cache. This should only happen in tests
> 2021-01-20 16:36:40,006 WARN
> org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil:
> Attempt to get an uncached RocksDB handle failed and an instance was
> retrieved from the cache. This should only happen in tests
> 2021-01-20 16:36:40,013 WARN
> org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil:
> Attempt to get an uncached RocksDB handle failed and an instance was
> retrieved from the cache. This should only happen in tests
> 2021-01-20 16:36:40,018 WARN
> org.apache.hadoop.ozone.container.keyvalue.helpers.KeyValueContainerUtil:
> Attempt to get an uncached RocksDB handle failed and an instance was
> retrieved from the cache. This should only happen in tests
> {code}
> The name changing of the datanode column families is actually not a backward
> compatible.
> This PR aims to revert the name from deletedBlocks, deleteTxns, blockData to
> block_data, deleted_blocks, delete_txns.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
