Sammi Chen created HDDS-14417:
---------------------------------
Summary: Don't wrap allocated but unused blocks as pseudo file and
save in deletedTable
Key: HDDS-14417
URL: https://issues.apache.org/jira/browse/HDDS-14417
Project: Apache Ozone
Issue Type: Improvement
Reporter: Sammi Chen
In the storage capacity feature testing, Prince has found that once he run the
CLI
{code:java}
"ozone freon ockg -n 100 -s 0 -v vol43 -b bucket -p dir1/dir2"
{code}
There are pending deletion block transactions reported by SCM
{code:java}
{
"totalBlocksize": 26843545600,
"totalReplicatedBlockSize": 80530636800,
"totalBlocksCount": 100
}
{code}
Since the files created by ockg are 0 length files, so it's not expected that
there will be blocks for deletion.
Investigation through SCM audit log and log file, shows that SCM does received
the block deletion requests from OM, as that above data of SCM is correct.
While there is no related DELETION in om audit log.
A further investigation shows that this is related with pre-allocated block of
OM during key/file creation. Here is the flow,
- OM receives a key1 creation request, created key1, and allocated a new block1
for key1
- OM receives key1's commit request, commit key1, and wrap the unused block1 as
a pseudo file key1-p, and put this key1-p into the deletedTable
- OM KeyDeletingService service scan the deletedTable, find key1-p, sent block
deletion request to SCM for key1-p
Since block1 doesn't used, its container is never created on DNs, no replica of
this container can be found. If user manually closes this container, then this
container will never get chance to be created, so the block deletion for block1
will stay in SCM DB forever.
If this container keeps open and SCM allocates another block for this
container, and data written to DN, this container got created. The block1
deletion command finally can be executed on DNs, but it will cause block file
not found error or block metadata not found error.
So the ideal state is we don't wrap this block allocated but unused as a pseudo
file and put into deletedTable, as this block doesn’t need delete, it doesn’t
exist.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
