umamaheswararao commented on PR #3514:
URL: https://github.com/apache/ozone/pull/3514#issuecomment-1168115808
@guihecheng, I have discovered several issues with this. So, I have just
kept this issue a side currently as I am focussing on other prioritized tasks.
Here just empty put block will not work:
1. Container may not be available. PutBlock will not create container on
absence. Container's usually be created on first write chunk. Since we don't
have write chunk, empty putBlocks will just fail.
2. Last putBlock with close flag true will fail even though container is
available as chunkFile may not exist. close flag will enforce to flush the
content to os file system at the DN.
So, the best way could be to create the container's from client on
initializing the streams first.
Here I found a problem that, container's may exist already and exist
createContainer API will just fail when container already exist. SO, we may
need a createContainerAPI which should not fail when container exist. May be we
need additional flag in createContainer API?
With offline recovery, we are creating the container all the time to target
nodes first. So, even though no blocks needs to be recovered, we may create the
container. So, first recovery task will solve the problem and from then onward
we will have empty containers I guess. Only thing we need to make sure is,
empty containers should not get delete in EC case from RM. With this, I felt
this can be low prioritized, until RM work is done.
Doing it as part of close or start of input stream should not have any
difference. In both cases, createContainer will fail if already exist.
Please not current code is just experimental changes. Not for commit or
review. Feel free to suggest or take up if you have any other thoughts which
may be simpler and low risk.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
