Xu Shao Hong created HDDS-6203:
----------------------------------
Summary: CleanUp gz files failed to be fully written during
Container move
Key: HDDS-6203
URL: https://issues.apache.org/jira/browse/HDDS-6203
Project: Apache Ozone
Issue Type: Bug
Reporter: Xu Shao Hong
Assignee: Xu Shao Hong
Right now, the container re-replication will be sent with GRPC as gz files to
the temporary dir. If the temporary dir is small, there will be a contest for
space left that concurrent threads downloading the containers will compete to
write the downloaded byte buffer to the actual files with FileOutputStream.
Once the thread fails to write the buffer to file, the current logic will not
clean up the failed and incomplete file and just complete exceptionally as the
code shows.
{code:java}
GrpcReplicationClient
@Override
public void onNext(CopyContainerResponseProto chunk) {
try {
chunk.getData().writeTo(stream);
} catch (IOException e) {
response.completeExceptionally(e);
}
} {code}
the exception will be caught at ```getContainerDataFromReplicas``` and only
will be logged as an error.
Thus it is necessary to clean up the possible incomplete files which failed in
such case.
(I have tested manually to mimic such a contest case and proved the incomplete
files remained, the example could be seen in attachment)
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
