[
https://issues.apache.org/jira/browse/HDDS-5908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharat Viswanadham updated HDDS-5908:
-------------------------------------
Description:
Failure was observed on this CI run:
[https://github.com/apache/ozone/runs/4015387580?check_suite_focus=true]
The output bundles expired before I could add them to this Jira, but the
failure can be reproduced on master by applying the patch attached to this Jira
and running the test. The patch speeds up repeated execution of the test by
repeatedly writing keys without having to spin up a new mini ozone cluster in
between each write. It usually takes about 4 minutes and 1200 iterations to
reproduce.
The failing assertion is:
{code:java}
OzoneInputStream inputStream = bucket.readKey(keyName);
Assert.assertTrue(inputStream instanceof MultipartCryptoKeyInputStream);{code}
Indicating a plain OzoneInputStream is returned, since the class has no other
sub classes.
*I have found the reason for this.*
1. If complete MPU is completed, it adds the entry to keyTable.
2. Now if getKey happens on this, if doublebuffer flush not completed flush and
cleaned up cache if entry is still in keyTable, the key info returned as Not
Mpu Key, this is due to a bug in OmKeyInfo#copyObject Which is not using
isMultipartKey.
was:
Failure was observed on this CI run:
[https://github.com/apache/ozone/runs/4015387580?check_suite_focus=true]
The output bundles expired before I could add them to this Jira, but the
failure can be reproduced on master by applying the patch attached to this Jira
and running the test. The patch speeds up repeated execution of the test by
repeatedly writing keys without having to spin up a new mini ozone cluster in
between each write. It usually takes about 4 minutes and 1200 iterations to
reproduce.
The failing assertion is:
{code:java}
OzoneInputStream inputStream = bucket.readKey(keyName);
Assert.assertTrue(inputStream instanceof MultipartCryptoKeyInputStream);{code}
Indicating a plain OzoneInputStream is returned, since the class has no other
sub classes.
I found the reason.
1. If complete MPU is completed, it adds the entry to keyTable.
2. Now if getKey happens on this, if doublebuffer flush not completed flush and
cleaned up cache if entry is still in keyTable, the key info returned as Not
Mpu Key, this is due to a bug in OmKeyInfo#copyObject Which is not using
isMultipartKey.
> MPU getKey can fail, if completeMPU result is still in cache
> ------------------------------------------------------------
>
> Key: HDDS-5908
> URL: https://issues.apache.org/jira/browse/HDDS-5908
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Ethan Rose
> Assignee: Bharat Viswanadham
> Priority: Major
> Labels: pull-request-available
> Attachments: repeat_mpu_test.diff
>
>
> Failure was observed on this CI run:
> [https://github.com/apache/ozone/runs/4015387580?check_suite_focus=true]
> The output bundles expired before I could add them to this Jira, but the
> failure can be reproduced on master by applying the patch attached to this
> Jira and running the test. The patch speeds up repeated execution of the test
> by repeatedly writing keys without having to spin up a new mini ozone cluster
> in between each write. It usually takes about 4 minutes and 1200 iterations
> to reproduce.
> The failing assertion is:
> {code:java}
> OzoneInputStream inputStream = bucket.readKey(keyName);
> Assert.assertTrue(inputStream instanceof MultipartCryptoKeyInputStream);{code}
> Indicating a plain OzoneInputStream is returned, since the class has no other
> sub classes.
> *I have found the reason for this.*
> 1. If complete MPU is completed, it adds the entry to keyTable.
> 2. Now if getKey happens on this, if doublebuffer flush not completed flush
> and cleaned up cache if entry is still in keyTable, the key info returned as
> Not Mpu Key, this is due to a bug in OmKeyInfo#copyObject Which is not using
> isMultipartKey.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
