[jira] [Updated] (HDFS-11913) Ozone: TestKeySpaceManager#testDeleteVolume fails
[
https://issues.apache.org/jira/browse/HDFS-11913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anu Engineer updated HDFS-11913:
Issue Type: Sub-task (was: Bug)
Parent: HDFS-7240
> Ozone: TestKeySpaceManager#testDeleteVolume fails
> -
>
> Key: HDFS-11913
> URL: https://issues.apache.org/jira/browse/HDFS-11913
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Fix For: HDFS-7240
>
> Attachments: HDFS-11913-HDFS-7240.001.patch
>
>
> HDFS-11774 introduces an UT failure,
> {{TestKeySpaceManager#testDeleteVolume}}, error as below
> {noformat}
> java.util.NoSuchElementException
> at
> org.fusesource.leveldbjni.internal.JniDBIterator.peekNext(JniDBIterator.java:84)
> at
> org.fusesource.leveldbjni.internal.JniDBIterator.next(JniDBIterator.java:98)
> at
> org.fusesource.leveldbjni.internal.JniDBIterator.next(JniDBIterator.java:45)
> at
> org.apache.hadoop.ozone.ksm.MetadataManagerImpl.isVolumeEmpty(MetadataManagerImpl.java:221)
> at
> org.apache.hadoop.ozone.ksm.VolumeManagerImpl.deleteVolume(VolumeManagerImpl.java:294)
> at
> org.apache.hadoop.ozone.ksm.KeySpaceManager.deleteVolume(KeySpaceManager.java:340)
> at
> org.apache.hadoop.ozone.protocolPB.KeySpaceManagerProtocolServerSideTranslatorPB.deleteVolume(KeySpaceManagerProtocolServerSideTranslatorPB.java:200)
> at
> org.apache.hadoop.ozone.protocol.proto.KeySpaceManagerProtocolProtos$KeySpaceManagerService$2.callBlockingMethod(KeySpaceManagerProtocolProtos.java:22742)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:522)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:867)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:813)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2659)
> {noformat}
> this is caused by a buggy code in {{MetadataManagerImpl#isVolumeEmpty}},
> there are 2 issues need to be fixed
> # Iterate next element will throw this exception if it doesn't have next.
> This always fail when a volume is empty.
> # The code was checking if the first bucket name start with "/volume_name",
> this will return a wrong value if I have several empty volumes with same
> prefix, e.g "/volA/", "/volAA/". Such case {{isVolumeEmpty}} will return
> false as the next element from "/volA/" is not a bucket, it's another volume
> "/volAA/" but matches the prefix.
> For now an empty volume with name "/volA/" is probably not valid, but if we
> make sure our bucket key starts with "/volA/" instead of just "/volA" is a
> good idea to leave us away from weird problems.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11913) Ozone: TestKeySpaceManager#testDeleteVolume fails
[
https://issues.apache.org/jira/browse/HDFS-11913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anu Engineer updated HDFS-11913:
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: HDFS-7240
Status: Resolved (was: Patch Available)
[~cheersyang] Thanks for the contribution. [~msingh] and [~yuanbo] Thanks for
the reviews. I have committed this to the feature branch.
> Ozone: TestKeySpaceManager#testDeleteVolume fails
> -
>
> Key: HDFS-11913
> URL: https://issues.apache.org/jira/browse/HDFS-11913
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ozone
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Fix For: HDFS-7240
>
> Attachments: HDFS-11913-HDFS-7240.001.patch
>
>
> HDFS-11774 introduces an UT failure,
> {{TestKeySpaceManager#testDeleteVolume}}, error as below
> {noformat}
> java.util.NoSuchElementException
> at
> org.fusesource.leveldbjni.internal.JniDBIterator.peekNext(JniDBIterator.java:84)
> at
> org.fusesource.leveldbjni.internal.JniDBIterator.next(JniDBIterator.java:98)
> at
> org.fusesource.leveldbjni.internal.JniDBIterator.next(JniDBIterator.java:45)
> at
> org.apache.hadoop.ozone.ksm.MetadataManagerImpl.isVolumeEmpty(MetadataManagerImpl.java:221)
> at
> org.apache.hadoop.ozone.ksm.VolumeManagerImpl.deleteVolume(VolumeManagerImpl.java:294)
> at
> org.apache.hadoop.ozone.ksm.KeySpaceManager.deleteVolume(KeySpaceManager.java:340)
> at
> org.apache.hadoop.ozone.protocolPB.KeySpaceManagerProtocolServerSideTranslatorPB.deleteVolume(KeySpaceManagerProtocolServerSideTranslatorPB.java:200)
> at
> org.apache.hadoop.ozone.protocol.proto.KeySpaceManagerProtocolProtos$KeySpaceManagerService$2.callBlockingMethod(KeySpaceManagerProtocolProtos.java:22742)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:522)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:867)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:813)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2659)
> {noformat}
> this is caused by a buggy code in {{MetadataManagerImpl#isVolumeEmpty}},
> there are 2 issues need to be fixed
> # Iterate next element will throw this exception if it doesn't have next.
> This always fail when a volume is empty.
> # The code was checking if the first bucket name start with "/volume_name",
> this will return a wrong value if I have several empty volumes with same
> prefix, e.g "/volA/", "/volAA/". Such case {{isVolumeEmpty}} will return
> false as the next element from "/volA/" is not a bucket, it's another volume
> "/volAA/" but matches the prefix.
> For now an empty volume with name "/volA/" is probably not valid, but if we
> make sure our bucket key starts with "/volA/" instead of just "/volA" is a
> good idea to leave us away from weird problems.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11913) Ozone: TestKeySpaceManager#testDeleteVolume fails
[
https://issues.apache.org/jira/browse/HDFS-11913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang updated HDFS-11913:
---
Description:
HDFS-11774 introduces an UT failure, {{TestKeySpaceManager#testDeleteVolume}},
error as below
{noformat}
java.util.NoSuchElementException
at
org.fusesource.leveldbjni.internal.JniDBIterator.peekNext(JniDBIterator.java:84)
at org.fusesource.leveldbjni.internal.JniDBIterator.next(JniDBIterator.java:98)
at org.fusesource.leveldbjni.internal.JniDBIterator.next(JniDBIterator.java:45)
at
org.apache.hadoop.ozone.ksm.MetadataManagerImpl.isVolumeEmpty(MetadataManagerImpl.java:221)
at
org.apache.hadoop.ozone.ksm.VolumeManagerImpl.deleteVolume(VolumeManagerImpl.java:294)
at
org.apache.hadoop.ozone.ksm.KeySpaceManager.deleteVolume(KeySpaceManager.java:340)
at
org.apache.hadoop.ozone.protocolPB.KeySpaceManagerProtocolServerSideTranslatorPB.deleteVolume(KeySpaceManagerProtocolServerSideTranslatorPB.java:200)
at
org.apache.hadoop.ozone.protocol.proto.KeySpaceManagerProtocolProtos$KeySpaceManagerService$2.callBlockingMethod(KeySpaceManagerProtocolProtos.java:22742)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:522)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:867)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:813)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2659)
{noformat}
this is caused by a buggy code in {{MetadataManagerImpl#isVolumeEmpty}}, there
are 2 issues need to be fixed
# Iterate next element will throw this exception if it doesn't have next. This
always fail when a volume is empty.
# The code was checking if the first bucket name start with "/volume_name",
this will return a wrong value if I have several empty volumes with same
prefix, e.g "/volA/", "/volAA/". Such case {{isVolumeEmpty}} will return false
as the next element from "/volA/" is not a bucket, it's another volume
"/volAA/" but matches the prefix.
For now an empty volume with name "/volA/" is probably not valid, but if we
make sure our bucket key starts with "/volA/" instead of just "/volA" is a good
idea to leave us away from weird problems.
was:
HDFS-11774 introduces an UT failure, {{TestKeySpaceManager#testDeleteVolume}},
error as below
{noformat}
java.util.NoSuchElementException
at
org.fusesource.leveldbjni.internal.JniDBIterator.peekNext(JniDBIterator.java:84)
at org.fusesource.leveldbjni.internal.JniDBIterator.next(JniDBIterator.java:98)
at org.fusesource.leveldbjni.internal.JniDBIterator.next(JniDBIterator.java:45)
at
org.apache.hadoop.ozone.ksm.MetadataManagerImpl.isVolumeEmpty(MetadataManagerImpl.java:221)
at
org.apache.hadoop.ozone.ksm.VolumeManagerImpl.deleteVolume(VolumeManagerImpl.java:294)
at
org.apache.hadoop.ozone.ksm.KeySpaceManager.deleteVolume(KeySpaceManager.java:340)
at
org.apache.hadoop.ozone.protocolPB.KeySpaceManagerProtocolServerSideTranslatorPB.deleteVolume(KeySpaceManagerProtocolServerSideTranslatorPB.java:200)
at
org.apache.hadoop.ozone.protocol.proto.KeySpaceManagerProtocolProtos$KeySpaceManagerService$2.callBlockingMethod(KeySpaceManagerProtocolProtos.java:22742)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:522)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:867)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:813)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2659)
{noformat}
this is caused by a buggy code in {{MetadataManagerImpl#isVolumeEmpty}}, there
are 2 issues need to be fixed
# Iterate next element will throw this exception if it doesn't have next. This
always fail when a volume is empty.
# The code was checking if the first bucket name start with "/volume_name",
this will return a wrong value if I have several empty volumes with same
prefix, e.g "/volA/", "/volAA/". Such case {{isVolumeEmpty}} will return false
as the next element from "/volA/" is not a bucket, it's another volume
"/volAA/" but matches the prefix.
> Ozone: TestKeySpaceManager#testDeleteVolume fails
> -
>
> Key: HDFS-11913
> URL: https://issues.apache.org/jira/browse/HDFS-11913
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ozone
>Reporter: Weiwei Yang
>
[jira] [Updated] (HDFS-11913) Ozone: TestKeySpaceManager#testDeleteVolume fails
[
https://issues.apache.org/jira/browse/HDFS-11913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang updated HDFS-11913:
---
Status: Patch Available (was: Open)
> Ozone: TestKeySpaceManager#testDeleteVolume fails
> -
>
> Key: HDFS-11913
> URL: https://issues.apache.org/jira/browse/HDFS-11913
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ozone
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Attachments: HDFS-11913-HDFS-7240.001.patch
>
>
> HDFS-11774 introduces an UT failure,
> {{TestKeySpaceManager#testDeleteVolume}}, error as below
> {noformat}
> java.util.NoSuchElementException
> at
> org.fusesource.leveldbjni.internal.JniDBIterator.peekNext(JniDBIterator.java:84)
> at
> org.fusesource.leveldbjni.internal.JniDBIterator.next(JniDBIterator.java:98)
> at
> org.fusesource.leveldbjni.internal.JniDBIterator.next(JniDBIterator.java:45)
> at
> org.apache.hadoop.ozone.ksm.MetadataManagerImpl.isVolumeEmpty(MetadataManagerImpl.java:221)
> at
> org.apache.hadoop.ozone.ksm.VolumeManagerImpl.deleteVolume(VolumeManagerImpl.java:294)
> at
> org.apache.hadoop.ozone.ksm.KeySpaceManager.deleteVolume(KeySpaceManager.java:340)
> at
> org.apache.hadoop.ozone.protocolPB.KeySpaceManagerProtocolServerSideTranslatorPB.deleteVolume(KeySpaceManagerProtocolServerSideTranslatorPB.java:200)
> at
> org.apache.hadoop.ozone.protocol.proto.KeySpaceManagerProtocolProtos$KeySpaceManagerService$2.callBlockingMethod(KeySpaceManagerProtocolProtos.java:22742)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:522)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:867)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:813)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2659)
> {noformat}
> this is caused by a buggy code in {{MetadataManagerImpl#isVolumeEmpty}},
> there are 2 issues need to be fixed
> # Iterate next element will throw this exception if it doesn't have next.
> This always fail when a volume is empty.
> # The code was checking if the first bucket name start with "/volume_name",
> this will return a wrong value if I have several empty volumes with same
> prefix, e.g "/volA/", "/volAA/". Such case {{isVolumeEmpty}} will return
> false as the next element from "/volA/" is not a bucket, it's another volume
> "/volAA/" but matches the prefix.
> For now an empty volume with name "/volA/" is probably not valid, but if we
> make sure our bucket key starts with "/volA/" instead of just "/volA" is a
> good idea to leave us away from weird problems.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11913) Ozone: TestKeySpaceManager#testDeleteVolume fails
[
https://issues.apache.org/jira/browse/HDFS-11913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weiwei Yang updated HDFS-11913:
---
Attachment: HDFS-11913-HDFS-7240.001.patch
> Ozone: TestKeySpaceManager#testDeleteVolume fails
> -
>
> Key: HDFS-11913
> URL: https://issues.apache.org/jira/browse/HDFS-11913
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: ozone
>Reporter: Weiwei Yang
>Assignee: Weiwei Yang
> Attachments: HDFS-11913-HDFS-7240.001.patch
>
>
> HDFS-11774 introduces an UT failure,
> {{TestKeySpaceManager#testDeleteVolume}}, error as below
> {noformat}
> java.util.NoSuchElementException
> at
> org.fusesource.leveldbjni.internal.JniDBIterator.peekNext(JniDBIterator.java:84)
> at
> org.fusesource.leveldbjni.internal.JniDBIterator.next(JniDBIterator.java:98)
> at
> org.fusesource.leveldbjni.internal.JniDBIterator.next(JniDBIterator.java:45)
> at
> org.apache.hadoop.ozone.ksm.MetadataManagerImpl.isVolumeEmpty(MetadataManagerImpl.java:221)
> at
> org.apache.hadoop.ozone.ksm.VolumeManagerImpl.deleteVolume(VolumeManagerImpl.java:294)
> at
> org.apache.hadoop.ozone.ksm.KeySpaceManager.deleteVolume(KeySpaceManager.java:340)
> at
> org.apache.hadoop.ozone.protocolPB.KeySpaceManagerProtocolServerSideTranslatorPB.deleteVolume(KeySpaceManagerProtocolServerSideTranslatorPB.java:200)
> at
> org.apache.hadoop.ozone.protocol.proto.KeySpaceManagerProtocolProtos$KeySpaceManagerService$2.callBlockingMethod(KeySpaceManagerProtocolProtos.java:22742)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:522)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:867)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:813)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2659)
> {noformat}
> this is caused by a buggy code in {{MetadataManagerImpl#isVolumeEmpty}},
> there are 2 issues need to be fixed
> # Iterate next element will throw this exception if it doesn't have next.
> This always fail when a volume is empty.
> # The code was checking if the first bucket name start with "/volume_name",
> this will return a wrong value if I have several empty volumes with same
> prefix, e.g "/volA/", "/volAA/". Such case {{isVolumeEmpty}} will return
> false as the next element from "/volA/" is not a bucket, it's another volume
> "/volAA/" but matches the prefix.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
