[
https://issues.apache.org/jira/browse/HDDS-10399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Saketa Chalamchala updated HDDS-10399:
--------------------------------------
Description:
There is a bug in the list key improvements made or OBS/LEGACY buckets bucket
listing errors out when access it via S3 in some cases.
{code:java}
ozone sh bucket create /s3v/testbucket -l OBJECT_STORE
hadoop fs -mkdir -p s3a://testbucket/data/test/
hadoop fs -mkdir -p s3a://testbucket/data/test/mr_job/input
24/02/20 23:56:17 WARN impl.MetricsConfig: Cannot locate configuration: tried
hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
24/02/20 23:56:17 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period
at 10 second(s).
24/02/20 23:56:17 INFO impl.MetricsSystemImpl: s3a-file-system metrics system
started
24/02/20 23:56:18 INFO Configuration.deprecation: No unit for
fs.s3a.connection.request.timeout(0) assuming SECONDS
24/02/20 23:56:18 INFO impl.DirectoryPolicyImpl: Directory markers will be kept
mkdir: getFileStatus on s3a://testbucket/data/test:
com.amazonaws.services.s3.model.AmazonS3Exception: Server Error (Service:
Amazon S3; Status Code: 500; Error Code: 500 Server Error; Request ID: null; S3
Extended Request ID: null; Proxy: null), S3 Extended Request ID: null:500
Server Error: Server Error (Service: Amazon S3; Status Code: 500; Error Code:
500 Server Error; Request ID: null; S3 Extended Request ID: null; Proxy: null)
In S3 gateway log:
Caused by: java.lang.IndexOutOfBoundsException: Index 0 out of bounds for
length 0
at
java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
at
java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
at
java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248)
at java.base/java.util.Objects.checkIndex(Objects.java:372)
at java.base/java.util.ArrayList.remove(ArrayList.java:535)
at
org.apache.hadoop.ozone.client.OzoneBucket$KeyIterator.getNextShallowListOfKeys(OzoneBucket.java:1234)
at
org.apache.hadoop.ozone.client.OzoneBucket$KeyIterator.getNextListOfKeys(OzoneBucket.java:1136)
at
org.apache.hadoop.ozone.client.OzoneBucket$KeyIterator.hasNext(OzoneBucket.java:1110)
at
org.apache.hadoop.ozone.s3.endpoint.BucketEndpoint.get(BucketEndpoint.java:208)
at jdk.internal.reflect.GeneratedMethodAccessor90.invoke(Unknown Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
at
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234)
at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)
at
org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
... 55 more {code}
was:
There is a bug in the list key improvements made or OBS/LEGACY buckets bucket
listing errors out when access it via S3 in some cases.
{code:java}
ozone sh bucket create /s3v/testbucket -l OBJECT_STORE
hadoop fs -mkdir -p s3a://testbucket/data/test/
hadoop fs -mkdir -p s3a://testbucket/data/test/mr_job/input
com.amazonaws.services.s3.model.AmazonS3Exception: Server Error (Service:
Amazon S3; Status Code: 500; Error Code: 500 Server Error; Request ID: null; S3
Extended Request ID: null; Proxy: null), S3 Extended Request ID: null:500
Server Error: Server Error (Service: Amazon S3; Status Code: 500; Error Code:
500 Server Error; Request ID: null; S3 Extended Request ID: null; Proxy: null)
In S3 gateway log:
Caused by: java.lang.IndexOutOfBoundsException: Index 0 out of bounds for
length 0
at
java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
at
java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
at
java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248)
at java.base/java.util.Objects.checkIndex(Objects.java:372)
at java.base/java.util.ArrayList.remove(ArrayList.java:535)
at
org.apache.hadoop.ozone.client.OzoneBucket$KeyIterator.getNextShallowListOfKeys(OzoneBucket.java:1234)
at
org.apache.hadoop.ozone.client.OzoneBucket$KeyIterator.getNextListOfKeys(OzoneBucket.java:1136)
at
org.apache.hadoop.ozone.client.OzoneBucket$KeyIterator.hasNext(OzoneBucket.java:1110)
at
org.apache.hadoop.ozone.s3.endpoint.BucketEndpoint.get(BucketEndpoint.java:208)
at jdk.internal.reflect.GeneratedMethodAccessor90.invoke(Unknown Source)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
at
org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
at
org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397)
at
org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
at
org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
at
org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
at
org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234)
at
org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)
at
org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
... 55 more {code}
> Error when listing OBS bucket
> -----------------------------
>
> Key: HDDS-10399
> URL: https://issues.apache.org/jira/browse/HDDS-10399
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone CLI
> Affects Versions: 1.4.0
> Reporter: Saketa Chalamchala
> Assignee: Saketa Chalamchala
> Priority: Major
>
> There is a bug in the list key improvements made or OBS/LEGACY buckets bucket
> listing errors out when access it via S3 in some cases.
>
> {code:java}
> ozone sh bucket create /s3v/testbucket -l OBJECT_STORE
> hadoop fs -mkdir -p s3a://testbucket/data/test/
> hadoop fs -mkdir -p s3a://testbucket/data/test/mr_job/input
> 24/02/20 23:56:17 WARN impl.MetricsConfig: Cannot locate configuration: tried
> hadoop-metrics2-s3a-file-system.properties,hadoop-metrics2.properties
> 24/02/20 23:56:17 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot
> period at 10 second(s).
> 24/02/20 23:56:17 INFO impl.MetricsSystemImpl: s3a-file-system metrics system
> started
> 24/02/20 23:56:18 INFO Configuration.deprecation: No unit for
> fs.s3a.connection.request.timeout(0) assuming SECONDS
> 24/02/20 23:56:18 INFO impl.DirectoryPolicyImpl: Directory markers will be
> kept
> mkdir: getFileStatus on s3a://testbucket/data/test:
> com.amazonaws.services.s3.model.AmazonS3Exception: Server Error (Service:
> Amazon S3; Status Code: 500; Error Code: 500 Server Error; Request ID: null;
> S3 Extended Request ID: null; Proxy: null), S3 Extended Request ID: null:500
> Server Error: Server Error (Service: Amazon S3; Status Code: 500; Error Code:
> 500 Server Error; Request ID: null; S3 Extended Request ID: null; Proxy: null)
> In S3 gateway log:
> Caused by: java.lang.IndexOutOfBoundsException: Index 0 out of bounds for
> length 0
> at
> java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
> at
> java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
> at
> java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248)
> at java.base/java.util.Objects.checkIndex(Objects.java:372)
> at java.base/java.util.ArrayList.remove(ArrayList.java:535)
> at
> org.apache.hadoop.ozone.client.OzoneBucket$KeyIterator.getNextShallowListOfKeys(OzoneBucket.java:1234)
> at
> org.apache.hadoop.ozone.client.OzoneBucket$KeyIterator.getNextListOfKeys(OzoneBucket.java:1136)
> at
> org.apache.hadoop.ozone.client.OzoneBucket$KeyIterator.hasNext(OzoneBucket.java:1110)
> at
> org.apache.hadoop.ozone.s3.endpoint.BucketEndpoint.get(BucketEndpoint.java:208)
> at jdk.internal.reflect.GeneratedMethodAccessor90.invoke(Unknown
> Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
> at
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124)
> at
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167)
> at
> org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
> at
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79)
> at
> org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475)
> at
> org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397)
> at
> org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
> at
> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255)
> at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
> at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
> at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
> at
> org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
> at
> org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234)
> at
> org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)
> at
> org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
> ... 55 more {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
