Re: [ceph-users] RADOSGW S3 - Continuation Token Ignored?
FYI, this PR just merged. I would expect to see backports at least as
far as N, and others would be possible.
regards,
Matt
On Fri, Jun 28, 2019 at 3:43 PM wrote:
>
> Matt;
>
> Yep, that would certainly explain it.
>
> My apologies, I almost searched for that information before sending the email.
>
> Thank you,
>
> Dominic L. Hilsbos, MBA
> Director – Information Technology
> Perform Air International Inc.
> dhils...@performair.com
> www.PerformAir.com
>
>
>
> -Original Message-
> From: Matt Benjamin [mailto:mbenj...@redhat.com]
> Sent: Friday, June 28, 2019 9:48 AM
> To: Dominic Hilsbos
> Cc: ceph-users
> Subject: Re: [ceph-users] RADOSGW S3 - Continuation Token Ignored?
>
> Hi Dominic,
>
> The reason is likely that RGW doesn't yet support ListObjectsV2.
>
> Support is nearly here though: https://github.com/ceph/ceph/pull/28102
>
> Matt
>
>
> On Fri, Jun 28, 2019 at 12:43 PM wrote:
> >
> > All;
> >
> > I've got a RADOSGW instance setup, backed by my demonstration Ceph cluster.
> > I'm using Amazon's S3 SDK, and I've run into an annoying little snag.
> >
> > My code looks like this:
> > amazonS3 = builder.build();
> >
> > ListObjectsV2Request req = new
> > ListObjectsV2Request().withBucketName("WorkOrder").withMaxKeys(MAX_KEYS);
> > ListObjectsV2Result result;
> >
> > do
> > {
> > result = amazonS3.listObjectsV2(req);
> >
> > for (S3ObjectSummary objectSummary : result.getObjectSummaries())
> > {
> > summaries.add(objectSummary);
> > }
> >
> > String token = result.getNextContinuationToken();
> > req.setContinuationToken(token);
> > }
> > while (result.isTruncated());
> >
> > The problem is, the ContinuationToken seems to be ignored, i.e. every call
> > to amazonS3.listObjectsV2(req) returns the same set, and the loop never
> > ends (until the summaries LinkedList overflows).
> >
> > Thoughts?
> >
> > Thank you,
> >
> > Dominic L. Hilsbos, MBA
> > Director - Information Technology
> > Perform Air International Inc.
> > dhils...@performair.com
> > www.PerformAir.com
> >
> >
> >
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
> >
>
>
> --
>
> Matt Benjamin
> Red Hat, Inc.
> 315 West Huron Street, Suite 140A
> Ann Arbor, Michigan 48103
>
> http://www.redhat.com/en/technologies/storage
>
> tel. 734-821-5101
> fax. 734-769-8938
> cel. 734-216-5309
--
Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103
http://www.redhat.com/en/technologies/storage
tel. 734-821-5101
fax. 734-769-8938
cel. 734-216-5309
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] RADOSGW S3 - Continuation Token Ignored?
Matt;
Yep, that would certainly explain it.
My apologies, I almost searched for that information before sending the email.
Thank you,
Dominic L. Hilsbos, MBA
Director – Information Technology
Perform Air International Inc.
dhils...@performair.com
www.PerformAir.com
-Original Message-
From: Matt Benjamin [mailto:mbenj...@redhat.com]
Sent: Friday, June 28, 2019 9:48 AM
To: Dominic Hilsbos
Cc: ceph-users
Subject: Re: [ceph-users] RADOSGW S3 - Continuation Token Ignored?
Hi Dominic,
The reason is likely that RGW doesn't yet support ListObjectsV2.
Support is nearly here though: https://github.com/ceph/ceph/pull/28102
Matt
On Fri, Jun 28, 2019 at 12:43 PM wrote:
>
> All;
>
> I've got a RADOSGW instance setup, backed by my demonstration Ceph cluster.
> I'm using Amazon's S3 SDK, and I've run into an annoying little snag.
>
> My code looks like this:
> amazonS3 = builder.build();
>
> ListObjectsV2Request req = new
> ListObjectsV2Request().withBucketName("WorkOrder").withMaxKeys(MAX_KEYS);
> ListObjectsV2Result result;
>
> do
> {
> result = amazonS3.listObjectsV2(req);
>
> for (S3ObjectSummary objectSummary : result.getObjectSummaries())
> {
> summaries.add(objectSummary);
> }
>
> String token = result.getNextContinuationToken();
> req.setContinuationToken(token);
> }
> while (result.isTruncated());
>
> The problem is, the ContinuationToken seems to be ignored, i.e. every call to
> amazonS3.listObjectsV2(req) returns the same set, and the loop never ends
> (until the summaries LinkedList overflows).
>
> Thoughts?
>
> Thank you,
>
> Dominic L. Hilsbos, MBA
> Director - Information Technology
> Perform Air International Inc.
> dhils...@performair.com
> www.PerformAir.com
>
>
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
--
Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103
http://www.redhat.com/en/technologies/storage
tel. 734-821-5101
fax. 734-769-8938
cel. 734-216-5309
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] RADOSGW S3 - Continuation Token Ignored?
Hi Dominic,
The reason is likely that RGW doesn't yet support ListObjectsV2.
Support is nearly here though: https://github.com/ceph/ceph/pull/28102
Matt
On Fri, Jun 28, 2019 at 12:43 PM wrote:
>
> All;
>
> I've got a RADOSGW instance setup, backed by my demonstration Ceph cluster.
> I'm using Amazon's S3 SDK, and I've run into an annoying little snag.
>
> My code looks like this:
> amazonS3 = builder.build();
>
> ListObjectsV2Request req = new
> ListObjectsV2Request().withBucketName("WorkOrder").withMaxKeys(MAX_KEYS);
> ListObjectsV2Result result;
>
> do
> {
> result = amazonS3.listObjectsV2(req);
>
> for (S3ObjectSummary objectSummary : result.getObjectSummaries())
> {
> summaries.add(objectSummary);
> }
>
> String token = result.getNextContinuationToken();
> req.setContinuationToken(token);
> }
> while (result.isTruncated());
>
> The problem is, the ContinuationToken seems to be ignored, i.e. every call to
> amazonS3.listObjectsV2(req) returns the same set, and the loop never ends
> (until the summaries LinkedList overflows).
>
> Thoughts?
>
> Thank you,
>
> Dominic L. Hilsbos, MBA
> Director - Information Technology
> Perform Air International Inc.
> dhils...@performair.com
> www.PerformAir.com
>
>
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
--
Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103
http://www.redhat.com/en/technologies/storage
tel. 734-821-5101
fax. 734-769-8938
cel. 734-216-5309
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] RADOSGW S3 - Continuation Token Ignored?
All;
I've got a RADOSGW instance setup, backed by my demonstration Ceph cluster.
I'm using Amazon's S3 SDK, and I've run into an annoying little snag.
My code looks like this:
amazonS3 = builder.build();
ListObjectsV2Request req = new
ListObjectsV2Request().withBucketName("WorkOrder").withMaxKeys(MAX_KEYS);
ListObjectsV2Result result;
do
{
result = amazonS3.listObjectsV2(req);
for (S3ObjectSummary objectSummary : result.getObjectSummaries())
{
summaries.add(objectSummary);
}
String token = result.getNextContinuationToken();
req.setContinuationToken(token);
}
while (result.isTruncated());
The problem is, the ContinuationToken seems to be ignored, i.e. every call to
amazonS3.listObjectsV2(req) returns the same set, and the loop never ends
(until the summaries LinkedList overflows).
Thoughts?
Thank you,
Dominic L. Hilsbos, MBA
Director - Information Technology
Perform Air International Inc.
dhils...@performair.com
www.PerformAir.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
