Re: [openstack-dev] swift3 Plugin Development

[Venkata R Edara]

Thanks for your reply. The reason we are looking into swift3 is we 
already integrated Gluster with openstack-swift which is providing obj 
store


(https://github.com/gluster/gluster-swift ). integration with RGW can be 
long-term solution, but for short term we would like to have ACL's support.


-Venkat

On 06/10/2017 01:23 AM, Pete Zaitcev wrote:

On Fri, 9 Jun 2017 10:37:15 +0530
Niels de Vos  wrote:


we are looking for S3 plugin with ACLS so that we can integrate gluster
with that.

Did you look into porting Ceph RGW on top of Gluster?

This is one of the longer term options that we have under consideration.
I am very interested in your reasons to suggest it, care to elaborate a
little?

RGW seems like the least worst starting point in terms of the end
result you're likely to get.

The swift3 does a good job for us in OpenStack Swift, providing a degree
of compatibility with S3. When Kota et.al. took over from Tomo, they revived
the development successfully. However, it remains fundamentally limited in
what it does, and its main function is to massage S3 to fit it on top
of Swift. If you place it in front of Gluster, you're saddled with
this fundamental incompatibility, unless you fork swift3 and rework it
beyond recognition.

In addition, surely you realize that swift3 is only a shim and you need
to have an object store to back it. Do you even have one in Gluster?

Fedora used to ship a self-contained S3 store "tabled", so unlike swift3
it's complete. It's written in C, so may be better compatible with Gluster's
development environment. However, it was out of development for years and
it only supports canned ACL. You aren't getting the full ACLs with it that
you're after.

The RGW gives you all that. It's well-compatible with S3, because it is
its native API (with Swift API being grafted on). Yehuda and crea maintain
a good compatibility. Yes, it's in C++, but the dialect is reasonable,
The worst downside is, yes, it's wedded to Ceph's RADOS and you need
a major surgery to place it on top of Gluster. Nonetheless, it seems like
a better defined task to me than trying to maintain your own webserver,
which you must do if you select swift3.

There are still some parts of RGW which will give you trouble. In particular,
it uses loadable classes, which run in the context of Ceph OSD. There's no
place in Gluster to run them. You may have to drag parts of OSD into the
project. But I didn't look closely enough to determine the feasibility.

In your shoes, I'd talk to Yehuda about this. He knows the problem domain
exceptionally and will give you a good advice, even though you're a
competitor in Open Source in general. Kinda like I do now :-)

Cheers,
-- Pete



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] swift3 Plugin Development

[Pete Zaitcev]

On Fri, 9 Jun 2017 10:37:15 +0530
Niels de Vos  wrote:

> > > we are looking for S3 plugin with ACLS so that we can integrate gluster 
> > > with that.
> > 
> > Did you look into porting Ceph RGW on top of Gluster?
> 
> This is one of the longer term options that we have under consideration.
> I am very interested in your reasons to suggest it, care to elaborate a
> little?

RGW seems like the least worst starting point in terms of the end
result you're likely to get.

The swift3 does a good job for us in OpenStack Swift, providing a degree
of compatibility with S3. When Kota et.al. took over from Tomo, they revived
the development successfully. However, it remains fundamentally limited in
what it does, and its main function is to massage S3 to fit it on top
of Swift. If you place it in front of Gluster, you're saddled with
this fundamental incompatibility, unless you fork swift3 and rework it
beyond recognition.

In addition, surely you realize that swift3 is only a shim and you need
to have an object store to back it. Do you even have one in Gluster?

Fedora used to ship a self-contained S3 store "tabled", so unlike swift3
it's complete. It's written in C, so may be better compatible with Gluster's
development environment. However, it was out of development for years and
it only supports canned ACL. You aren't getting the full ACLs with it that
you're after.

The RGW gives you all that. It's well-compatible with S3, because it is
its native API (with Swift API being grafted on). Yehuda and crea maintain
a good compatibility. Yes, it's in C++, but the dialect is reasonable,
The worst downside is, yes, it's wedded to Ceph's RADOS and you need
a major surgery to place it on top of Gluster. Nonetheless, it seems like
a better defined task to me than trying to maintain your own webserver,
which you must do if you select swift3.

There are still some parts of RGW which will give you trouble. In particular,
it uses loadable classes, which run in the context of Ceph OSD. There's no
place in Gluster to run them. You may have to drag parts of OSD into the
project. But I didn't look closely enough to determine the feasibility.

In your shoes, I'd talk to Yehuda about this. He knows the problem domain
exceptionally and will give you a good advice, even though you're a
competitor in Open Source in general. Kinda like I do now :-)

Cheers,
-- Pete

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Swift3 Plugin Development

[Nicolas Trangez]

On Thu, 2017-06-08 at 17:06 +0530, Venkata R Edara wrote:
> Hello,
> 
> we  have storage product called Gluster which is file storage system,
> we 
> are looking to support S3 APIs for it.

Hello Venkata,

Did you consider using the S3 Server project [1] to implement this
functionality? S3 Server supports object and bucket ACLs since its very
first release and was designed to provide a fully compatible AWS S3 API
(including e.g. object versioning) on top of existing storage systems
like Scality RING, Docker volumes and other cloud storage providers.
It's a fully open-source project, under the Apache-2 license.

See https://github.com/Scality/S3 for code and http://s3.scality.com
for some more background.

I believe it should be easy to integrate with Gluster, as it's meant to
have plugable meta-data and data back-ends.
 
Feel free to get in touch if you have any questions or would like to
discuss how to move forward with this project, we're happy to help and
collaborate!

Cheers,

Nicolas

-- 
 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Swift3 Plugin Development

[Niels de Vos]

On Thu, Jun 08, 2017 at 10:52:53PM -0600, Pete Zaitcev wrote:
> On Thu, 8 Jun 2017 17:06:02 +0530
> Venkata R Edara  wrote:
> 
> > we are looking for S3 plugin with ACLS so that we can integrate gluster 
> > with that.
> 
> Did you look into porting Ceph RGW on top of Gluster?

This is one of the longer term options that we have under consideration.
I am very interested in your reasons to suggest it, care to elaborate a
little?

Thanks,
Niels


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Swift3 Plugin Development

[Pete Zaitcev]

On Thu, 8 Jun 2017 17:06:02 +0530
Venkata R Edara  wrote:

> we are looking for S3 plugin with ACLS so that we can integrate gluster 
> with that.

Did you look into porting Ceph RGW on top of Gluster?

-- P

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Swift3 Plugin Development

[Venkata R Edara]

Hello,

we  have storage product called Gluster which is file storage system, we 
are looking to support S3 APIs for it.


openstack-swift project has the swift3 plugin to support S3 APIs , but 
according to https://github.com/openstack/swift3/blob/master/README.md


we see that S3 ACL's are still under development and not in production 
so far.


we are looking for S3 plugin with ACLS so that we can integrate gluster 
with that.


Are there any plans from dev community to develop full featured S3 ACL's 
into swift3 plugin? . if yes, please let us know the release version and 
time-frame.


Thanks in Advance

-Venkata R Edara





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev