Re: [ceph-users] osd max write size and large objects

[Wido den Hollander]

> Op 16 november 2017 om 16:46 schreef Robert Stanford 
> :
> 
> 
>  Thanks.  Does this mean that I can send >90MB objects to my RGWs and they
> will break it up into manageable (<=90MB) chunks before storing them?  Or
> if I'm going to store objects > 90MB do I need to change this parameter?
> 

Yes, the RGW breaks it up into 4MB chunks and writes that to RADOS.

In your case, don't worry about this, upload a 100MB object to RGW and it will 
take care of the rest.

>  I don't know if we'll be able to use libradosstriper, but thanks for
> bringing it to my attention.  We are using the S3 interface of RGW
> exclusively (nothing custom in there).
> 

Only a custom application will use libradosstriper. So don't worry.

Wido

> 
> On Thu, Nov 16, 2017 at 9:41 AM, Wido den Hollander  wrote:
> 
> >
> > > Op 16 november 2017 om 16:32 schreef Robert Stanford <
> > rstanford8...@gmail.com>:
> > >
> > >
> > >  Once 'osd max write size' (90MB by default I believe) is exceeded, does
> > > Ceph reject the object (which is coming in through RGW), or does it break
> > > it up into smaller objects (of max 'osd  max write size' size)?  If it
> > > breaks them up, does it read the fragments in parallel when they're
> > > requested by RGW?
> > >
> >
> > The OSD rejects the write. However, RGW already does striping over RADOS
> > objects.
> >
> > Usually you don't need to worry about this setting unless you are talking
> > to RADOS directly.
> >
> > However, 90MB objects are big and you should look into libradosstriper.
> >
> > Wido
> >
> > >  Thanks
> > > ___
> > > ceph-users mailing list
> > > ceph-users@lists.ceph.com
> > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] osd max write size and large objects

[Robert Stanford]

 Thanks.  Does this mean that I can send >90MB objects to my RGWs and they
will break it up into manageable (<=90MB) chunks before storing them?  Or
if I'm going to store objects > 90MB do I need to change this parameter?

 I don't know if we'll be able to use libradosstriper, but thanks for
bringing it to my attention.  We are using the S3 interface of RGW
exclusively (nothing custom in there).


On Thu, Nov 16, 2017 at 9:41 AM, Wido den Hollander  wrote:

>
> > Op 16 november 2017 om 16:32 schreef Robert Stanford <
> rstanford8...@gmail.com>:
> >
> >
> >  Once 'osd max write size' (90MB by default I believe) is exceeded, does
> > Ceph reject the object (which is coming in through RGW), or does it break
> > it up into smaller objects (of max 'osd  max write size' size)?  If it
> > breaks them up, does it read the fragments in parallel when they're
> > requested by RGW?
> >
>
> The OSD rejects the write. However, RGW already does striping over RADOS
> objects.
>
> Usually you don't need to worry about this setting unless you are talking
> to RADOS directly.
>
> However, 90MB objects are big and you should look into libradosstriper.
>
> Wido
>
> >  Thanks
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] osd max write size and large objects

[Wido den Hollander]

> Op 16 november 2017 om 16:32 schreef Robert Stanford 
> :
> 
> 
>  Once 'osd max write size' (90MB by default I believe) is exceeded, does
> Ceph reject the object (which is coming in through RGW), or does it break
> it up into smaller objects (of max 'osd  max write size' size)?  If it
> breaks them up, does it read the fragments in parallel when they're
> requested by RGW?
> 

The OSD rejects the write. However, RGW already does striping over RADOS 
objects.

Usually you don't need to worry about this setting unless you are talking to 
RADOS directly.

However, 90MB objects are big and you should look into libradosstriper.

Wido

>  Thanks
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] osd max write size and large objects

[Robert Stanford]

 Once 'osd max write size' (90MB by default I believe) is exceeded, does
Ceph reject the object (which is coming in through RGW), or does it break
it up into smaller objects (of max 'osd  max write size' size)?  If it
breaks them up, does it read the fragments in parallel when they're
requested by RGW?

 Thanks
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com