Thanks!
On Wed, Jan 23, 2013 at 9:31 PM, Sage Weil s...@inktank.com wrote:
Try wip-rados-get
On Wed, 23 Jan 2013, Sage Weil wrote:
Hi Nick-
The problem here looks to just be that do_get() in rados.cc isn't making
any attempt to read large objects in chunks. I'm not sure where the 2GB
This seems to be working ok for the most part, but I noticed that
using large files gives errors getting them (but not putting them).
The problems start after 2GB which, as you said, is larger than should
be used in this method. It shouldn't affect us since we shouldn't be
using this for files
Hi Nick-
The problem here looks to just be that do_get() in rados.cc isn't making
any attempt to read large objects in chunks. I'm not sure where the 2GB
limit is, but it well beyond non-optimal before it gets to that point.
That function needs to read in chunks of a few MB and keep going
Try wip-rados-get
On Wed, 23 Jan 2013, Sage Weil wrote:
Hi Nick-
The problem here looks to just be that do_get() in rados.cc isn't making
any attempt to read large objects in chunks. I'm not sure where the 2GB
limit is, but it well beyond non-optimal before it gets to that point.
Assuming that the clone is atomic so that the client only ever grabbed
a complete old or new version of the file, that method really seems
ideal. How much work/time would that be?
The objects will likely average around 10-20MB, but it's possible that
in some cases they may grow to a few hundred
I had thought about doing something like that, but I'm not sure how to
do it in a race-free way. For example if I was to set 'done=yes' on a
file, then check that before trying to download the file, the instant
I try to download the file the writer of the file could remove the
xattr and start
On Tue, 22 Jan 2013, Nick Bartos wrote:
Assuming that the clone is atomic so that the client only ever grabbed
a complete old or new version of the file, that method really seems
ideal. How much work/time would that be?
The objects will likely average around 10-20MB, but it's possible that
On Tue, 22 Jan 2013, Sage Weil wrote:
On Tue, 22 Jan 2013, Nick Bartos wrote:
Assuming that the clone is atomic so that the client only ever grabbed
a complete old or new version of the file, that method really seems
ideal. How much work/time would that be?
The objects will likely
Thanks! Is it safe to just apply that last commit to 0.56.1? Also,
is the rados command 'clonedata' instead of 'clone'? That's what it
looked like in the code.
On Tue, Jan 22, 2013 at 9:27 AM, Sage Weil s...@inktank.com wrote:
On Tue, 22 Jan 2013, Nick Bartos wrote:
Assuming that the clone
On Tue, 22 Jan 2013, Nick Bartos wrote:
Thanks! Is it safe to just apply that last commit to 0.56.1? Also,
is the rados command 'clonedata' instead of 'clone'? That's what it
looked like in the code.
Yep, and yep!
s
On Tue, Jan 22, 2013 at 9:27 AM, Sage Weil s...@inktank.com wrote:
I would like to store some objects in rados, and retrieve them in a
consistent manor. In my initial tests, if I do a 'rados -p foo put
test /tmp/test', while it is uploading I can do a 'rados -p foo get
test /tmp/blah' on another machine, and it will download a partially
written file without
On Monday, January 21, 2013 at 5:01 PM, Nick Bartos wrote:
I would like to store some objects in rados, and retrieve them in a
consistent manor. In my initial tests, if I do a 'rados -p foo put
test /tmp/test', while it is uploading I can do a 'rados -p foo get
test /tmp/blah' on another
On Mon, 21 Jan 2013, Gregory Farnum wrote:
On Monday, January 21, 2013 at 5:01 PM, Nick Bartos wrote:
I would like to store some objects in rados, and retrieve them in a
consistent manor. In my initial tests, if I do a 'rados -p foo put
test /tmp/test', while it is uploading I can do a
13 matches
Mail list logo
