Hi, > It was a 256 gig file > ERROR: S3 error: 416 (InvalidRange): I think this is a "MaxParts" default limit 1000, s3cmd default part size 15MB (see .s3cfg). I cant find right way how to change this limit on rgw side, so i change part size in .s3cfg or use multipart tar archive for large files.
-- Regards, Mikhail On Tue, 8 Apr 2014 12:52:52 +0200 Benedikt Fraunhofer <[email protected]> wrote: > Hello Yehuda, > > 2014-04-04 9:35 GMT+02:00 Benedikt Fraunhofer > <[email protected]>: > > > I'm currently trying to find a faster box with enough free space to > > reproduce that and capture logs. > > Here's the complete log with "debug rgw 20" and "debug ms 1" of a > failed large-ish multipart upload. > > As it's 71mb gzipped i placed it here: > http://www.traced.net/PMhpycoXcb9OirMzGQfaHzoQaC/radosgw11.obfus.log.gz > > 2014-04-08 10:57:14.328232 7f37907d8700 10 uid=bf2.0 requested perm > (type)=2, policy perm=2, user_perm_mask=2, acl perm=2 > 2014-04-08 10:57:14.328233 7f37907d8700 2 req 17480:0.000196:s3: > POST /file-262144:complete_multipart:verifying op params > 2014-04-08 10:57:14.328234 7f37907d8700 2 req 17480:0.000197:s3: > POST /file-262144:complete_multipart:executing > 2014-04-08 10:57:14.328284 7f37907d8700 2 req 17480:0.000247:s3: > POST /file-262144:complete_multipart:http status=416 > 2014-04-08 10:57:14.328289 7f37907d8700 1 ====== req done > req=0x7f37b80132d0 http_status=416 ====== > 2014-04-08 10:57:14.328290 7f37907d8700 20 process_request() returned > -34 > > this is what s3cmd outputs: > > file-262144 -> s3://7251b77d-ae36-449a-bb8a-99bd47f18947/file-262144 > [part 17476 of 17477, 15MB] > 15728640 of 15728640 100% in 0s 80.82 MB/s done > file-262144 -> s3://7251b77d-ae36-449a-bb8a-99bd47f18947/file-262144 > [part 17477 of 17477, 4MB] > 4194304 of 4194304 100% in 0s 56.07 MB/s done > ERROR: S3 error: 416 (InvalidRange): > > It was a 256 gig file, the script does no magic and is rather silly: > dd 256gig from /dev/zero to a file and upload it using s3cmd. > ======================== > #! /bin/bash > > set -x > > SIZE=$((256*1024)) > FN=file-${SIZE} > S3CMD=s3cmd > S3CMD=/home/bf/s3cmd-1.5.0-beta1/s3cmd > MULTIPART="--disable-multipart" > MULTIPART="" > > BUCK="s3://$(uuidgen)/" > if [ ! -f $FN ]; then > dd if=/dev/zero of=$FN bs=4M count=$(($SIZE/4)) & > fi > $S3CMD mb $BUCK > wait > > $S3CMD put $MULTIPART $FN $BUCK > #sleep 1 > #$S3CMD get --force ${BUCK}${FN} file1.1 > ======================== > > Thx > Benedikt > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
