Re: [ceph-users] do RGW have billing feature? If have, how do we use it ?

2014-08-27 Thread Yehuda Sadeh
There's the usage log that can be turned on and is in the granularity of 1 hour. It basically records amount of data transferred, bucket name, number of operations, and types of operations. Yehuda On Wed, Aug 27, 2014 at 1:59 PM, Craig Lewis cle...@centraldesktop.com wrote: Not directly, no.

Re: [ceph-users] Ceph object back up details

2014-09-08 Thread Yehuda Sadeh
Not sure I understand what you ask. Multiple zones within the same region configuration is described here: http://ceph.com/docs/master/radosgw/federated-config/#multi-site-data-replication Yehuda On Sun, Sep 7, 2014 at 10:32 PM, M Ranga Swami Reddy swamire...@gmail.com wrote: Hi Yahuda, I

Re: [ceph-users] why no likely() and unlikely() used in Ceph's source code?

2014-09-15 Thread Yehuda Sadeh
I agree with Greg. When dealing with the latencies that we deal with due to different IO operations (networking, storage), it's mostly not worth the trouble. I think the main reason we didn't actually put it to use is that we forgot we've had this macro defined, and it really wasn't worth the

Re: [ceph-users] Any way to remove possible orphaned files in a federated gateway configuration

2014-09-23 Thread Yehuda Sadeh
On Tue, Sep 23, 2014 at 3:05 PM, Lyn Mitchell mitc...@bellsouth.net wrote: Is anyone aware of a way to either reconcile or remove possible orphaned “shadow” files in a federated gateway configuration? The issue we’re seeing is the number of chunks/shadow files on the slave has many more

Re: [ceph-users] Merging two active ceph clusters: suggestions needed

2014-09-23 Thread Yehuda Sadeh
On Tue, Sep 23, 2014 at 7:23 PM, Robin H. Johnson robb...@gentoo.org wrote: On Tue, Sep 23, 2014 at 03:12:53PM -0600, John Nielsen wrote: Keep Cluster A intact and migrate it to your new hardware. You can do this with no downtime, assuming you have enough IOPS to support data migration and

Re: [ceph-users] Resetting RGW Federated replication

2014-09-23 Thread Yehuda Sadeh
On Tue, Sep 23, 2014 at 4:54 PM, Craig Lewis cle...@centraldesktop.com wrote: I've had some issues in my secondary cluster. I'd like to restart replication from the beginning, without destroying the data in the secondary cluster. Reading the radosgw-agent and Admin REST API code, I believe I

Re: [ceph-users] Merging two active ceph clusters: suggestions needed

2014-09-24 Thread Yehuda Sadeh
the primary zone. Yehuda Robin, are the mtimes in Cluster B's S3 data important? Just wondering if it would be easier to move the data from B to A, and move nodes from B to A as B shrinks. Then remove the old A nodes when it's all done. On Tue, Sep 23, 2014 at 10:04 PM, Yehuda Sadeh yeh

Re: [ceph-users] Merging two active ceph clusters: suggestions needed

2014-09-24 Thread Yehuda Sadeh
On Wed, Sep 24, 2014 at 2:12 PM, Robin H. Johnson robb...@gentoo.org wrote: On Wed, Sep 24, 2014 at 11:31:29AM -0700, Yehuda Sadeh wrote: On Wed, Sep 24, 2014 at 11:17 AM, Craig Lewis cle...@centraldesktop.com wrote: Yehuda, are there any potential problems there? I'm wondering if duplicate

Re: [ceph-users] Any way to remove possible orphaned files in a federated gateway configuration

2014-09-26 Thread Yehuda Sadeh
number of orphaned shadow files that are currently in the .region-1.zone-2.rgw.buckets pool are from the original sync performed back on Sept. 15. Thanks in advance, MLM -Original Message- From: yehud...@gmail.com [mailto:yehud...@gmail.com] On Behalf Of Yehuda Sadeh Sent: Tuesday

Re: [ceph-users] failed to sync object

2014-09-29 Thread Yehuda Sadeh
On Mon, Sep 29, 2014 at 10:44 AM, Lyn Mitchell mitc...@bellsouth.net wrote: Hello ceph users, We have a federated gateway configured to replicate between two zones. Replication seems to be working smoothly between the master and slave zone, however I have a recurring error in the

Re: [ceph-users] Can you assign ACLs to a virtual directory using Object Gateway's S3 API?

2014-09-30 Thread Yehuda Sadeh
On Tue, Sep 23, 2014 at 9:20 AM, Steve Kingsland steve.kingsl...@opower.com wrote: Using the S3 API to Object Gateway, let's say that I create an object named /some/path/foo.bar. When I browse this object in Ceph using a graphical S3 client, some and path show up as directories. I realize that

Re: [ceph-users] endpoints used during synchronization

2014-10-01 Thread Yehuda Sadeh
The agent itself only goes to the gateways it was configured to use. However, in a cross zone copy of objects, the gateway will round robin to any of the specified endpoints in its regionmap. Yehuda On Wed, Oct 1, 2014 at 3:46 PM, Lyn Mitchell mitc...@bellsouth.net wrote: Sorry all for the

Re: [ceph-users] v0.80.6 Firefly released

2014-10-01 Thread Yehuda Sadeh
pool alignment (#8442 Yehuda Sadeh) * rgw: don't send error body on HEAD for civetweb (#8539 Yehuda Sadeh) * rgw: fix crash in CORS preflight request (Yehuda Sadeh) * rgw: fix decoding of + in URL (#8702 Yehuda Sadeh) #8702 was actually fixed by Brian Rak. Yehuda

Re: [ceph-users] max_bucket limit -- safe to disable?

2014-10-06 Thread Yehuda Sadeh
It'd be interesting to see which rados operation is slowing down the requests. Can you provide a log dump of a request (with 'debug rgw = 20', and 'debug ms = 1'). This might give us a better idea as to what's going on. Thanks, Yehuda On Mon, Oct 6, 2014 at 10:05 AM, Daniel Schneller

Re: [ceph-users] max_bucket limit -- safe to disable?

2014-10-07 Thread Yehuda Sadeh
...@centerdevice.com | www.centerdevice.com On 06 Oct 2014, at 19:26, Yehuda Sadeh yeh...@redhat.com wrote: It'd be interesting to see which rados operation is slowing down the requests. Can you provide a log dump of a request (with 'debug rgw = 20', and 'debug ms = 1'). This might give us

Re: [ceph-users] max_bucket limit -- safe to disable?

2014-10-07 Thread Yehuda Sadeh
, I will not post the logs inline, but only attach them gzipped. As before, should the full data set be needed, I can provide an archived version. Thanks for your support! Daniel On 07 Oct 2014, at 22:45, Yehuda Sadeh yeh...@redhat.com wrote: The logs here don't include

Re: [ceph-users] RadosGW over HTTPS

2014-10-08 Thread Yehuda Sadeh
Try passing in 'Server-Port-Secure: 443' header to the auth request. Yehuda On Wed, Oct 8, 2014 at 7:41 AM, Marco Garcês ma...@garces.cc wrote: Hi David, I am indeed using Tengine 2.0.3, but I feel very strange that the default config is returning X-Storage-Url in the headers, in http, not

Re: [ceph-users] RadosGW over HTTPS

2014-10-08 Thread Yehuda Sadeh
: AUTH_rgwtk100066726f6e74656e643a737766303030323daad73c8234e91dfba33654a8ca962d64f0f2d492b4ec5b79aee87ac454bd38406d3bee X-Auth-Token: AUTH_rgwtk100066726f6e74656e643a737766303030323daad73c8234e91dfba33654a8ca962d64f0f2d492b4ec5b79aee87ac454bd38406d3bee On Wed, Oct 8, 2014 at 5:01 PM, Yehuda Sadeh yeh...@redhat.com wrote: Server-Port-Secure: 443 Marco

Re: [ceph-users] RadosGW over HTTPS

2014-10-08 Thread Yehuda Sadeh
WARNING: RGWRados::log_usage(): user name empty (bucket=), skipping On Wed, Oct 8, 2014 at 5:46 PM, Yehuda Sadeh yeh...@redhat.com wrote: debug rgw = 20 I Marco Garcês #sysadmin Maputo - Mozambique ___ ceph-users mailing list ceph-users

Re: [ceph-users] max_bucket limit -- safe to disable?

2014-10-08 Thread Yehuda Sadeh
: 8,355,934 wr KB: 0 .rgw.root = KB: 1 objects:3 rd: 524 rd KB: 346 wr: 3 wr KB: 3 Daniel On 08 Oct 2014, at 01:03, Yehuda Sadeh yeh...@redhat.com wrote

Re: [ceph-users] max_bucket limit -- safe to disable?

2014-10-08 Thread Yehuda Sadeh
On Wed, Oct 8, 2014 at 10:00 AM, Daniel Schneller daniel.schnel...@centerdevice.com wrote: Ok. How can I tell if stuff is stuck in a queue? What to look for? Correlate a slow request/response that you see in the rgw log to the same in the osd log. Check how much time it the osd thought it took

Re: [ceph-users] Rados Gateway and Swift create containers/buckets that cannot be opened

2014-10-09 Thread Yehuda Sadeh
I have a trivial fix for the issue that I'd like to check and get this one cleared, but never got to it due to some difficulties with a proper keystone setup in my environment. If you can and would like to test it so that we could get it merged it would be great. Thanks, Yehuda On Wed, Oct 8,

Re: [ceph-users] Rados Gateway and Swift create containers/buckets that cannot be opened

2014-10-09 Thread Yehuda Sadeh
share the fix/patch, we could test and confirm the fix status. Thanks Swami On Thu, Oct 9, 2014 at 10:42 PM, Yehuda Sadeh yeh...@redhat.com wrote: I have a trivial fix for the issue that I'd like to check and get this one cleared, but never got to it due to some difficulties with a proper

Re: [ceph-users] Rados Gateway and Swift create containers/buckets that cannot be opened

2014-10-09 Thread Yehuda Sadeh
. Installing a new patched radosgw binary and restarting got back a working swift. Cheers Mark On 10/10/14 07:19, Yehuda Sadeh wrote: Here's the fix, let me know if you need any help with that. Thanks, Yehuda diff --git a/src/rgw/rgw_swift.cc b/src/rgw/rgw_swift.cc index d9654a7..2445e17

Re: [ceph-users] rados gateway pools for users

2014-10-16 Thread Yehuda Sadeh
See this discussion: http://comments.gmane.org/gmane.comp.file-systems.ceph.user/4992 Yehuda On Thu, Oct 16, 2014 at 12:11 AM, Shashank Puntamkar spuntam...@gmail.com wrote: I am planning to use ceph object gateway to store data in ceph cluster.I need two different users of Rados gateway to

Re: [ceph-users] RGW Federated Gateways and Apache 2.4 problems

2014-10-24 Thread Yehuda Sadeh
On Thu, Oct 23, 2014 at 3:51 PM, Craig Lewis cle...@centraldesktop.com wrote: I'm having a problem getting RadosGW replication to work after upgrading to Apache 2.4 on my primary test cluster. Upgrading the secondary cluster to Apache 2.4 doesn't cause any problems. Both Ceph's apache packages

Re: [ceph-users] Object Storage Statistics

2014-10-24 Thread Yehuda Sadeh
On Fri, Oct 24, 2014 at 8:17 AM, Dane Elwell dane.elw...@gmail.com wrote: Hi list, We're using the object storage in production and billing people based on their usage, much like S3. We're also trying to produce things like hourly bandwidth graphs for our clients. We're having some issues

Re: [ceph-users] HTTP Get returns 404 Not Found for Swift API

2014-10-29 Thread Yehuda Sadeh
On Tue, Oct 28, 2014 at 2:23 PM, Pedro Miranda potter...@gmail.com wrote: Hi I'm new using Ceph and I have a very basic Ceph cluster with 1 mon in one node and 2 OSDs in two separate nodes (all CentOS 7). I followed the quick-ceph-deploy tutorial. All went well. Then I started the quick-rgw

Re: [ceph-users] 500 Internal Server Error when aborting large multipart upload through object storage

2014-10-31 Thread Yehuda Sadeh
On Fri, Oct 31, 2014 at 3:59 AM, Dane Elwell dane.elw...@gmail.com wrote: Hello list, When we upload a large multipart upload to RGW and it fails, we want to abort the upload. On large multipart uploads, with say 1000+ parts, it will consistently return 500 errors when trying to abort the

Re: [ceph-users] RADOSGW Logs

2014-10-31 Thread Yehuda Sadeh
On Fri, Oct 31, 2014 at 8:06 AM, Dane Elwell dane.elw...@gmail.com wrote: I think I may have answered my own question: http://tracker.ceph.com/issues/8553 Looks like this is fixed in Giant, which we'll be deploying as soon as 0.87.1 is out ;) Thanks Dane On 31 October 2014 09:08, Dane

Re: [ceph-users] logging, radosgw and pools questions

2014-10-31 Thread Yehuda Sadeh
On Fri, Oct 31, 2014 at 9:48 AM, Marco Garcês ma...@garces.cc wrote: Hi there, I have a few questions regarding pools, radosgw and logging: 1) How do I turn on radosgw logs for a specific pool? What do you mean? What do you want to log? I have this in my config: rgw enable ops log =

Re: [ceph-users] Swift + radosgw: How do I find accounts/containers/objects limitation?

2014-11-03 Thread Yehuda Sadeh
On Mon, Nov 3, 2014 at 9:37 AM, Narendra Trivedi (natrived) natri...@cisco.com wrote: Thanks. I think the limit is 100 by default and it can be disabled. As far as I understand, there are no object limit on radosgw side of things only from Swift end (i.e. 5GB) ….right? In short, if someone

Re: [ceph-users] Ceph Cluster with two radosgw

2014-11-07 Thread Yehuda Sadeh
On Wed, Nov 5, 2014 at 2:08 PM, lakshmi k s lux...@yahoo.com wrote: Hello - My ceph cluster needs to have two rados gateway nodes eventually interfacing with Openstack haproxy. I have been successful in bringing up one of them. What are the steps for additional rados gateway node to be

Re: [ceph-users] Creating RGW S3 User using the Admin Ops API

2014-11-15 Thread Yehuda Sadeh
On Sat, Nov 15, 2014 at 6:20 AM, Wido den Hollander w...@42on.com wrote: Hi, I'm having trouble with creating a new user using the Admin Ops API and I'm not sure where the problem lies. I'm using: http://eu.ceph.com/docs/master/radosgw/adminops/#create-user Using pycurl I send the

Re: [ceph-users] Creating RGW S3 User using the Admin Ops API

2014-11-17 Thread Yehuda Sadeh
On Sun, Nov 16, 2014 at 10:50 PM, Wido den Hollander w...@42on.com wrote: On 17-11-14 07:44, Lei Dong wrote: I think you should send the data (uid display-name) as arguments. I successfully create user via adminOps without any problems. To be clear: PUT

Re: [ceph-users] Radosgw agent only syncing metadata

2014-11-24 Thread Yehuda Sadeh
On Mon, Nov 24, 2014 at 2:43 PM, Mark Kirkwood mark.kirkw...@catalyst.net.nz wrote: On 22/11/14 10:54, Yehuda Sadeh wrote: On Thu, Nov 20, 2014 at 6:52 PM, Mark Kirkwood mark.kirkw...@catalyst.net.nz wrote: Fri Nov 21 02:13:31 2014 x-amz-copy-source:bucketbig/_multipart_big.dat.2

Re: [ceph-users] Deleting buckets and objects fails to reduce reported cluster usage

2014-11-26 Thread Yehuda Sadeh
On Wed, Nov 26, 2014 at 2:32 PM, b b@benjackson.email wrote: I've been deleting a bucket which originally had 60TB of data in it, with our cluster doing only 1 replication, the total usage was 120TB. I've been deleting the objects slowly using S3 browser, and I can see the bucket usage is now

Re: [ceph-users] Deleting buckets and objects fails to reduce reported cluster usage

2014-11-26 Thread Yehuda Sadeh
On Wed, Nov 26, 2014 at 3:49 PM, b b@benjackson.email wrote: On 2014-11-27 10:21, Yehuda Sadeh wrote: On Wed, Nov 26, 2014 at 3:09 PM, b b@benjackson.email wrote: On 2014-11-27 09:38, Yehuda Sadeh wrote: On Wed, Nov 26, 2014 at 2:32 PM, b b@benjackson.email wrote: I've been deleting

Re: [ceph-users] Deleting buckets and objects fails to reduce reported cluster usage

2014-11-27 Thread Yehuda Sadeh
On Thu, Nov 27, 2014 at 2:15 PM, b b@benjackson.email wrote: On 2014-11-27 11:36, Yehuda Sadeh wrote: On Wed, Nov 26, 2014 at 3:49 PM, b b@benjackson.email wrote: On 2014-11-27 10:21, Yehuda Sadeh wrote: On Wed, Nov 26, 2014 at 3:09 PM, b b@benjackson.email wrote: On 2014-11-27 09:38

Re: [ceph-users] Deleting buckets and objects fails to reduce reported cluster usage

2014-11-28 Thread Yehuda Sadeh
On Thu, Nov 27, 2014 at 9:22 PM, Ben b@benjackson.email wrote: On 2014-11-28 15:42, Yehuda Sadeh wrote: On Thu, Nov 27, 2014 at 2:15 PM, b b@benjackson.email wrote: On 2014-11-27 11:36, Yehuda Sadeh wrote: On Wed, Nov 26, 2014 at 3:49 PM, b b@benjackson.email wrote: On 2014-11-27 10:21

Re: [ceph-users] Deleting buckets and objects fails to reduce reported cluster usage

2014-11-28 Thread Yehuda Sadeh
On Fri, Nov 28, 2014 at 1:38 PM, Ben b@benjackson.email wrote: On 29/11/14 01:50, Yehuda Sadeh wrote: On Thu, Nov 27, 2014 at 9:22 PM, Ben b@benjackson.email wrote: On 2014-11-28 15:42, Yehuda Sadeh wrote: On Thu, Nov 27, 2014 at 2:15 PM, b b@benjackson.email wrote: On 2014-11-27 11:36

Re: [ceph-users] Deleting buckets and objects fails to reduce reported cluster usage

2014-12-01 Thread Yehuda Sadeh
On Sat, Nov 29, 2014 at 2:26 PM, Ben b@benjackson.email wrote: On 29/11/14 11:40, Yehuda Sadeh wrote: On Fri, Nov 28, 2014 at 1:38 PM, Ben b@benjackson.email wrote: On 29/11/14 01:50, Yehuda Sadeh wrote: On Thu, Nov 27, 2014 at 9:22 PM, Ben b@benjackson.email wrote: On 2014-11-28 15:42

Re: [ceph-users] Deleting buckets and objects fails to reduce reported cluster usage

2014-12-01 Thread Yehuda Sadeh
On Mon, Dec 1, 2014 at 2:10 PM, Ben b@benjackson.email wrote: On 2014-12-02 08:39, Yehuda Sadeh wrote: On Sat, Nov 29, 2014 at 2:26 PM, Ben b@benjackson.email wrote: On 29/11/14 11:40, Yehuda Sadeh wrote: On Fri, Nov 28, 2014 at 1:38 PM, Ben b@benjackson.email wrote: On 29/11/14 01:50

Re: [ceph-users] Deleting buckets and objects fails to reduce reported cluster usage

2014-12-01 Thread Yehuda Sadeh
On Mon, Dec 1, 2014 at 3:20 PM, Ben b@benjackson.email wrote: On 2014-12-02 09:25, Yehuda Sadeh wrote: On Mon, Dec 1, 2014 at 2:10 PM, Ben b@benjackson.email wrote: On 2014-12-02 08:39, Yehuda Sadeh wrote: On Sat, Nov 29, 2014 at 2:26 PM, Ben b@benjackson.email wrote: On 29/11/14 11:40

Re: [ceph-users] Deleting buckets and objects fails to reduce reported cluster usage

2014-12-01 Thread Yehuda Sadeh
On Mon, Dec 1, 2014 at 4:23 PM, Ben b@benjackson.email wrote: On 2014-12-02 11:21, Yehuda Sadeh wrote: On Mon, Dec 1, 2014 at 3:47 PM, Ben b@benjackson.email wrote: On 2014-12-02 10:40, Yehuda Sadeh wrote: On Mon, Dec 1, 2014 at 3:20 PM, Ben b@benjackson.email wrote: On 2014-12-02 09:25

Re: [ceph-users] Deleting buckets and objects fails to reduce reported cluster usage

2014-12-01 Thread Yehuda Sadeh
On Mon, Dec 1, 2014 at 3:47 PM, Ben b@benjackson.email wrote: On 2014-12-02 10:40, Yehuda Sadeh wrote: On Mon, Dec 1, 2014 at 3:20 PM, Ben b@benjackson.email wrote: On 2014-12-02 09:25, Yehuda Sadeh wrote: On Mon, Dec 1, 2014 at 2:10 PM, Ben b@benjackson.email wrote: On 2014-12-02 08:39

Re: [ceph-users] Deleting buckets and objects fails to reduce reported cluster usage

2014-12-01 Thread Yehuda Sadeh
On Mon, Dec 1, 2014 at 4:26 PM, Ben b@benjackson.email wrote: On 2014-12-02 11:25, Yehuda Sadeh wrote: On Mon, Dec 1, 2014 at 4:23 PM, Ben b@benjackson.email wrote: ... How can I tell if the shard has an object in it from the logs? Search for a different sequence (e.g., search

Re: [ceph-users] AWS SDK and MultiPart Problem

2014-12-05 Thread Yehuda Sadeh
It looks like a bug. Can you open an issue on tracker.ceph.com, describing what you see? Thanks, Yehuda On Fri, Dec 5, 2014 at 7:17 AM, Georgios Dimitrakakis gior...@acmac.uoc.gr wrote: It would be nice to see where and how uploadId is being calculated... Thanks, George For example

Re: [ceph-users] normalizing radosgw

2014-12-06 Thread Yehuda Sadeh
On Sat, Dec 6, 2014 at 10:39 AM, Sage Weil sw...@redhat.com wrote: Several things are different/annoying with radosgw than with other Ceph daemons: - binary/package are named 'radosgw' instead of 'ceph-rgw'. This is cosmetic, but it also makes it fit less well into the new /var/lib/ceph/*

Re: [ceph-users] AWS SDK and MultiPart Problem

2014-12-11 Thread Yehuda Sadeh
AM, Yehuda Sadeh yeh...@redhat.com wrote: I don't think it has been fixed recently. I'm looking at it now, and not sure why it hasn't triggered before in other areas. Yehuda On Thu, Dec 11, 2014 at 5:55 AM, Georgios Dimitrakakis gior...@acmac.uoc.gr wrote: This issue seems very similar

Re: [ceph-users] AWS SDK and MultiPart Problem

2014-12-11 Thread Yehuda Sadeh
Dimitrakakis wrote: I 've just created issues #10271 Best, George On Fri, 5 Dec 2014 09:30:45 -0800, Yehuda Sadeh wrote: It looks like a bug. Can you open an issue on tracker.ceph.com, describing what you see? Thanks, Yehuda On Fri, Dec 5, 2014 at 7:17 AM, Georgios Dimitrakakis gior

Re: [ceph-users] AWS SDK and MultiPart Problem

2014-12-11 Thread Yehuda Sadeh
Pushed a fix to wip-10271. Haven't tested it though, let me know if you try it. Thanks, Yehuda On Thu, Dec 11, 2014 at 8:38 AM, Yehuda Sadeh yeh...@redhat.com wrote: I don't think it has been fixed recently. I'm looking at it now, and not sure why it hasn't triggered before in other areas

Re: [ceph-users] AWS SDK and MultiPart Problem

2014-12-12 Thread Yehuda Sadeh
try it. Thanks, Yehuda On Thu, Dec 11, 2014 at 8:38 AM, Yehuda Sadeh yeh...@redhat.com wrote: I don't think it has been fixed recently. I'm looking at it now, and not sure why it hasn't triggered before in other areas. Yehuda On Thu, Dec 11, 2014 at 5:55 AM, Georgios Dimitrakakis

Re: [ceph-users] AWS SDK and MultiPart Problem

2014-12-12 Thread Yehuda Sadeh
update package is 0.80.5 Have I missed something? Regards, George Pushed a fix to wip-10271. Haven't tested it though, let me know if you try it. Thanks, Yehuda On Thu, Dec 11, 2014 at 8:38 AM, Yehuda Sadeh yeh...@redhat.com wrote: I don't think it has been fixed recently. I'm

Re: [ceph-users] Radosgw-Agent

2014-12-15 Thread Yehuda Sadeh
There's the 'radosgw-agent' package for debian, e.g., here: http://ceph.com/debian-giant/pool/main/r/radosgw-agent/radosgw-agent_1.2-1~bpo70+1_all.deb On Mon, Dec 15, 2014 at 5:12 AM, lakshmi k s lux...@yahoo.com wrote: Hello - Can anyone help me locate the Debian-type source packages for

Re: [ceph-users] Content-length error uploading big files to radosgw

2014-12-18 Thread Yehuda Sadeh
On Thu, Dec 18, 2014 at 11:24 AM, Gregory Farnum g...@gregs42.com wrote: On Thu, Dec 18, 2014 at 4:04 AM, Daniele Venzano li...@brownhat.org wrote: Hello, I have been trying to upload multi-gigabyte files to CEPH via the object gateway, using both the swift and s3 APIs. With file up to

Re: [ceph-users] radosgw multipart-uploaded downloads fail

2014-04-03 Thread Yehuda Sadeh
Ok, we were able to reproduce it. Opened issue #7978, and there's a fix pending for upstream. Thanks, Yehuda On Wed, Apr 2, 2014 at 7:20 AM, Yehuda Sadeh yeh...@inktank.com wrote: On Wed, Apr 2, 2014 at 2:08 AM, Benedikt Fraunhofer given.to.lists.ceph-users.ceph.com.toasta@traced.net wrote

Re: [ceph-users] RadosGW: bad request

2014-04-07 Thread Yehuda Sadeh
On Mon, Apr 7, 2014 at 5:38 AM, Gandalf Corvotempesta gandalf.corvotempe...@gmail.com wrote: I'm getting these trying to upload any file: 2014-04-07 14:33:27.084369 7f5268f86700 5 Getting permissions id=testuser owner=testuser perm=2 2014-04-07 14:33:27.084372 7f5268f86700 10 uid=testuser

Re: [ceph-users] RadosGW Quotas

2014-04-07 Thread Yehuda Sadeh
On Mon, Apr 7, 2014 at 4:34 PM, Derek Yarnell de...@umiacs.umd.edu wrote: Hi, Most likely Yehuda can speak to this. So there is some documentation on the way to set quotas for a user in master (src/rgw/rgw_rest_user.cc line 712) but these are not in the docs it seems yet. I have started to

Re: [ceph-users] [Ceph-rgw] pool assigment

2014-04-11 Thread Yehuda Sadeh
On Fri, Apr 11, 2014 at 3:20 AM, ghislain.cheval...@orange.com wrote: Hi all, Context : CEPH dumpling on Ubuntu 12.04 I would like to manage as accurately as possible the pools assigned to Rados gateway My goal is to apply specific SLA to applications which use http-driven storage. I 'd

Re: [ceph-users] Radosgw and s3cmd

2014-04-15 Thread Yehuda Sadeh
On Mon, Apr 14, 2014 at 10:12 PM, Shashank Puntamkar spuntam...@gmail.com wrote: I am setting up a ceph cluster with amazon s3 like capabilities. I have configured Ceph Object Gateway, radowgw on Ubuntu12.04 as described in ceph documentation.(http://ceph.com/docs/master/radosgw/config/).

Re: [ceph-users] Radosgw and s3cmd

2014-04-15 Thread Yehuda Sadeh
you will give hint where the error is? You're probably missing the 'rgw dns name' configuration in your ceph.conf. On 4/15/14, Yehuda Sadeh yeh...@inktank.com wrote: On Mon, Apr 14, 2014 at 10:12 PM, Shashank Puntamkar spuntam...@gmail.com wrote: I am setting up a ceph cluster with amazon

Re: [ceph-users] Access denied error

2014-04-16 Thread Yehuda Sadeh
On Tue, Apr 15, 2014 at 11:33 PM, Punit Dambiwal hypu...@gmail.com wrote: Hi, Still i am getting the same error,when i run the following :- -- curl -i 'http://xxx.xlinux.com/admin/usage?format=json' -X GET -H 'Authorization: AWS

Re: [ceph-users] Access denied error

2014-04-21 Thread Yehuda Sadeh
command execution getting an access denied error message. Is there any issues in the above header information ? or could you please explain how we can authenticate using access keys and access tokens ? or how we can pass the header information ? On Thu, Apr 17, 2014 at 12:10 AM, Yehuda Sadeh yeh

Re: [ceph-users] RadosGW Quotas

2014-04-22 Thread Yehuda Sadeh
On Tue, Apr 22, 2014 at 7:51 AM, Derek Yarnell de...@umiacs.umd.edu wrote: Hi Yehuda, So i finally got to testing the quota stuff per the info from lines 712-758 in rgw_rest_user.cc. I am using the query parameters version and getting a 409 error out. Below in the debug log of the gateway

Re: [ceph-users] Operation canceled on first operation after RadosGW restart

2014-04-22 Thread Yehuda Sadeh
On Tue, Apr 22, 2014 at 9:18 AM, Evgeni Golov ego+c...@dg-i.net wrote: Hi, we have a small (3x10GB) ceph cluster for testing, running Ceph 0.72.2 with RadosGW inside an Apache2. All the three nodes are identical: each has one osd, one mon, one radosgw. The application (self-written) uses

Re: [ceph-users] Access denied error

2014-04-24 Thread Yehuda Sadeh
RGWDataChangesLog::ChangesRenewThread: start 2014-04-22 09:37:59.047561 7ff16effd700 2 RGWDataChangesLog::ChangesRenewThread: start Thanks, On Tue, Apr 22, 2014 at 12:42 AM, Yehuda Sadeh yeh...@inktank.com wrote: On Fri, Apr 18, 2014 at 10:07 PM, Punit

Re: [ceph-users] [Bug]radosgw-agent can't sync files with Chinese filename

2014-04-24 Thread Yehuda Sadeh
Hi, sorry for the late response. I opened a ceph tracker issue for it (#8202). Thanks, Yehuda On Wed, Apr 16, 2014 at 1:00 AM, wsnote wsn...@163.com wrote: OS: CentOS 6.5 Version: Ceph 0.67.7 or 0.79 Hello, everyone! I have configured federated gateways for several. Now I can sync files

Re: [ceph-users] [Bug]radosgw-agent can't sync files with Chinese filename

2014-04-25 Thread Yehuda Sadeh
the sync agent issue or the gateway issue. My assumption is that the gateway fails to encode it correctly when sending the cross zones copy request. Yehuda At 2014-04-25 03:32:02,Yehuda Sadeh yeh...@inktank.com wrote: Hi, sorry for the late response. I opened a ceph tracker issue

Re: [ceph-users] Access denied error

2014-04-25 Thread Yehuda Sadeh
On Fri, Apr 25, 2014 at 1:03 AM, Punit Dambiwal hypu...@gmail.com wrote: Hi Yehuda, Thanks for your help...that missing date error gone but still i am getting the access denied error :- - 2014-04-25 15:52:56.988025 7f00d37c6700 1 == starting new request

Re: [ceph-users] Swift API and object versioning

2014-04-27 Thread Yehuda Sadeh
, between S3 and Swift, considering the stability and robustness, what it would be ? I ask because I think you have a whole and precise vision about S3 and Swift, which I havn't, yet ;-) Cheers ! Le 27/04/2014 17:04, Yehuda Sadeh a écrit : We discussed it internally a few days ago, and even

Re: [ceph-users] Swift API and object versioning

2014-04-27 Thread Yehuda Sadeh
We discussed it internally a few days ago, and even created some tickets for future work. The swift object versioning has some differences from the s3 one, and our plan at the moment is have the s3 working first and only then do swift. Yehuda On Sun, Apr 27, 2014 at 3:51 AM, Cedric Lemarchand

Re: [ceph-users] Creating a bucket failed

2014-04-28 Thread Yehuda Sadeh
This could happen if your client is uses the bucket through subdomain scheme, but the rgw is not resolving it correctly (either rgw_dns_name is misconfigured, or you were accessing it through different host name). Yehuda On Mon, Apr 28, 2014 at 2:02 PM, Seowon Jung seo...@hawaii.edu wrote:

Re: [ceph-users] List users not listing users

2014-05-08 Thread Yehuda Sadeh
You can use the metadata api to list the users, something along the lines of: GET /admin/metadata/user?format=json Yehuda On Thu, May 8, 2014 at 9:19 PM, Shanil S xielessha...@gmail.com wrote: Hi Punit, I hope someone in the community forum can solve this issue. I am waiting for their

Re: [ceph-users] List users not listing users

2014-05-08 Thread Yehuda Sadeh
, what do you think ? but i am using the admin user to do this action and users permission as * On Fri, May 9, 2014 at 9:52 AM, Yehuda Sadeh yeh...@inktank.com wrote: You can use the metadata api to list the users, something along the lines of: GET /admin/metadata/user?format=json Yehuda

Re: [ceph-users] List users not listing users

2014-05-08 Thread Yehuda Sadeh
: false, max_size_kb: -1, max_objects: -1}} On Fri, May 9, 2014 at 10:13 AM, Shanil S xielessha...@gmail.com wrote: Hi Yehuda, Okay.. Thanks.. I will add and check it. I will let you know the results On Fri, May 9, 2014 at 10:08 AM, Yehuda Sadeh yeh...@inktank.com wrote

Re: [ceph-users] Lost access to radosgw after crash?

2014-05-13 Thread Yehuda Sadeh
On Tue, May 13, 2014 at 8:52 AM, Brian Rak b...@gameservers.com wrote: I hit a bug where radosgw crashed with -101 2014-05-13 15:26:07.188494 7fde82886820 0 ERROR: FCGX_Accept_r returned -24 too many files opened. You probably need to adjust your limits. 0 2014-05-13 15:26:07.193772

Re: [ceph-users] Problem with radosgw and some file name characters

2014-05-15 Thread Yehuda Sadeh
Your rewrite rule might be off a bit. Can you provide log with 'debug rgw = 20'? Yehuda On Thu, May 15, 2014 at 8:02 AM, Andrei Mikhailovsky and...@arhont.com wrote: Hello guys, I am trying to figure out what is the problem here. Currently running Ubuntu 12.04 with latest updates and

Re: [ceph-users] Problem with radosgw and some file name characters

2014-05-16 Thread Yehuda Sadeh
%{QUERY_STRING} [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L] Or are you talking about something else? Cheers Andrei From: Yehuda Sadeh yeh...@inktank.com To: Andrei Mikhailovsky and...@arhont.com Cc: ceph-users@lists.ceph.com Sent: Thursday, 15 May, 2014 4:05

Re: [ceph-users] How to point custom domains to a bucket and set default page and error page

2014-05-19 Thread Yehuda Sadeh
On Sat, May 17, 2014 at 3:32 AM, wsnote wsn...@163.com wrote: Hi, everyone! I want to use ceph to host static websites. In Amazon S3, if I want to host test.domain.com, I create a bucket name test.domain.com, and set a CNAME record: test.domain.com CNAME test.domain.com.s3.amazonaws.com. In

Re: [ceph-users] Various file lengths while uploading the same file

2014-05-19 Thread Yehuda Sadeh
On Mon, May 19, 2014 at 12:19 AM, Arthur Tumanyan arthurtuman...@gmail.com wrote: Hi, I'm writing an fastcgi application, which just gets data from stdin and appends it to ceph. Please,have a look: while (0 != (written = fread(buffer, sizeof (char), sizeof buffer, stdin))) {

Re: [ceph-users] Problem with radosgw and some file name characters

2014-05-21 Thread Yehuda Sadeh
and...@arhont.com To: Yehuda Sadeh yeh...@inktank.com Cc: ceph-users@lists.ceph.com Sent: Monday, 19 May, 2014 9:30:03 AM Subject: Re: [ceph-users] Problem with radosgw and some file name characters Yehuda, I've tried the rewrite rule that you've suggested, but it is not working for me. I get

Re: [ceph-users] Radosgw Timeout

2014-05-22 Thread Yehuda Sadeh
On Thu, May 22, 2014 at 6:16 AM, Georg Höllrigl georg.hoellr...@xidras.com wrote: Hello List, Using the radosgw works fine, as long as the amount of data doesn't get too big. I have created one bucket that holds many small files, separated into different directories. But whenever I try to

Re: [ceph-users] Unable to update Swift ACL's on existing containers

2014-05-22 Thread Yehuda Sadeh
That looks like a bug; generally the permission checks there are broken. I opened issue #8428, and pushed a fix on top of the firefly branch to wip-8428. Thanks! Yehuda On Thu, May 22, 2014 at 2:49 PM, James Page james.p...@ubuntu.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi

Re: [ceph-users] Problem with radosgw and some file name characters

2014-06-02 Thread Yehuda Sadeh
-- Radosgw_Server_1) I do not have any issues with special characters. Any idea what I am missing? Perhaps something needs changing on the proxy server? Cheers Andrei From: Yehuda Sadeh yeh...@inktank.com To: Andrei Mikhailovsky and...@arhont.com Cc: ceph-users

Re: [ceph-users] RGW: Multi Part upload and resulting objects

2014-06-04 Thread Yehuda Sadeh
On Wed, Jun 4, 2014 at 8:49 AM, Gregory Farnum g...@inktank.com wrote: On Wed, Jun 4, 2014 at 7:58 AM, Sylvain Munaut s.mun...@whatever-company.com wrote: Hi, During a multi part upload you can't upload parts smaller than 5M, and radosgw also slices object in slices of 4M. Having those two

Re: [ceph-users] RGW: Multi Part upload and resulting objects

2014-06-05 Thread Yehuda Sadeh
Right. It only matters for newly created objects. On Thu, Jun 5, 2014 at 10:01 AM, Gregory Farnum g...@inktank.com wrote: I don't believe that should cause any issues; the chunk sizes are in the metadata. -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Thu, Jun 5,

Re: [ceph-users] radosgw-admin log show

2014-06-05 Thread Yehuda Sadeh
On Thu, Jun 5, 2014 at 8:42 PM, Derek Yarnell de...@umiacs.umd.edu wrote: On 11/28/13, 4:18 AM, Dan Van Der Ster wrote: Dear users/experts, Does anyone know how to use radosgw-admin log show? It seems to not properly read the --bucket parameter. # radosgw-admin log show --bucket=asdf

Re: [ceph-users] Swift API Authentication Failure

2014-06-06 Thread Yehuda Sadeh
On Wed, Jun 4, 2014 at 12:00 PM, David Curtiss dcurtiss_c...@dcurtiss.com wrote: Over the last two days, I set up ceph on a set of ubuntu 12.04 VMs (my first time working with ceph), and it seems to be working fine (I have HEALTH_OK, and can create a test document via the rados commandline

Re: [ceph-users] Swift API Authentication Failure

2014-06-10 Thread Yehuda Sadeh
radosgw-admin subuser create --uid=hive_cache --subuser=hive_cache:swift --access=full radosgw-admin key create --subuser=hive_cache:swift --key-type=swift --secret=QFAMEDSJP5DEKJO0DDXY - David On Mon, Jun 9, 2014 at 11:14 PM, Yehuda Sadeh yeh...@inktank.com wrote: It seems that the subuser

Re: [ceph-users] Swift API Authentication Failure

2014-06-11 Thread Yehuda Sadeh
00011 For comparison, the 'df' line for the .users pool lists 2 objects, which are as follows: $ rados ls -p .users 4U5H60BMDL7OSI5ZBL8P F7HZCI4SL12KVVSJ9UVZ - David On Tue, Jun 10, 2014 at 11:49 PM, Yehuda Sadeh yeh...@inktank.com

Re: [ceph-users] civetweb frontend issue with spurious response

2014-06-16 Thread Yehuda Sadeh
The second response is a response to an expected follow up request on the same connection (using keep alive). If you're seeing it as part of the response on the first request then it's either an issue with the client not handling keep alive connections correctly, or an issue with the content

Re: [ceph-users] Problem with RadosGW and special characters

2014-06-26 Thread Yehuda Sadeh
The gateway itself supports these kind of characters. Usually we see this issue when there's something in front of the web server (like a load balancer) that modifies the requests. Another possibility is the web server configuration that might be rewriting the requests. In this case it seems that

Re: [ceph-users] Issues with federated gateway sync

2014-07-21 Thread Yehuda Sadeh
On Mon, Jul 21, 2014 at 1:07 PM, Justice London just...@ministrycentered.com wrote: Hello, I am having issues getting FG working between east/west data-center test configurations. I have the sync default.conf configured like this: source: http://10.20.2.39:80; src_zone: us-west-1

Re: [ceph-users] Some questions of radosgw

2014-08-04 Thread Yehuda Sadeh
On Fri, Aug 1, 2014 at 9:49 AM, Osier Yang agedos...@gmail.com wrote: [ correct the URL ] On 2014年08月02日 00:42, Osier Yang wrote: Hi, list, I managed to setup radosgw in testing environment to see if it's stable/mature enough for production use these several days. In the meanwhile, I

Re: [ceph-users] RadosGW problems

2014-08-19 Thread Yehuda Sadeh
On Tue, Aug 19, 2014 at 5:32 AM, Marco Garcês ma...@garces.cc wrote: UPDATE: I have installed Tengine (nginx fork) and configured both HTTP and HTTPS to use radosgw socket. Looking back at this thread, and considering this solution it seems to me that you were running the wrong apache

Re: [ceph-users] Uploading large files to swift interface on radosgw

2013-09-19 Thread Yehuda Sadeh
Jakobovitsch g...@mandic.net.br wrote: Hello Yehuda, thank you for your help. On 09/17/2013 08:35 PM, Yehuda Sadeh wrote: On Tue, Sep 17, 2013 at 3:21 PM, Gerd Jakobovitsch g...@mandic.net.br wrote: Hi all, I am testing a ceph environment installed in debian wheezy, and, when testing file

Re: [ceph-users] poor radosgw performance

2013-09-19 Thread Yehuda Sadeh
On Thu, Sep 19, 2013 at 8:52 AM, Matt Thompson watering...@gmail.com wrote: Hi All, We're trying to test swift API performance of swift itself (1.9.0) and ceph's radosgw (0.67.3) using the following hardware configuration: Shared servers: * 1 server running keystone for authentication * 1

Re: [ceph-users] poor radosgw performance

2013-09-20 Thread Yehuda Sadeh
your specific user's keyring file (but I might be wrong). Thanks again for the help! -Matt On Thu, Sep 19, 2013 at 5:01 PM, Yehuda Sadeh yeh...@inktank.com wrote: On Thu, Sep 19, 2013 at 8:52 AM, Matt Thompson watering...@gmail.com wrote: Hi All, We're trying to test swift API

Re: [ceph-users] poor radosgw performance

2013-09-20 Thread Yehuda Sadeh
that set in our ceph.conf so should in theory be using it already. I actually meant to say ceph authentication, not ceph cache. Yehuda On Fri, Sep 20, 2013 at 3:57 PM, Yehuda Sadeh yeh...@inktank.com wrote: On Fri, Sep 20, 2013 at 3:51 AM, Matt Thompson watering...@gmail.com wrote: Hi

  1   2   3   4   5   >