Re: ceph-deploy osd destroy feature

2015-01-05 Thread Robert LeBlanc
I do think the find a journal partition code isn't particularly robust. I've had experiences with ceph-disk trying to create a new partition even though I had wiped/zapped a disk previously. It would make the operational component of Ceph much easier with replacing disks if the journal partition

Re: [ceph-users] What to do when a parent RBD clone becomes corrupted

2015-01-06 Thread Robert LeBlanc
On Mon, Jan 5, 2015 at 6:01 PM, Gregory Farnum g...@gregs42.com wrote: On Thu, Dec 18, 2014 at 1:21 PM, Robert LeBlanc rob...@leblancnet.us wrote: Before we base thousands of VM image clones off of one or more snapshots, I want to test what happens when the snapshot becomes corrupted. I don't

Fwd: [ceph-users] Prioritize Heartbeat packets

2015-03-06 Thread Robert LeBlanc
Hidden HTML ... trying agin... -- Forwarded message -- From: Robert LeBlanc rob...@leblancnet.us Date: Fri, Mar 6, 2015 at 5:20 PM Subject: Re: [ceph-users] Prioritize Heartbeat packets To: ceph-us...@lists.ceph.com ceph-us...@lists.ceph.com, ceph-devel ceph-devel@vger.kernel.org

Re: Prioritize Heartbeat packets

2015-03-09 Thread Robert LeBlanc
by the commit, this ought to do the trick: osd heartbeat use min delay socket = true On 07/03/15 01:20, Robert LeBlanc wrote: I see that Jian Wen has done work on this for 0.94. I tried looking through the code to see if I can figure out how to configure this new option, but it all went over my

Re: [ceph-users] Prioritize Heartbeat packets

2015-03-09 Thread Robert LeBlanc
at 3:36 AM, Robert LeBlanc rob...@leblancnet.us wrote: I've found commit 9b9a682fe035c985e416ee1c112fa58f9045a27c and I see that when 'osd heartbeat use min delay socket = true' it will mark the packet with DSCP CS6. Based on the setting of the socket in msg/simple/Pipe.cc is it possible

Fwd: Force an OSD to try to peer

2015-03-30 Thread Robert LeBlanc
Sorry HTML snuck in somewhere. -- Forwarded message -- From: Robert LeBlanc rob...@leblancnet.us Date: Mon, Mar 30, 2015 at 8:15 PM Subject: Force an OSD to try to peer To: Ceph-User ceph-us...@ceph.com, ceph-devel ceph-devel@vger.kernel.org I've been working at this peering

Where is the systemd files?

2015-03-26 Thread Robert LeBlanc
libcephfs1-0.93-0.el7.centos.x86_64 ceph-0.93-0.el7.centos.x86_64 ceph-deploy-1.5.22-0.noarch [ulhglive-root@mon1 systemd]# for i in $(rpm -qa | grep ceph); do rpm -ql $i | grep -i --color=always systemd; done [nothing returned] Thanks, Robert LeBlanc -- To unsubscribe from this list: send the line

Re: Force an OSD to try to peer

2015-03-31 Thread Robert LeBlanc
Turns out jumbo frames was not set on all the switch ports. Once that was resolved the cluster quickly became healthy. On Mon, Mar 30, 2015 at 8:15 PM, Robert LeBlanc rob...@leblancnet.us wrote: I've been working at this peering problem all day. I've done a lot of testing at the network layer

Re: [ceph-users] Force an OSD to try to peer

2015-03-31 Thread Robert LeBlanc
on peering ? In our setup so far, we haven't enabled jumbo frames other than performance reason (if at all). Thanks Regards Somnath -Original Message- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Robert LeBlanc Sent: Tuesday, March 31, 2015 11:08 AM

Re: Force an OSD to try to peer

2015-03-31 Thread Robert LeBlanc
I was desperate for anything after exhausting every other possibility I could think of. Maybe I should put a checklist in the Ceph docs of things to look for. Thanks, On Tue, Mar 31, 2015 at 11:36 AM, Sage Weil s...@newdream.net wrote: On Tue, 31 Mar 2015, Robert LeBlanc wrote: Turns out jumbo

Re: Clarification of SSD journals for BTRFS rotational HDD

2015-02-27 Thread Robert LeBlanc
Also sending to the devel list to see if they have some insight. On Wed, Feb 25, 2015 at 3:01 PM, Robert LeBlanc rob...@leblancnet.us wrote: I tried finding an answer to this on Google, but couldn't find it. Since BTRFS can parallel the journal with the write, does it make sense to have

Re: [ceph-users] who is using radosgw with civetweb?

2015-02-25 Thread Robert LeBlanc
would like to try it to offer some feedback on your question. Thanks, Robert LeBlanc On Wed, Feb 25, 2015 at 12:31 PM, Sage Weil sw...@redhat.com wrote: Hey, We are considering switching to civetweb (the embedded/standalone rgw web server) as the primary supported RGW frontend instead

Re: [ceph-users] who is using radosgw with civetweb?

2015-02-25 Thread Robert LeBlanc
Cool, I'll see if we have some cycles to look at it. On Wed, Feb 25, 2015 at 2:49 PM, Sage Weil sw...@redhat.com wrote: On Wed, 25 Feb 2015, Robert LeBlanc wrote: We tried to get radosgw working with Apache + mod_fastcgi, but due to the changes in radosgw, Apache, mode_*cgi, etc

Re: [ceph-users] who is using radosgw with civetweb?

2015-02-26 Thread Robert LeBlanc
Thanks, we were able to get it up and running very quickly. If it performs well, I don't see any reason to use Apache+fast_cgi. I don't have any problems just focusing on civetweb. On Wed, Feb 25, 2015 at 2:49 PM, Sage Weil sw...@redhat.com wrote: On Wed, 25 Feb 2015, Robert LeBlanc wrote: We

Re: [ceph-users] who is using radosgw with civetweb?

2015-02-26 Thread Robert LeBlanc
-ow...@vger.kernel.org [mailto:ceph-devel-ow...@vger.kernel.org] On Behalf Of Robert LeBlanc Sent: Thursday, February 26, 2015 12:27 PM To: Sage Weil Cc: Ceph-User; ceph-devel Subject: Re: [ceph-users] who is using radosgw with civetweb? Thanks, we were able to get it up and running very

Re: [ceph-users] who is using radosgw with civetweb?

2015-02-26 Thread Robert LeBlanc
+1 for proxy. Keep the civetweb lean and mean and if people need extras let the proxy handle this. Proxies are easy to set-up and a simple example could be included in the documentation. On Thu, Feb 26, 2015 at 11:43 AM, Wido den Hollander w...@42on.com wrote: Op 26 feb. 2015 om 18:22 heeft

Re: CRUSH decompile failes

2015-03-23 Thread Robert LeBlanc
it doesn't like the rack definition. I can move things around, like putting root before it and it always chokes on the first rack definition no matter which one it is. On Mon, Mar 23, 2015 at 12:53 PM, Robert LeBlanc rob...@leblancnet.us wrote: I was trying to decompile and edit the CRUSH map

Re: CRUSH decompile failes

2015-03-23 Thread Robert LeBlanc
are on). Saving for posterity's sake. Thanks Sage! On Mon, Mar 23, 2015 at 1:09 PM, Robert LeBlanc rob...@leblancnet.us wrote: Ok, so the decompile error is because I didn't download the CRUSH map (found that out using hexdump), but I still can't compile an unmodified CRUSH map. [root@nodezz

0.93 fresh cluster won't create PGs

2015-03-27 Thread Robert LeBlanc
or something (I know that Ceph journals pgmap changes and such). I'm concerned that this could be very detrimental in a production environment. There doesn't seem to be a way to recover from this. Any thoughts? Thanks, Robert LeBlanc -- To unsubscribe from this list: send the line unsubscribe ceph

Does crushtool --test --simulate do what cluster should do?

2015-03-23 Thread Robert LeBlanc
I'm trying to create a CRUSH ruleset and I'm using crushtool to test the rules, but it doesn't seem to mapping things correctly. I have two roots, on for spindles and another for SSD. I have two rules, one for each root. The output of crushtool on rule 0 shows objects being mapped to SSD OSDs when

Re: ceph repo for debian jessie ?

2015-04-29 Thread Robert LeBlanc
Should stretch be added as well? On Wed, Apr 29, 2015 at 12:18 AM, Alexandre DERUMIER aderum...@odiso.com wrote: Hi, Debian Jessie has been released this weekend, any plan to add jessie repositories soon ? Regards, Alexandre -- To unsubscribe from this list: send the line unsubscribe

Generating debug packages

2015-06-05 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 What is the feeling about generating debug packages along with the standard packages? Seems like it could be useful. That and even though I enjoying building my own packages, I am lazy. Thanks, - Robert LeBlanc GPG Fingerprint 79A2

Re: Generating debug packages

2015-06-05 Thread Robert LeBlanc
ng3hoFhiLxXev2+TwyCN8KP4AwbWC0oqPe88XI7U+Lvbc/95Vyj4B6mDU5Cx p9YLHNnEkEWWv0xivEukPwJP4rBbjx+dUqdU3kr34HsqWlkr0s1MvwEi8hkf 5CmbkMhpoLIrS5EumZD/hQIe6JKgpEh4is60iqiB9c2YwzDk2XRPpWFc5RRV R7Qo =jkb2 -END PGP SIGNATURE- Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654

Looking to improve small I/O performance

2015-06-05 Thread Robert LeBlanc
% ceph-osd ceph-osd [.] ReplicatedPG::execute_ctx +6.09% 0.15% ceph-osd [kernel.kallsyms] [k] ___sys_sendmsg +5.68% 0.00% ceph-osd [kernel.kallsyms] [k] do_sock_sendmsg Thanks for humoring me, - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4

Re: Looking to improve small I/O performance

2015-06-06 Thread Robert LeBlanc
/2rC74LvTscIIdf1dZQHWwXHelRm49 y1pxq07V7LlL6gM+zA6Zskm9QwlJ3D81mH7QpiaixKX8cEzcVifD7WUzP/YV Go8K =gB4o -END PGP SIGNATURE- Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Sat, Jun 6, 2015 at 12:07 AM, Dałek, Piotr piotr.da...@ts.fujitsu.com wrote

Re: Looking to improve small I/O performance

2015-06-07 Thread Robert LeBlanc
enable_experimental_unrecoverable_data_corrupting_features = ms-type-async BTW, what kind of workload you are trying , random read or write ? Also, is this SSD or HDD cluster ? Thanks Regards Somnath -Original Message- From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel-ow...@vger.kernel.org] On Behalf Of Robert LeBlanc Sent

Re: [ceph-users] rbd_cache, limiting read on high iops around 40k

2015-06-09 Thread Robert LeBlanc
+F0Uvs3xWAxxaIR9r83wMj9qQeBZTKVzQ 1aFIi15KqAwOp12yWCmrqKTeXhjwYQNd8viCQCGN7AQyPglmzfbuEHalVjz4 oSJX =k281 -END PGP SIGNATURE- Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Tue, Jun 9, 2015 at 6:02 AM, Alexandre DERUMIER aderum...@odiso.com wrote

Build RPMs from git

2015-06-09 Thread Robert LeBlanc
/ceph`' It coughs at the install/RPM package part. Any help with preventing the whole tar/untar process which I can't seem to get exactly right either would be very helpful. Thanks, - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 -BEGIN PGP

Re: 06/24/2015 Weekly Ceph Performance Meeting IS ON!

2015-06-25 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 +1 - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Thu, Jun 25, 2015 at 10:39 AM, Alexandre DERUMIER wrote: Thanks Mark ! - Mail original - De: Mark Nelson À: aderumier , Robert

Re: 06/24/2015 Weekly Ceph Performance Meeting IS ON!

2015-06-25 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Any update on the recorded sessions? Last I see is 10 Jun 2015 in the Etherpad. Thanks, - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Wed, Jun 24, 2015 at 7:28 AM, Mark Nelson wrote: 8AM

Re: Build RPMs from git

2015-06-18 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Cool, I'll have to give it a shot. - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Thu, Jun 18, 2015 at 7:44 AM, Owen Synge wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Robert

Fwd: Build RPMs from git

2015-06-10 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Brad, I've been able to do this just fine. I'm looking for a way to build right out of a git branch where the tarball isn't on the downloads page. - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1

Re: Build RPMs from git

2015-06-10 Thread Robert LeBlanc
-Stop: 0 1 6 on Debian, and leave the values blank on Red Hat. Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More

Re: [ceph-users] Ceph giant installation fails on rhel 7.0

2015-06-11 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Have you configured and enabled the epel repo? - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Thu, Jun 11, 2015 at 6:26 AM, Shambhu Rajak wrote: I am trying to install ceph gaint on rhel

rbd top

2015-06-11 Thread Robert LeBlanc
time because I don't know many of the tricks that help people get to speed quickly. Thanks, - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 -BEGIN PGP SIGNATURE- Version: Mailvelope v0.13.1 Comment: https://www.mailvelope.com

Re: Build RPMs from git

2015-06-10 Thread Robert LeBlanc
Rx6DfsdHfSN5ji4poAlpExwvRmwbdBSr16kVhPU07DdODdmlK8nF64g8U4yY WItD =sGNI -END PGP SIGNATURE- Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Wed, Jun 10, 2015 at 11:22 AM, Ken Dreyer kdre...@redhat.com wrote: On 06/10/2015 09:44 AM, Robert LeBlanc wrote: This is really

Re: rbd top

2015-06-15 Thread Robert LeBlanc
+8AjeVa45EDHkAQqa5t8Kb5+sKr/LyyKBmqw0suD77kqKqWtKs9+sVFd6X TXOhPKmAnd8TEGE86JzsfFeypfb76jil08MuIzvDvVj8hsRjpRHsZBQi+DPn VIIbONqNM4CknURJW17rmP0o9l+sF+KAeji0AGivZuawtD+vbThwfWNRyMjN 5IXZ =Sgkt -END PGP SIGNATURE- Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2

Re: rbd top

2015-06-17 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Well, I think this has gone well past my ability to implement. Should this be turned into a BP and see if someone is able to work on it? - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Tue

Re: rbd top

2015-06-15 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On Mon, Jun 15, 2015 at 7:52 AM, Sage Weil wrote: On Mon, 15 Jun 2015, Gregory Farnum wrote: On Thu, Jun 11, 2015 at 12:33 PM, Robert LeBlanc wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 One feature we would like is an rbd top

Re: MDS auth caps for cephfs

2015-05-28 Thread Robert LeBlanc
situation are we trying to target/solve? - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 -BEGIN PGP SIGNATURE- Version: Mailvelope v0.13.1 Comment: https://www.mailvelope.com wsFcBAEBCAAQBQJVZ154CRDmVDuy+mK58QAAXTcP/1bJDaE2KU1BKVk2O+o1

Re: MDS auth caps for cephfs

2015-05-28 Thread Robert LeBlanc
implementing Kerberos and preventing the client from mapping to other UID/GIDs, but that really takes the flexibility out of the system. Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 -- To unsubscribe from this list: send the line unsubscribe ceph-devel

Re: [ceph-users] Memory Allocators and Ceph

2015-05-27 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 The workload is on average, 17KB per read request and 13KB per write request with 73% read abd 27% write. This is a web hosting workload. - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Wed

Re: OSD-Based Object Stubs

2015-05-27 Thread Robert LeBlanc
storage tiering. Is the idea to be able to move data off to near-line storage like tape? Thanks, - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Wed, May 27, 2015 at 2:39 AM, Marcel Lauhoff wrote: Hi, I wrote a prototype for an OSD-based

Memory Allocators and Ceph

2015-05-27 Thread Robert LeBlanc
/spreadsheets/d/1n12IqAOuH2wH-A7Sq5boU8kSEYg_Pl20sPmM0idjj00/edit?usp=sharing Test script is multitest. The real world test is based off of the disk stats of about 100 of our servers which have uptimes of many months. - - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654

Re: Memory Allocators and Ceph

2015-05-27 Thread Robert LeBlanc
can determine if jemalloc is something that would be worth switching to eventually. - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 -BEGIN PGP SIGNATURE- Version: Mailvelope v0.13.1 Comment: https://www.mailvelope.com

Re: MDS auth caps for cephfs

2015-05-28 Thread Robert LeBlanc
this is one thing SMB got right and why I prefer Samba over NFS for multi-tenant environments. - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Wed, May 27, 2015 at 6:42 PM, Gregory Farnum wrote: On Wed, May 27, 2015 at 5:37 PM, Sage Weil wrote

Re: RFC: progress bars

2015-05-28 Thread Robert LeBlanc
/pYWtjcLZYB079Yjl5NKqWOdewStCM9JhTNwjc7bclQ4H0LLW3EW2P jeQE =KYfI -END PGP SIGNATURE- Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Thu, May 28, 2015 at 4:13 AM, John Spray john.sp...@redhat.com wrote: On 28/05/2015 06:47, Gregory Farnum wrote: Thread

Re: Memory Allocators and Ceph

2015-05-28 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 I've got some more tests running right now. Once those are done, I'll find a couple of tests that had extreme difference and gather some perf data for them. - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2

Re: MDS auth caps for cephfs

2015-05-28 Thread Robert LeBlanc
/GID to store unique UID/GIDs on the back end and just strip off the tenant ID when presented to the client so there are no collisions of UID/GIDs between tenants in the MDS. Hmm, that is another thought... -Greg - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E

Re: Ceph Hackathon: More Memory Allocator Testing

2015-08-21 Thread Robert LeBlanc
think I'm stating the obvious here, sorry). - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Fri, Aug 21, 2015 at 8:26 AM, Milosz Tanski wrote: On Fri, Aug 21, 2015 at 12:22 AM, Shishir Gowda wrote: Hi All, Have sent out a pull request

Re: Ceph Hackathon: More Memory Allocator Testing

2015-08-24 Thread Robert LeBlanc
://en.m.wikipedia.org/wiki/Dynamic_loading Robert LeBlanc Sent from a mobile device please excuse any typos. On Aug 22, 2015 7:55 AM, Sage Weil wrote: On Fri, 21 Aug 2015, Milosz Tanski wrote: On Fri, Aug 21, 2015 at 12:22 AM, Shishir Gowda wrote: Hi All, Have sent out a pull request which enables

Re: [ceph-users] dropping old distros: el6, precise 12.04, debian wheezy?

2015-07-30 Thread Robert LeBlanc
+HWBQTZgLi0bxIyBEo7Pg75+f8yzawTpWWx5tM4JR yKkADT8+hid8PK6YwUrsaYnRhdRunAZv9BdvOulXyJ9azcCHSzlJWeYWHEwq krdm =m+W6 -END PGP SIGNATURE- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Thu, Jul 30, 2015 at 7:54 AM, Sage Weil sw...@redhat.com wrote: As time

Re: systemd status

2015-07-30 Thread Robert LeBlanc
systemctl to shut down the process and unmount itself? - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Thu, Jul 30, 2015 at 6:45 AM, Sage Weil wrote: On Wed, 29 Jul 2015, Alex Elsayed wrote: Sage Weil wrote: On Wed, 29 Jul 2015, Alex

Cluster reliability

2015-07-14 Thread Robert LeBlanc
and report back to the list. Thank you, - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 -BEGIN PGP SIGNATURE- Version: Mailvelope v0.13.1 Comment: https://www.mailvelope.com wsFcBAEBCAAQBQJVpUQtCRDmVDuy+mK58QAA3d8P/3tSSFXInMDLde9IZFfE

Re: Fix OP dequeuing order

2015-10-29 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Here is the pull request against master. https://github.com/ceph/ceph/pull/6429 - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Wed, Oct 28, 2015 at 11:44 PM, Robert LeBlanc wrote

Re: Fix OP dequeuing order

2015-10-28 Thread Robert LeBlanc
/6BxREpKgXoQ9EgZMdDWntLUBU+QV+FedlXRTEHD3zQKwlT Hix1lTuNxVY+VZSXos18FaFz+duVjUS/O2yuy1bmLWD6PfouFFCMfRwmK1rl QaaE1k6vinAzGbwq48D94hMPBcNQmJTWd8GC8kOGP+F9EFHyCPaHVK0Lu/T2 jdnx9h/HcshOWh60CKu6N5w+JfvIcqyrLcDmiS3A9scgYOsdphLf1etxrKdq Xd8Z =evCv -END PGP SIGNATURE- Robert LeBlanc PGP

Re: Really slow cache-evict

2015-10-23 Thread Robert LeBlanc
at all. It is really weird. - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Fri, Oct 23, 2015 at 5:13 PM, Robert LeBlanc wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > We are testing out cache tiering, but w

Really slow cache-evict

2015-10-23 Thread Robert LeBlanc
cently and didn't see this problem (only took an hour or two to evict 700GB of data). I saw a very slow evict a while back, but just chalked it up to btrfs being dumb. [1] http://docs.ceph.com/docs/master/rados/operations/cache-tiering/#removing-a-writeback-cache - ---- Robert LeBla

Fix OP dequeuing order

2015-10-28 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 I created a pull request to fix an op dequeuing order problem. I'm not sure if I need to mention it here. https://github.com/ceph/ceph/pull/6417 - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1

Re: Request for Comments: Weighted Round Robin OP Queue

2015-11-09 Thread Robert LeBlanc
for that. Do you know the priorities of snap trim, scrub and recovery so that I can do some math/logic on applying costs in an efficient way as we talked about last week? Thanks, - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 -BEGIN PGP

Re: Request for Comments: Weighted Round Robin OP Queue

2015-11-09 Thread Robert LeBlanc
/tWqDNwg3TBXtWSFU5S1XgJzoAQm5 yE7dbMwhAObw3XQ/eGMTmyICs1vwD0+mxaNHHWzSubtFKcdblUDW6BUxc+lj ztfA =GSDL -END PGP SIGNATURE- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Mon, Nov 9, 2015 at 9:49 AM, Samuel Just <sj...@redhat.com> wrote:

Re: Request for Comments: Weighted Round Robin OP Queue

2015-11-09 Thread Robert LeBlanc
/I5rlCL5CKTxb2LR1/5WJt0eh7xtyKU1B0yh4G7JlMf/3kmrznOWu VEUUA3mJ1depDToadnECnCZMKHrGYC36XCy8xq3FDqhvl4BWV0VMA+yi1uhj zZ5udKKbN5Cxo/Sc48DG8wz9lQKn4LPCH2PD81oTcTfyd1iG2oNNkchrXa6K iwed =WjDS -END PGP SIGNATURE- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654

Re: Request for Comments: Weighted Round Robin OP Queue

2015-11-09 Thread Robert LeBlanc
o do next taking into account > fairness across io classes and ops with different costs. > On Mon, Nov 9, 2015 at 11:19 AM, Robert LeBlanc wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA256 >> >> Thanks, I think some of the fog is clearing. I was wondering how >

Re: Increasing # Shards vs multi-OSDs per device

2015-11-11 Thread Robert LeBlanc
difference there may be as it would help decide if this is a path to continue pursuing. - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Wed, Nov 11, 2015 at 3:44 PM, Blinick, Stephen L wrote: > Thanks for taking a look! > > First, the

Re: Request for Comments: Weighted Round Robin OP Queue

2015-11-09 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 It sounds like dmclock/mclock will alleviate a lot of the concerns I have as long as it can be smart like you said. It sounds like the queue thread was already tried so there is experience behind the current implementation vs. me thinking it might

Re: Request for Comments: Weighted Round Robin OP Queue

2015-11-04 Thread Robert LeBlanc
as well). What are some of the next steps for something like this, meaning a pretty significant change to core code? Thank you to all who took time to help point me in the right direction. - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Wed, Nov

Re: Request for Comments: Weighted Round Robin OP Queue

2015-11-06 Thread Robert LeBlanc
, osd_repop_reply). If the recovery is in another separate queue, then there is no reliable way to prioritize OPs between them. If I'm going off in to the weeds, please help me get back on the trail. Thanks, - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2

Request for Comments: Weighted Round Robin OP Queue

2015-11-04 Thread Robert LeBlanc
for the assert and pointers how to resolve it. 4. It seems like there are multiple threads of the queue keeping the queues pretty small. How can I limit the queue to one thread so all OPs have to be queued in one queue? I'd like to see the differences with changing this. 5. I'd like any pointers to improving this co

Re: Request for Comments: Weighted Round Robin OP Queue

2015-11-05 Thread Robert LeBlanc
as I've tried to figure this out. Thanks, - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Wed, Nov 4, 2015 at 8:20 PM, Gregory Farnum wrote: > On Wed, Nov 4, 2015 at 7:00 PM, Robert LeBlanc wrote: >> -BEGIN PGP SIGNE

Re: Request for Comments: Weighted Round Robin OP Queue

2015-11-06 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On Fri, Nov 6, 2015 at 3:12 AM, Sage Weil wrote: > On Thu, 5 Nov 2015, Robert LeBlanc wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA256 >> >> Thanks Gregory, >> >> People are most likely busy an

Re: Fix OP dequeuing order

2015-10-30 Thread Robert LeBlanc
0ryi4KDriU+zTUo7J0P6qQHm6ab nImA =e7Ar -END PGP SIGNATURE- -------- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Fri, Oct 30, 2015 at 12:08 PM, Gregory Farnum <gfar...@redhat.com> wrote: > As we've discussed on the PR, this isn't rig

Re: [ceph-users] Potential OSD deadlock?

2015-10-14 Thread Robert LeBlanc
XU8gYbG1um5YY8jrtvvkhnbHdeO/k24/cH7MGslxeezBPnMNzmqj3qVdiX1H EBbyBBtp8OF+pKExrmZc2w01W/Nxl6GbVoG+IKJ61FgwKOXEiMwb0wv5mu30 eP3D =R0O9 -END PGP SIGNATURE- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Wed, Oct 14, 2015 at 12:00 AM, Haomai Wang

Re: [ceph-users] Potential OSD deadlock?

2015-10-14 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 I'm sure I have a log of a 1,000 second block somewhere, I'll have to look around for it. I'll try turning that knob and see what happens. I'll come back with the results. Thanks, - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4

Re: [ceph-users] Potential OSD deadlock?

2015-10-07 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 We forgot to upload the ceph.log yesterday. It is there now. - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Tue, Oct 6, 2015 at 5:40 PM, Robert LeBlanc wrote: > -BEGIN PGP SIG

Re: newstore direction

2015-10-19 Thread Robert LeBlanc
- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Mon, Oct 19, 2015 at 1:49 PM, Sage Weil <sw...@redhat.com> wrote: > The current design is based on two simple ideas: > > 1) a key/value interface is better way to manage all of our internal >

Re: chooseleaf may cause some unnecessary pg migrations

2015-10-13 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Are you testing with straw or straw2? - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Tue, Oct 13, 2015 at 2:22 AM, Xusangdi wrote: > Hi Sage, > > Recently when I was learn

Re: [ceph-users] Potential OSD deadlock?

2015-10-06 Thread Robert LeBlanc
/ynVLLY8fOKb2/x1GEKYNz3cGKx3M0u S5wRtYvOOZEwb/B2bhvCSNtb2FgqpS4INfgKn+334ibd2X1o42oj52SA1lz/ baNh =LmYc -END PGP SIGNATURE- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Tue, Oct 6, 2015 at 10:19 AM, Sage Weil <sw...@redhat.com> wrote: &g

Re: [ceph-users] Potential OSD deadlock?

2015-10-06 Thread Robert LeBlanc
32M/ luUMXHZ8lTHk9C8MfZcpyPGvwp2FliD9LqaWOVPWtWZJcerEWcZVlEApg4qb fo5a =ahEi -END PGP SIGNATURE- ---- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Tue, Oct 6, 2015 at 6:37 AM, Sage Weil <sw...@redhat.com> wrote: > On Mon, 5 Oct 2015, Robert LeBlanc wrote: >&

Re: [ceph-users] Potential OSD deadlock?

2015-10-06 Thread Robert LeBlanc
. - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Tue, Oct 6, 2015 at 1:34 PM, Sage Weil wrote: > On Tue, 6 Oct 2015, Robert LeBlanc wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA256 >> >> I can't think of

Re: [ceph-users] Potential OSD deadlock?

2015-10-06 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On my second test (a much longer one), it took nearly an hour, but a few messages have popped up over a 20 window. Still far less than I have been seeing. - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2

Re: [ceph-users] Potential OSD deadlock?

2015-10-06 Thread Robert LeBlanc
clean build as that is my most limiting factor? Thanks, - ---- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Tue, Oct 6, 2015 at 12:32 PM, Sage Weil wrote: > On Tue, 6 Oct 2015, Robert LeBlanc wrote: >> -BEGIN PGP SIGNED MESSAGE-

Re: [ceph-users] Potential OSD deadlock?

2015-10-06 Thread Robert LeBlanc
D6hcAZGtdvpiBRXBC5gORfiFWFxwu5kQ+daUhgUIe/O/EWyeD0rirZoqlLnZ EDrG =BZVw -END PGP SIGNATURE- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Tue, Oct 6, 2015 at 2:36 PM, Robert LeBlanc <rob...@leblancnet.us> wrote: > -BEGIN PGP SIGNE

Re: [ceph-users] Potential OSD deadlock?

2015-10-08 Thread Robert LeBlanc
+aT2m1erdSyZn0ZC9a6CjWmwJXY6FCSGhE53BbefBxmCFxn+8tVav+Q8W 7s14TntP6ex4ca7eTwGuSXC9FU5fAVa+3+3aXDAC1QPAkeVkXyB716W1XG6b BCFo =GJL4 -END PGP SIGNATURE- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Wed, Oct 7, 2015 at 1:25 PM, Robert LeBlanc

Re: throttles

2015-10-13 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 In my experience with throttles, we had to restart the OSD, the admin socket would not apply the change to a running OSD. - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Tue, Oct 13, 2015

Re: Ceph Write Path Improvement

2015-09-03 Thread Robert LeBlanc
+jClEPU3GdDSrMluFCOH4Qu+F pvIy0PxdGlGDS4P4Xs+NiWIeCu12NjtO8dRg++BO0rPbladBlsF3ZD+G7R/q pG/QTKhf41s+pbDal+HJoNKiK7+WZgHhy8IBXdR4a8SQeMHRa/MYJk6MFLKr vK/6 =wcJB -END PGP SIGNATURE- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Wed, Sep 2

Re: Ceph Write Path Improvement

2015-09-03 Thread Robert LeBlanc
. - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Thu, Sep 3, 2015 at 10:42 AM, Mark Nelson wrote: > On 09/03/2015 11:23 AM, Robert LeBlanc wrote: >> >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA256 >> >> Somnath, >&

Re: Ceph Write Path Improvement

2015-09-10 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 This is great work. Thanks for sharing. I'm looking forward to the resolution of the scale out issues! - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Wed, Sep 9, 2015 at 6:33 PM, Somnath Roy

Re: Re: What should be in the next hammer/firefly release ?

2015-09-17 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Can we get http://tracker.ceph.com/issues/10399 into hammer. We hit this today. - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Thu, Sep 10, 2015 at 4:57 AM, Miyamae, Takeshi wrote: >

Re: [ceph-users] Potential OSD deadlock?

2015-10-03 Thread Robert LeBlanc
Bbev+JYEoYT 8/WX+fhueHt4dqUYWIl3HZ0CEzbXbug0xmFvhrbmL2f3t9XOkDZRbAjlYrGm lswiJMDueY8JkxSnPvCQrHXqjbCcy9rMG7nTnLFz98rTcHNCwtpv0qVYhheg 4YRNRVMbfNP/6xsJvG1wVOSQPwxZSPqJh42pDqMRePJl3Zn66MTx5wvdNDpk l7OF =OI++ -END PGP SIGNATURE- -------- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1

Re: try-restart on upgrade, and upgrade procedures in general

2015-09-09 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Sounds reasonable to me. - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Wed, Sep 9, 2015 at 2:48 AM, Nathan Cutler wrote: > Hi all: > > I have been tinkering with the %preun an

Re: [ceph-users] Potential OSD deadlock?

2015-10-05 Thread Robert LeBlanc
CN6w6xu1FHLeVobKAWe5ZzKY5lxw6b8YG+ce/E2dvW73gSASPTvtv68gaT04 2SPF9Ql0fERL5EDY9Pc4MHpQVcS0XxxJA69CgnWgaG6fzq2eY7fALeMBVWlB fRj3zQwqJls/X8JZ3c4P4G0R6DP9bmMwGr++oYc3gWGrvgzxw3N7+ornd0jd GdXC =Aigq -END PGP SIGNATURE- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1

Re: [ceph-users] Potential OSD deadlock?

2015-09-23 Thread Robert LeBlanc
to congest things. We probably already saw this issue, just didn't know it. - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Wed, Sep 23, 2015 at 1:10 PM, Mark Nelson wrote: > FWIW, we've got some 40GbE Intel cards in the commun

Re: full cluster/pool handling

2015-09-24 Thread Robert LeBlanc
; > sage > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html - Robert LeBlanc PGP Fingerprint 79A2 9

Re: [ceph-users] Potential OSD deadlock?

2015-09-22 Thread Robert LeBlanc
wVjQ43qLfuDTS4o+IJ4ztOMd26vIv6Mn6WVwKCjoCXJc8ajywR9Dy+6lL o8oJ+tn7hMc9Qy1iBhu3/QIP4WCsUf9RVeu60oahNEpde89qW32S9CZlrJDO gf4iTryRjkAhdmZIj9JiaE8jQ6dvN817D9cqs/CXKV9vhzYoM7p5YWHghBKB J3hS =0J7F -END PGP SIGNATURE- -------- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Tu

Re: [ceph-users] Potential OSD deadlock?

2015-09-22 Thread Robert LeBlanc
d reliable for ping, but still had the blocked I/O. I reduced the MTU to 1500 and checked pings (OK), but I'm still seeing the blocked I/O. - -------- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Tue, Sep 22, 2015 at 3:52 PM, Sage Weil wrote: > On Tue

Re: [ceph-users] Potential OSD deadlock?

2015-09-22 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 4.2.0-1.el7.elrepo.x86_64 - - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Tue, Sep 22, 2015 at 3:41 PM, Samuel Just wrote: > I loo

Re: Adding Data-At-Rest compression support to Ceph

2015-09-24 Thread Robert LeBlanc
Z_FULL_FLUSH may help provide some seek points within an archive to prevent decompressing the whole object for reads? - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Thu, Sep 24, 2015 at 10:25 AM, Igor Fedotov wrote: > > > On 24.09.2

Re: [ceph-users] Potential OSD deadlock?

2015-09-25 Thread Robert LeBlanc
and TIME_WAIT at this point. There are 130 OSDs, 8 KVM hosts hosting about 150 VMs. Open files is set at 32K for the OSD processes and 16K system wide. Does this seem like the right spot to be looking? What are some configuration items we should be looking at? Thanks, Robert LeBlanc PGP

Re: What should be in the next hammer/firefly release ?

2015-09-18 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Thanks, didn't find the original ticket number in the list. - Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Fri, Sep 18, 2015 at 12:57 AM, Loic Dachary wrote: > Hi Robert, >

Re: [ceph-users] Potential OSD deadlock?

2015-09-23 Thread Robert LeBlanc
/DC0mqpnmo0Clx3jkdHX+x1EXpJUtocSeI44LX KWIMhe9wXtKAoHQFEcJ0o0+wrXWMevvx33HPC4q1ULrFX0ILNx5Mo0Rp944X 4OEo =P33I -END PGP SIGNATURE- Robert LeBlanc PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Tue, Sep 22, 2015 at 4:15 PM, Robert LeBlanc <

  1   2   >