Re: [ceph-users] Best practice K/M-parameters EC pool

2014-08-26 Thread Christian Balzer
following the failure of the first disk (assuming AFR https://en.wikipedia.org/wiki/Annualized_failure_rate of every disk is 8%, divided by the number of hours during a year == (0.08 / 8760) ~= 1/100,000 * A given disk does not participate in more than 100 PG -- Christian Balzer

Re: [ceph-users] Best practice K/M-parameters EC pool

2014-08-26 Thread Christian Balzer
answered canonically by Inktank or Sage, if not then perhaps I'll see how far I get sticking this diatribe into the ICE support portal... -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com

Re: [ceph-users] Best practice K/M-parameters EC pool

2014-08-28 Thread Christian Balzer
On Thu, 28 Aug 2014 10:29:20 -0400 Mike Dawson wrote: On 8/28/2014 12:23 AM, Christian Balzer wrote: On Wed, 27 Aug 2014 13:04:48 +0200 Loic Dachary wrote: On 27/08/2014 04:34, Christian Balzer wrote: Hello, On Tue, 26 Aug 2014 20:21:39 +0200 Loic Dachary wrote: Hi Craig

Re: [ceph-users] Uneven OSD usage

2014-08-28 Thread Christian Balzer
Thanks! ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications

Re: [ceph-users] Uneven OSD usage

2014-08-29 Thread Christian Balzer
Hello, On Fri, 29 Aug 2014 02:32:39 -0400 J David wrote: On Thu, Aug 28, 2014 at 10:47 PM, Christian Balzer ch...@gol.com wrote: There are 1328 PG's in the pool, so about 110 per OSD. And just to be pedantic, the PGP_NUM is the same? Ah, ceph status reports 1328 pgs. But: $ sudo

Re: [ceph-users] Asked for emperor, got firefly. (You can't take the sky from me?)

2014-08-30 Thread Christian Balzer
-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi

Re: [ceph-users] Uneven OSD usage

2014-08-30 Thread Christian Balzer
Hello, On Sat, 30 Aug 2014 18:27:22 -0400 J David wrote: On Fri, Aug 29, 2014 at 2:53 AM, Christian Balzer ch...@gol.com wrote: Now, 1200 is not a power of two, but it makes sense. (12 x 100). Should have been 600 and then upped to 1024. At the time, there was a reason why doing

Re: [ceph-users] How to replace an node in ceph?

2014-09-04 Thread Christian Balzer
. -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph

Re: [ceph-users] Huge issues with slow requests

2014-09-04 Thread Christian Balzer
e13901) v4 currently waiting for subops from [12,29] Kind Regards, David ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer

Re: [ceph-users] SSD journal deployment experiences

2014-09-04 Thread Christian Balzer
to be done (retention of hot objects, etc) and there have been stability concerns raised here. Regards, Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com

Re: [ceph-users] How to replace an node in ceph?

2014-09-04 Thread Christian Balzer
Hello, On Fri, 5 Sep 2014 12:09:11 +0800 Ding Dinghua wrote: Please see my comment below: 2014-09-04 21:33 GMT+08:00 Christian Balzer ch...@gol.com: Hello, On Thu, 4 Sep 2014 20:56:31 +0800 Ding Dinghua wrote: Aside from what Loic wrote, why not replace the network

Re: [ceph-users] How to replace an node in ceph?

2014-09-05 Thread Christian Balzer
On Fri, 5 Sep 2014 13:46:17 +0800 Ding Dinghua wrote: 2014-09-05 13:19 GMT+08:00 Christian Balzer ch...@gol.com: Hello, On Fri, 5 Sep 2014 12:09:11 +0800 Ding Dinghua wrote: Please see my comment below: 2014-09-04 21:33 GMT+08:00 Christian Balzer ch...@gol.com

Re: [ceph-users] Huge issues with slow requests

2014-09-05 Thread Christian Balzer
not have this problem; I'm pretty sure you're dealing with a rogue/faulty osd/node somewhere. Cheers, Martin On Fri, Sep 5, 2014 at 2:28 AM, Christian Balzer ch...@gol.com wrote: On Thu, 4 Sep 2014 12:02:13 +0200 David wrote: Hi, We’re running a ceph cluster with version

Re: [ceph-users] Good way to monitor detailed latency/throughput

2014-09-06 Thread Christian Balzer
://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users

Re: [ceph-users] SSD journal deployment experiences

2014-09-06 Thread Christian Balzer
On Fri, 5 Sep 2014 09:42:02 + Dan Van Der Ster wrote: On 05 Sep 2014, at 11:04, Christian Balzer ch...@gol.com wrote: On Fri, 5 Sep 2014 07:46:12 + Dan Van Der Ster wrote: On 05 Sep 2014, at 03:09, Christian Balzer ch...@gol.com wrote: On Thu, 4 Sep 2014 14:49:39 -0700

Re: [ceph-users] Huge issues with slow requests

2014-09-06 Thread Christian Balzer
128GB of RAM on the OSDs and decided to beef them up to 256GB which helped. They’re running different workloads (shared hosting) but we’ve never encountered the issue we had yesterday even during our testing/benchmarking. Kind Regards, David 5 sep 2014 kl. 09:05 skrev Christian Balzer

Re: [ceph-users] SSD journal deployment experiences

2014-09-06 Thread Christian Balzer
/msg04152.html http://comments.gmane.org/gmane.comp.file-systems.ceph.devel/10021 And I'm not going to use BTRFS for mainly RBD backed VM images (fragmentation city), never mind the other stability issues that crop up here ever so often. September 6 2014 1:27 PM, Christian Balzer ch...@gol.com wrote

Re: [ceph-users] resizing the OSD

2014-09-06 Thread Christian Balzer
would of course never bring the cluster down. However taking an OSD out and/or adding a new one will cause data movement that might impact your cluster's performance. Regards, Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan

Re: [ceph-users] Huge issues with slow requests

2014-09-06 Thread Christian Balzer
Hello, On Sat, 6 Sep 2014 17:41:02 +0200 Josef Johansson wrote: Hi, On 06 Sep 2014, at 17:27, Christian Balzer ch...@gol.com wrote: Hello, On Sat, 6 Sep 2014 17:10:11 +0200 Josef Johansson wrote: We manage to go through the restore, but the performance degradation

Re: [ceph-users] SSD journal deployment experiences

2014-09-06 Thread Christian Balzer
a decent HW RAID controller. Christian On Sat Sep 06 2014 at 8:37:56 AM Christian Balzer ch...@gol.com wrote: On Sat, 6 Sep 2014 14:50:20 + Dan van der Ster wrote: September 6 2014 4:01 PM, Christian Balzer ch...@gol.com wrote: On Sat, 6 Sep 2014 13:07:27 + Dan van der Ster

Re: [ceph-users] resizing the OSD

2014-09-06 Thread Christian Balzer
Hello, On Sat, 06 Sep 2014 10:28:19 -0700 JIten Shah wrote: Thanks Christian. Replies inline. On Sep 6, 2014, at 8:04 AM, Christian Balzer ch...@gol.com wrote: Hello, On Fri, 05 Sep 2014 15:31:01 -0700 JIten Shah wrote: Hello Cephers, We created a ceph cluster with 100

Re: [ceph-users] Is ceph osd reweight always safe to use?

2014-09-08 Thread Christian Balzer
you'd easily get into a real near full or full situation. Regards, Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communult in data movement, so pick an appropriate time), would be to ications http://www.gol.com

Re: [ceph-users] resizing the OSD

2014-09-08 Thread Christian Balzer
Hello, On Mon, 08 Sep 2014 09:53:58 -0700 JIten Shah wrote: On Sep 6, 2014, at 8:22 PM, Christian Balzer ch...@gol.com wrote: Hello, On Sat, 06 Sep 2014 10:28:19 -0700 JIten Shah wrote: Thanks Christian. Replies inline. On Sep 6, 2014, at 8:04 AM, Christian Balzer ch

Re: [ceph-users] Updating the pg and pgp values

2014-09-08 Thread Christian Balzer
and push that change to all the MON, MSD and OSD’s ? Thanks. —Jiten -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users

Re: [ceph-users] Is ceph osd reweight always safe to use?

2014-09-08 Thread Christian Balzer
:09 PM, Christian Balzer wrote: Hello, On Mon, 08 Sep 2014 11:42:59 -0400 JR wrote: Greetings all, I have a small ceph cluster (4 nodes, 2 osds per node) which recently started showing: root@ocd45:~# ceph health HEALTH_WARN 1 near full osd(s) admin@node4:~$ for i in 2 3

Re: [ceph-users] Is ceph osd reweight always safe to use?

2014-09-08 Thread Christian Balzer
not an option at the moment to upgrade to firefly (can't make a big change before sending it out the door). On 9/8/2014 12:09 PM, Christian Balzer wrote: Hello, On Mon, 08 Sep 2014 11:42:59 -0400 JR wrote: Greetings all, I have a small ceph cluster (4 nodes, 2 osds per

Re: [ceph-users] SSD journal deployment experiences

2014-09-08 Thread Christian Balzer
-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Christian Balzer Sent: Sunday, 7 September 2014 1:38 AM To: ceph-users Subject: Re: [ceph-users] SSD journal deployment experiences On Sat, 6 Sep 2014 14:50:20 + Dan van der Ster wrote: September 6 2014 4:01 PM, Christian

Re: [ceph-users] Is ceph osd reweight always safe to use?

2014-09-09 Thread Christian Balzer
) warning; should have checked more carefully. I now have the expected data movement. Thanks alot! JR On 9/8/2014 10:04 PM, Christian Balzer wrote: Hello, On Mon, 08 Sep 2014 18:30:07 -0400 JR wrote: Hi Christian, all, Having researched this a bit more, it seemed

Re: [ceph-users] NAS on RBD

2014-09-09 Thread Christian Balzer
-- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users

Re: [ceph-users] number of PGs (global vs per pool)

2014-09-09 Thread Christian Balzer
has plenty of reserves, I would go with the 1024 PGs for big pools and 128 or 256 for the small ones. Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com

Re: [ceph-users] ceph data consistency

2014-09-09 Thread Christian Balzer
-correcting code http://en.wikipedia.org/wiki/Error-correcting_code (ECC). So if the data is not correct. the disk can recovery it or return i/o error. Does anyone can explain it? http://en.wikipedia.org/wiki/Data_corruption#Silent_data_corruption -- Christian BalzerNetwork

Re: [ceph-users] SSD journal deployment experiences

2014-09-10 Thread Christian Balzer
On Tue, 9 Sep 2014 10:57:26 -0700 Craig Lewis wrote: On Sat, Sep 6, 2014 at 9:27 AM, Christian Balzer ch...@gol.com wrote: On Sat, 06 Sep 2014 16:06:56 + Scott Laird wrote: Backing up slightly, have you considered RAID 5 over your SSDs? Practically speaking, there's

Re: [ceph-users] Why so much inconsistent error in 0.85?

2014-09-10 Thread Christian Balzer
a test cluster. So finding bugs is hardly surprising, especially since you're using an experimental backend on top of that. Also due it being a development version, you might get more feedback on the ceph-devel mailing list. Christian -- Christian BalzerNetwork/Systems Engineer

Re: [ceph-users] OSDs are crashing with Cannot fork or cannot create thread but plenty of memory is left

2014-09-12 Thread Christian Balzer
___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com

Re: [ceph-users] why no likely() and unlikely() used in Ceph's source code?

2014-09-15 Thread Christian Balzer
ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph

Re: [ceph-users] Newbie Ceph Design Questions

2014-09-18 Thread Christian Balzer
://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users

Re: [ceph-users] Newbie Ceph Design Questions

2014-09-20 Thread Christian Balzer
. ^o^ On Thu, Sep 18, 2014 at 6:36 AM, Christian Balzer ch...@gol.com wrote: Hello, On Thu, 18 Sep 2014 13:07:35 +0200 Christoph Adomeit wrote: Presently we use Solaris ZFS Boxes as NFS Storage for VMs. That sounds slower than I would Ceph RBD expect to be in nearly all

Re: [ceph-users] Merging two active ceph clusters: suggestions needed

2014-09-20 Thread Christian Balzer
and the mtimes as I don't use that functionality. Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users

Re: [ceph-users] Newbie Ceph Design Questions

2014-09-21 Thread Christian Balzer
Hello, On Sun, 21 Sep 2014 21:00:48 +0200 Udo Lembke wrote: Hi Christian, On 21.09.2014 07:18, Christian Balzer wrote: ... Personally I found ext4 to be faster than XFS in nearly all use cases and the lack of full, real kernel integration of ZFS is something that doesn't appeal

[ceph-users] IRQ balancing, distribution

2014-09-22 Thread Christian Balzer
if not all the traffic generated by that card will have to transferred to the other CPU anyway. Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com

Re: [ceph-users] IRQ balancing, distribution

2014-09-22 Thread Christian Balzer
___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications

Re: [ceph-users] Newbie Ceph Design Questions

2014-09-22 Thread Christian Balzer
On Mon, 22 Sep 2014 13:35:26 +0200 Udo Lembke wrote: Hi Christian, On 22.09.2014 05:36, Christian Balzer wrote: Hello, On Sun, 21 Sep 2014 21:00:48 +0200 Udo Lembke wrote: Hi Christian, On 21.09.2014 07:18, Christian Balzer wrote: ... Personally I found ext4 to be faster

Re: [ceph-users] IRQ balancing, distribution

2014-09-22 Thread Christian Balzer
Hello, On Mon, 22 Sep 2014 08:55:48 -0500 Mark Nelson wrote: On 09/22/2014 01:55 AM, Christian Balzer wrote: Hello, not really specific to Ceph, but since one of the default questions by the Ceph team when people are facing performance problems seems to be Have you tried turning

Re: [ceph-users] [Single OSD performance on SSD] Can't go over 3, 2K IOPS

2014-09-24 Thread Christian Balzer
/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users@lists.ceph.com http

Re: [ceph-users] SSD MTBF

2014-09-29 Thread Christian Balzer
on you unexpectedly. A 200 TB DC 3700S has a TBW of 1825, more than 10 times that of your Samsungs and would allow you to write 1TB each day for 5 years. Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications

Re: [ceph-users] IO wait spike in VM

2014-09-29 Thread Christian Balzer
___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com

Re: [ceph-users] SSD MTBF

2014-09-30 Thread Christian Balzer
On Tue, 30 Sep 2014 15:26:31 +0100 Kingsley Tart wrote: On Tue, 2014-09-30 at 00:30 +0900, Christian Balzer wrote: On Mon, 29 Sep 2014 11:15:21 +0200 Emmanuel Lacour wrote: On Mon, Sep 29, 2014 at 05:57:12PM +0900, Christian Balzer wrote: Given your SSDs, are they failing after

Re: [ceph-users] Why performance of benchmarks with small blocks is extremely small?

2014-09-30 Thread Christian Balzer
.tcp_window_scaling = 1 net.ipv4.tcp_timestamps = 1 net.ipv4.tcp_sack = 1 Do we have other ways to significantly improve CEPH storage performance? Any feedback and comments are welcome! Thank you! -- Christian BalzerNetwork/Systems Engineer ch...@gol.com

Re: [ceph-users] SSD MTBF

2014-10-01 Thread Christian Balzer
@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch

Re: [ceph-users] Why performance of benchmarks with small blocks is extremely small?

2014-10-01 Thread Christian Balzer
optimisation code will be added to the next maintenance release of firefy? Andrei - Original Message - From: Timur Nurlygayanov tnurlygaya...@mirantis.com To: Christian Balzer ch...@gol.com Cc: ceph-us...@ceph.com Sent: Wednesday, 1 October, 2014 1:11:25 PM Subject: Re

Re: [ceph-users] SSD MTBF

2014-10-01 Thread Christian Balzer
on the poor NANDs as Emmanuel's environment. Christian Cheers, Martin On Wed, Oct 1, 2014 at 10:18 AM, Christian Balzer ch...@gol.com wrote: On Wed, 1 Oct 2014 09:28:12 +0200 Kasper Dieter wrote: On Tue, Sep 30, 2014 at 04:38:41PM +0200, Mark Nelson wrote: On 09/29/2014 03:58 AM

Re: [ceph-users] OSD - choose the right controller card, HBA/IT mode explanation

2014-10-01 Thread Christian Balzer
second human opinion. Thanks you for any hint you'll give! -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph

Re: [ceph-users] OSD - choose the right controller card, HBA/IT mode explanation

2014-10-01 Thread Christian Balzer
On Wed, 01 Oct 2014 20:12:03 +0200 Massimiliano Cuttini wrote: Hello Christian, Il 01/10/2014 19:20, Christian Balzer ha scritto: Hello, On Wed, 01 Oct 2014 18:26:53 +0200 Massimiliano Cuttini wrote: Dear all, i need few tips about Ceph best solution for driver controller

Re: [ceph-users] Bad cluster benchmark results

2014-10-02 Thread Christian Balzer
network nodes. Monitor your nodes with atop and see where the bottlenecks are (I still bet disks). Re-read my mail below. Christian Thanks On Wed, Oct 1, 2014 at 7:24 PM, Christian Balzer ch...@gol.com wrote: Hello, On Wed, 1 Oct 2014 14:43:49 -0700 Jakes John wrote: Hi Ceph

Re: [ceph-users] Ceph SSD array with Intel DC S3500's

2014-10-02 Thread Christian Balzer
for a cache pool and 6 classic Ceph storage nodes for permanent storage might be good enough for your use case with a future version of Ceph. But unfortunately currently cache pools aren't quite there yet. Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com

Re: [ceph-users] ceph, ssds, hdds, journals and caching

2014-10-02 Thread Christian Balzer
(doubtful, but verify), more journals. Other than that, maybe create a 1TB (usable space) SSD pool for guests with special speed requirements... Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http

Re: [ceph-users] Ceph SSD array with Intel DC S3500's

2014-10-03 Thread Christian Balzer
- From: Christian Balzer ch...@gol.com To: ceph-users@lists.ceph.com Sent: Thursday, October 2, 2014 8:54:41 PM Subject: Re: [ceph-users] Ceph SSD array with Intel DC S3500's Hello, On Thu, 2 Oct 2014 13:48:27 -0400 (EDT) Adam Boyhan wrote: Hey everyone, loving Ceph so far

Re: [ceph-users] ceph, ssds, hdds, journals and caching

2014-10-03 Thread Christian Balzer
On Fri, 3 Oct 2014 11:24:38 +0100 (BST) Andrei Mikhailovsky wrote: From: Christian Balzer ch...@gol.com To: ceph-users@lists.ceph.com Sent: Friday, 3 October, 2014 2:06:48 AM Subject: Re: [ceph-users] ceph, ssds, hdds, journals and caching On Thu, 2 Oct 2014 21:54:54 +0100 (BST

Re: [ceph-users] ceph, ssds, hdds, journals and caching

2014-10-04 Thread Christian Balzer
? -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users

Re: [ceph-users] ceph, ssds, hdds, journals and caching

2014-10-04 Thread Christian Balzer
], ATTR{queue/rotational}==0, ATTR{queue/scheduler}=noop # # set cfq scheduler for rotating disks ACTION==add|change, KERNEL==sd[a-z], ATTR{queue/rotational}==1, ATTR{queue/scheduler}=cfq Is there anything else that I am missing? -- Christian Balzer Network/Systems Engineer

Re: [ceph-users] Ceph SSD array with Intel DC S3500's

2014-10-05 Thread Christian Balzer
-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion

Re: [ceph-users] Network hardware recommendations

2014-10-05 Thread Christian Balzer
___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com

Re: [ceph-users] Network hardware recommendations

2014-10-06 Thread Christian Balzer
On Mon, 6 Oct 2014 09:17:03 + Carl-Johan Schenström wrote: Christian Balzer ch...@gol.com wrote: Any decent switch with LACP will do really. And with that I mean Cisco, Brocade etc. But that won't give you redundancy if a switch fails, see below. TRILL ( http

Re: [ceph-users] OSD - choose the right controller card, HBA/IT mode explanation

2014-10-06 Thread Christian Balzer
On Fri, 03 Oct 2014 11:56:42 +0200 Massimiliano Cuttini wrote: Il 02/10/2014 17:24, Christian Balzer ha scritto: On Thu, 02 Oct 2014 12:20:06 +0200 Massimiliano Cuttini wrote: Il 02/10/2014 03:18, Christian Balzer ha scritto: On Wed, 01 Oct 2014 20:12:03 +0200 Massimiliano Cuttini wrote

Re: [ceph-users] Network hardware recommendations

2014-10-06 Thread Christian Balzer
. Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi

Re: [ceph-users] Network hardware recommendations

2014-10-07 Thread Christian Balzer
___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com

Re: [ceph-users] Network hardware recommendations

2014-10-08 Thread Christian Balzer
PM Christian Balzer ch...@gol.com wrote: On Tue, 07 Oct 2014 20:40:31 + Scott Laird wrote: I've done this two ways in the past. Either I'll give each machine an Infiniband network link and a 1000baseT link and use the Infiniband one as the private network for Ceph, or I'll throw

Re: [ceph-users] Ceph packages being blocked by epel packages on Centos6

2014-10-13 Thread Christian Balzer
mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com

Re: [ceph-users] Performance doesn't scale well on a full ssd cluster.

2014-10-16 Thread Christian Balzer
/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users@lists.ceph.com http

[ceph-users] OSD (and probably other settings) not being picked up outside of the [global] section

2014-10-16 Thread Christian Balzer
with 0.80.1 and 0.80.6)? Regards, Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users@lists.ceph.com http

Re: [ceph-users] OSD (and probably other settings) not being picked up outside of the [global] section

2014-10-20 Thread Christian Balzer
10' That I'm also aware of, but for the time being having everything in [global] resolves the problem and more importantly makes it reboot proof. Christian On Thu, Oct 16, 2014 at 6:54 PM, Christian Balzer ch...@gol.com wrote: Hello, Consider this rather basic configuration file

Re: [ceph-users] real beginner question

2014-10-21 Thread Christian Balzer
@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph

Re: [ceph-users] Few questions.

2014-10-21 Thread Christian Balzer
or recollection of seeing this before. In general you will want to monitor all your cluster nodes with something like atop in a situation like this to spot potential problems like slow disks, CPU or network starvation, etc. Christian Please help me. Best regards ! -- Christian BalzerNetwork

[ceph-users] mon_osd_down_out_subtree_limit stuck at rack

2014-10-21 Thread Christian Balzer
. Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi

Re: [ceph-users] mon_osd_down_out_subtree_limit stuck at rack

2014-10-22 Thread Christian Balzer
Hello, On Wed, 22 Oct 2014 17:41:45 -0300 Ricardo J. Barberis wrote: El Martes 21/10/2014, Christian Balzer escribió: Hello, I'm trying to change the value of mon_osd_down_out_subtree_limit from rack to something, anything else with ceph 0.80.(6|7). Using injectargs it tells me

Re: [ceph-users] Performance doesn't scale well on a full ssd cluster.

2014-10-23 Thread Christian Balzer
and seeing if that makes any difference. If you feel like it, it might be worth also running similar tests with something like fio just to verify that the same behaviour is present. Thanks! Mark -- Christian BalzerNetwork/Systems Engineer ch

[ceph-users] All SSD storage and journals

2014-10-24 Thread Christian Balzer
, but nobody cares about those in journals. ^o^ Obvious things that come to mind in this context would be the ability to disable journals (difficult, I know, not touching BTRFS, thank you) and probably K/V store in the future. Regards, Christian -- Christian BalzerNetwork/Systems Engineer

Re: [ceph-users] All SSD storage and journals

2014-10-27 Thread Christian Balzer
, at 07:58, Christian Balzer ch...@gol.com wrote: Hello, as others have reported in the past and now having tested things here myself, there really is no point in having journals for SSD backed OSDs on other SSDs. It is a zero sum game, because: a) using that journal SSD

Re: [ceph-users] What a maximum theoretical and practical capacity in ceph cluster?

2014-10-27 Thread Christian Balzer
. It's too big or normal use case for ceph? Not too big, but definitely needs a completely different design and lots of forethought, planning and testing. Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion

Re: [ceph-users] What a maximum theoretical and practical capacity in ceph cluster?

2014-10-28 Thread Christian Balzer
___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com

Re: [ceph-users] v0.87 Giant released

2014-10-29 Thread Christian Balzer
. Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph

Re: [ceph-users] journal on entire ssd device

2014-10-29 Thread Christian Balzer
the max to 10 seconds and since Ceph also starts flushing the journal when it becomes half full there's the above goal of having 20 seconds worth of space. That all said, I'd be very happy about some journal perf counters in Ceph that show how effective and utilized it is. Christian -- Christian Balzer

Re: [ceph-users] use ZFS for OSDs

2014-10-30 Thread Christian Balzer
note, have you done any tests using the ZFS compression? I'm wondering what the performance impact and efficiency are. Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com

Re: [ceph-users] use ZFS for OSDs

2014-11-01 Thread Christian Balzer
for a single hot 4KB of data is hardly efficient. Regards, Christian Cheers, -Original Message- From: Christian Balzer [mailto:ch...@gol.com] Sent: October-30-14 4:12 AM To: ceph-users Cc: Michal Kozanecki Subject: Re: [ceph-users] use ZFS for OSDs On Wed, 29 Oct 2014 15

Re: [ceph-users] giant release osd down

2014-11-02 Thread Christian Balzer
4.0G 21% /var/lib/ceph/osd/ceph-1 My Linux OS lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description:Ubuntu 14.04 LTS Release:14.04 Codename: trusty Regards Shiv -- Christian BalzerNetwork/Systems Engineer ch

Re: [ceph-users] giant release osd down

2014-11-02 Thread Christian Balzer
On Sun, 2 Nov 2014 14:07:23 -0800 (PST) Sage Weil wrote: On Mon, 3 Nov 2014, Christian Balzer wrote: c) But wait, you specified a pool size of 2 in your OSD section! Tough luck, because since Firefly there is a bug that at the very least prevents OSD and RGW parameters from being parsed

Re: [ceph-users] giant release osd down

2014-11-03 Thread Christian Balzer
Director of Software Engineering Red Hat (Inktank is now part of Red Hat!) http://www.linkedin.com/in/ircolle http://www.twitter.com/ircolle Cell: +1.303.601.7713 Email: ico...@redhat.com - Original Message - From: Christian Balzer ch...@gol.com To: ceph-us...@ceph.com Cc: Shiv Raj

Re: [ceph-users] giant release osd down

2014-11-03 Thread Christian Balzer
On Mon, 3 Nov 2014 06:02:08 -0800 (PST) Sage Weil wrote: On Mon, 3 Nov 2014, Mark Kirkwood wrote: On 03/11/14 14:56, Christian Balzer wrote: On Sun, 2 Nov 2014 14:07:23 -0800 (PST) Sage Weil wrote: On Mon, 3 Nov 2014, Christian Balzer wrote: c) But wait, you specified a pool

Re: [ceph-users] Statistic information about rbd bandwith/usage (from a rbd/kvm client)

2014-11-09 Thread Christian Balzer
the information you're after. If you're using libvirt, using virt-top should make really busy VMs stand out pretty quickly. I'm using ganeti, so I'd _really_ love to have what you're asking for implemented in Ceph. Regards, Christian best regards Danny -- Christian BalzerNetwork

[ceph-users] Deep scrub, cache pools, replica 1

2014-11-10 Thread Christian Balzer
reading all the data in the PGs. Why? And what is it comparing that data with, the cosmic background radiation? Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com

Re: [ceph-users] Weight field in osd dump osd tree

2014-11-11 Thread Christian Balzer
-- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Deep scrub, cache pools, replica 1

2014-11-11 Thread Christian Balzer
On Tue, 11 Nov 2014 10:21:49 -0800 Gregory Farnum wrote: On Mon, Nov 10, 2014 at 10:58 PM, Christian Balzer ch...@gol.com wrote: Hello, One of my clusters has become busy enough (I'm looking at you, evil Window VMs that I shall banish elsewhere soon) to experience client noticeable

Re: [ceph-users] Solaris 10 VMs extremely slow in KVM on Ceph RBD Devices

2014-11-13 Thread Christian Balzer
Christoph -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com

Re: [ceph-users] jbod + SMART : how to identify failing disks ?

2014-11-17 Thread Christian Balzer
(not) use in their systems ? Any magical way to “blink” a drive in linux ? Thanks regards ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian Balzer

Re: [ceph-users] Troubleshooting an erasure coded pool with a cache tier

2014-11-17 Thread Christian Balzer
@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer

Re: [ceph-users] Kernel memory allocation oops Centos 7

2014-11-20 Thread Christian Balzer
represent the views of the company. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global

Re: [ceph-users] Kernel memory allocation oops Centos 7

2014-11-20 Thread Christian Balzer
From: Christian Balzer ch...@gol.com Sent: Friday, 21 November 2014 10:06 AM To: 'ceph-users' Cc: Bond, Darryl Subject: Re: [ceph-users] Kernel memory allocation oops Centos 7 On Thu, 20 Nov 2014 22:10:02 + Bond, Darryl wrote: Brief outline: 6 Node production

Re: [ceph-users] Kernel memory allocation oops Centos 7

2014-11-23 Thread Christian Balzer
, at least it's good to know that. Guess I'll keep cargo-culting that little setting for some time to come. Darryl From: Christian Balzer ch...@gol.com Sent: Friday, 21 November 2014 2:39 PM To: 'ceph-users' Cc: Bond, Darryl Subject: Re: [ceph-users

Re: [ceph-users] Quetions abount osd journal configuration

2014-11-26 Thread Christian Balzer
tell osd.* injectargs '--filestore_max_sync_interval 30' Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph

[ceph-users] OSD trashed by simple reboot (Debian Jessie, systemd?)

2014-12-04 Thread Christian Balzer
. Is it really that easy to trash your OSDs? In the case a storage node crashes, am I to expect most if not all OSDs or at least their journals to require manual loving? Regards, Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine

  1   2   3   4   5   6   7   8   9   10   >