Re: [ceph-users] Requests blocked in degraded erasure coded pool

2017-06-06 Thread Gregory Farnum
On Tue, Jun 6, 2017 at 10:12 AM, Jonas Jaszkowic wrote: > I setup a simple Ceph cluster with 5 OSD nodes and 1 monitor node. Each OSD > is on a different host. > The erasure coded pool has 64 PGs and an initial state of HEALTH_OK. > > The goal is to deliberately break as many OSDs as possible up t

Re: [ceph-users] Write back mode Cach-tier behavior

2017-06-06 Thread Christian Balzer
Hello, On Tue, 6 Jun 2017 08:58:07 -0300 Webert de Souza Lima wrote: > Hey Christian. > > Which settings do you mean? I played a lot > with hit_set_count, hit_set_period, min_read_recency_for_promote > and min_write_recency_for_promote. > They showed no effect when hit_set_count = 0. > Yes, th

Re: [ceph-users] handling different disk sizes

2017-06-06 Thread Félix Barbeira
Hi, Thanks to your answers now I understand better this part of ceph. I did the change on the crushmap that Maxime suggested, after that the results are what I expect from the beginning: # ceph osd df ID WEIGHT REWEIGHT SIZE USE AVAIL %USE VAR PGS 0 7.27100 1.0 7445G 1830G 5614G 2

Re: [ceph-users] design guidance

2017-06-06 Thread Daniel K
I started down that path and got so deep that I couldn't even find where I went in. I couldn't make heads or tails out of what would or wouldn't work. We didn't need multiple hosts accessing a single datastore, so on the client side I just have a single VM guest running on each ESXi hosts, with th

Re: [ceph-users] design guidance

2017-06-06 Thread Daniel K
Christian, Thank you for the tips -- I certainly googled my eyes out for a good while before asking -- maybe my google-fu wasn't too good last night. > I love using IB, alas with just one port per host you're likely best off > ignoring it, unless you have a converged network/switches that can mak

Re: [ceph-users] RGW lifecycle not expiring objects

2017-06-06 Thread Ben Hines
If you have nothing listed in 'lc list', you probably need to add a lifecycle configuration using the S3 API. It's not automatic and has to be added per-bucket. Here's some sample code for doing so: http://tracker.ceph.com/issues/19587 -Ben On Tue, Jun 6, 2017 at 9:07 AM, Graham Allan wrote:

[ceph-users] PG that should not be on undersized+degraded on multi datacenter Ceph cluster

2017-06-06 Thread Alejandro Comisario
Hi all, i have a multi datacenter 6 nodes (6 osd) ceph jewel cluster. There are 3 pools in the cluster, all three with size 3 and min_size 2. Today, i shut down all three nodes (controlled and in order) on datacenter "CPD2" just to validate that everything keeps working on "CPD1", whitch did (incl

[ceph-users] Requests blocked in degraded erasure coded pool

2017-06-06 Thread Jonas Jaszkowic
I setup a simple Ceph cluster with 5 OSD nodes and 1 monitor node. Each OSD is on a different host. The erasure coded pool has 64 PGs and an initial state of HEALTH_OK. The goal is to deliberately break as many OSDs as possible up to the number of coding chunks m in order to evaluate the read p

Re: [ceph-users] Bug report: unexpected behavior when executing Lua object class

2017-06-06 Thread Noah Watkins
Thanks for the explanation Greg. I think that having even 64 bytes would be useful for most scenarios I can think of, and in the worst case it could be used to provide a unique identifier to retrieve a larger blob. On Mon, Jun 5, 2017 at 11:04 AM, Gregory Farnum wrote: > > > On Mon, Jun 5, 2017 a

Re: [ceph-users] RGW lifecycle not expiring objects

2017-06-06 Thread Graham Allan
I still haven't seen anything get expired from our kraken (11.2.0) system. When I run "radosgw-admin lc list" I get no output, besides debug output (I have "debug rgw = 10" at present): # radosgw-admin lc list 2017-06-06 10:57:49.319576 7f2b26ffd700 2 RGWDataChangesLog::ChangesRenewThread: s

Re: [ceph-users] radosgw refuses upload when Content-Type missing from POST policy

2017-06-06 Thread Dave Holland
On Sun, Jun 04, 2017 at 10:15:21PM +, Robin H. Johnson wrote: > Can you share the testcase and a capture on the user-side of it running? > (tcpdump showing the HTTP headers & complete payload for > request+response would be SUPERB). Thanks Robin, I put the captures I have in tracker #20201. C

Re: [ceph-users] Lumionous: bluestore 'tp_osd_tp thread tp_osd_tp' had timed out after 60

2017-06-06 Thread Jake Grimmett
Hi Mark, OK, I'll upgrade to the current master and retest... best, Jake On 06/06/17 15:46, Mark Nelson wrote: > Hi Jake, > > I just happened to notice this was on 12.0.3. Would it be possible to > test this out with current master and see if it still is a problem? > > Mark > > On 06/06/201

Re: [ceph-users] Lumionous: bluestore 'tp_osd_tp thread tp_osd_tp' had timed out after 60

2017-06-06 Thread Mark Nelson
Hi Jake, I just happened to notice this was on 12.0.3. Would it be possible to test this out with current master and see if it still is a problem? Mark On 06/06/2017 09:10 AM, Mark Nelson wrote: Hi Jake, Thanks much. I'm guessing at this point this is probably a bug. Would you (or nokiau

Re: [ceph-users] Lumionous: bluestore 'tp_osd_tp thread tp_osd_tp' had timed out after 60

2017-06-06 Thread Mark Nelson
Hi Jake, Thanks much. I'm guessing at this point this is probably a bug. Would you (or nokiauser) mind creating a bug in the tracker with a short description of what's going on and the collectl sample showing this is not IOs backing up on the disk? If you want to try it, we have a gdb base

Re: [ceph-users] [ceph] how to copy a cloned rbd including its parent infomation?

2017-06-06 Thread David Turner
The problem is that there are changes to the clone from the snapshot. Rbd copy and similar manual commands like dd will result in the same type of copy without the parent information. The other option is to snapshot the clone and create a new clone. It won't have the same parent information, but it

Re: [ceph-users] Write back mode Cach-tier behavior

2017-06-06 Thread Webert de Souza Lima
Hey Christian. Which settings do you mean? I played a lot with hit_set_count, hit_set_period, min_read_recency_for_promote and min_write_recency_for_promote. They showed no effect when hit_set_count = 0. On Mon, Jun 5, 2017 at 11:54 PM, Christian Balzer wrote: > > Hello, > > On Tue, 06 Jun 201

Re: [ceph-users] Write back mode Cach-tier behavior

2017-06-06 Thread Webert de Souza Lima
The hit set count/period is supposed to control whether the object will be in the cache pool or in the cold stage pool. By setting to 0, the object is always promoted. This is good for writings but on my use case, for example, I wouldn't want every read operation to make an object get promoted and

Re: [ceph-users] Lumionous: bluestore 'tp_osd_tp thread tp_osd_tp' had timed out after 60

2017-06-06 Thread Jake Grimmett
Hi Mark, Thanks again for looking into this problem. I ran the cluster overnight, with a script checking for dead OSDs every second, and restarting them. 40 OSD failures occurred in 12 hours, some OSDs failed multiple times, (there are 50 OSDs in the EC tier). Unfortunately, the output of colle

[ceph-users] [ceph] how to copy a cloned rbd including its parent infomation?

2017-06-06 Thread xiaoyang...@saiway.com.cn
Hi,all: 1)I have cloned an image named "vm-win7" from snapshot named "template-win7_snapshot" #rbd clone template-pool/template-win7@template-win7_snapshot template-pool/vm-win7 2) i want to copy the cloned rbd "vm-win7" to a new one #rbd copy template-pool/vm-win7 template-pool

Re: [ceph-users] design guidance

2017-06-06 Thread Maxime Guyot
Hi Daniel, The flexibility of Ceph is that you can start with your current config, scale out and upgrade (CPUs, journals etc...) as your performance requirement increase. 6x1.7Ghz, are we speaking about the Xeon E5 2603L v4? Any chance to bump that to 2620 v4 or 2630 v4? Test how the 6x1.7Ghz han

Re: [ceph-users] Write back mode Cach-tier behavior

2017-06-06 Thread TYLin
> On Jun 6, 2017, at 11:18 AM, jiajia zhong wrote: > > it's very similar to ours. but is there any need to seperate the osds for > different pools ? why ? > below's our crushmap. > > -98 6.29997 root tier_cache > -94 1.3 host cephn1-ssd

Re: [ceph-users] Write back mode Cach-tier behavior

2017-06-06 Thread TYLin
> On Jun 6, 2017, at 10:35 AM, Webert de Souza Lima > wrote: > > I'd like to add that, from all tests I did, the writing of new files only go > directly to the cache tier if you set hit set count = 0. > > Is there any concern or disadvantage once I set hit set count = 0? Looks like it is u

Re: [ceph-users] Kraken bluestore compression

2017-06-06 Thread ceph
I may fool myself, but as far as I know: - kraken introduces compression for RGW (not on the OSD level, not for rbd) - kraken stabilizes bluestore, a new OSD format, that introduces compression on the OSD level On 06/06/2017 04:36, Daniel K wrote: > Hi, > > I see several mentions that compressi

Re: [ceph-users] handling different disk sizes

2017-06-06 Thread Maxime Guyot
Hi Félix, Changing the failure domain to OSD is probably the easiest option if this is a test cluster. I think the commands would go like: - ceph osd getcrushmap -o map.bin - crushtool -d map.bin -o map.txt - sed -i 's/step chooseleaf firstn 0 type host/step chooseleaf firstn 0 type osd/' map.txt

Re: [ceph-users] ceph-users Digest, Vol 53, Issue 4

2017-06-06 Thread Zigor Ozamiz
Hi, > Is your min-size at least 2? Is it just one OSD affected? Yes, min_size is 2: # ceph osd pool get vmtier-10 min_size min_size: 2 Yes, affected OSD is only one. > > If yes and if it is only the journal that is corrupt, but the actual OSD > store is intact although lagging behind now in wri