[ceph-users] SSD Journal

2016-07-12 Thread Ashley Merrick
Hello, Looking at final stages of planning / setup for a CEPH Cluster. Per a Storage node looking @ 2 x SSD OS / Journal 10 x SATA Disk Will have a small Raid 1 Partition for the OS, however not sure if best to do: 5 x Journal Per a SSD 10 x Journal on Raid 1 of two SSD's Is the

Re: [ceph-users] SSD Journal

2016-07-13 Thread Ashley Merrick
[mailto:w...@42on.com] Sent: 13 July 2016 10:44 To: Ashley Merrick <ash...@amerrick.co.uk>; ceph-users@lists.ceph.com; Christian Balzer <ch...@gol.com> Subject: RE: [ceph-users] SSD Journal > Op 13 juli 2016 om 11:34 schreef Ashley Merrick <ash...@amerrick.co.uk>: >

Re: [ceph-users] SSD Journal

2016-07-13 Thread Ashley Merrick
: Christian Balzer [mailto:ch...@gol.com] Sent: 13 July 2016 01:12 To: ceph-users@lists.ceph.com Cc: Wido den Hollander <w...@42on.com>; Ashley Merrick <ash...@amerrick.co.uk> Subject: Re: [ceph-users] SSD Journal Hello, On Tue, 12 Jul 2016 19:14:14 +0200 (CEST) Wido den Hol

Re: [ceph-users] CEPH Replication

2016-07-01 Thread Ashley Merrick
/rados/operations/crush-map/#crush-map-bucket-types and http://docs.ceph.com/docs/hammer/rados/configuration/pool-pg-config-ref/) On 01/07/2016 13:49, Ashley Merrick wrote: > Hello, > > Looking at setting up a new CEPH Cluster, starting with the following. > > 3 x CEPH OSD Se

Re: [ceph-users] CEPH Replication

2016-07-01 Thread Ashley Merrick
Hello, Okie makes perfect sense. So if run CEPH with a replication of 3, is it still required to run an odd number of OSD Nodes. Or could I run 4 OSD Nodes to start with, with a replication of 3, with each replication on a separate server. ,Ashley Merrick -Original Message- From

[ceph-users] CEPH Replication

2016-07-01 Thread Ashley Merrick
Hello, Looking at setting up a new CEPH Cluster, starting with the following. 3 x CEPH OSD Servers Each Server: 20Gbps Network 12 OSD's SSD Journal Looking at running with replication of 2, will there be any issues using 3 nodes with a replication of two, this should "technically" give me ½

Re: [ceph-users] Ceph on XenServer

2017-02-25 Thread Ashley Merrick
Just to give me 50 cents, Proxmox uses full KVM virt, they offer their own GUI and storage management on top of standard QEMU/KVM. ,Ashley Sent from my iPhone On 25 Feb 2017, at 10:12 PM, Massimiliano Cuttini > wrote: Hi Brian, never listen

[ceph-users] OSD Repeated Failure

2017-02-10 Thread Ashley Merrick
Hello, I have a particular OSD (53), which at random will crash with the OSD process stopping. OS: Debian 8.x CEPH : ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367) >From the logs at the time of the OSD being marked as crashed I can only see >the following: -4> 2017-02-10

[ceph-users] CrushMap Rule Change

2017-03-02 Thread Ashley Merrick
Hello, I am currently doing some erasure code tests in a dev environment. I have set the following by "default" rule sas { ruleset 2 type erasure min_size 3 max_size 13 step set_chooseleaf_tries 5 step set_choose_tries 100 step take fourtb

[ceph-users] SSS Caching

2016-10-26 Thread Ashley Merrick
Hello All, Currently running a CEPH cluster connected to KVM via the KRBD and used only for this purpose. Is working perfectly fine, however would like to look at increasing / helping with random write performance and latency, specially from multiple VM's hitting the spinning disks at same

Re: [ceph-users] Antw: Re: SSS Caching

2016-10-27 Thread Ashley Merrick
;> 2016 um 04:07: Hi, > Hello, > > On Wed, 26 Oct 2016 15:40:00 + Ashley Merrick wrote: > >> Hello All, >> >> Currently running a CEPH cluster connected to KVM via the KRBD and >> used only > for this purpose. >> >> Is working perfe

[ceph-users] Monitoring Overhead

2016-10-23 Thread Ashley Merrick
Hello, This may come across as a simple question but just wanted to check. I am looking at importing live data from my cluster via ceph -s e.t.c into a graphical graph interface so I can monitor performance / iops / e.t.c overtime. I am looking to pull this data from one or more monitor nodes,

Re: [ceph-users] XFS no space left on device

2016-10-25 Thread Ashley Merrick
Sounds like 32bit Inode limit, if you mount with -o inode64 (not 100% how you would do in ceph), would allow data to continue to be wrote. ,Ashley -Original Message- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of ??? ??? Sent: 25 October 2016 12:38 To:

Re: [ceph-users] Monitoring Overhead

2016-10-24 Thread Ashley Merrick
] Sent: 24 October 2016 11:04 To: ceph-users@lists.ceph.com Cc: John Spray <jsp...@redhat.com>; Ashley Merrick <ash...@amerrick.co.uk> Subject: Re: [ceph-users] Monitoring Overhead Hello, On Mon, 24 Oct 2016 10:46:31 +0100 John Spray wrote: > On Mon, Oct 24, 2016 at 4:21 AM, Ashl

[ceph-users] Hammer Cache Tiering

2016-11-01 Thread Ashley Merrick
Hello, Currently using a Proxmox & CEPH cluster, currently they are running on Hammer looking to update to Jewel shortly, I know I can do a manual upgrade however would like to keep what is tested well with Proxmox. Looking to put a SSD Cache tier in front, however have seen and read there has

Re: [ceph-users] Hammer Cache Tiering

2016-11-01 Thread Ashley Merrick
Merrick <ash...@amerrick.co.uk> Subject: Re: [ceph-users] Hammer Cache Tiering Hello, On Tue, 1 Nov 2016 15:07:33 + Ashley Merrick wrote: > Hello, > > Currently using a Proxmox & CEPH cluster, currently they are running on > Hammer looking to update to Jewel sh

Re: [ceph-users] Hammer Cache Tiering

2016-11-01 Thread Ashley Merrick
: Wednesday, 2 November 2016 12:57 PM To: Ashley Merrick <ash...@amerrick.co.uk> Cc: Christian Balzer <ch...@gol.com>; ceph-us...@ceph.com Subject: Re: [ceph-users] Hammer Cache Tiering On Wed, Nov 2, 2016 at 5:19 PM, Ashley Merrick <ash...@amerrick.co.uk<mailto:ash...@amerrick.co

[ceph-users] Unbalanced OSD's

2016-12-29 Thread Ashley Merrick
Hello, I currently have 5 servers within my CEPH Cluster 2 x (10 * 8TB Disks) 3 x (10 * 4TB Disks) Currently seeing a larger difference in OSD use across the two separate server types, as well as within the server itself. For example on one 4TB server I have an OSD at 64% and one at 84%,

[ceph-users] Kraken + Bluestore

2017-03-27 Thread Ashley Merrick
Hi, Does anyone have any cluster of a decent scale running on Kraken and bluestore? How are you finding it? Have you had any big issues arise? Was it running non bluestore before and have you noticed any improvement? Read ? Write? IOPS? ,Ashley Sent from my iPhone

Re: [ceph-users] Understanding Ceph in case of a failure

2017-03-19 Thread Ashley Merrick
Might be good if you can attach the full decompiled crushmap so we can see exactly how things are listed/setup. -Original Message- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Karol Babioch Sent: 19 March 2017 20:42 To: ceph-users@lists.ceph.com Subject:

Re: [ceph-users] Latest Jewel New OSD Creation

2017-03-12 Thread Ashley Merrick
After rolling back to 10.2.5 the issue has gone, seems there has been a change in 10.2.6 which breaks this. ,Ashley From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Ashley Merrick Sent: Saturday, 11 March 2017 11:32 AM To: ceph-us...@ceph.com Subject: [ceph-users

[ceph-users] Latest Jewel New OSD Creation

2017-03-10 Thread Ashley Merrick
Hello, I am trying to add a new OSD to my CEPH Cluster, I am running Proxmox so attempted to as normal via the GUI as normal however received an error output at the following command: ceph-disk prepare --zap-disk --fs-type xfs --cluster ceph --cluster-uuid 51c1b5c5-e510-4ed3-8b09-417214edb3f4

[ceph-users] CEPH MON Updates Live

2017-04-24 Thread Ashley Merrick
Hey, Quick question hopefully have tried a few Google searches but noting concrete. I am running KVM VM's using KRBD, if I add and remove CEPH mon's are the running VM's updated with this information. Or do I need to reboot the VM's for them to be provided with the change of MON's? Thanks!

[ceph-users] BlueStore SSD

2017-08-14 Thread Ashley Merrick
Hello, Currently run 10x4TB , 2xSSD for Journal, planning to move fully to BS, looking at adding extra servers. With the removal of the double write on BS and from the testing so far of BS (having WAL & DB on SSD Seeing very minimal SSD use) Does it make sense for further servers to go with

Re: [ceph-users] PG Stuck EC Pool

2017-07-12 Thread Ashley Merrick
Is this planned to be merged into Luminous at some point? ,Ashley From: Gregory Farnum [mailto:gfar...@redhat.com] Sent: Tuesday, 6 June 2017 2:24 AM To: Ashley Merrick <ash...@amerrick.co.uk>; ceph-us...@ceph.com Cc: David Zafman <dzaf...@redhat.com> Subject: Re: [ceph-users] PG S

Re: [ceph-users] OSD Full Ratio Luminous - Unset

2017-07-10 Thread Ashley Merrick
Hello, Perfect thanks that fixed my issue! Still seems to be a bug on the ceph pg dump unless it has been moved out of the PG and directly into the OSD? ,Ashley -Original Message- From: Edward R Huyer [mailto:erh...@rit.edu] Sent: Tuesday, 11 July 2017 7:53 AM To: Ashley Merrick <

Re: [ceph-users] How to set up bluestore manually?

2017-07-07 Thread Ashley Merrick
Looks good, just one comment the drive I used was 12GB for the WAL and DB and CEPH still only set the small sizes from my earlier reply. So not sure if / what benefit their is of the big sizes, and how CEPH-DISK sets the size or if they are just hard coded. ,Ashley Sent from my iPhone On 7

Re: [ceph-users] Mon's crashing after updating

2017-07-18 Thread Ashley Merrick
On 18 Jul 2017, at 8:14 PM, John Spray <jsp...@redhat.com<mailto:jsp...@redhat.com>> wrote: On Tue, Jul 18, 2017 at 12:43 PM, Ashley Merrick <ash...@amerrick.co.uk<mailto:ash...@amerrick.co.uk>> wrote: Hello, I just updated to latest CEPH Lum RC, all was working fi

Re: [ceph-users] Mon's crashing after updating

2017-07-18 Thread Ashley Merrick
Perfect seems to have worked, so look's like it was the same bug. Thanks, Ashley -Original Message- From: John Spray [mailto:jsp...@redhat.com] Sent: Tuesday, 18 July 2017 8:27 PM To: Ashley Merrick <ash...@amerrick.co.uk> Cc: ceph-us...@ceph.com Subject: Re: [ceph-users] Mon's cr

[ceph-users] Mon's crashing after updating

2017-07-18 Thread Ashley Merrick
d43] 10: (DispatchQueue::entry()+0x7ca) [0x55ae71683fda] 11: (DispatchQueue::DispatchThread::entry()+0xd) [0x55ae714e554d] 12: (()+0x8064) [0x7f7cab6f9064] 13: (clone()+0x6d) [0x7f7ca8c0162d] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Thanks, Ashley Merr

[ceph-users] skewed osd utilization

2017-07-18 Thread Ashley Merrick
Hello, On a updated Lum cluster I am getting the following health warning (skewed osd utilization). The reason for this is I have a set of SSD’s in a cache which are much emptier than my standard SAS disks putting the ration off massively. Is it possible to tell it to exclude certain disks

Re: [ceph-users] OSD Full Ratio Luminous - Unset

2017-07-07 Thread Ashley Merrick
. ,Ashley From: Brad Hubbard <bhubb...@redhat.com> Sent: 07 July 2017 10:31:01 To: Ashley Merrick Cc: ceph-us...@ceph.com Subject: Re: [ceph-users] OSD Full Ratio Luminous - Unset On Fri, Jul 7, 2017 at 4:49 PM, Ashley Merrick <ash...@amerrick.co.uk> wrote: > After looking into th

Re: [ceph-users] Luminous RC OSD Crashing

2017-07-19 Thread Ashley Merrick
-- 0x55f1d5083180 con 0 -9998> 2017-07-19 12:50:46.692388 7f19330ee700 1 -- 172.16.3.3:6806/3482 --> 172.16.3.4:6800/27027 -- MOSDECSubOpReadReply(6.71s0 102354/102344 ECSubReadReply(tid=605721, attrs_read=0)) v2 -- 0x55f2412c1700 con 0 ,Ashley From: Ashley Merrick Sent: Wednesday, 19 July 2

Re: [ceph-users] Luminous RC OSD Crashing

2017-07-19 Thread Ashley Merrick
Logged a bug ticket, let me know if need anything further : http://tracker.ceph.com/issues/20687 From: Ashley Merrick Sent: Wednesday, 19 July 2017 8:05 PM To: ceph-us...@ceph.com Subject: RE: Luminous RC OSD Crashing Also found this error on some of the OSD's crashing: 2017-07-19 12:50

[ceph-users] Luminous RC OSD Crashing

2017-07-19 Thread Ashley Merrick
Hello, Getting the following on random OSD's crashing during a backfill/rebuilding on the latest RC, from the log's so far I have seen the following: 172.16.3.10:6802/21760 --> 172.16.3.6:6808/15997 -- pg_update_log_missing(6.19ds12 epoch 101931/101928 rep_tid 59 entries 101931'55683 (0'0)

Re: [ceph-users] Luminous RC OSD Crashing

2017-07-19 Thread Ashley Merrick
: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Ashley Merrick Sent: Wednesday, 19 July 2017 5:47 PM To: ceph-us...@ceph.com Subject: [ceph-users] Luminous RC OSD Crashing Hello, Getting the following on random OSD's crashing during a backfill/rebuilding on the latest RC, from

Re: [ceph-users] Luminous RC OSD Crashing

2017-07-19 Thread Ashley Merrick
-users-boun...@lists.ceph.com] On Behalf Of Ashley Merrick Sent: Wednesday, 19 July 2017 5:47 PM To: ceph-us...@ceph.com Subject: [ceph-users] Luminous RC OSD Crashing Hello, Getting the following on random OSD's crashing during a backfill/rebuilding on the latest RC, from the log's so far I have

Re: [ceph-users] slow cluster perfomance during snapshot restore

2017-06-29 Thread Ashley Merrick
Many others I’m sure will comment on the snapshot specifics. However running a cluster with some 8TB drives I have noticed huge differences between 4TB and 8TB drives and their peak latency’s when busy. So along with the known snapshot performance you may find the higher seek time and higher

[ceph-users] OSD Full Ratio Luminous - Unset

2017-07-03 Thread Ashley Merrick
Hello, On a Luminous upgraded from Jewel I am seeing the following in ceph -s : "Full ratio(s) out of order" and ceph pg dump | head dumped all version 44281 stamp 2017-07-04 05:52:08.337258 last_osdmap_epoch 0 last_pg_scan 0 full_ratio 0 nearfull_ratio 0 I have tried to inject the values

Re: [ceph-users] OSD Full Ratio Luminous - Unset

2017-07-04 Thread Ashley Merrick
Okie noticed their is a new command to set these. Tried these and still showing as 0 and error on full ratio out of order "ceph osd set-{full,nearfull,backfillfull}-ratio" ,Ashley From: ceph-users <ceph-users-boun...@lists.ceph.com> on

Re: [ceph-users] OSD Full Ratio Luminous - Unset

2017-07-06 Thread Ashley Merrick
Anyone have some feedback on this? Happy to log a bug ticket if it is one, but want to make sure not missing something Luminous change related. ,Ashley Sent from my iPhone On 4 Jul 2017, at 3:30 PM, Ashley Merrick <ash...@amerrick.co.uk<mailto:ash...@amerrick.co.uk>> wrote:

Re: [ceph-users] OSD Full Ratio Luminous - Unset

2017-07-07 Thread Ashley Merrick
. However does not seem to be giving the expected results. ,Ashley From: ceph-users <ceph-users-boun...@lists.ceph.com> on behalf of Ashley Merrick <ash...@amerrick.co.uk> Sent: 06 July 2017 12:44:09 To: ceph-us...@ceph.com Subject: Re: [ceph-users] OS

Re: [ceph-users] How to set up bluestore manually?

2017-07-07 Thread Ashley Merrick
I can run the following command with no issue however and have done for multiple OSD's which work fine, it just creates an sdd1 and sdd2 ceph-disk prepare --bluestore /dev/sdg --block.wal /dev/sdd --block.db /dev/sdd ,Ashley From: ceph-users

Re: [ceph-users] How to set up bluestore manually?

2017-07-07 Thread Ashley Merrick
system NameFlags 1 1049kB 1075MB 1074MB ceph block.db 2 1075MB 1679MB 604MBceph block.wal ,Ashley From: Martin Emrich <martin.emr...@empolis.com> Sent: 07 July 2017 08:12:24 To: Ashley Merrick; Vasu Ku

Re: [ceph-users] Very slow cache flush

2017-05-17 Thread Ashley Merrick
Hello, I have the same use case as you. After using the same settings as you seems to be running correctly and clearing it self. Will monitor and let you know. Thanks for that. ,Ashley From: David Turner [mailto:drakonst...@gmail.com] Sent: 17 May 2017 22:52 To: Ashley Merrick <

[ceph-users] Very slow cache flush

2017-05-17 Thread Ashley Merrick
Hello, I recently doubles the PG's on my cache pool, from 256 to 512. I have made sure all the PG's in question have been deep scrubbed however my cache no longer wish flush or evict at any decent speed. If left to it's own it does a couple of objects here and there, but eventually grows and

Re: [ceph-users] Flash for mon nodes ?

2017-06-21 Thread Ashley Merrick
If you just mean normal DC rated SSD’s then that’s what I am running across a ~120 OSD cluster. When checking they are very unbusy and minimal use, however I can imagine the lower random latency will always help. So if you can I would. ,Ashley Sent from my iPhone On 21 Jun 2017, at 6:39 PM,

[ceph-users] SSD OSD's Dual Use

2017-06-21 Thread Ashley Merrick
Hello, Currently have a pool of SSD's running as a Cache in front of a EC Pool. The cache is very under used and the SSD's spend most time idle, would like to create a small SSD Pool for a selection of very small RBD disk's as scratch disks within the OS, should I expect any issues running

Re: [ceph-users] Squeezing Performance of CEPH

2017-06-22 Thread Ashley Merrick
Hello, Also as Mark put, one minute your testing bandwidth capacity, next minute your testing disk capacity. No way is a small set of SSD’s going to be able to max your current bandwidth, even if you removed the CEPH / Journal overhead. I would say the speeds you are getting are what you

Re: [ceph-users] Squeezing Performance of CEPH

2017-06-23 Thread Ashley Merrick
Cuttini <m...@phoenixweb.it<mailto:m...@phoenixweb.it>> wrote: Ashley, but.. instead of use NVMe as a journal, why don't add 2 OSD to the cluster? Incresing number of OSD instead of improving performance of actual OSD? Il 23/06/2017 15:40, Ashley Merrick ha scritto: Sorry for the not

Re: [ceph-users] 6 osds on 2 hosts, does Ceph always write data in one osd on host1 and replica in osd on host2?

2017-06-26 Thread Ashley Merrick
Hello, Will need to see a full export of your crush map rules. Depends what the failure domain is set to. ,Ash Sent from my iPhone On 26 Jun 2017, at 4:11 PM, Stéphane Klein > wrote: Hi, I have this OSD:

Re: [ceph-users] 6 osds on 2 hosts, does Ceph always write data in one osd on host1 and replica in osd on host2?

2017-06-26 Thread Ashley Merrick
Your going across host’s so each replication will be on a different host. ,Ashley Sent from my iPhone On 26 Jun 2017, at 4:39 PM, Stéphane Klein <cont...@stephane-klein.info<mailto:cont...@stephane-klein.info>> wrote: 2017-06-26 11:15 GMT+02:00 Ashley Merrick <ash...

[ceph-users] Primary Affinity / EC Pool

2017-06-26 Thread Ashley Merrick
Have some 8TB drives I am looking to remove from cluster Long term however would like to make use of Primary Affinity to decrease the reads going to these drives. I have a replication and erasure code pool, I understand when setting the primary Affinity to 0 no PG’s will have their Primary PG

Re: [ceph-users] v12.1.0 Luminous RC released

2017-06-26 Thread Ashley Merrick
With the EC Overwite support, if currently running behind a cache tier in Jewel will the overwrite still be of benefit through the cache tier and remove the need to promote the full block to make any edits? Or we better totally removing the cache tier once fully upgraded? ,Ashley Sent from my

Re: [ceph-users] Squeezing Performance of CEPH

2017-06-23 Thread Ashley Merrick
You could move your Journal to another SSD this would remove the double write. Ideally you’d want one or two PCIe NVME in the servers for the Journal. Or if you can hold off a bit then bluestore, which removes the double write, however is still handy to move some of the services to a seperate

Re: [ceph-users] Squeezing Performance of CEPH

2017-06-23 Thread Ashley Merrick
Sorry for the not inline reply. If you can get 6 OSD’s per a NVME as long as your getting a decent rated NVME your bottle neck will be the NVME but will still improve over your current bottle neck. You could add two NVME OSD’s, but their higher performance would be lost along with the other

Re: [ceph-users] Network redundancy...

2017-05-29 Thread Ashley Merrick
The switches your using can they stack? If so you could spread the LACP across the two switches. Sent from my iPhone > On 29 May 2017, at 4:38 PM, Marco Gaiarin wrote: > > > I've setup a little Ceph cluster (3 host, 12 OSD), all belonging to a > single switch, using

Re: [ceph-users] Recovery stuck in active+undersized+degraded

2017-06-02 Thread Ashley Merrick
You only have 3 osd's hence with one down you only have 2 left for replication of 3 objects. No spare OSD to place the 3rd object on, if you was to add a 4th node the issue would be removed. ,Ashley On 2 Jun 2017, at 10:31 PM, Oleg Obleukhov >

Re: [ceph-users] PG Stuck EC Pool

2017-06-03 Thread Ashley Merrick
ing that could be causing it. ,Ashley From: Ashley Merrick Sent: 01 June 2017 23:39 To: ceph-us...@ceph.com Subject: RE: PG Stuck EC Pool Have attached the full pg query for the effected PG encase this shows anything of interest. Thanks From: ceph-users [mailto:ceph-users-boun...@lists.ceph.

Re: [ceph-users] PG Stuck EC Pool

2017-06-03 Thread Ashley Merrick
From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Ashley Merrick Sent: 03 June 2017 17:14 To: ceph-us...@ceph.com Subject: Re: [ceph-users] PG Stuck EC Pool This sender failed our fraud detection checks and may not be who they appear to be. Learn about spoofing<http:/

Re: [ceph-users] PG Stuck EC Pool

2017-06-03 Thread Ashley Merrick
314663 0 3883338397) 0x5569ea886b00 con 0x5569ea99c880 From: Ashley Merrick Sent: 03 June 2017 14:27 To: 'ceph-us...@ceph.com' <ceph-us...@ceph.com> Subject: RE: PG Stuck EC Pool >From this extract from pg query: "up": [ 11,

Re: [ceph-users] PG Stuck EC Pool

2017-06-01 Thread Ashley Merrick
Have attached the full pg query for the effected PG encase this shows anything of interest. Thanks From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Ashley Merrick Sent: 01 June 2017 17:19 To: ceph-us...@ceph.com Subject: [ceph-users] PG Stuck EC Pool This sender

[ceph-users] PG Stuck EC Pool

2017-06-01 Thread Ashley Merrick
Have a PG which is stuck in this state (Is an EC with K=10 M=3) pg 6.14 is active+undersized+degraded+remapped+inconsistent+backfilling, acting [2147483647,2147483647,84,83,22,26,69,72,53,59,8,4,46] Currently have no-recover set, if I unset no recover both OSD 83 + 84 start to flap and go

Re: [ceph-users] RBD I/O errors with QEMU [luminous upgrade/osd change]

2017-09-11 Thread Ashley Merrick
Could this also have an effect on kRBD client's? If so what ceph auth caps command should we use? From: ceph-users on behalf of Jason Dillaman Sent: 11 September 2017 22:00:47 To: Nico Schottelius Cc:

Re: [ceph-users] cephx

2017-10-13 Thread Ashley Merrick
r old. ,Ashley ________ From: Ashley Merrick Sent: 13 October 2017 07:54:27 To: Shinobu Kinjo Cc: ceph-us...@ceph.com Subject: Re: [ceph-users] cephx Hello, http://docs.ceph.com/docs/master/rados/operations/user-management/ User Management — Ceph Documentation<http://docs.c

[ceph-users] cephx

2017-10-13 Thread Ashley Merrick
Hello, Is it possible to limit a cephx user to one image? I have looked and seems it's possible per a pool, but can't find a per image option. ,Ashley ___ ceph-users mailing list ceph-users@lists.ceph.com

Re: [ceph-users] cephx

2017-10-13 Thread Ashley Merrick
t) ,Ashley From: Shinobu Kinjo <ski...@redhat.com> Sent: 13 October 2017 07:41 To: Ashley Merrick Cc: ceph-us...@ceph.com Subject: Re: [ceph-users] cephx On Fri, Oct 13, 2017 at 3:29 PM, Ashley Merrick <ash...@amerrick.co.uk> wrote: > Hello, &g

[ceph-users] Luminous Upgrade KRBD

2017-09-05 Thread Ashley Merrick
Hello, Have recently upgraded a cluster to Luminous (Running Proxmox), at the same time I have upgraded the Compute Cluster to 5.x meaning we now run the latest kernel version (Linux 4.10.15-1) Looking to do the following : ceph osd set-require-min-compat-client luminous Below is the output

Re: [ceph-users] Luminous Upgrade KRBD

2017-09-06 Thread Ashley Merrick
From: Henrik Korkuc <li...@kirneh.eu> Sent: 06 September 2017 06:58:52 To: Ashley Merrick; ceph-us...@ceph.com Subject: Re: [ceph-users] Luminous Upgrade KRBD On 17-09-06 07:33, Ashley Merrick wrote: Hello, Have recently upgraded a cluster to Luminous (Running Proxmox), at th

Re: [ceph-users] Luminous Upgrade KRBD

2017-09-06 Thread Ashley Merrick
Okie thanks all, will hold off  -Original Message- From: Ilya Dryomov [mailto:idryo...@gmail.com] Sent: 06 September 2017 17:58 To: Ashley Merrick <ash...@amerrick.co.uk> Cc: Henrik Korkuc <li...@kirneh.eu>; ceph-us...@ceph.com Subject: Re: [ceph-users] Luminous Upgrade

Re: [ceph-users] Luminous Upgrade KRBD

2017-09-06 Thread Ashley Merrick
[mailto:idryo...@gmail.com] Sent: 06 September 2017 17:09 To: Henrik Korkuc <li...@kirneh.eu> Cc: Ashley Merrick <ash...@amerrick.co.uk>; ceph-us...@ceph.com Subject: Re: [ceph-users] Luminous Upgrade KRBD On Wed, Sep 6, 2017 at 9:16 AM, Henrik Korkuc <li...@kirneh.eu> wrote: &

[ceph-users] Crush Map for test lab

2017-10-11 Thread Ashley Merrick
Hello, Setting up a new test lab, single server 5 disks/OSD. Want to run an EC Pool that has more shards than avaliable OSD's , is it possible to force crush to 're use an OSD for another shard? I know normally this is bad practice but is for testing only on a single server setup. Thanks,

[ceph-users] cephfs ceph-fuse performance

2017-10-18 Thread Ashley Merrick
Hello, I have been trying cephfs on the latest 12.x release. Performance under cephfs mounted via kernel seems to be as expected maxing out the underlying storage / resources using kernel version 4.13.4. However when it comes to mounting cephfs via ceph-fuse looking at performance of 5-10%

[ceph-users] OSD Random Failures - Latest Luminous

2017-11-15 Thread Ashley Merrick
Hello, After replacing a single OSD disk due to a failed disk I am now seeing 2-3 OSD's randomly stop and fail to start, do a boot loop get to load_pgs and then fail with the following (I tried setting OSD log's to 5/5 but didn't get any extra lines around the error just more information pre

Re: [ceph-users] OSD Random Failures - Latest Luminous

2017-11-16 Thread Ashley Merrick
Have created a ticket http://tracker.ceph.com/issues/22144 Feel free to add anything extra you have seen. From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Ashley Merrick Sent: 16 November 2017 17:27 To: Eric Nelson <ericnel...@gmail.com> Cc: ceph-us...@ceph.com S

Re: [ceph-users] Is the 12.2.1 really stable? Anybody have production cluster with Luminous Bluestore?

2017-11-16 Thread Ashley Merrick
Currently experiencing a nasty bug http://tracker.ceph.com/issues/21142 I would say wait a while for the next point release. ,Ashley -Original Message- From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Jack Sent: 16 November 2017 22:22 To:

Re: [ceph-users] OSD Random Failures - Latest Luminous

2017-11-18 Thread Ashley Merrick
55700 thread_name:tp_peering" ,Ashley From: Ashley Merrick Sent: 16 November 2017 17:27 To: Eric Nelson <ericnel...@gmail.com> Cc: ceph-us...@ceph.com Subject: Re: [ceph-users] OSD Random Failures - Latest Luminous Hello, Good to hear it's not just me, however have a cluster basically offline

Re: [ceph-users] OSD Random Failures - Latest Luminous

2017-11-18 Thread Ashley Merrick
Hello, Will try with the noup now and see if makes any difference. Is effecting both BS & FS OSD’s and effecting different host’s and different PG’s seems to be no form of pattern. ,Ashley From: David Turner [mailto:drakonst...@gmail.com] Sent: 18 November 2017 22:19 To: Ashley Merrick

Re: [ceph-users] OSD Random Failures - Latest Luminous

2017-11-18 Thread Ashley Merrick
a different PG listed on the failure. ,Ashley From: David Turner [mailto:drakonst...@gmail.com] Sent: 18 November 2017 22:19 To: Ashley Merrick <ash...@amerrick.co.uk> Cc: Eric Nelson <ericnel...@gmail.com>; ceph-us...@ceph.com Subject: Re: [ceph-users] OSD Random Failures - Latest

Re: [ceph-users] OSD Random Failures - Latest Luminous

2017-11-18 Thread Ashley Merrick
[mailto:sean.redmo...@gmail.com] Sent: 18 November 2017 22:40 To: Ashley Merrick <ash...@amerrick.co.uk> Cc: David Turner <drakonst...@gmail.com>; ceph-users <ceph-us...@ceph.com> Subject: Re: [ceph-users] OSD Random Failures - Latest Luminous Hi, Is it possible to add new

Re: [ceph-users] Erasure Coding Pools and PG calculation - documentation

2017-11-11 Thread Ashley Merrick
Hello, Are you having any issues with getting the pool working or just around the PG num you should use? ,Ashley Get Outlook for Android From: ceph-users on behalf of Tim Gipson

Re: [ceph-users] OSD Random Failures - Latest Luminous

2017-11-20 Thread Ashley Merrick
hley From: Ashley Merrick Sent: 20 November 2017 08:56:15 To: Gregory Farnum Cc: David Turner; ceph-us...@ceph.com Subject: Re: [ceph-users] OSD Random Failures - Latest Luminous Hello, So I tried as suggested marking one OSD that continuously failed as lost and add a new OSD to take it's p

Re: [ceph-users] OSD Random Failures - Latest Luminous

2017-11-18 Thread Ashley Merrick
Will add to ticket. But no the cluster does not see the OSD go up, just the OSD fails on the same assert. ,Ashley From: David Turner [mailto:drakonst...@gmail.com] Sent: 18 November 2017 23:19 To: Ashley Merrick <ash...@amerrick.co.uk> Cc: Eric Nelson <ericnel...@gmail.com&

Re: [ceph-users] OSD Random Failures - Latest Luminous

2017-11-18 Thread Ashley Merrick
0> 2017-11-18 16:31:23.653865 7f78721ae700 -1 *** Caught signal (Aborted) ** in thread 7f78721ae700 thread_name:tp_peering ,Ashley From: David Turner [mailto:drakonst...@gmail.com] Sent: 18 November 2017 23:19 To: Ashley Merrick <ash...@amerrick.co.uk> Cc: Eric Nelson <ericnel...@gmai

Re: [ceph-users] OSD Random Failures - Latest Luminous

2017-11-20 Thread Ashley Merrick
017 09:25:39 To: Ashley Merrick Cc: David Turner; ceph-us...@ceph.com Subject: Re: [ceph-users] OSD Random Failures - Latest Luminous I only see two asserts (in my local checkout) in that function; one is metadata assert(info.history.same_interval_since != 0); and the other is a sanity

Re: [ceph-users] OSD Random Failures - Latest Luminous

2017-11-16 Thread Ashley Merrick
tps://aka.ms/ghei36> From: Ashley Merrick Sent: Thursday, November 16, 2017 9:31:22 PM To: Ashley Merrick; Eric Nelson Cc: ceph-us...@ceph.com Subject: RE: [ceph-users] OSD Random Failures - Latest Luminous Have created a ticket http://tracker.ceph.com/issues

Re: [ceph-users] Is the 12.2.1 really stable? Anybody have production cluster with Luminous Bluestore?

2017-11-16 Thread Ashley Merrick
Hello Vasu, Sorry I linked to the wrong bug, the one that's causing me large issues is : http://tracker.ceph.com/issues/22144 ,Ashley -Original Message- From: Vasu Kulkarni [mailto:vakul...@redhat.com] Sent: 17 November 2017 03:36 To: Ashley Merrick <ash...@amerrick.co.uk> Cc: J

Re: [ceph-users] OSD Random Failures - Latest Luminous

2017-11-16 Thread Ashley Merrick
Hello, Good to hear it's not just me, however have a cluster basically offline due to too many OSD's dropping for this issue. Anybody have any suggestions? ,Ashley From: Eric Nelson <ericnel...@gmail.com> Sent: 16 November 2017 00:06:14 To: Ashley M

[ceph-users] EC Metadata Pool Storage

2018-10-31 Thread Ashley Merrick
Hello, I have a small EC Pool I am using with RBD to store a bunch of large files attached to some VM's for personal storage use. Currently I have the EC Meta Data Pool on some SSD's, I have noticed even though the EC Pool has TB's of data in the metadata pool is only in the 2MB Range. My

[ceph-users] EC K + M Size

2018-11-03 Thread Ashley Merrick
Hello, Tried to do some reading online but was unable to find much. I can imagine a higher K + M size with EC requires more CPU to re-compile the shards into the required object. But is there any benefit or negative going with a larger K + M, obviously their is the size benefit but technically

Re: [ceph-users] mount rbd read only

2018-11-08 Thread Ashley Merrick
What command are you using to mount the /dev/rbd0 to start with? You seem to have missed that on your copy and paste. On Thu, Nov 8, 2018 at 8:06 PM ST Wong (ITSC) wrote: > Hi, > > > > We created a testing rbd block device image as following: > > > > - cut here --- > > # rbd create

[ceph-users] Automated Deep Scrub always inconsistent

2018-11-08 Thread Ashley Merrick
Have in the past few days noticed that every single automated deep scrub comes back as inconsistent, once I run a manual deep-scrub it finishes fine and the PG is marked as clean. I am running the latest mimic but have noticed someone else under luminous is facing the same issue :

Re: [ceph-users] Disabling write cache on SATA HDDs reduces write latency 7 times

2018-11-10 Thread Ashley Merrick
I've just worked out I had the same issue, been trying to work out the cause for the past few days! However I am using brand new enterprise Toshiba drivers with 256MB write cache, was seeing I/O wait peaks of 40% even during a small writing operation to CEPH and commit / apply latency's in the

Re: [ceph-users] Disabling write cache on SATA HDDs reduces write latency 7 times

2018-11-11 Thread Ashley Merrick
is on hdd cluster only? > > > -Original Message----- > From: Ashley Merrick [mailto:singap...@amerrick.co.uk] > Sent: zondag 11 november 2018 6:24 > To: vita...@yourcmc.ru > Cc: ceph-users@lists.ceph.com > Subject: Re: [ceph-users] Disabling write cache on SATA HDDs reduces >

Re: [ceph-users] Disabling write cache on SATA HDDs reduces write latency 7 times

2018-11-11 Thread Ashley Merrick
Either more weird then, what drives is in the other cluster? On Sun, 11 Nov 2018 at 7:19 PM, Vitaliy Filippov wrote: > It seems no, I've just tested it on another small cluster with HDDs only > - > no change > > > Does it make sense to test disabling this on hdd cluster only? > > -- > With best

Re: [ceph-users] Disabling write cache on SATA HDDs reduces write latency 7 times

2018-11-11 Thread Ashley Merrick
brand / firmware? On Sun, Nov 11, 2018 at 8:54 PM Marc Roos wrote: > > WD Red here > > > > > > > -Original Message- > From: Ashley Merrick [mailto:singap...@amerrick.co.uk] > Sent: zondag 11 november 2018 13:47 > To: Vitaliy Filippov > Cc: Marc Roos

Re: [ceph-users] How to repair active+clean+inconsistent?

2018-11-11 Thread Ashley Merrick
Your need to run "ceph pg deep-scrub 1.65" first On Mon, Nov 12, 2018 at 2:20 PM K.C. Wong wrote: > Hi Brad, > > I got the following: > > [root@mgmt01 ~]# ceph health detail > HEALTH_ERR 1 pgs inconsistent; 1 scrub errors > pg 1.65 is active+clean+inconsistent, acting [62,67,47] > 1 scrub

Re: [ceph-users] Disabling write cache on SATA HDDs reduces write latency 7 times

2018-11-13 Thread Ashley Merrick
Looks like it as the Toshiba drives I use have their own version of that it seems. So would explain the same kind of results. On Tue, 13 Nov 2018 at 4:26 PM, Виталий Филиппов wrote: > This may be the explanation: > > >

Re: [ceph-users] Automated Deep Scrub always inconsistent

2018-11-12 Thread Ashley Merrick
the failed read) Thanks! On Mon, 12 Nov 2018 at 11:56 PM, Jonas Jelten wrote: > Maybe you are hitting the kernel bug worked around by > https://github.com/ceph/ceph/pull/23273 > > -- Jonas > > > On 12/11/2018 16.39, Ashley Merrick wrote: > > Is anyone else seeing this

Re: [ceph-users] Ceph or Gluster for implementing big NAS

2018-11-12 Thread Ashley Merrick
My 2 cents would be depends how H/A you need. Going with the monster VM you have a single point of failure and a single point of network congestion. If you go the CephFS route you remove that single point of failure if you mount to clients directly. And also can remove that single point of

  1   2   >