Re: [ceph-users] ceph at Universite de Lorraine

grasp on what you'll have to ask an integrator if you really need one. Best regards, Lionel Bouton ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph at Universite de Lorraine

grasp on what you'll have to ask an integrator if you really need one. Best regards, Lionel Bouton ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Ceph OSD very slow startup

it is harmless (at least these OSD don't show any other error/warning and have been restarted and their filesystem remounted on numerous occasions), but I'd like to be sure: is it? Best regards, Lionel Bouton ___ ceph-users mailing list ceph-users@lists.ceph.com

Re: [ceph-users] Ceph OSD very slow startup

Le 14/10/2014 01:28, Lionel Bouton a écrit : Hi, # First a short description of our Ceph setup You can skip to the next section (Main questions) to save time and come back to this one if you need more context. Missing important piece of information: this is Ceph 0.80.5 (guessable as I

Re: [ceph-users] Ceph OSD very slow startup

Le 14/10/2014 18:17, Gregory Farnum a écrit : On Monday, October 13, 2014, Lionel Bouton lionel+c...@bouton.name mailto:lionel%2bc...@bouton.name wrote: [...] What could explain such long startup times? Is the OSD init doing a lot of random disk accesses? Is it dependant

Re: [ceph-users] Ceph OSD very slow startup

Le 14/10/2014 18:51, Lionel Bouton a écrit : Le 14/10/2014 18:17, Gregory Farnum a écrit : On Monday, October 13, 2014, Lionel Bouton lionel+c...@bouton.name mailto:lionel%2bc...@bouton.name wrote: [...] What could explain such long startup times? Is the OSD init doing a lot

Re: [ceph-users] Ceph OSD very slow startup

Hi, More information on our Btrfs tests. Le 14/10/2014 19:53, Lionel Bouton a écrit : Current plan: wait at least a week to study 3.17.0 behavior and upgrade the 3.12.21 nodes to 3.17.0 if all goes well. 3.17.0 and 3.17.1 have a bug which remounts Btrfs filesystems read-only (no corruption

Re: [ceph-users] why the erasure code pool not support random write?

Le 20/10/2014 16:39, Wido den Hollander a écrit : On 10/20/2014 03:25 PM, 池信泽 wrote: hi, cephers: When I look into the ceph source code, I found the erasure code pool not support the random write, it only support the append write. Why? Is that random write of is erasure code high cost

Re: [ceph-users] why the erasure code pool not support random write?

for repair). If you are using Btrfs it will report an I/O error because it uses an internal checksum by default which will force Ceph to use other OSDs for repair. I'd be glad to be proven wrong on this subject though. Best regards, Lionel Bouton ___ ceph

[ceph-users] Question/idea about performance problems with a few overloaded OSDs

to avoid ping-pong situations where read requests overload OSDs before overloading another and coming round again. Any thought? Is it based on wrong assumptions? Would it prove to be a can of worms if someone tried to implement it? Best regards, Lionel Bouton

Re: [ceph-users] Question/idea about performance problems with a few overloaded OSDs

Hi Gregory, Le 21/10/2014 19:39, Gregory Farnum a écrit : On Tue, Oct 21, 2014 at 10:15 AM, Lionel Bouton lionel+c...@bouton.name wrote: [...] Any thought? Is it based on wrong assumptions? Would it prove to be a can of worms if someone tried to implement it? Yeah, there's one big thing

Re: [ceph-users] Problems with pgs incomplete

to be incomplete given your ceph osd tree output) but reducing min_size to 1 should be harmless and should unfreeze the recovering process. Best regards, Lionel Bouton ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph

Re: [ceph-users] Problems with pgs incomplete

Le 01/12/2014 17:08, Lionel Bouton a écrit : I may be wrong here (I'm surprised you only have 4 incomplete pgs, I'd expect ~1/3rd of your pgs to be incomplete given your ceph osd tree output) but reducing min_size to 1 should be harmless and should unfreeze the recovering process. Ignore

Re: [ceph-users] How to tell a VM to write more local ceph nodes than to the network.

actually perform worse than your current setup. Best regards, Lionel Bouton ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] btrfs backend with autodefrag mount option

. It might be possible to use snap and defrag, BTRFS was quite stable for us (but all our OSDs are on systems with at least 72GB RAM which have enough CPU power so memory wasn't much of an issue). Best regards, Lionel Bouton ___ ceph-users mailing list

Re: [ceph-users] OSDs with btrfs are down

gains? Additional Ceph features?). Best regards, Lionel Bouton ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Is ceph production ready? [was: Ceph PG Incomplete = Cluster unusable]

On 12/30/14 16:36, Nico Schottelius wrote: Good evening, we also tried to rescue data *from* our old / broken pool by map'ing the rbd devices, mounting them on a host and rsync'ing away as much as possible. However, after some time rsync got completly stuck and eventually the host which

Re: [ceph-users] OSDs with btrfs are down

On 01/06/15 02:36, Gregory Farnum wrote: [...] filestore btrfs snap controls whether to use btrfs snapshots to keep the journal and backing store in check. WIth that option disabled it handles things in basically the same way we do with xfs. filestore btrfs clone range I believe controls how

Re: [ceph-users] OSDs with btrfs are down

and report (may take some months before we create new OSDs though). Best regards, Lionel Bouton ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Ceph User Teething Problems

On 03/04/15 22:50, Travis Rhoden wrote: [...] Thanks for this feedback. I share a lot of your sentiments, especially that it is good to understand as much of the system as you can. Everyone's skill level and use-case is different, and ceph-deploy is targeted more towards PoC use-cases. It

Re: [ceph-users] Slow performance during recovery operations

On 04/02/15 21:02, Stillwell, Bryan wrote: With these settings and no deep-scrubs the load increased a bit in the VMs doing non negligible I/Os but this was manageable. Even disk thread ioprio settings (which is what you want to get the ionice behaviour for deep scrubs) didn't seem to make

Re: [ceph-users] Slow performance during recovery operations

On 04/02/15 21:02, Stillwell, Bryan wrote: I'm pretty sure setting 'nodeep-scrub' doesn't cancel any current deep-scrubs that are happening, Indeed it doesn't. but something like this would help prevent the problem from getting worse. If the cause of the recoveries/backfills are an OSD

Re: [ceph-users] long blocking with writes on rbds

On 04/22/15 19:50, Lionel Bouton wrote: On 04/22/15 17:57, Jeff Epstein wrote: On 04/10/2015 10:10 AM, Lionel Bouton wrote: On 04/10/15 15:41, Jeff Epstein wrote: [...] This seems highly unlikely. We get very good performance without ceph. Requisitioning and manupulating block devices

Re: [ceph-users] long blocking with writes on rbds

On 04/22/15 17:57, Jeff Epstein wrote: On 04/10/2015 10:10 AM, Lionel Bouton wrote: On 04/10/15 15:41, Jeff Epstein wrote: [...] This seems highly unlikely. We get very good performance without ceph. Requisitioning and manupulating block devices through LVM happens instantaneously. We

Re: [ceph-users] Cost- and Powerefficient OSD-Nodes

be *very* surprised if the drive would get any performance or life expectancy benefit when used at 19.2% instead of 32% or even 75%... Best regards, Lionel Bouton ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph

[ceph-users] Experience going through rebalancing with active VMs / questions

Hi, we are currently running the latest firefly (0.80.9) and we have difficulties maintaining good throughput when Ceph is backfilling/recovering and/or deep-scrubing after an outage. This got to the point where when the VM using rbd start misbehaving (load rising, some simple SQL update queries

[ceph-users] Btrfs defragmentation

Hi, we began testing one Btrfs OSD volume last week and for this first test we disabled autodefrag and began to launch manual btrfs fi defrag. During the tests, I monitored the number of extents of the journal (10GB) and it went through the roof (it currently sits at 8000+ extents for example).

Re: [ceph-users] Btrfs defragmentation

On 05/04/15 01:34, Sage Weil wrote: On Mon, 4 May 2015, Lionel Bouton wrote: Hi, we began testing one Btrfs OSD volume last week and for this first test we disabled autodefrag and began to launch manual btrfs fi defrag. During the tests, I monitored the number of extents of the journal (10GB

Re: [ceph-users] Btrfs defragmentation

On 05/06/15 19:51, Lionel Bouton wrote: During normal operation Btrfs OSD volumes continue to behave in the same way XFS ones do on the same system (sometimes faster/sometimes slower). What is really slow though it the OSD process startup. I've yet to make serious tests (umounting

Re: [ceph-users] Btrfs defragmentation

Hi, On 05/06/15 20:07, Timofey Titovets wrote: 2015-05-06 20:51 GMT+03:00 Lionel Bouton lionel+c...@bouton.name: Is there something that would explain why initially Btrfs creates the 4MB files with 128k extents (32 extents / file) ? Is it a bad thing for performance ? This kind of behaviour

Re: [ceph-users] Btrfs defragmentation

On 05/05/15 02:24, Lionel Bouton wrote: On 05/04/15 01:34, Sage Weil wrote: On Mon, 4 May 2015, Lionel Bouton wrote: Hi, we began testing one Btrfs OSD volume last week and for this first test we disabled autodefrag and began to launch manual btrfs fi defrag. [...] Cool.. let us know how

Re: [ceph-users] Btrfs defragmentation

On 05/05/15 06:30, Timofey Titovets wrote: Hi list, Excuse me, what I'm saying is off topic @Lionel, if you use btrfs, did you already try to use btrfs compression for OSD? If yes, сan you share the your experience? Btrfs compresses by default using zlib. We force lzo compression instead

Re: [ceph-users] Btrfs defragmentation

On 05/04/15 01:34, Sage Weil wrote: On Mon, 4 May 2015, Lionel Bouton wrote: Hi, we began testing one Btrfs OSD volume last week and for this first test we disabled autodefrag and began to launch manual btrfs fi defrag. [...] Cool.. let us know how things look after it ages! We had

Re: [ceph-users] Btrfs defragmentation

Hi, On 05/07/15 12:30, Burkhard Linke wrote: [...] Part of the OSD boot up process is also the handling of existing snapshots and journal replay. I've also had several btrfs based OSDs that took up to 20-30 minutes to start, especially after a crash. During journal replay the OSD daemon

Re: [ceph-users] Btrfs defragmentation

On 05/06/15 20:28, Lionel Bouton wrote: Hi, On 05/06/15 20:07, Timofey Titovets wrote: 2015-05-06 20:51 GMT+03:00 Lionel Bouton lionel+c...@bouton.name: Is there something that would explain why initially Btrfs creates the 4MB files with 128k extents (32 extents / file) ? Is it a bad thing

Re: [ceph-users] advantages of multiple pools?

On 04/17/15 16:01, Saverio Proto wrote: For example you can assign different read/write permissions and different keyrings to different pools. From memory you can set different replication settings, use a cache pool or not, use specific crush map rules too. Lionel Bouton

Re: [ceph-users] Slow performance during recovery operations

Hi, On 04/06/15 02:26, Francois Lafont wrote: Hi, Lionel Bouton wrote : Sorry this wasn't clear: I tried the ioprio settings before disabling the deep scrubs and it didn't seem to make a difference when deep scrubs occured. I have never tested these parameters

Re: [ceph-users] long blocking with writes on rbds

and unnecessarily increase the load on your OSDs. Best regards, Lionel Bouton ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Slow performance during recovery operations

it when they disappear. Best regards, Lionel Bouton ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Unexpected disk write activity with btrfs OSDs

On 06/22/15 17:21, Erik Logtenberg wrote: I have the journals on a separate disk too. How do you disable the snapshotting on the OSD? http://ceph.com/docs/master/rados/configuration/filestore-config-ref/ : filestore btrfs snap = false ___ ceph-users

Re: [ceph-users] Unexpected disk write activity with btrfs OSDs

On 06/19/15 13:23, Erik Logtenberg wrote: I believe this may be the same issue I reported some time ago, which is as of yet unsolved. https://www.mail-archive.com/ceph-users@lists.ceph.com/msg19770.html I used strace to figure out that the OSD's were doing an incredible amount of getxattr,

Re: [ceph-users] Unexpected disk write activity with btrfs OSDs

On 06/22/15 11:27, Jan Schermer wrote: I don’t run Ceph on btrfs, but isn’t this related to the btrfs snapshotting feature ceph uses to ensure a consistent journal? It's possible: if I understand correctly the code, the btrfs filestore backend creates a snapshot when syncing the journal. I'm a

Re: [ceph-users] Unexpected issues with simulated 'rack' outage

On 06/24/15 14:44, Romero Junior wrote: Hi, We are setting up a test environment using Ceph as the main storage solution for my QEMU-KVM virtualization platform, and everything works fine except for the following: When I simulate a failure by powering off the switches on one of our

Re: [ceph-users] Fwd: Re: Unexpected disk write activity with btrfs OSDs

...@computational.bio.uni-giessen.de To: Lionel Bouton lionel+c...@bouton.name Hi, On 06/18/2015 11:28 PM, Lionel Bouton wrote: Hi, *snipsnap* - Disks with btrfs OSD have a spike of activity every 30s (2 intervals of 10s with nearly 0 activity, one interval with a total amount of writes of ~120MB

[ceph-users] Unexpected disk write activity with btrfs OSDs

Hi, I've just noticed an odd behaviour with the btrfs OSDs. We monitor the amount of disk writes on each device, our granularity is 10s (every 10s the monitoring system collects the total amount of sector written and write io performed since boot and computes both the B/s and IO/s). With only

Re: [ceph-users] Unexpected disk write activity with btrfs OSDs

btrfs. On 06/18/15 23:28, Lionel Bouton wrote: Hi, I've just noticed an odd behaviour with the btrfs OSDs. We monitor the amount of disk writes on each device, our granularity is 10s (every 10s the monitoring system collects the total amount of sector written and write io performed since boot

Re: [ceph-users] Is Ceph right for me?

On 05/20/15 23:34, Trevor Robinson - Key4ce wrote: Hello, Could somebody please advise me if Ceph is suitable for our use? We are looking for a file system which is able to work over different locations which are connected by VPN. If one locations was to go offline then the

Re: [ceph-users] Discuss: New default recovery config settings

On 06/01/15 09:43, Jan Schermer wrote: We had to disable deep scrub or the cluster would me unusable - we need to turn it back on sooner or later, though. With minimal scrubbing and recovery settings, everything is mostly good. Turned out many issues we had were due to too few PGs - once we

Re: [ceph-users] any recommendation of using EnhanceIO?

On 07/02/15 13:49, German Anders wrote: output from iostat: CEPHOSD01: Device: rrqm/s wrqm/s r/s w/srMB/swMB/s avgrq-sz avgqu-sz await r_await w_await svctm %util sdc(ceph-0) 0.00 0.001.00 389.00 0.0035.98 188.9660.32 120.12

Re: [ceph-users] Performance and CPU load on HP servers running ceph (DL380 G6, should apply to others too)

On 05/26/15 10:06, Jan Schermer wrote: Turbo Boost will not hurt performance. Unless you have 100% load on all cores it will actually improve performance (vastly, in terms of bursty workloads). The issue you have could be related to CPU cores going to sleep mode. Another possibility is that

Re: [ceph-users] FW: Ceph data locality

On 07/07/15 18:20, Dmitry Meytin wrote: Exactly because of that issue I've reduced the number of Ceph replications to 2 and the number of HDFS copies is also 2 (so we're talking about 4 copies). I want (but didn't tried yet) to change Ceph replication to 1 and change HDFS back to 3. You are

Re: [ceph-users] FW: Ceph data locality

On 07/07/15 17:41, Dmitry Meytin wrote: Hi Lionel, Thanks for the answer. The missing info: 1) Ceph 0.80.9 Firefly 2) map-reduce makes sequential reads of blocks of 64MB (or 128 MB) 3) HDFS which is running on top of Ceph is replicating data for 3 times between VMs which could be located

Re: [ceph-users] Ceph for multi-site operation

Le 24/08/2015 15:11, Julien Escario a écrit : Hello, First, let me advise I'm really a noob with Cephsince I have only read some documentation. I'm now trying to deploy a Ceph cluster for testing purposes. The cluster is based on 3 (more if necessary) hypervisors running proxmox 3.4.

Re: [ceph-users] EXT4 for Production and Journal Question?

Le 24/08/2015 19:34, Robert LeBlanc a écrit : Building off a discussion earlier this month [1], how supported is EXT4 for OSDs? It seems that some people are getting good results with it and I'll be testing it in our environment. The other question is if the EXT4 journal is even necessary if

Re: [ceph-users] btrfs w/ centos 7.1

Le 07/08/2015 22:05, Ben Hines a écrit : Howdy, The Ceph docs still say btrfs is 'experimental' in one section, but say it's the long term ideal for ceph in the later section. Is this still accurate with Hammer? Is it mature enough on centos 7.1 for production use? (kernel is

Re: [ceph-users] CephFS vs RBD

Le 22/07/2015 21:17, Lincoln Bryant a écrit : Hi Hadi, AFAIK, you can’t safely mount RBD as R/W on multiple machines. You could re-export the RBD as NFS, but that’ll introduce a bottleneck and probably tank your performance gains over CephFS. For what it’s worth, some of our RBDs are mapped

Re: [ceph-users] how to recover from: 1 pgs down; 10 pgs incomplete; 10 pgs stuck inactive; 10 pgs stuck unclean

Le 15/07/2015 10:55, Jelle de Jong a écrit : On 13/07/15 15:40, Jelle de Jong wrote: I was testing a ceph cluster with osd_pool_default_size = 2 and while rebuilding the OSD on one ceph node a disk in an other node started getting read errors and ceph kept taking the OSD down, and instead of

Re: [ceph-users] Issue with journal on another drive

On 07/14/15 00:08, Rimma Iontel wrote: Hi all, [...] Is there something that needed to be done to journal partition to enable sharing between multiple OSDs? Or is there something else that's causing the isssue? IIRC you can't share a volume between multiple OSDs. What you could do if

Re: [ceph-users] O_DIRECT on deep-scrub read

Le 07/10/2015 13:44, Paweł Sadowski a écrit : > Hi, > > Can anyone tell if deep scrub is done using O_DIRECT flag or not? I'm > not able to verify that in source code. > > If not would it be possible to add such feature (maybe config option) to > help keeping Linux page cache in better shape?

Re: [ceph-users] FW: Ceph data locality

Hi Dmitry, On 07/07/15 14:42, Dmitry Meytin wrote: Hi Christian, Thanks for the thorough explanation. My case is Elastic Map Reduce on top of OpenStack with Ceph backend for everything (block, object, images). With default configuration, performance is 300% worse than bare metal. I did a

Re: [ceph-users] How to prefer faster disks in same pool

On 07/10/15 02:13, Christoph Adomeit wrote: Hi Guys, I have a ceph pool that is mixed with 10k rpm disks and 7.2 k rpm disks. There are 85 osds and 10 of them are 10k Size is not an issue, the pool is filled only 20% I want to somehow prefer the 10 k rpm disks so that they get more i/o

Re: [ceph-users] Real world benefit from SSD Journals for a more read than write cluster

On 07/12/15 05:55, Alex Gorbachev wrote: FWIW. Based on the excellent research by Mark Nelson (http://ceph.com/community/ceph-performance-part-2-write-throughput-without-ssd-journals/) we have dropped SSD journals altogether, and instead went for the battery protected controller writeback

Re: [ceph-users] Corruption of file systems on RBD images

Le 02/09/2015 18:16, Mathieu GAUTHIER-LAFAYE a écrit : > Hi Lionel, > > - Original Message - >> From: "Lionel Bouton" <lionel+c...@bouton.name> >> To: "Mathieu GAUTHIER-LAFAYE" <mathieu.gauthier-laf...@lapth.cnrs.fr>, >> ceph-us..

Re: [ceph-users] Corruption of file systems on RBD images

Hi Mathieu, Le 02/09/2015 14:10, Mathieu GAUTHIER-LAFAYE a écrit : > Hi All, > > We have some troubles regularly with virtual machines using RBD storage. When > we restart some virtual machines, they starts to do some filesystem checks. > Sometime it can rescue it, sometime the virtual machine

Re: [ceph-users] Hammer reduce recovery impact

Le 10/09/2015 22:56, Robert LeBlanc a écrit : > We are trying to add some additional OSDs to our cluster, but the > impact of the backfilling has been very disruptive to client I/O and > we have been trying to figure out how to reduce the impact. We have > seen some client I/O blocked for more

Re: [ceph-users] Hammer reduce recovery impact

Le 11/09/2015 00:20, Robert LeBlanc a écrit : > I don't think the script will help our situation as it is just setting > osd_max_backfill from 1 to 0. It looks like that change doesn't go > into effect until after it finishes the PG. That was what I was afraid of. Note that it should help a

Re: [ceph-users] Hammer reduce recovery impact

Le 11/09/2015 01:24, Lincoln Bryant a écrit : > On 9/10/2015 5:39 PM, Lionel Bouton wrote: >> For example deep-scrubs were a problem on our installation when at >> times there were several going on. We implemented a scheduler that >> enforces limits on simultaneous deep-scru

Re: [ceph-users] question on reusing OSD

Le 16/09/2015 01:21, John-Paul Robinson a écrit : > Hi, > > I'm working to correct a partitioning error from when our cluster was > first installed (ceph 0.56.4, ubuntu 12.04). This left us with 2TB > partitions for our OSDs, instead of the 2.8TB actually available on > disk, a 29% space hit.

Re: [ceph-users] Issue with journal on another drive

Hi, Le 29/09/2015 13:32, Jiri Kanicky a écrit : > Hi Lionel. > > Thank you for your reply. In this case I am considering to create > separate partitions for each disk on the SSD drive. Would be good to > know what is the performance difference, because creating partitions > is kind of waste of

Re: [ceph-users] Simultaneous CEPH OSD crashes

Le 27/09/2015 10:25, Lionel Bouton a écrit : > Le 27/09/2015 09:15, Lionel Bouton a écrit : >> Hi, >> >> we just had a quasi simultaneous crash on two different OSD which >> blocked our VMs (min_size = 2, size = 3) on Firefly 0.80.9. >> >> the first OSD to go

Re: [ceph-users] Simultaneous CEPH OSD crashes

Hi, Le 29/09/2015 19:06, Samuel Just a écrit : > It's an EIO. The osd got an EIO from the underlying fs. That's what > causes those asserts. You probably want to redirect to the relevant > fs maling list. Thanks. I didn't get any answer on this from BTRFS developers yet. The problem seems

Re: [ceph-users] Predict performance

Hi, Le 02/10/2015 18:15, Christian Balzer a écrit : > Hello, > On Fri, 2 Oct 2015 15:31:11 +0200 Javier C.A. wrote: > > Firstly, this has been discussed countless times here. > For one of the latest recurrences, check the archive for: > > "calculating maximum number of disk and node failure that

Re: [ceph-users] Simultaneous CEPH OSD crashes

Le 27/09/2015 09:15, Lionel Bouton a écrit : > Hi, > > we just had a quasi simultaneous crash on two different OSD which > blocked our VMs (min_size = 2, size = 3) on Firefly 0.80.9. > > the first OSD to go down had this error : > > 2015-09-27 06:30:33.257133 7f7ac7f

[ceph-users] Simultaneous CEPH OSD crashes

and the recent events) if needed. Can anyone put some light on why these OSDs died ? Best regards, Lionel Bouton ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] backfilling on a single OSD and caching controllers

Hi, just a tip I just validated on our hardware. I'm currently converting an OSD from xfs with journal on same platter to btrfs with journal on SSD. To avoid any unwanted movement, I reused the same OSD number, weight and placement : so Ceph is simply backfilling all PGs previously stored on the

Re: [ceph-users] SSD only pool without journal

Hi, Le 17/12/2015 16:47, Misa a écrit : > Hello everyone, > > does it make sense to create SSD only pool from OSDs without journal? No, because AFAIK you can't have OSDs without journals yet. IIRC there is work done for alternate stores where you wouldn't need journals anymore but it's not yet

Re: [ceph-users] Intel S3710 400GB and Samsung PM863 480GB fio results

Le 23/12/2015 16:18, Mart van Santen a écrit : > Hi all, > > > On 12/22/2015 01:55 PM, Wido den Hollander wrote: >> On 22-12-15 13:43, Andrei Mikhailovsky wrote: >>> Hello guys, >>> >>> Was wondering if anyone has done testing on Samsung PM863 120 GB version to >>> see how it performs? IMHO the

Re: [ceph-users] Intel S3710 400GB and Samsung PM863 480GB fio results

Le 23/12/2015 18:37, Mart van Santen a écrit : > So, maybe you are right and is the HBA the bottleneck (LSI Logic / > Symbios Logic MegaRAID SAS 2108). Under all cirumstances, I do not get > close to the numbers of the PM863 quoted by Sebastien. But his site > does not state what kind of HBA he is

Re: [ceph-users] Intel S3710 400GB and Samsung PM863 480GB fio results

Le 22/12/2015 13:43, Andrei Mikhailovsky a écrit : > Hello guys, > > Was wondering if anyone has done testing on Samsung PM863 120 GB version to > see how it performs? IMHO the 480GB version seems like a waste for the > journal as you only need to have a small disk size to fit 3-4 osd journals.

Re: [ceph-users] CEPH over SW-RAID

Le 23/11/2015 19:58, Jose Tavares a écrit : > > > On Mon, Nov 23, 2015 at 4:15 PM, Lionel Bouton > <lionel-subscript...@bouton.name > <mailto:lionel-subscript...@bouton.name>> wrote: > > Hi, > > Le 23/11/2015 18:37, Jose Tavares a écrit : > >

Re: [ceph-users] CEPH over SW-RAID

Hi, Le 23/11/2015 18:37, Jose Tavares a écrit : > Yes, but with SW-RAID, when we have a block that was read and does not match > its checksum, the device falls out of the array I don't think so. Under normal circumstances a device only falls out of a md array if it doesn't answer IO queries

Re: [ceph-users] CEPH over SW-RAID

Le 23/11/2015 21:01, Jose Tavares a écrit : > > > > > My new question regarding Ceph is if it isolates this bad sectors where > it found bad data when scrubbing? or there will be always a replica of > something over a known bad block..? > Ceph OSDs don't know about bad sectors, they

Re: [ceph-users] Global, Synchronous Blocked Requests

Hi, Le 28/11/2015 04:24, Brian Felton a écrit : > Greetings Ceph Community, > > We are running a Hammer cluster (0.94.3-1) in production that recently > experienced asymptotic performance degradation. We've been migrating > data from an older non-Ceph cluster at a fairly steady pace for the >

Re: [ceph-users] Scrubbing question

Le 26/11/2015 15:53, Tomasz Kuzemko a écrit : > ECC will not be able to recover the data, but it will always be able to > detect that data is corrupted. No. That's a theoretical impossibility as the detection is done by some kind of hash over the memory content which brings the possibility of

Re: [ceph-users] CEPH over SW-RAID

Le 23/11/2015 18:17, Jan Schermer a écrit : > SW-RAID doesn't help with bit-rot if that's what you're afraid of. > If you are afraid bit-rot you need to use a fully checksumming filesystem > like ZFS. > Ceph doesn't help there either when using replicas - not sure how strong > error

Re: [ceph-users] CEPH over SW-RAID

Le 23/11/2015 21:58, Jose Tavares a écrit : > > AFAIK, people are complaining about lots os bad blocks in the new big > disks. The hardware list seems to be small and unable to replace > theses blocks. Note that if by big disks you mean SMR-based disks, they can exhibit what looks like bad blocks

Re: [ceph-users] Ceph cache tier and rbd volumes/SSD primary, HDD replica crush rule!

Le 12/01/2016 18:27, Mihai Gheorghe a écrit : > One more question. Seeing that cache tier holds data on it untill it > reaches % ratio, i suppose i must set replication to 2 or higher on > the cache pool to not lose hot data not writen to the cold storage in > case of a drive failure, right? > >

Re: [ceph-users] Pinpointing performance bottleneck / would SSD journals help?

Le 27/06/2016 17:42, Daniel Schneller a écrit : > Hi! > > We are currently trying to pinpoint a bottleneck and are somewhat stuck. > > First things first, this is the hardware setup: > > 4x DELL PowerEdge R510, 12x4TB OSD HDDs, journal colocated on HDD > 96GB RAM, 2x6 Cores + HT > 2x1GbE bonded

Re: [ceph-users] pg scrub and auto repair in hammer

Hi, Le 28/06/2016 08:34, Stefan Priebe - Profihost AG a écrit : > [...] > Yes but at least BTRFS is still not working for ceph due to > fragmentation. I've even tested a 4.6 kernel a few weeks ago. But it > doubles it's I/O after a few days. BTRFS autodefrag is not working over the long term.

Re: [ceph-users] Increasing time to save RGW objects

Le 09/02/2016 20:07, Kris Jurka a écrit : > > > On 2/9/2016 10:11 AM, Lionel Bouton wrote: > >> Actually if I understand correctly how PG splitting works the next spike >> should be times smaller and spread over times the period (where >> is the number of subd

Re: [ceph-users] Increasing time to save RGW objects

Le 09/02/2016 20:18, Lionel Bouton a écrit : > Le 09/02/2016 20:07, Kris Jurka a écrit : >> >> On 2/9/2016 10:11 AM, Lionel Bouton wrote: >> >>> Actually if I understand correctly how PG splitting works the next spike >>> should be times smaller a

Re: [ceph-users] K is for Kraken

Le 08/02/2016 20:09, Robert LeBlanc a écrit : > Too bad K isn't an LTS. It was be fun to release the Kraken many times. Kraken is an awesome release name ! How I will miss being able to say/write to our clients that we just released the Kraken on their infra :-/ Lionel

Re: [ceph-users] Increasing time to save RGW objects

Hi, Le 09/02/2016 17:07, Kris Jurka a écrit : > > > On 2/8/2016 9:16 AM, Gregory Farnum wrote: >> On Mon, Feb 8, 2016 at 8:49 AM, Kris Jurka wrote: >>> >>> I've been testing the performance of ceph by storing objects through >>> RGW. >>> This is on Debian with Hammer using 40

Re: [ceph-users] Increasing time to save RGW objects

Le 09/02/2016 19:11, Lionel Bouton a écrit : > Actually if I understand correctly how PG splitting works the next spike > should be times smaller and spread over times the period (where > is the number of subdirectories created during each split which > seems to be 15 typo : 16

Re: [ceph-users] Reducing the impact of OSD restarts (noout ain't uptosnuff)

Le 13/02/2016 06:31, Christian Balzer a écrit : > [...] > --- > So from shutdown to startup about 2 seconds, not that bad. > However here is where the cookie crumbles massively: > --- > 2016-02-12 01:33:50.263152 7f75be4d57c0 0 filestore(/var/lib/ceph/osd/ceph-2) limited size xattrs > 2016-02-12

Re: [ceph-users] Reducing the impact of OSD restarts (noout ain't uptosnuff)

Hi, Le 13/02/2016 15:52, Christian Balzer a écrit : > [..] > > Hum that's surprisingly long. How much data (size and nb of files) do > you have on this OSD, which FS do you use, what are the mount options, > what is the hardware and the kind of access ? > > I already mentioned the HW, Areca RAID

Re: [ceph-users] SSD Journal

Le 29/01/2016 01:12, Jan Schermer a écrit : > [...] >> Second I'm not familiar with Ceph internals but OSDs must make sure that >> their PGs are synced so I was under the impression that the OSD content for >> a PG on the filesystem should always be guaranteed to be on all the other >> active

Re: [ceph-users] SSD Journal

Le 29/01/2016 16:25, Jan Schermer a écrit : > > [...] > > > But if I understand correctly, there is indeed a log of the recent > modifications in the filestore which is used when a PG is recovering > because another OSD is lagging behind (not when Ceph reports a full > backfill

Re: [ceph-users] SSD Journal

Le 28/01/2016 22:32, Jan Schermer a écrit : > P.S. I feel very strongly that this whole concept is broken > fundamentaly. We already have a journal for the filesystem which is > time proven, well behaved and above all fast. Instead there's this > reinvented wheel which supposedly does it better in

Re: [ceph-users] Help: pool not responding

Le 29/02/2016 22:50, Shinobu Kinjo a écrit : >> the fact that they are optimized for benchmarks and certainly not >> Ceph OSD usage patterns (with or without internal journal). > Are you assuming that SSHD is causing the issue? > If you could elaborate on this more, it would be helpful. Probably

  1   2   >