Hello,
On Wed, 20 Aug 2014 15:39:11 +0100 Hugo Mills wrote:
We have a ceph system here, and we're seeing performance regularly
descend into unusability for periods of minutes at a time (or longer).
This appears to be triggered by writing large numbers of small files.
Specifications:
Hi Hugo,
On 20 Aug 2014, at 17:54, Hugo Mills h.r.mi...@reading.ac.uk wrote:
What are you using for OSD journals?
On each machine, the three OSD journals live on the same ext4
filesystem on an SSD, which is also the root filesystem of the
machine.
Also check the CPU usage for the mons
Just to fill in some of the gaps from yesterday's mail:
On Wed, Aug 20, 2014 at 04:54:28PM +0100, Hugo Mills wrote:
Some questions below I can't answer immediately, but I'll spend
tomorrow morning irritating people by triggering these events (I think
I have a reproducer -- unpacking a
On Thu, Aug 21, 2014 at 07:40:45AM +, Dan Van Der Ster wrote:
On 20 Aug 2014, at 17:54, Hugo Mills h.r.mi...@reading.ac.uk wrote:
Does your hardware provide enough IOPS for what your users need?
(e.g. what is the op/s from ceph -w)
Not really an answer to your question, but: Before
Hi Hugo,
On 21 Aug 2014, at 14:17, Hugo Mills h.r.mi...@reading.ac.uk wrote:
Not sure what you mean about colocated journal/OSD. The journals
aren't on the same device as the OSDs. However, all three journals on
each machine are on the same SSD.
embarrassed I obviously didn’t drink
Hi,
Do you get slow requests during the slowness incidents? What about monitor
elections?
Are your MDSs using a lot of CPU? did you try tuning anything in the MDS (I
think the default config is still conservative, and there are options to cache
more entries, etc…)
What about iostat on the OSDs
Hi,
On 20 Aug 2014, at 16:55, German Anders
gand...@despegar.commailto:gand...@despegar.com wrote:
Hi Dan,
How are you? I want to know how you disable the indexing on the
/var/lib/ceph OSDs?
# grep ceph /etc/updatedb.conf
PRUNEPATHS = /afs /media /net /sfs /tmp /udev /var/cache/ccache
Hi, Dan,
Some questions below I can't answer immediately, but I'll spend
tomorrow morning irritating people by triggering these events (I think
I have a reproducer -- unpacking a 1.2 GiB tarball with 25 small
files in it) and giving you more details. For the ones I can answer
right now:
...@reading.ac.uk
To: Dan Van Der Ster daniel.vanders...@cern.ch
Cc: Ceph Users List ceph-users@lists.ceph.com
Sent: Wednesday, 20 August, 2014 4:54:28 PM
Subject: Re: [ceph-users] Serious performance problems with small file writes
Hi, Dan,
Some questions below I can't answer immediately
