On 04/02/15 21:02, Stillwell, Bryan wrote:
With these settings and no deep-scrubs the load increased a bit in the
VMs doing non negligible I/Os but this was manageable. Even disk thread
ioprio settings (which is what you want to get the ionice behaviour for
deep scrubs) didn't seem to make
Geddes
If you are still struggling with this , ping me in IRC #CEPH ( ksingh )
Karan Singh
Systems Specialist , Storage Platforms
CSC - IT Center for Science,
Keilaranta 14, P. O. Box 405, FIN-02101 Espoo, Finland
mobile: +358
On 04/02/15 21:02, Stillwell, Bryan wrote:
I'm pretty sure setting 'nodeep-scrub' doesn't cancel any current
deep-scrubs that are happening,
Indeed it doesn't.
but something like this would help prevent
the problem from getting worse.
If the cause of the recoveries/backfills are an OSD
Hi,
I am building Ceph Debian Packages off of the 0.80.9 (latest firefly) and on
top of that I am applying an optimization patch.
I am following the standard instructions from the README file and effectively
running commands in this order:
$ ./autogen.sh
$
Try changing src/ceph_ver.h and repeat what you did earlier.-Krishna
On Thursday, April 2, 2015 2:17 PM, Garg, Pankaj
pankaj.g...@caviumnetworks.com wrote:
!--#yiv6999103272 _filtered #yiv6999103272 {font-family:Calibri;panose-1:2
15 5 2 2 2 4 3 2 4;}#yiv6999103272 #yiv6999103272
No, but I've seen it in RadosGW too. I've been meaning to post about it.
I get about ten a day, out of about 50k objects/day.
clewis@clewis-mac ~ (-) $ s3cmd ls s3://live32/ | grep '1970-01' | head -1
1970-01-01 00:00 0
On Thu, Apr 2, 2015 at 11:18 PM, Barclay Jameson
almightybe...@gmail.com wrote:
I am using the Giant release. The OSDs and MON/MDS are using default
RHEL 7 kernel. Client is using elrepo 3.19 kernel. I am also using
cephaux.
I reproduced this issue by using giant release. It's a bug in the MDS
On 04/01/2015 02:42 AM, Jimmy Goffaux wrote:
English Version :
Hello,
I found a strange behavior in Ceph. This behavior is visible on Buckets
(RGW) and pools (RDB).
pools:
``
root@:~# qemu-img info rbd:pool/kibana2
image: rbd:pool/kibana2
file format: raw
virtual size: 30G (32212254720 bytes)
Fortunately Ceph Giant + OpenStack Juno works flawlessly for me.
If you have configured cinder / glance correctly , then after restarting
cinder and glance services , you should see something like this in cinder and
glance logs.
Cinder logs :
volume.log:2015-04-02 13:20:43.943 2085 INFO
On Wed, Apr 1, 2015 at 12:31 AM, Barclay Jameson
almightybe...@gmail.com wrote:
Here is the mds output from the command you requested. I did this
during the small data run . ( time cp small1/* small2/ )
It is 20MB in size so I couldn't find a place online that would accept
that much data.
We have the model with 25 disks per node.
We have just two 10G network interfaces per node. Does that not limit
the thgouthput and hence the load on the CPUs?
-Sreenath
On 4/2/15, Alexandre DERUMIER aderum...@odiso.com wrote:
Hi,
with HP SL4540 server?
this model
Thanks for the tips.
Could anyone share their experience building a SSD pool or a SSD cache
tier with HP SL4540 server?
rgds,
Sreenath
On 4/2/15, Christian Balzer ch...@gol.com wrote:
Hello,
On Wed, 1 Apr 2015 18:40:10 +0530 Sreenath BH wrote:
Hi all,
we are considering building all SSD
I'm probably going to get shot down for saying this...but here goes.
As a very rough guide, think of it more as you need around 10Mhz for every IO,
whether that IO is 4k or 4MB it uses roughly the same amount of CPU, as most of
the CPU usage is around ceph data placement rather than the actual
Hello everyone,
Is there anyone having experience to try to tune Kernel RBD device by changing
scheduler and other settings?
Currently we are trying it on RHEL 7.1 bundled rbd module, to change the
following setting under /sys/block/rbdX/queue:
1) scheduler: noop vs deadline, deadline seems
It's probably more a question of IOPS unless you have really slow SSDs. :)
Mark
On 04/02/2015 07:22 AM, Sreenath BH wrote:
We have the model with 25 disks per node.
We have just two 10G network interfaces per node. Does that not limit
the thgouthput and hence the load on the CPUs?
-Sreenath
Thanks Karan/Quentin/Erik,
I admit up front that this is all new to me as my background is optical
transport rather than server/storage admin!
I'm reassured to know that it should work and this is why I'm completely
willing to believe that it's something that I'm doing wrong ... but
Hi Chris,
according to your ceph osd tree capture, although the OSD reweight is set to 1,
the OSD CRUSH weight is set to 0 (2nd column). You need to assign the OSD a
CRUSH weight so that it can be selected by CRUSH: ceph osd crush reweight
osd.30 x.y (where 1.0=1TB)
Only when this is done
performance seem quite impressive (40iops 4k) :)
for endurance, It's looking like intel s3500
I still preferer s3610 for now,
400GB for $419 , with 3.0PB endurance.
- Mail original -
De: Mark Nelson mnel...@redhat.com
À: ceph-users ceph-users@lists.ceph.com
Envoyé: Jeudi 2
I think you want to do:
$ dch
$ dpkg-buildpackage
You can muck about with what the package is gonna be called (versions,
revisions etc) from dch, without changing the src.
Cheers
Mark
On 03/04/15 10:17, Garg, Pankaj wrote:
Hi,
I am building Ceph Debian Packages off of the 0.80.9 (latest
Recovery creates I/O performance drops in our VM too but it's manageable.
What really hurts us are deep scrubs.
Our current situation is Firefly 0.80.9 with a total of 24 identical OSDs
evenly distributed on 4 servers with the following relevant configuration:
osd recovery max active = 2
On Thursday, April 2, 2015, Nick Fisk n...@fisk.me.uk wrote:
I'm probably going to get shot down for saying this...but here goes.
As a very rough guide, think of it more as you need around 10Mhz for every
IO, whether that IO is 4k or 4MB it uses roughly the same amount of CPU, as
most
I am using the Giant release. The OSDs and MON/MDS are using default
RHEL 7 kernel. Client is using elrepo 3.19 kernel. I am also using
cephaux.
I may have found something.
I did the build manually as such I did _NOT_ set up these config settings:
filestore xattr use omap = false
filestore max
- Original Message -
From: Sergey Arkhipov sarkhi...@asdco.ru
To: ceph-users@lists.ceph.com
Sent: Monday, March 30, 2015 2:55:33 AM
Subject: [ceph-users] RADOS Gateway quota management
Hi,
Currently I am trying to figure out how to work with RADOS Gateway (ceph
0.87) limits and
As expected I had a typo in my config for cinder. Correcting that got
cinder working. Everything in glance looks correct according to the above
referenced page, but I'm not seeing any mention of rbd in the logs, and I
notice that the cephx authentication pieces that are present for cinder and
The RDO glance-store package had a bug in it that miscalculated the chunk
size. I should hope that it's been patched by Redhat now since the fix was
committed upstream before the first Juno rleease, but perhaps not. The
symptom of the bug was horribly slow uploads to glance.
Run this and send
Hi, can't pretend that I have all the answers (or any of them!) but I've
also been unable to deploy a mon node that doesn't appear in the 'mon
initial members' list. However, the No such file or directory is
something that I don't remember. Did you run ceph-deploy install against
the node first?
Yes, I did.
Furthermore, looking at node2, the ceph service is not starting, giving me
this error.
# service ceph start
=== mon.ceph-node2 ===
Starting Ceph mon.ceph-node2 on ceph-node2...
2015-04-02 18:08:50.387800 7f819f2667a0 -1 no public_addr or public_network
specified, and mon.ceph-node2
Hello Ceph Community,
I have an issue whenever I am creating an object (media file) on ceph using
the s3 api. I get the error message :
Tue Mar 24 14:41:00.079071 2015] [fastcgi:error] [pid 1606:tid
139809104328448] [client 192.168.88.1:18644] FastCGI: comm with server
/var/www/s3gw.fcgi
Hi,
Currently I am trying to figure out how to work with RADOS Gateway (ceph
0.87) limits and I've managed to produce such strange behavior:
{ bucket: test1-8,
pool: .rgw.buckets,
index_pool: .rgw.buckets.index,
id: default.17497.14,
marker: default.17497.14,
owner:
Yes. I have setup ceph and hadoop in each node. ceph health is OK and the
hadoop works fine when I use HDFS (I have ran the same command with HDFS
and it works). One node is the admin(job tracker running), other 4 are
slaves(tasktracker running). The problem occurs when I change the
Hi Greg,
Thank you for your response. I have understood that I should be starting
only the mapred daemons when using cephFS instead of HDFS. I have fixed
that and trying to run hadoop wordcount job using this instruction:
bin/hadoop jar hadoop*examples*.jar wordcount /tmp/wc-input /tmp/wc-output
i first had some major disaster i had 12 incomplete pgs that couldn't be fixed.
(due to several harddisk failures at once)
alls these incomplete pgs where all in the .rgw and .rgw.buckets pools
so the only option i could think of is to take my loses and delete and recreate
those pools.
the
Hi,
I just bought the book Learning Ceph. I'm trying to go through the pages
and set up a virtual lab. I managed to set up the servers etc but when it
comes to using ceph-deploy mon create to other nodes - it fails badly. I
don't see anything that I missed on the books.
Here's the output:
Thought folks might like to see this:
http://hothardware.com/reviews/intel-ssd-750-series-nvme-pci-express-solid-state-drive-review
Quick summary:
- PCIe SSD based on the P3700
- 400GB for $389!
- 1.2GB/s writes and 2.4GB/s reads
- power loss protection
- 219TB write endurance
So basically
All,
Whenever we're doing some kind of recovery operation on our ceph
clusters (cluster expansion or dealing with a drive failure), there
seems to be a fairly noticable performance drop while it does the
backfills (last time I measured it the performance during recovery was
something like 20% of
Glance should just require something like the following under [default]
rbd_store_user=glance
rbd_store_pool=images
rbd_store_ceph_conf=/etc/ceph/ceph.conf
rbd_store_chunk_size=8
default_store=rbd
Also make sure the keyring is in /etc/ceph and you may want to explicitly
define the user and
also from my tests, writes use a lot more cpus than read.
I think they are some optimisations in hammer for this.
(and also the tcmalloc bug which seem to impact cpu)
I will redo bench soon when hammer will be released.
- Mail original -
De: Nick Fisk n...@fisk.me.uk
À: ceph-users
Thanks Erik,
Maybe this is related as I have:
[DEFAULT]
verbose = True
notification_driver = noop
default_store = rbd
show_image_direct_url = true
debug=True
[database]
connection = mysql://glance:glancepw@ps-sw-ctrl1/glance
[keystone_authtoken]
auth_uri =
Hm, even lacking the mentions of rbd in the glance docs, and the lack of
cephx auth information in the config, glance seems to be working after all.
S, hooray! It was probably working all along, I just hadn't gotten to
really testing it since I was getting blocked by my typo on the cinder
On Thu, Apr 2, 2015 at 12:18 PM, Quentin Hartman
qhart...@direwolfdigital.com wrote:
Hm, even lacking the mentions of rbd in the glance docs, and the lack of
cephx auth information in the config, glance seems to be working after all.
S, hooray! It was probably working all along, I just
Well, 100% may be overstating things. When I try to create a volume from an
image it fails. I'm digging through the logs right now. glance alone works
(I can upload and delete images) and cinder alone works (I can create and
delete volumes) but when cinder tries to get the glance service it fails,
On Thu, Apr 2, 2015 at 10:03 AM, Mark Nelson mnel...@redhat.com wrote:
Thought folks might like to see this:
http://hothardware.com/reviews/intel-ssd-750-series-nvme-pci-express-solid-state-drive-review
Quick summary:
- PCIe SSD based on the P3700
- 400GB for $389!
- 1.2GB/s writes and
Hi,
On 04/02/15 19:31, Stillwell, Bryan wrote:
All,
Whenever we're doing some kind of recovery operation on our ceph
clusters (cluster expansion or dealing with a drive failure), there
seems to be a fairly noticable performance drop while it does the
backfills (last time I measured it the
On Thursday, April 2, 2015, Nick Fisk n...@fisk.me.uk wrote:
I'm probably going to get shot down for saying this...but here goes.
As a very rough guide, think of it more as you need around 10Mhz for every
IO, whether that IO is 4k or 4MB it uses roughly the same amount of CPU, as
most of the
Nope,
I redid the cluster with the above config options and it did not fix it.
It must have cached the files from the first copy.
Any thoughts on this?
On Thu, Apr 2, 2015 at 10:18 AM, Barclay Jameson
almightybe...@gmail.com wrote:
I am using the Giant release. The OSDs and MON/MDS are using
45 matches
Mail list logo
