Hi all,
I'm having some issues while trying to activate a new osd in a
new cluster, the prepare command run fine, but then the activate
command failed:
ceph@cephbkdeploy01:~/desp-bkp-cluster$ ceph-deploy --overwrite-conf
disk prepare --fs-type btrfs ceph-bkp-osd01:sdf:/dev/sdc
Hi German,
if i'm right the journal-creation on /dev/sdc1 failed (perhaps because
you only say /dev/sdc instead of /dev/sdc1?).
Do you have partitions on sdc?
Udo
On 31.10.2014 22:02, German Anders wrote:
Hi all,
I'm having some issues while trying to activate a new osd in a
new
We are looking to forward all of our Ceph logs to a centralized syslog
server. In the manual[1] it talks about log settings, but I'm not sure
about a few things.
1. What is clog?
2. If syslog is the logging facility are the logs from all daemons
merged into the same file? Is there a
Hi,
I've yet to install 0.80.7 on one node to confirm its stability and use
the new IO prirority tuning parameters enabling prioritized access to
data from client requests.
In the meantime, faced with large slowdowns caused by resync or external
IO load (although external IO load is not expected
On Tue, Oct 21, 2014 at 10:15 AM, Lionel Bouton lionel+c...@bouton.name wrote:
Hi,
I've yet to install 0.80.7 on one node to confirm its stability and use
the new IO prirority tuning parameters enabling prioritized access to
data from client requests.
In the meantime, faced with large
Hi Gregory,
Le 21/10/2014 19:39, Gregory Farnum a écrit :
On Tue, Oct 21, 2014 at 10:15 AM, Lionel Bouton lionel+c...@bouton.name
wrote:
[...]
Any thought? Is it based on wrong assumptions? Would it prove to be a
can of worms if someone tried to implement it?
Yeah, there's one big thing
On 10/21/2014 01:06 PM, Lionel Bouton wrote:
Hi Gregory,
Le 21/10/2014 19:39, Gregory Farnum a écrit :
On Tue, Oct 21, 2014 at 10:15 AM, Lionel Bouton lionel+c...@bouton.name wrote:
[...]
Any thought? Is it based on wrong assumptions? Would it prove to be a
can of worms if someone tried to
hi,all
take a look at the link ,
http://www.ceph.com/docs/master/architecture/#smart-daemons-enable-hyperscale
could you explain point 2, 3 in that picture.
1.
at point 2,3, before primary writes data to next osd, where is the data? it
is in momory or on disk already?
2. where is the
hi,all
my question is from my test.
let's take a example. object1(4MB)-- pg 0.1 -- osd 1,2,3,p1
when client is writing object1, during the write , osd1 is down. let suppose
2MB is writed.
1.
when the connection to osd1 is down, what does client do? ask monitor for
new osdmap? or only
Hi All!
1st of all thanks in advance.
If my running Ceph cluster is Centos6.5 with Ceph-firefly v80.1, will a
calamari server running on Ubuntu12.04 able to connect/monitor/manage this
cluster?
I would think Ubuntu base agents have to be installed on the storage-nodes in
order for calamari
hi, josh durgin:
please look at my test. inside vm using fio to test rbd performance.
fio paramters: dircet io, bs=4k, iodepth 4
from the infomation below, it does not match.
avgrq-sz is not approximately 8,
for avgqu-sz , its value is small and ruleless, lesser than 32. why?
in ceph ,
when I read RGW code, and can't understand master_ver inside struct
rgw_bucket_dir_header .
who can explain this struct , in especial master_ver and stats , thanks
baijia...@126.com___
ceph-users mailing list
ceph-users@lists.ceph.com
On 09/09/2014 07:06 AM, yuelongguang wrote:
hi, josh.durgin:
i want to know how librbd launch io request.
use case:
inside vm, i use fio to test rbd-disk's io performance.
fio's pramaters are bs=4k, direct io, qemu cache=none.
in this case, if librbd just send what it gets from vm， i mean no
[Moving this to ceph-devel, where you're more likely to get a response
from a developer!]
On Wed, 10 Sep 2014, baijia...@126.com wrote:
when I read RGW code, and can't understand master_ver inside struct
rgw_bucket_dir_header .
who can explain this struct , in especial master_ver and
hi, josh.durgin:
i want to know how librbd launch io request.
use case:
inside vm, i use fio to test rbd-disk's io performance.
fio's pramaters are bs=4k, direct io, qemu cache=none.
in this case, if librbd just send what it gets from vm， i mean no
gather/scatter. the rate , io inside vm :
If you want your data to be N+2 redundant (able to handle 2 failures, more
or less), then you need to set size=3 and have 3 replicas of your data.
If you want your monitors to be N+2 redundant, then you need 5 monitors.
If you feel that your data is worth size=3, then you should really try to
Thanks Greg, Joao and David,
The concept why odd no. of monitors are preferred is clear to me, but still I
am not clear about the working of Paxos algorithm:
#1. All changes in any data structure of monitor whether it is monitor map, OSD
map, PG map, MDS map or CRUSH map; are made through
On 08/30/2014 08:03 AM, pragya jain wrote:
Thanks Greg, Joao and David,
The concept why odd no. of monitors are preferred is clear to me, but
still I am not clear about the working of Paxos algorithm:
#1. All changes in any data structure of monitor whether it is monitor
map, OSD map, PG map,
On Thu, Aug 28, 2014 at 9:52 PM, pragya jain prag_2...@yahoo.co.in wrote:
I have some basic question about monitor and paxos relationship:
As the documents says, Ceph monitor contains cluster map, if there is any
change in the state of the cluster, the change is updated in the cluster
map.
On Fri, Aug 29, 2014 at 12:52 AM, pragya jain prag_2...@yahoo.co.in wrote:
#2: why odd no. of monitors are recommended for production cluster, not even
no.?
Because to achieve a quorum, you must always have participation of
more than 50% of the monitors. Not 50%. More than 50%. With an even
On 08/29/2014 11:22 PM, J David wrote:
So an even number N of monitors doesn't give you any better fault
resilience than N-1 monitors. And the more monitors you have, the
more traffic there is between them. So when N is even, N monitors
consume more resources and provide no extra benefit
I have some basic question about monitor and paxos relationship:
As the documents says, Ceph monitor contains cluster map, if there is any
change in the state of the cluster, the change is updated in the cluster map.
monitor use paxos algorithm to create the consensus among monitors to
hi,all
is there a way to get rbd,ko and ceph.ko for centos 6.X.
or i have to build them from source code? which is the least kernel version?
thanks___
ceph-users mailing list
ceph-users@lists.ceph.com
The rbd diff-related commands compare points in time of a single
image. Since children are identical to their parent when they're cloned,
if I created a snapshot right after it was cloned, I could export
the diff between the used child and the parent. Something like:
rbd clone child parent@snap
The rbd diff-related commands compare points in time of a single
image. Since children are identical to their parent when they're cloned,
if I created a snapshot right after it was cloned, I could export
the diff between the used child and the parent. Something like:
rbd clone child parent@snap
I understand the concept with Ceph being able to recover from the failure of an
OSD (presumably with a single OSD being on a single disk), but I'm wondering
what the scenario is if an OSD server node containing multiple disks should
fail. Presuming you have a server containing 8-10 disks,
, 2014 9:17 AM
To: ceph-us...@ceph.com
Subject: [ceph-users] Question on OSD node failure recovery
I understand the concept with Ceph being able to recover from the failure of an
OSD (presumably with a single OSD being on a single disk), but I’m wondering
what the scenario is if an OSD server node
: Thursday, August 21, 2014 9:17 AM
To: ceph-us...@ceph.com
Subject: [ceph-users] Question on OSD node failure recovery
I understand the concept with Ceph being able to recover from the failure
of an OSD (presumably with a single OSD being on a single disk), but I’m
wondering what the scenario
hi,all
recently i dive into the source code, i am a little confused about them,
maybe because of many threads,wait,seq.
1. what does apply_manager do? it is related to filestore and filejournal.
2. what does SubmitManager do?
3. how they interact and work together?
what a big question :),
These sorts of questions are good for ceph-de...@vger.kernel.org,
which I've added. :)
On Thu, Jul 31, 2014 at 12:24 PM, yuelongguang fasts...@163.com wrote:
hi,all
recently i dive into the source code, i am a little confused about them,
maybe because of many threads,wait,seq.
1. what does
Hi everyone,
I am reading the FileStore source code and confused with the read() and write()
process.
If journal is enable, write() will be acknowledged after the changes are
committed to the journal device but before they are applied to the file system.
However read() is always satisfied
Read will get the newest content. If the read arrives during the write
interval, it will wait for the write applied.
On Wed, Jul 23, 2014 at 10:22 AM, James Pan dev...@yahoo.com wrote:
Hi everyone,
I am reading the FileStore source code and confused with the read() and
write() process.
If
We want to have one replica per rack and more generally two replicas in
the first DC and one in the other one.
So now we are stuck on the crushmap: how to force the cluster to put two
replicas in the first dc?
Is that related to th bucket's weight?
You can fix that in the crush map bucket
Hi,
I have some osd's on hdd's and some on ssd's, just like the example in
these docs:
http://ceph.com/docs/firefly/rados/operations/crush-map/
Now I'd like to place an erasure encoded pool on the hdd's and a
replicated (cache) pool on the ssd's. In order to do that, I have to
split the crush
On 07/06/2014 11:29 PM, Erik Logtenberg wrote:
Hi,
I have some osd's on hdd's and some on ssd's, just like the example in
these docs:
http://ceph.com/docs/firefly/rados/operations/crush-map/
Now I'd like to place an erasure encoded pool on the hdd's and a
replicated (cache) pool on the ssd's.
On Fri, Jun 20, 2014 at 2:02 AM, Erik Logtenberg e...@logtenberg.eu wrote:
Hi Ilya,
Do you happen to know when this fix will be released?
Is upgrading to a newer kernel (client side) still a solution/workaround
too? If yes, which kernel version is required?
This fix is purely server-side,
On Tue, Jun 17, 2014 at 9:46 PM, Ke-fei Lin k...@kfei.net wrote:
2014-06-18 1:28 GMT+08:00 Gregory Farnum g...@inktank.com:
On Tue, Jun 17, 2014 at 3:22 AM, Ke-fei Lin k...@kfei.net wrote:
Hi list,
How does RADOS check an object and its replica are consistent? Is there
a checksum in object's
2014-06-18 22:44 GMT+08:00 Gregory Farnum g...@inktank.com:
On Tue, Jun 17, 2014 at 9:46 PM, Ke-fei Lin k...@kfei.net wrote:
2014-06-18 1:28 GMT+08:00 Gregory Farnum g...@inktank.com:
On Tue, Jun 17, 2014 at 3:22 AM, Ke-fei Lin k...@kfei.net wrote:
Hi list,
How does RADOS check an object and
On Wed, Jun 18, 2014 at 12:07 PM, Ke-fei Lin k...@kfei.net wrote:
2014-06-18 22:44 GMT+08:00 Gregory Farnum g...@inktank.com:
On Tue, Jun 17, 2014 at 9:46 PM, Ke-fei Lin k...@kfei.net wrote:
2014-06-18 1:28 GMT+08:00 Gregory Farnum g...@inktank.com:
On Tue, Jun 17, 2014 at 3:22 AM, Ke-fei Lin
Hi list,
How does RADOS check an object and its replica are consistent? Is there
a checksum in object's metadata or some other mechanisms? Does the
mechanism depend on
OSD's underlying file system?
And what would happen if a corrupted object being readed (like a
corrupted block in traditional
On Tue, Jun 17, 2014 at 3:22 AM, Ke-fei Lin k...@kfei.net wrote:
Hi list,
How does RADOS check an object and its replica are consistent? Is there
a checksum in object's metadata or some other mechanisms? Does the
mechanism depend on
OSD's underlying file system?
It does not check
2014-06-18 1:28 GMT+08:00 Gregory Farnum g...@inktank.com:
On Tue, Jun 17, 2014 at 3:22 AM, Ke-fei Lin k...@kfei.net wrote:
Hi list,
How does RADOS check an object and its replica are consistent? Is there
a checksum in object's metadata or some other mechanisms? Does the
mechanism depend on
Waegeman
kenneth.waege...@ugent.be wrote:
- Message from Igor Krstic igor.z.krs...@gmail.com -
Date: Fri, 06 Jun 2014 13:23:19 +0200
From: Igor Krstic igor.z.krs...@gmail.com
Subject: Re: [ceph-users] question about feature set mismatch
To: Ilya Dryomov ilya.dryo
: Fri, 06 Jun 2014 13:23:19 +0200
From: Igor Krstic igor.z.krs...@gmail.com
Subject: Re: [ceph-users] question about feature set mismatch
To: Ilya Dryomov ilya.dryo...@inktank.com
Cc: ceph-users@lists.ceph.com
On Fri, 2014-06-06 at 11:51 +0400, Ilya Dryomov wrote
On Thu, Jun 5, 2014 at 10:38 PM, Igor Krstic igor.z.krs...@gmail.com wrote:
Hello,
dmesg:
[ 690.181780] libceph: mon1 192.168.214.102:6789 feature set mismatch, my
4a042a42 server's 504a042a42, missing 50
[ 690.181907] libceph: mon1 192.168.214.102:6789 socket error on read
[
On Fri, 2014-06-06 at 11:51 +0400, Ilya Dryomov wrote:
On Thu, Jun 5, 2014 at 10:38 PM, Igor Krstic igor.z.krs...@gmail.com wrote:
Hello,
dmesg:
[ 690.181780] libceph: mon1 192.168.214.102:6789 feature set mismatch, my
4a042a42 server's 504a042a42, missing 50
[ 690.181907]
- Message from Igor Krstic igor.z.krs...@gmail.com -
Date: Fri, 06 Jun 2014 13:23:19 +0200
From: Igor Krstic igor.z.krs...@gmail.com
Subject: Re: [ceph-users] question about feature set mismatch
To: Ilya Dryomov ilya.dryo...@inktank.com
Cc: ceph-users@lists.ceph.com
On Fri, Jun 6, 2014 at 4:34 PM, Kenneth Waegeman
kenneth.waege...@ugent.be wrote:
- Message from Igor Krstic igor.z.krs...@gmail.com -
Date: Fri, 06 Jun 2014 13:23:19 +0200
From: Igor Krstic igor.z.krs...@gmail.com
Subject: Re: [ceph-users] question about feature set
Hello,
dmesg:
[ 690.181780] libceph: mon1 192.168.214.102:6789 feature set mismatch,
my 4a042a42 server's 504a042a42, missing 50
[ 690.181907] libceph: mon1 192.168.214.102:6789 socket error on read
[ 700.190342] libceph: mon0 192.168.214.101:6789 feature set mismatch,
my 4a042a42
Hi all
first off, we have yet to start with Ceph (and other clustered file
systems other than QFS), therefore please consider me a total newbie
w.r.t to Ceph.
We are trying to solve disk I/O problems we face and would like to
explore if we could utilize our currently underused network more in
On Mon, May 26, 2014 at 5:14 PM, Christian Balzer ch...@gol.com wrote:
Hello,
On Mon, 26 May 2014 10:28:12 +0200 Carsten Aulbert wrote:
Hi all
first off, we have yet to start with Ceph (and other clustered file
systems other than QFS), therefore please consider me a total newbie
w.r.t to
发自我的 iPhone
在 2014年5月22日，22:26，Gregory Farnum g...@inktank.com 写道：
On Thu, May 22, 2014 at 5:04 AM, Geert Lindemulder glindemul...@snow.nl
wrote:
Hello All
Trying to implement the osd leveldb backend at an existing ceph test
cluster.
The test cluster was updated from 0.72.1 to
Hello Greg and Haomai,
Thanks for the answers.
I was trying to implement the osd leveldb backend at an existing ceph
test cluster.
At the moment i am removing the osd's one by one and recreate them with
the objectstore = keyvaluestore-dev option in place in ceph.conf.
This works fine and the
Best Wishes!
在 2014年5月23日，19:27，Geert Lindemulder glindemul...@snow.nl 写道：
Hello Greg and Haomai,
Thanks for the answers.
I was trying to implement the osd leveldb backend at an existing ceph
test cluster.
At the moment i am removing the osd's one by one and recreate them with
the
Hello All
Trying to implement the osd leveldb backend at an existing ceph test
cluster.
The test cluster was updated from 0.72.1 to 0.80.1. The update was ok.
After the update, the osd objectstore = keyvaluestore-dev setting was
added to ceph.conf.
After restarting an osd it gives the
On Thu, May 22, 2014 at 5:04 AM, Geert Lindemulder glindemul...@snow.nl wrote:
Hello All
Trying to implement the osd leveldb backend at an existing ceph test
cluster.
The test cluster was updated from 0.72.1 to 0.80.1. The update was ok.
After the update, the osd objectstore =
On Thu, 17 Apr 2014 08:14:04 -0500 John-Paul Robinson wrote:
So in the mean time, are there any common work-arounds?
I'm assuming monitoring imageused/imagesize ratio and if its greater
than some tolerance create a new image and move file system content over
is an effective, if crude
: Wednesday, April 16, 2014 5:36 PM
To: ceph-users@lists.ceph.com
Subject: Re: [ceph-users] question on harvesting freed space
On Wed, 16 Apr 2014 13:12:15 -0500 John-Paul Robinson wrote:
So having learned some about fstrim, I ran it on an SSD backed file
system and it reported space freed. I ran
So in the mean time, are there any common work-arounds?
I'm assuming monitoring imageused/imagesize ratio and if its greater
than some tolerance create a new image and move file system content over
is an effective, if crude approach. I'm not clear on how to measure the
amount of storage an image
So having learned some about fstrim, I ran it on an SSD backed file
system and it reported space freed. I ran it on an RBD backed file
system and was told it's not implemented.
This is consistent with the test for FITRIM.
$ cat /sys/block/rbd3/queue/discard_max_bytes
0
On my SSD backed device
On Wed, 16 Apr 2014 13:12:15 -0500 John-Paul Robinson wrote:
So having learned some about fstrim, I ran it on an SSD backed file
system and it reported space freed. I ran it on an RBD backed file
system and was told it's not implemented.
This is consistent with the test for FITRIM.
$
:36 PM
To: ceph-users@lists.ceph.com
Subject: Re: [ceph-users] question on harvesting freed space
On Wed, 16 Apr 2014 13:12:15 -0500 John-Paul Robinson wrote:
So having learned some about fstrim, I ran it on an SSD backed file
system and it reported space freed. I ran it on an RBD backed file
Hi,
If I have an 1GB RBD image and format it with say xfs of ext4, then I
basically have thin provisioned disk. It takes up only as much space
from the Ceph pool as is needed to hold the data structure of the empty
file system.
If I add files to my file systems and then remove them, how does
I'm assuming Ceph/RBD doesn't have any direct awareness of this since
the file system doesn't traditionally have a give back blocks
operation to the block device. Is there anything special RBD does in
this case that communicates the release of the Ceph storage back to the
pool?
VMs running
pg query says the recovery state is:
might_have_unfound: [
{ osd: 11,
status: querying},
{ osd: 13,
status: already probed}],
I figured out why it wasn't probing osd.11.
When I manually replaced the disk, I added
*Craig Lewis*
Senior Systems Engineer
Office +1.714.602.1309
Email cle...@centraldesktop.com mailto:cle...@centraldesktop.com
*Central Desktop. Work together in ways you never thought possible.*
Connect with us Website http://www.centraldesktop.com/ | Twitter
Ceph is telling me that it can't find some data:
2014-04-07 11:15:09.901992 mon.0 [INF] pgmap v5436846: 2592 pgs: 2164
active+clean, 142 active+remapped+wait_backfill, 150
active+degraded+wait_backfill, 1 active+recovering+degraded, 2
active+degraded+backfilling, 133
The PG with the unfound object has been in active+recovering+degraded
state for much longer than usual. Most PGs spend about 20 minutes in
that state, then complete. This one has been in this in
active+recovering+degraded for about 4 hours now.
11.48388511885217974255906
Is this because it can't find the unfound object? Or is this because
I set osd flag noout and nodown?
Sorry, I meant to say, is it because I set:
root@ceph0c:~# ceph osd dump | grep 'flags'
flags nodown,noout,noscrub,nodeep-scrub
___
ceph-users
Hi Sage,
I have a similar question, I need 2 replicas (one on each rack) and I would
like to know whether the following rule always save primary on rack1?
rule data { ruleset 0 type replicated min_size 2 max_size 2 step take rack1
step chooseleaf firstn 1 type host step emit step take rack2
Hi,
I'm trying to understand the CRUSH algorithm and how it distribute data.
Let's say I simplify a small datacenter setup and map it up
hierarchically in the crush map as show below.
root datacenter
/\
/ \
/\
a b
On Mon, 20 Jan 2014, Arnulf Heimsbakk wrote:
Hi,
I'm trying to understand the CRUSH algorithm and how it distribute data.
Let's say I simplify a small datacenter setup and map it up
hierarchically in the crush map as show below.
root datacenter
/\
On this web page http://ceph.com/docs/master/start/quick-start-preflight/ where
it says Modify your ~/.ssh/config file of your admin node so that it defaults
to logging in as the user you created when no username is specified. Which
config file do I change?
I am using Ubuntu server 13.04.
1.
sshd looks for a per-user config file in ~/.ssh/config in addition to the
system level config in /etc/ssh/. If the file doesn't exist, create it.
More information is available from 'man ssh_config'
On Wed, Oct 2, 2013 at 1:18 PM, Nimish Patel npa...@guidewire.com wrote:
On this web page
User ceph
Host cephtest06
Hostname cephtest06.jf.intel.com
User ceph
ceph@cephtest01:/etc/ceph$
From: ceph-users-boun...@lists.ceph.com
[mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Nimish Patel
Sent: Wednesday, October 02, 2013 11:19 AM
To: ceph-us...@ceph.com
Subject: [ceph-users
Hi folks, I am Dafan from Yahoo! corp. We are really interested in Ceph now. I
wish to know where I can get some performance report about new released
DUMPLING? Like throughput, latency with different cluster scale and hardware
type? Thanks.
Dafan
On Sun, Sep 22, 2013 at 2:35 AM, Dafan Dong don...@yahoo-inc.com wrote:
Hi folks, I am Dafan from Yahoo! corp. We are really interested in Ceph now.
I wish to know where I can get some performance report about new released
DUMPLING? Like throughput, latency with different cluster scale and
Dave,OpenStack does"qemu-img snapshot" command to create a snapshot, here's the method:https://github.com/openstack/nova/blob/stable/folsom/nova/virt/libvirt/utils.py#L335-L347So the memory is _not_ saved, only the disk is. Note that it's always hard to make consistent snapshot. I assume that
Hi Dave,
Disclaimer: I'm not an OpenStack user.
On 04/05/2013 03:43 PM, Dave Spano wrote:
If I pause my instances in Openstack, then snapshot and clone my volumes, I
should have a consistent backup correct? Is freezing on snapshot creation like
LVM a potential future feature?
No, probably
If I pause my instances in Openstack, then snapshot and clone my volumes, I
should have a consistent backup correct? Is freezing on snapshot creation like
LVM a potential future feature?
I've considered Sebastien's method here(
On 03/04/2013 07:12 PM, Travis Rhoden wrote:
Joao,
Were you able to glean anything useful from the memory dump I provided?
Hey Travis,
Haven't had the chance to look into the dump, but it's still on my stack
to go over as soon as I'm able to get into it.
The mon did eventually crash,
201 - 281 of 281 matches
Mail list logo
