Hi,
Yes, im using bluestore.
there is no I/O on the ceph cluster. it's totally idle.
All the CPU usage are by OSD who don't have any workload on it.
Thanks!
On Thu, Nov 9, 2017 at 9:37 AM, Vy Nguyen Tan
wrote:
> Hello,
>
> I think it not normal behavior in Luminous.
On Fri, Nov 03, 2017 at 12:09:03PM +0100, Alwin Antreich wrote:
> Hi,
>
> I am confused by the %USED calculation in the output 'ceph df' in luminous.
> In the example below the pools use 2.92% "%USED" but with my calculation,
> taken from the source code it gives me a 8.28%. On a hammer cluster
Hi, Nick
Thank you for the answer!
It's still unclear for me, do those options have no effect at all?
Or disk thread is used for some other operations?
09.11.2017, 04:18, "Nick Fisk" :
>> -Original Message-
>> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com]
You're correct, if you were going to put the WAL and DB on the same device you
should just make one partition and allocate the DB to it, the WAL will
automatically be stored with the DB. It only makes sense to specify them
separately if they are going to go on different devices, and that itself
I get confused there because on the
documentation:http://docs.ceph.com/docs/master/rados/configuration/bluestore-config-ref/
"If there is more, provisioning a DB device makes more sense. The BlueStore
journal will always be placed on the fastest device available, so using a DB
device will
How/where can I see how eg. 'profile rbd' is defined?
As in
[client.rbd.client1]
key = xxx==
caps mon = "profile rbd"
caps osd = "profile rbd pool=rbd"
___
ceph-users mailing list
ceph-users@lists.ceph.com
2017-11-08 22:05 GMT+01:00 Marc Roos :
>
> Can anyone advice on a erasure pool config to store
>
> - files between 500MB and 8GB, total 8TB
> - just for archiving, not much reading (few files a week)
> - hdd pool
> - now 3 node cluster (4th coming)
> - would like to save
Hi Greg,
Thanks! This seems to have worked for at least 1 of 2 inconsistent pgs:
The inconsistency disappeared after a new scrub. Still waiting for the
result of the second pg. I tried to force deep-scrub with `ceph pg
deep-scrub ` yesterday, but today the last deep scrub is still from
a
On 2017-11-08T21:41:41, Sage Weil wrote:
> Who is running nfs-ganesha's FSAL to export CephFS? What has your
> experience been?
>
> (We are working on building proper testing and support for this into
> Mimic, but the ganesha FSAL has been around for years.)
We use it
Hi Sage,
As Lars mentioned, at SUSE, we use ganesha 2.5.2/luminous. We did a preliminary
performance comparison of cephfs client
and nfs-ganesha client. I have attached the results. The results are aggregate
bandwidth over 10 clients.
1. Test Setup:
We use fio to read/write to a single 5GB
What would be the correct way to convert the xml file rbdmapped images
to librbd?
I had this:
And for librbd this:
But this will give me a
> On Nov 9, 2017, at 5:25 AM, Sam Huracan wrote:
>
> root@radosgw system]# ceph --admin-daemon
> /var/run/ceph/ceph-client.rgw.radosgw.asok config show | grep log_file
> "log_file": "/var/log/ceph/ceph-client.rgw.radosgw.log”,
The .asok filename resembles what
On Thu, Nov 9, 2017 at 10:12 AM, Marc Roos wrote:
>
> How/where can I see how eg. 'profile rbd' is defined?
>
> As in
> [client.rbd.client1]
> key = xxx==
> caps mon = "profile rbd"
> caps osd = "profile rbd pool=rbd"
The profiles are defined
Hi!I would like to export my cephfs using Ganesha NFS to export a NFSv3 o NFSv4
I am a little lost while doing it. I managed to make it working with NFSv4 but
I can't make it work with NFSv3 as the server refuses the connection.
Has anyone managed to do it?
They are currently defined to the following (translated to cap syntax):
mon: 'allow service mon r, allow service osd r, allow service pg r,
allow command "osd blacklist" with blacklistop=add addr regex
"^[^/]+/[0-9]+$"'
osd: 'allow class-read object_prefix rbd_children, allow class-read
-sorry, wrong address
Hi Richard,
I have seen a few lectures about bluestore, and they made it abundantly
clear, that bluestore is superior to filestore in that manner, that it
writes data to the disc only once (this way they could achieve a 2x-3x
speed increase).
So this is true if there
Hi all,
I’ve experienced a strange issue with my cluster.
The cluster is composed by 10 HDDs nodes with 20 nodes + 4 journal each plus 4
SSDs nodes with 5 SSDs each.
All the nodes are behind 3 monitors and 2 different crush maps.
All the cluster is on 10.2.7
About 20 days ago I started to
One small point: It's a bit easier to observe distinct WAL and DB
behavior when they are on separate partitions. I often do this for
benchmarking and testing though I don't know that it would be enough of
a benefit to do it in production.
Mark
On 11/09/2017 04:16 AM, Richard Hesketh wrote:
The email was not delivered to ceph-de...@vger.kernel.org. So, re-sending it.
Few more things regarding the hardware and clients used in our benchmarking
setup:
- The cephfs benchmark were done using kernel cephfs client.
- NFS-Ganesha was mounted using nfs version 4.
- Single nfs-ganesha
Hi Alwin,
Thanx for the help.
I see now that I used the wrong wording in my email. I want to resize the
journal, not upgrade.
So, following your commands, I still sit with a 1GB journal:
oot@virt1:~# ceph-disk prepare --bluestore \
> --block.db /dev/sde --block.wal /dev/sde1 /dev/sda
Setting
Update: I noticed that there was a pg that remained scrubbing from the first
day I found the issue to when I reboot the node and problem disappeared.
Can this cause the behaviour I described before?
> Il giorno 09 nov 2017, alle ore 15:55, Matteo Dacrema ha
> scritto:
>
>
Hi Rudi,
On Thu, Nov 09, 2017 at 04:09:04PM +0200, Rudi Ahlers wrote:
> Hi,
>
> Can someone please tell me what the correct procedure is to upgrade a CEPH
> journal?
>
> I'm running ceph: 12.2.1 on Proxmox 5.1, which runs on Debian 9.1
>
> For a journal I have a 400GB Intel SSD drive and it seems
Please bear in mind that unless you've got a very good reason for separating
the WAL/DB into two partitions (i.e. you are testing/debugging and want to
observe their behaviour separately or they're actually going to go on different
devices which have different speeds) you should probably stick
2017-11-09 17:02 GMT+01:00 Alwin Antreich :
> Hi Rudi,
> On Thu, Nov 09, 2017 at 04:09:04PM +0200, Rudi Ahlers wrote:
> > Hi,
> >
> > Can someone please tell me what the correct procedure is to upgrade a
> CEPH
> > journal?
> >
> > I'm running ceph: 12.2.1 on Proxmox 5.1,
I installed rados-objclass-dev and objclass.h was installed successfully.
However, I failed to run the objclass following the steps as below:
1. copy https://github.com/ceph/ceph/blob/master/src/cls/sdk/cls_sdk.cc
into my machine. (cls_test.cpp)
2. make some changes on cls_test.cpp: 1) rename all
Rudi,
You can set the size of block.db and block.wal partitions in the ceph.conf
configuration file using:
bluestore_block_db_size = 16106127360 (which is 15GB, just calculate the
correct number for your needs)
bluestore_block_wal_size = 16106127360
Kind regards,
Caspar
2017-11-09 17:19
Hi Caspar,
Is this in the [global] or [osd] section of ceph.conf?
I am new to ceph so this is all still very vague to me.
What is the difference betwen the WAL and the DB?
And, lastly, if I want to setup the OSD in Proxmox beforehand and add the
journal to it, can I make these changes
On Thu, Nov 9, 2017 at 10:05 AM, Zheyuan Chen wrote:
> I installed rados-objclass-dev and objclass.h was installed successfully.
> However, I failed to run the objclass following the steps as below:
>
> 1. copy https://github.com/ceph/ceph/blob/master/src/cls/sdk/cls_sdk.cc
I changed this line into CLS_LOG(0, "loading cls_test");
https://github.com/ceph/ceph/blob/master/src/cls/sdk/cls_sdk.cc#L120
I don't think the test object class is loaded correctly since I don't have
the loading information in the log.
However I can see "loading cls_sdk" in the osd log.
On
I would like store objects with
rados -p ec32 put test2G.img test2G.img
error putting ec32/test2G.img: (27) File too large
Changing the pool application from custom to rgw did not help
___
ceph-users mailing list
ceph-users@lists.ceph.com
Marc,
If you're running luminous you may need to increase osd_max_object_size.
This snippet is from the Luminous change log.
"The default maximum size for a single RADOS object has been reduced
from 100GB to 128MB. The 100GB limit was completely impractical in
practice while the 128MB limit
It should be noted that the general advise is to not use such large
objects since cluster performance will suffer, see also this thread:
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2017-September/021051.html
libradosstriper might be an option which will automatically break the
object into
Hi everyone,
Beginner with Ceph, i’m looking for a way to do a 3-way replication between 2
datacenters as mention in ceph docs (but not describe).
My goal is to keep access to the data (at least read-only access) even when the
link between the 2 datacenters is cut and make sure at least one
Hi,
Can someone please tell me what the correct procedure is to upgrade a CEPH
journal?
I'm running ceph: 12.2.1 on Proxmox 5.1, which runs on Debian 9.1
For a journal I have a 400GB Intel SSD drive and it seems CEPH created a
1GB journal:
Disk /dev/sdf: 372.6 GiB, 400088457216 bytes,
I added an erasure k=3,m=2 coded pool on a 3 node test cluster and am
getting these errors.
pg 48.0 is stuck undersized for 23867.00, current state
active+undersized+degraded, last acting [9,13,2147483647,7,2147483647]
pg 48.1 is stuck undersized for 27479.944212, current state
Yes, I actually changed it back to the default after reading somewhat
about it (https://github.com/ceph/ceph/pull/15520). I wanted to store
5GB and 12GB files, that makes recovery not to nice. I thought there was
a setting to split them up automatically like with rbd pools.
-Original
Do you know of a rados client that uses this? Maybe a simple 'mount' so
I can cp the files on it?
-Original Message-
From: Christian Wuerdig [mailto:christian.wuer...@gmail.com]
Sent: donderdag 9 november 2017 22:01
To: Kevin Hrpcek
Cc: Marc Roos; ceph-users
Subject: Re:
In my cluster, rados bench shows about 1GB/s bandwidth. I've done some
tuning:
[osd]
osd op threads = 8
osd disk threads = 4
osd recovery max active = 7
I was hoping to get much better bandwidth. My network can handle it, and
my disks are pretty fast as well. Are there any major tunables I
38 matches
Mail list logo