Hi,
I've heard many time that to install multiple rados-gateway on the same server
is possible, just need to create on a different port.
However I've never managed to make it work.
Today I gave another try like this:
1. Created a new keyring: ceph auth get-or-create
Seems like someone is not testing cephadm on centos 7.9
Just tried installing cephadm from the repo, and ran
cephadm bootstrap --mon-ip=xxx
it blew up, with
ceph TypeError: __init__() got an unexpected keyword argument
'verbose_on_failure'
just after the firewall section.
I happen to have a
is the ceph insights plugin enabled? this caused huge huge bloat of the mon
stores for me. before i figured that out, i turned on leveldb compression
options on the mon store and got pretty significant savings, also.
On Tue, Mar 2, 2021 at 6:56 PM Lincoln Bryant wrote:
> Hi list,
>
> We
Hi Frank,
On Tue, Mar 02, 2021 at 02:58:05PM +, Frank Schilder wrote:
Hi all,
this is a follow-up on "reboot breaks OSDs converted from ceph-disk to ceph-volume
simple".
I converted a number of ceph-disk OSDs to ceph-volume using "simple scan" and
"simple activate". Somewhere along the
James,
Can you tell me what's the hardware config of your bcache? I use the 400G SATA
SSD as cache device and
10T HDD as the storage device. Hardware relationed?
On 2021/3/2 下午4:49, James Page wrote:
> Hi Norman
>
> On Mon, Mar 1, 2021 at 4:38 AM Norman.Kern wrote:
>
>> Hi, guys
>>
>> I am
On 3/2/21 7:17 PM, Stefan Kooman wrote:
What is output of "ceph daemon osd.0 config get ms_bind_ipv4" on the
osd0 node?
ceph daemon osd.0 config get ms_bind_ipv4
{
"ms_bind_ipv4": "true"
}
And
ceph daemon mds.mds1 config get ms_bind_ipv4
{
"ms_bind_ipv4": "true"
}
for that
Hi list,
We recently had a cluster outage over the weekend where several OSDs were
inaccessible over night for several hours. When I found the cluster in the
morning, the monitors' root disks (which contained both the monitor's leveldb
and the Ceph logs) had completely filled.
After
Hello Dave,
I recommend you read this docu:
https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/3/html/administration_guide/understanding-process-managemnet-for-ceph#powering-down-and-rebooting-a-red-hat-ceph-storage-cluster-management
Regards, Joachim
On 3/2/21 6:54 PM, Ilya Dryomov wrote:
--- snip ---
osd.0 up in weight 1 up_from 98071 up_thru 98719 down_at 98068
last_clean_interval [96047,98067)
[v2:[2001:7b8:80:1:0:1:2:1]:6848/505534,v1:[2001:7b8:80:1:0:1:2:1]:6854/505534,v2:0.0.0.0:6860/505534,v1:0.0.0.0:6866/505534]
Where did
On 3/2/21 6:00 PM, Jeff Layton wrote:
v2 support in the kernel is keyed on the ms_mode= mount option, so that
has to be passed in if you're connecting to a v2 port. Until the mount
helpers get support for that option you'll need to specify the address
and port manually if you want to use v2.
Phooey. :)
Do you know of any notification subsystems in libRADOS that might be
useful?
Will have to think on this...
Thanks
On Tue, Mar 2, 2021 at 4:05 PM Matt Benjamin wrote:
> Right. The elastic search integration--or something custom you could
> base on s3 bucket notifications--would
Right. The elastic search integration--or something custom you could
base on s3 bucket notifications--would both be working with events
generated in RGW.
Matt
On Tue, Mar 2, 2021 at 3:55 PM Cary FitzHugh wrote:
>
> Understood.
>
> With the RGW architecture comes more load balancing concerns,
On 3/2/21 5:42 PM, Ilya Dryomov wrote:
On Tue, Mar 2, 2021 at 9:26 AM Stefan Kooman wrote:
Hi,
On a CentOS 7 VM with mainline kernel (5.11.2-1.el7.elrepo.x86_64 #1 SMP
Fri Feb 26 11:54:18 EST 2021 x86_64 x86_64 x86_64 GNU/Linux) and with
Ceph Octopus 15.2.9 packages installed. The MDS server
Understood.
With the RGW architecture comes more load balancing concerns, more moving
parts, more tedious (to me) ACLs, less features (append and some other
things not supported in S3). Was hoping for a solution which didn't
require us to be hamstrung and only read / write to a pool via the
Hi Cary,
As you've said, these are well-developed features of RGW, I think that
would be the way to go, in the Ceph ecosystem.
Matt
On Tue, Mar 2, 2021 at 3:41 PM Cary FitzHugh wrote:
>
> Hello -
>
> We're trying to use native libRADOS and the only challenge we're running
> into is searching
On 3/2/21 5:16 PM, Jeff Layton wrote:
On Tue, 2021-03-02 at 09:25 +0100, Stefan Kooman wrote:
Hi,
On a CentOS 7 VM with mainline kernel (5.11.2-1.el7.elrepo.x86_64 #1 SMP
Fri Feb 26 11:54:18 EST 2021 x86_64 x86_64 x86_64 GNU/Linux) and with
I'm guessing this is a stable series kernel
It's
Hello -
We're trying to use native libRADOS and the only challenge we're running
into is searching metadata.
Using the rgw metadata sync seems to require all data to be pushed through
the rgw, which is not something we're interested in setting up at the
moment.
Are there hooks or features of
Hi all,
this is a follow-up on "reboot breaks OSDs converted from ceph-disk to
ceph-volume simple".
I converted a number of ceph-disk OSDs to ceph-volume using "simple scan" and
"simple activate". Somewhere along the way, the OSDs meta-data gets rigged and
the prominent symptom is that the
I wanted to revisit this - we're not on 15.2.9 and still have this one
cluster with 5 PGs "stuck" in pg_temp. Any idea how to clean this up,
or how it might have occurred? I'm fairly certain it showed up after
an autoscale up and autoscale down happened that overlapped each
other.
On Mon, Aug 10,
Hi all,
Following up on a previous issue.
My cephfs MDS is reporting damaged metadata following the addition (and
remapping) of 12 new OSDs.
`ceph tell mds.database-0 damage ls` reports ~85 files damaged. All of type
"backtrace".
` ceph tell mds.database-0 scrub start /
On Tue, Mar 2, 2021 at 6:02 PM Stefan Kooman wrote:
>
> On 3/2/21 5:42 PM, Ilya Dryomov wrote:
> > On Tue, Mar 2, 2021 at 9:26 AM Stefan Kooman wrote:
> >>
> >> Hi,
> >>
> >> On a CentOS 7 VM with mainline kernel (5.11.2-1.el7.elrepo.x86_64 #1 SMP
> >> Fri Feb 26 11:54:18 EST 2021 x86_64 x86_64
For comparison, the output of device discovery from ceph-disk and ceph-volume.
ceph-disk does it correctly, ceph-volume is screwed up:
[root@ceph-adm:ceph-18 ceph-241]# ceph-disk list /dev/sdb
/usr/lib/python2.7/site-packages/ceph_disk/main.py:5689: UserWarning:
Dear all,
ceph version: mimic 13.2.10
I'm facing a serious bug with devices converted from "ceph-disk" to
"ceph-volume simple". I "converted" all ceph-disk devices using "ceph-volume
simple scan ..." And everything worked fine at the beginning. Today I needed to
reboot an OSD host and since
On Tue, 2021-03-02 at 17:44 +0100, Stefan Kooman wrote:
> On 3/2/21 5:16 PM, Jeff Layton wrote:
> > On Tue, 2021-03-02 at 09:25 +0100, Stefan Kooman wrote:
> > > Hi,
> > >
> > > On a CentOS 7 VM with mainline kernel (5.11.2-1.el7.elrepo.x86_64 #1 SMP
> > > Fri Feb 26 11:54:18 EST 2021 x86_64
Dave,
Just to be certain of the terminology,
-
Step before Step 4: Quiesce client systems using Ceph
Step 4: Turn off everything that's not a MGR, MON, or OSD.
Step 5: Turn off OSDs
Step 6: Turn off MONs
Step 7: Turn off MGRs
If any of the above are running on the the same nodes
On Tue, Mar 2, 2021 at 9:26 AM Stefan Kooman wrote:
>
> Hi,
>
> On a CentOS 7 VM with mainline kernel (5.11.2-1.el7.elrepo.x86_64 #1 SMP
> Fri Feb 26 11:54:18 EST 2021 x86_64 x86_64 x86_64 GNU/Linux) and with
> Ceph Octopus 15.2.9 packages installed. The MDS server is running
> Nautilus 14.2.16.
Hi,
I've upgraded our test cluster to Octopus, and enabled the auto-scaler.
It's nearly finished:
PG autoscaler decreasing pool 11 PGs from 1024 to 32 (4d)
[==..] (remaining: 3h)
But I notice it looks to be making pool 11 smaller when HEALTH_WARN
thinks it
On Tue, 2021-03-02 at 09:25 +0100, Stefan Kooman wrote:
> Hi,
>
> On a CentOS 7 VM with mainline kernel (5.11.2-1.el7.elrepo.x86_64 #1 SMP
> Fri Feb 26 11:54:18 EST 2021 x86_64 x86_64 x86_64 GNU/Linux) and with
I'm guessing this is a stable series kernel
> Ceph Octopus 15.2.9 packages
Hi,
installation of cluster/osds went "by the book" https://docs.ceph.com/, but
now I want to setup Ceph Object Gateway, but documentation on
https://docs.ceph.com/en/latest/radosgw/ seems to lack information about
what and where to restart for example when setting [client.rgw.gateway-node1]
in
Hi,
On a CentOS 7 VM with mainline kernel (5.11.2-1.el7.elrepo.x86_64 #1 SMP
Fri Feb 26 11:54:18 EST 2021 x86_64 x86_64 x86_64 GNU/Linux) and with
Ceph Octopus 15.2.9 packages installed. The MDS server is running
Nautilus 14.2.16. Messenger v2 has been enabled. Poort 3300 of the
monitors is
On Tue, Mar 02, 2021 at 05:47:29PM +0800, Norman.Kern wrote:
> Matthias,
>
> I agreed with you for tuning. I ask this question just for that my OSDs have
> problems when the
>
> cache_available_percent less than 30, the SSDs almost useless and all I/Os
> bypass to HDDs with large latency.
Hello,
we clearly understood that. But in ceph we have the concept of "OSD
Journal on very fast different disk".
I just asked what in theory should be the advantage of caching on
bcache/NVME vs. Journal/NVME. I would not expect any performance
advantage for bcache (if the Journal is reasonably
On 2021/3/2 下午4:49, James Page wrote:
> Hi Norman
>
> On Mon, Mar 1, 2021 at 4:38 AM Norman.Kern wrote:
>
>> Hi, guys
>>
>> I am testing ceph on bcache devices, I found the performance is not good
>> as expected. Does anyone have any best practices for it? Thanks.
>>
> I've used bcache quite a
On 2021/3/2 上午5:09, Andreas John wrote:
> Hallo,
>
> do you expect that to be better (faster), than having the OSD's Journal
> on a different disk (ssd, nvme) ?
No, I created the OSD storage devices using bcache devices.
>
>
> rgds,
>
> derjohn
>
>
> On 01.03.21 05:37, Norman.Kern wrote:
>> Hi,
On 2021/3/1 下午6:32, Matthias Ferdinand wrote:
> On Mon, Mar 01, 2021 at 12:37:38PM +0800, Norman.Kern wrote:
>> Hi, guys
>>
>> I am testing ceph on bcache devices, I found the performance is not
>> good as expected. Does anyone have any best practices for it? Thanks.
> Hi,
>
> sorry to say, but
On 03/01 21:41, Dave Hall wrote:
> Hello,
>
> I've had a look at the instructions for clean shutdown given at
> https://ceph.io/planet/how-to-do-a-ceph-cluster-maintenance-shutdown/, but
> I'm not clear about some things on the steps about shutting down the
> various Ceph components.
>
> For my
Hi Norman
On Mon, Mar 1, 2021 at 4:38 AM Norman.Kern wrote:
> Hi, guys
>
> I am testing ceph on bcache devices, I found the performance is not good
> as expected. Does anyone have any best practices for it? Thanks.
>
I've used bcache quite a bit with Ceph with the following configuration
37 matches
Mail list logo
