[ceph-users] Re: Nautilus upgrade causes spike in MDS latency

2020-04-14 Thread Gregory Farnum
On Mon, Apr 13, 2020 at 3:16 PM Josh Haft wrote: > > On Mon, Apr 13, 2020 at 4:14 PM Gregory Farnum wrote: > > > > On Mon, Apr 13, 2020 at 1:33 PM Josh Haft wrote: > > > > > > Hi, > > > > > > I upgraded from 13.2.5 to 14.2.6 last week and am now seeing > > > significantly higher latency on

[ceph-users] Re: Existing Cluster to cephadm - mds start failing

2020-04-14 Thread Ashley Merrick
I have logged the following bug ticket for it : https://tracker.ceph.com/issues/45091 I have also noticed another bug with cephadm which I have logged under : https://tracker.ceph.com/issues/45092 Thanks On Mon, 13 Apr 2020 12:36:01 +0800 Ashley Merrick wrote Completed the

[ceph-users] Re: radosgw garbage collection seems stuck and mannual gc process didn't work

2020-04-14 Thread Peter Parker
Hi Matt, We upgraded our cluster to 13.2.8 yesterday . After restarting radosgw, gc process successfully cleaned up those objects and omap. thanks again ! By the way, for other users , in our case the backlog had been increased to more than 3 million. the cleanup after upgrading

[ceph-users] Re: MDS: what's the purpose of using LogEvent with empty metablob?

2020-04-14 Thread Yan, Zheng
On Wed, Apr 15, 2020 at 9:40 AM Xinying Song wrote: > > Hi, Greg: > Thanks for your reply! > I think master can always know if a request has been finished or not > no matter whether > there is a Commit-logevent, because it has written a EUpdate logevent > that records the > unfinished request. >

[ceph-users] Re: MDS: what's the purpose of using LogEvent with empty metablob?

2020-04-14 Thread Xinying Song
Hi, Greg: Thanks for your reply! I think master can always know if a request has been finished or not no matter whether there is a Commit-logevent, because it has written a EUpdate logevent that records the unfinished request. Of course, we need to do commit, in which we clean up mdcache and

[ceph-users] CephFS and Samba/CIFS permissions (xattr)

2020-04-14 Thread Victor Rodriguez
Hello, I have a CephFS running on v14.2.8 correctly. I also have a VM which runs Samba as AD controller and fileserver (Zentyal). My plan was to mount a CephFS path on that VM and make Samba share those files to a Windows network. But I cant make the shares work as Samba is asking to mount the

[ceph-users] Announcing go-ceph v0.3.0

2020-04-14 Thread John Mulligan
I'm happy to announce the another release of the go-ceph API bindings. This is a regular release following our every-two-months release cadence. https://github.com/ceph/go-ceph/releases/tag/v0.3.0 The bindings aim to play a similar role to the "pybind" python bindings in the ceph tree but for

[ceph-users] Re: MDS: cache pressure warnings with Ganesha exports

2020-04-14 Thread Jeff Layton
On Tue, 2020-04-14 at 06:27 +, Stolte, Felix wrote: > Hi Jeff, > > thank you for the hint. I set Entries_HWMark = 100 in MDCACHE Section > of ganesha.conf and upgraded ganesha to 3.2 this weekend. Cache > Pressure warnings still keep accuring, but not as frequent as before. > Is there another

[ceph-users] Re: Fwd: Question on rbd maps

2020-04-14 Thread Void Star Nill
Thanks Ilya. I am indeed using lock ls command with workload ID corresponding to the lock tag - works reasonably well. I was just wondering if there were better options. Thanks for all the inputs. Thanks Shridhar On Mon, Apr 13, 2020 at 4:23 AM Ilya Dryomov wrote: > Tying this with your other

[ceph-users] Re: Fwd: question on rbd locks

2020-04-14 Thread Void Star Nill
That makes sense. Thanks Ilya. On Mon, Apr 13, 2020 at 4:10 AM Ilya Dryomov wrote: > As Paul said, a lock is typically broken by a new client trying > to grab it. As part of that the existing lock holder needs to be > blacklisted, unless you fence using some type of STONITH. > > The question

[ceph-users] Re: ceph-mgr with large connections in CLOSE_WAIT state

2020-04-14 Thread Void Star Nill
Hi, Any thoughts on this? Regards Shridhar On Thu, Apr 9, 2020 at 5:17 PM Void Star Nill wrote: > Hi, > > I am seeing a large number of connections from ceph-mgr are stuck in > CLOSE_WAIT state with data stuck in the receive queue. Looks like ceph-mgr > process is not reading the data

[ceph-users] how to fix num_strays?

2020-04-14 Thread Dan van der Ster
Hi all, Following some cephfs issues today we have a stable cluster but the num_strays is incorrect. After starting the mds, the values are reasonable, but they very soon underflow and start showing 18E (2^64 - a few) ---mds --mds_cache--- --mds_log--

[ceph-users] Re: MDS: what's the purpose of using LogEvent with empty metablob?

2020-04-14 Thread Gregory Farnum
On Sun, Apr 12, 2020 at 5:19 AM Xinying Song wrote: > > Hi, cephers: > What's the purpose of using LogEvent with empty metablob? > For example in link/unlink operation cross two active mds, > when slave receives OP_FINISH it will write an ESlaveUpdate::OP_COMMIT > to the journal, then > send

[ceph-users] Re: radosgw garbage collection seems stuck and mannual gc process didn't work

2020-04-14 Thread Matt Benjamin
Hi Peter, You won't need to do anything--the gc process will clear the stall and begin clearing its backlog immediately after the upgrade. Matt On Sat, Apr 11, 2020 at 10:42 PM Peter Parker <346415...@qq.com> wrote: > > thanks a lot > i'm not sure if the PR is

[ceph-users] Re: Check if upmap is supported by client?

2020-04-14 Thread Paul Emmerich
Hi, On Mon, Apr 13, 2020 at 3:08 PM Frank Schilder wrote: > > Hi Paul, > > thanks for the fast reply. When you say "bit 21", do you mean "(feature_map & > 2^21) == true" (i.e., counting from 0 starting at the right-hand end)? yes > Assuming upmap is supported by all clients. If I understand

[ceph-users] Re: MDS: obscene buffer_anon memory use when scanning lots of files

2020-04-14 Thread Yan, Zheng
On Tue, Apr 14, 2020 at 9:41 PM Dan van der Ster wrote: > > On Tue, Apr 14, 2020 at 2:50 PM Dan van der Ster wrote: > > > > On Sun, Apr 12, 2020 at 9:33 PM Dan van der Ster > > wrote: > > > > > > Hi John, > > > > > > Did you make any progress on investigating this? > > > > > > Today I also saw

[ceph-users] Re: PGs unknown (osd down) after conversion to cephadm

2020-04-14 Thread Sebastian Wagner
Might be an issue with cephadm. Do you have the output of `ceph orch host ls --format json` and `ceph orch ls --format json`? Am 09.04.20 um 13:23 schrieb Dr. Marco Savoca: > Hi all, > >   > > last week I successfully upgraded my cluster to Octopus and converted it > to cephadm. The conversion

[ceph-users] Re: MDS: obscene buffer_anon memory use when scanning lots of files

2020-04-14 Thread Dan van der Ster
On Tue, Apr 14, 2020 at 2:50 PM Dan van der Ster wrote: > > On Sun, Apr 12, 2020 at 9:33 PM Dan van der Ster wrote: > > > > Hi John, > > > > Did you make any progress on investigating this? > > > > Today I also saw huge relative buffer_anon usage on our 2 active mds's > > running 14.2.8: > > > >

[ceph-users] Re: Fw: Incompatibilities (implicit_tenants & barbican) with Openstack after migrating from Ceph Luminous to Nautilus.

2020-04-14 Thread Scheurer François
Dear Casey I hope you had a good Easter and that this mail finds you in good health. I was wondering if you had some time to answer the question below regarding the backward compatibility of the RGW. Many thanks! Sincerely Francois From: Scheurer

[ceph-users] Re: MDS: obscene buffer_anon memory use when scanning lots of files

2020-04-14 Thread Dan van der Ster
On Sun, Apr 12, 2020 at 9:33 PM Dan van der Ster wrote: > > Hi John, > > Did you make any progress on investigating this? > > Today I also saw huge relative buffer_anon usage on our 2 active mds's > running 14.2.8: > > "mempool": { > "by_pool": { > "bloom_filter": { >

[ceph-users] MDS : replace a standby-replay daemon by an active one

2020-04-14 Thread Herve Ballans
Hello to all confined people (and the others too) ! On one of my Ceph cluster (Nautilus 14.2.3), I previously set up 3 MDS daemons in active/standy-replay/standby configuration. For design reasons, I would like to replace this configuration by an active/active/standby one. It means replace

[ceph-users] Re: MDS: cache pressure warnings with Ganesha exports

2020-04-14 Thread Stolte, Felix
Hi Jeff, thank you for the hint. I set Entries_HWMark = 100 in MDCACHE Section of ganesha.conf and upgraded ganesha to 3.2 this weekend. Cache Pressure warnings still keep accuring, but not as frequent as before. Is there another suggestion I did miss? Regards Felix