Re: [ceph-users] any recommendation of using EnhanceIO?

I did some (non-ceph) work on these, and concluded that bcache was the best supported, most stable, and fastest. This was ~1 year ago, to take it with a grain of salt, but that's what I would recommend. Daniel - Original Message - From: "Dominik Zalewski" To: "German Anders" Cc:

Re: [ceph-users] Status of ceph-docker

On 05/03/2016 04:17 PM, Vincenzo Pii wrote: https://github.com/ceph/ceph-docker Is someone using ceph-docker in production or the project is meant more for development and experimentation? Vincenzo Pii| TERALYTICS *DevOps Engineer * I'm not aware of anyone currently using it in production, bu

Re: [ceph-users] Running ceph in docker

On 06/30/2016 02:05 AM, F21 wrote: Hey all, I am interested in running ceph in docker containers. This is extremely attractive given the recent integration of swarm into the docker engine, making it really easy to set up a docker cluster. When running ceph in docker, should monitors, radosgw an

Re: [ceph-users] ceph/daemon mon not working and status exit (1)

On 07/07/2016 08:06 PM, Rahul Talari wrote: I am trying to use Ceph in Docker. I have built the ceph/base and ceph/daemon DockeFiles. I am trying to deploy a Ceph monitor according to the instructions given in the tutorial but when I execute the command without KV store and type: sudo docker ps

Re: [ceph-users] OSPF to the host

On 07/11/2016 08:23 AM, Saverio Proto wrote: I'm looking at the Dell S-ON switches which we can get in a Cumulus version. Any pro's and con's of using Cumulus vs old school switch OS's you may have come across? Nothing to declare here. Once configured properly the hardware works as expected

Re: [ceph-users] osd inside LXC

This is fairly standard for container deployment: one app per container instance. This is how we're deploying docker in our upstream ceph-docker / ceph-ansible as well. Daniel On 07/13/2016 08:41 PM, Łukasz Jagiełło wrote: Hi, Just wonder why you want each OSD inside separate LXC container?

Re: [ceph-users] Radosgw admin ops API command question

On 07/21/2016 05:04 AM, Horace wrote: Hello all, I've a question regarding Ceph Radosgw Admin ops API would like to ask your help. I couldn't find the API equivalent of the follow CLI command to look up the total bytes of a user. Your help will be appreciated, thanks in advance! radosgw-admin

Re: [ceph-users] RGW lifecycle not expiring objects

}, { "bucket": "::default.24713983.2", "status": "PROCESSING" }, At 10 loglevel, the lifecycle processor logs 'DELETED' each time it deletes som

Re: [ceph-users] Radosgw versioning S3 compatible?

On 06/28/2017 12:52 PM, Yehuda Sadeh-Weinraub wrote: On Wed, Jun 28, 2017 at 8:13 AM, Martin Emrich wrote: Correction: It’s about the Version expiration, not the versioning itself. We send this rule: Rules: [ { Status: 'Enabled', Prefix: '', N

Re: [ceph-users] RGW lifecycle not expiring objects

On 06/28/2017 02:30 PM, Graham Allan wrote: That seems to be it! I couldn't see a way to specify the auth version with aws cli (is there a way?). However it did work with s3cmd and v2 auth: % s3cmd --signature-v2 setlifecycle lifecycle.xml s3://testgta s3://testgta/: Lifecycle Policy updated G

Re: [ceph-users] Problems getting nfs-ganesha with cephfs backend to work.

On 07/19/2017 05:27 AM, Micha Krause wrote: Hi, Ganesha version 2.5.0.1 from the nfs-ganesha repo hosted on download.ceph.com I didn't know about that repo, and compiled ganesha myself. The developers in the #ganesha IRC channel pointed me to the libcephfs version

Re: [ceph-users] Kraken rgw lifeycle processing nightly crash

On 07/20/2017 04:48 PM, Ben Hines wrote: Still having this RGWLC crash once a day or so. I do plan to update to Luminous as soon as that is final, but it's possible this issue will still occur, so i was hoping one of the devs could take a look at it. My original suspicion was that it happens w

Re: [ceph-users] Ganesha NFS

Hi. NFSv3 is a bit different than v4. In the case of v3, you need to mount the full path, rather than the Pseudo path. This can cause problems for cephfs, because you probably exported / from cephfs. A good solution to this is to set mount_path_pseudo = true; in the NFS_CORE_PARAM block.

Re: [ceph-users] Ceph luminous nfs-ganesha-ceph

On 12/14/2017 09:46 AM, nigel davies wrote: Is this nfs-ganesha exporting Cephfs? Yes Are you using NFS for a Vmware Datastore? Yes What are you using for the NFS failover? (this is where i could be going wrong) When creating the NFS Datastore i added the two NFS servers ip address in NFS

Re: [ceph-users] Simple RGW Lifecycle processing questions (luminous 12.2.2)

On 12/19/2017 07:50 PM, Bryan Banister wrote: Hi All, Hey all, How often does the "lc process" run on RGW buckets in a cluster? Also is it configurable per bucket or anything? Tried searching man pages and ceph docs with no luck, so any help appreciated! This is controlled by the rgw_lif

Re: [ceph-users] nfs-ganesha rpm build script has not been adapted for this -

This was fixed on next (for 2.6, currently in -rc1) but not backported to 2.5. Daniel On 01/09/2018 12:41 PM, Marc Roos wrote: The script has not been adapted for this - at the end http://download.ceph.com/nfs-ganesha/rpm-V2.5-stable/luminous/x86_64/ nfs-ganesha-rgw-2.5.4-.el7.x86_64.rp

Re: [ceph-users] object lifecycle scope

Smallest scope is per-bucket. Daniel On 02/06/2018 02:24 PM, Robert Stanford wrote:  Hello Ceph users.  Is object lifecycle (currently expiration) for rgw implementable on a per-object basis, or is the smallest scope the bucket? ___ ceph-users

Re: [ceph-users] UID reset to root after chgrp on CephFS Ganesha export

I believe this is a Ganesha bug, as discussed on the Ganesha list. Daniel On 08/31/2016 06:55 AM, Wido den Hollander wrote: Op 31 augustus 2016 om 12:42 schreef John Spray : On Wed, Aug 31, 2016 at 11:23 AM, Wido den Hollander wrote: Hi, I have a CephFS filesystem which is re-exported th

Re: [ceph-users] UID reset to root after chgrp on CephFS Ganesha export

On 08/31/2016 02:15 PM, Wido den Hollander wrote: Op 31 augustus 2016 om 15:28 schreef Daniel Gryniewicz : I believe this is a Ganesha bug, as discussed on the Ganesha list. Ah, thanks. Do you maybe have a link or subject so I can chime in? Wido https://sourceforge.net/p/nfs-ganesha

Re: [ceph-users] radosgw - http status 400 while creating a bucket

Your RGW doesn't think it's the master, and cannot connect to the master, thus the create fails. Daniel On 11/08/2016 06:36 PM, Andrei Mikhailovsky wrote: Hello I am having issues with creating buckets in radosgw. It started with an upgrade to version 10.2.x When I am creating a bucket I get

Re: [ceph-users] Ceph and container

In addition, Red Hat is shipping a containerized Ceph (all daemons, not just mons) as a tech preview in RHCS, and the plan is to support it going forward. We have not seen performance issues related to being containerized. It's based on the ceph-docker and ceph-ansible projects. Daniel On 1

Re: [ceph-users] How exactly does rgw work?

Yes, this is common practice. Daniel On 12/22/2016 02:34 PM, Gerald Spencer wrote: Wonderful, just as I expected. Do folks normally have several RGW running on individual machines with a load balancer at larger scales? On Wed, Dec 21, 2016 at 8:22 AM, LOPEZ Jean-Charles mailto:jelo...@redhat.c

Re: [ceph-users] Secure way to wipe a Ceph cluster

On 07/27/2018 03:03 AM, Robert Sander wrote: Hi, On 27.07.2018 09:00, Christopher Kunz wrote: as part of deprovisioning customers, we regularly have the task of wiping their Ceph clusters. Is there a certifiable, GDPR compliant way to do so without physically shredding the disks? In the past

Re: [ceph-users] Cephfs fsal + nfs-ganesha + el7/centos7

To my knowledge, no one has done any work on ganesha + ceph and selinux. Fedora (and RHEL) includes config in it's selinux package for ganesha + gluster, but I'm sure there's missing bits for ceph. Daniel On 02/17/2018 03:15 PM, Oliver Freyermuth wrote: Hi together, many thanks for the RPMs

Re: [ceph-users] CephFS MDS stuck (failed to rdlock when getattr / lookup)

On 04/17/2018 11:40 AM, Oliver Freyermuth wrote: Am 17.04.2018 um 17:34 schrieb Paul Emmerich: [...] We are right now using the packages from https://eu.ceph.com/nfs-ganesha/ since we would like not to have to build NFS Ganesha against Ceph ourselves

Re: [ceph-users] CephFS MDS stuck (failed to rdlock when getattr / lookup)

On 05/01/2018 01:43 PM, Oliver Freyermuth wrote: Hi all, Am 17.04.2018 um 19:38 schrieb Oliver Freyermuth: Am 17.04.2018 um 19:35 schrieb Daniel Gryniewicz: On 04/17/2018 11:40 AM, Oliver Freyermuth wrote: Am 17.04.2018 um 17:34 schrieb Paul Emmerich: [...] We are right now using

Re: [ceph-users] RGW authentication fail with AWS S3 v4

It looks like, as it's now coded, the 15 minute time limit is hard coded. It checks that X-Amz-Expires is not exceeded, and then unconditionally checks that the request time is within 15 minutes of now. Daniel On 02/03/2017 04:06 AM, Khang Nguyễn Nhật wrote: Dear Wido, I have used X-Amz

Re: [ceph-users] Kraken release and RGW --> "S3 bucket lifecycle API has been added. Note that currently it only supports object expiration."

On 03/27/2017 04:28 PM, ceph.nov...@habmalnefrage.de wrote: Hi Cephers. Couldn't find any special documentation about the "S3 object expiration" so I assume it should work "AWS S3 like" (?!?) ... BUT ... we have a test cluster based on 11.2.0 - Kraken and I set some object expiration dates via

Re: [ceph-users] RGW lifecycle not expiring objects

Kraken has lifecycle, Jewel does not. Daniel On 06/04/2017 07:16 PM, ceph.nov...@habmalnefrage.de wrote: grrr... sorry && and again as text :| Gesendet: Montag, 05. Juni 2017 um 01:12 Uhr Von: ceph.nov...@habmalnefrage.de An: "Yehuda Sadeh-Weinraub" Cc: "ceph-users@lists.ceph.com" , ceph-d

Re: [ceph-users] ceph RGW NFS

On 02/28/2016 08:36 PM, David Wang wrote: Hi All, How the progress of NFS on RGW? Does it released on Infernalis? The contents of NFS on RGW is http://tracker.ceph.com/projects/ceph/wiki/RGW_-_NFS The FSAL has been integrated into upstream Ganesha (https://github.com/nfs-ganesha/nfs-gan

Re: [ceph-users] How to troubleshoot rsync to cephfs via nfs-ganesha stalling

This isn't something I've seen before. rsync generally works fine, even over cephfs. More inline. On 12/09/2018 09:42 AM, Marc Roos wrote: This rsync command fails and makes the local nfs unavailable (Have to stop nfs-ganesha, kill all rsync processes on the client and then start nfs-ganesh

Re: [ceph-users] How to troubleshoot rsync to cephfs via nfs-ganesha stalling

----- From: Daniel Gryniewicz [mailto:d...@redhat.com] Sent: 10 December 2018 15:54 To: ceph-users@lists.ceph.com Subject: Re: [ceph-users] How to troubleshoot rsync to cephfs via nfs-ganesha stalling This isn't something I've seen before. rsync generally works fine, even over cephfs. Mo

Re: [ceph-users] CEPH_FSAL Nfs-ganesha

Hi. Welcome to the community. On 01/14/2019 07:56 AM, David C wrote: Hi All I've been playing around with the nfs-ganesha 2.7 exporting a cephfs filesystem, it seems to be working pretty well so far. A few questions: 1) The docs say " For each NFS-Ganesha export, FSAL_CEPH uses a libcephfs

Re: [ceph-users] Deploy Cehp in multisite setup

On 3/5/19 2:15 PM, Paul Emmerich wrote: Choose two: * POSIX filesystem with a reliable storage underneath * Multiple sites with poor or high-latency connection between them * Performance If you can get away with S3/Object access, rather than POSIX FS, you could use the RadosGW from Ceph. It

Re: [ceph-users] Ceph expansion/deploy via ansible

On 4/17/19 4:24 AM, John Molefe wrote: Hi everyone, I currently have a ceph cluster running on SUSE and I have an expansion project that I will be starting with around June. Has anybody here deployed (from scratch) or expanded their ceph cluster via ansible?? I would appreciate it if you'd sha

Re: [ceph-users] RGW Beast frontend and ipv6 options

Based on past experience with this issue in other projects, I would propose this: 1. By default (rgw frontends=beast), we should bind to both IPv4 and IPv6, if available. 2. Just specifying port (rgw frontends=beast port=8000) should apply to both IPv4 and IPv6, if available. 3. If the use

Re: [ceph-users] RGW Beast frontend and ipv6 options

t we only use configured endpoints, but port= is now implicit endpoint configuration. Daniel On 5/2/19 10:08 AM, Daniel Gryniewicz wrote: Based on past experience with this issue in other projects, I would propose this: 1. By default (rgw frontends=beast), we should bind to both IPv4 and IPv6

Re: [ceph-users] Ceph nfs ganesha exports

User credentials in NFS are in the form of UID/GID pairs, just like in POSIX. These creds are specified by the NFS client, possibly modified by the server using NFS user mapping, and then passed down to CephFS by Ganesha. Whether or not the client user has permissions to do a particular op (f

Re: [ceph-users] NFS

Ganesha can export CephFS or RGW. It cannot export anything else (like iscsi or RBD). Config for RGW looks like this: EXPORT { Export_ID=1; Path = "/"; Pseudo = "/rgw"; Access_Type = RW; Protocols = 4; Transports = TCP; FSAL {

Re: [ceph-users] NFS

> as a NAS backend is new to me ( thus I might be over thinking this ) :) > > -Brent > > -Original Message- > From: Daniel Gryniewicz > Sent: Tuesday, October 1, 2019 8:20 AM > To: Marc Roos ; bkennedy ; > ceph-users > Subject: Re: [ceph-users] NFS > &

Re: [ceph-users] NFS

"Path" is either "/" to indicate the top of the tree, or a bucket name to indicate a limited export for a single bucket. It's not related to the user at all. On Thu, Oct 3, 2019 at 10:34 AM Marc Roos wrote: > > > How should a multi tenant RGW config look like, I am not able get this > working: >

Re: [ceph-users] [Nfs-ganesha-devel] 2.7.3 with CEPH_FSAL Crashing

_channel(cluster) log [WRN] : slow request 32.164260 seconds old, received at 2019-07-15 15:06:47.459980: client_request(client.16129440:1072326 getattr pAsLsXsFs #0x1000907443b 2019-07-15 15:00:21.118372 caller_uid=1161, caller_gid=1131{1131,4121,2330,2683,4115,2322,2779,2979,1503,3511,2783,2707,2942,2980,2258,2829,1238,1237,2793,1235,1249,2097,1154,2982,2983,3860,4101,1208,3638,3641,3644,3640,3643,3639,