Re: [ceph-users] MDS cluster degraded after upgrade to dumpling

Churchill wrote: > I've uploaded the complete log[0]. It's about 70MB just as a warning. > > [0] damoxc.net/ceph-mds.ceph2.log.1.gz > > On 21 August 2013 07:07, Gregory Farnum wrote: >> Do you have full logs from the beginning of replay? I believe you >>

Re: [ceph-users] librados pthread_create failure

On Mon, Aug 26, 2013 at 9:24 AM, Greg Poirier wrote: > So, in doing some testing last week, I believe I managed to exhaust the > number of threads available to nova-compute last week. After some > investigation, I found the pthread_create failure and increased nproc for > our Nova user to, what I

Re: [ceph-users] Sequential placement

In addition to that, Ceph uses full data journaling — if you have two journals on the OS drive then you'll be limited to what that OS drive can provide, divided by two (if you have two-copy happening). -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Mon, Aug 26, 2013 at 2:09

Re: [ceph-users] Reading from replica

Read-from-replica does not happen unless you go through some contortions with config and developer setups. However, all n OSDs should be the primary for about 1/n% of the data, so you should be seeing reads to to both OSDs as long as you touch several objects at a time. -Greg On Wednesday, August

Re: [ceph-users] Reading from replica

[ Please keep list discussions on the list. :) ] On Wed, Aug 28, 2013 at 12:54 PM, daniel pol wrote: > Hi ! > > Any pointers to where I can find the contortions ? You don't really want to — read-from-replica isn't safe except in very specific circumstances. > I agree with you on > should be see

Re: [ceph-users] Reading from replica

On Wed, Aug 28, 2013 at 1:22 PM, daniel pol wrote: > Sorry, my bad. Only my second post and forgot the "reply all" > > Thanks for the info. I'm looking at the impact of pg number on performance. > Just trying to learn more about how Ceph works. > I didn't set pgp_num. It came by default with 2 in

Re: [ceph-users] Reading from replica

On Wednesday, August 28, 2013, daniel pol wrote: > I don't recall. It might be that I started with PG=2. > Trying get an even distribution of PGs accros my 2 OSDs now. Tried > different numbers (keeping pgp_num same as pg_num :) but I keep getting one > OSD with more PGs compared to the other. Sin

Re: [ceph-users] Issues setting up ceph object storage

On Wednesday, August 28, 2013, Gerd Jakobovitsch wrote: > Hello all, > > I am trying to install a ceph environment for testing, focused on block > devices for virtualization and object storage. I am facing some issues. > > My environment consists of one server running ceph-deploy and mon; 3 > clus

Re: [ceph-users] rbd striping

On Thursday, August 29, 2013, Corin Langosch wrote: > Hi there, > > I read about how striping of rbd works at http://ceph.com/docs/next/man/** > 8/rbd/ and it seems rather complex > to me. As the individual objects are placed randomly over all osds taking > c

Re: [ceph-users] Upgraded Bobtail to Cuttlefish and unable to mount cephfs

Can you start up your mds with "dedug mds = 20" and "debug ms = 20"? The "failed to decode message" line is suspicious but there's not enough context here for me to be sure, and my pattern-matching isn't reminding me of any serious bugs. -Greg Software Engineer #42 @ http://inktank.com | http://cep

Re: [ceph-users] OSD to OSD Communication

Assuming the networks can intercommunicate, yes. -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Fri, Aug 30, 2013 at 1:09 PM, Geraint Jones wrote: > One Other thing > > If I set cluster_network on node0 and restart it, then do the same on > node1 will I be able to maintain

Re: [ceph-users] tons of "failed lossy con, dropping message" => root cause for bad performance ?

Looks like maybe your network is faulty. The crc error means the OSD received a message with a checksum that didn't match. The dropped message indicates that the connection (in this case to a client) has failed (probably because of the bad crc?) and so it's dropping the outgoing message. This is in

Re: [ceph-users] PG stuck in inconsistent state

On Tue, Sep 3, 2013 at 7:20 AM, Maciej Gałkiewicz wrote: > Hi > > I have recently discovered that one of my pg is in inconsistent state. I > have checked filesystem on osd.3, and re-run deep-scrub few times. Osd uses > xfs. Any suggestions how to fix it? You can use "repair" instead of "scrub" or

Re: [ceph-users] adding SSD only pool to existing ceph cluster

On Mon, Sep 2, 2013 at 5:09 AM, Jens-Christian Fischer wrote: > We have a ceph cluster with 64 OSD (3 TB SATA) disks on 10 servers, and run > an OpenStack cluster. > > We are planning to move the images of the running VM instances from the > physical machines to CephFS. Our plan is to add 10 SSDs

Re: [ceph-users] adding SSD only pool to existing ceph cluster

On Wed, Sep 4, 2013 at 2:24 AM, Jens-Christian Fischer wrote: > Hi Greg > >> If you saw your existing data migrate that means you changed its >> hierarchy somehow. It sounds like maybe you reorganized your existing >> nodes slightly, and that would certainly do it (although simply adding >> single

Re: [ceph-users] ceph install

On Fri, Aug 30, 2013 at 11:46 AM, Jimmy Lu [ Storage ] wrote: > Hello ceph-users, > > I am new to Ceph and would like to bring up a 5-node cluster for my PoC. I > am doing an installation from below link and ran into a problem. I am not > so sure how to deal with it. Can someone please shed some l

Re: [ceph-users] newbie question: rebooting the whole cluster, powerfailure

On Thu, Sep 5, 2013 at 9:31 AM, Alfredo Deza wrote: > On Thu, Sep 5, 2013 at 11:42 AM, Bernhard Glomm > wrote: >> >> Hi all, >> >> as a ceph newbie I got another question that is probably solved long ago. >> I have my testcluster consisting two OSDs that also host MONs >> plus one to five MONs. >

Re: [ceph-users] Remove HDD under OSD + rados request = not found

On Fri, Sep 6, 2013 at 12:06 AM, Mihály Árva-Tóth wrote: > Hello, > > I have a server with hot swappable SATA disks. When I remove HDD from a > working server, OSD does not noice missing of HDD. ceph healt status write > HEALTH_OK and all of OSD "in" and "up". When I run a swift client on another

Re: [ceph-users] CephFS no longer mounts and asserts in MDS after upgrade to 0.67.3

On Tue, Sep 10, 2013 at 10:54 AM, Oliver Daudey wrote: > Hey list, > > I just upgraded to Ceph 0.67.3. What I did on every node of my 3-node > cluster was: > - Unmount CephFS everywhere. > - Upgrade the Ceph-packages. > - Restart MON. > - Restart OSD. > - Restart MDS. > > As soon as I got to the

Re: [ceph-users] CephFS no longer mounts and asserts in MDS after upgrade to 0.67.3

M, Liu, Larry wrote: > This is scary. Should I hold on upgrade? > > On 9/10/13 11:33 AM, "Oliver Daudey" wrote: > >>Hey Gregory, >> >>On 10-09-13 20:21, Gregory Farnum wrote: >>> On Tue, Sep 10, 2013 at 10:54 AM, Oliver Daudey >>>wrote: >

Re: [ceph-users] CephFS no longer mounts and asserts in MDS after upgrade to 0.67.3

oftware Engineer #42 @ http://inktank.com | http://ceph.com On Tue, Sep 10, 2013 at 1:59 PM, Gregory Farnum wrote: > It's not an upgrade issue. There's an MDS object that is somehow > missing. If it exists, then on restart you'll be fine. > > Oliver, what is your general

Re: [ceph-users] CephFS no longer mounts and asserts in MDS after upgrade to 0.67.3

gt; see how I can scrub the PG for an object that doesn't appear to exist. > Please elaborate. > > >Regards, > > Oliver > > On di, 2013-09-10 at 14:06 -0700, Gregory Farnum wrote: >> Also, can you scrub the PG which contains the "mds_ancho

Re: [ceph-users] CephFS no longer mounts and asserts in MDS after upgrade to 0.67.3

#x27;d probably just shoot for that after doing a scrub. (If the scrub turns up something dirty then probably it can be recovered via a RADOS repair.) -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com >Regards, > > Oliver > > On di, 2013-09-10 at 13:

Re: [ceph-users] CephFS no longer mounts and asserts in MDS after upgrade to 0.67.3

dmap' > object 'mds_anchortable' -> 1.a7 -> [2,0] > # ceph pg scrub 1.a7 > > osd.2 logs: > 2013-09-11 00:41:15.843302 7faf56b1b700 0 log [INF] : 1.a7 scrub ok > > osd.0 didn't show anything in it's logs, though. Should I try a repair > next?

Re: [ceph-users] CephFS no longer mounts and asserts in MDS after upgrade to 0.67.3

On Wed, Sep 11, 2013 at 7:48 AM, Yan, Zheng wrote: > On Wed, Sep 11, 2013 at 10:06 PM, Oliver Daudey wrote: >> Hey Yan, >> >> On 11-09-13 15:12, Yan, Zheng wrote: >>> On Wed, Sep 11, 2013 at 7:51 PM, Oliver Daudey wrote: Hey Gregory, I wiped and re-created the MDS-cluster I just m

Re: [ceph-users] ocfs2 for OSDs?

On Wed, Sep 11, 2013 at 12:55 PM, David Disseldorp wrote: > Hi Sage, > > On Wed, 11 Sep 2013 09:18:13 -0700 (PDT) > Sage Weil wrote: > >> > REFLINKs (inode-based writeable snapshots) >> >> This is the one item on this list I see that the ceph-osds could take real >> advantage of; it would make ob

Re: [ceph-users] librados vs libcephfs performance for database broker

On Fri, Sep 6, 2013 at 2:08 AM, Serge Slipchenko wrote: > Hi, > > I am setting up a cluster that is using Hypertable as one of the key > components. > This had required some fixes of CephBroker, which I hope would be integrated > to the main Hypertable branch soon. However, it seems to me that Cep

Re: [ceph-users] [Ceph] 10Gb network support

What command did you use to get those results? Have you tried increasing parallelism? What bandwidth do you have between that machine and your OSDs? Have you run lower-level tests on individual disks and nodes to make sure they're performing as you expect? -Greg Software Engineer #42 @ http://inkta

Re: [ceph-users] many report failed after mon election

I believe that's too high of an allowed skew with the default lease etc settings. The actual complaint is "I got a lease which has ALREADY expired and can't do anything with that!" You'll need to either get your clock skew down to less than, say, 1/4 second (which is perfectly doable over three no

Re: [ceph-users] mds stuck in rejoin

What's the output of "ceph -s", and have you tried running the MDS with any logging enabled that we can check out? -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Sun, Sep 15, 2013 at 8:24 AM, Serge Slipchenko wrote: > Hi, > > I'm testing ceph 0.67.3 (408cd61584c72c0d97b774

Re: [ceph-users] errors after kernel-upgrade -- Help needed

Obviously your OSDs aren't getting all the PGs up and running. Have you followed the troubleshooting steps? (http://ceph.com/docs/master/rados/troubleshooting/troubleshooting-pg/) -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Mon, Sep 16, 2013 at 6:35 AM, Markus Goldberg

Re: [ceph-users] Rugged data distribution on OSDs

What is your PG count and what's the output of "ceph osd tree"? It's possible that you've just got a slightly off distribution since there still isn't much data in the cluster (probabilistic placement and all that), but let's cover the basics first. -Greg Software Engineer #42 @ http://inktank.com

Re: [ceph-users] mds stuck in rejoin

: >> >> Hi Gregory, >> >> On Sun, Sep 15, 2013 at 10:59 PM, Gregory Farnum wrote: >>> >>> What's the output of "ceph -s", and have you tried running the MDS >>> with any logging enabled that we can check out? >> >>

Re: [ceph-users] radosgw bucket question

[Removed ceph-devel] RGW, like Amazon S3, has a global bucket namespace. You already have a mybucket1, so the second user can't create another one. I believe Yehuda said recently that there's some work happening on multi tenancy that will give each user their own namespace? -Greg On Monday, Septe

Re: [ceph-users] VM storage and OSD Ceph failures

The VM read will hang until a replica gets promoted and the VM resends the read. In a healthy cluster with default settings this will take about 15 seconds. -Greg On Tuesday, September 17, 2013, Gandalf Corvotempesta wrote: > Hi to all. > Let's assume a Ceph cluster used to store VM disk images.

Re: [ceph-users] Ceph performance with 8K blocks.

Your 8k-block dd test is not nearly the same as your 8k-block rados bench or SQL tests. Both rados bench and SQL require the write to be committed to disk before moving on to the next one; dd is simply writing into the page cache. So you're not going to get 460 or even 273MB/s with sync 8k writes r

Re: [ceph-users] Ceph performance with 8K blocks.

Oh, and you should run some local sync benchmarks against these drives to figure out what sort of performance they can deliver with two write streams going on, too. Sometimes the drives don't behave the way one would expect. -Greg On Tuesday, September 17, 2013, Gregory Farnum wrote: >

Re: [ceph-users] Rugged data distribution on OSDs

ious test when I remove HDD from a working > cluster, but I think this is not relevant now) > > root@stor3:~# ceph osd pool get .rgw.buckets pg_num > pg_num: 250 > root@stor3:~# ceph osd pool get .rgw.buckets pgp_num > pgp_num: 250 > > pgmap v129814: 514 pgs: 514 active; 818 GB d

Re: [ceph-users] Ceph performance with 8K blocks.

rom >> that. >> >> RADOS performance from what I've seen is largely going to hinge on replica >> size and journal location. Are your journals on separate disks or on the >> same disk as the OSD? What is the replica size of your pool? >> >>

Re: [ceph-users] About ceph testing

On Tue, Sep 17, 2013 at 10:07 PM, david zhang wrote: > Hi ceph-users, > > Previously I sent one mail to ask for help on ceph unit test and function > test. Thanks to one of your guys, I got replied about unit test. > > Since we are planning to use ceph, but with strict quality bar inside, we > hav

Re: [ceph-users] Index document for radosgw buckets?

What do you mean by index documents? Objects in each bucket are already kept in an index object; it's how we do listing and things. -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Tue, Sep 17, 2013 at 11:37 PM, Jeppesen, Nelson wrote: > Is there a way to enable index docume

Re: [ceph-users] ulimit max user processes (-u) and non-root ceph clients

On Wed, Sep 18, 2013 at 6:33 AM, Dan Van Der Ster wrote: > Hi, > We just finished debugging a problem with RBD-backed Glance image creation > failures, and thought our workaround would be useful for others. Basically, > we found that during an image upload, librbd on the glance api server was >

Re: [ceph-users] PG distribution scattered

It will not lose any of your data. But it will try and move pretty much all of it, which will probably send performance down the toilet. -Greg On Thursday, September 19, 2013, Mark Nelson wrote: > Honestly I don't remember, but I would be wary if it's not a test system. > :) > > Mark > > On 09/19

Re: [ceph-users] ulimit max user processes (-u) and non-root ceph clients

On Wed, Sep 18, 2013 at 11:43 PM, Dan Van Der Ster wrote: > > On Sep 18, 2013, at 11:50 PM, Gregory Farnum > wrote: > >> On Wed, Sep 18, 2013 at 6:33 AM, Dan Van Der Ster >> wrote: >>> Hi, >>> We just finished debugging a problem with RBD-backed Glance

Re: [ceph-users] ceph-osd leak

On Fri, Sep 20, 2013 at 6:40 AM, Serge Slipchenko wrote: > Hi, > > I'm using CephFS 0.67.3 as a backend for Hypertable and ElasticSearch. > Active reading/writing to the cephfs causes uncontrolled OSD memory growth > and at the final stage swapping and server unavailability. What kind of memory g

Re: [ceph-users] ceph-osd leak

[ Re-added the list — please keep emails on there so everybody can benefit! ] On Fri, Sep 20, 2013 at 12:24 PM, Serge Slipchenko wrote: > > > > On Fri, Sep 20, 2013 at 5:59 PM, Gregory Farnum wrote: >> >> On Fri, Sep 20, 2013 at 6:40 AM, Serge Slipchenko >> wrote

Re: [ceph-users] Question about Ceph performance

On Sun, Sep 22, 2013 at 2:35 AM, Dafan Dong wrote: > > > Hi folks, I am Dafan from Yahoo! corp. We are really interested in Ceph now. > I wish to know where I can get some performance report about new released > DUMPLING? Like throughput, latency with different cluster scale and hardware > type?

Re: [ceph-users] clients in cluster network?

On Tue, Sep 24, 2013 at 1:14 AM, Kurt Bauer wrote: > > > John Wilkins schrieb: > > Clients use the public network. The cluster network is principally for > OSD-to-OSD communication--heartbeats, replication, backfill, etc. > > Hmm, well, I'm aware of this, but the question is, if it is nevertheless

Re: [ceph-users] ceph-osd leak

On Sun, Sep 22, 2013 at 10:00 AM, Serge Slipchenko wrote: > On Fri, Sep 20, 2013 at 11:44 PM, Gregory Farnum wrote: >> >> [ Re-added the list — please keep emails on there so everybody can >> benefit! ] >> >> On Fri, Sep 20, 2013 at 12:24 PM, Serge Slipchenko

Re: [ceph-users] Full OSD questions

On Sun, Sep 22, 2013 at 5:25 AM, Gaylord Holder wrote: > > > On 09/22/2013 02:12 AM, yy-nm wrote: >> >> On 2013/9/10 6:38, Gaylord Holder wrote: >>> >>> Indeed, that pool was created with the default 8 pg_nums. >>> >>> 8 pg_num * 2T/OSD / 2 repl ~ 8TB which about how far I got. >>> >>> I bumped up

Re: [ceph-users] few port per ceph-osd

On Sat, Sep 21, 2013 at 11:05 PM, yy-nm wrote: > On 2013/9/10 4:57, Samuel Just wrote: >> >> That's normal, each osd listens on a few different ports for different >> reasons. >> -Sam >> >> On Mon, Sep 9, 2013 at 12:27 AM, Timofey Koolin wrote: >>> >>> I use ceph 0.67.2. >>> When I start >>> ceph

Re: [ceph-users] Question regarding plugin class

On Wed, Sep 25, 2013 at 6:40 AM, Chen, Ching-Cheng (KFRM 1) wrote: > Hi: > > > > I have question regarding using the class plugin API. > > > > We finally able to make a test plugin class worked. We was able to invoke > the exec() call and execute our test plugin class successfully. > > > > Howev

Re: [ceph-users] CephFS Pool Specification?

On Fri, Sep 27, 2013 at 7:10 AM, Aronesty, Erik wrote: > Ø You can also create additional data pools and map directories to them, > but > > > this probably isn't what you need (yet). > > Is there a link to a web page where you can read how to map a directory to a > pool? (I googled ceph map dire

Re: [ceph-users] Can't mount CephFS - where to start troubleshooting?

On Fri, Sep 27, 2013 at 2:12 PM, Aaron Ten Clay wrote: > Hi, > > I probably did something wrong setting up my cluster with 0.67.3. I > previously built a cluster with 0.61 and everything went well, even after an > upgrade to 0.67.3. Now I built a fresh 0.67.3 cluster and when I try to > mount Ceph

Re: [ceph-users] Can't mount CephFS - where to start troubleshooting?

On Sat, Sep 28, 2013 at 12:05 AM, Aaron Ten Clay wrote: > Okay, looks like I was a bit hasty in my examination of log files. I > discovered this line in the monitor log: > > 2013-09-27 23:54:25.089967 7fe04e454700 1 mon.chekov@1(peon).auth v15 > unknown.0 10.42.6.159:0/2985411387 supports cephx b

Re: [ceph-users] Poor performance with three nodes

On Wed, Oct 2, 2013 at 1:59 PM, Eric Lee Green wrote: > I have three storage servers that provide NFS and iSCSI services to my > network, which serve data to four virtual machine compute hosts (two ESXi, > two libvirt/kvm) with several dozen virtual machines . I decided to test out > a Ceph deploy

Re: [ceph-users] Ceph with high disk densities?

On Mon, Oct 7, 2013 at 9:15 AM, Scott Devoid wrote: > I brought this up within the context of the RAID discussion, but it did not > garner any responses. [1] > > In our small test deployments (160 HDs and OSDs across 20 machines) our > performance is quickly bounded by CPU and memory overhead. The

Re: [ceph-users] Client Timeout on Rados Gateway

The ping tests you're running are connecting to different interfaces (10.23.37.175) than those you specify in the "mon_hosts" option (10.0.0.2, 10.0.0.3, 10.0.0.4). The client needs to be able to connect to the specified address; I'm guessing it's not routable from outside that network? The error

Re: [ceph-users] Client Timeout on Rados Gateway

t; ceph@cephclient01:~/cos$ rados --version > ceph version 0.67.3 (408cd61584c72c0d97b774b3d8f95c6b1b06341a) > > ceph@cephclient01:~/cos$ uname -a > Linux cephclient01 3.6.10-030610-generic #201212101650 SMP Mon Dec 10 > 21:51:40 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux > > Thanks, > Joe > >>

Re: [ceph-users] Client Timeout on Rados Gateway

On Mon, Oct 7, 2013 at 2:40 PM, Gruher, Joseph R wrote: > Could you clarify something for me... I have a cluster network (10.0.0.x) and > a public network (10.23.37.x). All the Ceph machines have one interface on > each network and clients (when configured normally) would only be on the > publ

Re: [ceph-users] How many rbds can you map?

I believe this is a result of how we used the kernel interfaces (allocating a major device ID for each RBD volume), and some kernel limits (only 8 bits for storing major device IDs, and some used for other purposes). See http://tracker.ceph.com/issues/5048 I believe we have discussed not using a m

Re: [ceph-users] increase # of mons and use haproxy to increase ceph performance?

On Tue, Oct 8, 2013 at 2:35 PM, Snider, Tim wrote: > Does inceasing the number of monitors affect Ceph cluster performance > (everything else remaining ==)? If it does I hope it’s positive. In general it won't affect performance at all since the monitors are out of the data path. If you managed

Re: [ceph-users] Dumpling ceph.conf looks different

Previously the Ceph startup scripts required an enumeration of the daemons in the ceph.conf in order to start them. We've been doing a lot of incremental work since last October or so to make the system do more self-bootstrapping, and by the time we released Dumpling that got far enough to be used

Re: [ceph-users] Dumpling ceph.conf looks different

ady assumed so. > But how do the OSD's then find their mon's? I believe this again has to be in > ceph.conf, right? > > wogri > -- > http://www.wogri.at > > On Oct 9, 2013, at 21:36 , Gregory Farnum wrote: > >> Previously the Ceph startup scripts re

Re: [ceph-users] Dumpling ceph.conf looks different

On Thursday, October 10, 2013, su kucherova wrote: > So let me understand again. > Ceph doesnt need [osd]\[msd] details in ceph.conf. Ceph can detect it > automatically.(not sure how exactly i guess looking into /var/lib/ceph/osd > folder) > Yeah. Suppose I give it in ceph.conf can I still con

Re: [ceph-users] show warning when make source code of ceph

Yeah, that write return value warning is only in cuttlefish, but it shouldn't cause any trouble. The second one is still around, but it's not a code warning so much as gcc going "ack, that's a lot of work!" And it's not a problem, either. :) -Greg On Wednesday, October 9, 2013, yy-nm wrote: > war

Re: [ceph-users] kernel: [ 8773.432358] libceph: osd1 192.168.0.131:6803 socket error on read

(Sorry for the delayed response, this was in my spam folder!) Has this issue persisted? Are you using the stock 13.04 kernel? Can you describe your setup a little more clearly? It sounds like maybe you're using CephFS now and were using rbd before; is that right? What data did you move, when, and

Re: [ceph-users] recovering from unhealthy state

On Wed, Oct 9, 2013 at 10:19 PM, Kees Bos wrote: > Hi, > > > I've managed to get cepth in a unhealthy state, from which it will not > recover automatically. I've done some 'ceph osd out X' and stopped > ceph-osd processes before the rebalancing was completed. (All in a test > environment :-) ) > >

Re: [ceph-users] recovering from unhealthy state

On Thu, Oct 10, 2013 at 10:40 AM, Kees Bos wrote: > On Thu, 2013-10-10 at 18:24 +0200, Gregory Farnum wrote: >> On Wed, Oct 9, 2013 at 10:19 PM, Kees Bos wrote: >> > Hi, >> > >> > >> > I've managed to get cepth in a unhealthy state, from which

Re: [ceph-users] kernel: [ 8773.432358] libceph: osd1 192.168.0.131:6803 socket error on read

And it works > like a charm. So for now I'm sticking with fuse :) > > Let me know if you want me to do some explicit testing. It may take some > time for me to do them as I'm using ceph but I can manage to have some time > for maintenances. > > Regards, > > >

Re: [ceph-users] Issue with OSDs not starting

On Fri, Oct 11, 2013 at 2:49 AM, wrote: > Hi > > > > I am installing Ceph using the chef cookbook recipes and I am having an > issue with ceph-osd-all-starter > > > > Here’s a dump from client.log > > > > > > Error e

Re: [ceph-users] Problems with RadosGW bench

Without more details it sounds like you're just overloading the cluster. How are the clients generating their load — is there any throttling? 4 gateways can probably process on the order of 15k ops/second; each of those PUT ops is going to require 3 writes to the disks on the backend (times whateve

Re: [ceph-users] SSD pool write performance

On Thu, Oct 10, 2013 at 12:47 PM, Sergey Pimkov wrote: > Hello! > > I'm testing small CEPH pool consists of some SSD drives (without any > spinners). Ceph version is 0.67.4. Seems like write performance of this > configuration is not so good as possible, when I testing it with small block > size

Re: [ceph-users] snapshots on CephFS

On Wed, Oct 16, 2013 at 8:01 AM, Kasper Dieter wrote: > Hi Greg, > > on http://comments.gmane.org/gmane.comp.file-systems.ceph.user/1705 > I found a statement from you regarding snapshots on cephfs: > > ---snip--- > Filesystem snapshots exist and you can experiment with them on CephFS > (there's a

Re: [ceph-users] Distribution of performance under load.

On Thu, Oct 17, 2013 at 6:19 AM, Robert van Leeuwen wrote: > Hi, > > I'm wondering how Ceph behaves when there are multiple sources writing > heavily to the same pool (e.g. Openstack nova compute) > Will each get its own "fair share" or will a very heavy user impact all > others? Ceph doesn't d

Re: [ceph-users] (no subject)

On Thu, Oct 17, 2013 at 12:40 PM, wrote: > I'd like to experiment with the ceph class methods technology. I've looked > at the cls_hello sample but I'm having trouble figuring out how to compile, > like and install. Are there any step-by-step documents on how to compile, > link and deploy the m

Re: [ceph-users] (no subject)

tial and protected from disclosure. If you are not the intended > recipient, any review, dissemination, distribution or copying is strictly > prohibited. If you have received this email message in error, please notify > the sender by reply email and delete the message and any attach

Re: [ceph-users] mounting RBD in linux containers

What kernel are you running, and which format is the RBD image? I thought we had a special return code for when the kernel doesn't support the features used by that image, but that could be the problem. -Greg On Thursday, October 17, 2013, Omar Marquez wrote: > > Strace produces below: > > … > >

Re: [ceph-users] Application HA & Scalability Via ceph

On Fri, Oct 18, 2013 at 12:23 PM, Noah Watkins wrote: > On Fri, Oct 18, 2013 at 12:04 PM, wrote: >> Hi all, >> >> Is this possible? >> Does it make sense? > > As far as constructing scriptable object interfaces (Java, LISP, > etc...) this is certainly possible, and pretty cool :) Currently we >

Re: [ceph-users] Rados bench result when increasing OSDs

On Mon, Oct 21, 2013 at 7:13 AM, Guang Yang wrote: > Dear ceph-users, > Recently I deployed a ceph cluster with RadosGW, from a small one (24 OSDs) > to a much bigger one (330 OSDs). > > When using rados bench to test the small cluster (24 OSDs), it showed the > average latency was around 3ms (o

Re: [ceph-users] Intermittent poor performance on 3 node cluster

On Mon, Oct 21, 2013 at 8:05 AM, Pieter Steyn wrote: > Hi all, > > I'm using Ceph as a filestore for my nginx web server, in order to have > shared storage, and redundancy with automatic failover. > > The cluster is not high spec, but given my use case (lots of images) - I am > very dissapointed w

Re: [ceph-users] SSD question

On Mon, Oct 21, 2013 at 7:05 PM, Martin Catudal wrote: > Hi, > I have purchase my hardware for my Ceph storage cluster but did not > open any of my 960GB SSD drive box since I need to answer my question first. > > Here's my hardware. > > THREE server Dual 6 core Xeon 2U capable with 8 hotswap

[ceph-users] CephFS & Project Manila (OpenStack)

Hey all, The OpenStack community has spawned a newish "Project Manila", an effort spearheaded by NetApp to provide a file-sharing service analogous to Cinder, but for filesystems instead of block devices. The elevator pitch: Isn't it great how OpenStack lets you manage block devices for your hosts?

Re: [ceph-users] saucy salamander support?

http://tracker.ceph.com/issues/6485 I don't believe it's in a release yet, but yes, that's the problem and it's fixed in the ceph-deploy source repo. :) -Greg On Wednesday, October 23, 2013, LaSalle, Jurvis wrote: > On 13/10/22 6:28 PM, "Dan Mick" > > wrote: > > >/etc/ceph should be installed by

Re: [ceph-users] CephFS & Project Manila (OpenStack)

On Wed, Oct 23, 2013 at 7:43 AM, Dimitri Maziuk wrote: > On 2013-10-22 22:41, Gregory Farnum wrote: > ... > >> Right now, unsurprisingly, the focus of the existing Manila developers >> is on Option 1: it's less work than the others and supports the most >> common

Re: [ceph-users] CephFS and clients [was: CephFS & Project Manila (OpenStack)]

On Wed, Oct 23, 2013 at 11:47 AM, Dimitri Maziuk wrote: > On 10/23/2013 12:53 PM, Gregory Farnum wrote: >> On Wed, Oct 23, 2013 at 7:43 AM, Dimitri Maziuk >> wrote: >>> On 2013-10-22 22:41, Gregory Farnum wrote: >>> ... >>> >>>> Right

Re: [ceph-users] CephFS and clients [was: CephFS & Project Manila (OpenStack)]

On Wed, Oct 23, 2013 at 1:28 PM, Dimitri Maziuk wrote: > On 10/23/2013 02:46 PM, Gregory Farnum wrote: > >> Ah, I see. No, each CephFS client needs to communicate with the whole >> cluster. Only the POSIX metadata changes flow through the MDS. > > Yeah, I thought you'

Re: [ceph-users] mds laggy or crashed

Looks like your journal has some bad events in it, probably due to bugs in the multi-MDS systems. Did you start out this cluster on 67.4, or has it been upgraded at some point? Why did you use two active MDS daemons? -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Mon, Oct 2

Re: [ceph-users] mds laggy or crashed

[ Adding back the list for archival and general edification. :) ] On Wed, Oct 23, 2013 at 5:53 PM, Gagandeep Arora wrote: > Hello Greg, > > mds was running fine for more than a month and last week on Thursday, we > created a snapshot to test the snapshot functionality of cephfs and the > snapshot

Re: [ceph-users] Rados bench result when increasing OSDs

On Thu, Oct 24, 2013 at 6:31 AM, Guang Yang wrote: > Hi Mark, Greg and Kyle, > Sorry to response this late, and thanks for providing the directions for me > to look at. > > We have exact the same setup for OSD, pool replica (and even I tried to > create the same number of PGs within the small clus

Re: [ceph-users] Nearly full OSDs with very little (apparent) FS utilization

Are you sure you're using only CephFS? Do you have any snapshots? -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Fri, Oct 25, 2013 at 2:59 AM, Miguel Afonso Oliveira wrote: > Hi, > > I have a recent ceph deployment with version: > > ceph version 0.67.4 (ad85b8bfafea6232d64

Re: [ceph-users] Seagate Kinetic

On Monday, October 28, 2013, wrote: > Not brand-new, but I've not seen it mentioned on here so far. Seagate > Kinetic essentially enables HDDs to present themselves directly over > Ethernet as Swift object storage: > > http://www.seagate.com/**solutions/cloud/data-center-** > cloud/platforms/?cmp

Re: [ceph-users] rbd on ubuntu 12.04 LTS

You've enabled some feature on your cluster which is not supported by that kernel client. It's probably the crush tunables (you can find info on them in the docs). -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Wed, Oct 30, 2013 at 3:59 AM, Fuchs, Andreas (SwissTXT) wrote:

Re: [ceph-users] radosgw - complete_multipart errors

On Thu, Oct 31, 2013 at 6:22 AM, Dominik Mostowiec wrote: > Hi, > I have strange radosgw error: > > == > 2013-10-26 21:18:29.844676 7f637beaf700 0 setting object > tag=_ZPeVs7d6W8GjU8qKr4dsilbGeo6NOgw > 2013-10-26 21:18:30.049588 7f637beaf700 0 WARNING: set_req_state_err > err_no=125 resorti

Re: [ceph-users] rbd on ubuntu 12.04 LTS

CentOS 6.4 kernel 3.8.0-32-generic > > Might this be an issue? > > > -Original Message- > > From: Gregory Farnum [mailto:g...@inktank.com ] > > Sent: Mittwoch, 30. Oktober 2013 23:57 > > To: Fuchs, Andreas (SwissTXT) > > Cc: Karan Singh; ceph-users@li

Re: [ceph-users] rbd on ubuntu 12.04 LTS

T) wrote: > Ok i halfway understand this. > > > > So I can either upgrade to a kernel version v3.9 or later > > Or > > Change the crushmap with ceph osd crush tunables {PROFILE} > > To a new profile. > > > > But to which profile do I have to change so that my u

Re: [ceph-users] "rbd map" says "bat option at rw"

I think this will be easier to help with if you provide the exact command you're running. :) -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Fri, Nov 1, 2013 at 3:07 AM, nicolasc wrote: > Hi every one, > > I finally and happily managed to get my Ceph cluster (3 monitors amo

Re: [ceph-users] rbd on ubuntu 12.04 LTS

On Mon, Nov 4, 2013 at 12:13 AM, Fuchs, Andreas (SwissTXT) wrote: > I tryed with: > ceph osd crush tunables default > ceph osd crush tunables argonaut > > while the command runs without error, I still get the feature set mismatch > error whe I try to mount > do I have to restart some service? Ah

Re: [ceph-users] rbd on ubuntu 12.04 LTS

t > step emit > } > > I'm I really looking in the right direction? > > > >> -Original Message- >> From: Gregory Farnum [mailto:g...@inktank.com] >> Sent: Montag, 4. November 2013 19:17 >> To: Fuchs, Andreas (SwissTXT) >> Cc: Karan Singh; cep

Re: [ceph-users] Pool without a name, how to remove it?

I don't remember how this has come up or been dealt with in the past, but I believe it has been. Have you tried just doing it via the ceph or rados CLI tools with an empty pool name? -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Tue, Nov 5, 2013 at 6:58 AM, Wido den Hollan

<    2   3   4   5   6   7   8   9   10   11   >