On Sat, Aug 2, 2008 at 9:45 PM, Padraig O'Sullivan <[EMAIL PROTECTED]> wrote: > On Tue, Jul 22, 2008 at 12:35 AM, Sage Weil <[EMAIL PROTECTED]> wrote: >> On Mon, 21 Jul 2008, Padraig O'Sullivan wrote: >>> You were right about the osds having trouble initializing. The output >>> I'm getting from the cmonctl commands looks like what you expected: >>> >>> # ./cmonctl mds stat >>> mon0 <- 'mds stat' >>> mon0 -> 'e2: 1 nodes: 1 up:creating' (0) >>> # ./cmonctl pg stat >>> mon0 <- 'pg stat' >>> mon0 -> 'v5: 1152 pgs: 1152 creating; 0 KB used, 0 KB / 0 KB free' (0) >>> # >>> >>> So there is definitely something up with btrfs here (I guess btrfs is >>> far from stable so I can't complain) since I have had no issues when >>> setting up a cluster with block devices. I have debugging enabled on >>> the osd daemons and have been having a look through their log files. I >>> have 2 storage nodes and in the log file for the first one (after some >>> initialization messages), I keep seeing the following lines repeated >>> over and over: >>> [...] >>> I'm not really sure what these messages mean in the log files as I'm >>> not that familiar with Ceph internally yet. Do you have any pointers >>> or ideas (based on this limited information) on what's going on here? >>> Or if you have any suggestions on other log files to look at? >> >> Since things behave with ebofs, a wild guess would be that you're not >> running cosd as root, and cosd isn't gracefully failing? And/or the >> transaction ioctls are failing gracefully. It's probably not btrfs's >> fault, either way. Can you email me the full osd output? >> > > I saw you submitted a patch today to the btrfs mailing list so I tried > this again after applying your patch to the latest unstable version of > btrfs. Just thought I'd let you know that everything works fine now. I > have not done any extensive testing with it but I have a small cluster > up and running. Thanks again! > > -Padraig
By the way, I also documented how I setup my small Ceph test cluster with btrfs. Its not pretty since its mostly for my purposes so I can remember what I did (I can easily copy and paste commands from the page) but I thought I would make it available in case anyone else finds it useful. Its available at the following location: http://www.ece.umd.edu/~posulliv/ceph/cluster_build.html -Padraig > >> sage >> > ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Ceph-devel mailing list Ceph-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ceph-devel
