Hi Sage
Good news, I have been able to resolve this issue. It requires the addition of
one line of code calling 'sync' to your script: /sbin/mkcephfs
At line 336, I have added in the block (see the comment and 'sync' line below
it):
modprobe btrfs || true
mkfs.btrfs $btrfs_devs
btrfs device scan || btrfsctl -a
#Mach: add 'sync' command below to force block changes, else btrfs mount
command is likely to fail
sync
mount -t btrfs $btrfs_opt $first_dev $btrfs_path
chown $osd_user $btrfs_path
chmod +w $btrfs_path
This has 100% resolved the issue for us.
Cheers,
Paul
-----Original Message-----
From: Sage Weil [mailto:[email protected]]
Sent: Wednesday, 11 July 2012 9:35 AM
To: Paul Pettigrew
Cc: [email protected]
Subject: RE: mkcephfs failing on v0.48 "argonaut"
Hi Paul,
Were you able to make any progress on this?
On Sun, 8 Jul 2012, Paul Pettigrew wrote:
> Hi Sage
>
> Confirming running the commands from a root prompt in the same sequence as
> requested:
> mkfs.btrfs /dev/sdc
> btrfs device scan
> mount /dev/sdc /srv/osd.0
>
>
> root@dsanb1-coy:~# mkfs.btrfs /dev/sdc
>
> WARNING! - Btrfs Btrfs v0.19 IS EXPERIMENTAL WARNING! - see
> http://btrfs.wiki.kernel.org before using
>
> fs created label (null) on /dev/sdc
> nodesize 4096 leafsize 4096 sectorsize 4096 size 1.82TB Btrfs
> Btrfs v0.19 root@dsanb1-coy:~# btrfs device scan Scanning for Btrfs
> filesystems root@dsanb1-coy:~# mount /dev/sdc /srv/osd.0
> mount: wrong fs type, bad option, bad superblock on /dev/sdc,
> missing codepage or helper program, or other error
> In some cases useful info is found in syslog - try
> dmesg | tail or so
>
> root@dsanb1-coy:~# mount | grep btrfs
> root@dsanb1-coy:~# mount -t btrfs /dev/sdc /srv/osd.0
> root@dsanb1-coy:~# mount | grep btrfs /dev/sdc on /srv/osd.0 type
> btrfs (rw)
>
>
> So - you can see I had to explicitly set the flag to mount "-t btrfs". It
> seems that when mkcepfs is running the command "mount -t btrfs -o noatime
> /dev/sdc /srv/osd.0" it is doing so with the effect that the "-t btrfs" was
> not present in the line, but it is (symptom the same at least). Crazy.
>
> Secondly, as requested I cannot run the command "sh -x /sbin/mkcephfs
> -d /tmp/mkcephfs.xgk025tjkQ --prepare-osdfs osd.0" because the
> contents of the /tmp/ directory are deleted each time mkcephfs
> finishes its run. I have however called the overall mkcephfs command
> with "sh -x" so you can see what is occurring.
Can you modify the mkcephfs command so that when it re-runs itself, it passes
in -x?
You can also remove the 'rm -r' bit that cleans up on error so that you can run
the command manually when it fails.
>
> See below:
>
> root@dsanb1-coy:~# sh -x /sbin/mkcephfs -c /etc/ceph/ceph.conf
> --allhosts --mkbtrfs -k /etc/ceph/keyring --crushmapsrc crushfile.txt
> -v <SNIP> === osd.0 ===
> + return 0
> + [ -n ]
> + rdir=/tmp/mkcephfs.kJjIwsEnfZ
> + [ 0 -eq 0 ]
> + cp /tmp/mkcephfs.kJjIwsEnfZ/conf /etc/ceph/ceph.conf [ 1 -eq 1 ] [
> + osd = osd ] do_root_cmd /sbin/mkcephfs -d /tmp/mkcephfs.kJjIwsEnfZ
> + --prepare-osdfs osd.0 [ -z ] [ 1 -eq 1 ] echo --- dsanb1-coy#
> + /sbin/mkcephfs -d /tmp/mkcephfs.kJjIwsEnfZ --prepare-osdfs osd.0
> --- dsanb1-coy# /sbin/mkcephfs -d /tmp/mkcephfs.kJjIwsEnfZ
> --prepare-osdfs osd.0
> + ulimit -c unlimited
> + whoami
> + whoami=root
> + [ root = root ]
> + bash -c /sbin/mkcephfs -d /tmp/mkcephfs.kJjIwsEnfZ --prepare-osdfs
> + osd.0
i.e., 'bash -c -x ...' here
> umount: /srv/osd.0: not mounted
> umount: /dev/sdc: not mounted
> /sbin/mkfs.btrfs
> RUNNING: mkfs.btrfs /dev/sdc
>
> WARNING! - Btrfs Btrfs v0.19 IS EXPERIMENTAL WARNING! - see
> http://btrfs.wiki.kernel.org before using
>
> fs created label (null) on /dev/sdc
> nodesize 4096 leafsize 4096 sectorsize 4096 size 1.82TB Btrfs
> Btrfs v0.19 Scanning for Btrfs filesystems /bin/mount
> RUNNING: mount -t btrfs -o noatime /dev/sdc /srv/osd.0
> mount: wrong fs type, bad option, bad superblock on /dev/sdc,
> missing codepage or helper program, or other error
> In some cases useful info is found in syslog - try
> dmesg | tail or so
>
> + echo failed: '/sbin/mkcephfs -d /tmp/mkcephfs.kJjIwsEnfZ --prepare-osdfs
> osd.0'
> failed: '/sbin/mkcephfs -d /tmp/mkcephfs.kJjIwsEnfZ --prepare-osdfs osd.0'
> + exit 1
> + rm -rf /tmp/mkcephfs.kJjIwsEnfZ
and comment out this command.
sage
> + exit
>
>
>
> -----Original Message-----
> From: Sage Weil [mailto:[email protected]]
> Sent: Saturday, 7 July 2012 2:20 PM
> To: Paul Pettigrew
> Cc: [email protected]
> Subject: RE: mkcephfs failing on v0.48 "argonaut"
>
> On Sat, 7 Jul 2012, Paul Pettigrew wrote:
> > Hi again Sage
> >
> > This is very perplexing. Confirming this system is a stock Ubuntu 12.04
> > x64, with no custom kernel or anything else, fully apt-get dist-upgrade'd
> > up to date.
> > root@dsanb1-coy:~# uname -r
> > 3.2.0-26-generic
> >
> > I have added in the suggestions you made to the script, we now have:
> >
> > modprobe btrfs || true
> > which mkfs.btrfs
> > echo "RUNNING: mkfs.btrfs $btrfs_devs"
> > mkfs.btrfs $btrfs_devs
> > btrfs device scan || btrfsctl -a which mount echo "RUNNING:
> > mount -t btrfs $btrfs_opt $first_dev $btrfs_path"
> > mount -t btrfs $btrfs_opt $first_dev $btrfs_path echo "DID I GET
> > HERE - OR CRASH OUT WITH mount ABOVE?"
> > chown $osd_user $btrfs_path
> >
> >
> > See below that the same command within the mkcephfs that is failing,
> > is working fine on a standard command line:
>
> Weirdness!
>
> >
> > === osd.0 ===
> > --- dsanb1-coy# /sbin/mkcephfs -d /tmp/mkcephfs.xgk025tjkQ
> > --prepare-osdfs osd.0
>
> Can you run this command with -x to see exactly what bash is doing?
>
> sh -x /sbin/mkcephfs -d /tmp/mkcephfs.xgk025tjkQ --prepare-osdfs
> osd.0
>
> In particular, I'm curious if you do
>
> mkfs.btrfs /dev/sdc
> btrfs device scan
> mount /dev/sdc /srv/osd.0
>
> (or whatever the exact sequence that mkcephfs does is) from the command line,
> does it give you the same error?
>
> sage
>
>
> > umount: /srv/osd.0: not mounted
> > umount: /dev/sdc: not mounted
> > /sbin/mkfs.btrfs
> > RUNNING: mkfs.btrfs /dev/sdc
> >
> > WARNING! - Btrfs Btrfs v0.19 IS EXPERIMENTAL WARNING! - see
> > http://btrfs.wiki.kernel.org before using
> >
> > fs created label (null) on /dev/sdc
> > nodesize 4096 leafsize 4096 sectorsize 4096 size 1.82TB
> > Btrfs Btrfs v0.19 Scanning for Btrfs filesystems /bin/mount
> > RUNNING: mount -t btrfs -o noatime /dev/sdc /srv/osd.0 /bin/mount
> > mount: wrong fs type, bad option, bad superblock on /dev/sdc,
> > missing codepage or helper program, or other error
> > In some cases useful info is found in syslog - try
> > dmesg | tail or so
> >
> > failed: '/sbin/mkcephfs -d /tmp/mkcephfs.xgk025tjkQ --prepare-osdfs osd.0'
> > root@dsanb1-coy:~# /bin/mount -t btrfs -o noatime /dev/sdc
> > /srv/osd.0 root@dsanb1-coy:~# mount | grep btrfs /dev/sdc on
> > /srv/osd.0 type btrfs (rw,noatime)
> >
> >
> > Remember, this is not isolated to btrfs, as per my original post it fails
> > when not specifying to use btrfs.
> >
> > I can only conclude that /bin/sh &/or /bin/bash and the way they interact
> > with the mkcephfs script, which does call itself etc, is somehow now become
> > fuddled up? Must be something wiggy, when the script output confirms it is
> > calling the same command ( /bin/mount ) but somehow finds a way for that to
> > not work and therefore cause the mkcephfs script terminate.
> >
> > Many thanks - will be a relief to sort this out, as all our Ceph project
> > works are on hold til we can sort this one out.
> >
> > Cheers
> >
> > Paul
> >
> >
> >
> > -----Original Message-----
> > From: Sage Weil [mailto:[email protected]]
> > Sent: Friday, 6 July 2012 2:09 PM
> > To: Paul Pettigrew
> > Cc: [email protected]
> > Subject: RE: mkcephfs failing on v0.48 "argonaut"
> >
> > On Fri, 6 Jul 2012, Paul Pettigrew wrote:
> > > Hi Sage - thanks so much for the quick response :-)
> > >
> > > Firstly, and it is a bit hard to see, but the command output below is run
> > > with the "-v" option. To help isolate what command line in the script is
> > > failing, I have added in some simple echo output, and the script now
> > > looks like:
> > >
> > >
> > > ### prepare-osdfs ###
> > >
> > > if [ -n "$prepareosdfs" ]; then
> > > <<SNIP>>
> > > modprobe btrfs || true
> > > echo "RUNNING: mkfs.btrfs $btrfs_devs"
> > > mkfs.btrfs $btrfs_devs
> > > btrfs device scan || btrfsctl -a echo "RUNNING: mount -t btrfs
> > > $btrfs_opt $first_dev $btrfs_path"
> > > mount -t btrfs $btrfs_opt $first_dev $btrfs_path echo "DID I
> > > GET HERE - OR CRASH OUT WITH mount ABOVE?"
> > > chown $osd_user $btrfs_path
> > > chmod +w $btrfs_path
> > >
> > > exit 0
> > > fi
> > >
> > > Per the modified script the above, here is the output displayed when
> > > running the script:
> > >
> > > root@dsanb1-coy:/srv# /sbin/mkcephfs -c /etc/ceph/ceph.conf
> > > --allhosts --mkbtrfs -k /etc/ceph/keyring --crushmapsrc
> > > crushfile.txt -v temp dir is /tmp/mkcephfs.uelzdJ82ej preparing
> > > monmap in /tmp/mkcephfs.uelzdJ82ej/monmap /usr/bin/monmaptool
> > > --create --clobber --add alpha 10.32.0.10:6789 --add bravo
> > > 10.32.0.25:6789 --add charlie
> > > 10.32.0.11:6789 --print /tmp/mkcephfs.uelzdJ82ej/monmap
> > > /usr/bin/monmaptool: monmap file /tmp/mkcephfs.uelzdJ82ej/monmap
> > > /usr/bin/monmaptool: generated fsid
> > > b254abdd-e036-4186-b6d5-e32b14e53b45
> > > epoch 0
> > > fsid b254abdd-e036-4186-b6d5-e32b14e53b45
> > > last_changed 2012-07-06 12:31:38.416848 created 2012-07-06
> > > 12:31:38.416848
> > > 0: 10.32.0.10:6789/0 mon.alpha
> > > 1: 10.32.0.11:6789/0 mon.charlie
> > > 2: 10.32.0.25:6789/0 mon.bravo
> > > /usr/bin/monmaptool: writing epoch 0 to
> > > /tmp/mkcephfs.uelzdJ82ej/monmap (3 monitors) /usr/bin/ceph-conf -c
> > > /etc/ceph/ceph.conf -n osd.0 "user"
> > > === osd.0 ===
> > > --- dsanb1-coy# /sbin/mkcephfs -d /tmp/mkcephfs.uelzdJ82ej
> > > --prepare-osdfs osd.0
> > > umount: /srv/osd.0: not mounted
> > > umount: /dev/sdc: not mounted
> > > RUNNING: mkfs.btrfs /dev/sdc
> > >
> > > WARNING! - Btrfs Btrfs v0.19 IS EXPERIMENTAL WARNING! - see
> > > http://btrfs.wiki.kernel.org before using
> > >
> > > fs created label (null) on /dev/sdc
> > > nodesize 4096 leafsize 4096 sectorsize 4096 size 1.82TB
> > > Btrfs Btrfs v0.19 Scanning for Btrfs filesystems
> > > RUNNING: mount -t btrfs -o noatime /dev/sdc /srv/osd.0
> > > mount: wrong fs type, bad option, bad superblock on /dev/sdc,
> > > missing codepage or helper program, or other error
> > > In some cases useful info is found in syslog - try
> > > dmesg | tail or so
> > >
> > > failed: '/sbin/mkcephfs -d /tmp/mkcephfs.uelzdJ82ej --prepare-osdfs osd.0'
> > >
> > >
> > > Which clearly isolates the issue to the "mount" command line.
> > >
> > > The trouble is, I can run this precise line on the command line directly
> > > without error:
> > >
> > > root@dsanb1-coy:/srv# mount -t btrfs -o noatime /dev/sdc
> > > /srv/osd.0 root@dsanb1-coy:/srv# mount | grep btrfs /dev/sdc on
> > > /srv/osd.0 type btrfs (rw,noatime)
> >
> > What if you run the exact sequence of commands that mkcephfs is doing?
> > (mkfs.btrfs, btrfs ..., mount ...). If that doesn't work, put `which
> > mkfs.btfs` etc in the script to make sure you're running the exact version
> > the script is...
> >
> > sage
> >
> >
> >
> > >
> > >
> > > Therefore, what could possibly be preventing the mkcephfs running a
> > > simple mount command on the first OSD disk it gets to, that otherwise
> > > works fine from the command line?
> > >
> > > Many thanks Sage
> > >
> > > Paul
> > >
> > > PS: changing the " btrfs device scan || btrfsctl -a" line as proposed had
> > > no effect, and neither did putting in a "sleep 10" immediately before the
> > > mount line.
> > > PPS: zerofilling the /dev/sdc and then re-creating a partition and
> > > mounting manually, then writing data to it is all fine. Same errors if we
> > > substitute any of the other HDD's in the server as 1st/osd.0. Ie, cannot
> > > see any issues with the hardware.
> > >
> > >
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: [email protected]
> > > [mailto:[email protected]] On Behalf Of Sage Weil
> > > Sent: Friday, 6 July 2012 8:18 AM
> > > To: Paul Pettigrew
> > > Cc: [email protected]
> > > Subject: Re: mkcephfs failing on v0.48 "argonaut"
> > >
> > > Hi Paul,
> > >
> > > On Wed, 4 Jul 2012, Paul Pettigrew wrote:
> > > > Firstly, well done guys on achieving this version milestone. I
> > > > successfully upgraded to the 0.48 format uneventfully on a live
> > > > (test) system.
> > > >
> > > > The same system was then going through "rebuild" testing, to
> > > > confirm that also worked fine.
> > > >
> > > >
> > > > Unfortunately, the mkcephfs command is failing:
> > > >
> > > > root@dsanb1-coy:~# mkcephfs -c /etc/ceph/ceph.conf --allhosts
> > > > --mkbtrfs -k /etc/ceph/keyring --crushmapsrc crushfile.txt -v
> > > > temp dir is /tmp/mkcephfs.GaRCZ9i06a preparing monmap in
> > > > /tmp/mkcephfs.GaRCZ9i06a/monmap /usr/bin/monmaptool --create
> > > > --clobber --add alpha 10.32.0.10:6789 --add bravo
> > > > 10.32.0.25:6789 --add charlie
> > > > 10.32.0.11:6789 --print /tmp/mkcephfs.GaRCZ9i06a/monmap
> > > > /usr/bin/monmaptool: monmap file /tmp/mkcephfs.GaRCZ9i06a/monmap
> > > > /usr/bin/monmaptool: generated fsid
> > > > c7202495-468c-4678-b678-115c3ee33402
> > > > epoch 0
> > > > fsid c7202495-468c-4678-b678-115c3ee33402
> > > > last_changed 2012-07-04 15:02:31.732275 created 2012-07-04
> > > > 15:02:31.732275
> > > > 0: 10.32.0.10:6789/0 mon.alpha
> > > > 1: 10.32.0.11:6789/0 mon.charlie
> > > > 2: 10.32.0.25:6789/0 mon.bravo
> > > > /usr/bin/monmaptool: writing epoch 0 to
> > > > /tmp/mkcephfs.GaRCZ9i06a/monmap (3 monitors) /usr/bin/ceph-conf -c
> > > > /etc/ceph/ceph.conf -n osd.0 "user"
> > > > === osd.0 ===
> > > > --- dsanb1-coy# /sbin/mkcephfs -d /tmp/mkcephfs.GaRCZ9i06a
> > > > --prepare-osdfs osd.0
> > > > umount: /srv/osd.0: not mounted
> > > > umount: /dev/disk/by-wwn/wwn-0x50014ee601246234: not mounted
> > > >
> > > > WARNING! - Btrfs Btrfs v0.19 IS EXPERIMENTAL WARNING! - see
> > > > http://btrfs.wiki.kernel.org before using
> > > >
> > > > fs created label (null) on /dev/disk/by-wwn/wwn-0x50014ee601246234
> > > > nodesize 4096 leafsize 4096 sectorsize 4096 size 1.82TB
> > > > Btrfs Btrfs v0.19 Scanning for Btrfs filesystems
> > > > mount: wrong fs type, bad option, bad superblock on /dev/sdc,
> > > > missing codepage or helper program, or other error
> > > > In some cases useful info is found in syslog - try
> > > > dmesg | tail or so
> > > >
> > > > failed: '/sbin/mkcephfs -d /tmp/mkcephfs.GaRCZ9i06a --prepare-osdfs
> > > > osd.0'
> > >
> > > Hmm. Can you try running with -v? That will tell us exactly which
> > > command it is running, and hopefully we can work backwards from there.
> > >
> > > > dmesg/syslog is spitting out at the time of this failure:
> > > >
> > > > Jul 4 15:02:31 dsanb1-coy kernel: [ 2306.751945] device fsid
> > > > 7de0d192-b710-4629-a201-849df1d9db17 devid 1 transid 27109
> > > > /dev/sdp Jul 4 15:02:31 dsanb1-coy kernel: [ 2306.751987]
> > > > device fsid 08fc3479-2fa2-4388-8b61-83e2a742a13e devid 1 transid
> > > > 28699 /dev/sdo Jul 4 15:02:31 dsanb1-coy kernel: [ 2306.752023]
> > > > device fsid
> > > > 8b4a7c43-1a05-4dcb-bbed-de2a5c933996 devid 1 transid 24346
> > > > /dev/sdn Jul 4 15:02:31 dsanb1-coy kernel: [ 2306.752068]
> > > > device fsid ba5fb1ca-c642-49b1-8a41-7f56f8e59fbd devid 1 transid
> > > > 27274 /dev/sdm Jul 4 15:02:31 dsanb1-coy kernel: [ 2306.761453]
> > > > device fsid 7fe8c5cf-bf8c-4276-90f2-c3f57f5275fb devid 1 transid
> > > > 28724 /dev/sdi Jul 4 15:02:31 dsanb1-coy kernel: [ 2306.761518]
> > > > device fsid 93fa3631-1202-4d42-8908-e5ef4d3e600d devid 1 transid
> > > > 25201 /dev/sdh Jul 4 15:02:31 dsanb1-coy kernel: [ 2306.761579]
> > > > device fsid b9a1b5e4-3e5e-4381-a29a-33470f4b870f devid 1 transid
> > > > 23375 /dev/sdg Jul 4 15:02:31 dsanb1-coy kernel: [ 2306.761635]
> > > > device fsid 280ea990-23f8-4c43-9e56-140c82340fdc devid 1 transid
> > > > 25559 /dev/sdf Jul 4 15:02:31 dsanb1-coy kernel: [ 2306.761693]
> > > > device fsid 2f724cde-6de5-4262-b195-1ba3eea2256e devid 1 transid
> > > > 176 /dev/sde Jul
> > > > 4 15:02:31 dsanb1-coy kernel: [ 2306.761732] device fsid
> > > > a66f890f-8b08-4393-aab0-f222637ca5a4 devid 1 transid 7 /dev/sdd
> > > > Jul
> > > > 4
> > > > 15:02:31 dsanb1-coy kernel: [ 2306.761769] device fsid
> > > > 6c181a94-697c-4e0c-af0d-05eb04d3626c devid 1 transid 7 /dev/sdc
> > > > Jul
> > > > 4
> > > > 15:02:31 dsanb1-coy kernel: [ 2306.775931] device fsid
> > > > 6c181a94-697c-4e0c-af0d-05eb04d3626c devid 1 transid 7 /dev/sdc
> > > > Jul
> > > > 4
> > > > 15:02:31 dsanb1-coy kernel: [ 2306.779716] btrfs bad fsid on
> > > > block
> > > > 20971520 Jul 4 15:02:31 dsanb1-coy kernel: [ 2306.791594] btrfs
> > > > bad fsid on block 20971520 Jul 4 15:02:31 dsanb1-coy kernel: [
> > > > 2306.803608] btrfs bad fsid on block 20971520 Jul 4 15:02:31
> > > > dsanb1-coy kernel: [ 2306.815541] btrfs bad fsid on block
> > > > 20971520 Jul
> > > > 4 15:02:31 dsanb1-coy kernel: [ 2306.815878] btrfs bad fsid on
> > > > block
> > > > 20971520 Jul 4 15:02:32 dsanb1-coy kernel: [ 2306.823554] btrfs
> > > > bad fsid on block 20971520 Jul 4 15:02:32 dsanb1-coy kernel: [
> > > > 2306.823797] btrfs bad fsid on block 20971520 Jul 4 15:02:32
> > > > dsanb1-coy kernel: [ 2306.823887] btrfs: failed to read chunk
> > > > root on sdc Jul 4 15:02:32 dsanb1-coy kernel: [ 2306.825622] btrfs:
> > > > open_ctree failed
> > >
> > > Long shot, but is the kernel on that machine recent?
> > >
> > > > Also fails if not forcing to use btrfs, eg:
> > > >
> > > > root@dsanb1-coy:~# mkcephfs -c /etc/ceph/ceph.conf --allhosts -k
> > > > /etc/ceph/keyring --crushmapsrc crushfile.txt -v temp dir is
> > > > /tmp/mkcephfs.ZOh6tBPAH0 preparing monmap in
> > > > /tmp/mkcephfs.ZOh6tBPAH0/monmap /usr/bin/monmaptool --create
> > > > --clobber --add alpha 10.32.0.10:6789 --add bravo
> > > > 10.32.0.25:6789 --add charlie
> > > > 10.32.0.11:6789 --print /tmp/mkcephfs.ZOh6tBPAH0/monmap
> > > > /usr/bin/monmaptool: monmap file /tmp/mkcephfs.ZOh6tBPAH0/monmap
> > > > /usr/bin/monmaptool: generated fsid
> > > > adb8d65c-a823-4dc2-9415-22b0d7252699
> > > > epoch 0
> > > > fsid adb8d65c-a823-4dc2-9415-22b0d7252699
> > > > last_changed 2012-07-04 15:04:17.423368 created 2012-07-04
> > > > 15:04:17.423368
> > > > 0: 10.32.0.10:6789/0 mon.alpha
> > > > 1: 10.32.0.11:6789/0 mon.charlie
> > > > 2: 10.32.0.25:6789/0 mon.bravo
> > > > /usr/bin/monmaptool: writing epoch 0 to
> > > > /tmp/mkcephfs.ZOh6tBPAH0/monmap (3 monitors) /usr/bin/ceph-conf -c
> > > > /etc/ceph/ceph.conf -n osd.0 "user"
> > > > === osd.0 ===
> > > > --- dsanb1-coy# /sbin/mkcephfs -d /tmp/mkcephfs.ZOh6tBPAH0
> > > > --init-daemon osd.0
> > > > 2012-07-04 15:04:17.789064 7fc7fadca780 -1 filestore(/srv/osd.0)
> > > > limited size xattrs -- enable filestore_xattr_use_omap
> > > > 2012-07-04 15:04:17.789120 7fc7fadca780 -1 OSD::mkfs: couldn't
> > > > mount
> > > > FileStore: error -95
> > > > 2012-07-04 15:04:17.789161 7fc7fadca780 -1 ** ERROR: error
> > > > creating empty object store in /srv/osd.0: (95) Operation not
> > > > supported
> > > > failed: '/sbin/mkcephfs -d /tmp/mkcephfs.ZOh6tBPAH0 --init-daemon osd.0'
> > > >
> > > >
> > > > Confirming all this was working previously, and the crushmap,
> > > > config file, etc are all proven to be OK (get same failure when
> > > > not specifying a custom crushmap also). Also note that whilst
> > > > the above is failing on
> > > > osd.0 creation, I have swapped disk references and still get the
> > > > same failure on different HDD's when they are hooked in as osd.0
> > >
> > > The only thing that changed from v0.47 is the below. Can you try
> > > replacing 'btrfs device scan || btrfsctl -a' with 'btrfs device scan ;
> > > btrfsctl -a'? Maybe the btrfs tool isn't being pendantic about return
> > > codes...
> > >
> > > sage
> > >
> > >
> > > commit a414fd51c7c5ae5dbe9e3af7db6f17741a58c1a7
> > > Author: Sage Weil <[email protected]>
> > > Date: Sat Feb 11 13:43:23 2012 -0800
> > >
> > > init-ceph, mkcephfs: try 'btrfs device scan' before 'btrfsctl -a'
> > >
> > > Fixes: #2023
> > > Reported-by: Wido den Hollander <[email protected]>
> > > Signed-off-by: Sage Weil <[email protected]>
> > >
> > > diff --git a/src/mkcephfs.in b/src/mkcephfs.in index
> > > 83fb932..17b6014
> > > 100644
> > > --- a/src/mkcephfs.in
> > > +++ b/src/mkcephfs.in
> > > @@ -332,7 +332,7 @@ if [ -n "$prepareosdfs" ]; then
> > >
> > > modprobe btrfs || true
> > > mkfs.btrfs $btrfs_devs
> > > - btrfsctl -a
> > > + btrfs device scan || btrfsctl -a
> > > mount -t btrfs $btrfs_opt $first_dev $btrfs_path
> > > chown $osd_user $btrfs_path
> > > chmod +w $btrfs_path
> > >
> > >
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> > > in the body of a message to [email protected] More
> > > majordomo info at http://vger.kernel.org/majordomo-info.html
> > >
> > >
> > >
> >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> > in the body of a message to [email protected] More majordomo
> > info at http://vger.kernel.org/majordomo-info.html
> >
> >
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> in the body of a message to [email protected] More majordomo
> info at http://vger.kernel.org/majordomo-info.html
>
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
