OK,
So I'll need to do the installation manually, but the rest of the commands
I should run via ceph-deploy? What version should I be trying to grab for
the manual compilation? Should I be grabbing from git or is there a better
place?
Shlomo
-----------------------------
Shlomo Dubrowin
The Solution to the water crisis in Israel:
# According to WikiPedia, the Kinneret can hold
# 4 km^3, so FULL here is in cubit meters
FULL="4000"
while [ "$LEVEL" -lt "$FULL" ]; do
cat /sea/med /sea/red |\
grep -vi "salt" |\
tee /sea/dead /lake/kinneret
LEVEL=`du -c /sea/dead /lake/kinneret | grep total | awk '{print $1}'`
done
On Tue, Dec 3, 2013 at 5:25 PM, Alfredo Deza <[email protected]>wrote:
> On Tue, Dec 3, 2013 at 10:21 AM, Mark Nelson <[email protected]>
> wrote:
> > Guys, I don't think we have pre-released packages of anything new that is
> > going to work on the pi regardless if you use ceph-deploy. Look at our
> > armhf packages file:
> >
> > http://ceph.com/debian-emperor/dists/wheezy/main/binary-armhf/Packages
> >
> > Unless I'm mistaken, you're going to have to compile it yourself. I
> think
> > Joao was going to try that, not sure if he ever got around to it though.
>
> Oh, good point. I was just assuming that because we support the Distro
> the packages would exist.
>
> >
> > Mark
> >
> >
> > On 12/03/2013 09:03 AM, Alfredo Deza wrote:
> >>
> >> On Tue, Dec 3, 2013 at 9:56 AM, Shlomo Dubrowin <[email protected]>
> >> wrote:
> >>>
> >>> Alfredo,
> >>>
> >>> I started that way, but I run into an error:
> >>>
> >>> $ ceph-deploy install baxter
> >>> [ceph_deploy.cli][INFO ] Invoked (1.3.3): /usr/bin/ceph-deploy install
> >>> baxter
> >>> [ceph_deploy.install][DEBUG ] Installing stable version emperor on
> >>> cluster
> >>> ceph hosts baxter
> >>> [ceph_deploy.install][DEBUG ] Detecting platform for host baxter ...
> >>> [baxter][DEBUG ] connected to host: baxter
> >>> [baxter][DEBUG ] detect platform information from remote host
> >>> [baxter][DEBUG ] detect machine type
> >>> [ceph_deploy.install][INFO ] Distro info: debian 7.0 wheezy
> >>> [baxter][INFO ] installing ceph on baxter
> >>> [baxter][INFO ] Running command: sudo env
> DEBIAN_FRONTEND=noninteractive
> >>> apt-get -q install --assume-yes ca-certificates
> >>> [baxter][DEBUG ] Reading package lists...
> >>> [baxter][DEBUG ] Building dependency tree...
> >>> [baxter][DEBUG ] Reading state information...
> >>> [baxter][DEBUG ] ca-certificates is already the newest version.
> >>> [baxter][DEBUG ] 0 upgraded, 0 newly installed, 0 to remove and 85 not
> >>> upgraded.
> >>> [baxter][INFO ] Running command: sudo wget -O release.asc
> >>> https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc
> >>> [baxter][WARNIN] --2013-12-03 16:32:45--
> >>> https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc
> >>> [baxter][WARNIN] Resolving ceph.com (ceph.com)... 208.113.241.137
> >>> [baxter][WARNIN] Connecting to ceph.com
> >>> (ceph.com)|208.113.241.137|:443...
> >>> connected.
> >>> [baxter][WARNIN] HTTP request sent, awaiting response... 200 OK
> >>> [baxter][WARNIN] Length: unspecified [text/plain]
> >>> [baxter][WARNIN] Saving to: `release.asc'
> >>> [baxter][WARNIN]
> >>> [baxter][WARNIN] 0K .
> >>> 1.06M=0.002s
> >>> [baxter][WARNIN]
> >>> [baxter][WARNIN] 2013-12-03 16:32:53 (1.06 MB/s) - `release.asc' saved
> >>> [1752]
> >>> [baxter][WARNIN]
> >>> [baxter][INFO ] Running command: sudo apt-key add release.asc
> >>> [baxter][DEBUG ] OK
> >>> [baxter][DEBUG ] add ceph deb repo to sources.list
> >>> [baxter][INFO ] Running command: sudo apt-get -q update
> >>> [baxter][DEBUG ] Get:1 http://mirrordirector.raspbian.org wheezy
> >>> Release.gpg
> >>> [490 B]
> >>> [baxter][DEBUG ] Hit http://raspberrypi.collabora.com wheezy
> Release.gpg
> >>> [baxter][DEBUG ] Get:2 http://archive.raspberrypi.org wheezy
> Release.gpg
> >>> [490 B]
> >>> [baxter][DEBUG ] Get:3 http://mirrordirector.raspbian.org wheezy
> Release
> >>> [14.4 kB]
> >>> [baxter][DEBUG ] Hit http://raspberrypi.collabora.com wheezy Release
> >>> [baxter][DEBUG ] Get:4 http://ceph.com wheezy Release.gpg [836 B]
> >>> [baxter][DEBUG ] Get:5 http://archive.raspberrypi.org wheezy Release
> >>> [7224
> >>> B]
> >>> [baxter][DEBUG ] Hit http://raspberrypi.collabora.com wheezy/rpi armhf
> >>> Packages
> >>> [baxter][DEBUG ] Get:6 http://mirrordirector.raspbian.org wheezy/main
> >>> armhf
> >>> Packages [7414 kB]
> >>> [baxter][DEBUG ] Get:7 http://archive.raspberrypi.org wheezy/main
> armhf
> >>> Packages [12.1 kB]
> >>> [baxter][DEBUG ] Get:8 http://ceph.com wheezy Release [5984 B]
> >>> [baxter][DEBUG ] Ign http://raspberrypi.collabora.com wheezy/rpi
> >>> Translation-en
> >>> [baxter][DEBUG ] Get:9 http://ceph.com wheezy/main armhf Packages
> [1010
> >>> B]
> >>> [baxter][DEBUG ] Ign http://archive.raspberrypi.org wheezy/main
> >>> Translation-en
> >>> [baxter][DEBUG ] Ign http://ceph.com wheezy/main Translation-en
> >>> [baxter][DEBUG ] Hit http://mirrordirector.raspbian.org wheezy/contrib
> >>> armhf
> >>> Packages
> >>> [baxter][DEBUG ] Hit http://mirrordirector.raspbian.orgwheezy/non-free
> >>> armhf Packages
> >>> [baxter][DEBUG ] Hit http://mirrordirector.raspbian.org wheezy/rpi
> armhf
> >>> Packages
> >>> [baxter][DEBUG ] Ign http://mirrordirector.raspbian.org wheezy/contrib
> >>> Translation-en
> >>> [baxter][DEBUG ] Ign http://mirrordirector.raspbian.org wheezy/main
> >>> Translation-en
> >>> [baxter][DEBUG ] Ign http://mirrordirector.raspbian.orgwheezy/non-free
> >>> Translation-en
> >>> [baxter][DEBUG ] Ign http://mirrordirector.raspbian.org wheezy/rpi
> >>> Translation-en
> >>> [baxter][DEBUG ] Fetched 7456 kB in 47s (157 kB/s)
> >>> [baxter][DEBUG ] Reading package lists...
> >>> [baxter][INFO ] Running command: sudo env
> DEBIAN_FRONTEND=noninteractive
> >>> DEBIAN_PRIORITY=critical apt-get -q -o Dpkg::Options::=--force-confnew
> >>> --no-install-recommends --assume-yes install -- ceph ceph-mds
> ceph-common
> >>> ceph-fs-common gdisk
> >>> [baxter][WARNIN] E: Unable to locate package ceph-mds
> >>> [baxter][WARNIN] E: Unable to locate package ceph-fs-common
> >>> [baxter][DEBUG ] Reading package lists...
> >>> [baxter][DEBUG ] Building dependency tree...
> >>> [baxter][DEBUG ] Reading state information...
> >>> [baxter][ERROR ] RuntimeError: command returned non-zero exit status:
> 100
> >>> [ceph_deploy][ERROR ] RuntimeError: Failed to execute command: env
> >>> DEBIAN_FRONTEND=noninteractive DEBIAN_PRIORITY=critical apt-get -q -o
> >>> Dpkg::Options::=--force-confnew --no-install-recommends --assume-yes
> >>> install
> >>> -- ceph ceph-mds ceph-common ceph-fs-common gdisk
> >>
> >>
> >> That output looks unexpected. I wonder if it is just a network hiccup
> >> to fail to get to those two packages.
> >>>
> >>>
> >>>
> >>> When running manually:
> >>>
> >>> $ env DEBIAN_FRONTEND=noninteractive DEBIAN_PRIORITY=critical apt-get
> -q
> >>> -o
> >>> Dpkg::Options::=--force-confnew --no-install-recommends --assume-yes
> >>> install
> >>> -- ceph ceph-mds ceph-common ceph-fs-common gdisk
> >>> E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission
> >>> denied)
> >>> E: Unable to lock the administration directory (/var/lib/dpkg/), are
> you
> >>> root?
> >>>
> >>>
> >>> Maybe this is supposed to be a sudo command?
> >>
> >>
> >> Yes, always with sudo unless you are root, which is the reason why
> >> ceph-deploy will not use that in the output.
> >>
> >>>
> >>> $ sudo env DEBIAN_FRONTEND=noninteractive DEBIAN_PRIORITY=critical
> >>> apt-get
> >>> -q -o Dpkg::Options::=--force-confnew --no-install-recommends
> >>> --assume-yes
> >>> install -- ceph ceph-mds ceph-common ceph-fs-common gdisk
> >>> Reading package lists...
> >>> Building dependency tree...
> >>> Reading state information...
> >>> E: Unable to locate package ceph-mds
> >>> E: Unable to locate package ceph-fs-common
> >>>
> >>>
> >>> This was the stage where I tried to install ceph with just apt-get
> >>> install
> >>> manually.
> >>
> >>
> >> If you run a `sudo apt-get update` does that come back correctly or
> >> does it complain?
> >>
> >>>
> >>> Shlomo
> >>>
> >>>
> >>> -----------------------------
> >>> Shlomo Dubrowin
> >>>
> >>> The Solution to the water crisis in Israel:
> >>>
> >>> # According to WikiPedia, the Kinneret can hold
> >>> # 4 km^3, so FULL here is in cubit meters
> >>> FULL="4000"
> >>> while [ "$LEVEL" -lt "$FULL" ]; do
> >>> cat /sea/med /sea/red |\
> >>> grep -vi "salt" |\
> >>> tee /sea/dead /lake/kinneret
> >>> LEVEL=`du -c /sea/dead /lake/kinneret | grep total | awk '{print
> $1}'`
> >>> done
> >>>
> >>>
> >>> On Tue, Dec 3, 2013 at 4:29 PM, Alfredo Deza <[email protected]
> >
> >>> wrote:
> >>>>
> >>>>
> >>>> On Tue, Dec 3, 2013 at 9:21 AM, Shlomo Dubrowin <[email protected]>
> >>>> wrote:
> >>>>>
> >>>>> Alfredo,
> >>>>>
> >>>>> Thank you for your response. I simply did apt-get install ceph on
> the
> >>>>> nodes.
> >>>>>
> >>>>> My /etc/apt/sources.list.d/ceph.list contains:
> >>>>>
> >>>>> deb http://ceph.com/debian-emperor/ wheezy main
> >>>>>
> >>>>
> >>>> Was that added manually? ceph-deploy can take care of handling the
> >>>> sources list for you.
> >>>>
> >>>> It is possible that you have something else in that machine that is
> >>>> saying that 0.43 is the version you should
> >>>> be getting.
> >>>>
> >>>> Have you tried to install with ceph-deploy?:
> >>>>
> >>>> ceph-deploy install baxter
> >>>>
> >>>>
> >>>>
> >>>>>
> >>>>> and the versions I received are what I got.
> >>>>>
> >>>>> Shlomo
> >>>>>
> >>>>> -----------------------------
> >>>>> Shlomo Dubrowin
> >>>>>
> >>>>> The Solution to the water crisis in Israel:
> >>>>>
> >>>>> # According to WikiPedia, the Kinneret can hold
> >>>>> # 4 km^3, so FULL here is in cubit meters
> >>>>> FULL="4000"
> >>>>> while [ "$LEVEL" -lt "$FULL" ]; do
> >>>>> cat /sea/med /sea/red |\
> >>>>> grep -vi "salt" |\
> >>>>> tee /sea/dead /lake/kinneret
> >>>>> LEVEL=`du -c /sea/dead /lake/kinneret | grep total | awk '{print
> >>>>> $1}'`
> >>>>> done
> >>>>>
> >>>>>
> >>>>> On Tue, Dec 3, 2013 at 4:15 PM, Alfredo Deza <
> [email protected]>
> >>>>> wrote:
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Dec 3, 2013 at 8:55 AM, Shlomo Dubrowin <
> [email protected]>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> I'm trying to deploy Ceph on a group of Raspberry Pis using the
> >>>>>>> procedure
> >>>>>>> documented in:
> http://ceph.com/docs/master/start/quick-ceph-deploy/
> >>>>>>>
> >>>>>>> There used to be a site:
> >>>>>>> http://ceph.com/docs/master/start/quick-start/
> >>>>>>> but
> >>>>>>> that page is no longer valid.
> >>>>>>>
> >>>>>>> The first thing I noticed is that the command lsb_release -sc
> >>>>>>> specified
> >>>>>>> in
> >>>>>>> the Ceph Deploy setup provides n/a even though the answer should be
> >>>>>>> wheezy.
> >>>>>>> I manually changed the /etc/apt/sources.list.d/ceph.list to specify
> >>>>>>> wheezy.
> >>>>>>>
> >>>>>>> I've installed ceph-deploy and setup the keys to communicate
> between
> >>>>>>> the
> >>>>>>> deploy Raspberry Pi and all the Raspberry Pis including the system
> >>>>>>> reunning
> >>>>>>> ceph-deploy.
> >>>>>>>
> >>>>>>> When I tried to run ceph-deploy install <node1> the installation
> >>>>>>> failed.
> >>>>>>> I went to each node and installed ceph manually and I see the
> >>>>>>> versions
> >>>>>>> are:
> >>>>>>>
> >>>>>>> $ ceph-deploy --version
> >>>>>>> 1.3.3
> >>>>>>>
> >>>>>>> $ ceph --version
> >>>>>>> ceph version 0.43 (commit:9fa8781c0147d66fcef7c2dd0e09cd3c69747d37)
> >>>>>>>
> >>>>>>>
> >>>>>>> All the nodes have the same ceph version.
> >>>>>>
> >>>>>>
> >>>>>> That looks like a *very* old ceph version. Is there any reason you
> are
> >>>>>> using 0.43 as opposed to the latest one?
> >>>>>>
> >>>>>> How did you installed Ceph on those nodes?
> >>>>>>>
> >>>>>>>
> >>>>>>> When I try to run a command, I get errors:
> >>>>>>>
> >>>>>>> $ ceph-deploy mon create baxter
> >>>>>>> [ceph_deploy.cli][INFO ] Invoked (1.3.3): /usr/bin/ceph-deploy mon
> >>>>>>> create
> >>>>>>> baxter
> >>>>>>> [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts baxter
> >>>>>>> [ceph_deploy.mon][DEBUG ] detecting platform for host baxter ...
> >>>>>>> [baxter][DEBUG ] connected to host: baxter
> >>>>>>> [baxter][DEBUG ] detect platform information from remote host
> >>>>>>> [baxter][DEBUG ] detect machine type
> >>>>>>> [ceph_deploy.mon][INFO ] distro info: debian 7.0 wheezy
> >>>>>>> [baxter][DEBUG ] determining if provided host has same hostname in
> >>>>>>> remote
> >>>>>>> [baxter][DEBUG ] get remote short hostname
> >>>>>>> [baxter][DEBUG ] deploying mon to baxter
> >>>>>>> [baxter][DEBUG ] get remote short hostname
> >>>>>>> [baxter][DEBUG ] remote hostname: baxter
> >>>>>>> [baxter][DEBUG ] write cluster configuration to
> >>>>>>> /etc/ceph/{cluster}.conf
> >>>>>>> [baxter][DEBUG ] create the mon path if it does not exist
> >>>>>>> [baxter][DEBUG ] checking for done path:
> >>>>>>> /var/lib/ceph/mon/ceph-baxter/done
> >>>>>>> [baxter][DEBUG ] done path does not exist:
> >>>>>>> /var/lib/ceph/mon/ceph-baxter/done
> >>>>>>> [baxter][INFO ] creating keyring file:
> >>>>>>> /var/lib/ceph/tmp/ceph-baxter.mon.keyring
> >>>>>>> [baxter][DEBUG ] create the monitor keyring file
> >>>>>>> [baxter][INFO ] Running command: sudo ceph-mon --cluster ceph
> --mkfs
> >>>>>>> -i
> >>>>>>> baxter --keyring /var/lib/ceph/tmp/ceph-baxter.mon.keyring
> >>>>>>> [baxter][WARNIN] too many arguments: [--cluster,ceph]
> >>>>>>> [baxter][WARNIN] usage: ceph-mon -i monid [--mon-data=pathtodata]
> >>>>>>> [flags]
> >>>>>>> [baxter][WARNIN] --debug_mon n
> >>>>>>> [baxter][WARNIN] debug monitor level (e.g. 10)
> >>>>>>> [baxter][WARNIN] --mkfs
> >>>>>>> [baxter][WARNIN] build fresh monitor fs
> >>>>>>> [baxter][DEBUG ] --conf/-c Read configuration from the given
> >>>>>>> configuration file
> >>>>>>> [baxter][DEBUG ] -d Run in foreground, log to stderr.
> >>>>>>> [baxter][DEBUG ] -f Run in foreground, log to usual
> >>>>>>> location.
> >>>>>>> [baxter][DEBUG ] --id/-i set ID portion of my name
> >>>>>>> [baxter][DEBUG ] --name/-n set name (TYPE.ID)
> >>>>>>> [baxter][DEBUG ] --version show version and quit
> >>>>>>> [baxter][DEBUG ]
> >>>>>>> [baxter][DEBUG ] --debug_ms N
> >>>>>>> [baxter][DEBUG ] set message debug level (e.g. 1)
> >>>>>>> [baxter][ERROR ] RuntimeError: command returned non-zero exit
> status:
> >>>>>>> 1
> >>>>>>> [ceph_deploy.mon][ERROR ] Failed to execute command: ceph-mon
> >>>>>>> --cluster
> >>>>>>> ceph
> >>>>>>> --mkfs -i baxter --keyring
> /var/lib/ceph/tmp/ceph-baxter.mon.keyring
> >>>>>>> [ceph_deploy][ERROR ] GenericError: Failed to create 1 monitors
> >>>>>>>
> >>>>>>>
> >>>>>>> If I try to run the same command as the user ceph on the local
> >>>>>>> machine,
> >>>>>>> I
> >>>>>>> get the same error:
> >>>>>>>
> >>>>>>> $ ceph-mon --cluster ceph --mkfs -i baxter --keyring
> >>>>>>> /var/lib/ceph/tmp/ceph-baxter.mon.keyring
> >>>>>>> failed to open log file '/var/log/ceph/mon.baxter.log': (13)
> >>>>>>> Permission
> >>>>>>> denied
> >>>>>>> too many arguments: [--cluster,ceph]
> >>>>>>> usage: ceph-mon -i monid [--mon-data=pathtodata] [flags]
> >>>>>>> --debug_mon n
> >>>>>>> debug monitor level (e.g. 10)
> >>>>>>> --mkfs
> >>>>>>> build fresh monitor fs
> >>>>>>> --conf/-c Read configuration from the given configuration
> file
> >>>>>>> -d Run in foreground, log to stderr.
> >>>>>>> -f Run in foreground, log to usual location.
> >>>>>>> --id/-i set ID portion of my name
> >>>>>>> --name/-n set name (TYPE.ID)
> >>>>>>> --version show version and quit
> >>>>>>>
> >>>>>>> --debug_ms N
> >>>>>>> set message debug level (e.g. 1)
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> This makes me think that the ceph-deploy installed doesn't work
> with
> >>>>>>> the
> >>>>>>> ceph installed on each node.
> >>>>>>>
> >>>>>>> So I guess I have 2 questions:
> >>>>>>>
> >>>>>>> How can I determine which ceph-deploy goes with which ceph and how
> >>>>>>> can I
> >>>>>>> correct this apparent mismatch?
> >>>>>>
> >>>>>>
> >>>>>> I don't think there was a ceph-deploy per-se that would've worked
> for
> >>>>>> that Ceph version.
> >>>>>>
> >>>>>> I believe that ceph-deploy is very backwards compatible for a few
> Ceph
> >>>>>> versions. At the very least we should
> >>>>>> fully support the latest 3 major versions and we special case a few
> >>>>>> commands depending on the version installed.
> >>>>>>
> >>>>>> But like I mentioned, the Ceph version you have installed seems
> like a
> >>>>>> very very old one.
> >>>>>>
> >>>>>>> Should I be looking at just using local ceph commands to configure
> >>>>>>> this
> >>>>>>> by
> >>>>>>> hand? Are there instructions on how to do this by hand?
> >>>>>>
> >>>>>>
> >>>>>> If you must have that version I don't think there is any option here
> >>>>>> but to do it by hand and avoid ceph-deploy.
> >>>>>>
> >>>>>> For docs, I think that might be even more difficult to do as you
> would
> >>>>>> need to go to the tag for that version and look at the docs
> >>>>>> for that tag in the repository.
> >>>>>>
> >>>>>> Then again, what is the reason for using that version?
> >>>>>>
> >>>>>>>
> >>>>>>> Thank you.
> >>>>>>>
> >>>>>>> Shlomo
> >>>>>>>
> >>>>>>> -----------------------------
> >>>>>>> Shlomo Dubrowin
> >>>>>>>
> >>>>>>> The Solution to the water crisis in Israel:
> >>>>>>>
> >>>>>>> # According to WikiPedia, the Kinneret can hold
> >>>>>>> # 4 km^3, so FULL here is in cubit meters
> >>>>>>> FULL="4000"
> >>>>>>> while [ "$LEVEL" -lt "$FULL" ]; do
> >>>>>>> cat /sea/med /sea/red |\
> >>>>>>> grep -vi "salt" |\
> >>>>>>> tee /sea/dead /lake/kinneret
> >>>>>>> LEVEL=`du -c /sea/dead /lake/kinneret | grep total | awk '{print
> >>>>>>> $1}'`
> >>>>>>> done
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> ceph-users mailing list
> >>>>>>> [email protected]
> >>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >>>>>>>
> >>>>>
> >>>>>
> >>>
> >>>
> >> _______________________________________________
> >> ceph-users mailing list
> >> [email protected]
> >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >>
> >
> > _______________________________________________
> > ceph-users mailing list
> > [email protected]
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> _______________________________________________
> ceph-users mailing list
> [email protected]
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
