[ceph-users] ceph-deploy from hammer server installs infernalis on nodes
Hi, running ceph-deploy install from a ceph version 0.94.6 (e832001feaf8c176593e0325c8298e3f16dfb403) cluster, will install ceph version 9.2.1 (752b6a3020c3de74e07d2a8b4c5e48dab5a6b6fd) on the node. Seen on centos 7.2 I was searching here now for 2 houres why strange things happens. I guess that this is why. I dont know if this is wanted behaviour, but if there is no argument given, ceph-deploy should install what is installed on the whole cluster, and not the very new one. At least i expect it to be like that. Greetings -- Mit freundlichen Gruessen / Best regards Oliver Dzombic IP-Interactive mailto:i...@ip-interactive.de Anschrift: IP Interactive UG ( haftungsbeschraenkt ) Zum Sonnenberg 1-3 63571 Gelnhausen HRB 93402 beim Amtsgericht Hanau Geschäftsführung: Oliver Dzombic Steuer Nr.: 35 236 3622 1 UST ID: DE274086107 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] ceph-deploy for Hammer
Hi Travis, These binaries are hosted on Canonical servers and are only for Ubuntu. Until the latest FireFly patch release 0.80.9, everything worked fine. I just tried the hammer binaries, and they seem to be failing in loading up erasure coding libraries. I have now built my own binaries and I was able to get the cluster up and running using ceph-deploy. You just have to skip the ceph installation step with ceph-deploy and rather do a manual install from deb files. Rest worked fine. Thanks Pankaj -Original Message- From: Travis Rhoden [mailto:trho...@gmail.com] Sent: Thursday, May 28, 2015 8:02 AM To: Garg, Pankaj Cc: ceph-users@lists.ceph.com Subject: Re: [ceph-users] ceph-deploy for Hammer Hi Pankaj, While there have been times in the past where ARM binaries were hosted on ceph.com, there is not currently any ARM hardware for builds. I don't think you will see any ARM binaries in http://ceph.com/debian-hammer/pool/main/c/ceph/, for example. Combine that with the fact that ceph-deploy is not intended to work with locally compiled binaries (only packages, as it relies on paths, conventions, and service definitions from the packages), and it is a very tricky combo to use ceph-deploy and ARM together. Your most recent error is indicative of the ceph-mon service not coming up successfully. when ceph-mon (the service, not the daemon) is started, it also calls ceph-create-keys, which waits for the monitor daemon to come up and the creates keys that are necessary for all clusters to run when using cephx (the admin key, the bootsraps keys). - Travis On Wed, May 27, 2015 at 8:27 PM, Garg, Pankaj pankaj.g...@caviumnetworks.com wrote: Actually the ARM binaries do exist and I have been using for previous releases. Somehow this library is the one that doesn’t load. Anyway I did compile my own Ceph for ARM, and now getting the following issue: [ceph_deploy.gatherkeys][WARNIN] Unable to find /etc/ceph/ceph.client.admin.keyring on ceph1 [ceph_deploy][ERROR ] KeyNotFoundError: Could not find keyring file: /etc/ceph/ceph.client.admin.keyring on host ceph1 From: Somnath Roy [mailto:somnath@sandisk.com] Sent: Wednesday, May 27, 2015 4:29 PM To: Garg, Pankaj Cc: ceph-users@lists.ceph.com Subject: RE: ceph-deploy for Hammer If you are trying to install the ceph repo hammer binaries, I don’t think it is built for ARM. Both binary and the .so needs to be built in ARM to make this work I guess. Try to build hammer code base in your ARM server and then retry. Thanks Regards Somnath From: Pankaj Garg [mailto:pankaj.g...@caviumnetworks.com] Sent: Wednesday, May 27, 2015 4:17 PM To: Somnath Roy Cc: ceph-users@lists.ceph.com Subject: RE: ceph-deploy for Hammer Yes I am on ARM. -Pankaj On May 27, 2015 3:58 PM, Somnath Roy somnath@sandisk.com wrote: Are you running this on ARM ? If not, it should not go for loading this library. Thanks Regards Somnath From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Garg, Pankaj Sent: Wednesday, May 27, 2015 2:26 PM To: Garg, Pankaj; ceph-users@lists.ceph.com Subject: Re: [ceph-users] ceph-deploy for Hammer I seem to be getting these errors in the Monitor Log : 2015-05-27 21:17:41.908839 3ff907368e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:41.978113 3ff969168e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16592 2015-05-27 21:17:41.984383 3ff969168e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:41.98 3ff969168e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:42.052415 3ff90cf68e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16604 2015-05-27 21:17:42.058656 3ff90cf68e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:42.058715 3ff90cf68e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:42.125279 3ffac4368e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16616 2015-05-27 21:17:42.131666 3ffac4368e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory
Re: [ceph-users] ceph-deploy for Hammer
Hi Pankaj, While there have been times in the past where ARM binaries were hosted on ceph.com, there is not currently any ARM hardware for builds. I don't think you will see any ARM binaries in http://ceph.com/debian-hammer/pool/main/c/ceph/, for example. Combine that with the fact that ceph-deploy is not intended to work with locally compiled binaries (only packages, as it relies on paths, conventions, and service definitions from the packages), and it is a very tricky combo to use ceph-deploy and ARM together. Your most recent error is indicative of the ceph-mon service not coming up successfully. when ceph-mon (the service, not the daemon) is started, it also calls ceph-create-keys, which waits for the monitor daemon to come up and the creates keys that are necessary for all clusters to run when using cephx (the admin key, the bootsraps keys). - Travis On Wed, May 27, 2015 at 8:27 PM, Garg, Pankaj pankaj.g...@caviumnetworks.com wrote: Actually the ARM binaries do exist and I have been using for previous releases. Somehow this library is the one that doesn’t load. Anyway I did compile my own Ceph for ARM, and now getting the following issue: [ceph_deploy.gatherkeys][WARNIN] Unable to find /etc/ceph/ceph.client.admin.keyring on ceph1 [ceph_deploy][ERROR ] KeyNotFoundError: Could not find keyring file: /etc/ceph/ceph.client.admin.keyring on host ceph1 From: Somnath Roy [mailto:somnath@sandisk.com] Sent: Wednesday, May 27, 2015 4:29 PM To: Garg, Pankaj Cc: ceph-users@lists.ceph.com Subject: RE: ceph-deploy for Hammer If you are trying to install the ceph repo hammer binaries, I don’t think it is built for ARM. Both binary and the .so needs to be built in ARM to make this work I guess. Try to build hammer code base in your ARM server and then retry. Thanks Regards Somnath From: Pankaj Garg [mailto:pankaj.g...@caviumnetworks.com] Sent: Wednesday, May 27, 2015 4:17 PM To: Somnath Roy Cc: ceph-users@lists.ceph.com Subject: RE: ceph-deploy for Hammer Yes I am on ARM. -Pankaj On May 27, 2015 3:58 PM, Somnath Roy somnath@sandisk.com wrote: Are you running this on ARM ? If not, it should not go for loading this library. Thanks Regards Somnath From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Garg, Pankaj Sent: Wednesday, May 27, 2015 2:26 PM To: Garg, Pankaj; ceph-users@lists.ceph.com Subject: Re: [ceph-users] ceph-deploy for Hammer I seem to be getting these errors in the Monitor Log : 2015-05-27 21:17:41.908839 3ff907368e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:41.978113 3ff969168e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16592 2015-05-27 21:17:41.984383 3ff969168e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:41.98 3ff969168e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:42.052415 3ff90cf68e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16604 2015-05-27 21:17:42.058656 3ff90cf68e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:42.058715 3ff90cf68e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:42.125279 3ffac4368e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16616 2015-05-27 21:17:42.131666 3ffac4368e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:42.131726 3ffac4368e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error The lib file exists, so not sure why this is happening. Any help appreciated. Thanks Pankaj From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Garg, Pankaj Sent: Wednesday, May 27, 2015 1:37 PM To: ceph-users@lists.ceph.com Subject: [ceph-users] ceph-deploy for Hammer Hi, Is there a particular verion of Ceph-Deploy that should be used with Hammer release? This is a brand new cluster. I’m getting the following error when running command : ceph-deploy mon create-initial [ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephuser
[ceph-users] ceph-deploy for Hammer
Hi, Is there a particular verion of Ceph-Deploy that should be used with Hammer release? This is a brand new cluster. I'm getting the following error when running command : ceph-deploy mon create-initial [ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephuser/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.25): /usr/local/bin/ceph-deploy mon create-initial [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts ceph1 [ceph_deploy.mon][DEBUG ] detecting platform for host ceph1 ... [ceph1][DEBUG ] connection detected need for sudo [ceph1][DEBUG ] connected to host: ceph1 [ceph1][DEBUG ] detect platform information from remote host [ceph1][DEBUG ] detect machine type [ceph_deploy.mon][INFO ] distro info: Ubuntu 14.04 trusty [ceph1][DEBUG ] determining if provided host has same hostname in remote [ceph1][DEBUG ] get remote short hostname [ceph1][DEBUG ] deploying mon to ceph1 [ceph1][DEBUG ] get remote short hostname [ceph1][DEBUG ] remote hostname: ceph1 [ceph1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf [ceph1][DEBUG ] create the mon path if it does not exist [ceph1][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-ceph1/done [ceph1][DEBUG ] create a done file to avoid re-doing the mon deployment [ceph1][DEBUG ] create the init path if it does not exist [ceph1][DEBUG ] locating the `service` executable... [ceph1][INFO ] Running command: sudo initctl emit ceph-mon cluster=ceph id=ceph1 [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory [ceph1][WARNIN] monitor: mon.ceph1, might not be running yet [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory [ceph1][WARNIN] monitor ceph1 does not exist in monmap [ceph_deploy.mon][INFO ] processing monitor mon.ceph1 [ceph1][DEBUG ] connection detected need for sudo [ceph1][DEBUG ] connected to host: ceph1 [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory [ceph_deploy.mon][WARNIN] mon.ceph1 monitor is not yet in quorum, tries left: 5 [ceph_deploy.mon][WARNIN] waiting 5 seconds before retrying [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory [ceph_deploy.mon][WARNIN] mon.ceph1 monitor is not yet in quorum, tries left: 4 [ceph_deploy.mon][WARNIN] waiting 10 seconds before retrying [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory [ceph_deploy.mon][WARNIN] mon.ceph1 monitor is not yet in quorum, tries left: 3 [ceph_deploy.mon][WARNIN] waiting 10 seconds before retrying [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory [ceph_deploy.mon][WARNIN] mon.ceph1 monitor is not yet in quorum, tries left: 2 [ceph_deploy.mon][WARNIN] waiting 15 seconds before retrying [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory [ceph_deploy.mon][WARNIN] mon.ceph1 monitor is not yet in quorum, tries left: 1 [ceph_deploy.mon][WARNIN] waiting 20 seconds before retrying [ceph_deploy.mon][ERROR ] Some monitors have still not reached quorum: [ceph_deploy.mon][ERROR ] ceph1 Thanks Pankaj ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] ceph-deploy for Hammer
I seem to be getting these errors in the Monitor Log : 2015-05-27 21:17:41.908839 3ff907368e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:41.978113 3ff969168e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16592 2015-05-27 21:17:41.984383 3ff969168e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:41.98 3ff969168e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:42.052415 3ff90cf68e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16604 2015-05-27 21:17:42.058656 3ff90cf68e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:42.058715 3ff90cf68e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:42.125279 3ffac4368e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16616 2015-05-27 21:17:42.131666 3ffac4368e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:42.131726 3ffac4368e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error The lib file exists, so not sure why this is happening. Any help appreciated. Thanks Pankaj From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Garg, Pankaj Sent: Wednesday, May 27, 2015 1:37 PM To: ceph-users@lists.ceph.com Subject: [ceph-users] ceph-deploy for Hammer Hi, Is there a particular verion of Ceph-Deploy that should be used with Hammer release? This is a brand new cluster. I'm getting the following error when running command : ceph-deploy mon create-initial [ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephuser/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.25): /usr/local/bin/ceph-deploy mon create-initial [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts ceph1 [ceph_deploy.mon][DEBUG ] detecting platform for host ceph1 ... [ceph1][DEBUG ] connection detected need for sudo [ceph1][DEBUG ] connected to host: ceph1 [ceph1][DEBUG ] detect platform information from remote host [ceph1][DEBUG ] detect machine type [ceph_deploy.mon][INFO ] distro info: Ubuntu 14.04 trusty [ceph1][DEBUG ] determining if provided host has same hostname in remote [ceph1][DEBUG ] get remote short hostname [ceph1][DEBUG ] deploying mon to ceph1 [ceph1][DEBUG ] get remote short hostname [ceph1][DEBUG ] remote hostname: ceph1 [ceph1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf [ceph1][DEBUG ] create the mon path if it does not exist [ceph1][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-ceph1/done [ceph1][DEBUG ] create a done file to avoid re-doing the mon deployment [ceph1][DEBUG ] create the init path if it does not exist [ceph1][DEBUG ] locating the `service` executable... [ceph1][INFO ] Running command: sudo initctl emit ceph-mon cluster=ceph id=ceph1 [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory [ceph1][WARNIN] monitor: mon.ceph1, might not be running yet [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory [ceph1][WARNIN] monitor ceph1 does not exist in monmap [ceph_deploy.mon][INFO ] processing monitor mon.ceph1 [ceph1][DEBUG ] connection detected need for sudo [ceph1][DEBUG ] connected to host: ceph1 [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory [ceph_deploy.mon][WARNIN] mon.ceph1 monitor is not yet in quorum, tries left: 5 [ceph_deploy.mon][WARNIN] waiting 5 seconds before retrying [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory [ceph_deploy.mon][WARNIN] mon.ceph1
Re: [ceph-users] ceph-deploy for Hammer
Actually the ARM binaries do exist and I have been using for previous releases. Somehow this library is the one that doesn’t load. Anyway I did compile my own Ceph for ARM, and now getting the following issue: [ceph_deploy.gatherkeys][WARNIN] Unable to find /etc/ceph/ceph.client.admin.keyring on ceph1 [ceph_deploy][ERROR ] KeyNotFoundError: Could not find keyring file: /etc/ceph/ceph.client.admin.keyring on host ceph1 From: Somnath Roy [mailto:somnath@sandisk.com] Sent: Wednesday, May 27, 2015 4:29 PM To: Garg, Pankaj Cc: ceph-users@lists.ceph.com Subject: RE: ceph-deploy for Hammer If you are trying to install the ceph repo hammer binaries, I don’t think it is built for ARM. Both binary and the .so needs to be built in ARM to make this work I guess. Try to build hammer code base in your ARM server and then retry. Thanks Regards Somnath From: Pankaj Garg [mailto:pankaj.g...@caviumnetworks.com] Sent: Wednesday, May 27, 2015 4:17 PM To: Somnath Roy Cc: ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com Subject: RE: ceph-deploy for Hammer Yes I am on ARM. -Pankaj On May 27, 2015 3:58 PM, Somnath Roy somnath@sandisk.commailto:somnath@sandisk.com wrote: Are you running this on ARM ? If not, it should not go for loading this library. Thanks Regards Somnath From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Garg, Pankaj Sent: Wednesday, May 27, 2015 2:26 PM To: Garg, Pankaj; ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com Subject: Re: [ceph-users] ceph-deploy for Hammer I seem to be getting these errors in the Monitor Log : 2015-05-27 21:17:41.908839 3ff907368e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:41.978113 3ff969168e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16592 2015-05-27 21:17:41.984383 3ff969168e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:41.98 3ff969168e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:42.052415 3ff90cf68e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16604 2015-05-27 21:17:42.058656 3ff90cf68e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:42.058715 3ff90cf68e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:42.125279 3ffac4368e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16616 2015-05-27 21:17:42.131666 3ffac4368e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:42.131726 3ffac4368e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error The lib file exists, so not sure why this is happening. Any help appreciated. Thanks Pankaj From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Garg, Pankaj Sent: Wednesday, May 27, 2015 1:37 PM To: ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com Subject: [ceph-users] ceph-deploy for Hammer Hi, Is there a particular verion of Ceph-Deploy that should be used with Hammer release? This is a brand new cluster. I’m getting the following error when running command : ceph-deploy mon create-initial [ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephuser/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.25): /usr/local/bin/ceph-deploy mon create-initial [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts ceph1 [ceph_deploy.mon][DEBUG ] detecting platform for host ceph1 ... [ceph1][DEBUG ] connection detected need for sudo [ceph1][DEBUG ] connected to host: ceph1 [ceph1][DEBUG ] detect platform information from remote host [ceph1][DEBUG ] detect machine type [ceph_deploy.mon][INFO ] distro info: Ubuntu 14.04 trusty [ceph1][DEBUG ] determining if provided host has same hostname in remote [ceph1][DEBUG ] get remote short hostname [ceph1][DEBUG ] deploying mon to ceph1 [ceph1][DEBUG ] get remote short hostname [ceph1][DEBUG ] remote hostname: ceph1 [ceph1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf [ceph1][DEBUG ] create the mon path if it does not exist [ceph1][DEBUG ] checking for done path: /var
Re: [ceph-users] ceph-deploy for Hammer
Are you running this on ARM ? If not, it should not go for loading this library. Thanks Regards Somnath From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Garg, Pankaj Sent: Wednesday, May 27, 2015 2:26 PM To: Garg, Pankaj; ceph-users@lists.ceph.com Subject: Re: [ceph-users] ceph-deploy for Hammer I seem to be getting these errors in the Monitor Log : 2015-05-27 21:17:41.908839 3ff907368e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:41.978113 3ff969168e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16592 2015-05-27 21:17:41.984383 3ff969168e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:41.98 3ff969168e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:42.052415 3ff90cf68e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16604 2015-05-27 21:17:42.058656 3ff90cf68e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:42.058715 3ff90cf68e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:42.125279 3ffac4368e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16616 2015-05-27 21:17:42.131666 3ffac4368e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:42.131726 3ffac4368e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error The lib file exists, so not sure why this is happening. Any help appreciated. Thanks Pankaj From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Garg, Pankaj Sent: Wednesday, May 27, 2015 1:37 PM To: ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com Subject: [ceph-users] ceph-deploy for Hammer Hi, Is there a particular verion of Ceph-Deploy that should be used with Hammer release? This is a brand new cluster. I'm getting the following error when running command : ceph-deploy mon create-initial [ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephuser/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.25): /usr/local/bin/ceph-deploy mon create-initial [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts ceph1 [ceph_deploy.mon][DEBUG ] detecting platform for host ceph1 ... [ceph1][DEBUG ] connection detected need for sudo [ceph1][DEBUG ] connected to host: ceph1 [ceph1][DEBUG ] detect platform information from remote host [ceph1][DEBUG ] detect machine type [ceph_deploy.mon][INFO ] distro info: Ubuntu 14.04 trusty [ceph1][DEBUG ] determining if provided host has same hostname in remote [ceph1][DEBUG ] get remote short hostname [ceph1][DEBUG ] deploying mon to ceph1 [ceph1][DEBUG ] get remote short hostname [ceph1][DEBUG ] remote hostname: ceph1 [ceph1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf [ceph1][DEBUG ] create the mon path if it does not exist [ceph1][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-ceph1/done [ceph1][DEBUG ] create a done file to avoid re-doing the mon deployment [ceph1][DEBUG ] create the init path if it does not exist [ceph1][DEBUG ] locating the `service` executable... [ceph1][INFO ] Running command: sudo initctl emit ceph-mon cluster=ceph id=ceph1 [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory [ceph1][WARNIN] monitor: mon.ceph1, might not be running yet [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory [ceph1][WARNIN] monitor ceph1 does not exist in monmap [ceph_deploy.mon][INFO ] processing monitor mon.ceph1 [ceph1][DEBUG ] connection detected need for sudo [ceph1][DEBUG ] connected to host: ceph1 [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory [ceph_deploy.mon][WARNIN] mon.ceph1 monitor
Re: [ceph-users] ceph-deploy for Hammer
Yes I am on ARM. -Pankaj On May 27, 2015 3:58 PM, Somnath Roy somnath@sandisk.com wrote: Are you running this on ARM ? If not, it should not go for loading this library. Thanks Regards Somnath From: ceph-users [mailto:ceph-users-bounces@lists.ceph.com] On Behalf Of Garg, Pankaj Sent: Wednesday, May 27, 2015 2:26 PM To: Garg, Pankaj; ceph-users@lists.ceph.com Subject: Re: [ceph-users] ceph-deploy for Hammer I seem to be getting these errors in the Monitor Log : 2015-05-27 21:17:41.908839 3ff907368e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:41.978113 3ff969168e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16592 2015-05-27 21:17:41.984383 3ff969168e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:41.98 3ff969168e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:42.052415 3ff90cf68e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16604 2015-05-27 21:17:42.058656 3ff90cf68e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:42.058715 3ff90cf68e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error 2015-05-27 21:17:42.125279 3ffac4368e0 0 ceph version 0.94.1 (e4bfad3a3c51054df7e537a724c8d0bf9be972ff), process ceph-mon, pid 16616 2015-05-27 21:17:42.131666 3ffac4368e0 -1 ErasureCodePluginSelectJerasure: load dlopen(/usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so): /usr/lib/aarch64-linux-gnu/ceph/erasure-code/libec_jerasure_neon.so: cannot open shared object file: No such file or directory 2015-05-27 21:17:42.131726 3ffac4368e0 -1 erasure_code_init(jerasure,/usr/lib/aarch64-linux-gnu/ceph/erasure-code): (5) Input/output error The lib file exists, so not sure why this is happening. Any help appreciated. Thanks Pankaj From: ceph-users [mailto:ceph-users-bounces@lists.ceph.com] On Behalf Of Garg, Pankaj Sent: Wednesday, May 27, 2015 1:37 PM To: ceph-users@lists.ceph.com Subject: [ceph-users] ceph-deploy for Hammer Hi, Is there a particular verion of Ceph-Deploy that should be used with Hammer release? This is a brand new cluster. I’m getting the following error when running command : ceph-deploy mon create-initial [ceph_deploy.conf][DEBUG ] found configuration file at: /home/cephuser/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.25): /usr/local/bin/ceph-deploy mon create-initial [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts ceph1 [ceph_deploy.mon][DEBUG ] detecting platform for host ceph1 ... [ceph1][DEBUG ] connection detected need for sudo [ceph1][DEBUG ] connected to host: ceph1 [ceph1][DEBUG ] detect platform information from remote host [ceph1][DEBUG ] detect machine type [ceph_deploy.mon][INFO ] distro info: Ubuntu 14.04 trusty [ceph1][DEBUG ] determining if provided host has same hostname in remote [ceph1][DEBUG ] get remote short hostname [ceph1][DEBUG ] deploying mon to ceph1 [ceph1][DEBUG ] get remote short hostname [ceph1][DEBUG ] remote hostname: ceph1 [ceph1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf [ceph1][DEBUG ] create the mon path if it does not exist [ceph1][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-ceph1/done [ceph1][DEBUG ] create a done file to avoid re-doing the mon deployment [ceph1][DEBUG ] create the init path if it does not exist [ceph1][DEBUG ] locating the service executable... [ceph1][INFO ] Running command: sudo initctl emit ceph-mon cluster=ceph id=ceph1 [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory [ceph1][WARNIN] monitor: mon.ceph1, might not be running yet [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR ] admin_socket: exception getting command descriptions: [Errno 2] No such file or directory [ceph1][WARNIN] monitor ceph1 does not exist in monmap [ceph_deploy.mon][INFO ] processing monitor mon.ceph1 [ceph1][DEBUG ] connection detected need for sudo [ceph1][DEBUG ] connected to host: ceph1 [ceph1][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.ceph1.asok mon_status [ceph1][ERROR