dmesg output please.
2014-08-11 2:16 GMT+04:00 Lorieri <[email protected]>: > same here, did you manage to fix it ? > > On Mon, Oct 28, 2013 at 3:13 PM, Kevin Weiler > <[email protected]> wrote: > > Hi Josh, > > > > We did map it directly to the host, and it seems to work just fine. I > > think this is a problem with how the container is accessing the rbd > module. > > > > -- > > > > Kevin Weiler > > > > IT > > > > > > IMC Financial Markets | 233 S. Wacker Drive, Suite 4300 | Chicago, IL > > 60606 | http://imc-chicago.com/ > > > > Phone: +1 312-204-7439 | Fax: +1 312-244-3301 | E-Mail: > > [email protected] > > > > > > > > > > > > > > > > On 10/18/13 7:50 PM, "Josh Durgin" <[email protected]> wrote: > > > >>On 10/18/2013 10:04 AM, Kevin Weiler wrote: > >>> The kernel is 3.11.4-201.fc19.x86_64, and the image format is 1. I did, > >>> however, try a map with an RBD that was format 2. I got the same error. > >> > >>To rule out any capability drops as the culprit, can you map an rbd > >>image on the same host outside of a container? > >> > >>Josh > >> > >>> -- > >>> > >>> *Kevin Weiler* > >>> > >>> IT > >>> > >>> IMC Financial Markets | 233 S. Wacker Drive, Suite 4300 | Chicago, IL > >>> 60606 | http://imc-chicago.com/ > >>> > >>> Phone: +1 312-204-7439 | Fax: +1 312-244-3301 | E-Mail: > >>> [email protected] <mailto:[email protected]>_ > >>> > >>> > >>> From: Gregory Farnum <[email protected] <mailto:[email protected]>> > >>> Date: Friday, October 18, 2013 10:26 AM > >>> To: Omar Marquez <[email protected] > >>> <mailto:[email protected]>> > >>> Cc: Kyle Bader <[email protected] <mailto:[email protected]>>, > >>> Kevin Weiler <[email protected] > >>> <mailto:[email protected]>>, "[email protected] > >>> <mailto:[email protected]>" <[email protected] > >>> <mailto:[email protected]>>, Khalid Goudeaux > >>> <[email protected] > >>><mailto:[email protected]>> > >>> Subject: Re: [ceph-users] mounting RBD in linux containers > >>> > >>> What kernel are you running, and which format is the RBD image? I > >>> thought we had a special return code for when the kernel doesn't > support > >>> the features used by that image, but that could be the problem. > >>> -Greg > >>> > >>> On Thursday, October 17, 2013, Omar Marquez wrote: > >>> > >>> > >>> Strace produces below: > >>> > >>> Š > >>> > >>> futex(0xb5637c, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0xb56378, > >>> {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 > >>> futex(0xb562f8, FUTEX_WAKE_PRIVATE, 1) = 1 > >>> add_key(0x424408, 0x7fff82c4e210, 0x7fff82c4e140, 0x22, > >>> 0xfffffffe) = 607085216 > >>> stat("/sys/bus/rbd", {st_mode=S_IFDIR|0755, st_size=0, ...}) = > 0 > >>> *open("/sys/bus/rbd/add", O_WRONLY) = 3* > >>> *write(3, "10.198.41.6:6789 > >>> <http://10.198.41.6:6789>,10.198.41.8:678 > >>> <http://10.198.41.8:678>"..., 96) = -1 EINVAL (Invalid > >>>argument)* > >>> close(3) = 0 > >>> rt_sigaction(SIGINT, {SIG_IGN, [], SA_RESTORER, > 0x7fbf8a7efa90}, > >>> {SIG_DFL, [], 0}, 8) = 0 > >>> rt_sigaction(SIGQUIT, {SIG_IGN, [], SA_RESTORER, > >>> 0x7fbf8a7efa90}, {SIG_DFL, [], 0}, 8) = 0 > >>> rt_sigprocmask(SIG_BLOCK, [CHLD], [PIPE], 8) = 0 > >>> clone(child_stack=0, flags=CLONE_PARENT_SETTID|SIGCHLD, > >>> parent_tidptr=0x7fff82c4e040) = 22 > >>> wait4(22, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = > 22 > >>> rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, > 0x7fbf8a7efa90}, > >>> NULL, 8) = 0 > >>> rt_sigaction(SIGQUIT, {SIG_DFL, [], SA_RESTORER, > >>> 0x7fbf8a7efa90}, NULL, 8) = 0 > >>> rt_sigprocmask(SIG_SETMASK, [PIPE], NULL, 8) = 0 > >>> write(2, "rbd: add failed: ", 17rbd: add failed: ) = 17 > >>> write(2, "(22) Invalid argument", 21(22) Invalid argument) = > >>>21 > >>> write(2, "\n", 1 > >>> ) = 1 > >>> exit_group(1) = ? > >>> +++ exited with 1 +++ > >>> > >>> > >>> The app is run inside the container with setuid = 0 and the > >>> container is able to mount all required filesystems Š could this > >>> still be a capability problem ? Also I do not see any call to > >>> capset() in the strafe log Š > >>> > >>> -- > >>> Om > >>> > >>> > >>> From: Kyle Bader <[email protected]> > >>> Date: Thursday, October 17, 2013 5:08 PM > >>> To: Kevin Weiler <[email protected]> > >>> Cc: "[email protected]" <[email protected]>, Omar > >>> Marquez <[email protected]>, Khalid Goudeaux > >>> <[email protected]> > >>> Subject: Re: [ceph-users] mounting RBD in linux containers > >>> > >>> My first guess would be that it's due to LXC dropping capabilities, > >>> I'd investigate whether CAP_SYS_ADMIN is being dropped. You need > >>> CAP_SYS_ADMIN for mount and block ioctls, if the container doesn't > >>> have those privs a map will likely fail. Maybe try tracing the > >>> command with strace? > >>> > >>> On Thu, Oct 17, 2013 at 2:45 PM, Kevin Weiler > >>> <[email protected]> wrote: > >>> > >>> Hi all, > >>> > >>> We're trying to mount an rbd image inside of a linux container > >>> that has been created with docker (https://www.docker.io/). We > >>> seem to have access to the rbd kernel module from inside the > >>> container: > >>> > >>> # lsmod | grep ceph > >>> libceph 218854 1 rbd > >>> libcrc32c 12603 3 xfs,libceph,dm_persistent_data > >>> > >>> And we can query the pool for available rbds and create rbds > >>> from inside the container: > >>> > >>> # rbd -p dockers --id dockers --keyring > >>> /etc/ceph/ceph.client.dockers.keyring create lxctest --size > >>>51200 > >>> # rbd -p dockers --id dockers --keyring > >>> /etc/ceph/ceph.client.dockers.keyring ls > >>> lxctest > >>> > >>> But for some reason, we can't seem to map the device to the > >>> container: > >>> > >>> # rbd -p dockers --id dockers --keyring > >>> /etc/ceph/ceph.client.dockers.keyring map lxctest > >>> rbd: add failed: (22) Invalid argument > >>> > >>> I don't see anything particularly interesting in dmesg or > >>> messages on either the container or the host box. Any ideas on > >>> how to troubleshoot this? > >>> > >>> Thanks! > >>> > >>> > >>> -- > >>> > >>> *Kevin Weiler* > >>> > >>> IT > >>> > >>> IMC Financial Markets | 233 S. Wacker Drive, Suite 4300 | > >>> Chicago, IL 60606 | http://imc-chicago.com/ > >>> > >>> Phone: +1 312-204-7439 | Fax: +1 312-244-3301 | E-Mail: > >>> [email protected]_ > >>> > >>> > >>> > >>>------------------------------------------------------------------------ > >>> > >>> The information in this e-mail is intended only for the person > >>> or entity to which it is addressed. > >>> > >>> It may contain confidential and /or privileged material. If > >>> someone other than the intended recipient should receive this > >>> e-mail, he / she shall not be entitled to read, disseminate, > >>> disclose or duplicate it. > >>> > >>> If you receive this e-mail unintentionally, please inform us > >>> immediately by "re > >>> > >>> > >>> > >>>------------------------------------------------------------------------ > >>> > >>> The information in this e-mail is intended only for the person or > >>> entity to which it is addressed. > >>> > >>> It may contain confidential and /or privileged material. If someone > >>> other than the intended recipient should receive this e-mail, he / > >>> she shall not be entitled to read, disseminate, disclose or > >>> duplicate it. > >>> > >>> If you receive this e-mail unintentionally, please inform us > >>> immediately by "reply" and then delete it from your system. > Although > >>> this information has been compiled with great care, neither IMC > >>> Financial Markets & Asset Management nor any of its related > entities > >>> shall accept any responsibility for any errors, omissions or other > >>> inaccuracies in this information or for the consequences thereof, > >>> nor shall it be bound in any way by the contents of this e-mail or > >>> its attachments. In the event of incomplete or incorrect > >>> transmission, please return the e-mail to the sender and > permanently > >>> delete this message and any attachments. > >>> > >>> Messages and attachments are scanned for all known viruses. Always > >>> scan attachments before opening them. > >>> > >>> > >>> > >>> -- > >>> Software Engineer #42 @ http://inktank.com | http://ceph.com > >>> > >>> > ------------------------------------------------------------------------ > >>> > >>> The information in this e-mail is intended only for the person or > entity > >>> to which it is addressed. > >>> > >>> It may contain confidential and /or privileged material. If someone > >>> other than the intended recipient should receive this e-mail, he / she > >>> shall not be entitled to read, disseminate, disclose or duplicate it. > >>> > >>> If you receive this e-mail unintentionally, please inform us > immediately > >>> by "reply" and then delete it from your system. Although this > >>> information has been compiled with great care, neither IMC Financial > >>> Markets & Asset Management nor any of its related entities shall accept > >>> any responsibility for any errors, omissions or other inaccuracies in > >>> this information or for the consequences thereof, nor shall it be bound > >>> in any way by the contents of this e-mail or its attachments. In the > >>> event of incomplete or incorrect transmission, please return the e-mail > >>> to the sender and permanently delete this message and any attachments. > >>> > >>> Messages and attachments are scanned for all known viruses. Always scan > >>> attachments before opening them. > >>> > >>> > >>> _______________________________________________ > >>> ceph-users mailing list > >>> [email protected] > >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >>> > >> > > > > > > ________________________________ > > > > The information in this e-mail is intended only for the person or entity > to which it is addressed. > > > > It may contain confidential and /or privileged material. If someone > other than the intended recipient should receive this e-mail, he / she > shall not be entitled to read, disseminate, disclose or duplicate it. > > > > If you receive this e-mail unintentionally, please inform us immediately > by "reply" and then delete it from your system. Although this information > has been compiled with great care, neither IMC Financial Markets & Asset > Management nor any of its related entities shall accept any responsibility > for any errors, omissions or other inaccuracies in this information or for > the consequences thereof, nor shall it be bound in any way by the contents > of this e-mail or its attachments. In the event of incomplete or incorrect > transmission, please return the e-mail to the sender and permanently delete > this message and any attachments. > > > > Messages and attachments are scanned for all known viruses. Always scan > attachments before opening them. > > _______________________________________________ > > ceph-users mailing list > > [email protected] > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- С уважением, Фасихов Ирек Нургаязович Моб.: +79229045757
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
