Re: [ceph-users] ceph-fuse segfaults ( jewel 10.2.2)

Mon, 04 Jul 2016 22:05:24 -0700 

Will do Brad. From you answer it should be a safe thing to do.

Will report later.

Thanks for the help

Cheers

Goncalo



On 07/05/2016 02:42 PM, Brad Hubbard wrote:

On Tue, Jul 5, 2016 at 1:34 PM, Patrick Donnelly <pdonn...@redhat.com> wrote:

Hi Goncalo,

I believe this segfault may be the one fixed here:

https://github.com/ceph/ceph/pull/10027

Ah, nice one Patrick.

Goncalo, the patch is fairly simple, just the addition of a lock on two lines to
resolve the race. Could you try recompiling with those changes and let
us know how
it goes?

Cheers,
Brad


(Sorry for brief top-post. Im on mobile.)

On Jul 4, 2016 9:16 PM, "Goncalo Borges" <goncalo.bor...@sydney.edu.au>
wrote:

Dear All...

We have recently migrated all our ceph infrastructure from 9.2.0 to
10.2.2.

We are currently using ceph-fuse to mount cephfs in a number of clients.

ceph-fuse 10.2.2 client is segfaulting in some situations. One of the
scenarios where ceph-fuse segfaults is when a user submits a parallel (mpi)
application requesting 4 hosts with 4 cores each (16 instances in total) .
According to the user, each instance has its own dedicated inputs and
outputs.

Please note that if we go back to ceph-fuse 9.2.0 client everything works
fine.

The ceph-fuse 10.2.2 client segfault is the following (we were able to
capture it mounting ceph-fuse in debug mode):

2016-07-04 21:21:00.074087 7f6aed92be40  0 ceph version 10.2.2
(45107e21c568dd033c2f0a3107dec8f0b0e58374), process ceph-fuse, pid 7346
ceph-fuse[7346]: starting ceph client
2016-07-04 21:21:00.107816 7f6aed92be40 -1 init, newargv = 0x7f6af8c12320
newargc=11
ceph-fuse[7346]: starting fuse
*** Caught signal (Segmentation fault) **
  in thread 7f69d7fff700 thread_name:ceph-fuse
  ceph version 10.2.2 (45107e21c568dd033c2f0a3107dec8f0b0e58374)
  1: (()+0x297ef2) [0x7f6aedbecef2]
  2: (()+0x3b88c0f7e0) [0x7f6aec64b7e0]
  3: (Client::get_root_ino()+0x10) [0x7f6aedaf0330]
  4: (CephFuse::Handle::make_fake_ino(inodeno_t, snapid_t)+0x175)
[0x7f6aedaee035]
  5: (()+0x199891) [0x7f6aedaee891]
  6: (()+0x15b76) [0x7f6aed50db76]
  7: (()+0x12aa9) [0x7f6aed50aaa9]
  8: (()+0x3b88c07aa1) [0x7f6aec643aa1]
  9: (clone()+0x6d) [0x7f6aeb8d193d]
2016-07-05 10:09:14.045131 7f69d7fff700 -1 *** Caught signal
(Segmentation fault) **
  in thread 7f69d7fff700 thread_name:ceph-fuse

  ceph version 10.2.2 (45107e21c568dd033c2f0a3107dec8f0b0e58374)
  1: (()+0x297ef2) [0x7f6aedbecef2]
  2: (()+0x3b88c0f7e0) [0x7f6aec64b7e0]
  3: (Client::get_root_ino()+0x10) [0x7f6aedaf0330]
  4: (CephFuse::Handle::make_fake_ino(inodeno_t, snapid_t)+0x175)
[0x7f6aedaee035]
  5: (()+0x199891) [0x7f6aedaee891]
  6: (()+0x15b76) [0x7f6aed50db76]
  7: (()+0x12aa9) [0x7f6aed50aaa9]
  8: (()+0x3b88c07aa1) [0x7f6aec643aa1]
  9: (clone()+0x6d) [0x7f6aeb8d193d]
  NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed
to interpret this.



The full dump is quite long. Here are the very last bits of it. Let me
know if you need the full dump.

--- begin dump of recent events ---
  -9999> 2016-07-05 10:09:13.956502 7f6a5ffff700  3 client.464559
_getxattr(1000037c789, "security.capability", 0) = -61
  -9998> 2016-07-05 10:09:13.956507 7f6aa96fa700  3 client.464559 ll_write
0x7f6a08028be0 1000037c78c 20094~34
  -9997> 2016-07-05 10:09:13.956527 7f6aa96fa700  3 client.464559 ll_write
0x7f6a08028be0 20094~34 = 34
  -9996> 2016-07-05 10:09:13.956535 7f69d7fff700  3 client.464559 ll_write
0x7f6a100145f0 1000037c78d 28526~34
  -9995> 2016-07-05 10:09:13.956553 7f69d7fff700  3 client.464559 ll_write
0x7f6a100145f0 28526~34 = 34
  -9994> 2016-07-05 10:09:13.956561 7f6ac0dfa700  3 client.464559
ll_forget 1000037c78c 1
  -9993> 2016-07-05 10:09:13.956569 7f6a5ffff700  3 client.464559
ll_forget 1000037c789 1
  -9992> 2016-07-05 10:09:13.956577 7f6a5ebfd700  3 client.464559 ll_write
0x7f6a94006350 1000037c789 22010~216
  -9991> 2016-07-05 10:09:13.956594 7f6a5ebfd700  3 client.464559 ll_write
0x7f6a94006350 22010~216 = 216
  -9990> 2016-07-05 10:09:13.956603 7f6aa8cf9700  3 client.464559
ll_getxattr 1000037c78c.head security.capability size 0
  -9989> 2016-07-05 10:09:13.956609 7f6aa8cf9700  3 client.464559
_getxattr(1000037c78c, "security.capability", 0) = -61

<snip>

   -160> 2016-07-05 10:09:14.043687 7f69d7fff700  3 client.464559
_getxattr(1000037c78a, "security.capability", 0) = -61
   -159> 2016-07-05 10:09:14.043694 7f6ac0dfa700  3 client.464559 ll_write
0x7f6a08042560 1000037c78b 11900~34
   -158> 2016-07-05 10:09:14.043712 7f6ac0dfa700  3 client.464559 ll_write
0x7f6a08042560 11900~34 = 34
   -157> 2016-07-05 10:09:14.043722 7f6ac17fb700  3 client.464559
ll_getattr 100001e9c80.head
   -156> 2016-07-05 10:09:14.043727 7f6ac17fb700  3 client.464559
ll_getattr 100001e9c80.head = 0
   -155> 2016-07-05 10:09:14.043734 7f69d7fff700  3 client.464559
ll_forget 1000037c78a 1
   -154> 2016-07-05 10:09:14.043738 7f6a5ebfd700  3 client.464559 ll_write
0x7f6a140d5930 1000037c78a 18292~34
   -153> 2016-07-05 10:09:14.043759 7f6a5ebfd700  3 client.464559 ll_write
0x7f6a140d5930 18292~34 = 34
   -152> 2016-07-05 10:09:14.043767 7f6ac17fb700  3 client.464559
ll_forget 100001e9c80 1
   -151> 2016-07-05 10:09:14.043784 7f6aa8cf9700  3 client.464559 ll_flush
0x7f6a00049fe0 100001e9c80
   -150> 2016-07-05 10:09:14.043794 7f6aa8cf9700  3 client.464559
ll_getxattr 1000037c78a.head security.capability size 0
   -149> 2016-07-05 10:09:14.043799 7f6aa8cf9700  3 client.464559
_getxattr(1000037c78a, "security.capability", 0) = -61
   -148> 2016-07-05 10:09:14.043804 7f6aab4fd700  3 client.464559
ll_release (fh)0x7f6a00049fe0 100001e9c80
   -147> 2016-07-05 10:09:14.043809 7f6aab4fd700  5 client.464559
_release_fh 0x7f6a00049fe0 mode 1 on 100001e9c80.head(faked_ino=0 ref=3
ll_ref=6 cap_refs={1024=0,2048=0} open={1=1} mode=100755 size=1208688/0
mtime=2016-05-25 12:20:23.667957 caps=pAsLsXsFscr(0=pAsLsXsFscr)
objectset[100001e9c80 ts 0/0 objects 1 dirty_or_tx 0] parents=0x7f6acc045ec0
0x7f6acc04a600)
   -146> 2016-07-05 10:09:14.043827 7f6aa8cf9700  3 client.464559
ll_forget 1000037c78a 1
   -145> 2016-07-05 10:09:14.043838 7f6a5ffff700  3 client.464559 ll_write
0x7f6a140d5930 1000037c78a 18326~34
   -144> 2016-07-05 10:09:14.043862 7f6a5ffff700  3 client.464559 ll_write
0x7f6a140d5930 18326~34 = 34
   -143> 2016-07-05 10:09:14.043872 7f6a5f5fe700  3 client.464559
ll_getxattr 1000037c78b.head security.capability size 0
   -142> 2016-07-05 10:09:14.043879 7f6a5f5fe700  3 client.464559
_getxattr(1000037c78b, "security.capability", 0) = -61
   -141> 2016-07-05 10:09:14.043889 7f6a5f5fe700  3 client.464559
ll_forget 1000037c78b 1
   -140> 2016-07-05 10:09:14.043894 7f6ac21fc700  3 client.464559
ll_getxattr 1000037c78a.head security.capability size 0
   -139> 2016-07-05 10:09:14.043902 7f6ac21fc700  3 client.464559
_getxattr(1000037c78a, "security.capability", 0) = -61
   -138> 2016-07-05 10:09:14.043909 7f6a5f5fe700  3 client.464559 ll_write
0x7f6a08042560 1000037c78b 11934~34
   -137> 2016-07-05 10:09:14.043931 7f6a5f5fe700  3 client.464559 ll_write
0x7f6a08042560 11934~34 = 34
   -136> 2016-07-05 10:09:14.043954 7f6ac0dfa700  3 client.464559
ll_getxattr 1000037c78b.head security.capability size 0
   -135> 2016-07-05 10:09:14.043960 7f6ac0dfa700  3 client.464559
_getxattr(1000037c78b, "security.capability", 0) = -61
   -134> 2016-07-05 10:09:14.043965 7f6aa96fa700  3 client.464559 ll_write
0x7f6a140d5930 1000037c78a 18360~34
   -133> 2016-07-05 10:09:14.043983 7f6aa96fa700  3 client.464559 ll_write
0x7f6a140d5930 18360~34 = 34
   -132> 2016-07-05 10:09:14.044005 7f6a5ebfd700  3 client.464559
ll_getxattr 1000037c78a.head security.capability size 0
   -131> 2016-07-05 10:09:14.044010 7f6a5ebfd700  3 client.464559
_getxattr(1000037c78a, "security.capability", 0) = -61
   -130> 2016-07-05 10:09:14.044015 7f6a5ebfd700  3 client.464559
ll_forget 1000037c78a 1
   -129> 2016-07-05 10:09:14.044021 7f6a5ebfd700  3 client.464559 ll_write
0x7f6a140d5930 1000037c78a 18394~34
   -128> 2016-07-05 10:09:14.044036 7f6a5ebfd700  3 client.464559 ll_write
0x7f6a140d5930 18394~34 = 34
   -127> 2016-07-05 10:09:14.044043 7f6ac0dfa700  3 client.464559
ll_forget 1000037c78b 1
   -126> 2016-07-05 10:09:14.044053 7f69d7fff700  3 client.464559 ll_write
0x7f6a08042560 1000037c78b 11968~34
   -125> 2016-07-05 10:09:14.044067 7f69d7fff700  3 client.464559 ll_write
0x7f6a08042560 11968~34 = 34
   -124> 2016-07-05 10:09:14.044091 7f6aab4fd700  3 client.464559
ll_getxattr 1000037c78b.head security.capability size 0
   -123> 2016-07-05 10:09:14.044095 7f6aab4fd700  3 client.464559
_getxattr(1000037c78b, "security.capability", 0) = -61
   -122> 2016-07-05 10:09:14.044100 7f6aab4fd700  3 client.464559
ll_forget 1000037c78b 1
   -121> 2016-07-05 10:09:14.044104 7f6aab4fd700  3 client.464559 ll_write
0x7f6a08042560 1000037c78b 12002~34
   -120> 2016-07-05 10:09:14.044117 7f6aab4fd700  3 client.464559 ll_write
0x7f6a08042560 12002~34 = 34
   -119> 2016-07-05 10:09:14.044124 7f6ac17fb700  3 client.464559
ll_getattr 1000009b5cb.head
   -118> 2016-07-05 10:09:14.044129 7f6ac17fb700  3 client.464559
ll_getattr 1000009b5cb.head = 0
   -117> 2016-07-05 10:09:14.044140 7f6ac0dfa700  3 client.464559
ll_getxattr 1000037c78a.head security.capability size 0
   -116> 2016-07-05 10:09:14.044148 7f6ac0dfa700  3 client.464559
_getxattr(1000037c78a, "security.capability", 0) = -61
   -115> 2016-07-05 10:09:14.044155 7f6ac17fb700  3 client.464559
ll_forget 1000009b5cb 1
   -114> 2016-07-05 10:09:14.044163 7f6ac17fb700  3 client.464559 ll_write
0x7f6a140d5930 1000037c78a 18428~34
   -113> 2016-07-05 10:09:14.044179 7f6ac17fb700  3 client.464559 ll_write
0x7f6a140d5930 18428~34 = 34
   -112> 2016-07-05 10:09:14.044186 7f6a5ffff700  3 client.464559
ll_lookup 0x7f6acc019b10 data
   -111> 2016-07-05 10:09:14.044194 7f6a5ffff700  3 client.464559
ll_lookup 0x7f6acc019b10 data -> 0 (1000009b9b4)
   -110> 2016-07-05 10:09:14.044201 7f6ac0dfa700  3 client.464559
ll_forget 1000037c78a 1
   -109> 2016-07-05 10:09:14.044209 7f6aa8cf9700  3 client.464559
ll_getxattr 1000037c78b.head security.capability size 0
   -108> 2016-07-05 10:09:14.044215 7f6aa8cf9700  3 client.464559
_getxattr(1000037c78b, "security.capability", 0) = -61
   -107> 2016-07-05 10:09:14.044221 7f6aa8cf9700  3 client.464559
ll_forget 1000037c78b 1
   -106> 2016-07-05 10:09:14.044227 7f6ac21fc700  3 client.464559
ll_forget 1000037c78a 1
   -105> 2016-07-05 10:09:14.044238 7f6a5f5fe700  3 client.464559
ll_getxattr 1000037c78a.head security.capability size 0
   -104> 2016-07-05 10:09:14.044243 7f6a5f5fe700  3 client.464559
_getxattr(1000037c78a, "security.capability", 0) = -61
   -103> 2016-07-05 10:09:14.044251 7f6a5f5fe700  3 client.464559
ll_forget 1000037c78a 1
   -102> 2016-07-05 10:09:14.044258 7f6aa96fa700  3 client.464559 ll_write
0x7f6a08042560 1000037c78b 12036~34
   -101> 2016-07-05 10:09:14.044273 7f6aa96fa700  3 client.464559 ll_write
0x7f6a08042560 12036~34 = 34
   -100> 2016-07-05 10:09:14.044281 7f6ac0dfa700  3 client.464559
ll_getattr 1000009b9b4.head
    -99> 2016-07-05 10:09:14.044286 7f6ac0dfa700  3 client.464559
ll_getattr 1000009b9b4.head = 0
    -98> 2016-07-05 10:09:14.044294 7f6a5f5fe700  3 client.464559 ll_write
0x7f6a140d5930 1000037c78a 18462~34
    -97> 2016-07-05 10:09:14.044310 7f6a5f5fe700  3 client.464559 ll_write
0x7f6a140d5930 18462~34 = 34
    -96> 2016-07-05 10:09:14.044320 7f6a5ebfd700  3 client.464559
ll_getxattr 1000037c78b.head security.capability size 0
    -95> 2016-07-05 10:09:14.044324 7f6a5ebfd700  3 client.464559
_getxattr(1000037c78b, "security.capability", 0) = -61
    -94> 2016-07-05 10:09:14.044331 7f6a5ffff700  3 client.464559
ll_forget 1000009b5cb 1
    -93> 2016-07-05 10:09:14.044348 7f6ac17fb700  3 client.464559 ll_write
0x7f6a08042560 1000037c78b 12070~34
    -92> 2016-07-05 10:09:14.044371 7f6ac17fb700  3 client.464559 ll_write
0x7f6a08042560 12070~34 = 34
    -91> 2016-07-05 10:09:14.044381 7f6ac0dfa700  3 client.464559
ll_forget 1000009b9b4 1
    -90> 2016-07-05 10:09:14.044389 7f6a5ebfd700  3 client.464559
ll_forget 1000037c78b 1
    -89> 2016-07-05 10:09:14.044397 7f6aab4fd700  3 client.464559
ll_getxattr 1000037c78a.head security.capability size 0
    -88> 2016-07-05 10:09:14.044401 7f6aab4fd700  3 client.464559
_getxattr(1000037c78a, "security.capability", 0) = -61
    -87> 2016-07-05 10:09:14.044407 7f6aa8cf9700  3 client.464559
ll_getxattr 1000037c78b.head security.capability size 0
    -86> 2016-07-05 10:09:14.044413 7f6aa8cf9700  3 client.464559
_getxattr(1000037c78b, "security.capability", 0) = -61
    -85> 2016-07-05 10:09:14.044423 7f69d7fff700  3 client.464559
ll_lookup 0x7f6acc026050 ruggerod
    -84> 2016-07-05 10:09:14.044428 7f69d7fff700  3 client.464559
ll_lookup 0x7f6acc026050 ruggerod -> 0 (10000174776)
    -83> 2016-07-05 10:09:14.044432 7f6ac21fc700  3 client.464559 ll_write
0x7f6a08042560 1000037c78b 12104~34
    -82> 2016-07-05 10:09:14.044454 7f6ac21fc700  3 client.464559 ll_write
0x7f6a08042560 12104~34 = 34
    -81> 2016-07-05 10:09:14.044464 7f6aab4fd700  3 client.464559
ll_forget 1000037c78a 1
    -80> 2016-07-05 10:09:14.044471 7f6aa8cf9700  3 client.464559
ll_forget 1000037c78b 1
    -79> 2016-07-05 10:09:14.044481 7f6aa96fa700  3 client.464559 ll_write
0x7f6a140d5930 1000037c78a 18496~34
    -78> 2016-07-05 10:09:14.044497 7f6aa96fa700  3 client.464559 ll_write
0x7f6a140d5930 18496~34 = 34
    -77> 2016-07-05 10:09:14.044507 7f6aa8cf9700  3 client.464559
ll_getxattr 1000037c78b.head security.capability size 0
    -76> 2016-07-05 10:09:14.044513 7f6aa8cf9700  3 client.464559
_getxattr(1000037c78b, "security.capability", 0) = -61
    -75> 2016-07-05 10:09:14.044519 7f6aa8cf9700  3 client.464559
ll_forget 1000037c78b 1
    -74> 2016-07-05 10:09:14.044525 7f6a5f5fe700  3 client.464559
ll_getxattr 1000037c78a.head security.capability size 0
    -73> 2016-07-05 10:09:14.044532 7f6a5f5fe700  3 client.464559
_getxattr(1000037c78a, "security.capability", 0) = -61
    -72> 2016-07-05 10:09:14.044539 7f6aa8cf9700  3 client.464559 ll_write
0x7f6a08042560 1000037c78b 12138~34
    -71> 2016-07-05 10:09:14.044558 7f6aa8cf9700  3 client.464559 ll_write
0x7f6a08042560 12138~34 = 34
    -70> 2016-07-05 10:09:14.044565 7f6a5f5fe700  3 client.464559
ll_forget 1000037c78a 1
    -69> 2016-07-05 10:09:14.044573 7f6a5ffff700  3 client.464559 ll_write
0x7f6a140d5930 1000037c78a 18530~34
    -68> 2016-07-05 10:09:14.044594 7f6a5ffff700  3 client.464559 ll_write
0x7f6a140d5930 18530~34 = 34
    -67> 2016-07-05 10:09:14.044602 7f6ac17fb700  3 client.464559
ll_getxattr 1000037c78b.head security.capability size 0
    -66> 2016-07-05 10:09:14.044607 7f6ac17fb700  3 client.464559
_getxattr(1000037c78b, "security.capability", 0) = -61
    -65> 2016-07-05 10:09:14.044612 7f6ac17fb700  3 client.464559
ll_forget 1000037c78b 1
    -64> 2016-07-05 10:09:14.044619 7f6ac17fb700  3 client.464559
ll_getxattr 1000037c78a.head security.capability size 0
    -63> 2016-07-05 10:09:14.044623 7f6ac17fb700  3 client.464559
_getxattr(1000037c78a, "security.capability", 0) = -61
    -62> 2016-07-05 10:09:14.044629 7f6a5ebfd700  3 client.464559 ll_write
0x7f6a08042560 1000037c78b 12172~34
    -61> 2016-07-05 10:09:14.044649 7f6a5ebfd700  3 client.464559 ll_write
0x7f6a08042560 12172~34 = 34
    -60> 2016-07-05 10:09:14.044657 7f6ac17fb700  3 client.464559
ll_forget 1000037c78a 1
    -59> 2016-07-05 10:09:14.044665 7f6ac21fc700  3 client.464559 ll_write
0x7f6a140d5930 1000037c78a 18564~34
    -58> 2016-07-05 10:09:14.044684 7f6ac21fc700  3 client.464559 ll_write
0x7f6a140d5930 18564~34 = 34
    -57> 2016-07-05 10:09:14.044692 7f6aab4fd700  3 client.464559
ll_getxattr 1000037c78b.head security.capability size 0
    -56> 2016-07-05 10:09:14.044697 7f6aab4fd700  3 client.464559
_getxattr(1000037c78b, "security.capability", 0) = -61
    -55> 2016-07-05 10:09:14.044701 7f6aab4fd700  3 client.464559
ll_forget 1000037c78b 1
    -54> 2016-07-05 10:09:14.044709 7f6aab4fd700  3 client.464559 ll_write
0x7f6a08042560 1000037c78b 12206~34
    -53> 2016-07-05 10:09:14.044723 7f6aab4fd700  3 client.464559 ll_write
0x7f6a08042560 12206~34 = 34
    -52> 2016-07-05 10:09:14.044732 7f6aa96fa700  3 client.464559
ll_getxattr 1000037c78a.head security.capability size 0
    -51> 2016-07-05 10:09:14.044738 7f6aa96fa700  3 client.464559
_getxattr(1000037c78a, "security.capability", 0) = -61
    -50> 2016-07-05 10:09:14.044745 7f6aa96fa700  3 client.464559
ll_forget 1000037c78a 1
    -49> 2016-07-05 10:09:14.044749 7f6ac0dfa700  3 client.464559
ll_getxattr 1000037c78b.head security.capability size 0
    -48> 2016-07-05 10:09:14.044757 7f6ac0dfa700  3 client.464559
_getxattr(1000037c78b, "security.capability", 0) = -61
    -47> 2016-07-05 10:09:14.044764 7f6aa96fa700  3 client.464559 ll_write
0x7f6a140d5930 1000037c78a 18598~34
    -46> 2016-07-05 10:09:14.044783 7f6aa96fa700  3 client.464559 ll_write
0x7f6a140d5930 18598~34 = 34
    -45> 2016-07-05 10:09:14.044790 7f6ac0dfa700  3 client.464559
ll_forget 1000037c78b 1
    -44> 2016-07-05 10:09:14.044798 7f6aa8cf9700  3 client.464559 ll_write
0x7f6a08042560 1000037c78b 12240~34
    -43> 2016-07-05 10:09:14.044815 7f6aa8cf9700  3 client.464559 ll_write
0x7f6a08042560 12240~34 = 34
    -42> 2016-07-05 10:09:14.044822 7f6a5f5fe700  3 client.464559
ll_getxattr 1000037c78a.head security.capability size 0
    -41> 2016-07-05 10:09:14.044828 7f6a5f5fe700  3 client.464559
_getxattr(1000037c78a, "security.capability", 0) = -61
    -40> 2016-07-05 10:09:14.044832 7f6a5f5fe700  3 client.464559
ll_forget 1000037c78a 1
    -39> 2016-07-05 10:09:14.044841 7f6a5f5fe700  3 client.464559
ll_getxattr 1000037c78b.head security.capability size 0
    -38> 2016-07-05 10:09:14.044845 7f6a5f5fe700  3 client.464559
_getxattr(1000037c78b, "security.capability", 0) = -61
    -37> 2016-07-05 10:09:14.044853 7f6a5ebfd700  3 client.464559 ll_write
0x7f6a140d5930 1000037c78a 18632~34
    -36> 2016-07-05 10:09:14.044869 7f6a5ebfd700  3 client.464559 ll_write
0x7f6a140d5930 18632~34 = 34
    -35> 2016-07-05 10:09:14.044876 7f6a5f5fe700  3 client.464559
ll_forget 1000037c78b 1
    -34> 2016-07-05 10:09:14.044889 7f6ac17fb700  3 client.464559 ll_write
0x7f6a08042560 1000037c78b 12274~34
    -33> 2016-07-05 10:09:14.044909 7f6ac17fb700  3 client.464559 ll_write
0x7f6a08042560 12274~34 = 34
    -32> 2016-07-05 10:09:14.044919 7f6ac21fc700  3 client.464559
ll_getxattr 1000037c78a.head security.capability size 0
    -31> 2016-07-05 10:09:14.044924 7f6ac21fc700  3 client.464559
_getxattr(1000037c78a, "security.capability", 0) = -61
    -30> 2016-07-05 10:09:14.044932 7f6ac21fc700  3 client.464559
ll_forget 1000037c78a 1
    -29> 2016-07-05 10:09:14.044936 7f6a5ffff700  3 client.464559
ll_getxattr 1000037c78b.head security.capability size 0
    -28> 2016-07-05 10:09:14.044942 7f6a5ffff700  3 client.464559
_getxattr(1000037c78b, "security.capability", 0) = -61
    -27> 2016-07-05 10:09:14.044948 7f6ac21fc700  3 client.464559 ll_write
0x7f6a140d5930 1000037c78a 18666~34
    -26> 2016-07-05 10:09:14.044970 7f6ac21fc700  3 client.464559 ll_write
0x7f6a140d5930 18666~34 = 34
    -25> 2016-07-05 10:09:14.044978 7f6a5ffff700  3 client.464559
ll_forget 1000037c78b 1
    -24> 2016-07-05 10:09:14.044987 7f6aab4fd700  3 client.464559 ll_write
0x7f6a08042560 1000037c78b 12308~34
    -23> 2016-07-05 10:09:14.045003 7f6aab4fd700  3 client.464559 ll_write
0x7f6a08042560 12308~34 = 34
    -22> 2016-07-05 10:09:14.045012 7f6aa96fa700  3 client.464559
ll_getxattr 1000037c78a.head security.capability size 0
    -21> 2016-07-05 10:09:14.045018 7f6aa96fa700  3 client.464559
_getxattr(1000037c78a, "security.capability", 0) = -61
    -20> 2016-07-05 10:09:14.045025 7f6aa96fa700  3 client.464559
ll_forget 1000037c78a 1
    -19> 2016-07-05 10:09:14.045028 7f6ac0dfa700  3 client.464559
ll_getxattr 1000037c78b.head security.capability size 0
    -18> 2016-07-05 10:09:14.045031 7f6ac0dfa700  3 client.464559
_getxattr(1000037c78b, "security.capability", 0) = -61
    -17> 2016-07-05 10:09:14.045037 7f6aa96fa700  3 client.464559 ll_write
0x7f6a140d5930 1000037c78a 18700~34
    -16> 2016-07-05 10:09:14.045053 7f6aa96fa700  3 client.464559 ll_write
0x7f6a140d5930 18700~34 = 34
    -15> 2016-07-05 10:09:14.045059 7f6ac0dfa700  3 client.464559
ll_forget 1000037c78b 1
    -14> 2016-07-05 10:09:14.045065 7f6aa8cf9700  3 client.464559 ll_write
0x7f6a08042560 1000037c78b 12342~34
    -13> 2016-07-05 10:09:14.045082 7f6aa8cf9700  3 client.464559 ll_write
0x7f6a08042560 12342~34 = 34
    -12> 2016-07-05 10:09:14.045090 7f6a5ebfd700  3 client.464559
ll_getxattr 1000037c78a.head security.capability size 0
    -11> 2016-07-05 10:09:14.045095 7f6a5ebfd700  3 client.464559
_getxattr(1000037c78a, "security.capability", 0) = -61
    -10> 2016-07-05 10:09:14.045101 7f6a5ebfd700  3 client.464559
ll_forget 1000037c78a 1
     -9> 2016-07-05 10:09:14.045112 7f6a5f5fe700  3 client.464559
ll_getxattr 1000037c78b.head security.capability size 0
     -8> 2016-07-05 10:09:14.045117 7f6a5f5fe700  3 client.464559
_getxattr(1000037c78b, "security.capability", 0) = -61
     -7> 2016-07-05 10:09:14.045122 7f6ac17fb700  3 client.464559 ll_write
0x7f6a140d5930 1000037c78a 18734~34
     -6> 2016-07-05 10:09:14.045139 7f6ac17fb700  3 client.464559 ll_write
0x7f6a140d5930 18734~34 = 34
     -5> 2016-07-05 10:09:14.045131 7f69d7fff700 -1 *** Caught signal
(Segmentation fault) **
  in thread 7f69d7fff700 thread_name:ceph-fuse

  ceph version 10.2.2 (45107e21c568dd033c2f0a3107dec8f0b0e58374)
  1: (()+0x297ef2) [0x7f6aedbecef2]
  2: (()+0x3b88c0f7e0) [0x7f6aec64b7e0]
  3: (Client::get_root_ino()+0x10) [0x7f6aedaf0330]
  4: (CephFuse::Handle::make_fake_ino(inodeno_t, snapid_t)+0x175)
[0x7f6aedaee035]
  5: (()+0x199891) [0x7f6aedaee891]
  6: (()+0x15b76) [0x7f6aed50db76]
  7: (()+0x12aa9) [0x7f6aed50aaa9]
  8: (()+0x3b88c07aa1) [0x7f6aec643aa1]
  9: (clone()+0x6d) [0x7f6aeb8d193d]
  NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed
to interpret this.

     -4> 2016-07-05 10:09:14.045146 7f6a5f5fe700  3 client.464559
ll_forget 1000037c78b 1
     -3> 2016-07-05 10:09:14.045152 7f6ac21fc700  3 client.464559 ll_write
0x7f6a08042560 1000037c78b 12376~34
     -2> 2016-07-05 10:09:14.045165 7f6ac21fc700  3 client.464559 ll_write
0x7f6a08042560 12376~34 = 34
     -1> 2016-07-05 10:09:14.045171 7f6a5ffff700  3 client.464559
ll_getxattr 1000037c78a.head security.capability size 0
      0> 2016-07-05 10:09:14.045175 7f6a5ffff700  3 client.464559
_getxattr(1000037c78a, "security.capability", 0) = -61
--- logging levels ---
    0/ 5 none
    0/ 1 lockdep
    0/ 1 context
    1/ 1 crush
    1/ 5 mds
    1/ 5 mds_balancer
    1/ 5 mds_locker
    1/ 5 mds_log
    1/ 5 mds_log_expire
    1/ 5 mds_migrator
    0/ 1 buffer
    0/ 1 timer
    0/ 1 filer
    0/ 1 striper
    0/ 1 objecter
    0/ 5 rados
    0/ 5 rbd
    0/ 5 rbd_mirror
    0/ 5 rbd_replay
    0/ 5 journaler
    0/ 5 objectcacher
    0/ 5 client
    0/ 5 osd
    0/ 5 optracker
    0/ 5 objclass
    1/ 3 filestore
    1/ 3 journal
    0/ 5 ms
    1/ 5 mon
    0/10 monc
    1/ 5 paxos
    0/ 5 tp
    1/ 5 auth
    1/ 5 crypto
    1/ 1 finisher
    1/ 5 heartbeatmap
    1/ 5 perfcounter
    1/ 5 rgw
    1/10 civetweb
    1/ 5 javaclient
    1/ 5 asok
    1/ 1 throttle
    0/ 0 refs
    1/ 5 xio
    1/ 5 compressor
    1/ 5 newstore
    1/ 5 bluestore
    1/ 5 bluefs
    1/ 3 bdev
    1/ 5 kstore
    4/ 5 rocksdb
    4/ 5 leveldb
    1/ 5 kinetic
    1/ 5 fuse
   -2/-2 (syslog threshold)
   99/99 (stderr threshold)
   max_recent     10000
   max_new         1000
   log_file
--- end dump of recent events ---


Note sure if this is relevant but we successfully compiled ceph-fuse
ourselfs (both 9.2.0 and 10.2.2) in an sl6 machine with with gcc 4.8.4,
python 2.7.6, boost 1.53.0-25 and fuse 2.9.7. We do this because of the
nature of the research performed by our scientists where there is (still)
the requirement to provide cephfs under sl6.

Any help in understanding why this is happening would be very appreciated.

Cheers

Goncalo

--
Goncalo Borges
Research Computing
ARC Centre of Excellence for Particle Physics at the Terascale
School of Physics A28 | University of Sydney, NSW  2006
T: +61 2 93511937


_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com






--
Goncalo Borges
Research Computing
ARC Centre of Excellence for Particle Physics at the Terascale
School of Physics A28 | University of Sydney, NSW  2006
T: +61 2 93511937


_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Reply via email to