Hello all,
I have a problem with my CephFS that I’m stumped on. I recently had to rebuild
a node who’s system disk failed. Once I did that, I re-created the osd
directory structure in /var/lib/ceph/osd and the osds came back into the
cluster, then had to backfill. However, I now have the problem that writing
more than a few MB into the filesystem causes the node (I’ve tried it on 6
different nodes now,) to reboot. Note that this did not happen until I brought
the osds back into the cluster. Shutting them down (or that entire node,)
doesn’t fix the problem. Updating to the latest (Centos7) kernel didn’t fix
the problem. This happens with both the kernel and fuse mounts, across several
data pools, including a newly created data pool. I’m running 14.2.6.
Sometimes I get the following message just before the system reboots (while
running an rsync job):
Message from syslogd@admin2 at Jan 24 17:21:07 ...
kernel:NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [rsync:4399]
after failing during write to kernel mounted FS, dmesg has (in the abrt crash
report):
[ 243.911318] ------------[ cut here ]------------
[ 243.911350] kernel BUG at mm/slub.c:3772!
[ 243.911368] invalid opcode: 0000 [#1] SMP
[ 243.911390] Modules linked in: fuse nfsv3 nfs_acl rpcsec_gss_krb5
auth_rpcgss nfsv4 ceph nfs libceph lockd dns_resolver grace fscache nf_nat_ftp
nf_conntrack_ftp nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter
ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat
ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6
nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat
nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle
iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter
ebtables ip6table_filter ip6_tables iptable_filter ib_isert iscsi_target_mod
ib_srpt target_core_mod ib_srp scsi_transport_srp scsi_tgt rpcrdma sunrpc
rdma_ucm ib_iser rdma_cm ib_umad iw_cm ib_ipoib libiscsi scsi_transport_iscsi
[ 243.911780] ib_cm iTCO_wdt iTCO_vendor_support sb_edac intel_powerclamp
coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass mlx4_ib crc32_pclmul
ghash_clmulni_intel ib_uverbs aesni_intel lrw gf128mul ib_core glue_helper
ablk_helper cryptd pcspkr joydev lpc_ich i2c_i801 sg mei_me mei ioatdma wmi
ipmi_ssif ipmi_si ipmi_devintf ipmi_msghandler pcc_cpufreq acpi_pad
acpi_power_meter ip_tables xfs libcrc32c mlx4_en sd_mod crc_t10dif
crct10dif_generic ast drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops ttm mlx4_core drm ahci igb libahci crct10dif_pclmul
crct10dif_common libata crc32c_intel ptp pps_core devlink
drm_panel_orientation_quirks dca i2c_algo_bit dm_mirror dm_region_hash dm_log
dm_mod
[ 243.912143] CPU: 1 PID: 0 Comm: swapper/1 Kdump: loaded Not tainted
3.10.0-1062.4.3.el7.x86_64 #1
[ 243.912177] Hardware name: Supermicro SYS-6018R-WTR/X10DRW-i, BIOS 2.0b
04/13/2017
[ 243.912207] task: ffff96b7e95a20e0 ti: ffff96b7e95b0000 task.ti:
ffff96b7e95b0000
[ 243.912236] RIP: 0010:[<ffffffffb322355c>] [<ffffffffb322355c>]
kfree+0x13c/0x140
[ 243.912271] RSP: 0018:ffff96c6bf843d60 EFLAGS: 00010246
[ 243.912293] RAX: 002fffff00000000 RBX: ffff96c6aaa60000 RCX: 0000000000000ec0
[ 243.912321] RDX: 002fffff00000000 RSI: 0000000000000001 RDI: ffff96c6aaa60000
[ 243.912349] RBP: ffff96c6bf843d78 R08: ffff96c6aaa60000 R09: ffff96b7e95b3e60
[ 243.912377] R10: 7fffffffffffffff R11: ffffe81100aa9800 R12: ffff96c6aaa60ec0
[ 243.912405] R13: ffffffffb36376c5 R14: 000000000000003d R15: ffff96c7691186a8
[ 243.912433] FS: 0000000000000000(0000) GS:ffff96c6bf840000(0000)
knlGS:0000000000000000
[ 243.912465] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 243.912488] CR2: 000055ba0b02bfb8 CR3: 000000203972e000 CR4: 00000000003607e0
[ 243.912516] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 243.912544] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 243.912572] Call Trace:
[ 243.912584] <IRQ>
[ 243.912598] [<ffffffffb36376c5>] skb_release_data+0xf5/0x140
[ 243.912625] [<ffffffffb3637734>] skb_release_all+0x24/0x30
[ 243.912650] [<ffffffffb3637c1c>] consume_skb+0x2c/0x90
[ 243.912674] [<ffffffffb364cb9d>] __dev_kfree_skb_any+0x3d/0x50
[ 243.912705] [<ffffffffc0ad8bd5>] ipoib_ib_handle_tx_wc+0x75/0x1d0 [ib_ipoib]
[ 243.912737] [<ffffffffc0ad8f3a>] ipoib_tx_poll+0x9a/0x100 [ib_ipoib]
[ 243.912765] [<ffffffffb364f57f>] net_rx_action+0x26f/0x390
[ 243.912790] [<ffffffffb30a5305>] __do_softirq+0xf5/0x280
[ 243.912815] [<ffffffffb379042c>] call_softirq+0x1c/0x30
[ 243.912840] [<ffffffffb302f715>] do_softirq+0x65/0xa0
[ 243.912862] [<ffffffffb30a5685>] irq_exit+0x105/0x110
[ 243.912884] [<ffffffffb3791886>] do_IRQ+0x56/0xf0
[ 243.913860] [<ffffffffb378336a>] common_interrupt+0x16a/0x16a
[ 243.914841] <EOI>
[ 243.914853] [<ffffffffb35c10f7>] ? cpuidle_enter_state+0x57/0xd0
[ 243.916792] [<ffffffffb35c124e>] cpuidle_idle_call+0xde/0x230
[ 243.917768] [<ffffffffb3037c6e>] arch_cpu_idle+0xe/0xc0
[ 243.918731] [<ffffffffb3100d3a>] cpu_startup_entry+0x14a/0x1e0
[ 243.919701] [<ffffffffb305a0b7>] start_secondary+0x1f7/0x270
[ 243.920650] [<ffffffffb30000d5>] start_cpu+0x5/0x14
[ 243.921567] Code: 49 8b 03 31 f6 f6 c4 40 74 04 41 8b 73 68 4c 89 df e8 89
2f fa ff eb 84 4c 8b 58 30 48 8b 10 80 e6 80 4c 0f 44 d8 e9 28 ff ff ff <0f> 0b
66 90 0f 1f 44 00 00 55 48 89 e5 41 57 41 56 41 55 41 54
[ 243.923534] RIP [<ffffffffb322355c>] kfree+0x13c/0x140
[ 243.924435] RSP <ffff96c6bf843d60>
Just before the crash, I get the following in my mds log (level 10):
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.server straydn is [dentry
#0x100/stray7/10003178634 [2,head] auth NULL (dversion lock) pv=0 v=129495990
ino=(nil) state=1342177345|new|bottomlru | request=1 0x55d0053208c0]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.server witnesses , have
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker acquire_locks
request(client.77200959:238582 cr=0x55d019672580)
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker must authpin [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json [2,head] auth
(dversion lock) v=13929147 ino=0x10003178634 state=1610612736 | request=0
lock=0 inodepin=1 dirty=1 waiter=0 authpin=0 0x55cfeeef3dc0]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker must authpin [inode 0x607
[...2,head] ~mds0/stray7/ auth v89339680 f(v40 m2020-01-24 16:08:43.778298
211=160+51) n(v130 rc2020-01-24 16:08:43.778298 b81704593 212=160+52) (inest
lock) (iversion lock) | dirtyscattered=0 lock=0 dirfrag=1 stickydirs=1 stray=1
dirtyrstat=0 dirty=1 authpin=0 0x55cfeff6f800]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker must authpin [inode 0x1
[...2,head] / auth v34216266 snaprealm=0x55cfeff4a280 f(v0 m2020-01-17
12:41:39.137684 5=0+5) n(v7229910 rc2020-01-24 16:08:38.595298 b441687361105836
23052011=22719256+332755)/n(v0 1=0+1) (inest lock dirty) (iversion lock)
caps={48654103=pAsLsXsFs/-@0,49033239=pAsLsXsFs/-@0,68004815=pAsLsXsFs/-@4,77200959=pAsLsXsFs/-@0,77246902=pAsLsXsFs/-@0}
| dirtyscattered=1 lock=0 dirfrag=1 caps=1 openingsnapparents=0 dirty=1
authpin=0 0x55cfeff94700]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker must authpin [inode
0x10000000001 [...2,head] /home/ auth v98722180 f(v0 m2020-01-20
09:56:37.772033 65=0+65) n(v7735020 rc2020-01-24 16:08:47.195918
b164276032854114 16350398=16221677+128721) (inest lock dirty) (iversion lock)
caps={49033239=pAsLsXs/-@0,75454214=pAsLsXsFs/-@4,76406416=pAsLsXsFs/-@0,76407061=pAsLsXsFs/-@0,76407436=pAsLsXsFs/-@0,77200959=pAsLsXsFs/-@0,77246902=pAsLsXsFs/AsLsXsFsx@4,77586859=pAsLsXsFs/-@0,77605502=pAsLsXsFs/-@2,77643571=pAsLsXsFs/-@1}
| dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirty=1
authpin=0 0x55cfeff94e00]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker must authpin [inode
0x10000000002 [...2,head] /home/tjr22/ auth v131045958 f(v31 m2020-01-24
16:08:21.091241 189=91+98) n(v765289 rc2020-01-24 16:08:47.195918
b8869319870466 1292992=1236041+56951) (inest lock dirty) (iversion lock)
caps={49033239=pAsLsXs/-@0,75454214=pAsLsXsFs/-@1513,77200959=pAsLsXsFs/-@327,77246902=pAsLsXs/-@2,77605502=pAsLsXs/p@50}
| dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirty=1
waiter=0 authpin=0 0x55cfeff95500]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker must authpin [inode
0x10000000a52 [...2,head] /home/tjr22/.config/ auth v129496029 f(v0 m2019-12-13
13:57:58.591154 31=6+25) n(v440444 rc2020-01-24 16:08:47.195918 b336426
108=63+45) (inest lock dirty) (iversion lock)
caps={49033239=pAsLsXs/-@0,75454214=pAsLsXsFs/-@69,77200959=pAsLsXsFs/-@64,77605502=pAsLsXsFs/-@2}
| dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirty=1
authpin=0 0x55cfeff95c00]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker must authpin [inode
0x1000268e7cb [...2,head] /home/tjr22/.config/configstore/ auth v15636199 f(v0
m2020-01-24 16:08:47.195918 2=2+0) n(v8 rc2020-01-24 16:08:47.246973 b700
3=2+1) (inest lock) (ifile excl) (iversion lock)
caps={77200959=pAsLsXsFsx/-@30413},l=77200959 | dirtyscattered=0 lock=0
dirfrag=1 caps=1 dirtyrstat=0 dirty=1 authpin=0 0x55cfeff96300]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker must authpin [dentry
#0x100/stray7/10003178634 [2,head] auth NULL (dversion lock) pv=0 v=129495990
ino=(nil) state=1342177345|new|bottomlru | request=1 0x55d0053208c0]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker must authpin [inode
0x10003178634 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json auth v13929147
dirtyparent s=350 n(v0 rc2020-01-24 16:08:43.778298 b350 1=1+0)/n(v0
rc2020-01-24 16:08:43.773910 b350 1=1+0) (iversion lock)
caps={77200959=pAsXsFscr/-@6} | ptrwaiter=0 request=0 lock=0 caps=1
dirtyparent=1 dirty=1 authpin=0 0x55d006c3a000]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker must authpin [inode
0x10003178635 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719 auth
v13929158 dirtyparent s=350 n(v0 rc2020-01-24 16:08:47.246973 b350 1=1+0)
(iversion lock) caps={77200959=pAsXsFscr/-@4} | ptrwaiter=0 request=0 lock=0
caps=1 dirtyparent=1 dirty=1 authpin=0 0x55d006c3b500]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker must authpin [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719
[2,head] auth (dversion lock) v=13929158 ino=0x10003178635 state=1610612736 |
request=0 lock=0 inodepin=1 dirty=1 authpin=0 0x55d018f10000]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker auth_pinning [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json [2,head] auth
(dversion lock) v=13929147 ino=0x10003178634 state=1610612736 | request=0
lock=0 inodepin=1 dirty=1 waiter=0 authpin=0 0x55cfeeef3dc0]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.den(0x1000268e7cb
tus-node-server-0.1.1.json) auth_pin by 0x55d01a129400 on [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json [2,head] auth
(dversion lock) v=13929147 ap=1 ino=0x10003178634 state=1610612736 | request=0
lock=0 inodepin=1 dirty=1 waiter=0 authpin=1 0x55cfeeef3dc0] now 1
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker auth_pinning [inode 0x607
[...2,head] ~mds0/stray7/ auth v89339680 f(v40 m2020-01-24 16:08:43.778298
211=160+51) n(v130 rc2020-01-24 16:08:43.778298 b81704593 212=160+52) (inest
lock) (iversion lock) | dirtyscattered=0 lock=0 dirfrag=1 stickydirs=1 stray=1
dirtyrstat=0 dirty=1 authpin=0 0x55cfeff6f800]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.ino(0x607) auth_pin by
0x55d01a129400 on [inode 0x607 [...2,head] ~mds0/stray7/ auth v89339680 ap=1
f(v40 m2020-01-24 16:08:43.778298 211=160+51) n(v130 rc2020-01-24
16:08:43.778298 b81704593 212=160+52) (inest lock) (iversion lock) |
dirtyscattered=0 lock=0 dirfrag=1 stickydirs=1 stray=1 dirtyrstat=0 dirty=1
authpin=1 0x55cfeff6f800] now 1
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker auth_pinning [inode 0x1
[...2,head] / auth v34216266 snaprealm=0x55cfeff4a280 f(v0 m2020-01-17
12:41:39.137684 5=0+5) n(v7229910 rc2020-01-24 16:08:38.595298 b441687361105836
23052011=22719256+332755)/n(v0 1=0+1) (inest lock dirty) (iversion lock)
caps={48654103=pAsLsXsFs/-@0,49033239=pAsLsXsFs/-@0,68004815=pAsLsXsFs/-@4,77200959=pAsLsXsFs/-@0,77246902=pAsLsXsFs/-@0}
| dirtyscattered=1 lock=0 dirfrag=1 caps=1 openingsnapparents=0 dirty=1
authpin=0 0x55cfeff94700]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.ino(0x1) auth_pin by
0x55d01a129400 on [inode 0x1 [...2,head] / auth v34216266 ap=1
snaprealm=0x55cfeff4a280 f(v0 m2020-01-17 12:41:39.137684 5=0+5) n(v7229910
rc2020-01-24 16:08:38.595298 b441687361105836 23052011=22719256+332755)/n(v0
1=0+1) (inest lock dirty) (iversion lock)
caps={48654103=pAsLsXsFs/-@0,49033239=pAsLsXsFs/-@0,68004815=pAsLsXsFs/-@4,77200959=pAsLsXsFs/-@0,77246902=pAsLsXsFs/-@0}
| dirtyscattered=1 lock=0 dirfrag=1 caps=1 openingsnapparents=0 dirty=1
authpin=1 0x55cfeff94700] now 1
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker auth_pinning [inode
0x10000000001 [...2,head] /home/ auth v98722180 f(v0 m2020-01-20
09:56:37.772033 65=0+65) n(v7735020 rc2020-01-24 16:08:47.195918
b164276032854114 16350398=16221677+128721) (inest lock dirty) (iversion lock)
caps={49033239=pAsLsXs/-@0,75454214=pAsLsXsFs/-@4,76406416=pAsLsXsFs/-@0,76407061=pAsLsXsFs/-@0,76407436=pAsLsXsFs/-@0,77200959=pAsLsXsFs/-@0,77246902=pAsLsXsFs/AsLsXsFsx@4,77586859=pAsLsXsFs/-@0,77605502=pAsLsXsFs/-@2,77643571=pAsLsXsFs/-@1}
| dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirty=1
authpin=0 0x55cfeff94e00]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.ino(0x10000000001) auth_pin
by 0x55d01a129400 on [inode 0x10000000001 [...2,head] /home/ auth v98722180
ap=1 f(v0 m2020-01-20 09:56:37.772033 65=0+65) n(v7735020 rc2020-01-24
16:08:47.195918 b164276032854114 16350398=16221677+128721) (inest lock dirty)
(iversion lock)
caps={49033239=pAsLsXs/-@0,75454214=pAsLsXsFs/-@4,76406416=pAsLsXsFs/-@0,76407061=pAsLsXsFs/-@0,76407436=pAsLsXsFs/-@0,77200959=pAsLsXsFs/-@0,77246902=pAsLsXsFs/AsLsXsFsx@4,77586859=pAsLsXsFs/-@0,77605502=pAsLsXsFs/-@2,77643571=pAsLsXsFs/-@1}
| dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirty=1
authpin=1 0x55cfeff94e00] now 1
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker auth_pinning [inode
0x10000000002 [...2,head] /home/tjr22/ auth v131045958 f(v31 m2020-01-24
16:08:21.091241 189=91+98) n(v765289 rc2020-01-24 16:08:47.195918
b8869319870466 1292992=1236041+56951) (inest lock dirty) (iversion lock)
caps={49033239=pAsLsXs/-@0,75454214=pAsLsXsFs/-@1513,77200959=pAsLsXsFs/-@327,77246902=pAsLsXs/-@2,77605502=pAsLsXs/p@50}
| dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirty=1
waiter=0 authpin=0 0x55cfeff95500]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.ino(0x10000000002) auth_pin
by 0x55d01a129400 on [inode 0x10000000002 [...2,head] /home/tjr22/ auth
v131045958 ap=1 f(v31 m2020-01-24 16:08:21.091241 189=91+98) n(v765289
rc2020-01-24 16:08:47.195918 b8869319870466 1292992=1236041+56951) (inest lock
dirty) (iversion lock)
caps={49033239=pAsLsXs/-@0,75454214=pAsLsXsFs/-@1513,77200959=pAsLsXsFs/-@327,77246902=pAsLsXs/-@2,77605502=pAsLsXs/p@50}
| dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirty=1
waiter=0 authpin=1 0x55cfeff95500] now 1
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker auth_pinning [inode
0x10000000a52 [...2,head] /home/tjr22/.config/ auth v129496029 f(v0 m2019-12-13
13:57:58.591154 31=6+25) n(v440444 rc2020-01-24 16:08:47.195918 b336426
108=63+45) (inest lock dirty) (iversion lock)
caps={49033239=pAsLsXs/-@0,75454214=pAsLsXsFs/-@69,77200959=pAsLsXsFs/-@64,77605502=pAsLsXsFs/-@2}
| dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirty=1
authpin=0 0x55cfeff95c00]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.ino(0x10000000a52) auth_pin
by 0x55d01a129400 on [inode 0x10000000a52 [...2,head] /home/tjr22/.config/ auth
v129496029 ap=1 f(v0 m2019-12-13 13:57:58.591154 31=6+25) n(v440444
rc2020-01-24 16:08:47.195918 b336426 108=63+45) (inest lock dirty) (iversion
lock)
caps={49033239=pAsLsXs/-@0,75454214=pAsLsXsFs/-@69,77200959=pAsLsXsFs/-@64,77605502=pAsLsXsFs/-@2}
| dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirty=1
authpin=1 0x55cfeff95c00] now 1
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker auth_pinning [inode
0x1000268e7cb [...2,head] /home/tjr22/.config/configstore/ auth v15636199 f(v0
m2020-01-24 16:08:47.195918 2=2+0) n(v8 rc2020-01-24 16:08:47.246973 b700
3=2+1) (inest lock) (ifile excl) (iversion lock)
caps={77200959=pAsLsXsFsx/-@30413},l=77200959 | dirtyscattered=0 lock=0
dirfrag=1 caps=1 dirtyrstat=0 dirty=1 authpin=0 0x55cfeff96300]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.ino(0x1000268e7cb) auth_pin
by 0x55d01a129400 on [inode 0x1000268e7cb [...2,head]
/home/tjr22/.config/configstore/ auth v15636199 ap=1 f(v0 m2020-01-24
16:08:47.195918 2=2+0) n(v8 rc2020-01-24 16:08:47.246973 b700 3=2+1) (inest
lock) (ifile excl) (iversion lock)
caps={77200959=pAsLsXsFsx/-@30413},l=77200959 | dirtyscattered=0 lock=0
dirfrag=1 caps=1 dirtyrstat=0 dirty=1 authpin=1 0x55cfeff96300] now 1
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker auth_pinning [dentry
#0x100/stray7/10003178634 [2,head] auth NULL (dversion lock) pv=0 v=129495990
ino=(nil) state=1342177345|new|bottomlru | request=1 0x55d0053208c0]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.den(0x607 10003178634)
auth_pin by 0x55d01a129400 on [dentry #0x100/stray7/10003178634 [2,head] auth
NULL (dversion lock) pv=0 v=129495990 ap=1 ino=(nil)
state=1342177345|new|bottomlru | request=1 authpin=1 0x55d0053208c0] now 1
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker auth_pinning [inode
0x10003178634 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json auth v13929147
dirtyparent s=350 n(v0 rc2020-01-24 16:08:43.778298 b350 1=1+0)/n(v0
rc2020-01-24 16:08:43.773910 b350 1=1+0) (iversion lock)
caps={77200959=pAsXsFscr/-@6} | ptrwaiter=0 request=0 lock=0 caps=1
dirtyparent=1 dirty=1 authpin=0 0x55d006c3a000]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.ino(0x10003178634) auth_pin
by 0x55d01a129400 on [inode 0x10003178634 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json auth v13929147 ap=1
dirtyparent s=350 n(v0 rc2020-01-24 16:08:43.778298 b350 1=1+0)/n(v0
rc2020-01-24 16:08:43.773910 b350 1=1+0) (iversion lock)
caps={77200959=pAsXsFscr/-@6} | ptrwaiter=0 request=0 lock=0 caps=1
dirtyparent=1 dirty=1 authpin=1 0x55d006c3a000] now 1
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker auth_pinning [inode
0x10003178635 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719 auth
v13929158 dirtyparent s=350 n(v0 rc2020-01-24 16:08:47.246973 b350 1=1+0)
(iversion lock) caps={77200959=pAsXsFscr/-@4} | ptrwaiter=0 request=0 lock=0
caps=1 dirtyparent=1 dirty=1 authpin=0 0x55d006c3b500]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.ino(0x10003178635) auth_pin
by 0x55d01a129400 on [inode 0x10003178635 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719 auth
v13929158 ap=1 dirtyparent s=350 n(v0 rc2020-01-24 16:08:47.246973 b350 1=1+0)
(iversion lock) caps={77200959=pAsXsFscr/-@4} | ptrwaiter=0 request=0 lock=0
caps=1 dirtyparent=1 dirty=1 authpin=1 0x55d006c3b500] now 1
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker auth_pinning [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719
[2,head] auth (dversion lock) v=13929158 ino=0x10003178635 state=1610612736 |
request=0 lock=0 inodepin=1 dirty=1 authpin=0 0x55d018f10000]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.den(0x1000268e7cb
tus-node-server-0.1.1.json.2465308719) auth_pin by 0x55d01a129400 on [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719
[2,head] auth (dversion lock) v=13929158 ap=1 ino=0x10003178635
state=1610612736 | request=0 lock=0 inodepin=1 dirty=1 authpin=1
0x55d018f10000] now 1
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker local_wrlock_start on
(dversion lock) on [dentry #0x100/stray7/10003178634 [2,head] auth NULL
(dversion lock) pv=0 v=129495990 ap=1 ino=(nil) state=1342177345|new|bottomlru
| request=1 authpin=1 0x55d0053208c0]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got wrlock on (dversion
lock w=1 last_client=77200959) [dentry #0x100/stray7/10003178634 [2,head] auth
NULL (dversion lock w=1 last_client=77200959) pv=0 v=129495990 ap=1 ino=(nil)
state=1342177345|new|bottomlru | request=1 lock=1 authpin=1 0x55d0053208c0]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker xlock_start on (dn sync)
on [dentry #0x100/stray7/10003178634 [2,head] auth NULL (dversion lock w=1
last_client=77200959) pv=0 v=129495990 ap=1 ino=(nil)
state=1342177345|new|bottomlru | request=1 lock=1 authpin=1 0x55d0053208c0]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker simple_lock on (dn sync)
on [dentry #0x100/stray7/10003178634 [2,head] auth NULL (dversion lock w=1
last_client=77200959) pv=0 v=129495990 ap=1 ino=(nil)
state=1342177345|new|bottomlru | request=1 lock=1 authpin=1 0x55d0053208c0]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker simple_xlock on (dn lock)
on [dentry #0x100/stray7/10003178634 [2,head] auth NULL (dn lock) (dversion
lock w=1 last_client=77200959) pv=0 v=129495990 ap=1 ino=(nil)
state=1342177345|new|bottomlru | request=1 lock=1 authpin=1 0x55d0053208c0]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.den(0x607 10003178634)
auth_pin by 0x55d005320980 on [dentry #0x100/stray7/10003178634 [2,head] auth
NULL (dn lock) (dversion lock w=1 last_client=77200959) pv=0 v=129495990 ap=2
ino=(nil) state=1342177345|new|bottomlru | request=1 lock=1 authpin=1
0x55d0053208c0] now 2
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got xlock on (dn xlock
x=1 by 0x55d01a129400) [dentry #0x100/stray7/10003178634 [2,head] auth NULL (dn
xlock x=1 by 0x55d01a129400) (dversion lock w=1 last_client=77200959) pv=0
v=129495990 ap=2 ino=(nil) state=1342177345|new|bottomlru | request=1 lock=2
authpin=1 0x55d0053208c0]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker local_wrlock_start on
(dversion lock) on [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json [2,head] auth
(dversion lock) v=13929147 ap=1 ino=0x10003178634 state=1610612736 | request=0
lock=0 inodepin=1 dirty=1 waiter=0 authpin=1 0x55cfeeef3dc0]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got wrlock on (dversion
lock w=1 last_client=77200959) [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json [2,head] auth
(dversion lock w=1 last_client=77200959) v=13929147 ap=1 ino=0x10003178634
state=1610612736 | request=0 lock=1 inodepin=1 dirty=1 waiter=0 authpin=1
0x55cfeeef3dc0]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker xlock_start on (dn sync)
on [dentry #0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json
[2,head] auth (dversion lock w=1 last_client=77200959) v=13929147 ap=1
ino=0x10003178634 state=1610612736 | request=0 lock=1 inodepin=1 dirty=1
waiter=0 authpin=1 0x55cfeeef3dc0]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker simple_lock on (dn sync)
on [dentry #0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json
[2,head] auth (dversion lock w=1 last_client=77200959) v=13929147 ap=1
ino=0x10003178634 state=1610612736 | request=0 lock=1 inodepin=1 dirty=1
waiter=0 authpin=1 0x55cfeeef3dc0]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker simple_xlock on (dn lock)
on [dentry #0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json
[2,head] auth (dn lock) (dversion lock w=1 last_client=77200959) v=13929147
ap=1 ino=0x10003178634 state=1610612736 | request=1 lock=1 inodepin=1 dirty=1
waiter=0 authpin=1 0x55cfeeef3dc0]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.den(0x1000268e7cb
tus-node-server-0.1.1.json) auth_pin by 0x55cfeeef3e80 on [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json [2,head] auth
(dn lock) (dversion lock w=1 last_client=77200959) v=13929147 ap=2
ino=0x10003178634 state=1610612736 | request=1 lock=1 inodepin=1 dirty=1
waiter=0 authpin=1 0x55cfeeef3dc0] now 2
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got xlock on (dn xlock
x=1 by 0x55d01a129400) [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json [2,head] auth
(dn xlock x=1 by 0x55d01a129400) (dversion lock w=1 last_client=77200959)
v=13929147 ap=2 ino=0x10003178634 state=1610612736 | request=1 lock=2
inodepin=1 dirty=1 waiter=0 authpin=1 0x55cfeeef3dc0]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker local_wrlock_start on
(dversion lock) on [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719
[2,head] auth (dversion lock) v=13929158 ap=1 ino=0x10003178635
state=1610612736 | request=0 lock=0 inodepin=1 dirty=1 authpin=1 0x55d018f10000]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got wrlock on (dversion
lock w=1 last_client=77200959) [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719
[2,head] auth (dversion lock w=1 last_client=77200959) v=13929158 ap=1
ino=0x10003178635 state=1610612736 | request=0 lock=1 inodepin=1 dirty=1
authpin=1 0x55d018f10000]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker xlock_start on (dn sync)
on [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719
[2,head] auth (dversion lock w=1 last_client=77200959) v=13929158 ap=1
ino=0x10003178635 state=1610612736 | request=0 lock=1 inodepin=1 dirty=1
authpin=1 0x55d018f10000]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker simple_lock on (dn sync)
on [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719
[2,head] auth (dversion lock w=1 last_client=77200959) v=13929158 ap=1
ino=0x10003178635 state=1610612736 | request=0 lock=1 inodepin=1 dirty=1
authpin=1 0x55d018f10000]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker simple_xlock on (dn lock)
on [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719
[2,head] auth (dn lock) (dversion lock w=1 last_client=77200959) v=13929158
ap=1 ino=0x10003178635 state=1610612736 | request=1 lock=1 inodepin=1 dirty=1
authpin=1 0x55d018f10000]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.den(0x1000268e7cb
tus-node-server-0.1.1.json.2465308719) auth_pin by 0x55d018f100c0 on [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719
[2,head] auth (dn lock) (dversion lock w=1 last_client=77200959) v=13929158
ap=2 ino=0x10003178635 state=1610612736 | request=1 lock=1 inodepin=1 dirty=1
authpin=1 0x55d018f10000] now 2
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got xlock on (dn xlock
x=1 by 0x55d01a129400) [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719
[2,head] auth (dn xlock x=1 by 0x55d01a129400) (dversion lock w=1
last_client=77200959) v=13929158 ap=2 ino=0x10003178635 state=1610612736 |
request=1 lock=2 inodepin=1 dirty=1 authpin=1 0x55d018f10000]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker rdlock_start on (isnap
sync) on [inode 0x1 [...2,head] / auth v34216266 ap=1 snaprealm=0x55cfeff4a280
f(v0 m2020-01-17 12:41:39.137684 5=0+5) n(v7229910 rc2020-01-24 16:08:38.595298
b441687361105836 23052011=22719256+332755)/n(v0 1=0+1) (inest lock dirty)
(iversion lock)
caps={48654103=pAsLsXsFs/-@0,49033239=pAsLsXsFs/-@0,68004815=pAsLsXsFs/-@4,77200959=pAsLsXsFs/-@0,77246902=pAsLsXsFs/-@0}
| dirtyscattered=1 lock=0 dirfrag=1 caps=1 openingsnapparents=0 dirty=1
authpin=1 0x55cfeff94700]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got rdlock on (isnap sync
r=1) [inode 0x1 [...2,head] / auth v34216266 ap=1 snaprealm=0x55cfeff4a280 f(v0
m2020-01-17 12:41:39.137684 5=0+5) n(v7229910 rc2020-01-24 16:08:38.595298
b441687361105836 23052011=22719256+332755)/n(v0 1=0+1) (isnap sync r=1) (inest
lock dirty) (iversion lock)
caps={48654103=pAsLsXsFs/-@0,49033239=pAsLsXsFs/-@0,68004815=pAsLsXsFs/-@4,77200959=pAsLsXsFs/-@0,77246902=pAsLsXsFs/-@0}
| dirtyscattered=1 lock=1 dirfrag=1 caps=1 openingsnapparents=0 dirty=1
authpin=1 0x55cfeff94700]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker wrlock_start (ifile sync)
on [inode 0x607 [...2,head] ~mds0/stray7/ auth v89339680 ap=1 f(v40 m2020-01-24
16:08:43.778298 211=160+51) n(v130 rc2020-01-24 16:08:43.778298 b81704593
212=160+52) (inest lock) (iversion lock) | dirtyscattered=0 lock=0 dirfrag=1
stickydirs=1 stray=1 dirtyrstat=0 dirty=1 authpin=1 0x55cfeff6f800]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker simple_lock on (ifile
sync) on [inode 0x607 [...2,head] ~mds0/stray7/ auth v89339680 ap=1 f(v40
m2020-01-24 16:08:43.778298 211=160+51) n(v130 rc2020-01-24 16:08:43.778298
b81704593 212=160+52) (inest lock) (iversion lock) | dirtyscattered=0 lock=0
dirfrag=1 stickydirs=1 stray=1 dirtyrstat=0 dirty=1 authpin=1 0x55cfeff6f800]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got wrlock on (ifile lock
w=1) [inode 0x607 [...2,head] ~mds0/stray7/ auth v89339680 ap=1 f(v40
m2020-01-24 16:08:43.778298 211=160+51) n(v130 rc2020-01-24 16:08:43.778298
b81704593 212=160+52) (inest lock) (ifile lock w=1) (iversion lock) |
dirtyscattered=0 lock=1 dirfrag=1 stickydirs=1 stray=1 dirtyrstat=0 dirty=1
authpin=1 0x55cfeff6f800]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker wrlock_start (inest lock)
on [inode 0x607 [...2,head] ~mds0/stray7/ auth v89339680 ap=1 f(v40 m2020-01-24
16:08:43.778298 211=160+51) n(v130 rc2020-01-24 16:08:43.778298 b81704593
212=160+52) (inest lock) (ifile lock w=1) (iversion lock) | dirtyscattered=0
lock=1 dirfrag=1 stickydirs=1 stray=1 dirtyrstat=0 dirty=1 authpin=1
0x55cfeff6f800]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got wrlock on (inest lock
w=1) [inode 0x607 [...2,head] ~mds0/stray7/ auth v89339680 ap=1 f(v40
m2020-01-24 16:08:43.778298 211=160+51) n(v130 rc2020-01-24 16:08:43.778298
b81704593 212=160+52) (inest lock w=1) (ifile lock w=1) (iversion lock) |
dirtyscattered=0 lock=2 dirfrag=1 stickydirs=1 stray=1 dirtyrstat=0 dirty=1
authpin=1 0x55cfeff6f800]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker rdlock_start on (isnap
sync) on [inode 0x10000000001 [...2,head] /home/ auth v98722180 ap=1 f(v0
m2020-01-20 09:56:37.772033 65=0+65) n(v7735020 rc2020-01-24 16:08:47.195918
b164276032854114 16350398=16221677+128721) (inest lock dirty) (iversion lock)
caps={49033239=pAsLsXs/-@0,75454214=pAsLsXsFs/-@4,76406416=pAsLsXsFs/-@0,76407061=pAsLsXsFs/-@0,76407436=pAsLsXsFs/-@0,77200959=pAsLsXsFs/-@0,77246902=pAsLsXsFs/AsLsXsFsx@4,77586859=pAsLsXsFs/-@0,77605502=pAsLsXsFs/-@2,77643571=pAsLsXsFs/-@1}
| dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirty=1
authpin=1 0x55cfeff94e00]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got rdlock on (isnap sync
r=1) [inode 0x10000000001 [...2,head] /home/ auth v98722180 ap=1 f(v0
m2020-01-20 09:56:37.772033 65=0+65) n(v7735020 rc2020-01-24 16:08:47.195918
b164276032854114 16350398=16221677+128721) (isnap sync r=1) (inest lock dirty)
(iversion lock)
caps={49033239=pAsLsXs/-@0,75454214=pAsLsXsFs/-@4,76406416=pAsLsXsFs/-@0,76407061=pAsLsXsFs/-@0,76407436=pAsLsXsFs/-@0,77200959=pAsLsXsFs/-@0,77246902=pAsLsXsFs/AsLsXsFsx@4,77586859=pAsLsXsFs/-@0,77605502=pAsLsXsFs/-@2,77643571=pAsLsXsFs/-@1}
| dirtyscattered=1 request=0 lock=1 dirfrag=1 caps=1 dirtyrstat=0 dirty=1
authpin=1 0x55cfeff94e00]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker rdlock_start on (isnap
sync) on [inode 0x10000000002 [...2,head] /home/tjr22/ auth v131045958 ap=1
f(v31 m2020-01-24 16:08:21.091241 189=91+98) n(v765289 rc2020-01-24
16:08:47.195918 b8869319870466 1292992=1236041+56951) (inest lock dirty)
(iversion lock)
caps={49033239=pAsLsXs/-@0,75454214=pAsLsXsFs/-@1513,77200959=pAsLsXsFs/-@327,77246902=pAsLsXs/-@2,77605502=pAsLsXs/p@50}
| dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirty=1
waiter=0 authpin=1 0x55cfeff95500]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got rdlock on (isnap sync
r=1) [inode 0x10000000002 [...2,head] /home/tjr22/ auth v131045958 ap=1 f(v31
m2020-01-24 16:08:21.091241 189=91+98) n(v765289 rc2020-01-24 16:08:47.195918
b8869319870466 1292992=1236041+56951) (isnap sync r=1) (inest lock dirty)
(iversion lock)
caps={49033239=pAsLsXs/-@0,75454214=pAsLsXsFs/-@1513,77200959=pAsLsXsFs/-@327,77246902=pAsLsXs/-@2,77605502=pAsLsXs/p@50}
| dirtyscattered=1 request=0 lock=1 dirfrag=1 caps=1 dirtyrstat=0 dirty=1
waiter=0 authpin=1 0x55cfeff95500]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker rdlock_start on (isnap
sync) on [inode 0x10000000a52 [...2,head] /home/tjr22/.config/ auth v129496029
ap=1 f(v0 m2019-12-13 13:57:58.591154 31=6+25) n(v440444 rc2020-01-24
16:08:47.195918 b336426 108=63+45) (inest lock dirty) (iversion lock)
caps={49033239=pAsLsXs/-@0,75454214=pAsLsXsFs/-@69,77200959=pAsLsXsFs/-@64,77605502=pAsLsXsFs/-@2}
| dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirty=1
authpin=1 0x55cfeff95c00]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got rdlock on (isnap sync
r=1) [inode 0x10000000a52 [...2,head] /home/tjr22/.config/ auth v129496029 ap=1
f(v0 m2019-12-13 13:57:58.591154 31=6+25) n(v440444 rc2020-01-24
16:08:47.195918 b336426 108=63+45) (isnap sync r=1) (inest lock dirty)
(iversion lock)
caps={49033239=pAsLsXs/-@0,75454214=pAsLsXsFs/-@69,77200959=pAsLsXsFs/-@64,77605502=pAsLsXsFs/-@2}
| dirtyscattered=1 request=0 lock=1 dirfrag=1 caps=1 dirtyrstat=0 dirty=1
authpin=1 0x55cfeff95c00]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker rdlock_start on (isnap
sync) on [inode 0x1000268e7cb [...2,head] /home/tjr22/.config/configstore/ auth
v15636199 ap=1 f(v0 m2020-01-24 16:08:47.195918 2=2+0) n(v8 rc2020-01-24
16:08:47.246973 b700 3=2+1) (inest lock) (ifile excl) (iversion lock)
caps={77200959=pAsLsXsFsx/-@30413},l=77200959 | dirtyscattered=0 lock=0
dirfrag=1 caps=1 dirtyrstat=0 dirty=1 authpin=1 0x55cfeff96300]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got rdlock on (isnap sync
r=1) [inode 0x1000268e7cb [...2,head] /home/tjr22/.config/configstore/ auth
v15636199 ap=1 f(v0 m2020-01-24 16:08:47.195918 2=2+0) n(v8 rc2020-01-24
16:08:47.246973 b700 3=2+1) (isnap sync r=1) (inest lock) (ifile excl)
(iversion lock) caps={77200959=pAsLsXsFsx/-@30413},l=77200959 |
dirtyscattered=0 lock=1 dirfrag=1 caps=1 dirtyrstat=0 dirty=1 authpin=1
0x55cfeff96300]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker wrlock_start (ifile excl)
on [inode 0x1000268e7cb [...2,head] /home/tjr22/.config/configstore/ auth
v15636199 ap=1 f(v0 m2020-01-24 16:08:47.195918 2=2+0) n(v8 rc2020-01-24
16:08:47.246973 b700 3=2+1) (isnap sync r=1) (inest lock) (ifile excl)
(iversion lock) caps={77200959=pAsLsXsFsx/-@30413},l=77200959 |
dirtyscattered=0 lock=1 dirfrag=1 caps=1 dirtyrstat=0 dirty=1 authpin=1
0x55cfeff96300]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got wrlock on (ifile excl
w=1) [inode 0x1000268e7cb [...2,head] /home/tjr22/.config/configstore/ auth
v15636199 ap=1 f(v0 m2020-01-24 16:08:47.195918 2=2+0) n(v8 rc2020-01-24
16:08:47.246973 b700 3=2+1) (isnap sync r=1) (inest lock) (ifile excl w=1)
(iversion lock) caps={77200959=pAsLsXsFsx/-@30413},l=77200959 |
dirtyscattered=0 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirty=1 authpin=1
0x55cfeff96300]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker wrlock_start (inest lock)
on [inode 0x1000268e7cb [...2,head] /home/tjr22/.config/configstore/ auth
v15636199 ap=1 f(v0 m2020-01-24 16:08:47.195918 2=2+0) n(v8 rc2020-01-24
16:08:47.246973 b700 3=2+1) (isnap sync r=1) (inest lock) (ifile excl w=1)
(iversion lock) caps={77200959=pAsLsXsFsx/-@30413},l=77200959 |
dirtyscattered=0 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirty=1 authpin=1
0x55cfeff96300]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got wrlock on (inest lock
w=1) [inode 0x1000268e7cb [...2,head] /home/tjr22/.config/configstore/ auth
v15636199 ap=1 f(v0 m2020-01-24 16:08:47.195918 2=2+0) n(v8 rc2020-01-24
16:08:47.246973 b700 3=2+1) (isnap sync r=1) (inest lock w=1) (ifile excl w=1)
(iversion lock) caps={77200959=pAsLsXsFsx/-@30413},l=77200959 |
dirtyscattered=0 lock=3 dirfrag=1 caps=1 dirtyrstat=0 dirty=1 authpin=1
0x55cfeff96300]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker local_wrlock_start on
(iversion lock) on [inode 0x10003178634 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json auth v13929147 ap=1
dirtyparent s=350 n(v0 rc2020-01-24 16:08:43.778298 b350 1=1+0)/n(v0
rc2020-01-24 16:08:43.773910 b350 1=1+0) (iversion lock)
caps={77200959=pAsXsFscr/-@6} | ptrwaiter=0 request=0 lock=0 caps=1
dirtyparent=1 dirty=1 authpin=1 0x55d006c3a000]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got wrlock on (iversion
lock w=1 last_client=77200959) [inode 0x10003178634 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json auth v13929147 ap=1
dirtyparent s=350 n(v0 rc2020-01-24 16:08:43.778298 b350 1=1+0)/n(v0
rc2020-01-24 16:08:43.773910 b350 1=1+0) (iversion lock w=1
last_client=77200959) caps={77200959=pAsXsFscr/-@6} | ptrwaiter=0 request=0
lock=1 caps=1 dirtyparent=1 dirty=1 authpin=1 0x55d006c3a000]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker xlock_start on (isnap
sync) on [inode 0x10003178634 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json auth v13929147 ap=1
dirtyparent s=350 n(v0 rc2020-01-24 16:08:43.778298 b350 1=1+0)/n(v0
rc2020-01-24 16:08:43.773910 b350 1=1+0) (iversion lock w=1
last_client=77200959) caps={77200959=pAsXsFscr/-@6} | ptrwaiter=0 request=0
lock=1 caps=1 dirtyparent=1 dirty=1 authpin=1 0x55d006c3a000]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker simple_lock on (isnap
sync) on [inode 0x10003178634 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json auth v13929147 ap=1
dirtyparent s=350 n(v0 rc2020-01-24 16:08:43.778298 b350 1=1+0)/n(v0
rc2020-01-24 16:08:43.773910 b350 1=1+0) (iversion lock w=1
last_client=77200959) caps={77200959=pAsXsFscr/-@6} | ptrwaiter=0 request=0
lock=1 caps=1 dirtyparent=1 dirty=1 authpin=1 0x55d006c3a000]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker simple_xlock on (isnap
lock) on [inode 0x10003178634 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json auth v13929147 ap=1
dirtyparent s=350 n(v0 rc2020-01-24 16:08:43.778298 b350 1=1+0)/n(v0
rc2020-01-24 16:08:43.773910 b350 1=1+0) (iversion lock w=1
last_client=77200959) caps={77200959=pAsXsFscr/-@6} | ptrwaiter=0 request=1
lock=1 caps=1 dirtyparent=1 dirty=1 authpin=1 0x55d006c3a000]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.ino(0x10003178634) auth_pin
by 0x55d006c3a5e0 on [inode 0x10003178634 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json auth v13929147 ap=2
dirtyparent s=350 n(v0 rc2020-01-24 16:08:43.778298 b350 1=1+0)/n(v0
rc2020-01-24 16:08:43.773910 b350 1=1+0) (iversion lock w=1
last_client=77200959) caps={77200959=pAsXsFscr/-@6} | ptrwaiter=0 request=1
lock=1 caps=1 dirtyparent=1 dirty=1 authpin=1 0x55d006c3a000] now 2
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got xlock on (isnap xlock
x=1 by 0x55d01a129400) [inode 0x10003178634 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json auth v13929147 ap=2
dirtyparent s=350 n(v0 rc2020-01-24 16:08:43.778298 b350 1=1+0)/n(v0
rc2020-01-24 16:08:43.773910 b350 1=1+0) (iversion lock w=1
last_client=77200959) caps={77200959=pAsXsFscr/-@6} | ptrwaiter=0 request=1
lock=2 caps=1 dirtyparent=1 dirty=1 authpin=1 0x55d006c3a000]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker xlock_start on (ilink
sync) on [inode 0x10003178634 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json auth v13929147 ap=2
dirtyparent s=350 n(v0 rc2020-01-24 16:08:43.778298 b350 1=1+0)/n(v0
rc2020-01-24 16:08:43.773910 b350 1=1+0) (iversion lock w=1
last_client=77200959) caps={77200959=pAsXsFscr/-@6} | ptrwaiter=0 request=1
lock=2 caps=1 dirtyparent=1 dirty=1 authpin=1 0x55d006c3a000]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker simple_lock on (ilink
sync) on [inode 0x10003178634 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json auth v13929147 ap=2
dirtyparent s=350 n(v0 rc2020-01-24 16:08:43.778298 b350 1=1+0)/n(v0
rc2020-01-24 16:08:43.773910 b350 1=1+0) (iversion lock w=1
last_client=77200959) caps={77200959=pAsXsFscr/-@6} | ptrwaiter=0 request=1
lock=2 caps=1 dirtyparent=1 dirty=1 authpin=1 0x55d006c3a000]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker simple_xlock on (ilink
lock) on [inode 0x10003178634 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json auth v13929147 ap=2
dirtyparent s=350 n(v0 rc2020-01-24 16:08:43.778298 b350 1=1+0)/n(v0
rc2020-01-24 16:08:43.773910 b350 1=1+0) (ilink lock) (iversion lock w=1
last_client=77200959) caps={77200959=pAsXsFscr/-@6} | ptrwaiter=0 request=1
lock=2 caps=1 dirtyparent=1 dirty=1 authpin=1 0x55d006c3a000]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.ino(0x10003178634) auth_pin
by 0x55d006c3a530 on [inode 0x10003178634 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json auth v13929147 ap=3
dirtyparent s=350 n(v0 rc2020-01-24 16:08:43.778298 b350 1=1+0)/n(v0
rc2020-01-24 16:08:43.773910 b350 1=1+0) (ilink lock) (iversion lock w=1
last_client=77200959) caps={77200959=pAsXsFscr/-@6} | ptrwaiter=0 request=1
lock=2 caps=1 dirtyparent=1 dirty=1 authpin=1 0x55d006c3a000] now 3
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got xlock on (ilink xlock
x=1 by 0x55d01a129400) [inode 0x10003178634 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json auth v13929147 ap=3
dirtyparent s=350 n(v0 rc2020-01-24 16:08:43.778298 b350 1=1+0)/n(v0
rc2020-01-24 16:08:43.773910 b350 1=1+0) (ilink xlock x=1 by 0x55d01a129400)
(iversion lock w=1 last_client=77200959) caps={77200959=pAsXsFscr/-@6} |
ptrwaiter=0 request=1 lock=3 caps=1 dirtyparent=1 dirty=1 authpin=1
0x55d006c3a000]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker local_wrlock_start on
(iversion lock) on [inode 0x10003178635 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719 auth
v13929158 ap=1 dirtyparent s=350 n(v0 rc2020-01-24 16:08:47.246973 b350 1=1+0)
(iversion lock) caps={77200959=pAsXsFscr/-@4} | ptrwaiter=0 request=0 lock=0
caps=1 dirtyparent=1 dirty=1 authpin=1 0x55d006c3b500]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got wrlock on (iversion
lock w=1 last_client=77200959) [inode 0x10003178635 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719 auth
v13929158 ap=1 dirtyparent s=350 n(v0 rc2020-01-24 16:08:47.246973 b350 1=1+0)
(iversion lock w=1 last_client=77200959) caps={77200959=pAsXsFscr/-@4} |
ptrwaiter=0 request=0 lock=1 caps=1 dirtyparent=1 dirty=1 authpin=1
0x55d006c3b500]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker xlock_start on (isnap
sync) on [inode 0x10003178635 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719 auth
v13929158 ap=1 dirtyparent s=350 n(v0 rc2020-01-24 16:08:47.246973 b350 1=1+0)
(iversion lock w=1 last_client=77200959) caps={77200959=pAsXsFscr/-@4} |
ptrwaiter=0 request=0 lock=1 caps=1 dirtyparent=1 dirty=1 authpin=1
0x55d006c3b500]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker simple_lock on (isnap
sync) on [inode 0x10003178635 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719 auth
v13929158 ap=1 dirtyparent s=350 n(v0 rc2020-01-24 16:08:47.246973 b350 1=1+0)
(iversion lock w=1 last_client=77200959) caps={77200959=pAsXsFscr/-@4} |
ptrwaiter=0 request=0 lock=1 caps=1 dirtyparent=1 dirty=1 authpin=1
0x55d006c3b500]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker simple_xlock on (isnap
lock) on [inode 0x10003178635 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719 auth
v13929158 ap=1 dirtyparent s=350 n(v0 rc2020-01-24 16:08:47.246973 b350 1=1+0)
(iversion lock w=1 last_client=77200959) caps={77200959=pAsXsFscr/-@4} |
ptrwaiter=0 request=1 lock=1 caps=1 dirtyparent=1 dirty=1 authpin=1
0x55d006c3b500]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.ino(0x10003178635) auth_pin
by 0x55d006c3bae0 on [inode 0x10003178635 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719 auth
v13929158 ap=2 dirtyparent s=350 n(v0 rc2020-01-24 16:08:47.246973 b350 1=1+0)
(iversion lock w=1 last_client=77200959) caps={77200959=pAsXsFscr/-@4} |
ptrwaiter=0 request=1 lock=1 caps=1 dirtyparent=1 dirty=1 authpin=1
0x55d006c3b500] now 2
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got xlock on (isnap xlock
x=1 by 0x55d01a129400) [inode 0x10003178635 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719 auth
v13929158 ap=2 dirtyparent s=350 n(v0 rc2020-01-24 16:08:47.246973 b350 1=1+0)
(iversion lock w=1 last_client=77200959) caps={77200959=pAsXsFscr/-@4} |
ptrwaiter=0 request=1 lock=2 caps=1 dirtyparent=1 dirty=1 authpin=1
0x55d006c3b500]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker xlock_start on (ilink
sync) on [inode 0x10003178635 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719 auth
v13929158 ap=2 dirtyparent s=350 n(v0 rc2020-01-24 16:08:47.246973 b350 1=1+0)
(iversion lock w=1 last_client=77200959) caps={77200959=pAsXsFscr/-@4} |
ptrwaiter=0 request=1 lock=2 caps=1 dirtyparent=1 dirty=1 authpin=1
0x55d006c3b500]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker simple_lock on (ilink
sync) on [inode 0x10003178635 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719 auth
v13929158 ap=2 dirtyparent s=350 n(v0 rc2020-01-24 16:08:47.246973 b350 1=1+0)
(iversion lock w=1 last_client=77200959) caps={77200959=pAsXsFscr/-@4} |
ptrwaiter=0 request=1 lock=2 caps=1 dirtyparent=1 dirty=1 authpin=1
0x55d006c3b500]
2020-01-24 16:08:47.250 7fbe050c6700 7 mds.0.locker simple_xlock on (ilink
lock) on [inode 0x10003178635 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719 auth
v13929158 ap=2 dirtyparent s=350 n(v0 rc2020-01-24 16:08:47.246973 b350 1=1+0)
(ilink lock) (iversion lock w=1 last_client=77200959)
caps={77200959=pAsXsFscr/-@4} | ptrwaiter=0 request=1 lock=2 caps=1
dirtyparent=1 dirty=1 authpin=1 0x55d006c3b500]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.ino(0x10003178635) auth_pin
by 0x55d006c3ba30 on [inode 0x10003178635 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719 auth
v13929158 ap=3 dirtyparent s=350 n(v0 rc2020-01-24 16:08:47.246973 b350 1=1+0)
(ilink lock) (iversion lock w=1 last_client=77200959)
caps={77200959=pAsXsFscr/-@4} | ptrwaiter=0 request=1 lock=2 caps=1
dirtyparent=1 dirty=1 authpin=1 0x55d006c3b500] now 3
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.locker got xlock on (ilink xlock
x=1 by 0x55d01a129400) [inode 0x10003178635 [2,head]
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719 auth
v13929158 ap=3 dirtyparent s=350 n(v0 rc2020-01-24 16:08:47.246973 b350 1=1+0)
(ilink xlock x=1 by 0x55d01a129400) (iversion lock w=1 last_client=77200959)
caps={77200959=pAsXsFscr/-@4} | ptrwaiter=0 request=1 lock=3 caps=1
dirtyparent=1 dirty=1 authpin=1 0x55d006c3b500]
2020-01-24 16:08:47.250 7fbe050c6700 10 MDSAuthCap is_capable inode(path
/home/tjr22/.config/configstore owner 1000:1003 mode 040700) by caller
1000:1000 mask 2 new 0:0 cap: MDSAuthCaps[allow *]
2020-01-24 16:08:47.250 7fbe050c6700 10 MDSAuthCap is_capable inode(path
/home/tjr22/.config/configstore owner 1000:1003 mode 040700) by caller
1000:1000 mask 2 new 0:0 cap: MDSAuthCaps[allow *]
2020-01-24 16:08:47.250 7fbe050c6700 10 MDSAuthCap is_capable inode(path
/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719 owner
1000:1000 mode 0100600) by caller 1000:1000 mask 2 new 0:0 cap:
MDSAuthCaps[allow *]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.snaprealm(0x1 seq 1
0x55cfeff4a280) have_past_parents_open [1,head]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.cache.snaprealm(0x1 seq 1
0x55cfeff4a280) have_past_parents_open [1,head]
2020-01-24 16:08:47.250 7fbe050c6700 10 mds.0.server _rename_prepare
request(client.77200959:238582 cr=0x55d019672580) [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json.2465308719
[2,head] auth (dn xlock x=1 by 0x55d01a129400) (dversion lock w=1
last_client=77200959) v=13929158 ap=2 ino=0x10003178635 state=1610612736 |
request=1 lock=2 inodepin=1 dirty=1 authpin=1 0x55d018f10000] [dentry
#0x1/home/tjr22/.config/configstore/tus-node-server-0.1.1.json [2,head] auth
(dn xlock x=1 by 0x55d01a129400) (dversion lock w=1 last_client=77200959)
v=13929147 ap=2 ino=0x10003178634 state=1610612736 | request=1 lock=2
inodepin=1 dirty=1 waiter=0 authpin=1 0x55cfeeef3dc0]
I have no idea if any of these are red herrings (sorry.) Please help, at this
point I’m baffled! Thank you.
-TJ
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
