Re: [ceph-users] hanging slow requests: failed to authpin, subtree is being exported
I think I am having this issue also (at least I had with luminous) I had to remove the hidden temp files rsync had left, when the cephfs mount 'stalled', otherwise I would never be able to complete the rsync. -Original Message- Cc: ceph-users Subject: Re: [ceph-users] hanging slow requests: failed to authpin, subtree is being exported I've made a ticket for this issue: https://tracker.ceph.com/issues/42338 Thanks again! K On 15/10/2019 18:00, Kenneth Waegeman wrote: > Hi Robert, all, > > > On 23/09/2019 17:37, Robert LeBlanc wrote: >> On Mon, Sep 23, 2019 at 4:14 AM Kenneth Waegeman >> wrote: >>> Hi all, >>> >>> When syncing data with rsync, I'm often getting blocked slow >>> requests, which also block access to this path. >>> >>>> 2019-09-23 11:25:49.477 7f4f401e8700 0 log_channel(cluster) log >>>> [WRN] >>>> : slow request 31.895478 seconds old, received at 2019-09-23 >>>> 11:25:17.598152: client_request(client.38352684:92684 lookup >>>> #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, >>>> caller_gid=0{0,}) currently failed to authpin, subtree is being >>>> exported >>>> 2019-09-23 11:26:19.477 7f4f401e8700 0 log_channel(cluster) log >>>> [WRN] >>>> : slow request 61.896079 seconds old, received at 2019-09-23 >>>> 11:25:17.598152: client_request(client.38352684:92684 lookup >>>> #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, >>>> caller_gid=0{0,}) currently failed to authpin, subtree is being >>>> exported >>>> 2019-09-23 11:27:19.478 7f4f401e8700 0 log_channel(cluster) log >>>> [WRN] >>>> : slow request 121.897268 seconds old, received at 2019-09-23 >>>> 11:25:17.598152: client_request(client.38352684:92684 lookup >>>> #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, >>>> caller_gid=0{0,}) currently failed to authpin, subtree is being >>>> exported >>>> 2019-09-23 11:29:19.488 7f4f401e8700 0 log_channel(cluster) log >>>> [WRN] >>>> : slow request 241.899467 seconds old, received at 2019-09-23 >>>> 11:25:17.598152: client_request(client.38352684:92684 lookup >>>> #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, >>>> caller_gid=0{0,}) currently failed to authpin, subtree is being >>>> exported >>>> 2019-09-23 11:33:19.680 7f4f401e8700 0 log_channel(cluster) log >>>> [WRN] >>>> : slow request 482.087927 seconds old, received at 2019-09-23 >>>> 11:25:17.598152: client_request(client.38352684:92684 lookup >>>> #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, >>>> caller_gid=0{0,}) currently failed to authpin, subtree is being >>>> exported >>>> 2019-09-23 11:36:09.881 7f4f401e8700 0 log_channel(cluster) log >>>> [WRN] >>>> : slow request 32.677511 seconds old, received at 2019-09-23 >>>> 11:35:37.217113: client_request(client.38347357:111963 lookup >>>> #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, >>>> caller_gid=0{0,}) currently failed to authpin, subtree is being >>>> exported >>>> 2019-09-23 11:36:39.881 7f4f401e8700 0 log_channel(cluster) log >>>> [WRN] >>>> : slow request 62.678132 seconds old, received at 2019-09-23 >>>> 11:35:37.217113: client_request(client.38347357:111963 lookup >>>> #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, >>>> caller_gid=0{0,}) currently failed to authpin, subtree is being >>>> exported >>>> 2019-09-23 11:37:39.891 7f4f401e8700 0 log_channel(cluster) log >>>> [WRN] >>>> : slow request 122.679273 seconds old, received at 2019-09-23 >>>> 11:35:37.217113: client_request(client.38347357:111963 lookup >>>> #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, >>>> caller_gid=0{0,}) currently failed to authpin, subtree is being >>>> exported >>>> 2019-09-23 11:39:39.892 7f4f401e8700 0 log_channel(cluster) log >>>> [WRN] >>>> : slow request 242.684667 seconds old, received at 2019-09-23 >>>> 11:35:37.217113: client_request(client.38347357:111963 lookup >>>> #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, >>>> caller_gid=0{0,}) currently failed to authpin, subtree is being >>>> exported >>>> 2019-09-23 11:41:19.893 7f4f401e8700 0 log_ch
Re: [ceph-users] hanging slow requests: failed to authpin, subtree is being exported
I've made a ticket for this issue: https://tracker.ceph.com/issues/42338 Thanks again! K On 15/10/2019 18:00, Kenneth Waegeman wrote: Hi Robert, all, On 23/09/2019 17:37, Robert LeBlanc wrote: On Mon, Sep 23, 2019 at 4:14 AM Kenneth Waegeman wrote: Hi all, When syncing data with rsync, I'm often getting blocked slow requests, which also block access to this path. 2019-09-23 11:25:49.477 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 31.895478 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:26:19.477 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 61.896079 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:27:19.478 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 121.897268 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:29:19.488 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 241.899467 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:33:19.680 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 482.087927 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:36:09.881 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 32.677511 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:36:39.881 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 62.678132 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:37:39.891 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 122.679273 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:39:39.892 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 242.684667 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:41:19.893 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 962.305681 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:43:39.923 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 482.712888 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:51:40.236 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 963.037049 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:57:20.308 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 1922.719287 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 12:07:40.621 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 1923.409501 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) cur
Re: [ceph-users] hanging slow requests: failed to authpin, subtree is being exported
Hi Robert, all, On 23/09/2019 17:37, Robert LeBlanc wrote: On Mon, Sep 23, 2019 at 4:14 AM Kenneth Waegeman wrote: Hi all, When syncing data with rsync, I'm often getting blocked slow requests, which also block access to this path. 2019-09-23 11:25:49.477 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 31.895478 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:26:19.477 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 61.896079 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:27:19.478 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 121.897268 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:29:19.488 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 241.899467 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:33:19.680 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 482.087927 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:36:09.881 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 32.677511 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:36:39.881 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 62.678132 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:37:39.891 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 122.679273 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:39:39.892 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 242.684667 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:41:19.893 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 962.305681 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:43:39.923 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 482.712888 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:51:40.236 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 963.037049 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:57:20.308 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 1922.719287 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 12:07:40.621 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 1923.409501 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 12:29:20.639 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 3843.057602 seconds old, r
Re: [ceph-users] hanging slow requests: failed to authpin, subtree is being exported
On Mon, Sep 23, 2019 at 4:14 AM Kenneth Waegeman wrote: > > Hi all, > > When syncing data with rsync, I'm often getting blocked slow requests, > which also block access to this path. > > > 2019-09-23 11:25:49.477 7f4f401e8700 0 log_channel(cluster) log [WRN] > > : slow request 31.895478 seconds old, received at 2019-09-23 > > 11:25:17.598152: client_request(client.38352684:92684 lookup > > #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, > > caller_gid=0{0,}) currently failed to authpin, subtree is being exported > > 2019-09-23 11:26:19.477 7f4f401e8700 0 log_channel(cluster) log [WRN] > > : slow request 61.896079 seconds old, received at 2019-09-23 > > 11:25:17.598152: client_request(client.38352684:92684 lookup > > #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, > > caller_gid=0{0,}) currently failed to authpin, subtree is being exported > > 2019-09-23 11:27:19.478 7f4f401e8700 0 log_channel(cluster) log [WRN] > > : slow request 121.897268 seconds old, received at 2019-09-23 > > 11:25:17.598152: client_request(client.38352684:92684 lookup > > #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, > > caller_gid=0{0,}) currently failed to authpin, subtree is being exported > > 2019-09-23 11:29:19.488 7f4f401e8700 0 log_channel(cluster) log [WRN] > > : slow request 241.899467 seconds old, received at 2019-09-23 > > 11:25:17.598152: client_request(client.38352684:92684 lookup > > #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, > > caller_gid=0{0,}) currently failed to authpin, subtree is being exported > > 2019-09-23 11:33:19.680 7f4f401e8700 0 log_channel(cluster) log [WRN] > > : slow request 482.087927 seconds old, received at 2019-09-23 > > 11:25:17.598152: client_request(client.38352684:92684 lookup > > #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, > > caller_gid=0{0,}) currently failed to authpin, subtree is being exported > > 2019-09-23 11:36:09.881 7f4f401e8700 0 log_channel(cluster) log [WRN] > > : slow request 32.677511 seconds old, received at 2019-09-23 > > 11:35:37.217113: client_request(client.38347357:111963 lookup > > #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, > > caller_gid=0{0,}) currently failed to authpin, subtree is being exported > > 2019-09-23 11:36:39.881 7f4f401e8700 0 log_channel(cluster) log [WRN] > > : slow request 62.678132 seconds old, received at 2019-09-23 > > 11:35:37.217113: client_request(client.38347357:111963 lookup > > #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, > > caller_gid=0{0,}) currently failed to authpin, subtree is being exported > > 2019-09-23 11:37:39.891 7f4f401e8700 0 log_channel(cluster) log [WRN] > > : slow request 122.679273 seconds old, received at 2019-09-23 > > 11:35:37.217113: client_request(client.38347357:111963 lookup > > #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, > > caller_gid=0{0,}) currently failed to authpin, subtree is being exported > > 2019-09-23 11:39:39.892 7f4f401e8700 0 log_channel(cluster) log [WRN] > > : slow request 242.684667 seconds old, received at 2019-09-23 > > 11:35:37.217113: client_request(client.38347357:111963 lookup > > #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, > > caller_gid=0{0,}) currently failed to authpin, subtree is being exported > > 2019-09-23 11:41:19.893 7f4f401e8700 0 log_channel(cluster) log [WRN] > > : slow request 962.305681 seconds old, received at 2019-09-23 > > 11:25:17.598152: client_request(client.38352684:92684 lookup > > #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, > > caller_gid=0{0,}) currently failed to authpin, subtree is being exported > > 2019-09-23 11:43:39.923 7f4f401e8700 0 log_channel(cluster) log [WRN] > > : slow request 482.712888 seconds old, received at 2019-09-23 > > 11:35:37.217113: client_request(client.38347357:111963 lookup > > #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, > > caller_gid=0{0,}) currently failed to authpin, subtree is being exported > > 2019-09-23 11:51:40.236 7f4f401e8700 0 log_channel(cluster) log [WRN] > > : slow request 963.037049 seconds old, received at 2019-09-23 > > 11:35:37.217113: client_request(client.38347357:111963 lookup > > #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, > > caller_gid=0{0,}) currently failed to authpin, subtree is being exported > > 2019-09-23 11:57:20.308 7f4f401e8700 0 log_channel(cluster) log [WRN] > > : slow request 1922.719287 seconds old, received at 2019-09-23 > > 11:25:17.598152: client_request(client.38352684:92684 lookup > > #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, > > caller_gid=0{0,}) currently failed to authpin, subtree is being exported > > 2019-09-23 12:07:40.621 7f4f401e8700 0 log_channel(cluster) log [WRN] > > : slow request 1923.409501 seconds old, received at 2019-09-23 > > 11:35:37.217113: client_request(client.38347357:111963 lookup > > #0x20005b0130c/testing 2019-
[ceph-users] hanging slow requests: failed to authpin, subtree is being exported
Hi all, When syncing data with rsync, I'm often getting blocked slow requests, which also block access to this path. 2019-09-23 11:25:49.477 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 31.895478 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:26:19.477 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 61.896079 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:27:19.478 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 121.897268 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:29:19.488 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 241.899467 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:33:19.680 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 482.087927 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:36:09.881 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 32.677511 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:36:39.881 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 62.678132 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:37:39.891 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 122.679273 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:39:39.892 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 242.684667 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:41:19.893 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 962.305681 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:43:39.923 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 482.712888 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:51:40.236 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 963.037049 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 11:57:20.308 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 1922.719287 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.38352684:92684 lookup #0x100152383ce/vsc42531 2019-09-23 11:25:17.598077 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 12:07:40.621 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 1923.409501 seconds old, received at 2019-09-23 11:35:37.217113: client_request(client.38347357:111963 lookup #0x20005b0130c/testing 2019-09-23 11:35:37.217015 caller_uid=0, caller_gid=0{0,}) currently failed to authpin, subtree is being exported 2019-09-23 12:29:20.639 7f4f401e8700 0 log_channel(cluster) log [WRN] : slow request 3843.057602 seconds old, received at 2019-09-23 11:25:17.598152: client_request(client.