Hi Yuhao,
On Mon, 6 Aug 2018, 15:26 Yuhao Zhang, wrote:
> Hello,
>
> I just experienced another hanging one hour ago and the server was not
> even under heavy IO.
>
> Atin, I attached the process monitoring results and another statedump.
>
> Xavi, ZFS was fine, during the hanging, I can still
Hi Yuhao,
On Mon, Aug 6, 2018 at 6:57 AM Yuhao Zhang wrote:
> Atin, that was my typo... I think it was glusterfsd, but not 100% sure. I
> will keep an eye when it happens next time.
>
> Thank you all for looking into this! I tried another transfer earlier
> today but it didn't get the chance to
Atin, that was my typo... I think it was glusterfsd, but not 100% sure. I will
keep an eye when it happens next time.
Thank you all for looking into this! I tried another transfer earlier today but
it didn't get the chance to reach the point where glusterfsd starts to fail
before we needed to
On Sun, 5 Aug 2018 at 13:29, Yuhao Zhang wrote:
> Sorry, what I meant was, if I start the transfer now and get glusterd into
> zombie status,
>
glusterd or glusterfsd?
it's unlikely that I can fully recover the server without a reboot.
>
>
> On Aug 5, 2018, at 02:55, Raghavendra Gowdappa
>
On Sun, Aug 5, 2018 at 1:29 PM, Yuhao Zhang wrote:
> Sorry, what I meant was, if I start the transfer now and get glusterd into
> zombie status, it's unlikely that I can fully recover the server without a
> reboot.
>
I missed it. Thanks for the explanation :).
>
> On Aug 5, 2018, at 02:55,
Sorry, what I meant was, if I start the transfer now and get glusterd into
zombie status, it's unlikely that I can fully recover the server without a
reboot.
> On Aug 5, 2018, at 02:55, Raghavendra Gowdappa wrote:
>
>
>
> On Sun, Aug 5, 2018 at 1:22 PM, Yuhao Zhang
On Sun, Aug 5, 2018 at 1:22 PM, Yuhao Zhang wrote:
> This is a semi-production server and I can't bring it down right now. Will
> try to get the monitoring output when I get a chance.
>
Collecting top output doesn't require to bring down servers.
> As I recall, the high CPU processes are
This is a semi-production server and I can't bring it down right now. Will try
to get the monitoring output when I get a chance.
As I recall, the high CPU processes are brick daemons (glusterfsd) and htop
showed they were in status D. However, I saw zero zpool IO as clients were all
hanging.
On Sun, Aug 5, 2018 at 12:44 PM, Yuhao Zhang wrote:
> Hi,
>
> I am running into a situation that heavy write causes Gluster server went
> into zombie with many high CPU processes and all clients hangs, it is
> almost 100% reproducible on my machine. Hope someone can help.
>
Can you give us the
