Re: [Gluster-devel] spurious test failure in tests/bugs/replicate/bug-1015990.t
On 05/08/2015 10:02 AM, Atin Mukherjee wrote: On 05/07/2015 03:00 PM, Krishnan Parthasarathi wrote: Atin would be doing this, since he is looking into it. HTH, KP - Original Message - On 05/07/2015 02:53 PM, Krishnan Parthasarathi wrote: - Original Message - On 05/07/2015 02:41 PM, Krishnan Parthasarathi wrote: Pranith, The above snippet says that the volume has to be stopped before deleted. It also says that volume-stop failed. I would look into glusterd logs to see why volume-stop failed, cmd-history.log tells us only so much. http://build.gluster.org/job/rackspace-regression-2GB-triggered/8522/consoleFull has the logs. I didn't find much information. Please feel free to take a look. What can we add to the code so that this failure can be debugged better in future? Please at least add that much for now? Atin is already looking into this. Without the root cause, it's not useful to speculate how we could help debugging this. As we root cause, I am sure we will find things that we could have logged to reduce time to root cause. Does that make sense? Cool. Could you please update the pad: https://public.pad.fsfe.org/p/gluster-spurious-failures with latest info on this issue. glusterd did log the following failure when volume stop was executed: [2015-05-06 13:09:58.534114] I [socket.c:3358:socket_submit_request] 0-management: not connected (priv->connected = 0) [2015-05-06 13:09:58.534137] W [rpc-clnt.c:1566:rpc_clnt_submit] 0-management: failed to submit rpc-request (XID: 0x1 Program: brick operations, ProgVers: 2, Proc: 1) to rpc-transport (management) This indicates the underlying transport connection was broken and glusterd failed to send the rpc request to the brick. For this case, glusterd didn't populate errstr because of which in cmd_history.log volume stop was logged with a failure and a blank error message. I've sent patch [1] to populate errstr for this failure. Thanks Atin, please move this test to resolved section in the pad if not already. Pranith [1] http://review.gluster.org/10659 ~Atin Pranith Pranith HTH, KP - Original Message - hi, Volume delete is failing without logging much about why it is failing. Know anything about this? (http://build.gluster.org/job/rackspace-regression-2GB-triggered/8522/consoleFull) 1 [2015-05-06 13:09:58.311519] : volume heal patchy statistics heal-count : SUCCESS 0 [2015-05-06 13:09:58.534917] : volume stop patchy : FAILED : 1 [2015-05-06 13:09:58.904333] : volume delete patchy : FAILED : Volume patchy has been started.Volume needs to be stopped before deletion. Pranith ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] spurious test failure in tests/bugs/replicate/bug-1015990.t
On 05/07/2015 03:00 PM, Krishnan Parthasarathi wrote: > Atin would be doing this, since he is looking into it. > > HTH, > KP > > - Original Message - >> >> On 05/07/2015 02:53 PM, Krishnan Parthasarathi wrote: >>> >>> - Original Message - On 05/07/2015 02:41 PM, Krishnan Parthasarathi wrote: > Pranith, > > The above snippet says that the volume has to be stopped before deleted. > It > also says that > volume-stop failed. I would look into glusterd logs to see why > volume-stop > failed, > cmd-history.log tells us only so much. http://build.gluster.org/job/rackspace-regression-2GB-triggered/8522/consoleFull has the logs. I didn't find much information. Please feel free to take a look. What can we add to the code so that this failure can be debugged better in future? Please at least add that much for now? >>> Atin is already looking into this. Without the root cause, it's not useful >>> to >>> speculate how we could help debugging this. As we root cause, I am sure we >>> will >>> find things that we could have logged to reduce time to root cause. Does >>> that make sense? >> Cool. Could you please update the pad: >> https://public.pad.fsfe.org/p/gluster-spurious-failures with latest info >> on this issue. glusterd did log the following failure when volume stop was executed: [2015-05-06 13:09:58.534114] I [socket.c:3358:socket_submit_request] 0-management: not connected (priv->connected = 0) [2015-05-06 13:09:58.534137] W [rpc-clnt.c:1566:rpc_clnt_submit] 0-management: failed to submit rpc-request (XID: 0x1 Program: brick operations, ProgVers: 2, Proc: 1) to rpc-transport (management) This indicates the underlying transport connection was broken and glusterd failed to send the rpc request to the brick. For this case, glusterd didn't populate errstr because of which in cmd_history.log volume stop was logged with a failure and a blank error message. I've sent patch [1] to populate errstr for this failure. [1] http://review.gluster.org/10659 ~Atin >> >> Pranith >>> Pranith > HTH, > KP > > - Original Message - >> hi, >>Volume delete is failing without logging much about why it is >> failing. Know anything about this? >> (http://build.gluster.org/job/rackspace-regression-2GB-triggered/8522/consoleFull) >> 1 [2015-05-06 13:09:58.311519] : volume heal patchy statistics >> heal-count : SUCCESS >> 0 [2015-05-06 13:09:58.534917] : volume stop patchy : FAILED : >> 1 [2015-05-06 13:09:58.904333] : volume delete patchy : FAILED : >> Volume patchy has been started.Volume needs to be stopped before >> deletion. >> >> Pranith >> >> >> -- ~Atin ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] spurious test failure in tests/bugs/replicate/bug-1015990.t
Atin would be doing this, since he is looking into it. HTH, KP - Original Message - > > On 05/07/2015 02:53 PM, Krishnan Parthasarathi wrote: > > > > - Original Message - > >> On 05/07/2015 02:41 PM, Krishnan Parthasarathi wrote: > >>> Pranith, > >>> > >>> The above snippet says that the volume has to be stopped before deleted. > >>> It > >>> also says that > >>> volume-stop failed. I would look into glusterd logs to see why > >>> volume-stop > >>> failed, > >>> cmd-history.log tells us only so much. > >> http://build.gluster.org/job/rackspace-regression-2GB-triggered/8522/consoleFull > >> has the logs. I didn't find much information. Please feel free to take a > >> look. What can we add to the code so that this failure can be debugged > >> better in future? Please at least add that much for now? > > Atin is already looking into this. Without the root cause, it's not useful > > to > > speculate how we could help debugging this. As we root cause, I am sure we > > will > > find things that we could have logged to reduce time to root cause. Does > > that make sense? > Cool. Could you please update the pad: > https://public.pad.fsfe.org/p/gluster-spurious-failures with latest info > on this issue. > > Pranith > > > >> Pranith > >> > >>> HTH, > >>> KP > >>> > >>> - Original Message - > hi, > Volume delete is failing without logging much about why it is > failing. Know anything about this? > (http://build.gluster.org/job/rackspace-regression-2GB-triggered/8522/consoleFull) > 1 [2015-05-06 13:09:58.311519] : volume heal patchy statistics > heal-count : SUCCESS > 0 [2015-05-06 13:09:58.534917] : volume stop patchy : FAILED : > 1 [2015-05-06 13:09:58.904333] : volume delete patchy : FAILED : > Volume patchy has been started.Volume needs to be stopped before > deletion. > > Pranith > > >> > > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] spurious test failure in tests/bugs/replicate/bug-1015990.t
On 05/07/2015 02:53 PM, Krishnan Parthasarathi wrote: - Original Message - On 05/07/2015 02:41 PM, Krishnan Parthasarathi wrote: Pranith, The above snippet says that the volume has to be stopped before deleted. It also says that volume-stop failed. I would look into glusterd logs to see why volume-stop failed, cmd-history.log tells us only so much. http://build.gluster.org/job/rackspace-regression-2GB-triggered/8522/consoleFull has the logs. I didn't find much information. Please feel free to take a look. What can we add to the code so that this failure can be debugged better in future? Please at least add that much for now? Atin is already looking into this. Without the root cause, it's not useful to speculate how we could help debugging this. As we root cause, I am sure we will find things that we could have logged to reduce time to root cause. Does that make sense? Cool. Could you please update the pad: https://public.pad.fsfe.org/p/gluster-spurious-failures with latest info on this issue. Pranith Pranith HTH, KP - Original Message - hi, Volume delete is failing without logging much about why it is failing. Know anything about this? (http://build.gluster.org/job/rackspace-regression-2GB-triggered/8522/consoleFull) 1 [2015-05-06 13:09:58.311519] : volume heal patchy statistics heal-count : SUCCESS 0 [2015-05-06 13:09:58.534917] : volume stop patchy : FAILED : 1 [2015-05-06 13:09:58.904333] : volume delete patchy : FAILED : Volume patchy has been started.Volume needs to be stopped before deletion. Pranith ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] spurious test failure in tests/bugs/replicate/bug-1015990.t
- Original Message - > > On 05/07/2015 02:41 PM, Krishnan Parthasarathi wrote: > > Pranith, > > > > The above snippet says that the volume has to be stopped before deleted. It > > also says that > > volume-stop failed. I would look into glusterd logs to see why volume-stop > > failed, > > cmd-history.log tells us only so much. > > http://build.gluster.org/job/rackspace-regression-2GB-triggered/8522/consoleFull > has the logs. I didn't find much information. Please feel free to take a > look. What can we add to the code so that this failure can be debugged > better in future? Please at least add that much for now? Atin is already looking into this. Without the root cause, it's not useful to speculate how we could help debugging this. As we root cause, I am sure we will find things that we could have logged to reduce time to root cause. Does that make sense? > > Pranith > > > > > HTH, > > KP > > > > - Original Message - > >> hi, > >> Volume delete is failing without logging much about why it is > >> failing. Know anything about this? > >> (http://build.gluster.org/job/rackspace-regression-2GB-triggered/8522/consoleFull) > >> 1 [2015-05-06 13:09:58.311519] : volume heal patchy statistics > >> heal-count : SUCCESS > >> 0 [2015-05-06 13:09:58.534917] : volume stop patchy : FAILED : > >> 1 [2015-05-06 13:09:58.904333] : volume delete patchy : FAILED : > >> Volume patchy has been started.Volume needs to be stopped before deletion. > >> > >> Pranith > >> > > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] spurious test failure in tests/bugs/replicate/bug-1015990.t
On 05/07/2015 02:41 PM, Krishnan Parthasarathi wrote: Pranith, The above snippet says that the volume has to be stopped before deleted. It also says that volume-stop failed. I would look into glusterd logs to see why volume-stop failed, cmd-history.log tells us only so much. http://build.gluster.org/job/rackspace-regression-2GB-triggered/8522/consoleFull has the logs. I didn't find much information. Please feel free to take a look. What can we add to the code so that this failure can be debugged better in future? Please at least add that much for now? Pranith HTH, KP - Original Message - hi, Volume delete is failing without logging much about why it is failing. Know anything about this? (http://build.gluster.org/job/rackspace-regression-2GB-triggered/8522/consoleFull) 1 [2015-05-06 13:09:58.311519] : volume heal patchy statistics heal-count : SUCCESS 0 [2015-05-06 13:09:58.534917] : volume stop patchy : FAILED : 1 [2015-05-06 13:09:58.904333] : volume delete patchy : FAILED : Volume patchy has been started.Volume needs to be stopped before deletion. Pranith ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] spurious test failure in tests/bugs/replicate/bug-1015990.t
Pranith, The above snippet says that the volume has to be stopped before deleted. It also says that volume-stop failed. I would look into glusterd logs to see why volume-stop failed, cmd-history.log tells us only so much. HTH, KP - Original Message - > hi, > Volume delete is failing without logging much about why it is > failing. Know anything about this? > (http://build.gluster.org/job/rackspace-regression-2GB-triggered/8522/consoleFull) >1 [2015-05-06 13:09:58.311519] : volume heal patchy statistics > heal-count : SUCCESS >0 [2015-05-06 13:09:58.534917] : volume stop patchy : FAILED : >1 [2015-05-06 13:09:58.904333] : volume delete patchy : FAILED : > Volume patchy has been started.Volume needs to be stopped before deletion. > > Pranith > ___ Gluster-devel mailing list Gluster-devel@gluster.org http://www.gluster.org/mailman/listinfo/gluster-devel
