Re: [openstack-dev] [Heat][Rant] About blank rechecks

[Sergey Kraynev]

On 22 October 2015 at 10:58, Thomas Herve  wrote:
> Hi all,
>
> You've seen me complain about people doing blank rechecks in Gerrit on IRC,
> and it seems it had little to no effect. So here I am trying to spread the
> word here. I'll try to stay calm.
>
> I'm seeing way too many rechecks on heat patches. It's not epidemic, but
> it's still enough to make me sad.
>
> First, it makes me sad as a developer. I don't know if it's just me, but one
> of the reason I code is curiosity, and debugging a gate failure is a great
> way to learn, pierce through the layers, and improve the situation.
>
> It then makes me sad as a team member. By doing a recheck you're basically
> implying that you don't care about the failure, and surely someone will care
> at some point. Except, the information will be lost, and we may have 100
> builds before that happen again, when a release already happened, and we
> have to backport it. Working early means working less.
>
> And finally, it makes me sad for the infra team. Doing a recheck is
> disrespecting all the work they're doing to create a reliable environment to
> run our tests. Sure, sometimes the environment is the reason the failure
> happens, but then it's even more important to give feedback about it. They
> provide a great deal of logs, we can use logstach to find patterns, the
> least we can do is trying. We're also using resources that other projects
> could be using. As much as we'd like to believe it, the cloud doesn't have
> free infinite resources.
>
> Recently, I've seen many cases where rechecks were made whereas:
> 1) The heat branch was broken. Generally for some external reason (a
> dependency updated), doing a recheck is a pure waste of resources until that
> failure is fixed. Most of the time, we say something on IRC when it's the
> case. We also try to open a bug, so looking at launchpad can show something.
> 2) THE PATCH WAS ACTUALLY BROKEN. And there I'm not sad anymore I'm
> particularly angry. It basically means that you didn't look at all at the
> build results, and just mindlessly typed rechecks hoping that some fairy
> will fix your broken code. Frankly, that makes want to go on a -2 rampage.
> ESPECIALLY where a core is doing it.
>
> To close, I'll try to provide a solution. I know we all have our agenda,
> debugging gate failures takes some time that you may not have, and obviously
> "my patch is fine it's not my fault" (who cares, that's what being in a team
> means). Still, I'd like everyone to look at the test failures, look if the
> patch is not the problem, and if not open a bug [1] mentioning the test
> name, pasting the traceback in the description, and linking the build
> result. Then do recheck bug #xyz. That's it. It shouldn't take you more than
> 3 minutes, and at least we didn't lose the information.
>
> Thanks for reading that far and sorry for the length,
>
>
> [1] https://bugs.launchpad.net/heat/+filebug
>
> --
> Thomas
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

Thomas,

Thank you for the pay attention on this issue.
I know that you constantly ask about it, especially core-team members :)
I appreciate it, because it's really helpful and more over important.

Unfortunately new approach with empty reverify made people more lazy.
I think, that we should follow your recommendation, because:
 - it will help fix our gate faster
 - makes our develop process more clear and useful for new comers

my +2 for this initiative.
Everybody please spent your 2-3 minutes to make our work better!

-- 
Regards,
Sergey.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Heat][Rant] About blank rechecks

[Thomas Herve]

Hi all,

You've seen me complain about people doing blank rechecks in Gerrit on IRC,
and it seems it had little to no effect. So here I am trying to spread the
word here. I'll try to stay calm.

I'm seeing way too many rechecks on heat patches. It's not epidemic, but
it's still enough to make me sad.

First, it makes me sad as a developer. I don't know if it's just me, but
one of the reason I code is curiosity, and debugging a gate failure is a
great way to learn, pierce through the layers, and improve the situation.

It then makes me sad as a team member. By doing a recheck you're basically
implying that you don't care about the failure, and surely someone will
care at some point. Except, the information will be lost, and we may have
100 builds before that happen again, when a release already happened, and
we have to backport it. Working early means working less.

And finally, it makes me sad for the infra team. Doing a recheck is
disrespecting all the work they're doing to create a reliable environment
to run our tests. Sure, sometimes the environment is the reason the failure
happens, but then it's even more important to give feedback about it. They
provide a great deal of logs, we can use logstach to find patterns, the
least we can do is trying. We're also using resources that other projects
could be using. As much as we'd like to believe it, the cloud doesn't have
free infinite resources.

Recently, I've seen many cases where rechecks were made whereas:
1) The heat branch was broken. Generally for some external reason (a
dependency updated), doing a recheck is a pure waste of resources until
that failure is fixed. Most of the time, we say something on IRC when it's
the case. We also try to open a bug, so looking at launchpad can show
something.
2) THE PATCH WAS ACTUALLY BROKEN. And there I'm not sad anymore I'm
particularly angry. It basically means that you didn't look at all at the
build results, and just mindlessly typed rechecks hoping that some fairy
will fix your broken code. Frankly, that makes want to go on a -2 rampage.
ESPECIALLY where a core is doing it.

To close, I'll try to provide a solution. I know we all have our agenda,
debugging gate failures takes some time that you may not have, and
obviously "my patch is fine it's not my fault" (who cares, that's what
being in a team means). Still, I'd like everyone to look at the test
failures, look if the patch is not the problem, and if not open a bug [1]
mentioning the test name, pasting the traceback in the description, and
linking the build result. Then do recheck bug #xyz. That's it. It shouldn't
take you more than 3 minutes, and at least we didn't lose the information.

Thanks for reading that far and sorry for the length,


[1] https://bugs.launchpad.net/heat/+filebug

-- 
Thomas
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat][Rant] About blank rechecks

[Jordan Pittier]

On Thu, Oct 22, 2015 at 9:58 AM, Thomas Herve  wrote:

> Hi all,
>
> You've seen me complain about people doing blank rechecks in Gerrit on
> IRC, and it seems it had little to no effect. So here I am trying to spread
> the word here. I'll try to stay calm.
>
> I'm seeing way too many rechecks on heat patches. It's not epidemic, but
> it's still enough to make me sad.
>
> First, it makes me sad as a developer. I don't know if it's just me, but
> one of the reason I code is curiosity, and debugging a gate failure is a
> great way to learn, pierce through the layers, and improve the situation.
>
> It then makes me sad as a team member. By doing a recheck you're basically
> implying that you don't care about the failure, and surely someone will
> care at some point. Except, the information will be lost, and we may have
> 100 builds before that happen again, when a release already happened, and
> we have to backport it. Working early means working less.
>
> And finally, it makes me sad for the infra team. Doing a recheck is
> disrespecting all the work they're doing to create a reliable environment
> to run our tests. Sure, sometimes the environment is the reason the failure
> happens, but then it's even more important to give feedback about it. They
> provide a great deal of logs, we can use logstach to find patterns, the
> least we can do is trying. We're also using resources that other projects
> could be using. As much as we'd like to believe it, the cloud doesn't have
> free infinite resources.
>
> Recently, I've seen many cases where rechecks were made whereas:
> 1) The heat branch was broken. Generally for some external reason (a
> dependency updated), doing a recheck is a pure waste of resources until
> that failure is fixed. Most of the time, we say something on IRC when it's
> the case. We also try to open a bug, so looking at launchpad can show
> something.
> 2) THE PATCH WAS ACTUALLY BROKEN. And there I'm not sad anymore I'm
> particularly angry. It basically means that you didn't look at all at the
> build results, and just mindlessly typed rechecks hoping that some fairy
> will fix your broken code. Frankly, that makes want to go on a -2 rampage.
> ESPECIALLY where a core is doing it.
>
> To close, I'll try to provide a solution. I know we all have our agenda,
> debugging gate failures takes some time that you may not have, and
> obviously "my patch is fine it's not my fault" (who cares, that's what
> being in a team means). Still, I'd like everyone to look at the test
> failures, look if the patch is not the problem, and if not open a bug [1]
> mentioning the test name, pasting the traceback in the description, and
> linking the build result. Then do recheck bug #xyz. That's it. It shouldn't
> take you more than 3 minutes, and at least we didn't lose the information.
>
> Thanks for reading that far and sorry for the length,
>
>
> [1] https://bugs.launchpad.net/heat/+filebug
>
> --
> Thomas
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> Hi Thomas,
I hear you and I have the exact same feeling on the projects I am involved
in.


Testing and our CI is the root of OpenStack's product quality, it deserves
care, attention, involvement from everybody.


I take the opportunity of this email to put a link to our logstash here
http://logstash.openstack.org/ and to our Elasticrecheck project here
http://status.openstack.org/elastic-recheck/ (I only learnt about this
tools a few month ago and I think they bring a lot of value)

Cheers,
Jordan
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev