Re: [openstack-dev] [all] Why do we have project specific hacking rules?

[Ian Cordasco]

 

-Original Message-
From: Ihar Hrachyshka <ihrac...@redhat.com>
Reply: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Date: October 5, 2016 at 10:30:32
To: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Subject:  Re: [openstack-dev] [all] Why do we have project specific hacking 
rules?

> Ian Cordasco wrote:
>  
> > -Original Message-
> > From: Tony Breeds  
> > Reply: OpenStack Development Mailing List (not for usage questions)
> > , OpenStack Development Mailing List
> > (not for usage questions)  
> > Date: October 5, 2016 at 08:14:40
> > To: OpenStack Development Mailing List (not for usage questions)
> >  
> > Subject: Re: [openstack-dev] [all] Why do we have project specific
> > hacking rules?
> >
> >> On Wed, Oct 05, 2016 at 07:56:15AM -0500, Ian Cordasco wrote:
> >>
> >>> So hacking doesn't push out to anyone. It's one of the projects that
> >>> doesn't
> >>> get updated by global-requirements updates, if I remember correctly.
> >>
> >> Just clarifying/confirming what Ian says.
> >>
> >> The proposal-bot does not generate updates to projects
> >> *requirements.txt. It's
> >> up to the projects to do that themselves.
> >>
> >> Having said that it *could* all the code is there but it was disabled
> >> for a reason.
> >
> > Right. Thank you for clarifying, Tony.
> >
> > I believe several projects didn't want Hacking to auto-update and break
> > things. With off-by-default rules (and the proliferation of them in
> > Hacking) I don't think this is the most valid of concerns anymore. The
> > only problem would be that pycodestyle and pyflakes frequently add new
> > checks in releases means that the way Hacking pins Flake8 and
> > itsdependencies is still necessary. We need to figure out how to keep
> > up-to-date with our upstream dependencies without causing problems for
> > projects. Until we do that, we should probably just keep our current
> > methodology of letting projects update when they want to and can afford
> > developer time to update.
>  
> I believe those transitive dependencies could be effectively pinned thru
> upper-constraints.txt mechanism.

They're already pinned (not using upper-constraints). I'm sorry I wasn't clear 
about that.

--  
Ian Cordasco


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Why do we have project specific hacking rules?

[Ihar Hrachyshka]

Ian Cordasco <sigmaviru...@gmail.com> wrote:


-Original Message-
From: Tony Breeds <t...@bakeyournoodle.com>
Reply: OpenStack Development Mailing List (not for usage questions)  
<openstack-dev@lists.openstack.org>, OpenStack Development Mailing List  
(not for usage questions) <openstack-dev@lists.openstack.org>

Date: October 5, 2016 at 08:14:40
To: OpenStack Development Mailing List (not for usage questions)  
<openstack-dev@lists.openstack.org>
Subject:  Re: [openstack-dev] [all] Why do we have project specific  
hacking rules?



On Wed, Oct 05, 2016 at 07:56:15AM -0500, Ian Cordasco wrote:

So hacking doesn't push out to anyone. It's one of the projects that  
doesn't

get updated by global-requirements updates, if I remember correctly.


Just clarifying/confirming what Ian says.

The proposal-bot does not generate updates to projects  
*requirements.txt. It's

up to the projects to do that themselves.

Having said that it *could* all the code is there but it was disabled  
for a reason.


Right. Thank you for clarifying, Tony.

I believe several projects didn't want Hacking to auto-update and break  
things. With off-by-default rules (and the proliferation of them in  
Hacking) I don't think this is the most valid of concerns anymore. The  
only problem would be that pycodestyle and pyflakes frequently add new  
checks in releases means that the way Hacking pins Flake8 and  
itsdependencies is still necessary. We need to figure out how to keep  
up-to-date with our upstream dependencies without causing problems for  
projects. Until we do that, we should probably just keep our current  
methodology of letting projects update when they want to and can afford  
developer time to update.


I believe those transitive dependencies could be effectively pinned thru  
upper-constraints.txt mechanism.


Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Why do we have project specific hacking rules?

[Ian Cordasco]

-Original Message-
From: Tony Breeds <t...@bakeyournoodle.com>
Reply: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>, OpenStack Development Mailing List (not 
for usage questions) <openstack-dev@lists.openstack.org>
Date: October 5, 2016 at 08:14:40
To: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Subject:  Re: [openstack-dev] [all] Why do we have project specific hacking 
rules?

> On Wed, Oct 05, 2016 at 07:56:15AM -0500, Ian Cordasco wrote:
>  
> > So hacking doesn't push out to anyone. It's one of the projects that doesn't
> > get updated by global-requirements updates, if I remember correctly.
>  
> Just clarifying/confirming what Ian says.
>  
> The proposal-bot does not generate updates to projects *requirements.txt. It's
> up to the projects to do that themselves.
>  
> Having said that it *could* all the code is there but it was disabled for a 
> reason.

Right. Thank you for clarifying, Tony.

I believe several projects didn't want Hacking to auto-update and break things. 
With off-by-default rules (and the proliferation of them in Hacking) I don't 
think this is the most valid of concerns anymore. The only problem would be 
that pycodestyle and pyflakes frequently add new checks in releases means that 
the way Hacking pins Flake8 and itsdependencies is still necessary. We need to 
figure out how to keep up-to-date with our upstream dependencies without 
causing problems for projects. Until we do that, we should probably just keep 
our current methodology of letting projects update when they want to and can 
afford developer time to update.

--  
Ian Cordasco


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Why do we have project specific hacking rules?

[Tony Breeds]

On Wed, Oct 05, 2016 at 07:56:15AM -0500, Ian Cordasco wrote:

> So hacking doesn't push out to anyone. It's one of the projects that doesn't
> get updated by global-requirements updates, if I remember correctly.

Just clarifying/confirming what Ian says.

The proposal-bot does not generate updates to projects *requirements.txt.  It's
up to the projects to do that themselves.

Having said that it *could* all the code is there but it was disabled for a 
reason.

Yours Tony.


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Why do we have project specific hacking rules?

[Ian Cordasco]

 

-Original Message-
From: Sean Dague <s...@dague.net>
Reply: OpenStack Development Mailing List (not for usage questions) 
<openstack-dev@lists.openstack.org>
Date: October 5, 2016 at 07:04:17
To: openstack-dev@lists.openstack.org <openstack-dev@lists.openstack.org>
Subject:  Re: [openstack-dev] [all] Why do we have project specific hacking 
rules?

> On 10/03/2016 03:01 AM, Ihar Hrachyshka wrote:
> > Andrew Laski wrote:
> >>
> >> A further hindrance to moving these checks to be OpenStack wide is that
> >> each check violation is a failure. So in order to roll a new check out
> >> across projects it requires a lot of churn in each project which
> >> violates the checks. I would prefer to leave it up to each project to
> >> make the decision to incorporate a new check and take the review load.
> >> Ultimately what I think sounds good would be a central listing of checks
> >> that are in use across projects and then leave it up to each project to
> >> replicate those that they like.
> >
> > Late flake8 releases support off_by_default decorator that allows to add
> > checks that are not triggered unless explicitly enabled in tox.ini with
> > select= directive. That should allow to maintain code in single place
> > while still having projects to opt-in new checks.
>  
> There could definitely be an effort to move more content up into hacking
> proper. It would require some genericizing and need folks focused on it.
>  
> One of the challenges is realizing that hacking updates push out to 50+
> projects. The project specific checks are often things where people have
> gotten the point of getting bored -1ing people for the same issues over
> and over again. Or using hacking a bit as a light weight unit test to
> make sure certain code doesn't exist (all those "don't use this code"
> checks).

So hacking doesn't push out to anyone. It's one of the projects that doesn't 
get updated by global-requirements updates, if I remember correctly.

> Joe Gordon used to be pretty aggressive in moving these checks up into
> hacking when they seemed to get enough concensus. But since he left the
> community, there has been less push on this.

The other problem with this is two-fold:

1. As one of the few people reviewing Hacking with more than a "The code looks 
good" point of view, it would be nice to have people from different projects 
affirming that more than one project would like the check in Hacking.

2. I'm not actually paid to work on Hacking or much upstream. What time I use 
to work on Hacking is my free time and that's already over-utilized.

If people think that absorbing all project-specific rules into Hacking is 
really that beneficial, I'd expect them to start participating a lot more 
actively in reviews of those changes. I won't have the time to pull down each 
one and verify it works as expected and won't break projects if it is enabled.

--  
Ian Cordasco


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Why do we have project specific hacking rules?

[Sean Dague]

On 10/03/2016 03:01 AM, Ihar Hrachyshka wrote:
> Andrew Laski  wrote:
>>
>> A further hindrance to moving these checks to be OpenStack wide is that
>> each check violation is a failure. So in order to roll a new check out
>> across projects it requires a lot of churn in each project which
>> violates the checks. I would prefer to leave it up to each project to
>> make the decision to incorporate a new check and take the review load.
>> Ultimately what I think sounds good would be a central listing of checks
>> that are in use across projects and then leave it up to each project to
>> replicate those that they like.
> 
> Late flake8 releases support off_by_default decorator that allows to add
> checks that are not triggered unless explicitly enabled in tox.ini with
> select= directive. That should allow to maintain code in single place
> while still having projects to opt-in new checks.

There could definitely be an effort to move more content up into hacking
proper. It would require some genericizing and need folks focused on it.

One of the challenges is realizing that hacking updates push out to 50+
projects. The project specific checks are often things where people have
gotten the point of getting bored -1ing people for the same issues over
and over again. Or using hacking a bit as a light weight unit test to
make sure certain code doesn't exist (all those "don't use this code"
checks).

Joe Gordon used to be pretty aggressive in moving these checks up into
hacking when they seemed to get enough concensus. But since he left the
community, there has been less push on this.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Why do we have project specific hacking rules?

[Ihar Hrachyshka]

Andrew Laski  wrote:


A further hindrance to moving these checks to be OpenStack wide is that
each check violation is a failure. So in order to roll a new check out
across projects it requires a lot of churn in each project which
violates the checks. I would prefer to leave it up to each project to
make the decision to incorporate a new check and take the review load.
Ultimately what I think sounds good would be a central listing of checks
that are in use across projects and then leave it up to each project to
replicate those that they like.


Late flake8 releases support off_by_default decorator that allows to add  
checks that are not triggered unless explicitly enabled in tox.ini with  
select= directive. That should allow to maintain code in single place while  
still having projects to opt-in new checks.


Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Why do we have project specific hacking rules?

[Andrew Laski]

On Sun, Oct 2, 2016, at 11:02 AM, Matt Riedemann wrote:
> On 10/2/2016 5:47 AM, Amrith Kumar wrote:
> > It was my understanding that hacking rules were like the 'Ten 
> > Commandments', the 'Four Opens'; things that were universally true across 
> > all projects and an attempt to bring standardization to all OpenStack code.
> >
> > How come we then have extensive project specific hacking rules? Why not 
> > make these "nova-specific" rules OpenStack wide?
> >
> > I looked at the checks.py file that Matt linked to below, and I can't see 
> > anything really "nova-specific"; i.e. all of them would translate just fine 
> > to Trove. Is there some reason they can't become OpenStack wide rules?

For some of them there are good reasons not to make them OpenStack wide
checks.

http://git.openstack.org/cgit/openstack/nova/tree/nova/hacking/checks.py?h=stable/newton#n155
no db usage in virt driver.

http://git.openstack.org/cgit/openstack/nova/tree/nova/hacking/checks.py?h=stable/newton#n168
no db session in public db api

http://git.openstack.org/cgit/openstack/nova/tree/nova/hacking/checks.py?h=stable/newton#n201
virt drivers aren't importing modules from other virt drivers

http://git.openstack.org/cgit/openstack/nova/tree/nova/hacking/checks.py?h=stable/newton#n220
virt drivers aren't using config options from other virt drivers

http://git.openstack.org/cgit/openstack/nova/tree/nova/hacking/checks.py?h=stable/newton#n406
api microversion decorator is first decorator when used

http://git.openstack.org/cgit/openstack/nova/tree/nova/hacking/checks.py?h=stable/newton#n616
check that nova specific utility method is used rather than
greenthread|eventlet.spawn|spawn_n

http://git.openstack.org/cgit/openstack/nova/tree/nova/hacking/checks.py?h=stable/newton#n643
config options are registered in a central place

http://git.openstack.org/cgit/openstack/nova/tree/nova/hacking/checks.py?h=stable/newton#n667
policy options are registered in a central place

http://git.openstack.org/cgit/openstack/nova/tree/nova/hacking/checks.py?h=stable/newton#n681
a nova specific context.can() method must be used for policy enforcement
rather than policy.enforce|authorize

All of these checks are either very specific to Nova or are
philosophical choices that have been made in Nova but don't need to be
shared by other projects.

And as Matt points out below some of the checks not listed above depend
on specific paths being used in the check which makes it more difficult
to share those checks.

A further hindrance to moving these checks to be OpenStack wide is that
each check violation is a failure. So in order to roll a new check out
across projects it requires a lot of churn in each project which
violates the checks. I would prefer to leave it up to each project to
make the decision to incorporate a new check and take the review load.
Ultimately what I think sounds good would be a central listing of checks
that are in use across projects and then leave it up to each project to
replicate those that they like.


> >
> > -amrith
> >
> >> -Original Message-
> >> From: Ihar Hrachyshka [mailto:ihrac...@redhat.com]
> >> Sent: Sunday, October 02, 2016 5:25 AM
> >> To: OpenStack Development Mailing List (not for usage questions)
> >> 
> >> Subject: Re: [openstack-dev] [all][i18n] do we need translation mark for
> >> strings in tests?
> >>
> >> Matt Riedemann  wrote:
> >>
> >>> On 10/1/2016 5:49 PM, Matt Riedemann wrote:
>  No you shouldn't need to mark strings for translation in test code. I
>  believe we have a hacking rule for marking LOG.info/warning/error
>  messages for translation but it should skip test directories.
> >>>
> >>> Ah I guess the hacking rule is nova-specific:
> >>>
> >>>
> >> https://github.com/openstack/nova/blob/2851ceaed3010c19f42e308be637b952eda
> >> b092a/nova/hacking/checks.py#L342
> >>
> >> We have a similar one in neutron; but note that it does not explicitly
> >> FAIL
> >> on translation markers in test code; it instead fails on no markers in
> >> production code, which is different different.
> >>
> >> Ihar
> >>
> >> __
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> > __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> 
> Well for one thing the specific one I pointed out has hard-coded nova 
> paths in it which might not work for other projects.
> 
> -- 
> 
> Thanks,
> 
> Matt Riedemann
> 
> 
> 

Re: [openstack-dev] [all] Why do we have project specific hacking rules?

[Matt Riedemann]

On 10/2/2016 5:47 AM, Amrith Kumar wrote:

It was my understanding that hacking rules were like the 'Ten Commandments', 
the 'Four Opens'; things that were universally true across all projects and an 
attempt to bring standardization to all OpenStack code.

How come we then have extensive project specific hacking rules? Why not make these 
"nova-specific" rules OpenStack wide?

I looked at the checks.py file that Matt linked to below, and I can't see anything really 
"nova-specific"; i.e. all of them would translate just fine to Trove. Is there 
some reason they can't become OpenStack wide rules?

-amrith


-Original Message-
From: Ihar Hrachyshka [mailto:ihrac...@redhat.com]
Sent: Sunday, October 02, 2016 5:25 AM
To: OpenStack Development Mailing List (not for usage questions)

Subject: Re: [openstack-dev] [all][i18n] do we need translation mark for
strings in tests?

Matt Riedemann  wrote:


On 10/1/2016 5:49 PM, Matt Riedemann wrote:

No you shouldn't need to mark strings for translation in test code. I
believe we have a hacking rule for marking LOG.info/warning/error
messages for translation but it should skip test directories.


Ah I guess the hacking rule is nova-specific:



https://github.com/openstack/nova/blob/2851ceaed3010c19f42e308be637b952eda
b092a/nova/hacking/checks.py#L342

We have a similar one in neutron; but note that it does not explicitly
FAIL
on translation markers in test code; it instead fails on no markers in
production code, which is different different.

Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



Well for one thing the specific one I pointed out has hard-coded nova 
paths in it which might not work for other projects.


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all] Why do we have project specific hacking rules?

[Amrith Kumar]

It was my understanding that hacking rules were like the 'Ten Commandments', 
the 'Four Opens'; things that were universally true across all projects and an 
attempt to bring standardization to all OpenStack code.

How come we then have extensive project specific hacking rules? Why not make 
these "nova-specific" rules OpenStack wide?

I looked at the checks.py file that Matt linked to below, and I can't see 
anything really "nova-specific"; i.e. all of them would translate just fine to 
Trove. Is there some reason they can't become OpenStack wide rules?

-amrith

> -Original Message-
> From: Ihar Hrachyshka [mailto:ihrac...@redhat.com]
> Sent: Sunday, October 02, 2016 5:25 AM
> To: OpenStack Development Mailing List (not for usage questions)
> 
> Subject: Re: [openstack-dev] [all][i18n] do we need translation mark for
> strings in tests?
> 
> Matt Riedemann  wrote:
> 
> > On 10/1/2016 5:49 PM, Matt Riedemann wrote:
> >> No you shouldn't need to mark strings for translation in test code. I
> >> believe we have a hacking rule for marking LOG.info/warning/error
> >> messages for translation but it should skip test directories.
> >
> > Ah I guess the hacking rule is nova-specific:
> >
> >
> https://github.com/openstack/nova/blob/2851ceaed3010c19f42e308be637b952eda
> b092a/nova/hacking/checks.py#L342
> 
> We have a similar one in neutron; but note that it does not explicitly
> FAIL
> on translation markers in test code; it instead fails on no markers in
> production code, which is different different.
> 
> Ihar
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev