Re: [Python-Dev] dear core-devs

[Steven D'Aprano]

On Thu, Oct 04, 2018 at 09:55:10AM +0200, Petr Viktorin wrote:

> Is paying the best way to get features into Python?

No, but it is *a* way to get features into Python.

It may be a good way to get a feature that is important to (generic) 
you, but the core devs don't have enough time, interest or energy to add 
in their own free time.

It is not a way to get features that are opposed by the core devs.


> Does becoming a core 
> dev mean you can now get paid for approving changes? Some of the 
> implications are quite disturbing :(

Naturally whenever money is involved, there is the risks of a conflict 
of interest, and of course we should be careful about such risks. But 
they are small and manageable risks.

We're unlikely to be talking about huge amounts of money, enough to 
corrupt people, and so long as there is transparency about who does what 
and why, open source and free software is compatible with payment for 
work done. Even Richard Stallman accepts money for code :-)


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dear core-devs

[Michael Felt]

On 10/4/2018 9:55 AM, Petr Viktorin wrote:
> On 10/4/18 9:34 AM, Victor Stinner wrote:
>> Hi,
>>
>> If IBM wants a better Python support, it would help a lot if IBM pays
>> for this development.
I agree. If IBM ...
>> ... Antoine Pitrou has been paid in the past to enhance Python
>> support in Solaris and it worked well.
>
FYI - as I now have access to the gccfarm, and in the spirit of more
generalized "posix" additions I looked for an HPUX and a Solais system
to build master on.

make test never finished (one test was still hanging after over 20
minutes, and I had to go. Of the 419, 17 or 18 had failed. Roughly where
AIX plus xlc was at last July without my PRs for tests.

So, while it worked - money stopped and Solaris is in no better
numerical shape (test wise) than AIX.
> Michael explicitly said this is a personal effort. IBM or other big
> money is not involved.
IBM is my employer. As I am not a developer (merely a systems and
management consultant) I do not face losing my job by working on OSS. I
have been called off certain OSS projects because IBM was providing
money and/or developers. This is one of the reasons (being called off
elsewhere) that I have been hesitant to be more involved than I was in
2015-2017.

So, let me be explicit - I can only speak for myself. And as long as no
manager says "No, cannot work on that" I have given a commitment to work
on this. "Some things cannot be bought" - such as un-biased (I call it
"maverick" rather than merely independent.) On the one hand IBM policy
is to encourage independent thought. The core goal is to help customers
succeed. But individual managers up and down the line occasionally have
additional business needs, and then workers as myself apologize and take
a step back - in a word - adjust.

Short answer: my involvement is mine to give at no price. I am
considered one of the worlds AIX experts on matters of integration,
performance and security.

So, I have just simple questions for you? Do you value my expertise? May
I assist?

>
> Is paying the best way to get features into Python? Does becoming a
> core dev mean you can now get paid for approving changes? Some of the
> implications are quite disturbing :(
>

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dear core-devs

[Petr Viktorin]

On 10/4/18 9:34 AM, Victor Stinner wrote:

Hi,

If IBM wants a better Python support, it would help a lot if IBM pays 
for this development. With money, you can easily find core dev 
contractors. Antoine Pitrou has been paid in the past to enhance Python 
support in Solaris and it worked well.


Michael explicitly said this is a personal effort. IBM or other big 
money is not involved.


Is paying the best way to get features into Python? Does becoming a core 
dev mean you can now get paid for approving changes? Some of the 
implications are quite disturbing :(



Le mercredi 3 octobre 2018, Michael Felt > a écrit :

 >
 >
 > On 10/2/2018 11:34 PM, Terry Reedy wrote:
 >> On 10/2/2018 12:41 PM, Simon Cross wrote:
 >>> Are there any core devs that Michael or Erik could collaborate with?
 >>> Rather than rely on adhoc patch review from random core developers.
 >>
 >> You two might collaborate with each other to the extent of reviewing
 >> some of each other's PRs.
 > Might be difficult. We both, or at least I, claim ignorance of the
 > others platform. I still have a lot of PEP to learn, and my idea of a
 > bug-fix (for Python2) was seen by core-dev as a feature change. I would
 > not feel comfortable trying to mentor someone in things PEP, etc..
 >> That still leaves the issue of merging.
 > How much confidence is there in all the "CI" tests? Does that not offer
 > sufficient confidence for a core-dev to press merge.
 > How about "master" continuing to be what it is, but insert a new
 > "pre-master" branch that the buildbots actually test on (e.g., what is
 > now the 3.X) and have a 3.8 buildbot - for what is now the "master".
 >
 > PR would still be done based on master, but an "initial" merge would be
 > via the pre-master aka 3.X buildbot tests.
 >
 > How "friendly" git is - that it not become such a workload to keep it
 > clean - I cannot say. Still learning to use git. Better, but still do
 > not want to assume it would be easy.
 >
 > My hope is that it would make it easier to consider a "merge" step that
 > gets all the buildbots involved for even broader CI tests.
 >
 >>
 >>> Michael and Eric: Question -- are you interested in becoming core
 >>> developers at least for the purposes of maintaining these platforms in
 >>> future?
 >>
 >> Since adhoc is not working to get merges, I had this same suggestion.
 >> Michael and Erik, I presume you have gotten some guidelines on what
 >> modifications to C code might be accepted, and what concerns people 
have.

 > imho: guidelines - paraphrased - as little as possible :)
 >
 > I have many assumptions, and one of those is that my assumptions are
 > probably incorrect.
 > Goal: have AIX recognized as a Stable platform, even if not in the
 > highest supported category.
 > And that implies, support as far as I am able, to keep it "Stable".
 >>
 >> I think for tests, a separate test_aix.py might be a good idea for
 >> aix-only tests
 > Unclear to me how this would work. Too young in Python I guess (or just
 > a very old dog), but what test would be needed for AIX, or any other
 > platform, that would not need to be tested in some fashion for the
 > 'other' platforms. At a hunch, where there are many platform.system()
 > dependencies expected (e.g., test_posix, maybe doing something in the
 > class definition (is there a "Root" Object/Class that all inherit from.
 > Maybe a (read-only) "root" attribute (or is property better?) could be
 > the value of platform.system(), and iirc, might be used by as @property
 > in unittest. (so, if not in "root" class, then in something like
 > unittest/__init__.py.
 >
 > I hope to be "close" in "Python thinking" - enough that someone who
 > actually knows how the pieces fit together could come with a better, and
 > more appropriate guideline/implementation.
 >
 >> , while modification of other tests might be limited to adding skips.
 >> The idea would be to make it easy to remove aix stuff in the future if
 >> it again became unsupported.
 > IMHO: IBM and AIX do not mention it, but for openstack cloudmanagement
 > (very specifically cloud-init) AIX needs a recognized stable Python
 > implementation. I am "surprised" in the level of communication of IBM
 > with Python community.
 >
 > Personally, I do not see AIX as a specialized platform. Feels more like
 > the "last-standing" fully supported (commercial OEM) 'POSIX-UNIX'. Of
 > course my focus is narrow - so maybe there is a lot of support for
 > commercial platforms such as HPUX, Solaris, and other mainstream UNIXes.
 > Feel free to correct me!!
 >> Ditto for other specialized platforms.
 >>
 >>
 >>
 >>
 >
 > ___
 > Python-Dev mailing list
 > Python-Dev@python.org 
 > https://mail.python.org/mailman/listinfo/python-dev
 > Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com

 >

___
Python-Dev mai

Re: [Python-Dev] dear core-devs

[Victor Stinner]

Hi,

If IBM wants a better Python support, it would help a lot if IBM pays for
this development. With money, you can easily find core dev contractors.
Antoine Pitrou has been paid in the past to enhance Python support in
Solaris and it worked well.

Victor

Le mercredi 3 octobre 2018, Michael Felt  a écrit :
>
>
> On 10/2/2018 11:34 PM, Terry Reedy wrote:
>> On 10/2/2018 12:41 PM, Simon Cross wrote:
>>> Are there any core devs that Michael or Erik could collaborate with?
>>> Rather than rely on adhoc patch review from random core developers.
>>
>> You two might collaborate with each other to the extent of reviewing
>> some of each other's PRs.
> Might be difficult. We both, or at least I, claim ignorance of the
> others platform. I still have a lot of PEP to learn, and my idea of a
> bug-fix (for Python2) was seen by core-dev as a feature change. I would
> not feel comfortable trying to mentor someone in things PEP, etc..
>> That still leaves the issue of merging.
> How much confidence is there in all the "CI" tests? Does that not offer
> sufficient confidence for a core-dev to press merge.
> How about "master" continuing to be what it is, but insert a new
> "pre-master" branch that the buildbots actually test on (e.g., what is
> now the 3.X) and have a 3.8 buildbot - for what is now the "master".
>
> PR would still be done based on master, but an "initial" merge would be
> via the pre-master aka 3.X buildbot tests.
>
> How "friendly" git is - that it not become such a workload to keep it
> clean - I cannot say. Still learning to use git. Better, but still do
> not want to assume it would be easy.
>
> My hope is that it would make it easier to consider a "merge" step that
> gets all the buildbots involved for even broader CI tests.
>
>>
>>> Michael and Eric: Question -- are you interested in becoming core
>>> developers at least for the purposes of maintaining these platforms in
>>> future?
>>
>> Since adhoc is not working to get merges, I had this same suggestion.
>> Michael and Erik, I presume you have gotten some guidelines on what
>> modifications to C code might be accepted, and what concerns people have.
> imho: guidelines - paraphrased - as little as possible :)
>
> I have many assumptions, and one of those is that my assumptions are
> probably incorrect.
> Goal: have AIX recognized as a Stable platform, even if not in the
> highest supported category.
> And that implies, support as far as I am able, to keep it "Stable".
>>
>> I think for tests, a separate test_aix.py might be a good idea for
>> aix-only tests
> Unclear to me how this would work. Too young in Python I guess (or just
> a very old dog), but what test would be needed for AIX, or any other
> platform, that would not need to be tested in some fashion for the
> 'other' platforms. At a hunch, where there are many platform.system()
> dependencies expected (e.g., test_posix, maybe doing something in the
> class definition (is there a "Root" Object/Class that all inherit from.
> Maybe a (read-only) "root" attribute (or is property better?) could be
> the value of platform.system(), and iirc, might be used by as @property
> in unittest. (so, if not in "root" class, then in something like
> unittest/__init__.py.
>
> I hope to be "close" in "Python thinking" - enough that someone who
> actually knows how the pieces fit together could come with a better, and
> more appropriate guideline/implementation.
>
>> , while modification of other tests might be limited to adding skips.
>> The idea would be to make it easy to remove aix stuff in the future if
>> it again became unsupported.
> IMHO: IBM and AIX do not mention it, but for openstack cloudmanagement
> (very specifically cloud-init) AIX needs a recognized stable Python
> implementation. I am "surprised" in the level of communication of IBM
> with Python community.
>
> Personally, I do not see AIX as a specialized platform. Feels more like
> the "last-standing" fully supported (commercial OEM) 'POSIX-UNIX'. Of
> course my focus is narrow - so maybe there is a lot of support for
> commercial platforms such as HPUX, Solaris, and other mainstream UNIXes.
> Feel free to correct me!!
>> Ditto for other specialized platforms.
>>
>>
>>
>>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/vstinner%40redhat.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dear core-devs

[Michael Felt]

On 10/3/2018 1:46 AM, Neil Schemenauer wrote:
> On 2018-10-02, Michael Felt wrote:
>> I am sorry, for myself obviously - but also for Python. Obviously, I am
>> doing it all wrong - as I see lots of other issues being picked up
>> immediately.
> I'm not sure that's the case.  There are a lot of PRs or bugs that
> sit there without getting reviews.  The problem is that few (or no)
> core developers get paid to work on Python.  So, the time they spend
> is motivated by their specific "itch".  Getting reviews on any PR is
> difficult, even for core developers.  In their case, they have to
> option of forcing the issue, I guess.
>
> This is a problem we should try to deal with somehow.  Turning off
> valuable contributors like you is bad.  I'm not sure how to do it
> though.  At the core Python sprint in September there was some talk
> about how CPython developers might get funding.  Maybe that could
> help deal with the backlog of reviews required.
>
>> And, while you may not give a damn about anything other than Windows,
>> macos and/or Linux - there are other platforms that would like a stable
>> Python.
> There is probably some truth in not caring about other platforms.
> The problem from the reviewer perspective is the question of "what
> is the potential downsides of this PR vs what are the benefits?".
> The safest thing is to not approve the PR.  No core developer wants
> to be the person who broke CPython.  You must admit, AIX is an
> extremely niche platform at this point.  I bet if you picked 1000
> software developers at random, it would be likely that zero of them
> have ever used AIX.  So, it's not that we don't care at all about
> AIX but that the cost/benefit equation makes accepting AIX specific
> changes more difficult.
Nods. However - this is a chicken/egg issue (imho). AIX is seen a weak
platform because noone has ever tackled these. When I started on this I
had never expected to have found a resolution to them all.

Platforms have differences and when the tests miss that difference that
the tests give a false result. e.g., one accepted PR was because AIX
libc printf() output for printf(NULL) is "" while other platforms output
"(null)".


>
> One specific suggestion I have about your PR is to try to make your
> changes not AIX specific.  Or at least, make the AIX checking as
> localized as possible.  So, as an example, in test_uuid you have:
>
> _notAIX = not sys.platform.startswith("aix")
a) I thought/hoped this was better practice and performance - calling 
sys.platform.startswith("aix")only once, rather than X times.
b) more maintainable (e.g., change to not platform.system()
c) iirc - this got changed to AIX = , and throughout the test is "if
not AIX"...
>
> then later in the module you check that flag.  While that is the
> most direct approach to fixing the issue and making the test pass,
> it is not good for the long term maintainability of the code.  You
> end up with boolean flags like _notAIX spread about the logic.  Over
> time, code like that becomes a nightmare to maintain.
>
> Instead, I would suggest test_uuid is making platform specific
> assumptions that are not true on AIX and possibly other platforms.
> So, do something like:
>
> 
> _IS_AIX = sys.platform.startswith("aix")
better name.
>
> _HAVE_MACADDR = (os.name == 'posix' and not _IS_AIX)
AIX has MACADDR, but formatted with '.' rather than ':' and uses a
single hex-digit when value between dots is < 16 (decimal)
>
> @unittest.skipUnless(_HAVE_MACADDR, 'requires Posix with macaddr')
> def test_arp_getnode(self):
> ...
>
> The _HAVE_MACADDR test is relatively simple and clear, does this
> platform have this capability.  Later in the code, a check for
> _HAVE_MACADDR is also quite clear.  If someone comes along with
> another platform that doesn't support macaddr, they only have to
> change one line of code.
>
> This kind of capability checking is similar to what happened with
> web browsers.  In that case, people discovered that checking the
> User Agent header was a bad idea.  Instead, you should probe for
> specific functionality and not assume based on browser IDs.  For the
> macaddr case, is there some way to you probe the arp command to see
> if supports macaddr? 
I suppose if someone had written the original test with "check program
to see if ..." it would have worked already.
I am trying to get current tests to work with minimal changes.

I am certainly not "blaming" anyone for not knowing this unique behavior
of this platform. Before debugging this I did not know of the difference
either. I agree that wherever a generic resolution is possible - it
should be done. However, as an "outsider" I do not feel empowered enough
to propose such a change to existing (and well proven for other
platforms) tests.
>  That way your test doesn't have to include any
> AIX specific check at all.  Further, it would have some hope of
> working on platforms other than AIX that also don't support mac

Re: [Python-Dev] dear core-devs

[Michael Felt]

On 10/3/2018 2:48 AM, Terry Reedy wrote:
> On 10/2/2018 7:16 PM, Michael Felt wrote:
>>
>>
>> On 10/2/2018 11:34 PM, Terry Reedy wrote:
>>> On 10/2/2018 12:41 PM, Simon Cross wrote:
 Are there any core devs that Michael or Erik could collaborate with?
 Rather than rely on adhoc patch review from random core developers.
>>>
>>> You two might collaborate with each other to the extent of reviewing
>>> some of each other's PRs.
>
>> Might be difficult. We both, or at least I, claim ignorance of the
>> others platform.
>
> Partial reviews, short of accept/change are better than no review and
> can make a merge decision easier for a core dev.  You should each be
> or become familiar with PEP 7 and somewhat familiar with local C
> idioms. Do names follow local standards.  Do C-API calls make sense.
Sounds simple enough. The tricky part is "the details".
>
> >>  I still have a lot of PEP to learn, and my idea of a
> >> bug-fix (for Python2) was seen by core-dev as a feature change.
>
> Failures of current tests would seem to me to be bugs.  However, some
> bug fixes require a feature change.  It is an awkward situation.  We
> are increasingly reluctant to patch 2.7.
Some are quite simple to fix, even if hard to find: such as:
"elif cmd is None:" -> "elif notcmd orcmd is None:"

Some are not bugs at all - very hard to find! Instead, "textual"
differences because a library is overly optimized - the expected
exception occurs - but no error message. Linking with a less optimized
(libssl.a and libcrypto.a) resolved many reported test "failures".

Nearly three years ago I was keen to see things in Python2(.7), but not
so much now. I also feel the time is to push hard towards current
Python3 versions.
>
>>> That still leaves the issue of merging.
>> How much confidence is there in all the "CI" tests? Does that not offer
>> sufficient confidence for a core-dev to press merge.
>
> Code for new features or bugs that escaped the tests should have new
> tests.  AIX-specific code should (as in must ;-) be tested before
> being submitted, since it will not be properly tested by CI.  With CI
> now covering Windows twice, Linux twice, and Mac, I believe it has
> become rarer for buildbots to fail after CI passes.  Victor would know.
>
> I  believe that you are initially dealing with bugs that do not pass
> current tests.
I am dealing with tests that do not pass. The dilemma: what is wrong -
the test, or what it is testing? Generally speaking, I cannot call
Python3 (master) broken. So I look for a "root cause" in a test
assumption that is wrong, and find a way to correct that.

Sometimes, it is a bit of both - and those are very hard to resolve
without feedback.

See the discussion, elsewhere, regarding MACADDR. It has never been that
platform Y does not have a MACADDR - rather, platform Y formats it
differently than (all) other platforms.

>
>> How about "master" continuing to be what it is, but insert a new
>> "pre-master" branch that the buildbots actually test on (e.g., what is
>> now the 3.X) and have a 3.8 buildbot - for what is now the "master".
>>
>> PR would still be done based on master, but an "initial" merge would be
>> via the pre-master aka 3.X buildbot tests.
>>
>> How "friendly" git is - that it not become such a workload to keep it
>> clean - I cannot say. Still learning to use git. Better, but still do
>> not want to assume it would be easy.
>
> Too complicated.
>
>> My hope is that it would make it easier to consider a "merge" step that
>> gets all the buildbots involved for even broader CI tests.
>
> I considered the wider buildbot fleet to be post-merge CI ;-).
>
>>> I think for tests, a separate test_aix.py might be a good idea for
>>> aix-only tests
>
> I may be wrong on this.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/aixtools%40felt.demon.nl

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dear core-devs

[Erik Bray]

On Tue, Oct 2, 2018 at 8:54 PM Michael Felt  wrote:
>
>
>
> On 10/2/2018 4:45 PM, Erik Bray wrote:
> > Michael, if there are any PRs you want to point me to that I might be
> > able to help review please do.
> A little trick I learned:
> https://github.com/python/cpython/pulls?q=is%3Aopen+is%3Apr+author%3Aaixtools+sort%3Aupdated-desc
> lists them all.

Cool, I'll have a look.

> What "flipped my switch" yesterday was discovering a PR that I was
> gifted (by an ex? core-dev) and put in the system back in January is now
> broken by a patch merged about two weeks ago. Worse, pieces of
> test_ctypes(bitfields) that previously worked when using __xlc__ seem to
> be broken. Which highlighted the "time pressure" of getting tests to
> pass so that regressions can be seen.

Yes, that can certainly happen.  I have many PRs floating around on
different projects that, you know, get stalled for months and months
and inevitably break.  It's extremely frustrating, but we've all been
there :)

> If you let me know what info you would need (I gave lots of debug info
> two years ago to get that initial fix).
>
> And, I guess the other "larger" change re: test_distutils. Also, some
> issues specific to xlc being different from gcc.
>
> Those two do not show on the gccfarm buildbot.
>
> Many thanks for the offer! I'll try to not take more than the hand offered!
> >   I don't know anything about AIX either
> > and am not a core dev so I can't have a final say.  But I've been
> > hacking on CPython for a long time anyways, and might be able to help
> > at least with some initial review.

I also about to ask if you have a buildbot for AIX, and I see now you
have several. So step in the right direction!
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dear core-devs

[Erik Bray]

On Tue, Oct 2, 2018 at 6:41 PM Simon Cross
 wrote:
>
> Are there any core devs that Michael or Erik could collaborate with?
> Rather than rely on adhoc patch review from random core developers.
>
> Michael and Eric: Question -- are you interested in becoming core
> developers at least for the purposes of maintaining these platforms in
> future?

I would be for the purposes of said platform maintenance.  I believe I
already have some maintainer permissions on bpo for exactly this
reason.

That said, while I'm sure it would help, I'm not exactly sure what it
would solve either.  I believe strongly in code review, and just
having a "core developer" status does not necessarily free one from
responsibility for obtaining code review.

It also partly depends on the issue.  If it's a change that touches
other parts of the code in ways that could impact it beyond the narrow
scope of platform support, I believe it definitely should get a second
pair of eyes.  Unfortunately many of the outstanding patches I have
for review fall in that category.  Though in the future there will be
fewer like that.  The majority of work needed for Cygwin, at least, is
tweaking some areas of the tests that make assumptions that don't
necessarily hold on that platform.*

Thanks,
E


* For example, there are some tests that assume there is a user with
UID 0.  While UID 0 is reserved for a "superuser", I don't know that
there's any requirement that such a user *must* exist (on Cygwin it
does not :)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dear core-devs

[Terry Reedy]

On 10/2/2018 7:16 PM, Michael Felt wrote:



On 10/2/2018 11:34 PM, Terry Reedy wrote:

On 10/2/2018 12:41 PM, Simon Cross wrote:

Are there any core devs that Michael or Erik could collaborate with?
Rather than rely on adhoc patch review from random core developers.


You two might collaborate with each other to the extent of reviewing
some of each other's PRs.



Might be difficult. We both, or at least I, claim ignorance of the
others platform.


Partial reviews, short of accept/change are better than no review and 
can make a merge decision easier for a core dev.  You should each be or 
become familiar with PEP 7 and somewhat familiar with local C idioms. 
Do names follow local standards.  Do C-API calls make sense.


>>  I still have a lot of PEP to learn, and my idea of a
>> bug-fix (for Python2) was seen by core-dev as a feature change.

Failures of current tests would seem to me to be bugs.  However, some 
bug fixes require a feature change.  It is an awkward situation.  We are 
increasingly reluctant to patch 2.7.



That still leaves the issue of merging.

How much confidence is there in all the "CI" tests? Does that not offer
sufficient confidence for a core-dev to press merge.


Code for new features or bugs that escaped the tests should have new 
tests.  AIX-specific code should (as in must ;-) be tested before being 
submitted, since it will not be properly tested by CI.  With CI now 
covering Windows twice, Linux twice, and Mac, I believe it has become 
rarer for buildbots to fail after CI passes.  Victor would know.


I  believe that you are initially dealing with bugs that do not pass 
current tests.



How about "master" continuing to be what it is, but insert a new
"pre-master" branch that the buildbots actually test on (e.g., what is
now the 3.X) and have a 3.8 buildbot - for what is now the "master".

PR would still be done based on master, but an "initial" merge would be
via the pre-master aka 3.X buildbot tests.

How "friendly" git is - that it not become such a workload to keep it
clean - I cannot say. Still learning to use git. Better, but still do
not want to assume it would be easy.


Too complicated.


My hope is that it would make it easier to consider a "merge" step that
gets all the buildbots involved for even broader CI tests.


I considered the wider buildbot fleet to be post-merge CI ;-).


I think for tests, a separate test_aix.py might be a good idea for
aix-only tests


I may be wrong on this.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dear core-devs

[Neil Schemenauer]

On 2018-10-02, Michael Felt wrote:
> I am sorry, for myself obviously - but also for Python. Obviously, I am
> doing it all wrong - as I see lots of other issues being picked up
> immediately.

I'm not sure that's the case.  There are a lot of PRs or bugs that
sit there without getting reviews.  The problem is that few (or no)
core developers get paid to work on Python.  So, the time they spend
is motivated by their specific "itch".  Getting reviews on any PR is
difficult, even for core developers.  In their case, they have to
option of forcing the issue, I guess.

This is a problem we should try to deal with somehow.  Turning off
valuable contributors like you is bad.  I'm not sure how to do it
though.  At the core Python sprint in September there was some talk
about how CPython developers might get funding.  Maybe that could
help deal with the backlog of reviews required.

> And, while you may not give a damn about anything other than Windows,
> macos and/or Linux - there are other platforms that would like a stable
> Python.

There is probably some truth in not caring about other platforms.
The problem from the reviewer perspective is the question of "what
is the potential downsides of this PR vs what are the benefits?".
The safest thing is to not approve the PR.  No core developer wants
to be the person who broke CPython.  You must admit, AIX is an
extremely niche platform at this point.  I bet if you picked 1000
software developers at random, it would be likely that zero of them
have ever used AIX.  So, it's not that we don't care at all about
AIX but that the cost/benefit equation makes accepting AIX specific
changes more difficult.

One specific suggestion I have about your PR is to try to make your
changes not AIX specific.  Or at least, make the AIX checking as
localized as possible.  So, as an example, in test_uuid you have:

_notAIX = not sys.platform.startswith("aix")

then later in the module you check that flag.  While that is the
most direct approach to fixing the issue and making the test pass,
it is not good for the long term maintainability of the code.  You
end up with boolean flags like _notAIX spread about the logic.  Over
time, code like that becomes a nightmare to maintain.

Instead, I would suggest test_uuid is making platform specific
assumptions that are not true on AIX and possibly other platforms.
So, do something like:


_IS_AIX = sys.platform.startswith("aix")

_HAVE_MACADDR = (os.name == 'posix' and not _IS_AIX)

@unittest.skipUnless(_HAVE_MACADDR, 'requires Posix with macaddr')
def test_arp_getnode(self):
...

The _HAVE_MACADDR test is relatively simple and clear, does this
platform have this capability.  Later in the code, a check for
_HAVE_MACADDR is also quite clear.  If someone comes along with
another platform that doesn't support macaddr, they only have to
change one line of code.

This kind of capability checking is similar to what happened with
web browsers.  In that case, people discovered that checking the
User Agent header was a bad idea.  Instead, you should probe for
specific functionality and not assume based on browser IDs.  For the
macaddr case, is there some way to you probe the arp command to see
if supports macaddr?  That way your test doesn't have to include any
AIX specific check at all.  Further, it would have some hope of
working on platforms other than AIX that also don't support macaddr
but are POSIX and have 'arp'.  The code could be something like:

_HAVE_MACADDR = False
if os.name == 'posix':
if :
_HAVE_MACADDR = True

Hope that is helpful.

  Neil
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dear core-devs

[Michael Felt]

On 10/2/2018 11:34 PM, Terry Reedy wrote:
> On 10/2/2018 12:41 PM, Simon Cross wrote:
>> Are there any core devs that Michael or Erik could collaborate with?
>> Rather than rely on adhoc patch review from random core developers.
>
> You two might collaborate with each other to the extent of reviewing
> some of each other's PRs. 
Might be difficult. We both, or at least I, claim ignorance of the
others platform. I still have a lot of PEP to learn, and my idea of a
bug-fix (for Python2) was seen by core-dev as a feature change. I would
not feel comfortable trying to mentor someone in things PEP, etc..
> That still leaves the issue of merging.
How much confidence is there in all the "CI" tests? Does that not offer
sufficient confidence for a core-dev to press merge.
How about "master" continuing to be what it is, but insert a new
"pre-master" branch that the buildbots actually test on (e.g., what is
now the 3.X) and have a 3.8 buildbot - for what is now the "master".

PR would still be done based on master, but an "initial" merge would be
via the pre-master aka 3.X buildbot tests.

How "friendly" git is - that it not become such a workload to keep it
clean - I cannot say. Still learning to use git. Better, but still do
not want to assume it would be easy.

My hope is that it would make it easier to consider a "merge" step that
gets all the buildbots involved for even broader CI tests.

>
>> Michael and Eric: Question -- are you interested in becoming core
>> developers at least for the purposes of maintaining these platforms in
>> future?
>
> Since adhoc is not working to get merges, I had this same suggestion.
> Michael and Erik, I presume you have gotten some guidelines on what
> modifications to C code might be accepted, and what concerns people have.
imho: guidelines - paraphrased - as little as possible :)

I have many assumptions, and one of those is that my assumptions are
probably incorrect.
Goal: have AIX recognized as a Stable platform, even if not in the
highest supported category.
And that implies, support as far as I am able, to keep it "Stable".
>
> I think for tests, a separate test_aix.py might be a good idea for
> aix-only tests
Unclear to me how this would work. Too young in Python I guess (or just
a very old dog), but what test would be needed for AIX, or any other
platform, that would not need to be tested in some fashion for the
'other' platforms. At a hunch, where there are many platform.system()
dependencies expected (e.g., test_posix, maybe doing something in the
class definition (is there a "Root" Object/Class that all inherit from.
Maybe a (read-only) "root" attribute (or is property better?) could be
the value of platform.system(), and iirc, might be used by as @property
in unittest. (so, if not in "root" class, then in something like
unittest/__init__.py.

I hope to be "close" in "Python thinking" - enough that someone who
actually knows how the pieces fit together could come with a better, and
more appropriate guideline/implementation.

> , while modification of other tests might be limited to adding skips. 
> The idea would be to make it easy to remove aix stuff in the future if
> it again became unsupported.
IMHO: IBM and AIX do not mention it, but for openstack cloudmanagement
(very specifically cloud-init) AIX needs a recognized stable Python
implementation. I am "surprised" in the level of communication of IBM
with Python community.

Personally, I do not see AIX as a specialized platform. Feels more like
the "last-standing" fully supported (commercial OEM) 'POSIX-UNIX'. Of
course my focus is narrow - so maybe there is a lot of support for
commercial platforms such as HPUX, Solaris, and other mainstream UNIXes.
Feel free to correct me!!
> Ditto for other specialized platforms.
>
>
>
>

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dear core-devs

[Terry Reedy]

On 10/2/2018 12:41 PM, Simon Cross wrote:

Are there any core devs that Michael or Erik could collaborate with?
Rather than rely on adhoc patch review from random core developers.


You two might collaborate with each other to the extent of reviewing 
some of each other's PRs.  That still leaves the issue of merging.



Michael and Eric: Question -- are you interested in becoming core
developers at least for the purposes of maintaining these platforms in
future?


Since adhoc is not working to get merges, I had this same suggestion. 
Michael and Erik, I presume you have gotten some guidelines on what 
modifications to C code might be accepted, and what concerns people have.


I think for tests, a separate test_aix.py might be a good idea for 
aix-only tests, while modification of other tests might be limited to 
adding skips.  The idea would be to make it easy to remove aix stuff in 
the future if it again became unsupported.  Ditto for other specialized 
platforms.





--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dear core-devs

[Michael Felt]

On 10/2/2018 4:45 PM, Erik Bray wrote:
> Michael, if there are any PRs you want to point me to that I might be
> able to help review please do.
A little trick I learned:
https://github.com/python/cpython/pulls?q=is%3Aopen+is%3Apr+author%3Aaixtools+sort%3Aupdated-desc
lists them all.

What "flipped my switch" yesterday was discovering a PR that I was
gifted (by an ex? core-dev) and put in the system back in January is now
broken by a patch merged about two weeks ago. Worse, pieces of
test_ctypes(bitfields) that previously worked when using __xlc__ seem to
be broken. Which highlighted the "time pressure" of getting tests to
pass so that regressions can be seen.

If you let me know what info you would need (I gave lots of debug info
two years ago to get that initial fix).

And, I guess the other "larger" change re: test_distutils. Also, some
issues specific to xlc being different from gcc.

Those two do not show on the gccfarm buildbot.

Many thanks for the offer! I'll try to not take more than the hand offered!
>   I don't know anything about AIX either
> and am not a core dev so I can't have a final say.  But I've been
> hacking on CPython for a long time anyways, and might be able to help
> at least with some initial review.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dear core-devs

[Michael Felt]

I am willing to assist as best I can with AIX - I seem to have the core
requirements re: time available: (i.e., over-comitted at work, but
'work' evenings and weekends on OSS :p)


On 10/2/2018 6:41 PM, Simon Cross wrote:
> Are there any core devs that Michael or Erik could collaborate with?
> Rather than rely on adhoc patch review from random core developers.
>
> Michael and Eric: Question -- are you interested in becoming core
> developers at least for the purposes of maintaining these platforms in
> future?
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/aixtools%40felt.demon.nl

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dear core-devs

[Simon Cross]

Are there any core devs that Michael or Erik could collaborate with?
Rather than rely on adhoc patch review from random core developers.

Michael and Eric: Question -- are you interested in becoming core
developers at least for the purposes of maintaining these platforms in
future?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dear core-devs

[Erik Bray]

On Tue, Oct 2, 2018 at 3:53 AM Tres Seaver  wrote:
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 10/01/2018 06:41 PM, Michael Felt wrote:
>
> > And, while you may not give a damn about anything other than Windows,
> > macos and/or Linux - there are other platforms that would like a
> > stable Python.
>
> Michael,
>
> I can understand the frustration you feel:  you have been putting effort
> into a labor of love geting Python support on AIX (back?) into shape, and
> feel that your efforts are unappreciated, or worse, that they will be waste
> d.
>
> The key thing to realize about the various core developers (and the
> broader Python and open source communities) is that their attention is a
> heavily over-committed resource:  it isn't that folks here aren't
> benevolent toward your efforts, but rather that each of them (us?) makes
> decisions every day juggling which projects / tasks to give the minutes /
> hours we have available.  In the common case, the "triage" involves
> scrathing an itch:  this bug affects me / my work, that feature would
> make my life / my employment simpler, etc.  Even where there are minutes
> available, the "is reviewing this feasible for me?" question kicks in.
>
> Because AIX is relatively narrow in the scope of folks it impacts, the
> average, overcommitted developer is likely to see a bug report, or even a
> pull request, which makes stuff build on AIX and say, "Hmm, I don't know
> enough to evalute that one, I'll leave it to folks who do know (and by
> implication, who have some skin in the game)."  Even for more
> consumer-focused platforms, it has historically been harder to get
> attention for bugs / patches which affect only a single platform (Windows
> file locking semantics, or the Mac installer, etc.)
>
> One key way to get past that hurdle is to slice the size of each "thing"
> down as fine as possible:  e.g., a pull request adding a single "#ifdef
> AIX" block to one file.  Anything more than a screenful of diff is likely
> to trigger the "let someone else review it" pattern, whereas being able
> to scan the patch at a glance lets even a non-itchy reviewer decide,
> "well, at least it can't hurt anything, give it a shot."
>
> Once you've gotten a number of those small patches merged, you will find
> that you've built a relationship with the folks who have been reviewing
> them, and that they are more likely to pass them, and to review larger
> ones, at least in part because *you* will have learned more about what is
> needed in terms of code style, documentation, test coverage, etc., and
> *they* will have learned to trust your judgement.
>
> I'm sorry it isn't easier,

I have thought of writing an almost verbatim post w.r.t. my efforts to
get Cygwin re-supported (which was never officially un-supported
either).  Victor asked me to set up a buildbot for Cygwin as a
prerequisite to much else, which I have done [1].  But it has been
turning out broken build after broken build and is all but useless
since, even at the time of setting it up, I pointed out that there are
two major blocker issues [2] [3] that prevent an even
partially-working build.  Naoki Inada provided some review of the
first one a while ago, and while we had some (I think valid)
disagreement on how to proceed, I updated the issue description with a
checklist of issues he raised that need some clear direction on how
they should be resolved (since at least on some of them we disagreed).
I'd be happy to do pretty much whatever so long as I knew it was
meeting a core dev's requirements while also meeting my own
requirements.

Obviously I'm sympathetic to the limited time and attention of core
devs--I am a maintainer on several projects myself and I know how it
goes, and I have tried not to make too much of a fuss about it.  But
there's a chicken-egg problem in the specific area of platform
support, which feels a little different from just "I need my pet bug
fixed", for someone who is not already a core developer: In order to
make any progress on the issue we need at least one core dev who is
interested in the same platform.  But if we're the only ones willing
to do the work who know or care anything about that platform, how are
we supposed to progress in the first place?

I, like Michael Felt, have a number of fixes waiting in the wings but
can't really progress until a little bit of bare minimum ground work
is at least done to get us up and running.

Michael, if there are any PRs you want to point me to that I might be
able to help review please do.  I don't know anything about AIX either
and am not a core dev so I can't have a final say.  But I've been
hacking on CPython for a long time anyways, and might be able to help
at least with some initial review.


[1] https://buildbot.python.org/all/#/builders/164
[2] https://github.com/python/cpython/pull/4348
[3] https://github.com/python/cpython/pull/8712
___
Python-Dev mailing list
Python-Dev@python.o

Re: [Python-Dev] dear core-devs

[Tres Seaver]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 10/01/2018 06:41 PM, Michael Felt wrote:

> And, while you may not give a damn about anything other than Windows, 
> macos and/or Linux - there are other platforms that would like a
> stable Python.

Michael,

I can understand the frustration you feel:  you have been putting effort
into a labor of love geting Python support on AIX (back?) into shape, and
feel that your efforts are unappreciated, or worse, that they will be waste
d.

The key thing to realize about the various core developers (and the
broader Python and open source communities) is that their attention is a
heavily over-committed resource:  it isn't that folks here aren't
benevolent toward your efforts, but rather that each of them (us?) makes
decisions every day juggling which projects / tasks to give the minutes /
hours we have available.  In the common case, the "triage" involves
scrathing an itch:  this bug affects me / my work, that feature would
make my life / my employment simpler, etc.  Even where there are minutes
available, the "is reviewing this feasible for me?" question kicks in.

Because AIX is relatively narrow in the scope of folks it impacts, the
average, overcommitted developer is likely to see a bug report, or even a
pull request, which makes stuff build on AIX and say, "Hmm, I don't know
enough to evalute that one, I'll leave it to folks who do know (and by
implication, who have some skin in the game)."  Even for more
consumer-focused platforms, it has historically been harder to get
attention for bugs / patches which affect only a single platform (Windows
file locking semantics, or the Mac installer, etc.)

One key way to get past that hurdle is to slice the size of each "thing"
down as fine as possible:  e.g., a pull request adding a single "#ifdef
AIX" block to one file.  Anything more than a screenful of diff is likely
to trigger the "let someone else review it" pattern, whereas being able
to scan the patch at a glance lets even a non-itchy reviewer decide,
"well, at least it can't hurt anything, give it a shot."

Once you've gotten a number of those small patches merged, you will find
that you've built a relationship with the folks who have been reviewing
them, and that they are more likely to pass them, and to review larger
ones, at least in part because *you* will have learned more about what is
needed in terms of code style, documentation, test coverage, etc., and
*they* will have learned to trust your judgement.

I'm sorry it isn't easier,


Tres.
- -- 
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   "Excellence by Design"http://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iEYEARECAAYFAluyzyYACgkQFXKVXuSL+CMAHQCfXxFKpKyBXQg3dBSPY8MYOwh1
djsAnitN3SjTt+xwDdnT2NvTs965wEjR
=Bl8Z
-END PGP SIGNATURE-

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] dear core-devs

[Steven D'Aprano]

Hi Michael, and welcome,

On Tue, Oct 02, 2018 at 12:41:56AM +0200, Michael Felt wrote:

[...]
> FYI: since the end of July I have dedicated 16 to 24 hours of my free
> time to get this done. All for Python; all in my freetime. My employer
> does not care - I do, or did.

[...]
> I am, to put it lightly, extremely frustrated, at this point.

Okay, but what are you frustrated *about*?


> I am sorry, for myself obviously - but also for Python. Obviously, I am
> doing it all wrong - as I see lots of other issues being picked up
> immediately.

Doing what wrong?


> All volunteers need some level of recognition to keep moving on.
> 
> And, while you may not give a damn about anything other than Windows,
> macos and/or Linux - there are other platforms that would like a stable
> Python.

???

Have the core devs decided to drop support for AIX? Have your patches 
been rejected or something? Please explain what the actual problem is.



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] dear core-devs

[Michael Felt]

Dear core-devs,

I have some bad characteristics.

I can be extremely enthusiastic - and write too much. I have been trying
to not write - anything - worried that my enthusiasm is not matched by
yours, or worse was a reason to ignore my work to get AIX passing all tests.

FYI: since the end of July I have dedicated 16 to 24 hours of my free
time to get this done. All for Python; all in my freetime. My employer
does not care - I do, or did.

I am grateful to Martin Panter - who helped me graciously when I knew
absolutely nothing when I first got started; Victor was kind enough to
answer some emails and help me along but also clear that he has zero
interest in AIX and my questions were taking too much of his time.
Regretfully for me.

Again - Victor - thank you for your time. I appreciated the assistance
and feedback.

(Others have helped from time to time, my apologies for not naming you
specifically.)

I am, to put it lightly, extremely frustrated, at this point.

I am sorry, for myself obviously - but also for Python. Obviously, I am
doing it all wrong - as I see lots of other issues being picked up
immediately.

All volunteers need some level of recognition to keep moving on.

And, while you may not give a damn about anything other than Windows,
macos and/or Linux - there are other platforms that would like a stable
Python.

Sincerely,

Michael




signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com