[Python-Dev] CFLAGS_NODIST and -qalias=noansi
Dear all, After several months of absence - my first manual build surprised me by the addition of the -qalias=noansi. Before I open an issue - maybe it is not that important - I am trying to find what brought it in. It looks to be a change in behavior in configure(.ac) - starting with Python3.9. Historical builds - and make.out: py38-3.8.7/.buildaix/make.out:0 py39-3.9.0/.buildaix/make.out:239 py39-3.9.1.0/.buildaix/make.out:254 py39-3.9.1/.buildaix/make.out:148 py3a-3.10/.buildaix/make.out:244 Prior to Python3.9 xlc_r -c -DNDEBUG -O -I/opt/include -O2 -qmaxmem=-1 -qarch=pwr5 -q64 -I/opt/include -O2 -qmaxmem=-1 -qarch=pwr5 -q64 -I../src/py38-3.8.7/Include/internal i.e., the flags became: xlc_r -c -DNDEBUG -O -I/opt/include -O2 -qmaxmem=-1 -qarch=pwr5 -q64 -I/opt/include -O2 -qmaxmem=-1 -qarch=pwr5 -q64 -qalias=noansi -qmaxmem=-1 -I../src/py38-3.8.7/Include/internal Note the flags have been repeating themselves for a long time, but now - after the repeated flags `-qalias=noansi -qmaxmem=-1` is being added. my gut repsonse is that I do not like seeing 'noansi' being passed as a flag to the compiler and I am trying to understand why - also how. Fyi: using `git log` I have tried to get any clue re: CFLAGS_NODIST and/or -qalias=noansi - but I do not seem to be skilled enough to find that information. Help is appreciated - and, thereafter, as needed, I'll open an issue. Regards, M Felt aka aixtools OpenPGP_0x722BFDB61F396FC2.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LNGKVRQWYQUXGHAP57UYSTC4RPCXVFSP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Python.org downloads page and javascript
Hi, Not clear on where to report this - so I hope someone else sees the same and can notify whoever needs to be notified. The main URL for downloads: https://www.python.org/downloads/ Opens with message : *Notice:* While Javascript is not essential for this website, your interaction with the content will be limited. Please turn Javascript on for the full experience. However, javascript is enabled in my browser (FF latest). Also tried with Chrome - same (messy) layout. The main page, and selected pages (e.g., https://www.python.org/doc/) work as expected. Hope this helps!, Michael OpenPGP_0x722BFDB61F396FC2.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6TFD7LVJOA33YFXSI4RGLBWRC7STFYCV/ Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-Dev] before I open an issue re: posix.stat and/or os.stat
Shortening the original mail to something shorter. The reason I am starting here, in -dev, rather than as an issue directly, is because I would like some direction/recommendation from concerned individuals before I take a "outsider" approach. Too often I have learned that I guessed wrong how the Python community "sees the world". > On 2/21/2019 11:26 AM, Michael wrote: > It seems so - however, Is there something such as PyUnsignedLong and is > that large enough for a "long long"? and if it exists, would that make > the value positive (for the first test). > > posix.major and os.major will need to mask away the MSB and > posix.makedev and os.makedev will need to add it back. > > OR - do I need to make the PyStat values "the same" in both 32-bit and > 64-bit? I think I already have the answer to this - follow the platform (as far as size goes). On 32-bit max(1) == 256 (2^^8) and on 64-bit 65536 (2^^16). However, I am still puzzeled re: the idiosyncratic MSB addition - that the value for st.st_dev on AIX 64-bit has always (as far back as I could look at least) had the MSB set so that, as a signed number st.st_dev is always negative. And MSB is not part of the major device number. It's just there. So, it has been a few days since I studied this (waiting for reactions) - but is the return suppossed to always be a value returned by the posixmodule.c (if so, can strip the MSB bit and make everything positive - even for st.st_dev (but that would break under a direct comparision with a C program that does not strip the MSB). Currently, the macros on AIX for 64-bit are broken, so I cannot guess how they will decide to implement major() and mkdevice() as far as striping (and adding) the MSB when "constructing" the st_dev number from two "regular" major and minor numbers. Parked for now, Best wishes, Michael > > Puzzled on what you think is the correct approach. > > Michael ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python 3.8 alpha and AIX buildbot “support” moving forward.
Congratulations on the official begin of the alpha phase of Python3-3.8. I hope there will be time to consider three of my PRs so that this phase has at least one of the AIX buildbots (not mine I fear) is passing all the tests and can finally serve it’s real purpose and signal when a change toggles it’s status from PASS to FAIL. Next week I hope to have some time to dig deeper and try to establish why my bot fails additional tests (in the multiprocessing module(s)) as well why AIX fails test_bdb when utf8 support is (additionally) installed but passes when utf8 support is not installed. While I am also concerned about AIX status I also hope that my inspection is helping to improve Python. Sincerely, Michael Sent from my iPhone___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Custom AIX build last week...
I did a bit of digging and it seems this is related to issue35059, and something about "inline" processing. Besides learning how to clone a PR manually (which I will need to do) - what are you hoping to find? As my bot is not using gcc ( but xlc) I could look at manually compiling a single file and add some extra flags (-qsource or -qlist) which generates a .lst file and shows what the compiler did (macro expansion, etc..) If you would like me to attempt this - just let me know. Although it may take 24 hours from that request before I can fulfill it. Regards, Michael > On 10/30/2018 8:08 AM, Michael Felt wrote: > I noticed that there was a "custom" build queued for my AIX build-bot last > week. (https://buildbot.python.org/all/#/builders/159/builds/1). > > It failed, but I hope that was due to the issue with install.sh. > > If it could be run again - and if it fails again, please let me know what the > test was, and I'll look at it on my server manually. > > If it succeeds - I would also appreciate a heads-up! > > Thanks, > > Michael > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/aixtools%40felt.demon.nl ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Custom AIX build last week...
I noticed that there was a "custom" build queued for my AIX build-bot last week. (https://buildbot.python.org/all/#/builders/159/builds/1). It failed, but I hope that was due to the issue with install.sh. If it could be run again - and if it fails again, please let me know what the test was, and I'll look at it on my server manually. If it succeeds - I would also appreciate a heads-up! Thanks, Michael ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Rename Include/internals/ to Include/pycore/
My short comment: +1 My longer comment: for someone who is not trying to be caught up in "internals" I find it extremely difficult to work with the "default" approach described below - trying to mentally understand, and remember what those macros mean/do as I got "bug-hunting". Ultimately, have a clear-cut division between "public" and "internal" make it much much easier for "the rest of us" to know our boundaries and thereby develop code that is based on the "published" aka public API and not a (guess what I found) internal trick. Isn´t that why there is a "public" API to begin with. Where, or how the separation is provided is not the key issue. But being committed to it is, and this sounds like a step towards commitment and clarity. > On 10/28/2018 6:20 PM, Victor Stinner wrote: > IMHO the current design of header files is done backward: by default, > everything is public. To exclude an API from core or stable, "#ifdef > Py_BUILD_CORE" and "#ifndef Py_LIMITED_API" are used. This design > caused issues in the past: functions, variables or something else > exposed whereas they were supposed to be "private". > > I propose a practical solution for that: Include/*.h files would only > be be public API. The "core" API would live in a new subdirectory: > Include/pycore/*.h. Moreover, files of this subdirectory would have > the prefix "pycore_". For example, Include/objimpl.h would have a > twin: Include/pycore/pycore_objimpl.h which extend the public API with > "core" APIs. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] AIX build-bots broken for 11 days
a) I wish I seen this earlier, but I have been away from OSS for the last weeks as my RL job has priority. b) to answer a very different question I needed to look at the current (Always FAIL) status of the the two AIX buildbots - and I saw that 11 days ago they both started failing with the following messages: renaming build/scripts-3.8/2to3 to build/scripts-3.8/2to3-3.8 ./install-sh -c -m 644 ./Tools/gdb/libpython.py python-gdb.py ./install-sh: not found make: 1254-004 The error code from the last command is 1. renaming build/scripts-3.8/2to3 to build/scripts-3.8/2to3-3.8 ./install-sh -c -m 644 ./Tools/gdb/libpython.py python-gdb.py make: execvp: ./install-sh: The file access permissions do not allow the specified action. Makefile:662: recipe for target 'python-gdb.py' failed I am quite surprised, because, afaik, gdb is not used on either of the AIX bots. In any case These two runs failed in the "normal" way https://buildbot.python.org/all/#/builders/10/builds/1644/steps/4/logs/stdio https://buildbot.python.org/all/#/builders/161/builds/327/steps/4/logs/stdio And these fail during the build. https://buildbot.python.org/all/#/builders/10/builds/1645/steps/2/logs/stdio https://buildbot.python.org/all/#/builders/161/builds/328/steps/2/logs/stdio Help appreciated. Michael ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] "Deprecation" of os.system in favor of subprocess?
Thanks for asking a question that triggered an enlightening discussion! On 10/25/2018 5:13 PM, Stephane Wirtel wrote: > Hi all, > > After your feedback, I have my answer. > > I understand the your points of view and I don't want to change any part > of code for os.system and subprocess, I don't want to deprecate > os.system in favor of subprocess. I just wanted to know your opinion > about this point. > > +1 to use os.system when we only need a call without portability, just a > quick code. > > +1 for subprocess for the portability. > > Thank you so much for your patience/time and your explanations. > > Stéphane ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] AIX to stable, what does that take?
On 10/4/2018 10:30 AM, INADA Naoki wrote: > Hello, > > First of all, congratulations on passing all test on AIX. > >> My assumption is that it needs to (at least) pass all tests - and that >> is why I keep asking for attention. All the PRs to fix individual tests >> mean less if they are not merged, for whatever reason. >> >> However, maybe there is another way, or even something additional >> needed. Maybe something I cannot provide and then I can adjust my >> expectations and goals. > As a one of core developer, I don't know anything about AIX. > If my change breaks AIX build, I can't investigate what's happened. > > So I think we need following in devguide: > > * Brief description about AIX, from developer's point of view. This I might be able to do. Bullet form: * Committed to POSIX standard (valid when release came out, so AIX 5.3 confirms to a different standard than AIX 7.2) * While Linux affinity is recognized - GNU (or GNP - GNU not POSIX) integration is not guaranteed. - GNU rte is not provided under support. There is a so-called Toolbox, GNU an other OSS utilities supplied by many packaged as RPMs. Unfortunately, different RPM providers (Michael Perlz, BULL Freeware, IBM, and others) have different numbering (the part after the package version, e.g., python-2.7.10-XXX so they do not mix well). Headache for both admins and developers trying to develop in a GNU-like environment. * As a consultant, fedup with what is called the "RPM hell" by many AIX admins - I do not use any RPMs. I build everything myself, using xlc (gcc introduces the need for a GNU RTE, e.g., glibc). I package using installp (the "native") AIX package manager, and strive to make the packages independent (these days). When there are dependencies I try to build them as static libraries so that they do not become an additional install dependency. * finally, a bit deeper: while the AIX linker loader supports svr4 shared libraries (it is the data, not the file name) it also supports having multiple shared libraries in a classic archive. So, rather that .../lib/libxxx.so and .../lib64/libxxx.so AIX prefers .../lib/libxxx.a with two so-called members, with same or different names. The one found is not it's name, but the symbol name and size of the ABI (32-bit or 64-bit) * Hope that is enough of the key issues for now. ** In general, GNU autotools (autoreconf -f -v) works well, as does configure.ac et al. for createing OSS Makefiles. > * How to run AIX on (VirtualBox, AWS EC2, Azure, GCP) easily. Easily! ? :) - well, on a POWER server it was easy enough for me to follow the step by step instructions for creating a buildbot. If I had a POWER server with more resources I would examine using the AIX internal WPAR - however, a POWER server configured to use PowerVC uses "EC2" aka cloud-init for creating a virtual machine. With that environment it should be "easy" to provide additional instructions to cloud-init-ec2. Or, I provide you a login on my personal server that I run the buildbot on. etc. etc. - Where there is a will, there is a way. > * How to set up a development environment for Python. Again, follow the instructions for setting up a buildbot. > * How to build Python. git clone ... autoreconf -v -f (as needed) ./configure --with-pydebug #gcc compiler ./configure --with-pydebug --without-computed-gotos # xlc compiler make make test > * How to debug C code. I learned, 40 years ago, using adb (a debugger) - I do a lot of single-stepping. gdb is not the default debugger. If I were a developer I would probably dig into the AIX debuggers (there are at least two, kdb (kernel debugger, which I do use occaisionally for performance issues) and another I try to avoid. I add fprintf statements and am looking at learning how to use probevue. In short, you probably have many much better ideas on how to debug C than I do :) > > And even though there is a developer guide, it will take more long time > than fixing issues on AIX, compared Linux, macOS, and Windows. > > But without this guide, it feels almost impossible to maintain AIX build to > me. IMHO: The AIX build is stable, but this is unrecognized because it does have differences that cause tests to fail. I can think of one test that PASSes, but should fail. And another test that passes, but should have failed (in test_uuid) I have submitted a PR. I tried to fix "all" in one PR, which confused people - so redid it as two (got _uuid working in Python 3.7 ! yes!!) but the "original" to fix uuid.py and test_uuid.py is still "awaiting change review". My gut feeling to maintaining AIX is: a) all test pass so a potential regression is flagged; b) someone such as myself who knows the platform and can establish a "root cause" on why it is failing with AIX so that c) a developer becomes aware and can decide to ignore or adjust; d) alternatives - such as work around an implementation limitation (as I try to do, e.g., for test_time and the related _datetime.c) is yet another path. In other wor
Re: [Python-Dev] dear core-devs
On 10/4/2018 9:55 AM, Petr Viktorin wrote: > On 10/4/18 9:34 AM, Victor Stinner wrote: >> Hi, >> >> If IBM wants a better Python support, it would help a lot if IBM pays >> for this development. I agree. If IBM ... >> ... Antoine Pitrou has been paid in the past to enhance Python >> support in Solaris and it worked well. > FYI - as I now have access to the gccfarm, and in the spirit of more generalized "posix" additions I looked for an HPUX and a Solais system to build master on. make test never finished (one test was still hanging after over 20 minutes, and I had to go. Of the 419, 17 or 18 had failed. Roughly where AIX plus xlc was at last July without my PRs for tests. So, while it worked - money stopped and Solaris is in no better numerical shape (test wise) than AIX. > Michael explicitly said this is a personal effort. IBM or other big > money is not involved. IBM is my employer. As I am not a developer (merely a systems and management consultant) I do not face losing my job by working on OSS. I have been called off certain OSS projects because IBM was providing money and/or developers. This is one of the reasons (being called off elsewhere) that I have been hesitant to be more involved than I was in 2015-2017. So, let me be explicit - I can only speak for myself. And as long as no manager says "No, cannot work on that" I have given a commitment to work on this. "Some things cannot be bought" - such as un-biased (I call it "maverick" rather than merely independent.) On the one hand IBM policy is to encourage independent thought. The core goal is to help customers succeed. But individual managers up and down the line occasionally have additional business needs, and then workers as myself apologize and take a step back - in a word - adjust. Short answer: my involvement is mine to give at no price. I am considered one of the worlds AIX experts on matters of integration, performance and security. So, I have just simple questions for you? Do you value my expertise? May I assist? > > Is paying the best way to get features into Python? Does becoming a > core dev mean you can now get paid for approving changes? Some of the > implications are quite disturbing :( > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] AIX to stable, what does that take?
In the buildbots AIX is marked as "unstable"? What is needed to get it marked as a "stable" platform - that being one of my short-term goals. My assumption is that it needs to (at least) pass all tests - and that is why I keep asking for attention. All the PRs to fix individual tests mean less if they are not merged, for whatever reason. However, maybe there is another way, or even something additional needed. Maybe something I cannot provide and then I can adjust my expectations and goals. Regards, Michael ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] dear core-devs
On 10/3/2018 1:46 AM, Neil Schemenauer wrote: > On 2018-10-02, Michael Felt wrote: >> I am sorry, for myself obviously - but also for Python. Obviously, I am >> doing it all wrong - as I see lots of other issues being picked up >> immediately. > I'm not sure that's the case. There are a lot of PRs or bugs that > sit there without getting reviews. The problem is that few (or no) > core developers get paid to work on Python. So, the time they spend > is motivated by their specific "itch". Getting reviews on any PR is > difficult, even for core developers. In their case, they have to > option of forcing the issue, I guess. > > This is a problem we should try to deal with somehow. Turning off > valuable contributors like you is bad. I'm not sure how to do it > though. At the core Python sprint in September there was some talk > about how CPython developers might get funding. Maybe that could > help deal with the backlog of reviews required. > >> And, while you may not give a damn about anything other than Windows, >> macos and/or Linux - there are other platforms that would like a stable >> Python. > There is probably some truth in not caring about other platforms. > The problem from the reviewer perspective is the question of "what > is the potential downsides of this PR vs what are the benefits?". > The safest thing is to not approve the PR. No core developer wants > to be the person who broke CPython. You must admit, AIX is an > extremely niche platform at this point. I bet if you picked 1000 > software developers at random, it would be likely that zero of them > have ever used AIX. So, it's not that we don't care at all about > AIX but that the cost/benefit equation makes accepting AIX specific > changes more difficult. Nods. However - this is a chicken/egg issue (imho). AIX is seen a weak platform because noone has ever tackled these. When I started on this I had never expected to have found a resolution to them all. Platforms have differences and when the tests miss that difference that the tests give a false result. e.g., one accepted PR was because AIX libc printf() output for printf(NULL) is "" while other platforms output "(null)". > > One specific suggestion I have about your PR is to try to make your > changes not AIX specific. Or at least, make the AIX checking as > localized as possible. So, as an example, in test_uuid you have: > > _notAIX = not sys.platform.startswith("aix") a) I thought/hoped this was better practice and performance - calling sys.platform.startswith("aix")only once, rather than X times. b) more maintainable (e.g., change to not platform.system() c) iirc - this got changed to AIX = , and throughout the test is "if not AIX"... > > then later in the module you check that flag. While that is the > most direct approach to fixing the issue and making the test pass, > it is not good for the long term maintainability of the code. You > end up with boolean flags like _notAIX spread about the logic. Over > time, code like that becomes a nightmare to maintain. > > Instead, I would suggest test_uuid is making platform specific > assumptions that are not true on AIX and possibly other platforms. > So, do something like: > > > _IS_AIX = sys.platform.startswith("aix") better name. > > _HAVE_MACADDR = (os.name == 'posix' and not _IS_AIX) AIX has MACADDR, but formatted with '.' rather than ':' and uses a single hex-digit when value between dots is < 16 (decimal) > > @unittest.skipUnless(_HAVE_MACADDR, 'requires Posix with macaddr') > def test_arp_getnode(self): > ... > > The _HAVE_MACADDR test is relatively simple and clear, does this > platform have this capability. Later in the code, a check for > _HAVE_MACADDR is also quite clear. If someone comes along with > another platform that doesn't support macaddr, they only have to > change one line of code. > > This kind of capability checking is similar to what happened with > web browsers. In that case, people discovered that checking the > User Agent header was a bad idea. Instead, you should probe for > specific functionality and not assume based on browser IDs. For the > macaddr case, is there some way to you probe the arp command to see > if supports macaddr? I suppose if someone had written the original test with "check program to see if ..." it would have worked already. I am trying to get current tests to work with minimal changes. I am certainly not "blaming" anyone for not knowing this unique behavior of this platform. Before debugging this I did not know of the difference eithe
Re: [Python-Dev] dear core-devs
On 10/3/2018 2:48 AM, Terry Reedy wrote: > On 10/2/2018 7:16 PM, Michael Felt wrote: >> >> >> On 10/2/2018 11:34 PM, Terry Reedy wrote: >>> On 10/2/2018 12:41 PM, Simon Cross wrote: >>>> Are there any core devs that Michael or Erik could collaborate with? >>>> Rather than rely on adhoc patch review from random core developers. >>> >>> You two might collaborate with each other to the extent of reviewing >>> some of each other's PRs. > >> Might be difficult. We both, or at least I, claim ignorance of the >> others platform. > > Partial reviews, short of accept/change are better than no review and > can make a merge decision easier for a core dev. You should each be > or become familiar with PEP 7 and somewhat familiar with local C > idioms. Do names follow local standards. Do C-API calls make sense. Sounds simple enough. The tricky part is "the details". > > >> I still have a lot of PEP to learn, and my idea of a > >> bug-fix (for Python2) was seen by core-dev as a feature change. > > Failures of current tests would seem to me to be bugs. However, some > bug fixes require a feature change. It is an awkward situation. We > are increasingly reluctant to patch 2.7. Some are quite simple to fix, even if hard to find: such as: "elif cmd is None:" -> "elif notcmd orcmd is None:" Some are not bugs at all - very hard to find! Instead, "textual" differences because a library is overly optimized - the expected exception occurs - but no error message. Linking with a less optimized (libssl.a and libcrypto.a) resolved many reported test "failures". Nearly three years ago I was keen to see things in Python2(.7), but not so much now. I also feel the time is to push hard towards current Python3 versions. > >>> That still leaves the issue of merging. >> How much confidence is there in all the "CI" tests? Does that not offer >> sufficient confidence for a core-dev to press merge. > > Code for new features or bugs that escaped the tests should have new > tests. AIX-specific code should (as in must ;-) be tested before > being submitted, since it will not be properly tested by CI. With CI > now covering Windows twice, Linux twice, and Mac, I believe it has > become rarer for buildbots to fail after CI passes. Victor would know. > > I believe that you are initially dealing with bugs that do not pass > current tests. I am dealing with tests that do not pass. The dilemma: what is wrong - the test, or what it is testing? Generally speaking, I cannot call Python3 (master) broken. So I look for a "root cause" in a test assumption that is wrong, and find a way to correct that. Sometimes, it is a bit of both - and those are very hard to resolve without feedback. See the discussion, elsewhere, regarding MACADDR. It has never been that platform Y does not have a MACADDR - rather, platform Y formats it differently than (all) other platforms. > >> How about "master" continuing to be what it is, but insert a new >> "pre-master" branch that the buildbots actually test on (e.g., what is >> now the 3.X) and have a 3.8 buildbot - for what is now the "master". >> >> PR would still be done based on master, but an "initial" merge would be >> via the pre-master aka 3.X buildbot tests. >> >> How "friendly" git is - that it not become such a workload to keep it >> clean - I cannot say. Still learning to use git. Better, but still do >> not want to assume it would be easy. > > Too complicated. > >> My hope is that it would make it easier to consider a "merge" step that >> gets all the buildbots involved for even broader CI tests. > > I considered the wider buildbot fleet to be post-merge CI ;-). > >>> I think for tests, a separate test_aix.py might be a good idea for >>> aix-only tests > > I may be wrong on this. > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/aixtools%40felt.demon.nl ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] dear core-devs
On 10/2/2018 11:34 PM, Terry Reedy wrote: > On 10/2/2018 12:41 PM, Simon Cross wrote: >> Are there any core devs that Michael or Erik could collaborate with? >> Rather than rely on adhoc patch review from random core developers. > > You two might collaborate with each other to the extent of reviewing > some of each other's PRs. Might be difficult. We both, or at least I, claim ignorance of the others platform. I still have a lot of PEP to learn, and my idea of a bug-fix (for Python2) was seen by core-dev as a feature change. I would not feel comfortable trying to mentor someone in things PEP, etc.. > That still leaves the issue of merging. How much confidence is there in all the "CI" tests? Does that not offer sufficient confidence for a core-dev to press merge. How about "master" continuing to be what it is, but insert a new "pre-master" branch that the buildbots actually test on (e.g., what is now the 3.X) and have a 3.8 buildbot - for what is now the "master". PR would still be done based on master, but an "initial" merge would be via the pre-master aka 3.X buildbot tests. How "friendly" git is - that it not become such a workload to keep it clean - I cannot say. Still learning to use git. Better, but still do not want to assume it would be easy. My hope is that it would make it easier to consider a "merge" step that gets all the buildbots involved for even broader CI tests. > >> Michael and Eric: Question -- are you interested in becoming core >> developers at least for the purposes of maintaining these platforms in >> future? > > Since adhoc is not working to get merges, I had this same suggestion. > Michael and Erik, I presume you have gotten some guidelines on what > modifications to C code might be accepted, and what concerns people have. imho: guidelines - paraphrased - as little as possible :) I have many assumptions, and one of those is that my assumptions are probably incorrect. Goal: have AIX recognized as a Stable platform, even if not in the highest supported category. And that implies, support as far as I am able, to keep it "Stable". > > I think for tests, a separate test_aix.py might be a good idea for > aix-only tests Unclear to me how this would work. Too young in Python I guess (or just a very old dog), but what test would be needed for AIX, or any other platform, that would not need to be tested in some fashion for the 'other' platforms. At a hunch, where there are many platform.system() dependencies expected (e.g., test_posix, maybe doing something in the class definition (is there a "Root" Object/Class that all inherit from. Maybe a (read-only) "root" attribute (or is property better?) could be the value of platform.system(), and iirc, might be used by as @property in unittest. (so, if not in "root" class, then in something like unittest/__init__.py. I hope to be "close" in "Python thinking" - enough that someone who actually knows how the pieces fit together could come with a better, and more appropriate guideline/implementation. > , while modification of other tests might be limited to adding skips. > The idea would be to make it easy to remove aix stuff in the future if > it again became unsupported. IMHO: IBM and AIX do not mention it, but for openstack cloudmanagement (very specifically cloud-init) AIX needs a recognized stable Python implementation. I am "surprised" in the level of communication of IBM with Python community. Personally, I do not see AIX as a specialized platform. Feels more like the "last-standing" fully supported (commercial OEM) 'POSIX-UNIX'. Of course my focus is narrow - so maybe there is a lot of support for commercial platforms such as HPUX, Solaris, and other mainstream UNIXes. Feel free to correct me!! > Ditto for other specialized platforms. > > > > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LDLAST variable in configure.ac
Yes, unintended. It was only supposed to be signed, but "Send Later" encrypts it. Unpacked version: On 10/2/2018 1:07 AM, Benjamin Peterson wrote: > On Mon, Oct 1, 2018, at 12:12, Michael Felt wrote: >> Hi all, >> >> Before I submit a patch to increase the default MAXDATA setting for AIX >> when in 32-bit mode - I want to know if I can put this LDFLAG setting in >> LDLAST, or if I should introduce a new AC_SUBST() variable (e.g., >> LDMAXDATA). > I think you should just put it in LDFLAGS. I was wanting to avoid that, as LDFLAGS is an environmental variable. At the surface, it appears Python is using PY_LDFLAGS (with CONFIGURE_LDFLAGS coming from LDFLAGS during the ./configure moment. A reason for a separate variable is that this particular option is only relevant for the python EXE, and not for shared libraries and "other things". IMHO, a reason for LDMAXDATA is because LDLAST is actually already too widely used: root@x066:[/data/prj/python/git/cpython-master]grep LDFLAGS *.in Makefile.pre.in:CONFIGURE_LDFLAGS= @LDFLAGS@ Makefile.pre.in:# Avoid assigning CFLAGS, LDFLAGS, etc. so users can use them on the Makefile.pre.in:# Both CPPFLAGS and LDFLAGS need to contain the shell's value for setup.py to Makefile.pre.in:PY_LDFLAGS= $(CONFIGURE_LDFLAGS) $(LDFLAGS) Makefile.pre.in:LDSHARED= @LDSHARED@ $(PY_LDFLAGS) Makefile.pre.in:BLDSHARED= @BLDSHARED@ $(PY_LDFLAGS) Makefile.pre.in:OPENSSL_LDFLAGS=@OPENSSL_LDFLAGS@ Makefile.pre.in: $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_GEN_FLAG)" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG)" LIBS="$(LIBS)" Makefile.pre.in: $(MAKE) @DEF_MAKE_RULE@ CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_USE_FLAG)" LDFLAGS="$(LDFLAGS)" Makefile.pre.in: $(LINKCC) $(PY_LDFLAGS) $(LINKFORSHARED) -o $@ Programs/python.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST) Makefile.pre.in: $(CC) -dynamiclib -Wl,-single_module $(PY_LDFLAGS) -undefined dynamic_lookup -Wl,-install_name,$(prefix)/lib/libpython$(LDVERSION).dylib -Wl,-compatibility_version,$(VERSION) -Wl,-current_version,$(VERSION) -o $@ $(LIBRARY_OBJS) $(SHLIBS) $(LIBC) $(LIBM) $(LDLAST); \ Makefile.pre.in: $(CC) -o $(LDLIBRARY) $(PY_LDFLAGS) -dynamiclib \ Makefile.pre.in: $(LINKCC) $(PY_LDFLAGS) $(LINKFORSHARED) -o $@ Programs/_testembed.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST) Makefile.pre.in: $(LINKCC) $(PY_LDFLAGS) -o $@ Programs/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST) Makefile.pre.in: $(CC) $(OPT) $(PY_LDFLAGS) $(PGENOBJS) $(LIBS) -o $(PGEN) The ONLY line that needs $LDMAXDATA is: Makefile.pre.in: $(LINKCC) $(PY_LDFLAGS) -o $@ Programs/_freeze_importlib.o $(LIBRARY_OBJS_OMIT_FROZEN) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST) $(LDMAXDATA) or set $(LDLAST) at the end rather than append $(LDMAXDATA) >> I have not looked yet, but I was thinking that MAYBE! LDLAST is intended >> as a last resort variable that can be modified in Makefile. > LDLAST looks vestigial from OSF/1 support and should probably be removed. On 10/2/2018 2:51 PM, Łukasz Langa wrote: >> On 2 Oct 2018, at 12:29, Michael Felt wrote: >> >> > Michael, this message looks encrypted on my end. For people without your > public key, it's impossible to read. This was probably unintentional on your > end. In either case I'd avoid encrypting messages that go to public mailing > lists. > > - Ł ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] dear core-devs
On 10/2/2018 4:45 PM, Erik Bray wrote: > Michael, if there are any PRs you want to point me to that I might be > able to help review please do. A little trick I learned: https://github.com/python/cpython/pulls?q=is%3Aopen+is%3Apr+author%3Aaixtools+sort%3Aupdated-desc lists them all. What "flipped my switch" yesterday was discovering a PR that I was gifted (by an ex? core-dev) and put in the system back in January is now broken by a patch merged about two weeks ago. Worse, pieces of test_ctypes(bitfields) that previously worked when using __xlc__ seem to be broken. Which highlighted the "time pressure" of getting tests to pass so that regressions can be seen. If you let me know what info you would need (I gave lots of debug info two years ago to get that initial fix). And, I guess the other "larger" change re: test_distutils. Also, some issues specific to xlc being different from gcc. Those two do not show on the gccfarm buildbot. Many thanks for the offer! I'll try to not take more than the hand offered! > I don't know anything about AIX either > and am not a core dev so I can't have a final say. But I've been > hacking on CPython for a long time anyways, and might be able to help > at least with some initial review. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] dear core-devs
I am willing to assist as best I can with AIX - I seem to have the core requirements re: time available: (i.e., over-comitted at work, but 'work' evenings and weekends on OSS :p) On 10/2/2018 6:41 PM, Simon Cross wrote: > Are there any core devs that Michael or Erik could collaborate with? > Rather than rely on adhoc patch review from random core developers. > > Michael and Eric: Question -- are you interested in becoming core > developers at least for the purposes of maintaining these platforms in > future? > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/aixtools%40felt.demon.nl ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] LDLAST variable in configure.ac
mime-attachment Description: application/pgp-encrypted encrypted.asc Description: Binary data ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] dear core-devs
Dear core-devs, I have some bad characteristics. I can be extremely enthusiastic - and write too much. I have been trying to not write - anything - worried that my enthusiasm is not matched by yours, or worse was a reason to ignore my work to get AIX passing all tests. FYI: since the end of July I have dedicated 16 to 24 hours of my free time to get this done. All for Python; all in my freetime. My employer does not care - I do, or did. I am grateful to Martin Panter - who helped me graciously when I knew absolutely nothing when I first got started; Victor was kind enough to answer some emails and help me along but also clear that he has zero interest in AIX and my questions were taking too much of his time. Regretfully for me. Again - Victor - thank you for your time. I appreciated the assistance and feedback. (Others have helped from time to time, my apologies for not naming you specifically.) I am, to put it lightly, extremely frustrated, at this point. I am sorry, for myself obviously - but also for Python. Obviously, I am doing it all wrong - as I see lots of other issues being picked up immediately. All volunteers need some level of recognition to keep moving on. And, while you may not give a damn about anything other than Windows, macos and/or Linux - there are other platforms that would like a stable Python. Sincerely, Michael signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Change in Python 3's "round" behavior
On 9/30/2018 2:17 PM, Steven D'Aprano wrote: > (It's also called Dutch Rounding.) Ah - as to why - and from school! (as so-called intuitive! rather desired!). A test score goes from 5.5 to 6.0 - which becomes passing. Oh, do I recall my children's frustrations when they had a X.4Y score - that became X.0. Tears! ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] LDLAST variable in configure.ac
Hi all, Before I submit a patch to increase the default MAXDATA setting for AIX when in 32-bit mode - I want to know if I can put this LDFLAG setting in LDLAST, or if I should introduce a new AC_SUBST() variable (e.g., LDMAXDATA). I have not looked yet, but I was thinking that MAYBE! LDLAST is intended as a last resort variable that can be modified in Makefile. Thanks! Michael signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [RELEASE] Python 3.7.1rc1 and 3.6.7rc1 now available for testing
Not critical - but I note a difference between Python3 3.6.7 and 3.7.1 - no support for the configure option --with-openssl. On AIX I was able to run both configure and "make install" without incident. I also ran the "make test" command. v3.7.1: 9 tests failed again: test_ctypes test_distutils test_httpservers test_importlib test_site test_socket test_time test_utf8_mode test_venv There are, for most of above, a PR for these waiting final review and merge. test_utf8_mode: I thought this was already merged. Will research. test_venv, test_site: new test failures (I am not familiar with). Will need more research. v3.6.1: 16 tests failed: test_asyncio test_ctypes test_distutils test_ftplib test_httplib test_httpservers test_importlib test_locale test_multiprocessing_fork test_multiprocessing_forkserver test_multiprocessing_spawn test_socket test_ssl test_strptime test_time test_tools FYI: again, there are PR for many of these, but, for now, I'll assume they will not be considered for backport. FYI only. On 9/27/2018 4:21 AM, Ned Deily wrote: > Assuming no > critical problems are found prior to 2018-10-06, no code changes are > planned between these release candidates and the final releases. These > release candidates are intended to give you the opportunity to test the > new security and bug fixes in 3.7.1 and 3.6.7. We strongly encourage you > to test your projects and report issues found to bugs.python.org as soon > as possible. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] make patchcheck and git path
When building out of tree there is no .git reference. If I understand the process it uses git to see what files have changed, and does further processing on those. As to workflow, that may be better, but other than the name, unknown to me. Sent from my iPhone > On 24 Aug 2018, at 19:37, Mariatta Wijaya wrote: > > I don't quite understand the problem you're facing with git and make > patchcheck? > > Also, perhaps this is more for core-workflow instead of python-dev. > > Mariatta > > ᐧ > >> On Fri, Aug 24, 2018 at 3:20 AM Michael wrote: >> I am trying to be a 'good scout' and run "make patchcheck" more >> regularly. However, I generally am not successful because I build and >> test in separate directories. >> >> There is access to git! just no ready reference in the build area. >> >> So, not calling it a bug - but if someone else also experiences this, >> and feels capable of makeing it more flexible - you will get thanks from >> me, in any case! >> >> Michael >> >> >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/mariatta.wijaya%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [issue17180] shutil copy* unsafe on POSIX - they preserve setuid/setgit bits
recent call last): File "/data/prj/python/git/python3-3.8/Lib/test/test_shutil.py", line 1491, in test_copy_remove_setuid self.assertEqual(oct(mode), oct(harmless_mode)) AssertionError: '0o4500' != '0o500' - 0o4500 ? - + 0o500 -- On 8/15/2018 1:01 PM, Michael Felt wrote: > Michael Felt added the comment: > > I am looking at this. pEpkey.asc Description: application/pgp-keys ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] AIX and python tests
Try again on this. a) Victor has said he will look, from time to time - after his vacation. b) our vacations do not overlap c) comment was also made privately, re: my starting a worker for buildbot, that there is not much sense in a bot if noone is working on the tests. I'll do my best, in the (limited) time I have to work on c) - but alone I cannot get anything done. So, Victor suggested I just ask for others to review for now - so I can have some semblance of moving forward - before my vacation starts (about when Victor gets back from his). In advance - many thanks. On 8/5/2018 10:59 PM, Michael wrote: > > As I have time, I'll dig into these. > > I have a couple of PR already 'out there', which I hope someone will > be looking at when/as he/she/they have time. My time will also be > intermittent. > > My next test - and I hope not too difficult - would be the test_utf8. > The test: > > FAIL: test_cmd_line (test.test_utf8_mode.UTF8ModeTests) fails - and I > am wondering if it is as simple as AIX default mode is ISO8559-1 and > the test looks to be comparing UTF8 with the locale_default. If that > is the case, obviously this test will never succeed - asis. Am I > understanding the test properly. If yes, then I'll see what I can come > up with for a patch to the test for AIX. If no, I'll need some hand > holding to help me understand the test A bigger challenge, and I think > a major issue with many of the test failures is test_ssl. Here I > already know I'll need so assistance. I am quite lost. I know AIX at > an expert level, but I do not know python (especially python > internals, macros, etc..) and after about 3 levels I am lost. I also > find it hard to get 'artifacts' from the tests to know what is > expected. Looking forward to assistance from various people - in > understanding the tests, and probably better python coding criticism. > Michael > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/aixtools%40felt.demon.nl pEpkey.asc Description: application/pgp-keys ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [python-committers] [RELEASED] Python 3.4.9 and Python 3.5.6 are now available
On 8/6/2018 11:38 AM, Charalampos Stratakis wrote: > A side note on your side note. Different distro's have different > standards, use/customer cases to address etc. In enterprise > distributions the usual scheme is that the version that you see is the > minimum one and many fixes coming from upstream or the redistributor > are incorporated on top of that version. Just check the package > changelogs. :) CVE's do get fixed and there is actually cooperation > with upstream on different levels in regards to those. And speaking > here as one of the people doing that for one of the enterprise > distros. > a) good to hear b) On AIX they stayed with ssh at version 6.0 for so long, that even with all the CVE et al included it was still extremely weak compared to 6.7 and later when they tightened the default ciphers. And yes, I fell over the change - but was glad, in the end, to rid of weak ssh clients. c) read package changelogs. The :) is because they are hard to read or non-existent. I do not mean to criticize any "enterprise" methods. My "enterprise" of choice is AIX and when it comes to OSS I dare say everyone else does a better job (which is why I got started with packaging in the first place - but only what I need and/or someone requests). However, I do find it very very hard to know what python 2.7.5 has or has not, that 2.7.15 now has. There are, iirc, quite a few important changes. The "hard" freeze seems to have come at roughly 2.7.8 or 2.7.9 (just a guess). Also, as I am trying to test on other platforms it gets a bit frustrating when the latest python3 I can find is a v3.4.X. Might be good project developers (in general, not meant as specific to python) to understand that version number changes are not followed - blindly - by enterprise patch management and being too quick with version number changes will make it more difficult for users to know what they have. p.s. I do not do this (packaging/patch management) for any "distro". In that sense I am "just a consumer" who "rolls his own" when/if needed. pEpkey.asc Description: application/pgp-keys ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] spwd and AIX
Sounds like i can skip this then. Thx. Sent from my iPhone > On 1 Aug 2018, at 17:52, Christian Heimes wrote: > >> On 2018-08-01 13:58, Michael wrote: >> a) I am looking at getting spwd integrated from AIX >> >> b) only the parameter sp_pwdp is my concern - as AIX really does not >> want to reveal the encrypted password. Rather, AIX will say '!' (meaning >> there is, or should be a shadow password, or '*' - no user password). >> Would this horribly break things if only '!' was returned, rather than >> the shadow password? >> >> It does not look terribly hard - but how do you deal with defaults such >> as 0 or -1 for the numeric values? > > Please note that I'm planning to deprecate and remove the spwd module. > The module has several limitation. Most importantly it bypasses the PAM > stack. Therefore it's not compatible with password policies or other > password storages like LDAP. > > Christian > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/aixtools%40felt.demon.nl ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] re performance
On 27/01/2017 22:24, MRAB wrote: I'm not bothered about it. It's quite a bit bigger than the re module, and, anyway, keeping it as a third-party module gives me more freedom to make updates, which are available for a range of Python versions. I tried packaging it (pip build) and ran into a minor syntax error (on AIX using xlc). How do I go about reporting what I found? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] C99
On 04-Jun-16 10:12, Sebastian Krause wrote: Sounds good for features that are well-supported by compilers that >people use. (Are there other compilers used than just GCC and MSVC?) Just thought I'd mention that "little ole me" is using IBM vac on AIX rather than gcc. This has more to do with maintaining the gcc rte requirements of anything built by gcc rather compiler features. There is one gcc compiler feature that has proved a show-stopper - constructor - in glibc (Gnome). There they also make an exception for Windows, but not for "other" compilers. FYI: calling the IBM C compiler as c99 magically makes it c99 compliant, as xlc makes it c99 plus several extensions - and take note - as cc it is "pre-c89". In other words, calling IBM compiler as "cc" is not a wise thing to do - I recommend to anyone listing to use either "xlc" or "xlc_r". Michael ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.6.0 Beta Phase Development
On 26/09/2016 17:41, Guido van Rossum wrote: The issue tracker is your friend! I shall remember this for future reference As you probably noticed - new "issue" https://bugs.python.org/issue28290 -- BETA report: Python-3.6 build messages to stderr: AIX and "not GCC" ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 3.6.0 Beta Phase Development
On 13/09/2016 02:15, Ned Deily wrote: the challenge is to put the finishing touches on the features and documentation, squash bugs, and test test test. The next preview release will be 3.6.0b2 Found one typo in Modules/_io/_iomodule.h on line 156 - #endif^L rather than #endif (posted as an issue, but I suppose just a note here would have been enough) I have a longish list of messages to stderr from the compiler (IBM xlc) on AIX. Rather than spam everyone with those - would opening an issue be the way forward, or just sending the file to a person - rather than the list. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] C99
I am using IBM xlc aka vac - version 11. afaik it will deal with c99 features (by default I set it to behave that way because a common 'issue' is C++ style comments, when they should not be that style (fyi: not seen that in Python). IMHO: GCC is not just a compiler - it brings with it a whole set of infrastructure requirements (aka run-time environment, rte). Certainly not an issue for GNU environments, but non-gnu (e.g., posix) will/may have continual side-effects from "competing" rte.. At least that was my experience when I was using gcc rather than xlc. On 6/4/2016 9:53 AM, Martin Panter wrote: Sounds good for features that are well-supported by compilers that people use. (Are there other compilers used than just GCC and MSVC?) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bitfields - short - and xlc compiler
On 2016-03-18 05:57, Andrew Barnert via Python-Dev wrote: Yeah, C99 (6.7.2.1) allows "a qualified or unqualified version of _Bool, signed int, unsigned int, or some other implementation-defined type", and same for C11. This means that a compiler could easily allow an implementation-defined type that's identical to and interconvertible with short, say "i16", to be used in bitfields, but not short itself. And yet, gcc still allows short "even in strictly conforming mode" (4.9), and it looks like Clang and Intel do the same. Meanwhile, MSVC specifically says it's illegal ("The type-specifier for the declarator must be unsigned int, signed int, or int") but then defines the semantics (you can't have a 17-bit short, bit fields act as the underlying type when accessed, alignment is forced to a boundary appropriate for the underlying type). They do mention that allowing char and long types is a Microsoft extension, but still nothing about short, even though it's used in most of the examples on the page. Anyway, is the question what ctypes should do? If a platform's compiler allows "short M: 1", especially if it has potentially different alignment than "int M: 1", ctypes on that platform had better make ("M", c_short, 1) match the former, right? So it sounds like you need some configure switch to test that your compiler doesn't allow short bit fields, so your ctypes build at least skips that part of _ctypes_test.c and test_bitfields.py, and maybe even doesn't allow them in Python code. >> test_short fails om AIX when using xlC in any case. How terrible is this? a) this does not look solveable using xlC, and I expect from the comment above re: MSVC, that it will, or should also fail there. And, imho, if anything is to done, it is a decision to be made by "Python". b) aka - it sounds like a defect, at least in the test. c) what danger is there to existing Python code if "short" is expected, per legacy when compilers did (and GCC still does - verified that when I compile with gcc the test does not signal failure) So, more with regard to c) - is there something I could/should be looking at in Python itself, in order to message that the code is not supported by the compiler? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bitfields - short - and xlc compiler
Update: Is this going to be impossible? test_short fails om AIX when using xlC in any case. How terrible is this? == FAIL: test_shorts (ctypes.test.test_bitfields.C_Test) -- Traceback (most recent call last): File "/data/prj/aixtools/python/python-2.7.11.2/Lib/ctypes/test/test_bitfields.py", line 48, in test_shorts self.assertEqual((name, i, getattr(b, name)), (name, i, func(byref(b), name))) AssertionError: Tuples differ: ('M', 1, -1) != ('M', 1, 1) First differing element 2: -1 1 - ('M', 1, -1) ? - + ('M', 1, 1) -- Ran 440 tests in 1.538s FAILED (failures=1, skipped=91) Traceback (most recent call last): File "./Lib/test/test_ctypes.py", line 15, in test_main() File "./Lib/test/test_ctypes.py", line 12, in test_main run_unittest(unittest.TestSuite(suites)) File "/data/prj/aixtools/python/python-2.7.11.2/Lib/test/test_support.py", line 1428, in run_unittest _run_suite(suite) File "/data/prj/aixtools/python/python-2.7.11.2/Lib/test/test_support.py", line 1411, in _run_suite raise TestFailed(err) test.test_support.TestFailed: Traceback (most recent call last): File "/data/prj/aixtools/python/python-2.7.11.2/Lib/ctypes/test/test_bitfields.py", line 48, in test_shorts self.assertEqual((name, i, getattr(b, name)), (name, i, func(byref(b), name))) AssertionError: Tuples differ: ('M', 1, -1) != ('M', 1, 1) First differing element 2: -1 1 - ('M', 1, -1) ? - + ('M', 1, 1) On 17-Mar-16 23:31, Michael Felt wrote: a) hope this is not something you expect to be on -list, if so - my apologies! Getting this message (here using c99 as compiler name, but same issue with xlc as compiler name) c99 -qarch=pwr4 -qbitfields=signed -DNDEBUG -O -I. -IInclude -I./Include -I/data/prj/aixtools/python/python-2.7.11.2/Include -I/data/prj/aixtools/python/python-2.7.11.2 -c /data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.c -o build/temp.aix-5.3-2.7/data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.o "/data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.c", line 387.5: 1506-009 (S) Bit field M must be of type signed int, unsigned int or int. "/data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.c", line 387.5: 1506-009 (S) Bit field N must be of type signed int, unsigned int or int. "/data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.c", line 387.5: 1506-009 (S) Bit field O must be of type signed int, unsigned int or int. "/data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.c", line 387.5: 1506-009 (S) Bit field P must be of type signed int, unsigned int or int. "/data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.c", line 387.5: 1506-009 (S) Bit field Q must be of type signed int, unsigned int or int. "/data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.c", line 387.5: 1506-009 (S) Bit field R must be of type signed int, unsigned int or int. "/data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.c", line 387.5: 1506-009 (S) Bit field S must be of type signed int, unsigned int or int. for: struct BITS { int A: 1, B:2, C:3, D:4, E: 5, F: 6, G: 7, H: 8, I: 9; short M: 1, N: 2, O: 3, P: 4, Q: 5, R: 6, S: 7; }; in short xlC v11 does not like short (xlC v7 might have accepted it, but "32-bit machines were common then". I am guessing that 16-bit is not well liked on 64-bit hw now. reference for xlC v7, where short was (apparently) still accepted: http://www.serc.iisc.ernet.in/facilities/ComputingFacilities/systems/cluster/vac-7.0/html/language/ref/clrc03defbitf.htm I am taking this is from xlC v7 documentation from the URL, not because I know it personally. So - my question: if "short" is unacceptable for POWER, or maybe only xlC (not tried with gcc) - how terrible is this, and is it possible to adjust the test so - the test is accurate? I am going to modify the test code so it is struct BITS { signed int A: 1, B:2, C:3, D:4, E: 5, F: 6, G: 7, H: 8, I: 9; unsigned int M: 1, N: 2, O: 3, P: 4, Q: 5, R: 6, S: 7; }; And see what happens - BUT - what does this have for impact on python - assuming that "short" bitfields are not supported? p.s. not submitting this a bug (now) as it may just be that "you" consider it a bug in xlC to not support (signed) short bit fields. p.p.s. Note: xlc, by default, considers bitfields to be unsigned. I was trying to force them to signe
[Python-Dev] bitfields - short - and xlc compiler
a) hope this is not something you expect to be on -list, if so - my apologies! Getting this message (here using c99 as compiler name, but same issue with xlc as compiler name) c99 -qarch=pwr4 -qbitfields=signed -DNDEBUG -O -I. -IInclude -I./Include -I/data/prj/aixtools/python/python-2.7.11.2/Include -I/data/prj/aixtools/python/python-2.7.11.2 -c /data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.c -o build/temp.aix-5.3-2.7/data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.o "/data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.c", line 387.5: 1506-009 (S) Bit field M must be of type signed int, unsigned int or int. "/data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.c", line 387.5: 1506-009 (S) Bit field N must be of type signed int, unsigned int or int. "/data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.c", line 387.5: 1506-009 (S) Bit field O must be of type signed int, unsigned int or int. "/data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.c", line 387.5: 1506-009 (S) Bit field P must be of type signed int, unsigned int or int. "/data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.c", line 387.5: 1506-009 (S) Bit field Q must be of type signed int, unsigned int or int. "/data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.c", line 387.5: 1506-009 (S) Bit field R must be of type signed int, unsigned int or int. "/data/prj/aixtools/python/python-2.7.11.2/Modules/_ctypes/_ctypes_test.c", line 387.5: 1506-009 (S) Bit field S must be of type signed int, unsigned int or int. for: struct BITS { int A: 1, B:2, C:3, D:4, E: 5, F: 6, G: 7, H: 8, I: 9; short M: 1, N: 2, O: 3, P: 4, Q: 5, R: 6, S: 7; }; in short xlC v11 does not like short (xlC v7 might have accepted it, but "32-bit machines were common then". I am guessing that 16-bit is not well liked on 64-bit hw now. reference for xlC v7, where short was (apparently) still accepted: http://www.serc.iisc.ernet.in/facilities/ComputingFacilities/systems/cluster/vac-7.0/html/language/ref/clrc03defbitf.htm I am taking this is from xlC v7 documentation from the URL, not because I know it personally. So - my question: if "short" is unacceptable for POWER, or maybe only xlC (not tried with gcc) - how terrible is this, and is it possible to adjust the test so - the test is accurate? I am going to modify the test code so it is struct BITS { signed int A: 1, B:2, C:3, D:4, E: 5, F: 6, G: 7, H: 8, I: 9; unsigned int M: 1, N: 2, O: 3, P: 4, Q: 5, R: 6, S: 7; }; And see what happens - BUT - what does this have for impact on python - assuming that "short" bitfields are not supported? p.s. not submitting this a bug (now) as it may just be that "you" consider it a bug in xlC to not support (signed) short bit fields. p.p.s. Note: xlc, by default, considers bitfields to be unsigned. I was trying to force them to signed with -qbitfields=signed - and I still got messages. So, going back to defaults. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Question about sys.path and sys.argv and how packaging (may) affects default values
On 2016-03-02 18:45, Thomas Wouters wrote: On Wed, Mar 2, 2016 at 3:50 AM, Michael Felt wrote: Hello all, 1) There are many lists to choose from - if this is the wrong one for questions about packaging - please forgive me, and point me in the right direction. It's hard to say where this belongs best, but python-list would probably have done as well. 2) Normally, I have just packaged python, and then moved on. However, recently I have been asked to help with packaging an 'easier to install' python by people using cloud-init, and more recently people wanting to use salt-stack (on AIX). FYI: I have been posting about my complete failure to build 2.7.11 ( http://bugs.python.org/issue26466) - so, what I am testing is based on 2.7.10 - which built easily for me. Going through the 'base documentation' I saw a reference to both sys.argv and sys.path. atm, I am looking for an easy way to get the program name (e.g., /opt/bin/python, versus ./python). I have my reasons (basically, looking for a compiled-in library search path to help with http://bugs.python.org/issue26439) I think the only way to get at the compiled-in search path is to recreate it based on the compiled-in prefix, which you can get through distutils. Python purposely only uses the compiled-in path as the last resort. Instead, it searches for its home relative to the executable and adds a set of directories relative to its home (if they exist). It's not clear to me why you're focusing on these differences, as (as I describe below) they are immaterial. Looking on two platforms (AIX, my build, and debian for power) I am surprised that sys.argv is empty in both cases, and sys.path returns /opt/lib/python27.zip with AIX, but not with debian. When you run python interactively, sys.argv[0] will be '', yes. Since you're not launching a program, there's nothing else to set it to. 'python' (or the path to the executable) wouldn't be the right thing to set it to, because python itself isn't a Python program :) The actual python executable is sys.executable, not sys.argv[0], but you shouldn't usually care about that, either. If you want to know where to install things, distutils is the thing to use. If you want to know where Python thinks it's installed (for debugging purposes only, really), sys.prefix will tell you. root@x064:[/data/prj/aixtools/python/python-2.7.10]/opt/bin/python Python 2.7.10 (default, Nov 3 2015, 14:36:51) [C] on aix5 Type "help", "copyright", "credits" or "license" for more information. import sys sys.argv [''] sys.path ['', '/opt/lib/python27.zip', '/opt/lib/python2.7', '/opt/lib/python2.7/plat-aix5', '/opt/lib/python2.7/lib-tk', '/opt/lib/python2.7/lib-old', '/opt/lib/python2.7/lib-dynload', '/opt/lib/python2.7/site-packages'] michael@ipv4:~$ python Python 2.7.9 (default, Mar 1 2015, 13:01:00) [GCC 4.9.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. import sys sys.argv [''] sys.path ['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-powerpc-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7'] In sys.path, you're seeing the difference between a vanilla Python and Debian's patched Python. Vanilla Python adds $prefix/lib/python27.zip to sys.path unconditionally, whereas Debian removes it when it doesn't exist. Likewise, the dist-packages directory is a local modification by Debian; in vanilla Python it's called 'site-packages' instead. The subdirectories in dist-packages that you see in the Debian case are added by .pth files installed in $prefix -- third-party packages, in other words, adding their own directories to the module search path. And I guess I would be interested in getting '/opt/lib/python2.7/dist-packages' in there as well (or learn a way to later add it for pre-compiled packages such as cloud-init AND that those would also look 'first' in /opt/lib/python2.7/dist-packages/cloud-init for modules added to support cloud-init - should I so choose (mainly in case of compatibility issues between say cloud-init and salt-stack that have common modules BUT may have conflicts) - Hopefully never needed for that reason, but it might also simplify packaging applications that depend on python. A vanilla Python (or non-Debian-built python, even) has no business looking in dist-packages. It
Re: [Python-Dev] Summary of Python tracker Issues
I guess I should have never changed the title - apparently the tracker loses track - there are more than 5 messages. On 2016-03-04 18:08, Python tracker wrote: #26439: ctypes.util.find_library fails when ldconfig/glibc not availab http://bugs.python.org/issue264395 msgs And, while I do not want to ping the list in a rude way, I submitted a patch - not perfect of course (seems to work as expected stand-alone, but not in a 'build' attempt (a previous 'half' patch that was the 'work in progress' did build) - so as I hope to have time in the coming days to dig further - some hints on how to debug the failed 'build moment during 'make install' would be greatly appreciated. Basically, the make install ends with: ... Compiling /var/aixtools/aixtools/python/2.7.11.2/opt/lib/python2.7/xml/sax/xmlreader.py ... Compiling /var/aixtools/aixtools/python/2.7.11.2/opt/lib/python2.7/xmllib.py ... Compiling /var/aixtools/aixtools/python/2.7.11.2/opt/lib/python2.7/xmlrpclib.py ... Compiling /var/aixtools/aixtools/python/2.7.11.2/opt/lib/python2.7/zipfile.py ... make: 1254-004 The error code from the last command is 1. Stop. root@x064:[/data/prj/aixtools/python/python-2.7.11.2] So, my question: how do I make the 'compile' of /var/aixtools/aixtools/python/2.7.11.2/opt/lib/python2.7/zipfile.py more verbose? I tried "make V=1 DESTDIR=/var/aixtools/aixtools/python/2.7.11.2 install". but the output was identical. Thanks, Michael ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] New OpenSSL - has anyone ever looked at (in)compatibility with LibreSSL
Can look at it. There has been a lot of discussion, iirc, between OpenSSL and LibreSSL re: version identification. Thx for the reference. On 08-Mar-16 14:55, Hasan Diwan wrote: On 8 March 2016 at 00:49, Michael Felt <mailto:mich...@felt.demon.nl>> wrote: As a relative newcomer I may have missed a long previous discussion re: linking with OpenSSL and/or LibreSSL. In an ideal world this would be rtl linking, i.e., underlying complexities of *SSL libraries are hidden from applications. In short, when I saw this http://bugs.python.org/issue26465 Title: Upgrade OpenSSL shipped with python installers, it reminded me I need to start looking at LibreSSL again - and that, if not already done - might be something "secure" for python as well. According to the libressl website, one of the projects primary goals is to remain "backwards-compatible with OpenSSL", which is to say, to either have code work without changes or to fail gracefully when it uses the deprecated bits. It does seem it ships with OpenBSD. There is an issue open on bugs to address whatever incompatibilities remain between LibreSSL and OpenSSL[1]. Perhaps you might want to take a look at that? -- H 1. https://bugs.python.org/issue23177 Michael ___ Python-Dev mailing list Python-Dev@python.org <mailto:Python-Dev@python.org> https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/hasan.diwan%40gmail.com -- OpenPGP: http://hasan.d8u.us/gpg.asc Sent from my mobile device Envoyé de mon portable ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] New OpenSSL - has anyone ever looked at (in)compatibility with LibreSSL
As a relative newcomer I may have missed a long previous discussion re: linking with OpenSSL and/or LibreSSL. In an ideal world this would be rtl linking, i.e., underlying complexities of *SSL libraries are hidden from applications. In short, when I saw this http://bugs.python.org/issue26465 Title: Upgrade OpenSSL shipped with python installers, it reminded me I need to start looking at LibreSSL again - and that, if not already done - might be something "secure" for python as well. Michael ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Question about sys.path and sys.argv and how packaging (may) affects default values
Hello all, 1) There are many lists to choose from - if this is the wrong one for questions about packaging - please forgive me, and point me in the right direction. 2) Normally, I have just packaged python, and then moved on. However, recently I have been asked to help with packaging an 'easier to install' python by people using cloud-init, and more recently people wanting to use salt-stack (on AIX). FYI: I have been posting about my complete failure to build 2.7.11 ( http://bugs.python.org/issue26466) - so, what I am testing is based on 2.7.10 - which built easily for me. Going through the 'base documentation' I saw a reference to both sys.argv and sys.path. atm, I am looking for an easy way to get the program name (e.g., /opt/bin/python, versus ./python). I have my reasons (basically, looking for a compiled-in library search path to help with http://bugs.python.org/issue26439) Looking on two platforms (AIX, my build, and debian for power) I am surprised that sys.argv is empty in both cases, and sys.path returns /opt/lib/python27.zip with AIX, but not with debian. root@x064:[/data/prj/aixtools/python/python-2.7.10]/opt/bin/python Python 2.7.10 (default, Nov 3 2015, 14:36:51) [C] on aix5 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.argv [''] >>> sys.path ['', '/opt/lib/python27.zip', '/opt/lib/python2.7', '/opt/lib/python2.7/plat-aix5', '/opt/lib/python2.7/lib-tk', '/opt/lib/python2.7/lib-old', '/opt/lib/python2.7/lib-dynload', '/opt/lib/python2.7/site-packages'] michael@ipv4:~$ python Python 2.7.9 (default, Mar 1 2015, 13:01:00) [GCC 4.9.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.argv [''] >>> sys.path ['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-powerpc-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7'] And I guess I would be interested in getting '/opt/lib/python2.7/dist-packages' in there as well (or learn a way to later add it for pre-compiled packages such as cloud-init AND that those would also look 'first' in /opt/lib/python2.7/dist-packages/cloud-init for modules added to support cloud-init - should I so choose (mainly in case of compatibility issues between say cloud-init and salt-stack that have common modules BUT may have conflicts) - Hopefully never needed for that reason, but it might also simplify packaging applications that depend on python. Many thanks for your time and pointers into the documentation, It is a bit daunting :) Michael ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com