Re: [Python-Dev] I hope this won't be my last comment here ~ yet it may well be...

2016-04-22 Thread Burkhard Meier
Ok. no more ellipses...what I was trying to share is an unhappy experience
I had with the open source Linux community.

I am sure this will not happen on this Python Dev list of professionals.

Please ignore my comments.

>From now on I will focus on contributing to Python (especially on a Windows
platform) and not taking up valuable reading time.


On Thu, Apr 21, 2016 at 3:43 PM, Chris Barker  wrote:

>  I'm really confused -- you had a handful of very positive responses to
> your offer to help with Python on Windows.
> Then a couple off the cuff remarks (at least one of which was serious)
> about what is often known as "the bus factor":
> But I think you may want to take into account the history here. This has
> been talked about A LOT in the Python community for years -- so we may be a
> bit blase about it. Note that Wikipedia's page on the bus factor:
> """An early instance of this sort of query was when Michael McLay publicly
> asked, in 1994, what would happen to the Python language
>  if Guido
> van Rossum  were hit by a
> bus.[8] """
> So this has been very, very well hashed out in the Python community.
> And a quick look at the existence of this list, the messages on it, and
> the source repo will tell you that Python is in no way a personal project
> of one person. (not to mentions the PSF)
> I think the lessons here are:
> - don't be too sensitive
> and, important for every open source community:
> - your comments and questions will be taken far more seriously if you have
> done your homework.
> -CHB
> On Thu, Apr 21, 2016 at 4:54 AM, Burkhard Meier 
> wrote:
>> Please do allow me to share my humble experiences of being a software
>> professional on a Windows platform.
>> Almost 20 years.
>> You know what; when I tried out 'sugar Linux' or Peppermint,,,the "admin'
>> dude kicked me out 5 times in one sole eve,
>> Maybe this is just *me*..
>> You know what: I did have my time with this *open source community*...
>> I was just asking a sincere question.
>> C'mon
>> This was rather very ridiculous.
>> ___
>> Python-Dev mailing list
>> Unsubscribe:
> --
> Christopher Barker, Ph.D.
> Oceanographer
> Emergency Response Division
> NOAA/NOS/OR(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
Python-Dev mailing list

[Python-Dev] Summary of Python tracker Issues

2016-04-22 Thread Python tracker

ACTIVITY SUMMARY (2016-04-15 - 2016-04-22)
Python tracker at

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open5491 ( +2)
  closed 33095 (+56)
  total  38586 (+58)

Open issues with patches: 2384 

Issues opened (41)

#26773: Shelve works inconsistently when carried over to child process  opened by Paul Ellenbogen

#26774: Elide Py_atomic fences when WITH_THREAD is disabled?  opened by larry

#26776: Determining the failure of C API call is ambiguous  opened by serhiy.storchaka

#26779: pdb continue followed by an exception in the same frame shows  opened by Sriram Rajagopalan

#26781: os.walk max_depth  opened by palaviv

#26786: bdist_msi duplicates directories with names in ALL CAPS to a b  opened by Ivan.Pozdeev

#26787: test_distutils fails when configured --with-lto  opened by gregory.p.smith

#26788: test_gdb fails all tests on a profile-opt build configured --w  opened by gregory.p.smith

#26789: Please do not log during shutdown  opened by smurfix

#26790: bdist_msi package duplicates everything to a bogus location wh  opened by Ivan.Pozdeev

#26791: shutil.move fails to move symlink (Invalid cross-device link)  opened by Unode

#26792: docstrings of runpy.run_{module,path} are rather sparse  opened by Antony.Lee

#26793: uuid causing thread issues when forking using os.fork py3.4+  opened by Steven Adams

#26794: curframe can be None in  opened by Jacek.Pliszka

#26796: BaseEventLoop.run_in_executor shouldn't specify max_workers fo  opened by Hans Lawrenz

#26797: Segafault in _PyObject_Alloc  opened by yselivanov

#26798: add BLAKE2 to hashlib  opened by Zooko.Wilcox-O'Hearn

#26800: Don't accept bytearray as filenames part 2  opened by pjenvey

#26801: Fix shutil.get_terminal_size() to catch AttributeError  opened by ebarry

#26803: syslog logging handler fails with address in unix abstract nam  opened by xdegaye

#26804: Prioritize lowercase proxy variables in urllib.request  opened by frispete

#26806: IDLE not displaying RecursionError tracebacks  opened by terry.reedy

#26807: mock_open()().readline() fails at EOF  opened by rbcollins

#26809: `string` exposes ChainMap from `collections`  opened by leewz

#26810: inconsistent garbage collector behavior across platforms when  opened by unsec treedee

#26811: segfault due to null pointer in tuple  opened by random832

#26812: ExtendedInterpolation drops user-defined 'vars' during _interp  opened by yab-arz

#26814: [WIP] Add a new _PyObject_FastCall() function which avoids the  opened by haypo

#26815: SIGBUS in test_ssl.test_dealloc_warn() on "AMD64 FreeBSD 10.0  opened by haypo

#26816: Make concurrent.futures.Executor an abc  opened by xiang.zhang

#26817: Docs for StringIO should link to io.BytesIO  opened by guettli

#26818: trace CLI doesn't respect -s option  opened by berker.peksag

#26819: _ProactorReadPipeTransport pause_reading()/resume_reading() br  opened by Fulvio Esposito

#26820: Prevent uses of format string based PyObject_Call* that do not  opened by josh.r

#26822: itemgetter/attrgetter/methodcaller objects ignore keyword argu  opened by serhiy.storchaka

#26823: Shrink recursive tracebacks  opened by ebarry

#26824: Make some macros use Py_TYPE  opened by xiang.zhang

#26826: Expose new copy_file_range() syscal in os module and use it to  opened by StyXman

#26827: PyObject *PyInit_myextention -> PyMODINIT_FUNC PyInit_myextent  opened by prinsherbert

#26828: Implement __length_hint__() on map() and filter() to optimize  opened by haypo

Re: [Python-Dev] Modify PyMem_Malloc to use pymalloc for performance

2016-04-22 Thread Victor Stinner

My pull request has been merged into numpy. numpy now uses
PyMem_RawMalloc() rather than PyMem_Malloc() since it uses the memory
allocator without holding the GIL:

It was proposed to modify numpy to hold the GIL. Maybe it will be done later.

It means that there are no more C extensions known to not use
correctly Python memory allocators. So I pushed my change in CPython
to use the pymalloc memory allocator in PyMem_Malloc():

I documented that porting C extensions to Python 3.6 require to run
tests with PYTHONMALLOC=debug. This environment variable enables
checks at runtime to validate the usage of Python memory allocators,
including checks on the GIL. PYTHONMALLOC=debug and the check on the
GIL are new in Python 3.6.

By the way, I modified the code to log the fatal error. if a buffer
overflow/underflow is detected in a free function like PyObject_Free()
and tracemalloc is enabled, the traceback where the memory block was
allocated is now displayed:

Moreover, the warning logger now also log where file, socket, etc.
were allocated on ResourceWarning:

It looks like Python 3.6 will help developers ;-)


2016-04-20 1:33 GMT+02:00 Victor Stinner :
> Ping? Is someone still opposed to my change #26249 "Change
> PyMem_Malloc to use pymalloc allocator"? If no, I think that I will
> push my change.
> My change only changes two lines, so it can be easily reverted before
> CPython 3.6 if we detect major issues in third-party extensions. And
> maybe it's better to push such change today to get more time to play
> with it, than pushing it late in the development of CPython 3.6.
> The new PYTHONMALLOC=debug feature allows to quickly and easily check
> the usage of the PyMem_Malloc() API, even if Python is compiled in
> release mode.
> I checked multiple Python extensions written in C. I only found one
> bug in numpy and I sent a patch (not merged yet).
> victor
> 2016-03-15 0:19 GMT+01:00 Victor Stinner :
>> 2016-02-12 14:31 GMT+01:00 M.-A. Lemburg :
> If your program has bugs, you can use a debug build of Python 3.5 to
> detect misusage of the API.
>>> Yes, but people don't necessarily do this, e.g. I have
>>> for a very long time ignored debug builds completely
>>> and when I started to try them, I found that some of the
>>> things I had been doing with e.g. free list implementations
>>> did not work in debug builds.
>> I just added support for debug hooks on Python memory allocators on
>> Python compiled in *release* mode. Set the environment variable
>> PYTHONMALLOC to debug to try with Python 3.6.
>> I added a check on PyObject_Malloc() debug hook to ensure that the
>> function is called with the GIL held. I opened an issue to add a
>> similar check on PyMem_Malloc():
>>> Yes, but those are part of the stdlib. You'd need to check
>>> a few C extensions which are not tested as part of the stdlib,
>>> e.g. numpy, scipy, lxml, pillow, etc. (esp. ones which implement custom
>>> types in C since these will often need the memory management
>>> APIs).
>>> It may also be a good idea to check wrapper generators such
>>> as cython, swig, cffi, etc.
>> I ran the test suite of numpy, lxml, Pillow and cryptography (used cffi).
>> I found a bug in numpy. numpy calls PyMem_Malloc() without holding the GIL:
>> Except of this bug, all other tests pass with PyMem_Malloc() using
>> pymalloc and all debug checks.
>> Victor
Python-Dev mailing list

[Python-Dev] Issue with DLL import library installation in Cygwin

2016-04-22 Thread Erik Bray
Hi all,

I've been working on compiling/installing Python on Cygwin and have
hit upon an odd issue in the Makefile that seems to have been around
for as long as there's been Cygwin support in it.

When building Python on Cygwin, both a libpython-X.Y.dll and a
libpython-X.Y.dll.a are created.  The latter is an "import library"
consisting of stubs for functions in the DLL so that it can be linked
to statically when building, for example, extension modules.

The odd bit is that in the altbininstall target (see [1]) if the
$(DLLLIBRARY) variable is defined then only it is installed, while
$(LDLIBRARY) (which in this cases references the import library) is
*not* installed, except in $(prefix)/lib/pythonX.Y/config, which is
not normally on the linker search path, or even included by
python-config --ldflags.  Therefore static linking to libpython fails,
unless the search path is explicitly modified, or a symlink is created
from $(prefix)/lib/pythonX.Y/config/libpython.dll.a to $(prefix)/lib.

In fact Cygwin's own package for Python manually creates the latter
symlink in its install script.  But it's not clear why Python's
Makefile doesn't install this file in the first place.  In other
words, why not install $LDLIBRARY regardless?


Python-Dev mailing list