[Python-Dev] Modifying the self-signed.pythontest.net certificate

2016-01-13 Thread Martin Panter
In order to fix the SSL test suite , I would like to modify the certificate used by https://self-signed.pythontest.net. So far I have a patch ready for the pythontestdotnet repository, but I want to know if I can just push to that repository, or if other steps

[Python-Dev] Discussion related to memory leaks requested

2016-01-13 Thread Matthew Paulson
Hi: I've spent some time performing memory leak analysis while using Python in an embedded configuration. The pattern is: Py_Initialize(); ... run empty python source file ... Py_Finalize(); I've identified several suspect areas including dictionary maitenace in import.c:~ 414

Re: [Python-Dev] Discussion related to memory leaks requested

2016-01-13 Thread Brett Cannon
Probably the best way to handle this, Matthew, is to open issues at bugs.python.org for each of the leaks you have found and then they can be discussed there. And thanks for being willing to report these! On Wed, 13 Jan 2016 at 11:42 Matthew Paulson wrote: > Hi: > > I've

Re: [Python-Dev] Modifying the self-signed.pythontest.net certificate

2016-01-13 Thread Benjamin Peterson
On Wed, Jan 13, 2016, at 13:15, Martin Panter wrote: > In order to fix the SSL test suite > , I would like to modify the > certificate used by https://self-signed.pythontest.net. So far I have > a patch ready for the pythontestdotnet repository, but I want to

Re: [Python-Dev] Discussion related to memory leaks requested

2016-01-13 Thread Benjamin Peterson
This is a "well-known" issue. Parts of the interpreter (and especially, extension modules) cheerfully stash objects in global variables with no way to clean them up. Fixing this is a large project, which probably involves implementing PEP 489. On Wed, Jan 13, 2016, at 11:32, Matthew Paulson

Re: [Python-Dev] Discussion related to memory leaks requested

2016-01-13 Thread Trent Nelson
Gflags/umdh is pretty useful on Windows, I used it to track down a few quirky PyParallel memory leaks. Steps: 1. Enable global flags: gflags –i python.exe +ust 2. Launch Python. 3. Enable the umdh tracer: umdh –p: -f:d1.log 4. Kill it after a short run. 5.

Re: [Python-Dev] Modifying the self-signed.pythontest.net certificate

2016-01-13 Thread Benjamin Peterson
On Wed, Jan 13, 2016, at 23:31, Martin Panter wrote: > On 14 January 2016 at 05:34, Benjamin Peterson > wrote: > > On Wed, Jan 13, 2016, at 13:15, Martin Panter wrote: > >> In order to fix the SSL test suite > >> , I would like to modify

Re: [Python-Dev] Modifying the self-signed.pythontest.net certificate

2016-01-13 Thread Martin Panter
On 14 January 2016 at 05:34, Benjamin Peterson wrote: > On Wed, Jan 13, 2016, at 13:15, Martin Panter wrote: >> In order to fix the SSL test suite >> , I would like to modify the >> certificate used by https://self-signed.pythontest.net. So

Re: [Python-Dev] Discussion related to memory leaks requested

2016-01-13 Thread Victor Stinner
Hi, 2016-01-13 20:32 GMT+01:00 Matthew Paulson : > I've spent some time performing memory leak analysis while using Python in an > embedded configuration. Hum, did you try tracemalloc? https://docs.python.org/dev/library/tracemalloc.html

Re: [Python-Dev] Discussion related to memory leaks requested

2016-01-13 Thread Andrew Barnert via Python-Dev
On Jan 13, 2016, at 14:49, Matthew Paulson wrote: > > Hi Victor: > > No, I'm using the new heap analysis functions in DS2015. Isn't that going to report any memory that Python's higher level allocators hold in their freelists as leaked, even though it isn't leaked? > We

Re: [Python-Dev] Discussion related to memory leaks requested

2016-01-13 Thread Matthew Paulson
Hi Andrew: These are all good points, and I defer to your experience -- I am new to python internals, but the fact remains that after multiple iterations of our embedded test case, we are seeing continued allocations (DS2015) and growth of the working set (windows task manager). If your are

Re: [Python-Dev] Debugging using VS 2015

2016-01-13 Thread Brett Cannon
Use PCbuild/get_externals.bat to download the external dependencies. On Wed, 13 Jan 2016 at 13:25 Eddy Quicksall wrote: > I am using 3.5.1. I’m adding an extension for my special case. > > > > I know this

Re: [Python-Dev] Debugging using VS 2015

2016-01-13 Thread Brett Cannon
Then your paths might be off; I would try with a fresh checkout and make sure you can build Python cleanly before trying your embedded case to make sure your tooling is set up (e.g., you don't have Mercurial installed so that's a cause of one of your error messages). On Wed, 13 Jan 2016 at 15:58

Re: [Python-Dev] Discussion related to memory leaks requested

2016-01-13 Thread Matthew Paulson
Hi Victor: No, I'm using the new heap analysis functions in DS2015. We think we have found one issue. In the following sequence, dict has no side effects, yet it is used -- unless someone can shed light on why dict is used in this case: /* Clear the modules dict. */

Re: [Python-Dev] PEP 510: Specialize functions with guards

2016-01-13 Thread Victor Stinner
I extracted a patch from my FAT Python project to implement the PEP 510: https://bugs.python.org/issue26098 FYI I also extracted the runtime part of the FAT Python optimizer and put it on GitHub: https://github.com/haypo/fat The fat module provides specialize(), get_specialized() and

Re: [Python-Dev] Discussion related to memory leaks requested

2016-01-13 Thread Nick Coghlan
On 14 January 2016 at 10:18, Matthew Paulson wrote: > Hi Andrew: > > These are all good points, and I defer to your experience -- I am new to > python internals, but the fact remains that after multiple iterations of > our embedded test case, we are seeing continued

Re: [Python-Dev] Debugging using VS 2015

2016-01-13 Thread Steve Dower
On 13Jan2016 1556, Eddy Quicksall wrote: Those files already exist: W:\Python-3.5.1>PCbuild\get_externals.bat Fetching external libraries... bzip2-1.0.6 already exists, skipping. nasm-2.11.06 already exists, skipping. openssl-1.0.2d already exists, skipping. sqlite-3.8.11.0 already exists,