Cython taking more time than regular Python
Hope this is good place to ask question about Cython as well. Following code of mine is taking 2.5 times more time than Native Python code: %%cython import numpy as np a = np.array([]) def large_sum2(int num_range): np.append(a,[i for i in xrange(num_range)]) return a.sum %timeit large_sum2(10) 10 loops, best of 3: 19 ms per loop on the other side python takes much less time: def large_sum(num_range): return sum([i for i in xrange(num_range)]) %timeit large_sum(10) 100 loops, best of 3: 7 ms per loop -- https://mail.python.org/mailman/listinfo/python-list
Python 3.5.1 C API, the global available available is not destroyed when delete the module
Hi,
I have a app loading python35.dll. Use python API PyImport_AddModule
to run a py file. And use PyDict_DelItemString to delete the module.
There is a global vailable in the py file. The global variable is not
destroyed when calling PyDict_DelItemString to delete the module. That
cause the memory leak.
But it is ok with python33.dll, the global variable can be destroyed
when calling PyDict_DelItemString to delete the module.
How to resolve the problem? Is there a workaround? I need to use
python35.dll and wish the global variable in a module can be released
automatically when call PyDict_DelItemString to delete the module.
Here is the python test code:
class Simple:
def __init__( self ):
print('Simple__init__')
def __del__( self ):
print('Simple__del__')
simple = Simple()
Thanks,
Jack
--
https://mail.python.org/mailman/listinfo/python-list
Re: Cython taking more time than regular Python
Arshpreet Singh wrote: > Hope this is good place to ask question about Cython as well. > Following code of mine is taking 2.5 times more time than Native Python > code: > > %%cython > import numpy as np > a = np.array([]) > def large_sum2(int num_range): > np.append(a,[i for i in xrange(num_range)]) > return a.sum > > %timeit large_sum2(10) > 10 loops, best of 3: 19 ms per loop > > on the other side python takes much less time: > > def large_sum(num_range): > return sum([i for i in xrange(num_range)]) > > %timeit large_sum(10) > 100 loops, best of 3: 7 ms per loop But the two versions aren't the same! If you compare similar code: In [2]: %%cython def sigma(n): return sum(i for i in range(n)) ...: In [3]: def tau(n): return sum(i for i in range(n)) In [4]: %timeit sigma(10) 100 loops, best of 3: 8.34 ms per loop In [5]: %timeit tau(10) 100 loops, best of 3: 12.7 ms per loop And if you decide to go lowlevel (there may be smarter ways, but I'm a complete amateur with Cython): In [7]: %%cython def omega(int n): cdef long i cdef long result = 0 for i in range(n): result += i return result ...: In [8]: %timeit omega(10) 1 loops, best of 3: 91.6 µs per loop In [9]: assert omega(10) == tau(10) == sigma(10) -- https://mail.python.org/mailman/listinfo/python-list
Using copyreg to pickle unpicklable oblects
Hi, I have an object (pygame.mixer.Sound) and, to convert it to a picklable format, I can run: sound_object.get_raw() and to turn that back into an object, I can run: sound_object = pygame.mixer.Sound(raw_data) Is it possible to use copyreg or something similar so it's done automatically when I pickle pygame.mixer.Sound() objects? -- https://mail.python.org/mailman/listinfo/python-list
Re: How to install Python.h on FreeBSD 10.0-RELEASE?
In article <201609...@crcomp.net>, Don Kuenz wrote: > >The installed python packages are shown below. Searches lead me to >believe that a PTH option make play a role. > > >$ uname -v >FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 >r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC > >$ pkg info | grep python >py27-dnspython-1.14.0 DNS toolkit for Python >py27-notify-0.1.1_11 python bindings for libnotify >py27-telepathy-python-0.15.19_1 Python bindings for the Telepathy framework >python-2.7_2,2 The "meta-port" for the default version >of Python interpreter >python2-2_3The "meta-port" for version 2 of the >Python interpreter >python27-2.7.12Interpreted object-oriented programming language >python3-3_3The "meta-port" for version 3 of the >Python interpreter >python34-3.4.5 Interpreted object-oriented programming language > >$ cd /usr/ports/lang/python >$ make config >===> No options to configure > >--- > >Thank you, > >-- >Don Kuenz KB7RPU > >There be triple ways to take, of the eagle or the snake, >Or the way of a man with a maid; >But the seetest way to me is a ship's upon the sea >In the heel of the Northeast Trade. > - Kipling Why not upgrade to 10.2 ? -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca God,Queen and country!Never Satan President Republic!Beware AntiChrist rising! http://www.fullyfollow.me/rootnl2k Look at Psalms 14 and 53 on Atheism Time for the USA to hold a referendum on its republic and vote to dissolve!! -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.5.1 C API, the global available available is not destroyed when delete the module
On Mon, Sep 19, 2016 at 4:47 PM, dl l wrote: > I have a app loading python35.dll. Use python API PyImport_AddModule > to run a py file. And use PyDict_DelItemString to delete the module. > There is a global vailable in the py file. The global variable is not > destroyed when calling PyDict_DelItemString to delete the module. That > cause the memory leak. Does the module still exist in sys.modules? If so, it's not leaked - it's still in use. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Using copyreg to pickle unpicklable oblects
kerbingamer376 wrote:
> Hi,
[No need to start a new thread for this]
> I have an object (pygame.mixer.Sound) and, to convert it to a picklable
> format, I can run:
>
> sound_object.get_raw()
>
> and to turn that back into an object, I can run:
>
> sound_object = pygame.mixer.Sound(raw_data)
>
> Is it possible to use copyreg or something similar so it's done
> automatically when I pickle pygame.mixer.Sound() objects?
Have a look at the example from the documentation:
>>> import copyreg, copy, pickle
>>> class C(object):
... def __init__(self, a):
... self.a = a
...
>>> def pickle_c(c):
... print("pickling a C instance...")
... return C, (c.a,)
...
>>> copyreg.pickle(C, pickle_c)
Translating that gives (untested)
import copyreg # copy_reg in Python 2
import pygame.mixer
def pickle_sound(sound):
return pygame.mixer.Sound, (sound.get_raw(),)
copyreg.pickle(pygame.mixer.Sound, pickle_sound)
Does that work?
--
https://mail.python.org/mailman/listinfo/python-list
Re: Using copyreg to pickle unpicklable oblects
On Monday, September 19, 2016 at 5:48:35 PM UTC+1, Peter Otten wrote:
> kerbingamer376 wrote:
>
> > Hi,
>
> [No need to start a new thread for this]
>
> > I have an object (pygame.mixer.Sound) and, to convert it to a picklable
> > format, I can run:
> >
> > sound_object.get_raw()
> >
> > and to turn that back into an object, I can run:
> >
> > sound_object = pygame.mixer.Sound(raw_data)
> >
> > Is it possible to use copyreg or something similar so it's done
> > automatically when I pickle pygame.mixer.Sound() objects?
>
> Have a look at the example from the documentation:
>
> >>> import copyreg, copy, pickle
> >>> class C(object):
> ... def __init__(self, a):
> ... self.a = a
> ...
> >>> def pickle_c(c):
> ... print("pickling a C instance...")
> ... return C, (c.a,)
> ...
> >>> copyreg.pickle(C, pickle_c)
>
>
> Translating that gives (untested)
>
> import copyreg # copy_reg in Python 2
> import pygame.mixer
>
> def pickle_sound(sound):
> return pygame.mixer.Sound, (sound.get_raw(),)
>
> copyreg.pickle(pygame.mixer.Sound, pickle_sound)
>
> Does that work?
I get:
_pickle.PicklingError: Can't pickle : attribute lookup Sound on
builtins failed
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to install Python.h on FreeBSD 10.3-RELEASE?
In article you wrote: > In article <201609...@crcomp.net>, Don Kuenz wrote: >> >>The installed python packages are shown below. Searches lead me to >>believe that a PTH option make play a role. >> >> >>$ uname -v >>FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 >>r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC >> >>$ pkg info | grep python >>py27-dnspython-1.14.0 DNS toolkit for Python >>py27-notify-0.1.1_11 python bindings for libnotify >>py27-telepathy-python-0.15.19_1 Python bindings for the Telepathy framework >>python-2.7_2,2 The "meta-port" for the default version >>of Python interpreter >>python2-2_3The "meta-port" for version 2 of the >>Python interpreter >>python27-2.7.12Interpreted object-oriented programming >>language >>python3-3_3The "meta-port" for version 3 of the >>Python interpreter >>python34-3.4.5 Interpreted object-oriented programming >>language >> >>$ cd /usr/ports/lang/python >>$ make config >>===> No options to configure >> >>--- >> > > Why not upgrade to 10.2 ? A newer host is loaded with 10.3. It uses identical python packages. It seems that FreeBSD python has been neglected for years. $ uname -v FreeBSD 10.3-RELEASE #0 r297264: Fri Mar 25 02:10:02 UTC 2016 r...@releng1.nyi.freebsd.or g:/usr/obj/usr/src/sys/GENERIC $ pkg info | grep python py27-dnspython-1.12.0 DNS toolkit for Python py27-notify-0.1.1_11 python bindings for libnotify py27-telepathy-python-0.15.19_1 Python bindings for the Telepathy framework python-2.7_2,2 The "meta-port" for the default version of Python interpreter python2-2_3The "meta-port" for version 2 of the Python interpreter python27-2.7.12Interpreted object-oriented programming language python3-3_3The "meta-port" for version 3 of the Python interpreter python34-3.4.5 Interpreted object-oriented programming language Thank you, -- Don Kuenz KB7RPU It's easier to mend neglect than to quicken love. - Saint Jerome -- https://mail.python.org/mailman/listinfo/python-list
Re: Using copyreg to pickle unpicklable oblects
kerbingamer376 wrote:
> On Monday, September 19, 2016 at 5:48:35 PM UTC+1, Peter Otten wrote:
>> kerbingamer376 wrote:
>>
>> > Hi,
>>
>> [No need to start a new thread for this]
>>
>> > I have an object (pygame.mixer.Sound) and, to convert it to a picklable
>> > format, I can run:
>> >
>> > sound_object.get_raw()
>> >
>> > and to turn that back into an object, I can run:
>> >
>> > sound_object = pygame.mixer.Sound(raw_data)
>> >
>> > Is it possible to use copyreg or something similar so it's done
>> > automatically when I pickle pygame.mixer.Sound() objects?
>>
>> Have a look at the example from the documentation:
>>
>> >>> import copyreg, copy, pickle
>> >>> class C(object):
>> ... def __init__(self, a):
>> ... self.a = a
>> ...
>> >>> def pickle_c(c):
>> ... print("pickling a C instance...")
>> ... return C, (c.a,)
>> ...
>> >>> copyreg.pickle(C, pickle_c)
>>
>>
>> Translating that gives (untested)
>>
>> import copyreg # copy_reg in Python 2
>> import pygame.mixer
>>
>> def pickle_sound(sound):
>> return pygame.mixer.Sound, (sound.get_raw(),)
>>
>> copyreg.pickle(pygame.mixer.Sound, pickle_sound)
>>
>> Does that work?
>
> I get:
>
> _pickle.PicklingError: Can't pickle : attribute lookup
> Sound on builtins failed
Looks like Sound doesn't set the __module__ attribute correctly.
Try injecting it into the built-in namespace:
import builtins
builtins.Sound = pygame.mixer.Sound
--
https://mail.python.org/mailman/listinfo/python-list
Re: how to automate java application in window using python
On Monday, September 19, 2016 at 11:29:24 AM UTC+12, Ned Batchelder wrote: > > On Sunday, September 18, 2016 at 6:45:32 PM UTC-4, Lawrence D’Oliveiro wrote: >> >> A CLI gives the user power over the computer. While a GUI is a great way to >> give the computer, and proprietary software companies, power over the user. > > This is completely beside the point of the original question. Which is meaningless, as I explained. -- https://mail.python.org/mailman/listinfo/python-list
Another å, ä, ö question
I am studying some of these tutorials: https://pythonprogramming.net/matplotlib-intro-tutorial/ I am recreating the code and I use my native Swedish for comments, labels and titles. Until now I have been doing so using Geany and executing using F5. This works fine. But -- now I tested using emacs instead using C-c C-c to execute. Noting happens so I try to run the program from command line and find that now Python can't stand my å, ä and ö. I am puzzled: With Geany there is no problem but outside Geany I am punished by Python for using Swedish. Any ideas? /Martin -- https://mail.python.org/mailman/listinfo/python-list
Re: Another å, ä, ö question
Am 19.09.16 um 22:21 schrieb Martin Schöön: I am studying some of these tutorials: https://pythonprogramming.net/matplotlib-intro-tutorial/ I am recreating the code and I use my native Swedish for comments, labels and titles. Until now I have been doing so using Geany and executing using F5. This works fine. But -- now I tested using emacs instead using C-c C-c to execute. Noting happens so I try to run the program from command line and find that now Python can't stand my å, ä and ö. I am puzzled: With Geany there is no problem but outside Geany I am punished by Python for using Swedish. you are not "punished for Swedish", you need to tell Python the encoding of the file it runs. See here: https://www.python.org/dev/peps/pep-0263/ Assuming that you use UTF-8 (you should check with an emacs expert, I am not an emacs user), try putting the header #!/usr/bin/python # -*- coding: utf-8 -*- on top of your files. Christian -- https://mail.python.org/mailman/listinfo/python-list
Re: Functions Of Functions Returning Functions
On Monday, September 19, 2016 at 6:54:31 PM UTC+12, dieter wrote: > Some time ago, we had a (quite heated) discussion here ... I have noticed that people get very defensive about things they don’t understand. > Often, functions returning functions are more difficult to understand > than "first order" functions (those returning simple values only) -- > at least for people not familiar with higher abstraction levels. Maybe if the alternative meant writing 200 extra lines of code (as in this case), they might feel differently... -- https://mail.python.org/mailman/listinfo/python-list
Re: Looking for tips and gotchas for working with Python 3.5 zipapp feature
On 16 September 2016 19:48 at 21:08, Malcolm Greene wrote:
> Looking for tips or edge case gotchas associated with using Python 3.5's
> new zipapp feature.
It's worth pointing out that support for executing zipped applications
has been part of Python since Python 2.6 or so. The pyz files created
by the zipapp module can be used with any version of Python back to
then (assuming the code doesn't use newer features of Python, of
course).
There aren't many "gotchas" that I know of - the main one being the
one you note below about locating data files - some modules (not many
these days) assume they are living in a normal directory and locate
data files by doing path calculations based on __file__. This doesn't
work in a zipfile.
The other main limitation (not so much a gotcha as a consequence of
how the OS works) is that you can't load C extensions (pyd or so
files) from a zipfile. If you need to do that, you'll have to bundle
the C extensions to work around that limitation, but that's pretty
advanced usage.
> For those of you wondering what this feature is, see
> the end of this post for a brief background [1].
>
> Two questions in particular:
>
> 1. Are there any issues with deploying scripts that sit in sub-
>folders beneath the directory being zipped, eg. does zipapp only
>support a flat folder of scripts or does it recursively zip and
>path sub-folders?
It will work fine with a folder structure. However, the way it works
is that it runs a __main__.py script which is at the root of the
zipfile, with the zipfile inserted on sys.path. So, if you have
"import foo" in your __main__.py file, foo.py must be at the root.
If you have something like the following layout in your zipfile, you
can do "import foo.bar.baz" from your __main__.py. Basically the usual
rules for imports apply.
__main__.py
foo
__init__.py
bar
__init__.py
baz.py
> 2. Can additional non-Python files like config files be added to a
>zipapp without breaking them and if so, how would your script
>reference these embedded files (by opening up the zipapp as a zip
>archive and navigating from there?).
They can be, but as you note, you need to reference those data files
specially, you can't find them via the __file__ attribute of a module,
and path manipulation.
The stdlib API for finding data files stored in a package is
pkgutil.get_data
(https://docs.python.org/3/library/pkgutil.html#pkgutil.get_data).
Basically, if you have a "data file" called myapp.ini" in the package
directory of the "foo" package, you can access it as
data = pkgutil.get_data('foo', 'myapp.ini')
The content of the file is returned as a bytes object.
File layout:
__main__.py
foo
__init__.py
myapp.ini
bar
__init__.py
baz.py
__main__.py
import pkgutil
import foo.bar.baz
data = pkgutil.get_data('foo', 'myapp.ini')
print(data)
The data access API is unfortunately very minimal, for example there's
no way to find what data files are present, you need to know the name
in advance. There have been proposals to add a richer API, but none
are present yet.
If you need more than this, you can as you say open your zipapp as a
zipfile (but by doing so, you commit to only working if the
application is zipped, which may or may not matter to you). Something
like this:
__main__.py
import zipfile
import os
my_archive = os.path.dirname(__file__)
zf = zipfile.ZipFile(my_archive)
...
> Thank you,
> Malcolm
Hope this helps :-)
> [1] The zipapp feature of Python 3.5 is pretty cool: It allows you to
> package your Python scripts in a single executable zip file. This
> isn't a replacement for tools like PyInstaller or Py2Exe, eg. it
> doesn't bundle the Python interpreter in the zip file, but it's a
> clean way to distribute multi-file scripts in environments where you
> have control over users' Python setups.
If you do want a completely standalone application, what you can do is
to write a small C program that embeds the Python interpreter, and
just starts your zipapp. Then ship that along with the "embedded"
Python distribution (available from the python.org pages) and you have
a standalone application. I'm currently working on such a wrapper -
the prototype is at https://github.com/pfmoore/pylaunch. If I can get
it into a suitable state, I may look at adding the wrapper to the
zipapp module for Python 3.7.
Paul
--
https://mail.python.org/mailman/listinfo/python-list
Re: Another å, ä, ö question
On Tuesday, September 20, 2016 at 8:21:25 AM UTC+12, Martin Schöön wrote: > But -- now I tested using emacs instead using C-c C-c to execute. > Noting happens so I try to run the program from command line and > find that now Python can't stand my å, ä and ö. What version of Python? Python 3 accepts Unicode UTF-8 as a matter of course. -- https://mail.python.org/mailman/listinfo/python-list
Re: How to install Python.h on FreeBSD 10.3-RELEASE?
In article <201609...@crcomp.net>, Don Kuenz wrote: > >In article you wrote: >> In article <201609...@crcomp.net>, Don Kuenz wrote: >>> >>>The installed python packages are shown below. Searches lead me to >>>believe that a PTH option make play a role. >>> >>> >>>$ uname -v >>>FreeBSD 10.0-RELEASE #0 r260789: Thu Jan 16 22:34:59 UTC 2014 >>>r...@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC >>> >>>$ pkg info | grep python >>>py27-dnspython-1.14.0 DNS toolkit for Python >>>py27-notify-0.1.1_11 python bindings for libnotify >>>py27-telepathy-python-0.15.19_1 Python bindings for the Telepathy framework >>>python-2.7_2,2 The "meta-port" for the default version >>>of Python interpreter >>>python2-2_3The "meta-port" for version 2 of the >>>Python interpreter >>>python27-2.7.12Interpreted object-oriented programming >language >>>python3-3_3The "meta-port" for version 3 of the >>>Python interpreter >>>python34-3.4.5 Interpreted object-oriented programming >language >>> >>>$ cd /usr/ports/lang/python >>>$ make config >>>===> No options to configure >>> >>>--- >>> >> >> Why not upgrade to 10.2 ? > >A newer host is loaded with 10.3. It uses identical python packages. >It seems that FreeBSD python has been neglected for years. > > > >$ uname -v >FreeBSD 10.3-RELEASE #0 r297264: Fri Mar 25 02:10:02 UTC 2016 >r...@releng1.nyi.freebsd.or >g:/usr/obj/usr/src/sys/GENERIC >$ pkg info | grep python >py27-dnspython-1.12.0 DNS toolkit for Python >py27-notify-0.1.1_11 python bindings for libnotify >py27-telepathy-python-0.15.19_1 Python bindings for the Telepathy framework >python-2.7_2,2 The "meta-port" for the default version >of Python interpreter >python2-2_3The "meta-port" for version 2 of the >Python interpreter >python27-2.7.12Interpreted object-oriented programming language >python3-3_3The "meta-port" for version 3 of the >Python interpreter >python34-3.4.5 Interpreted object-oriented programming language > > > >Thank you, > >-- >Don Kuenz KB7RPU > >It's easier to mend neglect than to quicken love. - Saint Jerome My servers are running uname -v FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11 18:38:15 UTC 2016 r...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC Run freebsd-update fetch ; freebsd-update install -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca God,Queen and country!Never Satan President Republic!Beware AntiChrist rising! http://www.fullyfollow.me/rootnl2k Look at Psalms 14 and 53 on Atheism Time for the USA to hold a referendum on its republic and vote to dissolve!! -- https://mail.python.org/mailman/listinfo/python-list
Re: how to automate java application in window using python
On Monday, September 19, 2016 at 4:24:31 PM UTC-4, Lawrence D’Oliveiro wrote: > On Monday, September 19, 2016 at 11:29:24 AM UTC+12, Ned Batchelder wrote: > > > > On Sunday, September 18, 2016 at 6:45:32 PM UTC-4, Lawrence D’Oliveiro > > wrote: > >> > >> A CLI gives the user power over the computer. While a GUI is a great way to > >> give the computer, and proprietary software companies, power over the user. > > > > This is completely beside the point of the original question. > > Which is meaningless, as I explained. We get it, you don't like GUIs. The question is clearly not meaningless, since others are helping with the question. --Ned. --Ned. -- https://mail.python.org/mailman/listinfo/python-list
Re: how to automate java application in window using python
Hi Matt Wheeler,
can it contorl Maplesoft's maple which is a java executable file?
On Sunday, September 18, 2016 at 5:02:15 PM UTC+8, Matt Wheeler wrote:
> On Thu, 15 Sep 2016, 08:12 meInvent bbird, wrote:
>
> > how to automate java application in window using python
> >
> > 1. scroll up or down of scroll bar
> > 2. click button
> > 3. type text in textbox
> >
>
> I would recommend having a look at pywinauto
> https://github.com/pywinauto/pywinauto
>
> It presents a very nice pythonic interface to Windows and the controls
> within them, allowing statements such as (copied from the Readme):
>
> app.UntitledNotepad.MenuSelect("Help->About Notepad")
> app.AboutNotepad.OK.Click()
> app.UntitledNotepad.Edit.TypeKeys ("pywinauto Works!", with_spaces = True)
>
> (I found it already quite stable when I first used it a couple of years
> ago, and looking at the Readme now it's clearly still an active project,
> the optional image capture feature using pillow is new)
>
> >
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.5.1 C API, the global available available is not destroyed when delete the module
Thanks for the replay. I run this script as __main__ module, this module is removed from sys.modules, and the ref count becomes 0. 2016-09-20 0:00 GMT+08:00 Chris Angelico : > On Mon, Sep 19, 2016 at 4:47 PM, dl l wrote: > > I have a app loading python35.dll. Use python API PyImport_AddModule > > to run a py file. And use PyDict_DelItemString to delete the module. > > There is a global vailable in the py file. The global variable is not > > destroyed when calling PyDict_DelItemString to delete the module. That > > cause the memory leak. > > Does the module still exist in sys.modules? If so, it's not leaked - > it's still in use. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: Looking for tips and gotchas for working with Python 3.5 zipapp feature
Hi Paul, WOW!:) I really appreciate the detailed response. You answered all my questions. I'm looking forward to testing out your pylaunch wrapper. Thank you very much! Malcolm -- https://mail.python.org/mailman/listinfo/python-list
Re: Cython taking more time than regular Python
Peter Otten schrieb am 19.09.2016 um 14:55: > In [7]: %%cython > def omega(int n): > cdef long i > cdef long result = 0 > for i in range(n): result += i > return result >...: > > In [8]: %timeit omega(10) > 1 loops, best of 3: 91.6 µs per loop Note that this is the worst benchmark ever. Any non-dump C compiler will happily apply Young Gauß and calculate the result in constant time. Stefan -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.5.1 C API, the global available available is not destroyed when delete the module
On Monday 19 September 2016 16:47, dl l wrote:
> Hi,
>
> I have a app loading python35.dll. Use python API PyImport_AddModule
> to run a py file. And use PyDict_DelItemString to delete the module.
> There is a global vailable in the py file. The global variable is not
> destroyed when calling PyDict_DelItemString to delete the module. That
> cause the memory leak.
See:
https://bugs.python.org/issue28202
> But it is ok with python33.dll, the global variable can be destroyed
> when calling PyDict_DelItemString to delete the module.
You are calling PyDict_DelItemString on sys.modules, but that won't clear the
reference to the module if there are any other references to it.
> How to resolve the problem? Is there a workaround? I need to use
> python35.dll and wish the global variable in a module can be released
> automatically when call PyDict_DelItemString to delete the module.
The global object will not be released so long as any other object refers to
it. If the global object is still alive, it will keep the module alive.
Are you sure that the object has no other references to it? Have you put it in
a list or dict or some other object?
> Here is the python test code:
>
> class Simple:
> def __init__( self ):
> print('Simple__init__')
> def __del__( self ):
> print('Simple__del__')
>
> simple = Simple()
You cannot rely on the __del__ method running at any specific time. It might
not run until your application exists.
That test code is not sufficient. How do you call the test code? If you do this
from main:
import mymodule # module above
del sys.modules['mymodule']
that will delete the sys.modules cache entry, but the module object is still
alive. If you are absolutely sure there are no other references to the global
variable and the module, then it sounds like a leak.
Can you confirm that the gc is enabled?
For a work-around, I can only suggest manually killing the global. What happens
if you do this:
del sys.modules['mymodule']
mymodule.simple = None
del mymodule
or equivalent?
--
Steven
git gets easier once you get the basic idea that branches are homeomorphic
endofunctors mapping submanifolds of a Hilbert space.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Cython taking more time than regular Python
On Tuesday, 20 September 2016 11:00:40 UTC+5:30, Stefan Behnel wrote: > > In [8]: %timeit omega(10) > > 1 loops, best of 3: 91.6 µs per loop > > Note that this is the worst benchmark ever. Any non-dump C compiler will > happily apply Young Gauß and calculate the result in constant time. Hey Stefan what else you suggest? -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3.5.1 C API, the global available available is not destroyed when delete the module
Hi Steven,
Thanks for reply.
I logged bug https://bugs.python.org/issue28202. I added more info in this
bug :).
Yes, it's a workaround to set the global variables to None. But My app just
provide a framework for my customers to run python scripts. That's means
the workaround requires my customers to update their python scripts. That
may make them unhappy :(. Is there a workaround in my code C++ side to call
Python C APIs to resolve the memory leak issue?
2016-09-20 13:49 GMT+08:00 Steven D'Aprano <
steve+comp.lang.pyt...@pearwood.info>:
> On Monday 19 September 2016 16:47, dl l wrote:
>
> > Hi,
> >
> > I have a app loading python35.dll. Use python API PyImport_AddModule
> > to run a py file. And use PyDict_DelItemString to delete the module.
> > There is a global vailable in the py file. The global variable is not
> > destroyed when calling PyDict_DelItemString to delete the module. That
> > cause the memory leak.
>
> See:
>
> https://bugs.python.org/issue28202
>
> > But it is ok with python33.dll, the global variable can be destroyed
> > when calling PyDict_DelItemString to delete the module.
>
> You are calling PyDict_DelItemString on sys.modules, but that won't clear
> the
> reference to the module if there are any other references to it.
>
>
> > How to resolve the problem? Is there a workaround? I need to use
> > python35.dll and wish the global variable in a module can be released
> > automatically when call PyDict_DelItemString to delete the module.
>
> The global object will not be released so long as any other object refers
> to
> it. If the global object is still alive, it will keep the module alive.
>
> Are you sure that the object has no other references to it? Have you put
> it in
> a list or dict or some other object?
>
>
> > Here is the python test code:
> >
> > class Simple:
> > def __init__( self ):
> > print('Simple__init__')
> > def __del__( self ):
> > print('Simple__del__')
> >
> > simple = Simple()
>
>
> You cannot rely on the __del__ method running at any specific time. It
> might
> not run until your application exists.
>
>
> That test code is not sufficient. How do you call the test code? If you do
> this
> from main:
>
>
> import mymodule # module above
> del sys.modules['mymodule']
>
>
>
> that will delete the sys.modules cache entry, but the module object is
> still
> alive. If you are absolutely sure there are no other references to the
> global
> variable and the module, then it sounds like a leak.
>
> Can you confirm that the gc is enabled?
>
>
> For a work-around, I can only suggest manually killing the global. What
> happens
> if you do this:
>
> del sys.modules['mymodule']
> mymodule.simple = None
> del mymodule
>
>
> or equivalent?
>
>
>
>
>
> --
> Steven
> git gets easier once you get the basic idea that branches are homeomorphic
> endofunctors mapping submanifolds of a Hilbert space.
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
