[Numpy-discussion] allclose() does not check shape of inputs
Hi, I think this is a bug: In [16]: np.allclose([1.0, 1.0], [1.1], rtol=0.1, atol=0.0) Out[16]: True Shall I create a ticket? r. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] allclose() does not check shape of inputs
Fri, 13 Nov 2009 11:54:51 +0100, Robert Cimrman wrote: I think this is a bug: In [16]: np.allclose([1.0, 1.0], [1.1], rtol=0.1, atol=0.0) Out[16]: True It's broadcasting. I'm not sure it is a bug: np.allclose([1.0, 1.0], [1.1, 1.1, 1.1], rtol=0.1, atol=0.0) False -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] allclose() does not check shape of inputs
Pauli Virtanen wrote: Fri, 13 Nov 2009 11:54:51 +0100, Robert Cimrman wrote: I think this is a bug: In [16]: np.allclose([1.0, 1.0], [1.1], rtol=0.1, atol=0.0) Out[16]: True It's broadcasting. I'm not sure it is a bug: np.allclose([1.0, 1.0], [1.1, 1.1, 1.1], rtol=0.1, atol=0.0) False I see, I hit the corner case... Sorry for the noise! r. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.4.0: Setting a firm release date for 1st december.
On Wed, Nov 4, 2009 at 11:05 AM, Jarrod Millman mill...@berkeley.eduwrote: On Wed, Nov 4, 2009 at 1:37 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: It would be good if we could also have one more merge of the work in the doc editor (close to 300 new/changed docstrings now). I can have it all reviewed by the 13th. That would be great. Thanks for taking care of that. All wiki changes are now reviewed and can be merged. Under numpy/doc there is a file HOWTO_MERGE_WIKI_DOCS.txt with details on how this is done. Cheers, Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Initial implementation of histogram_discrete()
13/11/09 @ 09:41 (+0200), thus spake Priit Laes: Does anyone have a scenario where one would actually have both negative and positive numbers (integers) in the list? Yes: when you have a random variable that is the difference of two (discrete) random variables. For example, if you measure the difference in number of days off per week because of sickness between two groups of people, you would end up with a discrete variable with both positive and negative integers. So, how about numpy.histogram_discrete() that returns data the way histogram() does: a list containing histogram values (ie counts) and list of sorted items from min(input)...max(input). ? In my humble opinion, it would be nice. Regards. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] bus error in embedded numpy
Hi, I'm trying to embed Python in a MATLAB mex file. I've been coming under some pressure to make my Python code available to my MATLAB colleagues so I am trying to come up with a relatively general way of calling numerical python code from Matlab. I am making some progress - but get a reliable crash from numpy. This only occurs the second time I am loading it. So I Py_Initialize, import numpy, Py_Finalize (all works fine), but then if I clear the mex file (clear funcname in matlab - which calls Py_Finalize through a mexAtExit handler) and try to run the function again (which will reinitialize interpreter and import numpy again) I get the followin stack trace from multisarray.so. Wondered if anyone could through any light. I have already run into this bug http://bugs.python.org/issue6869 which prevents me using ctypes... I wondered if this was related. For now its not such a big problem - I will just avoid unloading the mex function (with clear function). But I thought it might be indicative of a memory leak or some other problem since I think in theory it should work (It does if numpy isn't imported). Cheers Robin Bus error detected at Fri Nov 13 17:11:57 2009 Configuration: MATLAB Version: 7.8.0.347 (R2009a) MATLAB License: 161051 Operating System: Darwin 10.0.0 Darwin Kernel Version 10.0.0: Fri Jul 31 22:47:34 PDT 2009; root:xnu-1456.1.25~1/RELEASE_I386 i386 Window System:The X.Org Foundation (10402000), display /tmp/launch-2p1ZWg/:0 Current Visual: 0x24 (class 4, depth 24) Processor ID: x86 Family 6 Model 15 Stepping 10, GenuineIntel Virtual Machine: Java 1.6.0_15-b03-219 with Apple Inc. Java HotSpot(TM) Client VM mixed mode Default Encoding: ISO-8859-1 Fault Count: 1 Register State: eax = 0001 ebx = 307b12ab ecx = edx = 305ef148 esi = 305ef140 edi = 32a79d60 ebp = b097c6c8 esp = b097c670 eip = 307b14be flg = 00010202 Stack Trace: [0] multiarray.so:PyArray_FromScalar~(0x305ef140, 0, 2, 0x32e23287) + 542 bytes [1] multiarray.so:gentype_nonzero_number~(0x305ef140, 0x2eaa6db0, 0, 0x32e73cfe) + 36 bytes [2] Python:PyObject_IsTrue~(0x305ef140, 0x2ea95de0, 2, 0) + 63 bytes [3] Python:PyEval_EvalFrameEx~(0x33c95160, 0, 0x332941e0, 0) + 12598 bytes [4] Python:PyEval_EvalFrameEx~(0x32a70f80, 0, 0x332941e0, 0) + 24217 bytes [5] Python:PyEval_EvalCodeEx~(0x33f05068, 0x332941e0, 0, 0x32a96794) + 1819 bytes [6] Python:PyEval_EvalFrameEx~(0x32a96640, 0, 0x332941e0, 0) + 16561 bytes [7] Python:PyEval_EvalCodeEx~(0x33f051d0, 0x332941e0, 0, 0x32a6fcc0) + 1819 bytes [8] Python:PyEval_EvalFrameEx~(0x32a6fb60, 0, 0x33bc99c0, 0) + 16561 bytes [9] Python:PyEval_EvalCodeEx~(0x334abda0, 0x33bc99c0, 0, 0x32ad3b24) + 1819 bytes [10] Python:PyEval_EvalFrameEx~(0x32ad39d0, 0, 0x33bc94b0, 0) + 16561 bytes [11] Python:PyEval_EvalCodeEx~(0x332919b0, 0x33bc94b0, 0, 0x33ce5294) + 1819 bytes [12] Python:PyEval_EvalFrameEx~(0x33ce5150, 0, 0x33bc94b0, 0) + 16561 bytes [13] Python:PyEval_EvalCodeEx~(0x332918d8, 0x33bc94b0, 0, 0x34d5156c) + 1819 bytes [14] Python:PyEval_EvalFrameEx~(0x34d51430, 0, 0x33bc9300, 0x33bc9300) + 16561 bytes [15] Python:PyEval_EvalCodeEx~(0x33291968, 0x33bc9300, 0x33bc9300, 0) + 1819 bytes [16] Python:PyEval_EvalCode~(0x33291968, 0x33bc9300, 0x33bc9300, 0x9325378f) + 87 bytes [17] Python:PyImport_ExecCodeModuleEx~(0xb097e4cb numpy.core._internal, 0x33291968, 0xb097dbbf /Library/Frameworks/Python.frame.., 0x332a2000) + 193 bytes [18] Python:load_source_module~(1, 0, 0xb097e418 X™C†, 0xb097e41c) + 726 bytes [19] Python:import_submodule~(0xb097e4d6 _internal, 0x33d1ce1c _internal, 9, 0x32e2cfc9) + 293 bytes [20] Python:load_next~(0xb097e4cb numpy.core._internal, 0xb097e8cc, 0xb097e578, 0x32e8b5e6) + 195 bytes [21] Python:import_module_level~(0x32ee4aa0 TD, 0x, 0xee9e70b3, 0x32e6df4d «E) + 142 bytes [22] Python:PyImport_ImportModuleLevel~(0x33d1ce1c _internal, 0x33b25150, 0x33b25150, 0x32ee4aa0 TD) + 45 bytes [23] Python:builtin___import__~(0, 0x348854e0, 0, 0x32e73cfe) + 156 bytes [24] Python:PyObject_Call~(0x33da9a08, 0x348854e0, 0, 0x32e233cd) + 45 bytes [25] Python:PyEval_CallObjectWithKeywords~(0x33da9a08, 0x348854e0, 0, 0x33b25150) + 112 bytes [26] Python:PyEval_EvalFrameEx~(0x33cd2da0, 0, 0x33b25150, 0x33b25150) + 8138 bytes [27] Python:PyEval_EvalCodeEx~(0x2068, 0x33b25150, 0x33b25150, 0) + 1819 bytes [28] Python:PyEval_EvalCode~(0x2068, 0x33b25150, 0x33b25150, 0x9325378f) + 87 bytes [29] Python:PyImport_ExecCodeModuleEx~(0xb097fadb numpy.core, 0x2068, 0xb097ed7f /Library/Frameworks/Python.frame.., 0x4adc73cc) + 193 bytes [30] Python:load_source_module~(1, 0, 0xb097f5cc, 0) + 726 bytes [31] Python:load_package~(5, 0, 0xb097fa28, 0xb097fa2c) + 427 bytes [32] Python:import_submodule~(0xb097fae1
Re: [Numpy-discussion] bus error in embedded numpy
I forgot to add it doesn't happen if Py_Finalize isn't called - if I take that out and just let the OS/matlab unload the mex function then I can run it many times without the error, but it does leak a bit of memory each time. Cheers Robin On Fri, Nov 13, 2009 at 5:23 PM, Robin robi...@gmail.com wrote: Hi, I'm trying to embed Python in a MATLAB mex file. I've been coming under some pressure to make my Python code available to my MATLAB colleagues so I am trying to come up with a relatively general way of calling numerical python code from Matlab. I am making some progress - but get a reliable crash from numpy. This only occurs the second time I am loading it. So I Py_Initialize, import numpy, Py_Finalize (all works fine), but then if I clear the mex file (clear funcname in matlab - which calls Py_Finalize through a mexAtExit handler) and try to run the function again (which will reinitialize interpreter and import numpy again) I get the followin stack trace from multisarray.so. Wondered if anyone could through any light. I have already run into this bug http://bugs.python.org/issue6869 which prevents me using ctypes... I wondered if this was related. For now its not such a big problem - I will just avoid unloading the mex function (with clear function). But I thought it might be indicative of a memory leak or some other problem since I think in theory it should work (It does if numpy isn't imported). Cheers Robin Bus error detected at Fri Nov 13 17:11:57 2009 Configuration: MATLAB Version: 7.8.0.347 (R2009a) MATLAB License: 161051 Operating System: Darwin 10.0.0 Darwin Kernel Version 10.0.0: Fri Jul 31 22:47:34 PDT 2009; root:xnu-1456.1.25~1/RELEASE_I386 i386 Window System: The X.Org Foundation (10402000), display /tmp/launch-2p1ZWg/:0 Current Visual: 0x24 (class 4, depth 24) Processor ID: x86 Family 6 Model 15 Stepping 10, GenuineIntel Virtual Machine: Java 1.6.0_15-b03-219 with Apple Inc. Java HotSpot(TM) Client VM mixed mode Default Encoding: ISO-8859-1 Fault Count: 1 Register State: eax = 0001 ebx = 307b12ab ecx = edx = 305ef148 esi = 305ef140 edi = 32a79d60 ebp = b097c6c8 esp = b097c670 eip = 307b14be flg = 00010202 Stack Trace: [0] multiarray.so:PyArray_FromScalar~(0x305ef140, 0, 2, 0x32e23287) + 542 bytes [1] multiarray.so:gentype_nonzero_number~(0x305ef140, 0x2eaa6db0, 0, 0x32e73cfe) + 36 bytes [2] Python:PyObject_IsTrue~(0x305ef140, 0x2ea95de0, 2, 0) + 63 bytes [3] Python:PyEval_EvalFrameEx~(0x33c95160, 0, 0x332941e0, 0) + 12598 bytes [4] Python:PyEval_EvalFrameEx~(0x32a70f80, 0, 0x332941e0, 0) + 24217 bytes [5] Python:PyEval_EvalCodeEx~(0x33f05068, 0x332941e0, 0, 0x32a96794) + 1819 bytes [6] Python:PyEval_EvalFrameEx~(0x32a96640, 0, 0x332941e0, 0) + 16561 bytes [7] Python:PyEval_EvalCodeEx~(0x33f051d0, 0x332941e0, 0, 0x32a6fcc0) + 1819 bytes [8] Python:PyEval_EvalFrameEx~(0x32a6fb60, 0, 0x33bc99c0, 0) + 16561 bytes [9] Python:PyEval_EvalCodeEx~(0x334abda0, 0x33bc99c0, 0, 0x32ad3b24) + 1819 bytes [10] Python:PyEval_EvalFrameEx~(0x32ad39d0, 0, 0x33bc94b0, 0) + 16561 bytes [11] Python:PyEval_EvalCodeEx~(0x332919b0, 0x33bc94b0, 0, 0x33ce5294) + 1819 bytes [12] Python:PyEval_EvalFrameEx~(0x33ce5150, 0, 0x33bc94b0, 0) + 16561 bytes [13] Python:PyEval_EvalCodeEx~(0x332918d8, 0x33bc94b0, 0, 0x34d5156c) + 1819 bytes [14] Python:PyEval_EvalFrameEx~(0x34d51430, 0, 0x33bc9300, 0x33bc9300) + 16561 bytes [15] Python:PyEval_EvalCodeEx~(0x33291968, 0x33bc9300, 0x33bc9300, 0) + 1819 bytes [16] Python:PyEval_EvalCode~(0x33291968, 0x33bc9300, 0x33bc9300, 0x9325378f) + 87 bytes [17] Python:PyImport_ExecCodeModuleEx~(0xb097e4cb numpy.core._internal, 0x33291968, 0xb097dbbf /Library/Frameworks/Python.frame.., 0x332a2000) + 193 bytes [18] Python:load_source_module~(1, 0, 0xb097e418 X™C†, 0xb097e41c) + 726 bytes [19] Python:import_submodule~(0xb097e4d6 _internal, 0x33d1ce1c _internal, 9, 0x32e2cfc9) + 293 bytes [20] Python:load_next~(0xb097e4cb numpy.core._internal, 0xb097e8cc, 0xb097e578, 0x32e8b5e6) + 195 bytes [21] Python:import_module_level~(0x32ee4aa0 TD, 0x, 0xee9e70b3, 0x32e6df4d «E) + 142 bytes [22] Python:PyImport_ImportModuleLevel~(0x33d1ce1c _internal, 0x33b25150, 0x33b25150, 0x32ee4aa0 TD) + 45 bytes [23] Python:builtin___import__~(0, 0x348854e0, 0, 0x32e73cfe) + 156 bytes [24] Python:PyObject_Call~(0x33da9a08, 0x348854e0, 0, 0x32e233cd) + 45 bytes [25] Python:PyEval_CallObjectWithKeywords~(0x33da9a08, 0x348854e0, 0, 0x33b25150) + 112 bytes [26] Python:PyEval_EvalFrameEx~(0x33cd2da0, 0, 0x33b25150, 0x33b25150) + 8138 bytes [27] Python:PyEval_EvalCodeEx~(0x2068, 0x33b25150, 0x33b25150, 0) + 1819 bytes [28] Python:PyEval_EvalCode~(0x2068, 0x33b25150,
Re: [Numpy-discussion] 1.4.0: Setting a firm release date for 1st december.
On Fri, Nov 13, 2009 at 3:25 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: All wiki changes are now reviewed and can be merged. Under numpy/doc there is a file HOWTO_MERGE_WIKI_DOCS.txt with details on how this is done. I checked in the majority of the doc changes, but there are some minor problems with the remaining diffs. I have to run now, but I will look at it later today. Thanks, -- Jarrod Millman Helen Wills Neuroscience Institute 10 Giannini Hall, UC Berkeley http://cirl.berkeley.edu/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] bus error in embedded numpy
I'm trying to embed Python in a MATLAB mex file. I've been coming under some pressure to make my Python code available to my MATLAB colleagues so I am trying to come up with a relatively general way of calling numerical python code from Matlab. I get a similar but different error trying to do the same with scipy - (importing scipy after a Py_Finalize and matlab function clear) - this time in umath.so: Bus error detected at Fri Nov 13 17:53:43 2009 Configuration: MATLAB Version: 7.8.0.347 (R2009a) MATLAB License: 161051 Operating System: Darwin 10.0.0 Darwin Kernel Version 10.0.0: Fri Jul 31 22:47:34 PDT 2009; root:xnu-1456.1.25~1/RELEASE_I386 i386 Window System:The X.Org Foundation (10402000), display /tmp/launch-2p1ZWg/:0 Current Visual: 0x24 (class 4, depth 24) Processor ID: x86 Family 6 Model 15 Stepping 10, GenuineIntel Virtual Machine: Java 1.6.0_15-b03-219 with Apple Inc. Java HotSpot(TM) Client VM mixed mode Default Encoding: ISO-8859-1 Fault Count: 1 Register State: eax = 33028046 ebx = 327464cb ecx = 3382b110 edx = 330180b0 esi = 304d6ac0 edi = 336e6660 ebp = b08fa8f8 esp = b08fa8cc eip = 23810006 flg = 00010a92 Stack Trace: [0] 0x23810006(0x336e6660, 0x3352e415 remainder, 0x32824d74, 0) [1] umath.so:initumath~(0xb08faebb numpy.core.umath, 0xb08faec6 umath, 0xb08faa07 /Library/Frameworks/Python.frame.., 0xa043aab0) + 2152 bytes [2] Python:_PyImport_LoadDynamicModule~(0xb08faebb numpy.core.umath, 0xb08faa07 /Library/Frameworks/Python.frame.., 0xa043aab0, 0x331ae7d3) + 153 bytes [3] Python:load_module~(3, 0, 0xb08fae08 ∞™C†, 0xb08fae0c) + 203 bytes [4] Python:import_submodule~(0xb08faec6 umath, 0x326eecd4 umath, 5, 0x33148fc9) + 293 bytes [5] Python:load_next~(0xb08faebb numpy.core.umath, 0xb08fb2bc, 0xb08faf68, 0x331a75e6) + 195 bytes [6] Python:import_module_level~(0x33200aa0, 0x, 0xee9e70b3, 0x33189f4d «E) + 142 bytes [7] Python:PyImport_ImportModuleLevel~(0x326eecd4 umath, 0x336340c0, 0x336340c0, 0x33200aa0) + 45 bytes [8] Python:builtin___import__~(0, 0x33845ea0, 0, 0x3318fcfe) + 156 bytes [9] Python:PyObject_Call~(0x326ba7b0, 0x33845ea0, 0, 0x3313f3cd) + 45 bytes [10] Python:PyEval_CallObjectWithKeywords~(0x326ba7b0, 0x33845ea0, 0, 0x336340c0) + 112 bytes [11] Python:PyEval_EvalFrameEx~(0x304d5fd0, 0, 0x336340c0, 0x336340c0) + 8138 bytes [12] Python:PyEval_EvalCodeEx~(0x30386e30, 0x336340c0, 0x336340c0, 0) + 1819 bytes [13] Python:PyEval_EvalCode~(0x30386e30, 0x336340c0, 0x336340c0, 0x9325378f) + 87 bytes [14] Python:PyImport_ExecCodeModuleEx~(0xb08fc4cb numpy.core, 0x30386e30, 0xb08fb76f /Library/Frameworks/Python.frame.., 0x4adc73cc) + 193 bytes [15] Python:load_source_module~(1, 0, 0xb08fbfbc X™C†0D80†370, 0) + 726 bytes [16] Python:load_package~(5, 0, 0xb08fc418, 0xb08fc41c) + 427 bytes [17] Python:import_submodule~(0xb08fc4d1 core, 0x3384512a core.numeric, 4, 0x33148fc9) + 293 bytes [18] Python:load_next~(0xb08fc4cb numpy.core, 0xb08fc8cc, 9, 0x331a75e6) + 195 bytes [19] Python:import_module_level~(0x33200aa0, 0x, 0x746e6920, 0x33189f4d «E) + 213 bytes [20] Python:PyImport_ImportModuleLevel~(0x33845124 numpy.core.numeric, 0x33634ae0, 0x33634ae0, 0x33200aa0) + 45 bytes [21] Python:builtin___import__~(0, 0x33845f30, 0, 0x3318fcfe) + 156 bytes [22] Python:PyObject_Call~(0x326ba7b0, 0x33845f30, 0, 0x3313f3cd) + 45 bytes [23] Python:PyEval_CallObjectWithKeywords~(0x326ba7b0, 0x33845f30, 0, 0x33634ae0) + 112 bytes [24] Python:PyEval_EvalFrameEx~(0x304caf70, 0, 0x33634ae0, 0x33634ae0) + 8138 bytes [25] Python:PyEval_EvalCodeEx~(0x30386e78, 0x33634ae0, 0x33634ae0, 0) + 1819 bytes [26] Python:PyEval_EvalCode~(0x30386e78, 0x33634ae0, 0x33634ae0, 0x9325378f) + 87 bytes [27] Python:PyImport_ExecCodeModuleEx~(0xb08fd68b numpy.lib.type_check, 0x30386e78, 0xb08fcd7f /Library/Frameworks/Python.frame.., 6771) + 193 bytes [28] Python:load_source_module~(1, 0, 0xb08fd5d8, 0xb08fd5dc) + 726 bytes [29] Python:import_submodule~(0xb08fd695 type_check, 0x3038a494 type_check, 10, 0x33148fc9) + 293 bytes [30] Python:load_next~(0xb08fd68b numpy.lib.type_check, 0xb08fda8c, 0xb08fd738, 0x331a75e6) + 195 bytes [31] Python:import_module_level~(0x32703e30, 0x, 0xee9e70b3, 0x33189f4d «E) + 142 bytes [32] Python:PyImport_ImportModuleLevel~(0x3038a494 type_check, 0x33634150, 0x33634150, 0x32703e30) + 45 bytes [33] Python:builtin___import__~(0, 0x33845540, 0, 0x3318fcfe) + 156 bytes [34] Python:PyObject_Call~(0x326ba7b0, 0x33845540, 0, 0x3313f3cd) + 45 bytes [35] Python:PyEval_CallObjectWithKeywords~(0x326ba7b0, 0x33845540, 0, 0x33634150) + 112 bytes [36] Python:PyEval_EvalFrameEx~(0x304b4fd0, 0, 0x33634150, 0x33634150) + 8138 bytes [37] Python:PyEval_EvalCodeEx~(0x32727bf0, 0x33634150, 0x33634150, 0) + 1819
Re: [Numpy-discussion] bus error in embedded numpy
On Nov 13, 2009, at 11:23 AM, Robin wrote: Hi, I'm trying to embed Python in a MATLAB mex file. I've been coming under some pressure to make my Python code available to my MATLAB colleagues so I am trying to come up with a relatively general way of calling numerical python code from Matlab. I am making some progress - but get a reliable crash from numpy. This only occurs the second time I am loading it. So I Py_Initialize, import numpy, Py_Finalize (all works fine), but then if I clear the mex file (clear funcname in matlab - which calls Py_Finalize through a mexAtExit handler) and try to run the function again (which will reinitialize interpreter and import numpy again) I get the followin stack trace from multisarray.so. Wondered if anyone could through any light. I have already run into this bug http://bugs.python.org/issue6869 which prevents me using ctypes... I wondered if this was related. For now its not such a big problem - I will just avoid unloading the mex function (with clear function). But I thought it might be indicative of a memory leak or some other problem since I think in theory it should work (It does if numpy isn't imported). I wonder if this is related to the fact that you can't unload a dynamically linked module (like NumPy has). So, when you call Py_Finalize you are not really finalizing your usage of Python extension modules. I'm not sure though. -Travis ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Solaris Sparc build broken
On 11/12/2009 05:56 PM, David Cournapeau wrote: On Thu, Nov 12, 2009 at 11:34 PM, Michael Droettboommd...@stsci.edu wrote: I'm happy to make these changes, since I've got access to the finicky platform, but want to confirm how you would prefer it done first. Cool. The changes should be done for all platforms, and pointer cast should be done through union, not (*(new type*)(x)). I've committed this change in r7731. It's been tested in x86/gcc, x86_64/gcc (both by forcing it to use the npymath version of nextafterl) and sparc/Sun Studio. It probably could use some testing on platforms with other long double types. Mike ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] 1.4.0: Setting a firm release date for 1st december.
On Fri, Nov 13, 2009 at 6:51 PM, Jarrod Millman mill...@berkeley.eduwrote: On Fri, Nov 13, 2009 at 3:25 AM, Ralf Gommers ralf.gomm...@googlemail.com wrote: All wiki changes are now reviewed and can be merged. Under numpy/doc there is a file HOWTO_MERGE_WIKI_DOCS.txt with details on how this is done. I checked in the majority of the doc changes, but there are some minor problems with the remaining diffs. I have to run now, but I will look at it later today. Thanks Jarrod. With minor problems I assume you are talking about the error messages at the top of the diff. Most of those are not terribly important items, some are for things like constants for which only the html docs make sense anyway. Cheers, Ralf ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] bus error in embedded numpy
On Fri, Nov 13, 2009 at 12:05, Travis Oliphant oliph...@enthought.com wrote: On Nov 13, 2009, at 11:23 AM, Robin wrote: Hi, I'm trying to embed Python in a MATLAB mex file. I've been coming under some pressure to make my Python code available to my MATLAB colleagues so I am trying to come up with a relatively general way of calling numerical python code from Matlab. I am making some progress - but get a reliable crash from numpy. This only occurs the second time I am loading it. So I Py_Initialize, import numpy, Py_Finalize (all works fine), but then if I clear the mex file (clear funcname in matlab - which calls Py_Finalize through a mexAtExit handler) and try to run the function again (which will reinitialize interpreter and import numpy again) I get the followin stack trace from multisarray.so. Wondered if anyone could through any light. I have already run into this bug http://bugs.python.org/issue6869 which prevents me using ctypes... I wondered if this was related. For now its not such a big problem - I will just avoid unloading the mex function (with clear function). But I thought it might be indicative of a memory leak or some other problem since I think in theory it should work (It does if numpy isn't imported). I wonder if this is related to the fact that you can't unload a dynamically linked module (like NumPy has). So, when you call Py_Finalize you are not really finalizing your usage of Python extension modules. I'm not sure though. Right. We do some global things when numpy is imported. Since there is no unload step for extension modules, we can't undo them. The second time the interpreter starts up, it doesn't know that numpy has already been loaded and that numpy shouldn't try to do those global things again. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] bus error in embedded numpy
On Fri, Nov 13, 2009 at 6:13 PM, Robert Kern robert.k...@gmail.com wrote: On Fri, Nov 13, 2009 at 12:05, Travis Oliphant oliph...@enthought.com wrote: I wonder if this is related to the fact that you can't unload a dynamically linked module (like NumPy has). So, when you call Py_Finalize you are not really finalizing your usage of Python extension modules. I'm not sure though. Right. We do some global things when numpy is imported. Since there is no unload step for extension modules, we can't undo them. The second time the interpreter starts up, it doesn't know that numpy has already been loaded and that numpy shouldn't try to do those global things again. Thanks for the quick responses... I'm sure you're right - in fact it looks like there was a similar issue here: http://www.mathworks.co.uk/matlabcentral/newsreader/view_thread/262089 I had assumed when matlab unloads the mex function it would also unload python - but it looks like other dynamic libs pulled in from the mex function (in this case python and in turn numpy) aren't unloaded... I wonder if it would be possible to link python statically to my mex function, so it really is unloaded when the mex function is... but I'm getting a bit out of my depth with linker options, and I guess numpy is always loaded dynamically anyway and will stick around. Easy enough to work around it anyway - but just wanted to check what was going on. Cheers Robin ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] bus error in embedded numpy
Fri, 13 Nov 2009 17:23:19 +, Robin wrote: I'm trying to embed Python in a MATLAB mex file. I've been coming under some pressure to make my Python code available to my MATLAB colleagues so I am trying to come up with a relatively general way of calling numerical python code from Matlab. Out of curiosity, how are you handling the Matlab RTLD_GLOBAL issue. Last time [1] I did something similar, I had to hack around it in an ugly way. .. [1] http://www.iki.fi/pav/software/pythoncall/index.html -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] bus error in embedded numpy
On Fri, Nov 13, 2009 at 6:50 PM, Pauli Virtanen pav...@iki.fi wrote: Fri, 13 Nov 2009 17:23:19 +, Robin wrote: I'm trying to embed Python in a MATLAB mex file. I've been coming under some pressure to make my Python code available to my MATLAB colleagues so I am trying to come up with a relatively general way of calling numerical python code from Matlab. Out of curiosity, how are you handling the Matlab RTLD_GLOBAL issue. Last time [1] I did something similar, I had to hack around it in an ugly way. .. [1] http://www.iki.fi/pav/software/pythoncall/index.html Actually I was completely unaware of it (the RTLD_GLOBAL thing). I had googled for a while trying to find a similar project (I had assumed someone must have done it) but somehow didn't find your pythoncall project. It's great - the interface is very close to what I had in mind, but it's much more comprehensive then anything I thought of (I was hoping to handle just contiguous numpy arrays). How does the RTLD_GLOBAL thing manifest itself? So far I have only a very basic start which basically consists of: cmd = mxArrayToString(prhs[0]); PyRun_SimpleString(cmd); but I haven't noticed anything not working - I can run numpy testsuite, and do simple commands as expected (initiliase arrays in the interpreter, run numpy functions on them). Perhaps recent versions of Matlab behave differently (I am using R2009a on a mac). So far the only remotely tricky thing I did was redirect sys.stdout and sys.stderr to a wrapper that uses mexPrintf so output goes to the matlab console. Cheers Robin ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] bus error in embedded numpy
Fri, 13 Nov 2009 19:12:26 +, Robin wrote: [clip] How does the RTLD_GLOBAL thing manifest itself? So far I have only a very basic start which basically consists of: cmd = mxArrayToString(prhs[0]); PyRun_SimpleString(cmd); but I haven't noticed anything not working - I can run numpy testsuite, and do simple commands as expected (initiliase arrays in the interpreter, run numpy functions on them). Perhaps recent versions of Matlab behave differently (I am using R2009a on a mac). The RTLD_GLOBAL issue prevented Numpy from being imported. I think everything was well, until you tried to run import numpy in the embedded process -- loading multiarray.so would fail because of missing symbols. But if it worked for you without the hack, then it must have been changed in the Matlab versions since then (and Pythoncall needs updating...). -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] bus error in embedded numpy
Robin skrev: I had assumed when matlab unloads the mex function it would also unload python - but it looks like other dynamic libs pulled in from the mex function (in this case python and in turn numpy) aren't unloaded... Matlab MEX functions are DLLs, Python interpreter is a DLL, NumPy .pyd files are DLLs... If you cannot unload Python or NumPy, you cannot unload MEX functions either. The same OS constraints apply. If you are using Windows, I would recommend that you expose your NumPy code as a COM object using pywin32, and make a Matlab wrapper (ordinary .m file) that calls the COM object. If you are not using Windows, you could create an XMLRPC server in Python and call that using Matlab's built-in Java VM. Or you can spawn a separate Python process from Matlab, and communicate using pipes or a tcp/ip socket (Matlab's Java or MEX). There are many solutions that are easier than embedding Python in Matlab. Sturla ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] bus error in embedded numpy
Robin skrev: So far the only remotely tricky thing I did was redirect sys.stdout and sys.stderr to a wrapper that uses mexPrintf so output goes to the matlab console. Be careful when you are using file handles. You have to be sure that Matlab, Python and NumPy are all linked against the same CRT. Sturla ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] bus error in embedded numpy
On Fri, Nov 13, 2009 at 7:48 PM, Sturla Molden stu...@molden.no wrote: Robin skrev: I had assumed when matlab unloads the mex function it would also unload python - but it looks like other dynamic libs pulled in from the mex function (in this case python and in turn numpy) aren't unloaded... Matlab MEX functions are DLLs, Python interpreter is a DLL, NumPy .pyd files are DLLs... If you cannot unload Python or NumPy, you cannot unload MEX functions either. The same OS constraints apply. Ah, I hadn't realised it was an OS constraint - I thought it was possible to unload dlls - and that was why matlab provides the clear function. mex automatically clears a function when you rebuild it - I thought that was how you can rebuild and reload mex functions without restarting matlab. I thought it was just a quirk of matlab that there was no way to unload other libraries pulled in through the mex function. I must admit to being a bit out of my depth with this stuff though so I stand corrected. If you are using Windows, I would recommend that you expose your NumPy code as a COM object using pywin32, and make a Matlab wrapper (ordinary .m file) that calls the COM object. If you are not using Windows, you could create an XMLRPC server in Python and call that using Matlab's built-in Java VM. Or you can spawn a separate Python process from Matlab, and communicate using pipes or a tcp/ip socket (Matlab's Java or MEX). There are many solutions that are easier than embedding Python in Matlab. I really want a cross platform solution so that rules out COM. I thought about using web services or something which I guess would be the easiest way to do communication through a tcp socket (least work since I could use existing web services libraries on both sides). But actually I have found it pretty easy to embed Python so far... with about 5 lines of code I'm able run simple strings and with a small cython module I get stdout to the console. I haven't tried passing data back and forth yet but from Pauli's pythoncall it doesn't look like that is too difficult. I was sort of hoping that eventually I could create a numpy array from a matlab data pointer - at least for read only input - so that I could use python code to work on large amounts of data without so much memory overhead (returning smaller results by copying). Do you think I'm likely to run into more problems? I got the feeling from asking questions on IRC that embedding Python is kind of discouraged but I thought in this case it seemed the neatest way. Be careful when you are using file handles. You have to be sure that Matlab, Python and NumPy are all linked against the same CRT. I was aware of this - I thought I would be OK on the mac - at least Python and Numpy and my mex function are built with apple gcc although I'm not sure about Matlab. I guess Windows will be more difficult... But in any case I don't plan to pass any file handles around. Cheers Robin ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] finding close together points.
Anne Archibald wrote: 2009/11/10 Christopher Barker chris.bar...@noaa.gov: I have a bunch of points in 2-d space, and I need to find out which pairs of points are within a certain distance of one-another (regular old Euclidean norm). This is now implemented in SVN. Wow! great -- you sounded interested, but I had no idea you'd run out and do it! thanks! we'll check it out. I (tentatively?) used a set to store the collection of pairs, because my tree traversal is not smart enough to reliably uniquify the pairs without using sets. With more time and energy, I'm sure the algorithm could be improved to avoid using sets (both internally and on return), but I think that's something to save for the Cython version. I agree -- what's wrong with using a set? Thanks, we'll let you know how it works for us. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] bus error in embedded numpy
Robin skrev: Ah, I hadn't realised it was an OS constraint - I thought it was possible to unload dlls - and that was why matlab provides the clear function. mex automatically clears a function when you rebuild it - I thought that was how you can rebuild and reload mex functions without restarting matlab. It is OS dependent if DLLs can be unloaded or not. In Windows you can call CloseHandle on the DLL handle and it's gone. If you can unload a MEX function you can also unload Python or NumPy. But when you unload the MEX function, the DLLs loaded by the MEX are not automatically cleared. There is no garbage collector. In Windows, a load and an unload will result in calls to a function called DllMain in the DLL. And if the DLL has other DLLs to load or clear, that is where you need to place it. You can put a custom DllMain function in a MEX file. But be careful: a DLL is only loaded once, i.e. DLLs are imported as singletons in the process. If two MEX functions load Python26.dll, they get handles to the same instance. If you unload Python26.dll in one MEX, it is unloaded globally from the process, so the other MEX will crash. There is no reference counting by the OS kernel. The solution to this type of DLL Hell in Windows is COM. A COM object is a DLL but not a singleton. You can have multiple instances of the save COM object in one process. On Mac I guess your options are to either statically link everything to the MEX file, or find a way for multiple MEX files to share Python interpreter, e.g. implement some sort of reference counting scheme, which by the way is what COM does on Windows. I really want a cross platform solution so that rules out COM. CORBA or XMLRPC seem to be the standards. I'm not sure I would use either. Do you think I'm likely to run into more problems? I got the feeling from asking questions on IRC that embedding Python is kind of discouraged but I thought in this case it seemed the neatest way. It is discouraged because you get a singleton. You can create subinterpreters (cf. Apache's mod_python), but that will bring problems of it's own. For example you cannot use extensions that require the simplified GIL API (e.g. ctypes). I think this is a major design flaw of CPython. For example with Java's JNI, you get a context pointer to the VM, so you don't have any of these problems. But with CPython, both the interpreter and extensions are basically implemented to be loaded as singletons. I was aware of this - I thought I would be OK on the mac - at least Python and Numpy and my mex function are built with apple gcc although I'm not sure about Matlab. I guess Windows will be more difficult... But in any case I don't plan to pass any file handles around. It applies to any CRT resource, not just files. Compiler is not important, but which CRT is loaded. And if you statically link the same CRT twice, that becomes two CRTs that cannot share resources. In Windows, Microsoft has made sure there are multiple versions of their CRT (msvcrt.dll, msvcr71.dll, msvcr80.dll, msvcr90.dll, ...) that cannot share resources. And anyone not careful with this can experice crashes at random locations. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] finding close together points.
2009/11/13 Christopher Barker chris.bar...@noaa.gov: Anne Archibald wrote: 2009/11/10 Christopher Barker chris.bar...@noaa.gov: I have a bunch of points in 2-d space, and I need to find out which pairs of points are within a certain distance of one-another (regular old Euclidean norm). This is now implemented in SVN. Wow! great -- you sounded interested, but I had no idea you'd run out and do it! thanks! we'll check it out. No problem, it was a fairly minor modification of the two-tree code. I (tentatively?) used a set to store the collection of pairs, because my tree traversal is not smart enough to reliably uniquify the pairs without using sets. With more time and energy, I'm sure the algorithm could be improved to avoid using sets (both internally and on return), but I think that's something to save for the Cython version. I agree -- what's wrong with using a set? It should be possible to arrange the tree traversal algorithm so that each pair of subtrees is automatically traversed only once - akin to for i in range(n): for j in range(i): This would avoid having to build and modify a set every time you traverse the tree (sets are fairly cheap hash tables, so the cost is probably minor compared to other python overhead), and it would also naturally allow the result to be a list/inflatable array (which would save memory, if nothing else). But it would also require carefully thinking through the tree traversal algorithm. Thanks, we'll let you know how it works for us. Good luck! Anne -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] bus error in embedded numpy
pe, 2009-11-13 kello 22:36 +0100, Sturla Molden kirjoitti: [clip] I was aware of this - I thought I would be OK on the mac - at least Python and Numpy and my mex function are built with apple gcc although I'm not sure about Matlab. I guess Windows will be more difficult... But in any case I don't plan to pass any file handles around. It applies to any CRT resource, not just files. Compiler is not important, but which CRT is loaded. And if you statically link the same CRT twice, that becomes two CRTs that cannot share resources. In Windows, Microsoft has made sure there are multiple versions of their CRT (msvcrt.dll, msvcr71.dll, msvcr80.dll, msvcr90.dll, ...) that cannot share resources. And anyone not careful with this can experice crashes at random locations. Well, for Matlab/Python integration meant for numerical computations I would be surprised if CRT is really a problem. You essentially can pass data to Matlab only via Matlab's own interface -- CRT stuff like ownership of pointers to allocated memory, file handles etc. typically do not cross this boundary. -- Pauli Virtanen ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is a 4 -bit int dtype possible
On Fri, Nov 13, 2009 at 19:33, Brian Granger ellisonbg@gmail.com wrote: Hi, I have a large binary data set that has 4-bit integers in it. It is possible to create a numpy dtype for a 4-bit integer? Not really. We have 1-byte atomicity. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Is a 4 -bit int dtype possible
Robert, Thanks for the quick reply. I will just keep twiddling my bits then. Cheers, Brian On Fri, Nov 13, 2009 at 5:38 PM, Robert Kern robert.k...@gmail.com wrote: On Fri, Nov 13, 2009 at 19:33, Brian Granger ellisonbg@gmail.com wrote: Hi, I have a large binary data set that has 4-bit integers in it. It is possible to create a numpy dtype for a 4-bit integer? Not really. We have 1-byte atomicity. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Howto document constants?
Hi All, The documentation documentation says to document constants like functions. So if a module defines a constant is it documented like so: myconstant = 1 Blah and blah That doesn't seem quite right, but what is the proper method? Also, do we have a way to document static class methods? Class attributes? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Howto document constants?
On Fri, Nov 13, 2009 at 6:54 PM, Charles R Harris charlesr.har...@gmail.com wrote: Hi All, The documentation documentation says to document constants like functions. So if a module defines a constant is it documented like so: myconstant = 1 Blah and blah That doesn't seem quite right, but what is the proper method? Also, do we have a way to document static class methods? Class attributes? Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion Hi, Chuck. If you do not get a definitive answer (I'm afraid I don't have one) to these very reasonable questions, would you mind please filing a ticket for enhancement of the docstring standard. Thanks! DG ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Howto document constants?
On Nov 13, 2009, at 9:54 PM, Charles R Harris wrote: Hi All, The documentation documentation says to document constants like functions. So if a module defines a constant is it documented like so: myconstant = 1 Blah and blah That doesn't seem quite right, but what is the proper method? Why wouldn't you document the constant in the docstring of the module where it's defined ? Personally, I used ..data to describe the constants in numpy.ma http://docs.scipy.org/doc/numpy/_sources/reference/maskedarray.baseclass.txt ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
