[Python-Dev] The process I intend to follow for any proposed changes to NumPy

2015-09-13 Thread Travis Oliphant
Hey all, I just wanted to clarify, that I am very excited about a few ideas I have --- but I don't have time myself to engage in the community process to get these changes into NumPy. However, those are real processes --- I've been coaching a few people in those processes for the past several

[Python-Dev] The changes I am planning to NumPy I'd like to make only available on Python 3

2015-09-12 Thread Travis Oliphant
If it helps anyone in their interest level. My intention would be to make these changes to NumPy only available on Python 3 as a way to help continue adoption of Python 3. -Travis -- *Travis Oliphant* *Co-founder and CEO* @teoliphant 512-222-5440 http://www.continuum.io

[Python-Dev] Looking for a developer who will work with me for at least 6 months to fix NumPy's dtype system.

2015-09-12 Thread Travis Oliphant
if I can't find someone I will publish the ideas --- but that also takes time and effort that is in short supply for me right now. If there is someone willing to fund this work, please let me know as well -- that could free up more of my time. Best, -Travis -- *Travis Oliphant* *Co-found

[Python-Dev] A new webpage promoting Compiler technology for CPython

2013-02-15 Thread Travis Oliphant
spreading, and so we welcome any and all contributions. Thank you, Travis Oliphant ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive

Re: [Python-Dev] Allocation of shape and strides fields in Py_buffer

2008-12-10 Thread Travis Oliphant
Antoine Pitrou wrote: Hello, The Py_buffer struct has two pointers named `shape` and `strides`. Each points to an array of Py_ssize_t values whose length is equal to the number of dimensions of the buffer object. Unfortunately, the buffer protocol spec doesn't explain how allocation of these

Re: [Python-Dev] Allocation of shape and strides fields in Py_buffer

2008-12-10 Thread Travis Oliphant
Alexander Belopolsky wrote: On Mon, Dec 8, 2008 at 6:25 PM, Antoine Pitrou [EMAIL PROTECTED] wrote: .. Alexander's suggestion of going and looking at what the numpy folks have done in this area is probably a good idea too. Well, I'm open to others doing this, but I won't do it myself. My

Re: [Python-Dev] Allocation of shape and strides fields in Py_buffer

2008-12-10 Thread Travis Oliphant
Antoine Pitrou wrote: Alexander Belopolsky alexander.belopolsky at gmail.com writes: I did not follow numpy development for the last year or more, so I won't qualify as the numpy folks, but my understanding is that numpy does exactly what Nick recommended: the viewed object owns shape and

Re: [Python-Dev] Allocation of shape and strides fields in Py_buffer

2008-12-10 Thread Travis Oliphant
Antoine Pitrou wrote: Nick Coghlan ncoghlan at gmail.com writes: For the slicing problem in particular, memoryview is currently trying to get away with only one Py_buffer object when it needs TWO. Why should it need two? Why couldn't the embedded Py_buffer fullfill all the needs of the

Re: [Python-Dev] Allocation of shape and strides fields in Py_buffer

2008-12-10 Thread Travis Oliphant
Greg Ewing wrote: Antoine Pitrou wrote: Why should it need two? Why couldn't the embedded Py_buffer fullfill all the needs of the memoryview object? Two things here: 1) The memoryview should *not* be holding onto a Py_buffer in between calls to its getitem and setitem methods. It

Re: [Python-Dev] Allocation of shape and strides fields in Py_buffer

2008-12-10 Thread Travis Oliphant
Nick Coghlan wrote: Antoine Pitrou wrote: In all honesty, I admit I am annoyed by all the problems with the buffer API / memoryview object, many of which are caused by its utterly bizarre design (and the fact that the design team went missing in action after imposing such a bizarre and complex

Re: [Python-Dev] Allocation of shape and strides fields in Py_buffer

2008-12-10 Thread Travis Oliphant
Antoine Pitrou wrote: Nick Coghlan ncoghlan at gmail.com writes: I don't see anything wrong with the PEP 3118 protocol. Apart from the fact that: - it uses something (Py_buffer) which is not a PyObject and has totally different allocation/lifetime semantics (which makes it non-trivial to

Re: [Python-Dev] Allocation of shape and strides fields in Py_buffer

2008-12-10 Thread Travis Oliphant
Greg Ewing wrote: Nick Coghlan wrote: Maintaining a PyDict instance to map from view pointers to shapes and strides info doesn't strike me as a complex scheme though. I don't see why a given buffer provider should ever need more than one set of shape/strides arrays at a time. It can allocate

Re: [Python-Dev] PEP 361: Python 2.6/3.0 release schedule

2008-03-18 Thread Travis Oliphant
, I've planned to back-port the improvements to the struct module and the addition of the memoryview object (both in PEP 3118). If you have questions, we can talk tomorrow. Best regards, -Travis Oliphant ___ Python-Dev mailing list Python-Dev

Re: [Python-Dev] 2.6 and 3.0 tasks

2008-03-16 Thread Travis Oliphant
Guido van Rossum wrote: Moving this to a new subject to keep the discussion of tasks and the discussion of task tracking tools separate. On Sun, Mar 16, 2008 at 9:42 AM, Christian Heimes [EMAIL PROTECTED] wrote: I did a quick brainstorming with me, myself and I. I came up with a list of

Re: [Python-Dev] Error in PEP3118?

2008-02-19 Thread Travis Oliphant
Lisandro Dalcin wrote: On 2/11/08, Travis Oliphant [EMAIL PROTECTED] wrote: My perception is that you are seeing too much of a connection between the C-compiler and the PEP description of memory. Perhaps that's not it, and I'm missing something else. Travis, all this make me believe

Re: [Python-Dev] Error in PEP3118?

2008-01-23 Thread Travis Oliphant
Thomas Heller wrote: Hi Travis, The pep contains this sample: Nested array :: struct { int ival; double data[16*4]; } i:ival: (16,4)d:data: I think it is wrong and must be changed to the following;

[Python-Dev] Patch for adding offset to mmap

2007-10-22 Thread Travis Oliphant
Hi all, I think the latest patch for fixing Issue 708374 (adding offset to mmap) should be committed to SVN. I will do it, if nobody opposes the plan. I think it is a very important addition and greatly increases the capability of the mmap module. Thanks, -Travis Oliphant P.S. Initially

Re: [Python-Dev] Google spreadsheet to collaborate on backporting Py3K stuff to 2.6

2007-09-06 Thread Travis Oliphant
Brett Cannon wrote: Neal, Anthony, Thomas W., and I have a spreadsheet that was started to keep track of what needs to be done in what needs to be done in 2.6 for Py3K transitioning: http://spreadsheets.google.com/pub?key=pCKY4oaXnT81FrGo3ShGHGg . I am opening the spreadsheet up to everyone

Re: [Python-Dev] PEP 3118: Extended buffer protocol (new version)

2007-04-19 Thread Travis Oliphant
Carl Banks wrote: Travis Oliphant wrote: Carl Banks wrote: Ok, I've thought quite a bit about this, and I have an idea that I think will be ok with you, and I'll be able to drop my main objection. It's not a big change, either. The key is to explicitly say whether the flag allows

Re: [Python-Dev] PEP 3118: Extended buffer protocol (new version)

2007-04-18 Thread Travis Oliphant
Greg Ewing wrote: Carl Banks wrote: Py_BUF_REQUIRE_READONLY - Raise excpetion if the buffer is writable. Is there a use case for this? Yes. The idea is used in NumPy all the time. Suppose you want to write to an array but only have an algorithm that works with contiguous data. Then you

Re: [Python-Dev] PEP 3118: Extended buffer protocol (new version)

2007-04-18 Thread Travis Oliphant
Carl Banks wrote: Ok, I've thought quite a bit about this, and I have an idea that I think will be ok with you, and I'll be able to drop my main objection. It's not a big change, either. The key is to explicitly say whether the flag allows or requires. But I made a few other changes as

Re: [Python-Dev] Extended Buffer Interface/Protocol

2007-04-15 Thread Travis Oliphant
Greg Ewing wrote: But since the NumPy object has to know about the provider, it can simply pass the release call on to it if appropriate. I don't see how this case necessitates making the release call on a different object. I'm -1 on involving any other objects or returning object

Re: [Python-Dev] Extended Buffer Interface/Protocol

2007-04-15 Thread Travis Oliphant
Carl Banks wrote: Tr ITSM that we are using the word view very differently. Consider this example: A = zeros((100,100)) B = A.transpose() You are thinking of NumPy's particular use case. I'm thinking of a generic use case. So, yes I'm using the word view in two different contexts.

[Python-Dev] Extended buffer PEP

2007-04-15 Thread Travis Oliphant
on the extensions to the struct module until the rest is approved. Thank you for any and all comments: -Travis :PEP: XXX :Title: Revising the buffer protocol :Version: $Revision: $ :Last-Modified: $Date: $ :Authors: Travis Oliphant [EMAIL PROTECTED], Carl Banks [EMAIL PROTECTED] :Status: Draft :Type

Re: [Python-Dev] Extended buffer PEP

2007-04-15 Thread Travis Oliphant
Greg Ewing wrote: Travis Oliphant wrote: Carl Banks wrote: I'd like to see it accept a flags argument over what kind of buffer it's allowed to return. I'd rather not burden the user to check all the entries in bufferinfo to make sure it doesn't get something unexpected. Yes, I

Re: [Python-Dev] PEP 3118: Extended buffer protocol (new version)

2007-04-13 Thread Travis Oliphant
Carl Banks wrote: The thing that bothers me about this whole flags setup is that different flags can do opposite things. Some of the flags RESTRICT the kind of buffers that can be exported (Py_BUF_WRITABLE); other flags EXPAND the kind of buffers that can be exported (Py_BUF_INDIRECT).

Re: [Python-Dev] PEP 3118: Extended buffer protocol (new version)

2007-04-12 Thread Travis Oliphant
Neil Hodgson wrote: Travis Oliphant: PEP: 3118 ... I'd like to see the PEP include discussion of what to do when an incompatible request is received while locked. Should there be a standard Can't do that: my buffer has been got exception? I'm not sure what standard to make a decision

Re: [Python-Dev] PEP 3118: Extended buffer protocol (new version)

2007-04-12 Thread Travis Oliphant
Lisandro Dalcin wrote: On 4/9/07, Travis Oliphant [EMAIL PROTECTED] wrote: Travis, all this is far better and simpler than previous approaches... Just a few comments Thanks for your wonderful suggestions. I've incorporated many of them. 1) I assume that 'bufferinfo' structure will be part

Re: [Python-Dev] PEP 3118: Extended buffer protocol (new version)

2007-04-11 Thread Travis Oliphant
Greg Ewing wrote: From PEP 3118: A memory-view object is an extended buffer object that should replace the buffer object in Python 3K. typedef struct { PyObject_HEAD PyObject *base; struct bufferinfo view; int itemsize; int flags; } PyMemoryViewObject; If

Re: [Python-Dev] Extended Buffer Protocol - simple use examples

2007-04-09 Thread Travis Oliphant
Paul Moore wrote: Hi, I'll admit right off that I haven't followed all of the extended buffer protocol discussions - I have no real need for anything much beyond the existing here's a blob of memory level of functionality. I have skimmed (briefly, I'll admit!) the pre-PEP, but I've found it

[Python-Dev] PEP 3118: Extended buffer protocol (new version)

2007-04-09 Thread Travis Oliphant
: Revising the buffer protocol Version: $Revision$ Last-Modified: $Date$ Authors: Travis Oliphant [EMAIL PROTECTED], Carl Banks [EMAIL PROTECTED] Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Aug-2006 Python-Version: 3000 Abstract This PEP proposes re-designing

Re: [Python-Dev] PEP 3118: Extended buffer protocol (new version)

2007-04-09 Thread Travis Oliphant
Carl Banks wrote: Travis Oliphant wrote: Py_BUF_READONLY The returned buffer must be readonly and the underlying object should make its memory readonly if that is possible. I don't like the if possible thing. If it makes no guarantees, it pretty much useless over

Re: [Python-Dev] Extended buffer PEP

2007-04-08 Thread Travis Oliphant
Carl Banks wrote: Only one concern: typedef int (*getbufferproc)(PyObject *obj, struct bufferinfo *view) I'd like to see it accept a flags argument over what kind of buffer it's allowed to return. I'd rather not burden the user to check all the entries in bufferinfo to make sure

Re: [Python-Dev] An updated extended buffer PEP

2007-03-27 Thread Travis Oliphant
Lisandro Dalcin wrote: On 3/26/07, Travis Oliphant [EMAIL PROTECTED] wrote: Here is my updated PEP which incorporates several parts of the discussions we have been having. Travis, it looks really good, below my comments I hope you don't mind me replying to python-dev. 1- Is it hard

Re: [Python-Dev] An updated extended buffer PEP

2007-03-27 Thread Travis Oliphant
Carl Banks wrote: Travis E. Oliphant wrote: I think we are getting closer. What do you think about Greg's idea of basically making the provider the bufferinfo structure and having the exporter handle copying memory over for shape and strides if it wants to be able to change those before

Re: [Python-Dev] Extended Buffer Interface/Protocol

2007-03-26 Thread Travis Oliphant
Carl Banks wrote: We're done. Return pointer. Thank you for this detailed example. I will have to parse it in more depth but I think I can see what you are suggesting. First, I'm not sure why getbuffer needs to return a view object. The view object in your case would just be the

[Python-Dev] An updated extended buffer PEP

2007-03-26 Thread Travis Oliphant
Hi Carl and Greg, Here is my updated PEP which incorporates several parts of the discussions we have been having. -Travis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

Re: [Python-Dev] Extended Buffer Interface/Protocol

2007-03-22 Thread Travis Oliphant
Greg Ewing wrote: Travis Oliphant wrote: I'm talking about arrays of pointers to other arrays: i.e. if somebody defined in C float B[10][20] then B would B an array of pointers to arrays of floats. No, it wouldn't, it would be a contiguously stored 2-dimensional array of floats

[Python-Dev] Extended Buffer Interface/Protocol

2007-03-21 Thread Travis Oliphant
every element of any object that exposes the buffer protocol. Thanks for any feedback, -Travis Oliphant ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman

Re: [Python-Dev] Extended Buffer Interface/Protocol

2007-03-21 Thread Travis Oliphant
Attached is the PEP. :PEP: XXX :Title: Revising the buffer protocol :Version: $Revision: $ :Last-Modified: $Date: $ :Author: Travis Oliphant [EMAIL PROTECTED] :Status: Draft :Type: Standards Track :Content-Type: text/x-rst :Created: 28-Aug-2006 :Python-Version: 3000 Abstract This PEP

Re: [Python-Dev] Extended Buffer Interface/Protocol

2007-03-21 Thread Travis Oliphant
Greg Ewing wrote: Travis Oliphant wrote: The question is should we eliminate the possibility of sharing memory for objects that store data basically as arrays of arrays (i.e. true C-style arrays). Can you clarify what you mean by this? Are you talking about an array of pointers

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-11-03 Thread Travis Oliphant
Perhaps the most relevant thing to pull from this conversation is back to what Martin has asked about before: flexible array members. A TCP packet has no defined length (there isn't even a header field in the packet for this, so in fairness we can talk about IP packets which do). There

Re: [Python-Dev] idea for data-type (data-format) PEP

2006-11-03 Thread Travis Oliphant
Martin v. Löwis wrote: Travis Oliphant schrieb: r_field = PyDict_GetItemString(dtype,'r'); Actually it should read PyDict_GetItemString(dtype-fields).The r_field is a tuple (data-type object, offset). The fields attribute is (currently) a Python dictionary. Ok

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-11-02 Thread Travis Oliphant
Martin v. Löwis wrote: Travis E. Oliphant schrieb: 2. Should primitive type codes be characters or integers (from an enum) at C level? - I prefer integers 3. Should size be expressed in bits or bytes? - I prefer bits So, you want an integer enum for the kind and an integer for the

Re: [Python-Dev] idea for data-type (data-format) PEP

2006-11-02 Thread Travis Oliphant
T IIUC, so far the 'data-object' carries information about the structure of the data it describes. Couldn't it go a step further and have also some functionality? Converting the data into a Python object and back? Yes, I had considered it to do that. That's why the setfunc and getfunc

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-11-01 Thread Travis Oliphant
Jim Jewett wrote: I'm still not sure exactly what is missing from ctypes. To make this concrete: I think the only thing missing from ctypes expressiveness as far as I can tell in terms of what you can do is the byte-order representation. What is missing is ease-of use for producers and

Re: [Python-Dev] idea for data-type (data-format) PEP

2006-11-01 Thread Travis Oliphant
Martin v. Löwis wrote: Travis E. Oliphant schrieb: Or, if it does have uses independent of the buffer extension: what are those uses? So that NumPy and ctypes and audio libraries and video libraries and database libraries and image-file format libraries can communicate about

Re: [Python-Dev] PEP: Extending the buffer protocol to share array information.

2006-11-01 Thread Travis Oliphant
Fredrik Lundh wrote: Chris Barker wrote: While /F suggested we get off the PIL bandwagon I suggest we drop the obsession with pointers to memory areas that are supposed to have a specific format; modern data access API:s don't work that way for good reasons, so I don't see why Python

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-11-01 Thread Travis Oliphant
Martin v. Löwis wrote: Travis E. Oliphant schrieb: 2) complex-valued types (you might argue that it's just a 2-array of floats, but you could say the same thing about int as an array of bytes). The point is how do people interpret the data. Complex-valued data-types are very common. It is

Re: [Python-Dev] PEP: Extending the buffer protocol to share array information.

2006-11-01 Thread Travis Oliphant
Fredrik Lundh wrote: Chris Barker wrote: While /F suggested we get off the PIL bandwagon I suggest we drop the obsession with pointers to memory areas that are supposed to have a specific format; modern data access API:s don't work that way for good reasons, so I don't see why Python

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-11-01 Thread Travis Oliphant
Alexander Belopolsky wrote: Travis Oliphant oliphant.travis at ieee.org writes: b = buffer(array('d', [1,2,3])) there is not much that I can do with b. For example, if I want to pass it to numpy, I will have to provide the type and shape information myself: numpy.ndarray(shape=(3

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-11-01 Thread Travis Oliphant
Paul Moore wrote: Enough of the abstract. As a concrete example, suppose I have a (byte) string in my program containing some binary data - an ID3 header, or a TCP packet, or whatever. It doesn't really matter. Does your proposal offer anything to me in how I might manipulate that data

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-10-31 Thread Travis Oliphant
Martin v. Löwis wrote: Travis Oliphant schrieb: So, the big difference is that I think data-formats should be *instances* of a single type. This is nearly the case for ctypes as well. All layout descriptions are instances of the type type. Nearly, because they are instances of subtypes

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-10-31 Thread Travis Oliphant
Nick Coghlan wrote: Travis E. Oliphant wrote: However, the existence of an alternative strategy using a single Python type and multiple instances of that type to describe binary data (which is the NumPy approach and essentially the array module approach) means that we can't just a-priori

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-10-31 Thread Travis Oliphant
Martin v. Löwis wrote: Travis E. Oliphant schrieb: But, there are distinct disadvantages to this approach compared to what I'm trying to allow. Martin claims that the ctypes approach is *basically* equivalent but this is just not true. I may claim that, but primarily, my goal was to

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-10-31 Thread Travis Oliphant
Thomas Heller wrote: (I tried to read the whole thread again, but it is too large already.) There is a (badly named, probably) api to access information about ctypes types and instances of this type. The functions are PyObject_stgdict(obj) and PyType_stgdict(type). Both return a

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-10-31 Thread Travis Oliphant
Martin v. Löwis wrote: Stephan Tolksdorf schrieb: While Travis' proposal encompasses the data format functionality within the struct module and overlaps with what ctypes has to offer, it does not aim to replace ctypes. This discussion could have been a lot shorter if he had said so.

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-10-31 Thread Travis Oliphant
Paul Moore wrote: On 10/31/06, Travis Oliphant [EMAIL PROTECTED] wrote: Martin v. Löwis wrote: [...] because I still don't quite understand what the PEP wants to achieve. Are you saying you still don't understand after having read the extended buffer protocol PEP, yet? I can't speak

Re: [Python-Dev] PEP: Extending the buffer protocol to share array information.

2006-10-31 Thread Travis Oliphant
Martin v. Löwis wrote: Travis E. Oliphant schrieb: Several extensions to Python utilize the buffer protocol to share the location of a data-buffer that is really an N-dimensional array. However, there is no standard way to exchange the additional N-dimensional array information

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-10-30 Thread Travis Oliphant
Martin v. Löwis wrote: Josiah Carlson schrieb: One could also toss wxPython, VTK, or any one of the other GUI libraries into the mix for visualizing those images, of which wxPython just acquired no-copy display of PIL images, and being able to manipulate them with numpy (of which some wxPython

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-10-30 Thread Travis Oliphant
Jim Jewett wrote: Travis E. Oliphant wrote: Two packages need to share a chunk of memory (the package authors do not know each other and only have and Python as a common reference). They both want to describe that the memory they are sharing has some underlying binary structure. As a

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-10-30 Thread Travis Oliphant
Greg Ewing wrote: Travis E. Oliphant wrote: Greg Ewing wrote: What exactly does bit mean in that context? Do you mean big ? No, you've got a data type there called bit, which seems to imply a size, in contradiction to the size-independent nature of the other types. I'm asking

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-10-30 Thread Travis Oliphant
Martin v. Löwis wrote: Robert Kern schrieb: As I unification mechanism, I think it is insufficient. I doubt it can express all the concepts that ctypes supports. What do you think is missing that can't be added? I can factually only report what is missing. Whether it can be added, I

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-10-30 Thread Travis Oliphant
Armin Rigo wrote: Hi Travis, On Fri, Oct 27, 2006 at 02:05:31PM -0600, Travis E. Oliphant wrote: This PEP proposes adapting the data-type objects from NumPy for inclusion in standard Python, to provide a consistent and standard way to discuss the format of binary data. How

Re: [Python-Dev] PEP: Adding data-type objects to Python

2006-10-30 Thread Travis Oliphant
Greg Ewing wrote: Travis Oliphant wrote: Part of the problem is that ctypes uses a lot of different Python types (that's what I mean by multi-object to accomplish it's goal). What I'm looking for is a single Python type that can be passed around and explains binary data. It's

Re: [Python-Dev] __index__ clipping

2006-08-10 Thread Travis Oliphant
Guido van Rossum wrote: What do you think (10**10).__index__() should return (when called from Python)? I'm with Guido on this point. I think (10**10).__index__() should return the full long integer when called from within Python. -Travis ___

Re: [Python-Dev] adding Construct to the standard library?

2006-04-18 Thread Travis Oliphant
-talk between all of us different users of the notion of what we in the NumPy community call a data-type might be useful. -Travis Oliphant ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe

Re: [Python-Dev] PEP 359: The make Statement

2006-04-13 Thread Travis Oliphant
Steven Bethard wrote: I know 2.5's not out yet, but since I now have a PEP number, I'm going to go ahead and post this for discussion. Currently, the target version is Python 2.6. You can also see the PEP at: http://www.python.org/dev/peps/pep-0359/ Thanks in advance for the feedback!

[Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c

2006-03-27 Thread Travis Oliphant
If you have Numeric or numpy installed try this: #import Numeric as N import numpy as N a = range(10) b = N.arange(10) a.__iadd__(b) print a Result: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9] Contrast the returned output with import numpy as N a = range(10) b =

Re: [Python-Dev] Expose the array interface in Python 2.5?

2006-03-17 Thread Travis Oliphant
Edward C. Jones wrote: Travis E. Oliphant [EMAIL PROTECTED] wrote: It is very important for many people to get access to memory with some description of how that memory should be interpreted as an array. Several Python packages could benefit if Python had some notion of an array

Re: [Python-Dev] Please comment on PEP 357 -- adding nb_index slot to PyNumberMethods

2006-02-17 Thread Travis Oliphant
Thomas Wouters wrote: On Fri, Feb 17, 2006 at 05:29:32PM +0100, Armin Rigo wrote: Where obj must be either an int or a long or another object that has the __index__ special method (but not self). The anything but not self rule is not consistent with any other special

Re: [Python-Dev] ssize_t branch merged

2006-02-17 Thread Travis Oliphant
Martin v. Löwis wrote: Just in case you haven't noticed, I just merged the ssize_t branch (PEP 353). If you have any corrections to the code to make which you would consider bug fixes, just go ahead. If you are uncertain how specific problems should be resolved, feel free to ask. If

Re: [Python-Dev] ssize_t branch merged

2006-02-17 Thread Travis Oliphant
Thomas Wouters wrote: On Fri, Feb 17, 2006 at 04:40:08PM -0700, Travis Oliphant wrote: What is PY_SSIZE_T_MAX supposed to be? The definition in pyport.h doesn't compile. Maybe I have the wrong version of code. In my pyport.h (checked out from svn trunk) I have. #define PY_SSIZE_T_MAX

[Python-Dev] PEP for adding an sq_index slot so that any object, a or b, can be used in X[a:b] notation

2006-02-09 Thread Travis Oliphant
: $Date: 2006/02/09 $ Author: Travis Oliphant oliphant at ee.byu.edu Status: Draft Type: Standards Track Created: 09-Feb-2006 Python-Version: 2.5 Abstract This PEP proposes adding an sq_index slot in PySequenceMethods and an __index__ special method so that arbitrary objects can be used

Re: [Python-Dev] PEP for adding an sq_index slot so that any object, a or b, can be used in X[a:b] notation

2006-02-09 Thread Travis Oliphant
Eric Nieuwland wrote: Travis Oliphant wrote: PEP: ### Title: Allowing any object to be used for slicing [...] Rationale Currently integers and long integers play a special role in slice notation in that they are the only objects allowed in slice syntax. In other words, if X

Re: [Python-Dev] Help with Unicode arrays in NumPy

2006-02-08 Thread Travis Oliphant
Thank you, Martin and Stephen, for the suggestions and comments. For your information: We decided that all NumPy arrays of unicode strings will use UCS4 for internal representation. When an element of the array is selected, a unicodescalar (which inherits directly from the unicode builtin

Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-24 Thread Travis Oliphant
Martin v. Löwis wrote: Travis Oliphant wrote: So, I now believe that his code (plus the array scalar extension type) was actually exposing a real bug in the memory manager itself. In theory, the Python memory manager should have been able to re-use the memory for the array-scalar

Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-24 Thread Travis Oliphant
Martin v. Löwis wrote: Travis Oliphant wrote: As verified by removing usage of the Python PyObject_MALLOC function, it was the Python memory manager that was performing poorly. Even though the array-scalar objects were deleted, the memory manager would not re-use their memory for later

Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-17 Thread Travis Oliphant
Bingo. Yes, definitely allocating new _types_ (an awful lot of them...) --- that's what the array scalars are: new types created in C. Do you really mean that someArray[1] will create a new type to represent the second element of someArray? I would guess that you create an instance of

Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-16 Thread Travis Oliphant
[EMAIL PROTECTED] wrote: Travis More to the point, however, these scalar objects were allocated Travis using the standard PyObject_New and PyObject_Del functions which Travis of course use the Python memory manager. One user ported his Travis (long-running) code to the new scipy

Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-16 Thread Travis Oliphant
Josiah Carlson wrote: Robert Kern [EMAIL PROTECTED] wrote: [1] There *is* an array type for general PyObjects in scipy_core, but that's not being used in the code that blows up and has nothing to do with the problem Travis is talking about. I seemed to have misunderstood the

Re: [Python-Dev] Problems with the Python Memory Manager

2005-11-16 Thread Travis Oliphant
Jim Jewett wrote: Do you have the code that caused problems? Yes. I was able to reproduce his trouble and was trying to debug it. The things I would check first are (1) Is he allocating (peak usage) a type (such as integers) that never gets returned to the free pool, in case you need more

[Python-Dev] Why does __getitem__ slot of builtin call sequence methods first?

2005-10-01 Thread Travis Oliphant
as a .__getitem__ wrapper method. Why does this wrapper call the sequence getitem instead of the mapping getitem method? Is there anyway to get at a mapping-style __getitem__ method from Python? This looks like a bug to me (which is why I'm posting here...) Thanks for any help or insight. -Travis

Re: [Python-Dev] Why does __getitem__ slot of builtin call sequence methods first?

2005-10-01 Thread Travis Oliphant
Guido van Rossum wrote: On 10/1/05, Travis Oliphant [EMAIL PROTECTED] wrote: The new ndarray object of scipy core (successor to Numeric Python) is a C extension type that has a getitem defined in both the as_mapping and the as_sequence structure. The as_sequence mapping is just so

Re: [Python-Dev] Is PEP 237 final -- Unifying Long Integers and Integers

2005-07-14 Thread Travis Oliphant
but this could be modified as desired. The code is available in the Numeric3 CVS tree at the numeric python sourceforge site. -Travis Oliphant ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe

Re: [Python-Dev] Pickling buffer objects.

2005-04-19 Thread Travis Oliphant
Greg Ewing wrote: Travis Oliphant wrote: I'm proposing to pickle the buffer object so that it unpickles as a string. Wouldn't this mean you're only solving half the problem? Unpickling a Numeric array this way would still use an intermediate string. Well, actually, unpickling in the new numeric

[Python-Dev] Pickling buffer objects.

2005-04-18 Thread Travis Oliphant
to be reworked into something more along the lines of the previously-proposed bytes object before a separate bytecode for pickleable mutable-bytes is accepted, however. -Travis Oliphant ___ Python-Dev mailing list Python-Dev@python.org http

[Python-Dev] 64-bit sequence and buffer protocol

2005-03-29 Thread Travis Oliphant
I'm posting to this list to again generate open discussion on the problem in current Python that an int is used in both the Python sequence protocol and the Python buffer protocol. The problem is that a C-int is typically only 4 bytes long while there are many applications (mmap for example),

[Python-Dev] Using descriptors to dynamically attach methods written in Python to C-defined (new-style) types

2005-03-25 Thread Travis Oliphant
. (Perhaps a PythonMethod descriptor object to complement the Method Descriptor). Any hints will be helpful. -Travis Oliphant ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http

[Python-Dev] Fixing _PyEval_SliceIndex so that integer-like objects can be used

2005-02-18 Thread Travis Oliphant
. Thanks, -Travis Oliphant ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Fix _PyEval_SliceIndex (Take two)

2005-02-18 Thread Travis Oliphant
. If this is acceptable, it is an easy patch. Thanks, -Travis Oliphant ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail

Re: [Python-Dev] Fixing _PyEval_SliceIndex so that integer-like objects can be used

2005-02-18 Thread Travis Oliphant
Guido van Rossum wrote: Would it be possible to change _PyEval_SliceIndex in ceval.c so that rather than throwing an error if the indexing object is not an integer, the code first checks to see if the object has a tp_as_number-nb_int method and calls it instead. I don't think this is the

[Python-Dev] Re: [Numpy-discussion] Re: Numeric life as I see it

2005-02-10 Thread Travis Oliphant
One question we are pursuing is could the arrayobject get into the core without a particular ufunc object. Most see this as sub-optimal, but maybe it is the only way. Since all the artithmetic operations are in ufunc that would be suboptimal solution, but indeed still a workable one. I

[Python-Dev] Clarification sought about including a multidimensional array object into Python core

2005-02-09 Thread Travis Oliphant
into the Python core. Thanks for your comments. I think they will help the discussion currently taking place. -Travis Oliphant ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http

Re: [Python-Dev] Clarification sought about including a multidimensional array object into Python core

2005-02-09 Thread Travis Oliphant
Martin v. Löwis wrote: Travis Oliphant wrote: I am a co-author of the current PEP regarding inclusion of the multidimensional array object into the core. However, that PEP is sorely outdated. [...] 1) What specifically about Numeric prevented it from being acceptable as an addition

Re: [Python-Dev] Clarification sought about including a multidimensional array object into Python core

2005-02-09 Thread Travis Oliphant
David Ascher wrote: I've not followed the num* discussion in quite a while, but my impression back then was that there wasn't one such community. Instead, the technical differences in the approaches required in specific fields, regarding things like the relative importance of memory profiles,

[Python-Dev] Re: Numeric life as I see it

2005-02-09 Thread Travis Oliphant
Martin v. Löwis wrote: The PEP should list the options, include criteria for selection, and then propose a choice. People can then discuss whether the list of options is complete (if not, you need to extend it), whether the criteria are agreed (they might be not, and there might be difficult

Re: [Numpy-discussion] Re: [Python-Dev] Re: Numeric life as I see it

2005-02-09 Thread Travis Oliphant
[Travis] I appreciate some of what Paul is saying here, but I'm not fully convinced that this is still true with Python 2.2 and up new-style c-types. The concerns seem to be over the fact that you have to re-implement everything in the sub-class because the base-class will always return one