Re: [Python-Dev] An updated extended buffer PEP
Travis Oliphant wrote:
> Travis Oliphant wrote:
>> Hi Carl and Greg,
>>
>> Here is my updated PEP which incorporates several parts of the
>> discussions we have been having.
>
> And here is the actual link:
>
> http://projects.scipy.org/scipy/numpy/browser/trunk/numpy/doc/pep_buffer.txt
What's the purpose of void** segments in PyObject_GetBuffer? It seems
like it's leftover from an older incarnation?
I'd hope after more recent discussion, we'll end up simplifying
releasebuffer. It seems like it'd be a nightmare to keep track of what
you've released.
Finally, the isptr thing. It's just not sufficient. Frankly, I'm
having doubts whether it's a good idea to support multibuffer at all.
Sure, it brings generality, but I'm thinking its too hard to explain and
too hard to get one's head around, and will lead to lots of
misunderstanding and bugginess. OTOH, it really doen't burden anyone
except those who want to export multi-buffered arrays, and we only have
one shot to do it. I just hope it doesn't confuse everyone so much that
no one bothers.
Here's how I would update the isptr thing. I've changed "derefoff" to
"subbufferoffsets" to describe it better.
typedef PyObject *(*getbufferproc)(PyObject *obj, void **buf,
Py_ssize_t *len, int *writeable,
char **format, int *ndims,
Py_ssize_t **shape,
Py_ssize_t **strides,
Py_ssize_t **subbufferoffsets);
subbufferoffsets
Used to export information about multibuffer arrays. It is an
address of a ``Py_ssize_t *`` variable that will be set to point at
an array of ``Py_ssize_t`` of length ``*ndims``.
[I don't even want to try a verbal description.]
To demonstrate how subbufferoffsets works, here is am example of a
function that returns a pointer to an element of ANY N-dimensional
array, single- or multi-buffered.
void* get_item_pointer(int ndim, void* buf, Py_ssize_t* strides,
Py_ssize_t* subarrayoffs, Py_ssize_t *indices) {
char* pointer = (char*)buf;
int i;
for (i = 0; i < ndim; i++) {
pointer += strides[i]*indices[i];
if (subarraysoffs[i] >= 0) {
pointer = *(char**)pointer + subarraysoffs[i];
}
}
return (void*)pointer;
}
For single buffers, subbufferoffsets is negative in every dimension
and it reduces to normal single-buffer indexing. For multi-buffers,
subbufferoffsets indicates when to dereference the pointer and switch
to the new buffer, and gives the offset into the buffer to start at.
In most cases, the subbufferoffset would be zero (indicating it should
start at the beginning of the new buffer), but can be a positive
number if the following dimension has been sliced, and thus the 0th
entry in that dimension would not be at the beginning of the new
buffer.
Other than that, looks good. :)
Carl Banks
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Standard Image and Sound classes (was: SoC proposal: multimedia library)
On Tuesday 27 March 2007 11:03, Lino Mastrodomenico wrote: > 2007/3/26, Pete Shinners <[EMAIL PROTECTED]>: > > My main question is what is the image and sound container > > passed back to Python? This single thing along would be worth a > > SoC if it could be implemented across all libraries. > > > > Will your image objects be transferrable to PIL, Pygame, > > PyOpengl, Numpy, Pythonmagick, Pycairo, wxPython, etc? It would > > be best if this could avoid the "fromstring/tostring" mojo that > > always requires and extra copy of the data for each transfer. > > I agree. I withdrew my original "multimedia library" idea and > submitted a proposal for the creation of two standard Image and > Sound classes. Ideally you'd hook this into the standard library's existing sound file handling modules. Anthony -- Anthony Baxter <[EMAIL PROTECTED]> It's never too late to have a happy childhood. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] HTTP responses and errors
> Why only 200 and 206?
This kind of question can often be answered through the revision
history. If you do 'svn annotate', you see that the line testing
for 206 was last changed in r36262. Comparing that to the
previous revision, you see that it before said
if r.status == 200:
and that amk changed it with the log message
[Bug #912845] urllib2 only checks for a 200 return code, but 206 is also
legal if a Range: header was supplied.
(Actually, should the first 'if' statement be modified to allow any 2xx
status code?)
Going to bugs.python.org/912845, you see that the current form was
proposed by Ahmed F. (oneofone), apparently because it crashed for
him with 206. In 2006, heresiarch ask the same question that you
are now asking and that amk asked before.
Going further back, you see that HTTPErrorProcessor (along with
the handling of 200) as added by jhylton in 34909, which in turn
originates from #852995, where jjlee introduced the handlers
in order to support cookie handling. Looking at the change, you
see that it is just refactoring; the special-casing of 200 was
present before.
In earlier versions, the error handling was done using this block:
14267jhylton if code == 200:
14267jhylton return addinfourl(fp, hdrs,
req.get_full_url())
14267jhylton else:
14267jhylton return self.parent.error('http', req, fp,
code, msg, hdrs)
You then find that 14267 is the initial revision, checked in with
the comment
# EXPERIMENTAL
#
# An extensible library for opening URLs using a variety protocols.
# Intended as a replacement for urllib.
So it seems that it only tests for 200 and 206 because the experiments
never produced a need for anything else.
Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] HTTP responses and errors
Martin v. Löwis wrote: >> Why only 200 and 206? > This kind of question can often be answered through the revision > history. If you do 'svn annotate', you see that the line testing > ... > So it seems that it only tests for 200 and 206 because the experiments > never produced a need for anything else. Thanks for this detailed explanation, I learned a lot of how to "discover" the history of a piece of code (didn't know about "annotate"). Regarding the codes themselves: As the tests for 200 and 206 came from just needing them, I think there's no reason to not include the rest of 200. So, in the base that the RFC says that "2xx" codes means that the request was succeded, I think we shouldn't raise an Exception. Right now, it's a bug. Do you think it's safe to fix this or will break much code? Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] HTTP responses and errors
On Tue, Mar 27, 2007 at 04:12:06PM +, Facundo Batista wrote: > (didn't know about "annotate"). It is also known under the name "blame"! ;) Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[EMAIL PROTECTED] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] HTTP responses and errors
> Right now, it's a bug. Do you think it's safe to fix this or will break > much code? Who am I to judge whether a fix will break much code? Personally, I think it should treat all 2xx responses as success. Callers can then still check the response code themselves if they need to. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] HTTP responses and errors
Oleg Broytmann schrieb: > On Tue, Mar 27, 2007 at 04:12:06PM +, Facundo Batista wrote: >> (didn't know about "annotate"). > >It is also known under the name "blame"! ;) Or "praise", depending on your mood :-) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] HTTP responses and errors
On Tue, Mar 27, 2007 at 07:14:35PM +0200, "Martin v. L?wis" wrote: > Oleg Broytmann schrieb: > > On Tue, Mar 27, 2007 at 04:12:06PM +, Facundo Batista wrote: > >> (didn't know about "annotate"). > > > >It is also known under the name "blame"! ;) > > Or "praise", depending on your mood :-) But "blame" is its official primary name! > svn praise blame (praise, annotate, ann): Output the content of specified files or URLs with revision and author information in-line. usage: blame [EMAIL PROTECTED] See? (-: Oleg. -- Oleg Broytmannhttp://phd.pp.ru/[EMAIL PROTECTED] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] HTTP responses and errors
Martin v. Löwis wrote: > Who am I to judge whether a fix will break much code? Personally, I Sorry, this was an error. I thought "you" as in plural (in spanish there're two different words for third person of plural and singular), and wrote it as is; now, re-reading the parragraph, it's confusing. So, you-people-in-the-list, do you think fix this will be a problem? > think it should treat all 2xx responses as success. Callers can > then still check the response code themselves if they need to. The same I think. If nobody has a conflic with this decission, I'll fix this. Thank you! :) -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] An updated extended buffer PEP
Greg Ewing wrote:
> Here's another idea, to allow multiple views of the same
> buffer with different shape/stride info to coexist, but
> without extra provider objects or refcount weirdness.
> Also it avoids using calls with a brazillion arguments.
>
>struct bufferinfo {
> void **buf;
> Py_ssize_t *len;
> int *writeable;
> char **format;
> int *ndims;
> Py_ssize_t **shape;
> Py_ssize_t **strides;
> int **isptr;
>};
>
>int (*getbuffer)(PyObject *obj, struct bufferinfo *info);
>
>int (*releasebuffer)(PyObject *obj, struct bufferinfo *info);
>
This is not much different from my original "view" object. Stick a
PyObject_HEAD at the start of this bufferinfo and you have it.
Memory management was the big reason I wanted to do something like this.
I don't see why a PyObject_HEAD would make anything significantly
slower. Then we could use Python's memory management very easily to
create and destroy these things. This bufferinfo object would become
the "provider" I was talking about.
> If the object has constant shape/stride info, it just fills
> in the info struct with pointers to its own memory, and does
> nothing when releasebuffer is called (other than unlocking
> its buffer).
>
> If its shape/stride info can change, it mallocs memory for
> them and copies them into the info struct. When releasebuffer
> is called, it frees this memory.
>
> It is the responsibility of the consumer to ensure that the
> base object remains alive until releasebuffer has been called
> on the info struct (to avoid leaking any memory that has
> been malloced for shapes/strides).
This is a reasonable design choice. I actually prefer to place all the
buffer information in a single object rather than the multiple argument
design because it scales better and is easier to explain and understand.
-Travis
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] An updated extended buffer PEP
Carl Banks wrote:
> Travis Oliphant wrote:
>> Travis Oliphant wrote:
>>> Hi Carl and Greg,
>>>
>>> Here is my updated PEP which incorporates several parts of the
>>> discussions we have been having.
>> And here is the actual link:
>>
>> http://projects.scipy.org/scipy/numpy/browser/trunk/numpy/doc/pep_buffer.txt
>
>
> What's the purpose of void** segments in PyObject_GetBuffer? It seems
> like it's leftover from an older incarnation?
>
Yeah, I forgot to change that location.
> I'd hope after more recent discussion, we'll end up simplifying
> releasebuffer. It seems like it'd be a nightmare to keep track of what
> you've released.
Yeah, I agree. I think I'm leaning toward the bufferinfo structure
which allows the exporter to copy memory for things that it wants to be
free to change while the buffer is exported.
>
>
> Finally, the isptr thing. It's just not sufficient. Frankly, I'm
> having doubts whether it's a good idea to support multibuffer at all.
> Sure, it brings generality, but I'm thinking its too hard to explain and
> too hard to get one's head around, and will lead to lots of
> misunderstanding and bugginess. OTOH, it really doen't burden anyone
> except those who want to export multi-buffered arrays, and we only have
> one shot to do it. I just hope it doesn't confuse everyone so much that
> no one bothers.
People used to have doubts about explaining strides in NumPy as well. I
sure would have hated to see them eliminate the possiblity because of
those doubts. I think the addition you discuss is not difficult once you
get a hold of it.
I also understand now why subbufferoffsets is needed. I was thinking
that for slices you would just re-create a whole other array of pointers
to contain that addition. But, that is really not advisable. It makes
sense when you are talking about a single pointer variable (like in
NumPy) but it doesn't when you have an array of pointers.
Providing the example about how to extract the pointer from the returned
information goes a long way towards clearing up any remaining confusion.
Your ImageObject example is also helpful. I really like the addition
and think it is clear enough and supports a lot of use cases with very
little effort.
>
> Here's how I would update the isptr thing. I've changed "derefoff" to
> "subbufferoffsets" to describe it better.
>
>
> typedef PyObject *(*getbufferproc)(PyObject *obj, void **buf,
> Py_ssize_t *len, int *writeable,
> char **format, int *ndims,
> Py_ssize_t **shape,
> Py_ssize_t **strides,
> Py_ssize_t **subbufferoffsets);
>
>
> subbufferoffsets
>
>Used to export information about multibuffer arrays. It is an
>address of a ``Py_ssize_t *`` variable that will be set to point at
>an array of ``Py_ssize_t`` of length ``*ndims``.
>
>[I don't even want to try a verbal description.]
>
>To demonstrate how subbufferoffsets works, here is am example of a
>function that returns a pointer to an element of ANY N-dimensional
>array, single- or multi-buffered.
>
> void* get_item_pointer(int ndim, void* buf, Py_ssize_t* strides,
> Py_ssize_t* subarrayoffs, Py_ssize_t *indices) {
> char* pointer = (char*)buf;
> int i;
> for (i = 0; i < ndim; i++) {
> pointer += strides[i]*indices[i];
> if (subarraysoffs[i] >= 0) {
> pointer = *(char**)pointer + subarraysoffs[i];
> }
> }
> return (void*)pointer;
> }
>
>For single buffers, subbufferoffsets is negative in every dimension
>and it reduces to normal single-buffer indexing.
What about just having subbufferoffsets be NULL in this case? i.e. you
don't need it.If some of the dimensions did not need dereferencing
then they would be negative (how about we just say -1 to be explicit)?
>For multi-buffers,
>subbufferoffsets indicates when to dereference the pointer and switch
>to the new buffer, and gives the offset into the buffer to start at.
>In most cases, the subbufferoffset would be zero (indicating it should
>start at the beginning of the new buffer), but can be a positive
>number if the following dimension has been sliced, and thus the 0th
>entry in that dimension would not be at the beginning of the new
>buffer.
>
>
>
> Other than that, looks good. :)
>
I think we are getting closer. What do you think about Greg's idea of
basically making the provider the bufferinfo structure and having the
exporter handle copying memory over for shape and strides if it wants to
be able to change those before the lock is released.
-Travis
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailm
Re: [Python-Dev] HTTP responses and errors
On Tue, Mar 27, 2007, Facundo Batista wrote: > > Sorry, this was an error. I thought "you" as in plural (in spanish > there're two different words for third person of plural and singular), > and wrote it as is; now, re-reading the parragraph, it's confusing. > > So, you-people-in-the-list, do you think fix this will be a problem? The proper English word for plural "you" is "y'all". ;-) Except for "all y'all". Isn't English fun? -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ Need a book? Use your library! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PyPy 1.0: JIT compilers for free and more
Hi all, Sorry for the spamming. I hope this will be of interest to some of you. Armin == PyPy 1.0: JIT compilers for free and more == Welcome to the PyPy 1.0 release - a milestone integrating the results of four years of research, engineering, management and sprinting efforts, concluding the 28 months phase of EU co-funding! Although still not mature enough for general use, PyPy 1.0 materializes for the first time the full extent of our original vision: - A flexible Python interpreter, written in "RPython": - Mostly unaware of threading, memory and lower-level target platform aspects. - Showcasing advanced interpreter features and prototypes. - Passing core CPython regression tests, translatable to C, LLVM and .NET. - An advanced framework to translate such interpreters and programs: - That performs whole type-inference on RPython programs. - Can weave in threading, memory and target platform aspects. - Has low level (C, LLVM) and high level (CLI, Java, JavaScript) backends. - A **Just-In-Time Compiler generator** able to **automatically** enhance the low level versions of our Python interpreter, leading to run-time machine code that runs algorithmic examples at speeds typical of JITs! Previous releases, particularly the 0.99.0 release from February, already highlighted features of our Python implementation and the abilities of our translation approach but the **new JIT generator** clearly marks a major research result and gives weight to our vision that one can generate efficient interpreter implementations, starting from a description in a high level language. We have prepared several entry points to help you get started: * The main entry point for JIT documentation and status: http://codespeak.net/pypy/dist/pypy/doc/jit.html * The main documentation and getting-started PyPy entry point: http://codespeak.net/pypy/dist/pypy/doc/index.html * Our online "play1" demos showcasing various Python interpreters, features (and a new way to program AJAX applications): http://play1.codespeak.net/ * Our detailed and in-depth Reports about various aspects of the project: http://codespeak.net/pypy/dist/pypy/doc/index-report.html In the next few months we are going to discuss the goals and form of the next stage of development - now more than ever depending on your feedback and contributions - and we hope you appreciate PyPy 1.0 as an interesting basis for greater things to come, as much as we do ourselves! have fun, the PyPy release team, Samuele Pedroni, Armin Rigo, Holger Krekel, Michael Hudson, Carl Friedrich Bolz, Antonio Cuni, Anders Chrigstroem, Guido Wesdorp Maciej Fijalkowski, Alexandre Fayolle and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html What is PyPy? Technically, PyPy is both a Python interpreter implementation and an advanced compiler, or more precisely a framework for implementing dynamic languages and generating virtual machines for them. The framework allows for alternative frontends and for alternative backends, currently C, LLVM and .NET. For our main target "C", we can can "mix in" different garbage collectors and threading models, including micro-threads aka "Stackless". The inherent complexity that arises from this ambitious approach is mostly kept away from the Python interpreter implementation, our main frontend. PyPy is now also a Just-In-Time compiler generator. The translation framework contains the now-integrated JIT generation technology. This depends only on a few hints added to the interpreter source and should be able to cope with the changes to the interpreter and be generally applicable to other interpreters written using the framework. Socially, PyPy is a collaborative effort of many individuals working together in a distributed and sprint-driven way since 2003. PyPy would not have gotten as far as it has without the coding, feedback and general support from numerous people. Formally, many of the current developers were involved in executing an EU contract with the goal of exploring and researching new approaches to language and compiler development and software engineering. This contract's duration is about to end this month (March 2007) and we are working and preparing the according final review which is scheduled for May 2007. For the future, we are in the process of setting up structures to help maintain conceptual integrity of the project and to discuss and deal with funding opportunities related to further PyPy sprinting and developments. See here for results of the discussion so far: http://codespeak.net/pipermail/pypy-dev/2007q1/003577.html 1.0.0 Feature highlights == Here is a summary list of key features included in PyPy 1.0: - The Just-In-Time compiler generator, now capable of generating the fir
Re: [Python-Dev] PyPy 1.0: JIT compilers for free and more
On Tue, Mar 27, 2007, Armin Rigo wrote: > > Sorry for the spamming. I hope this will be of interest to some of you. This is not spamming, this is wonderful news! Congratulations! -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ Need a book? Use your library! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] An updated extended buffer PEP
Lisandro Dalcin wrote: > On 3/26/07, Travis Oliphant <[EMAIL PROTECTED]> wrote: >> Here is my updated PEP which incorporates several parts of the >> discussions we have been having. > > Travis, it looks really good, below my comments I hope you don't mind me replying to python-dev. > > 1- Is it hard to EXTEND PyBufferProcs in order to be able to use all > this machinery in Py 2.X series, not having to wait until Py3k? No, I don't think it will be hard. I just wanted to focus on Py3k since it is going to happen before Python 2.6 and I wanted it discussed in that world. > > 2- Its not clear for me if this PEP will enable object types defined > in the Python side to export buffer info. This is a feature I really > like in numpy, and simplifies my life a lot when I need to export > memory for C/C++ object wrapped with the help of tools like SWIG. This PEP does not address that. You will have to rely on the objects themselves for any such information. > > 3- Why not to constraint the returned 'view' object to be of a > specific type defined in the C side (and perhaps available in the > Python side)? This 'view' object could maintain a reference to the > base object containing the data, could call releasebuffer using the > base object when the view object is decref'ed, and can have a flag > field for think like OWN_MEMORY, OWN_SHAPE, etc in order to properly > manage memory deallocation. Does all this make sense? Yes, that was my original thinking and we are kind of coming back to it after several iterations. Perhaps, though we can stick with an object-less buffer interface but have this "view object" as an expanded buffer object. -Travis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] An updated extended buffer PEP
Travis E. Oliphant wrote:
> Greg Ewing wrote:
>> struct bufferinfo {
>> ...
>> };
>>
>> int (*getbuffer)(PyObject *obj, struct bufferinfo *info);
>> int (*releasebuffer)(PyObject *obj, struct bufferinfo *info);
> This is not much different from my original "view" object. Stick a
> PyObject_HEAD at the start of this bufferinfo and you have it.
The important difference is that it *doesn't* have
PyObject_HEAD at the start of it. :-)
> I don't see why a PyObject_HEAD would make anything significantly
> slower. Then we could use Python's memory management very easily to
> create and destroy these things.
In the case where the shape/stride info is constant, and
the caller is able to allocate the struct bufferinfo on
the stack, my proposal requires no memory allocations at
all. That's got to be faster than allocating and freeing
a Python object.
When it is necessary to allocate memory for the shape/stride,
some mallocs and frees (or Python equivalents) are going to
be needed either way. I don't see how using a Python object
makes this any easier.
--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] An updated extended buffer PEP
Travis Oliphant wrote: > Perhaps, though we can stick with an > object-less buffer interface but have this "view object" as an expanded > buffer object. I like this idea. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Fwd: PEP 0305 (small problem with the CSV reader)]
>First of all, let me say thank you for the CSV module. Thanks. >I've been using it and today is the first time I hit a minor bump in the road. >What happened is I opened this file with genome annotations with a >long field and the error "field larger than field limit" showed up. >From what I can see it is in the "static int parse_add_char(ReaderObj >*self, char c)" function. >This function uses the static long field_limit = 128 * 1024; /* max >parsed field size */ >I'm not sure if this is supposed to be recomputed or if there is >something I need to do to change it, but for right now it just says my >row is bigger than 131,072 and stops. >I don't think Python 2.5 has any such string length limitations and >this shouldn't be. This limit was added back in January 2005 to provide some protection against the situation where the parser is returning fields directly from a file, and the file contains a mismatched quote character: this would otherwise result in the entire file being unexpectedly read into memory. You can change the limit with the csv.field_size_limit() method. As you note, it defaults to 128K, but you can set it to anything up to (2**31)-1 or 2147483647 (about 2 billion). BTW, I've taken the liberty of CC'ing this to the python-dev list, so the motivation for this feature is recorded - it caused me some head scratching, and I added it. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Standard Image and Sound classes (was: SoC proposal: multimedia library)
2007/3/27, Anthony Baxter <[EMAIL PROTECTED]>: > On Tuesday 27 March 2007 11:03, Lino Mastrodomenico wrote: > > I agree. I withdrew my original "multimedia library" idea and > > submitted a proposal for the creation of two standard Image and > > Sound classes. > > Ideally you'd hook this into the standard library's existing sound > file handling modules. Yes, in my SoC application I listed the following stdlib modules (for both sound and images): Tkinter, Tix, wave, ossaudiodev, winsound, aifc, audioop, sunau, sunaudiodev and imageop. There are also a few Mac modules that can probably use the new classes: videoreader, PixMapWrapper and some Carbon modules. IRIX goodness: al, imgfile, jpeg and gl, but I can't test these. BTW, will they be still here in Python 3.0? -- Lino Mastrodomenico E-mail: [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Standard Image and Sound classes (was: SoC proposal: multimedia library)
2007/3/28, Lino Mastrodomenico <[EMAIL PROTECTED]>: > IRIX goodness: al, imgfile, jpeg and gl, but I can't test these. BTW, > will they be still here in Python 3.0? Ok, I found the answer: PEP 3108. Sorry for the noise. -- Lino Mastrodomenico E-mail: [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyPy 1.0: JIT compilers for free and more
Aahz wrote: > On Tue, Mar 27, 2007, Armin Rigo wrote: >> Sorry for the spamming. I hope this will be of interest to some of you. > > This is not spamming, this is wonderful news! Congratulations! Second the congrats! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [ 1688393 ] sock.recvfrom(-24) crashes
I applied the patch in this bug to the trunk. As it's a bug, and a very nasty one (it causes an ugly crash), please consider backporting it to 2.5.x. If you apply this to 2.5.x, just close the bug. Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] An updated extended buffer PEP
Carl Banks wrote: > Travis E. Oliphant wrote: >> I think we are getting closer. What do you think about Greg's idea >> of basically making the provider the bufferinfo structure and having >> the exporter handle copying memory over for shape and strides if it >> wants to be able to change those before the lock is released. > > It seems like it's just a different way to return the data. You could > do it by setting values through pointers, or do it by returning a > structure. Which way you choose is a minor detail in my opinion. I'd > probably favor returning the information in a structure. > > I would consider adding two fields to the structure: > > size_t structsize; /* size of the structure */ Why is this necessary? can't you get that by sizeof(bufferinfo)? > PyObject* releaser; /* the object you need to call releasebuffer on */ Is this so that another object could be used to manage releases if desired? -Travis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] The latest extended buffer PEP
The latest update is here. Carl and Greg, can I add your names to the PEP author list? I think we are very close. I'd like to start working on the implmentation. The modifications to the struct module is probably where I'll start. I really like the possibilities this will open up for sharing of video, images, audio, databases, between different objects. Algorithms could be written that are object agnostic and work for any object exporting the buffer interface. Are we ready for a pronouncement? -Travis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
