Re: [Python-Dev] PEP: Adding data-type objects to Python
Travis E. Oliphant wrote: > How to handle unicode data-formats could definitely be improved. > Suggestions are welcome. 'U4*10' string of 10 4-byte Unicode chars Then for consistency you'd want 'S*10' rather than just 'S10' (or at least allow it as an alternative). -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Adding data-type objects to Python
Travis E. Oliphant wrote: > The 'kind' does not specify how "big" the data-type (data-format) is. What exactly does "bit" mean in that context? -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Adding data-type objects to Python
Nick Coghlan wrote: > I'd say the answer to where we put it will be dependent on what happens to > the > idea of adding a NumArray style fixed dimension array type to the standard > library. If that gets exposed through the array module as array.dimarray, > then > it would make sense to expose the associated data layout descriptors as > array.datatype. Seem to me that arrays are a sub-concept of binary data, not the other way around. So maybe both arrays and data types should be in a module called 'binary' or some such. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Adding data-type objects to Python
Nick Coghlan wrote: > Greg Ewing wrote: >> Also, what if I want to refer to fields by name >> but don't want to have to work out all the offsets > Use the list definition form. With the changes I've > suggested above, you wouldn't even have to name the fields you don't > care about - just describe them. That would be okay. I still don't see a strong justification for having a one-big-string form as well as a list/tuple/dict form, though. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Adding data-type objects to Python
Martin v. Löwis wrote: > Travis E. Oliphant schrieb: >> In this case, the 'kind' does not specify how large the data-type is. >> You can have 'u1', 'u2', 'u4', etc. >> >> The same is true with Unicode. You can have 10-character unicode >> elements, 20-character, etc. But, we have to be clear about what a >> "character" is in the data-format. > > That is certainly confusing. In u1, u2, u4, the digit seems to indicate > the size of a single value (1 byte, 2 bytes, 4 bytes). Right? Yet, > in U20, it does *not* indicate the size of a single value but of an > array? And then, it's not the size, but the number of elements? > Good point. In NumPy, unicode support was added "in parallel" with string arrays where there is not the ambiguity. So, yes, it's true that the unicode case is a special-case. The other way to handle it would be to describe the 'code'-point size (i.e. 'U1', 'U2', 'U4' for UCS-1, UCS-2, UCS-4) and then have the length be encoded as an "array" of those types. This was not the direction we took with NumPy (which is what I'm using as a reference) because I wanted Unicode and string arrays to look the same and thought of strings differently. How to handle unicode data-formats could definitely be improved. Suggestions are welcome. -Travis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] build bots, log output
Martin v. Löwis wrote: > Georg Brandl schrieb: >> I wonder if it's possible that the build bot notification mails that go >> to python-checkins include the last 10-15 lines from the log. This would >> make it much easier to decide whether a buildbot failure is an old, >> esoteric one (e.g. > > It should be possible to implement that. To do so, one would have to > modify the source of the buildbot master. I'd volunteer to do it if I knew where the source of the buildbot master can be found :) Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 355 status
BJörn Lindqvist wrote: > I'd like to write a post mortem for PEP 355. But one important > question that haven't been answered is if there is a possibility for a > path-like PEP to succeed in the future? If so, does the path-object > implementation have to prove itself in the wild before it can be > included in Python? From earlier posts it seems like you don't like > the concept of path objects, which others have found very interesting. > If that is the case, then it would be nice to hear it explicitly. :) So...how's that post mortem coming along? Did you get a sufficient answer to your questions? And the more interesting question is, will the effort to reform Python's path functionality continue? From reading all the responses to your post, I feel that the community is on the whole supportive of the idea of refactoring os.path and friends, but they prefer a different approach; And several of the responses sketch out some suggestions for what that approach might be. So what happens next? -- Talin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Adding data-type objects to Python
Travis E. Oliphant wrote: > M.-A. Lemburg wrote: >> Travis E. Oliphant wrote: >>> M.-A. Lemburg wrote: Travis E. Oliphant wrote: > > > PEP: > Title: Adding data-type objects to the standard library > Attributes > > kind -- returns the basic "kind" of the data-type. The basic > kinds > are: >'t' - bit, >'b' - bool, >'i' - signed integer, >'u' - unsigned integer, >'f' - floating point, >'c' - complex floating point, >'S' - string (fixed-length sequence of char), >'U' - fixed length sequence of UCS4, Shouldn't this read "fixed length sequence of Unicode" ?! The underlying code unit format (UCS2 and UCS4) depends on the Python version. >>> Well, in NumPy 'U' always means UCS4. So, I just copied that over. See >>> my questions at the bottom which talk about how to handle this. A >>> data-format does not necessarily have to correspond to something Python >>> represents with an Object. >> Ok, but why are you being specific about UCS4 (which is an internal >> storage format), while you are not specific about e.g. the >> internal bit size of the integers (which could be 32 or 64 bit) ? >> > > The 'kind' does not specify how "big" the data-type (data-format) is. A > number is needed to represent the number of bytes. > > In this case, the 'kind' does not specify how large the data-type is. > You can have 'u1', 'u2', 'u4', etc. > > The same is true with Unicode. You can have 10-character unicode > elements, 20-character, etc. But, we have to be clear about what a > "character" is in the data-format. I understand and that's why I'm asking why you made the range explicit in the definition. The definition should talk about Unicode code points. The number of bytes then determines whether you can only represent the ASCII subset (1 byte), UCS2 (2 bytes, BMP only) or UCS4 (4 bytes, all currently assigned code points). This is similar to the range for integers (ie. ZZ_0), where the number of bytes determines the range of numbers that can be represented. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 28 2006) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] build bots, log output
Georg Brandl schrieb: > I wonder if it's possible that the build bot notification mails that go > to python-checkins include the last 10-15 lines from the log. This would > make it much easier to decide whether a buildbot failure is an old, > esoteric one (e.g. It should be possible to implement that. To do so, one would have to modify the source of the buildbot master. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Adding data-type objects to Python
Travis E. Oliphant schrieb: > In this case, the 'kind' does not specify how large the data-type is. > You can have 'u1', 'u2', 'u4', etc. > > The same is true with Unicode. You can have 10-character unicode > elements, 20-character, etc. But, we have to be clear about what a > "character" is in the data-format. That is certainly confusing. In u1, u2, u4, the digit seems to indicate the size of a single value (1 byte, 2 bytes, 4 bytes). Right? Yet, in U20, it does *not* indicate the size of a single value but of an array? And then, it's not the size, but the number of elements? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Adding data-type objects to Python
Armin Rigo wrote: > Hi Travis, > > On Fri, Oct 27, 2006 at 02:05:31PM -0600, Travis E. Oliphant wrote: >> This PEP proposes adapting the data-type objects from NumPy for >> inclusion in standard Python, to provide a consistent and standard >> way to discuss the format of binary data. > > How does this compare with ctypes? Do we really need yet another, > incompatible way to describe C-like data structures in the standard > library? Part of what the data-type, data-format object is trying to do is bring together all the disparate ways to represent data that *already* exists in the standard library. What is needed is a definitive way to describe data and then have array struct ctypes all be compatible with that same method.That's why I'm proposing the PEP. It's a unification effort not yet-another-method. One of the big reasons for it is to move something like the array interface into Python. There are tens to hundreds of people mostly in the scientific computing community that want to see Python grow more support for NumPy-like things. I keep getting requests to "do something" to make Python more aware of arrays. This PEP is part of that effort. In particular, something like the array interface should be available in Python. The easiest way to do this is to extend the buffer protocol to allow objects to share information about shape, strides, and data-format of a block of memory. But, how do you represent data-format in Python? What will the objects pass back and forth to each other to do it? C-types has a solution which creates multiple objects to do it. This is an un-wieldy over-complicated solution for the array interface. The array objects have a solution using the a single object that carries the data-format information. The solution we have for arrays deserves consideration. It could be placed inside the array module if desired, but again, I'm really looking for something that would allow the extend buffer protocol (to be proposed soon) to share data-type information. That could be done with the array-interface objects (strings, lists, and tuples), but then every body who uses the interface will have to write their own "decoders" to process the data-format information. I actually think ctypes would benefit from this data-format specification too. Recognizing all these diverging ways to essentially talk about the same thing is part of what prompted this PEP. -Travis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Adding data-type objects to Python
M.-A. Lemburg wrote: > Travis E. Oliphant wrote: >> M.-A. Lemburg wrote: >>> Travis E. Oliphant wrote: PEP: Title: Adding data-type objects to the standard library Attributes kind -- returns the basic "kind" of the data-type. The basic kinds are: 't' - bit, 'b' - bool, 'i' - signed integer, 'u' - unsigned integer, 'f' - floating point, 'c' - complex floating point, 'S' - string (fixed-length sequence of char), 'U' - fixed length sequence of UCS4, >>> Shouldn't this read "fixed length sequence of Unicode" ?! >>> The underlying code unit format (UCS2 and UCS4) depends on the >>> Python version. >> Well, in NumPy 'U' always means UCS4. So, I just copied that over. See >> my questions at the bottom which talk about how to handle this. A >> data-format does not necessarily have to correspond to something Python >> represents with an Object. > > Ok, but why are you being specific about UCS4 (which is an internal > storage format), while you are not specific about e.g. the > internal bit size of the integers (which could be 32 or 64 bit) ? > The 'kind' does not specify how "big" the data-type (data-format) is. A number is needed to represent the number of bytes. In this case, the 'kind' does not specify how large the data-type is. You can have 'u1', 'u2', 'u4', etc. The same is true with Unicode. You can have 10-character unicode elements, 20-character, etc. But, we have to be clear about what a "character" is in the data-format. -Travis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Adding data-type objects to Python
Josiah Carlson wrote: > I think that even on 64 bit platforms, using 'int' or 'long' generally > means 32 bit. In order to get 64 bit ints, one needs to use 'long long'. real 64-bit platforms use the LP64 standard, where long and pointers are both 64 bits: http://www.unix.org/version2/whatsnew/lp64_wp.html ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Adding data-type objects to Python
"M.-A. Lemburg" <[EMAIL PROTECTED]> wrote: > > Travis E. Oliphant wrote: > > M.-A. Lemburg wrote: > >> Travis E. Oliphant wrote: > >>> > >>> > >>> PEP: > >>> Title: Adding data-type objects to the standard library > >>> Attributes > >>> > >>> kind -- returns the basic "kind" of the data-type. The basic > >>> kinds > >>> are: > >>>'t' - bit, > >>>'b' - bool, > >>>'i' - signed integer, > >>>'u' - unsigned integer, > >>>'f' - floating point, > >>>'c' - complex floating point, > >>>'S' - string (fixed-length sequence of char), > >>>'U' - fixed length sequence of UCS4, > >> Shouldn't this read "fixed length sequence of Unicode" ?! > >> The underlying code unit format (UCS2 and UCS4) depends on the > >> Python version. > > > > Well, in NumPy 'U' always means UCS4. So, I just copied that over. See > > my questions at the bottom which talk about how to handle this. A > > data-format does not necessarily have to correspond to something Python > > represents with an Object. > > Ok, but why are you being specific about UCS4 (which is an internal > storage format), while you are not specific about e.g. the > internal bit size of the integers (which could be 32 or 64 bit) ? I think that even on 64 bit platforms, using 'int' or 'long' generally means 32 bit. In order to get 64 bit ints, one needs to use 'long long'. Sharing some of the codes with the struct module, though arbitrary, doesn't seem like a bad idea to me. Of course offering specifically 32 and 64 bit ints would make sense to me. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Adding data-type objects to Python
Travis E. Oliphant wrote: > M.-A. Lemburg wrote: >> Travis E. Oliphant wrote: >>> >>> >>> PEP: >>> Title: Adding data-type objects to the standard library >>> Attributes >>> >>> kind -- returns the basic "kind" of the data-type. The basic >>> kinds >>> are: >>>'t' - bit, >>>'b' - bool, >>>'i' - signed integer, >>>'u' - unsigned integer, >>>'f' - floating point, >>>'c' - complex floating point, >>>'S' - string (fixed-length sequence of char), >>>'U' - fixed length sequence of UCS4, >> Shouldn't this read "fixed length sequence of Unicode" ?! >> The underlying code unit format (UCS2 and UCS4) depends on the >> Python version. > > Well, in NumPy 'U' always means UCS4. So, I just copied that over. See > my questions at the bottom which talk about how to handle this. A > data-format does not necessarily have to correspond to something Python > represents with an Object. Ok, but why are you being specific about UCS4 (which is an internal storage format), while you are not specific about e.g. the internal bit size of the integers (which could be 32 or 64 bit) ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 28 2006) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Adding data-type objects to Python
Hi Travis, On Fri, Oct 27, 2006 at 02:05:31PM -0600, Travis E. Oliphant wrote: > This PEP proposes adapting the data-type objects from NumPy for > inclusion in standard Python, to provide a consistent and standard > way to discuss the format of binary data. How does this compare with ctypes? Do we really need yet another, incompatible way to describe C-like data structures in the standard library? A bientot, Armin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] build bots, log output
Hi, I wonder if it's possible that the build bot notification mails that go to python-checkins include the last 10-15 lines from the log. This would make it much easier to decide whether a buildbot failure is an old, esoteric one (e.g. test_wait4 sem_post: Success make: *** [buildbottest] Killed on the hppa one) or a new one, really caused by one's checkin. The alternative would be to fix the tests/buildbots not to have these esoteric failures anymore Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Adding data-type objects to Python
M.-A. Lemburg wrote: > Travis E. Oliphant wrote: >> >> >> >> PEP: >> Title: Adding data-type objects to the standard library >> Attributes >> >> kind -- returns the basic "kind" of the data-type. The basic kinds >> are: >>'t' - bit, >>'b' - bool, >>'i' - signed integer, >>'u' - unsigned integer, >>'f' - floating point, >>'c' - complex floating point, >>'S' - string (fixed-length sequence of char), >>'U' - fixed length sequence of UCS4, > > Shouldn't this read "fixed length sequence of Unicode" ?! > The underlying code unit format (UCS2 and UCS4) depends on the > Python version. Well, in NumPy 'U' always means UCS4. So, I just copied that over. See my questions at the bottom which talk about how to handle this. A data-format does not necessarily have to correspond to something Python represents with an Object. -Travis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP: Adding data-type objects to Python
Travis E. Oliphant wrote: > > > > > PEP: > Title: Adding data-type objects to the standard library > Attributes > > kind -- returns the basic "kind" of the data-type. The basic kinds > are: >'t' - bit, >'b' - bool, >'i' - signed integer, >'u' - unsigned integer, >'f' - floating point, >'c' - complex floating point, >'S' - string (fixed-length sequence of char), >'U' - fixed length sequence of UCS4, Shouldn't this read "fixed length sequence of Unicode" ?! The underlying code unit format (UCS2 and UCS4) depends on the Python version. >'O' - pointer to PyObject, >'V' - Void (anything else). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 28 2006) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] DRAFT: python-dev summary for 2006-09-16 to 2006-09-30
On Oct 28, 2006, at 1:50 AM, Martin v. Löwis wrote: Steven Bethard schrieb: Jack Howarth asked about creating universal binaries for OS X that would support 32-bit or 64-bit on both PPC and x86. Ronald Oussoren pointed out that the 32-bit part of this was already supported, but indicated that adding 64-bit support simultaneously might be more difficult. Ronald seemed to think that modifications to pyconfig.h.in might solve the problem, though he was worried that this might cause distutils to detect some architecture features incorrectly. Ronald can surely speak for himself, but I think the problem is slightly different. There were different strategies discussed for changing pyconfig.h (with an include, or with #ifdefs), and in all cases, distutils would fail to detect the architecture properly. That's not really a problem of pyconfig.h, but of the way that distutils uses to detect bitsizes - which inherently cannot work for universal binaries (i.e. you should look at the running interpreter, not at pyconfig.h). That depends on what you want to do. If you want to use the information about byteorder and bitsizes to drive the build of an extension you're better of using pyconfig.h instead of using the values of the currently running interpreter. If you want to use the information to generate raw data files in the platform byteorder and bitsizes you're better of using the struct module, so there's really no good reason to look at WORDS_BIGENDIAN and the various SIZEOF_ macros through distutils. An example of this was the build of expat: before I merged the universal binary patches setup.py looked at sys.byteorder and then added a define to the build flags for expat. With the universal patches I changed this to an include-file that looks at the value in pyconfig.h and sets the define that expat expects. This is needed because with universal binaries the byteorder and bitsizes are no longer configure-time constants but are compile-time constants. Note that adding support for universal builds for 32-bit architecturs was relatively easy because only one variable in pyconfig.h needed to be patched and GCC has explicit support for getting the required information. The patch for 32-bit/64-bit builds will probably require sniffing the current architecture (e.g. "#ifdef __i386__") and settings values that way. The cleanest way to do that is in introduction of an additional include file. It also requires changes to setup.py because all mac-specific modules won't build in 64-bit code in released versions of the OS (because OSX only has a 64-bit unix-layer in 10.4, 10.5 will be 64-bit throughout). Ronald smime.p7s Description: S/MIME cryptographic signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
