Re: [Python-ideas] Python multi-dimensional array constructor

2016-10-19 Thread Steven D'Aprano
On Wed, Oct 19, 2016 at 03:08:21PM -0400, Todd wrote: [taking your later comment out of the order it was written] > If this sort of thing doesn't interest you I won't be offended if you stop > reading now, and I apologize if it is considered off-topic for this ML. No problem Todd, we shouldn't be

Re: [Python-ideas] Py_SIZE of PyLongs

2016-10-19 Thread Tim Peters
[Elliot Gorokhovsky ] > I'm working on a special-case compare function for bounded integers for the > sort stuff. By looking at the implementation, I figured out that Py_SIZE of > a long is the sign times the number of digits (...right?). > ... Please ignore the other reply you got - they clearly

Re: [Python-ideas] Py_SIZE of PyLongs

2016-10-19 Thread Elliot Gorokhovsky
It's in the code. See longobject.c: Py_SIZE(v) = ndigits*sign; You can also see Py_SIZE(v) used on PyLongs all over the place in longobject.c, for example: v = (PyLongObject *)vv; i = Py_SIZE(v); Just do a ctrl-f for Py_SIZE(v) in longobject.c. Like I said, by looking in the implementat

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Terry Reedy
On 10/19/2016 12:38 AM, Nathaniel Smith wrote: I'd like to propose that Python's iterator protocol be enhanced to add a first-class notion of completion / cleanup. With respect the the standard iterator protocol, a very solid -1 from me. (I leave commenting specifically on __aiterclose__ to

Re: [Python-ideas] Py_SIZE of PyLongs

2016-10-19 Thread Thomas Nyberg
On 10/19/2016 09:04 PM, Elliot Gorokhovsky wrote: A quick note: I'm working on a special-case compare function for bounded integers for the sort stuff. By looking at the implementation, I figured out that Py_SIZE of a long is the sign times the number of digits (...right?). Before looking at the

[Python-ideas] Py_SIZE of PyLongs

2016-10-19 Thread Elliot Gorokhovsky
A quick note: I'm working on a special-case compare function for bounded integers for the sort stuff. By looking at the implementation, I figured out that Py_SIZE of a long is the sign times the number of digits (...right?). Before looking at the implementation, though, I had looked for this info

Re: [Python-ideas] Python multi-dimensional array constructor

2016-10-19 Thread Todd
On Wed, Oct 19, 2016 at 7:48 PM, Chris Barker wrote: > a few thoughts: > > On Wed, Oct 19, 2016 at 12:08 PM, Todd wrote: > >> I have been thinking about how to go about having a multidimensional >> array constructor in python. I know that Python doesn't have a built-in >> multidimensional array

Re: [Python-ideas] Python multi-dimensional array constructor

2016-10-19 Thread Alexander Belopolsky
On Wed, Oct 19, 2016 at 7:48 PM, Chris Barker wrote: > > > However, if you really don't like it, then you can pass a string to aconfsturctor function instead: > > a = arr_from_string(" | 0, 1, 2 || 3, 4, 5 | ") > > yeah, you need to type the extra quotes, but that's not much. > > NOTE: I'm pretty

Re: [Python-ideas] Python multi-dimensional array constructor

2016-10-19 Thread Chris Barker
a few thoughts: On Wed, Oct 19, 2016 at 12:08 PM, Todd wrote: > I have been thinking about how to go about having a multidimensional array > constructor in python. I know that Python doesn't have a built-in > multidimensional array class and won't for the foreseeable future. > no but it does h

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Yury Selivanov
On 2016-10-19 6:07 PM, Paul Moore wrote: I missed that you propose phasing this in, but it doesn't really alter much, I think the current behaviour is valuable and common, and I'm -1 on breaking it. It's just too much of a fundamental change to how loops and iterators interact for me to be comf

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Robert Collins
Hey Nathaniel - I like the intent here, but I think perhaps it would be better if the problem is approached differently. Seems to me that making *generators* have a special 'you are done now' interface is special casing, which usually makes things harder to learn and predict; and that more the net

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Paul Moore
On 19 October 2016 at 20:21, Nathaniel Smith wrote: > On Wed, Oct 19, 2016 at 11:38 AM, Paul Moore wrote: >> On 19 October 2016 at 19:13, Chris Angelico wrote: >>> Now it *won't* correctly call the end-of-iteration function, because >>> there's no 'for' loop. This is going to either (a) require

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Nathaniel Smith
On Wed, Oct 19, 2016 at 11:13 AM, Chris Angelico wrote: > On Thu, Oct 20, 2016 at 3:38 AM, Random832 wrote: >> On Wed, Oct 19, 2016, at 11:51, Yury Selivanov wrote: >>> I'm -1 on the idea. Here's why: >>> >>> >>> 1. Python is a very dynamic language with GC and that is one of its >>> fundamental

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Yury Selivanov
Nathaniel, On 2016-10-19 5:02 PM, Nathaniel Smith wrote: Hi Yury, Thanks for the detailed comments! Replies inline below. NP! On Wed, Oct 19, 2016 at 8:51 AM, Yury Selivanov wrote: I'm -1 on the idea. Here's why: 1. Python is a very dynamic language with GC and that is one of its fun

Re: [Python-ideas] Python multi-dimensional array constructor

2016-10-19 Thread Todd
On Wed, Oct 19, 2016 at 4:47 PM, Matt Gilson wrote: > FWIW, you probably _don't_ want to use `ndarray` directly. Normally, you > want to use the `np.array` factory function... > > >>> import numpy as np > >>> a = np.ndarray([0, 1, 2]) > >>> a > array([], shape=(0, 1, 2), dtype=float64) > > Aside

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Nathaniel Smith
On Wed, Oct 19, 2016 at 1:33 PM, Yury Selivanov wrote: > On 2016-10-19 3:33 PM, Nathaniel Smith wrote: > >>> lst = [1,2,3,4] >>> >it = iter(lst) >>> >for i in it: >>... if i == 2: break >> >>> >>> >for i in it: >>... print(i) >>3 >>

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Neil Girdhar
Ohhh, sorry, you want __iterclose__ to happen when iteration is terminated by a break statement as well? Okay, I understand, and that's fair. However, I would rather that people be explicit about when they're iterating (use the iteration protocol) and when they're managing a resource (use a c

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Nathaniel Smith
Hi Yury, Thanks for the detailed comments! Replies inline below. On Wed, Oct 19, 2016 at 8:51 AM, Yury Selivanov wrote: > I'm -1 on the idea. Here's why: > > > 1. Python is a very dynamic language with GC and that is one of its > fundamental properties. This proposal might make GC of iterators

Re: [Python-ideas] Python multi-dimensional array constructor

2016-10-19 Thread Matt Gilson
FWIW, you probably _don't_ want to use `ndarray` directly. Normally, you want to use the `np.array` factory function... >>> import numpy as np >>> a = np.ndarray([0, 1, 2]) >>> a array([], shape=(0, 1, 2), dtype=float64) Aside from that, my main problem with this proposal is that it seems to onl

Re: [Python-ideas] Python multi-dimensional array constructor

2016-10-19 Thread Mikhail V
On 19 October 2016 at 21:08, Todd wrote: > > a = np.ndarray(48, 11, 141, 13, -60, -37, 58, -52, -29, 134], > [-6, 96, -66, 137, -59, -147, -118, -104, -123, -7]], > [[-103, 50, -89, -12, 28, -12, 119, -131, -73, 21], > [-58, 105, 25, -138,

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Yury Selivanov
On 2016-10-19 3:33 PM, Nathaniel Smith wrote: lst = [1,2,3,4] >it = iter(lst) >for i in it: >>... if i == 2: break >> >for i in it: >>... print(i) >>3 >>4 > >> >>With the proposed behaviour, if I understand it, "it" would be closed >>after the first loop, so resuming "it

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Chris Angelico
On Thu, Oct 20, 2016 at 7:14 AM, Neil Girdhar wrote: > class AddIterclose: > > def __init__(self, iterable, iterclose): > self.iterable = iterable > self.iterclose = iterclose > > def __iter__(self): > try: > for x in self.iterable: > yie

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Neil Girdhar
On Wed, Oct 19, 2016 at 2:11 PM Nathaniel Smith wrote: > On Wed, Oct 19, 2016 at 10:08 AM, Neil Girdhar > wrote: > > > > > > On Wed, Oct 19, 2016 at 11:08 AM Todd wrote: > >> > >> On Wed, Oct 19, 2016 at 3:38 AM, Neil Girdhar > >> wrote: > >>> > >>> This is a very interesting proposal. I just

Re: [Python-ideas] Python multi-dimensional array constructor

2016-10-19 Thread Todd
On Wed, Oct 19, 2016 at 3:55 PM, Joseph Jevnik wrote: > You could add or prototype this with quasiquotes (http://quasiquotes. > readthedocs.io/en/latest/). You just need to be able to parse the body of > your expression as a string into an array. Here is a quick example with a > parser that only

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Brendan Barnwell
On 2016-10-19 12:21, Nathaniel Smith wrote: >Also, unless I'm misunderstanding the proposal, there's a fairly major >compatibility break. At present we have: > lst = [1,2,3,4] it = iter(lst) for i in it: >... if i == 2: break > for i in it: >... print(i) >3 >4 > >Wi

Re: [Python-ideas] Python multi-dimensional array constructor

2016-10-19 Thread Joseph Jevnik
You could add or prototype this with quasiquotes ( http://quasiquotes.readthedocs.io/en/latest/). You just need to be able to parse the body of your expression as a string into an array. Here is a quick example with a parser that only accepts 2d arrays: ``` # coding: quasiquotes import numpy as n

Re: [Python-ideas] Python multi-dimensional array constructor

2016-10-19 Thread Todd
On Wed, Oct 19, 2016 at 3:24 PM, Thomas Nyberg wrote: > Personally I like the way that numpy does it now better (even for > multidimensional arrays). Being able to index into the different sub > dimension using just [] iteratively matches naturally with the data > structure itself in my mind. Thi

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Nathaniel Smith
On Wed, Oct 19, 2016 at 12:21 PM, Nathaniel Smith wrote: > On Wed, Oct 19, 2016 at 11:38 AM, Paul Moore wrote: >> On 19 October 2016 at 19:13, Chris Angelico wrote: >>> Now it *won't* correctly call the end-of-iteration function, because >>> there's no 'for' loop. This is going to either (a) req

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Nathaniel Smith
On Wed, Oct 19, 2016 at 11:38 AM, Paul Moore wrote: > On 19 October 2016 at 19:13, Chris Angelico wrote: >> Now it *won't* correctly call the end-of-iteration function, because >> there's no 'for' loop. This is going to either (a) require that EVERY >> consumer of an iterator follow this new prot

Re: [Python-ideas] Python multi-dimensional array constructor

2016-10-19 Thread Thomas Nyberg
Personally I like the way that numpy does it now better (even for multidimensional arrays). Being able to index into the different sub dimension using just [] iteratively matches naturally with the data structure itself in my mind. This may also just be my fear of change though... Here is an

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Todd
On Wed, Oct 19, 2016 at 2:38 PM, Paul Moore wrote: > On 19 October 2016 at 19:13, Chris Angelico wrote: > > Now it *won't* correctly call the end-of-iteration function, because > > there's no 'for' loop. This is going to either (a) require that EVERY > > consumer of an iterator follow this new p

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Ethan Furman
On 10/19/2016 11:38 AM, Paul Moore wrote: Also, unless I'm misunderstanding the proposal, there's a fairly major compatibility break. At present we have: lst = [1,2,3,4] it = iter(lst) for i in it: ... if i == 2: break for i in it: ... print(i) 3 4 With the proposed behaviour, if I

[Python-ideas] Python multi-dimensional array constructor

2016-10-19 Thread Todd
I have been thinking about how to go about having a multidimensional array constructor in python. I know that Python doesn't have a built-in multidimensional array class and won't for the foreseeable future. However, some projects have come up with their own ways of making it simpler to create suc

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Paul Moore
On 19 October 2016 at 19:13, Chris Angelico wrote: > Now it *won't* correctly call the end-of-iteration function, because > there's no 'for' loop. This is going to either (a) require that EVERY > consumer of an iterator follow this new protocol, or (b) introduce a > ton of edge cases. Also, unles

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Chris Angelico
On Thu, Oct 20, 2016 at 3:38 AM, Random832 wrote: > On Wed, Oct 19, 2016, at 11:51, Yury Selivanov wrote: >> I'm -1 on the idea. Here's why: >> >> >> 1. Python is a very dynamic language with GC and that is one of its >> fundamental properties. This proposal might make GC of iterators more >> de

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Nathaniel Smith
On Wed, Oct 19, 2016 at 10:08 AM, Neil Girdhar wrote: > > > On Wed, Oct 19, 2016 at 11:08 AM Todd wrote: >> >> On Wed, Oct 19, 2016 at 3:38 AM, Neil Girdhar >> wrote: >>> >>> This is a very interesting proposal. I just wanted to share something I >>> found in my quick search: >>> >>> >>> http:/

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Neil Girdhar
On Wed, Oct 19, 2016 at 11:08 AM Todd wrote: > On Wed, Oct 19, 2016 at 3:38 AM, Neil Girdhar > wrote: > > This is a very interesting proposal. I just wanted to share something I > found in my quick search: > > > http://stackoverflow.com/questions/14797930/python-custom-iterator-close-a-file-on-

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Yury Selivanov
On 2016-10-19 12:38 PM, Random832 wrote: On Wed, Oct 19, 2016, at 11:51, Yury Selivanov wrote: I'm -1 on the idea. Here's why: 1. Python is a very dynamic language with GC and that is one of its fundamental properties. This proposal might make GC of iterators more deterministic, but that is

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Random832
On Wed, Oct 19, 2016, at 11:51, Yury Selivanov wrote: > I'm -1 on the idea. Here's why: > > > 1. Python is a very dynamic language with GC and that is one of its > fundamental properties. This proposal might make GC of iterators more > deterministic, but that is only one case. There is a hug

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Yury Selivanov
I'm -1 on the idea. Here's why: 1. Python is a very dynamic language with GC and that is one of its fundamental properties. This proposal might make GC of iterators more deterministic, but that is only one case. For instance, in some places in asyncio source code we have statements like t

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Todd
On Wed, Oct 19, 2016 at 3:38 AM, Neil Girdhar wrote: > This is a very interesting proposal. I just wanted to share something I > found in my quick search: > > http://stackoverflow.com/questions/14797930/python- > custom-iterator-close-a-file-on-stopiteration > > Could you explain why the accepte

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Neil Girdhar
This is a very interesting proposal. I just wanted to share something I found in my quick search: http://stackoverflow.com/questions/14797930/python-custom-iterator-close-a-file-on-stopiteration Could you explain why the accepted answer there doesn't address this issue? class Parse(object):

Re: [Python-ideas] Civility on this mailing list

2016-10-19 Thread Paul Moore
On 19 October 2016 at 12:29, Michel Desmoulin wrote: > I feel like people are really getting hyper sensitive about communications. > While I do prefer talking to calm rational people with a friendly tone, I > acknowledge this is not always the case and it's ok if somebody go overboard > from time

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Oscar Benjamin
On 19 October 2016 at 12:33, Oscar Benjamin wrote: > >> New convenience functions >> - >> >> The ``itertools`` module gains a new iterator wrapper that can be used >> to selectively disable the new ``__iterclose__`` behavior:: >> >> # XX FIXME: I feel like there might be

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Oscar Benjamin
On 17 October 2016 at 09:08, Nathaniel Smith wrote: > Hi all, Hi Nathaniel. I'm just reposting what I wrote on pypy-dev (as requested) but under the assumption that you didn't substantially alter your draft - I apologise if some of the quoted text below has already been edited. > Always inject r

Re: [Python-ideas] Deterministic iterator cleanup

2016-10-19 Thread Vincent Michel
Thanks Nathaniel for this great proposal. As I went through your mail, I realized all the comments I wanted to make were already covered in later paragraphs. And I don't think there's a single point I disagree with. I don't have a strong opinion about the synchronous part of the proposal. I

Re: [Python-ideas] Civility on this mailing list

2016-10-19 Thread Michel Desmoulin
+1. I read many disagreements, and people being rude and unprofessional on occasions, but nothing that would make me have a bad day, even when I was the target of it. I feel like people are really getting hyper sensitive about communications. While I do prefer talking to calm rational people