Re: [Numpy-discussion] FeatureRequest: support for array construction from iterators

Actually, while working on https://github.com/numpy/numpy/issues/7264 I realized that the memory efficiency (one-pass) argument is simply incorrect: import numpy as np class A: def __getitem__(self, i): print("A get item", i) return [np.int8(1), np.int8(2)][i] def

Re: [Numpy-discussion] FeatureRequest: support for array construction from iterators

just to not prevent it from the black hole - what about integrating fromiter into array? (see the post by Benjamin Root) for me personally, taking the first element for deducing the dtype would be a perfect default way to read generators. If one wants a specific other dtype, one could specify it

Re: [Numpy-discussion] FeatureRequest: support for array construction from iterators

Devil's advocate here: np.array() has become the de-facto "constructor" for numpy arrays. Right now, passing it a generator results in what, IMHO, is a useless result: >>> np.array((i for i in range(10))) array( at 0x7f28b2beca00>, dtype=object) Passing pretty much any dtype argument will cause

Re: [Numpy-discussion] FeatureRequest: support for array construction from iterators

I would like to further push Benjamin Root's suggestion: "Therefore, I think it is not out of the realm of reason that passing a generator object and a dtype could then delegate the work under the hood to np.fromiter()? I would even go so far as to raise an error if one passes a generator without

Re: [Numpy-discussion] FeatureRequest: support for array construction from iterators

Heh, never noticed that. Was it implemented more like a generator/iterator in older versions of Python? Thanks, Ben Root On Mon, Dec 14, 2015 at 12:38 PM, Robert Kern wrote: > On Mon, Dec 14, 2015 at 3:56 PM, Benjamin Root > wrote: > > > By the

Re: [Numpy-discussion] FeatureRequest: support for array construction from iterators

On Mon, Dec 14, 2015 at 3:56 PM, Benjamin Root wrote: > By the way, any reason why this works? > >>> np.array(xrange(10)) > array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) It's not a generator. It's a true sequence that just happens to have a special implementation rather than being

Re: [Numpy-discussion] FeatureRequest: support for array construction from iterators

On Mon, Dec 14, 2015 at 5:41 PM, Benjamin Root wrote: > > Heh, never noticed that. Was it implemented more like a generator/iterator in older versions of Python? No, it predates generators and iterators so it has always had to be implemented like that. -- Robert Kern

Re: [Numpy-discussion] FeatureRequest: support for array construction from iterators

On Fri, Dec 11, 2015 at 11:32 PM, Juan Nunez-Iglesias wrote: > Nathaniel, > >> IMO this is better than making np.array(iter) internally call list(iter) >> or equivalent > > Yeah but that's not the only option: > > from itertools import chain > def

Re: [Numpy-discussion] FeatureRequest: support for array

> > > > from itertools import chain > > def fromiter_awesome_edition(iterable): > > elem = next(iterable) > > dtype = whatever_numpy_does_to_infer_dtypes_from_lists(elem) > > return np.fromiter(chain([elem], iterable), dtype=dtype) > > > > I think this would be a huge win for

Re: [Numpy-discussion] FeatureRequest: support for array construction from iterators

Hey Nathaniel, Fascinating! Thanks for the primer! I didn't know that it would check dtype of values in the whole array. In that case, I would agree that it would be bad to infer it magically from just the first value, and this can be left to the users. Thanks! Juan. On Sat, Dec 12, 2015 at

Re: [Numpy-discussion] FeatureRequest: support for array construction from iterators

Constructing an array from an iterator is fundamentally different from constructing an array from an in-memory data structure like a list, because in the iterator case it's necessary to either use a single-pass algorithm or else create extra temporary buffers that cause much higher memory

Re: [Numpy-discussion] FeatureRequest: support for array construction from iterators

numpy.fromiter is neither numpy.array nor does it work similar to numpy.array(list(...)) as the dtype argument is necessary is there a reason, why np.array(...) should not work on iterators? I have the feeling that such requests get (repeatedly) dismissed, but until yet I haven't found a

Re: [Numpy-discussion] FeatureRequest: support for array construction from iterators

Nathaniel, > IMO this is better than making np.array(iter) internally call list(iter) or equivalent Yeah but that's not the only option: from itertools import chain def fromiter_awesome_edition(iterable): elem = next(iterable) dtype = whatever_numpy_does_to_infer_dtypes_from_lists(elem)

Re: [Numpy-discussion] FeatureRequest: support for array construction from iterators

On 11/27/2015 5:37 AM, Stephan Sahm wrote: I like to request a generator/iterator support for np.array(...) as far as list(...) supports it. http://docs.scipy.org/doc/numpy/reference/generated/numpy.fromiter.html hth, Alan Isaac ___

[Numpy-discussion] FeatureRequest: support for array construction from iterators

​​ [ this ​request /discussion refers to numpy issue ​​ ​​ #5863 ​ ​ ​​ https://github.com/numpy/numpy/pull/5863#issuecomment-159738368 ] Dear all, As far as I can think, the expected functionality of np.array(...) would be np.array(list(...)) or something even nicer. Therefore, I like to