Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-17 Thread Nathaniel Smith
On Wed, Jun 13, 2012 at 7:54 PM, Wes McKinney wesmck...@gmail.com wrote: It looks like the levels can only be strings. This is too limited for my needs. Why not support all possible NumPy dtypes? In pandas world, the levels can be any unique Index object It seems like there are three obvious

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-17 Thread Wes McKinney
On Sun, Jun 17, 2012 at 6:10 AM, Nathaniel Smith n...@pobox.com wrote: On Wed, Jun 13, 2012 at 7:54 PM, Wes McKinney wesmck...@gmail.com wrote: It looks like the levels can only be strings. This is too limited for my needs. Why not support all possible NumPy dtypes? In pandas world, the levels

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-17 Thread Nathaniel Smith
On Sun, Jun 17, 2012 at 9:04 PM, Wes McKinney wesmck...@gmail.com wrote: On Sun, Jun 17, 2012 at 6:10 AM, Nathaniel Smith n...@pobox.com wrote: On Wed, Jun 13, 2012 at 7:54 PM, Wes McKinney wesmck...@gmail.com wrote: It looks like the levels can only be strings. This is too limited for my

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-17 Thread Nathaniel Smith
On Wed, Jun 13, 2012 at 11:06 PM, Bryan Van de Ven bry...@continuum.io wrote: On 6/13/12 1:12 PM, Nathaniel Smith wrote: Yes, of course we *could* write the code to implement these open dtypes, and then write the documentation, examples, tutorials, etc. to help people work around their

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-14 Thread Dag Sverre Seljebotn
On 06/14/2012 12:06 AM, Bryan Van de Ven wrote: On 6/13/12 1:12 PM, Nathaniel Smith wrote: your-branch's-base-master but not in your-repo's-master are new stuff that you did on your branch. Solution is just to do git pushyour github remote name master Fixed, thanks. Yes, of course we

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-14 Thread Thouis (Ray) Jones
On Wed, Jun 13, 2012 at 8:54 PM, Wes McKinney wesmck...@gmail.com wrote: Nathaniel: my experience (see blog posting above for a bit more) is that khash really crushes PyDict for two reasons: you can use it with primitive types and avoid boxing, and secondly you can preallocate. Its memory

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-13 Thread Nathaniel Smith
On Tue, Jun 12, 2012 at 10:27 PM, Bryan Van de Ven bry...@continuum.io wrote: Hi all, It has been some time, but I do have an update regarding this proposed feature. I thought it would be helpful to flesh out some parts of a possible implementation to learn what can be spelled reasonably in

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-13 Thread Dag Sverre Seljebotn
On 06/13/2012 03:33 PM, Nathaniel Smith wrote: On Tue, Jun 12, 2012 at 10:27 PM, Bryan Van de Venbry...@continuum.io wrote: Hi all, It has been some time, but I do have an update regarding this proposed feature. I thought it would be helpful to flesh out some parts of a possible

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-13 Thread Nathaniel Smith
On Wed, Jun 13, 2012 at 5:04 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 06/13/2012 03:33 PM, Nathaniel Smith wrote: I'm inclined to say therefore that we should just drop the open type idea, since it adds complexity but doesn't seem to actually solve the problem it's

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-13 Thread Dag Sverre Seljebotn
Nathaniel Smith n...@pobox.com wrote: On Wed, Jun 13, 2012 at 5:04 PM, Dag Sverre Seljebotn d.s.seljeb...@astro.uio.no wrote: On 06/13/2012 03:33 PM, Nathaniel Smith wrote: I'm inclined to say therefore that we should just drop the open type idea, since it adds complexity but doesn't seem to

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-13 Thread Nathaniel Smith
On Wed, Jun 13, 2012 at 5:44 PM, Bryan Van de Ven bry...@continuum.io wrote: On 6/13/12 8:33 AM, Nathaniel Smith wrote: Hi Bryan, I skimmed over the diff:     https://github.com/bryevdv/numpy/compare/master...enum It was a bit hard to read since it seems like about half the changes in that

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-13 Thread Wes McKinney
On Wed, Jun 13, 2012 at 2:12 PM, Nathaniel Smith n...@pobox.com wrote: On Wed, Jun 13, 2012 at 5:44 PM, Bryan Van de Ven bry...@continuum.io wrote: On 6/13/12 8:33 AM, Nathaniel Smith wrote: Hi Bryan, I skimmed over the diff:     https://github.com/bryevdv/numpy/compare/master...enum It was

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-13 Thread Bryan Van de Ven
On 6/13/12 1:54 PM, Wes McKinney wrote: OK, I need to spend some time on this as it will directly impact me. Random thoughts here. It looks like the levels can only be strings. This is too limited for my needs. Why not support all possible NumPy dtypes? In pandas world, the levels can be any

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-13 Thread Wes McKinney
On Wed, Jun 13, 2012 at 5:19 PM, Bryan Van de Ven bry...@continuum.io wrote: On 6/13/12 1:54 PM, Wes McKinney wrote: OK, I need to spend some time on this as it will directly impact me. Random thoughts here. It looks like the levels can only be strings. This is too limited for my needs. Why

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-13 Thread Bryan Van de Ven
On 6/13/12 1:12 PM, Nathaniel Smith wrote: your-branch's-base-master but not in your-repo's-master are new stuff that you did on your branch. Solution is just to do git pushyour github remote name master Fixed, thanks. Yes, of course we *could* write the code to implement these open

Re: [Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-13 Thread Bryan Van de Ven
On 6/13/12 5:11 PM, Wes McKinney wrote: And retrieving group indicies/summing: In [8]: %timeit arr=='a' 1000 loops, best of 3: 1.52 ms per loop In [10]: vals = np.random.randn(100) In [20]: inds = [arr==x for x in lets] In [23]: %timeit for ind in inds: vals[ind].sum() 10 loops, best

[Numpy-discussion] Enum/Factor NEP (now with code)

2012-06-12 Thread Bryan Van de Ven
Hi all, It has been some time, but I do have an update regarding this proposed feature. I thought it would be helpful to flesh out some parts of a possible implementation to learn what can be spelled reasonably in NumPy. Mark Wiebe helped out greatly in navigating the NumPy code codebase.