On Mon, Apr 15, 2013 at 9:40 AM, Julio Trevisan <juliotrevi...@gmail.com>wrote:
> Hi Anthony
>
> Thanks for adding this issue.
>
> Is there a way to use CS indexes to get the row coordinates satisfying a
> simple condition sorted by the column in the condition? I would like to
> avoid using numpy.sort() since the sorting order is probably already
> available within the index information.
>
> My condition is simply (timestamp >= %d) & (timestamp <= %d)" % (ts1, ts2)
>
> If you could please give me some guidelines so that I could put together
> such a method, that would be great.
>
> Julio
>
>
> (like using getWhereList()), but using an index to get the coordinate list
> ordered by column indexes to get the coordinates ? I nhas a *sort*parameter
> that uses numpy.sort() to do the work, and that readSorted() uses
> a full index to get the sorted sequence. I couldn't make complete sense yet
> of "chunkmaps" being passed to numexpr evaluator inside _where().
>
Hi Julio,
Thanks for taking this on! You probably what to read [1] to figure out how
numexpr works and what chunkmaps is doing, if you haven't already.
However, probably the simplest implementation of this method would be
basically part of the read_where() method followed by the read_sorted()
body. It would look like this, but you'll have to try it:
def read_where_sorted(self, ...):
self._g_check_open()
condcoords = set([p.nrow for p in
self._where(condition, condvars, start, stop, step)])
self._where_condition = None # reset the conditions
index = self._check_sortby_csi(sortby, checkCSI)
sortcoords = index[start:stop:step]
coords = [c for c in sortcoords if c in condcoords]
return self.read_coordinates(coords, field)
There may be faster, more elegant solutions, but I think that something
like this would work.
Be Well
Anthony
1. http://code.google.com/p/numexpr/
>
>
> On Thu, Apr 11, 2013 at 1:14 PM, Anthony Scopatz <scop...@gmail.com>wrote:
>
>> Thanks for bringing this up, Julio.
>>
>> Hmm I don't think that this exists currently, but since there are
>> readWhere() and readSorted() it shouldn't be too hard to implement. I have
>> opened issue #225 to this effect. Pull requests welcome!
>>
>> https://github.com/PyTables/PyTables/issues/225
>>
>> Be Well
>> Anthony
>>
>>
>> On Wed, Apr 10, 2013 at 1:02 PM, Dr. Louis Wicker
>> <louis.wic...@noaa.gov>wrote:
>>
>>> I am also interested in the this capability, if it exists in some way...
>>>
>>> Lou
>>>
>>> On Apr 10, 2013, at 12:35 PM, Julio Trevisan <juliotrevi...@gmail.com>
>>> wrote:
>>>
>>> > Hi,
>>> >
>>> > Is there a way that I could have the ability of readWhere (i.e.,
>>> specify condition, and fast result) but also using a CSIndex so that the
>>> rows come sorted in a particular order?
>>> >
>>> > I checked readSorted() but it is iterative and does not allow to
>>> specify a condition.
>>> >
>>> > Julio
>>> >
>>> ------------------------------------------------------------------------------
>>> > Precog is a next-generation analytics platform capable of advanced
>>> > analytics on semi-structured data. The platform includes APIs for
>>> building
>>> > apps and a phenomenal toolset for data science. Developers can use
>>> > our toolset for easy data analysis & visualization. Get a free account!
>>> >
>>> http://www2.precog.com/precogplatform/slashdotnewsletter_______________________________________________
>>> > Pytables-users mailing list
>>> > Pytables-users@lists.sourceforge.net
>>> > https://lists.sourceforge.net/lists/listinfo/pytables-users
>>>
>>>
>>> ----------------------------------------------------------------------------
>>> | Dr. Louis J. Wicker
>>> | NSSL/WRDD Rm 4366
>>> | National Weather Center
>>> | 120 David L. Boren Boulevard, Norman, OK 73072
>>> |
>>> | E-mail: louis.wic...@noaa.gov
>>> | HTTP: http://www.nssl.noaa.gov/~lwicker
>>> | Phone: (405) 325-6340
>>> | Fax: (405) 325-6780
>>> |
>>> |
>>> I For every complex problem, there is a solution that is simple,
>>> | neat, and wrong.
>>> |
>>> | -- H. L. Mencken
>>> |
>>>
>>> ----------------------------------------------------------------------------
>>> | "The contents of this message are mine personally and
>>> | do not reflect any position of the Government or NOAA."
>>>
>>> ----------------------------------------------------------------------------
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Precog is a next-generation analytics platform capable of advanced
>>> analytics on semi-structured data. The platform includes APIs for
>>> building
>>> apps and a phenomenal toolset for data science. Developers can use
>>> our toolset for easy data analysis & visualization. Get a free account!
>>> http://www2.precog.com/precogplatform/slashdotnewsletter
>>> _______________________________________________
>>> Pytables-users mailing list
>>> Pytables-users@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/pytables-users
>>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Precog is a next-generation analytics platform capable of advanced
>> analytics on semi-structured data. The platform includes APIs for building
>> apps and a phenomenal toolset for data science. Developers can use
>> our toolset for easy data analysis & visualization. Get a free account!
>> http://www2.precog.com/precogplatform/slashdotnewsletter
>> _______________________________________________
>> Pytables-users mailing list
>> Pytables-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/pytables-users
>>
>>
>
>
> ------------------------------------------------------------------------------
> Precog is a next-generation analytics platform capable of advanced
> analytics on semi-structured data. The platform includes APIs for building
> apps and a phenomenal toolset for data science. Developers can use
> our toolset for easy data analysis & visualization. Get a free account!
> http://www2.precog.com/precogplatform/slashdotnewsletter
> _______________________________________________
> Pytables-users mailing list
> Pytables-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/pytables-users
>
>
------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
