Hi Anthony,
Thanks for your response.
I had come across that discussion, but I don't think the floating-point
precision thing really explains my results, because I'm querying for intervals,
not instants.
if I have a table containing, say, one-second samples between 500.0 and 1500.0,
and I use a where clause like this:
'(update_seconds >= 1000.0) & (update_seconds <= 1060.0)'
then I expect to get at least 58 samples, even with floating-point 'fuzziness'
- but in fact I get none.
However, I have now tried the approach of storing my epoch seconds in
Float64Cols and that seems to be working just fine.
The question I'm left with is - just what does a Time64Col represent? Since
there's no standard Python Time class with a float representation, I just
guessed I could assign it float seconds a la time.time(), but Float64 works
just as well for that (and as it turns out, better). How could you use a
Time64Col in practice?
Thanks again,
Charles de Villiers
"They have computers, and they may have other weapons of mass destruction."
(Janet Reno)
________________________________
From: Anthony Scopatz <scop...@gmail.com>
To: Charles de Villiers <chas...@yahoo.com>; Discussion list for PyTables
<pytables-users@lists.sourceforge.net>
Sent: Monday, April 15, 2013 5:13 PM
Subject: Re: [Pytables-users] PyTables in-kernel query using Time64Col returns
wrong results
Hi Charles,
We just discussed this last week and I am too lazy to retype it all so here is
a link to the archive post [1].
Be Well
Anthony
1. http://sourceforge.net/mailarchive/message.php?msg_id=30708089
On Mon, Apr 15, 2013 at 9:20 AM, Charles de Villiers <chas...@yahoo.com> wrote:
>0
>down vote
>favorite I'm using PyTables 2.4.0 and Python 2.7 I've got a database that
>contains the following typical table:
>/anc/asc_wind_speed (Table(87591,),shuffle,blosc(3))'Wind speed'description
>:={"value_seconds":Time64Col(shape=(),dflt=0.0,pos=0),"update_seconds":Time64Col(shape=(),dflt=0.0,pos=1),"status":UInt8Col(shape=(),dflt=0,pos=2),"value":Float64Col(shape=(),dflt=0.0,pos=3)}byteorder
> :='little'chunkshape :=(2621,)autoIndex :=Truecolindexes
>:={"update_seconds":Index(9,full,shuffle,zlib(1)).is_CSI=True,"value":Index(9,full,shuffle,zlib(1)).is_CSI=True}
>I populate the timestamp columns using float seconds.
>The data looks OK in my IPython session:
>array([(1343779432.2160001,1343779431.8529999,0,5.2975000000000003),(1343779433.2190001,1343779432.9430001,0,5.7474999999999996),(1343779434.217,1343779433.9809999,0,5.8600000000000003),...,(1343866301.934,1343866301.5139999,0,3.8424999999999998),(1343866302.934,1343866302.5799999,0,4.0599999999999996),(1343866303.934,1343866303.642,0,3.7825000000000002)],dtype=[('value_seconds','<f8'),('update_seconds','<f8'),('status','|u1'),('value','<f8')])
>.. but when I try to do an in-kernel search using the indexed column
>'update_seconds', everything goes pear-shaped:
>len(wstable.readWhere('(update_seconds <= 1343866303.642)'))0
>ie I get 0 rows returned when I was expecting all 87591 of them. Occasionally
>I do manage to get some rows with a '>=' query, but the timestamp columns are
>then returned as huge floats (~10^79). It seems that there is some implicit
>type-conversion going on that causes the Time64Col values to be
>misinterpreted. Can someone spot my mistake, or should I forget about
>Time64Cols and convert them all to Float64 (and how do I do this?)
>
>
>------------------------------------------------------------------------------
>Precog is a next-generation analytics platform capable of advanced
>analytics on semi-structured data. The platform includes APIs for building
>apps and a phenomenal toolset for data science. Developers can use
>our toolset for easy data analysis & visualization. Get a free account!
>http://www2.precog.com/precogplatform/slashdotnewsletter
>_______________________________________________
>Pytables-users mailing list
>Pytables-users@lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/pytables-users
>
>------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
