Re: Schema and indexes for efficient time range queries

2015-06-11 Thread Yiannis Gkoufas
Hi there, I did try to reverse the order of the Composite Key and it worked! I changed BIGINT to UNSIGNED_LONG because I was facing some issues with the timezone of the DATE field. There is one small detail I cannot understand why is happening: When I give the query: SELECT * FROM SENSOR_DATA WHE

Re: Schema and indexes for efficient time range queries

2015-06-09 Thread Yiannis Gkoufas
Thanks a lot for your replies! Will try the DATE field and change the order of the Composite Key. On 8 June 2015 at 17:37, James Taylor wrote: > Both DATE and TIME have millisecond granularity (both are stored as 8 > byte longs), so I'd recommend using either of those. Phoenix also > supports da

Re: Schema and indexes for efficient time range queries

2015-06-08 Thread James Taylor
Both DATE and TIME have millisecond granularity (both are stored as 8 byte longs), so I'd recommend using either of those. Phoenix also supports date arithmetic, so you can do queries like this to get the last weeks worth of data: SELECT * FROM SENSOR_DATA WHERE sid = 'ID1' AND dt > CURRENT_TIME()

Re: Schema and indexes for efficient time range queries

2015-06-08 Thread Vladimir Rodionov
There are several time data types, natively supported by Phoenix: TIME is probably most suitable for your case (it should have millisecond accuracy, but you better check it yourself.) -Vlad On Mon, Jun 8, 2015 at 9:02 AM, Yiannis Gkoufas wrote: > Hi Vladimir, > > thanks a lot for your input, ju

Re: Schema and indexes for efficient time range queries

2015-06-08 Thread Yiannis Gkoufas
Hi Vladimir, thanks a lot for your input, just some followup questions: 1) When you say "try to fit it in long" you mean UNSIGNED_LONG from https://phoenix.apache.org/language/datatypes.html right? 2) Would also string format be efficient? Like MMDDHHmm right? Thanks a lot! On 8 June 2015 at

Re: Schema and indexes for efficient time range queries

2015-06-08 Thread Vladimir Rodionov
PRIMARY KEY(dt,sid)) won't work well for your query. PRIMARY KEY(sid, dt)) is much better for time range queries for a particular sensor. In a latter case this query will be translated into efficient range scan. Do not use bigint for timestamp, try to fit it into long or use stringified version in

Schema and indexes for efficient time range queries

2015-06-08 Thread Yiannis Gkoufas
Hi there, I am investigating Phoenix as a potential data-store for time-series data on sensors. What I am really interested in, as a first milestone, is to have efficient time range queries for a particular sensor. From those queries the results would consist of 1 or 2 columns (so small rows). I w