Hi Welly,
Ignite perfectly fits for your task.
First, if I understand you properly there are will be many time series
for a give sensor ID.
If so then I would use a compound key like (sensorId, time) for all
cache related operations.
As an example you may want to use classes like this.
SampleKey {
int sernsorId;
long sampleTime;
}
Sample {
int sensorId;
long sampleTime;
byte[] data1;
byte[] data2;
etc.
}
And use them this way
cache.put(new SampleKey(1, time), sample);
cache.get(new SampleKey(2, time));
Second, to retrieve samples depending on sensor ID, time (time range) or
other parameters you can leverage Ignite SQL engine that is designed
exactly for the use cases you have. [1]
However, if you're going to use an object field in 'SELECT' or 'WHERE'
clause you have to annotate it properly or specify using
CacheTypeMetadata [2]
Third, when you need to update data series you can remove the old one
and insert the new one that should have new time.
To perform a remove I would suggest doing the following:
- select sensor ID and sampleTime of all the entries to delete;
- use cache.removeAll by passing SampleKeys that are created using the
data retrieved with SQL above.
Moreover, you can use an eviction or expire policy that is used in cases
when old data must be removed from cache automatically.
Just refer to these articles for more info - [3]
Finally, Ignite has bunch of cache and SQL related examples. They are
located in "datagrid" folder of "examples" module.
Have a look at them and probably you'll come up with better solution
based on Ignite that suggested by me above cause definitely you know all
the details of your case better ;)
[1] https://apacheignite.readme.io/docs/sql-queries
[2]
https://apacheignite.readme.io/docs/sql-queries#configuring-sql-indexes-by-annotations
[3] https://apacheignite.readme.io/docs/evictions
[4] https://apacheignite.readme.io/docs/expiry-policies
Regards,
Denis
On 12/3/2015 12:50 PM, Welly Tambunan wrote:
Hi Igniters,
Currently we are trying to asses the possibility of using Ignite on
our Architecture.
We have a case where we want to store time series data in memory. We
will have a lots of sensor data. So i think i will use sensor id as a
key to retrieve the time series from cache.
However i can't find any sorted list structure in ignite to store our
time series. The index can be long ( for time ). So it will need to be
sorted by index.
We also have a query for getting range of index, ex: Give me all
series from start idx to end idx.
For updating we also need to be able to update a range with new data
series.
We just don't want to re uploaded the data again over and over again
to cache everytime there's an update on the range. We want to be able
to update the cache partially based on the range.
Is there any way we can achieve this on Ignite ?
Any suggestion or reference would be really appreciated.
Cheers
--
Welly Tambunan
Triplelands
http://weltam.wordpress.com
http://www.triplelands.com <http://www.triplelands.com/blog/>
