Hi Denis, Thanks for your clear explanation.
Our data structure is something like this one <sensorid: UUID, time: Long, value: Double> When i put a data point with that composite key. <sensorid, time>, is there any guarantee that it will be store close together in a same node ? In case of update, however we need to be able to update in range, like replace the range <startId, endIdx, List[DataPoint]>, (where DataPoint => <time, value> ) so it will clear that range and insert the new data point into that range. So we need to do two step process to do that ? Select the key and then delete the key ? Thanks On Thu, Dec 3, 2015 at 7:00 PM, Denis Magda <[email protected]> wrote: > Hi Welly, > > Ignite perfectly fits for your task. > > First, if I understand you properly there are will be many time series for > a give sensor ID. > If so then I would use a compound key like (sensorId, time) for all cache > related operations. > > As an example you may want to use classes like this. > > SampleKey { > int sernsorId; > long sampleTime; > } > > Sample { > int sensorId; > long sampleTime; > byte[] data1; > byte[] data2; > etc. > } > > And use them this way > > cache.put(new SampleKey(1, time), sample); > cache.get(new SampleKey(2, time)); > > Second, to retrieve samples depending on sensor ID, time (time range) or > other parameters you can leverage Ignite SQL engine that is designed > exactly for the use cases you have. [1] > However, if you're going to use an object field in 'SELECT' or 'WHERE' > clause you have to annotate it properly or specify using CacheTypeMetadata > [2] > > Third, when you need to update data series you can remove the old one and > insert the new one that should have new time. > To perform a remove I would suggest doing the following: > - select sensor ID and sampleTime of all the entries to delete; > - use cache.removeAll by passing SampleKeys that are created using the > data retrieved with SQL above. > > Moreover, you can use an eviction or expire policy that is used in cases > when old data must be removed from cache automatically. > Just refer to these articles for more info - [3] > > Finally, Ignite has bunch of cache and SQL related examples. They are > located in "datagrid" folder of "examples" module. > Have a look at them and probably you'll come up with better solution based > on Ignite that suggested by me above cause definitely you know all the > details of your case better ;) > > [1] https://apacheignite.readme.io/docs/sql-queries > [2] > https://apacheignite.readme.io/docs/sql-queries#configuring-sql-indexes-by-annotations > [3] https://apacheignite.readme.io/docs/evictions > [4] https://apacheignite.readme.io/docs/expiry-policies > > > Regards, > Denis > > > On 12/3/2015 12:50 PM, Welly Tambunan wrote: > > Hi Igniters, > > Currently we are trying to asses the possibility of using Ignite on our > Architecture. > > We have a case where we want to store time series data in memory. We will > have a lots of sensor data. So i think i will use sensor id as a key to > retrieve the time series from cache. > > However i can't find any sorted list structure in ignite to store our time > series. The index can be long ( for time ). So it will need to be sorted by > index. > > > We also have a query for getting range of index, ex: Give me all series > from start idx to end idx. > For updating we also need to be able to update a range with new data > series. > > We just don't want to re uploaded the data again over and over again to > cache everytime there's an update on the range. We want to be able to > update the cache partially based on the range. > > Is there any way we can achieve this on Ignite ? > > Any suggestion or reference would be really appreciated. > > Cheers > > -- > Welly Tambunan > Triplelands > > http://weltam.wordpress.com > http://www.triplelands.com <http://www.triplelands.com/blog/> > > > -- Welly Tambunan Triplelands http://weltam.wordpress.com http://www.triplelands.com <http://www.triplelands.com/blog/>
