Hello Steve, Your use case perfectly well fits Ignite’s capabilities. Please see inline for the rest.
> On Dec 11, 2017, at 6:00 AM, Steven van der Merwe > <stevevanderme...@gmail.com> wrote: > > Hi All > > I am new to Ignite and I am trying to understand if Ignite will fit my use > case (I hope so). It seems to have all of the aspects I am looking for. > > I am trying to do the following: > I am building up a realtime analytics / business dashboard based on a whole > lot of metrics and messages in the integration space (utilising something > like a time series database). > I would like to stream messages from a number of different data collectors (I > am using queues at the moment for this task) > At the moment I am putting these messages into Graylog / Elasticsearch but I > am looking to change this > I need to store the time series data in a DB (About 1-5 million messages a > day) > I want to evict / delete the data after a month or so (if space is a problem) > I need to do some basic aggregation and CEP > I would like to aggregate the data over time (and thus reduce the granularity > / size) > > I guess my questions are the following: > Can I start using Ignite (in a single (or a few) nodes initially) as a SQL / > KV store (I will probably persist the data to disk very soon only keeping a > few 10-20GB in RAM and use it more like a traditional “DB”)? Yes, the documentation section below explains how to define how much memory to dedicate in RAM and how to enable the persistence: https://apacheignite.readme.io/docs/memory-configuration#section-data-regions > My concern is the Ignite was not designed for this (as it seems mostly in > memory) so I am unsure about how it will work persisting 500-1000GB of data > on a single node (I guess I am trying to use it more as a hybrid in memory > store) Ignite 2.x is based on a completely new memory architecture where disk stores a subset of data and RAM caches as much data as you can afford. https://apacheignite.readme.io/docs/durable-memory So, presently Ignite is also used as a system of records or classical DB. > The documentation seems to refer to using it as a cache in front of a RDBMS > but I would like to just use Ignite as the only DBMS What’s the documentation you’re referring to? There are many documentation and use cases pages Ignite is used for. For instance, the use cases on the site unveil how to use it as a DBMS: https://ignite.apache.org/usecases.html > Can I aggregate the data using time as a “grouping" field ? > I will delete the expired data from the db periodically > I want to do CEP on the data > Yes Denis > Thanks for the help in advance. > > Regards > Steve
