HI, Mikael
Thanks for you reply very much!
The type of data like this:
member [name, location, age, gender, hobby, level, credits, expense
...]
We need filter data by arbitrary fileds combination, so creating index
is not of much use. We thought traveling all data in memory works better.
We can keep all data in ram, but data may increase progressisvely,
single node is not scalable. So we plan to use a distribute memory cache.
We store data off heap and all in ram with default ignite
serialization. We just create table, then populate data with default
configuration in ignite, query by sql(one node, 4 million records ).
Is there anyway can improve query performance ?
Mikael <[email protected]> 于2019年11月21日周四 下午5:02写道：
> Hi!
>
> The comparison is not of much use, when you talk about ignite, it's not
> just to search a list, there is serialization/deserialization and other
> things to consider that will make it slower compared to a simple list
> search, a linear search on an Ignite cache depends on how you store data
> (off heap/on heap, in ram/partially on disk, type of serialization and
> so on.
>
> If you cannot keep all data in ram you are going to need some index to
> do a fast lookup, there is no way around it.
>
> If you can have all the data in ram, why do you need Ignite ? do you
> have some other requirements for it that Ignite gives you ? otherwise it
> might be simpler to just use a list in ram and go with that ?
>
> Is memory a limitation (cluster or single node ?) ? if not, could you
> explain why is it difficult to create an index on the data ?
>
> Could you explain what type of data it is ? maybe it is possible to
> arrange the data in some other way to improve everything
>
> Did you test with a single node or a cluster of nodes ? with more nodes
> you can improve performance as any search can be split up between the
> nodes, still, some kind of index will help a lot.
>
> Mikael
>
> Den 2019-11-21 kl. 08:49, skrev c c:
> > HI,
> > We have a table with about 30 million records and 15 fields. We
> > need implement function that user can filter record by arbitrary 12
> > fields( one，two, three...of them) with very low latency. It's
> > difficult to create index. We think ignite is a grid memory cache and
> > test it with 4 million records(one node) without creating index. It
> > took about 5 seconds to find a record match one field filter
> > condition. We have tested just travel a java List(10 million elements)
> > with 3 filter condition. It took about 0.1 second. We just want to
> > know whether ignite suit this use case? Thanks very much.
> >
>
