Hi Sven, I will let others comment on the materialized view suggestion, but I have another question.
*As we all know, joins are a nightmare in a distributed system* Have you considered collocation or replication? If a table is replicated, then it will be present on all the nodes and all joins will be fast. If two partitioned tables are colocated based on some affinity key, then joins on that affinity key will be fast as well. Both, colocation and replication are supported by Ignite. Will any of these approaches work for you? D. On Mon, Sep 17, 2018 at 11:04 AM Sven Beauprez <sven.beaup...@theglue.com> wrote: > All, > > > > We are in a situation where we have to query data over several caches. As > we all know, joins are a nightmare in a distributed system and I know there > are other means like denormalisation, but it is not sufficient anymore in > some cases we have and we need the joins. > > > > We mainly work in an OLTP context, where queries are known in advance (ie > dev time) and inpsired by following blog of several years ago, I was > wondering if the concept of “materialized views” could make it into Apache > Ignite. > > ( > https://www.confluent.io/blog/turning-the-database-inside-out-with-apache-samza/ > ) > > > > It would work as follows: > > - A query must register itself in Ignite at startup time (eg. via > configuration) or during run time (eg. API call) > - The registered query is parsed and a new “view” cache is created > which will ‘cache’ the resultset of the query (could take a while, but > intermediate status can be “warming up” and “hot” when ready) > - All caches involved in the joins are now monitored for CUD > operations and relevant data is stored in the new “view” cache so the view > gets updated in real time > - All operations must be ACID compliant > - The view is queried via a very trivial select statement > > > > Would that be feasible as a new feature? > > > > > > Regards, > > > > Sven > > > > > > > > [image: cid:image001.png@01D3007B.4D007790] > > > > SVEN BEAUPREZ > > > > L e a d A r c h i t e c t > > > > De Kleetlaan 5, B-1831 Diegem > > www.theglue.com >
