Raymond, Then Ignite Persistent Store is exactly for your use case. Please refer to this discussion on the dev list: http://apache-ignite-developers.2346864.n4.nabble.com/GridGain-Donates-Persistent-Distributed-Store-To-ASF-Apache-Ignite-td16788.html#a16838
Also it was covered a bit in that webinar: https://www.youtube.com/watch?v=bDrGueQ16UQ The store should be released by the community in the nearest. — Denis > On Jun 13, 2017, at 2:02 PM, Raymond Wilson <[email protected]> > wrote: > > Hi Pavel, > > It’s a little complicated. The system is essentially a DB in its own right; > actually it’s an IMDG a bit like Ignite, but developed 8 years ago to fulfill > a need we had. > > Today, I am looking to modernize that system and rather than continuing to > build and maintain all the core ‘infrastructure’ features of an IMDG such as > clustering, messaging, enterprise caching etc, I am looking to see how well > Ignite fits by running a Proof of Concept project. It turns out it fits quite > well, largely because the architectural structure of both systems (ie: IMDG) > is well aligned in terms of the problems being solved. > > The primary gap between the legacy system and IMDG is that IMDG does not > support persistence. The legacy system has a distributed cache that stores > objects that are aggregate collections (10’s of thousand’s) of relatively > simple spatial data records that are operated on by the clustered compute > engine. Sometimes billions of records need to be processed to satisfy a > single query. Your standard run of the mill SQL DB finds these sorts of > queries hard. > > I suppose you could use another DB (MS-SQL, AWS:RDS etc) to store those > aggregate blobs, but it seems like a bit of a ‘miss-use case’ when what I’m > really after is a persistence/storage layer J > > Thanks, > Raymond. > > > From: Pavel Tupitsyn [mailto:[email protected] > <mailto:[email protected]>] > Sent: Wednesday, June 14, 2017 1:59 AM > To: [email protected] <mailto:[email protected]> > Subject: Re: Write behind using Grid Gain > > Hi Raymond, > > I think your use case fits well into traditional Ignite model of > write-through cache store with backing database. > Why do you want to avoid a DB? Do you plan to store data on disk directly as > a set of files? > > Pavel > > On Mon, Jun 12, 2017 at 2:14 AM, Raymond Wilson <[email protected] > <mailto:[email protected]>> wrote: > Hi Pavel, > > Thanks for the blog – it explains it quite well. > > I have a slightly different use case where groups of records within a much > larger data set are clustered together for efficiency (ie: each of the cached > items in the Ignite grid cache has significant internal structure). You can > think of them as a large number of smallish files (a few Kb to a few Mb), but > file systems don’t like lots of small files. > > I have a legacy implementation that houses these small files within a single > larger file, but wanted to know if there was a clean way of supporting the > same structure using the Ignite read/write through support, perhaps with > another system providing relatively transparent persistency semantics but > which does not use a DB to store the data. > > Thanks, > Raymond. > > From: Pavel Tupitsyn [mailto:[email protected] > <mailto:[email protected]>] > Sent: Saturday, May 27, 2017 5:03 AM > To: [email protected] <mailto:[email protected]> > > Subject: Re: Write behind using Grid Gain > > I've decided to write a blog post, since this topic seems to be in demand: > https://ptupitsyn.github.io/Ado-Net-Cache-Store/ > <https://ptupitsyn.github.io/Ado-Net-Cache-Store/> > > Code: > https://github.com/ptupitsyn/ignite-net-examples/tree/master/AdoNetCacheStore > <https://github.com/ptupitsyn/ignite-net-examples/tree/master/AdoNetCacheStore> > > Let me know if this helps! > > On Fri, May 26, 2017 at 3:50 PM, Chetan D <[email protected] > <mailto:[email protected]>> wrote: > Thank you Pavel. > > waiting for your response. > > On Fri, May 26, 2017 at 6:03 PM, Pavel Tupitsyn <[email protected] > <mailto:[email protected]>> wrote: > To give everyone context, this is not about GridGain, but about Apache Ignite. > The blog post in question is > https://ptupitsyn.github.io/Entity-Framework-Cache-Store/ > <https://ptupitsyn.github.io/Entity-Framework-Cache-Store/> > > Chetan, I'll prepare an example with Ignite 2.0 / ado.net <http://ado.net/> > and post it some time later. > > Pavel > > On Fri, May 26, 2017 at 2:32 PM, Chetan D <[email protected] > <mailto:[email protected]>> wrote: > ++ User List > > any help much appreciated. > > Thanks And Regards > Chetan D > ---------- Forwarded message ---------- > From: Pavel Tupitsyn <[email protected] <mailto:[email protected]>> > Date: Fri, May 26, 2017 at 4:38 PM > Subject: Re: Write behind using Grid Gain > To: Chetan D <[email protected] <mailto:[email protected]>> > > Hi Chetan, can you please write this to our user list, [email protected] > <mailto:[email protected]>? > So that entire community can participate. > > Thanks, > Pavel > > On Fri, May 26, 2017 at 1:35 PM, Chetan D <[email protected] > <mailto:[email protected]>> wrote: > Hi Pavel Tupitsyn, > > I have posted a comment in your blog as well (entity framework as ignite .net > store) regarding write behind using ignite. > > I have been working on a project where i need to implement distributed > caching and i have been asked to look into grid gain. > > This is is the first time i am working on caching and this is entirely new > topic for me. > > The example which you have shared i was able to understand a little and the > sad part is even entity framework also i have never worked on. > > It would be helpful if you can share me a simple example using ado.net > <http://ado.net/> implementing read through, write through and write behind > even a simple table helps me understand the concept.
