That looks like something you could do with Kafka connect https://www.confluent.io/product/connectors/ using the jdbc source and the ignite sink. Just my 2 cents
On Mon, Aug 6, 2018, 07:19 Deepa Kolwalkar <[email protected]> wrote: > Thanks Prasad for your suggestions. > > The Legacy sytems are on different platforms and some of them are products > .. so there is no way of implementing any custom logic in such products to > send update messages. The Legacy Systems remain a black-box to us with only > its DB which is accessible for viewing.. > > Regards > > > > > > From: "Prasad Bhalerao" <[email protected]> > To: [email protected] > Date: 06-08-2018 16:51 > Subject: Re: what are the tehcniques to automatically detect > changes in Multiple DBs and automatically push them into Ignite Cache > ------------------------------ > > > > Can this back office legacy system send you a DB update message or can you > make this back office system to send you DB update message? > > If yes then you can have the Id/primary key, DB operation and table name > in this DB update message. > > In your application you use this information to refresh your cache using > read through mechanism. > > Thanks, > Prasad > > On Mon, Aug 6, 2018, 3:02 PM Deepa Kolwalkar <*[email protected]* > <[email protected]>> wrote: > Thanks Denis. > > But as I mentioned in earlier mail, the Caches are meant to be Read-Only > (only to be used by Microservices for fetching data). > The Databases are updated by backoffice legacy systems. Hence we cannot > do a Write-through to the DBs via the CacheStore API. > > If anyone has used the Gridgain GoldenGate Adapter, then we would be glad > to hear about any challenges/short-comings if any. > > Regards > > > > From: Denis Mekhanikov <*[email protected]* > <[email protected]>> > To: *[email protected]* <[email protected]> > Date: 06-08-2018 13:18 > Subject: Re: what are the tehcniques to automatically detect > changes in Multiple DBs and automatically push them into Ignite Cache > ------------------------------ > > > > There is no such feature in Ignite. > If you know, how to subscribe for events in the external database, then > you can implement this logic yourself. > You just need to perform put into Ignite cache for every insert into the > external DB. > > But the recommended way to do it is to perform writing on Ignite. > Cache store with *write-through* > <https://apacheignite.readme.io/v2.6/docs/3rd-party-store#section-read-through-and-write-through> > enabled > will take care of writing the data into the external DB. > > Denis > > вс, 5 авг. 2018 г. в 17:32, Deepa Kolwalkar <*[email protected]* > <[email protected]>>: > We have a requirement where Changes to data from Multiple DBs need to be > periodically & automatically Pushed (not sure how) into various Ignite > Caches > Once the Data is available in the Ignite Caches, it will be persisted > using Ignite Native Persistence, so that in the event of a crash, the Data > can be loaded from Native Persistence. > The Caches will be used in read-only manner by Clients (Microservices) . > > What is the Best technique for having Changes to data from Multiple DBs to > be automatically put into the Ignite Caches ? > > While searching for this solution i came across this link : > > *http://apache-ignite-users.70518.x6.nabble.com/Any-references-Syncing-Ignite-and-Oracle-DB-with-Oracle-GoldenGate-updates-from-DB-to-ignite-td20715.html* > <http://apache-ignite-users.70518.x6.nabble.com/Any-references-Syncing-Ignite-and-Oracle-DB-with-Oracle-GoldenGate-updates-from-DB-to-ignite-td20715.html> > which suggests the following : > ============== > Ignite does not provide such integration out of the box, however there > a commercial offering from GridGain for that: > *https://docs.gridgain.com/docs/goldengate-replication* > <https://docs.gridgain.com/docs/goldengate-replication> > ============== > > Was wondering whether we still need to use GoldenGate for such > replications OR whether newer versions of Ignite are now supporting such > asynchronous sync-ups with underlying DB changes > > Thanks > =====-----=====-----===== > Notice: The information contained in this e-mail > message and/or attachments to it may contain > confidential or privileged information. If you are > not the intended recipient, any dissemination, use, > review, distribution, printing or copying of the > information contained in this e-mail message > and/or attachments to it are strictly prohibited. If > you have received this communication in error, > please notify us by reply e-mail or telephone and > immediately and permanently delete the message > and any attachments. Thank you >
