Re: Using 3rd party DB together with native persistence (WAS: GettingInvalid state exception when Persistance is enabled.)
Hi It is very easy for user to shoot himself in a foot... and they do it again and again e.g. like this one [1]. [1] http://apache-ignite-users.70518.x6.nabble.com/Data-Loss-while-upgrading-custom-jar-from-old-jar-in-server-and-client-nodes-td20505.html On Thu, Mar 8, 2018 at 11:28 PM, Dmitriy Setrakyan wrote: > To my knowledge, the 2.4 release should have support for both persistence > mechanisms, native and 3rd party, working together. The release is out for > a vote already: > http://apache-ignite-developers.2346864.n4.nabble. > com/VOTE-Apache-Ignite-2-4-0-RC1-td27687.html > > D. > > On Mon, Feb 26, 2018 at 2:43 AM, Humphrey wrote: > >> I think he means when *write-through* and *read-through* modes are >> enabled on >> the 3rd party store, data might be written/read to/from one of those >> persistence storage (not on both). >> >> So if you save data "A" it might be stored in the 3rd party persistence, >> and >> not in the native. When data "A" is not in the cache it might try to look >> it >> up from the native persistence, where it's not available. Same could >> happen >> with updates, if "A" was updated to "B" it could have changed in the 3rd >> party but when requesting for the data again you might in one case get "A" >> an other case "B" depending on the stores it reads the data from. >> >> At least that is what I understand from his consistency between both >> stores. >> >> >> >> -- >> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >> > >
Re: Using 3rd party DB together with native persistence (WAS: GettingInvalid state exception when Persistance is enabled.)
To my knowledge, the 2.4 release should have support for both persistence mechanisms, native and 3rd party, working together. The release is out for a vote already: http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Apache-Ignite-2-4-0-RC1-td27687.html D. On Mon, Feb 26, 2018 at 2:43 AM, Humphrey wrote: > I think he means when *write-through* and *read-through* modes are enabled > on > the 3rd party store, data might be written/read to/from one of those > persistence storage (not on both). > > So if you save data "A" it might be stored in the 3rd party persistence, > and > not in the native. When data "A" is not in the cache it might try to look > it > up from the native persistence, where it's not available. Same could happen > with updates, if "A" was updated to "B" it could have changed in the 3rd > party but when requesting for the data again you might in one case get "A" > an other case "B" depending on the stores it reads the data from. > > At least that is what I understand from his consistency between both > stores. > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Using 3rd party DB together with native persistence (WAS: GettingInvalid state exception when Persistance is enabled.)
I think he means when *write-through* and *read-through* modes are enabled on the 3rd party store, data might be written/read to/from one of those persistence storage (not on both). So if you save data "A" it might be stored in the 3rd party persistence, and not in the native. When data "A" is not in the cache it might try to look it up from the native persistence, where it's not available. Same could happen with updates, if "A" was updated to "B" it could have changed in the 3rd party but when requesting for the data again you might in one case get "A" an other case "B" depending on the stores it reads the data from. At least that is what I understand from his consistency between both stores. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Using 3rd party DB together with native persistence (WAS: GettingInvalid state exception when Persistance is enabled.)
Hi Stan, Could you please elaborate more on this point "*but if cache configured with write-through or read-through modes enabled then data consistency between Ignite persistence and CacheStore is not guaranteed at all times*" . If I have enabled ignite persistence and my cache is also read-through and write-through then why does it not guarantee 100% data consistency? Thanks, Prasad On Wed, Feb 21, 2018 at 7:29 PM, Stanislav Lukyanov wrote: > Sorry, I wasn’t completely correct. Persistence and CacheStore can work > together, but if cache configured with write-through or read-through modes > enabled then data consistency between Ignite persistence and CacheStore is > not guaranteed at all times, so such configurations are usually avoided. > However, from what you’re saying it seems that you don’t need write-through > or read-through, so it should work fine. > > Your solution seems OK. If you only need a one-time setup I guess you > could also do it without a CacheStore and just get the data from Oracle and > put it into Ignite via putAll or DataStreamer. > > > > Thanks, > > Stan > > > > *From: *Prasad Bhalerao > *Sent: *21 февраля 2018 г. 14:33 > *To: *user@ignite.apache.org > *Subject: *Re: Using 3rd party DB together with native persistence (WAS: > GettingInvalid state exception when Persistance is enabled.) > > > > Hi Stan, > > > > Thank you for the reply. I will send different questions separately now > onwards. > > > > I do not understand what are you trying to say ( "*Ignite doesn’t support > using 3rd party DBs and native persistence with the same cache*" ) . > > > > If the persistence is enabled and if you create or load the cache from 3rd > party DB or any other source, data is always stored/backed in ignite native > persistence store. > > > > I have enable the persistence using IgniteConfiguration. I believe that > this configuration is global and so it is applicable to all the caches I > have created. As per my understanding, if persistence is enabled any cache > created will be stored/backed in ignite persistence store. > > So now if I load the data in cache from oracle or any other DB table, it > will be persisted in ignite native persistence file system. The point is I > want to load the cache from oracle tables only first time or when the cache > is empty. To load the cache from 3rd party DB I am using cache.loadCache > method. > > > > I am checking cache size, if the size is zero I call loadCache method. If > the cache is not empty it means that data is already loaded in previous > attempt so no need to call loadCache method. > > I just wanted to know if there is any better solution to achieve this. > > > > > > Thanks, > > Prasad > > > > On Wed, Feb 21, 2018 at 1:33 PM, Stanislav Lukyanov < > stanlukya...@gmail.com> wrote: > > Hi Prasad, > > > > // Please send different questions separately – this way it’s easier to > answer and to search for existing answers > > > > > Also, I am loading the cache from oracle table using loadCache method. > If the persistence is enabled and if the data is already persisted, I want > to make sure that the cache is loaded from persisted data instead of > loading it from oracle table using loadCache. Can someone please advise how > this can be achieved? > > > > > > Ignite doesn’t support using 3rd party DBs and native persistence with the > same cache. > > If you need to use both, I’d suggest to create two caches, one backed by > Oracle and one with enabled Ignite persistence, and alternate between them > in your application code. > > > > Thanks, > > Stan > > > > *From: *Prasad Bhalerao > *Sent: *20 февраля 2018 г. 15:24 > *To: *user@ignite.apache.org > *Subject: *Getting Invalid state exception when Persistance is enabled. > > > > Hi, > > > > I am starting ignite node in server mode in intellij. I am starting only > one instance of it. I am using IgniteSpringBean to set configuration and > start the node as shown below. But when I enable persistence, I get > following exception. > > > > Caused by: java.lang.IllegalStateException: Ignite is in invalid state to > perform this operation. It either not started yet or has already being or > have stopped [ignite=null, cfg=null] > > > > As per the doc, IgniteSpringBean is responsible for starting the ignite. > So how to set node to active state in case this case? > > > > Also, I am loading the cache from oracle table using loadCache method. If > the persistence is enabled and if the data is already persisted, I want to > make sure that t
RE: Using 3rd party DB together with native persistence (WAS: GettingInvalid state exception when Persistance is enabled.)
Sorry, I wasn’t completely correct. Persistence and CacheStore can work
together, but if cache configured with write-through or read-through modes
enabled then data consistency between Ignite persistence and CacheStore is not
guaranteed at all times, so such configurations are usually avoided. However,
from what you’re saying it seems that you don’t need write-through or
read-through, so it should work fine.
Your solution seems OK. If you only need a one-time setup I guess you could
also do it without a CacheStore and just get the data from Oracle and put it
into Ignite via putAll or DataStreamer.
Thanks,
Stan
From: Prasad Bhalerao
Sent: 21 февраля 2018 г. 14:33
To: user@ignite.apache.org
Subject: Re: Using 3rd party DB together with native persistence (WAS:
GettingInvalid state exception when Persistance is enabled.)
Hi Stan,
Thank you for the reply. I will send different questions separately now onwards.
I do not understand what are you trying to say ( "Ignite doesn’t support using
3rd party DBs and native persistence with the same cache" ) .
If the persistence is enabled and if you create or load the cache from 3rd
party DB or any other source, data is always stored/backed in ignite native
persistence store.
I have enable the persistence using IgniteConfiguration. I believe that this
configuration is global and so it is applicable to all the caches I have
created. As per my understanding, if persistence is enabled any cache created
will be stored/backed in ignite persistence store.
So now if I load the data in cache from oracle or any other DB table, it will
be persisted in ignite native persistence file system. The point is I want to
load the cache from oracle tables only first time or when the cache is empty.
To load the cache from 3rd party DB I am using cache.loadCache method.
I am checking cache size, if the size is zero I call loadCache method. If the
cache is not empty it means that data is already loaded in previous attempt so
no need to call loadCache method.
I just wanted to know if there is any better solution to achieve this.
Thanks,
Prasad
On Wed, Feb 21, 2018 at 1:33 PM, Stanislav Lukyanov
wrote:
Hi Prasad,
// Please send different questions separately – this way it’s easier to answer
and to search for existing answers
> Also, I am loading the cache from oracle table using loadCache method. If the
> persistence is enabled and if the data is already persisted, I want to make
> sure that the cache is loaded from persisted data instead of loading it from
> oracle table using loadCache. Can someone please advise how this can be
> achieved?
Ignite doesn’t support using 3rd party DBs and native persistence with the same
cache.
If you need to use both, I’d suggest to create two caches, one backed by Oracle
and one with enabled Ignite persistence, and alternate between them in your
application code.
Thanks,
Stan
From: Prasad Bhalerao
Sent: 20 февраля 2018 г. 15:24
To: user@ignite.apache.org
Subject: Getting Invalid state exception when Persistance is enabled.
Hi,
I am starting ignite node in server mode in intellij. I am starting only one
instance of it. I am using IgniteSpringBean to set configuration and start the
node as shown below. But when I enable persistence, I get following exception.
Caused by: java.lang.IllegalStateException: Ignite is in invalid state to
perform this operation. It either not started yet or has already being or have
stopped [ignite=null, cfg=null]
As per the doc, IgniteSpringBean is responsible for starting the ignite. So how
to set node to active state in case this case?
Also, I am loading the cache from oracle table using loadCache method. If the
persistence is enabled and if the data is already persisted, I want to make
sure that the cache is loaded from persisted data instead of loading it from
oracle table using loadCache. Can someone please advise how this can be
achieved?
Code to config ignite and cache:
@Bean
public IgniteSpringBean igniteInstance() {
IgniteSpringBean ignite = new IgniteSpringBean();
ignite.active(true);
ignite.setConfiguration(getIgniteConfiguration());
return ignite;
}
private IgniteConfiguration getIgniteConfiguration() {
String HOST = "127.0.0.1:47500..47509";
TcpDiscoveryVmIpFinder ipFinder = new TcpDiscoveryVmIpFinder();
ipFinder.setAddresses(Collections.singletonList(HOST));
TcpDiscoverySpi discoSpi = new TcpDiscoverySpi();
discoSpi.setIpFinder(ipFinder);
IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setDiscoverySpi(discoSpi);
cfg.setIgniteInstanceName("springDataNode");
cfg.setPeerClassLoadingEnabled(false);
cfg.setRebalanceThreadPoolSize(4);
DataStorageConfiguration storageCfg = new DataStorageConfiguration();
storageCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(true);
cfg.setDataStorageConfiguration(storageCfg);
