Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

[Denis Magda]

Ray,

This looks odd. Sure, data has to be available for you after the restart,
it's a basic capability. Then I tend to agree with the rest of the folks
that you might haven encountered the issue mentioned by Yakov. To to be
100% sure about that please do the following:

* Clean "work/db" directory completely to avoid side effects.
* Run the app that preloads data. Copy the generated logs somewhere.
* Restart the cluster and start the second app.
* Compare the first and second logs checking that the "page store" and WAL
directories are identical in both files.
* If the directories vary then you hit issue [1] and let us know if this
the case. We have to come up how to solve this automatically. As a
workaround try to send IgniteConfiguration.setConsistentId() to a
predefined value.

[1] https://issues.apache.org/jira/browse/IGNITE-6285

--
Denis

On Fri, Sep 15, 2017 at 6:52 PM, Ray  wrote:

> Yes, I tried to work with the data after restarting.
> As I stated in the original post, I can't see any cache after I restart the
> cluster.
> So I called
> https://apacheignite.readme.io/docs/rest-api#section-get-or-create-cache
> API
> with the exact cache name before restarting as parameter.
> Then I try to run a sql query with that cache name and I got an empty
> cache.
> If the data loading from disk to RAM is transparent, I should see the data
> before restarting, right?
>
> I agree with you Dmitriy, according to the debug log, the WAL folder is
> resolved.
> And I can confirm there're several WAL logs under that folder.
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

[Denis Magda]

Dmitriy,

Check up Ray’s initial message. There is no single cache.get command or SQL 
query that can trigger data movement from disk to RAM. Plus, the output 
generated after the restart suggests that the node found all the persistence 
directories. Thus, not sure how this issue is related to consistent ID mess 
discovered by Yakov before.

Ray, please chime in and clarify if you tried to work with the data after the 
restart. 

—
Denis

> On Sep 15, 2017, at 4:02 PM, Dmitry Pavlov  wrote:
> 
> Hi Denis,
> 
> I agree that data presence should be double-checked. In case check will show 
> that 1) cache doesn't exist at all after restart and 2) we can find two 
> folders with binary_meta and wal that means this question is similar with 
> recent question 
> http://apache-ignite-users.70518.x6.nabble.com/Specifying-location-of-persistent-storage-location-tc16636.html
>  
> 
>  
> 
> It is completely transparent to user that data is in RAM or on disk.
> 
> 
> Hi Yakov,
> 
> To my mind ticket is reasonable. I can suggest to generate such warning only 
> in case consistent ID is not overriden by IgniteConfiguration. Parameter 
> override may indicate that several nodes running at the same machine.
> 
> Sincerely,
> Dmitriy Pavlov
> 
> сб, 16 сент. 2017 г. в 1:47, Denis Magda  >:
> Alexey,
> 
> The data preloading from disk to RAM doesn’t occur automatically after the 
> restart. It will be moved to RAM on demand while your application executes 
> its logic.
> 
> Ray,
> 
> Have you executed any query (key-value, SQL) to confirm that the data is 
> missing after the shutdown?
> 
> —
> Denis
> 
> > On Sep 15, 2017, at 4:09 AM, Alexey Kukushkin  > > wrote:
> >
> > Ray,
> >
> > Yes, enabling persistence must automatically bring all your data and 
> > metadata upon the cluster startup. Please give us more details why you 
> > think the cache was not created. BTW, do you activate the cluster after 
> > startup? Enabling persistence requires explicit cluster activation:
> >
> > $IGNITE_HOME/bin/control.sh --host 127.0.0.1 --port 11211 --activate
> >
> >
> 

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

[Dmitry Pavlov]

Hi Denis,

I agree that data presence should be double-checked. In case check will
show that 1) cache doesn't exist at all after restart and 2) we can find
two folders with binary_meta and wal that means this question is similar
with recent question
http://apache-ignite-users.70518.x6.nabble.com/Specifying-location-of-persistent-storage-location-tc16636.html


It is completely transparent to user that data is in RAM or on disk.


Hi Yakov,

To my mind ticket is reasonable. I can suggest to generate such warning
only in case consistent ID is not overriden by IgniteConfiguration.
Parameter override may indicate that several nodes running at the same
machine.

Sincerely,
Dmitriy Pavlov

сб, 16 сент. 2017 г. в 1:47, Denis Magda :

> Alexey,
>
> The data preloading from disk to RAM doesn’t occur automatically after the
> restart. It will be moved to RAM on demand while your application executes
> its logic.
>
> Ray,
>
> Have you executed any query (key-value, SQL) to confirm that the data is
> missing after the shutdown?
>
> —
> Denis
>
> > On Sep 15, 2017, at 4:09 AM, Alexey Kukushkin 
> wrote:
> >
> > Ray,
> >
> > Yes, enabling persistence must automatically bring all your data and
> metadata upon the cluster startup. Please give us more details why you
> think the cache was not created. BTW, do you activate the cluster after
> startup? Enabling persistence requires explicit cluster activation:
> >
> > $IGNITE_HOME/bin/control.sh --host 127.0.0.1 --port 11211 --activate
> >
> >
>
>

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

[Alexey Kukushkin]

Ray,

Yakov just pointed me to this thread

where
user had a problem with IP address changing between the cluster restarts.
The persistent data directories inside work/db have names built as a
concatenation of local IP addresses and ports. Thus, if your node receives
a different IP address between the restarts, you will not have your
persistent data since the node will initialise from a different persistent
data file. You would need to bind your discovery SPI to specific address to
work around that. There is an issue opened to properly fix it.

Please also consider that as a potential cause.

Re: Ignite cluster with persistent store enabled did not load from wal after restarting.

[Yakov Zhdanov]

Guys, it seems that we have the same issue that was described in thread -
Specifying location of persistent storage location

I filed a ticket some time ago -
https://issues.apache.org/jira/browse/IGNITE-6285, Alex G, can you take a
look if this ticket makes sense?

--Yakov