Re: Persistence per memory policy configuration

2017-10-25 Thread Alexey Goncharuk
The changes are implemented in ignite-6748 branch. TC run is pending. 2017-10-25 11:26 GMT+03:00 Alexey Goncharuk : > Igniters, > > I've found one more shortcoming which we missed during the review. The > checkpoint page buffer is attributed to a data region, so

Re: Persistence per memory policy configuration

2017-10-25 Thread Alexey Goncharuk
Igniters, I've found one more shortcoming which we missed during the review. The checkpoint page buffer is attributed to a data region, so originally it was a mistake to place the checkpoint page buffer size to the global configuration. This property should go to the data region configuration. I

Re: Persistence per memory policy configuration

2017-10-17 Thread Denis Magda
Ivan, Please don’t forget to update all the persistence and memory pools related examples to the new configuration format. Let’s make sure non of our example prints out the class deprecated warning. Denis On Tuesday, October 17, 2017, Dmitriy Setrakyan wrote: > Thanks

Re: Persistence per memory policy configuration

2017-10-17 Thread Dmitriy Setrakyan
Thanks Ivan! Let's make sure that every property gets sufficient javadoc for our users to understand. We should also document this configuration on readme. On Tue, Oct 17, 2017 at 3:06 PM, Ivan Rakov wrote: > Dmitriy, > > Please check description of

Re: Persistence per memory policy configuration

2017-10-17 Thread Ivan Rakov
Dmitriy, Please check description of https://issues.apache.org/jira/browse/IGNITE-6030, I've updated it with actual list of properties. Best Regards, Ivan Rakov On 17.10.2017 21:46, Dmitriy Setrakyan wrote: I am now confused. Can I please ask for the final configuration again? What will it

Re: Persistence per memory policy configuration

2017-10-17 Thread Dmitriy Setrakyan
I am now confused. Can I please ask for the final configuration again? What will it look like? On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk < alexey.goncha...@gmail.com> wrote: > Agree with Ivan. If we implemented backward compatibility, this would be > completely counterintuitive behavior,

Re: Persistence per memory policy configuration

2017-10-17 Thread Alexey Goncharuk
Agree with Ivan. If we implemented backward compatibility, this would be completely counterintuitive behavior, so +1 to keep the behavior as is. As for the swap path, I see nothing wrong with having it for in-memory caches. This is a simple overflow mechanism that works fine if you do not need

Re: Persistence per memory policy configuration

2017-10-16 Thread Dmitriy Setrakyan
Igniters, we should not have Swap in the property names. Swap is something that behaves completely differently from Ignite persistence, and I don't think it should be present within Ignite altogether. D. On Mon, Oct 16, 2017 at 11:17 AM, Vladimir Ozerov wrote: > +1 to

Re: Persistence per memory policy configuration

2017-10-16 Thread Vladimir Ozerov
+1 to Ivan's suggestion. пн, 16 окт. 2017 г. в 21:00, Ivan Rakov : > *swapPath* is ok for me. It is also consistent with *walPath* and > *walArchivePath*. > > Regarding persistencePath/storagePath, I don't like the idea when path > for WAL is implicitly changed, especially

Re: Persistence per memory policy configuration

2017-10-16 Thread Ivan Rakov
*swapPath* is ok for me. It is also consistent with *walPath* and *walArchivePath*. Regarding persistencePath/storagePath, I don't like the idea when path for WAL is implicitly changed, especially when we have separate option for it. WAL and storage files are already located under same

Re: Persistence per memory policy configuration

2017-10-16 Thread Pavel Tupitsyn
Igniters, another thing to consider: DataRegionConfiguration.SwapFilePath should be SwapPath, since this is actually not a single file, but a directory path. On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda wrote: > Seems I've got what you’re talking about. > > I’ve tried to

Re: Persistence per memory policy configuration

2017-10-13 Thread Denis Magda
Seems I've got what you’re talking about. I’ve tried to change the root directory (*persistencePath*) and saw that only data/indexes were placed to it while wal stayed somewhere in my work dir. It works counterintuitive and causes non productive discussions like we are in arguing about

Re: Persistence per memory policy configuration

2017-10-13 Thread Ivan Rakov
Denis, Data/index storage and WAL are located under the same root by default. However, this is not mandatory: *storagePath* and *walPath* properties can contain both absolute and relative paths. If paths are absolute, storage and WAL can reside on different devices, like this: storagePath:

Re: Persistence per memory policy configuration

2017-10-12 Thread Dmitriy Setrakyan
On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda wrote: > From what I see after running an example they are under the same root > folder and in different subdirectories. The root folder should be defined > by setPersistencePath as I guess. > If that is the case, then you are

Re: Persistence per memory policy configuration

2017-10-12 Thread Denis Magda
>From what I see after running an example they are under the same root folder and in different subdirectories. The root folder should be defined by setPersistencePath as I guess. Denis On Thursday, October 12, 2017, Dmitriy Setrakyan wrote: > On Thu, Oct 12, 2017 at 4:05

Re: Persistence per memory policy configuration

2017-10-12 Thread Dmitriy Setrakyan
On Thu, Oct 12, 2017 at 4:05 PM, Denis Magda wrote: > Both terms “persistence” and “storage” mean same to me. If to choose > between them I would go for “persistence” because this is how > we name the feature - “Ignite Persistence” [1] > Furthermore, by default WAL files,

Re: Persistence per memory policy configuration

2017-10-12 Thread Denis Magda
Both terms “persistence” and “storage” mean same to me. If to choose between them I would go for “persistence” because this is how we name the feature - “Ignite Persistence” [1]. Furthermore, by default WAL files, the archive and data/index files are located under the same root which is

Re: Persistence per memory policy configuration

2017-10-12 Thread Ivan Rakov
Name *setPersistencePath* looks confusing to me because both WAL and index/partition files storage provide persistence. That's why we separated API methods as *setWalPath* and *setStoragePath*. I think, *setStoragePath* is good enough as long as it's supported by explaining javadoc.* * Best

Re: Persistence per memory policy configuration

2017-10-11 Thread Dmitriy Setrakyan
Is the storage path the root folder for the persistence or only the root path for the main storage? On Wed, Oct 11, 2017 at 3:54 PM, Denis Magda wrote: > Ivan, > > Instead of “setStoragePath” I would suggest “setPersistencePath”. Left > some extra notes in the ticket. > > —

Re: Persistence per memory policy configuration

2017-10-11 Thread Denis Magda
Ivan, Instead of “setStoragePath” I would suggest “setPersistencePath”. Left some extra notes in the ticket. — Denis > On Oct 11, 2017, at 4:30 AM, Ivan Rakov wrote: > > Vladimir, > > Thanks for focusing on existing namings. Most of your suggestions really > sound

Re: Persistence per memory policy configuration

2017-10-11 Thread Ivan Rakov
Typo in my previous reply: *walArchivePath* -> *setWalArchivePath* Best Regards, Ivan Rakov On 11.10.2017 14:30, Ivan Rakov wrote: Vladimir, Thanks for focusing on existing namings. Most of your suggestions really sound better. I've posted my thoughts under your comment. By the way, we

Re: Persistence per memory policy configuration

2017-10-11 Thread Ivan Rakov
Vladimir, Thanks for focusing on existing namings. Most of your suggestions really sound better. I've posted my thoughts under your comment. By the way, we should decide two things: 1) Naming for methods for configuring store path. I suggest the following: *setStoragePath* - for partition

Re: Persistence per memory policy configuration

2017-10-11 Thread Vladimir Ozerov
Ivan, I left some comments in the ticket [1], please take a look. [1] https://issues.apache.org/jira/browse/IGNITE-6030?focusedCommentId=16200050=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16200050 On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov

Re: Persistence per memory policy configuration

2017-10-11 Thread Ivan Rakov
Igniters, https://issues.apache.org/jira/browse/IGNITE-6030 is ready and enqueued for TC run. PR: https://github.com/apache/ignite/pull/2828 Everyone interested in new data storage configuration API, please pay attention and review. Best Regards, Ivan Rakov On 09.10.2017 12:40, Pavel

Re: Persistence per memory policy configuration

2017-10-09 Thread Pavel Tupitsyn
Sounds good to me. On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov wrote: > Pavel, > > Sounds reasonable. > I suggest to include both "data" and "configuration" to make it even more > obvious: > > set/getDefaultDataRegionConfiguration > set/getDataRegionConfigurations > >

Re: Persistence per memory policy configuration

2017-10-09 Thread Ivan Rakov
Pavel, Sounds reasonable. I suggest to include both "data" and "configuration" to make it even more obvious: set/getDefaultDataRegionConfiguration set/getDataRegionConfigurations Best Regards, Ivan Rakov On 09.10.2017 10:51, Pavel Tupitsyn wrote: Sorry that I'm late to the party, but this

Re: Persistence per memory policy configuration

2017-10-09 Thread Pavel Tupitsyn
Sorry that I'm late to the party, but this looks inconsistent: DataStorageConfiguration defaultRegionConfiguration DataRegionConfiguration[] getDataRegions defaultRegionConfiguration + getRegionConfigurations - or - defaultDataRegion + getDataRegions Thoughts? On Mon, Oct 2, 2017 at 9:10 PM,

Re: Persistence per memory policy configuration

2017-10-02 Thread Ivan Rakov
Denis, Yes, you're right. All cache groups without specific data region configured will be persistent. And if you want to add another persistent data region, you should set *isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly. Best Regards, Ivan Rakov On 02.10.2017 21:01,

Re: Persistence per memory policy configuration

2017-10-02 Thread Denis Magda
Missed the point with defaults. Makes sense to me now. So to wrap this up, if I want to enable the persistence globally and don’t have any regions configured explicitly I need to take the default region and switch the persistence on for it. Is my understanding correct? — Denis > On Oct 2,

Re: Persistence per memory policy configuration

2017-10-02 Thread Ivan Rakov
Denis, why do you need to access an instance of the default region bean? If you want to set any parameter, just instantiate new bean with this parameter set (like in XML snipped below). Other parameters will be automatically initialized with their default values. Best Regards, Ivan Rakov On

Re: Persistence per memory policy configuration

2017-10-02 Thread Dmitriy Setrakyan
On Mon, Oct 2, 2017 at 7:28 PM, Denis Magda wrote: > > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > > > > In other data regions persistence

Re: Persistence per memory policy configuration

2017-10-02 Thread Denis Magda
>> >> > class="org.apache.ignite.configuration.DataStorageConfiguration"> >> >> >> > class="org.apache.ignite.configuration.DataRegionConfiguration"> >> >> >> >> >> >

Re: Persistence per memory policy configuration

2017-10-02 Thread Ivan Rakov
Agree with Alexey. Properties like *defaultDataRegionSize*, *isDefaultPersistenceEnabled* can confuse users who don't know that there's such thing as default data region. They can decide they are inherited by all data regions where size and persistence flag are not explicitly set. Let's get

Re: Persistence per memory policy configuration

2017-10-02 Thread Denis Magda
> To resolve this, I suggest to > introduce just another field defaultRegionConfiguration and get rid of > other defaults in DataStorageConfiguration. Won’t it complicate the configuration from a Spring XML file? I’m not an expert in Spring so how do I get defaultRegionConfiguration bean first

Re: Persistence per memory policy configuration

2017-10-02 Thread Alexey Kuznetsov
Guys, Please, do not add Boolean (non-primitive) to configurations. It is really the pain to handle them in tools like WebConsole. Enum would be better. I'm not insisting, but just a minor note. On Mon, Oct 2, 2017 at 10:30 PM, Alexey Goncharuk < alexey.goncha...@gmail.com> wrote: > Agree

Re: Persistence per memory policy configuration

2017-10-02 Thread Alexey Goncharuk
Agree with Vladimir. If we are to implement this, we would either need to have a Boolean (non-primitive) for persistenceEnabled on DataRegionConfiguration, or introduce an enum for this field which is also an overkill. On the other hand, one can assume that the defaults we are talking about are

Re: Persistence per memory policy configuration

2017-10-02 Thread Ivan Rakov
Vladimir, I like your approach because it's easier to implement. However, user may be confused by setting *isDefaultPersistenceEnabled* flag and seeing that persistence is not enabled by default in custom memory region. I'll add clarifying Javadoc at this place. Best Regards, Ivan Rakov On

Re: Persistence per memory policy configuration

2017-10-02 Thread Vladimir Ozerov
Ivan, I do not think this is correct approach, because it will be hard to explain, and you will have to use "Boolean" instead of "boolean" for DataRegionConfiguration. I do not think we need default "persistence enabled" for all regions. Instead, we should have "persistence enabled" flag for

Re: Persistence per memory policy configuration

2017-10-02 Thread Ivan Rakov
Guys, I think I got the point now. Let's check the final design: *DataStorageConfiguration* will have *isDefaultPersistenceEnabled* property (default = false), which will be used for enabling persistence in default data region. *DataRegionConfiguration* will have *isPersistenceEnabled*

Re: Persistence per memory policy configuration

2017-10-01 Thread Denis Magda
> On Oct 1, 2017, at 9:49 PM, Dmitriy Setrakyan wrote: > > On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda > wrote: > >> >>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov >> >

Re: Persistence per memory policy configuration

2017-10-01 Thread Dmitriy Setrakyan
On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda wrote: > > > On Oct 1, 2017, at 4:41 AM, Ivan Rakov wrote: > > > > 1) You're right. I forgot to include the main flag in > DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be > enabled

Re: Persistence per memory policy configuration

2017-10-01 Thread Denis Magda
> On Oct 1, 2017, at 4:41 AM, Ivan Rakov wrote: > > 1) You're right. I forgot to include the main flag in DataRegionConfiguration > - *isPersistenceEnabled*. Persistence will be enabled globally if at least > one memory region has this flag set. I’m confused. Why the

Re: Persistence per memory policy configuration

2017-10-01 Thread Dmitriy Setrakyan
I am not sure I like the name "isDefaultDataRegionPersistenceEnabled" - too long. Can we just call it "isDefaultPersistenceEnabled"? D. On Sun, Oct 1, 2017 at 2:41 PM, Ivan Rakov wrote: > Denis, > > 1) You're right. I forgot to include the main flag in >

Re: Persistence per memory policy configuration

2017-10-01 Thread Ivan Rakov
Vladimir, *DataRegionMetrics* is for former memory metrics. *DataStorageMetrics* will contain metrics about persistence. If DataStorageMetrics is ok, let's go this way. Best Regards, Ivan Rakov On 01.10.2017 15:19, Vladimir Ozerov wrote: We merged memory and persistence on config level. So

Re: Persistence per memory policy configuration

2017-10-01 Thread Vladimir Ozerov
We merged memory and persistence on config level. So we should merge metrics in the same way. DataRegionMetrics is absolutely normal name, even if it contains only persistence-related stuff at the moment. вс, 1 окт. 2017 г. в 14:41, Ivan Rakov : > Denis, > > 1) You're

Re: Persistence per memory policy configuration

2017-10-01 Thread Ivan Rakov
Denis, 1) You're right. I forgot to include the main flag in DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be enabled globally if at least one memory region has this flag set. Regarding default data region, I've added *isDefaultDataRegionPersistenceEnabled *to the

Re: Persistence per memory policy configuration

2017-09-29 Thread Ivan Rakov
Guys, I've attached new configuration design draft to the ticket description: https://issues.apache.org/jira/browse/IGNITE-6030 Please, take a look. Right now is the best time to change name of any property. And question about metrics: are we going to rename MemoryMetrics and

Re: Persistence per memory policy configuration

2017-09-28 Thread Yakov Zhdanov
StorageConfiguration + StorageRegionConfiguration --Yakov

Re: Persistence per memory policy configuration

2017-09-28 Thread Vladimir Ozerov
Guys, But what is exact desicion? :-) I saw two final options: 1) StorageConfiguration + StorageRegionConfiguration 2) DataStorageConfiguration + DataRegionConfiguration Which one we choose? On Thu, Sep 28, 2017 at 5:10 PM, Yakov Zhdanov wrote: > > I guess it is safe to

Re: Persistence per memory policy configuration

2017-09-28 Thread Yakov Zhdanov
> I guess it is safe to assume that at this point we came to a consensus? Alex, I think so. Let's carve it in stone (code). --Yakov

Re: Persistence per memory policy configuration

2017-09-28 Thread Alexey Goncharuk
My vote also goes for 1. I guess it is safe to assume that at this point we came to a consensus? 2017-09-27 21:52 GMT+03:00 Denis Magda : > Vote for 1. > > — > Denis > > > On Sep 26, 2017, at 11:23 PM, Vladimir Ozerov > wrote: > > > > Folks, > > > > Let

Re: Persistence per memory policy configuration

2017-09-27 Thread Denis Magda
Vote for 1. — Denis > On Sep 26, 2017, at 11:23 PM, Vladimir Ozerov wrote: > > Folks, > > Let me summarize current naming ideas one more time: > > 1) [StorageConfiguration - StorageRegionConfiguration] > 2) [DurableMemoryConfiguration - DataRegionConfiguration] > 3)

Re: Persistence per memory policy configuration

2017-09-27 Thread Yakov Zhdanov
I like Vladimir's suggestion - StorageConfiguration - StorageRegionConfiguration --Yakov

Re: Persistence per memory policy configuration

2017-09-27 Thread Vladimir Ozerov
Folks, Let me summarize current naming ideas one more time: 1) [StorageConfiguration - StorageRegionConfiguration] 2) [DurableMemoryConfiguration - DataRegionConfiguration] 3) [DurableMemoryConfiguration - DurableMemoryRegionConfiguration] - out of question, as "durable memory region" is too

Re: Persistence per memory policy configuration

2017-09-26 Thread Dmitriy Setrakyan
On Tue, Sep 26, 2017 at 7:33 AM, Dmitry Pavlov wrote: > Hi Dmitriy, thank you for reply. Do you agree Memory Policy already became > Ignite's term? We call this configuration now MemoryPolicy(Configuration), > can we call new configuration elments by their existings name?

Re: Persistence per memory policy configuration

2017-09-26 Thread Dmitry Pavlov
Hi Dmitriy, thank you for reply. Do you agree Memory Policy already became Ignite's term? We call this configuration now MemoryPolicy(Configuration), can we call new configuration elments by their existings name? We can avoid introduction of second Ignite's term in that case. вт, 26 сент. 2017 г.

Re: Persistence per memory policy configuration

2017-09-26 Thread Dmitry Pavlov
Vladimir, it is not clear for me, why we need to rename existing configuration classes. Could you explain? And if we can't get consensus now, should we pospond solution? My idea is that user needs this feature more than elegant names in configuration. Moreover once MemoryPolicyConfiguration was

Re: Persistence per memory policy configuration

2017-09-26 Thread Vladimir Ozerov
I do not understand why we should delay with renames. Yes, it will cause questions, so we will have to put additional efforts to docs and JavaDocs. But the earlier we do that, the better. On Tue, Sep 26, 2017 at 4:50 PM, Dmitry Pavlov wrote: > Hi Igniters, sorry for late

Re: Persistence per memory policy configuration

2017-09-26 Thread Dmitry Pavlov
Hi Igniters, sorry for late response. I didn't catch idea of renaming. PersistentStoreConfiguration is intuitive, and MemoryPolicyConfiguration is intuitive also. If we rename these classes now, it will bring more questions to user list. Users may be confused by old and new names and by trying to

Re: Persistence per memory policy configuration

2017-09-26 Thread Alexey Goncharuk
I do not like DurableMemoryConfiguration, because it's quite confusing - we configure in-memory caches using DurableMemory class, which immediately suggests that everything will be persisted. I am not sure if this is a right wording choice for the documentation either. I would go with

Re: Persistence per memory policy configuration

2017-09-25 Thread Dmitriy Setrakyan
Given that we already have a notion of CacheStore which comes from JCache spec, I think having other stores may get confusing. I like DurableMemoryConfiguration. Other opinions? D. On Mon, Sep 25, 2017 at 12:24 PM, Vladimir Ozerov wrote: > Dima, let's finalize the design

Re: Persistence per memory policy configuration

2017-09-25 Thread Vladimir Ozerov
Dima, let's finalize the design first. As I understand, we are happy with idea to merge MemoryConfiguration and PersistentStoreConfiguration into something what I called DataConfiguration, and to rename MemoryPolicyConfiguration to DataRegionConfiguration. The only outstanding qurestion is

Re: Persistence per memory policy configuration

2017-09-25 Thread Dmitriy Setrakyan
Vladimir, Can you please add the configuration example in the ticket? D. On Mon, Sep 25, 2017 at 12:20 AM, Alexey Goncharuk < alexey.goncha...@gmail.com> wrote: > Guys, > > I suggest we finalize the configuration changes in the original ticket > then:

Re: Persistence per memory policy configuration

2017-09-25 Thread Alexey Goncharuk
Guys, I suggest we finalize the configuration changes in the original ticket then: https://issues.apache.org/jira/browse/IGNITE-6030 and proceed with the changes. 2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan : > Can we specify what metrics will look like? I think we

Re: Persistence per memory policy configuration

2017-09-23 Thread Dmitriy Setrakyan
Can we specify what metrics will look like? I think we should not just blindly merge them. On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov wrote: > Denis, > > Makes sense. Thanks for catching it! > > On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda wrote:

Re: Persistence per memory policy configuration

2017-09-23 Thread Vladimir Ozerov
Denis, Makes sense. Thanks for catching it! On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda wrote: > If we’re taking the consolidation path for Memory and Persistence > configurations then it makes sense to merge MemoryMetrics [1] and > PersistenceMetrics [2] plus their JMX

Re: Persistence per memory policy configuration

2017-09-22 Thread Denis Magda
If we’re taking the consolidation path for Memory and Persistence configurations then it makes sense to merge MemoryMetrics [1] and PersistenceMetrics [2] plus their JMX beans. Agree? [1] https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/MemoryMetrics.html

Re: Persistence per memory policy configuration

2017-09-22 Thread Dmitriy Setrakyan
I like it. Alexey G, can you please chime in? D. On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov wrote: > Guys, > > Here is my proposal: > > 1) MemoryPolicyConfiguration is renamed to *DataRegionConfiguration*. > 2) PersistenceConfiguration is merged with

Re: Persistence per memory policy configuration

2017-09-22 Thread Vladimir Ozerov
Guys, Here is my proposal: 1) MemoryPolicyConfiguration is renamed to *DataRegionConfiguration*. 2) PersistenceConfiguration is merged with MemoryConfiguration and renamed to ... *DataStorageConfiguration*! It has: common memory settings (e.g. default data region), persistence settings (e.g.

Re: Persistence per memory policy configuration

2017-09-20 Thread Dmitriy Setrakyan
Firstly all, why not call it DataPolicy instead of MemoryPolicy? Secondly, why not set data policies directly on IgniteConfiguration. And lastly, how about we combine memory and disk properties in one bean with clear naming convention? Here is the example. Note that all properties above must

Re: Persistence per memory policy configuration

2017-09-20 Thread Vladimir Ozerov
I prefer the second. Composition over inheritance - this is how all our configuration is crafted. E.g. we do not have "CacheConfiguration" and "StoreEnabledCacheConfiguration". Instead, we have "CacheConfiguration.setCacheStoreFactory". On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <

Re: Persistence per memory policy configuration

2017-09-20 Thread Alexey Goncharuk
Reiterating this based on some feedback from PDS users. It might be confusing to configure persistence with "MemoryPolicy", so another approach is to deprecate the old names and introduce a new name "DataRegion" because it reflects the actual state when data is stored on disk and partially in

Re: Persistence per memory policy configuration

2017-09-14 Thread Alexey Goncharuk
Vladimir, The configuration would look like so: IgniteConfiguration cfg = new IgniteConfiguration(); cfg.setMemoryConfiguration(new MemoryConfiguration() .setMemoryPolicies( new MemoryPolicyConfiguration().setName("InMemory").setMaxSize(...), new

Re: Persistence per memory policy configuration

2017-09-14 Thread Yakov Zhdanov
>>Can we have separate MemoryPolycy for every CacheGroup? In that case we could auto generate MemoryPolycy based on CacheGroup settings and have a "cache level" persistence settings. >>We can use some kind of MemoryPolicyTemplate to add default values and even re-use existing MemoryPolicy if all

Re: Persistence per memory policy configuration

2017-09-14 Thread Yakov Zhdanov
> I like the idea of having PersistentMemoryPolicy configuration class. I like the idea, too. --Yakov

Re: Persistence per memory policy configuration

2017-09-13 Thread Konstantin Dudkov
Can we have separate MemoryPolycy for every CacheGroup? In that case we could auto generate MemoryPolycy based on CacheGroup settings and have a "cache level" persistence settings. We can use some kind of MemoryPolicyTemplate to add default values and even re-use existing MemoryPolicy if all

Re: Persistence per memory policy configuration

2017-09-13 Thread Vladimir Ozerov
Sounds good. But can we see a full example of such configuration? E.g. we have two caches, one in-memory, another persistent. On Wed, Sep 13, 2017 at 10:54 AM, Alexey Goncharuk < alexey.goncha...@gmail.com> wrote: > I like the idea of having PersistentMemoryPolicy configuration class. >

Re: Persistence per memory policy configuration

2017-09-13 Thread Alexey Goncharuk
I like the idea of having PersistentMemoryPolicy configuration class. Vladimir, Yakov, any objections? 2017-09-13 3:05 GMT+03:00 Denis Magda : > I would go for PersistentMemoryPolicyConfiguration way if it will be > expanded by storage related settings furthers. > > For

Re: Persistence per memory policy configuration

2017-09-12 Thread Denis Magda
I would go for PersistentMemoryPolicyConfiguration way if it will be expanded by storage related settings furthers. For instance, from the discussion on @user I see a raising demand for the storage maximum size. If it’s reached the data will be evicted from there too. Just as an example. —

Re: Persistence per memory policy configuration

2017-09-12 Thread Denis Magda
While we are contemplating the best way to introduce this feature I have a side note to consider. Can the user enable CacheStore (RDBMS, Cassandra) for caches that are not persisted in Ignite Persistence? That’s one of the use cases I hear a lot about - some of the caches need persisted in

Re: Persistence per memory policy configuration

2017-09-12 Thread Alexey Goncharuk
Yakov, The default=true will be used only if the PersistentStoreConfiguration is set in the IgniteConfiguration, this is done only to maintain configuration compatibility. A user is allowed to use persisted and in-memory caches, he will be only forced to put them into separate memory policies.

Re: Persistence per memory policy configuration

2017-09-12 Thread Yakov Zhdanov
Agree that implicit actions are always source of issues of different kinds. Alex, having persistence enabled by default does not seem to be a good idea. >I think if we go with "persistence per cache" path, we must just enforce the correct configuration and let users configure the rest What do

Re: Persistence per memory policy configuration

2017-09-12 Thread Alexey Goncharuk
This may work. But I do not like any implicit memory policy cloning because this will double the expected memory consumption and result in questions "why does my Ignite take 2x more memory when I enable persistence" from users. I think if we go with "persistence per cache" path, we must just

Re: Persistence per memory policy configuration

2017-09-12 Thread Pavel Tupitsyn
Can we leave the default policy as is, but clone it with persistence enabled on demand? E.g: - user starts Ignite with default config - createCache without persistence - use default policy - createCache with persistence - clone default policy with enabled persistence and some predefined postfix -

Re: Persistence per memory policy configuration

2017-09-12 Thread Alexey Goncharuk
We surely can, but: * we should then have two configuration settings for default memory policy size (one in-memory and one persisted) * a user still may configure multiple custom memory policies. In this case, the requirement to have this flag the same in a memory policy is still valid, so a

Re: Persistence per memory policy configuration

2017-09-12 Thread Vladimir Ozerov
Alex, Can we have two default memory policies - one for in-memory and another one for persistence cases? On Tue, Sep 12, 2017 at 12:40 PM, Alexey Goncharuk < alexey.goncha...@gmail.com> wrote: > This is possible, but then if two caches belong to the same memory policy, > they must be both

Re: Persistence per memory policy configuration

2017-09-12 Thread Alexey Goncharuk
This is possible, but then if two caches belong to the same memory policy, they must be both either persistence-enabled or persistence-disabled. We can add this validation, but I think this will lead to a greater confusion for a user. 2017-09-12 12:34 GMT+03:00 Pavel Tupitsyn

Re: Persistence per memory policy configuration

2017-09-12 Thread Pavel Tupitsyn
Agree with Vladimir. Currently we enable persistence cluster-wide by setting IgniteConfiguration.persistentStoreConfiguration. Ideally CacheConfiguration.persistenceEnabled should be the only setting I need to set. Thanks, Pavel On Tue, Sep 12, 2017 at 12:28 PM, Vladimir Ozerov

Re: Persistence per memory policy configuration

2017-09-12 Thread Vladimir Ozerov
As a user I would definitely prefer to control persistence through flag on cache configuration. I do not even want to know what "memory policy" is. E.g. CacheConfiguration.persistenceEnabled. On Tue, Sep 12, 2017 at 12:24 PM, Alexey Goncharuk < alexey.goncha...@gmail.com> wrote: > Igniters, > >