Re: [DISCUSSION] Urgent Ignite bug fix release

We created the release build procedure on Apache CI server according to this ticket: https://issues.apache.org/jira/browse/IGNITE-5249 We also changed the checksum calculation method: https://issues.apache.org/jira/browse/IGNITE-5817 Oleg On Thu, Sep 7, 2017 at 9:49 AM, 李玉珏@163

Re: [DISCUSSION] Urgent Ignite bug fix release

Currently,the latest version is apache-ignite-fabric-2.1.0-bin.zip . and in http://www.gridgainsystems.com/nexus/content/repositories/external/org/apache/ignite/ignite-core/ the latest version is 2.1.4, As I

Re: [DISCUSSION] Urgent Ignite bug fix release

What’s the “new beta release process”? Haven’t seen anything like that on @dev? — Denis > On Sep 6, 2017, at 9:37 AM, Anton Vinogradov wrote: > > Denis, > > As you can remember, Cos told us that the only option to perform release is > to perform it in Apache way. >

Re: [DISCUSSION] Urgent Ignite bug fix release

Denis, As you can remember, Cos told us that the only option to perform release is to perform it in Apache way. So, we will be able once new release process beta will be ready. Also, https://issues.apache.org/jira/browse/IGNITE-6193 was reopened and fix in progress now. On Wed, Sep 6, 2017 at

Re: [DISCUSSION] Urgent Ignite bug fix release

Igor, That's an important fix. Please merge it to ignite-2.2. Don't forget to change fix version. On Tue, Sep 5, 2017 at 3:42 PM, Igor Sapego wrote: > Igniters, > > There is one more fix I would propose to include into 2.2: [1]. It has > been fixed a while ago and it is

Re: [DISCUSSION] Urgent Ignite bug fix release

Igniters, There is one more fix I would propose to include into 2.2: [1]. It has been fixed a while ago and it is critical for C++ users that would like to make use of persistence store, as currently they need to have separate Java node just to activate cluster. What do you think? [1] -

Re: [DISCUSSION] Urgent Ignite bug fix release

Igniters, All fixes related to release 2.2 merged. But, we still waiting for changes related to new release process ( https://issues.apache.org/jira/browse/IGNITE-5249). Beta version of "new release process" will be used to release 2.2, final changes will be merged to 2.3. On Fri, Sep 1, 2017

Re: [DISCUSSION] Urgent Ignite bug fix release

Hi, at least for this release I suggest to keep versioning as is and use 2.2 Without solved issue with http://apache-ignite-developers.2346864.n4.nabble.com/usage-analytics-td19504.html Ignite will not remind users about latest available ignite version. Versioning structure change will confuse

Re: [DISCUSSION] Urgent Ignite bug fix release

Hi, Reasonable question, but I propose to keep 2.2 as a release version for this urgent release, since we never released X.Y.Z version before. Let's discuss this at separate thread. Could you please start it? On Fri, Sep 1, 2017 at 4:49 AM, 李玉珏@163 <18624049...@163.com> wrote: > Since it's an

Re: [DISCUSSION] Urgent Ignite bug fix release

Since it's an urgent bugfix version, why can't the version number be defined as 2.1.1 or 2.1.4? After all, functionality has not increased and documents do not require big changes. If the version number is 2.1 series, there is no need to publish a new version of the document. 在 2017/8/28

Re: [DISCUSSION] Urgent Ignite bug fix release

To be honnest we check heap memory which is used by all Ignite nodes on the host: org.apache.ignite.internal.IgniteKernal#checkPhysicalRam Shouldn't we do the same for offheap? -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [DISCUSSION] Urgent Ignite bug fix release

Your suggestions sounds reasonable to me. Are they only about the calculations at the code level or you propose to change the message format too? — Denis > On Aug 30, 2017, at 3:24 AM, Vladimir Ozerov wrote: > > I looked at the code of IGNITE-6182 (memory policy

Re: [DISCUSSION] Urgent Ignite bug fix release

IGNITE-6128 is not merged yet. I already raised my concerns, please chime in. ср, 30 авг. 2017 г. в 21:45, Denis Magda : > Have the all pre-release tests passed? If so, let’s kick off the vote. > > — > Denis > > > On Aug 30, 2017, at 4:29 AM, Anton Vinogradov

Re: [DISCUSSION] Urgent Ignite bug fix release

Have the all pre-release tests passed? If so, let’s kick off the vote. — Denis > On Aug 30, 2017, at 4:29 AM, Anton Vinogradov > wrote: > > 1) Fixes for: > - https://issues.apache.org/jira/browse/IGNITE-6204 > - https://issues.apache.org/jira/browse/IGNITE-6193 >

Re: [DISCUSSION] Urgent Ignite bug fix release

1) Fixes for: - https://issues.apache.org/jira/browse/IGNITE-6204 - https://issues.apache.org/jira/browse/IGNITE-6193 merged 2) https://issues.apache.org/jira/browse/IGNITE-6003 resolved as duplicate to https://issues.apache.org/jira/browse/IGNITE-6182 3) Current list if issues related to

Re: [DISCUSSION] Urgent Ignite bug fix release

I looked at the code of IGNITE-6182 (memory policy suggestions), and I do not like it. Too complex. My proposal: 1) Do not take in count JVM heap. First, we do not need whether it will be really used or not. Second, we never checked it in heap-based Ignite 1.x, and never had a single compliant. 2)

Re: [DISCUSSION] Urgent Ignite bug fix release

Anton, thanks for stepping in! The tickets set looks complete for me! Let’s start the vote once the fixes are merged and tests pass. — Denis > On Aug 28, 2017, at 7:19 AM, Anton Vinogradov > wrote: > > Igniters, > > Seems 2.2 is a urgent bugfix release, so it

Re: [DISCUSSION] Urgent Ignite bug fix release

Backport issue: https://issues.apache.org/jira/browse/IGNITE-6204 Best Regards, Ivan Rakov On 28.08.2017 17:19, Anton Vinogradov wrote: Igniters, Seems 2.2 is a urgent bugfix release, so it should be based on 2.1, In this case all other issues with fixVersion = 2.2 should be moved to 2.3.

Re: [DISCUSSION] Urgent Ignite bug fix release

Igniters, Seems 2.2 is a urgent bugfix release, so it should be based on 2.1, In this case all other issues with fixVersion = 2.2 should be moved to 2.3. Currently, I see 835 issues with fixVersion = 2.2 Seems we should have only 4 issues with fixVersion = 2.2: - Change default max memory size

Re: [DISCUSSION] Urgent Ignite bug fix release

Denis, > BTW, who is considered to be the release manager of this release? I'll do it. On Mon, Aug 28, 2017 at 3:54 PM, Seliverstov Igor wrote: > Ok, the check happens at the node start time or on NODE_JOIN event > > in general it looks like: > > 1) calculate expected

Re: [DISCUSSION] Urgent Ignite bug fix release

Ok, the check happens at the node start time or on NODE_JOIN event in general it looks like: 1) calculate expected used memory = heap max + system cache max + all custom policies max + default policy size and put it into a node attribute 2) get total physycal memory, calculate expected safe to

Re: [DISCUSSION] Urgent Ignite bug fix release

Igor, can you please describe the algorithm with all the thresholds? On Mon, Aug 28, 2017 at 4:56 AM, Seliverstov Igor wrote: > The suggestion here is based on initial settings, and it's so because there > is no other nodes on the host in the example. > > The algorithm

Re: [DISCUSSION] Urgent Ignite bug fix release

The suggestion here is based on initial settings, and it's so because there is no other nodes on the host in the example. The algorithm tries to preserve the original ratio of memory policies keeping numbers reasonable (for example after some thresshold it will suggest not to use several memory

Re: [DISCUSSION] Urgent Ignite bug fix release

Looks good, but why in the example provided are we suggesting 8GB? 2 nodes with 8GB will completely exhaust the available memory. I would suggest 6 or 7GB. Also, why 100MB for default policy. Anything under 1GB seems too small. Can you please comment? D. On Mon, Aug 28, 2017 at 3:31 AM,

Re: [DISCUSSION] Urgent Ignite bug fix release

One more example of possible warning: - Excessive memory usage by Ignite node process (performance may drop) [requested=44613MB, available=15942MB]. Please tune the folowing settings as suggested: MemoryPolicyConfiguration.initialSize for

Re: [DISCUSSION] Urgent Ignite bug fix release

Guys, ML lib profile is missing in 2.1 release! That must be fixed and rolled out in this emergency release: https://issues.apache.org/jira/browse/IGNITE-6193 Oleg, Yuri, please step in and handle the issue. BTW, who is considered to be the

Re: [DISCUSSION] Urgent Ignite bug fix release

I like the format proposed by Denis, very clear. However, I also do not understand why a user should change the size of some system cache. How would a user ever know what value to put there? This value should be configured by Ignite automatically. D. On Fri, Aug 25, 2017 at 2:24 PM, Denis Magda

Re: [DISCUSSION] Urgent Ignite bug fix release

Igor, Let me suggest this format. - Excessive memory usage by Ignite node process (performance may drop) [requested=29251MB, available=15942MB] Please tune the following settings: [MemoryConfiguration.defaultMemoryPolicySize = suggested value]

Re: [DISCUSSION] Urgent Ignite bug fix release

The message without logging layout: Not enough memory for current process [required=29251MB, available=15942MB]. Please change MemoryConfiguration.systemCacheMaxSize and MemoryConfiguration.defaultMemoryPolicySize to decrease memory allocated for each node. Current settings: HeapInit=250MB

Re: [DISCUSSION] Urgent Ignite bug fix release

Igor, what is this flood of WARN messaging coming after the text? Are we really going to print this whole thing out? On Fri, Aug 25, 2017 at 9:49 AM, Seliverstov Igor wrote: > This message appears on topology change in case the available memory is > exceeded > > 2017-08-25

Re: [DISCUSSION] Urgent Ignite bug fix release

This message appears on topology change in case the available memory is exceeded 2017-08-25 19:47 GMT+03:00 Seliverstov Igor : > An example of current impl: > > > [2017-08-25 19:44:37,740][WARN ][disco-event-worker-#29%internal. >

Re: [DISCUSSION] Urgent Ignite bug fix release

An example of current impl: [2017-08-25 19:44:37,740][WARN ][disco-event-worker-#29%internal.GridHomePathSelfTest0%][GridDiscoveryManager] [2017-08-25 19:44:37,740][WARN ][disco-event-worker-#29%internal.GridHomePathSelfTest0%][GridDiscoveryManager] Not enough memory for current process

Re: [DISCUSSION] Urgent Ignite bug fix release

I suppose we should not forget JVM heap size and suggest to reduce both options On Fri, Aug 25, 2017 at 5:24 PM, Dmitriy Setrakyan wrote: > Igor, I would change the message. How about this: > > Required RAM size is larger than total physical memory available for OS. > >

Re: [DISCUSSION] Urgent Ignite bug fix release

Igor, I would change the message. How about this: Required RAM size is larger than total physical memory available for OS. > Please change MemoryConfiguration.WhichProperty and > MemoryPolicyConfiguration.WhichProperty to decrease memory allocated for > each node. Also, can we calculate what

Re: [DISCUSSION] Urgent Ignite bug fix release

What do you, guys think about next warning? [2017-08-25 17:17:04,718][INFO ][test-runner-#1%internal.GridHomePathSelfTest%][GridHomePathSelfTest0] System cache's MemoryPolicy size is configured to 40 MB. Use MemoryConfiguration.systemCacheMemorySize property to change the setting. [2017-08-25

Re: [DISCUSSION] Urgent Ignite bug fix release

Agree, let's release new version including tickets mentioned by Denis and Ivan. --Yakov

Re: [DISCUSSION] Urgent Ignite bug fix release

Let's include to the release tickets with optimizations of checkpointing algorithm: https://issues.apache.org/jira/browse/IGNITE-6178 https://issues.apache.org/jira/browse/IGNITE-6033 https://issues.apache.org/jira/browse/IGNITE-5961 This will help users who are experiencing problems with slow

Re: [DISCUSSION] Urgent Ignite bug fix release

+1 on the release, let's do it ASAP. On Thu, Aug 24, 2017 at 12:32 PM, Denis Magda wrote: > Igniters, > > Presently every Apache Ignite node requests 80% of RAM from an operating > system. We debated a lot about this default [1] and as a practice shows > those who voted for

[DISCUSSION] Urgent Ignite bug fix release

Igniters, Presently every Apache Ignite node requests 80% of RAM from an operating system. We debated a lot about this default [1] and as a practice shows those who voted for the lower value (20%-25%) were right. Our users started reporting about bizarre issues ([2], [3], [4]) when a cluster