[GitHub] ignite pull request #4792: Ignite 2.5 master

[alamar]

Github user alamar closed the pull request at:

https://github.com/apache/ignite/pull/4792


---

[GitHub] ignite pull request #4893: IGNITE-8879 Correct restoring of metastorage and ...

[ivandasch]

GitHub user ivandasch opened a pull request:

https://github.com/apache/ignite/pull/4893

IGNITE-8879 Correct restoring of metastorage and skipping updates of …

…other caches on localJoin of node not in baseline.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8879

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4893.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4893


commit c8b3fc73cc068507b3ad838c2a297b7fa0e56ff5
Author: Ivan Daschinskiy 
Date:   2018-10-02T13:34:10Z

IGNITE-8879 Correct restoring of metastorage and skipping updates of other 
caches on localJoin of node not in baseline.




---

Re: TDE Implementation details.

[Nikolay Izhikov]

Hello, Dmitriy.

Thank you for feedback!

> 1) I suggest changing encrypt() and decrypt() method would take the same type 
> of parameters. 
> This uniformity will avoid implementors questions about why encrypt takes 
> byte[] but decrypt takes ByteBuffer.

This is done by a reason.
encrypt and decrypt methods are on hot code path, so it important to avoid 
unnecessary type conversion.
We use ByteBuffer in places where we already operate with ByteBuffer and byte[] 
where we have it.

Anyway, ByteBuffer can be easily converted to byte[] and vice versa.

> 2) I suggest the renaming of method create()  to createKey() to make it> easy 
> to understand what method creates.> 

We already discussed new SPI and approved it with reviewers.
Anyway it's pretty easy to rename method, but I don't think I should do it at 
the moment.
All other reviewers are OK with simple `create`.

В Вт, 02/10/2018 в 17:34 +0300, Dmitriy Pavlov пишет:
> Hi Igniters,
> 
> I'm sorry I a little bit late to the party, but I found a couple of hours
> to take a look to TDE-1 implementation
> https://issues.apache.org/jira/browse/IGNITE-8485 . Nikolay kindly agreed
> to let me take a look.
> 
> Most of the change was done well and I totally agree with it. Current
> implementation seems to be extendable.
> 
> I've got several minor comments, I hope it will be fixed.
> https://github.com/apache/ignite/pull/4167
> Related to tests: let's make sure we won't fail new tests and compilation
> pass for all modules. I will be happy to help with failures analysis if it
> is required.
> 
> Proposals related to API
> 1) I suggest changing encrypt() and decrypt() method would take the same
> type of parameters. This uniformity will avoid implementors questions about
> why encrypt takes byte[] but decrypt takes ByteBuffer.
> 2) I suggest the renaming of method create()  to createKey() to make it
> easy to understand what method creates.
> 
> 
> Sincerely,
> Dmitriy Pavlov
> 
> ср, 12 сент. 2018 г. в 20:34, Denis Magda :
> 
> > Hello Nikolay,
> > 
> > Excellent progress, look forward to seeing the TDE released in 2.7!
> > 
> > --
> > Denis
> > 
> > On Wed, Sep 12, 2018 at 2:47 AM Nikolay Izhikov 
> > wrote:
> > 
> > > Hello, Denis.
> > > 
> > > > Could you please list the limitations of Phase 1?
> > > > I'm curious what won't be supported in 2.7.
> > > 
> > > 1. We will have ability to encrypt data stored on the disk for specific
> > > cache.
> > > 
> > > - There is no limitation on API usage or something for an
> > > encrypted cache.
> > > - If some cache in a cache group is encrypted, all other caches
> > 
> > in
> > > this group must be encrypted.
> > > - Setting up master key(standard jdk key storage) is prerequisite
> > > and should be done by an administrator.
> > > - `encryptionEnabled` flag setting up on cache creation and can't
> > > be changed in runtime.
> > > 
> > > 2. We won't be able to change encryption keys for existing cache(key
> > > rotation procedure).
> > > This will be implemented in Phase 2.
> > > 
> > > В Вт, 11/09/2018 в 23:41 -0400, Denis Magda пишет:
> > > > Nikolay,
> > > > 
> > > > Could you please list the limitations of Phase 1? I'm curious what
> > 
> > won't
> > > be
> > > > supported in 2.7.
> > > > 
> > > > --
> > > > Denis
> > > > 
> > > > On Tue, Sep 11, 2018 at 4:29 PM Nikolay Izhikov 
> > > 
> > > wrote:
> > > > 
> > > > > > As I understand the plan is to get TDE Phase 1 released in 2.7,
> > > 
> > > right?
> > > > > 
> > > > > Yes. We will release TDE in 2.7
> > > > > 
> > > > > > Could you also list what needs to be done at Phase 2 and how much
> > > 
> > > time
> > > > > 
> > > > > it might take.
> > > > > 
> > > > > Yes, I think Dmitry Ryabov will send Phase 2 design
> > > > > 
> > > > > 
> > > > > В Вт, 11/09/2018 в 23:27 +0300, Nikolay Izhikov пишет:
> > > > > > Hello, Denis.
> > > > > > 
> > > > > > Yes, Vladimir made 2 rounds of review.
> > > > > > I planning to fix all issues with implementation in a few days.
> > > > > > 
> > > > > > 
> > > > > > В Вт, 11/09/2018 в 10:40 -0400, Denis Magda пишет:
> > > > > > > Hi Nikolay,
> > > > > > > 
> > > > > > > Has anybody started looking into your request? As I understand
> > 
> > the
> > > > > 
> > > > > plan is
> > > > > > > to get TDE Phase 1 released in 2.7, right?
> > > > > > > 
> > > > > 
> > > > > 
> > 
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=89067473
> > > > > > > 
> > > > > > > Could you also list what needs to be done at Phase 2 and how much
> > > 
> > > time
> > > > > 
> > > > > it
> > > > > > > might take.
> > > > > > > 
> > > > > > > --
> > > > > > > Denis
> > > > > > > 
> > > > > > > On Thu, Aug 9, 2018 at 8:48 AM Nikolay Izhikov <
> > > 
> > > nizhi...@apache.org>
> > > > > 
> > > > > wrote:
> > > > > > > 
> > > > > > > > Hello, Igniters.
> > > > > > > > 
> > > > > > > > I want to share with you TDE implementation details.
> > > > > > > > I think it can simplify review and acception of 

[jira] [Created] (IGNITE-9770) Re-run possible blockers from pr.html

[Nikolai Kulagin (JIRA)]

Nikolai Kulagin created IGNITE-9770:
---

 Summary: Re-run possible blockers from pr.html
 Key: IGNITE-9770
 URL: https://issues.apache.org/jira/browse/IGNITE-9770
 Project: Ignite
  Issue Type: Sub-task
Reporter: Nikolai Kulagin
Assignee: Nikolai Kulagin


Show button for re-run possible blockers. Also show merged button with re-run 
possible blockers JIRA, so the user re-run the possible tests and does 
not wait for them to complete,but learn about the completed tests from the 
comment to GitHib.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9773) GridAffinityProcessorMemoryLeakTest should be fixed

[Andrey Aleksandrov (JIRA)]

Andrey Aleksandrov created IGNITE-9773:
--

 Summary: GridAffinityProcessorMemoryLeakTest should be fixed
 Key: IGNITE-9773
 URL: https://issues.apache.org/jira/browse/IGNITE-9773
 Project: Ignite
  Issue Type: Bug
Reporter: Andrey Aleksandrov


/** Max value for affinity history size name. Should be the same as in 
GridAffinityAssignmentCache.MAX_HIST_SIZE */
private final int MAX_HIST_SIZE = getInteger(IGNITE_AFFINITY_HISTORY_SIZE, 100);

The default value should be 500. (not 100)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9775) Cache.lostPartitions() is empty with single node remaining in the cluster

[Roman Novichenok (JIRA)]

Roman Novichenok created IGNITE-9775:


 Summary: Cache.lostPartitions() is empty with single node 
remaining in the cluster
 Key: IGNITE-9775
 URL: https://issues.apache.org/jira/browse/IGNITE-9775
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Affects Versions: 2.6, 2.5
Reporter: Roman Novichenok


When only one node remains in the cluster, cache.lostPartitions() is empty.

Pull request: [https://github.com/novicr/ignite/pull/1] contains a test to show 
the issue.

To reproduce:
 # Turn on native persistence
 # Create partitioned cache with 1 backup, sync replication, READ_ONLY_SAFE 
loss policy
 # Start 4 nodes
 # Persist enough data to make sure all nodes participate
 # Stop 3 nodes

Expected behavior:
 * cache.lostPartitions().size() > 0

Actual:
 * cache.lostPartitions().size() == 0

 

Note, if more than one node is operational, the lostPartitions() will contain 
some number of partitions.  Don't know if the list it contains is complete.

 

The issue seems to be in the GridDhtPartitionTopologyImpl.detectLostPartitions()

With a single node left in the cluster, the coordinator node is also the only 
one that contains any data, but localNode is skipped when examining node2Part 
maps to look for any partitions that should be marked LOST.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Release process documentation in wiki

[Dmitriy Pavlov]

Hi Igniters,

We had several hot discussions related to release, and one from reasons we
need time to come to consensus was my poor understanding of the process.

I decided to contribute currently found facts as a scheme of release
process here
https://cwiki.apache.org/confluence/display/IGNITE/Release+Process

Please find a minute and take a look. Feedback, as always, is more than
welcome.

Sincerely,
Dmitriy Pavlov

Re: TDE Implementation details.

[Dmitriy Pavlov]

Hi Igniters,

I'm sorry I a little bit late to the party, but I found a couple of hours
to take a look to TDE-1 implementation
https://issues.apache.org/jira/browse/IGNITE-8485 . Nikolay kindly agreed
to let me take a look.

Most of the change was done well and I totally agree with it. Current
implementation seems to be extendable.

I've got several minor comments, I hope it will be fixed.
https://github.com/apache/ignite/pull/4167
Related to tests: let's make sure we won't fail new tests and compilation
pass for all modules. I will be happy to help with failures analysis if it
is required.

Proposals related to API
1) I suggest changing encrypt() and decrypt() method would take the same
type of parameters. This uniformity will avoid implementors questions about
why encrypt takes byte[] but decrypt takes ByteBuffer.
2) I suggest the renaming of method create()  to createKey() to make it
easy to understand what method creates.


Sincerely,
Dmitriy Pavlov

ср, 12 сент. 2018 г. в 20:34, Denis Magda :

> Hello Nikolay,
>
> Excellent progress, look forward to seeing the TDE released in 2.7!
>
> --
> Denis
>
> On Wed, Sep 12, 2018 at 2:47 AM Nikolay Izhikov 
> wrote:
>
> > Hello, Denis.
> >
> > > Could you please list the limitations of Phase 1?
> > > I'm curious what won't be supported in 2.7.
> >
> > 1. We will have ability to encrypt data stored on the disk for specific
> > cache.
> >
> > - There is no limitation on API usage or something for an
> > encrypted cache.
> > - If some cache in a cache group is encrypted, all other caches
> in
> > this group must be encrypted.
> > - Setting up master key(standard jdk key storage) is prerequisite
> > and should be done by an administrator.
> > - `encryptionEnabled` flag setting up on cache creation and can't
> > be changed in runtime.
> >
> > 2. We won't be able to change encryption keys for existing cache(key
> > rotation procedure).
> > This will be implemented in Phase 2.
> >
> > В Вт, 11/09/2018 в 23:41 -0400, Denis Magda пишет:
> > > Nikolay,
> > >
> > > Could you please list the limitations of Phase 1? I'm curious what
> won't
> > be
> > > supported in 2.7.
> > >
> > > --
> > > Denis
> > >
> > > On Tue, Sep 11, 2018 at 4:29 PM Nikolay Izhikov 
> > wrote:
> > >
> > > > > As I understand the plan is to get TDE Phase 1 released in 2.7,
> > right?
> > > >
> > > > Yes. We will release TDE in 2.7
> > > >
> > > > > Could you also list what needs to be done at Phase 2 and how much
> > time
> > > >
> > > > it might take.
> > > >
> > > > Yes, I think Dmitry Ryabov will send Phase 2 design
> > > >
> > > >
> > > > В Вт, 11/09/2018 в 23:27 +0300, Nikolay Izhikov пишет:
> > > > > Hello, Denis.
> > > > >
> > > > > Yes, Vladimir made 2 rounds of review.
> > > > > I planning to fix all issues with implementation in a few days.
> > > > >
> > > > >
> > > > > В Вт, 11/09/2018 в 10:40 -0400, Denis Magda пишет:
> > > > > > Hi Nikolay,
> > > > > >
> > > > > > Has anybody started looking into your request? As I understand
> the
> > > >
> > > > plan is
> > > > > > to get TDE Phase 1 released in 2.7, right?
> > > > > >
> > > >
> > > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=89067473
> > > > > >
> > > > > > Could you also list what needs to be done at Phase 2 and how much
> > time
> > > >
> > > > it
> > > > > > might take.
> > > > > >
> > > > > > --
> > > > > > Denis
> > > > > >
> > > > > > On Thu, Aug 9, 2018 at 8:48 AM Nikolay Izhikov <
> > nizhi...@apache.org>
> > > >
> > > > wrote:
> > > > > >
> > > > > > > Hello, Igniters.
> > > > > > >
> > > > > > > I want to share with you TDE implementation details.
> > > > > > > I think it can simplify review and acception of TDE
> > implementation.
> > > > > > > This mail is copy of wiki page [1].
> > > > > > >
> > > > > > > Please, share your thoughts.
> > > > > > >
> > > > > > > TDE is ready for review [2].
> > > > > > > Please, let me know, who is able to make final review.
> > > > > > >
> > > > > > > This document describes some internal details of TDE.Phase 1
> > > > > > > implementation [3].
> > > > > > > I suggest that reader familiar with the general design
> described
> > in
> > > >
> > > > IEP-18
> > > > > > > [4].
> > > > > > >
> > > > > > >
> > > > > > > Cache group key management and node join enhancements:
> > > > > > >
> > > > > > > 1. GridEncryptionManager encapsulates all logic related
> > to
> > > >
> > > > key
> > > > > > > management:
> > > > > > > a. All group encryption keys are stored in
> > MetaStore.
> > > > > > >
> > > > > > > b. Joining node sends to cluster:
> > > > > > > * Master key digest.
> > > > > > > * All group keys saved locally
> > (Map > > > > > > byte[]>). Keys send over a network in encrypted form.
> > > > > > >
> > > > > > > c. Coordinator on new node join:
> > > > > > > * Compares new node master key 

[GitHub] ignite pull request #4894: IGNITE-9673

[NSAmelchev]

GitHub user NSAmelchev opened a pull request:

https://github.com/apache/ignite/pull/4894

IGNITE-9673

Fix deadlock of system threads.  

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/NSAmelchev/ignite ignite-9673

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4894.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4894


commit 680ddc7f4b5eeedc2ed3294ad3d756146f846dff
Author: NSAmelchev 
Date:   2018-10-02T14:31:18Z

Fix sys thread deadlock




---

[GitHub] ignite pull request #4891: IGNITE-9750

[devozerov]

GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/4891

IGNITE-9750



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9750-1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4891.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4891


commit ca0b0f7be4c3af1a8f0f0900dff8363e94cc77be
Author: devozerov 
Date:   2018-10-02T12:02:47Z

IGNITE-9750

commit 6f6a75cf0b98ca4a41dc2bcce83a53023cb112b8
Author: devozerov 
Date:   2018-10-02T12:58:15Z

Tests.

commit 597293670fd401f926e2f717141b45539901e951
Author: devozerov 
Date:   2018-10-02T13:00:09Z

Done.

commit 1d68659dc5f6426a40e30d48b62cd09bd65f1864
Author: devozerov 
Date:   2018-10-02T13:00:53Z

WPI.

commit 111d669d2e80ca589c9e614a8427c4fb76de2436
Author: devozerov 
Date:   2018-10-02T13:02:35Z

Done.




---

Re: Release process documentation in wiki

[Nikolay Izhikov]

Dmitriy, thank you.

I also making some notes about release.
Feedback are welcome.

https://cwiki.apache.org/confluence/display/IGNITE/Release+manager+Notes

В Вт, 02/10/2018 в 14:42 +0300, Dmitriy Pavlov пишет:
> Hi Igniters,
> 
> We had several hot discussions related to release, and one from reasons we
> need time to come to consensus was my poor understanding of the process.
> 
> I decided to contribute currently found facts as a scheme of release
> process here
> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process
> 
> Please find a minute and take a look. Feedback, as always, is more than
> welcome.
> 
> Sincerely,
> Dmitriy Pavlov


signature.asc
Description: This is a digitally signed message part

[jira] [Created] (IGNITE-9772) Remove term "fabric" from Apache Ignite project completely

[Peter Ivanov (JIRA)]

Peter Ivanov created IGNITE-9772:


 Summary: Remove term "fabric" from Apache Ignite project completely
 Key: IGNITE-9772
 URL: https://issues.apache.org/jira/browse/IGNITE-9772
 Project: Ignite
  Issue Type: Task
Reporter: Peter Ivanov
Assignee: Peter Ivanov


Some files (code, readmes, etc.) still have reference to "fabric" term. It is 
required to eliminate it completely.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4892: Ignite 2.5.3

[alamar]

GitHub user alamar opened a pull request:

https://github.com/apache/ignite/pull/4892

Ignite 2.5.3



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-2.5.3

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4892.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4892


commit 58e79e1b63225778117333139046820ceb4bb896
Author: Igor Sapego 
Date:   2018-07-17T09:36:59Z

IGNITE-9014: Thin C++ client included in binary release

(cherry picked from commit 8d1b1d6)

commit cac0879e87d63e5afbb2f353ab5a970b991092c2
Author: Igor Sapego 
Date:   2018-07-17T11:45:01Z

IGNITE-9002: Fixed C++ thin client crash with dynamic cache without
config

(cherry picked from commit 5e9d3c2)

commit e59efecc43e06891dda02344edf177e446588d12
Author: Slava Koptilin 
Date:   2018-07-16T13:40:56Z

IGNITE-1094 Fixed hanging during cache initialization

Signed-off-by: Andrey Gura 
(cherry picked from commit a393e696212ef0dd4f23f923bf1001e0a48db915)

commit cd6793a941d38f724f04c2ee024c040a6b405f11
Author: Dmitriy Govorukhin 
Date:   2018-07-17T13:48:43Z

IGNITE-8684 Fixed infinite loop of partition single/full messages when 
partition state does not change - Fixes #4287.

(cherry picked from commit dd47fab)

commit 5d62af42c6d8a281cd522bda78cdb44272b7e36c
Author: Ivan Daschinskiy 
Date:   2018-07-17T13:52:22Z

IGNITE-8975 Invalid initialization of compressed archived WAL segment when 
WAL compression is switched off. - Fixes #4345.

Signed-off-by: Ivan Rakov 

(cherry picked from commit 46db052)

commit f9d6cd72171845d36ffb904a81868a78c247ec4a
Author: ezagumennov 
Date:   2018-06-19T15:04:34Z

IGNITE-8798 Move transaction recovery logging to INFO level

Signed-off-by: Ivan Rakov 

(cherry picked from commit 49a565c)
Signed-off-by: EdShangGG 

commit 103673631588355445133d9206bd545c5d699331
Author: Eduard Shangareev 
Date:   2018-07-12T16:26:48Z

IGNITE-8955 Fix of test after Checkpoint can't get write lock if massive 
eviction on node start started

(cherry picked from commit 584a88d)
Signed-off-by: EdShangGG 

commit 992df05fc2fd8dfc737920375bc48ffcdff51b98
Author: DmitriyGovorukhin 
Date:   2018-07-18T13:16:58Z

IGNITE-8929 Do not disable WAL if node does not have MOVING partitions. 
Fixes #4372

(cherry picked from commit 2b22933)

commit 56790babf46d36925be89e55a71862f71a6850ab
Author: Denis Mekhanikov 
Date:   2018-07-19T13:11:38Z

IGNITE-8922 Fix for discovery message delivery guarantee can be violated - 
Fixes #4349.

Signed-off-by: Dmitriy Pavlov 
(cherry picked from commit e2cf5cf99907825e8e19cf162c24a816b8ffd2df)

commit e6ca011040b1a004773c457839e86fb3dc2d59a1
Author: Eduard Shangareev 
Date:   2018-07-20T14:22:52Z

IGNITE-9039 Fixed non-releasing pinned page on throttling fallback - Fixes 
#4390.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit 0b0)

commit f3ca205ab2606f191909e6541915ce9ff899eed6
Author: Evgeny Stanilovskiy 
Date:   2018-07-23T08:56:21Z

IGNITE-8892 Fixed OOME when scan query is used for a big partition - Fixes 
#4391.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit a164296)

commit 7840c857b697ec2a1bf9bb09f0dbff4d90e80780
Author: Dmitriy Govorukhin 
Date:   2018-07-23T08:25:49Z

IGNITE-9042 Fixed partial tranasaction state wheh transaction is timed out 
- Fixes #4397.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit 33f485a)

commit 6e29cbd67d64f2a3cae774c62a3bf5b058834473
Author: Andrey V. Mashenkov 
Date:   2018-07-23T09:31:42Z

Merge remote-tracking branch 'origin/ignite-2.5-master' into 
ignite-2.5-master

commit edb22bcbb83e611b0124933355a5401fee796cb2
Author: Andrey V. Mashenkov 
Date:   2018-07-23T09:32:41Z

Merge remote-tracking branch 'origin/ignite-2.5-master' into 
ignite-2.5-master

commit eb92eef801266aff3486459e7ba6434e327b0d6a
Author: Evgeny Stanilovskiy 
Date:   2018-07-23T08:56:21Z

IGNITE-8892 Fixed OOME when scan query is used for a big partition - Fixes 
#4391.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit a164296)

commit f850032f2b9fb5463450573930baf00ccd0db31a
Author: Ivan Daschinskiy 
Date:   2018-07-23T12:29:08Z

IGNITE-8820 Fix rollback logic when tx is initiated from client. - Fixes 
#4399.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit 5794eb0)

commit dbaf1df276d1919c2f93ca0480e7aceeb084f1f3
Author: Dmitriy Govorukhin 
Date:   2018-07-23T15:01:37Z

IGNITE-9049 Fixed write of SWITCH_SEGMENT_RECORD at the end of a segment 
file - Fixes #4401.

Signed-off-by: Alexey Goncharuk 

[jira] [Created] (IGNITE-9774) CacheMvccTransactionsTest.testPutAllGetAll_ClientServer_Backups1_Restart_Scan hangs

[Ivan Pavlukhin (JIRA)]

Ivan Pavlukhin created IGNITE-9774:
--

 Summary: 
CacheMvccTransactionsTest.testPutAllGetAll_ClientServer_Backups1_Restart_Scan 
hangs
 Key: IGNITE-9774
 URL: https://issues.apache.org/jira/browse/IGNITE-9774
 Project: Ignite
  Issue Type: Bug
  Components: mvcc
Reporter: Ivan Pavlukhin


Test 
{{CacheMvccTransactionsTest#testPutAllGetAll_ClientServer_Backups1_Restart_Scan 
hangs}} and can freeze a whole suite. Investigation is needed. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4888: IGNITE-9760 Fixed NPE on concurrent WAL flush for...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4888


---

[GitHub] ignite pull request #4895: TDR-18 Introduce read-only cluster mode

[alex-plekhanov]

GitHub user alex-plekhanov opened a pull request:

https://github.com/apache/ignite/pull/4895

TDR-18 Introduce read-only cluster mode



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite tdr-18

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4895.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4895


commit 9d2cdcdf8facb1a358699fc4da1a7e1f1f6cebab
Author: Ivan Rakov 
Date:   2018-07-18T20:55:36Z

TDR-4: Refactor consistent cut logic as a separate entity

Signed-off-by: Ivan Rakov 

commit f290da12c6a607d8160ef668bbb646bb595f7d9a
Author: Slava Koptilin 
Date:   2018-07-20T15:53:19Z

txdr: keys that are participating in write operations should be included in 
writeKeys collection

commit 323f27d459abff222cfc0cc797331fdb17ac661d
Author: Ivan Rakov 
Date:   2018-07-20T16:54:36Z

Merge branch 'master' into txdr-master

commit 2ad80b64076613c837ae2d3a19419f19c5984100
Author: Ivan Rakov 
Date:   2018-07-23T09:29:56Z

Merge branch 'master' into txdr-master

commit 8557ae7134c93715ecd867da0366d301954c485b
Author: Dmitriy Sorokin 
Date:   2018-07-25T11:49:22Z

TXDR: Memory leak with unlimited growth of 
LocalPendingTransactionsTracker.preparedCommittedTxsCounters fixed.

commit 3a9e50d6523922c591985d1cea1b2fc146938697
Author: Ivan Rakov 
Date:   2018-07-25T13:52:59Z

Merge branch 'master' into txdr-master

# Conflicts:
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/GridDistributedTxRemoteAdapter.java

commit 4e63980ddeb20243d349acc84e54156d4c9fc911
Author: Aleksey Plekhanov 
Date:   2018-07-27T16:25:04Z

System property for local transaction tracker

commit 9bbb9e0666ae7f60e418fae09168ef292383fc5e
Author: Ivan Rakov 
Date:   2018-08-01T09:38:24Z

Merge branch 'master' into txdr-master

commit 62acf55b3af0d8fa672ca4876556b8a53afdeb6b
Author: Aleksey Plekhanov 
Date:   2018-08-01T13:54:52Z

System property flor local transaction tracker

commit 7ce7c30700293f949fbef85b3aeb3f05a227de6e
Author: Ivan Rakov 
Date:   2018-08-03T13:31:32Z

TDR-13 Create public API interface for transactional replication

Signed-off-by: Ivan Rakov 

commit d7c559880cd1ecac6729ad96ecded8ef4f24d56d
Author: Slava Koptilin 
Date:   2018-08-05T21:49:29Z

TDR-6 Build CC dependent transaction graph using key hashes instead of keys

Signed-off-by: Ivan Rakov 

commit 3badc1ae77d3cd87271089e75338dc4419f42c16
Author: Aleksey Plekhanov 
Date:   2018-08-05T21:59:33Z

TDR-15 Create test that will detect memory leaks in 
LocalPendingTransactionsTracker

Signed-off-by: Ivan Rakov 

commit 3f12544c6cb629f40c3c740e2483075c72c34103
Author: Dmitriy Sorokin 
Date:   2018-08-05T22:56:59Z

TDR-11 Implement handling of already COMMITTING transactions

Signed-off-by: Ivan Rakov 

commit 8c41acae041494cc834f5f07f491232a73a1ec9a
Author: Ivan Rakov 
Date:   2018-08-08T19:19:51Z

TDR-24 Refactor replication manager into GridPluginProcessor with 
ExchangeAware interface

Signed-off-by: Ivan Rakov 

commit 2a289afb482a25b8d6e80393285d8b12b888282b
Author: Ivan Rakov 
Date:   2018-08-09T13:52:13Z

Merge branch 'master' into txdr-master

commit 79ae004afe969d4acd86836912b3fa625ba34540
Author: Aleksey Plekhanov 
Date:   2018-08-09T15:51:38Z

TDR-22 Сonistent cuts watcher mechanism for replica cluster

commit 4b09f481d5bd5b735c196ab2edffb9d123a2b450
Author: Ivan Rakov 
Date:   2018-08-15T11:09:34Z

Merge branch 'master' into txdr-master

# Conflicts:
#   
modules/core/src/test/java/org/apache/ignite/internal/processors/cache/persistence/db/wal/IgniteWalIteratorSwitchSegmentTest.java

commit e3615cc194d5ad3ec30adb06457b5d06a3b597d7
Author: Ivan Rakov 
Date:   2018-08-17T15:20:39Z

Merge branch 'master' into txdr-master

commit cd034c4dbab1174e1ee95d8d937b8260a04e203d
Author: Ivan Rakov 
Date:   2018-08-20T13:21:27Z

Merge branch 'master' into txdr-master

commit 1488751b1595be50a6cef24320dd1062a057bd54
Author: Andrey Gura 
Date:   2018-08-20T21:24:02Z

TDR-9 Applying consistent cuts.

commit ebdd6d03d14a8af40ed9e8b9d83b6fbc95f24f35
Author: Slava Koptilin 
Date:   2018-08-22T17:09:44Z

TDR-16: bootstrap master cluster

commit 620a657a84fcb2d040f0053adb17a06a092166c8
Author: Ivan Rakov 
Date:   2018-08-27T18:13:46Z

TDR-43 Create TEST_SNAPSHOT_CREATE command for purposes of consistent cut 
testing

commit 5b4eb8cf079373fad60312a66dee5cb95f3b42cc
Author: Ivan Rakov 
Date:   2018-08-29T09:08:28Z

Added package-info

commit c1d07ccc6f4c1b42ffac7c192058b622aecb56bb
Author: Ivan Rakov 
Date:   2018-08-29T12:12:34Z

Added package description for Spring Data 2.0

commit a15a4fff0c310327217d698dbf56256964bc0389
Author: Ivan 

[GitHub] dspavlov closed pull request #20: Ignite 9645

[GitBox]

dspavlov closed pull request #20: Ignite 9645
URL: https://github.com/apache/ignite-teamcity-bot/pull/20
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITeamcity.java 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITeamcity.java
index 9481318..de34613 100644
--- a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITeamcity.java
+++ b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/ITeamcity.java
@@ -42,6 +42,7 @@
 import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrence;
 import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrenceFull;
 import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrences;
+import org.apache.ignite.ci.tcmodel.result.tests.TestRef;
 import org.apache.ignite.ci.util.Base64Util;
 import org.jetbrains.annotations.NotNull;
 
@@ -152,6 +153,8 @@ default Build getBuild(int id) {
 
 TestOccurrences getTests(String href, String normalizedBranch);
 
+TestOccurrences getFailedUnmutedTests(String href, String 
normalizedBranch);
+
 Statistics getBuildStatistics(String href);
 
 CompletableFuture getTestFull(String href);
@@ -160,6 +163,8 @@ default Build getBuild(int id) {
 
 ChangesList getChangesList(String href);
 
+CompletableFuture getTestRef(TestOccurrence occurrence);
+
 /**
  * List of build's related issues.
  *
diff --git 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
index bbf96a2..9def938 100644
--- 
a/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
+++ 
b/ignite-tc-helper-web/src/main/java/org/apache/ignite/ci/IgnitePersistentTeamcity.java
@@ -72,6 +72,7 @@
 import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrence;
 import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrenceFull;
 import org.apache.ignite.ci.tcmodel.result.tests.TestOccurrences;
+import org.apache.ignite.ci.tcmodel.result.tests.TestRef;
 import org.apache.ignite.ci.util.CacheUpdateUtil;
 import org.apache.ignite.ci.util.CollectionUtil;
 import org.apache.ignite.ci.util.ObjectInterner;
@@ -103,6 +104,7 @@
 private static final String BUILD_HIST_FINISHED = "buildHistFinished";
 private static final String BUILD_HIST_FINISHED_OR_FAILED = 
"buildHistFinishedOrFailed";
 public static final String BOT_DETECTED_ISSUES = "botDetectedIssues";
+public static final String TEST_REFS = "testRefs";
 
 //todo need separate cache or separate key for 'execution time' because it 
is placed in statistics
 private static final String BUILDS_FAILURE_RUN_STAT = 
"buildsFailureRunStat";
@@ -124,6 +126,11 @@
  */
 private ConcurrentMap> 
testOccFullFutures = new ConcurrentHashMap<>();
 
+/**
+ * cached loads of test refs.
+ */
+private ConcurrentMap> testRefsFutures 
= new ConcurrentHashMap<>();
+
 /** cached running builds for branch. */
 private ConcurrentMap>> queuedBuilds = 
new ConcurrentHashMap<>();
 
@@ -217,6 +224,13 @@ public void init(String serverId) {
 return getOrCreateCacheV2(ignCacheNme(TEST_FULL));
 }
 
+/**
+ * @return {@link TestRef} instances cache, 32 parts.
+ */
+private IgniteCache testRefsCache() {
+return getOrCreateCacheV2(ignCacheNme(TEST_REFS));
+}
+
 /**
  * @return Build {@link ProblemOccurrences} instances cache, 32 parts.
  */
@@ -231,7 +245,6 @@ public void init(String serverId) {
 return getOrCreateCacheV2(ignCacheNme(BUILD_STATISTICS));
 }
 
-
 /**
  * @return Build history: {@link BuildRef} lists cache, 32 parts, 
transactional.
  */
@@ -824,6 +837,11 @@ private void registerCriticalBuildProblemInStat(Build 
build, ProblemOccurrences
 hrefIgnored -> teamcity.getTests(href, normalizedBranch));
 }
 
+@AutoProfiling
+@Override public TestOccurrences getFailedUnmutedTests(String href, String 
normalizedBranch) {
+return getTests(href + ",muted:false,status:FAILURE", 
normalizedBranch);
+}
+
 private void addTestOccurrencesToStat(TestOccurrences val) {
 for (TestOccurrence next : val.getTests())
 addTestOccurrenceToStat(next, ITeamcity.DEFAULT, null);
@@ -871,6 +889,23 @@ private void addTestOccurrencesToStat(TestOccurrences val) 
{
 });
 }
 
+@Override
+@AutoProfiling
+public CompletableFuture getTestRef(TestOccurrence 
testOccurrence) {
+return CacheUpdateUtil.loadAsyncIfAbsent(
+testRefsCache(),
+testOccurrence.name,
+testRefsFutures,
+name -> {
+try {
+  

Re: [ML] New features and improvement of ML module for 2.7 release

[Alexey Zinoviev]

Whole documentation for all significant features/improvements was prepared by
different members of ML community and I hope it will be added to readme.io
by @ybabak in a few days.



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[GitHub] ignite pull request #4890: IGNITE-9761 fixed deadlock SegmentArchivedStorage...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4890


---

[GitHub] ignite pull request #4896: Ignite 9084 2.5.1

[macrergate]

GitHub user macrergate opened a pull request:

https://github.com/apache/ignite/pull/4896

Ignite 9084 2.5.1



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9084-2.5.1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4896.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4896


commit dc1cc39ffc0cdb2abc2767852883d6c8307cf9ea
Author: Anton Kalashnikov 
Date:   2018-05-31T13:06:52Z

IGNITE-8530 fixed onNodeLeft for InitNewCoordinatorFuture - Fixes #4086.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit 49fe8cd)

commit 383f3ad52d20045477fe4fed61d1d56e97ece35d
Author: dgladkikh 
Date:   2018-06-01T15:34:50Z

IGNITE-8603 Add JMX-metric to cluster: baseline nodes - Fixes #4060.

Signed-off-by: Ivan Rakov 

(cherry picked from commit 1f6266c)

commit a716bbee641a123f313325140ded10dc0c2de9ba
Author: Sergey Skudnov 
Date:   2018-05-14T10:35:14Z

IGNITE-8138 Uptime output with days - Fixes #3775.
Cherry-picked from dfb0b9ee35afeb6adc546160c37b08a85d869f59
Signed-off-by: dpavlov 

commit 9c381fd3ef0f75e2fed4562f562904ce5ede6bc3
Author: Ivan Daschinskiy 
Date:   2018-05-10T11:56:35Z

IGNITE-8424 Add directory sorting and skip duplicated config file in order 
to consistently start nodes.

Signed-off-by: Andrey Gura 

(cherry picked from commit bec3e9b)

commit dc8ba0932a944f3e966ebbe2af053488d1fe9b28
Author: Sergey Chugunov 
Date:   2018-05-04T12:26:02Z

IGNITE-8421 new implementation of getChildren method is added tolerating 
KeeperException.NoNodeException - Fixes #3939.

Signed-off-by: dpavlov 

(cherry-picked from commit #02e9ca993178d4aa648d06cb93ce1a9277eb22b1)

commit 0d37bd6749829ce07fbd4b0d2344b62588177628
Author: Ivan Rakov 
Date:   2018-06-05T11:42:43Z

IGNITE-8682 Attempt to configure IGFS in persistent mode without specific 
data region ends with AssertionError

(cherry picked from commit 2fe0a10)

commit 0c20c51025e5a550b7f3d8458acae27ec8cdcef5
Author: a-polyakov 
Date:   2018-06-05T15:49:19Z

IGNITE-8602 Add support filter "label null" for control.sh tx utility - 
Fixes #4073.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit 3372590)

commit 455777d2cd2bd6b83d69dbdbbd7e95b4f375bb75
Author: a-polyakov 
Date:   2018-06-05T16:01:03Z

IGNITE-8467 Fixed filter minSize for transactions utility control.sh. Fixes 
#4069

(cherry picked from commit d61c068)

commit c40f7659f24b6037485a5ac5e1965b0c6bd573bb
Author: Sergey Kosarev 
Date:   2018-06-06T08:19:26Z

IGNITE-8587 GridToStringBuilder use ConcurrentHashMap to avoid global locks 
on classCache - Fixes #4059.

Signed-off-by: Alexey Goncharuk 
(cherry picked from commit ecd8261f1add62099fde39aa7dca49855a866eda)

commit af026c7840101e2c57c7dd882baabddb6df2fa82
Author: Ilya Lantukh 
Date:   2018-06-04T22:24:37Z

IGNITE-8693 SQL JOIN between PARTITIONED and REPLICATED cache fails - Fixes 
#4120.

Signed-off-by: Ivan Rakov 

(cherry picked from commit bc35ce0)

commit a84538a9df8ace432411736289ec8371dee215e2
Author: Sergey Kosarev 
Date:   2018-06-06T09:59:28Z

IGNITE-8696 control.sh utility does not show atomicity mode - Fixes #4127.

Signed-off-by: Ivan Rakov 
(cherry picked from commit 78e5d970be74c04b01857123b1a623038aa18440)

commit fff8a4cc48ae7d8a47f8911763a71d0d59042d87
Author: Andrey Kuznetsov 
Date:   2018-06-06T12:02:35Z

IGNITE-8642 Added thread dumping to FailureProcessor

Signed-off-by: Andrey Gura 

commit 6ea23844cfb0294a0cf3f3c21787aeff30939bcf
Author: Dmitriy Sorokin 
Date:   2018-06-06T11:34:04Z

IGNITE-8311 IgniteClientRejoinTest.testClientsReconnectDisabled causes 
exchange-worker to terminate via NPE

Signed-off-by: Andrey Gura 

commit bad52d3843925d831e6d357a4382731751ae6bb7
Author: Ilya Lantukh 
Date:   2018-06-04T14:07:52Z

GG-13862 : Disabled fast eviction.

(cherry picked from commit 786633d)

commit 6956097ceedd2e3e9edcb627d97e49afb7f0a26d
Author: Pavel Kovalenko 
Date:   2018-06-06T12:24:39Z

IGNITE-8482 Skip 2-phase partition release wait in case of activation or 
dynamic caches start - Fixes #4078.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit 7c565d2)

commit d777c76eadd0a2ccc50f1031c42626b135da5906
Author: Dmitriy Govorukhin 
Date:   2018-06-06T14:20:18Z

IGNITE-8685 Fixed switch segment record size - Fixes #4130.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit 2a048bd)

commit ef4782d8a60eea870b43933480f2c9695c9fc55d
Author: Alexey Goncharuk 
Date:   2018-06-07T11:55:13Z

IGNITE-8706 Fixed testMemoryUsageMultipleNodes

commit 

Re: [ML] New features and improvement of ML module for 2.7 release

[Alexey Zinoviev]

Currently, in release 2.7, the ignite ML has a parity with a Spark ML by ML
algorithms, feature preprocessing and other capabilities.

I'm going to talk about that in October on two conferences

1) [Ru] Yaroslavl, Open Source Distributed Machine Learning Library for
Apache Ignite https://yappidays.ru/talks.html#zinovev 

2) [En] Minsk, Nuances of Machine Learning with Ignite ML,
https://jfuture.by/#talkbyAlexeyZinoviev

After my previous event, JUG MSK, the new contributor @Ravil Galeyev joined
to our community, hope for new members from Yaroslavl and Minsk soon



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

RE: Volunteer needed: AWS Elastic Load Balancer IP Findersimplemented

[Stanislav Lukyanov]

Hi,

I took a look at the code, and I believe the patch needs to be enhanced.

The patch is about adding TcpDiscoveryApplicationElbIpFinder, but it also 
deprecates the existing TcpDiscoveryElbIpFinder
and replaces it with its copy named TcpDiscoveryClassicElbIpFinder.

I’d really prefer if the existing class were enhanced to support both classic 
ELB and Application ELB.
If it can’t be done, let’s  use inheritance to reduce the copypaste.
In any case, let’s avoid deprecating the existing class – I don’t think that 
changing the name is that important in this case.

Thanks,
Stan

From: Denis Magda
Sent: 26 сентября 2018 г. 18:52
To: dev
Subject: Re: Volunteer needed: AWS Elastic Load Balancer IP Findersimplemented

Igniters,

Don't we have experts in our networking component? I do believe that's a
small improvement that can be verified and merged quickly.

--
Denis

On Wed, Sep 26, 2018 at 8:50 AM Dmitriy Pavlov 
wrote:

> Igniters, Friendly reminder.
>
> Denis, could you advice how to proceed in this case? It seems feature can
> be useful, but committers are not involved in a review.
>
> Should we send a proposal with lazy consensus and automatically merge? Any
> alternative ideas on how to proceed with issues stuck in the review process
> are strongly appreciated.
>
> Sincerely,
> Dmitriy Pavlov
>
> ср, 15 авг. 2018 г. в 19:38, Dmitriy Pavlov :
>
> > Hi Igniters,
> >
> > I took a look at the proposed contribution
> > https://issues.apache.org/jira/browse/IGNITE-8617 and PR
> > https://github.com/apache/ignite/pull/4076.
> >
> > I found this change reasonable and I can update code according to code
> > style.
> >
> > But I would ask someone from the community with experience with AWS ELB
> to
> > join the review. It would also benefit us if someone could try to run a
> > cluster with AWS ELB IP Finder.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
>

RE: Volunteer needed: AWS Elastic Load Balancer IP Findersimplemented

[Stanislav Lukyanov]

Also, it makes me nervous that we’re lacking any sort of functional tests for 
AWS-based IP finders (same for GCE, actually).
I understand that it’s hard to include tests like that in regular TC runs 
because we’d have to include some sort of credentials.
But let’s think of some sort of a middle ground.

I’m thinking about a functional test suite in the ignite-aws module that would 
pick up a properties file with AWS credentials.
It wouldn’t be included in any of the TC runs, but someone making changes to 
ignite-aws would be able to run it locally providing their own credentials.
They’d then share the results of their testing together with the usual TC link.

Stan

From: Stanislav Lukyanov
Sent: 2 октября 2018 г. 19:08
To: dev@ignite.apache.org
Subject: RE: Volunteer needed: AWS Elastic Load Balancer IP Findersimplemented

Hi,

I took a look at the code, and I believe the patch needs to be enhanced.

The patch is about adding TcpDiscoveryApplicationElbIpFinder, but it also 
deprecates the existing TcpDiscoveryElbIpFinder
and replaces it with its copy named TcpDiscoveryClassicElbIpFinder.

I’d really prefer if the existing class were enhanced to support both classic 
ELB and Application ELB.
If it can’t be done, let’s  use inheritance to reduce the copypaste.
In any case, let’s avoid deprecating the existing class – I don’t think that 
changing the name is that important in this case.

Thanks,
Stan

From: Denis Magda
Sent: 26 сентября 2018 г. 18:52
To: dev
Subject: Re: Volunteer needed: AWS Elastic Load Balancer IP Findersimplemented

Igniters,

Don't we have experts in our networking component? I do believe that's a
small improvement that can be verified and merged quickly.

--
Denis

On Wed, Sep 26, 2018 at 8:50 AM Dmitriy Pavlov 
wrote:

> Igniters, Friendly reminder.
>
> Denis, could you advice how to proceed in this case? It seems feature can
> be useful, but committers are not involved in a review.
>
> Should we send a proposal with lazy consensus and automatically merge? Any
> alternative ideas on how to proceed with issues stuck in the review process
> are strongly appreciated.
>
> Sincerely,
> Dmitriy Pavlov
>
> ср, 15 авг. 2018 г. в 19:38, Dmitriy Pavlov :
>
> > Hi Igniters,
> >
> > I took a look at the proposed contribution
> > https://issues.apache.org/jira/browse/IGNITE-8617 and PR
> > https://github.com/apache/ignite/pull/4076.
> >
> > I found this change reasonable and I can update code according to code
> > style.
> >
> > But I would ask someone from the community with experience with AWS ELB
> to
> > join the review. It would also benefit us if someone could try to run a
> > cluster with AWS ELB IP Finder.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
>

Re: Volunteer needed: AWS Elastic Load Balancer IP Findersimplemented

[uday kale]

Thanks for the review Stan.
I renamed the existing class TcpDiscoveryElbIpFinder since the name is not
clear regarding the actual load balancer being used. The names
TcpDiscoveryClassicElbIpFinder
and TcpDiscoveryApplicationElbIpFinder provide a clear distinction.
I deprecated the existing class because of review checklist [1] point 1.a
under checklist. It requires methods to be deprecated instead of being
renamed. I assumed this will extend to classes too.
Regarding the your suggestion for having the tests I agree. It will be
helpful to know whether TC can accept properties while initiating a run.

[1] https://cwiki.apache.org/confluence/display/IGNITE/Review+Checklist

Uday

On Tue, Oct 2, 2018 at 9:58 PM Stanislav Lukyanov 
wrote:

> Also, it makes me nervous that we’re lacking any sort of functional tests
> for AWS-based IP finders (same for GCE, actually).
> I understand that it’s hard to include tests like that in regular TC runs
> because we’d have to include some sort of credentials.
> But let’s think of some sort of a middle ground.
>
> I’m thinking about a functional test suite in the ignite-aws module that
> would pick up a properties file with AWS credentials.
> It wouldn’t be included in any of the TC runs, but someone making changes
> to ignite-aws would be able to run it locally providing their own
> credentials.
> They’d then share the results of their testing together with the usual TC
> link.
>
> Stan
>
> From: Stanislav Lukyanov
> Sent: 2 октября 2018 г. 19:08
> To: dev@ignite.apache.org
> Subject: RE: Volunteer needed: AWS Elastic Load Balancer IP
> Findersimplemented
>
> Hi,
>
> I took a look at the code, and I believe the patch needs to be enhanced.
>
> The patch is about adding TcpDiscoveryApplicationElbIpFinder, but it also
> deprecates the existing TcpDiscoveryElbIpFinder
> and replaces it with its copy named TcpDiscoveryClassicElbIpFinder.
>
> I’d really prefer if the existing class were enhanced to support both
> classic ELB and Application ELB.
> If it can’t be done, let’s  use inheritance to reduce the copypaste.
> In any case, let’s avoid deprecating the existing class – I don’t think
> that changing the name is that important in this case.
>
> Thanks,
> Stan
>
> From: Denis Magda
> Sent: 26 сентября 2018 г. 18:52
> To: dev
> Subject: Re: Volunteer needed: AWS Elastic Load Balancer IP
> Findersimplemented
>
> Igniters,
>
> Don't we have experts in our networking component? I do believe that's a
> small improvement that can be verified and merged quickly.
>
> --
> Denis
>
> On Wed, Sep 26, 2018 at 8:50 AM Dmitriy Pavlov 
> wrote:
>
> > Igniters, Friendly reminder.
> >
> > Denis, could you advice how to proceed in this case? It seems feature can
> > be useful, but committers are not involved in a review.
> >
> > Should we send a proposal with lazy consensus and automatically merge?
> Any
> > alternative ideas on how to proceed with issues stuck in the review
> process
> > are strongly appreciated.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > ср, 15 авг. 2018 г. в 19:38, Dmitriy Pavlov :
> >
> > > Hi Igniters,
> > >
> > > I took a look at the proposed contribution
> > > https://issues.apache.org/jira/browse/IGNITE-8617 and PR
> > > https://github.com/apache/ignite/pull/4076.
> > >
> > > I found this change reasonable and I can update code according to code
> > > style.
> > >
> > > But I would ask someone from the community with experience with AWS ELB
> > to
> > > join the review. It would also benefit us if someone could try to run a
> > > cluster with AWS ELB IP Finder.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> >
>
>
>

Re: Apache Ignite 2.7 release

[Andrey Kuznetsov]

I've caught a bug [1] in FsyncModeFileWriteAheadLogManager. It looks like a
release blocker to me.

[1] https://issues.apache.org/jira/browse/IGNITE-9776

вт, 2 окт. 2018 г. в 13:14, Dmitriy Pavlov :

> Hi Anton,
>
>  I definitely agree it is a blocker.
>
> Sincerely,
> Dmitriy Pavlov
>
> вт, 2 окт. 2018 г. в 13:09, Anton Kalashnikov :
>
> > Hi Igniters.
> >
> > I have one more possible blockers - deadlock in archiver -
> > https://issues.apache.org/jira/browse/IGNITE-9761. I almost fixed it.
> > It seems it should be include to scope.
> >
> > --
> > Best regards,
> > Anton Kalashnikov
> >
> >
> > 02.10.2018, 00:08, "Dmitriy Setrakyan" :
> > > Thanks, got it.
> > >
> > > On Mon, Oct 1, 2018 at 1:14 PM Dmitriy Pavlov 
> > wrote:
> > >
> > >>  Here I agree with Vladimir. Furthermore, I do my absolute best to
> > finalize
> > >>  all reviews in all 2.7 tickets I'm related to. I think most of the
> > >>  contributors doing the same.
> > >>
> > >>  пн, 1 окт. 2018 г. в 23:03, Vladimir Ozerov :
> > >>
> > >>  > This is precisely the scope we have at the moment. All these
> tickets
> > were
> > >>  > considered carefully on whether to include them into AI 2.7 scope.
> I
> > >>  would
> > >>  > say that 10-15% of current tickets may be moved furhter.
> > >>  >
> > >>  > Third of current tickets are features on their final review stages
> > (e.g.
> > >>  > TDE, MVCC invoke, TensorFlow, Thin Clients), another big part is
> > >>  > stabilization tickets (mainly - various test failures), and another
> > big
> > >>  > part is infrastructure (adopting new modules, Java 9+ support,
> > etc.). So
> > >>  > despite big absolute number, most of these tickets are grouped
> around
> > >>  > several big areas, and overall progress over this week should be
> very
> > >>  good.
> > >>  >
> > >>  > On Mon, Oct 1, 2018 at 9:50 PM Dmitriy Setrakyan <
> > dsetrak...@apache.org>
> > >>  > wrote:
> > >>  >
> > >>  > > If this filter is for 2.7 release, then I do not believe all
> these
> > >>  > tickets
> > >>  > > will be closed. It would be nice to leave only "must-have"
> tickets
> > in
> > >>  2.7
> > >>  > > and move the rest to 2.8.
> > >>  > >
> > >>  > > D.
> > >>  > >
> > >>  > > On Mon, Oct 1, 2018 at 11:02 AM Vladimir Ozerov <
> > voze...@gridgain.com>
> > >>  > > wrote:
> > >>  > >
> > >>  > > > Igniters,
> > >>  > > >
> > >>  > > > Please use this filter, as it properly handles tickets without
> > >>  > > components:
> > >>  > > >
> > >>  > > >
> > >>  > > >
> > >>  > >
> > >>  >
> > >>
> >
> https://issues.apache.org/jira/issues/?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20(%272.7%27)%20AND%20status%20NOT%20IN%20(Resolved%2C%20Closed)%20and%20(component%20is%20null%20or%20component%20not%20in%20(documentation)))%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20%20
> > >>  > > >
> > >>  > > > On Mon, Oct 1, 2018 at 6:18 PM Nikolay Izhikov <
> > nizhi...@apache.org>
> > >>  > > > wrote:
> > >>  > > >
> > >>  > > > > Hello, Igniters.
> > >>  > > > >
> > >>  > > > > I announce scope freeze for an Apache Ignite 2.7 release.
> > >>  > > > >
> > >>  > > > > It means:
> > >>  > > > >
> > >>  > > > > 1. We add to 2.7 only critical bugs.
> > >>  > > > > 2. We merge to 2.7 branch only previously announces features.
> > >>  > > > > 3. I expect we should exclude or *MERGE ALL TASKS FOR 2.7 DUE
> > TO
> > >>  > > OCTOBER
> > >>  > > > > 10*.
> > >>  > > > > So the *October 10 is DEADLINE* for new features.
> > >>  > > > >
> > >>  > > > > Thoughts?
> > >>  > > > >
> > >>  > > > > For now we have 34 In Progress tickets planned to 2.7 version
> > [1].
> > >>  > > > > So is you assigned to any of this ticker friendly reminder
> #1,
> > *the
> > >>  > > > > deadline is near :)*.
> > >>  > > > >
> > >>  > > > > [1]
> > >>  > > > >
> > >>  > > > >
> > >>  > > >
> > >>  > >
> > >>  >
> > >>
> >
> https://issues.apache.org/jira/browse/IGNITE-8803?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20(%272.7%27)%20AND%20status%20NOT%20IN%20(Resolved%2C%20Closed)%20and%20component%20!%3D%20documentation%20)%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20%20
> > >>  > > > >
> > >>  > > > >
> > >>  > > > >
> > >>  > > > > В Пн, 01/10/2018 в 16:13 +0300, Andrey Gura пишет:
> > >>  > > > > > Agree with Andrey.
> > >>  > > > > >
> > >>  > > > > > IGNITE-9723 will be merged to ignite-2.7 branch soon.
> > >>  > > > > > On Mon, Oct 1, 2018 at 3:56 PM Andrey Kuznetsov <
> > >>  stku...@gmail.com
> > >>  > >
> > >>  > > > > wrote:
> > >>  > > > > > >
> > >>  > > > > > > Igniters,
> > >>  > > > > > >
> > >>  > > > > > > There is an inaccuracy in critical worker termination
> > >>  detection,
> > >>  > > and
> > >>  > > > > I'm
> > >>  > > > > > > working on a fix right now, see [1]. Also, we have
> trivial
> > yet
> > >>  > > > > important
> > >>  > > > > > > fix [2], this one is ready to merge.
> > >>  > > > > > >
> > >>  > > > > 

[GitHub] ignite pull request #4897: IGNITE-9769 IgniteCacheAtomicProtocolTest.testPut...

[SomeFire]

GitHub user SomeFire opened a pull request:

https://github.com/apache/ignite/pull/4897

IGNITE-9769 IgniteCacheAtomicProtocolTest.testPutReaderUpdate1 is flaky



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/SomeFire/ignite IGNITE-9769

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4897.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4897


commit e7addd989e230953277ae16c14d61aa003a585ad
Author: Dmitrii Ryabov 
Date:   2018-10-02T18:06:48Z

IGNITE-9769 IgniteCacheAtomicProtocolTest.testPutReaderUpdate1 is flaky




---

[GitHub] ignite pull request #4898: 9769 tests

[SomeFire]

GitHub user SomeFire opened a pull request:

https://github.com/apache/ignite/pull/4898

9769 tests



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/SomeFire/ignite 9769-tests

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4898.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4898


commit e7addd989e230953277ae16c14d61aa003a585ad
Author: Dmitrii Ryabov 
Date:   2018-10-02T18:06:48Z

IGNITE-9769 IgniteCacheAtomicProtocolTest.testPutReaderUpdate1 is flaky

commit 582ad7891d950d3c51f20752255fbfad9e9b2dcc
Author: Dmitrii Ryabov 
Date:   2018-10-02T18:16:06Z

test flakiness




---

[jira] [Created] (IGNITE-9776) FsyncModeFileWriteAheadLogManager can block forever in log() call

[Andrey Kuznetsov (JIRA)]

Andrey Kuznetsov created IGNITE-9776:


 Summary: FsyncModeFileWriteAheadLogManager can block forever in 
log() call
 Key: IGNITE-9776
 URL: https://issues.apache.org/jira/browse/IGNITE-9776
 Project: Ignite
  Issue Type: Bug
Reporter: Andrey Kuznetsov
 Fix For: 2.7
 Attachments: FsyncWalRolloverDoesNotBlockTest.java

If WAL archiver is disabled and WALRecord being logged has {{rollOver() == 
true}}, then {{log()}} call blocks forever in {{FileArchiver}}'s (!) method:

{noformat}
nextAbsoluteSegmentIndex:1707, FsyncModeFileWriteAheadLogManager$FileArchiver 
(org.apache.ignite.internal.processors.cache.persistence.wal)
access$3200:1437, FsyncModeFileWriteAheadLogManager$FileArchiver 
(org.apache.ignite.internal.processors.cache.persistence.wal)
pollNextFile:1384, FsyncModeFileWriteAheadLogManager 
(org.apache.ignite.internal.processors.cache.persistence.wal)
initNextWriteHandle:1243, FsyncModeFileWriteAheadLogManager 
(org.apache.ignite.internal.processors.cache.persistence.wal)
rollOver:1130, FsyncModeFileWriteAheadLogManager 
(org.apache.ignite.internal.processors.cache.persistence.wal)
log:712, FsyncModeFileWriteAheadLogManager 
(org.apache.ignite.internal.processors.cache.persistence.wal)
{noformat}

Reporoducer is attached.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Volunteer needed: AWS Elastic Load Balancer IP Findersimplemented

[Dmitriy Pavlov]

Hi Uday,

We should keep classes name as is within all minor releases (2.x). We can
schedule rename only to 3.0. We need to keep both classes and methods
naming as is to provide compilation guarantee. If someone used existing
TcpDiscoveryElbIpFinder from 2.6 release than his or her code should
compile and run well.

I've updated checklist, thank you for pointing to this.

So I suggest keeping existing naming of TcpDiscoveryElbIpFinder as it is
part of API.

Sincerely,
Dmitriy Pavlov

вт, 2 окт. 2018 г. в 21:04, uday kale :

> Thanks for the review Stan.
> I renamed the existing class TcpDiscoveryElbIpFinder since the name is not
> clear regarding the actual load balancer being used. The names
> TcpDiscoveryClassicElbIpFinder
> and TcpDiscoveryApplicationElbIpFinder provide a clear distinction.
> I deprecated the existing class because of review checklist [1] point 1.a
> under checklist. It requires methods to be deprecated instead of being
> renamed. I assumed this will extend to classes too.
> Regarding the your suggestion for having the tests I agree. It will be
> helpful to know whether TC can accept properties while initiating a run.
>
> [1] https://cwiki.apache.org/confluence/display/IGNITE/Review+Checklist
>
> Uday
>
> On Tue, Oct 2, 2018 at 9:58 PM Stanislav Lukyanov 
> wrote:
>
> > Also, it makes me nervous that we’re lacking any sort of functional tests
> > for AWS-based IP finders (same for GCE, actually).
> > I understand that it’s hard to include tests like that in regular TC runs
> > because we’d have to include some sort of credentials.
> > But let’s think of some sort of a middle ground.
> >
> > I’m thinking about a functional test suite in the ignite-aws module that
> > would pick up a properties file with AWS credentials.
> > It wouldn’t be included in any of the TC runs, but someone making changes
> > to ignite-aws would be able to run it locally providing their own
> > credentials.
> > They’d then share the results of their testing together with the usual TC
> > link.
> >
> > Stan
> >
> > From: Stanislav Lukyanov
> > Sent: 2 октября 2018 г. 19:08
> > To: dev@ignite.apache.org
> > Subject: RE: Volunteer needed: AWS Elastic Load Balancer IP
> > Findersimplemented
> >
> > Hi,
> >
> > I took a look at the code, and I believe the patch needs to be enhanced.
> >
> > The patch is about adding TcpDiscoveryApplicationElbIpFinder, but it also
> > deprecates the existing TcpDiscoveryElbIpFinder
> > and replaces it with its copy named TcpDiscoveryClassicElbIpFinder.
> >
> > I’d really prefer if the existing class were enhanced to support both
> > classic ELB and Application ELB.
> > If it can’t be done, let’s  use inheritance to reduce the copypaste.
> > In any case, let’s avoid deprecating the existing class – I don’t think
> > that changing the name is that important in this case.
> >
> > Thanks,
> > Stan
> >
> > From: Denis Magda
> > Sent: 26 сентября 2018 г. 18:52
> > To: dev
> > Subject: Re: Volunteer needed: AWS Elastic Load Balancer IP
> > Findersimplemented
> >
> > Igniters,
> >
> > Don't we have experts in our networking component? I do believe that's a
> > small improvement that can be verified and merged quickly.
> >
> > --
> > Denis
> >
> > On Wed, Sep 26, 2018 at 8:50 AM Dmitriy Pavlov 
> > wrote:
> >
> > > Igniters, Friendly reminder.
> > >
> > > Denis, could you advice how to proceed in this case? It seems feature
> can
> > > be useful, but committers are not involved in a review.
> > >
> > > Should we send a proposal with lazy consensus and automatically merge?
> > Any
> > > alternative ideas on how to proceed with issues stuck in the review
> > process
> > > are strongly appreciated.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > ср, 15 авг. 2018 г. в 19:38, Dmitriy Pavlov :
> > >
> > > > Hi Igniters,
> > > >
> > > > I took a look at the proposed contribution
> > > > https://issues.apache.org/jira/browse/IGNITE-8617 and PR
> > > > https://github.com/apache/ignite/pull/4076.
> > > >
> > > > I found this change reasonable and I can update code according to
> code
> > > > style.
> > > >
> > > > But I would ask someone from the community with experience with AWS
> ELB
> > > to
> > > > join the review. It would also benefit us if someone could try to
> run a
> > > > cluster with AWS ELB IP Finder.
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > >
> > >
> >
> >
> >
>

Re: Apache Ignite 2.7 release

[Vladimir Ozerov]

Andrey, Anton,

How do you conclude that these tickets are blockers? What is the impact to
users and in what circumstances users can met them?

Note that we have many hundreds opened bugs, and yet we do not strive to
include them all, because bug != blocker.

So -1 from my side to including these tickets to release scope, unless
impact is explained.

Vladimir.

вт, 2 окт. 2018 г. в 22:45, Andrey Kuznetsov :

> I've caught a bug [1] in FsyncModeFileWriteAheadLogManager. It looks like a
> release blocker to me.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-9776
>
> вт, 2 окт. 2018 г. в 13:14, Dmitriy Pavlov :
>
> > Hi Anton,
> >
> >  I definitely agree it is a blocker.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > вт, 2 окт. 2018 г. в 13:09, Anton Kalashnikov :
> >
> > > Hi Igniters.
> > >
> > > I have one more possible blockers - deadlock in archiver -
> > > https://issues.apache.org/jira/browse/IGNITE-9761. I almost fixed it.
> > > It seems it should be include to scope.
> > >
> > > --
> > > Best regards,
> > > Anton Kalashnikov
> > >
> > >
> > > 02.10.2018, 00:08, "Dmitriy Setrakyan" :
> > > > Thanks, got it.
> > > >
> > > > On Mon, Oct 1, 2018 at 1:14 PM Dmitriy Pavlov  >
> > > wrote:
> > > >
> > > >>  Here I agree with Vladimir. Furthermore, I do my absolute best to
> > > finalize
> > > >>  all reviews in all 2.7 tickets I'm related to. I think most of the
> > > >>  contributors doing the same.
> > > >>
> > > >>  пн, 1 окт. 2018 г. в 23:03, Vladimir Ozerov  >:
> > > >>
> > > >>  > This is precisely the scope we have at the moment. All these
> > tickets
> > > were
> > > >>  > considered carefully on whether to include them into AI 2.7
> scope.
> > I
> > > >>  would
> > > >>  > say that 10-15% of current tickets may be moved furhter.
> > > >>  >
> > > >>  > Third of current tickets are features on their final review
> stages
> > > (e.g.
> > > >>  > TDE, MVCC invoke, TensorFlow, Thin Clients), another big part is
> > > >>  > stabilization tickets (mainly - various test failures), and
> another
> > > big
> > > >>  > part is infrastructure (adopting new modules, Java 9+ support,
> > > etc.). So
> > > >>  > despite big absolute number, most of these tickets are grouped
> > around
> > > >>  > several big areas, and overall progress over this week should be
> > very
> > > >>  good.
> > > >>  >
> > > >>  > On Mon, Oct 1, 2018 at 9:50 PM Dmitriy Setrakyan <
> > > dsetrak...@apache.org>
> > > >>  > wrote:
> > > >>  >
> > > >>  > > If this filter is for 2.7 release, then I do not believe all
> > these
> > > >>  > tickets
> > > >>  > > will be closed. It would be nice to leave only "must-have"
> > tickets
> > > in
> > > >>  2.7
> > > >>  > > and move the rest to 2.8.
> > > >>  > >
> > > >>  > > D.
> > > >>  > >
> > > >>  > > On Mon, Oct 1, 2018 at 11:02 AM Vladimir Ozerov <
> > > voze...@gridgain.com>
> > > >>  > > wrote:
> > > >>  > >
> > > >>  > > > Igniters,
> > > >>  > > >
> > > >>  > > > Please use this filter, as it properly handles tickets
> without
> > > >>  > > components:
> > > >>  > > >
> > > >>  > > >
> > > >>  > > >
> > > >>  > >
> > > >>  >
> > > >>
> > >
> >
> https://issues.apache.org/jira/issues/?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20(%272.7%27)%20AND%20status%20NOT%20IN%20(Resolved%2C%20Closed)%20and%20(component%20is%20null%20or%20component%20not%20in%20(documentation)))%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20%20
> > > >>  > > >
> > > >>  > > > On Mon, Oct 1, 2018 at 6:18 PM Nikolay Izhikov <
> > > nizhi...@apache.org>
> > > >>  > > > wrote:
> > > >>  > > >
> > > >>  > > > > Hello, Igniters.
> > > >>  > > > >
> > > >>  > > > > I announce scope freeze for an Apache Ignite 2.7 release.
> > > >>  > > > >
> > > >>  > > > > It means:
> > > >>  > > > >
> > > >>  > > > > 1. We add to 2.7 only critical bugs.
> > > >>  > > > > 2. We merge to 2.7 branch only previously announces
> features.
> > > >>  > > > > 3. I expect we should exclude or *MERGE ALL TASKS FOR 2.7
> DUE
> > > TO
> > > >>  > > OCTOBER
> > > >>  > > > > 10*.
> > > >>  > > > > So the *October 10 is DEADLINE* for new features.
> > > >>  > > > >
> > > >>  > > > > Thoughts?
> > > >>  > > > >
> > > >>  > > > > For now we have 34 In Progress tickets planned to 2.7
> version
> > > [1].
> > > >>  > > > > So is you assigned to any of this ticker friendly reminder
> > #1,
> > > *the
> > > >>  > > > > deadline is near :)*.
> > > >>  > > > >
> > > >>  > > > > [1]
> > > >>  > > > >
> > > >>  > > > >
> > > >>  > > >
> > > >>  > >
> > > >>  >
> > > >>
> > >
> >
> https://issues.apache.org/jira/browse/IGNITE-8803?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20(%272.7%27)%20AND%20status%20NOT%20IN%20(Resolved%2C%20Closed)%20and%20component%20!%3D%20documentation%20)%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20%20
> > > >>  > > > >
> > > >>  > > > >
> > > >>  > > > >
> > > >>  > > > > В Пн, 01/10/2018 

Re: Apache Ignite 2.7 release

[Dmitriy Pavlov]

Hi Igniters, Vladimir,

NPEs or hangs in WAL is a completely non-functional grid (if persistence
enabled).

I see no reasons to release 2.7 with such symptoms until we're sure it is
too rare/impossible to reproduce. But it seems it is not the case. I will
definitely vote -1 for the release if I'm aware of such problems exist and
were not researched. Community guarantees the quality and usability of the
product.

We should ask and answer other questions:
1) why there are a lot of NPEs and hangs reported recently in the same area
2) and why we signed-off commit(s).

Probably we can identify and revert these commit(s) from 2.7 and research
these failures in master (with no rush).

Sincerely,
Dmitriy Pavlov

вт, 2 окт. 2018 г. в 23:54, Vladimir Ozerov :

> Andrey, Anton,
>
> How do you conclude that these tickets are blockers? What is the impact to
> users and in what circumstances users can met them?
>
> Note that we have many hundreds opened bugs, and yet we do not strive to
> include them all, because bug != blocker.
>
> So -1 from my side to including these tickets to release scope, unless
> impact is explained.
>
> Vladimir.
>
> вт, 2 окт. 2018 г. в 22:45, Andrey Kuznetsov :
>
> > I've caught a bug [1] in FsyncModeFileWriteAheadLogManager. It looks
> like a
> > release blocker to me.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-9776
> >
> > вт, 2 окт. 2018 г. в 13:14, Dmitriy Pavlov :
> >
> > > Hi Anton,
> > >
> > >  I definitely agree it is a blocker.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > вт, 2 окт. 2018 г. в 13:09, Anton Kalashnikov :
> > >
> > > > Hi Igniters.
> > > >
> > > > I have one more possible blockers - deadlock in archiver -
> > > > https://issues.apache.org/jira/browse/IGNITE-9761. I almost fixed
> it.
> > > > It seems it should be include to scope.
> > > >
> > > > --
> > > > Best regards,
> > > > Anton Kalashnikov
> > > >
> > > >
> > > > 02.10.2018, 00:08, "Dmitriy Setrakyan" :
> > > > > Thanks, got it.
> > > > >
> > > > > On Mon, Oct 1, 2018 at 1:14 PM Dmitriy Pavlov <
> dpavlov@gmail.com
> > >
> > > > wrote:
> > > > >
> > > > >>  Here I agree with Vladimir. Furthermore, I do my absolute best to
> > > > finalize
> > > > >>  all reviews in all 2.7 tickets I'm related to. I think most of
> the
> > > > >>  contributors doing the same.
> > > > >>
> > > > >>  пн, 1 окт. 2018 г. в 23:03, Vladimir Ozerov <
> voze...@gridgain.com
> > >:
> > > > >>
> > > > >>  > This is precisely the scope we have at the moment. All these
> > > tickets
> > > > were
> > > > >>  > considered carefully on whether to include them into AI 2.7
> > scope.
> > > I
> > > > >>  would
> > > > >>  > say that 10-15% of current tickets may be moved furhter.
> > > > >>  >
> > > > >>  > Third of current tickets are features on their final review
> > stages
> > > > (e.g.
> > > > >>  > TDE, MVCC invoke, TensorFlow, Thin Clients), another big part
> is
> > > > >>  > stabilization tickets (mainly - various test failures), and
> > another
> > > > big
> > > > >>  > part is infrastructure (adopting new modules, Java 9+ support,
> > > > etc.). So
> > > > >>  > despite big absolute number, most of these tickets are grouped
> > > around
> > > > >>  > several big areas, and overall progress over this week should
> be
> > > very
> > > > >>  good.
> > > > >>  >
> > > > >>  > On Mon, Oct 1, 2018 at 9:50 PM Dmitriy Setrakyan <
> > > > dsetrak...@apache.org>
> > > > >>  > wrote:
> > > > >>  >
> > > > >>  > > If this filter is for 2.7 release, then I do not believe all
> > > these
> > > > >>  > tickets
> > > > >>  > > will be closed. It would be nice to leave only "must-have"
> > > tickets
> > > > in
> > > > >>  2.7
> > > > >>  > > and move the rest to 2.8.
> > > > >>  > >
> > > > >>  > > D.
> > > > >>  > >
> > > > >>  > > On Mon, Oct 1, 2018 at 11:02 AM Vladimir Ozerov <
> > > > voze...@gridgain.com>
> > > > >>  > > wrote:
> > > > >>  > >
> > > > >>  > > > Igniters,
> > > > >>  > > >
> > > > >>  > > > Please use this filter, as it properly handles tickets
> > without
> > > > >>  > > components:
> > > > >>  > > >
> > > > >>  > > >
> > > > >>  > > >
> > > > >>  > >
> > > > >>  >
> > > > >>
> > > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20(%272.7%27)%20AND%20status%20NOT%20IN%20(Resolved%2C%20Closed)%20and%20(component%20is%20null%20or%20component%20not%20in%20(documentation)))%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20%20
> > > > >>  > > >
> > > > >>  > > > On Mon, Oct 1, 2018 at 6:18 PM Nikolay Izhikov <
> > > > nizhi...@apache.org>
> > > > >>  > > > wrote:
> > > > >>  > > >
> > > > >>  > > > > Hello, Igniters.
> > > > >>  > > > >
> > > > >>  > > > > I announce scope freeze for an Apache Ignite 2.7 release.
> > > > >>  > > > >
> > > > >>  > > > > It means:
> > > > >>  > > > >
> > > > >>  > > > > 1. We add to 2.7 only critical bugs.
> > > > >>  > > > > 2. We merge to 2.7 branch only 

[jira] [Created] (IGNITE-9771) Indirect SQL queries are failing

[Sergey Grimstad (JIRA)]

Sergey Grimstad created IGNITE-9771:
---

 Summary: Indirect SQL queries are failing
 Key: IGNITE-9771
 URL: https://issues.apache.org/jira/browse/IGNITE-9771
 Project: Ignite
  Issue Type: Bug
  Components: sql
Affects Versions: 2.7
Reporter: Sergey Grimstad
Assignee: Sergey Grimstad
 Fix For: 2.8


Attempt to sql query (not sql fields query ) another cache's type leads to 
exception:

Caused by: class 
org.apache.ignite.internal.processors.query.IgniteSQLException: Failed to find 
SQL table for type: ...

IgniteH2Indexing#queryDistributedSql in the beginning of the method logic there 
is check of schemaName, cacheName and type. First two parameters passed belong 
to called cache while third belongs to target cache. This combination leads to 
the exception mentioned.
 # Create tests to reproduce the situation
 # Fix problem
 # Enable commented out tests with 'todo' and this ticket number reference



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [ML] New features and improvement of ML module for 2.7 release

[Denis Magda]

Alexey,

Thanks for spreading the word about the ML capabilities! *Prachi*, please
help us to add the talks Alexey is going to give to Ignite events page:
https://ignite.apache.org/events.html

Btw, I gave a presentation about Ignite ML + TensorFlow integration today
at IMC Summit in the US. It was perceived really well, was bombarded with
many questions after the talk and think we've got some potential users ;)

--
Denis

On Tue, Oct 2, 2018 at 8:54 AM Alexey Zinoviev 
wrote:

> Currently, in release 2.7, the ignite ML has a parity with a Spark ML by ML
> algorithms, feature preprocessing and other capabilities.
>
> I'm going to talk about that in October on two conferences
>
> 1) [Ru] Yaroslavl, Open Source Distributed Machine Learning Library for
> Apache Ignite https://yappidays.ru/talks.html#zinovev
>
> 2) [En] Minsk, Nuances of Machine Learning with Ignite ML,
> https://jfuture.by/#talkbyAlexeyZinoviev
>
> After my previous event, JUG MSK, the new contributor @Ravil Galeyev joined
> to our community, hope for new members from Yaroslavl and Minsk soon
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

[GitHub] ignite pull request #4900: Network discovery using consul and S3 resolver to...

[rahulbsw]

GitHub user rahulbsw opened a pull request:

https://github.com/apache/ignite/pull/4900

Network discovery using consul and S3 resolver to handle public and private 
IP mapping

Module added to discover node using consul,
update aws module adding S3 base address resolution service which will help 
to handle accessing ignite cluster outside of aws using public and private ip 
map

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rahulbsw/ignite master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4900.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4900


commit 597ef0ba939e56720eae2a223de650664b91e271
Author: Rahul 
Date:   2018-04-09T22:17:21Z

AWS S3 IP Finder SPI ,
   replace depricated AWS APIs with new API
   added support to include public IP (for public address resolution)
Added new S3 Address resolution depends on update S3 IP Finders SPI
Add Consul IP Finder SPI and address resolvers

commit 18eda56c5be15d2dce55b0556ee8c85ee6b1dd7f
Author: Rahul 
Date:   2018-04-10T03:26:36Z

Merge pull request #1 from apache/master

Merge from apache/ignite master

commit 3202e71257418320e823093940461b0c03973812
Author: Rahul 
Date:   2018-04-21T20:11:53Z

AWS S3 IP Finder SPI ,
   replace depricated AWS APIs with new API
   added support to include public IP (for public address resolution)
Added new S3 Address resolution depends on update S3 IP Finders SPI
Add Consul IP Finder SPI and address resolvers

commit 80a0b997c884d649f7a36f0e84e93e4747c27f02
Author: Rahul 
Date:   2018-04-21T20:18:11Z

Merge remote-tracking branch 'origin/master' into aws-update_consul_support

commit 93a82501a4544a03e17d38ee5e747551438f19a9
Author: Rahul 
Date:   2018-10-03T03:56:30Z

Merge branch 'aws-update_consul_support' into pr/2

# Conflicts:
#   
modules/aws/src/main/java/org/apache/ignite/spi/discovery/tcp/ipfinder/s3/TcpDiscoveryS3IpFinder.java

commit 3145ab10b12a176693d96e127211a24df61e08e9
Author: Rahul 
Date:   2018-10-03T04:57:33Z

resolved merge conflicts




---

[GitHub] ignite pull request #4899: Network discovery using consul and S3 resolver to...

[rahulbsw]

Github user rahulbsw closed the pull request at:

https://github.com/apache/ignite/pull/4899


---

[GitHub] ignite pull request #4899: Network discovery using consul and S3 resolver to...

[rahulbsw]

GitHub user rahulbsw opened a pull request:

https://github.com/apache/ignite/pull/4899

Network discovery using consul and S3 resolver to handle public and private 
IP mapping

Module added to discover node using consul,
update aws module adding S3 base address resolution service which will help 
to handle accessing ignite cluster outside of aws using public and private ip 
map 


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rahulbsw/ignite master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4899.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4899


commit 597ef0ba939e56720eae2a223de650664b91e271
Author: Rahul 
Date:   2018-04-09T22:17:21Z

AWS S3 IP Finder SPI ,
   replace depricated AWS APIs with new API
   added support to include public IP (for public address resolution)
Added new S3 Address resolution depends on update S3 IP Finders SPI
Add Consul IP Finder SPI and address resolvers

commit 18eda56c5be15d2dce55b0556ee8c85ee6b1dd7f
Author: Rahul 
Date:   2018-04-10T03:26:36Z

Merge pull request #1 from apache/master

Merge from apache/ignite master

commit 3202e71257418320e823093940461b0c03973812
Author: Rahul 
Date:   2018-04-21T20:11:53Z

AWS S3 IP Finder SPI ,
   replace depricated AWS APIs with new API
   added support to include public IP (for public address resolution)
Added new S3 Address resolution depends on update S3 IP Finders SPI
Add Consul IP Finder SPI and address resolvers

commit 80a0b997c884d649f7a36f0e84e93e4747c27f02
Author: Rahul 
Date:   2018-04-21T20:18:11Z

Merge remote-tracking branch 'origin/master' into aws-update_consul_support




---

Re: Apache Ignite 2.7 release

[Nikolay Izhikov]

Hello, Dmitriy.

I'm sorry, but I don't understand your concern.

Vladimir just asks experienced Ignite contributor to *explain impact* of a bug.

Why are you scaring us with your "-1"?
Is it Apache Way to do so?
What should be done for you to return to a constructive discussion?

В Ср, 03/10/2018 в 00:23 +0300, Dmitriy Pavlov пишет:
> Hi Igniters, Vladimir,
> 
> NPEs or hangs in WAL is a completely non-functional grid (if persistence
> enabled).
> 
> I see no reasons to release 2.7 with such symptoms until we're sure it is
> too rare/impossible to reproduce. But it seems it is not the case. I will
> definitely vote -1 for the release if I'm aware of such problems exist and
> were not researched. Community guarantees the quality and usability of the
> product.
> 
> We should ask and answer other questions:
> 1) why there are a lot of NPEs and hangs reported recently in the same area
> 2) and why we signed-off commit(s).
> 
> Probably we can identify and revert these commit(s) from 2.7 and research
> these failures in master (with no rush).
> 
> Sincerely,
> Dmitriy Pavlov
> 
> вт, 2 окт. 2018 г. в 23:54, Vladimir Ozerov :
> 
> > Andrey, Anton,
> > 
> > How do you conclude that these tickets are blockers? What is the impact to
> > users and in what circumstances users can met them?
> > 
> > Note that we have many hundreds opened bugs, and yet we do not strive to
> > include them all, because bug != blocker.
> > 
> > So -1 from my side to including these tickets to release scope, unless
> > impact is explained.
> > 
> > Vladimir.
> > 
> > вт, 2 окт. 2018 г. в 22:45, Andrey Kuznetsov :
> > 
> > > I've caught a bug [1] in FsyncModeFileWriteAheadLogManager. It looks
> > 
> > like a
> > > release blocker to me.
> > > 
> > > [1] https://issues.apache.org/jira/browse/IGNITE-9776
> > > 
> > > вт, 2 окт. 2018 г. в 13:14, Dmitriy Pavlov :
> > > 
> > > > Hi Anton,
> > > > 
> > > >  I definitely agree it is a blocker.
> > > > 
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > > 
> > > > вт, 2 окт. 2018 г. в 13:09, Anton Kalashnikov :
> > > > 
> > > > > Hi Igniters.
> > > > > 
> > > > > I have one more possible blockers - deadlock in archiver -
> > > > > https://issues.apache.org/jira/browse/IGNITE-9761. I almost fixed
> > 
> > it.
> > > > > It seems it should be include to scope.
> > > > > 
> > > > > --
> > > > > Best regards,
> > > > > Anton Kalashnikov
> > > > > 
> > > > > 
> > > > > 02.10.2018, 00:08, "Dmitriy Setrakyan" :
> > > > > > Thanks, got it.
> > > > > > 
> > > > > > On Mon, Oct 1, 2018 at 1:14 PM Dmitriy Pavlov <
> > 
> > dpavlov@gmail.com
> > > > 
> > > > > wrote:
> > > > > > 
> > > > > > >  Here I agree with Vladimir. Furthermore, I do my absolute best to
> > > > > 
> > > > > finalize
> > > > > > >  all reviews in all 2.7 tickets I'm related to. I think most of
> > 
> > the
> > > > > > >  contributors doing the same.
> > > > > > > 
> > > > > > >  пн, 1 окт. 2018 г. в 23:03, Vladimir Ozerov <
> > 
> > voze...@gridgain.com
> > > > :
> > > > > > > 
> > > > > > >  > This is precisely the scope we have at the moment. All these
> > > > 
> > > > tickets
> > > > > were
> > > > > > >  > considered carefully on whether to include them into AI 2.7
> > > 
> > > scope.
> > > > I
> > > > > > >  would
> > > > > > >  > say that 10-15% of current tickets may be moved furhter.
> > > > > > >  >
> > > > > > >  > Third of current tickets are features on their final review
> > > 
> > > stages
> > > > > (e.g.
> > > > > > >  > TDE, MVCC invoke, TensorFlow, Thin Clients), another big part
> > 
> > is
> > > > > > >  > stabilization tickets (mainly - various test failures), and
> > > 
> > > another
> > > > > big
> > > > > > >  > part is infrastructure (adopting new modules, Java 9+ support,
> > > > > 
> > > > > etc.). So
> > > > > > >  > despite big absolute number, most of these tickets are grouped
> > > > 
> > > > around
> > > > > > >  > several big areas, and overall progress over this week should
> > 
> > be
> > > > very
> > > > > > >  good.
> > > > > > >  >
> > > > > > >  > On Mon, Oct 1, 2018 at 9:50 PM Dmitriy Setrakyan <
> > > > > 
> > > > > dsetrak...@apache.org>
> > > > > > >  > wrote:
> > > > > > >  >
> > > > > > >  > > If this filter is for 2.7 release, then I do not believe all
> > > > 
> > > > these
> > > > > > >  > tickets
> > > > > > >  > > will be closed. It would be nice to leave only "must-have"
> > > > 
> > > > tickets
> > > > > in
> > > > > > >  2.7
> > > > > > >  > > and move the rest to 2.8.
> > > > > > >  > >
> > > > > > >  > > D.
> > > > > > >  > >
> > > > > > >  > > On Mon, Oct 1, 2018 at 11:02 AM Vladimir Ozerov <
> > > > > 
> > > > > voze...@gridgain.com>
> > > > > > >  > > wrote:
> > > > > > >  > >
> > > > > > >  > > > Igniters,
> > > > > > >  > > >
> > > > > > >  > > > Please use this filter, as it properly handles tickets
> > > 
> > > without
> > > > > > >  > > components:
> > > > > > >  > > >
> > > > > > >  > > >
> > > > > > >  > > >
> > > > > > >  > >
> > > > > > >  >
> > > > > > > 
> > 
> > 

[jira] [Created] (IGNITE-9759) CacheAsyncOperationsFailoverTxTest#testAsyncFailover test hangs up

[Alexey Stelmak (JIRA)]

Alexey Stelmak created IGNITE-9759:
--

 Summary: CacheAsyncOperationsFailoverTxTest#testAsyncFailover test 
hangs up
 Key: IGNITE-9759
 URL: https://issues.apache.org/jira/browse/IGNITE-9759
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Stelmak


The test stops, because not all futures completed correctly when nodes are 
stopped.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4886: IGNITE-8803 Corrected output for quiet mode

[agoncharuk]

GitHub user agoncharuk opened a pull request:

https://github.com/apache/ignite/pull/4886

IGNITE-8803 Corrected output for quiet mode



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-8803

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4886.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4886


commit 6c7f93c3cf948d1d51ed4c3ed64397ec252a43e3
Author: Alexey Goncharuk 
Date:   2018-10-02T08:24:21Z

IGNITE-8803 Corrected output for quiet mode




---

[jira] [Created] (IGNITE-9760) NPE is possible during WAL flushing for FSYNC mode

[Anton Kalashnikov (JIRA)]

Anton Kalashnikov created IGNITE-9760:
-

 Summary: NPE is possible during WAL flushing for FSYNC mode
 Key: IGNITE-9760
 URL: https://issues.apache.org/jira/browse/IGNITE-9760
 Project: Ignite
  Issue Type: Bug
Reporter: Anton Kalashnikov
Assignee: Anton Kalashnikov


{noformat}
class org.apache.ignite.IgniteCheckedException: Failed to update keys (retry 
update if possible).: [9483]

at 
org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7409)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:261)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:172)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
at 
org.apache.ignite.testframework.GridTestUtils.lambda$runMultiThreadedAsync$96d302c5$1(GridTestUtils.java:853)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:349)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:337)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:497)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:476)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:464)
at 
org.apache.ignite.testframework.GridTestUtils.lambda$runAsync$2(GridTestUtils.java:1005)
at 
org.apache.ignite.testframework.GridTestUtils$7.call(GridTestUtils.java:1295)
at 
org.apache.ignite.testframework.GridTestThread.run(GridTestThread.java:86)
Caused by: org.apache.ignite.cache.CachePartialUpdateException: Failed to 
update keys (retry update if possible).: [9483]
at 
org.apache.ignite.internal.processors.cache.GridCacheUtils.convertToCacheException(GridCacheUtils.java:1307)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.cacheException(IgniteCacheProxyImpl.java:1742)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1092)
at 
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:820)
at 
org.apache.ignite.internal.processors.cache.persistence.db.wal.WalRolloverRecordLoggingTest.lambda$testAvoidInfinityWaitingOnRolloverOfSegment$0(WalRolloverRecordLoggingTest.java:119)
... 2 more
Caused by: class 
org.apache.ignite.internal.processors.cache.CachePartialUpdateCheckedException: 
Failed to update keys (retry update if possible).: [9483]
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.onPrimaryError(GridNearAtomicAbstractUpdateFuture.java:397)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.onPrimaryResponse(GridNearAtomicSingleUpdateFuture.java:253)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture$1.apply(GridNearAtomicAbstractUpdateFuture.java:303)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture$1.apply(GridNearAtomicAbstractUpdateFuture.java:300)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1855)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1668)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:299)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.map(GridNearAtomicSingleUpdateFuture.java:483)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicSingleUpdateFuture.mapOnTopology(GridNearAtomicSingleUpdateFuture.java:443)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:248)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update0(GridDhtAtomicCache.java:1153)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.put0(GridDhtAtomicCache.java:611)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2449)
at 

[jira] [Created] (IGNITE-9761) Deadlock SegmentArchivedStorage <-> SegmentLockStorage

[Anton Kalashnikov (JIRA)]

Anton Kalashnikov created IGNITE-9761:
-

 Summary: Deadlock SegmentArchivedStorage <-> SegmentLockStorage
 Key: IGNITE-9761
 URL: https://issues.apache.org/jira/browse/IGNITE-9761
 Project: Ignite
  Issue Type: Bug
Reporter: Anton Kalashnikov
Assignee: Anton Kalashnikov


{noformat}
Found one Java-level deadlock:
=
"wal-file-archiver%cache.IgniteClusterActivateDeactivateTestWithPersistence2-#11729%cache.IgniteClusterActivateDeactivateTestWithPersistence2%":
  waiting to lock monitor 0x7fa33c0121e8 (object 0xf7142560, a 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentLockStorage),
  which is held by 
"exchange-worker-#11646%cache.IgniteClusterActivateDeactivateTestWithPersistence2%"
"exchange-worker-#11646%cache.IgniteClusterActivateDeactivateTestWithPersistence2%":
  waiting to lock monitor 0x7fa3503b6058 (object 0xf7142578, a 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentArchivedStorage),
  which is held by 
"wal-file-archiver%cache.IgniteClusterActivateDeactivateTestWithPersistence2-#11729%cache.IgniteClusterActivateDeactivateTestWithPersistence2%"
Java stack information for the threads listed above:
===
"wal-file-archiver%cache.IgniteClusterActivateDeactivateTestWithPersistence2-#11729%cache.IgniteClusterActivateDeactivateTestWithPersistence2%":
at 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentLockStorage.locked(SegmentLockStorage.java:41)
- waiting to lock <0xf7142560> (a 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentLockStorage)
at 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentArchivedStorage.markAsMovedToArchive(SegmentArchivedStorage.java:101)
- locked <0xf7142578> (a 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentArchivedStorage)
at 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentAware.markAsMovedToArchive(SegmentAware.java:91)
at 
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$FileArchiver.body(FileWriteAheadLogManager.java:1643)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:748)
"exchange-worker-#11646%cache.IgniteClusterActivateDeactivateTestWithPersistence2%":
at 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentArchivedStorage.onSegmentUnlocked(SegmentArchivedStorage.java:135)
- waiting to lock <0xf7142578> (a 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentArchivedStorage)
at 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentArchivedStorage$$Lambda$2/2113450692.accept(Unknown
 Source)
at 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentObservable.lambda$notifyObservers$0(SegmentObservable.java:44)
at 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentObservable$$Lambda$6/688404745.accept(Unknown
 Source)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentObservable.notifyObservers(SegmentObservable.java:44)
- locked <0xf7142560> (a 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentLockStorage)
at 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentLockStorage.releaseWorkSegment(SegmentLockStorage.java:74)
- locked <0xf7142560> (a 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentLockStorage)
at 
org.apache.ignite.internal.processors.cache.persistence.wal.aware.SegmentAware.releaseWorkSegment(SegmentAware.java:226)
at 
org.apache.ignite.internal.processors.cache.persistence.wal.io.LockedReadFileInput.ensure(LockedReadFileInput.java:81)
at 
org.apache.ignite.internal.processors.cache.persistence.wal.serializer.RecordV1Serializer.readSegmentHeader(RecordV1Serializer.java:260)
at 
org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.initReadHandle(AbstractWalRecordsIterator.java:381)
at 
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.initReadHandle(FileWriteAheadLogManager.java:2942)
at 
org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$RecordsIterator.advanceSegment(FileWriteAheadLogManager.java:3024)
at 
org.apache.ignite.internal.processors.cache.persistence.wal.AbstractWalRecordsIterator.advance(AbstractWalRecordsIterator.java:163)
at 

[GitHub] ignite pull request #4883: Ignite 2.4.11

[Max-Pudov]

Github user Max-Pudov closed the pull request at:

https://github.com/apache/ignite/pull/4883


---

[GitHub] ignite pull request #4887: IGNITE-9390

[devozerov]

GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/4887

IGNITE-9390



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9390

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4887.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4887


commit 78c3fe86e3371e7ec5755529c6a2033cf986f6f2
Author: devozerov 
Date:   2018-10-02T07:59:57Z

WIP.

commit 5455c1d802ba2309b491cba63813b3565cd951f3
Author: devozerov 
Date:   2018-10-02T08:23:03Z

ALmost done.

commit d309a507ee915db86e796da8cf8a535fbac347ca
Author: devozerov 
Date:   2018-10-02T08:32:30Z

Done.

commit 91bc77ecaacb0fea50248ec5b238145972feac1a
Author: devozerov 
Date:   2018-10-02T08:50:07Z

Minors.




---

[jira] [Created] (IGNITE-9762) Network partition leads to failures in Ignite's cache

[Mo (JIRA)]

Mo created IGNITE-9762:
--

 Summary: Network partition leads to failures in Ignite's cache
 Key: IGNITE-9762
 URL: https://issues.apache.org/jira/browse/IGNITE-9762
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.4
Reporter: Mo


Creating a network partition in a replicated Ignite cluster leads to creating 
two independent clusters, each of which would operate independently from the 
other, even after the network partition is healed.

 

 

Setup: 3 servers (s1,s2,s3) two clients (c1,c2).

A partition created \{(s1,s2,c1),(s3,c2)}.

--> At this point two independent clusters form; one containing s1 and s2, 
while the other containing s3. The two never rejoin even after the partition is 
healed. 

 

This leads to a faulty cache in both sides of the partition:

 
 * *Stale reads*: An update to a cache in one side of the partition will not be 
propagated to the other side, hence, future reads to the other side's cache 
(using the updated key) will be stale reads.

 
 * *Data unavailability*: Inserting a new element to the cache on one side of 
the partition will not be added to the other side even after the partition is 
healed. This results in data unavailability for clients connected to the 
servers on the other side of the partition.

 

These are the settings used for the replicated cache:

 
cfg.setCacheMode(CacheMode.REPLICATED);
cfg.setAtomicityMode(CacheAtomicityMode.ATOMIC);
cfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
cfg.setReadFromBackup(false);
cfg.setPartitionLossPolicy(PartitionLossPolicy.READ_ONLY_SAFE);



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4889: IGNITE-7251 Remove term "fabric" from Ignite deli...

[vveider]

GitHub user vveider opened a pull request:

https://github.com/apache/ignite/pull/4889

IGNITE-7251 Remove term "fabric" from Ignite deliverables

 * removed additional "fabric" inclusion

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7251

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4889.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4889


commit 41674591cdc2f429a79c4ed030e81423afcdbc98
Author: Ivanov Petr 
Date:   2018-10-02T09:15:22Z

IGNITE-7251 Remove term "fabric" from Ignite deliverables
 * removed additional "fabric" inclusion




---

Re: Apache Ignite 2.7 release

[Anton Kalashnikov]

Hi Igniters.

I have one more possible blockers - deadlock in archiver - 
https://issues.apache.org/jira/browse/IGNITE-9761. I almost fixed it. 
It seems it should be include to scope.

-- 
Best regards,
Anton Kalashnikov


02.10.2018, 00:08, "Dmitriy Setrakyan" :
> Thanks, got it.
>
> On Mon, Oct 1, 2018 at 1:14 PM Dmitriy Pavlov  wrote:
>
>>  Here I agree with Vladimir. Furthermore, I do my absolute best to finalize
>>  all reviews in all 2.7 tickets I'm related to. I think most of the
>>  contributors doing the same.
>>
>>  пн, 1 окт. 2018 г. в 23:03, Vladimir Ozerov :
>>
>>  > This is precisely the scope we have at the moment. All these tickets were
>>  > considered carefully on whether to include them into AI 2.7 scope. I
>>  would
>>  > say that 10-15% of current tickets may be moved furhter.
>>  >
>>  > Third of current tickets are features on their final review stages (e.g.
>>  > TDE, MVCC invoke, TensorFlow, Thin Clients), another big part is
>>  > stabilization tickets (mainly - various test failures), and another big
>>  > part is infrastructure (adopting new modules, Java 9+ support, etc.). So
>>  > despite big absolute number, most of these tickets are grouped around
>>  > several big areas, and overall progress over this week should be very
>>  good.
>>  >
>>  > On Mon, Oct 1, 2018 at 9:50 PM Dmitriy Setrakyan 
>>  > wrote:
>>  >
>>  > > If this filter is for 2.7 release, then I do not believe all these
>>  > tickets
>>  > > will be closed. It would be nice to leave only "must-have" tickets in
>>  2.7
>>  > > and move the rest to 2.8.
>>  > >
>>  > > D.
>>  > >
>>  > > On Mon, Oct 1, 2018 at 11:02 AM Vladimir Ozerov 
>>  > > wrote:
>>  > >
>>  > > > Igniters,
>>  > > >
>>  > > > Please use this filter, as it properly handles tickets without
>>  > > components:
>>  > > >
>>  > > >
>>  > > >
>>  > >
>>  >
>>  
>> https://issues.apache.org/jira/issues/?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20(%272.7%27)%20AND%20status%20NOT%20IN%20(Resolved%2C%20Closed)%20and%20(component%20is%20null%20or%20component%20not%20in%20(documentation)))%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20%20
>>  > > >
>>  > > > On Mon, Oct 1, 2018 at 6:18 PM Nikolay Izhikov 
>>  > > > wrote:
>>  > > >
>>  > > > > Hello, Igniters.
>>  > > > >
>>  > > > > I announce scope freeze for an Apache Ignite 2.7 release.
>>  > > > >
>>  > > > > It means:
>>  > > > >
>>  > > > > 1. We add to 2.7 only critical bugs.
>>  > > > > 2. We merge to 2.7 branch only previously announces features.
>>  > > > > 3. I expect we should exclude or *MERGE ALL TASKS FOR 2.7 DUE TO
>>  > > OCTOBER
>>  > > > > 10*.
>>  > > > > So the *October 10 is DEADLINE* for new features.
>>  > > > >
>>  > > > > Thoughts?
>>  > > > >
>>  > > > > For now we have 34 In Progress tickets planned to 2.7 version [1].
>>  > > > > So is you assigned to any of this ticker friendly reminder #1, *the
>>  > > > > deadline is near :)*.
>>  > > > >
>>  > > > > [1]
>>  > > > >
>>  > > > >
>>  > > >
>>  > >
>>  >
>>  
>> https://issues.apache.org/jira/browse/IGNITE-8803?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20(%272.7%27)%20AND%20status%20NOT%20IN%20(Resolved%2C%20Closed)%20and%20component%20!%3D%20documentation%20)%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20%20
>>  > > > >
>>  > > > >
>>  > > > >
>>  > > > > В Пн, 01/10/2018 в 16:13 +0300, Andrey Gura пишет:
>>  > > > > > Agree with Andrey.
>>  > > > > >
>>  > > > > > IGNITE-9723 will be merged to ignite-2.7 branch soon.
>>  > > > > > On Mon, Oct 1, 2018 at 3:56 PM Andrey Kuznetsov <
>>  stku...@gmail.com
>>  > >
>>  > > > > wrote:
>>  > > > > > >
>>  > > > > > > Igniters,
>>  > > > > > >
>>  > > > > > > There is an inaccuracy in critical worker termination
>>  detection,
>>  > > and
>>  > > > > I'm
>>  > > > > > > working on a fix right now, see [1]. Also, we have trivial yet
>>  > > > > important
>>  > > > > > > fix [2], this one is ready to merge.
>>  > > > > > >
>>  > > > > > > I deem both should get to 2.7. Any objections?
>>  > > > > > >
>>  > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-9744
>>  > > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-9723
>>  > > > > > >
>>  > > > > > > пн, 1 окт. 2018 г. в 13:13, Dmitriy Pavlov <
>>  > dpavlov@gmail.com
>>  > > >:
>>  > > > > > >
>>  > > > > > > > Folks,
>>  > > > > > > >
>>  > > > > > > > We can consider this thread as an almost healthy discussion
>>  > about
>>  > > > SG
>>  > > > > > > > inclusion to 2.7.
>>  > > > > > > >
>>  > > > > > > > For the future, I encourage all Igniters to discuss things
>>  > before
>>  > > > > agreement
>>  > > > > > > > on dev.list instead of trying to log 'some pre-build
>>  > agreements'
>>  > > > > here. BTW
>>  > > > > > > > such logged 'agreements' are not always valid.
>>  > > > > > > >
>>  > > > > > > > Dev list is not a log of community actions; it is the
>>  

[jira] [Created] (IGNITE-9765) Network partition leads to failures in Ignite's queue

[Mo (JIRA)]

Mo created IGNITE-9765:
--

 Summary: Network partition leads to failures in Ignite's queue
 Key: IGNITE-9765
 URL: https://issues.apache.org/jira/browse/IGNITE-9765
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Mo


Creating a network partition in a replicated Ignite cluster leads to creating 
two independent clusters, each of which would operate independently from the 
other, even after the network partition is healed.

 

Setup: 3 servers (s1,s2,s3) two clients (c1,c2).

A partition created \{(s1,s2,c1),(s3,c2)}.

--> At this point two independent clusters form; one containing s1 and s2, 
while the other containing s3. The two never rejoin even after the partition is 
healed. 

 

Affected operations:  
 * *Queue add*: Inserting an element to *s3*'s queue ** will no be propagated 
to *s1* and *s2* even after the partition is healed. This leads to data 
unavailability.

 
 * *Queue remove:* Removing an element from the queue in *s3* will not be 
executed in the other servers. This leads to reappearance of deleted data.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9767) Network partition leads to failures in Ignite's semaphore

[Mo (JIRA)]

Mo created IGNITE-9767:
--

 Summary: Network partition leads to failures in Ignite's semaphore
 Key: IGNITE-9767
 URL: https://issues.apache.org/jira/browse/IGNITE-9767
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Mo


Creating a network partition in a replicated Ignite cluster leads to creating 
two independent clusters, each of which would operate independently from the 
other, even after the network partition is healed.

 

Setup: 3 servers (s1,s2,s3) two clients (c1,c2).

A partition created \{(s1,s2,c1),(s3,c2)}.

--> At this point two independent clusters form; one containing s1 and s2, 
while the other containing s3. The two never rejoin even after the partition is 
healed. 

 

This leads to a faulty semaphore on both sides of the partition. For example, 
if a semaphore with one permit is created in the cluster, after creating a 
network partition and healing it, both *c1* and *c2* can acquire that one 
permit.

 

System config: 

Release acquired permits if node, that owned them, left topology ==> Set to true



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9764) Node may hang on start if cluster state is in transition

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-9764:


 Summary: Node may hang on start if cluster state is in transition
 Key: IGNITE-9764
 URL: https://issues.apache.org/jira/browse/IGNITE-9764
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Goncharuk


The following sequence of events may cause node hang on start
Node starts, detects cluster state transition and waits for it to complete
{code}
"start-node-1" #11804 prio=5 os_prio=0 tid=0x7f9cc4022000 nid=0x1094 
waiting on condition [0x7f9ffc4c2000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1084)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2033)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1728)
- locked <0x9467c890> (a 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1156)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:654)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:917)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:855)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:843)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:809)
at 
org.apache.ignite.internal.processors.cache.IgniteClusterActivateDeactivateTest.lambda$testConcurrentJoinAndActivate$4(IgniteClusterActivateDeactivateTest.java:539)
at 
org.apache.ignite.internal.processors.cache.IgniteClusterActivateDeactivateTest$$Lambda$99/295822519.call(Unknown
 Source)
at 
org.apache.ignite.testframework.GridTestThread.run(GridTestThread.java:86)
{code}

Nio thread that is to process a message that would complete the exchange is 
attempting to create a session and get a local node ID
{code}
"grid-nio-worker-tcp-comm-3-#9833%cache.IgniteClusterActivateDeactivateTest3%" 
#11875 prio=5 os_prio=0 tid=0x7f9c8009e800 nid=0x10dc waiting on condition 
[0x7f9ff4d76000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  (a 
java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
at 
org.apache.ignite.internal.util.IgniteUtils.await(IgniteUtils.java:7577)
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.getSpiContext(TcpCommunicationSpi.java:2266)
at 
org.apache.ignite.spi.IgniteSpiAdapter.getLocalNode(IgniteSpiAdapter.java:156)
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.safeLocalNodeId(TcpCommunicationSpi.java:4006)
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.nodeIdMessage(TcpCommunicationSpi.java:3999)
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.access$300(TcpCommunicationSpi.java:271)
at 
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi$2.onConnected(TcpCommunicationSpi.java:412)
at 
org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onSessionOpened(GridNioFilterChain.java:251)
at 
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedSessionOpened(GridNioFilterAdapter.java:88)
at 
org.apache.ignite.internal.util.nio.GridNioCodecFilter.onSessionOpened(GridNioCodecFilter.java:67)
at 
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedSessionOpened(GridNioFilterAdapter.java:88)
at 
org.apache.ignite.internal.util.nio.GridConnectionBytesVerifyFilter.onSessionOpened(GridConnectionBytesVerifyFilter.java:58)
at 
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedSessionOpened(GridNioFilterAdapter.java:88)
at 
org.apache.ignite.internal.util.nio.GridNioServer$HeadFilter.onSessionOpened(GridNioServer.java:3499)
at 

Re: Apache Ignite 2.7 release

[Dmitriy Pavlov]

Hi Anton,

 I definitely agree it is a blocker.

Sincerely,
Dmitriy Pavlov

вт, 2 окт. 2018 г. в 13:09, Anton Kalashnikov :

> Hi Igniters.
>
> I have one more possible blockers - deadlock in archiver -
> https://issues.apache.org/jira/browse/IGNITE-9761. I almost fixed it.
> It seems it should be include to scope.
>
> --
> Best regards,
> Anton Kalashnikov
>
>
> 02.10.2018, 00:08, "Dmitriy Setrakyan" :
> > Thanks, got it.
> >
> > On Mon, Oct 1, 2018 at 1:14 PM Dmitriy Pavlov 
> wrote:
> >
> >>  Here I agree with Vladimir. Furthermore, I do my absolute best to
> finalize
> >>  all reviews in all 2.7 tickets I'm related to. I think most of the
> >>  contributors doing the same.
> >>
> >>  пн, 1 окт. 2018 г. в 23:03, Vladimir Ozerov :
> >>
> >>  > This is precisely the scope we have at the moment. All these tickets
> were
> >>  > considered carefully on whether to include them into AI 2.7 scope. I
> >>  would
> >>  > say that 10-15% of current tickets may be moved furhter.
> >>  >
> >>  > Third of current tickets are features on their final review stages
> (e.g.
> >>  > TDE, MVCC invoke, TensorFlow, Thin Clients), another big part is
> >>  > stabilization tickets (mainly - various test failures), and another
> big
> >>  > part is infrastructure (adopting new modules, Java 9+ support,
> etc.). So
> >>  > despite big absolute number, most of these tickets are grouped around
> >>  > several big areas, and overall progress over this week should be very
> >>  good.
> >>  >
> >>  > On Mon, Oct 1, 2018 at 9:50 PM Dmitriy Setrakyan <
> dsetrak...@apache.org>
> >>  > wrote:
> >>  >
> >>  > > If this filter is for 2.7 release, then I do not believe all these
> >>  > tickets
> >>  > > will be closed. It would be nice to leave only "must-have" tickets
> in
> >>  2.7
> >>  > > and move the rest to 2.8.
> >>  > >
> >>  > > D.
> >>  > >
> >>  > > On Mon, Oct 1, 2018 at 11:02 AM Vladimir Ozerov <
> voze...@gridgain.com>
> >>  > > wrote:
> >>  > >
> >>  > > > Igniters,
> >>  > > >
> >>  > > > Please use this filter, as it properly handles tickets without
> >>  > > components:
> >>  > > >
> >>  > > >
> >>  > > >
> >>  > >
> >>  >
> >>
> https://issues.apache.org/jira/issues/?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20(%272.7%27)%20AND%20status%20NOT%20IN%20(Resolved%2C%20Closed)%20and%20(component%20is%20null%20or%20component%20not%20in%20(documentation)))%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20%20
> >>  > > >
> >>  > > > On Mon, Oct 1, 2018 at 6:18 PM Nikolay Izhikov <
> nizhi...@apache.org>
> >>  > > > wrote:
> >>  > > >
> >>  > > > > Hello, Igniters.
> >>  > > > >
> >>  > > > > I announce scope freeze for an Apache Ignite 2.7 release.
> >>  > > > >
> >>  > > > > It means:
> >>  > > > >
> >>  > > > > 1. We add to 2.7 only critical bugs.
> >>  > > > > 2. We merge to 2.7 branch only previously announces features.
> >>  > > > > 3. I expect we should exclude or *MERGE ALL TASKS FOR 2.7 DUE
> TO
> >>  > > OCTOBER
> >>  > > > > 10*.
> >>  > > > > So the *October 10 is DEADLINE* for new features.
> >>  > > > >
> >>  > > > > Thoughts?
> >>  > > > >
> >>  > > > > For now we have 34 In Progress tickets planned to 2.7 version
> [1].
> >>  > > > > So is you assigned to any of this ticker friendly reminder #1,
> *the
> >>  > > > > deadline is near :)*.
> >>  > > > >
> >>  > > > > [1]
> >>  > > > >
> >>  > > > >
> >>  > > >
> >>  > >
> >>  >
> >>
> https://issues.apache.org/jira/browse/IGNITE-8803?jql=(project%20%3D%20%27Ignite%27%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20(%272.7%27)%20AND%20status%20NOT%20IN%20(Resolved%2C%20Closed)%20and%20component%20!%3D%20documentation%20)%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20%20
> >>  > > > >
> >>  > > > >
> >>  > > > >
> >>  > > > > В Пн, 01/10/2018 в 16:13 +0300, Andrey Gura пишет:
> >>  > > > > > Agree with Andrey.
> >>  > > > > >
> >>  > > > > > IGNITE-9723 will be merged to ignite-2.7 branch soon.
> >>  > > > > > On Mon, Oct 1, 2018 at 3:56 PM Andrey Kuznetsov <
> >>  stku...@gmail.com
> >>  > >
> >>  > > > > wrote:
> >>  > > > > > >
> >>  > > > > > > Igniters,
> >>  > > > > > >
> >>  > > > > > > There is an inaccuracy in critical worker termination
> >>  detection,
> >>  > > and
> >>  > > > > I'm
> >>  > > > > > > working on a fix right now, see [1]. Also, we have trivial
> yet
> >>  > > > > important
> >>  > > > > > > fix [2], this one is ready to merge.
> >>  > > > > > >
> >>  > > > > > > I deem both should get to 2.7. Any objections?
> >>  > > > > > >
> >>  > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-9744
> >>  > > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-9723
> >>  > > > > > >
> >>  > > > > > > пн, 1 окт. 2018 г. в 13:13, Dmitriy Pavlov <
> >>  > dpavlov@gmail.com
> >>  > > >:
> >>  > > > > > >
> >>  > > > > > > > Folks,
> >>  > > > > > > >
> >>  > > > > > > > We can consider this thread as an almost healthy
> discussion
> >>  > 

[jira] [Created] (IGNITE-9766) Network partition leads to failures in Ignite's set

[Mo (JIRA)]

Mo created IGNITE-9766:
--

 Summary: Network partition leads to failures in Ignite's set
 Key: IGNITE-9766
 URL: https://issues.apache.org/jira/browse/IGNITE-9766
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Mo


Creating a network partition in a replicated Ignite cluster leads to creating 
two independent clusters, each of which would operate independently from the 
other, even after the network partition is healed.

 

Setup: 3 servers (s1,s2,s3) two clients (c1,c2).

A partition created \{(s1,s2,c1),(s3,c2)}.

--> At this point two independent clusters form; one containing s1 and s2, 
while the other containing s3. The two never rejoin even after the partition is 
healed. 

 

This leads to a faulty set in both sides of the partition. For example, adding 
an element to the set in *s3* will not add that element *s1* and *s2*, even 
after the partition is healed. This leads to data unavailability.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Page IO statistics for Ignite

[Юрий]

Hi,

Thanks you for participate here.
I've prepared some draft of IEP-27
,
which aggregate information from the thread. Please take a look and give
your feedback.

Also there are some open questions:
1) We can simple gathering metrics on each nodes. But also we need cluster
wide metrics. How it can be achieved? I see that we already share some
metrics through TcpDiscoveryMetricsUpdateMessage
and TcpDiscoveryClientMetricsUpdateMessage. If we add all IO metrics here
it will grow up more then two times. Can it lead to performance degradation?
2) How it will be more convenient to reset statistics? My opinion it can be
done by time (for example once per hour) and by request + keep history of
the statistics. What do you think?

Please find below example of node statistics which can be simple gathered
on node. Such statistics can be obtained separately for physical read/write
and logical read:
1. Fine-grained:
{T_PAGE_LIST_NODE=1592, UNKNOWN=0, T_PART_META=865, T_H2_MVCC_REF_LEAF=0,
T_TX_LOG_INNER=0, T_DATA=592, T_DATA_METASTORAGE=0,
T_CACHE_ID_AWARE_PENDING_REF_INNER=0, T_CACHE_ID_DATA_REF_MVCC_LEAF=0,
T_PAGE_LIST_META=365, T_DATA_REF_INNER=0, T_H2_EX_REF_MVCC_INNER=0,
T_PENDING_REF_LEAF=408, T_DATA_REF_METASTORAGE_LEAF=0,
T_DATA_REF_MVCC_INNER=0, T_DATA_REF_METASTORAGE_INNER=0,
T_DATA_REF_MVCC_LEAF=0, T_CACHE_ID_AWARE_DATA_REF_LEAF=0,
T_CACHE_ID_AWARE_PENDING_REF_LEAF=0, T_TX_LOG_LEAF=0, T_DATA_REF_LEAF=1000,
T_H2_EX_REF_MVCC_LEAF=0, T_BPLUS_META=408, T_PAGE_UPDATE_TRACKING=0,
T_H2_REF_LEAF=0, T_METASTORE_INNER=0, T_META=0, T_H2_REF_INNER=0,
T_PART_CNTRS=0, T_H2_EX_REF_INNER=0, T_CACHE_ID_AWARE_DATA_REF_INNER=0,
T_PENDING_REF_INNER=0, T_CACHE_ID_DATA_REF_MVCC_INNER=0,
T_H2_EX_REF_LEAF=0, T_H2_MVCC_REF_INNER=0, T_METASTORE_LEAF=0}
2. Aggregated:
{DATA=592, INDEX=1000, OTHER=3638}


Maybe I missed something important, please share you opinion.


Thanks.


вт, 25 сент. 2018 г. в 22:20, Alex Plehanov :

> Hi,
>
> I've made some investigation a couple of months ago about a statistics
> collected by some RDBMS vendors (Oracle, Postgres, MySQL). These databases
> collect detailed IO statistics in dimensions such as queries, database
> objects (tables and indexes), files, sessions, users, event types etc.
>
> Some views where you can get IO statistics:
> Oracle: v$filestat, v$segment_statistics, v$sqlarea, v$sysstat, v$sesstat
> Postgres: pg_stat_database, pg_statio_all_tables, pg_statio_all_indexes,
> pg_statio_all_sequences
> MySQL: table_io_waits_summary_by_table,
> table_io_waits_summary_by_index_usage, io_global_by_file_by_bytes,
> io_global_by_wait_by_bytes, file_summary_by_event_name,
> file_summary_by_instance, host_summary_by_file_io, user_summary_by_file_io,
> metrics, etc.
>
> I think we can start by collecting statistics per FilePageStore (updating
> counters on read(...) and write(...) methods). Each FilePageStore is
> bounded to cache and partition or cache index, so we can easily aggregate
> values and get IO statistics per cache/index/node.
>
> вт, 25 сент. 2018 г. в 10:57, Vladimir Ozerov :
>
>> Hi Yuriy,
>>
>> I think this is great idea. But we need to collect more details on how and
>> what to collect. I think one of the most interesting parts for us would be
>> index and data page usages, split by different "dimensions":
>> 1) Global node statistics
>> 2) Per-cache statistics
>> 3) Per-index statistics
>>
>> We can start with a short summary of what is collected by other database
>> vendors.
>>
>> Vladimir.
>>
>>
>> On Sat, Sep 22, 2018 at 1:07 AM Denis Magda  wrote:
>>
>> > Hello Yuri,
>> >
>> > I might give useful feedback if see how the metrics will look like from
>> the
>> > API standpoint. If it's not difficult please create a draft.
>> >
>> > AS for the interface, in addition to JMX and SQL we need to ensure Visor
>> > CMD and Web Console gets updated. *Alex K.*, please join the thread and
>> > share your requirements.
>> >
>> > --
>> > Denis
>> >
>> > On Fri, Sep 21, 2018 at 8:16 AM Юрий 
>> wrote:
>> >
>> > > Hi Igniters,
>> > >
>> > > I started IGNITE-8580
>> > >  ticket
>> > > related to print page read/write metrics and did some investigation
>> what
>> > > other databases provide for the similar purposes.
>> > >
>> > > Based on the investigation I want to propose my raw vision of how to
>> > IGNITE
>> > > can be more transparent from performance perspective.
>> > >
>> > > Need to collect statistics for logical (from memory) and physical
>> (from
>> > > storage) page reads/writes. All these metrics should be separated by
>> next
>> > > dimensions:
>> > > 1) index/cache
>> > > 2) query level
>> > > 3) node/cluster
>> > > ...
>> > >
>> > > Seems the statistics should be limited by time.
>> > >
>> > > If we will have such statistics we could realize such things as:
>> > > 1) Get IO statistics per SQL query, global or/and splitted by
>> > > indexes/caches.
>> > 

[jira] [Created] (IGNITE-9768) Network partition leads to failures in Ignite's atomic data types.

[Mo (JIRA)]

Mo created IGNITE-9768:
--

 Summary: Network partition leads to failures in Ignite's atomic 
data types.
 Key: IGNITE-9768
 URL: https://issues.apache.org/jira/browse/IGNITE-9768
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Mo


Creating a network partition in a replicated Ignite cluster leads to creating 
two independent clusters, each of which would operate independently from the 
other, even after the network partition is healed.

 

Setup: 3 servers (s1,s2,s3) two clients (c1,c2).

A partition created \{(s1,s2,c1),(s3,c2)}.

--> At this point two independent clusters form; one containing s1 and s2, 
while the other containing s3. The two never rejoin even after the partition is 
healed. 

 

This leads to a faulty atomic types in Ignite. 

Affected data types:  
 * *Atomic Sequence*: An IncrementAndGet operation on *s3* will no affect the 
sequence in both *s1* and *s2* (even after the partition is healed).

 * *AtomicLong* and *AtomicRef*: Operations such as IncrementAndGet, 
CompareAndSet on *s3* will not be reflected to *s1* and *s2* even after the 
partition heals, which leads in faulty results for clients connected to these 
servers.

 * *CountDownLatch*: A CountDown Operation on the latch in *s3* will not be 
reflected to the other servers.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] SomeFire opened a new pull request #24: IGNITE-9697 Autocomplete branch for TC field

[GitBox]

SomeFire opened a new pull request #24: IGNITE-9697 Autocomplete branch for TC 
field
URL: https://github.com/apache/ignite-teamcity-bot/pull/24
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[jira] [Created] (IGNITE-9769) Replace sleep by fut.get() in the IgniteCacheAtomicProtocolTest.testPutReaderUpdate1 test

[Ryabov Dmitrii (JIRA)]

Ryabov Dmitrii created IGNITE-9769:
--

 Summary: Replace sleep by fut.get() in the 
IgniteCacheAtomicProtocolTest.testPutReaderUpdate1 test
 Key: IGNITE-9769
 URL: https://issues.apache.org/jira/browse/IGNITE-9769
 Project: Ignite
  Issue Type: Task
Reporter: Ryabov Dmitrii
Assignee: Ryabov Dmitrii


Replace sleep with fut.get(getTestTimeout()) to gather statistics. If test is 
flaky because 2 seconds isn't enough and test needs a bit more time - it's ok. 
If test really hangs - we should investigate the issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4887: IGNITE-9390

[devozerov]

Github user devozerov closed the pull request at:

https://github.com/apache/ignite/pull/4887


---