[jira] [Created] (IGNITE-9680) Web console: add component for status output

[Ilya Borisov (JIRA)]

Ilya Borisov created IGNITE-9680:


 Summary: Web console: add component for status output
 Key: IGNITE-9680
 URL: https://issues.apache.org/jira/browse/IGNITE-9680
 Project: Ignite
  Issue Type: Improvement
  Components: wizards
Reporter: Ilya Borisov
Assignee: Ilya Borisov


Create a component that output status:
1. It should accept a list of possible statuses as an argument, each item 
should specify label to display, model value, color (red, green or regular) and 
whether to show an inline spinner or not.
2. It should also accept status value as an argument.
3. Given that same the status might be displayed in various places, provide a 
way to omit repeating lengthy statuses list argument. Maybe a component factory 
that binds statuses to a new component?

Use the new component in:
1. Connected clusters dialog.

Remove the _ignite-status_ component.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Spring Session Ignite plugin

[akurbanov]

Hi Denis,

I've created pull request:  https://github.com/apache/ignite/pull/4820
   and filed an JIRA issue: 
https://issues.apache.org/jira/browse/IGNITE-9676
  . 

An initial review would be highly appreciated, thanks in advance. I will
clean up code, check dependencies and run TC.

Best regards,
Anton



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: Spring Session Ignite plugin

[Denis Magda]

Hello Anton,

I would merge the contribution to "ignite-spring-session" module. Create
it. We already have "ignite-spring", "ignite-spring-date" modules and the
addition of one more looks natural.

--
Denis

On Mon, Sep 24, 2018 at 3:31 AM Dmitriy Pavlov 
wrote:

> Hi Anton,
>
> There is GitHub mirror of Apache Ignite codebase here:
> https://github.com/apache/ignite
>
> You can create JIRA ticket and pull request from your Apache Ignite fork.
>
> Another option is to donate to some supplementary repository, but this way
> is much more complex and requires discussion from all community and grant
> agreement signing.
>
> So I suggest creating PR.
>
> Sincerely,
> Dmitriy Pavlov
>
> пн, 24 сент. 2018 г. в 13:03, akurbanov :
>
> > Hello Igniters,
> >
> > I have implemented small community extension to use Apache Ignite as
> > backing
> > repository for session clustering with Spring Session:  github link
> >   . I was wondering what
> > would be the best way to share this with community, is there any Ignite
> > repository existing for community integrations?
> >
> > Regards,
> > anton
> >
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >
>

Re: [TC Bot] Proposal of improvement

[Dmitriy Pavlov]

Hi Nikolay,

What about the case when committer creates ignite-9679 branch and tests it
without PR?

We have 1100+ open PRs and less than 100 open tickets. So scanning seems to
be possible only in JIRA. Mention probably will work for GitHub, but it
needs to be researched.

Two open PRs is not a valid situation in the majority of cases and How To
Contribute asks to avoid it. The bot can ignore closed PRs and the bot can
expect there is only one open PR per ticket.

Sincerely,
Dmitriy Pavlov

пн, 24 сент. 2018 г. в 23:41, Nikolay Izhikov :

> Hello, Dmitriy.
>
> > But it could be a lot of work to handle mentions (probably from the
> email> account and inbox).
>
> Actually, it can be done via GitHub REST API [1].
> It has 'since' param, so getting new GitHub comments is a very basic task.
>
> > Patch available ticket
>
> I think we shouldn't take a ticket as an entity that should be tested.
> For me, it's a PR.
>
> Moreover, it's a common case when we have several PR in a ticket.
> And it's a common case when both of them has to be tested.
>
> My vote goes to the closer integration with GitHub.
>
> [1]
> https://developer.github.com/v3/pulls/comments/#list-comments-in-a-repository
>
> В Пн, 24/09/2018 в 22:36 +0300, Dmitriy Pavlov пишет:
> > Hi Nikolay,
> >
> > The idea makes perfect sense for me, and we should definitely take the
> best
> > practices from other big Apache projects.
> >
> > But it could be a lot of work to handle mentions (probably from the email
> > account and inbox).
> >
> > I would like to suggest the following algorithm:
> >
> > Patch available ticket, which was never checked by the bot will be
> > processed in the following steps:
> > 1. check existing run all (by PR or by branch name), if found go to the
> > step 3
> > 2. run-all to be triggered by PR
> > 3. results should be analyzed for the presence of possible blockers. If
> > there is no blockers go to step 5.
> > 4. re-run of particular suites containing possible blockers should be
> > applied to try to get success for very rare flaky failures (<1%). Go to 3
> > (this go to should be done only once).
> > 5. comment should be added to JIRA ticket containing information about
> > results.
> >
> > If a ticket was processed by bot early (probably author added some fixes)
> > but still in PA state, the bot will check comments list and find possible
> > new mentions (made after the previous build complete date). If it finds
> > such comments it goes to step 1 (trying to find only new builds
> available).
> >
> > What do you think?
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пн, 24 сент. 2018 г. в 21:43, Nikolay Izhikov :
> >
> > > Hello, Igniters.
> > >
> > > I propose to implement following behaviour:
> > >
> > > 1. Execute "Run all" suite for specific PR when the author of PR makes
> a
> > > comment
> > > "@mtcga.bot Run Tests!" in GitHub comments.
> > >
> > > 2. Send a comment with "Run All" results both: to a Jira ticket and
> GitHub
> > > comment.
> > >
> > > 3. Label PR based on "Run All" results like it done in Apache Kafka [1]
> > >
> > > I've create ticket for this proposal [2]
> > >
> > > Thoughts?
> > >
> > > [1] https://github.com/apache/kafka/pulls
> > > [2] https://issues.apache.org/jira/browse/IGNITE-9678

Re: [TC Bot] Proposal of improvement

[Nikolay Izhikov]

Hello, Dmitriy.

> But it could be a lot of work to handle mentions (probably from the email> 
> account and inbox).

Actually, it can be done via GitHub REST API [1].
It has 'since' param, so getting new GitHub comments is a very basic task.

> Patch available ticket

I think we shouldn't take a ticket as an entity that should be tested.
For me, it's a PR.

Moreover, it's a common case when we have several PR in a ticket.
And it's a common case when both of them has to be tested.

My vote goes to the closer integration with GitHub.

[1] 
https://developer.github.com/v3/pulls/comments/#list-comments-in-a-repository

В Пн, 24/09/2018 в 22:36 +0300, Dmitriy Pavlov пишет:
> Hi Nikolay,
> 
> The idea makes perfect sense for me, and we should definitely take the best
> practices from other big Apache projects.
> 
> But it could be a lot of work to handle mentions (probably from the email
> account and inbox).
> 
> I would like to suggest the following algorithm:
> 
> Patch available ticket, which was never checked by the bot will be
> processed in the following steps:
> 1. check existing run all (by PR or by branch name), if found go to the
> step 3
> 2. run-all to be triggered by PR
> 3. results should be analyzed for the presence of possible blockers. If
> there is no blockers go to step 5.
> 4. re-run of particular suites containing possible blockers should be
> applied to try to get success for very rare flaky failures (<1%). Go to 3
> (this go to should be done only once).
> 5. comment should be added to JIRA ticket containing information about
> results.
> 
> If a ticket was processed by bot early (probably author added some fixes)
> but still in PA state, the bot will check comments list and find possible
> new mentions (made after the previous build complete date). If it finds
> such comments it goes to step 1 (trying to find only new builds available).
> 
> What do you think?
> 
> Sincerely,
> Dmitriy Pavlov
> 
> пн, 24 сент. 2018 г. в 21:43, Nikolay Izhikov :
> 
> > Hello, Igniters.
> > 
> > I propose to implement following behaviour:
> > 
> > 1. Execute "Run all" suite for specific PR when the author of PR makes a
> > comment
> > "@mtcga.bot Run Tests!" in GitHub comments.
> > 
> > 2. Send a comment with "Run All" results both: to a Jira ticket and GitHub
> > comment.
> > 
> > 3. Label PR based on "Run All" results like it done in Apache Kafka [1]
> > 
> > I've create ticket for this proposal [2]
> > 
> > Thoughts?
> > 
> > [1] https://github.com/apache/kafka/pulls
> > [2] https://issues.apache.org/jira/browse/IGNITE-9678

signature.asc
Description: This is a digitally signed message part

Re: [TC Bot] Proposal of improvement

[Dmitriy Pavlov]

Hi Nikolay,

The idea makes perfect sense for me, and we should definitely take the best
practices from other big Apache projects.

But it could be a lot of work to handle mentions (probably from the email
account and inbox).

I would like to suggest the following algorithm:

Patch available ticket, which was never checked by the bot will be
processed in the following steps:
1. check existing run all (by PR or by branch name), if found go to the
step 3
2. run-all to be triggered by PR
3. results should be analyzed for the presence of possible blockers. If
there is no blockers go to step 5.
4. re-run of particular suites containing possible blockers should be
applied to try to get success for very rare flaky failures (<1%). Go to 3
(this go to should be done only once).
5. comment should be added to JIRA ticket containing information about
results.

If a ticket was processed by bot early (probably author added some fixes)
but still in PA state, the bot will check comments list and find possible
new mentions (made after the previous build complete date). If it finds
such comments it goes to step 1 (trying to find only new builds available).

What do you think?

Sincerely,
Dmitriy Pavlov

пн, 24 сент. 2018 г. в 21:43, Nikolay Izhikov :

> Hello, Igniters.
>
> I propose to implement following behaviour:
>
> 1. Execute "Run all" suite for specific PR when the author of PR makes a
> comment
> "@mtcga.bot Run Tests!" in GitHub comments.
>
> 2. Send a comment with "Run All" results both: to a Jira ticket and GitHub
> comment.
>
> 3. Label PR based on "Run All" results like it done in Apache Kafka [1]
>
> I've create ticket for this proposal [2]
>
> Thoughts?
>
> [1] https://github.com/apache/kafka/pulls
> [2] https://issues.apache.org/jira/browse/IGNITE-9678

[GitHub] ignite pull request #4821: IGNITE-9655 DML benchmark suite (except selects)

[pavel-kuznetsov]

GitHub user pavel-kuznetsov opened a pull request:

https://github.com/apache/ignite/pull/4821

IGNITE-9655 DML benchmark suite (except selects)

Created benchmark suite for updates and insert+deletes.
Removed outdated benchmarks.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9655

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4821.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4821


commit 858426ca5b65262fde5e73d524caf9fa187637c8
Author: Pavel Kuznetsov 
Date:   2018-09-21T14:32:51Z

ignite-9655: created configuration with my benchmarks.

commit dce4e0ef3e5b6d0b95d27f246da1a13c6d0ce969
Author: Pavel Kuznetsov 
Date:   2018-09-23T23:20:56Z

ignite-9655: updated benchmarks.

Left questions in the end in comments. Use only native sql.

commit 43f7495b248478b92a725a803021b3ca85c864bd
Author: Pavel Kuznetsov 
Date:   2018-09-24T11:47:30Z

ignite-9655: fixed extra ','

commit 0cda529d54cec967daee05328cde85103ee57416
Author: Pavel Kuznetsov 
Date:   2018-09-24T14:52:29Z

ignite-9655: added other drivers.

commit c93abcfb700f82d71e2920e16a5ce6aee86dba97
Author: Pavel Kuznetsov 
Date:   2018-09-24T15:44:07Z

ignite-9655: removed duplicated -cl flag.

commit a05f486a30be05c0c9080407a85df2ce763164ba
Author: Pavel Kuznetsov 
Date:   2018-09-24T19:28:05Z

ignite-9655: removed incorrect benchmarks (dml).

Removed benchmarks from package org.apache.ignite.yardstick.cache.dml that 
don't have steady state.

commit 7304a2ecc86d2fd02cca885de1252576edb1e659
Author: Pavel Kuznetsov 
Date:   2018-09-24T18:40:49Z

ignite-9655: removed incorrect benchmarks (jdbc.store).

Removed benchmarks from package 
org.apache.ignite.yardstick.cache.store.jdbc since they are not needed.




---

Re: Critical worker threads liveness checking drawbacks

[Andrey Kuznetsov]

Denis,

I've created the ticket [1] with short description of the functionality.

[1] https://issues.apache.org/jira/browse/IGNITE-9679


пн, 24 сент. 2018 г. в 17:46, Denis Magda :

> Andrey K. and G.,
>
> Thanks, do we have a documentation ticket created? Prachi (copied) can help
> with the documentation.
>
> --
> Denis
>
> On Mon, Sep 24, 2018 at 5:51 AM Andrey Gura  wrote:
>
> > Andrey,
> >
> > finally your change is merged to master branch. Congratulations and
> > thank you very much! :)
> >
> > I think that the next step is feature that will allow signal about
> > blocked threads to the monitoring tools via MXBean.
> >
> > I hope you will continue development of this feature and provide your
> > vision in new JIRA issue.
> >
> >
> > On Tue, Sep 11, 2018 at 6:54 PM Andrey Kuznetsov 
> > wrote:
> > >
> > > David, Maxim!
> > >
> > > Thanks a lot for you ideas. Unfortunately, I can't adopt all of them
> > right
> > > now: the scope is much broader than the scope of the change I
> implement.
> > I
> > > have had a talk to a group of Ignite commiters, and we agreed to
> complete
> > > the change as follows.
> > > - Blocking instructions in system-critical which may resonably last
> long
> > > should be explicitly excluded from the monitoring.
> > > - Failure handlers should have a setting to suppress some failures on
> > > per-failure-type basis.
> > > According to this I have updated the implementation: [1]
> > >
> > > [1] https://github.com/apache/ignite/pull/4089
> > >
> > > пн, 10 сент. 2018 г. в 22:35, David Harvey :
> > >
> > > > When I've done this before,I've needed to find the oldest  thread,
> and
> > kill
> > > > the node running that.   From a language standpoint, Maxim's "without
> > > > progress" better than "heartbeat".   For example, what I'm most
> > interested
> > > > in on a distributed system is which thread started the work it has
> not
> > > > completed the earliest, and when did that thread last make forward
> > > > process. You don't want to kill a node because a thread is
> waiting
> > on a
> > > > lock held by a thread that went off-node and has not gotten a
> response.
> > > > If you don't understand the dependency relationships, you will make
> > > > incorrect recovery decisions.
> > > >
> > > > On Mon, Sep 10, 2018 at 4:08 AM Maxim Muzafarov 
> > > > wrote:
> > > >
> > > > > I think we should find exact answers to these questions:
> > > > >  1. What `critical` issue exactly is?
> > > > >  2. How can we find critical issues?
> > > > >  3. How can we handle critical issues?
> > > > >
> > > > > First,
> > > > >  - Ignore uninterruptable actions (e.g. worker\service shutdown)
> > > > >  - Long I/O operations (should be a configurable timeout for each
> > type of
> > > > > usage)
> > > > >  - Infinite loops
> > > > >  - Stalled\deadlocked threads (and\or too many parked threads,
> > exclude
> > > > I/O)
> > > > >
> > > > > Second,
> > > > >  - The working queue is without progress (e.g. disco, exchange
> > queues)
> > > > >  - Work hasn't been completed since the last heartbeat (checking
> > > > > milestones)
> > > > >  - Too many system resources used by a thread for the long period
> of
> > time
> > > > > (allocated memory, CPU)
> > > > >  - Timing fields associated with each thread status exceeded a
> > maximum
> > > > time
> > > > > limit.
> > > > >
> > > > > Third (not too many options here),
> > > > >  - `log everything` should be the default behaviour in all these
> > cases,
> > > > > since it may be difficult to find the cause after the restart.
> > > > >  - Wait some interval of time and kill the hanging node (cluster
> > should
> > > > be
> > > > > configured stable enough)
> > > > >
> > > > > Questions,
> > > > >  - Not sure, but can workers miss their heartbeat deadlines if CPU
> > loads
> > > > up
> > > > > to 80%-90%? Bursts of momentary overloads can be
> > > > > expected behaviour as a normal part of system operations.
> > > > >  - Why do we decide that critical thread should monitor each other?
> > For
> > > > > instance, if all the tasks were blocked and unable to run,
> > > > > node reset would never occur. As for me, a better solution is
> to
> > use
> > > > a
> > > > > separate monitor thread or pool (maybe both with software
> > > > > and hardware checks) that not only checks heartbeats but
> > monitors the
> > > > > other system as well.
> > > > >
> > > > > On Mon, 10 Sep 2018 at 00:07 David Harvey 
> > wrote:
> > > > >
> > > > > > It would be safer to restart the entire cluster than to remove
> the
> > last
> > > > > > node for a cache that should be redundant.
> > > > > >
> > > > > > On Sun, Sep 9, 2018, 4:00 PM Andrey Gura 
> wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I agree with Yakov that we can provide some option that manage
> > worker
> > > > > > > liveness checker behavior in case of observing that some worker
> > is
> > > > > > > blocked too long.
> > > > > > > At least it will  some workaround for cases when 

[jira] [Created] (IGNITE-9679) Document critical workers liveness checking implementation

[Andrey Kuznetsov (JIRA)]

Andrey Kuznetsov created IGNITE-9679:


 Summary: Document critical workers liveness checking implementation
 Key: IGNITE-9679
 URL: https://issues.apache.org/jira/browse/IGNITE-9679
 Project: Ignite
  Issue Type: Task
  Components: documentation
Reporter: Andrey Kuznetsov
Assignee: Denis Magda
 Fix For: 2.7


Newly implemented critical worker thread liveness checks should be mentioned in 
Ignite Documentation. Brief description of the functionality follows.

Ignite node has a number of critical worker threads that should be alive and 
responsive, otherwise node's health is not guaranteed. These threads monitor 
each other periodically and track two aspects for a thread being checked:
- whether it's alive;
- whether it updates its internal heartbeat timestamp.
Both checks use {{IgniteConfiguration.failureDetectionTimeout}} property as a 
threshold value.
Whenever at least one of the above conditions is violated, checker thread logs 
the error and calls currently configured {{FailureHandler}}.

Liveness checks are enabled by default, but can be disabled through 
{{WorkersControlMXBean.healthMonitoringEnabled}} property.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[TC Bot] Proposal of improvement

[Nikolay Izhikov]

Hello, Igniters.

I propose to implement following behaviour:

1. Execute "Run all" suite for specific PR when the author of PR makes a comment
"@mtcga.bot Run Tests!" in GitHub comments.

2. Send a comment with "Run All" results both: to a Jira ticket and GitHub 
comment.

3. Label PR based on "Run All" results like it done in Apache Kafka [1]

I've create ticket for this proposal [2]

Thoughts?

[1] https://github.com/apache/kafka/pulls
[2] https://issues.apache.org/jira/browse/IGNITE-9678

signature.asc
Description: This is a digitally signed message part

[jira] [Created] (IGNITE-9678) [TC Bot] Trigger Run All for github comment

[Nikolay Izhikov (JIRA)]

Nikolay Izhikov created IGNITE-9678:
---

 Summary: [TC Bot] Trigger Run All for github comment
 Key: IGNITE-9678
 URL: https://issues.apache.org/jira/browse/IGNITE-9678
 Project: Ignite
  Issue Type: Bug
Reporter: Nikolay Izhikov


Team City bot should work in the following way:
It should be able to:

1. Execute "Run all" suite for specific PR when the author of PR makes a 
comment "@mtcga.bot Run Tests!" in GitHub comments.
2. Send a comment with "Run All" results both: to a Jira ticket and GitHub 
comment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4820: IGNITE-9676 Spring Session integration implemente...

[antkr]

GitHub user antkr opened a pull request:

https://github.com/apache/ignite/pull/4820

IGNITE-9676 Spring Session integration implemented.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/antkr/ignite IGNITE-9676

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4820.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4820


commit a57e6520cbe14c24c9b1feee561183fcbe0d7e6d
Author: antkr 
Date:   2018-09-24T17:59:43Z

IGNITE-9676 Spring Session integration implemented.




---

[jira] [Created] (IGNITE-9677) [TC Bot] Automatic build trigger should track all branches defined in configuration.

[Pavel Pereslegin (JIRA)]

Pavel Pereslegin created IGNITE-9677:


 Summary: [TC Bot] Automatic build trigger should track all 
branches defined in configuration.
 Key: IGNITE-9677
 URL: https://issues.apache.org/jira/browse/IGNITE-9677
 Project: Ignite
  Issue Type: Bug
Reporter: Pavel Pereslegin


Automatic build trigger tracks only chains defined in branch with identifier 
"master", this should be changed to track all configured branches.
Also when bot starts build it checks that there is no self-started builds in 
queue already, this check should be improved to be able start another branch 
build simultaneously (skip triggering only if build's {{branchName}} is equal 
to {{branchForRest}}).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4819: IGNITE-9674 Optimize network usage on P2P class l...

[antonovsergey93]

GitHub user antonovsergey93 opened a pull request:

https://github.com/apache/ignite/pull/4819

IGNITE-9674 Optimize network usage on P2P class loading.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9674

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4819.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4819


commit 8c0ea8571e4f736b1d6d68af65b03f22cc8ef368
Author: Sergey Antonov 
Date:   2018-09-24T16:57:50Z

IGNITE-9674 WIP

commit 0c2f13bcace8a81a777800d6e27b29f18182277d
Author: Sergey Antonov 
Date:   2018-09-24T17:15:30Z

IGNITE-9674 Test was added.




---

Re: affinityBackupFilter for AWS Availability Zones

[David Harvey]

Yes, thanks Val!

On Mon, Sep 24, 2018 at 11:35 AM Dmitriy Pavlov 
wrote:

> Hi Val, many thanks for the review.
>
> ср, 12 сент. 2018 г. в 20:35, Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
>
> > Yes, will try to review this week.
> >
> > -Val
> >
> > On Wed, Sep 12, 2018 at 10:24 AM Dmitriy Pavlov 
> > wrote:
> >
> > > Hi Val,
> > >
> > > I'm not an expert in AWS, so could you please pick up the review?
> > >
> > > Thank you in advance!
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > вт, 11 сент. 2018 г. в 1:28, Dave Harvey :
> > >
> > > > Submitted a patch for this
> > > > https://issues.apache.org/jira/browse/IGNITE-9365
> > > >
> > > >
> > > >
> > > > --
> > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > > >
> > >
> >
>

Re: Sign Keys for release artifacts

[Anton Vinogradov]

Nikolay,

Please check all vote_* steps to make sure that's the only problem

пн, 24 сент. 2018 г. в 18:53, Petr Ivanov :

> How then you’ll be able to push artifacts to SVN repository for vote?
>
>
>
> > On 24 Sep 2018, at 18:28, Nikolay Izhikov  wrote:
> >
> > Hello, Igniters.
> >
> > Cos, I have a question regarding Apache Ignite release.
> >
> > I have committer priveledges in Apache Ignite.
> > I don't have PMC role.
> >
> > Since I don't have PMC role, I can't add my GPG key to KEYS [1] file.
> >
> > Is it OK if someone with PMC role adds my GPG key to KEYS file?
> >
> > [1] https://dist.apache.org/repos/dist/release/ignite/KEYS
>
>

Re: Ignite-2.7 branch created

[Dmitriy Pavlov]

Thank you, Petr!

Done with the Bot. I will not enable notifications to dev. list.

A user can come to its own preferences page and enable "Notify on all
failures in ignite-2.7"

пн, 24 сент. 2018 г. в 19:18, Petr Ivanov :

> Done with trigger.
>
>
>
> > On 24 Sep 2018, at 19:15, Dmitriy Pavlov  wrote:
> >
> > Hi Nikolay, Petr
> >
> > could you please create triggers for nightly builds?
> >
> > I will also add the branch to be tracked to TC Bot.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пн, 24 сент. 2018 г. в 18:24, Nikolay Izhikov :
> >
> >> Hello, Igniters.
> >>
> >> I've created branch for an Apache Ignite 2.7 release [1]
> >> Please, cherry-pick tickets that should be in 2.7 to that branch.
> >>
> >> [1]
> >>
> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=shortlog;h=refs/heads/ignite-2.7
>
>

Re: Ignite-2.7 branch created

[Petr Ivanov]

Done with trigger. 



> On 24 Sep 2018, at 19:15, Dmitriy Pavlov  wrote:
> 
> Hi Nikolay, Petr
> 
> could you please create triggers for nightly builds?
> 
> I will also add the branch to be tracked to TC Bot.
> 
> Sincerely,
> Dmitriy Pavlov
> 
> пн, 24 сент. 2018 г. в 18:24, Nikolay Izhikov :
> 
>> Hello, Igniters.
>> 
>> I've created branch for an Apache Ignite 2.7 release [1]
>> Please, cherry-pick tickets that should be in 2.7 to that branch.
>> 
>> [1]
>> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=shortlog;h=refs/heads/ignite-2.7

Re: Ignite-2.7 branch created

[Dmitriy Pavlov]

Hi Nikolay, Petr

could you please create triggers for nightly builds?

I will also add the branch to be tracked to TC Bot.

Sincerely,
Dmitriy Pavlov

пн, 24 сент. 2018 г. в 18:24, Nikolay Izhikov :

> Hello, Igniters.
>
> I've created branch for an Apache Ignite 2.7 release [1]
> Please, cherry-pick tickets that should be in 2.7 to that branch.
>
> [1]
> https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=shortlog;h=refs/heads/ignite-2.7

[GitHub] ignite pull request #4818: IGNITE-9451: Cache.size for key-value API

[pavlukhin]

GitHub user pavlukhin opened a pull request:

https://github.com/apache/ignite/pull/4818

IGNITE-9451: Cache.size for key-value API



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9451

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4818.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4818


commit b7d84760eb8dde4b1a922b4ee54885e86644e858
Author: Andrey V. Mashenkov 
Date:   2018-09-06T07:45:56Z

IGNITE-7764: WIP. Tests added.

Signed-off-by: Andrey V. Mashenkov 

commit 98ba5dc4e99e571056f4c0d7061c87f68c0d0aff
Author: Andrey V. Mashenkov 
Date:   2018-09-06T14:28:53Z

IGNITE-7764: WIP. Fix Get\GetAll operations.

Signed-off-by: Andrey V. Mashenkov 

commit b88c8b8aa37a493adb64e572e82e7e15c8be88e7
Author: Andrey V. Mashenkov 
Date:   2018-09-07T19:26:49Z

IGNITE-7764: WIP. Fix Put\PutAll operations. Naive implementation, 
optimization needed.

Signed-off-by: Andrey V. Mashenkov 

commit 09c0a2e8d321da968900af247f41b281cf505599
Author: Andrey V. Mashenkov 
Date:   2018-09-08T15:01:38Z

IGNITE-7764: WIP. Fix test.

Signed-off-by: Andrey V. Mashenkov 

commit 7971694048ab344353808072cbd8826ac958a48d
Author: Andrey V. Mashenkov 
Date:   2018-09-10T10:06:06Z

IGNITE-7764: WIP. Refactored query enlist futures.

Generify query enlist futures.

Signed-off-by: Andrey V. Mashenkov 

commit a06e80d0a3040e467163359016d5fb2cab8eb8a4
Author: Andrey V. Mashenkov 
Date:   2018-09-12T14:12:45Z

IGNITE-7764: WIP. Implemented MVCC protocol for basic 
put\putAll\remove\removeAll operations.

Signed-off-by: Andrey V. Mashenkov 

commit 043d42cda73c8d11ec270c8c5739f2d34d58d9fe
Author: Andrey V. Mashenkov 
Date:   2018-09-12T16:33:10Z

IGNITE-7764: WIP. Added tests to suite.

Signed-off-by: Andrey V. Mashenkov 

commit 3dffd70991cd782197749575770187705f780c52
Author: Andrey V. Mashenkov 
Date:   2018-09-13T10:06:06Z

IGNITE-7764: WIP. Added tests for remove\removeAll.

Signed-off-by: Andrey V. Mashenkov 

commit bfeed7184bece6b05fac3f2dbcee9e221f50
Author: Andrey V. Mashenkov 
Date:   2018-09-13T12:20:14Z

IGNITE-7764: WIP. Added tests for getAndPut\getAndRemove\putIfAbsent.

Added muted test replace/getAndReplace.

Signed-off-by: Andrey V. Mashenkov 

commit 7133ee487ce23a18b60f5fd9980db9a1bf204f25
Author: Andrey V. Mashenkov 
Date:   2018-09-13T14:07:10Z

IGNITE-7764: WIP. Fix tests.

Signed-off-by: Andrey V. Mashenkov 

commit 04925e8795a0db146aa9caf53e3aafadae4dea8c
Author: Andrey V. Mashenkov 
Date:   2018-09-13T15:14:28Z

IGNITE-7764: WIP. Mute tests.

Signed-off-by: Andrey V. Mashenkov 

commit f06157575650f5948f5c94a696f546b087d6a455
Author: Andrey V. Mashenkov 
Date:   2018-09-13T17:52:54Z

IGNITE-7764: WIP. Fixed putIfAbsent/replace/getAndReplace.

Signed-off-by: Andrey V. Mashenkov 

commit 8f4bafadeb22e3a420090a674e1c803d3add16c9
Author: Andrey V. Mashenkov 
Date:   2018-09-14T15:19:39Z

IGNITE-7764: WIP. Fixed putIfAbsent/replace/getAndReplace.

Signed-off-by: Andrey V. Mashenkov 

commit 8de70951b5864484fb552087eecf08fa45ace39b
Author: Andrey V. Mashenkov 
Date:   2018-09-14T15:36:00Z

IGNITE-7764: WIP. Minor.

Signed-off-by: Andrey V. Mashenkov 

commit bb8f375c9aed9d61a56b75513108e256b9c31a84
Author: Andrey V. Mashenkov 
Date:   2018-09-14T16:00:10Z

IGNITE-7764: WIP. Enable Full API MVCC tests.

Signed-off-by: Andrey V. Mashenkov 

commit a96a6e618bba269ef8c83185a352e5f6df2a1464
Author: Andrey V. Mashenkov 
Date:   2018-09-14T16:18:50Z

IGNITE-7764: WIP. Minors & Styles.

Signed-off-by: Andrey V. Mashenkov 

commit 7e5ca350360b56696f4238c97b5b8b7ee41d976f
Author: Andrey V. Mashenkov 
Date:   2018-09-14T17:41:03Z

IGNITE-7764: WIP. Minors.

Signed-off-by: Andrey V. Mashenkov 

commit f65324b1c69458128b5f5a2a0917bc1fe49e2cc1
Author: Andrey V. Mashenkov 
Date:   2018-09-14T17:53:12Z

IGNITE-7764: WIP. Minors after rebase.

Signed-off-by: Andrey V. Mashenkov 

commit 83116012412a4f24de8df2b64d953385060e8724
Author: Andrey V. Mashenkov 
Date:   2018-09-17T09:25:37Z

IGNITE-7764: WIP. Minors.

Signed-off-by: Andrey V. Mashenkov 

commit 0bd02e8939010891854b856c8d77f7f49f13d651
Author: Andrey V. Mashenkov 
Date:   2018-09-17T10:10:17Z

IGNITE-7764: WIP. Minors.

Signed-off-by: Andrey V. Mashenkov 

commit aafbe9c1a9d33086ff78c062f1a06381126e123d
Author: Andrey V. Mashenkov 
Date:   2018-09-17T10:49:22Z

IGNITE-7764: WIP. Fix non-versioned read.

Signed-off-by: Andrey V. Mashenkov 

commit 7cb3c54dfac32ba4eea5cb3e03e3daa5b4b7d9bf
Author: Andrey V. Mashenkov 
Date:   

Re: Sign Keys for release artifacts

[Petr Ivanov]

How then you’ll be able to push artifacts to SVN repository for vote?



> On 24 Sep 2018, at 18:28, Nikolay Izhikov  wrote:
> 
> Hello, Igniters.
> 
> Cos, I have a question regarding Apache Ignite release.
> 
> I have committer priveledges in Apache Ignite.
> I don't have PMC role. 
> 
> Since I don't have PMC role, I can't add my GPG key to KEYS [1] file.
> 
> Is it OK if someone with PMC role adds my GPG key to KEYS file?
> 
> [1] https://dist.apache.org/repos/dist/release/ignite/KEYS

Re: Wrong off-heap size is reported for a node

[Pavel Pereslegin]

Andrei,
I totally agree with you and I think that "ClusterMetrics" should also
be fixed, I'm just not sure that we should include this change in the
same ticket.
пн, 24 сент. 2018 г. в 18:43, aealexsandrov :
>
> Hi,
>
> OK, the user can use it to calculate the off-heap. But I think that the
> reason for your changes to fix the calculation of the nonHeap used in Ignite
> now. For example now REST return "-1" for nonHeapMemoryMaximum. I think that
> it can't be used somehow. So REST possible should be updated as you did for
> log metrics and it will require for the same logic.
>
> BR,
> Andrei
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: Wrong off-heap size is reported for a node

[aealexsandrov]

Hi,

OK, the user can use it to calculate the off-heap. But I think that the
reason for your changes to fix the calculation of the nonHeap used in Ignite
now. For example now REST return "-1" for nonHeapMemoryMaximum. I think that
it can't be used somehow. So REST possible should be updated as you did for
log metrics and it will require for the same logic.

BR,
Andrei



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [MTCGA]: new failures in builds [1888723] needs to be handled

[Dmitriy Pavlov]

Hi Igniters, I've found Parity check test was muted.

The test itself contains the possibility to exclude property. No reason to
mute this test.

Could you please ignore property in code?

Sincerely,
Dmitriy Pavlov

пн, 17 сент. 2018 г. в 17:25, :

> Hi Ignite Developer,
>
> I am MTCGA.Bot, and I've detected some issue on TeamCity to be addressed.
> I hope you can help.
>
>  *New test failure in master
> IgniteConfigurationParityTest.TestIgniteConfiguration
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=7406999233661835096=%3Cdefault%3E=testDetails
>
>  *New test failure in master DataRegionMetricsTest.TestMemoryMetrics
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=6742613397597284603=%3Cdefault%3E=testDetails
>
>  *New test failure in master MemoryMetricsTest.TestMemoryMetrics
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-7558087625238261420=%3Cdefault%3E=testDetails
>  Changes may led to failure were done by
>  - kondakov87
> http://ci.ignite.apache.org/viewModification.html?modId=831703=false
>
> - If your changes can led to this failure(s), please create issue
> with label MakeTeamCityGreenAgain and assign it to you.
> -- If you have fix, please set ticket to PA state and write to dev
> list fix is ready
> -- For case fix will require some time please mute test and set
> label Muted_Test to issue
> - If you know which change caused failure please contact change
> author directly
> - If you don't know which change caused failure please send
> message to dev list to find out
> Should you have any questions please contact dev@ignite.apache.org
> Best Regards,
> MTCGA.Bot
> Notification generated at Mon Sep 17 17:25:16 MSK 2018
>

Re: affinityBackupFilter for AWS Availability Zones

[Dmitriy Pavlov]

Hi Val, many thanks for the review.

ср, 12 сент. 2018 г. в 20:35, Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Yes, will try to review this week.
>
> -Val
>
> On Wed, Sep 12, 2018 at 10:24 AM Dmitriy Pavlov 
> wrote:
>
> > Hi Val,
> >
> > I'm not an expert in AWS, so could you please pick up the review?
> >
> > Thank you in advance!
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > вт, 11 сент. 2018 г. в 1:28, Dave Harvey :
> >
> > > Submitted a patch for this
> > > https://issues.apache.org/jira/browse/IGNITE-9365
> > >
> > >
> > >
> > > --
> > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > >
> >
>

[GitHub] ignite pull request #4014: IGNITE-6500

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4014


---

Sign Keys for release artifacts

[Nikolay Izhikov]

Hello, Igniters.

Cos, I have a question regarding Apache Ignite release.

I have committer priveledges in Apache Ignite.
I don't have PMC role. 

Since I don't have PMC role, I can't add my GPG key to KEYS [1] file.

Is it OK if someone with PMC role adds my GPG key to KEYS file?

[1] https://dist.apache.org/repos/dist/release/ignite/KEYS

signature.asc
Description: This is a digitally signed message part

Ignite-2.7 branch created

[Nikolay Izhikov]

Hello, Igniters.

I've created branch for an Apache Ignite 2.7 release [1]
Please, cherry-pick tickets that should be in 2.7 to that branch.

[1] 
https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=shortlog;h=refs/heads/ignite-2.7

signature.asc
Description: This is a digitally signed message part

Apache Ignite community update

[Tom Diederich]

Hello, Igniters! It’s been a busy September for us here in the Apache Ignite 
community. We’ve been preparing for the In-Memory Computing Summit North 
America 2018 happening Oct. 2-3 in San Francisco. I’ll have details on some 
Ignite sessions below.

Meantime, our conference preparations haven’t slowed us down! Our intrepid 
Apache Ignite technology evangelist Akmal Chaudhri has been on the road as part 
of a European Big Data Meetup roadshow – his stops to date: Copenhagen, 
Hamburg, Berlin, Frankfurt, Vienna, Munich,  Zurich, Paris, Amsterdam and 
London. And this week, he'll be speaking in Oslo.

Let’s take a look at some upcoming events for this week (and details on in 
In-Memory Computing Summit North America afterwards. If you are in the San 
Francisco Bay Area and would like to attend the conference, I have 3 free 
passes available if you use promo code “TomVIP” during checkout).

Meetups
Bay Area In-Memory Computing Meetup
Wednesday, September 26, 2018 (in Menlo Park, CA)
Valentin (Val) Kulichenko will present "Best Practices for Stream Ingestion, 
Processing and Analytics Using In-Memory Computing." He’ll share best practices 
used for real-time stream ingestion, processing and analytics using Apache® 
Ignite™, Apache Kafka™, Apache Spark™ and other technologies.

Big Data, Oslo V3.0 Meetup
Thursday, September 27, 2018
Apache Ignite technology evangelist Akmal Chaudhri will be a featured speaker 
at the Big Data, Oslo V3.0 Meetup on Sept. 27 from 6-9 p.m. His talk is titled, 
"How to become a Big Data Rockstar in 15 minutes!" Abstract: The secret? Apache 
Ignite! Apache Ignite is a memory-centric distributed database, caching, and 
processing platform. It is designed for transactional, analytical, and 
streaming workloads, delivering in-memory performance at scale.

Conferences
db tech showcase Tokyo 2018
Friday, September 21, 2018
Roman Shtykh, a Big Data Engineer at Yahoo! JAPAN, will be a featured speaker 
at the db tech showcase Tokyo 2018 conference, Sept. 19-21 in Tokyo. His talk 
is titled, "Apache Ignite: From In-Memory Data Grid to Memory-Centric 
Distributed Database," and is scheduled for Sept. 21 (Session D-33 #dbts2017).

Webinars
Apache Ignite: powering up banks and financial institutions with distributed 
systems
Tuesday, September 25, 2018
In this presentation, Akmal will teach attendees about Apache Ignite, and about 
the key capabilities and features important for financial applications, 
including ACID compliance, SQL compatibility, persistence, replication, 
security, fault tolerance, fraud detection and more.

IMC Summit (Oct. 2-3 in Burlingame) 
And as promised, here’s the lineup of Apache Ignite sessions at next week’s in 
In-Memory Computing Summit. The conference will include six sessions featuring 
Apache Ignite.

Here’s a listing of the Ignite-focused sessions:
Scalable Machine and Deep Learning with Apache Ignite with Denis Magda, vice 
president of the Apache Ignite PMC.

Exploring the Full Potential of Ignite Using Classless Design with David 
Follen, ING Belgium

How to implement data security in distributed systems - Transparent data 
encryption in Apache Ignite with Nikolay Izhikov, SberTech

Ingesting Streaming Data for Analysis in Apache Ignite with Pat Patterson, 
StreamSets

Apache Ignite — Using a Memory Grid for Distributed Computation Frameworks 
(Spark and Containerized Apps) with Chris Herrera, Hashmap, Inc.

Ultimate Guide to Successful Cross-Platform Deployments with Apache Ignite with 
Pavel Petroshenko, Electric Imp

Thanks everyone, and that’s all for this update!
Tom

[jira] [Created] (IGNITE-9676) Ignite as storage in Spring Session

[Anton Kurbanov (JIRA)]

Anton Kurbanov created IGNITE-9676:
--

 Summary: Ignite as storage in Spring Session
 Key: IGNITE-9676
 URL: https://issues.apache.org/jira/browse/IGNITE-9676
 Project: Ignite
  Issue Type: New Feature
Reporter: Anton Kurbanov
Assignee: Anton Kurbanov


Implement repository backed with Ignite for sessions clustering with Spring 
Session.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: TeamCity Helper's wiki page

[Dmitriy Pavlov]

Awesome, thanks! Added you to list of contributors.

Feel free to create a new page under
https://cwiki.apache.org/confluence/display/IGNITE/Testing+and+benchmarking

Sincerely,
Dmitriy Pavlov

пн, 24 сент. 2018 г. в 15:39, Dmitrii Ryabov :

> I signed up. My username - somefire.
>
> 2018-09-24 15:08 GMT+03:00 Dmitriy Pavlov :
>
>> Dmitrii,
>>
>> could you please sign up to the wiki and share your ID. I will set
>> necessary permissions.
>>
>> Sincerely,
>> Dmitriy Pavlov
>>
>> пн, 24 сент. 2018 г. в 14:57, Dmitrii Ryabov :
>>
>> > Hello, Igniters!
>> >
>> > I propose to create page about TeamCity Helper on the wiki and move here
>> > information about Helper from the MTCGA page.
>> >
>> > Also, I want to add instruction about how to use TCH to comment JIRA:
>> >
>> > 1. Start "Run All" build for PR on TeamCity.
>> >
>> > 2. Check failures and fix possible blockers.
>> > * Go to https://mtcga.gridgain.com/
>> > * Fill the form "Check branch/PR" with branch from TeamCity
>> > ("pull//head"). Leave base branch empty and press
>> > "Latest" button to compare with the latest master.
>> >
>> > 3. Comment JIRA ticket with TeamCity Helper message containing analisys
>> > results:
>> > * Go to https://mtcga.gridgain.com/services.html
>> > * Fill the form "Notify JIRA" with branch from TeamCity (or
>> > "" only). If PR is named according to contributing
>> > guide (name starts with "IGNITE-"), you can leave ticket field empty.
>> > Otherwise - enter JIRA ticket number.
>> >
>>
>
>

Re: Wrong off-heap size is reported for a node

[Pavel Pereslegin]

Hello Andrei,

All these metrics available through JMX (see "DataRegionMetrics" group) [1].

[1] https://apacheignite.readme.io/docs/memory-metrics
пн, 24 сент. 2018 г. в 17:58, aealexsandrov :
>
> Hi Igniters,
>
> Small notes according to these fix.
>
> As I see that all logic of calculation off-neap max size at the moment
> located in the ackNodeMetrics method in IgniteKernal.java. I think that it
> isn't ok because this logic should be added to other functionality too. I
> think that will be better to move this logic for example somewhere inside
> ClusterMetrics and update existed metrics.
>
> Also in case if you will push this change in current view then, for example,
> the user could be confused because he will see different off-heap values in
> the log and rest http://127.0.0.1:8080/ignite?cmd=node=true=127.0.0.1
>
> Log:
>
> Metrics for local node (to disable set 'metricsLogFrequency' to 0)
> ^-- Node [id=*3229aa83*, name=ignite1, uptime=00:00:41.202]
> ...
> ^-- Off-heap [used=8MB, free=99,82%, comm=1580MB]
> ...
>
> Rest:
>
> "nodeId":"*3229aa83*-bfcb-45b3-af05-be33da50aa2f"
> "nonHeapMemoryMaximum":-1
> "nonHeapMemoryUsed":58390208
> "nonHeapMemoryCommitted":59572224
>
> I understand that REST could be implemented later but looks like at the
> moment there is no way to get these metrics somehow except parsing the log
> files.
>
> I suggest creating the way to get it by the user.
>
> BR,
> Andrei
>
>
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[GitHub] ignite pull request #4817: IGNITE-9669 implemented

[SGrimstad]

GitHub user SGrimstad opened a pull request:

https://github.com/apache/ignite/pull/4817

IGNITE-9669 implemented



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite IGNITE-9669

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4817.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4817


commit c88061ca99cdb86952cc51e1bbf5d76738f49764
Author: SGrimstad 
Date:   2018-09-24T15:03:06Z

IGNITE-9669 implemented




---

[jira] [Created] (IGNITE-9675) Deadlock on Ignite:active() and stopping grid simultaneously calling

[Alexey Platonov (JIRA)]

Alexey Platonov created IGNITE-9675:
---

 Summary: Deadlock on Ignite:active() and stopping grid 
simultaneously calling
 Key: IGNITE-9675
 URL: https://issues.apache.org/jira/browse/IGNITE-9675
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Platonov
Assignee: Alexey Platonov


There is deadlock on Ignite:active() and stopping grid simultaneously calling
 # Trying to stop client node.

{code:java}
"main-ScalaTest-running-VisorInProcDriverSpec" #1 prio=5 os_prio=0 
tid=0x7f267800e800 nid=0x6574 sleeping[0x7f2681b7e000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at 
org.apache.ignite.internal.util.GridSpinReadWriteLock.writeLock(GridSpinReadWriteLock.java:206)
at 
org.apache.ignite.internal.processors.task.GridTaskProcessor.onKernalStop(GridTaskProcessor.java:190)
at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2135)
at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2083)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2590)
- locked <0x000797ecf7f8> (a 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2553)
at org.apache.ignite.internal.IgnitionEx.stop(IgnitionEx.java:374)
at org.apache.ignite.Ignition.stop(Ignition.java:225)
{code}
 

and
 2. Execute task that tries to get ignite.active() state, which also executes 
task 
GridClusterStateProcessor.sendComputeCheckGlobalState(GridClusterStateProcessor.java:1086)
{code:java}
"mgmt-#2470" #2965 prio=5 os_prio=0 tid=0x7f23fc001000 nid=0x730b waiting 
on condition [0x7f221f0ee000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
at 
org.apache.ignite.internal.AsyncSupportAdapter.saveOrGet(AsyncSupportAdapter.java:112)
at 
org.apache.ignite.internal.IgniteComputeImpl.call(IgniteComputeImpl.java:786)
at 
org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.sendComputeCheckGlobalState(GridClusterStateProcessor.java:1086)
at 
org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.publicApiActiveState(GridClusterStateProcessor.java:177)
at 
org.apache.ignite.internal.cluster.IgniteClusterImpl.active(IgniteClusterImpl.java:300)
at 
org.apache.ignite.internal.visor.node.VisorNodeDataCollectorTask.reduce(VisorNodeDataCollectorTask.java:75)
at 
org.gridgain.grid.internal.visor.node.VisorGridGainNodeDataCollectorTask.reduce0(VisorGridGainNodeDataCollectorTask.java:39)
at 
org.gridgain.grid.internal.visor.node.VisorGridGainNodeDataCollectorTask.reduce0(VisorGridGainNodeDataCollectorTask.java:26)
at 
org.apache.ignite.internal.visor.VisorMultiNodeTask.reduce(VisorMultiNodeTask.java:139)
at 
org.apache.ignite.internal.processors.task.GridTaskWorker$6.call(GridTaskWorker.java:1139)
at 
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6765)
at 
org.apache.ignite.internal.processors.task.GridTaskWorker.reduce(GridTaskWorker.java:1137)
at 
org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:964)
at 
org.apache.ignite.internal.processors.task.GridTaskProcessor.processJobExecuteResponse(GridTaskProcessor.java:1081)
at 
org.apache.ignite.internal.processors.task.GridTaskProcessor$JobMessageListener.onMessage(GridTaskProcessor.java:1316)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197)
at 
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
at 
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Wrong off-heap size is reported for a node

[aealexsandrov]

Hi Igniters,

Small notes according to these fix. 

As I see that all logic of calculation off-neap max size at the moment
located in the ackNodeMetrics method in IgniteKernal.java. I think that it
isn't ok because this logic should be added to other functionality too. I
think that will be better to move this logic for example somewhere inside
ClusterMetrics and update existed metrics.

Also in case if you will push this change in current view then, for example,
the user could be confused because he will see different off-heap values in
the log and rest http://127.0.0.1:8080/ignite?cmd=node=true=127.0.0.1

Log:

Metrics for local node (to disable set 'metricsLogFrequency' to 0)
^-- Node [id=*3229aa83*, name=ignite1, uptime=00:00:41.202]
...
^-- Off-heap [used=8MB, free=99,82%, comm=1580MB]
...

Rest:

"nodeId":"*3229aa83*-bfcb-45b3-af05-be33da50aa2f"
"nonHeapMemoryMaximum":-1
"nonHeapMemoryUsed":58390208
"nonHeapMemoryCommitted":59572224

I understand that REST could be implemented later but looks like at the
moment there is no way to get these metrics somehow except parsing the log
files.

I suggest creating the way to get it by the user.

BR,
Andrei





--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[jira] [Created] (IGNITE-9674) Double p2p class loading

[Sergey Antonov (JIRA)]

Sergey Antonov created IGNITE-9674:
--

 Summary: Double p2p class loading
 Key: IGNITE-9674
 URL: https://issues.apache.org/jira/browse/IGNITE-9674
 Project: Ignite
  Issue Type: Bug
Reporter: Sergey Antonov
 Attachments: AlwaysTrueTestPredicate.java, 
DoubleP2PClassLoadingTest.java, always-true-predicate.jar

On p2p class loading {{GridDeploymentRequest}} will be sent twice:

First time it will be in {{GridDeploymentPerVersionStore}}#checkLoadRemoteClass 
line 715

Second time it will be in {{GridDeploymentPerVersionStore#getDeployment line 
500}}

*How to reproduce:*
 # Remove {{AlwaysTrueTestPredicate}} from CLASSPATH, for example, pack to jar 
file.
 # Run {{DoubleP2PClassLoadingTest}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9673) Timeout in Java Client suite.

[Amelchev Nikita (JIRA)]

Amelchev Nikita created IGNITE-9673:
---

 Summary: Timeout in Java Client suite.
 Key: IGNITE-9673
 URL: https://issues.apache.org/jira/browse/IGNITE-9673
 Project: Ignite
  Issue Type: Bug
Reporter: Amelchev Nikita
Assignee: Amelchev Nikita


Example of timeout: [TC 
build|[https://ci.ignite.apache.org/viewLog.html?buildId=1919405=buildResultsDiv=IgniteTests24Java8_JavaClient].]

The possible reason is non-interruptable future:
{noformat}
"test-runner-#2440%redis.RedisProtocolStringSelfTest%" #3843 prio=5 os_prio=0 
tid=0x7f8f053fb000 nid=0x7b19 waiting on condition [0x7f8d74f8f000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2465)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2463)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4228)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put0(GridCacheAdapter.java:2463)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2444)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2421)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.put(IgniteCacheProxyImpl.java:1089)
at 
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.put(GatewayProtectedCacheProxy.java:820)
at 
org.apache.ignite.internal.processors.rest.protocols.tcp.redis.RedisProtocolStringSelfTest.testStrlen(RedisProtocolStringSelfTest.java:310)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at junit.framework.TestCase.runTest(TestCase.java:176)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2177)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:143)
at 
org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:2092)
at java.lang.Thread.run(Thread.java:748)
{noformat}

The main thread is waiting for runner thread interrupt:

{noformat}
"main" #1 prio=5 os_prio=0 tid=0x7f8f0400e000 nid=0x6c18 in Object.wait() 
[0x7f8f0db58000]
   java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1252)
- locked <0x00078bfaf668> (a org.apache.ignite.thread.IgniteThread)
at 
org.apache.ignite.internal.util.IgniteUtils.join(IgniteUtils.java:4662)
at 
org.apache.ignite.internal.util.IgniteUtils.join(IgniteUtils.java:4647)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.runTest(GridAbstractTest.java:2124)
at junit.framework.TestCase.runBare(TestCase.java:141)
at junit.framework.TestResult$1.protect(TestResult.java:122)
at junit.framework.TestResult.runProtected(TestResult.java:142)
at junit.framework.TestResult.run(TestResult.java:125)
at junit.framework.TestCase.run(TestCase.java:129)
at junit.framework.TestSuite.runTest(TestSuite.java:255)
at junit.framework.TestSuite.run(TestSuite.java:250)
at junit.framework.TestSuite.runTest(TestSuite.java:255)
at junit.framework.TestSuite.run(TestSuite.java:250)
at 
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:369)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:275)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:239)
at 
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:160)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 

Re: Critical worker threads liveness checking drawbacks

[Denis Magda]

Andrey K. and G.,

Thanks, do we have a documentation ticket created? Prachi (copied) can help
with the documentation.

--
Denis

On Mon, Sep 24, 2018 at 5:51 AM Andrey Gura  wrote:

> Andrey,
>
> finally your change is merged to master branch. Congratulations and
> thank you very much! :)
>
> I think that the next step is feature that will allow signal about
> blocked threads to the monitoring tools via MXBean.
>
> I hope you will continue development of this feature and provide your
> vision in new JIRA issue.
>
>
> On Tue, Sep 11, 2018 at 6:54 PM Andrey Kuznetsov 
> wrote:
> >
> > David, Maxim!
> >
> > Thanks a lot for you ideas. Unfortunately, I can't adopt all of them
> right
> > now: the scope is much broader than the scope of the change I implement.
> I
> > have had a talk to a group of Ignite commiters, and we agreed to complete
> > the change as follows.
> > - Blocking instructions in system-critical which may resonably last long
> > should be explicitly excluded from the monitoring.
> > - Failure handlers should have a setting to suppress some failures on
> > per-failure-type basis.
> > According to this I have updated the implementation: [1]
> >
> > [1] https://github.com/apache/ignite/pull/4089
> >
> > пн, 10 сент. 2018 г. в 22:35, David Harvey :
> >
> > > When I've done this before,I've needed to find the oldest  thread, and
> kill
> > > the node running that.   From a language standpoint, Maxim's "without
> > > progress" better than "heartbeat".   For example, what I'm most
> interested
> > > in on a distributed system is which thread started the work it has not
> > > completed the earliest, and when did that thread last make forward
> > > process. You don't want to kill a node because a thread is waiting
> on a
> > > lock held by a thread that went off-node and has not gotten a response.
> > > If you don't understand the dependency relationships, you will make
> > > incorrect recovery decisions.
> > >
> > > On Mon, Sep 10, 2018 at 4:08 AM Maxim Muzafarov 
> > > wrote:
> > >
> > > > I think we should find exact answers to these questions:
> > > >  1. What `critical` issue exactly is?
> > > >  2. How can we find critical issues?
> > > >  3. How can we handle critical issues?
> > > >
> > > > First,
> > > >  - Ignore uninterruptable actions (e.g. worker\service shutdown)
> > > >  - Long I/O operations (should be a configurable timeout for each
> type of
> > > > usage)
> > > >  - Infinite loops
> > > >  - Stalled\deadlocked threads (and\or too many parked threads,
> exclude
> > > I/O)
> > > >
> > > > Second,
> > > >  - The working queue is without progress (e.g. disco, exchange
> queues)
> > > >  - Work hasn't been completed since the last heartbeat (checking
> > > > milestones)
> > > >  - Too many system resources used by a thread for the long period of
> time
> > > > (allocated memory, CPU)
> > > >  - Timing fields associated with each thread status exceeded a
> maximum
> > > time
> > > > limit.
> > > >
> > > > Third (not too many options here),
> > > >  - `log everything` should be the default behaviour in all these
> cases,
> > > > since it may be difficult to find the cause after the restart.
> > > >  - Wait some interval of time and kill the hanging node (cluster
> should
> > > be
> > > > configured stable enough)
> > > >
> > > > Questions,
> > > >  - Not sure, but can workers miss their heartbeat deadlines if CPU
> loads
> > > up
> > > > to 80%-90%? Bursts of momentary overloads can be
> > > > expected behaviour as a normal part of system operations.
> > > >  - Why do we decide that critical thread should monitor each other?
> For
> > > > instance, if all the tasks were blocked and unable to run,
> > > > node reset would never occur. As for me, a better solution is to
> use
> > > a
> > > > separate monitor thread or pool (maybe both with software
> > > > and hardware checks) that not only checks heartbeats but
> monitors the
> > > > other system as well.
> > > >
> > > > On Mon, 10 Sep 2018 at 00:07 David Harvey 
> wrote:
> > > >
> > > > > It would be safer to restart the entire cluster than to remove the
> last
> > > > > node for a cache that should be redundant.
> > > > >
> > > > > On Sun, Sep 9, 2018, 4:00 PM Andrey Gura  wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I agree with Yakov that we can provide some option that manage
> worker
> > > > > > liveness checker behavior in case of observing that some worker
> is
> > > > > > blocked too long.
> > > > > > At least it will  some workaround for cases when node fails is
> too
> > > > > > annoying.
> > > > > >
> > > > > > Backups count threshold sounds good but I don't understand how it
> > > will
> > > > > > help in case of cluster hanging.
> > > > > >
> > > > > > The simplest solution here is alert in cases of blocking of some
> > > > > > critical worker (we can improve WorkersRegistry for this purpose
> and
> > > > > > expose list of blocked workers) and optionally call system
> configured
> > > > > > 

[GitHub] ignite pull request #4815: IGNITE-8022 Need to add README.txt for ignite-dir...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4815


---

[GitHub] ignite pull request #4816: Ignite 9004 2.5.1

[macrergate]

GitHub user macrergate opened a pull request:

https://github.com/apache/ignite/pull/4816

Ignite 9004 2.5.1



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9004-2.5.1

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4816.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4816


commit 0690bd492714f35cf01dd58a61623e50a8d3bab9
Author: Dmitriy Govorukhin 
Date:   2018-05-10T10:17:00Z

release notes 2.5.1-p1 updated

commit 2b4feaef1e5c84417f45d3efeb2185424375b215
Author: dpavlov 
Date:   2018-05-14T10:39:21Z

IGNITE-8138 Fix for incorrect uptime in Ignite metrics for long running 
server node
Cherry-picked from 4ea7f926e6f3803c616ab51e1d2e79765862efa1

commit bf6cef1468c44ece07cc3190bdff2d515195d12c
Author: Alexey Goncharuk 
Date:   2018-05-16T12:25:51Z

IGNITE-8508 Proper ordering of ZK discovery custom events ACKs

commit 1d096b35c8723ea6bbc7d3273134f4094f1fb62b
Author: Pavel Kovalenko 
Date:   2018-04-24T15:22:52Z

IGNITE-8313 Add trace logs on exchange phases and affinity calculation. - 
Fixes #3881.

Signed-off-by: dpavlov 

(cherry picked from commit f4646e4)

commit e536f3c1b1e28f4e6ed813148b3b7fae56303279
Author: Ivan Rakov 
Date:   2018-05-16T17:13:30Z

IGNITE-8499 validate_indexes command doesn't detect absent rows in cache 
data tree

(cherry picked from commit 88a6bfd)

commit 88479b21652aee66835ac0014c02dd378373e3ff
Author: Ivan Daschinskiy 
Date:   2018-05-10T17:00:12Z

IGNITE-7896 FilePageStore truncate now actually remove redundant partition 
page file.

Signed-off-by: Andrey Gura 

(cherry picked from commit d154eec)

commit fac043bd1d06a08b1426a6e83a267cbd843f
Author: Pavel Kovalenko 
Date:   2018-05-17T09:19:36Z

IGNITE-8320 Partition file can be truncated only after checkpoint - Fixes 
#3985.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit 8cb35e1)

commit 69d9272009cc6cbec07d8bbc99364ec777c851b5
Author: Sergey Chugunov 
Date:   2018-05-17T10:10:01Z

IGNITE-8474 Fixed WalStateNodeLeaveExchangeTask preventing exchange merge - 
Fixes #3990.

Signed-off-by: Alexey Goncharuk 

(cherry-picked from commit #ebd669e4c53cfd66708ff18dd59071e4aace38ae)

commit 9b80a49d642de27dd6a69daea3921d1c05919df6
Author: Evgeny Stanilovskiy 
Date:   2018-04-28T12:09:17Z

IGNITE-8401: errorneous WalPointers comparation. - Fixes #3931.

Signed-off-by: dspavlov 

commit 40ddce5d046e98b9da4c05a9a302326d4e15e08f
Author: Pavel Kovalenko 
Date:   2018-05-17T12:04:27Z

Merge branch 'ignite-2.5.1-master' of github.com:gridgain/apache-ignite 
into ignite-2.5.1-master

commit 31f8d7e445b1ef7986e3b58e700e119b8490c734
Author: Aleksei Scherbakov 
Date:   2018-05-17T17:11:41Z

IGNITE-8423 Control utility may block when joining node is watiting for 
partition map exchange. - Fixes #4018.

Signed-off-by: Alexey Goncharuk 

commit d9a80ff6c09ac92a78948fcdb292c5e5c28111d0
Author: Anton Kalashnikov 
Date:   2018-05-18T08:18:39Z

IGNITE-8464 Fixed race in WALIterator leading to skipped segments - Fixes 
#3982.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit fdad641)

commit 68bf746c47d445d5a469d29b7a4ae5ce040b3d9b
Author: Pavel Kovalenko 
Date:   2018-05-18T16:28:45Z

IGNITE-8531 Fixed NPE if checkpoint has no pages to write, but has 
partitions to destroy. - Fixes #4026.

Signed-off-by: Alexey Goncharuk 

(cherry picked from commit 5c8d9ff)

commit 18d913c6118b5132859fbc1e5bacfc97392f9bc2
Author: vd-pyatkov 
Date:   2018-05-18T13:59:14Z

IGNITE-8491 Add JMX flag: Is the node in baseline or not - Fixes #4010.

Signed-off-by: Ivan Rakov 
(cherry picked from commit f8ae30d)

commit 40bbe65df3cbd8b3f478812b0e204b2973b98d3d
Author: Alexey Goncharuk 
Date:   2018-05-21T12:13:37Z

IGNITE-8521 Do not attempt to unregister continuous query if query ID is 
null

commit affc9d47b986e054fb0c229dcf95e64a38cab13f
Author: Pavel Kovalenko 
Date:   2018-05-21T19:33:50Z

IGNITE-8544 Use exchange result topology version for local wal state 
management. - Fixes #4039.

Signed-off-by: Alexey Goncharuk 

commit 3f4b2ae4f594e8c73b5a9c4b86f2351c2bae591e
Author: Anton Kalashnikov 
Date:   2018-05-23T09:24:51Z

IGNITE-8561 SingleSegmentLogicalRecordsIterator is broken - Fixes #4045.

(cherry picked from commit 21678bc)

commit 49929b73a00f758ef099ea99f8c6f54373c30a95
Author: EdShangGG 
Date:   2018-04-28T16:27:27Z

IGNITE-7628 SqlQuery hangs indefinitely with additional not registered in 
baseline node.

Signed-off-by: Andrey Gura 

commit 95b749460b5fa32deecb946865944e854d9745d3
Author: Dmitriy Sorokin 
Date:   2018-05-24T13:03:41Z

 

[jira] [Created] (IGNITE-9672) Move o.a.i.i.processors.cache.persistence.tree.io.PageMetaIO to metastore.

[Alexei Scherbakov (JIRA)]

Alexei Scherbakov created IGNITE-9672:
-

 Summary: Move 
o.a.i.i.processors.cache.persistence.tree.io.PageMetaIO to metastore.
 Key: IGNITE-9672
 URL: https://issues.apache.org/jira/browse/IGNITE-9672
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexei Scherbakov
 Fix For: 2.8


We have in current implementation special meta page related to snapshot 
functionality.

Meta page is stored in index partition.

If index.bin is removed (for triggering index rebuild), all information is lost 
and incremental snapshot logic is broken.

Solution: move snapshot metadata in metastore.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4815: IGNITE-8022 Need to add README.txt for ignite-dir...

[dspavlov]

GitHub user dspavlov opened a pull request:

https://github.com/apache/ignite/pull/4815

IGNITE-8022 Need to add README.txt for ignite-direct-io



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/ignite ignite-8022

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4815.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4815


commit ab55b32d96a8a5f8913852c43112478022e44e17
Author: Dmitriy Pavlov 
Date:   2018-09-24T13:52:03Z

IGNITE-8022 Need to add README.txt for ignite-direct-io




---

[jira] [Created] (IGNITE-9671) Hanging the end of the CacheAsyncOperationsFailoverTxTest#testPutAllAsyncFailover test.

[Alexey Stelmak (JIRA)]

Alexey Stelmak created IGNITE-9671:
--

 Summary: Hanging the end of the 
CacheAsyncOperationsFailoverTxTest#testPutAllAsyncFailover test.
 Key: IGNITE-9671
 URL: https://issues.apache.org/jira/browse/IGNITE-9671
 Project: Ignite
  Issue Type: Bug
Reporter: Alexey Stelmak






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4814: IGNITE-9670: debug changes

[tledkov-gridgain]

GitHub user tledkov-gridgain opened a pull request:

https://github.com/apache/ignite/pull/4814

IGNITE-9670: debug changes



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-9670

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/4814.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4814


commit cd89c4f0322c6223dd4eda62a72050d121fbad24
Author: tledkov-gridgain 
Date:   2018-09-24T13:40:53Z

IGNITE-9670: debug changes




---

Re: java.lang.ClassCastException: org.apache.ignite.internal.pagemem.impl.PageMemoryNoStoreImpl cannot be cast to org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryEx

[Ilya Kasnacheev]

Hello!

What is the Ignite version used there? I remember seeing errors like this
one, but way back.

Regards,
-- 
Ilya Kasnacheev


вс, 23 сент. 2018 г. в 21:07, kcheng.mvp :

> I ran into this error with below configuration(IgniteSpringBean)
>
> I set * defaultDataRegionConfiguration* as   *  name="persistenceEnabled" value="false"/>* for unit test. If I set this to
> true then everything goes well.
>
>
> 
>  class="org.apache.ignite.configuration.DataStorageConfiguration">
>
>  value="${ignite.storage.storagePath}"/>
>  value="${ignite.storage.walPath}"/>
>  value="${ignite.storage.walArchivePath}"/>
>
> 
> 
>  class="org.apache.ignite.configuration.DataRegionConfiguration">
> 
>  value='#{${ignite.storage.dataRegion.region_uat.initialSize}}'/>
>  value='#{${ignite.storage.dataRegion.region_uat.maxSize}}'/>
>*  value="true"/>*
> 
>  class="org.apache.ignite.configuration.DataRegionConfiguration">
> 
>  value='#{${ignite.storage.dataRegion.region_sc.initialSize}}'/>
>  value='#{${ignite.storage.dataRegion.region_sc.maxSize}}'/>
>*  value="true"/>*
> 
> 
> 
> 
>  class="org.apache.ignite.configuration.DataRegionConfiguration">
> 
>
> 
>*  value="false"/>*
> 
> 
> 
> 
>
>
>
>
> here is the full error stack
>
>
> Caused by: class org.apache.ignite.IgniteException: Failed to activate
> cluster
> at
>
> org.apache.ignite.internal.util.IgniteUtils.convertException(IgniteUtils.java:990)
> at
>
> org.apache.ignite.internal.cluster.IgniteClusterImpl.active(IgniteClusterImpl.java:315)
> at
> org.apache.ignite.internal.IgniteKernal.active(IgniteKernal.java:3399)
> at
> org.apache.ignite.IgniteSpringBean.active(IgniteSpringBean.java:615)
> at
> com.nfhd.data.Application.lambda$commandLineRunner$0(Application.java:90)
> at
>
> org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:797)
> ... 30 more
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to
> activate cluster
> Suppressed: java.lang.ClassCastException:
> org.apache.ignite.internal.pagemem.impl.PageMemoryNoStoreImpl cannot be
> cast
> to
>
> org.apache.ignite.internal.processors.cache.persistence.pagemem.PageMemoryEx
> at
>
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.getPageMemoryForCacheGroup(GridCacheDatabaseSharedManager.java:2154)
> at
>
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:2007)
> at
>
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreMemory(GridCacheDatabaseSharedManager.java:1929)
> at
>
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readCheckpointAndRestoreMemory(GridCacheDatabaseSharedManager.java:755)
> at
>
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onClusterStateChangeRequest(GridDhtPartitionsExchangeFuture.java:894)
> at
>
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:641)
> at
>
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2419)
> at
>
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2299)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at java.lang.Thread.run(Thread.java:748)
>
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

[jira] [Created] (IGNITE-9670) JDK9: supports hadoop Ignite module

[Taras Ledkov (JIRA)]

Taras Ledkov created IGNITE-9670:


 Summary: JDK9: supports hadoop Ignite module
 Key: IGNITE-9670
 URL: https://issues.apache.org/jira/browse/IGNITE-9670
 Project: Ignite
  Issue Type: Task
  Components: hadoop
Affects Versions: 2.6
Reporter: Taras Ledkov
Assignee: Taras Ledkov
 Fix For: 2.7


# Hadoop processor doens't start with error: {{class 
org.apache.ignite.IgniteCheckedException: Java version 9 and above is not 
supported.}}
# Hadoop tests aren't run from IDEA because used classloaders chain is changed 
(related to IGNITE-8146).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9669) Spark Integration fails test after fix IGNITE-8552

[Sergey Grimstad (JIRA)]

Sergey Grimstad created IGNITE-9669:
---

 Summary: Spark Integration fails test after fix IGNITE-8552
 Key: IGNITE-9669
 URL: https://issues.apache.org/jira/browse/IGNITE-9669
 Project: Ignite
  Issue Type: Bug
  Components: sql
Affects Versions: 2.7
Reporter: Sergey Grimstad
 Fix For: 2.7






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-9668) Comment JIRA from pr.html

[Ryabov Dmitrii (JIRA)]

Ryabov Dmitrii created IGNITE-9668:
--

 Summary: Comment JIRA from pr.html
 Key: IGNITE-9668
 URL: https://issues.apache.org/jira/browse/IGNITE-9668
 Project: Ignite
  Issue Type: Sub-task
Reporter: Ryabov Dmitrii
Assignee: Ryabov Dmitrii


Show button to comment Jira ticket on the pr.html page. If ticket can't be 
detected automatically, then ticket should be gained from user.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #4737: IGNITE-9544 BinaryOutputStream#writeByte excessiv...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4737


---

Re: Critical worker threads liveness checking drawbacks

[Andrey Gura]

Andrey,

finally your change is merged to master branch. Congratulations and
thank you very much! :)

I think that the next step is feature that will allow signal about
blocked threads to the monitoring tools via MXBean.

I hope you will continue development of this feature and provide your
vision in new JIRA issue.


On Tue, Sep 11, 2018 at 6:54 PM Andrey Kuznetsov  wrote:
>
> David, Maxim!
>
> Thanks a lot for you ideas. Unfortunately, I can't adopt all of them right
> now: the scope is much broader than the scope of the change I implement. I
> have had a talk to a group of Ignite commiters, and we agreed to complete
> the change as follows.
> - Blocking instructions in system-critical which may resonably last long
> should be explicitly excluded from the monitoring.
> - Failure handlers should have a setting to suppress some failures on
> per-failure-type basis.
> According to this I have updated the implementation: [1]
>
> [1] https://github.com/apache/ignite/pull/4089
>
> пн, 10 сент. 2018 г. в 22:35, David Harvey :
>
> > When I've done this before,I've needed to find the oldest  thread, and kill
> > the node running that.   From a language standpoint, Maxim's "without
> > progress" better than "heartbeat".   For example, what I'm most interested
> > in on a distributed system is which thread started the work it has not
> > completed the earliest, and when did that thread last make forward
> > process. You don't want to kill a node because a thread is waiting on a
> > lock held by a thread that went off-node and has not gotten a response.
> > If you don't understand the dependency relationships, you will make
> > incorrect recovery decisions.
> >
> > On Mon, Sep 10, 2018 at 4:08 AM Maxim Muzafarov 
> > wrote:
> >
> > > I think we should find exact answers to these questions:
> > >  1. What `critical` issue exactly is?
> > >  2. How can we find critical issues?
> > >  3. How can we handle critical issues?
> > >
> > > First,
> > >  - Ignore uninterruptable actions (e.g. worker\service shutdown)
> > >  - Long I/O operations (should be a configurable timeout for each type of
> > > usage)
> > >  - Infinite loops
> > >  - Stalled\deadlocked threads (and\or too many parked threads, exclude
> > I/O)
> > >
> > > Second,
> > >  - The working queue is without progress (e.g. disco, exchange queues)
> > >  - Work hasn't been completed since the last heartbeat (checking
> > > milestones)
> > >  - Too many system resources used by a thread for the long period of time
> > > (allocated memory, CPU)
> > >  - Timing fields associated with each thread status exceeded a maximum
> > time
> > > limit.
> > >
> > > Third (not too many options here),
> > >  - `log everything` should be the default behaviour in all these cases,
> > > since it may be difficult to find the cause after the restart.
> > >  - Wait some interval of time and kill the hanging node (cluster should
> > be
> > > configured stable enough)
> > >
> > > Questions,
> > >  - Not sure, but can workers miss their heartbeat deadlines if CPU loads
> > up
> > > to 80%-90%? Bursts of momentary overloads can be
> > > expected behaviour as a normal part of system operations.
> > >  - Why do we decide that critical thread should monitor each other? For
> > > instance, if all the tasks were blocked and unable to run,
> > > node reset would never occur. As for me, a better solution is to use
> > a
> > > separate monitor thread or pool (maybe both with software
> > > and hardware checks) that not only checks heartbeats but monitors the
> > > other system as well.
> > >
> > > On Mon, 10 Sep 2018 at 00:07 David Harvey  wrote:
> > >
> > > > It would be safer to restart the entire cluster than to remove the last
> > > > node for a cache that should be redundant.
> > > >
> > > > On Sun, Sep 9, 2018, 4:00 PM Andrey Gura  wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I agree with Yakov that we can provide some option that manage worker
> > > > > liveness checker behavior in case of observing that some worker is
> > > > > blocked too long.
> > > > > At least it will  some workaround for cases when node fails is too
> > > > > annoying.
> > > > >
> > > > > Backups count threshold sounds good but I don't understand how it
> > will
> > > > > help in case of cluster hanging.
> > > > >
> > > > > The simplest solution here is alert in cases of blocking of some
> > > > > critical worker (we can improve WorkersRegistry for this purpose and
> > > > > expose list of blocked workers) and optionally call system configured
> > > > > failure processor. BTW, failure processor can be extended in order to
> > > > > perform any checks (e.g. backup count) and decide whether it should
> > > > > stop node or not.
> > > > > On Sat, Sep 8, 2018 at 3:42 PM Andrey Kuznetsov 
> > > > wrote:
> > > > > >
> > > > > > David, Yakov, I understand your fears. But liveness checks deal
> > with
> > > > > > _critical_ conditions, i.e. when such a condition is met we
> > conclude
> > > > the
> > > 

Re: TeamCity Helper's wiki page

[Dmitrii Ryabov]

I signed up. My username - somefire.

2018-09-24 15:08 GMT+03:00 Dmitriy Pavlov :

> Dmitrii,
>
> could you please sign up to the wiki and share your ID. I will set
> necessary permissions.
>
> Sincerely,
> Dmitriy Pavlov
>
> пн, 24 сент. 2018 г. в 14:57, Dmitrii Ryabov :
>
> > Hello, Igniters!
> >
> > I propose to create page about TeamCity Helper on the wiki and move here
> > information about Helper from the MTCGA page.
> >
> > Also, I want to add instruction about how to use TCH to comment JIRA:
> >
> > 1. Start "Run All" build for PR on TeamCity.
> >
> > 2. Check failures and fix possible blockers.
> > * Go to https://mtcga.gridgain.com/
> > * Fill the form "Check branch/PR" with branch from TeamCity
> > ("pull//head"). Leave base branch empty and press
> > "Latest" button to compare with the latest master.
> >
> > 3. Comment JIRA ticket with TeamCity Helper message containing analisys
> > results:
> > * Go to https://mtcga.gridgain.com/services.html
> > * Fill the form "Notify JIRA" with branch from TeamCity (or
> > "" only). If PR is named according to contributing
> > guide (name starts with "IGNITE-"), you can leave ticket field empty.
> > Otherwise - enter JIRA ticket number.
> >
>

[GitHub] ignite pull request #4808: IGNITE-9665: fix for RPM build

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4808


---

Re: Include TeamCity Helper into contribution process

[Dmitriy Pavlov]

++1 Only one concern - the domain name is now GridGain. I will ask infra
about Apache domain, but it will require some time to be done.

пн, 24 сент. 2018 г. в 15:07, Dmitrii Ryabov :

> Hello, Igniters!
>
> I propose to update the "How to Contribute" page to include TeamCity
> Helper's JIRA commenting functionality into contribution process.
>
> This will speedup review process by highlighting new failures, timeouts and
> othe critical failures in the JIRA ticket.
>
>
> For example, we can replace next text on the "How to Contribute" page
>
> * Once tests are passed, the pull request can be reviewed and merged by a
> committer. Move a corresponding JIRA ticket to "Patch Available" state by
> clicking on "Submit Patch" button and let the community know that you're
> ready for review.
>
> replace to
>
> * Once tests are passed, use TeamCity Helper(https://mtcga.gridgain.com/)
> to analyze tests and comment(https://mtcga.gridgain.com/services.html)
> JIRA
> ticket with the results.
> * Fix found failures (possible blockers) and rerun tests.
> * Restart suites with timeout if you sure that they failed not because
> of PR.
> * Once TeamCity Helper has shown the absence of possible blockers, the pull
> request can be reviewed and merged by a committer. Move a corresponding
> JIRA ticket to "Patch Available" state by clicking on "Submit Patch" button
> and let the community know that you're ready for review.
>

Re: Apache Ignite 2.7 - Release Procedure issues

[Nikolay Izhikov]

Hello, Petr.

My suggestion is to migrate to a newer version of GPG and throw an error 
message if one use old version.

В Пн, 24/09/2018 в 14:53 +0300, Petr Ivanov пишет:
> I’ve checked the changes and they are good both on old and latest versions of 
> Ubuntu.
> 
> 
> However, I’ve stumbled upon another problem — GPG: current release scripts do 
> not honour latest GPG versions.
> I can introduce corresponding changes, but question is — should release 
> script check for GPG version and have 2 version of signing commands or just 
> warn user about old version of GPG and exit?
> 
> 
> > On 21 Sep 2018, at 19:46, Nikolay Izhikov  wrote:
> > 
> > Hello, Petr.
> > 
> > Seems that rpm build script doesn't work on a lates Ubuntu Linux.
> > I've created a ticket [1] and found a fix for it [2]
> > 
> > With one line fix rpm build is working under my environment.
> > Can you check fix on your environment?
> > 
> > [1] https://issues.apache.org/jira/browse/IGNITE-9665
> > 
> > [2] https://github.com/apache/ignite/pull/4808
> > 
> > В Пт, 21/09/2018 в 16:22 +0300, Petr Ivanov пишет:
> > > Hi, Nikolay
> > > 
> > > 
> > > I’ve tested vote_3_step_1 and vote_3_step_2 scripts from [1] and they are 
> > > OK.
> > > My configuration:
> > > - generated gnupg key (~/.gnupg)
> > > - Ubuntu 16.04 (with latest updates)
> > > - packages: subversion git unzip alien rpm fakeroot gcc dpkg-sig 
> > > gnupg-agent
> > > 
> > > Please double check you environment for release procedure
> > > 
> > > 
> > > [1] 
> > > https://ci.ignite.apache.org/viewLog.html?buildId=1914618=ApacheIgniteReleaseJava8_PrepareVote=artifacts#!hkm8d5gqy4ii
> > > 
> > > 
> > > > On 20 Sep 2018, at 17:39, Nikolay Izhikov  wrote:
> > > > 
> > > > Hello, Igniters.
> > > > 
> > > > I've started to write Wiki article for a future Release Managers.
> > > > Since release process doesn't described anywhere public I do it while 
> > > > releasing 2.7:
> > > > 
> > > > https://cwiki.apache.org/confluence/display/IGNITE/Release+manager+Notes
> > > > 
> > > > Any feedback is strongly appreciated.
> > > > 
> > > > I've tried to walk through vote steps in release procedure and found 
> > > > some issues:
> > > > 
> > > > Some of them we already fixed with Petr Ivanod and Anton Vinogradov.
> > > > Thank you, guys!
> > > > 
> > > > For now, I stuck on the following issue:
> > > > 
> > > > `vote_3_step_1\[packages\]build.sh` is broken.
> > > > I got following output:
> > > > 
> > > > ```
> > > > RPM build errors:
> > > >   bogus date in %changelog: Fri Jun 17 2018 Peter Ivanov 
> > > >  - 2.6.0-1
> > > >   File not found: 
> > > > /tmp/tmp.EezfJVTwLm/BUILDROOT/apache-ignite-2.6.0-1.x86_64/usr/share/doc/apache-ignite-2.6.0/MIGRATION_GUIDE.txt
> > > > + processTrap
> > > > + echo 'Removing temporary work directories:  /tmp/tmp.EezfJVTwLm'
> > > > Removing temporary work directories:  /tmp/tmp.EezfJVTwLm
> > > > + rm -rf /tmp/tmp.EezfJVTwLm
> > > > ```
> > > > 
> > > > 1. Script uses version number 2.6 somehow.
> > > > 2. It fails because MIGRATION_GUIDE file doesn't exists.
> > > > 
> > > > Is there anybody who can help with this issue?
> > > > 
> > > > 
> > > > 
> 
> 

signature.asc
Description: This is a digitally signed message part

Re: TeamCity Helper's wiki page

[Dmitriy Pavlov]

Dmitrii,

could you please sign up to the wiki and share your ID. I will set
necessary permissions.

Sincerely,
Dmitriy Pavlov

пн, 24 сент. 2018 г. в 14:57, Dmitrii Ryabov :

> Hello, Igniters!
>
> I propose to create page about TeamCity Helper on the wiki and move here
> information about Helper from the MTCGA page.
>
> Also, I want to add instruction about how to use TCH to comment JIRA:
>
> 1. Start "Run All" build for PR on TeamCity.
>
> 2. Check failures and fix possible blockers.
> * Go to https://mtcga.gridgain.com/
> * Fill the form "Check branch/PR" with branch from TeamCity
> ("pull//head"). Leave base branch empty and press
> "Latest" button to compare with the latest master.
>
> 3. Comment JIRA ticket with TeamCity Helper message containing analisys
> results:
> * Go to https://mtcga.gridgain.com/services.html
> * Fill the form "Notify JIRA" with branch from TeamCity (or
> "" only). If PR is named according to contributing
> guide (name starts with "IGNITE-"), you can leave ticket field empty.
> Otherwise - enter JIRA ticket number.
>

Include TeamCity Helper into contribution process

[Dmitrii Ryabov]

Hello, Igniters!

I propose to update the "How to Contribute" page to include TeamCity
Helper's JIRA commenting functionality into contribution process.

This will speedup review process by highlighting new failures, timeouts and
othe critical failures in the JIRA ticket.


For example, we can replace next text on the "How to Contribute" page

* Once tests are passed, the pull request can be reviewed and merged by a
committer. Move a corresponding JIRA ticket to "Patch Available" state by
clicking on "Submit Patch" button and let the community know that you're
ready for review.

replace to

* Once tests are passed, use TeamCity Helper(https://mtcga.gridgain.com/)
to analyze tests and comment(https://mtcga.gridgain.com/services.html) JIRA
ticket with the results.
* Fix found failures (possible blockers) and rerun tests.
* Restart suites with timeout if you sure that they failed not because
of PR.
* Once TeamCity Helper has shown the absence of possible blockers, the pull
request can be reviewed and merged by a committer. Move a corresponding
JIRA ticket to "Patch Available" state by clicking on "Submit Patch" button
and let the community know that you're ready for review.

TeamCity Helper's wiki page

[Dmitrii Ryabov]

Hello, Igniters!

I propose to create page about TeamCity Helper on the wiki and move here
information about Helper from the MTCGA page.

Also, I want to add instruction about how to use TCH to comment JIRA:

1. Start "Run All" build for PR on TeamCity.

2. Check failures and fix possible blockers.
* Go to https://mtcga.gridgain.com/
* Fill the form "Check branch/PR" with branch from TeamCity
("pull//head"). Leave base branch empty and press
"Latest" button to compare with the latest master.

3. Comment JIRA ticket with TeamCity Helper message containing analisys
results:
* Go to https://mtcga.gridgain.com/services.html
* Fill the form "Notify JIRA" with branch from TeamCity (or
"" only). If PR is named according to contributing
guide (name starts with "IGNITE-"), you can leave ticket field empty.
Otherwise - enter JIRA ticket number.

Re: Apache Ignite 2.7 - Release Procedure issues

[Petr Ivanov]

I’ve checked the changes and they are good both on old and latest versions of 
Ubuntu.


However, I’ve stumbled upon another problem — GPG: current release scripts do 
not honour latest GPG versions.
I can introduce corresponding changes, but question is — should release script 
check for GPG version and have 2 version of signing commands or just warn user 
about old version of GPG and exit?


> On 21 Sep 2018, at 19:46, Nikolay Izhikov  wrote:
> 
> Hello, Petr.
> 
> Seems that rpm build script doesn't work on a lates Ubuntu Linux.
> I've created a ticket [1] and found a fix for it [2]
> 
> With one line fix rpm build is working under my environment.
> Can you check fix on your environment?
> 
> [1] https://issues.apache.org/jira/browse/IGNITE-9665
> 
> [2] https://github.com/apache/ignite/pull/4808
> 
> В Пт, 21/09/2018 в 16:22 +0300, Petr Ivanov пишет:
>> Hi, Nikolay
>> 
>> 
>> I’ve tested vote_3_step_1 and vote_3_step_2 scripts from [1] and they are OK.
>> My configuration:
>> - generated gnupg key (~/.gnupg)
>> - Ubuntu 16.04 (with latest updates)
>> - packages: subversion git unzip alien rpm fakeroot gcc dpkg-sig gnupg-agent
>> 
>> Please double check you environment for release procedure
>> 
>> 
>> [1] 
>> https://ci.ignite.apache.org/viewLog.html?buildId=1914618=ApacheIgniteReleaseJava8_PrepareVote=artifacts#!hkm8d5gqy4ii
>> 
>> 
>>> On 20 Sep 2018, at 17:39, Nikolay Izhikov  wrote:
>>> 
>>> Hello, Igniters.
>>> 
>>> I've started to write Wiki article for a future Release Managers.
>>> Since release process doesn't described anywhere public I do it while 
>>> releasing 2.7:
>>> 
>>> https://cwiki.apache.org/confluence/display/IGNITE/Release+manager+Notes
>>> 
>>> Any feedback is strongly appreciated.
>>> 
>>> I've tried to walk through vote steps in release procedure and found some 
>>> issues:
>>> 
>>> Some of them we already fixed with Petr Ivanod and Anton Vinogradov.
>>> Thank you, guys!
>>> 
>>> For now, I stuck on the following issue:
>>> 
>>> `vote_3_step_1\[packages\]build.sh` is broken.
>>> I got following output:
>>> 
>>> ```
>>> RPM build errors:
>>>   bogus date in %changelog: Fri Jun 17 2018 Peter Ivanov 
>>>  - 2.6.0-1
>>>   File not found: 
>>> /tmp/tmp.EezfJVTwLm/BUILDROOT/apache-ignite-2.6.0-1.x86_64/usr/share/doc/apache-ignite-2.6.0/MIGRATION_GUIDE.txt
>>> + processTrap
>>> + echo 'Removing temporary work directories:  /tmp/tmp.EezfJVTwLm'
>>> Removing temporary work directories:  /tmp/tmp.EezfJVTwLm
>>> + rm -rf /tmp/tmp.EezfJVTwLm
>>> ```
>>> 
>>> 1. Script uses version number 2.6 somehow.
>>> 2. It fails because MIGRATION_GUIDE file doesn't exists.
>>> 
>>> Is there anybody who can help with this issue?
>>> 
>>> 
>>> 
>> 

Re: IgniteEvents for MVCC caches

[Vladimir Ozerov]

Igniters,

Igniters, I see three distinct problems with cache events, if we are to put
SQL aside.

1) Strange "ENTRY_CREATED" and "ENTRY_DESTROYED" events
They simply look outdated. I propose to deprecate them in the next AI
release with optional flag to re-enalbe them if someone really used them.

2) Inconsistency for PUT/REMOVE/READ events
Currently for different transaction types these events are fired at
different times. For TRANSACTIONAL cache we fire them when entry is really
read or updated, what depends on transaction mode. E.g. for
PESSIMISTIC/READ_COMMITTED mode two subsequent get() will raise two events.
For PESSIMISTIC/REPEATABLE_READ it will be one event. The same goes for
puts. For MVCC caches it will work differently - every update will trigger
separate PUT event if executed through SQL request. But not every update
through PUT API will trigger an event. I think we need to come to clear
semantics here, which can be explained for user. For me it could sound like
this:
*"If entry is read inside a transaction, at least one read event is fired.
If entry is updated within transaction, at least one update event is
fired."*

What do you think?

3) Lock/unlock events inconsistency
>From what I know "LOCKED/UNLOCKED" events has no relation to
IgniteCache.lock() method family. These events are fired in any cache when
entry is locked for update. This is valid for all cache modes. But due to
arhcitecture differences, we cannot fire UNLOCK events for MVCC caches.
Because there is no explicit UNLOCK phase anymore. That is, we know when
current transaction locked it. But the only way to understand that entry is
unlocked is to check it in subsequent transaction. The only way to fix is
to store all affected keys in-memory, what will lead to OOME for large
transactions. My proposal here is not to fire "UNLOCK" event for MVCC
caches.

So, to summarize, my proposal is:
1) Deprecate ENTRY_CREATED/ENTRY_DESTROYED events
2) Formulate in JavaDocs semantics of PUT/REMOVE/READ events. MVCC doesn't
bring any new problems here.
3) Do not fire UNLOCK events for MVCC (explain in JavaDocs)

What do you think?

Vladimir.

On Mon, Sep 24, 2018 at 1:28 PM Павлухин Иван  wrote:

> Hi guys,
>
> Andrey, thanks for pointing out explicit lock/unlock operations. Actually,
> I think that lock/unlock events are perfectly valid for TRANSACTIONAL
> caches, so there is a little chance that they could be deprecated. On the
> other hand mvcc lock/unlock is not related to non-mvcc lock/unlock because
> we do not support transactions involving caches with different modes. I
> could imagine that lock/unlock events can be used for debugging purposes.
> Currently events documentation is quite humble, e.g. for
> EVT_CACHE_OBJECT_LOCKED event we have javadoc "Built-in event type: object
> locked.". What object? What does "locked" mean? And "locking" semantics is
> not slightly different for mvcc and non-mvcc caches (already mentioned
> unlock on commit). So, another idea which came to my mind is introducing
> additional event type, say MVCC_LOCKED.
>
> Denis, there are 2 groups of cache events: EVT_CACHE_ENTRY_<...> and
> EVT_CACHE_OBJECT_<...>. EVT_CACHE_ENTRY_CREATED and
> EVT_CACHE_ENTRY_DESTROYED are related to on-heap part of cache behavior.
> Correct me if I am wrong, but now all caches use off-heap storage. And
> EVT_CACHE_ENTRY_CREATED, EVT_CACHE_ENTRY_DESTROYED are triggered when
> GridCacheMapEntry object is created as part of an access chain to values
> stored in the cache. E.g. cache.put invocation causes following sequence of
> events: CACHE_ENTRY_CREATED, CACHE_OBJECT_PUT, CACHE_ENTRY_DESTROYED.
> Actually, there are no special problems with supporting CACHE_ENTRY_<...>
> events for MVCC. And yes, I do not see much user value in them.
>
> Also, it seems that I chose a little bit confusing name for the thread. I
> would like to keep current discussion mainly about key-value API events for
> MVCC caches. SQL and events is a little bit different topic with number of
> additional problems.
>
> 2018-09-22 17:35 GMT+03:00 Denis Magda :
>
> > Vladimir,
> >
> > That's a good question. I confused _ENTRY_ events with those mentioned by
> > you.
> >
> > Anyway, are you saying that the _ENTRY_ events are workable but have no
> > value or they are not workable any longer with the MVCC?
> >
> > --
> > Denis
> >
> >
> >
> > On Sat, Sep 22, 2018 at 4:10 AM Vladimir Ozerov 
> > wrote:
> >
> > > Denis,
> > >
> > > What is the point of these events, provided that we already has
> CACHE_PUT
> > > and CACHE_REMOVED?
> > >
> > > On Sat, Sep 22, 2018 at 1:01 AM Denis Magda  wrote:
> > >
> > > > Hello Ivan,
> > > >
> > > > First of all, it looks like that EVT_CACHE_ENTRY_CREATED,
> > > > > EVT_CACHE_ENTRY_DESTROYED are legacy came from Ignite 1.x time and
> > > have a
> > > > > little value nowadays. Does anyone have something against
> deprecating
> > > > them?
> > > >
> > > >
> > > > These events are used in a pure caching or in-memory data grid use
> 

Re: [MTCGA]: new failures in builds [1848302] needs to be handled

[Dmitriy Pavlov]

Hi Igniters,

It seems it is duplicate. But anyway Hadoop needs to be fixed.

Sincerely,
Dmitriy Pavlov

пн, 24 сент. 2018 г. в 0:14, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *Recently contributed test failed in master
> org.apache.ignite.testsuites.IgniteHadoopTestSuite.initializationError
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=6265644424903641182=%3Cdefault%3E=testDetails
>  Changes may lead to failure were done by
>  - verbalab
> http://ci.ignite.apache.org/viewModification.html?modId=831346=false
>  - verbalab
> http://ci.ignite.apache.org/viewModification.html?modId=831344=false
>  - dmitriyff
> http://ci.ignite.apache.org/viewModification.html?modId=831341=false
>  - verbalab
> http://ci.ignite.apache.org/viewModification.html?modId=831339=false
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 00:13:51 24-09-2018
>

[jira] [Created] (IGNITE-9667) Error in DataRegionMetrics.getTotalAllocatedSize in case of CREATE/INSERT/DROP/CREATE/INSERT

[Vasiliy Sisko (JIRA)]

Vasiliy Sisko created IGNITE-9667:
-

 Summary: Error in DataRegionMetrics.getTotalAllocatedSize in case 
of CREATE/INSERT/DROP/CREATE/INSERT
 Key: IGNITE-9667
 URL: https://issues.apache.org/jira/browse/IGNITE-9667
 Project: Ignite
  Issue Type: Bug
  Components: persistence
Reporter: Vasiliy Sisko
Assignee: Vasiliy Sisko


Implement test to check allocation of space in persistence store on second 
similar changes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Spring Session Ignite plugin

[Dmitriy Pavlov]

Hi Anton,

There is GitHub mirror of Apache Ignite codebase here:
https://github.com/apache/ignite

You can create JIRA ticket and pull request from your Apache Ignite fork.

Another option is to donate to some supplementary repository, but this way
is much more complex and requires discussion from all community and grant
agreement signing.

So I suggest creating PR.

Sincerely,
Dmitriy Pavlov

пн, 24 сент. 2018 г. в 13:03, akurbanov :

> Hello Igniters,
>
> I have implemented small community extension to use Apache Ignite as
> backing
> repository for session clustering with Spring Session:  github link
>   . I was wondering what
> would be the best way to share this with community, is there any Ignite
> repository existing for community integrations?
>
> Regards,
> anton
>
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

Re: IgniteEvents for MVCC caches

[Павлухин Иван]

Hi guys,

Andrey, thanks for pointing out explicit lock/unlock operations. Actually,
I think that lock/unlock events are perfectly valid for TRANSACTIONAL
caches, so there is a little chance that they could be deprecated. On the
other hand mvcc lock/unlock is not related to non-mvcc lock/unlock because
we do not support transactions involving caches with different modes. I
could imagine that lock/unlock events can be used for debugging purposes.
Currently events documentation is quite humble, e.g. for
EVT_CACHE_OBJECT_LOCKED event we have javadoc "Built-in event type: object
locked.". What object? What does "locked" mean? And "locking" semantics is
not slightly different for mvcc and non-mvcc caches (already mentioned
unlock on commit). So, another idea which came to my mind is introducing
additional event type, say MVCC_LOCKED.

Denis, there are 2 groups of cache events: EVT_CACHE_ENTRY_<...> and
EVT_CACHE_OBJECT_<...>. EVT_CACHE_ENTRY_CREATED and
EVT_CACHE_ENTRY_DESTROYED are related to on-heap part of cache behavior.
Correct me if I am wrong, but now all caches use off-heap storage. And
EVT_CACHE_ENTRY_CREATED, EVT_CACHE_ENTRY_DESTROYED are triggered when
GridCacheMapEntry object is created as part of an access chain to values
stored in the cache. E.g. cache.put invocation causes following sequence of
events: CACHE_ENTRY_CREATED, CACHE_OBJECT_PUT, CACHE_ENTRY_DESTROYED.
Actually, there are no special problems with supporting CACHE_ENTRY_<...>
events for MVCC. And yes, I do not see much user value in them.

Also, it seems that I chose a little bit confusing name for the thread. I
would like to keep current discussion mainly about key-value API events for
MVCC caches. SQL and events is a little bit different topic with number of
additional problems.

2018-09-22 17:35 GMT+03:00 Denis Magda :

> Vladimir,
>
> That's a good question. I confused _ENTRY_ events with those mentioned by
> you.
>
> Anyway, are you saying that the _ENTRY_ events are workable but have no
> value or they are not workable any longer with the MVCC?
>
> --
> Denis
>
>
>
> On Sat, Sep 22, 2018 at 4:10 AM Vladimir Ozerov 
> wrote:
>
> > Denis,
> >
> > What is the point of these events, provided that we already has CACHE_PUT
> > and CACHE_REMOVED?
> >
> > On Sat, Sep 22, 2018 at 1:01 AM Denis Magda  wrote:
> >
> > > Hello Ivan,
> > >
> > > First of all, it looks like that EVT_CACHE_ENTRY_CREATED,
> > > > EVT_CACHE_ENTRY_DESTROYED are legacy came from Ignite 1.x time and
> > have a
> > > > little value nowadays. Does anyone have something against deprecating
> > > them?
> > >
> > >
> > > These events are used in a pure caching or in-memory data grid use
> cases
> > > where key-value is a primary access pattern. I personally know several
> > > companies who rely on those events. So, we can't discontinue them, not
> > > everyone uses SQL. What's a hurdle of making them working with MVCC?
> > >
> > > --
> > > Denis
> > >
> > > On Fri, Sep 21, 2018 at 6:44 AM Павлухин Иван 
> > wrote:
> > >
> > > > Hi Igniters,
> > > >
> > > > As you might know MVCC was introduced in Apache Ignite. We started
> > > > IgniteEvents support implementation for MVCC caches and faced some
> > > > obstacles. I would like to start a discussion about next steps which
> > > should
> > > > be done to deal with current problems. The main theme is about
> > > > EventType.EVTS_CACHE and key-value API.
> > > >
> > > > First of all, it looks like that EVT_CACHE_ENTRY_CREATED,
> > > > EVT_CACHE_ENTRY_DESTROYED are legacy came from Ignite 1.x time and
> > have a
> > > > little value nowadays. Does anyone have something against deprecating
> > > them?
> > > >
> > > > Second, there could be problems with EVT_CACHE_OBJECT_UNLOCKED event,
> > > > because for MVCC all locks are released on transaction end. And it
> does
> > > not
> > > > sound good idea to track all locked entries during transaction
> > execution,
> > > > in MVCC we are not keeping entries modified participating in
> > transaction
> > > in
> > > > private working set in memory. One possible solution is deprecation
> of
> > > > lock, unlock events. Another one is introducing special lock event
> for
> > > > MVCC, but it will be confusing. Also, I see an option of firing only
> > > > EVT_CACHE_OBJECT_LOCKED.
> > > >
> > > > Last, is different number of events fired for similar scenarios and
> > > > different cache modes. Let's consider "put, remove, put" for the same
> > key
> > > > and different modes. For ATOMIC 2 put and 1 remove event will be
> fired.
> > > For
> > > > TRANSACTIONAL 1 put will be fired in case of commit and nothing in
> case
> > > of
> > > > rollback. For MVCC in current vision 2 put will be fired regardless
> > > whether
> > > > transaction was committed and rolled back. Currently I do not see
> > options
> > > > how to overcome it.
> > > >
> > > > Also, I hardly imagine current use cases for cache events. I think
> that
> > > > understanding them is the best way for developing working solution

Spring Session Ignite plugin

[akurbanov]

Hello Igniters,

I have implemented small community extension to use Apache Ignite as backing
repository for session clustering with Spring Session:  github link
  . I was wondering what
would be the best way to share this with community, is there any Ignite
repository existing for community integrations?

Regards,
anton




--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[MTCGA]: new failures in builds [1917802] needs to be handled

[dpavlov . tasks]

Hi Igniters,

 I've detected some new issue on TeamCity to be handled. You are more than 
welcomed to help.

 If your changes can lead to this failure(s): We're grateful that you were a 
volunteer to make the contribution to this project, but things change and you 
may no longer be able to finalize your contribution.
 Could you respond to this email and indicate if you wish to continue and fix 
test failures or step down and some committer may revert you commit. 

 *New test failure in master 
org.apache.ignite.spark.IgniteOptimizationDateFuncSpec.Supported optimized date 
functions - FORMATDATETIME 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-5031414754271757301=%3Cdefault%3E=testDetails
 Changes may lead to failure were done by 
 - maxmuzaf 
http://ci.ignite.apache.org/viewModification.html?modId=832238=false
 - slava.koptilin 
http://ci.ignite.apache.org/viewModification.html?modId=832230=false
 - sgrimstad 
http://ci.ignite.apache.org/viewModification.html?modId=832213=false
 - dpavlov 
http://ci.ignite.apache.org/viewModification.html?modId=832211=false
 - stanlukyanov 
http://ci.ignite.apache.org/viewModification.html?modId=832210=false
 - pudov.max 
http://ci.ignite.apache.org/viewModification.html?modId=832205=false
 - tledkov 
http://ci.ignite.apache.org/viewModification.html?modId=832200=false
 - sgrimstad 
http://ci.ignite.apache.org/viewModification.html?modId=832194=false
 - slukyanov 
http://ci.ignite.apache.org/viewModification.html?modId=832192=false
 - isapego 
http://ci.ignite.apache.org/viewModification.html?modId=832190=false
 - pivanov 
http://ci.ignite.apache.org/viewModification.html?modId=832180=false

 - Here's a reminder of what contributors were agreed to do 
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute 
 - Should you have any questions please contact dev@ignite.apache.org 

Best Regards,
Apache Ignite TeamCity Bot 
https://github.com/apache/ignite-teamcity-bot
Notification generated at 11:45:22 24-09-2018 

[GitHub] ignite pull request #4745: IGNITE-9411: mvcc timeouts

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/4745


---