[GitHub] ignite pull request #3549: Test run. config option to use off-heap map inste...

[xtern]

Github user xtern closed the pull request at:

https://github.com/apache/ignite/pull/3549


---

[GitHub] ignite pull request #3570: Introduce scala-test profile to hide scalatest-ma...

[nizhikov]

Github user nizhikov closed the pull request at:

https://github.com/apache/ignite/pull/3570


---

[GitHub] ignite pull request #3582: IGNITE-6868 Concurrency performance fix

[alex-plekhanov]

GitHub user alex-plekhanov opened a pull request:

https://github.com/apache/ignite/pull/3582

IGNITE-6868 Concurrency performance fix



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alex-plekhanov/ignite 
ignite-6868-concurrency-fix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3582.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3582


commit 97a606506b877c405f503a78778a2be1d1ead3c9
Author: Aleksey Plekhanov 
Date:   2018-02-28T07:06:46Z

IGNITE-6868 Concurrency performance fix




---

[jira] [Created] (IGNITE-7836) ExchangeFuture misses onBaselineTopologyChanged callback when forceReassignment is false

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-7836:


 Summary: ExchangeFuture misses onBaselineTopologyChanged callback 
when forceReassignment is false
 Key: IGNITE-7836
 URL: https://issues.apache.org/jira/browse/IGNITE-7836
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Alexey Goncharuk
 Fix For: 2.5






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Ignite | Webconsole Dockerfile unifications

[Petr Ivanov]

Hi, Denis.


I guess I’ll be able to cherry-pick my changes to release tag and use resulting 
project state to build and create docker images, however that won’t be 100% 
binary compatible with what was tested and release (uploaded to ASF). Though 
that must not be a problem currently, and at 2.5 I’d like to add docker image 
creation to release procedure.



> On 28 Feb 2018, at 07:20, Denis Magda  wrote:
> 
> Peter,
> 
> Thanks for keep automating and improving Ignite build/release environment!
> Igniters, who can review the changes?
> 
> However, I guess that it's too late to merge the changes into 2.4 branch
> which is frozen for merges.
> 
> Considering this, could you prepare docker as well as cloud images (AWS,
> google compute engine) for 2.4 manually?
> 
> --
> Denis
> 
> 
> On Tue, Feb 27, 2018 at 4:16 AM, vveider  wrote:
> 
>> Hi, Igniters!
>> 
>> 
>> In order to correctly process Apache Ignite 2.4 release, I've implemented
>> several changes in Apache Ignite's and Web Console's docker build procedure
>> [1].
>> My main motivation was to:
>> 1. Unify approach to docker image architecture. Previously, Apache Ignite
>> docker image build procedure was designed for DockerHub autobuild. Yet,
>> there are plans for a) moving Apache Ignite docker image to official docker
>> repository [2] and b) including docker image building into release
>> procedure
>> (+ QA, release binaries are tested before release). Now the architecture is
>> common with Web Console's one - Dockerfile redesigned to build image from
>> locally acquired Apache Ignite's binary archive.
>> 2. Unify approach for container deliveries. I've replaced Web Console's
>> custom build script with instructions about how to obtain necessary files
>> for docker image, thus simplifying it and making more transparent. Now
>> images are constructed from built sources in almost single command.
>> 
>> Need help with review [3] and merge to master and ignite-2.4 branches - so
>> it will be possible to create and upload docker release images without
>> pain.
>> Corresponding documentations is in preparation.
>> 
>> 
>> [1] https://issues.apache.org/jira/browse/IGNITE-7821
>> [2] https://hub.docker.com/explore
>> [3] https://github.com/apache/ignite/pull/3574
>> 
>> 
>> 
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>> 

[jira] [Created] (IGNITE-7835) Scala tests appeared in wrong configurations

[Nikolay Izhikov (JIRA)]

Nikolay Izhikov created IGNITE-7835:
---

 Summary: Scala tests appeared in wrong configurations
 Key: IGNITE-7835
 URL: https://issues.apache.org/jira/browse/IGNITE-7835
 Project: Ignite
  Issue Type: Bug
Reporter: Nikolay Izhikov
Assignee: Nikolay Izhikov
 Fix For: 2.5


Scala tests are executed in wrong configurations.

This happens because scalatest-maven-plugin doesn't properly handle {{-Dtest}} 
settings.

http://apache-ignite-developers.2346864.n4.nabble.com/TeamCity-Ignite-RDD-tests-tp27129p27343.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Thin client / Binary protocol: questions

[Alexey Kosenchuk]

Folks,

Designing node.js thin lib (IGNITE-) we have the following questions.
Could you please have a look and clarify.

Thanks,
-Alexey

- Supported standard types and their type codes.
  Are defined in the spec: 
https://apacheignite.readme.io/docs/binary-client-protocol#data-format
  But, as we see, java and .net libs (common parts which are used by 
thin clients as well) support additional types and type codes, not 
described in the spec. Eg. defined here: 
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/binary/GridBinaryMarshaller.java
  Just to double-check - we should support the types from the spec 
only. Right?


- Key-Value and SQL and Scan Queries.
  The most of operations has "Flag" field in Request: "Pass 0 for 
default, or 1 to keep the value in binary form."

  What is it for?
  Java and .net libs always pass 0. Why?

- OP_QUERY_SCAN: Filter platform.
  Why is it required?
  Why an app working remotely via the binary protocol should know the 
exact platform where the node is running?


- Binary Type operations (register/get type name, put/get type).
  What are they for?
  What is a use case?
  It seems possible to put data of any binary object type into a cache 
w/o registering type name and structure in advance.


- OP_REGISTER_BINARY_TYPE_NAME, OP_GET_BINARY_TYPE_NAME: Platform id.
  Is it a type of the node platform?
  Why a remote app should know it?
  Why non-identical values are used here and in OP_QUERY_SCAN: Filter 
platform?


- OP_REGISTER_BINARY_TYPE_NAME, OP_PUT_BINARY_TYPE
  Why the both - Type name and Java-style hash code of the type name - 
are in the request? (The same for Field name and Java-style hash code of 
the field name.)

  Seems redundant.

- Just interesting - why it was decided that hash code should be also 
calculated on a client side? Why it could not be returned by a server side?
  Eg. OP_CACHE_CREATE_WITH_NAME Response can return hash code for the 
provided Cache name.
  That would simplify thin client implementations, guarantee 
consistency/correctness of hash code calculations...


- Name/password authentication.
  When will it be available in the spec?
  Is there any draft we can look at?

- SSL/TLS communication.
  When will it be available in the spec?
  Is there any draft we can look at?

- As Denis said, we should implement a node failover algorithm 
(switching to another predefined node).

  Should the algorithm be the same in all thin libs?
  Is it described somewhere?
  We have not found it in java and .net libs. Will it be supported by 
them? When?


- Is there any recommendation/suggestion how to test a node failover 
case on a thin client side?


- We are going to use jsdoc comments for the API methods: 
http://usejsdoc.org/
  This tool, for example, may be used to generate the API spec from the 
comments: https://www.npmjs.com/package/jsdoc

  Any objections?

- We are going to write the lib readme/guide in the markdown format and 
place it at the repo (.md file).

  Is it OK? Or should we use dash.readme.io ?

- May we easily use 3rd-party components with the following licenses?
-- Apache 2.0
-- MIT
-- GNU LGPL v3
-- GNU GPL v2
  Should we pre-approve somehow / notify somebody about usage of 
concrete 3rd-party components?

Re: Ignite | Webconsole Dockerfile unifications in 2.4

[Denis Magda]

Peter,

Thanks for keep automating and improving Ignite build/release environment!
Igniters, who can review the changes?

However, I guess that it's too late to merge the changes into 2.4 branch
which is frozen for merges.

Considering this, could you prepare docker as well as cloud images (AWS,
google compute engine) for 2.4 manually?

--
Denis


On Tue, Feb 27, 2018 at 4:16 AM, vveider  wrote:

> Hi, Igniters!
>
>
> In order to correctly process Apache Ignite 2.4 release, I've implemented
> several changes in Apache Ignite's and Web Console's docker build procedure
> [1].
> My main motivation was to:
> 1. Unify approach to docker image architecture. Previously, Apache Ignite
> docker image build procedure was designed for DockerHub autobuild. Yet,
> there are plans for a) moving Apache Ignite docker image to official docker
> repository [2] and b) including docker image building into release
> procedure
> (+ QA, release binaries are tested before release). Now the architecture is
> common with Web Console's one - Dockerfile redesigned to build image from
> locally acquired Apache Ignite's binary archive.
> 2. Unify approach for container deliveries. I've replaced Web Console's
> custom build script with instructions about how to obtain necessary files
> for docker image, thus simplifying it and making more transparent. Now
> images are constructed from built sources in almost single command.
>
> Need help with review [3] and merge to master and ignite-2.4 branches - so
> it will be possible to create and upload docker release images without
> pain.
> Corresponding documentations is in preparation.
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-7821
> [2] https://hub.docker.com/explore
> [3] https://github.com/apache/ignite/pull/3574
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

Re: Next Steps: GA Grid: Request to contribute GA library to Apache Ignite

[Denis Magda]

>
> Based on Denis' request, I understand that Oleg/Yuri will handle merging
> 'pull-request' into main codebase.


Exactly, I'm expecting the same.

--
Denis


On Tue, Feb 27, 2018 at 1:22 PM, techbysample  wrote:

> Denis,
>
> Good to hear!  I am glad that GA Grid will officially become apart of
> Apache
> Ignite.
>
> Based on Denis' request, I understand that Oleg/Yuri will handle merging
> 'pull-request' into main codebase.
>
> Let me know if there is any action for me to take at this time.
>
> Also, Denis, I will be reaching out to you very soon to discuss next steps.
>
> Best,
> Turik Campbell
> NetMillennium, Inc.
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>

[GitHub] ignite pull request #3581: IGNITE-5357 replicated cache reads load balancing...

[DALukjanenko]

GitHub user DALukjanenko opened a pull request:

https://github.com/apache/ignite/pull/3581

IGNITE-5357 replicated cache reads load balancing was added



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/DALukjanenko/ignite IGNITE-5357.master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3581.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3581


commit 5cc0906838f4916060149a62b51d6904f63a11a7
Author: Dmitry Lukjanenko 
Date:   2018-02-28T00:14:36Z

IGNITE-5357 replicated cache reads load balancing was added




---

[GitHub] ignite pull request #3580: IGNITE-6868 Concurrency performance fix

[alex-plekhanov]

GitHub user alex-plekhanov opened a pull request:

https://github.com/apache/ignite/pull/3580

IGNITE-6868 Concurrency performance fix



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alex-plekhanov/ignite ignite-6868-perf-fix

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3580.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3580


commit b3544e263b9c22917fdf039441102668e5627cad
Author: Aleksey Plekhanov 
Date:   2018-02-27T15:35:43Z

IGNITE-6868 Concurrency performance fix

commit 515b478d493a42d3c871a65057878da0f0dfbfb8
Author: Aleksey Plekhanov 
Date:   2018-02-27T22:06:12Z

IGNITE-6868 WIP

commit d6127866d8e300a6de22fe37be5d7ba725790427
Author: Aleksey Plekhanov 
Date:   2018-02-27T22:13:56Z

IGNITE-6868 Concurrency performance fix




---

Re: Next Steps: GA Grid: Request to contribute GA library to Apache Ignite

[techbysample]

Denis,

Good to hear!  I am glad that GA Grid will officially become apart of Apache
Ignite.

Based on Denis' request, I understand that Oleg/Yuri will handle merging
'pull-request' into main codebase.

Let me know if there is any action for me to take at this time.

Also, Denis, I will be reaching out to you very soon to discuss next steps.

Best,
Turik Campbell
NetMillennium, Inc.



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [TeamCity] DataFrame Examples

[Dmitry Pavlov]

I prefer to use existing profile. We have a lot of suites, and it is hard
to manage and monitor.

вт, 27 февр. 2018 г. в 23:02, Nikolay Izhikov :

> I've tested it locally so yes, Spark examples can be tested in Ignite
> Examples.
>
> We need to enable `scala` profile for test plan and add
> `IgniteExamplesSparkSelfTestSuite` to Team City properties.
>
> Dmitry, please, give me advice.
> Should I edit Ignite Examples settings or should I create copy of test
> plan to adjust settings?
>
>
>
> В Вт, 27/02/2018 в 19:54 +, Dmitry Pavlov пишет:
> > Nikolay, Val,
> >
> > Could these examples be tested in standard Ignite Examples test suite? As
> > far as I know Ignite ML and Ignite IGFS are also tested there.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > вт, 27 февр. 2018 г. в 22:49, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com>:
> >
> > > Nikolay,
> > >
> > > Spark integration is not related to Scalar, the only thing they have in
> > > common is Scala. I think we should have a separate configuration for
> > > ignite-spark module. If anything Spark related is currently in Scalar
> > > suite, it should be moved from there.
> > >
> > > -Val
> > >
> > > On Tue, Feb 27, 2018 at 3:03 AM, Nikolay Izhikov 
> > > wrote:
> > >
> > > > Hello, Igniters.
> > > >
> > > > I've created several examples for a Ignite DataFrame.
> > > > Currently, they are not tested on Team City.
> > > >
> > > > As far as I can understand, they should be checked in "Ignite Scalar
> > > > Examples [Scala]".
> > > > But it marked as obsolete [1]
> > > >
> > > > Please, give me advice:
> > > >
> > > > 1. Should we test Ignite DataFrame examples on Team City?
> > > >
> > > > 2. Which plan should be used for that?
> > > >
> > > > [1]
> > >
> > > https://ci.ignite.apache.org/viewLog.html?buildId=1068099=
> > > > IgniteTestsForCodeCoverage_IgniteScalarExamples=buildParameters

Re: [TeamCity] DataFrame Examples

[Nikolay Izhikov]

I've tested it locally so yes, Spark examples can be tested in Ignite Examples.

We need to enable `scala` profile for test plan and add 
`IgniteExamplesSparkSelfTestSuite` to Team City properties.

Dmitry, please, give me advice.
Should I edit Ignite Examples settings or should I create copy of test plan to 
adjust settings?



В Вт, 27/02/2018 в 19:54 +, Dmitry Pavlov пишет:
> Nikolay, Val,
> 
> Could these examples be tested in standard Ignite Examples test suite? As
> far as I know Ignite ML and Ignite IGFS are also tested there.
> 
> Sincerely,
> Dmitriy Pavlov
> 
> вт, 27 февр. 2018 г. в 22:49, Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
> 
> > Nikolay,
> > 
> > Spark integration is not related to Scalar, the only thing they have in
> > common is Scala. I think we should have a separate configuration for
> > ignite-spark module. If anything Spark related is currently in Scalar
> > suite, it should be moved from there.
> > 
> > -Val
> > 
> > On Tue, Feb 27, 2018 at 3:03 AM, Nikolay Izhikov 
> > wrote:
> > 
> > > Hello, Igniters.
> > > 
> > > I've created several examples for a Ignite DataFrame.
> > > Currently, they are not tested on Team City.
> > > 
> > > As far as I can understand, they should be checked in "Ignite Scalar
> > > Examples [Scala]".
> > > But it marked as obsolete [1]
> > > 
> > > Please, give me advice:
> > > 
> > > 1. Should we test Ignite DataFrame examples on Team City?
> > > 
> > > 2. Which plan should be used for that?
> > > 
> > > [1]
> > 
> > https://ci.ignite.apache.org/viewLog.html?buildId=1068099=
> > > IgniteTestsForCodeCoverage_IgniteScalarExamples=buildParameters

signature.asc
Description: This is a digitally signed message part

Re: [TeamCity] DataFrame Examples

[Valentin Kulichenko]

If we can run Scala examples there, then yes.

-Val

On Tue, Feb 27, 2018 at 11:54 AM, Dmitry Pavlov 
wrote:

> Nikolay, Val,
>
> Could these examples be tested in standard Ignite Examples test suite? As
> far as I know Ignite ML and Ignite IGFS are also tested there.
>
> Sincerely,
> Dmitriy Pavlov
>
> вт, 27 февр. 2018 г. в 22:49, Valentin Kulichenko <
> valentin.kuliche...@gmail.com>:
>
> > Nikolay,
> >
> > Spark integration is not related to Scalar, the only thing they have in
> > common is Scala. I think we should have a separate configuration for
> > ignite-spark module. If anything Spark related is currently in Scalar
> > suite, it should be moved from there.
> >
> > -Val
> >
> > On Tue, Feb 27, 2018 at 3:03 AM, Nikolay Izhikov 
> > wrote:
> >
> > > Hello, Igniters.
> > >
> > > I've created several examples for a Ignite DataFrame.
> > > Currently, they are not tested on Team City.
> > >
> > > As far as I can understand, they should be checked in "Ignite Scalar
> > > Examples [Scala]".
> > > But it marked as obsolete [1]
> > >
> > > Please, give me advice:
> > >
> > > 1. Should we test Ignite DataFrame examples on Team City?
> > >
> > > 2. Which plan should be used for that?
> > >
> > > [1]
> > https://ci.ignite.apache.org/viewLog.html?buildId=1068099=
> > > IgniteTestsForCodeCoverage_IgniteScalarExamples=buildParameters
> >
>

Re: [TeamCity] DataFrame Examples

[Dmitry Pavlov]

Nikolay, Val,

Could these examples be tested in standard Ignite Examples test suite? As
far as I know Ignite ML and Ignite IGFS are also tested there.

Sincerely,
Dmitriy Pavlov

вт, 27 февр. 2018 г. в 22:49, Valentin Kulichenko <
valentin.kuliche...@gmail.com>:

> Nikolay,
>
> Spark integration is not related to Scalar, the only thing they have in
> common is Scala. I think we should have a separate configuration for
> ignite-spark module. If anything Spark related is currently in Scalar
> suite, it should be moved from there.
>
> -Val
>
> On Tue, Feb 27, 2018 at 3:03 AM, Nikolay Izhikov 
> wrote:
>
> > Hello, Igniters.
> >
> > I've created several examples for a Ignite DataFrame.
> > Currently, they are not tested on Team City.
> >
> > As far as I can understand, they should be checked in "Ignite Scalar
> > Examples [Scala]".
> > But it marked as obsolete [1]
> >
> > Please, give me advice:
> >
> > 1. Should we test Ignite DataFrame examples on Team City?
> >
> > 2. Which plan should be used for that?
> >
> > [1]
> https://ci.ignite.apache.org/viewLog.html?buildId=1068099=
> > IgniteTestsForCodeCoverage_IgniteScalarExamples=buildParameters
>

[GitHub] ignite pull request #3573: IGNITE-7816: DataFrame examples fixed.

[nizhikov]

Github user nizhikov closed the pull request at:

https://github.com/apache/ignite/pull/3573


---

Re: [TeamCity] DataFrame Examples

[Valentin Kulichenko]

Nikolay,

Spark integration is not related to Scalar, the only thing they have in
common is Scala. I think we should have a separate configuration for
ignite-spark module. If anything Spark related is currently in Scalar
suite, it should be moved from there.

-Val

On Tue, Feb 27, 2018 at 3:03 AM, Nikolay Izhikov 
wrote:

> Hello, Igniters.
>
> I've created several examples for a Ignite DataFrame.
> Currently, they are not tested on Team City.
>
> As far as I can understand, they should be checked in "Ignite Scalar
> Examples [Scala]".
> But it marked as obsolete [1]
>
> Please, give me advice:
>
> 1. Should we test Ignite DataFrame examples on Team City?
>
> 2. Which plan should be used for that?
>
> [1] https://ci.ignite.apache.org/viewLog.html?buildId=1068099=
> IgniteTestsForCodeCoverage_IgniteScalarExamples=buildParameters

Re: .NET suite failures

[Dmitry Pavlov]

Hi Pavel, yes, you're right. All tests passed.

I thought it was because of 'IGNITE-7329 .NET: Thin client: SSL connection
implemented' and did not look in the log

вт, 27 февр. 2018 г. в 19:22, Pavel Tupitsyn :

> Hi Dmitry,
>
> Thanks for bringing this up.
> Please see the build log, it indicates failed Java build:
>
> Failed to execute goal
> org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
> (default) on project ignite-storm: Error resolving project artifact: Could
> not transfer artifact ring-cors:ring-cors:pom:0.1.5 from/to central (
> http://repo.maven.apache.org/maven2): repo.maven.apache.org for project
> ring-cors:ring-cors:jar:0.1.5: Unknown host repo.maven.apache.org -> [Help
> 1]
>
> Looks like a Maven connection issue to me, let's wait for the next build.
>
> Pavel
>
> On Tue, Feb 27, 2018 at 7:13 PM, Dmitry Pavlov 
> wrote:
>
> > Hi Pavel, Igniters,
> >
> > .NET test failed after recent changes
> >
> >
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_
> > IgnitePlatformNet_IgniteTests24Java8=%3Cdefault%
> > 3E=buildTypeStatusDiv
> >
> > Failed suites are following:
> >Ignite Platform .NET [ tests 0 TC_EXIT_CODE ]
> >Ignite Platform .NET Long Running [ tests 0 TC_EXIT_CODE ]
> >
> > Is it related to recent commits to .NET ?
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
>

[jira] [Created] (IGNITE-7834) Ignite Queries 2 fail rate is more than 95%: DynamicColumnsAbstractConcurrentSelfTest and its subclasses are flaky

[Dmitriy Pavlov (JIRA)]

Dmitriy Pavlov created IGNITE-7834:
--

 Summary: Ignite Queries 2 fail rate is more than 95%: 
DynamicColumnsAbstractConcurrentSelfTest and its subclasses are flaky
 Key: IGNITE-7834
 URL: https://issues.apache.org/jira/browse/IGNITE-7834
 Project: Ignite
  Issue Type: Test
  Components: sql
Reporter: Dmitriy Pavlov


Queries 2 suite has unstable tests DynamicColumns...
Initially contrubuted by [~al.psc] in [IGNITE-5572]

https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_IgniteQueries2=%3Cdefault%3E=buildTypeStatusDiv

Last 10 runs statistics:
{noformat}
Ignite Queries 2 [ tests 16 ]
 [2] IgniteBinaryCacheQueryTestSuite2: 
DynamicColumnsConcurrentTransactionalPartitionedSelfTest.testNodeJoinOnPendingAddOperation
 (fail rate 5,8%) 
 IgniteBinaryCacheQueryTestSuite2: 
DynamicColumnsConcurrentTransactionalReplicatedSelfTest.testAddColumnCoordinatorChange
 (fail rate 5,8%) 
 IgniteBinaryCacheQueryTestSuite2: 
DynamicColumnsConcurrentTransactionalReplicatedSelfTest.testNodeJoinOnPendingAddOperation
 (fail rate 5,8%) 
 [2] IgniteBinaryCacheQueryTestSuite2: 
DynamicColumnsConcurrentAtomicPartitionedSelfTest.testDropColumnCoordinatorChange
 (fail rate 3,9%) 
 IgniteBinaryCacheQueryTestSuite2: 
DynamicColumnsConcurrentAtomicPartitionedSelfTest.testNodeJoinOnPendingDropOperation
 (fail rate 3,9%) 
 IgniteBinaryCacheQueryTestSuite2: 
DynamicColumnsConcurrentAtomicReplicatedSelfTest.testNodeJoinOnPendingAddOperation
 (fail rate 3,9%) 
 IgniteBinaryCacheQueryTestSuite2: 
DynamicIndexPartitionedAtomicConcurrentSelfTest.testClientReconnectWithCacheRestart
 (fail rate 3,6%) 
 IgniteBinaryCacheQueryTestSuite2: 
DynamicColumnsConcurrentTransactionalPartitionedSelfTest.testConcurrentOperationsAndNodeStartStopMultithreaded
 (fail rate 2,9%) 
 IgniteBinaryCacheQueryTestSuite2: 
DynamicColumnsConcurrentAtomicReplicatedSelfTest.testDropConcurrentCacheDestroy 
(fail rate 2,9%) 
 IgniteBinaryCacheQueryTestSuite2: 
DynamicColumnsConcurrentTransactionalPartitionedSelfTest.testAddConcurrentRebalance
 (fail rate 2,9%) 
 IgniteBinaryCacheQueryTestSuite2: 
DynamicIndexReplicatedTransactionalConcurrentSelfTest.testClientReconnectWithCacheRestart
 (fail rate 2,5%) 
 IgniteBinaryCacheQueryTestSuite2: 
DynamicColumnsConcurrentAtomicReplicatedSelfTest.testConcurrentPutRemove (fail 
rate 2,2%) 
 IgniteBinaryCacheQueryTestSuite2: 
DynamicColumnsConcurrentTransactionalReplicatedSelfTest.testDropConcurrentRebalance
 (fail rate 1,9%) 
 IgniteBinaryCacheQueryTestSuite2: 
DynamicColumnsConcurrentTransactionalPartitionedSelfTest.testClientReconnectWithCacheRestart
 (fail rate 1,8%) 
 IgniteBinaryCacheQueryTestSuite2: 
DynamicColumnsConcurrentTransactionalPartitionedSelfTest.testDropConcurrentCacheDestroy
 (fail rate 1,0%) 
IgniteBinaryCacheQueryTestSuite2: 
DynamicIndexReplicatedTransactionalConcurrentSelfTest.testClientReconnect (fail 
rate 0,4%) 
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3579: IGNITE-7467 Partition counters validation

[Jokser]

GitHub user Jokser opened a pull request:

https://github.com/apache/ignite/pull/3579

IGNITE-7467 Partition counters validation



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7467

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3579.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3579


commit d52d7a5395b86c3f35ba793751e3f0aa2cb35b8a
Author: Pavel Kovalenko 
Date:   2018-01-19T15:53:02Z

IGNITE-7467 WIP

commit cbb4a26f99bc006fe459799b84d01a9001a6ac4e
Author: Pavel Kovalenko 
Date:   2018-02-27T16:22:13Z

IGNITE-7467 Validate partition update counters and cache sizes.

commit 92588393b29f57ac27666b48efe859ae7ebb752b
Author: Pavel Kovalenko 
Date:   2018-02-27T16:22:19Z

Merge branch 'master' into ignite-7467

commit d9739847e2420a2ddfea00a42f03b261fe16c6f7
Author: Pavel Kovalenko 
Date:   2018-02-27T16:23:57Z

IGNITE-7467 Cleaned up test.

commit 1029c84cfec3bbb8922cbcda408d0703477aed82
Author: Pavel Kovalenko 
Date:   2018-02-27T16:28:22Z

IGNITE-7467 Missed java docs.




---

Re: .NET suite failures

[Pavel Tupitsyn]

Hi Dmitry,

Thanks for bringing this up.
Please see the build log, it indicates failed Java build:

Failed to execute goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
(default) on project ignite-storm: Error resolving project artifact: Could
not transfer artifact ring-cors:ring-cors:pom:0.1.5 from/to central (
http://repo.maven.apache.org/maven2): repo.maven.apache.org for project
ring-cors:ring-cors:jar:0.1.5: Unknown host repo.maven.apache.org -> [Help
1]

Looks like a Maven connection issue to me, let's wait for the next build.

Pavel

On Tue, Feb 27, 2018 at 7:13 PM, Dmitry Pavlov 
wrote:

> Hi Pavel, Igniters,
>
> .NET test failed after recent changes
>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_
> IgnitePlatformNet_IgniteTests24Java8=%3Cdefault%
> 3E=buildTypeStatusDiv
>
> Failed suites are following:
>Ignite Platform .NET [ tests 0 TC_EXIT_CODE ]
>Ignite Platform .NET Long Running [ tests 0 TC_EXIT_CODE ]
>
> Is it related to recent commits to .NET ?
>
> Sincerely,
> Dmitriy Pavlov
>

[jira] [Created] (IGNITE-7833) Find out possible ways to handle partition update counters inconsistency

[Pavel Kovalenko (JIRA)]

Pavel Kovalenko created IGNITE-7833:
---

 Summary: Find out possible ways to handle partition update 
counters inconsistency
 Key: IGNITE-7833
 URL: https://issues.apache.org/jira/browse/IGNITE-7833
 Project: Ignite
  Issue Type: Improvement
  Components: cache
Reporter: Pavel Kovalenko


We should think about possible ways to resolve the situation when we observe 
that partition update counters for the same partitions (primary-backup) are 
different on some nodes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: TeamCity. Ignite RDD tests

[Dmitry Pavlov]

Hi Nikolay,

IMO you can merge. Just check if following suites work correct and still
execute its tests:

Ignite Scalar Examples [Scala]
Ignite Visor Console [Scala]

Sincerely,
Dmitriy Pavlov

вт, 27 февр. 2018 г. в 19:15, Nikolay Izhikov :

> Hello,
>
> If there are no any objections I merge the fix tomorrow.
>
> https://github.com/apache/ignite/pull/3570
>
> В Вт, 27/02/2018 в 13:27 +0300, Nikolay Izhikov пишет:
> > Hello, Alexey, what do you think?
> >
> > Let's fix this issue!
> >
> > В Пн, 26/02/2018 в 16:29 +, Dmitry Pavlov пишет:
> > > I agree.
> > >
> > > Alexey K, what do you think?
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > пн, 26 февр. 2018 г. в 19:17, Nikolay Izhikov :
> > >
> > > > Hello, Dmitriy.
> > > >
> > > > scalatest-maven-plugin doesn't handle "test" property.
> > > > So when "scala" profile enabled it will execute all Scala Spark
> tests for
> > > > any test plan.
> > > >
> > > > I propose to hide scalatest plugin declaration into profile
> "scala-test"
> > > > [1]
> > > >
> > > > And enable it only for specific team city plans.
> > > > I.e "Ignite RDD" and "Ignite RDD spark 2_10".
> > > >
> > > > Thoughts?
> > > >
> > > > [1] https://github.com/apache/ignite/pull/3570
> > > >
> > > > В Пн, 26/02/2018 в 14:35 +0300, Nikolay Izhikov пишет:
> > > > > Hello, Dmitriy.
> > > > >
> > > > > I'm investigate issue and share results later today.
> > > > >
> > > > > В Пн, 26/02/2018 в 11:31 +, Dmitry Pavlov пишет:
> > > > > > Hi Nikolay,
> > > > > >
> > > > > > Are there any news about these tests apperance in suites?
> > > > > >
> > > > > > It seems I've found one more example in
> > > > > >
> > > >
> > > >
> https://ci.ignite.apache.org/viewLog.html?buildId=1106598=buildResultsDiv=IgniteTests24Java8_IgniteOSGi
> > > > > > (other
> > > > > > change PR run). I'm not sure that OSGI build configuration is
> > > >
> > > > appropriate
> > > > > > suite to run these tests.
> > > > > >
> > > > > > Sincerely,
> > > > > > Dmitriy Pavlov
> > > > > >
> > > > > > чт, 22 февр. 2018 г. в 16:13, Nikolay Izhikov <
> nizhi...@apache.org>:
> > > > > >
> > > > > > > Dmitriy.
> > > > > > >
> > > > > > > OK, I get it.
> > > > > > >
> > > > > > > Looks like scala-test-plugin I added in IGNITE-7042 doesn't
> handle
> > > > > > > `-Dtest=XXX` properly.
> > > > > > > I will investigate and fixed it.
> > > > > > >
> > > > > > > В Чт, 22/02/2018 в 10:28 +, Dmitry Pavlov пишет:
> > > > > > > > It seems that tests I've previously muted were started in
> wrong
> > > >
> > > > configs.
> > > > > > > >
> > > > > > > > Is it possible IgniteRDDSpec.IgniteRDD  became unit tests,
> which is
> > > > > > >
> > > > > > > started
> > > > > > > > with code compile?
> > > > > > > >
> > > > > > > > This can be reason these tests appeared in suites which is
> not
> > > >
> > > > related to
> > > > > > > > IgniteRDD, such as Scala Examples.
> > > > > > > >
> > > > > > > > чт, 22 февр. 2018 г. в 13:20, Nikolay Izhikov <
> nizhi...@apache.org
> > > > >
> > > > > :
> > > > > > > >
> > > > > > > > > Hello, Dmitriy
> > > > > > > > >
> > > > > > > > > I'm looked in TeamCity but not sure what should I do.
> > > > > > > > > Can you explain it to me, please?
> > > > > > > > >
> > > > > > > > > В Чт, 22/02/2018 в 10:13 +, Dmitry Pavlov пишет:
> > > > > > > > > > Hi Nikolay,
> > > > > > > > > >
> > > > > > > > > > It seems
> > > > > > > > > > IgniteRDDSpec.IgniteRDD  has appeared in configurations
> it is
> > > >
> > > > not
> > > > > > > > >
> > > > > > > > > expected
> > > > > > > > > > to run.
> > > > > > > > > >
> > > > > > > > > > Please see investigations assigned to you. What can we
> change
> > > >
> > > > to
> > > > > > >
> > > > > > > avoid
> > > > > > > > > > these tests to be included into run other suites?
> > > > > > > > > >
> > > > > > > > > > Sincerely,
> > > > > > > > > > Dmitriy Pavlov
> > > > > > > > > >
> > > > > > > > > > пт, 16 февр. 2018 г. в 7:35, Nikolay Izhikov <
> > > >
> > > > nizhi...@apache.org>:
> > > > > > > > > >
> > > > > > > > > > > Hello, Igniters.
> > > > > > > > > > >
> > > > > > > > > > > I'm working on issue [1].
> > > > > > > > > > >
> > > > > > > > > > > Team City doesn't collect info about scalatest
> execution
> > > >
> > > > because of
> > > > > > > > >
> > > > > > > > > wrong
> > > > > > > > > > > pom.xml
> > > > > > > > > > > I'm fixed it in PR [3]
> > > > > > > > > > >
> > > > > > > > > > > It happens there is a 2 broken tests written in Scala
> - [4]
> > > > > > > > > > >
> > > > > > > > > > > 1. IgniteRDDSpec.IgniteRDD should successfully store
> data to
> > > >
> > > > ignite
> > > > > > > > >
> > > > > > > > > using
> > > > > > > > > > > saveValues
> > > > > > > > > > > 2. IgniteRDDSpec.IgniteRDD should successfully store
> data to
> > > >
> > > > ignite
> > > > > > > > >
> > > > > > > > > using
> > > > > > > > > > > saveValues with inline transformation
> > > > > > > > > > >
> > > > > > > > > > > It seems they are been here 

Re: TeamCity. Ignite RDD tests

[Nikolay Izhikov]

Hello, 

If there are no any objections I merge the fix tomorrow.

https://github.com/apache/ignite/pull/3570

В Вт, 27/02/2018 в 13:27 +0300, Nikolay Izhikov пишет:
> Hello, Alexey, what do you think?
> 
> Let's fix this issue!
> 
> В Пн, 26/02/2018 в 16:29 +, Dmitry Pavlov пишет:
> > I agree.
> > 
> > Alexey K, what do you think?
> > 
> > Sincerely,
> > Dmitriy Pavlov
> > 
> > пн, 26 февр. 2018 г. в 19:17, Nikolay Izhikov :
> > 
> > > Hello, Dmitriy.
> > > 
> > > scalatest-maven-plugin doesn't handle "test" property.
> > > So when "scala" profile enabled it will execute all Scala Spark tests for
> > > any test plan.
> > > 
> > > I propose to hide scalatest plugin declaration into profile "scala-test"
> > > [1]
> > > 
> > > And enable it only for specific team city plans.
> > > I.e "Ignite RDD" and "Ignite RDD spark 2_10".
> > > 
> > > Thoughts?
> > > 
> > > [1] https://github.com/apache/ignite/pull/3570
> > > 
> > > В Пн, 26/02/2018 в 14:35 +0300, Nikolay Izhikov пишет:
> > > > Hello, Dmitriy.
> > > > 
> > > > I'm investigate issue and share results later today.
> > > > 
> > > > В Пн, 26/02/2018 в 11:31 +, Dmitry Pavlov пишет:
> > > > > Hi Nikolay,
> > > > > 
> > > > > Are there any news about these tests apperance in suites?
> > > > > 
> > > > > It seems I've found one more example in
> > > > > 
> > > 
> > > https://ci.ignite.apache.org/viewLog.html?buildId=1106598=buildResultsDiv=IgniteTests24Java8_IgniteOSGi
> > > > > (other
> > > > > change PR run). I'm not sure that OSGI build configuration is
> > > 
> > > appropriate
> > > > > suite to run these tests.
> > > > > 
> > > > > Sincerely,
> > > > > Dmitriy Pavlov
> > > > > 
> > > > > чт, 22 февр. 2018 г. в 16:13, Nikolay Izhikov :
> > > > > 
> > > > > > Dmitriy.
> > > > > > 
> > > > > > OK, I get it.
> > > > > > 
> > > > > > Looks like scala-test-plugin I added in IGNITE-7042 doesn't handle
> > > > > > `-Dtest=XXX` properly.
> > > > > > I will investigate and fixed it.
> > > > > > 
> > > > > > В Чт, 22/02/2018 в 10:28 +, Dmitry Pavlov пишет:
> > > > > > > It seems that tests I've previously muted were started in wrong
> > > 
> > > configs.
> > > > > > > 
> > > > > > > Is it possible IgniteRDDSpec.IgniteRDD  became unit tests, which 
> > > > > > > is
> > > > > > 
> > > > > > started
> > > > > > > with code compile?
> > > > > > > 
> > > > > > > This can be reason these tests appeared in suites which is not
> > > 
> > > related to
> > > > > > > IgniteRDD, such as Scala Examples.
> > > > > > > 
> > > > > > > чт, 22 февр. 2018 г. в 13:20, Nikolay Izhikov  > > > 
> > > > :
> > > > > > > 
> > > > > > > > Hello, Dmitriy
> > > > > > > > 
> > > > > > > > I'm looked in TeamCity but not sure what should I do.
> > > > > > > > Can you explain it to me, please?
> > > > > > > > 
> > > > > > > > В Чт, 22/02/2018 в 10:13 +, Dmitry Pavlov пишет:
> > > > > > > > > Hi Nikolay,
> > > > > > > > > 
> > > > > > > > > It seems
> > > > > > > > > IgniteRDDSpec.IgniteRDD  has appeared in configurations it is
> > > 
> > > not
> > > > > > > > 
> > > > > > > > expected
> > > > > > > > > to run.
> > > > > > > > > 
> > > > > > > > > Please see investigations assigned to you. What can we change
> > > 
> > > to
> > > > > > 
> > > > > > avoid
> > > > > > > > > these tests to be included into run other suites?
> > > > > > > > > 
> > > > > > > > > Sincerely,
> > > > > > > > > Dmitriy Pavlov
> > > > > > > > > 
> > > > > > > > > пт, 16 февр. 2018 г. в 7:35, Nikolay Izhikov <
> > > 
> > > nizhi...@apache.org>:
> > > > > > > > > 
> > > > > > > > > > Hello, Igniters.
> > > > > > > > > > 
> > > > > > > > > > I'm working on issue [1].
> > > > > > > > > > 
> > > > > > > > > > Team City doesn't collect info about scalatest execution
> > > 
> > > because of
> > > > > > > > 
> > > > > > > > wrong
> > > > > > > > > > pom.xml
> > > > > > > > > > I'm fixed it in PR [3]
> > > > > > > > > > 
> > > > > > > > > > It happens there is a 2 broken tests written in Scala - [4]
> > > > > > > > > > 
> > > > > > > > > > 1. IgniteRDDSpec.IgniteRDD should successfully store data to
> > > 
> > > ignite
> > > > > > > > 
> > > > > > > > using
> > > > > > > > > > saveValues
> > > > > > > > > > 2. IgniteRDDSpec.IgniteRDD should successfully store data to
> > > 
> > > ignite
> > > > > > > > 
> > > > > > > > using
> > > > > > > > > > saveValues with inline transformation
> > > > > > > > > > 
> > > > > > > > > > It seems they are been here for a while.
> > > > > > > > > > I propose to mute or disable it on TeamCity before merging
> > > 
> > > my PR.
> > > > > > > > > > I've created ticket for fixing tests - [5].
> > > > > > > > > > 
> > > > > > > > > > Thoughts?
> > > > > > > > > > 
> > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-7042
> > > > > > > > > > 
> > > > > > > > > > [2]
> > > > > > > > > > 
> > > > > > > > 
> > > > > > > > 
> > > > > > 
> > > > > > 
> > > 
> > > 

.NET suite failures

[Dmitry Pavlov]

Hi Pavel, Igniters,

.NET test failed after recent changes

https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_IgnitePlatformNet_IgniteTests24Java8=%3Cdefault%3E=buildTypeStatusDiv

Failed suites are following:
   Ignite Platform .NET [ tests 0 TC_EXIT_CODE ]
   Ignite Platform .NET Long Running [ tests 0 TC_EXIT_CODE ]

Is it related to recent commits to .NET ?

Sincerely,
Dmitriy Pavlov

[GitHub] ignite pull request #3578: neineighborhoods selection logic was added

[daradurvs]

GitHub user daradurvs opened a pull request:

https://github.com/apache/ignite/pull/3578

neineighborhoods selection logic was added



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/daradurvs/ignite ignite-5357

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3578.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3578


commit fc6cdf8cff5236254c9f3d4805561ca0baa8518d
Author: Vyacheslav Daradur 
Date:   2018-02-27T15:58:38Z

ignite-5357: neineighborhoods selection logic was added




---

[jira] [Created] (IGNITE-7832) Partition lost events looks broken.

[Andrew Mashenkov (JIRA)]

Andrew Mashenkov created IGNITE-7832:


 Summary: Partition lost events looks broken.
 Key: IGNITE-7832
 URL: https://issues.apache.org/jira/browse/IGNITE-7832
 Project: Ignite
  Issue Type: Task
  Components: cache
Reporter: Andrew Mashenkov
 Fix For: 2.5


Steps to reproduce:
 # Start 3 nodes with Partitioned cache with no backups.
 Register event handlers for EVT_CACHE_REBALANCE_PART_DATA_LOST event. 
PartitionLossPolicy should be smth other then IGNORED (which is default).
 # Put some data and make sure rebalance is finished.
 # Kill first node. 
I observer partition lost events on 2-nd node only however, some of them should 
be owned by 3-rd node according new topology. 
Cache.lostPartitions() returns non-empty resultset on 2-nd node only.
 # Kill 2-nd node. 
No partition lost events will be fired on 3-rd node. However data is obviously 
lost.
Cache.lostPartitions() returns empty resultset.



Looks like partition lost events mechanics is broken and should be fixed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7831) Throw Exceptions instead of AssertionErrors when reading from corrupted persistence

[Ilya Lantukh (JIRA)]

Ilya Lantukh created IGNITE-7831:


 Summary: Throw Exceptions instead of AssertionErrors when reading 
from corrupted persistence
 Key: IGNITE-7831
 URL: https://issues.apache.org/jira/browse/IGNITE-7831
 Project: Ignite
  Issue Type: Improvement
Reporter: Ilya Lantukh






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [IGNITE-5714] Design proposal of suspend/resume for pessimistic tx

[Nikolay Izhikov]

Hello, Alexey.

Great mail, by the way.
I think, it would be great to have this feature in Ignite.

> I haven't removed thread id completely from code.

Can we remove thread id completely from code?
Can you estimate how much effort do we need?

As far as I can see from parent task [1] we need some complex tests to be 
implemented.
Are they presented in prototype?

[1] 
https://issues.apache.org/jira/browse/IGNITE-4887?focusedCommentId=16069655=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16069655

В Пн, 26/02/2018 в 10:59 +, Aleksey Kuznetsov пишет:
> Hello Igniters!
> 
> Currently we have suspension/resuming implemented for optimistic
> transactions [1].
> Unless suspend/resume isn't supported for pessimistic tx JTA isn't fully
> supported [4].
> 
> I’m working on a ticket "Suspension/resuming for pessimistic transactions"
> [2].
> Goal of the ticket is to support transaction suspend/resume for pessimistic
> transactions.
> 
> # Benefits of suspending/resuming transaction.
> 
>   1. Full JTA standart support.
>   2. Increase of throughput in high-load scenarios.
>   Suspend operation would allow to release Ignite threads and
> optionally perform some other logic.
> 
> Note, current API has got suspend/resume only for optimistic transactions,
> which confuses users.
> 
> # Real life example.
> 
> Consider the following scenario:
> 
>   1. Application starts Ignite transaction.
>   2. Business logic is executed inside transaction.
>   3. For commit/rollback application need approval message from external
> agent.
>   4. Currently, thread inside Ignite is idle until approval is received.
>   4a. When suspend/resume support is implemented, application can perform
> suspend and release thread inside Ignite.
> 
> # How pessimistic transaction works.
> 
> When we perform put/get operations in pessimistic transactions, lock
> request is sent to remote nodes by `GridNearLockRequest`.
> Request contains thread id `IgniteTxAdapter#threadId`, in which operation
> was performed.
> In pessimistic mode, multiple transaction objects are created - on
> primary, on backup nodes, and on originating node:
> `GridNearTxLocal`, `GridDhtTxLocal`, `GridNearTxRemote`, `GridDhtTxRemote`.
> 
> Thread id is used in logic on these nodes.
> For instance, to check whether thread has successfully locked the key,
> after lock acquisition attempt.
> Or to check whether active transaction exists.
> 
> # Main challenge for implementation.
> 
> I've analysed implementation approaches and see the core issue:
> 
> The essential problem with suspending/resuming lies in thread id field
> transferred to remote nodes during put/get operations.
> 
> Imagine, we want to suspend transaction and resume it in another thread.
> See code snippet below:
> 
> ```
> tx = ignite.transactions().txStart(PESSIMISTIC, SERIALIZABLE);
> 
> cache.put(1, 1);
> 
> tx.suspend();
> 
> 
> // In another thread.
> tx.resume(); // Thread id will be changed in transaction instance.
> ```
> 
> Original thread id is transferred and saved on remote node.
> After resuming thread id on local node differs from remote node.
> I want to avoid one more network round trip to change thread Id on remote
> node after transaction resuming.
> 
> # Design proposal.
> 
> Transaction id (`xid`) can be used instead of thread id on remote nodes.
> The following solution is possible for the ticket :
> 
> Replace thread id by transaction id for sending to remote nodes.
> Thread id will be removed from the following classes:
> `IgniteTxAdapter`, `GridDistributedTxPrepareRequest`,
> `GridDistributedTxFinishRequest`, `GridDistributedTxFinishResponse`,
> `GridDistributedTxPrepareRequest`.
> 
> I haven't removed thread id completely from code. Thread id is moved to
> `GridNearTxLocal`.
> We still need it in near local transaction for many reasons, for example to
> assure only thread started transaction can suspend it in
> `GridNearTxLocal#suspend()`.
> In future we can remove thread id completely. I propose to study this
> question in another ticket.
> 
> Also, thread id is remained in `GridDistributedLockRequest`.
> Lock request used by cache locks and it need to transfer thread id to
> remote nodes.
> For example to use cache locks along with cache operations put/get, see
> `GridNearTxLocal#updateExplicitVersion`.
> As for pessimistic transaction, thread id in `GridDistributedLockRequest`
> is set to `UNDEFINED_THREAD_ID`, which means we must not use it remotely.
> 
> Note, that if user suspends transaction and forgets to resume it,
> transaction would be rolled back once timeout has occurred.
> 
> In my design when transaction is suspended, all locked keys remain locked.
> 
> Please see my prototype of proposal implementation [3].
> 
> Proposed changes are relatively small.
> They ensure consistency of information about locks, if thread Id will be
> changed within one transaction (by suspend/resume).
> There will be used correct id for locks on remote 

[jira] [Created] (IGNITE-7830) Adopt kNN model to the new Partitioned Dataset

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-7830:


 Summary: Adopt kNN model to the new Partitioned Dataset
 Key: IGNITE-7830
 URL: https://issues.apache.org/jira/browse/IGNITE-7830
 Project: Ignite
  Issue Type: Sub-task
  Components: ml
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7828) Adopt yardstick tests for the new version of kNN regression algorithm

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-7828:


 Summary: Adopt yardstick tests for the new version of kNN 
regression algorithm
 Key: IGNITE-7828
 URL: https://issues.apache.org/jira/browse/IGNITE-7828
 Project: Ignite
  Issue Type: Sub-task
  Components: ml
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7829) Adopt kNN regression example to the new Partitioned Dataset

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-7829:


 Summary: Adopt kNN regression example to the new Partitioned 
Dataset
 Key: IGNITE-7829
 URL: https://issues.apache.org/jira/browse/IGNITE-7829
 Project: Ignite
  Issue Type: Sub-task
  Components: ml
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7827) Adopt kNN regression to the new Partitioned Dataset

[Aleksey Zinoviev (JIRA)]

Aleksey Zinoviev created IGNITE-7827:


 Summary: Adopt kNN regression to the new Partitioned Dataset
 Key: IGNITE-7827
 URL: https://issues.apache.org/jira/browse/IGNITE-7827
 Project: Ignite
  Issue Type: Improvement
  Components: ml
Reporter: Aleksey Zinoviev
Assignee: Aleksey Zinoviev






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Stop nodes after test by default - IGNITE-6842

[Maxim Muzafarov]

Yep, link may not be correct.

Here is correct version:
TC: 
*https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8_IgniteTests24Java8=pull%2F3542%2Fhead
*


вт, 27 февр. 2018 г. в 16:41, Dmitry Pavlov :

> Hi Maxim,
>
> could you please provide link to TC run on your PR? It seems link provided
> points to run of master. In changes field you may select pull/3542/head
> before starting RunAll.
>
> Igniters,
>
> this change is related to our test framework, so change may affect your
> tests. Please join to review
> https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
>
> Sincerely,
> Dmitriy Pavlov
>
> вт, 27 февр. 2018 г. в 16:14, Maxim Muzafarov :
>
> > Hi all,
> >
> > I think, I've done with this issue, what should we do next?
> >
> > PR: https://github.com/apache/ignite/pull/3542
> > Upsource: https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
> > TC:
> >
> >
> https://ci.ignite.apache.org/viewModification.html?modId=723895=false==vcsModificationTests
> > JIRA: https://issues.apache.org/jira/browse/IGNITE-6842
> >
> >
> > чт, 22 февр. 2018 г. в 14:12, Dmitry Pavlov :
> >
> > > Hi Maxim,
> > >
> > > Thank you.
> > >
> > > To my mind stopAllGrids call should be kept in afterTestsStop(). If
> > > developer forgot to call super(), he will see exception from
> > > beforeTestsStart()
> > > added by you.
> > >
> > > If you think patch is ready to be reviewed, please change JIRA status
> to
> > > "Patch Available".
> > >
> > > Optionally you can create Upsource review. It would be easier for
> > multiple
> > > reviewers to handle this patch. This test framework is used by all
> > Igniters
> > > so Upsource would be good option (https://reviews.ignite.apache.org/
> ).
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > чт, 22 февр. 2018 г. в 13:44, Maxim Muzafarov :
> > >
> > > > Hi all,
> > > >
> > > > I've made some changes based on our previous discusstions, please see
> > PR
> > > > [1]:
> > > > 1) Remove duplicated code for stopGrid (by index and by name);
> > > > 2) Add new method that thows exception if not all grids haven't
> stopped
> > > > correctly;
> > > > 3)  Change tests that have been affected by this changes;
> > > >
> > > > Also, I have some thoughts for clarification:
> > > > 1) beforeTestsStart() - I expect here in subtests that grids are not
> > > > started yet. Am I right?
> > > > 2) I think we should call stopAllGrids in tearDown method. So, if in
> > some
> > > > cases we'll override afterTestsStop in subclasses and forgot to call
> > > > super() it won't lead us to exception.
> > > >
> > > > [1] https://github.com/apache/ignite/pull/3542
> > > > [2] https://ci.ignite.apache.org/viewModification.html?modId=717275
> > > > [3] https://issues.apache.org/jira/browse/IGNITE-6842
> > > >
> > > >
> > > > ср, 7 февр. 2018 г. в 18:28, Maxim Muzafarov :
> > > >
> > > > > Thank you all,
> > > > >
> > > > > I'll add this comment's for JIRA ticket, if you don't mind.
> > > > >
> > > > > ср, 7 февр. 2018 г. в 15:16, Dmitry Pavlov  >:
> > > > >
> > > > >> Yes, this solution allows to cover both cases:
> > > > >> a) not stopped node from previous test and
> > > > >> b) allows to remove useless code that stops Ignite nodes from each
> > > test.
> > > > >>
> > > > >> ср, 7 февр. 2018 г. в 15:13, Anton Vinogradov <
> > > avinogra...@gridgain.com
> > > > >:
> > > > >>
> > > > >> > Maxim,
> > > > >> >
> > > > >> > We discussed with Dima privately, and decided
> > > > >> >
> > > > >> > 1) We have to assert that there is no alive nodes at
> > > > GridAbstractTest's
> > > > >> > beforeTestsStarted
> > > > >> > 2) We have to kill all alive nodes (without force) at
> > > > GridAbstractTest's
> > > > >> > afterTestsStopped
> > > > >> > 3) In case of any exceptions at #2 we have to see test error
> > > > >> > 4) We can get rid of all useless stopAllGrids at
> > GridAbstractTest's
> > > > >> > subclasses.
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > On Wed, Feb 7, 2018 at 2:32 PM, Dmitry Pavlov <
> > > dpavlov@gmail.com>
> > > > >> > wrote:
> > > > >> >
> > > > >> > > > Let's just add stopAllGrids(flase) to GridAbstractTest 's
> > > > >> > > afterTestsStopped
> > > > >> > > method body.
> > > > >> > >
> > > > >> > > Can't agree with it becase implicit silent shutdown of nodes
> > from
> > > > test
> > > > >> > > framework may cause a lot of bugs introduced to Ignite.
> > > > >> > >
> > > > >> > > I think stop+test fail should occur.
> > > > >> > > In that case author of incorrect test or not consistent Ignite
> > > > >> shutdown
> > > > >> > > will see problem.
> > > > >> > >
> > > > >> > > 'Fail fast' if always better than hidding real problem under
> > > > automatic
> > > > >> > > framework feature.
> > > > >> > >
> > > > >> > > ср, 

[GitHub] ignite pull request #3577: IGNITE-7796: Adopt kNN classification example to ...

[zaleslaw]

GitHub user zaleslaw opened a pull request:

https://github.com/apache/ignite/pull/3577

IGNITE-7796: Adopt kNN classification example to the new datasets



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7796

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3577.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3577


commit ae9741f36d360c2deada2804baa2cec1fb0254ea
Author: zaleslaw 
Date:   2018-02-14T14:42:16Z

Added draft sample

commit 531b336d569456fdee11a424878bd119b3b63c28
Author: Zinoviev Alexey 
Date:   2018-02-22T11:42:05Z

IGNITE-7702: Added tests

commit ad71203ca565f9ef688bea1924bc77052818e236
Author: Zinoviev Alexey 
Date:   2018-02-22T17:06:51Z

IGNITE-7702: Fixed bug

commit fd8f976f3d42066139b0359709fb8ea6eded2f0f
Author: Zinoviev Alexey 
Date:   2018-02-22T17:09:32Z

IGNITE-7702: remove incorrect examples

commit 4622dc6df0bf48c800dc413a24278d227e96f6aa
Author: Zinoviev Alexey 
Date:   2018-02-22T17:10:17Z

IGNITE-7702: Rewrite KNN classifications

commit 3a661d9e0627b8696c580d4f3587f51b30800166
Author: Zinoviev Alexey 
Date:   2018-02-22T17:11:28Z

IGNITE-7702: Delete incorrect benchmarks

commit c0e2edfbd3893ad693a72462315f4b7c5bd61ac6
Author: Zinoviev Alexey 
Date:   2018-02-22T17:36:15Z

IGNITE-7702: Fixed tests

commit cee126f3a2eedbe88166111d11aa6d985444b924
Author: Zinoviev Alexey 
Date:   2018-02-27T13:41:37Z

IGNITE-7702: Fixed licenses

commit a7b8e8ad773e422ff43d484fa62e8ef58553a802
Author: Zinoviev Alexey 
Date:   2018-02-27T13:48:45Z

Merge branch 'master' into ignite-7702

commit c63ab415bad489efc93e43538d866fcc5780b642
Author: Zinoviev Alexey 
Date:   2018-02-27T13:49:25Z

IGNITE-7702: Fixed merged conflicts

commit c06a7a00f735e71d659cd6794a6817f9a9fba0e2
Author: Zinoviev Alexey 
Date:   2018-02-27T14:02:55Z

IGNITE-7796: Added example




---

[jira] [Created] (IGNITE-7826) DStatProbe benchmark empty result output

[Burov Ilya (JIRA)]

Burov Ilya created IGNITE-7826:
--

 Summary: DStatProbe benchmark empty result output
 Key: IGNITE-7826
 URL: https://issues.apache.org/jira/browse/IGNITE-7826
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 2.3
 Environment: Ubuntu 17.10

apache-ignite-fabric-2.3.0-bin

Dstat 0.7.3
Platform posix/linux2
Kernel 4.13.0-36-generic
Python 2.7.14 (default, Sep 23 2017, 22:06:14) 
[GCC 7.2.0]

Terminal type: xterm-256color (color support)
Reporter: Burov Ilya
 Attachments: DStatProbe.csv, dstat.log

DStatProbe creates empty result csv report due to dstat command output parsing 
error.

Sample csv file and part of driver log included.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Stop nodes after test by default - IGNITE-6842

[Dmitry Pavlov]

Hi Maxim,

could you please provide link to TC run on your PR? It seems link provided
points to run of master. In changes field you may select pull/3542/head
before starting RunAll.

Igniters,

this change is related to our test framework, so change may affect your
tests. Please join to review
https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502

Sincerely,
Dmitriy Pavlov

вт, 27 февр. 2018 г. в 16:14, Maxim Muzafarov :

> Hi all,
>
> I think, I've done with this issue, what should we do next?
>
> PR: https://github.com/apache/ignite/pull/3542
> Upsource: https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
> TC:
>
> https://ci.ignite.apache.org/viewModification.html?modId=723895=false==vcsModificationTests
> JIRA: https://issues.apache.org/jira/browse/IGNITE-6842
>
>
> чт, 22 февр. 2018 г. в 14:12, Dmitry Pavlov :
>
> > Hi Maxim,
> >
> > Thank you.
> >
> > To my mind stopAllGrids call should be kept in afterTestsStop(). If
> > developer forgot to call super(), he will see exception from
> > beforeTestsStart()
> > added by you.
> >
> > If you think patch is ready to be reviewed, please change JIRA status to
> > "Patch Available".
> >
> > Optionally you can create Upsource review. It would be easier for
> multiple
> > reviewers to handle this patch. This test framework is used by all
> Igniters
> > so Upsource would be good option (https://reviews.ignite.apache.org/ ).
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > чт, 22 февр. 2018 г. в 13:44, Maxim Muzafarov :
> >
> > > Hi all,
> > >
> > > I've made some changes based on our previous discusstions, please see
> PR
> > > [1]:
> > > 1) Remove duplicated code for stopGrid (by index and by name);
> > > 2) Add new method that thows exception if not all grids haven't stopped
> > > correctly;
> > > 3)  Change tests that have been affected by this changes;
> > >
> > > Also, I have some thoughts for clarification:
> > > 1) beforeTestsStart() - I expect here in subtests that grids are not
> > > started yet. Am I right?
> > > 2) I think we should call stopAllGrids in tearDown method. So, if in
> some
> > > cases we'll override afterTestsStop in subclasses and forgot to call
> > > super() it won't lead us to exception.
> > >
> > > [1] https://github.com/apache/ignite/pull/3542
> > > [2] https://ci.ignite.apache.org/viewModification.html?modId=717275
> > > [3] https://issues.apache.org/jira/browse/IGNITE-6842
> > >
> > >
> > > ср, 7 февр. 2018 г. в 18:28, Maxim Muzafarov :
> > >
> > > > Thank you all,
> > > >
> > > > I'll add this comment's for JIRA ticket, if you don't mind.
> > > >
> > > > ср, 7 февр. 2018 г. в 15:16, Dmitry Pavlov :
> > > >
> > > >> Yes, this solution allows to cover both cases:
> > > >> a) not stopped node from previous test and
> > > >> b) allows to remove useless code that stops Ignite nodes from each
> > test.
> > > >>
> > > >> ср, 7 февр. 2018 г. в 15:13, Anton Vinogradov <
> > avinogra...@gridgain.com
> > > >:
> > > >>
> > > >> > Maxim,
> > > >> >
> > > >> > We discussed with Dima privately, and decided
> > > >> >
> > > >> > 1) We have to assert that there is no alive nodes at
> > > GridAbstractTest's
> > > >> > beforeTestsStarted
> > > >> > 2) We have to kill all alive nodes (without force) at
> > > GridAbstractTest's
> > > >> > afterTestsStopped
> > > >> > 3) In case of any exceptions at #2 we have to see test error
> > > >> > 4) We can get rid of all useless stopAllGrids at
> GridAbstractTest's
> > > >> > subclasses.
> > > >> >
> > > >> >
> > > >> >
> > > >> > On Wed, Feb 7, 2018 at 2:32 PM, Dmitry Pavlov <
> > dpavlov@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> > > > Let's just add stopAllGrids(flase) to GridAbstractTest 's
> > > >> > > afterTestsStopped
> > > >> > > method body.
> > > >> > >
> > > >> > > Can't agree with it becase implicit silent shutdown of nodes
> from
> > > test
> > > >> > > framework may cause a lot of bugs introduced to Ignite.
> > > >> > >
> > > >> > > I think stop+test fail should occur.
> > > >> > > In that case author of incorrect test or not consistent Ignite
> > > >> shutdown
> > > >> > > will see problem.
> > > >> > >
> > > >> > > 'Fail fast' if always better than hidding real problem under
> > > automatic
> > > >> > > framework feature.
> > > >> > >
> > > >> > > ср, 7 февр. 2018 г. в 14:05, Anton Vinogradov <
> > > >> avinogra...@gridgain.com
> > > >> > >:
> > > >> > >
> > > >> > > > Hi all,
> > > >> > > >
> > > >> > > > > I've made some research about this problem and i think that
> in
> > > >> > general
> > > >> > > we
> > > >> > > > > should move stopAllGrids method in GridAbstractTest class to
> > > >> > > > > afterTestsStopped method with some changes. Am I right?
> > > >> > > > Let's just add stopAllGrids(flase) to GridAbstractTest 's
> > > >> > > > afterTestsStopped method
> > > >> > > > body.
> > > >> > > >
> > > >> > > > > I have a question about stopAllGrids(boolean 

[GitHub] ignite pull request #3576: Ignite gg 13533

[dkarachentsev]

GitHub user dkarachentsev opened a pull request:

https://github.com/apache/ignite/pull/3576

Ignite gg 13533



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-13533

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3576.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3576


commit 3e2ccfd30427ba0552eea8667c0129ae5ace9c0b
Author: Igor Sapego 
Date:   2016-11-25T11:26:54Z

IGNITE-4299: Fixes for examples.

commit 6fbaef45af8f40062a95058df7ec0984c99035b9
Author: Konstantin Dudkov 
Date:   2016-11-25T10:58:58Z

IGNITE-4305 marshalling fix in GridNearAtomicSingleUpdateInvokeRequest

commit 1a2de51f5807a91ce0d5dff28f24ed5bf7abebbc
Author: Konstantin Dudkov 
Date:   2016-11-28T09:59:02Z

IGNITE-4305 marshalling fix

commit c06e4017771603df7118974758d3d6b9cadc41b5
Author: Eduard Shangareev 
Date:   2016-11-30T11:34:47Z

ignite-4332 Usage of cache.getEntry inside GridCacheQueryManager.runQuery 
causes to remote operations

commit 066691098797be8c01daa0e8dc2ba94d4ad73561
Author: sboikov 
Date:   2016-12-01T14:16:28Z

ignite-4344 Do not create offheap map on client nodes.

commit e9ace7730773a6d4a1d30b271854f1fe8a7ba632
Author: Alexey Kuznetsov 
Date:   2016-12-02T09:06:41Z

Improved exception handling.

commit 12bdd6a031a33eda004a66e73cee9628f073ed68
Author: Alexey Kuznetsov 
Date:   2016-12-02T09:09:29Z

Updated classnames.properties for running nodes in IDE.

commit 33dda46061aae73e5c138851cfdd5f49a1c254cb
Author: sboikov 
Date:   2016-12-02T09:13:38Z

ignite-4285 For serializable txs allow multiple threads to get read lock 
for the same key

commit cc13d9d155f70e22e08ef203ac64e5cc0aa0a50f
Author: dkarachentsev 
Date:   2016-11-28T08:30:14Z

IGNITE-4026: Fixed BinaryObjectBuilder.build() can fail if one of the 
fields is Externalizable, enum from binary object. This closes #1281. This 
closes #1289.

(cherry picked from commit 0b7c62d)

commit b4aedfd5350b4a318f1608596a171789513835a4
Author: Igor Sapego 
Date:   2016-12-02T17:10:09Z

IGNITE-4347: Fixed NPE.

commit acf20b32d8fb68e42b904b091fb2b943f4558cef
Author: sboikov 
Date:   2016-12-05T11:01:28Z

ignite-4296 Optimize GridDhtPartitionsSingleMessage processing
- optimized processing of GridDhtPartitionsSingleMessage
- minor optimizations for RendezvousAffinityFunction
 - fixed heartbeats sending in tcp discovery

commit 6ba1711a1fa10d8276974227491136070c3ed43a
Author: Anton Vinogradov 
Date:   2016-12-06T09:55:41Z

IGNITE-4242 ExchangeManager should wait for cache rebalancing in async way

commit 0d4a1b7381fece47ee480f3a06bff7c51a7fead4
Author: Alexey Kuznetsov 
Date:   2016-12-07T11:02:49Z

Improved exception handling for failed queries.

commit ac8602dbdf2bbf5b16a611eaf6d520a0a7b0010b
Author: Sergi Vladykin 
Date:   2016-08-15T13:46:54Z

ignite-3685 - fixed

commit bbaa79af8ef526b5d2684db0e3d71d60a8f1ebe7
Author: agura 
Date:   2016-12-07T16:36:11Z

IGNITE-3770 GridLogThrottle.warn ignores the exception

commit 18598574bb2992aa193eed1d72ca333a1e21ad72
Author: Andrey V. Mashenkov 
Date:   2016-12-08T09:36:07Z

GG-11746: Backport of IGNITE-4379:  Fixed broken local SqlFieldsQuery.

commit 671a77a2d81cac401765dddf25f30fba4e4ab17f
Author: sboikov 
Date:   2016-12-08T09:56:46Z

ignite-4154 Fixed node version check for compression feature usage

commit 391f4be4c687a7f325aeec8b727c9c85ca003454
Author: agura 
Date:   2016-12-07T17:11:50Z

ignite-2358 toString() method for cache store implementations

commit bc977d3211906ef94e1f7d3f0f988efbed65034f
Author: Alexey Kuznetsov 
Date:   2016-12-09T09:11:31Z

IGNITE-4350 Reworked JdbcTypesDefaultTransformed logic. Improved improved 
error messages in CacheJdbcPojoStore.

commit b83ec8e57c7c48f2baa4780cf3b2e46df075f3df
Author: sboikov 
Date:   2016-12-09T11:32:42Z

IGNITE-500 CacheLoadingConcurrentGridStartSelfTest fails: prevent 
'localUpdate' execution while top read lock is held.

commit 6e485637e2738a7e809eac1a802f0964dc12383d
Author: Andrey V. Mashenkov 
Date:   2016-12-09T12:42:40Z

IGNITE-4379: Fixed local query execution. This closes #1323.

commit 6fd8bf6338470275e687a686044c7d935d3714ff
Author: Andrey V. Mashenkov 
Date:   2016-12-07T15:49:06Z

Fixed tests for IGNITE-4379 commit.

commit c143bc1a77baa13f61d6ba00509fa1fcb33757b1
Author: tledkov-gridgain 
Date:   2016-12-09T13:05:03Z

IGNITE-4063:  

[jira] [Created] (IGNITE-7825) ODBC: Update specification documentation

[Igor Sapego (JIRA)]

Igor Sapego created IGNITE-7825:
---

 Summary: ODBC: Update specification documentation
 Key: IGNITE-7825
 URL: https://issues.apache.org/jira/browse/IGNITE-7825
 Project: Ignite
  Issue Type: Task
  Components: documentation
Affects Versions: 2.3
Reporter: Igor Sapego
Assignee: Igor Sapego
 Fix For: 2.4


Need to update the following doc 
[https://apacheignite-sql.readme.io/v2.3/docs/specification] according to the 
newly implemented features in 2.4.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Stop nodes after test by default - IGNITE-6842

[Maxim Muzafarov]

Hi all,

I think, I've done with this issue, what should we do next?

PR: https://github.com/apache/ignite/pull/3542
Upsource: https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
TC:
https://ci.ignite.apache.org/viewModification.html?modId=723895=false==vcsModificationTests
JIRA: https://issues.apache.org/jira/browse/IGNITE-6842


чт, 22 февр. 2018 г. в 14:12, Dmitry Pavlov :

> Hi Maxim,
>
> Thank you.
>
> To my mind stopAllGrids call should be kept in afterTestsStop(). If
> developer forgot to call super(), he will see exception from
> beforeTestsStart()
> added by you.
>
> If you think patch is ready to be reviewed, please change JIRA status to
> "Patch Available".
>
> Optionally you can create Upsource review. It would be easier for multiple
> reviewers to handle this patch. This test framework is used by all Igniters
> so Upsource would be good option (https://reviews.ignite.apache.org/ ).
>
> Sincerely,
> Dmitriy Pavlov
>
> чт, 22 февр. 2018 г. в 13:44, Maxim Muzafarov :
>
> > Hi all,
> >
> > I've made some changes based on our previous discusstions, please see PR
> > [1]:
> > 1) Remove duplicated code for stopGrid (by index and by name);
> > 2) Add new method that thows exception if not all grids haven't stopped
> > correctly;
> > 3)  Change tests that have been affected by this changes;
> >
> > Also, I have some thoughts for clarification:
> > 1) beforeTestsStart() - I expect here in subtests that grids are not
> > started yet. Am I right?
> > 2) I think we should call stopAllGrids in tearDown method. So, if in some
> > cases we'll override afterTestsStop in subclasses and forgot to call
> > super() it won't lead us to exception.
> >
> > [1] https://github.com/apache/ignite/pull/3542
> > [2] https://ci.ignite.apache.org/viewModification.html?modId=717275
> > [3] https://issues.apache.org/jira/browse/IGNITE-6842
> >
> >
> > ср, 7 февр. 2018 г. в 18:28, Maxim Muzafarov :
> >
> > > Thank you all,
> > >
> > > I'll add this comment's for JIRA ticket, if you don't mind.
> > >
> > > ср, 7 февр. 2018 г. в 15:16, Dmitry Pavlov :
> > >
> > >> Yes, this solution allows to cover both cases:
> > >> a) not stopped node from previous test and
> > >> b) allows to remove useless code that stops Ignite nodes from each
> test.
> > >>
> > >> ср, 7 февр. 2018 г. в 15:13, Anton Vinogradov <
> avinogra...@gridgain.com
> > >:
> > >>
> > >> > Maxim,
> > >> >
> > >> > We discussed with Dima privately, and decided
> > >> >
> > >> > 1) We have to assert that there is no alive nodes at
> > GridAbstractTest's
> > >> > beforeTestsStarted
> > >> > 2) We have to kill all alive nodes (without force) at
> > GridAbstractTest's
> > >> > afterTestsStopped
> > >> > 3) In case of any exceptions at #2 we have to see test error
> > >> > 4) We can get rid of all useless stopAllGrids at GridAbstractTest's
> > >> > subclasses.
> > >> >
> > >> >
> > >> >
> > >> > On Wed, Feb 7, 2018 at 2:32 PM, Dmitry Pavlov <
> dpavlov@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > > Let's just add stopAllGrids(flase) to GridAbstractTest 's
> > >> > > afterTestsStopped
> > >> > > method body.
> > >> > >
> > >> > > Can't agree with it becase implicit silent shutdown of nodes from
> > test
> > >> > > framework may cause a lot of bugs introduced to Ignite.
> > >> > >
> > >> > > I think stop+test fail should occur.
> > >> > > In that case author of incorrect test or not consistent Ignite
> > >> shutdown
> > >> > > will see problem.
> > >> > >
> > >> > > 'Fail fast' if always better than hidding real problem under
> > automatic
> > >> > > framework feature.
> > >> > >
> > >> > > ср, 7 февр. 2018 г. в 14:05, Anton Vinogradov <
> > >> avinogra...@gridgain.com
> > >> > >:
> > >> > >
> > >> > > > Hi all,
> > >> > > >
> > >> > > > > I've made some research about this problem and i think that in
> > >> > general
> > >> > > we
> > >> > > > > should move stopAllGrids method in GridAbstractTest class to
> > >> > > > > afterTestsStopped method with some changes. Am I right?
> > >> > > > Let's just add stopAllGrids(flase) to GridAbstractTest 's
> > >> > > > afterTestsStopped method
> > >> > > > body.
> > >> > > >
> > >> > > > > I have a question about stopAllGrids(boolean cancel) this
> > "cancel"
> > >> > > > That's just a flag means "do not wait for any operations finish"
> > >> > > > See no reason to set it true in that case.
> > >> > > >
> > >> > > > > If you delegate closing to afterTestsStopped this will affect
> > only
> > >> > > > > last test (method).
> > >> > > > The idea is to stop all nodes at last test's method finish.
> > >> > > >
> > >> > > > >  Nodes that survive between tests can affect successive
> > >> > > > tests.
> > >> > > > Thats ok. we have a lot tests where nodes reusable between
> test's
> > >> > > methods.
> > >> > > >
> > >> > > >
> > >> > > > On Wed, Feb 7, 2018 at 1:20 PM, Dmitry Pavlov <
> > >> dpavlov@gmail.com>
> > >> > > > wrote:
> > >> > > 

[IGNITE-7789] Ignite Client Nodes testErrorOnDisconnect()

[Maxim Muzafarov]

Hi all,

I just briefly look at this issue [1] and found that for:

1) CollectionConfiguration[2] we have default  backup = 0;
2) AtomicConfiguration[3] we have DFLT_BACKUPS = 1;

Is this correct? Because, both of them used for testing
dataStructureOperationsTest(). After disconnet and recreation cache
GridCacheAtomicLongImpl the default value for backup used.

We have doTestIgniteOperationOnDisconnect() and it's argument of operations
List> which calls asynch. So thats why we
have unstable test-case.

OK - test case
[2018-02-27 15:51:05,065][INFO ][async-callable-runner-1][root] Queue
creation
[2018-02-27 15:51:05,065][INFO ][async-callable-runner-1][root] Set creation
[2018-02-27 15:51:05,065][INFO ][async-callable-runner-1][root] Atomic
creation

FAIL - test case
2018-02-27 15:52:23,304][INFO ][async-callable-runner-1][root] Atomic
creation
[2018-02-27 15:52:23,304][INFO ][async-callable-runner-1][root] Queue
creation
[2018-02-27 15:52:23,303][INFO ][async-callable-runner-1][root] Set creation

Can i take this issue for mysefl?

[1] https://issues.apache.org/jira/browse/IGNITE-7789
[2]
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/configuration/CollectionConfiguration.java#L47
[3]
https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/configuration/AtomicConfiguration.java#L32

[GitHub] ignite pull request #3575: IGNITE-7646 OSGI tests fail

[daradurvs]

GitHub user daradurvs opened a pull request:

https://github.com/apache/ignite/pull/3575

IGNITE-7646 OSGI tests fail



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/daradurvs/ignite ignite-7646

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3575.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3575


commit 9f8c6d7d9528c66e8ee0dcb82cb8b59fb1fc45ff
Author: Vyacheslav Daradur 
Date:   2018-02-27T12:15:49Z

ignite-7646: fix property name




---

Ignite | Webconsole Dockerfile unifications in 2.4

[vveider]

Hi, Igniters!


In order to correctly process Apache Ignite 2.4 release, I've implemented
several changes in Apache Ignite's and Web Console's docker build procedure
[1].
My main motivation was to:
1. Unify approach to docker image architecture. Previously, Apache Ignite
docker image build procedure was designed for DockerHub autobuild. Yet,
there are plans for a) moving Apache Ignite docker image to official docker
repository [2] and b) including docker image building into release procedure
(+ QA, release binaries are tested before release). Now the architecture is
common with Web Console's one - Dockerfile redesigned to build image from
locally acquired Apache Ignite's binary archive.
2. Unify approach for container deliveries. I've replaced Web Console's
custom build script with instructions about how to obtain necessary files
for docker image, thus simplifying it and making more transparent. Now
images are constructed from built sources in almost single command.

Need help with review [3] and merge to master and ignite-2.4 branches - so
it will be possible to create and upload docker release images without pain.
Corresponding documentations is in preparation.


[1] https://issues.apache.org/jira/browse/IGNITE-7821
[2] https://hub.docker.com/explore
[3] https://github.com/apache/ignite/pull/3574



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: IgniteSet implementation: changes required

[Andrey Kuznetsov]

As far as I know, Pavel P. is working on fixing existing sets currently.

As for {{asSet}} cache adapter, I filed the ticket [1].

[1] https://issues.apache.org/jira/browse/IGNITE-7823

2018-02-27 11:20 GMT+03:00 Vladimir Ozerov :

> I think the root issue is that we are trying to mix different cases in a
> single solution. What is the common usage patterns of sets?
> 1) Small mostly-read sets - current implementation is ideal for them -
> everything is available locally, on-heap and in deserialized form
> 2) Big data sets - IgniteCache API is the best candidate here; we can
> simply add a method "Set IgniteCache.asSet()" which will return thin
> wrapper for Set interface around cache. An you will have recovery and
> heap-resistance with almost no additional efforts.
> There is no need to choose between one or another solution. We can support
> both.
>
> As far as current igniteSet implementation - all issues described above
> will go away if you use approach suggested by Dmitry. I do not see a need
> for any major changes.
>
> Vladimir.
>

-- 
Best regards,
  Andrey Kuznetsov.

[jira] [Created] (IGNITE-7824) Rollback part of IGNITE-7170 changes

[Alexey Popov (JIRA)]

Alexey Popov created IGNITE-7824:


 Summary: Rollback part of IGNITE-7170 changes
 Key: IGNITE-7824
 URL: https://issues.apache.org/jira/browse/IGNITE-7824
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 2.4
Reporter: Alexey Popov


Please rollback changes done by mistake:

U.quietAndWarn(log, "Nodes started on local machine require 
more than 20% of physical RAM what can " +

back to:

U.quietAndWarn(log, "Nodes started on local machine require 
more than 80% of physical RAM what can " +




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3574: IGNITE-7821 Update Apache Ignite and Web Console ...

[vveider]

GitHub user vveider opened a pull request:

https://github.com/apache/ignite/pull/3574

IGNITE-7821 Update Apache Ignite and Web Console Dockerfile for 2.4 version 
and local build

 * removed Apache Ignite's Docker module obsolete autobuild directory layout
 * updated Apache Ignite's Dockerfile for local image build and use of 
lighter base image
 * removed Web Console's build script - replaced with pre-build instructions
 * introduced minor refactoring to Web Console's Dockerfile
 * fixed .dockerignore file with correct paths

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7821

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3574.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3574


commit ca9077dc6f56e62ef3dfe251d5e3477da929074a
Author: Ivanov Petr 
Date:   2018-02-27T09:10:57Z

IGNITE-7821 Update Apache Ignite and Web Console Dockerfile for 2.4 version 
and local build
 * removed Apache Ignite's Docker module obsolete autobuild directory layout
 * updated Apache Ignite's Dockerfile for local image build and use of 
lighter base image
 * removed Web Console's build script - replaced with pre-build instructions
 * introduced minor refactoring to Web Console's Dockerfile
 * fixed .dockerignore file with correct paths




---

[jira] [Created] (IGNITE-7823) Enrich IgniteCache with asSet method

[Andrey Kuznetsov (JIRA)]

Andrey Kuznetsov created IGNITE-7823:


 Summary: Enrich IgniteCache with asSet method
 Key: IGNITE-7823
 URL: https://issues.apache.org/jira/browse/IGNITE-7823
 Project: Ignite
  Issue Type: New Feature
  Components: data structures
Reporter: Andrey Kuznetsov
 Fix For: 2.5


Existing {{IgniteSet}} datastructure is good enough for small sets. For big 
sets it's too expensive to maintain redundant onheap data copies. Thus we'd 
better to add new {{IgniteCache::asSet}} method returning set adapter to 
existing cache. The difference between these two kinds of sets should be 
properly documented afterwards. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[TeamCity] DataFrame Examples

[Nikolay Izhikov]

Hello, Igniters.

I've created several examples for a Ignite DataFrame.
Currently, they are not tested on Team City.

As far as I can understand, they should be checked in "Ignite Scalar Examples 
[Scala]".
But it marked as obsolete [1]

Please, give me advice:

1. Should we test Ignite DataFrame examples on Team City? 

2. Which plan should be used for that?

[1] 
https://ci.ignite.apache.org/viewLog.html?buildId=1068099=IgniteTestsForCodeCoverage_IgniteScalarExamples=buildParameters

signature.asc
Description: This is a digitally signed message part

[GitHub] ignite pull request #3573: IGNITE-7816: DataFrame examples fixed.

[nizhikov]

GitHub user nizhikov opened a pull request:

https://github.com/apache/ignite/pull/3573

IGNITE-7816: DataFrame examples fixed.

 moved to `org.apache.ignite.scalar.examples.spark` package. Minor examples 
improvements.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nizhikov/ignite IGNITE-7816

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3573.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3573


commit 7c89b4b2d1bc03c0790862826d83777ba31fe0df
Author: Nikolay Izhikov 
Date:   2018-02-27T10:26:39Z

IGNITE-7816: DataFrame examples moved to 
`org.apache.ignite.scalar.examples.spark` package. Minor examples improvements.




---

Re: TeamCity. Ignite RDD tests

[Nikolay Izhikov]

Hello, Alexey, what do you think?

Let's fix this issue!

В Пн, 26/02/2018 в 16:29 +, Dmitry Pavlov пишет:
> I agree.
> 
> Alexey K, what do you think?
> 
> Sincerely,
> Dmitriy Pavlov
> 
> пн, 26 февр. 2018 г. в 19:17, Nikolay Izhikov :
> 
> > Hello, Dmitriy.
> > 
> > scalatest-maven-plugin doesn't handle "test" property.
> > So when "scala" profile enabled it will execute all Scala Spark tests for
> > any test plan.
> > 
> > I propose to hide scalatest plugin declaration into profile "scala-test"
> > [1]
> > 
> > And enable it only for specific team city plans.
> > I.e "Ignite RDD" and "Ignite RDD spark 2_10".
> > 
> > Thoughts?
> > 
> > [1] https://github.com/apache/ignite/pull/3570
> > 
> > В Пн, 26/02/2018 в 14:35 +0300, Nikolay Izhikov пишет:
> > > Hello, Dmitriy.
> > > 
> > > I'm investigate issue and share results later today.
> > > 
> > > В Пн, 26/02/2018 в 11:31 +, Dmitry Pavlov пишет:
> > > > Hi Nikolay,
> > > > 
> > > > Are there any news about these tests apperance in suites?
> > > > 
> > > > It seems I've found one more example in
> > > > 
> > 
> > https://ci.ignite.apache.org/viewLog.html?buildId=1106598=buildResultsDiv=IgniteTests24Java8_IgniteOSGi
> > > > (other
> > > > change PR run). I'm not sure that OSGI build configuration is
> > 
> > appropriate
> > > > suite to run these tests.
> > > > 
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > > 
> > > > чт, 22 февр. 2018 г. в 16:13, Nikolay Izhikov :
> > > > 
> > > > > Dmitriy.
> > > > > 
> > > > > OK, I get it.
> > > > > 
> > > > > Looks like scala-test-plugin I added in IGNITE-7042 doesn't handle
> > > > > `-Dtest=XXX` properly.
> > > > > I will investigate and fixed it.
> > > > > 
> > > > > В Чт, 22/02/2018 в 10:28 +, Dmitry Pavlov пишет:
> > > > > > It seems that tests I've previously muted were started in wrong
> > 
> > configs.
> > > > > > 
> > > > > > Is it possible IgniteRDDSpec.IgniteRDD  became unit tests, which is
> > > > > 
> > > > > started
> > > > > > with code compile?
> > > > > > 
> > > > > > This can be reason these tests appeared in suites which is not
> > 
> > related to
> > > > > > IgniteRDD, such as Scala Examples.
> > > > > > 
> > > > > > чт, 22 февр. 2018 г. в 13:20, Nikolay Izhikov  > > 
> > > :
> > > > > > 
> > > > > > > Hello, Dmitriy
> > > > > > > 
> > > > > > > I'm looked in TeamCity but not sure what should I do.
> > > > > > > Can you explain it to me, please?
> > > > > > > 
> > > > > > > В Чт, 22/02/2018 в 10:13 +, Dmitry Pavlov пишет:
> > > > > > > > Hi Nikolay,
> > > > > > > > 
> > > > > > > > It seems
> > > > > > > > IgniteRDDSpec.IgniteRDD  has appeared in configurations it is
> > 
> > not
> > > > > > > 
> > > > > > > expected
> > > > > > > > to run.
> > > > > > > > 
> > > > > > > > Please see investigations assigned to you. What can we change
> > 
> > to
> > > > > 
> > > > > avoid
> > > > > > > > these tests to be included into run other suites?
> > > > > > > > 
> > > > > > > > Sincerely,
> > > > > > > > Dmitriy Pavlov
> > > > > > > > 
> > > > > > > > пт, 16 февр. 2018 г. в 7:35, Nikolay Izhikov <
> > 
> > nizhi...@apache.org>:
> > > > > > > > 
> > > > > > > > > Hello, Igniters.
> > > > > > > > > 
> > > > > > > > > I'm working on issue [1].
> > > > > > > > > 
> > > > > > > > > Team City doesn't collect info about scalatest execution
> > 
> > because of
> > > > > > > 
> > > > > > > wrong
> > > > > > > > > pom.xml
> > > > > > > > > I'm fixed it in PR [3]
> > > > > > > > > 
> > > > > > > > > It happens there is a 2 broken tests written in Scala - [4]
> > > > > > > > > 
> > > > > > > > > 1. IgniteRDDSpec.IgniteRDD should successfully store data to
> > 
> > ignite
> > > > > > > 
> > > > > > > using
> > > > > > > > > saveValues
> > > > > > > > > 2. IgniteRDDSpec.IgniteRDD should successfully store data to
> > 
> > ignite
> > > > > > > 
> > > > > > > using
> > > > > > > > > saveValues with inline transformation
> > > > > > > > > 
> > > > > > > > > It seems they are been here for a while.
> > > > > > > > > I propose to mute or disable it on TeamCity before merging
> > 
> > my PR.
> > > > > > > > > I've created ticket for fixing tests - [5].
> > > > > > > > > 
> > > > > > > > > Thoughts?
> > > > > > > > > 
> > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-7042
> > > > > > > > > 
> > > > > > > > > [2]
> > > > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > 
> > > > > 
> > 
> > https://ci.ignite.apache.org/viewLog.html?buildId=1096059=buildResultsDiv=IgniteTests24Java8_IgniteRdd
> > > > > > > > > 
> > > > > > > > > [3] https://github.com/apache/ignite/pull/3530
> > > > > > > > > 
> > > > > > > > > [4]
> > > > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > 
> > > > > 
> > 
> > https://ci.ignite.apache.org/viewLog.html?buildId=1095218=IgniteTests24Java8_IgniteRdd=testsInfo
> > > > > > > > > 
> > > > > > > > > [5] https://issues.apache.org/jira/browse/IGNITE-7727

signature.asc
Description: This is a digitally signed 

Re: MTCGA: Tests of the week

[Dmitry Pavlov]

Hi Denis,

It is not easy to say exact number of tests, because test failures are
quite randomized. Correct metric can be obtained from mass 'RunAll' runs,
for example, during weekend.

In the same time brief estimation gives number of failures around 30-60 per
'RunAll' instead of 60-80 (we don't count 355 CPP test here because it is
one problem).

If we check issues create/resolve chart, it is available
https://cwiki.apache.org/confluence/display/IGNITE/Make+Teamcity+Green+Again,
progress is obivious, ~ 27 issues reported and ~ 13 were resolved.

Hope someone in the community can pick up remaining unsassigned issues:
https://issues.apache.org/jira/issues/?jql=labels%3DMakeTeamcityGreenAgain%20AND%20createdDate%3E-3w%20AND%20resolution%20is%20EMPTY%20and%20assignee%20is%20EMPTY

Sincerely,
Dmitriy Pavlov



вт, 27 февр. 2018 г. в 0:45, Denis Magda :

> Hi Dmitriy,
>
> Could you share some statistics? How many tests were fixed or merged in
> one? Just curiuos.
>
> --
> Denis
>
> On Mon, Feb 26, 2018 at 9:36 AM, Dmitry Pavlov 
> wrote:
>
> > Hi Folks,
> >
> > Fixing the tests already brings some results. The number of falling tests
> > in a single run was reduced. Thanks to all, especially to Peter Ivanov,
> > Alexey Goncharuk, Pavel Kovalenko
> >
> > There are 4 tickets that were not picked up by community (list at the end
> > of letter). Please look through and assign issue if you are able to help
> > with this.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> >  - IGNITE-7791 -Ignite Client Nodes: failed test
> > IgniteClientReconnectCacheTest.testReconnectCacheDestroyedAndCreated()
> > - IGNITE-7789 - Ignite Client Nodes: failed test
> > IgniteClientReconnectApiExceptionTest.testErrorOnDisconnect()
> > - IGNITE-7708 - Ignite Compute has flaky failures in ConfigVariationTests
> > - IGNITE-7692 - affinityCall and affinityRun may execute code on backup
> > partitions
> >
>

[jira] [Created] (IGNITE-7821) Update Apache Ignite and Web Console Dockerfile for 2.4 version and local build

[Peter Ivanov (JIRA)]

Peter Ivanov created IGNITE-7821:


 Summary: Update Apache Ignite and Web Console Dockerfile for 2.4 
version and local build
 Key: IGNITE-7821
 URL: https://issues.apache.org/jira/browse/IGNITE-7821
 Project: Ignite
  Issue Type: Improvement
Reporter: Peter Ivanov
Assignee: Peter Ivanov
 Fix For: 2.4


# Update Dockerfiles in order to match upcoming 2.4 release.
# Change Apache Ignite's Dockerfile to get binaries from local build.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7820) Investigate and fix perfromance drop of WAL for default mode

[Andrey Gura (JIRA)]

Andrey Gura created IGNITE-7820:
---

 Summary: Investigate and fix perfromance drop of WAL for default 
mode
 Key: IGNITE-7820
 URL: https://issues.apache.org/jira/browse/IGNITE-7820
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.4
Reporter: Andrey Gura
 Fix For: 2.5


WAL performance drop was introduced by 
https://issues.apache.org/jira/browse/IGNITE-6339 fix. In order to provide 
better performance for {{DEFAULT}} WAL mode 
{{FsyncModeFileWriteAheadLogManager}} implementation was added as result of fix 
issue https://issues.apache.org/jira/browse/IGNITE-7594.

What we know about this performance drop:

* It affects {{IgnitePutAllBenchmark}} and {{IgnitePutAllTxBenchmark}} and 
measurements show 10-15% drop and ~50% drop accordingly.
* It is reproducible not for all hardware configuration. That is for some 
configuration we see only performance improvements instead of drop.
* It is reproducible for [Many clients --> One server] topology.
* If {{IGNITE_WAL_MMAP == false}} then we have better performance.
* If {{fsyncDelay == 0}} then we have better performance.

What were tried during initial investigation:

* Replacing of {{LockSupport.park/unpark}} to spin leads to improvement about 
2%.
* Using {{FileWriteHandle.fsync(null)}} (unconditional flush) instead of 
{{FileWriteHandle.fsync(position)}} (conditional flush) doesn't affect 
benchmarks.

What should we do:

Investigate the problem and provide fix or recommendation for system tuning.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: IgniteSet implementation: changes required

[Vladimir Ozerov]

I think the root issue is that we are trying to mix different cases in a
single solution. What is the common usage patterns of sets?
1) Small mostly-read sets - current implementation is ideal for them -
everything is available locally, on-heap and in deserialized form
2) Big data sets - IgniteCache API is the best candidate here; we can
simply add a method "Set IgniteCache.asSet()" which will return thin
wrapper for Set interface around cache. An you will have recovery and
heap-resistance with almost no additional efforts.
There is no need to choose between one or another solution. We can support
both.

As far as current igniteSet implementation - all issues described above
will go away if you use approach suggested by Dmitry. I do not see a need
for any major changes.

Vladimir.

On Mon, Feb 19, 2018 at 10:44 AM, Pavel Pereslegin  wrote:

> Hello Vladimir,
>
> > What we can do is to optionally disable on-heap caching for specific set
> at the cost of lower performance if user wants so.
>
> I want to make sure that we are speaking about the same thing. By
> "on-heap caching" I mean custom datastructure and not standard on-heap
> cache, we can't disable it now, because it is the part of current
> IgniteSet implementation. Is it acceptable to use GridOffHeapMap
> optionally instead of ConcurrentHashMap for "off-heap mode"?
>
>
> 2018-02-15 23:20 GMT+03:00 Dmitriy Setrakyan :
> > On Thu, Feb 15, 2018 at 6:08 AM, Vladimir Ozerov 
> > wrote:
> >
> >> I do not think indexes is the right approach - set do not have indexes,
> and
> >> you will have to maintain additional counter for it in order to know
> when
> >> to stop.
> >>
> >> From what I see there are two distinct problems:
> >> 1) Broken recovery - this is just a bug which needs to be fixed. As
> soon as
> >> data is stored in real persistent cache, recovery of data structure
> state
> >> should be trivial task.
> >> 2) Heap items - this should not be a problem in common case when set
> >> contains moderate number of elements. If set is excessively large, then
> >> this is not the right structure for your use case and you should use
> >> standard IgniteCache API instead. What we can do is to optionally
> disable
> >> on-heap caching for specific set at the cost of lower performance if
> user
> >> wants so.
> >>
> >
> > Vladimir, I am not sure I agree. In my view, set should be similar to
> > cache, just a different API. I am not sure why we should make an
> > assumptions that set data should be smaller than cache, especially given
> > that it is a trivial task to implement a set based on Ignite cache API
> (we
> > could just store key-key mappings in cache instead of key-value mappings
> > internally).
> >
> > Can you clarify why you believe that IgniteSet should need to have
> on-heap
> > entries?
> >
> > D.
>

[jira] [Created] (IGNITE-7819) IgniteServiceConfigVariationsFullApiTest must stop services in finally block

[Alexey Goncharuk (JIRA)]

Alexey Goncharuk created IGNITE-7819:


 Summary: IgniteServiceConfigVariationsFullApiTest must stop 
services in finally block
 Key: IGNITE-7819
 URL: https://issues.apache.org/jira/browse/IGNITE-7819
 Project: Ignite
  Issue Type: Test
Reporter: Alexey Goncharuk
Assignee: Alexey Goncharuk
 Fix For: 2.5






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)