[jira] [Created] (IGNITE-7994) Web console: remove _auth method from frontend Auth service

[Ilya Borisov (JIRA)]

Ilya Borisov created IGNITE-7994:


 Summary: Web console: remove _auth method from frontend Auth 
service
 Key: IGNITE-7994
 URL: https://issues.apache.org/jira/browse/IGNITE-7994
 Project: Ignite
  Issue Type: Improvement
  Components: wizards
Reporter: Ilya Borisov


The current implementation uses a single method to call various backend API 
endpoints, which is sub-optimal. I've added proper separate methods to the 
service during IGNITE-7949, but these still use `_auth` under the hood. The 
`_auth` method should be removed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: readme.io weird interface element

[Petr Ivanov]

Prachi, has you filed the ticket or should I create one?



> On 15 Mar 2018, at 23:07, Denis Magda  wrote:
> 
> Please create a ticket and let's fix it in the nearest future.
> 
> --
> Denis
> 
> On Thu, Mar 15, 2018 at 12:47 PM, Prachi Garg  wrote:
> 
>> That's the download button. Tables on feature pages are downloadable in
>> pdf, csv format. There is a javascript code embedded with the table
>> element. Not sure why it doesn't work on the download page; works fine on
>> other pages.
>> 
>> On Thu, Mar 15, 2018 at 12:16 PM, Denis Magda  wrote:
>> 
>>> Have no idea, just makes the table looks nicer :)
>>> 
>>> Prachi, what is this for? It's not clear from the HTML sources.
>>> 
>>> --
>>> Denis
>>> 
>>> On Thu, Mar 15, 2018 at 12:09 PM, Petr Ivanov 
>> wrote:
>>> 
 Sorry for misleading. I meant https://ignite.apache.org/download.cgi
 indeed.
 
 
 
> On 15 Mar 2018, at 22:05, Denis Magda  wrote:
> 
> Petr,
> 
> What's that? It doesn't look like the readme.io doc that hosts all
 Ignite
> docs:
> https://apacheignite.readme.io/docs
> 
> --
> Denis
> 
> On Thu, Mar 15, 2018 at 1:52 AM, vveider  wrote:
> 
>> Hi, all!
>> 
>> 
>> Does anyone know what is this button with arrow and disk at the right
>> corner
>> of versions list header? [1]
>> Seems that it does nothing.
>> 
>> [1] https://ibb.co/eJf5uc
>> 
>> 
>> 
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>> 
 
 
>>> 
>> 

Re: IEP-14: Ignite failures handling (Discussion)

[Dmitriy Setrakyan]

On Mon, Mar 19, 2018 at 2:24 PM, Yakov Zhdanov  wrote:

> Andrey Gura,
>
> Why should we have any FailureHandler abstraction? We already have it -
> this is EventListener. In my view it is better (and cleaner design) to add
> events (similar to, for
> example, org.apache.ignite.events.EventType#EVT_NODE_SEGMENTED) like
> EVT_IGNITE_OOME, EVT_SYS_WORKER_FAILED and fire events accordingly to the
> situation + execute configured system logic. We have exactly same way with
> segmentation. We have policy which defines how system reacts and also allow
> user to add event listeners.
>

Yakov, how would it be possible to fire the events if Ignite is not in
operational state? For example, what can a user do if the Java application
ran out of memory?

[jira] [Created] (IGNITE-7993) Striped pool can't be disabled

[Valentin Kulichenko (JIRA)]

Valentin Kulichenko created IGNITE-7993:
---

 Summary: Striped pool can't be disabled
 Key: IGNITE-7993
 URL: https://issues.apache.org/jira/browse/IGNITE-7993
 Project: Ignite
  Issue Type: Bug
  Components: general
Affects Versions: 2.4
Reporter: Valentin Kulichenko
 Fix For: 2.5


Javadoc for {{IgniteConfiguration#setStripedPoolSize}} states that striped pool 
can be disabled by providing value less or equal than zero:
{noformat}
If set to non-positive value then requests get processed in system pool.
{noformat}
However, doing that prevents node from startup, it fails with the following 
exception:
{noformat}
Caused by: class org.apache.ignite.IgniteCheckedException: Invalid stripedPool 
thread pool size (must be greater than 0), actual value: 0
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.validateThreadPoolSize(IgnitionEx.java:2061)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1799)
at 
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1716)
at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1144)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:664)
at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:589)
at org.apache.ignite.Ignition.start(Ignition.java:322)
... 7 more
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3620: Keep marshaller mappings in consistent state

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3620


---

Re: Reconsider default WAL mode: we need something between LOG_ONLY and FSYNC

[Valentin Kulichenko]

Guys,

What do we understand under "data corruption" here? If a storage is in
corrupted state, does it mean that it needs to be completely removed and
cluster needs to be restarted without data? If so, I'm not sure any mode
that allows corruption makes much sense to me. How am I supposed to use a
database, if virtually any failure can end with complete loss of data?

In any case, this definitely should not be a default behavior. If user ever
switches to corruption-unsafe mode, there should be a clear warning about
this.

-Val

On Fri, Mar 16, 2018 at 1:06 AM, Ivan Rakov  wrote:

> Ticket to track changes: https://issues.apache.org/jira/browse/IGNITE-7754
>
> Best Regards,
> Ivan Rakov
>
>
> On 16.03.2018 10:58, Dmitriy Setrakyan wrote:
>
>> On Fri, Mar 16, 2018 at 12:55 AM, Ivan Rakov 
>> wrote:
>>
>> Vladimir,
>>>
>>> Unlike BACKGROUND, LOG_ONLY provides strict write guarantees unless power
>>> loss has happened.
>>> Seems like we need to measure performance difference to decide whether do
>>> we need separate WAL mode. If it will be invisible, we'll just fix these
>>> bugs without introducing new mode; if it will be perceptible, we'll
>>> continue the discussion about introducing LOG_ONLY_SAFE.
>>> Makes sense?
>>>
>>> Yes, this sounds like the right approach.
>>
>>
>

Re: (Partition Map) Exchange at wiki

[Dmitry Pavlov]

Thank you, Denis.

I'm really proud that the wiki now has a link even from official docs.

Made some updates in the article.

Sincerely
Dmitry Pavlov

вт, 20 мар. 2018 г., 1:54 Denis Magda :

> Sure,
>
> See a green callout block in this section:
> https://apacheignite.readme.io/v2.4/docs/cache-modes#section-overview
>
> --
> Denis
>
> On Mon, Mar 19, 2018 at 4:28 AM, Yakov Zhdanov 
> wrote:
>
> > Awesome article, Dmitry!
> >
> > Denis Magda, should we put a link to it from apacheignite.readme.io?
> >
> > --Yakov
> >
>

Re: (Partition Map) Exchange at wiki

[Denis Magda]

Sure,

See a green callout block in this section:
https://apacheignite.readme.io/v2.4/docs/cache-modes#section-overview

--
Denis

On Mon, Mar 19, 2018 at 4:28 AM, Yakov Zhdanov  wrote:

> Awesome article, Dmitry!
>
> Denis Magda, should we put a link to it from apacheignite.readme.io?
>
> --Yakov
>

[jira] [Created] (IGNITE-7992) Clarify how expiration policy works for durable memory and 3rd pary

[Denis Magda (JIRA)]

Denis Magda created IGNITE-7992:
---

 Summary: Clarify how expiration policy works for durable memory 
and 3rd pary
 Key: IGNITE-7992
 URL: https://issues.apache.org/jira/browse/IGNITE-7992
 Project: Ignite
  Issue Type: New Feature
  Components: documentation
Affects Versions: 2.4
Reporter: Denis Magda
Assignee: Prachi Garg
 Fix For: 2.5


Update the expiration policies page with the result of the following discussion:
http://apache-ignite-developers.2346864.n4.nabble.com/Data-eviction-expiration-from-Ignite-persistence-td24419i20.html#a28081



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-7991) MVCC TX Crash recovery

[Igor Seliverstov (JIRA)]

Igor Seliverstov created IGNITE-7991:


 Summary: MVCC TX Crash recovery
 Key: IGNITE-7991
 URL: https://issues.apache.org/jira/browse/IGNITE-7991
 Project: Ignite
  Issue Type: Task
  Components: cache
Reporter: Igor Seliverstov


Implement crash recovery for MVCC enabled caches



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

2 phase waiting for partitions release

[Pavel Kovalenko]

Hello Igniters,

Current implementation of
GridDhtPartitionsExchangeFuture#waitPartitionRelease function doesn't give
us 100% guarantees that
after this method completes there are no ongoing atomic or transactional
updates on current node during main stage of PME.
It gives us only guarantee that all primary updates will be finished on
that node, while we can still receive and process backup updates after this
method.
Example of such case is described in
https://issues.apache.org/jira/browse/IGNITE-7871

To avoid such situations we would like to implement second phase of
waitPartitionRelease method.
On this phase every server node participating in PME should wait while all
other server nodes will finish their ongoing updates.

Here is brief algorithm description:

Non-coordinator node:
1) Finish all ongoing atomic & transactional updates.
2) Send acknowledgement to coordinator.
3) Wait for final acknowledgement from coordinator, that all nodes finished
their updates.
4) Continue PME.

Coordinator node:
1) Finish all ongoing atomic & transactional updates.
2) Wait for all acknowledgements from all server nodes.
3) Send final acknowledgement to all server nodes.
4) Continue PME.

Acknowledgement messages have tiny size, so network pressure and overall
performance drop will be minimal.

Another solution of the problem is just cancelling atomic backup updates
and transactional backup updates on PREPARED phase if topology version is
changed.
But from user perspective it's not correct to catch transaction errors even
in cases when node is joining to the cluster.

Any thoughts?

Apache Ignite community update for the week of March 19

[Tom Diederich]

Hello Igniters! 

Akmal Chaudhri is in Atlanta this week for a mini-Meetup tour evangelizing 
Apache Ignite. He just wrapped the OpenIoT Summit North America in Portland 
last week, where his popular session demonstrated data streaming to an Apache 
Ignite cluster from embedded devices and real-time data processing with Apache 
Spark.

Also last week, and in the photo, Fotios Filacouris spoke at the NYC In-Memory 
Computing Meetup, where he walked attendees through a few reference 
architectures on various use cases that have benefited from using Apache 
Ignite. The room was filled to capacity in an open Manhattan office donated by 
WeWork via a pilot program with Meetup.com, which the company purchased last 
November. WeWork is building a worldwide network of co-working offices and 
apartments.

Here’s a look at this week

Monday
Our technology evangelist will speak at the first of his scheduled three 
meetups this evening. That meetup is called, Big Bang Data Science & Analytics 
Solutions.

Its members are obsessed with data science and Akmal's talk won't disappoint. 
He'll unpack some of Apache Ignite's main components (the Compute Grid, Data 
Grid and the Machine Learning Grid) and then, through examples, explain how 
Apache Ignite can be used for data analysis.

Tuesday
March 20 is the first day of Spring and two of our experts will be speaking at 
events some 5,000 miles apart. 

In Georgia, Akmal will be speaking at the Atlanta Apache Spark User Group. This 
session will demonstrate how to easily share state in-memory across multiple 
Spark jobs, either within the same application or between different Spark 
applications using an implementation of the Spark RDD abstraction provided in 
Apache Ignite

During the talk, Akmal will demonstrate how IgniteRDD — an implementation of 
native Spark RDD and DataFrame APIs — shares the state of the RDD across other 
Spark jobs, applications and workers. Examples will show how IgniteRDD allows 
execution of SQL queries many times faster than native Spark RDDs or Data 
Frames due to its advanced in-memory indexing capabilities.

Meanwhile, at the Moscow Apache Ignite Meetup, engineer Artem Schitow will 
unveil what’s new in Apache Ignite 2.4. Next up, architect Alexey Goncharuk 
will demonstrate the art of sizing Apache Ignite clusters (delivered at the 
architect level). My colleague in Moscow Kseniya Romanova organized and will 
also serve as MC at both events. Great job, Kseniya!

 Wednesday
Akmal concludes his visit to Atlanta with a talk at the DevOpsATL Meetup about 
distributed databases and Kubernetes. In his presentation, Akmal will show 
attendees how Kubernetes can orchestrate a distributed database like Apache 
Ignite, in particular:

 •    Cluster Assembling - database nodes auto-discovery in Kubernetes.

 •    Database Resilience - automated horizontal scalability.

 •    Database Availability - what's the role of Kubernetes and the 
database.

 •    Utilizing both RAM and disk - set up Apache Ignite in a way to 
get in-memory performance with the durability of disk.


That’s all for this update.

Tom

 

 

Re: [TeamCity] DataFrame Examples

[Nikolay Izhikov]

Petr, thanks for help with Team City configuration issue!

В Пн, 19/03/2018 в 18:14 +0300, Nikolay Izhikov пишет:
> Yes.
> 
> В Пн, 19/03/2018 в 18:13 +0300, Petr Ivanov пишет:
> > Have you detached template for build Ignite Examples?
> > 
> > 
> > > On 19 Mar 2018, at 18:08, Nikolay Izhikov  wrote:
> > > 
> > > Hello, guys.
> > > 
> > > I merged to master my fix of tests. [1]
> > > 
> > > I edited "Ignite Examples" configuration to Team City to enable testing 
> > > of Spark examples.
> > > 
> > > 1. `spark` profile enabled for test plan.
> > > 2. `IgniteExamplesSparkSelfTestSuite` added to tests suites list.
> > > 
> > > Test results [2] are OK.
> > > Please, Team City gurus, validate my edits.
> > > 
> > > [1] 
> > > https://github.com/apache/ignite/commit/ce6a15782c93a95c58ccb46c44f391b39d4c4ac9
> > > [2] 
> > > https://ci.ignite.apache.org/viewLog.html?buildId=1145334&buildTypeId=IgniteTests24Java8_IgniteExamples&tab=testsInfo
> > > 
> > > В Пн, 19/03/2018 в 06:12 -0700, vveider пишет:
> > > > +1 to sticking to one suite for now.
> > > > 
> > > > Maybe someday when examples' tests will surpass adequate build time, 
> > > > we'll
> > > > consider splitting them.
> > > > 
> > > > 
> > > > 
> > > > --
> > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

signature.asc
Description: This is a digitally signed message part

[jira] [Created] (IGNITE-7990) Integrate MLP with Distributed Dataset API

[Anton Dmitriev (JIRA)]

Anton Dmitriev created IGNITE-7990:
--

 Summary: Integrate MLP with Distributed Dataset API
 Key: IGNITE-7990
 URL: https://issues.apache.org/jira/browse/IGNITE-7990
 Project: Ignite
  Issue Type: Improvement
Reporter: Anton Dmitriev
Assignee: Anton Dmitriev






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [TeamCity] DataFrame Examples

[Nikolay Izhikov]

Yes.

В Пн, 19/03/2018 в 18:13 +0300, Petr Ivanov пишет:
> Have you detached template for build Ignite Examples?
> 
> 
> > On 19 Mar 2018, at 18:08, Nikolay Izhikov  wrote:
> > 
> > Hello, guys.
> > 
> > I merged to master my fix of tests. [1]
> > 
> > I edited "Ignite Examples" configuration to Team City to enable testing of 
> > Spark examples.
> > 
> > 1. `spark` profile enabled for test plan.
> > 2. `IgniteExamplesSparkSelfTestSuite` added to tests suites list.
> > 
> > Test results [2] are OK.
> > Please, Team City gurus, validate my edits.
> > 
> > [1] 
> > https://github.com/apache/ignite/commit/ce6a15782c93a95c58ccb46c44f391b39d4c4ac9
> > [2] 
> > https://ci.ignite.apache.org/viewLog.html?buildId=1145334&buildTypeId=IgniteTests24Java8_IgniteExamples&tab=testsInfo
> > 
> > В Пн, 19/03/2018 в 06:12 -0700, vveider пишет:
> > > +1 to sticking to one suite for now.
> > > 
> > > Maybe someday when examples' tests will surpass adequate build time, we'll
> > > consider splitting them.
> > > 
> > > 
> > > 
> > > --
> > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> 
> 

signature.asc
Description: This is a digitally signed message part

Re: [TeamCity] DataFrame Examples

[Petr Ivanov]

Have you detached template for build Ignite Examples?


> On 19 Mar 2018, at 18:08, Nikolay Izhikov  wrote:
> 
> Hello, guys.
> 
> I merged to master my fix of tests. [1]
> 
> I edited "Ignite Examples" configuration to Team City to enable testing of 
> Spark examples.
> 
> 1. `spark` profile enabled for test plan.
> 2. `IgniteExamplesSparkSelfTestSuite` added to tests suites list.
> 
> Test results [2] are OK.
> Please, Team City gurus, validate my edits.
> 
> [1] 
> https://github.com/apache/ignite/commit/ce6a15782c93a95c58ccb46c44f391b39d4c4ac9
> [2] 
> https://ci.ignite.apache.org/viewLog.html?buildId=1145334&buildTypeId=IgniteTests24Java8_IgniteExamples&tab=testsInfo
> 
> В Пн, 19/03/2018 в 06:12 -0700, vveider пишет:
>> +1 to sticking to one suite for now.
>> 
>> Maybe someday when examples' tests will surpass adequate build time, we'll
>> consider splitting them.
>> 
>> 
>> 
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [TeamCity] DataFrame Examples

[Nikolay Izhikov]

Hello, guys.

I merged to master my fix of tests. [1]

I edited "Ignite Examples" configuration to Team City to enable testing of 
Spark examples.

1. `spark` profile enabled for test plan.
2. `IgniteExamplesSparkSelfTestSuite` added to tests suites list.

Test results [2] are OK.
Please, Team City gurus, validate my edits.

[1] 
https://github.com/apache/ignite/commit/ce6a15782c93a95c58ccb46c44f391b39d4c4ac9
[2] 
https://ci.ignite.apache.org/viewLog.html?buildId=1145334&buildTypeId=IgniteTests24Java8_IgniteExamples&tab=testsInfo

В Пн, 19/03/2018 в 06:12 -0700, vveider пишет:
> +1 to sticking to one suite for now.
> 
> Maybe someday when examples' tests will surpass adequate build time, we'll
> consider splitting them.
> 
> 
> 
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

signature.asc
Description: This is a digitally signed message part

[jira] [Created] (IGNITE-7989) ClassNotFoundException org.jsr166.ConcurrentHashMap8

[Anton Kalashnikov (JIRA)]

Anton Kalashnikov created IGNITE-7989:
-

 Summary: ClassNotFoundException org.jsr166.ConcurrentHashMap8
 Key: IGNITE-7989
 URL: https://issues.apache.org/jira/browse/IGNITE-7989
 Project: Ignite
  Issue Type: Bug
Reporter: Anton Kalashnikov


[2018-03-17 15:54:52,042][ERROR][tcp-disco-msg-worker-#1075][TcpDiscoverySpi] 
Failed to unmarshal discovery data for component: 6
class org.apache.ignite.IgniteCheckedException: Failed to find class with given 
class loader for unmarshalling (make sure same versions of all classes are 
available on all nodes or enable peer-class-loading) 
[clsLdr=IsolatedClassLoader\{roleName='test'}, 
cls=org.jsr166.ConcurrentHashMap8]
    at 
org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:144)
    at 
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
    at 
org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:161)
    at 
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
    at 
org.apache.ignite.spi.discovery.tcp.internal.DiscoveryDataPacket.unmarshalData(DiscoveryDataPacket.java:277)
    at 
org.apache.ignite.spi.discovery.tcp.internal.DiscoveryDataPacket.unmarshalGridData(DiscoveryDataPacket.java:120)
    at 
org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.onExchange(TcpDiscoverySpi.java:1916)
    at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processNodeAddedMessage(ServerImpl.java:4305)
    at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2706)
    at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2519)
    at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6707)
    at 
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2602)
    at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
Caused by: java.lang.ClassNotFoundException: org.jsr166.ConcurrentHashMap8
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at 
org.apache.maven.surefire.booter.IsolatedClassLoader.loadClass(IsolatedClassLoader.java:100)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at 
org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8577)
    at 
org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.resolveClass(JdkMarshallerObjectInputStream.java:59)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1859)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1745)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2033)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1567)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:427)
    at java.util.ArrayList.readObject(ArrayList.java:797)
    at sun.reflect.GeneratedMethodAccessor87.invoke(Unknown Source)
    at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2060)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1567)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:427)
    at 
org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:139)
    ... 12 more



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3590: IGNITE-7816: Fixing Spark examples to run correct...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3590


---

[jira] [Created] (IGNITE-7988) SQL: Upload benchmarks in mvcc mode

[Pavel Kuznetsov (JIRA)]

Pavel Kuznetsov created IGNITE-7988:
---

 Summary: SQL: Upload benchmarks in mvcc mode
 Key: IGNITE-7988
 URL: https://issues.apache.org/jira/browse/IGNITE-7988
 Project: Ignite
  Issue Type: Task
Reporter: Pavel Kuznetsov
Assignee: Pavel Kuznetsov


We need to know how does mvcc sql feature affect performance.
dev branch: ignite-4191
cache atomicity mode: TRANSACTIONAL

We need to adopt existing jdbc benchmarks to be able to run in mvcc.
Only thin driver  and native sql should be benchmarked.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3660: IGNITE-7987 Use all discovery events for affinity...

[Jokser]

GitHub user Jokser opened a pull request:

https://github.com/apache/ignite/pull/3660

IGNITE-7987 Use all discovery events for affinity calculation.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7987

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3660.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3660


commit 65142464a68b170a8f9ee23b850620c06bc7f121
Author: Pavel Kovalenko 
Date:   2018-03-19T14:24:00Z

IGNITE-7987 Use all discovery events for affinity calculation.




---

[jira] [Created] (IGNITE-7987) Affinity may be not calculated properly in case of merged exchanges with client nodes

[Pavel Kovalenko (JIRA)]

Pavel Kovalenko created IGNITE-7987:
---

 Summary: Affinity may be not calculated properly in case of merged 
exchanges with client nodes
 Key: IGNITE-7987
 URL: https://issues.apache.org/jira/browse/IGNITE-7987
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.4
Reporter: Pavel Kovalenko
Assignee: Pavel Kovalenko
 Fix For: 2.5


Currently we pass only last (or first in some cases) discovery event for 
affinity calculation at GridAffinityAssignmentCache.

Affinity calculation can be skipped if such discovery event belongs to client 
node or node filtered by nodeFilter for optimization issues (because affinity 
will not be changed in such case).

Since we have exchange merging there can be several discovery events 
corresponds to one exchange. Passing only first or last event for affinity 
calculation is wrong, because calculation can be skipped, while exchange 
actually contains events changing affinity.

Instead of first/last event we should pass whole collection of discovery events 
(ExchangeDiscoveryEvents) and skip affinity calculation for a group only when 
ALL events doesn't change affinity for such group.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3659: IGNITE-7986: GridPartitionStateMap.entrySet() opt...

[BiryukovVA]

GitHub user BiryukovVA opened a pull request:

https://github.com/apache/ignite/pull/3659

IGNITE-7986: GridPartitionStateMap.entrySet() optimization.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/BiryukovVA/ignite IGNITE-7986

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3659.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3659


commit 255ea51a96a0c1b6ce36d189dc670f4bb9c54010
Author: Vitaliy Biryukov 
Date:   2018-03-19T13:47:01Z

IGNITE-7986: GridPartitionStateMap.entrySet() optimization.




---

[jira] [Created] (IGNITE-7986) GridPartitionStateMap.entrySet() optimization.

[Vitaliy Biryukov (JIRA)]

Vitaliy Biryukov created IGNITE-7986:


 Summary: GridPartitionStateMap.entrySet() optimization.
 Key: IGNITE-7986
 URL: https://issues.apache.org/jira/browse/IGNITE-7986
 Project: Ignite
  Issue Type: Improvement
Reporter: Vitaliy Biryukov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Code inspection

[Petr Ivanov]

Filed https://issues.apache.org/jira/browse/IGNITE-7985 
 [1].



> On 18 Mar 2018, at 00:56, Dmitry Pavlov  wrote:
> 
> Hello Petr,
> 
> Many members of the community would appreciate such additional code control, 
> and it's a pity that no one made this happen. Agree? 
> 
> Could you please pick up this activity?
> 
> It might be an idea to create 'IDEA Inspections' step to be run in parallel 
> with 'Build Apache Ignite'. WDYT? Would it work?
> 
> Sincerely,
> Dmitriy Pavlov
> 
> https://confluence.jetbrains.com/display/TCD10/Inspections 
> 
> 
> 
> пн, 12 мар. 2018 г. в 14:37, Dmitry Pavlov  >:
> Hi Dmitriy,
> 
> would you pick up this activity?
> 
> Sincerely,
> Dmitriy Pavlov
> 
> вт, 6 мар. 2018 г. в 14:09, Dmitry Pavlov  >:
> What I can suggest now it is to take XML file with existing as is from 
> previous topic (I remember someone in community already prepared settings) 
> and set up TeamCity Run configuration as part of Run All Basic Tests (per 
> commit basis).
> 
> If we don’t have XML, I suggest to enable build-in Idea inspections 'as is' 
> on TeamCity and iteratively improve it according to found issues.
> 
> Dmitriy G., would you prepare PR and proof-of-concept TC run configuration? 
> 
> As discussion became really active, I think that means community is 
> interested in static code checks.
> 
> вт, 6 мар. 2018 г. в 14:08, Dmitry Pavlov  >:
> I was thinking about some quick check, which will automatically require 
> minimum runs. Now, any committer can push changes to the master, which break 
> not only the inspection and style, but even the compilation. If this control 
> would be automatic, it can allow us make codebase better quite fast. But I am 
> afraid it is not realistic.
> 
> 
> 
> вт, 6 мар. 2018 г. в 13:42, Petr Ivanov  >:
> Sonar is powerful, yes, but it’s power in thoroughness. I.e. it does its job 
> well in cases of leisurely post-build analysis.
> 
> I’d suggest we use it (if we will use it) in the following scenarios:
>  — some basic checks Sonar profile for Blocker bugs (it is fast) — something 
> that cannot be passed to master;
>  — nightly or even weekly run with Full Sonar profile (600+ checks from 
> Firebug, Codestyle, Coverage, etc.) for regression and overall code quality 
> improvement goals.
> 
> Did not quite get you about push-to-master prohibition. Can you explain 
> scenario in more details?
> 
> 
> > On 6 Mar 2018, at 13:27, Dmitry Pavlov  > > wrote:
> >
> > Petr, I've heard Sonar is powerful tool.
> >
> > Would it help us to prohibit commits to master w/o test run / too much
> > failed tests / too much inspection errors appeared?
> >
> > вт, 6 мар. 2018 г. в 13:22, Alexey Goncharuk  > >:
> >
> >> Dmitriy,
> >>
> >> I like this idea a lot. For example, the inspection profile should have
> >> inspection 'Anonymous class can be converted to lambda' disabled because
> >> quite a lot of such classes can be sent over the network (although even
> >> anonymous classes are discourage for such purposes).
> >>
> >> I believe we can start with sharing somehow one of the profiles and then
> >> iteratively improving it until the community is satisfied with the result.
> >>
> >> Thoughts?
> >>
> >> 2018-03-06 12:06 GMT+03:00 Petr Ivanov  >> >:
> >>
> >>> We can use Sonar as instrument for code analysis and test coverage
> >>> inspections.
> >>>
> >>>
> >>>
>  On 6 Mar 2018, at 11:28, Dmitriy Govorukhin <
> >>> dmitriy.govoruk...@gmail.com > wrote:
> 
>  Dmitriy,
> 
>  As I understood, preview topic was of static code analysis in general.
>  In this topic, I want to discuss only idea inspection rule.
>  In future, of course, we can expаnd this rule to the TeamCity build.
> 
>  On Tue, Mar 6, 2018 at 11:16 AM, Nikolay Izhikov   >
>  wrote:
> 
> > Hello, Igniters.
> >
> > +1 to automatic code style tools.
> >
> > Let's make it already!
> > Do we have a ticket for it?
> >
> > Related discussion -
> >> http://apache-ignite-developers.2346864.n4.nabble 
> >> .
> > com/Static-code-analysis-for-Java-td22195.html
> >
> > В Вт, 06/03/2018 в 08:15 +, Dmitry Pavlov пишет:
> >> Hi Dmitriy,
> >>
> >> I think we should resurrect thread about addition of code
> >> inspections,
> > and
> >> later we can enable automatic control step to TeamCity.
> >>
> >> Could you help me to find it?
> >>
> >> вт, 6 мар. 2018 г. в 11:11, Dmitriy Govorukhin <
> > dmitriy.govoruk...@gmail.com 
> >>> 

[jira] [Created] (IGNITE-7985) Integration JetBrains IntelliJ IDEA code inspections as TC build

[Peter Ivanov (JIRA)]

Peter Ivanov created IGNITE-7985:


 Summary: Integration JetBrains IntelliJ IDEA code inspections as 
TC build
 Key: IGNITE-7985
 URL: https://issues.apache.org/jira/browse/IGNITE-7985
 Project: Ignite
  Issue Type: Task
Reporter: Peter Ivanov
Assignee: Peter Ivanov


https://confluence.jetbrains.com/display/TCD10/Inspections



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3658: ignite-7978 don't check tag on restore in GridCac...

[dmekhanikov]

GitHub user dmekhanikov opened a pull request:

https://github.com/apache/ignite/pull/3658

ignite-7978 don't check tag on restore in GridCacheDatabaseSharedManager



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7978

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3658.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3658


commit 7b0b32249f537cf21f422bb089cb9f0683016c6e
Author: Denis Mekhanikov 
Date:   2018-03-19T13:10:48Z

ignite-7978 don't check tag on restore in GridCacheDatabaseSharedManager




---

Re: [TeamCity] DataFrame Examples

[vveider]

+1 to sticking to one suite for now.

Maybe someday when examples' tests will surpass adequate build time, we'll
consider splitting them.



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: [TeamCity] DataFrame Examples

[Dmitry Pavlov]

Hi Igniters,

 It seems to me it is better to keep one suite. It is reasonable to
separate suites if tests requires too much time to run.

Using the same suite will save remarkable amount of agent time.

Sincerely,
Dmitriy Pavlov

вс, 18 мар. 2018 г. в 19:34, Nikolay Izhikov :

> Hello, guys.
>
> I think we are ready to enable testing of Spark examples on Team City.
> Can you help me with best choice to do it?
>
> Currently, we have 'Ignite Examples' to run `IgniteExamplesSelfTestSuite`.
>
> We also have a `IgniteExamplesSparkSelfTestSuite` to test spark example.
> `IgniteExamplesSparkSelfTestSuite` exists in source only if `spark`
> profile enabled so we can't just add it to `IgniteExamplesSelfTestSuite`.
>
> I see following option here:
>
> 1. Edit 'Ignite Examples' to run both `IgniteExamplesSelfTestSuite` and
> `IgniteExamplesSparkSelfTestSuite`.
> 2. Create new test plan to run `IgniteExamplesSparkSelfTestSuite`.
>
> Thoughts?
>
> P.S. We also need minor changes [1] to make sure Spark examples are
> running correctly.
>
> [1] https://github.com/apache/ignite/pull/3590
>
> В Вт, 27/02/2018 в 11:56 -0800, Valentin Kulichenko пишет:
> > If we can run Scala examples there, then yes.
> >
> > -Val
> >
> > On Tue, Feb 27, 2018 at 11:54 AM, Dmitry Pavlov 
> > wrote:
> >
> > > Nikolay, Val,
> > >
> > > Could these examples be tested in standard Ignite Examples test suite?
> As
> > > far as I know Ignite ML and Ignite IGFS are also tested there.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > вт, 27 февр. 2018 г. в 22:49, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com>:
> > >
> > > > Nikolay,
> > > >
> > > > Spark integration is not related to Scalar, the only thing they have
> in
> > > > common is Scala. I think we should have a separate configuration for
> > > > ignite-spark module. If anything Spark related is currently in Scalar
> > > > suite, it should be moved from there.
> > > >
> > > > -Val
> > > >
> > > > On Tue, Feb 27, 2018 at 3:03 AM, Nikolay Izhikov <
> nizhi...@apache.org>
> > > > wrote:
> > > >
> > > > > Hello, Igniters.
> > > > >
> > > > > I've created several examples for a Ignite DataFrame.
> > > > > Currently, they are not tested on Team City.
> > > > >
> > > > > As far as I can understand, they should be checked in "Ignite
> Scalar
> > > > > Examples [Scala]".
> > > > > But it marked as obsolete [1]
> > > > >
> > > > > Please, give me advice:
> > > > >
> > > > > 1. Should we test Ignite DataFrame examples on Team City?
> > > > >
> > > > > 2. Which plan should be used for that?
> > > > >
> > > > > [1]
> > > >
> > > >
> https://ci.ignite.apache.org/viewLog.html?buildId=1068099&buildTypeId=
> > > > > IgniteTestsForCodeCoverage_IgniteScalarExamples&tab=buildParameters

[GitHub] ignite pull request #3657: IGNITE-7888: Added support for SQL_ATTR_LOGIN_TIM...

[isapego]

GitHub user isapego opened a pull request:

https://github.com/apache/ignite/pull/3657

IGNITE-7888: Added support for SQL_ATTR_LOGIN_TIMEOUT



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7888

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3657.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3657


commit 36b9f21cdee6b5fc2d93845177db56aed082f610
Author: Igor Sapego 
Date:   2018-03-16T15:33:11Z

IGNITE-7888: Implemented login timeout.

commit f9d7e97b39be3af59be617a839ac7ed8b53b10e1
Author: Igor Sapego 
Date:   2018-03-19T12:08:17Z

IGNITE-7888: Added tests




---

Re: Stop nodes after test by default - IGNITE-6842

[Dmitry Pavlov]

I agree it is important, I'm going to do a review, but do not have time
slot to do.

Who could pick up this review?

Dmitriy G., could I ask you?

пн, 19 мар. 2018 г. в 15:13, Maxim Muzafarov :

> Dmitry and other igniters,
>
> Will you have time to review this issue?
> I've preperated PR and TC for this, also I've fixed all comments made by
> Andrey Kuznetsov and Vyacheslav Daradur.
>
> I think this is important issue and will make test framework more stable
> and clear.
>
>
> TC: https://ci.ignite.apache.org/viewLog.html?buildId=1138151
> JIRA: https://issues.apache.org/jira/browse/IGNITE-6842
> Upsource: https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
> PR: https://github.com/apache/ignite/pull/3542
>
> чт, 15 мар. 2018 г. в 13:31, Maxim Muzafarov :
>
>> Dmtry,
>>
>> Can we proceed with this change?
>> I've done with fixing review comments and tests that you mentioned before.
>>
>> TC: https://ci.ignite.apache.org/viewLog.html?buildId=1138151
>> JIRA: https://issues.apache.org/jira/browse/IGNITE-6842
>> Upsource: https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
>> PR: https://github.com/apache/ignite/pull/3542
>>
>>
>>
>> вт, 6 мар. 2018 г. в 20:42, Dmitry Pavlov :
>>
>>> Ok, thank you.
>>>
>>> Please let me know when we can proceed with review
>>> https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
>>>
>>>
>>> вт, 6 мар. 2018 г. в 20:17, Maxim Muzafarov :
>>>
>>> > Hello Dmitry,
>>> >
>>> > Yes, I've updated test classes as you metioned before.
>>> > Now i'm fixing review comments. Within next few days I'll prepare final
>>> > version of this PR.
>>> >
>>> > вт, 6 мар. 2018 г. в 20:12, Dmitry Pavlov :
>>> >
>>> > > Hi Maxim,
>>> > >
>>> > > are there any news on these test fails?
>>> > >
>>> > > Is issue ready for review?
>>> > >
>>> > > Sincerely,
>>> > > Dmitiry Pavlov
>>> > >
>>> > > вт, 27 февр. 2018 г. в 17:12, Dmitry Pavlov :
>>> > >
>>> > > > Hi, thank you!
>>> > > >
>>> > > > I've found several suspicious fails: such test fails have rate less
>>> > than
>>> > > > 1%, it is probably new failures.
>>> > > >
>>> > > > It would be great if we can fix it before merge. Could you address
>>> this
>>> > > > fails?
>>> > > >
>>> > > > Sincerely,
>>> > > > Dmitriy Pavlov
>>> > > >
>>> > > > IgniteCacheTestSuite5: IgniteCacheStoreCollectionTest.testStoreMap
>>> > (fail
>>> > > > rate 0,0%)
>>> > > > IgniteCacheTestSuite5:
>>> > > > CacheLateAffinityAssignmentTest.testDelayAssignmentClientJoin (fail
>>> > rate
>>> > > > 0,0%)
>>> > > > IgniteCacheWithIndexingTestSuite:
>>> > > > CacheRandomOperationsMultithreadedTest.testAtomicOffheapEviction
>>> (fail
>>> > > rate
>>> > > > 0,0%)
>>> > > > IgniteCacheWithIndexingTestSuite:
>>> > > >
>>> >
>>> CacheRandomOperationsMultithreadedTest.testAtomicOffheapEvictionIndexing
>>> > > > (fail rate 0,0%)
>>> > > > IgniteCacheWithIndexingTestSuite:
>>> > > > CacheRandomOperationsMultithreadedTest.testTxOffheapEviction (fail
>>> rate
>>> > > > 0,0%)
>>> > > > IgniteCacheWithIndexingTestSuite:
>>> > > >
>>> CacheRandomOperationsMultithreadedTest.testTxOffheapEvictionIndexing
>>> > > (fail
>>> > > > rate 0,0%)
>>> > > >
>>> > > > IgniteBinarySimpleNameMapperCacheFullApiTestSuite:
>>> > > >
>>> > >
>>> >
>>> GridCachePartitionedNearDisabledMultiNodeWithGroupFullApiSelfTest.testWithSkipStoreTx
>>> > > > (fail rate 0,0%)
>>> > > >
>>> > > > вт, 27 февр. 2018 г. в 17:04, Maxim Muzafarov >> >:
>>> > > >
>>> > > >> Yep, link may not be correct.
>>> > > >>
>>> > > >> Here is correct version:
>>> > > >> TC: *
>>> > > >>
>>> > >
>>> >
>>> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&branch_IgniteTests24Java8=pull%2F3542%2Fhead
>>> > > >> <
>>> > > >>
>>> > >
>>> >
>>> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&branch_IgniteTests24Java8=pull%2F3542%2Fhead
>>> > > >> >*
>>> > > >>
>>> > > >>
>>> > > >> вт, 27 февр. 2018 г. в 16:41, Dmitry Pavlov <
>>> dpavlov@gmail.com>:
>>> > > >>
>>> > > >> > Hi Maxim,
>>> > > >> >
>>> > > >> > could you please provide link to TC run on your PR? It seems
>>> link
>>> > > >> provided
>>> > > >> > points to run of master. In changes field you may select
>>> > > pull/3542/head
>>> > > >> > before starting RunAll.
>>> > > >> >
>>> > > >> > Igniters,
>>> > > >> >
>>> > > >> > this change is related to our test framework, so change may
>>> affect
>>> > > your
>>> > > >> > tests. Please join to review
>>> > > >> > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
>>> > > >> >
>>> > > >> > Sincerely,
>>> > > >> > Dmitriy Pavlov
>>> > > >> >
>>> > > >> > вт, 27 февр. 2018 г. в 16:14, Maxim Muzafarov <
>>> maxmu...@gmail.com>:
>>> > > >> >
>>> > > >> > > Hi all,
>>> > > >> > >
>>> > > >> > > I think, I've done with this issue, what should we do next?
>>> > > >> > >
>>> > > >> > > PR: https://github.com/apache/ignite/pull/3542
>>> > > >> > > Upsource:
>>> > > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
>>> > > >> > > TC:
>>> > > >> > >
>>> > > >>

[GitHub] ignite pull request #3638: IGNITE-7916: GA Grid examples should be ready for...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3638


---

Re: Stop nodes after test by default - IGNITE-6842

[Maxim Muzafarov]

Dmitry and other igniters,

Will you have time to review this issue?
I've preperated PR and TC for this, also I've fixed all comments made by
Andrey Kuznetsov and Vyacheslav Daradur.

I think this is important issue and will make test framework more stable
and clear.

TC: https://ci.ignite.apache.org/viewLog.html?buildId=1138151
JIRA: https://issues.apache.org/jira/browse/IGNITE-6842
Upsource: https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
PR: https://github.com/apache/ignite/pull/3542

чт, 15 мар. 2018 г. в 13:31, Maxim Muzafarov :

> Dmtry,
>
> Can we proceed with this change?
> I've done with fixing review comments and tests that you mentioned before.
>
> TC: https://ci.ignite.apache.org/viewLog.html?buildId=1138151
> JIRA: https://issues.apache.org/jira/browse/IGNITE-6842
> Upsource: https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
> PR: https://github.com/apache/ignite/pull/3542
>
>
>
> вт, 6 мар. 2018 г. в 20:42, Dmitry Pavlov :
>
>> Ok, thank you.
>>
>> Please let me know when we can proceed with review
>> https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
>>
>>
>> вт, 6 мар. 2018 г. в 20:17, Maxim Muzafarov :
>>
>> > Hello Dmitry,
>> >
>> > Yes, I've updated test classes as you metioned before.
>> > Now i'm fixing review comments. Within next few days I'll prepare final
>> > version of this PR.
>> >
>> > вт, 6 мар. 2018 г. в 20:12, Dmitry Pavlov :
>> >
>> > > Hi Maxim,
>> > >
>> > > are there any news on these test fails?
>> > >
>> > > Is issue ready for review?
>> > >
>> > > Sincerely,
>> > > Dmitiry Pavlov
>> > >
>> > > вт, 27 февр. 2018 г. в 17:12, Dmitry Pavlov :
>> > >
>> > > > Hi, thank you!
>> > > >
>> > > > I've found several suspicious fails: such test fails have rate less
>> > than
>> > > > 1%, it is probably new failures.
>> > > >
>> > > > It would be great if we can fix it before merge. Could you address
>> this
>> > > > fails?
>> > > >
>> > > > Sincerely,
>> > > > Dmitriy Pavlov
>> > > >
>> > > > IgniteCacheTestSuite5: IgniteCacheStoreCollectionTest.testStoreMap
>> > (fail
>> > > > rate 0,0%)
>> > > > IgniteCacheTestSuite5:
>> > > > CacheLateAffinityAssignmentTest.testDelayAssignmentClientJoin (fail
>> > rate
>> > > > 0,0%)
>> > > > IgniteCacheWithIndexingTestSuite:
>> > > > CacheRandomOperationsMultithreadedTest.testAtomicOffheapEviction
>> (fail
>> > > rate
>> > > > 0,0%)
>> > > > IgniteCacheWithIndexingTestSuite:
>> > > >
>> > CacheRandomOperationsMultithreadedTest.testAtomicOffheapEvictionIndexing
>> > > > (fail rate 0,0%)
>> > > > IgniteCacheWithIndexingTestSuite:
>> > > > CacheRandomOperationsMultithreadedTest.testTxOffheapEviction (fail
>> rate
>> > > > 0,0%)
>> > > > IgniteCacheWithIndexingTestSuite:
>> > > > CacheRandomOperationsMultithreadedTest.testTxOffheapEvictionIndexing
>> > > (fail
>> > > > rate 0,0%)
>> > > >
>> > > > IgniteBinarySimpleNameMapperCacheFullApiTestSuite:
>> > > >
>> > >
>> >
>> GridCachePartitionedNearDisabledMultiNodeWithGroupFullApiSelfTest.testWithSkipStoreTx
>> > > > (fail rate 0,0%)
>> > > >
>> > > > вт, 27 февр. 2018 г. в 17:04, Maxim Muzafarov :
>> > > >
>> > > >> Yep, link may not be correct.
>> > > >>
>> > > >> Here is correct version:
>> > > >> TC: *
>> > > >>
>> > >
>> >
>> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&branch_IgniteTests24Java8=pull%2F3542%2Fhead
>> > > >> <
>> > > >>
>> > >
>> >
>> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&branch_IgniteTests24Java8=pull%2F3542%2Fhead
>> > > >> >*
>> > > >>
>> > > >>
>> > > >> вт, 27 февр. 2018 г. в 16:41, Dmitry Pavlov > >:
>> > > >>
>> > > >> > Hi Maxim,
>> > > >> >
>> > > >> > could you please provide link to TC run on your PR? It seems link
>> > > >> provided
>> > > >> > points to run of master. In changes field you may select
>> > > pull/3542/head
>> > > >> > before starting RunAll.
>> > > >> >
>> > > >> > Igniters,
>> > > >> >
>> > > >> > this change is related to our test framework, so change may
>> affect
>> > > your
>> > > >> > tests. Please join to review
>> > > >> > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
>> > > >> >
>> > > >> > Sincerely,
>> > > >> > Dmitriy Pavlov
>> > > >> >
>> > > >> > вт, 27 февр. 2018 г. в 16:14, Maxim Muzafarov <
>> maxmu...@gmail.com>:
>> > > >> >
>> > > >> > > Hi all,
>> > > >> > >
>> > > >> > > I think, I've done with this issue, what should we do next?
>> > > >> > >
>> > > >> > > PR: https://github.com/apache/ignite/pull/3542
>> > > >> > > Upsource:
>> > > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-502
>> > > >> > > TC:
>> > > >> > >
>> > > >> > >
>> > > >> >
>> > > >>
>> > >
>> >
>> https://ci.ignite.apache.org/viewModification.html?modId=723895&personal=false&buildTypeId=&tab=vcsModificationTests
>> > > >> > > JIRA: https://issues.apache.org/jira/browse/IGNITE-6842
>> > > >> > >
>> > > >> > >
>> > > >> > > чт, 22 февр. 2018 г. в 14:12, Dmitry Pavlov <
>> > dpavlov@gmail.com
>> > > >:
>> > > >> > >
>> > > >> > > > Hi Maxim,
>> > > >>

[GitHub] ignite pull request #3656: IGNITE-7754 dirty fix for FSYNC on archivation

[glukos]

GitHub user glukos opened a pull request:

https://github.com/apache/ignite/pull/3656

IGNITE-7754 dirty fix for FSYNC on archivation



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7754

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3656.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3656


commit 8d7b64da584e95d951d89c357b8d96f41c173a0d
Author: Ivan Rakov 
Date:   2018-03-19T12:11:21Z

IGNITE-7754 dirty fix for FSYNC on archivation




---

Re: Ignite-7640 Refactor DiscoveryDataClusterState to be immutable (Done)

[Dmitry Pavlov]

Unfortunately in these tests a lot of timeouts occurred.

Could you please merge master into your banch and re-run tests?

In this case we could be sure failures is not caused by current patch
changes.

Sincerely,
Dmitriy Pavlov

пт, 16 мар. 2018 г. в 16:40, Dmitry Pavlov :

> Hi, It seems run all here was outdated, triggered one more run
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&branch_IgniteTests24Java8=pull%2F3515%2Fhead
>
>
> пн, 26 февр. 2018 г. в 13:22, Александр Меньшиков :
>
>> Hi to all.
>>
>> I have done issue ignite-7640. Please review.
>>
>>
>> JIRA: https://issues.apache.org/jira/browse/IGNITE-7640
>> PR: https://github.com/apache/ignite/pull/3515
>> TC:
>>
>> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&branch_IgniteTests24Java8=pull%2F3515%2Fhead
>>  CR: https://reviews.ignite.apache.org/ignite/review/IGNT-CR-492
>>
>

Re: IGNITE-5357 is ready for review (Replicated cache reads load balancing)

[Dmitry Pavlov]

Hi Vyacheslav,

I've raised 1 concern and several questions in
https://reviews.ignite.apache.org/ignite/review/IGNT-CR-509

Could you please address?

Sincerely,
Dmitriy Pavlov

чт, 15 мар. 2018 г. в 13:30, Vyacheslav Daradur :

> Dmitry, I'm looking forward to the news.
>
> Thanks in advance!
>
> On Thu, Mar 15, 2018 at 1:10 PM, Dmitry Pavlov 
> wrote:
> > I would like to take a look to code now, if you don't mind.
> >
> > But I need to check if there is critical bug introduced to Ignite by my
> > recent fix. So I need some time to research bug, and then can come back
> to
> > review.
> >
> > чт, 15 мар. 2018 г. в 13:04, Vyacheslav Daradur :
> >>
> >> So, what's the next step?
> >>
> >> On Thu, Mar 15, 2018 at 12:27 AM, Dmitry Pavlov 
> >> wrote:
> >> > Yes, I think I could move IgniteReproducingSuite to dev-utils module
> >> > later.
> >> > Thank you for this idea.
> >> >
> >> > Yes, It is probably it was Queries test flaky'ness.
> >> >
> >> > I hope Vladimir, you will find some time to make query tests more
> >> > stable. It
> >> > is not friendly to community members if their patches are rejected by
> >> > reasons not related to their change.
> >> >
> >> > Any assistance from the rest of community here is also appreciated.
> >> >
> >> > ср, 14 мар. 2018 г. в 22:24, Vyacheslav Daradur  >:
> >> >>
> >> >> Thank you for the advice!
> >> >>
> >> >> Unfortunately, *IgniteReproducingSuite* is in the core module while
> >> >> *IgniteSqlSplitterSelfTest* in the ignite-indexing module that means
> I
> >> >> am not able to add the test in this test suite without addition
> >> >> cycling dependency.
> >> >>
> >> >> I'd recommend you detaching *IgniteReproducingSuite* as a separate
> >> >> module in the project to include the test suites from any module in
> >> >> the project.
> >> >>
> >> >>
> >> >> But I've prepared *Ignite Queries* in the same way as you suggested
> in
> >> >> *IgniteReproducingSuite* [1] and ran all tests in
> >> >> *IgniteSqlSplitterSelfTest* 100 times [2].
> >> >>
> >> >> >> IgniteBinaryCacheQueryTestSuite:
> >> >> >>
> >> >> >>
> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
> >> >> >> (fail rate 0,0%)
> >> >> For this test "Green lite" 100 times of 100.
> >> >>
> >> >> Green lite for all tests in *IgniteSqlSplitterSelfTest* in the latest
> >> >> build of main PR [3].
> >> >>
> >> >>
> >> >> [1]
> >> >>
> >> >>
> https://github.com/daradurvs/ignite/blob/fd6abc915838599c2ebab3f803f90f2e641e8892/modules/indexing/src/test/java/org/apache/ignite/testsuites/IgniteCacheQuerySelfTestSuite.java
> >> >> [2] https://ci.ignite.apache.org/viewLog.html?buildId=1136780
> >> >> [3] https://ci.ignite.apache.org/viewLog.html?buildId=1136685
> >> >>
> >> >> On Wed, Mar 14, 2018 at 7:55 PM, Dmitry Pavlov <
> dpavlov@gmail.com>
> >> >> wrote:
> >> >> > It is possible that test is failing only on agents and is always
> >> >> > successfull
> >> >> > locally.
> >> >> >
> >> >> > For researching such test there was "Ignite reproducing suite"
> >> >> > introduced
> >> >> > early. This suite intentionally left blank on TC. Correspondent
> suite
> >> >> > in
> >> >> > code is IgniteReproducingSuite.
> >> >> >
> >> >> > You may add some extra debug info into test. Add this test in
> >> >> > IgniteReproducingSuite in code and then start suite on TC several
> >> >> > times.
> >> >> >
> >> >> > ср, 14 мар. 2018 г. в 19:42, Vyacheslav Daradur
> >> >> > :
> >> >> >>
> >> >> >> Dmitry, as I've written here before: I checked this test locally,
> >> >> >> many
> >> >> >> times (didn't have any falling on 100 starts).
> >> >> >>
> >> >> >> On Wed, Mar 14, 2018 at 7:31 PM, Dmitry Pavlov
> >> >> >> 
> >> >> >> wrote:
> >> >> >> > Hi, I've found test which never failed on master, but fails in
> >> >> >> > branch
> >> >> >> >
> >> >> >> >  Ignite Queries [ tests 1 ]
> >> >> >> >
> >> >> >> >  IgniteBinaryCacheQueryTestSuite:
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> >
> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCacheSegmentedClient
> >> >> >> > (fail rate 0,0%)
> >> >> >> >
> >> >> >> >
> >> >> >> > ср, 14 мар. 2018 г. в 19:26, Dmitry Pavlov
> >> >> >> > :
> >> >> >> >>
> >> >> >> >> Hi, let me check TC run
> >> >> >> >>
> >> >> >> >> вт, 13 мар. 2018 г. в 9:22, Vyacheslav Daradur
> >> >> >> >> :
> >> >> >> >>>
> >> >> >> >>> Dmitry,
> >> >> >> >>>
> >> >> >> >>> Nickolay accepted PR changes at Upsource [1].
> >> >> >> >>>
> >> >> >> >>> Latest ci.build [2] looks good in comparison with master [3].
> >> >> >> >>>
> >> >> >> >>> Following tests passed locally:
> >> >> >> >>>
> CacheAffinityCallSelfTest.testAffinityCallFromClientRestartNode
> >> >> >> >>> CacheAffinityCallSelfTest.testAffinityCallRestartNode
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>>
> IgniteOptimisticTxSuspendResumeMultiServerTest.testTxTimeoutOnSuspend
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>>
> IgniteSqlSplitterSelfTest.testReplicatedTablesUsingPartitionedCache

[GitHub] ignite pull request #3655: IGNITE-7984

[devozerov]

GitHub user devozerov opened a pull request:

https://github.com/apache/ignite/pull/3655

IGNITE-7984



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7984

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3655.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3655






---

[jira] [Created] (IGNITE-7984) DML: improve deadlock handling

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-7984:
---

 Summary: DML: improve deadlock handling 
 Key: IGNITE-7984
 URL: https://issues.apache.org/jira/browse/IGNITE-7984
 Project: Ignite
  Issue Type: Task
  Components: sql
Affects Versions: 2.4
Reporter: Vladimir Ozerov
 Fix For: 2.5


Current DML implementation is not transactional. It groups keys in batches by 
their affinity, and then flushes that batches synchronously. This could lead to 
deadlocks easily.

This could be improved if we sort key within a batch. However, this would 
require new comparison function for {{BinaryObjectImpl}}, as it is not 
comparable.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3654: IGNITE-7581 avoiding ConcurrentModificationExcept...

[sergey-chugunov-1985]

GitHub user sergey-chugunov-1985 opened a pull request:

https://github.com/apache/ignite/pull/3654

IGNITE-7581 avoiding ConcurrentModificationException when timeout worker 
handles tx



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-7581

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3654.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3654






---

[GitHub] ignite pull request #3643: IGNITE-7811: ODBC: Implemented connection failove...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3643


---

Re: (Partition Map) Exchange at wiki

[Yakov Zhdanov]

Awesome article, Dmitry!

Denis Magda, should we put a link to it from apacheignite.readme.io?

--Yakov

[jira] [Created] (IGNITE-7983) NPE in TxRollbackOnTimeoutNearCacheTest.testRandomMixedTxConfigurations

[Andrey Kuznetsov (JIRA)]

Andrey Kuznetsov created IGNITE-7983:


 Summary: NPE in 
TxRollbackOnTimeoutNearCacheTest.testRandomMixedTxConfigurations
 Key: IGNITE-7983
 URL: https://issues.apache.org/jira/browse/IGNITE-7983
 Project: Ignite
  Issue Type: Task
Affects Versions: 2.4
Reporter: Andrey Kuznetsov
Assignee: Andrey Kuznetsov
 Fix For: 2.5


{{get}} inside transaction sometimes returns {{null}}. This should be 
impossible.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: IEP-14: Ignite failures handling (Discussion)

[Yakov Zhdanov]

Andrey Gura,

Why should we have any FailureHandler abstraction? We already have it -
this is EventListener. In my view it is better (and cleaner design) to add
events (similar to, for
example, org.apache.ignite.events.EventType#EVT_NODE_SEGMENTED) like
EVT_IGNITE_OOME, EVT_SYS_WORKER_FAILED and fire events accordingly to the
situation + execute configured system logic. We have exactly same way with
segmentation. We have policy which defines how system reacts and also allow
user to add event listeners.

For better understanding please take a look
at org.apache.ignite.plugin.segmentation.SegmentationPolicy
and 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.DiscoveryWorker#onSegmentation.
Discovery manager records the event (allowing user to get notification on
it) and executes internal logic in case segmentation policy is not NOOP.

Thanks!

--Yakov

Re: IGNITE-6879

[Dmitry Pavlov]

Hi Alexey,

Did you find the patch is looking good and is ready to be merged?

Sincerely,
Dmitriy Pavlov

чт, 15 мар. 2018 г. в 11:19, Alexey Kukushkin :

> Just found the fix is ready - I will review it today or tomorrow.
>

[jira] [Created] (IGNITE-7982) IgniteRepositoryFactory are unable to instantiate non-lazy repository

[Pavel Vinokurov (JIRA)]

Pavel Vinokurov created IGNITE-7982:
---

 Summary: IgniteRepositoryFactory are unable to instantiate 
non-lazy repository
 Key: IGNITE-7982
 URL: https://issues.apache.org/jira/browse/IGNITE-7982
 Project: Ignite
  Issue Type: Bug
  Components: spring
Affects Versions: 2.4
Reporter: Pavel Vinokurov






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: MTCGA: IGNITE-7791 and GridDhtPartitionsSingleMessage

[Alexey Goncharuk]

Hello Maxim,

SingleMessage with exchId=null is sent when a node updates local
partitions' state and schedules a background cluster notification. In
contrast, when a partition map exchange happends, it is completed with
exchId != null.

I need more context regarding how this message interferes with the exchange
and what the difference between the two messages is so that during the
regular scenario the assertion does not happen.

2018-03-13 20:58 GMT+03:00 Maxim Muzafarov :

> Hi all,
>
> I'm working on [1] IgniteClientReconnectCacheTest class with frakly
> test-case testReconnectCacheDestroyedAndCreated with success rate 32.4%.
>
> I've leaved comment in JIRA [2] and new test-case with reproducing this
> issue.
> Basicly, when we receiving GridDhtPartitionsSingleMessage with exchId=null
> not
> in proper time we've got this Assertion error. Ignite client instance
> erases all it's caches after reconnect, so it has no information about
> cache named 'static-cache' that persists on server nodes and when he
> recieve this SignleMessage after reconnection it will have 'Failed to
> reinitialize local partitions (preloading will be stopped)'.
>
> Should we perform clean-up [3] client caches in case of reconnect client
> ignite instance?
> Why we should clean clinent caches after node reconnects? Can't catch the
> idea of it.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-7791
> [2]
> https://issues.apache.org/jira/browse/IGNITE-7791?
> focusedCommentId=16391409&page=com.atlassian.jira.
> plugin.system.issuetabpanels:comment-tabpanel#comment-16391409
> [3]
> https://github.com/apache/ignite/blob/master/modules/
> core/src/main/java/org/apache/ignite/internal/processors/cache/
> CacheAffinitySharedManager.java#L190
>

[jira] [Created] (IGNITE-7981) Thread hangs on ignite.cluster().active(false) method call

[Oleg Ostanin (JIRA)]

Oleg Ostanin created IGNITE-7981:


 Summary: Thread hangs on ignite.cluster().active(false) method call
 Key: IGNITE-7981
 URL: https://issues.apache.org/jira/browse/IGNITE-7981
 Project: Ignite
  Issue Type: Bug
Reporter: Oleg Ostanin


ignite.cluster().active(false)

After calling this method thread hangs for more than a few hours.

There is thread dump from client node:

"CheckAffinityTask-#62%TaskPool%" #99 prio=5 os_prio=0 tid=0x7f19dd861000 
nid=0x8f5 waiting on condition [0x7f18e5001000]
 java.lang.Thread.State: WAITING (parking)
 at sun.misc.Unsafe.park(Native Method)
 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
 at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
 at 
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
 at 
org.apache.ignite.internal.cluster.IgniteClusterImpl.active(IgniteClusterImpl.java:312)
 at 
org.apache.ignite.scenario.internal.AbstractTask.copyWorkDirs(AbstractTask.java:757)
 at 
org.apache.ignite.scenario.internal.AbstractTask.stopAndCollect(AbstractTask.java:750)
 - locked <0x0006408d3b68> (a java.lang.Class for 
org.apache.ignite.scenario.internal.AbstractTask)
 at 
org.apache.ignite.scenario.CheckAffinityTask.body0(CheckAffinityTask.java:111)
 at org.apache.ignite.scenario.internal.AbstractTask.body(AbstractTask.java:322)
 at org.apache.ignite.scenario.internal.TaskLooper.run(TaskLooper.java:85)
 at java.lang.Thread.run(Thread.java:748)

 

Thread dumps from server nodes can be found here:

https://drive.google.com/open?id=1fGmVrQ5Q5uggMadbdW0ChoYL5JW18T7K



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] ignite pull request #3585: IGNITE-7029 Add an ability to provide multiple co...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3585


---

[GitHub] ignite pull request #3392: ZookeeperDiscovery implementation

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3392


---