[jira] [Created] (IGNITE-11427) Document custom node fail functional.

[Stanilovsky Evgeny (JIRA)]

Stanilovsky Evgeny created IGNITE-11427:
---

 Summary: Document custom node fail functional.
 Key: IGNITE-11427
 URL: https://issues.apache.org/jira/browse/IGNITE-11427
 Project: Ignite
  Issue Type: Improvement
  Components: documentation
Affects Versions: 2.7
Reporter: Stanilovsky Evgeny
Assignee: Artem Budnikov
 Fix For: 2.8
 Attachments: Screenshot_20190227_100539.png

Append additional node fail documentation related to [1]

[1] https://issues.apache.org/jira/browse/IGNITE-11332

 

how it looks into jconsole:

!Screenshot_20190227_100539.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[MTCGA]: new failures in builds [3096664] needs to be handled

[dpavlov . tasks]

Hi Igniters,

 I've detected some new issue on TeamCity to be handled. You are more than 
welcomed to help.

 If your changes can lead to this failure(s): We're grateful that you were a 
volunteer to make the contribution to this project, but things change and you 
may no longer be able to finalize your contribution.
 Could you respond to this email and indicate if you wish to continue and fix 
test failures or step down and some committer may revert you commit. 

 *Recently contributed test failed in master 
tcp.TcpDiscoverySpiFailureTimeoutSelfTest 
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=3316347121927346093=%3Cdefault%3E=testDetails
 No changes in the build

 - Here's a reminder of what contributors were agreed to do 
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute 
 - Should you have any questions please contact dev@ignite.apache.org 

Best Regards,
Apache Ignite TeamCity Bot 
https://github.com/apache/ignite-teamcity-bot
Notification generated at 08:11:28 27-02-2019 

Re: On Multiple Endpoints Mode of JDBC Driver

[Denis Magda]

Hello,

You provide a list of IP addresses for the sake of high-availability - if
one of the servers goes down then the client will reconnect to the next IP
automatically. There is no any load balancing in place presently. But! In
the next Ignite version, we're planning to roll out partition-awareness
support - the client will send a request to the nodes who hold the data
needed for the request.

-
Denis


On Tue, Feb 26, 2019 at 2:48 PM 李玉珏  wrote:

> Hi,
>
> Does have load balancing function in Multiple Endpoints mode of JDBC
> driver?For example, "jdbc: ignite: thin://192.168.0.50:101,
> 192.188.5.40:101, 192.168.10.230:101"
> If not, will one node become the bottleneck of the whole system?
>

On Multiple Endpoints Mode of JDBC Driver

[李玉珏]

Hi,

Does have load balancing function in Multiple Endpoints mode of JDBC driver?For 
example, "jdbc: ignite: thin://192.168.0.50:101, 192.188.5.40:101, 
192.168.10.230:101"
If not, will one node become the bottleneck of the whole system?

Re: Wrong backup size with persistence + data expriation

[Denis Magda]

Evgeniy, Igniters,

Is this somehow related to the TTL issue we observed with other use cases
recently?

Ruslan, please share your configuration.

-
Denis


On Tue, Feb 26, 2019 at 5:56 AM Ilya Kasnacheev 
wrote:

> Hello!
>
> Indeed I can observe this problem:
>
> Baseline: [Server1, Server2, Server3]
> 1 records inserted
> 2019-02-26T16:50:00.603 :: [0] all=2, primary=1, backup=1,
> query=1
> 2019-02-26T16:50:10.692 :: [1] all=2, primary=1, backup=1,
> query=1
> 2019-02-26T16:50:20.750 :: [2] all=2, primary=1, backup=1,
> query=1
> 2019-02-26T16:50:30.775 :: [3] all=2, primary=1, backup=1,
> query=1
> 2019-02-26T16:50:40.851 :: [4] all=18564, primary=9285, backup=9262,
> query=9122
> 2019-02-26T16:50:50.897 :: [5] all=10143, primary=5073, backup=5061,
> query=4752
> 2019-02-26T16:51:00.925 :: [6] all=173, primary=86, backup=72, query=0
> 2019-02-26T16:51:10.934 :: [7] all=9, primary=0, backup=9, query=0
> 2019-02-26T16:51:20.952 :: [8] all=9, primary=0, backup=9, query=0
> 2019-02-26T16:51:30.964 :: [9] all=9, primary=0, backup=9, query=0
> 2019-02-26T16:51:40.976 :: [10] all=9, primary=0, backup=9, query=0
> 2019-02-26T16:51:50.988 :: [11] all=9, primary=0, backup=9, query=0
> 2019-02-26T16:52:01.015 :: [12] all=9, primary=0, backup=9, query=0
> 2019-02-26T16:52:11.040 :: [13] all=9, primary=0, backup=9, query=0
> 2019-02-26T16:52:21.053 :: [14] all=9, primary=0, backup=9, query=0
> 2019-02-26T16:52:31.072 :: [15] all=9, primary=0, backup=9, query=0
> [16:52:35] Ignite node stopped OK [uptime=00:03:02.734]
>
> As far as my understanding goes, all entries should expire and both primary
> and backup be reduced to zero, but it does not happen.
>
> Can anyone comment this? Should a ticket be filed?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 8 февр. 2019 г. в 22:53, Ruslan Kamashev :
>
> > Hi,
> >
> > I try to use data expiration with persistence, but I got wrong backup
> size
> > after cache records are expired.
> > Also sometimes I got result when expired records didn't deleted but I
> > can't reproduce this case again.
> >
> > Could you share more details about using data expiration with
> persistence?
> >
> > Source: https://github.com/rynffoll/ignite-tests
> > Logs:
> >
> https://github.com/rynffoll/ignite-tests/tree/master/logs/persistance_with_data_expiration
> >
> >
>

Re: Flaky tests

[Denis Magda]

Hello Stanislav,

Ignite also has some issues with Java 11 support. For a long time we have
been ignoring Java 11 existence at all. At the moment, the community has
been working on the full Java 11 support and this version should become a
default one soon. The tests will be done for both Java 8 and Java 11.

I'll let Dmitry Pavlov dig into the details since he is the one who is
driving or doing most of the adoption.

-
Denis


On Tue, Feb 26, 2019 at 2:39 PM Stanislav Kozlovski <
stanislav_kozlov...@outlook.com> wrote:

> Hey there Ignite community,
>
> I contribute to a fellow open-source project - Apache Kafka - and there we
> have been fighting flaky tests a lot. We run Java 8 and Java 11 builds on
> every Pull Request and due to test flakiness, almost all of them turn out
> red with 1 or 2 tests (completely unrelated to the change in the PR)
> failing. This has resulted in committers either ignoring them and merging
> the changes or in the worst case rerunning the hour-long build until it
> becomes green.
> This test flakiness has also slowed down our releases significantly.
>
> In general, I was just curious to understand if this is a problem that
> your project faces as well. Does your project have a lot of intermittently
> failing tests, do you have any active process of addressing such tests
> (during the initial review, after realizing it is flaky, etc). Any pointers
> will be greatly appreciated!
>
> Thanks,
> Stanislav
>
>
> 
>

Flaky tests

[Stanislav Kozlovski]

Hey there Ignite community,

I contribute to a fellow open-source project - Apache Kafka - and there we have 
been fighting flaky tests a lot. We run Java 8 and Java 11 builds on every Pull 
Request and due to test flakiness, almost all of them turn out red with 1 or 2 
tests (completely unrelated to the change in the PR) failing. This has resulted 
in committers either ignoring them and merging the changes or in the worst case 
rerunning the hour-long build until it becomes green.
This test flakiness has also slowed down our releases significantly.

In general, I was just curious to understand if this is a problem that your 
project faces as well. Does your project have a lot of intermittently failing 
tests, do you have any active process of addressing such tests (during the 
initial review, after realizing it is flaky, etc). Any pointers will be greatly 
appreciated!

Thanks,
Stanislav

[jira] [Created] (IGNITE-11426) Denial of Service Attack Vulnerability

[Gabriel Jimenez (JIRA)]

Gabriel Jimenez created IGNITE-11426:


 Summary: Denial of Service Attack Vulnerability
 Key: IGNITE-11426
 URL: https://issues.apache.org/jira/browse/IGNITE-11426
 Project: Ignite
  Issue Type: Bug
Affects Versions: 2.6
Reporter: Gabriel Jimenez


{{*Problem Statement*: The DiscoverySPI and CommunicationSPI have components 
that listen on open ports (Various GridNIOServer(Communication) and 
SocketReader(Discovery) instances). These open ports result on a vulnerability 
to denial of service attacks. Even more concerning is the fact that the 
rejection behavior for GridNIOServer relies on asserting instanceof for the 
incoming message (subsequently throwing an exception on failed assertion). This 
is relatively costly computationally, and can lead to OutOfMemory issues for 
the node JVM. Additionally, the exception is not properly handled by the 
GridNIOServer instances, and can result in error messages:}}

{{"}}
{{[ERROR] [grid-nio-worker-client-listener-0-#110] ClientListenerProcessor - 
Closing NIO session because of unhandled exception. 
org.apache.ignite.IgniteCheckedException: Invalid handshake message at 
org.apache.ignite.internal.processors.odbc.ClientListenerNioServerBuffer.read(ClientListenerNioServerBuffer.java:115)
 ~[bdp-ignite-core-2.6.0.jar:2.6.0] at 
org.apache.ignite.internal.processors.odbc.ClientListenerBufferedParser.decode(ClientListenerBufferedParser.java:60)
 ~[bdp-ignite-core-2.6.0.jar:2.6.0] at 
org.apache.ignite.internal.processors.odbc.ClientListenerBufferedParser.decode(ClientListenerBufferedParser.java:40)
 ~[bdp-ignite-core-2.6.0.jar:2.6.0] at 
org.apache.ignite.internal.util.nio.GridNioCodecFilter.onMessageReceived(GridNioCodecFilter.java:114)
 ~[bdp-ignite-core-2.6.0.jar:2.6.0] at 
org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
 ~[bdp-ignite-core-2.6.0.jar:2.6.0] at 
org.apache.ignite.internal.util.nio.GridNioServer$HeadFilter.onMessageReceived(GridNioServer.java:3490)
 ~[bdp-ignite-core-2.6.0.jar:2.6.0] at 
org.apache.ignite.internal.util.nio.GridNioFilterChain.onMessageReceived(GridNioFilterChain.java:175)
 ~[bdp-ignite-core-2.6.0.jar:2.6.0] at 
org.apache.ignite.internal.util.nio.GridNioServer$ByteBufferNioClientWorker.processRead(GridNioServer.java:1113)
 [bdp-ignite-core-2.6.0.jar:2.6.0] at 
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2339)
 [bdp-ignite-core-2.6.0.jar:2.6.0] at 
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2110)
 [bdp-ignite-core-2.6.0.jar:2.6.0] at 
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1764)
 [bdp-ignite-core-2.6.0.jar:2.6.0] at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) 
[bdp-ignite-core-2.6.0.jar:2.6.0] at java.lang.Thread.run(Thread.java:748) 
[?:1.8.0_172]}}
{{"}}

{{Relevant Lines:}}
{{[https://github.com/apache/ignite/blob/ignite-2.6/modules/core/src/main/java/org/apache/ignite/spi/communication/tcp/TcpCommunicationSpi.java#L483]}}

[https://github.com/apache/ignite/blob/ignite-2.6/modules/core/src/main/java/org/apache/ignite/spi/communication/tcp/TcpCommunicationSpi.java#L541]

 

*Solution*: On our internal build we opted to replace the assert statements 
with conditionals to simply close the session and log a warning if the incoming 
message isn't of the expected type. This approach is present throughout other 
parts of the codebase, thus it seemed fitting.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-11425) Log information about inaccessible nodes through Communication

[Vladislav Pyatkov (JIRA)]

Vladislav Pyatkov created IGNITE-11425:
--

 Summary: Log information about inaccessible nodes through 
Communication
 Key: IGNITE-11425
 URL: https://issues.apache.org/jira/browse/IGNITE-11425
 Project: Ignite
  Issue Type: Improvement
Reporter: Vladislav Pyatkov


In case of long getting communication TCP client (longe than this 
CONNECTION_ESTABLISH_THRESHOLD_MS = 100) message will printed:

{noformat}
[sys-#20167%dht.CacheGetReadFromBackupFailoverTest0%][TcpCommunicationSpi] TCP 
client created [client=GridTcpNioCommunicationClient 
[ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker 
[super=AbstractNioClientWorker [idx=3, bytesRcvd=0, bytesSent=0, bytesRcvd0=0, 
bytesSent0=0, select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-3, 
igniteInstanceName=dht.CacheGetReadFromBackupFailoverTest0, finished=false, 
heartbeatTs=1550512236151, hashCode=140561231, interrupted=false, 
runner=grid-nio-worker-tcp-comm-3-#20147%dht.CacheGetReadFromBackupFailoverTest0%]]],
 writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], 
readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], 
inRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=0, 
sentCnt=0, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode 
[id=8a660330-6ddb-4031-b955-4cb4f4b2, addrs=ArrayList [127.0.0.1], 
sockAddrs=HashSet [/127.0.0.1:47502], discPort=47502, order=5, intOrder=4, 
lastExchangeTime=1550512235890, loc=false, ver=2.8.0#20190218-sha1:29232e37, 
isClient=false], connected=false, connectCnt=2, queueLimit=4096, reserveCnt=2, 
pairedConnections=false], outRecovery=GridNioRecoveryDescriptor [acked=0, 
resendCnt=0, rcvCnt=0, sentCnt=0, reserved=true, lastAck=0, nodeLeft=false, 
node=TcpDiscoveryNode [id=8a660330-6ddb-4031-b955-4cb4f4b2, addrs=ArrayList 
[127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47502], discPort=47502, order=5, 
intOrder=4, lastExchangeTime=1550512235890, loc=false, 
ver=2.8.0#20190218-sha1:29232e37, isClient=false], connected=false, 
connectCnt=2, queueLimit=4096, reserveCnt=2, pairedConnections=false], 
super=GridNioSessionImpl [locAddr=/127.0.0.1:38770, rmtAddr=/127.0.0.1:45212, 
createTime=1550512236151, closeTime=0, bytesSent=0, bytesRcvd=0, bytesSent0=0, 
bytesRcvd0=0, sndSchedTime=1550512236151, lastSndTime=1550512236151, 
lastRcvTime=1550512236151, readsPaused=false, 
filterChain=FilterChain[filters=[GridNioCodecFilter 
[parser=org.apache.ignite.internal.util.nio.GridDirectParser@d240a48, 
directMode=true], GridConnectionBytesVerifyFilter], accepted=false, 
markedForClose=false]], super=GridAbstractCommunicationClient 
[lastUsed=1550512236151, closed=false, connIdx=0]], duration=211ms]
{noformt}

but in some cases we can not to get client during time out, and the message 
reduce to
 
TCP client created [client=null, duration=60004 ms]

According to the message you cannot understand which nodes were inaccessible.
Moreover, wants to see the connection trouble earlier than the 10 minutes after.

Should to log ip/host for clear understanding what was the node and log WARN 
message each time when need to increase timeout:
{code}
if (lastWaitingTimeout < 6)
  lastWaitingTimeout *= 2;
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Migration to JUnit 5

[Ivan Fedotov]

Ivan,
I will investigate GridAbstractTest refactoring issue more precisely when I
finish with JUnit3Legacy classes. Anyway, I will keep in touch with you and
the community on the most significant moments.

JUnit5 docs say that functionality is not full "especially with regard to
reporting". On the other hand, I also agree with docs that it is the
easiest way that does not require to touch CI infrastructure. I am going to
try @RunWith(JUnitPlatform.class) construction with features from IEP to
make sure that we will have the full support of them. The alternative way
is dynamic tests [1], but the problem is that we add methods to suites
manually, not via @Test annotation. It is some kind of rollback to JUnit3
syntax.

Anton,
thank you for the reminder, I will update IEP according to the conversation.

[1] https://www.baeldung.com/junit5-dynamic-tests

вт, 26 февр. 2019 г. в 17:56, Anton Vinogradov :

> Folks,
>
> Please make sure you keep IEP updated and each issue mentioned.
>
> On Tue, Feb 26, 2019 at 4:28 PM Павлухин Иван  wrote:
>
> > Ivan,
> >
> > Thank you for detailed answers! I would put a great care to
> > @RunWith(JUnitPlatform.class) construction. As stated in junit5 docs
> > [1] it does not support all features and unfortunately it is not clear
> > how limited it is. Also, it is some kind of transitional mechanism
> > which was not designed for being a long term solution.
> >
> > And I fully support an idea of refactoring GridAbstractTest. I think
> > it is possible to make a significant improvement here.
> >
> > [1]
> >
> https://junit.org/junit5/docs/current/user-guide/#running-tests-junit-platform-runner
> >
> > пн, 25 февр. 2019 г. в 17:41, Ivan Fedotov :
> > >
> > > Hello Nikolay.
> > >
> > > The prime benefits are more comfortable work with flaky tests, Java 8
> > tests
> > > compatibility, user-friendly syntaxis in parametrized tests and others.
> > > The most significant features list you can find in IEP-30 Motivation
> > > section.
> > >
> > > If you have any specific questions about JUnit5 feel free to ask me.
> > >
> > > пн, 25 февр. 2019 г. в 16:55, Nikolay Izhikov :
> > >
> > > > Hello, Ivan.
> > > >
> > > > May be I miss some mail - if yes, can you repeat it.
> > > > What is advantages of migration from junit 4 to 5?
> > > > Why we should do it?
> > > >
> > > >
> > > > пн, 25 февр. 2019 г. в 16:33, Ivan Fedotov :
> > > >
> > > > > Ivan,
> > > > > That is my thoughts according to your questions.
> > > > >
> > > > > 1. I tried to implement test suits with JUnit4 compatibility layer.
> > The
> > > > > basic concept is to use @RunWith(JUnitPlatform.class)
> @SelectClasses
> > > > > ({...})[1] and on
> > > > > CI Ignite it works fine.
> > > > >
> > > > > 2. According to @Rules, there are several ways to solve it:
> > > > > 2.1 Leave JUnit4 code without changes. It will work because of
> > > > Vintage
> > > > > module
> > > > > 2.2 Rewrite the @Rule as an Extension. The work of extension is
> > > > similar
> > > > > to the @Rules work, but it is extracted in an Extension class.
> > > > > For more information about extensions, please, follow the IEP
> > [2].
> > > > > In my opinion, the easiest and the most understandable way is to
> > leave
> > > > > GridAbstractTest in current form. It will work with JUnit5
> > > > > syntaxis and abilities.
> > > > >
> > > > > 3. I faced a couple of problems during dealing with dynamic and
> > static
> > > > > tests in one project with JUnit5. The problem occurs with surefire
> > > > version:
> > > > > static tests work fine with 2.21x and earlier and with dynamic
> > tests, the
> > > > > situation is vice versa, it works with > 2.21x surefire version.
> > > > > We can use helpful surefire dependency to use static tests with the
> > > > newest
> > > > > surefire version [3], but dynamic tests become unavailable from
> pure
> > > > > Maven and accordingly from CI Ignite (from IDE all is fine).
> > > > > I can suggest leaving this type of tests on JUnit4 on the current
> > stage -
> > > > > they are in the vast minority.
> > > > >
> > > > > Let me comment on your side notes.
> > > > >
> > > > > I am not against the stable and widely-used test library usage.
> All I
> > > > want
> > > > > to say that it is not necessary in case of the main testing Ignite
> > > > > framework (Junit) already provides the mentioned features.
> > > > >
> > > > > At the initial stage of improvements 3->4 I am planning to remove
> > > > > JUnit3TestLegacyAssert, JUnit3TestLegacySupport classes. I guess
> that
> > > > > during this work
> > > > > I will face with an issue that you are mentioned - turning instance
> > > > methods
> > > > > to static. It is because of beforeTestsStarted and
> afterTestsStarted
> > > > > methods - I want to replace them by methods with BeforeAll,
> AfterAll
> > > > > annotations. But the point is that methods under such annotations
> > must be
> > > > > static. Just now I am not sure about fully removing
> > > > > GridCommonAbstractTest 

Re: Migration to JUnit 5

[Anton Vinogradov]

Folks,

Please make sure you keep IEP updated and each issue mentioned.

On Tue, Feb 26, 2019 at 4:28 PM Павлухин Иван  wrote:

> Ivan,
>
> Thank you for detailed answers! I would put a great care to
> @RunWith(JUnitPlatform.class) construction. As stated in junit5 docs
> [1] it does not support all features and unfortunately it is not clear
> how limited it is. Also, it is some kind of transitional mechanism
> which was not designed for being a long term solution.
>
> And I fully support an idea of refactoring GridAbstractTest. I think
> it is possible to make a significant improvement here.
>
> [1]
> https://junit.org/junit5/docs/current/user-guide/#running-tests-junit-platform-runner
>
> пн, 25 февр. 2019 г. в 17:41, Ivan Fedotov :
> >
> > Hello Nikolay.
> >
> > The prime benefits are more comfortable work with flaky tests, Java 8
> tests
> > compatibility, user-friendly syntaxis in parametrized tests and others.
> > The most significant features list you can find in IEP-30 Motivation
> > section.
> >
> > If you have any specific questions about JUnit5 feel free to ask me.
> >
> > пн, 25 февр. 2019 г. в 16:55, Nikolay Izhikov :
> >
> > > Hello, Ivan.
> > >
> > > May be I miss some mail - if yes, can you repeat it.
> > > What is advantages of migration from junit 4 to 5?
> > > Why we should do it?
> > >
> > >
> > > пн, 25 февр. 2019 г. в 16:33, Ivan Fedotov :
> > >
> > > > Ivan,
> > > > That is my thoughts according to your questions.
> > > >
> > > > 1. I tried to implement test suits with JUnit4 compatibility layer.
> The
> > > > basic concept is to use @RunWith(JUnitPlatform.class) @SelectClasses
> > > > ({...})[1] and on
> > > > CI Ignite it works fine.
> > > >
> > > > 2. According to @Rules, there are several ways to solve it:
> > > > 2.1 Leave JUnit4 code without changes. It will work because of
> > > Vintage
> > > > module
> > > > 2.2 Rewrite the @Rule as an Extension. The work of extension is
> > > similar
> > > > to the @Rules work, but it is extracted in an Extension class.
> > > > For more information about extensions, please, follow the IEP
> [2].
> > > > In my opinion, the easiest and the most understandable way is to
> leave
> > > > GridAbstractTest in current form. It will work with JUnit5
> > > > syntaxis and abilities.
> > > >
> > > > 3. I faced a couple of problems during dealing with dynamic and
> static
> > > > tests in one project with JUnit5. The problem occurs with surefire
> > > version:
> > > > static tests work fine with 2.21x and earlier and with dynamic
> tests, the
> > > > situation is vice versa, it works with > 2.21x surefire version.
> > > > We can use helpful surefire dependency to use static tests with the
> > > newest
> > > > surefire version [3], but dynamic tests become unavailable from pure
> > > > Maven and accordingly from CI Ignite (from IDE all is fine).
> > > > I can suggest leaving this type of tests on JUnit4 on the current
> stage -
> > > > they are in the vast minority.
> > > >
> > > > Let me comment on your side notes.
> > > >
> > > > I am not against the stable and widely-used test library usage. All I
> > > want
> > > > to say that it is not necessary in case of the main testing Ignite
> > > > framework (Junit) already provides the mentioned features.
> > > >
> > > > At the initial stage of improvements 3->4 I am planning to remove
> > > > JUnit3TestLegacyAssert, JUnit3TestLegacySupport classes. I guess that
> > > > during this work
> > > > I will face with an issue that you are mentioned - turning instance
> > > methods
> > > > to static. It is because of beforeTestsStarted and afterTestsStarted
> > > > methods - I want to replace them by methods with BeforeAll, AfterAll
> > > > annotations. But the point is that methods under such annotations
> must be
> > > > static. Just now I am not sure about fully removing
> > > > GridCommonAbstractTest class, but the need for static methods is a
> fact.
> > > >
> > > > [1]
> > > >
> > > >
> > >
> https://github.com/apache/ignite/blob/85ba3a88d661bb05bbb749bd1feaf60cd9099ddc/examples/src/test/java/org/apache/ignite/testsuites/IgniteExamplesSelfTestSuite.java#L59
> > > > [2]
> > > >
> > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-30%3A+Migration+to+JUnit+5
> > > > [3] https://github.com/junit-team/junit5/issues/1778
> > > >
> > > > вс, 24 февр. 2019 г. в 10:15, Павлухин Иван :
> > > >
> > > > > Ivan,
> > > > >
> > > > > Indeed junit5 has a lot of powerful features which can improve
> testing
> > > > > process.
> > > > >
> > > > > But first we should go through a migration process. There are
> several
> > > > > items which looks quite challenging.
> > > > > 1. Test suites support. Correct me if I am missed it, but I have
> not
> > > > > found a concept of test suites similar to junit3/4 ones. CI in
> Ignite
> > > > > heavily depends on test suites. Is there an alternative in junit5?
> > > > > 2. The majority of our tests extend GridAbstractTest which in fact
> is
> > > > > a core 

Re: Wrong backup size with persistence + data expriation

[Ilya Kasnacheev]

Hello!

Indeed I can observe this problem:

Baseline: [Server1, Server2, Server3]
1 records inserted
2019-02-26T16:50:00.603 :: [0] all=2, primary=1, backup=1,
query=1
2019-02-26T16:50:10.692 :: [1] all=2, primary=1, backup=1,
query=1
2019-02-26T16:50:20.750 :: [2] all=2, primary=1, backup=1,
query=1
2019-02-26T16:50:30.775 :: [3] all=2, primary=1, backup=1,
query=1
2019-02-26T16:50:40.851 :: [4] all=18564, primary=9285, backup=9262,
query=9122
2019-02-26T16:50:50.897 :: [5] all=10143, primary=5073, backup=5061,
query=4752
2019-02-26T16:51:00.925 :: [6] all=173, primary=86, backup=72, query=0
2019-02-26T16:51:10.934 :: [7] all=9, primary=0, backup=9, query=0
2019-02-26T16:51:20.952 :: [8] all=9, primary=0, backup=9, query=0
2019-02-26T16:51:30.964 :: [9] all=9, primary=0, backup=9, query=0
2019-02-26T16:51:40.976 :: [10] all=9, primary=0, backup=9, query=0
2019-02-26T16:51:50.988 :: [11] all=9, primary=0, backup=9, query=0
2019-02-26T16:52:01.015 :: [12] all=9, primary=0, backup=9, query=0
2019-02-26T16:52:11.040 :: [13] all=9, primary=0, backup=9, query=0
2019-02-26T16:52:21.053 :: [14] all=9, primary=0, backup=9, query=0
2019-02-26T16:52:31.072 :: [15] all=9, primary=0, backup=9, query=0
[16:52:35] Ignite node stopped OK [uptime=00:03:02.734]

As far as my understanding goes, all entries should expire and both primary
and backup be reduced to zero, but it does not happen.

Can anyone comment this? Should a ticket be filed?

Regards,
-- 
Ilya Kasnacheev


пт, 8 февр. 2019 г. в 22:53, Ruslan Kamashev :

> Hi,
>
> I try to use data expiration with persistence, but I got wrong backup size
> after cache records are expired.
> Also sometimes I got result when expired records didn't deleted but I
> can't reproduce this case again.
>
> Could you share more details about using data expiration with persistence?
>
> Source: https://github.com/rynffoll/ignite-tests
> Logs:
> https://github.com/rynffoll/ignite-tests/tree/master/logs/persistance_with_data_expiration
>
>

[jira] [Created] (IGNITE-11424) Cannot build C++ from tarball of master branch

[Ilya Kasnacheev (JIRA)]

Ilya Kasnacheev created IGNITE-11424:


 Summary: Cannot build C++ from tarball of master branch
 Key: IGNITE-11424
 URL: https://issues.apache.org/jira/browse/IGNITE-11424
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Reporter: Ilya Kasnacheev
Assignee: Igor Sapego


In a tarball made by mvn initialize -Prelease on fresh master branch:

{code}
% libtoolize && aclocal && autoheader && automake --add-missing && autoreconf
libtoolize: putting auxiliary files in '.'.
libtoolize: linking file './ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: linking file 'm4/libtool.m4'
libtoolize: linking file 'm4/ltoptions.m4'
libtoolize: linking file 'm4/ltsugar.m4'
libtoolize: linking file 'm4/ltversion.m4'
libtoolize: linking file 'm4/lt~obsolete.m4'
configure.ac:36: installing './ar-lib'
configure.ac:35: installing './compile'
configure.ac:24: installing './config.guess'
configure.ac:24: installing './config.sub'
configure.ac:28: installing './install-sh'
configure.ac:28: installing './missing'
configure.ac:88: error: required file 'network/include/Makefile.in' not found
configure.ac:88: error: required file 'network/Makefile.in' not found
Makefile.am:27: error: required directory ./network does not exist
Makefile.am:22: error: required directory ./network does not exist
Makefile.am:51: error: required directory ./network does not exist
binary/Makefile.am: installing './depcomp'
{code}

Indeed it seems that network/ does not exist



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Migration to JUnit 5

[Павлухин Иван]

Ivan,

Thank you for detailed answers! I would put a great care to
@RunWith(JUnitPlatform.class) construction. As stated in junit5 docs
[1] it does not support all features and unfortunately it is not clear
how limited it is. Also, it is some kind of transitional mechanism
which was not designed for being a long term solution.

And I fully support an idea of refactoring GridAbstractTest. I think
it is possible to make a significant improvement here.

[1] 
https://junit.org/junit5/docs/current/user-guide/#running-tests-junit-platform-runner

пн, 25 февр. 2019 г. в 17:41, Ivan Fedotov :
>
> Hello Nikolay.
>
> The prime benefits are more comfortable work with flaky tests, Java 8 tests
> compatibility, user-friendly syntaxis in parametrized tests and others.
> The most significant features list you can find in IEP-30 Motivation
> section.
>
> If you have any specific questions about JUnit5 feel free to ask me.
>
> пн, 25 февр. 2019 г. в 16:55, Nikolay Izhikov :
>
> > Hello, Ivan.
> >
> > May be I miss some mail - if yes, can you repeat it.
> > What is advantages of migration from junit 4 to 5?
> > Why we should do it?
> >
> >
> > пн, 25 февр. 2019 г. в 16:33, Ivan Fedotov :
> >
> > > Ivan,
> > > That is my thoughts according to your questions.
> > >
> > > 1. I tried to implement test suits with JUnit4 compatibility layer. The
> > > basic concept is to use @RunWith(JUnitPlatform.class) @SelectClasses
> > > ({...})[1] and on
> > > CI Ignite it works fine.
> > >
> > > 2. According to @Rules, there are several ways to solve it:
> > > 2.1 Leave JUnit4 code without changes. It will work because of
> > Vintage
> > > module
> > > 2.2 Rewrite the @Rule as an Extension. The work of extension is
> > similar
> > > to the @Rules work, but it is extracted in an Extension class.
> > > For more information about extensions, please, follow the IEP [2].
> > > In my opinion, the easiest and the most understandable way is to leave
> > > GridAbstractTest in current form. It will work with JUnit5
> > > syntaxis and abilities.
> > >
> > > 3. I faced a couple of problems during dealing with dynamic and static
> > > tests in one project with JUnit5. The problem occurs with surefire
> > version:
> > > static tests work fine with 2.21x and earlier and with dynamic tests, the
> > > situation is vice versa, it works with > 2.21x surefire version.
> > > We can use helpful surefire dependency to use static tests with the
> > newest
> > > surefire version [3], but dynamic tests become unavailable from pure
> > > Maven and accordingly from CI Ignite (from IDE all is fine).
> > > I can suggest leaving this type of tests on JUnit4 on the current stage -
> > > they are in the vast minority.
> > >
> > > Let me comment on your side notes.
> > >
> > > I am not against the stable and widely-used test library usage. All I
> > want
> > > to say that it is not necessary in case of the main testing Ignite
> > > framework (Junit) already provides the mentioned features.
> > >
> > > At the initial stage of improvements 3->4 I am planning to remove
> > > JUnit3TestLegacyAssert, JUnit3TestLegacySupport classes. I guess that
> > > during this work
> > > I will face with an issue that you are mentioned - turning instance
> > methods
> > > to static. It is because of beforeTestsStarted and afterTestsStarted
> > > methods - I want to replace them by methods with BeforeAll, AfterAll
> > > annotations. But the point is that methods under such annotations must be
> > > static. Just now I am not sure about fully removing
> > > GridCommonAbstractTest class, but the need for static methods is a fact.
> > >
> > > [1]
> > >
> > >
> > https://github.com/apache/ignite/blob/85ba3a88d661bb05bbb749bd1feaf60cd9099ddc/examples/src/test/java/org/apache/ignite/testsuites/IgniteExamplesSelfTestSuite.java#L59
> > > [2]
> > >
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-30%3A+Migration+to+JUnit+5
> > > [3] https://github.com/junit-team/junit5/issues/1778
> > >
> > > вс, 24 февр. 2019 г. в 10:15, Павлухин Иван :
> > >
> > > > Ivan,
> > > >
> > > > Indeed junit5 has a lot of powerful features which can improve testing
> > > > process.
> > > >
> > > > But first we should go through a migration process. There are several
> > > > items which looks quite challenging.
> > > > 1. Test suites support. Correct me if I am missed it, but I have not
> > > > found a concept of test suites similar to junit3/4 ones. CI in Ignite
> > > > heavily depends on test suites. Is there an alternative in junit5?
> > > > 2. The majority of our tests extend GridAbstractTest which in fact is
> > > > a core class in Ignite testing. Writing a test without extending it is
> > > > not a good idea. Currently it employs number of junit4 concepts, e.g.
> > > > test rules which as I saw are not supported in junit5. So, it sounds
> > > > that some changes in GridAbstractTest need to be done. During
> > > > migration from junit 3 to 4 GridAbstractTest used kind of mimicry, it
> > > > 

[jira] [Created] (IGNITE-11422) Remove H2 console from documentation

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-11422:


 Summary: Remove H2 console from documentation
 Key: IGNITE-11422
 URL: https://issues.apache.org/jira/browse/IGNITE-11422
 Project: Ignite
  Issue Type: Task
  Components: documentation
Reporter: Vladimir Ozerov
Assignee: Artem Budnikov


H2 console was deprecated as a part of IGNITE-11333. Need to remove all 
mentions of "H2 console" from documentation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-11421) There's no AI version in ODBC driver available for a connection

[Sergey Kozlov (JIRA)]

Sergey Kozlov created IGNITE-11421:
--

 Summary: There's no AI version in ODBC driver available for a 
connection
 Key: IGNITE-11421
 URL: https://issues.apache.org/jira/browse/IGNITE-11421
 Project: Ignite
  Issue Type: Bug
  Components: platforms
Affects Versions: 2.7
Reporter: Sergey Kozlov


I use {{pyodbc}} and successfuly connected to AI cluster
But all connections attributes have no AI version:
{noformat}
print(conn.getinfo(SQL_DRIVER_NAME))
Apache Ignite
print(conn.getinfo(SQL_DRIVER_ODBC_VER))
03.00
print(conn.getinfo(SQL_DRIVER_VER))
02.04.
print(conn.getinfo(SQL_DBMS_VER))
02.04.
print(conn.getinfo(SQL_ODBC_VER ))
03.80.
{noformat}




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Java 11 maven profile for local builds and testing

[Dmitriy Pavlov]

It should be automatically detected. Profile activation uses JVM version.

 But during import into IDE it may have issues. I enabled it manually in
intellij.

вт, 26 февр. 2019 г., 12:17 Ilya Kasnacheev :

> Hello!
>
> That's awesome! Is this profile auto-detected when running under Java 9+?
> Is there a way to do that?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пн, 25 февр. 2019 г. в 18:43, Dmitriy Pavlov :
>
> > Hi Ignite Developers,
> >
> > Maven profile was modified to provide an opportunity to build a project
> > locally using java11 (you can't use results of such local build for java
> > 8).
> >
> > The profile is named 'java-9+'. If it is not enabled by default you will
> > probably need to enable it manually.
> >
> > The ignite-tools problem under java-11 was fixed by providing two
> versions
> > of this class for Java 8 & java 9+. These files are placed with different
> > source roots. During switching between java versions you may need to
> > re-import maven project. If you use IntelliJ than sometimes you will need
> > to manually remove not needed source root from .iml file.
> >
> > See also "modules/tools/ignite-tools.iml"
> >
> >  > isTestSource="false" />
> >
> > Sincerely,
> >
> > Dmitriy Pavlov
> >
> > BTW, Teamcity bot is up and running using Ignite 2.7 & java 11 so
> > limited number of functionality is working well: (Apache Ignite
> > Teamcity Bot, V20190218, Powered by Apache Ignite V2.7.0, Java
> > Version: 11.0.1)
> >
>

[jira] [Created] (IGNITE-11420) MVCC: Test fails due to unexpected txRollbackException.

[Andrew Mashenkov (JIRA)]

Andrew Mashenkov created IGNITE-11420:
-

 Summary: MVCC: Test fails due to unexpected txRollbackException.
 Key: IGNITE-11420
 URL: https://issues.apache.org/jira/browse/IGNITE-11420
 Project: Ignite
  Issue Type: Bug
  Components: mvcc
Reporter: Andrew Mashenkov


Next tests are failed with unexpected TxRollbackException in mvcc mode:

TxRollbackOnIncorrectParamsTest.testLabelFilledLocalGuarantee
 TxRollbackOnIncorrectParamsTest.testLabelFilledRemoteGuarantee
 
TxRollbackOnIncorrectParamsTest.testRollbackInsideLocalListenerAfterRemoteFilter
 TxRollbackOnIncorrectParamsTest.testTimeoutSetLocalGuarantee
 TxRollbackOnIncorrectParamsTest.testTimeoutSetRemoteGuarantee

Seems, smth was missed in IGNITE-10415 fix or during merge to master.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-11419) Memory leak after multiple restarts of server node within the same jvm

[Pavel Vinokurov (JIRA)]

Pavel Vinokurov created IGNITE-11419:


 Summary: Memory leak after multiple restarts of server node within 
the same jvm
 Key: IGNITE-11419
 URL: https://issues.apache.org/jira/browse/IGNITE-11419
 Project: Ignite
  Issue Type: Bug
Reporter: Pavel Vinokurov


Multiple restarts of a server node with enabled persistence and 20 caches lead 
to OutOfMemory.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: New PMC Member: Roman Shtykh

[Andrey Novikov]

Congrats, Roman!

On Tue, Feb 26, 2019 at 4:58 PM Maxim Muzafarov  wrote:
>
> Roman,
>
> My congratulations! Nice to work with you.
>
> On Tue, 26 Feb 2019 at 10:09 Dmitriy Pavlov  wrote:
>
> > Congrats, Roman!
> >
> > вт, 26 февр. 2019 г. в 04:28, Roman Shtykh :
> >
> > > Thank you! It's always pleasure to work with Ignite community, and I will
> > > keep on working on the project and spreading the Ignite word.
> > >
> > > Best regards,
> > > Roman
> > >
> > >
> > > On Tuesday, February 26, 2019, 7:59:58 a.m. GMT+9, Denis Magda <
> > > dma...@apache.org> wrote:
> > >
> > >  The Project Management Committee (PMC) for Apache Ignite
> > > has invited Roman Shtykh to become a PMC member and we are pleased to
> > > announce that he has accepted. Being a PMC member enables assistance with
> > > the management and to guide the direction of the project.
> > >
> > >
> > > Roman is one of the community veterans who is presently popularizing
> > Ignite
> > > in Japan and helping to adopt it in that country. Thanks a lot for this
> > > hard work and let's support Roman in this initiative.
> > >
> > > -
> > > Denis
> > >
> >
> --
> --
> Maxim Muzafarov

Re: New PMC Member: Roman Shtykh

[Maxim Muzafarov]

Roman,

My congratulations! Nice to work with you.

On Tue, 26 Feb 2019 at 10:09 Dmitriy Pavlov  wrote:

> Congrats, Roman!
>
> вт, 26 февр. 2019 г. в 04:28, Roman Shtykh :
>
> > Thank you! It's always pleasure to work with Ignite community, and I will
> > keep on working on the project and spreading the Ignite word.
> >
> > Best regards,
> > Roman
> >
> >
> > On Tuesday, February 26, 2019, 7:59:58 a.m. GMT+9, Denis Magda <
> > dma...@apache.org> wrote:
> >
> >  The Project Management Committee (PMC) for Apache Ignite
> > has invited Roman Shtykh to become a PMC member and we are pleased to
> > announce that he has accepted. Being a PMC member enables assistance with
> > the management and to guide the direction of the project.
> >
> >
> > Roman is one of the community veterans who is presently popularizing
> Ignite
> > in Japan and helping to adopt it in that country. Thanks a lot for this
> > hard work and let's support Roman in this initiative.
> >
> > -
> > Denis
> >
>
-- 
--
Maxim Muzafarov

Re: Java 11 maven profile for local builds and testing

[Ilya Kasnacheev]

Hello!

That's awesome! Is this profile auto-detected when running under Java 9+?
Is there a way to do that?

Regards,
-- 
Ilya Kasnacheev


пн, 25 февр. 2019 г. в 18:43, Dmitriy Pavlov :

> Hi Ignite Developers,
>
> Maven profile was modified to provide an opportunity to build a project
> locally using java11 (you can't use results of such local build for java
> 8).
>
> The profile is named 'java-9+'. If it is not enabled by default you will
> probably need to enable it manually.
>
> The ignite-tools problem under java-11 was fixed by providing two versions
> of this class for Java 8 & java 9+. These files are placed with different
> source roots. During switching between java versions you may need to
> re-import maven project. If you use IntelliJ than sometimes you will need
> to manually remove not needed source root from .iml file.
>
> See also "modules/tools/ignite-tools.iml"
>
>  isTestSource="false" />
>
> Sincerely,
>
> Dmitriy Pavlov
>
> BTW, Teamcity bot is up and running using Ignite 2.7 & java 11 so
> limited number of functionality is working well: (Apache Ignite
> Teamcity Bot, V20190218, Powered by Apache Ignite V2.7.0, Java
> Version: 11.0.1)
>

[jira] [Created] (IGNITE-11418) Document SQL IGNITE.INDEXES view

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-11418:


 Summary: Document SQL IGNITE.INDEXES view
 Key: IGNITE-11418
 URL: https://issues.apache.org/jira/browse/IGNITE-11418
 Project: Ignite
  Issue Type: Task
  Components: documentation
Reporter: Vladimir Ozerov
Assignee: Artem Budnikov


New {{IGNITE.INDEXES}} view was added, which displays indexes in specific 
columns. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (IGNITE-11417) Get rid of CacheRebalanceMode#NONE

[Roman Kondakov (JIRA)]

Roman Kondakov created IGNITE-11417:
---

 Summary: Get rid of CacheRebalanceMode#NONE
 Key: IGNITE-11417
 URL: https://issues.apache.org/jira/browse/IGNITE-11417
 Project: Ignite
  Issue Type: Improvement
  Components: cache
Reporter: Roman Kondakov
 Fix For: 3.0


We should get rid of erroneous and unusable rebalance mode {{NONE}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)