Re: [MTCGA]: new failures in builds [2818501] needs to be handled

2019-01-26 Thread Dmitriy Pavlov
It is reporting of an old failure. It occurs probably because of rebuilding
internal run history in the bot after cache corruption.

Now mentioned test is just rare-flaky, so please ignore.

сб, 26 янв. 2019 г. в 12:03, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *New stable failure of a flaky test in master
> IgniteClusterActivateDeactivateTestWithPersistence.testActivateCachesRestore_SingleNode_WithNewCaches
>
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-7923751657565628273=%3Cdefault%3E=testDetails
>  No changes in the build
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 12:02:54 26-01-2019
>


Re: edit access to Apache Ignite wiki

2019-01-25 Thread Dmitriy Pavlov
Done, I've added permission for Oleg Ignatenko user.

Just in case, let me share
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document with you
:)

пт, 25 янв. 2019 г. в 20:55, oignatenko :

> Hello,
>
> Is it possible to get edit access to Apache Ignite wiki?
>
> I've been working on particular JIRA ticket [1] and one of reviewers
> suggested me update particular wiki page [2] after the ticket is done.
>
> Now that the ticket is done I would like to make proposed edits to wiki
> page.
>
> TIA,
> regards, Oleg
>
> [1]: https://issues.apache.org/jira/browse/IGNITE-10178
> [2]: https://cwiki.apache.org/confluence/display/IGNITE/Coding+Guidelines
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>


Re: [MTCGA]: new failures in builds [2658036] needs to be handled

2019-01-25 Thread Dmitriy Pavlov
Who can advice if mentioned tests change their name from time to time?

The bot will not notify for such tests correctly because it is impossible
to say if a test is the same with some passing test(s).

чт, 24 янв. 2019 г. в 22:17, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *Recently contributed test failed in master tests:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase::testCacheGetWrongArgs
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-2711660524736740778=%3Cdefault%3E=testDetails
>
>  *Recently contributed test failed in master tests:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase::testCachePutWrongArgs
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-7419076455473690169=%3Cdefault%3E=testDetails
>
>  *Recently contributed test failed in master tests:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase::testCacheClearKeyWrongArgs
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=1014161704039071695=%3Cdefault%3E=testDetails
>
>  *Recently contributed test failed in master tests:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase::testCacheClearKeysWrongArgs
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=6066988179553985738=%3Cdefault%3E=testDetails
>
>  *Recently contributed test failed in master tests:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase::testCacheGetSizeWrongArgs
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=8039044665871263128=%3Cdefault%3E=testDetails
>
>  *Recently contributed test failed in master tests:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase::testContainsKeyWrongArgs
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=8278104022332781338=%3Cdefault%3E=testDetails
>
>  *Recently contributed test failed in master tests:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase::testCacheReplaceWrongArgs
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=2601421891678519225=%3Cdefault%3E=testDetails
>
>  *Recently contributed test failed in master tests:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase::testCacheGetAndPutIfAbsentWrongArgs
>
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=6918523075100081828=%3Cdefault%3E=testDetails
>
>  *Recently contributed test failed in master tests:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase::testCacheContainsKeysWrongArgs
>
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=8508115998050634179=%3Cdefault%3E=testDetails
>
>  *Recently contributed test failed in master tests:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase::testCacheGetAllWrongArgs
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=8010201874973788100=%3Cdefault%3E=testDetails
>
>  *Recently contributed test failed in master tests:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase::testRemoveIfEqualsWrongArgs
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=5613882862418653296=%3Cdefault%3E=testDetails
>
>  *Recently contributed test failed in master tests:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase::testCacheReplaceIfEqualsWrongArgs
>
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=7510292952369692624=%3Cdefault%3E=testDetails
>
>  *Recently contributed test failed in master tests:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase::testCacheGetAndRemoveWrongArgs
>
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=6767768552589233912=%3Cdefault%3E=testDetails
>
>  *Recently contributed test failed in master tests:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase:
> Apache\Ignite\Tests\CacheKeyValueOpsTestCase::testCacheGetAndPutWrongArgs
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=1431371577522230596=%3Cdefault%3E=testDetails
>
>  *Recently contributed 

Re: [MTCGA]: new failures in builds [2807830] needs to be handled

2019-01-25 Thread Dmitriy Pavlov
The test is still flaky and it seems to begin failing recently. Is there
any issue for this test fix?

вт, 22 янв. 2019 г. в 22:49, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *New stable failure of a flaky test in master
> PartitionLossTest.TestReadOnlyAll
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=5428922878479275743=%3Cdefault%3E=testDetails
>  Changes may lead to failure were done by
>  - amalykhgh
> https://ci.ignite.apache.org/viewModification.html?modId=872152
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 22:32:50 22-01-2019
>


Re: [MTCGA]: new failures in builds [2867589] needs to be handled

2019-01-25 Thread Dmitriy Pavlov
It seems to be passing now. Please ignore.

ср, 23 янв. 2019 г. в 00:03, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *New stable failure of a flaky test in master IgniteCoreTest:
> CacheQueryTestSuite: TestFieldsQueryPagesSeveral
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-4145688911096802577=%3Cdefault%3E=testDetails
>
>  *New stable failure of a flaky test in master IgniteCoreTest:
> CacheQueryTestSuite: TestFieldsQuerySeveral
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=7923965190700916819=%3Cdefault%3E=testDetails
>  Changes may lead to failure were done by
>  - sergi.vladykin
> https://ci.ignite.apache.org/viewModification.html?modId=872823
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 00:02:54 23-01-2019
>


Re: [MTCGA]: new failures in builds [2867594, 2639452] needs to be handled

2019-01-25 Thread Dmitriy Pavlov
Passing now, please ignore

вт, 22 янв. 2019 г. в 23:33, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *New stable failure of a flaky test in master IgniteCoreTest:
> CacheQueryTestSuite: TestFieldsQueryPagesSeveral
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-4145688911096802577=%3Cdefault%3E=testDetails
>
>  *New stable failure of a flaky test in master IgniteCoreTest:
> CacheQueryTestSuite: TestFieldsQuerySeveral
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=7923965190700916819=%3Cdefault%3E=testDetails
>  Changes may lead to failure were done by
>  - sergi.vladykin
> https://ci.ignite.apache.org/viewModification.html?modId=872823
>
>  *New stable failure of a flaky test in master
> IgniteStartStopTest.TestProcessorInit
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-8379686487933741046=%3Cdefault%3E=testDetails
>  No changes in the build
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 23:32:51 22-01-2019
>


Re: Ignite index corruption issue -> unrecoverable cluster

2019-01-24 Thread Dmitriy Pavlov
Denis, Whan do you think about a more general idea of creating FAQs for
Ignite users?

What if experts will once place their answer in a wiki page and then
develop answers for frequent problems.

And before diving into researching each problem, experienced community
members will ask users to check the FAQ first?

Sincerely,
Dmitriy Pavlov

P.S. here is an article, Apache guides have reference to
http://www.catb.org/~esr/faqs/smart-questions.html - one from required
actions from users is to search for information.

чт, 24 янв. 2019 г. в 01:55, Denis Magda :

> Another data/index corruption issue:
>
> https://stackoverflow.com/questions/54295401/ignite-transaction-failure-not-recoverable-with-persistance
>
> It's suggested to clean index.bin to be able to recover the cluster. Folks,
> let's prepare a list of actions to do if a cluster becomes unrecoverable
> due to data or index corruption issue. What should we do depending on an
> exception:
>
>- Remove index.bin if X or Y or Z
>- etc
>
>
> --
> Denis Magda
>
>
> On Sun, Dec 30, 2018 at 10:06 AM Denis Magda  wrote:
>
> > Ignite SQL and memory experts,
> >
> > The following issue was reported on SO:
> >
> >
> https://stackoverflow.com/questions/53979106/ignite-corruptedtreeexception-leads-to-cluster-failure
> >
> > The stack trace starts with the message below, more details are in that
> > forum:
> >
> > [SEVERE][data-streamer-stripe-2-#15][GridDhtAtomicCache] 
> > Unexpected exception during cache update
> > org.h2.message.DbException: General error: "class
> >
> org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
> > Runtime failure on row: Row@75ab6623[ key: CacheKey [idHash=242632156,
> > hash=-841684964, parentId=-8607237606486310912, hour=9,
> > id=-8607237528489033728, date=2018-09-09 00:00:00.0], val: CacheValue
> > [idHash=843227122, hash=-801894604, 
> >
> > Let's see if it's addressed in the latest release. Also, the user asked a
> > reasonable question - how to recover? Yes, it's possible to use snapshots
> > of GridGain if they are created before but I remember some discussions
> > around a recovery tool.
> >
> > --
> > Denis
> >
>


Re: Review and merge IGNITE-10921

2019-01-24 Thread Dmitriy Pavlov
Count on my support if nobody else would like to help with the merge.
Eventually, I will pick up.

ср, 23 янв. 2019 г. в 19:53, Stanislav Lukyanov :

> Hi Igniters,
>
> Can anyone please review and merge IGNITE-10921?
> Artur Muradimov contributed the change, I did a review myself and now we
> need a committer to finish this.
>
> Thanks,
> Stan
>
>


Re: [MTCGA]: new failures in builds [2872965] needs to be handled

2019-01-24 Thread Dmitriy Pavlov
Hi, it is probably related to
https://issues.apache.org/jira/browse/IGNITE-10822 Test was failing before.
I've muted it.

ср, 23 янв. 2019 г. в 12:33, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *New stable failure of a flaky test in master
> SysPropWalDeltaConsistencyTest.testPutRemoveMultinode
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=8000156635398196642=%3Cdefault%3E=testDetails
>  Changes may lead to failure were done by
>  - skosarev
> https://ci.ignite.apache.org/viewModification.html?modId=872921
>  - kondakov87
> https://ci.ignite.apache.org/viewModification.html?modId=872922
>  - y.chief
> https://ci.ignite.apache.org/viewModification.html?modId=872934
>  - zaleslaw.sin
> https://ci.ignite.apache.org/viewModification.html?modId=872912
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 12:32:51 23-01-2019
>


Re: [MTCGA]: new failures in builds [2872963] needs to be handled

2019-01-24 Thread Dmitriy Pavlov
Hi, it is an obvious and stable new test failure. This issue is probably
coming from

Roman Kondakov (rkondakov)
3 files
IGNITE-10753 : MVCC: Await all queries are done before checking versions
are cleaned. This closes #5786.
Sergey Kosarev (skosarev)
20 files
IGNITE-6564  Correct calculation of size and keySize for cluster cache
metrics.

Roman, Sergey,

Please chime in.

Sincerely,
Dmitriy Pavlov



ср, 23 янв. 2019 г. в 12:13, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *New stable failure of a flaky test in master
> CacheMetricsEntitiesCountTest.testEnitiesCount
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-9212566675620168727=%3Cdefault%3E=testDetails
>  Changes may lead to failure were done by
>  - skosarev
> https://ci.ignite.apache.org/viewModification.html?modId=872921
>  - kondakov87
> https://ci.ignite.apache.org/viewModification.html?modId=872922
>  - y.chief
> https://ci.ignite.apache.org/viewModification.html?modId=872934
>  - zaleslaw.sin
> https://ci.ignite.apache.org/viewModification.html?modId=872912
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 12:02:49 23-01-2019
>


[jira] [Created] (IGNITE-11049) [TC Bot] Reset Credentials is broken

2019-01-23 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-11049:
---

 Summary: [TC Bot] Reset Credentials is broken
 Key: IGNITE-11049
 URL: https://issues.apache.org/jira/browse/IGNITE-11049
 Project: Ignite
  Issue Type: Bug
Reporter: Dmitriy Pavlov


https://mtcga.gridgain.com/user.html
Reset credentials 
gives
Internal Server Error [500].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Contribute to ignite.net

2019-01-23 Thread Dmitriy Pavlov
Hi,

It seems some PMC already added your account to the list of contributors,
so now you can assign the issue.

Welcome to Apache Ignite community!

Sincerely,
Dmitriy Pavlov

вт, 22 янв. 2019 г. в 23:29, Лев Хенкин :

> Hi, folks. I'm using ignite.net and I want to try to improve it. I think
> that I'm able to do this ticket
> https://issues.apache.org/jira/browse/IGNITE-7128 but I cannot assign it
> to me in jira.
> My login  is lkhenkin, could you please help me?
>


Re: Greeting

2019-01-22 Thread Dmitriy Pavlov
Hi Artur,

You're in,  now you can assign an issue to yourself.

Thank you for interest in Apache Ignite and welcome to the Apache Ignite
Community.

Following is a couple of links should help during joining to the community
Short 'How to start' guide
https://github.com/apache/ignite/blob/master/CONTRIBUTING.md#contributing-to-apache-ignite

and full guide version
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute

Should you have any questions please do not hesitate to contact Ignite
developers here.

Sincerely,
Dmitriy Pavlov

вт, 22 янв. 2019 г. в 22:21, Artur Muradimov :

> Community,
>
> I am Artur Muradimov and I want to contribute Apache Ignite.
>
> I had a time to watch the project and I think, already ready to make some
> JIRA ticket.
> I want to start from the issue IGNITE-10921
>
> Please grant me contributer privileges to make possible assign.
> My jira login: arturmuradimov77
>


[jira] [Created] (IGNITE-11030) Teamcity Bot cache is in unrecoverable state after bot restart: need to check reasons

2019-01-22 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-11030:
---

 Summary: Teamcity Bot cache is in unrecoverable state after bot 
restart: need to check reasons
 Key: IGNITE-11030
 URL: https://issues.apache.org/jira/browse/IGNITE-11030
 Project: Ignite
  Issue Type: Task
Reporter: Dmitriy Pavlov


- TC Bot was stopped with kill command (default signal).
- Shutdown hook calls ignite close

But after updating to V190122 TC Bot DB was became unrecoverable for 
"teamcityTestRunHist" cache with exception

{noformat}
Failure periodic check failed: javax.cache.CacheException: class 
org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
 Runtime failure on lookup row: SearchRow 
[key=org.apache.ignite.ci.teamcity.ignited.runhist.RunHistKey 
[idHash=1028871081, hash=1241170874, srvId=1411517106, testOrSuiteName=11924, 
branch=281], hash=1241170874, cacheId=0]
com.google.common.util.concurrent.UncheckedExecutionException: 
javax.cache.CacheException: class 
org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
 Runtime failure on lookup row: SearchRow 
[key=org.apache.ignite.ci.teamcity.ignited.runhist.RunHistKey 
[idHash=1028871081, hash=1241170874, srvId=1411517106, testOrSuiteName=11924, 
branch=281], hash=1241170874, cacheId=0]
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2050)
at com.google.common.cache.LocalCache.get(LocalCache.java:3952)
at 
com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
at 
org.apache.ignite.ci.di.cache.GuavaCachedInterceptor.invoke(GuavaCachedInterceptor.java:59)
at 
org.apache.ignite.ci.teamcity.ignited.TeamcityIgnitedImpl.getTestRunHist(TeamcityIgnitedImpl.java:404)
at 
org.apache.ignite.ci.di.AutoProfilingInterceptor.invoke(AutoProfilingInterceptor.java:76)
at 
org.apache.ignite.ci.web.model.current.SuiteCurrentStatus.lambda$initFromContext$0(SuiteCurrentStatus.java:155)
at java.util.Comparator.lambda$comparing$77a9974f$1(Comparator.java:469)
at 
java.util.Collections$ReverseComparator2.compare(Collections.java:5178)
at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
at java.util.TimSort.sort(TimSort.java:220)
at java.util.Arrays.sort(Arrays.java:1512)
at java.util.ArrayList.sort(ArrayList.java:1462)
at 
org.apache.ignite.ci.web.model.current.SuiteCurrentStatus.initFromContext(SuiteCurrentStatus.java:160)
at 
org.apache.ignite.ci.web.model.current.ChainAtServerCurrentStatus.lambda$initFromContext$0(ChainAtServerCurrentStatus.java:171)
at 
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at 
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at 
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at 
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at 
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at 
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at 
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at 
org.apache.ignite.ci.web.model.current.ChainAtServerCurrentStatus.initFromContext(ChainAtServerCurrentStatus.java:167)
at 
org.apache.ignite.ci.tcbot.chain.TrackedBranchChainsProcessor.lambda$getTrackedBranchTestFailures$1(TrackedBranchChainsProcessor.java:121)
at 
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at 
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at 
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at 
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at 
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at 
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at 
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at 
org.apache.ignite.ci.tcbot.chain.TrackedBranchChainsProcessor.getTrackedBranchTestFailures(TrackedBranchChainsProcessor.java:125)
at 
org.apache.ignite.ci.di.AutoProfilingInterceptor.invoke(AutoProfilingInterceptor.java:76)
at 
org.apache.ignite.ci.tcbot.issue.IssueDetector.checkFailuresEx(IssueDetector.java:445)
at 
org.apache.ignite.ci.di.MonitoredTaskIntercep

[jira] [Created] (IGNITE-10989) [TC Bot] Finalize refactoring and remove REST data persistent caches

2019-01-18 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-10989:
---

 Summary: [TC Bot] Finalize refactoring and remove REST data 
persistent caches
 Key: IGNITE-10989
 URL: https://issues.apache.org/jira/browse/IGNITE-10989
 Project: Ignite
  Issue Type: Task
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov


TC Bot was migrated to compacted storage of builds, so now it is no reason to 
keep old-fashioned REST request caches for TeamCity data



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: IGNITE-10978

2019-01-18 Thread Dmitriy Pavlov
Hi Konstantin,

Thank you for interest in Apache Ignite and welcome to the Apache Ignite
Community.

I've added you to the list of contributors, so now you can assign an issue
to yourself.

Should you have any questions please do not hesitate to contact Ignite
developers here.

Sincerely,
Dmitriy Pavlov

P.S. Let me share with you
Short 'How to start' guide
https://github.com/apache/ignite/blob/master/CONTRIBUTING.md#contributing-to-apache-ignite

and full guide version
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute

пт, 18 янв. 2019 г. в 17:28, Konstantin Bolyandra :

> Hello Ignite Community!
>
> My name is Konstantin. I want to contribute to Apache Ignite and want to
> start with this issue - IGNITE-10978, my JIRA username kbolyandra. Any
> help on this will be appreciated.
>
> Thanks!
>
> --
> Best Regards,
> Konstantin
>
>


[jira] [Created] (IGNITE-10978) Remove unused tests marked with unclear todo

2019-01-18 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-10978:
---

 Summary: Remove unused tests marked with unclear todo
 Key: IGNITE-10978
 URL: https://issues.apache.org/jira/browse/IGNITE-10978
 Project: Ignite
  Issue Type: Task
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov



{noformat}
// TODO GG-11141.
//
GridTestUtils.addTestIfNeeded(suite,GridCacheOnCopyFlagTxPartitionedSelfTest.class,
 ignoredTests);
//
GridTestUtils.addTestIfNeeded(suite,GridCacheOnCopyFlagReplicatedSelfTest.class,
 ignoredTests);
//
GridTestUtils.addTestIfNeeded(suite,GridCacheOnCopyFlagLocalSelfTest.class, 
ignoredTests);
//
GridTestUtils.addTestIfNeeded(suite,GridCacheOnCopyFlagAtomicSelfTest.class, 
ignoredTests);
{noformat}
Test classes not used, so there is no reason to keep it



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Subscribe

2019-01-17 Thread Dmitriy Pavlov
Hi Igor,

Welcome to the Apache Ignite Community.

I've added you to the list of contributors, so now you can assign an issue
to yourself.

Should you have any questions please do not hesitate to contact Ignite
developers here.

Sincerely,
Dmitriy Pavlov

P.S. Let me share with you
Short 'How to start' guide
https://github.com/apache/ignite/blob/master/CONTRIBUTING.md#contributing-to-apache-ignite

and full guide version
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute

чт, 17 янв. 2019 г. в 20:22, Igor A :

> Hello Ignite Community!
>
> I want to contribute to Apache Ignite, I am maintenance engineer, and I
> can some Java, C# and Perl my JIRA username is "iakkuratov"
>
> and i want to assign IGNITE-9907 to myself.
>
> Thanks!
>
>


[jira] [Created] (IGNITE-10942) [TC Bot] Optimize backround JIRA tickets sync

2019-01-15 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-10942:
---

 Summary: [TC Bot] Optimize backround JIRA tickets sync
 Key: IGNITE-10942
 URL: https://issues.apache.org/jira/browse/IGNITE-10942
 Project: Ignite
  Issue Type: Task
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov


JIRA sync is now performed in not an efficient manner.
All project data is reloaded every 15 minutes, and this may create additional 
pressure for 3rd party services.

Introduce incremental updates (update is running only until the first 
non-modified element found) and full updates.

Check ticket data modification before saving into Ignite DB (will safe WAL & 
page store pressure).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: How to update Documentation 2.8 docs? - IGNITE-8532 [ML] GA Grid - Implement Roulette Wheel Selection

2019-01-15 Thread Dmitriy Pavlov
Hi Turik,

2.8 is not released, and dates were not discussed yet. So it is hard to say
when a doc package will be available.

Sincerely,
Dmitriy Pavlov


пн, 14 янв. 2019 г. в 14:18, techbysample :

> Igniters,
>
> Would you please advise on when 2.8 documentation will be available?
>
> I need to make updates to Genetic Algorithm section of ML to support
>
> ticket: https://issues.apache.org/jira/browse/IGNITE-8532
>
> Best,
> Turik
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>


[jira] [Created] (IGNITE-10930) [TC Bot] Support PR-less contributions

2019-01-14 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-10930:
---

 Summary: [TC Bot] Support PR-less contributions
 Key: IGNITE-10930
 URL: https://issues.apache.org/jira/browse/IGNITE-10930
 Project: Ignite
  Issue Type: Task
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov


An Apache Ignite committer can prepare issue fix with origin/ignite-1 
branch. And this contributions testing does not require any PR creations, 
because TC RunAll may be created without it.

Support contributions detection by provided TC run for branches matching
ignite-N, where N=0-9

If such RunAll presented for any tracked branch and JIRA issue is in PA state, 
we may display these branches as addition to PRs, show its report and setup 
VISA into IGNITE-N ticket.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: I want to contribute to Apache Ignite

2019-01-12 Thread Dmitriy Pavlov
Hi Andrey,

Welcome to Apache Ignite Community.

I've added you to the list of contributors, so now you can assign issues to
yourself.

Sincerely,
Dmitriy Pavlov

P.S. Let me share with you
Short 'How to start' guide
https://github.com/apache/ignite/blob/master/CONTRIBUTING.md#contributing-to-apache-ignite

and full guide version
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute


сб, 12 янв. 2019 г. в 21:04, Андрей Калинин :

> Hello Ignite Community!
>
> My name is Andrey. I want to contribute to Apache Ignite and want to start
> with this issue - IGNITE-10138, my JIRA username "6uest".
>
> Thanks!
>
> --
> C уважением,
> Андрей Калинин
>


Re: Time to remove automated messages from the devlist?

2019-01-11 Thread Dmitriy Pavlov
Hi Oleg,

AFAIK digest subscription works for individuals, who can voluntary
subscribe to digests. It is not about forwarding digests from one list
to another.

Forwarding digest may require us to develop some kind of our own tool for
this (like AI TC Bot), but I would like to avoid any development here :).

Actually, I thought only about someone who wants to monitor the list (to
understand what happens, to select issue he or she want to contribute/or to
discourage its addition to the product).

But this forwarding is also to be used by all community members to know if
there any issues related to their previous contributions. A user who
reports a new issue using JIRA usually has absolutely no clue who can now
help with resolving. So this forwarding should help us to figure out any
new problems should be addressed.

Sincerely,
Dmitriy Pavlov

пт, 11 янв. 2019 г. в 12:47, oignatenko :

> Hi Denis,
>
> Mixing JIRA and human mails in one list seems to be inconvenient both for
> those interested in monitoring human and JIRA mails and from this
> perspective separate lists approach looks superior.
>
> Also, your mention of filtering made me wonder if "mixed" approach may have
> effect opposite to what you want, that is folks will just blindly filter
> out
> JIRA mails so that these will be effectively hidden from target audience.
>
> That said, your reasons for sending JIRA notifications to mailing list
> readers look important to me and that also makes me unhappy with approach
> of
> separate JIRA list.
>
> Trying to figure if there is a way to somehow resolve above contradiction I
> re-checked Apache mailing lists documentation for maybe they have something
> to address these matters somehow.
>
> From what I read it looks that we can try sort of combined solution, when
> JIRA notification feed to separate mailing list which in turn forwards
> daily
> digest to human list. [1]
>
> WDYT?
>
> regards, Oleg
>
> [1]: https://apache.org/foundation/mailinglists.html#digest-subscriptions
>
>
> Denis Magda-2 wrote
> > -1 for this action.
> >
> > JIRA notifications are sent on purpose to the dev list - someone came
> > across an issue and want to report it to the dev community. The community
> > has to be aware of this. That's the purpose of the community to improve
> > and
> > fix the project. It's just wrong to force to subscribe to 2 lists, people
> > won't do this, we're lazy by nature. Just use email filters, that's XXI
> > century but please don't separate discussions from issues reporting.
> >
> > --
> > Denis
> >
> >
> > On Thu, Jan 10, 2019 at 3:55 AM Dmitriy Pavlov 
>
> > dpavlov@
>
> >  wrote:
> >
> >> Hi Igniters,
> >>
> >> After removal of GitHub Comments from the list I have (a very
> subjective)
> >> feeling, that there became more human-human interaction, which is
> >> definitely more important that opportunity to control new JIRA tickets
> >> using the list.
> >>
> >> I suggest coming back to the idea of moving JIRA to a separate list.
> >> Please
> >> share your vision on this topic. Should it be
>
> > jira@.apache
>
> >  or we
> >> should reuse
>
> > notifications@.apache
>
> >>
> >> Sincerely,
> >> Dmitriy Pavlov
> >>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>


Re: Time to remove automated messages from the devlist?

2019-01-10 Thread Dmitriy Pavlov
Ok, Denis, Thank you for sharing your vision.

BTW, we have now separate list for archiving all JIRA actions here
https://lists.apache.org/list.html?iss...@ignite.apache.org I didn't know
about it before yesterday. This probably proves that newcomers will ignore
a separate list.

пт, 11 янв. 2019 г. в 02:32, Denis Magda :

> -1 for this action.
>
> JIRA notifications are sent on purpose to the dev list - someone came
> across an issue and want to report it to the dev community. The community
> has to be aware of this. That's the purpose of the community to improve and
> fix the project. It's just wrong to force to subscribe to 2 lists, people
> won't do this, we're lazy by nature. Just use email filters, that's XXI
> century but please don't separate discussions from issues reporting.
>
> --
> Denis
>
>
> On Thu, Jan 10, 2019 at 3:55 AM Dmitriy Pavlov  wrote:
>
> > Hi Igniters,
> >
> > After removal of GitHub Comments from the list I have (a very subjective)
> > feeling, that there became more human-human interaction, which is
> > definitely more important that opportunity to control new JIRA tickets
> > using the list.
> >
> > I suggest coming back to the idea of moving JIRA to a separate list.
> Please
> > share your vision on this topic. Should it be j...@ignite.apache.org or
> we
> > should reuse notificati...@ignite.apache.org
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > ср, 21 нояб. 2018 г. в 15:25, Dmitriy Pavlov :
> >
> > > Please start a vote according to
> > > https://www.apache.org/foundation/voting.html
> > > Anyone can start a vote, you don't need to be a PMC.
> > >
> > > I don't feel it is a very important issue to remove notifications from
> > the
> > > list, as it can be easily filtered out using mail setup. But if someone
> > > feels it is really disturbing, please go ahead. I'm ok with GitHub
> > > redirection, but I will not drive this topic.
> > >
> > > ср, 21 нояб. 2018 г. в 11:40, Павлухин Иван :
> > >
> > >> Dmitriy, let's proceed with it.
> > >> вт, 20 нояб. 2018 г. в 23:20, Dmitriy Pavlov :
> > >> >
> > >> > One more thing I want to emphasize here. We can't just remove
> > messages,
> > >> it
> > >> > _must_ be sent to some list, which is why we need some additional
> > list,
> > >> > e.g. notifications@ for this.
> > >> >
> > >> > So only one option to proceed here is to run a formal vote on list
> > >> creation
> > >> > and redirection of github/gitbox messages to a new list.
> > >> >
> > >> > пн, 19 нояб. 2018 г. в 18:23, Dmitriy Pavlov :
> > >> >
> > >> > > Denis, we need because contributors do not announce their
> > >> > > intent/designs/etc manually. It is the best way ever? No, of
> course.
> > >> > >
> > >> > > We have consensus on PR removal, so let's do it and see results.
> > >> > >
> > >> > > пн, 19 нояб. 2018 г. в 18:11, Denis Mekhanikov <
> > dmekhani...@gmail.com
> > >> >:
> > >> > >
> > >> > >> Dmitriy,
> > >> > >>
> > >> > >> If a person wants to track all new tickets, then he may go to
> JIRA,
> > >> create
> > >> > >> a filter for Ignite tickets
> > >> > >> and subscribe to it. JIRA has a pretty flexible configuration of
> > >> filters
> > >> > >> and subscriptions, so you can
> > >> > >> specify exactly what issues you are interested in, and how often
> > you
> > >> want
> > >> > >> to receive these emails.
> > >> > >> This is much more convenient and more flexible than filtering
> > emails
> > >> from
> > >> > >> a
> > >> > >> bot.
> > >> > >>
> > >> > >> So, most people ignore JIRA messages, and the ones who want to
> > track
> > >> new
> > >> > >> tickets,
> > >> > >> may go to JIRA and configure their own filters. I don't see, why
> we
> > >> need
> > >> > >> to
> > >> > >> keep the forwarding to dev list.
> > >> > >>
> > >> > >> Denis
> > >> > >>
> > >> > >> пт, 16 нояб. 2018 г. в 22:30, Павлухин Иван  >:
> &g

Re: Configuration suggestion of Java version at Ignite startup

2019-01-10 Thread Dmitriy Pavlov
Yes, and we can run tests on J9,10,11 but it is controlled by JavaHome
build parameter during runAll start. I suggest first to change this
parameter as default to newer Java in 2-4 weeks before updating our
recommendations.

чт, 10 янв. 2019 г. в 15:10, Vyacheslav Daradur :

> AFAIK some test-agents are configured to use JDK 9/10/11, isn't it?
>
> On Thu, Jan 10, 2019 at 2:59 PM Dmitriy Pavlov  wrote:
> >
> > I feel the first step there should be to switch our tests running on TC
> to
> > java > 8.
> >
> > After some time having newer java as default we can change suggestions
> for
> > users.
> >
> > чт, 10 янв. 2019 г. в 14:08, Vyacheslav Daradur :
> >
> > > Igniters,
> > >
> > > As you know, Ignite logs configuration tuning suggestions at node
> > > startup to improve performance.
> > >
> > > At the moment, it recommends using Java 8 even if later releases are
> used.
> > >
> > > Public updates will not be available for business users since January
> > > of 2019 if I'm not mistaken.
> > >
> > > It may be useful to change recommendation to use the latest Java
> > > version supported by Ignite.
> > >
> > > What do you think?
> > >
> > > --
> > > Best Regards, Vyacheslav D.
> > >
>
>
>
> --
> Best Regards, Vyacheslav D.
>


Re: Configuration suggestion of Java version at Ignite startup

2019-01-10 Thread Dmitriy Pavlov
I feel the first step there should be to switch our tests running on TC to
java > 8.

After some time having newer java as default we can change suggestions for
users.

чт, 10 янв. 2019 г. в 14:08, Vyacheslav Daradur :

> Igniters,
>
> As you know, Ignite logs configuration tuning suggestions at node
> startup to improve performance.
>
> At the moment, it recommends using Java 8 even if later releases are used.
>
> Public updates will not be available for business users since January
> of 2019 if I'm not mistaken.
>
> It may be useful to change recommendation to use the latest Java
> version supported by Ignite.
>
> What do you think?
>
> --
> Best Regards, Vyacheslav D.
>


Re: Time to remove automated messages from the devlist?

2019-01-10 Thread Dmitriy Pavlov
Hi Igniters,

After removal of GitHub Comments from the list I have (a very subjective)
feeling, that there became more human-human interaction, which is
definitely more important that opportunity to control new JIRA tickets
using the list.

I suggest coming back to the idea of moving JIRA to a separate list. Please
share your vision on this topic. Should it be j...@ignite.apache.org or we
should reuse notificati...@ignite.apache.org

Sincerely,
Dmitriy Pavlov

ср, 21 нояб. 2018 г. в 15:25, Dmitriy Pavlov :

> Please start a vote according to
> https://www.apache.org/foundation/voting.html
> Anyone can start a vote, you don't need to be a PMC.
>
> I don't feel it is a very important issue to remove notifications from the
> list, as it can be easily filtered out using mail setup. But if someone
> feels it is really disturbing, please go ahead. I'm ok with GitHub
> redirection, but I will not drive this topic.
>
> ср, 21 нояб. 2018 г. в 11:40, Павлухин Иван :
>
>> Dmitriy, let's proceed with it.
>> вт, 20 нояб. 2018 г. в 23:20, Dmitriy Pavlov :
>> >
>> > One more thing I want to emphasize here. We can't just remove messages,
>> it
>> > _must_ be sent to some list, which is why we need some additional list,
>> > e.g. notifications@ for this.
>> >
>> > So only one option to proceed here is to run a formal vote on list
>> creation
>> > and redirection of github/gitbox messages to a new list.
>> >
>> > пн, 19 нояб. 2018 г. в 18:23, Dmitriy Pavlov :
>> >
>> > > Denis, we need because contributors do not announce their
>> > > intent/designs/etc manually. It is the best way ever? No, of course.
>> > >
>> > > We have consensus on PR removal, so let's do it and see results.
>> > >
>> > > пн, 19 нояб. 2018 г. в 18:11, Denis Mekhanikov > >:
>> > >
>> > >> Dmitriy,
>> > >>
>> > >> If a person wants to track all new tickets, then he may go to JIRA,
>> create
>> > >> a filter for Ignite tickets
>> > >> and subscribe to it. JIRA has a pretty flexible configuration of
>> filters
>> > >> and subscriptions, so you can
>> > >> specify exactly what issues you are interested in, and how often you
>> want
>> > >> to receive these emails.
>> > >> This is much more convenient and more flexible than filtering emails
>> from
>> > >> a
>> > >> bot.
>> > >>
>> > >> So, most people ignore JIRA messages, and the ones who want to track
>> new
>> > >> tickets,
>> > >> may go to JIRA and configure their own filters. I don't see, why we
>> need
>> > >> to
>> > >> keep the forwarding to dev list.
>> > >>
>> > >> Denis
>> > >>
>> > >> пт, 16 нояб. 2018 г. в 22:30, Павлухин Иван :
>> > >>
>> > >> > Hi,
>> > >> >
>> > >> > Can we collect opinions about keeping messages of mentioned types
>> on
>> > >> > dev list? From my side (+ means keeping on dev list):
>> > >> > TC bot +
>> > >> > Jira -
>> > >> > GitHub -
>> > >> > пт, 16 нояб. 2018 г. в 22:25, Dmitriy Pavlov :
>> > >> > >
>> > >> > > Importance is hardly definable and it is not possible that
>> importance
>> > >> is
>> > >> > > equal for everyone. You can say about other human emails it is
>> not
>> > >> > > important if some product area is not interesting for you. So I
>> can
>> > >> only
>> > >> > > understand the terms: email needs action/does not need action.
>> > >> > >
>> > >> > > If some contributor never reacted to JIRA notification he or she
>> may
>> > >> > think
>> > >> > > it is not important. But even we have a majority of contributors
>> who
>> > >> > > ignores JIRA, it does not mean it is a right decision to switch
>> it
>> > >> off.
>> > >> > We
>> > >> > > don't play in a democracy, hopefully.
>> > >> > >
>> > >> > > My suggestion now: keep showing an excellent example of
>> human-human
>> > >> > > interaction, announces, etc from all Ignite veterans (especially,
>> > >> PMCs),
>> > >>

Re: How to free up space on disc after removing entries from IgniteCache with enabled PDS?

2019-01-09 Thread Dmitriy Pavlov
In the TC Bot, I used to create the second cache with CacheV2 name and
migrate needed data from Cache  V1 to V2.

After CacheV1 destroy(), files are removed and disk space is freed.

ср, 9 янв. 2019 г. в 12:04, Павлухин Иван :

> Vyacheslav,
>
> Have you investigated how other vendors (Oracle, Postgres) tackle this
> problem?
>
> I have one wild idea. Could the problem be solved by stopping a node
> which need to be defragmented, clearing persistence files and
> restarting the node? After rebalance the node will receive all data
> back without fragmentation. I see a big downside -- sending data
> across the network. But perhaps we can play with affinity and start
> new node on the same host which will receive the same data, after that
> old node can be stopped. It looks more as kind of workaround but
> perhaps it can be turned into workable solution.
>
> ср, 9 янв. 2019 г. в 10:49, Vyacheslav Daradur :
> >
> > Yes, it's about Page Memory defragmentation.
> >
> > Pages in partitions files are stored sequentially, possible, it makes
> > sense to defragment pages first to avoid interpages gaps since we use
> > pages offset to manage them.
> >
> > I filled an issue [1], I hope we will be able to find resources to
> > solve the issue before 2.8 release.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-10862
> >
> > On Sat, Dec 29, 2018 at 10:47 AM Павлухин Иван 
> wrote:
> > >
> > > I suppose it is about Ignite Page Memory pages defragmentation.
> > >
> > > We can get 100 allocated pages each of which becomes only e.g. 50%
> > > filled after removal some entries. But they will occupy a space for
> > > 100 pages on a hard drive.
> > >
> > > пт, 28 дек. 2018 г. в 20:45, Denis Magda :
> > > >
> > > > Shouldn't the OS care of defragmentation? What we need to do is to
> give a
> > > > way to remove stale data and "release" the allocated space somehow
> through
> > > > the tools, MBeans or API methods.
> > > >
> > > > --
> > > > Denis
> > > >
> > > >
> > > > On Fri, Dec 28, 2018 at 6:24 AM Vladimir Ozerov <
> voze...@gridgain.com>
> > > > wrote:
> > > >
> > > > > Hi Vyacheslav,
> > > > >
> > > > > AFAIK this is not implemented. Shrinking/defragmentation is
> important
> > > > > optimization. Not only because it releases free space, but also
> because it
> > > > > decreases total number of pages. But is it not very easy to
> implement, as
> > > > > you have to both reshuffle data entries and index entries,
> maintaining
> > > > > consistency for concurrent reads and updates at the same time. Or
> > > > > alternatively we can think of offline defragmentation. It will be
> easier to
> > > > > implement and faster, but concurrent operations will be prohibited.
> > > > >
> > > > > On Fri, Dec 28, 2018 at 4:08 PM Vyacheslav Daradur <
> daradu...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Igniters, we have faced with the following problem on one of our
> > > > > > deployments.
> > > > > >
> > > > > > Let's imagine that we have used IgniteCache with enabled PDS
> during the
> > > > > > time:
> > > > > > - hardware disc space has been occupied during growing up of an
> amount
> > > > > > of data, e.g. 100Gb;
> > > > > > - then, we removed non-actual data, e.g 50Gb, which became
> useless for
> > > > > us;
> > > > > > - disc space stopped growing up with new data, but it was not
> > > > > > released, and still took 100Gb, instead of expected 50Gb;
> > > > > >
> > > > > > Another use case:
> > > > > > - a user extracts data from IgniteCache to store it in separate
> > > > > > IgniteCache or another store;
> > > > > > - disc still is occupied and the user is not able to store data
> in the
> > > > > > different cache at the same cluster because of disc limitation;
> > > > > >
> > > > > > How can we help the user to free up the disc space, if an amount
> of
> > > > > > data in IgniteCache has been reduced many times and will not be
> > > > > > increased in the nearest future?
> > > > > >
> > > > > > AFAIK, we have mechanics of reusing memory pages, that allows us
> to
> > > > > > use pages which have been allocated and stored removed data for
> > > > > > storing new data.
> > > > > > Are there any chances to shrink data and free up space on disc
> (with
> > > > > > defragmentation if possible)?
> > > > > >
> > > > > > --
> > > > > > Best Regards, Vyacheslav D.
> > > > > >
> > > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> >
> >
> >
> > --
> > Best Regards, Vyacheslav D.
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>


Re: [DISCUSSION] Relocation of Apache git repositories from git-wip-us.apache.org to GitBox

2019-01-09 Thread Dmitriy Pavlov
Done for the wiki, we need update site.

ср, 9 янв. 2019 г. в 12:28, Andrey Mashenkov :

> Hi Dmitry,
>
> Looks like repository has been transferred to GitBox,
> but for now we still have incorrect links on wiki and ignite web site.
>
> I've created a ticket [1].  Can someone fix this?
>
> https://issues.apache.org/jira/browse/IGNITE-10863
>
> On Sun, Dec 30, 2018 at 1:28 AM Dmitriy Pavlov  wrote:
>
>> Hi Igniters,
>>
>> Thanks to Peter and Alexey for sharing their opinion.
>>
>> I've filed https://issues.apache.org/jira/browse/INFRA-17516 to migrate.
>>
>> Sincerely,
>> Dmitriy Pavlov
>>
>> чт, 13 дек. 2018 г. в 21:47, Peter Ivanov :
>>
>> > There will be no problem with TC, as we use GitHub as main VCS root,
>> using
>> > ASF only for release (which I will reconfigure during release builds
>> > refactoring and optimisations for AI 2.8).
>> >
>> >
>> > > On 11 Dec 2018, at 19:25, Alexey Goncharuk <
>> alexey.goncha...@gmail.com>
>> > wrote:
>> > >
>> > > Given that there is no option to stay on the old repository and
>> > mass-migration is scheduled for Feb, 7th, I think it is better to
>> prepare
>> > and move the repository before the date.
>> > >
>> > > As far as I understand, from developers standpoint we only need to
>> > change the git remote entry. Petr, Sergey, can you asses what changes
>> are
>> > need to be done in order to support this migration for TC and release
>> > procedure?
>> > >
>> > > вт, 11 дек. 2018 г. в 19:20, Dmitriy Pavlov > > > dpav...@apache.org>>:
>> > > Hi All,
>> > >
>> > > The Apache Ignite still has a repository on git-wip-us for its code,
>> as
>> > > well, for release.
>> > >
>> > > Does anyone have a problem with moving over the Ignite git-wip-us
>> > > repository to gitbox voluntarily? This means integrated access and
>> easy
>> > PRs
>> > > (write access to the GitHub repo).
>> > >
>> > > We need to document support for the decision from a mailing list
>> post, so
>> > > here it is.
>> > >
>> > > Sincerely,
>> > > Dmitriy Pavlov
>> > >
>> > > сб, 8 дек. 2018 г. в 00:16, Dmitriy Pavlov > > > dpav...@apache.org>>:
>> > >
>> > > > Hi Igniters,
>> > > >
>> > > > What do you think about the voluntary migration of project
>> repositories
>> > > > from git-wip to GitBox? (See details in the forwarded email.)
>> > > >
>> > > > Redirection of emails originated by PR actions to notifications@
>> > seems to
>> > > > be almost finished, so email flood issue seems to be already solved
>> > for dev@
>> > > > .
>> > > >
>> > > > It is up to us to decide if we would like to migrate sooner. Later
>> all
>> > > > repositories will be migrated anyway.
>> > > >
>> > > > Affected repositories
>> > > > - Ignite Release
>> > > > - Ignite
>> > > >
>> > > > Please share your opinion.
>> > > >
>> > > > Sincerely,
>> > > > Dmitriy Pavlov
>> > > >
>> > > > -- Forwarded message -
>> > > > From: Daniel Gruno > humbed...@apache.org
>> > >>
>> > > > Date: пт, 7 дек. 2018 г. в 19:52
>> > > > Subject: [NOTICE] Mandatory relocation of Apache git repositories on
>> > > > git-wip-us.apache.org <http://git-wip-us.apache.org/>
>> > > > To: us...@infra.apache.org <mailto:us...@infra.apache.org> <
>> > us...@infra.apache.org <mailto:us...@infra.apache.org>>
>> > > >
>> > > >
>> > > > [IF YOUR PROJECT DOES NOT HAVE GIT REPOSITORIES ON GIT-WIP-US PLEASE
>> > > >   DISREGARD THIS EMAIL; IT WAS MASS-MAILED TO ALL APACHE PROJECTS]
>> > > >
>> > > > Hello Apache projects,
>> > > >
>> > > > I am writing to you because you may have git repositories on the
>> > > > git-wip-us server, which is slated to be decommissioned in the
>> coming
>> > > > months. All repositories will be moved to the new gitbox service
>> which
>> > > > includes direct write access on github as well as the

Fwd: The ASF needs your help! "The Apache Way To Me..."

2019-01-09 Thread Dmitriy Pavlov
Hi Igniters,

Please find a free minute and fill survey:

https://s.apache.org/oxTr   DEADLINE: 15 February 2019

Sincerely,
Dmitriy Pavlov

-- Forwarded message -
From: Sally Khudairi 
Date: ср, 9 янв. 2019 г. в 09:13
Subject: Fwd: The ASF needs your help! "The Apache Way To Me..."
To: ASF Marketing & Publicity 


Hello Apache PMCs (in blind copy) --I hope you are all well.

For those of you who missed the email to committers@ , we'd love your
cooperation with this project.

Your help is much appreciated!

Warm thanks,
Sally

- - -
Vice President Marketing & Publicity
Vice President Sponsor Relations
The Apache Software Foundation

Tel +1 617 921 8656 <(617)%20921-8656> | s...@apache.org

- Original message -
From: Sally Khudairi 
To: committ...@apache.org
Subject: The ASF needs your help! "The Apache Way To Me..."
Date: Wed, 09 Jan 2019 00:02:08 -0500

Hello fellow ASF Committers --the ASF needs your help.

We'll be celebrating our 20th Anniversary at the end of March!

As you know, the key to our success of stewarding 300+ Apache projects is
community-led development "The Apache Way".

And as Apache Committers, you are entrusted to help educate and propagate
The Apache Way across your projects and their communities. We would love to
hear your thoughts and experiences on The Apache Way by taking the short
survey below.

https://s.apache.org/oxTr  DEADLINE: 15 February 2019

Please feel free to share this questionnaire with others as broadly as
possible. We'll also be announcing to the PMCs and promoting across our
social media channels.

Keep an eye out for other Anniversary details to be shared in the coming
months.

Thank you in advance for your help with this. As always, feel free to let
me know if you have any questions.

Warmest regards,
Sally

- - -
Vice President Marketing & Publicity
Vice President Sponsor Relations
The Apache Software Foundation

Tel +1 617 921 8656 <(617)%20921-8656> | s...@apache.org


Re: IGNITE-10856 cassandra-store seems to be broken by incorrect guava version

2019-01-08 Thread Dmitriy Pavlov
Hi Saikat,

Fix itself is ok. But from the top of my head, I don't know how can we
auto-test this scenario in the future. Do you have an idea?

Sincerely,
Dmitriy Pavlov

ср, 9 янв. 2019 г. в 07:08, Saikat Maitra :

> Hi,
>
> I have raised a PR for the below issue.
>
> IGNITE-10856 cassandra-store seems to be broken by incorrect guava version
>
> PR - https://github.com/apache/ignite/pull/5780/
>
> Please take a look and share feedback.
>
> Regards,
> Saikat
>


Re: Problem with reading incomplete payload - IGNITE-7153

2019-01-03 Thread Dmitriy Pavlov
Hi Igniters,

I'm trying to reach the author of the fix because the ticket is still in In
Progress.

Could you please advice me how to handle it (because fix seems to be
useful)? Can we set Patch Available status by lazy consensus and review
possibly incomplete fix?
https://issues.apache.org/jira/browse/IGNITE-7153

Sincerely,
Dmitriy Pavlov

пт, 2 нояб. 2018 г. в 13:20, Michael Fong :

> Hi Yakov,
>
> Thanks so much for your analysis.
>
> Parser expects chunks to be complete and has all the data to read entire
> > message, but this is not guaranteed and single message can arrive in
> > several chunks.
>
> This is indeed the the assumption to my implementation. I have not come up
> a another parsing algorithm to handle this rainy day case. Perhaps, it
> would require more refactoring on existing code. In addition, I might need
> to check how Redis dev implements the parser state machine.
>
> I would be interested to see how current implementation (based on
> 2.6/master) behaves if we intentionally split the message into chunks as
> you suggested for the reproducer.
>
> Regards,
>
> Michael
>
> On Wed, Oct 31, 2018 at 7:08 PM Yakov Zhdanov  wrote:
>
> > Hi Mike!
> >
> > Thanks for reproducer. Now I understand the problem. NIO worker reads
> > chunks from the network and notifies the parser on data read. Parser
> > expects chunks to be complete and has all the data to read entire
> message,
> > but this is not guaranteed and single message can arrive in several
> chunks.
> > Which is demostrated by your test.
> >
> > The problem is inside GridRedisProtocolParser. We should add ability to
> > store the parsing context if we do not have all the data to complete
> > message parsing, as it is done, for example in GridBufferedParser. So, it
> > is definitely an issue and should be fixed by adding parsing state. I see
> > you attempted to do so in PR
> > https://github.com/apache/ignite/pull/5044/files. I did not do a formal
> > review, so let's ask community to review your patch.
> >
> > Couple of comments about your reproducer.
> >
> > 1. Let's dump a proper Redis message bytes sent by Jedis.
> > 2. Let's split this dump into 5 chunks and send them with 100 ms delays.
> >
> > This should fail before fix is applied, and should pass with proper
> message
> > parsed after we have the issue fixed.
> >
> > Thanks!
> >
> > --Yakov
> >
>


Apache Ignite Codebase has been migrated to GitBox

2018-12-30 Thread Dmitriy Pavlov
Dear Ignite Developers,

I would like to repeat announce from other thread with more details:

Apache Ignite *New* Git Remotes for repositories hosted on git-wip-us:
https://gitbox.apache.org/repos/asf/ignite.git
https://gitbox.apache.org/repos/asf/ignite-release.git

Mirrors are the same:
https://github.com/apache/ignite/
https://github.com/apache/ignite-release/

Ignite-doc has been deleted as it has no commits. PMC can create new
additional repositories https://selfserve.apache.org/

Committers can set up account matching between GitHub and Apache. After
this setup, you can merge pull requests from GitHub Web UI and commit files
directly to master branch via GitHub. You can run this setup here
https://gitbox.apache.org/setup/

Sincerely,
Dmitriy Pavlov


Re: [DISCUSSION] Relocation of Apache git repositories from git-wip-us.apache.org to GitBox

2018-12-30 Thread Dmitriy Pavlov
Hi Igniters,

the main Git repository for Apache Ignite and Ignite release have been
migrated to GitBox.

New Remotes:
https://gitbox.apache.org/repos/asf/ignite.git
https://gitbox.apache.org/repos/asf?p=ignite-release.git

Committers, please update your remotes.

Please share your vision if we can for now simply delete empty
https://git-wip-us.apache.org/repos/asf?p=ignite-doc.git

Sincerely,
Dmitriy Pavlov


вс, 30 дек. 2018 г. в 01:28, Dmitriy Pavlov :

> Hi Igniters,
>
> Thanks to Peter and Alexey for sharing their opinion.
>
> I've filed https://issues.apache.org/jira/browse/INFRA-17516 to migrate.
>
> Sincerely,
> Dmitriy Pavlov
>
> чт, 13 дек. 2018 г. в 21:47, Peter Ivanov :
>
>> There will be no problem with TC, as we use GitHub as main VCS root,
>> using ASF only for release (which I will reconfigure during release builds
>> refactoring and optimisations for AI 2.8).
>>
>>
>> > On 11 Dec 2018, at 19:25, Alexey Goncharuk 
>> wrote:
>> >
>> > Given that there is no option to stay on the old repository and
>> mass-migration is scheduled for Feb, 7th, I think it is better to prepare
>> and move the repository before the date.
>> >
>> > As far as I understand, from developers standpoint we only need to
>> change the git remote entry. Petr, Sergey, can you asses what changes are
>> need to be done in order to support this migration for TC and release
>> procedure?
>> >
>> > вт, 11 дек. 2018 г. в 19:20, Dmitriy Pavlov > <mailto:dpav...@apache.org>>:
>> > Hi All,
>> >
>> > The Apache Ignite still has a repository on git-wip-us for its code, as
>> > well, for release.
>> >
>> > Does anyone have a problem with moving over the Ignite git-wip-us
>> > repository to gitbox voluntarily? This means integrated access and easy
>> PRs
>> > (write access to the GitHub repo).
>> >
>> > We need to document support for the decision from a mailing list post,
>> so
>> > here it is.
>> >
>> > Sincerely,
>> > Dmitriy Pavlov
>> >
>> > сб, 8 дек. 2018 г. в 00:16, Dmitriy Pavlov > dpav...@apache.org>>:
>> >
>> > > Hi Igniters,
>> > >
>> > > What do you think about the voluntary migration of project
>> repositories
>> > > from git-wip to GitBox? (See details in the forwarded email.)
>> > >
>> > > Redirection of emails originated by PR actions to notifications@
>> seems to
>> > > be almost finished, so email flood issue seems to be already solved
>> for dev@
>> > > .
>> > >
>> > > It is up to us to decide if we would like to migrate sooner. Later all
>> > > repositories will be migrated anyway.
>> > >
>> > > Affected repositories
>> > > - Ignite Release
>> > > - Ignite
>> > >
>> > > Please share your opinion.
>> > >
>> > > Sincerely,
>> > > Dmitriy Pavlov
>> > >
>> > > -- Forwarded message -
>> > > From: Daniel Gruno mailto:humbed...@apache.org
>> >>
>> > > Date: пт, 7 дек. 2018 г. в 19:52
>> > > Subject: [NOTICE] Mandatory relocation of Apache git repositories on
>> > > git-wip-us.apache.org <http://git-wip-us.apache.org/>
>> > > To: us...@infra.apache.org <mailto:us...@infra.apache.org> <
>> us...@infra.apache.org <mailto:us...@infra.apache.org>>
>> > >
>> > >
>> > > [IF YOUR PROJECT DOES NOT HAVE GIT REPOSITORIES ON GIT-WIP-US PLEASE
>> > >   DISREGARD THIS EMAIL; IT WAS MASS-MAILED TO ALL APACHE PROJECTS]
>> > >
>> > > Hello Apache projects,
>> > >
>> > > I am writing to you because you may have git repositories on the
>> > > git-wip-us server, which is slated to be decommissioned in the coming
>> > > months. All repositories will be moved to the new gitbox service which
>> > > includes direct write access on github as well as the standard ASF
>> > > commit access via gitbox.apache.org <http://gitbox.apache.org/>.
>> > >
>> > > ## Why this move? ##
>> > > The move comes as a result of retiring the git-wip service, as the
>> > > hardware it runs on is longing for retirement. In lieu of this, we
>> > > have decided to consolidate the two services (git-wip and gitbox), to
>> > > ease the management of our repository systems and future-proof the
>> > > underlying 

Re: [MTCGA]: new failures in builds [2662229, 2662228] needs to be handled

2018-12-27 Thread Dmitriy Pavlov
Nikolay, thank you for fixing

чт, 27 дек. 2018 г. в 23:00, Dmitriy Pavlov :

> Thanks to Michail Petrov we have now TC Bot notifications on compilation
> failures.
>
>
> чт, 27 дек. 2018 г. в 22:07, :
>
>> Hi Igniters,
>>
>>  I've detected some new issue on TeamCity to be handled. You are more
>> than welcomed to help.
>>
>>  If your changes can lead to this failure(s): We're grateful that you
>> were a volunteer to make the contribution to this project, but things
>> change and you may no longer be able to finalize your contribution.
>>  Could you respond to this email and indicate if you wish to continue and
>> fix test failures or step down and some committer may revert you commit.
>>
>>  *New Critical Failure in master _Javadoc_
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Javadoc=%3Cdefault%3E=buildTypeStatusDiv
>>  Changes may lead to failure were done by
>>  - eshangareev
>> https://ci.ignite.apache.org/viewModification.html?modId=855404
>>  - oignatenko
>> https://ci.ignite.apache.org/viewModification.html?modId=855401
>>  - oignatenko
>> https://ci.ignite.apache.org/viewModification.html?modId=855397
>>
>>  *New Critical Failure in master ~Build Apache Ignite~
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_BuildApacheIgnite=%3Cdefault%3E=buildTypeStatusDiv
>>  Changes may lead to failure were done by
>>  - eshangareev
>> https://ci.ignite.apache.org/viewModification.html?modId=855404
>>  - oignatenko
>> https://ci.ignite.apache.org/viewModification.html?modId=855401
>>  - oignatenko
>> https://ci.ignite.apache.org/viewModification.html?modId=855397
>>
>>  - Here's a reminder of what contributors were agreed to do
>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>>  - Should you have any questions please contact
>> dev@ignite.apache.org
>>
>> Best Regards,
>> Apache Ignite TeamCity Bot
>> https://github.com/apache/ignite-teamcity-bot
>> Notification generated at 22:07:01 27-12-2018
>>
>


Re: [MTCGA]: new failures in builds [2662229, 2662228] needs to be handled

2018-12-27 Thread Dmitriy Pavlov
Thanks to Michail Petrov we have now TC Bot notifications on compilation
failures.


чт, 27 дек. 2018 г. в 22:07, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *New Critical Failure in master _Javadoc_
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Javadoc=%3Cdefault%3E=buildTypeStatusDiv
>  Changes may lead to failure were done by
>  - eshangareev
> https://ci.ignite.apache.org/viewModification.html?modId=855404
>  - oignatenko
> https://ci.ignite.apache.org/viewModification.html?modId=855401
>  - oignatenko
> https://ci.ignite.apache.org/viewModification.html?modId=855397
>
>  *New Critical Failure in master ~Build Apache Ignite~
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_BuildApacheIgnite=%3Cdefault%3E=buildTypeStatusDiv
>  Changes may lead to failure were done by
>  - eshangareev
> https://ci.ignite.apache.org/viewModification.html?modId=855404
>  - oignatenko
> https://ci.ignite.apache.org/viewModification.html?modId=855401
>  - oignatenko
> https://ci.ignite.apache.org/viewModification.html?modId=855397
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 22:07:01 27-12-2018
>


[jira] [Created] (IGNITE-10832) [TC Bot] Auto-trigger check win agents availability before triggering a build

2018-12-27 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-10832:
---

 Summary: [TC Bot] Auto-trigger check win agents availability 
before triggering a build
 Key: IGNITE-10832
 URL: https://issues.apache.org/jira/browse/IGNITE-10832
 Project: Ignite
  Issue Type: Task
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov


Currently, Win Agents may be a bottle-neck in TC throughput.

So it is reasonable to check constraint availability, but not only overall pool 
of agents



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Let's force -Dfile.encoding=UTF-8

2018-12-26 Thread Dmitriy Pavlov
+1 from my side. I think it is reasonable

ср, 26 дек. 2018 г. в 19:20, Ilya Kasnacheev :

> Hello!
>
> It was recently discovered that there's some issue in H2 which leads to
> differing string encodings when they come into reducer from nodes with
> different system encoding. Even if we were able to fix this, I suspect that
> user code may too be bitten by mismatch of this setting.
>
> I propose to force UTF-8 as system encoding at all times when we control
> how JVM is launched.
> This includes ignite.sh, ignite.bat, Apache.Ignite.exe and C++'s ./ignite.
>
> This will mainly affect Windows systems as I expect that Linux will most
> always use UTF-8 locale and Mac OS X should always be UTF-8.
>
> file.encoding is somewhat misleading name since it specifies the default
> string encoding, such as the one used for String.getBytes(). It is a common
> convention to set ut to UTF-8, for example, IDEA will do that.
>
> WDYT?
>
> There's a pull request: https://github.com/apache/ignite/pull/5725
> If somebody could contribute C++ and .Net tests I would be also grateful.
>
> Regards,
> --
> Ilya Kasnacheev
>


Re: Request for review : IGNITE-3303 Apache Flink Integration - Flink source

2018-12-25 Thread Dmitriy Pavlov
Hi All,

Please check the commit
https://git-wip-us.apache.org/repos/asf?p=ignite.git;a=commit;h=3b9c415ee2c4b6461c18f87f46f1950f324f5662

Thanks to everyone involved.

Sincerely,
Dmitriy Pavlov

вт, 25 дек. 2018 г. в 19:38, Dmitriy Pavlov :

> Hi Saikat,
>
> I'm going to slightly change the code before commit.
>
> First of all, I will add more code style and abbreviation rules related
> changes.
> Second of all, I add pom.xml variables usages where possible.
> And last but not least, I'm going to change the implementation of
> IgniteSource.
>
> AFAIK drainTo() method is not blocking so IgniteSource will be
> continuously spinning if there are no events coming from a cache. So I'm
> going to commit this variant:
>
> while (isRunning) {
> // block here for some time if there is no events from source
> CacheEvent firstEvt = evtBuf.poll(1, TimeUnit.SECONDS);
>
> if (firstEvt != null)
> evts.add(firstEvt);
>
> if (evtBuf.drainTo(evts, evtBatchSize) > 0) {
> synchronized (ctx.getCheckpointLock()) {
> for (CacheEvent evt : evts)
> ctx.collect(evt);
>
> evts.clear();
> }
> }
> }
>
>
> I hope you don't mind. This option allows us both to stop and does not spin 
> in the while.
>
>
> Sincerely,
>
> Dmitriy Pavlov
>
>
> сб, 13 окт. 2018 г. в 20:37, Saikat Maitra :
>
>> Hi Alex ,
>>
>> Can you please review and let me know if the PR looks good to merge.
>>
>> I have completed the requested changes.
>>
>> Regards,
>> Saikat
>>
>> On Mon, Oct 1, 2018 at 9:24 PM Saikat Maitra 
>> wrote:
>>
>> > Thank you everyone for reviewing the changes. As discussed I have
>> removed
>> > the FlinkIgniteSourceSelfExample and also verified that
>> FlinkIgniteSourceSelfTestSuite
>> > is part of Ignite Streamers in Team City.
>> >
>> > Please let me know if any further changes are required.
>> >
>> > Alex , will you please review and let me know if the PR looks good?
>> >
>> > Regards,
>> > Saikat
>> >
>> > On Mon, Oct 1, 2018 at 11:58 AM Nikolay Izhikov 
>> > wrote:
>> >
>> >> Hello, Andrey.
>> >>
>> >> Yes, I know it.
>> >> I've looked at the PR befor mailing :)
>> >>
>> >> Do you think we can include this improvement to the 2.7 release?
>> >> Do you have time to assist Saikat with TC setup and other tasks?
>> >>
>> >>
>> >> В Пн, 01/10/2018 в 19:54 +0300, Andrey Mashenkov пишет:
>> >> > Dmitry, Nikolay,
>> >> >
>> >> > Ignite-3303 is a new Ignite module and there is no changes related to
>> >> core or other existed modules.
>> >> > So, PR should not affected existed functional ity and can be safely
>> >> merged.
>> >> >
>> >> > Thanks.
>> >> >
>> >> >
>> >> > пн, 1 окт. 2018 г., 16:04 Dmitriy Pavlov :
>> >> > > Hi Saikat,
>> >> > >
>> >> > > I don't mind merging to master, but I have concern if it will go to
>> >> 2.7. In
>> >> > > the separate discussion, we agreed on code freeze should happen
>> >> during last
>> >> > > weekend, September, 30.
>> >> > >
>> >> > > So it is now up to community and release manager to decide if fix
>> >> should go
>> >> > > to the upcoming release. Usually, after the freeze, only bug/test
>> >> fixes can
>> >> > > be merged to release branch.
>> >> > >
>> >> > > Hi Nikolay,
>> >> > >
>> >> > > could you please announce that code freeze happened?
>> >> > >
>> >> > > Sincerely,
>> >> > > Dmitriy Pavlov
>> >> > >
>> >> > > пн, 1 окт. 2018 г. в 3:58, Saikat Maitra > >:
>> >> > >
>> >> > > > Hi Alex, Nicolay
>> >> > > >
>> >> > > > As discussed with Andrew the changes looks good. Would it be ok
>> to
>> >> merge
>> >> > > > this change to master considering the 2.7 release plan?
>> >> > > >
>> >> > > > Regards,
>> >> > > > Saikat
>> >> > > >
>> >> > > > On Fri, Sep 28, 2018 at 7:15 PM Saikat Maitra <
>> >&g

Re: Request for review : IGNITE-3303 Apache Flink Integration - Flink source

2018-12-25 Thread Dmitriy Pavlov
Hi Saikat,

I'm going to slightly change the code before commit.

First of all, I will add more code style and abbreviation rules related
changes.
Second of all, I add pom.xml variables usages where possible.
And last but not least, I'm going to change the implementation of
IgniteSource.

AFAIK drainTo() method is not blocking so IgniteSource will be continuously
spinning if there are no events coming from a cache. So I'm going to commit
this variant:

while (isRunning) {
// block here for some time if there is no events from source
CacheEvent firstEvt = evtBuf.poll(1, TimeUnit.SECONDS);

if (firstEvt != null)
evts.add(firstEvt);

if (evtBuf.drainTo(evts, evtBatchSize) > 0) {
synchronized (ctx.getCheckpointLock()) {
for (CacheEvent evt : evts)
ctx.collect(evt);

evts.clear();
}
}
}


I hope you don't mind. This option allows us both to stop and does not
spin in the while.


Sincerely,

Dmitriy Pavlov


сб, 13 окт. 2018 г. в 20:37, Saikat Maitra :

> Hi Alex ,
>
> Can you please review and let me know if the PR looks good to merge.
>
> I have completed the requested changes.
>
> Regards,
> Saikat
>
> On Mon, Oct 1, 2018 at 9:24 PM Saikat Maitra 
> wrote:
>
> > Thank you everyone for reviewing the changes. As discussed I have removed
> > the FlinkIgniteSourceSelfExample and also verified that
> FlinkIgniteSourceSelfTestSuite
> > is part of Ignite Streamers in Team City.
> >
> > Please let me know if any further changes are required.
> >
> > Alex , will you please review and let me know if the PR looks good?
> >
> > Regards,
> > Saikat
> >
> > On Mon, Oct 1, 2018 at 11:58 AM Nikolay Izhikov 
> > wrote:
> >
> >> Hello, Andrey.
> >>
> >> Yes, I know it.
> >> I've looked at the PR befor mailing :)
> >>
> >> Do you think we can include this improvement to the 2.7 release?
> >> Do you have time to assist Saikat with TC setup and other tasks?
> >>
> >>
> >> В Пн, 01/10/2018 в 19:54 +0300, Andrey Mashenkov пишет:
> >> > Dmitry, Nikolay,
> >> >
> >> > Ignite-3303 is a new Ignite module and there is no changes related to
> >> core or other existed modules.
> >> > So, PR should not affected existed functional ity and can be safely
> >> merged.
> >> >
> >> > Thanks.
> >> >
> >> >
> >> > пн, 1 окт. 2018 г., 16:04 Dmitriy Pavlov :
> >> > > Hi Saikat,
> >> > >
> >> > > I don't mind merging to master, but I have concern if it will go to
> >> 2.7. In
> >> > > the separate discussion, we agreed on code freeze should happen
> >> during last
> >> > > weekend, September, 30.
> >> > >
> >> > > So it is now up to community and release manager to decide if fix
> >> should go
> >> > > to the upcoming release. Usually, after the freeze, only bug/test
> >> fixes can
> >> > > be merged to release branch.
> >> > >
> >> > > Hi Nikolay,
> >> > >
> >> > > could you please announce that code freeze happened?
> >> > >
> >> > > Sincerely,
> >> > > Dmitriy Pavlov
> >> > >
> >> > > пн, 1 окт. 2018 г. в 3:58, Saikat Maitra :
> >> > >
> >> > > > Hi Alex, Nicolay
> >> > > >
> >> > > > As discussed with Andrew the changes looks good. Would it be ok to
> >> merge
> >> > > > this change to master considering the 2.7 release plan?
> >> > > >
> >> > > > Regards,
> >> > > > Saikat
> >> > > >
> >> > > > On Fri, Sep 28, 2018 at 7:15 PM Saikat Maitra <
> >> saikat.mai...@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Thank you Andrew
> >> > > > >
> >> > > > > Regards,
> >> > > > > Saikat
> >> > > > >
> >> > > > > On Fri, Sep 28, 2018 at 7:00 PM Andrey Mashenkov <
> >> > > > > andrey.mashen...@gmail.com> wrote:
> >> > > > >
> >> > > > >> Hi Saikat,
> >> > > > >>
> >> > > > >> Sorry for late answer. I've checked changes a day ago. Now,
> >> looks good.
> >> > > > >> Hope, it will be merged soo

[jira] [Created] (IGNITE-10787) [TC Bot] Setup TC bot profiling on the server using SJK/stcap and prepare rules for Ignite thread groups

2018-12-21 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-10787:
---

 Summary: [TC Bot] Setup TC bot profiling on the server using 
SJK/stcap and prepare rules for Ignite thread groups
 Key: IGNITE-10787
 URL: https://issues.apache.org/jira/browse/IGNITE-10787
 Project: Ignite
  Issue Type: Task
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov


https://github.com/aragozin/jvm-tools

Integrate tool into start scrips, prepare settings for capturing
Prepare traces analysis and include into server monitoring page
https://mtcga.gridgain.com/monitoring.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Critical worker threads liveness checking drawbacks

2018-12-19 Thread Dmitriy Pavlov
Hi,

Sorry for being too formal here, but IGNITE-10003
<https://issues.apache.org/jira/browse/IGNITE-10003> is in progress.

Also, I've tried to find anything related to it in the list. So according
to the list, no one was asking to include.

Sincerely,
Dmitriy Pavlov

ср, 19 дек. 2018 г. в 13:24, Nikolay Izhikov :

> Hello, Alexey.
>
> No, we don't include this ticket to 2.7.
> Should we?
>
> ср, 19 дек. 2018 г. в 12:55, Alexey Goncharuk  >:
>
> > Folks, why did not we include IGNITE-10003 to ignite-2.7 release scope?
> > This causes an Ignite node to be stopped by default when checkpoint read
> > lock acquire times out. I expect a lot of Ignite 2.7 users will be
> affected
> > by this mistake.
> >
> > We should at least update the documentation and make users aware of a
> > workaround.
> >
> > чт, 25 окт. 2018 г. в 16:35, Alexey Goncharuk <
> alexey.goncha...@gmail.com
> > >:
> >
> > > Andrey,
> > >
> > > I still see that checkpoint read lock acquisition raises a
> > CRITICAL_ERROR,
> > > which by default will shut down local node. As far as I remember, we
> > > decided that by default thread timeout should not trigger node failure.
> > > Now, however, it does, because we ignore SYSTEM_WORKER_BLOCKED events
> in
> > > default configuration.
> > >
> > > Should we introduce another critical failure type
> > > CHECKPOINT_READ_LOCK_BLOCKED or use SYSTEM_WORKER_BLOCKED for
> checkpoint
> > > read lock acquire failure?
> > >
> > > --AG
> > >
> > > пт, 12 окт. 2018 г. в 8:29, Andrey Kuznetsov :
> > >
> > >> Igniters,
> > >>
> > >> Now I spot blocking / long-running code arising from
> > >> {{GridDhtPartitionsExchangeFuture#init}} calls in partition-exchanger
> > >> thread, see [1]. Ideally, all blocking operations along all possible
> > code
> > >> paths should be guarded implicitly from critical failure detector to
> > avoid
> > >> the thread from being considered blocked. There is a pull request [2]
> > that
> > >> provides shallow solution. I didn't change code outside
> > >> {{GridDhtPartitionsExchangeFuture}}, otherwise it could be broken by
> any
> > >> upcoming change. Also, I didn't touch the code runnable by threads
> other
> > >> than partition-exchanger. So I have a number of guarded sections that
> > are
> > >> wider than they could be, and this potentially hides issues from
> failure
> > >> detector. Does this PR make sense? Or maybe it's better to exclude
> > >> partition-exchanger from critical threads registry at all?
> > >>
> > >> [1] https://issues.apache.org/jira/browse/IGNITE-9710
> > >> [2] https://github.com/apache/ignite/pull/4962
> > >>
> > >>
> > >> пт, 28 сент. 2018 г. в 18:56, Maxim Muzafarov :
> > >>
> > >> > Andrey, Andrey
> > >> >
> > >> > > Thanks for being attentive! It's definitely a typo. Could you
> please
> > >> > create
> > >> > > an issue?
> > >> >
> > >> > I've created an issue [1] and prepared PR [2].
> > >> > Please, review this change.
> > >> >
> > >> > [1] https://issues.apache.org/jira/browse/IGNITE-9723
> > >> > [2] https://github.com/apache/ignite/pull/4862
> > >> >
> > >> > On Fri, 28 Sep 2018 at 16:58 Yakov Zhdanov 
> > wrote:
> > >> >
> > >> > > Config option + mbean access. Does that make sense?
> > >> > >
> > >> > > Yakov
> > >> > >
> > >> > > On Fri, Sep 28, 2018, 17:17 Vladimir Ozerov  >
> > >> > wrote:
> > >> > >
> > >> > > > Then it should be config option.
> > >> > > >
> > >> > > > пт, 28 сент. 2018 г. в 13:15, Andrey Gura :
> > >> > > >
> > >> > > > > Guys,
> > >> > > > >
> > >> > > > > why we need both config option and system property? I believe
> > one
> > >> way
> > >> > > is
> > >> > > > > enough.
> > >> > > > > On Fri, Sep 28, 2018 at 12:38 PM Nikolay Izhikov <
> > >> > nizhi...@apache.org>
> > >> > > > > wrote:
> > >> > > > > &

Re: Is it time to move forward to JUnit4 (5)?

2018-12-19 Thread Dmitriy Pavlov
Hi Igniters,

I've tried yesterday to find any of @Ignore d tests with a link to the
issue in its value on TeamCity, but I did not manage to.

Do you know if JUnit Ignored tests are completely invisible in the TC and
its REST? I've also tried to google it, but not found anything yet.

I remember that .NET Ignores are at least visible in the UI and in a REST
request (see, for example,
https://ci.ignite.apache.org/viewLog.html?buildId=2590524=buildResultsDiv=IgniteTests24Java8_PlatformNet_IgniteTests24Java8=%3Cdefault%3E
).

Please advice me or share a link to a build with Java JUnit Ignores, which
is visible in TC.

Sincerely,
Dmitriy Pavlov

ср, 19 дек. 2018 г. в 11:59, Павлухин Иван :

> Hi Oleg,
>
> I have not quite understood who is going to call (from standpoint of
> test framework) beforeTestsStarted, beforeTest, afterTest,
> afterTestsStarted methods?
> вт, 18 дек. 2018 г. в 23:31, oignatenko :
> >
> > Hi Ivan,
> >
> > To answer your last question, yes, all the tests are to be marked with
> @Test
> > annotations, and those that are meant to be ignored are going to be
> marked
> > with @Ignore annotations. There is no exceptions to that, and these
> > annotations will work just as well on tests using our home brewed
> > beforeTestsStarted, beforeTest, afterTest, afterTestsStarted.
> >
> > For the sake of completeness, developers will also be able to use
> Before* /
> > After* annotations on their own methods in tests. The only exception
> > (clarified in respective javadocs) is that these annotations shouldn't be
> > used on overrides of our home brewed methods - and these methods, in
> > addition, will be recommended (not mandated) to avoid wia deprecation
> > notices.
> >
> > -
> >
> > As for accessing tests which have problems running under junit4, the way
> how
> > I search for these in current master is regex search in IDEA for
> > "addTestSuite.*class", that is I search in testsuites for entries that
> are
> > added using method "addTestSuite" with parameter class.
> >
> > Probably worth noting that some of the problems that were previously
> > blocking addition of particular tests have been resolved in the course of
> > working on IGNITE-10177 (https://github.com/apache/ignite/pull/5615).
> One
> > riddle that currently looks particularly difficult to crack is Teamcity
> > failures in "Queries 1", I even haven't yet figured how to reproduce
> these
> > locally.
> >
> > regards, Oleg
> >
> >
> > Павлухин Иван wrote
> > > Hi Oleg,
> > >
> > > Now concerns about junit3 elimination are clear for me. And I agree
> > > that it is worth to do it. By the way is it possible to access tests
> > > which have problems running under junit4? I would like to take a look.
> > >
> > > Also a clarifying bit regarding next migration steps is needed. Sorry
> > > if it was described but I missed it. Currently we have tons of tests
> > > which rely on our home brewed beforeTestsStarted, beforeTest,
> > > afterTest, afterTestsStarted. Are you going to mark them all with
> > > corresponding junit4 annotations?
> > > пн, 17 дек. 2018 г. в 19:13, oignatenko 
> >
> > > oignatenko@
> >
> > > :
> > >>
> > >> Hi Ivan,
> > >>
> > >> Per my cursory check of the code currently in master, we have 239 test
> > >> classes that couldn't migrate (that's probably about 500 or something
> > >> test
> > >> cases). For comparison, over 1600 classes migrated without problems.
> This
> > >> is
> > >> to answer your questions about how many tests are affected by change
> and
> > >> how many tests were not migrated yet.
> > >>
> > >> -
> > >>
> > >> I think we can say that only small percent of tests turned out
> sensitive
> > >> to
> > >> JUnit framework change.
> > >>
> > >> In the same time, due to very large overall amount of our tests we
> have
> > >> quite a big number as an absolute value. I point this because if
> amount
> > >> of
> > >> troublesome tests was smaller (say, order of magnitude smaller) I
> would
> > >> consider delaying migration until all tests reworked to blend totally
> > >> seamlessly into new version JUnit. This is doable - as sort of "pilot
> > >> exercise" me and Ed adjusted one test case this way - but the sheer
> > >> amount
> > >> of work on 200+

Re: Code inspection

2018-12-18 Thread Dmitriy Pavlov
Both patches were applied. Maxim, thank you!

What about 1. An `Unexpected error during build messages processing in
TeamCity`, what can we do as the next step to fix it?

Sincerely,
Dmitriy Pavlov

пн, 17 дек. 2018 г. в 18:31, Andrey Mashenkov :

> Maxim,
>
> Looks ok. Let's apply IGNITE-10682.
>
> All,
>
> Also, I'd like to publish idea.logs into artefacts by default.
> This will give us more details for investigation in future if any failure
> will occurs.
> It will costs 1-10 kB.
>
> On Mon, Dec 17, 2018 at 3:21 PM Maxim Muzafarov 
> wrote:
>
> > Dmitry,
> >
> > It seems to me that we have two independent issues here.
> > 1. An `Unexpected error during build messages processing in TeamCity`
> > error message which is related to TC agent configuration. Suppose,
> > server.log will provide us more details about it. I have to access
> > there.
> > 2. A new set of inspection rules was introduced in 2018+ IntelliJ IDEA
> > and they should be disabled in our ignite_inspections_teamcity.xml
> > configuration file. They are not fixed in the Apache Ignite project
> > code yet. I've prepared the issue [1] for it. Please, take a look.
> >
> >
> > Andrey,
> >
> > I've fixed disabled plugins file according to your suggestions. The
> > issue [2] is ready. I've re-run `Excluded [Inspections] Core Debug`
> > suite and the log details show me that now only 3 plugins are enabled:
> > IDEA CORE, Maven Integration, Properties Support. It seems to me that
> > it's correct.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-10709
> > [2] https://issues.apache.org/jira/browse/IGNITE-10682
> >
> > On Sat, 15 Dec 2018 at 15:22, Dmitriy Pavlov  wrote:
> > >
> > > Folks,
> > >
> > > There is a strange error on TC
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=2556875=IgniteTests24Java8_InspectionsCore
> > >
> > > It appeared after TC update to the latest version.
> > >
> > > Sincerely,
> > > Dmitry Pavlov
> > >
> > > пт, 14 дек. 2018 г. в 16:09, Andrey Mashenkov <
> > andrey.mashen...@gmail.com>:
> > >
> > > > Maxim,
> > > >
> > > > PR is incomplete. Some plugins should be disabled with different
> > id\name.
> > > > Maven plugin shouldn't be disabled as Idea Inspector use it to use
> > Ignite
> > > > project pom file.
> > > >
> > > > Please, find details in ticket.
> > > >
> > > >
> > > > On Fri, Dec 14, 2018 at 12:00 PM Andrey Mashenkov <
> > > > andrey.mashen...@gmail.com> wrote:
> > > >
> > > > > Maxim,
> > > > >
> > > > > Thanks, I'll check PR and let you know about results.
> > > > >
> > > > > For now, Inspections task execution time looks much better (15-22
> > min),
> > > > > but fluctuation is still noticeable.
> > > > >
> > > > > On Fri, Dec 14, 2018 at 11:13 AM Maxim Muzafarov <
> maxmu...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > >> Andrey,
> > > > >>
> > > > >> Thanks! I've consulted with the IntelliJ IDEA source code and
> found
> > > > >> how this disabled plugins file should look like. I've created a
> new
> > > > >> issue [1] and prepared PR [2] with the set of disabled plugins
> > (maybe
> > > > >> not complete set). I don't have access to change corresponding
> > > > >> `~Excluded [Inspections] Core Debug` test suite properties.
> > > > >> Can we test this PR?
> > > > >>
> > > > >> [1] https://issues.apache.org/jira/browse/IGNITE-10682
> > > > >> [2] https://github.com/apache/ignite/pull/5666
> > > > >> On Thu, 13 Dec 2018 at 17:35, Andrey Mashenkov
> > > > >>  wrote:
> > > > >> >
> > > > >> > Maxim,
> > > > >> >
> > > > >> > Idea has a file in config directory
> ./config/disabled_plugins.txt
> > ,
> > > > you
> > > > >> can easily find it at you local machine.
> > > > >> > Teamcity Inspections runner has an option "Disabled plugins"
> where
> > > > >> disabled_plugins.txt file content can be set.
> > > > >> >
> > > > >> > So, looks like we can disable useless plug

Re: Write access to Apache Ignite Confluence

2018-12-18 Thread Dmitriy Pavlov
Hi Andrey,

I've updated permissions, please check.

Sincerely,
Dmitriy Pavlov

вт, 18 дек. 2018 г. в 12:34, Andrey Kuznetsov :

> Hi, Igniters.
>
> I'd like to add new wiki page with suggestions for future Ignite 3.0
> release. We can collect any ideas there and then discuss and refine them on
> dev-list.
>
> Could someone grant me (andrey-kuznetsov) write permission?
>


Re: How to begin to contribute

2018-12-17 Thread Dmitriy Pavlov
Hi Michael,

I'm trying to reach you in https://issues.apache.org/jira/browse/IGNITE-7153
 mentions.

To proceed with patch according to
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-SubmittingforReview
we
need to set the ticket to Patch Available state. If you feel your patch is
ready, please set ticket status.

Sincerely,
Dmitriy Pavlov

ср, 31 окт. 2018 г. в 06:04, Michael Fong :

> Absolutely, I am glad to see this problem solved regardless whom the
> assignee is. After all, we are suffering this Redis problem in our Ignite
> PoC, and we are in urgency to solve this problem.
> I would be glad to provide my solution for you guys to apply if you think
> it is a good (enough) solution. :-)
>
> Regards,
>
> Michael
>
> On Tue, Oct 30, 2018 at 7:53 PM Dmitriy Pavlov 
> wrote:
>
> > Hi Michael,
> >
> > I can see that https://issues.apache.org/jira/browse/IGNITE-7153 is
> > already
> > assigned to Roman Shtykh and is In Progress status.
> >
> > I hope Roman can step in and comment if he is interested in continuing of
> > developing this issue/or he could apply you PR as well.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пн, 22 окт. 2018 г. в 11:33, Michael Fong :
> >
> > > Ah, never mind. I found out that I chose the wrong test suite to begin
> > with
> > > on Team City. :-)
> > >
> > > On Sun, Oct 21, 2018 at 10:20 PM Michael Fong 
> > > wrote:
> > >
> > > > Hi, all,
> > > >
> > > > I happened to have a fix for IGNITE-7153 and created pull request for
> > > it. (
> > > > https://github.com/apache/ignite/pull/5044)
> > > >
> > > > I tried to follow the How to Contribute Workflow
> > > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute#HowtoContribute-Workflow
> > >
> > > but
> > > > kind of got stuck on TeamCity CI, where all the test suite build
> failed
> > > at
> > > > unrelated library (ignite-tool). Is there any other easier or more
> > > > intuitive CI or documentation to guide through the testing process?
> > > >
> > > > Furthermore, I wonder what would the code review process be, still
> via
> > > > github comment review?
> > > >
> > > > Regards,
> > > >
> > > > Michael
> > > >
> > >
> >
>


Re: Apache Ignite TeamCity Bot - muted tests

2018-12-17 Thread Dmitriy Pavlov
Investigation parameters: Can leave as empty
Unmute: Manual (don't use auto because of flaky tests)
Text: https://issues.apache.org/jira/browse/IGNITE-N

But anyway, I hope in the nearest future best way to mute test will be to
ignore it using:
@IgniteIgnore
or @Ignore
or Assume.that()

In this case, we will not need TeamCity mutes anymore.

I'm not sure which way of ignoring will be best.

пн, 17 дек. 2018 г. в 17:02, Anton Vinogradov :

> Dmitrii,
>
> My question was mostly about how to mute test properly.
>
>
> On Mon, Dec 17, 2018 at 4:27 PM Dmitrii Ryabov 
> wrote:
>
> > Anton,
> >
> > In the "Mute" column you can find link to the test on TeamCity. On
> > this page you can see test details like run history and current mutes.
> > Here you should be able to unmute existing mutes.
> > пн, 17 дек. 2018 г. в 16:21, Anton Vinogradov :
> > >
> > > Great feature!
> > >
> > > Could you please explain how to have a deal with it?
> > > How should I mite/unmute tests for now?
> > >
> > > On Fri, Dec 14, 2018 at 6:50 PM Dmitriy Pavlov 
> > wrote:
> > >
> > > > We have IgniteIgnore, so we can use it. It seems behavior is the
> same.
> > We
> > > > need just to understand in which case we can get issue ID from TC
> REST
> > (for
> > > > the Bot).
> > > >
> > > > пт, 14 дек. 2018 г. в 17:51, Vyacheslav Daradur  >:
> > > >
> > > > > Dmitry Pavlov,
> > > > >
> > > > > As far as I know, we are able to use following construction to
> ignore
> > > > test:
> > > > > Assume.assumeTrue("link-to-issue", false);
> > > > >
> > > > > On Fri, Dec 14, 2018 at 5:44 PM Dmitriy Pavlov  >
> > > > wrote:
> > > > > >
> > > > > > Thank you, Dmitrii,
> > > > > >
> > > > > >  I've started unmuting tests starting from yesterday.
> > > > > >
> > > > > > Now muted test count was decreased from 3540 to 3408
> > > > > >
> > > > >
> > > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=mutedProblems_IgniteTests24Java8=%3Cdefault%3E
> > > > > >
> > > > > >
> > > > > > Probably we could automate this activity by auto-unmute, but
> > hopefully,
> > > > > > JUnit4 & @Ignore opportunity will come earlier.
> > > > > >
> > > > > > Sincerely,
> > > > > > Dmitriy Pavlov
> > > > > >
> > > > > > пт, 14 дек. 2018 г. в 17:24, Dmitrii Ryabov <
> somefire...@gmail.com
> > >:
> > > > > >
> > > > > > > Hello, Igniters!
> > > > > > >
> > > > > > > Our Bot got new functionality - it shows tests muted on
> TeamCity
> > and
> > > > > their
> > > > > > > ticket status [1].
> > > > > > >
> > > > > > > So, we can see forgotten mutes for resolved tickets now.
> > > > > > >
> > > > > > > Igniters with rights to unmute tests on TC - please, check this
> > page
> > > > > > > periodically for forgotten mutes.
> > > > > > >
> > > > > > > [1] https://mtcga.gridgain.com/mutes.html
> > > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best Regards, Vyacheslav D.
> > > > >
> > > >
> >
>


Re: Place Ignite Abbrev Plugin to ASF Ignite supplementary git repo

2018-12-17 Thread Dmitriy Pavlov
Hi Igniters,

I've created an empty repository :
New repository ignite-abbrev-plugin.git was created, as requested by
dpavlov.
You may view it at: https://gitbox.apache.org/repos/asf/ignite-abbrev
-plugin.git

Alex G., could you please prepare a PR with your initial changes from
https://github.com/dspavlov/ignite-abbrev-plugin/commit/4b30c1a2cff4ee4a615bc0940327035705198cbc

Later we can start to create changes if other community members agree to
place their work in the ASF repo.

Sincerely,
Dmitriy Pavlov


чт, 19 июл. 2018 г. в 14:04, Dmitry Pavlov :

> Hi Vyacheslav,
>
> For Abbrev plugin I also thought that ASF git repo as primary and
> https://github.com/apacheignite mirror would cover all cases.
>
> Thank you for sharing your vision.
>
> Sincerely,
> Dmitriy Pavlov
>
> чт, 19 июл. 2018 г. в 9:08, Vyacheslav Daradur :
>
>> I vote for a separate repo for the Ignite Abbrev Plugin project.
>>
>> The reason is:
>> Ignite Abbrev Plugin - build on top of IntelliJ Platform SDK [1] and
>> can't be easily packaged without it, moreover, it doesn't depend on
>> Ignite internals (unlike .NET/C++ clients).
>>
>> One more place where we could place the project (if this repo
>> maintained by Ignite's commiters) [2].
>>
>> [1] http://www.jetbrains.org/intellij/sdk/docs/welcome.html
>> [2] https://github.com/apacheignite
>> On Thu, Jul 19, 2018 at 12:48 AM Dmitry Pavlov 
>> wrote:
>> >
>> > Hi Denis,
>> >
>> > This option can be considered also. I have no arguments against this
>> > solution.
>> >
>> > In the same time I think Ignite developer will need pre-build
>> distributive
>> > (Jar) of plugin, probably in /idea subfolder.
>> >
>> > So standalone ASF repo & [build/link to build] in main repo can be still
>> > considered. This option can save checkout time, and reduce main repo
>> size.
>> >
>> > I hope this make sense. I would appreciate Igniter's opinion on this
>> topic.
>> >
>> > Sincerely,
>> > Dmitriy Pavlov
>> >
>> > ср, 18 июл. 2018 г. в 23:05, Denis Magda :
>> >
>> > > Hi Dmitriy,
>> > >
>> > > Yes, I would have this tool at hands soon as I check out Ignite repo
>> and
>> > > start setting it up. However, instead of a new repo let's place it
>> under
>> > > {ignite}/dev-tools folder. What do you think?
>> > >
>> > > --
>> > > Denis
>> > >
>> > > On Wed, Jul 18, 2018 at 4:37 AM Dmitry Pavlov 
>> > > wrote:
>> > >
>> > > > Hi Igniters,
>> > > >
>> > > > There is one mode widely used tool in Apache Ignite, abbreviation
>> plugin
>> > > > for Intelli J Idea. This plugin is used by almost all experienced
>> Ignite
>> > > > contributors.
>> > > >
>> > > > I would like to say thanks to all contributors which created this
>> plugin:
>> > > > vkazakov, sevdokimov, daradurvs, agoncharuk. And because this
>> plugin is
>> > > > also a part of our process I also want to place plugin code to ASF
>> > > > repository.
>> > > >
>> > > > What do you think about placing plugin code to supplementary Apache
>> > > > repository? Please share your vision till 24 July.
>> > > >
>> > > > Sincerely,
>> > > > Dmitriy Pavlov
>> > > >
>> > > >
>> > > >
>> > >
>> https://cwiki.apache.org/confluence/display/IGNITE/Abbreviation+Rules#AbbreviationRules-IntelliJIdeaPlugin
>> > > >
>> > > >
>> > > > https://github.com/dspavlov/ignite-abbrev-plugin
>> > > >
>> > >
>>
>>
>>
>> --
>> Best Regards, Vyacheslav D.
>>
>


Re: Abbreviation code-style requirement.

2018-12-17 Thread Dmitriy Pavlov
Hi Ivan,

I've merged your PR and added permissions to the wiki, please check.

I still feel we need to migrate code to ASF repo. I will try to do
something for that.

Sincerely,
Dmitriy Pavlov

пн, 17 дек. 2018 г. в 13:44, Павлухин Иван :

> Hi,
>
> I did some investigation regarding scala support and it seems that
> version used today (3.0.0) was build without scala support. If nobody
> minds I suggest to build a new version without scala as well.
>
> Also there is a thing that bothers me a little. IDEA throws exception
> in log when a name is abbreviated by plugin (see at the very bottom of
> this message). I build old version on my machine as well and received
> the same. It seems that it was there before my changes. Or it might be
> somehow related to version of IDEA which I used to build the plugin.
>
> 2018-12-17 13:35:10,849 [  21351]  ERROR -
> oring.BaseRefactoringProcessor - Refactorings should not be started
> inside write action
>  because they start progress inside and any read action from the
> progress task would cause the deadlock
> java.lang.Exception
> at
> com.intellij.refactoring.BaseRefactoringProcessor.run(BaseRefactoringProcessor.java:560)
> at
> com.intellij.refactoring.RefactoringImpl.run(RefactoringImpl.java:73)
> at
> org.apache.ignite.idea.inspection.abbrev.IgniteAbbreviationInspection$RenameToFix.rename(IgniteAbbreviationInspection.java:163)
> at
> org.apache.ignite.idea.inspection.abbrev.IgniteAbbreviationInspection$RenameToFix.applyFix(IgniteAbbreviationInspection.java:148)
> at
> org.apache.ignite.idea.inspection.abbrev.IgniteAbbreviationInspection$RenameToFix.applyFix(IgniteAbbreviationInspection.java:121)
> at
> com.intellij.codeInspection.ex.QuickFixWrapper.invoke(QuickFixWrapper.java:75)
> at
> com.intellij.codeInsight.intention.impl.IntentionActionWithTextCaching$MyIntentionAction.invoke(IntentionActionWithTextCaching.java:173)
> at
> com.intellij.codeInsight.intention.impl.ShowIntentionActionsHandler.lambda$invokeIntention$3(ShowIntentionActionsHandler.java:202)
> at
> com.intellij.openapi.application.WriteAction.run(WriteAction.java:105)
> at
> com.intellij.codeInsight.intention.impl.ShowIntentionActionsHandler.invokeIntention(ShowIntentionActionsHandler.java:204)
> at
> com.intellij.codeInsight.intention.impl.ShowIntentionActionsHandler.lambda$null$1(ShowIntentionActionsHandler.java:179)
> at
> com.intellij.openapi.application.TransactionGuardImpl.runSyncTransaction(TransactionGuardImpl.java:88)
> at
> com.intellij.openapi.application.TransactionGuardImpl.submitTransactionAndWait(TransactionGuardImpl.java:153)
> at
> com.intellij.codeInsight.intention.impl.ShowIntentionActionsHandler.lambda$chooseActionAndInvoke$2(ShowIntentionActionsHandler.java:178)
> at
> com.intellij.openapi.command.impl.CoreCommandProcessor.executeCommand(CoreCommandProcessor.java:139)
> at
> com.intellij.openapi.command.impl.CoreCommandProcessor.executeCommand(CoreCommandProcessor.java:97)
> at
> com.intellij.openapi.command.impl.CoreCommandProcessor.executeCommand(CoreCommandProcessor.java:87)
> at
> com.intellij.openapi.command.impl.CoreCommandProcessor.executeCommand(CoreCommandProcessor.java:73)
> at
> com.intellij.codeInsight.intention.impl.ShowIntentionActionsHandler.chooseActionAndInvoke(ShowIntentionActionsHandler.java:177)
> at
> com.intellij.codeInsight.intention.impl.IntentionListStep.lambda$applyAction$0(IntentionListStep.java:118)
> at
> com.intellij.openapi.application.TransactionGuardImpl.performUserActivity(TransactionGuardImpl.java:195)
> at
> com.intellij.ui.popup.AbstractPopup.lambda$dispose$8(AbstractPopup.java:1417)
> at com.intellij.util.ui.UIUtil.invokeLaterIfNeeded(UIUtil.java:3097)
> at
> com.intellij.ide.IdeEventQueue.ifFocusEventsInTheQueue(IdeEventQueue.java:183)
> at
> com.intellij.ide.IdeEventQueue.executeWhenAllFocusEventsLeftTheQueue(IdeEventQueue.java:132)
> at
> com.intellij.openapi.wm.impl.FocusManagerImpl.doWhenFocusSettlesDown(FocusManagerImpl.java:190)
> at
> com.intellij.openapi.wm.impl.IdeFocusManagerImpl.doWhenFocusSettlesDown(IdeFocusManagerImpl.java:58)
> at com.intellij.ui.popup.AbstractPopup.dispose(AbstractPopup.java:1411)
> at com.intellij.ui.popup.WizardPopup.dispose(WizardPopup.java:160)
> at
> com.intellij.ui.popup.list.ListPopupImpl.dispose(ListPopupImpl.java:307)
> at com.intellij.openapi.util.Disposer$1.execute(Disposer.java:48)
> at com.intellij.openapi.util.Disposer$1.execute(Disposer.java:44)
> at
> com.intellij.openapi.util.objectTree.ObjectNode$1.execute(ObjectNode.java:138)
> at
> com.intellij.openapi.util.objectTr

Re: Broken layout of Ignite javadoc on web

2018-12-17 Thread Dmitriy Pavlov
I see the same, browsers checked: Edge & Chrome. I guess it is not a layout
is broken, but a style missing.

Unfortunately, I don't know how to fix. Probably we should start research
from ignite/pom.xml:191

${basedir}/assembly/docfiles/javadoc.css


пн, 17 дек. 2018 г. в 14:32, Павлухин Иван :

> Hi,
>
> I noticed that Ignite javadoc layout on web for latest versions has
> some problems. For example, I see no links near the "Class" at heading
> of the page [1]. Here is a screenshot [2]. Do you see the same?
>
> I was able to build javadoc on my machine. With current configuration
> I see the same broken layout. After using commenting out customized
> style sheet configuration (referring to assembly/docfiles/javadoc.css)
> I was able to obtain an html which renders fine for me.
>
> Can anyone suggest a simple solution for this?
>
> [1]
> https://ignite.apache.org/releases/2.7.0/javadoc/org/apache/ignite/cache/eviction/EvictionPolicy.html
> [2]
> https://gist.githubusercontent.com/pavlukhin/c8c7c6266eeab56048c31f5cdfb31d20/raw/1b257ad73f81cb4698f6105a9d1646295ba55795/javadoc_layout.png
>
> --
> Best regards,
> Ivan Pavlukhin
>


Re: Code inspection

2018-12-15 Thread Dmitriy Pavlov
Folks,

There is a strange error on TC
https://ci.ignite.apache.org/viewLog.html?buildId=2556875=IgniteTests24Java8_InspectionsCore

It appeared after TC update to the latest version.

Sincerely,
Dmitry Pavlov

пт, 14 дек. 2018 г. в 16:09, Andrey Mashenkov :

> Maxim,
>
> PR is incomplete. Some plugins should be disabled with different id\name.
> Maven plugin shouldn't be disabled as Idea Inspector use it to use Ignite
> project pom file.
>
> Please, find details in ticket.
>
>
> On Fri, Dec 14, 2018 at 12:00 PM Andrey Mashenkov <
> andrey.mashen...@gmail.com> wrote:
>
> > Maxim,
> >
> > Thanks, I'll check PR and let you know about results.
> >
> > For now, Inspections task execution time looks much better (15-22 min),
> > but fluctuation is still noticeable.
> >
> > On Fri, Dec 14, 2018 at 11:13 AM Maxim Muzafarov 
> > wrote:
> >
> >> Andrey,
> >>
> >> Thanks! I've consulted with the IntelliJ IDEA source code and found
> >> how this disabled plugins file should look like. I've created a new
> >> issue [1] and prepared PR [2] with the set of disabled plugins (maybe
> >> not complete set). I don't have access to change corresponding
> >> `~Excluded [Inspections] Core Debug` test suite properties.
> >> Can we test this PR?
> >>
> >> [1] https://issues.apache.org/jira/browse/IGNITE-10682
> >> [2] https://github.com/apache/ignite/pull/5666
> >> On Thu, 13 Dec 2018 at 17:35, Andrey Mashenkov
> >>  wrote:
> >> >
> >> > Maxim,
> >> >
> >> > Idea has a file in config directory ./config/disabled_plugins.txt ,
> you
> >> can easily find it at you local machine.
> >> > Teamcity Inspections runner has an option "Disabled plugins" where
> >> disabled_plugins.txt file content can be set.
> >> >
> >> > So, looks like we can disable useless plugins.
> >> > But I'm not expert and can't suggest changes we can safely apply.
> >> >
> >> > On Thu, Dec 13, 2018 at 4:59 PM Maxim Muzafarov 
> >> wrote:
> >> >>
> >> >> Andrey,
> >> >>
> >> >> Thank you for solving this issue with GC pauses! I've checked the
> >> >> given report. The inspections configuration is correct, but it seems
> >> >> to me that we have enabled by default rules of included plugins (for
> >> >> instance, KotlinInternalInJava in the report is enabled).
> >> >>
> >> >> Can you share more details about `disable plugin` option you found?
> >> >>
> >> >> I see that idea instance starts with the default -Didea.plugins.path
> >> >> system property, can we change it so the plugins will be not loaded
> by
> >> >> default?
> >> >> On Thu, 13 Dec 2018 at 15:45, Andrey Mashenkov
> >> >>  wrote:
> >> >> >
> >> >> > Maxim,
> >> >> >
> >> >> > It looks like we can't make logs more verbose due to possible bug,
> >> I've create a ticket in Jetbrains Jira [1].
> >> >> > We can just publish idea logs in artefacts as suggested in this
> >> manual [2].
> >> >> >
> >> >> > For now, Inspections logs looks like this one [3].
> >> >> > Also, would you please to take a look at inspection report and
> check
> >> if we missed smth and there are any unwanted inspection turned on.
> >> >> >
> >> >> > [1] https://youtrack.jetbrains.com/issue/TW-58422
> >> >> > [2]
> >>
> https://confluence.jetbrains.com/display/TCD10/Reporting+Issues#ReportingIssues-IntelliJIDEAInspections
> >> >> > [3]
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=2538111=IgniteTests24Java8_ExcludedInspections2=artifacts
> >> >> >
> >> >> > On Thu, Dec 13, 2018 at 3:19 PM Dmitriy Pavlov  >
> >> wrote:
> >> >> >>
> >> >> >> Maxim M, do you know if we can disable inspections by wildcard?
> E.g.
> >> >> >> Android* ?
> >> >> >>
> >> >> >> чт, 13 дек. 2018 г. в 14:59, Andrey Mashenkov <
> >> andrey.mashen...@gmail.com>:
> >> >> >>
> >> >> >> > Fixed memory issues with increasing heap size and forcing G1GC.
> >> >> >> >
> >> >> >> > Do we need all these plugins loaded for inspections?
> >

Re: CONTRIBUTING.md first version

2018-12-14 Thread Dmitriy Pavlov
Sure,  I will do

пт, 14 дек. 2018 г., 20:37 Denis Magda :

> >
> > `Contributing.md` is more not about how to contribute in day by day
> basis,
> > but a short step-by-step guide on how to join and start contributing.
>
>
> Could we rework and refine the wiki version in a similar way - most
> important stuff first, old is either removed and reworked.
>
> --
> Denis
>
> On Fri, Dec 14, 2018 at 2:43 AM Dmitriy Pavlov  wrote:
>
> > Thank you for sharing your vision, I also agree to have short in GH and
> > full in the wiki.
> >
> > Denis, I see a number of outdated info in How To Contribute, but most
> > sections there are for a strong reason. A lot of cases covered, and a lot
> > of cases could be covered in addition to existing.
> >
> > `Contributing.md` is more not about how to contribute in day by day
> basis,
> > but a short step-by-step guide on how to join and start contributing.
> >
> > Moreover, I would like to refactor website section
> > https://ignite.apache.org/community/contribute.html#contribute in the
> same
> > manner.
> >
> > пт, 14 дек. 2018 г. в 11:29, Petr Ivanov :
> >
> > > Agree. GitHub can have shorter quick notes with link to full
> > documentation.
> > >
> > >
> > > > On 14 Dec 2018, at 08:05, Alexey Kuznetsov 
> > > wrote:
> > > >
> > > > Hi,
> > > >
> > > > From my experience it is better when all needed stuff available on
> > > GitHub.
> > > >
> > > > It is very comfortable when thinks like, "read me", "change log",
> "how
> > to
> > > > build", "how to contribute" and so on available on GitHub.
> > > >
> > > > No need to open some external links.
> > > >
> > > > --
> > > > Alexey Kuznetsov
> > >
> > >
> >
>


Re: Async debugging in IDEA

2018-12-14 Thread Dmitriy Pavlov
Looks cool, I will try

пт, 14 дек. 2018 г. в 16:46, Павлухин Иван :

> Hi Igniters,
>
> I would like to share with you that now it is possible to use IDEA
> async debugger for debugging Ignite futures. And we should thank Ivan
> Bessonov for that awesome contribution [1].
>
> Most of your might be already familiar with async debugger in IDEA.
> For the rest I prepared two screenshots [2], [3]. We can see regular
> stacktrace on breakpoint hit in future completion callback which is
> extended with an additional stacktrace showing a path where the
> callback was installed.
>
> Thank you Ivan!
>
> [1] https://issues.apache.org/jira/browse/IGNITE-10475
> [2]
> https://gist.githubusercontent.com/pavlukhin/c8c7c6266eeab56048c31f5cdfb31d20/raw/bfe32b8c11a67ccc3bcbbb8b398e39a8a818cad6/adebug1.png
> [3]
> https://gist.githubusercontent.com/pavlukhin/c8c7c6266eeab56048c31f5cdfb31d20/raw/bfe32b8c11a67ccc3bcbbb8b398e39a8a818cad6/adebug2.png
>
> --
> Best regards,
> Ivan Pavlukhin
>


Re: Apache Ignite TeamCity Bot - muted tests

2018-12-14 Thread Dmitriy Pavlov
We have IgniteIgnore, so we can use it. It seems behavior is the same. We
need just to understand in which case we can get issue ID from TC REST (for
the Bot).

пт, 14 дек. 2018 г. в 17:51, Vyacheslav Daradur :

> Dmitry Pavlov,
>
> As far as I know, we are able to use following construction to ignore test:
> Assume.assumeTrue("link-to-issue", false);
>
> On Fri, Dec 14, 2018 at 5:44 PM Dmitriy Pavlov  wrote:
> >
> > Thank you, Dmitrii,
> >
> >  I've started unmuting tests starting from yesterday.
> >
> > Now muted test count was decreased from 3540 to 3408
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=mutedProblems_IgniteTests24Java8=%3Cdefault%3E
> >
> >
> > Probably we could automate this activity by auto-unmute, but hopefully,
> > JUnit4 & @Ignore opportunity will come earlier.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пт, 14 дек. 2018 г. в 17:24, Dmitrii Ryabov :
> >
> > > Hello, Igniters!
> > >
> > > Our Bot got new functionality - it shows tests muted on TeamCity and
> their
> > > ticket status [1].
> > >
> > > So, we can see forgotten mutes for resolved tickets now.
> > >
> > > Igniters with rights to unmute tests on TC - please, check this page
> > > periodically for forgotten mutes.
> > >
> > > [1] https://mtcga.gridgain.com/mutes.html
> > >
>
>
>
> --
> Best Regards, Vyacheslav D.
>


Re: Apache Ignite TeamCity Bot - muted tests

2018-12-14 Thread Dmitriy Pavlov
Thank you, Dmitrii,

 I've started unmuting tests starting from yesterday.

Now muted test count was decreased from 3540 to 3408
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=mutedProblems_IgniteTests24Java8=%3Cdefault%3E


Probably we could automate this activity by auto-unmute, but hopefully,
JUnit4 & @Ignore opportunity will come earlier.

Sincerely,
Dmitriy Pavlov

пт, 14 дек. 2018 г. в 17:24, Dmitrii Ryabov :

> Hello, Igniters!
>
> Our Bot got new functionality - it shows tests muted on TeamCity and their
> ticket status [1].
>
> So, we can see forgotten mutes for resolved tickets now.
>
> Igniters with rights to unmute tests on TC - please, check this page
> periodically for forgotten mutes.
>
> [1] https://mtcga.gridgain.com/mutes.html
>


Re: CONTRIBUTING.md first version

2018-12-14 Thread Dmitriy Pavlov
Thank you for sharing your vision, I also agree to have short in GH and
full in the wiki.

Denis, I see a number of outdated info in How To Contribute, but most
sections there are for a strong reason. A lot of cases covered, and a lot
of cases could be covered in addition to existing.

`Contributing.md` is more not about how to contribute in day by day basis,
but a short step-by-step guide on how to join and start contributing.

Moreover, I would like to refactor website section
https://ignite.apache.org/community/contribute.html#contribute in the same
manner.

пт, 14 дек. 2018 г. в 11:29, Petr Ivanov :

> Agree. GitHub can have shorter quick notes with link to full documentation.
>
>
> > On 14 Dec 2018, at 08:05, Alexey Kuznetsov 
> wrote:
> >
> > Hi,
> >
> > From my experience it is better when all needed stuff available on
> GitHub.
> >
> > It is very comfortable when thinks like, "read me", "change log", "how to
> > build", "how to contribute" and so on available on GitHub.
> >
> > No need to open some external links.
> >
> > --
> > Alexey Kuznetsov
>
>


[jira] [Created] (IGNITE-10677) [TC Bot] Include build failure on metric & exit code into critical failures

2018-12-13 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-10677:
---

 Summary: [TC Bot] Include build failure on metric & exit code into 
critical failures
 Key: IGNITE-10677
 URL: https://issues.apache.org/jira/browse/IGNITE-10677
 Project: Ignite
  Issue Type: Task
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov


It is necessary to setup dev.list notifications for _License_ failure 
and for Inspections: Core failures notification



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: [ML] Metric calculation for classification models

2018-12-13 Thread Dmitriy Pavlov
Folks, I sometimes hear complains related to metrics and its clearness for
end-users.

Would you add a couple of words related to each value to wiki/readme.io?

чт, 13 дек. 2018 г. в 17:13, Alexey Zinoviev :

> So, I agree that we should avoid ineffective metrics calculations.
> I think that in 2.8 release we should have
>
>1. BinaryClassificationMetric with all metrics from Wikipedia
>2. Metric interface with 1 or two implementations in example folder or
>in metric package like roc auc and accuracy
>3. BinaryClassificationMetric and MultiClassClassificationMetrics should
>implement new interface MetricGroup
>
> Will totally change the current PR according your recommendation
>
> чт, 13 дек. 2018 г. в 16:06, Алексей Платонов :
>
> > You can compute just TP (true-positive), FP, TN and FN counters and use
> > them to evaluate Recall, Precision, Accuracy, ect. If you want to specify
> > class for Pr evaluation, then you can compute Pr for first label as
> > TP/(TP+FP) and for second label as TN/(TN+FN) for example. After it we
> can
> > unite all one-point metrics evaluation.
> >
> > In my opinion we can redesign metrics calculation and provide one-point
> > metrics (like Pr, Re) and integral metrics like ROC AUC where one-point
> > metrics can be calculated through TP,FP etc.
> >
> > Maybe you should design class BinaryClassificationMetric that computes
> > these counters and provide methods like recall :: () -> double, precision
> > :: () -> double, etc.
> >
> > чт, 13 дек. 2018 г. в 13:26, Yuriy Babak :
> >
> > > Igniters, Alexey
> > >
> > > I want to discuss the ticket 10371 [1], currently, we calculate 4
> numbers
> > > (true positive, true negative, false positive, false negative) for each
> > > "point metric" like accuracy, recall, f-score and precision for each
> > label.
> > >
> > > So for the full score we need calculates those 4 numbers 8 times. But
> we
> > > could calculate all 8 metrics(4 for the first label and 4 for the
> second
> > > label).
> > >
> > > I suggest introducing new API "point metric" for metrics like those
> > > 4(accuracy, recall, f-score, and precision) and "integral metric" for
> > > metrics like ROC AUC [2].
> > >
> > > Any thoughts would be appreciated.
> > >
> > > [1] - https://issues.apache.org/jira/browse/IGNITE-10371
> > > [2] - https://issues.apache.org/jira/browse/IGNITE-10145
> > >
> >
>


Re: [MTCGA]: new failures in builds [2364053] needs to be handled

2018-12-13 Thread Dmitriy Pavlov
The failure occurred a long time ago, it is outside of TC history now. The
test is still flaky.

чт, 13 дек. 2018 г. в 07:56, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *New stable failure of a flaky test in master
> CacheAbstractTest.TestCacheConfigurationExpiryPolicy
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-7892614422846595039=%3Cdefault%3E=testDetails
>  Changes may lead to failure were done by
>  - aplatonovv
> https://ci.ignite.apache.org/viewModification.html?modId=839641
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 07:55:53 13-12-2018
>


Re: [MTCGA]: new failures in builds [2372233] needs to be handled

2018-12-13 Thread Dmitriy Pavlov
Mentioned failure is not recent, but occurred in the past. Hopefully, new
services implementation will fix the test.

чт, 13 дек. 2018 г. в 04:11, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *New stable failure of a flaky test in master
> GridServiceProcessorBatchDeploySelfTest.testDeployAllTopologyChangeFail
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=5756858704963167219=%3Cdefault%3E=testDetails
>  No changes in the build
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 04:10:57 13-12-2018
>


Re: [MTCGA]: new failures in builds [2527852] needs to be handled

2018-12-13 Thread Dmitriy Pavlov
https://issues.apache.org/jira/browse/IGNITE-10245

ср, 12 дек. 2018 г. в 17:11, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *Recently contributed test failed in master
> ClientSslParametersTest.testNonExistentCipherSuite
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-2251849292403456778=%3Cdefault%3E=testDetails
>
>  *Recently contributed test failed in master
> ClientSslParametersTest.testNonExistentProtocol
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=7837765355690376078=%3Cdefault%3E=testDetails
>  Changes may lead to failure were done by
>  - akuznetsov
> https://ci.ignite.apache.org/viewModification.html?modId=844317
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 17:10:54 12-12-2018
>


Re: Code inspection

2018-12-13 Thread Dmitriy Pavlov
Maxim M, do you know if we can disable inspections by wildcard? E.g.
Android* ?

чт, 13 дек. 2018 г. в 14:59, Andrey Mashenkov :

> Fixed memory issues with increasing heap size and forcing G1GC.
>
> Do we need all these plugins loaded for inspections?
> I've found a 'disable plugin' option in TC Inspections build configuration,
> but it is unclear how to disable plugin correctly.
> Can someone take over this?
>
> > 46 plugins initialized in 1031 ms
> > 2018-12-13 10:55:24,875 [ 1342] INFO - llij.ide.plugins.PluginManager -
> > Loaded bundled plugins: Android Support (10.2.3), Ant Support (1.0), CSS
> > Support (172.4574.11), Database Tools and SQL (172.4574.11), Eclipse
> > Integration (3.0), FreeMarker support (1.0), GWT Support (1.0), Gradle
> > (172.4574.11), Groovy (9.0), Guice (8.0), HTML Tools (2.0), Hibernate
> > Support (1.0), I18n for Java (172.4574.11), IDEA CORE (172.4574.11),
> > IntelliLang (8.0), JBoss Seam Support (1.0), JUnit (1.0), Java EE: Bean
> > Validation Support (1.1), Java EE: Contexts and Dependency Injection
> (1.1),
> > Java EE: EJB, JPA, Servlets (1.0), Java EE: Java Server Faces (2.2.X.),
> > Java EE: Web Services (JAX-WS) (1.9), Java Server Pages (JSP) Integration
> > (1.0), JavaScript Support (1.0), Kotlin (1.1.4-release-IJ2017.2-3), Maven
> > Integration (172.4574.11), Persistence Frameworks Support (1.0), Plugin
> > DevKit (1.0), Properties Support (172.4574.11), QuirksMode (172.4574.11),
> > Spring AOP/@AspectJ (1.0), Spring Batch (1.0), Spring Data (1.0), Spring
> > Integration Patterns (1.0), Spring Security (1.0), Spring Support (1.0),
> > Spring Web Flow (1.0), Spring Web Services (1.0), Struts 1.x (2.0),
> Struts
> > 2 (1.0), TestNG-J (8.0), UI Designer (172.4574.11), Velocity support
> (1.0),
> > W3C Validators (2.0), WebLogic Integration (1.0), XPathView + XSLT
> Support
> > (4)
>
>
> Kotlin plugins fails to start, let's disable it.
>
> >
> > 2018-12-13 10:55:27,623 [   4090]   INFO -
> il.indexing.FileBasedIndexImpl - Rebuild requested for index
> org.jetbrains.kotlin.idea.versions.KotlinJvmMetadataVersionIndex
> > java.lang.Throwable
> >   at
> com.intellij.util.indexing.FileBasedIndex.requestRebuild(FileBasedIndex.java:68)
> >   at
> org.jetbrains.kotlin.idea.versions.KotlinUpdatePluginComponent.initComponent(KotlinUpdatePluginComponent.kt:54)
> >   at
> com.intellij.openapi.components.impl.ComponentManagerImpl$ComponentConfigComponentAdapter.getComponentInstance(ComponentManagerImpl.java:492)
> >   at
> com.intellij.openapi.components.impl.ComponentManagerImpl.createComponents(ComponentManagerImpl.java:118)
> >   at
> com.intellij.openapi.application.impl.ApplicationImpl.a(ApplicationImpl.java:462)
> >   at
> com.intellij.openapi.application.impl.ApplicationImpl.createComponents(ApplicationImpl.java:466)
> >   at
> com.intellij.openapi.components.impl.ComponentManagerImpl.init(ComponentManagerImpl.java:102)
> >   at
> com.intellij.openapi.application.impl.ApplicationImpl.load(ApplicationImpl.java:421)
> >   at
> com.intellij.openapi.application.impl.ApplicationImpl.load(ApplicationImpl.java:407)
> >   at com.intellij.idea.IdeaApplication.run(IdeaApplication.java:203)
>
>
>
> On Thu, Dec 13, 2018 at 1:45 PM Dmitriy Pavlov  wrote:
>
> > Sure, let's apply. I hope all TC agents may handle 4G heap.
> >
> > чт, 13 дек. 2018 г. в 12:54, Andrey Mashenkov <
> andrey.mashen...@gmail.com
> > >:
> >
> > > Guys,
> > >
> > > I've just creates a copy of Inspections TC build task with GC logs
> turned
> > > on to check if there is any issues
> > > and found Inspections task spent too much time in STW due to long Full
> GC
> > > pauses.
> > >
> > > I've tried to increase Xmx up to 4Gb and use G1GC got 2+ times better
> > > execution time down to ~15 min (~17 for 2G heap).
> > > Increasing heap size only is not very helpful as it just postpone Full
> GC
> > > issues, but changing GC to G1GC gives noticeable result.
> > >
> > > Let's apply this optimization.
> > > Thoughts?
> > >
> > >
> > > On Sun, Dec 9, 2018 at 12:43 PM Vyacheslav Daradur <
> daradu...@gmail.com>
> > > wrote:
> > >
> > > > Hi, Maxim, Nikolay, I have the following questions regarding
> > inspections:
> > > >
> > > > Should 'gnite_inspections_teamcity.xml' been imported into IDEA,
> since
> > > > 'ignite_inspections.xml' has been removed in actual master?
> > > >
> > > > Also, I'

Re: [MTCGA]: new failures in builds [2421292] needs to be handled

2018-12-13 Thread Dmitriy Pavlov
This failure bot complains was occurred in November. It is not a recent
failure. But the test is still flaky.

ср, 12 дек. 2018 г. в 03:26, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *New stable failure of a flaky test in master
> JdbcThinTransactionsServerAutoCommitComplexSelfTest.testInsertAndQueryMultipleCaches
>
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=695613062704149117=%3Cdefault%3E=testDetails
>  No changes in the build
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 03:25:55 12-12-2018
>


Re: [MTCGA]: new failures in builds [2481986] needs to be handled

2018-12-13 Thread Dmitriy Pavlov
Flaky timeout. Probably already fixed

пн, 10 дек. 2018 г. в 04:47, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *New Critical Failure in master Client Nodes
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_ClientNodes=%3Cdefault%3E=buildTypeStatusDiv
>  No changes in the build
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 04:47:19 10-12-2018
>


Re: [MTCGA]: new failures in builds [2375823] needs to be handled

2018-12-13 Thread Dmitriy Pavlov
TC Bot detected some stable failure in the past. The mentioned build is
outside of TC storage now.  Failure history in the latest Bot release is
longer, so this is not a recent failure.  But anyway, the test needs to be
fixed as it is flaky.

ср, 12 дек. 2018 г. в 00:11, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *New stable failure of a flaky test in master
> IgniteClientReconnectMassiveShutdownTest.testMassiveServersShutdown1
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-6728945354254258306=%3Cdefault%3E=testDetails
>  No changes in the build
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 00:10:55 12-12-2018
>


MVCC: [MTCGA]: new failures in builds [2512587] needs to be handled

2018-12-13 Thread Dmitriy Pavlov
It is really recently broken test:
https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=7945125576049372508=%3Cdefault%3E=testDetails


Now it fails stable.

Igor S, Vladimir O. please chime in.

вт, 11 дек. 2018 г. в 09:17, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *New stable failure of a flaky test in master
> CacheMvccPartitionedSqlCoordinatorFailoverTest.testPutAllGetAll_ClientServer_Backups1_RestartRandomSrv_SqlDml
>
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=7945125576049372508=%3Cdefault%3E=testDetails
>
>  *New stable failure of a flaky test in master
> CacheMvccPartitionedSqlCoordinatorFailoverTest.testPutAllGetAll_ClientServer_Backups2_RestartRandomSrv_SqlDml
>
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=8412476034648229784=%3Cdefault%3E=testDetails
>
>  *New stable failure of a flaky test in master
> CacheMvccPartitionedSqlCoordinatorFailoverTest.testPutAllGetAll_Server_Backups2_RestartRandomSrv_SqlDml
>
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=254244004184327163=%3Cdefault%3E=testDetails
>  Changes may lead to failure were done by
>  - gvvinblade
> https://ci.ignite.apache.org/viewModification.html?modId=843774
>  - ygerzhedovich
> https://ci.ignite.apache.org/viewModification.html?modId=843694
>  - dpavlov
> https://ci.ignite.apache.org/viewModification.html?modId=843864
>  - isapego
> https://ci.ignite.apache.org/viewModification.html?modId=843817
>  - vozerov
> https://ci.ignite.apache.org/viewModification.html?modId=843801
>  - andrey.mashenkov
> https://ci.ignite.apache.org/viewModification.html?modId=843780
>  - gvvinblade
> https://ci.ignite.apache.org/viewModification.html?modId=843765
>  - andrey.mashenkov
> https://ci.ignite.apache.org/viewModification.html?modId=843701
>  - vozerov
> https://ci.ignite.apache.org/viewModification.html?modId=843815
>  - nsamelchev
> https://ci.ignite.apache.org/viewModification.html?modId=843792
>  - kondakov87
> https://ci.ignite.apache.org/viewModification.html?modId=843777
>  - palmihal
> https://ci.ignite.apache.org/viewModification.html?modId=843699
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 09:17:19 11-12-2018
>


Re: [MTCGA]: new failures in builds [2446283, 2368367] needs to be handled

2018-12-13 Thread Dmitriy Pavlov
Hi All, the Bot detected some stable failure in the past because failure
history in the latest Bot release is longer, so this is not a recent
failure.

But anyway both tests need to be fixed as they are flaky.

вт, 11 дек. 2018 г. в 23:34, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *New stable failure of a flaky test in master
> SpringEncryptedCacheRestartTest.testCreateEncryptedCache
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=7227670937558724712=%3Cdefault%3E=testDetails
>  No changes in the build
>
>  *New stable failure of a flaky test in master
> IgniteConfigurationTest.TestSpringXml
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=8068196411489261503=%3Cdefault%3E=testDetails
>  No changes in the build
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 23:25:55 11-12-2018
>


Re: Code inspection

2018-12-13 Thread Dmitriy Pavlov
Sure, let's apply. I hope all TC agents may handle 4G heap.

чт, 13 дек. 2018 г. в 12:54, Andrey Mashenkov :

> Guys,
>
> I've just creates a copy of Inspections TC build task with GC logs turned
> on to check if there is any issues
> and found Inspections task spent too much time in STW due to long Full GC
> pauses.
>
> I've tried to increase Xmx up to 4Gb and use G1GC got 2+ times better
> execution time down to ~15 min (~17 for 2G heap).
> Increasing heap size only is not very helpful as it just postpone Full GC
> issues, but changing GC to G1GC gives noticeable result.
>
> Let's apply this optimization.
> Thoughts?
>
>
> On Sun, Dec 9, 2018 at 12:43 PM Vyacheslav Daradur 
> wrote:
>
> > Hi, Maxim, Nikolay, I have the following questions regarding inspections:
> >
> > Should 'gnite_inspections_teamcity.xml' been imported into IDEA, since
> > 'ignite_inspections.xml' has been removed in actual master?
> >
> > Also, I've faced mismatching: if I use
> > '@SuppressWarnings("ErrorNotRethrown")' in code, then this will be
> > marked on TC as "Redundant suppression". If I removed this suppression
> > in "main" code base (not in tests) then it's fine and IDE does not
> > mark the code by inspection. But, if I use
> > 'GridTestUtils#assertThrows' in 'tests' code base, then IDE requires
> > to suppress the inspection, if I have done it then TC marks this as
> > "Redundant suppression".
> >
> > What should I do in this case?
> >
> > On Mon, Dec 3, 2018 at 10:26 PM Andrey Mashenkov
> >  wrote:
> > >
> > > Hi,
> > >
> > > Have someone tried to investigate the issue related to Inspection TC
> task
> > > execution time variation (from 0.5 up to 1,5 hours)?
> > > Can we enable GC logs for this task or may be even get CPU, Disk,
> Network
> > > metrics?
> > > Can someone check if there are unnecessary Idea plugins starts that can
> > be
> > > safely disabled?
> > >
> > >
> > > On Tue, Nov 27, 2018 at 5:52 PM Dmitriy Pavlov 
> > wrote:
> > >
> > > > I'm totally with you in this decision, let's move the file.
> > > >
> > > > вт, 27 нояб. 2018 г. в 16:24, Maxim Muzafarov :
> > > >
> > > > > Igniters,
> > > > >
> > > > > I propose to make inspection configuration default on the project
> > > > > level. I've created a new issue [1] for it. It can be easily done
> and
> > > > > recommend by IntelliJ documentation [2].
> > > > > Thoughts?
> > > > >
> > > > >
> > > > > Vyacheslav,
> > > > >
> > > > > Can you share an example of your warnings?
> > > > > Currently, we have different inspection configurations:
> > > > > - ignite_inspections.xml - to import inspections as default and use
> > it
> > > > > daily.
> > > > > - ignite_inspections_teamcity.xml - config to run it on TC. Only
> > fixed
> > > > > rules in the project code are enabled. Each of these rules are
> marked
> > > > > with ERROR level.
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-10422
> > > > > [2] https://www.jetbrains.com/help/idea/code-inspection.html
> > > > > On Tue, 20 Nov 2018 at 13:58, Nikolay Izhikov  >
> > > > wrote:
> > > > > >
> > > > > > Hello, Vyacheslav.
> > > > > >
> > > > > > Yes, we have.
> > > > > >
> > > > > > Maxim Muzafarov, can you fix it, please?
> > > > > >
> > > > > > вт, 20 нояб. 2018 г., 13:10 Vyacheslav Daradur
> daradu...@gmail.com
> > :
> > > > > >
> > > > > > > Guys, why we have 2 different inspection files in the repo?
> > > > > > > idea\ignite_inspections.xml
> > > > > > > idea\ignite_inspections_teamcity.xml
> > > > > > >
> > > > > > > AFAIK TeamCity is able to use the same inspection file with
> IDE.
> > > > > > >
> > > > > > > I've imported 'idea\ignite_inspections.xml' in the IDE, but now
> > see
> > > > > > > inspection warnings for my PR on TC because of different rules.
> > > > > > >
> > > > > > >
> > > > > > > On Sun, Nov 11, 2018 at 6:06 PM Maxim Muzafarov <
>

Re: CONTRIBUTING.md first version

2018-12-13 Thread Dmitriy Pavlov
I'll fix. There is an ignored files list.

чт, 13 дек. 2018 г. в 12:02, Petr Ivanov :

> There is an error in Build project [1] section.
> How can it be fixed? Ticket to IGNITE Jira?
>
>
> [1]
> https://github.com/apache/ignite/blob/master/CONTRIBUTING.md#building-project
>
> > On 12 Dec 2018, at 20:21, Dmitriy Pavlov  wrote:
> >
> > Hi Igniters,
> >
> > Just to inform, the very first version of the Contribution step by step
> > guide was added to GitHub. It is one more entry point for newcomers to
> find
> > out what and how should be done to contribute:
> >
> https://github.com/apache/ignite/blob/master/CONTRIBUTING.md#contributing-to-apache-ignite
> >
> >
> > Feel free to share with newcomers and contribute.
> >
> > Sincerely,
> > Dmitriy Pavlov
>
>


Re: CONTRIBUTING.md first version

2018-12-12 Thread Dmitriy Pavlov
There is nothing wrong with wiki (excepting it is sometimes way too slow
and it may be annoying ).

Github markdown is a way to
1. Keep documenting under version control system
2. Github recommends to provide this doc in the standard place.

Of course, we can move this new doc content to wiki and just provide link
from contributing.md to wiki. Should we?

I can also add a copy to a wiki. Denis, folks,  please share your vision.

чт, 13 дек. 2018 г., 0:32 Denis Magda :

> What is wrong with Ignite wiki? It seems like the right place for the doc
> like this.
>
> --
> Denis
>
> On Wed, Dec 12, 2018 at 9:21 AM Dmitriy Pavlov  wrote:
>
> > Hi Igniters,
> >
> > Just to inform, the very first version of the Contribution step by step
> > guide was added to GitHub. It is one more entry point for newcomers to
> find
> > out what and how should be done to contribute:
> >
> >
> https://github.com/apache/ignite/blob/master/CONTRIBUTING.md#contributing-to-apache-ignite
> >
> >
> > Feel free to share with newcomers and contribute.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
>


Re: [DISCUSSION] Relocation of Apache git repositories from git-wip-us.apache.org to GitBox

2018-12-12 Thread Dmitriy Pavlov
Folks, Incubator finished its migration to GitBox -
https://gitbox.apache.org/repos/asf?p=incubator.git

Because it seems majority don't care too much about migration I'm going to
consider this decision approved by silence and could be made by lazy
consensus in 72 hours.


вт, 11 дек. 2018 г. в 19:25, Alexey Goncharuk :

> Given that there is no option to stay on the old repository and
> mass-migration is scheduled for Feb, 7th, I think it is better to prepare
> and move the repository before the date.
>
> As far as I understand, from developers standpoint we only need to change
> the git remote entry. Petr, Sergey, can you asses what changes are need to
> be done in order to support this migration for TC and release procedure?
>
> вт, 11 дек. 2018 г. в 19:20, Dmitriy Pavlov :
>
> > Hi All,
> >
> > The Apache Ignite still has a repository on git-wip-us for its code, as
> > well, for release.
> >
> > Does anyone have a problem with moving over the Ignite git-wip-us
> > repository to gitbox voluntarily? This means integrated access and easy
> PRs
> > (write access to the GitHub repo).
> >
> > We need to document support for the decision from a mailing list post, so
> > here it is.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > сб, 8 дек. 2018 г. в 00:16, Dmitriy Pavlov :
> >
> > > Hi Igniters,
> > >
> > > What do you think about the voluntary migration of project repositories
> > > from git-wip to GitBox? (See details in the forwarded email.)
> > >
> > > Redirection of emails originated by PR actions to notifications@ seems
> > to
> > > be almost finished, so email flood issue seems to be already solved for
> > dev@
> > > .
> > >
> > > It is up to us to decide if we would like to migrate sooner. Later all
> > > repositories will be migrated anyway.
> > >
> > > Affected repositories
> > > - Ignite Release
> > > - Ignite
> > >
> > > Please share your opinion.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > -- Forwarded message -
> > > From: Daniel Gruno 
> > > Date: пт, 7 дек. 2018 г. в 19:52
> > > Subject: [NOTICE] Mandatory relocation of Apache git repositories on
> > > git-wip-us.apache.org
> > > To: us...@infra.apache.org 
> > >
> > >
> > > [IF YOUR PROJECT DOES NOT HAVE GIT REPOSITORIES ON GIT-WIP-US PLEASE
> > >   DISREGARD THIS EMAIL; IT WAS MASS-MAILED TO ALL APACHE PROJECTS]
> > >
> > > Hello Apache projects,
> > >
> > > I am writing to you because you may have git repositories on the
> > > git-wip-us server, which is slated to be decommissioned in the coming
> > > months. All repositories will be moved to the new gitbox service which
> > > includes direct write access on github as well as the standard ASF
> > > commit access via gitbox.apache.org.
> > >
> > > ## Why this move? ##
> > > The move comes as a result of retiring the git-wip service, as the
> > > hardware it runs on is longing for retirement. In lieu of this, we
> > > have decided to consolidate the two services (git-wip and gitbox), to
> > > ease the management of our repository systems and future-proof the
> > > underlying hardware. The move is fully automated, and ideally, nothing
> > > will change in your workflow other than added features and access to
> > > GitHub.
> > >
> > > ## Timeframe for relocation ##
> > > Initially, we are asking that projects voluntarily request to move
> > > their repositories to gitbox, hence this email. The voluntary
> > > timeframe is between now and January 9th 2019, during which projects
> > > are free to either move over to gitbox or stay put on git-wip. After
> > > this phase, we will be requiring the remaining projects to move within
> > > one month, after which we will move the remaining projects over.
> > >
> > > To have your project moved in this initial phase, you will need:
> > >
> > > - Consensus in the project (documented via the mailing list)
> > > - File a JIRA ticket with INFRA to voluntarily move your project repos
> > >over to gitbox (as stated, this is highly automated and will take
> > >between a minute and an hour, depending on the size and number of
> > >your repositories)
> > >
> > > To sum up the preliminary timeline;
> > >
> > > - December 9th 2018 -> January 9th 2019: Voluntary (coordinate

CONTRIBUTING.md first version

2018-12-12 Thread Dmitriy Pavlov
Hi Igniters,

Just to inform, the very first version of the Contribution step by step
guide was added to GitHub. It is one more entry point for newcomers to find
out what and how should be done to contribute:
https://github.com/apache/ignite/blob/master/CONTRIBUTING.md#contributing-to-apache-ignite


Feel free to share with newcomers and contribute.

Sincerely,
Dmitriy Pavlov


Re: [ML][DISCUSSION] ML Package reorganization

2018-12-12 Thread Dmitriy Pavlov
Hi Yuri,

Would it affect users?

Ignite-code has separation of API classes (not internal) and internal
stuff. Internal classes may be changed from version to version, but API
can't be renamed/moved/and so on. What about ML?

Sincerely,
Dmitriy Pavlov

вт, 11 дек. 2018 г. в 16:56, Yuriy Babak :

> Igniters,
>
> I want to discuss package structure for ML module. Actually I don't like
> our current package organization. The main problem is that we have lots of
> different packages under root package.
>
> For examples algorithm related packages on same level like some utility
> infrastructure packages such as environment(package with some classes
> related with training environment).
>
> So I created branch [1] with example of new package structure. Please share
> any feedback about this package structure and about this idea itself.
>
> Regards,
> Yuriy.
>
> [1] -
>
> https://github.com/YuriBabak/ignite/tree/ignite-10641/modules/ml/src/main/java/org/apache/ignite/ml
>


Re: [MTCGA]: new failures in builds [2342751] needs to be handled

2018-12-11 Thread Dmitriy Pavlov
This notification probably came from some old TC Bot DB records, while the
test is flaky - it is not a mistake.

Now server uses time-based statistics, so bot keeps data for a longer time.

вт, 11 дек. 2018 г. в 20:11, :

> Hi Igniters,
>
>  I've detected some new issue on TeamCity to be handled. You are more than
> welcomed to help.
>
>  If your changes can lead to this failure(s): We're grateful that you were
> a volunteer to make the contribution to this project, but things change and
> you may no longer be able to finalize your contribution.
>  Could you respond to this email and indicate if you wish to continue and
> fix test failures or step down and some committer may revert you commit.
>
>  *New stable failure of a flaky test in master
> IgniteWalRebalanceTest.testWithLocalWalChange
> https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=8228076339467741112=%3Cdefault%3E=testDetails
>  No changes in the build
>
>  - Here's a reminder of what contributors were agreed to do
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
>  - Should you have any questions please contact
> dev@ignite.apache.org
>
> Best Regards,
> Apache Ignite TeamCity Bot
> https://github.com/apache/ignite-teamcity-bot
> Notification generated at 20:11:00 11-12-2018
>


Re: [DISCUSSION] Relocation of Apache git repositories from git-wip-us.apache.org to GitBox

2018-12-11 Thread Dmitriy Pavlov
Hi All,

The Apache Ignite still has a repository on git-wip-us for its code, as
well, for release.

Does anyone have a problem with moving over the Ignite git-wip-us
repository to gitbox voluntarily? This means integrated access and easy PRs
(write access to the GitHub repo).

We need to document support for the decision from a mailing list post, so
here it is.

Sincerely,
Dmitriy Pavlov

сб, 8 дек. 2018 г. в 00:16, Dmitriy Pavlov :

> Hi Igniters,
>
> What do you think about the voluntary migration of project repositories
> from git-wip to GitBox? (See details in the forwarded email.)
>
> Redirection of emails originated by PR actions to notifications@ seems to
> be almost finished, so email flood issue seems to be already solved for dev@
> .
>
> It is up to us to decide if we would like to migrate sooner. Later all
> repositories will be migrated anyway.
>
> Affected repositories
> - Ignite Release
> - Ignite
>
> Please share your opinion.
>
> Sincerely,
> Dmitriy Pavlov
>
> -- Forwarded message -
> From: Daniel Gruno 
> Date: пт, 7 дек. 2018 г. в 19:52
> Subject: [NOTICE] Mandatory relocation of Apache git repositories on
> git-wip-us.apache.org
> To: us...@infra.apache.org 
>
>
> [IF YOUR PROJECT DOES NOT HAVE GIT REPOSITORIES ON GIT-WIP-US PLEASE
>   DISREGARD THIS EMAIL; IT WAS MASS-MAILED TO ALL APACHE PROJECTS]
>
> Hello Apache projects,
>
> I am writing to you because you may have git repositories on the
> git-wip-us server, which is slated to be decommissioned in the coming
> months. All repositories will be moved to the new gitbox service which
> includes direct write access on github as well as the standard ASF
> commit access via gitbox.apache.org.
>
> ## Why this move? ##
> The move comes as a result of retiring the git-wip service, as the
> hardware it runs on is longing for retirement. In lieu of this, we
> have decided to consolidate the two services (git-wip and gitbox), to
> ease the management of our repository systems and future-proof the
> underlying hardware. The move is fully automated, and ideally, nothing
> will change in your workflow other than added features and access to
> GitHub.
>
> ## Timeframe for relocation ##
> Initially, we are asking that projects voluntarily request to move
> their repositories to gitbox, hence this email. The voluntary
> timeframe is between now and January 9th 2019, during which projects
> are free to either move over to gitbox or stay put on git-wip. After
> this phase, we will be requiring the remaining projects to move within
> one month, after which we will move the remaining projects over.
>
> To have your project moved in this initial phase, you will need:
>
> - Consensus in the project (documented via the mailing list)
> - File a JIRA ticket with INFRA to voluntarily move your project repos
>over to gitbox (as stated, this is highly automated and will take
>between a minute and an hour, depending on the size and number of
>your repositories)
>
> To sum up the preliminary timeline;
>
> - December 9th 2018 -> January 9th 2019: Voluntary (coordinated)
>relocation
> - January 9th -> February 6th: Mandated (coordinated) relocation
> - February 7th: All remaining repositories are mass migrated.
>
> This timeline may change to accommodate various scenarios.
>
> ## Using GitHub with ASF repositories ##
> When your project has moved, you are free to use either the ASF
> repository system (gitbox.apache.org) OR GitHub for your development
> and code pushes. To be able to use GitHub, please follow the primer
> at: https://reference.apache.org/committer/github
>
>
> We appreciate your understanding of this issue, and hope that your
> project can coordinate voluntarily moving your repositories in a
> timely manner.
>
> All settings, such as commit mail targets, issue linking, PR
> notification schemes etc will automatically be migrated to gitbox as
> well.
>
> With regards, Daniel on behalf of ASF Infra.
>
> PS:For inquiries, please reply to us...@infra.apache.org, not your
> project's dev list :-).
>
>
>


Re: [Result][VOTE] Creation dedicated list for github notifiacations

2018-12-11 Thread Dmitriy Pavlov
IMO it is not possible, but you can always ask Infra.

One from ideas behind lists is forever-archived interaction, so I guess it
is not possible to remove something from a list.

вт, 11 дек. 2018 г. в 15:27, Denis Mekhanikov :

> Great news!
>
> Is it possible to move all GitHub messages from the archive of the
> developers list
> to the newly-created one?
>
> Denis
>
> вт, 11 дек. 2018 г. в 10:05, Dmitriy Pavlov :
>
> > Hi Igniters,
> >
> > Infra changed notifications, so now GitHub emails are being sent to
> > notifications@
> >
> > Web UI:
> https://lists.apache.org/list.html?notificati...@ignite.apache.org
> >
> > You may subscribe to the list notifications-subscr...@ignite.apache.org
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > ср, 5 дек. 2018 г. в 12:57, Eduard Shangareev <
> eduard.shangar...@gmail.com
> > >:
> >
> > > https://issues.apache.org/jira/browse/INFRA-17351
> > > A ticket was created.
> > >
> > > On Fri, Nov 30, 2018 at 12:04 AM Denis Magda 
> wrote:
> > >
> > > > A request has been submitted.
> > > >
> > > > --
> > > > Denis
> > > >
> > > > On Thu, Nov 29, 2018 at 11:45 AM Dmitriy Pavlov 
> > > > wrote:
> > > >
> > > > > Denis, could you please create a new list for Apache Ignite, e.g.
> > > > > notificati...@ignite.apache.org  ?
> > > > >
> > > > > Only PMC Chair can create a list https://infra.apache.org/
> > > > >
> > > > > Create list feature has been restricted to ASF members and PMC
> chairs
> > > > only.
> > > > > Thank you in advance.
> > > > >
> > > > > Sincerely,
> > > > > Dmitriy Pavlov
> > > > >
> > > > > чт, 29 нояб. 2018 г. в 21:44, Eduard Shangareev <
> > > > > eduard.shangar...@gmail.com>:
> > > > >
> > > > >> Igniters,
> > > > >> The result is successful.
> > > > >>
> > > > >> No "-1".
> > > > >> 11 "+1".
> > > > >> 2 "0".
> > > > >>
> > > > >> Vote thread:
> > > > >>
> > > > >>
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Creation-dedicated-list-for-github-notifiacations-td38485.html
> > > > >>
> > > > >
> > > >
> > >
> >
>


[jira] [Created] (IGNITE-10639) Proof of concept for internal description for packages

2018-12-11 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-10639:
---

 Summary: Proof of concept for internal description for packages
 Key: IGNITE-10639
 URL: https://issues.apache.org/jira/browse/IGNITE-10639
 Project: Ignite
  Issue Type: Task
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov


Need to test opportunities for documenting packages structure using
package-info/readme.txt/readme.md





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


Re: Clean up of our PRs and IEPs before 2019

2018-12-10 Thread Dmitriy Pavlov
My point is - close your outdated PRs/if you don't need. It is not a
pattern nor enforcement nor bureaucracy.

You're absolutely free to use PRs in any way you like to (e.g. experiments
with TC runs).

Now count of PRs shocks newcomers.

вт, 11 дек. 2018 г. в 10:10, Nikolay Izhikov :

> Hello, Ivan.
>
> Personally, I keep my PR's clear.
> So, I don't have dozens of opened PR.
>
> But, I don't support Dmitriy proposal for several reasons:
>
> 1. We introduce some new, not required, level of bureaucracy.
> From my experience - not required bureaucracy is a BAD thing.
>
> 2. We spread our work pattern to whole community.
> I believe there are many patterns of dealing with *YOUR OWN* PRs.
> Some of them can lead to dozens of opened PRs to master.
> Whats wrong with it?
>
> 3. I dont' see any issues with many opened PRs.
> What problem we trying to solve?
>
> 4. Closing abanodned PRs doesn't force anybody to review the rest.
> Instead of ordering something to one way or another, let's solve real
> problem:
>
> - help the community doing PR review.
> - fixing failing tests.
> - introducing new code inspections to make our code base clear.
> - making Ignite improvements
>
> 5. I don't see how our numbers differs from other Apache projects
>
> Apache Kafka - 533 PR opened.
> Apache Spark - 484 PR opened.
> Apache Flink - 430 PR opened.
>
> В Вт, 11/12/2018 в 09:24 +0300, Pavel Tupitsyn пишет:
> > Agree with Dmitriy.
> >
> > We use GitHub PRs in our workflow, therefore we should keep them in
> order.
> >
> > We can close PRs that refer to closed tickets, this can be done with a
> > simple script.
> >
> > On Tue, Dec 11, 2018 at 9:15 AM Павлухин Иван 
> wrote:
> >
> > > Nikolay,
> > >
> > > I must say that when I first saw 1K+ open PRs my first thought was
> > > that something was wrong with a review process. In my mind in not very
> > > big project open PR list can reflect very well the real work in
> > > progress. For bigger projects things become more complicated.
> > >
> > > Dmitriy,
> > >
> > > Do you have some cleanup automation in mind? Immediately I think that
> > > it is fully safe to close all PRs that were not touched more than a
> > > year.
> > > пн, 10 дек. 2018 г. в 20:01, Dmitriy Pavlov :
> > > >
> > > > The main concern is related to chances that newcomer will have to
> obtain
> > >
> > > a
> > > > review support from the community.
> > > >
> > > > Actually, a lot of people doing their best to provide a feedback to
> > > > newcomers, and count of issues still in PA state goes down (84 is a
> > > > relatively small count of issues in PA state). But 1428 PRs may
> imply we
> > > > don't review here, as we have tons of incomplete PRs. Actually, most
> of
> > > > these PRs were merged (but not using ./apply-pull-request.sh script,
> but
> > > > manually, without reference to PRs).
> > > >
> > > > Another benefit of revising this list, if there are any changes which
> > > > were not accomplished with a proper ticket with PA status, we will
> > >
> > > identify
> > > > a number of additional contributions to be applied to the codebase.
> > > >
> > > >
> > > > пн, 10 дек. 2018 г. в 19:53, Nikolay Izhikov :
> > > >
> > > > > Hello, Dmitriy.
> > > > >
> > > > > What, exactly concerns newcomers?
> > > > > What is wrong with opened PR?
> > > > > How project will benefit from closed PR?
> > > > >
> > > > > > The same proposal is related to IEP statuses. If you were
> involved
> > >
> > > in an
> > > > > IEP, please validate its status
> > > > >
> > > > > +1. We should maintain IEP description up to date.
> > > > >
> > > > > пн, 10 дек. 2018 г., 19:15 Dmitriy Pavlov dpav...@apache.org:
> > > > >
> > > > > > Hi Igniters,
> > > > > >
> > > > > > Newcomers to Apache Ignite sometimes became concerned about many
> > >
> > > open PRs
> > > > > > in the project. Apache Ignite TC Bot also performs runs checks
> with
> > >
> > > a PR
> > > > > > open. Apache Ignite pulls list
> > >
> > > https://github.com/apache/ignite/pulls
> > > > > > contains
> > > > > > 1442 PRs open while only 84 issues are waiting for review.
> > > > > >
> > > > > > Could you please verify the list of your PRs in Apache Ignite
> > > > > > https://github.com/pulls  and close every not needed/already
> merged
> > > > > > change?
> > > > > >
> > > > > > The same proposal is related to IEP statuses. If you were
> involved
> > >
> > > in an
> > > > > > IEP, please validate its status here
> > > > > >
> https://cwiki.apache.org/confluence/display/IGNITE/Active+Proposals
> > > > > > Please
> > > > > > set a correct state for your IEP, as well.
> > > > > >
> > > > > > Should you have any questions, please don't hesitate to ask here.
> > >
> > > Thank
> > > > > you
> > > > > > in advance!
> > > > > >
> > > > > > Sincerely,
> > > > > > Dmitriy Pavlov
> > > > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
>


Re: [Result][VOTE] Creation dedicated list for github notifiacations

2018-12-10 Thread Dmitriy Pavlov
Hi Igniters,

Infra changed notifications, so now GitHub emails are being sent to
notifications@

Web UI: https://lists.apache.org/list.html?notificati...@ignite.apache.org

You may subscribe to the list notifications-subscr...@ignite.apache.org

Sincerely,
Dmitriy Pavlov

ср, 5 дек. 2018 г. в 12:57, Eduard Shangareev :

> https://issues.apache.org/jira/browse/INFRA-17351
> A ticket was created.
>
> On Fri, Nov 30, 2018 at 12:04 AM Denis Magda  wrote:
>
> > A request has been submitted.
> >
> > --
> > Denis
> >
> > On Thu, Nov 29, 2018 at 11:45 AM Dmitriy Pavlov 
> > wrote:
> >
> > > Denis, could you please create a new list for Apache Ignite, e.g.
> > > notificati...@ignite.apache.org  ?
> > >
> > > Only PMC Chair can create a list https://infra.apache.org/
> > >
> > > Create list feature has been restricted to ASF members and PMC chairs
> > only.
> > > Thank you in advance.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > чт, 29 нояб. 2018 г. в 21:44, Eduard Shangareev <
> > > eduard.shangar...@gmail.com>:
> > >
> > >> Igniters,
> > >> The result is successful.
> > >>
> > >> No "-1".
> > >> 11 "+1".
> > >> 2 "0".
> > >>
> > >> Vote thread:
> > >>
> > >>
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Creation-dedicated-list-for-github-notifiacations-td38485.html
> > >>
> > >
> >
>


Re: Clean up of our PRs and IEPs before 2019

2018-12-10 Thread Dmitriy Pavlov
Hi Ivan,

Unfortunately, I don't have the necessary rights to close PRs. Only PR
author can close.

I can only imagine the following way: Produce fake commit(s) with message
Fixes #. But this may be acceptable if PR author is not responding.

Simple expiration policy may have its own issues. There can be
- an enthusiast who didn't provide correctly set up ticket
- But prepared just PR with a good addition to the product.

It is not so correct to accept such contribution (HTC requires JIRA), but
anyway, having PR open we can ask him or her to set up a ticket.

Sincerely,
Dmitriy Pavlov

вт, 11 дек. 2018 г. в 09:24, Pavel Tupitsyn :

> Agree with Dmitriy.
>
> We use GitHub PRs in our workflow, therefore we should keep them in order.
>
> We can close PRs that refer to closed tickets, this can be done with a
> simple script.
>
> On Tue, Dec 11, 2018 at 9:15 AM Павлухин Иван  wrote:
>
> > Nikolay,
> >
> > I must say that when I first saw 1K+ open PRs my first thought was
> > that something was wrong with a review process. In my mind in not very
> > big project open PR list can reflect very well the real work in
> > progress. For bigger projects things become more complicated.
> >
> > Dmitriy,
> >
> > Do you have some cleanup automation in mind? Immediately I think that
> > it is fully safe to close all PRs that were not touched more than a
> > year.
> > пн, 10 дек. 2018 г. в 20:01, Dmitriy Pavlov :
> > >
> > > The main concern is related to chances that newcomer will have to
> obtain
> > a
> > > review support from the community.
> > >
> > > Actually, a lot of people doing their best to provide a feedback to
> > > newcomers, and count of issues still in PA state goes down (84 is a
> > > relatively small count of issues in PA state). But 1428 PRs may imply
> we
> > > don't review here, as we have tons of incomplete PRs. Actually, most of
> > > these PRs were merged (but not using ./apply-pull-request.sh script,
> but
> > > manually, without reference to PRs).
> > >
> > > Another benefit of revising this list, if there are any changes which
> > > were not accomplished with a proper ticket with PA status, we will
> > identify
> > > a number of additional contributions to be applied to the codebase.
> > >
> > >
> > > пн, 10 дек. 2018 г. в 19:53, Nikolay Izhikov :
> > >
> > > > Hello, Dmitriy.
> > > >
> > > > What, exactly concerns newcomers?
> > > > What is wrong with opened PR?
> > > > How project will benefit from closed PR?
> > > >
> > > > > The same proposal is related to IEP statuses. If you were involved
> > in an
> > > > IEP, please validate its status
> > > >
> > > > +1. We should maintain IEP description up to date.
> > > >
> > > > пн, 10 дек. 2018 г., 19:15 Dmitriy Pavlov dpav...@apache.org:
> > > >
> > > > > Hi Igniters,
> > > > >
> > > > > Newcomers to Apache Ignite sometimes became concerned about many
> > open PRs
> > > > > in the project. Apache Ignite TC Bot also performs runs checks with
> > a PR
> > > > > open. Apache Ignite pulls list
> > https://github.com/apache/ignite/pulls
> > > > > contains
> > > > > 1442 PRs open while only 84 issues are waiting for review.
> > > > >
> > > > > Could you please verify the list of your PRs in Apache Ignite
> > > > > https://github.com/pulls  and close every not needed/already
> merged
> > > > > change?
> > > > >
> > > > > The same proposal is related to IEP statuses. If you were involved
> > in an
> > > > > IEP, please validate its status here
> > > > >
> https://cwiki.apache.org/confluence/display/IGNITE/Active+Proposals
> > > > > Please
> > > > > set a correct state for your IEP, as well.
> > > > >
> > > > > Should you have any questions, please don't hesitate to ask here.
> > Thank
> > > > you
> > > > > in advance!
> > > > >
> > > > > Sincerely,
> > > > > Dmitriy Pavlov
> > > > >
> > > >
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >
>


Re: Is it time to move forward to JUnit4 (5)?

2018-12-10 Thread Dmitriy Pavlov
Vladimir, AFAIK there will be no single change for overall migration. All
migration will be in a step-by-step manner and mostly transparent for
contributors.

Both approaches 3&4 will be available for some time.

First of all, we need an opportunity to run existing integration tests in
JUnit4.

пн, 10 дек. 2018 г. в 20:38, Vladimir Ozerov :

> Ed,
>
> Several questions from my side:
> 1) When the change is expected to be merged?
> 2) What contributors with opened PRs and new or updated JUnit3 tests are
> supposed to do? Rewrite to JUnit4?
>
> If yes, then we should give them time to have a chance to get used to new
> approach and resolve possible conflicts.
>
> Vladimir.
>
> пн, 10 дек. 2018 г. в 20:32, Eduard Shangareev <
> eduard.shangar...@gmail.com
> >:
>
> > Ivan,
> >
> > So, suggested actions with the new approach:
> > 1. Add @Test annotation on test methods.
> > 2. Add @RunWith(JUnit4.class) annotation on test class.
> > 3. Add @Before, @After on methods which should run before, after a
> > test (setUp, tearDown in current approach).
> > 4. Add your test-class to a suite using suite.addTest(new
> > JUnit4TestAdapter(YourTestClass.class));
> > 5. Use @Ignore instead fail(); for muting test.
> > 6. You could start using @Parametrized instead of inheritance.
> >
> >
> > On Mon, Dec 3, 2018 at 1:15 PM Павлухин Иван 
> wrote:
> >
> > > Hi Oleg,
> > >
> > > I noticed that GridAbstractTest is now capable to run junit4 tests.
> > > What are the current recommendations for writing new tests? Can we use
> > > junit4 annotation for new tests?
> > > пн, 12 нояб. 2018 г. в 19:58, oignatenko :
> > > >
> > > > Hi Ivan,
> > > >
> > > > I am currently testing approach you used in pull/5354 in the "pilot"
> > > > sub-task with examples tests (IGNITE-10174).
> > > >
> > > > So far it looks more and more like the way to go. The most promising
> > > thing I
> > > > observed is that after I changed classes in our test framework the
> way
> > > you
> > > > did, execution of (unchanged) examples tests went exactly the same as
> > it
> > > was
> > > > before changes.
> > > >
> > > > This indicates that existing tests won't be affected making it indeed
> > low
> > > > risk.
> > > >
> > > > After that I converted examples tests to Junit 4 by adding @RunWith
> and
> > > > @Test annotations and tried a few, and these looked okay.
> > > >
> > > > Currently I am running full examples test suite and after it is over
> I
> > > will
> > > > compare results to the reference list I made by running it prior to
> > > > migration.
> > > >
> > > > regards, Oleg
> > > >
> > > >
> > > >
> > > > --
> > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
> >
>


Re: Clean up of our PRs and IEPs before 2019

2018-12-10 Thread Dmitriy Pavlov
The main concern is related to chances that newcomer will have to obtain a
review support from the community.

Actually, a lot of people doing their best to provide a feedback to
newcomers, and count of issues still in PA state goes down (84 is a
relatively small count of issues in PA state). But 1428 PRs may imply we
don't review here, as we have tons of incomplete PRs. Actually, most of
these PRs were merged (but not using ./apply-pull-request.sh script, but
manually, without reference to PRs).

Another benefit of revising this list, if there are any changes which
were not accomplished with a proper ticket with PA status, we will identify
a number of additional contributions to be applied to the codebase.


пн, 10 дек. 2018 г. в 19:53, Nikolay Izhikov :

> Hello, Dmitriy.
>
> What, exactly concerns newcomers?
> What is wrong with opened PR?
> How project will benefit from closed PR?
>
> > The same proposal is related to IEP statuses. If you were involved in an
> IEP, please validate its status
>
> +1. We should maintain IEP description up to date.
>
> пн, 10 дек. 2018 г., 19:15 Dmitriy Pavlov dpav...@apache.org:
>
> > Hi Igniters,
> >
> > Newcomers to Apache Ignite sometimes became concerned about many open PRs
> > in the project. Apache Ignite TC Bot also performs runs checks with a PR
> > open. Apache Ignite pulls list https://github.com/apache/ignite/pulls
> > contains
> > 1442 PRs open while only 84 issues are waiting for review.
> >
> > Could you please verify the list of your PRs in Apache Ignite
> > https://github.com/pulls  and close every not needed/already merged
> > change?
> >
> > The same proposal is related to IEP statuses. If you were involved in an
> > IEP, please validate its status here
> > https://cwiki.apache.org/confluence/display/IGNITE/Active+Proposals
> > Please
> > set a correct state for your IEP, as well.
> >
> > Should you have any questions, please don't hesitate to ask here. Thank
> you
> > in advance!
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
>


Clean up of our PRs and IEPs before 2019

2018-12-10 Thread Dmitriy Pavlov
Hi Igniters,

Newcomers to Apache Ignite sometimes became concerned about many open PRs
in the project. Apache Ignite TC Bot also performs runs checks with a PR
open. Apache Ignite pulls list https://github.com/apache/ignite/pulls contains
1442 PRs open while only 84 issues are waiting for review.

Could you please verify the list of your PRs in Apache Ignite
https://github.com/pulls  and close every not needed/already merged change?

The same proposal is related to IEP statuses. If you were involved in an
IEP, please validate its status here
https://cwiki.apache.org/confluence/display/IGNITE/Active+Proposals  Please
set a correct state for your IEP, as well.

Should you have any questions, please don't hesitate to ask here. Thank you
in advance!

Sincerely,
Dmitriy Pavlov


Re: Default failure handler was changed for tests

2018-12-10 Thread Dmitriy Pavlov
Reverted.

https://issues.apache.org/jira/browse/IGNITE-8227 reopened

пн, 10 дек. 2018 г. в 16:23, Dmitriy Pavlov :

> Anton, I was expecting that you revert, because you wanted to do it.
>
> Provided that I agree that fix could be reverted because of both
> functional and style possible improvements, does not mean I believe it is
> the only option and it should be reverted.
>
> Even if I agree to revert doesn't mean all community agrees, so reverting
> just 1 minute after writing to dev list would be strange. I believe we
> should be courteous enough to give a couple of days for people to come and
> give feedback.
>
> So if you have a spare minute, please go ahead. If not, I can do it later.
>
> пн, 10 дек. 2018 г. в 14:23, Anton Vinogradov :
>
>> Dmitriy,
>>
>> You confirmed that fix should be reverted and reworked last Friday.
>> Why it still not reverted?
>>
>> On Mon, Dec 10, 2018 at 12:46 AM Dmitrii Ryabov 
>> wrote:
>>
>> > Agree, it is reasonable to revert.
>> > пт, 7 дек. 2018 г. в 18:44, Dmitriy Pavlov :
>> > >
>> > > Hi Ilya,
>> > >
>> > > thank you for noticing.
>> > >
>> > > Calling to fail is equal to re-throw,
>> > >
>> > > throw new AssertionFailedError(message);
>> > >
>> > > So, yes, for now it is absolutely valid reason to revert and rework
>> fix
>> > >
>> > > - as Nikolay suggested to reduce method override ocurrences.
>> > > - and with transferring this exception into GridAbstractTest and
>> > > correctly failing test.
>> > >
>> > > Sincerely,
>> > > Dmitriy Pavlov
>> > >
>> > >
>> > > пт, 7 дек. 2018 г. в 18:38, Ilya Lantukh :
>> > >
>> > > > Unfortunately, this FailureHandler doesn't seem to work. I wrote a
>> test
>> > > > that reproduces a bug and should fail. It prints the following text
>> > into
>> > > > log, but the test still passes "successfully":
>> > > >
>> > > > [2018-12-07
>> > > >
>> > > >
>> >
>> 18:28:23,800][ERROR][sys-stripe-1-#345%recovery.GridPointInTimeRecoveryCacheNoAffinityExchangeTest1%][IgniteTestResources]
>> > > > Critical system error detected. Will be handled accordingly to
>> > configured
>> > > > handler [hnd=TestFailingFailureHandler [], failureCtx=FailureContext
>> > > > [type=CRITICAL_ERROR, err=java.lang.IllegalStateException: Unable to
>> > find
>> > > > consistentId by UUID [nodeId=80dd2ec6-1913-4a5c-a839-630315c3,
>> > > > topVer=AffinityTopologyVersion [topVer=12, minorTopVer=0
>> > > > java.lang.IllegalStateException: Unable to find consistentId by UUID
>> > > > [nodeId=80dd2ec6-1913-4a5c-a839-630315c3,
>> > > > topVer=AffinityTopologyVersion [topVer=12, minorTopVer=0]]
>> > > > at
>> > > >
>> > > >
>> >
>> org.apache.ignite.internal.managers.discovery.ConsistentIdMapper.mapToCompactId(ConsistentIdMapper.java:62)
>> > > > at
>> > > >
>> > > >
>> >
>> org.apache.ignite.internal.managers.discovery.ConsistentIdMapper.mapToCompactIds(ConsistentIdMapper.java:123)
>> > > > at
>> > > >
>> > > >
>> >
>> org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager.newTxRecord(IgniteTxManager.java:2507)
>> > > > at
>> > > >
>> > > >
>> >
>> org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager.logTxRecord(IgniteTxManager.java:2483)
>> > > > at
>> > > >
>> > > >
>> >
>> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.state(IgniteTxAdapter.java:1226)
>> > > > at
>> > > >
>> > > >
>> >
>> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.state(IgniteTxAdapter.java:1054)
>> > > > at
>> > > >
>> > > >
>> >
>> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.startRemoteTx(IgniteTxHandler.java:1836)
>> > > > at
>> > > >
>> > > >
>> >
>> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processDhtTxPrepareRequest(IgniteTxHandler.java:1180)
>> > > > 

Re: Default failure handler was changed for tests

2018-12-10 Thread Dmitriy Pavlov
Anton, I was expecting that you revert, because you wanted to do it.

Provided that I agree that fix could be reverted because of both functional
and style possible improvements, does not mean I believe it is the only
option and it should be reverted.

Even if I agree to revert doesn't mean all community agrees, so reverting
just 1 minute after writing to dev list would be strange. I believe we
should be courteous enough to give a couple of days for people to come and
give feedback.

So if you have a spare minute, please go ahead. If not, I can do it later.

пн, 10 дек. 2018 г. в 14:23, Anton Vinogradov :

> Dmitriy,
>
> You confirmed that fix should be reverted and reworked last Friday.
> Why it still not reverted?
>
> On Mon, Dec 10, 2018 at 12:46 AM Dmitrii Ryabov 
> wrote:
>
> > Agree, it is reasonable to revert.
> > пт, 7 дек. 2018 г. в 18:44, Dmitriy Pavlov :
> > >
> > > Hi Ilya,
> > >
> > > thank you for noticing.
> > >
> > > Calling to fail is equal to re-throw,
> > >
> > > throw new AssertionFailedError(message);
> > >
> > > So, yes, for now it is absolutely valid reason to revert and rework fix
> > >
> > > - as Nikolay suggested to reduce method override ocurrences.
> > > - and with transferring this exception into GridAbstractTest and
> > > correctly failing test.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > >
> > > пт, 7 дек. 2018 г. в 18:38, Ilya Lantukh :
> > >
> > > > Unfortunately, this FailureHandler doesn't seem to work. I wrote a
> test
> > > > that reproduces a bug and should fail. It prints the following text
> > into
> > > > log, but the test still passes "successfully":
> > > >
> > > > [2018-12-07
> > > >
> > > >
> >
> 18:28:23,800][ERROR][sys-stripe-1-#345%recovery.GridPointInTimeRecoveryCacheNoAffinityExchangeTest1%][IgniteTestResources]
> > > > Critical system error detected. Will be handled accordingly to
> > configured
> > > > handler [hnd=TestFailingFailureHandler [], failureCtx=FailureContext
> > > > [type=CRITICAL_ERROR, err=java.lang.IllegalStateException: Unable to
> > find
> > > > consistentId by UUID [nodeId=80dd2ec6-1913-4a5c-a839-630315c3,
> > > > topVer=AffinityTopologyVersion [topVer=12, minorTopVer=0
> > > > java.lang.IllegalStateException: Unable to find consistentId by UUID
> > > > [nodeId=80dd2ec6-1913-4a5c-a839-630315c3,
> > > > topVer=AffinityTopologyVersion [topVer=12, minorTopVer=0]]
> > > > at
> > > >
> > > >
> >
> org.apache.ignite.internal.managers.discovery.ConsistentIdMapper.mapToCompactId(ConsistentIdMapper.java:62)
> > > > at
> > > >
> > > >
> >
> org.apache.ignite.internal.managers.discovery.ConsistentIdMapper.mapToCompactIds(ConsistentIdMapper.java:123)
> > > > at
> > > >
> > > >
> >
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager.newTxRecord(IgniteTxManager.java:2507)
> > > > at
> > > >
> > > >
> >
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager.logTxRecord(IgniteTxManager.java:2483)
> > > > at
> > > >
> > > >
> >
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.state(IgniteTxAdapter.java:1226)
> > > > at
> > > >
> > > >
> >
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.state(IgniteTxAdapter.java:1054)
> > > > at
> > > >
> > > >
> >
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.startRemoteTx(IgniteTxHandler.java:1836)
> > > > at
> > > >
> > > >
> >
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processDhtTxPrepareRequest(IgniteTxHandler.java:1180)
> > > > at
> > > >
> > > >
> >
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$400(IgniteTxHandler.java:118)
> > > > at
> > > >
> > > >
> >
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$5.apply(IgniteTxHandler.java:222)
> > > > at
> > > >
> > > >
> >
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$5.apply(IgniteTxHandler.java:220)
> > > > at

[jira] [Created] (IGNITE-10620) [TC Bot] Implement Tracked Branches processor and IssueDetector unit tests

2018-12-10 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-10620:
---

 Summary: [TC Bot] Implement Tracked Branches processor and 
IssueDetector unit tests
 Key: IGNITE-10620
 URL: https://issues.apache.org/jira/browse/IGNITE-10620
 Project: Ignite
  Issue Type: Task
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov


Implement actual tests for tracked branches processing and issue detectors. 
Skeletons:

org.apache.ignite.ci.tcbot.issue.IssueDetectorTest#testDetector

org.apache.ignite.ci.tcbot.chain.TrackedBranchProcessorTest#testTrackedBranchChainsProcessor

The actual challenge is refactoring of configuration to use not singletons, but 
some injected interfaces with mock-able alternatives, refactor scheduler, and 
any other refactorings needed to cover issue detection by test. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[DISCUSSION] Relocation of Apache git repositories from git-wip-us.apache.org to GitBox

2018-12-07 Thread Dmitriy Pavlov
Hi Igniters,

What do you think about the voluntary migration of project repositories
from git-wip to GitBox? (See details in the forwarded email.)

Redirection of emails originated by PR actions to notifications@ seems to
be almost finished, so email flood issue seems to be already solved for dev@
.

It is up to us to decide if we would like to migrate sooner. Later all
repositories will be migrated anyway.

Affected repositories
- Ignite Release
- Ignite

Please share your opinion.

Sincerely,
Dmitriy Pavlov

-- Forwarded message -
From: Daniel Gruno 
Date: пт, 7 дек. 2018 г. в 19:52
Subject: [NOTICE] Mandatory relocation of Apache git repositories on
git-wip-us.apache.org
To: us...@infra.apache.org 


[IF YOUR PROJECT DOES NOT HAVE GIT REPOSITORIES ON GIT-WIP-US PLEASE
  DISREGARD THIS EMAIL; IT WAS MASS-MAILED TO ALL APACHE PROJECTS]

Hello Apache projects,

I am writing to you because you may have git repositories on the
git-wip-us server, which is slated to be decommissioned in the coming
months. All repositories will be moved to the new gitbox service which
includes direct write access on github as well as the standard ASF
commit access via gitbox.apache.org.

## Why this move? ##
The move comes as a result of retiring the git-wip service, as the
hardware it runs on is longing for retirement. In lieu of this, we
have decided to consolidate the two services (git-wip and gitbox), to
ease the management of our repository systems and future-proof the
underlying hardware. The move is fully automated, and ideally, nothing
will change in your workflow other than added features and access to
GitHub.

## Timeframe for relocation ##
Initially, we are asking that projects voluntarily request to move
their repositories to gitbox, hence this email. The voluntary
timeframe is between now and January 9th 2019, during which projects
are free to either move over to gitbox or stay put on git-wip. After
this phase, we will be requiring the remaining projects to move within
one month, after which we will move the remaining projects over.

To have your project moved in this initial phase, you will need:

- Consensus in the project (documented via the mailing list)
- File a JIRA ticket with INFRA to voluntarily move your project repos
   over to gitbox (as stated, this is highly automated and will take
   between a minute and an hour, depending on the size and number of
   your repositories)

To sum up the preliminary timeline;

- December 9th 2018 -> January 9th 2019: Voluntary (coordinated)
   relocation
- January 9th -> February 6th: Mandated (coordinated) relocation
- February 7th: All remaining repositories are mass migrated.

This timeline may change to accommodate various scenarios.

## Using GitHub with ASF repositories ##
When your project has moved, you are free to use either the ASF
repository system (gitbox.apache.org) OR GitHub for your development
and code pushes. To be able to use GitHub, please follow the primer
at: https://reference.apache.org/committer/github


We appreciate your understanding of this issue, and hope that your
project can coordinate voluntarily moving your repositories in a
timely manner.

All settings, such as commit mail targets, issue linking, PR
notification schemes etc will automatically be migrated to gitbox as
well.

With regards, Daniel on behalf of ASF Infra.

PS:For inquiries, please reply to us...@infra.apache.org, not your
project's dev list :-).


Re: [GitHub] ignite pull request #5612: IGNITE-10371: [ML] Add multiple metrics calculati...

2018-12-07 Thread Dmitriy Pavlov
It seems we really need a reviewer field in JIRA. The previous discussion
was completed with consensus, and now I should create a ticket to INFRA.

пт, 7 дек. 2018 г. в 21:54, Alexey Zinoviev :

> Dear Igniters, please have a look and summarise your thougths about that PR
> here, in developer list.
> Especially, I'm waiting review from @aplatonov
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>


Re: Default failure handler was changed for tests

2018-12-07 Thread Dmitriy Pavlov
Hi Ilya,

thank you for noticing.

Calling to fail is equal to re-throw,

throw new AssertionFailedError(message);

So, yes, for now it is absolutely valid reason to revert and rework fix

- as Nikolay suggested to reduce method override ocurrences.
- and with transferring this exception into GridAbstractTest and
correctly failing test.

Sincerely,
Dmitriy Pavlov


пт, 7 дек. 2018 г. в 18:38, Ilya Lantukh :

> Unfortunately, this FailureHandler doesn't seem to work. I wrote a test
> that reproduces a bug and should fail. It prints the following text into
> log, but the test still passes "successfully":
>
> [2018-12-07
>
> 18:28:23,800][ERROR][sys-stripe-1-#345%recovery.GridPointInTimeRecoveryCacheNoAffinityExchangeTest1%][IgniteTestResources]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=TestFailingFailureHandler [], failureCtx=FailureContext
> [type=CRITICAL_ERROR, err=java.lang.IllegalStateException: Unable to find
> consistentId by UUID [nodeId=80dd2ec6-1913-4a5c-a839-630315c3,
> topVer=AffinityTopologyVersion [topVer=12, minorTopVer=0
> java.lang.IllegalStateException: Unable to find consistentId by UUID
> [nodeId=80dd2ec6-1913-4a5c-a839-630315c3,
> topVer=AffinityTopologyVersion [topVer=12, minorTopVer=0]]
> at
>
> org.apache.ignite.internal.managers.discovery.ConsistentIdMapper.mapToCompactId(ConsistentIdMapper.java:62)
> at
>
> org.apache.ignite.internal.managers.discovery.ConsistentIdMapper.mapToCompactIds(ConsistentIdMapper.java:123)
> at
>
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager.newTxRecord(IgniteTxManager.java:2507)
> at
>
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxManager.logTxRecord(IgniteTxManager.java:2483)
> at
>
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.state(IgniteTxAdapter.java:1226)
> at
>
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.state(IgniteTxAdapter.java:1054)
> at
>
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.startRemoteTx(IgniteTxHandler.java:1836)
> at
>
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processDhtTxPrepareRequest(IgniteTxHandler.java:1180)
> at
>
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$400(IgniteTxHandler.java:118)
> at
>
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$5.apply(IgniteTxHandler.java:222)
> at
>
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$5.apply(IgniteTxHandler.java:220)
> at
>
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1059)
> at
>
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:584)
> at
>
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:383)
> at
>
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:309)
> at
>
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:100)
> at
>
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:299)
> at
>
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1568)
> at
>
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1196)
> at
>
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127)
> at
>
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1092)
> at
>
> org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:505)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
> at java.lang.Thread.run(Thread.java:748)
>
>
> On Thu, Dec 6, 2018 at 4:01 PM Anton Vinogradov  wrote:
>
> > >> We stop, for now, then you will chill a
> > >> little bit, then you will have an absolutely fantastic weekend, and
> then
> > on
> > >> Monday, Dec 10 we will continue this discussion in a positive and
> > >> constructive manner.
> > Agree
> >
> > On Thu, Dec 6, 2018 at 3:55 PM Nikolay Izhikov 
> > wrote:
> >
> > > Anton.
> > >
> > > I discussed this fix privately with Dmitriy Pavlov.
> > >
> > > 1. We had NoOpHandler for ALL tests before this merge.
&

Re: Contribution: Gaurav Sharma

2018-12-07 Thread Dmitriy Pavlov
Hello, Gaurav Sharma!

Welcome to the Apache Software Foundation and to the Apache Ignite
Community.

I've added your account to the list of contributors. Now you should be able
to assign an issue to yourself.

Should you have any questions please do not hesitate to ask here. Looking
forward to your contributions.

Sincerely,
Dmitriy Pavlov

P.S. Additional references that should boost your onboarding.

Please subscribe to both dev@ and user@ lists, optionally you may subscribe
to notifications@:
https://ignite.apache.org/community/resources.html#mail-lists

Get familiar with Apache Ignite development process described here:
https://cwiki.apache.org/confluence/display/IGNITE/Development+Process

Instructions on how to contribute can be found here:
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute

Project setup in Intellij IDEA:
https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup

Once you got familiar and were able to run a few examples
These are the easy tickets to start with:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20labels%20in%20(newbie)%20and%20status%20%3D%20OPEN



пт, 7 дек. 2018 г. в 11:39, Gaurav Sharma :

> Hello Igniteries,
>
> I would like to contribute to the Ignite community.
>
> My JIRA username is gaurav_sharma.
>
> Sincerely,
> Gaurav Sharma
>


Re: [jira] [Created] (IGNITE-10577) ignite-kubernetes is missing jackson-annotations dependency

2018-12-07 Thread Dmitriy Pavlov
I've checked this patch and it looks good to me, it is really simple.

But probably we should consider auto-test it somehow. Nikolay Kulagin,
could you chime in?

Stephen, could it be a workaround for end-user code? I guess user can just
add this dependency in the pom and everything will work well.

Because release 2.7 is already done, the nearest time the issue could be
fixed is 2.8.

чт, 6 дек. 2018 г. в 20:43, Stephen Darlington <
stephen.darling...@gridgain.com>:

> Not sure what the etiquette on this is, but this is a regression in 2.7. I
> added a simple patch to the ticket.
>
> Regards,
> Stephen
>
> > On 6 Dec 2018, at 14:42, Stephen Darlington (JIRA) 
> wrote:
> >
> > Stephen Darlington created IGNITE-10577:
> > ---
> >
> > Summary: ignite-kubernetes is missing jackson-annotations
> dependency
> > Key: IGNITE-10577
> > URL: https://issues.apache.org/jira/browse/IGNITE-10577
> > Project: Ignite
> >  Issue Type: Bug
> >  Components: build
> >Affects Versions: 2.7
> >Reporter: Stephen Darlington
> >Assignee: Stephen Darlington
> >
> >
> > When starting 2.7 with the ignite-kubernetes option I get the following
> exception on startup:
> >
> > {{[13:44:34,605][SEVERE][main][IgniteKernal] Got exception while
> starting (will rollback startup
> routine).}}{{java.lang.NoClassDefFoundError:
> com/fasterxml/jackson/annotation/JsonView}}{{ at
> com.fasterxml.jackson.databind.introspect.JacksonAnnotationIntrospector.(JacksonAnnotationIntrospector.java:37)}}{{
> at
> com.fasterxml.jackson.databind.ObjectMapper.(ObjectMapper.java:291)}}{{
> at
> org.apache.ignite.spi.discovery.tcp.ipfinder.kubernetes.TcpDiscoveryKubernetesIpFinder.getRegisteredAddresses(TcpDiscoveryKubernetesIpFinder.java:151)}}{{
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.registeredAddresses(TcpDiscoverySpi.java:1900)}}{{
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.resolvedAddresses(TcpDiscoverySpi.java:1848)}}{{
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl.sendJoinRequestMessage(ServerImpl.java:1049)}}{{
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:910)}}
> >
> > It's clearly missing a dependency.
> >
> >
> >
> > --
> > This message was sent by Atlassian JIRA
> > (v7.6.3#76005)
>
>
>


[ANNOUNCE] Apache Ignite 2.7.0 Vulnerable Dependecies Updates

2018-12-07 Thread Dmitriy Pavlov
The Apache Ignite Community is pleased to announce that recently released
Apache Ignite 2.7.0 replaces some vulnerable dependencies to versions with
fixes.



Apache Ignite https://ignite.apache.org/  is a memory-centric distributed
database, caching, and processing platform for transactional, analytical,
and streaming workloads delivering in-memory speeds at petabyte scale.



Apache Ignite 2.7 replaced following dependencies in to avoid usage of
vulnerable 3rd party software by end users:



Apache Log4j
https://nvd.nist.gov/vuln/detail/CVE-2017-5645

FasterXML jackson-databind
https://nvd.nist.gov/vuln/detail/CVE-2017-15095 ,

https://nvd.nist.gov/vuln/detail/CVE-2017-17485 ,

https://nvd.nist.gov/vuln/detail/CVE-2017-7525 ,

https://nvd.nist.gov/vuln/detail/CVE-2018-5968 ,

https://nvd.nist.gov/vuln/detail/CVE-2018-7489



Scala
https://nvd.nist.gov/vuln/detail/CVE-2017-15288

Apache Commons
https://nvd.nist.gov/vuln/detail/CVE-2015-6420 ,

https://nvd.nist.gov/vuln/detail/CVE-2015-7501 ,

https://nvd.nist.gov/vuln/detail/CVE-2017-15708



Netty Project

https://nvd.nist.gov/vuln/detail/CVE-2016-4970

JCraft

https://nvd.nist.gov/vuln/detail/CVE-2016-5725



Apache Tomcat
https://nvd.nist.gov/vuln/detail/CVE-2016-3092 ,

https://nvd.nist.gov/vuln/detail/CVE-2016-8735 ,

https://nvd.nist.gov/vuln/detail/CVE-2018-8014


Guava
https://nvd.nist.gov/vuln/detail/CVE-2018-10237

Apache Camel
https://nvd.nist.gov/vuln/detail/CVE-2015-5344 ,

https://nvd.nist.gov/vuln/detail/CVE-2015-5348 ,

https://nvd.nist.gov/vuln/detail/CVE-2016-8749 ,

https://nvd.nist.gov/vuln/detail/CVE-2017-12633 ,

https://nvd.nist.gov/vuln/detail/CVE-2017-12634  ,

https://nvd.nist.gov/vuln/detail/CVE-2017-3159 ,

https://nvd.nist.gov/vuln/detail/CVE-2017-5643

Spring Framework

https://nvd.nist.gov/vuln/detail/CVE-2018-1257 ,

https://nvd.nist.gov/vuln/detail/CVE-2018-1258



Spring Data Commons

https://nvd.nist.gov/vuln/detail/CVE-2018-1259 ,

https://nvd.nist.gov/vuln/detail/CVE-2018-1273



Jetty

https://nvd.nist.gov/vuln/detail/CVE-2016-4800 ,

https://nvd.nist.gov/vuln/detail/CVE-2017-9735 ,

https://nvd.nist.gov/vuln/detail/CVE-2016-4800 ,

https://nvd.nist.gov/vuln/detail/CVE-2017-9735 ,

https://nvd.nist.gov/vuln/detail/CVE-2016-4800 ,

https://nvd.nist.gov/vuln/detail/CVE-2017-7658



Lucene
https://nvd.nist.gov/vuln/detail/CVE-2017-12629

Mitigation:
•Upgrade to Apache Ignite 2.7 or later version



Credit:
Segu Riluvan discovered the usage of vulnerable modules in dependencies of
Apache Ignite.


Thanks for everyone who was involved into dependencies migration.

Best Regards,

Dmitriy Pavlov on behalf of Apache Ignite community


Re: Hello!

2018-12-06 Thread Dmitriy Pavlov
Hello, Masaki Nakano!

Welcome to the Apache Software Foundation and to the Apache Ignite
Community.

I've added your account to the list of contributors. Now you should be able
to assign an issue to yourself.

Should you have any questions please do not hesitate to ask here. Looking
forward to your contributions.

Sincerely,
Dmitriy Pavlov

P.S. Additional references that should boost your onboarding.

Please subscribe to both dev@ and user@ lists, optionally you may subscribe
to notifications@:
https://ignite.apache.org/community/resources.html#mail-lists

Get familiar with Apache Ignite development process described here:
https://cwiki.apache.org/confluence/display/IGNITE/Development+Process

Instructions on how to contribute can be found here:
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute

Project setup in Intellij IDEA:
https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup

Once you got familiar and were able to run a few examples
These are the easy tickets to start with:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20IGNITE%20AND%20labels%20in%20(newbie)%20and%20status%20%3D%20OPEN


пт, 7 дек. 2018 г. в 07:42, Masaki Nakano :

> Hello, Igniters!
>
> I want to contribute Apache Ignite.
> My JIRA username is `kanosaki`.
>
> Sincerely,
> Masaki Nakano
>


Re: Default failure handler was changed for tests

2018-12-06 Thread Dmitriy Pavlov
Anton, I have another proposal. We stop, for now, then you will chill a
little bit, then you will have an absolutely fantastic weekend, and then on
Monday, Dec 10 we will continue this discussion in a positive and
constructive manner.

Trying to win in a match "my revert is bigger than yours/my code is better
than yours" will not help to anyone.

чт, 6 дек. 2018 г. в 15:02, Anton Vinogradov :

> >> I still hope Anton will do the first bunch of tests research to
> demonstrate
> >> the idea.
>
> Dmitriy,
> Just want to remind you that we already spend time here because of
> unacceptable code merge situation.
> Such merges should NEVER happen again.
> Please, next time make sure that code you merge has no massive duplication
> and fixes without proper reason investigation.
> Committer always MUST be ready to explain each symbol inside code he
> merged.
> The situation when you have no clue why it written this way unacceptable.
>
> Feel free to start a discussion at private in case you have some
> objections.
> But, hope you agree and will help us to solve the issue instead.
>
> Dmitrii,
> >> Anton, I mean `copy-paste reduce` ticket. I'll try to describe the
> reasons for
> >> no-op in tests. Then, we can create tickets to fix this cases if needed.
>
> In case no-one will be ready to start a proper fix (investigate why every
> no-op required and create tickets for each problem) before Friday evening,
> the code will be rolled back.
> Simple no-op is better that same but overcomplicated.
>
> On Thu, Dec 6, 2018 at 2:14 PM Dmitrii Ryabov 
> wrote:
>
> > Anton, I mean `copy-paste reduce` ticket. I'll try to describe reasons
> for
> > no-op in tests. Then, we can create tickets to fix this cases if needed.
> >
> > чт, 6 дек. 2018 г., 13:53 Dmitriy Pavlov dpav...@apache.org:
> >
> > > BTW, No-Op or StopNode-FailTest in case of a deep investigation will
> > always
> > > require to understand what test does and what it tests.
> > >
> > > So we can get a positive outcome from this research if we agree to add
> > > - a small description to each test about the reason for existing of
> this
> > > test,
> > > - what is the expected behavior of the product in the test, and how it
> is
> > > checked?
> > > - failure handler influence, etc.
> > >
> > > I still hope Anton will do the first bunch of tests research to
> > demonstrate
> > > the idea.
> > >
> > > чт, 6 дек. 2018 г. в 13:39, Anton Vinogradov :
> > >
> > > > Dmitrii,
> > > >
> > > > >> I agree with Nikolay's solution. If no one minds, I'll create
> ticket
> > > for
> > > > >> appropriate changes and recheck issues.
> > > > Do you mean 'copy-paste reduce' ticket or check/fix of all tests with
> > > no-op
> > > > to have a proper handler?
> > > >
> > > > Just want to make sure that copy-paste minimization is not the final
> > > step.
> > > >
> > > > On Thu, Dec 6, 2018 at 1:24 PM Павлухин Иван 
> > > wrote:
> > > >
> > > > > Dmitrii Ryabov,
> > > > >
> > > > > Your comments sounds reasonable to me. Marker base class approach
> > > > > looks good to me so far.
> > > > >
> > > > > P.S. I had even worse name in mind 'StopGaps' =)
> > > > > чт, 6 дек. 2018 г. в 13:08, Dmitrii Ryabov  >:
> > > > > >
> > > > > > Ivan, I think `Workarounds` class isn't good idea, because it
> looks
> > > > like
> > > > > we
> > > > > > create stable workarounds, which will never be fixed.
> > > > > >
> > > > > > I agree with Nikolay's solution. If no one minds, I'll create
> > ticket
> > > > for
> > > > > > appropriate changes and recheck issues.
> > > > > >
> > > > > > чт, 6 дек. 2018 г., 12:17 Anton Vinogradov a...@apache.org:
> > > > > >
> > > > > > > Folks, thank's everyone for solution research.
> > > > > > > I'm ok with Nikolay approach in case that's not a final step.
> > > > > > >
> > > > > > > On Thu, Dec 6, 2018 at 12:11 PM Павлухин Иван <
> > vololo...@gmail.com
> > > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Nikolay,
> > > > > > > >
> > > > > > > > I meant "

Re: [ANNOUNCE] Apache Ignite 2.7.0 Released

2018-12-06 Thread Dmitriy Pavlov
According to previous
https://lists.apache.org/thread.html/4dd772a607ba89b3aaa2084a1ebcd2f4e6a9790decad4d167b2dbd73@%3Cdev.ignite.apache.org%3E
it
seems to be To: annou...@apache.org, dev ,
u...@ignite.apache.org

чт, 6 дек. 2018 г. в 14:30, Nikolay Izhikov :

> Dmitriy.
>
> Yes, we should. Please, tell me mail of this channel.
>
> чт, 6 дек. 2018 г., 14:13 Dmitriy Pavlov dpav...@apache.org:
>
> > Nikolay, thank you for your efforts!
> >
> > Should we announce a release in a special Apache-wide channel?
> >
> > чт, 6 дек. 2018 г. в 10:36, Nikolay Izhikov :
> >
> > > The Apache Ignite Community is pleased to announce the release of
> > > Apache Ignite 2.7.0.
> > >
> > > Apache Ignite [1] is a memory-centric distributed database, caching,
> > > and processing platform for transactional, analytical, and streaming
> > > workloads delivering in-memory speeds at petabyte scale.
> > >
> > > This release introduce several major features and fixes some critical
> > > issues
> > > https://ignite.apache.org/releases/2.7.0/release_notes.html
> > >
> > > Download the latest Ignite version from here:
> > > https://ignite.apache.org/download.cgi
> > >
> > > Please let us know [2] if you encounter any problems.
> > >
> > > Regards,
> > > Nikolay Izhikov on behalf of Apache Ignite community
> > >
> > > [1] https://ignite.apache.org
> > > [2] https://ignite.apache.org/community/resources.html#ask
> > >
> >
>


Re: [ANNOUNCE] Apache Ignite 2.7.0 Released

2018-12-06 Thread Dmitriy Pavlov
Nikolay, thank you for your efforts!

Should we announce a release in a special Apache-wide channel?

чт, 6 дек. 2018 г. в 10:36, Nikolay Izhikov :

> The Apache Ignite Community is pleased to announce the release of
> Apache Ignite 2.7.0.
>
> Apache Ignite [1] is a memory-centric distributed database, caching,
> and processing platform for transactional, analytical, and streaming
> workloads delivering in-memory speeds at petabyte scale.
>
> This release introduce several major features and fixes some critical
> issues
> https://ignite.apache.org/releases/2.7.0/release_notes.html
>
> Download the latest Ignite version from here:
> https://ignite.apache.org/download.cgi
>
> Please let us know [2] if you encounter any problems.
>
> Regards,
> Nikolay Izhikov on behalf of Apache Ignite community
>
> [1] https://ignite.apache.org
> [2] https://ignite.apache.org/community/resources.html#ask
>


Re: Default failure handler was changed for tests

2018-12-06 Thread Dmitriy Pavlov
Nikolay,

Answering your questions a couple of emails ago.

Only one valid reason to avoid NoOp it the risk
- we don't correctly understand test meaning by class name, we don't catch
it's expected flow and
- there is some test which uses NoOp now, but should not.

Any failure in such test included to set of NoOp handlers test can be
hidden and the test will pass for whatever reason, e.g. poor validation of
expected behavior. So suggested analog to this is muted failures, which
runs on TC but don't signal us that something is wrong.

Sincerely,
Dmitriy Pavlov

чт, 6 дек. 2018 г. в 13:53, Dmitriy Pavlov :

> BTW, No-Op or StopNode-FailTest in case of a deep investigation will
> always require to understand what test does and what it tests.
>
> So we can get a positive outcome from this research if we agree to add
> - a small description to each test about the reason for existing of this
> test,
> - what is the expected behavior of the product in the test, and how it is
> checked?
> - failure handler influence, etc.
>
> I still hope Anton will do the first bunch of tests research to
> demonstrate the idea.
>
> чт, 6 дек. 2018 г. в 13:39, Anton Vinogradov :
>
>> Dmitrii,
>>
>> >> I agree with Nikolay's solution. If no one minds, I'll create ticket
>> for
>> >> appropriate changes and recheck issues.
>> Do you mean 'copy-paste reduce' ticket or check/fix of all tests with
>> no-op
>> to have a proper handler?
>>
>> Just want to make sure that copy-paste minimization is not the final step.
>>
>> On Thu, Dec 6, 2018 at 1:24 PM Павлухин Иван  wrote:
>>
>> > Dmitrii Ryabov,
>> >
>> > Your comments sounds reasonable to me. Marker base class approach
>> > looks good to me so far.
>> >
>> > P.S. I had even worse name in mind 'StopGaps' =)
>> > чт, 6 дек. 2018 г. в 13:08, Dmitrii Ryabov :
>> > >
>> > > Ivan, I think `Workarounds` class isn't good idea, because it looks
>> like
>> > we
>> > > create stable workarounds, which will never be fixed.
>> > >
>> > > I agree with Nikolay's solution. If no one minds, I'll create ticket
>> for
>> > > appropriate changes and recheck issues.
>> > >
>> > > чт, 6 дек. 2018 г., 12:17 Anton Vinogradov a...@apache.org:
>> > >
>> > > > Folks, thank's everyone for solution research.
>> > > > I'm ok with Nikolay approach in case that's not a final step.
>> > > >
>> > > > On Thu, Dec 6, 2018 at 12:11 PM Павлухин Иван 
>> > wrote:
>> > > >
>> > > > > Nikolay,
>> > > > >
>> > > > > I meant "not expensive" by "cheap". And I meant that it is good
>> that
>> > > > > it cheap =). And I said it to contrast with "expensive" ~100 tests
>> > > > > investigation. And if we agree (mostly I would like an opinion
>> from
>> > > > > Dmitriy Ryabov as an original author) on a way how to improve the
>> > > > > patch then let's do it.
>> > > > > чт, 6 дек. 2018 г. в 10:41, Nikolay Izhikov > >:
>> > > > > >
>> > > > > > Dmitriy Ryabov, Dmitriy Pavlov, sorry.
>> > > > > >
>> > > > > > Of course it should be "NOT to blame author".
>> > > > > >
>> > > > > > Sorry, one more time.
>> > > > > >
>> > > > > > чт, 6 дек. 2018 г., 10:40 Dmitriy Pavlov dpav...@apache.org:
>> > > > > >
>> > > > > > > I hope you've misprinted here
>> > > > > > > > I'm here to blame the author.
>> > > > > > >
>> > > > > > > We can blame code but never coders.
>> > > > > > >
>> > > > > > > Please see https://discourse.pi-hole.net/faq - has absolutely
>> > > > nothing
>> > > > > in
>> > > > > > > common with Apache Guides, but says the same things. It is a
>> > > > practical
>> > > > > > > necessity to maintain a friendly atmosphere.
>> > > > > > >
>> > > > > > > чт, 6 дек. 2018 г. в 10:31, Nikolay Izhikov <
>> nizhi...@apache.org
>> > >:
>> > > > > > >
>> > > > > > > > Ivan.
>> > > > > > > >
>> > > > > > > > > 1. Accept the patch and

Re: Default failure handler was changed for tests

2018-12-06 Thread Dmitriy Pavlov
BTW, No-Op or StopNode-FailTest in case of a deep investigation will always
require to understand what test does and what it tests.

So we can get a positive outcome from this research if we agree to add
- a small description to each test about the reason for existing of this
test,
- what is the expected behavior of the product in the test, and how it is
checked?
- failure handler influence, etc.

I still hope Anton will do the first bunch of tests research to demonstrate
the idea.

чт, 6 дек. 2018 г. в 13:39, Anton Vinogradov :

> Dmitrii,
>
> >> I agree with Nikolay's solution. If no one minds, I'll create ticket for
> >> appropriate changes and recheck issues.
> Do you mean 'copy-paste reduce' ticket or check/fix of all tests with no-op
> to have a proper handler?
>
> Just want to make sure that copy-paste minimization is not the final step.
>
> On Thu, Dec 6, 2018 at 1:24 PM Павлухин Иван  wrote:
>
> > Dmitrii Ryabov,
> >
> > Your comments sounds reasonable to me. Marker base class approach
> > looks good to me so far.
> >
> > P.S. I had even worse name in mind 'StopGaps' =)
> > чт, 6 дек. 2018 г. в 13:08, Dmitrii Ryabov :
> > >
> > > Ivan, I think `Workarounds` class isn't good idea, because it looks
> like
> > we
> > > create stable workarounds, which will never be fixed.
> > >
> > > I agree with Nikolay's solution. If no one minds, I'll create ticket
> for
> > > appropriate changes and recheck issues.
> > >
> > > чт, 6 дек. 2018 г., 12:17 Anton Vinogradov a...@apache.org:
> > >
> > > > Folks, thank's everyone for solution research.
> > > > I'm ok with Nikolay approach in case that's not a final step.
> > > >
> > > > On Thu, Dec 6, 2018 at 12:11 PM Павлухин Иван 
> > wrote:
> > > >
> > > > > Nikolay,
> > > > >
> > > > > I meant "not expensive" by "cheap". And I meant that it is good
> that
> > > > > it cheap =). And I said it to contrast with "expensive" ~100 tests
> > > > > investigation. And if we agree (mostly I would like an opinion from
> > > > > Dmitriy Ryabov as an original author) on a way how to improve the
> > > > > patch then let's do it.
> > > > > чт, 6 дек. 2018 г. в 10:41, Nikolay Izhikov :
> > > > > >
> > > > > > Dmitriy Ryabov, Dmitriy Pavlov, sorry.
> > > > > >
> > > > > > Of course it should be "NOT to blame author".
> > > > > >
> > > > > > Sorry, one more time.
> > > > > >
> > > > > > чт, 6 дек. 2018 г., 10:40 Dmitriy Pavlov dpav...@apache.org:
> > > > > >
> > > > > > > I hope you've misprinted here
> > > > > > > > I'm here to blame the author.
> > > > > > >
> > > > > > > We can blame code but never coders.
> > > > > > >
> > > > > > > Please see https://discourse.pi-hole.net/faq - has absolutely
> > > > nothing
> > > > > in
> > > > > > > common with Apache Guides, but says the same things. It is a
> > > > practical
> > > > > > > necessity to maintain a friendly atmosphere.
> > > > > > >
> > > > > > > чт, 6 дек. 2018 г. в 10:31, Nikolay Izhikov <
> nizhi...@apache.org
> > >:
> > > > > > >
> > > > > > > > Ivan.
> > > > > > > >
> > > > > > > > > 1. Accept the patch and bring an improvement to Ignite (and
> > > > create
> > > > > a>
> > > > > > > > ticket for further investigation).
> > > > > > > >
> > > > > > > > I support this idea.
> > > > > > > > Do we create the tickets already?
> > > > > > > >
> > > > > > > > > Nikolay's patch [1] suggests a slightly different approach
> > how to
> > > > > the
> > > > > > > > > same thing. And implementing that idea looks like a cheap
> > > > > refactoring.
> > > > > > > >
> > > > > > > > I don't agree with your term "cheap".
> > > > > > > > Do you think reducing copy paste code not worth it?
> > > > > > > >
> > > > > > > > I see a hundreds iss

Re: Default failure handler was changed for tests

2018-12-05 Thread Dmitriy Pavlov
I hope you've misprinted here
> I'm here to blame the author.

We can blame code but never coders.

Please see https://discourse.pi-hole.net/faq - has absolutely nothing in
common with Apache Guides, but says the same things. It is a practical
necessity to maintain a friendly atmosphere.

чт, 6 дек. 2018 г. в 10:31, Nikolay Izhikov :

> Ivan.
>
> > 1. Accept the patch and bring an improvement to Ignite (and create a>
> ticket for further investigation).
>
> I support this idea.
> Do we create the tickets already?
>
> > Nikolay's patch [1] suggests a slightly different approach how to the
> > same thing. And implementing that idea looks like a cheap refactoring.
>
> I don't agree with your term "cheap".
> Do you think reducing copy paste code not worth it?
>
> I see a hundreds issues that bring copypasted code in the product(Ignite
> and others).
> I insist, that we shouldn't accept patches with it.
>
> I'm here to blame the author.
> I want to improve this patch and make it easier to find all places with
> NoOp handler to do the further investigation.
>
> В Чт, 06/12/2018 в 10:19 +0300, Павлухин Иван пишет:
> > Guys,
> >
> > I asked what harm will applying the patch bring I have not got a
> > direct answer. But I think I got some pain points:
> > 1. Anton does not like that reasons why ~100 tests require noop
> > handler are not clear. And might be several problems are covered
> > there.
> > 2. Nikolay suggests some code improvements.
> >
> > Nikolay's patch [1] suggests a slightly different approach how to the
> > same thing. And implementing that idea looks like a cheap refactoring.
> > But the idea of course could be discussed. Straight away I can suggest
> > another slightly different trick [2].
> >
> > Investigating why ~100 tests require noop handler could be costly. So,
> > in that direction I see following options which can happen for sure:
> > 1. Accept the patch and bring an improvement to Ignite (and create a
> > ticket for further investigation).
> > 2. Revert the patch and loose an improvement.
> >
> > One might say that there is an option "Revert the patch and then do it
> > better" but I does not see anything (anyone) what can guarantee it.
> > So, I personally prefer an option 1 against 2 because I believe that
> > it is good if the system "can make a progress".
> >
> > [1] https://github.com/apache/ignite/pull/5584/files
> > [2] https://github.com/apache/ignite/pull/5586/files
> > ср, 5 дек. 2018 г. в 21:22, Nikolay Izhikov :
> > >
> > > Dmitriy.
> > >
> > > > The closest analog to Noop handler is mute of test failure.
> > > > By this commit, we had unmuted (possible) failures in
> ~5-~100=~49900
> > >
> > > tests, and we’re still concerned about style or minor details if no-op
> was
> > > copy-pasted, aren’t we?
> > >
> > > Can you explain this idea a bit more?
> > > I don't understand what is unmuted by discussed commit.
> > >
> > > ср, 5 дек. 2018 г. в 20:40, Nikolay Izhikov :
> > >
> > > > > Thanks, as an improvement to the code, this may be better.
> > > >
> > > > I can prepare a full patch for NoOp handler.
> > > > What do you think?
> > > >
> > > > Anton Vinogradov, do you agree with this approach?
> > > >
> > > >
> > > >
> > > > ср, 5 дек. 2018 г. в 20:33, Dmitriy Pavlov :
> > > >
> > > > > Thanks, as an improvement to the code, this may be better. But
> still, it
> > > > > is
> > > > > not a reason to revert. And Anton mentioned something with better
> > > > > exception
> > > > > handling/logging. Probably we will see an implementation as well.
> > > > >
> > > > > This case here is a big thing related to The Apache Way, - and I'll
> > > > > explain
> > > > > why it makes me switched into fight-mode - until we stop this
> nonsense. If
> > > > > PMCs (at least) are aware of patterns and anti-patterns in the
> community,
> > > > > we will succeed as a project much more as with (only) perfect code.
> > > > >
> > > > > The closest analog to Noop handler is mute of test failure. By this
> > > > > commit,
> > > > > we had unmuted (possible) failures in ~5-~100=~49900 tests,
> and we’re
> > > > > still concerned about style or minor details if no-op was
> copy-pasted,
> > &

[jira] [Created] (IGNITE-10562) [TC Bot] DB contains invalid values for build references ID maps to value with other ID

2018-12-05 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-10562:
---

 Summary: [TC Bot] DB contains invalid values for build references 
ID maps to value with other ID
 Key: IGNITE-10562
 URL: https://issues.apache.org/jira/browse/IGNITE-10562
 Project: Ignite
  Issue Type: Task
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov


{noformat}

http://ci.ignite.apache.org/viewLog.html?buildId=2253798buildTypeId=IgniteTests24Java8_Cloud;
 href="/app/rest/latest/builds/id:2253798">

20181106T114741+0300
20181106T130017+0300
20181106T130156+0300

http://ci.ignite.apache.org/viewLog.html?buildId=2253788buildTypeId=IgniteTests24Java8_BuildApacheIgnite;
 href="/app/rest/latest/builds/id:2253788"/>





http://ci.ignite.apache.org/viewModification.html?modId=837621personal=false;
 href="/app/rest/latest/changes/id:837621"/>




http://ci.ignite.apache.org/viewLog.html?buildId=2253900buildTypeId=IgniteTests24Java8_RunAll;
 href="/app/rest/latest/builds/id:2253900"/>


{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


<    1   2   3   4   5   6   7   8   9   10   >