Re: Hello, Ignite Community
Welcome, Albert! You are in, look forward to your contribution! -- Denis On Thu, Nov 8, 2018 at 10:39 PM Альберт Исхаков wrote: > Hello, Ignite Community! > > My name is Albert Iskhakov. I want to contribute to Apache Ignite and want > to start with this issue [1]. > > JIRA username is aliskhakov. > > [1] https://issues.apache.org/jira/browse/IGNITE-10184 >
Re: Service grid redesign
Vyacheslav,
What are the cases when the service can be redeployed? Affinity, failure,
etc., right. It would be good to list all the cases on the wiki and then
our tech writers will get everything documented.
--
Denis
On Thu, Nov 8, 2018 at 11:06 PM Vyacheslav Daradur
wrote:
> Denis,
>
> Services reassignment process takes into account previous assignments
> to avoid redundant redeployments.
> So, in the described case, ServiceA won't be moved from node1 to node2.
> On Fri, Nov 9, 2018 at 4:41 AM Denis Magda wrote:
> >
> > Vyacheslav,
> >
> > First of all, thanks for archiving this milestone and rolling out these
> new
> > capabilities.
> >
> > Speaking of the topology change events [1], does the new architecture
> avoid
> > a running service redeployment when a new node joins? For instance, let's
> > say I have ServiceA running node1, then node2 joins and I don't want the
> > service to be redeployed to any other node.
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95654584#ServiceGridredesign.Phase1.Implementationdetails.-Topologychange
> >
> > --
> > Denis
> >
> > On Wed, Nov 7, 2018 at 7:04 AM Vyacheslav Daradur
> > wrote:
> >
> > > Dmitriy, I published documentation in wiki:
> > >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95654584
> > >
> > > Thank you!
> > > On Wed, Nov 7, 2018 at 5:10 PM Dmitriy Pavlov
> > > wrote:
> > > >
> > > > Hi I think wiki is better than any attached docs. Could you please
> > > create a
> > > > page?
> > > >
> > > > ср, 7 нояб. 2018 г., 14:39 Vyacheslav Daradur :
> > > >
> > > > > I prepared a description of the implemented solution and attached
> it
> > > > > to the issue [1].
> > > > >
> > > > > This should help during a review. Should I post the document into
> wiki
> > > or
> > > > > IEP?
> > > > >
> > > > > I'd like to ask Ignite's experts review the solution [1] [2],
> please?
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-9607
> > > > > [2] https://github.com/apache/ignite/pull/4434
> > > > > On Wed, Oct 31, 2018 at 5:04 PM Vyacheslav Daradur <
> > > daradu...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > Hi, Igniters! Good news!
> > > > > >
> > > > > > Service Grid Redesign Phase 1 - is in Patch Available now.
> > > > > >
> > > > > > Nikolay Izhikov has reviewed implementation.
> > > > > >
> > > > > > However, we need additional review from other Ignite experts.
> > > > > >
> > > > > > Here is an umbrella ticket [1] and PR [2].
> > > > > >
> > > > > > Could someone step in and do the review?
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-9607
> > > > > > [2] https://github.com/apache/ignite/pull/4434
> > > > > > On Sat, Aug 18, 2018 at 11:44 AM Denis Mekhanikov <
> > > dmekhani...@gmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > > Pavel, could you assist?
> > > > > > >
> > > > > > > Does it make sense for .Net to specify service class name
> instead
> > > of
> > > > > its
> > > > > > > implementation?
> > > > > > >
> > > > > > > I think, it shouldn't be a problem.
> > > > > > >
> > > > > > > Denis
> > > > > > >
> > > > > > > On Sat, Aug 18, 2018, 11:33 Vyacheslav Daradur <
> > > daradu...@gmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > > > I think that the replacement of serialized instance makes
> sense
> > > to me
> > > > > > > > for Java part.
> > > > > > > >
> > > > > > > > But how it should work for .NET client?
> > > > > > > >
> > > > > > > > On Tue, Aug 14, 2018 at 4:07 PM Dmitriy Setrakyan <
> > > > > dsetrak...@apache.org>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > On Tue, Aug 14, 2018 at 6:10 AM, Nikita Amelchev <
> > > > > nsamelc...@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hello, Igniters.
> > > > > > > > > >
> > > > > > > > > > I am working on task [1] that would replace serialized
> > > service's
> > > > > > > > instance
> > > > > > > > > > by service's class name and properties map in
> > > > > {ServiceConfiguration}.
> > > > > > > > > >
> > > > > > > > > > The task describes that we should use
> > > > > > > > > > {String className} + {Map properties}
> instead
> > > > > {Service
> > > > > > > > > > srvc}.
> > > > > > > > > >
> > > > > > > > > > I'd like to clarify the following questions:
> > > > > > > > > >
> > > > > > > > > > 1. What about public methods?
> > > > > > > > > > I suggest to mark them as deprecated and use class name
> of
> > > > > provided
> > > > > > > > > > instance.
> > > > > > > > > > Also to add deploying methods with new parameters:
> > > > > > > > > >
> > > > > > > > > > @Deprecated
> > > > > > > > > > public IgniteInternalFuture
> > > deployNodeSingleton(ClusterGroup
> > > > > prj,
> > > > > > > > > > String
> > > > > > > > > > name, Service svc)
> > > > > > > > > >
> > > > > > > > > > public IgniteInternalFuture
> > > deployNodeSingleton(ClusterGroup
> > > > > prj,
> > > > > > > > > > String
> > > > > > > > > > name, String srvcClsName, Map prop)
[GitHub] ignite pull request #5027: IGNITE-9843 If metadata is not available due to r...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/5027 ---
[GitHub] ignite pull request #5359: IGNITE-10209 some tests in IgniteExamplesMLTestSu...
GitHub user oignatenko opened a pull request: https://github.com/apache/ignite/pull/5359 IGNITE-10209 some tests in IgniteExamplesMLTestSuite fail with FileNotFoundException You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-10209 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5359.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5359 commit 1eeca908a8076a8317947dac8a46845964d1d7ea Author: Oleg Ignatenko Date: 2018-08-23T13:13:28Z IGNITE-9348 ML examples improvements - wip (logging improved) -- verified with diffs overview, executing the examples and studying execution logs commit e50b89c392568ba9b93935c4fa6c7f7f93f5ec6f Author: Oleg Ignatenko Date: 2018-08-23T14:45:57Z Revert "IGNITE-9348 ML examples improvements" This reverts commit 1eeca908a8076a8317947dac8a46845964d1d7ea. commit 474024b4c5bbdb3c0a4ed2f0a66238c8054c6674 Author: Oleg Ignatenko Date: 2018-08-23T14:57:34Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit 9642b233b5701bdad47ebea163079160227c582a Author: Oleg Ignatenko Date: 2018-08-28T14:01:11Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit 7fc16d013ab725d2ff2e1a1b042c983f11d0c4d4 Author: Oleg Ignatenko Date: 2018-08-28T15:13:02Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit d2caba67b156674f051f50faebeafe0871bf0914 Author: Oleg Ignatenko Date: 2018-08-29T13:14:07Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit 16775dff51d71ea68b4a3dea98be552130c493ed Author: Oleg Ignatenko Date: 2018-08-30T09:00:56Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit aedb59929974fe205b949225c1a338c68c60cfc8 Author: Oleg Ignatenko Date: 2018-08-30T09:42:38Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit 39c6482fcdca506aa33011ed21c98060b4a8c68b Author: Oleg Ignatenko Date: 2018-08-30T11:28:05Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit 33b32a2760a6559c78283b927e3191180d8ed9e1 Author: Oleg Ignatenko Date: 2018-08-30T12:31:16Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit 9531028ddd1aef9e95f7e8c8b528086739bbb1b0 Author: Oleg Ignatenko Date: 2018-08-30T14:06:34Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit 28f22c6e2fffcb82717ba5da7be2cfd39715c4e3 Author: Oleg Ignatenko Date: 2018-08-30T16:41:51Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit aacac88db519187413b0fc5ff9d0e55b8f8cff22 Author: Oleg Ignatenko Date: 2018-08-31T10:12:32Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit 897f920dde46022849b13f9fb86dba8e54119a56 Author: Oleg Ignatenko Date: 2018-08-31T13:57:14Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit 114c79e54c1b316006ccc3ff22d20d902f9313df Author: Oleg Ignatenko Date: 2018-08-31T17:39:16Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit fd6b659bb8be1992618ce4ce91f568a0988b3afa Author: Oleg Ignatenko Date: 2018-09-02T06:11:42Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit 6ae6d1d3cf9743d8d466be0330511ddc8589e944 Author: Oleg Ignatenko Date: 2018-09-03T10:27:35Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit e8b27dbd3d0c1cbdb7a7659175f5c7bb447482bf Author: Oleg Ignatenko Date: 2018-09-04T09:49:44Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit 622c82efdd0aa182fadea6b7ffa5d4615521a3f5 Author: Oleg Ignatenko Date: 2018-09-05T10:50:28Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit fb844fe3751e2e8ae87e6b8030b0e4bd659df9c2 Author: Oleg Ignatenko Date: 2018-09-05T11:45:58Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit 480ed78869277d7e32f725ab71bec9621f1ac03a Author: Oleg Ignatenko Date: 2018-09-06T07:52:55Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit c99762498f617c0e98ea3062a43c0b30092166ef Author: Oleg Ignatenko Date: 2018-09-06T14:45:04Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit 2e17175225c72f747d370b5fee96f8be69d6d2cb Author: Oleg Ignatenko Date: 2018-09-06T17:33:54Z Merge branch 'master' of https://github.com/apache/ignite into master-ml commit 9ebcd9a2fe5966b0bf42a95484395867c15d863f Author: Oleg Ignatenko Date: 2018-09-07T13:38:51Z Merge branch
[GitHub] ignite pull request #5227: ignite-9970 Local hostname fix
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/5227 ---
[jira] [Created] (IGNITE-10210) SQL: jdbc thin connected to newly started client, misses "old" tables
Pavel Kuznetsov created IGNITE-10210:
Summary: SQL: jdbc thin connected to newly started client, misses
"old" tables
Key: IGNITE-10210
URL: https://issues.apache.org/jira/browse/IGNITE-10210
Project: Ignite
Issue Type: Bug
Components: jdbc, sql
Affects Versions: 2.6
Reporter: Pavel Kuznetsov
Given cluster.
We are creating table using thin driver.
After that we are starting new client node and connecting to it.
Metadata of this new connection doesn't contain information about this table.
Affected all methods that use
{{org.apache.ignite.internal.processors.query.GridQueryProcessor#types}} under
the hood.
Such as:
1) getPrimaryKeys()
2) getColumns()
3) getSchemas() (if no tables are created getSchemas() returns empty list.)
4) getTables()
5) getIndexes()
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Is it time to move forward to JUnit4 (5)?
Hi Oleg, I can outline some motivating ideas. There is no agreed solutions yet, but junit4 might help us in reusing existing tests for improving MVCC coverage. Another area is managing RunAll execution time with help of some kind of tests parameterization. Regarding concrete tests changes from my experiment [1]. They were changed in order to demonstrate certain capabilities in action. 1. CacheMvccDmlSimpleTest simply demonstrates that it is a valid junit4 test. Test names without "test" prefix make it evident that tests are run by junit4 engine. 2. IgniteCacheCommitDelayTxRecoveryTest shows how a pattern common for our junit3 tests could be improved by junit4. I did not have each test method renaming in mind (but I suppose we cannot avoid marking each test with @Test annotation). And I also have run RunAll after introducing changes in GridAbstractTest [2]. There were several rebuilds because of problems found along the way. I will return to it in order to receive good enough single RunAll result eligible for analysis by human. [1] https://github.com/apache/ignite/pull/5354/files [2] https://mtcga.gridgain.com/pr.html?serverId=apache=IgniteTests24Java8_RunAll=pull/5354/head=Latest пт, 9 нояб. 2018 г. в 16:48, oignatenko : > Hi Ivan, > > That's a very good question. I would say the primary motivation in the task > we discuss is to make the change at the minimal cost and risk. Or, as > stated > in description of IGNITE-10173 "move... gradually, smoothly and safely". > > With this in mind I would say that main expected benefit is laid out at the > top of ticket description: > > Currently unit tests in the project are mix of two versions Junit 3 and > 4. > > This makes it hard to develop and maintain. Making all tests use the same > > version Junit is intended to address this problem. > > Above goal is probably modest but given that this change is expected to > take > minimal effort it looks good enough for me. > > On a more general note I don't expect strong technical benefits of this > change, at least not immediately. This is based on my studies of prior > discussions related to this matter and of the existing tests. I believe > that > if there were any strong and obvious technical benefits we would be long > aware of these and we would migrate to newer Junit long time ago. > > Observing that migration hasn't happened - despite JUnit 3 being obsolete > for over 10 years now, and despite many more fundamental changes that > successfully made it to Ignite in past years - it looks natural to assume > that there are just no technical reasons that would be strong enough to > justify investing much effort in the migration. > > --- > > Now about your experiment at ignite/pull/5354, I briefly skimmed it and > here > are some initial impressions. > > First the bad news, some parts look indeed "insane" to me. I am talking > specifically about renames in CacheMvccDmlSimpleTest and changes to test > design in IgniteCacheCommitDelayTxRecoveryTest. I believe that these are > wildly out of scope of the discussed change because I firmly want to keep > it > limited to absolute minimum of changes (ideally only annotations with > involved imports). > > This is because there are just too many tests to (manually) change. My > rough > estimate is there about 7,000 (seven thousands!) test cases to change in > core module alone; and I guess there are few thousands more in other parts > of the project. Doing any extra changes to these tests makes things much > more complicated, effort consuming and risky than was initially supposed. > > On a brighter side, changes made to GridAbstractTest look worth further > studying. At this point I still prefer to keep this class unchanged > (because > this gives us a bulletproof guarantee against regressions in all prior > tests) but this may change after we do "pilot" changes to examples tests > (IGNITE-10174). > > These pilot changes will help us estimate with concrete code how much > effort > could be involved in maintaining "twin" classes in sync (you are absolutely > right pointing that this is concerning). If it turns out too complicated to > sync we would probably switch to something like you have done in pull/5354 > > regards, Oleg > > > Павлухин Иван wrote > > Hi Oleg, > > > > Migrating to Junit 4 sounds as great idea. I believe everybody > > is quite surprised when finds Junit 3 in Ignite. > > But for me personally it is good to understand what problem we > > are trying to solve? What benefits will Junit 4 give us? What are > > the most painful moments with current testing framework? > > (I my mind I draw a proposal which contains sections "Motivation", > > "Alternatives", etc.) > > > > Also I must say that I made some experiments related to Junit4 > > migration. And from the first glance it seems that the most > > important Ignite test framework classes contain a huge amount of > > utility methods and a little amount of code related to a test lifecycle. > > These classes are
[jira] [Created] (IGNITE-10209) some tests in IgniteExamplesMLTestSuite fail with FileNotFoundException at SandboxMLCache#fillCacheWith(MLSandboxDatasets)
Oleg Ignatenko created IGNITE-10209:
---
Summary: some tests in IgniteExamplesMLTestSuite fail with
FileNotFoundException at SandboxMLCache#fillCacheWith(MLSandboxDatasets)
Key: IGNITE-10209
URL: https://issues.apache.org/jira/browse/IGNITE-10209
Project: Ignite
Issue Type: Bug
Components: ml
Affects Versions: 2.8
Reporter: Oleg Ignatenko
Tests for examples that use {{SandboxMLCache#fillCacheWith(MLSandboxDatasets)}}
fail with FNFE:
{noformat}
[ERROR] Tests run: 34, Failures: 0, Errors: 13, Skipped: 0, Time elapsed:
638.752 s <<< FAILURE! - in
org.apache.ignite.testsuites.IgniteExamplesMLTestSuite
[ERROR]
testExample(org.apache.ignite.examples.ml.KMeansClusterizationExampleSelfName)
Time elapsed: 21.701 s <<< ERROR!
java.io.FileNotFoundException:
examples\src\main\resources\datasets\two_classed_iris.csv (Системе не удается
найти указанный путь)
at
org.apache.ignite.examples.ml.KMeansClusterizationExampleSelfName.testExample(KMeansClusterizationExampleSelfName.java)
[ERROR]
testExample(org.apache.ignite.examples.ml.KNNClassificationExampleSelfName)
Time elapsed: 13.81 s <<< ERROR!
java.io.FileNotFoundException: examples\src\main\resources\datasets\iris.txt
(Системе не удается найти указанный путь)
at
org.apache.ignite.examples.ml.KNNClassificationExampleSelfName.testExample(KNNClassificationExampleSelfName.java)
[ERROR] testExample(org.apache.ignite.examples.ml.KNNRegressionExampleSelfName)
Time elapsed: 13.633 s <<< ERROR!
java.io.FileNotFoundException:
examples\src\main\resources\datasets\cleared_machines.csv (Системе не удается
найти указанный путь)
at
org.apache.ignite.examples.ml.KNNRegressionExampleSelfName.testExample(KNNRegressionExampleSelfName.java)
[ERROR]
testExample(org.apache.ignite.examples.ml.LinearRegressionLSQRTrainerExampleSelfName)
Time elapsed: 13.71 s <<< ERROR!
java.io.FileNotFoundException:
examples\src\main\resources\datasets\mortalitydata.csv (Системе не удается
найти указанный путь)
at
org.apache.ignite.examples.ml.LinearRegressionLSQRTrainerExampleSelfName.testExample(LinearRegressionLSQRTrainerExampleSelfName.java)
[ERROR]
testExample(org.apache.ignite.examples.ml.LinearRegressionLSQRTrainerWithMinMaxScalerExampleSelfName)
Time elapsed: 13.728 s <<< ERROR!
java.io.FileNotFoundException:
examples\src\main\resources\datasets\mortalitydata.csv (Системе не удается
найти указанный путь)
at
org.apache.ignite.examples.ml.LinearRegressionLSQRTrainerWithMinMaxScalerExampleSelfName.testExample(LinearRegressionLSQRTrainerWithMinMaxScalerExampleSelfName.java)
[ERROR]
testExample(org.apache.ignite.examples.ml.LinearRegressionSGDTrainerExampleSelfName)
Time elapsed: 13.874 s <<< ERROR!
java.io.FileNotFoundException:
examples\src\main\resources\datasets\mortalitydata.csv (Системе не удается
найти указанный путь)
at
org.apache.ignite.examples.ml.LinearRegressionSGDTrainerExampleSelfName.testExample(LinearRegressionSGDTrainerExampleSelfName.java)
[ERROR]
testExample(org.apache.ignite.examples.ml.LogisticRegressionSGDTrainerExampleSelfName)
Time elapsed: 13.694 s <<< ERROR!
java.io.FileNotFoundException:
examples\src\main\resources\datasets\two_classed_iris.csv (Системе не удается
найти указанный путь)
at
org.apache.ignite.examples.ml.LogisticRegressionSGDTrainerExampleSelfName.testExample(LogisticRegressionSGDTrainerExampleSelfName.java)
[ERROR]
testExample(org.apache.ignite.examples.ml.LogRegressionMultiClassClassificationExampleSelfName)
Time elapsed: 13.823 s <<< ERROR!
java.io.FileNotFoundException:
examples\src\main\resources\datasets\glass_identification.csv (Системе не
удается найти указанный путь)
at
org.apache.ignite.examples.ml.LogRegressionMultiClassClassificationExampleSelfName.testExample(LogRegressionMultiClassClassificationExampleSelfName.java)
[ERROR]
testExample(org.apache.ignite.examples.ml.TrainTestDatasetSplitterExampleSelfName)
Time elapsed: 12.875 s <<< ERROR!
java.io.FileNotFoundException:
examples\src\main\resources\datasets\mortalitydata.csv (Системе не удается
найти указанный путь)
at
org.apache.ignite.examples.ml.TrainTestDatasetSplitterExampleSelfName.testExample(TrainTestDatasetSplitterExampleSelfName.java)
[ERROR]
testExample(org.apache.ignite.examples.ml.SVMBinaryClassificationExampleSelfName)
Time elapsed: 12.795 s <<< ERROR!
java.io.FileNotFoundException:
examples\src\main\resources\datasets\two_classed_iris.csv (Системе не удается
найти указанный путь)
at
org.apache.ignite.examples.ml.SVMBinaryClassificationExampleSelfName.testExample(SVMBinaryClassificationExampleSelfName.java)
[ERROR]
testExample(org.apache.ignite.examples.ml.SVMMultiClassClassificationExampleSelfName)
Time elapsed: 12.948 s <<< ERROR!
java.io.FileNotFoundException:
[GitHub] sergey-chugunov-1985 opened a new pull request #64: New Page for TC Bot: Long Running Tests Report
sergey-chugunov-1985 opened a new pull request #64: New Page for TC Bot: Long Running Tests Report URL: https://github.com/apache/ignite-teamcity-bot/pull/64 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (IGNITE-10208) Verify list of tests after migration to Junit 4 against some prior reference (follow-up to IGNITE-10177)
Oleg Ignatenko created IGNITE-10208:
---
Summary: Verify list of tests after migration to Junit 4 against
some prior reference (follow-up to IGNITE-10177)
Key: IGNITE-10208
URL: https://issues.apache.org/jira/browse/IGNITE-10208
Project: Ignite
Issue Type: Sub-task
Reporter: Oleg Ignatenko
Migration from Junit 3 to 4 involves manually adding {{@Test}} annotation to
existing test cases. Since Ignite contains many thousands test cases there is a
substantial risk that we may miss some of these in such a transition.
In order to mitigate the risk, suggest to do a check after completion of
IGNITE-10177 - somehow generate list of tests in the project and compare it
against similar list of tests of some "known good" prior project version (at
this point, 2.7 release branch looks like a good candidate for such a
reference).
If comparison shows that some test cases from older version are missed in newer
one, open a ticket to investigate that.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Is it time to move forward to JUnit4 (5)?
Hi Ivan, That's a very good question. I would say the primary motivation in the task we discuss is to make the change at the minimal cost and risk. Or, as stated in description of IGNITE-10173 "move... gradually, smoothly and safely". With this in mind I would say that main expected benefit is laid out at the top of ticket description: > Currently unit tests in the project are mix of two versions Junit 3 and 4. > This makes it hard to develop and maintain. Making all tests use the same > version Junit is intended to address this problem. Above goal is probably modest but given that this change is expected to take minimal effort it looks good enough for me. On a more general note I don't expect strong technical benefits of this change, at least not immediately. This is based on my studies of prior discussions related to this matter and of the existing tests. I believe that if there were any strong and obvious technical benefits we would be long aware of these and we would migrate to newer Junit long time ago. Observing that migration hasn't happened - despite JUnit 3 being obsolete for over 10 years now, and despite many more fundamental changes that successfully made it to Ignite in past years - it looks natural to assume that there are just no technical reasons that would be strong enough to justify investing much effort in the migration. --- Now about your experiment at ignite/pull/5354, I briefly skimmed it and here are some initial impressions. First the bad news, some parts look indeed "insane" to me. I am talking specifically about renames in CacheMvccDmlSimpleTest and changes to test design in IgniteCacheCommitDelayTxRecoveryTest. I believe that these are wildly out of scope of the discussed change because I firmly want to keep it limited to absolute minimum of changes (ideally only annotations with involved imports). This is because there are just too many tests to (manually) change. My rough estimate is there about 7,000 (seven thousands!) test cases to change in core module alone; and I guess there are few thousands more in other parts of the project. Doing any extra changes to these tests makes things much more complicated, effort consuming and risky than was initially supposed. On a brighter side, changes made to GridAbstractTest look worth further studying. At this point I still prefer to keep this class unchanged (because this gives us a bulletproof guarantee against regressions in all prior tests) but this may change after we do "pilot" changes to examples tests (IGNITE-10174). These pilot changes will help us estimate with concrete code how much effort could be involved in maintaining "twin" classes in sync (you are absolutely right pointing that this is concerning). If it turns out too complicated to sync we would probably switch to something like you have done in pull/5354 regards, Oleg Павлухин Иван wrote > Hi Oleg, > > Migrating to Junit 4 sounds as great idea. I believe everybody > is quite surprised when finds Junit 3 in Ignite. > But for me personally it is good to understand what problem we > are trying to solve? What benefits will Junit 4 give us? What are > the most painful moments with current testing framework? > (I my mind I draw a proposal which contains sections "Motivation", > "Alternatives", etc.) > > Also I must say that I made some experiments related to Junit4 > migration. And from the first glance it seems that the most > important Ignite test framework classes contain a huge amount of > utility methods and a little amount of code related to a test lifecycle. > These classes are GridAbstractTest and GridCommonAbstractTest. > Both are quite thick, but as already said it is mostly utility code > which could be extracted. I would be great to come to very thin base > test classes (or even to baseclass-less tests at all) and include all > kind of utilities without inheritance. > > But massive refactoring perhaps is more for future. The main point > here is that Ignite test framework is not complex at all and could be > easily adopted to Junit 4. > > Regarding twin classes. How to keep them in sync? If the migration > process will be paused halfway it could make things complicated in > future. > > Additionally, I have a wild idea how to marry junit 3 and 4 without > introducing twins. The idea looks ugly from OOP perspective, but > here it is. We can simply annotate overridden Junit 3 setUp and > tearDown methods with @Before and @After annotations. And > use runTest overridden method for Junit 3 and TestRule for Junit 4 > in order to run test methods in newly started thread. > > For all details you can see an experiment on github [1]. Among them > are ticks with @RunWith annotation making possible to run a test > inherited from TestCase as Junit 4 test. Still, I might have missed > something. > > Does it sound sane anyhow? > > [1] https://github.com/apache/ignite/pull/5354/files > > > ср, 7 нояб. 2018 г. в 19:57, oignatenko > oignatenko@ > : > >>
[jira] [Created] (IGNITE-10206) Allow specifying query parallelism in CREATE TABLE's WITH "" clause
Ilya Kasnacheev created IGNITE-10206:
Summary: Allow specifying query parallelism in CREATE TABLE's WITH
"" clause
Key: IGNITE-10206
URL: https://issues.apache.org/jira/browse/IGNITE-10206
Project: Ignite
Issue Type: Improvement
Components: sql
Affects Versions: 2.6
Reporter: Ilya Kasnacheev
As in
{code}
CREATE TABLE foo (id long primary key, val varchar) WITH "backups=1,
query_parallelism=4";
{code}
Currently it is possible to specify e.g. backups but not query_parallelism.
This leads to necessity to specify cache template for such tables, which is
cumbersome.
Moreover, people may ignore this option outright and get worse performance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (IGNITE-10207) Missed loss policy checks
Dmitriy Govorukhin created IGNITE-10207: --- Summary: Missed loss policy checks Key: IGNITE-10207 URL: https://issues.apache.org/jira/browse/IGNITE-10207 Project: Ignite Issue Type: Bug Reporter: Dmitriy Govorukhin In some cases (client reconnect, new client join, etc) PartitionLossPolicy may incorrectly validate operation. Return null for READ_ONLY_SAFE for loss partition. To reproduce run CacheResultIsNotNullOnPartitionLossTest (1000 times ) with random node stop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #5021: IGNITE-9930: Split Zookeeper Discovery 2 onto sev...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/5021 ---
[GitHub] ignite pull request #5358: IGNITE-10073 Ignite NuGet package without embedde...
GitHub user kukushal opened a pull request: https://github.com/apache/ignite/pull/5358 IGNITE-10073 Ignite NuGet package without embedded Ignite JARs You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-10073 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5358.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5358 commit a9897350f415567caab29ef7514d74b2a5cd8e92 Author: Alexey Kukushkin Date: 2018-11-05T23:18:22Z IGNITE-10073 Ignite NuGet package without embedded Ignite JARs ---
[GitHub] ignite pull request #5330: IGNITE-9870 GridDhtPartitionsFullMessage#prepareM...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/5330 ---
[GitHub] ignite pull request #5013: IGNITE-9909 Extract file handle from WAL manager
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/5013 ---
[jira] [Created] (IGNITE-10205) add to utility command - ./control.sh --cache idle_verify --dump abbility to exclude cache from output file
ARomantsov created IGNITE-10205: --- Summary: add to utility command - ./control.sh --cache idle_verify --dump abbility to exclude cache from output file Key: IGNITE-10205 URL: https://issues.apache.org/jira/browse/IGNITE-10205 Project: Ignite Issue Type: Improvement Reporter: ARomantsov -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #5357: IGNITE-10167: MVCC-compatible IgniteCache.localEn...
GitHub user pavlukhin opened a pull request: https://github.com/apache/ignite/pull/5357 IGNITE-10167: MVCC-compatible IgniteCache.localEntries You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-10167 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5357.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5357 commit 0359a3ad885c869a6d492af23e84fa8507bac848 Author: ipavlukhin Date: 2018-11-02T15:32:56Z mvcc-compatible peek draft commit 514b793910a88ab4718829d1f77055f45740e349 Author: ipavlukhin Date: 2018-11-06T11:09:40Z do "peek" operation similarly to SQL query (use mvcc tracker) commit 751048bed4012be6608bb01fdc42a2ab58ef5395 Author: ipavlukhin Date: 2018-11-06T11:13:23Z add todo commit 067782a81e7851f3c84455fdfb281a537fd142a3 Author: ipavlukhin Date: 2018-11-06T11:28:25Z Merge branch 'master' into mvcc-peek commit 1b2a19969039c9a53da9de9b5802c7aa751a6a7c Author: ipavlukhin Date: 2018-11-06T11:29:56Z cleanup commit c1f774ee78b46587cb42010782b8b435df640159 Author: ipavlukhin Date: 2018-11-06T14:02:45Z use special MAX snapshot which can see all committed versions for mvccPeek commit 1d46b4b20f49dd2cdacebc45b993b4972fe2c533 Author: ipavlukhin Date: 2018-11-06T17:45:33Z tune MVCC_MAX_SNAPSHOT commit 35b822e87b80ba428dec23d4068cead3ea306988 Author: ipavlukhin Date: 2018-11-06T17:47:30Z dummy for a code style commit c90c3f34c0b7b732f139024e24e42716c03e02ac Author: ipavlukhin Date: 2018-11-07T06:28:20Z move peek test to core module commit e88d70910ef43f0c7803928d6732e0b6ea0bcaac Author: ipavlukhin Date: 2018-11-07T06:31:17Z remove test checking that peek is not supported for mvcc commit 61b019bbf7a270d4f80b4acfeb9b1c113a9fbef4 Author: ipavlukhin Date: 2018-11-07T08:45:25Z * call checkObsolete() in mvccPeek; * javadoc for mvccPeek. commit c4e2807c53ce545d46f6b0161eaa7c484487c600 Author: ipavlukhin Date: 2018-11-07T09:31:54Z update IgniteCache.peek javadoc commit 0db14098fe2e36cc74c1a0a35b0eefb4a709453d Author: ipavlukhin Date: 2018-11-07T09:38:27Z minor javadoc fix commit 63e8de61963b10b3c71800275d0f44c8169550a6 Author: ipavlukhin Date: 2018-11-07T09:43:58Z comments for special mvcc constants commit b57533135cace90828caa1cea0d964e709876213 Author: ipavlukhin Date: 2018-11-07T10:03:35Z rework MvccCachePeekTest to stop grid only once commit 25653c9b255294268722e2e62dfeb422b6a8c21b Author: ipavlukhin Date: 2018-11-07T10:41:20Z Merge branch 'master' into mvcc-peek commit edb496b9489615852d2c48960d79b4d8325e375a Author: ipavlukhin Date: 2018-11-07T13:28:22Z adopt existing peek tests and add them to MVCC suite commit 16219757547e0fe14a96622ad9d53cce1c7ff331 Author: ipavlukhin Date: 2018-11-08T13:27:36Z Merge branch 'master' into mvcc-peek commit 977f13a2d39e0f4785e06ea4775d25a3e45ce264 Author: ipavlukhin Date: 2018-11-08T16:08:57Z IgniteCache.localEntries draft commit 5d71df8da19211aedc428bf96ea95eb2359b4789 Author: ipavlukhin Date: 2018-11-09T10:59:27Z use MvccUtils.MVCC_MAX_SNAPSHOT for "localEntries" implementation commit eea412024fcb58072c7ddfcc92a9e36ae28ac943 Author: ipavlukhin Date: 2018-11-09T11:03:29Z minor ---
[GitHub] ignite pull request #5356: IGNITE-10196 Change scope of kafka-clients-*-test...
GitHub user Max-Pudov opened a pull request: https://github.com/apache/ignite/pull/5356 IGNITE-10196 Change scope of kafka-clients-*-test dependency to test You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-10196 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5356.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5356 commit 0b6a30d71004b0746fe9290c606b10a26062d529 Author: Max-Pudov Date: 2018-11-09T11:19:05Z IGNITE-10196 Change scope of kafka-clients-*-test dependency to test ---
[jira] [Created] (IGNITE-10204) Web Console: Rename "Allow non-collocated joins" checkbox
Vica Abramova created IGNITE-10204: -- Summary: Web Console: Rename "Allow non-collocated joins" checkbox Key: IGNITE-10204 URL: https://issues.apache.org/jira/browse/IGNITE-10204 Project: Ignite Issue Type: Improvement Components: UI, wizards Reporter: Vica Abramova Assignee: Alexey Kuznetsov Rename checkbox "Allow non-collocated joins" => "Allow distributed joins" In the Web Console's query tab, it says "Allow non-collocated joins". I propose that this be renamed to "Allow distributed joins". I say that because in your documentation it refers to it as distributed joins. Inconsistent naming caused some confusion to our development teams. Consistent naming can solve this. https://apacheignite-sql.readme.io/docs/distributed-joins Attached is screenshot of what I get from google when search for non-collocated joins which seems to suggests that distrbuted joins is the preferred terminology. https://www.google.com/search?q=GridGain+non-collocated+joins -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #5316: IGNITE-10151 Fix BinaryMarshallerSelfTest#testDef...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/5316 ---
[GitHub] ignite pull request #5056: IGNITE-9976
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/5056 ---
[GitHub] ignite pull request #4314: IGNITE-8619 Remote node could not start in ssh co...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4314 ---
[GitHub] ololo3000 opened a new pull request #63: IGNITE-10181 [Tc Bot] Add fail-rate handling for test's page
ololo3000 opened a new pull request #63: IGNITE-10181 [Tc Bot] Add fail-rate handling for test's page URL: https://github.com/apache/ignite-teamcity-bot/pull/63 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (IGNITE-10203) [TC Bot] Support for alternative configurations for PR testing
Nikolai Kulagin created IGNITE-10203: Summary: [TC Bot] Support for alternative configurations for PR testing Key: IGNITE-10203 URL: https://issues.apache.org/jira/browse/IGNITE-10203 Project: Ignite Issue Type: Task Reporter: Nikolai Kulagin Assignee: Nikolai Kulagin Support for alternative configurations for PR testing (for example, IgniteTests24Java8_RunMl) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ignite pull request #5241: IGNITE-10128 Race in read\write cache configurati...
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/5241 ---
Re: Is it time to move forward to JUnit4 (5)?
Hi Vyacheslav, GridCommonAbstractTest is expected to change too. As pointed in IGNITE-10173 our approach involves all the subclasses of GridAbstractTest - including but not limited to one you mentioned (GridCommonAbstractTest). regards, Oleg -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
[jira] [Created] (IGNITE-10202) ML: Create a POC of TensorFlow model inference in Java
Anton Dmitriev created IGNITE-10202: --- Summary: ML: Create a POC of TensorFlow model inference in Java Key: IGNITE-10202 URL: https://issues.apache.org/jira/browse/IGNITE-10202 Project: Ignite Issue Type: Sub-task Components: ml Affects Versions: 2.8 Reporter: Anton Dmitriev Assignee: Anton Dmitriev Fix For: 2.8 TensorFlow models are represented internally as computation graphs that are saved as a Protobuf objects with some kind of metadata. TensorFlow provides a good API for Python that allows to work with them, but Java API is very poor. TensorFlow allows to import computation graph in Java API and manipulate it using low-level API and it looks like a one way to make model inference in Java. Another way is based on the fact that the most popular way to build a TensorFlow model is using of the Keras API. It allows to save model in _.h5_ format that can be imported into Java using Deeplearning4J. So, the goal of this task is to *prepare a POC that demonstrates*: * How to import TensorFlow model into Java and make inference using TensorFlow Java API. * How to import TensorFlow model into Java and make inference using Deeplearning4J. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10201) ML: TensorFlow model inference on Apache Ignite
Anton Dmitriev created IGNITE-10201: --- Summary: ML: TensorFlow model inference on Apache Ignite Key: IGNITE-10201 URL: https://issues.apache.org/jira/browse/IGNITE-10201 Project: Ignite Issue Type: New Feature Components: ml Affects Versions: 2.8 Reporter: Anton Dmitriev Assignee: Anton Dmitriev Fix For: 2.8 Machine learning pipeline consists of two stages: *model training* and *model inference* _(model training is a process of training a model using existing data with known target values, model inference is a process of making predictions on a new data using trained model)._ It's important that a model can be trained in one environment/system and after that is used for inference in another. A trained model is an immutable object without any side-effects (a pure mathematical function in math language). As result of that, an inference process has an excellent linear scalability characteristics because different inferences can be done in parallel in different threads or on different nodes. The goal of "TensorFlow model inference on Apache Ignite" is to allow user to easily import pre-trained TensorFlow model into Apache Ignite, distribute it across nodes in a cluster, provide a common interface to call these models to make inference and finally perform load balancing so that all node resources are properly utilized. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
