Re: [DISCUSSION] Apache Ignite Release 2.10 (time, scope, manager)
Hi Denis, Maxim, I'd like to clarify that I'm pretty sure to complete the tasks below before the voting date: https://issues.apache.org/jira/browse/IGNITE-13659 https://issues.apache.org/jira/browse/IGNITE-13796 I'll start to work on them right after the long Russian NY holidays. But I'm not quite sure that we'll meet the deadline with other tickets that should be on the list. So we should probably allocate more time for this work. I'd be glad to get your opinion on this. Regards, Nikita сб, 19 дек. 2020 г. в 11:39, Maxim Muzafarov : > Denis, Nikita > > Yes, the list of important documentation issues will be ready prior to > the scope freeze date. I'll prepare it. > > I mean if we want to start working right now on some documentation > task we can choose any important from those list of opened tasks. > > On Fri, 18 Dec 2020 at 19:55, Denis Magda wrote: > > > > Maxim, > > > > Presently, there are 70 documentation tickets. Probably, these are all > the > > tickets that we are moving from a release to a release. Could you help to > > prepare a list of 2.10-specific tickets (features, enhancements, required > > updates due to a change in the behavior)? As before, we can track these > > important 2.10-specific tickets under the "Documentation tasks for > > important tasks" features: > > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.10#ApacheIgnite2.10-Documentationtasksforimportantfeatures > > > > > > - > > Denis > > > > > > On Fri, Dec 18, 2020 at 8:14 AM Maxim Muzafarov > wrote: > > > > > Hello Nikita, > > > > > > Thanks for your support. > > > > > > You can find the 2.10 related documentation issues on the release wiki > > > page [1]. Currently, some of the important issues don't have the right > > > priorities, so it's better to sort them first. > > > > > > As for the time estimates, we've previously agreed that documentation > > > should be ready prior to the release voting date. You can find the > > > actual release dates here [2]. > > > > > > [1] > > > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.10#ApacheIgnite2.10-Allunresolveddocumentationtasks > > > [2] > > > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.10#ApacheIgnite2.10-Releasephases > > > > > > On Fri, 18 Dec 2020 at 18:48, Никита Сафонов < > vlasovpavel2...@gmail.com> > > > wrote: > > > > > > > > Hi guys, > > > > > > > > I'd be more than happy to support the 2.10 release and contribute to > the > > > > documentation. > > > > So could you please give me a scope of documentation tasks for 2.10 > with > > > > the approximate time estimates? > > > > > > > > Regards, > > > > Nikita > > > > > > > > пн, 7 дек. 2020 г. в 16:23, Alexey Zinoviev >: > > > > > > > > > Support it > > > > > > > > > > пн, 7 дек. 2020 г. в 14:03, Maxim Muzafarov : > > > > > > > > > > > Folks, > > > > > > > > > > > > > > > > > > I propose to shift a bit the release date since 2.9.1 has not > > > released > > > > > yet. > > > > > > > > > > > > > > > > > > Scope Freeze: December 28, 2020 > > > > > > Code Freeze: December 28, 2020 > > > > > > Voting Date: January 25, 2021 > > > > > > Release Date: February 1, 2021 > > > > > > > > > > > > WDYT? > > > > > > > > > > > > On Tue, 17 Nov 2020 at 12:59, Maxim Muzafarov > > > > wrote: > > > > > > > > > > > > > > Folks, > > > > > > > > > > > > > > I've marked all the valuable features of 2.10 release with an > > > > > > > appropriate label. Please, let me know if I've missed anything. > > > > > > > > > > > > > > Here is the link to the JIRA: > > > > > > > https://s.apache.org/1hyfu > > > > > > > > > > > > > > > > > > > > > On Thu, 12 Nov 2020 at 12:28, Anton Vinogradov > > > wrote: > > > > > > > > > > > > > > > > > - Distributed environment tests [2] (Nikolay Izhikov, Anton > > > > > > > > > Vinogradov, Ivan Daschinskiy) > > > > > > > > > Will all related issues be included in the 2.10? > > > > > > > > We're at the final preparations phase. Going to start the > merge > > > > > > discussion > > > > > > > > soon. > > > > > > > > > > > > > > > > On Wed, Nov 11, 2020 at 4:48 PM Alexey Zinoviev < > > > > > > zaleslaw@gmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Yes, Model Export/Import for ML will be finished, now it's > > > ready, > > > > > > but under > > > > > > > > > testing on my side. > > > > > > > > > > > > > > > > > > ср, 11 нояб. 2020 г. в 16:08, Nikolay Izhikov < > > > nizhi...@apache.org > > > > > >: > > > > > > > > > > > > > > > > > > > I suggest to include CDC feature to the 2.10 scope. > > > > > > > > > > > > > > > > > > > > > 11 нояб. 2020 г., в 16:06, Maxim Muzafarov < > > > mmu...@apache.org> > > > > > > > > > > написал(а): > > > > > > > > > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you clarify the issue statuses according to the > Apache > > > > > Ignite > > > > > > > > > > > Roadmap [1] and the 2.10 Apache Ignite Release? Here > are > > > some > > > > > >
Re: [DISCUSS] Missed (non-suited) tests
Hi! Ilya thanks for the reply! I agree that it's a valid case when a test is part of multiple suites in different modules. But it is definitely a bug that the test is written in a module where it can't be run at all and aimed to run within different modules (core tests in core that require h2). I propose to fix this issue. I'm going to check all such tests and move them to the right module. As I can see there are about 100 such test classes, but I hope that most of them follow only a few patterns. On Fri, Dec 25, 2020 at 2:58 PM Ivan Daschinsky wrote: > Hi! > >> I'm not sure that we should assume every test is only run from one test > suite. One test may be run from different test suites located in different > modules. > Agree. For example, if we introduce this limitation, zk suites will be > broken. > > > пт, 25 дек. 2020 г. в 14:48, Ilya Kasnacheev : > > > Hello! > > > > Sorry for the long wait. > > > > I'm not sure that we should assume every test is only run from one test > > suite. One test may be run from different test suites located in > different > > modules. > > > > I wonder if we can drop this requirement, check all the modules > > transitively for used/unused tests. > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > ср, 2 дек. 2020 г. в 18:23, Max Timonin : > > > > > Hi, > > > > > > I don't think so. It looks like a bug that tests fail if one runs them > > > within their module (actually, what is the goal of this test?). The > check > > > showed us this problem, there is no need to fix the check. > > > > > > Currently I see two ways: > > > 1. Find the right module for every misplaced test. There are 104 tests, > > > maybe just move them all to the target module? If TeamCity runs them > > within > > > the indexing module only is there a reason to have a test in the core > > > module at all? > > > 2. Back to my previous proposal - create fake suites within a module, > > then > > > replace test classes in a target suite with the single class of the > fake > > > suite. > > > > > > > > > > > > On Wed, Dec 2, 2020 at 5:38 PM Ilya Kasnacheev < > > ilya.kasnach...@gmail.com> > > > wrote: > > > > > > > Hello! > > > > > > > > I think this means that we should abandon the plan of moving tests > > > between > > > > suites, and that your tool has to understand the dependency graph > > > > between modules' tests when assessing what's included and what's not. > > > > > > > > Regards, > > > > -- > > > > Ilya Kasnacheev > > > > > > > > > > > > ср, 2 дек. 2020 г. в 15:56, Max Timonin : > > > > > > > > > Hi, Ilya! > > > > > > > > > > I've checked testsuites. There is an issue. For example > > > > > *IgniteBinaryCacheQueryTestSuite* suite is now in 2 modules: > > > ignite-core, > > > > > ignite-indexing. On TeamCity it runs by "Query 1" suite. Simplified > > > maven > > > > > command for the suite is > > > > > > > > > > mvn -DtestIgniteBinaryCacheQueryTestSuite -am -pl :ignite-indexing > > > > > surefire:test > > > > > > > > > > Sequence of actions is: > > > > > 1. Find modules dependencies (*-am* flag): ignite-tools, > ignite-core; > > > > > 2. Run the test command for every module. In this step the maven > > tries > > > to > > > > > find the specified test for every module. This is good news, so we > > > don't > > > > > need to create new TeamCity suites for such splitted suites. > > > > > > > > > > But the run performs within the current module classpath, so for > the > > > core > > > > > module the test suite fails with error "Add module > 'ignite-indexing' > > to > > > > the > > > > > classpath of all Ignite nodes". Maven can't resolve it. > > > > > > > > > > The only way to work with it is to specify additional classpath > > > elements > > > > > for tests with setting > > > > *-Dmaven.test.additionalClasspath=/path/to/m2/jar*. > > > > > I did it by filling MAVEN_OPTS with the setting. Please check the > job > > > > > parameters [1]. After that the core module part ran successfully. > It > > > > means > > > > > for every TC suite that runs such splitted suite we need to set the > > > > > setting. What do you think, is it a valid way to handle the issue? > If > > > > there > > > > > are no objections, I will check other such suites. > > > > > > > > > > Also to mention there, the work directory contains a *repository/* > > > folder > > > > > with all required .jars. But usage of this path in the setting > didn't > > > > help. > > > > > I'm not sure, but I think it's an issue due to usage of > Classworlds. > > > So, > > > > > using dependency from .m2 is the only way. > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > https://ci.ignite.apache.org/viewLog.html?buildId=5770727=IgniteTests24Java8_Queries1=buildParameters > > > > > > > > > > > > > > > > > > > > On Fri, Nov 27, 2020 at 3:55 PM Max Timonin < > timonin.ma...@gmail.com > > > > > > > > wrote: > > > > > > > > > > > Sure, I'll do that. > > > > > > > > > > > > On Fri, Nov 27, 2020 at 2:00 PM Ilya Kasnacheev < > > > > >
Re: [DISCUSSION] Apache Ignite Release 2.10 (time, scope, manager)
Igniters, I'm going to create the `ignite-2.10` release branch on Monday 28-th December. Please, keep this in mind when merging issues to the master branch. I'll also exclude all issues with `Open`, `In progress` states from the release. As for the `Patch Available` issues, we will discuss this list on dev-list. Please, let me know what we definitely must include to the 2.10 release and review. On Sat, 19 Dec 2020 at 11:39, Maxim Muzafarov wrote: > > Denis, Nikita > > Yes, the list of important documentation issues will be ready prior to > the scope freeze date. I'll prepare it. > > I mean if we want to start working right now on some documentation > task we can choose any important from those list of opened tasks. > > On Fri, 18 Dec 2020 at 19:55, Denis Magda wrote: > > > > Maxim, > > > > Presently, there are 70 documentation tickets. Probably, these are all the > > tickets that we are moving from a release to a release. Could you help to > > prepare a list of 2.10-specific tickets (features, enhancements, required > > updates due to a change in the behavior)? As before, we can track these > > important 2.10-specific tickets under the "Documentation tasks for > > important tasks" features: > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.10#ApacheIgnite2.10-Documentationtasksforimportantfeatures > > > > > > - > > Denis > > > > > > On Fri, Dec 18, 2020 at 8:14 AM Maxim Muzafarov wrote: > > > > > Hello Nikita, > > > > > > Thanks for your support. > > > > > > You can find the 2.10 related documentation issues on the release wiki > > > page [1]. Currently, some of the important issues don't have the right > > > priorities, so it's better to sort them first. > > > > > > As for the time estimates, we've previously agreed that documentation > > > should be ready prior to the release voting date. You can find the > > > actual release dates here [2]. > > > > > > [1] > > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.10#ApacheIgnite2.10-Allunresolveddocumentationtasks > > > [2] > > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.10#ApacheIgnite2.10-Releasephases > > > > > > On Fri, 18 Dec 2020 at 18:48, Никита Сафонов > > > wrote: > > > > > > > > Hi guys, > > > > > > > > I'd be more than happy to support the 2.10 release and contribute to the > > > > documentation. > > > > So could you please give me a scope of documentation tasks for 2.10 with > > > > the approximate time estimates? > > > > > > > > Regards, > > > > Nikita > > > > > > > > пн, 7 дек. 2020 г. в 16:23, Alexey Zinoviev : > > > > > > > > > Support it > > > > > > > > > > пн, 7 дек. 2020 г. в 14:03, Maxim Muzafarov : > > > > > > > > > > > Folks, > > > > > > > > > > > > > > > > > > I propose to shift a bit the release date since 2.9.1 has not > > > released > > > > > yet. > > > > > > > > > > > > > > > > > > Scope Freeze: December 28, 2020 > > > > > > Code Freeze: December 28, 2020 > > > > > > Voting Date: January 25, 2021 > > > > > > Release Date: February 1, 2021 > > > > > > > > > > > > WDYT? > > > > > > > > > > > > On Tue, 17 Nov 2020 at 12:59, Maxim Muzafarov > > > wrote: > > > > > > > > > > > > > > Folks, > > > > > > > > > > > > > > I've marked all the valuable features of 2.10 release with an > > > > > > > appropriate label. Please, let me know if I've missed anything. > > > > > > > > > > > > > > Here is the link to the JIRA: > > > > > > > https://s.apache.org/1hyfu > > > > > > > > > > > > > > > > > > > > > On Thu, 12 Nov 2020 at 12:28, Anton Vinogradov > > > wrote: > > > > > > > > > > > > > > > > > - Distributed environment tests [2] (Nikolay Izhikov, Anton > > > > > > > > > Vinogradov, Ivan Daschinskiy) > > > > > > > > > Will all related issues be included in the 2.10? > > > > > > > > We're at the final preparations phase. Going to start the merge > > > > > > discussion > > > > > > > > soon. > > > > > > > > > > > > > > > > On Wed, Nov 11, 2020 at 4:48 PM Alexey Zinoviev < > > > > > > zaleslaw@gmail.com> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Yes, Model Export/Import for ML will be finished, now it's > > > ready, > > > > > > but under > > > > > > > > > testing on my side. > > > > > > > > > > > > > > > > > > ср, 11 нояб. 2020 г. в 16:08, Nikolay Izhikov < > > > nizhi...@apache.org > > > > > >: > > > > > > > > > > > > > > > > > > > I suggest to include CDC feature to the 2.10 scope. > > > > > > > > > > > > > > > > > > > > > 11 нояб. 2020 г., в 16:06, Maxim Muzafarov < > > > mmu...@apache.org> > > > > > > > > > > написал(а): > > > > > > > > > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can you clarify the issue statuses according to the Apache > > > > > Ignite > > > > > > > > > > > Roadmap [1] and the 2.10 Apache Ignite Release? Here are > > > some > > > > > > major > > > > > > > > > > > issues which are expected to be included in 2.10 with the > > > > > unknown > > > > > > > > > > > status: > > > > >
[jira] [Created] (IGNITE-13915) Calcite improvements. Extend tests coverage, use both client and server for starting queries.
Stanilovsky Evgeny created IGNITE-13915: --- Summary: Calcite improvements. Extend tests coverage, use both client and server for starting queries. Key: IGNITE-13915 URL: https://issues.apache.org/jira/browse/IGNITE-13915 Project: Ignite Issue Type: Improvement Components: sql Reporter: Stanilovsky Evgeny Assignee: Stanilovsky Evgeny We need to extend tests, for example for all _CalciteQueryProcessorTest_, we need to check resulsets from both servers and clients. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13914) wrong error message for bad selector (config get)
Malchikov Fedor created IGNITE-13914:
Summary: wrong error message for bad selector (config get)
Key: IGNITE-13914
URL: https://issues.apache.org/jira/browse/IGNITE-13914
Project: Ignite
Issue Type: Bug
Affects Versions: 3.0.0-alpha1
Reporter: Malchikov Fedor
{code:bash}
x:~/ignite-cli-test$ ./ignite config get --selector=/
Connection issues while trying to send http request{code}
expected:
{code:java}
Can't get configuration{
"error" :
{ "type" : "CONFIG_PATH_UNRECOGNIZED", "message" : "wrong sab path" }}
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [DISCUSS] Missed (non-suited) tests
Hi! >> I'm not sure that we should assume every test is only run from one test suite. One test may be run from different test suites located in different modules. Agree. For example, if we introduce this limitation, zk suites will be broken. пт, 25 дек. 2020 г. в 14:48, Ilya Kasnacheev : > Hello! > > Sorry for the long wait. > > I'm not sure that we should assume every test is only run from one test > suite. One test may be run from different test suites located in different > modules. > > I wonder if we can drop this requirement, check all the modules > transitively for used/unused tests. > > Regards, > -- > Ilya Kasnacheev > > > ср, 2 дек. 2020 г. в 18:23, Max Timonin : > > > Hi, > > > > I don't think so. It looks like a bug that tests fail if one runs them > > within their module (actually, what is the goal of this test?). The check > > showed us this problem, there is no need to fix the check. > > > > Currently I see two ways: > > 1. Find the right module for every misplaced test. There are 104 tests, > > maybe just move them all to the target module? If TeamCity runs them > within > > the indexing module only is there a reason to have a test in the core > > module at all? > > 2. Back to my previous proposal - create fake suites within a module, > then > > replace test classes in a target suite with the single class of the fake > > suite. > > > > > > > > On Wed, Dec 2, 2020 at 5:38 PM Ilya Kasnacheev < > ilya.kasnach...@gmail.com> > > wrote: > > > > > Hello! > > > > > > I think this means that we should abandon the plan of moving tests > > between > > > suites, and that your tool has to understand the dependency graph > > > between modules' tests when assessing what's included and what's not. > > > > > > Regards, > > > -- > > > Ilya Kasnacheev > > > > > > > > > ср, 2 дек. 2020 г. в 15:56, Max Timonin : > > > > > > > Hi, Ilya! > > > > > > > > I've checked testsuites. There is an issue. For example > > > > *IgniteBinaryCacheQueryTestSuite* suite is now in 2 modules: > > ignite-core, > > > > ignite-indexing. On TeamCity it runs by "Query 1" suite. Simplified > > maven > > > > command for the suite is > > > > > > > > mvn -DtestIgniteBinaryCacheQueryTestSuite -am -pl :ignite-indexing > > > > surefire:test > > > > > > > > Sequence of actions is: > > > > 1. Find modules dependencies (*-am* flag): ignite-tools, ignite-core; > > > > 2. Run the test command for every module. In this step the maven > tries > > to > > > > find the specified test for every module. This is good news, so we > > don't > > > > need to create new TeamCity suites for such splitted suites. > > > > > > > > But the run performs within the current module classpath, so for the > > core > > > > module the test suite fails with error "Add module 'ignite-indexing' > to > > > the > > > > classpath of all Ignite nodes". Maven can't resolve it. > > > > > > > > The only way to work with it is to specify additional classpath > > elements > > > > for tests with setting > > > *-Dmaven.test.additionalClasspath=/path/to/m2/jar*. > > > > I did it by filling MAVEN_OPTS with the setting. Please check the job > > > > parameters [1]. After that the core module part ran successfully. It > > > means > > > > for every TC suite that runs such splitted suite we need to set the > > > > setting. What do you think, is it a valid way to handle the issue? If > > > there > > > > are no objections, I will check other such suites. > > > > > > > > Also to mention there, the work directory contains a *repository/* > > folder > > > > with all required .jars. But usage of this path in the setting didn't > > > help. > > > > I'm not sure, but I think it's an issue due to usage of Classworlds. > > So, > > > > using dependency from .m2 is the only way. > > > > > > > > [1] > > > > > > > > > > > > > > https://ci.ignite.apache.org/viewLog.html?buildId=5770727=IgniteTests24Java8_Queries1=buildParameters > > > > > > > > > > > > > > > > On Fri, Nov 27, 2020 at 3:55 PM Max Timonin > > > > > wrote: > > > > > > > > > Sure, I'll do that. > > > > > > > > > > On Fri, Nov 27, 2020 at 2:00 PM Ilya Kasnacheev < > > > > ilya.kasnach...@gmail.com> > > > > > wrote: > > > > > > > > > >> Hello! > > > > >> > > > > >> You can override these values (module, suites) values when > running a > > > > suite > > > > >> on TC. Can you please run these ones which need to be changed > > > > individually > > > > >> on TC, make sure they run without errors and contain all the > needed > > > > tests, > > > > >> and link to these runs in the ticket? Then I can modify the suites > > to > > > > fit > > > > >> those. > > > > >> > > > > >> I'm not sure that class shadowing will work as we want it to work, > > > e.g., > > > > >> we > > > > >> now have two IgniteCacheQuerySelfTestSuite6 with the same FQDN, > I'm > > > not > > > > >> sure if maven/TC is going to pick both or just one. > > > > >> Maybe they should go to a different package, e.g., testsuites/core > > for > > > > >> every suite already present in
[jira] [Created] (IGNITE-13913) Corrupted poms in cli modules
Kirill Gusakov created IGNITE-13913:
---
Summary: Corrupted poms in cli modules
Key: IGNITE-13913
URL: https://issues.apache.org/jira/browse/IGNITE-13913
Project: Ignite
Issue Type: Sub-task
Reporter: Kirill Gusakov
{code:java}
/usr/lib/jvm/java-1.11.0-openjdk-amd64/bin/java
-Dmaven.multiModuleProjectDirectory=/home/prom1se/apache/ignite-3
-Dmaven.home=/snap/intellij-idea-community/267/plugins/maven/lib/maven3
-Dclassworlds.conf=/snap/intellij-idea-community/267/plugins/maven/lib/maven3/bin/m2.conf
-Dmaven.ext.class.path=/snap/intellij-idea-community/267/plugins/maven/lib/maven-event-listener.jar
-javaagent:/snap/intellij-idea-community/267/lib/idea_rt.jar=35791:/snap/intellij-idea-community/267/bin
-Dfile.encoding=UTF-8 -classpath
/snap/intellij-idea-community/267/plugins/maven/lib/maven3/boot/plexus-classworlds-2.6.0.jar:/snap/intellij-idea-community/267/plugins/maven/lib/maven3/boot/plexus-classworlds.license
org.codehaus.classworlds.Launcher -Didea.version=2020.3 clean install[INFO]
Scanning for projects...[ERROR] [ERROR] Some problems were encountered while
processing the POMs:[FATAL] Non-resolvable parent POM for
org.apache.ignite:ignite-cli-common:3.0-SNAPSHOT: Could not find artifact
org.apache.ignite:apache-ignite:pom:3.0.0-SNAPSHOT and 'parent.relativePath'
points at wrong local POM @ line 24, column 13[FATAL] Non-resolvable parent POM
for org.apache.ignite:ignite-cli:3.0-SNAPSHOT: Could not find artifact
org.apache.ignite:apache-ignite:pom:3.0.0-SNAPSHOT and 'parent.relativePath'
points at wrong local POM @ line 26, column 13 @ [ERROR] The build could not
read 2 projects -> [Help 1][ERROR] [ERROR] The project
org.apache.ignite:ignite-cli-common:3.0-SNAPSHOT
(/home/prom1se/apache/ignite-3/modules/cli-common/pom.xml) has 1 error[ERROR]
Non-resolvable parent POM for
org.apache.ignite:ignite-cli-common:3.0-SNAPSHOT: Could not find artifact
org.apache.ignite:apache-ignite:pom:3.0.0-SNAPSHOT and 'parent.relativePath'
points at wrong local POM @ line 24, column 13 -> [Help 2][ERROR] [ERROR]
The project org.apache.ignite:ignite-cli:3.0-SNAPSHOT
(/home/prom1se/apache/ignite-3/modules/cli/pom.xml) has 1 error[ERROR]
Non-resolvable parent POM for org.apache.ignite:ignite-cli:3.0-SNAPSHOT: Could
not find artifact org.apache.ignite:apache-ignite:pom:3.0.0-SNAPSHOT and
'parent.relativePath' points at wrong local POM @ line 26, column 13 -> [Help
2][ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with
the -e switch.[ERROR] Re-run Maven using the -X switch to enable full debug
logging.[ERROR] [ERROR] For more information about the errors and possible
solutions, please read the following articles:[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException[ERROR]
[Help 2]
http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelExceptionProcess
finished with exit code 1
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [DISCUSS] Missed (non-suited) tests
Hello! Sorry for the long wait. I'm not sure that we should assume every test is only run from one test suite. One test may be run from different test suites located in different modules. I wonder if we can drop this requirement, check all the modules transitively for used/unused tests. Regards, -- Ilya Kasnacheev ср, 2 дек. 2020 г. в 18:23, Max Timonin : > Hi, > > I don't think so. It looks like a bug that tests fail if one runs them > within their module (actually, what is the goal of this test?). The check > showed us this problem, there is no need to fix the check. > > Currently I see two ways: > 1. Find the right module for every misplaced test. There are 104 tests, > maybe just move them all to the target module? If TeamCity runs them within > the indexing module only is there a reason to have a test in the core > module at all? > 2. Back to my previous proposal - create fake suites within a module, then > replace test classes in a target suite with the single class of the fake > suite. > > > > On Wed, Dec 2, 2020 at 5:38 PM Ilya Kasnacheev > wrote: > > > Hello! > > > > I think this means that we should abandon the plan of moving tests > between > > suites, and that your tool has to understand the dependency graph > > between modules' tests when assessing what's included and what's not. > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > ср, 2 дек. 2020 г. в 15:56, Max Timonin : > > > > > Hi, Ilya! > > > > > > I've checked testsuites. There is an issue. For example > > > *IgniteBinaryCacheQueryTestSuite* suite is now in 2 modules: > ignite-core, > > > ignite-indexing. On TeamCity it runs by "Query 1" suite. Simplified > maven > > > command for the suite is > > > > > > mvn -DtestIgniteBinaryCacheQueryTestSuite -am -pl :ignite-indexing > > > surefire:test > > > > > > Sequence of actions is: > > > 1. Find modules dependencies (*-am* flag): ignite-tools, ignite-core; > > > 2. Run the test command for every module. In this step the maven tries > to > > > find the specified test for every module. This is good news, so we > don't > > > need to create new TeamCity suites for such splitted suites. > > > > > > But the run performs within the current module classpath, so for the > core > > > module the test suite fails with error "Add module 'ignite-indexing' to > > the > > > classpath of all Ignite nodes". Maven can't resolve it. > > > > > > The only way to work with it is to specify additional classpath > elements > > > for tests with setting > > *-Dmaven.test.additionalClasspath=/path/to/m2/jar*. > > > I did it by filling MAVEN_OPTS with the setting. Please check the job > > > parameters [1]. After that the core module part ran successfully. It > > means > > > for every TC suite that runs such splitted suite we need to set the > > > setting. What do you think, is it a valid way to handle the issue? If > > there > > > are no objections, I will check other such suites. > > > > > > Also to mention there, the work directory contains a *repository/* > folder > > > with all required .jars. But usage of this path in the setting didn't > > help. > > > I'm not sure, but I think it's an issue due to usage of Classworlds. > So, > > > using dependency from .m2 is the only way. > > > > > > [1] > > > > > > > > > https://ci.ignite.apache.org/viewLog.html?buildId=5770727=IgniteTests24Java8_Queries1=buildParameters > > > > > > > > > > > > On Fri, Nov 27, 2020 at 3:55 PM Max Timonin > > > wrote: > > > > > > > Sure, I'll do that. > > > > > > > > On Fri, Nov 27, 2020 at 2:00 PM Ilya Kasnacheev < > > > ilya.kasnach...@gmail.com> > > > > wrote: > > > > > > > >> Hello! > > > >> > > > >> You can override these values (module, suites) values when running a > > > suite > > > >> on TC. Can you please run these ones which need to be changed > > > individually > > > >> on TC, make sure they run without errors and contain all the needed > > > tests, > > > >> and link to these runs in the ticket? Then I can modify the suites > to > > > fit > > > >> those. > > > >> > > > >> I'm not sure that class shadowing will work as we want it to work, > > e.g., > > > >> we > > > >> now have two IgniteCacheQuerySelfTestSuite6 with the same FQDN, I'm > > not > > > >> sure if maven/TC is going to pick both or just one. > > > >> Maybe they should go to a different package, e.g., testsuites/core > for > > > >> every suite already present in indexing/spring/etc. Maybe you can > > rename > > > >> them just now? This will mean a lot less of work reconfiguring > suites. > > > >> In TC configurations, suite names are simple class names, not FQ, so > > no > > > >> changes may be needed at all. > > > >> > > > >> Regards, > > > >> -- > > > >> Ilya Kasnacheev > > > >> > > > >> > > > >> пт, 27 нояб. 2020 г. в 13:03, Max Timonin >: > > > >> > > > >> > Hi, sorry for the misleading. I mean "adding ignite-core module > > > >> *suites* to > > > >> > the TeamCity Queries* suite" > > > >> > > > > >> > On Fri, Nov 27, 2020 at 12:44 PM Ilya Kasnacheev < > >
Re: [DISCUSS] Python thin client development approach.
Hello, Ivan. I’m +1 for your proposal. > 25 дек. 2020 г., в 13:14, Ivan Daschinsky написал(а): > > Hi folks! > > Since we already have a separate repo for thin-clients [1], [2] > I'd like to propose some improvements in development process/ > > 1. We should simplify and automate unit tests run for different versions of > python > 2. We should add travis integration per commit and pr. Tests could be run > against latest dockered image of ignite > 3. There should be ability to run tests against multiple pythons on TC > 4. For thin client development process, travis should be the first option. > TC suite should be used only to check that compatibility is not broken > and when new functionality is developed (rare case). > > I've prepared fix [3], you can see successful builds for travis. It uses > tox [5], the most common tool to run tests in multiple environments. > There are few environments set up in tox.ini -- with and without docker, > with or without ssl, etc. This helped a lot > to setup travis CI build (you can see in commits list of PR) and simplify > run tests for developers. Also docker-compose was introduced > to help python thin client developers. > > But I need some assistance for TC: > 1. There is outdated python setuptools on TC agents, so tests cannot be run > with updated pytest etc. > 2. Multiple pythons should be installed on TC agents (via > https://github.com/pyenv/pyenv), latest minor versions > for 3.6, 3.7 and 3.8 > 3. After that, TC job should be changed to utilize tox > > WDYT about this initiative? > > > [1] -- https://issues.apache.org/jira/browse/IGNITE-13767 > [2] -- https://github.com/apache/ignite-python-thin-client > [3] -- https://issues.apache.org/jira/browse/IGNITE-13903 > [4] -- https://github.com/apache/ignite-python-thin-client/pull/1/commits > [5] -- https://tox.readthedocs.io/en/latest/ > > -- > Sincerely yours, Ivan Daschinskiy
Re: Limiting number of active connection for thin client
>> However, sometimes it can create excessive load for cluster in use cases when there are a lot of thin clients. Could you please provide some figures? I suppose, that our NIO server can handle thousands connections easily, and problem is not in multiple connections. пт, 25 дек. 2020 г. в 11:33, Pavel Tupitsyn : > Igor, > > The idea is to keep partition awareness enabled > even when the limit is reached, right? > So it works when the corresponding server is available, > and ignored otherwise. > > I think this could be useful, but we should document it thoroughly, > so the negative effect on partition awareness performance is clear. > > > On Fri, Dec 25, 2020 at 12:02 AM Igor Sapego wrote: > > > Hi Igniters, > > > > As you may know thin clients now can establish connections with > > multiple servers simultaneously. It is implemented this way to make > > partition awareness [1] work or for fast failover if partition awareness > > is not used. However, sometimes it can create excessive load for > > cluster in use cases when there are a lot of thin clients. I think, we > > should give user a possibility to limit maximum number of active > > connections established by a client. What do you think? > > > > [1] - > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients > > > > Best Regards, > > Igor > > > -- Sincerely yours, Ivan Daschinskiy
[jira] [Created] (IGNITE-13912) Incorrect calculation of WAL segments that should be deleted from WAL archive
Kirill Tkalenko created IGNITE-13912: Summary: Incorrect calculation of WAL segments that should be deleted from WAL archive Key: IGNITE-13912 URL: https://issues.apache.org/jira/browse/IGNITE-13912 Project: Ignite Issue Type: Bug Components: persistence Reporter: Kirill Tkalenko Assignee: Kirill Tkalenko Fix For: 2.10 Now there is an incorrect calculation of WAL segments that should be deleted from WAL archive. Since we delete only those segments whose total size should not exceed DataStorageConfiguration#maxWalArchiveSize * IGNITE_THRESHOLD_WAL_ARCHIVE_SIZE_PERCENTAGE, but should DataStorageConfiguration#maxWalArchiveSize - (DataStorageConfiguration#maxWalArchiveSize * IGNITE_THRESHOLD_WAL_ARCHIVE_SIZE_PERCENTAGE). Therefore, an excess of DataStorageConfiguration#maxWalArchiveSize occurs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13911) asyncio version of python ignite thin client
Ivan Daschinskiy created IGNITE-13911: - Summary: asyncio version of python ignite thin client Key: IGNITE-13911 URL: https://issues.apache.org/jira/browse/IGNITE-13911 Project: Ignite Issue Type: Improvement Components: python Reporter: Ivan Daschinskiy Assignee: Ivan Daschinskiy Currently, asyncio is default event-loop and coroutine engine for python 3.6+. This approach can drastically improve performance of IO-bound tasks. So it is important to implement asyncio version of python ignite client. Old synchronous version should remain and share common code with asyncio version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[DISCUSS] Python thin client development approach.
Hi folks! Since we already have a separate repo for thin-clients [1], [2] I'd like to propose some improvements in development process/ 1. We should simplify and automate unit tests run for different versions of python 2. We should add travis integration per commit and pr. Tests could be run against latest dockered image of ignite 3. There should be ability to run tests against multiple pythons on TC 4. For thin client development process, travis should be the first option. TC suite should be used only to check that compatibility is not broken and when new functionality is developed (rare case). I've prepared fix [3], you can see successful builds for travis. It uses tox [5], the most common tool to run tests in multiple environments. There are few environments set up in tox.ini -- with and without docker, with or without ssl, etc. This helped a lot to setup travis CI build (you can see in commits list of PR) and simplify run tests for developers. Also docker-compose was introduced to help python thin client developers. But I need some assistance for TC: 1. There is outdated python setuptools on TC agents, so tests cannot be run with updated pytest etc. 2. Multiple pythons should be installed on TC agents (via https://github.com/pyenv/pyenv), latest minor versions for 3.6, 3.7 and 3.8 3. After that, TC job should be changed to utilize tox WDYT about this initiative? [1] -- https://issues.apache.org/jira/browse/IGNITE-13767 [2] -- https://github.com/apache/ignite-python-thin-client [3] -- https://issues.apache.org/jira/browse/IGNITE-13903 [4] -- https://github.com/apache/ignite-python-thin-client/pull/1/commits [5] -- https://tox.readthedocs.io/en/latest/ -- Sincerely yours, Ivan Daschinskiy
Re: Re[2]: [DISCUSS] Page replacement improvement
Guys, I've implemented Segmented-LRU page replacement algorithm and benchmarked results, it gives some boost (5-10%) when page replacement is heavily used, but, unfortunately, when replacement is not used it gives about 2% drop compared to the current Random-LRU page replacement implementation. Due to this, I think Segmented-LRU can't be used as the only option or option by default. Also, I've implemented CLOCK page replacement algorithm (basic, not scan-resistant version) and benchmark results are much better. It gives about the same performance as Segmented-LRU when page replacement is used and about the same performance as Random-LRU where there is no page replacement. There are advanced modifications of CLOCK algorithm exists, but usually, they require additional maintenance cost and we can again get drop on environments without page replacements compared to Random-LRU. I've written a benchmark with background full cache scans and even in this case basic CLOCK version looks promising. So, my proposals: 1. Include all 3 implementations (Random-LRU, Segmented-LRU, CLOCK) to the product. 2. Make page replacement algorithm configurable. 3. Recommend to use Random-LRU for environments with no page replacements (as it has zero maintenance cost). Recommend to use Segmented-LRU for environments with a high page replacement rate and a large amount of one-time scans (as it has near to optimal page to replace selection policy and scan-resistant). Recommend to use CLOCK for all other cases (as it has near to zero maintenance cost and efficiency of page replacement near to Segmented-LRU). 4. Set CLOCK as the default page replacement algorithm. WDYT? I've filled the IEP [1] for this discussion and create the pull request [2] for the last proposal. I would appreciate for review of the patch. [1]: https://cwiki.apache.org/confluence/display/IGNITE/IEP-62+Page+replacement+improvements [2]: https://github.com/apache/ignite/pull/8513 пн, 23 нояб. 2020 г. в 11:12, Zhenya Stanilovsky : > > > Nikolay, i hope such case ignite users already observed) > I suggest to: put data bigger then available, full scan, get data only for > available inmem data in loop, check PageReplacement metric, how match > iterations will bring to zero this metric. > > >Hello, Alex. > > > >> Perhaps we can implement a special benchmark for this case with > manually triggered "batch page replacement" using yardstick (but I'm not > sure if ducktape can help here). > > > >I think we should carefully describe the issues with the current approach > and then we can choose right tool to benchmark improvements. > >You said: > > > >> we use Random-LRU algorithm … it has many disadvantages and affects > performance very much when replacement is started > > > >Can you list disadvantages of the Random-LRU? > > > >AFAIU the first benchmark should be: > > > >1. Start cluster with persistence and put data bigger then available RAM > to it. > >2. Measure performance of the queries that selects one entry from the > cache. > >3. Make some queries that will iterate throw all data - `SELECT SUM(x) > FROM t` or something similar. > >4. Repeat step 2. in this time performance of random queries should be > lower due to the page replacement. > > > >Is this scenario correct? > > > >> 23 нояб. 2020 г., в 09:12, Alex Plehanov < plehanov.a...@gmail.com > > написал(а): > >> > >> Nikolay, Zhenya, > >> > >> Benchmark from IGNITE-13034 is fully synthetic, it makes random puts > >> uniformly. It can be used to compare different page replacement > algorithms > >> (random-LRU vs segmented-LRU), but it is totally inapplicable to > benchmark > >> batch page replacement. > >> Perhaps we can implement a special benchmark for this case with manually > >> triggered "batch page replacement" using yardstick (but I'm not sure > >> if ducktape can help here). > >> > >>> I understand case you described, but who will pull the switch ? Human, > >> artificial intelligence ? > >> As I described before, we can implement both manual and automatic "batch > >> page replacement" triggering. For automatic triggering, there is no > >> artificial intelligence needed, just several conditions with reasonable > >> thresholds. Automatic triggering also can be disabled by default. > >> > >> пт, 20 нояб. 2020 г. в 13:32, Zhenya Stanilovsky < > arzamas...@mail.ru.invalid > >>> : > >> > >>> > >>> > >>> > >>> > >>> > >>> > Zhenya, > > > Alexey, we already have changes that partially fixes this issue [1] > IGNITE-13086 it's a minor improvement. We still have major problems > with > our page replacement algorithm (slow page selection and non-optimal > page-fault rate). I think changing from random 5 pages to 7 will make > things even worse (it's better for page-fault rate, but page selection > >>> will > be slower). > >>> All this words above need to be proven, i hope. + 1 with Nikolay, we > need > >>> correct reproduces or some graphs from 2.9 ver. > >>> > > >
Re: Limiting number of active connection for thin client
Igor, The idea is to keep partition awareness enabled even when the limit is reached, right? So it works when the corresponding server is available, and ignored otherwise. I think this could be useful, but we should document it thoroughly, so the negative effect on partition awareness performance is clear. On Fri, Dec 25, 2020 at 12:02 AM Igor Sapego wrote: > Hi Igniters, > > As you may know thin clients now can establish connections with > multiple servers simultaneously. It is implemented this way to make > partition awareness [1] work or for fast failover if partition awareness > is not used. However, sometimes it can create excessive load for > cluster in use cases when there are a lot of thin clients. I think, we > should give user a possibility to limit maximum number of active > connections established by a client. What do you think? > > [1] - > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients > > Best Regards, > Igor >
Re: [DISCUSS] Ignite 3.0 development approach
Rather strange, cause the repository on GitHub is only a mirror of Apache's GitBox. Although — I guess Apache applied the same policy to its repositories, if not was it's author... > On 22 Dec 2020, at 13:34, Pavel Tupitsyn wrote: > > Ivan, it is the new GitHub default > > "On Oct. 1, 2020, any new repositories you create will use main as the > default branch, instead of master" [1] > > [1] > https://www.zdnet.com/article/github-to-replace-master-with-main-starting-next-month/ > > On Tue, Dec 22, 2020 at 1:12 PM Ivan Pavlukhin wrote: > >> Also I noticed that ignite-3 repository has "main" but not "master" >> branch. Who can shed light on this? Did not find an explanation in >> this thread. >> >> 2020-12-22 13:09 GMT+03:00, Ivan Pavlukhin : >>> I noticed some free-from commit messages in ignite-3 repository. I >>> think we should use ticket-based workflow and commit messages as >>> usual. >>> >>> [1] https://github.com/apache/ignite-3/commits/main >>> >>> 2020-12-21 10:55 GMT+03:00, Petr Ivanov : There is no problem to have both in new repository, if skilled >> enthusiast will take over that job. I guess we will stick to Maven for time being but development of Gradle-based building system can be done in parallel. I can even add corresponding development build configurations for TeamCity, or even introduce some kind of switch — so that we can test old and new build approaches and provide seamless transition if we agree on that. > On 19 Dec 2020, at 01:00, Valentin Kulichenko > wrote: > > Hi Ivan, > > There was a very brief discussion around this. Basically, it looks like > switching from Maven to something else is not going to bring much >> value, > but at the same time will be quite demanding because the community has > much > more experience with Maven. However, I would say that it is still > debatable at this point -- please feel free to share your thoughts on > this. > > -Val > > On Thu, Dec 17, 2020 at 10:53 PM Ivan Pavlukhin > wrote: > >> Hi Igniters, >> >> Forgive me that I am not reading dev list carefully these days. Was it >> explicitly decided that Maven should be used as a build system for >> Ignite 3? As there is a new repository we possibly can update our >> build tools as well. What do you think? >> >> 2020-12-17 22:45 GMT+03:00, Valentin Kulichenko < >> valentin.kuliche...@gmail.com>: >>> Hi Dmitriy, >>> >>> I don't think there is any reason for concern at this point. The >> community >>> agreed on the scope of the changes for 3.0 - it is described on Wiki. >>> The >>> scope is quite big, so it is clear that 2.x and 3.x will have to >> exist >>> in >>> parallel for a significant amount of time, so we need a place where >> we >> can >>> merge the code for 3.x. Thus, I've created this new repo. We already >>> have >>> multiple IEPs, as well as several contributors who are actively >>> involved >> in >>> development. Some of the first PRs were merged today. >>> >>> I didn't hear any objections since the repo was created. >>> >>> -Val >>> >>> On Wed, Dec 16, 2020 at 7:28 AM Dmitriy Pavlov >> wrote: >>> Folks, I'm a little bit concerned about the simultaneous - existence of the repo https://github.com/apache/ignite-3 and PRs for that repo - and a couple of downvotes from PMC members. Is it all fine here? Was there any vote /discussion where it was discussed and consensus approved? What is the status of the ignite-3 repo? вт, 15 дек. 2020 г. в 17:30, Carbone, Adam >> : > I don't believe Java 7 was LTS, and I hope that others will have > upgraded > long before that. If that is the release timeframe for 3.0, then I suppose > that would makes sense, I would still doubt that people would be > going > newer than java 11, just my opinion of what I'm seeing. > > Regards > ~adam > > Adam Carbone | Director of Innovation – Intelligent Platform Team | > Bottomline Technologies > Office: 603-501-6446 | Mobile: 603-570-8418 > www.bottomline.com > > > > On 12/15/20, 4:25 AM, "Ilya Kasnacheev" < >> ilya.kasnach...@gmail.com> > wrote: > > Hello! > > I guess Ignite 3.0 will be ready for production use somewhere in 2022, > realistically. By that time, Java 8 will be long enough out of support > so > that most companies will actually forbid its use, WRT > vulnerabilities > et > all. > > After all we have managed to
