[jira] [Created] (IGNITE-14278) Design public Index API
Maksim Timonin created IGNITE-14278: --- Summary: Design public Index API Key: IGNITE-14278 URL: https://issues.apache.org/jira/browse/IGNITE-14278 Project: Ignite Issue Type: New Feature Reporter: Maksim Timonin Assignee: Maksim Timonin There is a feature request - provide opportunity to index non-key fields for non-SQL caches. Need to check opportunity to implement it and design API. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Apache Ignite 3.x: order in repository
Petr, Any suggestions on how to fix this? One of the ideas that come to my mind is to unify the process. For example, by enforcing that all changes are merged only through PRs. That, however, means that all contributors are forced to work with GitHub, which is not necessarily a great thing. Thoughts? -Val On Wed, Feb 24, 2021 at 12:45 AM Petr Ivanov wrote: > I'd guess that one of the main problem with inactive PRs is in creation of > PR for reviewing but merging it from command line (not via GitHub > interface). > Also, of course, there are lots of efforts which are abandoned after first > review, or even do not have a chance to be reviewed at all. > > > > On 22 Feb 2021, at 11:51, Stephen Darlington < > stephen.darling...@gridgain.com> wrote: > > > > I think we need to be able answer the question “Why are there so many > inactive PRs?" before we automate their removal. If perfectly good changes > are being ignored, we have a problem. > > > > Removing branches of merged PRs and protecting the main branch make > sense. > > > >> On 20 Feb 2021, at 18:30, Pavel Tupitsyn wrote: > >> > >> +1 > >> > >> - Close inactive PRs (1 month or so?) > >> - Enable main branch protection (no force pushes, require linear > history, > >> require status checks) > >> > >> On Sat, Feb 20, 2021 at 2:31 PM Petr Ivanov > wrote: > >> > >>> Hi, Igniters! > >>> > >>> > >>> When we started Ignite 3.x in new repository, not only we have > received a > >>> chance to cleanup codebase, but to maintain some order in development > >>> tools, like GitHub. > >>> Currently in 2.x repository we have lots of stalled PRs and branches, > >>> which not only clog the repository, but also indirectly influence TC > >>> performance (due to necessity to check for updates every ref: branches > and > >>> PRs). > >>> > >>> Could I suggest we devise some recommendations for using PR's and > branches > >>> in new repo and add some rules about stalled PRs at least, like closing > >>> them if inactive for some time. > >>> Also we can activate some settings in repo's configuration, like auto > >>> delete branch after PR is merged. > >>> > >>> > >>> WDYT? > > > > > >
New committer: Mauricio Stekl
The Project Management Committee (PMC) for Apache Ignite has invited Mauricio Stekl to become a committer and we are pleased to announce that he has accepted. Mauricio is the contributor who has been maintaining and improving our website for the last two years. There are countless small and significant changes he has made throughout that time. He helped to redesign the website, integrated the website into the documentation publication process, regularly optimizes the pages to ensure our project is easily discoverable by search engines. Being a committer enables easier contribution to the project since there is no need to go via the patch submission process. This should enable better productivity. - Denis
Re: Review Request
Atri, I've added my comments in the PR. -Val On Wed, Mar 3, 2021 at 7:29 AM Denis Magda wrote: > @Valentin Kulichenko , @Nikolay Izhikov > , @samvi...@yandex.ru , > > I saw you reviewing the ticket. Could you please double-check the changes? > "IGNITE-2399: Implement acquireAndExecute In IgniteSemaphore" > > > Atri, please put a ticket number and name in the title of an email, so that > community member can see right away what change is being contributed. > > - > Denis > > > On Wed, Mar 3, 2021 at 7:27 AM Atri Sharma wrote: > > > Hi, > > > > Please help in reviewing: > > > > https://github.com/apache/ignite/pull/8820 > > > > Atri > > >
Re: [DISCUSS] Missed (non-suited) tests
Max, > There is an overhead for running git + mvn test twice, but it is a cost for > the flexibility. Yes, I mean this issue. I have no objections, the build queue seems to be empty most of the time. On Wed, 3 Mar 2021 at 21:01, Max Timonin wrote: > > Yes, they can be performed as parallel, as doesn't depend on each other. > There is an overhead for running git + mvn test twice, but it is a cost for > the flexibility. > > On Wed, Mar 3, 2021 at 8:55 PM Max Timonin wrote: > > > I mean that any TC job with tests depends on both [Build], [Sanity > > Checks]. No tests run if any of those jobs failed. > > > > [Build] prepares ignite.zip for distribution between TC agents (mvn > > install). > > [Sanity Checks] checks that code is correct in terms of our static checks > > (mvn test). > > > > Indeed it can be run as a single job, but in favor of flexibility in > > configuration (enable / disable checks) it is OK to separate it in 2 steps. > > > > Do you have some objections to do it that way? > > > > On Wed, Mar 3, 2021 at 8:45 PM Maxim Muzafarov wrote: > > > >> Maxim, > >> > >> Can you clarify what means '[Sanity Checks] runs in parallel with > >> [Build]'? AFAIK the checks need the build results to run themselves. > >> > >> On Wed, 3 Mar 2021 at 18:48, Max Timonin wrote: > >> > > >> > Discussed with Petr privately. Proposal is: > >> > > >> > 1. The [Build] job runs without any checks. > >> > 2. There will be a new job [Sanity Checks], that runs all checks - > >> > checkstyle, licenses, javadoc, check-suites. > >> > 3. [Sanity Checks] runs in parallel with [Build]. > >> > 4. All TC jobs with tests depend on a result of the [Sanity Checks] > >> job. If > >> > the check job fails then a test job won't be started. > >> > 5. Users can disable the [Sanity Checks] job with a selector on the > >> > Parameters tab of custom TC build. > >> > > >> > If no one has objections I will create a JIRA ticket for that. > >> > > >> > > >> > On Wed, Mar 3, 2021 at 5:11 PM Max Timonin > >> wrote: > >> > > >> > > Hi Petr! My proposal is: > >> > > > >> > > 1. Create a parameter in [Build] TC suite - MAVEN_CHECKS, default > >> value is > >> > > "-Plicenses,checkstyle,check-licenses,check-test-suites". > >> > > 2. Use it in a command along with MAVEN_MODULES_STRING. > >> > > -U -Pall-java,all-scala,scala,lgpl,examples %MAVEN_CHECKS% > >> > > %MAVEN_MODULES_STRING% > >> > > > >> > > 3. Provide a global param for test suites "reverse.dep.MAVEN_CHECKS" > >> that > >> > > is possible to override in a custom build. If I understand it > >> correctly is > >> > > possible to do by editing the job [1]. > >> > > 4. This param should be represented to a user as a selector with 2 > >> > > options: > >> > > - default (see point 1.) > >> > > - "-DskipTests=true" - that ignores all checks, skip tests and just > >> build > >> > > a .zip of Ignite. > >> > > > >> > > Could you please review this solution? Is it OK for you? > >> > > > >> > > [1] > >> > > > >> https://ci.ignite.apache.org/admin/editBuildParams.html?id=template:IgniteTests24Java8_RunTestSuitesJava > >> > > > >> > > On Thu, Feb 25, 2021 at 1:47 PM Petr Ivanov > >> wrote: > >> > > > >> > >> If profile can handle this — its ok. > >> > >> > >> > >> For choosing build type — we can introduce select, that will choose > >> > >> between -p and -DskipTests=true (defaulting to profile). > >> > >> Thus [Build] will pass either way. > >> > >> > >> > >> > >> > >> Regards, > >> > >> Petr Ivanov > >> > >> Head of IT > >> > >> IT & Development Solutions | GRIDGAIN SYSTEMS > >> > >> +7 (911) 945-00-59 > >> > >> > >> > >> > On 25 Feb 2021, at 13:23, Max Timonin > >> wrote: > >> > >> > > >> > >> > Yes, it's correct that "mvn install" runs also the "mvn test" > >> command, > >> > >> and > >> > >> > this is OK as the check-test-suites profile handles all tests > >> > >> > without running them. If the skipTests flag is triggered then this > >> > >> check is > >> > >> > useless. It will take only about 2 min to run "mvn test" with this > >> > >> profile. > >> > >> > Travis does that as one of steps. > >> > >> > > >> > >> > So, there are no issues with tests. Should I provide more info how > >> this > >> > >> > check works? > >> > >> > > >> > >> > Also, discussed with Anton Vinogradov, Alex Plekhanov. There can > >> be an > >> > >> > issue, that sometimes it's required to run custom test suites to > >> debug > >> > >> > flaky tests. Sequence of steps is the following: > >> > >> > 1. Find a test suite with flaky tests (that reproducible only on an > >> > >> > TeamCity agent); > >> > >> > 2. Comment some tests in the suite to isolate; > >> > >> > 3. Push it, and run related TC suite; > >> > >> > 4. TC suite depends on [Build] job, run the job - it will fail on > >> the > >> > >> check > >> > >> > "check-test-suites". > >> > >> > > >> > >> > So it is needed to provide a configuration to disable this check > >> such > >> > >> runs. > >> > >> > I'll have a look on next week how to implement this. > >> > >>
[VOTE] Release Apache Ignite 2.10.0 RC1
Dear Community,
The release candidate is ready. The rest of the documentation pages
will be completed prior to the release announcement message. Please,
see the links below.
I have uploaded a release candidate to:
https://dist.apache.org/repos/dist/dev/ignite/2.10.0-rc1/
https://dist.apache.org/repos/dist/dev/ignite/packages_2.10.0-rc1/
The following staging can be used for testing:
https://repository.apache.org/content/repositories/orgapacheignite-1506
https://www.myget.org/feed/apache-ignite-staging/package/nuget/Apache.Ignite
Tag name is 2.10.0-rc1:
https://gitbox.apache.org/repos/asf?p=ignite.git;a=commit;h=refs/tags/2.10.0-rc1
RELEASE_NOTES:
https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob_plain;f=RELEASE_NOTES.txt;hb=ignite-2.10
Complete list of resolved issues:
https://issues.apache.org/jira/issues/?jql=(project%20%3D%20'Ignite'%20AND%20fixVersion%20is%20not%20empty%20AND%20fixVersion%20in%20('2.10'))
DEVNOTES:
https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob_plain;f=DEVNOTES.txt;hb=ignite-2.10
Additional checks have been performed (available for users included into
the release group on TeamCity).
TC [Check RC: Licenses, compile, chksum]
https://ci.ignite.apache.org/viewLog.html?buildId=5901349=ApacheIgniteReleaseJava8_PrepareVote4CheckRcLicensesChecksum
TC [3] Build & Upload Nuget Staging Packages
https://ci.ignite.apache.org/viewLog.html?buildId=5901347=ApacheIgniteReleaseJava8_PrepareVote3BuildNuGetPackages
TC [2] Compare w/ Previous Release
https://ci.ignite.apache.org/viewLog.html?buildId=5901351=ApacheIgniteReleaseJava8_IgniteRelease72CheckFileConsistency=artifacts_Releases_ApacheIgniteMain=ignite-2.9.1#%2Fresults
The vote is formal, see voting guidelines
https://www.apache.org/foundation/voting.html
+1 - to accept Apache Ignite 2.10.0-rc1
0 - don't care either way
-1 - DO NOT accept Apache Ignite Ignite 2.10.0-rc1 (explain why)
See notes on how to verify release here
https://www.apache.org/info/verification.html
and
https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-P5.VotingonReleaseandReleaseVerification
This vote will be open until Mon Mar 8, 15:00 UTC.
Please, write me down the thread if you need additional time to check
the release.
https://www.timeanddate.com/countdown/vote?iso=20210308T18=166=VOTE+on+the+Apache+Ignite+Release+2.10.0+RC1=sanserif
Re: [DISCUSS] Missed (non-suited) tests
Yes, they can be performed as parallel, as doesn't depend on each other. There is an overhead for running git + mvn test twice, but it is a cost for the flexibility. On Wed, Mar 3, 2021 at 8:55 PM Max Timonin wrote: > I mean that any TC job with tests depends on both [Build], [Sanity > Checks]. No tests run if any of those jobs failed. > > [Build] prepares ignite.zip for distribution between TC agents (mvn > install). > [Sanity Checks] checks that code is correct in terms of our static checks > (mvn test). > > Indeed it can be run as a single job, but in favor of flexibility in > configuration (enable / disable checks) it is OK to separate it in 2 steps. > > Do you have some objections to do it that way? > > On Wed, Mar 3, 2021 at 8:45 PM Maxim Muzafarov wrote: > >> Maxim, >> >> Can you clarify what means '[Sanity Checks] runs in parallel with >> [Build]'? AFAIK the checks need the build results to run themselves. >> >> On Wed, 3 Mar 2021 at 18:48, Max Timonin wrote: >> > >> > Discussed with Petr privately. Proposal is: >> > >> > 1. The [Build] job runs without any checks. >> > 2. There will be a new job [Sanity Checks], that runs all checks - >> > checkstyle, licenses, javadoc, check-suites. >> > 3. [Sanity Checks] runs in parallel with [Build]. >> > 4. All TC jobs with tests depend on a result of the [Sanity Checks] >> job. If >> > the check job fails then a test job won't be started. >> > 5. Users can disable the [Sanity Checks] job with a selector on the >> > Parameters tab of custom TC build. >> > >> > If no one has objections I will create a JIRA ticket for that. >> > >> > >> > On Wed, Mar 3, 2021 at 5:11 PM Max Timonin >> wrote: >> > >> > > Hi Petr! My proposal is: >> > > >> > > 1. Create a parameter in [Build] TC suite - MAVEN_CHECKS, default >> value is >> > > "-Plicenses,checkstyle,check-licenses,check-test-suites". >> > > 2. Use it in a command along with MAVEN_MODULES_STRING. >> > > -U -Pall-java,all-scala,scala,lgpl,examples %MAVEN_CHECKS% >> > > %MAVEN_MODULES_STRING% >> > > >> > > 3. Provide a global param for test suites "reverse.dep.MAVEN_CHECKS" >> that >> > > is possible to override in a custom build. If I understand it >> correctly is >> > > possible to do by editing the job [1]. >> > > 4. This param should be represented to a user as a selector with 2 >> > > options: >> > > - default (see point 1.) >> > > - "-DskipTests=true" - that ignores all checks, skip tests and just >> build >> > > a .zip of Ignite. >> > > >> > > Could you please review this solution? Is it OK for you? >> > > >> > > [1] >> > > >> https://ci.ignite.apache.org/admin/editBuildParams.html?id=template:IgniteTests24Java8_RunTestSuitesJava >> > > >> > > On Thu, Feb 25, 2021 at 1:47 PM Petr Ivanov >> wrote: >> > > >> > >> If profile can handle this — its ok. >> > >> >> > >> For choosing build type — we can introduce select, that will choose >> > >> between -p and -DskipTests=true (defaulting to profile). >> > >> Thus [Build] will pass either way. >> > >> >> > >> >> > >> Regards, >> > >> Petr Ivanov >> > >> Head of IT >> > >> IT & Development Solutions | GRIDGAIN SYSTEMS >> > >> +7 (911) 945-00-59 >> > >> >> > >> > On 25 Feb 2021, at 13:23, Max Timonin >> wrote: >> > >> > >> > >> > Yes, it's correct that "mvn install" runs also the "mvn test" >> command, >> > >> and >> > >> > this is OK as the check-test-suites profile handles all tests >> > >> > without running them. If the skipTests flag is triggered then this >> > >> check is >> > >> > useless. It will take only about 2 min to run "mvn test" with this >> > >> profile. >> > >> > Travis does that as one of steps. >> > >> > >> > >> > So, there are no issues with tests. Should I provide more info how >> this >> > >> > check works? >> > >> > >> > >> > Also, discussed with Anton Vinogradov, Alex Plekhanov. There can >> be an >> > >> > issue, that sometimes it's required to run custom test suites to >> debug >> > >> > flaky tests. Sequence of steps is the following: >> > >> > 1. Find a test suite with flaky tests (that reproducible only on an >> > >> > TeamCity agent); >> > >> > 2. Comment some tests in the suite to isolate; >> > >> > 3. Push it, and run related TC suite; >> > >> > 4. TC suite depends on [Build] job, run the job - it will fail on >> the >> > >> check >> > >> > "check-test-suites". >> > >> > >> > >> > So it is needed to provide a configuration to disable this check >> such >> > >> runs. >> > >> > I'll have a look on next week how to implement this. >> > >> > >> > >> > On Thu, Feb 25, 2021 at 11:02 AM Petr Ivanov > > >> > >> wrote: >> > >> > >> > >> >> I am telling that INSTALL goal for maven will trigger TEST goal >> for the >> > >> >> whole project and it cannot be prevented until the flag is >> specified >> > >> either >> > >> >> as command line parameter, or in profile somehow to be inherited >> by >> > >> other >> > >> >> modules. >> > >> >> Thats why I am suggesting this as separate suite. >> > >> >> >> > >> >> >> > >> >> Regards, >> > >> >>
Re: [DISCUSS] Missed (non-suited) tests
I mean that any TC job with tests depends on both [Build], [Sanity Checks]. No tests run if any of those jobs failed. [Build] prepares ignite.zip for distribution between TC agents (mvn install). [Sanity Checks] checks that code is correct in terms of our static checks (mvn test). Indeed it can be run as a single job, but in favor of flexibility in configuration (enable / disable checks) it is OK to separate it in 2 steps. Do you have some objections to do it that way? On Wed, Mar 3, 2021 at 8:45 PM Maxim Muzafarov wrote: > Maxim, > > Can you clarify what means '[Sanity Checks] runs in parallel with > [Build]'? AFAIK the checks need the build results to run themselves. > > On Wed, 3 Mar 2021 at 18:48, Max Timonin wrote: > > > > Discussed with Petr privately. Proposal is: > > > > 1. The [Build] job runs without any checks. > > 2. There will be a new job [Sanity Checks], that runs all checks - > > checkstyle, licenses, javadoc, check-suites. > > 3. [Sanity Checks] runs in parallel with [Build]. > > 4. All TC jobs with tests depend on a result of the [Sanity Checks] job. > If > > the check job fails then a test job won't be started. > > 5. Users can disable the [Sanity Checks] job with a selector on the > > Parameters tab of custom TC build. > > > > If no one has objections I will create a JIRA ticket for that. > > > > > > On Wed, Mar 3, 2021 at 5:11 PM Max Timonin > wrote: > > > > > Hi Petr! My proposal is: > > > > > > 1. Create a parameter in [Build] TC suite - MAVEN_CHECKS, default > value is > > > "-Plicenses,checkstyle,check-licenses,check-test-suites". > > > 2. Use it in a command along with MAVEN_MODULES_STRING. > > > -U -Pall-java,all-scala,scala,lgpl,examples %MAVEN_CHECKS% > > > %MAVEN_MODULES_STRING% > > > > > > 3. Provide a global param for test suites "reverse.dep.MAVEN_CHECKS" > that > > > is possible to override in a custom build. If I understand it > correctly is > > > possible to do by editing the job [1]. > > > 4. This param should be represented to a user as a selector with 2 > > > options: > > > - default (see point 1.) > > > - "-DskipTests=true" - that ignores all checks, skip tests and just > build > > > a .zip of Ignite. > > > > > > Could you please review this solution? Is it OK for you? > > > > > > [1] > > > > https://ci.ignite.apache.org/admin/editBuildParams.html?id=template:IgniteTests24Java8_RunTestSuitesJava > > > > > > On Thu, Feb 25, 2021 at 1:47 PM Petr Ivanov > wrote: > > > > > >> If profile can handle this — its ok. > > >> > > >> For choosing build type — we can introduce select, that will choose > > >> between -p and -DskipTests=true (defaulting to profile). > > >> Thus [Build] will pass either way. > > >> > > >> > > >> Regards, > > >> Petr Ivanov > > >> Head of IT > > >> IT & Development Solutions | GRIDGAIN SYSTEMS > > >> +7 (911) 945-00-59 > > >> > > >> > On 25 Feb 2021, at 13:23, Max Timonin > wrote: > > >> > > > >> > Yes, it's correct that "mvn install" runs also the "mvn test" > command, > > >> and > > >> > this is OK as the check-test-suites profile handles all tests > > >> > without running them. If the skipTests flag is triggered then this > > >> check is > > >> > useless. It will take only about 2 min to run "mvn test" with this > > >> profile. > > >> > Travis does that as one of steps. > > >> > > > >> > So, there are no issues with tests. Should I provide more info how > this > > >> > check works? > > >> > > > >> > Also, discussed with Anton Vinogradov, Alex Plekhanov. There can be > an > > >> > issue, that sometimes it's required to run custom test suites to > debug > > >> > flaky tests. Sequence of steps is the following: > > >> > 1. Find a test suite with flaky tests (that reproducible only on an > > >> > TeamCity agent); > > >> > 2. Comment some tests in the suite to isolate; > > >> > 3. Push it, and run related TC suite; > > >> > 4. TC suite depends on [Build] job, run the job - it will fail on > the > > >> check > > >> > "check-test-suites". > > >> > > > >> > So it is needed to provide a configuration to disable this check > such > > >> runs. > > >> > I'll have a look on next week how to implement this. > > >> > > > >> > On Thu, Feb 25, 2021 at 11:02 AM Petr Ivanov > > >> wrote: > > >> > > > >> >> I am telling that INSTALL goal for maven will trigger TEST goal > for the > > >> >> whole project and it cannot be prevented until the flag is > specified > > >> either > > >> >> as command line parameter, or in profile somehow to be inherited by > > >> other > > >> >> modules. > > >> >> Thats why I am suggesting this as separate suite. > > >> >> > > >> >> > > >> >> Regards, > > >> >> *Petr Ivanov* > > >> >> Head of IT > > >> >> IT & Development Solutions | > > >> >> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59 > > >> >> > > >> >> On 25 Feb 2021, at 10:44, Max Timonin > wrote: > > >> >> > > >> >> Hi, Petr! > > >> >> > > >> >> Profile "check-test-suites" handles all tests in another way, it > just > > >> >> verifies that all tests are suited. No tests run
Re: [DISCUSS] Missed (non-suited) tests
Maxim, Can you clarify what means '[Sanity Checks] runs in parallel with [Build]'? AFAIK the checks need the build results to run themselves. On Wed, 3 Mar 2021 at 18:48, Max Timonin wrote: > > Discussed with Petr privately. Proposal is: > > 1. The [Build] job runs without any checks. > 2. There will be a new job [Sanity Checks], that runs all checks - > checkstyle, licenses, javadoc, check-suites. > 3. [Sanity Checks] runs in parallel with [Build]. > 4. All TC jobs with tests depend on a result of the [Sanity Checks] job. If > the check job fails then a test job won't be started. > 5. Users can disable the [Sanity Checks] job with a selector on the > Parameters tab of custom TC build. > > If no one has objections I will create a JIRA ticket for that. > > > On Wed, Mar 3, 2021 at 5:11 PM Max Timonin wrote: > > > Hi Petr! My proposal is: > > > > 1. Create a parameter in [Build] TC suite - MAVEN_CHECKS, default value is > > "-Plicenses,checkstyle,check-licenses,check-test-suites". > > 2. Use it in a command along with MAVEN_MODULES_STRING. > > -U -Pall-java,all-scala,scala,lgpl,examples %MAVEN_CHECKS% > > %MAVEN_MODULES_STRING% > > > > 3. Provide a global param for test suites "reverse.dep.MAVEN_CHECKS" that > > is possible to override in a custom build. If I understand it correctly is > > possible to do by editing the job [1]. > > 4. This param should be represented to a user as a selector with 2 > > options: > > - default (see point 1.) > > - "-DskipTests=true" - that ignores all checks, skip tests and just build > > a .zip of Ignite. > > > > Could you please review this solution? Is it OK for you? > > > > [1] > > https://ci.ignite.apache.org/admin/editBuildParams.html?id=template:IgniteTests24Java8_RunTestSuitesJava > > > > On Thu, Feb 25, 2021 at 1:47 PM Petr Ivanov wrote: > > > >> If profile can handle this — its ok. > >> > >> For choosing build type — we can introduce select, that will choose > >> between -p and -DskipTests=true (defaulting to profile). > >> Thus [Build] will pass either way. > >> > >> > >> Regards, > >> Petr Ivanov > >> Head of IT > >> IT & Development Solutions | GRIDGAIN SYSTEMS > >> +7 (911) 945-00-59 > >> > >> > On 25 Feb 2021, at 13:23, Max Timonin wrote: > >> > > >> > Yes, it's correct that "mvn install" runs also the "mvn test" command, > >> and > >> > this is OK as the check-test-suites profile handles all tests > >> > without running them. If the skipTests flag is triggered then this > >> check is > >> > useless. It will take only about 2 min to run "mvn test" with this > >> profile. > >> > Travis does that as one of steps. > >> > > >> > So, there are no issues with tests. Should I provide more info how this > >> > check works? > >> > > >> > Also, discussed with Anton Vinogradov, Alex Plekhanov. There can be an > >> > issue, that sometimes it's required to run custom test suites to debug > >> > flaky tests. Sequence of steps is the following: > >> > 1. Find a test suite with flaky tests (that reproducible only on an > >> > TeamCity agent); > >> > 2. Comment some tests in the suite to isolate; > >> > 3. Push it, and run related TC suite; > >> > 4. TC suite depends on [Build] job, run the job - it will fail on the > >> check > >> > "check-test-suites". > >> > > >> > So it is needed to provide a configuration to disable this check such > >> runs. > >> > I'll have a look on next week how to implement this. > >> > > >> > On Thu, Feb 25, 2021 at 11:02 AM Petr Ivanov > >> wrote: > >> > > >> >> I am telling that INSTALL goal for maven will trigger TEST goal for the > >> >> whole project and it cannot be prevented until the flag is specified > >> either > >> >> as command line parameter, or in profile somehow to be inherited by > >> other > >> >> modules. > >> >> Thats why I am suggesting this as separate suite. > >> >> > >> >> > >> >> Regards, > >> >> *Petr Ivanov* > >> >> Head of IT > >> >> IT & Development Solutions | > >> >> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59 > >> >> > >> >> On 25 Feb 2021, at 10:44, Max Timonin wrote: > >> >> > >> >> Hi, Petr! > >> >> > >> >> Profile "check-test-suites" handles all tests in another way, it just > >> >> verifies that all tests are suited. No tests run then. > >> >> As I understand the [BUILD] job goal is preparing a .zip archive to > >> >> distribute it for other jobs. I think it is a valid place to put sanity > >> >> checks. If a check fails then no archive is prepared. WDYT? > >> >> > >> >> Also I see that there is a flag -Dmaven.javadoc.skip=true. I'd propose > >> to > >> >> change it to the profile "skip-docs", that was introduced in ticket [1] > >> >> IGNITE-13623. As the setting "maven.javadoc.skip" does not > >> >> affect scaladocs. > >> >> > >> >> [1] https://issues.apache.org/jira/browse/IGNITE-13623 > >> >> > >> >> On Thu, Feb 25, 2021 at 7:34 AM Petr Ivanov > >> wrote: > >> >> > >> >>> Won't the absence of -DskipTests flag trigger ALL the tests for all > >> >>> modules? > >> >>> This flag was added intentionally. > >> >>> >
[jira] [Created] (IGNITE-14277) Calcite.
Konstantin Orlov created IGNITE-14277:
-
Summary: Calcite.
Key: IGNITE-14277
URL: https://issues.apache.org/jira/browse/IGNITE-14277
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Konstantin Orlov
Currently SqlToRelConverter rewrites queries with IN predicate into semi-join
but without actually using semi-join. The resulting plan includes two joins and
several aggregates over IN argument list to calculate some sort of indicators.
This plan is quite cumbersome, it contains a lot of nodes, thus boost a search
space.
As workaround this optimization was disabled by increasing inSubQueryThreshold
to a MAX_INT value.
But a safer solution would be to rewrite the IN predicate as a true semi-join,
or better yet, an inner join. To achieve this, we need to convert the list of
values to an inline table with only single values as the left shoulder of the
inner join, and place the original table as the right shoulder. Thus, we could
take advantage of the Indexed Nested Loop in case there is an index on a column
that is part of the IN predicate.
Starting point for this ticket is
{{org.apache.calcite.sql2rel.SqlToRelConverter#substituteSubQuery}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [DISCUSS] Missed (non-suited) tests
Discussed with Petr privately. Proposal is: 1. The [Build] job runs without any checks. 2. There will be a new job [Sanity Checks], that runs all checks - checkstyle, licenses, javadoc, check-suites. 3. [Sanity Checks] runs in parallel with [Build]. 4. All TC jobs with tests depend on a result of the [Sanity Checks] job. If the check job fails then a test job won't be started. 5. Users can disable the [Sanity Checks] job with a selector on the Parameters tab of custom TC build. If no one has objections I will create a JIRA ticket for that. On Wed, Mar 3, 2021 at 5:11 PM Max Timonin wrote: > Hi Petr! My proposal is: > > 1. Create a parameter in [Build] TC suite - MAVEN_CHECKS, default value is > "-Plicenses,checkstyle,check-licenses,check-test-suites". > 2. Use it in a command along with MAVEN_MODULES_STRING. > -U -Pall-java,all-scala,scala,lgpl,examples %MAVEN_CHECKS% > %MAVEN_MODULES_STRING% > > 3. Provide a global param for test suites "reverse.dep.MAVEN_CHECKS" that > is possible to override in a custom build. If I understand it correctly is > possible to do by editing the job [1]. > 4. This param should be represented to a user as a selector with 2 > options: > - default (see point 1.) > - "-DskipTests=true" - that ignores all checks, skip tests and just build > a .zip of Ignite. > > Could you please review this solution? Is it OK for you? > > [1] > https://ci.ignite.apache.org/admin/editBuildParams.html?id=template:IgniteTests24Java8_RunTestSuitesJava > > On Thu, Feb 25, 2021 at 1:47 PM Petr Ivanov wrote: > >> If profile can handle this — its ok. >> >> For choosing build type — we can introduce select, that will choose >> between -p and -DskipTests=true (defaulting to profile). >> Thus [Build] will pass either way. >> >> >> Regards, >> Petr Ivanov >> Head of IT >> IT & Development Solutions | GRIDGAIN SYSTEMS >> +7 (911) 945-00-59 >> >> > On 25 Feb 2021, at 13:23, Max Timonin wrote: >> > >> > Yes, it's correct that "mvn install" runs also the "mvn test" command, >> and >> > this is OK as the check-test-suites profile handles all tests >> > without running them. If the skipTests flag is triggered then this >> check is >> > useless. It will take only about 2 min to run "mvn test" with this >> profile. >> > Travis does that as one of steps. >> > >> > So, there are no issues with tests. Should I provide more info how this >> > check works? >> > >> > Also, discussed with Anton Vinogradov, Alex Plekhanov. There can be an >> > issue, that sometimes it's required to run custom test suites to debug >> > flaky tests. Sequence of steps is the following: >> > 1. Find a test suite with flaky tests (that reproducible only on an >> > TeamCity agent); >> > 2. Comment some tests in the suite to isolate; >> > 3. Push it, and run related TC suite; >> > 4. TC suite depends on [Build] job, run the job - it will fail on the >> check >> > "check-test-suites". >> > >> > So it is needed to provide a configuration to disable this check such >> runs. >> > I'll have a look on next week how to implement this. >> > >> > On Thu, Feb 25, 2021 at 11:02 AM Petr Ivanov >> wrote: >> > >> >> I am telling that INSTALL goal for maven will trigger TEST goal for the >> >> whole project and it cannot be prevented until the flag is specified >> either >> >> as command line parameter, or in profile somehow to be inherited by >> other >> >> modules. >> >> Thats why I am suggesting this as separate suite. >> >> >> >> >> >> Regards, >> >> *Petr Ivanov* >> >> Head of IT >> >> IT & Development Solutions | >> >> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59 >> >> >> >> On 25 Feb 2021, at 10:44, Max Timonin wrote: >> >> >> >> Hi, Petr! >> >> >> >> Profile "check-test-suites" handles all tests in another way, it just >> >> verifies that all tests are suited. No tests run then. >> >> As I understand the [BUILD] job goal is preparing a .zip archive to >> >> distribute it for other jobs. I think it is a valid place to put sanity >> >> checks. If a check fails then no archive is prepared. WDYT? >> >> >> >> Also I see that there is a flag -Dmaven.javadoc.skip=true. I'd propose >> to >> >> change it to the profile "skip-docs", that was introduced in ticket [1] >> >> IGNITE-13623. As the setting "maven.javadoc.skip" does not >> >> affect scaladocs. >> >> >> >> [1] https://issues.apache.org/jira/browse/IGNITE-13623 >> >> >> >> On Thu, Feb 25, 2021 at 7:34 AM Petr Ivanov >> wrote: >> >> >> >>> Won't the absence of -DskipTests flag trigger ALL the tests for all >> >>> modules? >> >>> This flag was added intentionally. >> >>> >> >>> Instead, I'd put Non-Suited tests into some kind of sanity check, >> group >> >>> all sanity checks in single Run All, and make tests depend on it's >> >>> successful pass. >> >>> >> >>> >> >>> Regards, >> >>> *Petr Ivanov* >> >>> Head of IT >> >>> IT & Development Solutions | >> >>> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59 >> >>> >> >>> On 24 Feb 2021, at 19:58, Max Timonin >> wrote: >> >>> >> >>> Hi, all! >> >>> >> >>> What do you
Re: Review Request
@Valentin Kulichenko , @Nikolay Izhikov , @samvi...@yandex.ru , I saw you reviewing the ticket. Could you please double-check the changes? "IGNITE-2399: Implement acquireAndExecute In IgniteSemaphore" Atri, please put a ticket number and name in the title of an email, so that community member can see right away what change is being contributed. - Denis On Wed, Mar 3, 2021 at 7:27 AM Atri Sharma wrote: > Hi, > > Please help in reviewing: > > https://github.com/apache/ignite/pull/8820 > > Atri >
[jira] [Created] (IGNITE-14276) Local maven build runs tests while they are switched off by skipTests option
Andrey N. Gura created IGNITE-14276:
---
Summary: Local maven build runs tests while they are switched off
by skipTests option
Key: IGNITE-14276
URL: https://issues.apache.org/jira/browse/IGNITE-14276
Project: Ignite
Issue Type: New Feature
Reporter: Andrey N. Gura
Assignee: Peter Ivanov
Fix For: 3.0.0-alpha2
Local maven build runs unit-tests in despite of skipTests option.
{code}
mvn clean install -DskipTests
{code}
Expected behavior: build procedure will compiled tests but will not run
Actual behavior: build procedure runs tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-14275) SQL. Calcite: MapReduce aggregates for hash distribution
Yury Gerzhedovich created IGNITE-14275: -- Summary: SQL. Calcite: MapReduce aggregates for hash distribution Key: IGNITE-14275 URL: https://issues.apache.org/jira/browse/IGNITE-14275 Project: Ignite Issue Type: Improvement Components: sql Reporter: Yury Gerzhedovich Assignee: Yury Gerzhedovich Currently, we don't choice map-reduce aggregates algorithm for hash distribution, also the same issue there is for aggregates which could used with no GROUP BY clouse like a COUNT function. Need to implement this possibility. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14274) Calcite integration: refactoring aggregates converter rules: single and map/reduce
Taras Ledkov created IGNITE-14274: - Summary: Calcite integration: refactoring aggregates converter rules: single and map/reduce Key: IGNITE-14274 URL: https://issues.apache.org/jira/browse/IGNITE-14274 Project: Ignite Issue Type: Improvement Components: sql Reporter: Taras Ledkov Assignee: Taras Ledkov We have to use converter rules to create single & map+reduce aggregates instead of creates map/reduce aggregates on handle traits at the single aggregates. Now: SortAggregateConverterRule HashAggregateConverterRule Must be: SortSingleAggregateConverterRule SortMapReduceAggregateConverterRule HashSingleAggregateConverterRule HashMapReduceAggregateConverterRule -- This message was sent by Atlassian Jira (v8.3.4#803005)
Review Request
Hi, Please help in reviewing: https://github.com/apache/ignite/pull/8820 Atri
Minor PR
Hi, Please help review this PR: https://github.com/apache/ignite/pull/8845 Regards, Atri
Re: [DISCUSS] Missed (non-suited) tests
Hi Petr! My proposal is: 1. Create a parameter in [Build] TC suite - MAVEN_CHECKS, default value is "-Plicenses,checkstyle,check-licenses,check-test-suites". 2. Use it in a command along with MAVEN_MODULES_STRING. -U -Pall-java,all-scala,scala,lgpl,examples %MAVEN_CHECKS% %MAVEN_MODULES_STRING% 3. Provide a global param for test suites "reverse.dep.MAVEN_CHECKS" that is possible to override in a custom build. If I understand it correctly is possible to do by editing the job [1]. 4. This param should be represented to a user as a selector with 2 options: - default (see point 1.) - "-DskipTests=true" - that ignores all checks, skip tests and just build a .zip of Ignite. Could you please review this solution? Is it OK for you? [1] https://ci.ignite.apache.org/admin/editBuildParams.html?id=template:IgniteTests24Java8_RunTestSuitesJava On Thu, Feb 25, 2021 at 1:47 PM Petr Ivanov wrote: > If profile can handle this — its ok. > > For choosing build type — we can introduce select, that will choose > between -p and -DskipTests=true (defaulting to profile). > Thus [Build] will pass either way. > > > Regards, > Petr Ivanov > Head of IT > IT & Development Solutions | GRIDGAIN SYSTEMS > +7 (911) 945-00-59 > > > On 25 Feb 2021, at 13:23, Max Timonin wrote: > > > > Yes, it's correct that "mvn install" runs also the "mvn test" command, > and > > this is OK as the check-test-suites profile handles all tests > > without running them. If the skipTests flag is triggered then this check > is > > useless. It will take only about 2 min to run "mvn test" with this > profile. > > Travis does that as one of steps. > > > > So, there are no issues with tests. Should I provide more info how this > > check works? > > > > Also, discussed with Anton Vinogradov, Alex Plekhanov. There can be an > > issue, that sometimes it's required to run custom test suites to debug > > flaky tests. Sequence of steps is the following: > > 1. Find a test suite with flaky tests (that reproducible only on an > > TeamCity agent); > > 2. Comment some tests in the suite to isolate; > > 3. Push it, and run related TC suite; > > 4. TC suite depends on [Build] job, run the job - it will fail on the > check > > "check-test-suites". > > > > So it is needed to provide a configuration to disable this check such > runs. > > I'll have a look on next week how to implement this. > > > > On Thu, Feb 25, 2021 at 11:02 AM Petr Ivanov > wrote: > > > >> I am telling that INSTALL goal for maven will trigger TEST goal for the > >> whole project and it cannot be prevented until the flag is specified > either > >> as command line parameter, or in profile somehow to be inherited by > other > >> modules. > >> Thats why I am suggesting this as separate suite. > >> > >> > >> Regards, > >> *Petr Ivanov* > >> Head of IT > >> IT & Development Solutions | > >> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59 > >> > >> On 25 Feb 2021, at 10:44, Max Timonin wrote: > >> > >> Hi, Petr! > >> > >> Profile "check-test-suites" handles all tests in another way, it just > >> verifies that all tests are suited. No tests run then. > >> As I understand the [BUILD] job goal is preparing a .zip archive to > >> distribute it for other jobs. I think it is a valid place to put sanity > >> checks. If a check fails then no archive is prepared. WDYT? > >> > >> Also I see that there is a flag -Dmaven.javadoc.skip=true. I'd propose > to > >> change it to the profile "skip-docs", that was introduced in ticket [1] > >> IGNITE-13623. As the setting "maven.javadoc.skip" does not > >> affect scaladocs. > >> > >> [1] https://issues.apache.org/jira/browse/IGNITE-13623 > >> > >> On Thu, Feb 25, 2021 at 7:34 AM Petr Ivanov > wrote: > >> > >>> Won't the absence of -DskipTests flag trigger ALL the tests for all > >>> modules? > >>> This flag was added intentionally. > >>> > >>> Instead, I'd put Non-Suited tests into some kind of sanity check, group > >>> all sanity checks in single Run All, and make tests depend on it's > >>> successful pass. > >>> > >>> > >>> Regards, > >>> *Petr Ivanov* > >>> Head of IT > >>> IT & Development Solutions | > >>> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59 > >>> > >>> On 24 Feb 2021, at 19:58, Max Timonin wrote: > >>> > >>> Hi, all! > >>> > >>> What do you think if we add the check in the TC [Build] job. Currently > >>> [Build] runs also check for licences, checkstyle [1]: > >>> > >>> mvn *install* -Pall-java,all-scala,scala,*licenses*,lgpl,examples, > >>> *checkstyle* -DskipTests -Dmaven.javadoc.skip=true > >>> %MAVEN_MODULES_STRING%. > >>> > >>> So let's add the check too to block other jobs. As if there missed > tests > >>> then TC run may be invalid - missed tests may be broken and then the > MTCGA > >>> visa too. To made this we should change command line parameters: > >>> 1. Add profile check-test-suites; > >>> 2. Remove -Dskiptests flag. > >>> > >>> -Pall-java,all-scala,scala,licenses,lgpl,examples,checkstyle, > >>> *check-test-suites *-DskipTests -Dmaven.javadoc.skip=true > >>>
[jira] [Created] (IGNITE-14273) ignite-extensions: Export data from snapshot to custom format
Maxim Muzafarov created IGNITE-14273: Summary: ignite-extensions: Export data from snapshot to custom format Key: IGNITE-14273 URL: https://issues.apache.org/jira/browse/IGNITE-14273 Project: Ignite Issue Type: Improvement Components: extensions Reporter: Maxim Muzafarov Assignee: Maxim Muzafarov The user must be able to use offline command-line tools to export data from a snapshot to an external appropriate format e.g. CSV, parquet, JSON. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[MTCGA]: new failures in builds [5900469] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. *New Critical Failure in master [Build] https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_BuildApacheIgnite?branch=%3Cdefault%3E No changes in the build - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 16:10:20 03-03-2021
Re: [MEETUP] March 2: Moving from Postgres to Apache Ignite to boost Jira cloud app development
Thanks for joining us yesterday and for the inspiring Q session. Here are the materials: Alliedium app: https://alliedium.alliedtesting.com/ Slides with all the links: t.ly/9BOf Video: https://youtu.be/BvpF68NWOao An article about this case: https://medium.com/alliedium/boosting-jira-cloud-app-development-with-apache-ignite-7eebc7bb3d48 пт, 26 февр. 2021 г. в 19:53, Kseniya Romanova : > Hi Igniters! > Join Virtual Apache Ignite Meetup on March 2nd[1]. Peter Gagarinov from > Alliedium will share his experience of moving to Ignite from Postgres and > integrating Ignite with Atlassian Connect Spring Boot. In this > presentation, Peter will speak about Apache Ignite features such as SQL and > NoSQL queries, thin and thick clients, caching, distributed messaging and > events, distributed computations, database-schema change tracking, > Kubernetes deployment. > > [1] https://www.meetup.com/Apache-Ignite-Virtual-Meetup/events/276416802/ > > -- > Cheers, > Kseniya > > Devrel at GridGain > Don't miss upcoming community events, join > https://www.meetup.com/Apache-Ignite-Virtual-Meetup/ >
[jira] [Created] (IGNITE-14272) Merge modules/DEVNOTES.txt and DEVNOTES.txt
Alexey Goncharuk created IGNITE-14272: - Summary: Merge modules/DEVNOTES.txt and DEVNOTES.txt Key: IGNITE-14272 URL: https://issues.apache.org/jira/browse/IGNITE-14272 Project: Ignite Issue Type: Improvement Components: documentation Reporter: Alexey Goncharuk Assignee: Alexey Goncharuk Fix For: 3.0.0-alpha2 The modules/DEVNOTES.txt was mistakenly introduced, the contents should be moved to the root DEVNOTES.txt Also, we may add some structure to modules README.md files and link them to DEVNOTES.txt -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14271) Update Metrics page descriptions
Nikita Safonov created IGNITE-14271:
---
Summary: Update Metrics page descriptions
Key: IGNITE-14271
URL: https://issues.apache.org/jira/browse/IGNITE-14271
Project: Ignite
Issue Type: Improvement
Components: documentation
Affects Versions: 2.10
Reporter: Nikita Safonov
Assignee: Nikita Safonov
Update the *{{IndexRebuildKeyProcessed}}* and *{{IsIndexRebuildInProgress}}*
descriptions on the {{new-metrics.adoc}} page.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
