Re: [DISCUSS] Missed (non-suited) tests

[Petr Ivanov]

Yeap!


I'll start implementing it next week.
I think I'll create new jobs and when everything is OK — will switch settings 
in template to new chain.

It will take some time to do everything right though.

> On 5 Mar 2021, at 09:28, Maksim Timonin  wrote:
> 
> Hi, Petr!
> 
> I've created a ticket [1] for that and assigned it on you. Could you please
> catch it when you have time?
> 
> [1] https://issues.apache.org/jira/browse/IGNITE-14283
> 
> On Wed, Mar 3, 2021 at 11:06 PM Maxim Muzafarov  wrote:
> 
>> Max,
>> 
>>> There is an overhead for running git + mvn test twice, but it is a cost
>> for the flexibility.
>> 
>> Yes, I mean this issue. I have no objections, the build queue seems to
>> be empty most of the time.
>> 
>> On Wed, 3 Mar 2021 at 21:01, Max Timonin  wrote:
>>> 
>>> Yes, they can be performed as parallel, as doesn't depend on each other.
>>> There is an overhead for running git + mvn test twice, but it is a cost
>> for
>>> the flexibility.
>>> 
>>> On Wed, Mar 3, 2021 at 8:55 PM Max Timonin 
>> wrote:
>>> 
 I mean that any TC job with tests depends on both [Build], [Sanity
 Checks]. No tests run if any of those jobs failed.
 
 [Build] prepares ignite.zip for distribution between TC agents (mvn
 install).
 [Sanity Checks] checks that code is correct in terms of our static
>> checks
 (mvn test).
 
 Indeed it can be run as a single job, but in favor of flexibility in
 configuration (enable / disable checks) it is OK to separate it in 2
>> steps.
 
 Do you have some objections to do it that way?
 
 On Wed, Mar 3, 2021 at 8:45 PM Maxim Muzafarov 
>> wrote:
 
> Maxim,
> 
> Can you clarify what means '[Sanity Checks] runs in parallel with
> [Build]'? AFAIK the checks need the build results to run themselves.
> 
> On Wed, 3 Mar 2021 at 18:48, Max Timonin 
>> wrote:
>> 
>> Discussed with Petr privately. Proposal is:
>> 
>> 1. The [Build] job runs without any checks.
>> 2. There will be a new job [Sanity Checks], that runs all checks -
>> checkstyle, licenses, javadoc, check-suites.
>> 3. [Sanity Checks] runs in parallel with [Build].
>> 4. All TC jobs with tests depend on a result of the [Sanity Checks]
> job. If
>> the check job fails then a test job won't be started.
>> 5. Users can disable the [Sanity Checks] job with a selector on the
>> Parameters tab of custom TC build.
>> 
>> If no one has objections I will create a JIRA ticket for that.
>> 
>> 
>> On Wed, Mar 3, 2021 at 5:11 PM Max Timonin >> 
> wrote:
>> 
>>> Hi Petr! My proposal is:
>>> 
>>> 1. Create a parameter in [Build] TC suite - MAVEN_CHECKS, default
> value is
>>> "-Plicenses,checkstyle,check-licenses,check-test-suites".
>>> 2. Use it in a command along with MAVEN_MODULES_STRING.
>>> -U -Pall-java,all-scala,scala,lgpl,examples %MAVEN_CHECKS%
>>> %MAVEN_MODULES_STRING%
>>> 
>>> 3. Provide a global param for test suites
>> "reverse.dep.MAVEN_CHECKS"
> that
>>> is possible to override in a custom build. If I understand it
> correctly is
>>> possible to do by editing the job [1].
>>> 4. This param should be represented to a user as a selector with 2
>>> options:
>>> - default (see point 1.)
>>> - "-DskipTests=true" - that ignores all checks, skip tests and
>> just
> build
>>> a .zip of Ignite.
>>> 
>>> Could you please review this solution? Is it OK for you?
>>> 
>>> [1]
>>> 
> 
>> https://ci.ignite.apache.org/admin/editBuildParams.html?id=template:IgniteTests24Java8_RunTestSuitesJava
>>> 
>>> On Thu, Feb 25, 2021 at 1:47 PM Petr Ivanov >> 
> wrote:
>>> 
 If profile can handle this — its ok.
 
 For choosing build type — we can introduce select, that will
>> choose
 between -p  and -DskipTests=true (defaulting to
>> profile).
 Thus [Build] will pass either way.
 
 
 Regards,
 Petr Ivanov
 Head of IT
 IT & Development Solutions | GRIDGAIN SYSTEMS
 +7 (911) 945-00-59
 
> On 25 Feb 2021, at 13:23, Max Timonin >> 
> wrote:
> 
> Yes, it's correct that "mvn install" runs also the "mvn test"
> command,
 and
> this is OK as the check-test-suites profile handles all tests
> without running them. If the skipTests flag is triggered then
>> this
 check is
> useless. It will take only about 2 min to run "mvn test" with
>> this
 profile.
> Travis does that as one of steps.
> 
> So, there are no issues with tests. Should I provide more info
>> how
> this
> check works?
> 
> Also, discussed with Anton Vinogradov, Alex Plekhanov. There
>> can
> be an
> issue, that sometimes it's required to run custom test suites
>> to
> debug

Re: [DISCUSS] Missed (non-suited) tests

[Maksim Timonin]

Hi, Petr!

I've created a ticket [1] for that and assigned it on you. Could you please
catch it when you have time?

[1] https://issues.apache.org/jira/browse/IGNITE-14283

On Wed, Mar 3, 2021 at 11:06 PM Maxim Muzafarov  wrote:

> Max,
>
> > There is an overhead for running git + mvn test twice, but it is a cost
> for the flexibility.
>
> Yes, I mean this issue. I have no objections, the build queue seems to
> be empty most of the time.
>
> On Wed, 3 Mar 2021 at 21:01, Max Timonin  wrote:
> >
> > Yes, they can be performed as parallel, as doesn't depend on each other.
> > There is an overhead for running git + mvn test twice, but it is a cost
> for
> > the flexibility.
> >
> > On Wed, Mar 3, 2021 at 8:55 PM Max Timonin 
> wrote:
> >
> > > I mean that any TC job with tests depends on both [Build], [Sanity
> > > Checks]. No tests run if any of those jobs failed.
> > >
> > > [Build] prepares ignite.zip for distribution between TC agents (mvn
> > > install).
> > > [Sanity Checks] checks that code is correct in terms of our static
> checks
> > > (mvn test).
> > >
> > > Indeed it can be run as a single job, but in favor of flexibility in
> > > configuration (enable / disable checks) it is OK to separate it in 2
> steps.
> > >
> > > Do you have some objections to do it that way?
> > >
> > > On Wed, Mar 3, 2021 at 8:45 PM Maxim Muzafarov 
> wrote:
> > >
> > >> Maxim,
> > >>
> > >> Can you clarify what means '[Sanity Checks] runs in parallel with
> > >> [Build]'? AFAIK the checks need the build results to run themselves.
> > >>
> > >> On Wed, 3 Mar 2021 at 18:48, Max Timonin 
> wrote:
> > >> >
> > >> > Discussed with Petr privately. Proposal is:
> > >> >
> > >> > 1. The [Build] job runs without any checks.
> > >> > 2. There will be a new job [Sanity Checks], that runs all checks -
> > >> > checkstyle, licenses, javadoc, check-suites.
> > >> > 3. [Sanity Checks] runs in parallel with [Build].
> > >> > 4. All TC jobs with tests depend on a result of the [Sanity Checks]
> > >> job. If
> > >> > the check job fails then a test job won't be started.
> > >> > 5. Users can disable the [Sanity Checks] job with a selector on the
> > >> > Parameters tab of custom TC build.
> > >> >
> > >> > If no one has objections I will create a JIRA ticket for that.
> > >> >
> > >> >
> > >> > On Wed, Mar 3, 2021 at 5:11 PM Max Timonin  >
> > >> wrote:
> > >> >
> > >> > > Hi Petr! My proposal is:
> > >> > >
> > >> > > 1. Create a parameter in [Build] TC suite - MAVEN_CHECKS, default
> > >> value is
> > >> > > "-Plicenses,checkstyle,check-licenses,check-test-suites".
> > >> > > 2. Use it in a command along with MAVEN_MODULES_STRING.
> > >> > > -U -Pall-java,all-scala,scala,lgpl,examples %MAVEN_CHECKS%
> > >> > > %MAVEN_MODULES_STRING%
> > >> > >
> > >> > > 3. Provide a global param for test suites
> "reverse.dep.MAVEN_CHECKS"
> > >> that
> > >> > > is possible to override in a custom build. If I understand it
> > >> correctly is
> > >> > > possible to do by editing the job [1].
> > >> > > 4. This param should be represented to a user as a selector with 2
> > >> > > options:
> > >> > > - default (see point 1.)
> > >> > > - "-DskipTests=true" - that ignores all checks, skip tests and
> just
> > >> build
> > >> > > a .zip of Ignite.
> > >> > >
> > >> > > Could you please review this solution? Is it OK for you?
> > >> > >
> > >> > > [1]
> > >> > >
> > >>
> https://ci.ignite.apache.org/admin/editBuildParams.html?id=template:IgniteTests24Java8_RunTestSuitesJava
> > >> > >
> > >> > > On Thu, Feb 25, 2021 at 1:47 PM Petr Ivanov  >
> > >> wrote:
> > >> > >
> > >> > >> If profile can handle this — its ok.
> > >> > >>
> > >> > >> For choosing build type — we can introduce select, that will
> choose
> > >> > >> between -p  and -DskipTests=true (defaulting to
> profile).
> > >> > >> Thus [Build] will pass either way.
> > >> > >>
> > >> > >>
> > >> > >> Regards,
> > >> > >> Petr Ivanov
> > >> > >> Head of IT
> > >> > >> IT & Development Solutions | GRIDGAIN SYSTEMS
> > >> > >> +7 (911) 945-00-59
> > >> > >>
> > >> > >> > On 25 Feb 2021, at 13:23, Max Timonin  >
> > >> wrote:
> > >> > >> >
> > >> > >> > Yes, it's correct that "mvn install" runs also the "mvn test"
> > >> command,
> > >> > >> and
> > >> > >> > this is OK as the check-test-suites profile handles all tests
> > >> > >> > without running them. If the skipTests flag is triggered then
> this
> > >> > >> check is
> > >> > >> > useless. It will take only about 2 min to run "mvn test" with
> this
> > >> > >> profile.
> > >> > >> > Travis does that as one of steps.
> > >> > >> >
> > >> > >> > So, there are no issues with tests. Should I provide more info
> how
> > >> this
> > >> > >> > check works?
> > >> > >> >
> > >> > >> > Also, discussed with Anton Vinogradov, Alex Plekhanov. There
> can
> > >> be an
> > >> > >> > issue, that sometimes it's required to run custom test suites
> to
> > >> debug
> > >> > >> > flaky tests. Sequence of steps is the following:
> > >> > >> > 1. Find a test suite with 

Re: [DISCUSS] Missed (non-suited) tests

[Maxim Muzafarov]

Max,

> There is an overhead for running git + mvn test twice, but it is a cost for 
> the flexibility.

Yes, I mean this issue. I have no objections, the build queue seems to
be empty most of the time.

On Wed, 3 Mar 2021 at 21:01, Max Timonin  wrote:
>
> Yes, they can be performed as parallel, as doesn't depend on each other.
> There is an overhead for running git + mvn test twice, but it is a cost for
> the flexibility.
>
> On Wed, Mar 3, 2021 at 8:55 PM Max Timonin  wrote:
>
> > I mean that any TC job with tests depends on both [Build], [Sanity
> > Checks]. No tests run if any of those jobs failed.
> >
> > [Build] prepares ignite.zip for distribution between TC agents (mvn
> > install).
> > [Sanity Checks] checks that code is correct in terms of our static checks
> > (mvn test).
> >
> > Indeed it can be run as a single job, but in favor of flexibility in
> > configuration (enable / disable checks) it is OK to separate it in 2 steps.
> >
> > Do you have some objections to do it that way?
> >
> > On Wed, Mar 3, 2021 at 8:45 PM Maxim Muzafarov  wrote:
> >
> >> Maxim,
> >>
> >> Can you clarify what means '[Sanity Checks] runs in parallel with
> >> [Build]'? AFAIK the checks need the build results to run themselves.
> >>
> >> On Wed, 3 Mar 2021 at 18:48, Max Timonin  wrote:
> >> >
> >> > Discussed with Petr privately. Proposal is:
> >> >
> >> > 1. The [Build] job runs without any checks.
> >> > 2. There will be a new job [Sanity Checks], that runs all checks -
> >> > checkstyle, licenses, javadoc, check-suites.
> >> > 3. [Sanity Checks] runs in parallel with [Build].
> >> > 4. All TC jobs with tests depend on a result of the [Sanity Checks]
> >> job. If
> >> > the check job fails then a test job won't be started.
> >> > 5. Users can disable the [Sanity Checks] job with a selector on the
> >> > Parameters tab of custom TC build.
> >> >
> >> > If no one has objections I will create a JIRA ticket for that.
> >> >
> >> >
> >> > On Wed, Mar 3, 2021 at 5:11 PM Max Timonin 
> >> wrote:
> >> >
> >> > > Hi Petr! My proposal is:
> >> > >
> >> > > 1. Create a parameter in [Build] TC suite - MAVEN_CHECKS, default
> >> value is
> >> > > "-Plicenses,checkstyle,check-licenses,check-test-suites".
> >> > > 2. Use it in a command along with MAVEN_MODULES_STRING.
> >> > > -U -Pall-java,all-scala,scala,lgpl,examples %MAVEN_CHECKS%
> >> > > %MAVEN_MODULES_STRING%
> >> > >
> >> > > 3. Provide a global param for test suites "reverse.dep.MAVEN_CHECKS"
> >> that
> >> > > is possible to override in a custom build. If I understand it
> >> correctly is
> >> > > possible to do by editing the job [1].
> >> > > 4. This param should be represented to a user as a selector with 2
> >> > > options:
> >> > > - default (see point 1.)
> >> > > - "-DskipTests=true" - that ignores all checks, skip tests and just
> >> build
> >> > > a .zip of Ignite.
> >> > >
> >> > > Could you please review this solution? Is it OK for you?
> >> > >
> >> > > [1]
> >> > >
> >> https://ci.ignite.apache.org/admin/editBuildParams.html?id=template:IgniteTests24Java8_RunTestSuitesJava
> >> > >
> >> > > On Thu, Feb 25, 2021 at 1:47 PM Petr Ivanov 
> >> wrote:
> >> > >
> >> > >> If profile can handle this — its ok.
> >> > >>
> >> > >> For choosing build type — we can introduce select, that will choose
> >> > >> between -p  and -DskipTests=true (defaulting to profile).
> >> > >> Thus [Build] will pass either way.
> >> > >>
> >> > >>
> >> > >> Regards,
> >> > >> Petr Ivanov
> >> > >> Head of IT
> >> > >> IT & Development Solutions | GRIDGAIN SYSTEMS
> >> > >> +7 (911) 945-00-59
> >> > >>
> >> > >> > On 25 Feb 2021, at 13:23, Max Timonin 
> >> wrote:
> >> > >> >
> >> > >> > Yes, it's correct that "mvn install" runs also the "mvn test"
> >> command,
> >> > >> and
> >> > >> > this is OK as the check-test-suites profile handles all tests
> >> > >> > without running them. If the skipTests flag is triggered then this
> >> > >> check is
> >> > >> > useless. It will take only about 2 min to run "mvn test" with this
> >> > >> profile.
> >> > >> > Travis does that as one of steps.
> >> > >> >
> >> > >> > So, there are no issues with tests. Should I provide more info how
> >> this
> >> > >> > check works?
> >> > >> >
> >> > >> > Also, discussed with Anton Vinogradov, Alex Plekhanov. There can
> >> be an
> >> > >> > issue, that sometimes it's required to run custom test suites to
> >> debug
> >> > >> > flaky tests. Sequence of steps is the following:
> >> > >> > 1. Find a test suite with flaky tests (that reproducible only on an
> >> > >> > TeamCity agent);
> >> > >> > 2. Comment some tests in the suite to isolate;
> >> > >> > 3. Push it, and run related TC suite;
> >> > >> > 4. TC suite depends on [Build] job, run the job - it will fail on
> >> the
> >> > >> check
> >> > >> > "check-test-suites".
> >> > >> >
> >> > >> > So it is needed to provide a configuration to disable this check
> >> such
> >> > >> runs.
> >> > >> > I'll have a look on next week how to implement this.
> >> > >> 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Yes, they can be performed as parallel, as doesn't depend on each other.
There is an overhead for running git + mvn test twice, but it is a cost for
the flexibility.

On Wed, Mar 3, 2021 at 8:55 PM Max Timonin  wrote:

> I mean that any TC job with tests depends on both [Build], [Sanity
> Checks]. No tests run if any of those jobs failed.
>
> [Build] prepares ignite.zip for distribution between TC agents (mvn
> install).
> [Sanity Checks] checks that code is correct in terms of our static checks
> (mvn test).
>
> Indeed it can be run as a single job, but in favor of flexibility in
> configuration (enable / disable checks) it is OK to separate it in 2 steps.
>
> Do you have some objections to do it that way?
>
> On Wed, Mar 3, 2021 at 8:45 PM Maxim Muzafarov  wrote:
>
>> Maxim,
>>
>> Can you clarify what means '[Sanity Checks] runs in parallel with
>> [Build]'? AFAIK the checks need the build results to run themselves.
>>
>> On Wed, 3 Mar 2021 at 18:48, Max Timonin  wrote:
>> >
>> > Discussed with Petr privately. Proposal is:
>> >
>> > 1. The [Build] job runs without any checks.
>> > 2. There will be a new job [Sanity Checks], that runs all checks -
>> > checkstyle, licenses, javadoc, check-suites.
>> > 3. [Sanity Checks] runs in parallel with [Build].
>> > 4. All TC jobs with tests depend on a result of the [Sanity Checks]
>> job. If
>> > the check job fails then a test job won't be started.
>> > 5. Users can disable the [Sanity Checks] job with a selector on the
>> > Parameters tab of custom TC build.
>> >
>> > If no one has objections I will create a JIRA ticket for that.
>> >
>> >
>> > On Wed, Mar 3, 2021 at 5:11 PM Max Timonin 
>> wrote:
>> >
>> > > Hi Petr! My proposal is:
>> > >
>> > > 1. Create a parameter in [Build] TC suite - MAVEN_CHECKS, default
>> value is
>> > > "-Plicenses,checkstyle,check-licenses,check-test-suites".
>> > > 2. Use it in a command along with MAVEN_MODULES_STRING.
>> > > -U -Pall-java,all-scala,scala,lgpl,examples %MAVEN_CHECKS%
>> > > %MAVEN_MODULES_STRING%
>> > >
>> > > 3. Provide a global param for test suites "reverse.dep.MAVEN_CHECKS"
>> that
>> > > is possible to override in a custom build. If I understand it
>> correctly is
>> > > possible to do by editing the job [1].
>> > > 4. This param should be represented to a user as a selector with 2
>> > > options:
>> > > - default (see point 1.)
>> > > - "-DskipTests=true" - that ignores all checks, skip tests and just
>> build
>> > > a .zip of Ignite.
>> > >
>> > > Could you please review this solution? Is it OK for you?
>> > >
>> > > [1]
>> > >
>> https://ci.ignite.apache.org/admin/editBuildParams.html?id=template:IgniteTests24Java8_RunTestSuitesJava
>> > >
>> > > On Thu, Feb 25, 2021 at 1:47 PM Petr Ivanov 
>> wrote:
>> > >
>> > >> If profile can handle this — its ok.
>> > >>
>> > >> For choosing build type — we can introduce select, that will choose
>> > >> between -p  and -DskipTests=true (defaulting to profile).
>> > >> Thus [Build] will pass either way.
>> > >>
>> > >>
>> > >> Regards,
>> > >> Petr Ivanov
>> > >> Head of IT
>> > >> IT & Development Solutions | GRIDGAIN SYSTEMS
>> > >> +7 (911) 945-00-59
>> > >>
>> > >> > On 25 Feb 2021, at 13:23, Max Timonin 
>> wrote:
>> > >> >
>> > >> > Yes, it's correct that "mvn install" runs also the "mvn test"
>> command,
>> > >> and
>> > >> > this is OK as the check-test-suites profile handles all tests
>> > >> > without running them. If the skipTests flag is triggered then this
>> > >> check is
>> > >> > useless. It will take only about 2 min to run "mvn test" with this
>> > >> profile.
>> > >> > Travis does that as one of steps.
>> > >> >
>> > >> > So, there are no issues with tests. Should I provide more info how
>> this
>> > >> > check works?
>> > >> >
>> > >> > Also, discussed with Anton Vinogradov, Alex Plekhanov. There can
>> be an
>> > >> > issue, that sometimes it's required to run custom test suites to
>> debug
>> > >> > flaky tests. Sequence of steps is the following:
>> > >> > 1. Find a test suite with flaky tests (that reproducible only on an
>> > >> > TeamCity agent);
>> > >> > 2. Comment some tests in the suite to isolate;
>> > >> > 3. Push it, and run related TC suite;
>> > >> > 4. TC suite depends on [Build] job, run the job - it will fail on
>> the
>> > >> check
>> > >> > "check-test-suites".
>> > >> >
>> > >> > So it is needed to provide a configuration to disable this check
>> such
>> > >> runs.
>> > >> > I'll have a look on next week how to implement this.
>> > >> >
>> > >> > On Thu, Feb 25, 2021 at 11:02 AM Petr Ivanov > >
>> > >> wrote:
>> > >> >
>> > >> >> I am telling that INSTALL goal for maven will trigger TEST goal
>> for the
>> > >> >> whole project and it cannot be prevented until the flag is
>> specified
>> > >> either
>> > >> >> as command line parameter, or in profile somehow to be inherited
>> by
>> > >> other
>> > >> >> modules.
>> > >> >> Thats why I am suggesting this as separate suite.
>> > >> >>
>> > >> >>
>> > >> >> Regards,
>> > >> >> 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

I mean that any TC job with tests depends on both [Build], [Sanity Checks].
No tests run if any of those jobs failed.

[Build] prepares ignite.zip for distribution between TC agents (mvn
install).
[Sanity Checks] checks that code is correct in terms of our static checks
(mvn test).

Indeed it can be run as a single job, but in favor of flexibility in
configuration (enable / disable checks) it is OK to separate it in 2 steps.

Do you have some objections to do it that way?

On Wed, Mar 3, 2021 at 8:45 PM Maxim Muzafarov  wrote:

> Maxim,
>
> Can you clarify what means '[Sanity Checks] runs in parallel with
> [Build]'? AFAIK the checks need the build results to run themselves.
>
> On Wed, 3 Mar 2021 at 18:48, Max Timonin  wrote:
> >
> > Discussed with Petr privately. Proposal is:
> >
> > 1. The [Build] job runs without any checks.
> > 2. There will be a new job [Sanity Checks], that runs all checks -
> > checkstyle, licenses, javadoc, check-suites.
> > 3. [Sanity Checks] runs in parallel with [Build].
> > 4. All TC jobs with tests depend on a result of the [Sanity Checks] job.
> If
> > the check job fails then a test job won't be started.
> > 5. Users can disable the [Sanity Checks] job with a selector on the
> > Parameters tab of custom TC build.
> >
> > If no one has objections I will create a JIRA ticket for that.
> >
> >
> > On Wed, Mar 3, 2021 at 5:11 PM Max Timonin 
> wrote:
> >
> > > Hi Petr! My proposal is:
> > >
> > > 1. Create a parameter in [Build] TC suite - MAVEN_CHECKS, default
> value is
> > > "-Plicenses,checkstyle,check-licenses,check-test-suites".
> > > 2. Use it in a command along with MAVEN_MODULES_STRING.
> > > -U -Pall-java,all-scala,scala,lgpl,examples %MAVEN_CHECKS%
> > > %MAVEN_MODULES_STRING%
> > >
> > > 3. Provide a global param for test suites "reverse.dep.MAVEN_CHECKS"
> that
> > > is possible to override in a custom build. If I understand it
> correctly is
> > > possible to do by editing the job [1].
> > > 4. This param should be represented to a user as a selector with 2
> > > options:
> > > - default (see point 1.)
> > > - "-DskipTests=true" - that ignores all checks, skip tests and just
> build
> > > a .zip of Ignite.
> > >
> > > Could you please review this solution? Is it OK for you?
> > >
> > > [1]
> > >
> https://ci.ignite.apache.org/admin/editBuildParams.html?id=template:IgniteTests24Java8_RunTestSuitesJava
> > >
> > > On Thu, Feb 25, 2021 at 1:47 PM Petr Ivanov 
> wrote:
> > >
> > >> If profile can handle this — its ok.
> > >>
> > >> For choosing build type — we can introduce select, that will choose
> > >> between -p  and -DskipTests=true (defaulting to profile).
> > >> Thus [Build] will pass either way.
> > >>
> > >>
> > >> Regards,
> > >> Petr Ivanov
> > >> Head of IT
> > >> IT & Development Solutions | GRIDGAIN SYSTEMS
> > >> +7 (911) 945-00-59
> > >>
> > >> > On 25 Feb 2021, at 13:23, Max Timonin 
> wrote:
> > >> >
> > >> > Yes, it's correct that "mvn install" runs also the "mvn test"
> command,
> > >> and
> > >> > this is OK as the check-test-suites profile handles all tests
> > >> > without running them. If the skipTests flag is triggered then this
> > >> check is
> > >> > useless. It will take only about 2 min to run "mvn test" with this
> > >> profile.
> > >> > Travis does that as one of steps.
> > >> >
> > >> > So, there are no issues with tests. Should I provide more info how
> this
> > >> > check works?
> > >> >
> > >> > Also, discussed with Anton Vinogradov, Alex Plekhanov. There can be
> an
> > >> > issue, that sometimes it's required to run custom test suites to
> debug
> > >> > flaky tests. Sequence of steps is the following:
> > >> > 1. Find a test suite with flaky tests (that reproducible only on an
> > >> > TeamCity agent);
> > >> > 2. Comment some tests in the suite to isolate;
> > >> > 3. Push it, and run related TC suite;
> > >> > 4. TC suite depends on [Build] job, run the job - it will fail on
> the
> > >> check
> > >> > "check-test-suites".
> > >> >
> > >> > So it is needed to provide a configuration to disable this check
> such
> > >> runs.
> > >> > I'll have a look on next week how to implement this.
> > >> >
> > >> > On Thu, Feb 25, 2021 at 11:02 AM Petr Ivanov 
> > >> wrote:
> > >> >
> > >> >> I am telling that INSTALL goal for maven will trigger TEST goal
> for the
> > >> >> whole project and it cannot be prevented until the flag is
> specified
> > >> either
> > >> >> as command line parameter, or in profile somehow to be inherited by
> > >> other
> > >> >> modules.
> > >> >> Thats why I am suggesting this as separate suite.
> > >> >>
> > >> >>
> > >> >> Regards,
> > >> >> *Petr Ivanov*
> > >> >> Head of IT
> > >> >> IT & Development Solutions |
> > >> >> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
> > >> >>
> > >> >> On 25 Feb 2021, at 10:44, Max Timonin 
> wrote:
> > >> >>
> > >> >> Hi, Petr!
> > >> >>
> > >> >> Profile "check-test-suites" handles all tests in another way, it
> just
> > >> >> verifies that all tests are suited. No tests run 

Re: [DISCUSS] Missed (non-suited) tests

[Maxim Muzafarov]

Maxim,

Can you clarify what means '[Sanity Checks] runs in parallel with
[Build]'? AFAIK the checks need the build results to run themselves.

On Wed, 3 Mar 2021 at 18:48, Max Timonin  wrote:
>
> Discussed with Petr privately. Proposal is:
>
> 1. The [Build] job runs without any checks.
> 2. There will be a new job [Sanity Checks], that runs all checks -
> checkstyle, licenses, javadoc, check-suites.
> 3. [Sanity Checks] runs in parallel with [Build].
> 4. All TC jobs with tests depend on a result of the [Sanity Checks] job. If
> the check job fails then a test job won't be started.
> 5. Users can disable the [Sanity Checks] job with a selector on the
> Parameters tab of custom TC build.
>
> If no one has objections I will create a JIRA ticket for that.
>
>
> On Wed, Mar 3, 2021 at 5:11 PM Max Timonin  wrote:
>
> > Hi Petr! My proposal is:
> >
> > 1. Create a parameter in [Build] TC suite - MAVEN_CHECKS, default value is
> > "-Plicenses,checkstyle,check-licenses,check-test-suites".
> > 2. Use it in a command along with MAVEN_MODULES_STRING.
> > -U -Pall-java,all-scala,scala,lgpl,examples %MAVEN_CHECKS%
> > %MAVEN_MODULES_STRING%
> >
> > 3. Provide a global param for test suites "reverse.dep.MAVEN_CHECKS" that
> > is possible to override in a custom build. If I understand it correctly is
> > possible to do by editing the job [1].
> > 4. This param should be represented to a user as a selector with 2
> > options:
> > - default (see point 1.)
> > - "-DskipTests=true" - that ignores all checks, skip tests and just build
> > a .zip of Ignite.
> >
> > Could you please review this solution? Is it OK for you?
> >
> > [1]
> > https://ci.ignite.apache.org/admin/editBuildParams.html?id=template:IgniteTests24Java8_RunTestSuitesJava
> >
> > On Thu, Feb 25, 2021 at 1:47 PM Petr Ivanov  wrote:
> >
> >> If profile can handle this — its ok.
> >>
> >> For choosing build type — we can introduce select, that will choose
> >> between -p  and -DskipTests=true (defaulting to profile).
> >> Thus [Build] will pass either way.
> >>
> >>
> >> Regards,
> >> Petr Ivanov
> >> Head of IT
> >> IT & Development Solutions | GRIDGAIN SYSTEMS
> >> +7 (911) 945-00-59
> >>
> >> > On 25 Feb 2021, at 13:23, Max Timonin  wrote:
> >> >
> >> > Yes, it's correct that "mvn install" runs also the "mvn test" command,
> >> and
> >> > this is OK as the check-test-suites profile handles all tests
> >> > without running them. If the skipTests flag is triggered then this
> >> check is
> >> > useless. It will take only about 2 min to run "mvn test" with this
> >> profile.
> >> > Travis does that as one of steps.
> >> >
> >> > So, there are no issues with tests. Should I provide more info how this
> >> > check works?
> >> >
> >> > Also, discussed with Anton Vinogradov, Alex Plekhanov. There can be an
> >> > issue, that sometimes it's required to run custom test suites to debug
> >> > flaky tests. Sequence of steps is the following:
> >> > 1. Find a test suite with flaky tests (that reproducible only on an
> >> > TeamCity agent);
> >> > 2. Comment some tests in the suite to isolate;
> >> > 3. Push it, and run related TC suite;
> >> > 4. TC suite depends on [Build] job, run the job - it will fail on the
> >> check
> >> > "check-test-suites".
> >> >
> >> > So it is needed to provide a configuration to disable this check such
> >> runs.
> >> > I'll have a look on next week how to implement this.
> >> >
> >> > On Thu, Feb 25, 2021 at 11:02 AM Petr Ivanov 
> >> wrote:
> >> >
> >> >> I am telling that INSTALL goal for maven will trigger TEST goal for the
> >> >> whole project and it cannot be prevented until the flag is specified
> >> either
> >> >> as command line parameter, or in profile somehow to be inherited by
> >> other
> >> >> modules.
> >> >> Thats why I am suggesting this as separate suite.
> >> >>
> >> >>
> >> >> Regards,
> >> >> *Petr Ivanov*
> >> >> Head of IT
> >> >> IT & Development Solutions |
> >> >> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
> >> >>
> >> >> On 25 Feb 2021, at 10:44, Max Timonin  wrote:
> >> >>
> >> >> Hi, Petr!
> >> >>
> >> >> Profile "check-test-suites" handles all tests in another way, it just
> >> >> verifies that all tests are suited. No tests run then.
> >> >> As I understand the [BUILD] job goal is preparing a .zip archive to
> >> >> distribute it for other jobs. I think it is a valid place to put sanity
> >> >> checks. If a check fails then no archive is prepared. WDYT?
> >> >>
> >> >> Also I see that there is a flag -Dmaven.javadoc.skip=true. I'd propose
> >> to
> >> >> change it to the profile "skip-docs", that was introduced in ticket [1]
> >> >> IGNITE-13623. As the setting "maven.javadoc.skip" does not
> >> >> affect scaladocs.
> >> >>
> >> >> [1] https://issues.apache.org/jira/browse/IGNITE-13623
> >> >>
> >> >> On Thu, Feb 25, 2021 at 7:34 AM Petr Ivanov 
> >> wrote:
> >> >>
> >> >>> Won't the absence of -DskipTests flag trigger ALL the tests for all
> >> >>> modules?
> >> >>> This flag was added intentionally.
> >> >>>
> 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Discussed with Petr privately. Proposal is:

1. The [Build] job runs without any checks.
2. There will be a new job [Sanity Checks], that runs all checks -
checkstyle, licenses, javadoc, check-suites.
3. [Sanity Checks] runs in parallel with [Build].
4. All TC jobs with tests depend on a result of the [Sanity Checks] job. If
the check job fails then a test job won't be started.
5. Users can disable the [Sanity Checks] job with a selector on the
Parameters tab of custom TC build.

If no one has objections I will create a JIRA ticket for that.


On Wed, Mar 3, 2021 at 5:11 PM Max Timonin  wrote:

> Hi Petr! My proposal is:
>
> 1. Create a parameter in [Build] TC suite - MAVEN_CHECKS, default value is
> "-Plicenses,checkstyle,check-licenses,check-test-suites".
> 2. Use it in a command along with MAVEN_MODULES_STRING.
> -U -Pall-java,all-scala,scala,lgpl,examples %MAVEN_CHECKS%
> %MAVEN_MODULES_STRING%
>
> 3. Provide a global param for test suites "reverse.dep.MAVEN_CHECKS" that
> is possible to override in a custom build. If I understand it correctly is
> possible to do by editing the job [1].
> 4. This param should be represented to a user as a selector with 2
> options:
> - default (see point 1.)
> - "-DskipTests=true" - that ignores all checks, skip tests and just build
> a .zip of Ignite.
>
> Could you please review this solution? Is it OK for you?
>
> [1]
> https://ci.ignite.apache.org/admin/editBuildParams.html?id=template:IgniteTests24Java8_RunTestSuitesJava
>
> On Thu, Feb 25, 2021 at 1:47 PM Petr Ivanov  wrote:
>
>> If profile can handle this — its ok.
>>
>> For choosing build type — we can introduce select, that will choose
>> between -p  and -DskipTests=true (defaulting to profile).
>> Thus [Build] will pass either way.
>>
>>
>> Regards,
>> Petr Ivanov
>> Head of IT
>> IT & Development Solutions | GRIDGAIN SYSTEMS
>> +7 (911) 945-00-59
>>
>> > On 25 Feb 2021, at 13:23, Max Timonin  wrote:
>> >
>> > Yes, it's correct that "mvn install" runs also the "mvn test" command,
>> and
>> > this is OK as the check-test-suites profile handles all tests
>> > without running them. If the skipTests flag is triggered then this
>> check is
>> > useless. It will take only about 2 min to run "mvn test" with this
>> profile.
>> > Travis does that as one of steps.
>> >
>> > So, there are no issues with tests. Should I provide more info how this
>> > check works?
>> >
>> > Also, discussed with Anton Vinogradov, Alex Plekhanov. There can be an
>> > issue, that sometimes it's required to run custom test suites to debug
>> > flaky tests. Sequence of steps is the following:
>> > 1. Find a test suite with flaky tests (that reproducible only on an
>> > TeamCity agent);
>> > 2. Comment some tests in the suite to isolate;
>> > 3. Push it, and run related TC suite;
>> > 4. TC suite depends on [Build] job, run the job - it will fail on the
>> check
>> > "check-test-suites".
>> >
>> > So it is needed to provide a configuration to disable this check such
>> runs.
>> > I'll have a look on next week how to implement this.
>> >
>> > On Thu, Feb 25, 2021 at 11:02 AM Petr Ivanov 
>> wrote:
>> >
>> >> I am telling that INSTALL goal for maven will trigger TEST goal for the
>> >> whole project and it cannot be prevented until the flag is specified
>> either
>> >> as command line parameter, or in profile somehow to be inherited by
>> other
>> >> modules.
>> >> Thats why I am suggesting this as separate suite.
>> >>
>> >>
>> >> Regards,
>> >> *Petr Ivanov*
>> >> Head of IT
>> >> IT & Development Solutions |
>> >> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>> >>
>> >> On 25 Feb 2021, at 10:44, Max Timonin  wrote:
>> >>
>> >> Hi, Petr!
>> >>
>> >> Profile "check-test-suites" handles all tests in another way, it just
>> >> verifies that all tests are suited. No tests run then.
>> >> As I understand the [BUILD] job goal is preparing a .zip archive to
>> >> distribute it for other jobs. I think it is a valid place to put sanity
>> >> checks. If a check fails then no archive is prepared. WDYT?
>> >>
>> >> Also I see that there is a flag -Dmaven.javadoc.skip=true. I'd propose
>> to
>> >> change it to the profile "skip-docs", that was introduced in ticket [1]
>> >> IGNITE-13623. As the setting "maven.javadoc.skip" does not
>> >> affect scaladocs.
>> >>
>> >> [1] https://issues.apache.org/jira/browse/IGNITE-13623
>> >>
>> >> On Thu, Feb 25, 2021 at 7:34 AM Petr Ivanov 
>> wrote:
>> >>
>> >>> Won't the absence of -DskipTests flag trigger ALL the tests for all
>> >>> modules?
>> >>> This flag was added intentionally.
>> >>>
>> >>> Instead, I'd put Non-Suited tests into some kind of sanity check,
>> group
>> >>> all sanity checks in single Run All, and make tests depend on it's
>> >>> successful pass.
>> >>>
>> >>>
>> >>> Regards,
>> >>> *Petr Ivanov*
>> >>> Head of IT
>> >>> IT & Development Solutions |
>> >>> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>> >>>
>> >>> On 24 Feb 2021, at 19:58, Max Timonin 
>> wrote:
>> >>>
>> >>> Hi, all!
>> >>>
>> >>> What do you 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi Petr! My proposal is:

1. Create a parameter in [Build] TC suite - MAVEN_CHECKS, default value is
"-Plicenses,checkstyle,check-licenses,check-test-suites".
2. Use it in a command along with MAVEN_MODULES_STRING.
-U -Pall-java,all-scala,scala,lgpl,examples %MAVEN_CHECKS%
%MAVEN_MODULES_STRING%

3. Provide a global param for test suites "reverse.dep.MAVEN_CHECKS" that
is possible to override in a custom build. If I understand it correctly is
possible to do by editing the job [1].
4. This param should be represented to a user as a selector with 2 options:
- default (see point 1.)
- "-DskipTests=true" - that ignores all checks, skip tests and just build a
.zip of Ignite.

Could you please review this solution? Is it OK for you?

[1]
https://ci.ignite.apache.org/admin/editBuildParams.html?id=template:IgniteTests24Java8_RunTestSuitesJava

On Thu, Feb 25, 2021 at 1:47 PM Petr Ivanov  wrote:

> If profile can handle this — its ok.
>
> For choosing build type — we can introduce select, that will choose
> between -p  and -DskipTests=true (defaulting to profile).
> Thus [Build] will pass either way.
>
>
> Regards,
> Petr Ivanov
> Head of IT
> IT & Development Solutions | GRIDGAIN SYSTEMS
> +7 (911) 945-00-59
>
> > On 25 Feb 2021, at 13:23, Max Timonin  wrote:
> >
> > Yes, it's correct that "mvn install" runs also the "mvn test" command,
> and
> > this is OK as the check-test-suites profile handles all tests
> > without running them. If the skipTests flag is triggered then this check
> is
> > useless. It will take only about 2 min to run "mvn test" with this
> profile.
> > Travis does that as one of steps.
> >
> > So, there are no issues with tests. Should I provide more info how this
> > check works?
> >
> > Also, discussed with Anton Vinogradov, Alex Plekhanov. There can be an
> > issue, that sometimes it's required to run custom test suites to debug
> > flaky tests. Sequence of steps is the following:
> > 1. Find a test suite with flaky tests (that reproducible only on an
> > TeamCity agent);
> > 2. Comment some tests in the suite to isolate;
> > 3. Push it, and run related TC suite;
> > 4. TC suite depends on [Build] job, run the job - it will fail on the
> check
> > "check-test-suites".
> >
> > So it is needed to provide a configuration to disable this check such
> runs.
> > I'll have a look on next week how to implement this.
> >
> > On Thu, Feb 25, 2021 at 11:02 AM Petr Ivanov 
> wrote:
> >
> >> I am telling that INSTALL goal for maven will trigger TEST goal for the
> >> whole project and it cannot be prevented until the flag is specified
> either
> >> as command line parameter, or in profile somehow to be inherited by
> other
> >> modules.
> >> Thats why I am suggesting this as separate suite.
> >>
> >>
> >> Regards,
> >> *Petr Ivanov*
> >> Head of IT
> >> IT & Development Solutions |
> >> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
> >>
> >> On 25 Feb 2021, at 10:44, Max Timonin  wrote:
> >>
> >> Hi, Petr!
> >>
> >> Profile "check-test-suites" handles all tests in another way, it just
> >> verifies that all tests are suited. No tests run then.
> >> As I understand the [BUILD] job goal is preparing a .zip archive to
> >> distribute it for other jobs. I think it is a valid place to put sanity
> >> checks. If a check fails then no archive is prepared. WDYT?
> >>
> >> Also I see that there is a flag -Dmaven.javadoc.skip=true. I'd propose
> to
> >> change it to the profile "skip-docs", that was introduced in ticket [1]
> >> IGNITE-13623. As the setting "maven.javadoc.skip" does not
> >> affect scaladocs.
> >>
> >> [1] https://issues.apache.org/jira/browse/IGNITE-13623
> >>
> >> On Thu, Feb 25, 2021 at 7:34 AM Petr Ivanov 
> wrote:
> >>
> >>> Won't the absence of -DskipTests flag trigger ALL the tests for all
> >>> modules?
> >>> This flag was added intentionally.
> >>>
> >>> Instead, I'd put Non-Suited tests into some kind of sanity check, group
> >>> all sanity checks in single Run All, and make tests depend on it's
> >>> successful pass.
> >>>
> >>>
> >>> Regards,
> >>> *Petr Ivanov*
> >>> Head of IT
> >>> IT & Development Solutions |
> >>> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
> >>>
> >>> On 24 Feb 2021, at 19:58, Max Timonin  wrote:
> >>>
> >>> Hi, all!
> >>>
> >>> What do you think if we add the check in the TC [Build] job. Currently
> >>> [Build] runs also check for licences, checkstyle [1]:
> >>>
> >>> mvn *install* -Pall-java,all-scala,scala,*licenses*,lgpl,examples,
> >>> *checkstyle* -DskipTests -Dmaven.javadoc.skip=true
> >>> %MAVEN_MODULES_STRING%.
> >>>
> >>> So let's add the check too to block other jobs. As if there missed
> tests
> >>> then TC run may be invalid - missed tests may be broken and then the
> MTCGA
> >>> visa too. To made this we should change command line parameters:
> >>> 1. Add profile check-test-suites;
> >>> 2. Remove -Dskiptests flag.
> >>>
> >>> -Pall-java,all-scala,scala,licenses,lgpl,examples,checkstyle,
> >>> *check-test-suites *-DskipTests -Dmaven.javadoc.skip=true
> >>> 

Re: [DISCUSS] Missed (non-suited) tests

[Petr Ivanov]

If profile can handle this — its ok.

For choosing build type — we can introduce select, that will choose between -p 
 and -DskipTests=true (defaulting to profile).
Thus [Build] will pass either way.


Regards,
Petr Ivanov
Head of IT
IT & Development Solutions | GRIDGAIN SYSTEMS
+7 (911) 945-00-59

> On 25 Feb 2021, at 13:23, Max Timonin  wrote:
> 
> Yes, it's correct that "mvn install" runs also the "mvn test" command, and
> this is OK as the check-test-suites profile handles all tests
> without running them. If the skipTests flag is triggered then this check is
> useless. It will take only about 2 min to run "mvn test" with this profile.
> Travis does that as one of steps.
> 
> So, there are no issues with tests. Should I provide more info how this
> check works?
> 
> Also, discussed with Anton Vinogradov, Alex Plekhanov. There can be an
> issue, that sometimes it's required to run custom test suites to debug
> flaky tests. Sequence of steps is the following:
> 1. Find a test suite with flaky tests (that reproducible only on an
> TeamCity agent);
> 2. Comment some tests in the suite to isolate;
> 3. Push it, and run related TC suite;
> 4. TC suite depends on [Build] job, run the job - it will fail on the check
> "check-test-suites".
> 
> So it is needed to provide a configuration to disable this check such runs.
> I'll have a look on next week how to implement this.
> 
> On Thu, Feb 25, 2021 at 11:02 AM Petr Ivanov  wrote:
> 
>> I am telling that INSTALL goal for maven will trigger TEST goal for the
>> whole project and it cannot be prevented until the flag is specified either
>> as command line parameter, or in profile somehow to be inherited by other
>> modules.
>> Thats why I am suggesting this as separate suite.
>> 
>> 
>> Regards,
>> *Petr Ivanov*
>> Head of IT
>> IT & Development Solutions |
>> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>> 
>> On 25 Feb 2021, at 10:44, Max Timonin  wrote:
>> 
>> Hi, Petr!
>> 
>> Profile "check-test-suites" handles all tests in another way, it just
>> verifies that all tests are suited. No tests run then.
>> As I understand the [BUILD] job goal is preparing a .zip archive to
>> distribute it for other jobs. I think it is a valid place to put sanity
>> checks. If a check fails then no archive is prepared. WDYT?
>> 
>> Also I see that there is a flag -Dmaven.javadoc.skip=true. I'd propose to
>> change it to the profile "skip-docs", that was introduced in ticket [1]
>> IGNITE-13623. As the setting "maven.javadoc.skip" does not
>> affect scaladocs.
>> 
>> [1] https://issues.apache.org/jira/browse/IGNITE-13623
>> 
>> On Thu, Feb 25, 2021 at 7:34 AM Petr Ivanov  wrote:
>> 
>>> Won't the absence of -DskipTests flag trigger ALL the tests for all
>>> modules?
>>> This flag was added intentionally.
>>> 
>>> Instead, I'd put Non-Suited tests into some kind of sanity check, group
>>> all sanity checks in single Run All, and make tests depend on it's
>>> successful pass.
>>> 
>>> 
>>> Regards,
>>> *Petr Ivanov*
>>> Head of IT
>>> IT & Development Solutions |
>>> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>>> 
>>> On 24 Feb 2021, at 19:58, Max Timonin  wrote:
>>> 
>>> Hi, all!
>>> 
>>> What do you think if we add the check in the TC [Build] job. Currently
>>> [Build] runs also check for licences, checkstyle [1]:
>>> 
>>> mvn *install* -Pall-java,all-scala,scala,*licenses*,lgpl,examples,
>>> *checkstyle* -DskipTests -Dmaven.javadoc.skip=true
>>> %MAVEN_MODULES_STRING%.
>>> 
>>> So let's add the check too to block other jobs. As if there missed tests
>>> then TC run may be invalid - missed tests may be broken and then the MTCGA
>>> visa too. To made this we should change command line parameters:
>>> 1. Add profile check-test-suites;
>>> 2. Remove -Dskiptests flag.
>>> 
>>> -Pall-java,all-scala,scala,licenses,lgpl,examples,checkstyle,
>>> *check-test-suites *-DskipTests -Dmaven.javadoc.skip=true
>>> %MAVEN_MODULES_STRING%
>>> 
>>> WDYT?
>>> 
>>> [1]
>>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_BuildApacheIgnite=buildTypeSettings_IgniteTests24Java8=%3Cdefault
>>> 
>>> On Tue, Feb 9, 2021 at 4:48 PM Ilya Kasnacheev 
>>> wrote:
>>> 
 Hello again!
 
 Of course it's 20 minutes, not 20 seconds.
 
 Regards,
 --
 Ilya Kasnacheev
 
 
 вт, 9 февр. 2021 г. в 16:45, Ilya Kasnacheev  :
 
> Hello!
> 
> Java part kicks in if the target not found in pom.xml. Ideally we
 should
> skip this build if target check-test-suites is not in pom.xml
> 
> I have changed its timeout to 20 second which should terminate its
> progression on older builds. Maybe that would be sufficient for now.
> 
> Regards,
> --
> Ilya Kasnacheev
> 
> 
> вт, 9 февр. 2021 г. в 14:09, Petr Ivanov :
> 
>> As much as I understood — we execute internal class as plugin, where
 all
>> the magic is done.
>> Seems pretty solid in Maven part. Java part, unfortunately, cannot
 check.

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Yes, it's correct that "mvn install" runs also the "mvn test" command, and
this is OK as the check-test-suites profile handles all tests
without running them. If the skipTests flag is triggered then this check is
useless. It will take only about 2 min to run "mvn test" with this profile.
Travis does that as one of steps.

So, there are no issues with tests. Should I provide more info how this
check works?

Also, discussed with Anton Vinogradov, Alex Plekhanov. There can be an
issue, that sometimes it's required to run custom test suites to debug
flaky tests. Sequence of steps is the following:
1. Find a test suite with flaky tests (that reproducible only on an
TeamCity agent);
2. Comment some tests in the suite to isolate;
3. Push it, and run related TC suite;
4. TC suite depends on [Build] job, run the job - it will fail on the check
"check-test-suites".

So it is needed to provide a configuration to disable this check such runs.
I'll have a look on next week how to implement this.

On Thu, Feb 25, 2021 at 11:02 AM Petr Ivanov  wrote:

> I am telling that INSTALL goal for maven will trigger TEST goal for the
> whole project and it cannot be prevented until the flag is specified either
> as command line parameter, or in profile somehow to be inherited by other
> modules.
> Thats why I am suggesting this as separate suite.
>
>
> Regards,
> *Petr Ivanov*
> Head of IT
> IT & Development Solutions |
> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>
> On 25 Feb 2021, at 10:44, Max Timonin  wrote:
>
> Hi, Petr!
>
> Profile "check-test-suites" handles all tests in another way, it just
> verifies that all tests are suited. No tests run then.
> As I understand the [BUILD] job goal is preparing a .zip archive to
> distribute it for other jobs. I think it is a valid place to put sanity
> checks. If a check fails then no archive is prepared. WDYT?
>
> Also I see that there is a flag -Dmaven.javadoc.skip=true. I'd propose to
> change it to the profile "skip-docs", that was introduced in ticket [1]
> IGNITE-13623. As the setting "maven.javadoc.skip" does not
> affect scaladocs.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-13623
>
> On Thu, Feb 25, 2021 at 7:34 AM Petr Ivanov  wrote:
>
>> Won't the absence of -DskipTests flag trigger ALL the tests for all
>> modules?
>> This flag was added intentionally.
>>
>> Instead, I'd put Non-Suited tests into some kind of sanity check, group
>> all sanity checks in single Run All, and make tests depend on it's
>> successful pass.
>>
>>
>> Regards,
>> *Petr Ivanov*
>> Head of IT
>> IT & Development Solutions |
>> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>>
>> On 24 Feb 2021, at 19:58, Max Timonin  wrote:
>>
>> Hi, all!
>>
>> What do you think if we add the check in the TC [Build] job. Currently
>> [Build] runs also check for licences, checkstyle [1]:
>>
>> mvn *install* -Pall-java,all-scala,scala,*licenses*,lgpl,examples,
>> *checkstyle* -DskipTests -Dmaven.javadoc.skip=true
>> %MAVEN_MODULES_STRING%.
>>
>> So let's add the check too to block other jobs. As if there missed tests
>> then TC run may be invalid - missed tests may be broken and then the MTCGA
>> visa too. To made this we should change command line parameters:
>> 1. Add profile check-test-suites;
>> 2. Remove -Dskiptests flag.
>>
>> -Pall-java,all-scala,scala,licenses,lgpl,examples,checkstyle,
>> *check-test-suites *-DskipTests -Dmaven.javadoc.skip=true
>> %MAVEN_MODULES_STRING%
>>
>> WDYT?
>>
>> [1]
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_BuildApacheIgnite=buildTypeSettings_IgniteTests24Java8=%3Cdefault
>>
>> On Tue, Feb 9, 2021 at 4:48 PM Ilya Kasnacheev 
>> wrote:
>>
>>> Hello again!
>>>
>>> Of course it's 20 minutes, not 20 seconds.
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> вт, 9 февр. 2021 г. в 16:45, Ilya Kasnacheev >> >:
>>>
>>> > Hello!
>>> >
>>> > Java part kicks in if the target not found in pom.xml. Ideally we
>>> should
>>> > skip this build if target check-test-suites is not in pom.xml
>>> >
>>> > I have changed its timeout to 20 second which should terminate its
>>> > progression on older builds. Maybe that would be sufficient for now.
>>> >
>>> > Regards,
>>> > --
>>> > Ilya Kasnacheev
>>> >
>>> >
>>> > вт, 9 февр. 2021 г. в 14:09, Petr Ivanov :
>>> >
>>> >> As much as I understood — we execute internal class as plugin, where
>>> all
>>> >> the magic is done.
>>> >> Seems pretty solid in Maven part. Java part, unfortunately, cannot
>>> check.
>>> >>
>>> >>
>>> >>
>>> >> Regards,
>>> >> *Petr Ivanov*
>>> >> Head of IT
>>> >> IT & Development Solutions |
>>> >> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>>> >>
>>> >> On 9 Feb 2021, at 12:32, Ilya Kasnacheev 
>>> >> wrote:
>>> >>
>>> >> Hello Peter,
>>> >>
>>> >> Thanks for chiming in. The code is under
>>> >> https://github.com/apache/ignite/pull/8367
>>> >>
>>> >> Regards,
>>> >> --
>>> >> Ilya Kasnacheev
>>> >>
>>> >>
>>> >> вт, 9 февр. 2021 г. в 12:20, Petr Ivanov :
>>> >>
>>> >>> Hi, Ilya.
>>> >>>
>>> >>>
>>> >>> 

Re: [DISCUSS] Missed (non-suited) tests

[Petr Ivanov]

I am telling that INSTALL goal for maven will trigger TEST goal for the whole 
project and it cannot be prevented until the flag is specified either as 
command line parameter, or in profile somehow to be inherited by other modules.
Thats why I am suggesting this as separate suite.


Regards,
Petr Ivanov
Head of IT
IT & Development Solutions | GRIDGAIN SYSTEMS
+7 (911) 945-00-59

> On 25 Feb 2021, at 10:44, Max Timonin  wrote:
> 
> Hi, Petr!
> 
> Profile "check-test-suites" handles all tests in another way, it just 
> verifies that all tests are suited. No tests run then. 
> As I understand the [BUILD] job goal is preparing a .zip archive to 
> distribute it for other jobs. I think it is a valid place to put sanity 
> checks. If a check fails then no archive is prepared. WDYT?
> 
> Also I see that there is a flag -Dmaven.javadoc.skip=true. I'd propose to 
> change it to the profile "skip-docs", that was introduced in ticket [1] 
> IGNITE-13623. As the setting "maven.javadoc.skip" does not affect scaladocs. 
> 
> [1] https://issues.apache.org/jira/browse/IGNITE-13623 
> 
> On Thu, Feb 25, 2021 at 7:34 AM Petr Ivanov  > wrote:
> Won't the absence of -DskipTests flag trigger ALL the tests for all modules?
> This flag was added intentionally.
> 
> Instead, I'd put Non-Suited tests into some kind of sanity check, group all 
> sanity checks in single Run All, and make tests depend on it's successful 
> pass.
> 
> 
> Regards,
> Petr Ivanov
> Head of IT
> IT & Development Solutions | GRIDGAIN SYSTEMS
> +7 (911) 945-00-59
> 
>> On 24 Feb 2021, at 19:58, Max Timonin > > wrote:
>> 
>> Hi, all!
>> 
>> What do you think if we add the check in the TC [Build] job. Currently 
>> [Build] runs also check for licences, checkstyle [1]:
>> 
>> mvn install -Pall-java,all-scala,scala,licenses,lgpl,examples,checkstyle 
>> -DskipTests -Dmaven.javadoc.skip=true %MAVEN_MODULES_STRING%.
>> 
>> So let's add the check too to block other jobs. As if there missed tests 
>> then TC run may be invalid - missed tests may be broken and then the MTCGA 
>> visa too. To made this we should change command line parameters:
>> 1. Add profile check-test-suites;
>> 2. Remove -Dskiptests flag.
>> 
>> -Pall-java,all-scala,scala,licenses,lgpl,examples,checkstyle,check-test-suites
>>  -DskipTests -Dmaven.javadoc.skip=true %MAVEN_MODULES_STRING%
>> 
>> WDYT?
>> 
>> [1] 
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_BuildApacheIgnite=buildTypeSettings_IgniteTests24Java8=%3Cdefault
>>  
>> 
>> On Tue, Feb 9, 2021 at 4:48 PM Ilya Kasnacheev > > wrote:
>> Hello again!
>> 
>> Of course it's 20 minutes, not 20 seconds.
>> 
>> Regards,
>> -- 
>> Ilya Kasnacheev
>> 
>> 
>> вт, 9 февр. 2021 г. в 16:45, Ilya Kasnacheev > >:
>> 
>> > Hello!
>> >
>> > Java part kicks in if the target not found in pom.xml. Ideally we should
>> > skip this build if target check-test-suites is not in pom.xml
>> >
>> > I have changed its timeout to 20 second which should terminate its
>> > progression on older builds. Maybe that would be sufficient for now.
>> >
>> > Regards,
>> > --
>> > Ilya Kasnacheev
>> >
>> >
>> > вт, 9 февр. 2021 г. в 14:09, Petr Ivanov > > >:
>> >
>> >> As much as I understood — we execute internal class as plugin, where all
>> >> the magic is done.
>> >> Seems pretty solid in Maven part. Java part, unfortunately, cannot check.
>> >>
>> >>
>> >>
>> >> Regards,
>> >> *Petr Ivanov*
>> >> Head of IT
>> >> IT & Development Solutions |
>> >> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>> >>
>> >> On 9 Feb 2021, at 12:32, Ilya Kasnacheev > >> >
>> >> wrote:
>> >>
>> >> Hello Peter,
>> >>
>> >> Thanks for chiming in. The code is under
>> >> https://github.com/apache/ignite/pull/8367 
>> >> 
>> >>
>> >> Regards,
>> >> --
>> >> Ilya Kasnacheev
>> >>
>> >>
>> >> вт, 9 февр. 2021 г. в 12:20, Petr Ivanov > >> >:
>> >>
>> >>> Hi, Ilya.
>> >>>
>> >>>
>> >>> I've added Inspections to Run All.
>> >>> Checkstyle is currently integrated to Build and can be deleted.
>> >>>
>> >>>
>> >>> Where can I find the code for check-test-suites profile?
>> >>>
>> >>>
>> >>> Regards,
>> >>> *Petr Ivanov*
>> >>> Head of IT
>> >>> IT & Development Solutions |
>> >>> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>> >>>
>> >>> On 9 Feb 2021, at 12:16, Ilya Kasnacheev > >>> >
>> >>> wrote:
>> >>>
>> >>> Hello!
>> >>>
>> >>> I have found one issue where it would cause tests to be run if the
>> >>> change is not present in the target branch.
>> >>>
>> >>> This includes e.g. 2.10 nightlies.
>> >>>
>> 

Re: [DISCUSS] Missed (non-suited) tests

[Petr Ivanov]

Won't the absence of -DskipTests flag trigger ALL the tests for all modules?
This flag was added intentionally.

Instead, I'd put Non-Suited tests into some kind of sanity check, group all 
sanity checks in single Run All, and make tests depend on it's successful pass.


Regards,
Petr Ivanov
Head of IT
IT & Development Solutions | GRIDGAIN SYSTEMS
+7 (911) 945-00-59

> On 24 Feb 2021, at 19:58, Max Timonin  wrote:
> 
> Hi, all!
> 
> What do you think if we add the check in the TC [Build] job. Currently 
> [Build] runs also check for licences, checkstyle [1]:
> 
> mvn install -Pall-java,all-scala,scala,licenses,lgpl,examples,checkstyle 
> -DskipTests -Dmaven.javadoc.skip=true %MAVEN_MODULES_STRING%.
> 
> So let's add the check too to block other jobs. As if there missed tests then 
> TC run may be invalid - missed tests may be broken and then the MTCGA visa 
> too. To made this we should change command line parameters:
> 1. Add profile check-test-suites;
> 2. Remove -Dskiptests flag.
> 
> -Pall-java,all-scala,scala,licenses,lgpl,examples,checkstyle,check-test-suites
>  -DskipTests -Dmaven.javadoc.skip=true %MAVEN_MODULES_STRING%
> 
> WDYT?
> 
> [1] 
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_BuildApacheIgnite=buildTypeSettings_IgniteTests24Java8=%3Cdefault
>  
> 
> On Tue, Feb 9, 2021 at 4:48 PM Ilya Kasnacheev  > wrote:
> Hello again!
> 
> Of course it's 20 minutes, not 20 seconds.
> 
> Regards,
> -- 
> Ilya Kasnacheev
> 
> 
> вт, 9 февр. 2021 г. в 16:45, Ilya Kasnacheev  >:
> 
> > Hello!
> >
> > Java part kicks in if the target not found in pom.xml. Ideally we should
> > skip this build if target check-test-suites is not in pom.xml
> >
> > I have changed its timeout to 20 second which should terminate its
> > progression on older builds. Maybe that would be sufficient for now.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > вт, 9 февр. 2021 г. в 14:09, Petr Ivanov  > >:
> >
> >> As much as I understood — we execute internal class as plugin, where all
> >> the magic is done.
> >> Seems pretty solid in Maven part. Java part, unfortunately, cannot check.
> >>
> >>
> >>
> >> Regards,
> >> *Petr Ivanov*
> >> Head of IT
> >> IT & Development Solutions |
> >> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
> >>
> >> On 9 Feb 2021, at 12:32, Ilya Kasnacheev  >> >
> >> wrote:
> >>
> >> Hello Peter,
> >>
> >> Thanks for chiming in. The code is under
> >> https://github.com/apache/ignite/pull/8367 
> >> 
> >>
> >> Regards,
> >> --
> >> Ilya Kasnacheev
> >>
> >>
> >> вт, 9 февр. 2021 г. в 12:20, Petr Ivanov  >> >:
> >>
> >>> Hi, Ilya.
> >>>
> >>>
> >>> I've added Inspections to Run All.
> >>> Checkstyle is currently integrated to Build and can be deleted.
> >>>
> >>>
> >>> Where can I find the code for check-test-suites profile?
> >>>
> >>>
> >>> Regards,
> >>> *Petr Ivanov*
> >>> Head of IT
> >>> IT & Development Solutions |
> >>> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
> >>>
> >>> On 9 Feb 2021, at 12:16, Ilya Kasnacheev  >>> >
> >>> wrote:
> >>>
> >>> Hello!
> >>>
> >>> I have found one issue where it would cause tests to be run if the
> >>> change is not present in the target branch.
> >>>
> >>> This includes e.g. 2.10 nightlies.
> >>>
> >>> What can we do to avoid that? Is specifying -DskipTests sufficient? Any
> >>> chance that it will break the missed tests check?
> >>>
> >>> Regards,
> >>> --
> >>> Ilya Kasnacheev
> >>>
> >>>
> >>> пн, 8 февр. 2021 г. в 14:13, Ilya Kasnacheev  >>> 
> >>> >:
> >>>
>  Hello!
> 
>  I have created a TC suite:
> 
>  https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_MissingTests?mode=builds
>   
>  
> 
>  + Peter Ivanov
> 
>  Can you please check if everything is alright?
> 
>  BTW, it seems that Inspections [Core] is only in Run All Basic (but not
>  in Run All), and Check Code Style is not triggered by either one. Is it
>  correct?
> 
>  Regards,
>  --
>  Ilya Kasnacheev
> 
> 
>  пн, 8 февр. 2021 г. в 10:22, Max Timonin   >:
> 
> > Hi!
> >
> > Yes, now it's a part of the Travis check, and there is already a first
> > successful build [1]. But I think it's also required to run the check
> > on TC
> > too, along with jobs for checking licenses, code style, and core
> > inspections.
> >
> >
> > [1] https://travis-ci.com/github/apache/ignite/builds/216363067 
> > 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi, Petr!

Profile "check-test-suites" handles all tests in another way, it just
verifies that all tests are suited. No tests run then.
As I understand the [BUILD] job goal is preparing a .zip archive to
distribute it for other jobs. I think it is a valid place to put sanity
checks. If a check fails then no archive is prepared. WDYT?

Also I see that there is a flag -Dmaven.javadoc.skip=true. I'd propose to
change it to the profile "skip-docs", that was introduced in ticket [1]
IGNITE-13623. As the setting "maven.javadoc.skip" does not
affect scaladocs.

[1] https://issues.apache.org/jira/browse/IGNITE-13623

On Thu, Feb 25, 2021 at 7:34 AM Petr Ivanov  wrote:

> Won't the absence of -DskipTests flag trigger ALL the tests for all
> modules?
> This flag was added intentionally.
>
> Instead, I'd put Non-Suited tests into some kind of sanity check, group
> all sanity checks in single Run All, and make tests depend on it's
> successful pass.
>
>
> Regards,
> *Petr Ivanov*
> Head of IT
> IT & Development Solutions |
> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>
> On 24 Feb 2021, at 19:58, Max Timonin  wrote:
>
> Hi, all!
>
> What do you think if we add the check in the TC [Build] job. Currently
> [Build] runs also check for licences, checkstyle [1]:
>
> mvn *install* -Pall-java,all-scala,scala,*licenses*,lgpl,examples,
> *checkstyle* -DskipTests -Dmaven.javadoc.skip=true %MAVEN_MODULES_STRING%.
>
> So let's add the check too to block other jobs. As if there missed tests
> then TC run may be invalid - missed tests may be broken and then the MTCGA
> visa too. To made this we should change command line parameters:
> 1. Add profile check-test-suites;
> 2. Remove -Dskiptests flag.
>
> -Pall-java,all-scala,scala,licenses,lgpl,examples,checkstyle,
> *check-test-suites *-DskipTests -Dmaven.javadoc.skip=true
> %MAVEN_MODULES_STRING%
>
> WDYT?
>
> [1]
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_BuildApacheIgnite=buildTypeSettings_IgniteTests24Java8=%3Cdefault
>
> On Tue, Feb 9, 2021 at 4:48 PM Ilya Kasnacheev 
> wrote:
>
>> Hello again!
>>
>> Of course it's 20 minutes, not 20 seconds.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> вт, 9 февр. 2021 г. в 16:45, Ilya Kasnacheev :
>>
>> > Hello!
>> >
>> > Java part kicks in if the target not found in pom.xml. Ideally we should
>> > skip this build if target check-test-suites is not in pom.xml
>> >
>> > I have changed its timeout to 20 second which should terminate its
>> > progression on older builds. Maybe that would be sufficient for now.
>> >
>> > Regards,
>> > --
>> > Ilya Kasnacheev
>> >
>> >
>> > вт, 9 февр. 2021 г. в 14:09, Petr Ivanov :
>> >
>> >> As much as I understood — we execute internal class as plugin, where
>> all
>> >> the magic is done.
>> >> Seems pretty solid in Maven part. Java part, unfortunately, cannot
>> check.
>> >>
>> >>
>> >>
>> >> Regards,
>> >> *Petr Ivanov*
>> >> Head of IT
>> >> IT & Development Solutions |
>> >> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>> >>
>> >> On 9 Feb 2021, at 12:32, Ilya Kasnacheev 
>> >> wrote:
>> >>
>> >> Hello Peter,
>> >>
>> >> Thanks for chiming in. The code is under
>> >> https://github.com/apache/ignite/pull/8367
>> >>
>> >> Regards,
>> >> --
>> >> Ilya Kasnacheev
>> >>
>> >>
>> >> вт, 9 февр. 2021 г. в 12:20, Petr Ivanov :
>> >>
>> >>> Hi, Ilya.
>> >>>
>> >>>
>> >>> I've added Inspections to Run All.
>> >>> Checkstyle is currently integrated to Build and can be deleted.
>> >>>
>> >>>
>> >>> Where can I find the code for check-test-suites profile?
>> >>>
>> >>>
>> >>> Regards,
>> >>> *Petr Ivanov*
>> >>> Head of IT
>> >>> IT & Development Solutions |
>> >>> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>> >>>
>> >>> On 9 Feb 2021, at 12:16, Ilya Kasnacheev 
>> >>> wrote:
>> >>>
>> >>> Hello!
>> >>>
>> >>> I have found one issue where it would cause tests to be run if the
>> >>> change is not present in the target branch.
>> >>>
>> >>> This includes e.g. 2.10 nightlies.
>> >>>
>> >>> What can we do to avoid that? Is specifying -DskipTests sufficient?
>> Any
>> >>> chance that it will break the missed tests check?
>> >>>
>> >>> Regards,
>> >>> --
>> >>> Ilya Kasnacheev
>> >>>
>> >>>
>> >>> пн, 8 февр. 2021 г. в 14:13, Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com
>> >>> >:
>> >>>
>>  Hello!
>> 
>>  I have created a TC suite:
>> 
>> 
>> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_MissingTests?mode=builds
>> 
>>  + Peter Ivanov
>> 
>>  Can you please check if everything is alright?
>> 
>>  BTW, it seems that Inspections [Core] is only in Run All Basic (but
>> not
>>  in Run All), and Check Code Style is not triggered by either one. Is
>> it
>>  correct?
>> 
>>  Regards,
>>  --
>>  Ilya Kasnacheev
>> 
>> 
>>  пн, 8 февр. 2021 г. в 10:22, Max Timonin :
>> 
>> > Hi!
>> >
>> > Yes, now it's a part of the Travis check, and there is already a
>> first
>> > successful build [1]. But I 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi, all!

What do you think if we add the check in the TC [Build] job. Currently
[Build] runs also check for licences, checkstyle [1]:

mvn *install* -Pall-java,all-scala,scala,*licenses*,lgpl,examples,
*checkstyle* -DskipTests -Dmaven.javadoc.skip=true %MAVEN_MODULES_STRING%.

So let's add the check too to block other jobs. As if there missed tests
then TC run may be invalid - missed tests may be broken and then the MTCGA
visa too. To made this we should change command line parameters:
1. Add profile check-test-suites;
2. Remove -Dskiptests flag.

-Pall-java,all-scala,scala,licenses,lgpl,examples,checkstyle,
*check-test-suites *-DskipTests -Dmaven.javadoc.skip=true
%MAVEN_MODULES_STRING%

WDYT?

[1]
https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_BuildApacheIgnite=buildTypeSettings_IgniteTests24Java8=%3Cdefault

On Tue, Feb 9, 2021 at 4:48 PM Ilya Kasnacheev 
wrote:

> Hello again!
>
> Of course it's 20 minutes, not 20 seconds.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 9 февр. 2021 г. в 16:45, Ilya Kasnacheev :
>
> > Hello!
> >
> > Java part kicks in if the target not found in pom.xml. Ideally we should
> > skip this build if target check-test-suites is not in pom.xml
> >
> > I have changed its timeout to 20 second which should terminate its
> > progression on older builds. Maybe that would be sufficient for now.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > вт, 9 февр. 2021 г. в 14:09, Petr Ivanov :
> >
> >> As much as I understood — we execute internal class as plugin, where all
> >> the magic is done.
> >> Seems pretty solid in Maven part. Java part, unfortunately, cannot
> check.
> >>
> >>
> >>
> >> Regards,
> >> *Petr Ivanov*
> >> Head of IT
> >> IT & Development Solutions |
> >> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
> >>
> >> On 9 Feb 2021, at 12:32, Ilya Kasnacheev 
> >> wrote:
> >>
> >> Hello Peter,
> >>
> >> Thanks for chiming in. The code is under
> >> https://github.com/apache/ignite/pull/8367
> >>
> >> Regards,
> >> --
> >> Ilya Kasnacheev
> >>
> >>
> >> вт, 9 февр. 2021 г. в 12:20, Petr Ivanov :
> >>
> >>> Hi, Ilya.
> >>>
> >>>
> >>> I've added Inspections to Run All.
> >>> Checkstyle is currently integrated to Build and can be deleted.
> >>>
> >>>
> >>> Where can I find the code for check-test-suites profile?
> >>>
> >>>
> >>> Regards,
> >>> *Petr Ivanov*
> >>> Head of IT
> >>> IT & Development Solutions |
> >>> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
> >>>
> >>> On 9 Feb 2021, at 12:16, Ilya Kasnacheev 
> >>> wrote:
> >>>
> >>> Hello!
> >>>
> >>> I have found one issue where it would cause tests to be run if the
> >>> change is not present in the target branch.
> >>>
> >>> This includes e.g. 2.10 nightlies.
> >>>
> >>> What can we do to avoid that? Is specifying -DskipTests sufficient? Any
> >>> chance that it will break the missed tests check?
> >>>
> >>> Regards,
> >>> --
> >>> Ilya Kasnacheev
> >>>
> >>>
> >>> пн, 8 февр. 2021 г. в 14:13, Ilya Kasnacheev <
> ilya.kasnach...@gmail.com
> >>> >:
> >>>
>  Hello!
> 
>  I have created a TC suite:
> 
> 
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_MissingTests?mode=builds
> 
>  + Peter Ivanov
> 
>  Can you please check if everything is alright?
> 
>  BTW, it seems that Inspections [Core] is only in Run All Basic (but
> not
>  in Run All), and Check Code Style is not triggered by either one. Is
> it
>  correct?
> 
>  Regards,
>  --
>  Ilya Kasnacheev
> 
> 
>  пн, 8 февр. 2021 г. в 10:22, Max Timonin :
> 
> > Hi!
> >
> > Yes, now it's a part of the Travis check, and there is already a
> first
> > successful build [1]. But I think it's also required to run the check
> > on TC
> > too, along with jobs for checking licenses, code style, and core
> > inspections.
> >
> >
> > [1] https://travis-ci.com/github/apache/ignite/builds/216363067
> >
> > On Fri, Feb 5, 2021 at 7:13 PM Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com>
> > wrote:
> >
> > > Hello!
> > >
> > > I have merged it to master!
> > >
> > > I wonder what happens next. It will run as a part of travis check?
> > Do we
> > > also need to add it as a TC suite?
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > ср, 3 февр. 2021 г. в 18:50, Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com>:
> > >
> > > > Hello!
> > > >
> > > > Code mostly looks good, I have added a minor request. I will
> check
> > how it
> > > > works and then we may commit.
> > > >
> > > > + zaleslaw@gmail.com
> > > >
> > > > Can you please check whether the new ML suites make sense?
> > > > math/distances/DistancesTestSuite.java
> > > > naivebayes/NaiveBayesTestSuite.java
> > > >
> > > > Would we need to add them to some TC runs?
> > > >
> > > > Regards,
> > > > --
> > > > 

Re: [DISCUSS] Missed (non-suited) tests

[Ilya Kasnacheev]

Hello again!

Of course it's 20 minutes, not 20 seconds.

Regards,
-- 
Ilya Kasnacheev


вт, 9 февр. 2021 г. в 16:45, Ilya Kasnacheev :

> Hello!
>
> Java part kicks in if the target not found in pom.xml. Ideally we should
> skip this build if target check-test-suites is not in pom.xml
>
> I have changed its timeout to 20 second which should terminate its
> progression on older builds. Maybe that would be sufficient for now.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 9 февр. 2021 г. в 14:09, Petr Ivanov :
>
>> As much as I understood — we execute internal class as plugin, where all
>> the magic is done.
>> Seems pretty solid in Maven part. Java part, unfortunately, cannot check.
>>
>>
>>
>> Regards,
>> *Petr Ivanov*
>> Head of IT
>> IT & Development Solutions |
>> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>>
>> On 9 Feb 2021, at 12:32, Ilya Kasnacheev 
>> wrote:
>>
>> Hello Peter,
>>
>> Thanks for chiming in. The code is under
>> https://github.com/apache/ignite/pull/8367
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> вт, 9 февр. 2021 г. в 12:20, Petr Ivanov :
>>
>>> Hi, Ilya.
>>>
>>>
>>> I've added Inspections to Run All.
>>> Checkstyle is currently integrated to Build and can be deleted.
>>>
>>>
>>> Where can I find the code for check-test-suites profile?
>>>
>>>
>>> Regards,
>>> *Petr Ivanov*
>>> Head of IT
>>> IT & Development Solutions |
>>> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>>>
>>> On 9 Feb 2021, at 12:16, Ilya Kasnacheev 
>>> wrote:
>>>
>>> Hello!
>>>
>>> I have found one issue where it would cause tests to be run if the
>>> change is not present in the target branch.
>>>
>>> This includes e.g. 2.10 nightlies.
>>>
>>> What can we do to avoid that? Is specifying -DskipTests sufficient? Any
>>> chance that it will break the missed tests check?
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> пн, 8 февр. 2021 г. в 14:13, Ilya Kasnacheev >> >:
>>>
 Hello!

 I have created a TC suite:

 https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_MissingTests?mode=builds

 + Peter Ivanov

 Can you please check if everything is alright?

 BTW, it seems that Inspections [Core] is only in Run All Basic (but not
 in Run All), and Check Code Style is not triggered by either one. Is it
 correct?

 Regards,
 --
 Ilya Kasnacheev


 пн, 8 февр. 2021 г. в 10:22, Max Timonin :

> Hi!
>
> Yes, now it's a part of the Travis check, and there is already a first
> successful build [1]. But I think it's also required to run the check
> on TC
> too, along with jobs for checking licenses, code style, and core
> inspections.
>
>
> [1] https://travis-ci.com/github/apache/ignite/builds/216363067
>
> On Fri, Feb 5, 2021 at 7:13 PM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com>
> wrote:
>
> > Hello!
> >
> > I have merged it to master!
> >
> > I wonder what happens next. It will run as a part of travis check?
> Do we
> > also need to add it as a TC suite?
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > ср, 3 февр. 2021 г. в 18:50, Ilya Kasnacheev <
> ilya.kasnach...@gmail.com>:
> >
> > > Hello!
> > >
> > > Code mostly looks good, I have added a minor request. I will check
> how it
> > > works and then we may commit.
> > >
> > > + zaleslaw@gmail.com
> > >
> > > Can you please check whether the new ML suites make sense?
> > > math/distances/DistancesTestSuite.java
> > > naivebayes/NaiveBayesTestSuite.java
> > >
> > > Would we need to add them to some TC runs?
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > пн, 25 янв. 2021 г. в 22:07, Max Timonin  >:
> > >
> > >> Hi, Ilya!
> > >>
> > >> I made a fix to the check. Now it aggregates info about tests and
> suites
> > >> from all modules and then validates it. Could you please review
> the PR
> > >> [1]?
> > >>
> > >> I tried to move some tests between modules, but unfortunately it
> still
> > >> looks like spaghetti. So I reverted all changes to testsuites
> (new and
> > >> splitted suites) and reworked the check.
> > >>
> > >> [1] https://github.com/apache/ignite/pull/8367
> > >>
> > >> On Mon, Dec 28, 2020 at 2:03 PM Ilya Kasnacheev <
> > >> ilya.kasnach...@gmail.com>
> > >> wrote:
> > >>
> > >> > Hello!
> > >> >
> > >> > You could try to move these tests as .java files between
> modules in a
> > >> > separate commit. I think I could review it.
> > >> >
> > >> > Regards,
> > >> > --
> > >> > Ilya Kasnacheev
> > >> >
> > >> >
> > >> > пт, 25 дек. 2020 г. в 17:19, Max Timonin <
> timonin.ma...@gmail.com>:
> > >> >
> > >> > > Hi!
> > >> > >
> > >> > > Ilya thanks for the reply! I agree that it's a 

Re: [DISCUSS] Missed (non-suited) tests

[Ilya Kasnacheev]

Hello!

Java part kicks in if the target not found in pom.xml. Ideally we should
skip this build if target check-test-suites is not in pom.xml

I have changed its timeout to 20 second which should terminate its
progression on older builds. Maybe that would be sufficient for now.

Regards,
-- 
Ilya Kasnacheev


вт, 9 февр. 2021 г. в 14:09, Petr Ivanov :

> As much as I understood — we execute internal class as plugin, where all
> the magic is done.
> Seems pretty solid in Maven part. Java part, unfortunately, cannot check.
>
>
>
> Regards,
> *Petr Ivanov*
> Head of IT
> IT & Development Solutions |
> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>
> On 9 Feb 2021, at 12:32, Ilya Kasnacheev 
> wrote:
>
> Hello Peter,
>
> Thanks for chiming in. The code is under
> https://github.com/apache/ignite/pull/8367
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 9 февр. 2021 г. в 12:20, Petr Ivanov :
>
>> Hi, Ilya.
>>
>>
>> I've added Inspections to Run All.
>> Checkstyle is currently integrated to Build and can be deleted.
>>
>>
>> Where can I find the code for check-test-suites profile?
>>
>>
>> Regards,
>> *Petr Ivanov*
>> Head of IT
>> IT & Development Solutions |
>> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>>
>> On 9 Feb 2021, at 12:16, Ilya Kasnacheev 
>> wrote:
>>
>> Hello!
>>
>> I have found one issue where it would cause tests to be run if the change
>> is not present in the target branch.
>>
>> This includes e.g. 2.10 nightlies.
>>
>> What can we do to avoid that? Is specifying -DskipTests sufficient? Any
>> chance that it will break the missed tests check?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пн, 8 февр. 2021 г. в 14:13, Ilya Kasnacheev :
>>
>>> Hello!
>>>
>>> I have created a TC suite:
>>>
>>> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_MissingTests?mode=builds
>>>
>>> + Peter Ivanov
>>>
>>> Can you please check if everything is alright?
>>>
>>> BTW, it seems that Inspections [Core] is only in Run All Basic (but not
>>> in Run All), and Check Code Style is not triggered by either one. Is it
>>> correct?
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> пн, 8 февр. 2021 г. в 10:22, Max Timonin :
>>>
 Hi!

 Yes, now it's a part of the Travis check, and there is already a first
 successful build [1]. But I think it's also required to run the check
 on TC
 too, along with jobs for checking licenses, code style, and core
 inspections.


 [1] https://travis-ci.com/github/apache/ignite/builds/216363067

 On Fri, Feb 5, 2021 at 7:13 PM Ilya Kasnacheev <
 ilya.kasnach...@gmail.com>
 wrote:

 > Hello!
 >
 > I have merged it to master!
 >
 > I wonder what happens next. It will run as a part of travis check? Do
 we
 > also need to add it as a TC suite?
 >
 > Regards,
 > --
 > Ilya Kasnacheev
 >
 >
 > ср, 3 февр. 2021 г. в 18:50, Ilya Kasnacheev <
 ilya.kasnach...@gmail.com>:
 >
 > > Hello!
 > >
 > > Code mostly looks good, I have added a minor request. I will check
 how it
 > > works and then we may commit.
 > >
 > > + zaleslaw@gmail.com
 > >
 > > Can you please check whether the new ML suites make sense?
 > > math/distances/DistancesTestSuite.java
 > > naivebayes/NaiveBayesTestSuite.java
 > >
 > > Would we need to add them to some TC runs?
 > >
 > > Regards,
 > > --
 > > Ilya Kasnacheev
 > >
 > >
 > > пн, 25 янв. 2021 г. в 22:07, Max Timonin :
 > >
 > >> Hi, Ilya!
 > >>
 > >> I made a fix to the check. Now it aggregates info about tests and
 suites
 > >> from all modules and then validates it. Could you please review
 the PR
 > >> [1]?
 > >>
 > >> I tried to move some tests between modules, but unfortunately it
 still
 > >> looks like spaghetti. So I reverted all changes to testsuites (new
 and
 > >> splitted suites) and reworked the check.
 > >>
 > >> [1] https://github.com/apache/ignite/pull/8367
 > >>
 > >> On Mon, Dec 28, 2020 at 2:03 PM Ilya Kasnacheev <
 > >> ilya.kasnach...@gmail.com>
 > >> wrote:
 > >>
 > >> > Hello!
 > >> >
 > >> > You could try to move these tests as .java files between modules
 in a
 > >> > separate commit. I think I could review it.
 > >> >
 > >> > Regards,
 > >> > --
 > >> > Ilya Kasnacheev
 > >> >
 > >> >
 > >> > пт, 25 дек. 2020 г. в 17:19, Max Timonin <
 timonin.ma...@gmail.com>:
 > >> >
 > >> > > Hi!
 > >> > >
 > >> > > Ilya thanks for the reply! I agree that it's a valid case when
 a
 > test
 > >> is
 > >> > > part of multiple suites in different modules. But it is
 definitely a
 > >> bug
 > >> > > that the test is written in a module where it can't be run at
 all
 > and
 > >> > aimed
 > >> > > to run within different modules (core tests in core that
 

Re: [DISCUSS] Missed (non-suited) tests

[Petr Ivanov]

As much as I understood — we execute internal class as plugin, where all the 
magic is done.
Seems pretty solid in Maven part. Java part, unfortunately, cannot check.



Regards,
Petr Ivanov
Head of IT
IT & Development Solutions | GRIDGAIN SYSTEMS
+7 (911) 945-00-59

> On 9 Feb 2021, at 12:32, Ilya Kasnacheev  wrote:
> 
> Hello Peter,
> 
> Thanks for chiming in. The code is under 
> https://github.com/apache/ignite/pull/8367 
> 
> 
> Regards,
> -- 
> Ilya Kasnacheev
> 
> 
> вт, 9 февр. 2021 г. в 12:20, Petr Ivanov  >:
> Hi, Ilya.
> 
> 
> I've added Inspections to Run All.
> Checkstyle is currently integrated to Build and can be deleted.
> 
> 
> Where can I find the code for check-test-suites profile?
> 
> 
> Regards,
> Petr Ivanov
> Head of IT
> IT & Development Solutions | GRIDGAIN SYSTEMS
> +7 (911) 945-00-59
> 
>> On 9 Feb 2021, at 12:16, Ilya Kasnacheev > > wrote:
>> 
>> Hello!
>> 
>> I have found one issue where it would cause tests to be run if the change is 
>> not present in the target branch.
>> 
>> This includes e.g. 2.10 nightlies.
>> 
>> What can we do to avoid that? Is specifying -DskipTests sufficient? Any 
>> chance that it will break the missed tests check?
>> 
>> Regards,
>> -- 
>> Ilya Kasnacheev
>> 
>> 
>> пн, 8 февр. 2021 г. в 14:13, Ilya Kasnacheev > >:
>> Hello!
>> 
>> I have created a TC suite:
>> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_MissingTests?mode=builds
>>  
>> 
>> 
>> + Peter Ivanov
>> 
>> Can you please check if everything is alright?
>> 
>> BTW, it seems that Inspections [Core] is only in Run All Basic (but not in 
>> Run All), and Check Code Style is not triggered by either one. Is it correct?
>> 
>> Regards,
>> -- 
>> Ilya Kasnacheev
>> 
>> 
>> пн, 8 февр. 2021 г. в 10:22, Max Timonin > >:
>> Hi!
>> 
>> Yes, now it's a part of the Travis check, and there is already a first
>> successful build [1]. But I think it's also required to run the check on TC
>> too, along with jobs for checking licenses, code style, and core
>> inspections.
>> 
>> 
>> [1] https://travis-ci.com/github/apache/ignite/builds/216363067 
>> 
>> 
>> On Fri, Feb 5, 2021 at 7:13 PM Ilya Kasnacheev > >
>> wrote:
>> 
>> > Hello!
>> >
>> > I have merged it to master!
>> >
>> > I wonder what happens next. It will run as a part of travis check? Do we
>> > also need to add it as a TC suite?
>> >
>> > Regards,
>> > --
>> > Ilya Kasnacheev
>> >
>> >
>> > ср, 3 февр. 2021 г. в 18:50, Ilya Kasnacheev > > >:
>> >
>> > > Hello!
>> > >
>> > > Code mostly looks good, I have added a minor request. I will check how it
>> > > works and then we may commit.
>> > >
>> > > + zaleslaw@gmail.com 
>> > >
>> > > Can you please check whether the new ML suites make sense?
>> > > math/distances/DistancesTestSuite.java
>> > > naivebayes/NaiveBayesTestSuite.java
>> > >
>> > > Would we need to add them to some TC runs?
>> > >
>> > > Regards,
>> > > --
>> > > Ilya Kasnacheev
>> > >
>> > >
>> > > пн, 25 янв. 2021 г. в 22:07, Max Timonin > > > >:
>> > >
>> > >> Hi, Ilya!
>> > >>
>> > >> I made a fix to the check. Now it aggregates info about tests and suites
>> > >> from all modules and then validates it. Could you please review the PR
>> > >> [1]?
>> > >>
>> > >> I tried to move some tests between modules, but unfortunately it still
>> > >> looks like spaghetti. So I reverted all changes to testsuites (new and
>> > >> splitted suites) and reworked the check.
>> > >>
>> > >> [1] https://github.com/apache/ignite/pull/8367 
>> > >> 
>> > >>
>> > >> On Mon, Dec 28, 2020 at 2:03 PM Ilya Kasnacheev <
>> > >> ilya.kasnach...@gmail.com >
>> > >> wrote:
>> > >>
>> > >> > Hello!
>> > >> >
>> > >> > You could try to move these tests as .java files between modules in a
>> > >> > separate commit. I think I could review it.
>> > >> >
>> > >> > Regards,
>> > >> > --
>> > >> > Ilya Kasnacheev
>> > >> >
>> > >> >
>> > >> > пт, 25 дек. 2020 г. в 17:19, Max Timonin > > >> > >:
>> > >> >
>> > >> > > Hi!
>> > >> > >
>> > >> > > Ilya thanks for the reply! I agree that it's a valid case when a
>> > test
>> > >> is
>> > >> > > part of multiple suites in different modules. But it is definitely a
>> > >> bug
>> > >> > > that the test is written in a module where it can't be run at all
>> > and
>> > >> > aimed
>> > >> > > to run within different modules (core tests in core that require
>> > h2).
>> > >> I
>> > >> > > propose to fix this issue.
>> > >> > >
>> > >> > > I'm 

Re: [DISCUSS] Missed (non-suited) tests

[Petr Ivanov]

Hi, Ilya.


I've added Inspections to Run All.
Checkstyle is currently integrated to Build and can be deleted.


Where can I find the code for check-test-suites profile?


Regards,
Petr Ivanov
Head of IT
IT & Development Solutions | GRIDGAIN SYSTEMS
+7 (911) 945-00-59

> On 9 Feb 2021, at 12:16, Ilya Kasnacheev  wrote:
> 
> Hello!
> 
> I have found one issue where it would cause tests to be run if the change is 
> not present in the target branch.
> 
> This includes e.g. 2.10 nightlies.
> 
> What can we do to avoid that? Is specifying -DskipTests sufficient? Any 
> chance that it will break the missed tests check?
> 
> Regards,
> -- 
> Ilya Kasnacheev
> 
> 
> пн, 8 февр. 2021 г. в 14:13, Ilya Kasnacheev  >:
> Hello!
> 
> I have created a TC suite:
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_MissingTests?mode=builds
>  
> 
> 
> + Peter Ivanov
> 
> Can you please check if everything is alright?
> 
> BTW, it seems that Inspections [Core] is only in Run All Basic (but not in 
> Run All), and Check Code Style is not triggered by either one. Is it correct?
> 
> Regards,
> -- 
> Ilya Kasnacheev
> 
> 
> пн, 8 февр. 2021 г. в 10:22, Max Timonin  >:
> Hi!
> 
> Yes, now it's a part of the Travis check, and there is already a first
> successful build [1]. But I think it's also required to run the check on TC
> too, along with jobs for checking licenses, code style, and core
> inspections.
> 
> 
> [1] https://travis-ci.com/github/apache/ignite/builds/216363067 
> 
> 
> On Fri, Feb 5, 2021 at 7:13 PM Ilya Kasnacheev  >
> wrote:
> 
> > Hello!
> >
> > I have merged it to master!
> >
> > I wonder what happens next. It will run as a part of travis check? Do we
> > also need to add it as a TC suite?
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > ср, 3 февр. 2021 г. в 18:50, Ilya Kasnacheev  > >:
> >
> > > Hello!
> > >
> > > Code mostly looks good, I have added a minor request. I will check how it
> > > works and then we may commit.
> > >
> > > + zaleslaw@gmail.com 
> > >
> > > Can you please check whether the new ML suites make sense?
> > > math/distances/DistancesTestSuite.java
> > > naivebayes/NaiveBayesTestSuite.java
> > >
> > > Would we need to add them to some TC runs?
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > пн, 25 янв. 2021 г. в 22:07, Max Timonin  > > >:
> > >
> > >> Hi, Ilya!
> > >>
> > >> I made a fix to the check. Now it aggregates info about tests and suites
> > >> from all modules and then validates it. Could you please review the PR
> > >> [1]?
> > >>
> > >> I tried to move some tests between modules, but unfortunately it still
> > >> looks like spaghetti. So I reverted all changes to testsuites (new and
> > >> splitted suites) and reworked the check.
> > >>
> > >> [1] https://github.com/apache/ignite/pull/8367 
> > >> 
> > >>
> > >> On Mon, Dec 28, 2020 at 2:03 PM Ilya Kasnacheev <
> > >> ilya.kasnach...@gmail.com >
> > >> wrote:
> > >>
> > >> > Hello!
> > >> >
> > >> > You could try to move these tests as .java files between modules in a
> > >> > separate commit. I think I could review it.
> > >> >
> > >> > Regards,
> > >> > --
> > >> > Ilya Kasnacheev
> > >> >
> > >> >
> > >> > пт, 25 дек. 2020 г. в 17:19, Max Timonin  > >> > >:
> > >> >
> > >> > > Hi!
> > >> > >
> > >> > > Ilya thanks for the reply! I agree that it's a valid case when a
> > test
> > >> is
> > >> > > part of multiple suites in different modules. But it is definitely a
> > >> bug
> > >> > > that the test is written in a module where it can't be run at all
> > and
> > >> > aimed
> > >> > > to run within different modules (core tests in core that require
> > h2).
> > >> I
> > >> > > propose to fix this issue.
> > >> > >
> > >> > > I'm going to check all such tests and move them to the right module.
> > >> As I
> > >> > > can see there are about 100 such test classes, but I hope that most
> > of
> > >> > them
> > >> > > follow only a few patterns.
> > >> > >
> > >> > > On Fri, Dec 25, 2020 at 2:58 PM Ivan Daschinsky <
> > ivanda...@gmail.com >
> > >> > > wrote:
> > >> > >
> > >> > > > Hi!
> > >> > > > >> I'm not sure that we should assume every test is only run from
> > >> one
> > >> > > test
> > >> > > > suite. One test may be run from different test suites located in
> > >> > > different
> > >> > > > modules.
> > >> > > > Agree. For example, if we introduce this limitation, zk suites
> > will
> > >> be
> > >> > > > broken.
> > >> > > >
> > >> > > >
> > >> > > > пт, 25 дек. 2020 г. в 14:48, Ilya 

Re: [DISCUSS] Missed (non-suited) tests

[Ilya Kasnacheev]

Hello Peter,

Thanks for chiming in. The code is under
https://github.com/apache/ignite/pull/8367

Regards,
-- 
Ilya Kasnacheev


вт, 9 февр. 2021 г. в 12:20, Petr Ivanov :

> Hi, Ilya.
>
>
> I've added Inspections to Run All.
> Checkstyle is currently integrated to Build and can be deleted.
>
>
> Where can I find the code for check-test-suites profile?
>
>
> Regards,
> *Petr Ivanov*
> Head of IT
> IT & Development Solutions |
> *GRIDGAIN SYSTEMS*+7 (911) 945-00-59
>
> On 9 Feb 2021, at 12:16, Ilya Kasnacheev 
> wrote:
>
> Hello!
>
> I have found one issue where it would cause tests to be run if the change
> is not present in the target branch.
>
> This includes e.g. 2.10 nightlies.
>
> What can we do to avoid that? Is specifying -DskipTests sufficient? Any
> chance that it will break the missed tests check?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пн, 8 февр. 2021 г. в 14:13, Ilya Kasnacheev :
>
>> Hello!
>>
>> I have created a TC suite:
>>
>> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_MissingTests?mode=builds
>>
>> + Peter Ivanov
>>
>> Can you please check if everything is alright?
>>
>> BTW, it seems that Inspections [Core] is only in Run All Basic (but not
>> in Run All), and Check Code Style is not triggered by either one. Is it
>> correct?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пн, 8 февр. 2021 г. в 10:22, Max Timonin :
>>
>>> Hi!
>>>
>>> Yes, now it's a part of the Travis check, and there is already a first
>>> successful build [1]. But I think it's also required to run the check on
>>> TC
>>> too, along with jobs for checking licenses, code style, and core
>>> inspections.
>>>
>>>
>>> [1] https://travis-ci.com/github/apache/ignite/builds/216363067
>>>
>>> On Fri, Feb 5, 2021 at 7:13 PM Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com>
>>> wrote:
>>>
>>> > Hello!
>>> >
>>> > I have merged it to master!
>>> >
>>> > I wonder what happens next. It will run as a part of travis check? Do
>>> we
>>> > also need to add it as a TC suite?
>>> >
>>> > Regards,
>>> > --
>>> > Ilya Kasnacheev
>>> >
>>> >
>>> > ср, 3 февр. 2021 г. в 18:50, Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com>:
>>> >
>>> > > Hello!
>>> > >
>>> > > Code mostly looks good, I have added a minor request. I will check
>>> how it
>>> > > works and then we may commit.
>>> > >
>>> > > + zaleslaw@gmail.com
>>> > >
>>> > > Can you please check whether the new ML suites make sense?
>>> > > math/distances/DistancesTestSuite.java
>>> > > naivebayes/NaiveBayesTestSuite.java
>>> > >
>>> > > Would we need to add them to some TC runs?
>>> > >
>>> > > Regards,
>>> > > --
>>> > > Ilya Kasnacheev
>>> > >
>>> > >
>>> > > пн, 25 янв. 2021 г. в 22:07, Max Timonin :
>>> > >
>>> > >> Hi, Ilya!
>>> > >>
>>> > >> I made a fix to the check. Now it aggregates info about tests and
>>> suites
>>> > >> from all modules and then validates it. Could you please review the
>>> PR
>>> > >> [1]?
>>> > >>
>>> > >> I tried to move some tests between modules, but unfortunately it
>>> still
>>> > >> looks like spaghetti. So I reverted all changes to testsuites (new
>>> and
>>> > >> splitted suites) and reworked the check.
>>> > >>
>>> > >> [1] https://github.com/apache/ignite/pull/8367
>>> > >>
>>> > >> On Mon, Dec 28, 2020 at 2:03 PM Ilya Kasnacheev <
>>> > >> ilya.kasnach...@gmail.com>
>>> > >> wrote:
>>> > >>
>>> > >> > Hello!
>>> > >> >
>>> > >> > You could try to move these tests as .java files between modules
>>> in a
>>> > >> > separate commit. I think I could review it.
>>> > >> >
>>> > >> > Regards,
>>> > >> > --
>>> > >> > Ilya Kasnacheev
>>> > >> >
>>> > >> >
>>> > >> > пт, 25 дек. 2020 г. в 17:19, Max Timonin >> >:
>>> > >> >
>>> > >> > > Hi!
>>> > >> > >
>>> > >> > > Ilya thanks for the reply! I agree that it's a valid case when a
>>> > test
>>> > >> is
>>> > >> > > part of multiple suites in different modules. But it is
>>> definitely a
>>> > >> bug
>>> > >> > > that the test is written in a module where it can't be run at
>>> all
>>> > and
>>> > >> > aimed
>>> > >> > > to run within different modules (core tests in core that require
>>> > h2).
>>> > >> I
>>> > >> > > propose to fix this issue.
>>> > >> > >
>>> > >> > > I'm going to check all such tests and move them to the right
>>> module.
>>> > >> As I
>>> > >> > > can see there are about 100 such test classes, but I hope that
>>> most
>>> > of
>>> > >> > them
>>> > >> > > follow only a few patterns.
>>> > >> > >
>>> > >> > > On Fri, Dec 25, 2020 at 2:58 PM Ivan Daschinsky <
>>> > ivanda...@gmail.com>
>>> > >> > > wrote:
>>> > >> > >
>>> > >> > > > Hi!
>>> > >> > > > >> I'm not sure that we should assume every test is only run
>>> from
>>> > >> one
>>> > >> > > test
>>> > >> > > > suite. One test may be run from different test suites located
>>> in
>>> > >> > > different
>>> > >> > > > modules.
>>> > >> > > > Agree. For example, if we introduce this limitation, zk suites
>>> > will
>>> > >> be
>>> > >> > > > broken.
>>> > >> > > >
>>> > >> > > >
>>> > >> > > > пт, 25 

Re: [DISCUSS] Missed (non-suited) tests

[Ilya Kasnacheev]

Hello!

I have found one issue where it would cause tests to be run if the change
is not present in the target branch.

This includes e.g. 2.10 nightlies.

What can we do to avoid that? Is specifying -DskipTests sufficient? Any
chance that it will break the missed tests check?

Regards,
-- 
Ilya Kasnacheev


пн, 8 февр. 2021 г. в 14:13, Ilya Kasnacheev :

> Hello!
>
> I have created a TC suite:
>
> https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_MissingTests?mode=builds
>
> + Peter Ivanov
>
> Can you please check if everything is alright?
>
> BTW, it seems that Inspections [Core] is only in Run All Basic (but not in
> Run All), and Check Code Style is not triggered by either one. Is it
> correct?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пн, 8 февр. 2021 г. в 10:22, Max Timonin :
>
>> Hi!
>>
>> Yes, now it's a part of the Travis check, and there is already a first
>> successful build [1]. But I think it's also required to run the check on
>> TC
>> too, along with jobs for checking licenses, code style, and core
>> inspections.
>>
>>
>> [1] https://travis-ci.com/github/apache/ignite/builds/216363067
>>
>> On Fri, Feb 5, 2021 at 7:13 PM Ilya Kasnacheev > >
>> wrote:
>>
>> > Hello!
>> >
>> > I have merged it to master!
>> >
>> > I wonder what happens next. It will run as a part of travis check? Do we
>> > also need to add it as a TC suite?
>> >
>> > Regards,
>> > --
>> > Ilya Kasnacheev
>> >
>> >
>> > ср, 3 февр. 2021 г. в 18:50, Ilya Kasnacheev > >:
>> >
>> > > Hello!
>> > >
>> > > Code mostly looks good, I have added a minor request. I will check
>> how it
>> > > works and then we may commit.
>> > >
>> > > + zaleslaw@gmail.com
>> > >
>> > > Can you please check whether the new ML suites make sense?
>> > > math/distances/DistancesTestSuite.java
>> > > naivebayes/NaiveBayesTestSuite.java
>> > >
>> > > Would we need to add them to some TC runs?
>> > >
>> > > Regards,
>> > > --
>> > > Ilya Kasnacheev
>> > >
>> > >
>> > > пн, 25 янв. 2021 г. в 22:07, Max Timonin :
>> > >
>> > >> Hi, Ilya!
>> > >>
>> > >> I made a fix to the check. Now it aggregates info about tests and
>> suites
>> > >> from all modules and then validates it. Could you please review the
>> PR
>> > >> [1]?
>> > >>
>> > >> I tried to move some tests between modules, but unfortunately it
>> still
>> > >> looks like spaghetti. So I reverted all changes to testsuites (new
>> and
>> > >> splitted suites) and reworked the check.
>> > >>
>> > >> [1] https://github.com/apache/ignite/pull/8367
>> > >>
>> > >> On Mon, Dec 28, 2020 at 2:03 PM Ilya Kasnacheev <
>> > >> ilya.kasnach...@gmail.com>
>> > >> wrote:
>> > >>
>> > >> > Hello!
>> > >> >
>> > >> > You could try to move these tests as .java files between modules
>> in a
>> > >> > separate commit. I think I could review it.
>> > >> >
>> > >> > Regards,
>> > >> > --
>> > >> > Ilya Kasnacheev
>> > >> >
>> > >> >
>> > >> > пт, 25 дек. 2020 г. в 17:19, Max Timonin > >:
>> > >> >
>> > >> > > Hi!
>> > >> > >
>> > >> > > Ilya thanks for the reply! I agree that it's a valid case when a
>> > test
>> > >> is
>> > >> > > part of multiple suites in different modules. But it is
>> definitely a
>> > >> bug
>> > >> > > that the test is written in a module where it can't be run at all
>> > and
>> > >> > aimed
>> > >> > > to run within different modules (core tests in core that require
>> > h2).
>> > >> I
>> > >> > > propose to fix this issue.
>> > >> > >
>> > >> > > I'm going to check all such tests and move them to the right
>> module.
>> > >> As I
>> > >> > > can see there are about 100 such test classes, but I hope that
>> most
>> > of
>> > >> > them
>> > >> > > follow only a few patterns.
>> > >> > >
>> > >> > > On Fri, Dec 25, 2020 at 2:58 PM Ivan Daschinsky <
>> > ivanda...@gmail.com>
>> > >> > > wrote:
>> > >> > >
>> > >> > > > Hi!
>> > >> > > > >> I'm not sure that we should assume every test is only run
>> from
>> > >> one
>> > >> > > test
>> > >> > > > suite. One test may be run from different test suites located
>> in
>> > >> > > different
>> > >> > > > modules.
>> > >> > > > Agree. For example, if we introduce this limitation, zk suites
>> > will
>> > >> be
>> > >> > > > broken.
>> > >> > > >
>> > >> > > >
>> > >> > > > пт, 25 дек. 2020 г. в 14:48, Ilya Kasnacheev <
>> > >> > ilya.kasnach...@gmail.com
>> > >> > > >:
>> > >> > > >
>> > >> > > > > Hello!
>> > >> > > > >
>> > >> > > > > Sorry for the long wait.
>> > >> > > > >
>> > >> > > > > I'm not sure that we should assume every test is only run
>> from
>> > one
>> > >> > test
>> > >> > > > > suite. One test may be run from different test suites
>> located in
>> > >> > > > different
>> > >> > > > > modules.
>> > >> > > > >
>> > >> > > > > I wonder if we can drop this requirement, check all the
>> modules
>> > >> > > > > transitively for used/unused tests.
>> > >> > > > >
>> > >> > > > > Regards,
>> > >> > > > > --
>> > >> > > > > Ilya Kasnacheev
>> > >> > > > >
>> > >> > > > >
>> > >> > > > > ср, 2 дек. 2020 г. в 18:23, Max Timonin 

Re: [DISCUSS] Missed (non-suited) tests

[Ilya Kasnacheev]

Hello!

I have created a TC suite:
https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_MissingTests?mode=builds

+ Peter Ivanov

Can you please check if everything is alright?

BTW, it seems that Inspections [Core] is only in Run All Basic (but not in
Run All), and Check Code Style is not triggered by either one. Is it
correct?

Regards,
-- 
Ilya Kasnacheev


пн, 8 февр. 2021 г. в 10:22, Max Timonin :

> Hi!
>
> Yes, now it's a part of the Travis check, and there is already a first
> successful build [1]. But I think it's also required to run the check on TC
> too, along with jobs for checking licenses, code style, and core
> inspections.
>
>
> [1] https://travis-ci.com/github/apache/ignite/builds/216363067
>
> On Fri, Feb 5, 2021 at 7:13 PM Ilya Kasnacheev 
> wrote:
>
> > Hello!
> >
> > I have merged it to master!
> >
> > I wonder what happens next. It will run as a part of travis check? Do we
> > also need to add it as a TC suite?
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > ср, 3 февр. 2021 г. в 18:50, Ilya Kasnacheev  >:
> >
> > > Hello!
> > >
> > > Code mostly looks good, I have added a minor request. I will check how
> it
> > > works and then we may commit.
> > >
> > > + zaleslaw@gmail.com
> > >
> > > Can you please check whether the new ML suites make sense?
> > > math/distances/DistancesTestSuite.java
> > > naivebayes/NaiveBayesTestSuite.java
> > >
> > > Would we need to add them to some TC runs?
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > пн, 25 янв. 2021 г. в 22:07, Max Timonin :
> > >
> > >> Hi, Ilya!
> > >>
> > >> I made a fix to the check. Now it aggregates info about tests and
> suites
> > >> from all modules and then validates it. Could you please review the PR
> > >> [1]?
> > >>
> > >> I tried to move some tests between modules, but unfortunately it still
> > >> looks like spaghetti. So I reverted all changes to testsuites (new and
> > >> splitted suites) and reworked the check.
> > >>
> > >> [1] https://github.com/apache/ignite/pull/8367
> > >>
> > >> On Mon, Dec 28, 2020 at 2:03 PM Ilya Kasnacheev <
> > >> ilya.kasnach...@gmail.com>
> > >> wrote:
> > >>
> > >> > Hello!
> > >> >
> > >> > You could try to move these tests as .java files between modules in
> a
> > >> > separate commit. I think I could review it.
> > >> >
> > >> > Regards,
> > >> > --
> > >> > Ilya Kasnacheev
> > >> >
> > >> >
> > >> > пт, 25 дек. 2020 г. в 17:19, Max Timonin :
> > >> >
> > >> > > Hi!
> > >> > >
> > >> > > Ilya thanks for the reply! I agree that it's a valid case when a
> > test
> > >> is
> > >> > > part of multiple suites in different modules. But it is
> definitely a
> > >> bug
> > >> > > that the test is written in a module where it can't be run at all
> > and
> > >> > aimed
> > >> > > to run within different modules (core tests in core that require
> > h2).
> > >> I
> > >> > > propose to fix this issue.
> > >> > >
> > >> > > I'm going to check all such tests and move them to the right
> module.
> > >> As I
> > >> > > can see there are about 100 such test classes, but I hope that
> most
> > of
> > >> > them
> > >> > > follow only a few patterns.
> > >> > >
> > >> > > On Fri, Dec 25, 2020 at 2:58 PM Ivan Daschinsky <
> > ivanda...@gmail.com>
> > >> > > wrote:
> > >> > >
> > >> > > > Hi!
> > >> > > > >> I'm not sure that we should assume every test is only run
> from
> > >> one
> > >> > > test
> > >> > > > suite. One test may be run from different test suites located in
> > >> > > different
> > >> > > > modules.
> > >> > > > Agree. For example, if we introduce this limitation, zk suites
> > will
> > >> be
> > >> > > > broken.
> > >> > > >
> > >> > > >
> > >> > > > пт, 25 дек. 2020 г. в 14:48, Ilya Kasnacheev <
> > >> > ilya.kasnach...@gmail.com
> > >> > > >:
> > >> > > >
> > >> > > > > Hello!
> > >> > > > >
> > >> > > > > Sorry for the long wait.
> > >> > > > >
> > >> > > > > I'm not sure that we should assume every test is only run from
> > one
> > >> > test
> > >> > > > > suite. One test may be run from different test suites located
> in
> > >> > > > different
> > >> > > > > modules.
> > >> > > > >
> > >> > > > > I wonder if we can drop this requirement, check all the
> modules
> > >> > > > > transitively for used/unused tests.
> > >> > > > >
> > >> > > > > Regards,
> > >> > > > > --
> > >> > > > > Ilya Kasnacheev
> > >> > > > >
> > >> > > > >
> > >> > > > > ср, 2 дек. 2020 г. в 18:23, Max Timonin <
> > timonin.ma...@gmail.com
> > >> >:
> > >> > > > >
> > >> > > > > > Hi,
> > >> > > > > >
> > >> > > > > > I don't think so. It looks like a bug that tests fail if one
> > >> runs
> > >> > > them
> > >> > > > > > within their module (actually, what is the goal of this
> > test?).
> > >> The
> > >> > > > check
> > >> > > > > > showed us this problem, there is no need to fix the check.
> > >> > > > > >
> > >> > > > > > Currently I see two ways:
> > >> > > > > > 1. Find the right module for every misplaced test. There are
> > 104
> > >> > > tests,
> > >> > 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi!

Yes, now it's a part of the Travis check, and there is already a first
successful build [1]. But I think it's also required to run the check on TC
too, along with jobs for checking licenses, code style, and core
inspections.


[1] https://travis-ci.com/github/apache/ignite/builds/216363067

On Fri, Feb 5, 2021 at 7:13 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> I have merged it to master!
>
> I wonder what happens next. It will run as a part of travis check? Do we
> also need to add it as a TC suite?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 3 февр. 2021 г. в 18:50, Ilya Kasnacheev :
>
> > Hello!
> >
> > Code mostly looks good, I have added a minor request. I will check how it
> > works and then we may commit.
> >
> > + zaleslaw@gmail.com
> >
> > Can you please check whether the new ML suites make sense?
> > math/distances/DistancesTestSuite.java
> > naivebayes/NaiveBayesTestSuite.java
> >
> > Would we need to add them to some TC runs?
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > пн, 25 янв. 2021 г. в 22:07, Max Timonin :
> >
> >> Hi, Ilya!
> >>
> >> I made a fix to the check. Now it aggregates info about tests and suites
> >> from all modules and then validates it. Could you please review the PR
> >> [1]?
> >>
> >> I tried to move some tests between modules, but unfortunately it still
> >> looks like spaghetti. So I reverted all changes to testsuites (new and
> >> splitted suites) and reworked the check.
> >>
> >> [1] https://github.com/apache/ignite/pull/8367
> >>
> >> On Mon, Dec 28, 2020 at 2:03 PM Ilya Kasnacheev <
> >> ilya.kasnach...@gmail.com>
> >> wrote:
> >>
> >> > Hello!
> >> >
> >> > You could try to move these tests as .java files between modules in a
> >> > separate commit. I think I could review it.
> >> >
> >> > Regards,
> >> > --
> >> > Ilya Kasnacheev
> >> >
> >> >
> >> > пт, 25 дек. 2020 г. в 17:19, Max Timonin :
> >> >
> >> > > Hi!
> >> > >
> >> > > Ilya thanks for the reply! I agree that it's a valid case when a
> test
> >> is
> >> > > part of multiple suites in different modules. But it is definitely a
> >> bug
> >> > > that the test is written in a module where it can't be run at all
> and
> >> > aimed
> >> > > to run within different modules (core tests in core that require
> h2).
> >> I
> >> > > propose to fix this issue.
> >> > >
> >> > > I'm going to check all such tests and move them to the right module.
> >> As I
> >> > > can see there are about 100 such test classes, but I hope that most
> of
> >> > them
> >> > > follow only a few patterns.
> >> > >
> >> > > On Fri, Dec 25, 2020 at 2:58 PM Ivan Daschinsky <
> ivanda...@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > Hi!
> >> > > > >> I'm not sure that we should assume every test is only run from
> >> one
> >> > > test
> >> > > > suite. One test may be run from different test suites located in
> >> > > different
> >> > > > modules.
> >> > > > Agree. For example, if we introduce this limitation, zk suites
> will
> >> be
> >> > > > broken.
> >> > > >
> >> > > >
> >> > > > пт, 25 дек. 2020 г. в 14:48, Ilya Kasnacheev <
> >> > ilya.kasnach...@gmail.com
> >> > > >:
> >> > > >
> >> > > > > Hello!
> >> > > > >
> >> > > > > Sorry for the long wait.
> >> > > > >
> >> > > > > I'm not sure that we should assume every test is only run from
> one
> >> > test
> >> > > > > suite. One test may be run from different test suites located in
> >> > > > different
> >> > > > > modules.
> >> > > > >
> >> > > > > I wonder if we can drop this requirement, check all the modules
> >> > > > > transitively for used/unused tests.
> >> > > > >
> >> > > > > Regards,
> >> > > > > --
> >> > > > > Ilya Kasnacheev
> >> > > > >
> >> > > > >
> >> > > > > ср, 2 дек. 2020 г. в 18:23, Max Timonin <
> timonin.ma...@gmail.com
> >> >:
> >> > > > >
> >> > > > > > Hi,
> >> > > > > >
> >> > > > > > I don't think so. It looks like a bug that tests fail if one
> >> runs
> >> > > them
> >> > > > > > within their module (actually, what is the goal of this
> test?).
> >> The
> >> > > > check
> >> > > > > > showed us this problem, there is no need to fix the check.
> >> > > > > >
> >> > > > > > Currently I see two ways:
> >> > > > > > 1. Find the right module for every misplaced test. There are
> 104
> >> > > tests,
> >> > > > > > maybe just move them all to the target module? If TeamCity
> runs
> >> > them
> >> > > > > within
> >> > > > > > the indexing module only is there a reason to have a test in
> the
> >> > core
> >> > > > > > module at all?
> >> > > > > > 2. Back to my previous proposal - create fake suites within a
> >> > module,
> >> > > > > then
> >> > > > > > replace test classes in a target suite with the single class
> of
> >> the
> >> > > > fake
> >> > > > > > suite.
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > On Wed, Dec 2, 2020 at 5:38 PM Ilya Kasnacheev <
> >> > > > > ilya.kasnach...@gmail.com>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Hello!
> >> > > > > > >
> >> > > > > > > I think this means that we should 

Re: [DISCUSS] Missed (non-suited) tests

[Ilya Kasnacheev]

Hello!

I have merged it to master!

I wonder what happens next. It will run as a part of travis check? Do we
also need to add it as a TC suite?

Regards,
-- 
Ilya Kasnacheev


ср, 3 февр. 2021 г. в 18:50, Ilya Kasnacheev :

> Hello!
>
> Code mostly looks good, I have added a minor request. I will check how it
> works and then we may commit.
>
> + zaleslaw@gmail.com
>
> Can you please check whether the new ML suites make sense?
> math/distances/DistancesTestSuite.java
> naivebayes/NaiveBayesTestSuite.java
>
> Would we need to add them to some TC runs?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пн, 25 янв. 2021 г. в 22:07, Max Timonin :
>
>> Hi, Ilya!
>>
>> I made a fix to the check. Now it aggregates info about tests and suites
>> from all modules and then validates it. Could you please review the PR
>> [1]?
>>
>> I tried to move some tests between modules, but unfortunately it still
>> looks like spaghetti. So I reverted all changes to testsuites (new and
>> splitted suites) and reworked the check.
>>
>> [1] https://github.com/apache/ignite/pull/8367
>>
>> On Mon, Dec 28, 2020 at 2:03 PM Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com>
>> wrote:
>>
>> > Hello!
>> >
>> > You could try to move these tests as .java files between modules in a
>> > separate commit. I think I could review it.
>> >
>> > Regards,
>> > --
>> > Ilya Kasnacheev
>> >
>> >
>> > пт, 25 дек. 2020 г. в 17:19, Max Timonin :
>> >
>> > > Hi!
>> > >
>> > > Ilya thanks for the reply! I agree that it's a valid case when a test
>> is
>> > > part of multiple suites in different modules. But it is definitely a
>> bug
>> > > that the test is written in a module where it can't be run at all and
>> > aimed
>> > > to run within different modules (core tests in core that require h2).
>> I
>> > > propose to fix this issue.
>> > >
>> > > I'm going to check all such tests and move them to the right module.
>> As I
>> > > can see there are about 100 such test classes, but I hope that most of
>> > them
>> > > follow only a few patterns.
>> > >
>> > > On Fri, Dec 25, 2020 at 2:58 PM Ivan Daschinsky 
>> > > wrote:
>> > >
>> > > > Hi!
>> > > > >> I'm not sure that we should assume every test is only run from
>> one
>> > > test
>> > > > suite. One test may be run from different test suites located in
>> > > different
>> > > > modules.
>> > > > Agree. For example, if we introduce this limitation, zk suites will
>> be
>> > > > broken.
>> > > >
>> > > >
>> > > > пт, 25 дек. 2020 г. в 14:48, Ilya Kasnacheev <
>> > ilya.kasnach...@gmail.com
>> > > >:
>> > > >
>> > > > > Hello!
>> > > > >
>> > > > > Sorry for the long wait.
>> > > > >
>> > > > > I'm not sure that we should assume every test is only run from one
>> > test
>> > > > > suite. One test may be run from different test suites located in
>> > > > different
>> > > > > modules.
>> > > > >
>> > > > > I wonder if we can drop this requirement, check all the modules
>> > > > > transitively for used/unused tests.
>> > > > >
>> > > > > Regards,
>> > > > > --
>> > > > > Ilya Kasnacheev
>> > > > >
>> > > > >
>> > > > > ср, 2 дек. 2020 г. в 18:23, Max Timonin > >:
>> > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > > I don't think so. It looks like a bug that tests fail if one
>> runs
>> > > them
>> > > > > > within their module (actually, what is the goal of this test?).
>> The
>> > > > check
>> > > > > > showed us this problem, there is no need to fix the check.
>> > > > > >
>> > > > > > Currently I see two ways:
>> > > > > > 1. Find the right module for every misplaced test. There are 104
>> > > tests,
>> > > > > > maybe just move them all to the target module? If TeamCity runs
>> > them
>> > > > > within
>> > > > > > the indexing module only is there a reason to have a test in the
>> > core
>> > > > > > module at all?
>> > > > > > 2. Back to my previous proposal - create fake suites within a
>> > module,
>> > > > > then
>> > > > > > replace test classes in a target suite with the single class of
>> the
>> > > > fake
>> > > > > > suite.
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On Wed, Dec 2, 2020 at 5:38 PM Ilya Kasnacheev <
>> > > > > ilya.kasnach...@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hello!
>> > > > > > >
>> > > > > > > I think this means that we should abandon the plan of moving
>> > tests
>> > > > > > between
>> > > > > > > suites, and that your tool has to understand the dependency
>> graph
>> > > > > > > between modules' tests when assessing what's included and
>> what's
>> > > not.
>> > > > > > >
>> > > > > > > Regards,
>> > > > > > > --
>> > > > > > > Ilya Kasnacheev
>> > > > > > >
>> > > > > > >
>> > > > > > > ср, 2 дек. 2020 г. в 15:56, Max Timonin <
>> timonin.ma...@gmail.com
>> > >:
>> > > > > > >
>> > > > > > > > Hi, Ilya!
>> > > > > > > >
>> > > > > > > > I've checked testsuites. There is an issue. For example
>> > > > > > > > *IgniteBinaryCacheQueryTestSuite* suite is now in 2 modules:
>> > > > > > ignite-core,
>> > > > > > > > ignite-indexing. On TeamCity it runs 

Re: [DISCUSS] Missed (non-suited) tests

[Ilya Kasnacheev]

Hello!

Code mostly looks good, I have added a minor request. I will check how it
works and then we may commit.

+ zaleslaw@gmail.com

Can you please check whether the new ML suites make sense?
math/distances/DistancesTestSuite.java
naivebayes/NaiveBayesTestSuite.java

Would we need to add them to some TC runs?

Regards,
-- 
Ilya Kasnacheev


пн, 25 янв. 2021 г. в 22:07, Max Timonin :

> Hi, Ilya!
>
> I made a fix to the check. Now it aggregates info about tests and suites
> from all modules and then validates it. Could you please review the PR [1]?
>
> I tried to move some tests between modules, but unfortunately it still
> looks like spaghetti. So I reverted all changes to testsuites (new and
> splitted suites) and reworked the check.
>
> [1] https://github.com/apache/ignite/pull/8367
>
> On Mon, Dec 28, 2020 at 2:03 PM Ilya Kasnacheev  >
> wrote:
>
> > Hello!
> >
> > You could try to move these tests as .java files between modules in a
> > separate commit. I think I could review it.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > пт, 25 дек. 2020 г. в 17:19, Max Timonin :
> >
> > > Hi!
> > >
> > > Ilya thanks for the reply! I agree that it's a valid case when a test
> is
> > > part of multiple suites in different modules. But it is definitely a
> bug
> > > that the test is written in a module where it can't be run at all and
> > aimed
> > > to run within different modules (core tests in core that require h2). I
> > > propose to fix this issue.
> > >
> > > I'm going to check all such tests and move them to the right module.
> As I
> > > can see there are about 100 such test classes, but I hope that most of
> > them
> > > follow only a few patterns.
> > >
> > > On Fri, Dec 25, 2020 at 2:58 PM Ivan Daschinsky 
> > > wrote:
> > >
> > > > Hi!
> > > > >> I'm not sure that we should assume every test is only run from one
> > > test
> > > > suite. One test may be run from different test suites located in
> > > different
> > > > modules.
> > > > Agree. For example, if we introduce this limitation, zk suites will
> be
> > > > broken.
> > > >
> > > >
> > > > пт, 25 дек. 2020 г. в 14:48, Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com
> > > >:
> > > >
> > > > > Hello!
> > > > >
> > > > > Sorry for the long wait.
> > > > >
> > > > > I'm not sure that we should assume every test is only run from one
> > test
> > > > > suite. One test may be run from different test suites located in
> > > > different
> > > > > modules.
> > > > >
> > > > > I wonder if we can drop this requirement, check all the modules
> > > > > transitively for used/unused tests.
> > > > >
> > > > > Regards,
> > > > > --
> > > > > Ilya Kasnacheev
> > > > >
> > > > >
> > > > > ср, 2 дек. 2020 г. в 18:23, Max Timonin :
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I don't think so. It looks like a bug that tests fail if one runs
> > > them
> > > > > > within their module (actually, what is the goal of this test?).
> The
> > > > check
> > > > > > showed us this problem, there is no need to fix the check.
> > > > > >
> > > > > > Currently I see two ways:
> > > > > > 1. Find the right module for every misplaced test. There are 104
> > > tests,
> > > > > > maybe just move them all to the target module? If TeamCity runs
> > them
> > > > > within
> > > > > > the indexing module only is there a reason to have a test in the
> > core
> > > > > > module at all?
> > > > > > 2. Back to my previous proposal - create fake suites within a
> > module,
> > > > > then
> > > > > > replace test classes in a target suite with the single class of
> the
> > > > fake
> > > > > > suite.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Dec 2, 2020 at 5:38 PM Ilya Kasnacheev <
> > > > > ilya.kasnach...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hello!
> > > > > > >
> > > > > > > I think this means that we should abandon the plan of moving
> > tests
> > > > > > between
> > > > > > > suites, and that your tool has to understand the dependency
> graph
> > > > > > > between modules' tests when assessing what's included and
> what's
> > > not.
> > > > > > >
> > > > > > > Regards,
> > > > > > > --
> > > > > > > Ilya Kasnacheev
> > > > > > >
> > > > > > >
> > > > > > > ср, 2 дек. 2020 г. в 15:56, Max Timonin <
> timonin.ma...@gmail.com
> > >:
> > > > > > >
> > > > > > > > Hi, Ilya!
> > > > > > > >
> > > > > > > > I've checked testsuites. There is an issue. For example
> > > > > > > > *IgniteBinaryCacheQueryTestSuite* suite is now in 2 modules:
> > > > > > ignite-core,
> > > > > > > > ignite-indexing. On TeamCity it runs by "Query 1" suite.
> > > Simplified
> > > > > > maven
> > > > > > > > command for the suite is
> > > > > > > >
> > > > > > > > mvn -DtestIgniteBinaryCacheQueryTestSuite -am -pl
> > > :ignite-indexing
> > > > > > > > surefire:test
> > > > > > > >
> > > > > > > > Sequence of actions is:
> > > > > > > > 1. Find modules dependencies (*-am* flag): ignite-tools,
> > > > ignite-core;
> > > > > > > > 2. Run the test command for every module. 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi, Ilya!

I made a fix to the check. Now it aggregates info about tests and suites
from all modules and then validates it. Could you please review the PR [1]?

I tried to move some tests between modules, but unfortunately it still
looks like spaghetti. So I reverted all changes to testsuites (new and
splitted suites) and reworked the check.

[1] https://github.com/apache/ignite/pull/8367

On Mon, Dec 28, 2020 at 2:03 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> You could try to move these tests as .java files between modules in a
> separate commit. I think I could review it.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 25 дек. 2020 г. в 17:19, Max Timonin :
>
> > Hi!
> >
> > Ilya thanks for the reply! I agree that it's a valid case when a test is
> > part of multiple suites in different modules. But it is definitely a bug
> > that the test is written in a module where it can't be run at all and
> aimed
> > to run within different modules (core tests in core that require h2). I
> > propose to fix this issue.
> >
> > I'm going to check all such tests and move them to the right module. As I
> > can see there are about 100 such test classes, but I hope that most of
> them
> > follow only a few patterns.
> >
> > On Fri, Dec 25, 2020 at 2:58 PM Ivan Daschinsky 
> > wrote:
> >
> > > Hi!
> > > >> I'm not sure that we should assume every test is only run from one
> > test
> > > suite. One test may be run from different test suites located in
> > different
> > > modules.
> > > Agree. For example, if we introduce this limitation, zk suites will be
> > > broken.
> > >
> > >
> > > пт, 25 дек. 2020 г. в 14:48, Ilya Kasnacheev <
> ilya.kasnach...@gmail.com
> > >:
> > >
> > > > Hello!
> > > >
> > > > Sorry for the long wait.
> > > >
> > > > I'm not sure that we should assume every test is only run from one
> test
> > > > suite. One test may be run from different test suites located in
> > > different
> > > > modules.
> > > >
> > > > I wonder if we can drop this requirement, check all the modules
> > > > transitively for used/unused tests.
> > > >
> > > > Regards,
> > > > --
> > > > Ilya Kasnacheev
> > > >
> > > >
> > > > ср, 2 дек. 2020 г. в 18:23, Max Timonin :
> > > >
> > > > > Hi,
> > > > >
> > > > > I don't think so. It looks like a bug that tests fail if one runs
> > them
> > > > > within their module (actually, what is the goal of this test?). The
> > > check
> > > > > showed us this problem, there is no need to fix the check.
> > > > >
> > > > > Currently I see two ways:
> > > > > 1. Find the right module for every misplaced test. There are 104
> > tests,
> > > > > maybe just move them all to the target module? If TeamCity runs
> them
> > > > within
> > > > > the indexing module only is there a reason to have a test in the
> core
> > > > > module at all?
> > > > > 2. Back to my previous proposal - create fake suites within a
> module,
> > > > then
> > > > > replace test classes in a target suite with the single class of the
> > > fake
> > > > > suite.
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Dec 2, 2020 at 5:38 PM Ilya Kasnacheev <
> > > > ilya.kasnach...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hello!
> > > > > >
> > > > > > I think this means that we should abandon the plan of moving
> tests
> > > > > between
> > > > > > suites, and that your tool has to understand the dependency graph
> > > > > > between modules' tests when assessing what's included and what's
> > not.
> > > > > >
> > > > > > Regards,
> > > > > > --
> > > > > > Ilya Kasnacheev
> > > > > >
> > > > > >
> > > > > > ср, 2 дек. 2020 г. в 15:56, Max Timonin  >:
> > > > > >
> > > > > > > Hi, Ilya!
> > > > > > >
> > > > > > > I've checked testsuites. There is an issue. For example
> > > > > > > *IgniteBinaryCacheQueryTestSuite* suite is now in 2 modules:
> > > > > ignite-core,
> > > > > > > ignite-indexing. On TeamCity it runs by "Query 1" suite.
> > Simplified
> > > > > maven
> > > > > > > command for the suite is
> > > > > > >
> > > > > > > mvn -DtestIgniteBinaryCacheQueryTestSuite -am -pl
> > :ignite-indexing
> > > > > > > surefire:test
> > > > > > >
> > > > > > > Sequence of actions is:
> > > > > > > 1. Find modules dependencies (*-am* flag): ignite-tools,
> > > ignite-core;
> > > > > > > 2. Run the test command for every module. In this step the
> maven
> > > > tries
> > > > > to
> > > > > > > find the specified test for every module. This is good news, so
> > we
> > > > > don't
> > > > > > > need to create new TeamCity suites for such splitted suites.
> > > > > > >
> > > > > > > But the run performs within the current module classpath, so
> for
> > > the
> > > > > core
> > > > > > > module the test suite fails with error "Add module
> > > 'ignite-indexing'
> > > > to
> > > > > > the
> > > > > > > classpath of all Ignite nodes".  Maven can't resolve it.
> > > > > > >
> > > > > > > The only way to work with it is to specify additional classpath
> > > > > elements
> > > > > > > for tests with setting
> > > > > > 

Re: [DISCUSS] Missed (non-suited) tests

[Ilya Kasnacheev]

Hello!

You could try to move these tests as .java files between modules in a
separate commit. I think I could review it.

Regards,
-- 
Ilya Kasnacheev


пт, 25 дек. 2020 г. в 17:19, Max Timonin :

> Hi!
>
> Ilya thanks for the reply! I agree that it's a valid case when a test is
> part of multiple suites in different modules. But it is definitely a bug
> that the test is written in a module where it can't be run at all and aimed
> to run within different modules (core tests in core that require h2). I
> propose to fix this issue.
>
> I'm going to check all such tests and move them to the right module. As I
> can see there are about 100 such test classes, but I hope that most of them
> follow only a few patterns.
>
> On Fri, Dec 25, 2020 at 2:58 PM Ivan Daschinsky 
> wrote:
>
> > Hi!
> > >> I'm not sure that we should assume every test is only run from one
> test
> > suite. One test may be run from different test suites located in
> different
> > modules.
> > Agree. For example, if we introduce this limitation, zk suites will be
> > broken.
> >
> >
> > пт, 25 дек. 2020 г. в 14:48, Ilya Kasnacheev  >:
> >
> > > Hello!
> > >
> > > Sorry for the long wait.
> > >
> > > I'm not sure that we should assume every test is only run from one test
> > > suite. One test may be run from different test suites located in
> > different
> > > modules.
> > >
> > > I wonder if we can drop this requirement, check all the modules
> > > transitively for used/unused tests.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > ср, 2 дек. 2020 г. в 18:23, Max Timonin :
> > >
> > > > Hi,
> > > >
> > > > I don't think so. It looks like a bug that tests fail if one runs
> them
> > > > within their module (actually, what is the goal of this test?). The
> > check
> > > > showed us this problem, there is no need to fix the check.
> > > >
> > > > Currently I see two ways:
> > > > 1. Find the right module for every misplaced test. There are 104
> tests,
> > > > maybe just move them all to the target module? If TeamCity runs them
> > > within
> > > > the indexing module only is there a reason to have a test in the core
> > > > module at all?
> > > > 2. Back to my previous proposal - create fake suites within a module,
> > > then
> > > > replace test classes in a target suite with the single class of the
> > fake
> > > > suite.
> > > >
> > > >
> > > >
> > > > On Wed, Dec 2, 2020 at 5:38 PM Ilya Kasnacheev <
> > > ilya.kasnach...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hello!
> > > > >
> > > > > I think this means that we should abandon the plan of moving tests
> > > > between
> > > > > suites, and that your tool has to understand the dependency graph
> > > > > between modules' tests when assessing what's included and what's
> not.
> > > > >
> > > > > Regards,
> > > > > --
> > > > > Ilya Kasnacheev
> > > > >
> > > > >
> > > > > ср, 2 дек. 2020 г. в 15:56, Max Timonin :
> > > > >
> > > > > > Hi, Ilya!
> > > > > >
> > > > > > I've checked testsuites. There is an issue. For example
> > > > > > *IgniteBinaryCacheQueryTestSuite* suite is now in 2 modules:
> > > > ignite-core,
> > > > > > ignite-indexing. On TeamCity it runs by "Query 1" suite.
> Simplified
> > > > maven
> > > > > > command for the suite is
> > > > > >
> > > > > > mvn -DtestIgniteBinaryCacheQueryTestSuite -am -pl
> :ignite-indexing
> > > > > > surefire:test
> > > > > >
> > > > > > Sequence of actions is:
> > > > > > 1. Find modules dependencies (*-am* flag): ignite-tools,
> > ignite-core;
> > > > > > 2. Run the test command for every module. In this step the maven
> > > tries
> > > > to
> > > > > > find the specified test for every module. This is good news, so
> we
> > > > don't
> > > > > > need to create new TeamCity suites for such splitted suites.
> > > > > >
> > > > > > But the run performs within the current module classpath, so for
> > the
> > > > core
> > > > > > module the test suite fails with error "Add module
> > 'ignite-indexing'
> > > to
> > > > > the
> > > > > > classpath of all Ignite nodes".  Maven can't resolve it.
> > > > > >
> > > > > > The only way to work with it is to specify additional classpath
> > > > elements
> > > > > > for tests with setting
> > > > > *-Dmaven.test.additionalClasspath=/path/to/m2/jar*.
> > > > > > I did it by filling MAVEN_OPTS with the setting. Please check the
> > job
> > > > > > parameters [1]. After that the core module part ran successfully.
> > It
> > > > > means
> > > > > > for every TC suite that runs such splitted suite we need to set
> the
> > > > > > setting. What do you think, is it a valid way to handle the
> issue?
> > If
> > > > > there
> > > > > > are no objections, I will check other such suites.
> > > > > >
> > > > > > Also to mention there, the work directory contains a
> *repository/*
> > > > folder
> > > > > > with all required .jars. But usage of this path in the setting
> > didn't
> > > > > help.
> > > > > > I'm not sure, but I think it's an issue due to usage of
> > Classworlds.
> > > > So,
> 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi!

Ilya thanks for the reply! I agree that it's a valid case when a test is
part of multiple suites in different modules. But it is definitely a bug
that the test is written in a module where it can't be run at all and aimed
to run within different modules (core tests in core that require h2). I
propose to fix this issue.

I'm going to check all such tests and move them to the right module. As I
can see there are about 100 such test classes, but I hope that most of them
follow only a few patterns.

On Fri, Dec 25, 2020 at 2:58 PM Ivan Daschinsky  wrote:

> Hi!
> >> I'm not sure that we should assume every test is only run from one test
> suite. One test may be run from different test suites located in different
> modules.
> Agree. For example, if we introduce this limitation, zk suites will be
> broken.
>
>
> пт, 25 дек. 2020 г. в 14:48, Ilya Kasnacheev :
>
> > Hello!
> >
> > Sorry for the long wait.
> >
> > I'm not sure that we should assume every test is only run from one test
> > suite. One test may be run from different test suites located in
> different
> > modules.
> >
> > I wonder if we can drop this requirement, check all the modules
> > transitively for used/unused tests.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > ср, 2 дек. 2020 г. в 18:23, Max Timonin :
> >
> > > Hi,
> > >
> > > I don't think so. It looks like a bug that tests fail if one runs them
> > > within their module (actually, what is the goal of this test?). The
> check
> > > showed us this problem, there is no need to fix the check.
> > >
> > > Currently I see two ways:
> > > 1. Find the right module for every misplaced test. There are 104 tests,
> > > maybe just move them all to the target module? If TeamCity runs them
> > within
> > > the indexing module only is there a reason to have a test in the core
> > > module at all?
> > > 2. Back to my previous proposal - create fake suites within a module,
> > then
> > > replace test classes in a target suite with the single class of the
> fake
> > > suite.
> > >
> > >
> > >
> > > On Wed, Dec 2, 2020 at 5:38 PM Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com>
> > > wrote:
> > >
> > > > Hello!
> > > >
> > > > I think this means that we should abandon the plan of moving tests
> > > between
> > > > suites, and that your tool has to understand the dependency graph
> > > > between modules' tests when assessing what's included and what's not.
> > > >
> > > > Regards,
> > > > --
> > > > Ilya Kasnacheev
> > > >
> > > >
> > > > ср, 2 дек. 2020 г. в 15:56, Max Timonin :
> > > >
> > > > > Hi, Ilya!
> > > > >
> > > > > I've checked testsuites. There is an issue. For example
> > > > > *IgniteBinaryCacheQueryTestSuite* suite is now in 2 modules:
> > > ignite-core,
> > > > > ignite-indexing. On TeamCity it runs by "Query 1" suite. Simplified
> > > maven
> > > > > command for the suite is
> > > > >
> > > > > mvn -DtestIgniteBinaryCacheQueryTestSuite -am -pl :ignite-indexing
> > > > > surefire:test
> > > > >
> > > > > Sequence of actions is:
> > > > > 1. Find modules dependencies (*-am* flag): ignite-tools,
> ignite-core;
> > > > > 2. Run the test command for every module. In this step the maven
> > tries
> > > to
> > > > > find the specified test for every module. This is good news, so we
> > > don't
> > > > > need to create new TeamCity suites for such splitted suites.
> > > > >
> > > > > But the run performs within the current module classpath, so for
> the
> > > core
> > > > > module the test suite fails with error "Add module
> 'ignite-indexing'
> > to
> > > > the
> > > > > classpath of all Ignite nodes".  Maven can't resolve it.
> > > > >
> > > > > The only way to work with it is to specify additional classpath
> > > elements
> > > > > for tests with setting
> > > > *-Dmaven.test.additionalClasspath=/path/to/m2/jar*.
> > > > > I did it by filling MAVEN_OPTS with the setting. Please check the
> job
> > > > > parameters [1]. After that the core module part ran successfully.
> It
> > > > means
> > > > > for every TC suite that runs such splitted suite we need to set the
> > > > > setting. What do you think, is it a valid way to handle the issue?
> If
> > > > there
> > > > > are no objections, I will check other such suites.
> > > > >
> > > > > Also to mention there, the work directory contains a *repository/*
> > > folder
> > > > > with all required .jars. But usage of this path in the setting
> didn't
> > > > help.
> > > > > I'm not sure, but I think it's an issue due to usage of
> Classworlds.
> > > So,
> > > > > using dependency from .m2 is the only way.
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=5770727=IgniteTests24Java8_Queries1=buildParameters
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Nov 27, 2020 at 3:55 PM Max Timonin <
> timonin.ma...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Sure, I'll do that.
> > > > > >
> > > > > > On Fri, Nov 27, 2020 at 2:00 PM Ilya Kasnacheev <
> > > > > 

Re: [DISCUSS] Missed (non-suited) tests

[Ivan Daschinsky]

Hi!
>> I'm not sure that we should assume every test is only run from one test
suite. One test may be run from different test suites located in different
modules.
Agree. For example, if we introduce this limitation, zk suites will be
broken.


пт, 25 дек. 2020 г. в 14:48, Ilya Kasnacheev :

> Hello!
>
> Sorry for the long wait.
>
> I'm not sure that we should assume every test is only run from one test
> suite. One test may be run from different test suites located in different
> modules.
>
> I wonder if we can drop this requirement, check all the modules
> transitively for used/unused tests.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 2 дек. 2020 г. в 18:23, Max Timonin :
>
> > Hi,
> >
> > I don't think so. It looks like a bug that tests fail if one runs them
> > within their module (actually, what is the goal of this test?). The check
> > showed us this problem, there is no need to fix the check.
> >
> > Currently I see two ways:
> > 1. Find the right module for every misplaced test. There are 104 tests,
> > maybe just move them all to the target module? If TeamCity runs them
> within
> > the indexing module only is there a reason to have a test in the core
> > module at all?
> > 2. Back to my previous proposal - create fake suites within a module,
> then
> > replace test classes in a target suite with the single class of the fake
> > suite.
> >
> >
> >
> > On Wed, Dec 2, 2020 at 5:38 PM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com>
> > wrote:
> >
> > > Hello!
> > >
> > > I think this means that we should abandon the plan of moving tests
> > between
> > > suites, and that your tool has to understand the dependency graph
> > > between modules' tests when assessing what's included and what's not.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > ср, 2 дек. 2020 г. в 15:56, Max Timonin :
> > >
> > > > Hi, Ilya!
> > > >
> > > > I've checked testsuites. There is an issue. For example
> > > > *IgniteBinaryCacheQueryTestSuite* suite is now in 2 modules:
> > ignite-core,
> > > > ignite-indexing. On TeamCity it runs by "Query 1" suite. Simplified
> > maven
> > > > command for the suite is
> > > >
> > > > mvn -DtestIgniteBinaryCacheQueryTestSuite -am -pl :ignite-indexing
> > > > surefire:test
> > > >
> > > > Sequence of actions is:
> > > > 1. Find modules dependencies (*-am* flag): ignite-tools, ignite-core;
> > > > 2. Run the test command for every module. In this step the maven
> tries
> > to
> > > > find the specified test for every module. This is good news, so we
> > don't
> > > > need to create new TeamCity suites for such splitted suites.
> > > >
> > > > But the run performs within the current module classpath, so for the
> > core
> > > > module the test suite fails with error "Add module 'ignite-indexing'
> to
> > > the
> > > > classpath of all Ignite nodes".  Maven can't resolve it.
> > > >
> > > > The only way to work with it is to specify additional classpath
> > elements
> > > > for tests with setting
> > > *-Dmaven.test.additionalClasspath=/path/to/m2/jar*.
> > > > I did it by filling MAVEN_OPTS with the setting. Please check the job
> > > > parameters [1]. After that the core module part ran successfully. It
> > > means
> > > > for every TC suite that runs such splitted suite we need to set the
> > > > setting. What do you think, is it a valid way to handle the issue? If
> > > there
> > > > are no objections, I will check other such suites.
> > > >
> > > > Also to mention there, the work directory contains a *repository/*
> > folder
> > > > with all required .jars. But usage of this path in the setting didn't
> > > help.
> > > > I'm not sure, but I think it's an issue due to usage of Classworlds.
> > So,
> > > > using dependency from .m2 is the only way.
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=5770727=IgniteTests24Java8_Queries1=buildParameters
> > > >
> > > >
> > > >
> > > > On Fri, Nov 27, 2020 at 3:55 PM Max Timonin  >
> > > > wrote:
> > > >
> > > > > Sure, I'll do that.
> > > > >
> > > > > On Fri, Nov 27, 2020 at 2:00 PM Ilya Kasnacheev <
> > > > ilya.kasnach...@gmail.com>
> > > > > wrote:
> > > > >
> > > > >> Hello!
> > > > >>
> > > > >> You can override these values (module, suites) values when
> running a
> > > > suite
> > > > >> on TC. Can you please run these ones which need to be changed
> > > > individually
> > > > >> on TC, make sure they run without errors and contain all the
> needed
> > > > tests,
> > > > >> and link to these runs in the ticket? Then I can modify the suites
> > to
> > > > fit
> > > > >> those.
> > > > >>
> > > > >> I'm not sure that class shadowing will work as we want it to work,
> > > e.g.,
> > > > >> we
> > > > >> now have two IgniteCacheQuerySelfTestSuite6 with the same FQDN,
> I'm
> > > not
> > > > >> sure if maven/TC is going to pick both or just one.
> > > > >> Maybe they should go to a different package, e.g., testsuites/core
> > for
> > > > >> every suite already present in 

Re: [DISCUSS] Missed (non-suited) tests

[Ilya Kasnacheev]

Hello!

Sorry for the long wait.

I'm not sure that we should assume every test is only run from one test
suite. One test may be run from different test suites located in different
modules.

I wonder if we can drop this requirement, check all the modules
transitively for used/unused tests.

Regards,
-- 
Ilya Kasnacheev


ср, 2 дек. 2020 г. в 18:23, Max Timonin :

> Hi,
>
> I don't think so. It looks like a bug that tests fail if one runs them
> within their module (actually, what is the goal of this test?). The check
> showed us this problem, there is no need to fix the check.
>
> Currently I see two ways:
> 1. Find the right module for every misplaced test. There are 104 tests,
> maybe just move them all to the target module? If TeamCity runs them within
> the indexing module only is there a reason to have a test in the core
> module at all?
> 2. Back to my previous proposal - create fake suites within a module, then
> replace test classes in a target suite with the single class of the fake
> suite.
>
>
>
> On Wed, Dec 2, 2020 at 5:38 PM Ilya Kasnacheev 
> wrote:
>
> > Hello!
> >
> > I think this means that we should abandon the plan of moving tests
> between
> > suites, and that your tool has to understand the dependency graph
> > between modules' tests when assessing what's included and what's not.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > ср, 2 дек. 2020 г. в 15:56, Max Timonin :
> >
> > > Hi, Ilya!
> > >
> > > I've checked testsuites. There is an issue. For example
> > > *IgniteBinaryCacheQueryTestSuite* suite is now in 2 modules:
> ignite-core,
> > > ignite-indexing. On TeamCity it runs by "Query 1" suite. Simplified
> maven
> > > command for the suite is
> > >
> > > mvn -DtestIgniteBinaryCacheQueryTestSuite -am -pl :ignite-indexing
> > > surefire:test
> > >
> > > Sequence of actions is:
> > > 1. Find modules dependencies (*-am* flag): ignite-tools, ignite-core;
> > > 2. Run the test command for every module. In this step the maven tries
> to
> > > find the specified test for every module. This is good news, so we
> don't
> > > need to create new TeamCity suites for such splitted suites.
> > >
> > > But the run performs within the current module classpath, so for the
> core
> > > module the test suite fails with error "Add module 'ignite-indexing' to
> > the
> > > classpath of all Ignite nodes".  Maven can't resolve it.
> > >
> > > The only way to work with it is to specify additional classpath
> elements
> > > for tests with setting
> > *-Dmaven.test.additionalClasspath=/path/to/m2/jar*.
> > > I did it by filling MAVEN_OPTS with the setting. Please check the job
> > > parameters [1]. After that the core module part ran successfully. It
> > means
> > > for every TC suite that runs such splitted suite we need to set the
> > > setting. What do you think, is it a valid way to handle the issue? If
> > there
> > > are no objections, I will check other such suites.
> > >
> > > Also to mention there, the work directory contains a *repository/*
> folder
> > > with all required .jars. But usage of this path in the setting didn't
> > help.
> > > I'm not sure, but I think it's an issue due to usage of Classworlds.
> So,
> > > using dependency from .m2 is the only way.
> > >
> > > [1]
> > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=5770727=IgniteTests24Java8_Queries1=buildParameters
> > >
> > >
> > >
> > > On Fri, Nov 27, 2020 at 3:55 PM Max Timonin 
> > > wrote:
> > >
> > > > Sure, I'll do that.
> > > >
> > > > On Fri, Nov 27, 2020 at 2:00 PM Ilya Kasnacheev <
> > > ilya.kasnach...@gmail.com>
> > > > wrote:
> > > >
> > > >> Hello!
> > > >>
> > > >> You can override these values (module, suites) values when running a
> > > suite
> > > >> on TC. Can you please run these ones which need to be changed
> > > individually
> > > >> on TC, make sure they run without errors and contain all the needed
> > > tests,
> > > >> and link to these runs in the ticket? Then I can modify the suites
> to
> > > fit
> > > >> those.
> > > >>
> > > >> I'm not sure that class shadowing will work as we want it to work,
> > e.g.,
> > > >> we
> > > >> now have two IgniteCacheQuerySelfTestSuite6 with the same FQDN, I'm
> > not
> > > >> sure if maven/TC is going to pick both or just one.
> > > >> Maybe they should go to a different package, e.g., testsuites/core
> for
> > > >> every suite already present in indexing/spring/etc. Maybe you can
> > rename
> > > >> them just now? This will mean a lot less of work reconfiguring
> suites.
> > > >> In TC configurations, suite names are simple class names, not FQ, so
> > no
> > > >> changes may be needed at all.
> > > >>
> > > >> Regards,
> > > >> --
> > > >> Ilya Kasnacheev
> > > >>
> > > >>
> > > >> пт, 27 нояб. 2020 г. в 13:03, Max Timonin  >:
> > > >>
> > > >> > Hi, sorry for the misleading. I mean "adding ignite-core module
> > > >> *suites* to
> > > >> > the TeamCity Queries* suite"
> > > >> >
> > > >> > On Fri, Nov 27, 2020 at 12:44 PM Ilya Kasnacheev <
> > 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi,

I don't think so. It looks like a bug that tests fail if one runs them
within their module (actually, what is the goal of this test?). The check
showed us this problem, there is no need to fix the check.

Currently I see two ways:
1. Find the right module for every misplaced test. There are 104 tests,
maybe just move them all to the target module? If TeamCity runs them within
the indexing module only is there a reason to have a test in the core
module at all?
2. Back to my previous proposal - create fake suites within a module, then
replace test classes in a target suite with the single class of the fake
suite.



On Wed, Dec 2, 2020 at 5:38 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> I think this means that we should abandon the plan of moving tests between
> suites, and that your tool has to understand the dependency graph
> between modules' tests when assessing what's included and what's not.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 2 дек. 2020 г. в 15:56, Max Timonin :
>
> > Hi, Ilya!
> >
> > I've checked testsuites. There is an issue. For example
> > *IgniteBinaryCacheQueryTestSuite* suite is now in 2 modules: ignite-core,
> > ignite-indexing. On TeamCity it runs by "Query 1" suite. Simplified maven
> > command for the suite is
> >
> > mvn -DtestIgniteBinaryCacheQueryTestSuite -am -pl :ignite-indexing
> > surefire:test
> >
> > Sequence of actions is:
> > 1. Find modules dependencies (*-am* flag): ignite-tools, ignite-core;
> > 2. Run the test command for every module. In this step the maven tries to
> > find the specified test for every module. This is good news, so we don't
> > need to create new TeamCity suites for such splitted suites.
> >
> > But the run performs within the current module classpath, so for the core
> > module the test suite fails with error "Add module 'ignite-indexing' to
> the
> > classpath of all Ignite nodes".  Maven can't resolve it.
> >
> > The only way to work with it is to specify additional classpath elements
> > for tests with setting
> *-Dmaven.test.additionalClasspath=/path/to/m2/jar*.
> > I did it by filling MAVEN_OPTS with the setting. Please check the job
> > parameters [1]. After that the core module part ran successfully. It
> means
> > for every TC suite that runs such splitted suite we need to set the
> > setting. What do you think, is it a valid way to handle the issue? If
> there
> > are no objections, I will check other such suites.
> >
> > Also to mention there, the work directory contains a *repository/* folder
> > with all required .jars. But usage of this path in the setting didn't
> help.
> > I'm not sure, but I think it's an issue due to usage of Classworlds. So,
> > using dependency from .m2 is the only way.
> >
> > [1]
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=5770727=IgniteTests24Java8_Queries1=buildParameters
> >
> >
> >
> > On Fri, Nov 27, 2020 at 3:55 PM Max Timonin 
> > wrote:
> >
> > > Sure, I'll do that.
> > >
> > > On Fri, Nov 27, 2020 at 2:00 PM Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com>
> > > wrote:
> > >
> > >> Hello!
> > >>
> > >> You can override these values (module, suites) values when running a
> > suite
> > >> on TC. Can you please run these ones which need to be changed
> > individually
> > >> on TC, make sure they run without errors and contain all the needed
> > tests,
> > >> and link to these runs in the ticket? Then I can modify the suites to
> > fit
> > >> those.
> > >>
> > >> I'm not sure that class shadowing will work as we want it to work,
> e.g.,
> > >> we
> > >> now have two IgniteCacheQuerySelfTestSuite6 with the same FQDN, I'm
> not
> > >> sure if maven/TC is going to pick both or just one.
> > >> Maybe they should go to a different package, e.g., testsuites/core for
> > >> every suite already present in indexing/spring/etc. Maybe you can
> rename
> > >> them just now? This will mean a lot less of work reconfiguring suites.
> > >> In TC configurations, suite names are simple class names, not FQ, so
> no
> > >> changes may be needed at all.
> > >>
> > >> Regards,
> > >> --
> > >> Ilya Kasnacheev
> > >>
> > >>
> > >> пт, 27 нояб. 2020 г. в 13:03, Max Timonin :
> > >>
> > >> > Hi, sorry for the misleading. I mean "adding ignite-core module
> > >> *suites* to
> > >> > the TeamCity Queries* suite"
> > >> >
> > >> > On Fri, Nov 27, 2020 at 12:44 PM Ilya Kasnacheev <
> > >> > ilya.kasnach...@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Hello!
> > >> > >
> > >> > > What do you mean by "adding ignite-core to suite"? ignite-core is
> a
> > >> top
> > >> > > dependency and its tests are also included in all other modules'
> > tests
> > >> > > classpath since it provides GridAbstractTest.
> > >> > >
> > >> > > Regards,
> > >> > > --
> > >> > > Ilya Kasnacheev
> > >> > >
> > >> > >
> > >> > > пт, 27 нояб. 2020 г. в 01:24, Max Timonin <
> timonin.ma...@gmail.com
> > >:
> > >> > >
> > >> > > > Hi, Ilya!
> > >> > > >
> > >> > > > So, I've updated PR, fixed comments and removed Core* prefixes.
> > >> 

Re: [DISCUSS] Missed (non-suited) tests

[Ilya Kasnacheev]

Hello!

I think this means that we should abandon the plan of moving tests between
suites, and that your tool has to understand the dependency graph
between modules' tests when assessing what's included and what's not.

Regards,
-- 
Ilya Kasnacheev


ср, 2 дек. 2020 г. в 15:56, Max Timonin :

> Hi, Ilya!
>
> I've checked testsuites. There is an issue. For example
> *IgniteBinaryCacheQueryTestSuite* suite is now in 2 modules: ignite-core,
> ignite-indexing. On TeamCity it runs by "Query 1" suite. Simplified maven
> command for the suite is
>
> mvn -DtestIgniteBinaryCacheQueryTestSuite -am -pl :ignite-indexing
> surefire:test
>
> Sequence of actions is:
> 1. Find modules dependencies (*-am* flag): ignite-tools, ignite-core;
> 2. Run the test command for every module. In this step the maven tries to
> find the specified test for every module. This is good news, so we don't
> need to create new TeamCity suites for such splitted suites.
>
> But the run performs within the current module classpath, so for the core
> module the test suite fails with error "Add module 'ignite-indexing' to the
> classpath of all Ignite nodes".  Maven can't resolve it.
>
> The only way to work with it is to specify additional classpath elements
> for tests with setting *-Dmaven.test.additionalClasspath=/path/to/m2/jar*.
> I did it by filling MAVEN_OPTS with the setting. Please check the job
> parameters [1]. After that the core module part ran successfully. It means
> for every TC suite that runs such splitted suite we need to set the
> setting. What do you think, is it a valid way to handle the issue? If there
> are no objections, I will check other such suites.
>
> Also to mention there, the work directory contains a *repository/* folder
> with all required .jars. But usage of this path in the setting didn't help.
> I'm not sure, but I think it's an issue due to usage of Classworlds. So,
> using dependency from .m2 is the only way.
>
> [1]
>
> https://ci.ignite.apache.org/viewLog.html?buildId=5770727=IgniteTests24Java8_Queries1=buildParameters
>
>
>
> On Fri, Nov 27, 2020 at 3:55 PM Max Timonin 
> wrote:
>
> > Sure, I'll do that.
> >
> > On Fri, Nov 27, 2020 at 2:00 PM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com>
> > wrote:
> >
> >> Hello!
> >>
> >> You can override these values (module, suites) values when running a
> suite
> >> on TC. Can you please run these ones which need to be changed
> individually
> >> on TC, make sure they run without errors and contain all the needed
> tests,
> >> and link to these runs in the ticket? Then I can modify the suites to
> fit
> >> those.
> >>
> >> I'm not sure that class shadowing will work as we want it to work, e.g.,
> >> we
> >> now have two IgniteCacheQuerySelfTestSuite6 with the same FQDN, I'm not
> >> sure if maven/TC is going to pick both or just one.
> >> Maybe they should go to a different package, e.g., testsuites/core for
> >> every suite already present in indexing/spring/etc. Maybe you can rename
> >> them just now? This will mean a lot less of work reconfiguring suites.
> >> In TC configurations, suite names are simple class names, not FQ, so no
> >> changes may be needed at all.
> >>
> >> Regards,
> >> --
> >> Ilya Kasnacheev
> >>
> >>
> >> пт, 27 нояб. 2020 г. в 13:03, Max Timonin :
> >>
> >> > Hi, sorry for the misleading. I mean "adding ignite-core module
> >> *suites* to
> >> > the TeamCity Queries* suite"
> >> >
> >> > On Fri, Nov 27, 2020 at 12:44 PM Ilya Kasnacheev <
> >> > ilya.kasnach...@gmail.com>
> >> > wrote:
> >> >
> >> > > Hello!
> >> > >
> >> > > What do you mean by "adding ignite-core to suite"? ignite-core is a
> >> top
> >> > > dependency and its tests are also included in all other modules'
> tests
> >> > > classpath since it provides GridAbstractTest.
> >> > >
> >> > > Regards,
> >> > > --
> >> > > Ilya Kasnacheev
> >> > >
> >> > >
> >> > > пт, 27 нояб. 2020 г. в 01:24, Max Timonin  >:
> >> > >
> >> > > > Hi, Ilya!
> >> > > >
> >> > > > So, I've updated PR, fixed comments and removed Core* prefixes.
> >> MTCGA
> >> > > shows
> >> > > > no blockers, but it was 2 weeks ago, so I've started it again.
> >> > > >
> >> > > > If PR is OK then there are some suites that should be updated on
> TC.
> >> > > Could
> >> > > > you please tell me how we can do it?
> >> > > >
> >> > > > 1. Add ignite-cassandra-serializers suite:
> >> > > >
> >> > > >1. org.apache.ignite.tests.SerializerSuite
> >> > > >
> >> > > > 2. Add ignite-core to Queries* TC suite:
> >> > > >
> >> > > >1. org.apache.ignite.client.IgniteClientTestSuite
> >> > > >2. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite
> >> > > >3. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite2
> >> > > >4. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite3
> >> > > >5. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite4
> >> > > >6. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite5
> >> > > >7. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite6
> >> 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi, Ilya!

I've checked testsuites. There is an issue. For example
*IgniteBinaryCacheQueryTestSuite* suite is now in 2 modules: ignite-core,
ignite-indexing. On TeamCity it runs by "Query 1" suite. Simplified maven
command for the suite is

mvn -DtestIgniteBinaryCacheQueryTestSuite -am -pl :ignite-indexing
surefire:test

Sequence of actions is:
1. Find modules dependencies (*-am* flag): ignite-tools, ignite-core;
2. Run the test command for every module. In this step the maven tries to
find the specified test for every module. This is good news, so we don't
need to create new TeamCity suites for such splitted suites.

But the run performs within the current module classpath, so for the core
module the test suite fails with error "Add module 'ignite-indexing' to the
classpath of all Ignite nodes".  Maven can't resolve it.

The only way to work with it is to specify additional classpath elements
for tests with setting *-Dmaven.test.additionalClasspath=/path/to/m2/jar*.
I did it by filling MAVEN_OPTS with the setting. Please check the job
parameters [1]. After that the core module part ran successfully. It means
for every TC suite that runs such splitted suite we need to set the
setting. What do you think, is it a valid way to handle the issue? If there
are no objections, I will check other such suites.

Also to mention there, the work directory contains a *repository/* folder
with all required .jars. But usage of this path in the setting didn't help.
I'm not sure, but I think it's an issue due to usage of Classworlds. So,
using dependency from .m2 is the only way.

[1]
https://ci.ignite.apache.org/viewLog.html?buildId=5770727=IgniteTests24Java8_Queries1=buildParameters



On Fri, Nov 27, 2020 at 3:55 PM Max Timonin  wrote:

> Sure, I'll do that.
>
> On Fri, Nov 27, 2020 at 2:00 PM Ilya Kasnacheev 
> wrote:
>
>> Hello!
>>
>> You can override these values (module, suites) values when running a suite
>> on TC. Can you please run these ones which need to be changed individually
>> on TC, make sure they run without errors and contain all the needed tests,
>> and link to these runs in the ticket? Then I can modify the suites to fit
>> those.
>>
>> I'm not sure that class shadowing will work as we want it to work, e.g.,
>> we
>> now have two IgniteCacheQuerySelfTestSuite6 with the same FQDN, I'm not
>> sure if maven/TC is going to pick both or just one.
>> Maybe they should go to a different package, e.g., testsuites/core for
>> every suite already present in indexing/spring/etc. Maybe you can rename
>> them just now? This will mean a lot less of work reconfiguring suites.
>> In TC configurations, suite names are simple class names, not FQ, so no
>> changes may be needed at all.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пт, 27 нояб. 2020 г. в 13:03, Max Timonin :
>>
>> > Hi, sorry for the misleading. I mean "adding ignite-core module
>> *suites* to
>> > the TeamCity Queries* suite"
>> >
>> > On Fri, Nov 27, 2020 at 12:44 PM Ilya Kasnacheev <
>> > ilya.kasnach...@gmail.com>
>> > wrote:
>> >
>> > > Hello!
>> > >
>> > > What do you mean by "adding ignite-core to suite"? ignite-core is a
>> top
>> > > dependency and its tests are also included in all other modules' tests
>> > > classpath since it provides GridAbstractTest.
>> > >
>> > > Regards,
>> > > --
>> > > Ilya Kasnacheev
>> > >
>> > >
>> > > пт, 27 нояб. 2020 г. в 01:24, Max Timonin :
>> > >
>> > > > Hi, Ilya!
>> > > >
>> > > > So, I've updated PR, fixed comments and removed Core* prefixes.
>> MTCGA
>> > > shows
>> > > > no blockers, but it was 2 weeks ago, so I've started it again.
>> > > >
>> > > > If PR is OK then there are some suites that should be updated on TC.
>> > > Could
>> > > > you please tell me how we can do it?
>> > > >
>> > > > 1. Add ignite-cassandra-serializers suite:
>> > > >
>> > > >1. org.apache.ignite.tests.SerializerSuite
>> > > >
>> > > > 2. Add ignite-core to Queries* TC suite:
>> > > >
>> > > >1. org.apache.ignite.client.IgniteClientTestSuite
>> > > >2. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite
>> > > >3. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite2
>> > > >4. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite3
>> > > >5. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite4
>> > > >6. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite5
>> > > >7. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite6
>> > > >8. org.apache.ignite.suites.IgnitePdsWithIndexingCoreTestSuite
>> > > >9. org.apache.ignite.suites.IgniteCacheMvccSqlTestSuite
>> > > >
>> > > > 3. Remove ignite-indexing from TC suites:
>> > > >
>> > > >1. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite3
>> > > >2. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite4
>> > > >3. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite5
>> > > >
>> > > > 4. Add ignite-core to Spring* TC suite:
>> > > >
>> > > >1. 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Sure, I'll do that.

On Fri, Nov 27, 2020 at 2:00 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> You can override these values (module, suites) values when running a suite
> on TC. Can you please run these ones which need to be changed individually
> on TC, make sure they run without errors and contain all the needed tests,
> and link to these runs in the ticket? Then I can modify the suites to fit
> those.
>
> I'm not sure that class shadowing will work as we want it to work, e.g., we
> now have two IgniteCacheQuerySelfTestSuite6 with the same FQDN, I'm not
> sure if maven/TC is going to pick both or just one.
> Maybe they should go to a different package, e.g., testsuites/core for
> every suite already present in indexing/spring/etc. Maybe you can rename
> them just now? This will mean a lot less of work reconfiguring suites.
> In TC configurations, suite names are simple class names, not FQ, so no
> changes may be needed at all.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 27 нояб. 2020 г. в 13:03, Max Timonin :
>
> > Hi, sorry for the misleading. I mean "adding ignite-core module *suites*
> to
> > the TeamCity Queries* suite"
> >
> > On Fri, Nov 27, 2020 at 12:44 PM Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com>
> > wrote:
> >
> > > Hello!
> > >
> > > What do you mean by "adding ignite-core to suite"? ignite-core is a top
> > > dependency and its tests are also included in all other modules' tests
> > > classpath since it provides GridAbstractTest.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > пт, 27 нояб. 2020 г. в 01:24, Max Timonin :
> > >
> > > > Hi, Ilya!
> > > >
> > > > So, I've updated PR, fixed comments and removed Core* prefixes. MTCGA
> > > shows
> > > > no blockers, but it was 2 weeks ago, so I've started it again.
> > > >
> > > > If PR is OK then there are some suites that should be updated on TC.
> > > Could
> > > > you please tell me how we can do it?
> > > >
> > > > 1. Add ignite-cassandra-serializers suite:
> > > >
> > > >1. org.apache.ignite.tests.SerializerSuite
> > > >
> > > > 2. Add ignite-core to Queries* TC suite:
> > > >
> > > >1. org.apache.ignite.client.IgniteClientTestSuite
> > > >2. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite
> > > >3. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite2
> > > >4. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite3
> > > >5. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite4
> > > >6. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite5
> > > >7. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite6
> > > >8. org.apache.ignite.suites.IgnitePdsWithIndexingCoreTestSuite
> > > >9. org.apache.ignite.suites.IgniteCacheMvccSqlTestSuite
> > > >
> > > > 3. Remove ignite-indexing from TC suites:
> > > >
> > > >1. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite3
> > > >2. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite4
> > > >3. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite5
> > > >
> > > > 4. Add ignite-core to Spring* TC suite:
> > > >
> > > >1. org.apache.ignite.testsuites.IgniteSpringTestSuite
> > > >
> > > > 5. Add ignite-core suite (depends on uri-deployment module):
> > > >
> > > >1. org.apache.ignite.testsuites.IgniteUriDeploymentTestSuite
> > > >
> > > > 6. Add ignite-core suite to Zookeeper TC suite:
> > > >
> > > >1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3
> > > >
> > > > 7. Remove ignite-zookeeper test suite:
> > > >
> > > >1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3
> > > >
> > > > 8. Add ignite-ml test suites:
> > > >
> > > >1. org.apache.ignite.ml.math.distances.DistancesTestSuite
> > > >2. org.apache.ignite.ml.naivebayes.NaiveBayesTestSuite
> > > >
> > > >
> > > > On Wed, Nov 25, 2020 at 4:26 PM Ilya Kasnacheev <
> > > ilya.kasnach...@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hello!
> > > > >
> > > > > Yes, we have such tests which depend on ignite-indexing or
> > > ignite-spring.
> > > > > They just need to be included in Spring* or Queries* test suite.
> Then
> > > > they
> > > > > will be executed on TC in the correct context. You can also run
> these
> > > > tests
> > > > > from IDEA by specifying other module as classpath. No need to move
> > the
> > > > > classes around.
> > > > >
> > > > > I will check the PR.
> > > > >
> > > > > Regards,
> > > > > --
> > > > > Ilya Kasnacheev
> > > > >
> > > > >
> > > > > ср, 25 нояб. 2020 г. в 00:22, Max Timonin  >:
> > > > >
> > > > > > Ilya, Anton, Ivan, hi!
> > > > > >
> > > > > > I fix some comments you leave in the PR. Also I checked some test
> > > > suites
> > > > > > and found that some tests are written in the core module but
> depend
> > > on
> > > > > the
> > > > > > indexing module (or other modules). Some of such test classes
> > contain
> > > > > tests
> > > > > > that are related to the core functionality, but some to indexing.
> > I'm
> > > > not
> > > > > > sure 

Re: [DISCUSS] Missed (non-suited) tests

[Ilya Kasnacheev]

Hello!

You can override these values (module, suites) values when running a suite
on TC. Can you please run these ones which need to be changed individually
on TC, make sure they run without errors and contain all the needed tests,
and link to these runs in the ticket? Then I can modify the suites to fit
those.

I'm not sure that class shadowing will work as we want it to work, e.g., we
now have two IgniteCacheQuerySelfTestSuite6 with the same FQDN, I'm not
sure if maven/TC is going to pick both or just one.
Maybe they should go to a different package, e.g., testsuites/core for
every suite already present in indexing/spring/etc. Maybe you can rename
them just now? This will mean a lot less of work reconfiguring suites.
In TC configurations, suite names are simple class names, not FQ, so no
changes may be needed at all.

Regards,
-- 
Ilya Kasnacheev


пт, 27 нояб. 2020 г. в 13:03, Max Timonin :

> Hi, sorry for the misleading. I mean "adding ignite-core module *suites* to
> the TeamCity Queries* suite"
>
> On Fri, Nov 27, 2020 at 12:44 PM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com>
> wrote:
>
> > Hello!
> >
> > What do you mean by "adding ignite-core to suite"? ignite-core is a top
> > dependency and its tests are also included in all other modules' tests
> > classpath since it provides GridAbstractTest.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > пт, 27 нояб. 2020 г. в 01:24, Max Timonin :
> >
> > > Hi, Ilya!
> > >
> > > So, I've updated PR, fixed comments and removed Core* prefixes. MTCGA
> > shows
> > > no blockers, but it was 2 weeks ago, so I've started it again.
> > >
> > > If PR is OK then there are some suites that should be updated on TC.
> > Could
> > > you please tell me how we can do it?
> > >
> > > 1. Add ignite-cassandra-serializers suite:
> > >
> > >1. org.apache.ignite.tests.SerializerSuite
> > >
> > > 2. Add ignite-core to Queries* TC suite:
> > >
> > >1. org.apache.ignite.client.IgniteClientTestSuite
> > >2. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite
> > >3. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite2
> > >4. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite3
> > >5. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite4
> > >6. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite5
> > >7. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite6
> > >8. org.apache.ignite.suites.IgnitePdsWithIndexingCoreTestSuite
> > >9. org.apache.ignite.suites.IgniteCacheMvccSqlTestSuite
> > >
> > > 3. Remove ignite-indexing from TC suites:
> > >
> > >1. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite3
> > >2. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite4
> > >3. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite5
> > >
> > > 4. Add ignite-core to Spring* TC suite:
> > >
> > >1. org.apache.ignite.testsuites.IgniteSpringTestSuite
> > >
> > > 5. Add ignite-core suite (depends on uri-deployment module):
> > >
> > >1. org.apache.ignite.testsuites.IgniteUriDeploymentTestSuite
> > >
> > > 6. Add ignite-core suite to Zookeeper TC suite:
> > >
> > >1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3
> > >
> > > 7. Remove ignite-zookeeper test suite:
> > >
> > >1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3
> > >
> > > 8. Add ignite-ml test suites:
> > >
> > >1. org.apache.ignite.ml.math.distances.DistancesTestSuite
> > >2. org.apache.ignite.ml.naivebayes.NaiveBayesTestSuite
> > >
> > >
> > > On Wed, Nov 25, 2020 at 4:26 PM Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hello!
> > > >
> > > > Yes, we have such tests which depend on ignite-indexing or
> > ignite-spring.
> > > > They just need to be included in Spring* or Queries* test suite. Then
> > > they
> > > > will be executed on TC in the correct context. You can also run these
> > > tests
> > > > from IDEA by specifying other module as classpath. No need to move
> the
> > > > classes around.
> > > >
> > > > I will check the PR.
> > > >
> > > > Regards,
> > > > --
> > > > Ilya Kasnacheev
> > > >
> > > >
> > > > ср, 25 нояб. 2020 г. в 00:22, Max Timonin :
> > > >
> > > > > Ilya, Anton, Ivan, hi!
> > > > >
> > > > > I fix some comments you leave in the PR. Also I checked some test
> > > suites
> > > > > and found that some tests are written in the core module but depend
> > on
> > > > the
> > > > > indexing module (or other modules). Some of such test classes
> contain
> > > > tests
> > > > > that are related to the core functionality, but some to indexing.
> I'm
> > > not
> > > > > sure if it is correct to move a whole suite with all tests from the
> > > > > indexing module to the core, as it will hide some core tests from
> the
> > > > core
> > > > > module.
> > > > >
> > > > > I believe that the correct solution is to investigate every such
> test
> > > and
> > > > > move it to the right module. But I think this work will take a lot
> 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi, sorry for the misleading. I mean "adding ignite-core module *suites* to
the TeamCity Queries* suite"

On Fri, Nov 27, 2020 at 12:44 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> What do you mean by "adding ignite-core to suite"? ignite-core is a top
> dependency and its tests are also included in all other modules' tests
> classpath since it provides GridAbstractTest.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 27 нояб. 2020 г. в 01:24, Max Timonin :
>
> > Hi, Ilya!
> >
> > So, I've updated PR, fixed comments and removed Core* prefixes. MTCGA
> shows
> > no blockers, but it was 2 weeks ago, so I've started it again.
> >
> > If PR is OK then there are some suites that should be updated on TC.
> Could
> > you please tell me how we can do it?
> >
> > 1. Add ignite-cassandra-serializers suite:
> >
> >1. org.apache.ignite.tests.SerializerSuite
> >
> > 2. Add ignite-core to Queries* TC suite:
> >
> >1. org.apache.ignite.client.IgniteClientTestSuite
> >2. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite
> >3. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite2
> >4. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite3
> >5. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite4
> >6. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite5
> >7. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite6
> >8. org.apache.ignite.suites.IgnitePdsWithIndexingCoreTestSuite
> >9. org.apache.ignite.suites.IgniteCacheMvccSqlTestSuite
> >
> > 3. Remove ignite-indexing from TC suites:
> >
> >1. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite3
> >2. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite4
> >3. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite5
> >
> > 4. Add ignite-core to Spring* TC suite:
> >
> >1. org.apache.ignite.testsuites.IgniteSpringTestSuite
> >
> > 5. Add ignite-core suite (depends on uri-deployment module):
> >
> >1. org.apache.ignite.testsuites.IgniteUriDeploymentTestSuite
> >
> > 6. Add ignite-core suite to Zookeeper TC suite:
> >
> >1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3
> >
> > 7. Remove ignite-zookeeper test suite:
> >
> >1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3
> >
> > 8. Add ignite-ml test suites:
> >
> >1. org.apache.ignite.ml.math.distances.DistancesTestSuite
> >2. org.apache.ignite.ml.naivebayes.NaiveBayesTestSuite
> >
> >
> > On Wed, Nov 25, 2020 at 4:26 PM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com
> > >
> > wrote:
> >
> > > Hello!
> > >
> > > Yes, we have such tests which depend on ignite-indexing or
> ignite-spring.
> > > They just need to be included in Spring* or Queries* test suite. Then
> > they
> > > will be executed on TC in the correct context. You can also run these
> > tests
> > > from IDEA by specifying other module as classpath. No need to move the
> > > classes around.
> > >
> > > I will check the PR.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > ср, 25 нояб. 2020 г. в 00:22, Max Timonin :
> > >
> > > > Ilya, Anton, Ivan, hi!
> > > >
> > > > I fix some comments you leave in the PR. Also I checked some test
> > suites
> > > > and found that some tests are written in the core module but depend
> on
> > > the
> > > > indexing module (or other modules). Some of such test classes contain
> > > tests
> > > > that are related to the core functionality, but some to indexing. I'm
> > not
> > > > sure if it is correct to move a whole suite with all tests from the
> > > > indexing module to the core, as it will hide some core tests from the
> > > core
> > > > module.
> > > >
> > > > I believe that the correct solution is to investigate every such test
> > and
> > > > move it to the right module. But I think this work will take a lot of
> > > time
> > > > and should be performed in a separate ticket, I will do it in the
> > > > background.
> > > >
> > > > I think currently we should proceed with a way I introduced in PR:
> > > > 1. Create fake suites for all such tests (written in core, suited in
> > > other
> > > > modules: indexing/spring/zookeeper/etc) in the core module. I named
> > such
> > > > suites with prefix Core*.
> > > > 2. Replace tests in modules with links to fake suites.
> > > > 3. Create an umbrella Jira ticket to discover every fake suite and
> > > replace
> > > > it with a new one in the right module.
> > > > 4. Merge this PR for introducing a new travis check to avoid losing
> > > > new tests.
> > > >
> > > > WDYT?
> > > >
> > > > List of such mixed suites:
> > > >
> > > > 1. suite IgniteBinaryCacheQueryTestSuite
> > > >
> > > > test GridCacheQueryIndexingDisabledSelfTest
> > > > test IgniteCacheBinaryObjectsScanSelfTest
> > > > test IgniteCacheBinaryObjectsScanWithEventsSelfTest)
> > > >
> > > >
> > > > 2. suite IgniteCacheQuerySelfTestSuite3
> > > >
> > > > test GridCacheContinuousQueryNodesFilteringTest
> > > >
> > > >
> > > > 3. suite IgniteCacheQuerySelfTestSuite5
> > > 

Re: [DISCUSS] Missed (non-suited) tests

[Ilya Kasnacheev]

Hello!

What do you mean by "adding ignite-core to suite"? ignite-core is a top
dependency and its tests are also included in all other modules' tests
classpath since it provides GridAbstractTest.

Regards,
-- 
Ilya Kasnacheev


пт, 27 нояб. 2020 г. в 01:24, Max Timonin :

> Hi, Ilya!
>
> So, I've updated PR, fixed comments and removed Core* prefixes. MTCGA shows
> no blockers, but it was 2 weeks ago, so I've started it again.
>
> If PR is OK then there are some suites that should be updated on TC. Could
> you please tell me how we can do it?
>
> 1. Add ignite-cassandra-serializers suite:
>
>1. org.apache.ignite.tests.SerializerSuite
>
> 2. Add ignite-core to Queries* TC suite:
>
>1. org.apache.ignite.client.IgniteClientTestSuite
>2. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite
>3. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite2
>4. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite3
>5. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite4
>6. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite5
>7. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite6
>8. org.apache.ignite.suites.IgnitePdsWithIndexingCoreTestSuite
>9. org.apache.ignite.suites.IgniteCacheMvccSqlTestSuite
>
> 3. Remove ignite-indexing from TC suites:
>
>1. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite3
>2. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite4
>3. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite5
>
> 4. Add ignite-core to Spring* TC suite:
>
>1. org.apache.ignite.testsuites.IgniteSpringTestSuite
>
> 5. Add ignite-core suite (depends on uri-deployment module):
>
>1. org.apache.ignite.testsuites.IgniteUriDeploymentTestSuite
>
> 6. Add ignite-core suite to Zookeeper TC suite:
>
>1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3
>
> 7. Remove ignite-zookeeper test suite:
>
>1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3
>
> 8. Add ignite-ml test suites:
>
>1. org.apache.ignite.ml.math.distances.DistancesTestSuite
>2. org.apache.ignite.ml.naivebayes.NaiveBayesTestSuite
>
>
> On Wed, Nov 25, 2020 at 4:26 PM Ilya Kasnacheev  >
> wrote:
>
> > Hello!
> >
> > Yes, we have such tests which depend on ignite-indexing or ignite-spring.
> > They just need to be included in Spring* or Queries* test suite. Then
> they
> > will be executed on TC in the correct context. You can also run these
> tests
> > from IDEA by specifying other module as classpath. No need to move the
> > classes around.
> >
> > I will check the PR.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > ср, 25 нояб. 2020 г. в 00:22, Max Timonin :
> >
> > > Ilya, Anton, Ivan, hi!
> > >
> > > I fix some comments you leave in the PR. Also I checked some test
> suites
> > > and found that some tests are written in the core module but depend on
> > the
> > > indexing module (or other modules). Some of such test classes contain
> > tests
> > > that are related to the core functionality, but some to indexing. I'm
> not
> > > sure if it is correct to move a whole suite with all tests from the
> > > indexing module to the core, as it will hide some core tests from the
> > core
> > > module.
> > >
> > > I believe that the correct solution is to investigate every such test
> and
> > > move it to the right module. But I think this work will take a lot of
> > time
> > > and should be performed in a separate ticket, I will do it in the
> > > background.
> > >
> > > I think currently we should proceed with a way I introduced in PR:
> > > 1. Create fake suites for all such tests (written in core, suited in
> > other
> > > modules: indexing/spring/zookeeper/etc) in the core module. I named
> such
> > > suites with prefix Core*.
> > > 2. Replace tests in modules with links to fake suites.
> > > 3. Create an umbrella Jira ticket to discover every fake suite and
> > replace
> > > it with a new one in the right module.
> > > 4. Merge this PR for introducing a new travis check to avoid losing
> > > new tests.
> > >
> > > WDYT?
> > >
> > > List of such mixed suites:
> > >
> > > 1. suite IgniteBinaryCacheQueryTestSuite
> > >
> > > test GridCacheQueryIndexingDisabledSelfTest
> > > test IgniteCacheBinaryObjectsScanSelfTest
> > > test IgniteCacheBinaryObjectsScanWithEventsSelfTest)
> > >
> > >
> > > 2. suite IgniteCacheQuerySelfTestSuite3
> > >
> > > test GridCacheContinuousQueryNodesFilteringTest
> > >
> > >
> > > 3. suite IgniteCacheQuerySelfTestSuite5
> > >
> > > test ContinuousQueryRemoteFilterMissingInClassPathSelfTest
> > >
> > > 4. suite IgniteCacheQuerySelfTestSuite6
> > >
> > > test CacheContinuousQueryOperationP2PTest
> > >
> > > test CacheContinuousQueryFilterDeploymentFailedTest
> > >
> > >
> > > 5. all tests in suite IgnitePdsWithIndexingCoreTestSuite
> > >
> > >
> > > 6. and some others.
> > >
> > > On Wed, Nov 18, 2020 at 12:38 PM Max Timonin 
> > > wrote:
> > >
> > > > Hi Ilya! Thank you for up the topic. I will 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi, Ilya!

So, I've updated PR, fixed comments and removed Core* prefixes. MTCGA shows
no blockers, but it was 2 weeks ago, so I've started it again.

If PR is OK then there are some suites that should be updated on TC. Could
you please tell me how we can do it?

1. Add ignite-cassandra-serializers suite:

   1. org.apache.ignite.tests.SerializerSuite

2. Add ignite-core to Queries* TC suite:

   1. org.apache.ignite.client.IgniteClientTestSuite
   2. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite
   3. org.apache.ignite.suites.IgniteBinaryCacheQueryTestSuite2
   4. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite3
   5. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite4
   6. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite5
   7. org.apache.ignite.suites.IgniteCacheQuerySelfTestSuite6
   8. org.apache.ignite.suites.IgnitePdsWithIndexingCoreTestSuite
   9. org.apache.ignite.suites.IgniteCacheMvccSqlTestSuite

3. Remove ignite-indexing from TC suites:

   1. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite3
   2. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite4
   3. org.apache.ignite.testsuites.IgniteCacheQuerySelfTestSuite5

4. Add ignite-core to Spring* TC suite:

   1. org.apache.ignite.testsuites.IgniteSpringTestSuite

5. Add ignite-core suite (depends on uri-deployment module):

   1. org.apache.ignite.testsuites.IgniteUriDeploymentTestSuite

6. Add ignite-core suite to Zookeeper TC suite:

   1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3

7. Remove ignite-zookeeper test suite:

   1. org.apache.ignite.testsuites.ZookeeperDiscoverySpiTestSuite3

8. Add ignite-ml test suites:

   1. org.apache.ignite.ml.math.distances.DistancesTestSuite
   2. org.apache.ignite.ml.naivebayes.NaiveBayesTestSuite


On Wed, Nov 25, 2020 at 4:26 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> Yes, we have such tests which depend on ignite-indexing or ignite-spring.
> They just need to be included in Spring* or Queries* test suite. Then they
> will be executed on TC in the correct context. You can also run these tests
> from IDEA by specifying other module as classpath. No need to move the
> classes around.
>
> I will check the PR.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 25 нояб. 2020 г. в 00:22, Max Timonin :
>
> > Ilya, Anton, Ivan, hi!
> >
> > I fix some comments you leave in the PR. Also I checked some test suites
> > and found that some tests are written in the core module but depend on
> the
> > indexing module (or other modules). Some of such test classes contain
> tests
> > that are related to the core functionality, but some to indexing. I'm not
> > sure if it is correct to move a whole suite with all tests from the
> > indexing module to the core, as it will hide some core tests from the
> core
> > module.
> >
> > I believe that the correct solution is to investigate every such test and
> > move it to the right module. But I think this work will take a lot of
> time
> > and should be performed in a separate ticket, I will do it in the
> > background.
> >
> > I think currently we should proceed with a way I introduced in PR:
> > 1. Create fake suites for all such tests (written in core, suited in
> other
> > modules: indexing/spring/zookeeper/etc) in the core module. I named such
> > suites with prefix Core*.
> > 2. Replace tests in modules with links to fake suites.
> > 3. Create an umbrella Jira ticket to discover every fake suite and
> replace
> > it with a new one in the right module.
> > 4. Merge this PR for introducing a new travis check to avoid losing
> > new tests.
> >
> > WDYT?
> >
> > List of such mixed suites:
> >
> > 1. suite IgniteBinaryCacheQueryTestSuite
> >
> > test GridCacheQueryIndexingDisabledSelfTest
> > test IgniteCacheBinaryObjectsScanSelfTest
> > test IgniteCacheBinaryObjectsScanWithEventsSelfTest)
> >
> >
> > 2. suite IgniteCacheQuerySelfTestSuite3
> >
> > test GridCacheContinuousQueryNodesFilteringTest
> >
> >
> > 3. suite IgniteCacheQuerySelfTestSuite5
> >
> > test ContinuousQueryRemoteFilterMissingInClassPathSelfTest
> >
> > 4. suite IgniteCacheQuerySelfTestSuite6
> >
> > test CacheContinuousQueryOperationP2PTest
> >
> > test CacheContinuousQueryFilterDeploymentFailedTest
> >
> >
> > 5. all tests in suite IgnitePdsWithIndexingCoreTestSuite
> >
> >
> > 6. and some others.
> >
> > On Wed, Nov 18, 2020 at 12:38 PM Max Timonin 
> > wrote:
> >
> > > Hi Ilya! Thank you for up the topic. I will come back with fixes and
> > > comments in a couple of days.
> > >
> > > On Tue, Nov 17, 2020 at 4:26 PM Ilya Kasnacheev <
> > ilya.kasnach...@gmail.com>
> > > wrote:
> > >
> > >> Hello!
> > >>
> > >> I have left some comments and there's also more discussion there. Can
> > you
> > >> please look?
> > >>
> > >> Thanks,
> > >> --
> > >> Ilya Kasnacheev
> > >>
> > >>
> > >> вт, 3 нояб. 2020 г. в 00:03, Max Timonin :
> > >>
> > >> > Hi!
> > >> >
> > >> > I've updated PR: https://github.com/apache/ignite/pull/8367. Anton,
> > >> Ivan,
> > >> > 

Re: [DISCUSS] Missed (non-suited) tests

[Ilya Kasnacheev]

Hello!

Yes, we have such tests which depend on ignite-indexing or ignite-spring.
They just need to be included in Spring* or Queries* test suite. Then they
will be executed on TC in the correct context. You can also run these tests
from IDEA by specifying other module as classpath. No need to move the
classes around.

I will check the PR.

Regards,
-- 
Ilya Kasnacheev


ср, 25 нояб. 2020 г. в 00:22, Max Timonin :

> Ilya, Anton, Ivan, hi!
>
> I fix some comments you leave in the PR. Also I checked some test suites
> and found that some tests are written in the core module but depend on the
> indexing module (or other modules). Some of such test classes contain tests
> that are related to the core functionality, but some to indexing. I'm not
> sure if it is correct to move a whole suite with all tests from the
> indexing module to the core, as it will hide some core tests from the core
> module.
>
> I believe that the correct solution is to investigate every such test and
> move it to the right module. But I think this work will take a lot of time
> and should be performed in a separate ticket, I will do it in the
> background.
>
> I think currently we should proceed with a way I introduced in PR:
> 1. Create fake suites for all such tests (written in core, suited in other
> modules: indexing/spring/zookeeper/etc) in the core module. I named such
> suites with prefix Core*.
> 2. Replace tests in modules with links to fake suites.
> 3. Create an umbrella Jira ticket to discover every fake suite and replace
> it with a new one in the right module.
> 4. Merge this PR for introducing a new travis check to avoid losing
> new tests.
>
> WDYT?
>
> List of such mixed suites:
>
> 1. suite IgniteBinaryCacheQueryTestSuite
>
> test GridCacheQueryIndexingDisabledSelfTest
> test IgniteCacheBinaryObjectsScanSelfTest
> test IgniteCacheBinaryObjectsScanWithEventsSelfTest)
>
>
> 2. suite IgniteCacheQuerySelfTestSuite3
>
> test GridCacheContinuousQueryNodesFilteringTest
>
>
> 3. suite IgniteCacheQuerySelfTestSuite5
>
> test ContinuousQueryRemoteFilterMissingInClassPathSelfTest
>
> 4. suite IgniteCacheQuerySelfTestSuite6
>
> test CacheContinuousQueryOperationP2PTest
>
> test CacheContinuousQueryFilterDeploymentFailedTest
>
>
> 5. all tests in suite IgnitePdsWithIndexingCoreTestSuite
>
>
> 6. and some others.
>
> On Wed, Nov 18, 2020 at 12:38 PM Max Timonin 
> wrote:
>
> > Hi Ilya! Thank you for up the topic. I will come back with fixes and
> > comments in a couple of days.
> >
> > On Tue, Nov 17, 2020 at 4:26 PM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com>
> > wrote:
> >
> >> Hello!
> >>
> >> I have left some comments and there's also more discussion there. Can
> you
> >> please look?
> >>
> >> Thanks,
> >> --
> >> Ilya Kasnacheev
> >>
> >>
> >> вт, 3 нояб. 2020 г. в 00:03, Max Timonin :
> >>
> >> > Hi!
> >> >
> >> > I've updated PR: https://github.com/apache/ignite/pull/8367. Anton,
> >> Ivan,
> >> > Ivan could you please review it?
> >> >
> >> > Some moments to mention:
> >> > 1. I've added new suites: SerializerSuite
> >> (ignite-cassandra-serializers),
> >> > DistanceTestSuite, NaiveBayesTestSuite (ignite-ml). Should we
> configure
> >> a
> >> > TeamCity to run them?
> >> >
> >> > 2. Some tests marked as failed, I'll create corresponding tickets for
> >> them
> >> > after PR approved:
> >> > - IgnitePKIndexesMigrationToUnwrapPkTest
> >> > - P2PGridifySelfTest
> >> > - GridCacheMultithreadedFailoverAbstractTest
> >> > - WalCompactionAfterRestartTest
> >> > - GridTcpCommunicationSpiLogTest
> >> > - ComplexSecondaryKeyUnwrapSelfTest
> >> > - SqlTransactionsSelfTest
> >> >
> >> > 3. Add docs to DEVNOTES.txt
> >> >
> >> > On Mon, Nov 2, 2020 at 11:44 AM Anton Vinogradov 
> wrote:
> >> >
> >> > > > As I understand we
> >> > > > can't just move suites between modules, as TeamCity may depend on
> >> the
> >> > > path
> >> > > > to them.
> >> > > See no problem to update TC as well.
> >> > >
> >> > > On Fri, Oct 30, 2020 at 4:32 PM Ivan Daschinsky <
> ivanda...@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > I suggests to mark these tests with @Ignore and file tickets to
> fix
> >> > them.
> >> > > >
> >> > > > пт, 30 окт. 2020 г. в 16:26, Ivan Daschinsky  >:
> >> > > >
> >> > > > > Hi
> >> > > > >
> >> > > > > WalCompactionAfterRestartTest -- yes we need it. This test
> failed
> >> > > because
> >> > > > > of race (test shold be rewritten a little bit)
> >> > > > >
> >> > > > > пт, 30 окт. 2020 г. в 16:15, Max Timonin <
> timonin.ma...@gmail.com
> >> >:
> >> > > > >
> >> > > > >> Hi!
> >> > > > >>
> >> > > > >> Yes, you're correct. I've developed the check and started to
> >> clean
> >> > > tests
> >> > > > >> (move them to suites, mark some tests with Ignore, etc.). I
> >> finish
> >> > > work
> >> > > > on
> >> > > > >> the core module. I hope it was the biggest one, and others are
> >> less.
> >> > > If
> >> > > > >> so,
> >> > > > >> I think I will finish the work on other modules in 1 or 2
> weeks,
> >> as
> >> > I
> >> > 

Re: [DISCUSS] Missed (non-suited) tests

[Anton Vinogradov]

>> WDYT?
LGTM

On Wed, Nov 25, 2020 at 12:23 AM Max Timonin 
wrote:

> Ilya, Anton, Ivan, hi!
>
> I fix some comments you leave in the PR. Also I checked some test suites
> and found that some tests are written in the core module but depend on the
> indexing module (or other modules). Some of such test classes contain tests
> that are related to the core functionality, but some to indexing. I'm not
> sure if it is correct to move a whole suite with all tests from the
> indexing module to the core, as it will hide some core tests from the core
> module.
>
> I believe that the correct solution is to investigate every such test and
> move it to the right module. But I think this work will take a lot of time
> and should be performed in a separate ticket, I will do it in the
> background.
>
> I think currently we should proceed with a way I introduced in PR:
> 1. Create fake suites for all such tests (written in core, suited in other
> modules: indexing/spring/zookeeper/etc) in the core module. I named such
> suites with prefix Core*.
> 2. Replace tests in modules with links to fake suites.
> 3. Create an umbrella Jira ticket to discover every fake suite and replace
> it with a new one in the right module.
> 4. Merge this PR for introducing a new travis check to avoid losing
> new tests.
>
> WDYT?
>
> List of such mixed suites:
>
> 1. suite IgniteBinaryCacheQueryTestSuite
>
> test GridCacheQueryIndexingDisabledSelfTest
> test IgniteCacheBinaryObjectsScanSelfTest
> test IgniteCacheBinaryObjectsScanWithEventsSelfTest)
>
>
> 2. suite IgniteCacheQuerySelfTestSuite3
>
> test GridCacheContinuousQueryNodesFilteringTest
>
>
> 3. suite IgniteCacheQuerySelfTestSuite5
>
> test ContinuousQueryRemoteFilterMissingInClassPathSelfTest
>
> 4. suite IgniteCacheQuerySelfTestSuite6
>
> test CacheContinuousQueryOperationP2PTest
>
> test CacheContinuousQueryFilterDeploymentFailedTest
>
>
> 5. all tests in suite IgnitePdsWithIndexingCoreTestSuite
>
>
> 6. and some others.
>
> On Wed, Nov 18, 2020 at 12:38 PM Max Timonin 
> wrote:
>
> > Hi Ilya! Thank you for up the topic. I will come back with fixes and
> > comments in a couple of days.
> >
> > On Tue, Nov 17, 2020 at 4:26 PM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com>
> > wrote:
> >
> >> Hello!
> >>
> >> I have left some comments and there's also more discussion there. Can
> you
> >> please look?
> >>
> >> Thanks,
> >> --
> >> Ilya Kasnacheev
> >>
> >>
> >> вт, 3 нояб. 2020 г. в 00:03, Max Timonin :
> >>
> >> > Hi!
> >> >
> >> > I've updated PR: https://github.com/apache/ignite/pull/8367. Anton,
> >> Ivan,
> >> > Ivan could you please review it?
> >> >
> >> > Some moments to mention:
> >> > 1. I've added new suites: SerializerSuite
> >> (ignite-cassandra-serializers),
> >> > DistanceTestSuite, NaiveBayesTestSuite (ignite-ml). Should we
> configure
> >> a
> >> > TeamCity to run them?
> >> >
> >> > 2. Some tests marked as failed, I'll create corresponding tickets for
> >> them
> >> > after PR approved:
> >> > - IgnitePKIndexesMigrationToUnwrapPkTest
> >> > - P2PGridifySelfTest
> >> > - GridCacheMultithreadedFailoverAbstractTest
> >> > - WalCompactionAfterRestartTest
> >> > - GridTcpCommunicationSpiLogTest
> >> > - ComplexSecondaryKeyUnwrapSelfTest
> >> > - SqlTransactionsSelfTest
> >> >
> >> > 3. Add docs to DEVNOTES.txt
> >> >
> >> > On Mon, Nov 2, 2020 at 11:44 AM Anton Vinogradov 
> wrote:
> >> >
> >> > > > As I understand we
> >> > > > can't just move suites between modules, as TeamCity may depend on
> >> the
> >> > > path
> >> > > > to them.
> >> > > See no problem to update TC as well.
> >> > >
> >> > > On Fri, Oct 30, 2020 at 4:32 PM Ivan Daschinsky <
> ivanda...@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > I suggests to mark these tests with @Ignore and file tickets to
> fix
> >> > them.
> >> > > >
> >> > > > пт, 30 окт. 2020 г. в 16:26, Ivan Daschinsky  >:
> >> > > >
> >> > > > > Hi
> >> > > > >
> >> > > > > WalCompactionAfterRestartTest -- yes we need it. This test
> failed
> >> > > because
> >> > > > > of race (test shold be rewritten a little bit)
> >> > > > >
> >> > > > > пт, 30 окт. 2020 г. в 16:15, Max Timonin <
> timonin.ma...@gmail.com
> >> >:
> >> > > > >
> >> > > > >> Hi!
> >> > > > >>
> >> > > > >> Yes, you're correct. I've developed the check and started to
> >> clean
> >> > > tests
> >> > > > >> (move them to suites, mark some tests with Ignore, etc.). I
> >> finish
> >> > > work
> >> > > > on
> >> > > > >> the core module. I hope it was the biggest one, and others are
> >> less.
> >> > > If
> >> > > > >> so,
> >> > > > >> I think I will finish the work on other modules in 1 or 2
> weeks,
> >> as
> >> > I
> >> > > do
> >> > > > >> this activity in the background (~10% of my work time).
> Actually
> >> > I've
> >> > > > >> found
> >> > > > >> 3 failed tests in the core module that aren't in any suite, so
> I
> >> > need
> >> > > > time
> >> > > > >> to discover reason of failures and if we actually need those
> >> tests:
> >> > > > >>
> >> > > > >> 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Ilya, Anton, Ivan, hi!

I fix some comments you leave in the PR. Also I checked some test suites
and found that some tests are written in the core module but depend on the
indexing module (or other modules). Some of such test classes contain tests
that are related to the core functionality, but some to indexing. I'm not
sure if it is correct to move a whole suite with all tests from the
indexing module to the core, as it will hide some core tests from the core
module.

I believe that the correct solution is to investigate every such test and
move it to the right module. But I think this work will take a lot of time
and should be performed in a separate ticket, I will do it in the
background.

I think currently we should proceed with a way I introduced in PR:
1. Create fake suites for all such tests (written in core, suited in other
modules: indexing/spring/zookeeper/etc) in the core module. I named such
suites with prefix Core*.
2. Replace tests in modules with links to fake suites.
3. Create an umbrella Jira ticket to discover every fake suite and replace
it with a new one in the right module.
4. Merge this PR for introducing a new travis check to avoid losing
new tests.

WDYT?

List of such mixed suites:

1. suite IgniteBinaryCacheQueryTestSuite

test GridCacheQueryIndexingDisabledSelfTest
test IgniteCacheBinaryObjectsScanSelfTest
test IgniteCacheBinaryObjectsScanWithEventsSelfTest)


2. suite IgniteCacheQuerySelfTestSuite3

test GridCacheContinuousQueryNodesFilteringTest


3. suite IgniteCacheQuerySelfTestSuite5

test ContinuousQueryRemoteFilterMissingInClassPathSelfTest

4. suite IgniteCacheQuerySelfTestSuite6

test CacheContinuousQueryOperationP2PTest

test CacheContinuousQueryFilterDeploymentFailedTest


5. all tests in suite IgnitePdsWithIndexingCoreTestSuite


6. and some others.

On Wed, Nov 18, 2020 at 12:38 PM Max Timonin 
wrote:

> Hi Ilya! Thank you for up the topic. I will come back with fixes and
> comments in a couple of days.
>
> On Tue, Nov 17, 2020 at 4:26 PM Ilya Kasnacheev 
> wrote:
>
>> Hello!
>>
>> I have left some comments and there's also more discussion there. Can you
>> please look?
>>
>> Thanks,
>> --
>> Ilya Kasnacheev
>>
>>
>> вт, 3 нояб. 2020 г. в 00:03, Max Timonin :
>>
>> > Hi!
>> >
>> > I've updated PR: https://github.com/apache/ignite/pull/8367. Anton,
>> Ivan,
>> > Ivan could you please review it?
>> >
>> > Some moments to mention:
>> > 1. I've added new suites: SerializerSuite
>> (ignite-cassandra-serializers),
>> > DistanceTestSuite, NaiveBayesTestSuite (ignite-ml). Should we configure
>> a
>> > TeamCity to run them?
>> >
>> > 2. Some tests marked as failed, I'll create corresponding tickets for
>> them
>> > after PR approved:
>> > - IgnitePKIndexesMigrationToUnwrapPkTest
>> > - P2PGridifySelfTest
>> > - GridCacheMultithreadedFailoverAbstractTest
>> > - WalCompactionAfterRestartTest
>> > - GridTcpCommunicationSpiLogTest
>> > - ComplexSecondaryKeyUnwrapSelfTest
>> > - SqlTransactionsSelfTest
>> >
>> > 3. Add docs to DEVNOTES.txt
>> >
>> > On Mon, Nov 2, 2020 at 11:44 AM Anton Vinogradov  wrote:
>> >
>> > > > As I understand we
>> > > > can't just move suites between modules, as TeamCity may depend on
>> the
>> > > path
>> > > > to them.
>> > > See no problem to update TC as well.
>> > >
>> > > On Fri, Oct 30, 2020 at 4:32 PM Ivan Daschinsky 
>> > > wrote:
>> > >
>> > > > I suggests to mark these tests with @Ignore and file tickets to fix
>> > them.
>> > > >
>> > > > пт, 30 окт. 2020 г. в 16:26, Ivan Daschinsky :
>> > > >
>> > > > > Hi
>> > > > >
>> > > > > WalCompactionAfterRestartTest -- yes we need it. This test failed
>> > > because
>> > > > > of race (test shold be rewritten a little bit)
>> > > > >
>> > > > > пт, 30 окт. 2020 г. в 16:15, Max Timonin > >:
>> > > > >
>> > > > >> Hi!
>> > > > >>
>> > > > >> Yes, you're correct. I've developed the check and started to
>> clean
>> > > tests
>> > > > >> (move them to suites, mark some tests with Ignore, etc.). I
>> finish
>> > > work
>> > > > on
>> > > > >> the core module. I hope it was the biggest one, and others are
>> less.
>> > > If
>> > > > >> so,
>> > > > >> I think I will finish the work on other modules in 1 or 2 weeks,
>> as
>> > I
>> > > do
>> > > > >> this activity in the background (~10% of my work time). Actually
>> > I've
>> > > > >> found
>> > > > >> 3 failed tests in the core module that aren't in any suite, so I
>> > need
>> > > > time
>> > > > >> to discover reason of failures and if we actually need those
>> tests:
>> > > > >>
>> > > > >> GridCacheMultithreadedFailoverAbstractTest
>> > > > >> WalCompactionAfterRestartTest
>> > > > >> GridTcpCommunicationSpiLogTest
>> > > > >>
>> > > > >> Also we should decide how to be with wrongly located es. As I
>> > > understand
>> > > > >> we
>> > > > >> can't just move suites between modules, as TeamCity may depend on
>> > the
>> > > > path
>> > > > >> to them. So, for such cases I've just created suites in the right
>> > > > module,
>> > > > >> and replaced 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi Ilya! Thank you for up the topic. I will come back with fixes and
comments in a couple of days.

On Tue, Nov 17, 2020 at 4:26 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> I have left some comments and there's also more discussion there. Can you
> please look?
>
> Thanks,
> --
> Ilya Kasnacheev
>
>
> вт, 3 нояб. 2020 г. в 00:03, Max Timonin :
>
> > Hi!
> >
> > I've updated PR: https://github.com/apache/ignite/pull/8367. Anton,
> Ivan,
> > Ivan could you please review it?
> >
> > Some moments to mention:
> > 1. I've added new suites: SerializerSuite (ignite-cassandra-serializers),
> > DistanceTestSuite, NaiveBayesTestSuite (ignite-ml). Should we configure a
> > TeamCity to run them?
> >
> > 2. Some tests marked as failed, I'll create corresponding tickets for
> them
> > after PR approved:
> > - IgnitePKIndexesMigrationToUnwrapPkTest
> > - P2PGridifySelfTest
> > - GridCacheMultithreadedFailoverAbstractTest
> > - WalCompactionAfterRestartTest
> > - GridTcpCommunicationSpiLogTest
> > - ComplexSecondaryKeyUnwrapSelfTest
> > - SqlTransactionsSelfTest
> >
> > 3. Add docs to DEVNOTES.txt
> >
> > On Mon, Nov 2, 2020 at 11:44 AM Anton Vinogradov  wrote:
> >
> > > > As I understand we
> > > > can't just move suites between modules, as TeamCity may depend on the
> > > path
> > > > to them.
> > > See no problem to update TC as well.
> > >
> > > On Fri, Oct 30, 2020 at 4:32 PM Ivan Daschinsky 
> > > wrote:
> > >
> > > > I suggests to mark these tests with @Ignore and file tickets to fix
> > them.
> > > >
> > > > пт, 30 окт. 2020 г. в 16:26, Ivan Daschinsky :
> > > >
> > > > > Hi
> > > > >
> > > > > WalCompactionAfterRestartTest -- yes we need it. This test failed
> > > because
> > > > > of race (test shold be rewritten a little bit)
> > > > >
> > > > > пт, 30 окт. 2020 г. в 16:15, Max Timonin  >:
> > > > >
> > > > >> Hi!
> > > > >>
> > > > >> Yes, you're correct. I've developed the check and started to clean
> > > tests
> > > > >> (move them to suites, mark some tests with Ignore, etc.). I finish
> > > work
> > > > on
> > > > >> the core module. I hope it was the biggest one, and others are
> less.
> > > If
> > > > >> so,
> > > > >> I think I will finish the work on other modules in 1 or 2 weeks,
> as
> > I
> > > do
> > > > >> this activity in the background (~10% of my work time). Actually
> > I've
> > > > >> found
> > > > >> 3 failed tests in the core module that aren't in any suite, so I
> > need
> > > > time
> > > > >> to discover reason of failures and if we actually need those
> tests:
> > > > >>
> > > > >> GridCacheMultithreadedFailoverAbstractTest
> > > > >> WalCompactionAfterRestartTest
> > > > >> GridTcpCommunicationSpiLogTest
> > > > >>
> > > > >> Also we should decide how to be with wrongly located es. As I
> > > understand
> > > > >> we
> > > > >> can't just move suites between modules, as TeamCity may depend on
> > the
> > > > path
> > > > >> to them. So, for such cases I've just created suites in the right
> > > > module,
> > > > >> and replaced the test list with the new class suite. It does not
> > look
> > > > >> pretty enough, but I think It's a path of least resistance. WDYT?
> > > > >>
> > > > >> BEFORE:
> > > > >> Module A -> SuiteA -> testA1, testA2, testB1, testB2
> > > > >> Module B -> testB1, testB2
> > > > >>
> > > > >> AFTER:
> > > > >> Module A -> SuiteA, SuiteB
> > > > >> Module B -> SuiteB -> testB1, testB2
> > > > >>
> > > > >> On Fri, Oct 30, 2020 at 3:38 PM Anton Vinogradov 
> > > wrote:
> > > > >>
> > > > >> > Folks,
> > > > >> > What's the current state of this thread?
> > > > >> > AFAIU, we found unused and wrongly located tests and developed
> > some
> > > > >> > checker, could we split this to some PRs?
> > > > >> > Let's merge tests usage fix and location fixes first, this will
> > > > provide
> > > > >> us
> > > > >> > an ability to automate check using Travis.
> > > > >> >
> > > > >> > On Tue, Oct 20, 2020 at 12:06 PM Ivan Pavlukhin <
> > > vololo...@gmail.com>
> > > > >> > wrote:
> > > > >> >
> > > > >> > > Max, Ivan,
> > > > >> > >
> > > > >> > > Using explicit @Ignore and the automated check sounds good to
> > me.
> > > If
> > > > >> > > nobody has arguments against it I think we should do it.
> > > > >> > >
> > > > >> > > 2020-10-19 19:30 GMT+03:00, Max Timonin <
> > timonin.ma...@gmail.com
> > > >:
> > > > >> > > > Hi Ivan,
> > > > >> > > >
> > > > >> > > > I've checked the ticket you provide. It contains subtasks to
> > > > >> uncomment
> > > > >> > or
> > > > >> > > > to remove some unused tests. It definitely describes some
> > cases
> > > > I've
> > > > >> > > found.
> > > > >> > > > So what do you think if I uncomment them in suites, add
> > @Ignore
> > > > >> > > annotation
> > > > >> > > > for those tests while the tickets are open? This will help
> to
> > > find
> > > > >> out
> > > > >> > > > tests that were forgiven in a recent time.
> > > > >> > > >
> > > > >> > > > Also I believe that this check must be automated. I didn't
> > find
> > > a
> > > > >> way
> > > > 

Re: [DISCUSS] Missed (non-suited) tests

[Ilya Kasnacheev]

Hello!

I have left some comments and there's also more discussion there. Can you
please look?

Thanks,
-- 
Ilya Kasnacheev


вт, 3 нояб. 2020 г. в 00:03, Max Timonin :

> Hi!
>
> I've updated PR: https://github.com/apache/ignite/pull/8367. Anton, Ivan,
> Ivan could you please review it?
>
> Some moments to mention:
> 1. I've added new suites: SerializerSuite (ignite-cassandra-serializers),
> DistanceTestSuite, NaiveBayesTestSuite (ignite-ml). Should we configure a
> TeamCity to run them?
>
> 2. Some tests marked as failed, I'll create corresponding tickets for them
> after PR approved:
> - IgnitePKIndexesMigrationToUnwrapPkTest
> - P2PGridifySelfTest
> - GridCacheMultithreadedFailoverAbstractTest
> - WalCompactionAfterRestartTest
> - GridTcpCommunicationSpiLogTest
> - ComplexSecondaryKeyUnwrapSelfTest
> - SqlTransactionsSelfTest
>
> 3. Add docs to DEVNOTES.txt
>
> On Mon, Nov 2, 2020 at 11:44 AM Anton Vinogradov  wrote:
>
> > > As I understand we
> > > can't just move suites between modules, as TeamCity may depend on the
> > path
> > > to them.
> > See no problem to update TC as well.
> >
> > On Fri, Oct 30, 2020 at 4:32 PM Ivan Daschinsky 
> > wrote:
> >
> > > I suggests to mark these tests with @Ignore and file tickets to fix
> them.
> > >
> > > пт, 30 окт. 2020 г. в 16:26, Ivan Daschinsky :
> > >
> > > > Hi
> > > >
> > > > WalCompactionAfterRestartTest -- yes we need it. This test failed
> > because
> > > > of race (test shold be rewritten a little bit)
> > > >
> > > > пт, 30 окт. 2020 г. в 16:15, Max Timonin :
> > > >
> > > >> Hi!
> > > >>
> > > >> Yes, you're correct. I've developed the check and started to clean
> > tests
> > > >> (move them to suites, mark some tests with Ignore, etc.). I finish
> > work
> > > on
> > > >> the core module. I hope it was the biggest one, and others are less.
> > If
> > > >> so,
> > > >> I think I will finish the work on other modules in 1 or 2 weeks, as
> I
> > do
> > > >> this activity in the background (~10% of my work time). Actually
> I've
> > > >> found
> > > >> 3 failed tests in the core module that aren't in any suite, so I
> need
> > > time
> > > >> to discover reason of failures and if we actually need those tests:
> > > >>
> > > >> GridCacheMultithreadedFailoverAbstractTest
> > > >> WalCompactionAfterRestartTest
> > > >> GridTcpCommunicationSpiLogTest
> > > >>
> > > >> Also we should decide how to be with wrongly located es. As I
> > understand
> > > >> we
> > > >> can't just move suites between modules, as TeamCity may depend on
> the
> > > path
> > > >> to them. So, for such cases I've just created suites in the right
> > > module,
> > > >> and replaced the test list with the new class suite. It does not
> look
> > > >> pretty enough, but I think It's a path of least resistance. WDYT?
> > > >>
> > > >> BEFORE:
> > > >> Module A -> SuiteA -> testA1, testA2, testB1, testB2
> > > >> Module B -> testB1, testB2
> > > >>
> > > >> AFTER:
> > > >> Module A -> SuiteA, SuiteB
> > > >> Module B -> SuiteB -> testB1, testB2
> > > >>
> > > >> On Fri, Oct 30, 2020 at 3:38 PM Anton Vinogradov 
> > wrote:
> > > >>
> > > >> > Folks,
> > > >> > What's the current state of this thread?
> > > >> > AFAIU, we found unused and wrongly located tests and developed
> some
> > > >> > checker, could we split this to some PRs?
> > > >> > Let's merge tests usage fix and location fixes first, this will
> > > provide
> > > >> us
> > > >> > an ability to automate check using Travis.
> > > >> >
> > > >> > On Tue, Oct 20, 2020 at 12:06 PM Ivan Pavlukhin <
> > vololo...@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> > > Max, Ivan,
> > > >> > >
> > > >> > > Using explicit @Ignore and the automated check sounds good to
> me.
> > If
> > > >> > > nobody has arguments against it I think we should do it.
> > > >> > >
> > > >> > > 2020-10-19 19:30 GMT+03:00, Max Timonin <
> timonin.ma...@gmail.com
> > >:
> > > >> > > > Hi Ivan,
> > > >> > > >
> > > >> > > > I've checked the ticket you provide. It contains subtasks to
> > > >> uncomment
> > > >> > or
> > > >> > > > to remove some unused tests. It definitely describes some
> cases
> > > I've
> > > >> > > found.
> > > >> > > > So what do you think if I uncomment them in suites, add
> @Ignore
> > > >> > > annotation
> > > >> > > > for those tests while the tickets are open? This will help to
> > find
> > > >> out
> > > >> > > > tests that were forgiven in a recent time.
> > > >> > > >
> > > >> > > > Also I believe that this check must be automated. I didn't
> find
> > a
> > > >> way
> > > >> > how
> > > >> > > > uncomment / unused tests are found in the ticket. If there is
> no
> > > >> any -
> > > >> > I
> > > >> > > > propose mine PR for this purpose.
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > > On Mon, Oct 19, 2020 at 5:24 PM Ivan Daschinsky <
> > > >> ivanda...@gmail.com>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > >> Ivan, as far as I understand, Max also created verification
> > check
> > > >> for
> > > >> > > not
> > > >> 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi!

I've updated PR: https://github.com/apache/ignite/pull/8367. Anton, Ivan,
Ivan could you please review it?

Some moments to mention:
1. I've added new suites: SerializerSuite (ignite-cassandra-serializers),
DistanceTestSuite, NaiveBayesTestSuite (ignite-ml). Should we configure a
TeamCity to run them?

2. Some tests marked as failed, I'll create corresponding tickets for them
after PR approved:
- IgnitePKIndexesMigrationToUnwrapPkTest
- P2PGridifySelfTest
- GridCacheMultithreadedFailoverAbstractTest
- WalCompactionAfterRestartTest
- GridTcpCommunicationSpiLogTest
- ComplexSecondaryKeyUnwrapSelfTest
- SqlTransactionsSelfTest

3. Add docs to DEVNOTES.txt

On Mon, Nov 2, 2020 at 11:44 AM Anton Vinogradov  wrote:

> > As I understand we
> > can't just move suites between modules, as TeamCity may depend on the
> path
> > to them.
> See no problem to update TC as well.
>
> On Fri, Oct 30, 2020 at 4:32 PM Ivan Daschinsky 
> wrote:
>
> > I suggests to mark these tests with @Ignore and file tickets to fix them.
> >
> > пт, 30 окт. 2020 г. в 16:26, Ivan Daschinsky :
> >
> > > Hi
> > >
> > > WalCompactionAfterRestartTest -- yes we need it. This test failed
> because
> > > of race (test shold be rewritten a little bit)
> > >
> > > пт, 30 окт. 2020 г. в 16:15, Max Timonin :
> > >
> > >> Hi!
> > >>
> > >> Yes, you're correct. I've developed the check and started to clean
> tests
> > >> (move them to suites, mark some tests with Ignore, etc.). I finish
> work
> > on
> > >> the core module. I hope it was the biggest one, and others are less.
> If
> > >> so,
> > >> I think I will finish the work on other modules in 1 or 2 weeks, as I
> do
> > >> this activity in the background (~10% of my work time). Actually I've
> > >> found
> > >> 3 failed tests in the core module that aren't in any suite, so I need
> > time
> > >> to discover reason of failures and if we actually need those tests:
> > >>
> > >> GridCacheMultithreadedFailoverAbstractTest
> > >> WalCompactionAfterRestartTest
> > >> GridTcpCommunicationSpiLogTest
> > >>
> > >> Also we should decide how to be with wrongly located es. As I
> understand
> > >> we
> > >> can't just move suites between modules, as TeamCity may depend on the
> > path
> > >> to them. So, for such cases I've just created suites in the right
> > module,
> > >> and replaced the test list with the new class suite. It does not look
> > >> pretty enough, but I think It's a path of least resistance. WDYT?
> > >>
> > >> BEFORE:
> > >> Module A -> SuiteA -> testA1, testA2, testB1, testB2
> > >> Module B -> testB1, testB2
> > >>
> > >> AFTER:
> > >> Module A -> SuiteA, SuiteB
> > >> Module B -> SuiteB -> testB1, testB2
> > >>
> > >> On Fri, Oct 30, 2020 at 3:38 PM Anton Vinogradov 
> wrote:
> > >>
> > >> > Folks,
> > >> > What's the current state of this thread?
> > >> > AFAIU, we found unused and wrongly located tests and developed some
> > >> > checker, could we split this to some PRs?
> > >> > Let's merge tests usage fix and location fixes first, this will
> > provide
> > >> us
> > >> > an ability to automate check using Travis.
> > >> >
> > >> > On Tue, Oct 20, 2020 at 12:06 PM Ivan Pavlukhin <
> vololo...@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Max, Ivan,
> > >> > >
> > >> > > Using explicit @Ignore and the automated check sounds good to me.
> If
> > >> > > nobody has arguments against it I think we should do it.
> > >> > >
> > >> > > 2020-10-19 19:30 GMT+03:00, Max Timonin  >:
> > >> > > > Hi Ivan,
> > >> > > >
> > >> > > > I've checked the ticket you provide. It contains subtasks to
> > >> uncomment
> > >> > or
> > >> > > > to remove some unused tests. It definitely describes some cases
> > I've
> > >> > > found.
> > >> > > > So what do you think if I uncomment them in suites, add @Ignore
> > >> > > annotation
> > >> > > > for those tests while the tickets are open? This will help to
> find
> > >> out
> > >> > > > tests that were forgiven in a recent time.
> > >> > > >
> > >> > > > Also I believe that this check must be automated. I didn't find
> a
> > >> way
> > >> > how
> > >> > > > uncomment / unused tests are found in the ticket. If there is no
> > >> any -
> > >> > I
> > >> > > > propose mine PR for this purpose.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Mon, Oct 19, 2020 at 5:24 PM Ivan Daschinsky <
> > >> ivanda...@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > >> Ivan, as far as I understand, Max also created verification
> check
> > >> for
> > >> > > not
> > >> > > >> included test and found a few tests, that have never been
> > included
> > >> in
> > >> > > any
> > >> > > >> testsuites.
> > >> > > >>
> > >> > > >> Also, I suppose, that even if we cannot run some tests, these
> > tests
> > >> > > >> should
> > >> > > >> be ignored using annotation, but not commented.
> > >> > > >>
> > >> > > >> пн, 19 окт. 2020 г. в 16:33, Ivan Pavlukhin <
> vololo...@gmail.com
> > >:
> > >> > > >>
> > >> > > >> > Hi Max,
> > >> > > >> >
> > >> > > >> > There is an 

Re: [DISCUSS] Missed (non-suited) tests

[Anton Vinogradov]

> As I understand we
> can't just move suites between modules, as TeamCity may depend on the path
> to them.
See no problem to update TC as well.

On Fri, Oct 30, 2020 at 4:32 PM Ivan Daschinsky  wrote:

> I suggests to mark these tests with @Ignore and file tickets to fix them.
>
> пт, 30 окт. 2020 г. в 16:26, Ivan Daschinsky :
>
> > Hi
> >
> > WalCompactionAfterRestartTest -- yes we need it. This test failed because
> > of race (test shold be rewritten a little bit)
> >
> > пт, 30 окт. 2020 г. в 16:15, Max Timonin :
> >
> >> Hi!
> >>
> >> Yes, you're correct. I've developed the check and started to clean tests
> >> (move them to suites, mark some tests with Ignore, etc.). I finish work
> on
> >> the core module. I hope it was the biggest one, and others are less. If
> >> so,
> >> I think I will finish the work on other modules in 1 or 2 weeks, as I do
> >> this activity in the background (~10% of my work time). Actually I've
> >> found
> >> 3 failed tests in the core module that aren't in any suite, so I need
> time
> >> to discover reason of failures and if we actually need those tests:
> >>
> >> GridCacheMultithreadedFailoverAbstractTest
> >> WalCompactionAfterRestartTest
> >> GridTcpCommunicationSpiLogTest
> >>
> >> Also we should decide how to be with wrongly located es. As I understand
> >> we
> >> can't just move suites between modules, as TeamCity may depend on the
> path
> >> to them. So, for such cases I've just created suites in the right
> module,
> >> and replaced the test list with the new class suite. It does not look
> >> pretty enough, but I think It's a path of least resistance. WDYT?
> >>
> >> BEFORE:
> >> Module A -> SuiteA -> testA1, testA2, testB1, testB2
> >> Module B -> testB1, testB2
> >>
> >> AFTER:
> >> Module A -> SuiteA, SuiteB
> >> Module B -> SuiteB -> testB1, testB2
> >>
> >> On Fri, Oct 30, 2020 at 3:38 PM Anton Vinogradov  wrote:
> >>
> >> > Folks,
> >> > What's the current state of this thread?
> >> > AFAIU, we found unused and wrongly located tests and developed some
> >> > checker, could we split this to some PRs?
> >> > Let's merge tests usage fix and location fixes first, this will
> provide
> >> us
> >> > an ability to automate check using Travis.
> >> >
> >> > On Tue, Oct 20, 2020 at 12:06 PM Ivan Pavlukhin 
> >> > wrote:
> >> >
> >> > > Max, Ivan,
> >> > >
> >> > > Using explicit @Ignore and the automated check sounds good to me. If
> >> > > nobody has arguments against it I think we should do it.
> >> > >
> >> > > 2020-10-19 19:30 GMT+03:00, Max Timonin :
> >> > > > Hi Ivan,
> >> > > >
> >> > > > I've checked the ticket you provide. It contains subtasks to
> >> uncomment
> >> > or
> >> > > > to remove some unused tests. It definitely describes some cases
> I've
> >> > > found.
> >> > > > So what do you think if I uncomment them in suites, add @Ignore
> >> > > annotation
> >> > > > for those tests while the tickets are open? This will help to find
> >> out
> >> > > > tests that were forgiven in a recent time.
> >> > > >
> >> > > > Also I believe that this check must be automated. I didn't find a
> >> way
> >> > how
> >> > > > uncomment / unused tests are found in the ticket. If there is no
> >> any -
> >> > I
> >> > > > propose mine PR for this purpose.
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Mon, Oct 19, 2020 at 5:24 PM Ivan Daschinsky <
> >> ivanda...@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > >> Ivan, as far as I understand, Max also created verification check
> >> for
> >> > > not
> >> > > >> included test and found a few tests, that have never been
> included
> >> in
> >> > > any
> >> > > >> testsuites.
> >> > > >>
> >> > > >> Also, I suppose, that even if we cannot run some tests, these
> tests
> >> > > >> should
> >> > > >> be ignored using annotation, but not commented.
> >> > > >>
> >> > > >> пн, 19 окт. 2020 г. в 16:33, Ivan Pavlukhin  >:
> >> > > >>
> >> > > >> > Hi Max,
> >> > > >> >
> >> > > >> > There is an existing effort about "abandoned" tests
> >> > > >> > https://issues.apache.org/jira/browse/IGNITE-9210
> >> > > >> >
> >> > > >> > 2020-10-19 16:25 GMT+03:00, Max Timonin <
> timonin.ma...@gmail.com
> >> >:
> >> > > >> > > Hi Igniters!
> >> > > >> > >
> >> > > >> > > I made a research into tests that aren't included in any test
> >> > suite.
> >> > > >> > > As
> >> > > >> > > TeamCity runs tests by suites so there could be tests that
> >> never
> >> > run
> >> > > >> > > on
> >> > > >> > TC.
> >> > > >> > > So I tried implementing a simple check for such tests and
> >> include
> >> > it
> >> > > >> > > in
> >> > > >> > > Ignite's travis config.
> >> > > >> > >
> >> > > >> > > The check runs while "mvn test" command and piggy-backs on
> the
> >> > maven
> >> > > >> > > surefire plugin. I replaced the junit provider with a custom
> >> one
> >> > > that
> >> > > >> > > checks if a class is a test or a suite (there are some Ignite
> >> > > >> > > specific
> >> > > >> > > stuff), marks tests that are in suites and raises an
> exception
> 

Re: [DISCUSS] Missed (non-suited) tests

[Ivan Daschinsky]

I suggests to mark these tests with @Ignore and file tickets to fix them.

пт, 30 окт. 2020 г. в 16:26, Ivan Daschinsky :

> Hi
>
> WalCompactionAfterRestartTest -- yes we need it. This test failed because
> of race (test shold be rewritten a little bit)
>
> пт, 30 окт. 2020 г. в 16:15, Max Timonin :
>
>> Hi!
>>
>> Yes, you're correct. I've developed the check and started to clean tests
>> (move them to suites, mark some tests with Ignore, etc.). I finish work on
>> the core module. I hope it was the biggest one, and others are less. If
>> so,
>> I think I will finish the work on other modules in 1 or 2 weeks, as I do
>> this activity in the background (~10% of my work time). Actually I've
>> found
>> 3 failed tests in the core module that aren't in any suite, so I need time
>> to discover reason of failures and if we actually need those tests:
>>
>> GridCacheMultithreadedFailoverAbstractTest
>> WalCompactionAfterRestartTest
>> GridTcpCommunicationSpiLogTest
>>
>> Also we should decide how to be with wrongly located es. As I understand
>> we
>> can't just move suites between modules, as TeamCity may depend on the path
>> to them. So, for such cases I've just created suites in the right module,
>> and replaced the test list with the new class suite. It does not look
>> pretty enough, but I think It's a path of least resistance. WDYT?
>>
>> BEFORE:
>> Module A -> SuiteA -> testA1, testA2, testB1, testB2
>> Module B -> testB1, testB2
>>
>> AFTER:
>> Module A -> SuiteA, SuiteB
>> Module B -> SuiteB -> testB1, testB2
>>
>> On Fri, Oct 30, 2020 at 3:38 PM Anton Vinogradov  wrote:
>>
>> > Folks,
>> > What's the current state of this thread?
>> > AFAIU, we found unused and wrongly located tests and developed some
>> > checker, could we split this to some PRs?
>> > Let's merge tests usage fix and location fixes first, this will provide
>> us
>> > an ability to automate check using Travis.
>> >
>> > On Tue, Oct 20, 2020 at 12:06 PM Ivan Pavlukhin 
>> > wrote:
>> >
>> > > Max, Ivan,
>> > >
>> > > Using explicit @Ignore and the automated check sounds good to me. If
>> > > nobody has arguments against it I think we should do it.
>> > >
>> > > 2020-10-19 19:30 GMT+03:00, Max Timonin :
>> > > > Hi Ivan,
>> > > >
>> > > > I've checked the ticket you provide. It contains subtasks to
>> uncomment
>> > or
>> > > > to remove some unused tests. It definitely describes some cases I've
>> > > found.
>> > > > So what do you think if I uncomment them in suites, add @Ignore
>> > > annotation
>> > > > for those tests while the tickets are open? This will help to find
>> out
>> > > > tests that were forgiven in a recent time.
>> > > >
>> > > > Also I believe that this check must be automated. I didn't find a
>> way
>> > how
>> > > > uncomment / unused tests are found in the ticket. If there is no
>> any -
>> > I
>> > > > propose mine PR for this purpose.
>> > > >
>> > > >
>> > > >
>> > > > On Mon, Oct 19, 2020 at 5:24 PM Ivan Daschinsky <
>> ivanda...@gmail.com>
>> > > > wrote:
>> > > >
>> > > >> Ivan, as far as I understand, Max also created verification check
>> for
>> > > not
>> > > >> included test and found a few tests, that have never been included
>> in
>> > > any
>> > > >> testsuites.
>> > > >>
>> > > >> Also, I suppose, that even if we cannot run some tests, these tests
>> > > >> should
>> > > >> be ignored using annotation, but not commented.
>> > > >>
>> > > >> пн, 19 окт. 2020 г. в 16:33, Ivan Pavlukhin :
>> > > >>
>> > > >> > Hi Max,
>> > > >> >
>> > > >> > There is an existing effort about "abandoned" tests
>> > > >> > https://issues.apache.org/jira/browse/IGNITE-9210
>> > > >> >
>> > > >> > 2020-10-19 16:25 GMT+03:00, Max Timonin > >:
>> > > >> > > Hi Igniters!
>> > > >> > >
>> > > >> > > I made a research into tests that aren't included in any test
>> > suite.
>> > > >> > > As
>> > > >> > > TeamCity runs tests by suites so there could be tests that
>> never
>> > run
>> > > >> > > on
>> > > >> > TC.
>> > > >> > > So I tried implementing a simple check for such tests and
>> include
>> > it
>> > > >> > > in
>> > > >> > > Ignite's travis config.
>> > > >> > >
>> > > >> > > The check runs while "mvn test" command and piggy-backs on the
>> > maven
>> > > >> > > surefire plugin. I replaced the junit provider with a custom
>> one
>> > > that
>> > > >> > > checks if a class is a test or a suite (there are some Ignite
>> > > >> > > specific
>> > > >> > > stuff), marks tests that are in suites and raises an exception
>> if
>> > > >> > > there
>> > > >> > are
>> > > >> > > non-suited tests. It's implemented as a part of maven command
>> so
>> > it
>> > > >> runs
>> > > >> > > for every module separately.
>> > > >> > >
>> > > >> > > I've prepared draft PR with this check:
>> > > >> > > https://github.com/apache/ignite/pull/8367
>> > > >> > > Travis check report is here:
>> > > >> > > https://travis-ci.org/github/apache/ignite/jobs/737046387
>> > > >> > > As It's a draft, so I skip some maven configuration steps for a
>> > > >> > 

Re: [DISCUSS] Missed (non-suited) tests

[Ivan Daschinsky]

Hi

WalCompactionAfterRestartTest -- yes we need it. This test failed because
of race (test shold be rewritten a little bit)

пт, 30 окт. 2020 г. в 16:15, Max Timonin :

> Hi!
>
> Yes, you're correct. I've developed the check and started to clean tests
> (move them to suites, mark some tests with Ignore, etc.). I finish work on
> the core module. I hope it was the biggest one, and others are less. If so,
> I think I will finish the work on other modules in 1 or 2 weeks, as I do
> this activity in the background (~10% of my work time). Actually I've found
> 3 failed tests in the core module that aren't in any suite, so I need time
> to discover reason of failures and if we actually need those tests:
>
> GridCacheMultithreadedFailoverAbstractTest
> WalCompactionAfterRestartTest
> GridTcpCommunicationSpiLogTest
>
> Also we should decide how to be with wrongly located es. As I understand we
> can't just move suites between modules, as TeamCity may depend on the path
> to them. So, for such cases I've just created suites in the right module,
> and replaced the test list with the new class suite. It does not look
> pretty enough, but I think It's a path of least resistance. WDYT?
>
> BEFORE:
> Module A -> SuiteA -> testA1, testA2, testB1, testB2
> Module B -> testB1, testB2
>
> AFTER:
> Module A -> SuiteA, SuiteB
> Module B -> SuiteB -> testB1, testB2
>
> On Fri, Oct 30, 2020 at 3:38 PM Anton Vinogradov  wrote:
>
> > Folks,
> > What's the current state of this thread?
> > AFAIU, we found unused and wrongly located tests and developed some
> > checker, could we split this to some PRs?
> > Let's merge tests usage fix and location fixes first, this will provide
> us
> > an ability to automate check using Travis.
> >
> > On Tue, Oct 20, 2020 at 12:06 PM Ivan Pavlukhin 
> > wrote:
> >
> > > Max, Ivan,
> > >
> > > Using explicit @Ignore and the automated check sounds good to me. If
> > > nobody has arguments against it I think we should do it.
> > >
> > > 2020-10-19 19:30 GMT+03:00, Max Timonin :
> > > > Hi Ivan,
> > > >
> > > > I've checked the ticket you provide. It contains subtasks to
> uncomment
> > or
> > > > to remove some unused tests. It definitely describes some cases I've
> > > found.
> > > > So what do you think if I uncomment them in suites, add @Ignore
> > > annotation
> > > > for those tests while the tickets are open? This will help to find
> out
> > > > tests that were forgiven in a recent time.
> > > >
> > > > Also I believe that this check must be automated. I didn't find a way
> > how
> > > > uncomment / unused tests are found in the ticket. If there is no any
> -
> > I
> > > > propose mine PR for this purpose.
> > > >
> > > >
> > > >
> > > > On Mon, Oct 19, 2020 at 5:24 PM Ivan Daschinsky  >
> > > > wrote:
> > > >
> > > >> Ivan, as far as I understand, Max also created verification check
> for
> > > not
> > > >> included test and found a few tests, that have never been included
> in
> > > any
> > > >> testsuites.
> > > >>
> > > >> Also, I suppose, that even if we cannot run some tests, these tests
> > > >> should
> > > >> be ignored using annotation, but not commented.
> > > >>
> > > >> пн, 19 окт. 2020 г. в 16:33, Ivan Pavlukhin :
> > > >>
> > > >> > Hi Max,
> > > >> >
> > > >> > There is an existing effort about "abandoned" tests
> > > >> > https://issues.apache.org/jira/browse/IGNITE-9210
> > > >> >
> > > >> > 2020-10-19 16:25 GMT+03:00, Max Timonin  >:
> > > >> > > Hi Igniters!
> > > >> > >
> > > >> > > I made a research into tests that aren't included in any test
> > suite.
> > > >> > > As
> > > >> > > TeamCity runs tests by suites so there could be tests that never
> > run
> > > >> > > on
> > > >> > TC.
> > > >> > > So I tried implementing a simple check for such tests and
> include
> > it
> > > >> > > in
> > > >> > > Ignite's travis config.
> > > >> > >
> > > >> > > The check runs while "mvn test" command and piggy-backs on the
> > maven
> > > >> > > surefire plugin. I replaced the junit provider with a custom one
> > > that
> > > >> > > checks if a class is a test or a suite (there are some Ignite
> > > >> > > specific
> > > >> > > stuff), marks tests that are in suites and raises an exception
> if
> > > >> > > there
> > > >> > are
> > > >> > > non-suited tests. It's implemented as a part of maven command so
> > it
> > > >> runs
> > > >> > > for every module separately.
> > > >> > >
> > > >> > > I've prepared draft PR with this check:
> > > >> > > https://github.com/apache/ignite/pull/8367
> > > >> > > Travis check report is here:
> > > >> > > https://travis-ci.org/github/apache/ignite/jobs/737046387
> > > >> > > As It's a draft, so I skip some maven configuration steps for a
> > > >> > > while.
> > > >> > Also
> > > >> > > I run the check only for the core module.
> > > >> > >
> > > >> > > But I have some results that want to discuss before continue the
> > > >> > > work:
> > > >> > > 1. Currently in the core module there are 53 tests that aren't
> > part
> > > >> > > of
> > > >> > any
> 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi!

Yes, you're correct. I've developed the check and started to clean tests
(move them to suites, mark some tests with Ignore, etc.). I finish work on
the core module. I hope it was the biggest one, and others are less. If so,
I think I will finish the work on other modules in 1 or 2 weeks, as I do
this activity in the background (~10% of my work time). Actually I've found
3 failed tests in the core module that aren't in any suite, so I need time
to discover reason of failures and if we actually need those tests:

GridCacheMultithreadedFailoverAbstractTest
WalCompactionAfterRestartTest
GridTcpCommunicationSpiLogTest

Also we should decide how to be with wrongly located es. As I understand we
can't just move suites between modules, as TeamCity may depend on the path
to them. So, for such cases I've just created suites in the right module,
and replaced the test list with the new class suite. It does not look
pretty enough, but I think It's a path of least resistance. WDYT?

BEFORE:
Module A -> SuiteA -> testA1, testA2, testB1, testB2
Module B -> testB1, testB2

AFTER:
Module A -> SuiteA, SuiteB
Module B -> SuiteB -> testB1, testB2

On Fri, Oct 30, 2020 at 3:38 PM Anton Vinogradov  wrote:

> Folks,
> What's the current state of this thread?
> AFAIU, we found unused and wrongly located tests and developed some
> checker, could we split this to some PRs?
> Let's merge tests usage fix and location fixes first, this will provide us
> an ability to automate check using Travis.
>
> On Tue, Oct 20, 2020 at 12:06 PM Ivan Pavlukhin 
> wrote:
>
> > Max, Ivan,
> >
> > Using explicit @Ignore and the automated check sounds good to me. If
> > nobody has arguments against it I think we should do it.
> >
> > 2020-10-19 19:30 GMT+03:00, Max Timonin :
> > > Hi Ivan,
> > >
> > > I've checked the ticket you provide. It contains subtasks to uncomment
> or
> > > to remove some unused tests. It definitely describes some cases I've
> > found.
> > > So what do you think if I uncomment them in suites, add @Ignore
> > annotation
> > > for those tests while the tickets are open? This will help to find out
> > > tests that were forgiven in a recent time.
> > >
> > > Also I believe that this check must be automated. I didn't find a way
> how
> > > uncomment / unused tests are found in the ticket. If there is no any -
> I
> > > propose mine PR for this purpose.
> > >
> > >
> > >
> > > On Mon, Oct 19, 2020 at 5:24 PM Ivan Daschinsky 
> > > wrote:
> > >
> > >> Ivan, as far as I understand, Max also created verification check for
> > not
> > >> included test and found a few tests, that have never been included in
> > any
> > >> testsuites.
> > >>
> > >> Also, I suppose, that even if we cannot run some tests, these tests
> > >> should
> > >> be ignored using annotation, but not commented.
> > >>
> > >> пн, 19 окт. 2020 г. в 16:33, Ivan Pavlukhin :
> > >>
> > >> > Hi Max,
> > >> >
> > >> > There is an existing effort about "abandoned" tests
> > >> > https://issues.apache.org/jira/browse/IGNITE-9210
> > >> >
> > >> > 2020-10-19 16:25 GMT+03:00, Max Timonin :
> > >> > > Hi Igniters!
> > >> > >
> > >> > > I made a research into tests that aren't included in any test
> suite.
> > >> > > As
> > >> > > TeamCity runs tests by suites so there could be tests that never
> run
> > >> > > on
> > >> > TC.
> > >> > > So I tried implementing a simple check for such tests and include
> it
> > >> > > in
> > >> > > Ignite's travis config.
> > >> > >
> > >> > > The check runs while "mvn test" command and piggy-backs on the
> maven
> > >> > > surefire plugin. I replaced the junit provider with a custom one
> > that
> > >> > > checks if a class is a test or a suite (there are some Ignite
> > >> > > specific
> > >> > > stuff), marks tests that are in suites and raises an exception if
> > >> > > there
> > >> > are
> > >> > > non-suited tests. It's implemented as a part of maven command so
> it
> > >> runs
> > >> > > for every module separately.
> > >> > >
> > >> > > I've prepared draft PR with this check:
> > >> > > https://github.com/apache/ignite/pull/8367
> > >> > > Travis check report is here:
> > >> > > https://travis-ci.org/github/apache/ignite/jobs/737046387
> > >> > > As It's a draft, so I skip some maven configuration steps for a
> > >> > > while.
> > >> > Also
> > >> > > I run the check only for the core module.
> > >> > >
> > >> > > But I have some results that want to discuss before continue the
> > >> > > work:
> > >> > > 1. Currently in the core module there are 53 tests that aren't
> part
> > >> > > of
> > >> > any
> > >> > > test suite. I'm not sure about the reason for every test. So I
> just
> > >> > > put
> > >> > > below a list of the tests and last contributor to a file that
> > >> > > contains
> > >> a
> > >> > > test.
> > >> > >
> > >> > > 2. Some tests are located in the core module, but suites are in a
> > >> > > different, for example ignite-indexing suite
> > >> > > IgniteCacheQuerySelfTestSuite3 contains
> > >> > > only tests written in 

Re: [DISCUSS] Missed (non-suited) tests

[Anton Vinogradov]

Folks,
What's the current state of this thread?
AFAIU, we found unused and wrongly located tests and developed some
checker, could we split this to some PRs?
Let's merge tests usage fix and location fixes first, this will provide us
an ability to automate check using Travis.

On Tue, Oct 20, 2020 at 12:06 PM Ivan Pavlukhin  wrote:

> Max, Ivan,
>
> Using explicit @Ignore and the automated check sounds good to me. If
> nobody has arguments against it I think we should do it.
>
> 2020-10-19 19:30 GMT+03:00, Max Timonin :
> > Hi Ivan,
> >
> > I've checked the ticket you provide. It contains subtasks to uncomment or
> > to remove some unused tests. It definitely describes some cases I've
> found.
> > So what do you think if I uncomment them in suites, add @Ignore
> annotation
> > for those tests while the tickets are open? This will help to find out
> > tests that were forgiven in a recent time.
> >
> > Also I believe that this check must be automated. I didn't find a way how
> > uncomment / unused tests are found in the ticket. If there is no any - I
> > propose mine PR for this purpose.
> >
> >
> >
> > On Mon, Oct 19, 2020 at 5:24 PM Ivan Daschinsky 
> > wrote:
> >
> >> Ivan, as far as I understand, Max also created verification check for
> not
> >> included test and found a few tests, that have never been included in
> any
> >> testsuites.
> >>
> >> Also, I suppose, that even if we cannot run some tests, these tests
> >> should
> >> be ignored using annotation, but not commented.
> >>
> >> пн, 19 окт. 2020 г. в 16:33, Ivan Pavlukhin :
> >>
> >> > Hi Max,
> >> >
> >> > There is an existing effort about "abandoned" tests
> >> > https://issues.apache.org/jira/browse/IGNITE-9210
> >> >
> >> > 2020-10-19 16:25 GMT+03:00, Max Timonin :
> >> > > Hi Igniters!
> >> > >
> >> > > I made a research into tests that aren't included in any test suite.
> >> > > As
> >> > > TeamCity runs tests by suites so there could be tests that never run
> >> > > on
> >> > TC.
> >> > > So I tried implementing a simple check for such tests and include it
> >> > > in
> >> > > Ignite's travis config.
> >> > >
> >> > > The check runs while "mvn test" command and piggy-backs on the maven
> >> > > surefire plugin. I replaced the junit provider with a custom one
> that
> >> > > checks if a class is a test or a suite (there are some Ignite
> >> > > specific
> >> > > stuff), marks tests that are in suites and raises an exception if
> >> > > there
> >> > are
> >> > > non-suited tests. It's implemented as a part of maven command so it
> >> runs
> >> > > for every module separately.
> >> > >
> >> > > I've prepared draft PR with this check:
> >> > > https://github.com/apache/ignite/pull/8367
> >> > > Travis check report is here:
> >> > > https://travis-ci.org/github/apache/ignite/jobs/737046387
> >> > > As It's a draft, so I skip some maven configuration steps for a
> >> > > while.
> >> > Also
> >> > > I run the check only for the core module.
> >> > >
> >> > > But I have some results that want to discuss before continue the
> >> > > work:
> >> > > 1. Currently in the core module there are 53 tests that aren't part
> >> > > of
> >> > any
> >> > > test suite. I'm not sure about the reason for every test. So I just
> >> > > put
> >> > > below a list of the tests and last contributor to a file that
> >> > > contains
> >> a
> >> > > test.
> >> > >
> >> > > 2. Some tests are located in the core module, but suites are in a
> >> > > different, for example ignite-indexing suite
> >> > > IgniteCacheQuerySelfTestSuite3 contains
> >> > > only tests written in the core module, and none from the indexing
> >> module.
> >> > > Also there are suites in spring, uri-deploy, zookeeper modules. In
> my
> >> PR
> >> > > I've just copied the test suites to the core module.
> >> > >
> >> > > 3. Some test classes are named with the "Abstract" suffix but don't
> >> have
> >> > > the corresponding modifier (for example,
> >> > > IgniteTxTimeoutAbstractTest).
> >> > So,
> >> > > I add the modifier for every such file if it's not a part of any
> >> > > suite.
> >> > >
> >> > > What do you think about this check? If Ignite needs it, let's
> discuss
> >> > next
> >> > > things:
> >> > > 1. Mark tests that should never be in any suite by some reason;
> >> > > 2. Fix the missed tests;
> >> > > 3. How to declare suites that contains tests from a different
> module;
> >> > > 4. How to check if TC runs all suites.
> >> > >
> >> > > List of non-suited tests in the core module:
> >> > >
> >> > > maksim.stepac...@gmail.com:
> >> > > GridTcpCommunicationSpiLogTest
> >> > >
> >> > > nizhi...@apache.org:
> >> > > IgniteCacheClientMultiNodeUpdateTopologyLockTest
> >> > > CacheClientsConcurrentStartTest
> >> > > IgniteOutOfMemoryPropagationTest
> >> > > GridCacheP2PUndeploySelfTest
> >> > > GridCacheRebalancingOrderingTest
> >> > > IgniteMassLoadSandboxTest
> >> > > PageLockTrackerMXBeanImplTest
> >> > > 

Re: [DISCUSS] Missed (non-suited) tests

[Ivan Pavlukhin]

Max, Ivan,

Using explicit @Ignore and the automated check sounds good to me. If
nobody has arguments against it I think we should do it.

2020-10-19 19:30 GMT+03:00, Max Timonin :
> Hi Ivan,
>
> I've checked the ticket you provide. It contains subtasks to uncomment or
> to remove some unused tests. It definitely describes some cases I've found.
> So what do you think if I uncomment them in suites, add @Ignore annotation
> for those tests while the tickets are open? This will help to find out
> tests that were forgiven in a recent time.
>
> Also I believe that this check must be automated. I didn't find a way how
> uncomment / unused tests are found in the ticket. If there is no any - I
> propose mine PR for this purpose.
>
>
>
> On Mon, Oct 19, 2020 at 5:24 PM Ivan Daschinsky 
> wrote:
>
>> Ivan, as far as I understand, Max also created verification check for not
>> included test and found a few tests, that have never been included in any
>> testsuites.
>>
>> Also, I suppose, that even if we cannot run some tests, these tests
>> should
>> be ignored using annotation, but not commented.
>>
>> пн, 19 окт. 2020 г. в 16:33, Ivan Pavlukhin :
>>
>> > Hi Max,
>> >
>> > There is an existing effort about "abandoned" tests
>> > https://issues.apache.org/jira/browse/IGNITE-9210
>> >
>> > 2020-10-19 16:25 GMT+03:00, Max Timonin :
>> > > Hi Igniters!
>> > >
>> > > I made a research into tests that aren't included in any test suite.
>> > > As
>> > > TeamCity runs tests by suites so there could be tests that never run
>> > > on
>> > TC.
>> > > So I tried implementing a simple check for such tests and include it
>> > > in
>> > > Ignite's travis config.
>> > >
>> > > The check runs while "mvn test" command and piggy-backs on the maven
>> > > surefire plugin. I replaced the junit provider with a custom one that
>> > > checks if a class is a test or a suite (there are some Ignite
>> > > specific
>> > > stuff), marks tests that are in suites and raises an exception if
>> > > there
>> > are
>> > > non-suited tests. It's implemented as a part of maven command so it
>> runs
>> > > for every module separately.
>> > >
>> > > I've prepared draft PR with this check:
>> > > https://github.com/apache/ignite/pull/8367
>> > > Travis check report is here:
>> > > https://travis-ci.org/github/apache/ignite/jobs/737046387
>> > > As It's a draft, so I skip some maven configuration steps for a
>> > > while.
>> > Also
>> > > I run the check only for the core module.
>> > >
>> > > But I have some results that want to discuss before continue the
>> > > work:
>> > > 1. Currently in the core module there are 53 tests that aren't part
>> > > of
>> > any
>> > > test suite. I'm not sure about the reason for every test. So I just
>> > > put
>> > > below a list of the tests and last contributor to a file that
>> > > contains
>> a
>> > > test.
>> > >
>> > > 2. Some tests are located in the core module, but suites are in a
>> > > different, for example ignite-indexing suite
>> > > IgniteCacheQuerySelfTestSuite3 contains
>> > > only tests written in the core module, and none from the indexing
>> module.
>> > > Also there are suites in spring, uri-deploy, zookeeper modules. In my
>> PR
>> > > I've just copied the test suites to the core module.
>> > >
>> > > 3. Some test classes are named with the "Abstract" suffix but don't
>> have
>> > > the corresponding modifier (for example,
>> > > IgniteTxTimeoutAbstractTest).
>> > So,
>> > > I add the modifier for every such file if it's not a part of any
>> > > suite.
>> > >
>> > > What do you think about this check? If Ignite needs it, let's discuss
>> > next
>> > > things:
>> > > 1. Mark tests that should never be in any suite by some reason;
>> > > 2. Fix the missed tests;
>> > > 3. How to declare suites that contains tests from a different module;
>> > > 4. How to check if TC runs all suites.
>> > >
>> > > List of non-suited tests in the core module:
>> > >
>> > > maksim.stepac...@gmail.com:
>> > > GridTcpCommunicationSpiLogTest
>> > >
>> > > nizhi...@apache.org:
>> > > IgniteCacheClientMultiNodeUpdateTopologyLockTest
>> > > CacheClientsConcurrentStartTest
>> > > IgniteOutOfMemoryPropagationTest
>> > > GridCacheP2PUndeploySelfTest
>> > > GridCacheRebalancingOrderingTest
>> > > IgniteMassLoadSandboxTest
>> > > PageLockTrackerMXBeanImplTest
>> > > IgniteBinaryMetadataUpdateNodeRestartTest
>> > > CacheLockCandidatesThreadTest
>> > > GridMBeanBaselineTest
>> > > RendezvousAffinityFunctionSimpleBenchmark
>> > >
>> > > samvi...@yandex.ru:
>> > > IgnitePdsNoSpaceLeftOnDeviceTest
>> > >
>> > > maxmu...@gmail.com:
>> > > GridCacheOnCopyFlagReplicatedSelfTest
>> > > GridCacheOnCopyFlagLocalSelfTest
>> > > GridCacheReplicatedAtomicReferenceMultiNodeTest
>> > > GridCacheReplicatedMarshallerTxTest
>> > > GridCacheReplicatedTxConcurrentGetTest
>> > > 

Re: [DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi Ivan,

I've checked the ticket you provide. It contains subtasks to uncomment or
to remove some unused tests. It definitely describes some cases I've found.
So what do you think if I uncomment them in suites, add @Ignore annotation
for those tests while the tickets are open? This will help to find out
tests that were forgiven in a recent time.

Also I believe that this check must be automated. I didn't find a way how
uncomment / unused tests are found in the ticket. If there is no any - I
propose mine PR for this purpose.



On Mon, Oct 19, 2020 at 5:24 PM Ivan Daschinsky  wrote:

> Ivan, as far as I understand, Max also created verification check for not
> included test and found a few tests, that have never been included in any
> testsuites.
>
> Also, I suppose, that even if we cannot run some tests, these tests should
> be ignored using annotation, but not commented.
>
> пн, 19 окт. 2020 г. в 16:33, Ivan Pavlukhin :
>
> > Hi Max,
> >
> > There is an existing effort about "abandoned" tests
> > https://issues.apache.org/jira/browse/IGNITE-9210
> >
> > 2020-10-19 16:25 GMT+03:00, Max Timonin :
> > > Hi Igniters!
> > >
> > > I made a research into tests that aren't included in any test suite. As
> > > TeamCity runs tests by suites so there could be tests that never run on
> > TC.
> > > So I tried implementing a simple check for such tests and include it in
> > > Ignite's travis config.
> > >
> > > The check runs while "mvn test" command and piggy-backs on the maven
> > > surefire plugin. I replaced the junit provider with a custom one that
> > > checks if a class is a test or a suite (there are some Ignite specific
> > > stuff), marks tests that are in suites and raises an exception if there
> > are
> > > non-suited tests. It's implemented as a part of maven command so it
> runs
> > > for every module separately.
> > >
> > > I've prepared draft PR with this check:
> > > https://github.com/apache/ignite/pull/8367
> > > Travis check report is here:
> > > https://travis-ci.org/github/apache/ignite/jobs/737046387
> > > As It's a draft, so I skip some maven configuration steps for a while.
> > Also
> > > I run the check only for the core module.
> > >
> > > But I have some results that want to discuss before continue the work:
> > > 1. Currently in the core module there are 53 tests that aren't part of
> > any
> > > test suite. I'm not sure about the reason for every test. So I just put
> > > below a list of the tests and last contributor to a file that contains
> a
> > > test.
> > >
> > > 2. Some tests are located in the core module, but suites are in a
> > > different, for example ignite-indexing suite
> > > IgniteCacheQuerySelfTestSuite3 contains
> > > only tests written in the core module, and none from the indexing
> module.
> > > Also there are suites in spring, uri-deploy, zookeeper modules. In my
> PR
> > > I've just copied the test suites to the core module.
> > >
> > > 3. Some test classes are named with the "Abstract" suffix but don't
> have
> > > the corresponding modifier (for example, IgniteTxTimeoutAbstractTest).
> > So,
> > > I add the modifier for every such file if it's not a part of any suite.
> > >
> > > What do you think about this check? If Ignite needs it, let's discuss
> > next
> > > things:
> > > 1. Mark tests that should never be in any suite by some reason;
> > > 2. Fix the missed tests;
> > > 3. How to declare suites that contains tests from a different module;
> > > 4. How to check if TC runs all suites.
> > >
> > > List of non-suited tests in the core module:
> > >
> > > maksim.stepac...@gmail.com:
> > > GridTcpCommunicationSpiLogTest
> > >
> > > nizhi...@apache.org:
> > > IgniteCacheClientMultiNodeUpdateTopologyLockTest
> > > CacheClientsConcurrentStartTest
> > > IgniteOutOfMemoryPropagationTest
> > > GridCacheP2PUndeploySelfTest
> > > GridCacheRebalancingOrderingTest
> > > IgniteMassLoadSandboxTest
> > > PageLockTrackerMXBeanImplTest
> > > IgniteBinaryMetadataUpdateNodeRestartTest
> > > CacheLockCandidatesThreadTest
> > > GridMBeanBaselineTest
> > > RendezvousAffinityFunctionSimpleBenchmark
> > >
> > > samvi...@yandex.ru:
> > > IgnitePdsNoSpaceLeftOnDeviceTest
> > >
> > > maxmu...@gmail.com:
> > > GridCacheOnCopyFlagReplicatedSelfTest
> > > GridCacheOnCopyFlagLocalSelfTest
> > > GridCacheReplicatedAtomicReferenceMultiNodeTest
> > > GridCacheReplicatedMarshallerTxTest
> > > GridCacheReplicatedTxConcurrentGetTest
> > > GridCacheOnCopyFlagTxPartitionedSelfTest
> > > GridCacheReplicatedTxReadTest
> > > GridCachePartitionedAtomicReferenceMultiNodeTest
> > > GridCacheOnCopyFlagAtomicSelfTest
> > >
> > > mmu...@apache.org:
> > > GridActivateExtensionTest
> > > IgniteChangeGlobalStateCacheTest
> > > IgniteChangeGlobalStateTest
> > > IgniteChangeGlobalStateServiceTest
> > > 

Re: [DISCUSS] Missed (non-suited) tests

[Ivan Daschinsky]

Ivan, as far as I understand, Max also created verification check for not
included test and found a few tests, that have never been included in any
testsuites.

Also, I suppose, that even if we cannot run some tests, these tests should
be ignored using annotation, but not commented.

пн, 19 окт. 2020 г. в 16:33, Ivan Pavlukhin :

> Hi Max,
>
> There is an existing effort about "abandoned" tests
> https://issues.apache.org/jira/browse/IGNITE-9210
>
> 2020-10-19 16:25 GMT+03:00, Max Timonin :
> > Hi Igniters!
> >
> > I made a research into tests that aren't included in any test suite. As
> > TeamCity runs tests by suites so there could be tests that never run on
> TC.
> > So I tried implementing a simple check for such tests and include it in
> > Ignite's travis config.
> >
> > The check runs while "mvn test" command and piggy-backs on the maven
> > surefire plugin. I replaced the junit provider with a custom one that
> > checks if a class is a test or a suite (there are some Ignite specific
> > stuff), marks tests that are in suites and raises an exception if there
> are
> > non-suited tests. It's implemented as a part of maven command so it runs
> > for every module separately.
> >
> > I've prepared draft PR with this check:
> > https://github.com/apache/ignite/pull/8367
> > Travis check report is here:
> > https://travis-ci.org/github/apache/ignite/jobs/737046387
> > As It's a draft, so I skip some maven configuration steps for a while.
> Also
> > I run the check only for the core module.
> >
> > But I have some results that want to discuss before continue the work:
> > 1. Currently in the core module there are 53 tests that aren't part of
> any
> > test suite. I'm not sure about the reason for every test. So I just put
> > below a list of the tests and last contributor to a file that contains a
> > test.
> >
> > 2. Some tests are located in the core module, but suites are in a
> > different, for example ignite-indexing suite
> > IgniteCacheQuerySelfTestSuite3 contains
> > only tests written in the core module, and none from the indexing module.
> > Also there are suites in spring, uri-deploy, zookeeper modules. In my PR
> > I've just copied the test suites to the core module.
> >
> > 3. Some test classes are named with the "Abstract" suffix but don't have
> > the corresponding modifier (for example, IgniteTxTimeoutAbstractTest).
> So,
> > I add the modifier for every such file if it's not a part of any suite.
> >
> > What do you think about this check? If Ignite needs it, let's discuss
> next
> > things:
> > 1. Mark tests that should never be in any suite by some reason;
> > 2. Fix the missed tests;
> > 3. How to declare suites that contains tests from a different module;
> > 4. How to check if TC runs all suites.
> >
> > List of non-suited tests in the core module:
> >
> > maksim.stepac...@gmail.com:
> > GridTcpCommunicationSpiLogTest
> >
> > nizhi...@apache.org:
> > IgniteCacheClientMultiNodeUpdateTopologyLockTest
> > CacheClientsConcurrentStartTest
> > IgniteOutOfMemoryPropagationTest
> > GridCacheP2PUndeploySelfTest
> > GridCacheRebalancingOrderingTest
> > IgniteMassLoadSandboxTest
> > PageLockTrackerMXBeanImplTest
> > IgniteBinaryMetadataUpdateNodeRestartTest
> > CacheLockCandidatesThreadTest
> > GridMBeanBaselineTest
> > RendezvousAffinityFunctionSimpleBenchmark
> >
> > samvi...@yandex.ru:
> > IgnitePdsNoSpaceLeftOnDeviceTest
> >
> > maxmu...@gmail.com:
> > GridCacheOnCopyFlagReplicatedSelfTest
> > GridCacheOnCopyFlagLocalSelfTest
> > GridCacheReplicatedAtomicReferenceMultiNodeTest
> > GridCacheReplicatedMarshallerTxTest
> > GridCacheReplicatedTxConcurrentGetTest
> > GridCacheOnCopyFlagTxPartitionedSelfTest
> > GridCacheReplicatedTxReadTest
> > GridCachePartitionedAtomicReferenceMultiNodeTest
> > GridCacheOnCopyFlagAtomicSelfTest
> >
> > mmu...@apache.org:
> > GridActivateExtensionTest
> > IgniteChangeGlobalStateCacheTest
> > IgniteChangeGlobalStateTest
> > IgniteChangeGlobalStateServiceTest
> > IgniteChangeGlobalStateDataStructureTest
> >
> > oignate...@gridgain.com:
> > CacheEntryProcessorCopySelfTest
> > MemoryLeaksOnRestartNodeTest
> > GridCacheAtomicPreloadSelfTest
> > WalCompactionAfterRestartTest
> > IgniteCacheConcurrentPutGetRemove
> > GridIoManagerBenchmark0
> >
> > nsamelc...@gmail.com:
> > GridLongRunningInitNewCrdFutureDiagnosticsTest
> > GridCacheMultithreadedFailoverAbstractTest
> >
> > alexey.goncha...@gmail.com:
> > GridCacheBinaryObjectsAtomicOnheapSelfTest
> > GridCacheBinaryObjectsAtomicNearDisabledOnheapSelfTest
> > GridCacheBinaryObjectsPartitionedOnheapSelfTest
> > GridCacheBinaryObjectsPartitionedNearDisabledOnheapSelfTest
> >
> > vladis...@gmail.com:
> > 

Re: [DISCUSS] Missed (non-suited) tests

[Ivan Pavlukhin]

Hi Max,

There is an existing effort about "abandoned" tests
https://issues.apache.org/jira/browse/IGNITE-9210

2020-10-19 16:25 GMT+03:00, Max Timonin :
> Hi Igniters!
>
> I made a research into tests that aren't included in any test suite. As
> TeamCity runs tests by suites so there could be tests that never run on TC.
> So I tried implementing a simple check for such tests and include it in
> Ignite's travis config.
>
> The check runs while "mvn test" command and piggy-backs on the maven
> surefire plugin. I replaced the junit provider with a custom one that
> checks if a class is a test or a suite (there are some Ignite specific
> stuff), marks tests that are in suites and raises an exception if there are
> non-suited tests. It's implemented as a part of maven command so it runs
> for every module separately.
>
> I've prepared draft PR with this check:
> https://github.com/apache/ignite/pull/8367
> Travis check report is here:
> https://travis-ci.org/github/apache/ignite/jobs/737046387
> As It's a draft, so I skip some maven configuration steps for a while. Also
> I run the check only for the core module.
>
> But I have some results that want to discuss before continue the work:
> 1. Currently in the core module there are 53 tests that aren't part of any
> test suite. I'm not sure about the reason for every test. So I just put
> below a list of the tests and last contributor to a file that contains a
> test.
>
> 2. Some tests are located in the core module, but suites are in a
> different, for example ignite-indexing suite
> IgniteCacheQuerySelfTestSuite3 contains
> only tests written in the core module, and none from the indexing module.
> Also there are suites in spring, uri-deploy, zookeeper modules. In my PR
> I've just copied the test suites to the core module.
>
> 3. Some test classes are named with the "Abstract" suffix but don't have
> the corresponding modifier (for example, IgniteTxTimeoutAbstractTest). So,
> I add the modifier for every such file if it's not a part of any suite.
>
> What do you think about this check? If Ignite needs it, let's discuss next
> things:
> 1. Mark tests that should never be in any suite by some reason;
> 2. Fix the missed tests;
> 3. How to declare suites that contains tests from a different module;
> 4. How to check if TC runs all suites.
>
> List of non-suited tests in the core module:
>
> maksim.stepac...@gmail.com:
> GridTcpCommunicationSpiLogTest
>
> nizhi...@apache.org:
> IgniteCacheClientMultiNodeUpdateTopologyLockTest
> CacheClientsConcurrentStartTest
> IgniteOutOfMemoryPropagationTest
> GridCacheP2PUndeploySelfTest
> GridCacheRebalancingOrderingTest
> IgniteMassLoadSandboxTest
> PageLockTrackerMXBeanImplTest
> IgniteBinaryMetadataUpdateNodeRestartTest
> CacheLockCandidatesThreadTest
> GridMBeanBaselineTest
> RendezvousAffinityFunctionSimpleBenchmark
>
> samvi...@yandex.ru:
> IgnitePdsNoSpaceLeftOnDeviceTest
>
> maxmu...@gmail.com:
> GridCacheOnCopyFlagReplicatedSelfTest
> GridCacheOnCopyFlagLocalSelfTest
> GridCacheReplicatedAtomicReferenceMultiNodeTest
> GridCacheReplicatedMarshallerTxTest
> GridCacheReplicatedTxConcurrentGetTest
> GridCacheOnCopyFlagTxPartitionedSelfTest
> GridCacheReplicatedTxReadTest
> GridCachePartitionedAtomicReferenceMultiNodeTest
> GridCacheOnCopyFlagAtomicSelfTest
>
> mmu...@apache.org:
> GridActivateExtensionTest
> IgniteChangeGlobalStateCacheTest
> IgniteChangeGlobalStateTest
> IgniteChangeGlobalStateServiceTest
> IgniteChangeGlobalStateDataStructureTest
>
> oignate...@gridgain.com:
> CacheEntryProcessorCopySelfTest
> MemoryLeaksOnRestartNodeTest
> GridCacheAtomicPreloadSelfTest
> WalCompactionAfterRestartTest
> IgniteCacheConcurrentPutGetRemove
> GridIoManagerBenchmark0
>
> nsamelc...@gmail.com:
> GridLongRunningInitNewCrdFutureDiagnosticsTest
> GridCacheMultithreadedFailoverAbstractTest
>
> alexey.goncha...@gmail.com:
> GridCacheBinaryObjectsAtomicOnheapSelfTest
> GridCacheBinaryObjectsAtomicNearDisabledOnheapSelfTest
> GridCacheBinaryObjectsPartitionedOnheapSelfTest
> GridCacheBinaryObjectsPartitionedNearDisabledOnheapSelfTest
>
> vladis...@gmail.com:
> IgnitePartitionedLockSelfTest
>
> alexandr.bel...@xored.com:
> IgniteStableBaselineCachePutAllFailoverTest
> IgniteStableBaselineCacheRemoveFailoverTest
>
> ilant...@gridgain.com:
> IgniteCacheAtomicOnheapExpiryPolicyTest
> IgniteCacheAtomicLocalOnheapExpiryPolicyTest
> GridCacheReplicatedOnheapFullApiSelfTest
> GridCacheBinaryObjectsLocalOnheapSelfTest
>
> oignate...@users.noreply.github.com:
> GridCacheTtlManagerEvictionSelfTest
>
> ira...@apache.org:
> CommonPoolStarvationCheckpointTest
>
> 

[DISCUSS] Missed (non-suited) tests

[Max Timonin]

Hi Igniters!

I made a research into tests that aren't included in any test suite. As
TeamCity runs tests by suites so there could be tests that never run on TC.
So I tried implementing a simple check for such tests and include it in
Ignite's travis config.

The check runs while "mvn test" command and piggy-backs on the maven
surefire plugin. I replaced the junit provider with a custom one that
checks if a class is a test or a suite (there are some Ignite specific
stuff), marks tests that are in suites and raises an exception if there are
non-suited tests. It's implemented as a part of maven command so it runs
for every module separately.

I've prepared draft PR with this check:
https://github.com/apache/ignite/pull/8367
Travis check report is here:
https://travis-ci.org/github/apache/ignite/jobs/737046387
As It's a draft, so I skip some maven configuration steps for a while. Also
I run the check only for the core module.

But I have some results that want to discuss before continue the work:
1. Currently in the core module there are 53 tests that aren't part of any
test suite. I'm not sure about the reason for every test. So I just put
below a list of the tests and last contributor to a file that contains a
test.

2. Some tests are located in the core module, but suites are in a
different, for example ignite-indexing suite
IgniteCacheQuerySelfTestSuite3 contains
only tests written in the core module, and none from the indexing module.
Also there are suites in spring, uri-deploy, zookeeper modules. In my PR
I've just copied the test suites to the core module.

3. Some test classes are named with the "Abstract" suffix but don't have
the corresponding modifier (for example, IgniteTxTimeoutAbstractTest). So,
I add the modifier for every such file if it's not a part of any suite.

What do you think about this check? If Ignite needs it, let's discuss next
things:
1. Mark tests that should never be in any suite by some reason;
2. Fix the missed tests;
3. How to declare suites that contains tests from a different module;
4. How to check if TC runs all suites.

List of non-suited tests in the core module:

maksim.stepac...@gmail.com:
GridTcpCommunicationSpiLogTest

nizhi...@apache.org:
IgniteCacheClientMultiNodeUpdateTopologyLockTest
CacheClientsConcurrentStartTest
IgniteOutOfMemoryPropagationTest
GridCacheP2PUndeploySelfTest
GridCacheRebalancingOrderingTest
IgniteMassLoadSandboxTest
PageLockTrackerMXBeanImplTest
IgniteBinaryMetadataUpdateNodeRestartTest
CacheLockCandidatesThreadTest
GridMBeanBaselineTest
RendezvousAffinityFunctionSimpleBenchmark

samvi...@yandex.ru:
IgnitePdsNoSpaceLeftOnDeviceTest

maxmu...@gmail.com:
GridCacheOnCopyFlagReplicatedSelfTest
GridCacheOnCopyFlagLocalSelfTest
GridCacheReplicatedAtomicReferenceMultiNodeTest
GridCacheReplicatedMarshallerTxTest
GridCacheReplicatedTxConcurrentGetTest
GridCacheOnCopyFlagTxPartitionedSelfTest
GridCacheReplicatedTxReadTest
GridCachePartitionedAtomicReferenceMultiNodeTest
GridCacheOnCopyFlagAtomicSelfTest

mmu...@apache.org:
GridActivateExtensionTest
IgniteChangeGlobalStateCacheTest
IgniteChangeGlobalStateTest
IgniteChangeGlobalStateServiceTest
IgniteChangeGlobalStateDataStructureTest

oignate...@gridgain.com:
CacheEntryProcessorCopySelfTest
MemoryLeaksOnRestartNodeTest
GridCacheAtomicPreloadSelfTest
WalCompactionAfterRestartTest
IgniteCacheConcurrentPutGetRemove
GridIoManagerBenchmark0

nsamelc...@gmail.com:
GridLongRunningInitNewCrdFutureDiagnosticsTest
GridCacheMultithreadedFailoverAbstractTest

alexey.goncha...@gmail.com:
GridCacheBinaryObjectsAtomicOnheapSelfTest
GridCacheBinaryObjectsAtomicNearDisabledOnheapSelfTest
GridCacheBinaryObjectsPartitionedOnheapSelfTest
GridCacheBinaryObjectsPartitionedNearDisabledOnheapSelfTest

vladis...@gmail.com:
IgnitePartitionedLockSelfTest

alexandr.bel...@xored.com:
IgniteStableBaselineCachePutAllFailoverTest
IgniteStableBaselineCacheRemoveFailoverTest

ilant...@gridgain.com:
IgniteCacheAtomicOnheapExpiryPolicyTest
IgniteCacheAtomicLocalOnheapExpiryPolicyTest
GridCacheReplicatedOnheapFullApiSelfTest
GridCacheBinaryObjectsLocalOnheapSelfTest

oignate...@users.noreply.github.com:
GridCacheTtlManagerEvictionSelfTest

ira...@apache.org:
CommonPoolStarvationCheckpointTest

alievmi...@gmail.com:
RemoveAllDeadlockTest

schugu...@gridgain.com:
FullyConnectedComponentSearcherTest

sboi...@gridgain.com:
IgniteDataStructuresNoClassOnServerTest

timonin.ma...@gmail.com:
ReliableChannelTest
ThinClientPartitionAwarenessDiscoveryTest