Oleg, Thanks for clarification.
On Tue, Dec 25, 2018 at 2:24 PM oignatenko <oignate...@gridgain.com> wrote: > Hi Andrey, > > As far as I know tests annotated by @Ignore can be conveniently tracked on > Teamcity but this requires such tests to be included into JUnit 4 test > suite > (org.junit.runners.Suite). > > Above requirement currently isn't met by many Ignite tests which are > included in JUnit 3 suites (junit.framework.TestSuite). This means that > test > cases annotated @Ignore simply are hidden when present in such > old-fashioned > suites. > > We are currently working on migrating Junit 3 suites to JUnit 4. If you are > interested, work breakdown and progress can be found in JIRA: > - https://issues.apache.org/jira/browse/IGNITE-10762 > > ----- > > With regards to Assume API, it seems more complicated. I have recently read > that tests that are conditionally skipped using this API are reported by > JUnit 4 test framework as passed. This is not very convenient. > > I think I have seen somewhere mentioned that proper reporting of > conditionally skipped tests can be achieved only with JUnit 5. But I didn't > dig deeper there because we haven't yet decided whether it is worth it for > Ignite to move to JUnit 5 > (https://issues.apache.org/jira/browse/IGNITE-10180). > > Given above, I believe that the proper way to mute tests is currently with > @Ignore and Assume API. It's not 100% ideal but overall looks better than > the old one. > > regards, Oleg > > > Andrew Mashenkov wrote > > Dmitry, > > > > As far as I understand, @IgniteIgnore and fail("") works in similar way > > and > > requires test to be muted on TC manually. > > And we have to unmute these tests on TC right after fix will be merged to > > master, isn't it? > > > > @Ignore and Assume.that works different way comparing to previous two. > > Junit just exclude such tests from run. > > So, there is no need to mute\unmute them on TC at all, but... > > Does TC treat such tests as muted ones and we can see them on "muted > > tests" > > page? > > This may be useful to track tests state if someone forget to remove > > @ignore > > while fixing related ticket. > > > > > > On Mon, Dec 17, 2018 at 5:54 PM Anton Vinogradov < > > > av@ > > > > wrote: > > > >> >> In this case, we will not need TeamCity mutes anymore. > >> Sounds great! > >> > >> On Mon, Dec 17, 2018 at 5:23 PM Dmitrii Ryabov < > > > somefireone@ > > > > > >> wrote: > >> > >> > Dmitriy, > >> > > >> > with ignore annotations we will need to check forgotten tests > >> > manually. Thats bad, but we will save some machine time in common > >> > runs. > >> > пн, 17 дек. 2018 г. в 17:06, Dmitriy Pavlov < > > > dpavlov@ > > > >: > >> > > > >> > > Investigation parameters: Can leave as empty > >> > > Unmute: Manual (don't use auto because of flaky tests) > >> > > Text: https://issues.apache.org/jira/browse/IGNITE-NNNNN > >> > > > >> > > But anyway, I hope in the nearest future best way to mute test will > >> be > >> to > >> > > ignore it using: > >> > > @IgniteIgnore > >> > > or @Ignore > >> > > or Assume.that() > >> > > > >> > > In this case, we will not need TeamCity mutes anymore. > >> > > > >> > > I'm not sure which way of ignoring will be best. > >> > > > >> > > пн, 17 дек. 2018 г. в 17:02, Anton Vinogradov < > > > av@ > > > >: > >> > > > >> > > > Dmitrii, > >> > > > > >> > > > My question was mostly about how to mute test properly. > >> > > > > >> > > > > >> > > > On Mon, Dec 17, 2018 at 4:27 PM Dmitrii Ryabov < > >> > > > somefireone@ > > >> > >> > > > wrote: > >> > > > > >> > > > > Anton, > >> > > > > > >> > > > > In the "Mute" column you can find link to the test on TeamCity. > >> On > >> > > > > this page you can see test details like run history and current > >> > mutes. > >> > > > > Here you should be able to unmute existing mutes. > >> > > > > пн, 17 дек. 2018 г. в 16:21, Anton Vinogradov < > > > av@ > > > >: > >> > > > > > > >> > > > > > Great feature! > >> > > > > > > >> > > > > > Could you please explain how to have a deal with it? > >> > > > > > How should I mite/unmute tests for now? > >> > > > > > > >> > > > > > On Fri, Dec 14, 2018 at 6:50 PM Dmitriy Pavlov < > >> > > > dpavlov@ > > >> > > > >> > > > > wrote: > >> > > > > > > >> > > > > > > We have IgniteIgnore, so we can use it. It seems behavior is > >> the > >> > > > same. > >> > > > > We > >> > > > > > > need just to understand in which case we can get issue ID > >> from > >> TC > >> > > > REST > >> > > > > (for > >> > > > > > > the Bot). > >> > > > > > > > >> > > > > > > пт, 14 дек. 2018 г. в 17:51, Vyacheslav Daradur < > >> > > > > daradurvs@ > > >> > > > >: > >> > > > > > > > >> > > > > > > > Dmitry Pavlov, > >> > > > > > > > > >> > > > > > > > As far as I know, we are able to use following > construction > >> to > >> > > > ignore > >> > > > > > > test: > >> > > > > > > > Assume.assumeTrue("link-to-issue", false); > >> > > > > > > > > >> > > > > > > > On Fri, Dec 14, 2018 at 5:44 PM Dmitriy Pavlov < > >> > > > > dpavlov@ > > >> > > > > > >> > > > > > > wrote: > >> > > > > > > > > > >> > > > > > > > > Thank you, Dmitrii, > >> > > > > > > > > > >> > > > > > > > > I've started unmuting tests starting from yesterday. > >> > > > > > > > > > >> > > > > > > > > Now muted test count was decreased from 3540 to 3408 > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > >> > > >> > https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8&tab=mutedProblems&branch_IgniteTests24Java8=%3Cdefault%3E > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > Probably we could automate this activity by auto-unmute, > >> but > >> > > > > hopefully, > >> > > > > > > > > JUnit4 & @Ignore opportunity will come earlier. > >> > > > > > > > > > >> > > > > > > > > Sincerely, > >> > > > > > > > > Dmitriy Pavlov > >> > > > > > > > > > >> > > > > > > > > пт, 14 дек. 2018 г. в 17:24, Dmitrii Ryabov < > >> > > > > > > somefireone@ > > >> > > > > >: > >> > > > > > > > > > >> > > > > > > > > > Hello, Igniters! > >> > > > > > > > > > > >> > > > > > > > > > Our Bot got new functionality - it shows tests muted > on > >> > > > TeamCity > >> > > > > and > >> > > > > > > > their > >> > > > > > > > > > ticket status [1]. > >> > > > > > > > > > > >> > > > > > > > > > So, we can see forgotten mutes for resolved tickets > >> now. > >> > > > > > > > > > > >> > > > > > > > > > Igniters with rights to unmute tests on TC - please, > >> check > >> > this > >> > > > > page > >> > > > > > > > > > periodically for forgotten mutes. > >> > > > > > > > > > > >> > > > > > > > > > [1] https://mtcga.gridgain.com/mutes.html > >> > > > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > -- > >> > > > > > > > Best Regards, Vyacheslav D. > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > >> > > >> > > > > > > -- > > Best regards, > > Andrey V. Mashenkov > > > > > > -- > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > -- Best regards, Andrey V. Mashenkov
