adoroszlai commented on a change in pull request #3148: URL: https://github.com/apache/ozone/pull/3148#discussion_r819922590
########## File path: hadoop-hdds/test-utils/src/main/java/org/apache/ozone/test/tag/Flaky.java ########## @@ -0,0 +1,40 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * <p> + * http://www.apache.org/licenses/LICENSE-2.0 + * <p> + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.ozone.test.tag; + +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +import org.junit.jupiter.api.Tag; + +/** + * Annotation to mark test classes or methods with some intermittent failures. + * These are handled separately from the normal tests. (Not required to pass, + * may be repeated automatically, etc.) + */ +@Target({ ElementType.TYPE, ElementType.METHOD }) +@Retention(RetentionPolicy.RUNTIME) +@Tag("flaky") +public @interface Flaky { Review comment: Thanks @ghuangups for taking a look. Initially I proposed to use text files listing tests for each category. That's what we discussed offline. In practice, however, I found it has a problem: using both include/exclude file and include/exclude list in POM (which we have for CI) is not really compatible. Test list in text file also has two disadvantages compared to annotation-based tagging: * easier to make typo in test class/method name or pattern, * any renaming or other refactoring needs to be applied to the list as well, which is easy to forget. Both of these are non-issues with tags. Regarding the set of categories being defined: * Stable is everything that's not tagged, i.e. it is an implicit category. * Probation does not need to be tagged. One can explicitly select any subset of the tests for execution via `-Dtest`, include file or Maven profile. The CI-specific profiles are not active in this case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
