steveloughran commented on a change in pull request #1115: HADOOP-16207 testMR
failures
URL: https://github.com/apache/hadoop/pull/1115#discussion_r330939535
##########
File path:
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/AbstractYarnClusterITest.java
##########
@@ -61,45 +69,70 @@
* If two subclasses of this class are instantiated in the same JVM, in order,
* they are guaranteed to be isolated.
*
- * History: this is a superclass extracted from
- * {@link AbstractITCommitMRJob} while adding support for testing terasorting.
- *
*/
public abstract class AbstractYarnClusterITest extends AbstractCommitITest {
private static final Logger LOG =
LoggerFactory.getLogger(AbstractYarnClusterITest.class);
- private static final int TEST_FILE_COUNT = 2;
- private static final int SCALE_TEST_FILE_COUNT = 50;
+ private static final int TEST_FILE_COUNT = 1;
+ private static final int SCALE_TEST_FILE_COUNT = 2;
- public static final int SCALE_TEST_KEYS = 1000;
+ public static final int SCALE_TEST_KEYS = 100;
public static final int BASE_TEST_KEYS = 10;
private boolean scaleTest;
- private boolean uniqueFilenames = false;
+ /**
+ * The static cluster binding with the lifecycle of this test; served
+ * through instance-level methods for sharing across methods in the
+ * suite.
+ */
+ @SuppressWarnings("StaticNonFinalField")
+ private static ClusterBinding clusterBinding;
Review comment:
now we have parameterized tests there will never be >1 CommitMRJob
test running in parallel. That was a problem we had before, where you could
have multiple ITest*CommitMRJob test runs spawning so many processes that your
dev box comes to a halt if you do a many threaded run. With these settings the
delegation token test runs can overlap with the CommitMRJob sequence -i plan to
follow up this patch with parameterization there too.
As all the tests have opted not to use a mini HDFS cluster we avoid the
extra overhead of those processes too
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
