ayushtkn commented on a change in pull request #2861: URL: https://github.com/apache/hadoop/pull/2861#discussion_r611748559
########## File path: hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/contract/AbstractContractDistCpTest.java ########## @@ -628,8 +629,15 @@ public void testDistCpWithIterator() throws Exception { GenericTestUtils .createFiles(remoteFS, source, getDepth(), getWidth(), getWidth()); + GenericTestUtils.LogCapturer log = Review comment: Thanx @steveloughran, do you suggest that we should have two options like -useiteratorforsource and -useiteratorfortarget. Do you think in that case we would be able to save out on memory? since the target list is being build as part of CopyCommitter, so even if one takes the normal path, We would get OOM, just `when` will differ? Regarding the log stuff, That was the only thing I could think of, to confirm if iterator was used. And during migration to Log4J2, Will moving to something like this will also be of no help: https://github.com/apache/hive/blob/master/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java#L285 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org