[jira] [Commented] (HBASE-6175) TestFSUtils flaky on hdfs getFileStatus method
[ https://issues.apache.org/jira/browse/HBASE-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13544410#comment-13544410 ] Hudson commented on HBASE-6175: --- Integrated in HBase-0.94-security-on-Hadoop-23 #10 (See [https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/10/]) HBASE-6175 TestFSUtils flaky on hdfs getFileStatus method (Revision 1422503) Result = FAILURE nkeywal : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java TestFSUtils flaky on hdfs getFileStatus method -- Key: HBASE-6175 URL: https://issues.apache.org/jira/browse/HBASE-6175 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0, 0.94.4 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.96.0, 0.94.4 Attachments: 6175.v1.patch This is a simplified version of a TestFSUtils issue: a sleep and the test works 100% of the time. No sleep and it becomes flaky. Root cause unknown. While the issue appears on the tests, the root cause could be an issue on real production system as well. {noformat} @Test public void testFSUTils() throws Exception { final String hosts[] = {host1, host2, host3, host4}; Path testFile = new Path(/test1.txt); HBaseTestingUtility htu = new HBaseTestingUtility(); try { htu.startMiniDFSCluster(hosts).waitActive(); FileSystem fs = htu.getDFSCluster().getFileSystem(); for (int i = 0; i 100; ++i) { FSDataOutputStream out = fs.create(testFile); byte[] data = new byte[1]; out.write(data, 0, 1); out.close(); // Put a sleep here to make me work //Thread.sleep(2000); FileStatus status = fs.getFileStatus(testFile); HDFSBlocksDistribution blocksDistribution = FSUtils.computeHDFSBlocksDistribution(fs, status, 0, status.getLen()); assertEquals(Wrong number of hosts distributing blocks. at iteration +i, 3, blocksDistribution.getTopHosts().size()); fs.delete(testFile, true); } } finally { htu.shutdownMiniDFSCluster(); } } {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6175) TestFSUtils flaky on hdfs getFileStatus method
[ https://issues.apache.org/jira/browse/HBASE-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13537932#comment-13537932 ] Hudson commented on HBASE-6175: --- Integrated in HBase-0.94-security #87 (See [https://builds.apache.org/job/HBase-0.94-security/87/]) HBASE-6175 TestFSUtils flaky on hdfs getFileStatus method (Revision 1422503) Result = SUCCESS nkeywal : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java TestFSUtils flaky on hdfs getFileStatus method -- Key: HBASE-6175 URL: https://issues.apache.org/jira/browse/HBASE-6175 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0, 0.94.4 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.96.0, 0.94.4 Attachments: 6175.v1.patch This is a simplified version of a TestFSUtils issue: a sleep and the test works 100% of the time. No sleep and it becomes flaky. Root cause unknown. While the issue appears on the tests, the root cause could be an issue on real production system as well. {noformat} @Test public void testFSUTils() throws Exception { final String hosts[] = {host1, host2, host3, host4}; Path testFile = new Path(/test1.txt); HBaseTestingUtility htu = new HBaseTestingUtility(); try { htu.startMiniDFSCluster(hosts).waitActive(); FileSystem fs = htu.getDFSCluster().getFileSystem(); for (int i = 0; i 100; ++i) { FSDataOutputStream out = fs.create(testFile); byte[] data = new byte[1]; out.write(data, 0, 1); out.close(); // Put a sleep here to make me work //Thread.sleep(2000); FileStatus status = fs.getFileStatus(testFile); HDFSBlocksDistribution blocksDistribution = FSUtils.computeHDFSBlocksDistribution(fs, status, 0, status.getLen()); assertEquals(Wrong number of hosts distributing blocks. at iteration +i, 3, blocksDistribution.getTopHosts().size()); fs.delete(testFile, true); } } finally { htu.shutdownMiniDFSCluster(); } } {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6175) TestFSUtils flaky on hdfs getFileStatus method
[ https://issues.apache.org/jira/browse/HBASE-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13533364#comment-13533364 ] Hudson commented on HBASE-6175: --- Integrated in HBase-0.94 #628 (See [https://builds.apache.org/job/HBase-0.94/628/]) HBASE-6175 TestFSUtils flaky on hdfs getFileStatus method (Revision 1422503) Result = FAILURE nkeywal : Files : * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestFSUtils.java TestFSUtils flaky on hdfs getFileStatus method -- Key: HBASE-6175 URL: https://issues.apache.org/jira/browse/HBASE-6175 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0, 0.94.4 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.96.0, 0.94.4 Attachments: 6175.v1.patch This is a simplified version of a TestFSUtils issue: a sleep and the test works 100% of the time. No sleep and it becomes flaky. Root cause unknown. While the issue appears on the tests, the root cause could be an issue on real production system as well. {noformat} @Test public void testFSUTils() throws Exception { final String hosts[] = {host1, host2, host3, host4}; Path testFile = new Path(/test1.txt); HBaseTestingUtility htu = new HBaseTestingUtility(); try { htu.startMiniDFSCluster(hosts).waitActive(); FileSystem fs = htu.getDFSCluster().getFileSystem(); for (int i = 0; i 100; ++i) { FSDataOutputStream out = fs.create(testFile); byte[] data = new byte[1]; out.write(data, 0, 1); out.close(); // Put a sleep here to make me work //Thread.sleep(2000); FileStatus status = fs.getFileStatus(testFile); HDFSBlocksDistribution blocksDistribution = FSUtils.computeHDFSBlocksDistribution(fs, status, 0, status.getLen()); assertEquals(Wrong number of hosts distributing blocks. at iteration +i, 3, blocksDistribution.getTopHosts().size()); fs.delete(testFile, true); } } finally { htu.shutdownMiniDFSCluster(); } } {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6175) TestFSUtils flaky on hdfs getFileStatus method
[ https://issues.apache.org/jira/browse/HBASE-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13407149#comment-13407149 ] stack commented on HBASE-6175: -- Is the loop right the way it sets ok = true at the top of the loop each time? Indents seem off too. Would suggest you document too why the loop (your findings above). Else patch LGTM. TestFSUtils flaky on hdfs getFileStatus method -- Key: HBASE-6175 URL: https://issues.apache.org/jira/browse/HBASE-6175 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.96.0 Attachments: 6175.v1.patch This is a simplified version of a TestFSUtils issue: a sleep and the test works 100% of the time. No sleep and it becomes flaky. Root cause unknown. While the issue appears on the tests, the root cause could be an issue on real production system as well. {noformat} @Test public void testFSUTils() throws Exception { final String hosts[] = {host1, host2, host3, host4}; Path testFile = new Path(/test1.txt); HBaseTestingUtility htu = new HBaseTestingUtility(); try { htu.startMiniDFSCluster(hosts).waitActive(); FileSystem fs = htu.getDFSCluster().getFileSystem(); for (int i = 0; i 100; ++i) { FSDataOutputStream out = fs.create(testFile); byte[] data = new byte[1]; out.write(data, 0, 1); out.close(); // Put a sleep here to make me work //Thread.sleep(2000); FileStatus status = fs.getFileStatus(testFile); HDFSBlocksDistribution blocksDistribution = FSUtils.computeHDFSBlocksDistribution(fs, status, 0, status.getLen()); assertEquals(Wrong number of hosts distributing blocks. at iteration +i, 3, blocksDistribution.getTopHosts().size()); fs.delete(testFile, true); } } finally { htu.shutdownMiniDFSCluster(); } } {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6175) TestFSUtils flaky on hdfs getFileStatus method
[ https://issues.apache.org/jira/browse/HBASE-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13407156#comment-13407156 ] nkeywal commented on HBASE-6175: bq. Is the loop right the way it sets ok = true at the top of the loop each time? Yes, we set it to false when the condition it not met in ok = (ok uniqueBlocksTotalWeight == weight); bq. Would suggest you document too why the loop (your findings above). The is this comment at the end of the loop. You want me to add something? // NameNode is informed asynchronously, so we may have a delay. See HBASE-6175 bq. Indents seem off too. I'm gonna check. I've already committed the patch (yesterday), I will update if necessary. TestFSUtils flaky on hdfs getFileStatus method -- Key: HBASE-6175 URL: https://issues.apache.org/jira/browse/HBASE-6175 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.96.0 Attachments: 6175.v1.patch This is a simplified version of a TestFSUtils issue: a sleep and the test works 100% of the time. No sleep and it becomes flaky. Root cause unknown. While the issue appears on the tests, the root cause could be an issue on real production system as well. {noformat} @Test public void testFSUTils() throws Exception { final String hosts[] = {host1, host2, host3, host4}; Path testFile = new Path(/test1.txt); HBaseTestingUtility htu = new HBaseTestingUtility(); try { htu.startMiniDFSCluster(hosts).waitActive(); FileSystem fs = htu.getDFSCluster().getFileSystem(); for (int i = 0; i 100; ++i) { FSDataOutputStream out = fs.create(testFile); byte[] data = new byte[1]; out.write(data, 0, 1); out.close(); // Put a sleep here to make me work //Thread.sleep(2000); FileStatus status = fs.getFileStatus(testFile); HDFSBlocksDistribution blocksDistribution = FSUtils.computeHDFSBlocksDistribution(fs, status, 0, status.getLen()); assertEquals(Wrong number of hosts distributing blocks. at iteration +i, 3, blocksDistribution.getTopHosts().size()); fs.delete(testFile, true); } } finally { htu.shutdownMiniDFSCluster(); } } {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6175) TestFSUtils flaky on hdfs getFileStatus method
[ https://issues.apache.org/jira/browse/HBASE-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13407161#comment-13407161 ] stack commented on HBASE-6175: -- Don't worry about it; none of my comments are substantial enough to require redo (I saw the comment about hbase-6175 on one of the loops only). TestFSUtils flaky on hdfs getFileStatus method -- Key: HBASE-6175 URL: https://issues.apache.org/jira/browse/HBASE-6175 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.96.0 Attachments: 6175.v1.patch This is a simplified version of a TestFSUtils issue: a sleep and the test works 100% of the time. No sleep and it becomes flaky. Root cause unknown. While the issue appears on the tests, the root cause could be an issue on real production system as well. {noformat} @Test public void testFSUTils() throws Exception { final String hosts[] = {host1, host2, host3, host4}; Path testFile = new Path(/test1.txt); HBaseTestingUtility htu = new HBaseTestingUtility(); try { htu.startMiniDFSCluster(hosts).waitActive(); FileSystem fs = htu.getDFSCluster().getFileSystem(); for (int i = 0; i 100; ++i) { FSDataOutputStream out = fs.create(testFile); byte[] data = new byte[1]; out.write(data, 0, 1); out.close(); // Put a sleep here to make me work //Thread.sleep(2000); FileStatus status = fs.getFileStatus(testFile); HDFSBlocksDistribution blocksDistribution = FSUtils.computeHDFSBlocksDistribution(fs, status, 0, status.getLen()); assertEquals(Wrong number of hosts distributing blocks. at iteration +i, 3, blocksDistribution.getTopHosts().size()); fs.delete(testFile, true); } } finally { htu.shutdownMiniDFSCluster(); } } {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6175) TestFSUtils flaky on hdfs getFileStatus method
[ https://issues.apache.org/jira/browse/HBASE-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13402990#comment-13402990 ] nkeywal commented on HBASE-6175: Here is the fix. Without 'no go' I'll commit it this week end. TestFSUtils flaky on hdfs getFileStatus method -- Key: HBASE-6175 URL: https://issues.apache.org/jira/browse/HBASE-6175 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.96.0 Attachments: 6175.v1.patch This is a simplified version of a TestFSUtils issue: a sleep and the test works 100% of the time. No sleep and it becomes flaky. Root cause unknown. While the issue appears on the tests, the root cause could be an issue on real production system as well. {noformat} @Test public void testFSUTils() throws Exception { final String hosts[] = {host1, host2, host3, host4}; Path testFile = new Path(/test1.txt); HBaseTestingUtility htu = new HBaseTestingUtility(); try { htu.startMiniDFSCluster(hosts).waitActive(); FileSystem fs = htu.getDFSCluster().getFileSystem(); for (int i = 0; i 100; ++i) { FSDataOutputStream out = fs.create(testFile); byte[] data = new byte[1]; out.write(data, 0, 1); out.close(); // Put a sleep here to make me work //Thread.sleep(2000); FileStatus status = fs.getFileStatus(testFile); HDFSBlocksDistribution blocksDistribution = FSUtils.computeHDFSBlocksDistribution(fs, status, 0, status.getLen()); assertEquals(Wrong number of hosts distributing blocks. at iteration +i, 3, blocksDistribution.getTopHosts().size()); fs.delete(testFile, true); } } finally { htu.shutdownMiniDFSCluster(); } } {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6175) TestFSUtils flaky on hdfs getFileStatus method
[ https://issues.apache.org/jira/browse/HBASE-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403014#comment-13403014 ] Hadoop QA commented on HBASE-6175: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12533802/6175.v1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 6 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestStore Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2280//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2280//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2280//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2280//console This message is automatically generated. TestFSUtils flaky on hdfs getFileStatus method -- Key: HBASE-6175 URL: https://issues.apache.org/jira/browse/HBASE-6175 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.96.0 Attachments: 6175.v1.patch This is a simplified version of a TestFSUtils issue: a sleep and the test works 100% of the time. No sleep and it becomes flaky. Root cause unknown. While the issue appears on the tests, the root cause could be an issue on real production system as well. {noformat} @Test public void testFSUTils() throws Exception { final String hosts[] = {host1, host2, host3, host4}; Path testFile = new Path(/test1.txt); HBaseTestingUtility htu = new HBaseTestingUtility(); try { htu.startMiniDFSCluster(hosts).waitActive(); FileSystem fs = htu.getDFSCluster().getFileSystem(); for (int i = 0; i 100; ++i) { FSDataOutputStream out = fs.create(testFile); byte[] data = new byte[1]; out.write(data, 0, 1); out.close(); // Put a sleep here to make me work //Thread.sleep(2000); FileStatus status = fs.getFileStatus(testFile); HDFSBlocksDistribution blocksDistribution = FSUtils.computeHDFSBlocksDistribution(fs, status, 0, status.getLen()); assertEquals(Wrong number of hosts distributing blocks. at iteration +i, 3, blocksDistribution.getTopHosts().size()); fs.delete(testFile, true); } } finally { htu.shutdownMiniDFSCluster(); } } {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6175) TestFSUtils flaky on hdfs getFileStatus method
[ https://issues.apache.org/jira/browse/HBASE-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13291625#comment-13291625 ] nkeywal commented on HBASE-6175: Todd said on hdfs mailing list: {noformat} This is the expected behavior based on the default configuration of dfs.replication.min. When you close the file, the client waits until all of the DNs have the block fully written, but the DNs report the replica to the NN asychronously. So with the default configuration, the client then only waits for 1 replica to be available before allowing the file to be closed. If you need to wait for more replicas, I would recommend polling after closing the file. {noformat} So I need to check if it's just the test or if HBase really needs to know the exact number of replica. TestFSUtils flaky on hdfs getFileStatus method -- Key: HBASE-6175 URL: https://issues.apache.org/jira/browse/HBASE-6175 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.96.0 This is a simplified version of a TestFSUtils issue: a sleep and the test works 100% of the time. No sleep and it becomes flaky. Root cause unknown. While the issue appears on the tests, the root cause could be an issue on real production system as well. {noformat} @Test public void testFSUTils() throws Exception { final String hosts[] = {host1, host2, host3, host4}; Path testFile = new Path(/test1.txt); HBaseTestingUtility htu = new HBaseTestingUtility(); try { htu.startMiniDFSCluster(hosts).waitActive(); FileSystem fs = htu.getDFSCluster().getFileSystem(); for (int i = 0; i 100; ++i) { FSDataOutputStream out = fs.create(testFile); byte[] data = new byte[1]; out.write(data, 0, 1); out.close(); // Put a sleep here to make me work //Thread.sleep(2000); FileStatus status = fs.getFileStatus(testFile); HDFSBlocksDistribution blocksDistribution = FSUtils.computeHDFSBlocksDistribution(fs, status, 0, status.getLen()); assertEquals(Wrong number of hosts distributing blocks. at iteration +i, 3, blocksDistribution.getTopHosts().size()); fs.delete(testFile, true); } } finally { htu.shutdownMiniDFSCluster(); } } {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6175) TestFSUtils flaky on hdfs getFileStatus method
[ https://issues.apache.org/jira/browse/HBASE-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13291667#comment-13291667 ] nkeywal commented on HBASE-6175: It's used mainly to estimate and in a cache to prioritize. It's not an issue if we miss one replica sometimes. So it's just a question of fixing the test itself. TestFSUtils flaky on hdfs getFileStatus method -- Key: HBASE-6175 URL: https://issues.apache.org/jira/browse/HBASE-6175 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.96.0 This is a simplified version of a TestFSUtils issue: a sleep and the test works 100% of the time. No sleep and it becomes flaky. Root cause unknown. While the issue appears on the tests, the root cause could be an issue on real production system as well. {noformat} @Test public void testFSUTils() throws Exception { final String hosts[] = {host1, host2, host3, host4}; Path testFile = new Path(/test1.txt); HBaseTestingUtility htu = new HBaseTestingUtility(); try { htu.startMiniDFSCluster(hosts).waitActive(); FileSystem fs = htu.getDFSCluster().getFileSystem(); for (int i = 0; i 100; ++i) { FSDataOutputStream out = fs.create(testFile); byte[] data = new byte[1]; out.write(data, 0, 1); out.close(); // Put a sleep here to make me work //Thread.sleep(2000); FileStatus status = fs.getFileStatus(testFile); HDFSBlocksDistribution blocksDistribution = FSUtils.computeHDFSBlocksDistribution(fs, status, 0, status.getLen()); assertEquals(Wrong number of hosts distributing blocks. at iteration +i, 3, blocksDistribution.getTopHosts().size()); fs.delete(testFile, true); } } finally { htu.shutdownMiniDFSCluster(); } } {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6175) TestFSUtils flaky on hdfs getFileStatus method
[ https://issues.apache.org/jira/browse/HBASE-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13290903#comment-13290903 ] nkeywal commented on HBASE-6175: Yes, without much success :-). You're right, I will try on hdfs list. If it doesn't work out I will push a first patch to make the test non-flaky but keep this jira open as the root cause remains. TestFSUtils flaky on hdfs getFileStatus method -- Key: HBASE-6175 URL: https://issues.apache.org/jira/browse/HBASE-6175 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.96.0 This is a simplified version of a TestFSUtils issue: a sleep and the test works 100% of the time. No sleep and it becomes flaky. Root cause unknown. While the issue appears on the tests, the root cause could be an issue on real production system as well. {noformat} @Test public void testFSUTils() throws Exception { final String hosts[] = {host1, host2, host3, host4}; Path testFile = new Path(/test1.txt); HBaseTestingUtility htu = new HBaseTestingUtility(); try { htu.startMiniDFSCluster(hosts).waitActive(); FileSystem fs = htu.getDFSCluster().getFileSystem(); for (int i = 0; i 100; ++i) { FSDataOutputStream out = fs.create(testFile); byte[] data = new byte[1]; out.write(data, 0, 1); out.close(); // Put a sleep here to make me work //Thread.sleep(2000); FileStatus status = fs.getFileStatus(testFile); HDFSBlocksDistribution blocksDistribution = FSUtils.computeHDFSBlocksDistribution(fs, status, 0, status.getLen()); assertEquals(Wrong number of hosts distributing blocks. at iteration +i, 3, blocksDistribution.getTopHosts().size()); fs.delete(testFile, true); } } finally { htu.shutdownMiniDFSCluster(); } } {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6175) TestFSUtils flaky on hdfs getFileStatus method
[ https://issues.apache.org/jira/browse/HBASE-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13291376#comment-13291376 ] nkeywal commented on HBASE-6175: The hbase-free version of the test: {noformat} package org.apache.hadoop.test; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FSDataOutputStream; import org.apache.hadoop.fs.FileStatus; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; import org.apache.hadoop.hdfs.MiniDFSCluster; import org.junit.Test; import static junit.framework.Assert.assertEquals; public class TestHDFS { @Test public void testFSUTils() throws Exception { final Configuration conf = new Configuration(); final String hosts[] = {host1, host2, host3, host4}; final byte[] data = new byte[1]; // Will fit in one block final Path testFile = new Path(/test1.txt); MiniDFSCluster dfsCluster = new MiniDFSCluster(0, conf, hosts.length, true, true, true, null, null, hosts, null); try { FileSystem fs = dfsCluster.getFileSystem(); dfsCluster.waitClusterUp(); for (int i = 0; i 200; ++i) { FSDataOutputStream out = fs.create(testFile); out.write(data, 0, 1); out.close(); // Put a sleep here to make me work //Thread.sleep(1000); FileStatus status = fs.getFileStatus(testFile); int nbHosts = fs.getFileBlockLocations(status, 0, status.getLen())[0].getHosts().length; assertEquals(1, fs.getFileBlockLocations(status, 0, status.getLen()).length); assertEquals(Wrong number of hosts distributing blocks at iteration + i, 3, nbHosts); fs.delete(testFile, true); } } finally { dfsCluster.shutdown(); } } } {noformat} TestFSUtils flaky on hdfs getFileStatus method -- Key: HBASE-6175 URL: https://issues.apache.org/jira/browse/HBASE-6175 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.96.0 This is a simplified version of a TestFSUtils issue: a sleep and the test works 100% of the time. No sleep and it becomes flaky. Root cause unknown. While the issue appears on the tests, the root cause could be an issue on real production system as well. {noformat} @Test public void testFSUTils() throws Exception { final String hosts[] = {host1, host2, host3, host4}; Path testFile = new Path(/test1.txt); HBaseTestingUtility htu = new HBaseTestingUtility(); try { htu.startMiniDFSCluster(hosts).waitActive(); FileSystem fs = htu.getDFSCluster().getFileSystem(); for (int i = 0; i 100; ++i) { FSDataOutputStream out = fs.create(testFile); byte[] data = new byte[1]; out.write(data, 0, 1); out.close(); // Put a sleep here to make me work //Thread.sleep(2000); FileStatus status = fs.getFileStatus(testFile); HDFSBlocksDistribution blocksDistribution = FSUtils.computeHDFSBlocksDistribution(fs, status, 0, status.getLen()); assertEquals(Wrong number of hosts distributing blocks. at iteration +i, 3, blocksDistribution.getTopHosts().size()); fs.delete(testFile, true); } } finally { htu.shutdownMiniDFSCluster(); } } {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6175) TestFSUtils flaky on hdfs getFileStatus method
[ https://issues.apache.org/jira/browse/HBASE-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13290510#comment-13290510 ] stack commented on HBASE-6175: -- This is the question you asked on the list Nicolas (why getFileStatus lags?)? You never got an answer? Ask on hdfs mailing list? TestFSUtils flaky on hdfs getFileStatus method -- Key: HBASE-6175 URL: https://issues.apache.org/jira/browse/HBASE-6175 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.96.0 Reporter: nkeywal Assignee: nkeywal Priority: Trivial Fix For: 0.96.0 This is a simplified version of a TestFSUtils issue: a sleep and the test works 100% of the time. No sleep and it becomes flaky. Root cause unknown. While the issue appears on the tests, the root cause could be an issue on real production system as well. {noformat} @Test public void testFSUTils() throws Exception { final String hosts[] = {host1, host2, host3, host4}; Path testFile = new Path(/test1.txt); HBaseTestingUtility htu = new HBaseTestingUtility(); try { htu.startMiniDFSCluster(hosts).waitActive(); FileSystem fs = htu.getDFSCluster().getFileSystem(); for (int i = 0; i 100; ++i) { FSDataOutputStream out = fs.create(testFile); byte[] data = new byte[1]; out.write(data, 0, 1); out.close(); // Put a sleep here to make me work //Thread.sleep(2000); FileStatus status = fs.getFileStatus(testFile); HDFSBlocksDistribution blocksDistribution = FSUtils.computeHDFSBlocksDistribution(fs, status, 0, status.getLen()); assertEquals(Wrong number of hosts distributing blocks. at iteration +i, 3, blocksDistribution.getTopHosts().size()); fs.delete(testFile, true); } } finally { htu.shutdownMiniDFSCluster(); } } {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira