[jira] [Commented] (HDFS-13676) TestEditLogRace fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514120#comment-16514120 ] Zuoming Zhang commented on HDFS-13676: -- [~elgoiri] Here is the issue link: https://issues.apache.org/jira/browse/HDFS-6440 Here is the github link: [https://github.com/apache/hadoop/commit/49dfad942970459297f72632ed8dfd353e0c86de] >From the issue description, it's checked-in in 3.0.0. > TestEditLogRace fails on Windows > > > Key: HDFS-13676 > URL: https://issues.apache.org/jira/browse/HDFS-13676 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.1.0, 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Attachments: HDFS-13676-branch-2.000.patch, HDFS-13676.000.patch, > HDFS-13676.001.patch, TestEditLogRace-Report-branch-2.001.txt, > TestEditLogRace-Report.000.txt > > > _TestEditLogRace_ fails on Windows > > Problem: > When try to call _FSImage.saveFSImageInAllDirs_, there's actually no > directories existing. This is because the _getConf()_ function doesn't > specify creating any directories. > > Fix: > Remove the comment for the two lines that config directories to be created. > > Concern: > Not for sure why it was commented in change > [https://github.com/apache/hadoop/commit/3cb7ae11a839c01b8be629774874c1873f51b747] > And it should also fail for Linux I guess. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13676) TestEditLogRace fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16513270#comment-16513270 ] Zuoming Zhang commented on HDFS-13676: -- [~elgoiri] It doesn't seem like these errors are related to my change, just double check with it. > TestEditLogRace fails on Windows > > > Key: HDFS-13676 > URL: https://issues.apache.org/jira/browse/HDFS-13676 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.1.0, 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Attachments: HDFS-13676-branch-2.000.patch, HDFS-13676.000.patch, > HDFS-13676.001.patch, TestEditLogRace-Report-branch-2.001.txt, > TestEditLogRace-Report.000.txt > > > _TestEditLogRace_ fails on Windows > > Problem: > When try to call _FSImage.saveFSImageInAllDirs_, there's actually no > directories existing. This is because the _getConf()_ function doesn't > specify creating any directories. > > Fix: > Remove the comment for the two lines that config directories to be created. > > Concern: > Not for sure why it was commented in change > [https://github.com/apache/hadoop/commit/3cb7ae11a839c01b8be629774874c1873f51b747] > And it should also fail for Linux I guess. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13676) TestEditLogRace fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zuoming Zhang updated HDFS-13676: - Attachment: HDFS-13676.001.patch > TestEditLogRace fails on Windows > > > Key: HDFS-13676 > URL: https://issues.apache.org/jira/browse/HDFS-13676 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.1.0, 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Attachments: HDFS-13676-branch-2.000.patch, HDFS-13676.000.patch, > HDFS-13676.001.patch, TestEditLogRace-Report-branch-2.001.txt, > TestEditLogRace-Report.000.txt > > > _TestEditLogRace_ fails on Windows > > Problem: > When try to call _FSImage.saveFSImageInAllDirs_, there's actually no > directories existing. This is because the _getConf()_ function doesn't > specify creating any directories. > > Fix: > Remove the comment for the two lines that config directories to be created. > > Concern: > Not for sure why it was commented in change > [https://github.com/apache/hadoop/commit/3cb7ae11a839c01b8be629774874c1873f51b747] > And it should also fail for Linux I guess. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13676) TestEditLogRace fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zuoming Zhang updated HDFS-13676: - Affects Version/s: 3.1.0 > TestEditLogRace fails on Windows > > > Key: HDFS-13676 > URL: https://issues.apache.org/jira/browse/HDFS-13676 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.1.0, 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Attachments: HDFS-13676-branch-2.000.patch, HDFS-13676.000.patch, > TestEditLogRace-Report-branch-2.001.txt, TestEditLogRace-Report.000.txt > > > _TestEditLogRace_ fails on Windows > > Problem: > When try to call _FSImage.saveFSImageInAllDirs_, there's actually no > directories existing. This is because the _getConf()_ function doesn't > specify creating any directories. > > Fix: > Remove the comment for the two lines that config directories to be created. > > Concern: > Not for sure why it was commented in change > [https://github.com/apache/hadoop/commit/3cb7ae11a839c01b8be629774874c1873f51b747] > And it should also fail for Linux I guess. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13676) TestEditLogRace fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zuoming Zhang updated HDFS-13676: - Target Version/s: 2.9.1, 3.1.0 (was: 2.9.1) > TestEditLogRace fails on Windows > > > Key: HDFS-13676 > URL: https://issues.apache.org/jira/browse/HDFS-13676 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.1.0, 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Attachments: HDFS-13676-branch-2.000.patch, HDFS-13676.000.patch, > TestEditLogRace-Report-branch-2.001.txt, TestEditLogRace-Report.000.txt > > > _TestEditLogRace_ fails on Windows > > Problem: > When try to call _FSImage.saveFSImageInAllDirs_, there's actually no > directories existing. This is because the _getConf()_ function doesn't > specify creating any directories. > > Fix: > Remove the comment for the two lines that config directories to be created. > > Concern: > Not for sure why it was commented in change > [https://github.com/apache/hadoop/commit/3cb7ae11a839c01b8be629774874c1873f51b747] > And it should also fail for Linux I guess. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13676) TestEditLogRace fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zuoming Zhang updated HDFS-13676: - Fix Version/s: (was: 2.9.1) > TestEditLogRace fails on Windows > > > Key: HDFS-13676 > URL: https://issues.apache.org/jira/browse/HDFS-13676 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.1.0, 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Attachments: HDFS-13676-branch-2.000.patch, HDFS-13676.000.patch, > TestEditLogRace-Report-branch-2.001.txt, TestEditLogRace-Report.000.txt > > > _TestEditLogRace_ fails on Windows > > Problem: > When try to call _FSImage.saveFSImageInAllDirs_, there's actually no > directories existing. This is because the _getConf()_ function doesn't > specify creating any directories. > > Fix: > Remove the comment for the two lines that config directories to be created. > > Concern: > Not for sure why it was commented in change > [https://github.com/apache/hadoop/commit/3cb7ae11a839c01b8be629774874c1873f51b747] > And it should also fail for Linux I guess. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13676) TestEditLogRace fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zuoming Zhang updated HDFS-13676: - Attachment: TestEditLogRace-Report-branch-2.001.txt > TestEditLogRace fails on Windows > > > Key: HDFS-13676 > URL: https://issues.apache.org/jira/browse/HDFS-13676 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Fix For: 2.9.1 > > Attachments: HDFS-13676-branch-2.000.patch, HDFS-13676.000.patch, > TestEditLogRace-Report-branch-2.001.txt, TestEditLogRace-Report.000.txt > > > _TestEditLogRace_ fails on Windows > > Problem: > When try to call _FSImage.saveFSImageInAllDirs_, there's actually no > directories existing. This is because the _getConf()_ function doesn't > specify creating any directories. > > Fix: > Remove the comment for the two lines that config directories to be created. > > Concern: > Not for sure why it was commented in change > [https://github.com/apache/hadoop/commit/3cb7ae11a839c01b8be629774874c1873f51b747] > And it should also fail for Linux I guess. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13676) TestEditLogRace fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zuoming Zhang updated HDFS-13676: - Fix Version/s: (was: 3.1.0) > TestEditLogRace fails on Windows > > > Key: HDFS-13676 > URL: https://issues.apache.org/jira/browse/HDFS-13676 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Fix For: 2.9.1 > > Attachments: HDFS-13676-branch-2.000.patch, HDFS-13676.000.patch, > TestEditLogRace-Report.000.txt > > > _TestEditLogRace_ fails on Windows > > Problem: > When try to call _FSImage.saveFSImageInAllDirs_, there's actually no > directories existing. This is because the _getConf()_ function doesn't > specify creating any directories. > > Fix: > Remove the comment for the two lines that config directories to be created. > > Concern: > Not for sure why it was commented in change > [https://github.com/apache/hadoop/commit/3cb7ae11a839c01b8be629774874c1873f51b747] > And it should also fail for Linux I guess. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13676) TestEditLogRace fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zuoming Zhang updated HDFS-13676: - Target Version/s: 2.9.1 (was: 3.1.0, 2.9.1) > TestEditLogRace fails on Windows > > > Key: HDFS-13676 > URL: https://issues.apache.org/jira/browse/HDFS-13676 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Fix For: 2.9.1 > > Attachments: HDFS-13676-branch-2.000.patch, HDFS-13676.000.patch, > TestEditLogRace-Report.000.txt > > > _TestEditLogRace_ fails on Windows > > Problem: > When try to call _FSImage.saveFSImageInAllDirs_, there's actually no > directories existing. This is because the _getConf()_ function doesn't > specify creating any directories. > > Fix: > Remove the comment for the two lines that config directories to be created. > > Concern: > Not for sure why it was commented in change > [https://github.com/apache/hadoop/commit/3cb7ae11a839c01b8be629774874c1873f51b747] > And it should also fail for Linux I guess. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13676) TestEditLogRace fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zuoming Zhang updated HDFS-13676: - Affects Version/s: (was: 3.1.0) > TestEditLogRace fails on Windows > > > Key: HDFS-13676 > URL: https://issues.apache.org/jira/browse/HDFS-13676 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Fix For: 3.1.0, 2.9.1 > > Attachments: HDFS-13676-branch-2.000.patch, HDFS-13676.000.patch, > TestEditLogRace-Report.000.txt > > > _TestEditLogRace_ fails on Windows > > Problem: > When try to call _FSImage.saveFSImageInAllDirs_, there's actually no > directories existing. This is because the _getConf()_ function doesn't > specify creating any directories. > > Fix: > Remove the comment for the two lines that config directories to be created. > > Concern: > Not for sure why it was commented in change > [https://github.com/apache/hadoop/commit/3cb7ae11a839c01b8be629774874c1873f51b747] > And it should also fail for Linux I guess. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13676) TestEditLogRace fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zuoming Zhang updated HDFS-13676: - Attachment: HDFS-13676-branch-2.000.patch > TestEditLogRace fails on Windows > > > Key: HDFS-13676 > URL: https://issues.apache.org/jira/browse/HDFS-13676 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.1.0, 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Fix For: 3.1.0, 2.9.1 > > Attachments: HDFS-13676-branch-2.000.patch, HDFS-13676.000.patch, > TestEditLogRace-Report.000.txt > > > _TestEditLogRace_ fails on Windows > > Problem: > When try to call _FSImage.saveFSImageInAllDirs_, there's actually no > directories existing. This is because the _getConf()_ function doesn't > specify creating any directories. > > Fix: > Remove the comment for the two lines that config directories to be created. > > Concern: > Not for sure why it was commented in change > [https://github.com/apache/hadoop/commit/3cb7ae11a839c01b8be629774874c1873f51b747] > And it should also fail for Linux I guess. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13673) TestNameNodeMetrics fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16512744#comment-16512744 ] Zuoming Zhang commented on HDFS-13673: -- So you mean testCorruptBlock is another issue? Just double confirm to make sure it's not caused by my patch. [~elgoiri] > TestNameNodeMetrics fails on Windows > > > Key: HDFS-13673 > URL: https://issues.apache.org/jira/browse/HDFS-13673 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.1.0, 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Fix For: 3.1.0, 2.9.1 > > Attachments: HDFS-13673.000.patch, HDFS-13673.001.patch, > TestNameNodeMetrics-testVolumeFailures-Report.000.txt, > TestNameNodeMetrics-testVolumeFailures-Report.001.txt > > > _TestNameNodeMetrics_ fails on Windows > > Problem: > This is because in _testVolumeFailures_, it tries to call > _DataNodeTestUtils.injectDataDirFailure_ on a volume folder. What > _injectDataDirFailure_does is actually modifying the folder name from > _volume_name_ to _volume_name_._origin_ and create a new file named as > _volume_name_. Inside the folder, it has two things: 1. a directory named as > "_current_", 2. a file named as "_in_use.lock_". Windows behaves different > from Linux when renaming the parent folder of a locked file. Windows prevent > you from renaming while Linux allows. > Fix: > So in order to inject data failure on to the volume. Instead of renaming the > volume folder itself. Rename the folder inside it which doesn't hold a lock. > Since the folder inside the volume is "_current_". Then we only need to > inject data failure to _volume_name/current_. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13673) TestNameNodeMetrics fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16512720#comment-16512720 ] Zuoming Zhang commented on HDFS-13673: -- Interesting, for the error in TestNameNodeMetrics.testCorruptBlock, it doesn't seem to be related to my change, since the test has first passed in my previous patch, and the new patch is no difference in logic. I think the TestNameNodeMetrics.testCorruptBlock itself has random failure since it sometimes fails on my local machine. > TestNameNodeMetrics fails on Windows > > > Key: HDFS-13673 > URL: https://issues.apache.org/jira/browse/HDFS-13673 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.1.0, 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Fix For: 3.1.0, 2.9.1 > > Attachments: HDFS-13673.000.patch, HDFS-13673.001.patch, > TestNameNodeMetrics-testVolumeFailures-Report.000.txt, > TestNameNodeMetrics-testVolumeFailures-Report.001.txt > > > _TestNameNodeMetrics_ fails on Windows > > Problem: > This is because in _testVolumeFailures_, it tries to call > _DataNodeTestUtils.injectDataDirFailure_ on a volume folder. What > _injectDataDirFailure_does is actually modifying the folder name from > _volume_name_ to _volume_name_._origin_ and create a new file named as > _volume_name_. Inside the folder, it has two things: 1. a directory named as > "_current_", 2. a file named as "_in_use.lock_". Windows behaves different > from Linux when renaming the parent folder of a locked file. Windows prevent > you from renaming while Linux allows. > Fix: > So in order to inject data failure on to the volume. Instead of renaming the > volume folder itself. Rename the folder inside it which doesn't hold a lock. > Since the folder inside the volume is "_current_". Then we only need to > inject data failure to _volume_name/current_. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13673) TestNameNodeMetrics fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511864#comment-16511864 ] Zuoming Zhang commented on HDFS-13673: -- [~elgoiri] Uploaded > TestNameNodeMetrics fails on Windows > > > Key: HDFS-13673 > URL: https://issues.apache.org/jira/browse/HDFS-13673 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.1.0, 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Fix For: 3.1.0, 2.9.1 > > Attachments: HDFS-13673.000.patch, HDFS-13673.001.patch, > TestNameNodeMetrics-testVolumeFailures-Report.000.txt, > TestNameNodeMetrics-testVolumeFailures-Report.001.txt > > > _TestNameNodeMetrics_ fails on Windows > > Problem: > This is because in _testVolumeFailures_, it tries to call > _DataNodeTestUtils.injectDataDirFailure_ on a volume folder. What > _injectDataDirFailure_does is actually modifying the folder name from > _volume_name_ to _volume_name_._origin_ and create a new file named as > _volume_name_. Inside the folder, it has two things: 1. a directory named as > "_current_", 2. a file named as "_in_use.lock_". Windows behaves different > from Linux when renaming the parent folder of a locked file. Windows prevent > you from renaming while Linux allows. > Fix: > So in order to inject data failure on to the volume. Instead of renaming the > volume folder itself. Rename the folder inside it which doesn't hold a lock. > Since the folder inside the volume is "_current_". Then we only need to > inject data failure to _volume_name/current_. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13673) TestNameNodeMetrics fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zuoming Zhang updated HDFS-13673: - Attachment: HDFS-13673.001.patch > TestNameNodeMetrics fails on Windows > > > Key: HDFS-13673 > URL: https://issues.apache.org/jira/browse/HDFS-13673 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.1.0, 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Fix For: 3.1.0, 2.9.1 > > Attachments: HDFS-13673.000.patch, HDFS-13673.001.patch, > TestNameNodeMetrics-testVolumeFailures-Report.000.txt, > TestNameNodeMetrics-testVolumeFailures-Report.001.txt > > > _TestNameNodeMetrics_ fails on Windows > > Problem: > This is because in _testVolumeFailures_, it tries to call > _DataNodeTestUtils.injectDataDirFailure_ on a volume folder. What > _injectDataDirFailure_does is actually modifying the folder name from > _volume_name_ to _volume_name_._origin_ and create a new file named as > _volume_name_. Inside the folder, it has two things: 1. a directory named as > "_current_", 2. a file named as "_in_use.lock_". Windows behaves different > from Linux when renaming the parent folder of a locked file. Windows prevent > you from renaming while Linux allows. > Fix: > So in order to inject data failure on to the volume. Instead of renaming the > volume folder itself. Rename the folder inside it which doesn't hold a lock. > Since the folder inside the volume is "_current_". Then we only need to > inject data failure to _volume_name/current_. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13676) TestEditLogRace fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zuoming Zhang updated HDFS-13676: - Attachment: HDFS-13676.000.patch Target Version/s: 2.9.1, 3.1.0 (was: 3.1.0, 2.9.1) Status: Patch Available (was: Open) > TestEditLogRace fails on Windows > > > Key: HDFS-13676 > URL: https://issues.apache.org/jira/browse/HDFS-13676 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 2.9.1, 3.1.0 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Fix For: 3.1.0, 2.9.1 > > Attachments: HDFS-13676.000.patch, TestEditLogRace-Report.000.txt > > > _TestEditLogRace_ fails on Windows > > Problem: > When try to call _FSImage.saveFSImageInAllDirs_, there's actually no > directories existing. This is because the _getConf()_ function doesn't > specify creating any directories. > > Fix: > Remove the comment for the two lines that config directories to be created. > > Concern: > Not for sure why it was commented in change > [https://github.com/apache/hadoop/commit/3cb7ae11a839c01b8be629774874c1873f51b747] > And it should also fail for Linux I guess. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13676) TestEditLogRace fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zuoming Zhang updated HDFS-13676: - Attachment: TestEditLogRace-Report.000.txt > TestEditLogRace fails on Windows > > > Key: HDFS-13676 > URL: https://issues.apache.org/jira/browse/HDFS-13676 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.1.0, 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Fix For: 3.1.0, 2.9.1 > > Attachments: HDFS-13676.000.patch, TestEditLogRace-Report.000.txt > > > _TestEditLogRace_ fails on Windows > > Problem: > When try to call _FSImage.saveFSImageInAllDirs_, there's actually no > directories existing. This is because the _getConf()_ function doesn't > specify creating any directories. > > Fix: > Remove the comment for the two lines that config directories to be created. > > Concern: > Not for sure why it was commented in change > [https://github.com/apache/hadoop/commit/3cb7ae11a839c01b8be629774874c1873f51b747] > And it should also fail for Linux I guess. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13673) TestNameNodeMetrics fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zuoming Zhang updated HDFS-13673: - Attachment: TestNameNodeMetrics-testVolumeFailures-Report.001.txt > TestNameNodeMetrics fails on Windows > > > Key: HDFS-13673 > URL: https://issues.apache.org/jira/browse/HDFS-13673 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.1.0, 2.9.1 >Reporter: Zuoming Zhang >Assignee: Zuoming Zhang >Priority: Minor > Labels: Windows > Fix For: 3.1.0, 2.9.1 > > Attachments: HDFS-13673.000.patch, HDFS-13673.001.patch, > TestNameNodeMetrics-testVolumeFailures-Report.000.txt, > TestNameNodeMetrics-testVolumeFailures-Report.001.txt > > > _TestNameNodeMetrics_ fails on Windows > > Problem: > This is because in _testVolumeFailures_, it tries to call > _DataNodeTestUtils.injectDataDirFailure_ on a volume folder. What > _injectDataDirFailure_does is actually modifying the folder name from > _volume_name_ to _volume_name_._origin_ and create a new file named as > _volume_name_. Inside the folder, it has two things: 1. a directory named as > "_current_", 2. a file named as "_in_use.lock_". Windows behaves different > from Linux when renaming the parent folder of a locked file. Windows prevent > you from renaming while Linux allows. > Fix: > So in order to inject data failure on to the volume. Instead of renaming the > volume folder itself. Rename the folder inside it which doesn't hold a lock. > Since the folder inside the volume is "_current_". Then we only need to > inject data failure to _volume_name/current_. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13676) TestEditLogRace fails on Windows
Zuoming Zhang created HDFS-13676: Summary: TestEditLogRace fails on Windows Key: HDFS-13676 URL: https://issues.apache.org/jira/browse/HDFS-13676 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 2.9.1, 3.1.0 Reporter: Zuoming Zhang Fix For: 3.1.0, 2.9.1 _TestEditLogRace_ fails on Windows Problem: When try to call _FSImage.saveFSImageInAllDirs_, there's actually no directories existing. This is because the _getConf()_ function doesn't specify creating any directories. Fix: Remove the comment for the two lines that config directories to be created. Concern: Not for sure why it was commented in change [https://github.com/apache/hadoop/commit/3cb7ae11a839c01b8be629774874c1873f51b747] And it should also fail for Linux I guess. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13673) TestNameNodeMetrics fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511843#comment-16511843 ] Zuoming Zhang commented on HDFS-13673: -- [~elgoiri] This error doesn't seem to be related to mine, any idea? > TestNameNodeMetrics fails on Windows > > > Key: HDFS-13673 > URL: https://issues.apache.org/jira/browse/HDFS-13673 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.1.0, 2.9.1 >Reporter: Zuoming Zhang >Priority: Minor > Labels: Windows > Fix For: 3.1.0, 2.9.1 > > Attachments: HDFS-13673.000.patch, > TestNameNodeMetrics-testVolumeFailures-Report.000.txt > > > _TestNameNodeMetrics_ fails on Windows > > Problem: > This is because in _testVolumeFailures_, it tries to call > _DataNodeTestUtils.injectDataDirFailure_ on a volume folder. What > _injectDataDirFailure_does is actually modifying the folder name from > _volume_name_ to _volume_name_._origin_ and create a new file named as > _volume_name_. Inside the folder, it has two things: 1. a directory named as > "_current_", 2. a file named as "_in_use.lock_". Windows behaves different > from Linux when renaming the parent folder of a locked file. Windows prevent > you from renaming while Linux allows. > Fix: > So in order to inject data failure on to the volume. Instead of renaming the > volume folder itself. Rename the folder inside it which doesn't hold a lock. > Since the folder inside the volume is "_current_". Then we only need to > inject data failure to _volume_name/current_. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13673) TestNameNodeMetrics fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511682#comment-16511682 ] Zuoming Zhang commented on HDFS-13673: -- Thanks [~elgoiri] Answers to your questions: * Nope. Actually for other places that are calling _DataNodeTestUtils.injectDataDirFailure_, they are not calling on the volume folder itself. So not affected by _in_use.lock_. I've also checked with all other tests that call the _injectDataDirFailure_, and the tests are not failing. * What do you mean to extract the variable? I think this is only used once in this file? > TestNameNodeMetrics fails on Windows > > > Key: HDFS-13673 > URL: https://issues.apache.org/jira/browse/HDFS-13673 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.1.0, 2.9.1 >Reporter: Zuoming Zhang >Priority: Minor > Labels: Windows > Fix For: 3.1.0, 2.9.1 > > Attachments: HDFS-13673.000.patch, > TestNameNodeMetrics-testVolumeFailures-Report.000.txt > > > _TestNameNodeMetrics_ fails on Windows > > Problem: > This is because in _testVolumeFailures_, it tries to call > _DataNodeTestUtils.injectDataDirFailure_ on a volume folder. What > _injectDataDirFailure_does is actually modifying the folder name from > _volume_name_ to _volume_name_._origin_ and create a new file named as > _volume_name_. Inside the folder, it has two things: 1. a directory named as > "_current_", 2. a file named as "_in_use.lock_". Windows behaves different > from Linux when renaming the parent folder of a locked file. Windows prevent > you from renaming while Linux allows. > Fix: > So in order to inject data failure on to the volume. Instead of renaming the > volume folder itself. Rename the folder inside it which doesn't hold a lock. > Since the folder inside the volume is "_current_". Then we only need to > inject data failure to _volume_name/current_. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13673) TestNameNodeMetrics fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zuoming Zhang updated HDFS-13673: - Attachment: HDFS-13673.000.patch Target Version/s: 2.9.1, 3.1.0 (was: 3.1.0, 2.9.1) Status: Patch Available (was: Open) > TestNameNodeMetrics fails on Windows > > > Key: HDFS-13673 > URL: https://issues.apache.org/jira/browse/HDFS-13673 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 2.9.1, 3.1.0 >Reporter: Zuoming Zhang >Priority: Minor > Labels: Windows > Fix For: 3.1.0, 2.9.1 > > Attachments: HDFS-13673.000.patch, > TestNameNodeMetrics-testVolumeFailures-Report.000.txt > > > _TestNameNodeMetrics_ fails on Windows > > Problem: > This is because in _testVolumeFailures_, it tries to call > _DataNodeTestUtils.injectDataDirFailure_ on a volume folder. What > _injectDataDirFailure_does is actually modifying the folder name from > _volume_name_ to _volume_name_._origin_ and create a new file named as > _volume_name_. Inside the folder, it has two things: 1. a directory named as > "_current_", 2. a file named as "_in_use.lock_". Windows behaves different > from Linux when renaming the parent folder of a locked file. Windows prevent > you from renaming while Linux allows. > Fix: > So in order to inject data failure on to the volume. Instead of renaming the > volume folder itself. Rename the folder inside it which doesn't hold a lock. > Since the folder inside the volume is "_current_". Then we only need to > inject data failure to _volume_name/current_. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13673) TestNameNodeMetrics fails on Windows
[ https://issues.apache.org/jira/browse/HDFS-13673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zuoming Zhang updated HDFS-13673: - Attachment: TestNameNodeMetrics-testVolumeFailures-Report.000.txt > TestNameNodeMetrics fails on Windows > > > Key: HDFS-13673 > URL: https://issues.apache.org/jira/browse/HDFS-13673 > Project: Hadoop HDFS > Issue Type: Bug > Components: test >Affects Versions: 3.1.0, 2.9.1 >Reporter: Zuoming Zhang >Priority: Minor > Labels: Windows > Fix For: 3.1.0, 2.9.1 > > Attachments: HDFS-13673.000.patch, > TestNameNodeMetrics-testVolumeFailures-Report.000.txt > > > _TestNameNodeMetrics_ fails on Windows > > Problem: > This is because in _testVolumeFailures_, it tries to call > _DataNodeTestUtils.injectDataDirFailure_ on a volume folder. What > _injectDataDirFailure_does is actually modifying the folder name from > _volume_name_ to _volume_name_._origin_ and create a new file named as > _volume_name_. Inside the folder, it has two things: 1. a directory named as > "_current_", 2. a file named as "_in_use.lock_". Windows behaves different > from Linux when renaming the parent folder of a locked file. Windows prevent > you from renaming while Linux allows. > Fix: > So in order to inject data failure on to the volume. Instead of renaming the > volume folder itself. Rename the folder inside it which doesn't hold a lock. > Since the folder inside the volume is "_current_". Then we only need to > inject data failure to _volume_name/current_. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13673) TestNameNodeMetrics fails on Windows
Zuoming Zhang created HDFS-13673: Summary: TestNameNodeMetrics fails on Windows Key: HDFS-13673 URL: https://issues.apache.org/jira/browse/HDFS-13673 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 2.9.1, 3.1.0 Reporter: Zuoming Zhang Fix For: 3.1.0, 2.9.1 _TestNameNodeMetrics_ fails on Windows Problem: This is because in _testVolumeFailures_, it tries to call _DataNodeTestUtils.injectDataDirFailure_ on a volume folder. What _injectDataDirFailure_does is actually modifying the folder name from _volume_name_ to _volume_name_._origin_ and create a new file named as _volume_name_. Inside the folder, it has two things: 1. a directory named as "_current_", 2. a file named as "_in_use.lock_". Windows behaves different from Linux when renaming the parent folder of a locked file. Windows prevent you from renaming while Linux allows. Fix: So in order to inject data failure on to the volume. Instead of renaming the volume folder itself. Rename the folder inside it which doesn't hold a lock. Since the folder inside the volume is "_current_". Then we only need to inject data failure to _volume_name/current_. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
