Why DFSClient in HA-cluster tries to resolve HA cluster name?

2015-03-31 Thread Dmitry Simonov
Hello! Why DFSClient in HA-cluster tries to resolve HA cluster name? I have just been investigating a problem - FileSystem.get() method sometimes takes too long time to complete, about 2-3 seconds, and found that: 1. DFSClient makes call to NameNodeProxies.createProxy() 2. createProxy() method,

[jira] [Created] (HDFS-8027) Update CHANGES-HDFS-7285.txt with branch commits

2015-03-31 Thread Vinayakumar B (JIRA)
Vinayakumar B created HDFS-8027: --- Summary: Update CHANGES-HDFS-7285.txt with branch commits Key: HDFS-8027 URL: https://issues.apache.org/jira/browse/HDFS-8027 Project: Hadoop HDFS Issue Type:

[jira] [Resolved] (HDFS-8027) Erasure Coding: Update CHANGES-HDFS-7285.txt with branch commits

2015-03-31 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B resolved HDFS-8027. - Resolution: Fixed Fix Version/s: HDFS-7285 Committed to HDFS-7285 branch, Committed

Re: Why DFSClient in HA-cluster tries to resolve HA cluster name?

2015-03-31 Thread Vinayakumar B
Thats a good find Dmitry Simonov. Even in DFSAdmin, everywhere its used just to print to console. You can raise a jira and post a patch to refactor this. -Vinay On Tue, Mar 31, 2015 at 12:34 PM, Dmitry Simonov dimmobor...@gmail.com wrote: Hello! Why DFSClient in HA-cluster tries to resolve

[jira] [Created] (HDFS-8028) TestNNHandlesBlockReportPerStorage/TestNNHandlesCombinedBlockReport Failed after patched HDFS-7704

2015-03-31 Thread hongyu bi (JIRA)
hongyu bi created HDFS-8028: --- Summary: TestNNHandlesBlockReportPerStorage/TestNNHandlesCombinedBlockReport Failed after patched HDFS-7704 Key: HDFS-8028 URL: https://issues.apache.org/jira/browse/HDFS-8028

[jira] [Created] (HDFS-8037) WebHDFS: CheckAccess silently accepts certain malformed FsActions

2015-03-31 Thread Jake Low (JIRA)
Jake Low created HDFS-8037: -- Summary: WebHDFS: CheckAccess silently accepts certain malformed FsActions Key: HDFS-8037 URL: https://issues.apache.org/jira/browse/HDFS-8037 Project: Hadoop HDFS

[jira] [Created] (HDFS-8036) Use snapshot path as source when using snapshot diff report in DistCp

2015-03-31 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8036: --- Summary: Use snapshot path as source when using snapshot diff report in DistCp Key: HDFS-8036 URL: https://issues.apache.org/jira/browse/HDFS-8036 Project: Hadoop HDFS

Build failed in Jenkins: Hadoop-Hdfs-trunk-Java8 #140

2015-03-31 Thread Apache Jenkins Server
See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/140/changes Changes: [kihwal] HDFS-7742. Favoring decommissioning node for replication can cause a block to stay [aajisaka] HDFS-8002. Website refers to /trash directory. Contributd by Brahma Reddy Battula. [vinodkv] YARN-3304.

[jira] [Created] (HDFS-8029) NPE during disk usage calculation on snapshot directory, after a sub folder is deleted

2015-03-31 Thread kanaka kumar avvaru (JIRA)
kanaka kumar avvaru created HDFS-8029: - Summary: NPE during disk usage calculation on snapshot directory, after a sub folder is deleted Key: HDFS-8029 URL: https://issues.apache.org/jira/browse/HDFS-8029

Uncertainty of pre-commit builds

2015-03-31 Thread Karthik Kambatla
Hi devs, I am sure people must have noticed, but thought I would bring this up. I have noticed that our pre-commit builds could end up running the wrong set of unit tests for patches. For instance, YARN-3412 https://issues.apache.org/jira/browse/YARN-3412 changes only YARN files, but the test

Re: Uncertainty of pre-commit builds

2015-03-31 Thread Allen Wittenauer
Very likely. One of the things I noticed during HADOOP-11746 is that there is a HUGE, catastrophic race if Jenkins doesn’t setup the environment correctly or leaks variables between runs. shellcheck prints out so many messages on the current code I’m surprised it doesn’t crash. On

Hadoop-Hdfs-trunk - Build # 2081 - Failure

2015-03-31 Thread Apache Jenkins Server
See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2081/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 6518 lines...] [INFO] [INFO] ---

Build failed in Jenkins: Hadoop-Hdfs-trunk #2081

2015-03-31 Thread Apache Jenkins Server
See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2081/changes Changes: [kihwal] HDFS-7742. Favoring decommissioning node for replication can cause a block to stay [aajisaka] HDFS-8002. Website refers to /trash directory. Contributd by Brahma Reddy Battula. [vinodkv] YARN-3304. Cleaning up

Hadoop-Hdfs-trunk-Java8 - Build # 140 - Still Failing

2015-03-31 Thread Apache Jenkins Server
See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/140/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 7840 lines...] [INFO] [INFO] ---

[jira] [Created] (HDFS-8032) Erasure coding: use sentinels in BlockInfoStriped and shrink size of indices

2015-03-31 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8032: --- Summary: Erasure coding: use sentinels in BlockInfoStriped and shrink size of indices Key: HDFS-8032 URL: https://issues.apache.org/jira/browse/HDFS-8032 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-7729) Add logic to DFSOutputStream to support writing a file in striping layout

2015-03-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7729. - Resolution: Duplicate Add logic to DFSOutputStream to support writing a file in striping layout

[jira] [Resolved] (HDFS-7617) Add unit tests for editlog transactions for EC

2015-03-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7617. - Resolution: Fixed Fix Version/s: HDFS-7285 Add unit tests for editlog transactions for EC

[jira] [Created] (HDFS-8030) HDFS Erasure Coding Phase II -- EC with contiguous layout

2015-03-31 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8030: --- Summary: HDFS Erasure Coding Phase II -- EC with contiguous layout Key: HDFS-8030 URL: https://issues.apache.org/jira/browse/HDFS-8030 Project: Hadoop HDFS Issue

Re: Uncertainty of pre-commit builds

2015-03-31 Thread Karthik Kambatla
Ravi just pointed me to HADOOP-11746. Thanks a bunch for working on improving our test-patch scripts, Allen. On Tue, Mar 31, 2015 at 9:37 AM, Allen Wittenauer a...@altiscale.com wrote: Very likely. One of the things I noticed during HADOOP-11746 is that there is a HUGE, catastrophic

[jira] [Resolved] (HDFS-7783) Support appending to a striping layout file

2015-03-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7783. - Resolution: Duplicate Support appending to a striping layout file

[jira] [Created] (HDFS-8031) Follow-on work for erasure coding phase I (striping layout)

2015-03-31 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8031: --- Summary: Follow-on work for erasure coding phase I (striping layout) Key: HDFS-8031 URL: https://issues.apache.org/jira/browse/HDFS-8031 Project: Hadoop HDFS Issue

[jira] [Resolved] (HDFS-7689) Add periodic checker to find the corrupted EC blocks/files

2015-03-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7689. - Resolution: Duplicate Add periodic checker to find the corrupted EC blocks/files

[jira] [Resolved] (HDFS-7981) getStoragePolicy() regards HOT policy as EC policy

2015-03-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7981. - Resolution: Won't Fix HDFS-7839 is close to finish. getStoragePolicy() regards HOT policy as EC policy

[jira] [Created] (HDFS-8033) Erasure coding: stateful (non-positional) read from files in striped layout

2015-03-31 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8033: --- Summary: Erasure coding: stateful (non-positional) read from files in striped layout Key: HDFS-8033 URL: https://issues.apache.org/jira/browse/HDFS-8033 Project: Hadoop HDFS

[jira] [Created] (HDFS-8034) Fix TestDFSClientRetries#testDFSClientConfigurationLocateFollowingBlockInitialDelay for Windows

2015-03-31 Thread Xiaoyu Yao (JIRA)
Xiaoyu Yao created HDFS-8034: Summary: Fix TestDFSClientRetries#testDFSClientConfigurationLocateFollowingBlockInitialDelay for Windows Key: HDFS-8034 URL: https://issues.apache.org/jira/browse/HDFS-8034

[jira] [Created] (HDFS-8035) Move checking replication and get client DN to BM and DM respectively

2015-03-31 Thread Haohui Mai (JIRA)
Haohui Mai created HDFS-8035: Summary: Move checking replication and get client DN to BM and DM respectively Key: HDFS-8035 URL: https://issues.apache.org/jira/browse/HDFS-8035 Project: Hadoop HDFS