Doubt on ReplicaInPipeline.stopWriter

We are using hadoop 2.6.0 and recently we found that sometimes datanode can hang(not HDFS-7489, we had already patched it). This one was caused by pipeline recovery of blk_1150368526_76663248 datanode log 2015-04-28 17:51:55,297 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving

[jira] [Created] (HDFS-8295) Add MODIFY and REMOVE ECSchema editlog operations

Xinwei Qin created HDFS-8295: - Summary: Add MODIFY and REMOVE ECSchema editlog operations Key: HDFS-8295 URL: https://issues.apache.org/jira/browse/HDFS-8295 Project: Hadoop HDFS Issue Type:

[jira] [Created] (HDFS-8296) BlockManager.getUnderReplicatedBlocksCount() is not giving correct count if namenode in safe mode.

surendra singh lilhore created HDFS-8296: Summary: BlockManager.getUnderReplicatedBlocksCount() is not giving correct count if namenode in safe mode. Key: HDFS-8296 URL:

[jira] [Created] (HDFS-8297) Ability to online trigger data dir rescan for blocks

Hari Sekhon created HDFS-8297: - Summary: Ability to online trigger data dir rescan for blocks Key: HDFS-8297 URL: https://issues.apache.org/jira/browse/HDFS-8297 Project: Hadoop HDFS Issue Type:

Hadoop-Hdfs-trunk-Java8 - Build # 170 - Still Failing

See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/170/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 5183 lines...] [INFO] [INFO] ---

[jira] [Created] (HDFS-8298) HA: NameNode should not shut down completely without quorum, doesn't recover from temporary failures

Hari Sekhon created HDFS-8298: - Summary: HA: NameNode should not shut down completely without quorum, doesn't recover from temporary failures Key: HDFS-8298 URL: https://issues.apache.org/jira/browse/HDFS-8298

[jira] [Created] (HDFS-8299) HDFS reporting missing blocks when they are actually present due to read-only filesystem

Hari Sekhon created HDFS-8299: - Summary: HDFS reporting missing blocks when they are actually present due to read-only filesystem Key: HDFS-8299 URL: https://issues.apache.org/jira/browse/HDFS-8299

Build failed in Jenkins: Hadoop-Hdfs-trunk #2111

See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2111/changes Changes: [jing9] HDFS-8283. DataStreamer cleanup and some minor improvement. Contributed by Tsz Wo Nicholas Sze. [wheat9] HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when

Hadoop-Hdfs-trunk - Build # 2111 - Still Failing

See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2111/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 5182 lines...] [INFO] [INFO] ---

[jira] [Created] (HDFS-8302) Consolidate log purging logic in QJM and FJM

Zhe Zhang created HDFS-8302: --- Summary: Consolidate log purging logic in QJM and FJM Key: HDFS-8302 URL: https://issues.apache.org/jira/browse/HDFS-8302 Project: Hadoop HDFS Issue Type: Improvement

Block level compression

Hi, Is there any way I can get the size of each compressed (Gzip) block without actually compressing it. For example, I have 200mb uncompressed data in HDFS and the block size is 64 MB. I want to get the size of each of the 4 compressed blocks. The result might look like, the first block is 15

[jira] [Created] (HDFS-8301) Create a separate document for tag storage type explicitly.

Xiaoyu Yao created HDFS-8301: Summary: Create a separate document for tag storage type explicitly. Key: HDFS-8301 URL: https://issues.apache.org/jira/browse/HDFS-8301 Project: Hadoop HDFS

[jira] [Created] (HDFS-8300) Fix unit test failures and findbugs warning caused by HDFS-8283

Jing Zhao created HDFS-8300: --- Summary: Fix unit test failures and findbugs warning caused by HDFS-8283 Key: HDFS-8300 URL: https://issues.apache.org/jira/browse/HDFS-8300 Project: Hadoop HDFS

[jira] [Created] (HDFS-8304) Separate out shared log purging methods for QJM (Paxos directory) and FJM

Zhe Zhang created HDFS-8304: --- Summary: Separate out shared log purging methods for QJM (Paxos directory) and FJM Key: HDFS-8304 URL: https://issues.apache.org/jira/browse/HDFS-8304 Project: Hadoop HDFS

[jira] [Created] (HDFS-8303) QJM should purge old logs in the current directory through FJM

Zhe Zhang created HDFS-8303: --- Summary: QJM should purge old logs in the current directory through FJM Key: HDFS-8303 URL: https://issues.apache.org/jira/browse/HDFS-8303 Project: Hadoop HDFS

[jira] [Created] (HDFS-8306) Generate ACL and Xattr outputs in OIV XML outputs

Lei (Eddy) Xu created HDFS-8306: --- Summary: Generate ACL and Xattr outputs in OIV XML outputs Key: HDFS-8306 URL: https://issues.apache.org/jira/browse/HDFS-8306 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-8307) Spurious DNS Queries from hdfs shell

Anu Engineer created HDFS-8307: -- Summary: Spurious DNS Queries from hdfs shell Key: HDFS-8307 URL: https://issues.apache.org/jira/browse/HDFS-8307 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-8305) HDFS INotify: the destination argument to RenameOp should always end with the file name

Colin Patrick McCabe created HDFS-8305: -- Summary: HDFS INotify: the destination argument to RenameOp should always end with the file name Key: HDFS-8305 URL: https://issues.apache.org/jira/browse/HDFS-8305

Re: Planning Hadoop 2.6.1 release

HDFS candidates for back-porting to Hadoop 2.6.1. The first two were requested in [1]. HADOOP-11674. oneByteBuf in CryptoInputStream and CryptoOutputStream should be non static HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization HDFS-7009. Active NN and

[jira] [Resolved] (HDFS-8308) Erasure Coding: NameNode may get blocked in waitForLoadingFSImage() when loading editlog

[ https://issues.apache.org/jira/browse/HDFS-8308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8308. - Resolution: Fixed Fix Version/s: HDFS-7285 Thanks for the review, Nicholas! I've committed this.

Update: checkstyle, shellcheck, and whitespace tests

Hey gang. With the commit of HADOOP-11866, the output for checkstyle and whitespace has been greatly enhanced to actually be useful now. Instead of getting weird output, you'll see the file, the line number and (in the case of checkstyle) error. Note that line numbers are

[jira] [Created] (HDFS-8308) Erasure Coding: NameNode may get blocked in waitForLoadingFSImage() when loading editlog

Jing Zhao created HDFS-8308: --- Summary: Erasure Coding: NameNode may get blocked in waitForLoadingFSImage() when loading editlog Key: HDFS-8308 URL: https://issues.apache.org/jira/browse/HDFS-8308 Project:

Re: Planning Hadoop 2.6.1 release

Thank you, Arpit. In addition, I suggest we include the following: HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification pipe is full HADOOP-11604. Prevent ConcurrentModificationException while closing domain sockets during shutdown of DomainSocketWatcher thread. HADOOP-11648.