[jira] [Updated] (HDFS-4937) ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom()

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-4937: - Fix Version/s: 2.8.0 > ReplicationMonitor can infinite-loop in > BlockPlacementPolicyDefault#chooseRandom(

[jira] [Updated] (HDFS-8219) setStoragePolicy with folder behavior is different after cluster restart

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8219: - Fix Version/s: 2.8.0 > setStoragePolicy with folder behavior is different after cluster restart > -

[jira] [Updated] (HDFS-8245) Standby namenode doesn't process DELETED_BLOCK if the add block request is in edit log.

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8245: - Fix Version/s: 2.8.0 > Standby namenode doesn't process DELETED_BLOCK if the add block request is in > edi

[jira] [Commented] (HDFS-11296) Maintenance state expiry should be an epoch time and not jvm monotonic

2017-01-05 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803118#comment-15803118 ] Manoj Govindassamy commented on HDFS-11296: --- Couldn't find the failed test detai

[jira] [Updated] (HDFS-8270) create() always retried with hardcoded timeout when file already exists with open lease

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8270: - Fix Version/s: 2.8.0 > create() always retried with hardcoded timeout when file already exists with > open

[jira] [Updated] (HDFS-8305) HDFS INotify: the destination field of RenameOp should always end with the file name

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8305: - Fix Version/s: 2.8.0 > HDFS INotify: the destination field of RenameOp should always end with the > file n

[jira] [Updated] (HDFS-7314) When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-7314: - Fix Version/s: 2.8.0 > When the DFSClient lease cannot be renewed, abort open-for-write files rather > tha

[jira] [Commented] (HDFS-11296) Maintenance state expiry should be an epoch time and not jvm monotonic

2017-01-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803104#comment-15803104 ] Hadoop QA commented on HDFS-11296: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

[jira] [Commented] (HDFS-11121) Add assertions to BlockInfo#addStorage to protect from breaking reportedBlock-blockGroup mapping

2017-01-05 Thread Takanobu Asanuma (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803091#comment-15803091 ] Takanobu Asanuma commented on HDFS-11121: - Hi, [~jojochuang]. Happy New Year! If

[jira] [Updated] (HDFS-9880) TestDatanodeRegistration fails occasionally

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9880: - Fix Version/s: 2.8.0 > TestDatanodeRegistration fails occasionally > --

[jira] [Updated] (HDFS-9958) BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages.

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9958: - Fix Version/s: 2.8.0 > BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed > storag

[jira] [Updated] (HDFS-10239) Fsshell mv fails if port usage doesn't match in src and destination paths

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10239: -- Fix Version/s: 2.8.0 > Fsshell mv fails if port usage doesn't match in src and destination paths > -

[jira] [Updated] (HDFS-10372) Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10372: -- Fix Version/s: 2.8.0 > Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume > ---

[jira] [Updated] (HDFS-10261) TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10261: -- Fix Version/s: 2.8.0 > TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports > ---

[jira] [Updated] (HDFS-10275) TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10275: -- Fix Version/s: 2.8.0 > TestDataNodeMetrics failing intermittently due to TotalWriteTime counted > incor

[jira] [Updated] (HDFS-10264) Logging improvements in FSImageFormatProtobuf.Saver

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10264: -- Fix Version/s: 2.8.0 > Logging improvements in FSImageFormatProtobuf.Saver > ---

[jira] [Updated] (HDFS-10182) Hedged read might overwrite user's buf

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10182: -- Fix Version/s: 2.8.0 > Hedged read might overwrite user's buf > -- >

[jira] [Updated] (HDFS-5215) dfs.datanode.du.reserved is not considered while computing available space

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5215: - Fix Version/s: 2.8.0 > dfs.datanode.du.reserved is not considered while computing available space > ---

[jira] [Updated] (HDFS-4946) Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-4946: - Fix Version/s: 2.8.0 > Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable > --

[jira] [Updated] (HDFS-7163) WebHdfsFileSystem should retry reads according to the configured retry policy.

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-7163: - Fix Version/s: 2.8.0 > WebHdfsFileSystem should retry reads according to the configured retry policy. > ---

[jira] [Updated] (HDFS-8806) Inconsistent metrics: number of missing blocks with replication factor 1 not properly cleared

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8806: - Fix Version/s: 2.8.0 > Inconsistent metrics: number of missing blocks with replication factor 1 not > prop

[jira] [Updated] (HDFS-8846) Add a unit test for INotify functionality across a layout version upgrade

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8846: - Fix Version/s: 2.8.0 > Add a unit test for INotify functionality across a layout version upgrade >

[jira] [Updated] (HDFS-9917) IBR accumulate more objects when SNN was down for sometime.

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9917: - Fix Version/s: 2.8.0 > IBR accumulate more objects when SNN was down for sometime. > --

[jira] [Updated] (HDFS-8863) The remaining space check in BlockPlacementPolicyDefault is flawed

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8863: - Fix Version/s: 2.8.0 > The remaining space check in BlockPlacementPolicyDefault is flawed > ---

[jira] [Updated] (HDFS-8995) Flaw in registration bookeeping can make DN die on reconnect

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8995: - Fix Version/s: 2.8.0 > Flaw in registration bookeeping can make DN die on reconnect > -

[jira] [Updated] (HDFS-9724) Degraded performance in WebHDFS listing as it does not reuse ObjectMapper

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9724: - Fix Version/s: 2.8.0 > Degraded performance in WebHDFS listing as it does not reuse ObjectMapper >

[jira] [Updated] (HDFS-9106) Transfer failure during pipeline recovery causes permanent write failures

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9106: - Fix Version/s: 2.8.0 > Transfer failure during pipeline recovery causes permanent write failures >

[jira] [Updated] (HDFS-9273) ACLs on root directory may be lost after NN restart

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9273: - Fix Version/s: 2.8.0 > ACLs on root directory may be lost after NN restart > --

[jira] [Updated] (HDFS-9413) getContentSummary() on standby should throw StandbyException

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9413: - Fix Version/s: 2.8.0 > getContentSummary() on standby should throw StandbyException > -

[jira] [Updated] (HDFS-9470) Encryption zone on root not loaded from fsimage after NN restart

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9470: - Fix Version/s: 2.8.0 > Encryption zone on root not loaded from fsimage after NN restart > -

[jira] [Updated] (HDFS-9178) Slow datanode I/O can cause a wrong node to be marked bad

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9178: - Fix Version/s: 2.8.0 > Slow datanode I/O can cause a wrong node to be marked bad >

[jira] [Updated] (HDFS-6945) BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-6945: - Fix Version/s: 2.8.0 > BlockManager should remove a block from excessReplicateMap and decrement > ExcessBl

[jira] [Updated] (HDFS-6300) Prevent multiple balancers from running simultaneously

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-6300: - Fix Version/s: 2.8.0 > Prevent multiple balancers from running simultaneously > ---

[jira] [Updated] (HDFS-9289) Make DataStreamer#block thread safe and verify genStamp in commitBlock

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9289: - Fix Version/s: 2.8.0 > Make DataStreamer#block thread safe and verify genStamp in commitBlock > ---

[jira] [Updated] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8578: - Fix Version/s: 2.8.0 > On upgrade, Datanode should process all storage/data dirs in parallel >

[jira] [Updated] (HDFS-8656) Preserve compatibility of ClientProtocol#rollingUpgrade after finalization

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8656: - Fix Version/s: 2.8.0 > Preserve compatibility of ClientProtocol#rollingUpgrade after finalization > ---

[jira] [Updated] (HDFS-8676) Delayed rolling upgrade finalization can cause heartbeat expiration and write failures

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8676: - Fix Version/s: 2.8.0 > Delayed rolling upgrade finalization can cause heartbeat expiration and write > fai

[jira] [Updated] (HDFS-8722) Optimize datanode writes for small writes and flushes

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8722: - Fix Version/s: 2.8.0 > Optimize datanode writes for small writes and flushes >

[jira] [Updated] (HDFS-8767) RawLocalFileSystem.listStatus() returns null for UNIX pipefile

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8767: - Fix Version/s: 2.8.0 > RawLocalFileSystem.listStatus() returns null for UNIX pipefile > ---

[jira] [Updated] (HDFS-10178) Permanent write failures can happen if pipeline recoveries occur for the first packet

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10178: -- Fix Version/s: 2.8.0 > Permanent write failures can happen if pipeline recoveries occur for the > first

[jira] [Commented] (HDFS-11282) Document the missing metrics of DataNode Volume IO operations

2017-01-05 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803052#comment-15803052 ] Arpit Agarwal commented on HDFS-11282: -- Thanks [~linyiqun]. The patch looks good and

[jira] [Updated] (HDFS-10335) Mover$Processor#chooseTarget() always chooses the first matching target storage group

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10335: -- Fix Version/s: 2.8.0 > Mover$Processor#chooseTarget() always chooses the first matching target > storag

[jira] [Updated] (HDFS-10271) Extra bytes are getting released from reservedSpace for append

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10271: -- Fix Version/s: 2.8.0 > Extra bytes are getting released from reservedSpace for append >

[jira] [Updated] (HDFS-8431) hdfs crypto class not found in Windows

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8431: - Fix Version/s: 2.8.0 > hdfs crypto class not found in Windows > -- > >

[jira] [Updated] (HDFS-7609) Avoid retry cache collision when Standby NameNode loading edits

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-7609: - Fix Version/s: 2.8.0 > Avoid retry cache collision when Standby NameNode loading edits > --

[jira] [Updated] (HDFS-8213) DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8213: - Fix Version/s: 2.8.0 > DFSClient should use hdfs.client.htrace HTrace configuration prefix rather > than h

[jira] [Updated] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8480: - Fix Version/s: 2.8.0 > Fix performance and timeout issues in HDFS-7929 by using hard-links to > preserve o

[jira] [Updated] (HDFS-7934) Update RollingUpgrade rollback documentation: should use bootstrapstandby for standby NN

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-7934: - Fix Version/s: 2.8.0 > Update RollingUpgrade rollback documentation: should use bootstrapstandby for > sta

[jira] [Updated] (HDFS-7916) 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-7916: - Fix Version/s: 2.8.0 > 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for > infinite

[jira] [Updated] (HDFS-7894) Rolling upgrade readiness is not updated in jmx until query command is issued.

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-7894: - Fix Version/s: 2.8.0 > Rolling upgrade readiness is not updated in jmx until query command is issued. > ---

[jira] [Updated] (HDFS-4660) Block corruption can happen during pipeline recovery

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-4660: - Fix Version/s: 2.8.0 > Block corruption can happen during pipeline recovery > -

[jira] [Commented] (HDFS-9666) Enable hdfs-client to read even remote SSD/RAM prior to local disk replica to improve random read

2017-01-05 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803039#comment-15803039 ] Arpit Agarwal commented on HDFS-9666: - Unset the Target Version for now due to lack of

[jira] [Updated] (HDFS-8486) DN startup may cause severe data loss

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8486: - Fix Version/s: 2.8.0 > DN startup may cause severe data loss > - > >

[jira] [Updated] (HDFS-9666) Enable hdfs-client to read even remote SSD/RAM prior to local disk replica to improve random read

2017-01-05 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-9666: Target Version/s: (was: 2.9.0) > Enable hdfs-client to read even remote SSD/RAM prior to local disk

[jira] [Updated] (HDFS-9666) Enable hdfs-client to read even remote SSD/RAM prior to local disk replica to improve random read

2017-01-05 Thread Ravi Prakash (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash updated HDFS-9666: --- Target Version/s: 2.9.0 (was: 2.7.4) > Enable hdfs-client to read even remote SSD/RAM prior to local d

[jira] [Commented] (HDFS-9483) Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS.

2017-01-05 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803025#comment-15803025 ] Hudson commented on HDFS-9483: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #1107

[jira] [Commented] (HDFS-11209) SNN can't checkpoint when rolling upgrade is not finalized

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803018#comment-15803018 ] Junping Du commented on HDFS-11209: --- Hi [~xyao], do we plan to fix this issue in short t

[jira] [Commented] (HDFS-11194) Maintain aggregated peer performance metrics on NameNode

2017-01-05 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15803009#comment-15803009 ] Arpit Agarwal commented on HDFS-11194: -- HDFS-11194-03-04.delta has the changes from t

[jira] [Updated] (HDFS-11194) Maintain aggregated peer performance metrics on NameNode

2017-01-05 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-11194: - Attachment: HDFS-11194.04.patch Thanks for the review [~xyao]. The v4 patch addresses your feedbac

[jira] [Updated] (HDFS-11194) Maintain aggregated peer performance metrics on NameNode

2017-01-05 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-11194: - Attachment: HDFS-11194-03-04.delta > Maintain aggregated peer performance metrics on NameNode > --

[jira] [Updated] (HDFS-9483) Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS.

2017-01-05 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-9483: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved

[jira] [Updated] (HDFS-9483) Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS.

2017-01-05 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-9483: Fix Version/s: 3.0.0-alpha2 > Documentation does not cover use of "swebhdfs" as URL scheme for SSL-se

[jira] [Updated] (HDFS-11296) Maintenance state expiry should be an epoch time and not jvm monotonic

2017-01-05 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HDFS-11296: -- Status: Patch Available (was: Open) > Maintenance state expiry should be an epoch time

[jira] [Updated] (HDFS-11296) Maintenance state expiry should be an epoch time and not jvm monotonic

2017-01-05 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HDFS-11296: -- Description: Currently it is possible to configure an expiry time in milliseconds for a

[jira] [Updated] (HDFS-11296) Maintenance state expiry should be an epoch time and not jvm monotonic

2017-01-05 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HDFS-11296: -- Attachment: HDFS-11296.01.patch Attached v01 patch to address the following 1. Made {{Da

[jira] [Commented] (HDFS-11292) log lastWrittenTxId etc info in logSyncAll

2017-01-05 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802812#comment-15802812 ] Wei-Chiu Chuang commented on HDFS-11292: Hello [~yzhangal] thanks for providing th

[jira] [Updated] (HDFS-8891) HDFS concat should keep srcs order

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8891: - Fix Version/s: 2.8.0 > HDFS concat should keep srcs order > -- > >

[jira] [Updated] (HDFS-9445) Datanode may deadlock while handling a bad volume

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9445: - Fix Version/s: 2.8.0 > Datanode may deadlock while handling a bad volume >

[jira] [Updated] (HDFS-9426) Rollingupgrade finalization is not backward compatible

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9426: - Fix Version/s: 2.8.0 > Rollingupgrade finalization is not backward compatible > ---

[jira] [Updated] (HDFS-9290) DFSClient#callAppend() is not backward compatible for slightly older NameNodes

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9290: - Fix Version/s: 2.8.0 > DFSClient#callAppend() is not backward compatible for slightly older NameNodes > ---

[jira] [Updated] (HDFS-9294) DFSClient deadlock when close file and failed to renew lease

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9294: - Fix Version/s: 2.8.0 > DFSClient deadlock when close file and failed to renew lease >

[jira] [Updated] (HDFS-8273) FSNamesystem#Delete() should not call logSync() when holding the lock

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8273: - Fix Version/s: 2.8.0 > FSNamesystem#Delete() should not call logSync() when holding the lock >

[jira] [Updated] (HDFS-8070) Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8070: - Fix Version/s: 2.8.0 > Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode > --

[jira] [Updated] (HDFS-8596) TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8596: - Fix Version/s: 2.8.0 > TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect > set

[jira] [Updated] (HDFS-9220) Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9220: - Fix Version/s: 2.8.0 > Reading small file (< 512 bytes) that is open for append fails due to > incorrect c

[jira] [Updated] (HDFS-8572) DN always uses HTTP/localhost@REALM principals in SPNEGO

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8572: - Fix Version/s: 2.8.0 > DN always uses HTTP/localhost@REALM principals in SPNEGO > -

[jira] [Updated] (HDFS-8681) BlockScanner is incorrectly disabled by default

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8681: - Fix Version/s: 2.8.0 > BlockScanner is incorrectly disabled by default > --

[jira] [Updated] (HDFS-8626) Reserved RBW space is not released if creation of RBW File fails

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8626: - Fix Version/s: 2.8.0 > Reserved RBW space is not released if creation of RBW File fails > -

[jira] [Updated] (HDFS-8127) NameNode Failover during HA upgrade can cause DataNode to finalize upgrade

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8127: - Fix Version/s: 2.8.0 > NameNode Failover during HA upgrade can cause DataNode to finalize upgrade > ---

[jira] [Updated] (HDFS-8163) Using monotonicNow for block report scheduling causes test failures on recently restarted systems

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8163: - Fix Version/s: 2.8.0 > Using monotonicNow for block report scheduling causes test failures on > recently r

[jira] [Updated] (HDFS-8179) DFSClient#getServerDefaults returns null within 1 hour of system start

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8179: - Fix Version/s: 2.8.0 > DFSClient#getServerDefaults returns null within 1 hour of system start > ---

[jira] [Updated] (HDFS-8451) DFSClient probe for encryption testing interprets empty URI property for "enabled"

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8451: - Fix Version/s: 2.8.0 > DFSClient probe for encryption testing interprets empty URI property for > "enabled

[jira] [Updated] (HDFS-8226) Non-HA rollback compatibility broken

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8226: - Fix Version/s: 2.8.0 > Non-HA rollback compatibility broken > > >

[jira] [Updated] (HDFS-8269) getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8269: - Fix Version/s: 2.8.0 > getBlockLocations() does not resolve the .reserved path and generates > incorrect e

[jira] [Commented] (HDFS-9083) Replication violates block placement policy.

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802687#comment-15802687 ] Junping Du commented on HDFS-9083: -- Got it. Thanks for the confirmation! > Replication vi

[jira] [Commented] (HDFS-9083) Replication violates block placement policy.

2017-01-05 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802662#comment-15802662 ] Kihwal Lee commented on HDFS-9083: -- [~djp], that's correct. > Replication violates block

[jira] [Resolved] (HDFS-9896) WebHDFS API may return invalid JSON

2017-01-05 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-9896. --- Resolution: Duplicate > WebHDFS API may return invalid JSON > --- > >

[jira] [Reopened] (HDFS-9896) WebHDFS API may return invalid JSON

2017-01-05 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HDFS-9896: --- > WebHDFS API may return invalid JSON > --- > > Key: HDFS-9

[jira] [Updated] (HDFS-9896) WebHDFS API may return invalid JSON

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9896: - Assignee: (was: John Zhuge) > WebHDFS API may return invalid JSON > ---

[jira] [Updated] (HDFS-9896) WebHDFS API may return invalid JSON

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9896: - Fix Version/s: (was: 2.7.3) > WebHDFS API may return invalid JSON > ---

[jira] [Updated] (HDFS-11160) VolumeScanner reports write-in-progress replicas as corrupt incorrectly

2017-01-05 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-11160: --- Fix Version/s: 2.8.0 > VolumeScanner reports write-in-progress replicas as corrupt incorrectly

[jira] [Commented] (HDFS-11160) VolumeScanner reports write-in-progress replicas as corrupt incorrectly

2017-01-05 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802579#comment-15802579 ] Wei-Chiu Chuang commented on HDFS-11160: My bad. I had it cherry picked in my loca

[jira] [Commented] (HDFS-11072) Add ability to unset and change directory EC policy

2017-01-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802526#comment-15802526 ] Hadoop QA commented on HDFS-11072: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

[jira] [Commented] (HDFS-9083) Replication violates block placement policy.

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802521#comment-15802521 ] Junping Du commented on HDFS-9083: -- Hi [~shahrs87], [~kihwal] and [~sjlee0], I assume the

[jira] [Updated] (HDFS-10214) Checkpoint Can not be done by StandbyNameNode.Because checkpoint may cause DataNode blockReport.blockReceivedAndDeleted.heartbeat rpc timeout when the object num > 100000

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10214: -- Fix Version/s: (was: 2.7.2) > Checkpoint Can not be done by StandbyNameNode.Because checkpoint may c

[jira] [Commented] (HDFS-7967) Reduce the performance impact of the balancer

2017-01-05 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802511#comment-15802511 ] Kihwal Lee commented on HDFS-7967: -- The 2.8 patch works fine with jdk8. The unit tests ran

[jira] [Updated] (HDFS-11296) Maintenance state expiry should be an epoch time and not jvm monotonic

2017-01-05 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HDFS-11296: -- Summary: Maintenance state expiry should be an epoch time and not jvm monotonic (was: M

[jira] [Created] (HDFS-11296) Maintenance state expiry should be a epoch time and not jvm monotonic

2017-01-05 Thread Manoj Govindassamy (JIRA)
Manoj Govindassamy created HDFS-11296: - Summary: Maintenance state expiry should be a epoch time and not jvm monotonic Key: HDFS-11296 URL: https://issues.apache.org/jira/browse/HDFS-11296 Project

[jira] [Commented] (HDFS-11293) FsDatasetImpl throws ReplicaAlreadyExistsException in a wrong situation

2017-01-05 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802413#comment-15802413 ] Uma Maheswara Rao G commented on HDFS-11293: [~yuanbo], {quote} A[DISK], not

[jira] [Commented] (HDFS-11273) Move TransferFsImage#doGetUrl function to a Util class

2017-01-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802396#comment-15802396 ] Hadoop QA commented on HDFS-11273: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote

<    1   2   3   >