[jira] [Updated] (HDFS-10178) Permanent write failures can happen if pipeline recoveries occur for the first packet

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10178: -- Fix Version/s: 2.8.0 > Permanent write failures can happen if pipeline recoveries occur for the >

[jira] [Updated] (HDFS-10335) Mover$Processor#chooseTarget() always chooses the first matching target storage group

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10335: -- Fix Version/s: 2.8.0 > Mover$Processor#chooseTarget() always chooses the first matching target >

[jira] [Updated] (HDFS-10271) Extra bytes are getting released from reservedSpace for append

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10271: -- Fix Version/s: 2.8.0 > Extra bytes are getting released from reservedSpace for append >

[jira] [Updated] (HDFS-8431) hdfs crypto class not found in Windows

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8431: - Fix Version/s: 2.8.0 > hdfs crypto class not found in Windows > -- > >

[jira] [Updated] (HDFS-7609) Avoid retry cache collision when Standby NameNode loading edits

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-7609: - Fix Version/s: 2.8.0 > Avoid retry cache collision when Standby NameNode loading edits >

[jira] [Updated] (HDFS-8213) DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8213: - Fix Version/s: 2.8.0 > DFSClient should use hdfs.client.htrace HTrace configuration prefix rather > than

[jira] [Updated] (HDFS-8480) Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8480: - Fix Version/s: 2.8.0 > Fix performance and timeout issues in HDFS-7929 by using hard-links to > preserve

[jira] [Updated] (HDFS-7934) Update RollingUpgrade rollback documentation: should use bootstrapstandby for standby NN

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-7934: - Fix Version/s: 2.8.0 > Update RollingUpgrade rollback documentation: should use bootstrapstandby for >

[jira] [Updated] (HDFS-7916) 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-7916: - Fix Version/s: 2.8.0 > 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for >

[jira] [Updated] (HDFS-7894) Rolling upgrade readiness is not updated in jmx until query command is issued.

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-7894: - Fix Version/s: 2.8.0 > Rolling upgrade readiness is not updated in jmx until query command is issued. >

[jira] [Updated] (HDFS-4660) Block corruption can happen during pipeline recovery

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-4660: - Fix Version/s: 2.8.0 > Block corruption can happen during pipeline recovery >

[jira] [Updated] (HDFS-8486) DN startup may cause severe data loss

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8486: - Fix Version/s: 2.8.0 > DN startup may cause severe data loss > - > >

[jira] [Commented] (HDFS-11209) SNN can't checkpoint when rolling upgrade is not finalized

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15803018#comment-15803018 ] Junping Du commented on HDFS-11209: --- Hi [~xyao], do we plan to fix this issue in short term? If not,

[jira] [Updated] (HDFS-8891) HDFS concat should keep srcs order

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8891: - Fix Version/s: 2.8.0 > HDFS concat should keep srcs order > -- > >

[jira] [Updated] (HDFS-9445) Datanode may deadlock while handling a bad volume

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9445: - Fix Version/s: 2.8.0 > Datanode may deadlock while handling a bad volume >

[jira] [Updated] (HDFS-9426) Rollingupgrade finalization is not backward compatible

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9426: - Fix Version/s: 2.8.0 > Rollingupgrade finalization is not backward compatible >

[jira] [Updated] (HDFS-9290) DFSClient#callAppend() is not backward compatible for slightly older NameNodes

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9290: - Fix Version/s: 2.8.0 > DFSClient#callAppend() is not backward compatible for slightly older NameNodes >

[jira] [Updated] (HDFS-9294) DFSClient deadlock when close file and failed to renew lease

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9294: - Fix Version/s: 2.8.0 > DFSClient deadlock when close file and failed to renew lease >

[jira] [Updated] (HDFS-8273) FSNamesystem#Delete() should not call logSync() when holding the lock

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8273: - Fix Version/s: 2.8.0 > FSNamesystem#Delete() should not call logSync() when holding the lock >

[jira] [Updated] (HDFS-8070) Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8070: - Fix Version/s: 2.8.0 > Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode >

[jira] [Updated] (HDFS-8596) TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8596: - Fix Version/s: 2.8.0 > TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect >

[jira] [Updated] (HDFS-9220) Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9220: - Fix Version/s: 2.8.0 > Reading small file (< 512 bytes) that is open for append fails due to > incorrect

[jira] [Updated] (HDFS-8572) DN always uses HTTP/localhost@REALM principals in SPNEGO

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8572: - Fix Version/s: 2.8.0 > DN always uses HTTP/localhost@REALM principals in SPNEGO >

[jira] [Updated] (HDFS-8681) BlockScanner is incorrectly disabled by default

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8681: - Fix Version/s: 2.8.0 > BlockScanner is incorrectly disabled by default >

[jira] [Updated] (HDFS-8626) Reserved RBW space is not released if creation of RBW File fails

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8626: - Fix Version/s: 2.8.0 > Reserved RBW space is not released if creation of RBW File fails >

[jira] [Updated] (HDFS-8127) NameNode Failover during HA upgrade can cause DataNode to finalize upgrade

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8127: - Fix Version/s: 2.8.0 > NameNode Failover during HA upgrade can cause DataNode to finalize upgrade >

[jira] [Updated] (HDFS-8179) DFSClient#getServerDefaults returns null within 1 hour of system start

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8179: - Fix Version/s: 2.8.0 > DFSClient#getServerDefaults returns null within 1 hour of system start >

[jira] [Updated] (HDFS-8163) Using monotonicNow for block report scheduling causes test failures on recently restarted systems

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8163: - Fix Version/s: 2.8.0 > Using monotonicNow for block report scheduling causes test failures on > recently

[jira] [Updated] (HDFS-8451) DFSClient probe for encryption testing interprets empty URI property for "enabled"

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8451: - Fix Version/s: 2.8.0 > DFSClient probe for encryption testing interprets empty URI property for >

[jira] [Updated] (HDFS-8226) Non-HA rollback compatibility broken

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8226: - Fix Version/s: 2.8.0 > Non-HA rollback compatibility broken > > >

[jira] [Updated] (HDFS-8269) getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8269: - Fix Version/s: 2.8.0 > getBlockLocations() does not resolve the .reserved path and generates > incorrect

[jira] [Commented] (HDFS-9083) Replication violates block placement policy.

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802687#comment-15802687 ] Junping Du commented on HDFS-9083: -- Got it. Thanks for the confirmation! > Replication violates block

[jira] [Updated] (HDFS-9896) WebHDFS API may return invalid JSON

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9896: - Assignee: (was: John Zhuge) > WebHDFS API may return invalid JSON >

[jira] [Updated] (HDFS-9896) WebHDFS API may return invalid JSON

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9896: - Fix Version/s: (was: 2.7.3) > WebHDFS API may return invalid JSON >

[jira] [Commented] (HDFS-9083) Replication violates block placement policy.

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802521#comment-15802521 ] Junping Du commented on HDFS-9083: -- Hi [~shahrs87], [~kihwal] and [~sjlee0], I assume the issue fixed here

[jira] [Updated] (HDFS-10214) Checkpoint Can not be done by StandbyNameNode.Because checkpoint may cause DataNode blockReport.blockReceivedAndDeleted.heartbeat rpc timeout when the object num > 100000

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10214: -- Fix Version/s: (was: 2.7.2) > Checkpoint Can not be done by StandbyNameNode.Because checkpoint may

[jira] [Updated] (HDFS-8519) Patch for PPC64

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8519: - Fix Version/s: (was: 2.7.1) > Patch for PPC64 > --- > > Key: HDFS-8519 >

[jira] [Updated] (HDFS-8518) Patch for PPC64

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8518: - Fix Version/s: (was: 2.7.1) > Patch for PPC64 > --- > > Key: HDFS-8518 >

[jira] [Updated] (HDFS-9293) FSEditLog's 'OpInstanceCache' instance of threadLocal cache exists dirty 'rpcId',which may cause standby NN too busy to communicate

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9293: - Fix Version/s: (was: 2.7.1) > FSEditLog's 'OpInstanceCache' instance of threadLocal cache exists

[jira] [Updated] (HDFS-5180) Output the processing time of slow RPC request to node's log

2017-01-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-5180: - Fix Version/s: (was: 2.8.0) > Output the processing time of slow RPC request to node's log >

[jira] [Updated] (HDFS-8061) Create an Offline FSImage Viewer tool

2017-01-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8061: - Fix Version/s: (was: 2.8.0) > Create an Offline FSImage Viewer tool >

[jira] [Updated] (HDFS-8306) Outputs Xattr in OIV XML format

2017-01-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8306: - Fix Version/s: (was: 2.8.0) > Outputs Xattr in OIV XML format > --- > >

[jira] [Updated] (HDFS-8872) Reporting of missing blocks is different in fsck and namenode ui/metasave

2017-01-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8872: - Fix Version/s: (was: 2.8.0) > Reporting of missing blocks is different in fsck and namenode

[jira] [Updated] (HDFS-9310) TestDataNodeHotSwapVolumes fails occasionally

2017-01-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9310: - Fix Version/s: (was: 2.8.0) > TestDataNodeHotSwapVolumes fails occasionally >

[jira] [Commented] (HDFS-9940) Balancer should not use property dfs.datanode.balance.max.concurrent.moves

2017-01-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15799935#comment-15799935 ] Junping Du commented on HDFS-9940: -- The patch is not get in, so we should set target version instead of

[jira] [Updated] (HDFS-9287) Block placement completely fails if too many nodes are decommissioning

2017-01-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9287: - Fix Version/s: (was: 2.8.0) > Block placement completely fails if too many nodes are decommissioning >

[jira] [Updated] (HDFS-9940) Balancer should not use property dfs.datanode.balance.max.concurrent.moves

2017-01-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9940: - Fix Version/s: (was: 2.8.0) > Balancer should not use property

[jira] [Updated] (HDFS-9940) Balancer should not use property dfs.datanode.balance.max.concurrent.moves

2017-01-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9940: - Target Version/s: 2.9.0 (was: 2.8.0) > Balancer should not use property

[jira] [Updated] (HDFS-10171) Balancer should log config values

2017-01-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10171: -- Fix Version/s: (was: 2.8.0) > Balancer should log config values > -

[jira] [Updated] (HDFS-10198) File browser web UI should split to pages when files/dirs are too many

2017-01-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10198: -- Fix Version/s: (was: 2.8.0) > File browser web UI should split to pages when files/dirs are too

[jira] [Updated] (HDFS-10315) Fix TestRetryCacheWithHA and TestNamenodeRetryCache failures

2017-01-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10315: -- Fix Version/s: (was: 2.8.0) > Fix TestRetryCacheWithHA and TestNamenodeRetryCache failures >

[jira] [Commented] (HDFS-10716) In Balancer, the target task should be removed when its size < 0.

2017-01-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15799925#comment-15799925 ] Junping Du commented on HDFS-10716: --- Sound like we were missing jira number in commit log. {noformat}

[jira] [Updated] (HDFS-10888) dfshealth.html#tab-datanode

2017-01-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10888: -- Fix Version/s: (was: 2.8.0) > dfshealth.html#tab-datanode > --- > >

[jira] [Commented] (HDFS-11160) VolumeScanner reports write-in-progress replicas as corrupt incorrectly

2017-01-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15799919#comment-15799919 ] Junping Du commented on HDFS-11160: --- >From commit log, it indicate the commit only land in branch-2 but

[jira] [Updated] (HDFS-11160) VolumeScanner reports write-in-progress replicas as corrupt incorrectly

2017-01-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-11160: -- Fix Version/s: (was: 2.8.0) 2.9.0 > VolumeScanner reports write-in-progress

[jira] [Commented] (HDFS-11225) NameNode crashed because deleteSnapshot held FSNamesystem lock too long

2016-12-15 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15753114#comment-15753114 ] Junping Du commented on HDFS-11225: --- Looks like a nice feature, but could be too late for 2.8. Move it

[jira] [Updated] (HDFS-11225) NameNode crashed because deleteSnapshot held FSNamesystem lock too long

2016-12-15 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-11225: -- Target Version/s: 2.9.0, 3.0.0-beta1 (was: 2.8.0, 3.0.0-beta1) > NameNode crashed because

[jira] [Commented] (HDFS-10684) WebHDFS DataNode calls fail without parameter createparent

2016-12-06 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727021#comment-15727021 ] Junping Du commented on HDFS-10684: --- I see. Thanks for update, John. I would like to understand the

[jira] [Commented] (HDFS-10581) Hide redundant table on NameNode WebUI when no nodes are decomissioning

2016-12-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15724206#comment-15724206 ] Junping Du commented on HDFS-10581: --- The latest patch indeed becomes a trivial fix which looks good to

[jira] [Updated] (HDFS-10581) Hide redundant table on NameNode WebUI when no nodes are decomissioning

2016-12-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10581: -- Priority: Trivial (was: Major) > Hide redundant table on NameNode WebUI when no nodes are

[jira] [Commented] (HDFS-10684) WebHDFS DataNode calls fail without parameter createparent

2016-12-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723956#comment-15723956 ] Junping Du commented on HDFS-10684: --- Thanks for comments, Andrew. [~jzhuge], do we make any progress so

[jira] [Commented] (HDFS-10684) WebHDFS DataNode calls fail without parameter createparent

2016-12-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15723837#comment-15723837 ] Junping Du commented on HDFS-10684: --- Hi, this doesn't sounds like a blocker for our 2.8 release. Can

[jira] [Commented] (HDFS-10581) Redundant table on Datanodes page when no nodes under decomissioning

2016-12-02 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15717053#comment-15717053 ] Junping Du commented on HDFS-10581: --- Remove elements in UI is not a trivial thing, especially some UI

[jira] [Updated] (HDFS-10581) Redundant table on Datanodes page when no nodes under decomissioning

2016-12-02 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10581: -- Priority: Major (was: Trivial) > Redundant table on Datanodes page when no nodes under decomissioning

[jira] [Updated] (HDFS-8845) DiskChecker should not traverse the entire tree

2016-12-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8845: - Fix Version/s: 2.8.0 > DiskChecker should not traverse the entire tree >

[jira] [Updated] (HDFS-8850) VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks

2016-12-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8850: - Fix Version/s: 2.8.0 > VolumeScanner thread exits with exception if there is no block pool to be >

[jira] [Updated] (HDFS-8852) HDFS architecture documentation of version 2.x is outdated about append write support

2016-12-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8852: - Fix Version/s: 2.8.0 > HDFS architecture documentation of version 2.x is outdated about append write >

[jira] [Updated] (HDFS-7645) Rolling upgrade is restoring blocks from trash multiple times

2016-12-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-7645: - Fix Version/s: 2.8.0 > Rolling upgrade is restoring blocks from trash multiple times >

[jira] [Updated] (HDFS-7725) Incorrect "nodes in service" metrics caused all writes to fail

2016-12-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-7725: - Fix Version/s: 2.8.0 > Incorrect "nodes in service" metrics caused all writes to fail >

[jira] [Updated] (HDFS-9874) Long living DataXceiver threads cause volume shutdown to block.

2016-12-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9874: - Fix Version/s: 2.8.0 > Long living DataXceiver threads cause volume shutdown to block. >

[jira] [Updated] (HDFS-9904) testCheckpointCancellationDuringUpload occasionally fails

2016-12-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9904: - Fix Version/s: 2.8.0 > testCheckpointCancellationDuringUpload occasionally fails >

[jira] [Updated] (HDFS-10474) hftp copy fails when file name with Chinese+special char in branch-2

2016-12-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-10474: -- Fix Version/s: 2.8.0 > hftp copy fails when file name with Chinese+special char in branch-2 >

[jira] [Updated] (HDFS-2580) NameNode#main(...) can make use of GenericOptionsParser.

2016-12-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-2580: - Fix Version/s: 2.8.0 > NameNode#main(...) can make use of GenericOptionsParser. >

[jira] [Commented] (HDFS-11179) LightWeightHashSet can't remove blocks correctly which have a large number blockId

2016-11-28 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15703429#comment-15703429 ] Junping Du commented on HDFS-11179: --- Agree with [~jojochuang]. Especially, this EC staff doesn't show up

[jira] [Updated] (HDFS-11179) LightWeightHashSet can't remove blocks correctly which have a large number blockId

2016-11-28 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-11179: -- Target Version/s: 3.0.0-alpha2 (was: 2.8.0, 3.0.0-alpha2) > LightWeightHashSet can't remove blocks

[jira] [Updated] (HDFS-9456) BlockPlacementPolicyWithNodeGroup should override verifyBlockPlacement

2016-02-16 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9456: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved

[jira] [Commented] (HDFS-9456) BlockPlacementPolicyWithNodeGroup should override verifyBlockPlacement

2016-02-16 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15148467#comment-15148467 ] Junping Du commented on HDFS-9456: -- Thanks [~xiaobingo] for updating the patch. The patch looks pretty

[jira] [Updated] (HDFS-9629) Update the footer of Web UI to show year 2016

2016-02-11 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9629: - Fix Version/s: 2.7.3 2.8.0 > Update the footer of Web UI to show year 2016 >

[jira] [Commented] (HDFS-9629) Update the footer of Web UI to show year 2016

2016-02-11 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15143908#comment-15143908 ] Junping Du commented on HDFS-9629: -- Add 2.7.3 and 2.8.0 in fix version given it is already land on

[jira] [Updated] (HDFS-8722) Optimize datanode writes for small writes and flushes

2016-02-02 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8722: - Fix Version/s: 2.6.4 > Optimize datanode writes for small writes and flushes >

[jira] [Commented] (HDFS-8722) Optimize datanode writes for small writes and flushes

2016-02-02 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128736#comment-15128736 ] Junping Du commented on HDFS-8722: -- I have merge the 2.6 patch to branch-2.6. Thanks [~kihwal] for help on

[jira] [Commented] (HDFS-4660) Block corruption can happen during pipeline recovery

2016-02-02 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128690#comment-15128690 ] Junping Du commented on HDFS-4660: -- I have commit the 2.6 patch to branch-2.6. Thanks [~kihwal] for

[jira] [Updated] (HDFS-4660) Block corruption can happen during pipeline recovery

2016-02-02 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-4660: - Fix Version/s: 2.6.4 > Block corruption can happen during pipeline recovery >

[jira] [Commented] (HDFS-9178) Slow datanode I/O can cause a wrong node to be marked bad

2016-02-02 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128715#comment-15128715 ] Junping Du commented on HDFS-9178: -- Thanks [~kihwal] for review the patch! I have commit the patch to

[jira] [Updated] (HDFS-9178) Slow datanode I/O can cause a wrong node to be marked bad

2016-02-02 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9178: - Fix Version/s: 2.6.4 > Slow datanode I/O can cause a wrong node to be marked bad >

[jira] [Updated] (HDFS-8995) Flaw in registration bookeeping can make DN die on reconnect

2016-02-02 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8995: - Target Version/s: 2.7.2, 2.6.5 (was: 2.7.2, 2.6.4) > Flaw in registration bookeeping can make DN die on

[jira] [Commented] (HDFS-9456) BlockPlacementPolicyWithNodeGroup should override verifyBlockPlacement

2016-02-02 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15129776#comment-15129776 ] Junping Du commented on HDFS-9456: -- Thanks for updating the patch, [~xiaobingo]! There is an issue with

[jira] [Updated] (HDFS-9178) Slow datanode I/O can cause a wrong node to be marked bad

2016-02-02 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9178: - Attachment: 002-HDFS-9178.branch-2.6.patch The patch for branch-2.6 is stale. Update 002 patch that sync

[jira] [Updated] (HDFS-9220) Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum

2016-02-02 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9220: - Fix Version/s: 2.6.4 > Reading small file (< 512 bytes) that is open for append fails due to > incorrect

[jira] [Commented] (HDFS-9220) Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum

2016-02-02 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15129494#comment-15129494 ] Junping Du commented on HDFS-9220: -- Thanks [~kihwal]! Agree that only minor conflict with HDFS-9220 for

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-02-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8791: - Target Version/s: 2.8.0, 2.7.3, 2.6.5 (was: 2.8.0, 2.7.3, 2.6.4) > block ID-based DN storage layout can

[jira] [Updated] (HDFS-9106) Transfer failure during pipeline recovery causes permanent write failures

2016-02-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9106: - Target Version/s: 2.8.0, 2.6.5 (was: 2.8.0, 2.6.4) > Transfer failure during pipeline recovery causes

[jira] [Updated] (HDFS-9698) Long running Balancer should renew TGT

2016-02-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9698: - Target Version/s: 2.6.5 (was: 2.6.4) > Long running Balancer should renew TGT >

[jira] [Commented] (HDFS-8995) Flaw in registration bookeeping can make DN die on reconnect

2016-02-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127560#comment-15127560 ] Junping Du commented on HDFS-8995: -- Move it out of 2.6.4 to 2.6.5 as haven't updated for a while. > Flaw

[jira] [Commented] (HDFS-8676) Delayed rolling upgrade finalization can cause heartbeat expiration and write failures

2016-02-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127552#comment-15127552 ] Junping Du commented on HDFS-8676: -- Move it out of 2.6.4 to 2.6.5 as this jira haven't update for a while.

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-02-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127566#comment-15127566 ] Junping Du commented on HDFS-8791: -- Move this out of 2.6.4 to 2.6.5 as haven't updated for a while. >

[jira] [Updated] (HDFS-8676) Delayed rolling upgrade finalization can cause heartbeat expiration and write failures

2016-02-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-8676: - Target Version/s: 2.7.2, 2.6.5 (was: 2.7.2, 2.6.4) > Delayed rolling upgrade finalization can cause

[jira] [Commented] (HDFS-9106) Transfer failure during pipeline recovery causes permanent write failures

2016-02-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127557#comment-15127557 ] Junping Du commented on HDFS-9106: -- Move it out of 2.6.4 to 2.6.5 as this JIRA haven't being updated for a

[jira] [Updated] (HDFS-9688) Test the effect of nested encryption zones in HDFS downgrade

2016-02-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9688: - Target Version/s: 2.8.0, 2.7.3, 2.6.5 (was: 2.8.0, 2.7.3, 2.6.4) > Test the effect of nested encryption

[jira] [Updated] (HDFS-9263) tests are using /test/build/data; breaking Jenkins

2016-02-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HDFS-9263: - Target Version/s: 3.0.0, 2.7.3, 2.6.5 (was: 3.0.0, 2.7.3, 2.6.4) > tests are using /test/build/data;

<    1   2   3   4   5   6   7   8   9   10   >