[jira] [Commented] (HDFS-7285) Erasure Coding Support inside HDFS

2021-04-26 Thread Zhe Zhang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17332809#comment-17332809 ] Zhe Zhang commented on HDFS-7285: - [~iostream2...@163.com] Unfortunately I don't have permission to the

[jira] [Commented] (HDFS-7285) Erasure Coding Support inside HDFS

2021-04-21 Thread Zhe Zhang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17327055#comment-17327055 ] Zhe Zhang commented on HDFS-7285: - [~iostream2...@163.com]: the image cannot be viewed. Can you upload

[jira] [Updated] (HDFS-15289) Allow viewfs mounts with hdfs scheme and centralized mount table

2020-04-23 Thread Zhe Zhang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-15289: - Fix Version/s: (was: 3.4.0) > Allow viewfs mounts with hdfs scheme and centralized mount table >

[jira] [Commented] (HDFS-12943) Consistent Reads from Standby Node

2019-02-21 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774857#comment-16774857 ] Zhe Zhang commented on HDFS-12943: -- [~vagarychen] has tested the current version of the feature on a

[jira] [Comment Edited] (HDFS-9098) Erasure coding: emulate race conditions among striped streamers in write pipeline

2018-10-22 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964015#comment-14964015 ] Zhe Zhang edited comment on HDFS-9098 at 10/22/18 7:14 PM: --- WIP patch to

[jira] [Commented] (HDDS-120) Adding HDDS datanode Audit Log

2018-05-24 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDDS-120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489887#comment-16489887 ] Zhe Zhang commented on HDDS-120: Thanks [~xyao], [~dineshchitlangia]. DN audit log is actually useful for

[jira] [Commented] (HDFS-13616) Batch listing of multiple directories

2018-05-24 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16489882#comment-16489882 ] Zhe Zhang commented on HDFS-13616: -- Interesting idea! Any thought on the potential of DDoSing the

[jira] [Commented] (HDFS-12284) RBF: Support for Kerberos authentication

2018-03-27 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415802#comment-16415802 ] Zhe Zhang commented on HDFS-12284: -- Thanks [~zhengxg3], nice work! Could you click "Submit Patch" to

[jira] [Assigned] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads

2018-02-08 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reassigned HDFS-12345: Assignee: Erik Krogen (was: Zhe Zhang) > Scale testing HDFS NameNode with real metadata and

[jira] [Comment Edited] (HDFS-12943) Consistent Reads from Standby Node

2017-12-20 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16298099#comment-16298099 ] Zhe Zhang edited comment on HDFS-12943 at 12/20/17 8:54 AM: Thanks [~csun],

[jira] [Comment Edited] (HDFS-12943) Consistent Reads from Standby Node

2017-12-20 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16298099#comment-16298099 ] Zhe Zhang edited comment on HDFS-12943 at 12/20/17 8:54 AM: Thanks [~csun],

[jira] [Commented] (HDFS-12943) Consistent Reads from Standby Node

2017-12-20 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16298099#comment-16298099 ] Zhe Zhang commented on HDFS-12943: -- Thanks [~csun], interesting results! If I understand correctly, both

[jira] [Updated] (HDFS-12888) NameNode web UI shows stale config values after cli refresh

2017-12-04 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12888: - Description: To reproduce: # Run webui /conf # Use {{hdfs --refreshSuperUserGroupsConfiguration}} to

[jira] [Created] (HDFS-12888) NameNode web UI shows stale config values after cli refresh

2017-12-04 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-12888: Summary: NameNode web UI shows stale config values after cli refresh Key: HDFS-12888 URL: https://issues.apache.org/jira/browse/HDFS-12888 Project: Hadoop HDFS

[jira] [Updated] (HDFS-10967) Add configuration for BlockPlacementPolicy to avoid near-full DataNodes

2017-12-04 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-10967: - Target Version/s: (was: 2.7.5) Removing the Target version since 2.7.5 is releasing soon. > Add

[jira] [Commented] (HDFS-12638) NameNode exits due to ReplicationMonitor thread received Runtime exception in ReplicationWork#chooseTargets

2017-11-30 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273621#comment-16273621 ] Zhe Zhang commented on HDFS-12638: -- Thanks [~shv]. +1 on the v3 patch (pretty clear fix to remove orphan

[jira] [Commented] (HDFS-12832) INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit

2017-11-28 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269744#comment-16269744 ] Zhe Zhang commented on HDFS-12832: -- +1 on branch-2 and branch-2.7 patches as well. They are pretty clean

[jira] [Commented] (HDFS-12832) INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit

2017-11-28 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268446#comment-16268446 ] Zhe Zhang commented on HDFS-12832: -- Thanks [~shv] for the fix. Trunk patch LGTM, +1. I'll finish

[jira] [Commented] (HDFS-12791) NameNode Fsck http Connection can timeout for directories with multiple levels

2017-11-21 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16261456#comment-16261456 ] Zhe Zhang commented on HDFS-12791: -- Thanks for the work [~msingh] and [~vagarychen]. What should be the

[jira] [Updated] (HDFS-12823) Backport HDFS-9259 "Make SO_SNDBUF size configurable at DFSClient" to branch-2.7

2017-11-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12823: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.7.5 Status: Resolved

[jira] [Commented] (HDFS-8829) Make SO_RCVBUF and SO_SNDBUF size configurable for DataTransferProtocol sockets and allow configuring auto-tuning

2017-11-14 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252780#comment-16252780 ] Zhe Zhang commented on HDFS-8829: - This is an useful improvement for 2.7 as well. I just backported to

[jira] [Updated] (HDFS-8829) Make SO_RCVBUF and SO_SNDBUF size configurable for DataTransferProtocol sockets and allow configuring auto-tuning

2017-11-14 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8829: Fix Version/s: 2.7.5 > Make SO_RCVBUF and SO_SNDBUF size configurable for DataTransferProtocol > sockets

[jira] [Updated] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-24 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12502: - Fix Version/s: (was: 3.1.0) (was: 3.0.0) (was: 2.8.3)

[jira] [Comment Edited] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-24 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1621#comment-1621 ] Zhe Zhang edited comment on HDFS-12502 at 10/24/17 10:07 PM: - For some reason

[jira] [Reopened] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-24 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-12502: -- > nntop should support a category based on FilesInGetListingOps >

[jira] [Updated] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-24 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12502: - Fix Version/s: (was: 2.7.5) > nntop should support a category based on FilesInGetListingOps >

[jira] [Comment Edited] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-24 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1621#comment-1621 ] Zhe Zhang edited comment on HDFS-12502 at 10/24/17 9:59 PM: For some reason we

[jira] [Commented] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-24 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1621#comment-1621 ] Zhe Zhang commented on HDFS-12502: -- For some reason we were getting over 600k~700k FilesInGetListing per

[jira] [Commented] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-24 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16217704#comment-16217704 ] Zhe Zhang commented on HDFS-12502: -- Thanks [~daryn], I agree that the name {{NNTopUserOpCounts}} has an

[jira] [Commented] (HDFS-11743) Revert the incompatible fsck reporting output in HDFS-7933 from branch-2.7

2017-10-24 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16217178#comment-16217178 ] Zhe Zhang commented on HDFS-11743: -- Thanks [~xiaochen] and [~brahmareddy] for the discussion (and sorry I

[jira] [Updated] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-19 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12502: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.0 3.0.0

[jira] [Updated] (HDFS-10984) Expose nntop output as metrics

2017-10-19 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-10984: - Fix Version/s: 2.7.5 Thanks [~swagle] and [~xyao]. I backported this to branch-2.7 as well. > Expose

[jira] [Updated] (HDFS-12420) Add an option to disallow 'namenode format -force'

2017-10-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12420: - Fix Version/s: 3.0.0 2.7.5 2.8.2 2.9.0 Thanks

[jira] [Updated] (HDFS-12420) Add an option to disallow 'namenode format -force'

2017-10-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12420: - Fix Version/s: 3.1.0 > Add an option to disallow 'namenode format -force' >

[jira] [Updated] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-16 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12502: - Attachment: HDFS-12502.04.patch Thanks [~ywskycn]! Attaching new patch to address the comment. > nntop

[jira] [Commented] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193644#comment-16193644 ] Zhe Zhang commented on HDFS-12502: -- Among the reported UT failures, {{TestUnderReplicatedBlocks}} seems

[jira] [Updated] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12502: - Attachment: (was: HDFS-12502.03.patch) > nntop should support a category based on

[jira] [Updated] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12502: - Attachment: HDFS-12502.03.patch > nntop should support a category based on FilesInGetListingOps >

[jira] [Updated] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12502: - Attachment: HDFS-12502.03.patch Thanks [~xkrogen], good points. Updating the patch to address. > nntop

[jira] [Updated] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-10-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12502: - Attachment: HDFS-12502.02.patch Thanks [~shv]! Updating the patch to address comments. Actually the

[jira] [Commented] (HDFS-7964) Add support for async edit logging

2017-09-28 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185037#comment-16185037 ] Zhe Zhang commented on HDFS-7964: - Very impressive numbers [~daryn]. How many handler threads are used for

[jira] [Updated] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-09-21 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12502: - Attachment: HDFS-12502.01.patch v00 patch actually doesn't work, because {{logAuditEvent}} for

[jira] [Updated] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-09-20 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12502: - Status: Patch Available (was: Open) > nntop should support a category based on FilesInGetListingOps >

[jira] [Assigned] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-09-20 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reassigned HDFS-12502: Assignee: Zhe Zhang > nntop should support a category based on FilesInGetListingOps >

[jira] [Updated] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-09-20 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12502: - Attachment: HDFS-12502.00.patch Initial patch attached. > nntop should support a category based on

[jira] [Updated] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-09-20 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12502: - Description: Large listing ops can oftentimes be the main contributor to NameNode slowness. The aggregate

[jira] [Updated] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-09-20 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12502: - Description: Large listing ops can oftentimes be the main contributor to NameNode slowness. The aggregate

[jira] [Updated] (HDFS-12502) nntop should support category based on FilesInGetListingOps

2017-09-20 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12502: - Component/s: metrics > nntop should support category based on FilesInGetListingOps >

[jira] [Updated] (HDFS-12502) nntop should support a category based on FilesInGetListingOps

2017-09-20 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12502: - Summary: nntop should support a category based on FilesInGetListingOps (was: nntop should support

[jira] [Created] (HDFS-12502) nntop should support category based on FilesInGetListingOps

2017-09-20 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-12502: Summary: nntop should support category based on FilesInGetListingOps Key: HDFS-12502 URL: https://issues.apache.org/jira/browse/HDFS-12502 Project: Hadoop HDFS

[jira] [Updated] (HDFS-12421) Balancer to emit standard metrics

2017-09-11 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12421: - Component/s: metrics > Balancer to emit standard metrics > - > >

[jira] [Updated] (HDFS-1068) Reduce NameNode GC by reusing HdfsFileStatus objects in RPC handlers

2017-09-01 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-1068: Attachment: HDFS-1068.00.patch Attaching a very rough PoC patch to demonstrate idea. > Reduce NameNode GC

[jira] [Updated] (HDFS-1068) Reduce NameNode GC by reusing HdfsFileStatus objects in RPC handlers

2017-09-01 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-1068: Status: Patch Available (was: Open) > Reduce NameNode GC by reusing HdfsFileStatus objects in RPC handlers

[jira] [Commented] (HDFS-1068) Reduce NameNode GC by reusing HdfsFileStatus objects in RPC handlers

2017-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149741#comment-16149741 ] Zhe Zhang commented on HDFS-1068: - Reviving this ticket since GC from RPCs is actually a major contributor

[jira] [Updated] (HDFS-1068) Reduce NameNode GC by reusing HdfsFileStatus objects in RPC handlers

2017-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-1068: Attachment: Screen Shot 2017-08-31 at 3.58.15 PM.png > Reduce NameNode GC by reusing HdfsFileStatus objects

[jira] [Assigned] (HDFS-1068) Reduce NameNode GC by reusing HdfsFileStatus objects in RPC handlers

2017-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reassigned HDFS-1068: --- Assignee: Zhe Zhang (was: Hairong Kuang) > Reduce NameNode GC by reusing HdfsFileStatus objects in

[jira] [Updated] (HDFS-1068) Reduce NameNode GC by reusing HdfsFileStatus objects in RPC handlers

2017-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-1068: Summary: Reduce NameNode GC by reusing HdfsFileStatus objects in RPC handlers (was: Reduce getFileInfo's

[jira] [Comment Edited] (HDFS-958) Remove uneccessary creation of FileStatus objects in NameNode

2017-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148519#comment-16148519 ] Zhe Zhang edited comment on HDFS-958 at 8/31/17 6:34 AM: - Actually it does make

[jira] [Commented] (HDFS-958) Remove uneccessary creation of FileStatus objects in NameNode

2017-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148519#comment-16148519 ] Zhe Zhang commented on HDFS-958: Actually it does make sense to at least make it an option to avoid creating

[jira] [Commented] (HDFS-12379) NameNode getListing should use FileStatus instead of HdfsFileStatus

2017-08-30 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148492#comment-16148492 ] Zhe Zhang commented on HDFS-12379: -- Pinging [~daryn] [~wheat9] [~kihwal] for opinions based on HDFS-11641

[jira] [Created] (HDFS-12379) NameNode getListing should use FileStatus instead of HdfsFileStatus

2017-08-30 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-12379: Summary: NameNode getListing should use FileStatus instead of HdfsFileStatus Key: HDFS-12379 URL: https://issues.apache.org/jira/browse/HDFS-12379 Project: Hadoop HDFS

[jira] [Updated] (HDFS-12248) SNN will not upload fsimage on IOE and Interrupted exceptions

2017-08-29 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12248: - Component/s: ha > SNN will not upload fsimage on IOE and Interrupted exceptions >

[jira] [Updated] (HDFS-12248) SNN will not upload fsimage on IOE and Interrupted exceptions

2017-08-29 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12248: - Component/s: namenode > SNN will not upload fsimage on IOE and Interrupted exceptions >

[jira] [Commented] (HDFS-9079) Erasure coding: preallocate multiple generation stamps and serialize updates from data streamers

2017-08-28 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16144094#comment-16144094 ] Zhe Zhang commented on HDFS-9079: - Unassigning myself from the JIRA as I won't be able to work on it in the

[jira] [Assigned] (HDFS-9079) Erasure coding: preallocate multiple generation stamps and serialize updates from data streamers

2017-08-28 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reassigned HDFS-9079: --- Assignee: (was: Zhe Zhang) > Erasure coding: preallocate multiple generation stamps and serialize

[jira] [Updated] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads

2017-08-23 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12345: - Description: Creating this JIRA to better track

[jira] [Updated] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads

2017-08-23 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12345: - Issue Type: New Feature (was: Bug) > Scale testing HDFS NameNode with real metadata and workloads >

[jira] [Assigned] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads

2017-08-23 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reassigned HDFS-12345: Assignee: Zhe Zhang > Scale testing HDFS NameNode with real metadata and workloads >

[jira] [Updated] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads

2017-08-23 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12345: - Component/s: test namenode > Scale testing HDFS NameNode with real metadata and

[jira] [Updated] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads

2017-08-23 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12345: - Summary: Scale testing HDFS NameNode with real metadata and workloads (was: Scale testing HDFS NameNode)

[jira] [Created] (HDFS-12345) Scale testing HDFS NameNode

2017-08-23 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-12345: Summary: Scale testing HDFS NameNode Key: HDFS-12345 URL: https://issues.apache.org/jira/browse/HDFS-12345 Project: Hadoop HDFS Issue Type: Bug

[jira] [Updated] (HDFS-12284) Router support for Kerberos authentication and delegation tokens

2017-08-09 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-12284: - Summary: Router support for Kerberos authentication and delegation tokens (was: Router support for

[jira] [Created] (HDFS-12284) Router support for Kerberos and delegation tokens

2017-08-09 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-12284: Summary: Router support for Kerberos and delegation tokens Key: HDFS-12284 URL: https://issues.apache.org/jira/browse/HDFS-12284 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2017-07-28 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-10702: - Issue Type: New Feature (was: Improvement) > Add a Client API and Proxy Provider to enable stale read

[jira] [Updated] (HDFS-11896) Non-dfsUsed will be doubled on dead node re-registration

2017-07-27 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-11896: - Attachment: HDFS-11896-branch-2.7-006.patch Uploading a new branch-2.7 patch to fix the test. Now it

[jira] [Commented] (HDFS-11896) Non-dfsUsed will be doubled on dead node re-registration

2017-07-26 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16102719#comment-16102719 ] Zhe Zhang commented on HDFS-11896: -- Thanks for the work [~brahmareddy]. I modified the code base to use

[jira] [Commented] (HDFS-11882) Client fails if acknowledged size is greater than bytes sent

2017-06-26 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16063592#comment-16063592 ] Zhe Zhang commented on HDFS-11882: -- Sorry I missed this ping earlier. bq. Although it says that "at

[jira] [Comment Edited] (HDFS-11634) Optimize BlockIterator when iterating starts in the middle.

2017-06-21 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16058203#comment-16058203 ] Zhe Zhang edited comment on HDFS-11634 at 6/21/17 8:51 PM: --- On one of our

[jira] [Updated] (HDFS-11634) Optimize BlockIterator when iterating starts in the middle.

2017-06-21 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-11634: - Attachment: Screen Shot 2017-06-21 at 1.39.21 PM.png On our cluster with ~150 million blocks,

[jira] [Updated] (HDFS-11743) Revert the incompatible fsck reporting output in HDFS-7933 from branch-2.7

2017-06-08 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-11743: - Labels: (was: release-blocker) > Revert the incompatible fsck reporting output in HDFS-7933 from

[jira] [Updated] (HDFS-11743) Revert the incompatible fsck reporting output in HDFS-7933 from branch-2.7

2017-06-07 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-11743: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.7.4 Status: Resolved

[jira] [Commented] (HDFS-11743) Revert the incompatible fsck reporting output in HDFS-7933 from branch-2.7

2017-06-06 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039879#comment-16039879 ] Zhe Zhang commented on HDFS-11743: -- [~brahmareddy] [~shv] Please kindly review v01 patch. I have verified

[jira] [Commented] (HDFS-11743) Revert the incompatible fsck reporting output in HDFS-7933 from branch-2.7

2017-06-06 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039666#comment-16039666 ] Zhe Zhang commented on HDFS-11743: -- Thanks for confirming [~shv]. I verified reported unit test failures

[jira] [Updated] (HDFS-11743) Revert the incompatible fsck reporting output in HDFS-7933 from branch-2.7

2017-06-06 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-11743: - Attachment: HDFS-11743-branch-2.7.01.patch Thanks [~brahmareddy] for taking a look. Yes you are right,

[jira] [Comment Edited] (HDFS-11743) Revert the incompatible fsck reporting output in HDFS-7933 from branch-2.7

2017-06-06 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038193#comment-16038193 ] Zhe Zhang edited comment on HDFS-11743 at 6/6/17 3:02 PM: -- Attaching v00 patch to

[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.

2017-06-06 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038247#comment-16038247 ] Zhe Zhang commented on HDFS-7933: - Thanks for the feedback [~brahmareddy]. I decided to revert only the

[jira] [Updated] (HDFS-11743) Revert the incompatible fsck reporting output in HDFS-7933 from branch-2.7

2017-06-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-11743: - Attachment: HDFS-11743-branch-2.7.00.patch Just reverted {{NamenodeFsck}} and {{TestFsck}}. > Revert the

[jira] [Updated] (HDFS-11743) Revert the incompatible fsck reporting output in HDFS-7933 from branch-2.7

2017-06-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-11743: - Status: Patch Available (was: In Progress) > Revert the incompatible fsck reporting output in HDFS-7933

[jira] [Work started] (HDFS-11743) Revert the incompatible fsck reporting output in HDFS-7933 from branch-2.7

2017-06-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-11743 started by Zhe Zhang. > Revert the incompatible fsck reporting output in HDFS-7933 from branch-2.7 >

[jira] [Updated] (HDFS-11743) Revert the incompatible fsck reporting output in HDFS-7933 from branch-2.7

2017-06-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-11743: - Summary: Revert the incompatible fsck reporting output in HDFS-7933 from branch-2.7 (was: Revert

[jira] [Commented] (HDFS-7933) fsck should also report decommissioning replicas.

2017-06-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037822#comment-16037822 ] Zhe Zhang commented on HDFS-7933: - I agree with the above suggestion from [~shv]. > fsck should also

[jira] [Commented] (HDFS-7964) Add support for async edit logging

2017-06-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16037810#comment-16037810 ] Zhe Zhang commented on HDFS-7964: - I took some time trying to fix HDFS-11737 unit test failures but they

[jira] [Updated] (HDFS-11737) Backport HDFS-7964 to branch-2.7: add support for async edit logging

2017-06-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-11737: - Attachment: HDFS-11737-branch-2.7.00.patch > Backport HDFS-7964 to branch-2.7: add support for async edit

[jira] [Updated] (HDFS-11737) Backport HDFS-7964 to branch-2.7: add support for async edit logging

2017-06-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-11737: - Status: Patch Available (was: Open) > Backport HDFS-7964 to branch-2.7: add support for async edit

[jira] [Updated] (HDFS-11737) ibdcgditutibrcn

2017-06-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-11737: - Summary: ibdcgditutibrcn (was: Backport HDFS-7964 to branch-2.7: add support for async edit logging) >

[jira] [Updated] (HDFS-11737) Backport HDFS-7964 to branch-2.7: add support for async edit logging

2017-06-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-11737: - Summary: Backport HDFS-7964 to branch-2.7: add support for async edit logging (was: ibdcgditutibrcn) >

[jira] [Updated] (HDFS-11743) Revert HDFS-7933 from branch-2.7 (fsck reporting decommissioning replicas)

2017-06-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-11743: - Labels: release-blocker (was: ) > Revert HDFS-7933 from branch-2.7 (fsck reporting decommissioning

[jira] [Updated] (HDFS-11648) Lazy construct the IIP pathname

2017-05-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-11648: - Fix Version/s: 2.7.4 I think this is a good improvement for 2.7.4; just committed to branch-2.7. Ran

[jira] [Updated] (HDFS-10619) Cache path in InodesInPath

2017-05-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-10619: - I think this is a good improvement for 2.7.4; just committed to branch-2.7. I ran test-patch locally and

[jira] [Updated] (HDFS-10619) Cache path in InodesInPath

2017-05-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-10619: - Fix Version/s: 2.7.4 > Cache path in InodesInPath > -- > > Key:

  1   2   3   4   5   6   7   8   9   10   >