[jira] [Commented] (HDFS-5241) Provide alternate queuing audit logger to reduce logging contention

[ https://issues.apache.org/jira/browse/HDFS-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481444#comment-14481444 ] Arpit Agarwal commented on HDFS-5241: - Thanks for confirming that Kihwal. Provide

[jira] [Updated] (HDFS-7936) Erasure coding: resolving conflicts in the branch when merging trunk changes.

[ https://issues.apache.org/jira/browse/HDFS-7936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-7936: Summary: Erasure coding: resolving conflicts in the branch when merging trunk changes. (was: Erasure

[jira] [Updated] (HDFS-7936) Erasure coding: resolving conflicts in the branch when merging

[ https://issues.apache.org/jira/browse/HDFS-7936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-7936: Description: This will be used to track and resolve conflicts when merging trunk changes. Below is a list

[jira] [Updated] (HDFS-7936) Erasure coding: resolving conflicts in the branch when merging

[ https://issues.apache.org/jira/browse/HDFS-7936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-7936: Attachment: HDFS-7936-005.patch This is to resolve conflicts with HDFS-8035 Erasure coding: resolving

[jira] [Created] (HDFS-8068) Do not retry rpc calls If the proxy contains unresolved address

Kihwal Lee created HDFS-8068: Summary: Do not retry rpc calls If the proxy contains unresolved address Key: HDFS-8068 URL: https://issues.apache.org/jira/browse/HDFS-8068 Project: Hadoop HDFS

[jira] [Updated] (HDFS-8055) NullPointerException when topology script is missing.

[ https://issues.apache.org/jira/browse/HDFS-8055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-8055: --- Fix Version/s: 2.7.0 Status: Patch Available (was: Open) NullPointerException when

[jira] [Updated] (HDFS-8068) Do not retry rpc calls If the proxy contains unresolved address

[ https://issues.apache.org/jira/browse/HDFS-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8068: - Description: When the InetSocketAddress object happens to be unresolvable (e.g. due to transient DNS

[jira] [Commented] (HDFS-7923) The DataNodes should rate-limit their full block reports by asking the NN on heartbeat messages

[ https://issues.apache.org/jira/browse/HDFS-7923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481715#comment-14481715 ] Charles Lamb commented on HDFS-7923: Here is a description of the heuristic that my

[jira] [Commented] (HDFS-8032) Erasure coding: shrink size of indices in BlockInfoStriped

[ https://issues.apache.org/jira/browse/HDFS-8032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481828#comment-14481828 ] Zhe Zhang commented on HDFS-8032: - Thanks Yi for the comment. I should have put a more

[jira] [Commented] (HDFS-8068) Do not retry rpc calls If the proxy contains unresolved address

[ https://issues.apache.org/jira/browse/HDFS-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481923#comment-14481923 ] Kihwal Lee commented on HDFS-8068: -- The patch adds a check in

[jira] [Updated] (HDFS-7969) Erasure coding: NameNode support for lease recovery of striped block groups

[ https://issues.apache.org/jira/browse/HDFS-7969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-7969: Attachment: HDFS-7969-001.patch Thanks Yi for the review! The updated patch address the 2 comments and I

[jira] [Resolved] (HDFS-7969) Erasure coding: NameNode support for lease recovery of striped block groups

[ https://issues.apache.org/jira/browse/HDFS-7969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7969. - Resolution: Fixed Fix Version/s: HDFS-7285 Erasure coding: NameNode support for lease recovery of

[jira] [Commented] (HDFS-6440) Support more than 2 NameNodes

[ https://issues.apache.org/jira/browse/HDFS-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481831#comment-14481831 ] Aaron T. Myers commented on HDFS-6440: -- Sorry, [~jesse_yates], been busy. I got

[jira] [Updated] (HDFS-8038) PBImageDelimitedTextWriter#getEntry output HDFS path in platform-specific format.

[ https://issues.apache.org/jira/browse/HDFS-8038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-8038: - Attachment: HDFS-8038.02.patch Thanks [~cnauroth] for the review. I've fixed the path issues in the

[jira] [Updated] (HDFS-7923) The DataNodes should rate-limit their full block reports by asking the NN on heartbeat messages

[ https://issues.apache.org/jira/browse/HDFS-7923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb updated HDFS-7923: --- Attachment: HDFS-7923.000.patch Attached is a patch that implements the behavior I described. The

[jira] [Work started] (HDFS-7923) The DataNodes should rate-limit their full block reports by asking the NN on heartbeat messages

[ https://issues.apache.org/jira/browse/HDFS-7923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-7923 started by Charles Lamb. -- The DataNodes should rate-limit their full block reports by asking the NN on

[jira] [Created] (HDFS-8070) ShortCircuitShmManager goes into dead mode, stopping all operations

Gopal V created HDFS-8070: - Summary: ShortCircuitShmManager goes into dead mode, stopping all operations Key: HDFS-8070 URL: https://issues.apache.org/jira/browse/HDFS-8070 Project: Hadoop HDFS

[jira] [Commented] (HDFS-6666) Abort NameNode and DataNode startup if security is enabled but block access token is not enabled.

[ https://issues.apache.org/jira/browse/HDFS-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481610#comment-14481610 ] Arpit Agarwal commented on HDFS-: - Hi Vijay, the code change looks fine. You don't

[jira] [Updated] (HDFS-8068) Do not retry rpc calls If the proxy contains unresolved address

[ https://issues.apache.org/jira/browse/HDFS-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8068: - Target Version/s: 2.8.0 Status: Patch Available (was: Open) Do not retry rpc calls If the

[jira] [Comment Edited] (HDFS-6666) Abort NameNode and DataNode startup if security is enabled but block access token is not enabled.

[ https://issues.apache.org/jira/browse/HDFS-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481610#comment-14481610 ] Arpit Agarwal edited comment on HDFS- at 4/6/15 6:54 PM: -

[jira] [Commented] (HDFS-6440) Support more than 2 NameNodes

[ https://issues.apache.org/jira/browse/HDFS-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481709#comment-14481709 ] Jesse Yates commented on HDFS-6440: --- Us too. We are waiting on a committer to have time

[jira] [Updated] (HDFS-8032) Erasure coding: shrink size of indices in BlockInfoStriped

[ https://issues.apache.org/jira/browse/HDFS-8032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8032: Description: In a _m+k_ erasure coding schema, the first _(m+k)_ slots in {{BlockInfoStriped}} do not need

[jira] [Updated] (HDFS-8055) NullPointerException when topology script is missing.

[ https://issues.apache.org/jira/browse/HDFS-8055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-8055: --- Attachment: hdfs-8055.001.patch Handles the null returned by the ShellExecutor correctly in case of

[jira] [Commented] (HDFS-8050) Separate the client conf key from DFSConfigKeys

[ https://issues.apache.org/jira/browse/HDFS-8050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481674#comment-14481674 ] Steve Loughran commented on HDFS-8050: -- Given past experience, moving stuff out of

[jira] [Commented] (HDFS-6440) Support more than 2 NameNodes

[ https://issues.apache.org/jira/browse/HDFS-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481679#comment-14481679 ] Patrick White commented on HDFS-6440: - We're pretty interested in this as well, how's

[jira] [Created] (HDFS-8069) Tracing implementation on DFSInputStream seriously degrades performance

Josh Elser created HDFS-8069: Summary: Tracing implementation on DFSInputStream seriously degrades performance Key: HDFS-8069 URL: https://issues.apache.org/jira/browse/HDFS-8069 Project: Hadoop HDFS

[jira] [Commented] (HDFS-6440) Support more than 2 NameNodes

[ https://issues.apache.org/jira/browse/HDFS-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481863#comment-14481863 ] Patrick White commented on HDFS-6440: - [~jesse_yates] I'm not sure I know any HDFS

[jira] [Assigned] (HDFS-8068) Do not retry rpc calls If the proxy contains unresolved address

[ https://issues.apache.org/jira/browse/HDFS-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee reassigned HDFS-8068: Assignee: Kihwal Lee Do not retry rpc calls If the proxy contains unresolved address

[jira] [Updated] (HDFS-8068) Do not retry rpc calls If the proxy contains unresolved address

[ https://issues.apache.org/jira/browse/HDFS-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8068: - Attachment: HDFS-8068.v1.patch Do not retry rpc calls If the proxy contains unresolved address

[jira] [Commented] (HDFS-6440) Support more than 2 NameNodes

[ https://issues.apache.org/jira/browse/HDFS-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481957#comment-14481957 ] Lars Hofhansl commented on HDFS-6440: - Let me also restate that we are running this in

[jira] [Commented] (HDFS-5215) dfs.datanode.du.reserved is not taking effect as it's not considered while getting the available space

[ https://issues.apache.org/jira/browse/HDFS-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481985#comment-14481985 ] Yongjun Zhang commented on HDFS-5215: - Hi [~brahmareddy], Sorry for the delay, I was

[jira] [Commented] (HDFS-8059) Erasure coding: move dataBlockNum and parityBlockNum from BlockInfoStriped to INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482080#comment-14482080 ] Jing Zhao commented on HDFS-8059: - Thanks for working on this, Yi! The patch looks good to

[jira] [Commented] (HDFS-8055) NullPointerException when topology script is missing.

[ https://issues.apache.org/jira/browse/HDFS-8055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482128#comment-14482128 ] Hadoop QA commented on HDFS-8055: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-8068) Do not retry rpc calls If the proxy contains unresolved address

[ https://issues.apache.org/jira/browse/HDFS-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482238#comment-14482238 ] Hadoop QA commented on HDFS-8068: - {color:red}-1 overall{color}. Here are the results of

[jira] [Updated] (HDFS-7792) Add links to FaultInjectFramework and SLGUserGuide to site index

[ https://issues.apache.org/jira/browse/HDFS-7792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HDFS-7792: --- Attachment: HDFS-7792.001.patch LoadGenerator works on current trunk. Attached patch adds link

[jira] [Updated] (HDFS-8071) Redundant checkFileProgress() in PART II of getAdditionalBlock()

[ https://issues.apache.org/jira/browse/HDFS-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-8071: -- Attachment: HDFS-8071-01.patch This is minor optimization. I removed

[jira] [Commented] (HDFS-7937) Erasure Coding: INodeFile quota computation unit tests

[ https://issues.apache.org/jira/browse/HDFS-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482181#comment-14482181 ] Zhe Zhang commented on HDFS-7937: - To create a file in striping / EC format, we should set

[jira] [Commented] (HDFS-8069) Tracing implementation on DFSInputStream seriously degrades performance

[ https://issues.apache.org/jira/browse/HDFS-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482224#comment-14482224 ] Billie Rinaldi commented on HDFS-8069: -- We aren't tracing in the span collector. We

[jira] [Commented] (HDFS-8068) Do not retry rpc calls If the proxy contains unresolved address

[ https://issues.apache.org/jira/browse/HDFS-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481989#comment-14481989 ] Kihwal Lee commented on HDFS-8068: -- {{FailoverOnNetworkExceptionRetry#shouldRetry()}}

[jira] [Updated] (HDFS-8017) Erasure Coding: perform non-stripping erasure decoding/recovery work given block reader and writer

[ https://issues.apache.org/jira/browse/HDFS-8017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-8017: Description: This assumes the facilities like block reader and writer are ready, implements and performs

[jira] [Updated] (HDFS-8017) Erasure Coding: perform non-stripping erasure decoding/recovery work given block reader and writer

[ https://issues.apache.org/jira/browse/HDFS-8017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-8017: Summary: Erasure Coding: perform non-stripping erasure decoding/recovery work given block reader and writer

[jira] [Commented] (HDFS-8071) Redundant checkFileProgress() in PART II of getAdditionalBlock()

[ https://issues.apache.org/jira/browse/HDFS-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482219#comment-14482219 ] Tsz Wo Nicholas Sze commented on HDFS-8071: --- Patch looks good. The readLock()

[jira] [Commented] (HDFS-8069) Tracing implementation on DFSInputStream seriously degrades performance

[ https://issues.apache.org/jira/browse/HDFS-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482261#comment-14482261 ] Josh Elser commented on HDFS-8069: -- Thanks for chiming in, [~billie.rinaldi]. I had been

[jira] [Created] (HDFS-8071) Redundant checkFileProgress() in PART II of getAdditionalBlock()

Konstantin Shvachko created HDFS-8071: - Summary: Redundant checkFileProgress() in PART II of getAdditionalBlock() Key: HDFS-8071 URL: https://issues.apache.org/jira/browse/HDFS-8071 Project:

[jira] [Updated] (HDFS-7782) Erasure coding: pread from files in striped layout

[ https://issues.apache.org/jira/browse/HDFS-7782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-7782: Attachment: HDFS-7782-010.patch Thanks a lot Jing! The test suite {{testPlanReadPortions}} looks great! I

[jira] [Commented] (HDFS-7715) Implement the Hitchhiker erasure coding algorithm

[ https://issues.apache.org/jira/browse/HDFS-7715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482258#comment-14482258 ] Kai Zheng commented on HDFS-7715: - A quick review: High level: 1. Please write a

[jira] [Updated] (HDFS-7782) Erasure coding: pread from files in striped layout

[ https://issues.apache.org/jira/browse/HDFS-7782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7782: Attachment: HDFS-7782.009.patch Thanks Zhe! The 009 patch adds more unit tests, and also fixes several small

[jira] [Comment Edited] (HDFS-8069) Tracing implementation on DFSInputStream seriously degrades performance

[ https://issues.apache.org/jira/browse/HDFS-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482094#comment-14482094 ] Colin Patrick McCabe edited comment on HDFS-8069 at 4/6/15 10:32 PM:

[jira] [Commented] (HDFS-8069) Tracing implementation on DFSInputStream seriously degrades performance

[ https://issues.apache.org/jira/browse/HDFS-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482094#comment-14482094 ] Colin Patrick McCabe commented on HDFS-8069: Hi [~elserj], I'm not familiar

[jira] [Comment Edited] (HDFS-8069) Tracing implementation on DFSInputStream seriously degrades performance

[ https://issues.apache.org/jira/browse/HDFS-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482120#comment-14482120 ] Colin Patrick McCabe edited comment on HDFS-8069 at 4/6/15 10:48 PM:

[jira] [Updated] (HDFS-8062) Remove hard-coded values in favor of EC schema

[ https://issues.apache.org/jira/browse/HDFS-8062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Sasaki updated HDFS-8062: - Attachment: HDFS-8062.1.patch Remove hard-coded values in favor of EC schema

[jira] [Commented] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482164#comment-14482164 ] Zhe Zhang commented on HDFS-8058: - Thanks for the work Yi! The overall approach looks fine

[jira] [Commented] (HDFS-8038) PBImageDelimitedTextWriter#getEntry output HDFS path in platform-specific format.

[ https://issues.apache.org/jira/browse/HDFS-8038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482182#comment-14482182 ] Hadoop QA commented on HDFS-8038: - {color:red}-1 overall{color}. Here are the results of

[jira] [Updated] (HDFS-8071) Redundant checkFileProgress() in PART II of getAdditionalBlock()

[ https://issues.apache.org/jira/browse/HDFS-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-8071: -- Status: Patch Available (was: Open) Redundant checkFileProgress() in PART II of

[jira] [Commented] (HDFS-8069) Tracing implementation on DFSInputStream seriously degrades performance

[ https://issues.apache.org/jira/browse/HDFS-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482120#comment-14482120 ] Colin Patrick McCabe commented on HDFS-8069: I can think of a few ways to solve

[jira] [Commented] (HDFS-3233) Move IP to FQDN conversion from DatanodeJSPHelper to DatanodeID

[ https://issues.apache.org/jira/browse/HDFS-3233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482117#comment-14482117 ] Eli Collins commented on HDFS-3233: --- Looks like DatanodeJSPHelper has been removed, and

[jira] [Updated] (HDFS-7348) Erasure Coding: perform stripping erasure decoding work given block reader and writer

[ https://issues.apache.org/jira/browse/HDFS-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-7348: Description: This assumes the facilities like block reader and writer are ready, implements and performs

[jira] [Updated] (HDFS-7348) Erasure Coding: perform stripping erasure decoding/recovery work given block reader and writer

[ https://issues.apache.org/jira/browse/HDFS-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-7348: Summary: Erasure Coding: perform stripping erasure decoding/recovery work given block reader and writer

[jira] [Resolved] (HDFS-8064) Erasure coding: DataNode support for block recovery of striped block groups

[ https://issues.apache.org/jira/browse/HDFS-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng resolved HDFS-8064. - Resolution: Duplicate Release Note: This is a duplicate of HDFS-7348. Erasure coding: DataNode

[jira] [Work started] (HDFS-8062) Remove hard-coded values in favor of EC schema

[ https://issues.apache.org/jira/browse/HDFS-8062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-8062 started by Kai Sasaki. Remove hard-coded values in favor of EC schema

[jira] [Commented] (HDFS-8049) Annotation client implementation as private

[ https://issues.apache.org/jira/browse/HDFS-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482202#comment-14482202 ] Tsz Wo Nicholas Sze commented on HDFS-8049: --- The patch looks good. Could you

[jira] [Commented] (HDFS-7955) Improve naming of classes, methods, and variables related to block replication and recovery

[ https://issues.apache.org/jira/browse/HDFS-7955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482200#comment-14482200 ] Zhe Zhang commented on HDFS-7955: - [~rakeshr] Thanks for the question. Yes I will create a

[jira] [Commented] (HDFS-8071) Redundant checkFileProgress() in PART II of getAdditionalBlock()

[ https://issues.apache.org/jira/browse/HDFS-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482402#comment-14482402 ] Hadoop QA commented on HDFS-8071: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-8069) Tracing implementation on DFSInputStream seriously degrades performance

[ https://issues.apache.org/jira/browse/HDFS-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482440#comment-14482440 ] Masatake Iwasaki commented on HDFS-8069: bq. Is it possible introduce sampling of

[jira] [Updated] (HDFS-8072) Reserved RBW space is not released if client terminates while writing block

[ https://issues.apache.org/jira/browse/HDFS-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8072: Attachment: HDFS-8072.01.patch Patch to release reserved space when the BlockReceiver encounters an

[jira] [Updated] (HDFS-8072) Reserved RBW space is not released if client terminates while writing block

[ https://issues.apache.org/jira/browse/HDFS-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8072: Status: Patch Available (was: Open) Reserved RBW space is not released if client terminates while

[jira] [Commented] (HDFS-8067) haadmin commands doesn't work in Federation with HA

[ https://issues.apache.org/jira/browse/HDFS-8067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482498#comment-14482498 ] Ajith S commented on HDFS-8067: --- Thanks for the comment Dave. But i think the -ns option is

[jira] [Updated] (HDFS-8071) Redundant checkFileProgress() in PART II of getAdditionalBlock()

[ https://issues.apache.org/jira/browse/HDFS-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-8071: -- Attachment: HDFS-8071-02.patch Good point. New patch removes redundant readLock().

[jira] [Updated] (HDFS-8071) Redundant checkFileProgress() in PART II of getAdditionalBlock()

[ https://issues.apache.org/jira/browse/HDFS-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8071: -- Hadoop Flags: Reviewed +1 patch looks good. Redundant checkFileProgress() in PART II

[jira] [Commented] (HDFS-7792) Add links to FaultInjectFramework and SLGUserGuide to site index

[ https://issues.apache.org/jira/browse/HDFS-7792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482267#comment-14482267 ] Masatake Iwasaki commented on HDFS-7792: I just left FaultInjectFramework.md as is.

[jira] [Updated] (HDFS-7792) Add links to FaultInjectFramework and SLGUserGuide to site index

[ https://issues.apache.org/jira/browse/HDFS-7792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HDFS-7792: --- Status: Patch Available (was: Open) Add links to FaultInjectFramework and SLGUserGuide to

[jira] [Commented] (HDFS-8059) Erasure coding: move dataBlockNum and parityBlockNum from BlockInfoStriped to INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482483#comment-14482483 ] Yi Liu commented on HDFS-8059: -- Jing, Thanks for your detailed and nice comments ! I agree

[jira] [Commented] (HDFS-7715) Implement the Hitchhiker erasure coding algorithm

[ https://issues.apache.org/jira/browse/HDFS-7715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482497#comment-14482497 ] jack liuquan commented on HDFS-7715: High level: 1. Please write a comprehensive class

[jira] [Updated] (HDFS-8071) Redundant checkFileProgress() in PART II of getAdditionalBlock()

[ https://issues.apache.org/jira/browse/HDFS-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8071: - Assignee: Konstantin Shvachko Redundant checkFileProgress() in PART II of getAdditionalBlock()

[jira] [Commented] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482332#comment-14482332 ] Jing Zhao commented on HDFS-8058: - Thanks for working on this, Yi! bq. The only concern I

[jira] [Commented] (HDFS-3233) Move IP to FQDN conversion from DatanodeJSPHelper to DatanodeID

[ https://issues.apache.org/jira/browse/HDFS-3233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482358#comment-14482358 ] Gabor Liptak commented on HDFS-3233: Eli, is this issue ready to be resolved? Thanks

[jira] [Commented] (HDFS-7792) Add links to FaultInjectFramework and SLGUserGuide to site index

[ https://issues.apache.org/jira/browse/HDFS-7792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482411#comment-14482411 ] Hadoop QA commented on HDFS-7792: - {color:green}+1 overall{color}. Here are the results of

[jira] [Updated] (HDFS-8049) Annotation client implementation as private

[ https://issues.apache.org/jira/browse/HDFS-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Fukudome updated HDFS-8049: -- Attachment: HDFS-8049.2.patch Hi Nicholas, Thank you for the review. I attached a new patch

[jira] [Commented] (HDFS-8072) Reserved RBW space is not released if client terminates while writing block

[ https://issues.apache.org/jira/browse/HDFS-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482456#comment-14482456 ] Arpit Agarwal commented on HDFS-8072: - Found by [~cnauroth] (thanks!). Reserved RBW

[jira] [Created] (HDFS-8072) Reserved RBW space is not released if client terminates while writing block

Arpit Agarwal created HDFS-8072: --- Summary: Reserved RBW space is not released if client terminates while writing block Key: HDFS-8072 URL: https://issues.apache.org/jira/browse/HDFS-8072 Project:

[jira] [Commented] (HDFS-8069) Tracing implementation on DFSInputStream seriously degrades performance

[ https://issues.apache.org/jira/browse/HDFS-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482274#comment-14482274 ] Josh Elser commented on HDFS-8069: -- With regards to your other points: Comments to

[jira] [Commented] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482423#comment-14482423 ] Yi Liu commented on HDFS-8058: -- Zhe, Jing, thanks for your comments! {quote} The only concern

[jira] [Updated] (HDFS-8072) Reserved RBW space is not released if client terminates while writing block

[ https://issues.apache.org/jira/browse/HDFS-8072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8072: Attachment: HDFS-8072.02.patch v02 patch with updated test case (explicitly set replica count to 3).

[jira] [Commented] (HDFS-7980) Incremental BlockReport will dramatically slow down the startup of a namenode

[ https://issues.apache.org/jira/browse/HDFS-7980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482513#comment-14482513 ] Walter Su commented on HDFS-7980: - I don't think we need to add a test case. It's too

[jira] [Commented] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482568#comment-14482568 ] Zhe Zhang commented on HDFS-8058: - Thanks Yi for clarifying. Yes {{assertBlock}} is a good

[jira] [Updated] (HDFS-8073) Split BlockPlacementPolicyDefault.chooseTarget(..) so it can be easily overrided.

[ https://issues.apache.org/jira/browse/HDFS-8073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8073: Description: We want to implement a placement policy( eg. HDFS-7891) based on default policy. It'll easier

[jira] [Commented] (HDFS-8059) Erasure coding: move dataBlockNum and parityBlockNum from BlockInfoStriped to INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482620#comment-14482620 ] Jing Zhao commented on HDFS-8059: - bq. Otherwise we should use other ways to get these two

[jira] [Created] (HDFS-8073) Split BlockPlacementPolicyDefault.chooseTarget(..) so it can be easily overrided.

Walter Su created HDFS-8073: --- Summary: Split BlockPlacementPolicyDefault.chooseTarget(..) so it can be easily overrided. Key: HDFS-8073 URL: https://issues.apache.org/jira/browse/HDFS-8073 Project: Hadoop

[jira] [Updated] (HDFS-7937) Erasure Coding: INodeFile quota computation unit tests

[ https://issues.apache.org/jira/browse/HDFS-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Sasaki updated HDFS-7937: - Attachment: HDFS-7937.5.patch Erasure Coding: INodeFile quota computation unit tests

[jira] [Commented] (HDFS-8071) Redundant checkFileProgress() in PART II of getAdditionalBlock()

[ https://issues.apache.org/jira/browse/HDFS-8071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482607#comment-14482607 ] Hadoop QA commented on HDFS-8071: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-7980) Incremental BlockReport will dramatically slow down the startup of a namenode

[ https://issues.apache.org/jira/browse/HDFS-7980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482590#comment-14482590 ] Yongjun Zhang commented on HDFS-7980: - Hi Guys, Thanks for reporting and working on

[jira] [Commented] (HDFS-7889) Subclass DFSOutputStream to support writing striping layout files

[ https://issues.apache.org/jira/browse/HDFS-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482606#comment-14482606 ] Zhe Zhang commented on HDFS-7889: - Would also be nice to verify the parity data content in

[jira] [Commented] (HDFS-7937) Erasure Coding: INodeFile quota computation unit tests

[ https://issues.apache.org/jira/browse/HDFS-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482633#comment-14482633 ] Kai Sasaki commented on HDFS-7937: -- Thank you. I updated patch to create {{INodeFile}}

[jira] [Updated] (HDFS-8073) Split BlockPlacementPolicyDefault.chooseTarget(..) so it can be easily overrided.

[ https://issues.apache.org/jira/browse/HDFS-8073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8073: Description: We want to implement a placement policy( eg. HDFS-7891) based on default policy. Split

[jira] [Commented] (HDFS-7969) Erasure coding: NameNode support for lease recovery of striped block groups

[ https://issues.apache.org/jira/browse/HDFS-7969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14480995#comment-14480995 ] Yi Liu commented on HDFS-7969: -- Hi Zhe, The logic in {{initializeBlockRecovery}} to choose

[jira] [Commented] (HDFS-7617) Add unit tests for editlog transactions for EC

[ https://issues.apache.org/jira/browse/HDFS-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481024#comment-14481024 ] Hui Zheng commented on HDFS-7617: - I am sorry for late reply. Add unit tests for editlog

[jira] [Commented] (HDFS-7617) Add unit tests for editlog transactions for EC

[ https://issues.apache.org/jira/browse/HDFS-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481023#comment-14481023 ] Hui Zheng commented on HDFS-7617: - I am sorry for reply.Thanks [~zhz]. Add unit tests

[jira] [Resolved] (HDFS-8066) Erasure coding: Decommission handle for EC blocks.

[ https://issues.apache.org/jira/browse/HDFS-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu resolved HDFS-8066. -- Resolution: Implemented Take a look at the latest branch, this has been implemented in branch, so close it as

[jira] [Created] (HDFS-8065) Support truncate at striped group boundary.

Yi Liu created HDFS-8065: Summary: Support truncate at striped group boundary. Key: HDFS-8065 URL: https://issues.apache.org/jira/browse/HDFS-8065 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Updated] (HDFS-8049) Annotation client implementation as private

[ https://issues.apache.org/jira/browse/HDFS-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Fukudome updated HDFS-8049: -- Attachment: HDFS-8049.1.patch I have attached a patch which adds InterfaceAudience Annotation to

  1   2   >