[jira] [Commented] (HDFS-6261) Add document for enabling node group layer in HDFS

[ https://issues.apache.org/jira/browse/HDFS-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092565#comment-14092565 ] Hadoop QA commented on HDFS-6261: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-6664) HDFS permissions guide documentation states incorrect default group mapping class.

[ https://issues.apache.org/jira/browse/HDFS-6664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092567#comment-14092567 ] Hadoop QA commented on HDFS-6664: - {color:green}+1 overall{color}. Here are the results of

[jira] [Updated] (HDFS-6833) DirectoryScanner should not register a deleting block with memory of DataNode

[ https://issues.apache.org/jira/browse/HDFS-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shinichi Yamashita updated HDFS-6833: - Attachment: HDFS-6833.patch I attach a patch file which I added the processing about a

[jira] [Updated] (HDFS-6833) DirectoryScanner should not register a deleting block with memory of DataNode

[ https://issues.apache.org/jira/browse/HDFS-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shinichi Yamashita updated HDFS-6833: - Status: Patch Available (was: Open) DirectoryScanner should not register a deleting

[jira] [Commented] (HDFS-6833) DirectoryScanner should not register a deleting block with memory of DataNode

[ https://issues.apache.org/jira/browse/HDFS-6833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092737#comment-14092737 ] Hadoop QA commented on HDFS-6833: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-6659) Create a Block List

[ https://issues.apache.org/jira/browse/HDFS-6659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092871#comment-14092871 ] Daryn Sharp commented on HDFS-6659: --- I'd suggest trying to create a subclass of

[jira] [Commented] (HDFS-6660) Use int instead of object reference to DatanodeStorageInfo in BlockInfo triplets,

[ https://issues.apache.org/jira/browse/HDFS-6660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092874#comment-14092874 ] Daryn Sharp commented on HDFS-6660: --- As a safety precaution, is it possible to add at

[jira] [Commented] (HDFS-6803) Documenting DFSClient#DFSInputStream expectations reading and preading in concurrent context

[ https://issues.apache.org/jira/browse/HDFS-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092958#comment-14092958 ] Steve Loughran commented on HDFS-6803: -- I'm very tempted to push for a lax model which

[jira] [Commented] (HDFS-6793) Missing changes in HftpFileSystem when Reintroduce dfs.http.port / dfs.https.port in branch-2

[ https://issues.apache.org/jira/browse/HDFS-6793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14092983#comment-14092983 ] Juan Yu commented on HDFS-6793: --- Thanks Andrew and Yongjun! Missing changes in

[jira] [Commented] (HDFS-6569) OOB message can't be sent to the client when DataNode shuts down for upgrade

[ https://issues.apache.org/jira/browse/HDFS-6569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093039#comment-14093039 ] Brandon Li commented on HDFS-6569: -- The test failure was not introduced by this patch.

[jira] [Commented] (HDFS-6826) Plugin interface to enable delegation of HDFS authorization assertions

[ https://issues.apache.org/jira/browse/HDFS-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093045#comment-14093045 ] Daryn Sharp commented on HDFS-6826: --- I understand the motivation but there has to be a

[jira] [Commented] (HDFS-6837) Code cleanup for Balancer and Dispatcher

[ https://issues.apache.org/jira/browse/HDFS-6837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093060#comment-14093060 ] Jing Zhao commented on HDFS-6837: - +1. I will commit the patch shortly. Code cleanup for

[jira] [Updated] (HDFS-6837) Code cleanup for Balancer and Dispatcher

[ https://issues.apache.org/jira/browse/HDFS-6837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-6837: Resolution: Fixed Fix Version/s: 2.6.0 Hadoop Flags: Reviewed Status: Resolved (was:

[jira] [Created] (HDFS-6838) Code cleanup for unnecessary INode replacement

Jing Zhao created HDFS-6838: --- Summary: Code cleanup for unnecessary INode replacement Key: HDFS-6838 URL: https://issues.apache.org/jira/browse/HDFS-6838 Project: Hadoop HDFS Issue Type: Bug

[jira] [Updated] (HDFS-6838) Code cleanup for unnecessary INode replacement

[ https://issues.apache.org/jira/browse/HDFS-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-6838: Attachment: HDFS-6838.000.patch Main changes: # INode#recordModification on longer needs to return an

[jira] [Updated] (HDFS-6838) Code cleanup for unnecessary INode replacement

[ https://issues.apache.org/jira/browse/HDFS-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-6838: Status: Patch Available (was: Open) Code cleanup for unnecessary INode replacement

[jira] [Updated] (HDFS-6838) Code cleanup for unnecessary INode replacement

[ https://issues.apache.org/jira/browse/HDFS-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-6838: Priority: Minor (was: Major) Code cleanup for unnecessary INode replacement

[jira] [Created] (HDFS-6839) Fix TestCLI to expect new output

Charles Lamb created HDFS-6839: -- Summary: Fix TestCLI to expect new output Key: HDFS-6839 URL: https://issues.apache.org/jira/browse/HDFS-6839 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Work started] (HDFS-6839) Fix TestCLI to expect new output

[ https://issues.apache.org/jira/browse/HDFS-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-6839 started by Charles Lamb. Fix TestCLI to expect new output Key: HDFS-6839 URL:

[jira] [Updated] (HDFS-6839) Fix TestCLI to expect new output

[ https://issues.apache.org/jira/browse/HDFS-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb updated HDFS-6839: --- Attachment: HDFS-6839.001.patch A patch with the new output is attached. Might as well wait for the

[jira] [Commented] (HDFS-6825) Edit log corruption due to delayed block removal

[ https://issues.apache.org/jira/browse/HDFS-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093183#comment-14093183 ] Aaron T. Myers commented on HDFS-6825: -- Hey Yongjun, thanks a lot for working on this.

[jira] [Updated] (HDFS-6826) Plugin interface to enable delegation of HDFS authorization assertions

[ https://issues.apache.org/jira/browse/HDFS-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alejandro Abdelnur updated HDFS-6826: - Attachment: HDFS-6826-idea.patch [~daryn], A custom plugin, would have a list of region

[jira] [Commented] (HDFS-6826) Plugin interface to enable delegation of HDFS authorization assertions

[ https://issues.apache.org/jira/browse/HDFS-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093239#comment-14093239 ] Daryn Sharp commented on HDFS-6826: --- It doesn't matter which or how many paths go through

[jira] [Commented] (HDFS-6826) Plugin interface to enable delegation of HDFS authorization assertions

[ https://issues.apache.org/jira/browse/HDFS-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093249#comment-14093249 ] Alejandro Abdelnur commented on HDFS-6826: -- [~daryn], looking forward to your

[jira] [Commented] (HDFS-6803) Documenting DFSClient#DFSInputStream expectations reading and preading in concurrent context

[ https://issues.apache.org/jira/browse/HDFS-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093266#comment-14093266 ] Colin Patrick McCabe commented on HDFS-6803: [~stev...@iseran.com]: Hmm. I

[jira] [Commented] (HDFS-6757) Simplify lease manager with INodeID

[ https://issues.apache.org/jira/browse/HDFS-6757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093276#comment-14093276 ] Haohui Mai commented on HDFS-6757: -- Thanks [~cmccabe] and [~daryn] for the comments. Just

[jira] [Commented] (HDFS-6838) Code cleanup for unnecessary INode replacement

[ https://issues.apache.org/jira/browse/HDFS-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093303#comment-14093303 ] Hadoop QA commented on HDFS-6838: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-6838) Code cleanup for unnecessary INode replacement

[ https://issues.apache.org/jira/browse/HDFS-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093319#comment-14093319 ] Haohui Mai commented on HDFS-6838: -- Looks good to me. +1 Code cleanup for unnecessary

[jira] [Updated] (HDFS-6838) Code cleanup for unnecessary INode replacement

[ https://issues.apache.org/jira/browse/HDFS-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-6838: - Resolution: Fixed Fix Version/s: 2.6.0 Hadoop Flags: Reviewed Status: Resolved

[jira] [Commented] (HDFS-6582) Missing null check in RpcProgramNfs3#read(XDR, SecurityHandler)

[ https://issues.apache.org/jira/browse/HDFS-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093337#comment-14093337 ] Brandon Li commented on HDFS-6582: -- +1 Missing null check in RpcProgramNfs3#read(XDR,

[jira] [Commented] (HDFS-6582) Missing null check in RpcProgramNfs3#read(XDR, SecurityHandler)

[ https://issues.apache.org/jira/browse/HDFS-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093339#comment-14093339 ] Brandon Li commented on HDFS-6582: -- I've committed the patch. Thank you, [~abutala], for

[jira] [Updated] (HDFS-6582) Missing null check in RpcProgramNfs3#read(XDR, SecurityHandler)

[ https://issues.apache.org/jira/browse/HDFS-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-6582: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available)

[jira] [Updated] (HDFS-6582) Missing null check in RpcProgramNfs3#read(XDR, SecurityHandler)

[ https://issues.apache.org/jira/browse/HDFS-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-6582: - Fix Version/s: 2.6.0 Missing null check in RpcProgramNfs3#read(XDR, SecurityHandler)

[jira] [Updated] (HDFS-6582) Missing null check in RpcProgramNfs3#read(XDR, SecurityHandler)

[ https://issues.apache.org/jira/browse/HDFS-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-6582: - Affects Version/s: 2.2.0 Missing null check in RpcProgramNfs3#read(XDR, SecurityHandler)

[jira] [Commented] (HDFS-6582) Missing null check in RpcProgramNfs3#read(XDR, SecurityHandler)

[ https://issues.apache.org/jira/browse/HDFS-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093350#comment-14093350 ] Hudson commented on HDFS-6582: -- FAILURE: Integrated in Hadoop-trunk-Commit #6048 (See

[jira] [Commented] (HDFS-6837) Code cleanup for Balancer and Dispatcher

[ https://issues.apache.org/jira/browse/HDFS-6837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093352#comment-14093352 ] Hudson commented on HDFS-6837: -- FAILURE: Integrated in Hadoop-trunk-Commit #6048 (See

[jira] [Commented] (HDFS-6838) Code cleanup for unnecessary INode replacement

[ https://issues.apache.org/jira/browse/HDFS-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093353#comment-14093353 ] Hudson commented on HDFS-6838: -- FAILURE: Integrated in Hadoop-trunk-Commit #6048 (See

[jira] [Updated] (HDFS-6425) Large postponedMisreplicatedBlocks has impact on blockReport latency

[ https://issues.apache.org/jira/browse/HDFS-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated HDFS-6425: -- Attachment: HDFS-6425-2.patch Updated patch for latest trunk. Large postponedMisreplicatedBlocks has impact

[jira] [Updated] (HDFS-6818) FSShell's get operation should have the ability to take a length argument

[ https://issues.apache.org/jira/browse/HDFS-6818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6818: --- Summary: FSShell's get operation should have the ability to take a length argument

[jira] [Commented] (HDFS-6819) make HDFS fault injection framework working with maven

[ https://issues.apache.org/jira/browse/HDFS-6819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093478#comment-14093478 ] Colin Patrick McCabe commented on HDFS-6819: I don't see a lot of value in

[jira] [Commented] (HDFS-6634) inotify in HDFS

[ https://issues.apache.org/jira/browse/HDFS-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093481#comment-14093481 ] Andrew Wang commented on HDFS-6634: --- Thanks for revving James, few more comments. I

[jira] [Commented] (HDFS-6820) Namenode fails to boot if the file system reorders rename operations

[ https://issues.apache.org/jira/browse/HDFS-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093483#comment-14093483 ] Colin Patrick McCabe commented on HDFS-6820: Hmm, good find. You can fix this

[jira] [Updated] (HDFS-6835) Archival Storage: Add a new API to set storage policy

[ https://issues.apache.org/jira/browse/HDFS-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-6835: Attachment: HDFS-6835.000.patch Initial patch. Add a new API setStoragePolicy(String src, String

[jira] [Comment Edited] (HDFS-6134) Transparent data at rest encryption

[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093538#comment-14093538 ] Sanjay Radia edited comment on HDFS-6134 at 8/12/14 12:19 AM: --

[jira] [Commented] (HDFS-6134) Transparent data at rest encryption

[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093538#comment-14093538 ] Sanjay Radia commented on HDFS-6134: .bq. Charles posted a design doc for how distcp

[jira] [Commented] (HDFS-6425) Large postponedMisreplicatedBlocks has impact on blockReport latency

[ https://issues.apache.org/jira/browse/HDFS-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093558#comment-14093558 ] Hadoop QA commented on HDFS-6425: - {color:red}-1 overall{color}. Here are the results of

[jira] [Created] (HDFS-6840) Clients are always sent to the same datanode when read is off rack

Jason Lowe created HDFS-6840: Summary: Clients are always sent to the same datanode when read is off rack Key: HDFS-6840 URL: https://issues.apache.org/jira/browse/HDFS-6840 Project: Hadoop HDFS

[jira] [Commented] (HDFS-6134) Transparent data at rest encryption

[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093579#comment-14093579 ] Sanjay Radia commented on HDFS-6134: Wrt to webhdfs, the document says that the

[jira] [Updated] (HDFS-6774) Make FsDataset and DataStore support removing volumes.

[ https://issues.apache.org/jira/browse/HDFS-6774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-6774: Attachment: HDFS-6774.001.patch Refactor the patch and fix inconsistent comments. Make FsDataset

[jira] [Commented] (HDFS-6840) Clients are always sent to the same datanode when read is off rack

[ https://issues.apache.org/jira/browse/HDFS-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093584#comment-14093584 ] Jason Lowe commented on HDFS-6840: -- HDFS-6701 gives the option to randomize the returned

[jira] [Commented] (HDFS-6134) Transparent data at rest encryption

[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093583#comment-14093583 ] Sanjay Radia commented on HDFS-6134: One of the items raised at the meeting and

[jira] [Commented] (HDFS-6840) Clients are always sent to the same datanode when read is off rack

[ https://issues.apache.org/jira/browse/HDFS-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093595#comment-14093595 ] Andrew Wang commented on HDFS-6840: --- Hey Jason, did you bisect this behavior to

[jira] [Updated] (HDFS-6268) Better sorting in NetworkTopology#pseudoSortByDistance when no local node is found

[ https://issues.apache.org/jira/browse/HDFS-6268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated HDFS-6268: - Fix Version/s: 2.5.0 We also ran into the massive skew issue during localization that [~ashwinshankar77]

[jira] [Commented] (HDFS-6134) Transparent data at rest encryption

[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093603#comment-14093603 ] Andrew Wang commented on HDFS-6134: --- Hey Sanjay, thanks for reviewing things, Regarding

[jira] [Commented] (HDFS-6840) Clients are always sent to the same datanode when read is off rack

[ https://issues.apache.org/jira/browse/HDFS-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093613#comment-14093613 ] Jason Lowe commented on HDFS-6840: -- I think the previous behavior was not deterministic

[jira] [Updated] (HDFS-6839) Fix TestCLI to expect new output

[ https://issues.apache.org/jira/browse/HDFS-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb updated HDFS-6839: --- Status: Patch Available (was: In Progress) HADOOP-10919 changed the output of cp's usage. Submitting

[jira] [Updated] (HDFS-6839) Fix TestCLI to expect new output

[ https://issues.apache.org/jira/browse/HDFS-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb updated HDFS-6839: --- Status: In Progress (was: Patch Available) Fix TestCLI to expect new output

[jira] [Commented] (HDFS-6774) Make FsDataset and DataStore support removing volumes.

[ https://issues.apache.org/jira/browse/HDFS-6774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14093694#comment-14093694 ] Hadoop QA commented on HDFS-6774: - {color:red}-1 overall{color}. Here are the results of

Re: Synchronization among Mappers in map-reduce task

Hi Folks , I have been writing a map-reduce application where I am having an input file containing records and every field in the record is separated by some delimiter. In addition to this user will also provide a list of columns that he wants to lookup in a master properties file (stored