[jira] [Updated] (HDFS-6394) HDFS encryption documentation

2014-08-05 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6394: -- Attachment: hdfs-6394.005.patch Thanks Charles, again incorporated some and made additional tweaks of my

[jira] [Resolved] (HDFS-6394) HDFS encryption documentation

2014-08-05 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-6394. --- Resolution: Fixed Fix Version/s: fs-encryption (HADOOP-10150 and HDFS-6134) Committed, thanks C

[jira] [Created] (HDFS-6824) Additional user documentation for HDFS encryption.

2014-08-05 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-6824: - Summary: Additional user documentation for HDFS encryption. Key: HDFS-6824 URL: https://issues.apache.org/jira/browse/HDFS-6824 Project: Hadoop HDFS Issue Type: Su

[jira] [Commented] (HDFS-6824) Additional user documentation for HDFS encryption.

2014-08-05 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086971#comment-14086971 ] Andrew Wang commented on HDFS-6824: --- [~tucu00] relayed a few clarifications he'd like her

[jira] [Commented] (HDFS-6834) Improve the configuration guidance in DFSClient when there are no Codec classes found in configs

2014-08-07 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14089758#comment-14089758 ] Andrew Wang commented on HDFS-6834: --- +1 thanks Uma > Improve the configuration guidance

[jira] [Commented] (HDFS-6825) Edit log corruption due to delayed block removal

2014-08-08 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14091479#comment-14091479 ] Andrew Wang commented on HDFS-6825: --- Hi Yongjun, tricky bug, but this fix looks good. Jus

[jira] [Commented] (HDFS-6634) inotify in HDFS

2014-08-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093481#comment-14093481 ] Andrew Wang commented on HDFS-6634: --- Thanks for revving James, few more comments. I haven

[jira] [Commented] (HDFS-6840) Clients are always sent to the same datanode when read is off rack

2014-08-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093595#comment-14093595 ] Andrew Wang commented on HDFS-6840: --- Hey Jason, did you bisect this behavior to HDFS-6268

[jira] [Commented] (HDFS-6134) Transparent data at rest encryption

2014-08-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093603#comment-14093603 ] Andrew Wang commented on HDFS-6134: --- Hey Sanjay, thanks for reviewing things, Regarding

[jira] [Commented] (HDFS-6840) Clients are always sent to the same datanode when read is off rack

2014-08-12 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14094340#comment-14094340 ] Andrew Wang commented on HDFS-6840: --- Alright, I'm sufficiently convinced :) Is the fix as

[jira] [Commented] (HDFS-6825) Edit log corruption due to delayed block removal

2014-08-12 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14094977#comment-14094977 ] Andrew Wang commented on HDFS-6825: --- +1 from me as well, thanks for revving Yongjun. Let'

[jira] [Commented] (HDFS-6843) Create FileStatus.isEncrypted() method

2014-08-13 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14095797#comment-14095797 ] Andrew Wang commented on HDFS-6843: --- I looked into how we'd do this, and one major issue

[jira] [Commented] (HDFS-6783) Fix HDFS CacheReplicationMonitor rescan logic

2014-08-13 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096405#comment-14096405 ] Andrew Wang commented on HDFS-6783: --- +1 from me too, thanks for working on this Yi and Co

[jira] [Commented] (HDFS-6843) Create FileStatus.isEncrypted() method

2014-08-14 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097555#comment-14097555 ] Andrew Wang commented on HDFS-6843: --- +1 to that sentiment in the abstract. The other one

[jira] [Assigned] (HDFS-6840) Clients are always sent to the same datanode when read is off rack

2014-08-14 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reassigned HDFS-6840: - Assignee: Andrew Wang > Clients are always sent to the same datanode when read is off rack > -

[jira] [Updated] (HDFS-6825) Edit log corruption due to delayed block removal

2014-08-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6825: -- Resolution: Fixed Fix Version/s: 2.6.0 Status: Resolved (was: Patch Available) Thanks

[jira] [Updated] (HDFS-6134) Transparent data at rest encryption

2014-08-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6134: -- Attachment: fs-encryption.2014-08-18.patch New consolidated patch attached. The merge vote has passed, s

[jira] [Commented] (HDFS-6873) Constants in CommandWithDestination should be static

2014-08-19 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102789#comment-14102789 ] Andrew Wang commented on HDFS-6873: --- +1 > Constants in CommandWithDestination should be

[jira] [Commented] (HDFS-6872) Fix TestOptionsParser

2014-08-19 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14102790#comment-14102790 ] Andrew Wang commented on HDFS-6872: --- +1 > Fix TestOptionsParser > -

[jira] [Updated] (HDFS-6134) Transparent data at rest encryption

2014-08-19 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6134: -- Attachment: fs-encryption.2014-08-19.patch New consolidated patch, Charles cleared out the Jenkins issue

[jira] [Commented] (HDFS-6826) Plugin interface to enable delegation of HDFS authorization assertions

2014-08-19 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103189#comment-14103189 ] Andrew Wang commented on HDFS-6826: --- I don't think we need the addSnapshotDir and removeS

[jira] [Updated] (HDFS-6134) Transparent data at rest encryption

2014-08-19 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6134: -- Attachment: (was: fs-encryption.2014-08-19.patch) > Transparent data at rest encryption > --

[jira] [Updated] (HDFS-6134) Transparent data at rest encryption

2014-08-19 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6134: -- Attachment: fs-encryption.2014-08-19.patch > Transparent data at rest encryption > -

[jira] [Created] (HDFS-6882) Ability to fetch the KMS ACLs for a given key

2014-08-19 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-6882: - Summary: Ability to fetch the KMS ACLs for a given key Key: HDFS-6882 URL: https://issues.apache.org/jira/browse/HDFS-6882 Project: Hadoop HDFS Issue Type: Improve

[jira] [Commented] (HDFS-6882) Ability to fetch the KMS ACLs for a given key

2014-08-19 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103229#comment-14103229 ] Andrew Wang commented on HDFS-6882: --- One suggestion from [~tucu00] was to make this into

[jira] [Created] (HDFS-6883) Multiple Kerberos principals for KMS

2014-08-19 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-6883: - Summary: Multiple Kerberos principals for KMS Key: HDFS-6883 URL: https://issues.apache.org/jira/browse/HDFS-6883 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Commented] (HDFS-6134) Transparent data at rest encryption

2014-08-19 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103250#comment-14103250 ] Andrew Wang commented on HDFS-6134: --- I've filed MAPREDUCE-6040 to address Sanjay's point

[jira] [Created] (HDFS-6884) Include the hostname in HTTPFS log filenames

2014-08-19 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-6884: - Summary: Include the hostname in HTTPFS log filenames Key: HDFS-6884 URL: https://issues.apache.org/jira/browse/HDFS-6884 Project: Hadoop HDFS Issue Type: Improvem

[jira] [Commented] (HDFS-6134) Transparent data at rest encryption

2014-08-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14104266#comment-14104266 ] Andrew Wang commented on HDFS-6134: --- These tests all passed locally for me, so I'm going

[jira] [Updated] (HDFS-6872) Fix TestOptionsParser

2014-08-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6872: -- Issue Type: Bug (was: Sub-task) Parent: (was: HDFS-6134) > Fix TestOptionsParser >

[jira] [Created] (HDFS-6891) Follow-on work for transparent data at rest encryption

2014-08-20 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-6891: - Summary: Follow-on work for transparent data at rest encryption Key: HDFS-6891 URL: https://issues.apache.org/jira/browse/HDFS-6891 Project: Hadoop HDFS Issue Type

[jira] [Updated] (HDFS-6851) Flush EncryptionZoneWithId and add an id field to EncryptionZone

2014-08-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6851: -- Parent Issue: HDFS-6891 (was: HDFS-6134) > Flush EncryptionZoneWithId and add an id field to Encryption

[jira] [Updated] (HDFS-6843) Create FSDataInputStream & FSDataOutputStream isEncrypted() method

2014-08-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6843: -- Parent Issue: HDFS-6891 (was: HDFS-6134) > Create FSDataInputStream & FSDataOutputStream isEncrypted()

[jira] [Updated] (HDFS-6824) Additional user documentation for HDFS encryption.

2014-08-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6824: -- Parent Issue: HDFS-6891 (was: HDFS-6134) > Additional user documentation for HDFS encryption. > ---

[jira] [Updated] (HDFS-6705) Create an XAttr that disallows the HDFS admin from accessing a file

2014-08-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6705: -- Target Version/s: (was: fs-encryption (HADOOP-10150 and HDFS-6134)) > Create an XAttr that disallows t

[jira] [Updated] (HDFS-6668) Add and expose KeyProvider-related metrics on NameNode

2014-08-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6668: -- Affects Version/s: (was: fs-encryption (HADOOP-10150 and HDFS-6134)) 3.0.0 >

[jira] [Updated] (HDFS-6668) Add and expose KeyProvider-related metrics on NameNode

2014-08-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6668: -- Parent Issue: HDFS-6891 (was: HDFS-6134) > Add and expose KeyProvider-related metrics on NameNode > ---

[jira] [Updated] (HDFS-6705) Create an XAttr that disallows the HDFS admin from accessing a file

2014-08-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6705: -- Parent Issue: HDFS-6891 (was: HDFS-6134) > Create an XAttr that disallows the HDFS admin from accessing

[jira] [Updated] (HDFS-6815) Verify that alternate access methods work properly with Data at Rest Encryption

2014-08-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6815: -- Parent Issue: HDFS-6891 (was: HDFS-6134) > Verify that alternate access methods work properly with Data

[jira] [Updated] (HDFS-6668) Add and expose KeyProvider-related metrics on NameNode

2014-08-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6668: -- Target Version/s: (was: fs-encryption (HADOOP-10150 and HDFS-6134)) > Add and expose KeyProvider-relat

[jira] [Updated] (HDFS-6815) Verify that alternate access methods work properly with Data at Rest Encryption

2014-08-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6815: -- Affects Version/s: (was: fs-encryption (HADOOP-10150 and HDFS-6134)) 3.0.0 >

[jira] [Updated] (HDFS-6555) Specify file encryption attributes at create time

2014-08-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6555: -- Parent Issue: HDFS-6891 (was: HDFS-6134) > Specify file encryption attributes at create time >

[jira] [Updated] (HDFS-6705) Create an XAttr that disallows the HDFS admin from accessing a file

2014-08-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6705: -- Affects Version/s: 3.0.0 > Create an XAttr that disallows the HDFS admin from accessing a file > ---

[jira] [Updated] (HDFS-6555) Specify file encryption attributes at create time

2014-08-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6555: -- Affects Version/s: (was: fs-encryption (HADOOP-10150 and HDFS-6134)) 3.0.0 >

[jira] [Updated] (HDFS-6134) Transparent data at rest encryption

2014-08-20 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6134: -- Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) I've m

[jira] [Commented] (HDFS-6581) Write to single replica in memory

2014-08-21 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106245#comment-14106245 ] Andrew Wang commented on HDFS-6581: --- I also took a look at the doc, seems pretty reasonab

[jira] [Created] (HDFS-6947) Enhance HAR integration with encryption zones

2014-08-26 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-6947: - Summary: Enhance HAR integration with encryption zones Key: HDFS-6947 URL: https://issues.apache.org/jira/browse/HDFS-6947 Project: Hadoop HDFS Issue Type: Sub-tas

[jira] [Updated] (HDFS-6891) Follow-on work for transparent data at rest encryption

2014-08-26 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6891: -- Component/s: encryption > Follow-on work for transparent data at rest encryption > -

[jira] [Commented] (HDFS-6705) Create an XAttr that disallows the HDFS admin from accessing a file

2014-08-27 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14112804#comment-14112804 ] Andrew Wang commented on HDFS-6705: --- Hi Charles, thanks for working on this. Had some rev

[jira] [Commented] (HDFS-6843) Create FSDataInputStream & FSDataOutputStream isEncrypted() method

2014-08-27 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14112872#comment-14112872 ] Andrew Wang commented on HDFS-6843: --- Overall LGTM, just one nit: let's not c+p the Javado

[jira] [Commented] (HDFS-6951) Saving namespace and restarting NameNode will remove existing encryption zones

2014-08-27 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14112947#comment-14112947 ] Andrew Wang commented on HDFS-6951: --- Hi Charles, thanks for taking a look at this. Had so

[jira] [Commented] (HDFS-6036) Forcibly timeout misbehaving DFSClients that try to do no-checksum reads that extend too long

2014-08-27 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14113137#comment-14113137 ] Andrew Wang commented on HDFS-6036: --- Hi Colin, nice work here, few review comments: Nitt

[jira] [Updated] (HDFS-6840) Clients are always sent to the same datanode when read is off rack

2014-08-28 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6840: -- Status: Patch Available (was: Open) > Clients are always sent to the same datanode when read is off rac

[jira] [Updated] (HDFS-6840) Clients are always sent to the same datanode when read is off rack

2014-08-28 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6840: -- Attachment: hdfs-6840.001.patch Here's a patch that removes the "random seed" stuff. Fortunately HDFS-67

[jira] [Commented] (HDFS-6860) BlockStateChange logs are too noisy

2014-08-28 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14114304#comment-14114304 ] Andrew Wang commented on HDFS-6860: --- Is this a good time to also switch BlockStateChange

[jira] [Updated] (HDFS-6840) Clients are always sent to the same datanode when read is off rack

2014-08-28 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6840: -- Attachment: hdfs-6840.002.patch Nice catch Yi, new patch that removes that param from the XML file. > C

[jira] [Commented] (HDFS-6634) inotify in HDFS

2014-08-28 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14114511#comment-14114511 ] Andrew Wang commented on HDFS-6634: --- Hey James, nice work here. This is all pretty much c

[jira] [Updated] (HDFS-6968) Configurable replication degree in the hadoop archive command

2014-08-28 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6968: -- Target Version/s: 2.6.0 Affects Version/s: 2.6.0 > Configurable replication degree in the hadoop ar

[jira] [Commented] (HDFS-6968) Configurable replication degree in the hadoop archive command

2014-08-28 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14114523#comment-14114523 ] Andrew Wang commented on HDFS-6968: --- Hi Zhe, patch looks basically good, just a few comme

[jira] [Commented] (HDFS-6843) Create FSDataInputStream & FSDataOutputStream isEncrypted() method

2014-08-28 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14114585#comment-14114585 ] Andrew Wang commented on HDFS-6843: --- Steve, are we now requiring all API changes to also

[jira] [Commented] (HDFS-6954) With crypto, no native lib systems are too verbose

2014-08-28 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14114620#comment-14114620 ] Andrew Wang commented on HDFS-6954: --- Hi Charles, thanks for picking this up (and thanks A

[jira] [Updated] (HDFS-6891) Follow-on work for transparent data at rest encryption

2014-08-28 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6891: -- Target Version/s: 2.6.0 (was: 3.0.0) Affects Version/s: (was: 3.0.0) 2.5

[jira] [Created] (HDFS-6970) Move startFile EDEK retries to the DFSClient

2014-08-28 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-6970: - Summary: Move startFile EDEK retries to the DFSClient Key: HDFS-6970 URL: https://issues.apache.org/jira/browse/HDFS-6970 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-6971) Bounded staleness of EDEK caches on the NN

2014-08-28 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-6971: - Summary: Bounded staleness of EDEK caches on the NN Key: HDFS-6971 URL: https://issues.apache.org/jira/browse/HDFS-6971 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Commented] (HDFS-6954) With crypto, no native lib systems are too verbose

2014-08-28 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14114700#comment-14114700 ] Andrew Wang commented on HDFS-6954: --- Hey Charles, could you use slf4j style "{}" template

[jira] [Commented] (HDFS-6843) Create FSDataInputStream & FSDataOutputStream isEncrypted() method

2014-08-28 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14114772#comment-14114772 ] Andrew Wang commented on HDFS-6843: --- I'm aware, I did review the original FS contract pat

[jira] [Commented] (HDFS-6843) Create FSDataInputStream & FSDataOutputStream isEncrypted() method

2014-08-29 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14115629#comment-14115629 ] Andrew Wang commented on HDFS-6843: --- If we're going to start requiring this, it's somethi

[jira] [Commented] (HDFS-6843) Create FSDataInputStream & FSDataOutputStream isEncrypted() method

2014-08-29 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14115846#comment-14115846 ] Andrew Wang commented on HDFS-6843: --- Hi Steve W, I do not want to raise the bar to code

[jira] [Commented] (HDFS-6634) inotify in HDFS

2014-08-29 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14115862#comment-14115862 ] Andrew Wang commented on HDFS-6634: --- Just one last nitty nit: could we use "{}" template

[jira] [Commented] (HDFS-6972) TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration doesn't decode url correctly

2014-08-29 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14115986#comment-14115986 ] Andrew Wang commented on HDFS-6972: --- Thanks Yongjun, +1 LGTM. Basic issue is that URLs do

[jira] [Updated] (HDFS-6972) TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration doesn't decode url correctly

2014-08-29 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6972: -- Resolution: Fixed Fix Version/s: 2.6.0 Status: Resolved (was: Patch Available) Commit

[jira] [Updated] (HDFS-6634) inotify in HDFS

2014-09-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6634: -- Resolution: Fixed Fix Version/s: 2.6.0 Status: Resolved (was: Patch Available) Committ

[jira] [Updated] (HDFS-6954) With crypto, no native lib systems are too verbose

2014-09-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6954: -- Resolution: Fixed Fix Version/s: 2.6.0 Status: Resolved (was: Patch Available) Committ

[jira] [Created] (HDFS-6987) Move CipherSuite xattr information up to the encryption zone root

2014-09-02 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-6987: - Summary: Move CipherSuite xattr information up to the encryption zone root Key: HDFS-6987 URL: https://issues.apache.org/jira/browse/HDFS-6987 Project: Hadoop HDFS

[jira] [Updated] (HDFS-6986) DistributedFileSystem must get delegation tokens from configured KeyProvider

2014-09-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6986: -- Summary: DistributedFileSystem must get delegation tokens from configured KeyProvider (was: DistributedF

[jira] [Assigned] (HDFS-6971) Bounded staleness of EDEK caches on the NN

2014-09-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reassigned HDFS-6971: - Assignee: Zhe Zhang (was: Andrew Wang) > Bounded staleness of EDEK caches on the NN > ---

[jira] [Updated] (HDFS-6986) DistributedFileSystem must get delegation tokens from configured KeyProvider

2014-09-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6986: -- Assignee: Zhe Zhang > DistributedFileSystem must get delegation tokens from configured KeyProvider >

[jira] [Commented] (HDFS-6987) Move CipherSuite xattr information up to the encryption zone root

2014-09-02 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118884#comment-14118884 ] Andrew Wang commented on HDFS-6987: --- It'd be good to also protobuf the EZ root xattr as p

[jira] [Commented] (HDFS-6036) Forcibly timeout misbehaving DFSClients that try to do no-checksum reads that extend too long

2014-09-03 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120752#comment-14120752 ] Andrew Wang commented on HDFS-6036: --- Any reason you didn't add a metric? I think it shoul

[jira] [Commented] (HDFS-6036) Forcibly timeout misbehaving DFSClients that try to do no-checksum reads that extend too long

2014-09-03 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14120885#comment-14120885 ] Andrew Wang commented on HDFS-6036: --- SGTM thanks Colin, +1 again. > Forcibly timeout mis

[jira] [Created] (HDFS-7004) Update KeyProvider instantiation to create by URI

2014-09-05 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-7004: - Summary: Update KeyProvider instantiation to create by URI Key: HDFS-7004 URL: https://issues.apache.org/jira/browse/HDFS-7004 Project: Hadoop HDFS Issue Type: Sub

[jira] [Commented] (HDFS-6621) Hadoop Balancer prematurely exits iterations

2014-09-05 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123557#comment-14123557 ] Andrew Wang commented on HDFS-6621: --- I took a quick look at this, and I'm wondering about

[jira] [Commented] (HDFS-6951) Saving namespace and restarting NameNode will remove existing encryption zones

2014-09-05 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123703#comment-14123703 ] Andrew Wang commented on HDFS-6951: --- Charles, do you mind rebasing this? It doesn't apply

[jira] [Commented] (HDFS-6705) Create an XAttr that disallows the HDFS admin from accessing a file

2014-09-07 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125209#comment-14125209 ] Andrew Wang commented on HDFS-6705: --- Hi Charles, thanks for working on this, a few review

[jira] [Commented] (HDFS-6843) Create FileStatus isEncrypted() method

2014-09-07 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125228#comment-14125228 ] Andrew Wang commented on HDFS-6843: --- Hi Charles, thanks for sticking with this. Hopefully

[jira] [Commented] (HDFS-6951) Saving namespace and restarting NameNode will remove existing encryption zones

2014-09-07 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125230#comment-14125230 ] Andrew Wang commented on HDFS-6951: --- Blech, it looks like test-patch doesn't like binary

[jira] [Commented] (HDFS-6951) Saving namespace and restarting NameNode will remove existing encryption zones

2014-09-07 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14125240#comment-14125240 ] Andrew Wang commented on HDFS-6951: --- Overall looks good to me, just some test nits: XAtt

[jira] [Updated] (HDFS-6951) Correctly persist raw namespace xattrs to edit log and fsimage

2014-09-08 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6951: -- Summary: Correctly persist raw namespace xattrs to edit log and fsimage (was: Saving namespace and resta

[jira] [Commented] (HDFS-6951) Correctly persist raw namespace xattrs to edit log and fsimage

2014-09-08 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126348#comment-14126348 ] Andrew Wang commented on HDFS-6951: --- Committed to trunk and branch-2, btw. > Correctly p

[jira] [Updated] (HDFS-6951) Correctly persist raw namespace xattrs to edit log and fsimage

2014-09-08 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6951: -- Resolution: Fixed Fix Version/s: 2.6.0 Status: Resolved (was: Patch Available) +1 LGTM

[jira] [Commented] (HDFS-6966) Add additional unit tests for encryption zones

2014-09-08 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126422#comment-14126422 ] Andrew Wang commented on HDFS-6966: --- Nice patch Stephen, lots of good testing happening i

[jira] [Created] (HDFS-7047) Expose FileStatus#isEncrypted in libhdfs

2014-09-11 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-7047: - Summary: Expose FileStatus#isEncrypted in libhdfs Key: HDFS-7047 URL: https://issues.apache.org/jira/browse/HDFS-7047 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Commented] (HDFS-6966) Add additional unit tests for encryption zones

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130329#comment-14130329 ] Andrew Wang commented on HDFS-6966: --- +1 LGTM, will commit shortly. > Add additional unit

[jira] [Commented] (HDFS-6621) Hadoop Balancer prematurely exits iterations

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130337#comment-14130337 ] Andrew Wang commented on HDFS-6621: --- +1 LGTM, I'll commit this shortly. > Hadoop Balance

[jira] [Updated] (HDFS-6966) Add additional unit tests for encryption zones

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6966: -- Resolution: Fixed Fix Version/s: 2.6.0 Status: Resolved (was: Patch Available) Committ

[jira] [Assigned] (HDFS-6621) Hadoop Balancer prematurely exits iterations

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reassigned HDFS-6621: - Assignee: Rafal Wojdyla > Hadoop Balancer prematurely exits iterations > -

[jira] [Updated] (HDFS-6621) Hadoop Balancer prematurely exits iterations

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6621: -- Resolution: Fixed Fix Version/s: 2.6.0 Status: Resolved (was: Patch Available) Committ

[jira] [Commented] (HDFS-6621) Hadoop Balancer prematurely exits iterations

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130355#comment-14130355 ] Andrew Wang commented on HDFS-6621: --- Filed HDFS-7048 for problem 2, hopefully I captured

[jira] [Created] (HDFS-7048) Incorrect Balancer#Source wait/notify leads to early termination

2014-09-11 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-7048: - Summary: Incorrect Balancer#Source wait/notify leads to early termination Key: HDFS-7048 URL: https://issues.apache.org/jira/browse/HDFS-7048 Project: Hadoop HDFS

[jira] [Commented] (HDFS-7045) Fix NameNode deadlock when opening file under /.reserved path

2014-09-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130397#comment-14130397 ] Andrew Wang commented on HDFS-7045: --- +1, nice find Yi. I'll commit this shortly. > Fix N

<    5   6   7   8   9   10   11   12   13   14   >