[jira] [Created] (HDFS-10758) ReconfigurableBase can log sensitive information

2016-08-12 Thread Sean Mackrory (JIRA)
Sean Mackrory created HDFS-10758: Summary: ReconfigurableBase can log sensitive information Key: HDFS-10758 URL: https://issues.apache.org/jira/browse/HDFS-10758 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-09-01 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15456729#comment-15456729 ] Sean Mackrory commented on HDFS-10797: -- Just a note for anyone reading my comments above, I learned

[jira] [Updated] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-09-14 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10797: - Attachment: HDFS-10797.001.patch Attaching a patch that tries to identify files where the

[jira] [Updated] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-09-15 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10797: - Assignee: Sean Mackrory Status: Patch Available (was: Open) > Disk usage summary of

[jira] [Updated] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-09-15 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10797: - Attachment: HDFS-10797.002.patch Adding a doc comment to the test and fixing the checkstyle

[jira] [Updated] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-09-29 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10797: - Status: Open (was: Patch Available) > Disk usage summary of snapshots causes renamed blocks to

[jira] [Updated] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-09-29 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10797: - Attachment: HDFS-10797.005.patch Now attaching a patch that should give what I think we agree are

[jira] [Updated] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-09-29 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10797: - Attachment: HDFS-10797.004.patch Attaching the patch I meant to attach yesterday: this adds test

[jira] [Comment Edited] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-09-29 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15533078#comment-15533078 ] Sean Mackrory edited comment on HDFS-10797 at 9/29/16 3:27 PM: --- Now

[jira] [Updated] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-09-29 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10797: - Attachment: HDFS-10797.006.patch Great! During manual testing, I found a bug where if you

[jira] [Updated] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-09-29 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10797: - Status: Patch Available (was: Open) > Disk usage summary of snapshots causes renamed blocks to

[jira] [Commented] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-10-05 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549321#comment-15549321 ] Sean Mackrory commented on HDFS-10797: -- Thanks, [~xiaochen]. Except as noted below, I'll incorporate

[jira] [Updated] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-10-05 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10797: - Attachment: HDFS-10797.010.patch > Disk usage summary of snapshots causes renamed blocks to get

[jira] [Updated] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-10-05 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10797: - Attachment: HDFS-10797.010.patch > Disk usage summary of snapshots causes renamed blocks to get

[jira] [Commented] (HDFS-9926) ozone : Add volume commands to CLI

2016-09-22 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15513649#comment-15513649 ] Sean Mackrory commented on HDFS-9926: - Note for future git log greppers: HDFS-9226 was committed with

[jira] [Updated] (HDFS-10877) Make RemoteEditLogManifest.committedTxnId optional in Protocol Buffers

2016-09-20 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10877: - Attachment: HDFS-10877.001.patch Attaching a patch that modifies the .proto definition and the

[jira] [Moved] (HDFS-10877) Make RemoteEditLogManifest.committedTxnId optional in Protocol Buffers

2016-09-20 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory moved HADOOP-13629 to HDFS-10877: --- Issue Type: Improvement (was: Bug) Key: HDFS-10877 (was:

[jira] [Updated] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-09-16 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10797: - Attachment: HDFS-10797.003.patch Thanks for the review, [~xiaochen]. I do think that test is

[jira] [Updated] (HDFS-10877) Make RemoteEditLogManifest.committedTxnId optional in Protocol Buffers

2016-09-21 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10877: - Attachment: HDFS-10877.002.patch > Make RemoteEditLogManifest.committedTxnId optional in Protocol

[jira] [Created] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-08-25 Thread Sean Mackrory (JIRA)
Sean Mackrory created HDFS-10797: Summary: Disk usage summary of snapshots causes renamed blocks to get counted twice Key: HDFS-10797 URL: https://issues.apache.org/jira/browse/HDFS-10797 Project:

[jira] [Commented] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-08-26 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15439451#comment-15439451 ] Sean Mackrory commented on HDFS-10797: -- It's worth pointing out that the df command also shows some

[jira] [Commented] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-08-25 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15438128#comment-15438128 ] Sean Mackrory commented on HDFS-10797: -- To reproduce the discrepancy you can follow the following

[jira] [Commented] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-09-28 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531199#comment-15531199 ] Sean Mackrory commented on HDFS-10797: -- Thanks for pointing that out [~jingzhao]. I added test cases

[jira] [Updated] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-09-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10797: - Attachment: HDFS-10797.007.patch Thanks, [~xiaochen]. Removed the count variable and fixed the

[jira] [Updated] (HDFS-7642) NameNode should periodically log DataNode decommissioning progress

2016-10-03 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-7642: Attachment: HDFS-7642.001.patch Attaching an initial implementation. I'm wondering if hostname is

[jira] [Commented] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-09-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15536805#comment-15536805 ] Sean Mackrory commented on HDFS-10797: -- findbugs output is correct: I'm using synchronization to

[jira] [Assigned] (HDFS-7642) NameNode should periodically log DataNode decommissioning progress

2016-09-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory reassigned HDFS-7642: --- Assignee: Sean Mackrory > NameNode should periodically log DataNode decommissioning progress

[jira] [Updated] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-09-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10797: - Attachment: HDFS-10797.008.patch Attaching a patch that address the findbugs warnings. > Disk

[jira] [Commented] (HDFS-7642) NameNode should periodically log DataNode decommissioning progress

2016-09-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15536701#comment-15536701 ] Sean Mackrory commented on HDFS-7642: - Hey [~zhz] - I'd like to work on this, if that's alright with

[jira] [Updated] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-09-30 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10797: - Attachment: HDFS-10797.009.patch Fixing style issues, and always using getCounts() to retrieve

[jira] [Commented] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2016-10-26 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15608805#comment-15608805 ] Sean Mackrory commented on HDFS-10702: -- I think we should also consider ways to not send requests to

[jira] [Comment Edited] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2016-10-26 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15608805#comment-15608805 ] Sean Mackrory edited comment on HDFS-10702 at 10/26/16 3:39 PM: I think we

[jira] [Commented] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2016-11-15 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15667607#comment-15667607 ] Sean Mackrory commented on HDFS-10702: -- {quote}My concern is that if a significant portion of read

[jira] [Updated] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-11-15 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10797: - Release Note: Disk usage summaries previously incorrectly counted files twice if they had been

[jira] [Created] (HDFS-11077) RequestHedgingProxyProvider does not behave correctly with more than 3+ NameNodes

2016-10-31 Thread Sean Mackrory (JIRA)
Sean Mackrory created HDFS-11077: Summary: RequestHedgingProxyProvider does not behave correctly with more than 3+ NameNodes Key: HDFS-11077 URL: https://issues.apache.org/jira/browse/HDFS-11077

[jira] [Created] (HDFS-11080) Update HttpFS to use ConfigRedactor

2016-11-01 Thread Sean Mackrory (JIRA)
Sean Mackrory created HDFS-11080: Summary: Update HttpFS to use ConfigRedactor Key: HDFS-11080 URL: https://issues.apache.org/jira/browse/HDFS-11080 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-11080) Update HttpFS to use ConfigRedactor

2016-11-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-11080: - Attachment: HDFS-11080.001.patch Attaching a patch. Some keys were already redacted, so added

[jira] [Updated] (HDFS-11080) Update HttpFS to use ConfigRedactor

2016-11-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-11080: - Target Version/s: 3.0.0-alpha2 Status: Patch Available (was: Open) > Update HttpFS

[jira] [Updated] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2016-11-03 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10702: - Attachment: HDFS-10797.006.patch I had a conversation with [~clouderajiayi] about taking this

[jira] [Updated] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2016-11-03 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10702: - Attachment: (was: HDFS-10797.006.patch) > Add a Client API and Proxy Provider to enable stale

[jira] [Updated] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2016-11-03 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10702: - Attachment: HDFS-10702.006.patch It might be helpful if I don't submit patches for other JIRAs.

[jira] [Updated] (HDFS-10797) Disk usage summary of snapshots causes renamed blocks to get counted twice

2016-10-10 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10797: - Release Note: Disk usage summaries previously incorrectly counted files twice if they had been

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2016-12-15 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15752889#comment-15752889 ] Sean Mackrory commented on HDFS-11096: -- I've been through the JACC output [~andrew.wang] linked to

[jira] [Comment Edited] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-01-10 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816405#comment-15816405 ] Sean Mackrory edited comment on HDFS-11096 at 1/10/17 10:46 PM: After

[jira] [Updated] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2017-01-10 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10702: - Attachment: HDFS-10702.007.patch Attaching a patch that I believe wraps up everyone's feedback so

[jira] [Comment Edited] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2017-01-10 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816062#comment-15816062 ] Sean Mackrory edited comment on HDFS-10702 at 1/10/17 8:25 PM: --- Attaching a

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-01-10 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816405#comment-15816405 ] Sean Mackrory commented on HDFS-11096: -- After looking at where SortedMapWritable is used some more,

[jira] [Created] (HDFS-11312) Discrepancy in nonDfsUsed index in protobuf

2017-01-10 Thread Sean Mackrory (JIRA)
Sean Mackrory created HDFS-11312: Summary: Discrepancy in nonDfsUsed index in protobuf Key: HDFS-11312 URL: https://issues.apache.org/jira/browse/HDFS-11312 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-11312) Discrepancy in nonDfsUsed index in protobuf

2017-01-10 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-11312: - Attachment: HDFS-11312.001.patch > Discrepancy in nonDfsUsed index in protobuf >

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-01-10 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816505#comment-15816505 ] Sean Mackrory commented on HDFS-11096: -- The getHdfsBlockLocations removal is documented in HDFS-8895

[jira] [Comment Edited] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-01-05 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15799802#comment-15799802 ] Sean Mackrory edited comment on HDFS-11096 at 1/5/17 8:03 PM: -- I've been

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-01-05 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802380#comment-15802380 ] Sean Mackrory commented on HDFS-11096: -- In the SortedMapWritable case, the class is actually still

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-01-04 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15799802#comment-15799802 ] Sean Mackrory commented on HDFS-11096: -- I've been doing a lot of testing. I've posted some automation

[jira] [Updated] (HDFS-12151) Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes

2017-07-28 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-12151: - Attachment: HDFS-12151.005.patch > Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes >

[jira] [Updated] (HDFS-12151) Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes

2017-08-01 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-12151: - Resolution: Fixed Fix Version/s: 3.0.0-beta1 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-12151) Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes

2017-08-01 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109715#comment-16109715 ] Sean Mackrory commented on HDFS-12151: -- Pushed, as [~andrew.wang]'s "pendings" have been resolved.

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-07-17 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089926#comment-16089926 ] Sean Mackrory commented on HDFS-11096: -- Filed HDFS-12151 after looking into the logs deeper than I

[jira] [Created] (HDFS-12151) Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes

2017-07-17 Thread Sean Mackrory (JIRA)
Sean Mackrory created HDFS-12151: Summary: Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes Key: HDFS-12151 URL: https://issues.apache.org/jira/browse/HDFS-12151 Project: Hadoop HDFS

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-07-10 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080798#comment-16080798 ] Sean Mackrory commented on HDFS-11096: -- For anyone following this thread, I've come back to this and

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-07-10 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16081348#comment-16081348 ] Sean Mackrory commented on HDFS-11096: -- That's a good idea, [~rchiang]. Adding in some delays between

[jira] [Updated] (HDFS-12151) Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes

2017-07-17 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-12151: - Affects Version/s: 3.0.0-alpha4 Target Version/s: 3.0.0-beta1 Status: Patch

[jira] [Commented] (HDFS-12151) Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes

2017-07-17 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090677#comment-16090677 ] Sean Mackrory commented on HDFS-12151: -- The problem is in lines added by HDFS-9807. When serving

[jira] [Updated] (HDFS-12151) Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes

2017-07-17 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-12151: - Attachment: HDFS-12151.001.patch Attaching trivial patch. Yetus complains that this doesn't

[jira] [Updated] (HDFS-12151) Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes

2017-07-17 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-12151: - Description: Trying to write to a Hadoop 3 DataNode with a Hadoop 2 client currently fails. On

[jira] [Updated] (HDFS-12151) Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes

2017-07-27 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-12151: - Attachment: HDFS-12151.003.patch Attaching a patch with the checkstyle issues fixed and also

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-07-27 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103867#comment-16103867 ] Sean Mackrory commented on HDFS-11096: -- Updating: it was a recent change that broke this, I've posted

[jira] [Updated] (HDFS-12151) Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes

2017-07-26 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-12151: - Attachment: HDFS-12151.002.patch Attaching a test that ensures you can pass in null-ish values

[jira] [Commented] (HDFS-11661) GetContentSummary uses excessive amounts of memory

2017-04-21 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15979551#comment-15979551 ] Sean Mackrory commented on HDFS-11661: -- +1 to the revert - I too would still like to see the original

[jira] [Updated] (HDFS-12151) Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes

2017-07-28 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-12151: - Attachment: HDFS-12151.006.patch So the problem was that it couldn't connect to a socket on the

[jira] [Updated] (HDFS-12151) Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes

2017-07-31 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-12151: - Attachment: HDFS-12151.007.patch > Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes >

[jira] [Commented] (HDFS-12151) Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes

2017-07-31 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108308#comment-16108308 ] Sean Mackrory commented on HDFS-12151: -- Ah sorry about that - I seem to be blind to the yellow

[jira] [Comment Edited] (HDFS-9806) Allow HDFS block replicas to be provided by an external storage system

2017-08-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118654#comment-16118654 ] Sean Mackrory edited comment on HDFS-9806 at 8/8/17 5:14 PM: - Could someone

[jira] [Commented] (HDFS-9806) Allow HDFS block replicas to be provided by an external storage system

2017-08-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118654#comment-16118654 ] Sean Mackrory commented on HDFS-9806: - Could someone more familiar with the design here comment on any

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-08-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118936#comment-16118936 ] Sean Mackrory commented on HDFS-11096: -- Just pushed some more updates. HDFS-12151 is fixed, and I'm

[jira] [Updated] (HDFS-12151) Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes

2017-07-27 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-12151: - Attachment: HDFS-12151.004.patch I apologize for the noise everyone, but since I can't reproduce

[jira] [Assigned] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2017-06-20 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory reassigned HDFS-10702: Assignee: Sean Mackrory (was: Jiayi Zhou) > Add a Client API and Proxy Provider to enable

[jira] [Updated] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2017-06-21 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10702: - Attachment: HDFS-10702.008.patch Rebasing patch on more recent changes. I had also discovered

[jira] [Updated] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2017-06-21 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-10702: - Status: Open (was: Patch Available) > Add a Client API and Proxy Provider to enable stale read

[jira] [Comment Edited] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2017-09-19 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16171883#comment-16171883 ] Sean Mackrory edited comment on HDFS-10702 at 9/19/17 3:38 PM: --- The

[jira] [Commented] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2017-09-19 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16171883#comment-16171883 ] Sean Mackrory commented on HDFS-10702: -- The assumption of this feature is that an application is

[jira] [Comment Edited] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2017-09-19 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16171883#comment-16171883 ] Sean Mackrory edited comment on HDFS-10702 at 9/19/17 3:36 PM: --- The

[jira] [Updated] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-09-15 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-11096: - Attachment: HDFS-11096.003.patch So attaching a 3rd patch. YARN appears to be failing, so I need

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-10-02 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188872#comment-16188872 ] Sean Mackrory commented on HDFS-11096: -- Thanks, [~atm]! I have also not forgotten the comment about

[jira] [Created] (HDFS-12595) Isolation of native libraries in JARs

2017-10-05 Thread Sean Mackrory (JIRA)
Sean Mackrory created HDFS-12595: Summary: Isolation of native libraries in JARs Key: HDFS-12595 URL: https://issues.apache.org/jira/browse/HDFS-12595 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-09-08 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158780#comment-16158780 ] Sean Mackrory commented on HDFS-11096: -- Thanks for the reviews. I was hoping you'd take a look [~aw]!

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-09-07 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157122#comment-16157122 ] Sean Mackrory commented on HDFS-11096: -- {quote} It would be good to document the order of running the

[jira] [Updated] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-09-06 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-11096: - Attachment: HDFS-11096.002.patch Attaching a patch with a bunch of improvements: * Addresses most

[jira] [Commented] (HDFS-7878) API - expose an unique file identifier

2017-09-26 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181721#comment-16181721 ] Sean Mackrory commented on HDFS-7878: - {quote}Neither API has been part of a release, yet. {quote} But

[jira] [Comment Edited] (HDFS-7878) API - expose an unique file identifier

2017-09-26 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181634#comment-16181634 ] Sean Mackrory edited comment on HDFS-7878 at 9/26/17 9:41 PM: -- Looks like this

[jira] [Commented] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2017-09-26 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181648#comment-16181648 ] Sean Mackrory commented on HDFS-10702: -- I was just chatting with [~yzhangal] about this feature and

[jira] [Commented] (HDFS-7878) API - expose an unique file identifier

2017-09-26 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181634#comment-16181634 ] Sean Mackrory commented on HDFS-7878: - Looks like this patch is violating some of compatibility

[jira] [Updated] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-09-29 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-11096: - Attachment: HDFS-11096.004.patch .004 patch: * Added more helpful logging so set -v and set -x

[jira] [Comment Edited] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-08-28 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16143965#comment-16143965 ] Sean Mackrory edited comment on HDFS-11096 at 8/28/17 4:40 PM: --- So Docker

[jira] [Updated] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-08-28 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-11096: - Attachment: HDFS-11096.001.patch So Docker support has been added for the rolling-upgrade and

[jira] [Updated] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-08-28 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-11096: - Status: Patch Available (was: Open) > Support rolling upgrade between 2.x and 3.x >

[jira] [Updated] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-10-11 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Mackrory updated HDFS-11096: - Attachment: HDFS-11096.005.patch * Restored the old hadoop_actual_ssh. I did try a few cases of

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-11-01 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234197#comment-16234197 ] Sean Mackrory commented on HDFS-11096: -- >From an HDFS standpoint, definitely - I've run many

[jira] [Comment Edited] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-11-01 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234197#comment-16234197 ] Sean Mackrory edited comment on HDFS-11096 at 11/1/17 3:16 PM: --- >From an

[jira] [Commented] (HDFS-11096) Support rolling upgrade between 2.x and 3.x

2017-11-09 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246667#comment-16246667 ] Sean Mackrory commented on HDFS-11096: -- Note that the fix to the sbin/start-yarn.sh shell script has

[jira] [Commented] (HDFS-10702) Add a Client API and Proxy Provider to enable stale read from Standby

2017-12-05 Thread Sean Mackrory (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278900#comment-16278900 ] Sean Mackrory commented on HDFS-10702: -- Any other comments on the consistency issue, [~mingma] /

  1   2   >