[jira] [Updated] (HDFS-16540) Data locality is lost when DataNode pod restarts in kubernetes

2023-09-21 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HDFS-16540: - Fix Version/s: (was: 3.3.5) > Data locality is lost when DataNode pod restarts in kubernetes

[jira] [Commented] (HDFS-16540) Data locality is lost when DataNode pod restarts in kubernetes

2023-09-21 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767668#comment-17767668 ] Michael Stack commented on HDFS-16540: -- Let me do the latter [~dmanning] ...  I'll let folks ask for

[jira] [Commented] (HDFS-16540) Data locality is lost when DataNode pod restarts in kubernetes

2023-09-21 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767640#comment-17767640 ] Michael Stack commented on HDFS-16540: -- [~dmanning] You are right. Looks like I messed up the

[jira] [Commented] (HDFS-16755) Unit test can fail due to unexpected host resolution

2022-08-31 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17598612#comment-17598612 ] Michael Stack commented on HDFS-16755: -- {quote}Could someone transform HADOOP-18431 to HDFS-*?

[jira] [Moved] (HDFS-16755) Unit test can fail due to unexpected host resolution

2022-08-31 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack moved HADOOP-18431 to HDFS-16755: --- Component/s: test (was: test)

[jira] [Resolved] (HDFS-16684) Exclude self from JournalNodeSyncer when using a bind host

2022-08-28 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HDFS-16684. -- Hadoop Flags: Reviewed Resolution: Fixed Merged to trunk and branch-3.3. Resolving.

[jira] [Resolved] (HDFS-16586) Purge FsDatasetAsyncDiskService threadgroup; it causes BPServiceActor$CommandProcessingThread IllegalThreadStateException 'fatal exception and exit'

2022-05-25 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HDFS-16586. -- Fix Version/s: 3.4.0 3.2.4 3.3.4 Hadoop Flags:

[jira] [Commented] (HDFS-14997) BPServiceActor processes commands from NameNode asynchronously

2022-05-20 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17540321#comment-17540321 ] Michael Stack commented on HDFS-14997: -- Linking an issue where the change in how we do command

[jira] [Created] (HDFS-16586) Purge FsDatasetAsyncDiskService threadgroup; it causes BPServiceActor$CommandProcessingThread IllegalThreadStateException 'fatal exception and exit'

2022-05-20 Thread Michael Stack (Jira)
Michael Stack created HDFS-16586: Summary: Purge FsDatasetAsyncDiskService threadgroup; it causes BPServiceActor$CommandProcessingThread IllegalThreadStateException 'fatal exception and exit' Key: HDFS-16586

[jira] [Resolved] (HDFS-16540) Data locality is lost when DataNode pod restarts in kubernetes

2022-05-15 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HDFS-16540. -- Hadoop Flags: Reviewed Resolution: Fixed Merged to branch-3.3. and to trunk. > Data

[jira] [Updated] (HDFS-16540) Data locality is lost when DataNode pod restarts in kubernetes

2022-05-15 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HDFS-16540: - Fix Version/s: 3.4.0 3.3.4 > Data locality is lost when DataNode pod restarts

[jira] [Assigned] (HDFS-16540) Data locality is lost when DataNode pod restarts in kubernetes

2022-04-13 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack reassigned HDFS-16540: Assignee: Huaxiang Sun > Data locality is lost when DataNode pod restarts in kubernetes

[jira] [Updated] (HDFS-16090) Fine grained locking for datanodeNetworkCounts

2021-06-30 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HDFS-16090: - Fix Version/s: 3.3.2 3.4.0 Hadoop Flags: Reviewed Resolution:

[jira] [Commented] (HDFS-13613) RegionServer log is flooded with "Execution rejected, Executing in current thread"

2019-11-07 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969596#comment-16969596 ] Michael Stack commented on HDFS-13613: -- Thanks [~ndimiduk] and [~inigoiri] for taking a look.

[jira] [Commented] (HDFS-13613) RegionServer log is flooded with "Execution rejected, Executing in current thread"

2019-11-04 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967028#comment-16967028 ] Michael Stack commented on HDFS-13613: -- Attached proposed patch. The log is useless. Besides, we

[jira] [Updated] (HDFS-13613) RegionServer log is flooded with "Execution rejected, Executing in current thread"

2019-11-04 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack updated HDFS-13613: - Attachment: 0001-HDFS-13613-RegionServer-log-is-flooded-with-Executio.patch > RegionServer log

[jira] [Comment Edited] (HDFS-13613) RegionServer log is flooded with "Execution rejected, Executing in current thread"

2019-11-04 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967021#comment-16967021 ] Michael Stack edited comment on HDFS-13613 at 11/4/19 9:40 PM: --- Just ran

[jira] [Comment Edited] (HDFS-13613) RegionServer log is flooded with "Execution rejected, Executing in current thread"

2019-11-04 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967021#comment-16967021 ] Michael Stack edited comment on HDFS-13613 at 11/4/19 9:39 PM: --- Just ran

[jira] [Commented] (HDFS-13613) RegionServer log is flooded with "Execution rejected, Executing in current thread"

2019-11-04 Thread Michael Stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967021#comment-16967021 ] Michael Stack commented on HDFS-13613: -- Just ran into this one. Thread dump showded loads of

[jira] [Commented] (HDFS-14837) Review of Block.java

2019-09-09 Thread stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926301#comment-16926301 ] stack commented on HDFS-14837: -- One question, is Long.hashCode same as (int)(blockId^(blockId>>>32)) (I've

[jira] [Commented] (HDFS-14837) Review of Block.java

2019-09-09 Thread stack (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926214#comment-16926214 ] stack commented on HDFS-14837: -- +1 nice cleanup > Review of Block.java > > >

[jira] [Commented] (HDFS-14483) Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9

2019-07-09 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881339#comment-16881339 ] stack commented on HDFS-14483: -- Reverted from branch-2 and branch-2.9 and then reapplied to both branches

[jira] [Updated] (HDFS-14483) Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9

2019-07-09 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HDFS-14483: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.9.3 2.10.0

[jira] [Commented] (HDFS-14483) Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9

2019-07-07 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880012#comment-16880012 ] stack commented on HDFS-14483: -- [~leosun08] Thanks. Looking at history of hdfs builds, I see that it files

[jira] [Comment Edited] (HDFS-14483) Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9

2019-07-07 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880012#comment-16880012 ] stack edited comment on HDFS-14483 at 7/8/19 3:52 AM: -- [~leosun08] Thanks. Looking

[jira] [Commented] (HDFS-14483) Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9

2019-07-07 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880008#comment-16880008 ] stack commented on HDFS-14483: -- ...and +1 on patch. Lets just figure the story on this last flakey...and

[jira] [Updated] (HDFS-14483) Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9

2019-07-07 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HDFS-14483: - Attachment: HDFS-14585.branch-2.9.v3.patch > Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to

[jira] [Commented] (HDFS-14483) Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9

2019-07-07 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880004#comment-16880004 ] stack commented on HDFS-14483: -- Thanks for fixing the short circuit unit test [~leosun08]. Seems to have

[jira] [Updated] (HDFS-14483) Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9

2019-07-06 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HDFS-14483: - Attachment: HDFS-14585.branch-2.9.v3.patch > Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to

[jira] [Updated] (HDFS-14483) Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9

2019-07-03 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HDFS-14483: - Attachment: HDFS-14483.branch-2.9.v2 (2).patch > Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to

[jira] [Commented] (HDFS-14483) Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9

2019-07-03 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877963#comment-16877963 ] stack commented on HDFS-14483: -- I looked back over recent hdfs qa builds

[jira] [Commented] (HDFS-14483) Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9

2019-07-02 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877402#comment-16877402 ] stack commented on HDFS-14483: -- Are the test failures related [~leosun08]? Thanks. > Backport

[jira] [Updated] (HDFS-14483) Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9

2019-07-02 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HDFS-14483: - Attachment: HDFS-14483.branch-2.9.v2.patch > Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to

[jira] [Commented] (HDFS-14483) Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9

2019-07-02 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877023#comment-16877023 ] stack commented on HDFS-14483: -- What about my other comments [~leosun08]? Mind responding to them? There is

[jira] [Commented] (HDFS-14483) Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9

2019-07-01 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876673#comment-16876673 ] stack commented on HDFS-14483: -- Retry. All but one failure look like they could be related. Lets see. >

[jira] [Updated] (HDFS-14483) Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to branch-2.9

2019-07-01 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HDFS-14483: - Attachment: HDFS-14483.branch-2.9.v1.patch > Backport HDFS-14111,HDFS-3246 ByteBuffer pread interface to

[jira] [Resolved] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9

2019-07-01 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HDFS-14585. -- Resolution: Fixed Reapplied w/ proper commit message. Re-resolving. > Backport HDFS-8901 Use ByteBuffer in

[jira] [Reopened] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9

2019-07-01 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack reopened HDFS-14585: -- Reopening. Commit message was missing the JIRA # so revert and reapply with fixed commit message. > Backport

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9

2019-07-01 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HDFS-14585: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.9.3 2.10.0

[jira] [Commented] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9

2019-06-29 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16875570#comment-16875570 ] stack commented on HDFS-14585: -- Yes. It passed on second attempt. Flakey. The findbugs has a covering JIRA

[jira] [Updated] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9

2019-06-28 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HDFS-14585: - Attachment: HDFS-14585.branch-2.9.v2.patch > Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to

[jira] [Commented] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9

2019-06-28 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16875047#comment-16875047 ] stack commented on HDFS-14585: -- bq. Sorry, I don't quite understand thse meaning of this comments. I am

[jira] [Commented] (HDFS-14483) Backport HDFS-3246 ByteBuffer pread interface to branch-2.8.x

2019-06-27 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874426#comment-16874426 ] stack commented on HDFS-14483: -- Test failures seem related, no? nit: Was expecting

[jira] [Commented] (HDFS-14585) Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2 and branch2.9

2019-06-27 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874403#comment-16874403 ] stack commented on HDFS-14585: -- nit: why the change in import order for nonnull and Preconditions? Is it

[jira] [Commented] (HDFS-3246) pRead equivalent for direct read path

2019-03-01 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781902#comment-16781902 ] stack commented on HDFS-3246: - This patch looks beautiful going by the Interface. Being able to ask HDFS fill

[jira] [Commented] (HDFS-13702) HTrace hooks taking 10-15% CPU in DFS client when disabled

2018-06-27 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525612#comment-16525612 ] stack commented on HDFS-13702: -- bq. I do want a trace layer in there; I do want it broader than just HDFS,

[jira] [Commented] (HDFS-13643) Implement basic async rpc client

2018-06-26 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524429#comment-16524429 ] stack commented on HDFS-13643: -- Yeah, we can take a look at [~daryn] stuff when it shows up. On the patch,

[jira] [Commented] (HDFS-13702) HTrace hooks taking 10-15% CPU in DFS client when disabled

2018-06-26 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16524156#comment-16524156 ] stack commented on HDFS-13702: -- bq. what do you think? I think we need to be able to trace end-to-end

[jira] [Commented] (HDFS-13643) Implement basic async rpc client

2018-06-04 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500387#comment-16500387 ] stack commented on HDFS-13643: -- [~daryn] Thanks boss. Would like to see it. > Implement basic async rpc

[jira] [Updated] (HDFS-13643) Implement basic async rpc client

2018-05-31 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HDFS-13643: - Fix Version/s: HDFS-13572 > Implement basic async rpc client > > >

[jira] [Commented] (HDFS-13643) Implement basic async rpc client

2018-05-31 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497356#comment-16497356 ] stack commented on HDFS-13643: -- Thanks for the input [~daryn] bq. It's a nice POC but not implementing

[jira] [Commented] (HDFS-13572) [umbrella] Non-blocking HDFS Access for H3

2018-05-31 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497220#comment-16497220 ] stack commented on HDFS-13572: -- I made a branch named for this issue by cloning trunk ('trunk' is head and

[jira] [Commented] (HDFS-13572) [umbrella] Non-blocking HDFS Access for H3

2018-05-30 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16495291#comment-16495291 ] stack commented on HDFS-13572: -- Unless objection, was planning on creating a branch to work on this issue.

[jira] [Resolved] (HDFS-13565) [um

2018-05-16 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HDFS-13565. -- Resolution: Invalid Smile [~ebadger] Yeah, sorry about that lads. Bad wifi. Resolving as invalid. > [um >

[jira] [Commented] (HDFS-9924) [umbrella] Nonblocking HDFS Access

2018-05-15 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476812#comment-16476812 ] stack commented on HDFS-9924: - I moved the design doc over to a new issue, HDFS-13572, for the new effort

[jira] [Created] (HDFS-13572) [umbrella] Non-blocking HDFS Access for H3

2018-05-15 Thread stack (JIRA)
stack created HDFS-13572: Summary: [umbrella] Non-blocking HDFS Access for H3 Key: HDFS-13572 URL: https://issues.apache.org/jira/browse/HDFS-13572 Project: Hadoop HDFS Issue Type: New Feature

[jira] [Created] (HDFS-13565) [um

2018-05-15 Thread stack (JIRA)
stack created HDFS-13565: Summary: [um Key: HDFS-13565 URL: https://issues.apache.org/jira/browse/HDFS-13565 Project: Hadoop HDFS Issue Type: New Feature Reporter: stack -- This

[jira] [Commented] (HDFS-7240) Object store in HDFS

2017-11-06 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16240584#comment-16240584 ] stack commented on HDFS-7240: - The posted document needs author, date, and ref to this issue. Can it be made a

[jira] [Commented] (HDFS-12711) deadly hdfs test

2017-11-03 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16238181#comment-16238181 ] stack commented on HDFS-12711: -- bq. For now, though, I'm sort of tired at looking at this problem and will go

[jira] [Commented] (HDFS-12711) deadly hdfs test

2017-11-03 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16238056#comment-16238056 ] stack commented on HDFS-12711: -- This is excellent work. Would a kill -QUIT before you do actual kill of the

[jira] [Commented] (HDFS-12711) deadly hdfs test

2017-10-25 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16219178#comment-16219178 ] stack commented on HDFS-12711: -- Rah rah [~aw]! Thanks for digging in. > deadly hdfs test >

[jira] [Commented] (HDFS-11644) DFSStripedOutputStream should not implement Syncable

2017-04-13 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968113#comment-15968113 ] stack commented on HDFS-11644: -- bq. "The current behavior of throwing an exception is safer." ... but

[jira] [Commented] (HDFS-11170) Add create API in filesystem public class to support assign parameter through builder

2017-03-15 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927487#comment-15927487 ] stack commented on HDFS-11170: -- I took a look at 03 * CreateBuilder.java needs license. * I was going to

[jira] [Commented] (HDFS-6450) Support non-positional hedged reads in HDFS

2017-03-08 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902444#comment-15902444 ] stack commented on HDFS-6450: - [~elgoiri] Ignore my comment above. I thought this the resolved positional

[jira] [Commented] (HDFS-6450) Support non-positional hedged reads in HDFS

2017-03-08 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901691#comment-15901691 ] stack commented on HDFS-6450: - Open a new issue I'd say [~elgoiri]. Link it back here. > Support

[jira] [Commented] (HDFS-9924) [umbrella] Nonblocking HDFS Access

2017-02-03 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851725#comment-15851725 ] stack commented on HDFS-9924: - [~Apache9] Good luck (I didn't fully grok #3. It would be coolio if you could

[jira] [Commented] (HDFS-9924) [umbrella] Nonblocking HDFS Access

2017-01-25 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839393#comment-15839393 ] stack commented on HDFS-9924: - bq. Add a port unification service in front of the grpc server and the old rpc

[jira] [Created] (HDFS-11368) LocalFS does not allow setting storage policy so spew running in local mode

2017-01-25 Thread stack (JIRA)
stack created HDFS-11368: Summary: LocalFS does not allow setting storage policy so spew running in local mode Key: HDFS-11368 URL: https://issues.apache.org/jira/browse/HDFS-11368 Project: Hadoop HDFS

[jira] [Commented] (HDFS-11303) Hedged read might hang infinitely if read data from all DN failed

2017-01-24 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836371#comment-15836371 ] stack commented on HDFS-11303: -- Hey [~alicezhangchen] You got an email when Andrew made state changes to this

[jira] [Updated] (HDFS-11303) Hedged read might hang infinitely if read data from all DN failed

2017-01-24 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HDFS-11303: - Attachment: HDFS-11303-001.patch Retry > Hedged read might hang infinitely if read data from all DN failed >

[jira] [Commented] (HDFS-11303) Hedged read might hang infinitely if read data from all DN failed

2017-01-09 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812572#comment-15812572 ] stack commented on HDFS-11303: -- Patch LGTM. Your patch allows that the primary read might still complete

[jira] [Commented] (HDFS-10690) Optimize insertion/removal of replica in ShortCircuitCache.java

2016-09-28 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531228#comment-15531228 ] stack commented on HDFS-10690: -- Skimmed. Patch LGTM. Unfortunate we leave behind some perf but agree on

[jira] [Commented] (HDFS-9924) [umbrella] Nonblocking HDFS Access

2016-06-17 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15336228#comment-15336228 ] stack commented on HDFS-9924: - [~steve_l] Thanks for the context. Need to make sure this usecase and its

[jira] [Commented] (HDFS-9924) [umbrella] Nonblocking HDFS Access

2016-06-16 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15335420#comment-15335420 ] stack commented on HDFS-9924: - [~xiaobingo] Thanks for posting the compare. It helps. It looks like the

[jira] [Commented] (HDFS-9924) [umbrella] Nonblocking HDFS Access

2016-06-16 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333922#comment-15333922 ] stack commented on HDFS-9924: - +1 for a branch. Otherwise we have this mess going on late in a mature branch

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-06-15 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332670#comment-15332670 ] stack commented on HDFS-9924: - bq. Quoting Tsz Wo Nicholas Sze words, I understand your concern but it is a

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-06-14 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330867#comment-15330867 ] stack commented on HDFS-9924: - I see. Thank you. I see what you want now. You just need renames or you need

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-06-14 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15330173#comment-15330173 ] stack commented on HDFS-9924: - bq. There are multiple comments from both sides indicating that

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-06-13 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328974#comment-15328974 ] stack commented on HDFS-9924: - Your summary and characterization of where the discussion is at is not correct

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-06-13 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15328290#comment-15328290 ] stack commented on HDFS-9924: - [~ashutoshc] Can you make a bit of a better argument than citing the mighty

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-06-09 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15322864#comment-15322864 ] stack commented on HDFS-9924: - bq. This JIRA and the current implementation originally aim to support a basic

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-06-06 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317717#comment-15317717 ] stack commented on HDFS-9924: - Ugh. I meant to say, we risk having different 'async' API/implementations if we

[jira] [Commented] (HDFS-9924) [umbrella] Asynchronous HDFS Access

2016-06-06 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316850#comment-15316850 ] stack commented on HDFS-9924: - I'd suggest we work out a coherent, global filesystem async API/strategy before

[jira] [Commented] (HDFS-7240) Object store in HDFS

2016-05-26 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15302344#comment-15302344 ] stack commented on HDFS-7240: - bq. It is unfair to say that you are being rebuffed. Can we please move to

[jira] [Commented] (HDFS-7240) Object store in HDFS

2016-05-26 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15302191#comment-15302191 ] stack commented on HDFS-7240: - bq. Now, can people stop being territorial or making any form of criticism of

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-04-29 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264590#comment-15264590 ] stack commented on HDFS-3702: - Any chance of getting this on 2.8 branch? Thanks. > Add an option for NOT

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-04-13 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15239291#comment-15239291 ] stack commented on HDFS-3702: - bq. Suppose we find that the CreateFlag.NO_LOCAL_WRITE is bad. How do we remove

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-04-12 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15237558#comment-15237558 ] stack commented on HDFS-3702: - bq. I suggest HBase should do the same way of how it today is using

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-04-08 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15233363#comment-15233363 ] stack commented on HDFS-3702: - bq. I suggest adding a new create(..) method to DistributedFileSystem, either

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-04-08 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15231686#comment-15231686 ] stack commented on HDFS-3702: - bq. So let add this flag later so that it allows us to test the feature and see

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-23 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15209177#comment-15209177 ] stack commented on HDFS-3702: - I skimmed #9 patch. Seems good to me other than issues [~szetszwo] raises (we

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-23 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15209164#comment-15209164 ] stack commented on HDFS-3702: - bq. stack, let's add a boolean noLoaclWrite to DistributedFileSystem or just

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-22 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207327#comment-15207327 ] stack commented on HDFS-3702: - And more(getting an emotional) a downstreamer is hampered spending

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-22 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207307#comment-15207307 ] stack commented on HDFS-3702: - bq. I am very uncomfortable to add CreateFlag.NO_LOCAL_WRITE and AddBlockFlag

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-22 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207267#comment-15207267 ] stack commented on HDFS-3702: - [~szetszwo] [~arpitagarwal] is -0 if bq. AddBlockFlag should be tagged as

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-22 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207034#comment-15207034 ] stack commented on HDFS-3702: - bq. If the region server has write permissions on /hbase/.logs, which I assume

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-21 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205361#comment-15205361 ] stack commented on HDFS-3702: - bq. ...could you comment on the usability of providing node lists to this API?

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-21 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205343#comment-15205343 ] stack commented on HDFS-3702: - bq. Hi stack, the attribute could be set by an installer script or an API call

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-21 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205337#comment-15205337 ] stack commented on HDFS-3702: - bq. How would DFSClient know which nodes are disfavored nodes? How could it

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-03-21 Thread stack (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205142#comment-15205142 ] stack commented on HDFS-3702: - For uniform distribution of files over a cluster, I think users would prefer

  1   2   3   4   5   >