[jira] [Updated] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

2019-03-18 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14211: --- Attachment: HDFS-14211.002.patch > [Consistent Observer Reads] Allow for configurable "always msync"

[jira] [Commented] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

2019-03-18 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16795221#comment-16795221 ] Erik Krogen commented on HDFS-14211: Hey [~ayushtkn], thanks for your concerns, they

[jira] [Commented] (HDFS-14349) Edit log may be rolled more frequently than necessary with multiple Standby nodes

2019-03-18 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16795105#comment-16795105 ] Erik Krogen commented on HDFS-14349: Hi [~starphin], thanks for the input. I agree th

[jira] [Commented] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

2019-03-14 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792903#comment-16792903 ] Erik Krogen commented on HDFS-14211: The {{TestConsistentReadsObserver}} failure is i

[jira] [Commented] (HDFS-14276) [SBN read] Reduce tailing overhead

2019-03-13 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792101#comment-16792101 ] Erik Krogen commented on HDFS-14276: As [~csun] mentioned, that was a legitimate issu

[jira] [Commented] (HDFS-14370) Edit log tailing fast-path should allow for backoff

2019-03-13 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16792096#comment-16792096 ] Erik Krogen commented on HDFS-14370: [~jojochuang] I would love your input on this >

[jira] [Created] (HDFS-14370) Edit log tailing fast-path should allow for backoff

2019-03-13 Thread Erik Krogen (JIRA)
Erik Krogen created HDFS-14370: -- Summary: Edit log tailing fast-path should allow for backoff Key: HDFS-14370 URL: https://issues.apache.org/jira/browse/HDFS-14370 Project: Hadoop HDFS Issue Typ

[jira] [Updated] (HDFS-14346) Better time precision in getTimeDuration

2019-03-13 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14346: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.3 3.2.1

[jira] [Commented] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

2019-03-13 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791865#comment-16791865 ] Erik Krogen commented on HDFS-14211: I agree that there is potential issue with that

[jira] [Updated] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

2019-03-13 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14211: --- Attachment: HDFS-14211.001.patch > [Consistent Observer Reads] Allow for configurable "always msync"

[jira] [Commented] (HDFS-14346) Better time precision in getTimeDuration

2019-03-13 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791778#comment-16791778 ] Erik Krogen commented on HDFS-14346: Nice, LGTM but I think we can fix these two chec

[jira] [Commented] (HDFS-14346) Better time precision in getTimeDuration

2019-03-12 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791016#comment-16791016 ] Erik Krogen commented on HDFS-14346: Thanks [~csun], makes sense. I gave the patch a

[jira] [Commented] (HDFS-14346) EditLogTailer loses precision for sub-second edit log tailing and rolling interval

2019-03-11 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16789677#comment-16789677 ] Erik Krogen commented on HDFS-14346: Hey [~csun], the approach LGTM. We should probab

[jira] [Updated] (HDFS-14349) Edit log may be rolled more frequently than necessary with multiple Standby nodes

2019-03-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14349: --- Target Version/s: 3.0.4, 3.3.0, 3.2.1, 3.1.3 > Edit log may be rolled more frequently than necessary

[jira] [Updated] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

2019-03-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14317: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Standby does not trigger edit lo

[jira] [Commented] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

2019-03-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16788365#comment-16788365 ] Erik Krogen commented on HDFS-14317: [~ekanth] I filed HDFS-14349 and assigned to you

[jira] [Created] (HDFS-14349) Edit log may be rolled more frequently than necessary with multiple Standby nodes

2019-03-08 Thread Erik Krogen (JIRA)
Erik Krogen created HDFS-14349: -- Summary: Edit log may be rolled more frequently than necessary with multiple Standby nodes Key: HDFS-14349 URL: https://issues.apache.org/jira/browse/HDFS-14349 Project:

[jira] [Commented] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

2019-03-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16788289#comment-16788289 ] Erik Krogen commented on HDFS-14317: I synced with [~ekanth] offline and he informed

[jira] [Updated] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

2019-03-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14211: --- Status: Patch Available (was: In Progress) Just posted up a v000 patch of this. It is pretty simple:

[jira] [Updated] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

2019-03-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14211: --- Attachment: HDFS-14211.000.patch > [Consistent Observer Reads] Allow for configurable "always msync"

[jira] [Commented] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

2019-03-07 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787267#comment-16787267 ] Erik Krogen commented on HDFS-12345: [~jojochuang], I think I agree with you except f

[jira] [Commented] (HDFS-14346) EditLogTailer loses precision for sub-second edit log tailing and rolling interval

2019-03-07 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787226#comment-16787226 ] Erik Krogen commented on HDFS-14346: Hey [~csun], thanks for reporting. This actuall

[jira] [Commented] (HDFS-14084) Need for more stats in DFSClient

2019-03-07 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787206#comment-16787206 ] Erik Krogen commented on HDFS-14084: Hey [~dannytbecker], I think that would fix this

[jira] [Comment Edited] (HDFS-14084) Need for more stats in DFSClient

2019-03-07 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787206#comment-16787206 ] Erik Krogen edited comment on HDFS-14084 at 3/7/19 9:19 PM: H

[jira] [Work started] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

2019-03-07 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-14211 started by Erik Krogen. -- > [Consistent Observer Reads] Allow for configurable "always msync" mode > -

[jira] [Assigned] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

2019-03-07 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reassigned HDFS-14211: -- Assignee: Erik Krogen > [Consistent Observer Reads] Allow for configurable "always msync" mode

[jira] [Updated] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

2019-03-07 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14317: --- Hadoop Flags: Reviewed > Standby does not trigger edit log rolling when in-progress edit log tailing

[jira] [Commented] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

2019-03-07 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786971#comment-16786971 ] Erik Krogen commented on HDFS-14317: Thanks for the explanation, [~ekanth]. +1 from m

[jira] [Updated] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

2019-03-07 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14317: --- Fix Version/s: 3.1.3 3.2.1 3.3.0 3.0.4 > Sta

[jira] [Comment Edited] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

2019-03-06 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785876#comment-16785876 ] Erik Krogen edited comment on HDFS-14317 at 3/6/19 5:10 PM: H

[jira] [Commented] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

2019-03-06 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785876#comment-16785876 ] Erik Krogen commented on HDFS-14317: Hey [~ekanth], can you explain why the new test

[jira] [Commented] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

2019-03-05 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16784597#comment-16784597 ] Erik Krogen commented on HDFS-12345: Hey [~smeng], as you're pursuing this more activ

[jira] [Assigned] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

2019-03-05 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reassigned HDFS-12345: -- Assignee: Siyao Meng (was: Erik Krogen) > Scale testing HDFS NameNode with real metadata and

[jira] [Updated] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

2019-03-01 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14272: --- Resolution: Fixed Fix Version/s: 3.3.0 Status: Resolved (was: Patch Available) > [

[jira] [Commented] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

2019-03-01 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16782064#comment-16782064 ] Erik Krogen commented on HDFS-14272: Based on the +1 from [~shv] and the earlier revi

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-03-01 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16781851#comment-16781851 ] Erik Krogen commented on HDFS-14305: Backported to branch-3.2, branch-3.1, branch-3.0

[jira] [Updated] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-03-01 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14305: --- Fix Version/s: 3.1.3 3.2.1 3.0.4 > Serial number in BlockTokenS

[jira] [Commented] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

2019-03-01 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16781829#comment-16781829 ] Erik Krogen commented on HDFS-14317: {quote} the value is converted to seconds which

[jira] [Commented] (HDFS-14084) Need for more stats in DFSClient

2019-03-01 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16781768#comment-16781768 ] Erik Krogen commented on HDFS-14084: Sounds good, thanks [~jojochuang] :) > Need for

[jira] [Commented] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

2019-02-28 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16781094#comment-16781094 ] Erik Krogen commented on HDFS-14317: {quote} Unfortunately, dfs.ha.tail-edits.period

[jira] [Commented] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

2019-02-28 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16780994#comment-16780994 ] Erik Krogen commented on HDFS-12345: Hm. I wonder if it even makes sense for MiniDFSC

[jira] [Commented] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

2019-02-28 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16780780#comment-16780780 ] Erik Krogen commented on HDFS-14317: The production code all LGTM. I have a few comme

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-28 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16780746#comment-16780746 ] Erik Krogen commented on HDFS-14305: [~arpitagarwal], thanks for asking, it's a good

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-28 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16780742#comment-16780742 ] Erik Krogen commented on HDFS-14305: Thanks [~hexiaoqiao], I agree. I updated the rel

[jira] [Updated] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-28 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14305: --- Resolution: Fixed Fix Version/s: 3.3.0 Status: Resolved (was: Patch Available) > S

[jira] [Updated] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-28 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14305: --- Release Note: NameNodes rely on independent block token key ranges to communicate block token identit

[jira] [Updated] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-28 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14305: --- Release Note: NameNodes rely on independent block token key ranges to communicate block token identit

[jira] [Commented] (HDFS-14084) Need for more stats in DFSClient

2019-02-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16779791#comment-16779791 ] Erik Krogen commented on HDFS-14084: Hi [~pranay_singh], do you plan to complete this

[jira] [Commented] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

2019-02-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16779663#comment-16779663 ] Erik Krogen commented on HDFS-14317: Gotcha. Thank you for the detailed explanation -

[jira] [Updated] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14305: --- Release Note: NameNodes rely on independent block token key ranges to communicate block token identit

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16779521#comment-16779521 ] Erik Krogen commented on HDFS-14305: v006 patch LGTM. I agree that test failures are

[jira] [Updated] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14305: --- Component/s: namenode > Serial number in BlockTokenSecretManager could overlap between different > n

[jira] [Commented] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

2019-02-26 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778226#comment-16778226 ] Erik Krogen commented on HDFS-14317: {quote} I presume the reason why it also removes

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-26 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778137#comment-16778137 ] Erik Krogen commented on HDFS-14305: It looks fine to me. [~vagarychen] has done some

[jira] [Commented] (HDFS-14317) Standby does not trigger edit log rolling when in-progress edit log tailing is enabled

2019-02-26 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778130#comment-16778130 ] Erik Krogen commented on HDFS-14317: Hey [~ekanth], thanks for reporting this. It see

[jira] [Updated] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

2019-02-25 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14272: --- Attachment: (was: HDFS-14272.002.patch) > [SBN read] ObserverReadProxyProvider should sync with a

[jira] [Updated] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

2019-02-25 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14272: --- Attachment: HDFS-14272.002.patch > [SBN read] ObserverReadProxyProvider should sync with active txnID

[jira] [Commented] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

2019-02-25 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777406#comment-16777406 ] Erik Krogen commented on HDFS-14272: [~shv]: {code} Since proxies to individual NNs h

[jira] [Updated] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

2019-02-25 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14272: --- Attachment: HDFS-14272.002.patch > [SBN read] ObserverReadProxyProvider should sync with active txnID

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-25 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777046#comment-16777046 ] Erik Krogen commented on HDFS-14305: The idea seems sound to me. The helper method fo

[jira] [Commented] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

2019-02-22 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775639#comment-16775639 ] Erik Krogen commented on HDFS-14279: Thanks for the review, [~shv]. I just committed

[jira] [Updated] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

2019-02-22 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14279: --- Resolution: Fixed Fix Version/s: 3.3.0 Status: Resolved (was: Patch Available) > [

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-02-22 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775329#comment-16775329 ] Erik Krogen commented on HDFS-14305: Thanks for reporting this [~csun]. Given that th

[jira] [Commented] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

2019-02-21 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774602#comment-16774602 ] Erik Krogen commented on HDFS-14279: [~shv], does the new patch look good to you? >

[jira] [Commented] (HDFS-14130) Make ZKFC ObserverNode aware

2019-02-21 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774286#comment-16774286 ] Erik Krogen commented on HDFS-14130: Agreed that {{System.in}} is not the right appro

[jira] [Comment Edited] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

2019-02-21 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774265#comment-16774265 ] Erik Krogen edited comment on HDFS-14272 at 2/21/19 4:28 PM: -

[jira] [Commented] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

2019-02-21 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774265#comment-16774265 ] Erik Krogen commented on HDFS-14272: {quote} msync() on active NN if it is a new clie

[jira] [Commented] (HDFS-14292) Introduce Java ExecutorService to DataXceiverServer

2019-02-21 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16774216#comment-16774216 ] Erik Krogen commented on HDFS-14292: [~kihwal] or [~daryn], I'm curious if either of

[jira] [Commented] (HDFS-14130) Make ZKFC ObserverNode aware

2019-02-20 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773366#comment-16773366 ] Erik Krogen commented on HDFS-14130: In the new test, it seems weird to be mixing the

[jira] [Commented] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

2019-02-20 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773276#comment-16773276 ] Erik Krogen commented on HDFS-14279: I ran some benchmarks offline and found that the

[jira] [Updated] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

2019-02-20 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14279: --- Attachment: HDFS-14279.001.patch > [SBN Read] Race condition in ObserverReadProxyProvider > -

[jira] [Comment Edited] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

2019-02-20 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773262#comment-16773262 ] Erik Krogen edited comment on HDFS-14272 at 2/20/19 6:37 PM: -

[jira] [Updated] (HDFS-14272) [SBN read] ObserverReadProxyProvider should sync with active txnID on startup

2019-02-20 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14272: --- Summary: [SBN read] ObserverReadProxyProvider should sync with active txnID on startup (was: [SBN re

[jira] [Commented] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

2019-02-20 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773262#comment-16773262 ] Erik Krogen commented on HDFS-14272: [~shv], [~vagarychen], [~csun], appreciate if an

[jira] [Commented] (HDFS-14297) Add cache for getContentSummary() result

2019-02-20 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773160#comment-16773160 ] Erik Krogen commented on HDFS-14297: Sure, it seems reasonable to me. I see the value

[jira] [Commented] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-02-20 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773146#comment-16773146 ] Erik Krogen commented on HDFS-14216: Sure, I see the logic for using DEBUG level. I'm

[jira] [Commented] (HDFS-14292) Introduce Java ExecutorService to DataXceiverServer

2019-02-19 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16772457#comment-16772457 ] Erik Krogen commented on HDFS-14292: Hi [~belugabehr], the reasoning makes sense to m

[jira] [Commented] (HDFS-14297) Add cache for getContentSummary() result

2019-02-19 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16772449#comment-16772449 ] Erik Krogen commented on HDFS-14297: Hi [~Tao Jie], do the directories you are monito

[jira] [Commented] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-02-19 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16772109#comment-16772109 ] Erik Krogen commented on HDFS-14216: Hi [~xiaoheipangzi], thanks for fixing this impo

[jira] [Commented] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

2019-02-19 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16772079#comment-16772079 ] Erik Krogen commented on HDFS-14279: The intent is to avoid routing through a {{synch

[jira] [Commented] (HDFS-14284) RBF: Log Router identifier when reporting exceptions

2019-02-15 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16769816#comment-16769816 ] Erik Krogen commented on HDFS-14284: Makes sense to me. IIRC that information may not

[jira] [Commented] (HDFS-14284) RBF: Log Router identifier when reporting exceptions

2019-02-15 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16769714#comment-16769714 ] Erik Krogen commented on HDFS-14284: Interesting problem. Can you maybe post a sample

[jira] [Commented] (HDFS-14277) [SBN read] Observer benchmark results

2019-02-14 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768706#comment-16768706 ] Erik Krogen commented on HDFS-14277: Okay, interesting. Seems that it's at least wort

[jira] [Commented] (HDFS-14162) Balancer should work with ObserverNode

2019-02-14 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768668#comment-16768668 ] Erik Krogen commented on HDFS-14162: Just committed to trunk based off of Konstantin'

[jira] [Updated] (HDFS-14162) Balancer should work with ObserverNode

2019-02-14 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14162: --- Resolution: Fixed Fix Version/s: 3.3.0 Status: Resolved (was: Patch Available) > B

[jira] [Commented] (HDFS-14277) [SBN read] Observer benchmark results

2019-02-14 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768666#comment-16768666 ] Erik Krogen commented on HDFS-14277: Thanks Wei-Chiu. I don't see any calls to {{isCo

[jira] [Commented] (HDFS-14277) [SBN read] Observer benchmark results

2019-02-14 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768629#comment-16768629 ] Erik Krogen commented on HDFS-14277: Thanks for reporting this [~jojochuang]! Do you

[jira] [Comment Edited] (HDFS-14277) [SBN read] Observer benchmark results

2019-02-14 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768629#comment-16768629 ] Erik Krogen edited comment on HDFS-14277 at 2/14/19 7:18 PM: -

[jira] [Commented] (HDFS-14276) [SBN read] Reduce tailing overhead

2019-02-14 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768627#comment-16768627 ] Erik Krogen commented on HDFS-14276: Hey [~jojochuang], great find, thanks for report

[jira] [Updated] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

2019-02-14 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14272: --- Attachment: HDFS-14272.001.patch > [SBN read] HDFS command line tools does not guarantee consistency

[jira] [Commented] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

2019-02-14 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768530#comment-16768530 ] Erik Krogen commented on HDFS-14272: Attached v001 patch which adds some optimization

[jira] [Updated] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

2019-02-14 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14279: --- Status: Patch Available (was: Open) Attaching v000 patch which addresses this problem by having {{ch

[jira] [Updated] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

2019-02-14 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14279: --- Attachment: HDFS-14279.000.patch > [SBN Read] Race condition in ObserverReadProxyProvider > -

[jira] [Created] (HDFS-14279) [SBN Read] Race condition in ObserverReadProxyProvider

2019-02-14 Thread Erik Krogen (JIRA)
Erik Krogen created HDFS-14279: -- Summary: [SBN Read] Race condition in ObserverReadProxyProvider Key: HDFS-14279 URL: https://issues.apache.org/jira/browse/HDFS-14279 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-14262) [SBN read] Unclear Log.WARN message in GlobalStateIdContext

2019-02-14 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16768470#comment-16768470 ] Erik Krogen commented on HDFS-14262: +1 from me. Just committed to trunk. Thanks [~sh

[jira] [Updated] (HDFS-14262) [SBN read] Unclear Log.WARN message in GlobalStateIdContext

2019-02-14 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14262: --- Resolution: Fixed Status: Resolved (was: Patch Available) > [SBN read] Unclear Log.WARN mess

[jira] [Updated] (HDFS-14262) [SBN read] Unclear Log.WARN message in GlobalStateIdContext

2019-02-14 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14262: --- Fix Version/s: 3.3.0 > [SBN read] Unclear Log.WARN message in GlobalStateIdContext >

[jira] [Commented] (HDFS-14162) Balancer should work with ObserverNode

2019-02-13 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767660#comment-16767660 ] Erik Krogen commented on HDFS-14162: Attached a v004 patch addressing the checkstyle

[jira] [Updated] (HDFS-14162) Balancer should work with ObserverNode

2019-02-13 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14162: --- Attachment: HDFS-14162.004.patch > Balancer should work with ObserverNode > -

[jira] [Commented] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

2019-02-13 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16767613#comment-16767613 ] Erik Krogen commented on HDFS-14272: This problem is harder than I thought. Initiall

<    2   3   4   5   6   7   8   9   10   11   >