[jira] [Updated] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14272: --- Status: Patch Available (was: Open) > [SBN read] HDFS command line tools does not guarantee

[jira] [Comment Edited] (HDFS-14271) [SBN read] StandbyException is logged if Observer is the first NameNode

[ https://issues.apache.org/jira/browse/HDFS-14271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16767454#comment-16767454 ] Erik Krogen edited comment on HDFS-14271 at 2/13/19 6:45 PM: - Hey

[jira] [Commented] (HDFS-14271) [SBN read] StandbyException is logged if Observer is the first NameNode

[ https://issues.apache.org/jira/browse/HDFS-14271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16767454#comment-16767454 ] Erik Krogen commented on HDFS-14271: Hey [~jojochuang], thanks for reporting. What I found from

[jira] [Commented] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16767413#comment-16767413 ] Erik Krogen commented on HDFS-14272: Hey [~jojochuang], thanks for reporting. This should be handled.

[jira] [Commented] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16767418#comment-16767418 ] Erik Krogen commented on HDFS-14272: I don't see us handling this logic anywhere. I think we need to

[jira] [Assigned] (HDFS-14272) [SBN read] HDFS command line tools does not guarantee consistency

[ https://issues.apache.org/jira/browse/HDFS-14272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reassigned HDFS-14272: -- Assignee: Erik Krogen > [SBN read] HDFS command line tools does not guarantee consistency >

[jira] [Comment Edited] (HDFS-14262) [SBN read] Unclear Log.WARN message in GlobalStateIdContext

[ https://issues.apache.org/jira/browse/HDFS-14262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16767393#comment-16767393 ] Erik Krogen edited comment on HDFS-14262 at 2/13/19 5:21 PM: - [~brahmareddy]

[jira] [Commented] (HDFS-14262) [SBN read] Unclear Log.WARN message in GlobalStateIdContext

[ https://issues.apache.org/jira/browse/HDFS-14262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16767393#comment-16767393 ] Erik Krogen commented on HDFS-14262: [~brahmareddy] It should never happen. This log statement was

[jira] [Updated] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14162: --- Attachment: HDFS-14162.003.patch > Balancer should work with ObserverNode >

[jira] [Comment Edited] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16767374#comment-16767374 ] Erik Krogen edited comment on HDFS-14162 at 2/13/19 4:38 PM: - I ran a JMH

[jira] [Commented] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16767374#comment-16767374 ] Erik Krogen commented on HDFS-14162: I ran a JMH benchmark to see about performance differences (

[jira] [Updated] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14162: --- Attachment: ReflectionBenchmark.java > Balancer should work with ObserverNode >

[jira] [Commented] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16766553#comment-16766553 ] Erik Krogen commented on HDFS-14162: Thanks [~shv]. I just attached v002 proxy with the pluralization

[jira] [Updated] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14162: --- Attachment: HDFS-14162.002.patch > Balancer should work with ObserverNode >

[jira] [Commented] (HDFS-14262) [SBN read] Unclear Log.WARN message in GlobalStateIdContext

[ https://issues.apache.org/jira/browse/HDFS-14262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16765297#comment-16765297 ] Erik Krogen commented on HDFS-14262: Thanks for the ping [~jojochuang]! I don't think throwing an

[jira] [Commented] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763971#comment-16763971 ] Erik Krogen commented on HDFS-14162: I had some discussion offline with [~shv] and he pointed me

[jira] [Updated] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14162: --- Attachment: HDFS-14162.001.patch > Balancer should work with ObserverNode >

[jira] [Commented] (HDFS-14250) [SBN read] msync should sync with active NameNode to fetch the latest stateID

[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16761988#comment-16761988 ] Erik Krogen commented on HDFS-14250: Jenkins failure does not look related to this patch: {code}

[jira] [Updated] (HDFS-14250) [SBN read] msync should sync with active NameNode to fetch the latest stateID

[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14250: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.3.0 Status: Resolved

[jira] [Commented] (HDFS-14250) [SBN read] msync should sync with active NameNode to fetch the latest stateID

[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16761980#comment-16761980 ] Erik Krogen commented on HDFS-14250: The exceptions do not look related.

[jira] [Comment Edited] (HDFS-14250) [SBN read] msync should sync with active NameNode to fetch the latest stateID

[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16761980#comment-16761980 ] Erik Krogen edited comment on HDFS-14250 at 2/6/19 5:55 PM: The test failures

[jira] [Commented] (HDFS-14245) Class cast error in GetGroups with ObserverReadProxyProvider

[ https://issues.apache.org/jira/browse/HDFS-14245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16761968#comment-16761968 ] Erik Krogen commented on HDFS-14245: HAProxy will attempt to create an {{ObserverReadProxyProvider}},

[jira] [Commented] (HDFS-14084) Need for more stats in DFSClient

[ https://issues.apache.org/jira/browse/HDFS-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758738#comment-16758738 ] Erik Krogen commented on HDFS-14084: Also, if the metrics system _is_ initialized properly, you get

[jira] [Commented] (HDFS-14084) Need for more stats in DFSClient

[ https://issues.apache.org/jira/browse/HDFS-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758718#comment-16758718 ] Erik Krogen commented on HDFS-14084: Hey folks, I was testing out this patch and noticed an issue

[jira] [Commented] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758715#comment-16758715 ] Erik Krogen commented on HDFS-14250: Great ! +1 from me pending Jenkins, will commit early next week

[jira] [Commented] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16758643#comment-16758643 ] Erik Krogen commented on HDFS-14250: v000 patch LGTM, only minor nit is that I would probably switch

[jira] [Commented] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757722#comment-16757722 ] Erik Krogen commented on HDFS-14250: I don't think we want to make it write anything per se, but IIRC

[jira] [Commented] (HDFS-14250) [Standby Reads] msync should sync with active NameNode to fetch the latest stateID

[ https://issues.apache.org/jira/browse/HDFS-14250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16757704#comment-16757704 ] Erik Krogen commented on HDFS-14250: I think #2 is not necessary -- this will be handled

[jira] [Commented] (HDFS-14238) A log in NNThroughputBenchmark should change log level to "INFO" instead of "ERROR"

[ https://issues.apache.org/jira/browse/HDFS-14238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16755307#comment-16755307 ] Erik Krogen commented on HDFS-14238: I could be wrong, but I always interpreted this as an attempt to

[jira] [Commented] (HDFS-14214) IMax and IMin is not work in MutableRatesWithAggregation

[ https://issues.apache.org/jira/browse/HDFS-14214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746614#comment-16746614 ] Erik Krogen commented on HDFS-14214: Hi [~xuzq_zander], thanks for reporting this issue.

[jira] [Commented] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

[ https://issues.apache.org/jira/browse/HDFS-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746509#comment-16746509 ] Erik Krogen commented on HDFS-14211: {quote} Does that mean every read operation will be lead to

[jira] [Commented] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

[ https://issues.apache.org/jira/browse/HDFS-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744383#comment-16744383 ] Erik Krogen commented on HDFS-14211: Thanks for looking [~csun]! {quote}Currently the msync doesn't

[jira] [Updated] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

[ https://issues.apache.org/jira/browse/HDFS-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14211: --- Description: To allow for reads to be serviced from an ObserverNode (see HDFS-12943) in a

[jira] [Created] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

Erik Krogen created HDFS-14211: -- Summary: [Consistent Observer Reads] Allow for configurable "always msync" mode Key: HDFS-14211 URL: https://issues.apache.org/jira/browse/HDFS-14211 Project: Hadoop

[jira] [Commented] (HDFS-13688) Introduce msync API call

[ https://issues.apache.org/jira/browse/HDFS-13688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744299#comment-16744299 ] Erik Krogen commented on HDFS-13688: Hey [~hexiaoqiao], that's a great question. It is definitely on

[jira] [Updated] (HDFS-14211) [Consistent Observer Reads] Allow for configurable "always msync" mode

[ https://issues.apache.org/jira/browse/HDFS-14211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14211: --- Description: To allow for reads to be serviced from an ObserverNode (see HDFS-12943) in a

[jira] [Updated] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-12345: --- Status: Patch Available (was: In Progress) It took a while to finally get to this, but I'm happy to

[jira] [Updated] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-12345: --- Summary: Scale testing HDFS NameNode with real metadata and workloads (Dynamometer) (was: Scale

[jira] [Updated] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-12345: --- Attachment: HDFS-12345.000.patch > Scale testing HDFS NameNode with real metadata and workloads

[jira] [Work started] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads

[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-12345 started by Erik Krogen. -- > Scale testing HDFS NameNode with real metadata and workloads >

[jira] [Commented] (HDFS-14186) blockreport storm slow down namenode restart seriously in large cluster

[ https://issues.apache.org/jira/browse/HDFS-14186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742313#comment-16742313 ] Erik Krogen commented on HDFS-14186: [~kihwal] - Can you elaborate more on the below? How did you

[jira] [Commented] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740767#comment-16740767 ] Erik Krogen commented on HDFS-14162: Thanks for looking [~shv]. Unfortunately, not quite. Let's take

[jira] [Assigned] (HDFS-12533) NNThroughputBenchmark threads get stuck on UGI.getCurrentUser()

[ https://issues.apache.org/jira/browse/HDFS-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reassigned HDFS-12533: -- Assignee: (was: Erik Krogen) > NNThroughputBenchmark threads get stuck on

[jira] [Commented] (HDFS-12533) NNThroughputBenchmark threads get stuck on UGI.getCurrentUser()

[ https://issues.apache.org/jira/browse/HDFS-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736522#comment-16736522 ] Erik Krogen commented on HDFS-12533: I looked into this a bit further, and attempted to fix it by

[jira] [Assigned] (HDFS-12533) NNThroughputBenchmark threads get stuck on UGI.getCurrentUser()

[ https://issues.apache.org/jira/browse/HDFS-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reassigned HDFS-12533: -- Assignee: (was: Erik Krogen) > NNThroughputBenchmark threads get stuck on

[jira] [Assigned] (HDFS-12533) NNThroughputBenchmark threads get stuck on UGI.getCurrentUser()

[ https://issues.apache.org/jira/browse/HDFS-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reassigned HDFS-12533: -- Assignee: Erik Krogen > NNThroughputBenchmark threads get stuck on UGI.getCurrentUser() >

[jira] [Work stopped] (HDFS-12533) NNThroughputBenchmark threads get stuck on UGI.getCurrentUser()

[ https://issues.apache.org/jira/browse/HDFS-12533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-12533 stopped by Erik Krogen. -- > NNThroughputBenchmark threads get stuck on UGI.getCurrentUser() >

[jira] [Commented] (HDFS-13923) Add a configuration to turn on/off observer reads

[ https://issues.apache.org/jira/browse/HDFS-13923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736017#comment-16736017 ] Erik Krogen commented on HDFS-13923: {quote} The cache never returns the same file system object in

[jira] [Updated] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14162: --- Attachment: HDFS-14162.000.patch > Balancer should work with ObserverNode >

[jira] [Updated] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14162: --- Status: Patch Available (was: Open) Attaching v000 patch implementing essentially what Konstantin

[jira] [Assigned] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reassigned HDFS-14162: -- Assignee: Erik Krogen > Balancer should work with ObserverNode >

[jira] [Commented] (HDFS-13946) Log longest FSN write/read lock held stack trace

[ https://issues.apache.org/jira/browse/HDFS-13946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726981#comment-16726981 ] Erik Krogen commented on HDFS-13946: [~elgoiri] or [~jojochuang], are either of you interested in

[jira] [Commented] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726395#comment-16726395 ] Erik Krogen commented on HDFS-14162: I attached a quick-and-dirty demonstration patch

[jira] [Updated] (HDFS-14162) Balancer should work with ObserverNode

[ https://issues.apache.org/jira/browse/HDFS-14162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14162: --- Attachment: HDFS-14162-HDFS-12943.wip0.patch > Balancer should work with ObserverNode >

[jira] [Commented] (HDFS-14154) Document dfs.ha.tail-edits.period in user guide.

[ https://issues.apache.org/jira/browse/HDFS-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725307#comment-16725307 ] Erik Krogen commented on HDFS-14154: +1, new v001 LGTM! > Document dfs.ha.tail-edits.period in user

[jira] [Commented] (HDFS-14154) Document dfs.ha.tail-edits.period in user guide.

[ https://issues.apache.org/jira/browse/HDFS-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725289#comment-16725289 ] Erik Krogen commented on HDFS-14154: v1 patch still has "cingure" typo :) > Document

[jira] [Commented] (HDFS-14154) Document dfs.ha.tail-edits.period in user guide.

[ https://issues.apache.org/jira/browse/HDFS-14154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725200#comment-16725200 ] Erik Krogen commented on HDFS-14154: Hey Chao, thanks for tackling this. You have one typo, "cingure"

[jira] [Commented] (HDFS-12943) Consistent Reads from Standby Node

[ https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723295#comment-16723295 ] Erik Krogen commented on HDFS-12943: {quote}Bytheway My intent was when read/write are combined in

[jira] [Resolved] (HDFS-14155) Update "Consistent Read from Observer" User Guide with Edit Tailing Frequency

[ https://issues.apache.org/jira/browse/HDFS-14155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-14155. Resolution: Duplicate > Update "Consistent Read from Observer" User Guide with Edit Tailing

[jira] [Commented] (HDFS-12943) Consistent Reads from Standby Node

[ https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723300#comment-16723300 ] Erik Krogen commented on HDFS-12943: Whoops, took too long writing my comment. Thanks for also

[jira] [Comment Edited] (HDFS-12943) Consistent Reads from Standby Node

[ https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16723295#comment-16723295 ] Erik Krogen edited comment on HDFS-12943 at 12/17/18 7:38 PM: --

[jira] [Created] (HDFS-14155) Update "Consistent Read from Observer" User Guide with Edit Tailing Frequency

Erik Krogen created HDFS-14155: -- Summary: Update "Consistent Read from Observer" User Guide with Edit Tailing Frequency Key: HDFS-14155 URL: https://issues.apache.org/jira/browse/HDFS-14155 Project:

[jira] [Updated] (HDFS-14146) Handle exception from internalQueueCall

[ https://issues.apache.org/jira/browse/HDFS-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14146: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Handle exception from

[jira] [Commented] (HDFS-14146) Handle exception from internalQueueCall

[ https://issues.apache.org/jira/browse/HDFS-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721821#comment-16721821 ] Erik Krogen commented on HDFS-14146: +1. Thanks for this [~csun]! I just committed it to branch

[jira] [Updated] (HDFS-14146) Handle exception from internalQueueCall

[ https://issues.apache.org/jira/browse/HDFS-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14146: --- Fix Version/s: HDFS-12943 > Handle exception from internalQueueCall >

[jira] [Comment Edited] (HDFS-12943) Consistent Reads from Standby Node

[ https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721595#comment-16721595 ] Erik Krogen edited comment on HDFS-12943 at 12/14/18 4:50 PM: -- Hey

[jira] [Commented] (HDFS-12943) Consistent Reads from Standby Node

[ https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721595#comment-16721595 ] Erik Krogen commented on HDFS-12943: Hey [~brahmareddy], thanks for trying it out and for the

[jira] [Updated] (HDFS-13873) ObserverNode should reject read requests when it is too far behind.

[ https://issues.apache.org/jira/browse/HDFS-13873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-13873: --- Fix Version/s: HDFS-12943 > ObserverNode should reject read requests when it is too far behind. >

[jira] [Commented] (HDFS-13873) ObserverNode should reject read requests when it is too far behind.

[ https://issues.apache.org/jira/browse/HDFS-13873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720660#comment-16720660 ] Erik Krogen commented on HDFS-13873: +1 LGTM. Just committed based off of v004 Jenkins run given that

[jira] [Resolved] (HDFS-13873) ObserverNode should reject read requests when it is too far behind.

[ https://issues.apache.org/jira/browse/HDFS-13873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-13873. Resolution: Fixed > ObserverNode should reject read requests when it is too far behind. >

[jira] [Commented] (HDFS-14146) Handle exception from internalQueueCall

[ https://issues.apache.org/jira/browse/HDFS-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720569#comment-16720569 ] Erik Krogen commented on HDFS-14146: +1 from me pending Jenkins. Thanks [~csun]! > Handle exception

[jira] [Commented] (HDFS-14146) Handle exception from internalQueueCall

[ https://issues.apache.org/jira/browse/HDFS-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720468#comment-16720468 ] Erik Krogen commented on HDFS-14146: Nice job [~csun], this looks great! I have a few very minor

[jira] [Comment Edited] (HDFS-14146) Handle exception from internalQueueCall

[ https://issues.apache.org/jira/browse/HDFS-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719587#comment-16719587 ] Erik Krogen edited comment on HDFS-14146 at 12/13/18 12:28 AM: --- {quote} I

[jira] [Comment Edited] (HDFS-14146) Handle exception from internalQueueCall

[ https://issues.apache.org/jira/browse/HDFS-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719587#comment-16719587 ] Erik Krogen edited comment on HDFS-14146 at 12/13/18 12:28 AM: --- {quote} I

[jira] [Commented] (HDFS-13977) NameNode can kill itself if it tries to send too many txns to a QJM simultaneously

[ https://issues.apache.org/jira/browse/HDFS-13977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719582#comment-16719582 ] Erik Krogen commented on HDFS-13977: It's not clear to me if this bug will still be present after

[jira] [Commented] (HDFS-14146) Handle exception from internalQueueCall

[ https://issues.apache.org/jira/browse/HDFS-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719587#comment-16719587 ] Erik Krogen commented on HDFS-14146: {quote} I couldn't come up with an example (It would be great if

[jira] [Comment Edited] (HDFS-14146) Handle exception from internalQueueCall

[ https://issues.apache.org/jira/browse/HDFS-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719557#comment-16719557 ] Erik Krogen edited comment on HDFS-14146 at 12/13/18 12:05 AM: --- Hey

[jira] [Commented] (HDFS-14146) Handle exception from internalQueueCall

[ https://issues.apache.org/jira/browse/HDFS-14146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719557#comment-16719557 ] Erik Krogen commented on HDFS-14146: Hey [~csun], this is a good find. I actually think this problem

[jira] [Commented] (HDFS-13873) ObserverNode should reject read requests when it is too far behind.

[ https://issues.apache.org/jira/browse/HDFS-13873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719539#comment-16719539 ] Erik Krogen commented on HDFS-13873: The logic seems fine to me now that the exception handling has

[jira] [Commented] (HDFS-14084) Need for more stats in DFSClient

[ https://issues.apache.org/jira/browse/HDFS-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719424#comment-16719424 ] Erik Krogen commented on HDFS-14084: Looks good, I have one comment in addition to [~elgoiri]'s

[jira] [Commented] (HDFS-14131) Create user guide for "Consistent reads from Observer" feature.

[ https://issues.apache.org/jira/browse/HDFS-14131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718159#comment-16718159 ] Erik Krogen commented on HDFS-14131: Whoops, sorry [~elgoiri]! Trunk sounds good to me. > Create

[jira] [Updated] (HDFS-14131) Create user guide for "Consistent reads from Observer" feature.

[ https://issues.apache.org/jira/browse/HDFS-14131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14131: --- Fix Version/s: HDFS-12943 > Create user guide for "Consistent reads from Observer" feature. >

[jira] [Updated] (HDFS-14131) Create user guide for "Consistent reads from Observer" feature.

[ https://issues.apache.org/jira/browse/HDFS-14131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14131: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) >

[jira] [Commented] (HDFS-14131) Create user guide for "Consistent reads from Observer" feature.

[ https://issues.apache.org/jira/browse/HDFS-14131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717937#comment-16717937 ] Erik Krogen commented on HDFS-14131: Just committed this to branch {{HDFS-12943}}. Thanks [~csun]! >

[jira] [Commented] (HDFS-14131) Create user guide for "Consistent reads from Observer" feature.

[ https://issues.apache.org/jira/browse/HDFS-14131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717864#comment-16717864 ] Erik Krogen commented on HDFS-14131: +1 LGTM > Create user guide for "Consistent reads from

[jira] [Commented] (HDFS-13873) ObserverNode should reject read requests when it is too far behind.

[ https://issues.apache.org/jira/browse/HDFS-13873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717580#comment-16717580 ] Erik Krogen commented on HDFS-13873: I like your logic, [~shv], but I think it could still result in

[jira] [Commented] (HDFS-14131) Create user guide for "Consistent reads from Observer" feature.

[ https://issues.apache.org/jira/browse/HDFS-14131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715837#comment-16715837 ] Erik Krogen commented on HDFS-14131: Nice work, thanks [~csun]! My comments are pretty minor but

[jira] [Commented] (HDFS-14084) Need for more stats in DFSClient

[ https://issues.apache.org/jira/browse/HDFS-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16713168#comment-16713168 ] Erik Krogen commented on HDFS-14084: Nice, v002 became surprisingly easy :) A few comments: * Can we

[jira] [Commented] (HDFS-14084) Need for more stats in DFSClient

[ https://issues.apache.org/jira/browse/HDFS-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16712079#comment-16712079 ] Erik Krogen commented on HDFS-14084: One additional advantage to doing it at the FileSystem level is

[jira] [Commented] (HDFS-12861) Track speed in DFSClient

[ https://issues.apache.org/jira/browse/HDFS-12861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711852#comment-16711852 ] Erik Krogen commented on HDFS-12861: Hi [~mf_borge], are you still planning on working on this? It

[jira] [Commented] (HDFS-14109) Improve hdfs auditlog format and support federation friendly

[ https://issues.apache.org/jira/browse/HDFS-14109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711766#comment-16711766 ] Erik Krogen commented on HDFS-14109: I think as with most recent additions to the audit log, it

[jira] [Commented] (HDFS-14084) Need for more stats in DFSClient

[ https://issues.apache.org/jira/browse/HDFS-14084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711760#comment-16711760 ] Erik Krogen commented on HDFS-14084: [~pranay_singh] - thanks for taking this on, this is really

[jira] [Commented] (HDFS-13999) Bogus missing block warning if the file is under construction when NN starts

[ https://issues.apache.org/jira/browse/HDFS-13999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16711732#comment-16711732 ] Erik Krogen commented on HDFS-13999: Thanks for fixing this [~jojochuang]! This has plagued us for a

[jira] [Commented] (HDFS-14069) Better debuggability for datanode decommissioning

[ https://issues.apache.org/jira/browse/HDFS-14069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702541#comment-16702541 ] Erik Krogen commented on HDFS-14069: Hey [~dannytbecker], thanks for the ping. I don't have too much

[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration

[ https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689892#comment-16689892 ] Erik Krogen edited comment on HDFS-14017 at 11/16/18 7:02 PM: -- Even stranger

[jira] [Commented] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration

[ https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689892#comment-16689892 ] Erik Krogen commented on HDFS-14017: Even stranger now: This time around it claims that everything

[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration

[ https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689764#comment-16689764 ] Erik Krogen edited comment on HDFS-14017 at 11/16/18 5:58 PM: -- Hm..

[jira] [Commented] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration

[ https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689764#comment-16689764 ] Erik Krogen commented on HDFS-14017: Hm.. Something is pretty wrong with Jenkins. It's not actually

[jira] [Commented] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration

[ https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688831#comment-16688831 ] Erik Krogen commented on HDFS-14017: You have a bunch of unused imports in ORPPWithIPFailover, but

[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration

[ https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688316#comment-16688316 ] Erik Krogen edited comment on HDFS-14017 at 11/15/18 4:37 PM: -- Hey

<    3   4   5   6   7   8   9   10   11   12   >