[jira] [Updated] (HDFS-14640) [Dynamometer] Fix TestDynamometerInfra failures

2019-07-10 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14640: --- Status: Patch Available (was: In Progress) > [Dynamometer] Fix TestDynamometerInfra failures >

[jira] [Updated] (HDFS-14640) [Dynamometer] Fix TestDynamometerInfra failures

2019-07-10 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14640: --- Attachment: HDFS-14640.000.patch > [Dynamometer] Fix TestDynamometerInfra failures >

[jira] [Updated] (HDFS-14281) Dynamometer Phase 2

2019-07-10 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14281: --- Component/s: (was: namenode) > Dynamometer Phase 2 > --- > >

[jira] [Updated] (HDFS-14281) Dynamometer Phase 2

2019-07-10 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14281: --- Component/s: tools > Dynamometer Phase 2 > --- > > Key: HDFS-14281 >

[jira] [Updated] (HDFS-14640) [Dynamometer] Fix TestDynamometerInfra failures

2019-07-10 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14640: --- Component/s: tools test > [Dynamometer] Fix TestDynamometerInfra failures >

[jira] [Updated] (HDFS-14622) [Dynamometer] State transition err when CCM( HDFS Centralized Cache Management) feature is used

2019-07-10 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14622: --- Issue Type: Sub-task (was: Bug) Parent: HDFS-14281 > [Dynamometer] State transition err

[jira] [Commented] (HDFS-14622) [Dynamometer] State transition err when CCM( HDFS Centralized Cache Management) feature is used

2019-07-10 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882268#comment-16882268 ] Erik Krogen commented on HDFS-14622: Thanks [~jojochuang]. I'm working on addressing the findbugs

[jira] [Updated] (HDFS-14622) [Dynamometer] State transition err when CCM( HDFS Centralized Cache Management) feature is used

2019-07-10 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14622: --- Resolution: Fixed Fix Version/s: 3.3.0 Status: Resolved (was: Patch Available) >

[jira] [Updated] (HDFS-14410) Make Dynamometer documentation properly compile onto the Hadoop site

2019-07-10 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14410: --- Attachment: (was: HDFS-14410.001.patch) > Make Dynamometer documentation properly compile onto

[jira] [Updated] (HDFS-14410) Make Dynamometer documentation properly compile onto the Hadoop site

2019-07-10 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14410: --- Attachment: HDFS-14410.001.patch > Make Dynamometer documentation properly compile onto the Hadoop

[jira] [Updated] (HDFS-14410) Make Dynamometer documentation properly compile onto the Hadoop site

2019-07-10 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14410: --- Attachment: HDFS-14410.001.patch > Make Dynamometer documentation properly compile onto the Hadoop

[jira] [Updated] (HDFS-14410) Make Dynamometer documentation properly compile onto the Hadoop site

2019-07-10 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14410: --- Attachment: dynamometer_guide_rendered.001.png > Make Dynamometer documentation properly compile

[jira] [Commented] (HDFS-14410) Make Dynamometer documentation properly compile onto the Hadoop site

2019-07-10 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882266#comment-16882266 ] Erik Krogen commented on HDFS-14410: Thanks for the review [~linyiqun]. I've added the "step"

[jira] [Commented] (HDFS-12748) NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY

2019-07-10 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16882234#comment-16882234 ] Erik Krogen commented on HDFS-12748: +1 on branch-3.1 backport > NameNode memory leak when accessing

[jira] [Created] (HDFS-14643) [Dynamometer] Merge extra commits from GitHub to Hadoop

2019-07-10 Thread Erik Krogen (JIRA)
Erik Krogen created HDFS-14643: -- Summary: [Dynamometer] Merge extra commits from GitHub to Hadoop Key: HDFS-14643 URL: https://issues.apache.org/jira/browse/HDFS-14643 Project: Hadoop HDFS

[jira] [Work started] (HDFS-14640) [Dynamometer] Fix TestDynamometerInfra failures

2019-07-09 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-14640 started by Erik Krogen. -- > [Dynamometer] Fix TestDynamometerInfra failures >

[jira] [Created] (HDFS-14640) [Dynamometer] Fix TestDynamometerInfra failures

2019-07-09 Thread Erik Krogen (JIRA)
Erik Krogen created HDFS-14640: -- Summary: [Dynamometer] Fix TestDynamometerInfra failures Key: HDFS-14640 URL: https://issues.apache.org/jira/browse/HDFS-14640 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-14410) Make Dynamometer documentation properly compile onto the Hadoop site

2019-07-09 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881534#comment-16881534 ] Erik Krogen commented on HDFS-14410: The whitespace error is complaining about a line having tabs,

[jira] [Updated] (HDFS-14639) [Dynamometer] Unnecessary duplicate bin directory appears in dist layout

2019-07-09 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14639: --- Status: Patch Available (was: In Progress) Attaching v000 patch which fixes the issue. It looks

[jira] [Updated] (HDFS-14639) [Dynamometer] Unnecessary duplicate bin directory appears in dist layout

2019-07-09 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14639: --- Attachment: HDFS-14639.000.patch > [Dynamometer] Unnecessary duplicate bin directory appears in dist

[jira] [Work started] (HDFS-14639) [Dynamometer] Unnecessary duplicate bin directory appears in dist layout

2019-07-09 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-14639 started by Erik Krogen. -- > [Dynamometer] Unnecessary duplicate bin directory appears in dist layout >

[jira] [Commented] (HDFS-14410) Make Dynamometer documentation properly compile onto the Hadoop site

2019-07-09 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881481#comment-16881481 ] Erik Krogen commented on HDFS-14410: Attached v000 patch which creates a {{Dynamometer.md}} and links

[jira] [Updated] (HDFS-14410) Make Dynamometer documentation properly compile onto the Hadoop site

2019-07-09 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14410: --- Status: Patch Available (was: In Progress) > Make Dynamometer documentation properly compile onto

[jira] [Updated] (HDFS-14410) Make Dynamometer documentation properly compile onto the Hadoop site

2019-07-09 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14410: --- Attachment: HDFS-14410.000.patch > Make Dynamometer documentation properly compile onto the Hadoop

[jira] [Assigned] (HDFS-14639) [Dynamometer] Unnecessary duplicate bin directory appears in dist layout

2019-07-09 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reassigned HDFS-14639: -- Assignee: Erik Krogen > [Dynamometer] Unnecessary duplicate bin directory appears in dist

[jira] [Created] (HDFS-14639) [Dynamometer] Unnecessary duplicate bin directory appears in dist layout

2019-07-09 Thread Erik Krogen (JIRA)
Erik Krogen created HDFS-14639: -- Summary: [Dynamometer] Unnecessary duplicate bin directory appears in dist layout Key: HDFS-14639 URL: https://issues.apache.org/jira/browse/HDFS-14639 Project: Hadoop

[jira] [Created] (HDFS-14638) [Dynamometer] Fix scripts to refer to current build structure

2019-07-09 Thread Erik Krogen (JIRA)
Erik Krogen created HDFS-14638: -- Summary: [Dynamometer] Fix scripts to refer to current build structure Key: HDFS-14638 URL: https://issues.apache.org/jira/browse/HDFS-14638 Project: Hadoop HDFS

[jira] [Work started] (HDFS-14410) Make Dynamometer documentation properly compile onto the Hadoop site

2019-07-09 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-14410 started by Erik Krogen. -- > Make Dynamometer documentation properly compile onto the Hadoop site >

[jira] [Commented] (HDFS-14622) [Dynamometer] State transition err when CCM( HDFS Centralized Cache Management) feature is used

2019-07-09 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16881306#comment-16881306 ] Erik Krogen commented on HDFS-14622: Great, now we just need to find a committer for review :)

[jira] [Updated] (HDFS-14547) DirectoryWithQuotaFeature.quota costs additional memory even the storage type quota is not set.

2019-07-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14547: --- Fix Version/s: 3.1.3 3.2.1 3.3.0 3.0.4 >

[jira] [Commented] (HDFS-14547) DirectoryWithQuotaFeature.quota costs additional memory even the storage type quota is not set.

2019-07-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880758#comment-16880758 ] Erik Krogen commented on HDFS-14547: I was unable to reproduce the failing tests locally. Surprising

[jira] [Commented] (HDFS-14622) [Dynamometer] State transition err when CCM( HDFS Centralized Cache Management) feature is used

2019-07-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880752#comment-16880752 ] Erik Krogen commented on HDFS-14622: Thanks [~tanyuxin] and [~hexiaoqiao], I just submitted a patch

[jira] [Work started] (HDFS-14622) [Dynamometer] State transition err when CCM( HDFS Centralized Cache Management) feature is used

2019-07-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-14622 started by Erik Krogen. -- > [Dynamometer] State transition err when CCM( HDFS Centralized Cache >

[jira] [Updated] (HDFS-14622) [Dynamometer] State transition err when CCM( HDFS Centralized Cache Management) feature is used

2019-07-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14622: --- Status: Patch Available (was: In Progress) > [Dynamometer] State transition err when CCM( HDFS

[jira] [Updated] (HDFS-14622) [Dynamometer] State transition err when CCM( HDFS Centralized Cache Management) feature is used

2019-07-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14622: --- Attachment: HDFS-14622.000.patch > [Dynamometer] State transition err when CCM( HDFS Centralized

[jira] [Commented] (HDFS-14636) SBN : If you configure the default proxy provider still read Request going to Observer namenode only.

2019-07-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880659#comment-16880659 ] Erik Krogen commented on HDFS-14636: Hi [~Harsha1206], can you more fully explain your issue? I don't

[jira] [Comment Edited] (HDFS-12748) NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY

2019-07-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880600#comment-16880600 ] Erik Krogen edited comment on HDFS-12748 at 7/8/19 6:13 PM: Sorry for my late

[jira] [Commented] (HDFS-12748) NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY

2019-07-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880600#comment-16880600 ] Erik Krogen commented on HDFS-12748: Sorry for my late reply; I was on vacation last week. The

[jira] [Assigned] (HDFS-14622) [Dynamometer] State transition err when CCM( HDFS Centralized Cache Management) feature is used

2019-07-08 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reassigned HDFS-14622: -- Assignee: Erik Krogen > [Dynamometer] State transition err when CCM( HDFS Centralized Cache

[jira] [Commented] (HDFS-14547) DirectoryWithQuotaFeature.quota costs additional memory even the storage type quota is not set.

2019-06-28 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16875022#comment-16875022 ] Erik Krogen commented on HDFS-14547: [~LiJinglun], the new patch looks really good, I have a few more

[jira] [Commented] (HDFS-14547) DirectoryWithQuotaFeature.quota costs additional memory even the storage type quota is not set.

2019-06-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874493#comment-16874493 ] Erik Krogen commented on HDFS-14547: [~LiJinglun] I just took a look, thanks a lot for this work. It

[jira] [Commented] (HDFS-14611) Move handshake secret field from Token to BlockAccessToken

2019-06-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874357#comment-16874357 ] Erik Krogen commented on HDFS-14611: Makes sense, thank you for the detailed explanation! > Move

[jira] [Commented] (HDFS-14611) Move handshake secret field from Token to BlockAccessToken

2019-06-27 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874282#comment-16874282 ] Erik Krogen commented on HDFS-14611: Hi [~vagarychen], I haven't yet taken a look at the patch, but

[jira] [Assigned] (HDFS-14410) Make Dynamometer documentation properly compile onto the Hadoop site

2019-06-25 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reassigned HDFS-14410: -- Assignee: Erik Krogen > Make Dynamometer documentation properly compile onto the Hadoop site

[jira] [Commented] (HDFS-14410) Make Dynamometer documentation properly compile onto the Hadoop site

2019-06-25 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16872435#comment-16872435 ] Erik Krogen commented on HDFS-14410: We can start from the README of the Dynamometer GitHub repo

[jira] [Commented] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

2019-06-25 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16872433#comment-16872433 ] Erik Krogen commented on HDFS-12345: I just committed this to trunk. Huge thank you to [~jojochuang]

[jira] [Updated] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

2019-06-25 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-12345: --- Resolution: Fixed Fix Version/s: 3.3.0 Status: Resolved (was: Patch Available) >

[jira] [Updated] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-06-25 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14403: --- Resolution: Fixed Fix Version/s: 3.3.0 Status: Resolved (was: Patch Available) >

[jira] [Commented] (HDFS-12748) NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY

2019-06-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871776#comment-16871776 ] Erik Krogen commented on HDFS-12748: [~cheersyang] are you still interested in working on this? The

[jira] [Commented] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-06-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871702#comment-16871702 ] Erik Krogen commented on HDFS-14403: Just committed this to trunk. Thanks for the reviews [~elgoiri]

[jira] [Commented] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-06-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871396#comment-16871396 ] Erik Krogen commented on HDFS-14403: Thanks [~elgoiri]! Attached v13 without the unused import. I'll

[jira] [Updated] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-06-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14403: --- Attachment: HDFS-14403.013.patch > Cost-Based RPC FairCallQueue > > >

[jira] [Commented] (HDFS-14547) DirectoryWithQuotaFeature.quota costs additional memory even the storage type quota is not set.

2019-06-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871395#comment-16871395 ] Erik Krogen commented on HDFS-14547: Thanks for the ping, I will try to review this week >

[jira] [Commented] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

2019-06-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16871256#comment-16871256 ] Erik Krogen commented on HDFS-12345: I think this is ready to be committed, for follow-up tasks to be

[jira] [Updated] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

2019-06-21 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-12345: --- Attachment: HDFS-12345.009.patch > Scale testing HDFS NameNode with real metadata and workloads

[jira] [Commented] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

2019-06-21 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869951#comment-16869951 ] Erik Krogen commented on HDFS-12345: Attached v009 fixing a few of the last remaining checkstyle /

[jira] [Updated] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

2019-06-21 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-12345: --- Attachment: HDFS-12345.008.patch > Scale testing HDFS NameNode with real metadata and workloads

[jira] [Commented] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

2019-06-21 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869780#comment-16869780 ] Erik Krogen commented on HDFS-12345: Thanks [~jojochuang], that's a great catch. I changed it to the

[jira] [Commented] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-06-21 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869737#comment-16869737 ] Erik Krogen commented on HDFS-14403: Thanks [~elgoiri]! Great suggestions. I have applied them all in

[jira] [Updated] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-06-21 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14403: --- Attachment: HDFS-14403.012.patch > Cost-Based RPC FairCallQueue > > >

[jira] [Commented] (HDFS-14034) Support getQuotaUsage API in WebHDFS

2019-06-21 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16869723#comment-16869723 ] Erik Krogen commented on HDFS-14034: We want to tackle it pretty soon, but if you have bandwidth now,

[jira] [Commented] (HDFS-14588) Client retries Standby NN continuously even if Active NN is available (WebHDFS)

2019-06-20 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868968#comment-16868968 ] Erik Krogen commented on HDFS-14588: Seems like bad behavior. Am I correct in saying that your

[jira] [Commented] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-06-20 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868939#comment-16868939 ] Erik Krogen commented on HDFS-14403: Hey [~elgoiri], interesting question... I haven't heard of any

[jira] [Commented] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-06-20 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868819#comment-16868819 ] Erik Krogen commented on HDFS-14403: Shoot... I did indeed think you were a committer. My mistake.

[jira] [Commented] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

2019-06-20 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868718#comment-16868718 ] Erik Krogen commented on HDFS-12345: Attached v007 patch which: * Adds assumptions about not running

[jira] [Updated] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

2019-06-20 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-12345: --- Attachment: HDFS-12345.007.patch > Scale testing HDFS NameNode with real metadata and workloads

[jira] [Commented] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-06-20 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868614#comment-16868614 ] Erik Krogen commented on HDFS-14403: Forgot to update {{TestProcessingDetails}} when I removed

[jira] [Updated] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-06-20 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14403: --- Attachment: HDFS-14403.011.patch > Cost-Based RPC FairCallQueue > > >

[jira] [Commented] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-06-19 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868004#comment-16868004 ] Erik Krogen commented on HDFS-14403: Thanks [~csun]! # Yeah, it works the same as all of the IPC/call

[jira] [Updated] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-06-19 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14403: --- Attachment: HDFS-14403.010.patch > Cost-Based RPC FairCallQueue > > >

[jira] [Commented] (HDFS-14573) Backport Standby Read to branch-3

2019-06-19 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867744#comment-16867744 ] Erik Krogen commented on HDFS-14573: For both this and HDFS-14204, let's make sure that we get all of

[jira] [Commented] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-06-17 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16866024#comment-16866024 ] Erik Krogen commented on HDFS-14403: Just attached v009 patch. The v008 -> v009 changes are: * Fix 1

[jira] [Updated] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-06-17 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14403: --- Attachment: HDFS-14403.009.patch > Cost-Based RPC FairCallQueue > > >

[jira] [Commented] (HDFS-14537) Journaled Edits Cache is not cleared when formatting the JN

2019-06-13 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16863161#comment-16863161 ] Erik Krogen commented on HDFS-14537: +1 from me as well, thanks [~RANith] and [~brahmareddy]! >

[jira] [Commented] (HDFS-14547) DirectoryWithQuotaFeature.quota costs additional memory even the storage type quota is not set.

2019-06-12 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16862277#comment-16862277 ] Erik Krogen commented on HDFS-14547: [~LiJinglun], thanks for taking this up, it seems like a

[jira] [Commented] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-06-10 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860476#comment-16860476 ] Erik Krogen commented on HDFS-14403: Thanks a lot for the review [~csun]. These are all great points!

[jira] [Updated] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-06-10 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14403: --- Attachment: HDFS-14403.008.patch > Cost-Based RPC FairCallQueue > > >

[jira] [Commented] (HDFS-14537) Journaled Edits Cache is not cleared when formatting the JN

2019-06-04 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855850#comment-16855850 ] Erik Krogen commented on HDFS-14537: Thanks for the ping [~brahmareddy]! Instead of using a redundant

[jira] [Commented] (HDFS-14090) RBF: Improved isolation for downstream name nodes.

2019-06-03 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855160#comment-16855160 ] Erik Krogen commented on HDFS-14090: Is it possible to at least make it a subclass of

[jira] [Commented] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

2019-06-03 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855156#comment-16855156 ] Erik Krogen commented on HDFS-12345: Dropped this for a while, but would really like to push it

[jira] [Created] (HDFS-14539) Remove Dynamometer's reliance on the tar utility

2019-06-03 Thread Erik Krogen (JIRA)
Erik Krogen created HDFS-14539: -- Summary: Remove Dynamometer's reliance on the tar utility Key: HDFS-14539 URL: https://issues.apache.org/jira/browse/HDFS-14539 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-14090) RBF: Improved isolation for downstream name nodes.

2019-06-03 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855129#comment-16855129 ] Erik Krogen commented on HDFS-14090: [~crh], I took a look at the design document and think your

[jira] [Commented] (HDFS-12979) StandbyNode should upload FsImage to ObserverNode after checkpointing.

2019-06-03 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16854937#comment-16854937 ] Erik Krogen commented on HDFS-12979: Hey [~vagarychen], the new test case LGTM, although you need to

[jira] [Comment Edited] (HDFS-14528) [SBN Read]Failover from Active to Standby Failed

2019-06-03 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16854726#comment-16854726 ] Erik Krogen edited comment on HDFS-14528 at 6/3/19 3:38 PM: Thanks for filing

[jira] [Commented] (HDFS-14528) [SBN Read]Failover from Active to Standby Failed

2019-06-03 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16854726#comment-16854726 ] Erik Krogen commented on HDFS-14528: I'm not too familiar with ZKFC so would prefer to let [~csun]

[jira] [Commented] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-05-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847975#comment-16847975 ] Erik Krogen commented on HDFS-14403: Now that HADOOP-16266 has been committed, we can resume progress

[jira] [Updated] (HDFS-14403) Cost-Based RPC FairCallQueue

2019-05-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14403: --- Attachment: HDFS-14403.007.patch > Cost-Based RPC FairCallQueue > > >

[jira] [Resolved] (HDFS-14500) NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished

2019-05-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-14500. Resolution: Fixed > NameNode StartupProgress continues to report edit log segments after the >

[jira] [Comment Edited] (HDFS-14500) NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished

2019-05-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16847867#comment-16847867 ] Erik Krogen edited comment on HDFS-14500 at 5/24/19 8:57 PM: - Just committed

[jira] [Updated] (HDFS-14500) NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished

2019-05-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14500: --- Fix Version/s: 3.1.3 3.2.1 3.0.4 2.10.0 >

[jira] [Updated] (HDFS-14500) NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished

2019-05-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14500: --- Attachment: HDFS-14500-branch-2.001.patch > NameNode StartupProgress continues to report edit log

[jira] [Reopened] (HDFS-14500) NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished

2019-05-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen reopened HDFS-14500: > NameNode StartupProgress continues to report edit log segments after the > LOADING_EDITS phase is

[jira] [Updated] (HDFS-14500) NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished

2019-05-24 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14500: --- Resolution: Fixed Fix Version/s: 3.3.0 Status: Resolved (was: Patch Available)

[jira] [Commented] (HDFS-12979) StandbyNode should upload FsImage to ObserverNode after checkpointing.

2019-05-22 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16846144#comment-16846144 ] Erik Krogen commented on HDFS-12979: Regarding the use of the static variable to pass the tests, it

[jira] [Updated] (HDFS-14500) NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished

2019-05-22 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14500: --- Attachment: HDFS-14500.001.patch > NameNode StartupProgress continues to report edit log segments

[jira] [Commented] (HDFS-14500) NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished

2019-05-22 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16846127#comment-16846127 ] Erik Krogen commented on HDFS-14500: Yes, your understanding is correct. Considering that the changes

[jira] [Commented] (HDFS-14502) keepResults option in NNThroughputBenchmark should call saveNamespace()

2019-05-21 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844916#comment-16844916 ] Erik Krogen commented on HDFS-14502: Thanks for filing this, [~shv]. I have found myself doing this

[jira] [Commented] (HDFS-14500) NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished

2019-05-17 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16842412#comment-16842412 ] Erik Krogen commented on HDFS-14500: Attached v000 patch which fixes this issue by turning the

[jira] [Updated] (HDFS-14500) NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished

2019-05-17 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14500: --- Attachment: HDFS-14500.000.patch > NameNode StartupProgress continues to report edit log segments

[jira] [Updated] (HDFS-14500) NameNode StartupProgress continues to report edit log segments after the LOADING_EDITS phase is finished

2019-05-17 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-14500: --- Status: Patch Available (was: In Progress) > NameNode StartupProgress continues to report edit log

<    1   2   3   4   5   6   7   8   9   10   >