[jira] [Commented] (HDFS-14953) [Dynamometer] Missing blocks gradually increase after NN starts

2019-12-09 Thread Takanobu Asanuma (Jira) 


[ 
https://issues.apache.org/jira/browse/HDFS-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992267#comment-16992267
 ] 

Takanobu Asanuma commented on HDFS-14953:
-

Sorry for late response. Actually, I ported your PR and tested with it, but it 
didn't fix my problem. Will investigate it further.

> [Dynamometer] Missing blocks gradually increase after NN starts
> ---
>
> Key: HDFS-14953
> URL: https://issues.apache.org/jira/browse/HDFS-14953
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: tools
>Reporter: Takanobu Asanuma
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-14953) [Dynamometer] Missing blocks gradually increase after NN starts

2019-11-06 Thread Wei-Chiu Chuang (Jira) 


[ 
https://issues.apache.org/jira/browse/HDFS-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968497#comment-16968497
 ] 

Wei-Chiu Chuang commented on HDFS-14953:


Yep. How about porting my PR over here?

> [Dynamometer] Missing blocks gradually increase after NN starts
> ---
>
> Key: HDFS-14953
> URL: https://issues.apache.org/jira/browse/HDFS-14953
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: tools
>Reporter: Takanobu Asanuma
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-14953) [Dynamometer] Missing blocks gradually increase after NN starts

2019-11-05 Thread Takanobu Asanuma (Jira) 


[ 
https://issues.apache.org/jira/browse/HDFS-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16968113#comment-16968113
 ] 

Takanobu Asanuma commented on HDFS-14953:
-

The case is very similar to [this 
issue|https://github.com/linkedin/dynamometer/issues/64]. (Thanks for reporting 
it, [~weichiu].)

{quote}
After HDFS-9260, NN expects block replicas to be reported in ascending order of 
block id. If a block id is not in order, NN discards it silently. Because 
simulated DataNode in Dynamometer uses hash map to store block replicas, the 
replicas are not reported in order. The Dynamometer cluster would then see 
missing blocks gradually increase several minutes after NN starts.
{quote}

> [Dynamometer] Missing blocks gradually increase after NN starts
> ---
>
> Key: HDFS-14953
> URL: https://issues.apache.org/jira/browse/HDFS-14953
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: tools
>Reporter: Takanobu Asanuma
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org