[jira] [Issue Comment Deleted] (HADOOP-14604) Fix the bug of aggregation log have one replication in HDFS Federation environment

2018-12-06 Thread Elvis (JIRA) 


 [ 
https://issues.apache.org/jira/browse/HADOOP-14604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Elvis updated HADOOP-14604:
---
Comment: was deleted

(was: This patch may have some problems. Before applying this patch, the time 
between LOCALIZING and LOCALIZED is very fast. But after applying this patch, 
the time between LOCALIZING and LOCALIZED  becomes more than 10s)

> Fix the bug of aggregation log have one replication in HDFS Federation 
> environment
> --
>
> Key: HADOOP-14604
> URL: https://issues.apache.org/jira/browse/HADOOP-14604
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: fs, viewfs
>Affects Versions: 2.6.0
> Environment: Hadoop 2.6.0 with HDFS Federation
>Reporter: dingguotao
>Assignee: Doris Gu
>Priority: Major
> Fix For: 2.9.0, 2.7.4, 3.0.0-alpha4, 2.8.2
>
> Attachments: HADOOP-14604.patch
>
>
> With HDFS Federation configured, the aggregation log have a default 
> replication count of 1, rather than the value of dfs.replication in 
> hdfs-site.xml configuration file.
> Before: 
> {code:java}
> [op@NM-304-SA5212M4-BIGDATA-640 ~]$ hdfs dfs -ls 
> /yarn/apps/op/logs/application_1498544648312_0001
> Found 3 items
> -rw-r-   1 op hadoop   8270 2017-06-27 14:25 
> /yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-645_8041
> -rw-r-   1 op hadoop  54469 2017-06-27 14:25 
> /yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-646_8041
> -rw-r-   1 op hadoop   6537 2017-06-27 14:25 
> /yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-672_8041
> {code}
> The aggregation log have only 1 replication in default. But  the value of 
> dfs.replication in hdfs-site.xml configuration file is 3
> After apply this patch:
> {code:java}
> [op@NM-304-SA5212M4-BIGDATA-640 ~]$ hdfs dfs -ls 
> /yarn/apps/op/logs/application_1498635214020_0001
> Found 3 items
> -rw-r-   3 op hadoop   5729 2017-06-28 15:34 
> /yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-645_8041
> -rw-r-   3 op hadoop  38439 2017-06-28 15:34 
> /yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-646_8041
> -rw-r-   3 op hadoop   8270 2017-06-28 15:34 
> /yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-671_8041
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-14604) Fix the bug of aggregation log have one replication in HDFS Federation environment

2018-11-21 Thread Elvis (JIRA) 


[ 
https://issues.apache.org/jira/browse/HADOOP-14604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16694510#comment-16694510
 ] 

Elvis commented on HADOOP-14604:


This patch may have some problems. Before applying this patch, the time between 
LOCALIZING and LOCALIZED is very fast. But after applying this patch, the time 
between LOCALIZING and LOCALIZED  becomes more than 10s

> Fix the bug of aggregation log have one replication in HDFS Federation 
> environment
> --
>
> Key: HADOOP-14604
> URL: https://issues.apache.org/jira/browse/HADOOP-14604
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: fs, viewfs
>Affects Versions: 2.6.0
> Environment: Hadoop 2.6.0 with HDFS Federation
>Reporter: dingguotao
>Assignee: Doris Gu
>Priority: Major
> Fix For: 2.9.0, 2.7.4, 3.0.0-alpha4, 2.8.2
>
> Attachments: HADOOP-14604.patch
>
>
> With HDFS Federation configured, the aggregation log have a default 
> replication count of 1, rather than the value of dfs.replication in 
> hdfs-site.xml configuration file.
> Before: 
> {code:java}
> [op@NM-304-SA5212M4-BIGDATA-640 ~]$ hdfs dfs -ls 
> /yarn/apps/op/logs/application_1498544648312_0001
> Found 3 items
> -rw-r-   1 op hadoop   8270 2017-06-27 14:25 
> /yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-645_8041
> -rw-r-   1 op hadoop  54469 2017-06-27 14:25 
> /yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-646_8041
> -rw-r-   1 op hadoop   6537 2017-06-27 14:25 
> /yarn/apps/op/logs/application_1498544648312_0001/NM-304-SA5212M4-BIGDATA-672_8041
> {code}
> The aggregation log have only 1 replication in default. But  the value of 
> dfs.replication in hdfs-site.xml configuration file is 3
> After apply this patch:
> {code:java}
> [op@NM-304-SA5212M4-BIGDATA-640 ~]$ hdfs dfs -ls 
> /yarn/apps/op/logs/application_1498635214020_0001
> Found 3 items
> -rw-r-   3 op hadoop   5729 2017-06-28 15:34 
> /yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-645_8041
> -rw-r-   3 op hadoop  38439 2017-06-28 15:34 
> /yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-646_8041
> -rw-r-   3 op hadoop   8270 2017-06-28 15:34 
> /yarn/apps/op/logs/application_1498635214020_0001/NM-304-SA5212M4-BIGDATA-671_8041
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org