Hu Ziqian created YARN-9947:
---
Summary: lazy-init-appLogAggregatorImpl-when-log-aggregation
Key: YARN-9947
URL: https://issues.apache.org/jira/browse/YARN-9947
Project: Hadoop YARN
Issue Type: Impro
[
https://issues.apache.org/jira/browse/YARN-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-9947:
Summary: lazy init appLogAggregatorImpl when log aggregation (was:
lazy-init-appLogAggregatorImpl-when-log-
Hu Ziqian created YARN-9948:
---
Summary: Remove attempts that are beyond max-attempt limit from
RMAppImpl
Key: YARN-9948
URL: https://issues.apache.org/jira/browse/YARN-9948
Project: Hadoop YARN
Iss
[
https://issues.apache.org/jira/browse/YARN-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian reassigned YARN-9947:
---
Assignee: Hu Ziqian
> lazy init appLogAggregatorImpl when log aggregation
> -
[
https://issues.apache.org/jira/browse/YARN-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-9947:
Description: In current version, app log aggregator will check
> lazy init appLogAggregatorImpl when log agg
[
https://issues.apache.org/jira/browse/YARN-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-9947:
Description:
This issue introduce an method to lazy init
In current version, app log aggregator will ch
[
https://issues.apache.org/jira/browse/YARN-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-9947:
Description:
This issue introduce an method to lazy init appLogAggregatorImpl, which let it
access hdfs as
[
https://issues.apache.org/jira/browse/YARN-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-9947:
Description:
This issue introduce an method to lazy init appLogAggregatorImpl, which let it
access hdfs as
[
https://issues.apache.org/jira/browse/YARN-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-9947:
Attachment: YARN-9947.001.patch
> lazy init appLogAggregatorImpl when log aggregation
>
[
https://issues.apache.org/jira/browse/YARN-9948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-9948:
Description:
RM will store app attempt in both state store and RMAppImpl. YARN-3480 removes
attempts that a
[
https://issues.apache.org/jira/browse/YARN-9948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-9948:
Attachment: YARN-9948.001.patch
> Remove attempts that are beyond max-attempt limit from RMAppImpl
> ---
[
https://issues.apache.org/jira/browse/YARN-9948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16966447#comment-16966447
]
Hu Ziqian commented on YARN-9948:
-
[~taoyang], could you help me to review this issue?
>
[
https://issues.apache.org/jira/browse/YARN-9947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16966448#comment-16966448
]
Hu Ziqian commented on YARN-9947:
-
[~taoyang], [~leftnoteasy] could you help me to review
[
https://issues.apache.org/jira/browse/YARN-9948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16967341#comment-16967341
]
Hu Ziqian commented on YARN-9948:
-
[~hex108], [~jianhe] , this issue is based on YARN-34
[
https://issues.apache.org/jira/browse/YARN-9948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16970038#comment-16970038
]
Hu Ziqian commented on YARN-9948:
-
Hi [~hex108], in our cluster, one attempt will use more
[
https://issues.apache.org/jira/browse/YARN-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16979115#comment-16979115
]
Hu Ziqian commented on YARN-4766:
-
hi [~haibochen], should you tell me what's problem this
Hu Ziqian created YARN-9163:
---
Summary: Deadlock when use yarn rmadmin -refreshQueues
Key: YARN-9163
URL: https://issues.apache.org/jira/browse/YARN-9163
Project: Hadoop YARN
Issue Type: Bug
[
https://issues.apache.org/jira/browse/YARN-9163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian reassigned YARN-9163:
---
Assignee: Hu Ziqian
Affects Version/s: 3.1.1
Description:
We have a cluster wi
[
https://issues.apache.org/jira/browse/YARN-9163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-9163:
Attachment: YARN-9163.001.patch
> Deadlock when use yarn rmadmin -refreshQueues
> --
[
https://issues.apache.org/jira/browse/YARN-9163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-9163:
Description:
We have a cluster with 4000+ node and 10w+ app per-day in our production
environment. When we
[
https://issues.apache.org/jira/browse/YARN-9163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-9163:
Attachment: rm.jstack.ziqian.log
> Deadlock when use yarn rmadmin -refreshQueues
> -
[
https://issues.apache.org/jira/browse/YARN-9163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-9163:
Attachment: (was: rm.jstack.ziqian.log)
> Deadlock when use yarn rmadmin -refreshQueues
> --
[
https://issues.apache.org/jira/browse/YARN-9163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-9163:
Attachment: rm.jstack.ziqian.log
> Deadlock when use yarn rmadmin -refreshQueues
> -
[
https://issues.apache.org/jira/browse/YARN-9163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16735426#comment-16735426
]
Hu Ziqian commented on YARN-9163:
-
[~Tao Yang],
Sorry, not found any new evidence yet.
>
[
https://issues.apache.org/jira/browse/YARN-9163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16735424#comment-16735424
]
Hu Ziqian commented on YARN-9163:
-
[~cheersyang], actually we used a internal version of
[
https://issues.apache.org/jira/browse/YARN-9163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16735423#comment-16735423
]
Hu Ziqian commented on YARN-9163:
-
[~leftnoteasy],here‘s the jstack. [^rm.jstack.ziqian.lo
[
https://issues.apache.org/jira/browse/YARN-8479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16777559#comment-16777559
]
Hu Ziqian commented on YARN-8479:
-
Hi [~leftnoteasy],[~daemon]
"As a result of our test t
Hu Ziqian created YARN-8232:
---
Summary: RMContainer lost Queue name when recovered by RM
Key: YARN-8232
URL: https://issues.apache.org/jira/browse/YARN-8232
Project: Hadoop YARN
Issue Type: Bug
[
https://issues.apache.org/jira/browse/YARN-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8232:
Description:
RMContainer has a member variable queuename to store which queue the container
belongs to. Pree
[
https://issues.apache.org/jira/browse/YARN-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8232:
Flags: Patch
Attachment: YARN_8232.patch
Description:
RMContainer has a member variable queuen
[
https://issues.apache.org/jira/browse/YARN-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8232:
Description:
RMContainer has a member variable queuename to store which queue the container
belongs to. When
Hu Ziqian created YARN-8234:
---
Summary: Improve timeline server v1's preformance
Key: YARN-8234
URL: https://issues.apache.org/jira/browse/YARN-8234
Project: Hadoop YARN
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8234:
Flags: (was: Patch)
Attachment: YARN_8234.patch
Description:
When system metrics publisher i
[
https://issues.apache.org/jira/browse/YARN-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8232:
Attachment: (was: YARN_8232.patch)
> RMContainer lost queue name when RM HA happens
> ---
[
https://issues.apache.org/jira/browse/YARN-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8232:
Attachment: YARN-8232-branch-2.8.3.001.patch
YARN-8232.001.patch
> RMContainer lost queue nam
[
https://issues.apache.org/jira/browse/YARN-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458367#comment-16458367
]
Hu Ziqian commented on YARN-8232:
-
[~leftnoteasy], thanks for your comment.
I resubmit 2 v
[
https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8234:
Attachment: YARN-8234-branch-2.8.3.001.patch
> Improve RM system metrics publisher's performance by pushing e
[
https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8234:
Attachment: (was: YARN_8234.patch)
> Improve RM system metrics publisher's performance by pushing events
[
https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458375#comment-16458375
]
Hu Ziqian commented on YARN-8234:
-
[~rohithsharma], we are using ATS1
> Improve RM system
[
https://issues.apache.org/jira/browse/YARN-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8232:
Attachment: YARN-8232.002.patch
> RMContainer lost queue name when RM HA happens
> --
[
https://issues.apache.org/jira/browse/YARN-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459656#comment-16459656
]
Hu Ziqian commented on YARN-8232:
-
hi [~leftnoteasy], I've add a new patch with an unit tes
[
https://issues.apache.org/jira/browse/YARN-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459656#comment-16459656
]
Hu Ziqian edited comment on YARN-8232 at 5/1/18 12:29 PM:
--
hi [~le
[
https://issues.apache.org/jira/browse/YARN-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8232:
Attachment: YARN-8232.003.patch
> RMContainer lost queue name when RM HA happens
> --
[
https://issues.apache.org/jira/browse/YARN-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460447#comment-16460447
]
Hu Ziqian commented on YARN-8232:
-
YARN-8232.003.patch fix the checkstyle.
> RMContainer l
[
https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16465394#comment-16465394
]
Hu Ziqian commented on YARN-8234:
-
[~leftnoteasy], I'll check the Jenkins report later.
>
[
https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8234:
Attachment: YARN-8234-branch-2.8.3.002.patch
> Improve RM system metrics publisher's performance by pushing e
[
https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8234:
Attachment: YARN-8234-branch-2.8.3.003.patch
> Improve RM system metrics publisher's performance by pushing e
[
https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478467#comment-16478467
]
Hu Ziqian commented on YARN-8234:
-
Hi [~leftnoteasy],
hadoop.yarn.server.resourcemanager
[
https://issues.apache.org/jira/browse/YARN-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480217#comment-16480217
]
Hu Ziqian commented on YARN-8232:
-
Thanks [~leftnoteasy] for reviewing it
> RMContainer lo
Hu Ziqian created YARN-8382:
---
Summary: cgroup file leak in NM
Key: YARN-8382
URL: https://issues.apache.org/jira/browse/YARN-8382
Project: Hadoop YARN
Issue Type: Bug
Components: nodemana
[
https://issues.apache.org/jira/browse/YARN-8382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8382:
Description:
As Jiandan said in YARN-6525, NM may delete Cgroup container file timeout with
logs like
org
[
https://issues.apache.org/jira/browse/YARN-8382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8382:
Description:
As Jiandan said in YARN-6525, NM may delete Cgroup container file timeout with
logs like
org
[
https://issues.apache.org/jira/browse/YARN-8382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8382:
Attachment: YARN-8382-branch-2.8.3.001.patch
YARN-8382.001.patch
> cgroup file leak in NM
>
[
https://issues.apache.org/jira/browse/YARN-8382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8382:
Description:
As Jiandan said in YARN-6525, NM may delete Cgroup container file timeout with
logs like belo
[
https://issues.apache.org/jira/browse/YARN-8382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496049#comment-16496049
]
Hu Ziqian commented on YARN-8382:
-
Hi [~leftnoteasy], can you help me reviewing this patch
[
https://issues.apache.org/jira/browse/YARN-8382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8382:
Description:
As Jiandan said in YARN-6525, NM may delete Cgroup container file timeout with
logs like belo
[
https://issues.apache.org/jira/browse/YARN-8382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8382:
Attachment: YARN-8382.002.patch
YARN-8382-branch-2.8.3.002.patch
> cgroup file leak in NM
>
[
https://issues.apache.org/jira/browse/YARN-8382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16499324#comment-16499324
]
Hu Ziqian commented on YARN-8382:
-
[~miklos.szeg...@cloudera.com], I added a new patch whi
[
https://issues.apache.org/jira/browse/YARN-8382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16501301#comment-16501301
]
Hu Ziqian commented on YARN-8382:
-
[~miklos.szeg...@cloudera.com], thank you for reviewing
[
https://issues.apache.org/jira/browse/YARN-8382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8382:
Description:
As Jiandan said in YARN-6562, NM may delete Cgroup container file timeout with
logs like belo
[
https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8234:
Attachment: YARN-8234.001.patch
> Improve RM system metrics publisher's performance by pushing events to
>
[
https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8234:
Attachment: YARN-8234.002.patch
> Improve RM system metrics publisher's performance by pushing events to
>
[
https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8234:
Attachment: YARN-8234.003.patch
> Improve RM system metrics publisher's performance by pushing events to
>
[
https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16516585#comment-16516585
]
Hu Ziqian commented on YARN-8234:
-
Hi [~leftnoteasy],i‘ve added another patch YARN-8234.0
Hu Ziqian created YARN-8743:
---
Summary: capacity scheduler doesn't set node label to reserved
container
Key: YARN-8743
URL: https://issues.apache.org/jira/browse/YARN-8743
Project: Hadoop YARN
Issu
[
https://issues.apache.org/jira/browse/YARN-8743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian reassigned YARN-8743:
---
Assignee: Hu Ziqian
> capacity scheduler doesn't set node label to reserved container
> -
[
https://issues.apache.org/jira/browse/YARN-8743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8743:
Description:
capacity scheduler doesn't set node label when new a reserved container's
RMContainerImpl. Whe
[
https://issues.apache.org/jira/browse/YARN-8743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8743:
Attachment: YARN-8743.001.patch
> capacity scheduler doesn't set node label to reserved container
>
[
https://issues.apache.org/jira/browse/YARN-8743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604016#comment-16604016
]
Hu Ziqian commented on YARN-8743:
-
I check the failed UT is timeout of
org.apache.hadoop.
[
https://issues.apache.org/jira/browse/YARN-8743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8743:
Attachment: YARN-8743.002.patch
> capacity scheduler doesn't set node label to reserved container
>
[
https://issues.apache.org/jira/browse/YARN-8743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16604018#comment-16604018
]
Hu Ziqian commented on YARN-8743:
-
fix checkstyle in 002.patch
> capacity scheduler doesn
Hu Ziqian created YARN-8764:
---
Summary: Comparison method violates its general contract in Async
Schedule
Key: YARN-8764
URL: https://issues.apache.org/jira/browse/YARN-8764
Project: Hadoop YARN
Is
[
https://issues.apache.org/jira/browse/YARN-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8764:
Description:
We use node label and async schedule in our cluster, and found our rm throw
many {{IllegalArgu
[
https://issues.apache.org/jira/browse/YARN-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8764:
Attachment: YARN-8764.001.patch
> Comparison method violates its general contract in Async Schedule
> --
[
https://issues.apache.org/jira/browse/YARN-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8764:
Attachment: (was: YARN-8764.001.patch)
> Comparison method violates its general contract in Async Schedu
[
https://issues.apache.org/jira/browse/YARN-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8764:
Attachment: YARN-8764-branch-2.8.4.001.patch
> Comparison method violates its general contract in Async Sche
[
https://issues.apache.org/jira/browse/YARN-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8764:
Description:
We use node label and async schedule in our cluster, and found our rm throw
many {{IllegalArgu
[
https://issues.apache.org/jira/browse/YARN-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16610566#comment-16610566
]
Hu Ziqian commented on YARN-8764:
-
hi [~wangda], i find this error in our cluster which ba
[
https://issues.apache.org/jira/browse/YARN-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611515#comment-16611515
]
Hu Ziqian commented on YARN-8764:
-
Hi [~Tao Yang], did your problem only happen when you c
[
https://issues.apache.org/jira/browse/YARN-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian reassigned YARN-8764:
---
Assignee: Hu Ziqian
> Comparison method violates its general contract in Async Schedule
> ---
[
https://issues.apache.org/jira/browse/YARN-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8764:
Affects Version/s: (was: 2.8.0)
2.8.4
> Comparison method violates its general co
[
https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8234:
Attachment: YARN-8234.004.patch
> Improve RM system metrics publisher's performance by pushing events to
>
[
https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hu Ziqian updated YARN-8234:
Attachment: YARN-8234-branch-2.8.3.004.patch
> Improve RM system metrics publisher's performance by pushing
[
https://issues.apache.org/jira/browse/YARN-8234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16613077#comment-16613077
]
Hu Ziqian commented on YARN-8234:
-
[~rohithsharma], sorry for replying too late.
I add pa
84 matches
Mail list logo