Hey Everyone,
I am stuck in an issue of logs rotation with spark streaming job which run
on yarn. Yarn continuously write container stderr and stdout logs to
containers/ folder and it fills up disk space and crash cluster. I want to
continuously move logs to hdfs or s3 and then truncate source
[
https://issues.apache.org/jira/browse/YARN-6300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902391#comment-15902391
]
Yuanbo Liu commented on YARN-6300:
--
[~haibochen] Thanks for your review.
> NULL_UPDATE_REQUESTS is
[
https://issues.apache.org/jira/browse/YARN-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902269#comment-15902269
]
Steven Rand commented on YARN-6308:
---
Attached a new patch to HADOOP-14062, though I think this issue
[
https://issues.apache.org/jira/browse/YARN-6264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yufei Gu updated YARN-6264:
---
Attachment: YARN-6264.003.patch
> Resource comparison should depends on policy
>
[
https://issues.apache.org/jira/browse/YARN-6120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steven Rand resolved YARN-6120.
---
Resolution: Duplicate
I now have the ability to submit a patch for YARN-2985, so this duplicate JIRA
[
https://issues.apache.org/jira/browse/YARN-6312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuchang updated YARN-6312:
--
Description:
{code}
17/03/08 10:26:39 INFO retry.RetryInvocationHandler: Exception while invoking
allocate of
[
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902469#comment-15902469
]
Sunil G commented on YARN-6164:
---
[~benson.qiu], Thanks for the taking interest to contribute. Really
[
https://issues.apache.org/jira/browse/YARN-6307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902482#comment-15902482
]
Tao Jie commented on YARN-6307:
---
Thank you [~yufeigu], FairShareComparator#compare is called very frequently
[
https://issues.apache.org/jira/browse/YARN-2985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steven Rand reassigned YARN-2985:
-
Assignee: Steven Rand
> YARN should support to delete the aggregated logs for Non-MapReduce
>
Shane Kumpf created YARN-6305:
-
Summary: Improve signaling of short lived containers
Key: YARN-6305
URL: https://issues.apache.org/jira/browse/YARN-6305
Project: Hadoop YARN
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/YARN-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901678#comment-15901678
]
Robert Kanter commented on YARN-6050:
-
Test failure looks to be unrelated (YARN-5548).
> AMs can't be
[
https://issues.apache.org/jira/browse/YARN-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901717#comment-15901717
]
Haibo Chen commented on YARN-5169:
--
[~sjlee0] [~gtCarrera9], IIUC from the above discussion, RM is already
[
https://issues.apache.org/jira/browse/YARN-6299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901645#comment-15901645
]
Yufei Gu commented on YARN-6299:
It makes sense to me that an app is always needy when its demand is less
[
https://issues.apache.org/jira/browse/YARN-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Hung updated YARN-5948:
Attachment: YARN-5948-YARN-5734.007.patch
> Implement MutableConfigurationManager for handling
[
https://issues.apache.org/jira/browse/YARN-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haibo Chen reassigned YARN-5169:
Assignee: Haibo Chen
> most YARN events have timestamp of -1
>
[
https://issues.apache.org/jira/browse/YARN-6297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901771#comment-15901771
]
Hudson commented on YARN-6297:
--
FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #11373 (See
[
https://issues.apache.org/jira/browse/YARN-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901696#comment-15901696
]
Benson Qiu commented on YARN-6164:
--
[~sunilg], [~leftnoteasy]: Sure, I can add {{QueueCapacities}} to
Jian He created YARN-6306:
-
Summary: NMClient API change for container upgrade
Key: YARN-6306
URL: https://issues.apache.org/jira/browse/YARN-6306
Project: Hadoop YARN
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/YARN-6297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901740#comment-15901740
]
Robert Kanter commented on YARN-6297:
-
+1
> TestAppLogAggregatorImp.verifyFilesUploaded() should check
[
https://issues.apache.org/jira/browse/YARN-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shane Kumpf reassigned YARN-6305:
-
Assignee: Shane Kumpf
> Improve signaling of short lived containers
>
[
https://issues.apache.org/jira/browse/YARN-5818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shane Kumpf reassigned YARN-5818:
-
Assignee: Shane Kumpf
> Support the Docker Live Restore feature
>
[
https://issues.apache.org/jira/browse/YARN-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901655#comment-15901655
]
Shane Kumpf commented on YARN-6305:
---
I've been looking into this, so I'll take ownership and will put
[
https://issues.apache.org/jira/browse/YARN-6196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902630#comment-15902630
]
Sunil G commented on YARN-6196:
---
Change looks fine to me.. Committing later today if there are no other
[
https://issues.apache.org/jira/browse/YARN-5496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902631#comment-15902631
]
Sunil G commented on YARN-5496:
---
+1. Will commit later today if there are no objections.
> Make Node Heatmap
[
https://issues.apache.org/jira/browse/YARN-6312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuchang updated YARN-6312:
--
Description:
{quote}
java.net.ConnectException: Call From 12.23.45.2:8030 failed on connection
exception:
[
https://issues.apache.org/jira/browse/YARN-6312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wuchang updated YARN-6312:
--
Description:
{code}
java.net.ConnectException: Call From 12.23.45.2:8030 failed on connection
exception:
wuchang created YARN-6312:
-
Summary: Applicatioin trying to connect standby resourcemanager's
8030 port and failed
Key: YARN-6312
URL: https://issues.apache.org/jira/browse/YARN-6312
Project: Hadoop YARN
[
https://issues.apache.org/jira/browse/YARN-6301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902618#comment-15902618
]
Tao Jie commented on YARN-6301:
---
[~templedf], today queue's weight is allowed to be zero even negative. It
[
https://issues.apache.org/jira/browse/YARN-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901562#comment-15901562
]
Eric Badger commented on YARN-4266:
---
[~sidharta-s], I agree with your assessment. I don't see this
[
https://issues.apache.org/jira/browse/YARN-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901600#comment-15901600
]
Jason Lowe commented on YARN-4236:
--
Oops spoke too soon. Just before committing I noticed there's a place
[
https://issues.apache.org/jira/browse/YARN-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901534#comment-15901534
]
Shane Kumpf commented on YARN-4266:
---
I took another look at the progress being made on user namespaces in
[
https://issues.apache.org/jira/browse/YARN-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901607#comment-15901607
]
Eric Badger commented on YARN-4266:
---
Looks like I should have reloaded the page before commenting as I
[
https://issues.apache.org/jira/browse/YARN-4051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901553#comment-15901553
]
Jason Lowe commented on YARN-4051:
--
Thanks for updating the patch! In the future, please don't delete
[
https://issues.apache.org/jira/browse/YARN-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901575#comment-15901575
]
Jason Lowe commented on YARN-4236:
--
+1 lgtm. Committing this.
> Metric for aggregated resources
[
https://issues.apache.org/jira/browse/YARN-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901892#comment-15901892
]
Hadoop QA commented on YARN-5948:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem ||
[
https://issues.apache.org/jira/browse/YARN-6165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901915#comment-15901915
]
Jason Lowe commented on YARN-6165:
--
I'd like to take a quick look, and this needs a Jenkins run anyway.
>
[
https://issues.apache.org/jira/browse/YARN-6165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901921#comment-15901921
]
Eric Payne commented on YARN-6165:
--
bq. this needs a Jenkins run anyway
Thanks [~jlowe]. I kicked the
[
https://issues.apache.org/jira/browse/YARN-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901859#comment-15901859
]
Hadoop QA commented on YARN-5948:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem ||
[
https://issues.apache.org/jira/browse/YARN-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Saxena updated YARN-6237:
---
Summary: Move UID constant to TimelineReaderUtils (was: Move UID constant
into TimelineReaderUtils)
[
https://issues.apache.org/jira/browse/YARN-6165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901886#comment-15901886
]
Eric Payne commented on YARN-6165:
--
bq. I think patch looks fine for me.
Thanks [~sunilg]. Will you be
[
https://issues.apache.org/jira/browse/YARN-6237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901885#comment-15901885
]
Varun Saxena commented on YARN-6237:
Committed to YARN-5355, YARN-5355-branch-2.
Thanks [~rohithsharma]
Yufei Gu created YARN-6307:
--
Summary: Refactor FairShareComparator#compare
Key: YARN-6307
URL: https://issues.apache.org/jira/browse/YARN-6307
Project: Hadoop YARN
Issue Type: Bug
[
https://issues.apache.org/jira/browse/YARN-6302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901945#comment-15901945
]
ASF GitHub Bot commented on YARN-6302:
--
GitHub user szegedim opened a pull request:
[
https://issues.apache.org/jira/browse/YARN-6165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901981#comment-15901981
]
Hadoop QA commented on YARN-6165:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem ||
[
https://issues.apache.org/jira/browse/YARN-6165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901993#comment-15901993
]
Hadoop QA commented on YARN-6165:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem ||
[
https://issues.apache.org/jira/browse/YARN-6298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yufei Gu reassigned YARN-6298:
--
Assignee: (was: Yufei Gu)
> Metric preemptCall is not used in new preemption.
>
[
https://issues.apache.org/jira/browse/YARN-6300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902021#comment-15902021
]
Haibo Chen commented on YARN-6300:
--
Thanks [~yuanbo] for the patch. +1 nonbinding.
> NULL_UPDATE_REQUESTS
Manoj Govindassamy created YARN-6308:
Summary: Fix TestAMRMClient compilation errors
Key: YARN-6308
URL: https://issues.apache.org/jira/browse/YARN-6308
Project: Hadoop YARN
Issue Type:
[
https://issues.apache.org/jira/browse/YARN-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901986#comment-15901986
]
Manoj Govindassamy commented on YARN-6308:
--
bq. Yes, it looks like the changes in YARN-6218
Daniel Templeton created YARN-6309:
--
Summary: Fair scheduler docs should have the queue and
queuePlacementPolicy elements listed in bold so that they're easier to see
Key: YARN-6309
URL:
[
https://issues.apache.org/jira/browse/YARN-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902032#comment-15902032
]
Haibo Chen commented on YARN-6290:
--
Title says it is fair scheduler, but the stacktrace indicates Capacity
[
https://issues.apache.org/jira/browse/YARN-349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J reassigned YARN-349:
Assignee: (was: Harsh J)
> Send out last-minute load averages in TaskTrackerStatus
>
[
https://issues.apache.org/jira/browse/YARN-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901377#comment-15901377
]
Eric Badger commented on YARN-4236:
---
Thanks, [~lichangleo]!
[~jlowe], could you review?
> Metric for
[
https://issues.apache.org/jira/browse/YARN-6280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901427#comment-15901427
]
Sunil G commented on YARN-6280:
---
[~cltlfcjin]
Thanks for the patch.
I think we need to keep the default
[
https://issues.apache.org/jira/browse/YARN-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902184#comment-15902184
]
Haibo Chen commented on YARN-6288:
--
[~ajisakaa] Have you considered making LogWriter closable so that try
[
https://issues.apache.org/jira/browse/YARN-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902193#comment-15902193
]
Wangda Tan commented on YARN-5948:
--
Thanks [~jhung] for working on the patch, few minor comments:
1) In
Haibo Chen created YARN-6310:
Summary: OutputStreams in AggregatedLogFormat.LogWriter can be
left open upon exceptions
Key: YARN-6310
URL: https://issues.apache.org/jira/browse/YARN-6310
Project: Hadoop
[
https://issues.apache.org/jira/browse/YARN-6165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902149#comment-15902149
]
Hudson commented on YARN-6165:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11375 (See
[
https://issues.apache.org/jira/browse/YARN-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated YARN-5311:
-
Labels: (was: oct16-easy)
> Document graceful decommission CLI and usage
>
[
https://issues.apache.org/jira/browse/YARN-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902101#comment-15902101
]
Junping Du commented on YARN-5311:
--
Sorry for coming late on this as reviewing document is always a
[
https://issues.apache.org/jira/browse/YARN-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902101#comment-15902101
]
Junping Du edited comment on YARN-5311 at 3/8/17 10:41 PM:
---
Sorry for coming late
[
https://issues.apache.org/jira/browse/YARN-6165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902110#comment-15902110
]
Jason Lowe commented on YARN-6165:
--
+1 lgtm. The TestRMRestart failure is unrelated and will be fixed by
[
https://issues.apache.org/jira/browse/YARN-5956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901299#comment-15901299
]
Kai Sasaki commented on YARN-5956:
--
[~sunilg] Thanks. I updated the patch. The failed test seems fails
Ravi Prakash created YARN-6311:
--
Summary: We should write documentation for
RollingLevelDBTimelineStore
Key: YARN-6311
URL: https://issues.apache.org/jira/browse/YARN-6311
Project: Hadoop YARN
[
https://issues.apache.org/jira/browse/YARN-3448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902243#comment-15902243
]
Ravi Prakash commented on YARN-3448:
Thanks for the awesome design and fix Jon! I've opened YARN-6311
[
https://issues.apache.org/jira/browse/YARN-6207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900878#comment-15900878
]
Hudson commented on YARN-6207:
--
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11369 (See
[
https://issues.apache.org/jira/browse/YARN-6300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuanbo Liu reassigned YARN-6300:
Assignee: Yuanbo Liu
> NULL_UPDATE_REQUESTS is redundant in TestFairScheduler
>
[
https://issues.apache.org/jira/browse/YARN-6300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuanbo Liu updated YARN-6300:
-
Attachment: YARN-6300.001.patch
upload v1 patch for this JIRA.
> NULL_UPDATE_REQUESTS is redundant in
[
https://issues.apache.org/jira/browse/YARN-6300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901063#comment-15901063
]
Hadoop QA commented on YARN-6300:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem ||
69 matches
Mail list logo