Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-69037810
Alright LGTM I'm merging this into master thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3467#discussion_r22591785
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -329,6 +319,17 @@ private[history] class
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3467#discussion_r22591679
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -180,14 +176,15 @@ private[history] class
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/3467
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user tsudukim commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-68837548
resolved conflicts!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-68096064
[Test build #24818 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24818/consoleFull)
for PR 3467 at commit
Github user tsudukim commented on a diff in the pull request:
https://github.com/apache/spark/pull/3467#discussion_r22270614
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -180,14 +176,15 @@ private[history] class
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-68098394
[Test build #24818 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24818/consoleFull)
for PR 3467 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-68098398
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-68122296
[Test build #24830 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24830/consoleFull)
for PR 3467 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-68124725
[Test build #24830 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24830/consoleFull)
for PR 3467 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-68124728
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user tsudukim commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-68043299
Thank you for your comments! I'm going to do it in a few days!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3467#discussion_r22139289
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -180,14 +176,15 @@ private[history] class
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3467#discussion_r22139290
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/HistoryPage.scala ---
@@ -117,8 +129,9 @@ private[spark] class HistoryPage(parent:
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3467#discussion_r22139294
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/HistoryPage.scala ---
@@ -117,8 +129,9 @@ private[spark] class HistoryPage(parent:
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3467#discussion_r22139306
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/ApplicationHistoryProvider.scala
---
@@ -25,7 +25,8 @@ private[spark] case class
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3467#discussion_r22139319
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/HistoryPage.scala ---
@@ -65,11 +67,12 @@ private[spark] class HistoryPage(parent:
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/3467#discussion_r22139329
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ---
@@ -180,14 +176,15 @@ private[history] class
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-67722347
Hey @tsudukim this patch is pretty straightforward and it looks good.
Unfortunately you will need to resolve a non-trivial merge conflict with
another patch that I
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-67545486
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-67546098
[Test build #24596 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24596/consoleFull)
for PR 3467 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-67557713
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-67557700
[Test build #24596 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24596/consoleFull)
for PR 3467 at commit
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-67559755
By the way I'm going to block this on #1222, since that one has been open
for a while and I would like to get it merged first before @vanzin has to
resolve more
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-66809813
[Test build #24413 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24413/consoleFull)
for PR 3467 at commit
Github user tsudukim commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-66810500
@andrewor14 Thank you for following this ticket.
I finished rebasing to master.
---
If your project is set up for it, you can reply to this email and have your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-66820869
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-66820853
[Test build #24413 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24413/consoleFull)
for PR 3467 at commit
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-66519222
Hey @tsudukim you will need to rebase to master because there have been
changes to the HS that went in recently. Once you do that I will take a
detailed pass through
Github user ryan-williams commented on a diff in the pull request:
https://github.com/apache/spark/pull/3467#discussion_r21033452
--- Diff:
core/src/main/scala/org/apache/spark/deploy/history/HistoryPage.scala ---
@@ -61,6 +72,15 @@ private[spark] class HistoryPage(parent:
Github user tsudukim commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-64922238
@ryan-williams Thank you for your review!
I fixed them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-64922504
[Test build #23941 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23941/consoleFull)
for PR 3467 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-64926902
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-64601739
[Test build #23892 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23892/consoleFull)
for PR 3467 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-64653027
[Test build #23892 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23892/consoleFull)
for PR 3467 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-64653042
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
GitHub user tsudukim opened a pull request:
https://github.com/apache/spark/pull/3467
[SPARK-2458] Make failed application log visible on History Server
Enabled HistoryServer to show incomplete applications.
We can see the log for incomplete applications by clicking the bottom
Github user tsudukim commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-64509912
In this PR, the sort order is (-endTime, -startTime) which means that
the sorting is still end time for completed apps but is start time for
incomplete apps
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-64510056
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user tsudukim commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-64510079
And these are the screenshots of new UI.
A new link for the page of incomplete applications is added at bottom.
(Show incomplete applications)
Github user tsudukim commented on the pull request:
https://github.com/apache/spark/pull/1558#issuecomment-64510185
@andrewor14 I created a new PR (#3467) as your comment. Please check it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-64513458
Looks great. I will take a detailed pass once the 1.2 release is out of the
window.
---
If your project is set up for it, you can reply to this email and have your
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-64513464
add to whitelist
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-64513684
[Test build #23875 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23875/consoleFull)
for PR 3467 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-64513767
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/3467#issuecomment-64513766
[Test build #23875 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/23875/consoleFull)
for PR 3467 at commit
Github user tsudukim commented on the pull request:
https://github.com/apache/spark/pull/1558#issuecomment-57430441
Thank you @andrewor14
I've researched this problem these days with our environment and it turned
out to be a very rare case as @vanzin suggested first.
(like
Github user tsudukim closed the pull request at:
https://github.com/apache/spark/pull/1558
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1558#issuecomment-57500764
@tsudukim Actually the high-level fix here is not a bad idea. Right now if
the logs don't show up here the user has to manually figure out whether the
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1558#issuecomment-55333925
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1558#issuecomment-55334577
Hi @tsudukim, how does the user see the incomplete applications? As @vanzin
suggested, the semantics of a history server is that it displays completed
applications
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1558#issuecomment-55334702
QA tests have started for PR 1558. This patch DID NOT merge cleanly!
brView progress:
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1558#issuecomment-55334841
QA results for PR 1558:br- This patch FAILED unit tests.brbrFor more
information see test
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1558#issuecomment-55334845
Also, looks like this has merge conflicts. It would be great if you could
rebase to master. Thanks!
---
If your project is set up for it, you can reply to this email
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1558#issuecomment-55164137
QA tests have started for PR 1558. This patch DID NOT merge cleanly!
brView progress:
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1558#issuecomment-55164303
QA results for PR 1558:br- This patch FAILED unit tests.brbrFor more
information see test
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1558#issuecomment-54694566
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/1558#issuecomment-50052724
Disclaimer: haven't looked at the code yet.
I'm a little conflicted about exposing running apps in the history server,
especially this way. First, history sort of
Github user tsudukim commented on the pull request:
https://github.com/apache/spark/pull/1558#issuecomment-50060159
Thank you for following this PR.
Let me explain a little.
I'm sorry I made you misunderstand my purpose with the improper word
uncompleted. The purpose of this
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/1558#issuecomment-50060945
Hmm. A properly-written app that fails should still show up as finished:
val sc = new SparkContext(blah)
try {
doStuff()
} finally {
GitHub user tsudukim opened a pull request:
https://github.com/apache/spark/pull/1558
[SPARK-2458] Make failed application log visible on History Server
Modified to show uncompleted applications in History Server ui.
Modified apps sort rule to startTime base (originally it was
Github user tsudukim commented on the pull request:
https://github.com/apache/spark/pull/1558#issuecomment-49956854
We get the same ui as now by default.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1558#issuecomment-49956952
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
64 matches
Mail list logo
