[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 @jerryshao Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 Cannot cleanly merge to 2.2, so this will only land to master branch. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.a

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 LGTM, merging to master, if possible to 2.2. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81768/ Test PASSed. ---

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81768 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81768/testReport)** for PR 19132 at commit [`06035f7`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81768 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81768/testReport)** for PR 19132 at commit [`06035f7`](https://github.com/apache/spark/commit/06

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81764/ Test FAILed. ---

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81764 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81764/testReport)** for PR 19132 at commit [`8ec4319`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81764 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81764/testReport)** for PR 19132 at commit [`8ec4319`](https://github.com/apache/spark/commit/8e

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81710/ Test PASSed. ---

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81710 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81710/testReport)** for PR 19132 at commit [`f0e1ded`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81710/testReport)** for PR 19132 at commit [`f0e1ded`](https://github.com/apache/spark/commit/f0

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81709/ Test FAILed. ---

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81709/testReport)** for PR 19132 at commit [`560d442`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81709/testReport)** for PR 19132 at commit [`560d442`](https://github.com/apache/spark/commit/56

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/19132 I've been following, still LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-m

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 Overall LGTM, @ajbozarth can you please review again? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For ad

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 @jerryshao @HyukjinKwon Jenkins done.Thanks too much. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For addi

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81678/ Test PASSed. ---

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81678 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81678/testReport)** for PR 19132 at commit [`25fe22c`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 Thanks @HyukjinKwon , I will ping Josh about this thing 😄 . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.o

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19132 @jerryshao, for triggering tests on Jenkins, I think this should be added by its admin manually as well if I understood correctly. In my case, I asked this to Josh Rosen before via email private

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81678 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81678/testReport)** for PR 19132 at commit [`25fe22c`](https://github.com/apache/spark/commit/25

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19132 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-11 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 Yes i have confirmed.Below is a testing job: @jerryshao ![_001](https://user-images.githubusercontent.com/26762018/30306401-7c682822-97aa-11e7-82ae-9c5802680dd5.png) ![_002](https://user-im

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-11 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 Looks like I don't have the Jenkins permission to trigger UT 😞 . Let me ping @srowen to trigger the test. --- - To unsubscr

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-11 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 ok to test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-11 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 @jerryshao Thanks for your time. IIUC, event log is completed since driver has not dropped any event of executor which has problem described above.See below,driver only drop two events after shuttin

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-10 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 Still I have a question about history server, is you event log an incomplete event log or completed when you met such issue? --- -

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-10 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 Done @jerryshao --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-10 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 So your PR description is quite confusing, would you please elaborate your problem in detail and describe how to reproduce the issue. ---

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-08 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 I just appear on history ui since Live ui will use heartbeat metric result @jerryshao(From my understanding) .It appears when Driver exit unnormally,then status of taskinfo can not be updated correc

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-08 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 Is this a problem only in History UI, or it also has issues in Live UI? From my understanding you only pass a last update time for history UI, so is it intended? Also you mentioned "When

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-07 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 @vanzin @zsxwing could you help reivew this?Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additi

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-06 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 Ping @kiszk @jerryshao can you help reivew this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For addition

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-05 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 @ajbozarth I have updated the implementation which only access FS in FSHistoryServerProvider --- - To unsubscribe, e-mail: revie

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-05 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 Thanks for your recommendation @ajbozarth .Could you put a link for your pr?For the problem you mentioned,i have thought about them. 1、FsHistoryServer will always use FS to get event log 2ã

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-05 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/19132 Though this may fix the problem it is 100% the wrong way to do it for multiple reasons: 1. The UI should not touch the FS, FS access it abstracted out so the UI is can work with multiple FS im

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional