[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 @jerryshao Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 Cannot cleanly merge to 2.2, so this will only land to master branch. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 LGTM, merging to master, if possible to 2.2. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81768/ Test PASSed. ---

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81768 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81768/testReport)** for PR 19132 at commit

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81768 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81768/testReport)** for PR 19132 at commit

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81764/ Test FAILed. ---

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81764 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81764/testReport)** for PR 19132 at commit

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81764 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81764/testReport)** for PR 19132 at commit

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81710/ Test PASSed. ---

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81710 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81710/testReport)** for PR 19132 at commit

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81710/testReport)** for PR 19132 at commit

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81709/ Test FAILed. ---

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81709/testReport)** for PR 19132 at commit

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81709/testReport)** for PR 19132 at commit

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/19132 I've been following, still LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 Overall LGTM, @ajbozarth can you please review again? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 @jerryshao @HyukjinKwon Jenkins done.Thanks too much. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81678/ Test PASSed. ---

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81678 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81678/testReport)** for PR 19132 at commit

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 Thanks @HyukjinKwon , I will ping Josh about this thing 😄 . --- - To unsubscribe, e-mail:

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19132 @jerryshao, for triggering tests on Jenkins, I think this should be added by its admin manually as well if I understood correctly. In my case, I asked this to Josh Rosen before via email

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19132 **[Test build #81678 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81678/testReport)** for PR 19132 at commit

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19132 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-11 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 Yes i have confirmed.Below is a testing job: @jerryshao ![_001](https://user-images.githubusercontent.com/26762018/30306401-7c682822-97aa-11e7-82ae-9c5802680dd5.png)

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-11 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 Looks like I don't have the Jenkins permission to trigger UT 😞 . Let me ping @srowen to trigger the test. --- - To

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-11 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 ok to test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-11 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 @jerryshao Thanks for your time. IIUC, event log is completed since driver has not dropped any event of executor which has problem described above.See below,driver only drop two events after

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-11 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 Still I have a question about history server, is you event log an incomplete event log or completed when you met such issue? ---

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-10 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 Done @jerryshao --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-10 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 So your PR description is quite confusing, would you please elaborate your problem in detail and describe how to reproduce the issue. ---

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-08 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 I just appear on history ui since Live ui will use heartbeat metric result @jerryshao(From my understanding) .It appears when Driver exit unnormally,then status of taskinfo can not be updated

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-08 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19132 Is this a problem only in History UI, or it also has issues in Live UI? From my understanding you only pass a last update time for history UI, so is it intended? Also you mentioned "When

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-07 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 @vanzin @zsxwing could you help reivew this?Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-06 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 Ping @kiszk @jerryshao can you help reivew this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-05 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 @ajbozarth I have updated the implementation which only access FS in FSHistoryServerProvider --- - To unsubscribe, e-mail:

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-05 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19132 Thanks for your recommendation @ajbozarth .Could you put a link for your pr?For the problem you mentioned,i have thought about them. 1、FsHistoryServer will always use FS to get event log

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-05 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/19132 Though this may fix the problem it is 100% the wrong way to do it for multiple reasons: 1. The UI should not touch the FS, FS access it abstracted out so the UI is can work with multiple FS

[GitHub] spark issue #19132: [SPARK-21922] Fix duration always updating when task fai...

2017-09-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19132 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional