[GitHub] spark issue #23241: [SPARK-26283][CORE] Enable reading from open frames of z...

2018-12-09 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23241 Thanks a lot @srowen . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #23160: [SPARK-26196][SPARK-26281][WebUI] Total tasks title in t...

2018-12-07 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23160 Thanks @srowen. This issue happens only in master branch. Thank you all for the review and comments

[GitHub] spark issue #23241: [SPARK-26283][CORE] Enable reading from open frames of z...

2018-12-06 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23241 Thanks @srowen . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #23241: [SPARK-26283][CORE] Enable reading from open fram...

2018-12-06 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23241#discussion_r239618167 --- Diff: core/src/main/scala/org/apache/spark/io/CompressionCodec.scala --- @@ -197,4 +201,8 @@ class ZStdCompressionCodec(conf: SparkConf) extends

[GitHub] spark pull request #23241: [SPARK-26283][CORE] Enable reading from open fram...

2018-12-06 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23241#discussion_r239614561 --- Diff: core/src/main/scala/org/apache/spark/io/CompressionCodec.scala --- @@ -197,4 +201,8 @@ class ZStdCompressionCodec(conf: SparkConf) extends

[GitHub] spark pull request #23241: [SPARK-26283][CORE] Enable reading from open fram...

2018-12-06 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23241#discussion_r239534469 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ReplayListenerBus.scala --- @@ -118,10 +118,12 @@ private[spark] class ReplayListenerBus

[GitHub] spark pull request #23241: [SPARK-26283][CORE] Enable reading from open fram...

2018-12-06 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23241#discussion_r239530668 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ReplayListenerBus.scala --- @@ -118,10 +118,12 @@ private[spark] class ReplayListenerBus

[GitHub] spark pull request #23241: [SPARK-26283][CORE] Enable reading from open fram...

2018-12-06 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23241#discussion_r239521593 --- Diff: core/src/main/scala/org/apache/spark/io/CompressionCodec.scala --- @@ -197,4 +201,8 @@ class ZStdCompressionCodec(conf: SparkConf) extends

[GitHub] spark pull request #23241: [SPARK-26283][CORE] Enable reading from open fram...

2018-12-06 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23241#discussion_r239516496 --- Diff: core/src/main/scala/org/apache/spark/io/CompressionCodec.scala --- @@ -197,4 +201,8 @@ class ZStdCompressionCodec(conf: SparkConf) extends

[GitHub] spark issue #23160: [SPARK-26196][SPARK-26281][WebUI] Total tasks title in t...

2018-12-06 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23160 cc @tgravescs @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #23241: [SPARK-26283][CORE] Enable reading from open fram...

2018-12-06 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23241#discussion_r239496266 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ReplayListenerBus.scala --- @@ -118,8 +118,6 @@ private[spark] class ReplayListenerBus extends

[GitHub] spark pull request #23241: [SPARK-26283][CORE] Enable reading from open fram...

2018-12-06 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23241#discussion_r239494478 --- Diff: core/src/main/scala/org/apache/spark/io/CompressionCodec.scala --- @@ -197,4 +201,8 @@ class ZStdCompressionCodec(conf: SparkConf) extends

[GitHub] spark issue #23241: [SPARK-26283][CORE] Enable reading from open frames of z...

2018-12-06 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23241 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #23241: [SPARK-26283][CORE] Enable reading from open frames of z...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23241 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #23241: [SPARK-26283][CORE] Enable reading from open frames of z...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23241 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #23241: [SPARK-26283][CORE] Enable reading open frames of zstd, ...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23241 Thanks @vanzin I updated the title --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark pull request #23241: [SPARK-26283][CORE]When zstd compression enabled,...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23241#discussion_r239232597 --- Diff: core/src/main/scala/org/apache/spark/io/CompressionCodec.scala --- @@ -197,4 +201,8 @@ class ZStdCompressionCodec(conf: SparkConf) extends

[GitHub] spark issue #23240: [SPARK-26281][WebUI] Duration column of task table shoul...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23240 Thanks. I will update the PR title --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23241 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #23241: [SPARK-26283][CORE]When zstd compression enabled,...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23241#discussion_r239219205 --- Diff: core/src/main/scala/org/apache/spark/io/CompressionCodec.scala --- @@ -197,4 +201,8 @@ class ZStdCompressionCodec(conf: SparkConf) extends

[GitHub] spark pull request #23241: [SPARK-26283][CORE]When zstd compression enabled,...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23241#discussion_r239216282 --- Diff: core/src/main/scala/org/apache/spark/io/CompressionCodec.scala --- @@ -197,4 +201,8 @@ class ZStdCompressionCodec(conf: SparkConf) extends

[GitHub] spark issue #23241: [SPARK-26283][CORE]When zstd compression enabled, Inprog...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23241 cc @vanzin @srowen Kindly review --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark pull request #23241: [SPARK-26283][CORE]When zstd compression enabled,...

2018-12-05 Thread shahidki31
GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/23241 [SPARK-26283][CORE]When zstd compression enabled, Inprogress application in the history server appUI showing finished job as running. ## What changes were proposed in this pull request

[GitHub] spark issue #23240: [SPARK-26281][WebUI] Duration column of task table shoul...

2018-12-05 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23240 Hi @gengliangwang , It seems, this was already handled in the PR, https://github.com/apache/spark/pull/23160

[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...

2018-12-04 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23088 Thanks @vanzin @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...

2018-12-04 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23088 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...

2018-12-03 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23088 Retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...

2018-12-03 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23088 Hi @vanzin , I have opened a JIRA for disk store case https://issues.apache.org/jira/browse/SPARK-26260. I will try to work on the same

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-12-03 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r238492653 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusStoreSuite.scala --- @@ -77,6 +77,34 @@ class AppStatusStoreSuite extends SparkFunSuite

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-12-03 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r238492582 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -148,11 +148,20 @@ private[spark] class AppStatusStore

[GitHub] spark pull request #23191: [SPARK-26219][CORE][branch-2.4] Executor summary ...

2018-12-03 Thread shahidki31
Github user shahidki31 closed the pull request at: https://github.com/apache/spark/pull/23191 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #23191: [SPARK-26219][CORE][branch-2.4] Executor summary should ...

2018-12-03 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23191 Thanks a lot @vanzin @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #23205: [SPARK-26253][WebUI] : Task Summary Metrics Table on Sta...

2018-12-03 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23205 Thanks @pgandhi999 :+1: --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...

2018-12-02 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23088 Thanks @pgandhi999 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-12-02 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r238135277 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -148,11 +148,20 @@ private[spark] class AppStatusStore

[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...

2018-12-02 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23088 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...

2018-12-01 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23088 Hi @pgandhi999 , It seems after your checkins, when there is no summary metrics, it is displaying empty table rather than a message which shown in the PR title. could you please help me to fix

[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...

2018-12-01 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23088 Hi @srowen . Yes. Currently for disk store case, we need to have a more optimized code. > While it makes some sense I have two concerns: different answers based on disk vs memory st

[GitHub] spark issue #23160: [SPARK-26196][WebUI] Total tasks title in the stage page...

2018-12-01 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23160 > So as far as I can think, I expect that if you sort the Duration column, it will perform sorting on row.duration instead of row.taskMetrics.executorRunTime, thus, not getting the desi

[GitHub] spark pull request #23160: [SPARK-26196][WebUI] Total tasks title in the sta...

2018-12-01 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23160#discussion_r238060278 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -661,8 +662,8 @@ $(document).ready(function

[GitHub] spark issue #23191: [SPARK-26219][CORE][branch-2.4] Executor summary should ...

2018-12-01 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23191 cc @vanzin . Kindly review --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #23160: [SPARK-26196][WebUI] Total tasks title in the stage page...

2018-11-30 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23160 Hi @pgandhi999 , There is a small nit there. Recently we fixed a bug related to the "duration" metrics in the tasks table (see https://github.com/apache/spark/pull/23081), but t

[GitHub] spark issue #23191: [SPARK-26219][CORE][branch-2.4] Executor summary should ...

2018-11-30 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23191 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #23191: [SPARK-26219][CORE][branch-2.4] Executor summary should ...

2018-11-30 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23191 Jenkins, test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #23191: [SPARK-26219][CORE][branch-2.4] Executor summary ...

2018-11-30 Thread shahidki31
GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/23191 [SPARK-26219][CORE][branch-2.4] Executor summary should get updated for failure jobs in the history server UI ## What changes were proposed in this pull request? Back port the commit https

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

2018-11-30 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23181 Thanks @vanzin. I will open a PR in 2.4 branch --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

2018-11-30 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23181 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #23181: [SPARK-26219][CORE] Executor summary should get u...

2018-11-30 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23181#discussion_r237959492 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala --- @@ -1274,47 +1274,69 @@ class AppStatusListenerSuite extends

[GitHub] spark pull request #23181: [SPARK-26219][CORE] Executor summary should get u...

2018-11-30 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23181#discussion_r237959457 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala --- @@ -1274,47 +1274,69 @@ class AppStatusListenerSuite extends

[GitHub] spark pull request #23160: [SPARK-26196][WebUI] Total tasks title in the sta...

2018-11-30 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23160#discussion_r237907702 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -610,7 +610,8 @@ $(document).ready(function

[GitHub] spark issue #23160: [SPARK-26196][WebUI] Total tasks title in the stage page...

2018-11-30 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23160 @tgravescs Yes. Task table is showing all the tasks. Sorry, I didn't put the whole screen shot. --- - To unsubscribe, e-mail

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

2018-11-29 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23181 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #23181: [SPARK-26219][CORE] Executor summary should get updated ...

2018-11-29 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23181 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #23168: [SPARK-26207][doc]add PowerIterationClustering (P...

2018-11-29 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23168#discussion_r237637501 --- Diff: docs/ml-clustering.md --- @@ -265,3 +265,38 @@ Refer to the [R API docs](api/R/spark.gaussianMixture.html) for more details

[GitHub] spark issue #23181: [SPARK-26100][CORE] Executor summary should get updated ...

2018-11-29 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23181 cc @vanzin Kindly review --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #23181: [SPARK-26100][CORE] Executor summary should get updated ...

2018-11-29 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23181 Before patch: ![screenshot from 2018-11-29 22-13-34](https://user-images.githubusercontent.com/23054875/49246338-a21ead00-f43a-11e8-8214-f1020420be52.png) After patch

[GitHub] spark issue #23181: [SPARK-26100][CORE] Executor summary should get updated ...

2018-11-29 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23181 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #23181: Executor summary should update for history events

2018-11-29 Thread shahidki31
GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/23181 Executor summary should update for history events ## What changes were proposed in this pull request? The root cause of the problem is, whenever the taskEnd event comes after stageCompleted

[GitHub] spark pull request #23180: Executor summary should update for history events

2018-11-29 Thread shahidki31
Github user shahidki31 closed the pull request at: https://github.com/apache/spark/pull/23180 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #23180: Executor summary should update for history events

2018-11-29 Thread shahidki31
GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/23180 Executor summary should update for history events ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested

[GitHub] spark issue #23158: [SPARK-26186][SPARK-26184][CORE] Last updated time is no...

2018-11-29 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23158 Thanks a lot @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #23158: [SPARK-26186][SPARK-26184][CORE] Last updated time is no...

2018-11-29 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23158 It is random failure. Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #23158: [SPARK-26186][SPARK-26184][CORE] Last updated time is no...

2018-11-28 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23158 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #23158: [SPARK-26186][SPARK-26184][CORE] Last updated time is no...

2018-11-28 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23158 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #23158: [SPARK-26186][SPARK-26184] Last updated time is n...

2018-11-28 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23158#discussion_r237295187 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala --- @@ -334,6 +334,42 @@ class FsHistoryProviderSuite

[GitHub] spark pull request #23158: [SPARK-26186][SPARK-26184] Last updated time is n...

2018-11-28 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23158#discussion_r237292237 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala --- @@ -334,6 +334,42 @@ class FsHistoryProviderSuite

[GitHub] spark pull request #23158: [SPARK-26186][SPARK-26184] Last updated time is n...

2018-11-28 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23158#discussion_r237291446 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/FsHistoryProviderSuite.scala --- @@ -334,6 +334,42 @@ class FsHistoryProviderSuite

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-28 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r237154542 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -222,29 +223,20 @@ private[spark] class AppStatusStore( val

[GitHub] spark pull request #23160: [SPARK-26196]Total tasks title in the stage page ...

2018-11-27 Thread shahidki31
GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/23160 [SPARK-26196]Total tasks title in the stage page is incorrect when there are failed or killed tasks ## What changes were proposed in this pull request? Total tasks = numCompleteTasks

[GitHub] spark issue #23158: [SPARK-26186][SPARK-26184] Last updated time is not gett...

2018-11-27 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23158 Expired was determined by the `lastUpdateTime` which we need to update when ever an eventLog update happens https://github.com/apache/spark/blob/2d89d109e19d1e84c4ada3c9d5d48cfcf3d997ea/core

[GitHub] spark issue #23158: [SPARK-26186][SPARK-26184] Last updated time is not gett...

2018-11-27 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23158 cc @vanzin @srowen Kindly review --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark pull request #23158: [SPARK-26186][SPARK-26184] Last updated time is n...

2018-11-27 Thread shahidki31
GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/23158 [SPARK-26186][SPARK-26184] Last updated time is not getting updated for the Inprogress application ## What changes were proposed in this pull request? When

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-26 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236455725 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -222,29 +223,20 @@ private[spark] class AppStatusStore( val

[GitHub] spark issue #23038: [SPARK-25451][SPARK-26100][CORE]Aggregated metrics table...

2018-11-26 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23038 Oh. I could have re based and tested in local. Thanks @vanzin for the fix. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #23038: [SPARK-25451][SPARK-26100][CORE]Aggregated metrics table...

2018-11-26 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23038 Thank you @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-26 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236409746 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusStoreSuite.scala --- @@ -95,10 +123,18 @@ class AppStatusStoreSuite extends SparkFunSuite

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-26 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236409661 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -222,29 +223,20 @@ private[spark] class AppStatusStore( val

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-26 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236409557 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -222,29 +223,20 @@ private[spark] class AppStatusStore( val

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-26 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236234523 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -150,8 +150,9 @@ private[spark] class AppStatusStore

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-26 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236234400 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -222,29 +223,20 @@ private[spark] class AppStatusStore( val

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-25 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236120634 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -222,29 +223,20 @@ private[spark] class AppStatusStore( val

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-25 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236087784 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -222,29 +223,20 @@ private[spark] class AppStatusStore( val

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-25 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236087746 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusStoreSuite.scala --- @@ -77,6 +77,30 @@ class AppStatusStoreSuite extends SparkFunSuite

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-25 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236087749 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusStoreSuite.scala --- @@ -77,6 +77,30 @@ class AppStatusStoreSuite extends SparkFunSuite

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-25 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236087731 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -222,29 +223,20 @@ private[spark] class AppStatusStore( val

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-25 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236087741 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -222,29 +223,20 @@ private[spark] class AppStatusStore( val

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-25 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236087736 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -222,29 +223,20 @@ private[spark] class AppStatusStore( val

[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...

2018-11-25 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23088 Thank you @srowen for the review. I will update the PR. > When did the behavior change? you said you found the commit. https://issues.apache.org/jira/browse/SPARK-20

[GitHub] spark pull request #23134: [SPARK-25504][Docs] Update doc about retained tas...

2018-11-25 Thread shahidki31
Github user shahidki31 closed the pull request at: https://github.com/apache/spark/pull/23134 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #23134: [SPARK-25504][Docs] Update doc about retained tasks, job...

2018-11-25 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23134 In the JIRA, when the user want retainedTasks as 6, actually retained around 58000. As per the code, it retained atleast 0.9*retainedTasks to retainedTasks. if we can clearly specify

[GitHub] spark issue #23134: [SPARK-25504][Docs] Update doc about retained tasks, job...

2018-11-25 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23134 Hi @srowen , Some of the users seems getting confused with the threshold values, is it worth to update the docs? Thanks

[GitHub] spark pull request #23134: [SPARK-25504][Docs] Update doc about retained tas...

2018-11-25 Thread shahidki31
GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/23134 [SPARK-25504][Docs] Update doc about retained tasks, jobs and executions ## What changes were proposed in this pull request? Updated the documentation about spark.ui.retainedTasks

[GitHub] spark issue #23038: [SPARK-25451][SPARK-26100][CORE]Aggregated metrics table...

2018-11-25 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23038 @vanzin could you please check the updated changes, thanks --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...

2018-11-25 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23088 @srowen , could you please review the PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23016: [SPARK-26006][mllib] unpersist 'dataInternalRepr' in the...

2018-11-25 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23016 > I am currently using it with spark 2.3 as > > org.apache.spark > spark-mllib_2.11 > > How can i get this fix? You can cherry pick the commit from

[GitHub] spark issue #23016: [SPARK-26006][mllib] unpersist 'dataInternalRepr' in the...

2018-11-25 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23016 @idlevi Actually, input and output of the prefix span are RDD. Earlier intermediate rdd was cached, now final rdd is cached, and materialized it. So, if you materialize the model, earlier

[GitHub] spark issue #23088: [WIP][SPARK-26119][CORE][WEBUI]Task summary table should...

2018-11-21 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23088 (Still need more things to test, so changed to WIP) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #23088: [WIP][SPARK-26119][CORE][WEBUI]Task summary table should...

2018-11-21 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23088 @srowen Yes. I did check the commit history, and the PR which modified the behavior didn't mention about this behavior change. If it is intentional, then we should change the table title

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-21 Thread shahidki31
GitHub user shahidki31 reopened a pull request: https://github.com/apache/spark/pull/23088 [SPARK-26119][CORE][WEBUI]Task summary table should contain only successful tasks' metrics ## What changes were proposed in this pull request? Task summary table in the stage page

[GitHub] spark issue #23081: [SPARK-26109][WebUI]Duration in the task summary metrics...

2018-11-21 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23081 Thank you @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

  1   2   3   4   >