Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23081
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23081
again, failure unrelated to the PR.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23081
It is also a random failure.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23081
Thank you @gengliangwang
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23081#discussion_r235075045
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---
@@ -996,7 +996,7 @@ private[ui] object ApiHelper {
HEADER_EXECUTOR
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23081#discussion_r235075470
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---
@@ -996,7 +996,7 @@ private[ui] object ApiHelper {
HEADER_EXECUTOR
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23081#discussion_r235069217
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---
@@ -996,7 +996,7 @@ private[ui] object ApiHelper {
HEADER_EXECUTOR
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23081
Seems random failure, unrelated to the PR.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23081
Thank you @srowen . Till Spark2.2, duration (which was 'executorRunTime')
in the task table and the summary table were consistent. Apart from the
duration, all the metrics in the task table
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23088
Previous versions of Spark were showing only completed tasks summary
metrics. I think it make more sense to show only finished tasks, rather than
showing metrics of killed and failed
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23088#discussion_r235028381
--- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala
---
@@ -238,8 +239,16 @@ private[spark] class AppStatusStore
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23081
cc @vanzin @srowen Kindly review.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user shahidki31 closed the pull request at:
https://github.com/apache/spark/pull/23088
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23088
Need to handle more scenarios and optimization, I'm closing this for some
moment. Will update the PR.
---
-
To unsubscribe
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23088
cc @vanzin @srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23038
It is a random failure. Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23081
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user shahidki31 opened a pull request:
https://github.com/apache/spark/pull/23088
[SPARK-26119][CORE][WEBUI]Task summary table should contain only successful
tasks' metrics
## What changes were proposed in this pull request?
Task summary table in the stage page
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23038#discussion_r234741288
--- Diff:
core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala ---
@@ -1275,6 +1275,49 @@ class AppStatusListenerSuite extends
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23081
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23081
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user shahidki31 opened a pull request:
https://github.com/apache/spark/pull/23081
[SPARK-26109][WebUI]Duration in the task summary metrics table and the task
table are different
## What changes were proposed in this pull request?
Task summary displays the summary
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23038
It is random failure
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23016
Thank you @srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23038
**App UI from History server** after the patch:
 , and the current fix alone
is not sufficient for history server case
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23038#discussion_r234400684
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -565,7 +567,11 @@ private[spark] class AppStatusListener
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23038#discussion_r234399022
--- Diff:
core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala ---
@@ -1275,6 +1275,49 @@ class AppStatusListenerSuite extends
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23038#discussion_r234399018
--- Diff:
core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala ---
@@ -1275,6 +1275,49 @@ class AppStatusListenerSuite extends
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23038#discussion_r234399010
--- Diff:
core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala ---
@@ -1275,6 +1275,49 @@ class AppStatusListenerSuite extends
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23038#discussion_r234399006
--- Diff:
core/src/test/scala/org/apache/spark/status/AppStatusListenerSuite.scala ---
@@ -1275,6 +1275,49 @@ class AppStatusListenerSuite extends
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23016#discussion_r234396276
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/fpm/PrefixSpan.scala
---
@@ -174,6 +174,10 @@ class PrefixSpan private (
val
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23038#discussion_r234367532
--- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala ---
@@ -376,6 +376,8 @@ private class LiveStage extends LiveEntity
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23038#discussion_r234340680
--- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala ---
@@ -376,6 +376,8 @@ private class LiveStage extends LiveEntity
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23038#discussion_r234337780
--- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala ---
@@ -376,6 +376,8 @@ private class LiveStage extends LiveEntity
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23038#discussion_r234072070
--- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala ---
@@ -63,6 +63,7 @@ case class ApplicationAttemptInfo private[spark
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23038#discussion_r234034325
--- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala ---
@@ -63,6 +63,7 @@ case class ApplicationAttemptInfo private[spark
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23038#discussion_r234032510
--- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala ---
@@ -63,6 +63,7 @@ case class ApplicationAttemptInfo private[spark
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23024
Thank you @srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23038
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23038
Retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23016
Thanks @srowen , I will update the PR
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23038
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23038
I'll update the PR, for test failures
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23016
cc @srowen Kindly review
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/23038
cc @vanzin @srowen kindly review
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
GitHub user shahidki31 opened a pull request:
https://github.com/apache/spark/pull/23038
[SPARK-25451][CORE][WEBUI]Aggregated metrics table doesn't show the right
number of the total tasks
## What changes were proposed in this pull request?
Total tasks in the aggregated table
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23024#discussion_r233557648
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/ExecutorTable.scala
---
@@ -92,16 +93,7 @@ private[ui] class ExecutorTable(stage: StageData
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23024#discussion_r233555975
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/ExecutorTable.scala
---
@@ -70,6 +70,7 @@ private[ui] class ExecutorTable(stage: StageData, store
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/23024#discussion_r233554755
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/ExecutorTable.scala
---
@@ -92,16 +93,7 @@ private[ui] class ExecutorTable(stage: StageData
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/21627
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/21627
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user shahidki31 opened a pull request:
https://github.com/apache/spark/pull/23016
[SPARK-26006][mllib] unpersist 'dataInternalRepr' in the PrefixSpan
## What changes were proposed in this pull request?
Mllib's Prefixspan - run method - cached RDD stays in cache. After run
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22502
@cloud-fan Thanks. I will check and update the PR.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22502#discussion_r230811926
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFsRelation.scala
---
@@ -86,10 +89,28 @@ case class HadoopFsRelation
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
Thank you @srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
Hi @HyukjinKwon, Could you please retrigger the PR builder?
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
@srowen @gengliangwang There is one more place where the WEBUI can throw an
exception.
https://github.com/apache/spark/blob/1a7abf3f453f7d6012d7e842cf05f29f3afbb3bc/core/src/main/scala/org
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22864
Thank you @srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22864
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
Thank you @gengliangwang
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
I have tested all the scenarios in the WEBUI, and it is working as per
expected.
---
-
To unsubscribe, e-mail: reviews
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
We can remove the method 'prevPageSizeFormField' from pagedTable and
corresponding page classes. Because this method is no longer required, as it
was for checking IndexOutOfBound
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22864
The failure is unrelated to the PR. It is a known failure.
https://issues.apache.org/jira/browse/SPARK-25923
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
I have modified based on @gengliangwang 's suggestion. Also, we can remove
the check for page size in all the page class, because we handle the same in
the pagedTable class
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
Hi @srowen , After the PR, the following check also can be removed, because
the check is for preventing the OutOfBoundException, But no OutOfBound
exception happens after the PR
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
Hi @gengliangwang , Yes. That also doable. Currently when the
IndexOutOfBound Exception comes, the page is navigating to the first page. Also
to make the behavior consistent with the page size
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22864
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
Hi @srowen , The current behavior of WEBUI is,
1) When user enters a page size more than current page size, It falls back
to first page (In all the pages)
2) When the user enters a page
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22864
Seems random error.
Jenkins, retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
The current behavior is, If we enter a value more than the maximum page
number, the page navigation bar shows the user is in first page and throws an
exception. So, if we really want to throw
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
@gengliangwang IMHO, We should try to avoid exceptions in the WEBUI.
User will come to know which page he is, from the page navigation bar
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
Yes. Thank you @srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
The code here (and all the other pages) intended to check out of bound
exception and fall back to the first page.
https://github.com/apache/spark/blob
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22914
cc @srowen . Kindly review
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
GitHub user shahidki31 opened a pull request:
https://github.com/apache/spark/pull/22914
[SPARK-25900][WEBUI]If the When the page number is more than the total page
size, then fall back to the first page
## What changes were proposed in this pull request?
When we enter
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22864
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22864
I will fix the build error.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22864
Hi @gengliangwang , Yes the parameter is not used from the beginning.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
Thanks a lot @srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22864
cc @dongjoon-hyun @felixcheung
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22784#discussion_r228788331
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/PCA.scala ---
@@ -49,7 +50,16 @@ class PCA @Since("1.4.0") (@Since("1.4
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22784#discussion_r228729214
--- Diff:
mllib/src/test/scala/org/apache/spark/mllib/feature/PCASuite.scala ---
@@ -54,4 +55,21 @@ class PCASuite extends SparkFunSuite
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22784#discussion_r228729215
--- Diff:
mllib/src/test/scala/org/apache/spark/mllib/feature/PCASuite.scala ---
@@ -54,4 +55,21 @@ class PCASuite extends SparkFunSuite
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22784#discussion_r228729208
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/feature/PCA.scala ---
@@ -49,7 +50,16 @@ class PCA @Since("1.4.0") (@Since("1.4
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22784#discussion_r228729201
--- Diff:
mllib/src/test/scala/org/apache/spark/mllib/feature/PCASuite.scala ---
@@ -54,4 +55,14 @@ class PCASuite extends SparkFunSuite
GitHub user shahidki31 opened a pull request:
https://github.com/apache/spark/pull/22864
[Minor][WEBUI] Remove refresh interval parameter from the headerSparkPage
method.
## What changes were proposed in this pull request?
'refreshInterval' is not used any where
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
Thank you @srowen for the review. I have addressed the comments.
> I wonder if the SVD should be used at even smaller scales? as you point
out, it's pretty hard to compute a gram
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22784#discussion_r228719122
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/RowMatrix.scala
---
@@ -384,18 +384,28 @@ class RowMatrix @Since("
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22784#discussion_r228719009
--- Diff:
mllib/src/test/scala/org/apache/spark/mllib/feature/PCASuite.scala ---
@@ -54,4 +55,14 @@ class PCASuite extends SparkFunSuite
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22784#discussion_r228718969
--- Diff:
mllib/src/test/scala/org/apache/spark/mllib/feature/PCASuite.scala ---
@@ -54,4 +55,14 @@ class PCASuite extends SparkFunSuite
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
Hi @srowen , Thanks for the comment. As per my knowledge, PCA/SVD is not
limited on row size.
1) Currently row size is not a constraint. Ultimately we need to compute
graminan matrix
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
Hi @kiszk Maximum it can go upto the following limit.
https://github.com/apache/spark/blob/23cfda1547355a823a3b2b2d374e64608c9ce175/mllib/src/main/scala/org/apache/spark/mllib/linalg
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
Hi @kiszk ,
I think, INT_MAX is 2147483647, so n ~= sqrt(2*2147483647) = 65536.
Thanks
---
-
To unsubscribe, e-mail
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
All the UTs are passing locally. Seems random error.
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user shahidki31 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22784#discussion_r226847504
--- Diff: mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala
---
@@ -424,6 +424,28 @@ object Vectors
Github user shahidki31 commented on the issue:
https://github.com/apache/spark/pull/22784
cc @srowen
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
101 - 200 of 352 matches
Mail list logo
