[GitHub] spark pull request #20046: Adding unit tests for missing aggregate functions

2017-12-21 Thread attilapiros
GitHub user attilapiros opened a pull request: https://github.com/apache/spark/pull/20046 Adding unit tests for missing aggregate functions ## What changes were proposed in this pull request? Improving the test coverage of window functions focusing on missing test for

[GitHub] spark pull request #20046: [SPARK-22362][SQL] Add unit test for Window Aggre...

2017-12-22 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20046#discussion_r158456767 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameWindowFunctionsSuite.scala --- @@ -154,6 +154,217 @@ class

[GitHub] spark pull request #20045: [Spark-22360][SQL] Add unit tests for Window Spec...

2017-12-22 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20045#discussion_r158300159 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameWindowFunctionsSuite.scala --- @@ -32,6 +32,209 @@ import

[GitHub] spark pull request #20069: [SPARK-22895] [SQL] Push down the deterministic p...

2017-12-27 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20069#discussion_r158885112 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala --- @@ -47,6 +48,15 @@ trait Predicate extends

[GitHub] spark issue #20046: [SPARK-22362][SQL] Add unit test for Window Aggregate Fu...

2018-01-03 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20046 cc @gatorsmile @hvanhovell @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20203: [SPARK-22577] [core] executor page blacklist stat...

2018-01-09 Thread attilapiros
GitHub user attilapiros opened a pull request: https://github.com/apache/spark/pull/20203 [SPARK-22577] [core] executor page blacklist status should update with TaskSet level blacklisting ## What changes were proposed in this pull request? In this PR stage blacklisting is

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-10 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20203 cc @tgravescs --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #20235: [Spark-22887][ML][TESTS][WIP] ML test for Structu...

2018-01-11 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20235#discussion_r160979218 --- Diff: mllib/src/test/scala/org/apache/spark/ml/fpm/FPGrowthSuite.scala --- @@ -34,86 +35,122 @@ class FPGrowthSuite extends SparkFunSuite with

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-11 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20203 @tgravescs: You are welcome. Running Spark as (with lower application.maxFailedTasksPerExecutor than stage.maxFailedTasksPerExecutor): ``` $ bin/spark-shell --master "

[GitHub] spark pull request #20203: [SPARK-22577] [core] executor page blacklist stat...

2018-01-11 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20203#discussion_r161070422 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -211,6 +211,11 @@ private[spark] class AppStatusListener

[GitHub] spark pull request #20249: [SPARK-23057][SPARK-19235][SQL] SET LOCATION shou...

2018-01-15 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20249#discussion_r161480210 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -1869,6 +1869,65 @@ abstract class DDLSuite extends

[GitHub] spark pull request #20249: [SPARK-23057][SPARK-19235][SQL] SET LOCATION shou...

2018-01-15 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20249#discussion_r161488605 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -1869,6 +1869,65 @@ abstract class DDLSuite extends

[GitHub] spark pull request #20249: [SPARK-23057][SPARK-19235][SQL] SET LOCATION shou...

2018-01-15 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20249#discussion_r161488179 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -1869,6 +1869,65 @@ abstract class DDLSuite extends

[GitHub] spark pull request #20249: [SPARK-23057][SPARK-19235][SQL] SET LOCATION shou...

2018-01-15 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20249#discussion_r161477649 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -800,6 +802,15 @@ case class AlterTableSetLocationCommand

[GitHub] spark pull request #20249: [SPARK-23057][SPARK-19235][SQL] SET LOCATION shou...

2018-01-15 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20249#discussion_r161488850 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -1869,6 +1869,65 @@ abstract class DDLSuite extends

[GitHub] spark pull request #20249: [SPARK-23057][SPARK-19235][SQL] SET LOCATION shou...

2018-01-15 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20249#discussion_r161487862 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -1869,6 +1869,65 @@ abstract class DDLSuite extends

[GitHub] spark pull request #20249: [SPARK-23057][SPARK-19235][SQL] SET LOCATION shou...

2018-01-15 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20249#discussion_r161491069 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveDDLSuite.scala --- @@ -751,6 +751,25 @@ class HiveDDLSuite

[GitHub] spark pull request #19224: [SPARK-20990][SQL] Read all JSON documents in fil...

2018-01-16 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/19224#discussion_r161567546 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala --- @@ -361,3 +361,78 @@ class JacksonParser

[GitHub] spark pull request #19224: [SPARK-20990][SQL] Read all JSON documents in fil...

2018-01-16 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/19224#discussion_r161745397 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1853,7 +1853,7 @@ class JsonSuite extends

[GitHub] spark pull request #19224: [SPARK-20990][SQL] Read all JSON documents in fil...

2018-01-16 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/19224#discussion_r161744616 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonUtils.scala --- @@ -39,7 +38,7 @@ object JsonUtils

[GitHub] spark pull request #19224: [SPARK-20990][SQL] Read all JSON documents in fil...

2018-01-16 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/19224#discussion_r161742050 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonDataSource.scala --- @@ -178,21 +166,15 @@ object

[GitHub] spark pull request #19224: [SPARK-20990][SQL] Read all JSON documents in fil...

2018-01-16 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/19224#discussion_r161733393 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala --- @@ -361,3 +361,78 @@ class JacksonParser

[GitHub] spark pull request #19224: [SPARK-20990][SQL] Read all JSON documents in fil...

2018-01-16 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/19224#discussion_r161566402 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonDataSource.scala --- @@ -143,28 +139,20 @@ object

[GitHub] spark pull request #19224: [SPARK-20990][SQL] Read all JSON documents in fil...

2018-01-16 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/19224#discussion_r161704328 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonDataSource.scala --- @@ -143,28 +139,20 @@ object

[GitHub] spark pull request #21131: [SPARK-23433][CORE] Late zombie task completions ...

2018-04-24 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21131#discussion_r183774355 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -917,4 +917,111 @@ class TaskSchedulerImplSuite extends

[GitHub] spark pull request #21131: [SPARK-23433][CORE] Late zombie task completions ...

2018-04-24 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21131#discussion_r183775077 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -917,4 +917,111 @@ class TaskSchedulerImplSuite extends

[GitHub] spark pull request #21131: [SPARK-23433][CORE] Late zombie task completions ...

2018-04-24 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21131#discussion_r183783006 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSchedulerImplSuite.scala --- @@ -917,4 +917,111 @@ class TaskSchedulerImplSuite extends

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-05-02 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r185432837 --- Diff: core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala --- @@ -73,6 +73,7 @@ class HeartbeatReceiverSuite sc = spy(new

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-05-02 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r185434976 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/FailureTracker.scala --- @@ -0,0 +1,85 @@ +/* + * Licensed to the

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-05-02 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r185435494 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/FailureTracker.scala --- @@ -0,0 +1,85 @@ +/* + * Licensed to the

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-05-02 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r185440657 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTracker.scala --- @@ -0,0 +1,150

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-05-02 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r185441810 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTracker.scala --- @@ -0,0 +1,150

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-05-02 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r185447393 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -170,8 +170,7 @@ class

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-05-02 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/21068 I assume it is just a flaky R test. Jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-05-02 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/21068 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-05-03 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r185791498 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTrackerSuite.scala --- @@ -0,0 +1,144

[GitHub] spark pull request #21280: [SPARK-19181][Core] Fixing flaky "SparkListenerSu...

2018-05-09 Thread attilapiros
GitHub user attilapiros opened a pull request: https://github.com/apache/spark/pull/21280 [SPARK-19181][Core] Fixing flaky "SparkListenerSuite.local metrics" ## What changes were proposed in this pull request? Sometimes "SparkListenerSuite.local metrics"

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-05-30 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r191762829 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config.scala --- @@ -328,4 +328,19 @@ package object config

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-05-30 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/21068 @tgravescs what about removing YARN_BLACKLIST_MAX_NODE_BLACKLIST_RATIO config and when the set of backlisted nodes reaches numClusterNodes I stop synchronising the backlisted nodes toward YARN

[GitHub] spark pull request #20852: [SPARK-23728][BRANCH-2.3] Fix ML tests with expec...

2018-03-20 Thread attilapiros
Github user attilapiros closed the pull request at: https://github.com/apache/spark/pull/20852 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20852: [SPARK-23728][BRANCH-2.3] Fix ML tests with expected exc...

2018-03-20 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20852 Closing manually as it was merged to branch-2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20925: [SPARK-22941][core] Do not exit JVM when submit f...

2018-04-06 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20925#discussion_r179834098 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java --- @@ -400,6 +419,11 @@ private boolean isThriftServer

[GitHub] spark pull request #20925: [SPARK-22941][core] Do not exit JVM when submit f...

2018-04-06 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20925#discussion_r179825806 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java --- @@ -88,7 +88,8 @@ SparkLauncher.NO_RESOURCE

[GitHub] spark pull request #20925: [SPARK-22941][core] Do not exit JVM when submit f...

2018-04-06 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20925#discussion_r179814761 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -289,27 +288,26 @@ private[deploy] class SparkSubmitArguments

[GitHub] spark pull request #20925: [SPARK-22941][core] Do not exit JVM when submit f...

2018-04-06 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20925#discussion_r179816905 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -499,20 +497,18 @@ private[deploy] class SparkSubmitArguments

[GitHub] spark pull request #20925: [SPARK-22941][core] Do not exit JVM when submit f...

2018-04-06 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20925#discussion_r179832847 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkSubmitCommandBuilder.java --- @@ -99,17 +100,27 @@ */ private boolean

[GitHub] spark issue #20925: [SPARK-22941][core] Do not exit JVM when submit fails wi...

2018-04-09 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20925 I have finished my review and have not found any additional issue. LGTM --- - To unsubscribe, e-mail: reviews

[GitHub] spark issue #20046: [SPARK-22362][SQL] Add unit test for Window Aggregate Fu...

2018-04-09 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20046 ping @hvanhovell @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-04-13 Thread attilapiros
GitHub user attilapiros opened a pull request: https://github.com/apache/spark/pull/21068 [SPARK-16630][YARN] Blacklist a node if executors won't launch on it ## What changes were proposed in this pull request? This change extends YARN resource allocation handling

[GitHub] spark issue #20046: [SPARK-22362][SQL] Add unit test for Window Aggregate Fu...

2018-04-13 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20046 gentle reminder @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-04-17 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/21068 Yes we can create an abstract class from `YarnAllocatorBlacklistTracker` (like `AbstractAllocatorBlacklistTracker`) where the method `synchronizeBlacklistedNodes` can have different

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-04-18 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r182404677 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTracker.scala --- @@ -0,0 +1,155

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-04-18 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r182405470 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTracker.scala --- @@ -0,0 +1,155

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-04-18 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r182407936 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTracker.scala --- @@ -0,0 +1,155

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-04-18 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r182411922 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/FailureWithinTimeIntervalTracker.scala --- @@ -0,0 +1,80

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-04-18 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r182418247 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/FailureWithinTimeIntervalTracker.scala --- @@ -0,0 +1,80

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-04-18 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r182419122 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/FailureWithinTimeIntervalTracker.scala --- @@ -0,0 +1,80

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-04-18 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r182427448 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config.scala --- @@ -328,4 +328,26 @@ package object config

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-04-18 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r182428059 --- Diff: core/src/main/scala/org/apache/spark/scheduler/BlacklistTracker.scala --- @@ -216,6 +216,10 @@ private[scheduler] class BlacklistTracker

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-04-18 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r182447935 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config.scala --- @@ -328,4 +328,26 @@ package object config

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-04-18 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r182448168 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config.scala --- @@ -328,4 +328,26 @@ package object config

[GitHub] spark issue #20046: [SPARK-22362][SQL] Add unit test for Window Aggregate Fu...

2018-04-18 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20046 @hvanhovell I would like to ask you to take another quick glance to these change --- - To unsubscribe, e-mail: reviews

[GitHub] spark pull request #19224: [SPARK-20990][SQL] Read all JSON documents in fil...

2018-01-16 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/19224#discussion_r161807543 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala --- @@ -361,3 +361,78 @@ class JacksonParser

[GitHub] spark pull request #19224: [SPARK-20990][SQL] Read all JSON documents in fil...

2018-01-16 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/19224#discussion_r161854693 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala --- @@ -361,3 +361,78 @@ class JacksonParser

[GitHub] spark pull request #20203: [SPARK-22577] [core] executor page blacklist stat...

2018-01-17 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20203#discussion_r162041751 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetBlacklist.scala --- @@ -128,13 +130,17 @@ private[scheduler] class TaskSetBlacklist(val

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-17 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20203 Of course I can update the executors page too. @tgravescs I think via handling the onStageCompleted I can do what you would like to see here

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-17 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20203 @tgravescs @squito: But what about doing it in a separate PR as this is already quite big? So node backlisting would be still within the scope but executor page changes would be done for a

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-19 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20203 The node blacklisting is tested by unit tests: - HistoryServerSuite - TaskSetBlacklistSuite - AppStatusListenerSuite And manually with a 2 node cluster: https

[GitHub] spark pull request #20203: [SPARK-22577] [core] executor page blacklist stat...

2018-01-19 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20203#discussion_r162716271 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetBlacklistSuite.scala --- @@ -59,31 +60,55 @@ class TaskSetBlacklistSuite extends

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-20 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20203 One more reason to run tests in sbt / maven. In intelliJ somehow the complete suite was successful. But the current failure seems to me unrelated, as org.apache.spark.deploy.history has 0

[GitHub] spark pull request #20046: [SPARK-22362][SQL] Add unit test for Window Aggre...

2018-01-22 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20046#discussion_r162894686 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameWindowFunctionsSuite.scala --- @@ -86,6 +93,429 @@ class DataFrameWindowFunctionsSuite

[GitHub] spark issue #20046: [SPARK-22362][SQL] Add unit test for Window Aggregate Fu...

2018-01-22 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20046 I have already extended sql/core/src/test/resources/sql-tests/inputs/window.sql with the missing window aggregate functions but if you would like I can move it to a different PR too

[GitHub] spark issue #20046: [SPARK-22362][SQL] Add unit test for Window Aggregate Fu...

2018-01-22 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20046 PySpark failure must be unrelated as only unit tests are added. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-22 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20203 Test failure seems to be unrelated as it it is in org.apache.spark.sql.hive.client.HiveClientSuites. @squito, @tgravescs could you please restart testing

[GitHub] spark issue #20046: [SPARK-22362][SQL] Add unit test for Window Aggregate Fu...

2018-01-26 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20046 cc @gatorsmile @hvanhovell @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20408: [SPARK-23189][Core][UI] Reflect stage level black...

2018-01-26 Thread attilapiros
GitHub user attilapiros opened a pull request: https://github.com/apache/spark/pull/20408 [SPARK-23189][Core][UI] Reflect stage level blacklisting on executor tab ## What changes were proposed in this pull request? The purpose of this PR to reflect the stage level

[GitHub] spark issue #20408: [SPARK-23189][Core][Web UI] Reflect stage level blacklis...

2018-01-26 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20408 cc @squito @tgravescs --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #20408: [SPARK-23189][Core][Web UI] Reflect stage level b...

2018-01-27 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20408#discussion_r164292079 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -594,12 +606,24 @@ private[spark] class AppStatusListener

[GitHub] spark issue #20408: [SPARK-23189][Core][Web UI] Reflect stage level blacklis...

2018-01-29 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20408 @ajbozarth What about `Active except for Stages: [...]`? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #20408: [SPARK-23189][Core][Web UI] Reflect stage level b...

2018-01-29 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20408#discussion_r164368745 --- Diff: core/src/main/resources/org/apache/spark/ui/static/executorspage.js --- @@ -25,9 +25,13 @@ function getThreadDumpEnabled() { return

[GitHub] spark pull request #20408: [SPARK-23189][Core][Web UI] Reflect stage level b...

2018-01-29 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20408#discussion_r164594314 --- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala --- @@ -254,6 +255,7 @@ private class LiveExecutor(val executorId: String

[GitHub] spark issue #20408: [SPARK-23189][Core][Web UI] Reflect stage level blacklis...

2018-01-29 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20408 @ajbozarth Ok. For tomorrow I can update both the label and the screenshots. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #20408: [SPARK-23189][Core][Web UI] Reflect stage level b...

2018-01-30 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20408#discussion_r164776427 --- Diff: core/src/main/resources/org/apache/spark/ui/static/executorspage.js --- @@ -416,8 +422,7 @@ $(document).ready(function

[GitHub] spark pull request #20408: [SPARK-23189][Core][Web UI] Reflect stage level b...

2018-01-30 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20408#discussion_r164856639 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -594,12 +606,24 @@ private[spark] class AppStatusListener

[GitHub] spark pull request #20408: [SPARK-23189][Core][Web UI] Reflect stage level b...

2018-01-30 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20408#discussion_r164893438 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -228,9 +230,12 @@ private[spark] class AppStatusListener

[GitHub] spark pull request #20408: [SPARK-23189][Core][Web UI] Reflect stage level b...

2018-01-30 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20408#discussion_r164893958 --- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala --- @@ -423,7 +425,8 @@ private class LiveStage extends LiveEntity

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-01 Thread attilapiros
GitHub user attilapiros opened a pull request: https://github.com/apache/spark/pull/20474 [SPARK-23235][Core] Add executor Threaddump to api ## What changes were proposed in this pull request? Extending api with the executor thread dump data. For this new REST URL

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-01 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20474 cc @squito @ajbozarth --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-01 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20474#discussion_r165387234 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2168,7 +2168,17 @@ private[spark] object Utils extends Logging { // We

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-01 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20474#discussion_r165407125 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala --- @@ -51,6 +52,21 @@ private[v1] class

[GitHub] spark issue #20474: [SPARK-23235][Core] Add executor Threaddump to api

2018-02-01 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20474 Unrelated failure: "org.apache.spark.sql.execution.datasources.orc.OrcQuerySuite" --- - To unsubscribe, e-mai

[GitHub] spark issue #20408: [SPARK-23189][Core][Web UI] Reflect stage level blacklis...

2018-02-01 Thread attilapiros
Github user attilapiros commented on the issue: https://github.com/apache/spark/pull/20408 cc @tgravescs --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-01 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20474#discussion_r165509830 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala --- @@ -51,6 +52,21 @@ private[v1] class

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-01 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20474#discussion_r165510071 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala --- @@ -51,6 +52,21 @@ private[v1] class

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-02 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20474#discussion_r165604504 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala --- @@ -51,6 +52,46 @@ private[v1] class

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-02 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20474#discussion_r165717844 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/ApiRootResource.scala --- @@ -83,6 +83,8 @@ private[spark] trait UIRoot { def

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-02 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20474#discussion_r165719025 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala --- @@ -51,6 +52,43 @@ private[v1] class

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-02 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20474#discussion_r165720298 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala --- @@ -51,6 +52,43 @@ private[v1] class

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-02 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20474#discussion_r165739588 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala --- @@ -51,6 +52,43 @@ private[v1] class

[GitHub] spark pull request #20474: [SPARK-23235][Core] Add executor Threaddump to ap...

2018-02-02 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20474#discussion_r165741532 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala --- @@ -51,6 +52,43 @@ private[v1] class

  1   2   3   >