Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/23223#discussion_r239052799
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
---
@@ -612,11 +612,14 @@ private[yarn] class
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/23223#discussion_r239070925
--- Diff:
resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorSuite.scala
---
@@ -114,13 +116,20 @@ class
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/23223#discussion_r239059997
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala
---
@@ -612,11 +612,14 @@ private[yarn] class
Github user attilapiros closed the pull request at:
https://github.com/apache/spark/pull/23188
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/23188
I am closing this issue as I see one of the third party tool uses Netty3.
So as a transitive dependency Netty3 is really needed
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/23188
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/23188
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user attilapiros opened a pull request:
https://github.com/apache/spark/pull/23188
[SPARK-26232][Build] Remove unused dependency to Netty3
## What changes were proposed in this pull request?
Removing dependency to Netty3 as it is unused.
The reason
Github user attilapiros closed the pull request at:
https://github.com/apache/spark/pull/23115
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user attilapiros closed the pull request at:
https://github.com/apache/spark/pull/23114
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/23090
Backport PR for 2.2: https://github.com/apache/spark/pull/23115
Both 2.2 and 2.3 are retested.
---
-
To unsubscribe
GitHub user attilapiros opened a pull request:
https://github.com/apache/spark/pull/23115
[SPARK-26118][WEB UI] Introducing spark.ui.requestHeaderSize for setting
HTTP requestHeaderSize
## What changes were proposed in this pull request?
Introducing
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/23090
Backport PR for 2.3: https://github.com/apache/spark/pull/23114
---
-
To unsubscribe, e-mail: reviews-unsubscr
GitHub user attilapiros opened a pull request:
https://github.com/apache/spark/pull/23114
[SPARK-26118][BACKPORT-2.3][WEB UI] Introducing spark.ui.requestHeaderSize
for setting HTTP requestHeaderSize
## What changes were proposed in this pull request?
Introducing
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/23090
Thanks @dongjoon-hyun, of course I will create those PRs.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/23090
I would prefer to override this 8k limit when it is really necessary (and
only with the extent which justified by the production system
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/23090
@dongjoon-hyun, @squito it is my bad, this problem is present in earlier
releases as well. If I can change the issue type I modify it to Bug
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/23095#discussion_r235091764
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryStatus.scala
---
@@ -28,9 +28,10 @@ import
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/23000#discussion_r234992152
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala
---
@@ -410,6 +410,30 @@ class DateTimeUtilsSuite
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/23000#discussion_r234986530
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala
---
@@ -410,6 +410,30 @@ class DateTimeUtilsSuite
GitHub user attilapiros opened a pull request:
https://github.com/apache/spark/pull/23090
[SPARK-26118][UI] Introducing spark.ui.requestHeaderSize for setting HTTP
requestHeaderSize
## What changes were proposed in this pull request?
Introducing spark.ui.requestHeaderSize
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/23000
ping @10110346 @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
GitHub user attilapiros opened a pull request:
https://github.com/apache/spark/pull/23000
[SPARK-26002][SQL] Fix day of year calculation for Julian calendar days
## What changes were proposed in this pull request?
Fixing leap year calculations for date operators (year/month
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/22938
@HyukjinKwon Sure, the test would be for invalid JSON array:
```Scala
test("from_json - input=invalid JSON array, schema=array, output=array") {
val input = &
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/22938
@MaxGekk I have checked out your PR and played a little bit with it:
created a new unit test as a copy of "from_json - input=array, schema=array,
output=array" with an invalid JSON.
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22938#discussion_r231189880
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala
---
@@ -550,15 +550,23 @@ case class
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22938#discussion_r231156386
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala
---
@@ -550,15 +550,23 @@ case class
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22818#discussion_r231056276
--- Diff:
core/src/main/scala/org/apache/spark/internal/config/package.scala ---
@@ -488,17 +489,19 @@ package object config {
"othe
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22818#discussion_r231059198
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -287,7 +286,7 @@ class Dataset[T] private[sql](
_numRows: Int
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22880#discussion_r231047086
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRowConverter.scala
---
@@ -202,11 +204,15 @@ private
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22880#discussion_r229732302
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRowConverter.scala
---
@@ -202,11 +204,15 @@ private
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21063#discussion_r228223151
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala
---
@@ -111,7 +112,12 @@ trait ProgressReporter
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22279#discussion_r225310671
--- Diff: core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala
---
@@ -103,6 +103,14 @@ private[spark] class MetricsSystem private
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22279#discussion_r225138301
--- Diff: core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala
---
@@ -103,6 +103,14 @@ private[spark] class MetricsSystem private
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22279#discussion_r225086020
--- Diff: core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala
---
@@ -103,6 +103,14 @@ private[spark] class MetricsSystem private
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22624#discussion_r223796403
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosHadoopDelegationTokenManager.scala
---
@@ -34,15 +34,9
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22624#discussion_r223605049
--- Diff:
core/src/main/scala/org/apache/spark/deploy/security/AbstractCredentialRenewer.scala
---
@@ -0,0 +1,224 @@
+/*
+ * Licensed
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22624#discussion_r223590938
--- Diff:
core/src/main/scala/org/apache/spark/deploy/security/AbstractCredentialRenewer.scala
---
@@ -0,0 +1,224 @@
+/*
+ * Licensed
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22624#discussion_r223616283
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala
---
@@ -60,8 +60,13
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22624#discussion_r223622626
--- Diff:
core/src/main/scala/org/apache/spark/deploy/security/AbstractCredentialRenewer.scala
---
@@ -0,0 +1,224 @@
+/*
+ * Licensed
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22624#discussion_r223624353
--- Diff:
core/src/main/scala/org/apache/spark/deploy/security/AbstractCredentialRenewer.scala
---
@@ -0,0 +1,224 @@
+/*
+ * Licensed
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22624#discussion_r223398529
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosHadoopDelegationTokenManager.scala
---
@@ -14,147
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22624#discussion_r223607997
--- Diff:
core/src/main/scala/org/apache/spark/deploy/security/AbstractCredentialRenewer.scala
---
@@ -0,0 +1,224 @@
+/*
+ * Licensed
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22598#discussion_r221822533
--- Diff:
core/src/test/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManagerSuite.scala
---
@@ -111,6 +113,17 @@ class
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22598#discussion_r221821710
--- Diff:
core/src/main/scala/org/apache/spark/deploy/security/KafkaDelegationTokenProvider.scala
---
@@ -0,0 +1,66 @@
+/*
+ * Licensed
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22598#discussion_r221589931
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceProvider.scala
---
@@ -560,25 +553,56 @@ private[kafka010
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22598#discussion_r221587545
--- Diff:
core/src/test/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManagerSuite.scala
---
@@ -46,6 +47,7 @@ class
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22598#discussion_r221586357
--- Diff:
core/src/main/scala/org/apache/spark/deploy/security/KafkaDelegationTokenProvider.scala
---
@@ -0,0 +1,66 @@
+/*
+ * Licensed
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22598#discussion_r221588791
--- Diff:
core/src/test/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManagerSuite.scala
---
@@ -111,6 +113,17 @@ class
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22598#discussion_r221591857
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/TokenUtil.scala
---
@@ -0,0 +1,116 @@
+/*
+ * Licensed
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/20958#discussion_r214063180
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/sources/TextSocketStreamSuite.scala
---
@@ -256,6 +257,58 @@ class
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22138#discussion_r213869998
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/InternalKafkaConsumerPool.scala
---
@@ -0,0 +1,260
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22138#discussion_r213621710
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/InternalKafkaConsumerPool.scala
---
@@ -0,0 +1,260
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22138#discussion_r213639419
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/InternalKafkaConsumerPool.scala
---
@@ -0,0 +1,260
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22138#discussion_r213615553
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/InternalKafkaConsumerPool.scala
---
@@ -0,0 +1,260
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/22138#discussion_r213615086
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/InternalKafkaConsumerPool.scala
---
@@ -0,0 +1,260
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21971#discussion_r207898987
--- Diff: core/src/main/scala/org/apache/spark/rdd/AsyncRDDActions.scala ---
@@ -61,6 +62,36 @@ class AsyncRDDActions[T: ClassTag](self: RDD[T]) extends
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21222#discussion_r207857933
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala ---
@@ -513,6 +515,125 @@ class StreamSuite extends StreamTest
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21635#discussion_r203840572
--- Diff: docs/monitoring.md ---
@@ -435,6 +435,7 @@ set of sinks to which metrics are reported. The
following instances are currentl
* `executor
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21635#discussion_r203594956
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMasterSource.scala
---
@@ -0,0 +1,49
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21635#discussion_r202056881
--- Diff: docs/monitoring.md ---
@@ -435,6 +435,7 @@ set of sinks to which metrics are reported. The
following instances are currentl
* `executor
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21635#discussion_r202035811
--- Diff: docs/monitoring.md ---
@@ -435,6 +435,7 @@ set of sinks to which metrics are reported. The
following instances are currentl
* `executor
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21635#discussion_r201979140
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
---
@@ -309,6 +312,11 @@ private[spark] class
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21635#discussion_r201331447
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnClusterSchedulerSource.scala
---
@@ -0,0 +1,49
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21635#discussion_r199527182
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnClusterSchedulerSource.scala
---
@@ -0,0 +1,53
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21635#discussion_r198641094
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnClusterSchedulerSource.scala
---
@@ -0,0 +1,49
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21635#discussion_r198630307
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
---
@@ -309,6 +312,9 @@ private[spark] class
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21635#discussion_r198629311
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnClusterSchedulerSource.scala
---
@@ -0,0 +1,49
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21635#discussion_r198538655
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
---
@@ -309,6 +312,9 @@ private[spark] class
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21623#discussion_r198269733
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -270,6 +277,29 @@ private[parquet
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21623#discussion_r198245076
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -270,6 +277,29 @@ private[parquet
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21623#discussion_r198241792
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
---
@@ -270,6 +277,29 @@ private[parquet
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21644#discussion_r198185350
--- Diff:
core/src/main/scala/org/apache/spark/status/api/v1/ApiRootResource.scala ---
@@ -148,38 +148,36 @@ private[v1] trait BaseAppResource extends
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21635#discussion_r198116931
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnClusterSchedulerSource.scala
---
@@ -0,0 +1,35
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21635#discussion_r198116371
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
---
@@ -67,6 +68,14 @@ private[spark] class
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21635#discussion_r198115953
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
---
@@ -771,6 +784,7 @@ object ApplicationMaster
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21279#discussion_r197919386
--- Diff: bin/docker-image-tool.sh ---
@@ -44,15 +44,37 @@ function image_ref {
function build {
local BUILD_ARGS
local IMG_PATH
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21279#discussion_r197916290
--- Diff: bin/docker-image-tool.sh ---
@@ -44,15 +44,37 @@ function image_ref {
function build {
local BUILD_ARGS
local IMG_PATH
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/21623
Just a question regarding `sources.StringStartsWith("_1", null))`: if you
have a nullable string column and some of the values are null will this
operator (parameterized with nul
GitHub user attilapiros opened a pull request:
https://github.com/apache/spark/pull/21635
[SPARK-24594][YARN] Introducing metrics for YARN executor allocation
problems
## What changes were proposed in this pull request?
In this PR metrics are introduced for YARN allocation
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/21601
I read your changes and in the test I was searching for an check/assert but
found none. I understand it is about checking no exception is thrown during the
directory content reading but I still
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21068#discussion_r197449373
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTracker.scala
---
@@ -0,0 +1,187
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21068#discussion_r197444068
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTracker.scala
---
@@ -0,0 +1,187
Github user attilapiros closed the pull request at:
https://github.com/apache/spark/pull/21592
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
GitHub user attilapiros opened a pull request:
https://github.com/apache/spark/pull/21592
[SPARK-24578][Core] Avoid timeout at reading remote cache block
## What changes were proposed in this pull request?
In MessageWithHeader the copyByteBuf() method is called several
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/21068
Here is the new task for the metrics:
https://issues.apache.org/jira/browse/SPARK-24594.
---
-
To unsubscribe, e-mail
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21068#discussion_r195922985
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTracker.scala
---
@@ -0,0 +1,187
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21068#discussion_r195750196
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTracker.scala
---
@@ -0,0 +1,187
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/21068
Retested manually on a cluster with the result the PR's description is
updated.
---
-
To unsubscribe, e-mail: reviews
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21068#discussion_r195476043
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTracker.scala
---
@@ -0,0 +1,189
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/21068
@tgravescs what about removing YARN_BLACKLIST_MAX_NODE_BLACKLIST_RATIO
config and when the set of backlisted nodes reaches numClusterNodes I stop
synchronising the backlisted nodes toward YARN
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21068#discussion_r191762829
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config.scala
---
@@ -328,4 +328,19 @@ package object config
GitHub user attilapiros opened a pull request:
https://github.com/apache/spark/pull/21280
[SPARK-19181][Core] Fixing flaky "SparkListenerSuite.local metrics"
## What changes were proposed in this pull request?
Sometimes "SparkListenerSuite.local metrics"
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21068#discussion_r185791498
--- Diff:
resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTrackerSuite.scala
---
@@ -0,0 +1,144
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/21068
Jenkins, retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user attilapiros commented on the issue:
https://github.com/apache/spark/pull/21068
I assume it is just a flaky R test.
Jenkins retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21068#discussion_r185447393
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -170,8 +170,7 @@ class
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21068#discussion_r185441810
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTracker.scala
---
@@ -0,0 +1,150
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21068#discussion_r185440657
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocatorBlacklistTracker.scala
---
@@ -0,0 +1,150
Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/21068#discussion_r185435494
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/FailureTracker.scala
---
@@ -0,0 +1,85 @@
+/*
+ * Licensed
1 - 100 of 246 matches
Mail list logo
