Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21955#discussion_r207394183
--- Diff:
external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaTestUtils.scala
---
@@ -109,7 +109,7 @@ private[kafka010] class
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21222#discussion_r206342173
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/debug/package.scala ---
@@ -116,6 +177,30 @@ package object debug
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21222#discussion_r206342211
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/debug/package.scala ---
@@ -88,23 +100,72 @@ package object debug
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21222#discussion_r206342230
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/debug/package.scala ---
@@ -88,23 +100,72 @@ package object debug
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21222#discussion_r207392732
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/debug/package.scala ---
@@ -116,6 +177,30 @@ package object debug
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21955
There are some configs need to tune to make the tests faster. See:
https://github.com/apache/spark/pull/21488/files#diff-1aecc5a638048c27f97cbd9579cb53b5R311
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21854
LGTM pending tests
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21913
Yeah, let's also fix other instances.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21913#discussion_r206704908
--- Diff: core/src/main/scala/org/apache/spark/util/ThreadUtils.scala ---
@@ -254,4 +254,49 @@ private[spark] object ThreadUtils
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21913#discussion_r206704241
--- Diff: core/src/main/scala/org/apache/spark/util/ThreadUtils.scala ---
@@ -254,4 +254,49 @@ private[spark] object ThreadUtils
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21913#discussion_r206704548
--- Diff: core/src/main/scala/org/apache/spark/util/ThreadUtils.scala ---
@@ -254,4 +254,49 @@ private[spark] object ThreadUtils
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21913#discussion_r206704050
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/util/FileBasedWriteAheadLog.scala
---
@@ -313,9 +313,7 @@ private[streaming] object
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21913
> What problem does this solve?
@srowen `readParquetFootersInParallel` is called in executors. When a task
is cancelled, it will still keeping run. If it reads lots of files, it will
t
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
Thanks! Merging to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
@srowen May I read your comment as "no objections"? The current PR looks
good to me. If you don't have objections, I will go ahead
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
@ijuma I see. I was looking at 0.10 jar. Thanks for correcting me.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
@srowen just to be clear, `AdminClient` is not in `kafka-clients` jar. The
user has to add `kafka` jar as a dependency to use `AdminClient`.
In addition, even if this upgrade has some
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
> I might have missed this in the shuffle here -- is this fully compatible
with 0.10.x brokers too?
Yep. @ijuma could you conf
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21854#discussion_r206595006
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
@@ -2124,6 +2124,9 @@ class Analyzer(
override
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21854#discussion_r206594867
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -1446,3 +1448,17 @@ object
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21634#discussion_r206346537
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala
---
@@ -504,4 +504,38 @@ class SQLMetricsSuite extends
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21634#discussion_r206346272
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala
---
@@ -504,4 +504,38 @@ class SQLMetricsSuite extends
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21222#discussion_r205918043
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala ---
@@ -829,6 +955,18 @@ class StreamSuite extends StreamTest
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21222#discussion_r205917174
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/debug/package.scala ---
@@ -88,23 +100,70 @@ package object debug
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21222#discussion_r205918179
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamSuite.scala ---
@@ -513,6 +514,131 @@ class StreamSuite extends StreamTest
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21222#discussion_r205917365
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/debug/package.scala ---
@@ -88,23 +100,70 @@ package object debug
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21222#discussion_r205916804
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/debug/package.scala ---
@@ -116,6 +175,30 @@ package object debug
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
@tedyu fyi, here is the last fix for 2.0.0:
https://github.com/zsxwing/spark/commit/7ccc520b540106ca4c2dac2e42834984332d6804
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
@tedyu I just realized topic deletion might be used by some users. This
Kafka version will introduce a regression for them. Since 2.0 fixes this issue
and is going out soon, how about just wait
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
@tedyu I forgot one place:
https://github.com/apache/spark/blob/master/external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaContinuousSourceSuite.scala#L32
Could you also
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
@tedyu could you update the PR title and description to reflect the latest
changes, such as `[SPARK-18057][SS] Update Kafka client version from 0.10.0.1
to 1.1.0`? Otherwise, LGTM
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
Okey. In Kafka 1.1.0, deleting a topic when a Kafka client is running may
make the client hang at this line forever:
https://github.com/apache/kafka/blob/1.1.0/clients/src/main/java/org/apache
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21746
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21831#discussion_r204177925
--- Diff: resource-managers/kubernetes/integration-tests/pom.xml ---
@@ -25,7 +25,7 @@
spark-kubernetes-integration-tests_2.11
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21831#discussion_r204165909
--- Diff: resource-managers/kubernetes/integration-tests/pom.xml ---
@@ -25,7 +25,7 @@
spark-kubernetes-integration-tests_2.11
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21831
cc @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/21831
[SPARK-24880][BUILD]Fix the group id for spark-kubernetes-integration-tests
## What changes were proposed in this pull request?
The correct group id should be `org.apache.spark
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21739
LGTM again
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21488#discussion_r203561847
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReader.scala
---
@@ -115,7 +116,7 @@ private[kafka010] class
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
@tedyu my branch https://github.com/zsxwing/spark/commits/review-pr21488
should fix most of unit tests. Could you check and add to your PRs? Several
tests are still flaky and ignored. If you have
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
@tedyu this patch should fix most of unit tests:
https://github.com/zsxwing/spark/commit/c2834e2aeaa979f3c4a3f550404856c2de573463
I'm still thinking about how to fix the stress test
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
I see. It should be a test dependency. You don't need to add it into the
compile scope.
---
-
To unsubscribe, e-mail: reviews
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21488#discussion_r203109788
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReader.scala
---
@@ -115,7 +116,7 @@ private[kafka010] class
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
@tedyu We don't need to block on the release. I would expect the Kafka
client is backward complicity. In addition, most of Spark users will not
upgrade their Kafka at once after the new version
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21488#discussion_r203104176
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReader.scala
---
@@ -115,7 +116,7 @@ private[kafka010] class
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21488#discussion_r203103522
--- Diff: external/kafka-0-10-sql/pom.xml ---
@@ -74,6 +74,11 @@
${kafka.version}
test
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21488
@tedyu could you please just bump to 1.1.0, the current official latest
release from Apache Kafka?
---
-
To unsubscribe, e-mail
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21739
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21701
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21701#discussion_r201071243
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/WatermarkTracker.scala
---
@@ -20,15 +20,66 @@ package
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21701#discussion_r200771488
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/OffsetSeq.scala
---
@@ -115,8 +130,20 @@ object OffsetSeqMetadata extends
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21701#discussion_r200761267
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -875,6 +875,16 @@ object SQLConf {
.stringConf
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21575
LGTM. Thanks! Merging to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21575#discussion_r198579275
--- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala ---
@@ -74,17 +75,17 @@ private[spark] class HeartbeatReceiver(sc:
SparkContext
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21575#discussion_r198579348
--- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala ---
@@ -21,6 +21,7 @@ import java.util.concurrent.{ScheduledFuture, TimeUnit
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21575#discussion_r198576712
--- Diff: core/src/test/scala/org/apache/spark/SparkConfSuite.scala ---
@@ -371,6 +371,23 @@ class SparkConfSuite extends SparkFunSuite
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21575#discussion_r198578198
--- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala ---
@@ -74,17 +75,17 @@ private[spark] class HeartbeatReceiver(sc:
SparkContext
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21593
Thanks! Merging to master and 2.3.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21593#discussion_r196947295
--- Diff:
common/network-common/src/main/java/org/apache/spark/network/protocol/MessageWithHeader.java
---
@@ -137,30 +137,15 @@ protected void deallocate
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21575#discussion_r196924766
--- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala ---
@@ -75,16 +76,18 @@ private[spark] class HeartbeatReceiver(sc:
SparkContext
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21593#discussion_r196918296
--- Diff:
common/network-common/src/main/java/org/apache/spark/network/protocol/MessageWithHeader.java
---
@@ -137,30 +137,15 @@ protected void deallocate
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21571
LGTM. Merging to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21571#discussion_r195880511
--- Diff: python/pyspark/sql/utils.py ---
@@ -62,6 +62,7 @@ def deco(*a, **kw):
try:
return f(*a, **kw)
except
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21571#discussion_r195878970
--- Diff: python/pyspark/sql/streaming.py ---
@@ -1016,6 +1018,35 @@ def func_with_open_process_close(partition_id,
iterator):
self
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21571#discussion_r195875712
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala
---
@@ -322,6 +338,45 @@ final class DataStreamWriter[T] private
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21477
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21428
Thanks! Merging to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21428
LGTM pending tests
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21428#discussion_r194958139
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/RPCContinuousShuffleWriter.scala
---
@@ -0,0 +1,54
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21428#discussion_r194957906
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/RPCContinuousShuffleReader.scala
---
@@ -68,7 +66,7
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21428#discussion_r194914332
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/shuffle/ContinuousShuffleSuite.scala
---
@@ -288,4 +264,153 @@ class
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21428#discussion_r194911574
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/RPCContinuousShuffleWriter.scala
---
@@ -0,0 +1,54
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21428#discussion_r194914390
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/shuffle/ContinuousShuffleSuite.scala
---
@@ -288,4 +264,153 @@ class
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21428#discussion_r194907764
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/RPCContinuousShuffleWriter.scala
---
@@ -0,0 +1,54
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21428#discussion_r194916037
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/RPCContinuousShuffleReader.scala
---
@@ -79,10 +77,10
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21428#discussion_r194905420
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/RPCContinuousShuffleReader.scala
---
@@ -68,7 +66,7
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21428#discussion_r194912456
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/shuffle/ContinuousShuffleReadRDD.scala
---
@@ -34,8 +34,10 @@ case
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21477#discussion_r193839997
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/ForeachWriter.scala
---
@@ -71,23 +110,17 @@ abstract class ForeachWriter[T] extends Serializable
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21477#discussion_r193862649
--- Diff: python/pyspark/sql/tests.py ---
@@ -1884,7 +1885,164 @@ def test_query_manager_await_termination(self):
finally
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21477#discussion_r192483275
--- Diff: python/pyspark/sql/streaming.py ---
@@ -843,6 +844,169 @@ def trigger(self, processingTime=None, once=None,
continuous=None):
self
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21194
This is just a feature to warm up JVMs. Why do we need a smooth increasing
for this?
---
-
To unsubscribe, e-mail: reviews
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21491
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21400
Thanks! Merging to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21400
LGTM pending tests
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21400
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21454#discussion_r191944805
--- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala ---
@@ -448,6 +473,22 @@ class SparkConf(loadDefaults: Boolean) extends
Cloneable
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21454#discussion_r191944797
--- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala ---
@@ -448,6 +473,22 @@ class SparkConf(loadDefaults: Boolean) extends
Cloneable
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21454#discussion_r191945182
--- Diff: core/src/test/scala/org/apache/spark/SparkConfSuite.scala ---
@@ -339,6 +340,38 @@ class SparkConfSuite extends SparkFunSuite
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21454#discussion_r191883353
--- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala ---
@@ -448,6 +473,20 @@ class SparkConf(loadDefaults: Boolean) extends
Cloneable
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21454#discussion_r191899959
--- Diff: core/src/test/scala/org/apache/spark/SparkConfSuite.scala ---
@@ -339,6 +341,38 @@ class SparkConfSuite extends SparkFunSuite
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21382
Thanks! Merging to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21382
@felixcheung `spark.network.timeout` is one of the most weird configs as it
may have different default values in different places. I think this is why it's
not added to the global
GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/21382
[SPARK-24332][SS][MESOS]Fix places reading 'spark.network.timeout' as
milliseconds
## What changes were proposed in this pull request?
This PR replaces `getTimeAsMs
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21356
The code here is just propagating the interrupt state and is not doing
anything wrong. @squito do you know what would cause this in the first place
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21356
@vanzin I just want to understand the real issue here to make sure the
current solution is correct.
---
-
To unsubscribe, e
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21345
LGTM again. Merging to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21316
Thanks! Merging to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21356
> hdfs can interrupt the thread.
@squito Could you clarify this? This sounds like a bug in HDFS. I'm curious
when this will happen. Although we don't interrupt the AsyncEventQueue thr
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/21316
LGTM pending tests
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21316#discussion_r188871413
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -1607,7 +1607,9 @@ class Dataset[T] private[sql
201 - 300 of 6049 matches
Mail list logo
