Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20648
@HyukjinKwon @cloud-fan Thanks for the comment! Yes, I agreed we need to
keep the CSV's behavior. I will check how much we can clean up with it.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20622
**[Test build #87636 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87636/testReport)**
for PR 20622 at commit
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20622#discussion_r170389827
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala
---
@@ -266,6 +264,12 @@ class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20622
**[Test build #87637 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87637/testReport)**
for PR 20622 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20653
LGTM
Thanks! Merged to master
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20604
**[Test build #87635 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87635/testReport)**
for PR 20604 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20604
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1022/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19222
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/20604#discussion_r170383918
--- Diff:
core/src/main/scala/org/apache/spark/ExecutorAllocationClient.scala ---
@@ -55,18 +55,18 @@ private[spark] trait ExecutorAllocationClient {
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20604
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user tdas commented on the issue:
https://github.com/apache/spark/pull/20622
LGTM, assuming tests pass.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19222
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1023/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19222
**[Test build #87638 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87638/testReport)**
for PR 19222 at commit
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/20622#discussion_r170392408
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala
---
@@ -266,6 +264,12 @@ class
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20604
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87635/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20604
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1024/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20604
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/20647
Thanks for removing the equality methods. This changes equality for the
scan and streaming relation, though. Are those significant changes?
I still think this should not be committed until
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20648
> Yup, +1 for starting this by disallowing but up to my knowledge R's
read.csv allows then the legnth of tokens are shorter then its schema, putting
nulls (or NA) into missing fields, as a valid
Github user sujith71955 commented on the issue:
https://github.com/apache/spark/pull/20611
@gatorsmile Seems to be a random failures, each time random set of test
cases are failing. Please let me know for any suggestions
---
Github user sethah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20632#discussion_r170412046
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala ---
@@ -631,6 +651,160 @@ class RandomForestSuite extends SparkFunSuite
Github user sethah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20632#discussion_r170410747
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala ---
@@ -402,20 +405,40 @@ class RandomForestSuite extends SparkFunSuite
Github user sethah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20632#discussion_r170410687
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala ---
@@ -402,20 +407,35 @@ class RandomForestSuite extends SparkFunSuite
Github user sethah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20632#discussion_r170410905
--- Diff:
mllib/src/test/scala/org/apache/spark/mllib/tree/DecisionTreeSuite.scala ---
@@ -541,7 +541,7 @@ object DecisionTreeSuite extends SparkFunSuite {
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20604
**[Test build #87635 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87635/testReport)**
for PR 20604 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20604
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20604
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20662
Merging to 2.3. Please close the PR manually.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20622
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87636/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20622
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20622
**[Test build #87636 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87636/testReport)**
for PR 20622 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20622
**[Test build #87637 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87637/testReport)**
for PR 20622 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19222
**[Test build #87638 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87638/testReport)**
for PR 19222 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20648
_To me_ I have been roughly thinking that we should better match it to R's
read.csv and explicitly document this. I believe this is a good reference our
CSV has resembled so far.
BTW,
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20648
I think at least we should update the document for this behavior of csv
reader.
---
-
To unsubscribe, e-mail:
Github user sethah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20632#discussion_r170410775
--- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala ---
@@ -283,10 +292,12 @@ private[tree] class LearningNode(
// Here we want
Github user sethah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20632#discussion_r170412098
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala ---
@@ -631,6 +651,160 @@ class RandomForestSuite extends SparkFunSuite
Github user sethah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20632#discussion_r170410834
--- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/Node.scala ---
@@ -270,11 +269,21 @@ private[tree] class LearningNode(
* Convert this
Github user sethah commented on a diff in the pull request:
https://github.com/apache/spark/pull/20632#discussion_r170410851
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/tree/impl/RandomForestSuite.scala ---
@@ -18,17 +18,20 @@
package org.apache.spark.ml.tree.impl
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20647
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1025/
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20647
> This changes equality for the scan and streaming relation, though.
If we think this is the right equality for `DataSourceV2Relation`, it
should also be the right equality for scan and
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20647
**[Test build #87640 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87640/testReport)**
for PR 20647 at commit
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20663#discussion_r170408040
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllStagesPage.scala
---
@@ -143,76 +72,105 @@ private[ui] class AllStagesPage(parent: StagesTab)
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20663#discussion_r170407867
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllStagesPage.scala
---
@@ -143,76 +72,105 @@ private[ui] class AllStagesPage(parent: StagesTab)
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20663#discussion_r170407936
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllStagesPage.scala
---
@@ -143,76 +72,105 @@ private[ui] class AllStagesPage(parent: StagesTab)
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20663#discussion_r170407811
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllStagesPage.scala
---
@@ -19,46 +19,22 @@ package org.apache.spark.ui.jobs
import
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20663#discussion_r170407883
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/AllStagesPage.scala
---
@@ -143,76 +72,105 @@ private[ui] class AllStagesPage(parent: StagesTab)
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20622
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87637/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20622
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20647
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20604
**[Test build #87639 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87639/testReport)**
for PR 20604 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19222
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87638/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19222
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20604
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87639/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20604
**[Test build #87639 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87639/testReport)**
for PR 20604 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20604
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20666
cc @cloud-fan @HyukjinKwon To keep CSV reader behavior for corrupted
records, we don't bother to refactoring. But we should update the document and
explicitly disable partial results for corrupted
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20666
**[Test build #87641 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87641/testReport)**
for PR 20666 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20666
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20666
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1026/
GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/20666
[SPARK-23448][SQL] Clarify JSON and CSV parser behavior in document
## What changes were proposed in this pull request?
Clarify JSON and CSV reader behavior in document.
JSON
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/20647#discussion_r170311026
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala
---
@@ -23,11 +23,11 @@ import
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20647
**[Test build #87634 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87634/testReport)**
for PR 20647 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20663
**[Test build #87631 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87631/testReport)**
for PR 20663 at commit
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/19222#discussion_r170318693
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/memory/ByteArrayMemoryBlock.java
---
@@ -0,0 +1,190 @@
+/*
+ * Licensed to the Apache
Github user liyinan926 commented on the issue:
https://github.com/apache/spark/pull/20553
`spark.kubernetes.executor.cores` has nothing to do with dynamic resource
allocation. It's just a way of letting users specify a value for the cpu
resource request that conforms to Kubernetes
Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/20647#discussion_r170307194
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala
---
@@ -35,15 +35,14 @@ case class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/20647#discussion_r170307903
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala
---
@@ -23,11 +23,11 @@ import
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20647
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1021/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20647
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20572#discussion_r170279950
--- Diff:
external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaRDDSuite.scala
---
@@ -64,6 +69,41 @@ class KafkaRDDSuite extends
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20572#discussion_r170278078
--- Diff:
external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaRDDSuite.scala
---
@@ -64,6 +69,41 @@ class KafkaRDDSuite extends
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20572#discussion_r170277915
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaRDD.scala
---
@@ -172,57 +187,138 @@ private[spark] class KafkaRDD[K,
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20572#discussion_r170279150
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaRDD.scala
---
@@ -87,47 +89,60 @@ private[spark] class KafkaRDD[K,
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/19222#discussion_r170296358
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/memory/HeapMemoryAllocator.java
---
@@ -58,7 +58,8 @@ public MemoryBlock allocate(long size)
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20663
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87631/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20663
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20663
Could you file a separate bug for this cleanup? Thx
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/20647#discussion_r170185948
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
---
@@ -77,31 +79,32 @@ class MicroBatchExecution(
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20658
**[Test build #87627 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87627/testReport)**
for PR 20658 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20658
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87627/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20647
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20658
**[Test build #87627 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87627/testReport)**
for PR 20658 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20647
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87624/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20658
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20658
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20647
**[Test build #87624 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87624/testReport)**
for PR 20647 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20648
+1 for disallowing it anyway if it was Wenchen's opinion too. Please go
ahead. Will help double check anyway.
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19222
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1020/
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/19222#discussion_r170296478
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java ---
@@ -45,38 +44,149 @@
*/
public static final int
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/19222#discussion_r170296403
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java ---
@@ -22,10 +22,9 @@
import org.apache.spark.unsafe.Platform;
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19222
**[Test build #87633 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87633/testReport)**
for PR 19222 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19222
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/20553
also cc @cloud-fan @jerryshao
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/20664#discussion_r170279656
--- Diff: core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala ---
@@ -1129,6 +1129,36 @@ class RDDSuite extends SparkFunSuite with
SharedSparkContext
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19222#discussion_r170305944
--- Diff:
common/unsafe/src/main/java/org/apache/spark/unsafe/memory/ByteArrayMemoryBlock.java
---
@@ -0,0 +1,190 @@
+/*
+ * Licensed to the
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/20553
IIUC the `spark.kubernetes.executor.cores` here is just a special case for
`spark.executor.cores`, for k8s backend, you shall still have to handle float
values if you're to read the value of
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20572#discussion_r170278317
--- Diff:
external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaTestUtils.scala
---
@@ -162,17 +162,22 @@ private[kafka010] class
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20572#discussion_r170279504
--- Diff:
external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaRDDSuite.scala
---
@@ -64,6 +69,41 @@ class KafkaRDDSuite extends
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20572#discussion_r170278931
--- Diff:
external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/CachedKafkaConsumer.scala
---
@@ -71,25 +69,62 @@ class
1 - 100 of 174 matches
Mail list logo
