Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22333
Hi, @shaneknapp and @srowen .
Can we build and use the zinc-installed docker images in our build system?
-
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/jo
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/22112#discussion_r215070653
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -1513,37 +1513,34 @@ private[spark] class DAGScheduler(
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/22112
yeah you would have to be able to handle network partitioning somehow. I
don't know how difficult it is but its definitely work we may not want to do
here. I was trying to clarify and make sure
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/22333
moving any parts of the spark build infrastructure to use docker is a big
project and not happening in the next few months.
---
---
Github user wmellouli commented on the issue:
https://github.com/apache/spark/pull/22332
@mgaido91 Thank you for your suggestion, I updated the PR name, description
and sources with a new version using a parameter `atPosition` instead of a flag
`atTheEnd`. Let me know what you think a
Github user huaxingao commented on the issue:
https://github.com/apache/spark/pull/20442
Any more comments? @MLnick @jkbradley
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comman
GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/22334
[SPARK-25336][SS]Revert SPARK-24863 and SPARK 24748
## What changes were proposed in this pull request?
Revert SPARK-24863 and SPARK 24748 as per discussion in #21721. We will
revisit them
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22334
**[Test build #95684 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95684/testReport)**
for PR 22334 at commit
[`3d59df1`](https://github.com/apache/spark/commit/3d
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22334
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2846/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22334
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user dbtsai commented on the issue:
https://github.com/apache/spark/pull/21756
add @jerryshao for more feedback. Thanks.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comman
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22333
Oh, I assumed that it's already dockerized. Sorry, never mind about that
@shaneknapp . And, thanks!
---
-
To unsubscribe,
Github user HeartSaVioR commented on the issue:
https://github.com/apache/spark/pull/22138
@zsxwing
If it means code freeze for 2.4 is just around the corner then sure! We can
focus on blockers for releasing 2.4, and revisit this again. Let me reflect
@gaborgsomogyi review commen
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22313
**[Test build #95680 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95680/testReport)**
for PR 22313 at commit
[`3cd4443`](https://github.com/apache/spark/commit/3
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22313
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22313
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95680/
Test FAILed.
---
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22313
At this time, R failure.
```
DONE
===
Had test warnings or failures; see logs.
```
---
---
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22313
Retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: rev
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22313
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22313
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2847/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22313
**[Test build #95685 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95685/testReport)**
for PR 22313 at commit
[`3cd4443`](https://github.com/apache/spark/commit/3c
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22218
**[Test build #4331 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4331/testReport)**
for PR 22218 at commit
[`e72966e`](https://github.com/apache/spark/commit/
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22171
@vinodkc Could you answer the question from @cloud-fan ?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Fo
Github user ifilonenko commented on the issue:
https://github.com/apache/spark/pull/22298
@felixcheung @holdenk I have moved the PySpark example files to a more
appropriate location. Any other comments before merge?
---
--
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22298
**[Test build #95686 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95686/testReport)**
for PR 22298 at commit
[`7dc26ce`](https://github.com/apache/spark/commit/7d
Github user tigerquoll commented on the issue:
https://github.com/apache/spark/pull/21308
I am assuming this API was intended to support the "drop partition"
use-case. I'm arguing that adding and deleting partitions deal with a concept
that is a slightly higher concept than just a bu
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22298
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2848/
---
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22234
Did we introduce any behavior change in
https://github.com/apache/spark/pull/21273? Does this PR resolve it?
---
-
To unsubsc
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/spark/pull/22282#discussion_r215092933
--- Diff:
external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaWriteTask.scala
---
@@ -88,7 +92,30 @@ private[kafka010] abstract c
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22298
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22298
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2848/
---
--
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22298
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2848/
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21308
@tigerquoll, what we come up with needs to work across a variety of data
sources, including those like JDBC that can delete at a lower granularity than
partition.
For Hive tables, the partit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22334
**[Test build #95684 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95684/testReport)**
for PR 22334 at commit
[`3d59df1`](https://github.com/apache/spark/commit/3
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22334
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22334
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95684/
Test FAILed.
---
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/22334
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22334
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22334
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2849/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22334
**[Test build #95687 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95687/testReport)**
for PR 22334 at commit
[`3d59df1`](https://github.com/apache/spark/commit/3d
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/22332
I also can't find a strong reason to append a new API in `Dataset`... btw,
to add a new API there, you'd be better to discuss in jira before making a pr,
I think. cc: @rxin @cloud-fan @HyukjinKwon
Github user tigerquoll commented on the issue:
https://github.com/apache/spark/pull/21306
Sure,
I am looking at the point of view of supporting Kudu. Check out
https://kudu.apache.org/docs/schema_design.html#partitioning for some of the
details. In particular
https://kudu.apach
Github user tigerquoll commented on the issue:
https://github.com/apache/spark/pull/21306
So Kudu range partitions support arbitrary sized partition intervals, like
the example below, where the first and last range partition are six months in
size, but the middle partition is one year
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22138
**[Test build #95688 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95688/testReport)**
for PR 22138 at commit
[`9685cc5`](https://github.com/apache/spark/commit/96
Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/22320#discussion_r215106921
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala
---
@@ -56,7 +56,7 @@ case class Inser
Github user bomeng commented on the issue:
https://github.com/apache/spark/pull/21638
Here is the test code, not sure it is right or not ---
```
test("Number of partitions") {
sc = new SparkContext(new
SparkConf().setAppName("test").setMaster("local")
.set(
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/21638
Ideally the last test should have 50 partitions? is it because we really
need the test data to be at least 50 bytes? ideally a multiple of 50, I guess.
---
-
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/22324
ping @srowen @HyukjinKwon
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: rev
Github user fangshil commented on the issue:
https://github.com/apache/spark/pull/21310
To summarize our discussion in this pr:
Spark-avro is now merged into Spark as a built-in data source. Upstream
community is not merging the AvroEncoder to support Avro types in Dataset,
inste
Github user fangshil closed the pull request at:
https://github.com/apache/spark/pull/21310
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user hindog commented on the issue:
https://github.com/apache/spark/pull/17174
I believe another performance impact related to this may be attributed to
the `cast` operator failing to match during filter-pushdown, meaning that the
filter on the timestamp will NOT get pushed dow
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22334
**[Test build #95687 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95687/testReport)**
for PR 22334 at commit
[`3d59df1`](https://github.com/apache/spark/commit/3
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22324#discussion_r215111327
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/FileBasedDataSourceSuite.scala ---
@@ -473,6 +476,27 @@ class FileBasedDataSourceSuite extends QueryTe
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22334
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22334
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95687/
Test FAILed.
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22112
@tgravescs yes you are right about the problem here. Instead of asking
executors to remove old committed shuffle data, I prefer #6648 , which just
write new shuffle data with a different file name
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21669
**[Test build #95682 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95682/testReport)**
for PR 21669 at commit
[`aa3779c`](https://github.com/apache/spark/commit/a
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21669
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95682/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21669
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22334
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22333
**[Test build #95683 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95683/testReport)**
for PR 22333 at commit
[`ca99634`](https://github.com/apache/spark/commit/c
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22334
**[Test build #95689 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95689/testReport)**
for PR 22334 at commit
[`3d59df1`](https://github.com/apache/spark/commit/3d
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22333
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22333
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95683/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22334
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2850/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22334
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/22333#discussion_r215115678
--- Diff: build/mvn ---
@@ -91,15 +92,23 @@ install_mvn() {
# Install zinc under the build/ folder
install_zinc() {
- local zinc_path="zi
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/22219#discussion_r215115685
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -3237,6 +3238,28 @@ class Dataset[T] private[sql](
files.toSet.toArray
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22306
cc @gatorsmile @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: revi
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22329
**[Test build #95690 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95690/testReport)**
for PR 22329 at commit
[`2ad350c`](https://github.com/apache/spark/commit/2a
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22329
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22329
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2851/
Github user mhamilton723 commented on the issue:
https://github.com/apache/spark/pull/22328
@WeichenXu123. Awesome work! I have not had a chance to go through this in
depth but I did this in the originating project, [MMLSpark](www.aka.ms/spark),
a while back and have been meaning to s
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/21721
BTW I think this is probably SPIP-worthy. At the very least we should write
a design doc on this, similar to the other docs for dsv2 sub-components. We
should really think about whether it'd be possibl
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22313
**[Test build #95685 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95685/testReport)**
for PR 22313 at commit
[`3cd4443`](https://github.com/apache/spark/commit/3
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22313
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22313
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95685/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22319
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22319
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2852/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22319
**[Test build #95691 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95691/testReport)**
for PR 22319 at commit
[`9e060a4`](https://github.com/apache/spark/commit/9e
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22313
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22313
@dongjoon-hyun please also update the title of the JIRA ticket, thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22192
**[Test build #95681 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95681/testReport)**
for PR 22192 at commit
[`5a2852f`](https://github.com/apache/spark/commit/5
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22313
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95681/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22192
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22329
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95690/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22329
**[Test build #95690 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95690/testReport)**
for PR 22329 at commit
[`2ad350c`](https://github.com/apache/spark/commit/2
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22329
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22313
Thank you, @cloud-fan . Sure. I'll update them.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For addi
Github user seancxmao closed the pull request at:
https://github.com/apache/spark/pull/22183
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22306
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22306
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user Dooyoung-Hwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/22219#discussion_r215122865
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -3237,6 +3238,28 @@ class Dataset[T] private[sql](
files.toSet.to
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22234
From my understanding, yea. The problem here is sounds like ambiguity in
empty strings since they can be interpreted as empty strings and also `null`.
To me, this is actually rather a bug since
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/7#discussion_r215123978
--- Diff:
common/unsafe/src/test/java/org/apache/spark/unsafe/types/UTF8StringSuite.java
---
@@ -394,12 +394,14 @@ public void substringSQL() {
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/7#discussion_r215124064
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -2546,15 +2546,39 @@ object functions {
def soundex(e: Column): Colu
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/22333#discussion_r215124159
--- Diff: build/mvn ---
@@ -91,15 +92,23 @@ install_mvn() {
# Install zinc under the build/ folder
install_zinc() {
- local zinc_p
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22329
cc @gatorsmile and @BryanCutler
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22332
Can't we simply `select` after the the column is added? I wouldn't add this
as well - it can look confusing to be honest IMO.
---
--
401 - 500 of 584 matches
Mail list logo