Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/21698
Taking a step back and analyzing the solution for the problem at hand.
There are three main issues with the proposal:
* It does not solve the problem in a general manner.
* I g
Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21769#discussion_r202526423
--- Diff:
external/avro/src/main/scala/org/apache/spark/sql/avro/AvroFileFormat.scala ---
@@ -64,7 +64,7 @@ private[avro] class AvroFileFormat extends FileF
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21769
**[Test build #93008 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93008/testReport)**
for PR 21769 at commit
[`a7d078e`](https://github.com/apache/spark/commit/a7
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21769
**[Test build #93008 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93008/testReport)**
for PR 21769 at commit
[`a7d078e`](https://github.com/apache/spark/commit/a
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21769
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93008/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21769
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21770
cc @rednaxelafx
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21769
**[Test build #93009 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93009/testReport)**
for PR 21769 at commit
[`3b75c27`](https://github.com/apache/spark/commit/3b
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21766
They are not exactly the same in ANSI SQL, although MS SQL Server looks
like use them interchangeably.
https://docs.microsoft.com/en-us/sql/t-sql/data-types/decimal-and-numeric-transact-sql?view=
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21657
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21657
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@sp
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21769
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93009/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21769
**[Test build #93009 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93009/testReport)**
for PR 21769 at commit
[`3b75c27`](https://github.com/apache/spark/commit/3
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21769
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21657
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21657
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/956/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21657
**[Test build #93010 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93010/testReport)**
for PR 21657 at commit
[`dd5bb59`](https://github.com/apache/spark/commit/dd
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/20611#discussion_r202528097
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ---
@@ -303,94 +303,44 @@ case class LoadDataCommand(
s"
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21657#discussion_r202527784
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -38,24 +38,29 @@ class UnivocityParser(
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21657#discussion_r202527982
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
---
@@ -1579,4 +1579,20 @@ class CSVSuite extends QueryT
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21657#discussion_r202528247
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -38,24 +38,29 @@ class UnivocityParser(
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21657#discussion_r202528267
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -38,24 +38,29 @@ class UnivocityParser(
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21657#discussion_r202528372
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -223,8 +234,8 @@ class UnivocityParser(
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21657#discussion_r202528391
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -197,15 +202,21 @@ class UnivocityParser(
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/21657#discussion_r202528931
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -223,8 +234,8 @@ class UnivocityParser(
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/21657#discussion_r202528933
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -38,24 +38,29 @@ class UnivocityParser(
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/21657#discussion_r202528977
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/UnivocityParser.scala
---
@@ -197,15 +202,21 @@ class UnivocityParser(
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21766
aha, I didn't know that and postgresql also uses them interchangeably, too.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spar
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21770
**[Test build #93011 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93011/testReport)**
for PR 21770 at commit
[`d817f9d`](https://github.com/apache/spark/commit/d8
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21770
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/957/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21770
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21770
Also, `SparkException` needs to extend `RuntimeException` instead of
`Exception` because some generated codes do not have code to catch the
exception. But, the change causes Mima test failures, so it
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21764
@gatorsmile aha, ok. We need to make this option not `internal` but
`external`?
BTW, the interfaces to add/delete optimizer rules (addition via
`ExperimentalMethods` and deletion via `SQLCon
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21770
**[Test build #93011 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93011/testReport)**
for PR 21770 at commit
[`d817f9d`](https://github.com/apache/spark/commit/d
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21770
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21770
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93011/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21657
**[Test build #93010 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93010/testReport)**
for PR 21657 at commit
[`dd5bb59`](https://github.com/apache/spark/commit/d
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21657
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21657
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93010/
Test PASSed.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21741
Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21770
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h..
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21741
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21770
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21770
**[Test build #93012 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93012/testReport)**
for PR 21770 at commit
[`d817f9d`](https://github.com/apache/spark/commit/d8
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21770
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/958/
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21771#discussion_r202530411
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging
{
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21771#discussion_r202530444
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging
{
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21771#discussion_r202530492
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging
{
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21771#discussion_r202530515
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging
{
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21771
Seems fine to me.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21770
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21770
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93012/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21770
**[Test build #93012 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93012/testReport)**
for PR 21770 at commit
[`d817f9d`](https://github.com/apache/spark/commit/d
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21711#discussion_r202531090
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
---
@@ -2005,6 +2005,24 @@ class SQLQuerySuite extends QueryTe
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21711#discussion_r202531122
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -138,17 +138,35 @@ private[spark] class HiveExternalCatalog(
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21711#discussion_r202531134
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ---
@@ -138,17 +138,35 @@ private[spark] class HiveExternalCatalog(
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21768
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21764
@maropu This is for advanced end users or Spark developers. External conf
looks fine, but I have to admit this might be rarely used. BTW, after having
this conf, we can deprecate a few internal c
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21770
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h..
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21770
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/959/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21770
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21770
**[Test build #93013 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93013/testReport)**
for PR 21770 at commit
[`d817f9d`](https://github.com/apache/spark/commit/d8
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21771#discussion_r202531553
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging
{
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21771
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.ap
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21764
ok, thx for the kind explanation.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-ma
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21771#discussion_r202531555
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging
{
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21771
Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21771
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21556
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21556
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/960/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21556
**[Test build #93014 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93014/testReport)**
for PR 21556 at commit
[`e31c201`](https://github.com/apache/spark/commit/e3
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21740#discussion_r202531686
--- Diff:
mllib/src/test/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModelSuite.scala
---
@@ -72,6 +72,22 @@ class MatrixFactorizationMod
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21740#discussion_r202531693
--- Diff:
mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala
---
@@ -75,10 +75,22 @@ class MatrixFactorizationModel @
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21770
**[Test build #93013 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93013/testReport)**
for PR 21770 at commit
[`d817f9d`](https://github.com/apache/spark/commit/d
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21770
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93013/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21770
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21771#discussion_r202531879
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging
{
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21771#discussion_r202531920
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1849,6 +1858,9 @@ class SparkContext(config: SparkConf) extends Logging
{
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21771
We could have updated the doc for `spark.files.overwrite` too since the
confusion probably with this configuration.
---
-
To
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21771#discussion_r202531999
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1555,6 +1559,9 @@ class SparkContext(config: SparkConf) extends Logging
{
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/21589
AFAIK, we always have num of executor and then num of core per executor
right?
https://spark.apache.org/docs/latest/configuration.html#execution-behavior
maybe we should have the get
Github user maropu commented on the issue:
https://github.com/apache/spark/pull/21770
I'll fix the failures soon.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: re
101 - 182 of 182 matches
Mail list logo