[GitHub] spark issue #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with return ty...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20507 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/586/

[GitHub] spark issue #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with return ty...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20507 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with return ty...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20507 **[Test build #87063 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87063/testReport)** for PR 20507 at commit

[GitHub] spark pull request #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tes...

2018-02-05 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20487#discussion_r165940636 --- Diff: pom.xml --- @@ -185,6 +185,10 @@ 2.8 1.8 1.0.0 +

[GitHub] spark pull request #20499: [SPARK-23328][PYTHON] Disallow default value None...

2018-02-05 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20499#discussion_r165950192 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1557,6 +1557,9 @@ def replace(self, to_replace, value=None, subset=None): For

[GitHub] spark pull request #20481: [SPARK-23307][WEBUI]Sort jobs/stages/tasks/querie...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20481#discussion_r165934452 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -888,8 +888,8 @@ private[spark] class AppStatusListener(

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20477 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/587/

[GitHub] spark pull request #20481: [SPARK-23307][WEBUI]Sort jobs/stages/tasks/querie...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20481#discussion_r165934385 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -875,8 +875,8 @@ private[spark] class AppStatusListener(

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20477 **[Test build #87064 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87064/testReport)** for PR 20477 at commit

[GitHub] spark issue #20508: [SPARK-23335][SQL] Should not convert to double when the...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20508 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20509: [SPARK-23268][SQL][followup] Reorganize packages ...

2018-02-05 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/20509 [SPARK-23268][SQL][followup] Reorganize packages in data source V2 ## What changes were proposed in this pull request? This is a followup of https://github.com/apache/spark/pull/20435.

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2018-02-05 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/18555 cc @HyukjinKwon,@cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tests for ...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20487 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread wangyum
GitHub user wangyum opened a pull request: https://github.com/apache/spark/pull/20510 [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4 ## What changes were proposed in this pull request? This PR upgrade snappy-java to 1.1.4. release notes: - Fix a 1% performance

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20510 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20499 **[Test build #87068 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87068/testReport)** for PR 20499 at commit

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20499 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87068/ Test PASSed. ---

[GitHub] spark pull request #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with re...

2018-02-05 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/20507 [SPARK-23334][SQL][PYTHON] Fix pandas_udf with return type StringType() to handle str type properly in Python 2. ## What changes were proposed in this pull request? In Python 2, when

[GitHub] spark pull request #20226: [SPARK-23034][SQL] Override `nodeName` for all *S...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20226#discussion_r165932670 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -86,6 +86,9 @@ case class RowDataSourceScanExec(

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20477 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20481: [SPARK-23307][WEBUI]Sort jobs/stages/tasks/querie...

2018-02-05 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20481 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20508: [SPARK-23335][SQL] Should not convert to double w...

2018-02-05 Thread caneGuy
GitHub user caneGuy opened a pull request: https://github.com/apache/spark/pull/20508 [SPARK-23335][SQL] Should not convert to double when there is an Integra… …l value in BinaryArithmetic which will loss precison ## What changes were proposed in this pull request?

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20509 cc @gengliangwang @jose-torres @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.2 plus ...

2018-02-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20373 FYI, I am trying to make a minor release of cloudpickle to match with this to deduplicate our efforts. We put many efforts to find and backport bug fixes here. :-). ---

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20509 **[Test build #87065 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87065/testReport)** for PR 20509 at commit

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20509 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87065/ Test FAILed. ---

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20499 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20499 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/591/

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20499 **[Test build #87068 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87068/testReport)** for PR 20499 at commit

[GitHub] spark pull request #20495: [SPARK-23327] [SQL] Update the description and te...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20495#discussion_r165928335 --- Diff: python/pyspark/sql/functions.py --- @@ -1705,10 +1705,12 @@ def unhex(col): @ignore_unicode_prefix @since(1.5) def length(col):

[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20226 By default `simpleString` is defined as `s"$nodeName $argString".trim`, if we overwrite `nodeName` in some node, we should also overwrite `argString`, otherwise we may have duplicated information

[GitHub] spark issue #20481: [SPARK-23307][WEBUI]Sort jobs/stages/tasks/queries with ...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20481 thanks, merging to master/2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with return ty...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20507 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87063/ Test PASSed. ---

[GitHub] spark issue #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with return ty...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20507 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with return ty...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20507 **[Test build #87063 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87063/testReport)** for PR 20507 at commit

[GitHub] spark issue #20508: [SPARK-23335][SQL] Should not convert to double when the...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20508 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20509 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20509 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/588/

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20509 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20499: [SPARK-23328][PYTHON] Disallow default value None...

2018-02-05 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20499#discussion_r165951235 --- Diff: python/pyspark/sql/tests.py --- @@ -2186,7 +2186,7 @@ def test_replace(self): # replace with subset specified with one column

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20510 **[Test build #87067 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87067/testReport)** for PR 20510 at commit

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20510 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/590/

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20499 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with return ty...

2018-02-05 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20507 cc @BryanCutler @icexelloss @HyukjinKwon Could you help me double-check this? Since seems like this happens only in Python 2 environment, Jenkins will skip the tests. And let me know if

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20509 **[Test build #87065 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87065/testReport)** for PR 20509 at commit

[GitHub] spark issue #20373: [SPARK-23159][PYTHON] Update cloudpickle to v0.4.2 plus ...

2018-02-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20373 To me, yup. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tests for ...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20487 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/589/

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tests for ...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20487 **[Test build #87066 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87066/testReport)** for PR 20487 at commit

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-05 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/20509 The proposal sounds good to me 👍 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for ...

2018-02-05 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20506#discussion_r165980562 --- Diff: python/pyspark/sql/types.py --- @@ -1694,6 +1694,21 @@ def from_arrow_schema(arrow_schema): for field in arrow_schema])

[GitHub] spark pull request #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable l...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20387#discussion_r165981421 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,17 +17,151 @@ package

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20509 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/593/

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20509 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for ...

2018-02-05 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20506#discussion_r165987965 --- Diff: python/pyspark/sql/types.py --- @@ -1694,6 +1694,21 @@ def from_arrow_schema(arrow_schema): for field in arrow_schema])

[GitHub] spark issue #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for date ty...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20506 **[Test build #87071 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87071/testReport)** for PR 20506 at commit

[GitHub] spark issue #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with return ty...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20507 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with return ty...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20507 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87069/ Test PASSed. ---

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-05 Thread Fokko
Github user Fokko commented on the issue: https://github.com/apache/spark/pull/20057 Any idea when this will be merged into master? We could use this since we are ditching sqoop 👍 --- - To unsubscribe, e-mail:

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20477 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87064/ Test PASSed. ---

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20477 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18555: [SPARK-21353][CORE]add checkValue in spark.internal.conf...

2018-02-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18555 Hmm .. why not addressing https://github.com/apache/spark/pull/18555#discussion_r126293557? I think that comment makes sense. ---

[GitHub] spark issue #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with return ty...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20507 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with return ty...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20507 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/592/

[GitHub] spark issue #20057: [SPARK-22880][SQL] Add cascadeTruncate option to JDBC da...

2018-02-05 Thread danielvdende
Github user danielvdende commented on the issue: https://github.com/apache/spark/pull/20057 @Stephan202 thanks for pointing out those docs issues, just pushed the changes :-). @gatorsmile @dongjoon-hyun would you have a chance to take a look at this again? ---

[GitHub] spark pull request #20508: [SPARK-23335][SQL] Should not convert to double w...

2018-02-05 Thread wangyum
Github user wangyum commented on a diff in the pull request: https://github.com/apache/spark/pull/20508#discussion_r165968094 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -327,6 +327,14 @@ object TypeCoercion {

[GitHub] spark pull request #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with re...

2018-02-05 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20507#discussion_r165972212 --- Diff: python/pyspark/sql/tests.py --- @@ -3920,6 +3920,14 @@ def test_vectorized_udf_null_string(self): res =

[GitHub] spark pull request #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with re...

2018-02-05 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20507#discussion_r165968902 --- Diff: python/pyspark/sql/tests.py --- @@ -3920,6 +3920,14 @@ def test_vectorized_udf_null_string(self): res =

[GitHub] spark issue #20477: [SPARK-23303][SQL] improve the explain result for data s...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20477 **[Test build #87064 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87064/testReport)** for PR 20477 at commit

[GitHub] spark pull request #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with re...

2018-02-05 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20507#discussion_r165980594 --- Diff: python/pyspark/sql/tests.py --- @@ -3920,6 +3920,14 @@ def test_vectorized_udf_null_string(self): res = df.select(str_f(col('str')))

[GitHub] spark pull request #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with re...

2018-02-05 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20507#discussion_r165980572 --- Diff: python/pyspark/sql/tests.py --- @@ -3920,6 +3920,14 @@ def test_vectorized_udf_null_string(self): res = df.select(str_f(col('str')))

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-05 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20387 For doing pushdown at logical or physical phase, I don't have a strong preference. I think at logical phase we should try our best to push down data-size-reduction operators(like filter,

[GitHub] spark issue #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with return ty...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20507 **[Test build #87069 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87069/testReport)** for PR 20507 at commit

[GitHub] spark issue #20509: [SPARK-23268][SQL][followup] Reorganize packages in data...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20509 **[Test build #87070 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87070/testReport)** for PR 20509 at commit

[GitHub] spark issue #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for date ty...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20506 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/594/

[GitHub] spark issue #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for date ty...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20506 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2018-02-05 Thread DaimonPl
Github user DaimonPl commented on the issue: https://github.com/apache/spark/pull/16578 So if it's not going to be included in `2.3.0` maybe we could change `spark.sql.nestedSchemaPruning.enabled` to default `true` ? I hope that this time this PR could be finalized at the early stage

[GitHub] spark issue #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with return ty...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20507 **[Test build #87069 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87069/testReport)** for PR 20507 at commit

[GitHub] spark pull request #20167: [SPARK-16501] [MESOS] Allow providing Mesos princ...

2018-02-05 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/20167#discussion_r165994809 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -71,40 +74,64 @@ trait

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20510 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87067/ Test FAILed. ---

[GitHub] spark issue #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for date ty...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20506 **[Test build #87071 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87071/testReport)** for PR 20506 at commit

[GitHub] spark issue #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for date ty...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20506 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87071/ Test PASSed. ---

[GitHub] spark issue #20506: [SPARK-23290][SQL][PYTHON] Use datetime.date for date ty...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20506 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20510 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20510 **[Test build #87067 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87067/testReport)** for PR 20510 at commit

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tests for ...

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20487 **[Test build #87066 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87066/testReport)** for PR 20487 at commit

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tests for ...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20487 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87066/ Test PASSed. ---

[GitHub] spark issue #20487: [SPARK-23319][TESTS] Explicitly skips PySpark tests for ...

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20487 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20510 **[Test build #87072 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87072/testReport)** for PR 20510 at commit

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20510 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20510 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/595/

[GitHub] spark issue #17190: [SPARK-19478][SS] JDBC Sink

2018-02-05 Thread GaalDornick
Github user GaalDornick commented on the issue: https://github.com/apache/spark/pull/17190 Sorry for abandoning this. Micheal Armburst had indicated to me that this should be really be a Spark package, and not part of Spark itself. So, it is unlikely that this will get merged. I

[GitHub] spark pull request #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20510#discussion_r166030499 --- Diff: dev/deps/spark-deps-hadoop-2.6 --- @@ -182,7 +182,7 @@ slf4j-api-1.7.16.jar slf4j-log4j12-1.7.16.jar snakeyaml-1.15.jar

[GitHub] spark pull request #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable l...

2018-02-05 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20387#discussion_r166030958 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,17 +17,151 @@ package

[GitHub] spark issue #20510: [SPARK-23336][BUILD] Upgrade snappy-java to 1.1.4

2018-02-05 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20510 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20507: [SPARK-23334][SQL][PYTHON] Fix pandas_udf with re...

2018-02-05 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20507#discussion_r166018470 --- Diff: python/pyspark/serializers.py --- @@ -230,6 +230,9 @@ def create_array(s, t): s =

[GitHub] spark pull request #20511: [SPARK-23340][BUILD] Update ORC to 1.4.2

2018-02-05 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/20511 [SPARK-23340][BUILD] Update ORC to 1.4.2 ## What changes were proposed in this pull request? This PR updates Apache ORC dependencies to 1.4.2 released on January 23rd. Apache ORC

[GitHub] spark issue #20511: [SPARK-23340][BUILD] Update ORC to 1.4.2

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20511 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20511: [SPARK-23340][BUILD] Update ORC to 1.4.2

2018-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20511 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/596/

[GitHub] spark issue #20511: [SPARK-23340][BUILD] Update ORC to 1.4.2

2018-02-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20511 **[Test build #87073 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87073/testReport)** for PR 20511 at commit

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-05 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20387 > For safety, I wanna keep it unchanged, and start something new for data source v2 only. I disagree. * **#20476 addresses a bug caused by the new implementation that is not a

  1   2   3   4   >