[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20525 **[Test build #87250 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87250/testReport)** for PR 20525 at commit

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20556 **[Test build #87249 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87249/testReport)** for PR 20556 at commit

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20521 **[Test build #87245 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87245/testReport)** for PR 20521 at commit

[GitHub] spark issue #20545: [SPARK-23359][SQL] Adds an alias 'names' of 'fieldNames'...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20545 **[Test build #87246 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87246/testReport)** for PR 20545 at commit

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20521 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/745/

[GitHub] spark issue #20545: [SPARK-23359][SQL] Adds an alias 'names' of 'fieldNames'...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20545 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20545: [SPARK-23359][SQL] Adds an alias 'names' of 'fieldNames'...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20545 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/744/

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20521 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20521 **[Test build #87252 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87252/testReport)** for PR 20521 at commit

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r167162376 --- Diff: docs/sql-programming-guide.md --- @@ -1930,6 +1930,8 @@ working with timestamps in `pandas_udf`s to get the best performance, see -

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-09 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r167163193 --- Diff: docs/sql-programming-guide.md --- @@ -1930,6 +1930,8 @@ working with timestamps in `pandas_udf`s to get the best performance, see -

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-09 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r167163936 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala --- @@ -301,7 +301,6 @@ class DataFrameReaderWriterSuite

[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20525 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/747/

[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20525 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/748/

[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20525 **[Test build #87255 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87255/testReport)** for PR 20525 at commit

[GitHub] spark pull request #20520: [SPARK-23344][PYTHON][ML] Add distanceMeasure par...

2018-02-09 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20520#discussion_r167165450 --- Diff: python/pyspark/ml/tests.py --- @@ -1620,6 +1621,23 @@ def test_kmeans_summary(self): self.assertEqual(s.k, 2) +class

[GitHub] spark pull request #20558: [SPARK-21860][CORE][FOLLOWUP] fix java style erro...

2018-02-09 Thread mgaido91
GitHub user mgaido91 opened a pull request: https://github.com/apache/spark/pull/20558 [SPARK-21860][CORE][FOLLOWUP] fix java style error ## What changes were proposed in this pull request? #19077 introduced a Java style error (too long line). Quick fix. ## How was

[GitHub] spark issue #20558: [SPARK-21860][CORE][FOLLOWUP] fix java style error

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20558 **[Test build #87257 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87257/testReport)** for PR 20558 at commit

[GitHub] spark issue #20520: [SPARK-23344][PYTHON][ML] Add distanceMeasure param to K...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20520 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20520: [SPARK-23344][PYTHON][ML] Add distanceMeasure param to K...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20520 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87256/ Test PASSed. ---

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r167176642 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala --- @@ -301,7 +301,6 @@ class DataFrameReaderWriterSuite

[GitHub] spark issue #20167: [SPARK-16501] [MESOS] Allow providing Mesos principal & ...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20167 **[Test build #87258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87258/testReport)** for PR 20167 at commit

[GitHub] spark issue #20167: [SPARK-16501] [MESOS] Allow providing Mesos principal & ...

2018-02-09 Thread rvesse
Github user rvesse commented on the issue: https://github.com/apache/spark/pull/20167 @vanzin Ok I should hopefully have all those addressed and the documentation clarified appropriately --- - To unsubscribe,

[GitHub] spark issue #20167: [SPARK-16501] [MESOS] Allow providing Mesos principal & ...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20167 **[Test build #87259 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87259/testReport)** for PR 20167 at commit

[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20525 **[Test build #87254 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87254/testReport)** for PR 20525 at commit

[GitHub] spark issue #20167: [SPARK-16501] [MESOS] Allow providing Mesos principal & ...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20167 **[Test build #87258 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87258/testReport)** for PR 20167 at commit

[GitHub] spark issue #20167: [SPARK-16501] [MESOS] Allow providing Mesos principal & ...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20167 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20167: [SPARK-16501] [MESOS] Allow providing Mesos principal & ...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20167 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87258/ Test PASSed. ---

[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20525 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87254/ Test FAILed. ---

[GitHub] spark issue #20525: [SPARK-23271[SQL] Parquet output contains only _SUCCESS ...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20525 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20167: [SPARK-16501] [MESOS] Allow providing Mesos principal & ...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20167 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20167: [SPARK-16501] [MESOS] Allow providing Mesos principal & ...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20167 **[Test build #87259 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87259/testReport)** for PR 20167 at commit

[GitHub] spark issue #20167: [SPARK-16501] [MESOS] Allow providing Mesos principal & ...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20167 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87259/ Test PASSed. ---

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-09 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r167160198 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala --- @@ -301,7 +301,6 @@ class DataFrameReaderWriterSuite

[GitHub] spark pull request #20525: [SPARK-23271[SQL] Parquet output contains only _S...

2018-02-09 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/20525#discussion_r167160086 --- Diff: docs/sql-programming-guide.md --- @@ -1930,6 +1930,9 @@ working with timestamps in `pandas_udf`s to get the best performance, see -

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20521 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87245/ Test FAILed. ---

[GitHub] spark issue #20521: [SPARK-22977][SQL] fix web UI SQL tab for CTAS

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20521 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20556 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20556: [SPARK-23367][Build] Include python document style check...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20556 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87249/ Test FAILed. ---

[GitHub] spark issue #20545: [SPARK-23359][SQL] Adds an alias 'names' of 'fieldNames'...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20545 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20549: SPARK-18844[MLLIB] Add more binary classification metric...

2018-02-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20549 As I say, I don't think this would be merged anyway. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #19431: [SPARK-18580] [DStreams] [external/kafka-0-10][ex...

2018-02-09 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19431#discussion_r167281176 --- Diff: external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/DirectKafkaStreamSuite.scala --- @@ -22,6 +22,7 @@ import

[GitHub] spark pull request #19431: [SPARK-18580] [DStreams] [external/kafka-0-10][ex...

2018-02-09 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19431#discussion_r167281995 --- Diff: external/kafka-0-8/src/test/scala/org/apache/spark/streaming/kafka/DirectKafkaStreamSuite.scala --- @@ -539,6 +456,58 @@ class

[GitHub] spark pull request #19431: [SPARK-18580] [DStreams] [external/kafka-0-10][ex...

2018-02-09 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/19431#discussion_r167281427 --- Diff: external/kafka-0-8/src/test/scala/org/apache/spark/streaming/kafka/DirectKafkaStreamSuite.scala --- @@ -21,6 +21,7 @@ import java.io.File

[GitHub] spark pull request #20557: [SPARK-23364][SQL]'desc table' command in spark-s...

2018-02-09 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20557#discussion_r167244246 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -539,15 +539,15 @@ case class DescribeTableCommand(

[GitHub] spark issue #20550: [MINOR][HIVE] Typo fixes

2018-02-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20550 Not sure what you mean; I think typo fixes are always logically related, and are trivial, so can easily be part of one change. For example, I skimmed with my IDE spell checker and found 40 a few

[GitHub] spark issue #20519: [Spark-23240][python] Don't let python site customizatio...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20519 **[Test build #87262 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87262/testReport)** for PR 20519 at commit

[GitHub] spark issue #20546: [SPARK-20659][Core] Removing sc.getExecutorStorageStatus...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20546 **[Test build #87265 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87265/testReport)** for PR 20546 at commit

[GitHub] spark pull request #20546: [SPARK-20659][Core] Removing sc.getExecutorStorag...

2018-02-09 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20546#discussion_r167272305 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -610,7 +610,7 @@ class

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-09 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20387 > See FindDataSourceTable.readDataSourceTable about how we handle the path option. Sorry, what do you want to change? ---

[GitHub] spark pull request #20561: [SPARK-23376][SQL] creating UnsafeKVExternalSorte...

2018-02-09 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/20561#discussion_r167299807 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/UnsafeKVExternalSorter.java --- @@ -98,10 +99,20 @@ public UnsafeKVExternalSorter(

[GitHub] spark pull request #20561: [SPARK-23376][SQL] creating UnsafeKVExternalSorte...

2018-02-09 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/20561#discussion_r167299716 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/UnsafeKVExternalSorter.java --- @@ -98,10 +99,20 @@ public UnsafeKVExternalSorter(

[GitHub] spark pull request #20520: [SPARK-23344][PYTHON][ML] Add distanceMeasure par...

2018-02-09 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20520#discussion_r167308210 --- Diff: python/pyspark/ml/tests.py --- @@ -1620,6 +1621,23 @@ def test_kmeans_summary(self): self.assertEqual(s.k, 2)

[GitHub] spark issue #20359: [SPARK-23186][SQL] Initialize DriverManager first before...

2018-02-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20359 @cloud-fan . Can we have this in branch-2.2, too? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable l...

2018-02-09 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20387#discussion_r167320953 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,17 +17,130 @@ package

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20387 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/755/

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20387 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20167: [SPARK-16501] [MESOS] Allow providing Mesos princ...

2018-02-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20167#discussion_r167321486 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -106,6 +99,40 @@ trait

[GitHub] spark pull request #20167: [SPARK-16501] [MESOS] Allow providing Mesos princ...

2018-02-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20167#discussion_r167321536 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -106,6 +99,40 @@ trait

[GitHub] spark pull request #20167: [SPARK-16501] [MESOS] Allow providing Mesos princ...

2018-02-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20167#discussion_r167321443 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -106,6 +99,40 @@ trait

[GitHub] spark pull request #20559: [WIP][SPARK-23360][SQL][PYTHON] Get local timezon...

2018-02-09 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20559#discussion_r167322999 --- Diff: python/pyspark/sql/types.py --- @@ -1709,6 +1709,15 @@ def _check_dataframe_convert_date(pdf, schema): return pdf

[GitHub] spark pull request #20559: [WIP][SPARK-23360][SQL][PYTHON] Get local timezon...

2018-02-09 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20559#discussion_r167318107 --- Diff: python/pyspark/sql/tests.py --- @@ -4124,7 +4126,7 @@ def test_vectorized_udf_timestamps(self): data = [(0, datetime(1969, 1, 1,

[GitHub] spark issue #20351: [SPARK-23014][SS] Fully remove V1 memory sink.

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20351 **[Test build #87271 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87271/testReport)** for PR 20351 at commit

[GitHub] spark issue #20520: [SPARK-23344][PYTHON][ML] Add distanceMeasure param to K...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20520 **[Test build #87270 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87270/testReport)** for PR 20520 at commit

[GitHub] spark issue #19775: [SPARK-22343][core] Add support for publishing Spark met...

2018-02-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19775 Regardless of the discussion about whether this should live in Spark, does it need to live in core? Can it be kept in a separate module like the Ganglia stuff (even though that one is for

[GitHub] spark issue #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable logical ...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20387 **[Test build #87268 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87268/testReport)** for PR 20387 at commit

[GitHub] spark pull request #20557: [SPARK-23364][SQL]'desc table' command in spark-s...

2018-02-09 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20557#discussion_r167330494 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -539,15 +539,15 @@ case class DescribeTableCommand(

[GitHub] spark pull request #20561: [SPARK-23376][SQL] creating UnsafeKVExternalSorte...

2018-02-09 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/20561#discussion_r167300330 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/UnsafeKVExternalSorterSuite.scala --- @@ -205,4 +206,42 @@ class

[GitHub] spark pull request #20490: [SPARK-23323][SQL]: Support commit coordinator fo...

2018-02-09 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20490#discussion_r167306077 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceWriter.java --- @@ -62,6 +62,16 @@ */ DataWriterFactory

[GitHub] spark issue #20561: [SPARK-23376][SQL] creating UnsafeKVExternalSorter with ...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20561 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87263/ Test PASSed. ---

[GitHub] spark issue #20561: [SPARK-23376][SQL] creating UnsafeKVExternalSorter with ...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20561 **[Test build #87263 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87263/testReport)** for PR 20561 at commit

[GitHub] spark issue #20561: [SPARK-23376][SQL] creating UnsafeKVExternalSorter with ...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20561 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20490 **[Test build #87269 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87269/testReport)** for PR 20490 at commit

[GitHub] spark pull request #20167: [SPARK-16501] [MESOS] Allow providing Mesos princ...

2018-02-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20167 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20546: [SPARK-20659][Core] Removing sc.getExecutorStorageStatus...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20546 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87266/ Test FAILed. ---

[GitHub] spark issue #20520: [SPARK-23344][PYTHON][ML] Add distanceMeasure param to K...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20520 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87270/ Test PASSed. ---

[GitHub] spark issue #20520: [SPARK-23344][PYTHON][ML] Add distanceMeasure param to K...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20520 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20520: [SPARK-23344][PYTHON][ML] Add distanceMeasure param to K...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20520 **[Test build #87270 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87270/testReport)** for PR 20520 at commit

[GitHub] spark pull request #20546: [SPARK-20659][Core] Removing sc.getExecutorStorag...

2018-02-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20546#discussion_r167303527 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -1715,7 +1715,7 @@ class SparkContext(config: SparkConf) extends Logging {

[GitHub] spark issue #20560: [SPARK-23375][SQL] Eliminate unneeded Sort in Optimizer

2018-02-09 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20560 Thanks! This should be added as a separate rule. It is actually to resolve the comment in https://github.com/apache/spark/pull/11480#discussion_r54840644 I did not review it carefully,

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20490 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/756/

[GitHub] spark issue #20490: [SPARK-23323][SQL]: Support commit coordinator for DataS...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20490 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20561: [SPARK-23376][SQL] creating UnsafeKVExternalSorte...

2018-02-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20561#discussion_r167307229 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/UnsafeKVExternalSorter.java --- @@ -98,10 +99,20 @@ public UnsafeKVExternalSorter(

[GitHub] spark issue #20519: [Spark-23240][python] Don't let python site customizatio...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20519 **[Test build #87267 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87267/testReport)** for PR 20519 at commit

[GitHub] spark pull request #20490: [SPARK-23323][SQL]: Support commit coordinator fo...

2018-02-09 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20490#discussion_r167321301 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceWriter.java --- @@ -62,6 +62,16 @@ */ DataWriterFactory

[GitHub] spark issue #20520: [SPARK-23344][PYTHON][ML] Add distanceMeasure param to K...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20520 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/757/

[GitHub] spark issue #20520: [SPARK-23344][PYTHON][ML] Add distanceMeasure param to K...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20520 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20546: [SPARK-20659][Core] Removing sc.getExecutorStorag...

2018-02-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20546#discussion_r167303770 --- Diff: core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala --- @@ -610,7 +610,7 @@ class StandaloneDynamicAllocationSuite

[GitHub] spark issue #20546: [SPARK-20659][Core] Removing sc.getExecutorStorageStatus...

2018-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20546 **[Test build #87266 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87266/testReport)** for PR 20546 at commit

[GitHub] spark issue #20546: [SPARK-20659][Core] Removing sc.getExecutorStorageStatus...

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20546 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20558: [SPARK-21860][CORE][FOLLOWUP] fix java style error

2018-02-09 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20558 Merged to master. (Agree @HyukjinKwon but this one only affects master) --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20537: [SPARK-23314][PYTHON] Add ambiguous=False when lo...

2018-02-09 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20537#discussion_r167266079 --- Diff: python/pyspark/sql/types.py --- @@ -1744,8 +1744,27 @@ def _check_series_convert_timestamps_internal(s, timezone): from

[GitHub] spark pull request #20387: [SPARK-23203][SQL]: DataSourceV2: Use immutable l...

2018-02-09 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20387#discussion_r167288341 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,17 +17,130 @@ package

[GitHub] spark issue #20560: [SPARK-23375][SQL] Eliminate unneeded Sort in Optimizer

2018-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20560 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87261/ Test PASSed. ---

[GitHub] spark issue #20559: [WIP][SPARK-23360][SQL][PYTHON] Get local timezone from ...

2018-02-09 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/20559 Thanks @ueshin for the quick patch! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20558: [SPARK-21860][CORE][FOLLOWUP] fix java style erro...

2018-02-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20558 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20559: [WIP][SPARK-23360][SQL][PYTHON] Get local timezon...

2018-02-09 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20559#discussion_r167246316 --- Diff: python/pyspark/sql/types.py --- @@ -1709,6 +1709,15 @@ def _check_dataframe_convert_date(pdf, schema): return pdf +def

[GitHub] spark pull request #20544: [SPARK-23358][CORE]When the number of partitions ...

2018-02-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20544 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20561: [SPARK-23376][SQL] creating UnsafeKVExternalSorte...

2018-02-09 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/20561 [SPARK-23376][SQL] creating UnsafeKVExternalSorter with BytesToBytesMap may fail ## What changes were proposed in this pull request? This is a long-standing bug in

[GitHub] spark pull request #20490: [SPARK-23323][SQL]: Support commit coordinator fo...

2018-02-09 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20490#discussion_r167280511 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceWriter.java --- @@ -62,6 +62,16 @@ */ DataWriterFactory

  1   2   3   4   >