[GitHub] spark issue #22206: SPARK-25213: Add project to v2 scans before python filte...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22206 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-23 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21977 Looks like tests are passing now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22211: [SPARK-23207][SPARK-22905][SPARK-24564][SPARK-25114][SQL...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22211 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22211: [SPARK-23207][SPARK-22905][SPARK-24564][SPARK-25114][SQL...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22211 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95186/ Test FAILed. ---

[GitHub] spark issue #22211: [SPARK-23207][SPARK-22905][SPARK-24564][SPARK-25114][SQL...

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22211 **[Test build #95186 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95186/testReport)** for PR 22211 at commit

[GitHub] spark issue #22211: [SPARK-23207][SPARK-22905][SPARK-24564][SPARK-25114][SQL...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22211 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22211: [SPARK-23207][SPARK-22905][SPARK-24564][SPARK-25114][SQL...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22211 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22211: [SPARK-23207][SPARK-22905][SPARK-24564][SPARK-25114][SQL...

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22211 **[Test build #95186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95186/testReport)** for PR 22211 at commit

[GitHub] spark pull request #22211: [SPARK-23207][SPARK-22905][SPARK-24564][SPARK-251...

2018-08-23 Thread henryr
GitHub user henryr opened a pull request: https://github.com/apache/spark/pull/22211 [SPARK-23207][SPARK-22905][SPARK-24564][SPARK-25114][SQL][BACKPORT-2.… ## What changes were proposed in this pull request? Back port of #20393 and #22079. Currently

[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...

2018-08-23 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/22136 Well, this merged successfully with master but not with 2.3; it seemed to pull in code from another PR, strangely. Would you mind sending a backport PR against branch-2.3? Thank you! ---

[GitHub] spark pull request #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSi...

2018-08-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22136 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...

2018-08-23 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/22136 LGTM Merging with master. I'll try to backport it to 2.3 too. Thanks! --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22206: SPARK-25213: Add project to v2 scans before pytho...

2018-08-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22206#discussion_r212484229 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala --- @@ -130,10 +133,22 @@ object

[GitHub] spark pull request #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSi...

2018-08-23 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/22136#discussion_r212483950 --- Diff: python/pyspark/ml/tests.py --- @@ -844,6 +844,28 @@ def test_string_indexer_from_labels(self):

[GitHub] spark issue #22193: [SPARK-25186][SQL] Remove v2 save mode.

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22193 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22193: [SPARK-25186][SQL] Remove v2 save mode.

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22193 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95181/ Test PASSed. ---

[GitHub] spark issue #22193: [SPARK-25186][SQL] Remove v2 save mode.

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22193 **[Test build #95181 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95181/testReport)** for PR 22193 at commit

[GitHub] spark issue #21320: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21320 LGTM, as I explained above. https://github.com/apache/spark/pull/21320#issuecomment-415526369 Thanks for your patience and great work! @mallman Sorry, it takes two years to merge the

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-08-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r212476709 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala --- @@ -31,6 +32,7 @@ import

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-08-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r212476400 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala --- @@ -31,6 +32,7 @@ import

[GitHub] spark pull request #21320: [SPARK-4502][SQL] Parquet nested column pruning -...

2018-08-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21320#discussion_r212476268 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaPruningSuite.scala --- @@ -31,6 +32,7 @@ import

[GitHub] spark issue #21320: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21320 **[Test build #95185 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95185/testReport)** for PR 21320 at commit

[GitHub] spark issue #21320: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21320 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21320: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21320 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21320: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-23 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/21320 > @mallman Could you remove the changes made in ParquetRowConverter.scala and also turn off spark.sql.nestedSchemaPruning.enabled by default in this PR? Done. ---

[GitHub] spark pull request #22157: [SPARK-25126][SQL] Avoid creating Reader for all ...

2018-08-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22157#discussion_r212475271 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcUtils.scala --- @@ -79,9 +79,10 @@ object OrcUtils extends Logging {

[GitHub] spark issue #22208: [SPARK-25216][SQL] Improve error message when a column c...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22208 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95178/ Test PASSed. ---

[GitHub] spark issue #22208: [SPARK-25216][SQL] Improve error message when a column c...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22208 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22208: [SPARK-25216][SQL] Improve error message when a column c...

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22208 **[Test build #95178 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95178/testReport)** for PR 22208 at commit

[GitHub] spark pull request #22209: [SPARK-24415][Core] Fixed the issue where stage p...

2018-08-23 Thread ankuriitg
Github user ankuriitg commented on a diff in the pull request: https://github.com/apache/spark/pull/22209#discussion_r212471192 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -350,11 +350,16 @@ private[spark] class AppStatusListener(

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-08-23 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r212470424 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -225,6 +225,18 @@ private[spark] object

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-08-23 Thread erikerlandson
Github user erikerlandson commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r212469389 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -225,6 +225,18 @@ private[spark] object

[GitHub] spark pull request #22209: [SPARK-24415][Core] Fixed the issue where stage p...

2018-08-23 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/22209#discussion_r212468772 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -350,11 +350,16 @@ private[spark] class AppStatusListener(

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95175/ Test FAILed. ---

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21977 **[Test build #95175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95175/testReport)** for PR 21977 at commit

[GitHub] spark issue #22209: [SPARK-24415][Core] Fixed the issue where stage page agg...

2018-08-23 Thread ankuriitg
Github user ankuriitg commented on the issue: https://github.com/apache/spark/pull/22209 Just checked, this fixes the Spark History Server issue (SPARK-24539) as well, I just needed to restart SHS in order for my changes to take affect. ---

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22104 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22104 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95171/ Test PASSed. ---

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22104 **[Test build #95171 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95171/testReport)** for PR 22104 at commit

[GitHub] spark pull request #22112: [SPARK-23243][Core] Fix RDD.repartition() data co...

2018-08-23 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/22112#discussion_r212462874 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1502,6 +1502,53 @@ private[spark] class DAGScheduler(

[GitHub] spark issue #22079: [SPARK-23207][SPARK-22905][SPARK-24564][SPARK-25114][SQL...

2018-08-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22079 https://github.com/apache/spark/commit/d7c3aae2074b3dd3923dd754c0a3c97308c66893 Done --- - To unsubscribe, e-mail:

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95167/ Test PASSed. ---

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21977 **[Test build #95167 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95167/testReport)** for PR 21977 at commit

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-08-23 Thread erikerlandson
Github user erikerlandson commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r212460665 --- Diff: docs/running-on-kubernetes.md --- @@ -775,4 +787,183 @@ specific to Spark on Kubernetes. This sets the major Python version of the

[GitHub] spark pull request #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in...

2018-08-23 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/22104#discussion_r212460124 --- Diff: python/pyspark/sql/tests.py --- @@ -3367,6 +3367,35 @@ def test_ignore_column_of_all_nulls(self): finally:

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-23 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/22104 @HyukjinKwon I addressed the comments. Do you mind taking a another look? --- - To unsubscribe, e-mail:

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22104 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22104 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95169/ Test PASSed. ---

[GitHub] spark issue #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in FileSo...

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22104 **[Test build #95169 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95169/testReport)** for PR 22104 at commit

[GitHub] spark issue #22206: SPARK-25213: Add project to v2 scans before python filte...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22206 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22206: SPARK-25213: Add project to v2 scans before python filte...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22206 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95173/ Test FAILed. ---

[GitHub] spark issue #22206: SPARK-25213: Add project to v2 scans before python filte...

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22206 **[Test build #95173 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95173/testReport)** for PR 22206 at commit

[GitHub] spark pull request #22112: [SPARK-23243][Core] Fix RDD.repartition() data co...

2018-08-23 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/22112#discussion_r212452014 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2627,6 +2632,81 @@ class DAGSchedulerSuite extends SparkFunSuite

[GitHub] spark pull request #22112: [SPARK-23243][Core] Fix RDD.repartition() data co...

2018-08-23 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/22112#discussion_r212451081 --- Diff: core/src/main/scala/org/apache/spark/rdd/ZippedPartitionsRDD.scala --- @@ -95,6 +99,18 @@ private[spark] class ZippedPartitionsRDD2[A: ClassTag,

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-08-23 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r212448189 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -225,6 +225,18 @@ private[spark] object Config

[GitHub] spark issue #22187: [SPARK-25178][SQL] Directly ship the StructType objects ...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22187 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22187: [SPARK-25178][SQL] Directly ship the StructType objects ...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22187 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95166/ Test PASSed. ---

[GitHub] spark issue #22187: [SPARK-25178][SQL] Directly ship the StructType objects ...

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22187 **[Test build #95166 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95166/testReport)** for PR 22187 at commit

[GitHub] spark issue #22209: [SPARK-24415][Core] Fixed the issue where stage page agg...

2018-08-23 Thread ankuriitg
Github user ankuriitg commented on the issue: https://github.com/apache/spark/pull/22209 Thanks for your comment @tgravescs. This fix is indeed for the SparkUI (running jobs) issue. It does not fix it for history server yet. ---

[GitHub] spark pull request #22063: [WIP][SPARK-25044][SQL] Address translation of LM...

2018-08-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22063#discussion_r212445066 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala --- @@ -57,8 +59,21 @@ case class ScalaUDF(

[GitHub] spark issue #22209: [SPARK-24415][Core] HistoryServer does not display metri...

2018-08-23 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22209 Just a note the jira I filed for this was actually affecting running jobs as well (https://issues.apache.org/jira/browse/SPARK-24415) I think we have seen a similar issue on the history

[GitHub] spark issue #21698: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-23 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/21698 @jiangxb1987 I am guessing we should close this PR ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22210: [SPARK-25218][Core]Fix potential resource leaks in Trans...

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22210 **[Test build #95184 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95184/testReport)** for PR 22210 at commit

[GitHub] spark issue #22210: [SPARK-25218][Core]Fix potential resource leaks in Trans...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22210 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22210: [SPARK-25218][Core]Fix potential resource leaks i...

2018-08-23 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/22210#discussion_r212443117 --- Diff: common/network-common/src/main/java/org/apache/spark/network/buffer/FileSegmentManagedBuffer.java --- @@ -95,26 +95,24 @@ public ByteBuffer

[GitHub] spark pull request #22210: [SPARK-25218][Core]Fix potential resource leaks i...

2018-08-23 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/22210#discussion_r212443039 --- Diff: common/network-common/src/main/java/org/apache/spark/network/buffer/FileSegmentManagedBuffer.java --- @@ -77,16 +77,16 @@ public ByteBuffer

[GitHub] spark issue #22210: [SPARK-25218][Core]Fix potential resource leaks in Trans...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22210 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22210: [SPARK-25218][Core]Fix potential resource leaks i...

2018-08-23 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/22210 [SPARK-25218][Core]Fix potential resource leaks in TransportServer and SocketAuthHelper ## What changes were proposed in this pull request? Make sure TransportServer and SocketAuthHelper

[GitHub] spark issue #22209: [SPARK-24415][Core] HistoryServer does not display metri...

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22209 **[Test build #95183 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95183/testReport)** for PR 22209 at commit

[GitHub] spark issue #22079: [SPARK-23207][SPARK-22905][SPARK-24564][SPARK-25114][SQL...

2018-08-23 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/22079 @gatorsmile Weird, I don't see it on branch-2.2. Is that a sync issue? --- - To unsubscribe, e-mail:

[GitHub] spark issue #22209: [SPARK-24415][Core] HistoryServer does not display metri...

2018-08-23 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22209 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22203: [SPARK-25029][BUILD][CORE] Janino "Two non-abstract meth...

2018-08-23 Thread rednaxelafx
Github user rednaxelafx commented on the issue: https://github.com/apache/spark/pull/22203 Thanks @srowen ! In that case could we add a few mentions of such test cases that failed into the PR description itself, perhaps a few lines under the "How was this patch tested?" section to

[GitHub] spark issue #22209: [SPARK-24415][Core] HistoryServer does not display metri...

2018-08-23 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22209 PR title should describe the fix, not the problem. The problem is already described on the bug. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-08-23 Thread erikerlandson
Github user erikerlandson commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r212438850 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -225,6 +225,18 @@ private[spark] object

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-08-23 Thread erikerlandson
Github user erikerlandson commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r212423082 --- Diff: docs/running-on-kubernetes.md --- @@ -775,4 +787,183 @@ specific to Spark on Kubernetes. This sets the major Python version of the

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-08-23 Thread erikerlandson
Github user erikerlandson commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r212424515 --- Diff: docs/running-on-kubernetes.md --- @@ -775,4 +787,183 @@ specific to Spark on Kubernetes. This sets the major Python version of the

[GitHub] spark issue #22203: [SPARK-25029][BUILD][CORE] Janino "Two non-abstract meth...

2018-08-23 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22203 Good point @rednaxelafx though I know some existing tests already definitely failed without this change. They're in the JIRA. It's pervasive enough that we'd see a number of failures due to this

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-08-23 Thread erikerlandson
Github user erikerlandson commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r212423941 --- Diff: docs/running-on-kubernetes.md --- @@ -775,4 +787,183 @@ specific to Spark on Kubernetes. This sets the major Python version of the

[GitHub] spark issue #22209: [SPARK-24415][Core] HistoryServer does not display metri...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22209 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22209: [SPARK-24415][Core] HistoryServer does not display metri...

2018-08-23 Thread ankuriitg
Github user ankuriitg commented on the issue: https://github.com/apache/spark/pull/22209 @vanzin @squito --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22190: [SPARK-25188][SQL] Add WriteConfig to v2 write API.

2018-08-23 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/22190 @rxin, @cloud-fan, @jose-torres: this is the update to add `WriteConfig`. There's one failed test that I think is unrelated, so this is ready for you to have a look. This will probably need to be

[GitHub] spark issue #22209: [SPARK-24415][Core] HistoryServer does not display metri...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22209 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22209: [SPARK-24415][Core] HistoryServer does not display metri...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22209 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22209: [SPARK-24415][Core] HistoryServer does not displa...

2018-08-23 Thread ankuriitg
GitHub user ankuriitg opened a pull request: https://github.com/apache/spark/pull/22209 [SPARK-24415][Core] HistoryServer does not display metrics from tasks… … that complete after stage failure ## What changes were proposed in this pull request? The

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22192 **[Test build #95182 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95182/testReport)** for PR 22192 at commit

[GitHub] spark issue #22192: [SPARK-24918][Core] Executor Plugin API

2018-08-23 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22192 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22203: [SPARK-25029][BUILD][CORE] Janino "Two non-abstract meth...

2018-08-23 Thread rednaxelafx
Github user rednaxelafx commented on the issue: https://github.com/apache/spark/pull/22203 I'm wondering, though, either in the same PR or in a separate PR, could we add a test case in Spark that would trigger the "two non-abstract method" error so that we'd show an example scenario

[GitHub] spark issue #22203: [SPARK-25029][BUILD][CORE] Janino "Two non-abstract meth...

2018-08-23 Thread rednaxelafx
Github user rednaxelafx commented on the issue: https://github.com/apache/spark/pull/22203 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21596 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95161/ Test PASSed. ---

[GitHub] spark issue #22190: [SPARK-25188][SQL] Add WriteConfig to v2 write API.

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22190 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21596 **[Test build #95161 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95161/testReport)** for PR 21596 at commit

[GitHub] spark issue #22190: [SPARK-25188][SQL] Add WriteConfig to v2 write API.

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22190 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95174/ Test FAILed. ---

[GitHub] spark issue #22190: [SPARK-25188][SQL] Add WriteConfig to v2 write API.

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22190 **[Test build #95174 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95174/testReport)** for PR 22190 at commit

[GitHub] spark issue #22193: [SPARK-25186][SQL] Remove v2 save mode.

2018-08-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22193 **[Test build #95181 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95181/testReport)** for PR 22193 at commit

[GitHub] spark issue #22207: [SPARK-25214][SS]Fix the issue that Kafka v2 source may ...

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22207 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95179/ Test PASSed. ---

[GitHub] spark issue #22193: [SPARK-25186][SQL] Remove v2 save mode.

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22193 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22193: [SPARK-25186][SQL] Remove v2 save mode.

2018-08-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22193 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

<    1   2   3   4   5   6   >