[GitHub] spark issue #21044: [SPARK-9312][ML] Add RawPrediction, numClasses, and numF...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21044 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21044: [SPARK-9312][ML] Add RawPrediction, numClasses, and numF...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21044 **[Test build #89216 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89216/testReport)** for PR 21044 at commit [`0cfc20a`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #21044: [SPARK-9312][ML] Add RawPrediction, numClasses, and numF...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21044 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89216/ Test PASSed. ---

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20959 **[Test build #89220 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89220/testReport)** for PR 20959 at commit [`1427f73`](https://github.com/apache/spark/commit/14

[GitHub] spark pull request #21046: Fixing the BUG : SPARK-23955 Typo in classificati...

2018-04-11 Thread codeforfun15
GitHub user codeforfun15 opened a pull request: https://github.com/apache/spark/pull/21046 Fixing the BUG : SPARK-23955 Typo in classification.py ## What changes were proposed in this pull request? Fix for Jira -SPARK-23955 typo in parameter name 'rawPredicition'

[GitHub] spark pull request #21024: [SPARK-23917][SQL] Add array_max function

2018-04-11 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21024#discussion_r180892505 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -287,3 +287,70 @@ case class ArrayContai

[GitHub] spark issue #21046: [SPARK-23955][PYSPARK]Typo in classification.py

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21046 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21046: Fixing the BUG : SPARK-23955 Typo in classification.py

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21046 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21025: [SPARK-23918][SQL] Add array_min function

2018-04-11 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21025#discussion_r180894697 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -287,3 +287,70 @@ case class ArrayContai

[GitHub] spark issue #20871: [SPARK-23762][SQL] UTF8StringBuffer uses MemoryBlock

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20871 **[Test build #89211 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89211/testReport)** for PR 20871 at commit [`a7665fd`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #20871: [SPARK-23762][SQL] UTF8StringBuffer uses MemoryBlock

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20871 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20871: [SPARK-23762][SQL] UTF8StringBuffer uses MemoryBlock

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20871 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89211/ Test PASSed. ---

[GitHub] spark issue #21023: [SPARK-23949] makes && supports the function of predicat...

2018-04-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21023 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark issue #21046: [SPARK-23955][PYSPARK]Typo in classification.py

2018-04-11 Thread codeforfun15
Github user codeforfun15 commented on the issue: https://github.com/apache/spark/pull/21046 @chunshengji Can you review please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional command

[GitHub] spark issue #21004: [SPARK-23896][SQL]Improve PartitioningAwareFileIndex

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21004 **[Test build #89215 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89215/testReport)** for PR 21004 at commit [`d12efab`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #21004: [SPARK-23896][SQL]Improve PartitioningAwareFileIndex

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21004 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21004: [SPARK-23896][SQL]Improve PartitioningAwareFileIndex

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21004 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89215/ Test FAILed. ---

[GitHub] spark issue #21023: [SPARK-23949] makes && supports the function of predicat...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21023 **[Test build #89221 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89221/testReport)** for PR 21023 at commit [`d76d1b8`](https://github.com/apache/spark/commit/d7

[GitHub] spark pull request #21047: [SPARK-23956][YARN] Use effective RPC port in AM ...

2018-04-11 Thread gerashegalov
GitHub user gerashegalov opened a pull request: https://github.com/apache/spark/pull/21047 [SPARK-23956][YARN] Use effective RPC port in AM registration ## What changes were proposed in this pull request? We propose not to hard-code the RPC port in the AM registration.

[GitHub] spark issue #21047: [SPARK-23956][YARN] Use effective RPC port in AM registr...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21047 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21047: [SPARK-23956][YARN] Use effective RPC port in AM registr...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21047 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21047: [SPARK-23956][YARN] Use effective RPC port in AM registr...

2018-04-11 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21047 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.

[GitHub] spark issue #21047: [SPARK-23956][YARN] Use effective RPC port in AM registr...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21047 **[Test build #89222 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89222/testReport)** for PR 21047 at commit [`cc67ccb`](https://github.com/apache/spark/commit/cc

[GitHub] spark issue #20874: [SPARK-23763][SQL] OffHeapColumnVector uses MemoryBlock

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20874 **[Test build #89214 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89214/testReport)** for PR 20874 at commit [`9ef19df`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #20874: [SPARK-23763][SQL] OffHeapColumnVector uses MemoryBlock

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20874 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89214/ Test FAILed. ---

[GitHub] spark issue #20874: [SPARK-23763][SQL] OffHeapColumnVector uses MemoryBlock

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20874 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21047: [SPARK-23956][YARN] Use effective RPC port in AM registr...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21047 **[Test build #89222 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89222/testReport)** for PR 21047 at commit [`cc67ccb`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #21047: [SPARK-23956][YARN] Use effective RPC port in AM registr...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21047 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89222/ Test PASSed. ---

[GitHub] spark issue #21047: [SPARK-23956][YARN] Use effective RPC port in AM registr...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21047 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20959 **[Test build #89220 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89220/testReport)** for PR 20959 at commit [`1427f73`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20959 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20959: [SPARK-23846][SQL] The samplingRatio option for CSV data...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20959 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89220/ Test FAILed. ---

[GitHub] spark issue #21044: [SPARK-9312][ML] Add RawPrediction, numClasses, and numF...

2018-04-11 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/21044 Thanks for the PR! Quick high-level comment: We'll need to have rawPredictionCol be optional. If it's not set or is an empty string, then it should not be added to the output DataFrame. ---

[GitHub] spark pull request #21044: [SPARK-9312][ML] Add RawPrediction, numClasses, a...

2018-04-11 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/21044#discussion_r180920806 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/OneVsRest.scala --- @@ -195,14 +205,18 @@ final class OneVsRestModel private[ml] (

[GitHub] spark issue #21030: typo rawPredicition changed to rawPrediction

2018-04-11 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/21030 Actually, forget the JIRA; I'll just merge it with master and branch-2.3 as is. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #21030: typo rawPredicition changed to rawPrediction

2018-04-11 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21030 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21034 **[Test build #89219 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89219/testReport)** for PR 21034 at commit [`638a0ff`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21034 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89219/ Test FAILed. ---

[GitHub] spark issue #21034: [SPARK-23926][SQL] Extending reverse function to support...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21034 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21037: [SPARK-23919][SQL] Add array_position function

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21037 **[Test build #89218 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89218/testReport)** for PR 21037 at commit [`ce5df02`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #21048: [SPARK-23966][SS] Refactoring all checkpoint file writin...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21048 **[Test build #89223 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89223/testReport)** for PR 21048 at commit [`df7b339`](https://github.com/apache/spark/commit/df

[GitHub] spark issue #21037: [SPARK-23919][SQL] Add array_position function

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21037 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89218/ Test FAILed. ---

[GitHub] spark issue #21037: [SPARK-23919][SQL] Add array_position function

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21037 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21048: [SPARK-23966][SS] Refactoring all checkpoint file writin...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21048 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2229/ Tes

[GitHub] spark issue #21048: [SPARK-23966][SS] Refactoring all checkpoint file writin...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21048 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21048: [SPARK-23966][SS] Refactoring all checkpoint file...

2018-04-11 Thread tdas
GitHub user tdas opened a pull request: https://github.com/apache/spark/pull/21048 [SPARK-23966][SS] Refactoring all checkpoint file writing logic in a common CheckpointFileManager interface ## What changes were proposed in this pull request? Checkpoint files (offset log fi

[GitHub] spark issue #21004: [SPARK-23896][SQL]Improve PartitioningAwareFileIndex

2018-04-11 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/21004 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #21004: [SPARK-23896][SQL]Improve PartitioningAwareFileIndex

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21004 **[Test build #89224 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89224/testReport)** for PR 21004 at commit [`d12efab`](https://github.com/apache/spark/commit/d1

[GitHub] spark issue #21033: [SPARK-19320][MESOS][WIP]allow specifying a hard limit o...

2018-04-11 Thread susanxhuynh
Github user susanxhuynh commented on the issue: https://github.com/apache/spark/pull/21033 @yanji84 Thanks for the patch. I tested your previous PR on GPUs running on DC/OS and everything worked fine. Would you mind updating the documentation as well - https://github.com/apache/spark/

[GitHub] spark issue #21004: [SPARK-23896][SQL]Improve PartitioningAwareFileIndex

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21004 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2230/ Tes

[GitHub] spark issue #21004: [SPARK-23896][SQL]Improve PartitioningAwareFileIndex

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21004 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21031: [SPARK-23923][SQL] Add cardinality function

2018-04-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21031#discussion_r180929145 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -34,28 +76,38 @@ import org.apache.spark.s

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-04-11 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20345 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20345 **[Test build #89225 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89225/testReport)** for PR 20345 at commit [`94d9171`](https://github.com/apache/spark/commit/94

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2231/ Tes

[GitHub] spark issue #20345: [SPARK-23172][SQL] Expand the ReorderJoin rule to handle...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20345 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21048: [SPARK-23966][SS] Refactoring all checkpoint file writin...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21048 **[Test build #89226 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89226/testReport)** for PR 21048 at commit [`f1fc175`](https://github.com/apache/spark/commit/f1

[GitHub] spark pull request #20828: [SPARK-23687][SS] Add a memory source for continu...

2018-04-11 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20828#discussion_r180933380 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -53,32 +53,24 @@ class ContinuousSuiteBase extends St

[GitHub] spark issue #21039: [SPARK-23960][SQL][MINOR] Mark HashAggregateExec.bufVars...

2018-04-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21039 Reverted the PR due to the test failure: https://amplab.cs.berkeley.edu/jenkins/job/spark-master-test-maven-hadoop-2.7/4694/testReport/junit/org.apache.spark.sql/TPCDSQuerySuite/q61/ Plea

[GitHub] spark issue #21048: [SPARK-23966][SS] Refactoring all checkpoint file writin...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21048 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2232/ Tes

[GitHub] spark issue #21048: [SPARK-23966][SS] Refactoring all checkpoint file writin...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21048 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21048: [SPARK-23966][SS] Refactoring all checkpoint file writin...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21048 **[Test build #89227 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89227/testReport)** for PR 21048 at commit [`f9965f1`](https://github.com/apache/spark/commit/f9

[GitHub] spark issue #21048: [SPARK-23966][SS] Refactoring all checkpoint file writin...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21048 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21048: [SPARK-23966][SS] Refactoring all checkpoint file writin...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21048 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2233/ Tes

[GitHub] spark issue #21039: [SPARK-23960][SQL][MINOR] Mark HashAggregateExec.bufVars...

2018-04-11 Thread rednaxelafx
Github user rednaxelafx commented on the issue: https://github.com/apache/spark/pull/21039 Thanks for reverting it for me. The test failure was definitely related to the explicit nulling from this PR, but I can't see how that's possible yet. First of all, in the build that fir

[GitHub] spark issue #21037: [SPARK-23919][SQL] Add array_position function

2018-04-11 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/21037 Is array_position a string function? The function array_contains, for example, works on arrays. Also, this new function *seems* very similar as the existing instr ("Locate the position

[GitHub] spark pull request #20998: [SPARK-23888][CORE] speculative task should not r...

2018-04-11 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/20998#discussion_r180937626 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -880,8 +880,8 @@ class TaskSetManagerSuite extends SparkFunSuite w

[GitHub] spark issue #21023: [SPARK-23949] makes && supports the function of predicat...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21023 **[Test build #89221 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89221/testReport)** for PR 21023 at commit [`d76d1b8`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #21023: [SPARK-23949] makes && supports the function of predicat...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21023 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21023: [SPARK-23949] makes && supports the function of predicat...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21023 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89221/ Test PASSed. ---

[GitHub] spark issue #20998: [SPARK-23888][CORE] speculative task should not run on a...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20998 **[Test build #89228 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89228/testReport)** for PR 20998 at commit [`5901728`](https://github.com/apache/spark/commit/59

[GitHub] spark pull request #21048: [SPARK-23966][SS] Refactoring all checkpoint file...

2018-04-11 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/21048#discussion_r180937241 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CheckpointFileManager.scala --- @@ -0,0 +1,344 @@ +/* + * Licensed to the

[GitHub] spark pull request #21048: [SPARK-23966][SS] Refactoring all checkpoint file...

2018-04-11 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/21048#discussion_r180936292 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CheckpointFileManager.scala --- @@ -0,0 +1,344 @@ +/* + * Licensed to the

[GitHub] spark pull request #21048: [SPARK-23966][SS] Refactoring all checkpoint file...

2018-04-11 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/21048#discussion_r180938649 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/CheckpointFileManagerSuite.scala --- @@ -0,0 +1,188 @@ +/* + * Licensed t

[GitHub] spark pull request #21048: [SPARK-23966][SS] Refactoring all checkpoint file...

2018-04-11 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/21048#discussion_r180938989 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreSuite.scala --- @@ -471,6 +470,41 @@ class StateStoreSuite extend

[GitHub] spark pull request #21048: [SPARK-23966][SS] Refactoring all checkpoint file...

2018-04-11 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/21048#discussion_r180935752 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CheckpointFileManager.scala --- @@ -0,0 +1,344 @@ +/* + * Licensed to the

[GitHub] spark pull request #20828: [SPARK-23687][SS] Add a memory source for continu...

2018-04-11 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20828#discussion_r180939125 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala --- @@ -53,32 +53,24 @@ class ContinuousSuiteBase ext

[GitHub] spark pull request #21048: [SPARK-23966][SS] Refactoring all checkpoint file...

2018-04-11 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/21048#discussion_r180936744 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CheckpointFileManager.scala --- @@ -0,0 +1,344 @@ +/* + * Licensed to the

[GitHub] spark pull request #21048: [SPARK-23966][SS] Refactoring all checkpoint file...

2018-04-11 Thread brkyvz
Github user brkyvz commented on a diff in the pull request: https://github.com/apache/spark/pull/21048#discussion_r180938118 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CheckpointFileManager.scala --- @@ -0,0 +1,344 @@ +/* + * Licensed to the

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka partition revoked issue

2018-04-11 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21038 Thanks @koeninger for your comments. I think your suggestion is valid, the log here is just pasted from JIRA, but we also got the same issue from customer's report. Here in the PR descri

[GitHub] spark pull request #20959: [SPARK-23846][SQL] The samplingRatio option for C...

2018-04-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20959#discussion_r180940274 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -528,6 +529,7 @@ class DataFrameReader private[sql](sparkSession: S

[GitHub] spark pull request #21048: [SPARK-23966][SS] Refactoring all checkpoint file...

2018-04-11 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21048#discussion_r180940389 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/CheckpointFileManagerSuite.scala --- @@ -0,0 +1,188 @@ +/* + * Licensed to

[GitHub] spark pull request #20888: [SPARK-23775][TEST] Make DataFrameRangeSuite not ...

2018-04-11 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20888#discussion_r180940952 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameRangeSuite.scala --- @@ -152,22 +154,28 @@ class DataFrameRangeSuite extends QueryTest with

[GitHub] spark pull request #21048: [SPARK-23966][SS] Refactoring all checkpoint file...

2018-04-11 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21048#discussion_r180940932 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CheckpointFileManager.scala --- @@ -0,0 +1,344 @@ +/* + * Licensed to the A

[GitHub] spark pull request #21048: [SPARK-23966][SS] Refactoring all checkpoint file...

2018-04-11 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21048#discussion_r180940991 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CheckpointFileManager.scala --- @@ -0,0 +1,344 @@ +/* + * Licensed to the A

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka partition revoked issue

2018-04-11 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/21038 I can't think of a valid reason to create a configuration to allow it. It just fundamentally doesn't make sense to run different apps with the same group id. Trying to catch and re

[GitHub] spark issue #21048: [SPARK-23966][SS] Refactoring all checkpoint file writin...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21048 **[Test build #89223 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89223/testReport)** for PR 21048 at commit [`df7b339`](https://github.com/apache/spark/commit/d

[GitHub] spark pull request #20959: [SPARK-23846][SQL] The samplingRatio option for C...

2018-04-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20959#discussion_r180942647 --- Diff: python/pyspark/sql/tests.py --- @@ -3009,6 +3009,15 @@ def test_sort_with_nulls_order(self): df.select(df.name).orderBy(funct

[GitHub] spark issue #21048: [SPARK-23966][SS] Refactoring all checkpoint file writin...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21048 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89223/ Test FAILed. ---

[GitHub] spark issue #21048: [SPARK-23966][SS] Refactoring all checkpoint file writin...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21048 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20959: [SPARK-23846][SQL] The samplingRatio option for C...

2018-04-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20959#discussion_r180942841 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1279,4 +1278,68 @@ class CSVSuite extends Query

[GitHub] spark pull request #20959: [SPARK-23846][SQL] The samplingRatio option for C...

2018-04-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20959#discussion_r180942904 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1279,4 +1278,68 @@ class CSVSuite extends Query

[GitHub] spark pull request #20959: [SPARK-23846][SQL] The samplingRatio option for C...

2018-04-11 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20959#discussion_r180943744 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1279,4 +1278,68 @@ class CSVSuite extends Query

[GitHub] spark issue #21038: [SPARK-22968][DStream] Fix Kafka partition revoked issue

2018-04-11 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21038 Thanks @koeninger , then I will just improve the exception message. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apa

[GitHub] spark issue #21009: [SPARK-23905][SQL] Add UDF weekday

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21009 **[Test build #89229 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89229/testReport)** for PR 21009 at commit [`363ff1c`](https://github.com/apache/spark/commit/36

[GitHub] spark issue #21038: [SPARK-22968][DStream] Throw an exception on partition r...

2018-04-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21038 **[Test build #89230 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89230/testReport)** for PR 21038 at commit [`a7770e9`](https://github.com/apache/spark/commit/a7

[GitHub] spark issue #21038: [SPARK-22968][DStream] Throw an exception on partition r...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21038 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21038: [SPARK-22968][DStream] Throw an exception on partition r...

2018-04-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21038 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2234/ Tes

[GitHub] spark issue #20874: [SPARK-23763][SQL] OffHeapColumnVector uses MemoryBlock

2018-04-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20874 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #21037: [SPARK-23919][SQL] Add array_position function

2018-04-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21037 @bersprockets You are absolutely right. I made mistake. I will completely reimplement this soon. --- - To unsubscribe, e-mail: rev

<    1   2   3   4   5   6   >