[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-12-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/16030 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16129: [SPARK-18678][ML] Skewed feature subsampling in Random f...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16129 **[Test build #69618 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69618/consoleFull)** for PR 16129 at commit

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-12-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/16030 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-12-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/16030 The failure seems to be not related to this pr? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16129: [SPARK-18678][ML] Skewed feature subsampling in Random f...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16129 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16129: [SPARK-18678][ML] Skewed feature subsampling in Random f...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16129 **[Test build #69618 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69618/consoleFull)** for PR 16129 at commit

[GitHub] spark issue #16129: [SPARK-18678][ML] Skewed feature subsampling in Random f...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16129 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69618/ Test FAILed. ---

[GitHub] spark issue #16120: [SPARK-18634][PySpark][SQL][WIP] Corruption and Correctn...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16120 **[Test build #69615 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69615/consoleFull)** for PR 16120 at commit

[GitHub] spark issue #16120: [SPARK-18634][PySpark][SQL][WIP] Corruption and Correctn...

2016-12-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16120 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #16102: [SPARK-18586][BUILD] netty-3.8.0.Final.jar has vu...

2016-12-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16102 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16037: [SPARK-18471][MLLIB] In LBFGS, avoid sending huge vector...

2016-12-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16037 Yes I'm pretty OK with merging this. If you can dig up any results, that's all the better. Will check in with you next week. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13909 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69616/ Test PASSed. ---

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16030 **[Test build #69617 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69617/consoleFull)** for PR 16030 at commit

[GitHub] spark issue #16114: [SPARK-18620][Streaming][Kinesis] Flatten input rates in...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16114 **[Test build #69620 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69620/consoleFull)** for PR 16114 at commit

[GitHub] spark issue #16098: [SPARK-18672][CORE] Close recordwriter in SparkHadoopMap...

2016-12-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16098 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16116: [SPARK-18685][TESTS] Fix URI and release resource...

2016-12-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16116 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16116: [SPARK-18685][TESTS] Fix URI and release resources after...

2016-12-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16116 Thank you !! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16030 **[Test build #69617 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69617/consoleFull)** for PR 16030 at commit

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13909 **[Test build #69616 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69616/consoleFull)** for PR 13909 at commit

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13909 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13909 **[Test build #69616 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69616/consoleFull)** for PR 13909 at commit

[GitHub] spark issue #16098: [SPARK-18672][CORE] Close recordwriter in SparkHadoopMap...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16098 **[Test build #69619 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69619/consoleFull)** for PR 16098 at commit

[GitHub] spark issue #16120: [SPARK-18634][PySpark][SQL][WIP] Corruption and Correctn...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16120 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16031: [SPARK-18606][HISTORYSERVER]remove useless elemen...

2016-12-03 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16031#discussion_r90754812 --- Diff: core/src/main/resources/org/apache/spark/ui/static/historypage.js --- @@ -78,6 +78,12 @@ jQuery.extend( jQuery.fn.dataTableExt.oSort, { }

[GitHub] spark pull request #16103: [SPARK-18374][ML]Incorrect words in StopWords/eng...

2016-12-03 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16103#discussion_r90754782 --- Diff: mllib/src/main/resources/org/apache/spark/ml/feature/stopwords/english.txt --- @@ -149,5 +149,58 @@ shan shouldn wasn weren -won

[GitHub] spark pull request #16114: [SPARK-18620][Streaming][Kinesis] Flatten input r...

2016-12-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/16114#discussion_r90754922 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala --- @@ -56,6 +56,31 @@ private[kinesis] class

[GitHub] spark pull request #16069: [SPARK-18638][BUILD] Upgrade sbt, Zinc, and Maven...

2016-12-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16069 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16114: [SPARK-18620][Streaming][Kinesis] Flatten input rates in...

2016-12-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/16114 @srowen Do u know qualified maintainers on this component? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #16114: [SPARK-18620][Streaming][Kinesis] Flatten input rates in...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16114 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69620/ Test PASSed. ---

[GitHub] spark issue #16120: [SPARK-18634][PySpark][SQL][WIP] Corruption and Correctn...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16120 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16098: [SPARK-18672][CORE] Close recordwriter in SparkHadoopMap...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16098 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69614/ Test FAILed. ---

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16030 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16030 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69611/ Test FAILed. ---

[GitHub] spark issue #16120: [SPARK-18634][PySpark][SQL][WIP] Corruption and Correctn...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16120 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69610/ Test FAILed. ---

[GitHub] spark issue #16098: [SPARK-18672][CORE] Close recordwriter in SparkHadoopMap...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16098 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16129: [SPARK-18678][ML] Skewed feature subsampling in R...

2016-12-03 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/16129 [SPARK-18678][ML] Skewed feature subsampling in Random forest ## What changes were proposed in this pull request? Fix reservoir sampling bias for small k. An off-by-one error meant that the

[GitHub] spark issue #16102: [SPARK-18586][BUILD] netty-3.8.0.Final.jar has vulnerabi...

2016-12-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16102 Merged to master, though as I say I don't think the CVE actually impacted Spark to begin with. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16030 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16030 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69617/ Test FAILed. ---

[GitHub] spark issue #12004: [SPARK-7481] [build] Add spark-cloud module to pull in o...

2016-12-03 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/12004 Test failure due to new artifacts ``` +++ b/dev/pr-deps/spark-deps-hadoop-2.7 @@ -16,8 +16,6 @@ arpack_combined_all-0.1.jar avro-1.7.7.jar avro-ipc-1.7.7.jar

[GitHub] spark pull request #11105: [SPARK-12469][CORE] Data Property accumulators fo...

2016-12-03 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/11105#discussion_r90755993 --- Diff: core/src/test/scala/org/apache/spark/DataPropertyAccumulatorSuite.scala --- @@ -0,0 +1,395 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-12-03 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/11105 I'm down the idea of having add and merge not be final with huge warning signs and we could switch it up in 3.X to be final. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-12-03 Thread eyalfa
Github user eyalfa commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r90752975 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypes.scala --- @@ -0,0 +1,78 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #16116: [SPARK-18685][TESTS] Fix URI and release resources after...

2016-12-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16116 Merged to master/2.1/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16120: [SPARK-18634][PySpark][SQL][WIP] Corruption and Correctn...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16120 **[Test build #69615 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69615/consoleFull)** for PR 16120 at commit

[GitHub] spark pull request #16114: [SPARK-18620][Streaming][Kinesis] Flatten input r...

2016-12-03 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16114#discussion_r90754731 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala --- @@ -56,6 +56,31 @@ private[kinesis] class

[GitHub] spark issue #16120: [SPARK-18634][PySpark][SQL][WIP] Corruption and Correctn...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16120 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69615/ Test PASSed. ---

[GitHub] spark issue #16069: [SPARK-18638][BUILD] Upgrade sbt, Zinc, and Maven plugin...

2016-12-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16069 Merged to master. It's a build change and probably fine for 2.1 but it's non-trivial. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16030 **[Test build #69621 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69621/consoleFull)** for PR 16030 at commit

[GitHub] spark issue #16114: [SPARK-18620][Streaming][Kinesis] Flatten input rates in...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16114 **[Test build #69620 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69620/consoleFull)** for PR 16114 at commit

[GitHub] spark issue #16114: [SPARK-18620][Streaming][Kinesis] Flatten input rates in...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16114 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16122: [SPARK-18681][SQL] Fix filtering to compatible with part...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16122 **[Test build #69622 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69622/consoleFull)** for PR 16122 at commit

[GitHub] spark issue #15995: [SPARK-18566][SQL] remove OverwriteOptions

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15995 **[Test build #69623 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69623/consoleFull)** for PR 15995 at commit

[GitHub] spark pull request #16068: [SPARK-18637][SQL]Stateful UDF should be consider...

2016-12-03 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16068#discussion_r90756326 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFs.scala --- @@ -144,7 +144,7 @@ private[hive] case class HiveGenericUDF(

[GitHub] spark pull request #16068: [SPARK-18637][SQL]Stateful UDF should be consider...

2016-12-03 Thread zhzhan
Github user zhzhan commented on a diff in the pull request: https://github.com/apache/spark/pull/16068#discussion_r90763121 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveUDFSuite.scala --- @@ -487,6 +488,29 @@ class HiveUDFSuite extends QueryTest with

[GitHub] spark issue #16068: [SPARK-18637][SQL]Stateful UDF should be considered as n...

2016-12-03 Thread zhzhan
Github user zhzhan commented on the issue: https://github.com/apache/spark/pull/16068 @gatorsmile we cannot use deterministic = true/false, as there are existing udf with deterministic as true, but stateful as true as well. --- If your project is set up for it, you can reply to

[GitHub] spark issue #16129: [SPARK-18678][ML] Skewed feature subsampling in Random f...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16129 **[Test build #3467 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3467/consoleFull)** for PR 16129 at commit

[GitHub] spark pull request #16046: [SPARK-18582][SQL] Whitelist LogicalPlan operator...

2016-12-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16046 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16130: Update location of Spark YARN shuffle jar

2016-12-03 Thread nchammas
GitHub user nchammas opened a pull request: https://github.com/apache/spark/pull/16130 Update location of Spark YARN shuffle jar Looking at the distributions provided on spark.apache.org, I see that the Spark YARN shuffle jar is under `yarn/` and not `lib/`. You can merge this

[GitHub] spark issue #16130: Update location of Spark YARN shuffle jar

2016-12-03 Thread nchammas
Github user nchammas commented on the issue: https://github.com/apache/spark/pull/16130 cc @vanzin? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #16119: [SPARK-18687][Pyspark][SQL]Backward compatibility - crea...

2016-12-03 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16119 Since the current tests pass without this change I'd say that we should add a test for the behaviour we are planning to support that isn't currently supported (would also make the purpose of the

[GitHub] spark issue #16130: Update location of Spark YARN shuffle jar

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16130 **[Test build #69628 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69628/consoleFull)** for PR 16130 at commit

[GitHub] spark issue #16068: [SPARK-18637][SQL]Stateful UDF should be considered as n...

2016-12-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16068 https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L1373-L1378 Copied the code from Hive `FunctionRegistry.java`: ```JAVA

[GitHub] spark issue #16130: Update location of Spark YARN shuffle jar

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16130 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69628/ Test PASSed. ---

[GitHub] spark issue #16130: Update location of Spark YARN shuffle jar

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16130 **[Test build #69628 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69628/consoleFull)** for PR 16130 at commit

[GitHub] spark issue #16130: Update location of Spark YARN shuffle jar

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16130 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16103: [SPARK-18374][ML]Incorrect words in StopWords/eng...

2016-12-03 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/16103#discussion_r90765451 --- Diff: mllib/src/main/resources/org/apache/spark/ml/feature/stopwords/english.txt --- @@ -149,5 +149,58 @@ shan shouldn wasn weren -won

[GitHub] spark issue #16068: [SPARK-18637][SQL]Stateful UDF should be considered as n...

2016-12-03 Thread zhzhan
Github user zhzhan commented on the issue: https://github.com/apache/spark/pull/16068 My understanding is that the non-deterministic udf does not need to be stageful, but a stateful udf has to be non-deterministic. Here is the comments in hive regarding this property

[GitHub] spark issue #16068: [SPARK-18637][SQL]Stateful UDF should be considered as n...

2016-12-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16068 Could we directly use `@UDFType(deterministic = true/false)`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #16068: [SPARK-18637][SQL]Stateful UDF should be considered as n...

2016-12-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16068 Found the link: [HIVE-1994: Support new annotation @UDFType(stateful = true)](https://issues.apache.org/jira/browse/HIVE-1994 ) --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #16129: [SPARK-18678][ML] Skewed feature subsampling in Random f...

2016-12-03 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16129 @felixcheung maybe you can advise me on this. I think this is a correct fix, but ends up changing the results of decision forests a little bit. The SparkR test you wrote fails: ```

[GitHub] spark issue #16046: [SPARK-18582][SQL] Whitelist LogicalPlan operators allow...

2016-12-03 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16046 Merging to master/2.1/2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16094: [SPARK-18541][Python]Add metadata parameter to py...

2016-12-03 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/16094#discussion_r90764328 --- Diff: python/pyspark/sql/column.py --- @@ -298,19 +299,34 @@ def isin(self, *cols): isNotNull = _unary_op("isNotNull", "True if the current

[GitHub] spark issue #16121: [SPARK-16589][PYTHON] Chained cartesian produces incorre...

2016-12-03 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/16121 I was hesistant with the previous PR since it seemed like we didn't fully understand why we were changing what we were at the time, I can try and take a closer look at this over the next few days

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16030 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16030 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69621/ Test PASSed. ---

[GitHub] spark issue #15995: [SPARK-18566][SQL] remove OverwriteOptions

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15995 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69623/ Test PASSed. ---

[GitHub] spark issue #16114: [SPARK-18620][Streaming][Kinesis] Flatten input rates in...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16114 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16114: [SPARK-18620][Streaming][Kinesis] Flatten input rates in...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16114 **[Test build #69627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69627/consoleFull)** for PR 16114 at commit

[GitHub] spark issue #16122: [SPARK-18681][SQL] Fix filtering to compatible with part...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16122 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69625/ Test FAILed. ---

[GitHub] spark issue #16122: [SPARK-18681][SQL] Fix filtering to compatible with part...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16122 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16098: [SPARK-18672][CORE] Close recordwriter in SparkHadoopMap...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16098 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69619/ Test PASSed. ---

[GitHub] spark issue #16098: [SPARK-18672][CORE] Close recordwriter in SparkHadoopMap...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16098 **[Test build #69619 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69619/consoleFull)** for PR 16098 at commit

[GitHub] spark issue #16098: [SPARK-18672][CORE] Close recordwriter in SparkHadoopMap...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16098 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-12-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r90757729 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypes.scala --- @@ -0,0 +1,78 @@ +/* + * Licensed to the

[GitHub] spark issue #16122: [SPARK-18681][SQL] Fix filtering to compatible with part...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16122 **[Test build #69625 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69625/consoleFull)** for PR 16122 at commit

[GitHub] spark issue #16114: [SPARK-18620][Streaming][Kinesis] Flatten input rates in...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16114 **[Test build #69624 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69624/consoleFull)** for PR 16114 at commit

[GitHub] spark issue #16114: [SPARK-18620][Streaming][Kinesis] Flatten input rates in...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16114 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69624/ Test FAILed. ---

[GitHub] spark issue #16114: [SPARK-18620][Streaming][Kinesis] Flatten input rates in...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16114 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16114: [SPARK-18620][Streaming][Kinesis] Flatten input r...

2016-12-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/16114#discussion_r90758322 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala --- @@ -56,6 +56,27 @@ private[kinesis] class

[GitHub] spark issue #16129: [SPARK-18678][ML] Skewed feature subsampling in Random f...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16129 **[Test build #3466 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3466/consoleFull)** for PR 16129 at commit

[GitHub] spark pull request #16114: [SPARK-18620][Streaming][Kinesis] Flatten input r...

2016-12-03 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16114#discussion_r90756693 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala --- @@ -56,6 +56,27 @@ private[kinesis] class

[GitHub] spark issue #16030: [SPARK-18108][SQL] Fix a bug to fail partition schema in...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16030 **[Test build #69621 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69621/consoleFull)** for PR 16030 at commit

[GitHub] spark issue #16122: [SPARK-18681][SQL] Fix filtering to compatible with part...

2016-12-03 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/16122 This patch fails because hive-0.12 and hive-0.13 doesn't has `getMetaConf` method. see [HIVE-7532](https://issues.apache.org/jira/browse/HIVE-7532), --- If your project is set up for it, you

[GitHub] spark issue #16114: [SPARK-18620][Streaming][Kinesis] Flatten input rates in...

2016-12-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/16114 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16114: [SPARK-18620][Streaming][Kinesis] Flatten input r...

2016-12-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/16114#discussion_r90758182 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala --- @@ -56,6 +56,27 @@ private[kinesis] class

[GitHub] spark issue #15995: [SPARK-18566][SQL] remove OverwriteOptions

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15995 **[Test build #69623 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69623/consoleFull)** for PR 15995 at commit

[GitHub] spark pull request #16114: [SPARK-18620][Streaming][Kinesis] Flatten input r...

2016-12-03 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16114#discussion_r90756702 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisRecordProcessor.scala --- @@ -56,6 +56,27 @@ private[kinesis] class

[GitHub] spark issue #16129: [SPARK-18678][ML] Skewed feature subsampling in Random f...

2016-12-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16129 **[Test build #3466 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3466/consoleFull)** for PR 16129 at commit

[GitHub] spark issue #16122: [SPARK-18681][SQL] Fix filtering to compatible with part...

2016-12-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16122 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69622/ Test FAILed. ---

  1   2   >