[GitHub] spark pull request #18932: Add reduceVertices method to GraphOps

2017-08-13 Thread gilcu2
GitHub user gilcu2 opened a pull request: https://github.com/apache/spark/pull/18932 Add reduceVertices method to GraphOps ## What changes were proposed in this pull request? Add reduceVertices method to GraphOps class in graphX ## How was this patch tested?

[GitHub] spark issue #18930: [SPARK-21677][SQL] json_tuple throws NullPointException ...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18930 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18920: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-13 Thread DonnyZone
Github user DonnyZone commented on the issue: https://github.com/apache/spark/pull/18920 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18932: Add reduceVertices method to GraphOps

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18932 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80594/ Test FAILed. ---

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #80594 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80594/testReport)** for PR 18931 at commit

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18648: [SPARK-21428] Turn IsolatedClientLoader off while using ...

2017-08-13 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18648 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18920: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18920 **[Test build #80591 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80591/testReport)** for PR 18920 at commit

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18914 **[Test build #80588 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80588/testReport)** for PR 18914 at commit

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #80590 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80590/testReport)** for PR 18931 at commit

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80590/ Test FAILed. ---

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18931 Ran with the same benchmark in SPARK-21603. Before this patch: After this patch: Java HotSpot(TM) 64-Bit Server VM 1.8.0_102-b14 on Linux 4.9.36-moby

[GitHub] spark pull request #18931: [SPARK-21717][SQL][WIP] Decouple consume function...

2017-08-13 Thread viirya
GitHub user viirya reopened a pull request: https://github.com/apache/spark/pull/18931 [SPARK-21717][SQL][WIP] Decouple consume functions of physical operators in whole-stage codegen ## What changes were proposed in this pull request? It has been observed in SPARK-21603

[GitHub] spark pull request #18931: [SPARK-21717][SQL][WIP] Decouple consume function...

2017-08-13 Thread viirya
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/18931 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18914 **[Test build #80592 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80592/testReport)** for PR 18914 at commit

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18914 @heary-cao I already understand what you want to do. It makes sense to me to remove `sql("UNCACHE TABLE testData")`. My question is about `JoinSuite.afterEach()`. Do you want to call

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18914 **[Test build #80588 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80588/testReport)** for PR 18914 at commit

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18914 **[Test build #80593 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80593/testReport)** for PR 18914 at commit

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #80594 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80594/testReport)** for PR 18931 at commit

[GitHub] spark issue #18916: [SPARK-21705][CORE][DOC]Add spark.internal.config parame...

2017-08-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18916 I am -0 by the same reason above ^. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18875 **[Test build #80596 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80596/testReport)** for PR 18875 at commit

[GitHub] spark pull request #18926: [SPARK-21712] [PySpark] Clarify type error for Co...

2017-08-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18926#discussion_r132847782 --- Diff: python/pyspark/sql/column.py --- @@ -406,7 +406,13 @@ def substr(self, startPos, length): [Row(col=u'Ali'), Row(col=u'Bob')]

[GitHub] spark pull request #18930: [SPARK-21677][SQL] json_tuple throws NullPointExc...

2017-08-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18930#discussion_r132846456 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -361,10 +361,18 @@ case class

[GitHub] spark pull request #18920: [SPARK-19471][SQL]AggregationIterator does not in...

2017-08-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18920#discussion_r132847406 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -449,6 +449,28 @@ class DataFrameFunctionsSuite extends

[GitHub] spark pull request #18914: [MINOR][SQL][TEST]no uncache table in joinsuite t...

2017-08-13 Thread heary-cao
Github user heary-cao commented on a diff in the pull request: https://github.com/apache/spark/pull/18914#discussion_r132837521 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala --- @@ -30,6 +30,15 @@ import org.apache.spark.TestUtils.{assertNotSpilled,

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18914 @heary-cao Why do we need to call `spark.sharedState.cacheManager.clearCache()` in `JoinSuite` while we have already call `clearCache()` in `SharedSQLContext`? --- If your project is set up for it,

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18875 **[Test build #80585 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80585/testReport)** for PR 18875 at commit

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #80595 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80595/testReport)** for PR 18931 at commit

[GitHub] spark pull request #18932: Add reduceVertices method to GraphOps

2017-08-13 Thread gilcu2
Github user gilcu2 closed the pull request at: https://github.com/apache/spark/pull/18932 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18931 Ran with the same benchmark in #18810. Before this patch: Java HotSpot(TM) 64-Bit Server VM 1.8.0_102-b14 on Linux 4.9.36-moby Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz

[GitHub] spark issue #18932: Add reduceVertices method to GraphOps

2017-08-13 Thread gilcu2
Github user gilcu2 commented on the issue: https://github.com/apache/spark/pull/18932 Need a Jira First --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18920: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18920 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80591/ Test PASSed. ---

[GitHub] spark issue #18920: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18920 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80596/ Test PASSed. ---

[GitHub] spark pull request #18931: [SPARK-21717][SQL][WIP] Decouple consume function...

2017-08-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r132837981 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -149,14 +149,65 @@ trait CodegenSupport extends

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/18914 @gatorsmile , @viirya, @kiszk, With all the comments, I have modify it again. please review it again if you have time. thanks. --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #18920: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18920 **[Test build #80591 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80591/testReport)** for PR 18920 at commit

[GitHub] spark issue #18648: [SPARK-21428] Turn IsolatedClientLoader off while using ...

2017-08-13 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/18648 ping @jiangxb1987 @cloud-fan again --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18914 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18914 **[Test build #80593 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80593/testReport)** for PR 18914 at commit

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18914 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80593/ Test PASSed. ---

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #80589 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80589/testReport)** for PR 18931 at commit

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #80589 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80589/testReport)** for PR 18931 at commit

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80589/ Test FAILed. ---

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80585/ Test FAILed. ---

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18875 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18930: [SPARK-21677][SQL] json_tuple throws NullPointException ...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18930 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80587/ Test PASSed. ---

[GitHub] spark issue #18930: [SPARK-21677][SQL] json_tuple throws NullPointException ...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18930 **[Test build #80587 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80587/testReport)** for PR 18930 at commit

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/18914 @kiszk The original reason of this PR was a simple . test("broadcasted hash outer join operator selection") { spark.sharedState.cacheManager.clearCache() sql("CACHE

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/18914 okay,Looks like we just need something simple to remove _uncache table_ . thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #18875: [SPARK-21513][SQL] Allow UDF to_json support converting ...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18875 **[Test build #80596 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80596/testReport)** for PR 18875 at commit

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18652: [SPARK-21497][SQL] Pull non-deterministic equi join keys...

2017-08-13 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18652 Once we pull out them into downstream project, should we still worry about call orders? They are evaluated before sort or shuffle added later. --- If your project is set up for it, you can reply to

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80595/ Test FAILed. ---

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #80595 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80595/testReport)** for PR 18931 at commit

[GitHub] spark issue #18866: [SPARK-21649][SQL] Support writing data into hive bucket...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18866 **[Test build #80597 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80597/testReport)** for PR 18866 at commit

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18914 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80592/ Test PASSed. ---

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18914 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18317: [SPARK-21113][CORE] Read ahead input stream to am...

2017-08-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18317#discussion_r132846768 --- Diff: core/src/main/java/org/apache/spark/io/ReadAheadInputStream.java --- @@ -0,0 +1,288 @@ +/* + * Licensed under the Apache License, Version

[GitHub] spark issue #18866: [SPARK-21649][SQL] Support writing data into hive bucket...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18866 **[Test build #80597 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80597/testReport)** for PR 18866 at commit

[GitHub] spark issue #18866: [SPARK-21649][SQL] Support writing data into hive bucket...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18866 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80597/ Test FAILed. ---

[GitHub] spark issue #18866: [SPARK-21649][SQL] Support writing data into hive bucket...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18866 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18914: [MINOR][SQL][TEST]no uncache table in joinsuite t...

2017-08-13 Thread heary-cao
Github user heary-cao commented on a diff in the pull request: https://github.com/apache/spark/pull/18914#discussion_r132833028 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala --- @@ -30,6 +30,12 @@ import org.apache.spark.TestUtils.{assertNotSpilled,

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #80581 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80581/testReport)** for PR 18931 at commit

[GitHub] spark issue #18920: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18920 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80581/ Test FAILed. ---

[GitHub] spark issue #18930: [SPARK-21677][SQL] json_tuple throws NullPointException ...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18930 **[Test build #80579 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80579/testReport)** for PR 18930 at commit

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18914 **[Test build #80580 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80580/testReport)** for PR 18914 at commit

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18920: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18920 **[Test build #80582 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80582/testReport)** for PR 18920 at commit

[GitHub] spark issue #18920: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18920 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80582/ Test FAILed. ---

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18914 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18931: [SPARK-21717][SQL][WIP] Decouple consume functions of ph...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #80581 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80581/testReport)** for PR 18931 at commit

[GitHub] spark issue #18920: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18920 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18930: [SPARK-21677][SQL] json_tuple throws NullPointException ...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18930 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18930: [SPARK-21677][SQL] json_tuple throws NullPointException ...

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18930 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80579/ Test FAILed. ---

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18914 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80580/ Test FAILed. ---

[GitHub] spark issue #18916: [SPARK-21705][CORE][DOC]Add spark.internal.config parame...

2017-08-13 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/18916 cc @HyukjinKwon @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18914 **[Test build #80580 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80580/testReport)** for PR 18914 at commit

[GitHub] spark pull request #18931: [SPARK-21717][SQL][WIP] Decouple consume function...

2017-08-13 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/18931 [SPARK-21717][SQL][WIP] Decouple consume functions of physical operators in whole-stage codegen ## What changes were proposed in this pull request? It has been observed in SPARK-21603 that

[GitHub] spark issue #18700: [SPARK-21499] [SQL] Support creating persistent function...

2017-08-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18700 cc @cloud-fan @ueshin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18920: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18920 **[Test build #80582 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80582/testReport)** for PR 18920 at commit

[GitHub] spark pull request #18914: [MINOR][SQL][TEST]no uncache table in joinsuite t...

2017-08-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18914#discussion_r132833474 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala --- @@ -30,6 +30,15 @@ import org.apache.spark.TestUtils.{assertNotSpilled,

[GitHub] spark issue #18929: [MINOR][LAUNCHER]remove never used String in SparkLaunch...

2017-08-13 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/18929 @srowen In fact, we can replace spark.executor.memory with _SparkLauncher.EXECUTOR_MEMORY_ and replace spark.executor.cores with _SparkLauncher.EXECUTOR_CORES_. But in the process of

[GitHub] spark issue #18914: [MINOR][SQL][TEST]no uncache table in joinsuite test

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18914 **[Test build #80583 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80583/testReport)** for PR 18914 at commit

[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18899 OK maybe include some of this text in the scaladoc for it, to make it clear it is always intended to be called with the value of `numNonZeroes`. --- If your project is set up for it, you can reply

[GitHub] spark issue #18916: [SPARK-21705][CORE][DOC]Add spark.internal.config parame...

2017-08-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18916 In Spark SQL, the description of all the SQLConf can be displayed through the SQL commands. However, I am not sure how end users can get the descriptions of these Spark Conf. --- If your

[GitHub] spark issue #18692: [SPARK-21417][SQL] Infer join conditions using propagate...

2017-08-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18692 @aokolnychyi Thanks for finding the non-convergent case! Let me see how to fix it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #18926: [SPARK-21712] [PySpark] Clarify type error for Co...

2017-08-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18926#discussion_r132856266 --- Diff: python/pyspark/sql/column.py --- @@ -406,7 +406,13 @@ def substr(self, startPos, length): [Row(col=u'Ali'), Row(col=u'Bob')]

[GitHub] spark issue #18652: [SPARK-21497][SQL] Pull non-deterministic equi join keys...

2017-08-13 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18652 When we join two tables, given there are equi-join keys, and they are non-deterministic, for example `t1.a = rand(t2.b)` and `t1.c = rand(t2.d)`. We pull out them to downstream project:

[GitHub] spark issue #18920: [SPARK-19471][SQL]AggregationIterator does not initializ...

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18920 **[Test build #80599 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80599/testReport)** for PR 18920 at commit

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-13 Thread lvdongr
Github user lvdongr commented on the issue: https://github.com/apache/spark/pull/18756 ok, I will solve the problems left first, and hold this PR @gatorsmile. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #9518: [SPARK-11574][Core] Add metrics StatsD sink

2017-08-13 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/9518 Sorry I don't have the permission to merge this. Ping @cloud-fan @JoshRosen to review again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-13 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/18899 Thanks @sethah @srowen . The comment is added. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18899: [SPARK-21680][ML][MLLIB]optimzie Vector coompress

2017-08-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18899 **[Test build #80600 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80600/testReport)** for PR 18899 at commit

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-08-13 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17980 Retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18909: [MINOR][SQL] Additional test case for CheckCartesianProd...

2017-08-13 Thread aokolnychyi
Github user aokolnychyi commented on the issue: https://github.com/apache/spark/pull/18909 @gatorsmile sure, this PR is only about tests, I was just wondering what is planned regarding cross joins with inequality conditions. I borrowed several tests from PR #16762 and added

  1   2   3   >