[GitHub] spark pull request #18306: [SPARK-21029][SS] All StreamingQuery should be st...

2017-06-14 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/18306#discussion_r122092151 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala --- @@ -690,6 +690,7 @@ class SparkSession private( * @since 2.0.0

[GitHub] spark issue #18301: [SPARK-21052][SQL] Add hash map metrics to join

2017-06-14 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/18301 Can you put a screenshot of the UI up, for both join and aggregate? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #18311: Branch 2.0

2017-06-14 Thread yhqairqq
GitHub user yhqairqq opened a pull request: https://github.com/apache/spark/pull/18311 Branch 2.0 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how this patch

[GitHub] spark issue #18202: [SPARK-20980] [SQL] Rename `wholeFile` to `multiLine` fo...

2017-06-14 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18202 oops - found this https://github.com/apache/spark/blame/ae33abf71b353c638487948b775e966c7127cd46/docs/sql-programming-guide.md#L1001 --- If your project is set up for it, you can reply to

[GitHub] spark issue #18284: [SPARK-21072][SQL] `TreeNode.mapChildren` should only ap...

2017-06-14 Thread ConeyLiu
Github user ConeyLiu commented on the issue: https://github.com/apache/spark/pull/18284 @cloud-fan thanks for reviewing, code has updated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #18306: [SPARK-21029][SS] All StreamingQuery should be st...

2017-06-14 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18306#discussion_r122122675 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryManagerSuite.scala --- @@ -239,6 +237,40 @@ class

[GitHub] spark pull request #18312: [SPARK-20980][DOCS] update doc to reflect multiLi...

2017-06-14 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/18312 [SPARK-20980][DOCS] update doc to reflect multiLine change ## What changes were proposed in this pull request? doc only change ## How was this patch tested? manually

[GitHub] spark issue #18084: [SPARK-19900][core]Remove driver when relaunching.

2017-06-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18084 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17723: [SPARK-20434][YARN][CORE] Move Hadoop delegation ...

2017-06-14 Thread mgummelt
Github user mgummelt commented on a diff in the pull request: https://github.com/apache/spark/pull/17723#discussion_r122096373 --- Diff: docs/running-on-yarn.md --- @@ -504,10 +504,10 @@ spark.yarn.access.hadoopFileSystems hdfs://ireland.example.org:8020/,webhdfs://f ```

[GitHub] spark pull request #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-14 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18300#discussion_r122098055 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1764,6 +1765,56 @@ class Dataset[T] private[sql]( } /**

[GitHub] spark issue #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18300 **[Test build #78073 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78073/testReport)** for PR 18300 at commit

[GitHub] spark issue #18284: [SPARK-21072][SQL] `TreeNode.mapChildren` should only ap...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18284 **[Test build #78075 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78075/testReport)** for PR 18284 at commit

[GitHub] spark issue #18299: [SPARK-21092][SQL] Wire SQLConf in logical plan and expr...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18299 **[Test build #78071 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78071/testReport)** for PR 18299 at commit

[GitHub] spark issue #18299: [SPARK-21092][SQL] Wire SQLConf in logical plan and expr...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18299 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78070/ Test FAILed. ---

[GitHub] spark issue #18299: [SPARK-21092][SQL] Wire SQLConf in logical plan and expr...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18299 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18299: [SPARK-21092][SQL] Wire SQLConf in logical plan and expr...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18299 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78071/ Test FAILed. ---

[GitHub] spark issue #18299: [SPARK-21092][SQL] Wire SQLConf in logical plan and expr...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18299 **[Test build #78070 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78070/testReport)** for PR 18299 at commit

[GitHub] spark issue #18299: [SPARK-21092][SQL] Wire SQLConf in logical plan and expr...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18299 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18309: [SPARK-21079] [SQL] Calculate total size of a par...

2017-06-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/18309#discussion_r122102110 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala --- @@ -81,6 +83,19 @@ case class

[GitHub] spark issue #18302: [SPARK-17237][SQL][FOLLOWUP] Add a qualifier in pretty e...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18302 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18302: [SPARK-17237][SQL][FOLLOWUP] Add a qualifier in pretty e...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18302 **[Test build #78074 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78074/testReport)** for PR 18302 at commit

[GitHub] spark issue #18284: [SPARK-21072][SQL] `TreeNode.mapChildren` should only ap...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18284 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18284: [SPARK-21072][SQL] `TreeNode.mapChildren` should only ap...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18284 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78075/ Test PASSed. ---

[GitHub] spark pull request #18301: [SPARK-21052][SQL] Add hash map metrics to join

2017-06-14 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18301#discussion_r122114390 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala --- @@ -573,8 +586,11 @@ private[execution] final class

[GitHub] spark issue #18290: [SPARK-20989][Core] Fail to start multiple workers on on...

2017-06-14 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18290 @jerryshao Thank you for your suggestion, since this PR is aimed to resolve the conflict of spark configuration, I think the user behavior you described is out of scope here. --- If your

[GitHub] spark pull request #18301: [SPARK-21052][SQL] Add hash map metrics to join

2017-06-14 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/18301#discussion_r122116294 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala --- @@ -573,8 +586,11 @@ private[execution] final class

[GitHub] spark issue #18231: [SPARK-20994] Remove redundant characters in OpenBlocks ...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18231 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18284: [SPARK-21072][SQL] `TreeNode.mapChildren` should ...

2017-06-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18284#discussion_r122120866 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/trees/TreeNodeSuite.scala --- @@ -146,6 +154,23 @@ class TreeNodeSuite extends

[GitHub] spark issue #18231: [SPARK-20994] Remove redundant characters in OpenBlocks ...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18231 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78079/ Test PASSed. ---

[GitHub] spark pull request #18284: [SPARK-21072][SQL] `TreeNode.mapChildren` should ...

2017-06-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18284#discussion_r122120851 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/trees/TreeNodeSuite.scala --- @@ -61,6 +61,14 @@ case class ExpressionInMap(map:

[GitHub] spark pull request #18284: [SPARK-21072][SQL] `TreeNode.mapChildren` should ...

2017-06-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18284#discussion_r122120916 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/trees/TreeNodeSuite.scala --- @@ -146,6 +154,23 @@ class TreeNodeSuite extends

[GitHub] spark pull request #18075: [SPARK-18016][SQL][CATALYST] Code Generation: Con...

2017-06-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18075#discussion_r122122817 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/GeneratedProjectionSuite.scala --- @@ -62,13 +62,63 @@ class

[GitHub] spark pull request #18075: [SPARK-18016][SQL][CATALYST] Code Generation: Con...

2017-06-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18075#discussion_r122122765 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/GeneratedProjectionSuite.scala --- @@ -33,10 +33,10 @@ class

[GitHub] spark pull request #18075: [SPARK-18016][SQL][CATALYST] Code Generation: Con...

2017-06-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18075#discussion_r122122867 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala --- @@ -93,7 +93,7 @@ private[sql] trait ColumnarBatchScan

[GitHub] spark issue #18312: [SPARK-20980][DOCS] update doc to reflect multiLine chan...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18312 **[Test build #78084 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78084/testReport)** for PR 18312 at commit

[GitHub] spark issue #18312: [SPARK-20980][DOCS] update doc to reflect multiLine chan...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18312 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18262: [SPARK-21045][PYSPARK]Fixed executor blocked because tra...

2017-06-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18262 cc @viirya here too. (This one is similar one with few PRs you and I are reviewing). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #18302: [SPARK-17237][SQL][FOLLOWUP] Add a qualifier in pretty e...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18302 **[Test build #78074 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78074/testReport)** for PR 18302 at commit

[GitHub] spark issue #18290: [SPARK-20989][Core] Fail to start multiple workers on on...

2017-06-14 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18290 @jiangxb1987 , I think using environment variable to check if there's number of same-node workers connected id potentially vulnerable, user can manually start worker one by one without setting

[GitHub] spark pull request #18309: [SPARK-21079] [SQL] Calculate total size of a par...

2017-06-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/18309#discussion_r122102333 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala --- @@ -81,6 +83,19 @@ case class

[GitHub] spark issue #13585: [SPARK-15859][SQL] Optimize the partition pruning within...

2017-06-14 Thread chenghao-intel
Github user chenghao-intel commented on the issue: https://github.com/apache/spark/pull/13585 Oh, yes, I am closing it, will reopen it when we have another idea. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #13585: [SPARK-15859][SQL] Optimize the partition pruning...

2017-06-14 Thread chenghao-intel
Github user chenghao-intel closed the pull request at: https://github.com/apache/spark/pull/13585 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18310: [SPARK-21103][SQL] QueryPlanConstraints should be part o...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18310 **[Test build #78078 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78078/testReport)** for PR 18310 at commit

[GitHub] spark issue #18301: [SPARK-21052][SQL] Add hash map metrics to join

2017-06-14 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18301 The screenshot of BroadcastHashJoin: https://user-images.githubusercontent.com/68855/27163920-324127ea-51bd-11e7-8bee-924a20e161b8.png;> --- If your project is set up for it, you can reply

[GitHub] spark issue #18309: [SPARK-21079] [SQL] Calculate total size of a partition ...

2017-06-14 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/18309 Can you add a test case? In the test, we can add partitions with different paths by ALTER TABLE SET LOCATION command. I think that can reproduce your scenario, right? --- If your project is set up

[GitHub] spark issue #18284: [SPARK-21072][SQL] `TreeNode.mapChildren` should only ap...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18284 **[Test build #78081 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78081/testReport)** for PR 18284 at commit

[GitHub] spark pull request #18202: [SPARK-20980] [SQL] Rename `wholeFile` to `multiL...

2017-06-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18202 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18202: [SPARK-20980] [SQL] Rename `wholeFile` to `multiLine` fo...

2017-06-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18202 ah lucky :) merging to master/2.2! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18309: [SPARK-21079] [SQL] Calculate total size of a par...

2017-06-14 Thread mbasmanova
GitHub user mbasmanova opened a pull request: https://github.com/apache/spark/pull/18309 [SPARK-21079] [SQL] Calculate total size of a partition table as a sum of individual partitions ## What changes were proposed in this pull request? When calculating total size of a

[GitHub] spark pull request #18309: [SPARK-21079] [SQL] Calculate total size of a par...

2017-06-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/18309#discussion_r122102041 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala --- @@ -109,16 +124,16 @@ object

[GitHub] spark issue #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18300 **[Test build #78073 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78073/testReport)** for PR 18300 at commit

[GitHub] spark issue #18299: [SPARK-21092][SQL] Wire SQLConf in logical plan and expr...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18299 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78077/ Test PASSed. ---

[GitHub] spark pull request #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-14 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18300#discussion_r122116160 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1764,6 +1765,58 @@ class Dataset[T] private[sql]( } /**

[GitHub] spark pull request #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18300#discussion_r122116053 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1764,6 +1765,58 @@ class Dataset[T] private[sql]( } /**

[GitHub] spark issue #18299: [SPARK-21092][SQL] Wire SQLConf in logical plan and expr...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18299 **[Test build #78077 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78077/testReport)** for PR 18299 at commit

[GitHub] spark issue #18299: [SPARK-21092][SQL] Wire SQLConf in logical plan and expr...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18299 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18309: [SPARK-21079] [SQL] Calculate total size of a partition ...

2017-06-14 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/18309 Please explain the failed scenario in PR description. That's why our test cases cannot catch it and we need to test it manually. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #18310: [SPARK-21103][SQL] QueryPlanConstraints should be...

2017-06-14 Thread rxin
Github user rxin closed the pull request at: https://github.com/apache/spark/pull/18310 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #18284: [SPARK-21072][SQL] `TreeNode.mapChildren` should ...

2017-06-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18284#discussion_r122120790 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/trees/TreeNodeSuite.scala --- @@ -61,6 +61,14 @@ case class ExpressionInMap(map:

[GitHub] spark issue #18231: [SPARK-20994] Remove redundant characters in OpenBlocks ...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18231 **[Test build #78079 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78079/testReport)** for PR 18231 at commit

[GitHub] spark issue #18299: [SPARK-21092][SQL] Wire SQLConf in logical plan and expr...

2017-06-14 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/18299 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #18284: [SPARK-21072][SQL] `TreeNode.mapChildren` should ...

2017-06-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18284#discussion_r122120741 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala --- @@ -340,11 +340,21 @@ abstract class TreeNode[BaseType <:

[GitHub] spark issue #18310: [SPARK-21103][SQL] QueryPlanConstraints should be part o...

2017-06-14 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/18310 Closing for now, since @sameeragarwal said it might be useful in physical planning in the future. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #18284: [SPARK-21072][SQL] `TreeNode.mapChildren` should only ap...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18284 **[Test build #78082 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78082/testReport)** for PR 18284 at commit

[GitHub] spark issue #18144: [SPARK-20912][SQL] Allow column name in map functions.

2017-06-14 Thread liyichao
Github user liyichao commented on the issue: https://github.com/apache/spark/pull/18144 As the idea is not that good, this is closed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18144: [SPARK-20912][SQL] Allow column name in map funct...

2017-06-14 Thread liyichao
Github user liyichao closed the pull request at: https://github.com/apache/spark/pull/18144 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18300 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18300 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78073/ Test PASSed. ---

[GitHub] spark pull request #18301: [SPARK-21052][SQL] Add hash map metrics to join

2017-06-14 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18301#discussion_r122114832 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala --- @@ -573,8 +586,11 @@ private[execution] final class

[GitHub] spark pull request #18309: [SPARK-21079] [SQL] Calculate total size of a par...

2017-06-14 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/18309#discussion_r122116932 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala --- @@ -81,6 +83,19 @@ case class AnalyzeTableCommand(

[GitHub] spark pull request #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-14 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18300#discussion_r122116812 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1764,6 +1765,58 @@ class Dataset[T] private[sql]( } /**

[GitHub] spark pull request #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-14 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18300#discussion_r122121031 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1764,6 +1765,58 @@ class Dataset[T] private[sql]( } /**

[GitHub] spark pull request #18299: [SPARK-21092][SQL] Wire SQLConf in logical plan a...

2017-06-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18299 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #18284: [SPARK-21072][SQL] `TreeNode.mapChildren` should ...

2017-06-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18284#discussion_r122120954 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/trees/TreeNodeSuite.scala --- @@ -146,6 +154,23 @@ class TreeNodeSuite extends

[GitHub] spark issue #18312: [SPARK-20980][DOCS] update doc to reflect multiLine chan...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18312 **[Test build #78084 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78084/testReport)** for PR 18312 at commit

[GitHub] spark pull request #18075: [SPARK-18016][SQL][CATALYST] Code Generation: Con...

2017-06-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18075 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18312: [SPARK-20980][DOCS] update doc to reflect multiLine chan...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18312 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78084/ Test PASSed. ---

[GitHub] spark issue #18284: [SPARK-21072][SQL] `TreeNode.mapChildren` should only ap...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18284 **[Test build #78075 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78075/testReport)** for PR 18284 at commit

[GitHub] spark issue #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18300 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18300 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78076/ Test PASSed. ---

[GitHub] spark pull request #18306: [SPARK-21029][SS] All StreamingQuery should be st...

2017-06-14 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/18306#discussion_r122091309 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala --- @@ -321,6 +321,17 @@ class StreamingQueryManager

[GitHub] spark pull request #18084: [SPARK-19900][core]Remove driver when relaunching...

2017-06-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18084 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18309: [SPARK-21079] [SQL] Calculate total size of a partition ...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18309 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18300 **[Test build #78076 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78076/testReport)** for PR 18300 at commit

[GitHub] spark issue #18299: [SPARK-21092][SQL] Wire SQLConf in logical plan and expr...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18299 **[Test build #78077 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78077/testReport)** for PR 18299 at commit

[GitHub] spark issue #18311: Branch 2.0

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18311 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18306: [SPARK-21029][SS] All StreamingQuery should be st...

2017-06-14 Thread aray
Github user aray commented on a diff in the pull request: https://github.com/apache/spark/pull/18306#discussion_r122112073 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala --- @@ -690,6 +690,7 @@ class SparkSession private( * @since 2.0.0

[GitHub] spark issue #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18300 **[Test build #78076 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78076/testReport)** for PR 18300 at commit

[GitHub] spark issue #18301: [SPARK-21052][SQL] Add hash map metrics to join

2017-06-14 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18301 The screenshot of HashAggregate: https://user-images.githubusercontent.com/68855/27163956-5a0b5e6c-51bd-11e7-9159-f4962ac1bc36.png;> --- If your project is set up for it, you can reply to

[GitHub] spark issue #18301: [SPARK-21052][SQL] Add hash map metrics to join

2017-06-14 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18301 The screenshot of ShuffledHashJoin: https://user-images.githubusercontent.com/68855/27163988-86e82492-51bd-11e7-866e-199467c35817.png;> --- If your project is set up for it, you can reply

[GitHub] spark issue #18310: [SPARK-21103][SQL] QueryPlanConstraints should be part o...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18310 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78078/ Test PASSed. ---

[GitHub] spark issue #18310: [SPARK-21103][SQL] QueryPlanConstraints should be part o...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18310 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18310: [SPARK-21103][SQL] QueryPlanConstraints should be part o...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18310 **[Test build #78078 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78078/testReport)** for PR 18310 at commit

[GitHub] spark pull request #18075: [SPARK-18016][SQL][CATALYST] Code Generation: Con...

2017-06-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18075#discussion_r122122445 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -233,10 +222,118 @@ class

[GitHub] spark issue #18046: [SPARK-20749][SQL] Built-in SQL Function Support - all v...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18046 **[Test build #78083 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78083/testReport)** for PR 18046 at commit

[GitHub] spark issue #18312: [SPARK-20980][DOCS] update doc to reflect multiLine chan...

2017-06-14 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18312 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18298: [SPARK-21091][SQL] Move constraint code into QueryPlanCo...

2017-06-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18298 **[Test build #78067 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78067/testReport)** for PR 18298 at commit

[GitHub] spark issue #18298: [SPARK-21091][SQL] Move constraint code into QueryPlanCo...

2017-06-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18298 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78067/ Test PASSed. ---

[GitHub] spark issue #18303: [SPARK-19824][Core] Update JsonProtocol to keep consiste...

2017-06-14 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18303 @gatorsmile Sure, let me do it later! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18308: [SPARK-21099][Spark Core] INFO Log Message Using Incorre...

2017-06-14 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18308 LGTM. BTW can you please complement the PR description, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

  1   2   3   4   5   >