[GitHub] spark issue #18687: [SPARK-21484][SQL][WIP] Fix inconsistent query plans of ...

2017-07-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18687 I think you might push more changes in the following few hours. If the changes are clean and safe, we still need a SQLConf to ensure the behaviors can be changed back. We might not be able to

[GitHub] spark issue #18388: [SPARK-21175] Reject OpenBlocks when memory shortage on ...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18388 **[Test build #79879 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79879/testReport)** for PR 18388 at commit

[GitHub] spark issue #18687: [SPARK-21484][SQL][WIP] Fix inconsistent query plans of ...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18687 **[Test build #79878 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79878/testReport)** for PR 18687 at commit

[GitHub] spark issue #18718: [SPARK-21357][DStreams] FileInputDStream not remove out ...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18718 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18718: [SPARK-21357][DStreams] FileInputDStream not remo...

2017-07-22 Thread shaofei007
GitHub user shaofei007 opened a pull request: https://github.com/apache/spark/pull/18718 [SPARK-21357][DStreams] FileInputDStream not remove out of date RDD ## What changes were proposed in this pull request? ```DStreams class FileInputDStream [line

[GitHub] spark issue #18687: [SPARK-21484][SQL][WIP] Fix inconsistent query plans of ...

2017-07-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18687 > Once a dataset is uncached, the same fragment of logical plan in other datasets should fallback to uncached plan. This is true, although the plan looks like the same. However, the

[GitHub] spark issue #18687: [SPARK-21484][SQL][WIP] Fix inconsistent query plans of ...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18687 **[Test build #79877 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79877/testReport)** for PR 18687 at commit

[GitHub] spark pull request #18658: [SPARK-20871][SQL] limit logging of Janino code

2017-07-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18658#discussion_r128910412 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -562,6 +562,12 @@ object SQLConf { .intConf

[GitHub] spark pull request #18658: [SPARK-20871][SQL] limit logging of Janino code

2017-07-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18658#discussion_r128910402 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1037,25 +1038,27 @@ object

[GitHub] spark pull request #18658: [SPARK-20871][SQL] limit logging of Janino code

2017-07-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18658#discussion_r128910407 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1037,25 +1038,27 @@ object

[GitHub] spark issue #18687: [SPARK-21484][SQL][WIP] Fix inconsistent query plans of ...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18687 **[Test build #79876 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79876/testReport)** for PR 18687 at commit

[GitHub] spark issue #18717: [SPARK-21510] [SQL] Add isMaterialized() and eager persi...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18717 **[Test build #79875 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79875/testReport)** for PR 18717 at commit

[GitHub] spark pull request #18717: [SPARK-21510] [SQL] Add isMaterialized() and eage...

2017-07-22 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/18717 [SPARK-21510] [SQL] Add isMaterialized() and eager persist() to Dataset APIs ### What changes were proposed in this pull request? Currently, when using Spark, the beginners do not realize

[GitHub] spark issue #18715: [minor] Remove **** in test case names in FlatMapGroupsW...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18715 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79873/ Test PASSed. ---

[GitHub] spark issue #18715: [minor] Remove **** in test case names in FlatMapGroupsW...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18715 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18716: [SPARK-10063] Follow-up: remove a useless test related t...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18716 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18716: [SPARK-10063] Follow-up: remove a useless test related t...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18716 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79874/ Test PASSed. ---

[GitHub] spark issue #18716: [SPARK-10063] Follow-up: remove a useless test related t...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18716 **[Test build #79874 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79874/testReport)** for PR 18716 at commit

[GitHub] spark issue #18715: [minor] Remove **** in test case names in FlatMapGroupsW...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18715 **[Test build #79873 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79873/testReport)** for PR 18715 at commit

[GitHub] spark issue #18696: [SPARK-21490][core] Make sure SparkLauncher redirects ne...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18696 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79872/ Test PASSed. ---

[GitHub] spark issue #18696: [SPARK-21490][core] Make sure SparkLauncher redirects ne...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18696 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18696: [SPARK-21490][core] Make sure SparkLauncher redirects ne...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18696 **[Test build #79872 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79872/testReport)** for PR 18696 at commit

[GitHub] spark issue #17234: [SPARK-19892][MLlib] Implement findAnalogies method for ...

2017-07-22 Thread benradford
Github user benradford commented on the issue: https://github.com/apache/spark/pull/17234 @srowen @felixcheung: I believe all concerns have been addressed. Please let me know if there are any remaining issues. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #17234: [SPARK-19892][MLlib] Implement findAnalogies meth...

2017-07-22 Thread benradford
Github user benradford commented on a diff in the pull request: https://github.com/apache/spark/pull/17234#discussion_r123727450 --- Diff: R/pkg/DESCRIPTION --- @@ -54,5 +54,5 @@ Collate: 'types.R' 'utils.R' 'window.R' -RoxygenNote: 5.0.1

[GitHub] spark pull request #18714: [SPARK-20236][SQL] hive style partition overwrite

2017-07-22 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/18714#discussion_r128909042 --- Diff: core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala --- @@ -52,12 +55,22 @@ class

[GitHub] spark issue #18468: [SPARK-20783][SQL] Create CachedBatchColumnVector to abs...

2017-07-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18468 ping @cloud-fan, @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #18704: [SPARK-20783][SQL] Create CachedBatchColumnVector to abs...

2017-07-22 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18704 ping @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18658: [SPARK-20871][SQL] limit logging of Janino code

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18658 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18658: [SPARK-20871][SQL] limit logging of Janino code

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18658 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79871/ Test PASSed. ---

[GitHub] spark issue #18658: [SPARK-20871][SQL] limit logging of Janino code

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18658 **[Test build #79871 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79871/testReport)** for PR 18658 at commit

[GitHub] spark pull request #18714: [SPARK-20236][SQL] hive style partition overwrite

2017-07-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18714#discussion_r128908523 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -881,6 +881,16 @@ object SQLConf { .intConf

[GitHub] spark pull request #18714: [SPARK-20236][SQL] hive style partition overwrite

2017-07-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18714#discussion_r128908519 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -881,6 +881,16 @@ object SQLConf { .intConf

[GitHub] spark issue #18716: [SPARK-10063] Follow-up: remove a useless test related t...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18716 **[Test build #79874 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79874/testReport)** for PR 18716 at commit

[GitHub] spark pull request #18716: [SPARK-10063] Follow-up: remove a useless test re...

2017-07-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18716#discussion_r128908504 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/sources/HadoopFsRelationTest.scala --- @@ -783,52 +780,6 @@ abstract class HadoopFsRelationTest

[GitHub] spark issue #18716: [SPARK-10063] Follow-up: remove a useless test related t...

2017-07-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18716 cc @jiangxb1987 @robert3005 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18716: [SPARK-10063] Follow-up: remove a useless test re...

2017-07-22 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/18716 [SPARK-10063] Follow-up: remove a useless test related to an old output committer ## What changes were proposed in this pull request? It's a follow-up of

[GitHub] spark issue #18715: [minor] Remove **** in test case names in FlatMapGroupsW...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18715 **[Test build #79873 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79873/testReport)** for PR 18715 at commit

[GitHub] spark issue #18645: [SPARK-14280][BUILD][WIP] Update change-version.sh and p...

2017-07-22 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/18645 It is still source breaking change, and this is why I was saying it would be a lot of work to upgrade to Scala 2.12 without breaking existing source code. For 2.12 we should get rid of the functions

[GitHub] spark issue #18715: [minor] Remove **** in test case names in FlatMapGroupsW...

2017-07-22 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/18715 cc @tdas Was there a reason to use ``? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18715: [minor] Remove **** in test case names in FlatMap...

2017-07-22 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/18715 [minor] Remove in test case names in FlatMapGroupsWithStateSuite ## What changes were proposed in this pull request? This patch removes the `` string from test names in

[GitHub] spark issue #18709: [SPARK-21504] [SQL] Add spark version info into table me...

2017-07-22 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/18709 "Create Version" isn't a good user facing description. It'd make more sense to just say "Created by Spark xxx" --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #18714: [SPARK-20236][SQL] hive style partition overwrite

2017-07-22 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/18714#discussion_r128908118 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -881,6 +881,16 @@ object SQLConf { .intConf

[GitHub] spark issue #18696: [SPARK-21490][core] Make sure SparkLauncher redirects ne...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18696 **[Test build #79872 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79872/testReport)** for PR 18696 at commit

[GitHub] spark issue #18689: [SPARK-10063] Follow-up: remove dead code related to an ...

2017-07-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18689 interesting, that test should be removed, or the test should fail, let me take a closer look. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #18687: [SPARK-21484][SQL][WIP] Fix inconsistent query plans of ...

2017-07-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18687 Since you said it is well defined, then I suppose that the coded behavior of cache: * Once a dataset is cached, the same fragment of logical plan in other datasets should use the cached

[GitHub] spark issue #18689: [SPARK-10063] Follow-up: remove dead code related to an ...

2017-07-22 Thread robert3005
Github user robert3005 commented on the issue: https://github.com/apache/spark/pull/18689 Shouldn't https://github.com/apache/spark/blob/master/sql/hive/src/test/scala/org/apache/spark/sql/sources/HadoopFsRelationTest.scala#L786 also be removed? As I understand it it checks whether

[GitHub] spark issue #18658: [SPARK-20871][SQL] limit logging of Janino code

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18658 **[Test build #79871 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79871/testReport)** for PR 18658 at commit

[GitHub] spark pull request #18658: [SPARK-20871][SQL] limit logging of Janino code

2017-07-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18658#discussion_r128906812 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1037,25 +1037,25 @@ object

[GitHub] spark pull request #18658: [SPARK-20871][SQL] limit logging of Janino code

2017-07-22 Thread pjfanning
Github user pjfanning commented on a diff in the pull request: https://github.com/apache/spark/pull/18658#discussion_r128906682 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1037,25 +1037,25 @@ object

[GitHub] spark pull request #18658: [SPARK-20871][SQL] limit logging of Janino code

2017-07-22 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18658#discussion_r128906085 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -1037,25 +1037,25 @@ object

[GitHub] spark issue #18714: [SPARK-20236][SQL] hive style partition overwrite

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18714 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79870/ Test PASSed. ---

[GitHub] spark issue #18714: [SPARK-20236][SQL] hive style partition overwrite

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18714 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18714: [SPARK-20236][SQL] hive style partition overwrite

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18714 **[Test build #79870 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79870/testReport)** for PR 18714 at commit

[GitHub] spark issue #18687: [SPARK-21484][SQL][WIP] Fix inconsistent query plans of ...

2017-07-22 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18687 It is well defined, but never well documented. So far, there are multiple issues in the existing cache mechanism. We hesitate to do any major change until we figure out what is the next step of

[GitHub] spark issue #18713: [SPARK-21509][SQL] Add a config to enable adaptive query...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18713 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18713: [SPARK-21509][SQL] Add a config to enable adaptive query...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18713 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79869/ Test PASSed. ---

[GitHub] spark issue #18713: [SPARK-21509][SQL] Add a config to enable adaptive query...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18713 **[Test build #79869 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79869/testReport)** for PR 18713 at commit

[GitHub] spark issue #18714: [SPARK-20236][SQL] hive style partition overwrite

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18714 **[Test build #79870 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79870/testReport)** for PR 18714 at commit

[GitHub] spark issue #18714: [SPARK-20236][SQL] hive style partition overwrite

2017-07-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18714 cc @gatorsmile @ericl --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18714: [SPARK-20236][SQL] hive style partition overwrite

2017-07-22 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/18714 [SPARK-20236][SQL] hive style partition overwrite ## What changes were proposed in this pull request? When overwriting a partitioned table with dynamic partition columns, the behavior

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18702 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79868/ Test PASSed. ---

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18702 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18702 **[Test build #79868 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79868/testReport)** for PR 18702 at commit

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18702 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18702 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79867/ Test PASSed. ---

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18702 **[Test build #79867 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79867/testReport)** for PR 18702 at commit

[GitHub] spark issue #18712: [SPARK-17528][SQL][followup] remove unnecessary data cop...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18712 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18712: [SPARK-17528][SQL][followup] remove unnecessary data cop...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18712 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79866/ Test PASSed. ---

[GitHub] spark issue #18712: [SPARK-17528][SQL][followup] remove unnecessary data cop...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18712 **[Test build #79866 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79866/testReport)** for PR 18712 at commit

[GitHub] spark issue #18713: [SPARK-21509][SQL] Add a config to enable adaptive query...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18713 **[Test build #79869 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79869/testReport)** for PR 18713 at commit

[GitHub] spark pull request #18713: [SPARK-21509][SQL] Add a config to enable adaptiv...

2017-07-22 Thread jinxing64
GitHub user jinxing64 opened a pull request: https://github.com/apache/spark/pull/18713 [SPARK-21509][SQL] Add a config to enable adaptive query execution only for the last que… ## What changes were proposed in this pull request? Feature of adaptive query execution is a good

[GitHub] spark issue #18687: [SPARK-21484][SQL][WIP] Fix inconsistent query plans of ...

2017-07-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18687 We need to well define the cache behavior of Dataset. I agree that once a table is cached, other Dataset should use the cached table without reading it again. Then once a table is

[GitHub] spark issue #18687: [SPARK-21484][SQL] Fix inconsistent query plans of Datas...

2017-07-22 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18687 There is another case we should fix. You can see the storage level of ds2 is StorageLevel.NONE, but its executed plan is still cached version. scala> Seq("1",

[GitHub] spark issue #18543: [SPARK-21319][SQL] Fix memory leak in UnsafeExternalRowS...

2017-07-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18543 ping --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #18160: [SPARK-20938][SQL] Keep/print the string represen...

2017-07-22 Thread HyukjinKwon
Github user HyukjinKwon closed the pull request at: https://github.com/apache/spark/pull/18160 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #18647: [MINOR][PYTHON] Remove obsolete codes for parsing abstra...

2017-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18647 gentle ping @davies and @JoshRosen. I guess it is mainly just about a decision - whether we expose it in any way or remove it out if I am not mistaken. --- If your project is set up for it,

[GitHub] spark pull request #18618: [SPARK-20090][PYTHON] Add StructType.fieldNames i...

2017-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18618#discussion_r128896314 --- Diff: python/pyspark/sql/types.py --- @@ -445,9 +445,12 @@ class StructType(DataType): This is the data type representing a

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18702 **[Test build #79868 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79868/testReport)** for PR 18702 at commit

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18702 **[Test build #79867 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79867/testReport)** for PR 18702 at commit

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18702 @rxin, while fixing minor cleanup, the comment was folded but please see https://github.com/apache/spark/pull/18702#discussion_r128892162. --- If your project is set up for it, you can reply

[GitHub] spark issue #18712: [SPARK-17528][SQL][followup] remove unnecessary data cop...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18712 **[Test build #79866 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79866/testReport)** for PR 18712 at commit

[GitHub] spark issue #18712: [SPARK-17528][SQL][followup] remove unnecessary data cop...

2017-07-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18712 cc @liancheng @WeichenXu123 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18712: [SPARK-17528][SQL][followup] remove unnecessary d...

2017-07-22 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/18712 [SPARK-17528][SQL][followup] remove unnecessary data copy in object hash aggregate ## What changes were proposed in this pull request? In #18483 , we fixed the data copy bug when saving

[GitHub] spark issue #18711: [SPARK-21506][DOC]The description of "spark.executor.cor...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18711 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79865/ Test PASSed. ---

[GitHub] spark issue #18711: [SPARK-21506][DOC]The description of "spark.executor.cor...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18711 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18711: [SPARK-21506][DOC]The description of "spark.executor.cor...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18711 **[Test build #79865 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79865/testReport)** for PR 18711 at commit

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18702 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18702 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79864/ Test PASSed. ---

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18702 **[Test build #79864 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79864/testReport)** for PR 18702 at commit

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18702 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18702 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79863/ Test PASSed. ---

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18702 **[Test build #79863 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79863/testReport)** for PR 18702 at commit

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18702 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18702 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79862/ Test PASSed. ---

[GitHub] spark issue #18702: [SPARK-21485][SQL][DOCS] SQL documentation generation fo...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18702 **[Test build #79862 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79862/testReport)** for PR 18702 at commit

[GitHub] spark issue #18711: [SPARK-21506][DOC]The description of "spark.executor.cor...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18711 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18711: [SPARK-21506][DOC]The description of "spark.executor.cor...

2017-07-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18711 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79861/ Test PASSed. ---

[GitHub] spark issue #18711: [SPARK-21506][DOC]The description of "spark.executor.cor...

2017-07-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18711 **[Test build #79861 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79861/testReport)** for PR 18711 at commit

[GitHub] spark issue #18540: [SPARK-19451][SQL] rangeBetween method should accept Lon...

2017-07-22 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18540 LGTM except some minor comments --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #18540: [SPARK-19451][SQL] rangeBetween method should acc...

2017-07-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18540#discussion_r128894965 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/trees/TreeNodeSuite.scala --- @@ -436,21 +436,22 @@ class TreeNodeSuite extends

  1   2   >