[GitHub] spark issue #14712: [SPARK-17072] [SQL] support table-level statistics gener...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14712 **[Test build #64459 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64459/consoleFull)** for PR 14712 at commit [`aee4139`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #14712: [SPARK-17072] [SQL] support table-level statistics gener...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14712 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #14618: [SPARK-17030] [SQL] Remove/Cleanup HiveMetastoreC...

2016-08-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14618#discussion_r76371129 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala --- @@ -83,29 +103,30 @@ private[sql] class HiveSessionCatalog(

[GitHub] spark issue #14746: [SPARK-17180] [SQL] Fix View Resolution Order in ALTER V...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14746 **[Test build #64458 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64458/consoleFull)** for PR 14746 at commit [`51dbf72`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #14746: [SPARK-17180] [SQL] Fix View Resolution Order in ...

2016-08-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14746#discussion_r76370760 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -105,7 +105,13 @@ case class CreateViewCommand( }

[GitHub] spark pull request #14618: [SPARK-17030] [SQL] Remove/Cleanup HiveMetastoreC...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14618#discussion_r76370061 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala --- @@ -83,29 +103,30 @@ private[sql] class HiveSessionCatalog(

[GitHub] spark pull request #14618: [SPARK-17030] [SQL] Remove/Cleanup HiveMetastoreC...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14618#discussion_r76369902 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -74,3 +86,321 @@ private[hive] trait HiveStrategies { }

[GitHub] spark pull request #14618: [SPARK-17030] [SQL] Remove/Cleanup HiveMetastoreC...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14618#discussion_r76369704 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala --- @@ -63,9 +65,27 @@ private[sql] class HiveSessionCatalog( o

[GitHub] spark pull request #14618: [SPARK-17030] [SQL] Remove/Cleanup HiveMetastoreC...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14618#discussion_r76369538 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -498,6 +498,22 @@ class SessionCatalog( }

[GitHub] spark pull request #14746: [SPARK-17180] [SQL] Fix View Resolution Order in ...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14746#discussion_r76368945 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -105,7 +105,13 @@ case class CreateViewCommand( }

[GitHub] spark issue #14822: [SPARK-17187][SQL][follow-up] improve document of TypedI...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14822 **[Test build #64457 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64457/consoleFull)** for PR 14822 at commit [`846d282`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #14728: [SPARK-17165][SQL] FileStreamSource should not track the...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14728 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64454/ Test PASSed. ---

[GitHub] spark issue #14728: [SPARK-17165][SQL] FileStreamSource should not track the...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14728 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14728: [SPARK-17165][SQL] FileStreamSource should not track the...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14728 **[Test build #64454 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64454/consoleFull)** for PR 14728 at commit [`9a5ed19`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14618: [SPARK-17030] [SQL] Remove/Cleanup HiveMetastoreCatalog....

2016-08-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14618 @cloud-fan The conflicts are resolved. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark pull request #14822: [SPARK-17187][SQL][follow-up] improve document of...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14822#discussion_r76368623 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/interfaces.scala --- @@ -489,25 +510,23 @@ abstract class TypedI

[GitHub] spark pull request #14822: [SPARK-17187][SQL][follow-up] improve document of...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14822#discussion_r76368571 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/interfaces.scala --- @@ -390,48 +390,69 @@ abstract class Declar

[GitHub] spark issue #14822: [SPARK-17187][SQL][follow-up] improve document of TypedI...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14822 cc @clockfly @yhuai @liancheng --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request #14822: [SPARK-17187][SQL][follow-up] improve document of...

2016-08-25 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/14822 [SPARK-17187][SQL][follow-up] improve document of TypedImperativeAggregate ## What changes were proposed in this pull request? improve the document to make it easier to understand and als

[GitHub] spark issue #14618: [SPARK-17030] [SQL] Remove/Cleanup HiveMetastoreCatalog....

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14618 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64453/ Test PASSed. ---

[GitHub] spark issue #14618: [SPARK-17030] [SQL] Remove/Cleanup HiveMetastoreCatalog....

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14618 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14618: [SPARK-17030] [SQL] Remove/Cleanup HiveMetastoreCatalog....

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14618 **[Test build #64453 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64453/consoleFull)** for PR 14618 at commit [`ebdfad1`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14746: [SPARK-17180] [SQL] Fix View Resolution Order in ...

2016-08-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/14746#discussion_r76366559 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -105,7 +105,13 @@ case class CreateViewCommand( }

[GitHub] spark issue #14572: [SPARK-17192] [SQL] Issue Exception when Users Specify t...

2016-08-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14572 ping @yhuai : ) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, o

[GitHub] spark issue #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/8880 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64451/ Test PASSed. --- I

[GitHub] spark issue #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/8880 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark issue #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/8880 **[Test build #64451 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64451/consoleFull)** for PR 8880 at commit [`a9a05c5`](https://github.com/apache/spark/commit/a9

[GitHub] spark issue #14572: [SPARK-17192] [SQL] Issue Exception when Users Specify t...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14572 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64452/ Test PASSed. ---

[GitHub] spark issue #14572: [SPARK-17192] [SQL] Issue Exception when Users Specify t...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14572 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14572: [SPARK-17192] [SQL] Issue Exception when Users Specify t...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14572 **[Test build #64452 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64452/consoleFull)** for PR 14572 at commit [`d3a79c8`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14801: [SPARK-17234] [SQL] Table Existence Checking when Index ...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14801 **[Test build #64456 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64456/consoleFull)** for PR 14801 at commit [`8bcd946`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #14801: [SPARK-17234] [SQL] Table Existence Checking when Index ...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14801 **[Test build #64455 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64455/consoleFull)** for PR 14801 at commit [`3f75605`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #14808: [SPARK-17156][ML][EXAMPLE] Add multiclass logistic regre...

2016-08-25 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14808 This is going to have to be changed after [SPARK-17163](https://issues.apache.org/jira/browse/SPARK-17163). Sorry about the confusion! We'll still want to make an example with multiclass, though, so

[GitHub] spark issue #14818: [SPARK-17157][SPARKR][WIP]: Add multiclass logistic regr...

2016-08-25 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14818 In fact, we are actually just eliminating the `MultinomialLogisticRegression` interface and merging into the existing `LogisticRegression` estimator. So, maybe we won't need a change after all? I'm

[GitHub] spark issue #13231: [SPARK-15453] [SQL] Sort Merge Join to use bucketing met...

2016-08-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13231 @tejasapatil any chance to update it soon? If not, I am interested in implement it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #14821: [SPARK-17250] [SQL] Remove HiveClient and setCurrentData...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14821 LGTM, cc @yhuai to confirm. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark issue #14617: [SPARK-17019][Core] Expose on-heap and off-heap memory u...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14617 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64450/ Test PASSed. ---

[GitHub] spark issue #14617: [SPARK-17019][Core] Expose on-heap and off-heap memory u...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14617 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14617: [SPARK-17019][Core] Expose on-heap and off-heap memory u...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14617 **[Test build #64450 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64450/consoleFull)** for PR 14617 at commit [`838840d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14728: [SPARK-17165][SQL] FileStreamSource should not track the...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14728 **[Test build #64454 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64454/consoleFull)** for PR 14728 at commit [`9a5ed19`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #14802: [SPARK-17235][SQL] Support purging of old logs in Metada...

2016-08-25 Thread petermaxlee
Github user petermaxlee commented on the issue: https://github.com/apache/spark/pull/14802 @zsxwing yup I plan to consolidate them. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark pull request #14814: [SPARK-17242][Document]Update links of external d...

2016-08-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14814 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14819: [SPARK-17246][SQL] Add BigDecimal literal

2016-08-25 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14819 Does other database do this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wish

[GitHub] spark issue #14814: [SPARK-17242][Document]Update links of external dstream ...

2016-08-25 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14814 Merging in master/2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #14618: [SPARK-17030] [SQL] Remove/Cleanup HiveMetastoreCatalog....

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14618 **[Test build #64453 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64453/consoleFull)** for PR 14618 at commit [`ebdfad1`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #14818: [SPARK-17157][SPARKR][WIP]: Add multiclass logistic regr...

2016-08-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14818 cool, thanks for the heads up @sethah - please loop us in for the R side changes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #14572: [SPARK-17192] [SQL] Issue Exception when Users Specify t...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14572 **[Test build #64452 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64452/consoleFull)** for PR 14572 at commit [`d3a79c8`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #14572: [SPARK-17192] [SQL] Issue Exception when Users Specify t...

2016-08-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14572 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #14818: [SPARK-17157][SPARKR][WIP]: Add multiclass logistic regr...

2016-08-25 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/14818 This is going to have to wait. We are changing the interface completely. See https://issues.apache.org/jira/browse/SPARK-17163. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/8880 **[Test build #64451 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64451/consoleFull)** for PR 8880 at commit [`a9a05c5`](https://github.com/apache/spark/commit/a9a

[GitHub] spark issue #14821: [SPARK-17250] [SQL] Remove HiveClient and setCurrentData...

2016-08-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14821 cc @cloud-fan @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14809: [SPARK-17238][SQL] simplify the logic for converting dat...

2016-08-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14809 Yeah, that is a bug. We did not get an exception when we read it, but we can get the error when trying to write it. The error message is confusing ``` Can only write data to relations with

[GitHub] spark issue #13452: [SPARK-15718][SQL] better error message for writing buck...

2016-08-25 Thread Downchuck
Github user Downchuck commented on the issue: https://github.com/apache/spark/pull/13452 Regarding the reason for disallowing bucket writes: "we have no idea [on read] if the data is bucketed or not, so it doesn't make sense to use save to write bucketed data" It's easy enoug

[GitHub] spark issue #14537: [SPARK-16948][SQL] Support empty orc table when converti...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14537 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64449/ Test PASSed. ---

[GitHub] spark issue #14537: [SPARK-16948][SQL] Support empty orc table when converti...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14537 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14537: [SPARK-16948][SQL] Support empty orc table when converti...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14537 **[Test build #64449 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64449/consoleFull)** for PR 14537 at commit [`9ecb2ed`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14821: [SPARK-17250] [SQL] Remove HiveClient and setCurrentData...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14821 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64448/ Test PASSed. ---

[GitHub] spark issue #14821: [SPARK-17250] [SQL] Remove HiveClient and setCurrentData...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14821 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14821: [SPARK-17250] [SQL] Remove HiveClient and setCurrentData...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14821 **[Test build #64448 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64448/consoleFull)** for PR 14821 at commit [`1c9a1e3`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14712: [SPARK-17072] [SQL] support table-level statistic...

2016-08-25 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/14712#discussion_r76356157 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala --- @@ -88,14 +89,70 @@ case class AnalyzeTableCommand(tabl

[GitHub] spark issue #14809: [SPARK-17238][SQL] simplify the logic for converting dat...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14809 ah i see, btw in your example when will we throw exception? when we read it? a file-based external table without path is invalid. --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #14712: [SPARK-17072] [SQL] support table-level statistics gener...

2016-08-25 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/14712 @cloud-fan Can you please launch test for this pr? thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #14819: [SPARK-17246][SQL] Add BigDecimal literal

2016-08-25 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/14819#discussion_r76355514 --- Diff: sql/core/src/test/resources/sql-tests/inputs/literals.sql --- @@ -27,6 +27,12 @@ select 9223372036854775807L, -9223372036854775808L; -- out of r

[GitHub] spark pull request #14537: [SPARK-16948][SQL] Support empty orc table when c...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14537#discussion_r76355262 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -237,21 +237,27 @@ private[hive] class HiveMetastoreCatalog(

[GitHub] spark issue #14617: [SPARK-17019][Core] Expose on-heap and off-heap memory u...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14617 **[Test build #64450 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64450/consoleFull)** for PR 14617 at commit [`838840d`](https://github.com/apache/spark/commit/8

[GitHub] spark pull request #14712: [SPARK-17072] [SQL] support table-level statistic...

2016-08-25 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/14712#discussion_r76354939 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala --- @@ -88,14 +89,70 @@ case class AnalyzeTableCommand(tabl

[GitHub] spark issue #14537: [SPARK-16948][SQL] Support empty orc table when converti...

2016-08-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14537 BTW, @rajeshbalamohan as you directly use metastore schema now, the PR description looks not correct anymore, can you also update it? Thanks. --- If your project is set up for it, you can reply to t

[GitHub] spark issue #14819: [SPARK-17246][SQL] Add BigDecimal literal

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14819 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14819: [SPARK-17246][SQL] Add BigDecimal literal

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14819 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64445/ Test PASSed. ---

[GitHub] spark issue #14617: [SPARK-17019][Core] Expose on-heap and off-heap memory u...

2016-08-25 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/14617 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14819: [SPARK-17246][SQL] Add BigDecimal literal

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14819 **[Test build #64445 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64445/consoleFull)** for PR 14819 at commit [`fda100f`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14712: [SPARK-17072] [SQL] support table-level statistic...

2016-08-25 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/14712#discussion_r76354802 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala --- @@ -88,14 +89,70 @@ case class AnalyzeTableCommand(tabl

[GitHub] spark issue #14537: [SPARK-16948][SQL] Support empty orc table when converti...

2016-08-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14537 @gatorsmile Thanks for cc'ing me. As `spark.sql.hive.convertMetastoreOrc` is set to `false` by default, this change looks fine. However, if setting the config to `true`, and hitting with inc

[GitHub] spark issue #14239: [SPARK-16593] [CORE] [WIP] Provide a pre-fetch mechanism...

2016-08-25 Thread f7753
Github user f7753 commented on the issue: https://github.com/apache/spark/pull/14239 @tgravescs To make it more readable and answer the question above. **1. Are you saying that you are loading all the data for all the maps from disk into memory and caching it waiting for the reduc

[GitHub] spark pull request #14712: [SPARK-17072] [SQL] support table-level statistic...

2016-08-25 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/14712#discussion_r76354055 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala --- @@ -88,14 +89,70 @@ case class AnalyzeTableCommand(tabl

[GitHub] spark issue #14710: [SPARK-16533][CORE] resolve deadlocking in driver when e...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14710 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64442/ Test PASSed. ---

[GitHub] spark issue #14710: [SPARK-16533][CORE] resolve deadlocking in driver when e...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14710 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14710: [SPARK-16533][CORE] resolve deadlocking in driver when e...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14710 **[Test build #64442 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64442/consoleFull)** for PR 14710 at commit [`380291b`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14537: [SPARK-16948][SQL] Support empty orc table when converti...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14537 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14537: [SPARK-16948][SQL] Support empty orc table when converti...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14537 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64446/ Test PASSed. ---

[GitHub] spark issue #14537: [SPARK-16948][SQL] Support empty orc table when converti...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14537 **[Test build #64446 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64446/consoleFull)** for PR 14537 at commit [`fc14e2d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14809: [SPARK-17238][SQL] simplify the logic for converting dat...

2016-08-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14809 If we do not specify the schema, it will behave like what you said. For example, ```Scala sparkSession.catalog.createExternalTable( "createdParquetTable",

[GitHub] spark issue #14710: [SPARK-16533][CORE] resolve deadlocking in driver when e...

2016-08-25 Thread angolon
Github user angolon commented on the issue: https://github.com/apache/spark/pull/14710 Thanks for the feedback, @vanzin - all good points. I'll fix them up. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #14537: [SPARK-16948][SQL] Querying empty partitioned orc tables...

2016-08-25 Thread rajeshbalamohan
Github user rajeshbalamohan commented on the issue: https://github.com/apache/spark/pull/14537 Thanks @gatorsmile . Removed the changes related to OrcFileFormat --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark issue #14537: [SPARK-16948][SQL] Querying empty partitioned orc tables...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14537 **[Test build #64449 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64449/consoleFull)** for PR 14537 at commit [`9ecb2ed`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #14815: [SPARK-17244] Catalyst should not pushdown non-de...

2016-08-25 Thread sameeragarwal
Github user sameeragarwal commented on a diff in the pull request: https://github.com/apache/spark/pull/14815#discussion_r76351420 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1386,15 +1386,17 @@ object EliminateOuterJoin ex

[GitHub] spark issue #14821: [SPARK-17250] [SQL] Remove HiveClient and setCurrentData...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14821 **[Test build #64448 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64448/consoleFull)** for PR 14821 at commit [`1c9a1e3`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #14820: [SparkR][Minor] Fix example of spark.naiveBayes

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14820 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64447/ Test PASSed. ---

[GitHub] spark pull request #14811: [SPARK-17231][CORE] Avoid building debug or trace...

2016-08-25 Thread mallman
Github user mallman closed the pull request at: https://github.com/apache/spark/pull/14811 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] spark issue #14811: [SPARK-17231][CORE] Avoid building debug or trace log me...

2016-08-25 Thread mallman
Github user mallman commented on the issue: https://github.com/apache/spark/pull/14811 Done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featu

[GitHub] spark issue #14820: [SparkR][Minor] Fix example of spark.naiveBayes

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14820 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14820: [SparkR][Minor] Fix example of spark.naiveBayes

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14820 **[Test build #64447 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64447/consoleFull)** for PR 14820 at commit [`607f117`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14816: [SPARK-17245] [SQL] [BRANCH-1.6] Do not rely on Hive's s...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14816 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14816: [SPARK-17245] [SQL] [BRANCH-1.6] Do not rely on Hive's s...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14816 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64443/ Test PASSed. ---

[GitHub] spark issue #14809: [SPARK-17238][SQL] simplify the logic for converting dat...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14809 @gatorsmile can you explain more about this example? I think we will throw exception in `CreateDataSourceTableCommand` when we create a `DataSource` and call its `resolveRelation`. --- If your p

[GitHub] spark issue #14816: [SPARK-17245] [SQL] [BRANCH-1.6] Do not rely on Hive's s...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14816 **[Test build #64443 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64443/consoleFull)** for PR 14816 at commit [`8b57886`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14821: [SPARK-17250] [SQL] Remove HiveClient and setCurr...

2016-08-25 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/14821 [SPARK-17250] [SQL] Remove HiveClient and setCurrentDatabase from HiveSessionCatalog ### What changes were proposed in this pull request? This is the first step to remove `HiveClient` from `

[GitHub] spark pull request #14753: [SPARK-17187][SQL] Supports using arbitrary Java ...

2016-08-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14753 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #14786: [SPARK-17212][SQL] TypeCoercion supports widening...

2016-08-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14786 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14786: [SPARK-17212][SQL] TypeCoercion supports widening conver...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14786 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

  1   2   3   4   5   6   >