[GitHub] spark issue #18709: [SPARK-21504] [SQL] Add spark version info into table me...

2017-08-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18709 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[GitHub] spark pull request #18790: [SPARK-21587][SS] Added pushdown through watermar...

2017-08-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18790#discussion_r132094610 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -410,6 +413,14 @@ object PushProjectionThroughUnion

[GitHub] spark issue #18756: [SPARK-21548][SQL] "Support insert into serial columns o...

2017-08-08 Thread lvdongr
Github user lvdongr commented on the issue: https://github.com/apache/spark/pull/18756 The target of this pr is support to insert into specified columns, all columns is no need , like insert into t(a, c) values (1, 0.8) . --- If your project is set up for it, you can repl

[GitHub] spark issue #18707: [SPARK-21503][UI]: Spark UI shows incorrect task status ...

2017-08-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18707 I was just asking about different fixes, I'm ok with the current fix. thanks, merging to master/2.2! --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #18707: [SPARK-21503][UI]: Spark UI shows incorrect task ...

2017-08-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18707 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #18790: [SPARK-21587][SS] Added pushdown through watermar...

2017-08-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18790#discussion_r132095065 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -410,6 +413,14 @@ object PushProjectionThroughUnion

[GitHub] spark issue #18846: [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS ins...

2017-08-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18846 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18846: [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS ins...

2017-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18846 **[Test build #80443 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80443/testReport)** for PR 18846 at commit [`afc07ee`](https://github.com/apache/spark/commit/af

[GitHub] spark issue #18790: [SPARK-21587][SS] Added pushdown through watermarks.

2017-08-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18790 It sounds like we need to revisit all the optimizer rules for `EventTimeWatermark`? We can do it in the separate PRs. --- If your project is set up for it, you can reply to this email and have y

[GitHub] spark issue #16774: [SPARK-19357][ML] Adding parallel model evaluation in ML...

2017-08-08 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/16774 @BryanCutler You are right. Once `Future` complete the model can be cleaned by GC. So the memory cost of the code has been optimized already. I didn't look at the code carefully a few days ago.

[GitHub] spark issue #18866: [SPARK-21649][SQL] Support writing data into hive bucket...

2017-08-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18866 Hash function is not the only issue, one important difference is: hive will shuffle before write, and make sure one bucket has only one file. Spark doesn't shuffle and each write task may write a

[GitHub] spark issue #18790: [SPARK-21587][SS] Added pushdown through watermarks.

2017-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18790 **[Test build #80434 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80434/testReport)** for PR 18790 at commit [`4cae897`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #18890: [SPARK-21596][SS] Ensure places calling HDFSMetadataLog....

2017-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18890 **[Test build #80436 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80436/testReport)** for PR 18890 at commit [`532529a`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #18790: [SPARK-21587][SS] Added pushdown through watermarks.

2017-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18790 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18890: [SPARK-21596][SS] Ensure places calling HDFSMetadataLog....

2017-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18890 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80436/ Test PASSed. ---

[GitHub] spark issue #18790: [SPARK-21587][SS] Added pushdown through watermarks.

2017-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18790 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80434/ Test PASSed. ---

[GitHub] spark issue #18890: [SPARK-21596][SS] Ensure places calling HDFSMetadataLog....

2017-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18890 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18880: [SPARK-21665][Core]Need to close resources after use

2017-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18880 **[Test build #80444 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80444/testReport)** for PR 18880 at commit [`7bd2904`](https://github.com/apache/spark/commit/7b

[GitHub] spark issue #18709: [SPARK-21504] [SQL] Add spark version info into table me...

2017-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18709 **[Test build #80445 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80445/testReport)** for PR 18709 at commit [`f5a26cc`](https://github.com/apache/spark/commit/f5

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18869 @viirya Could you hold this PR at first? There is a related PR https://github.com/apache/spark/pull/18818 that is changing the underlying semantics. We also need to update the description after t

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18869 **[Test build #80435 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80435/testReport)** for PR 18869 at commit [`1369fd5`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18869 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80435/ Test PASSed. ---

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18869 @gatorsmile Sure. Let's wait after #18818. I'll look at it too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark issue #18869: [SPARK-21654][SQL] Complement SQL predicates expression ...

2017-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18869 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18818: [SPARK-21110][SQL] Structs, arrays, and other orderable ...

2017-08-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18818 Also add `NULL` in the test case? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enable

[GitHub] spark issue #18421: [SPARK-21213][SQL] Support collecting partition-level st...

2017-08-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18421 There are still two remaining comments. Could you also address them? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark issue #18787: [SPARK-21583][SQL] Create a ColumnarBatch from ArrowColu...

2017-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18787 **[Test build #80438 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80438/testReport)** for PR 18787 at commit [`23d19df`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #18787: [SPARK-21583][SQL] Create a ColumnarBatch from ArrowColu...

2017-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18787 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18787: [SPARK-21583][SQL] Create a ColumnarBatch from ArrowColu...

2017-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18787 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80438/ Test PASSed. ---

[GitHub] spark pull request #18421: [SPARK-21213][SQL] Support collecting partition-l...

2017-08-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18421#discussion_r132099286 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -1033,25 +998,126 @@ private[spark] class HiveExternalCatalo

[GitHub] spark pull request #18421: [SPARK-21213][SQL] Support collecting partition-l...

2017-08-08 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18421#discussion_r132099901 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -256,6 +257,222 @@ class StatisticsSuite extends StatisticsColl

[GitHub] spark issue #18499: [SPARK-21176][WEB UI] Use a single ProxyServlet to proxy...

2017-08-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18499 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] spark issue #18421: [SPARK-21213][SQL] Support collecting partition-level st...

2017-08-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18421 @mbasmanova Really thank you for your contribution! LGTM except the above four comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #18499: [SPARK-21176][WEB UI] Use a single ProxyServlet t...

2017-08-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18499 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18499: [SPARK-21176][WEB UI] Use a single ProxyServlet to proxy...

2017-08-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18499 @aosagie do we still need the previous fix for SPARK-21176? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark issue #18789: SPARK-20433 Bump jackson from 2.6.5 to 2.6.7.1

2017-08-08 Thread ash211
Github user ash211 commented on the issue: https://github.com/apache/spark/pull/18789 @srowen sorry for not picking up on this -- thanks for pushing it over the finish line in your PR! --- If your project is set up for it, you can reply to this email and have your reply appear on Git

[GitHub] spark issue #18797: [SPARK-21523][ML] update breeze to 0.13.2 for an emergen...

2017-08-08 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/18797 Merged into master and branch-2.2. Thanks for all. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request #18797: [SPARK-21523][ML] update breeze to 0.13.2 for an ...

2017-08-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18797 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18890: [SPARK-21596][SS] Ensure places calling HDFSMetadataLog....

2017-08-08 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/18890 Thanks! Merging to branch-2.2. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18867: [SPARK-21663][TESTS]test("remote fetch below max RPC mes...

2017-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18867 **[Test build #80437 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80437/testReport)** for PR 18867 at commit [`5fb7403`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #18867: [SPARK-21663][TESTS]test("remote fetch below max RPC mes...

2017-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18867 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80437/ Test PASSed. ---

[GitHub] spark issue #18867: [SPARK-21663][TESTS]test("remote fetch below max RPC mes...

2017-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18867 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18890: [SPARK-21596][SS] Ensure places calling HDFSMetadataLog....

2017-08-08 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/18890 Merged. Could you close the PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18846: [SPARK-21642][CORE] Use FQDN for DRIVER_HOST_ADDRESS ins...

2017-08-08 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/18846 @thide I think DRIVER_HOST_ADDRESS will be used to generate the driver url. Could you check if this line still works after your change? https://github.com/apache/spark/blob/b35660dd0e930f4b484a0

<    1   2   3   4   5   6