[GitHub] spark issue #15717: [SPARK-17910][SQL] Allow users to update the comment of ...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15717 **[Test build #68335 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68335/consoleFull)** for PR 15717 at commit [`0097809`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15808: [SPARK-17703][SQL][BACKPORT-2.0] Add unnamed version of ...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15808 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...

2016-11-08 Thread windpiger
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/15803 UTC timestamp can translate to different zone's local time,which the user see from the history ui. It is reasonable to return UTC/GMT time to the user browser, and show it by translate to the

[GitHub] spark issue #15808: [SPARK-17703][SQL][BACKPORT-2.0] Add unnamed version of ...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15808 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68333/ Test PASSed. ---

[GitHub] spark issue #15808: [SPARK-17703][SQL][BACKPORT-2.0] Add unnamed version of ...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15808 **[Test build #68333 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68333/consoleFull)** for PR 15808 at commit [`140f929`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15807: [SPARK-18147][SQL] do not fail for very complex aggregat...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15807 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15807: [SPARK-18147][SQL] do not fail for very complex aggregat...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15807 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68334/ Test FAILed. ---

[GitHub] spark issue #15807: [SPARK-18147][SQL] do not fail for very complex aggregat...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15807 **[Test build #68334 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68334/consoleFull)** for PR 15807 at commit [`0103bb4`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15297: [SPARK-9862]Handling data skew

2016-11-08 Thread scwf
Github user scwf commented on the issue: https://github.com/apache/spark/pull/15297 @YuhuWang2002 We should limit the use case for outer join: For left outer join, such as A left join B, this implementation now can not handle the case of skew of table B. That's because the

[GitHub] spark issue #15484: [SPARK-17868][SQL] Do not use bitmasks during parsing an...

2016-11-08 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/15484 ping @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15769: [SPARK-18191][CORE] Port RDD API to use commit protocol

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15769 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15769: [SPARK-18191][CORE] Port RDD API to use commit protocol

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15769 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68330/ Test PASSed. ---

[GitHub] spark issue #15769: [SPARK-18191][CORE] Port RDD API to use commit protocol

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15769 **[Test build #68330 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68330/consoleFull)** for PR 15769 at commit [`9380f91`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15807: [SPARK-18147][SQL] do not fail for very complex aggregat...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15807 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15807: [SPARK-18147][SQL] do not fail for very complex aggregat...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15807 **[Test build #68331 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68331/consoleFull)** for PR 15807 at commit [`3917630`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15797: [SPARK-17990][SPARK-18302][SQL] correct several partitio...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15797 **[Test build #68336 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68336/consoleFull)** for PR 15797 at commit [`f85bb27`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #15413: [SPARK-17847][ML] Reduce shuffled data size of GaussianM...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15413 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15413: [SPARK-17847][ML] Reduce shuffled data size of GaussianM...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15413 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68332/ Test PASSed. ---

[GitHub] spark issue #15413: [SPARK-17847][ML] Reduce shuffled data size of GaussianM...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15413 **[Test build #68332 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68332/consoleFull)** for PR 15413 at commit [`b2c2fa0`](https://github.com/apache/spark/commit/

[GitHub] spark issue #12162: [SPARK-14289][WIP] Support multiple eviction strategies ...

2016-11-08 Thread mozinrat
Github user mozinrat commented on the issue: https://github.com/apache/spark/pull/12162 @Earne is something relevant merged in spark 2.0.1, do we have FIFO eviction policy?. If yes how can I leverage it? --- If your project is set up for it, you can reply to this email and have y

[GitHub] spark issue #15413: [SPARK-17847][ML] Reduce shuffled data size of GaussianM...

2016-11-08 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15413 @sethah Yeah, I totally agree we can get around the issues I mentioned and make ```mllib.GaussianMixture``` call into ```ml.GaussianMixture```. But I'd prefer more to the current approach based o

[GitHub] spark issue #15805: [SPARK-18346][SQL] TRUNCATE TABLE should fail if no part...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15805 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68328/ Test PASSed. ---

[GitHub] spark issue #15805: [SPARK-18346][SQL] TRUNCATE TABLE should fail if no part...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15805 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15805: [SPARK-18346][SQL] TRUNCATE TABLE should fail if no part...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15805 **[Test build #68328 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68328/consoleFull)** for PR 15805 at commit [`0c18e27`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15751: [SPARK-18246][SQL] Throws an exception before execution ...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15751 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68326/ Test PASSed. ---

[GitHub] spark issue #15751: [SPARK-18246][SQL] Throws an exception before execution ...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15751 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15751: [SPARK-18246][SQL] Throws an exception before execution ...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15751 **[Test build #68326 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68326/consoleFull)** for PR 15751 at commit [`bae8db8`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15413: [SPARK-17847][ML] Reduce shuffled data size of Ga...

2016-11-08 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/15413#discussion_r86948993 --- Diff: mllib/src/test/scala/org/apache/spark/ml/clustering/GaussianMixtureSuite.scala --- @@ -130,4 +230,29 @@ object GaussianMixtureSuite {

[GitHub] spark pull request #15413: [SPARK-17847][ML] Reduce shuffled data size of Ga...

2016-11-08 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/15413#discussion_r86948383 --- Diff: python/pyspark/ml/clustering.py --- @@ -95,15 +95,10 @@ class GaussianMixture(JavaEstimator, HasFeaturesCol, HasPredictionCol, HasMaxIte

[GitHub] spark pull request #15413: [SPARK-17847][ML] Reduce shuffled data size of Ga...

2016-11-08 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/15413#discussion_r86947971 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/GaussianMixture.scala --- @@ -350,6 +458,145 @@ object GaussianMixture extends DefaultParam

[GitHub] spark issue #15807: [SPARK-18147][SQL] do not fail for very complex aggregat...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15807 **[Test build #68334 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68334/consoleFull)** for PR 15807 at commit [`0103bb4`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #15717: [SPARK-17910][SQL] Allow users to update the comment of ...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15717 **[Test build #68335 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68335/consoleFull)** for PR 15717 at commit [`0097809`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #15808: [SPARK-17703][SQL][BACKPORT-2.0] Add unnamed version of ...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15808 **[Test build #68333 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68333/consoleFull)** for PR 15808 at commit [`140f929`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #15276: [SPARK-17703][SQL] Add unnamed version of addReferenceOb...

2016-11-08 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/15276 I sent a new PR #15808. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #15797: [SPARK-17990][SPARK-18302][SQL] correct several p...

2016-11-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/15797#discussion_r86946976 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalogSuite.scala --- @@ -591,25 +673,25 @@ abstract class ExternalCa

[GitHub] spark pull request #15808: [SPARK-17703][SQL][BACKPORT-2.0] Add unnamed vers...

2016-11-08 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/15808 [SPARK-17703][SQL][BACKPORT-2.0] Add unnamed version of addReferenceObj for minor objects. ## What changes were proposed in this pull request? This is a backport of #15276. There a

[GitHub] spark issue #15803: [SPARK-18298][Web UI]change gmt time to local zone time ...

2016-11-08 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/spark/pull/15803 thanks for the fix. This patch parse the timestamp instead of the Date String returned. The REST api still return the GMT time, which is insistent with UI showing. I've goog

[GitHub] spark issue #15797: [SPARK-17990][SPARK-18302][SQL] correct several partitio...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15797 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15797: [SPARK-17990][SPARK-18302][SQL] correct several partitio...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15797 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68329/ Test FAILed. ---

[GitHub] spark issue #15797: [SPARK-17990][SPARK-18302][SQL] correct several partitio...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15797 **[Test build #68329 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68329/consoleFull)** for PR 15797 at commit [`dddee47`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15413: [SPARK-17847][ML] Reduce shuffled data size of GaussianM...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15413 **[Test build #68332 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68332/consoleFull)** for PR 15413 at commit [`b2c2fa0`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #15717: [SPARK-17910][SQL] Allow users to update the comment of ...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15717 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68327/ Test FAILed. ---

[GitHub] spark issue #15717: [SPARK-17910][SQL] Allow users to update the comment of ...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15717 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15717: [SPARK-17910][SQL] Allow users to update the comment of ...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15717 **[Test build #68327 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68327/consoleFull)** for PR 15717 at commit [`fda6d3a`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15012: [SPARK-17446] [SQL] No total size for data source tables...

2016-11-08 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/15012 The issue is resolved after merging https://github.com/apache/spark/pull/15024, so I'll close this pr. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #15012: [SPARK-17446] [SQL] No total size for data source...

2016-11-08 Thread wzhfy
Github user wzhfy closed the pull request at: https://github.com/apache/spark/pull/15012 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark issue #15807: [SPARK-18147][SQL] do not fail for very complex aggregat...

2016-11-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15807 Sorry I haven't noticed that https://github.com/apache/spark/pull/15693 was merged. Then this PR becomes a cleanup, not a bug fix. But I'd like to keep the regression test as it's from another JIR

[GitHub] spark issue #15717: [SPARK-17910][SQL] Allow users to update the comment of ...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15717 **[Test build #68324 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68324/consoleFull)** for PR 15717 at commit [`cba5bbd`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15717: [SPARK-17910][SQL] Allow users to update the comment of ...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15717 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68324/ Test FAILed. ---

[GitHub] spark issue #15717: [SPARK-17910][SQL] Allow users to update the comment of ...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15717 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15807: [SPARK-18147][SQL] do not fail for very complex aggregat...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15807 **[Test build #68331 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68331/consoleFull)** for PR 15807 at commit [`3917630`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #15276: [SPARK-17703][SQL] Add unnamed version of addReferenceOb...

2016-11-08 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/15276 Oh, I see, wait for a while. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wis

[GitHub] spark issue #15668: [SPARK-18137][SQL]Fix RewriteDistinctAggregates Unresolv...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15668 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15668: [SPARK-18137][SQL]Fix RewriteDistinctAggregates Unresolv...

2016-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15668 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68323/ Test PASSed. ---

[GitHub] spark issue #15668: [SPARK-18137][SQL]Fix RewriteDistinctAggregates Unresolv...

2016-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15668 **[Test build #68323 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68323/consoleFull)** for PR 15668 at commit [`6e58167`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15276: [SPARK-17703][SQL] Add unnamed version of addReferenceOb...

2016-11-08 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15276 Please do! I was hoping you'd do it...! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request #15807: [SPARK-18147][SQL] do not fail for very complex a...

2016-11-08 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/15807 [SPARK-18147][SQL] do not fail for very complex aggregator result type ## What changes were proposed in this pull request? In `TypedAggregateExpression.evaluateExpression`, we may create

[GitHub] spark issue #15276: [SPARK-17703][SQL] Add unnamed version of addReferenceOb...

2016-11-08 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/15276 @davies @rxin Should I send a PR to backport this to `branch-2.0` if you don't have time? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as w

[GitHub] spark issue #15047: [SPARK-17495] [SQL] Add Hash capability semantically equ...

2016-11-08 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15047 One testing technique we have used internally at Databricks (not for Spark) is to use random data generator to generate a bunch of data, and run through the reference implementation to get the results,

[GitHub] spark issue #15659: [SPARK-1267][SPARK-18129] Allow PySpark to be pip instal...

2016-11-08 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/15659 @viirya: the pip install sanity check is done assuming it was installed into an activated virtual env of just that Python version. You can see how to use it in 'run-pip-tests-2'. If you have multipl

<    1   2   3   4   5   6