[GitHub] spark pull request #16349: [Doc] bucketing is applicable to all file-based d...

2016-12-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16349 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16349: [Doc] bucketing is applicable to all file-based data sou...

2016-12-21 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16349 Merging in master/branch-2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16378: [SQL] Minor readability improvement for partition handli...

2016-12-21 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16378 I've also cherry picked this into branch-2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16382: [SPARK-18975][Core] Add an API to remove SparkListener

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16382 **[Test build #70511 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70511/testReport)** for PR 16382 at commit

[GitHub] spark pull request #16380: [SPARK-18972][Core]Fix the netty thread names for...

2016-12-21 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16380#discussion_r93577731 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -315,6 +315,10 @@ final class ShuffleBlockFetcherIterator(

[GitHub] spark pull request #16382: [SPARK-18975][Core] Add an API to remove SparkLis...

2016-12-21 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/16382 [SPARK-18975][Core] Add an API to remove SparkListener ## What changes were proposed in this pull request? In current Spark we could add customized SparkListener through

[GitHub] spark issue #16378: [SQL] Minor readability improvement for partition handli...

2016-12-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16378 LGTM, merging to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16378: [SQL] Minor readability improvement for partition...

2016-12-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16378 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16119: [SPARK-18687][Pyspark][SQL]Backward compatibility...

2016-12-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16119#discussion_r93576154 --- Diff: python/pyspark/sql/context.py --- @@ -72,8 +72,13 @@ def __init__(self, sparkContext, sparkSession=None, jsqlContext=None):

[GitHub] spark pull request #16119: [SPARK-18687][Pyspark][SQL]Backward compatibility...

2016-12-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16119#discussion_r93575937 --- Diff: python/pyspark/sql/tests.py --- @@ -1851,6 +1851,71 @@ def test_hivecontext(self): self.assertIn("default", out.decode('utf-8'))

[GitHub] spark issue #16381: [SPARK-18973][SQL] Remove SortPartitions and Redistribut...

2016-12-21 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16381 Note that this is code from the initial Spark SQL commit! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16381: [SPARK-18973][SQL] Remove SortPartitions and Redistribut...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16381 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70509/ Test PASSed. ---

[GitHub] spark issue #16381: [SPARK-18973][SQL] Remove SortPartitions and Redistribut...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16381 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16381: [SPARK-18973][SQL] Remove SortPartitions and Redistribut...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16381 **[Test build #70509 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70509/testReport)** for PR 16381 at commit

[GitHub] spark issue #15211: [SPARK-14709][ML] spark.ml API for linear SVM

2016-12-21 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/15211 Thanks a lot for the review. @jkbradley About the class name. AFAIK, typically "linear SVM" and "general SVM" use different algorithms for implementations. Just like the difference between

[GitHub] spark issue #16380: [SPARK-18972][Core]Fix the netty thread names for RPC

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16380 **[Test build #70510 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70510/testReport)** for PR 16380 at commit

[GitHub] spark issue #16380: [SPARK-18972][Core]Fix the netty thread names for RPC

2016-12-21 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16380 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16380: [SPARK-18972][Core]Fix the netty thread names for RPC

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16380 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70508/ Test FAILed. ---

[GitHub] spark issue #16380: [SPARK-18972][Core]Fix the netty thread names for RPC

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16380 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16380: [SPARK-18972][Core]Fix the netty thread names for RPC

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16380 **[Test build #70508 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70508/testReport)** for PR 16380 at commit

[GitHub] spark pull request #16380: [SPARK-18972][Core]Fix the netty thread names for...

2016-12-21 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16380#discussion_r93570841 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -315,6 +315,10 @@ final class ShuffleBlockFetcherIterator(

[GitHub] spark pull request #16380: [SPARK-18972][Core]Fix the netty thread names for...

2016-12-21 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/16380#discussion_r93570750 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -315,6 +315,10 @@ final class ShuffleBlockFetcherIterator(

[GitHub] spark pull request #16322: [SPARK-18908][SS] Creating StreamingQueryExceptio...

2016-12-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16322 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16322: [SPARK-18908][SS] Creating StreamingQueryException shoul...

2016-12-21 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16322 Thanks! Merging to master and 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16253: [SPARK-18537][Web UI] Add a REST api to serve spark stre...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16253 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16253: [SPARK-18537][Web UI] Add a REST api to serve spark stre...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16253 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70506/ Test PASSed. ---

[GitHub] spark issue #16253: [SPARK-18537][Web UI] Add a REST api to serve spark stre...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16253 **[Test build #70506 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70506/testReport)** for PR 16253 at commit

[GitHub] spark issue #16323: [SPARK-18911] [SQL] Define CatalogStatistics to interact...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16323 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16323: [SPARK-18911] [SQL] Define CatalogStatistics to interact...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16323 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70505/ Test PASSed. ---

[GitHub] spark issue #16323: [SPARK-18911] [SQL] Define CatalogStatistics to interact...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16323 **[Test build #70505 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70505/testReport)** for PR 16323 at commit

[GitHub] spark pull request #16053: [SPARK-17931] Eliminate unncessary task (de) seri...

2016-12-21 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16053#discussion_r93568327 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala --- @@ -17,27 +17,123 @@ package org.apache.spark.scheduler

[GitHub] spark pull request #16053: [SPARK-17931] Eliminate unncessary task (de) seri...

2016-12-21 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16053#discussion_r93567628 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala --- @@ -17,27 +17,123 @@ package org.apache.spark.scheduler

[GitHub] spark pull request #16053: [SPARK-17931] Eliminate unncessary task (de) seri...

2016-12-21 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16053#discussion_r93567927 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala --- @@ -17,27 +17,123 @@ package org.apache.spark.scheduler

[GitHub] spark pull request #16053: [SPARK-17931] Eliminate unncessary task (de) seri...

2016-12-21 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16053#discussion_r93568599 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala --- @@ -17,27 +17,123 @@ package org.apache.spark.scheduler

[GitHub] spark pull request #16053: [SPARK-17931] Eliminate unncessary task (de) seri...

2016-12-21 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16053#discussion_r93568367 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala --- @@ -17,27 +17,123 @@ package org.apache.spark.scheduler

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70503/ Test PASSed. ---

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13599 **[Test build #70503 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70503/testReport)** for PR 13599 at commit

[GitHub] spark issue #16371: [SPARK-18932][SQL] Support partial aggregation for colle...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16371 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16371: [SPARK-18932][SQL] Support partial aggregation for colle...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16371 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70504/ Test PASSed. ---

[GitHub] spark issue #16371: [SPARK-18932][SQL] Support partial aggregation for colle...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16371 **[Test build #70504 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70504/testReport)** for PR 16371 at commit

[GitHub] spark pull request #16372: [SPARK-18949] [SQL] [BACKPORT-2.1] Add recoverPar...

2016-12-21 Thread gatorsmile
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/16372 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #16372: [SPARK-18949] [SQL] [BACKPORT-2.1] Add recoverPartitions...

2016-12-21 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16372 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #16380: [SPARK-18972][Core]Fix the netty thread names for RPC

2016-12-21 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16380 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16322: [SPARK-18908][SS] Creating StreamingQueryException shoul...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16322 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70502/ Test PASSed. ---

[GitHub] spark issue #16322: [SPARK-18908][SS] Creating StreamingQueryException shoul...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16322 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16344: [SPARK-18929][ML] Add Tweedie distribution in GLM

2016-12-21 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/16344 @srowen @yanboliang Thanks much for the feedback. I now have a better understanding of the code and the issue. I have made new commits reflecting your suggestions. The major changes are

[GitHub] spark issue #16322: [SPARK-18908][SS] Creating StreamingQueryException shoul...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16322 **[Test build #70502 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70502/testReport)** for PR 16322 at commit

[GitHub] spark pull request #16380: [SPARK-18972][Core]Fix the netty thread names for...

2016-12-21 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16380#discussion_r93566371 --- Diff: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala --- @@ -315,6 +315,10 @@ final class ShuffleBlockFetcherIterator(

[GitHub] spark pull request #16380: [SPARK-18972][Core]Fix the netty thread names for...

2016-12-21 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16380#discussion_r93566199 --- Diff: common/network-common/src/main/java/org/apache/spark/network/server/TransportChannelHandler.java --- @@ -88,29 +88,29 @@ public void

[GitHub] spark issue #15505: [SPARK-17931][CORE] taskScheduler has some unneeded seri...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15505 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15505: [SPARK-17931][CORE] taskScheduler has some unneeded seri...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15505 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70501/ Test FAILed. ---

[GitHub] spark issue #16291: [SPARK-18838][CORE] Use separate executor service for ea...

2016-12-21 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/16291 @markhamstra, @vanzin thanks for your comments! >> so it is entirely possible. for example, for one ListenerEventExecutor to process a task end event for a particular task before another

[GitHub] spark issue #15505: [SPARK-17931][CORE] taskScheduler has some unneeded seri...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15505 **[Test build #70501 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70501/testReport)** for PR 15505 at commit

[GitHub] spark issue #16366: [SPARK-18953][CORE][WEB UI] Do now show the link to a de...

2016-12-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16366 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #16344: [SPARK-18929][ML] Add Tweedie distribution in GLM

2016-12-21 Thread actuaryzhang
Github user actuaryzhang commented on a diff in the pull request: https://github.com/apache/spark/pull/16344#discussion_r93565567 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -303,14 +341,15 @@ object

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16370 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #16344: [SPARK-18929][ML] Add Tweedie distribution in GLM

2016-12-21 Thread actuaryzhang
Github user actuaryzhang commented on a diff in the pull request: https://github.com/apache/spark/pull/16344#discussion_r93565335 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -64,6 +64,27 @@ private[regression] trait

[GitHub] spark pull request #12436: [SPARK-14649][CORE] DagScheduler should not run d...

2016-12-21 Thread sitalkedia
Github user sitalkedia closed the pull request at: https://github.com/apache/spark/pull/12436 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #12436: [SPARK-14649][CORE] DagScheduler should not run duplicat...

2016-12-21 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/12436 @kayousterhout - Thanks for taking a look at the PR. Currently I don't have time to work on it. I will close the PR and open a new PR with issues addressed. --- If your project is set up for

[GitHub] spark issue #16381: [SPARK-18973][SQL] Remove SortPartitions and Redistribut...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16381 **[Test build #70509 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70509/testReport)** for PR 16381 at commit

[GitHub] spark issue #16381: [SPARK-18973][SQL] Remove SortPartitions and Redistribut...

2016-12-21 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16381 Please also merge this into branch-2.1 to minimize backport conflicts ... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #16381: [SPARK-18973][SQL] Remove SortPartitions and Redi...

2016-12-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/16381#discussion_r93564401 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/partitioning.scala --- @@ -1,49 +0,0 @@ -/* - * Licensed to the

[GitHub] spark pull request #16381: [SPARK-18973][SQL] Remove SortPartitions and Redi...

2016-12-21 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/16381 [SPARK-18973][SQL] Remove SortPartitions and RedistributeData ## What changes were proposed in this pull request? SortPartitions and RedistributeData logical operators are not actually used and

[GitHub] spark issue #16380: [SPARK-18972][Core]Fix the netty thread names for RPC

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16380 **[Test build #70508 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70508/testReport)** for PR 16380 at commit

[GitHub] spark pull request #16380: [SPARK-18972][Core]Fix the netty thread names for...

2016-12-21 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/16380 [SPARK-18972][Core]Fix the netty thread names for RPC ## What changes were proposed in this pull request? Right now the name of threads created by Netty for Spark RPC are

[GitHub] spark issue #15505: [SPARK-17931][CORE] taskScheduler has some unneeded seri...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15505 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15505: [SPARK-17931][CORE] taskScheduler has some unneeded seri...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15505 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70507/ Test FAILed. ---

[GitHub] spark issue #15505: [SPARK-17931][CORE] taskScheduler has some unneeded seri...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15505 **[Test build #70507 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70507/testReport)** for PR 15505 at commit

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16294 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70500/ Test FAILed. ---

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16294 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16294 **[Test build #70500 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70500/testReport)** for PR 16294 at commit

[GitHub] spark issue #16053: [SPARK-17931] Eliminate unncessary task (de) serializati...

2016-12-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/16053 I think the code change in core/ looks pretty good to me. Is there someone who can look at the mesos changes ? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #16053: [SPARK-17931] Eliminate unncessary task (de) serializati...

2016-12-21 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/16053 @kayousterhout it shows there are some conflicts ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15505: [SPARK-17931][CORE] taskScheduler has some unneeded seri...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15505 **[Test build #70507 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70507/testReport)** for PR 15505 at commit

[GitHub] spark issue #16053: [SPARK-17931] Eliminate unncessary task (de) serializati...

2016-12-21 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/16053 I fixed all of the issues mentioned with the commits I pushed a few days ago, in case anyone was waiting for that and didn't notice! --- If your project is set up for it, you can reply to

[GitHub] spark issue #15505: [SPARK-17931][CORE] taskScheduler has some unneeded seri...

2016-12-21 Thread witgo
Github user witgo commented on the issue: https://github.com/apache/spark/pull/15505 @kayousterhout @squito I think Kay's approach is a good idea. We can first merging #16053, SPARK-18890 related code(including multi-threaded serialization TaskDescription) to stay in the

[GitHub] spark issue #16350: [SPARK-18700][SQL][BACKPORT-2.0] Add StripedLock for eac...

2016-12-21 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/16350 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #16350: [SPARK-18700][SQL][BACKPORT-2.0] Add StripedLock ...

2016-12-21 Thread xuanyuanking
Github user xuanyuanking closed the pull request at: https://github.com/apache/spark/pull/16350 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #16379: [SPARK-18969][SQL] Support grouping by nondetermi...

2016-12-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/16379#discussion_r93560851 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/PullOutNondeterministicSuite.scala --- @@ -0,0 +1,56 @@ +/* + * Licensed

[GitHub] spark issue #16378: [SQL] Minor readability improvement for partition handli...

2016-12-21 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16378 cc @cloud-fan too --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16253: [SPARK-18537][Web UI] Add a REST api to serve spark stre...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16253 **[Test build #70506 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70506/testReport)** for PR 16253 at commit

[GitHub] spark pull request #16253: [SPARK-18537][Web UI] Add a REST api to serve spa...

2016-12-21 Thread saturday-shi
Github user saturday-shi commented on a diff in the pull request: https://github.com/apache/spark/pull/16253#discussion_r93560749 --- Diff: streaming/src/main/scala/org/apache/spark/status/api/v1/streaming/api.scala --- @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #16323: [SPARK-18911] [SQL] Define CatalogStatistics to interact...

2016-12-21 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/16323 Updated. Please review @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16323: [SPARK-18911] [SQL] Define CatalogStatistics to interact...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16323 **[Test build #70505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70505/testReport)** for PR 16323 at commit

[GitHub] spark issue #16371: [SPARK-18932][SQL] Support partial aggregation for colle...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16371 **[Test build #70504 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70504/testReport)** for PR 16371 at commit

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13599 **[Test build #70503 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70503/testReport)** for PR 13599 at commit

[GitHub] spark issue #16322: [SPARK-18908][SS] Creating StreamingQueryException shoul...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16322 **[Test build #70502 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70502/testReport)** for PR 16322 at commit

[GitHub] spark pull request #15505: [SPARK-17931][CORE] taskScheduler has some unneed...

2016-12-21 Thread witgo
Github user witgo commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r93558564 --- Diff: core/src/main/scala/org/apache/spark/scheduler/local/LocalSchedulerBackend.scala --- @@ -59,6 +62,12 @@ private[spark] class LocalEndpoint(

[GitHub] spark pull request #15505: [SPARK-17931][CORE] taskScheduler has some unneed...

2016-12-21 Thread witgo
Github user witgo commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r93558472 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala --- @@ -17,27 +17,179 @@ package org.apache.spark.scheduler

[GitHub] spark pull request #15505: [SPARK-17931][CORE] taskScheduler has some unneed...

2016-12-21 Thread witgo
Github user witgo commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r93558527 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala --- @@ -17,27 +17,179 @@ package org.apache.spark.scheduler

[GitHub] spark pull request #15505: [SPARK-17931][CORE] taskScheduler has some unneed...

2016-12-21 Thread witgo
Github user witgo commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r93558450 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -993,6 +993,12 @@ class DAGScheduler(

[GitHub] spark pull request #15505: [SPARK-17931][CORE] taskScheduler has some unneed...

2016-12-21 Thread witgo
Github user witgo commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r93558431 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -232,6 +225,13 @@ private[spark] class Executor( }

[GitHub] spark pull request #15505: [SPARK-17931][CORE] taskScheduler has some unneed...

2016-12-21 Thread witgo
Github user witgo commented on a diff in the pull request: https://github.com/apache/spark/pull/15505#discussion_r93558401 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -136,14 +136,10 @@ private[spark] class Executor( startDriverHeartbeater()

[GitHub] spark issue #15505: [SPARK-17931][CORE] taskScheduler has some unneeded seri...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15505 **[Test build #70501 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70501/testReport)** for PR 15505 at commit

[GitHub] spark pull request #16119: [SPARK-18687][Pyspark][SQL]Backward compatibility...

2016-12-21 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16119#discussion_r93556600 --- Diff: python/pyspark/sql/context.py --- @@ -72,8 +72,13 @@ def __init__(self, sparkContext, sparkSession=None, jsqlContext=None): self._sc =

[GitHub] spark issue #16294: [SPARK-18669][SS][DOCS] Update Apache docs for Structure...

2016-12-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16294 **[Test build #70500 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70500/testReport)** for PR 16294 at commit

[GitHub] spark pull request #16119: [SPARK-18687][Pyspark][SQL]Backward compatibility...

2016-12-21 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16119#discussion_r93556537 --- Diff: python/pyspark/sql/context.py --- @@ -72,8 +72,13 @@ def __init__(self, sparkContext, sparkSession=None, jsqlContext=None): self._sc =

[GitHub] spark issue #16378: [SQL] Minor readability improvement for partition handli...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16378 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70497/ Test PASSed. ---

[GitHub] spark issue #16378: [SQL] Minor readability improvement for partition handli...

2016-12-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16378 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

  1   2   3   4   5   >