spark git commit: [SPARK-10749][MESOS] Support multiple roles with mesos cluster mode.

Repository: spark Updated Branches: refs/heads/master 40e6d40fe -> 00461bb91 [SPARK-10749][MESOS] Support multiple roles with mesos cluster mode. Currently the Mesos cluster dispatcher is not using offers from multiple roles correctly, as it simply aggregates all the offers resource values

spark git commit: [SPARK-12632][PYSPARK][DOC] PySpark fpm and als parameter desc to consistent format

Repository: spark Updated Branches: refs/heads/master 024482bf5 -> e298ac91e [SPARK-12632][PYSPARK][DOC] PySpark fpm and als parameter desc to consistent format Part of task for [SPARK-11219](https://issues.apache.org/jira/browse/SPARK-11219) to make PySpark MLlib parameter description

spark git commit: [SPARK-13334][ML] ML KMeansModel / BisectingKMeansModel / QuantileDiscretizer should set parent

Repository: spark Updated Branches: refs/heads/master e298ac91e -> 40e6d40fe [SPARK-13334][ML] ML KMeansModel / BisectingKMeansModel / QuantileDiscretizer should set parent ML ```KMeansModel / BisectingKMeansModel / QuantileDiscretizer``` should set parent. cc mengxr Author: Yanbo Liang

spark git commit: [SPARK-13429][MLLIB] Unify Logistic Regression convergence tolerance of ML & MLlib

Repository: spark Updated Branches: refs/heads/master 4fd199369 -> 72427c3e1 [SPARK-13429][MLLIB] Unify Logistic Regression convergence tolerance of ML & MLlib ## What changes were proposed in this pull request? In order to provide better and consistent result, let's change the default

spark git commit: [SPARK-13355][MLLIB] replace GraphImpl.fromExistingRDDs by Graph.apply

Repository: spark Updated Branches: refs/heads/master 72427c3e1 -> 764ca1803 [SPARK-13355][MLLIB] replace GraphImpl.fromExistingRDDs by Graph.apply `GraphImpl.fromExistingRDDs` expects preprocessed vertex RDD as input. We call it in LDA without validating this requirement. So it might

spark git commit: [SPARK-13355][MLLIB] replace GraphImpl.fromExistingRDDs by Graph.apply

Repository: spark Updated Branches: refs/heads/branch-1.6 d31854da5 -> 0784e02fd [SPARK-13355][MLLIB] replace GraphImpl.fromExistingRDDs by Graph.apply `GraphImpl.fromExistingRDDs` expects preprocessed vertex RDD as input. We call it in LDA without validating this requirement. So it might

spark git commit: [SPARK-13355][MLLIB] replace GraphImpl.fromExistingRDDs by Graph.apply

Repository: spark Updated Branches: refs/heads/branch-1.5 399e13d20 -> 4695eb832 [SPARK-13355][MLLIB] replace GraphImpl.fromExistingRDDs by Graph.apply `GraphImpl.fromExistingRDDs` expects preprocessed vertex RDD as input. We call it in LDA without validating this requirement. So it might

spark git commit: [SPARK-12746][ML] ArrayType(_, true) should also accept ArrayType(_, false) fix for branch-1.6

Repository: spark Updated Branches: refs/heads/branch-1.6 290279808 -> d31854da5 [SPARK-12746][ML] ArrayType(_, true) should also accept ArrayType(_, false) fix for branch-1.6 https://issues.apache.org/jira/browse/SPARK-13359 Author: Earthson Lu Closes #11237 from

spark git commit: [SPARK-13257][IMPROVEMENT] Refine naive Bayes example by checking model after loading it

Repository: spark Updated Branches: refs/heads/master 764ca1803 -> 5cd3e6f60 [SPARK-13257][IMPROVEMENT] Refine naive Bayes example by checking model after loading it Refine naive Bayes example by checking model after loading it Author: movelikeriver Closes #11125

spark git commit: [SPARK-12363][MLLIB][BACKPORT-1.4] Remove setRun and fix PowerIterationClustering failed test

Repository: spark Updated Branches: refs/heads/branch-1.4 c961c2788 -> 6598590f9 [SPARK-12363][MLLIB][BACKPORT-1.4] Remove setRun and fix PowerIterationClustering failed test JIRA: https://issues.apache.org/jira/browse/SPARK-12363 ## What changes were proposed in this pull request? Backport

spark git commit: [SPARK-13413] Remove SparkContext.metricsSystem

Repository: spark Updated Branches: refs/heads/master 00461bb91 -> 4a91806a4 [SPARK-13413] Remove SparkContext.metricsSystem ## What changes were proposed in this pull request? This patch removes SparkContext.metricsSystem. SparkContext.metricsSystem returns MetricsSystem, which is a

spark git commit: [SPARK-12546][SQL] Change default number of open parquet files

Repository: spark Updated Branches: refs/heads/master 4a91806a4 -> 173aa949c [SPARK-12546][SQL] Change default number of open parquet files A common problem that users encounter with Spark 1.6.0 is that writing to a partitioned parquet table OOMs. The root cause is that parquet allocates a

spark git commit: [SPARK-13422][SQL] Use HashedRelation instead of HashSet in Left Semi Joins

Repository: spark Updated Branches: refs/heads/master 173aa949c -> 206378184 [SPARK-13422][SQL] Use HashedRelation instead of HashSet in Left Semi Joins Use the HashedRelation which is a more optimized datastructure and reduce code complexity Author: Xiu Guo Closes

spark git commit: [SPARK-13016][DOCUMENTATION] Replace example code in mllib-dimensionality-reduction.md using include_example

Repository: spark Updated Branches: refs/heads/master 206378184 -> 9f410871c [SPARK-13016][DOCUMENTATION] Replace example code in mllib-dimensionality-reduction.md using include_example Replaced example example code in mllib-dimensionality-reduction.md using include_example Author: Devaraj

spark git commit: [SPARK-12546][SQL] Change default number of open parquet files

Repository: spark Updated Branches: refs/heads/branch-1.6 16f35c4c6 -> 699644c69 [SPARK-12546][SQL] Change default number of open parquet files A common problem that users encounter with Spark 1.6.0 is that writing to a partitioned parquet table OOMs. The root cause is that parquet

spark git commit: [SPARK-13012][DOCUMENTATION] Replace example code in ml-guide.md using include_example

Repository: spark Updated Branches: refs/heads/master 9f410871c -> 02b1fefff [SPARK-13012][DOCUMENTATION] Replace example code in ml-guide.md using include_example Replaced example code in ml-guide.md using include_example Author: Devaraj K Closes #11053 from

spark git commit: [SPARK-13295][ ML, MLLIB ] AFTSurvivalRegression.AFTAggregator improvements - avoid creating new instances of arrays/vectors for each record

Repository: spark Updated Branches: refs/heads/master 02b1fefff -> 33ef3aa7e [SPARK-13295][ ML, MLLIB ] AFTSurvivalRegression.AFTAggregator improvements - avoid creating new instances of arrays/vectors for each record As also mentioned/marked by TODO in AFTAggregator.AFTAggregator.add(data:

spark git commit: [SPARK-13298][CORE][UI] Escape "label" to avoid DAG being broken by some special character

Repository: spark Updated Branches: refs/heads/master 33ef3aa7e -> a11b39951 [SPARK-13298][CORE][UI] Escape "label" to avoid DAG being broken by some special character ## What changes were proposed in this pull request? When there are some special characters (e.g., `"`, `\`) in `label`, DAG

spark git commit: [SPARK-13298][CORE][UI] Escape "label" to avoid DAG being broken by some special character

Repository: spark Updated Branches: refs/heads/branch-1.6 699644c69 -> 85e6a2205 [SPARK-13298][CORE][UI] Escape "label" to avoid DAG being broken by some special character ## What changes were proposed in this pull request? When there are some special characters (e.g., `"`, `\`) in `label`,

spark git commit: [SPARK-11624][SPARK-11972][SQL] fix commands that need hive to exec

Repository: spark Updated Branches: refs/heads/master a11b39951 -> 5d80fac58 [SPARK-11624][SPARK-11972][SQL] fix commands that need hive to exec In SparkSQLCLI, we have created a `CliSessionState`, but then we call `SparkSQLEnv.init()`, which will start another `SessionState`. This would

spark git commit: [SPARK-11624][SPARK-11972][SQL] fix commands that need hive to exec

Repository: spark Updated Branches: refs/heads/branch-1.6 85e6a2205 -> f7898f9e2 [SPARK-11624][SPARK-11972][SQL] fix commands that need hive to exec In SparkSQLCLI, we have created a `CliSessionState`, but then we call `SparkSQLEnv.init()`, which will start another `SessionState`. This would

spark git commit: Update branch-1.6 for 1.6.1 release

Repository: spark Updated Branches: refs/heads/branch-1.6 f7898f9e2 -> 40d11d049 Update branch-1.6 for 1.6.1 release Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/40d11d04 Tree:

[2/2] spark git commit: Preparing development version 1.6.1-SNAPSHOT

Preparing development version 1.6.1-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/29027980 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/29027980 Diff:

[1/2] spark git commit: Preparing Spark release v1.6.1-rc1

Repository: spark Updated Branches: refs/heads/branch-1.6 40d11d049 -> 290279808 Preparing Spark release v1.6.1-rc1 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/152252f1 Tree:

spark git commit: [SPARK-12723][SQL] Comprehensive Verification and Fixing of SQL Generation Support for Expressions

Repository: spark Updated Branches: refs/heads/master 5d80fac58 -> 9dd5399d7 [SPARK-12723][SQL] Comprehensive Verification and Fixing of SQL Generation Support for Expressions What changes were proposed in this pull request? Ensure that all built-in expressions can be mapped to its SQL

spark git commit: [SPARK-6761][SQL] Approximate quantile for DataFrame

Repository: spark Updated Branches: refs/heads/master 01e10c9fe -> 4fd199369 [SPARK-6761][SQL] Approximate quantile for DataFrame JIRA: https://issues.apache.org/jira/browse/SPARK-6761 Compute approximate quantile based on the paper Greenwald, Michael and Khanna, Sanjeev, "Space-efficient

spark git commit: [MINOR][DOCS] Fix all typos in markdown files of `doc` and similar patterns in other comments

Repository: spark Updated Branches: refs/heads/master 1b144455b -> 024482bf5 [MINOR][DOCS] Fix all typos in markdown files of `doc` and similar patterns in other comments ## What changes were proposed in this pull request? This PR tries to fix all typos in all markdown files under `docs`

spark git commit: [SPARK-13186][STREAMING] migrate away from SynchronizedMap

Repository: spark Updated Branches: refs/heads/master 39ff15457 -> 8f35d3eac [SPARK-13186][STREAMING] migrate away from SynchronizedMap trait SynchronizedMap in package mutable is deprecated: Synchronization via traits is deprecated as it is inherently unreliable. Change to

spark git commit: [SPARK-13399][STREAMING] Fix checkpointsuite type erasure warnings

Repository: spark Updated Branches: refs/heads/master ef1047fca -> 1b144455b [SPARK-13399][STREAMING] Fix checkpointsuite type erasure warnings ## What changes were proposed in this pull request? Change the checkpointsuite getting the outputstreams to explicitly be unchecked on the generic

spark git commit: [SPARK-12153][SPARK-7617][MLLIB] add support of arbitrary length sentence and other tuning for Word2Vec

Repository: spark Updated Branches: refs/heads/master 8f35d3eac -> ef1047fca [SPARK-12153][SPARK-7617][MLLIB] add support of arbitrary length sentence and other tuning for Word2Vec add support of arbitrary length sentence by using the nature representation of sentences in the input. add