spark git commit: [SPARK-12031][CORE][BUG] Integer overflow when do sampling

Repository: spark Updated Branches: refs/heads/branch-1.6 0be792aad -> b5a76b4a4 [SPARK-12031][CORE][BUG] Integer overflow when do sampling Author: uncleGen Closes #10023 from uncleGen/1.6-bugfix. (cherry picked from commit a113216865fd45ea39ae8f104e784af2cf667dcf)

spark git commit: [SPARK-12012][SQL] Show more comprehensive PhysicalRDD metadata when visualizing SQL query plan

Repository: spark Updated Branches: refs/heads/master a11321686 -> 6e1c55eac [SPARK-12012][SQL] Show more comprehensive PhysicalRDD metadata when visualizing SQL query plan This PR adds a `private[sql]` method `metadata` to `SparkPlan`, which can be used to describe detail information about

spark git commit: [SPARK-10299][ML] word2vec should allow users to specify the window size

Repository: spark Updated Branches: refs/heads/branch-1.6 b5a76b4a4 -> acd462420 [SPARK-10299][ML] word2vec should allow users to specify the window size Currently word2vec has the window hard coded at 5, some users may want different sizes (for example if using on n-gram input or similar).

spark git commit: [SPARK-10299][ML] word2vec should allow users to specify the window size

Repository: spark Updated Branches: refs/heads/master 6e1c55eac -> 22b9a8740 [SPARK-10299][ML] word2vec should allow users to specify the window size Currently word2vec has the window hard coded at 5, some users may want different sizes (for example if using on n-gram input or similar). User

spark git commit: [SPARK-10582][YARN][CORE] Fix AM failure situation for dynamic allocation

Repository: spark Updated Branches: refs/heads/master 22b9a8740 -> 6900f0173 [SPARK-10582][YARN][CORE] Fix AM failure situation for dynamic allocation Because of AM failure, the target executor number between driver and AM will be different, which will lead to unexpected behavior in dynamic

spark git commit: [SPARK-12241][YARN] Improve failure reporting in Yarn client obtainTokenForHBase()

Repository: spark Updated Branches: refs/heads/master 6900f0173 -> 442a7715a [SPARK-12241][YARN] Improve failure reporting in Yarn client obtainTokenForHBase() This lines up the HBase token logic with that done for Hive in SPARK-11265: reflection with only CFNE being swallowed. There is a

spark git commit: [SPARK-11678][SQL][DOCS] Document basePath in the programming guide.

Repository: spark Updated Branches: refs/heads/branch-1.6 d86a88da6 -> 9fe8dc916 [SPARK-11678][SQL][DOCS] Document basePath in the programming guide. This PR adds document for `basePath`, which is a new parameter used by `HadoopFsRelation`. The compiled doc is shown below.

spark git commit: [SPARK-11678][SQL][DOCS] Document basePath in the programming guide.

Repository: spark Updated Branches: refs/heads/master 8770bd121 -> ac8cdf1cd [SPARK-11678][SQL][DOCS] Document basePath in the programming guide. This PR adds document for `basePath`, which is a new parameter used by `HadoopFsRelation`. The compiled doc is shown below.

spark git commit: [SPARK-12211][DOC][GRAPHX] Fix version number in graphx doc for migration from 1.1

Repository: spark Updated Branches: refs/heads/master 051c6a066 -> 7a8e587dc [SPARK-12211][DOC][GRAPHX] Fix version number in graphx doc for migration from 1.1 Migration from 1.1 section added to the GraphX doc in 1.2.0 (see

spark git commit: [SPARK-12211][DOC][GRAPHX] Fix version number in graphx doc for migration from 1.1

Repository: spark Updated Branches: refs/heads/branch-1.6 bfb420139 -> 9bc6a27fd [SPARK-12211][DOC][GRAPHX] Fix version number in graphx doc for migration from 1.1 Migration from 1.1 section added to the GraphX doc in 1.2.0 (see

spark git commit: [SPARK-12165][ADDENDUM] Fix outdated comments on unroll test

Repository: spark Updated Branches: refs/heads/master 7a8e587dc -> 8770bd121 [SPARK-12165][ADDENDUM] Fix outdated comments on unroll test JoshRosen Author: Andrew Or Closes #10229 from andrewor14/unroll-test-comments. Project:

spark git commit: [SPARK-12165][ADDENDUM] Fix outdated comments on unroll test

Repository: spark Updated Branches: refs/heads/branch-1.6 9bc6a27fd -> d86a88da6 [SPARK-12165][ADDENDUM] Fix outdated comments on unroll test JoshRosen Author: Andrew Or Closes #10229 from andrewor14/unroll-test-comments. (cherry picked from commit

[1/2] spark git commit: [SPARK-12244][SPARK-12245][STREAMING] Rename trackStateByKey to mapWithState and change tracking function signature

Repository: spark Updated Branches: refs/heads/branch-1.6 699f497cf -> f6d866173 http://git-wip-us.apache.org/repos/asf/spark/blob/f6d86617/streaming/src/test/java/org/apache/spark/streaming/JavaTrackStateByKeySuite.java --

[2/2] spark git commit: [SPARK-12244][SPARK-12245][STREAMING] Rename trackStateByKey to mapWithState and change tracking function signature

[SPARK-12244][SPARK-12245][STREAMING] Rename trackStateByKey to mapWithState and change tracking function signature SPARK-12244: Based on feedback from early users and personal experience attempting to explain it, the name trackStateByKey had two problem. "trackState" is a completely new term

spark git commit: [SPARK-11796] Fix httpclient and httpcore depedency issues related to docker-client

Repository: spark Updated Branches: refs/heads/master ac8cdf1cd -> 2166c2a75 [SPARK-11796] Fix httpclient and httpcore depedency issues related to docker-client This commit fixes dependency issues which prevented the Docker-based JDBC integration tests from running in the Maven build.

[2/2] spark git commit: [SPARK-12244][SPARK-12245][STREAMING] Rename trackStateByKey to mapWithState and change tracking function signature

[SPARK-12244][SPARK-12245][STREAMING] Rename trackStateByKey to mapWithState and change tracking function signature SPARK-12244: Based on feedback from early users and personal experience attempting to explain it, the name trackStateByKey had two problem. "trackState" is a completely new term

[1/2] spark git commit: [SPARK-12244][SPARK-12245][STREAMING] Rename trackStateByKey to mapWithState and change tracking function signature

Repository: spark Updated Branches: refs/heads/master 2166c2a75 -> bd2cd4f53 http://git-wip-us.apache.org/repos/asf/spark/blob/bd2cd4f5/streaming/src/test/java/org/apache/spark/streaming/JavaTrackStateByKeySuite.java -- diff

spark git commit: [SPARK-12252][SPARK-12131][SQL] refactor MapObjects to make it less hacky

Repository: spark Updated Branches: refs/heads/master bd2cd4f53 -> d8ec081c9 [SPARK-12252][SPARK-12131][SQL] refactor MapObjects to make it less hacky in https://github.com/apache/spark/pull/10133 we found that, we shoud ensure the children of `TreeNode` are all accessible in the

spark git commit: [SPARK-12165][SPARK-12189] Fix bugs in eviction of storage memory by execution

Repository: spark Updated Branches: refs/heads/master 442a7715a -> aec5ea000 [SPARK-12165][SPARK-12189] Fix bugs in eviction of storage memory by execution This patch fixes a bug in the eviction of storage memory by execution. ## The bug: In general, execution should be able to evict

spark git commit: [SPARK-12165][SPARK-12189] Fix bugs in eviction of storage memory by execution

Repository: spark Updated Branches: refs/heads/branch-1.6 acd462420 -> 05e441e12 [SPARK-12165][SPARK-12189] Fix bugs in eviction of storage memory by execution This patch fixes a bug in the eviction of storage memory by execution. ## The bug: In general, execution should be able to evict

spark git commit: [SPARK-11824][WEBUI] WebUI does not render descriptions with 'bad' HTML, throws console error

Repository: spark Updated Branches: refs/heads/branch-1.6 05e441e12 -> ee0a6e722 [SPARK-11824][WEBUI] WebUI does not render descriptions with 'bad' HTML, throws console error Don't warn when description isn't valid HTML since it may properly be like "SELECT ... where foo <= 1" The tests

[3/3] spark git commit: [SPARK-11551][DOC] Replace example code in ml-features.md using include_example

[SPARK-11551][DOC] Replace example code in ml-features.md using include_example PR on behalf of somideshmukh, thanks! Author: Xusen Yin Author: somideshmukh Closes #10219 from yinxusen/SPARK-11551. (cherry picked from commit

[3/3] spark git commit: [SPARK-11551][DOC] Replace example code in ml-features.md using include_example

[SPARK-11551][DOC] Replace example code in ml-features.md using include_example PR on behalf of somideshmukh, thanks! Author: Xusen Yin Author: somideshmukh Closes #10219 from yinxusen/SPARK-11551. Project:

[1/3] spark git commit: [SPARK-11551][DOC] Replace example code in ml-features.md using include_example

Repository: spark Updated Branches: refs/heads/master 1eb7c22ce -> 051c6a066 http://git-wip-us.apache.org/repos/asf/spark/blob/051c6a06/examples/src/main/scala/org/apache/spark/examples/ml/StringIndexerExample.scala -- diff

[2/3] spark git commit: [SPARK-11551][DOC] Replace example code in ml-features.md using include_example

http://git-wip-us.apache.org/repos/asf/spark/blob/051c6a06/examples/src/main/java/org/apache/spark/examples/ml/JavaPolynomialExpansionExample.java -- diff --git

[1/3] spark git commit: [SPARK-11551][DOC] Replace example code in ml-features.md using include_example

Repository: spark Updated Branches: refs/heads/branch-1.6 ee0a6e722 -> bfb420139 http://git-wip-us.apache.org/repos/asf/spark/blob/bfb42013/examples/src/main/scala/org/apache/spark/examples/ml/StringIndexerExample.scala --

[2/3] spark git commit: [SPARK-11551][DOC] Replace example code in ml-features.md using include_example

http://git-wip-us.apache.org/repos/asf/spark/blob/bfb42013/examples/src/main/java/org/apache/spark/examples/ml/JavaPolynomialExpansionExample.java -- diff --git