[2/3] spark git commit: [SPARK-11551][DOC][EXAMPLE] Revert PR #10002

http://git-wip-us.apache.org/repos/asf/spark/blob/da2012a0/examples/src/main/java/org/apache/spark/examples/ml/JavaRFormulaExample.java -- diff --git a/examples/src/main/java/org/apache/spark/examples/ml/JavaRFormulaExample.java

[3/3] spark git commit: [SPARK-11551][DOC][EXAMPLE] Revert PR #10002

[SPARK-11551][DOC][EXAMPLE] Revert PR #10002 This reverts PR #10002, commit 78209b0ccaf3f22b5e2345dfb2b98edfdb746819. The original PR wasn't tested on Jenkins before being merged. Author: Cheng Lian Closes #10200 from liancheng/revert-pr-10002. Project:

spark git commit: [SPARK-12166][TEST] Unset hadoop related environment in testing

Repository: spark Updated Branches: refs/heads/branch-1.6 c9e5274ae -> 870f43562 [SPARK-12166][TEST] Unset hadoop related environment in testing Author: Jeff Zhang Closes #10172 from zjffdu/SPARK-12166. (cherry picked from commit 708129187a460aca30790281e9221c0cd5e271df)

[2/3] spark git commit: [SPARK-11551][DOC][EXAMPLE] Revert PR #10002

http://git-wip-us.apache.org/repos/asf/spark/blob/8a791a32/examples/src/main/java/org/apache/spark/examples/ml/JavaRFormulaExample.java -- diff --git a/examples/src/main/java/org/apache/spark/examples/ml/JavaRFormulaExample.java

[3/3] spark git commit: [SPARK-11551][DOC][EXAMPLE] Revert PR #10002

[SPARK-11551][DOC][EXAMPLE] Revert PR #10002 This reverts PR #10002, commit 78209b0ccaf3f22b5e2345dfb2b98edfdb746819. The original PR wasn't tested on Jenkins before being merged. Author: Cheng Lian Closes #10200 from liancheng/revert-pr-10002. (cherry picked from commit

[1/3] spark git commit: [SPARK-11551][DOC][EXAMPLE] Revert PR #10002

Repository: spark Updated Branches: refs/heads/branch-1.6 870f43562 -> 8a791a327 http://git-wip-us.apache.org/repos/asf/spark/blob/8a791a32/examples/src/main/scala/org/apache/spark/examples/ml/TokenizerExample.scala -- diff

spark git commit: [SPARK-12103][STREAMING][KAFKA][DOC] document that K means Key and V …

Repository: spark Updated Branches: refs/heads/branch-1.6 5c8216920 -> c9e5274ae [SPARK-12103][STREAMING][KAFKA][DOC] document that K means Key and V … …means Value Author: cody koeninger Closes #10132 from koeninger/SPARK-12103. (cherry picked from commit

spark git commit: [SPARK-12103][STREAMING][KAFKA][DOC] document that K means Key and V …

Repository: spark Updated Branches: refs/heads/master 4a39b5a1b -> 48a9804b2 [SPARK-12103][STREAMING][KAFKA][DOC] document that K means Key and V … …means Value Author: cody koeninger Closes #10132 from koeninger/SPARK-12103. Project:

spark git commit: [SPARK-11652][CORE] Remote code execution with InvokerTransformer

Repository: spark Updated Branches: refs/heads/branch-1.4 b6ba2dab2 -> c7c99857d [SPARK-11652][CORE] Remote code execution with InvokerTransformer Fix commons-collection group ID to commons-collections for version 3.x Patches earlier PR at https://github.com/apache/spark/pull/9731 Author:

spark git commit: [SPARK-11652][CORE] Remote code execution with InvokerTransformer

Repository: spark Updated Branches: refs/heads/branch-1.5 2f30927a5 -> 4b99f72f7 [SPARK-11652][CORE] Remote code execution with InvokerTransformer Fix commons-collection group ID to commons-collections for version 3.x Patches earlier PR at https://github.com/apache/spark/pull/9731 Author:

spark git commit: [SPARK-11652][CORE] Remote code execution with InvokerTransformer

Repository: spark Updated Branches: refs/heads/master da2012a0e -> e3735ce16 [SPARK-11652][CORE] Remote code execution with InvokerTransformer Fix commons-collection group ID to commons-collections for version 3.x Patches earlier PR at https://github.com/apache/spark/pull/9731 Author: Sean

spark git commit: [SPARK-11652][CORE] Remote code execution with InvokerTransformer

Repository: spark Updated Branches: refs/heads/branch-1.6 8a791a327 -> c8f9eb749 [SPARK-11652][CORE] Remote code execution with InvokerTransformer Fix commons-collection group ID to commons-collections for version 3.x Patches earlier PR at https://github.com/apache/spark/pull/9731 Author:

spark git commit: [SPARK-10393] use ML pipeline in LDA example

Repository: spark Updated Branches: refs/heads/master 5d96a710a -> 872a2ee28 [SPARK-10393] use ML pipeline in LDA example jira: https://issues.apache.org/jira/browse/SPARK-10393 Since the logic of the text processing part has been moved to ML estimators/transformers, replace the related

spark git commit: [SPARK-11155][WEB UI] Stage summary json should include stage duration

Repository: spark Updated Branches: refs/heads/master e3735ce16 -> 6cb06e871 [SPARK-11155][WEB UI] Stage summary json should include stage duration The json endpoint for stages doesn't include information on the stage duration that is present in the UI. This looks like a simple oversight,

spark git commit: [SPARK-12074] Avoid memory copy involving ByteBuffer.wrap(ByteArrayOutputStream.toByteArray)

Repository: spark Updated Branches: refs/heads/master 6cb06e871 -> 75c60bf4b [SPARK-12074] Avoid memory copy involving ByteBuffer.wrap(ByteArrayOutputStream.toByteArray) SPARK-12060 fixed JavaSerializerInstance.serialize This PR applies the same technique on two other classes. zsxwing

spark git commit: [SPARK-12195][SQL] Adding BigDecimal, Date and Timestamp into Encoder

Repository: spark Updated Branches: refs/heads/master 381f17b54 -> c0b13d556 [SPARK-12195][SQL] Adding BigDecimal, Date and Timestamp into Encoder This PR is to add three more data types into Encoder, including `BigDecimal`, `Date` and `Timestamp`. marmbrus cloud-fan rxin Could you take a

spark git commit: [SPARK-12201][SQL] add type coercion rule for greatest/least

Repository: spark Updated Branches: refs/heads/master 75c60bf4b -> 381f17b54 [SPARK-12201][SQL] add type coercion rule for greatest/least checked with hive, greatest/least should cast their children to a tightest common type, i.e. `(int, long) => long`, `(int, string) => error`,

spark git commit: [SPARK-12201][SQL] add type coercion rule for greatest/least

Repository: spark Updated Branches: refs/heads/branch-1.6 c8f9eb749 -> 8ef33aa1f [SPARK-12201][SQL] add type coercion rule for greatest/least checked with hive, greatest/least should cast their children to a tightest common type, i.e. `(int, long) => long`, `(int, string) => error`,

spark git commit: [SPARK-12188][SQL] Code refactoring and comment correction in Dataset APIs

Repository: spark Updated Branches: refs/heads/master c0b13d556 -> 5d96a710a [SPARK-12188][SQL] Code refactoring and comment correction in Dataset APIs This PR contains the following updates: - Created a new private variable `boundTEncoder` that can be shared by multiple functions, `RDD`,

spark git commit: [SPARK-12188][SQL] Code refactoring and comment correction in Dataset APIs

Repository: spark Updated Branches: refs/heads/branch-1.6 9eeb0f25a -> be0fe9b45 [SPARK-12188][SQL] Code refactoring and comment correction in Dataset APIs This PR contains the following updates: - Created a new private variable `boundTEncoder` that can be shared by multiple functions,

spark git commit: [SPARK-10393] use ML pipeline in LDA example

Repository: spark Updated Branches: refs/heads/branch-1.6 be0fe9b45 -> 1c8451b5e [SPARK-10393] use ML pipeline in LDA example jira: https://issues.apache.org/jira/browse/SPARK-10393 Since the logic of the text processing part has been moved to ML estimators/transformers, replace the related

spark git commit: [SPARK-12195][SQL] Adding BigDecimal, Date and Timestamp into Encoder

Repository: spark Updated Branches: refs/heads/branch-1.6 8ef33aa1f -> 9eeb0f25a [SPARK-12195][SQL] Adding BigDecimal, Date and Timestamp into Encoder This PR is to add three more data types into Encoder, including `BigDecimal`, `Date` and `Timestamp`. marmbrus cloud-fan rxin Could you take

spark git commit: [SPARK-12205][SQL] Pivot fails Analysis when aggregate is UnresolvedFunction

Repository: spark Updated Branches: refs/heads/master 872a2ee28 -> 4bcb89494 [SPARK-12205][SQL] Pivot fails Analysis when aggregate is UnresolvedFunction Delays application of ResolvePivot until all aggregates are resolved to prevent problems with UnresolvedFunction and adds unit test

spark git commit: [SPARK-11605][MLLIB] ML 1.6 QA: API: Java compatibility, docs

Repository: spark Updated Branches: refs/heads/master 4bcb89494 -> 5cb469505 [SPARK-11605][MLLIB] ML 1.6 QA: API: Java compatibility, docs jira: https://issues.apache.org/jira/browse/SPARK-11605 Check Java compatibility for MLlib for this release. fix: 1. `StreamingTest.registerStream`

spark git commit: [SPARK-11605][MLLIB] ML 1.6 QA: API: Java compatibility, docs

Repository: spark Updated Branches: refs/heads/branch-1.6 9145bfb81 -> 7e45feb00 [SPARK-11605][MLLIB] ML 1.6 QA: API: Java compatibility, docs jira: https://issues.apache.org/jira/browse/SPARK-11605 Check Java compatibility for MLlib for this release. fix: 1. `StreamingTest.registerStream`

spark git commit: [SPARK-12205][SQL] Pivot fails Analysis when aggregate is UnresolvedFunction

Repository: spark Updated Branches: refs/heads/branch-1.6 1c8451b5e -> 9145bfb81 [SPARK-12205][SQL] Pivot fails Analysis when aggregate is UnresolvedFunction Delays application of ResolvePivot until all aggregates are resolved to prevent problems with UnresolvedFunction and adds unit test

spark git commit: [SPARK-12159][ML] Add user guide section for IndexToString transformer

Repository: spark Updated Branches: refs/heads/branch-1.6 7e45feb00 -> 3e31e7e24 [SPARK-12159][ML] Add user guide section for IndexToString transformer Documentation regarding the `IndexToString` label transformer with code snippets in Scala/Java/Python. Author: BenFradet

spark git commit: [SPARK-12187] *MemoryPool classes should not be fully public

Repository: spark Updated Branches: refs/heads/master 2ff17bcfb -> 949452169 [SPARK-12187] *MemoryPool classes should not be fully public This patch tightens them to `private[memory]`. Author: Andrew Or Closes #10182 from andrewor14/memory-visibility. Project:

spark git commit: [SPARK-12187] *MemoryPool classes should not be fully public

Repository: spark Updated Branches: refs/heads/branch-1.6 3e31e7e24 -> 25249d1ec [SPARK-12187] *MemoryPool classes should not be fully public This patch tightens them to `private[memory]`. Author: Andrew Or Closes #10182 from andrewor14/memory-visibility. (cherry

spark git commit: [SPARK-12069][SQL] Update documentation with Datasets

Repository: spark Updated Branches: refs/heads/master 949452169 -> 395948942 [SPARK-12069][SQL] Update documentation with Datasets Author: Michael Armbrust Closes #10060 from marmbrus/docs. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-12069][SQL] Update documentation with Datasets

Repository: spark Updated Branches: refs/heads/branch-1.6 25249d1ec -> 2a5e4d157 [SPARK-12069][SQL] Update documentation with Datasets Author: Michael Armbrust Closes #10060 from marmbrus/docs. (cherry picked from commit 39594894232e0b70c5ca8b0df137da0d61223fd5)

spark git commit: [SPARK-8517][ML][DOC] Reorganizes the spark.ml user guide

Repository: spark Updated Branches: refs/heads/master 395948942 -> 765c67f5f [SPARK-8517][ML][DOC] Reorganizes the spark.ml user guide This PR moves pieces of the spark.ml user guide to reflect suggestions in SPARK-8517. It does not introduce new content, as requested.

spark git commit: [SPARK-11676][SQL] Parquet filter tests all pass if filters are not really pushed down

Repository: spark Updated Branches: refs/heads/master 3934562d3 -> f6883bb7a [SPARK-11676][SQL] Parquet filter tests all pass if filters are not really pushed down Currently Parquet predicate tests all pass even if filters are not pushed down or this is disabled. In this PR, For checking

spark git commit: [SPARK-12222] [CORE] Deserialize RoaringBitmap using Kryo serializer throw Buffer underflow exception

Repository: spark Updated Branches: refs/heads/branch-1.6 9e82273af -> 0be792aad [SPARK-1] [CORE] Deserialize RoaringBitmap using Kryo serializer throw Buffer underflow exception Jira: https://issues.apache.org/jira/browse/SPARK-1 Deserialize RoaringBitmap using Kryo serializer

spark git commit: [SPARK-8517][ML][DOC] Reorganizes the spark.ml user guide

Repository: spark Updated Branches: refs/heads/branch-1.6 2a5e4d157 -> b1d5a7859 [SPARK-8517][ML][DOC] Reorganizes the spark.ml user guide This PR moves pieces of the spark.ml user guide to reflect suggestions in SPARK-8517. It does not introduce new content, as requested.

spark git commit: [SPARK-11343][ML] Documentation of float and double prediction/label columns in RegressionEvaluator

Repository: spark Updated Branches: refs/heads/branch-1.6 b1d5a7859 -> 9e82273af [SPARK-11343][ML] Documentation of float and double prediction/label columns in RegressionEvaluator felixcheung , mengxr Just added a message to require() Author: Dominik Dahlem

spark git commit: [SPARK-12222] [CORE] Deserialize RoaringBitmap using Kryo serializer throw Buffer underflow exception

Repository: spark Updated Branches: refs/heads/master a0046e379 -> 3934562d3 [SPARK-1] [CORE] Deserialize RoaringBitmap using Kryo serializer throw Buffer underflow exception Jira: https://issues.apache.org/jira/browse/SPARK-1 Deserialize RoaringBitmap using Kryo serializer throw

spark git commit: [SPARK-3873][BUILD] Add style checker to enforce import ordering.

Repository: spark Updated Branches: refs/heads/master 06746b300 -> 2ff17bcfb [SPARK-3873][BUILD] Add style checker to enforce import ordering. The checker tries to follow as closely as possible the guidelines of the code style document, and makes some decisions where the guide is not clear.