spark git commit: [SPARK-16870][DOCS] Summary:add "spark.sql.broadcastTimeout" into docs/sql-programming-gu…

2016-08-07 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 c0364485e -> 3f8a95be8 [SPARK-16870][DOCS] Summary:add "spark.sql.broadcastTimeout" into docs/sql-programming-gu… ## What changes were proposed in this pull request? default value for spark.sql.broadcastTimeout is 300s. and this prop

spark git commit: [SPARK-16911] Fix the links in the programming guide

2016-08-07 Thread srowen
Repository: spark Updated Branches: refs/heads/master 1275f6469 -> 6c1ecb191 [SPARK-16911] Fix the links in the programming guide ## What changes were proposed in this pull request? Fix the broken links in the programming guide of the Graphx Migration and understanding closures ## How was

spark git commit: [SPARK-16911] Fix the links in the programming guide

2016-08-07 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 3f8a95be8 -> 739a333f6 [SPARK-16911] Fix the links in the programming guide ## What changes were proposed in this pull request? Fix the broken links in the programming guide of the Graphx Migration and understanding closures ## How

spark git commit: [SPARK-16909][SPARK CORE] Streaming for postgreSQL JDBC driver

2016-08-07 Thread srowen
Repository: spark Updated Branches: refs/heads/master 6c1ecb191 -> bdfab9f94 [SPARK-16909][SPARK CORE] Streaming for postgreSQL JDBC driver As per the postgreSQL JDBC driver [implementation](https://github.com/pgjdbc/pgjdbc/blob/ab2a6d89081fc2c1fdb2a8600f413db33669022c/pgjdbc/src/main/java/or

spark git commit: [SPARK-16409][SQL] regexp_extract with optional groups causes NPE

2016-08-07 Thread srowen
but the requested optional group did not. This makes it return an empty string, as apparently designed. ## How was this patch tested? Additional unit test Author: Sean Owen Closes #14504 from srowen/SPARK-16409. (cherry picked from commit 8d8725208771a8815a60160a5a30dc6ea87a7e6a) Signed-off-by: S

spark git commit: [SPARK-16409][SQL] regexp_extract with optional groups causes NPE

2016-08-07 Thread srowen
equested optional group did not. This makes it return an empty string, as apparently designed. ## How was this patch tested? Additional unit test Author: Sean Owen Closes #14504 from srowen/SPARK-16409. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.ap

spark git commit: [SPARK-16409][SQL] regexp_extract with optional groups causes NPE

2016-08-07 Thread srowen
but the requested optional group did not. This makes it return an empty string, as apparently designed. ## How was this patch tested? Additional unit test Author: Sean Owen Closes #14504 from srowen/SPARK-16409. (cherry picked from commit 8d8725208771a8815a60160a5a30dc6ea87a7e6a) Signed-off-by: S

spark git commit: [SPARK-16939][SQL] Fix build error by using `Tuple1` explicitly in StringFunctionsSuite

2016-08-07 Thread srowen
Repository: spark Updated Branches: refs/heads/master 8d8725208 -> a16983c97 [SPARK-16939][SQL] Fix build error by using `Tuple1` explicitly in StringFunctionsSuite ## What changes were proposed in this pull request? This PR aims to fix a build error on branch 1.6 at https://github.com/apac

spark git commit: [SPARK-16939][SQL] Fix build error by using `Tuple1` explicitly in StringFunctionsSuite

2016-08-07 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 fd828e14b -> f37ed6e70 [SPARK-16939][SQL] Fix build error by using `Tuple1` explicitly in StringFunctionsSuite ## What changes were proposed in this pull request? This PR aims to fix a build error on branch 1.6 at https://github.com/

spark git commit: [SPARK-16939][SQL] Fix build error by using `Tuple1` explicitly in StringFunctionsSuite

2016-08-07 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 1a5e762ff -> a3b06ae0b [SPARK-16939][SQL] Fix build error by using `Tuple1` explicitly in StringFunctionsSuite ## What changes were proposed in this pull request? This PR aims to fix a build error on branch 1.6 at https://github.com/

spark git commit: [SPARK-16919] Configurable update interval for console progress bar

2016-08-07 Thread srowen
Repository: spark Updated Branches: refs/heads/master a16983c97 -> e076fb05a [SPARK-16919] Configurable update interval for console progress bar ## What changes were proposed in this pull request? Currently the update interval for the console progress bar is hardcoded. This PR makes it confi

spark git commit: [SPARK-16945] Fix Java Lint errors

2016-08-08 Thread srowen
Repository: spark Updated Branches: refs/heads/master 1db1c6567 -> e10ca8de4 [SPARK-16945] Fix Java Lint errors ## What changes were proposed in this pull request? This PR is to fix the minor Java linter errors as following: [ERROR] src/main/java/org/apache/spark/sql/catalyst/expressions/Vari

spark git commit: [SPARK-16606][CORE] Misleading warning for SparkContext.getOrCreate "WARN SparkContext: Use an existing SparkContext, some configuration may not take effect."

2016-08-09 Thread srowen
pull request? SparkContext.getOrCreate shouldn't warn about ignored config if - it wasn't ignored because a new context is created with it or - no config was actually provided ## How was this patch tested? Jenkins + existing tests. Author: Sean Owen Closes #14533 from srowen/SPARK-16606

spark git commit: [SPARK-16522][MESOS] Spark application throws exception on exit.

2016-08-09 Thread srowen
Repository: spark Updated Branches: refs/heads/master 801e4d097 -> af710e5bd [SPARK-16522][MESOS] Spark application throws exception on exit. ## What changes were proposed in this pull request? Spark applications running on Mesos throw exception upon exit. For details, refer to https://issues

spark git commit: [SPARK-16940][SQL] `checkAnswer` should raise `TestFailedException` for wrong results

2016-08-09 Thread srowen
Repository: spark Updated Branches: refs/heads/master af710e5bd -> 2154345b6 [SPARK-16940][SQL] `checkAnswer` should raise `TestFailedException` for wrong results ## What changes were proposed in this pull request? This PR fixes the following to make `checkAnswer` raise `TestFailedException`

spark git commit: [SPARK-16809] enable history server links in dispatcher UI

2016-08-09 Thread srowen
Repository: spark Updated Branches: refs/heads/master 2154345b6 -> 62e621244 [SPARK-16809] enable history server links in dispatcher UI ## What changes were proposed in this pull request? Links the Spark Mesos Dispatcher UI to the history server UI - adds spark.mesos.dispatcher.historyServer

spark git commit: [SPARK-16522][MESOS] Spark application throws exception on exit.

2016-08-10 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 475ee3815 -> 2285de734 [SPARK-16522][MESOS] Spark application throws exception on exit. This is backport of https://github.com/apache/spark/pull/14175 to branch 2.0 Author: Sun Rui Closes #14575 from sun-rui/SPARK-16522-branch-2.0.

spark git commit: Typo: Fow -> For

2016-08-10 Thread srowen
Repository: spark Updated Branches: refs/heads/master 2b10ebe6a -> bfda53f63 Typo: Fow -> For Author: Andrew Ash Closes #14563 from ash211/patch-8. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/bfda53f6 Tree: http://g

spark git commit: [SPARK-16927][SPARK-16923] Override task properties at dispatcher.

2016-08-10 Thread srowen
Repository: spark Updated Branches: refs/heads/master bfda53f63 -> eca58755f [SPARK-16927][SPARK-16923] Override task properties at dispatcher. ## What changes were proposed in this pull request? - enable setting default properties for all jobs submitted through the dispatcher [SPARK-16927]

spark git commit: [SPARK-16324][SQL] regexp_extract should doc that it returns empty string when match fails

2016-08-10 Thread srowen
oes not match ## How was this patch tested? Jenkins test, with a few new test cases Author: Sean Owen Closes #14525 from srowen/SPARK-16324. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0578ff96 Tree: http://git-

spark git commit: [SPARK-16324][SQL] regexp_extract should doc that it returns empty string when match fails

2016-08-10 Thread srowen
oup does not match ## How was this patch tested? Jenkins test, with a few new test cases Author: Sean Owen Closes #14525 from srowen/SPARK-16324. (cherry picked from commit 0578ff9681edbaab4ae68f67272dc3d4d890d53b) Signed-off-by: Sean Owen Project: http://git-wip-us.apache.org/repos/asf/sp

spark git commit: [SPARK-15899][SQL] Fix the construction of the file path with hadoop Path

2016-08-10 Thread srowen
Repository: spark Updated Branches: refs/heads/master b9f8a1170 -> 11a6844be [SPARK-15899][SQL] Fix the construction of the file path with hadoop Path ## What changes were proposed in this pull request? Fix the construction of the file path. Previous way of construction caused the creation o

spark git commit: [SPARK-15899][SQL] Fix the construction of the file path with hadoop Path

2016-08-10 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 20efb7969 -> 719ac5f37 [SPARK-15899][SQL] Fix the construction of the file path with hadoop Path ## What changes were proposed in this pull request? Fix the construction of the file path. Previous way of construction caused the creati

spark git commit: Revert "[SPARK-15899][SQL] Fix the construction of the file path with hadoop Path"

2016-08-10 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 719ac5f37 -> 15637f735 Revert "[SPARK-15899][SQL] Fix the construction of the file path with hadoop Path" This reverts commit 719ac5f37ccf32c34c70524b8cf9a2699c71a353. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit

spark git commit: Correct example value for spark.ssl.YYY.XXX settings

2016-08-11 Thread srowen
Repository: spark Updated Branches: refs/heads/master a7b02db45 -> 8a6b7037b Correct example value for spark.ssl.YYY.XXX settings Docs adjustment to: - link to other relevant section of docs - correct statement about the only value when actually other values are supported Author: Andrew Ash

spark git commit: Correct example value for spark.ssl.YYY.XXX settings

2016-08-11 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 4b434e7da -> 0ed6236e9 Correct example value for spark.ssl.YYY.XXX settings Docs adjustment to: - link to other relevant section of docs - correct statement about the only value when actually other values are supported Author: Andrew

spark git commit: [SPARK-16941] Use concurrentHashMap instead of scala Map in SparkSQLOperationManager.

2016-08-11 Thread srowen
Repository: spark Updated Branches: refs/heads/master 8a6b7037b -> a45fefd17 [SPARK-16941] Use concurrentHashMap instead of scala Map in SparkSQLOperationManager. ## What changes were proposed in this pull request? ThriftServer will have some thread-safe problem in **SparkSQLOperationManager*

spark git commit: [SPARK-16886][EXAMPLES][DOC] Fix some examples to be consistent and indentation in documentation

2016-08-11 Thread srowen
Repository: spark Updated Branches: refs/heads/master a45fefd17 -> 7186e8c31 [SPARK-16886][EXAMPLES][DOC] Fix some examples to be consistent and indentation in documentation ## What changes were proposed in this pull request? Originally this PR was based on #14491 but I realised that fixing

spark git commit: [SPARK-16952] don't lookup spark home directory when executor uri is set

2016-08-11 Thread srowen
Repository: spark Updated Branches: refs/heads/master 7186e8c31 -> 4d496802f [SPARK-16952] don't lookup spark home directory when executor uri is set ## What changes were proposed in this pull request? remove requirement to set spark.mesos.executor.home when spark.executor.uri is used ## Ho

spark git commit: [SPARK-15899][SQL] Fix the construction of the file path with hadoop Path for Spark 2.0

2016-08-11 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 0ed6236e9 -> 33a213f33 [SPARK-15899][SQL] Fix the construction of the file path with hadoop Path for Spark 2.0 This PR contains the adaptation of https://github.com/apache/spark/pull/13868 for Spark 2.0 ## What changes were proposed

spark git commit: Revert "[SPARK-16831][PYTHON] Fixed bug in CrossValidator.avgMetrics"

2016-08-11 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 ace458f03 -> b3ecff640 Revert "[SPARK-16831][PYTHON] Fixed bug in CrossValidator.avgMetrics" This reverts commit 92ee6fbf5d5096245d9f1a84cd3a8e66062dd945. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-w

spark git commit: [SPARK-16985] Change dataFormat from yyyyMMddHHmm to yyyyMMddHHmmss

2016-08-12 Thread srowen
Repository: spark Updated Branches: refs/heads/master 00e103a6e -> 993923c8f [SPARK-16985] Change dataFormat from MMddHHmm to MMddHHmmss ## What changes were proposed in this pull request? In our cluster, sometimes the sql output maybe overrided. When I submit some sql, all insert in

spark git commit: [MINOR][DOC] Fix style in examples across documentation

2016-08-12 Thread srowen
Repository: spark Updated Branches: refs/heads/master 993923c8f -> f4482225c [MINOR][DOC] Fix style in examples across documentation ## What changes were proposed in this pull request? This PR fixes the documentation as below: - Python has 4 spaces and Java and Scala has 2 spaces (See ht

spark git commit: [SPARK-16598][SQL][TEST] Added a test case for verifying the table identifier parsing

2016-08-12 Thread srowen
Repository: spark Updated Branches: refs/heads/master f4482225c -> 79e2caa13 [SPARK-16598][SQL][TEST] Added a test case for verifying the table identifier parsing What changes were proposed in this pull request? So far, the test cases of `TableIdentifierParserSuite` do not cover the quot

spark git commit: [DOC] add config option spark.ui.enabled into document

2016-08-12 Thread srowen
Repository: spark Updated Branches: refs/heads/master 2a105134e -> 91f2735a1 [DOC] add config option spark.ui.enabled into document ## What changes were proposed in this pull request? The configuration doc lost the config option `spark.ui.enabled` (default value is `true`) I think this optio

spark git commit: [SPARK-12370][DOCUMENTATION] Documentation should link to examples …

2016-08-13 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 bde94cd71 -> 38378f59f [SPARK-12370][DOCUMENTATION] Documentation should link to examples … ## What changes were proposed in this pull request? When documentation is built is should reference examples from the same build. There are

spark git commit: [SPARK-12370][DOCUMENTATION] Documentation should link to examples …

2016-08-13 Thread srowen
Repository: spark Updated Branches: refs/heads/master 91f2735a1 -> e46cb78b3 [SPARK-12370][DOCUMENTATION] Documentation should link to examples … ## What changes were proposed in this pull request? When documentation is built is should reference examples from the same build. There are time

spark git commit: [MINOR][CORE] fix warnings on depreciated methods in MesosClusterSchedulerSuite and DiskBlockObjectWriterSuite

2016-08-13 Thread srowen
Repository: spark Updated Branches: refs/heads/master e46cb78b3 -> 7f7133bdc [MINOR][CORE] fix warnings on depreciated methods in MesosClusterSchedulerSuite and DiskBlockObjectWriterSuite ## What changes were proposed in this pull request? Fixed warnings below after scanning through warnings

spark git commit: [SPARK-16968] Add additional options in jdbc when creating a new table

2016-08-13 Thread srowen
Repository: spark Updated Branches: refs/heads/master 7f7133bdc -> 8c8acdec9 [SPARK-16968] Add additional options in jdbc when creating a new table ## What changes were proposed in this pull request? In the PR, we just allow the user to add additional options when create a new table in JDBC

spark git commit: [SPARK-17023][BUILD] Upgrade to Kafka 0.10.0.1 release

2016-08-13 Thread srowen
Repository: spark Updated Branches: refs/heads/master 8c8acdec9 -> 67f025d90 [SPARK-17023][BUILD] Upgrade to Kafka 0.10.0.1 release ## What changes were proposed in this pull request? Update Kafka streaming connector to use Kafka 0.10.0.1 release ## How was this patch tested? Tested via Spark

spark git commit: [SPARK-17023][BUILD] Upgrade to Kafka 0.10.0.1 release

2016-08-13 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 38378f59f -> a21ecc996 [SPARK-17023][BUILD] Upgrade to Kafka 0.10.0.1 release ## What changes were proposed in this pull request? Update Kafka streaming connector to use Kafka 0.10.0.1 release ## How was this patch tested? Tested via S

spark git commit: Change check for particular missing file message to accommodate the message that would occur, it seems, only in Hadoop 1.x (and therefore in Spark 1.x)

2016-08-13 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 909231d7a -> 8a2b8fcbb Change check for particular missing file message to accommodate the message that would occur, it seems, only in Hadoop 1.x (and therefore in Spark 1.x) Project: http://git-wip-us.apache.org/repos/asf/spark/repo

spark git commit: [SPARK-17027][ML] Avoid integer overflow in PolynomialExpansion.getPolySize

2016-08-14 Thread srowen
Repository: spark Updated Branches: refs/heads/master cdaa562c9 -> 0ebf7c1bf [SPARK-17027][ML] Avoid integer overflow in PolynomialExpansion.getPolySize ## What changes were proposed in this pull request? Replaces custom choose function with o.a.commons.math3.CombinatoricsUtils.binomialCoeff

spark git commit: [SPARK-17027][ML] Avoid integer overflow in PolynomialExpansion.getPolySize

2016-08-14 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 750f88045 -> e02d0d085 [SPARK-17027][ML] Avoid integer overflow in PolynomialExpansion.getPolySize ## What changes were proposed in this pull request? Replaces custom choose function with o.a.commons.math3.CombinatoricsUtils.binomialC

spark git commit: [SPARK-17027][ML] Avoid integer overflow in PolynomialExpansion.getPolySize

2016-08-14 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 8a2b8fcbb -> b54a586af [SPARK-17027][ML] Avoid integer overflow in PolynomialExpansion.getPolySize Replaces custom choose function with o.a.commons.math3.CombinatoricsUtils.binomialCoefficient Spark unit tests Author: zero323 Close

spark git commit: Revert "[SPARK-17027][ML] Avoid integer overflow in PolynomialExpansion.getPolySize"

2016-08-14 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 b54a586af -> 4d64c7fd1 Revert "[SPARK-17027][ML] Avoid integer overflow in PolynomialExpansion.getPolySize" This reverts commit b54a586af4b8ca7e8b97311bf5e75e00797de899. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Comm

spark git commit: [WIP][MINOR][TYPO] Fix several trivival typos

2016-08-14 Thread srowen
Repository: spark Updated Branches: refs/heads/master 0ebf7c1bf -> 2a3d286f3 [WIP][MINOR][TYPO] Fix several trivival typos ## What changes were proposed in this pull request? * Fixed one typo `"overriden"` as `"overridden"`, also make sure no other same typo. * Fixed one typo `"lowcase"` as

spark git commit: [SPARK-11714][MESOS] Make Spark on Mesos honor port restrictions on coarse grain mode

2016-08-15 Thread srowen
Repository: spark Updated Branches: refs/heads/master 2a3d286f3 -> 1a028bdef [SPARK-11714][MESOS] Make Spark on Mesos honor port restrictions on coarse grain mode - Make mesos coarse grained scheduler accept port offers and pre-assign ports Previous attempt was for fine grained: https://git

spark git commit: [TRIVIAL][ML] Fix LogisticRegression typo in error message.

2016-08-15 Thread srowen
Repository: spark Updated Branches: refs/heads/master 1a028bdef -> ddf0d1e3f [TRIVIAL][ML] Fix LogisticRegression typo in error message. ## What changes were proposed in this pull request? Fix ```LogisticRegression``` typo in error message. ## How was this patch tested? Docs change, no new te

spark git commit: [MINOR][DOC] Correct code snippet results in quick start documentation

2016-08-16 Thread srowen
Repository: spark Updated Branches: refs/heads/master 8fdc6ce40 -> 6f0988b12 [MINOR][DOC] Correct code snippet results in quick start documentation ## What changes were proposed in this pull request? As README.md file is updated over time. Some code snippet outputs are not correct based on n

spark git commit: [SPARK-17087][MESOS] Documentation for Making Spark on Mesos honor port restrictions

2016-08-18 Thread srowen
Repository: spark Updated Branches: refs/heads/master e82dbe600 -> b81421afb [SPARK-17087][MESOS] Documentation for Making Spark on Mesos honor port restrictions ## What changes were proposed in this pull request? - adds documentation for https://issues.apache.org/jira/browse/SPARK-11714 ##

spark git commit: [SPARK-16961][CORE] Fixed off-by-one error that biased randomizeInPlace

2016-08-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 287bea130 -> 5377fc623 [SPARK-16961][CORE] Fixed off-by-one error that biased randomizeInPlace JIRA issue link: https://issues.apache.org/jira/browse/SPARK-16961 Changed one line of Utils.randomizeInPlace to allow elements to stay in place

spark git commit: [SPARK-16961][CORE] Fixed off-by-one error that biased randomizeInPlace

2016-08-19 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 05b180faa -> d55d1f454 [SPARK-16961][CORE] Fixed off-by-one error that biased randomizeInPlace JIRA issue link: https://issues.apache.org/jira/browse/SPARK-16961 Changed one line of Utils.randomizeInPlace to allow elements to stay in p

spark git commit: [SPARK-16965][MLLIB][PYSPARK] Fix bound checking for SparseVector.

2016-08-19 Thread srowen
Repository: spark Updated Branches: refs/heads/master 864be9359 -> 072acf5e1 [SPARK-16965][MLLIB][PYSPARK] Fix bound checking for SparseVector. ## What changes were proposed in this pull request? 1. In scala, add negative low bound checking and put all the low/upper bound checking in one pla

spark git commit: [SPARK-17002][CORE] Document that spark.ssl.protocol. is required for SSL

2016-08-21 Thread srowen
Repository: spark Updated Branches: refs/heads/master 7f08a60b6 -> e328f577e [SPARK-17002][CORE] Document that spark.ssl.protocol. is required for SSL ## What changes were proposed in this pull request? `spark.ssl.enabled`=true, but failing to set `spark.ssl.protocol` will fail and throw mea

spark git commit: [SPARK-17127] Make unaligned access in unsafe available for AArch64

2016-08-22 Thread srowen
Repository: spark Updated Branches: refs/heads/master b2074b664 -> 083de00cb [SPARK-17127] Make unaligned access in unsafe available for AArch64 ## # What changes were proposed in this pull request? >From the spark of version 2.0.0 , when MemoryMode.OFF_HEAP is set , whether >the architectur

spark git commit: [SPARK-16968] Document additional options in jdbc Writer

2016-08-22 Thread srowen
Repository: spark Updated Branches: refs/heads/master 083de00cb -> 4b6c2cbcb [SPARK-16968] Document additional options in jdbc Writer ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) This is the document for previous JDBC Writer options. ## Ho

spark git commit: [SPARK-17085][STREAMING][DOCUMENTATION AND ACTUAL CODE DIFFERS - UNSUPPORTED OPERATIONS]

2016-08-22 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 49cc44de3 -> 2add45fab [SPARK-17085][STREAMING][DOCUMENTATION AND ACTUAL CODE DIFFERS - UNSUPPORTED OPERATIONS] Changes in Spark Stuctured Streaming doc in this link https://spark.apache.org/docs/2.0.0/structured-streaming-programming

spark git commit: [SPARK-17085][STREAMING][DOCUMENTATION AND ACTUAL CODE DIFFERS - UNSUPPORTED OPERATIONS]

2016-08-22 Thread srowen
Repository: spark Updated Branches: refs/heads/master 8d35a6f68 -> bd9655063 [SPARK-17085][STREAMING][DOCUMENTATION AND ACTUAL CODE DIFFERS - UNSUPPORTED OPERATIONS] Changes in Spark Stuctured Streaming doc in this link https://spark.apache.org/docs/2.0.0/structured-streaming-programming-gui

spark git commit: [SPARK-17095] [Documentation] [Latex and Scala doc do not play nicely]

2016-08-23 Thread srowen
Repository: spark Updated Branches: refs/heads/master 9d376ad76 -> 97d461b75 [SPARK-17095] [Documentation] [Latex and Scala doc do not play nicely] ## What changes were proposed in this pull request? In Latex, it is common to find "}}}" when closing several expressions at once. [SPARK-16822]

spark git commit: [MINOR][DOC] Use standard quotes instead of "curly quote" marks from Mac in structured streaming programming guides

2016-08-23 Thread srowen
Repository: spark Updated Branches: refs/heads/master 8fd63e808 -> 588559911 [MINOR][DOC] Use standard quotes instead of "curly quote" marks from Mac in structured streaming programming guides ## What changes were proposed in this pull request? This PR fixes curly quotes (`“` and `”` ) t

spark git commit: [MINOR][DOC] Use standard quotes instead of "curly quote" marks from Mac in structured streaming programming guides

2016-08-23 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 cc4018996 -> a2a7506d0 [MINOR][DOC] Use standard quotes instead of "curly quote" marks from Mac in structured streaming programming guides This PR fixes curly quotes (`“` and `”` ) to standard quotes (`"`). This will be a actual p

spark git commit: [TRIVIAL] Typo Fix

2016-08-23 Thread srowen
Repository: spark Updated Branches: refs/heads/master 588559911 -> 6555ef0cc [TRIVIAL] Typo Fix ## What changes were proposed in this pull request? Fix a typo ## How was this patch tested? no tests Author: Zheng RuiFeng Closes #14772 from zhengruifeng/minor_numClasses. Project: http://gi

spark git commit: [MINOR][BUILD] Fix Java CheckStyle Error

2016-08-24 Thread srowen
Repository: spark Updated Branches: refs/heads/master 52fa45d62 -> 673a80d22 [MINOR][BUILD] Fix Java CheckStyle Error ## What changes were proposed in this pull request? As Spark 2.0.1 will be released soon (mentioned in the spark dev mailing list), besides the critical bugs, it's better to f

spark git commit: [MINOR][BUILD] Fix Java CheckStyle Error

2016-08-24 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 df87f161c -> ce7dce175 [MINOR][BUILD] Fix Java CheckStyle Error As Spark 2.0.1 will be released soon (mentioned in the spark dev mailing list), besides the critical bugs, it's better to fix the code style errors before the release. B

spark git commit: [SPARK-17086][ML] Fix InvalidArgumentException issue in QuantileDiscretizer when some quantiles are duplicated

2016-08-24 Thread srowen
Repository: spark Updated Branches: refs/heads/master 673a80d22 -> 92c0eaf34 [SPARK-17086][ML] Fix InvalidArgumentException issue in QuantileDiscretizer when some quantiles are duplicated ## What changes were proposed in this pull request? In cases when QuantileDiscretizerSuite is called upo

spark git commit: [SPARK-17086][ML] Fix InvalidArgumentException issue in QuantileDiscretizer when some quantiles are duplicated

2016-08-24 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 ce7dce175 -> 33d79b587 [SPARK-17086][ML] Fix InvalidArgumentException issue in QuantileDiscretizer when some quantiles are duplicated ## What changes were proposed in this pull request? In cases when QuantileDiscretizerSuite is called

spark git commit: [SPARK-16781][PYSPARK] java launched by PySpark as gateway may not be the same java used in the spark environment

2016-08-24 Thread srowen
OME support ## How was this patch tested? Pyspark tests Author: Sean Owen Closes #14748 from srowen/SPARK-16781. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0b3a4be9 Tree: http://git-wip-us.apache.org/repos/asf/spark/t

spark git commit: [SPARK-16781][PYSPARK] java launched by PySpark as gateway may not be the same java used in the spark environment

2016-08-24 Thread srowen
OME support ## How was this patch tested? Pyspark tests Author: Sean Owen Closes #14748 from srowen/SPARK-16781. (cherry picked from commit 0b3a4be92ca6b38eef32ea5ca240d9f91f68aa65) Signed-off-by: Sean Owen Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-

spark git commit: [SPARK-17193][CORE] HadoopRDD NPE at DEBUG log level when getLocationInfo == null

2016-08-25 Thread srowen
and logging) exception ## How was this patch tested? Jenkins tests Author: Sean Owen Closes #14760 from srowen/SPARK-17193. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2bcd5d5c Tree: http://git-wip-us.apache.org/repos/asf/sp

spark git commit: [SPARK-17193][CORE] HadoopRDD NPE at DEBUG log level when getLocationInfo == null

2016-08-25 Thread srowen
and logging) exception ## How was this patch tested? Jenkins tests Author: Sean Owen Closes #14760 from srowen/SPARK-17193. (cherry picked from commit 2bcd5d5ce3eaf0eb1600a12a2b55ddb40927533b) Signed-off-by: Sean Owen Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://

spark git commit: [SPARK-15382][SQL] Fix a bug in sampling with replacement

2016-08-27 Thread srowen
Repository: spark Updated Branches: refs/heads/master 718b6bad2 -> cd0ed31ea [SPARK-15382][SQL] Fix a bug in sampling with replacement ## What changes were proposed in this pull request? This pr to fix a bug below in sampling with replacement ``` val df = Seq((1, 0), (2, 0), (3, 0)).toDF("a",

spark git commit: [SPARK-15382][SQL] Fix a bug in sampling with replacement

2016-08-27 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 901ab0694 -> 56a842635 [SPARK-15382][SQL] Fix a bug in sampling with replacement ## What changes were proposed in this pull request? This pr to fix a bug below in sampling with replacement ``` val df = Seq((1, 0), (2, 0), (3, 0)).toDF("

spark git commit: [ML][MLLIB] The require condition and message doesn't match in SparseMatrix.

2016-08-27 Thread srowen
Repository: spark Updated Branches: refs/heads/master cd0ed31ea -> 40168dbe7 [ML][MLLIB] The require condition and message doesn't match in SparseMatrix. ## What changes were proposed in this pull request? The require condition and message doesn't match, and the condition also should be optim

spark git commit: [ML][MLLIB] The require condition and message doesn't match in SparseMatrix.

2016-08-27 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 56a842635 -> 7306c5f90 [ML][MLLIB] The require condition and message doesn't match in SparseMatrix. ## What changes were proposed in this pull request? The require condition and message doesn't match, and the condition also should be o

spark git commit: [SPARK-17216][UI] fix event timeline bars length

2016-08-27 Thread srowen
Repository: spark Updated Branches: refs/heads/master 40168dbe7 -> 9fbced5b2 [SPARK-17216][UI] fix event timeline bars length ## What changes were proposed in this pull request? Make event timeline bar expand to full length of the bar (which is total time) This issue occurs only on chrome, f

spark git commit: [SPARK-17216][UI] fix event timeline bars length

2016-08-27 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 7306c5f90 -> 5487fa0b8 [SPARK-17216][UI] fix event timeline bars length ## What changes were proposed in this pull request? Make event timeline bar expand to full length of the bar (which is total time) This issue occurs only on chrom

spark git commit: [SPARK-17001][ML] Enable standardScaler to standardize sparse vectors when withMean=True

2016-08-27 Thread srowen
if requested. This is for compatibility with `VectorAssembler` in common usages. ## How was this patch tested? Jenkins tests, including new caes to reflect the new behavior. Author: Sean Owen Closes #14663 from srowen/SPARK-17001. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Com

spark git commit: [BUILD] Closes some stale PRs.

2016-08-29 Thread srowen
ses #10572 ## How was this patch tested? N/A Author: Sean Owen Closes #14849 from srowen/CloseStalePRs. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1a48c004 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/1a48c

spark git commit: fixed a typo

2016-08-29 Thread srowen
Repository: spark Updated Branches: refs/heads/master 1a48c0047 -> 08913ce00 fixed a typo idempotant -> idempotent Author: Seigneurin, Alexis (CONT) Closes #14833 from aseigneurin/fix-typo. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/rep

spark-website git commit: Add Abraham Zhan to 2.0.0 contribs; wrap and dedupe the list.

2016-08-29 Thread srowen
Repository: spark-website Updated Branches: refs/heads/asf-site 9700f2f4a -> d37a3afce Add Abraham Zhan to 2.0.0 contribs; wrap and dedupe the list. Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/d37a3afc

spark git commit: [SPARK-17276][CORE][TEST] Stop env params output on Jenkins job page

2016-08-30 Thread srowen
Repository: spark Updated Branches: refs/heads/master bca79c823 -> 2d76cb11f [SPARK-17276][CORE][TEST] Stop env params output on Jenkins job page https://issues.apache.org/jira/browse/SPARK-17276 ## What changes were proposed in this pull request? When trying to find error msg in a failed Je

spark git commit: [SPARK-17264][SQL] DataStreamWriter should document that it only supports Parquet for now

2016-08-30 Thread srowen
ter now ## How was this patch tested? (Doc build -- no functional changes to test) Author: Sean Owen Closes #14860 from srowen/SPARK-17264. (cherry picked from commit befab9c1c6b59ad90f63a7d10e12b186be897f15) Signed-off-by: Sean Owen Project: http://git-wip-us.apache.org/repos/asf/spark/r

spark git commit: [SPARK-17264][SQL] DataStreamWriter should document that it only supports Parquet for now

2016-08-30 Thread srowen
now ## How was this patch tested? (Doc build -- no functional changes to test) Author: Sean Owen Closes #14860 from srowen/SPARK-17264. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/befab9c1 Tree: http://git-wip-us.apache.

spark git commit: [MINOR][DOCS] Fix minor typos in python example code

2016-08-30 Thread srowen
Repository: spark Updated Branches: refs/heads/master befab9c1c -> d4eee9932 [MINOR][DOCS] Fix minor typos in python example code ## What changes were proposed in this pull request? Fix minor typos python example code in streaming programming guide ## How was this patch tested? N/A Author:

spark git commit: [MINOR][MLLIB][SQL] Clean up unused variables and unused import

2016-08-30 Thread srowen
Repository: spark Updated Branches: refs/heads/master d4eee9932 -> 27209252f [MINOR][MLLIB][SQL] Clean up unused variables and unused import ## What changes were proposed in this pull request? Clean up unused variables and unused import statements, unnecessary `return` and `toArray`, and som

spark-website git commit: Hot-fix ec2-scripts.html in Spark docs/2.0.0, which for some reason built correctly in 2.0.0-preview, builds in master, but didn't deploy in 2.0.0

2016-08-31 Thread srowen
Repository: spark-website Updated Branches: refs/heads/asf-site d37a3afce -> b2cf71427 Hot-fix ec2-scripts.html in Spark docs/2.0.0, which for some reason built correctly in 2.0.0-preview, builds in master, but didn't deploy in 2.0.0 Project: http://git-wip-us.apache.org/repos/asf/spark-webs

spark-website git commit: Push only fix to contributor list to 2.0.0 for Spark

2016-08-31 Thread srowen
Repository: spark-website Updated Branches: refs/heads/asf-site b2cf71427 -> fcd0bc3dd Push only fix to contributor list to 2.0.0 for Spark Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/fcd0bc3d Tree: htt

[1/2] spark-website git commit: Re-sync Spark site HTML to output of latest jekyll

2016-08-31 Thread srowen
Repository: spark-website Updated Branches: refs/heads/asf-site fcd0bc3dd -> 0845f49de http://git-wip-us.apache.org/repos/asf/spark-website/blob/0845f49d/site/releases/spark-release-1-1-0.html -- diff --git a/site/releases/spar

[2/2] spark-website git commit: Re-sync Spark site HTML to output of latest jekyll

2016-08-31 Thread srowen
Re-sync Spark site HTML to output of latest jekyll Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/0845f49d Tree: http://git-wip-us.apache.org/repos/asf/spark-website/tree/0845f49d Diff: http://git-wip-us.apach

spark git commit: [SPARK-17329][BUILD] Don't build PRs with -Pyarn unless YARN code changed

2016-09-01 Thread srowen
h tested? Jenkins tests (will look to verify whether -Pyarn was included in the PR builder for this one.) Author: Sean Owen Closes #14892 from srowen/SPARK-17329. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/536fa911 Tree: ht

spark git commit: fixed typos

2016-09-01 Thread srowen
Repository: spark Updated Branches: refs/heads/master a18c169fd -> dd859f95c fixed typos fixed 2 typos Author: Seigneurin, Alexis (CONT) Closes #14877 from aseigneurin/fix-typo-2. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spa

spark git commit: [SPARK-17342][WEBUI] Style of event timeline is broken

2016-09-02 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 dd377a522 -> f9463238d [SPARK-17342][WEBUI] Style of event timeline is broken ## What changes were proposed in this pull request? SPARK-15373 (#13158) updated the version of vis.js to 4.16.1. As of 4.0.0, some class was renamed like '

spark git commit: [SPARK-17342][WEBUI] Style of event timeline is broken

2016-09-02 Thread srowen
Repository: spark Updated Branches: refs/heads/master f2d6e2ef2 -> 2ab8dbdda [SPARK-17342][WEBUI] Style of event timeline is broken ## What changes were proposed in this pull request? SPARK-15373 (#13158) updated the version of vis.js to 4.16.1. As of 4.0.0, some class was renamed like 'time

spark git commit: [SQL][DOC][MINOR] Add (Scala-specific) and (Java-specific)

2016-09-02 Thread srowen
Repository: spark Updated Branches: refs/heads/master 6969dcc79 -> a3097e2b3 [SQL][DOC][MINOR] Add (Scala-specific) and (Java-specific) ## What changes were proposed in this pull request? Adds (Scala-specific) and (Java-specific) to Scaladoc. ## How was this patch tested? local build Autho

spark git commit: [SPARK-17352][WEBUI] Executor computing time can be negative-number because of calculation error

2016-09-02 Thread srowen
Repository: spark Updated Branches: refs/heads/master a3097e2b3 -> 7ee24dac8 [SPARK-17352][WEBUI] Executor computing time can be negative-number because of calculation error ## What changes were proposed in this pull request? In StagePage, executor-computing-time is calculated but calculatio

spark git commit: [SPARK-17352][WEBUI] Executor computing time can be negative-number because of calculation error

2016-09-02 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 171bdfd96 -> d9d10ffb9 [SPARK-17352][WEBUI] Executor computing time can be negative-number because of calculation error ## What changes were proposed in this pull request? In StagePage, executor-computing-time is calculated but calcul

spark git commit: [SPARK-17363][ML][MLLIB] fix MultivariantOnlineSummerizer.numNonZeros

2016-09-03 Thread srowen
Repository: spark Updated Branches: refs/heads/master d2fde6b72 -> 7a8a81d79 [SPARK-17363][ML][MLLIB] fix MultivariantOnlineSummerizer.numNonZeros ## What changes were proposed in this pull request? fix `MultivariantOnlineSummerizer.numNonZeros` method, return `nnz` array, instead of `weight

spark git commit: [SPARK-17347][SQL][EXAMPLES] Encoder in Dataset example has incorrect type

2016-09-03 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 12a2e2a5a -> 949544d01 [SPARK-17347][SQL][EXAMPLES] Encoder in Dataset example has incorrect type ## What changes were proposed in this pull request? We propose to fix the Encoder type in the Dataset example ## How was this patch test

spark git commit: [SPARK-17347][SQL][EXAMPLES] Encoder in Dataset example has incorrect type

2016-09-03 Thread srowen
Repository: spark Updated Branches: refs/heads/master 7a8a81d79 -> 97da41039 [SPARK-17347][SQL][EXAMPLES] Encoder in Dataset example has incorrect type ## What changes were proposed in this pull request? We propose to fix the Encoder type in the Dataset example ## How was this patch tested?

<    7   8   9   10   11   12   13   14   15   16   >