Repository: spark
Updated Branches:
refs/heads/master bd70244b3 -> 99f5f9886
http://git-wip-us.apache.org/repos/asf/spark/blob/99f5f988/streaming/src/test/scala/org/apache/spark/streaming/rdd/TrackStateRDDSuite.scala
--
diff --
[SPARK-11290][STREAMING] Basic implementation of trackStateByKey
Current updateStateByKey provides stateful processing in Spark Streaming. It
allows the user to maintain per-key state and manage that state using an
updateFunction. The updateFunction is called for each key, and it uses new data
Repository: spark
Updated Branches:
refs/heads/branch-1.6 85bc72908 -> daa74be6f
http://git-wip-us.apache.org/repos/asf/spark/blob/daa74be6/streaming/src/test/scala/org/apache/spark/streaming/rdd/TrackStateRDDSuite.scala
--
dif
[SPARK-11290][STREAMING] Basic implementation of trackStateByKey
Current updateStateByKey provides stateful processing in Spark Streaming. It
allows the user to maintain per-key state and manage that state using an
updateFunction. The updateFunction is called for each key, and it uses new data
Repository: spark
Updated Branches:
refs/heads/master 1a8e0468a -> bd70244b3
[SPARK-11463] [PYSPARK] only install signal in main thread
Only install signal in main thread, or it will fail to create context in
not-main thread.
Author: Davies Liu
Closes #9574 from davies/python_signal.
Pro
Repository: spark
Updated Branches:
refs/heads/branch-1.6 649a0a357 -> 85bc72908
[SPARK-11463] [PYSPARK] only install signal in main thread
Only install signal in main thread, or it will fail to create context in
not-main thread.
Author: Davies Liu
Closes #9574 from davies/python_signal.
Repository: spark
Updated Branches:
refs/heads/branch-1.6 d9a39048f -> 649a0a357
[SPARK-11468] [SPARKR] add stddev/variance agg functions for Column
Checked names, none of them should conflict with anything in base
shivaram davies rxin
Author: felixcheung
Closes #9489 from felixcheung/rstd
Repository: spark
Updated Branches:
refs/heads/master fac53d8ec -> 1a8e0468a
[SPARK-11468] [SPARKR] add stddev/variance agg functions for Column
Checked names, none of them should conflict with anything in base
shivaram davies rxin
Author: felixcheung
Closes #9489 from felixcheung/rstddev.
Repository: spark
Updated Branches:
refs/heads/branch-1.6 bc3bfa04d -> d9a39048f
[SPARK-10192][HOTFIX] Fix NPE in test that was added in #8402
This fixes an NPE introduced in SPARK-10192 / #8402.
Author: Josh Rosen
Closes #9620 from JoshRosen/SPARK-10192-hotfix.
(cherry picked from commit
Repository: spark
Updated Branches:
refs/heads/master 6e101d2e9 -> fac53d8ec
[SPARK-10192][HOTFIX] Fix NPE in test that was added in #8402
This fixes an NPE introduced in SPARK-10192 / #8402.
Author: Josh Rosen
Closes #9620 from JoshRosen/SPARK-10192-hotfix.
Project: http://git-wip-us.apa
Repository: spark
Updated Branches:
refs/heads/master 745e45d5f -> 6e101d2e9
[SPARK-6726][ML] Import/export for spark.ml LogisticRegressionModel
This PR adds model save/load for spark.ml's LogisticRegressionModel. It also
does minor refactoring of the default save/load classes to reuse code.
Repository: spark
Updated Branches:
refs/heads/branch-1.6 b34819c7b -> bc3bfa04d
[SPARK-6726][ML] Import/export for spark.ml LogisticRegressionModel
This PR adds model save/load for spark.ml's LogisticRegressionModel. It also
does minor refactoring of the default save/load classes to reuse c
Repository: spark
Updated Branches:
refs/heads/master 12c7635dc -> 745e45d5f
[MINOR] License header formatting fix
The header wasn't indented properly.
Author: Marc Prud'hommeaux
Closes #9312 from mprudhom/patch-1.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://
Repository: spark
Updated Branches:
refs/heads/master 6600786dd -> 12c7635dc
[MINOR] Fix typo in AggregationQuerySuite.scala
Author: Forest Fang
Closes #9357 from saurfang/patch-1.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spa
Repository: spark
Updated Branches:
refs/heads/master 900917541 -> 6600786dd
[SPARK-11361][STREAMING] Show scopes of RDD operations inside
DStream.foreachRDD and DStream.transform in DAG viz
Currently, when a DStream sets the scope for RDD generated by it, that scope is
not allowed to be ove
Repository: spark
Updated Branches:
refs/heads/branch-1.6 fafeca36e -> b34819c7b
[SPARK-11361][STREAMING] Show scopes of RDD operations inside
DStream.foreachRDD and DStream.transform in DAG viz
Currently, when a DStream sets the scope for RDD generated by it, that scope is
not allowed to be
Repository: spark
Updated Branches:
refs/heads/branch-1.6 93ac30741 -> fafeca36e
[SPARK-11615] Drop @VisibleForTesting annotation
See http://search-hadoop.com/m/q3RTtjpe8r1iRbTj2 for discussion.
Summary: addition of VisibleForTesting annotation resulted in spark-shell
malfunctioning.
Author
Repository: spark
Updated Branches:
refs/heads/master 3e0a6cf1e -> 900917541
[SPARK-11615] Drop @VisibleForTesting annotation
See http://search-hadoop.com/m/q3RTtjpe8r1iRbTj2 for discussion.
Summary: addition of VisibleForTesting annotation resulted in spark-shell
malfunctioning.
Author: te
Repository: spark
Updated Branches:
refs/heads/branch-1.6 10272d5c9 -> 93ac30741
[SPARK-11572] Exit AsynchronousListenerBus thread when stop() is called
As vonnagy reported in the following thread:
http://search-hadoop.com/m/q3RTtk982kvIow22
Attempts to join the thread in AsynchronousListener
Repository: spark
Updated Branches:
refs/heads/master 33112f9c4 -> 3e0a6cf1e
[SPARK-11572] Exit AsynchronousListenerBus thread when stop() is called
As vonnagy reported in the following thread:
http://search-hadoop.com/m/q3RTtk982kvIow22
Attempts to join the thread in AsynchronousListenerBus
Repository: spark
Updated Branches:
refs/heads/master c0e48dfa6 -> 33112f9c4
[SPARK-10192][CORE] simple test w/ failure involving a shared dependency
just trying to increase test coverage in the scheduler, this already works. It
includes a regression test for SPARK-9809
copied some test uti
Repository: spark
Updated Branches:
refs/heads/branch-1.6 d8bfc025c -> 10272d5c9
[SPARK-10192][CORE] simple test w/ failure involving a shared dependency
just trying to increase test coverage in the scheduler, this already works. It
includes a regression test for SPARK-9809
copied some test
Repository: spark
Updated Branches:
refs/heads/branch-1.6 9ccd1bb80 -> d8bfc025c
[SPARK-11566] [MLLIB] [PYTHON] Refactoring GaussianMixtureModel.gaussians in
Python
cc jkbradley
Author: Yu ISHIKAWA
Closes #9534 from yu-iskw/SPARK-11566.
(cherry picked from commit c0e48dfa611fa5d94132af7e6
Repository: spark
Updated Branches:
refs/heads/master a3989058c -> c0e48dfa6
[SPARK-11566] [MLLIB] [PYTHON] Refactoring GaussianMixtureModel.gaussians in
Python
cc jkbradley
Author: Yu ISHIKAWA
Closes #9534 from yu-iskw/SPARK-11566.
Project: http://git-wip-us.apache.org/repos/asf/spark/r
Repository: spark
Updated Branches:
refs/heads/master 21c562fa0 -> a3989058c
[SPARK-10827][CORE] AppClient should not use `askWithReply` in `receiveAndReply`
Changed AppClient to be non-blocking in `receiveAndReply` by using a separate
thread to wait for response and reply to the context. Th
Repository: spark
Updated Branches:
refs/heads/branch-1.6 ce5aba32f -> 9ccd1bb80
[SPARK-10827][CORE] AppClient should not use `askWithReply` in `receiveAndReply`
Changed AppClient to be non-blocking in `receiveAndReply` by using a separate
thread to wait for response and reply to the context.
Repository: spark
Updated Branches:
refs/heads/branch-1.6 ff7d869c4 -> ce5aba32f
[SPARK-9241][SQL] Supporting multiple DISTINCT columns - follow-up (3)
This PR is a 2nd follow-up for
[SPARK-9241](https://issues.apache.org/jira/browse/SPARK-9241). It contains the
following improvements:
* Fix
Repository: spark
Updated Branches:
refs/heads/master 3121e7816 -> 21c562fa0
[SPARK-9241][SQL] Supporting multiple DISTINCT columns - follow-up (3)
This PR is a 2nd follow-up for
[SPARK-9241](https://issues.apache.org/jira/browse/SPARK-9241). It contains the
following improvements:
* Fix for
Repository: spark
Updated Branches:
refs/heads/branch-1.6 b79c1bd1e -> ff7d869c4
[SPARK-9830][SPARK-11641][SQL][FOLLOW-UP] Remove AggregateExpression1 and
update toString of Exchange
https://issues.apache.org/jira/browse/SPARK-9830
This is the follow-up pr for https://github.com/apache/spark
Repository: spark
Updated Branches:
refs/heads/master e281b8739 -> 3121e7816
[SPARK-9830][SPARK-11641][SQL][FOLLOW-UP] Remove AggregateExpression1 and
update toString of Exchange
https://issues.apache.org/jira/browse/SPARK-9830
This is the follow-up pr for https://github.com/apache/spark/pul
Repository: spark
Updated Branches:
refs/heads/branch-1.6 d0828d224 -> b79c1bd1e
[SPARK-5565][ML] LDA wrapper for Pipelines API
This adds LDA to spark.ml, the Pipelines API. It follows the design doc in the
JIRA: [https://issues.apache.org/jira/browse/SPARK-5565], with one major change:
* I
Repository: spark
Updated Branches:
refs/heads/master 1dde39d79 -> e281b8739
[SPARK-5565][ML] LDA wrapper for Pipelines API
This adds LDA to spark.ml, the Pipelines API. It follows the design doc in the
JIRA: [https://issues.apache.org/jira/browse/SPARK-5565], with one major change:
* I elim
Repository: spark
Updated Branches:
refs/heads/master 32790fe72 -> 1dde39d79
[SPARK-9818] Re-enable Docker tests for JDBC data source
This patch re-enables tests for the Docker JDBC data source. These tests were
reverted in #4872 due to transitive dependency conflicts introduced by the
`dock
Repository: spark
Updated Branches:
refs/heads/branch-1.6 6616f4da3 -> d0828d224
[SPARK-9818] Re-enable Docker tests for JDBC data source
This patch re-enables tests for the Docker JDBC data source. These tests were
reverted in #4872 due to transitive dependency conflicts introduced by the
`
Repository: spark
Updated Branches:
refs/heads/master 638c51d93 -> 32790fe72
[SPARK-11567] [PYTHON] Add Python API for corr Aggregate function
like `df.agg(corr("col1", "col2")`
davies
Author: felixcheung
Closes #9536 from felixcheung/pyfunc.
Project: http://git-wip-us.apache.org/repos/a
Repository: spark
Updated Branches:
refs/heads/branch-1.6 6a74efab0 -> 6616f4da3
[SPARK-11567] [PYTHON] Add Python API for corr Aggregate function
like `df.agg(corr("col1", "col2")`
davies
Author: felixcheung
Closes #9536 from felixcheung/pyfunc.
(cherry picked from commit 32790fe7249b0ef
Repository: spark
Updated Tags: refs/tags/v1.5.2-rc2 [deleted] 49c30c1f6
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
Repository: spark
Updated Branches:
refs/heads/master 724cf7a38 -> 638c51d93
[SPARK-11550][DOCS] Replace example code in mllib-optimization.md using
include_example
Author: Pravin Gadakh
Closes #9516 from pravingadakh/SPARK-11550.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Repository: spark
Updated Branches:
refs/heads/branch-1.6 8fb7b8304 -> 6a74efab0
[SPARK-11550][DOCS] Replace example code in mllib-optimization.md using
include_example
Author: Pravin Gadakh
Closes #9516 from pravingadakh/SPARK-11550.
(cherry picked from commit 638c51d9380081b3b8182be2c246
Repository: spark
Updated Branches:
refs/heads/branch-1.6 f0180106a -> 8fb7b8304
[SPARK-11616][SQL] Improve toString for Dataset
Author: Michael Armbrust
Closes #9586 from marmbrus/dataset-toString.
(cherry picked from commit 724cf7a38c551bf2a79b87a8158bbe1725f9f888)
Signed-off-by: Michael
Repository: spark
Updated Branches:
refs/heads/master dba1a62cf -> 724cf7a38
[SPARK-11616][SQL] Improve toString for Dataset
Author: Michael Armbrust
Closes #9586 from marmbrus/dataset-toString.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org
Repository: spark
Updated Branches:
refs/heads/branch-1.6 80641c4fa -> f0180106a
[SPARK-7316][MLLIB] RDD sliding window with step
Implementation of step capability for sliding window function in MLlib's RDD.
Though one can use current sliding window with step 1 and then filter every Nth
wind
Repository: spark
Updated Branches:
refs/heads/master 18350a570 -> dba1a62cf
[SPARK-7316][MLLIB] RDD sliding window with step
Implementation of step capability for sliding window function in MLlib's RDD.
Though one can use current sliding window with step 1 and then filter every Nth
window,
Repository: spark
Updated Branches:
refs/heads/master f14e95115 -> 18350a570
[SPARK-11618][ML] Minor refactoring of basic ML import/export
Refactoring
* separated overwrite and param save logic in DefaultParamsWriter
* added sparkVersion to DefaultParamsWriter
CC: mengxr
Author: Joseph K. Br
Repository: spark
Updated Branches:
refs/heads/branch-1.6 54685fa36 -> 80641c4fa
[SPARK-11618][ML] Minor refactoring of basic ML import/export
Refactoring
* separated overwrite and param save logic in DefaultParamsWriter
* added sparkVersion to DefaultParamsWriter
CC: mengxr
Author: Joseph K
Repository: spark
Updated Branches:
refs/heads/master 87aedc48c -> f14e95115
[ML][R] SparkR::glm summary result to compare with native R
Follow up #9561. Due to
[SPARK-11587](https://issues.apache.org/jira/browse/SPARK-11587) has been
fixed, we should compare SparkR::glm summary result with
Repository: spark
Updated Branches:
refs/heads/branch-1.6 f38509a76 -> 54685fa36
[ML][R] SparkR::glm summary result to compare with native R
Follow up #9561. Due to
[SPARK-11587](https://issues.apache.org/jira/browse/SPARK-11587) has been
fixed, we should compare SparkR::glm summary result w
Repository: spark
Updated Branches:
refs/heads/branch-1.6 5ccc1eb08 -> f38509a76
[SPARK-10371][SQL] Implement subexpr elimination for UnsafeProjections
This patch adds the building blocks for codegening subexpr elimination and
implements
it end to end for UnsafeProjection. The building blocks
Repository: spark
Updated Branches:
refs/heads/master 53600854c -> 87aedc48c
[SPARK-10371][SQL] Implement subexpr elimination for UnsafeProjections
This patch adds the building blocks for codegening subexpr elimination and
implements
it end to end for UnsafeProjection. The building blocks can
Repository: spark
Updated Branches:
refs/heads/branch-1.6 6e2e84f3e -> 5ccc1eb08
[SPARK-11590][SQL] use native json_tuple in lateral view
Author: Wenchen Fan
Closes #9562 from cloud-fan/json-tuple.
(cherry picked from commit 53600854c270d4c953fe95fbae528740b5cf6603)
Signed-off-by: Yin Huai
Repository: spark
Updated Branches:
refs/heads/master dfcfcbcc0 -> 53600854c
[SPARK-11590][SQL] use native json_tuple in lateral view
Author: Wenchen Fan
Closes #9562 from cloud-fan/json-tuple.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/
Repository: spark
Updated Branches:
refs/heads/branch-1.6 d2405cb5e -> 6e2e84f3e
[SPARK-11578][SQL][FOLLOW-UP] complete the user facing api for typed aggregation
Currently the user facing api for typed aggregation has some limitations:
* the customized typed aggregation must be the first of a
Repository: spark
Updated Branches:
refs/heads/master 47735cdc2 -> dfcfcbcc0
[SPARK-11578][SQL][FOLLOW-UP] complete the user facing api for typed aggregation
Currently the user facing api for typed aggregation has some limitations:
* the customized typed aggregation must be the first of aggre
Repository: spark
Updated Branches:
refs/heads/master e0701c756 -> 47735cdc2
[SPARK-10863][SPARKR] Method coltypes() (New version)
This is a follow up on PR #8984, as the corresponding branch for such PR was
damaged.
Author: Oscar D. Lara Yejas
Closes #9579 from olarayej/SPARK-10863_NEW14.
Repository: spark
Updated Branches:
refs/heads/branch-1.6 7c4ade0d7 -> d2405cb5e
[SPARK-10863][SPARKR] Method coltypes() (New version)
This is a follow up on PR #8984, as the corresponding branch for such PR was
damaged.
Author: Oscar D. Lara Yejas
Closes #9579 from olarayej/SPARK-10863_NE
http://git-wip-us.apache.org/repos/asf/spark/blob/7c4ade0d/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Utils.scala
--
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expre
http://git-wip-us.apache.org/repos/asf/spark/blob/7c4ade0d/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala
--
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/opt
Repository: spark
Updated Branches:
refs/heads/master 6e5fc3788 -> e0701c756
http://git-wip-us.apache.org/repos/asf/spark/blob/e0701c75/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala
--
diff --git a/sql/hive/src
[SPARK-9830][SQL] Remove AggregateExpression1 and Aggregate Operator used to
evaluate AggregateExpression1s
https://issues.apache.org/jira/browse/SPARK-9830
This PR contains the following main changes.
* Removing `AggregateExpression1`.
* Removing `Aggregate` operator, which is used to evaluate
[SPARK-9830][SQL] Remove AggregateExpression1 and Aggregate Operator used to
evaluate AggregateExpression1s
https://issues.apache.org/jira/browse/SPARK-9830
This PR contains the following main changes.
* Removing `AggregateExpression1`.
* Removing `Aggregate` operator, which is used to evaluate
Repository: spark
Updated Branches:
refs/heads/branch-1.6 825e971d0 -> 7c4ade0d7
http://git-wip-us.apache.org/repos/asf/spark/blob/7c4ade0d/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala
--
diff --git a/sql/hive
http://git-wip-us.apache.org/repos/asf/spark/blob/e0701c75/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Utils.scala
--
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expre
http://git-wip-us.apache.org/repos/asf/spark/blob/e0701c75/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala
--
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/opt
Repository: spark
Updated Branches:
refs/heads/branch-1.6 4dad3c51b -> 825e971d0
[SPARK-11252][NETWORK] ShuffleClient should release connection after fetching
blocks had been completed for external shuffle
with yarn's external shuffle, ExternalShuffleClient of executors reserve its
connectio
Repository: spark
Updated Branches:
refs/heads/master 689386b1c -> 6e5fc3788
[SPARK-11252][NETWORK] ShuffleClient should release connection after fetching
blocks had been completed for external shuffle
with yarn's external shuffle, ExternalShuffleClient of executors reserve its
connections f
Repository: spark
Updated Branches:
refs/heads/branch-1.6 6c1409e0b -> 4dad3c51b
[SPARK-7841][BUILD] Stop using retrieveManaged to retrieve dependencies in SBT
This patch modifies Spark's SBT build so that it no longer uses
`retrieveManaged` / `lib_managed` to store its dependencies. The moti
Repository: spark
Updated Branches:
refs/heads/master a81f47ff7 -> 689386b1c
[SPARK-7841][BUILD] Stop using retrieveManaged to retrieve dependencies in SBT
This patch modifies Spark's SBT build so that it no longer uses
`retrieveManaged` / `lib_managed` to store its dependencies. The motivati
Repository: spark
Updated Branches:
refs/heads/branch-1.6 39c1ebcd3 -> 6c1409e0b
[SPARK-11382] Replace example code in mllib-decision-tree.md using
include_example
https://issues.apache.org/jira/browse/SPARK-11382
B.T.W. I fix an error in naive_bayes_example.py.
Author: Xusen Yin
Closes #
Repository: spark
Updated Branches:
refs/heads/master 5507a9d09 -> a81f47ff7
[SPARK-11382] Replace example code in mllib-decision-tree.md using
include_example
https://issues.apache.org/jira/browse/SPARK-11382
B.T.W. I fix an error in naive_bayes_example.py.
Author: Xusen Yin
Closes #9596
Author: rxin
Date: Tue Nov 10 16:46:33 2015
New Revision: 1713691
URL: http://svn.apache.org/viewvc?rev=1713691&view=rev
Log:
update latest doc
Modified:
spark/_layouts/global.html
spark/site/community.html
spark/site/docs/latest
spark/site/documentation.html
spark/site/downlo
Added: spark/site/news/spark-1-5-2-released.html
URL:
http://svn.apache.org/viewvc/spark/site/news/spark-1-5-2-released.html?rev=1713690&view=auto
==
--- spark/site/news/spark-1-5-2-released.html (added)
+++ spark/site/new
Modified: spark/site/releases/spark-release-1-2-1.html
URL:
http://svn.apache.org/viewvc/spark/site/releases/spark-release-1-2-1.html?rev=1713690&r1=1713689&r2=1713690&view=diff
==
--- spark/site/releases/spark-release-1-2
Author: rxin
Date: Tue Nov 10 16:44:07 2015
New Revision: 1713690
URL: http://svn.apache.org/viewvc?rev=1713690&view=rev
Log:
1.5.2 release
Added:
spark/news/_posts/2015-11-09-spark-1-5-2-released.md
spark/releases/_posts/2015-11-09-spark-release-1-5-2.md
spark/site/news/spark-1-5-2-r
Repository: spark
Updated Branches:
refs/heads/branch-1.6 b426d24db -> 39c1ebcd3
Fix typo in driver page
"Comamnd property" => "Command property"
Author: Paul Chandler
Closes #9578 from pestilence669/fix_spelling.
(cherry picked from commit 5507a9d0935aa42d65c3a4fa65da680b5af14faf)
Signed-
Repository: spark
Updated Branches:
refs/heads/branch-1.5 0512960fc -> 6e823b4d7
Fix typo in driver page
"Comamnd property" => "Command property"
Author: Paul Chandler
Closes #9578 from pestilence669/fix_spelling.
(cherry picked from commit 5507a9d0935aa42d65c3a4fa65da680b5af14faf)
Signed-
Repository: spark
Updated Branches:
refs/heads/master 521b3cae1 -> 5507a9d09
Fix typo in driver page
"Comamnd property" => "Command property"
Author: Paul Chandler
Closes #9578 from pestilence669/fix_spelling.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-w
76 matches
Mail list logo