Repository: spark
Updated Branches:
refs/heads/branch-1.5 63922fa4d -> 71aa54755
[SPARK-10128] [STREAMING] Used correct classloader to deserialize WAL data
Recovering Kinesis sequence numbers from WAL leads to classnotfoundexception
because the ObjectInputStream does not use the correct class
Repository: spark
Updated Branches:
refs/heads/master 73431d8af -> b762f9920
[SPARK-10128] [STREAMING] Used correct classloader to deserialize WAL data
Recovering Kinesis sequence numbers from WAL leads to classnotfoundexception
because the ObjectInputStream does not use the correct classload
Repository: spark
Updated Branches:
refs/heads/branch-1.5 a3ed2c31e -> 63922fa4d
[SPARK-10125] [STREAMING] Fix a potential deadlock in JobGenerator.stop
Because `lazy val` uses `this` lock, if JobGenerator.stop and
JobGenerator.doCheckpoint (JobGenerator.shouldCheckpoint has not yet been
ini
Repository: spark
Updated Branches:
refs/heads/branch-1.5 16414dae0 -> a3ed2c31e
[SPARK-10124] [MESOS] Fix removing queued driver in mesos cluster mode.
Currently the spark applications can be queued to the Mesos cluster dispatcher,
but when multiple jobs are in queue we don't handle removing
Repository: spark
Updated Branches:
refs/heads/master affc8a887 -> 73431d8af
[SPARK-10124] [MESOS] Fix removing queued driver in mesos cluster mode.
Currently the spark applications can be queued to the Mesos cluster dispatcher,
but when multiple jobs are in queue we don't handle removing job
Repository: spark
Updated Branches:
refs/heads/master 1f29d502e -> affc8a887
[SPARK-10125] [STREAMING] Fix a potential deadlock in JobGenerator.stop
Because `lazy val` uses `this` lock, if JobGenerator.stop and
JobGenerator.doCheckpoint (JobGenerator.shouldCheckpoint has not yet been
initial
Repository: spark
Updated Branches:
refs/heads/branch-1.5 321cb99ca -> 16414dae0
[SPARK-9812] [STREAMING] Fix Python 3 compatibility issue in PySpark Streaming
and some docs
This PR includes the following fixes:
1. Use `range` instead of `xrange` in `queue_stream.py` to support Python 3.
2. F
Repository: spark
Updated Branches:
refs/heads/master 2f2686a73 -> 1f29d502e
[SPARK-9812] [STREAMING] Fix Python 3 compatibility issue in PySpark Streaming
and some docs
This PR includes the following fixes:
1. Use `range` instead of `xrange` in `queue_stream.py` to support Python 3.
2. Fix t
[SPARK-9242] [SQL] Audit UDAF interface.
A few minor changes:
1. Improved documentation
2. Rename apply(distinct) to distinct.
3. Changed MutableAggregationBuffer from a trait to an abstract class.
4. Renamed returnDataType to dataType to be more consistent with other
expressions.
And unrel
http://git-wip-us.apache.org/repos/asf/spark/blob/2f2686a7/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeFunctions.scala
--
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/exp
[SPARK-9242] [SQL] Audit UDAF interface.
A few minor changes:
1. Improved documentation
2. Rename apply(distinct) to distinct.
3. Changed MutableAggregationBuffer from a trait to an abstract class.
4. Renamed returnDataType to dataType to be more consistent with other
expressions.
And unrel
http://git-wip-us.apache.org/repos/asf/spark/blob/2f2686a7/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/math.scala
--
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/math
http://git-wip-us.apache.org/repos/asf/spark/blob/321cb99c/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/math.scala
--
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/math
Repository: spark
Updated Branches:
refs/heads/branch-1.5 56a37b01f -> 321cb99ca
http://git-wip-us.apache.org/repos/asf/spark/blob/321cb99c/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringOperations.scala
--
http://git-wip-us.apache.org/repos/asf/spark/blob/321cb99c/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala
--
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/exp
http://git-wip-us.apache.org/repos/asf/spark/blob/2f2686a7/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala
--
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/exp
Repository: spark
Updated Branches:
refs/heads/master ba5f7e184 -> 2f2686a73
http://git-wip-us.apache.org/repos/asf/spark/blob/2f2686a7/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringOperations.scala
--
http://git-wip-us.apache.org/repos/asf/spark/blob/321cb99c/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeFunctions.scala
--
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/exp
Repository: spark
Updated Branches:
refs/heads/master 8e0a072f7 -> ba5f7e184
[SPARK-10035] [SQL] Parquet filters does not process EqualNullSafe filter.
As I talked with Lian,
1. I added EquelNullSafe to ParquetFilters
- It uses the same equality comparison filter with EqualTo since the Parqu
Repository: spark
Updated Branches:
refs/heads/master b0dbaec4f -> 8e0a072f7
[SPARK-9895] User Guide for RFormula Feature Transformer
mengxr
Author: Eric Liang
Closes #8293 from ericl/docs-2.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/r
Repository: spark
Updated Branches:
refs/heads/branch-1.5 5c749c82c -> 56a37b01f
[SPARK-9895] User Guide for RFormula Feature Transformer
mengxr
Author: Eric Liang
Closes #8293 from ericl/docs-2.
(cherry picked from commit 8e0a072f78b4902d5f7ccc6b15232ed202a117f9)
Signed-off-by: Xiangrui M
Repository: spark
Updated Branches:
refs/heads/branch-1.5 a59475f5b -> 5c749c82c
[SPARK-6489] [SQL] add column pruning for Generate
This PR takes over https://github.com/apache/spark/pull/5358
Author: Wenchen Fan
Closes #8268 from cloud-fan/6489.
(cherry picked from commit b0dbaec4f942a47a
Repository: spark
Updated Branches:
refs/heads/master e0dd1309a -> b0dbaec4f
[SPARK-6489] [SQL] add column pruning for Generate
This PR takes over https://github.com/apache/spark/pull/5358
Author: Wenchen Fan
Closes #8268 from cloud-fan/6489.
Project: http://git-wip-us.apache.org/repos/as
Repository: spark
Updated Branches:
refs/heads/master 373a376c0 -> e0dd1309a
[SPARK-10119] [CORE] Fix isDynamicAllocationEnabled when config is expliticly
disabled.
Author: Marcelo Vanzin
Closes #8316 from vanzin/SPARK-10119.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Comm
Repository: spark
Updated Branches:
refs/heads/branch-1.5 1494d58d2 -> a59475f5b
[SPARK-10119] [CORE] Fix isDynamicAllocationEnabled when config is expliticly
disabled.
Author: Marcelo Vanzin
Closes #8316 from vanzin/SPARK-10119.
(cherry picked from commit e0dd1309ac248375f429639801923570f
Repository: spark
Updated Branches:
refs/heads/branch-1.5 b32a31d64 -> 1494d58d2
[SPARK-10083] [SQL] CaseWhen should support type coercion of DecimalType and
FractionalType
create t1 (a decimal(7, 2), b long);
select case when 1=1 then a else 1.0 end from t1;
select case when 1=1 then a else
Repository: spark
Updated Branches:
refs/heads/master f3ff4c41d -> 373a376c0
[SPARK-10083] [SQL] CaseWhen should support type coercion of DecimalType and
FractionalType
create t1 (a decimal(7, 2), b long);
select case when 1=1 then a else 1.0 end from t1;
select case when 1=1 then a else b en
Repository: spark
Updated Branches:
refs/heads/branch-1.5 d9dfd43d4 -> b32a31d64
[SPARK-9899] [SQL] Disables customized output committer when speculation is on
Speculation hates direct output committer, as there are multiple corner cases
that may cause data corruption and/or data loss.
Pleas
Repository: spark
Updated Branches:
refs/heads/master 1f4c4fe6d -> f3ff4c41d
[SPARK-9899] [SQL] Disables customized output committer when speculation is on
Speculation hates direct output committer, as there are multiple corner cases
that may cause data corruption and/or data loss.
Please se
Repository: spark
Updated Branches:
refs/heads/branch-1.5 77269fcb5 -> d9dfd43d4
[SPARK-10090] [SQL] fix decimal scale of division
We should rounding the result of multiply/division of decimal to expected
precision/scale, also check overflow.
Author: Davies Liu
Closes #8287 from davies/dec
Repository: spark
Updated Branches:
refs/heads/master 21bdbe9fe -> 1f4c4fe6d
[SPARK-10090] [SQL] fix decimal scale of division
We should rounding the result of multiply/division of decimal to expected
precision/scale, also check overflow.
Author: Davies Liu
Closes #8287 from davies/decimal
Repository: spark
Updated Branches:
refs/heads/branch-1.5 afaed7ef4 -> 77269fcb5
[SPARK-9627] [SQL] Stops using Scala runtime reflection in DictionaryEncoding
`DictionaryEncoding` uses Scala runtime reflection to avoid boxing costs while
building the directory array. However, this code path m
Repository: spark
Updated Branches:
refs/heads/master 08887369c -> 21bdbe9fe
[SPARK-9627] [SQL] Stops using Scala runtime reflection in DictionaryEncoding
`DictionaryEncoding` uses Scala runtime reflection to avoid boxing costs while
building the directory array. However, this code path may h
Repository: spark
Updated Branches:
refs/heads/branch-1.5 829c33a4b -> afaed7ef4
[SPARK-10073] [SQL] Python withColumn should replace the old column
DataFrame.withColumn in Python should be consistent with the Scala one
(replacing the existing column that has the same name).
cc marmbrus
Au
Repository: spark
Updated Branches:
refs/heads/master e05da5cb5 -> 08887369c
[SPARK-10073] [SQL] Python withColumn should replace the old column
DataFrame.withColumn in Python should be consistent with the Scala one
(replacing the existing column that has the same name).
cc marmbrus
Author
Repository: spark
Updated Branches:
refs/heads/branch-1.5 8c0a5a248 -> 1038f677b
[SPARK-10107] [SQL] fix NPE in format_number
Author: Davies Liu
Closes #8305 from davies/format_number.
(cherry picked from commit e05da5cb5ea253e6372f648fc8203204f2a8df8d)
Signed-off-by: Reynold Xin
Project
Repository: spark
Updated Branches:
refs/heads/branch-1.5 1038f677b -> 829c33a4b
[SPARK-10087] [CORE] [BRANCH-1.5] Disable spark.shuffle.reduceLocality.enabled
by default.
https://issues.apache.org/jira/browse/SPARK-10087
In some cases, when spark.shuffle.reduceLocality.enabled is enabled, w
Repository: spark
Updated Branches:
refs/heads/master f3391ff2b -> e05da5cb5
[SPARK-10107] [SQL] fix NPE in format_number
Author: Davies Liu
Closes #8305 from davies/format_number.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spa
Repository: spark
Updated Branches:
refs/heads/master 5b62bef8c -> f3391ff2b
[SPARK-8889] [CORE] Fix for OOM for graph creation
Fix for OOM for graph creation
Author: Joshi
Author: Rekha Joshi
Closes #7602 from rekhajoshm/SPARK-8889.
Project: http://git-wip-us.apache.org/repos/asf/spark/
Repository: spark
Updated Branches:
refs/heads/branch-1.5 ba369258d -> 8c0a5a248
[SPARK-8918] [MLLIB] [DOC] Add @since tags to mllib.clustering
This continues the work from #8256. I removed `since` tags from
private/protected/local methods/variables (see
https://github.com/apache/spark/commi
Repository: spark
Updated Branches:
refs/heads/master d898c33f7 -> 5b62bef8c
[SPARK-8918] [MLLIB] [DOC] Add @since tags to mllib.clustering
This continues the work from #8256. I removed `since` tags from
private/protected/local methods/variables (see
https://github.com/apache/spark/commit/72
Repository: spark
Updated Branches:
refs/heads/master 28a98464e -> d898c33f7
[SPARK-10106] [SPARKR] Add `ifelse` Column function to SparkR
### JIRA
[[SPARK-10106] Add `ifelse` Column function to SparkR - ASF
JIRA](https://issues.apache.org/jira/browse/SPARK-10106)
Author: Yu ISHIKAWA
Close
Repository: spark
Updated Branches:
refs/heads/branch-1.5 f25c32475 -> ba369258d
[SPARK-10106] [SPARKR] Add `ifelse` Column function to SparkR
### JIRA
[[SPARK-10106] Add `ifelse` Column function to SparkR - ASF
JIRA](https://issues.apache.org/jira/browse/SPARK-10106)
Author: Yu ISHIKAWA
C
Author: meng
Date: Wed Aug 19 19:11:08 2015
New Revision: 1696648
URL: http://svn.apache.org/r1696648
Log:
update MLlib page for 1.5
Modified:
spark/mllib/index.md
spark/site/mllib/index.html
Modified: spark/mllib/index.md
URL:
http://svn.apache.org/viewvc/spark/mllib/index.md?rev=16966
Repository: spark
Updated Branches:
refs/heads/branch-1.5 a8e880818 -> f25c32475
[SPARK-10097] Adds `shouldMaximize` flag to `ml.evaluation.Evaluator`
Previously, users of evaluator (`CrossValidator` and `TrainValidationSplit`)
would only maximize the metric in evaluator, leading to a hacky s
Repository: spark
Updated Branches:
refs/heads/master 5fd53c64b -> 28a98464e
[SPARK-10097] Adds `shouldMaximize` flag to `ml.evaluation.Evaluator`
Previously, users of evaluator (`CrossValidator` and `TrainValidationSplit`)
would only maximize the metric in evaluator, leading to a hacky solut
Repository: spark
Updated Branches:
refs/heads/master 2fcb9cb95 -> 5fd53c64b
[SPARK-9833] [YARN] Add options to disable delegation token retrieval.
This allows skipping the code that tries to talk to Hive and HBase to
fetch delegation tokens, in case that somehow conflicts with the application
Repository: spark
Updated Branches:
refs/heads/branch-1.5 bebe63dfe -> a8e880818
[SPARK-9856] [SPARKR] Add expression functions into SparkR whose params are
complicated
I added lots of Column functinos into SparkR. And I also added `rand(seed:
Int)` and `randn(seed: Int)` in Scala. Since we
Repository: spark
Updated Branches:
refs/heads/master f3e177917 -> 2fcb9cb95
[SPARK-9856] [SPARKR] Add expression functions into SparkR whose params are
complicated
I added lots of Column functinos into SparkR. And I also added `rand(seed:
Int)` and `randn(seed: Int)` in Scala. Since we need
Repository: spark
Updated Branches:
refs/heads/master 802b5b879 -> f3e177917
[SPARK-5754] [YARN] Spark/Yarn/Windows driver/executor escaping Fix
This is my retry to suggest a fix for using Spark on Yarn on Windows. The
former request lacked coding style which I hope to have learned to do bett
Repository: spark
Updated Branches:
refs/heads/branch-1.5 f8dc42770 -> bebe63dfe
[SPARK-10084] [MLLIB] [DOC] Add Python example for mllib FP-growth user guide
1, Add Python example for mllib FP-growth user guide.
2, Correct mistakes of Scala and Java examples.
Author: Yanbo Liang
Closes #82
Repository: spark
Updated Branches:
refs/heads/master 39e4ebd52 -> 802b5b879
[SPARK-10084] [MLLIB] [DOC] Add Python example for mllib FP-growth user guide
1, Add Python example for mllib FP-growth user guide.
2, Correct mistakes of Scala and Java examples.
Author: Yanbo Liang
Closes #8279 f
Repository: spark
Updated Branches:
refs/heads/branch-1.5 522b0b646 -> f8dc42770
[SPARK-10060] [ML] [DOC] spark.ml DecisionTree user guide
New user guide section ml-decision-tree.md, including code examples.
I have run all examples, including the Java ones.
CC: manishamde yanboliang mengxr
Repository: spark
Updated Branches:
refs/heads/master 3d16a5450 -> 39e4ebd52
[SPARK-10060] [ML] [DOC] spark.ml DecisionTree user guide
New user guide section ml-decision-tree.md, including code examples.
I have run all examples, including the Java ones.
CC: manishamde yanboliang mengxr
Auth
Repository: spark
Updated Branches:
refs/heads/branch-1.5 5553f02be -> 522b0b646
[SPARK-8949] Print warnings when using preferred locations feature
Add warnings according to SPARK-8949 in `SparkContext`
- warnings in scaladoc
- log warnings when preferred locations feature is used through
Repository: spark
Updated Branches:
refs/heads/master ba2a07e2b -> 3d16a5450
[SPARK-8949] Print warnings when using preferred locations feature
Add warnings according to SPARK-8949 in `SparkContext`
- warnings in scaladoc
- log warnings when preferred locations feature is used through
`Sp
Repository: spark
Updated Branches:
refs/heads/branch-1.5 e56bcc638 -> 5553f02be
[SPARK-9977] [DOCS] Update documentation for StringIndexer
By using `StringIndexer`, we can obtain indexed label on new column. So a
following estimator should use this new column through pipeline if it wants to
Repository: spark
Updated Branches:
refs/heads/master 865a3df3d -> ba2a07e2b
[SPARK-9977] [DOCS] Update documentation for StringIndexer
By using `StringIndexer`, we can obtain indexed label on new column. So a
following estimator should use this new column through pipeline if it wants to
use
Repository: spark
Updated Branches:
refs/heads/branch-1.5 561390dbc -> e56bcc638
[DOCS] [SQL] [PYSPARK] Fix typo in ntile function
Fix typo in ntile function.
Author: Moussa Taifi
Closes #8261 from moutai/patch-2.
(cherry picked from commit 865a3df3d578c0442c97d749c81f554b560da406)
Signed-
Repository: spark
Updated Branches:
refs/heads/master f141efeaf -> 865a3df3d
[DOCS] [SQL] [PYSPARK] Fix typo in ntile function
Fix typo in ntile function.
Author: Moussa Taifi
Closes #8261 from moutai/patch-2.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-w
Repository: spark
Updated Branches:
refs/heads/master b23c4d3ff -> f141efeaf
[SPARK-10070] [DOCS] Remove Guava dependencies in user guides
`Lists.newArrayList` -> `Arrays.asList`
CC jkbradley feynmanliang
Anybody into replacing usages of `Lists.newArrayList` in the examples / source
code to
Repository: spark
Updated Branches:
refs/heads/branch-1.5 417852f7c -> 561390dbc
[SPARK-10070] [DOCS] Remove Guava dependencies in user guides
`Lists.newArrayList` -> `Arrays.asList`
CC jkbradley feynmanliang
Anybody into replacing usages of `Lists.newArrayList` in the examples / source
cod
Repository: spark
Updated Branches:
refs/heads/branch-1.5 392bd19d6 -> 417852f7c
Fix Broken Link
Link was broken because it included tick marks.
Author: Bill Chambers
Closes #8302 from anabranch/patch-1.
(cherry picked from commit b23c4d3ffc36e47c057360c611d8ab1a13877699)
Signed-off-by: Re
Repository: spark
Updated Branches:
refs/heads/master bc9a0e032 -> b23c4d3ff
Fix Broken Link
Link was broken because it included tick marks.
Author: Bill Chambers
Closes #8302 from anabranch/patch-1.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apac
64 matches
Mail list logo
