Repository: spark
Updated Branches:
refs/heads/branch-1.4 dfdae5800 - c58b9c616
[SPARK-7233] [CORE] Detect REPL mode once
h3Description/h3
Detect REPL mode once per JVM lifespan.
Previous behavior was to check presence of interpreter mode every time a job
was submitted. In the case of
Repository: spark
Updated Branches:
refs/heads/master 8f4aaba0e - b1b9d5802
[SPARK-7233] [CORE] Detect REPL mode once
h3Description/h3
Detect REPL mode once per JVM lifespan.
Previous behavior was to check presence of interpreter mode every time a job
was submitted. In the case of execution
Repository: spark
Updated Branches:
refs/heads/master b1b9d5802 - 270d4b518
[CORE] Protect additional test vars from early GC
Fix more places in which some test variables could be collected early by
aggressive JVM optimization.
Added a couple of comments to note where existing references are
Repository: spark
Updated Branches:
refs/heads/branch-1.4 c58b9c616 - 866e4b520
[CORE] Protect additional test vars from early GC
Fix more places in which some test variables could be collected early by
aggressive JVM optimization.
Added a couple of comments to note where existing references
/assets/2133137/7675241/de1a3da6-fcea-11e4-8101-88055cef78c5.png;
width=300px /
Author: Andrew Or and...@databricks.com
Closes #6171 from andrewor14/dag-viz-skipped and squashes the following commits:
f261797 [Andrew Or] Merge branch 'master' of github.com:apache/spark into
dag-viz-skipped
Repository: spark
Updated Branches:
refs/heads/master 0b6f503d5 - fcf90b75c
[HOTFIX] Fix ORC build break
Fix break caused by merging #6225 and #6194.
Author: Michael Armbrust mich...@databricks.com
Closes #6244 from marmbrus/fixOrcBuildBreak and squashes the following commits:
b10e47b
Repository: spark
Updated Branches:
refs/heads/branch-1.4 ba502ab20 - a475cbc97
http://git-wip-us.apache.org/repos/asf/spark/blob/a475cbc9/core/src/main/scala/org/apache/spark/SparkContext.scala
--
diff --git
http://git-wip-us.apache.org/repos/asf/spark/blob/a475cbc9/core/src/main/resources/org/apache/spark/ui/static/dagre-d3.min.js
--
diff --git a/core/src/main/resources/org/apache/spark/ui/static/dagre-d3.min.js
/
--
**After**
img
src=https://cloud.githubusercontent.com/assets/2133137/7625997/e0878f8c-f9b4-11e4-8df3-7dd611b13c87.png;
width=650px/
Author: Andrew Or and...@databricks.com
Closes #6034 from andrewor14/dag-viz-streaming and squashes the following
commits:
932a64a [Andrew Or] Merge branch
Repository: spark
Updated Branches:
refs/heads/branch-1.4 fe3c7340f - e319719e0
[SPARK-7664] [WEBUI] DAG visualization: Fix incorrect link paths of DAG.
In JobPage, we can jump a StagePage when we click corresponding box of DAG viz
but the link path is incorrect.
When we click a box like as
Repository: spark
Updated Branches:
refs/heads/branch-1.4 e319719e0 - 7dc0ff3f1
[SPARK-7504] [YARN] NullPointerException when initializing SparkContext in
YARN-cluster mode
Added a simple checking for SparkContext.
Also added two rational checking against null at AM object.
Author: ehnalis
Repository: spark
Updated Branches:
refs/heads/master 971738936 - 30a6e0dcc
[SPARK-5634] [core] Show correct message in HS when no incomplete apps f...
...ound.
Author: Marcelo Vanzin van...@cloudera.com
Closes #5515 from vanzin/SPARK-5634 and squashes the following commits:
f74ecf1
Repository: spark
Updated Branches:
refs/heads/master 6de282e2d - 971738936
[SPARK-6890] [core] Fix launcher lib work with SPARK_PREPEND_CLASSES.
The fix for SPARK-6406 broke the case where sub-processes are launched
when SPARK_PREPEND_CLASSES is set, because the code now would only add
the
Repository: spark
Updated Branches:
refs/heads/branch-1.2 3c13936aa - 5845a6236
[SPARK-5634] [core] Show correct message in HS when no incomplete apps f...
...ound.
Author: Marcelo Vanzin van...@cloudera.com
Closes #5515 from vanzin/SPARK-5634 and squashes the following commits:
f74ecf1
Repository: spark
Updated Branches:
refs/heads/master 320bca450 - f63b44a5c
[SPARK-6894]spark.executor.extraLibraryOptions =
spark.executor.extraLibraryPath
https://issues.apache.org/jira/browse/SPARK-6894
cc vanzin
Author: WangTaoTheTonic wangtao...@huawei.com
Closes #5506 from
Repository: spark
Updated Branches:
refs/heads/master f63b44a5c - dcf8a9f33
[CORE] SPARK-6880: Fixed null check when all the dependent stages are cancelled
due to previous stage failure
Fixed null check when all the dependent stages are cancelled due to previous
stage failure. This happens
Repository: spark
Updated Branches:
refs/heads/master 51b306b93 - 320bca450
[SPARK-6081] Support fetching http/https uris in driver runner.
Currently if passed uris such as http/https, it won't able to fetch them as it
only calls HadoopFs get.
This fix utilizes the existing util method to
Repository: spark
Updated Branches:
refs/heads/master dcf8a9f33 - 25998e4d7
[SPARK-2033] Automatically cleanup checkpoint
Author: GuoQiang Li wi...@qq.com
Closes #855 from witgo/cleanup_checkpoint_date and squashes the following
commits:
1649850 [GuoQiang Li] review commit
c0087e0
Repository: spark
Updated Branches:
refs/heads/master 25998e4d7 - 8f8dc45f6
SPARK-1706: Allow multiple executors per worker in Standalone mode
resubmit of https://github.com/apache/spark/pull/636 for a totally different
algorithm
https://issues.apache.org/jira/browse/SPARK-1706
In current
Repository: spark
Updated Branches:
refs/heads/master 8f8dc45f6 - b075e4b72
[SPARK-6700] [yarn] Re-enable flaky test.
Test runs have been successful on jenkins. So let's re-enable the test and look
out for any failures, and fix things appropriately.
Author: Marcelo Vanzin
Repository: spark
Updated Branches:
refs/heads/master 6adb8bcbf - 65774370a
[SPARK-5808] [build] Package pyspark files in sbt assembly.
This turned out to be more complicated than I wanted because the
layout of python/ doesn't really follow the usual maven conventions.
So some extra code is
Repository: spark
Updated Branches:
refs/heads/master 6fbeb82e1 - 199133733
[SPARK-5933] [core] Move config deprecation warnings to SparkConf.
I didn't find many deprecated configs after a grep-based search,
but the ones I could find were moved to the centralized location
in SparkConf.
While
Repository: spark
Updated Branches:
refs/heads/master 199133733 - d850b4bd3
[SPARK-6975][Yarn] Fix argument validation error
`numExecutors` checking is failed when dynamic allocation is enabled with
default configuration. Details can be seen is
Repository: spark
Updated Branches:
refs/heads/master c5ed51013 - 6fbeb82e1
[SPARK-6350][Mesos] Make mesosExecutorCores configurable in mesos
fine-grained mode
- Defined executorCores from spark.mesos.executor.cores
- Changed the amount of mesosExecutor's cores to executorCores.
- Added new
Repository: spark
Updated Branches:
refs/heads/master a452c5921 - c5ed51013
[SPARK-6703][Core] Provide a way to discover existing SparkContext's
I've added a static getOrCreate method to the static SparkContext object that
allows one to either retrieve a previously created SparkContext or to
Repository: spark
Updated Branches:
refs/heads/master 65774370a - 4d4b24927
[SPARK-6769][YARN][TEST] Usage of the ListenerBus in YarnClusterSuite is wrong
In YarnClusterSuite, a test case uses `SaveExecutorInfo` to handle
ExecutorAddedEvent as follows.
```
private class SaveExecutorInfo
Repository: spark
Updated Branches:
refs/heads/branch-1.3 f17a2fe9b - 3366af67b
[SPARK-6700] disable flaky test
Author: Davies Liu dav...@databricks.com
Closes #5356 from davies/flaky and squashes the following commits:
08955f4 [Davies Liu] disable flaky test
(cherry picked from commit
Repository: spark
Updated Branches:
refs/heads/master 26b415e15 - 9b40c17ab
[SPARK-6700] disable flaky test
Author: Davies Liu dav...@databricks.com
Closes #5356 from davies/flaky and squashes the following commits:
08955f4 [Davies Liu] disable flaky test
Project:
Repository: spark
Updated Branches:
refs/heads/branch-1.3 ce137b8ed - 3e8b0406a
[HOTFIX] [BUILD] Fix Maven build; add core test jar
This was added later in ee11be258251adf900680927ba200bf46512cc04
in branch-1.4 but not branch-1.3. This patch adds a step in the
maven jar plugin to make a jar
Repository: spark
Updated Branches:
refs/heads/branch-1.3 5e77d69c7 - 5b96b6933
[SPARK-7205] [SPARK-7224] [SPARK-7306] Backport packages fixes
Main motivation is to fix the flaky `SparkSubmitUtilsSuite` in branch-1.3.
brkyvz
Author: Burak Yavuz brk...@gmail.com
Closes #6657 from andrewor14
Repository: spark
Updated Branches:
refs/heads/branch-1.4 bfab61f39 - 31e0ae9e1
[MINOR] [UI] Improve confusing message on log page
It's good practice to check if the input path is in the directory
we expect to avoid potentially confusing error messages.
Project:
Repository: spark
Updated Branches:
refs/heads/branch-1.3 e5747ee3a - 744599609
[MINOR] [UI] Improve confusing message on log page
It's good practice to check if the input path is in the directory
we expect to avoid potentially confusing error messages.
Project:
Repository: spark
Updated Branches:
refs/heads/master aa40c4420 - 1d8669f15
[SPARK-8001] [CORE] Make AsynchronousListenerBus.waitUntilEmpty throw
TimeoutException if timeout
Some places forget to call `assert` to check the return value of
`AsynchronousListenerBus.waitUntilEmpty`. Instead of
Repository: spark
Updated Branches:
refs/heads/master d8662cd90 - bfbdab12d
[HOTFIX] [TYPO] Fix typo in #6546
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/bfbdab12
Tree:
Repository: spark
Updated Branches:
refs/heads/master 566cb5947 - 51898b515
[SPARK-8088] don't attempt to lower number of executors by 0
Author: Ryan Williams ryan.blake.willi...@gmail.com
Closes #6624 from ryan-williams/execs and squashes the following commits:
b6f71d4 [Ryan Williams]
Repository: spark
Updated Branches:
refs/heads/master bfbf12b34 - aa40c4420
[SPARK-8059] [YARN] Wake up allocation thread when new requests arrive.
This should help reduce latency for new executor allocations.
Author: Marcelo Vanzin van...@cloudera.com
Closes #6600 from vanzin/SPARK-8059
Repository: spark
Updated Branches:
refs/heads/master f27134782 - a8f1f1543
[HOTFIX] Fix Hadoop-1 build caused by #5792.
Replaced `fs.listFiles` with Hadoop-1 friendly `fs.listStatus` method.
Author: Hari Shreedharan hshreedha...@apache.org
Closes #6619 from
Repository: spark
Updated Branches:
refs/heads/master 20a26b595 - c6a6dd0d0
[MINOR] [UI] Improve confusing message on log page
It's good practice to check if the input path is in the directory
we expect to avoid potentially confusing error messages.
Project:
Repository: spark
Updated Branches:
refs/heads/master c6a6dd0d0 - bfbf12b34
[SPARK-8083] [MESOS] Use the correct base path in mesos driver page.
Author: Timothy Chen tnac...@gmail.com
Closes #6615 from tnachen/mesos_driver_path and squashes the following commits:
4f47b7c [Timothy Chen] Use
Repository: spark
Updated Branches:
refs/heads/branch-1.4 b2a22a651 - d0be9508f
[HOTFIX] Unbreak build from backporting #6546
This is caused by 7e46ea0228f142f6b384331d62cec8f86e61c9d1.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit:
Repository: spark
Updated Branches:
refs/heads/master 0576c3c4f - e35cd36e0
[BUILD] Increase Jenkins test timeout
Currently hive tests alone take 40m. The right thing to do is
to reduce the test time. However, that is a bigger project and
we currently have PRs blocking on tests not timing
Repository: spark
Updated Branches:
refs/heads/branch-1.4 c2c129073 - 96f71b105
[BUILD] Increase Jenkins test timeout
Currently hive tests alone take 40m. The right thing to do is
to reduce the test time. However, that is a bigger project and
we currently have PRs blocking on tests not timing
Repository: spark
Updated Branches:
refs/heads/master 1d8669f15 - f27134782
[SPARK-7989] [CORE] [TESTS] Fix flaky tests in ExternalShuffleServiceSuite and
SparkListenerWithClusterSuite
The flaky tests in ExternalShuffleServiceSuite and
SparkListenerWithClusterSuite will fail if there are
Repository: spark
Updated Branches:
refs/heads/branch-1.4 7e46ea022 - ca21fff7d
[SPARK-3674] [EC2] Clear SPARK_WORKER_INSTANCES when using YARN
cc andrewor14
Author: Shivaram Venkataraman shiva...@cs.berkeley.edu
Closes #6424 from shivaram/spark-worker-instances-yarn-ec2 and squashes
Repository: spark
Updated Branches:
refs/heads/branch-1.4 0bc9a3ec4 - 16748694b
[SPARK-8088] don't attempt to lower number of executors by 0
Author: Ryan Williams ryan.blake.willi...@gmail.com
Closes #6624 from ryan-williams/execs and squashes the following commits:
b6f71d4 [Ryan Williams]
Repository: spark
Updated Branches:
refs/heads/master bfbdab12d - 566cb5947
[HOTFIX] History Server API docs error fix.
Minor error in the monitoring docs. Also made indentation changes in
`ApiRootResource`
Author: Hari Shreedharan hshreedha...@apache.org
Closes #6628 from
against the target branch itself.
Author: Andrew Or and...@databricks.com
Closes #6629 from andrewor14/build-check-hive and squashes the following
commits:
450fbbd [Andrew Or] [BUILD] Use right branch when checking against Hive
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http
find a way
to fix that separately.
Author: Andrew Or and...@databricks.com
Closes #6711 from andrewor14/fix-spark-shell and squashes the following commits:
bf62ecc [Andrew Or] Prevent NPE in spark-shell
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip
should find a way
to fix that separately.
Author: Andrew Or and...@databricks.com
Closes #6711 from andrewor14/fix-spark-shell and squashes the following commits:
bf62ecc [Andrew Or] Prevent NPE in spark-shell
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip
Repository: spark
Updated Branches:
refs/heads/master 12f5eaeee - 4036d05ce
Revert [MINOR] [BUILD] Use custom temp directory during build.
This reverts commit b16b5434ff44c42e4b3a337f9af147669ba44896.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit:
Repository: spark
Updated Branches:
refs/heads/branch-1.4 3e3151e75 - 429c65851
Revert [MINOR] [BUILD] Use custom temp directory during build.
This reverts commit 9b3e4c187123baa22666b8e119ddd55dafecbf89.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit:
Repository: spark
Updated Branches:
refs/heads/branch-1.4 d8a53fb80 - 6faaf15ba
[SPARK-7955] [CORE] Ensure executors with cached RDD blocks are not reâ¦
â¦moved if dynamic allocation is enabled.
This is a work in progress. This patch ensures that an executor that has cached
RDD blocks are
Repository: spark
Updated Branches:
refs/heads/master 18c4fcebb - ed2cc3ee8
[SPARK-8136] [YARN] Fix flakiness in YarnClusterSuite.
Instead of actually downloading the logs, just verify that the logs link is
actually
a URL and is in the expected format.
Author: Hari Shreedharan
Repository: spark
Updated Branches:
refs/heads/master 5aa804f3c - 18c4fcebb
[SPARK-7169] [CORE] Allow metrics system to be configured through SparkConf.
Author: Marcelo Vanzin van...@cloudera.com
Author: Jacek Lewandowski lewandowski.ja...@gmail.com
Closes #6560 from vanzin/SPARK-7169 and
Repository: spark
Updated Branches:
refs/heads/master 901a552c5 - 081db9479
[SPARK-8145] [WEBUI] Trigger a double click on the span to show full job
description.
When using the Spark SQL, Jobs tab and Stages tab display only part of SQL. I
change it to display full SQL by double-click on
Repository: spark
Updated Branches:
refs/heads/branch-1.4 b6fdc6cf1 - 9d1f4d66b
[SPARK-8145] [WEBUI] Trigger a double click on the span to show full job
description.
When using the Spark SQL, Jobs tab and Stages tab display only part of SQL. I
change it to display full SQL by double-click
Repository: spark
Updated Branches:
refs/heads/branch-1.3 ad5daa3a3 - 476b87d31
[MINOR] [UI] Improve error message on log page
Currently if a bad log type if specified, then we get blank.
We should provide a more informative error message.
Project:
the testing Thrift server process to ensure proper
logging behavior.
cc andrewor14 yhuai
Author: Cheng Lian l...@databricks.com
Closes #6493 from liancheng/override-log4j and squashes the following commits:
c489e0e [Cheng Lian] Fixes minor Scala styling issue
b46ef0d [Cheng Lian] Uses
build. This PR replaces those
APIs with equivalent Guava ones to ensure Java 6 friendliness.
cc andrewor14 pwendell, this should also be back ported to branch-1.4.
Author: Cheng Lian l...@databricks.com
Closes #6547 from liancheng/override-log4j and squashes the following commits:
c900cfd [Cheng
6 build. This PR replaces those
APIs with equivalent Guava ones to ensure Java 6 friendliness.
cc andrewor14 pwendell, this should also be back ported to branch-1.4.
Author: Cheng Lian l...@databricks.com
Closes #6547 from liancheng/override-log4j and squashes the following commits:
c900cfd
to test it out here.
Author: Andrew Or and...@databricks.com
Closes #6630 from andrewor14/build-check-hive-1.4 and squashes the following
commits:
186ec65 [Andrew Or] [BUILD] Use right branch when checking against Hive
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git
http://git-wip-us.apache.org/repos/asf/spark/blob/bfe74b34/core/src/test/scala/org/apache/spark/util/collection/AppendOnlyMapSuite.scala
--
diff --git
[SPARK-7558] Demarcate tests in unit-tests.log (1.4)
This includes the following commits:
original: 9eb222c
hotfix1: 8c99793
hotfix2: a4f2412
scalastyle check: 609c492
---
Original patch #6441
Branch-1.3 patch #6602
Author: Andrew Or and...@databricks.com
Closes #6598 from andrewor14
http://git-wip-us.apache.org/repos/asf/spark/blob/bfe74b34/core/src/test/scala/org/apache/spark/network/netty/NettyBlockTransferSecuritySuite.scala
--
diff --git
Repository: spark
Updated Branches:
refs/heads/branch-1.4 bfe74b34a - 84da65319
[BUILD] Fix Maven build for Kinesis
A necessary dependency that is transitively referenced is not
provided, causing compilation failures in builds that provide
the kinesis-asl profile.
Project:
Repository: spark
Updated Branches:
refs/heads/master 9cf740f35 - 984ad6014
[BUILD] Fix Maven build for Kinesis
A necessary dependency that is transitively referenced is not
provided, causing compilation failures in builds that provide
the kinesis-asl profile.
Project:
Repository: spark
Updated Branches:
refs/heads/branch-1.4 584a2ba21 - bfe74b34a
http://git-wip-us.apache.org/repos/asf/spark/blob/bfe74b34/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientDistributedCacheManagerSuite.scala
http://git-wip-us.apache.org/repos/asf/spark/blob/bfe74b34/mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala
--
diff --git
Repository: spark
Updated Branches:
refs/heads/master 490d5a72e - 0d5892dc7
[MINOR] [UI] DAG visualization: trim whitespace from input
Just as a safeguard against DOM rewriting.
Author: Andrew Or and...@databricks.com
Closes #6732 from andrewor14/dag-viz-trim and squashes the following
Repository: spark
Updated Branches:
refs/heads/branch-1.4 a7b7a194a - 1175cfe08
[MINOR] [UI] DAG visualization: trim whitespace from input
Just as a safeguard against DOM rewriting.
Author: Andrew Or and...@databricks.com
Closes #6732 from andrewor14/dag-viz-trim and squashes the following
Repository: spark
Updated Branches:
refs/heads/branch-1.3 0b8dce0c0 - 45b4527e3
[SPARK-8095][BACKPORT] Resolve dependencies of --packages in local ivy cache
Backported PR #6788
cc andrewor14
Author: Burak Yavuz brk...@gmail.com
Closes #6923 from brkyvz/backport-local-ivy and squashes
Repository: spark
Updated Branches:
refs/heads/master 96aa01378 - 1dfb0f7b2
[HOTFIX] [TESTS] Typo mqqt - mqtt
This was introduced in #6866.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1dfb0f7b
Tree:
Repository: spark
Updated Branches:
refs/heads/master 093c34838 - bec40e52b
[HOTFIX] [SPARK-8489] Correct JIRA number in previous commit
It should be SPARK-8489, not SPARK-8498.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit:
Repository: spark
Updated Branches:
refs/heads/branch-1.4 a7b773a8b - 9b16508d2
[HOTFIX] [SPARK-8489] Correct JIRA number in previous commit
It should be SPARK-8489, not SPARK-8498.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit:
Repository: spark
Updated Branches:
refs/heads/branch-1.4 93793237e - 792ed7a4b
[SPARK-8567] [SQL] Increase the timeout of HiveSparkSubmitSuite
https://issues.apache.org/jira/browse/SPARK-8567
Author: Yin Huai yh...@databricks.com
Closes #6957 from yhuai/SPARK-8567 and squashes the
Repository: spark
Updated Branches:
refs/heads/master dca21a83a - 7daa70292
[SPARK-8567] [SQL] Increase the timeout of HiveSparkSubmitSuite
https://issues.apache.org/jira/browse/SPARK-8567
Author: Yin Huai yh...@databricks.com
Closes #6957 from yhuai/SPARK-8567 and squashes the following
Repository: spark
Updated Branches:
refs/heads/master 0c32fc125 - a9858036b
Add example that reads a local file, writes to a DFS path provided by th...
...e user, reads the file back from the DFS, and compares word counts on the
local and DFS versions. Useful for verifying DFS correctness.
: Andrew Or and...@databricks.com
Closes #6886 from andrewor14/spark-submit-suite-fix and squashes the following
commits:
3f99ff1 [Andrew Or] Move destroy to finally block
9a62188 [Andrew Or] Re-enable ignored tests
2382672 [Andrew Or] Check for exit code
(cherry picked from commit
: Andrew Or and...@databricks.com
Closes #6886 from andrewor14/spark-submit-suite-fix and squashes the following
commits:
3f99ff1 [Andrew Or] Move destroy to finally block
9a62188 [Andrew Or] Re-enable ignored tests
2382672 [Andrew Or] Check for exit code
Project: http://git-wip-us.apache.org/repos
Repository: spark
Updated Branches:
refs/heads/master 68a2dca29 - 4be53d039
[SPARK-5836] [DOCS] [STREAMING] Clarify what may cause long-running Spark apps
to preserve shuffle files
Clarify what may cause long-running Spark apps to preserve shuffle files
Author: Sean Owen so...@cloudera.com
: Andrew Or and...@databricks.com
Closes #6886 from andrewor14/spark-submit-suite-fix and squashes the following
commits:
3f99ff1 [Andrew Or] Move destroy to finally block
9a62188 [Andrew Or] Re-enable ignored tests
2382672 [Andrew Or] Check for exit code
(cherry picked from commit
Repository: spark
Updated Branches:
refs/heads/branch-1.4 aedd893b4 - 3415fb978
[SPARK-5836] [DOCS] [STREAMING] Clarify what may cause long-running Spark apps
to preserve shuffle files
Clarify what may cause long-running Spark apps to preserve shuffle files
Author: Sean Owen
Repository: spark
Updated Branches:
refs/heads/branch-1.3 1d441475d - 0b8dce0c0
[SPARK-5836] [DOCS] [STREAMING] Clarify what may cause long-running Spark apps
to preserve shuffle files
Clarify what may cause long-running Spark apps to preserve shuffle files
Author: Sean Owen
Repository: spark
Updated Branches:
refs/heads/master 2c59d5c12 - 9baf09301
[SPARK-8430] ExternalShuffleBlockResolver of shuffle service should support
UnsafeShuffleManager
andrewor14 can you take a look?thanks
Author: Lianhui Wang lianhuiwan...@gmail.com
Closes #6873 from lianhuiwang
Repository: spark
Updated Branches:
refs/heads/branch-1.4 1f2dafb77 - 6f2e41108
[SPARK-8430] ExternalShuffleBlockResolver of shuffle service should support
UnsafeShuffleManager
andrewor14 can you take a look?thanks
Author: Lianhui Wang lianhuiwan...@gmail.com
Closes #6873 from lianhuiwang
Repository: spark
Updated Branches:
refs/heads/master c5876e529 - 4a462c282
[HOTFIX] Fix scala style in DFSReadWriteTest that causes tests failed
This scala style problem causes tested failed.
Author: Liang-Chi Hsieh vii...@gmail.com
Closes #6907 from viirya/hotfix_style and squashes the
Repository: spark
Updated Branches:
refs/heads/branch-1.4 7b88e6a1e - 28e8a6ea6
[SPARK-8282] [SPARKR] Make number of threads used in RBackend configurable
Read number of threads for RBackend from configuration.
[SPARK-8282] #comment Linking with JIRA
Author: Hossein hoss...@databricks.com
Repository: spark
Updated Branches:
refs/heads/master 38112905b - 30ebf1a23
[SPARK-8282] [SPARKR] Make number of threads used in RBackend configurable
Read number of threads for RBackend from configuration.
[SPARK-8282] #comment Linking with JIRA
Author: Hossein hoss...@databricks.com
Repository: spark
Updated Branches:
refs/heads/master 30ebf1a23 - 19e30b48f
[SPARK-7756] CORE RDDOperationScope fix for IBM Java
IBM Java has an extra method when we do getStackTrace(): this is
getStackTraceImpl, a native method. This causes two tests to fail within
DStreamScopeSuite when
Repository: spark
Updated Branches:
refs/heads/branch-1.4 28e8a6ea6 - 568d1d51d
[SPARK-7756] CORE RDDOperationScope fix for IBM Java
IBM Java has an extra method when we do getStackTrace(): this is
getStackTraceImpl, a native method. This causes two tests to fail within
DStreamScopeSuite
Repository: spark
Updated Branches:
refs/heads/master e90c9d92d - 80043e9e7
[SPARK-7261] [CORE] Change default log level to WARN in the REPL
1. Add `log4j-defaults-repl.properties` that has log level WARN.
2. When logging is initialized, check whether inside the REPL. If so, use
Repository: spark
Updated Branches:
refs/heads/master cb871c44c - 5014d0ed7
[SPARK-8273] Driver hangs up when yarn shutdown in client mode
In client mode, if yarn was shut down with spark application running, the
application will hang up after several retries(default: 30) because the
Repository: spark
Updated Branches:
refs/heads/master 80043e9e7 - cb871c44c
[SPARK-8290] spark class command builder need read SPARK_JAVA_OPTS and
SPARK_DRIVER_MEMORY properly
SPARK_JAVA_OPTS was missed in reconstructing the launcher part, we should add
it back so process launched by
Repository: spark
Updated Branches:
refs/heads/branch-1.4 568d1d51d - 2846a357f
[SPARK-8273] Driver hangs up when yarn shutdown in client mode
In client mode, if yarn was shut down with spark application running, the
application will hang up after several retries(default: 30) because the
Repository: spark
Updated Branches:
refs/heads/master 8f7308f9c - 38112905b
[SPARK-5479] [YARN] Handle --py-files correctly in YARN.
The bug description is a little misleading: the actual issue is that
.py files are not handled correctly when distributed by YARN. They're
added to
Repository: spark
Updated Branches:
refs/heads/master 19e30b48f - e90c9d92d
[SPARK-7527] [CORE] Fix createNullValue to return the correct null values and
REPL mode detection
The root cause of SPARK-7527 is `createNullValue` returns an incompatible value
`Byte(0)` for `char` and `boolean`.
Repository: spark
Updated Branches:
refs/heads/branch-1.4 f0513733d - 5e7973df0
[SPARK-8373] [PYSPARK] Add emptyRDD to pyspark and fix the issue when calling
sum on an empty RDD
This PR fixes the sum issue and also adds `emptyRDD` so that it's easy to
create a test case.
Author: zsxwing
Repository: spark
Updated Branches:
refs/heads/branch-1.4 320c4420b - a7f6979d0
[SPARK-7515] [DOC] Update documentation for PySpark on YARN with cluster mode
Now PySpark on YARN with cluster mode is supported so let's update doc.
Author: Kousuke Saruta sarutakoss.nttdata.co.jp
Closes #6040
Repository: spark
Updated Branches:
refs/heads/master 2837e0670 - 0fc4b96f3
[SPARK-8373] [PYSPARK] Add emptyRDD to pyspark and fix the issue when calling
sum on an empty RDD
This PR fixes the sum issue and also adds `emptyRDD` so that it's easy to
create a test case.
Author: zsxwing
be.
cc andrewor14
Author: Burak Yavuz brk...@gmail.com
Closes #6788 from brkyvz/local-ivy-fix and squashes the following commits:
2875bf4 [Burak Yavuz] fix temp dir bug
48cc648 [Burak Yavuz] improve deletion
a69e3e6 [Burak Yavuz] delete cache before test as well
0037197 [Burak Yavuz] fix merge
601 - 700 of 1431 matches
Mail list logo
