[jira] [Commented] (SPARK-21866) SPIP: Image support in Spark

[ https://issues.apache.org/jira/browse/SPARK-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16152915#comment-16152915 ] Matei Zaharia commented on SPARK-21866: --- Just to chime in on this, I've also seen feedback that the

[jira] [Updated] (SPARK-18278) SPIP: Support native submission of spark jobs to a kubernetes cluster

[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-18278: -- Labels: SPIP (was: ) > SPIP: Support native submission of spark jobs to a kubernetes cluster

[jira] [Updated] (SPARK-21866) SPIP: Image support in Spark

[ https://issues.apache.org/jira/browse/SPARK-21866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-21866: -- Labels: SPIP (was: ) > SPIP: Image support in Spark > > >

[jira] [Commented] (SPARK-17445) Reference an ASF page as the main place to find third-party packages

[ https://issues.apache.org/jira/browse/SPARK-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484732#comment-15484732 ] Matei Zaharia commented on SPARK-17445: --- Sounds good to me. > Reference an ASF page as the main

[jira] [Commented] (SPARK-17445) Reference an ASF page as the main place to find third-party packages

[ https://issues.apache.org/jira/browse/SPARK-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15480419#comment-15480419 ] Matei Zaharia commented on SPARK-17445: --- Sounds good, but IMO just keep the current supplemental

[jira] [Commented] (SPARK-17445) Reference an ASF page as the main place to find third-party packages

[ https://issues.apache.org/jira/browse/SPARK-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15479121#comment-15479121 ] Matei Zaharia commented on SPARK-17445: --- The powered by wiki page is a bit of a mess IMO, so I'd

[jira] [Commented] (SPARK-17445) Reference an ASF page as the main place to find third-party packages

[ https://issues.apache.org/jira/browse/SPARK-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15474543#comment-15474543 ] Matei Zaharia commented on SPARK-17445: --- I think one part you're missing, Josh, is that

[jira] [Created] (SPARK-17445) Reference an ASF page as the main place to find third-party packages

Matei Zaharia created SPARK-17445: - Summary: Reference an ASF page as the main place to find third-party packages Key: SPARK-17445 URL: https://issues.apache.org/jira/browse/SPARK-17445 Project:

[jira] [Commented] (SPARK-16031) Add debug-only socket source in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-16031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15337182#comment-15337182 ] Matei Zaharia commented on SPARK-16031: --- FYI I'll post a PR for this soon. > Add debug-only socket

[jira] [Created] (SPARK-16031) Add debug-only socket source in Structured Streaming

Matei Zaharia created SPARK-16031: - Summary: Add debug-only socket source in Structured Streaming Key: SPARK-16031 URL: https://issues.apache.org/jira/browse/SPARK-16031 Project: Spark Issue

[jira] [Created] (SPARK-15879) Update logo in UI and docs to add "Apache"

Matei Zaharia created SPARK-15879: - Summary: Update logo in UI and docs to add "Apache" Key: SPARK-15879 URL: https://issues.apache.org/jira/browse/SPARK-15879 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-14356) Update spark.sql.execution.debug to work on Datasets

[ https://issues.apache.org/jira/browse/SPARK-14356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia reassigned SPARK-14356: - Assignee: Matei Zaharia > Update spark.sql.execution.debug to work on Datasets >

[jira] [Created] (SPARK-14356) Update spark.sql.execution.debug to work on Datasets

Matei Zaharia created SPARK-14356: - Summary: Update spark.sql.execution.debug to work on Datasets Key: SPARK-14356 URL: https://issues.apache.org/jira/browse/SPARK-14356 Project: Spark Issue

[jira] [Commented] (SPARK-10854) MesosExecutorBackend: Received launchTask but executor was null

[ https://issues.apache.org/jira/browse/SPARK-10854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15038058#comment-15038058 ] Matei Zaharia commented on SPARK-10854: --- Just a note, I saw a log where this happened, and the

[jira] [Created] (SPARK-11733) Allow shuffle readers to request data from just one mapper

Matei Zaharia created SPARK-11733: - Summary: Allow shuffle readers to request data from just one mapper Key: SPARK-11733 URL: https://issues.apache.org/jira/browse/SPARK-11733 Project: Spark

[jira] [Commented] (SPARK-9999) RDD-like API on top of Catalyst/DataFrame

[ https://issues.apache.org/jira/browse/SPARK-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961567#comment-14961567 ] Matei Zaharia commented on SPARK-: -- Beyond tuples, you'll also want encoders for other generic

[jira] [Commented] (SPARK-9850) Adaptive execution in Spark

[ https://issues.apache.org/jira/browse/SPARK-9850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14907518#comment-14907518 ] Matei Zaharia commented on SPARK-9850: -- Hey Imran, this could make sense, but note that the problem

[jira] [Resolved] (SPARK-9852) Let reduce tasks fetch multiple map output partitions

[ https://issues.apache.org/jira/browse/SPARK-9852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-9852. -- Resolution: Fixed Fix Version/s: 1.6.0 > Let reduce tasks fetch multiple map output

[jira] [Updated] (SPARK-9852) Let reduce tasks fetch multiple map output partitions

[ https://issues.apache.org/jira/browse/SPARK-9852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-9852: - Summary: Let reduce tasks fetch multiple map output partitions (was: Let HashShuffleFetcher

[jira] [Resolved] (SPARK-9851) Support submitting map stages individually in DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-9851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-9851. -- Resolution: Fixed Fix Version/s: 1.6.0 > Support submitting map stages individually in

[jira] [Assigned] (SPARK-9853) Optimize shuffle fetch of contiguous partition IDs

[ https://issues.apache.org/jira/browse/SPARK-9853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia reassigned SPARK-9853: Assignee: Matei Zaharia Optimize shuffle fetch of contiguous partition IDs

[jira] [Resolved] (SPARK-10008) Shuffle locality can take precedence over narrow dependencies for RDDs with both

[ https://issues.apache.org/jira/browse/SPARK-10008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-10008. --- Resolution: Fixed Fix Version/s: 1.5.0 Shuffle locality can take precedence over

[jira] [Assigned] (SPARK-10008) Shuffle locality can take precedence over narrow dependencies for RDDs with both

[ https://issues.apache.org/jira/browse/SPARK-10008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia reassigned SPARK-10008: - Assignee: Matei Zaharia Shuffle locality can take precedence over narrow dependencies

[jira] [Created] (SPARK-10008) Shuffle locality can take precedence over narrow dependencies for RDDs with both

Matei Zaharia created SPARK-10008: - Summary: Shuffle locality can take precedence over narrow dependencies for RDDs with both Key: SPARK-10008 URL: https://issues.apache.org/jira/browse/SPARK-10008

[jira] [Updated] (SPARK-9851) Support submitting map stages individually in DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-9851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-9851: - Summary: Support submitting map stages individually in DAGScheduler (was: Add support for

[jira] [Updated] (SPARK-9923) ShuffleMapStage.numAvailableOutputs should be an Int instead of Long

[ https://issues.apache.org/jira/browse/SPARK-9923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-9923: - Labels: Starter (was: ) ShuffleMapStage.numAvailableOutputs should be an Int instead of Long

[jira] [Created] (SPARK-9923) ShuffleMapStage.numAvailableOutputs should be an Int instead of Long

Matei Zaharia created SPARK-9923: Summary: ShuffleMapStage.numAvailableOutputs should be an Int instead of Long Key: SPARK-9923 URL: https://issues.apache.org/jira/browse/SPARK-9923 Project: Spark

[jira] [Updated] (SPARK-9850) Adaptive execution in Spark

[ https://issues.apache.org/jira/browse/SPARK-9850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-9850: - Issue Type: Epic (was: New Feature) Adaptive execution in Spark ---

[jira] [Updated] (SPARK-9850) Adaptive execution in Spark

[ https://issues.apache.org/jira/browse/SPARK-9850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-9850: - Assignee: Yin Huai Adaptive execution in Spark ---

[jira] [Assigned] (SPARK-9851) Add support for submitting map stages individually in DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-9851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia reassigned SPARK-9851: Assignee: Matei Zaharia Add support for submitting map stages individually in

[jira] [Created] (SPARK-9852) Let HashShuffleFetcher fetch multiple map output partitions

Matei Zaharia created SPARK-9852: Summary: Let HashShuffleFetcher fetch multiple map output partitions Key: SPARK-9852 URL: https://issues.apache.org/jira/browse/SPARK-9852 Project: Spark

[jira] [Assigned] (SPARK-9852) Let HashShuffleFetcher fetch multiple map output partitions

[ https://issues.apache.org/jira/browse/SPARK-9852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia reassigned SPARK-9852: Assignee: Matei Zaharia Let HashShuffleFetcher fetch multiple map output partitions

[jira] [Created] (SPARK-9851) Add support for submitting map stages individually in DAGScheduler

Matei Zaharia created SPARK-9851: Summary: Add support for submitting map stages individually in DAGScheduler Key: SPARK-9851 URL: https://issues.apache.org/jira/browse/SPARK-9851 Project: Spark

[jira] [Updated] (SPARK-9850) Adaptive execution in Spark

[ https://issues.apache.org/jira/browse/SPARK-9850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-9850: - Attachment: AdaptiveExecutionInSpark.pdf Adaptive execution in Spark

[jira] [Created] (SPARK-9850) Adaptive execution in Spark

Matei Zaharia created SPARK-9850: Summary: Adaptive execution in Spark Key: SPARK-9850 URL: https://issues.apache.org/jira/browse/SPARK-9850 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-9853) Optimize shuffle fetch of contiguous partition IDs

Matei Zaharia created SPARK-9853: Summary: Optimize shuffle fetch of contiguous partition IDs Key: SPARK-9853 URL: https://issues.apache.org/jira/browse/SPARK-9853 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-9244) Increase some default memory limits

[ https://issues.apache.org/jira/browse/SPARK-9244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-9244. -- Resolution: Fixed Fix Version/s: 1.5.0 Increase some default memory limits

[jira] [Created] (SPARK-9244) Increase some default memory limits

Matei Zaharia created SPARK-9244: Summary: Increase some default memory limits Key: SPARK-9244 URL: https://issues.apache.org/jira/browse/SPARK-9244 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-8110) DAG visualizations sometimes look weird in Python

[ https://issues.apache.org/jira/browse/SPARK-8110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-8110: - Attachment: Screen Shot 2015-06-04 at 1.51.32 PM.png Screen Shot 2015-06-04 at

[jira] [Created] (SPARK-8110) DAG visualizations sometimes look weird in Python

Matei Zaharia created SPARK-8110: Summary: DAG visualizations sometimes look weird in Python Key: SPARK-8110 URL: https://issues.apache.org/jira/browse/SPARK-8110 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-7298) Harmonize style of new UI visualizations

[ https://issues.apache.org/jira/browse/SPARK-7298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-7298. -- Resolution: Fixed Fix Version/s: 1.4.0 Harmonize style of new UI visualizations

[jira] [Commented] (SPARK-7261) Change default log level to WARN in the REPL

[ https://issues.apache.org/jira/browse/SPARK-7261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14520366#comment-14520366 ] Matei Zaharia commented on SPARK-7261: -- IMO we can do this even without SPARK-7260 in

[jira] [Created] (SPARK-6778) SQL contexts in spark-shell and pyspark should both be called sqlContext

Matei Zaharia created SPARK-6778: Summary: SQL contexts in spark-shell and pyspark should both be called sqlContext Key: SPARK-6778 URL: https://issues.apache.org/jira/browse/SPARK-6778 Project:

[jira] [Commented] (SPARK-6646) Spark 2.0: Rearchitecting Spark for Mobile Platforms

[ https://issues.apache.org/jira/browse/SPARK-6646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14391456#comment-14391456 ] Matei Zaharia commented on SPARK-6646: -- Not to rain on the parade here, but I worry

[jira] [Commented] (SPARK-1564) Add JavaScript into Javadoc to turn ::Experimental:: and such into badges

[ https://issues.apache.org/jira/browse/SPARK-1564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359017#comment-14359017 ] Matei Zaharia commented on SPARK-1564: -- This is still a valid issue AFAIK, isn't it?

[jira] [Commented] (SPARK-5654) Integrate SparkR into Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14309782#comment-14309782 ] Matei Zaharia commented on SPARK-5654: -- Yup, there's a tradeoff, but given that this

[jira] [Resolved] (SPARK-5608) Improve SEO of Spark documentation site to let Google find latest docs

[ https://issues.apache.org/jira/browse/SPARK-5608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-5608. -- Resolution: Fixed Fix Version/s: 1.3.0 Improve SEO of Spark documentation site to let

[jira] [Updated] (SPARK-5088) Use spark-class for running executors directly on mesos

[ https://issues.apache.org/jira/browse/SPARK-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-5088: - Fix Version/s: (was: 1.2.1) Use spark-class for running executors directly on mesos

[jira] [Updated] (SPARK-5088) Use spark-class for running executors directly on mesos

[ https://issues.apache.org/jira/browse/SPARK-5088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-5088: - Target Version/s: 1.3.0 (was: 1.3.0, 1.2.1) Use spark-class for running executors directly on

[jira] [Resolved] (SPARK-3619) Upgrade to Mesos 0.21 to work around MESOS-1688

[ https://issues.apache.org/jira/browse/SPARK-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-3619. -- Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Jongyoul Lee (was: Timothy

[jira] [Commented] (SPARK-4660) JavaSerializer uses wrong classloader

[ https://issues.apache.org/jira/browse/SPARK-4660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14260544#comment-14260544 ] Matei Zaharia commented on SPARK-4660: -- [~pkolaczk] mind sending a pull request

[jira] [Commented] (SPARK-3247) Improved support for external data sources

[ https://issues.apache.org/jira/browse/SPARK-3247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14243253#comment-14243253 ] Matei Zaharia commented on SPARK-3247: -- For those looking to learn about the

[jira] [Commented] (SPARK-4690) AppendOnlyMap seems not using Quadratic probing as the JavaDoc

[ https://issues.apache.org/jira/browse/SPARK-4690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1429#comment-1429 ] Matei Zaharia commented on SPARK-4690: -- Yup, that's the definition of it.

[jira] [Closed] (SPARK-4690) AppendOnlyMap seems not using Quadratic probing as the JavaDoc

[ https://issues.apache.org/jira/browse/SPARK-4690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia closed SPARK-4690. Resolution: Invalid AppendOnlyMap seems not using Quadratic probing as the JavaDoc

[jira] [Created] (SPARK-4683) Add a beeline.cmd to run on Windows

Matei Zaharia created SPARK-4683: Summary: Add a beeline.cmd to run on Windows Key: SPARK-4683 URL: https://issues.apache.org/jira/browse/SPARK-4683 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-4684) Add a script to run JDBC server on Windows

Matei Zaharia created SPARK-4684: Summary: Add a script to run JDBC server on Windows Key: SPARK-4684 URL: https://issues.apache.org/jira/browse/SPARK-4684 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-4685) Update JavaDoc settings to include spark.ml and all spark.mllib subpackages in the right sections

[ https://issues.apache.org/jira/browse/SPARK-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-4685: - Priority: Trivial (was: Major) Update JavaDoc settings to include spark.ml and all spark.mllib

[jira] [Created] (SPARK-4685) Update JavaDoc settings to include spark.ml and all spark.mllib subpackages in the right sections

Matei Zaharia created SPARK-4685: Summary: Update JavaDoc settings to include spark.ml and all spark.mllib subpackages in the right sections Key: SPARK-4685 URL: https://issues.apache.org/jira/browse/SPARK-4685

[jira] [Updated] (SPARK-4685) Update JavaDoc settings to include spark.ml and all spark.mllib subpackages in the right sections

[ https://issues.apache.org/jira/browse/SPARK-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-4685: - Target Version/s: 1.2.1 (was: 1.2.0) Update JavaDoc settings to include spark.ml and all

[jira] [Resolved] (SPARK-4613) Make JdbcRDD easier to use from Java

[ https://issues.apache.org/jira/browse/SPARK-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-4613. -- Resolution: Fixed Fix Version/s: 1.2.0 Make JdbcRDD easier to use from Java

[jira] [Updated] (SPARK-4613) Make JdbcRDD easier to use from Java

[ https://issues.apache.org/jira/browse/SPARK-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-4613: - Issue Type: Improvement (was: Bug) Make JdbcRDD easier to use from Java

[jira] [Resolved] (SPARK-3628) Don't apply accumulator updates multiple times for tasks in result stages

[ https://issues.apache.org/jira/browse/SPARK-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-3628. -- Resolution: Fixed Fix Version/s: 1.2.0 Target Version/s: 1.1.2 (was: 0.9.3,

[jira] [Commented] (SPARK-3628) Don't apply accumulator updates multiple times for tasks in result stages

[ https://issues.apache.org/jira/browse/SPARK-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227077#comment-14227077 ] Matei Zaharia commented on SPARK-3628: -- FYI I merged this into 1.2.0, since the patch

[jira] [Commented] (SPARK-732) Recomputation of RDDs may result in duplicated accumulator updates

[ https://issues.apache.org/jira/browse/SPARK-732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14227108#comment-14227108 ] Matei Zaharia commented on SPARK-732: - As discussed on

[jira] [Reopened] (SPARK-3628) Don't apply accumulator updates multiple times for tasks in result stages

[ https://issues.apache.org/jira/browse/SPARK-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia reopened SPARK-3628: -- Don't apply accumulator updates multiple times for tasks in result stages

[jira] [Created] (SPARK-4613) Make JdbcRDD easier to use from Java

Matei Zaharia created SPARK-4613: Summary: Make JdbcRDD easier to use from Java Key: SPARK-4613 URL: https://issues.apache.org/jira/browse/SPARK-4613 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4613) Make JdbcRDD easier to use from Java

[ https://issues.apache.org/jira/browse/SPARK-4613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225615#comment-14225615 ] Matei Zaharia commented on SPARK-4613: -- BTW the strawman for this would be a version

[jira] [Commented] (SPARK-3633) Fetches failure observed after SPARK-2711

[ https://issues.apache.org/jira/browse/SPARK-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222545#comment-14222545 ] Matei Zaharia commented on SPARK-3633: -- [~stephen] you can try the 1.1.1 RC in

[jira] [Commented] (SPARK-4452) Shuffle data structures can starve others on the same thread for memory

[ https://issues.apache.org/jira/browse/SPARK-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14216691#comment-14216691 ] Matei Zaharia commented on SPARK-4452: -- BTW I've thought about this more and here's

[jira] [Commented] (SPARK-4452) Shuffle data structures can starve others on the same thread for memory

[ https://issues.apache.org/jira/browse/SPARK-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14217331#comment-14217331 ] Matei Zaharia commented on SPARK-4452: -- Forced spilling is orthogonal to how you set

[jira] [Commented] (SPARK-4452) Shuffle data structures can starve others on the same thread for memory

[ https://issues.apache.org/jira/browse/SPARK-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14215425#comment-14215425 ] Matei Zaharia commented on SPARK-4452: -- How much of this gets fixed if you fix the

[jira] [Commented] (SPARK-4452) Shuffle data structures can starve others on the same thread for memory

[ https://issues.apache.org/jira/browse/SPARK-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14215557#comment-14215557 ] Matei Zaharia commented on SPARK-4452: -- BTW we may also want to create a separate

[jira] [Commented] (SPARK-4452) Shuffle data structures can starve others on the same thread for memory

[ https://issues.apache.org/jira/browse/SPARK-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14215556#comment-14215556 ] Matei Zaharia commented on SPARK-4452: -- Got it. It would be fine to do this if you

[jira] [Updated] (SPARK-4306) LogisticRegressionWithLBFGS support for PySpark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-4306: - Target Version/s: 1.2.0 LogisticRegressionWithLBFGS support for PySpark MLlib

[jira] [Commented] (SPARK-4306) LogisticRegressionWithLBFGS support for PySpark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14214134#comment-14214134 ] Matei Zaharia commented on SPARK-4306: -- [~srinathsmn] I've assigned it to you. When

[jira] [Updated] (SPARK-4306) LogisticRegressionWithLBFGS support for PySpark MLlib

[ https://issues.apache.org/jira/browse/SPARK-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-4306: - Assignee: Varadharajan LogisticRegressionWithLBFGS support for PySpark MLlib

[jira] [Created] (SPARK-4435) Add setThreshold in Python LogisticRegressionModel and SVMModel

Matei Zaharia created SPARK-4435: Summary: Add setThreshold in Python LogisticRegressionModel and SVMModel Key: SPARK-4435 URL: https://issues.apache.org/jira/browse/SPARK-4435 Project: Spark

[jira] [Commented] (SPARK-4434) spark-submit cluster deploy mode JAR URLs are broken in 1.1.1

[ https://issues.apache.org/jira/browse/SPARK-4434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14214155#comment-14214155 ] Matei Zaharia commented on SPARK-4434: -- [~joshrosen] make sure to revert this on 1.2

[jira] [Created] (SPARK-4439) Export RandomForest in Python

Matei Zaharia created SPARK-4439: Summary: Export RandomForest in Python Key: SPARK-4439 URL: https://issues.apache.org/jira/browse/SPARK-4439 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-4439) Expose RandomForest in Python

[ https://issues.apache.org/jira/browse/SPARK-4439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-4439: - Summary: Expose RandomForest in Python (was: Export RandomForest in Python) Expose RandomForest

[jira] [Resolved] (SPARK-4330) Link to proper URL for YARN overview

[ https://issues.apache.org/jira/browse/SPARK-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-4330. -- Resolution: Fixed Fix Version/s: 1.2.0 1.1.1 Target

[jira] [Updated] (SPARK-4330) Link to proper URL for YARN overview

[ https://issues.apache.org/jira/browse/SPARK-4330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-4330: - Assignee: Kousuke Saruta Link to proper URL for YARN overview

[jira] [Commented] (SPARK-4303) [MLLIB] Use Long IDs instead of Int in ALS.Rating class

[ https://issues.apache.org/jira/browse/SPARK-4303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14203147#comment-14203147 ] Matei Zaharia commented on SPARK-4303: -- Yup, this will actually become easier with

[jira] [Resolved] (SPARK-4186) Support binaryFiles and binaryRecords API in Python

[ https://issues.apache.org/jira/browse/SPARK-4186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-4186. -- Resolution: Fixed Fix Version/s: 1.2.0 Support binaryFiles and binaryRecords API in

[jira] [Resolved] (SPARK-644) Jobs canceled due to repeated executor failures may hang

[ https://issues.apache.org/jira/browse/SPARK-644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-644. - Resolution: Fixed Jobs canceled due to repeated executor failures may hang

[jira] [Resolved] (SPARK-643) Standalone master crashes during actor restart

[ https://issues.apache.org/jira/browse/SPARK-643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-643. - Resolution: Fixed Standalone master crashes during actor restart

[jira] [Commented] (SPARK-677) PySpark should not collect results through local filesystem

[ https://issues.apache.org/jira/browse/SPARK-677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200514#comment-14200514 ] Matei Zaharia commented on SPARK-677: - [~joshrosen] is this fixed now? PySpark should

[jira] [Resolved] (SPARK-681) Optimize hashtables used in Spark

[ https://issues.apache.org/jira/browse/SPARK-681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-681. - Resolution: Fixed Optimize hashtables used in Spark -

[jira] [Resolved] (SPARK-993) Don't reuse Writable objects in HadoopRDDs by default

[ https://issues.apache.org/jira/browse/SPARK-993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-993. - Resolution: Won't Fix We investigated this for 1.0 but found that many InputFormats behave wrongly

[jira] [Commented] (SPARK-993) Don't reuse Writable objects in HadoopRDDs by default

[ https://issues.apache.org/jira/browse/SPARK-993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14200531#comment-14200531 ] Matei Zaharia commented on SPARK-993: - Arun, you'd see this issue if you do collect()

[jira] [Closed] (SPARK-1000) Crash when running SparkPi example with local-cluster

[ https://issues.apache.org/jira/browse/SPARK-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia closed SPARK-1000. Resolution: Cannot Reproduce Crash when running SparkPi example with local-cluster

[jira] [Resolved] (SPARK-1023) Remove Thread.sleep(5000) from TaskSchedulerImpl

[ https://issues.apache.org/jira/browse/SPARK-1023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1023. -- Resolution: Fixed Remove Thread.sleep(5000) from TaskSchedulerImpl

[jira] [Resolved] (SPARK-1185) In Spark Programming Guide, Master URLs should mention yarn-client

[ https://issues.apache.org/jira/browse/SPARK-1185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-1185. -- Resolution: Fixed In Spark Programming Guide, Master URLs should mention yarn-client

[jira] [Closed] (SPARK-2237) Add ZLIBCompressionCodec code

[ https://issues.apache.org/jira/browse/SPARK-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia closed SPARK-2237. Resolution: Won't Fix Add ZLIBCompressionCodec code -

[jira] [Updated] (SPARK-2348) In Windows having a enviorinment variable named 'classpath' gives error

[ https://issues.apache.org/jira/browse/SPARK-2348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-2348: - Priority: Critical (was: Major) In Windows having a enviorinment variable named 'classpath'

[jira] [Updated] (SPARK-4222) FixedLengthBinaryRecordReader should readFully

[ https://issues.apache.org/jira/browse/SPARK-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-4222: - Assignee: Jascha Swisher FixedLengthBinaryRecordReader should readFully

[jira] [Resolved] (SPARK-4222) FixedLengthBinaryRecordReader should readFully

[ https://issues.apache.org/jira/browse/SPARK-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-4222. -- Resolution: Fixed Fix Version/s: 1.2.0 FixedLengthBinaryRecordReader should readFully

[jira] [Updated] (SPARK-4040) Update spark documentation for local mode and spark-streaming.

[ https://issues.apache.org/jira/browse/SPARK-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia updated SPARK-4040: - Assignee: jay vyas Update spark documentation for local mode and spark-streaming.

[jira] [Resolved] (SPARK-4040) Update spark documentation for local mode and spark-streaming.

[ https://issues.apache.org/jira/browse/SPARK-4040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-4040. -- Resolution: Fixed Update spark documentation for local mode and spark-streaming.

[jira] [Resolved] (SPARK-565) Integrate spark in scala standard collection API

[ https://issues.apache.org/jira/browse/SPARK-565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-565. - Resolution: Won't Fix FYI I'm going to close this because we've locked down the API for 1.X, and

  1   2   3   4   5   6   7   8   9   10   >