[jira] [Resolved] (SPARK-13778) Master's ApplicationPage displays wrong application executor state when a worker is lost

[ https://issues.apache.org/jira/browse/SPARK-13778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13778. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Master's

[jira] [Created] (SPARK-13749) Faster pivot implementation for many distinct values with two phase aggregation

Andrew Ray created SPARK-13749: -- Summary: Faster pivot implementation for many distinct values with two phase aggregation Key: SPARK-13749 URL: https://issues.apache.org/jira/browse/SPARK-13749 Project:

[jira] [Commented] (SPARK-11857) Remove Mesos fine-grained mode subject to discussions

[ https://issues.apache.org/jira/browse/SPARK-11857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15185511#comment-15185511 ] Andrew Or commented on SPARK-11857: --- [~dragos][~tnachen] any thoughts on at least deprecating this? >

[jira] [Resolved] (SPARK-13659) Remove returnValues from BlockStore APIs

[ https://issues.apache.org/jira/browse/SPARK-13659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13659. --- Resolution: Fixed Fix Version/s: 2.0.0 > Remove returnValues from BlockStore APIs >

[jira] [Resolved] (SPARK-13689) Move some methods in CatalystQl to a util object

[ https://issues.apache.org/jira/browse/SPARK-13689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13689. --- Resolution: Fixed Fix Version/s: 2.0.0 > Move some methods in CatalystQl to a util object >

[jira] [Created] (SPARK-13689) Move some methods in CatalystQl to a util object

Andrew Or created SPARK-13689: - Summary: Move some methods in CatalystQl to a util object Key: SPARK-13689 URL: https://issues.apache.org/jira/browse/SPARK-13689 Project: Spark Issue Type:

[jira] [Created] (SPARK-13685) Rename catalog.Catalog to ExternalCatalog

Andrew Or created SPARK-13685: - Summary: Rename catalog.Catalog to ExternalCatalog Key: SPARK-13685 URL: https://issues.apache.org/jira/browse/SPARK-13685 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-13633) Move parser classes to o.a.s.sql.catalyst.parser package

[ https://issues.apache.org/jira/browse/SPARK-13633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13633. --- Resolution: Fixed Fix Version/s: 2.0.0 > Move parser classes to o.a.s.sql.catalyst.parser

[jira] [Updated] (SPARK-13633) Move parser classes to o.a.s.sql.catalyst.parser package

[ https://issues.apache.org/jira/browse/SPARK-13633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13633: -- Summary: Move parser classes to o.a.s.sql.catalyst.parser package (was: Create new

[jira] [Created] (SPARK-13633) Create new o.a.s.sql.catalyst.parser package

Andrew Or created SPARK-13633: - Summary: Create new o.a.s.sql.catalyst.parser package Key: SPARK-13633 URL: https://issues.apache.org/jira/browse/SPARK-13633 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-13632) Create new o.a.s.sql.execution.commands package

Andrew Or created SPARK-13632: - Summary: Create new o.a.s.sql.execution.commands package Key: SPARK-13632 URL: https://issues.apache.org/jira/browse/SPARK-13632 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-12817) Remove CacheManager and replace it with new BlockManager.getOrElseUpdate method

[ https://issues.apache.org/jira/browse/SPARK-12817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12817. --- Resolution: Fixed Fix Version/s: 2.0.0 > Remove CacheManager and replace it with new

[jira] [Created] (SPARK-13610) Create a Transformer to disassemble vectors in DataFrames

Andrew MacKinlay created SPARK-13610: Summary: Create a Transformer to disassemble vectors in DataFrames Key: SPARK-13610 URL: https://issues.apache.org/jira/browse/SPARK-13610 Project: Spark

[jira] [Issue Comment Deleted] (SPARK-13571) Track current database in SQL/HiveContext

[ https://issues.apache.org/jira/browse/SPARK-13571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13571: -- Comment: was deleted (was: User 'andrewor14' has created a pull request for this issue:

[jira] [Created] (SPARK-13571) Track current database in SQL/HiveContext

Andrew Or created SPARK-13571: - Summary: Track current database in SQL/HiveContext Key: SPARK-13571 URL: https://issues.apache.org/jira/browse/SPARK-13571 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-13522) Executor should kill itself when it's unable to heartbeat to the driver more than N times

[ https://issues.apache.org/jira/browse/SPARK-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13522. --- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.0.0 Target

[jira] [Created] (SPARK-13526) Refactor: Move SQLContext/HiveContext per-session state to separate class

Andrew Or created SPARK-13526: - Summary: Refactor: Move SQLContext/HiveContext per-session state to separate class Key: SPARK-13526 URL: https://issues.apache.org/jira/browse/SPARK-13526 Project: Spark

[jira] [Resolved] (SPARK-13519) Driver should tell Executor to stop itself when cleaning executor's state

[ https://issues.apache.org/jira/browse/SPARK-13519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13519. --- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.0.0 Target

[jira] [Created] (SPARK-13524) Remove BroadcastManager

Andrew Or created SPARK-13524: - Summary: Remove BroadcastManager Key: SPARK-13524 URL: https://issues.apache.org/jira/browse/SPARK-13524 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-12757) Use reference counting to prevent blocks from being evicted during reads

[ https://issues.apache.org/jira/browse/SPARK-12757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12757. --- Resolution: Fixed Fix Version/s: 2.0.0 > Use reference counting to prevent blocks from being

[jira] [Resolved] (SPARK-13387) Add support for SPARK_DAEMON_JAVA_OPTS with MesosClusterDispatcher.

[ https://issues.apache.org/jira/browse/SPARK-13387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13387. --- Resolution: Fixed Assignee: Timothy Chen Fix Version/s: 2.0.0 Target

[jira] [Resolved] (SPARK-13501) Remove use of Guava Stopwatch class

[ https://issues.apache.org/jira/browse/SPARK-13501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13501. --- Resolution: Fixed Fix Version/s: 2.0.0 > Remove use of Guava Stopwatch class >

[jira] [Updated] (SPARK-12009) Avoid re-allocate yarn container while driver want to stop all Executors

[ https://issues.apache.org/jira/browse/SPARK-12009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12009: -- Fix Version/s: (was: 1.6.1) > Avoid re-allocate yarn container while driver want to stop all

[jira] [Updated] (SPARK-12009) Avoid re-allocate yarn container while driver want to stop all Executors

[ https://issues.apache.org/jira/browse/SPARK-12009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12009: -- Assignee: SuYan > Avoid re-allocate yarn container while driver want to stop all Executors >

[jira] [Resolved] (SPARK-12009) Avoid re-allocate yarn container while driver want to stop all Executors

[ https://issues.apache.org/jira/browse/SPARK-12009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12009. --- Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 > Avoid re-allocate yarn

[jira] [Resolved] (SPARK-7729) Executor which has been killed should also be displayed on Executors Tab.

[ https://issues.apache.org/jira/browse/SPARK-7729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-7729. -- Resolution: Fixed Assignee: Lianhui Wang Fix Version/s: 2.0.0 Target

[jira] [Resolved] (SPARK-13298) DAG visualization does not render correctly for jobs

[ https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13298. --- Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 > DAG visualization does not

[jira] [Updated] (SPARK-13298) DAG visualization does not render correctly for jobs

[ https://issues.apache.org/jira/browse/SPARK-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13298: -- Target Version/s: 1.6.1, 2.0.0 > DAG visualization does not render correctly for jobs >

[jira] [Updated] (SPARK-12757) Use reference counting to prevent blocks from being evicted during reads

[ https://issues.apache.org/jira/browse/SPARK-12757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12757: -- Target Version/s: 2.0.0 > Use reference counting to prevent blocks from being evicted during reads >

[jira] [Updated] (SPARK-12583) spark shuffle fails with mesos after 2mins

[ https://issues.apache.org/jira/browse/SPARK-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12583: -- Target Version/s: 2.0.0 (was: 1.6.1) > spark shuffle fails with mesos after 2mins >

[jira] [Resolved] (SPARK-10749) Support multiple roles with Spark Mesos dispatcher

[ https://issues.apache.org/jira/browse/SPARK-10749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-10749. --- Resolution: Fixed Fix Version/s: 2.0.0 > Support multiple roles with Spark Mesos dispatcher >

[jira] [Updated] (SPARK-13329) Considering output for statistics of logical plan

[ https://issues.apache.org/jira/browse/SPARK-13329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13329: -- Summary: Considering output for statistics of logical plan (was: Considering output for statistics of

[jira] [Updated] (SPARK-12864) Fetch failure from AM restart

[ https://issues.apache.org/jira/browse/SPARK-12864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12864: -- Summary: Fetch failure from AM restart (was: initialize executorIdCounter after ApplicationMaster

[jira] [Issue Comment Deleted] (SPARK-13054) Always post TaskEnd event for tasks in cancelled stages

[ https://issues.apache.org/jira/browse/SPARK-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13054: -- Comment: was deleted (was: User 'andrewor14' has created a pull request for this issue:

[jira] [Updated] (SPARK-13371) TaskSetManager.dequeueSpeculativeTask compares Option[String] and String directly.

[ https://issues.apache.org/jira/browse/SPARK-13371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13371: -- Assignee: Sean Owen > TaskSetManager.dequeueSpeculativeTask compares Option[String] and String >

[jira] [Resolved] (SPARK-13371) TaskSetManager.dequeueSpeculativeTask compares Option[String] and String directly.

[ https://issues.apache.org/jira/browse/SPARK-13371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13371. --- Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 Target Version/s:

[jira] [Updated] (SPARK-13344) Tests have many "accumulator not found" exceptions

[ https://issues.apache.org/jira/browse/SPARK-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13344: -- Summary: Tests have many "accumulator not found" exceptions (was: SaveLoadSuite has many accumulator

[jira] [Updated] (SPARK-13344) Tests have many "accumulator not found" exceptions

[ https://issues.apache.org/jira/browse/SPARK-13344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13344: -- Description: This is because SparkFunSuite clears all accumulators after every single test. This

[jira] [Updated] (SPARK-13346) Using DataFrames iteratively leads to massive query plans, which slows execution

[ https://issues.apache.org/jira/browse/SPARK-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13346: -- Summary: Using DataFrames iteratively leads to massive query plans, which slows execution (was:

[jira] [Created] (SPARK-13344) SaveLoadSuite has many accumulator exceptions

Andrew Or created SPARK-13344: - Summary: SaveLoadSuite has many accumulator exceptions Key: SPARK-13344 URL: https://issues.apache.org/jira/browse/SPARK-13344 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-13220) Deprecate "yarn-client" and "yarn-cluster"

[ https://issues.apache.org/jira/browse/SPARK-13220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15147037#comment-15147037 ] Andrew Or commented on SPARK-13220: --- I've assigned it to you > Deprecate "yarn-client" and

[jira] [Updated] (SPARK-13220) Deprecate "yarn-client" and "yarn-cluster"

[ https://issues.apache.org/jira/browse/SPARK-13220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13220: -- Target Version/s: 2.0.0 > Deprecate "yarn-client" and "yarn-cluster" >

[jira] [Updated] (SPARK-13220) Deprecate "yarn-client" and "yarn-cluster"

[ https://issues.apache.org/jira/browse/SPARK-13220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13220: -- Assignee: Saisai Shao > Deprecate "yarn-client" and "yarn-cluster" >

[jira] [Updated] (SPARK-12154) Upgrade to Jersey 2

[ https://issues.apache.org/jira/browse/SPARK-12154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-12154: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-11806 > Upgrade to Jersey 2 >

[jira] [Commented] (SPARK-12154) Upgrade to Jersey 2

[ https://issues.apache.org/jira/browse/SPARK-12154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145746#comment-15145746 ] Andrew Ash commented on SPARK-12154: [~khajavi] would you please give it a go? [~mcheah] must have

[jira] [Resolved] (SPARK-5095) Support launching multiple mesos executors in coarse grained mesos mode

[ https://issues.apache.org/jira/browse/SPARK-5095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-5095. -- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Support launching

[jira] [Updated] (SPARK-13269) Expose more executor stats in stable status API

[ https://issues.apache.org/jira/browse/SPARK-13269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13269: -- Issue Type: Improvement (was: Bug) > Expose more executor stats in stable status API >

[jira] [Created] (SPARK-13269) Expose more executor stats in stable status API

Andrew Or created SPARK-13269: - Summary: Expose more executor stats in stable status API Key: SPARK-13269 URL: https://issues.apache.org/jira/browse/SPARK-13269 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-13220) Deprecate "yarn-client" and "yarn-cluster"

[ https://issues.apache.org/jira/browse/SPARK-13220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13220: -- Description: We currently allow `\-\-master yarn-client`. Instead, the user should do `\-\-master yarn

[jira] [Updated] (SPARK-9307) Logging: Make it either stable or private[spark]

[ https://issues.apache.org/jira/browse/SPARK-9307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9307: - Affects Version/s: 1.0.0 Target Version/s: 2.0.0 > Logging: Make it either stable or private[spark]

[jira] [Updated] (SPARK-13220) Deprecate "yarn-client" and "yarn-cluster"

[ https://issues.apache.org/jira/browse/SPARK-13220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13220: -- Description: We currently allow `\--master yarn-client`. Instead, the user should do `--master yarn

[jira] [Updated] (SPARK-9307) Logging: Make it either stable or private[spark]

[ https://issues.apache.org/jira/browse/SPARK-9307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9307: - Assignee: Sean Owen > Logging: Make it either stable or private[spark] >

[jira] [Updated] (SPARK-13220) Deprecate "yarn-client" and "yarn-cluster"

[ https://issues.apache.org/jira/browse/SPARK-13220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13220: -- Description: We currently allow `--master yarn-client`. Instead, the user should do `--master yarn

[jira] [Updated] (SPARK-9307) Logging: Make it either stable or private[spark]

[ https://issues.apache.org/jira/browse/SPARK-9307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9307: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-11806 > Logging: Make it either stable or

[jira] [Assigned] (SPARK-13080) Implementation of the internal catalog API using Hive

[ https://issues.apache.org/jira/browse/SPARK-13080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or reassigned SPARK-13080: - Assignee: Andrew Or > Implementation of the internal catalog API using Hive >

[jira] [Updated] (SPARK-13220) Deprecate "yarn-client" and "yarn-cluster"

[ https://issues.apache.org/jira/browse/SPARK-13220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13220: -- Description: We currently allow `\-\-master yarn-client`. Instead, the user should do `\-\-master yarn

[jira] [Issue Comment Deleted] (SPARK-13054) Always post TaskEnd event for tasks in cancelled stages

[ https://issues.apache.org/jira/browse/SPARK-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13054: -- Comment: was deleted (was: User 'andrewor14' has created a pull request for this issue:

[jira] [Updated] (SPARK-13214) Fix dynamic allocation docs

[ https://issues.apache.org/jira/browse/SPARK-13214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13214: -- Affects Version/s: 1.6.0 Target Version/s: 1.6.1, 2.0.0 Component/s: Documentation >

[jira] [Updated] (SPARK-13214) Fix dynamic allocation docs

[ https://issues.apache.org/jira/browse/SPARK-13214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13214: -- Assignee: Bill Chambers > Fix dynamic allocation docs > --- > >

[jira] [Resolved] (SPARK-13002) Mesos scheduler backend does not follow the property spark.dynamicAllocation.initialExecutors

[ https://issues.apache.org/jira/browse/SPARK-13002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13002. --- Resolution: Fixed Assignee: Luc Bourlier Fix Version/s: 2.0.0 > Mesos scheduler

[jira] [Resolved] (SPARK-13214) Fix dynamic allocation docs

[ https://issues.apache.org/jira/browse/SPARK-13214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13214. --- Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 > Fix dynamic allocation docs

[jira] [Created] (SPARK-13220) Deprecate "yarn-client" and "yarn-cluster"

Andrew Or created SPARK-13220: - Summary: Deprecate "yarn-client" and "yarn-cluster" Key: SPARK-13220 URL: https://issues.apache.org/jira/browse/SPARK-13220 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-13052) waitingApps metric doesn't show the number of apps currently in the WAITING state

[ https://issues.apache.org/jira/browse/SPARK-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13052: -- Assignee: Raafat Akkad > waitingApps metric doesn't show the number of apps currently in the WAITING

[jira] [Resolved] (SPARK-13052) waitingApps metric doesn't show the number of apps currently in the WAITING state

[ https://issues.apache.org/jira/browse/SPARK-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13052. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > waitingApps metric

[jira] [Resolved] (SPARK-13168) Collapse adjacent Repartition operations

[ https://issues.apache.org/jira/browse/SPARK-13168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13168. --- Resolution: Fixed Fix Version/s: 2.0.0 > Collapse adjacent Repartition operations >

[jira] [Resolved] (SPARK-12330) Mesos coarse executor does not cleanup blockmgr properly on termination if data is stored on disk

[ https://issues.apache.org/jira/browse/SPARK-12330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12330. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Mesos coarse

[jira] [Resolved] (SPARK-13164) Replace deprecated synchronizedBuffer in core

[ https://issues.apache.org/jira/browse/SPARK-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13164. --- Resolution: Fixed Assignee: holdenk Fix Version/s: 2.0.0 Target

[jira] [Resolved] (SPARK-13162) Standalone mode does not respect `spark.dynamicAllocation.initialExecutors`

[ https://issues.apache.org/jira/browse/SPARK-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13162. --- Resolution: Fixed Fix Version/s: 2.0.0 > Standalone mode does not respect

[jira] [Resolved] (SPARK-13053) Rectify ignored tests in InternalAccumulatorSuite

[ https://issues.apache.org/jira/browse/SPARK-13053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13053. --- Resolution: Fixed Fix Version/s: 2.0.0 > Rectify ignored tests in InternalAccumulatorSuite >

[jira] [Updated] (SPARK-13151) Investigate replacing SynchronizedBuffer as it is deprecated/unreliable

[ https://issues.apache.org/jira/browse/SPARK-13151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13151: -- Target Version/s: 2.0.0 > Investigate replacing SynchronizedBuffer as it is deprecated/unreliable >

[jira] [Resolved] (SPARK-13152) Fix task metrics deprecation warning

[ https://issues.apache.org/jira/browse/SPARK-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13152. --- Resolution: Fixed Assignee: holdenk Fix Version/s: 2.0.0 Target

[jira] [Created] (SPARK-13162) Standalone mode does not respect `spark.dynamicAllocation.initialExecutors`

Andrew Or created SPARK-13162: - Summary: Standalone mode does not respect `spark.dynamicAllocation.initialExecutors` Key: SPARK-13162 URL: https://issues.apache.org/jira/browse/SPARK-13162 Project: Spark

[jira] [Updated] (SPARK-13162) Standalone mode does not respect `spark.dynamicAllocation.initialExecutors`

[ https://issues.apache.org/jira/browse/SPARK-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13162: -- Description: On the Master we always set `ApplicationInfo#executorLimit` to infinity to start with. >

[jira] [Commented] (SPARK-13065) streaming-twitter pass twitter4j.FilterQuery argument to TwitterUtils.createStream()

[ https://issues.apache.org/jira/browse/SPARK-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15130923#comment-15130923 ] Andrew Davidson commented on SPARK-13065: - The code looks really nice well done Andy >

[jira] [Comment Edited] (SPARK-13065) streaming-twitter pass twitter4j.FilterQuery argument to TwitterUtils.createStream()

[ https://issues.apache.org/jira/browse/SPARK-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128832#comment-15128832 ] Andrew Davidson edited comment on SPARK-13065 at 2/2/16 7:20 PM: - Hi

[jira] [Updated] (SPARK-13065) streaming-twitter pass twitter4j.FilterQuery argument to TwitterUtils.createStream()

[ https://issues.apache.org/jira/browse/SPARK-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Davidson updated SPARK-13065: Description: The twitter stream api is very powerful provides a lot of support for

[jira] [Updated] (SPARK-13065) streaming-twitter pass twitter4j.FilterQuery argument to TwitterUtils.createStream()

[ https://issues.apache.org/jira/browse/SPARK-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Davidson updated SPARK-13065: Attachment: twitterFilterQueryPatch.tar.gz sorry bad name. its not in patch format >

[jira] [Commented] (SPARK-13009) spark-streaming-twitter_2.10 does not make it possible to access the raw twitter json

[ https://issues.apache.org/jira/browse/SPARK-13009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128843#comment-15128843 ] Andrew Davidson commented on SPARK-13009: - I Sean I total agree with you. The Twitter4j people

[jira] [Updated] (SPARK-13122) Race condition in MemoryStore.unrollSafely() causes memory leak

[ https://issues.apache.org/jira/browse/SPARK-13122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-13122: -- Assignee: Adam Budde > Race condition in MemoryStore.unrollSafely() causes memory leak >

[jira] [Resolved] (SPARK-13122) Race condition in MemoryStore.unrollSafely() causes memory leak

[ https://issues.apache.org/jira/browse/SPARK-13122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-13122. --- Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 Target Version/s:

[jira] [Resolved] (SPARK-12465) Remove spark.deploy.mesos.zookeeper.dir and use spark.deploy.zookeeper.dir

[ https://issues.apache.org/jira/browse/SPARK-12465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12465. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Remove

[jira] [Updated] (SPARK-10749) Support multiple roles with Spark Mesos dispatcher

[ https://issues.apache.org/jira/browse/SPARK-10749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10749: -- Assignee: Timothy Chen > Support multiple roles with Spark Mesos dispatcher >

[jira] [Updated] (SPARK-10749) Support multiple roles with Spark Mesos dispatcher

[ https://issues.apache.org/jira/browse/SPARK-10749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-10749: -- Target Version/s: 2.0.0 > Support multiple roles with Spark Mesos dispatcher >

[jira] [Resolved] (SPARK-12265) Spark calls System.exit inside driver instead of throwing exception

[ https://issues.apache.org/jira/browse/SPARK-12265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12265. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Spark calls

[jira] [Resolved] (SPARK-6847) Stack overflow on updateStateByKey which followed by a dstream with checkpoint set

[ https://issues.apache.org/jira/browse/SPARK-6847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-6847. -- Resolution: Fixed Fix Version/s: 2.0.0 > Stack overflow on updateStateByKey which followed by a

[jira] [Updated] (SPARK-6847) Stack overflow on updateStateByKey which followed by a dstream with checkpoint set

[ https://issues.apache.org/jira/browse/SPARK-6847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6847: - Target Version/s: 2.0.0 > Stack overflow on updateStateByKey which followed by a dstream with >

[jira] [Commented] (SPARK-13009) spark-streaming-twitter_2.10 does not make it possible to access the raw twitter json

[ https://issues.apache.org/jira/browse/SPARK-13009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126936#comment-15126936 ] Andrew Davidson commented on SPARK-13009: - Agreed. they ask me to file a spark RFE. I post your

[jira] [Resolved] (SPARK-12463) Remove spark.deploy.mesos.recoveryMode and use spark.deploy.recoveryMode

[ https://issues.apache.org/jira/browse/SPARK-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12463. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Remove

[jira] [Resolved] (SPARK-12464) Remove spark.deploy.mesos.zookeeper.url and use spark.deploy.zookeeper.url

[ https://issues.apache.org/jira/browse/SPARK-12464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12464. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Remove

[jira] [Resolved] (SPARK-10647) Mesos HA mode misuses spark.deploy.zookeeper.dir property; configs should be documented

[ https://issues.apache.org/jira/browse/SPARK-10647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-10647. --- Resolution: Fixed Fix Version/s: 2.0.0 Target Version/s: 2.0.0 > Mesos HA mode

[jira] [Commented] (SPARK-13065) streaming-twitter pass twitter4j.FilterQuery argument to TwitterUtils.createStream()

[ https://issues.apache.org/jira/browse/SPARK-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126931#comment-15126931 ] Andrew Davidson commented on SPARK-13065: - Thanks Sachin I do not know Scala. As a temporary

[jira] [Updated] (SPARK-12265) Spark calls System.exit inside driver instead of throwing exception

[ https://issues.apache.org/jira/browse/SPARK-12265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12265: -- Assignee: Iulian Dragos > Spark calls System.exit inside driver instead of throwing exception >

[jira] [Updated] (SPARK-12832) Mesos cluster mode should handle constraints

[ https://issues.apache.org/jira/browse/SPARK-12832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12832: -- Summary: Mesos cluster mode should handle constraints (was: spark on mesos dispacher need a

[jira] [Updated] (SPARK-12265) Spark calls System.exit inside driver instead of throwing exception

[ https://issues.apache.org/jira/browse/SPARK-12265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12265: -- Assignee: Nilanjan Raychaudhuri (was: Iulian Dragos) > Spark calls System.exit inside driver instead

[jira] [Commented] (SPARK-12423) Mesos executor home should not be resolved on the driver's file system

[ https://issues.apache.org/jira/browse/SPARK-12423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127083#comment-15127083 ] Andrew Or commented on SPARK-12423: --- Is this resolved then now that its duplicate is resolved? > Mesos

[jira] [Resolved] (SPARK-12979) Paths are resolved relative to the local file system

[ https://issues.apache.org/jira/browse/SPARK-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12979. --- Resolution: Fixed Assignee: Iulian Dragos Fix Version/s: 2.0.0 Target

[jira] [Updated] (SPARK-11638) Run Spark on Mesos with bridge networking

[ https://issues.apache.org/jira/browse/SPARK-11638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-11638: -- Summary: Run Spark on Mesos with bridge networking (was: Apache Spark in Docker with Bridge

[jira] [Resolved] (SPARK-12790) Remove HistoryServer old multiple files format

[ https://issues.apache.org/jira/browse/SPARK-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12790. --- Resolution: Fixed Fix Version/s: 2.0.0 > Remove HistoryServer old multiple files format >

[jira] [Updated] (SPARK-12637) Print stage info of finished stages properly

[ https://issues.apache.org/jira/browse/SPARK-12637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12637: -- Assignee: Navis > Print stage info of finished stages properly >

[jira] [Resolved] (SPARK-12637) Print stage info of finished stages properly

[ https://issues.apache.org/jira/browse/SPARK-12637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12637. --- Resolution: Fixed Fix Version/s: 2.0.0 > Print stage info of finished stages properly >

<    3   4   5   6   7   8   9   10   11   12   >