[jira] [Commented] (SPARK-12062) Master rebuilding historical SparkUI should be asynchronous

[ https://issues.apache.org/jira/browse/SPARK-12062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054829#comment-15054829 ] Andrew Or commented on SPARK-12062: --- I see, if you already have a patch then this is worth fixing.

[jira] [Resolved] (SPARK-12267) Standalone master keeps references to disassociated workers until they sent no heartbeats

[ https://issues.apache.org/jira/browse/SPARK-12267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-12267. -- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.0.0

[jira] [Assigned] (SPARK-12309) Use sqlContext from MLlibTestSparkContext for spark.ml test suites

[ https://issues.apache.org/jira/browse/SPARK-12309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12309: Assignee: (was: Apache Spark) > Use sqlContext from MLlibTestSparkContext for

[jira] [Assigned] (SPARK-12309) Use sqlContext from MLlibTestSparkContext for spark.ml test suites

[ https://issues.apache.org/jira/browse/SPARK-12309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12309: Assignee: Apache Spark > Use sqlContext from MLlibTestSparkContext for spark.ml test

[jira] [Updated] (SPARK-12309) Use sqlContext from MLlibTestSparkContext for spark.ml test suites

[ https://issues.apache.org/jira/browse/SPARK-12309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-12309: Description: Use sqlContext from MLlibTestSparkContext rather than creating new one for spark.ml

[jira] [Created] (SPARK-12308) Better onDisconnected logic for Master

Shixiong Zhu created SPARK-12308: Summary: Better onDisconnected logic for Master Key: SPARK-12308 URL: https://issues.apache.org/jira/browse/SPARK-12308 Project: Spark Issue Type:

[jira] [Updated] (SPARK-12309) Use sqlContext from MLlibTestSparkContext for spark.ml test suites

[ https://issues.apache.org/jira/browse/SPARK-12309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-12309: Description: Use sqlContext from MLlibTestSparkContext rather than creating new one for each

[jira] [Created] (SPARK-12309) Use sqlContext from MLlibTestSparkContext for spark.ml test suites

Yanbo Liang created SPARK-12309: --- Summary: Use sqlContext from MLlibTestSparkContext for spark.ml test suites Key: SPARK-12309 URL: https://issues.apache.org/jira/browse/SPARK-12309 Project: Spark

[jira] [Assigned] (SPARK-12300) Fix schema inferance on local collections

[ https://issues.apache.org/jira/browse/SPARK-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12300: Assignee: Apache Spark > Fix schema inferance on local collections >

[jira] [Assigned] (SPARK-12300) Fix schema inferance on local collections

[ https://issues.apache.org/jira/browse/SPARK-12300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12300: Assignee: (was: Apache Spark) > Fix schema inferance on local collections >

[jira] [Updated] (SPARK-12305) Adds Receiver scheduling info onto Spark Streaming web UI

[ https://issues.apache.org/jira/browse/SPARK-12305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12305: -- Summary: Adds Receiver scheduling info onto Spark Streaming web UI (was: Adds Receiver scheduling

[jira] [Updated] (SPARK-12305) Adds Receiver scheduling info onto Spark Streaming web UI

[ https://issues.apache.org/jira/browse/SPARK-12305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12305: -- Component/s: Streaming > Adds Receiver scheduling info onto Spark Streaming web UI >

[jira] [Updated] (SPARK-12305) Add Receiver scheduling info onto Spark Streaming web UI

[ https://issues.apache.org/jira/browse/SPARK-12305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12305: -- Summary: Add Receiver scheduling info onto Spark Streaming web UI (was: Adds Receiver scheduling info

[jira] [Updated] (SPARK-12306) Add an option to ignore BlockRDD partition data loss

[ https://issues.apache.org/jira/browse/SPARK-12306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12306: -- Summary: Add an option to ignore BlockRDD partition data loss (was: ToEdit) > Add an option to ignore

[jira] [Created] (SPARK-12305) Adds Receiver scheduling info onto

Liwei Lin created SPARK-12305: - Summary: Adds Receiver scheduling info onto Key: SPARK-12305 URL: https://issues.apache.org/jira/browse/SPARK-12305 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-12305) Adds Receiver scheduling info onto Spark Streaming web UI

[ https://issues.apache.org/jira/browse/SPARK-12305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12305: -- Affects Version/s: 1.5.2 > Adds Receiver scheduling info onto Spark Streaming web UI >

[jira] [Updated] (SPARK-12305) Adds Receiver scheduling info onto

[ https://issues.apache.org/jira/browse/SPARK-12305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12305: -- Priority: Minor (was: Critical) > Adds Receiver scheduling info onto >

[jira] [Assigned] (SPARK-12302) Example for servlet filter used by spark.ui.filters

[ https://issues.apache.org/jira/browse/SPARK-12302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12302: Assignee: Apache Spark > Example for servlet filter used by spark.ui.filters >

[jira] [Assigned] (SPARK-12304) Make Spark Streaming web UI display more friendly Receiver graphs

[ https://issues.apache.org/jira/browse/SPARK-12304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12304: Assignee: Apache Spark > Make Spark Streaming web UI display more friendly Receiver

[jira] [Assigned] (SPARK-12302) Example for servlet filter used by spark.ui.filters

[ https://issues.apache.org/jira/browse/SPARK-12302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12302: Assignee: (was: Apache Spark) > Example for servlet filter used by spark.ui.filters >

[jira] [Assigned] (SPARK-12304) Make Spark Streaming web UI display more friendly Receiver graphs

[ https://issues.apache.org/jira/browse/SPARK-12304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12304: Assignee: (was: Apache Spark) > Make Spark Streaming web UI display more friendly

[jira] [Updated] (SPARK-12304) Make Spark Streaming web UI display more friendly Receiver graphs

[ https://issues.apache.org/jira/browse/SPARK-12304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12304: -- Summary: Make Spark Streaming web UI display more friendly Receiver graphs (was: Make Spark Streaming

[jira] [Updated] (SPARK-12304) Make Spark Streaming web UI display more friendly Receiver graphs

[ https://issues.apache.org/jira/browse/SPARK-12304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12304: -- Description: Currently, the Spark Streaming web UI uses the same maxY when displays 'Input Rate

[jira] [Updated] (SPARK-12304) Make Spark Streaming web UI display more friendly Receiver graphs

[ https://issues.apache.org/jira/browse/SPARK-12304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12304: -- Description: Currently, the Spark Streaming web UI uses the same maxY when displays 'Input Rate

[jira] [Resolved] (SPARK-2870) Thorough schema inference directly on RDDs of Python dictionaries

[ https://issues.apache.org/jira/browse/SPARK-2870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2870. -- Resolution: Not A Problem > Thorough schema inference directly on RDDs of Python dictionaries >

[jira] [Updated] (SPARK-12303) Configuration parameter by which can choose if we want the REPL generated class directory name to be random or fixed name.

[ https://issues.apache.org/jira/browse/SPARK-12303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12303: -- Priority: Minor (was: Major) Issue Type: New Feature (was: Wish) What would be the purpose of

[jira] [Created] (SPARK-12306) ToEdit

Liwei Lin created SPARK-12306: - Summary: ToEdit Key: SPARK-12306 URL: https://issues.apache.org/jira/browse/SPARK-12306 Project: Spark Issue Type: Improvement Reporter: Liwei Lin

[jira] [Commented] (SPARK-12218) Boolean logic in sql does not work "not (A and B)" is not the same as "(not A) or (not B)"

[ https://issues.apache.org/jira/browse/SPARK-12218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054142#comment-15054142 ] Xiao Li commented on SPARK-12218: - Found the fix, but that fix is not merged into 1.5.2. Tomorrow I will

[jira] [Resolved] (SPARK-11193) Spark 1.5+ Kinesis Streaming - ClassCastException when starting KinesisReceiver

[ https://issues.apache.org/jira/browse/SPARK-11193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11193. --- Resolution: Fixed Fix Version/s: 1.6.1 2.0.0 Issue resolved by pull

[jira] [Updated] (SPARK-12305) Add Receiver scheduling info onto Spark Streaming web UI

[ https://issues.apache.org/jira/browse/SPARK-12305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12305: -- Shepherd: Shixiong Zhu > Add Receiver scheduling info onto Spark Streaming web UI >

[jira] [Created] (SPARK-12304) Make Spark Streaming web UI display more friendly Receiver graph

Liwei Lin created SPARK-12304: - Summary: Make Spark Streaming web UI display more friendly Receiver graph Key: SPARK-12304 URL: https://issues.apache.org/jira/browse/SPARK-12304 Project: Spark

[jira] [Updated] (SPARK-12304) Make Spark Streaming web UI display more friendly Receiver graphs

[ https://issues.apache.org/jira/browse/SPARK-12304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12304: -- Attachment: after-5.png before-5.png > Make Spark Streaming web UI display more

[jira] [Updated] (SPARK-12306) Add an option to ignore BlockRDD partition data loss

[ https://issues.apache.org/jira/browse/SPARK-12306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12306: -- Shepherd: Shixiong Zhu > Add an option to ignore BlockRDD partition data loss >

[jira] [Updated] (SPARK-12306) Add an option to ignore BlockRDD partition data loss

[ https://issues.apache.org/jira/browse/SPARK-12306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12306: -- Component/s: Streaming > Add an option to ignore BlockRDD partition data loss >

[jira] [Comment Edited] (SPARK-12306) Add an option to ignore BlockRDD partition data loss

[ https://issues.apache.org/jira/browse/SPARK-12306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054184#comment-15054184 ] Liwei Lin edited comment on SPARK-12306 at 12/12/15 9:51 AM: - This issue is

[jira] [Commented] (SPARK-12306) Add an option to ignore BlockRDD partition data loss

[ https://issues.apache.org/jira/browse/SPARK-12306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054184#comment-15054184 ] Liwei Lin commented on SPARK-12306: --- This is reported by me, and I'm working on it. :-) > Add an

[jira] [Commented] (SPARK-12305) Add Receiver scheduling info onto Spark Streaming web UI

[ https://issues.apache.org/jira/browse/SPARK-12305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054185#comment-15054185 ] Liwei Lin commented on SPARK-12305: --- This issue is reported by me, and I'm working on it. :-) > Add

[jira] [Updated] (SPARK-12306) Add an option to ignore BlockRDD partition data loss

[ https://issues.apache.org/jira/browse/SPARK-12306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-12306: -- Affects Version/s: 1.5.2 > Add an option to ignore BlockRDD partition data loss >

[jira] [Commented] (SPARK-12305) Add Receiver scheduling info onto Spark Streaming web UI

[ https://issues.apache.org/jira/browse/SPARK-12305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054201#comment-15054201 ] Sean Owen commented on SPARK-12305: --- Same [~proflin] please don't open nearly blank JIRAs like this.

[jira] [Commented] (SPARK-12306) Add an option to ignore BlockRDD partition data loss

[ https://issues.apache.org/jira/browse/SPARK-12306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054199#comment-15054199 ] Sean Owen commented on SPARK-12306: --- [~proflin] there's no detail here. It doesn't sound like something

[jira] [Updated] (SPARK-4621) Shuffle index can cached for SortShuffleManager in ExternalShuffle in order to reduce indexFile's io

[ https://issues.apache.org/jira/browse/SPARK-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianhui Wang updated SPARK-4621: Summary: Shuffle index can cached for SortShuffleManager in ExternalShuffle in order to reduce

[jira] [Updated] (SPARK-4621) Shuffle index can be cached for SortShuffleManager in ExternalShuffle in order to reduce indexFile's io

[ https://issues.apache.org/jira/browse/SPARK-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianhui Wang updated SPARK-4621: Description: in ExternalShuffle, we can use LRUCache to store recently finished shuffle index and

[jira] [Comment Edited] (SPARK-12137) Spark Streaming State Recovery limitations

[ https://issues.apache.org/jira/browse/SPARK-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054339#comment-15054339 ] Ravindar edited comment on SPARK-12137 at 12/12/15 2:55 PM: Sean, thanks for

[jira] [Assigned] (SPARK-4621) Shuffle index can be cached for SortShuffleManager in ExternalShuffle in order to reduce indexFile's io

[ https://issues.apache.org/jira/browse/SPARK-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4621: --- Assignee: Apache Spark > Shuffle index can be cached for SortShuffleManager in

[jira] [Commented] (SPARK-4621) Shuffle index can be cached for SortShuffleManager in ExternalShuffle in order to reduce indexFile's io

[ https://issues.apache.org/jira/browse/SPARK-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054346#comment-15054346 ] Apache Spark commented on SPARK-4621: - User 'lianhuiwang' has created a pull request for this issue:

[jira] [Updated] (SPARK-4621) Shuffle index can be cached for SortShuffleManager in ExternalShuffle in order to reduce indexFile's io

[ https://issues.apache.org/jira/browse/SPARK-4621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lianhui Wang updated SPARK-4621: Summary: Shuffle index can be cached for SortShuffleManager in ExternalShuffle in order to reduce

[jira] [Commented] (SPARK-12137) Spark Streaming State Recovery limitations

[ https://issues.apache.org/jira/browse/SPARK-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054339#comment-15054339 ] Ravindar commented on SPARK-12137: -- Sean, thanks for the clarification on the current functionality. One

[jira] [Assigned] (SPARK-12218) Boolean logic in sql does not work "not (A and B)" is not the same as "(not A) or (not B)"

[ https://issues.apache.org/jira/browse/SPARK-12218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12218: Assignee: Apache Spark > Boolean logic in sql does not work "not (A and B)" is not the

[jira] [Updated] (SPARK-11785) When deployed against remote Hive metastore with lower versions, JDBC metadata calls throws exception

[ https://issues.apache.org/jira/browse/SPARK-11785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-11785: - Target Version/s: (was: 1.6.0) > When deployed against remote Hive metastore with lower versions, JDBC

[jira] [Updated] (SPARK-12199) Follow-up: Refine example code in ml-features.md

[ https://issues.apache.org/jira/browse/SPARK-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-12199: -- Shepherd: Joseph K. Bradley (was: Xiangrui Meng) > Follow-up: Refine example code in

[jira] [Commented] (SPARK-12303) Configuration parameter by which can choose if we want the REPL generated class directory name to be random or fixed name.

[ https://issues.apache.org/jira/browse/SPARK-12303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054515#comment-15054515 ] Neelesh Srinivas Salian commented on SPARK-12303: - Hi piyush, Thanks for opening the

[jira] [Assigned] (SPARK-12218) Boolean logic in sql does not work "not (A and B)" is not the same as "(not A) or (not B)"

[ https://issues.apache.org/jira/browse/SPARK-12218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12218: Assignee: (was: Apache Spark) > Boolean logic in sql does not work "not (A and B)"

[jira] [Updated] (SPARK-9686) Spark Thrift server doesn't return correct JDBC metadata

[ https://issues.apache.org/jira/browse/SPARK-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-9686: Target Version/s: (was: 1.6.0) > Spark Thrift server doesn't return correct JDBC metadata >

[jira] [Resolved] (SPARK-12199) Follow-up: Refine example code in ml-features.md

[ https://issues.apache.org/jira/browse/SPARK-12199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-12199. --- Resolution: Fixed Fix Version/s: 1.6.1 2.0.0 Issue

[jira] [Commented] (SPARK-7425) spark.ml Predictor should support other numeric types for label

[ https://issues.apache.org/jira/browse/SPARK-7425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054664#comment-15054664 ] Benjamin Fradet commented on SPARK-7425: Is there anyone working on this? Because I'm considering

[jira] [Created] (SPARK-12307) ParquetFormat options should be exposed through the DataFrameReader/Writer options API

holdenk created SPARK-12307: --- Summary: ParquetFormat options should be exposed through the DataFrameReader/Writer options API Key: SPARK-12307 URL: https://issues.apache.org/jira/browse/SPARK-12307