[jira] [Commented] (SPARK-6370) RDD sampling with replacement intermittently yields incorrect number of samples

[ https://issues.apache.org/jira/browse/SPARK-6370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368599#comment-14368599 ] Marko Bonaci commented on SPARK-6370: - Before sending PR, would something like this be

[jira] [Resolved] (SPARK-4012) Uncaught OOM in ContextCleaner

[ https://issues.apache.org/jira/browse/SPARK-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Davidson resolved SPARK-4012. --- Resolution: Fixed Fix Version/s: 1.4.0 Uncaught OOM in ContextCleaner

[jira] [Resolved] (SPARK-6222) [STREAMING] All data may not be recovered from WAL when driver is killed

[ https://issues.apache.org/jira/browse/SPARK-6222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-6222. -- Resolution: Fixed [STREAMING] All data may not be recovered from WAL when driver is killed

[jira] [Commented] (SPARK-5682) Add encrypted shuffle in spark

[ https://issues.apache.org/jira/browse/SPARK-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368621#comment-14368621 ] liyunzhang_intel commented on SPARK-5682: - sorry to reply so late. If run spark on

[jira] [Comment Edited] (SPARK-6354) Replace the plan which is part of cached query

[ https://issues.apache.org/jira/browse/SPARK-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368649#comment-14368649 ] Liang-Chi Hsieh edited comment on SPARK-6354 at 3/19/15 7:31 AM:

[jira] [Commented] (SPARK-5387) parquet writer runs into OOM during writing when number of rows is large

[ https://issues.apache.org/jira/browse/SPARK-5387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368578#comment-14368578 ] Chaozhong Yang commented on SPARK-5387: --- We can locate the bug in parquet-hadoop

[jira] [Comment Edited] (SPARK-6354) Replace the plan which is part of cached query

[ https://issues.apache.org/jira/browse/SPARK-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368649#comment-14368649 ] Liang-Chi Hsieh edited comment on SPARK-6354 at 3/19/15 7:28 AM:

[jira] [Commented] (SPARK-5387) parquet writer runs into OOM during writing when number of rows is large

[ https://issues.apache.org/jira/browse/SPARK-5387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368652#comment-14368652 ] Chaozhong Yang commented on SPARK-5387: ---

[jira] [Commented] (SPARK-6354) Replace the plan which is part of cached query

[ https://issues.apache.org/jira/browse/SPARK-6354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368649#comment-14368649 ] Liang-Chi Hsieh commented on SPARK-6354: h2. Introduction Currently we use the

[jira] [Created] (SPARK-6419) GenerateOrdering does not support BinaryType and complex types.

Yin Huai created SPARK-6419: --- Summary: GenerateOrdering does not support BinaryType and complex types. Key: SPARK-6419 URL: https://issues.apache.org/jira/browse/SPARK-6419 Project: Spark Issue

[jira] [Commented] (SPARK-6367) Use the proper data type for those expressions that are hijacking existing data types.

[ https://issues.apache.org/jira/browse/SPARK-6367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370220#comment-14370220 ] Apache Spark commented on SPARK-6367: - User 'yhuai' has created a pull request for

[jira] [Commented] (SPARK-6373) Add SSL/TLS for the Netty based BlockTransferService

[ https://issues.apache.org/jira/browse/SPARK-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370272#comment-14370272 ] Kenji Matsuoka commented on SPARK-6373: --- I also have a use case which requires data

[jira] [Updated] (SPARK-6398) Improve utility of GaussianMixture for higher dimensional data

[ https://issues.apache.org/jira/browse/SPARK-6398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6398: - Summary: Improve utility of GaussianMixture for higher dimensional data (was: Improve

[jira] [Updated] (SPARK-6398) Improve utility of GaussianMixture for higher dimensional data

[ https://issues.apache.org/jira/browse/SPARK-6398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6398: - Assignee: Travis Galoppo Improve utility of GaussianMixture for higher dimensional data

[jira] [Created] (SPARK-6421) _regression_train_wrapper does not test initialWeights correctly

Joseph K. Bradley created SPARK-6421: Summary: _regression_train_wrapper does not test initialWeights correctly Key: SPARK-6421 URL: https://issues.apache.org/jira/browse/SPARK-6421 Project:

[jira] [Updated] (SPARK-5508) Arrays and Maps stored with Hive Parquet Serde may not be able to read by the Parquet support in the Data Souce API

[ https://issues.apache.org/jira/browse/SPARK-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-5508: Description: *The root cause of this bug is explained below ([see here|])* When the table is saved as

[jira] [Updated] (SPARK-5508) Arrays and Maps stored with Hive Parquet Serde may not be able to read by the Parquet support in the Data Souce API

[ https://issues.apache.org/jira/browse/SPARK-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-5508: Description: *The root cause of this bug is explained below ([see

[jira] [Created] (SPARK-6420) Driver's Block Manager does not use spark.driver.host in Yarn-Client mode

Liangliang Gu created SPARK-6420: Summary: Driver's Block Manager does not use spark.driver.host in Yarn-Client mode Key: SPARK-6420 URL: https://issues.apache.org/jira/browse/SPARK-6420 Project:

[jira] [Updated] (SPARK-3789) [GRAPHX] Python bindings for GraphX

[ https://issues.apache.org/jira/browse/SPARK-3789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kushal Datta updated SPARK-3789: Summary: [GRAPHX] Python bindings for GraphX (was: Python bindings for GraphX) [GRAPHX] Python

[jira] [Commented] (SPARK-6398) Improve utility of GaussianMixture for higher dimensional data

[ https://issues.apache.org/jira/browse/SPARK-6398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370305#comment-14370305 ] Joseph K. Bradley commented on SPARK-6398: -- I know we've discussed this, but it

[jira] [Commented] (SPARK-6420) Driver's Block Manager does not use spark.driver.host in Yarn-Client mode

[ https://issues.apache.org/jira/browse/SPARK-6420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370326#comment-14370326 ] Apache Spark commented on SPARK-6420: - User 'marsishandsome' has created a pull

[jira] [Updated] (SPARK-1529) Support setting spark.local.dirs to a hadoop FileSystem

[ https://issues.apache.org/jira/browse/SPARK-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kannan Rajah updated SPARK-1529: Attachment: (was: SparkShuffleUsingHDFS_API.pdf) Support setting spark.local.dirs to a hadoop

[jira] [Updated] (SPARK-1529) Support setting spark.local.dirs to a hadoop FileSystem

[ https://issues.apache.org/jira/browse/SPARK-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kannan Rajah updated SPARK-1529: Attachment: Spark Shuffle using HDFS.pdf Support setting spark.local.dirs to a hadoop FileSystem

[jira] [Commented] (SPARK-6419) GenerateOrdering does not support BinaryType and complex types.

[ https://issues.apache.org/jira/browse/SPARK-6419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370257#comment-14370257 ] Yin Huai commented on SPARK-6419: - For now, the workaround is to disable code gen for

[jira] [Updated] (SPARK-6419) GenerateOrdering does not support BinaryType and complex types.

[ https://issues.apache.org/jira/browse/SPARK-6419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6419: Description: When user want to order by binary columns or columns with complex types and code gen is

[jira] [Resolved] (SPARK-3823) Spark Hive SQL readColumn is not reset each time for a new query

[ https://issues.apache.org/jira/browse/SPARK-3823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-3823. - Resolution: Duplicate I believe that it has been resolved by SPARK-3559. Spark Hive SQL readColumn is

[jira] [Updated] (SPARK-5508) Arrays and Maps stored with Hive Parquet Serde may not be able to read by the Parquet support in the Data Souce API

[ https://issues.apache.org/jira/browse/SPARK-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-5508: Description: *The root cause of this bug is explained below ([see

[jira] [Commented] (SPARK-5654) Integrate SparkR into Apache Spark

[ https://issues.apache.org/jira/browse/SPARK-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370353#comment-14370353 ] Apache Spark commented on SPARK-5654: - User 'shivaram' has created a pull request for

[jira] [Commented] (SPARK-6370) RDD sampling with replacement intermittently yields incorrect number of samples

[ https://issues.apache.org/jira/browse/SPARK-6370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370463#comment-14370463 ] Apache Spark commented on SPARK-6370: - User 'mbonaci' has created a pull request for

[jira] [Commented] (SPARK-6370) RDD sampling with replacement intermittently yields incorrect number of samples

[ https://issues.apache.org/jira/browse/SPARK-6370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370462#comment-14370462 ] Marko Bonaci commented on SPARK-6370: - Here's PR:

[jira] [Commented] (SPARK-6250) Types are now reserved words in DDL parser.

[ https://issues.apache.org/jira/browse/SPARK-6250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370240#comment-14370240 ] Nitay Joffe commented on SPARK-6250: Sorry for delay I'll try it out asap, hopefully

[jira] [Commented] (SPARK-6320) Adding new query plan strategy to SQLContext

[ https://issues.apache.org/jira/browse/SPARK-6320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368683#comment-14368683 ] Santiago M. Mola commented on SPARK-6320: - [~marmbrus] We could change strategies

[jira] [Comment Edited] (SPARK-6320) Adding new query plan strategy to SQLContext

[ https://issues.apache.org/jira/browse/SPARK-6320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368683#comment-14368683 ] Santiago M. Mola edited comment on SPARK-6320 at 3/19/15 8:12 AM:

[jira] [Updated] (SPARK-6410) Build error on Windows: polymorphic expression cannot be instantiated to expected type

[ https://issues.apache.org/jira/browse/SPARK-6410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santiago M. Mola updated SPARK-6410: Description: $ bash build/sbt -Phadoop-2.3 assembly [...] [error]

[jira] [Updated] (SPARK-6410) Build error on Windows: polymorphic expression cannot be instantiated to expected type

[ https://issues.apache.org/jira/browse/SPARK-6410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santiago M. Mola updated SPARK-6410: Component/s: SQL Build error on Windows: polymorphic expression cannot be instantiated to

[jira] [Created] (SPARK-6408) JDBCRDD fails on where clause with string literal

Pei-Lun Lee created SPARK-6408: -- Summary: JDBCRDD fails on where clause with string literal Key: SPARK-6408 URL: https://issues.apache.org/jira/browse/SPARK-6408 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6286) Handle TASK_ERROR in TaskState

[ https://issues.apache.org/jira/browse/SPARK-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368744#comment-14368744 ] Apache Spark commented on SPARK-6286: - User 'jongyoul' has created a pull request for

[jira] [Commented] (SPARK-6363) make scala 2.11 default language

[ https://issues.apache.org/jira/browse/SPARK-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368846#comment-14368846 ] antonkulaga commented on SPARK-6363: is already cross-built for 2.10 and 2.11, and

[jira] [Commented] (SPARK-6386) add association rule mining algorithm to MLLib

[ https://issues.apache.org/jira/browse/SPARK-6386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368719#comment-14368719 ] zhangyouhua commented on SPARK-6386: I think association rule algorithms is a new

[jira] [Created] (SPARK-6410) Build error on Windows: polymorphic expression cannot be instantiated to expected type

Santiago M. Mola created SPARK-6410: --- Summary: Build error on Windows: polymorphic expression cannot be instantiated to expected type Key: SPARK-6410 URL: https://issues.apache.org/jira/browse/SPARK-6410

[jira] [Updated] (SPARK-5682) Add encrypted shuffle in spark

[ https://issues.apache.org/jira/browse/SPARK-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyunzhang_intel updated SPARK-5682: Attachment: Design Document of Encrypted Spark Shuffle_20150318.docx [~srowen], i have

[jira] [Updated] (SPARK-6409) It is not necessary that avoid old inteface of hive that will make some UDAF can work.

[ https://issues.apache.org/jira/browse/SPARK-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DoingDone9 updated SPARK-6409: -- Description: I run SQL like that CREATE TEMPORARY FUNCTION test_avg AS

[jira] [Updated] (SPARK-6409) It is not necessary that avoid old inteface of hive that will make some UDAF can work.

[ https://issues.apache.org/jira/browse/SPARK-6409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DoingDone9 updated SPARK-6409: -- Summary: It is not necessary that avoid old inteface of hive that will make some UDAF can work. (was:

[jira] [Updated] (SPARK-6410) Build error on Windows: polymorphic expression cannot be instantiated to expected type

[ https://issues.apache.org/jira/browse/SPARK-6410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santiago M. Mola updated SPARK-6410: Description: $ bash build/sbt -Phadoop-2.3 assembly [...] [error]

[jira] [Updated] (SPARK-6410) Build error on Windows: polymorphic expression cannot be instantiated to expected type

[ https://issues.apache.org/jira/browse/SPARK-6410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santiago M. Mola updated SPARK-6410: Attachment: output.log Full error log. Build error on Windows: polymorphic expression

[jira] [Created] (SPARK-6409) Is it necessary that avoid old inteface of hive that will make some UDAF can work.

DoingDone9 created SPARK-6409: - Summary: Is it necessary that avoid old inteface of hive that will make some UDAF can work. Key: SPARK-6409 URL: https://issues.apache.org/jira/browse/SPARK-6409 Project:

[jira] [Resolved] (SPARK-6410) Build error on Windows: polymorphic expression cannot be instantiated to expected type

[ https://issues.apache.org/jira/browse/SPARK-6410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santiago M. Mola resolved SPARK-6410. - Resolution: Not a Problem This has something to do with the incremental compiler. I got

[jira] [Comment Edited] (SPARK-5387) parquet writer runs into OOM during writing when number of rows is large

[ https://issues.apache.org/jira/browse/SPARK-5387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368578#comment-14368578 ] Chaozhong Yang edited comment on SPARK-5387 at 3/19/15 10:58 AM:

[jira] [Commented] (SPARK-6370) RDD sampling with replacement intermittently yields incorrect number of samples

[ https://issues.apache.org/jira/browse/SPARK-6370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14368877#comment-14368877 ] Sean Owen commented on SPARK-6370: -- Looking at the code, I think that's accurate. The

[jira] [Updated] (SPARK-5821) JSONRelation should check if delete is successful for the overwrite operation.

[ https://issues.apache.org/jira/browse/SPARK-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-5821: -- Description: When you run CTAS command such as {code sql} CREATE TEMPORARY TABLE jsonTable USING

[jira] [Updated] (SPARK-6402) EC2 script and job scheduling documentation still refer to Shark

[ https://issues.apache.org/jira/browse/SPARK-6402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6402: - Component/s: EC2 Documentation Priority: Trivial (was: Major) Assignee:

[jira] [Resolved] (SPARK-6402) EC2 script and job scheduling documentation still refer to Shark

[ https://issues.apache.org/jira/browse/SPARK-6402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6402. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5083

[jira] [Commented] (SPARK-5387) parquet writer runs into OOM during writing when number of rows is large

[ https://issues.apache.org/jira/browse/SPARK-5387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369259#comment-14369259 ] Apache Spark commented on SPARK-5387: - User 'debugger87' has created a pull request

[jira] [Updated] (SPARK-5821) JSONRelation should check if delete is successful for the overwrite operation.

[ https://issues.apache.org/jira/browse/SPARK-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-5821: -- Description: When you run CTAS command such as {code:sql} CREATE TEMPORARY TABLE jsonTable USING

[jira] [Updated] (SPARK-5821) JSONRelation should check if delete is successful for the overwrite operation.

[ https://issues.apache.org/jira/browse/SPARK-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-5821: -- Description: When you run CTAS command such as {code sql} CREATE TEMPORARY TABLE jsonTable USING

[jira] [Resolved] (SPARK-5843) Expose all parameters in JavaPairRDD.combineByKey()

[ https://issues.apache.org/jira/browse/SPARK-5843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5843. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4634

[jira] [Updated] (SPARK-5843) Expose all parameters in JavaPairRDD.combineByKey()

[ https://issues.apache.org/jira/browse/SPARK-5843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5843: - Assignee: Matthew Cheah Expose all parameters in JavaPairRDD.combineByKey()

[jira] [Created] (SPARK-6411) PySpark DataFrames can't be created if any datetimes have timezones

Harry Brundage created SPARK-6411: - Summary: PySpark DataFrames can't be created if any datetimes have timezones Key: SPARK-6411 URL: https://issues.apache.org/jira/browse/SPARK-6411 Project: Spark

[jira] [Created] (SPARK-6423) MemoryUtils should use memoryOverhead if it's set.

Jongyoul Lee created SPARK-6423: --- Summary: MemoryUtils should use memoryOverhead if it's set. Key: SPARK-6423 URL: https://issues.apache.org/jira/browse/SPARK-6423 Project: Spark Issue Type:

[jira] [Commented] (SPARK-4105) FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370598#comment-14370598 ] Harry Brundage commented on SPARK-4105: --- Would just like to add that we are seeing

[jira] [Created] (SPARK-6424) Support user-defined aggregators in AggregateFunction

Takeshi Yamamuro created SPARK-6424: --- Summary: Support user-defined aggregators in AggregateFunction Key: SPARK-6424 URL: https://issues.apache.org/jira/browse/SPARK-6424 Project: Spark

[jira] [Updated] (SPARK-6308) VectorUDT is displayed as `vecto` in dtypes

[ https://issues.apache.org/jira/browse/SPARK-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6308: - Assignee: (was: Xiangrui Meng) VectorUDT is displayed as `vecto` in dtypes

[jira] [Commented] (SPARK-6424) Support user-defined aggregators in AggregateFunction

[ https://issues.apache.org/jira/browse/SPARK-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370654#comment-14370654 ] Apache Spark commented on SPARK-6424: - User 'maropu' has created a pull request for

[jira] [Commented] (SPARK-6422) support customized akka system for actor-based receiver

[ https://issues.apache.org/jira/browse/SPARK-6422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370515#comment-14370515 ] Apache Spark commented on SPARK-6422: - User 'CodingCat' has created a pull request for

[jira] [Updated] (SPARK-6423) MemoryUtils should use memoryOverhead if it's set

[ https://issues.apache.org/jira/browse/SPARK-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jongyoul Lee updated SPARK-6423: Summary: MemoryUtils should use memoryOverhead if it's set (was: MemoryUtils should use

[jira] [Commented] (SPARK-5682) Add encrypted shuffle in spark

[ https://issues.apache.org/jira/browse/SPARK-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370631#comment-14370631 ] liyunzhang_intel commented on SPARK-5682: - Hi all: There are two methods to not

[jira] [Created] (SPARK-6422) support customized akka system for actor-based receiver

Nan Zhu created SPARK-6422: -- Summary: support customized akka system for actor-based receiver Key: SPARK-6422 URL: https://issues.apache.org/jira/browse/SPARK-6422 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-4105) FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370598#comment-14370598 ] Harry Brundage edited comment on SPARK-4105 at 3/20/15 2:46 AM:

[jira] [Created] (SPARK-6425) Add parallel Q-learning algorithm to MLLib

zhangyouhua created SPARK-6425: -- Summary: Add parallel Q-learning algorithm to MLLib Key: SPARK-6425 URL: https://issues.apache.org/jira/browse/SPARK-6425 Project: Spark Issue Type: New Feature

[jira] [Closed] (SPARK-6395) Rebuild the schema from a GenericRow

[ https://issues.apache.org/jira/browse/SPARK-6395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Song closed SPARK-6395. Resolution: Not a Problem Rebuild the schema from a GenericRow

[jira] [Commented] (SPARK-4105) FAILED_TO_UNCOMPRESS(5) errors when fetching shuffle data with sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370613#comment-14370613 ] Mark Khaitman commented on SPARK-4105: -- Can also confirm it's happened once so far

[jira] [Updated] (SPARK-6423) MemoryUtils should use memoryOverhead if it's set

[ https://issues.apache.org/jira/browse/SPARK-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jongyoul Lee updated SPARK-6423: Fix Version/s: 1.3.1 1.4.0 MemoryUtils should use memoryOverhead if it's set

[jira] [Commented] (SPARK-6423) MemoryUtils should use memoryOverhead if it's set

[ https://issues.apache.org/jira/browse/SPARK-6423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370601#comment-14370601 ] Apache Spark commented on SPARK-6423: - User 'jongyoul' has created a pull request for

[jira] [Commented] (SPARK-5981) pyspark ML models should support predict/transform on vector within map

[ https://issues.apache.org/jira/browse/SPARK-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370747#comment-14370747 ] Joseph K. Bradley commented on SPARK-5981: -- I agree that it should not work for

[jira] [Updated] (SPARK-6112) Provide OffHeap support through HDFS RAM_DISK

[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6112: -- Summary: Provide OffHeap support through HDFS RAM_DISK (was: Leverage HDFS RAM_DISK capacity to

[jira] [Created] (SPARK-6416) RDD.fold() requires the operator to be commutative

Josh Rosen created SPARK-6416: - Summary: RDD.fold() requires the operator to be commutative Key: SPARK-6416 URL: https://issues.apache.org/jira/browse/SPARK-6416 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-6415) Spark Streaming fail-fast: Stop scheduling jobs when a batch fails, and kills the app

[ https://issues.apache.org/jira/browse/SPARK-6415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6415: --- Component/s: Streaming Spark Streaming fail-fast: Stop scheduling jobs when a batch fails,

[jira] [Created] (SPARK-6418) Add simple per-stage visualization to the UI

Kay Ousterhout created SPARK-6418: - Summary: Add simple per-stage visualization to the UI Key: SPARK-6418 URL: https://issues.apache.org/jira/browse/SPARK-6418 Project: Spark Issue Type:

[jira] [Updated] (SPARK-6219) Expand Python lint checks to check for compilation errors

[ https://issues.apache.org/jira/browse/SPARK-6219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-6219: -- Assignee: Nicholas Chammas Expand Python lint checks to check for compilation errors

[jira] [Resolved] (SPARK-6219) Expand Python lint checks to check for compilation errors

[ https://issues.apache.org/jira/browse/SPARK-6219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-6219. --- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 4941

[jira] [Updated] (SPARK-6418) Add simple per-stage visualization to the UI

[ https://issues.apache.org/jira/browse/SPARK-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-6418: -- Attachment: Screen Shot 2015-03-18 at 6.13.04 PM.png Add simple per-stage visualization to the

[jira] [Commented] (SPARK-1200) Make it possible to use unmanaged AM in yarn-client mode

[ https://issues.apache.org/jira/browse/SPARK-1200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369339#comment-14369339 ] Steve Loughran commented on SPARK-1200: --- You know, we could benefit all YARN apps if

[jira] [Updated] (SPARK-6408) JDBCRDD fails on where clause with string literal

[ https://issues.apache.org/jira/browse/SPARK-6408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6408: -- Priority: Blocker (was: Critical) Target Version/s: 1.3.1 JDBCRDD fails on where clause

[jira] [Commented] (SPARK-5821) JSONRelation should check if delete is successful for the overwrite operation.

[ https://issues.apache.org/jira/browse/SPARK-5821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369305#comment-14369305 ] Cheng Lian commented on SPARK-5821: --- Left comments on GitHub. JSONRelation should

[jira] [Created] (SPARK-6415) Spark Streaming fail-fast: Stop scheduling jobs when a batch fails, and kills the app

Hari Shreedharan created SPARK-6415: --- Summary: Spark Streaming fail-fast: Stop scheduling jobs when a batch fails, and kills the app Key: SPARK-6415 URL: https://issues.apache.org/jira/browse/SPARK-6415

[jira] [Created] (SPARK-6414) Spark driver failed with NPE on job cancelation

Yuri Makhno created SPARK-6414: -- Summary: Spark driver failed with NPE on job cancelation Key: SPARK-6414 URL: https://issues.apache.org/jira/browse/SPARK-6414 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2344) Add Fuzzy C-Means algorithm to MLlib

[ https://issues.apache.org/jira/browse/SPARK-2344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369787#comment-14369787 ] Beniamino commented on SPARK-2344: -- Hi Alex, Sorry for the late response but I'm very

[jira] [Updated] (SPARK-6291) GLM toString should not output full weight vector

[ https://issues.apache.org/jira/browse/SPARK-6291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6291: - Assignee: Yanbo Liang GLM toString should not output full weight vector

[jira] [Resolved] (SPARK-6291) GLM toString should not output full weight vector

[ https://issues.apache.org/jira/browse/SPARK-6291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6291. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5038

[jira] [Commented] (SPARK-5981) pyspark ML models should support predict/transform on vector within map

[ https://issues.apache.org/jira/browse/SPARK-5981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369383#comment-14369383 ] Manoj Kumar commented on SPARK-5981: [~josephkb] Do you agree with my observation that

[jira] [Created] (SPARK-6417) Add Linear Programming algorithm

Fan Jiang created SPARK-6417: Summary: Add Linear Programming algorithm Key: SPARK-6417 URL: https://issues.apache.org/jira/browse/SPARK-6417 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-6418) Add simple per-stage visualization to the UI

[ https://issues.apache.org/jira/browse/SPARK-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370042#comment-14370042 ] Sean Owen commented on SPARK-6418: -- I like how it looks, sounds like a fairly simple

[jira] [Commented] (SPARK-4123) Show new dependencies added in pull requests

[ https://issues.apache.org/jira/browse/SPARK-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370138#comment-14370138 ] Apache Spark commented on SPARK-4123: - User 'brennonyork' has created a pull request

[jira] [Created] (SPARK-6413) For data source tables, we should provide better output for described extended/formatted.

Yin Huai created SPARK-6413: --- Summary: For data source tables, we should provide better output for described extended/formatted. Key: SPARK-6413 URL: https://issues.apache.org/jira/browse/SPARK-6413

[jira] [Commented] (SPARK-5988) Model import/export for PowerIterationClusteringModel

[ https://issues.apache.org/jira/browse/SPARK-5988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14369601#comment-14369601 ] Xusen Yin commented on SPARK-5988: -- [~mengxr] Can I take this for the next step? Thanks!

[jira] [Updated] (SPARK-6373) Add SSL/TLS for the Netty based BlockTransferService

[ https://issues.apache.org/jira/browse/SPARK-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Turpin updated SPARK-6373: -- Priority: Major (was: Minor) Add SSL/TLS for the Netty based BlockTransferService

[jira] [Updated] (SPARK-6401) Unable to load a old API input format in Spark streaming

[ https://issues.apache.org/jira/browse/SPARK-6401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6401: - Component/s: Streaming Unable to load a old API input format in Spark streaming

[jira] [Resolved] (SPARK-5313) Create simple framework for highlighting changes introduced in a PR

[ https://issues.apache.org/jira/browse/SPARK-5313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5313. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5072

[jira] [Updated] (SPARK-5313) Create simple framework for highlighting changes introduced in a PR

[ https://issues.apache.org/jira/browse/SPARK-5313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-5313: - Assignee: Brennon York Create simple framework for highlighting changes introduced in a PR

[jira] [Updated] (SPARK-6406) Launcher backward compatibility issues

[ https://issues.apache.org/jira/browse/SPARK-6406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6406: - Component/s: Deploy Launcher backward compatibility issues --

  1   2   >