[jira] [Commented] (SPARK-6917) Broken data returned to PySpark dataframe if any large numbers used in Scala land

2015-05-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546160#comment-14546160 ] Davies Liu commented on SPARK-6917: --- [~yhuai] It's a bug in SQL or Parquet library: [[co

[jira] [Comment Edited] (SPARK-6917) Broken data returned to PySpark dataframe if any large numbers used in Scala land

2015-05-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546160#comment-14546160 ] Davies Liu edited comment on SPARK-6917 at 5/15/15 8:58 PM: [~

[jira] [Updated] (SPARK-6917) Broken data returned to PySpark dataframe if any large numbers used in Scala land

2015-05-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-6917: -- Assignee: Yin Huai (was: Davies Liu) > Broken data returned to PySpark dataframe if any large numbers u

[jira] [Resolved] (SPARK-7296) Timeline view for Stage page

2015-05-15 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-7296. --- Resolution: Fixed Fix Version/s: 1.4.0 Target Version/s: 1.4.0 (was: 1.4.0, 1

[jira] [Commented] (SPARK-7080) Binary processing based aggregate operator

2015-05-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546163#comment-14546163 ] Michael Armbrust commented on SPARK-7080: - That sounds like a good idea to me. >

[jira] [Resolved] (SPARK-6438) Indicate which tasks ran on which executors in per-stage visualization in UI

2015-05-15 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-6438. --- Resolution: Fixed > Indicate which tasks ran on which executors in per-stage visualization in

[jira] [Resolved] (SPARK-6418) Add simple per-stage visualization to the UI

2015-05-15 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-6418. --- Resolution: Fixed Assignee: (was: Pradyumn Shroff) > Add simple per-stage visualizat

[jira] [Resolved] (SPARK-6439) Show per-task metrics when you hover over a task in the web UI visualization

2015-05-15 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-6439. --- Resolution: Fixed Fix Version/s: 1.4.0 > Show per-task metrics when you hover over a ta

[jira] [Updated] (SPARK-6418) Add simple per-stage visualization to the UI

2015-05-15 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-6418: -- Fix Version/s: 1.4.0 > Add simple per-stage visualization to the UI > --

[jira] [Resolved] (SPARK-7226) Support math functions in R DataFrame

2015-05-15 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-7226. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 617

[jira] [Created] (SPARK-7676) Cleanup unnecessary code in the stage timeline view

2015-05-15 Thread Kay Ousterhout (JIRA)
Kay Ousterhout created SPARK-7676: - Summary: Cleanup unnecessary code in the stage timeline view Key: SPARK-7676 URL: https://issues.apache.org/jira/browse/SPARK-7676 Project: Spark Issue Typ

[jira] [Commented] (SPARK-6820) Convert NAs to null type in SparkR DataFrames

2015-05-15 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546174#comment-14546174 ] Shivaram Venkataraman commented on SPARK-6820: -- PR open at https://github.com

[jira] [Updated] (SPARK-6820) Convert NAs to null type in SparkR DataFrames

2015-05-15 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-6820: - Assignee: Qian Huang > Convert NAs to null type in SparkR DataFrames > ---

[jira] [Commented] (SPARK-6917) Broken data returned to PySpark dataframe if any large numbers used in Scala land

2015-05-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546201#comment-14546201 ] Yin Huai commented on SPARK-6917: - [~adrian-wang] Can you take a look? > Broken data retu

[jira] [Commented] (SPARK-7655) Akka timeout exception

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546204#comment-14546204 ] Apache Spark commented on SPARK-7655: - User 'zsxwing' has created a pull request for t

[jira] [Commented] (SPARK-7661) Support for dynamic allocation of executors in Kinesis Spark Streaming

2015-05-15 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546216#comment-14546216 ] Tathagata Das commented on SPARK-7661: -- What do you mean by the currently the logic i

[jira] [Updated] (SPARK-7676) Cleanup unnecessary code in the stage timeline view

2015-05-15 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-7676: -- Description: SPARK-7296 added a per-stage visualization to the UI. There's some unneeded code

[jira] [Updated] (SPARK-7676) Cleanup unnecessary code and fix small bug in the stage timeline view

2015-05-15 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-7676: -- Summary: Cleanup unnecessary code and fix small bug in the stage timeline view (was: Cleanup un

[jira] [Commented] (SPARK-7543) Break dataframe.py into multiple files

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546228#comment-14546228 ] Apache Spark commented on SPARK-7543: - User 'davies' has created a pull request for th

[jira] [Assigned] (SPARK-7543) Break dataframe.py into multiple files

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7543: --- Assignee: Davies Liu (was: Apache Spark) > Break dataframe.py into multiple files >

[jira] [Assigned] (SPARK-7543) Break dataframe.py into multiple files

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7543: --- Assignee: Apache Spark (was: Davies Liu) > Break dataframe.py into multiple files >

[jira] [Created] (SPARK-7677) Enable Kafka In Scala 2.11 Build

2015-05-15 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-7677: -- Summary: Enable Kafka In Scala 2.11 Build Key: SPARK-7677 URL: https://issues.apache.org/jira/browse/SPARK-7677 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-7651) PySpark GMM predict, predictSoft should fail on bad input

2015-05-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7651: - Fix Version/s: 1.3.2 > PySpark GMM predict, predictSoft should fail on bad input > ---

[jira] [Updated] (SPARK-7511) PySpark ML seed Param should be varied per class

2015-05-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7511: - Summary: PySpark ML seed Param should be varied per class (was: PySpark ML seed Param sho

[jira] [Updated] (SPARK-7677) Enable Kafka In Scala 2.11 Build

2015-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7677: --- Description: Now that we upgraded Kafka in SPARK-2808 we can enable it in the Scala 2.11 build

[jira] [Assigned] (SPARK-7676) Cleanup unnecessary code and fix small bug in the stage timeline view

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7676: --- Assignee: Apache Spark (was: Kay Ousterhout) > Cleanup unnecessary code and fix small bug in

[jira] [Updated] (SPARK-7511) PySpark ML seed Param should be varied per class

2015-05-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7511: - Description: Currently, Scala's HasSeed mix-in uses a random Long as the default value for

[jira] [Assigned] (SPARK-7676) Cleanup unnecessary code and fix small bug in the stage timeline view

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7676: --- Assignee: Kay Ousterhout (was: Apache Spark) > Cleanup unnecessary code and fix small bug in

[jira] [Commented] (SPARK-7676) Cleanup unnecessary code and fix small bug in the stage timeline view

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546239#comment-14546239 ] Apache Spark commented on SPARK-7676: - User 'kayousterhout' has created a pull request

[jira] [Updated] (SPARK-7511) PySpark ML seed Param should be varied per class

2015-05-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7511: - Description: Currently, Scala's HasSeed mix-in uses a random Long as the default value for

[jira] [Updated] (SPARK-7511) PySpark ML seed Param should be varied per class

2015-05-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7511: - Description: Currently, Scala's HasSeed mix-in uses a random Long as the default value for

[jira] [Resolved] (SPARK-7677) Enable Kafka In Scala 2.11 Build

2015-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell resolved SPARK-7677. Resolution: Fixed Fix Version/s: 1.4.0 Fixed by pull request: https://github.com/apac

[jira] [Created] (SPARK-7678) Scala ML seed Param should vary per class

2015-05-15 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-7678: Summary: Scala ML seed Param should vary per class Key: SPARK-7678 URL: https://issues.apache.org/jira/browse/SPARK-7678 Project: Spark Issue Type: I

[jira] [Updated] (SPARK-7678) Scala ML seed Param should be fixed but vary per class

2015-05-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7678: - Summary: Scala ML seed Param should be fixed but vary per class (was: Scala ML seed Param

[jira] [Resolved] (SPARK-7556) User guide update for feature transformer: Binarizer

2015-05-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-7556. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 6116 [https

[jira] [Commented] (SPARK-6902) Row() object can be mutated even though it should be immutable

2015-05-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546274#comment-14546274 ] Davies Liu commented on SPARK-6902: --- [~jarfa] Python is a dynamic language, it's not com

[jira] [Commented] (SPARK-6216) Check Python version in worker before run PySpark job

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546321#comment-14546321 ] Apache Spark commented on SPARK-6216: - User 'davies' has created a pull request for th

[jira] [Commented] (SPARK-7621) Report KafkaReceiver MessageHandler errors so StreamingListeners can take action

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546326#comment-14546326 ] Apache Spark commented on SPARK-7621: - User 'jerluc' has created a pull request for th

[jira] [Assigned] (SPARK-7621) Report KafkaReceiver MessageHandler errors so StreamingListeners can take action

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7621: --- Assignee: Apache Spark > Report KafkaReceiver MessageHandler errors so StreamingListeners can

[jira] [Assigned] (SPARK-7621) Report KafkaReceiver MessageHandler errors so StreamingListeners can take action

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7621: --- Assignee: (was: Apache Spark) > Report KafkaReceiver MessageHandler errors so StreamingLi

[jira] [Created] (SPARK-7679) Update AWS SDK and KCL versions to 1.2.1

2015-05-15 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-7679: Summary: Update AWS SDK and KCL versions to 1.2.1 Key: SPARK-7679 URL: https://issues.apache.org/jira/browse/SPARK-7679 Project: Spark Issue Type: Improvemen

[jira] [Assigned] (SPARK-7679) Update AWS SDK and KCL versions to 1.2.1

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7679: --- Assignee: Tathagata Das (was: Apache Spark) > Update AWS SDK and KCL versions to 1.2.1 > ---

[jira] [Commented] (SPARK-7679) Update AWS SDK and KCL versions to 1.2.1

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546341#comment-14546341 ] Apache Spark commented on SPARK-7679: - User 'tdas' has created a pull request for this

[jira] [Assigned] (SPARK-7679) Update AWS SDK and KCL versions to 1.2.1

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7679: --- Assignee: Apache Spark (was: Tathagata Das) > Update AWS SDK and KCL versions to 1.2.1 > ---

[jira] [Updated] (SPARK-7672) Number format exception with spark.kryoserializer.buffer.mb

2015-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7672: --- Priority: Critical (was: Major) > Number format exception with spark.kryoserializer.buffer.mb

[jira] [Updated] (SPARK-7672) Number format exception with spark.kryoserializer.buffer.mb

2015-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7672: --- Component/s: Spark Core > Number format exception with spark.kryoserializer.buffer.mb > --

[jira] [Updated] (SPARK-7284) Update streaming documentation for Spark 1.4.0 release

2015-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7284: --- Priority: Critical (was: Blocker) > Update streaming documentation for Spark 1.4.0 release >

[jira] [Updated] (SPARK-7644) Ensure all scoped RDD operations are tested and cleaned

2015-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7644: --- Priority: Critical (was: Blocker) > Ensure all scoped RDD operations are tested and cleaned >

[jira] [Updated] (SPARK-7355) FlakyTest - o.a.s.DriverSuite

2015-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7355: --- Priority: Critical (was: Blocker) > FlakyTest - o.a.s.DriverSuite > -

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546394#comment-14546394 ] Patrick Wendell commented on SPARK-2883: Since this is a feature I'm going to drop

[jira] [Updated] (SPARK-2883) Spark Support for ORCFile format

2015-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-2883: --- Priority: Critical (was: Blocker) > Spark Support for ORCFile format > --

[jira] [Updated] (SPARK-6811) Building binary R packages for SparkR

2015-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-6811: --- Assignee: Shivaram Venkataraman > Building binary R packages for SparkR >

[jira] [Commented] (SPARK-6902) Row() object can be mutated even though it should be immutable

2015-05-15 Thread Jonathan Arfa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546417#comment-14546417 ] Jonathan Arfa commented on SPARK-6902: -- [~davies] it works for me simply because I *n

[jira] [Commented] (SPARK-6289) PySpark doesn't maintain SQL date Types

2015-05-15 Thread Michael Nazario (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546425#comment-14546425 ] Michael Nazario commented on SPARK-6289: This does work for me, but it seems odd t

[jira] [Commented] (SPARK-6980) Akka timeout exceptions indicate which conf controls them

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546426#comment-14546426 ] Apache Spark commented on SPARK-6980: - User 'BryanCutler' has created a pull request f

[jira] [Updated] (SPARK-7676) Cleanup unnecessary code and fix small bug in the stage timeline view

2015-05-15 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-7676: -- Component/s: Web UI > Cleanup unnecessary code and fix small bug in the stage timeline view > --

[jira] [Resolved] (SPARK-7676) Cleanup unnecessary code and fix small bug in the stage timeline view

2015-05-15 Thread Kay Ousterhout (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-7676. --- Resolution: Fixed Fix Version/s: 1.4.0 > Cleanup unnecessary code and fix small bug in

[jira] [Commented] (SPARK-6820) Convert NAs to null type in SparkR DataFrames

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546451#comment-14546451 ] Apache Spark commented on SPARK-6820: - User 'hqzizania' has created a pull request for

[jira] [Assigned] (SPARK-6820) Convert NAs to null type in SparkR DataFrames

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6820: --- Assignee: Apache Spark (was: Qian Huang) > Convert NAs to null type in SparkR DataFrames > -

[jira] [Assigned] (SPARK-6820) Convert NAs to null type in SparkR DataFrames

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6820: --- Assignee: Qian Huang (was: Apache Spark) > Convert NAs to null type in SparkR DataFrames > -

[jira] [Commented] (SPARK-7073) Clean up Python data type hierarchy

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546450#comment-14546450 ] Apache Spark commented on SPARK-7073: - User 'davies' has created a pull request for th

[jira] [Assigned] (SPARK-7073) Clean up Python data type hierarchy

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7073: --- Assignee: Apache Spark (was: Davies Liu) > Clean up Python data type hierarchy > ---

[jira] [Assigned] (SPARK-7073) Clean up Python data type hierarchy

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7073: --- Assignee: Davies Liu (was: Apache Spark) > Clean up Python data type hierarchy > ---

[jira] [Commented] (SPARK-6411) PySpark DataFrames can't be created if any datetimes have timezones

2015-05-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546463#comment-14546463 ] Davies Liu commented on SPARK-6411: --- Since TimestampType in Spark SQL does not support t

[jira] [Comment Edited] (SPARK-6411) PySpark DataFrames can't be created if any datetimes have timezones

2015-05-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546463#comment-14546463 ] Davies Liu edited comment on SPARK-6411 at 5/16/15 1:02 AM: Si

[jira] [Updated] (SPARK-7563) OutputCommitCoordinator.stop() should only be executed in driver

2015-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7563: --- Target Version/s: 1.3.2, 1.4.0 > OutputCommitCoordinator.stop() should only be executed in dri

[jira] [Updated] (SPARK-7563) OutputCommitCoordinator.stop() should only be executed in driver

2015-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-7563: --- Fix Version/s: 1.4.0 > OutputCommitCoordinator.stop() should only be executed in driver >

[jira] [Commented] (SPARK-7563) OutputCommitCoordinator.stop() should only be executed in driver

2015-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546468#comment-14546468 ] Patrick Wendell commented on SPARK-7563: I pulled the fix into 1.4.0, but not yet

[jira] [Updated] (SPARK-7673) DataSourceStrategy''s buildPartitionedTableScan always list list file status for all data files

2015-05-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-7673: Target Version/s: 1.4.0 > DataSourceStrategy''s buildPartitionedTableScan always list list file status > fo

[jira] [Updated] (SPARK-7673) DataSourceStrategy''s buildPartitionedTableScan always list list file status for all data files

2015-05-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-7673: Fix Version/s: (was: 1.4.0) > DataSourceStrategy''s buildPartitionedTableScan always list list file stat

[jira] [Updated] (SPARK-7673) DataSourceStrategy''s buildPartitionedTableScan always list list file status for all data files

2015-05-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-7673: Assignee: Cheng Lian > DataSourceStrategy''s buildPartitionedTableScan always list list file status > for a

[jira] [Updated] (SPARK-7654) DataFrameReader and DataFrameWriter for input/output API

2015-05-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-7654: --- Priority: Blocker (was: Major) > DataFrameReader and DataFrameWriter for input/output API > -

[jira] [Resolved] (SPARK-7575) Example code for OneVsRest

2015-05-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-7575. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 6115 [https

[jira] [Assigned] (SPARK-6964) Support Cancellation in the Thrift Server

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6964: --- Assignee: (was: Apache Spark) > Support Cancellation in the Thrift Server > -

[jira] [Commented] (SPARK-6964) Support Cancellation in the Thrift Server

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546518#comment-14546518 ] Apache Spark commented on SPARK-6964: - User 'dongwang218' has created a pull request f

[jira] [Assigned] (SPARK-6964) Support Cancellation in the Thrift Server

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6964: --- Assignee: Apache Spark > Support Cancellation in the Thrift Server >

[jira] [Created] (SPARK-7680) Add a fake Receiver that generates random strings, useful for prototyping

2015-05-15 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-7680: Summary: Add a fake Receiver that generates random strings, useful for prototyping Key: SPARK-7680 URL: https://issues.apache.org/jira/browse/SPARK-7680 Project: Spar

[jira] [Resolved] (SPARK-7073) Clean up Python data type hierarchy

2015-05-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-7073. Resolution: Fixed Fix Version/s: 1.4.0 > Clean up Python data type hierarchy > --

[jira] [Resolved] (SPARK-7543) Break dataframe.py into multiple files

2015-05-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-7543. Resolution: Fixed Fix Version/s: 1.4.0 > Break dataframe.py into multiple files > ---

[jira] [Updated] (SPARK-7621) Report KafkaReceiver MessageHandler errors so StreamingListeners can take action

2015-05-15 Thread Jeremy A. Lucas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy A. Lucas updated SPARK-7621: --- Fix Version/s: (was: 1.3.1) > Report KafkaReceiver MessageHandler errors so StreamingListe

[jira] [Resolved] (SPARK-7473) Use reservoir sample in RandomForest when choosing features per node

2015-05-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-7473. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5988 [https

[jira] [Updated] (SPARK-7673) DataSourceStrategy's buildPartitionedTableScan always list list file status for all data files

2015-05-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-7673: -- Summary: DataSourceStrategy's buildPartitionedTableScan always list list file status for all data files

[jira] [Commented] (SPARK-7473) Use reservoir sample in RandomForest when choosing features per node

2015-05-15 Thread Ai He (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546546#comment-14546546 ] Ai He commented on SPARK-7473: -- Hi Joseph, it's AiHe. Thank you for reviewing and merging thi

[jira] [Commented] (SPARK-7473) Use reservoir sample in RandomForest when choosing features per node

2015-05-15 Thread Ai He (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546550#comment-14546550 ] Ai He commented on SPARK-7473: -- Hi Joseph, it's AiHe. Thank you for reviewing and merging thi

[jira] [Issue Comment Deleted] (SPARK-7473) Use reservoir sample in RandomForest when choosing features per node

2015-05-15 Thread Ai He (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ai He updated SPARK-7473: - Comment: was deleted (was: Hi Joseph, it's AiHe. Thank you for reviewing and merging this PR. ) > Use reservoir s

[jira] [Assigned] (SPARK-6649) DataFrame created through SQLContext.jdbc() failed if columns table must be quoted

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6649: --- Assignee: Apache Spark > DataFrame created through SQLContext.jdbc() failed if columns table

[jira] [Commented] (SPARK-6649) DataFrame created through SQLContext.jdbc() failed if columns table must be quoted

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546552#comment-14546552 ] Apache Spark commented on SPARK-6649: - User 'frreiss' has created a pull request for t

[jira] [Assigned] (SPARK-6649) DataFrame created through SQLContext.jdbc() failed if columns table must be quoted

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6649: --- Assignee: (was: Apache Spark) > DataFrame created through SQLContext.jdbc() failed if col

[jira] [Created] (SPARK-7681) Add SparseVector support for gemv with DenseMatrix

2015-05-15 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-7681: -- Summary: Add SparseVector support for gemv with DenseMatrix Key: SPARK-7681 URL: https://issues.apache.org/jira/browse/SPARK-7681 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-7681) Add SparseVector support for gemv with DenseMatrix

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7681: --- Assignee: Apache Spark > Add SparseVector support for gemv with DenseMatrix > ---

[jira] [Commented] (SPARK-7681) Add SparseVector support for gemv with DenseMatrix

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546561#comment-14546561 ] Apache Spark commented on SPARK-7681: - User 'viirya' has created a pull request for th

[jira] [Assigned] (SPARK-7681) Add SparseVector support for gemv with DenseMatrix

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7681: --- Assignee: (was: Apache Spark) > Add SparseVector support for gemv with DenseMatrix >

[jira] [Updated] (SPARK-7473) Use reservoir sample in RandomForest when choosing features per node

2015-05-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-7473: - Assignee: Ai He > Use reservoir sample in RandomForest when choosing features per node > -

[jira] [Updated] (SPARK-7661) Support for dynamic allocation of executors in Kinesis Spark Streaming

2015-05-15 Thread Murtaza Kanchwala (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Murtaza Kanchwala updated SPARK-7661: - Description: Currently the no. of cores is (N + 1), where N is no. of shards in a Kinesis

[jira] [Commented] (SPARK-7661) Support for dynamic allocation of executors in Kinesis Spark Streaming

2015-05-15 Thread Murtaza Kanchwala (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546598#comment-14546598 ] Murtaza Kanchwala commented on SPARK-7661: -- Ok I'll correct my terms, My case is

<    1   2   3