[jira] [Assigned] (SPARK-13593) add a `schema()` method to convert python RDD to DataFrame easily

2016-03-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13593: Assignee: (was: Apache Spark) > add a `schema()` method to convert python RDD to DataF

[jira] [Created] (SPARK-13593) add a `schema()` method to convert python RDD to DataFrame easily

2016-03-01 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-13593: --- Summary: add a `schema()` method to convert python RDD to DataFrame easily Key: SPARK-13593 URL: https://issues.apache.org/jira/browse/SPARK-13593 Project: Spark

[jira] [Commented] (SPARK-10795) FileNotFoundException while deploying pyspark job on cluster

2016-03-01 Thread Carlos Bribiescas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173723#comment-15173723 ] Carlos Bribiescas commented on SPARK-10795: --- Have you tried just specifying the

[jira] [Commented] (SPARK-13510) Shuffle may throw FetchFailedException: Direct buffer memory

2016-03-01 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173675#comment-15173675 ] Hong Shen commented on SPARK-13510: --- I have resolve in our own edition, I will add a pu

[jira] [Updated] (SPARK-13510) Shuffle may throw FetchFailedException: Direct buffer memory

2016-03-01 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated SPARK-13510: -- Description: In our cluster, when I test spark-1.6.0 with a sql, it throw exception and failed. {code}

[jira] [Updated] (SPARK-13510) Shuffle may throw FetchFailedException: Direct buffer memory

2016-03-01 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated SPARK-13510: -- Description: In our cluster, when I test spark-1.6.0 with a sql, it throw exception and failed. {code}

[jira] [Commented] (SPARK-13510) Shuffle may throw FetchFailedException: Direct buffer memory

2016-03-01 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173658#comment-15173658 ] Hong Shen commented on SPARK-13510: --- We can't resolve all the shuffle OOM by allocate m

[jira] [Resolved] (SPARK-13510) Shuffle may throw FetchFailedException: Direct buffer memory

2016-03-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13510. --- Resolution: Not A Problem > Shuffle may throw FetchFailedException: Direct buffer memory > --

[jira] [Closed] (SPARK-13510) Shuffle may throw FetchFailedException: Direct buffer memory

2016-03-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-13510. - > Shuffle may throw FetchFailedException: Direct buffer memory >

[jira] [Commented] (SPARK-10795) FileNotFoundException while deploying pyspark job on cluster

2016-03-01 Thread Daniel Jouany (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173650#comment-15173650 ] Daniel Jouany commented on SPARK-10795: --- I am using spark 1.4.1 on HDP 2.3.2. My c

[jira] [Comment Edited] (SPARK-10795) FileNotFoundException while deploying pyspark job on cluster

2016-03-01 Thread Daniel Jouany (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173489#comment-15173489 ] Daniel Jouany edited comment on SPARK-10795 at 3/1/16 11:51 AM: ---

[jira] [Updated] (SPARK-13584) ContinuousQueryManagerSuite floods the logs with garbage

2016-03-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13584: -- Component/s: Tests > ContinuousQueryManagerSuite floods the logs with garbage > ---

[jira] [Updated] (SPARK-13583) Support `UnusedImports` Java checkstyle rule

2016-03-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13583: -- Component/s: Streaming Spark Core > Support `UnusedImports` Java checkstyle rule > ---

[jira] [Commented] (SPARK-13510) Shuffle may throw FetchFailedException: Direct buffer memory

2016-03-01 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173648#comment-15173648 ] Hong Shen commented on SPARK-13510: --- In our cluster, we have lot of sql run on hive, I

[jira] [Commented] (SPARK-13510) Shuffle may throw FetchFailedException: Direct buffer memory

2016-03-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173647#comment-15173647 ] Sean Owen commented on SPARK-13510: --- [~shenhong] don't reopen JIRAs unless the discussi

[jira] [Updated] (SPARK-13574) Improve parquet dictionary decoding for strings

2016-03-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-13574: -- Component/s: SQL > Improve parquet dictionary decoding for strings > --

[jira] [Reopened] (SPARK-13510) Shuffle may throw FetchFailedException: Direct buffer memory

2016-03-01 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen reopened SPARK-13510: --- > Shuffle may throw FetchFailedException: Direct buffer memory >

[jira] [Resolved] (SPARK-13510) Shuffle may throw FetchFailedException: Direct buffer memory

2016-03-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13510. --- Resolution: Not A Problem As far as I can tell, your problem reduces to "out of memory", which is not

[jira] [Commented] (SPARK-13244) Unify DataFrame and Dataset API

2016-03-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173616#comment-15173616 ] Apache Spark commented on SPARK-13244: -- User 'liancheng' has created a pull request

[jira] [Resolved] (SPARK-13546) GBT with many trees consistently giving java.lang.StackOverflowError

2016-03-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13546. --- Resolution: Not A Problem Yes, it's likely to be required to avoid a stack overflow. Exactly how many

[jira] [Commented] (SPARK-13587) Support virtualenv in PySpark

2016-03-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173590#comment-15173590 ] Sean Owen commented on SPARK-13587: --- CC [~j_houg] for interest, comment > Support virt

[jira] [Commented] (SPARK-13117) WebUI should use the local ip not 0.0.0.0

2016-03-01 Thread Jeremiah Jordan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173511#comment-15173511 ] Jeremiah Jordan commented on SPARK-13117: - If you want minimal side effects to th

[jira] [Commented] (SPARK-10795) FileNotFoundException while deploying pyspark job on cluster

2016-03-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173503#comment-15173503 ] Jeff Zhang commented on SPARK-10795: What's your spark version ? And is it possible f

[jira] [Commented] (SPARK-10795) FileNotFoundException while deploying pyspark job on cluster

2016-03-01 Thread Daniel Jouany (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173489#comment-15173489 ] Daniel Jouany commented on SPARK-10795: --- Hi - I am facing the exact same problem.

[jira] [Assigned] (SPARK-13592) pyspark failed to launch on Windows client

2016-03-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13592: Assignee: Apache Spark > pyspark failed to launch on Windows client >

[jira] [Commented] (SPARK-13592) pyspark failed to launch on Windows client

2016-03-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173474#comment-15173474 ] Apache Spark commented on SPARK-13592: -- User 'tsudukim' has created a pull request f

[jira] [Assigned] (SPARK-13592) pyspark failed to launch on Windows client

2016-03-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13592: Assignee: (was: Apache Spark) > pyspark failed to launch on Windows client > -

[jira] [Commented] (SPARK-13592) pyspark failed to launch on Windows client

2016-03-01 Thread Masayoshi TSUZUKI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173472#comment-15173472 ] Masayoshi TSUZUKI commented on SPARK-13592: --- The error message says {quote} 's

[jira] [Comment Edited] (SPARK-13587) Support virtualenv in PySpark

2016-03-01 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173228#comment-15173228 ] Jeff Zhang edited comment on SPARK-13587 at 3/1/16 8:30 AM: T

[jira] [Created] (SPARK-13592) pyspark failed to launch on Windows client

2016-03-01 Thread Masayoshi TSUZUKI (JIRA)
Masayoshi TSUZUKI created SPARK-13592: - Summary: pyspark failed to launch on Windows client Key: SPARK-13592 URL: https://issues.apache.org/jira/browse/SPARK-13592 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-13591) Remove Back-ticks in Attribute/Alias Names

2016-03-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13591: Assignee: Apache Spark > Remove Back-ticks in Attribute/Alias Names >

[jira] [Assigned] (SPARK-13591) Remove Back-ticks in Attribute/Alias Names

2016-03-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13591: Assignee: (was: Apache Spark) > Remove Back-ticks in Attribute/Alias Names > -

[jira] [Commented] (SPARK-13591) Remove Back-ticks in Attribute/Alias Names

2016-03-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173418#comment-15173418 ] Apache Spark commented on SPARK-13591: -- User 'gatorsmile' has created a pull request

<    1   2   3