[jira] [Commented] (SPARK-17815) Report committed offsets

2016-10-09 Thread Ofir Manor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15559541#comment-15559541 ] Ofir Manor commented on SPARK-17815: As far as I understand, there is a clear single-

[jira] [Commented] (SPARK-14136) Spark 2.0 can't start with yarn mode with ClassNotFoundException: org.apache.spark.deploy.yarn.history.YarnHistoryService

2016-10-09 Thread Mike Fang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15559540#comment-15559540 ] Mike Fang commented on SPARK-14136: --- To be more specific, it should be SPARK_CONF_DIR i

[jira] [Commented] (SPARK-17815) Report committed offsets

2016-10-09 Thread Ofir Manor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15559549#comment-15559549 ] Ofir Manor commented on SPARK-17815: Good, we are on the same page, no argument that

[jira] [Commented] (SPARK-17820) Spark sqlContext.sql() performs only first insert for HiveQL "FROM target INSERT INTO dest" command to insert into multiple target tables from same source

2016-10-09 Thread Jiang Xingbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15559565#comment-15559565 ] Jiang Xingbo commented on SPARK-17820: -- [~kmbeyond]I tried on Spark 1.6.0 and was un

[jira] [Commented] (SPARK-17219) QuantileDiscretizer does strange things with NaN values

2016-10-09 Thread Vincent (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15559597#comment-15559597 ] Vincent commented on SPARK-17219: - No problem. I will try to submit another PR based on a

[jira] [Resolved] (SPARK-17287) PySpark sc.AddFile method does not support the recursive keyword argument

2016-10-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-17287. --- Resolution: Duplicate > PySpark sc.AddFile method does not support the recursive keyword argument > -

[jira] [Resolved] (SPARK-11326) Support for authentication and encryption in standalone mode

2016-10-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11326. --- Resolution: Won't Fix > Support for authentication and encryption in standalone mode > --

[jira] [Commented] (SPARK-17607) --driver-url doesn't point to my master_ip.

2016-10-09 Thread Sasi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15559636#comment-15559636 ] Sasi commented on SPARK-17607: -- Hey, Is there any workaround? my system run on production a

[jira] [Commented] (SPARK-17607) --driver-url doesn't point to my master_ip.

2016-10-09 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15559882#comment-15559882 ] Sean Owen commented on SPARK-17607: --- Same as https://issues.apache.org/jira/browse/SPAR

[jira] [Commented] (SPARK-8780) Move Python doctest code example from models to algorithms

2016-10-09 Thread Jiayue Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15559886#comment-15559886 ] Jiayue Zhang commented on SPARK-8780: - I will work on this, thanks. > Move Python doc

[jira] [Commented] (SPARK-17607) --driver-url doesn't point to my master_ip.

2016-10-09 Thread Sasi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15559888#comment-15559888 ] Sasi commented on SPARK-17607: -- I'll check it out. I ping because I don't find any relative

[jira] [Commented] (SPARK-17815) Report committed offsets

2016-10-09 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15559911#comment-15559911 ] Cody Koeninger commented on SPARK-17815: The WAL cannot be the only source of tru

[jira] [Commented] (SPARK-8780) Move Python doctest code example from models to algorithms

2016-10-09 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560103#comment-15560103 ] Yanbo Liang commented on SPARK-8780: Since spark.mllib package is in maintenance mode,

[jira] [Closed] (SPARK-8780) Move Python doctest code example from models to algorithms

2016-10-09 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang closed SPARK-8780. -- Resolution: Won't Fix > Move Python doctest code example from models to algorithms > ---

[jira] [Assigned] (SPARK-15917) Define the number of executors in standalone mode with an easy-to-use property

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15917: Assignee: Apache Spark > Define the number of executors in standalone mode with an easy-to

[jira] [Assigned] (SPARK-15917) Define the number of executors in standalone mode with an easy-to-use property

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15917: Assignee: (was: Apache Spark) > Define the number of executors in standalone mode with

[jira] [Commented] (SPARK-15917) Define the number of executors in standalone mode with an easy-to-use property

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560115#comment-15560115 ] Apache Spark commented on SPARK-15917: -- User 'JonathanTaws' has created a pull reque

[jira] [Commented] (SPARK-10838) Repeat to join one DataFrame twice,there will be AnalysisException.

2016-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560164#comment-15560164 ] Xiao Li commented on SPARK-10838: - The root cause is the same as Spark-13801. Close it no

[jira] [Closed] (SPARK-10838) Repeat to join one DataFrame twice,there will be AnalysisException.

2016-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li closed SPARK-10838. --- Resolution: Duplicate > Repeat to join one DataFrame twice,there will be AnalysisException. > ---

[jira] [Created] (SPARK-17839) UnsafeSorterSpillReader should use Nio's directbuffer to read the spill files in order to avoid additional copy

2016-10-09 Thread Sital Kedia (JIRA)
Sital Kedia created SPARK-17839: --- Summary: UnsafeSorterSpillReader should use Nio's directbuffer to read the spill files in order to avoid additional copy Key: SPARK-17839 URL: https://issues.apache.org/jira/browse/

[jira] [Commented] (SPARK-17745) Update Python API for NB to support weighted instances

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560231#comment-15560231 ] Apache Spark commented on SPARK-17745: -- User 'WeichenXu123' has created a pull reque

[jira] [Assigned] (SPARK-17745) Update Python API for NB to support weighted instances

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17745: Assignee: Apache Spark > Update Python API for NB to support weighted instances >

[jira] [Assigned] (SPARK-17745) Update Python API for NB to support weighted instances

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17745: Assignee: (was: Apache Spark) > Update Python API for NB to support weighted instances

[jira] [Updated] (SPARK-8842) Spark SQL - Insert into table Issue

2016-10-09 Thread James Greenwood (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Greenwood updated SPARK-8842: --- Affects Version/s: 2.0.1 > Spark SQL - Insert into table Issue > -

[jira] [Commented] (SPARK-17147) Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets

2016-10-09 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560241#comment-15560241 ] Cody Koeninger commented on SPARK-17147: [~graphex] My WIP is at https://github.c

[jira] [Created] (SPARK-17840) Add some pointers for wiki/CONTRIBUTING.md in README.md and some warnings in PULL_REQUEST_TEMPLATE

2016-10-09 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-17840: Summary: Add some pointers for wiki/CONTRIBUTING.md in README.md and some warnings in PULL_REQUEST_TEMPLATE Key: SPARK-17840 URL: https://issues.apache.org/jira/browse/SPARK-17840

[jira] [Updated] (SPARK-17840) Add some pointers for wiki/CONTRIBUTING.md in README.md and some warnings in PULL_REQUEST_TEMPLATE

2016-10-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-17840: - Description: It seems it'd better to give best efforts for describing the guides for creating JI

[jira] [Updated] (SPARK-17840) Add some pointers for wiki/CONTRIBUTING.md in README.md and some warnings in PULL_REQUEST_TEMPLATE

2016-10-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-17840: - Description: It seems it'd better to give best efforts for describing the guides for creating JI

[jira] [Created] (SPARK-17841) Kafka 0.10 commitQueue needs to be drained

2016-10-09 Thread Cody Koeninger (JIRA)
Cody Koeninger created SPARK-17841: -- Summary: Kafka 0.10 commitQueue needs to be drained Key: SPARK-17841 URL: https://issues.apache.org/jira/browse/SPARK-17841 Project: Spark Issue Type: Bu

[jira] [Commented] (SPARK-17841) Kafka 0.10 commitQueue needs to be drained

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560370#comment-15560370 ] Apache Spark commented on SPARK-17841: -- User 'koeninger' has created a pull request

[jira] [Assigned] (SPARK-17841) Kafka 0.10 commitQueue needs to be drained

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17841: Assignee: Apache Spark > Kafka 0.10 commitQueue needs to be drained >

[jira] [Assigned] (SPARK-17841) Kafka 0.10 commitQueue needs to be drained

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17841: Assignee: (was: Apache Spark) > Kafka 0.10 commitQueue needs to be drained > -

[jira] [Assigned] (SPARK-17839) UnsafeSorterSpillReader should use Nio's directbuffer to read the spill files in order to avoid additional copy

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17839: Assignee: Apache Spark > UnsafeSorterSpillReader should use Nio's directbuffer to read the

[jira] [Assigned] (SPARK-17839) UnsafeSorterSpillReader should use Nio's directbuffer to read the spill files in order to avoid additional copy

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17839: Assignee: (was: Apache Spark) > UnsafeSorterSpillReader should use Nio's directbuffer

[jira] [Commented] (SPARK-17839) UnsafeSorterSpillReader should use Nio's directbuffer to read the spill files in order to avoid additional copy

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560381#comment-15560381 ] Apache Spark commented on SPARK-17839: -- User 'sitalkedia' has created a pull request

[jira] [Commented] (SPARK-17820) Spark sqlContext.sql() performs only first insert for HiveQL "FROM target INSERT INTO dest" command to insert into multiple target tables from same source

2016-10-09 Thread Kiran Miryala (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560423#comment-15560423 ] Kiran Miryala commented on SPARK-17820: --- Yes, data exists in sqoop_import.departmen

[jira] [Commented] (SPARK-17147) Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets

2016-10-09 Thread Sean McKibben (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560439#comment-15560439 ] Sean McKibben commented on SPARK-17147: --- I just had time to read through the change

[jira] [Commented] (SPARK-17815) Report committed offsets

2016-10-09 Thread Ofir Manor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560661#comment-15560661 ] Ofir Manor commented on SPARK-17815: I hope others with better understanding of Spark

[jira] [Commented] (SPARK-17815) Report committed offsets

2016-10-09 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560684#comment-15560684 ] Cody Koeninger commented on SPARK-17815: Regarding kafka consumer behavior, I'm n

[jira] [Commented] (SPARK-17812) More granular control of starting offsets

2016-10-09 Thread Ofir Manor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560696#comment-15560696 ] Ofir Manor commented on SPARK-17812: Cody, you are absolutely right that the option n

[jira] [Comment Edited] (SPARK-17812) More granular control of starting offsets

2016-10-09 Thread Ofir Manor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560696#comment-15560696 ] Ofir Manor edited comment on SPARK-17812 at 10/9/16 10:00 PM: -

[jira] [Created] (SPARK-17842) Thread and memory leak in WindowDstream (UnionRDD ) when parallelPartition computation gets enabled.

2016-10-09 Thread Sreelal S L (JIRA)
Sreelal S L created SPARK-17842: --- Summary: Thread and memory leak in WindowDstream (UnionRDD ) when parallelPartition computation gets enabled. Key: SPARK-17842 URL: https://issues.apache.org/jira/browse/SPARK-1784

[jira] [Commented] (SPARK-17812) More granular control of starting offsets

2016-10-09 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560719#comment-15560719 ] Cody Koeninger commented on SPARK-17812: Generally agree with the direction of wh

[jira] [Commented] (SPARK-17815) Report committed offsets

2016-10-09 Thread Ofir Manor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560749#comment-15560749 ] Ofir Manor commented on SPARK-17815: Thanks Cody, this is much clearer. (BTW - I've b

[jira] [Commented] (SPARK-17812) More granular control of starting offsets

2016-10-09 Thread Ofir Manor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560820#comment-15560820 ] Ofir Manor commented on SPARK-17812: I'll have to think about it - not sure if it eve

[jira] [Commented] (SPARK-17815) Report committed offsets

2016-10-09 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560837#comment-15560837 ] Cody Koeninger commented on SPARK-17815: My personal concerns about complexity ar

[jira] [Commented] (SPARK-17812) More granular control of starting offsets

2016-10-09 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560839#comment-15560839 ] Cody Koeninger commented on SPARK-17812: That totally kills the usability of Subs

[jira] [Created] (SPARK-17843) History Server gives no feedback about the application list being incomplete

2016-10-09 Thread Vinayak Joshi (JIRA)
Vinayak Joshi created SPARK-17843: - Summary: History Server gives no feedback about the application list being incomplete Key: SPARK-17843 URL: https://issues.apache.org/jira/browse/SPARK-17843 Projec

[jira] [Commented] (SPARK-17812) More granular control of starting offsets

2016-10-09 Thread Ofir Manor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560865#comment-15560865 ] Ofir Manor commented on SPARK-17812: Why? If you choose oldest, latest or by timestam

[jira] [Commented] (SPARK-14761) PySpark DataFrame.join should reject invalid join methods even when join columns are not specified

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560874#comment-15560874 ] Apache Spark commented on SPARK-14761: -- User 'bkpathak' has created a pull request f

[jira] [Assigned] (SPARK-17843) History Server gives no feedback about the application list being incomplete

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17843: Assignee: Apache Spark > History Server gives no feedback about the application list being

[jira] [Assigned] (SPARK-17843) History Server gives no feedback about the application list being incomplete

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17843: Assignee: (was: Apache Spark) > History Server gives no feedback about the application

[jira] [Commented] (SPARK-17843) History Server gives no feedback about the application list being incomplete

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560892#comment-15560892 ] Apache Spark commented on SPARK-17843: -- User 'vijoshi' has created a pull request fo

[jira] [Commented] (SPARK-17815) Report committed offsets

2016-10-09 Thread Ofir Manor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560906#comment-15560906 ] Ofir Manor commented on SPARK-17815: I totally understand your concerns, and yes, I h

[jira] [Commented] (SPARK-17840) Add some pointers for wiki/CONTRIBUTING.md in README.md and some warnings in PULL_REQUEST_TEMPLATE

2016-10-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560914#comment-15560914 ] Reynold Xin commented on SPARK-17840: - My comment on the dev list email thread: Gith

[jira] [Commented] (SPARK-17812) More granular control of starting offsets

2016-10-09 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560925#comment-15560925 ] Cody Koeninger commented on SPARK-17812: I want to start a pattern subscription a

[jira] [Commented] (SPARK-14393) monotonicallyIncreasingId not monotonically increasing with downstream coalesce

2016-10-09 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560941#comment-15560941 ] Takeshi Yamamuro commented on SPARK-14393: -- [~hyukjin.kwon] What do u think the

[jira] [Created] (SPARK-17844) DataFrame API should simplify defining frame boundaries without partitioning/ordering

2016-10-09 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-17844: --- Summary: DataFrame API should simplify defining frame boundaries without partitioning/ordering Key: SPARK-17844 URL: https://issues.apache.org/jira/browse/SPARK-17844 P

[jira] [Commented] (SPARK-17844) DataFrame API should simplify defining frame boundaries without partitioning/ordering

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560966#comment-15560966 ] Apache Spark commented on SPARK-17844: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-17844) DataFrame API should simplify defining frame boundaries without partitioning/ordering

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17844: Assignee: Reynold Xin (was: Apache Spark) > DataFrame API should simplify defining frame

[jira] [Assigned] (SPARK-17844) DataFrame API should simplify defining frame boundaries without partitioning/ordering

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17844: Assignee: Apache Spark (was: Reynold Xin) > DataFrame API should simplify defining frame

[jira] [Commented] (SPARK-17840) Add some pointers for wiki/CONTRIBUTING.md in README.md and some warnings in PULL_REQUEST_TEMPLATE

2016-10-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560968#comment-15560968 ] Hyukjin Kwon commented on SPARK-17840: -- I do agree and thank you for your prompt res

[jira] [Commented] (SPARK-14393) monotonicallyIncreasingId not monotonically increasing with downstream coalesce

2016-10-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560991#comment-15560991 ] Hyukjin Kwon commented on SPARK-14393: -- Thanks for pinging me!, in my personal opini

[jira] [Commented] (SPARK-14393) monotonicallyIncreasingId not monotonically increasing with downstream coalesce

2016-10-09 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561008#comment-15561008 ] Takeshi Yamamuro commented on SPARK-14393: -- yea, since this ticket is totally in

[jira] [Commented] (SPARK-16938) Cannot resolve column name after a join

2016-10-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561067#comment-15561067 ] Dongjoon Hyun commented on SPARK-16938: --- This issue is not in progress anymore. Ple

[jira] [Comment Edited] (SPARK-16938) Cannot resolve column name after a join

2016-10-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561067#comment-15561067 ] Dongjoon Hyun edited comment on SPARK-16938 at 10/10/16 2:30 AM: --

[jira] [Comment Edited] (SPARK-16938) Cannot resolve column name after a join

2016-10-09 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561067#comment-15561067 ] Dongjoon Hyun edited comment on SPARK-16938 at 10/10/16 2:30 AM: --

[jira] [Commented] (SPARK-14958) Failed task hangs if error is encountered when getting task result

2016-10-09 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561094#comment-15561094 ] Rui Li commented on SPARK-14958: I hit the issue with Spark 1.6.1 > Failed task hangs if

[jira] [Commented] (SPARK-17842) Thread and memory leak in WindowDstream (UnionRDD ) when parallelPartition computation gets enabled.

2016-10-09 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561099#comment-15561099 ] Liwei Lin commented on SPARK-17842: --- [~sreelalsl] thanks for the very clear reproducer

[jira] [Created] (SPARK-17845) Improve window function frame boundary API in DataFrame

2016-10-09 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-17845: --- Summary: Improve window function frame boundary API in DataFrame Key: SPARK-17845 URL: https://issues.apache.org/jira/browse/SPARK-17845 Project: Spark Issue T

[jira] [Created] (SPARK-17846) A bad state of Running Applications with spark standalone HA

2016-10-09 Thread dylanzhou (JIRA)
dylanzhou created SPARK-17846: - Summary: A bad state of Running Applications with spark standalone HA Key: SPARK-17846 URL: https://issues.apache.org/jira/browse/SPARK-17846 Project: Spark Issu

[jira] [Updated] (SPARK-17846) A bad state of Running Applications with spark standalone HA

2016-10-09 Thread dylanzhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dylanzhou updated SPARK-17846: -- Attachment: QQ截图20161010121745.jpg Problem screenshots > A bad state of Running Applications with spa

[jira] [Updated] (SPARK-17846) A bad state of Running Applications with spark standalone HA

2016-10-09 Thread dylanzhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dylanzhou updated SPARK-17846: -- Attachment: (was: QQ截图20161010121745.jpg) > A bad state of Running Applications with spark standalo

[jira] [Updated] (SPARK-17846) A bad state of Running Applications with spark standalone HA

2016-10-09 Thread dylanzhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dylanzhou updated SPARK-17846: -- Attachment: Problem screenshots.jpg > A bad state of Running Applications with spark standalone HA > -

[jira] [Assigned] (SPARK-17656) Decide on the variant of @scala.annotation.varargs and use consistently

2016-10-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-17656: --- Assignee: Reynold Xin > Decide on the variant of @scala.annotation.varargs and use consisten

[jira] [Updated] (SPARK-17845) Improve window function frame boundary API in DataFrame

2016-10-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17845: Description: In SQL, we use the following to specify the frame boundaries for window functions:

[jira] [Updated] (SPARK-17845) Improve window function frame boundary API in DataFrame

2016-10-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17845: Description: ANSI SQL uses the following to specify the frame boundaries for window functions: {c

[jira] [Updated] (SPARK-17845) Improve window function frame boundary API in DataFrame

2016-10-09 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17845: Description: ANSI SQL uses the following to specify the frame boundaries for window functions: {c

[jira] [Resolved] (SPARK-17832) TableIdentifier.quotedString creates un-parseable names when name contains a backtick

2016-10-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17832. --- Resolution: Fixed Assignee: Jiang Xingbo Fix Version/s: 2.1.0

[jira] [Created] (SPARK-17847) Copy GaussianMixture implementation from mllib to ml

2016-10-09 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-17847: --- Summary: Copy GaussianMixture implementation from mllib to ml Key: SPARK-17847 URL: https://issues.apache.org/jira/browse/SPARK-17847 Project: Spark Issue Type

[jira] [Resolved] (SPARK-17741) Grammar to parse top level and nested data fields separately

2016-10-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17741. --- Resolution: Fixed Assignee: Jiang Xingbo Fix Version/s: 2.1.0 > Gramm

[jira] [Assigned] (SPARK-17847) Copy GaussianMixture implementation from mllib to ml

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17847: Assignee: (was: Apache Spark) > Copy GaussianMixture implementation from mllib to ml >

[jira] [Commented] (SPARK-17847) Copy GaussianMixture implementation from mllib to ml

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561294#comment-15561294 ] Apache Spark commented on SPARK-17847: -- User 'yanboliang' has created a pull request

[jira] [Assigned] (SPARK-17847) Copy GaussianMixture implementation from mllib to ml

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17847: Assignee: Apache Spark > Copy GaussianMixture implementation from mllib to ml > --

[jira] [Commented] (SPARK-17845) Improve window function frame boundary API in DataFrame

2016-10-09 Thread Russell Spitzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561305#comment-15561305 ] Russell Spitzer commented on SPARK-17845: - FWIW, I Find (2) much more readable >

[jira] [Comment Edited] (SPARK-17845) Improve window function frame boundary API in DataFrame

2016-10-09 Thread Russell Spitzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561305#comment-15561305 ] Russell Spitzer edited comment on SPARK-17845 at 10/10/16 5:14 AM:

[jira] [Created] (SPARK-17848) Move LabelCol datatype cast into Predictor.fit

2016-10-09 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-17848: Summary: Move LabelCol datatype cast into Predictor.fit Key: SPARK-17848 URL: https://issues.apache.org/jira/browse/SPARK-17848 Project: Spark Issue Type: Im

[jira] [Resolved] (SPARK-11584) The attribute of temporay table shows false

2016-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-11584. - Resolution: Not A Problem This has been resolved in the native DDL support since 2.0 Close it now. Thanks

[jira] [Commented] (SPARK-10848) Applied JSON Schema Works for json RDD but not when loading json file

2016-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561328#comment-15561328 ] Xiao Li commented on SPARK-10848: - This should be resolved in the latest file format refa

[jira] [Reopened] (SPARK-10848) Applied JSON Schema Works for json RDD but not when loading json file

2016-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reopened SPARK-10848: - > Applied JSON Schema Works for json RDD but not when loading json file > ---

[jira] [Resolved] (SPARK-10848) Applied JSON Schema Works for json RDD but not when loading json file

2016-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-10848. - Resolution: Fixed > Applied JSON Schema Works for json RDD but not when loading json file > -

[jira] [Resolved] (SPARK-10848) Applied JSON Schema Works for json RDD but not when loading json file

2016-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-10848. - Resolution: Not A Problem > Applied JSON Schema Works for json RDD but not when loading json file > -

[jira] [Commented] (SPARK-9762) ALTER TABLE cannot find column

2016-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561337#comment-15561337 ] Xiao Li commented on SPARK-9762: Now, we do not support ALTER TABLE CHANGE COLUMN since 2.

[jira] [Resolved] (SPARK-9762) ALTER TABLE cannot find column

2016-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-9762. Resolution: Not A Problem > ALTER TABLE cannot find column > -- > >

[jira] [Updated] (SPARK-6962) Netty BlockTransferService hangs in the middle of SQL query

2016-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-6962: --- Component/s: Spark Core > Netty BlockTransferService hangs in the middle of SQL query > --

[jira] [Resolved] (SPARK-10673) spark.sql.hive.verifyPartitionPath Attempts to Verify Unregistered Partitions

2016-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-10673. - Resolution: Not A Problem > spark.sql.hive.verifyPartitionPath Attempts to Verify Unregistered Partitions

[jira] [Commented] (SPARK-10673) spark.sql.hive.verifyPartitionPath Attempts to Verify Unregistered Partitions

2016-10-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561348#comment-15561348 ] Xiao Li commented on SPARK-10673: - This should be resolved in 2.0. Please retry it. If it

[jira] [Commented] (SPARK-17848) Move LabelCol datatype cast into Predictor.fit

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561367#comment-15561367 ] Apache Spark commented on SPARK-17848: -- User 'zhengruifeng' has created a pull reque

[jira] [Assigned] (SPARK-17848) Move LabelCol datatype cast into Predictor.fit

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17848: Assignee: (was: Apache Spark) > Move LabelCol datatype cast into Predictor.fit > -

[jira] [Assigned] (SPARK-17848) Move LabelCol datatype cast into Predictor.fit

2016-10-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17848: Assignee: Apache Spark > Move LabelCol datatype cast into Predictor.fit >

  1   2   >