[jira] [Updated] (SPARK-12637) Print stage info of finished stages properly

2016-02-01 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-12637: -- Assignee: Navis > Print stage info of finished stages properly >

[jira] [Assigned] (SPARK-13123) Add wholestage codegen for sort

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13123: Assignee: Apache Spark > Add wholestage codegen for sort >

[jira] [Assigned] (SPARK-13087) Grouping by a complex expression may lead to incorrect AttributeReferences in aggregations

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13087: Assignee: (was: Apache Spark) > Grouping by a complex expression may lead to

[jira] [Commented] (SPARK-13087) Grouping by a complex expression may lead to incorrect AttributeReferences in aggregations

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127377#comment-15127377 ] Apache Spark commented on SPARK-13087: -- User 'marmbrus' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13087) Grouping by a complex expression may lead to incorrect AttributeReferences in aggregations

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13087: Assignee: Apache Spark > Grouping by a complex expression may lead to incorrect

[jira] [Closed] (SPARK-12715) Improve test coverage

2016-02-01 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-12715. -- Resolution: Won't Fix > Improve test coverage > - > > Key:

[jira] [Commented] (SPARK-12987) Drop fails when columns contain dots

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127348#comment-15127348 ] Apache Spark commented on SPARK-12987: -- User 'kevinyu98' has created a pull request for this issue:

[jira] [Commented] (SPARK-13087) Grouping by a complex expression may lead to incorrect AttributeReferences in aggregations

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127397#comment-15127397 ] Apache Spark commented on SPARK-13087: -- User 'marmbrus' has created a pull request for this issue:

[jira] [Updated] (SPARK-13124) Adding JQuery DataTables messed up the Web UI css and js

2016-02-01 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Bozarth updated SPARK-13124: - Attachment: js_issue.png css_issue.png > Adding JQuery DataTables messed up the

[jira] [Commented] (SPARK-13120) Shade protobuf-java

2016-02-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127555#comment-15127555 ] Sean Owen commented on SPARK-13120: --- Why does shading help there? I don't see any info on the cause. In

[jira] [Resolved] (SPARK-12637) Print stage info of finished stages properly

2016-02-01 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-12637. --- Resolution: Fixed Fix Version/s: 2.0.0 > Print stage info of finished stages properly >

[jira] [Commented] (SPARK-13122) Race condition in MemoryStore.unrollSafely() causes memory leak

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127381#comment-15127381 ] Apache Spark commented on SPARK-13122: -- User 'budde' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13122) Race condition in MemoryStore.unrollSafely() causes memory leak

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13122: Assignee: (was: Apache Spark) > Race condition in MemoryStore.unrollSafely() causes

[jira] [Assigned] (SPARK-13122) Race condition in MemoryStore.unrollSafely() causes memory leak

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13122: Assignee: Apache Spark > Race condition in MemoryStore.unrollSafely() causes memory leak

[jira] [Updated] (SPARK-13122) Race condition in MemoryStore.unrollSafely() causes memory leak

2016-02-01 Thread Adam Budde (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Budde updated SPARK-13122: --- Description: The

[jira] [Commented] (SPARK-13124) Adding JQuery DataTables messed up the Web UI css and js

2016-02-01 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127509#comment-15127509 ] Alex Bozarth commented on SPARK-13124: -- I'm working on a fix for this right now and will open a PR

[jira] [Created] (SPARK-13123) Add wholestage codegen for sort

2016-02-01 Thread Nong Li (JIRA)
Nong Li created SPARK-13123: --- Summary: Add wholestage codegen for sort Key: SPARK-13123 URL: https://issues.apache.org/jira/browse/SPARK-13123 Project: Spark Issue Type: Bug Components:

[jira] [Updated] (SPARK-13122) Race condition in MemoryStore.unrollSafely() causes memory leak

2016-02-01 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-13122: - Target Version/s: 1.6.1 > Race condition in MemoryStore.unrollSafely() causes memory

[jira] [Comment Edited] (SPARK-9141) DataFrame recomputed instead of using cached parent.

2016-02-01 Thread Placek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15125491#comment-15125491 ] Placek edited comment on SPARK-9141 at 2/2/16 1:21 AM: --- Can it be that this problem

[jira] [Updated] (SPARK-13122) Race condition in MemoryStore.unrollSafely() causes memory leak

2016-02-01 Thread Adam Budde (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Budde updated SPARK-13122: --- Description: The

[jira] [Commented] (SPARK-11918) WLS can not resolve some kinds of equation

2016-02-01 Thread Imran Younus (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127437#comment-15127437 ] Imran Younus commented on SPARK-11918: -- Several columns in the given dataset contain only zeros. In

[jira] [Created] (SPARK-13122) Race condition in MemoryStore.unrollSafely() causes memory leak

2016-02-01 Thread Adam Budde (JIRA)
Adam Budde created SPARK-13122: -- Summary: Race condition in MemoryStore.unrollSafely() causes memory leak Key: SPARK-13122 URL: https://issues.apache.org/jira/browse/SPARK-13122 Project: Spark

[jira] [Commented] (SPARK-13087) Grouping by a complex expression may lead to incorrect AttributeReferences in aggregations

2016-02-01 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127353#comment-15127353 ] Michael Armbrust commented on SPARK-13087: -- Here's a self-contained test case: {code}

[jira] [Assigned] (SPARK-12987) Drop fails when columns contain dots

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12987: Assignee: Apache Spark > Drop fails when columns contain dots >

[jira] [Commented] (SPARK-12715) Improve test coverage

2016-02-01 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127359#comment-15127359 ] Davies Liu commented on SPARK-12715: It was a guess, unfortunately it's wrong, we can close this

[jira] [Assigned] (SPARK-12718) SQL generation support for window functions

2016-02-01 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-12718: -- Assignee: Xiao Li > SQL generation support for window functions >

[jira] [Created] (SPARK-13109) SBT publishLocal failed to publish to local ivy repo

2016-02-01 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-13109: --- Summary: SBT publishLocal failed to publish to local ivy repo Key: SPARK-13109 URL: https://issues.apache.org/jira/browse/SPARK-13109 Project: Spark Issue

[jira] [Commented] (SPARK-13033) PySpark ml.regression support export/import

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15125945#comment-15125945 ] Apache Spark commented on SPARK-13033: -- User 'Wenpei' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13033) PySpark ml.regression support export/import

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13033: Assignee: Apache Spark > PySpark ml.regression support export/import >

[jira] [Assigned] (SPARK-13033) PySpark ml.regression support export/import

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13033: Assignee: (was: Apache Spark) > PySpark ml.regression support export/import >

[jira] [Commented] (SPARK-12739) Details of batch in Streaming tab uses two Duration columns

2016-02-01 Thread Mario Briggs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15125974#comment-15125974 ] Mario Briggs commented on SPARK-12739: -- Maybe the 1st duration can be named 'Output Op Duration' and

[jira] [Commented] (SPARK-13009) spark-streaming-twitter_2.10 does not make it possible to access the raw twitter json

2016-02-01 Thread Mario Briggs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15125940#comment-15125940 ] Mario Briggs commented on SPARK-13009: -- Andrew, as you yourself noted, i think it is more

[jira] [Commented] (SPARK-12950) Improve performance of BytesToBytesMap

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126039#comment-15126039 ] Apache Spark commented on SPARK-12950: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12950) Improve performance of BytesToBytesMap

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12950: Assignee: (was: Apache Spark) > Improve performance of BytesToBytesMap >

[jira] [Commented] (SPARK-13106) KafkaUtils.createDirectStream method with messageHandler and topics

2016-02-01 Thread Luis Alves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126054#comment-15126054 ] Luis Alves commented on SPARK-13106: In my case, I want to store in a database the last offset

[jira] [Commented] (SPARK-13109) SBT publishLocal failed to publish to local ivy repo

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15125991#comment-15125991 ] Apache Spark commented on SPARK-13109: -- User 'jerryshao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13109) SBT publishLocal failed to publish to local ivy repo

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13109: Assignee: Apache Spark > SBT publishLocal failed to publish to local ivy repo >

[jira] [Assigned] (SPARK-13109) SBT publishLocal failed to publish to local ivy repo

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13109: Assignee: (was: Apache Spark) > SBT publishLocal failed to publish to local ivy repo

[jira] [Assigned] (SPARK-12950) Improve performance of BytesToBytesMap

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12950: Assignee: Apache Spark > Improve performance of BytesToBytesMap >

[jira] [Updated] (SPARK-13110) How to configure the access of the Spark History Web UI with Kerberos authentication?

2016-02-01 Thread Pierre Beauvois (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Beauvois updated SPARK-13110: Description: Hello, Spark is installed on several machines of my cluster. These machines

[jira] [Created] (SPARK-13110) How to configure the access of the Spark History Web UI with Kerberos authentication?

2016-02-01 Thread Pierre Beauvois (JIRA)
Pierre Beauvois created SPARK-13110: --- Summary: How to configure the access of the Spark History Web UI with Kerberos authentication? Key: SPARK-13110 URL: https://issues.apache.org/jira/browse/SPARK-13110

[jira] [Created] (SPARK-13111) Spark UI is showing negative number of sessions

2016-02-01 Thread Kaushal Prajapati (JIRA)
Kaushal Prajapati created SPARK-13111: - Summary: Spark UI is showing negative number of sessions Key: SPARK-13111 URL: https://issues.apache.org/jira/browse/SPARK-13111 Project: Spark

[jira] [Updated] (SPARK-13111) Spark UI is showing negative number of sessions

2016-02-01 Thread Kaushal Prajapati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaushal Prajapati updated SPARK-13111: -- Attachment: spark_ui_session.jpg > Spark UI is showing negative number of sessions >

[jira] [Issue Comment Deleted] (SPARK-12279) Requesting a HBase table with kerberos is not working

2016-02-01 Thread Pierre Beauvois (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Beauvois updated SPARK-12279: Comment: was deleted (was: Hi [~yabodnar], I recently had the opportunity to do more tests

[jira] [Created] (SPARK-13114) java.lang.NegativeArraySizeException in CSV

2016-02-01 Thread Davies Liu (JIRA)
Davies Liu created SPARK-13114: -- Summary: java.lang.NegativeArraySizeException in CSV Key: SPARK-13114 URL: https://issues.apache.org/jira/browse/SPARK-13114 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-13087) Grouping by a complex expression may lead to incorrect AttributeReferences in aggregations

2016-02-01 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126012#comment-15126012 ] Liang-Chi Hsieh commented on SPARK-13087: - On latest build, looks like there is no this problem.

[jira] [Commented] (SPARK-12739) Details of batch in Streaming tab uses two Duration columns

2016-02-01 Thread Jacek Laskowski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126005#comment-15126005 ] Jacek Laskowski commented on SPARK-12739: - Looks like I am. If you could give it a shot, I'd

[jira] [Commented] (SPARK-12279) Requesting a HBase table with kerberos is not working

2016-02-01 Thread Pierre Beauvois (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126024#comment-15126024 ] Pierre Beauvois commented on SPARK-12279: - Hi [~yabodnar], I recently had the opportunity to do

[jira] [Commented] (SPARK-12279) Requesting a HBase table with kerberos is not working

2016-02-01 Thread Pierre Beauvois (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126026#comment-15126026 ] Pierre Beauvois commented on SPARK-12279: - Hi [~yabodnar], I recently had the opportunity to do

[jira] [Assigned] (SPARK-13065) streaming-twitter pass twitter4j.FilterQuery argument to TwitterUtils.createStream()

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13065: Assignee: (was: Apache Spark) > streaming-twitter pass twitter4j.FilterQuery argument

[jira] [Assigned] (SPARK-13065) streaming-twitter pass twitter4j.FilterQuery argument to TwitterUtils.createStream()

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13065: Assignee: Apache Spark > streaming-twitter pass twitter4j.FilterQuery argument to >

[jira] [Commented] (SPARK-13065) streaming-twitter pass twitter4j.FilterQuery argument to TwitterUtils.createStream()

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126227#comment-15126227 ] Apache Spark commented on SPARK-13065: -- User 'agsachin' has created a pull request for this issue:

[jira] [Commented] (SPARK-13083) Small spark sql queries get blocked if there is a long running query over a lot a partitions

2016-02-01 Thread Vishal Gupta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126133#comment-15126133 ] Vishal Gupta commented on SPARK-13083: -- cc [~marmbrus] , [~rxin] > Small spark sql queries get

[jira] [Created] (SPARK-13112) CoarsedExecutorBackend register to driver should wait Executor was ready

2016-02-01 Thread SuYan (JIRA)
SuYan created SPARK-13112: - Summary: CoarsedExecutorBackend register to driver should wait Executor was ready Key: SPARK-13112 URL: https://issues.apache.org/jira/browse/SPARK-13112 Project: Spark

[jira] [Commented] (SPARK-11416) Upgrade kryo package to version 3.0

2016-02-01 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126173#comment-15126173 ] Steve Loughran commented on SPARK-11416: there was some discussion on twitter's chill about this;

[jira] [Commented] (SPARK-7481) Add Hadoop 2.6+ profile to pull in object store FS accessors

2016-02-01 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126175#comment-15126175 ] Steve Loughran commented on SPARK-7481: --- having a lib/* would be fantastic, as it stops spark having

[jira] [Commented] (SPARK-10605) collect_list() and collect_set() should accept struct types as argument

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126473#comment-15126473 ] Apache Spark commented on SPARK-10605: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10605) collect_list() and collect_set() should accept struct types as argument

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10605: Assignee: Apache Spark > collect_list() and collect_set() should accept struct types as

[jira] [Assigned] (SPARK-10605) collect_list() and collect_set() should accept struct types as argument

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10605: Assignee: (was: Apache Spark) > collect_list() and collect_set() should accept struct

[jira] [Created] (SPARK-13133) When the option --master of spark-submit script is inconsistent with SparkConf.setMaster in Spark appliction code, the behavior is Difficult to understand

2016-02-01 Thread Li Ye (JIRA)
Li Ye created SPARK-13133: - Summary: When the option --master of spark-submit script is inconsistent with SparkConf.setMaster in Spark appliction code, the behavior is Difficult to understand Key: SPARK-13133 URL:

[jira] [Created] (SPARK-13134) add 'spark.streaming.kafka.partition.multiplier' into SparkConf

2016-02-01 Thread zhengcanbin (JIRA)
zhengcanbin created SPARK-13134: --- Summary: add 'spark.streaming.kafka.partition.multiplier' into SparkConf Key: SPARK-13134 URL: https://issues.apache.org/jira/browse/SPARK-13134 Project: Spark

[jira] [Closed] (SPARK-12710) Create local CoGroup operator

2016-02-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-12710. --- Resolution: Later > Create local CoGroup operator > - > >

[jira] [Closed] (SPARK-9988) Create local (external) sort operator

2016-02-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-9988. -- Resolution: Later > Create local (external) sort operator > - > >

[jira] [Closed] (SPARK-9989) Create local sort-merge join operator

2016-02-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-9989. -- Resolution: Later > Create local sort-merge join operator > - > >

[jira] [Closed] (SPARK-9987) Create local aggregate operator

2016-02-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-9987. -- Resolution: Later > Create local aggregate operator > --- > >

[jira] [Closed] (SPARK-9995) Create local Python evaluation operator

2016-02-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-9995. -- Resolution: Later > Create local Python evaluation operator > --- >

[jira] [Closed] (SPARK-9983) Local physical operators for query execution

2016-02-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-9983. -- Resolution: Later I'm going to close this as later, and we will come back to this after Spark 2.0. >

[jira] [Closed] (SPARK-10665) Connect the local iterators with the planner

2016-02-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-10665. --- Resolution: Later > Connect the local iterators with the planner >

[jira] [Updated] (SPARK-12705) Sorting column can't be resolved if it's not in projection

2016-02-01 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-12705: - Assignee: Xiao Li > Sorting column can't be resolved if it's not in projection >

[jira] [Updated] (SPARK-10777) order by fails when column is aliased and projection includes windowed aggregate

2016-02-01 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-10777: - Assignee: Xiao Li > order by fails when column is aliased and projection includes

[jira] [Created] (SPARK-13119) SparkR Ser/De fail to handle "columns(df)"

2016-02-01 Thread Xusen Yin (JIRA)
Xusen Yin created SPARK-13119: - Summary: SparkR Ser/De fail to handle "columns(df)" Key: SPARK-13119 URL: https://issues.apache.org/jira/browse/SPARK-13119 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-13087) Grouping by a complex expression may lead to incorrect AttributeReferences in aggregations

2016-02-01 Thread Trystan Leftwich (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126980#comment-15126980 ] Trystan Leftwich commented on SPARK-13087: -- I re-built 1.6.1 with commit

[jira] [Updated] (SPARK-13133) When the option --master of spark-submit script is inconsistent with SparkConf.setMaster in Spark appliction code, the behavior is Difficult to understand

2016-02-01 Thread Li Ye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Ye updated SPARK-13133: -- Description: When the option --master of spark-submit script is inconsistent with SparkConf.setMaster in

[jira] [Commented] (SPARK-13114) java.lang.NegativeArraySizeException in CSV

2016-02-01 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127792#comment-15127792 ] Hyukjin Kwon commented on SPARK-13114: -- This is already fixed in Spark,

[jira] [Closed] (SPARK-13134) add 'spark.streaming.kafka.partition.multiplier' into SparkConf

2016-02-01 Thread zhengcanbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengcanbin closed SPARK-13134. --- Resolution: Incomplete > add 'spark.streaming.kafka.partition.multiplier' into SparkConf >

[jira] [Resolved] (SPARK-13130) Make whole stage codegen variable names slightly easier to read

2016-02-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13130. - Resolution: Fixed Fix Version/s: 2.0.0 > Make whole stage codegen variable names slightly

[jira] [Created] (SPARK-13136) Data exchange (shuffle, broadcast) should only be handled by the exchange operator

2016-02-01 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-13136: --- Summary: Data exchange (shuffle, broadcast) should only be handled by the exchange operator Key: SPARK-13136 URL: https://issues.apache.org/jira/browse/SPARK-13136

[jira] [Comment Edited] (SPARK-13110) How to configure the access of the Spark History Web UI with Kerberos authentication?

2016-02-01 Thread Pierre Beauvois (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127837#comment-15127837 ] Pierre Beauvois edited comment on SPARK-13110 at 2/2/16 7:52 AM: - Hi

[jira] [Created] (SPARK-13137) NullPoingException in schema inference for CSV when the first line is empty

2016-02-01 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-13137: Summary: NullPoingException in schema inference for CSV when the first line is empty Key: SPARK-13137 URL: https://issues.apache.org/jira/browse/SPARK-13137 Project:

[jira] [Updated] (SPARK-6761) Approximate quantile

2016-02-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6761: --- Target Version/s: 2.0.0 (was: ) > Approximate quantile > > >

[jira] [Updated] (SPARK-13133) When the option --master of spark-submit script is inconsistent with SparkConf.setMaster in Spark appliction code, the behavior is Difficult to understand

2016-02-01 Thread Li Ye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Ye updated SPARK-13133: -- Description: When the option --master of spark-submit script is inconsistent with SparkConf.setMaster in

[jira] [Commented] (SPARK-13114) java.lang.NegativeArraySizeException in CSV

2016-02-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127801#comment-15127801 ] Reynold Xin commented on SPARK-13114: - See if we have an existing test. If we don't, we should add

[jira] [Created] (SPARK-13135) Don't print expressions recursively in generated code

2016-02-01 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-13135: --- Summary: Don't print expressions recursively in generated code Key: SPARK-13135 URL: https://issues.apache.org/jira/browse/SPARK-13135 Project: Spark Issue

[jira] [Commented] (SPARK-12725) SQL generation suffers from name conficts introduced by some analysis rules

2016-02-01 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127817#comment-15127817 ] Xiao Li commented on SPARK-12725: - Let me work on this at first. Will submit a PR tomorrow. : ) Thank

[jira] [Closed] (SPARK-12709) Create local Except operator

2016-02-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-12709. --- Resolution: Not A Problem This is not needed anymore assuming we implement anti-join. See

[jira] [Commented] (SPARK-13110) How to configure the access of the Spark History Web UI with Kerberos authentication?

2016-02-01 Thread Pierre Beauvois (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127837#comment-15127837 ] Pierre Beauvois commented on SPARK-13110: - Hi [~vanzin], Thank you for your feedback. I'm going

[jira] [Updated] (SPARK-13133) When the option --master of spark-submit script is inconsistent with SparkConf.setMaster in Spark appliction code, the behavior is Difficult to understand

2016-02-01 Thread Li Ye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Ye updated SPARK-13133: -- Description: When the option --master of spark-submit script is inconsistent with SparkConf.setMaster in

[jira] [Updated] (SPARK-13133) When the option --master of spark-submit script is inconsistent with SparkConf.setMaster in Spark appliction code, the behavior is Difficult to understand

2016-02-01 Thread Li Ye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Ye updated SPARK-13133: -- Description: When the option --master of spark-submit script is inconsistent with SparkConf.setMaster in

[jira] [Updated] (SPARK-13133) When the option --master of spark-submit script is inconsistent with SparkConf.setMaster in Spark appliction code, the behavior of Spark application is difficult to unde

2016-02-01 Thread Li Ye (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Ye updated SPARK-13133: -- Summary: When the option --master of spark-submit script is inconsistent with SparkConf.setMaster in Spark

[jira] [Commented] (SPARK-12344) Remove env-based configurations

2016-02-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127794#comment-15127794 ] Felix Cheung commented on SPARK-12344: -- I think we should remove `SPARK_YARN_USER_ENV` as well.

[jira] [Commented] (SPARK-13135) Don't print expressions recursively in generated code

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127804#comment-15127804 ] Apache Spark commented on SPARK-13135: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-13135) Don't print expressions recursively in generated code

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13135: Assignee: Reynold Xin (was: Apache Spark) > Don't print expressions recursively in

[jira] [Assigned] (SPARK-13135) Don't print expressions recursively in generated code

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13135: Assignee: Apache Spark (was: Reynold Xin) > Don't print expressions recursively in

[jira] [Updated] (SPARK-13136) Data exchange (shuffle, broadcast) should only be handled by the exchange operator

2016-02-01 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-13136: Description: In an ideal architecture, we have a very small number of physical operators that

[jira] [Assigned] (SPARK-13114) java.lang.NegativeArraySizeException in CSV

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13114: Assignee: (was: Apache Spark) > java.lang.NegativeArraySizeException in CSV >

[jira] [Commented] (SPARK-13114) java.lang.NegativeArraySizeException in CSV

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127836#comment-15127836 ] Apache Spark commented on SPARK-13114: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-13114) java.lang.NegativeArraySizeException in CSV

2016-02-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13114: Assignee: Apache Spark > java.lang.NegativeArraySizeException in CSV >

[jira] [Created] (SPARK-13126) History Server page always has horizontal scrollbar

2016-02-01 Thread Alex Bozarth (JIRA)
Alex Bozarth created SPARK-13126: Summary: History Server page always has horizontal scrollbar Key: SPARK-13126 URL: https://issues.apache.org/jira/browse/SPARK-13126 Project: Spark Issue

[jira] [Commented] (SPARK-13108) Encoding not working with non-ascii compatible encodings (UTF-16/32 etc.)

2016-02-01 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15127623#comment-15127623 ] Hyukjin Kwon commented on SPARK-13108: -- I think both would be great. > Encoding not working with

[jira] [Updated] (SPARK-13125) makes the ratio of KafkaRDD partition to kafka topic partition configurable.

2016-02-01 Thread zhengcanbin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengcanbin updated SPARK-13125: Shepherd: zhengcanbin > makes the ratio of KafkaRDD partition to kafka topic partition

<    1   2   3   >