[jira] [Commented] (SPARK-12113) Add timing metrics to blocking phases for spark sql

2016-06-15 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333160#comment-15333160 ] Takeshi Yamamuro commented on SPARK-12113: -- [~rxin] okay, I'll rework based on the #10116 patch.

[jira] [Updated] (SPARK-15983) Remove FileFormat.prepareRead()

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15983: --- Summary: Remove FileFormat.prepareRead() (was: Remove FileFormat.prepareRead) > Remove

[jira] [Assigned] (SPARK-15983) Remove FileFormat.prepareRead

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15983: Assignee: Cheng Lian (was: Apache Spark) > Remove FileFormat.prepareRead >

[jira] [Assigned] (SPARK-15983) Remove FileFormat.prepareRead

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15983: Assignee: Apache Spark (was: Cheng Lian) > Remove FileFormat.prepareRead >

[jira] [Commented] (SPARK-15983) Remove FileFormat.prepareRead

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333141#comment-15333141 ] Apache Spark commented on SPARK-15983: -- User 'liancheng' has created a pull request for this issue:

[jira] [Updated] (SPARK-15983) Remove FileFormat.prepareRead

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15983: --- Description: Interface method {{FileFormat.prepareRead()}} was added in [PR

[jira] [Created] (SPARK-15983) Remove FileFormat.prepareRead

2016-06-15 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-15983: -- Summary: Remove FileFormat.prepareRead Key: SPARK-15983 URL: https://issues.apache.org/jira/browse/SPARK-15983 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-12922) Implement gapply() on DataFrame in SparkR

2016-06-15 Thread Narine Kokhlikyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333125#comment-15333125 ] Narine Kokhlikyan edited comment on SPARK-12922 at 6/16/16 5:25 AM:

[jira] [Commented] (SPARK-12922) Implement gapply() on DataFrame in SparkR

2016-06-15 Thread Narine Kokhlikyan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333125#comment-15333125 ] Narine Kokhlikyan commented on SPARK-12922: --- FYI, [~olarayej], [~aloknsingh], [~vijayrb]! >

[jira] [Commented] (SPARK-15919) DStream "saveAsTextFile" doesn't update the prefix after each checkpoint

2016-06-15 Thread Aamir Abbas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333119#comment-15333119 ] Aamir Abbas commented on SPARK-15919: - I have tried the solution you suggested, i-e window()

[jira] [Assigned] (SPARK-15981) Fix bug in python DataStreamReader

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15981: Assignee: Tathagata Das (was: Apache Spark) > Fix bug in python DataStreamReader >

[jira] [Commented] (SPARK-15981) Fix bug in python DataStreamReader

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333113#comment-15333113 ] Apache Spark commented on SPARK-15981: -- User 'tdas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15981) Fix bug in python DataStreamReader

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15981: Assignee: Apache Spark (was: Tathagata Das) > Fix bug in python DataStreamReader >

[jira] [Updated] (SPARK-15906) Complementary Naive Bayes Algorithm Implementation

2016-06-15 Thread MIN-FU YANG (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] MIN-FU YANG updated SPARK-15906: Description: Improve the Naive Bayes algorithm on skew data according to "Tackling the Poor

[jira] [Updated] (SPARK-15906) Complementary Naive Bayes Algorithm Implementation

2016-06-15 Thread MIN-FU YANG (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] MIN-FU YANG updated SPARK-15906: Description: Improve the Naive Bayes algorithm on skew data according to "Tackling the Poor

[jira] [Commented] (SPARK-15906) Complementary Naive Bayes Algorithm Implementation

2016-06-15 Thread MIN-FU YANG (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333098#comment-15333098 ] MIN-FU YANG commented on SPARK-15906: - OK, the description is updated. > Complementary Naive Bayes

[jira] [Created] (SPARK-15982) DataFrameReader.orc() should support varargs like json, csv, and parquet

2016-06-15 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-15982: - Summary: DataFrameReader.orc() should support varargs like json, csv, and parquet Key: SPARK-15982 URL: https://issues.apache.org/jira/browse/SPARK-15982 Project:

[jira] [Resolved] (SPARK-12922) Implement gapply() on DataFrame in SparkR

2016-06-15 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-12922. --- Resolution: Fixed Assignee: Narine Kokhlikyan Fix Version/s:

[jira] [Created] (SPARK-15981) Fix bug in python DataStreamReader

2016-06-15 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-15981: - Summary: Fix bug in python DataStreamReader Key: SPARK-15981 URL: https://issues.apache.org/jira/browse/SPARK-15981 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-15980) Add PushPredicateThroughObjectConsumer rule to Optimizer.

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15980: Assignee: (was: Apache Spark) > Add PushPredicateThroughObjectConsumer rule to

[jira] [Assigned] (SPARK-15980) Add PushPredicateThroughObjectConsumer rule to Optimizer.

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15980: Assignee: Apache Spark > Add PushPredicateThroughObjectConsumer rule to Optimizer. >

[jira] [Commented] (SPARK-15980) Add PushPredicateThroughObjectConsumer rule to Optimizer.

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333089#comment-15333089 ] Apache Spark commented on SPARK-15980: -- User 'ueshin' has created a pull request for this issue:

[jira] [Created] (SPARK-15980) Add PushPredicateThroughObjectConsumer rule to Optimizer.

2016-06-15 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-15980: - Summary: Add PushPredicateThroughObjectConsumer rule to Optimizer. Key: SPARK-15980 URL: https://issues.apache.org/jira/browse/SPARK-15980 Project: Spark

[jira] [Commented] (SPARK-14048) Aggregation operations on structs fail when the structs have fields with special characters

2016-06-15 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333076#comment-15333076 ] Simeon Simeonov commented on SPARK-14048: - Yes, I get the exact same failure with 1.6.1. >

[jira] [Comment Edited] (SPARK-14048) Aggregation operations on structs fail when the structs have fields with special characters

2016-06-15 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333076#comment-15333076 ] Simeon Simeonov edited comment on SPARK-14048 at 6/16/16 4:46 AM: -- Yes,

[jira] [Commented] (SPARK-15817) Spark client picking hive 1.2.1 by default which failed to alter a table name

2016-06-15 Thread Nataraj Gorantla (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333069#comment-15333069 ] Nataraj Gorantla commented on SPARK-15817: -- Can some one please add an update . Thanks,

[jira] [Resolved] (SPARK-15824) Run 'with ... insert ... select' failed when use spark thriftserver

2016-06-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-15824. - Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13678

[jira] [Updated] (SPARK-15824) Run 'with ... insert ... select' failed when use spark thriftserver

2016-06-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-15824: Assignee: Herman van Hovell > Run 'with ... insert ... select' failed when use spark thriftserver

[jira] [Updated] (SPARK-12492) SQL page of Spark-sql is always blank

2016-06-15 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-12492: - Issue Type: Improvement (was: Bug) > SQL page of Spark-sql is always blank >

[jira] [Resolved] (SPARK-12492) SQL page of Spark-sql is always blank

2016-06-15 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-12492. -- Resolution: Fixed Assignee: KaiXinXIaoLei Fix Version/s: 2.0.0 > SQL page of

[jira] [Commented] (SPARK-15639) Try to push down filter at RowGroups level for parquet reader

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333030#comment-15333030 ] Apache Spark commented on SPARK-15639: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15977) TRUNCATE TABLE does not work with Datasource tables outside of Hive

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15977: Assignee: Herman van Hovell (was: Apache Spark) > TRUNCATE TABLE does not work with

[jira] [Commented] (SPARK-15977) TRUNCATE TABLE does not work with Datasource tables outside of Hive

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15333022#comment-15333022 ] Apache Spark commented on SPARK-15977: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15977) TRUNCATE TABLE does not work with Datasource tables outside of Hive

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15977: Assignee: Apache Spark (was: Herman van Hovell) > TRUNCATE TABLE does not work with

[jira] [Commented] (SPARK-15979) Rename various Parquet support classes

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332993#comment-15332993 ] Apache Spark commented on SPARK-15979: -- User 'rxin' has created a pull request for this issue:

[jira] [Resolved] (SPARK-15851) Spark 2.0 does not compile in Windows 7

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15851. - Resolution: Fixed Assignee: Reynold Xin Fix Version/s: 2.0.0 > Spark 2.0 does

[jira] [Resolved] (SPARK-13498) JDBCRDD should update some input metrics

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-13498. - Resolution: Fixed Assignee: Wayne Song Fix Version/s: 2.0.0 > JDBCRDD should

[jira] [Assigned] (SPARK-15979) Rename various Parquet support classes

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15979: Assignee: Apache Spark (was: Reynold Xin) > Rename various Parquet support classes >

[jira] [Assigned] (SPARK-15979) Rename various Parquet support classes

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15979: Assignee: Reynold Xin (was: Apache Spark) > Rename various Parquet support classes >

[jira] [Created] (SPARK-15979) Rename various Parquet support classes

2016-06-15 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-15979: --- Summary: Rename various Parquet support classes Key: SPARK-15979 URL: https://issues.apache.org/jira/browse/SPARK-15979 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-15958) Make initial buffer size for the Sorter configurable

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15958: Assignee: Apache Spark > Make initial buffer size for the Sorter configurable >

[jira] [Resolved] (SPARK-15979) Rename various Parquet support classes

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15979. - Resolution: Fixed > Rename various Parquet support classes >

[jira] [Commented] (SPARK-15979) Rename various Parquet support classes

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332968#comment-15332968 ] Apache Spark commented on SPARK-15979: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15958) Make initial buffer size for the Sorter configurable

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15958: Assignee: (was: Apache Spark) > Make initial buffer size for the Sorter configurable

[jira] [Commented] (SPARK-15958) Make initial buffer size for the Sorter configurable

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332966#comment-15332966 ] Apache Spark commented on SPARK-15958: -- User 'sitalkedia' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-15690) Fast single-node (single-process) in-memory shuffle

2016-06-15 Thread Joseph Fourny (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332960#comment-15332960 ] Joseph Fourny edited comment on SPARK-15690 at 6/16/16 3:00 AM: I am

[jira] [Commented] (SPARK-15690) Fast single-node (single-process) in-memory shuffle

2016-06-15 Thread Joseph Fourny (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332960#comment-15332960 ] Joseph Fourny commented on SPARK-15690: --- +1 on this. I am trying to develop single-node clusters on

[jira] [Assigned] (SPARK-15978) Some improvement of "Show Tables"

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15978: Assignee: Apache Spark > Some improvement of "Show Tables" >

[jira] [Assigned] (SPARK-15978) Some improvement of "Show Tables"

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15978: Assignee: (was: Apache Spark) > Some improvement of "Show Tables" >

[jira] [Assigned] (SPARK-15978) Some improvement of "Show Tables"

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15978: Assignee: Apache Spark > Some improvement of "Show Tables" >

[jira] [Commented] (SPARK-15978) Some improvement of "Show Tables"

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332802#comment-15332802 ] Apache Spark commented on SPARK-15978: -- User 'bomeng' has created a pull request for this issue:

[jira] [Commented] (SPARK-15782) --packages doesn't work with the spark-shell

2016-06-15 Thread Nezih Yigitbasi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332797#comment-15332797 ] Nezih Yigitbasi commented on SPARK-15782: - reopened, will submit a PR including Marcelo's fix on

[jira] [Reopened] (SPARK-15782) --packages doesn't work with the spark-shell

2016-06-15 Thread Nezih Yigitbasi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nezih Yigitbasi reopened SPARK-15782: - > --packages doesn't work with the spark-shell >

[jira] [Assigned] (SPARK-15782) --packages doesn't work with the spark-shell

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15782: Assignee: Apache Spark (was: Nezih Yigitbasi) > --packages doesn't work with the

[jira] [Assigned] (SPARK-15782) --packages doesn't work with the spark-shell

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15782: Assignee: Nezih Yigitbasi (was: Apache Spark) > --packages doesn't work with the

[jira] [Created] (SPARK-15978) Some improvement of "Show Tables"

2016-06-15 Thread Bo Meng (JIRA)
Bo Meng created SPARK-15978: --- Summary: Some improvement of "Show Tables" Key: SPARK-15978 URL: https://issues.apache.org/jira/browse/SPARK-15978 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-13498) JDBCRDD should update some input metrics

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332776#comment-15332776 ] Apache Spark commented on SPARK-13498: -- User 'rxin' has created a pull request for this issue:

[jira] [Commented] (SPARK-14048) Aggregation operations on structs fail when the structs have fields with special characters

2016-06-15 Thread Sean Zhong (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332775#comment-15332775 ] Sean Zhong commented on SPARK-14048: [~simeons] Are you able to reproduce this case any longer? I

[jira] [Created] (SPARK-15977) TRUNCATE TABLE does not work with Datasource tables outside of Hive

2016-06-15 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-15977: - Summary: TRUNCATE TABLE does not work with Datasource tables outside of Hive Key: SPARK-15977 URL: https://issues.apache.org/jira/browse/SPARK-15977

[jira] [Resolved] (SPARK-7848) Update SparkStreaming docs to incorporate FAQ and/or bullets w/ "knobs" information.

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-7848. Resolution: Fixed Assignee: Nirman Narang Fix Version/s: 2.0.0 > Update

[jira] [Resolved] (SPARK-8813) Combine files when there're many small files in table

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-8813. Resolution: Fixed Assignee: Michael Armbrust Fix Version/s: 2.0.0 > Combine files

[jira] [Commented] (SPARK-15974) Create a socket on YARN AM start-up

2016-06-15 Thread Mingyu Kim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332749#comment-15332749 ] Mingyu Kim commented on SPARK-15974: I agree this is not ideal. A lot of times setting up a server

[jira] [Resolved] (SPARK-12114) ColumnPruning rule fails in case of "Project <- Filter <- Join"

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12114. - Resolution: Fixed Fix Version/s: 2.0.0 > ColumnPruning rule fails in case of "Project <-

[jira] [Commented] (SPARK-12032) Filter can't be pushed down to correct Join because of bad order of Join

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332721#comment-15332721 ] Apache Spark commented on SPARK-12032: -- User 'flyson' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12329) spark-sql prints out SET commands to stdout instead of stderr

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12329: Assignee: Apache Spark > spark-sql prints out SET commands to stdout instead of stderr >

[jira] [Assigned] (SPARK-12329) spark-sql prints out SET commands to stdout instead of stderr

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12329: Assignee: (was: Apache Spark) > spark-sql prints out SET commands to stdout instead

[jira] [Resolved] (SPARK-9689) Cache doesn't refresh for HadoopFsRelation based table

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-9689. Resolution: Fixed Assignee: (was: Cheng Hao) Fix Version/s: 2.0.0 I think this

[jira] [Commented] (SPARK-4944) Table Not Found exception in "Create Table Like registered RDD table"

2016-06-15 Thread Derek Sabry (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332704#comment-15332704 ] Derek Sabry commented on SPARK-4944: This email account is inactive. Please contact another person at

[jira] [Resolved] (SPARK-4944) Table Not Found exception in "Create Table Like registered RDD table"

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-4944. Resolution: Auto Closed > Table Not Found exception in "Create Table Like registered RDD table" >

[jira] [Commented] (SPARK-15976) Make Stage Numbering determinstic

2016-06-15 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332686#comment-15332686 ] Imran Rashid commented on SPARK-15976: -- cc [~kayousterhout] [~markhamstra] > Make Stage Numbering

[jira] [Updated] (SPARK-15177) SparkR 2.0 QA: New R APIs and API docs for mllib.R

2016-06-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15177: -- Issue Type: Documentation (was: Improvement) > SparkR 2.0 QA: New R APIs and API docs

[jira] [Updated] (SPARK-15129) Clarify conventions for calling Spark and MLlib from R

2016-06-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15129: -- Assignee: Gayathri Murali > Clarify conventions for calling Spark and MLlib from R >

[jira] [Created] (SPARK-15976) Make Stage Numbering determinstic

2016-06-15 Thread Imran Rashid (JIRA)
Imran Rashid created SPARK-15976: Summary: Make Stage Numbering determinstic Key: SPARK-15976 URL: https://issues.apache.org/jira/browse/SPARK-15976 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-15782) --packages doesn't work with the spark-shell

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332674#comment-15332674 ] Apache Spark commented on SPARK-15782: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-15974) Create a socket on YARN AM start-up

2016-06-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332637#comment-15332637 ] Marcelo Vanzin commented on SPARK-15974: bq. Spark's ApplicationMaster should optionally create

[jira] [Resolved] (SPARK-15934) Return binary mode in ThriftServer

2016-06-15 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15934. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13667

[jira] [Resolved] (SPARK-15776) Type coercion incorrect

2016-06-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-15776. - Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13651

[jira] [Updated] (SPARK-15776) Type coercion incorrect

2016-06-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-15776: Assignee: Sean Zhong > Type coercion incorrect > --- > > Key:

[jira] [Updated] (SPARK-15319) Fix SparkR doc layout for corr and other DataFrame stats functions

2016-06-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15319: -- Issue Type: Documentation (was: Bug) > Fix SparkR doc layout for corr and other

[jira] [Updated] (SPARK-15319) Fix SparkR doc layout for corr and other DataFrame stats functions

2016-06-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-15319: -- Affects Version/s: 2.0.0 > Fix SparkR doc layout for corr and other DataFrame stats

[jira] [Created] (SPARK-15974) Create a socket on YARN AM start-up

2016-06-15 Thread Mingyu Kim (JIRA)
Mingyu Kim created SPARK-15974: -- Summary: Create a socket on YARN AM start-up Key: SPARK-15974 URL: https://issues.apache.org/jira/browse/SPARK-15974 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-15851) Spark 2.0 does not compile in Windows 7

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332612#comment-15332612 ] Apache Spark commented on SPARK-15851: -- User 'rxin' has created a pull request for this issue:

[jira] [Created] (SPARK-15975) Improper Popen.wait() return code handling in dev/run-tests

2016-06-15 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-15975: -- Summary: Improper Popen.wait() return code handling in dev/run-tests Key: SPARK-15975 URL: https://issues.apache.org/jira/browse/SPARK-15975 Project: Spark

[jira] [Commented] (SPARK-15975) Improper Popen.wait() return code handling in dev/run-tests

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332620#comment-15332620 ] Apache Spark commented on SPARK-15975: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15975) Improper Popen.wait() return code handling in dev/run-tests

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15975: Assignee: Josh Rosen (was: Apache Spark) > Improper Popen.wait() return code handling in

[jira] [Assigned] (SPARK-15975) Improper Popen.wait() return code handling in dev/run-tests

2016-06-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15975: Assignee: Apache Spark (was: Josh Rosen) > Improper Popen.wait() return code handling in

[jira] [Commented] (SPARK-15930) Add Row count property to FPGrowth model

2016-06-15 Thread Gayathri Murali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332588#comment-15332588 ] Gayathri Murali commented on SPARK-15930: - [~yuhaoyan] If you havent already started working on

[jira] [Commented] (SPARK-3847) Enum.hashCode is only consistent within the same JVM

2016-06-15 Thread Brett Stime (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332587#comment-15332587 ] Brett Stime commented on SPARK-3847: Another option: have the default behavior be 'safe' and not share

[jira] [Commented] (SPARK-15447) Performance test for ALS in Spark 2.0

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332578#comment-15332578 ] Reynold Xin commented on SPARK-15447: - We can close this one now can't we? > Performance test for

[jira] [Updated] (SPARK-15901) Test Cases for CONVERT_METASTORE_ORC and CONVERT_METASTORE_PARQUET

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15901: --- Assignee: Xiao Li > Test Cases for CONVERT_METASTORE_ORC and CONVERT_METASTORE_PARQUET >

[jira] [Resolved] (SPARK-15901) Test Cases for CONVERT_METASTORE_ORC and CONVERT_METASTORE_PARQUET

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15901. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13622

[jira] [Closed] (SPARK-12173) Consider supporting DataSet API in SparkR

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-12173. --- Resolution: Won't Fix Marking as won't fix. R doesn't have compile time safety and as a result it

[jira] [Resolved] (SPARK-15782) --packages doesn't work with the spark-shell

2016-06-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-15782. Resolution: Fixed Assignee: Nezih Yigitbasi Fix Version/s: 2.0.0 >

[jira] [Commented] (SPARK-14953) LocalBackend should revive offers periodically

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332560#comment-15332560 ] Cheng Lian commented on SPARK-14953: Marked this as won't fix since this isn't causing any actual

[jira] [Commented] (SPARK-15930) Add Row count property to FPGrowth model

2016-06-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332559#comment-15332559 ] Joseph K. Bradley commented on SPARK-15930: --- This seems like a reasonable field to add to

[jira] [Updated] (SPARK-14953) LocalBackend should revive offers periodically

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14953: Target Version/s: (was: 2.0.0) > LocalBackend should revive offers periodically >

[jira] [Updated] (SPARK-14953) LocalBackend should revive offers periodically

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14953: Affects Version/s: 2.0.0 > LocalBackend should revive offers periodically >

[jira] [Resolved] (SPARK-14953) LocalBackend should revive offers periodically

2016-06-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-14953. Resolution: Won't Fix > LocalBackend should revive offers periodically >

[jira] [Resolved] (SPARK-15715) Altering partition storage information doesn't work in Hive

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15715. - Resolution: Fixed Fix Version/s: 2.0.0 > Altering partition storage information doesn't

[jira] [Updated] (SPARK-12978) Skip unnecessary final group-by when input data already clustered with group-by keys

2016-06-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12978: Target Version/s: 2.1.0 (was: 2.0.0) > Skip unnecessary final group-by when input data already

  1   2   3   >