[jira] [Assigned] (SPARK-12284) Output UnsafeRow from window function

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12284: Assignee: (was: Apache Spark) > Output UnsafeRow from window function > --

[jira] [Assigned] (SPARK-12284) Output UnsafeRow from window function

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12284: Assignee: Apache Spark > Output UnsafeRow from window function > -

[jira] [Commented] (SPARK-12580) Remove string concatenations from usage and extended in @ExpressionDescription

2015-12-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074738#comment-15074738 ] Kazuaki Ishizaki commented on SPARK-12580: -- Thank you, however I have already do

[jira] [Comment Edited] (SPARK-11437) createDataFrame shouldn't .take() when provided schema

2015-12-29 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074733#comment-15074733 ] Maciej Bryński edited comment on SPARK-11437 at 12/30/15 7:39 AM: -

[jira] [Comment Edited] (SPARK-11437) createDataFrame shouldn't .take() when provided schema

2015-12-29 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074733#comment-15074733 ] Maciej Bryński edited comment on SPARK-11437 at 12/30/15 7:38 AM: -

[jira] [Commented] (SPARK-11437) createDataFrame shouldn't .take() when provided schema

2015-12-29 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-11437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074733#comment-15074733 ] Maciej Bryński commented on SPARK-11437: There is such an API, but not public. {

[jira] [Comment Edited] (SPARK-12554) Standalone app scheduler will hang when app.coreToAssign < minCoresPerExecutor

2015-12-29 Thread Lijie Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074728#comment-15074728 ] Lijie Xu edited comment on SPARK-12554 at 12/30/15 7:33 AM: C

[jira] [Closed] (SPARK-12291) Support UnsafeRow in BroadcastLeftSemiJoinHash

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-12291. -- Resolution: Not A Problem > Support UnsafeRow in BroadcastLeftSemiJoinHash > --

[jira] [Commented] (SPARK-12554) Standalone app scheduler will hang when app.coreToAssign < minCoresPerExecutor

2015-12-29 Thread Lijie Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074728#comment-15074728 ] Lijie Xu commented on SPARK-12554: -- Case 2 may be categorized to misconfiguration. Howev

[jira] [Commented] (SPARK-12291) Support UnsafeRow in BroadcastLeftSemiJoinHash

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074729#comment-15074729 ] Davies Liu commented on SPARK-12291: Thanks, I will close this one. > Support Unsafe

[jira] [Assigned] (SPARK-12540) Support all TPCDS queries

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12540: -- Assignee: Davies Liu > Support all TPCDS queries > - > >

[jira] [Closed] (SPARK-11855) Catalyst breaks backwards compatibility in branch-1.6

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-11855. -- Resolution: Won't Fix > Catalyst breaks backwards compatibility in branch-1.6 > ---

[jira] [Commented] (SPARK-11437) createDataFrame shouldn't .take() when provided schema

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074726#comment-15074726 ] Davies Liu commented on SPARK-11437: [~maver1ck] Maybe we should provide a API to ver

[jira] [Assigned] (SPARK-12562) DataFrame.write.format("text") requires the column name to be called value

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12562: Assignee: Apache Spark > DataFrame.write.format("text") requires the column name to be cal

[jira] [Commented] (SPARK-12580) Remove string concatenations from usage and extended in @ExpressionDescription

2015-12-29 Thread somil deshmukh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074711#comment-15074711 ] somil deshmukh commented on SPARK-12580: I would like to work on this issue > Re

[jira] [Assigned] (SPARK-12541) Support rollup/cube in SQL query

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-12541: -- Assignee: Davies Liu > Support rollup/cube in SQL query > > >

[jira] [Updated] (SPARK-12541) Support rollup/cube in SQL query

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12541: --- Description: We have DataFrame API for rollup/cube, but do not support them in SQL parser (both SQLCo

[jira] [Updated] (SPARK-12541) Support rollup/cube in SQL query

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12541: --- Description: We have DataFrame API for rollup/cube, but do not support them in SQL parser (both SQLC

[jira] [Updated] (SPARK-12541) Support rollup/cube in SQL query

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12541: --- Summary: Support rollup/cube in SQL query (was: Support rollup/cube/povit in SQL query) > Support r

[jira] [Resolved] (SPARK-12564) Improve missing column AnalysisException

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12564. - Resolution: Fixed Assignee: Xiao Li (was: Apache Spark) Fix Version/s: 2.0.0 > I

[jira] [Commented] (SPARK-12558) AnalysisException when multiple functions applied in GROUP BY clause

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074675#comment-15074675 ] Apache Spark commented on SPARK-12558: -- User 'dilipbiswal' has created a pull reques

[jira] [Assigned] (SPARK-11559) Make `runs` no effect in k-means

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11559: Assignee: (was: Apache Spark) > Make `runs` no effect in k-means > ---

[jira] [Assigned] (SPARK-11559) Make `runs` no effect in k-means

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11559: Assignee: Apache Spark > Make `runs` no effect in k-means > --

[jira] [Commented] (SPARK-12537) Add option to accept quoting of all character backslash quoting mechanism

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074654#comment-15074654 ] Reynold Xin commented on SPARK-12537: - [~srowen] I think the problem right now is tha

[jira] [Commented] (SPARK-12537) Add option to accept quoting of all character backslash quoting mechanism

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074647#comment-15074647 ] Apache Spark commented on SPARK-12537: -- User 'Cazen' has created a pull request for

[jira] [Assigned] (SPARK-12558) AnalysisException when multiple functions applied in GROUP BY clause

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12558: Assignee: (was: Apache Spark) > AnalysisException when multiple functions applied in G

[jira] [Assigned] (SPARK-12558) AnalysisException when multiple functions applied in GROUP BY clause

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12558: Assignee: Apache Spark > AnalysisException when multiple functions applied in GROUP BY cla

[jira] [Commented] (SPARK-12537) Add option to accept quoting of all character backslash quoting mechanism

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074640#comment-15074640 ] Apache Spark commented on SPARK-12537: -- User 'Cazen' has created a pull request for

[jira] [Commented] (SPARK-11745) Enable more JSON parsing options for parsing non-standard JSON files

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074630#comment-15074630 ] Reynold Xin commented on SPARK-11745: - No particular reason -- I just didn't run into

[jira] [Updated] (SPARK-12580) Remove string concatenations from usage and extended in @ExpressionDescription

2015-12-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-12580: - Description: More than one string concatenation in {{usage}} or {{extended}} for {{@Expr

[jira] [Commented] (SPARK-2827) Add DegreeDist function support

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074612#comment-15074612 ] Apache Spark commented on SPARK-2827: - User 'maropu' has created a pull request for th

[jira] [Updated] (SPARK-12580) Remove string concatenations from usage and extended in @ExpressionDescription

2015-12-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-12580: - External issue ID: SPARK-12454, SPARK-12530, SPARK-12455,SPARK-12456,SPARK-12457,SPARK-12

[jira] [Updated] (SPARK-12580) Remove string concatenations from usage and extended in @ExpressionDescription

2015-12-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-12580: - External issue ID: SPARK-12454, SPARK-12530 (was: 12454, 12530) > Remove string concaten

[jira] [Created] (SPARK-12580) Remove string concatenations from usage and extended in @ExpressionDescription

2015-12-29 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-12580: Summary: Remove string concatenations from usage and extended in @ExpressionDescription Key: SPARK-12580 URL: https://issues.apache.org/jira/browse/SPARK-12580

[jira] [Assigned] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12511: Assignee: Shixiong Zhu (was: Apache Spark) > streaming driver with checkpointing unable t

[jira] [Assigned] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12511: Assignee: Apache Spark (was: Shixiong Zhu) > streaming driver with checkpointing unable t

[jira] [Assigned] (SPARK-12579) User-specified JDBC driver should always take precedence

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12579: Assignee: Apache Spark (was: Josh Rosen) > User-specified JDBC driver should always take

[jira] [Assigned] (SPARK-12579) User-specified JDBC driver should always take precedence

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12579: Assignee: Josh Rosen (was: Apache Spark) > User-specified JDBC driver should always take

[jira] [Commented] (SPARK-12554) Standalone app scheduler will hang when app.coreToAssign < minCoresPerExecutor

2015-12-29 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074588#comment-15074588 ] Saisai Shao commented on SPARK-12554: - For case 2, I think it is really a misconfigur

[jira] [Created] (SPARK-12579) User-specified JDBC driver should always take precedence

2015-12-29 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-12579: -- Summary: User-specified JDBC driver should always take precedence Key: SPARK-12579 URL: https://issues.apache.org/jira/browse/SPARK-12579 Project: Spark Issue Ty

[jira] [Updated] (SPARK-2356) Exception: Could not locate executable null\bin\winutils.exe in the Hadoop

2015-12-29 Thread Kostiantyn Kudriavtsev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kostiantyn Kudriavtsev updated SPARK-2356: -- Affects Version/s: 1.1.1 1.2.1 1.2.

[jira] [Updated] (SPARK-12571) AWS credentials not available for read.parquet in SQLContext

2015-12-29 Thread Kostiantyn Kudriavtsev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kostiantyn Kudriavtsev updated SPARK-12571: --- Affects Version/s: 1.5.1 > AWS credentials not available for read.parquet in

[jira] [Created] (SPARK-12577) better support of parentheses in partition by and order by clause of window function's over clause

2015-12-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-12577: --- Summary: better support of parentheses in partition by and order by clause of window function's over clause Key: SPARK-12577 URL: https://issues.apache.org/jira/browse/SPARK-12577

[jira] [Created] (SPARK-12578) Parser should not silently ignore the distinct keyword used in an aggregate function when OVER clause is used

2015-12-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-12578: --- Summary: Parser should not silently ignore the distinct keyword used in an aggregate function when OVER clause is used Key: SPARK-12578 URL: https://issues.apache.org/jira/browse/SP

[jira] [Updated] (SPARK-12576) Enable expression parsing (used in DataFrames)

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12576: Description: We use the existing SQL parser to parse expressions. We need to make sure the absorbe

[jira] [Created] (SPARK-12576) Enable expression parsing (used in DataFrames)

2015-12-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-12576: --- Summary: Enable expression parsing (used in DataFrames) Key: SPARK-12576 URL: https://issues.apache.org/jira/browse/SPARK-12576 Project: Spark Issue Type: Sub-

[jira] [Created] (SPARK-12575) Grammar parity with existing SQL parser

2015-12-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-12575: --- Summary: Grammar parity with existing SQL parser Key: SPARK-12575 URL: https://issues.apache.org/jira/browse/SPARK-12575 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-12564) Improve missing column AnalysisException

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12564: Assignee: Apache Spark > Improve missing column AnalysisException > --

[jira] [Created] (SPARK-12574) Move parser from hive module to catalyst (or sql-core) module

2015-12-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-12574: --- Summary: Move parser from hive module to catalyst (or sql-core) module Key: SPARK-12574 URL: https://issues.apache.org/jira/browse/SPARK-12574 Project: Spark

[jira] [Updated] (SPARK-12362) Create a full-fledged built-in SQL parser

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12362: Assignee: Herman van Hovell (was: Apache Spark) > Create a full-fledged built-in SQL parser >

[jira] [Commented] (SPARK-12362) Create a full-fledged built-in SQL parser

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074574#comment-15074574 ] Reynold Xin commented on SPARK-12362: - [~hvanhovell] please create additional subtask

[jira] [Updated] (SPARK-12362) Create a full-fledged built-in SQL parser

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12362: Priority: Critical (was: Major) > Create a full-fledged built-in SQL parser >

[jira] [Resolved] (SPARK-12572) Initial import of the Hive parser

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12572. - Resolution: Fixed Fix Version/s: 2.0.0 > Initial import of the Hive parser > -

[jira] [Created] (SPARK-12573) Add acknowledge that the parser was initially from Hive

2015-12-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-12573: --- Summary: Add acknowledge that the parser was initially from Hive Key: SPARK-12573 URL: https://issues.apache.org/jira/browse/SPARK-12573 Project: Spark Issue T

[jira] [Updated] (SPARK-12572) Initial import of the Hive parser

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12572: Description: Pull request: https://github.com/apache/spark/pull/10509 > Initial import of the Hive

[jira] [Created] (SPARK-12572) Initial import of the Hive parser

2015-12-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-12572: --- Summary: Initial import of the Hive parser Key: SPARK-12572 URL: https://issues.apache.org/jira/browse/SPARK-12572 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12148: Target Version/s: 2.0.0 > SparkR: rename DataFrame to SparkDataFrame >

[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074570#comment-15074570 ] Reynold Xin commented on SPARK-12148: - +1 We should consider just renaming this to Sp

[jira] [Commented] (SPARK-12564) Improve missing column AnalysisException

2015-12-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074546#comment-15074546 ] Xiao Li commented on SPARK-12564: - Let me push one PR: https://github.com/apache/spark/pu

[jira] [Comment Edited] (SPARK-12554) Standalone app scheduler will hang when app.coreToAssign < minCoresPerExecutor

2015-12-29 Thread Lijie Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074543#comment-15074543 ] Lijie Xu edited comment on SPARK-12554 at 12/30/15 2:11 AM: I

[jira] [Updated] (SPARK-12554) Standalone app scheduler will hang when app.coreToAssign < minCoresPerExecutor

2015-12-29 Thread Lijie Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lijie Xu updated SPARK-12554: - Description: In scheduleExecutorsOnWorker() in Master.scala, {{val keepScheduling = coresToAssign >= minC

[jira] [Commented] (SPARK-12554) Standalone app scheduler will hang when app.coreToAssign < minCoresPerExecutor

2015-12-29 Thread Lijie Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074543#comment-15074543 ] Lijie Xu commented on SPARK-12554: -- I agree that it is a design problem. {{spark.schedu

[jira] [Created] (SPARK-12571) AWS credentials not available for read.parquet in SQLContext

2015-12-29 Thread Kostiantyn Kudriavtsev (JIRA)
Kostiantyn Kudriavtsev created SPARK-12571: -- Summary: AWS credentials not available for read.parquet in SQLContext Key: SPARK-12571 URL: https://issues.apache.org/jira/browse/SPARK-12571 Proj

[jira] [Updated] (SPARK-12554) Standalone app scheduler will hang when app.coreToAssign < minCoresPerExecutor

2015-12-29 Thread Lijie Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lijie Xu updated SPARK-12554: - Description: In scheduleExecutorsOnWorker() in Master.scala, *val keepScheduling = coresToAssign >= minCo

[jira] [Assigned] (SPARK-12568) Add BINARY to Encoders

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12568: Assignee: Apache Spark (was: Michael Armbrust) > Add BINARY to Encoders > ---

[jira] [Commented] (SPARK-6724) Model import/export for FPGrowth

2015-12-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074512#comment-15074512 ] Joseph K. Bradley commented on SPARK-6724: -- Thanks for your patience. I'll take

[jira] [Commented] (SPARK-12567) Add aes_{encrypt,decrypt} UDFs

2015-12-29 Thread Kai Jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074511#comment-15074511 ] Kai Jiang commented on SPARK-12567: --- already working on it. > Add aes_{encrypt,decrypt

[jira] [Resolved] (SPARK-12549) UDFs' input type specification should take Option[Seq[DataType]] rather than Seq[DataType]

2015-12-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12549. - Resolution: Fixed Assignee: Reynold Xin (was: Apache Spark) Fix Version/s: 2.0.0

[jira] [Commented] (SPARK-12558) AnalysisException when multiple functions applied in GROUP BY clause

2015-12-29 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074508#comment-15074508 ] Dilip Biswal commented on SPARK-12558: -- I would like to work on this one. > Analysi

[jira] [Created] (SPARK-12570) DecisionTreeRegressor: provide variance of prediction: user guide update

2015-12-29 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-12570: - Summary: DecisionTreeRegressor: provide variance of prediction: user guide update Key: SPARK-12570 URL: https://issues.apache.org/jira/browse/SPARK-12570 Pr

[jira] [Created] (SPARK-12569) DecisionTreeRegressor: provide variance of prediction: Python API

2015-12-29 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-12569: - Summary: DecisionTreeRegressor: provide variance of prediction: Python API Key: SPARK-12569 URL: https://issues.apache.org/jira/browse/SPARK-12569 Project:

[jira] [Commented] (SPARK-12568) Add BINARY to Encoders

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074499#comment-15074499 ] Apache Spark commented on SPARK-12568: -- User 'marmbrus' has created a pull request f

[jira] [Created] (SPARK-12568) Add BINARY to Encoders

2015-12-29 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-12568: Summary: Add BINARY to Encoders Key: SPARK-12568 URL: https://issues.apache.org/jira/browse/SPARK-12568 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074496#comment-15074496 ] Apache Spark commented on SPARK-12511: -- User 'zsxwing' has created a pull request fo

[jira] [Created] (SPARK-12567) Add aes_{encrypt,decrypt} UDFs

2015-12-29 Thread Kai Jiang (JIRA)
Kai Jiang created SPARK-12567: - Summary: Add aes_{encrypt,decrypt} UDFs Key: SPARK-12567 URL: https://issues.apache.org/jira/browse/SPARK-12567 Project: Spark Issue Type: New Feature Co

[jira] [Resolved] (SPARK-12002) offsetRanges attribute missing in Kafka RDD when resuming from checkpoint

2015-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-12002. -- Resolution: Fixed Assignee: Saisai Shao Fix Version/s: 1.6.0 > offsetRanges att

[jira] [Resolved] (SPARK-12004) RDD checkpointing does not preserve partitioner

2015-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-12004. -- Resolution: Fixed Fix Version/s: 1.6.0 > RDD checkpointing does not preserve partitioner

[jira] [Commented] (SPARK-10638) spark streaming stop gracefully keeps the spark context

2015-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074467#comment-15074467 ] Shixiong Zhu commented on SPARK-10638: -- Do you have any stack trace or reproduce cod

[jira] [Comment Edited] (SPARK-12407) ClassCast Exception when restarting spark streaming from checkpoint

2015-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074461#comment-15074461 ] Shixiong Zhu edited comment on SPARK-12407 at 12/30/15 12:08 AM: --

[jira] [Commented] (SPARK-5159) Thrift server does not respect hive.server2.enable.doAs=true

2015-12-29 Thread Sen Fang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074462#comment-15074462 ] Sen Fang commented on SPARK-5159: - As of Spark 1.5.2, we have a similar issue that might b

[jira] [Resolved] (SPARK-12407) ClassCast Exception when restarting spark streaming from checkpoint

2015-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-12407. -- Resolution: Duplicate Fix Version/s: SPARK-5206 The cause is Streaming checkpointing doe

[jira] [Updated] (SPARK-12407) ClassCast Exception when restarting spark streaming from checkpoint

2015-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-12407: - Fix Version/s: (was: SPARK-5206) > ClassCast Exception when restarting spark streaming from c

[jira] [Commented] (SPARK-12562) DataFrame.write.format("text") requires the column name to be called value

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074456#comment-15074456 ] Apache Spark commented on SPARK-12562: -- User 'xguo27' has created a pull request for

[jira] [Commented] (SPARK-11745) Enable more JSON parsing options for parsing non-standard JSON files

2015-12-29 Thread Cazen Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074445#comment-15074445 ] Cazen Lee commented on SPARK-11745: --- Good Day [~rxin] This is Cazen I'm sorry for aski

[jira] [Assigned] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12511: Assignee: Apache Spark (was: Shixiong Zhu) > streaming driver with checkpointing unable t

[jira] [Assigned] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12511: Assignee: Shixiong Zhu (was: Apache Spark) > streaming driver with checkpointing unable t

[jira] [Assigned] (SPARK-12560) SqlTestUtils.stripSparkFilter needs to copy utf8strings

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12560: Assignee: Apache Spark > SqlTestUtils.stripSparkFilter needs to copy utf8strings > ---

[jira] [Assigned] (SPARK-12560) SqlTestUtils.stripSparkFilter needs to copy utf8strings

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12560: Assignee: (was: Apache Spark) > SqlTestUtils.stripSparkFilter needs to copy utf8string

[jira] [Commented] (SPARK-12511) streaming driver with checkpointing unable to finalize leading to OOM

2015-12-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074427#comment-15074427 ] Shixiong Zhu commented on SPARK-12511: -- I submitted a PR to py4j to fix this issue:

[jira] [Assigned] (SPARK-12560) SqlTestUtils.stripSparkFilter needs to copy utf8strings

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12560: Assignee: Apache Spark > SqlTestUtils.stripSparkFilter needs to copy utf8strings > ---

[jira] [Assigned] (SPARK-12560) SqlTestUtils.stripSparkFilter needs to copy utf8strings

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12560: Assignee: (was: Apache Spark) > SqlTestUtils.stripSparkFilter needs to copy utf8string

[jira] [Created] (SPARK-12566) GLM model family, link function support

2015-12-29 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-12566: - Summary: GLM model family, link function support Key: SPARK-12566 URL: https://issues.apache.org/jira/browse/SPARK-12566 Project: Spark Issue Type:

[jira] [Commented] (SPARK-12385) Push projection into Join

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074383#comment-15074383 ] Davies Liu commented on SPARK-12385: [~smilegator] Thanks for working on this, could

[jira] [Commented] (SPARK-11607) Update MLlib website for 1.6

2015-12-29 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074384#comment-15074384 ] Joseph K. Bradley commented on SPARK-11607: --- This one is probably OK and can be

[jira] [Commented] (SPARK-4819) Remove Guava's "Optional" from public API

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074380#comment-15074380 ] Apache Spark commented on SPARK-4819: - User 'srowen' has created a pull request for th

[jira] [Assigned] (SPARK-4819) Remove Guava's "Optional" from public API

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4819: --- Assignee: Apache Spark (was: Sean Owen) > Remove Guava's "Optional" from public API > --

[jira] [Assigned] (SPARK-4819) Remove Guava's "Optional" from public API

2015-12-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-4819: --- Assignee: Sean Owen (was: Apache Spark) > Remove Guava's "Optional" from public API > --

[jira] [Commented] (SPARK-12544) Support window functions in SQLContext

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074369#comment-15074369 ] Davies Liu commented on SPARK-12544: [~hvanhovell] Right now, the SQL parser can't p

[jira] [Commented] (SPARK-12542) Support union/intersect/except in Hive SQL

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074368#comment-15074368 ] Davies Liu commented on SPARK-12542: [~viirya] [~xiaol] That's true, updated the titl

[jira] [Updated] (SPARK-12542) Support union/intersect/except in Hive SQL

2015-12-29 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12542: --- Assignee: Xiao Li > Support union/intersect/except in Hive SQL >

  1   2   3   >