[jira] [Created] (SPARK-9401) Fully implement code generation for ConcatWs

2015-07-27 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-9401: -- Summary: Fully implement code generation for ConcatWs Key: SPARK-9401 URL: https://issues.apache.org/jira/browse/SPARK-9401 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-9379) org.apache.spark.sql.catalyst.SqlParser should be extensible easily

2015-07-27 Thread Rishi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643951#comment-14643951 ] Rishi commented on SPARK-9379: -- Thanks Michael for the reply. Precisely for plug-able SQL dia

[jira] [Created] (SPARK-9400) Implement code generation for StringLocate

2015-07-27 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-9400: -- Summary: Implement code generation for StringLocate Key: SPARK-9400 URL: https://issues.apache.org/jira/browse/SPARK-9400 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-9362) Exception when using DataFrame groupby().sum on Decimal type in Python

2015-07-27 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643942#comment-14643942 ] Davies Liu commented on SPARK-9362: --- Could you test this with master? I think master al

[jira] [Resolved] (SPARK-9373) Support StructType in Tungsten style Projection

2015-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-9373. Resolution: Fixed Fix Version/s: 1.5.0 > Support StructType in Tungsten style Projection > --

[jira] [Closed] (SPARK-9399) Assorted micro-optimizations in PythonRDD

2015-07-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen closed SPARK-9399. - Resolution: Won't Fix > Assorted micro-optimizations in PythonRDD > --

[jira] [Commented] (SPARK-9373) Support StructType in Tungsten style Projection

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643929#comment-14643929 ] Apache Spark commented on SPARK-9373: - User 'rxin' has created a pull request for this

[jira] [Updated] (SPARK-9399) Assorted micro-optimizations in PythonRDD

2015-07-27 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-9399: -- Description: There are a few minor optimizations in PythonRDD which may avoid garbage creation or Scala

[jira] [Assigned] (SPARK-9399) Assorted micro-optimizations in PythonRDD

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9399: --- Assignee: Apache Spark (was: Josh Rosen) > Assorted micro-optimizations in PythonRDD > -

[jira] [Assigned] (SPARK-9399) Assorted micro-optimizations in PythonRDD

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9399: --- Assignee: Josh Rosen (was: Apache Spark) > Assorted micro-optimizations in PythonRDD > -

[jira] [Commented] (SPARK-9399) Assorted micro-optimizations in PythonRDD

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643921#comment-14643921 ] Apache Spark commented on SPARK-9399: - User 'JoshRosen' has created a pull request for

[jira] [Commented] (SPARK-9333) Add `registerTable` method to hivecontext

2015-07-27 Thread Kuldeep (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643920#comment-14643920 ] Kuldeep commented on SPARK-9333: For persisting past life of spark context you can either

[jira] [Commented] (SPARK-9398) Add unit test for null inputs for date functions

2015-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643919#comment-14643919 ] Reynold Xin commented on SPARK-9398: cc [~adrian-wang] When you add new expressions,

[jira] [Created] (SPARK-9399) Assorted micro-optimizations in PythonRDD

2015-07-27 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-9399: - Summary: Assorted micro-optimizations in PythonRDD Key: SPARK-9399 URL: https://issues.apache.org/jira/browse/SPARK-9399 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-9398) Add unit test for null inputs for date functions

2015-07-27 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-9398: -- Summary: Add unit test for null inputs for date functions Key: SPARK-9398 URL: https://issues.apache.org/jira/browse/SPARK-9398 Project: Spark Issue Type: Sub-ta

[jira] [Commented] (SPARK-7503) Resources in .sparkStaging directory can't be cleaned up on error

2015-07-27 Thread Sumit Nigam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643918#comment-14643918 ] Sumit Nigam commented on SPARK-7503: For spark on yarn, would yarn not remove all loca

[jira] [Comment Edited] (SPARK-7503) Resources in .sparkStaging directory can't be cleaned up on error

2015-07-27 Thread Sumit Nigam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643918#comment-14643918 ] Sumit Nigam edited comment on SPARK-7503 at 7/28/15 6:21 AM: -

[jira] [Updated] (SPARK-9398) Add unit test for null inputs for date functions

2015-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9398: --- Description: Unit tests in DateExpressionsSuite didn't cover null inputs. For each expression, we sho

[jira] [Commented] (SPARK-7439) Should delete temporary local directories

2015-07-27 Thread Sumit Nigam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643916#comment-14643916 ] Sumit Nigam commented on SPARK-7439: For spark on YARN, would YARN not remove all loca

[jira] [Issue Comment Deleted] (SPARK-7439) Should delete temporary local directories

2015-07-27 Thread Sumit Nigam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumit Nigam updated SPARK-7439: --- Comment: was deleted (was: For spark on YARN, would YARN not remove all local directories created by

[jira] [Commented] (SPARK-8196) date/time function: next_day

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643914#comment-14643914 ] Apache Spark commented on SPARK-8196: - User 'rxin' has created a pull request for this

[jira] [Resolved] (SPARK-8828) Revert the change of SPARK-5680

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-8828. - Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7667 [https:/

[jira] [Assigned] (SPARK-9397) DataFrame should provide an API to find source data files if applicable

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9397: --- Assignee: Aaron Davidson (was: Apache Spark) > DataFrame should provide an API to find sourc

[jira] [Commented] (SPARK-9397) DataFrame should provide an API to find source data files if applicable

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643893#comment-14643893 ] Apache Spark commented on SPARK-9397: - User 'aarondav' has created a pull request for

[jira] [Assigned] (SPARK-9397) DataFrame should provide an API to find source data files if applicable

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9397: --- Assignee: Apache Spark (was: Aaron Davidson) > DataFrame should provide an API to find sourc

[jira] [Updated] (SPARK-9397) DataFrame should provide an API to find source data files if applicable

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-9397: Target Version/s: 1.5.0 Priority: Critical (was: Major) > DataFrame should prov

[jira] [Created] (SPARK-9397) DataFrame should provide an API to find source data files if applicable

2015-07-27 Thread Aaron Davidson (JIRA)
Aaron Davidson created SPARK-9397: - Summary: DataFrame should provide an API to find source data files if applicable Key: SPARK-9397 URL: https://issues.apache.org/jira/browse/SPARK-9397 Project: Spar

[jira] [Commented] (SPARK-9345) Failure to cleanup on exceptions causes persistent I/O problems later on

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643882#comment-14643882 ] Michael Armbrust commented on SPARK-9345: - We changed the way that we instantiate

[jira] [Commented] (SPARK-9345) Failure to cleanup on exceptions causes persistent I/O problems later on

2015-07-27 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643874#comment-14643874 ] Simeon Simeonov commented on SPARK-9345: [~skp33][~marmbrus] How does the absence

[jira] [Resolved] (SPARK-6648) Reading Parquet files with different sub-files doesn't work

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6648. - Resolution: Fixed I think this has been fixed by a rewrite of our parquet functionality.

[jira] [Resolved] (SPARK-7273) The SQLContext.jsonFile() api has a problem when load a format json file?

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-7273. - Resolution: Cannot Reproduce > The SQLContext.jsonFile() api has a problem when load a for

[jira] [Resolved] (SPARK-6984) Operations on tables with many partitions _very_slow

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6984. - Resolution: Duplicate Fix Version/s: 1.5.0 This should be fixed by [SPARK-6910]. >

[jira] [Updated] (SPARK-8465) ScalaReflectionException with DataFrames in 1.4

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-8465: Description: We are seeing class exceptions when converting to a DataFrame. Anyone out ther

[jira] [Resolved] (SPARK-8465) ScalaReflectionException with DataFrames in 1.4

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-8465. - Resolution: Duplicate Fix Version/s: 1.4.1 1.5.0 I believe this

[jira] [Resolved] (SPARK-8679) remove dummy java class BaseRow and BaseMutableRow

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-8679. - Resolution: Fixed Fixed by [SPARK-8610] > remove dummy java class BaseRow and BaseMutable

[jira] [Commented] (SPARK-8658) AttributeReference equals method only compare name, exprId and dataType

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643860#comment-14643860 ] Michael Armbrust commented on SPARK-8658: - +1. Would you like to submit a PR? >

[jira] [Resolved] (SPARK-8955) Replace a duplicated initialize() in HiveGenericUDTF with new one

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-8955. - Resolution: Won't Fix > Replace a duplicated initialize() in HiveGenericUDTF with new one

[jira] [Resolved] (SPARK-9181) round bracket used in naming aggregations is not allowed by parquet writer

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-9181. - Resolution: Won't Fix I'm sorry, but I don't think there is much we can do here as the lim

[jira] [Updated] (SPARK-9228) Adjust Spark SQL Configs

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-9228: Target Version/s: 1.5.0 > Adjust Spark SQL Configs > > >

[jira] [Updated] (SPARK-9255) SQL codegen fails with "value < is not a member of TimestampType.this.InternalType"

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-9255: Target Version/s: 1.5.0 > SQL codegen fails with "value < is not a member of > TimestampTyp

[jira] [Commented] (SPARK-9275) IsolatedClientLoader could not load shared JNI libraries

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643853#comment-14643853 ] Michael Armbrust commented on SPARK-9275: - Have you tried the solution suggested i

[jira] [Commented] (SPARK-9338) Aliases from SELECT not available in GROUP BY

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643852#comment-14643852 ] Michael Armbrust commented on SPARK-9338: - This is optimized internally and the ex

[jira] [Resolved] (SPARK-9347) spark load of existing parquet files extremely slow if large number of files

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-9347. - Resolution: Duplicate This should be fixed by [SPARK-8125] > spark load of existing parqu

[jira] [Commented] (SPARK-9379) org.apache.spark.sql.catalyst.SqlParser should be extensible easily

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643850#comment-14643850 ] Michael Armbrust commented on SPARK-9379: - This is a private API that is not inten

[jira] [Resolved] (SPARK-9345) Failure to cleanup on exceptions causes persistent I/O problems later on

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-9345. - Resolution: Not A Problem > Failure to cleanup on exceptions causes persistent I/O problem

[jira] [Commented] (SPARK-9345) Failure to cleanup on exceptions causes persistent I/O problems later on

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643849#comment-14643849 ] Michael Armbrust commented on SPARK-9345: - {{file:/user/hive/warehouse/test}} is n

[jira] [Resolved] (SPARK-9280) New HiveContext object unexpectedly loads configuration settings from history

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-9280. - Resolution: Won't Fix As stated in the other comment, this is because configuration is sto

[jira] [Resolved] (SPARK-9283) Join on Spark DataFrame with multiple columns

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-9283. - Resolution: Duplicate You need to use {{&}} instead of {{and}}. We throw an error stating

[jira] [Resolved] (SPARK-9282) Filter on Spark DataFrame with multiple columns

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-9282. - Resolution: Duplicate You need to use {{&}} instead of {{and}}. In Spark 1.4 we print an

[jira] [Commented] (SPARK-9333) Add `registerTable` method to hivecontext

2015-07-27 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643844#comment-14643844 ] Michael Armbrust commented on SPARK-9333: - This is difficult because RDDs can be c

[jira] [Resolved] (SPARK-9395) Create a SpecializedGetters interface to track all the specialized getters

2015-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-9395. Resolution: Fixed Fix Version/s: 1.5.0 > Create a SpecializedGetters interface to track all t

[jira] [Comment Edited] (SPARK-9396) Spark yarn allocator does not call "removeContainerRequest" for allocated Container requests, resulting in bloated ask[] toYarn RM.

2015-07-27 Thread prakhar jauhari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643822#comment-14643822 ] prakhar jauhari edited comment on SPARK-9396 at 7/28/15 4:18 AM: ---

[jira] [Commented] (SPARK-9396) Spark yarn allocator does not call "removeContainerRequest" for allocated Container requests, resulting in bloated ask[] toYarn RM.

2015-07-27 Thread prakhar jauhari (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643822#comment-14643822 ] prakhar jauhari commented on SPARK-9396: This is because Yarn's AM client does not

[jira] [Commented] (SPARK-9375) The total number of executor(s) requested by the driver may be negative

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643821#comment-14643821 ] Apache Spark commented on SPARK-9375: - User 'KaiXinXiaoLei' has created a pull request

[jira] [Resolved] (SPARK-8195) date/time function: last_day

2015-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-8195. Resolution: Fixed Assignee: Adrian Wang Fix Version/s: 1.5.0 > date/time function: l

[jira] [Resolved] (SPARK-8196) date/time function: next_day

2015-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-8196. Resolution: Fixed Assignee: Adrian Wang Fix Version/s: 1.5.0 > date/time function: n

[jira] [Reopened] (SPARK-9375) The total number of executor(s) requested by the driver may be negative

2015-07-27 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei reopened SPARK-9375: -- I test and find, this issue is exist and is different of https://issues.apache.org/jira/browse/SPAR

[jira] [Commented] (SPARK-8712) Hive's Parser does not support distinct aggregations with OVER clause

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643818#comment-14643818 ] Apache Spark commented on SPARK-8712: - User 'hvanhovell' has created a pull request fo

[jira] [Assigned] (SPARK-8712) Hive's Parser does not support distinct aggregations with OVER clause

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8712: --- Assignee: (was: Apache Spark) > Hive's Parser does not support distinct aggregations with

[jira] [Assigned] (SPARK-8641) Native Spark Window Functions

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8641: --- Assignee: Apache Spark > Native Spark Window Functions > - > >

[jira] [Assigned] (SPARK-8641) Native Spark Window Functions

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8641: --- Assignee: (was: Apache Spark) > Native Spark Window Functions > -

[jira] [Commented] (SPARK-8641) Native Spark Window Functions

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643817#comment-14643817 ] Apache Spark commented on SPARK-8641: - User 'hvanhovell' has created a pull request fo

[jira] [Assigned] (SPARK-8712) Hive's Parser does not support distinct aggregations with OVER clause

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8712: --- Assignee: Apache Spark > Hive's Parser does not support distinct aggregations with OVER claus

[jira] [Created] (SPARK-9396) Spark yarn allocator does not call "removeContainerRequest" for allocated Container requests, resulting in bloated ask[] toYarn RM.

2015-07-27 Thread prakhar jauhari (JIRA)
prakhar jauhari created SPARK-9396: -- Summary: Spark yarn allocator does not call "removeContainerRequest" for allocated Container requests, resulting in bloated ask[] toYarn RM. Key: SPARK-9396 URL: https://issu

[jira] [Updated] (SPARK-9375) The total number of executor(s) requested by the driver may be negative

2015-07-27 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-9375: - Attachment: The total number of executor(s) is negative in AM log.png In ApplicationMaster log, th

[jira] [Updated] (SPARK-9375) The total number of executor(s) requested by the driver may be negative

2015-07-27 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-9375: - Summary: The total number of executor(s) requested by the driver may be negative (was: The tota

[jira] [Updated] (SPARK-9375) The total number of executor(s) requested by the driver maybe negative

2015-07-27 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-9375: - Summary: The total number of executor(s) requested by the driver maybe negative (was: the total

[jira] [Updated] (SPARK-9375) The total number of executor(s) is negative

2015-07-27 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-9375: - Summary: The total number of executor(s) is negative (was: The total number of executor(s) is neg

[jira] [Updated] (SPARK-9375) the total number of executor(s) requested by the driver is negative

2015-07-27 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-9375: - Summary: the total number of executor(s) requested by the driver is negative (was: The total nu

[jira] [Updated] (SPARK-9375) The total number of executor(s) is negative

2015-07-27 Thread KaiXinXIaoLei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KaiXinXIaoLei updated SPARK-9375: - Description: I set "spark.dynamicAllocation.enabled = true”. I run a big job. I find a problem in

[jira] [Assigned] (SPARK-9202) Worker should not have data structures which grow without bound

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9202: --- Assignee: (was: Apache Spark) > Worker should not have data structures which grow without

[jira] [Assigned] (SPARK-9202) Worker should not have data structures which grow without bound

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9202: --- Assignee: Apache Spark > Worker should not have data structures which grow without bound > --

[jira] [Commented] (SPARK-9202) Worker should not have data structures which grow without bound

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643806#comment-14643806 ] Apache Spark commented on SPARK-9202: - User 'CodingCat' has created a pull request for

[jira] [Commented] (SPARK-7264) SparkR API for parallel functions

2015-07-27 Thread Jeff Allen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643798#comment-14643798 ] Jeff Allen commented on SPARK-7264: --- I too added a round of comments to the document --

[jira] [Assigned] (SPARK-5561) Generalize PeriodicGraphCheckpointer for RDDs

2015-07-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-5561: Assignee: Joseph K. Bradley > Generalize PeriodicGraphCheckpointer for RDDs > -

[jira] [Commented] (SPARK-5561) Generalize PeriodicGraphCheckpointer for RDDs

2015-07-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643797#comment-14643797 ] Joseph K. Bradley commented on SPARK-5561: -- I'm generalizing this to use for GBTs

[jira] [Updated] (SPARK-5561) Generalize PeriodicGraphCheckpointer for RDDs

2015-07-27 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-5561: - Target Version/s: 1.5.0 > Generalize PeriodicGraphCheckpointer for RDDs >

[jira] [Updated] (SPARK-9390) Create an array abstract class ArrayData and a default implementation backed by Array[Object]

2015-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9390: --- Description: {code} interface ArrayData implements SpecializedGetters { int numElements(); int siz

[jira] [Updated] (SPARK-9390) Create an array abstract class ArrayData and a default implementation backed by Array[Object]

2015-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9390: --- Summary: Create an array abstract class ArrayData and a default implementation backed by Array[Object]

[jira] [Updated] (SPARK-9390) Create an array abstract class ArrayData

2015-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9390: --- Description: {code} interface ArrayData implements SpecializedGetters { int numElements(); int siz

[jira] [Commented] (SPARK-9395) Create a SpecializedGetters interface to track all the specialized getters

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643792#comment-14643792 ] Apache Spark commented on SPARK-9395: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-9395) Create a SpecializedGetters interface to track all the specialized getters

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9395: --- Assignee: Reynold Xin (was: Apache Spark) > Create a SpecializedGetters interface to track a

[jira] [Assigned] (SPARK-9395) Create a SpecializedGetters interface to track all the specialized getters

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9395: --- Assignee: Apache Spark (was: Reynold Xin) > Create a SpecializedGetters interface to track a

[jira] [Updated] (SPARK-9395) Create a SpecializedGetters interface to track all the specialized getters

2015-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9395: --- Description: As we are adding more and more specialized getters to more classes, we should have an in

[jira] [Updated] (SPARK-9390) Create an array abstract class ArrayData

2015-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9390: --- Description: {code} interface ArrayData implements SpecializedGetters { public int numElements();

[jira] [Updated] (SPARK-9395) Create a SpecializedGetters interface to track all the specialized getters

2015-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9395: --- Description: As we are adding more and more specialized getters to more classes, we should have an in

[jira] [Created] (SPARK-9395) Create a SpecializedGetters interface to track all the specialized getters

2015-07-27 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-9395: -- Summary: Create a SpecializedGetters interface to track all the specialized getters Key: SPARK-9395 URL: https://issues.apache.org/jira/browse/SPARK-9395 Project: Spark

[jira] [Updated] (SPARK-9389) Support ArrayType in Tungsten

2015-07-27 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9389: --- Summary: Support ArrayType in Tungsten (was: Support array type in Tungsten) > Support ArrayType in

[jira] [Commented] (SPARK-9394) CodeFormatter should handle parentheses

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643778#comment-14643778 ] Apache Spark commented on SPARK-9394: - User 'rxin' has created a pull request for this

[jira] [Assigned] (SPARK-9394) CodeFormatter should handle parentheses

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9394: --- Assignee: Reynold Xin (was: Apache Spark) > CodeFormatter should handle parentheses > --

[jira] [Assigned] (SPARK-9394) CodeFormatter should handle parentheses

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9394: --- Assignee: Apache Spark (was: Reynold Xin) > CodeFormatter should handle parentheses > --

[jira] [Commented] (SPARK-9335) Kinesis test hits rate limit

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643777#comment-14643777 ] Apache Spark commented on SPARK-9335: - User 'zsxwing' has created a pull request for t

[jira] [Created] (SPARK-9394) CodeFormatter should handle parentheses

2015-07-27 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-9394: -- Summary: CodeFormatter should handle parentheses Key: SPARK-9394 URL: https://issues.apache.org/jira/browse/SPARK-9394 Project: Spark Issue Type: Bug C

[jira] [Assigned] (SPARK-9393) Fix several error-handling bugs in ScriptTransform operator

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9393: --- Assignee: Josh Rosen (was: Apache Spark) > Fix several error-handling bugs in ScriptTransfor

[jira] [Commented] (SPARK-9393) Fix several error-handling bugs in ScriptTransform operator

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643774#comment-14643774 ] Apache Spark commented on SPARK-9393: - User 'JoshRosen' has created a pull request for

[jira] [Assigned] (SPARK-9393) Fix several error-handling bugs in ScriptTransform operator

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9393: --- Assignee: Apache Spark (was: Josh Rosen) > Fix several error-handling bugs in ScriptTransfor

[jira] [Created] (SPARK-9393) Fix several error-handling bugs in ScriptTransform operator

2015-07-27 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-9393: - Summary: Fix several error-handling bugs in ScriptTransform operator Key: SPARK-9393 URL: https://issues.apache.org/jira/browse/SPARK-9393 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-8266) string function: translate

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8266: --- Assignee: Apache Spark (was: Cheng Hao) > string function: translate > -

[jira] [Commented] (SPARK-8266) string function: translate

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643766#comment-14643766 ] Apache Spark commented on SPARK-8266: - User 'zhichao-li' has created a pull request fo

[jira] [Assigned] (SPARK-8266) string function: translate

2015-07-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8266: --- Assignee: Cheng Hao (was: Apache Spark) > string function: translate > -

  1   2   3   >