[jira] [Commented] (SPARK-12311) [CORE] Restore previous value of "os.arch" property in test suites after forcing to set specific value to "os.arch" property

2015-12-14 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056317#comment-15056317 ] Kazuaki Ishizaki commented on SPARK-12311: -- Yes. Even when I run core test on a ppc64le machine,

[jira] [Created] (SPARK-12311) [CORE] Restore previous "os.arch" property in test suites after forcing to set specific value to "os.arch" property

2015-12-13 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-12311: Summary: [CORE] Restore previous "os.arch" property in test suites after forcing to set specific value to "os.arch" property Key: SPARK-12311 URL:

[jira] [Updated] (SPARK-12311) [CORE] Restore previous value of "os.arch" property in test suites after forcing to set specific value to "os.arch" property

2015-12-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-12311: - Summary: [CORE] Restore previous value of "os.arch" property in test suites after

[jira] [Updated] (SPARK-12311) [CORE] Restore previous value of "os.arch" property in test suites after forcing to set specific value to "os.arch" property

2015-12-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-12311: - Description: Although current BlockManagerSuite.scala and SizeEstimatorSuite.scala set

[jira] [Commented] (SPARK-3785) Support off-loading computations to a GPU

2016-01-03 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15080653#comment-15080653 ] Kazuaki Ishizaki commented on SPARK-3785: - Let us reopen this thread :) We are working for

[jira] [Reopened] (SPARK-3785) Support off-loading computations to a GPU

2016-01-03 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki reopened SPARK-3785: - Add a comment of our prototype to offload to GPU > Support off-loading computations to a GPU

[jira] [Issue Comment Deleted] (SPARK-3785) Support off-loading computations to a GPU

2016-01-03 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-3785: Comment: was deleted (was: Add a comment of our prototype to offload to GPU) > Support

[jira] [Comment Edited] (SPARK-3785) Support off-loading computations to a GPU

2016-01-03 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15080653#comment-15080653 ] Kazuaki Ishizaki edited comment on SPARK-3785 at 1/4/16 3:44 AM: - Let us

[jira] [Created] (SPARK-12620) Proposal of GPU exploitation for Spark

2016-01-04 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-12620: Summary: Proposal of GPU exploitation for Spark Key: SPARK-12620 URL: https://issues.apache.org/jira/browse/SPARK-12620 Project: Spark Issue Type:

[jira] [Commented] (SPARK-3785) Support off-loading computations to a GPU

2016-01-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15081184#comment-15081184 ] Kazuaki Ishizaki commented on SPARK-3785: - I am sorry to reopen this. Based on your suggestion, I

[jira] [Commented] (SPARK-3785) Support off-loading computations to a GPU

2016-01-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082596#comment-15082596 ] Kazuaki Ishizaki commented on SPARK-3785: - [~rxin][~srowen] thank you for your directions, let us

[jira] [Closed] (SPARK-12620) Proposal of GPU exploitation for Spark

2016-01-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki closed SPARK-12620. Move to SPARK-3875. > Proposal of GPU exploitation for Spark >

[jira] [Commented] (SPARK-12620) Proposal of GPU exploitation for Spark

2016-01-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082589#comment-15082589 ] Kazuaki Ishizaki commented on SPARK-12620: -- I will close this, and continue a discussion at

[jira] [Commented] (SPARK-12620) Proposal of GPU exploitation for Spark

2016-01-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15081332#comment-15081332 ] Kazuaki Ishizaki commented on SPARK-12620: -- [~srowen] would it be better to continue discussion

[jira] [Commented] (SPARK-3785) Support off-loading computations to a GPU

2016-01-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082859#comment-15082859 ] Kazuaki Ishizaki commented on SPARK-3785: - To use CUDA is an intermediate approach to evaluate

[jira] [Commented] (SPARK-3785) Support off-loading computations to a GPU

2016-01-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15082881#comment-15082881 ] Kazuaki Ishizaki commented on SPARK-3785: - # You can specify cpu-cores by using conventional Spark

[jira] [Created] (SPARK-12502) Script /dev/run-tests fails when IBM Java is used

2015-12-23 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-12502: Summary: Script /dev/run-tests fails when IBM Java is used Key: SPARK-12502 URL: https://issues.apache.org/jira/browse/SPARK-12502 Project: Spark

[jira] [Commented] (SPARK-12502) Script /dev/run-tests fails when IBM Java is used

2015-12-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069991#comment-15069991 ] Kazuaki Ishizaki commented on SPARK-12502: -- Yes, I will do it. I will change a format of

[jira] [Created] (SPARK-12530) Build break at Spark-Master-Maven-Snapshots from #1293

2015-12-26 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-12530: Summary: Build break at Spark-Master-Maven-Snapshots from #1293 Key: SPARK-12530 URL: https://issues.apache.org/jira/browse/SPARK-12530 Project: Spark

[jira] [Updated] (SPARK-12530) Build break at Spark-Master-Maven-Snapshots from #1293

2015-12-26 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-12530: - Description: Build break happens at Spark-Master-Maven-Snapshots from #1293 due to

[jira] [Created] (SPARK-12580) Remove string concatenations from usage and extended in @ExpressionDescription

2015-12-29 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-12580: Summary: Remove string concatenations from usage and extended in @ExpressionDescription Key: SPARK-12580 URL: https://issues.apache.org/jira/browse/SPARK-12580

[jira] [Commented] (SPARK-12580) Remove string concatenations from usage and extended in @ExpressionDescription

2015-12-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074738#comment-15074738 ] Kazuaki Ishizaki commented on SPARK-12580: -- Thank you, however I have already done in my local

[jira] [Updated] (SPARK-12580) Remove string concatenations from usage and extended in @ExpressionDescription

2015-12-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-12580: - External issue ID: SPARK-12454, SPARK-12530,

[jira] [Updated] (SPARK-12580) Remove string concatenations from usage and extended in @ExpressionDescription

2015-12-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-12580: - External issue ID: SPARK-12454, SPARK-12530 (was: 12454, 12530) > Remove string

[jira] [Updated] (SPARK-12580) Remove string concatenations from usage and extended in @ExpressionDescription

2015-12-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-12580: - Description: More than one string concatenation in {{usage}} or {{extended}} for

[jira] [Comment Edited] (SPARK-15687) Columnar execution engine

2016-06-06 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316858#comment-15316858 ] Kazuaki Ishizaki edited comment on SPARK-15687 at 6/6/16 5:49 PM: -- Thank

[jira] [Commented] (SPARK-15687) Columnar execution engine

2016-06-06 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316858#comment-15316858 ] Kazuaki Ishizaki commented on SPARK-15687: -- Thank you for your answers * How we pass columnar

[jira] [Commented] (SPARK-15899) file scheme should be used correctly

2016-06-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15326537#comment-15326537 ] Kazuaki Ishizaki commented on SPARK-15899: -- Thank you for your comments. It is a little bit

[jira] [Created] (SPARK-15899) file scheme should be used correctly

2016-06-11 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-15899: Summary: file scheme should be used correctly Key: SPARK-15899 URL: https://issues.apache.org/jira/browse/SPARK-15899 Project: Spark Issue Type:

[jira] [Issue Comment Deleted] (SPARK-15899) file scheme should be used correctly

2016-06-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-15899: - Comment: was deleted (was: When I added the two extra slashes, it works on Linux. But,

[jira] [Commented] (SPARK-15899) file scheme should be used correctly

2016-06-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327824#comment-15327824 ] Kazuaki Ishizaki commented on SPARK-15899: -- When I added the two extra slashes, it works on

[jira] [Commented] (SPARK-15899) file scheme should be used correctly

2016-06-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15327825#comment-15327825 ] Kazuaki Ishizaki commented on SPARK-15899: -- When I added the two extra slashes, it works on

[jira] [Created] (SPARK-15950) Eliminate unreachable code at projection for complex types

2016-06-14 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-15950: Summary: Eliminate unreachable code at projection for complex types Key: SPARK-15950 URL: https://issues.apache.org/jira/browse/SPARK-15950 Project: Spark

[jira] [Created] (SPARK-15962) Introduce additonal implementation with a dense format for UnsafeArrayData

2016-06-15 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-15962: Summary: Introduce additonal implementation with a dense format for UnsafeArrayData Key: SPARK-15962 URL: https://issues.apache.org/jira/browse/SPARK-15962

[jira] [Commented] (SPARK-15687) Columnar execution engine

2016-06-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15310663#comment-15310663 ] Kazuaki Ishizaki commented on SPARK-15687: -- Thank you for creating interesting JIRA entry. Based

[jira] [Commented] (SPARK-16070) DataFrame/Parquet issues with primitive arrays

2016-06-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341408#comment-15341408 ] Kazuaki Ishizaki commented on SPARK-16070: -- [~mengxr], thank you for creating an umbrella. I

[jira] [Commented] (SPARK-16070) DataFrame/Parquet issues with primitive arrays

2016-06-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341427#comment-15341427 ] Kazuaki Ishizaki commented on SPARK-16070: -- Other JIRAs for DataFrame issues with primitive

[jira] [Comment Edited] (SPARK-16070) DataFrame/Parquet issues with primitive arrays

2016-06-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341408#comment-15341408 ] Kazuaki Ishizaki edited comment on SPARK-16070 at 6/21/16 9:22 AM: ---

[jira] [Comment Edited] (SPARK-16070) DataFrame/Parquet issues with primitive arrays

2016-06-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341427#comment-15341427 ] Kazuaki Ishizaki edited comment on SPARK-16070 at 6/21/16 9:22 AM: ---

[jira] [Commented] (SPARK-15467) Getting stack overflow when attempting to query a wide Dataset (>200 fields)

2016-06-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341376#comment-15341376 ] Kazuaki Ishizaki commented on SPARK-15467: -- Thank you for letting me know it. Now, it looks [an

[jira] [Created] (SPARK-16043) Prepare GenericArrayData implementation specialized for a primitive array

2016-06-18 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-16043: Summary: Prepare GenericArrayData implementation specialized for a primitive array Key: SPARK-16043 URL: https://issues.apache.org/jira/browse/SPARK-16043

[jira] [Created] (SPARK-16042) Eliminate nullcheck code at projection for an array type

2016-06-18 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-16042: Summary: Eliminate nullcheck code at projection for an array type Key: SPARK-16042 URL: https://issues.apache.org/jira/browse/SPARK-16042 Project: Spark

[jira] [Commented] (SPARK-15899) file scheme should be used correctly

2016-06-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345620#comment-15345620 ] Kazuaki Ishizaki commented on SPARK-15899: -- I think so. As [~sowen] proposed, we may need a

[jira] [Created] (SPARK-15985) Reduce runtime overhead of a program that reads an primitive array in Dataset

2016-06-16 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-15985: Summary: Reduce runtime overhead of a program that reads an primitive array in Dataset Key: SPARK-15985 URL: https://issues.apache.org/jira/browse/SPARK-15985

[jira] [Commented] (SPARK-15467) Getting stack overflow when attempting to query a wide Dataset (>200 fields)

2016-06-16 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15334443#comment-15334443 ] Kazuaki Ishizaki commented on SPARK-15467: -- We are waiting for author's review at

[jira] [Created] (SPARK-16213) Reduce runtime overhead of a program that creates an primitive array in DataFrame

2016-06-25 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-16213: Summary: Reduce runtime overhead of a program that creates an primitive array in DataFrame Key: SPARK-16213 URL: https://issues.apache.org/jira/browse/SPARK-16213

[jira] [Created] (SPARK-16215) Reduce runtime overhead of a program that writes an primitive array in Dataframe/Dataset

2016-06-25 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-16215: Summary: Reduce runtime overhead of a program that writes an primitive array in Dataframe/Dataset Key: SPARK-16215 URL: https://issues.apache.org/jira/browse/SPARK-16215

[jira] [Commented] (SPARK-16070) DataFrame/Parquet issues with primitive arrays

2016-06-26 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350005#comment-15350005 ] Kazuaki Ishizaki commented on SPARK-16070: -- I added two JIRA entries, which address DataFrame

[jira] [Created] (SPARK-16223) Codegen failure with a Dataframe program using an array

2016-06-27 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-16223: Summary: Codegen failure with a Dataframe program using an array Key: SPARK-16223 URL: https://issues.apache.org/jira/browse/SPARK-16223 Project: Spark

[jira] [Created] (SPARK-13437) Add InternalColumn

2016-02-22 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-13437: Summary: Add InternalColumn Key: SPARK-13437 URL: https://issues.apache.org/jira/browse/SPARK-13437 Project: Spark Issue Type: Improvement

[jira] [Closed] (SPARK-12907) Use BitSet to represent null fields in ColumnVector

2016-02-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki closed SPARK-12907. Resolution: Won't Fix > Use BitSet to represent null fields in ColumnVector >

[jira] [Commented] (SPARK-13421) Make output of a SparkPlan configurable

2016-02-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160157#comment-15160157 ] Kazuaki Ishizaki commented on SPARK-13421: -- I am willing to work for and contribute to this. Are

[jira] [Commented] (SPARK-13431) Maven build fails due to: Method code too large! in Catalyst

2016-02-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15159649#comment-15159649 ] Kazuaki Ishizaki commented on SPARK-13431: -- I identified why this problem occurs only in maven.

[jira] [Commented] (SPARK-13530) Add ShortType support to UnsafeRowParquetRecordReader

2016-02-27 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15170636#comment-15170636 ] Kazuaki Ishizaki commented on SPARK-13530: -- I saw the same exception in my environment. Thank

[jira] [Commented] (SPARK-13431) Maven build fails due to: Method code too large! in Catalyst

2016-02-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157104#comment-15157104 ] Kazuaki Ishizaki commented on SPARK-13431: -- According to AMPLAB's Jenkins, the first failure

[jira] [Commented] (SPARK-13431) Maven build fails due to: Method code too large! in Catalyst

2016-02-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157208#comment-15157208 ] Kazuaki Ishizaki commented on SPARK-13431: -- I am using mvn, and executed the following command

[jira] [Commented] (SPARK-13431) Maven build fails due to: Method code too large! in Catalyst

2016-02-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157182#comment-15157182 ] Kazuaki Ishizaki commented on SPARK-13431: -- I succeeded to build Spark by commenting out lines

[jira] [Comment Edited] (SPARK-13431) Maven build fails due to: Method code too large! in Catalyst

2016-02-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157297#comment-15157297 ] Kazuaki Ishizaki edited comment on SPARK-13431 at 2/22/16 5:17 PM: --- The

[jira] [Commented] (SPARK-13431) Maven build fails due to: Method code too large! in Catalyst

2016-02-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157297#comment-15157297 ] Kazuaki Ishizaki commented on SPARK-13431: -- The size of a static initializer method of

[jira] [Created] (SPARK-12907) Use BitSet to represent null fields in ColumnVector

2016-01-19 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-12907: Summary: Use BitSet to represent null fields in ColumnVector Key: SPARK-12907 URL: https://issues.apache.org/jira/browse/SPARK-12907 Project: Spark

[jira] [Created] (SPARK-13432) Add the origin of the source code into a generated Java code

2016-02-22 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-13432: Summary: Add the origin of the source code into a generated Java code Key: SPARK-13432 URL: https://issues.apache.org/jira/browse/SPARK-13432 Project: Spark

[jira] [Created] (SPARK-13805) Direct consume ColumnVector in generated code when ColumnarBatch is used

2016-03-10 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-13805: Summary: Direct consume ColumnVector in generated code when ColumnarBatch is used Key: SPARK-13805 URL: https://issues.apache.org/jira/browse/SPARK-13805

[jira] [Created] (SPARK-13844) Generate better code for filters with a non-nullable column

2016-03-12 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-13844: Summary: Generate better code for filters with a non-nullable column Key: SPARK-13844 URL: https://issues.apache.org/jira/browse/SPARK-13844 Project: Spark

[jira] [Commented] (SPARK-14083) Analyze JVM bytecode and turn closures into Catalyst expressions

2016-03-31 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15219637#comment-15219637 ] Kazuaki Ishizaki commented on SPARK-14083: -- Looks interesting. I am also interested in

[jira] [Created] (SPARK-14072) Show JVM information when we run Benchmark

2016-03-22 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-14072: Summary: Show JVM information when we run Benchmark Key: SPARK-14072 URL: https://issues.apache.org/jira/browse/SPARK-14072 Project: Spark Issue

[jira] [Created] (SPARK-14098) Generate code that get a value in each column from CachedBatch when DataFrame.cache() is called

2016-03-23 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-14098: Summary: Generate code that get a value in each column from CachedBatch when DataFrame.cache() is called Key: SPARK-14098 URL:

[jira] [Comment Edited] (SPARK-14098) Generate code that get a value in each column from CachedBatch when DataFrame.cache() is called

2016-03-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15209047#comment-15209047 ] Kazuaki Ishizaki edited comment on SPARK-14098 at 3/23/16 8:02 PM: ---

[jira] [Commented] (SPARK-14098) Generate code that get a value in each column from CachedBatch when DataFrame.cache() is called

2016-03-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15209047#comment-15209047 ] Kazuaki Ishizaki commented on SPARK-14098: -- After fixing some trivial compilation failures, I

[jira] [Commented] (SPARK-14138) Generated SpecificColumnarIterator code can exceed JVM size limit for cached DataFrames

2016-03-26 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213071#comment-15213071 ] Kazuaki Ishizaki commented on SPARK-14138: -- I will make a PR that includes two solutions: 1.

[jira] [Created] (SPARK-13644) [SQL] add the source file name and line into Logger when an exception occurs in the generated code

2016-03-03 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-13644: Summary: [SQL] add the source file name and line into Logger when an exception occurs in the generated code Key: SPARK-13644 URL:

[jira] [Updated] (SPARK-13644) Add the source file name and line into Logger when an exception occurs in the generated code

2016-03-03 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-13644: - Summary: Add the source file name and line into Logger when an exception occurs in the

[jira] [Updated] (SPARK-13644) add the source file name and line into Logger when an exception occurs in the generated code

2016-03-03 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-13644: - Issue Type: Improvement (was: Bug) Summary: add the source file name and line

[jira] [Commented] (SPARK-14083) Analyze JVM bytecode and turn closures into Catalyst expressions

2016-03-31 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15220055#comment-15220055 ] Kazuaki Ishizaki commented on SPARK-14083: -- Thank you. Makes sense to me. It seems that this

[jira] [Commented] (SPARK-14083) Analyze JVM bytecode and turn closures into Catalyst expressions

2016-03-28 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15213959#comment-15213959 ] Kazuaki Ishizaki commented on SPARK-14083: -- Another possible bytecode analysis library is WALA

[jira] [Commented] (SPARK-14083) Analyze JVM bytecode and turn closures into Catalyst expressions

2016-04-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15222634#comment-15222634 ] Kazuaki Ishizaki commented on SPARK-14083: -- Good point. On the other hand, should we follow JVM

[jira] [Updated] (SPARK-14098) Generate code that get a value in each column from CachedBatch when DataFrame.cache() is called

2016-04-25 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-14098: - Target Version/s: 2.0.0 > Generate code that get a value in each column from CachedBatch

[jira] [Commented] (SPARK-14138) Generated SpecificColumnarIterator code can exceed JVM size limit for cached DataFrames

2016-04-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254061#comment-15254061 ] Kazuaki Ishizaki commented on SPARK-14138: -- Yes, it will be included in 1.6.2 and 2.0.0. >

[jira] [Commented] (SPARK-15467) Getting stack overflow when attempting to query a wide Dataset (>200 fields)

2016-05-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296743#comment-15296743 ] Kazuaki Ishizaki commented on SPARK-15467: -- The following code can reproduce the same issue.

[jira] [Commented] (SPARK-15467) Getting stack overflow when attempting to query a wide Dataset (>200 fields)

2016-05-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296929#comment-15296929 ] Kazuaki Ishizaki commented on SPARK-15467: -- I created test cases to reproduce this problem and a

[jira] [Comment Edited] (SPARK-15467) Getting stack overflow when attempting to query a wide Dataset (>200 fields)

2016-05-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296743#comment-15296743 ] Kazuaki Ishizaki edited comment on SPARK-15467 at 5/24/16 3:54 AM: --- The

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295014#comment-15295014 ] Kazuaki Ishizaki commented on SPARK-15285: -- I see, I started doing this. > Generated

[jira] [Commented] (SPARK-15467) Getting stack overflow when attempting to query a wide Dataset (>200 fields)

2016-05-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296102#comment-15296102 ] Kazuaki Ishizaki commented on SPARK-15467: -- I will try to submit a patch against Janino. >

[jira] [Commented] (SPARK-15467) Getting stack overflow when attempting to query a wide Dataset (>200 fields)

2016-05-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295499#comment-15295499 ] Kazuaki Ishizaki commented on SPARK-15467: -- IMHO, this issue is due to Janino's implementation.

[jira] [Commented] (SPARK-15467) Getting stack overflow when attempting to query a wide Dataset (>200 fields)

2016-05-24 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297784#comment-15297784 ] Kazuaki Ishizaki commented on SPARK-15467: -- I submitted [a pull

[jira] [Commented] (SPARK-15258) Nested/Chained case statements generate codegen over 64k exception

2016-05-24 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297896#comment-15297896 ] Kazuaki Ishizaki commented on SPARK-15258: -- {{commit d642b273544bb77ef7f584326aa2d214649ac61b}}

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295299#comment-15295299 ] Kazuaki Ishizaki commented on SPARK-15285: -- I created a

[jira] [Comment Edited] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295299#comment-15295299 ] Kazuaki Ishizaki edited comment on SPARK-15285 at 5/21/16 11:12 PM: I

[jira] [Comment Edited] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295299#comment-15295299 ] Kazuaki Ishizaki edited comment on SPARK-15285 at 5/21/16 11:33 PM: I

[jira] [Commented] (SPARK-15258) Nested/Chained case statements generate codegen over 64k exception

2016-05-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295402#comment-15295402 ] Kazuaki Ishizaki commented on SPARK-15258: -- This PR is not for SPARK-15258. This is for

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294494#comment-15294494 ] Kazuaki Ishizaki commented on SPARK-15285: -- I can take it today if they are busy. > Generated

[jira] [Created] (SPARK-15380) Generate code that stores a float/double value in each column from ColumnarBatch when DataFrame.cache() is used

2016-05-18 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-15380: Summary: Generate code that stores a float/double value in each column from ColumnarBatch when DataFrame.cache() is used Key: SPARK-15380 URL:

[jira] [Commented] (SPARK-14082) Add support for GPU resource when running on Mesos

2016-05-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15281371#comment-15281371 ] Kazuaki Ishizaki commented on SPARK-14082: -- Sounds great. We expect that our

[jira] [Created] (SPARK-15117) Generate code that get a value in each compressed column from CachedBatch when DataFrame.cache() is called

2016-05-04 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-15117: Summary: Generate code that get a value in each compressed column from CachedBatch when DataFrame.cache() is called Key: SPARK-15117 URL:

[jira] [Updated] (SPARK-15117) Generate code that get a value in each compressed column from CachedBatch when DataFrame.cache() is called

2016-05-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-15117: - Target Version/s: 2.1.0 > Generate code that get a value in each compressed column from

[jira] [Updated] (SPARK-14098) Generate code that get a float/double value in each column from CachedBatch when DataFrame.cache() is called

2016-05-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-14098: - Description: When DataFrame.cache() is called, data is stored as column-oriented storage

[jira] [Updated] (SPARK-14098) Generate code that get a float/double value in each column from CachedBatch when DataFrame.cache() is called

2016-05-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-14098: - Summary: Generate code that get a float/double value in each column from CachedBatch

[jira] [Closed] (SPARK-14690) [SQL] SPARK-8020 fails in Jenkins for master

2016-04-17 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki closed SPARK-14690. Add a link to the original JIRA > [SQL] SPARK-8020 fails in Jenkins for master >

[jira] [Commented] (SPARK-14690) [SQL] SPARK-8020 fails in Jenkins for master

2016-04-17 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15244798#comment-15244798 ] Kazuaki Ishizaki commented on SPARK-14690: -- I see. I will reopen the original JIRA soon. >

[jira] [Created] (SPARK-14690) [SQL] SPARK-9757 fails in Jenkins for master

2016-04-17 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-14690: Summary: [SQL] SPARK-9757 fails in Jenkins for master Key: SPARK-14690 URL: https://issues.apache.org/jira/browse/SPARK-14690 Project: Spark Issue

[jira] [Issue Comment Deleted] (SPARK-14690) [SQL] SPARK-8020 fails in Jenkins for master

2016-04-17 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-14690: - Comment: was deleted (was: Add a link to the original JIRA) > [SQL] SPARK-8020 fails in

  1   2   3   4   5   6   7   8   >