[jira] [Comment Edited] (SPARK-23415) BufferHolderSparkSubmitSuite is flaky

2018-02-15 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365931#comment-16365931 ] Kazuaki Ishizaki edited comment on SPARK-23415 at 2/15/18 5:02 PM: --- I

[jira] [Commented] (SPARK-23427) spark.sql.autoBroadcastJoinThreshold causing OOM in the driver

2018-02-14 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365065#comment-16365065 ] Kazuaki Ishizaki commented on SPARK-23427: -- We would appreciate it if you could post a program

[jira] [Commented] (SPARK-23415) BufferHolderSparkSubmitSuite is flaky

2018-02-14 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364682#comment-16364682 ] Kazuaki Ishizaki commented on SPARK-23415: -- I am working for this. >

[jira] [Created] (SPARK-23424) Add codegenStageId in comment

2018-02-14 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-23424: Summary: Add codegenStageId in comment Key: SPARK-23424 URL: https://issues.apache.org/jira/browse/SPARK-23424 Project: Spark Issue Type:

[jira] [Commented] (SPARK-23310) Perf regression introduced by SPARK-21113

2018-02-10 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359429#comment-16359429 ] Kazuaki Ishizaki commented on SPARK-23310: -- got it, thanks > Perf regression introduced by

[jira] [Commented] (SPARK-23309) Spark 2.3 cached query performance 20-30% worse then spark 2.2

2018-02-08 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357441#comment-16357441 ] Kazuaki Ishizaki commented on SPARK-23309: -- When there is a repro, I am happy to investigate the

[jira] [Commented] (SPARK-23310) Perf regression introduced by SPARK-21113

2018-02-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352754#comment-16352754 ] Kazuaki Ishizaki commented on SPARK-23310: -- [~juliuszsompolski] We would appreciate it if you

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-02-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348980#comment-16348980 ] Kazuaki Ishizaki commented on SPARK-18016: -- This issue has been fixed in Spark 2.3 or master. It

[jira] [Resolved] (SPARK-20826) Support compression/decompression of ColumnVector in generated code

2018-01-31 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-20826. -- Resolution: Duplicate > Support compression/decompression of ColumnVector in generated

[jira] [Resolved] (SPARK-20823) Generate code to build table cache using ColumnarBatch and to get value from ColumnVector for other types

2018-01-31 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-20823. -- Resolution: Won't Fix > Generate code to build table cache using ColumnarBatch and to

[jira] [Resolved] (SPARK-20821) Add compression/decompression of data to ColumnVector for other data types

2018-01-31 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-20821. -- Resolution: Won't Fix > Add compression/decompression of data to ColumnVector for

[jira] [Resolved] (SPARK-20820) Add compression/decompression of data to ColumnVector for other compression schemes

2018-01-31 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-20820. -- Resolution: Duplicate > Add compression/decompression of data to ColumnVector for

[jira] [Resolved] (SPARK-20824) Generate code to get value from table cache with wider column in ColumnarBatch

2018-01-31 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-20824. -- Resolution: Won't Fix > Generate code to get value from table cache with wider column

[jira] [Resolved] (SPARK-20825) Generate code to get value from table cache with wider column in ColumnarBatch for other data types

2018-01-31 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-20825. -- Resolution: Not A Problem > Generate code to get value from table cache with wider

[jira] [Resolved] (SPARK-22747) Shorten lifetime of global variables used in HashAggregateExec

2018-01-31 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-22747. -- Resolution: Won't Fix > Shorten lifetime of global variables used in HashAggregateExec

[jira] [Resolved] (SPARK-22935) Dataset with Java Beans for java.sql.Date throws CompileException

2018-01-31 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-22935. -- Resolution: Invalid > Dataset with Java Beans for java.sql.Date throws

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-31 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347160#comment-16347160 ] Kazuaki Ishizaki commented on SPARK-18016: -- [~gaurav.garg] I found the message {{18/01/30

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-31 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16347092#comment-16347092 ] Kazuaki Ishizaki commented on SPARK-18016: -- [~gaurav.garg] Thank you for attaching the log file.

[jira] [Comment Edited] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-30 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16345551#comment-16345551 ] Kazuaki Ishizaki edited comment on SPARK-18016 at 1/30/18 6:27 PM: ---

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-30 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16345551#comment-16345551 ] Kazuaki Ishizaki commented on SPARK-18016: -- [~gaurav.garg] Thank you for your confirmation. I am

[jira] [Comment Edited] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16336957#comment-16336957 ] Kazuaki Ishizaki edited comment on SPARK-18016 at 1/24/18 5:49 AM: --- I

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16336957#comment-16336957 ] Kazuaki Ishizaki commented on SPARK-18016: -- I cannot reproduce this issue using the following

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333897#comment-16333897 ] Kazuaki Ishizaki commented on SPARK-18016: -- Thanks, I will look at this. > Code Generation:

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333298#comment-16333298 ] Kazuaki Ishizaki commented on SPARK-18016: -- Thank you for reporting an issue. Could you post a

[jira] [Commented] (SPARK-22935) Dataset with Java Beans for java.sql.Date throws CompileException

2018-01-02 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16308194#comment-16308194 ] Kazuaki Ishizaki commented on SPARK-22935: -- [~jlaskowski] When you see the scheme of this

[jira] [Updated] (SPARK-22935) Dataset with Java Beans for java.sql.Date throws CompileException

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22935: - Description: The following code can throw an exception with or without whole-stage

[jira] [Resolved] (SPARK-21413) Multiple projections with CASE WHEN fails to run generated codes

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-21413. -- Resolution: Duplicate > Multiple projections with CASE WHEN fails to run generated

[jira] [Commented] (SPARK-21413) Multiple projections with CASE WHEN fails to run generated codes

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307531#comment-16307531 ] Kazuaki Ishizaki commented on SPARK-21413: -- SPARK-22520 solved this issue. > Multiple

[jira] [Commented] (SPARK-22935) Dataset with Java Beans for java.sql.Date produces incorrect result

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307520#comment-16307520 ] Kazuaki Ishizaki commented on SPARK-22935: -- I am working for this. > Dataset with Java Beans

[jira] [Created] (SPARK-22935) Dataset with Java Beans for java.sql.Date produces incorrect result

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22935: Summary: Dataset with Java Beans for java.sql.Date produces incorrect result Key: SPARK-22935 URL: https://issues.apache.org/jira/browse/SPARK-22935 Project:

[jira] [Updated] (SPARK-22935) Dataset with Java Beans for java.sql.Date produces incorrect result

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22935: - Description: The following code prints {{c=0}}. The value of {{c}} must be 2. {{ds}}

[jira] [Commented] (SPARK-22500) 64KB JVM bytecode limit problem with cast

2017-12-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16306244#comment-16306244 ] Kazuaki Ishizaki commented on SPARK-22500: -- Thank you for your report. I think that this is the

[jira] [Updated] (SPARK-22868) 64KB JVM bytecode limit problem with aggregation

2017-12-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22868: - Issue Type: Sub-task (was: Bug) Parent: SPARK-22510 > 64KB JVM bytecode limit

[jira] [Updated] (SPARK-22869) 64KB JVM bytecode limit problem with filter

2017-12-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22869: - Issue Type: Sub-task (was: Bug) Parent: SPARK-22510 > 64KB JVM bytecode limit

[jira] [Commented] (SPARK-22869) 64KB JVM bytecode limit problem with filter

2017-12-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16300946#comment-16300946 ] Kazuaki Ishizaki commented on SPARK-22869: -- I am working for this and waiting for merging

[jira] [Commented] (SPARK-22868) 64KB JVM bytecode limit problem with aggregation

2017-12-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16300945#comment-16300945 ] Kazuaki Ishizaki commented on SPARK-22868: -- I am working for this and waiting for merging

[jira] [Created] (SPARK-22869) 64KB JVM bytecode limit problem with filter

2017-12-21 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22869: Summary: 64KB JVM bytecode limit problem with filter Key: SPARK-22869 URL: https://issues.apache.org/jira/browse/SPARK-22869 Project: Spark Issue

[jira] [Created] (SPARK-22868) 64KB JVM bytecode limit problem with aggregation

2017-12-21 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22868: Summary: 64KB JVM bytecode limit problem with aggregation Key: SPARK-22868 URL: https://issues.apache.org/jira/browse/SPARK-22868 Project: Spark

[jira] [Updated] (SPARK-22848) Avoid the generation of useless mutable states by Stack function

2017-12-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22848: - Description: Current Spark generates two mutable states in generated code for

[jira] [Updated] (SPARK-22848) Avoid the generation of useless mutable states by Stack function

2017-12-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22848: - Description: Current Spark generates two mutable states in generated code for {Stack}.

[jira] [Updated] (SPARK-22848) Avoid the generation of useless mutable states by Stack function

2017-12-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22848: - Description: Current Spark generates two mutable states in generated code for {{Stack}}.

[jira] [Updated] (SPARK-22848) Avoid the generation of useless mutable states by Stack function

2017-12-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22848: - Summary: Avoid the generation of useless mutable states by Stack function (was: Do not

[jira] [Updated] (SPARK-22848) Do not execute generated code of functions in Generator

2017-12-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22848: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-22692 > Do not execute

[jira] [Created] (SPARK-22848) Do not execute generated code of functions in Generator

2017-12-20 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22848: Summary: Do not execute generated code of functions in Generator Key: SPARK-22848 URL: https://issues.apache.org/jira/browse/SPARK-22848 Project: Spark

[jira] [Commented] (SPARK-22813) run-tests.py fails when /usr/sbin/lsof does not exist

2017-12-15 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16293570#comment-16293570 ] Kazuaki Ishizaki commented on SPARK-22813: -- Thank you for pointing the PR that we worked. I was

[jira] [Updated] (SPARK-22813) run-tests.py fails when /usr/sbin/lsof does not exist

2017-12-15 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22813: - Description: Running ./dev/run-tests.py for mvn on OS that does not have /usr/sbin/lsof

[jira] [Created] (SPARK-22813) run-tests.py fails when /usr/sbin/lsof does not exist

2017-12-15 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22813: Summary: run-tests.py fails when /usr/sbin/lsof does not exist Key: SPARK-22813 URL: https://issues.apache.org/jira/browse/SPARK-22813 Project: Spark

[jira] [Commented] (SPARK-22776) Increase default value of spark.sql.codegen.maxFields

2017-12-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16290351#comment-16290351 ] Kazuaki Ishizaki commented on SPARK-22776: -- Wait for re-merging SPARK-22600 > Increase default

[jira] [Created] (SPARK-22776) Increase default value of spark.sql.codegen.maxFields

2017-12-13 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22776: Summary: Increase default value of spark.sql.codegen.maxFields Key: SPARK-22776 URL: https://issues.apache.org/jira/browse/SPARK-22776 Project: Spark

[jira] [Commented] (SPARK-22774) Add compilation check for generated code in TPCDSQuerySuite

2017-12-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289471#comment-16289471 ] Kazuaki Ishizaki commented on SPARK-22774: -- I will commit a PR for this soon. > Add compilation

[jira] [Comment Edited] (SPARK-22774) Add compilation check for generated code in TPCDSQuerySuite

2017-12-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289471#comment-16289471 ] Kazuaki Ishizaki edited comment on SPARK-22774 at 12/13/17 4:24 PM: I

[jira] [Created] (SPARK-22774) Add compilation check for generated code in TPCDSQuerySuite

2017-12-13 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22774: Summary: Add compilation check for generated code in TPCDSQuerySuite Key: SPARK-22774 URL: https://issues.apache.org/jira/browse/SPARK-22774 Project: Spark

[jira] [Created] (SPARK-22747) Shorten lifetime of global variables used in HashAggregateExec

2017-12-09 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22747: Summary: Shorten lifetime of global variables used in HashAggregateExec Key: SPARK-22747 URL: https://issues.apache.org/jira/browse/SPARK-22747 Project:

[jira] [Updated] (SPARK-22746) Avoid the generation of useless mutable states by SortMergeJoin

2017-12-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22746: - Issue Type: Sub-task (was: Bug) Parent: SPARK-22692 > Avoid the generation of

[jira] [Created] (SPARK-22746) Avoid the generation of useless mutable states by SortMergeJoin

2017-12-09 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22746: Summary: Avoid the generation of useless mutable states by SortMergeJoin Key: SPARK-22746 URL: https://issues.apache.org/jira/browse/SPARK-22746 Project:

[jira] [Created] (SPARK-22705) Reduce # of mutable variables in Case, Coalesce, and In

2017-12-05 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22705: Summary: Reduce # of mutable variables in Case, Coalesce, and In Key: SPARK-22705 URL: https://issues.apache.org/jira/browse/SPARK-22705 Project: Spark

[jira] [Created] (SPARK-22704) Reduce # of mutable variables in Least and greatest

2017-12-05 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22704: Summary: Reduce # of mutable variables in Least and greatest Key: SPARK-22704 URL: https://issues.apache.org/jira/browse/SPARK-22704 Project: Spark

[jira] [Updated] (SPARK-22688) Upgrade Janino version to 3.0.8

2017-12-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22688: - Summary: Upgrade Janino version to 3.0.8 (was: Upgrade Janino version 3.0.8) > Upgrade

[jira] [Updated] (SPARK-22688) Upgrade Janino version 3.0.8

2017-12-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22688: - Description: [Janino 3.0.8|https://janino-compiler.github.io/janino/changelog.html]

[jira] [Updated] (SPARK-22688) Upgrade Janino version 3.0.8

2017-12-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22688: - Summary: Upgrade Janino version 3.0.8 (was: Upgrade Janino version 0.3.8) > Upgrade

[jira] [Created] (SPARK-22688) Upgrade Janino version 0.3.8

2017-12-04 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22688: Summary: Upgrade Janino version 0.3.8 Key: SPARK-22688 URL: https://issues.apache.org/jira/browse/SPARK-22688 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-22668) CodegenContext.splitExpressions() creates incorrect results with global variable arguments

2017-12-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22668: - Summary: CodegenContext.splitExpressions() creates incorrect results with global

[jira] [Updated] (SPARK-22668) CodegenContext.splitExpressions() creates incorrect results with global variable arguments

2017-12-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22668: - Description: {{CodegenContext.splitExpressions()}} creates incorrect results with

[jira] [Commented] (SPARK-22668) CodegenContext.splitExpression() creates incorrect results with global variable arguments

2017-12-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16274128#comment-16274128 ] Kazuaki Ishizaki commented on SPARK-22668: -- I am working for this. >

[jira] [Created] (SPARK-22668) CodegenContext.splitExpression() creates incorrect results with global variable arguments

2017-12-01 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22668: Summary: CodegenContext.splitExpression() creates incorrect results with global variable arguments Key: SPARK-22668 URL:

[jira] [Commented] (SPARK-22373) Intermittent NullPointerException in org.codehaus.janino.IClass.isAssignableFrom

2017-11-27 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16267901#comment-16267901 ] Kazuaki Ishizaki commented on SPARK-22373: -- [~mshen] Thank you for reporting this issue. We

[jira] [Updated] (SPARK-22608) Avoid code duplication regarding CodeGeneration.splitExpressions()

2017-11-26 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22608: - Issue Type: Improvement (was: Bug) > Avoid code duplication regarding

[jira] [Updated] (SPARK-22608) Avoid code duplication regarding CodeGeneration.splitExpressions()

2017-11-26 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22608: - Priority: Minor (was: Major) > Avoid code duplication regarding

[jira] [Updated] (SPARK-22608) Avoid code duplication regarding CodeGeneration.splitExpressions()

2017-11-26 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22608: - Summary: Avoid code duplication regarding CodeGeneration.splitExpressions() (was: Add

[jira] [Created] (SPARK-22608) Add new API to CodeGeneration.splitExpressions()

2017-11-26 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22608: Summary: Add new API to CodeGeneration.splitExpressions() Key: SPARK-22608 URL: https://issues.apache.org/jira/browse/SPARK-22608 Project: Spark

[jira] [Updated] (SPARK-22603) 64KB JVM bytecode limit problem with FormatString

2017-11-24 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22603: - Issue Type: Sub-task (was: Bug) Parent: SPARK-22510 > 64KB JVM bytecode limit

[jira] [Updated] (SPARK-22603) 64KB JVM bytecode limit problem with FormatString

2017-11-24 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22603: - Description: {{FormatString}} can throw an exception due to the 64KB JVM bytecode limit

[jira] [Created] (SPARK-22603) 64KB JVM bytecode limit problem with FormatString

2017-11-24 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22603: Summary: 64KB JVM bytecode limit problem with FormatString Key: SPARK-22603 URL: https://issues.apache.org/jira/browse/SPARK-22603 Project: Spark

[jira] [Commented] (SPARK-22595) flaky test: CastSuite.SPARK-22500: cast for struct should not generate codes beyond 64KB

2017-11-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264663#comment-16264663 ] Kazuaki Ishizaki commented on SPARK-22595: -- I see. I will look at this. > flaky test:

[jira] [Updated] (SPARK-22570) Create a lot of global variables to reuse an object in generated code

2017-11-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22570: - Description: Generated code for several operations such as {{Cast}}, {{RegExpReplace}},

[jira] [Updated] (SPARK-22570) Create a lot of global variables to reuse an object

2017-11-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22570: - Summary: Create a lot of global variables to reuse an object (was: Cast may create a

[jira] [Updated] (SPARK-22570) Create a lot of global variables to reuse an object in generated code

2017-11-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22570: - Summary: Create a lot of global variables to reuse an object in generated code (was:

[jira] [Updated] (SPARK-22570) Cast may create a lot of UTF8String.IntWrapper or UTF8String.longWrapper instances

2017-11-22 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22570: - Issue Type: Sub-task (was: Bug) Parent: SPARK-22510 > Cast may create a lot of

[jira] [Updated] (SPARK-22510) Exceptions caused by 64KB JVM bytecode or constant pool entry limits

2017-11-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22510: - Description: Codegen can throw an exception due to the 64KB JVM bytecode or 64K constant

[jira] [Updated] (SPARK-22510) Exceptions caused by 64KB JVM bytecode or 64K constant pool entry limit

2017-11-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22510: - Summary: Exceptions caused by 64KB JVM bytecode or 64K constant pool entry limit (was:

[jira] [Updated] (SPARK-22510) Exceptions caused by 64KB JVM bytecode or constant pool entry limits

2017-11-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22510: - Summary: Exceptions caused by 64KB JVM bytecode or constant pool entry limits (was:

[jira] [Commented] (SPARK-22570) Cast may create a lot of UTF8String.IntWrapper or UTF8String.longWrapper instances

2017-11-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260363#comment-16260363 ] Kazuaki Ishizaki commented on SPARK-22570: -- I am working on this with SPARK-22500 > Cast may

[jira] [Created] (SPARK-22570) Cast may create a lot of UTF8String.IntWrapper or UTF8String.longWrapper instances

2017-11-20 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22570: Summary: Cast may create a lot of UTF8String.IntWrapper or UTF8String.longWrapper instances Key: SPARK-22570 URL: https://issues.apache.org/jira/browse/SPARK-22570

[jira] [Updated] (SPARK-22550) 64KB JVM bytecode limit problem with elt

2017-11-17 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22550: - Issue Type: Sub-task (was: Bug) Parent: SPARK-22510 > 64KB JVM bytecode limit

[jira] [Created] (SPARK-22550) 64KB JVM bytecode limit problem with elt

2017-11-17 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22550: Summary: 64KB JVM bytecode limit problem with elt Key: SPARK-22550 URL: https://issues.apache.org/jira/browse/SPARK-22550 Project: Spark Issue Type:

[jira] [Updated] (SPARK-22549) 64KB JVM bytecode limit problem with concat_ws

2017-11-17 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22549: - Issue Type: Sub-task (was: Bug) Parent: SPARK-22510 > 64KB JVM bytecode limit

[jira] [Created] (SPARK-22549) 64KB JVM bytecode limit problem with concat_ws

2017-11-17 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22549: Summary: 64KB JVM bytecode limit problem with concat_ws Key: SPARK-22549 URL: https://issues.apache.org/jira/browse/SPARK-22549 Project: Spark Issue

[jira] [Updated] (SPARK-22498) 64KB JVM bytecode limit problem with concat

2017-11-17 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22498: - Summary: 64KB JVM bytecode limit problem with concat (was: 64KB JVM bytecode limit

[jira] [Updated] (SPARK-22498) 64KB JVM bytecode limit problem with concat

2017-11-17 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22498: - Description: {{concat}} can throw an exception due to the 64KB JVM bytecode limit when

[jira] [Commented] (SPARK-22517) NullPointerException in ShuffleExternalSorter.spill()

2017-11-15 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16254706#comment-16254706 ] Kazuaki Ishizaki commented on SPARK-22517: -- Could you please post a program that can reproduce

[jira] [Updated] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

2017-11-14 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-21337: - Issue Type: Sub-task (was: Bug) Parent: SPARK-22510 > SQL which has large ‘case

[jira] [Commented] (SPARK-22510) Exceptions caused by 64KB JVM bytecode limit

2017-11-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16250839#comment-16250839 ] Kazuaki Ishizaki commented on SPARK-22510: -- [~smilegator] Thanks, good idea. I will add other

[jira] [Created] (SPARK-22508) 64KB JVM bytecode limit problem with GenerateUnsafeRowJoiner.create()

2017-11-13 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22508: Summary: 64KB JVM bytecode limit problem with GenerateUnsafeRowJoiner.create() Key: SPARK-22508 URL: https://issues.apache.org/jira/browse/SPARK-22508

[jira] [Commented] (SPARK-22501) 64KB JVM bytecode limit problem with in

2017-11-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16249029#comment-16249029 ] Kazuaki Ishizaki commented on SPARK-22501: -- [~mgaido] I am working on this. > 64KB JVM bytecode

[jira] [Created] (SPARK-22501) 64KB JVM bytecode limit problem with in

2017-11-12 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22501: Summary: 64KB JVM bytecode limit problem with in Key: SPARK-22501 URL: https://issues.apache.org/jira/browse/SPARK-22501 Project: Spark Issue Type:

[jira] [Updated] (SPARK-22499) 64KB JVM bytecode limit problem with least and greatest

2017-11-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22499: - Description: Both {{least}} and {{greatest}} can throw an exception due to the 64KB JVM

[jira] [Created] (SPARK-22500) 64KB JVM bytecode limit problem with cast

2017-11-12 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22500: Summary: 64KB JVM bytecode limit problem with cast Key: SPARK-22500 URL: https://issues.apache.org/jira/browse/SPARK-22500 Project: Spark Issue

[jira] [Updated] (SPARK-22498) 64KB JVM bytecode limit problem with concat and concat_ws

2017-11-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22498: - Description: Both {{concat}} and {{concat_ws}} can throw an exception due to the 64KB

[jira] [Updated] (SPARK-22499) 64KB JVM bytecode limit problem with least and greatest

2017-11-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22499: - Description: Both {{least}} and {{greatest}} can thow an exception due to the 64KB JVM

[jira] [Updated] (SPARK-22499) 64KB JVM bytecode limit problem with least and greatest

2017-11-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22499: - Summary: 64KB JVM bytecode limit problem with least and greatest (was: 64KB JVM

<    1   2   3   4   5   6   7   8   >