[jira] [Commented] (SPARK-24754) Minhash integer overflow

2018-07-07 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535657#comment-16535657 ] Kazuaki Ishizaki commented on SPARK-24754: -- In test cases, we would appreciate it if you will

[jira] [Commented] (SPARK-24579) SPIP: Standardize Optimized Data Exchange between Spark and DL/AI frameworks

2018-07-02 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16530640#comment-16530640 ] Kazuaki Ishizaki commented on SPARK-24579: -- I cannot see comments on the doc, too. > SPIP:

[jira] [Commented] (SPARK-24498) Add JDK compiler for runtime codegen

2018-06-19 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517097#comment-16517097 ] Kazuaki Ishizaki commented on SPARK-24498: -- [~maropu] thank you, let us use this as a start

[jira] [Commented] (SPARK-24529) Add spotbugs into maven build process

2018-06-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509569#comment-16509569 ] Kazuaki Ishizaki commented on SPARK-24529: -- I am working for this > Add spotbugs into maven

[jira] [Created] (SPARK-24529) Add spotbugs into maven build process

2018-06-12 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-24529: Summary: Add spotbugs into maven build process Key: SPARK-24529 URL: https://issues.apache.org/jira/browse/SPARK-24529 Project: Spark Issue Type:

[jira] [Commented] (SPARK-24498) Add JDK compiler for runtime codegen

2018-06-08 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506239#comment-16506239 ] Kazuaki Ishizaki commented on SPARK-24498: -- Hi [~smilegator] Definetely, I am interested in

[jira] [Commented] (SPARK-24486) Slow performance reading ArrayType columns

2018-06-07 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504845#comment-16504845 ] Kazuaki Ishizaki commented on SPARK-24486: -- Thank you for reporting a problem. Could you please

[jira] [Created] (SPARK-24452) long = int*int or long = int+int may cause overflow.

2018-06-01 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-24452: Summary: long = int*int or long = int+int may cause overflow. Key: SPARK-24452 URL: https://issues.apache.org/jira/browse/SPARK-24452 Project: Spark

[jira] [Created] (SPARK-24323) Java lint errors

2018-05-19 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-24323: Summary: Java lint errors Key: SPARK-24323 URL: https://issues.apache.org/jira/browse/SPARK-24323 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-24314) interpreted element_at or GetMapValue does not work for complex types

2018-05-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480482#comment-16480482 ] Kazuaki Ishizaki commented on SPARK-24314: -- I am working for this. > interpreted element_at or

[jira] [Reopened] (SPARK-24314) interpreted element_at or GetMapValue does not work for complex types

2018-05-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki reopened SPARK-24314: -- > interpreted element_at or GetMapValue does not work for complex types >

[jira] [Updated] (SPARK-24314) interpreted element_at or GetMapValue does not work for complex types

2018-05-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-24314: - Summary: interpreted element_at or GetMapValue does not work for complex types (was:

[jira] [Resolved] (SPARK-24314) interpreted array_position does not work for complex types

2018-05-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-24314. -- Resolution: Duplicate > interpreted array_position does not work for complex types >

[jira] [Created] (SPARK-24314) interpreted array_position does not work for complex types

2018-05-18 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-24314: Summary: interpreted array_position does not work for complex types Key: SPARK-24314 URL: https://issues.apache.org/jira/browse/SPARK-24314 Project: Spark

[jira] [Commented] (SPARK-24273) Failure while using .checkpoint method

2018-05-15 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475941#comment-16475941 ] Kazuaki Ishizaki commented on SPARK-24273: -- Thank you for reporting this issue? Would it be

[jira] [Commented] (SPARK-24220) java.lang.NullPointerException at org.apache.spark.sql.execution.UnsafeExternalRowSorter.(UnsafeExternalRowSorter.java:83)

2018-05-11 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472195#comment-16472195 ] Kazuaki Ishizaki commented on SPARK-24220: -- Thank you for reporting an issue. Would it be

[jira] [Commented] (SPARK-24119) Add interpreted execution to SortPrefix expression

2018-04-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458297#comment-16458297 ] Kazuaki Ishizaki commented on SPARK-24119: -- It seems to make sense. It would be good to set

[jira] [Comment Edited] (SPARK-23933) High-order function: map(array, array) → map<K,V>

2018-04-25 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452404#comment-16452404 ] Kazuaki Ishizaki edited comment on SPARK-23933 at 4/25/18 6:48 PM: ---

[jira] [Commented] (SPARK-23933) High-order function: map(array, array) → map<K,V>

2018-04-25 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16452404#comment-16452404 ] Kazuaki Ishizaki commented on SPARK-23933: -- Thank you for your comment. The current map can take

[jira] [Commented] (SPARK-23933) High-order function: map(array, array) → map<K,V>

2018-04-24 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450262#comment-16450262 ] Kazuaki Ishizaki commented on SPARK-23933: -- cc [~smilegator] > High-order function: map(array,

[jira] [Commented] (SPARK-10399) Off Heap Memory Access for non-JVM libraries (C++)

2018-04-20 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446166#comment-16446166 ] Kazuaki Ishizaki commented on SPARK-10399: -- https://issues.apache.org/jira/browse/SPARK-23879 is

[jira] [Comment Edited] (SPARK-23933) High-order function: map(array, array) → map<K,V>

2018-04-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436976#comment-16436976 ] Kazuaki Ishizaki edited comment on SPARK-23933 at 4/18/18 4:22 PM: ---

[jira] [Commented] (SPARK-23933) High-order function: map(array, array) → map<K,V>

2018-04-17 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16441268#comment-16441268 ] Kazuaki Ishizaki commented on SPARK-23933: -- ping [~smilegator] > High-order function:

[jira] [Commented] (SPARK-23986) CompileException when using too many avg aggregation after joining

2018-04-15 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438885#comment-16438885 ] Kazuaki Ishizaki commented on SPARK-23986: -- While I also checked it with branch-2.3, it works

[jira] [Comment Edited] (SPARK-23986) CompileException when using too many avg aggregation after joining

2018-04-15 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438818#comment-16438818 ] Kazuaki Ishizaki edited comment on SPARK-23986 at 4/15/18 7:36 PM: ---

[jira] [Commented] (SPARK-23986) CompileException when using too many avg aggregation after joining

2018-04-15 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438818#comment-16438818 ] Kazuaki Ishizaki commented on SPARK-23986: -- Thank for reporting an issue with deep dive. When I

[jira] [Created] (SPARK-23976) UTF8String.concat() or ByteArray.concat() may allocate shorter structure.

2018-04-13 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-23976: Summary: UTF8String.concat() or ByteArray.concat() may allocate shorter structure. Key: SPARK-23976 URL: https://issues.apache.org/jira/browse/SPARK-23976

[jira] [Commented] (SPARK-23933) High-order function: map(array, array) → map<K,V>

2018-04-13 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436976#comment-16436976 ] Kazuaki Ishizaki commented on SPARK-23933: -- [~smilegator] [~ueshin] Could you favor us? SparkSQL

[jira] [Commented] (SPARK-23933) High-order function: map(array, array) → map<K,V>

2018-04-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436136#comment-16436136 ] Kazuaki Ishizaki commented on SPARK-23933: -- I will work for this, thank you. > High-order

[jira] [Commented] (SPARK-23915) High-order function: array_except(x, y) → array

2018-04-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16435666#comment-16435666 ] Kazuaki Ishizaki commented on SPARK-23915: -- I will work for this, thanks. > High-order

[jira] [Commented] (SPARK-23914) High-order function: array_union(x, y) → array

2018-04-11 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434494#comment-16434494 ] Kazuaki Ishizaki commented on SPARK-23914: -- I will work for this, thank you. > High-order

[jira] [Commented] (SPARK-23913) High-order function: array_intersect(x, y) → array

2018-04-11 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434491#comment-16434491 ] Kazuaki Ishizaki commented on SPARK-23913: -- I will work for this, thank you. > High-order

[jira] [Commented] (SPARK-23916) High-order function: array_join(x, delimiter, null_replacement) → varchar

2018-04-10 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431795#comment-16431795 ] Kazuaki Ishizaki commented on SPARK-23916: -- Sorry for my mistake regarding a PR with wrong JIRA

[jira] [Comment Edited] (SPARK-23923) High-order function: cardinality(x) → bigint

2018-04-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431009#comment-16431009 ] Kazuaki Ishizaki edited comment on SPARK-23923 at 4/9/18 6:36 PM: -- I

[jira] [Commented] (SPARK-23921) High-order function: array_sort(x) → array

2018-04-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431037#comment-16431037 ] Kazuaki Ishizaki commented on SPARK-23921: -- I am working for this > High-order function:

[jira] [Comment Edited] (SPARK-23919) High-order function: array_position(x, element) → bigint

2018-04-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431004#comment-16431004 ] Kazuaki Ishizaki edited comment on SPARK-23919 at 4/9/18 6:19 PM: -- I

[jira] [Commented] (SPARK-23923) High-order function: cardinality(x) → bigint

2018-04-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431009#comment-16431009 ] Kazuaki Ishizaki commented on SPARK-23923: -- I am working for this. > High-order function:

[jira] [Commented] (SPARK-23919) High-order function: array_position(x, element) → bigint

2018-04-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431004#comment-16431004 ] Kazuaki Ishizaki commented on SPARK-23919: -- I am working for this. > High-order function:

[jira] [Commented] (SPARK-23924) High-order function: element_at

2018-04-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431002#comment-16431002 ] Kazuaki Ishizaki commented on SPARK-23924: -- I will work for this. > High-order function:

[jira] [Created] (SPARK-23893) Possible overflow in long = int * int

2018-04-07 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-23893: Summary: Possible overflow in long = int * int Key: SPARK-23893 URL: https://issues.apache.org/jira/browse/SPARK-23893 Project: Spark Issue Type:

[jira] [Updated] (SPARK-23892) Improve coverage and fix lint error in UTF8String-related Suite

2018-04-06 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-23892: - Description: The following code in {{UTF8StringSuite}} has no sense. {code}

[jira] [Updated] (SPARK-23892) Improve coverage and fix lint error in UTF8String-related Suite

2018-04-06 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-23892: - Description: The following code in {{UTF8StringSuite}} has no sense. {code}

[jira] [Updated] (SPARK-23892) Improve coverage and fix lint error in UTF8String-related Suite

2018-04-06 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-23892: - Summary: Improve coverage and fix lint error in UTF8String-related Suite (was: Improve

[jira] [Created] (SPARK-23892) Improve coverage and fix lint error in UTF8StringSuite

2018-04-06 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-23892: Summary: Improve coverage and fix lint error in UTF8StringSuite Key: SPARK-23892 URL: https://issues.apache.org/jira/browse/SPARK-23892 Project: Spark

[jira] [Created] (SPARK-23882) Is UTF8StringSuite.writeToOutputStreamUnderflow() supported?

2018-04-06 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-23882: Summary: Is UTF8StringSuite.writeToOutputStreamUnderflow() supported? Key: SPARK-23882 URL: https://issues.apache.org/jira/browse/SPARK-23882 Project: Spark

[jira] [Updated] (SPARK-23762) UTF8StringBuilder uses MemoryBlock

2018-04-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-23762: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-23879 > UTF8StringBuilder

[jira] [Updated] (SPARK-23763) OffHeapColumnVector uses MemoryBlock

2018-04-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-23763: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-23879 >

[jira] [Updated] (SPARK-23779) TaskMemoryManager and UnsafeSorter use MemoryBlock

2018-04-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-23779: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-23879 > TaskMemoryManager

[jira] [Updated] (SPARK-23879) Introduce MemoryBlock API instead of Platform API with Object

2018-04-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-23879: - Component/s: Spark Core > Introduce MemoryBlock API instead of Platform API with Object

[jira] [Created] (SPARK-23879) Introduce MemoryBlock API instead of Platform API with Object

2018-04-05 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-23879: Summary: Introduce MemoryBlock API instead of Platform API with Object Key: SPARK-23879 URL: https://issues.apache.org/jira/browse/SPARK-23879 Project:

[jira] [Commented] (SPARK-23801) Consistent SIGSEGV after upgrading to Spark v2.3.0

2018-04-02 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422955#comment-16422955 ] Kazuaki Ishizaki commented on SPARK-23801: -- If you use WebUI, you can easily see SQL plans in a

[jira] [Commented] (SPARK-23801) Consistent SIGSEGV after upgrading to Spark v2.3.0

2018-03-27 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16416835#comment-16416835 ] Kazuaki Ishizaki commented on SPARK-23801: -- While we know it is time-consuming work to narrow

[jira] [Created] (SPARK-23779) TaskMemoryManager and UnsafeSorter use MemoryBlock

2018-03-23 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-23779: Summary: TaskMemoryManager and UnsafeSorter use MemoryBlock Key: SPARK-23779 URL: https://issues.apache.org/jira/browse/SPARK-23779 Project: Spark

[jira] [Updated] (SPARK-23763) OffHeapColumnVector uses MemoryBlock

2018-03-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-23763: - Description: This JIRA entry tries to use {{MemoryBlock}} in {{OffHeapColumnVector}}.

[jira] [Updated] (SPARK-23763) BufferHolder uses MemoryBlock

2018-03-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-23763: - Description: This JIRA entry tries to use {{MemoryBlock}} in {{BufferHolder}}. was:

[jira] [Created] (SPARK-23763) UTF8StringBuilder uses MemoryBlock

2018-03-21 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-23763: Summary: UTF8StringBuilder uses MemoryBlock Key: SPARK-23763 URL: https://issues.apache.org/jira/browse/SPARK-23763 Project: Spark Issue Type:

[jira] [Created] (SPARK-23762) UTF8StringBuilder uses MemoryBlock

2018-03-21 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-23762: Summary: UTF8StringBuilder uses MemoryBlock Key: SPARK-23762 URL: https://issues.apache.org/jira/browse/SPARK-23762 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-23713) Clean-up UnsafeWriter classes

2018-03-16 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403246#comment-16403246 ] Kazuaki Ishizaki edited comment on SPARK-23713 at 3/17/18 3:42 AM: --- It

[jira] [Commented] (SPARK-23713) Clean-up UnsafeWriter classes

2018-03-16 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403246#comment-16403246 ] Kazuaki Ishizaki commented on SPARK-23713: -- It will make sense. I will work for this. >

[jira] [Commented] (SPARK-23673) PySpark dayofweek does not conform with ISO 8601

2018-03-15 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401392#comment-16401392 ] Kazuaki Ishizaki commented on SPARK-23673: -- In Spark, `dayofweek` comes from SQL. The result

[jira] [Updated] (SPARK-23656) Assertion in XXH64Suite is not performed on big endian platform

2018-03-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-23656: - Description: {{XXH64Suite.testKnownByteArrayInputs()}} performs assertions only on

[jira] [Updated] (SPARK-23656) Assertion in XXH64Suite.testKnownByteArrayInputs() is not performed on big endian platform

2018-03-12 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-23656: - Summary: Assertion in XXH64Suite.testKnownByteArrayInputs() is not performed on big

[jira] [Created] (SPARK-23656) Assertion in XXH64Suite is not performed on big endian platform

2018-03-12 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-23656: Summary: Assertion in XXH64Suite is not performed on big endian platform Key: SPARK-23656 URL: https://issues.apache.org/jira/browse/SPARK-23656 Project:

[jira] [Commented] (SPARK-23605) Conflicting dependencies for janino in 2.3.0

2018-03-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394035#comment-16394035 ] Kazuaki Ishizaki commented on SPARK-23605: -- Thank you for providing information. If I am

[jira] [Comment Edited] (SPARK-23605) Conflicting dependencies for janino in 2.3.0

2018-03-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393793#comment-16393793 ] Kazuaki Ishizaki edited comment on SPARK-23605 at 3/9/18 11:30 PM: ---

[jira] [Commented] (SPARK-23605) Conflicting dependencies for janino in 2.3.0

2018-03-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393793#comment-16393793 ] Kazuaki Ishizaki commented on SPARK-23605: -- Would it be possible to post a program that can

[jira] [Commented] (SPARK-23610) Cast of ArrayType of NullType to ArrayType of nullable material type does not work

2018-03-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393173#comment-16393173 ] Kazuaki Ishizaki commented on SPARK-23610: -- The following code works without an exception using

[jira] [Updated] (SPARK-22246) UnsafeRow, UnsafeArrayData, and UnsafeMapData use MemoryBlock

2018-03-08 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22246: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-10399 > UnsafeRow,

[jira] [Comment Edited] (SPARK-23598) WholeStageCodegen can lead to IllegalAccessError calling append for HashAggregateExec

2018-03-08 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391834#comment-16391834 ] Kazuaki Ishizaki edited comment on SPARK-23598 at 3/9/18 3:20 AM: --

[jira] [Commented] (SPARK-23598) WholeStageCodegen can lead to IllegalAccessError calling append for HashAggregateExec

2018-03-08 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391834#comment-16391834 ] Kazuaki Ishizaki commented on SPARK-23598: -- Thanks, I confirmed that I can reproduce this issue

[jira] [Comment Edited] (SPARK-23549) Spark SQL unexpected behavior when comparing timestamp to date

2018-03-08 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391386#comment-16391386 ] Kazuaki Ishizaki edited comment on SPARK-23549 at 3/8/18 5:33 PM: -- I

[jira] [Commented] (SPARK-23549) Spark SQL unexpected behavior when comparing timestamp to date

2018-03-08 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391386#comment-16391386 ] Kazuaki Ishizaki commented on SPARK-23549: -- I see. Make sense. It would be good to cast

[jira] [Commented] (SPARK-23549) Spark SQL unexpected behavior when comparing timestamp to date

2018-03-08 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391344#comment-16391344 ] Kazuaki Ishizaki commented on SPARK-23549: -- I think that this is a problem in Spark. My question

[jira] [Comment Edited] (SPARK-23598) WholeStageCodegen can lead to IllegalAccessError calling append for HashAggregateExec

2018-03-07 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389766#comment-16389766 ] Kazuaki Ishizaki edited comment on SPARK-23598 at 3/7/18 4:39 PM: -- I

[jira] [Commented] (SPARK-23598) WholeStageCodegen can lead to IllegalAccessError calling append for HashAggregateExec

2018-03-07 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389766#comment-16389766 ] Kazuaki Ishizaki commented on SPARK-23598: -- I also think that the easiest way is to make

[jira] [Commented] (SPARK-18492) GeneratedIterator grows beyond 64 KB

2018-03-07 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389761#comment-16389761 ] Kazuaki Ishizaki commented on SPARK-18492: -- [~imranshaik] I ran the following program with

[jira] [Commented] (SPARK-23582) Add interpreted execution to StaticInvoke expression

2018-03-06 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388729#comment-16388729 ] Kazuaki Ishizaki commented on SPARK-23582: -- I see. Now, I have a prototype using old-school

[jira] [Commented] (SPARK-23582) Add interpreted execution to StaticInvoke expression

2018-03-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387230#comment-16387230 ] Kazuaki Ishizaki commented on SPARK-23582: -- I am working for this > Add interpreted execution

[jira] [Commented] (SPARK-23583) Add interpreted execution to Invoke expression

2018-03-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387231#comment-16387231 ] Kazuaki Ishizaki commented on SPARK-23583: -- I am working for this > Add interpreted execution

[jira] [Commented] (SPARK-23580) Interpreted mode fallback should be implemented for all expressions & projections

2018-03-05 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16386991#comment-16386991 ] Kazuaki Ishizaki commented on SPARK-23580: -- Sure, I will work for them > Interpreted mode

[jira] [Commented] (SPARK-23573) Create linter rule to prevent misuse of SparkContext.hadoopConfiguration in SQL modules

2018-03-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16385026#comment-16385026 ] Kazuaki Ishizaki commented on SPARK-23573: -- Sorry for my unintentional changes to the

[jira] [Updated] (SPARK-23573) Create linter rule to prevent misuse of SparkContext.hadoopConfiguration in SQL modules

2018-03-04 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-23573: - Description: SparkContext.hadoopConfiguration should not be used directly in SQL module.

[jira] [Comment Edited] (SPARK-18492) GeneratedIterator grows beyond 64 KB

2018-03-02 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384428#comment-16384428 ] Kazuaki Ishizaki edited comment on SPARK-18492 at 3/3/18 2:08 AM: --

[jira] [Commented] (SPARK-18492) GeneratedIterator grows beyond 64 KB

2018-03-02 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384428#comment-16384428 ] Kazuaki Ishizaki commented on SPARK-18492: -- Thanks, what is the definition of

[jira] [Commented] (SPARK-18492) GeneratedIterator grows beyond 64 KB

2018-03-02 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383936#comment-16383936 ] Kazuaki Ishizaki commented on SPARK-18492: -- According to the screenshot, the number of the

[jira] [Commented] (SPARK-18492) GeneratedIterator grows beyond 64 KB

2018-03-02 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383933#comment-16383933 ] Kazuaki Ishizaki commented on SPARK-18492: -- Thank you for reporting the issue. We would

[jira] [Created] (SPARK-23546) Refactor non-stateful methods/values in CodegenContext

2018-03-01 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-23546: Summary: Refactor non-stateful methods/values in CodegenContext Key: SPARK-23546 URL: https://issues.apache.org/jira/browse/SPARK-23546 Project: Spark

[jira] [Commented] (SPARK-23427) spark.sql.autoBroadcastJoinThreshold causing OOM exception in the driver

2018-03-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381815#comment-16381815 ] Kazuaki Ishizaki commented on SPARK-23427: -- I am working for this (now trying to collect heap

[jira] [Commented] (SPARK-23498) Accuracy problem in comparison with string and integer

2018-02-28 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16380722#comment-16380722 ] Kazuaki Ishizaki commented on SPARK-23498: -- Is SPARK-21774 related to this issue, too? >

[jira] [Commented] (SPARK-23509) Upgrade commons-net from 2.2 to 3.1

2018-02-25 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376198#comment-16376198 ] Kazuaki Ishizaki commented on SPARK-23509: -- During the build using sbt, we are seeing the

[jira] [Commented] (SPARK-23512) Complex operations on Dataframe corrupts data

2018-02-25 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376171#comment-16376171 ] Kazuaki Ishizaki commented on SPARK-23512: -- This problem can be reproduced with master using the

[jira] [Commented] (SPARK-23498) Accuracy problem in comparison with string and integer

2018-02-25 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16376137#comment-16376137 ] Kazuaki Ishizaki commented on SPARK-23498: -- SPARK implicitly casts {{String}} to type of another

[jira] [Commented] (SPARK-23459) Improve the error message when unknown column is specified in partition columns

2018-02-21 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371474#comment-16371474 ] Kazuaki Ishizaki commented on SPARK-23459: -- I am working for this. > Improve the error message

[jira] [Created] (SPARK-23466) Remove redundant null checks in generated Java code by GenerateUnsafeProjection

2018-02-19 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-23466: Summary: Remove redundant null checks in generated Java code by GenerateUnsafeProjection Key: SPARK-23466 URL: https://issues.apache.org/jira/browse/SPARK-23466

[jira] [Commented] (SPARK-23459) Improve the error message when unknown column is specified in partition columns

2018-02-19 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369346#comment-16369346 ] Kazuaki Ishizaki commented on SPARK-23459: -- Have you started working for this? If not, I can

[jira] [Comment Edited] (SPARK-23427) spark.sql.autoBroadcastJoinThreshold causing OOM exception in the driver

2018-02-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368823#comment-16368823 ] Kazuaki Ishizaki edited comment on SPARK-23427 at 2/19/18 7:01 AM: --- I

[jira] [Commented] (SPARK-23427) spark.sql.autoBroadcastJoinThreshold causing OOM exception in the driver

2018-02-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368823#comment-16368823 ] Kazuaki Ishizaki commented on SPARK-23427: -- I got the OOM with the same stack trace for both

[jira] [Comment Edited] (SPARK-23427) spark.sql.autoBroadcastJoinThreshold causing OOM exception in the driver

2018-02-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368728#comment-16368728 ] Kazuaki Ishizaki edited comment on SPARK-23427 at 2/19/18 12:49 AM:

[jira] [Commented] (SPARK-23427) spark.sql.autoBroadcastJoinThreshold causing OOM exception in the driver

2018-02-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368728#comment-16368728 ] Kazuaki Ishizaki commented on SPARK-23427: -- Thank you. I ran this program several times with

[jira] [Commented] (SPARK-23415) BufferHolderSparkSubmitSuite is flaky

2018-02-15 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365931#comment-16365931 ] Kazuaki Ishizaki commented on SPARK-23415: -- I realized that an issue in this test case. I think

<    1   2   3   4   5   6   7   8   >