[jira] [Commented] (SPARK-22226) Code generation fails for dataframes with 10000 columns

[ https://issues.apache.org/jira/browse/SPARK-6?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197164#comment-16197164 ] Kazuaki Ishizaki commented on SPARK-6: -- [This PR|https://github.com/apache/spark/pull/16648]

[jira] [Created] (SPARK-22219) Refector "spark.sql.codegen.comments"

Kazuaki Ishizaki created SPARK-22219: Summary: Refector "spark.sql.codegen.comments" Key: SPARK-22219 URL: https://issues.apache.org/jira/browse/SPARK-22219 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19984) ERROR codegen.CodeGenerator: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java'

[ https://issues.apache.org/jira/browse/SPARK-19984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194937#comment-16194937 ] Kazuaki Ishizaki commented on SPARK-19984: -- [~JohnSteidley] Here is my program that produced the

[jira] [Commented] (SPARK-19984) ERROR codegen.CodeGenerator: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java'

[ https://issues.apache.org/jira/browse/SPARK-19984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16193201#comment-16193201 ] Kazuaki Ishizaki commented on SPARK-19984: -- [~JohnSteidley] Thank you for your comment. Now, I

[jira] [Commented] (SPARK-19984) ERROR codegen.CodeGenerator: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java'

[ https://issues.apache.org/jira/browse/SPARK-19984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16190050#comment-16190050 ] Kazuaki Ishizaki commented on SPARK-19984: -- [~JohnSteidley] Thank you for providing valuable

[jira] [Commented] (SPARK-19984) ERROR codegen.CodeGenerator: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java'

[ https://issues.apache.org/jira/browse/SPARK-19984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188320#comment-16188320 ] Kazuaki Ishizaki commented on SPARK-19984: -- [~JohnSteidley] Thank you for your report. I also

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184206#comment-16184206 ] Kazuaki Ishizaki commented on SPARK-18016: -- Thank you for reporting this again. While I pinged

[jira] [Updated] (SPARK-22130) UTF8String.trim() inefficiently scans all white-space string twice.

[ https://issues.apache.org/jira/browse/SPARK-22130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22130: - Issue Type: Improvement (was: Bug) > UTF8String.trim() inefficiently scans all

[jira] [Commented] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181190#comment-16181190 ] Kazuaki Ishizaki commented on SPARK-16845: -- [~mvelusce] Thank you for reporting an issue with

[jira] [Commented] (SPARK-22130) UTF8String.trim() inefficiently scans all white-space string twice.

[ https://issues.apache.org/jira/browse/SPARK-22130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181070#comment-16181070 ] Kazuaki Ishizaki commented on SPARK-22130: -- I will submit a PR soon. > UTF8String.trim()

[jira] [Created] (SPARK-22130) UTF8String.trim() inefficiently scans all white-space string twice.

Kazuaki Ishizaki created SPARK-22130: Summary: UTF8String.trim() inefficiently scans all white-space string twice. Key: SPARK-22130 URL: https://issues.apache.org/jira/browse/SPARK-22130 Project:

[jira] [Comment Edited] (SPARK-22105) Dataframe has poor performance when computing on many columns with codegen

[ https://issues.apache.org/jira/browse/SPARK-22105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16176655#comment-16176655 ] Kazuaki Ishizaki edited comment on SPARK-22105 at 9/22/17 4:22 PM: --- Can

[jira] [Commented] (SPARK-22105) Dataframe has poor performance when computing on many columns with codegen

[ https://issues.apache.org/jira/browse/SPARK-22105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16176655#comment-16176655 ] Kazuaki Ishizaki commented on SPARK-22105: -- Can this PR at

[jira] [Commented] (SPARK-22000) org.codehaus.commons.compiler.CompileException: toString method is not declared

[ https://issues.apache.org/jira/browse/SPARK-22000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170319#comment-16170319 ] Kazuaki Ishizaki commented on SPARK-22000: -- If there is no sample code, it may take a long time

[jira] [Commented] (SPARK-22033) BufferHolder size checks should account for the specific VM array size limitations

[ https://issues.apache.org/jira/browse/SPARK-22033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169318#comment-16169318 ] Kazuaki Ishizaki commented on SPARK-22033: -- I think {{ColumnVector}} and

[jira] [Commented] (SPARK-22000) org.codehaus.commons.compiler.CompileException: toString method is not declared

[ https://issues.apache.org/jira/browse/SPARK-22000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165902#comment-16165902 ] Kazuaki Ishizaki commented on SPARK-22000: -- Thank you for good suggestion. I will try to use

[jira] [Commented] (SPARK-22000) org.codehaus.commons.compiler.CompileException: toString method is not declared

[ https://issues.apache.org/jira/browse/SPARK-22000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165883#comment-16165883 ] Kazuaki Ishizaki commented on SPARK-22000: -- It would be good to generate

[jira] [Commented] (SPARK-21907) NullPointerException in UnsafeExternalSorter.spill()

[ https://issues.apache.org/jira/browse/SPARK-21907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158995#comment-16158995 ] Kazuaki Ishizaki commented on SPARK-21907: -- If you cannot provide a repro, could you please run

[jira] [Commented] (SPARK-21905) ClassCastException when call sqlContext.sql on temp table

[ https://issues.apache.org/jira/browse/SPARK-21905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158496#comment-16158496 ] Kazuaki Ishizaki commented on SPARK-21905: -- While I ran the following code (I do not have

[jira] [Commented] (SPARK-21946) Flaky test: InMemoryCatalogedDDLSuite.`alter table: rename cached table`

[ https://issues.apache.org/jira/browse/SPARK-21946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158044#comment-16158044 ] Kazuaki Ishizaki commented on SPARK-21946: -- If someone has not worked for this, I will create a

[jira] [Commented] (SPARK-21907) NullPointerException in UnsafeExternalSorter.spill()

[ https://issues.apache.org/jira/browse/SPARK-21907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156015#comment-16156015 ] Kazuaki Ishizaki commented on SPARK-21907: -- Thank you for your report. Could you please attach a

[jira] [Commented] (SPARK-21894) Some Netty errors do not propagate to the top level driver

[ https://issues.apache.org/jira/browse/SPARK-21894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151930#comment-16151930 ] Kazuaki Ishizaki commented on SPARK-21894: -- Thank you for reporting this issue. Could you please

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141650#comment-16141650 ] Kazuaki Ishizaki commented on SPARK-18016: -- The issue {{Caused by:

[jira] [Commented] (SPARK-21828) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB...again

[ https://issues.apache.org/jira/browse/SPARK-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141361#comment-16141361 ] Kazuaki Ishizaki commented on SPARK-21828: -- Thank you for your report. Some fixes solved this

[jira] [Commented] (SPARK-21828) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB...again

[ https://issues.apache.org/jira/browse/SPARK-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140282#comment-16140282 ] Kazuaki Ishizaki commented on SPARK-21828: -- Thank you for reporting a problem. First, IIUC, this

[jira] [Commented] (SPARK-21750) Use arrow 0.6.0

[ https://issues.apache.org/jira/browse/SPARK-21750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136920#comment-16136920 ] Kazuaki Ishizaki commented on SPARK-21750: -- Closed this since to upgrade Arrow requires to

[jira] [Closed] (SPARK-21750) Use arrow 0.6.0

[ https://issues.apache.org/jira/browse/SPARK-21750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki closed SPARK-21750. Resolution: Won't Fix > Use arrow 0.6.0 > --- > > Key:

[jira] [Commented] (SPARK-21794) exception about reading task serial data(broadcast) value when the storage memory is not enough to unroll

[ https://issues.apache.org/jira/browse/SPARK-21794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134440#comment-16134440 ] Kazuaki Ishizaki commented on SPARK-21794: -- Thank you for reporting this issue. Could you please

[jira] [Commented] (SPARK-21776) How to use the memory-mapped file on Spark??

[ https://issues.apache.org/jira/browse/SPARK-21776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16131681#comment-16131681 ] Kazuaki Ishizaki commented on SPARK-21776: -- Is this a question? It this is a kind of questions,

[jira] [Commented] (SPARK-21720) Filter predicate with many conditions throw stackoverflow error

[ https://issues.apache.org/jira/browse/SPARK-21720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130231#comment-16130231 ] Kazuaki Ishizaki commented on SPARK-21720: -- I identified issues in {{predicates.scala}}. I am

[jira] [Created] (SPARK-21751) CodeGeneraor.splitExpressions counts code size more precisely

Kazuaki Ishizaki created SPARK-21751: Summary: CodeGeneraor.splitExpressions counts code size more precisely Key: SPARK-21751 URL: https://issues.apache.org/jira/browse/SPARK-21751 Project: Spark

[jira] [Commented] (SPARK-21750) Use arrow 0.6.0

[ https://issues.apache.org/jira/browse/SPARK-21750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129133#comment-16129133 ] Kazuaki Ishizaki commented on SPARK-21750: -- Waiting for it on mvnrepository > Use arrow 0.6.0 >

[jira] [Created] (SPARK-21750) Use arrow 0.6.0

Kazuaki Ishizaki created SPARK-21750: Summary: Use arrow 0.6.0 Key: SPARK-21750 URL: https://issues.apache.org/jira/browse/SPARK-21750 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-21720) Filter predicate with many conditions throw stackoverflow error

[ https://issues.apache.org/jira/browse/SPARK-21720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127477#comment-16127477 ] Kazuaki Ishizaki edited comment on SPARK-21720 at 8/15/17 4:26 PM: --- In

[jira] [Commented] (SPARK-21720) Filter predicate with many conditions throw stackoverflow error

[ https://issues.apache.org/jira/browse/SPARK-21720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127477#comment-16127477 ] Kazuaki Ishizaki commented on SPARK-21720: -- In this case, to add JVM option {{-Xss512m}}

[jira] [Commented] (SPARK-21720) Filter predicate with many conditions throw stackoverflow error

[ https://issues.apache.org/jira/browse/SPARK-21720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16125253#comment-16125253 ] Kazuaki Ishizaki commented on SPARK-21720: -- I confirmed that this occurs in the master branch. I

[jira] [Comment Edited] (SPARK-19372) Code generation for Filter predicate including many OR conditions exceeds JVM method size limit

[ https://issues.apache.org/jira/browse/SPARK-19372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124982#comment-16124982 ] Kazuaki Ishizaki edited comment on SPARK-19372 at 8/13/17 5:05 PM: ---

[jira] [Commented] (SPARK-19372) Code generation for Filter predicate including many OR conditions exceeds JVM method size limit

[ https://issues.apache.org/jira/browse/SPARK-19372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124982#comment-16124982 ] Kazuaki Ishizaki commented on SPARK-19372: -- [~srinivasanm] I can reproduce this issue by using

[jira] [Commented] (SPARK-19372) Code generation for Filter predicate including many OR conditions exceeds JVM method size limit

[ https://issues.apache.org/jira/browse/SPARK-19372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16124856#comment-16124856 ] Kazuaki Ishizaki commented on SPARK-19372: -- Thank you for letting us know the problem. I

[jira] [Commented] (SPARK-21276) Update lz4-java to remove custom LZ4BlockInputStream

[ https://issues.apache.org/jira/browse/SPARK-21276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16118402#comment-16118402 ] Kazuaki Ishizaki commented on SPARK-21276: -- Is it better to update affected version? > Update

[jira] [Commented] (SPARK-21390) Dataset filter api inconsistency

[ https://issues.apache.org/jira/browse/SPARK-21390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110576#comment-16110576 ] Kazuaki Ishizaki commented on SPARK-21390: -- Thank you very much for pointing out the good JIRA

[jira] [Commented] (SPARK-21591) Implement treeAggregate on Dataset API

[ https://issues.apache.org/jira/browse/SPARK-21591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108644#comment-16108644 ] Kazuaki Ishizaki commented on SPARK-21591: -- I like this idea > Implement treeAggregate on

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16104284#comment-16104284 ] Kazuaki Ishizaki commented on SPARK-18016: -- [~jamcon] Thank you reporting the problem. We fixed

[jira] [Commented] (SPARK-21496) Support codegen for TakeOrderedAndProjectExec

[ https://issues.apache.org/jira/browse/SPARK-21496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099840#comment-16099840 ] Kazuaki Ishizaki commented on SPARK-21496: -- Is there any good benchmark program for this? >

[jira] [Commented] (SPARK-21517) Fetch local data via block manager cause oom

[ https://issues.apache.org/jira/browse/SPARK-21517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099654#comment-16099654 ] Kazuaki Ishizaki commented on SPARK-21517: -- Does it occur in Spark 2.2? > Fetch local data via

[jira] [Commented] (SPARK-21501) Spark shuffle index cache size should be memory based

[ https://issues.apache.org/jira/browse/SPARK-21501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099351#comment-16099351 ] Kazuaki Ishizaki commented on SPARK-21501: -- I see. I misunderstood the description. You expect

[jira] [Closed] (SPARK-21387) org.apache.spark.memory.TaskMemoryManager.allocatePage causes OOM

[ https://issues.apache.org/jira/browse/SPARK-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki closed SPARK-21387. Resolution: Cannot Reproduce > org.apache.spark.memory.TaskMemoryManager.allocatePage

[jira] [Closed] (SPARK-21387) org.apache.spark.memory.TaskMemoryManager.allocatePage causes OOM

[ https://issues.apache.org/jira/browse/SPARK-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki closed SPARK-21387. Resolution: Fixed > org.apache.spark.memory.TaskMemoryManager.allocatePage causes OOM >

[jira] [Reopened] (SPARK-21387) org.apache.spark.memory.TaskMemoryManager.allocatePage causes OOM

[ https://issues.apache.org/jira/browse/SPARK-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki reopened SPARK-21387: -- > org.apache.spark.memory.TaskMemoryManager.allocatePage causes OOM >

[jira] [Commented] (SPARK-21387) org.apache.spark.memory.TaskMemoryManager.allocatePage causes OOM

[ https://issues.apache.org/jira/browse/SPARK-21387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098545#comment-16098545 ] Kazuaki Ishizaki commented on SPARK-21387: -- While I got OOM in my unit test, I have to

[jira] [Commented] (SPARK-21501) Spark shuffle index cache size should be memory based

[ https://issues.apache.org/jira/browse/SPARK-21501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098531#comment-16098531 ] Kazuaki Ishizaki commented on SPARK-21501: -- I guess that to use Spark 2.1 or later version

[jira] [Created] (SPARK-21516) overriding afterEach() in DatasetCacheSuite must call super.afterEach()

Kazuaki Ishizaki created SPARK-21516: Summary: overriding afterEach() in DatasetCacheSuite must call super.afterEach() Key: SPARK-21516 URL: https://issues.apache.org/jira/browse/SPARK-21516

[jira] [Comment Edited] (SPARK-21512) DatasetCacheSuite needs to execute unpersistent after executing peristent

[ https://issues.apache.org/jira/browse/SPARK-21512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097565#comment-16097565 ] Kazuaki Ishizaki edited comment on SPARK-21512 at 7/24/17 4:53 AM: ---

[jira] [Commented] (SPARK-21512) DatasetCacheSuite needs to execute unpersistent after executing peristent

[ https://issues.apache.org/jira/browse/SPARK-21512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097565#comment-16097565 ] Kazuaki Ishizaki commented on SPARK-21512: -- When {DatasetCacheSuite} is executed, the following

[jira] [Updated] (SPARK-21512) DatasetCacheSuite needs to execute unpersistent after executing peristent

[ https://issues.apache.org/jira/browse/SPARK-21512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-21512: - Summary: DatasetCacheSuite needs to execute unpersistent after executing peristent

[jira] [Updated] (SPARK-21512) DatasetCacheSuites need to execute unpersistent after executing peristent

[ https://issues.apache.org/jira/browse/SPARK-21512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-21512: - Summary: DatasetCacheSuites need to execute unpersistent after executing peristent

[jira] [Created] (SPARK-21512) DatasetCacheSuite need to execute unpersistent after executing peristent

Kazuaki Ishizaki created SPARK-21512: Summary: DatasetCacheSuite need to execute unpersistent after executing peristent Key: SPARK-21512 URL: https://issues.apache.org/jira/browse/SPARK-21512

[jira] [Updated] (SPARK-20822) Generate code to get value from ColumnVector in ColumnarBatch

[ https://issues.apache.org/jira/browse/SPARK-20822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-20822: - Summary: Generate code to get value from ColumnVector in ColumnarBatch (was: Generate

[jira] [Updated] (SPARK-20822) Generate code to get value from CachedBatchColumnVector in ColumnarBatch

[ https://issues.apache.org/jira/browse/SPARK-20822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-20822: - Summary: Generate code to get value from CachedBatchColumnVector in ColumnarBatch (was:

[jira] [Commented] (SPARK-21443) Very long planning duration for queries with lots of operations

[ https://issues.apache.org/jira/browse/SPARK-21443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090242#comment-16090242 ] Kazuaki Ishizaki commented on SPARK-21443: -- These two optimizations

[jira] [Commented] (SPARK-21415) Triage scapegoat warnings, part 1

[ https://issues.apache.org/jira/browse/SPARK-21415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089915#comment-16089915 ] Kazuaki Ishizaki commented on SPARK-21415: -- I see. When another JIRA will happen for these

[jira] [Commented] (SPARK-21390) Dataset filter api inconsistency

[ https://issues.apache.org/jira/browse/SPARK-21390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089012#comment-16089012 ] Kazuaki Ishizaki commented on SPARK-21390: -- cc: [~ueshin] Is there any thought on this? >

[jira] [Commented] (SPARK-21390) Dataset filter api inconsistency

[ https://issues.apache.org/jira/browse/SPARK-21390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088977#comment-16088977 ] Kazuaki Ishizaki commented on SPARK-21390: -- When I ran the following test suite in

[jira] [Comment Edited] (SPARK-21418) NoSuchElementException: None.get on DataFrame.rdd

[ https://issues.apache.org/jira/browse/SPARK-21418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088659#comment-16088659 ] Kazuaki Ishizaki edited comment on SPARK-21418 at 7/16/17 8:25 AM: --- I

[jira] [Comment Edited] (SPARK-21393) spark (pyspark) crashes unpredictably when using show() or toPandas()

[ https://issues.apache.org/jira/browse/SPARK-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088660#comment-16088660 ] Kazuaki Ishizaki edited comment on SPARK-21393 at 7/15/17 4:53 PM: --- I

[jira] [Commented] (SPARK-21393) spark (pyspark) crashes unpredictably when using show() or toPandas()

[ https://issues.apache.org/jira/browse/SPARK-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088660#comment-16088660 ] Kazuaki Ishizaki commented on SPARK-21393: -- I confirmed that this python program works well

[jira] [Comment Edited] (SPARK-21418) NoSuchElementException: None.get on DataFrame.rdd

[ https://issues.apache.org/jira/browse/SPARK-21418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088659#comment-16088659 ] Kazuaki Ishizaki edited comment on SPARK-21418 at 7/15/17 4:43 PM: --- I

[jira] [Commented] (SPARK-21418) NoSuchElementException: None.get on DataFrame.rdd

[ https://issues.apache.org/jira/browse/SPARK-21418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088659#comment-16088659 ] Kazuaki Ishizaki commented on SPARK-21418: -- I am curious why

[jira] [Commented] (SPARK-21393) spark (pyspark) crashes unpredictably when using show() or toPandas()

[ https://issues.apache.org/jira/browse/SPARK-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087391#comment-16087391 ] Kazuaki Ishizaki commented on SPARK-21393: -- Not yet, however I created a patch not to cause

[jira] [Commented] (SPARK-21415) Triage scapegoat warnings, part 1

[ https://issues.apache.org/jira/browse/SPARK-21415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087097#comment-16087097 ] Kazuaki Ishizaki commented on SPARK-21415: -- Thank you. Is it better to create an umbrella JIRA

[jira] [Commented] (SPARK-21413) Multiple projections with CASE WHEN fails to run generated codes

[ https://issues.apache.org/jira/browse/SPARK-21413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086927#comment-16086927 ] Kazuaki Ishizaki commented on SPARK-21413: -- Thank you for preparing a good repro. I can

[jira] [Commented] (SPARK-21393) spark (pyspark) crashes unpredictably when using show() or toPandas()

[ https://issues.apache.org/jira/browse/SPARK-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086254#comment-16086254 ] Kazuaki Ishizaki commented on SPARK-21393: -- This program can cause the same exception {code}

[jira] [Comment Edited] (SPARK-21393) spark (pyspark) crashes unpredictably when using show() or toPandas()

[ https://issues.apache.org/jira/browse/SPARK-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086189#comment-16086189 ] Kazuaki Ishizaki edited comment on SPARK-21393 at 7/13/17 6:39 PM: ---

[jira] [Commented] (SPARK-21393) spark (pyspark) crashes unpredictably when using show() or toPandas()

[ https://issues.apache.org/jira/browse/SPARK-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086189#comment-16086189 ] Kazuaki Ishizaki commented on SPARK-21393: -- Thank you for uploading files. When I insert

[jira] [Updated] (SPARK-21393) spark (pyspark) crashes unpredictably when using show() or toPandas()

[ https://issues.apache.org/jira/browse/SPARK-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-21393: - Affects Version/s: 2.2.0 > spark (pyspark) crashes unpredictably when using show() or

[jira] [Commented] (SPARK-21391) Cannot convert a Seq of Map whose value type is again a seq, into a dataset

[ https://issues.apache.org/jira/browse/SPARK-21391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085939#comment-16085939 ] Kazuaki Ishizaki commented on SPARK-21391: -- I created [a

[jira] [Updated] (SPARK-21390) Dataset filter api inconsistency

[ https://issues.apache.org/jira/browse/SPARK-21390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-21390: - Affects Version/s: 2.1.0 > Dataset filter api inconsistency >

[jira] [Updated] (SPARK-21390) Dataset filter api inconsistency

[ https://issues.apache.org/jira/browse/SPARK-21390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-21390: - Affects Version/s: 2.2.0 > Dataset filter api inconsistency >

[jira] [Commented] (SPARK-21393) spark (pyspark) crashes unpredictably when using show() or toPandas()

[ https://issues.apache.org/jira/browse/SPARK-21393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085726#comment-16085726 ] Kazuaki Ishizaki commented on SPARK-21393: -- This program seems to require 7 csv files to execute

[jira] [Commented] (SPARK-21391) Cannot convert a Seq of Map whose value type is again a seq, into a dataset

[ https://issues.apache.org/jira/browse/SPARK-21391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085342#comment-16085342 ] Kazuaki Ishizaki commented on SPARK-21391: -- [~neelrr] Do you want to have fix in future release

[jira] [Comment Edited] (SPARK-21391) Cannot convert a Seq of Map whose value type is again a seq, into a dataset

[ https://issues.apache.org/jira/browse/SPARK-21391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085099#comment-16085099 ] Kazuaki Ishizaki edited comment on SPARK-21391 at 7/13/17 3:42 AM: ---

[jira] [Commented] (SPARK-21391) Cannot convert a Seq of Map whose value type is again a seq, into a dataset

[ https://issues.apache.org/jira/browse/SPARK-21391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085099#comment-16085099 ] Kazuaki Ishizaki commented on SPARK-21391: -- [~hyukjin.kwon] I think that

[jira] [Comment Edited] (SPARK-21391) Cannot convert a Seq of Map whose value type is again a seq, into a dataset

[ https://issues.apache.org/jira/browse/SPARK-21391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084333#comment-16084333 ] Kazuaki Ishizaki edited comment on SPARK-21391 at 7/12/17 5:19 PM: ---

[jira] [Comment Edited] (SPARK-21391) Cannot convert a Seq of Map whose value type is again a seq, into a dataset

[ https://issues.apache.org/jira/browse/SPARK-21391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084333#comment-16084333 ] Kazuaki Ishizaki edited comment on SPARK-21391 at 7/12/17 5:19 PM: ---

[jira] [Commented] (SPARK-21391) Cannot convert a Seq of Map whose value type is again a seq, into a dataset

[ https://issues.apache.org/jira/browse/SPARK-21391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084333#comment-16084333 ] Kazuaki Ishizaki commented on SPARK-21391: -- This program works with the master. {code}

[jira] [Commented] (SPARK-21390) Dataset filter api inconsistency

[ https://issues.apache.org/jira/browse/SPARK-21390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084306#comment-16084306 ] Kazuaki Ishizaki commented on SPARK-21390: -- Another interesting results with Spark-2.2: On IDE

[jira] [Comment Edited] (SPARK-21390) Dataset filter api inconsistency

[ https://issues.apache.org/jira/browse/SPARK-21390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084306#comment-16084306 ] Kazuaki Ishizaki edited comment on SPARK-21390 at 7/12/17 5:09 PM: ---

[jira] [Commented] (SPARK-21390) Dataset filter api inconsistency

[ https://issues.apache.org/jira/browse/SPARK-21390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16084266#comment-16084266 ] Kazuaki Ishizaki commented on SPARK-21390: -- Thank you for reporting this. I can reproduce this

[jira] [Created] (SPARK-21387) org.apache.spark.memory.TaskMemoryManager.allocatePage causes OOM

Kazuaki Ishizaki created SPARK-21387: Summary: org.apache.spark.memory.TaskMemoryManager.allocatePage causes OOM Key: SPARK-21387 URL: https://issues.apache.org/jira/browse/SPARK-21387 Project:

[jira] [Comment Edited] (SPARK-21373) Update Jetty to 9.3.20.v20170531

[ https://issues.apache.org/jira/browse/SPARK-21373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082426#comment-16082426 ] Kazuaki Ishizaki edited comment on SPARK-21373 at 7/11/17 3:56 PM: ---

[jira] [Updated] (SPARK-21373) Update Jetty to 9.3.20.v20170531

[ https://issues.apache.org/jira/browse/SPARK-21373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-21373: - Summary: Update Jetty to 9.3.20.v20170531 (was: Update Jetty to 9.3.20.v20170531 to fix

[jira] [Commented] (SPARK-21373) Update Jetty to 9.3.20.v20170531

[ https://issues.apache.org/jira/browse/SPARK-21373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082426#comment-16082426 ] Kazuaki Ishizaki commented on SPARK-21373: -- Since I have not clarified, I changed the title. I

[jira] [Created] (SPARK-21373) Update Jetty to 9.3.20.v20170531 to fix CVE-2017-9735

Kazuaki Ishizaki created SPARK-21373: Summary: Update Jetty to 9.3.20.v20170531 to fix CVE-2017-9735 Key: SPARK-21373 URL: https://issues.apache.org/jira/browse/SPARK-21373 Project: Spark

[jira] [Updated] (SPARK-21373) Update Jetty to 9.3.20.v20170531 to fix CVE-2017-9735

[ https://issues.apache.org/jira/browse/SPARK-21373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-21373: - Description: This is derived from https://issues.apache.org/jira/browse/FELIX-5664.

[jira] [Commented] (SPARK-21364) IndexOutOfBoundsException on equality check of two complex array elements

[ https://issues.apache.org/jira/browse/SPARK-21364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16080963#comment-16080963 ] Kazuaki Ishizaki commented on SPARK-21364: -- When I ran the following test case that is derived

[jira] [Commented] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079032#comment-16079032 ] Kazuaki Ishizaki commented on SPARK-21337: -- I cannot reproduce this by using the latest or v2.1

[jira] [Commented] (SPARK-21344) BinaryType comparison does signed byte array comparison

[ https://issues.apache.org/jira/browse/SPARK-21344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16079010#comment-16079010 ] Kazuaki Ishizaki commented on SPARK-21344: -- I will work for this if anyone has finished a PR. >

[jira] [Commented] (SPARK-21337) SQL which has large ‘case when’ expressions may cause code generation beyond 64KB

[ https://issues.apache.org/jira/browse/SPARK-21337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16077952#comment-16077952 ] Kazuaki Ishizaki commented on SPARK-21337: -- In the master branch, I cannot see a huge dump and

[jira] [Comment Edited] (SPARK-21271) UnsafeRow.hashCode assertion when sizeInBytes not multiple of 8

[ https://issues.apache.org/jira/browse/SPARK-21271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070973#comment-16070973 ] Kazuaki Ishizaki edited comment on SPARK-21271 at 7/1/17 3:18 AM: -- I

[jira] [Commented] (SPARK-21271) UnsafeRow.hashCode assertion when sizeInBytes not multiple of 8

[ https://issues.apache.org/jira/browse/SPARK-21271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070973#comment-16070973 ] Kazuaki Ishizaki commented on SPARK-21271: -- I see. I will work for this. Thank you for letting

<    2   3   4   5   6   7   8   9   10   >