[jira] [Issue Comment Deleted] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kohki Nishio updated SPARK-36827: - Comment: was deleted (was: https://github.com/apache/spark/pull/34090) > Task/Stage/Job data re

[jira] [Commented] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419548#comment-17419548 ] Kohki Nishio commented on SPARK-36827: -- https://github.com/apache/spark/pull/34090

[jira] [Commented] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418849#comment-17418849 ] Kohki Nishio commented on SPARK-36827: -- i still need to ramp up the process for get

[jira] [Comment Edited] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418832#comment-17418832 ] Kohki Nishio edited comment on SPARK-36827 at 9/22/21, 9:04 PM: --

[jira] [Commented] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418832#comment-17418832 ] Kohki Nishio commented on SPARK-36827: -- this line https://github.com/apache/spark/b

[jira] [Commented] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418829#comment-17418829 ] Kohki Nishio commented on SPARK-36827: -- I do have multiple heap dumps, however it w

[jira] [Updated] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kohki Nishio updated SPARK-36827: - Description: Noticing memory-leak like behavior, steady increase of heap after GC and eventuall

[jira] [Updated] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kohki Nishio updated SPARK-36827: - Description: Noticing memory-leak like behavior, steady increase of heap after GC and eventuall

[jira] [Commented] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418742#comment-17418742 ] Kohki Nishio commented on SPARK-36827: -- one thread shows slightly different codepoi

[jira] [Updated] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kohki Nishio updated SPARK-36827: - Attachment: mem1.txt > Task/Stage/Job data remain in memory leads memory leak >

[jira] [Updated] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

[ https://issues.apache.org/jira/browse/SPARK-36827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kohki Nishio updated SPARK-36827: - Attachment: worker.txt > Task/Stage/Job data remain in memory leads memory leak > --

[jira] [Created] (SPARK-36827) Task/Stage/Job data remain in memory leads memory leak

Kohki Nishio created SPARK-36827: Summary: Task/Stage/Job data remain in memory leads memory leak Key: SPARK-36827 URL: https://issues.apache.org/jira/browse/SPARK-36827 Project: Spark Issue

[jira] [Updated] (SPARK-36733) Perf issue in SchemaPruning when a struct has many fields

[ https://issues.apache.org/jira/browse/SPARK-36733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kohki Nishio updated SPARK-36733: - Summary: Perf issue in SchemaPruning when a struct has many fields (was: Perf issue in SchemaPr

[jira] [Comment Edited] (SPARK-36733) Perf issue in SchemaPruning when a struct has million fields

[ https://issues.apache.org/jira/browse/SPARK-36733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413882#comment-17413882 ] Kohki Nishio edited comment on SPARK-36733 at 9/13/21, 3:23 AM: --

[jira] [Commented] (SPARK-36733) Perf issue in SchemaPruning when a struct has million fields

[ https://issues.apache.org/jira/browse/SPARK-36733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413882#comment-17413882 ] Kohki Nishio commented on SPARK-36733: -- [https://github.com/apache/spark/blob/maste

[jira] [Created] (SPARK-36733) Perf issue in SchemaPruning when a struct has million fields

Kohki Nishio created SPARK-36733: Summary: Perf issue in SchemaPruning when a struct has million fields Key: SPARK-36733 URL: https://issues.apache.org/jira/browse/SPARK-36733 Project: Spark

[jira] [Commented] (SPARK-22303) [SQL] Getting java.sql.SQLException: Unsupported type 101 for BINARY_DOUBLE

[ https://issues.apache.org/jira/browse/SPARK-22303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16214163#comment-16214163 ] Kohki Nishio commented on SPARK-22303: -- https://github.com/apache/spark/pull/19548

[jira] [Reopened] (SPARK-22303) [SQL] Getting java.sql.SQLException: Unsupported type 101 for BINARY_DOUBLE

[ https://issues.apache.org/jira/browse/SPARK-22303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kohki Nishio reopened SPARK-22303: -- Spark supports Oracle specific types, I'm working on a PR, please keep this open > [SQL] Getting j

[jira] [Created] (SPARK-22303) [SQL] Getting java.sql.SQLException: Unsupported type 101 for BINARY_DOUBLE

Kohki Nishio created SPARK-22303: Summary: [SQL] Getting java.sql.SQLException: Unsupported type 101 for BINARY_DOUBLE Key: SPARK-22303 URL: https://issues.apache.org/jira/browse/SPARK-22303 Project:

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15884809#comment-15884809 ] Kohki Nishio commented on SPARK-19675: -- here's a pr for this issue https://github.co

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877350#comment-15877350 ] Kohki Nishio commented on SPARK-19675: -- Turned out that this is a dup of https://is

[jira] [Comment Edited] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877290#comment-15877290 ] Kohki Nishio edited comment on SPARK-19675 at 2/22/17 2:11 AM:

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877290#comment-15877290 ] Kohki Nishio commented on SPARK-19675: -- not in a separate process, I'm running it in

[jira] [Comment Edited] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877273#comment-15877273 ] Kohki Nishio edited comment on SPARK-19675 at 2/22/17 2:00 AM:

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877273#comment-15877273 ] Kohki Nishio commented on SPARK-19675: -- [~zsxwing] That's only when findClass is use

[jira] [Comment Edited] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877256#comment-15877256 ] Kohki Nishio edited comment on SPARK-19675 at 2/22/17 1:58 AM:

[jira] [Comment Edited] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877256#comment-15877256 ] Kohki Nishio edited comment on SPARK-19675 at 2/22/17 1:54 AM:

[jira] [Comment Edited] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877256#comment-15877256 ] Kohki Nishio edited comment on SPARK-19675 at 2/22/17 1:54 AM:

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877256#comment-15877256 ] Kohki Nishio commented on SPARK-19675: -- My application does notebook thing, this is

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877235#comment-15877235 ] Kohki Nishio commented on SPARK-19675: -- I don't think this is something clearly a wr

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877234#comment-15877234 ] Kohki Nishio commented on SPARK-19675: -- Overriding loadClass fixes my scenario {cod

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877219#comment-15877219 ] Kohki Nishio commented on SPARK-19675: -- First of all, sbt (0.13) is built with Scala

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15875526#comment-15875526 ] Kohki Nishio commented on SPARK-19675: -- Spark is picking up a class from difference

[jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15875504#comment-15875504 ] Kohki Nishio commented on SPARK-19675: -- Here's a simple repro, both should say '/tmp

[jira] [Reopened] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

[ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kohki Nishio reopened SPARK-19675: -- SBT can manage 2.11 applications, an applicaiton runs completely with 2.11 binary however SBT is b

[jira] [Created] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader

Kohki Nishio created SPARK-19675: Summary: ExecutorClassLoader loads classes from SystemClassLoader Key: SPARK-19675 URL: https://issues.apache.org/jira/browse/SPARK-19675 Project: Spark Issu