[jira] [Commented] (SPARK-1154) Spark fills up disk with app-* folders

[ https://issues.apache.org/jira/browse/SPARK-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13999562#comment-13999562 ] Mingyu Kim commented on SPARK-1154: --- I looked at the commit, and it seems like it wipes

[jira] [Commented] (SPARK-1154) Spark fills up disk with app-* folders

[ https://issues.apache.org/jira/browse/SPARK-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14000905#comment-14000905 ] Mingyu Kim commented on SPARK-1154: --- Yes, I'd be happy to take it. Please feel free to

[jira] [Comment Edited] (SPARK-1860) Standalone Worker cleanup should not clean up running applications

[ https://issues.apache.org/jira/browse/SPARK-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004747#comment-14004747 ] Mingyu Kim edited comment on SPARK-1860 at 5/21/14 3:10 PM:

[jira] [Commented] (SPARK-1860) Standalone Worker cleanup should not clean up running applications

[ https://issues.apache.org/jira/browse/SPARK-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14004747#comment-14004747 ] Mingyu Kim commented on SPARK-1860: --- [~aash], is there a reliable way to check folder is

[jira] [Commented] (SPARK-2292) NullPointerException in JavaPairRDD.mapToPair

[ https://issues.apache.org/jira/browse/SPARK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046791#comment-14046791 ] Mingyu Kim commented on SPARK-2292: --- Given that this was added to better support lambda

[jira] [Commented] (SPARK-2292) NullPointerException in JavaPairRDD.mapToPair

[ https://issues.apache.org/jira/browse/SPARK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047009#comment-14047009 ] Mingyu Kim commented on SPARK-2292: --- Interesting. I could repro the same behavior as

[jira] [Commented] (SPARK-2292) NullPointerException in JavaPairRDD.mapToPair

[ https://issues.apache.org/jira/browse/SPARK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047073#comment-14047073 ] Mingyu Kim commented on SPARK-2292: --- For now, the work around would be to run the

[jira] [Commented] (SPARK-1860) Standalone Worker cleanup should not clean up running applications

[ https://issues.apache.org/jira/browse/SPARK-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047074#comment-14047074 ] Mingyu Kim commented on SPARK-1860: --- [~pwendell], would there be an easy way to tell

[jira] [Created] (SPARK-3050) Spark program running with 1.0.2 jar cannot run against a 1.0.1 cluster

Mingyu Kim created SPARK-3050: - Summary: Spark program running with 1.0.2 jar cannot run against a 1.0.1 cluster Key: SPARK-3050 URL: https://issues.apache.org/jira/browse/SPARK-3050 Project: Spark

[jira] [Created] (SPARK-3996) Shade Jetty in Spark deliverables

Mingyu Kim created SPARK-3996: - Summary: Shade Jetty in Spark deliverables Key: SPARK-3996 URL: https://issues.apache.org/jira/browse/SPARK-3996 Project: Spark Issue Type: Task

[jira] [Commented] (SPARK-3996) Shade Jetty in Spark deliverables

[ https://issues.apache.org/jira/browse/SPARK-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177697#comment-14177697 ] Mingyu Kim commented on SPARK-3996: --- Cool. [~mcheah] will take this. Can you assign this

[jira] [Created] (SPARK-4906) Spark master OOMs with exception stack trace stored in JobProgressListener

Mingyu Kim created SPARK-4906: - Summary: Spark master OOMs with exception stack trace stored in JobProgressListener Key: SPARK-4906 URL: https://issues.apache.org/jira/browse/SPARK-4906 Project: Spark

[jira] [Commented] (SPARK-4906) Spark master OOMs with exception stack trace stored in JobProgressListener

[ https://issues.apache.org/jira/browse/SPARK-4906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14277849#comment-14277849 ] Mingyu Kim commented on SPARK-4906: --- typically once a few tasks have failed the stage

[jira] [Commented] (SPARK-1823) ExternalAppendOnlyMap can still OOM if one key is very large

[ https://issues.apache.org/jira/browse/SPARK-1823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14323768#comment-14323768 ] Mingyu Kim commented on SPARK-1823: --- [~andrewor14], is anyone working on this actively?

[jira] [Created] (SPARK-5915) Spillable should check every N bytes rather than every 32 elements

Mingyu Kim created SPARK-5915: - Summary: Spillable should check every N bytes rather than every 32 elements Key: SPARK-5915 URL: https://issues.apache.org/jira/browse/SPARK-5915 Project: Spark

[jira] [Commented] (SPARK-4808) Spark fails to spill with small number of large objects

[ https://issues.apache.org/jira/browse/SPARK-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364016#comment-14364016 ] Mingyu Kim commented on SPARK-4808: --- [~andrewor14], should this now be closed with the

[jira] [Created] (SPARK-6619) Jar cache on Executors should use file content hash as the key

Mingyu Kim created SPARK-6619: - Summary: Jar cache on Executors should use file content hash as the key Key: SPARK-6619 URL: https://issues.apache.org/jira/browse/SPARK-6619 Project: Spark

[jira] [Commented] (SPARK-6619) Improve Jar caching on executors

[ https://issues.apache.org/jira/browse/SPARK-6619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387783#comment-14387783 ] Mingyu Kim commented on SPARK-6619: --- [~li-zhihui], [~joshrosen] since you worked on

[jira] [Updated] (SPARK-6619) Improve Jar caching on executors

[ https://issues.apache.org/jira/browse/SPARK-6619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingyu Kim updated SPARK-6619: -- Description: Taking SPARK-2713 one step further so that - The cached jars can be used by multiple

[jira] [Updated] (SPARK-6619) Jar cache on Executors should use file content hash as the key

[ https://issues.apache.org/jira/browse/SPARK-6619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingyu Kim updated SPARK-6619: -- Description: Taking SPARK-2713 one step further so that the cached jars can be used by multiple

[jira] [Created] (SPARK-8413) DirectParquetOutputCommitter doesn't clean up the file on task failure

Mingyu Kim created SPARK-8413: - Summary: DirectParquetOutputCommitter doesn't clean up the file on task failure Key: SPARK-8413 URL: https://issues.apache.org/jira/browse/SPARK-8413 Project: Spark

[jira] [Created] (SPARK-11081) Shade Jersey dependency to work around the compatibility issue with Jersey2

Mingyu Kim created SPARK-11081: -- Summary: Shade Jersey dependency to work around the compatibility issue with Jersey2 Key: SPARK-11081 URL: https://issues.apache.org/jira/browse/SPARK-11081 Project:

[jira] [Commented] (SPARK-7917) Spark doesn't clean up Application Directories (local dirs)

[ https://issues.apache.org/jira/browse/SPARK-7917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618179#comment-14618179 ] Mingyu Kim commented on SPARK-7917: --- This looks like a duplicate of SPARK-5970, which is

[jira] [Updated] (SPARK-10611) Configuration object thread safety issue in NewHadoopRDD

[ https://issues.apache.org/jira/browse/SPARK-10611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingyu Kim updated SPARK-10611: --- Attachment: Screen Shot 2015-09-09 at 12.58.13 PM.png > Configuration object thread safety issue in

[jira] [Created] (SPARK-10611) Configuration object thread safety issue in NewHadoopRDD

Mingyu Kim created SPARK-10611: -- Summary: Configuration object thread safety issue in NewHadoopRDD Key: SPARK-10611 URL: https://issues.apache.org/jira/browse/SPARK-10611 Project: Spark Issue

[jira] [Updated] (SPARK-10611) Configuration object thread safety issue in NewHadoopRDD

[ https://issues.apache.org/jira/browse/SPARK-10611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingyu Kim updated SPARK-10611: --- Attachment: (was: Screen Shot 2015-09-09 at 12.58.13 PM.png) > Configuration object thread

[jira] [Updated] (SPARK-10611) Configuration object thread safety issue in NewHadoopRDD

[ https://issues.apache.org/jira/browse/SPARK-10611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingyu Kim updated SPARK-10611: --- Attachment: Screen Shot 2015-09-15 at 12.20.38 AM.png > Configuration object thread safety issue in

[jira] [Created] (SPARK-10703) Physical filter operators should replace the general AND/OR/equality/etc with a special version that treats null as false

Mingyu Kim created SPARK-10703: -- Summary: Physical filter operators should replace the general AND/OR/equality/etc with a special version that treats null as false Key: SPARK-10703 URL:

[jira] [Commented] (SPARK-15974) Create a socket on YARN AM start-up

[ https://issues.apache.org/jira/browse/SPARK-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332749#comment-15332749 ] Mingyu Kim commented on SPARK-15974: I agree this is not ideal. A lot of times setting up a server

[jira] [Created] (SPARK-15974) Create a socket on YARN AM start-up

Mingyu Kim created SPARK-15974: -- Summary: Create a socket on YARN AM start-up Key: SPARK-15974 URL: https://issues.apache.org/jira/browse/SPARK-15974 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-10063) Remove DirectParquetOutputCommitter

[ https://issues.apache.org/jira/browse/SPARK-10063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229953#comment-15229953 ] Mingyu Kim commented on SPARK-10063: [~rxin], can you clarify why this is a problem even when