[jira] [Commented] (SPARK-29992) Make access to getActive public

[ https://issues.apache.org/jira/browse/SPARK-29992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16979794#comment-16979794 ] Nishkam Ravi commented on SPARK-29992: -- [https://github.com/apache/spark/pull/26631] > Make access

[jira] [Created] (SPARK-29992) Make access to getActive public

Nishkam Ravi created SPARK-29992: Summary: Make access to getActive public Key: SPARK-29992 URL: https://issues.apache.org/jira/browse/SPARK-29992 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-14140) Futures timeout exception in executor logs

[ https://issues.apache.org/jira/browse/SPARK-14140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15211122#comment-15211122 ] Nishkam Ravi commented on SPARK-14140: -- It only appears for yarn-cluster mode (and not yarn-client).

[jira] [Created] (SPARK-14140) Futures timeout exception in executor logs

Nishkam Ravi created SPARK-14140: Summary: Futures timeout exception in executor logs Key: SPARK-14140 URL: https://issues.apache.org/jira/browse/SPARK-14140 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-13796) Lock release errors occur frequently in executor logs

[ https://issues.apache.org/jira/browse/SPARK-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi resolved SPARK-13796. -- Resolution: Fixed > Lock release errors occur frequently in executor logs >

[jira] [Commented] (SPARK-13796) Lock release errors occur frequently in executor logs

[ https://issues.apache.org/jira/browse/SPARK-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15194171#comment-15194171 ] Nishkam Ravi commented on SPARK-13796: -- Looks like this issue has been fixed by a recent commit

[jira] [Commented] (SPARK-13796) Lock release errors occur frequently in executor logs

[ https://issues.apache.org/jira/browse/SPARK-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189957#comment-15189957 ] Nishkam Ravi commented on SPARK-13796: -- Would it make sense to change the condition to check for

[jira] [Commented] (SPARK-13796) Lock release errors occur frequently in executor logs

[ https://issues.apache.org/jira/browse/SPARK-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189930#comment-15189930 ] Nishkam Ravi commented on SPARK-13796: -- [~joshrosen] Retested with latest master. FWIW I haven't

[jira] [Commented] (SPARK-13796) Lock release errors occur frequently in executor logs

[ https://issues.apache.org/jira/browse/SPARK-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15189919#comment-15189919 ] Nishkam Ravi commented on SPARK-13796: -- Hey [~srowen] this doesn't seem to impact functionality. I

[jira] [Commented] (SPARK-13796) Lock release errors occur frequently in executor logs

[ https://issues.apache.org/jira/browse/SPARK-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188762#comment-15188762 ] Nishkam Ravi commented on SPARK-13796: -- Running with master from March 7th

[jira] [Created] (SPARK-13796) Lock release errors occur frequently in executor logs

Nishkam Ravi created SPARK-13796: Summary: Lock release errors occur frequently in executor logs Key: SPARK-13796 URL: https://issues.apache.org/jira/browse/SPARK-13796 Project: Spark Issue

[jira] [Created] (SPARK-12967) NettyRPC races with SparkContext.stop() and throws exception

Nishkam Ravi created SPARK-12967: Summary: NettyRPC races with SparkContext.stop() and throws exception Key: SPARK-12967 URL: https://issues.apache.org/jira/browse/SPARK-12967 Project: Spark

[jira] [Updated] (SPARK-11278) PageRank fails with unified memory manager

[ https://issues.apache.org/jira/browse/SPARK-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi updated SPARK-11278: - Attachment: executor_logs_legacyModeFalse.html > PageRank fails with unified memory manager >

[jira] [Updated] (SPARK-11278) PageRank fails with unified memory manager

[ https://issues.apache.org/jira/browse/SPARK-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi updated SPARK-11278: - Attachment: executor_log_legacyModeTrue.html > PageRank fails with unified memory manager >

[jira] [Commented] (SPARK-11278) PageRank fails with unified memory manager

[ https://issues.apache.org/jira/browse/SPARK-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014732#comment-15014732 ] Nishkam Ravi commented on SPARK-11278: -- Attaching executor logs for most recent runs against master

[jira] [Commented] (SPARK-11278) PageRank fails with unified memory manager

[ https://issues.apache.org/jira/browse/SPARK-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15012691#comment-15012691 ] Nishkam Ravi commented on SPARK-11278: -- [~andrewor14] This was last tested on Nov 11th, which would

[jira] [Resolved] (SPARK-11648) IllegalReferenceCountException in Spark workloads

[ https://issues.apache.org/jira/browse/SPARK-11648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi resolved SPARK-11648. -- Resolution: Fixed > IllegalReferenceCountException in Spark workloads >

[jira] [Commented] (SPARK-11648) IllegalReferenceCountException in Spark workloads

[ https://issues.apache.org/jira/browse/SPARK-11648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15004976#comment-15004976 ] Nishkam Ravi commented on SPARK-11648: -- The updated patch seems to have resolved this issue. Closing

[jira] [Created] (SPARK-11708) 20-25% performance regression in TeraSort

Nishkam Ravi created SPARK-11708: Summary: 20-25% performance regression in TeraSort Key: SPARK-11708 URL: https://issues.apache.org/jira/browse/SPARK-11708 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11648) IllegalReferenceCountException in Spark workloads

[ https://issues.apache.org/jira/browse/SPARK-11648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001291#comment-15001291 ] Nishkam Ravi commented on SPARK-11648: -- The two issues are closely related, not sure if they are

[jira] [Created] (SPARK-11648) IllegalReferenceCountException in Spark workloads

Nishkam Ravi created SPARK-11648: Summary: IllegalReferenceCountException in Spark workloads Key: SPARK-11648 URL: https://issues.apache.org/jira/browse/SPARK-11648 Project: Spark Issue

[jira] [Commented] (SPARK-11648) IllegalReferenceCountException in Spark workloads

[ https://issues.apache.org/jira/browse/SPARK-11648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15000103#comment-15000103 ] Nishkam Ravi commented on SPARK-11648: -- Traced this back to 'stream data using network lib' commit

[jira] [Closed] (SPARK-11635) Avoid Guava version dependency for Hashcodes

[ https://issues.apache.org/jira/browse/SPARK-11635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi closed SPARK-11635. > Avoid Guava version dependency for Hashcodes > > >

[jira] [Created] (SPARK-11635) Avoid Guava version dependency for Hashcodes

Nishkam Ravi created SPARK-11635: Summary: Avoid Guava version dependency for Hashcodes Key: SPARK-11635 URL: https://issues.apache.org/jira/browse/SPARK-11635 Project: Spark Issue Type:

[jira] [Commented] (SPARK-11635) Avoid Guava version dependency for Hashcodes

[ https://issues.apache.org/jira/browse/SPARK-11635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999308#comment-14999308 ] Nishkam Ravi commented on SPARK-11635: -- I see. So we end up with a dependency either way. HashCode

[jira] [Commented] (SPARK-11278) PageRank fails with unified memory manager

[ https://issues.apache.org/jira/browse/SPARK-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999743#comment-14999743 ] Nishkam Ravi commented on SPARK-11278: -- Would it make sense to set useLegacyMode to true for 1.6

[jira] [Created] (SPARK-11501) spark.rpc config not propagated to executors

Nishkam Ravi created SPARK-11501: Summary: spark.rpc config not propagated to executors Key: SPARK-11501 URL: https://issues.apache.org/jira/browse/SPARK-11501 Project: Spark Issue Type:

[jira] [Commented] (SPARK-6234) 10% Performance regression with Breeze upgrade

[ https://issues.apache.org/jira/browse/SPARK-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14975542#comment-14975542 ] Nishkam Ravi commented on SPARK-6234: - The regression in breeze has been fixed:

[jira] [Created] (SPARK-11278) PageRank fails with unified memory manager

Nishkam Ravi created SPARK-11278: Summary: PageRank fails with unified memory manager Key: SPARK-11278 URL: https://issues.apache.org/jira/browse/SPARK-11278 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11278) PageRank fails with unified memory manager

[ https://issues.apache.org/jira/browse/SPARK-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972155#comment-14972155 ] Nishkam Ravi commented on SPARK-11278: -- Yeah, the problem goes away with useLegacyMode = true. In

[jira] [Comment Edited] (SPARK-11278) PageRank fails with unified memory manager

[ https://issues.apache.org/jira/browse/SPARK-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14972155#comment-14972155 ] Nishkam Ravi edited comment on SPARK-11278 at 10/24/15 12:05 AM: - Yeah,

[jira] [Commented] (SPARK-8881) Scheduling fails if num_executors num_workers

[ https://issues.apache.org/jira/browse/SPARK-8881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618362#comment-14618362 ] Nishkam Ravi commented on SPARK-8881: - There's more to it. Consider the following:

[jira] [Commented] (SPARK-8881) Scheduling fails if num_executors num_workers

[ https://issues.apache.org/jira/browse/SPARK-8881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618369#comment-14618369 ] Nishkam Ravi commented on SPARK-8881: - This isn't the best example because the third

[jira] [Created] (SPARK-8881) Scheduling fails if num_executors num_workers

Nishkam Ravi created SPARK-8881: --- Summary: Scheduling fails if num_executors num_workers Key: SPARK-8881 URL: https://issues.apache.org/jira/browse/SPARK-8881 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-8881) Scheduling fails if num_executors num_workers

[ https://issues.apache.org/jira/browse/SPARK-8881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14617732#comment-14617732 ] Nishkam Ravi commented on SPARK-8881: - No that's not the problem. You have 4 workers

[jira] [Updated] (SPARK-8881) Scheduling fails if num_executors num_workers

[ https://issues.apache.org/jira/browse/SPARK-8881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi updated SPARK-8881: Description: Current scheduling algorithm (in Master.scala) has two issues: 1. cores are allocated

[jira] [Commented] (SPARK-7960) Serialization problem when multiple receivers are specified in a loop

[ https://issues.apache.org/jira/browse/SPARK-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14568373#comment-14568373 ] Nishkam Ravi commented on SPARK-7960: - Yep, the problem persists. Serialization

[jira] [Created] (SPARK-7988) Mechanism to control receiver scheduling

Nishkam Ravi created SPARK-7988: --- Summary: Mechanism to control receiver scheduling Key: SPARK-7988 URL: https://issues.apache.org/jira/browse/SPARK-7988 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-7959) Uneven distribution of receivers in the cluster

[ https://issues.apache.org/jira/browse/SPARK-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14566882#comment-14566882 ] Nishkam Ravi commented on SPARK-7959: - I see. There may be some confusion with regards

[jira] [Commented] (SPARK-7988) Mechanism to control receiver scheduling

[ https://issues.apache.org/jira/browse/SPARK-7988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14566896#comment-14566896 ] Nishkam Ravi commented on SPARK-7988: - Receiver.preferredLocation already helps to

[jira] [Commented] (SPARK-7988) Mechanism to control receiver scheduling

[ https://issues.apache.org/jira/browse/SPARK-7988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14566898#comment-14566898 ] Nishkam Ravi commented on SPARK-7988: - Feel free to assign this JIRA to me if needed.

[jira] [Created] (SPARK-7959) Uneven distribution of receivers in the cluster

Nishkam Ravi created SPARK-7959: --- Summary: Uneven distribution of receivers in the cluster Key: SPARK-7959 URL: https://issues.apache.org/jira/browse/SPARK-7959 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-7960) Serialization problem when multiple receivers are specified in a loop

Nishkam Ravi created SPARK-7960: --- Summary: Serialization problem when multiple receivers are specified in a loop Key: SPARK-7960 URL: https://issues.apache.org/jira/browse/SPARK-7960 Project: Spark

[jira] [Updated] (SPARK-7959) Uneven distribution of receivers in the cluster

[ https://issues.apache.org/jira/browse/SPARK-7959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi updated SPARK-7959: Attachment: Screen Shot 2015-05-29 at 3.52.01 PM.png Uneven distribution of receivers in the

[jira] [Commented] (SPARK-7672) Number format exception with spark.kryoserializer.buffer.mb

[ https://issues.apache.org/jira/browse/SPARK-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546020#comment-14546020 ] Nishkam Ravi commented on SPARK-7672: - In translating deprecated

[jira] [Created] (SPARK-7672) Number format exception with spark.kryoserializer.buffer.mb

Nishkam Ravi created SPARK-7672: --- Summary: Number format exception with spark.kryoserializer.buffer.mb Key: SPARK-7672 URL: https://issues.apache.org/jira/browse/SPARK-7672 Project: Spark

[jira] [Commented] (SPARK-7213) Exception while copying Hadoop config files due to permission issues

[ https://issues.apache.org/jira/browse/SPARK-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518188#comment-14518188 ] Nishkam Ravi commented on SPARK-7213: - Exception in thread main

[jira] [Commented] (SPARK-7213) Exception while copying Hadoop config files due to permission issues

[ https://issues.apache.org/jira/browse/SPARK-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518197#comment-14518197 ] Nishkam Ravi commented on SPARK-7213: - PR: https://github.com/apache/spark/pull/5760/

[jira] [Created] (SPARK-7213) Exception while copying Hadoop config files due to permission issues

Nishkam Ravi created SPARK-7213: --- Summary: Exception while copying Hadoop config files due to permission issues Key: SPARK-7213 URL: https://issues.apache.org/jira/browse/SPARK-7213 Project: Spark

[jira] [Commented] (SPARK-6406) Launcher backward compatibility issues

[ https://issues.apache.org/jira/browse/SPARK-6406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14370815#comment-14370815 ] Nishkam Ravi commented on SPARK-6406: - Also, we should not look for a separate

[jira] [Created] (SPARK-6406) Launcher backward compatibility issues

Nishkam Ravi created SPARK-6406: --- Summary: Launcher backward compatibility issues Key: SPARK-6406 URL: https://issues.apache.org/jira/browse/SPARK-6406 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6234) 10% Performance regression with Breeze upgrade

[ https://issues.apache.org/jira/browse/SPARK-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356016#comment-14356016 ] Nishkam Ravi commented on SPARK-6234: - If Spark doesn't directly/indirectly invoke

[jira] [Created] (SPARK-6234) 10% Performance regression with Breeze upgrade

Nishkam Ravi created SPARK-6234: --- Summary: 10% Performance regression with Breeze upgrade Key: SPARK-6234 URL: https://issues.apache.org/jira/browse/SPARK-6234 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6234) 10% Performance regression with Breeze upgrade

[ https://issues.apache.org/jira/browse/SPARK-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353793#comment-14353793 ] Nishkam Ravi commented on SPARK-6234: - [~mengxr] Variant of

[jira] [Commented] (SPARK-6234) 10% Performance regression with Breeze upgrade

[ https://issues.apache.org/jira/browse/SPARK-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353843#comment-14353843 ] Nishkam Ravi commented on SPARK-6234: - Are we saying that Breeze's performance is

[jira] [Commented] (SPARK-6234) 10% Performance regression with Breeze upgrade

[ https://issues.apache.org/jira/browse/SPARK-6234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14353877#comment-14353877 ] Nishkam Ravi commented on SPARK-6234: - Right. This particular implementation can be

[jira] [Commented] (SPARK-6142) 10-12% Performance regression with finalize

[ https://issues.apache.org/jira/browse/SPARK-6142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350154#comment-14350154 ] Nishkam Ravi commented on SPARK-6142: - [~srowen] GC takes at least one extra cycle for

[jira] [Commented] (SPARK-6142) 10-12% Performance regression with finalize

[ https://issues.apache.org/jira/browse/SPARK-6142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14349646#comment-14349646 ] Nishkam Ravi commented on SPARK-6142: - [~zsxwing]Yes, the 10-12% regression was

[jira] [Created] (SPARK-6142) 10-12% Performance regression with finalize

Nishkam Ravi created SPARK-6142: --- Summary: 10-12% Performance regression with finalize Key: SPARK-6142 URL: https://issues.apache.org/jira/browse/SPARK-6142 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6142) 10-12% Performance regression with finalize

[ https://issues.apache.org/jira/browse/SPARK-6142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14346086#comment-14346086 ] Nishkam Ravi commented on SPARK-6142: - Fixed (master

[jira] [Resolved] (SPARK-6142) 10-12% Performance regression with finalize

[ https://issues.apache.org/jira/browse/SPARK-6142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi resolved SPARK-6142. - Resolution: Fixed 10-12% Performance regression with finalize

[jira] [Reopened] (SPARK-5423) ExternalAppendOnlyMap won't delete temp spilled file if some exception happens during using it

[ https://issues.apache.org/jira/browse/SPARK-5423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi reopened SPARK-5423: - The commit was reverted due to performance issues. Re-opening this JIRA. ExternalAppendOnlyMap

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225313#comment-14225313 ] Nishkam Ravi commented on SPARK-4584: - Looked into this issue a bit more. Source of

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225321#comment-14225321 ] Nishkam Ravi commented on SPARK-4584: - Looks like java.lang.SecurityManager is a hog.

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225471#comment-14225471 ] Nishkam Ravi commented on SPARK-4584: - I don't see SecurityManager anywhere else in

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225479#comment-14225479 ] Nishkam Ravi commented on SPARK-4584: - Use of SecurityManager may be suppressing a JIT

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225563#comment-14225563 ] Nishkam Ravi commented on SPARK-4584: - [~andrewor14] Just curious, why is that

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225567#comment-14225567 ] Nishkam Ravi commented on SPARK-4584: - Around 2GB in the map stage and 1.5GB in the

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225603#comment-14225603 ] Nishkam Ravi commented on SPARK-4584: - I would recommend working with JavaWordCount.

[jira] [Created] (SPARK-4584) 2x Performance regression for Spark-on-YARN

Nishkam Ravi created SPARK-4584: --- Summary: 2x Performance regression for Spark-on-YARN Key: SPARK-4584 URL: https://issues.apache.org/jira/browse/SPARK-4584 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14223930#comment-14223930 ] Nishkam Ravi commented on SPARK-4584: - In YARN cluster mode. 2x Performance

[jira] [Updated] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi updated SPARK-4584: Description: Significant performance regression observed for Spark-on-YARN (upto 2x) after 1.2

[jira] [Issue Comment Deleted] (SPARK-4584) 2x Performance regression for Spark-on-YARN

[ https://issues.apache.org/jira/browse/SPARK-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi updated SPARK-4584: Comment: was deleted (was: In YARN cluster mode.) 2x Performance regression for Spark-on-YARN

[jira] [Commented] (SPARK-4515) OOM/GC errors with sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-4515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14220724#comment-14220724 ] Nishkam Ravi commented on SPARK-4515: - Yep. SPARK-4467 fixes this issue. Closing this

[jira] [Resolved] (SPARK-4515) OOM/GC errors with sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-4515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi resolved SPARK-4515. - Resolution: Fixed OOM/GC errors with sort-based shuffle -

[jira] [Commented] (SPARK-3633) Fetches failure observed after SPARK-2711

[ https://issues.apache.org/jira/browse/SPARK-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14220728#comment-14220728 ] Nishkam Ravi commented on SPARK-3633: - Fixed by PR for SPARK-4480. Fetches failure

[jira] [Resolved] (SPARK-3633) Fetches failure observed after SPARK-2711

[ https://issues.apache.org/jira/browse/SPARK-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi resolved SPARK-3633. - Resolution: Fixed Fetches failure observed after SPARK-2711

[jira] [Created] (SPARK-4515) OOM/GC errors with sort-based shuffle

Nishkam Ravi created SPARK-4515: --- Summary: OOM/GC errors with sort-based shuffle Key: SPARK-4515 URL: https://issues.apache.org/jira/browse/SPARK-4515 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4515) OOM/GC errors with sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-4515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi updated SPARK-4515: Description: OOM/GC errors with sort-based shuffle that go away when shuffle.manager is set to

[jira] [Commented] (SPARK-4515) OOM/GC errors with sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-4515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14219908#comment-14219908 ] Nishkam Ravi commented on SPARK-4515: - I run a slightly modified version of PageRank,

[jira] [Commented] (SPARK-4515) OOM/GC errors with sort-based shuffle

[ https://issues.apache.org/jira/browse/SPARK-4515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14220483#comment-14220483 ] Nishkam Ravi commented on SPARK-4515: - Looks like this issue got fixed through one of

[jira] [Created] (SPARK-4484) Treat maxResultSize as unlimited when set to 0; improve error message

Nishkam Ravi created SPARK-4484: --- Summary: Treat maxResultSize as unlimited when set to 0; improve error message Key: SPARK-4484 URL: https://issues.apache.org/jira/browse/SPARK-4484 Project: Spark

[jira] [Commented] (SPARK-4484) Treat maxResultSize as unlimited when set to 0; improve error message

[ https://issues.apache.org/jira/browse/SPARK-4484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14217577#comment-14217577 ] Nishkam Ravi commented on SPARK-4484: - PR: https://github.com/apache/spark/pull/3360/

[jira] [Commented] (SPARK-4484) Treat maxResultSize as unlimited when set to 0; improve error message

[ https://issues.apache.org/jira/browse/SPARK-4484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14217653#comment-14217653 ] Nishkam Ravi commented on SPARK-4484: - Yeah, most workloads break with a large enough

[jira] [Updated] (SPARK-4484) [CORE] Treat maxResultSize as unlimited when set to 0; improve error message

[ https://issues.apache.org/jira/browse/SPARK-4484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishkam Ravi updated SPARK-4484: Summary: [CORE] Treat maxResultSize as unlimited when set to 0; improve error message (was: Treat

[jira] [Commented] (SPARK-2398) Trouble running Spark 1.0 on Yarn

[ https://issues.apache.org/jira/browse/SPARK-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14211896#comment-14211896 ] Nishkam Ravi commented on SPARK-2398: - [~srowen] yes, this has been resolved by

[jira] [Commented] (SPARK-3633) Fetches failure observed after SPARK-2711

[ https://issues.apache.org/jira/browse/SPARK-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14159059#comment-14159059 ] Nishkam Ravi commented on SPARK-3633: - Quick update: reverting 4fde28c gets rid of the

[jira] [Commented] (SPARK-3633) Fetches failure observed after SPARK-2711

[ https://issues.apache.org/jira/browse/SPARK-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14156074#comment-14156074 ] Nishkam Ravi commented on SPARK-3633: - For a different workload (variant of TeraSort),

[jira] [Commented] (SPARK-3633) Fetches failure observed after SPARK-2711

[ https://issues.apache.org/jira/browse/SPARK-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148026#comment-14148026 ] Nishkam Ravi commented on SPARK-3633: - Increasing the value of

[jira] [Commented] (SPARK-3633) Fetches failure observed after SPARK-2711

[ https://issues.apache.org/jira/browse/SPARK-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14145676#comment-14145676 ] Nishkam Ravi commented on SPARK-3633: - [~pwendell] I see a large number of open files

[jira] [Commented] (SPARK-3633) Fetches failure observed after SPARK-2711

[ https://issues.apache.org/jira/browse/SPARK-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14145713#comment-14145713 ] Nishkam Ravi commented on SPARK-3633: - Looks ok (still waiting on the run to

[jira] [Commented] (SPARK-3633) Fetches failure observed after SPARK-2711

[ https://issues.apache.org/jira/browse/SPARK-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14145749#comment-14145749 ] Nishkam Ravi commented on SPARK-3633: - Unsurprisingly, the run goes through. It would

[jira] [Commented] (SPARK-3633) Fetches failure observed after SPARK-2711

[ https://issues.apache.org/jira/browse/SPARK-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14145763#comment-14145763 ] Nishkam Ravi commented on SPARK-3633: - With higher values of ulimit and timeout

[jira] [Commented] (SPARK-2444) Make spark.yarn.executor.memoryOverhead a first class citizen

[ https://issues.apache.org/jira/browse/SPARK-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059147#comment-14059147 ] Nishkam Ravi commented on SPARK-2444: - [~tgraves] Thanks for the pointer. I guess I

[jira] [Commented] (SPARK-2398) Trouble running Spark 1.0 on Yarn

[ https://issues.apache.org/jira/browse/SPARK-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059489#comment-14059489 ] Nishkam Ravi commented on SPARK-2398: - [~gq] [~sowen] I don't specify XmX, I'm only

[jira] [Commented] (SPARK-2398) Trouble running Spark 1.0 on Yarn

[ https://issues.apache.org/jira/browse/SPARK-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14059494#comment-14059494 ] Nishkam Ravi commented on SPARK-2398: - In this case, it should be Spark taking care of

[jira] [Created] (SPARK-2444) Make spark.yarn.executor.memoryOverhead a first class citizen

Nishkam Ravi created SPARK-2444: --- Summary: Make spark.yarn.executor.memoryOverhead a first class citizen Key: SPARK-2444 URL: https://issues.apache.org/jira/browse/SPARK-2444 Project: Spark

[jira] [Commented] (SPARK-2398) Trouble running Spark 1.0 on Yarn

[ https://issues.apache.org/jira/browse/SPARK-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056858#comment-14056858 ] Nishkam Ravi commented on SPARK-2398: - A bit unclear on the root cause. Q1. Are the

[jira] [Commented] (SPARK-2398) Trouble running Spark 1.0 on Yarn

[ https://issues.apache.org/jira/browse/SPARK-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055637#comment-14055637 ] Nishkam Ravi commented on SPARK-2398: - Thanks [~gq], looks related. I can see the

[jira] [Created] (SPARK-2398) Trouble running Spark 1.0 on Yarn

Nishkam Ravi created SPARK-2398: --- Summary: Trouble running Spark 1.0 on Yarn Key: SPARK-2398 URL: https://issues.apache.org/jira/browse/SPARK-2398 Project: Spark Issue Type: Bug

  1   2   >