[jira] [Created] (SPARK-5385) Calling textFile, parallelize, zip, then partitions causes failure on some local[*]

Pedro Rodriguez created SPARK-5385: -- Summary: Calling textFile, parallelize, zip, then partitions causes failure on some local[*] Key: SPARK-5385 URL: https://issues.apache.org/jira/browse/SPARK-5385

[jira] [Resolved] (SPARK-2906) FileLogger throws a invocation target exception.

[ https://issues.apache.org/jira/browse/SPARK-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2906. -- Resolution: Duplicate FileLogger throws a invocation target exception.

[jira] [Resolved] (SPARK-3064) It would be very useful to specifies line terminate when use the textFile function

[ https://issues.apache.org/jira/browse/SPARK-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3064. -- Resolution: Not a Problem I think Cheng's comment indicates the right way to deal with this. It would

[jira] [Commented] (SPARK-5385) Calling textFile, parallelize, zip, then partitions causes failure on some local[*]

[ https://issues.apache.org/jira/browse/SPARK-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289586#comment-14289586 ] Sean Owen commented on SPARK-5385: -- Based on this description, this does not sound like a

[jira] [Commented] (SPARK-2639) Under execute tab in web UI, # Completed task is more than # Total tasks

[ https://issues.apache.org/jira/browse/SPARK-2639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289646#comment-14289646 ] Sean Owen commented on SPARK-2639: -- (Just browsing some old issues here --) Is this

[jira] [Commented] (SPARK-5386) Reduce fails with vectors of big length

[ https://issues.apache.org/jira/browse/SPARK-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289677#comment-14289677 ] Alexander Ulanov commented on SPARK-5386: - My spark-env.sh contains: export

[jira] [Commented] (SPARK-5384) Vectors.sqdist return inconsistent result for sparse/dense vectors when the vectors have different lengths

[ https://issues.apache.org/jira/browse/SPARK-5384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289480#comment-14289480 ] Apache Spark commented on SPARK-5384: - User 'hhbyyh' has created a pull request for

[jira] [Commented] (SPARK-5386) Reduce fails with vectors of big length

[ https://issues.apache.org/jira/browse/SPARK-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289607#comment-14289607 ] Sean Owen commented on SPARK-5386: -- Yes, you're creating ~5GB vectors and have at least 2

[jira] [Closed] (SPARK-5385) Calling textFile, parallelize, zip, then partitions causes failure on some local[*]

[ https://issues.apache.org/jira/browse/SPARK-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pedro Rodriguez closed SPARK-5385. -- Resolution: Fixed Indeed, not a bug, fixed by calling textFiles, then passing partitions.size

[jira] [Created] (SPARK-5387) parquet writer runs into OOM during writing when number of rows is large

Shirley Wu created SPARK-5387: - Summary: parquet writer runs into OOM during writing when number of rows is large Key: SPARK-5387 URL: https://issues.apache.org/jira/browse/SPARK-5387 Project: Spark

[jira] [Resolved] (SPARK-2719) Add Mima binary checks to Flume-Sink

[ https://issues.apache.org/jira/browse/SPARK-2719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2719. -- Resolution: Duplicate Target Version/s: (was: 1.2.0) I think this is just the same as

[jira] [Updated] (SPARK-5386) Reduce fails with vectors of big length

[ https://issues.apache.org/jira/browse/SPARK-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-5386: Description: Code: import org.apache.spark.mllib.rdd.RDDFunctions._ import breeze.linalg._

[jira] [Comment Edited] (SPARK-5386) Reduce fails with vectors of big length

[ https://issues.apache.org/jira/browse/SPARK-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289607#comment-14289607 ] Sean Owen edited comment on SPARK-5386 at 1/23/15 5:58 PM: --- Yes,

[jira] [Commented] (SPARK-2823) GraphX jobs throw IllegalArgumentException

[ https://issues.apache.org/jira/browse/SPARK-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289499#comment-14289499 ] Pedro Rodriguez commented on SPARK-2823: I looked into this more and it looks like

[jira] [Commented] (SPARK-5385) Calling textFile, parallelize, zip, then partitions causes failure on some local[*]

[ https://issues.apache.org/jira/browse/SPARK-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289604#comment-14289604 ] Pedro Rodriguez commented on SPARK-5385: Perhaps its not a bug then, if so, then

[jira] [Commented] (SPARK-5386) Reduce fails with vectors of big length

[ https://issues.apache.org/jira/browse/SPARK-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289633#comment-14289633 ] Sean Owen commented on SPARK-5386: -- You are allocating 8G for executors? or just the

[jira] [Commented] (SPARK-5386) Reduce fails with vectors of big length

[ https://issues.apache.org/jira/browse/SPARK-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289679#comment-14289679 ] Shivaram Venkataraman commented on SPARK-5386: -- Couple of things that might

[jira] [Created] (SPARK-5386) Reduce fails with vectors of big length

Alexander Ulanov created SPARK-5386: --- Summary: Reduce fails with vectors of big length Key: SPARK-5386 URL: https://issues.apache.org/jira/browse/SPARK-5386 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-5386) Reduce fails with vectors of big length

[ https://issues.apache.org/jira/browse/SPARK-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-5386: Environment: Overall: 6 machine cluster (Xeon 3.3GHz 4 cores, 16GB RAM, Ubuntu), each runs

[jira] [Commented] (SPARK-5385) Calling textFile, parallelize, zip, then partitions causes failure on some local[*]

[ https://issues.apache.org/jira/browse/SPARK-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289610#comment-14289610 ] Sean Owen commented on SPARK-5385: -- The number of partitions is not the problem, although

[jira] [Commented] (SPARK-2812) convert maven to archetype based build

[ https://issues.apache.org/jira/browse/SPARK-2812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289682#comment-14289682 ] Sean Owen commented on SPARK-2812: -- Is this obsolete now that the Scala 2.11 support is

[jira] [Commented] (SPARK-5385) Calling textFile, parallelize, zip, then partitions causes failure on some local[*]

[ https://issues.apache.org/jira/browse/SPARK-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289623#comment-14289623 ] Pedro Rodriguez commented on SPARK-5385: On your prior comment, I know

[jira] [Commented] (SPARK-5386) Reduce fails with vectors of big length

[ https://issues.apache.org/jira/browse/SPARK-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289621#comment-14289621 ] Alexander Ulanov commented on SPARK-5386: - I allocate 8G for driver and each

[jira] [Commented] (SPARK-2645) Spark driver calls System.exit(50) after calling SparkContext.stop() the second time

[ https://issues.apache.org/jira/browse/SPARK-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289654#comment-14289654 ] Sean Owen commented on SPARK-2645: -- Is this still an issue? I do not see any call to

[jira] [Resolved] (SPARK-3384) Potential thread unsafe Breeze vector addition in KMeans

[ https://issues.apache.org/jira/browse/SPARK-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3384. -- Resolution: Not a Problem Given the discussion, I do not see evidence that there is a problem in the

[jira] [Commented] (SPARK-3437) Adapt maven build to work without the need of hardcoding scala binary version in artifact id.

[ https://issues.apache.org/jira/browse/SPARK-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289737#comment-14289737 ] Sean Owen commented on SPARK-3437: -- Is this obsolete? both pull requests are closed and

[jira] [Commented] (SPARK-5386) Reduce fails with vectors of big length

[ https://issues.apache.org/jira/browse/SPARK-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289731#comment-14289731 ] Shivaram Venkataraman commented on SPARK-5386: -- Note that having 2 worker

[jira] [Commented] (SPARK-3437) Adapt maven build to work without the need of hardcoding scala binary version in artifact id.

[ https://issues.apache.org/jira/browse/SPARK-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289751#comment-14289751 ] Prashant Sharma commented on SPARK-3437: It is not resolved in my opinion. We

[jira] [Commented] (SPARK-5386) Reduce fails with vectors of big length

[ https://issues.apache.org/jira/browse/SPARK-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289708#comment-14289708 ] Alexander Ulanov commented on SPARK-5386: - Thank you for suggestions. 1. count()

[jira] [Resolved] (SPARK-2688) Need a way to run multiple data pipeline concurrently

[ https://issues.apache.org/jira/browse/SPARK-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2688. -- Resolution: Not a Problem Given that persist() directly addresses the problem identified here, that

[jira] [Updated] (SPARK-5388) Provide a stable application submission gateway

[ https://issues.apache.org/jira/browse/SPARK-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5388: - Description: The existing submission gateway in standalone mode is not compatible across Spark versions.

[jira] [Updated] (SPARK-5388) Provide a stable application submission gateway

[ https://issues.apache.org/jira/browse/SPARK-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5388: - Attachment: Stable Spark Standalone Submission.pdf Provide a stable application submission gateway

[jira] [Commented] (SPARK-2688) Need a way to run multiple data pipeline concurrently

[ https://issues.apache.org/jira/browse/SPARK-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289813#comment-14289813 ] Sandy Ryza commented on SPARK-2688: --- I agree that this is worth keeping open. Allowing

[jira] [Comment Edited] (SPARK-5386) Reduce fails with vectors of big length

[ https://issues.apache.org/jira/browse/SPARK-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289708#comment-14289708 ] Alexander Ulanov edited comment on SPARK-5386 at 1/23/15 6:52 PM:

[jira] [Updated] (SPARK-5386) Reduce fails with vectors of big length

[ https://issues.apache.org/jira/browse/SPARK-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ulanov updated SPARK-5386: Description: Code: import org.apache.spark.mllib.rdd.RDDFunctions._ import breeze.linalg._

[jira] [Closed] (SPARK-2345) ForEachDStream should have an option of running the foreachfunc on Spark

[ https://issues.apache.org/jira/browse/SPARK-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das closed SPARK-2345. Resolution: Not a Problem ForEachDStream should have an option of running the foreachfunc on Spark

[jira] [Commented] (SPARK-5386) Reduce fails with vectors of big length

[ https://issues.apache.org/jira/browse/SPARK-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289880#comment-14289880 ] Alexander Ulanov commented on SPARK-5386: - Thank you, it might be the problem. I

[jira] [Updated] (SPARK-2688) Need a way to run multiple data pipeline concurrently

[ https://issues.apache.org/jira/browse/SPARK-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated SPARK-2688: -- Issue Type: New Feature (was: Improvement) Need a way to run multiple data pipeline concurrently

[jira] [Reopened] (SPARK-2688) Need a way to run multiple data pipeline concurrently

[ https://issues.apache.org/jira/browse/SPARK-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza reopened SPARK-2688: --- Need a way to run multiple data pipeline concurrently

[jira] [Commented] (SPARK-2688) Need a way to run multiple data pipeline concurrently

[ https://issues.apache.org/jira/browse/SPARK-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289798#comment-14289798 ] Sean Owen commented on SPARK-2688: -- [~airhorns] Persisting does not mean hitting disk, if

[jira] [Commented] (SPARK-2688) Need a way to run multiple data pipeline concurrently

[ https://issues.apache.org/jira/browse/SPARK-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289771#comment-14289771 ] Harry Brundage commented on SPARK-2688: --- I respectfully disagree :) Persist is one

[jira] [Updated] (SPARK-5388) Provide a stable application submission gateway

[ https://issues.apache.org/jira/browse/SPARK-5388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-5388: - Summary: Provide a stable application submission gateway (was: Provide a stable application submission

[jira] [Created] (SPARK-5388) Provide a stable application submission gateway in standalone mode

Andrew Or created SPARK-5388: Summary: Provide a stable application submission gateway in standalone mode Key: SPARK-5388 URL: https://issues.apache.org/jira/browse/SPARK-5388 Project: Spark

[jira] [Commented] (SPARK-5386) Reduce fails with vectors of big length

[ https://issues.apache.org/jira/browse/SPARK-5386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289957#comment-14289957 ] Shivaram Venkataraman commented on SPARK-5386: -- Results are merged on the

[jira] [Commented] (SPARK-5390) Encourage users to post on Stack Overflow in Community Docs

[ https://issues.apache.org/jira/browse/SPARK-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290155#comment-14290155 ] Sean Owen commented on SPARK-5390: -- I don't believe the thrust of the outcome was please

[jira] [Comment Edited] (SPARK-5368) Spark should support NAT (via akka improvements)

[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290057#comment-14290057 ] Timothy St. Clair edited comment on SPARK-5368 at 1/23/15 9:54 PM:

[jira] [Updated] (SPARK-5389) spark-shell.cmd does not run from DOS Windows 7

[ https://issues.apache.org/jira/browse/SPARK-5389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yana Kadiyska updated SPARK-5389: - Description: spark-shell.cmd crashes in DOS prompt Windows 7. Works fine under PowerShell.

[jira] [Created] (SPARK-5390) Encourage users to post on Stack Overflow in Community Docs

Nicholas Chammas created SPARK-5390: --- Summary: Encourage users to post on Stack Overflow in Community Docs Key: SPARK-5390 URL: https://issues.apache.org/jira/browse/SPARK-5390 Project: Spark

[jira] [Commented] (SPARK-5281) Registering table on RDD is giving MissingRequirementError

[ https://issues.apache.org/jira/browse/SPARK-5281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290206#comment-14290206 ] Peter Prettenhofer commented on SPARK-5281: --- my problem was resolved when not

[jira] [Commented] (SPARK-5368) Spark should support NAT (via akka improvements)

[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290047#comment-14290047 ] jay vyas commented on SPARK-5368: - Akka 2.4 will support NAT, according to conversation w/

[jira] [Created] (SPARK-5389) spark-shell.cmd does not run from DOS Windows 7

Yana Kadiyska created SPARK-5389: Summary: spark-shell.cmd does not run from DOS Windows 7 Key: SPARK-5389 URL: https://issues.apache.org/jira/browse/SPARK-5389 Project: Spark Issue Type:

[jira] [Updated] (SPARK-5389) spark-shell.cmd does not run from DOS Windows 7

[ https://issues.apache.org/jira/browse/SPARK-5389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yana Kadiyska updated SPARK-5389: - Attachment: SparkShell_Win7.JPG spark-shell.cmd does not run from DOS Windows 7

[jira] [Updated] (SPARK-5390) Encourage users to post on Stack Overflow in Community Docs

[ https://issues.apache.org/jira/browse/SPARK-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-5390: Description: As [discussed extensively on the user

[jira] [Updated] (SPARK-5390) Encourage users to post on Stack Overflow in Community Docs

[ https://issues.apache.org/jira/browse/SPARK-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-5390: Description: As [discussed extensively on the user

[jira] [Updated] (SPARK-5390) Encourage users to post on Stack Overflow in Community Docs

[ https://issues.apache.org/jira/browse/SPARK-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-5390: Description: As [discussed extensively on the user

[jira] [Commented] (SPARK-5390) Encourage users to post on Stack Overflow in Community Docs

[ https://issues.apache.org/jira/browse/SPARK-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290185#comment-14290185 ] Nicholas Chammas commented on SPARK-5390: - Updated accordingly. Encourage users

[jira] [Commented] (SPARK-5368) Spark should support NAT (via akka improvements)

[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290057#comment-14290057 ] Timothy St. Clair commented on SPARK-5368: -- akka 2.4 apparently has NAT support,

[jira] [Comment Edited] (SPARK-5390) Encourage users to post on Stack Overflow in Community Docs

[ https://issues.apache.org/jira/browse/SPARK-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290187#comment-14290187 ] Nicholas Chammas edited comment on SPARK-5390 at 1/23/15 11:08 PM:

[jira] [Commented] (SPARK-5390) Encourage users to post on Stack Overflow in Community Docs

[ https://issues.apache.org/jira/browse/SPARK-5390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290187#comment-14290187 ] Nicholas Chammas commented on SPARK-5390: - cc [~pwendell] Encourage users to

[jira] [Comment Edited] (SPARK-5209) Jobs fail with unexpected value exception in certain environments

[ https://issues.apache.org/jira/browse/SPARK-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290153#comment-14290153 ] Amo A edited comment on SPARK-5209 at 1/23/15 10:42 PM: So after

[jira] [Commented] (SPARK-5209) Jobs fail with unexpected value exception in certain environments

[ https://issues.apache.org/jira/browse/SPARK-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290153#comment-14290153 ] Amo A commented on SPARK-5209: -- So after some further testing as promised, we managed to find

[jira] [Updated] (SPARK-5368) Spark should support NAT (via akka improvements)

[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jay vyas updated SPARK-5368: Summary: Spark should support NAT (via akka improvements) (was: Spark should support NAT) Spark should

[jira] [Updated] (SPARK-5368) Spark should support NAT

[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jay vyas updated SPARK-5368: Summary: Spark should support NAT (was: Support user configurable akka parameters. ) Spark should

[jira] [Commented] (SPARK-1980) problems introduced by broadcast

[ https://issues.apache.org/jira/browse/SPARK-1980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289210#comment-14289210 ] Sean Owen commented on SPARK-1980: -- This error doesn't say anything except that the app

[jira] [Resolved] (SPARK-2499) spark-default.conf parser doesn't ignore inline comments

[ https://issues.apache.org/jira/browse/SPARK-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2499. -- Resolution: Won't Fix spark-default.conf parser doesn't ignore inline comments

[jira] [Commented] (SPARK-2499) spark-default.conf parser doesn't ignore inline comments

[ https://issues.apache.org/jira/browse/SPARK-2499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289247#comment-14289247 ] Sean Owen commented on SPARK-2499: -- I agree. These are parsed by java.util.Properties so

[jira] [Commented] (SPARK-3284) saveAsParquetFile not working on windows

[ https://issues.apache.org/jira/browse/SPARK-3284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289252#comment-14289252 ] Sean Owen commented on SPARK-3284: -- Are you saying that the NPE happens in your code, not

[jira] [Resolved] (SPARK-3733) Support for programmatically submitting Spark jobs

[ https://issues.apache.org/jira/browse/SPARK-3733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-3733. -- Resolution: Duplicate This looks like it is subsumed by SPARK-4924 which has an active PR. Support

[jira] [Commented] (SPARK-1472) Go through YARN api used in Spark to make sure we aren't using Private Apis

[ https://issues.apache.org/jira/browse/SPARK-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289256#comment-14289256 ] Sean Owen commented on SPARK-1472: -- [~tgraves] Is this as resolved as it can be? now that

[jira] [Commented] (SPARK-5293) Enable Spark user applications to use different versions of Akka

[ https://issues.apache.org/jira/browse/SPARK-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289258#comment-14289258 ] Nan Zhu commented on SPARK-5293: shall we make this JIRA as an umbrella task, so that the

[jira] [Commented] (SPARK-1844) Support maven-style dependency resolution in sbt build

[ https://issues.apache.org/jira/browse/SPARK-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289261#comment-14289261 ] Sean Owen commented on SPARK-1844: -- Is this still relevant now that Maven is the primary

[jira] [Updated] (SPARK-5268) CoarseGrainedExecutorBackend exits for irrelevant DisassociatedEvent

[ https://issues.apache.org/jira/browse/SPARK-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-5268: --- Target Version/s: 1.2.1 CoarseGrainedExecutorBackend exits for irrelevant DisassociatedEvent

[jira] [Commented] (SPARK-1472) Go through YARN api used in Spark to make sure we aren't using Private Apis

[ https://issues.apache.org/jira/browse/SPARK-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289263#comment-14289263 ] Thomas Graves commented on SPARK-1472: -- No there are a bunch of others we can get rid

[jira] [Commented] (SPARK-1967) Using parallelize method to create RDD, wordcount app just hanging there without errors or warnings

[ https://issues.apache.org/jira/browse/SPARK-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289266#comment-14289266 ] Sean Owen commented on SPARK-1967: -- I can't reproduce this. Is it still a problem? can

[jira] [Commented] (SPARK-5097) Adding data frame APIs to SchemaRDD

[ https://issues.apache.org/jira/browse/SPARK-5097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14288947#comment-14288947 ] Apache Spark commented on SPARK-5097: - User 'rxin' has created a pull request for this

[jira] [Commented] (SPARK-5350) There are issues when combining Spark and CDK (https://github.com/egonw/cdk).

[ https://issues.apache.org/jira/browse/SPARK-5350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14288960#comment-14288960 ] Staffan Arvidsson commented on SPARK-5350: -- From what I can find out, you only

[jira] [Commented] (SPARK-3650) Triangle Count handles reverse edges incorrectly

[ https://issues.apache.org/jira/browse/SPARK-3650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289012#comment-14289012 ] Apache Spark commented on SPARK-3650: - User 'Leolh' has created a pull request for

[jira] [Commented] (SPARK-5209) Jobs fail with unexpected value exception in certain environments

[ https://issues.apache.org/jira/browse/SPARK-5209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290224#comment-14290224 ] Sven Krasser commented on SPARK-5209: - Thanks Amo! Assuming this is the root cause,

[jira] [Created] (SPARK-5394) kafka link in streaming docs goes to nowhere

Jon Haddad created SPARK-5394: - Summary: kafka link in streaming docs goes to nowhere Key: SPARK-5394 URL: https://issues.apache.org/jira/browse/SPARK-5394 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-5351) Can't zip RDDs with unequal numbers of partitions in ReplicatedVertexView.upgrade()

[ https://issues.apache.org/jira/browse/SPARK-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-5351. --- Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Issue resolved by pull request

[jira] [Commented] (SPARK-4779) PySpark Shuffle Fails Looking for Files that Don't Exist when low on Memory

[ https://issues.apache.org/jira/browse/SPARK-4779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290298#comment-14290298 ] Sven Krasser commented on SPARK-4779: - Here's a potentially related issue occurring on

[jira] [Created] (SPARK-5392) Shuffle spill size is shown as negative

Sven Krasser created SPARK-5392: --- Summary: Shuffle spill size is shown as negative Key: SPARK-5392 URL: https://issues.apache.org/jira/browse/SPARK-5392 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-5392) Shuffle spill size is shown as negative

[ https://issues.apache.org/jira/browse/SPARK-5392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sven Krasser updated SPARK-5392: Attachment: Screen Shot 2015-01-23 at 5.13.55 PM.png Shuffle spill size is shown as negative

[jira] [Commented] (SPARK-5051) python: module pyspark.daemon not found

[ https://issues.apache.org/jira/browse/SPARK-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14290271#comment-14290271 ] Sven Krasser commented on SPARK-5051: - Do you see

[jira] [Created] (SPARK-5393) Flood of util.RackResolver log messages after SPARK-1714

Sandy Ryza created SPARK-5393: - Summary: Flood of util.RackResolver log messages after SPARK-1714 Key: SPARK-5393 URL: https://issues.apache.org/jira/browse/SPARK-5393 Project: Spark Issue Type:

[jira] [Closed] (SPARK-1660) Centralize the definition of property names and default values

[ https://issues.apache.org/jira/browse/SPARK-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kousuke Saruta closed SPARK-1660. - Resolution: Won't Fix Closing for now. Centralize the definition of property names and default

[jira] [Updated] (SPARK-5392) Shuffle spill size is shown as negative

[ https://issues.apache.org/jira/browse/SPARK-5392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sven Krasser updated SPARK-5392: Description: The Shuffle Spill (Memory) metric on the Stage Detail Web UI shows as negative for

[jira] [Created] (SPARK-5391) SparkSQL fails to create tables with custom JSON SerDe

David Ross created SPARK-5391: - Summary: SparkSQL fails to create tables with custom JSON SerDe Key: SPARK-5391 URL: https://issues.apache.org/jira/browse/SPARK-5391 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-5063) Display more helpful error messages for several invalid operations

[ https://issues.apache.org/jira/browse/SPARK-5063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-5063. --- Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Issue resolved by pull request

[jira] [Resolved] (SPARK-2823) GraphX jobs throw IllegalArgumentException

[ https://issues.apache.org/jira/browse/SPARK-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankur Dave resolved SPARK-2823. --- Resolution: Fixed Fix Version/s: 1.2.1 1.3.0 Issue resolved by pull request

[jira] [Commented] (SPARK-3298) [SQL] registerAsTable / registerTempTable overwrites old tables

[ https://issues.apache.org/jira/browse/SPARK-3298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289004#comment-14289004 ] Apache Spark commented on SPARK-3298: - User 'OopsOutOfMemory' has created a pull

[jira] [Updated] (SPARK-5262) widen types for parameters of coalesce()

[ https://issues.apache.org/jira/browse/SPARK-5262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrian Wang updated SPARK-5262: --- Summary: widen types for parameters of coalesce() (was: coalesce should allow NullType and 1 another

[jira] [Commented] (SPARK-4988) Create table ..as select ..from..order by .. limit 10 report error when one col is a Decimal

[ https://issues.apache.org/jira/browse/SPARK-4988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289116#comment-14289116 ] Kuldeep commented on SPARK-4988: This is also happening also for comparison select * from

[jira] [Resolved] (SPARK-980) NullPointerException for single-host setup with S3 URLs

[ https://issues.apache.org/jira/browse/SPARK-980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-980. - Resolution: Fixed NullPointerException for single-host setup with S3 URLs

[jira] [Commented] (SPARK-980) NullPointerException for single-host setup with S3 URLs

[ https://issues.apache.org/jira/browse/SPARK-980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289139#comment-14289139 ] Sean Owen commented on SPARK-980: - I believe Nan is correct that this was subsequently

[jira] [Commented] (SPARK-960) JobCancellationSuite two jobs sharing the same stage is broken

[ https://issues.apache.org/jira/browse/SPARK-960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14289136#comment-14289136 ] Apache Spark commented on SPARK-960: User 'srowen' has created a pull request for this

[jira] [Updated] (SPARK-1529) Support setting spark.local.dirs to a hadoop FileSystem

[ https://issues.apache.org/jira/browse/SPARK-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kannan Rajah updated SPARK-1529: Attachment: SparkShuffleUsingHDFS_API.pdf Support setting spark.local.dirs to a hadoop FileSystem

[jira] [Updated] (SPARK-1529) Support setting spark.local.dirs to a hadoop FileSystem

[ https://issues.apache.org/jira/browse/SPARK-1529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kannan Rajah updated SPARK-1529: Attachment: (was: MapRShuffleOnSpark.pdf) Support setting spark.local.dirs to a hadoop

[jira] [Commented] (SPARK-4877) userClassPathFirst doesn't handle user classes inheriting from parent

[ https://issues.apache.org/jira/browse/SPARK-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14288945#comment-14288945 ] Stephen Haberman commented on SPARK-4877: - Hi Matt, I don't doubt you are right,

[jira] [Updated] (SPARK-5380) There will be an ArrayIndexOutOfBoundsException if the format of the source file is wrong

[ https://issues.apache.org/jira/browse/SPARK-5380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leo_lh updated SPARK-5380: -- Description: When I build a graph with a file format error, there will be an ArrayIndexOutOfBoundsException

  1   2   >