[
https://issues.apache.org/jira/browse/SPARK-5921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14328764#comment-14328764
]
Michael Nitschinger commented on SPARK-5921:
--------------------------------------------
Interestingly the logs indicate it's complete. Here I tried with 2 instead of
1, both are RUNNING in the UI but the logs indicate this:
15/02/20 10:52:11 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0,
10.0.1.6, PROCESS_LOCAL, 1341 bytes)
15/02/20 10:52:11 INFO TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1,
10.0.1.6, PROCESS_LOCAL, 1341 bytes)
15/02/20 10:52:11 INFO BlockManagerMasterActor: Registering block manager
10.0.1.6:51400 with 265.1 MB RAM, BlockManagerId(0, 10.0.1.6, 51400)
15/02/20 10:52:12 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on
10.0.1.6:51400 (size: 884.0 B, free: 265.1 MB)
15/02/20 10:52:12 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0)
in 895 ms on 10.0.1.6 (1/2)
15/02/20 10:52:12 INFO TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1)
in 889 ms on 10.0.1.6 (2/2)
> Task/Stage always in pending on foreachPartition
> ------------------------------------------------
>
> Key: SPARK-5921
> URL: https://issues.apache.org/jira/browse/SPARK-5921
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, Web UI
> Affects Versions: 1.2.0, 1.2.1
> Environment: OS X, standalone cluster with 1 or more slaves.
> Reporter: Michael Nitschinger
>
> I've got some very simple code:
> sc
> .parallelize(Seq("foo", "bar), 1)
> .foreachPartition({ iter =>
> println(iter)
> })
> when I run this against a standalone cluster the application itself
> completes, but the job is always active (never completes),because the stage
> is always pending.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
