You would have needed to configure it by
setting yarn.scheduler.capacity.resource-calculator to something ending in
DominantResourceCalculator. If you haven't configured it, there's a high
probability that the recently committed
https://issues.apache.org/jira/browse/SPARK-6050 will fix your proble
We're using the capacity scheduler, to the best of my knowledge. Unsure if
multi resource scheduling is used, but if you know of an easy way to figure
that out, then let me know.
Thanks,
Anders
On Sat, Feb 21, 2015 at 12:05 AM, Sandy Ryza
wrote:
> Are you using the capacity scheduler or fifo sc
Are you using the capacity scheduler or fifo scheduler without multi
resource scheduling by any chance?
On Thu, Feb 12, 2015 at 1:51 PM, Anders Arpteg wrote:
> The nm logs only seems to contain similar to the following. Nothing else
> in the same time range. Any help?
>
> 2015-02-12 20:47:31,245
The nm logs only seems to contain similar to the following. Nothing else in
the same time range. Any help?
2015-02-12 20:47:31,245 WARN
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl:
Event EventType: KILL_CONTAINER sent to absent container
container_1422406067005_
It seems unlikely to me that it would be a 2.2 issue, though not entirely
impossible. Are you able to find any of the container logs? Is the
NodeManager launching containers and reporting some exit code?
-Sandy
On Thu, Feb 12, 2015 at 1:21 PM, Anders Arpteg wrote:
> No, not submitting from wi
No, not submitting from windows, from a debian distribution. Had a quick
look at the rm logs, and it seems some containers are allocated but then
released again for some reason. Not easy to make sense of the logs, but
here is a snippet from the logs (from a test in our small test cluster) if
you'd
I ran against 2.6, not 2.2.
For that yarn-client run, do you have the application master log?
On Thu, Feb 12, 2015 at 6:11 AM, Aniket Bhatnagar <
aniket.bhatna...@gmail.com> wrote:
> This is tricky to debug. Check logs of node and resource manager of YARN
> to see if you can trace the error. In
This is tricky to debug. Check logs of node and resource manager of YARN to
see if you can trace the error. In the past I have to closely look at
arguments getting passed to YARN container (they get logged before
attempting to launch containers). If I still don't get a clue, I had to
check the scri
Interesting to hear that it works for you. Are you using Yarn 2.2 as well?
No strange log message during startup, and can't see any other log messages
since no executer gets launched. Does not seems to work in yarn-client mode
either, failing with the exception below.
Exception in thread "main" or
Hi Anders,
I just tried this out and was able to successfully acquire executors. Any
strange log messages or additional color you can provide on your setup?
Does yarn-client mode work?
-Sandy
On Wed, Feb 11, 2015 at 1:28 PM, Anders Arpteg wrote:
> Hi,
>
> Compiled the latest master of Spark y
10 matches
Mail list logo