Hongtaosun, you need to have at least one small node manager running or the
map reduce job won't start. It's mentioned in the myriad configuration,
though it should be more explicit in the fgs documentation. I'll try to
update it.
On May 23, 2017 1:34 AM, "Hongtaosun (JIRA)" <j...@apache.org> wrote:
[ https://issues.apache.org/jira/browse/MYRIAD-254?page=
com.atlassian.jira.plugin.system.issuetabpanels:comment-
tabpanel&focusedCommentId=16020678#comment-16020678 ]
Hongtaosun commented on MYRIAD-254:
-----------------------------------
[~darinj]thanks for your replying.
I just want to test Fine-grained Scaling.
>From the document,My understanding is bellow:
1、I flexed up some zero nodemanagers.
2、I run MR jobs.
3、These nodemangers can automatic acquisition of resource from mesos.
Is all right?
I can't get through here。
Thank you.
> can not autoscale with zero profile on myriad 0.2.0
> ---------------------------------------------------
>
> Key: MYRIAD-254
> URL: https://issues.apache.org/jira/browse/MYRIAD-254
> Project: Myriad
> Issue Type: Bug
> Components: Scheduler
> Affects Versions: Myriad 0.2.0
> Environment: mesos 1.1.0
> docker 1.12.6
> Reporter: Hongtaosun
>
> [root@csv-dcosstorage60 hadoop]# hadoop jar /usr/hadoop-2.7.0/share/
hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar wordcount
/wordcount/1.txt /wordcount/output
> 17/03/30 17:11:13 INFO client.RMProxy: Connecting to ResourceManager at /
20.26.28.246:8032
> 17/03/30 17:11:14 INFO input.FileInputFormat: Total input paths to
process : 1
> 17/03/30 17:11:14 INFO mapreduce.JobSubmitter: number of splits:1
> 17/03/30 17:11:14 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1490859953345_0002
> 17/03/30 17:11:15 INFO mapreduce.JobSubmitter: Cleaning up the staging
area /tmp/hadoop-yarn/staging/root/.staging/job_1490859953345_0002
> java.io.IOException: org.apache.hadoop.yarn.exceptions.
InvalidResourceRequestException: Invalid resource request, requested memory
< 0, or requested memory > max configured, requestedMemory=512, maxMemory=0
> at org.apache.hadoop.yarn.server.resourcemanager.scheduler.
SchedulerUtils.validateResourceRequest(SchedulerUtils.java:204)
> at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.
validateAndCreateResourceRequest(RMAppManager.java:385)
> at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.
createAndPopulateNewRMApp(RMAppManager.java:328)
> at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.
submitApplication(RMAppManager.java:281)
> at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.
submitApplication(ClientRMService.java:580)
> at org.apache.hadoop.yarn.api.impl.pb.service.
ApplicationClientProtocolPBServiceImpl.submitApplication(
ApplicationClientProtocolPBServiceImpl.java:218)
> at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$
ApplicationClientProtocolService$2.callBlockingMethod(
ApplicationClientProtocol.java:419)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$
ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(
UserGroupInformation.java:1657)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
> at org.apache.hadoop.mapred.YARNRunner.submitJob(
YARNRunner.java:306)
> at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(
JobSubmitter.java:243)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(
UserGroupInformation.java:1657)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)
> at org.apache.hadoop.examples.WordCount.main(WordCount.java:87)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(
ProgramDriver.java:71)
> at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
> at org.apache.hadoop.examples.ExampleDriver.main(
ExampleDriver.java:74)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: org.apache.hadoop.yarn.exceptions.
InvalidResourceRequestException: Invalid resource request, requested memory
< 0, or requested memory > max configured, requestedMemory=512, maxMemory=0
> when I run a mapreduce job,It cann't get resource from mesos.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
