[jira] [Commented] (MYRIAD-252) chown: cannot access '/sys/fs/cgroup/cpu/mesos/330e5ca9-b8ba-4e30-822a-817f7c905891': No such file or directory

2017-03-29 Thread Hongtaosun (JIRA) 

[ 
https://issues.apache.org/jira/browse/MYRIAD-252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15947285#comment-15947285
 ] 

Hongtaosun commented on MYRIAD-252:
---

Hi,Do you test the autoscale with zero profile on myriad framework ? Thank you.

> chown: cannot access 
> '/sys/fs/cgroup/cpu/mesos/330e5ca9-b8ba-4e30-822a-817f7c905891': No such file 
> or directory
> ---
>
> Key: MYRIAD-252
> URL: https://issues.apache.org/jira/browse/MYRIAD-252
> Project: Myriad
>  Issue Type: Bug
>Reporter: yangjunfeng
>
> I have configured myriad these days and some probloms shown below:
> 14:49:45.216470 23824 fetcher.cpp:409] Fetching URI 
> 'hdfs://188.188.0.189:9000/usr/hadoop-2.7.3.tgz'
> I0123 14:49:45.216486 23824 fetcher.cpp:250] Fetching directly into the 
> sandbox directory
> I0123 14:49:45.216532 23824 fetcher.cpp:187] Fetching URI 
> 'hdfs://188.188.0.189:9000/usr/hadoop-2.7.3.tgz'
> I0123 14:49:45.851863 23824 fetcher.cpp:109] Downloading resource with Hadoop 
> client from 'hdfs://188.188.0.189:9000/usr/hadoop-2.7.3.tgz' to 
> '/var/lib/mesos/slaves/3337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/frameworks/1ff6c07d-a86c-400e-abac-e9618a5504db-0014/executors/myriad_executor1ff6c07d-a86c-400e-abac-e9618a5504db-00141ff6c07d-a86c-400e-abac-e9618a5504db-O93123337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/runs/330e5ca9-b8ba-4e30-822a-817f7c905891/hadoop-2.7.3.tgz'
> I0123 14:49:50.121711 23824 fetcher.cpp:547] Fetched 
> 'hdfs://188.188.0.189:9000/usr/hadoop-2.7.3.tgz' to 
> '/var/lib/mesos/slaves/3337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/frameworks/1ff6c07d-a86c-400e-abac-e9618a5504db-0014/executors/myriad_executor1ff6c07d-a86c-400e-abac-e9618a5504db-00141ff6c07d-a86c-400e-abac-e9618a5504db-O93123337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/runs/330e5ca9-b8ba-4e30-822a-817f7c905891/hadoop-2.7.3.tgz'
> I0123 14:49:50.121835 23824 fetcher.cpp:409] Fetching URI 
> 'http://188.188.0.189:8088/conf'
> I0123 14:49:50.121855 23824 fetcher.cpp:250] Fetching directly into the 
> sandbox directory
> I0123 14:49:50.121888 23824 fetcher.cpp:187] Fetching URI 
> 'http://188.188.0.189:8088/conf'
> I0123 14:49:50.121911 23824 fetcher.cpp:134] Downloading resource from 
> 'http://188.188.0.189:8088/conf' to 
> '/var/lib/mesos/slaves/3337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/frameworks/1ff6c07d-a86c-400e-abac-e9618a5504db-0014/executors/myriad_executor1ff6c07d-a86c-400e-abac-e9618a5504db-00141ff6c07d-a86c-400e-abac-e9618a5504db-O93123337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/runs/330e5ca9-b8ba-4e30-822a-817f7c905891/conf'
> W0123 14:49:50.139974 23824 fetcher.cpp:289] Copying instead of extracting 
> resource from URI with 'extract' flag, because it does not seem to be an 
> archive: http://188.188.0.189:8088/conf
> I0123 14:49:50.140010 23824 fetcher.cpp:547] Fetched 
> 'http://188.188.0.189:8088/conf' to 
> '/var/lib/mesos/slaves/3337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/frameworks/1ff6c07d-a86c-400e-abac-e9618a5504db-0014/executors/myriad_executor1ff6c07d-a86c-400e-abac-e9618a5504db-00141ff6c07d-a86c-400e-abac-e9618a5504db-O93123337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/runs/330e5ca9-b8ba-4e30-822a-817f7c905891/conf'
> chown: cannot access 
> '/sys/fs/cgroup/cpu/mesos/330e5ca9-b8ba-4e30-822a-817f7c905891': No such file 
> or directory
> env: /bin/yarn: No such file or directory
> How can I fix this problom?
> Thanks very much!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MYRIAD-252) chown: cannot access '/sys/fs/cgroup/cpu/mesos/330e5ca9-b8ba-4e30-822a-817f7c905891': No such file or directory

2017-03-17 Thread Hongtaosun (JIRA) 

[ 
https://issues.apache.org/jira/browse/MYRIAD-252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929510#comment-15929510
 ] 

Hongtaosun commented on MYRIAD-252:
---

Oh no. The right method is :I configure the hadoop_home when start the 
mesos-slave.

> chown: cannot access 
> '/sys/fs/cgroup/cpu/mesos/330e5ca9-b8ba-4e30-822a-817f7c905891': No such file 
> or directory
> ---
>
> Key: MYRIAD-252
> URL: https://issues.apache.org/jira/browse/MYRIAD-252
> Project: Myriad
>  Issue Type: Bug
>Reporter: yangjunfeng
>
> I have configured myriad these days and some probloms shown below:
> 14:49:45.216470 23824 fetcher.cpp:409] Fetching URI 
> 'hdfs://188.188.0.189:9000/usr/hadoop-2.7.3.tgz'
> I0123 14:49:45.216486 23824 fetcher.cpp:250] Fetching directly into the 
> sandbox directory
> I0123 14:49:45.216532 23824 fetcher.cpp:187] Fetching URI 
> 'hdfs://188.188.0.189:9000/usr/hadoop-2.7.3.tgz'
> I0123 14:49:45.851863 23824 fetcher.cpp:109] Downloading resource with Hadoop 
> client from 'hdfs://188.188.0.189:9000/usr/hadoop-2.7.3.tgz' to 
> '/var/lib/mesos/slaves/3337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/frameworks/1ff6c07d-a86c-400e-abac-e9618a5504db-0014/executors/myriad_executor1ff6c07d-a86c-400e-abac-e9618a5504db-00141ff6c07d-a86c-400e-abac-e9618a5504db-O93123337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/runs/330e5ca9-b8ba-4e30-822a-817f7c905891/hadoop-2.7.3.tgz'
> I0123 14:49:50.121711 23824 fetcher.cpp:547] Fetched 
> 'hdfs://188.188.0.189:9000/usr/hadoop-2.7.3.tgz' to 
> '/var/lib/mesos/slaves/3337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/frameworks/1ff6c07d-a86c-400e-abac-e9618a5504db-0014/executors/myriad_executor1ff6c07d-a86c-400e-abac-e9618a5504db-00141ff6c07d-a86c-400e-abac-e9618a5504db-O93123337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/runs/330e5ca9-b8ba-4e30-822a-817f7c905891/hadoop-2.7.3.tgz'
> I0123 14:49:50.121835 23824 fetcher.cpp:409] Fetching URI 
> 'http://188.188.0.189:8088/conf'
> I0123 14:49:50.121855 23824 fetcher.cpp:250] Fetching directly into the 
> sandbox directory
> I0123 14:49:50.121888 23824 fetcher.cpp:187] Fetching URI 
> 'http://188.188.0.189:8088/conf'
> I0123 14:49:50.121911 23824 fetcher.cpp:134] Downloading resource from 
> 'http://188.188.0.189:8088/conf' to 
> '/var/lib/mesos/slaves/3337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/frameworks/1ff6c07d-a86c-400e-abac-e9618a5504db-0014/executors/myriad_executor1ff6c07d-a86c-400e-abac-e9618a5504db-00141ff6c07d-a86c-400e-abac-e9618a5504db-O93123337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/runs/330e5ca9-b8ba-4e30-822a-817f7c905891/conf'
> W0123 14:49:50.139974 23824 fetcher.cpp:289] Copying instead of extracting 
> resource from URI with 'extract' flag, because it does not seem to be an 
> archive: http://188.188.0.189:8088/conf
> I0123 14:49:50.140010 23824 fetcher.cpp:547] Fetched 
> 'http://188.188.0.189:8088/conf' to 
> '/var/lib/mesos/slaves/3337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/frameworks/1ff6c07d-a86c-400e-abac-e9618a5504db-0014/executors/myriad_executor1ff6c07d-a86c-400e-abac-e9618a5504db-00141ff6c07d-a86c-400e-abac-e9618a5504db-O93123337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/runs/330e5ca9-b8ba-4e30-822a-817f7c905891/conf'
> chown: cannot access 
> '/sys/fs/cgroup/cpu/mesos/330e5ca9-b8ba-4e30-822a-817f7c905891': No such file 
> or directory
> env: /bin/yarn: No such file or directory
> How can I fix this problom?
> Thanks very much!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MYRIAD-252) chown: cannot access '/sys/fs/cgroup/cpu/mesos/330e5ca9-b8ba-4e30-822a-817f7c905891': No such file or directory

2017-03-16 Thread Sarjeet Singh (JIRA) 

[ 
https://issues.apache.org/jira/browse/MYRIAD-252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15928476#comment-15928476
 ] 

Sarjeet Singh commented on MYRIAD-252:
--

>> E0316 15:32:03.124871 12039 shell.hpp:107] Command 'hadoop version 2>&1' 
>> failed; this is the output:
sh: hadoop: command not found
Failed to fetch 'hdfs://csv-dcos02:9000/dist/hadoop-myriad.tgz': Failed to 
create HDFS client: Failed to execute 'hadoop version 2>&1'; the command was 
either not found or exited with a non-zero exit status: 127

@Hongtaosun, can you check the following:
1. Is hadoop installed on all mesos agents?
2. Can you run hadoop commands from agent machines? (also try "/bin/sh hadoop" 
command)
3. Can you check if HADOOP_HOME is set or you have hadoop bin location in PATH 
environment variable?

> chown: cannot access 
> '/sys/fs/cgroup/cpu/mesos/330e5ca9-b8ba-4e30-822a-817f7c905891': No such file 
> or directory
> ---
>
> Key: MYRIAD-252
> URL: https://issues.apache.org/jira/browse/MYRIAD-252
> Project: Myriad
>  Issue Type: Bug
>Reporter: yangjunfeng
>
> I have configured myriad these days and some probloms shown below:
> 14:49:45.216470 23824 fetcher.cpp:409] Fetching URI 
> 'hdfs://188.188.0.189:9000/usr/hadoop-2.7.3.tgz'
> I0123 14:49:45.216486 23824 fetcher.cpp:250] Fetching directly into the 
> sandbox directory
> I0123 14:49:45.216532 23824 fetcher.cpp:187] Fetching URI 
> 'hdfs://188.188.0.189:9000/usr/hadoop-2.7.3.tgz'
> I0123 14:49:45.851863 23824 fetcher.cpp:109] Downloading resource with Hadoop 
> client from 'hdfs://188.188.0.189:9000/usr/hadoop-2.7.3.tgz' to 
> '/var/lib/mesos/slaves/3337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/frameworks/1ff6c07d-a86c-400e-abac-e9618a5504db-0014/executors/myriad_executor1ff6c07d-a86c-400e-abac-e9618a5504db-00141ff6c07d-a86c-400e-abac-e9618a5504db-O93123337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/runs/330e5ca9-b8ba-4e30-822a-817f7c905891/hadoop-2.7.3.tgz'
> I0123 14:49:50.121711 23824 fetcher.cpp:547] Fetched 
> 'hdfs://188.188.0.189:9000/usr/hadoop-2.7.3.tgz' to 
> '/var/lib/mesos/slaves/3337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/frameworks/1ff6c07d-a86c-400e-abac-e9618a5504db-0014/executors/myriad_executor1ff6c07d-a86c-400e-abac-e9618a5504db-00141ff6c07d-a86c-400e-abac-e9618a5504db-O93123337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/runs/330e5ca9-b8ba-4e30-822a-817f7c905891/hadoop-2.7.3.tgz'
> I0123 14:49:50.121835 23824 fetcher.cpp:409] Fetching URI 
> 'http://188.188.0.189:8088/conf'
> I0123 14:49:50.121855 23824 fetcher.cpp:250] Fetching directly into the 
> sandbox directory
> I0123 14:49:50.121888 23824 fetcher.cpp:187] Fetching URI 
> 'http://188.188.0.189:8088/conf'
> I0123 14:49:50.121911 23824 fetcher.cpp:134] Downloading resource from 
> 'http://188.188.0.189:8088/conf' to 
> '/var/lib/mesos/slaves/3337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/frameworks/1ff6c07d-a86c-400e-abac-e9618a5504db-0014/executors/myriad_executor1ff6c07d-a86c-400e-abac-e9618a5504db-00141ff6c07d-a86c-400e-abac-e9618a5504db-O93123337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/runs/330e5ca9-b8ba-4e30-822a-817f7c905891/conf'
> W0123 14:49:50.139974 23824 fetcher.cpp:289] Copying instead of extracting 
> resource from URI with 'extract' flag, because it does not seem to be an 
> archive: http://188.188.0.189:8088/conf
> I0123 14:49:50.140010 23824 fetcher.cpp:547] Fetched 
> 'http://188.188.0.189:8088/conf' to 
> '/var/lib/mesos/slaves/3337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/frameworks/1ff6c07d-a86c-400e-abac-e9618a5504db-0014/executors/myriad_executor1ff6c07d-a86c-400e-abac-e9618a5504db-00141ff6c07d-a86c-400e-abac-e9618a5504db-O93123337a77d-41ff-4d1c-b9ce-6b0c971e7ba1-S2/runs/330e5ca9-b8ba-4e30-822a-817f7c905891/conf'
> chown: cannot access 
> '/sys/fs/cgroup/cpu/mesos/330e5ca9-b8ba-4e30-822a-817f7c905891': No such file 
> or directory
> env: /bin/yarn: No such file or directory
> How can I fix this problom?
> Thanks very much!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (MYRIAD-252) chown: cannot access '/sys/fs/cgroup/cpu/mesos/330e5ca9-b8ba-4e30-822a-817f7c905891': No such file or directory

2017-02-03 Thread yangjunfeng (JIRA) 

[ 
https://issues.apache.org/jira/browse/MYRIAD-252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852458#comment-15852458
 ] 

yangjunfeng commented on MYRIAD-252:


sorry to reply this issue until today. the release I have downloaded is myriad 
0.2.0
my myriad -config-default.yml is shown below:
mesosMaster: zk://ms01:2181,ms02:2181,ms03:2181/mesos
checkpoint: false
frameworkFailoverTimeout: 4320
frameworkName: Myriad
frameworkRole: 
frameworkUser: root # User the Node Manager runs as, required if 
nodeManagerURI set, otherwise defaults to the user
  # running the resource manager.
frameworkSuperUser: root  # To be depricated, currently permissions need set by 
a superuser due to Mesos-1790.  Must be
  # root or have passwordless sudo. Required if 
nodeManagerURI set, ignored otherwise.
nativeLibrary: /usr/local/lib/libmesos.so
zkServers: ms01:2181,ms02:2181,ms03:2181
zkTimeout: 2
restApiPort: 8192
#servedConfigPath: dist/config.tgz
#servedBinaryPath: dist/binary.tgz
profiles:
  zero:  # NMs launched with this profile dynamically obtain cpu/mem from Mesos
cpu: 0
mem: 0
  small:
cpu: 1
mem: 1024
  medium:
cpu: 2
mem: 2048
  large:
cpu: 4
mem: 4096
nmInstances: # NMs to start with. Requires at least 1 NM with a non-zero 
profile.
  medium: 1 # 
rebalancer: false
haEnabled: false
nodemanager:
  jvmMaxMemoryMB: 1024
  cpus: 0.2
  cgroups: false
executor:
  jvmMaxMemoryMB: 256
  #path: file:///usr/local/libexec/mesos/myriad-executor-runnable-0.1.0.jar
  #The following should be used for a remotely distributed URI, hdfs assumed 
but other URI types valid.
  #nodeManagerUri: hdfs://hdfs/hadoop-2.7.0.tar.gz
  nodeManagerUri: hdfs://usr/hadoop-2.7.3.tgz
  #configUri: hdfs://hdfs/config.tgz
  configUri: hdfs://usr/config.tgz
  #jvmUri: hdfs://hdfs/dist/jre-7u76-linux-x64.tar.gz
yarnEnvironment:
  #HADOOP_CONF_DIR=config
  #HADOOP_TMP_DIR=$MESOS_SANDBOX
  YARN_HOME: hadoop-2.7.3 #this should be relative if nodeManagerUri is set
  #JAVA_HOME: /usr/lib/jvm/java-default #System dependent, but sometimes 
necessary
  #JAVA_HOME: jre-7u76-linux-x64.tar.gz # Path to JRE distribution, relative to 
sandbox directory
  #JAVA_LIBRARY_PATH: /opt/mycompany/lib
#mesosAuthenticationPrincipal:
#mesosAuthenticationSecretFilename:


And yarn-site.xml before started yarn


 


yarn.nodemanager.aux-services
mapreduce_shuffle,myriad_executor



yarn.nodemanager.aux-services.mapreduce_shuffle.class
org.apache.hadoop.mapred.ShuffleHandler


yarn.nodemanager.aux-services.myriad_executor.class
org.apache.myriad.executor.MyriadExecutorAuxService


yarn.nm.liveness-monitor.expiry-interval-ms
2000


yarn.am.liveness-monitor.expiry-interval-ms
1


yarn.resourcemanager.nm.liveness-monitor.interval-ms
1000



yarn.nodemanager.resource.cpu-vcores
${nodemanager.resource.cpu-vcores}


yarn.nodemanager.resource.memory-mb
${nodemanager.resource.memory-mb}



  yarn.nodemanager.pmem-check-enabled
  false


  yarn.nodemanager.vmem-check-enabled
  false

 


yarn.nodemanager.address
${myriad.yarn.nodemanager.address}


yarn.nodemanager.webapp.address
${myriad.yarn.nodemanager.webapp.address}


yarn.nodemanager.webapp.https.address
${myriad.yarn.nodemanager.webapp.address}


yarn.nodemanager.localizer.address
${myriad.yarn.nodemanager.localizer.address}

 


yarn.resourcemanager.scheduler.class
org.apache.myriad.scheduler.yarn.MyriadFairScheduler

 


yarn.scheduler.minimum-allocation-vcores
0


yarn.scheduler.minimum-allocation-mb
0


 
yarn.resourcemanager.hostname
188.188.0.189
 




After I started yarn, some configs write in yarn-site.xml and shown the error 
cannot access '/sys/fs/cgroup/cpu/mesos/...

can you help me to check this error? thanks very much



> chown: cannot access 
> '/sys/fs/cgroup/cpu/mesos/330e5ca9-b8ba-4e30-822a-817f7c905891': No such file 
> or directory
> ---
>
> Key: MYRIAD-252
> URL: https://issues.apache.org/jira/browse/MYRIAD-252
> Project: Myriad
>  Issue Type: Bug
>Reporter: yangjunfeng
>
> I have configured myriad these days and some probloms shown below:
> 14:49:45.216470 23824 fetcher.cpp:409] Fetching URI 
> 'hdfs://188.188.0.189:9000/usr/hadoop-2.7.3.tgz'
> I0123 14:49:45.216486 23824 fetcher.cpp:250] Fetching directly into the 
> sandbox directory
> I0123