Re: Library Issue

Mon, 28 Feb 2011 01:42:36 -0800 

Harsh J wrote:

You're facing a permissions issue with a device, not a Hadoop-related
issue. Find a way to let users access the required devices
(/dev/nvidiactl is what's reported in your ST, for starters).

On Mon, Feb 28, 2011 at 12:05 PM, Adarsh Sharma
<[email protected]> wrote:

Greetings to all,
Today i came across a strange problem about non-root users in Linux ( CentOS 
).
I am able to compile & run a Java Program through below commands properly : 

[root@cuda1 hadoop-0.20.2]# javac EnumDevices.java
[root@cuda1 hadoop-0.20.2]# java EnumDevices
Total number of devices: 1
Name: Tesla C1060
Version: 1.3
Clock rate: 1296000 MHz
Threads per block: 512

But I need to run it through other user [B]hadoop[/B]  in CentOS

[hadoop@ws37-mah-lin hadoop-0.20.2]$ javac EnumDevices.java
[hadoop@ws37-mah-lin hadoop-0.20.2]$ java EnumDevices
NVIDIA: could not open the device file /dev/nvidiactl (Permission denied). 
Exception in thread "main" CUDA Driver error: 100
      at jcuda.CUDA.setError(CUDA.java:1874)
      at jcuda.CUDA.init(CUDA.java:62)
      at jcuda.CUDA.<init>(CUDA.java:42)
      at EnumDevices.main(EnumDevices.java:20)
*I settled the above issue by setting permission to /dev/nvidia files to Hadoop user & group. 
Now, I am able to compile programs from command

[hadoop@cuda1 hadoop-0.20.2]# javac EnumDevices.java
[hadoop@cuda1 hadoop-0.20.2]# java EnumDevices
Total number of devices: 1
Name: Tesla C1060
Version: 1.3
Clock rate: 1296000 MHz
Threads per block: 512

But Still don't know why it fails in Map-reduce job.
[hadoop@ws37-mah-lin hadoop-0.20.2]$ bin/hadoop jar wordcount1.jar org.myorg.WordCount /user/hadoop/gutenberg /user/hadoop/output1 11/02/28 15:01:45 INFO input.FileInputFormat: Total input paths to process : 3 
11/02/28 15:01:45 INFO mapred.JobClient: Running job: job_201102281104_0006
11/02/28 15:01:46 INFO mapred.JobClient:  map 0% reduce 0%
11/02/28 15:01:56 INFO mapred.JobClient: Task Id : attempt_201102281104_0006_m_000000_0, Status : FAILED 
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115) 
       at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
       at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) 
       at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113) 
       ... 3 more
Caused by: java.lang.UnsatisfiedLinkError: no jcuda in java.library.path
       at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1734)
       at java.lang.Runtime.loadLibrary0(Runtime.java:823)
       at java.lang.System.loadLibrary(System.java:1028)
       at jcuda.driver.CUDADriver.<clinit>(CUDADriver.java:909)
       at jcuda.CUDA.init(CUDA.java:62)
       at jcuda.CUDA.<init>(CUDA.java:42)
       at org.myorg.WordCount$TokenizerMapper.<init>(WordCount.java:28)
       ... 8 more
11/02/28 15:01:56 INFO mapred.JobClient: Task Id : attempt_201102281104_0006_m_000001_0, Status : FAILED 
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115) 
       at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
       at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) 
       at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113) 
       ... 3 more
Caused by: java.lang.UnsatisfiedLinkError: no jcuda in java.library.path
       at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1734)
       at java.lang.Runtime.loadLibrary0(Runtime.java:823)
       at java.lang.System.loadLibrary(System.java:1028)
       at jcuda.driver.CUDADriver.<clinit>(CUDADriver.java:909)
       at jcuda.CUDA.init(CUDA.java:62)
       at jcuda.CUDA.<init>(CUDA.java:42)
       at org.myorg.WordCount$TokenizerMapper.<init>(WordCount.java:28)
       ... 8 more
11/02/28 15:02:05 INFO mapred.JobClient: Task Id : attempt_201102281104_0006_m_000002_1, Status : FAILED 
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115) 
       at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
       at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) 
       at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113) 
       ... 3 more
Caused by: java.lang.UnsatisfiedLinkError: no jcuda in java.library.path
       at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1734)
       at java.lang.Runtime.loadLibrary0(Runtime.java:823)
       at java.lang.System.loadLibrary(System.java:1028)
       at jcuda.driver.CUDADriver.<clinit>(CUDADriver.java:909)
       at jcuda.CUDA.init(CUDA.java:62)
       at jcuda.CUDA.<init>(CUDA.java:42)
       at org.myorg.WordCount$TokenizerMapper.<init>(WordCount.java:28)
       ... 8 more
11/02/28 15:02:09 INFO mapred.JobClient: Task Id : attempt_201102281104_0006_m_000000_2, Status : FAILED 
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115) 
       at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
       at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) 
       at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113) 
       ... 3 more
Caused by: java.lang.UnsatisfiedLinkError: no jcuda in java.library.path
       at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1709)
       at java.lang.Runtime.loadLibrary0(Runtime.java:823)
       at java.lang.System.loadLibrary(System.java:1028)
       at jcuda.driver.CUDADriver.<clinit>(CUDADriver.java:909)
       at jcuda.CUDA.init(CUDA.java:62)
       at jcuda.CUDA.<init>(CUDA.java:42)
       at org.myorg.WordCount$TokenizerMapper.<init>(WordCount.java:28)
       ... 8 more
11/02/28 15:02:09 INFO mapred.JobClient: Task Id : attempt_201102281104_0006_m_000001_2, Status : FAILED 
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115) 
       at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
       at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) 
       at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113) 
       ... 3 more
Caused by: java.lang.UnsatisfiedLinkError: no jcuda in java.library.path
       at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1709)
       at java.lang.Runtime.loadLibrary0(Runtime.java:823)
       at java.lang.System.loadLibrary(System.java:1028)
       at jcuda.driver.CUDADriver.<clinit>(CUDADriver.java:909)
       at jcuda.CUDA.init(CUDA.java:62)
       at jcuda.CUDA.<init>(CUDA.java:42)
       at org.myorg.WordCount$TokenizerMapper.<init>(WordCount.java:28)
       ... 8 more
11/02/28 15:02:11 INFO mapred.JobClient: Task Id : attempt_201102281104_0006_m_000002_2, Status : FAILED 
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115) 
       at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:569)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
       at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) 
       at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113) 
       ... 3 more
Caused by: java.lang.UnsatisfiedLinkError: no jcuda in java.library.path
       at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1734)
       at java.lang.Runtime.loadLibrary0(Runtime.java:823)
       at java.lang.System.loadLibrary(System.java:1028)
       at jcuda.driver.CUDADriver.<clinit>(CUDADriver.java:909)
       at jcuda.CUDA.init(CUDA.java:62)
       at jcuda.CUDA.<init>(CUDA.java:42)
       at org.myorg.WordCount$TokenizerMapper.<init>(WordCount.java:28)
       ... 8 more

11/02/28 15:02:19 INFO mapred.JobClient: Job complete: job_201102281104_0006
11/02/28 15:02:19 INFO mapred.JobClient: Counters: 3
11/02/28 15:02:19 INFO mapred.JobClient:   Job Counters
11/02/28 15:02:19 INFO mapred.JobClient:     Launched map tasks=12
11/02/28 15:02:19 INFO mapred.JobClient:     Data-local map tasks=12
11/02/28 15:02:19 INFO mapred.JobClient:     Failed map tasks=1
[hadoop@ws37-mah-lin hadoop-0.20.2]$

Any suggestions.


Thanks

*

[hadoop@ws37-mah-lin hadoop-0.20.2]$
Actually I need to run a map-reduce code but first if it runs through simple 
then I will go for it.
Please guide me how to solve this issue as CLASSPATH is same through all 
users.



Thanks & best Regards,
Adarsh Sharma

Reply via email to