Maybe you are out of space in a local disk? That location[1] looks like
the local dir where MR places some intermediate files. Can you check the
output of df -h on a shell?
[1] /var/local/hadoop/cache/mapred/local
On 22/05/14 09:04, Natarajan, Prabakaran 1. (NSN - IN/Bangalore) wrote:
Hi
When I run a query in Hive, I get below exception. I noticed the
error "No space left on device".
Then I did "hadoop fs -count -q /var/local/hadoop" -- which gave below
output
none inf none inf 69
275 288034318 hdfs://nnode:54310/var/local/hadoop
Why I am getting none and inf for space and remaining space quota? Is
this meaning is unlimited space or is there is any space left?
I tried "hadoop dfsadmin -setSpaceQuota 100G /var/local/hadoop" --
Not sure 100G is correct or not? How much I need to set and how to
calculate this?
After setting 100G , I get the below output for "hadoop fs -count -q
/var/local/hadoop"
none inf 107374182400 104408308039
73 286 297777777 hdfs://nnode:54310/var/local/hadoop
I have to wait to see whether 100G is going to give me an exception or
not....
--------------
2014-05-22 10:48:43,585 ERROR
org.apache.hadoop.security.UserGroupInformation:
PriviledgedActionException as:hadoop
cause:java.io.FileNotFoundException:
/var/local/hadoop/cache/mapred/local/taskTracker/hadoop/jobcache/job_201405211712_0625/jars/org/iq80/snappy/SnappyCompressor.class
(No space left on device)
2014-05-22 10:48:43,585 WARN org.apache.hadoop.mapred.TaskTracker:
Error initializing attempt_201405211712_0625_r_000001_2:
java.io.FileNotFoundException:
/var/local/hadoop/cache/mapred/local/taskTracker/hadoop/jobcache/job_201405211712_0625/jars/org/iq80/snappy/SnappyCompressor.class
*(No space left on device)*
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
at org.apache.hadoop.util.RunJar.unJar(RunJar.java:51)
at
org.apache.hadoop.mapred.JobLocalizer.localizeJobJarFile(JobLocalizer.java:277)
at
org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:377)
at
org.apache.hadoop.mapred.JobLocalizer.localizeJobFiles(JobLocalizer.java:367)
at
org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:202)
at
org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1228)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at
org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1203)
at
org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1118)
at
org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2430)
at java.lang.Thread.run(Thread.java:744)
*Thanks and Regards*
Prabakaran.N aka NP
nsn, Bangalore
*/When "I" is replaced by "We" - even Illness becomes "Wellness"/*
--
*Aitor Pérez*
/Big Data System Engineer/
Telf.: +34 917 680 490
Fax: +34 913 833 301
C/Manuel Tovar, 49-53 - 28034 Madrid - Spain
_http://www.bidoop.es_
