[ https://issues.apache.org/jira/browse/PIG-689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich resolved PIG-689. -------------------------------- Resolution: Fixed The error message looks adequate to me since it does state that the problem is with the quota. We can't special case every error we encounter > Error Handling - improve message for hadoop18.3 quotas feature > -------------------------------------------------------------- > > Key: PIG-689 > URL: https://issues.apache.org/jira/browse/PIG-689 > Project: Pig > Issue Type: Bug > Environment: linux > hadoop18.3 > Reporter: Araceli Henley > Priority: Minor > > TEST: Set disk quota and attempt to create directory or load a file such that > the quota is exceeded > RESULT: Throws an error 2998 that indicates an unhandled exception, but the > hadoop message is correct. > EXPECTED: a message to the effect that the "disk quota was exceeed" > The error message from hadoop is correct and adequate, but pig is throwing a > "2998" error which is an "unhandled internal error". It should throw "Quota > exceed error". > Log shows: > ERROR 2998: Unhandled internal error. > org.apache.hadoop.dfs.QuotaExceededException: The quota of /user/hadoopqa/foo > is exceeded: quota=2 count=3 > org.apache.hadoop.dfs.QuotaExceededException: > org.apache.hadoop.dfs.QuotaExceededException: The quota of /user/hadoopqa/foo > is exceeded: quota=2 count=3 > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) > at java.lang.reflect.Constructor.newInstance(Constructor.java:513) > at > org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:90) > at > org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:52) > at > org.apache.hadoop.dfs.DFSClient$DFSOutputStream.<init>(DFSClient.java:2311) > at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:477) > at > org.apache.hadoop.dfs.DistributedFileSystem.create(DistributedFileSystem.java:178) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:503) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:484) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:391) > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:213) > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:150) > at org.apache.pig.backend.hadoop.datastorage.HPath.copy(HPath.java:86) > at > org.apache.pig.backend.hadoop.datastorage.HPath.copy(HPath.java:165) > at > org.apache.pig.tools.grunt.GruntParser.processCopyFromLocal(GruntParser.java:499) > at > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:178) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:98) > at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:72) > at org.apache.pig.Main.main(Main.java:296) > Caused by: org.apache.hadoop.ipc.RemoteException: > org.apache.hadoop.dfs.QuotaExceededException: The quota of /user/hadoopqa/foo > is exceeded: quota=2 count=3 > at > org.apache.hadoop.dfs.INodeDirectoryWithQuota.verifyQuota(INode.java:782) > at > org.apache.hadoop.dfs.INodeDirectoryWithQuota.updateNumItemsInTree(INode.java:761) > at org.apache.hadoop.dfs.FSDirectory.updateCount(FSDirectory.java:767) > at org.apache.hadoop.dfs.FSDirectory.addChild(FSDirectory.java:896) > at org.apache.hadoop.dfs.FSDirectory.addNode(FSDirectory.java:886) > at org.apache.hadoop.dfs.FSDirectory.addFile(FSDirectory.java:151) > at > org.apache.hadoop.dfs.FSNamesystem.startFileInternal(FSNamesystem.java:1014) > at org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:909) > at org.apache.hadoop.dfs.NameNode.create(NameNode.java:284) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:481) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:890) > at org.apache.hadoop.ipc.Client.call(Client.java:716) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216) > at org.apache.hadoop.dfs.$Proxy0.create(Unknown Source) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) > at org.apache.hadoop.dfs.$Proxy0.create(Unknown Source) > at > org.apache.hadoop.dfs.DFSClient$DFSOutputStream.<init>(DFSClient.java:2308) > ... 14 > TEST CASES > General setup: > hadoop --config /grid/2/pig_harness/tmp/hod dfs -mkdir /user/hadoopqa/foo > /hadoop --config /grid/2/pig_harness/tmp/hod dfsadmin -setQuota 2 > /user/hadoopqa/foo > From the pig shell test: > 1) creation of directory > 2) try loading a file > STEPS TO REPRODUCE: > Allocate Hod > 2) Create directories and set quota using hadoop > /homes/hadoopqa> hadoop --config /grid/2/pig_harness/tmp/hod dfs -mkdir > /user/hadoopqa/foo > /homes/hadoopqa> hadoop --config /grid/2/pig_harness/tmp/hod dfsadmin > -setQuota 2 /user/hadoopqa/foo > /homes/hadoopqa> hadoop --config /grid/2/pig_harness/tmp/hod dfs -mkdir > /user/hadoopqa/foo/foo1 > /homes/hadoopqa> hadoop --config /grid/2/pig_harness/tmp/hod dfs -mkdir > /user/hadoopqa/foo/foo2 > mkdir: org.apache.hadoop.dfs.QuotaExceededException: The quota of > /user/hadoopqa/foo is exceeded: quota=2 count=3/homes/hadoopqa> hadoop > --config /grid/2/pig_harness/tmp/hod dfs -copyFromLocal > /grid/2/pig/in/studentcolon10k /user/hadoopqa > /homes/hadoopqa> hadoop --config /grid/2/pig_harness/tmp/hod dfs > -copyFromLocal /grid/2/pig/in/studentcolon10k /user/hadoopqa/foo > > 3) open grunt shell and load file into directory that does not have quota > set, then into directory that does have quota set. > grunt> copyFromLocal /grid/2/pig/in/studentcolon10k > /user/hadoopqa/studentcolon10K.p1 > grunt> ls /user/hadoopqa/ > hdfs://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/user/hadoopqa/foo <dir> > hdfs://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/user/hadoopqa/mapredsystem <dir> > hdfs://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/user/hadoopqa/studentcolon10K.p1<r 3> > 218620 > hdfs://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/user/hadoopqa/studentcolon10k<r 3> > 218620 > grunt> copyFromLocal /grid/2/pig/in/studentcolon10k > /user/hadoopqa/foo/studentcolon10K.p1 > 2009-02-26 22:54:42,858 main ERROR org.apache.pig.tools.grunt.Grunt - ERROR > 2998: Unhandled internal error. org.apache.hadoop.dfs.QuotaExceededException: > The quota of /user/hadoopqa/foo is exceeded: quota=2 count=3 > Details at logfile: /homes/hadoopqa/pig_1235688818725.log -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.