[jira] [Created] (MAPREDUCE-3772) MultipleOutputs output lost if baseOutputPath starts with ../
MultipleOutputs output lost if baseOutputPath starts with ../ - Key: MAPREDUCE-3772 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3772 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1 Affects Versions: 0.22.0 Environment: FreeBSD Reporter: Radim Kolar Priority: Minor Lets say you have output directory set: FileOutputFormat.setOutputPath(job, /tmp/multi1/out); and want to place output from MultipleOutputs into /tmp/multi1/extra I expect following code to work: mos = new MultipleOutputsText, IntWritable(context); mos.write(new Text(zrr), value, ../extra/); but no Exception is throw and expected output directory /tmp/multi1/extra does not even exists. All data written to this output vanish without trace. To make it work fullpath must be used mos.write(new Text(zrr), value, /tmp/multi1/extra/); Output is listed in statistics from MultipleOutputs correctly: org.apache.hadoop.mapreduce.lib.output.MultipleOutputs ../gaja1/=1 (* everything is lost *) /tmp/multi1/out/../ksd34/=1 (* this using full path works *) list1=6667 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3773) Add queue metrics with buckets for job run times
Add queue metrics with buckets for job run times Key: MAPREDUCE-3773 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3773 Project: Hadoop Map/Reduce Issue Type: New Feature Components: jobtracker Reporter: Owen O'Malley Assignee: Owen O'Malley It would be nice to have queue metrics that reflect the number of jobs in each queue that have been running for different ranges of time. Reasonable time ranges are probably 0-1 hr, 1-5 hr, 5-24 hr, 24+ hrs; but they should be configurable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-3766) dumptb fails with filenotfound
[ https://issues.apache.org/jira/browse/MAPREDUCE-3766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antonio Piccolboni resolved MAPREDUCE-3766. --- Resolution: Not A Problem Seems to be reading each part just fine. Maybe it should get a more getmerge like functionality, maybe it should be documented, but most likely this is not a bug. dumptb fails with filenotfound -- Key: MAPREDUCE-3766 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3766 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/streaming Affects Versions: 0.20.2 Environment: AWS, ubuntu, cdh3u3 Reporter: Antonio Piccolboni dumptb can't find an existing file antonio@ip-10-9-65-123:~$ hadoop jar /usr/lib/hadoop/contrib/streaming/hadoop-streaming-0.20.2-cdh3u3.jar dumptb /tmp/Rtmp5h8lIO/file179e4fc 12/01/31 00:27:48 INFO security.UserGroupInformation: JAAS Configuration already set up for Hadoop, not re-installing. 12/01/31 00:27:49 WARN snappy.LoadSnappy: Snappy native library is available 12/01/31 00:27:49 INFO util.NativeCodeLoader: Loaded the native-hadoop library 12/01/31 00:27:49 INFO snappy.LoadSnappy: Snappy native library loaded Exception in thread main java.io.FileNotFoundException: File does not exist: /tmp/Rtmp5h8lIO/file179e4fc/_logs at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1822) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.init(DFSClient.java:1813) at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:544) at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:187) at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:456) at org.apache.hadoop.streaming.AutoInputFormat.getRecordReader(AutoInputFormat.java:56) at org.apache.hadoop.streaming.DumpTypedBytes.dumpTypedBytes(DumpTypedBytes.java:102) at org.apache.hadoop.streaming.DumpTypedBytes.run(DumpTypedBytes.java:83) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) at org.apache.hadoop.streaming.HadoopStreaming.main(HadoopStreaming.java:41) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:197) But the file is there antonio@ip-10-9-65-123:~$ hadoop dfs -ls /tmp/Rtmp5h8lIO/file179e4fc/_logs 12/01/31 00:28:47 INFO security.UserGroupInformation: JAAS Configuration already set up for Hadoop, not re-installing. Found 1 items drwxrwxrwx - antonio supergroup 0 2012-01-31 00:18 /tmp/Rtmp5h8lIO/file179e4fc/_logs/history -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3774) yarn-default.xml should be moved to hadoop-yarn-common.
yarn-default.xml should be moved to hadoop-yarn-common. --- Key: MAPREDUCE-3774 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3774 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.1 yarn-default.xml right now resides in hadoop-yarn-server-common jars which is not the right thing to do since this jar might not be needed in some cases when depending upon yarn. We should move it to hadoop-yarn-common which is a required dependency for all the yarn components (client/server). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname
Change MiniYarnCluster to escape special chars in testname -- Key: MAPREDUCE-3775 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3775 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor When using MiniYarnCluster with the testname set to a nested classname, the $ within the class name creates issues with the container launch scripts as they try to expand the $... within the paths/variables in use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3776) org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator fails intermittently
org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator fails intermittently
Key: MAPREDUCE-3776
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3776
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.24.0
Reporter: Robert Joseph Evans
org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator fails intermittently
{noformat}
Tests run: 9, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 42.267 sec
FAILURE!
testReportedAppProgressWithOnlyMaps(org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator)
Time elapsed: 11.908 sec FAILURE!
junit.framework.AssertionFailedError: Task state is not correct (timedout)
expected:RUNNING but was:SCHEDULED
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.failNotEquals(Assert.java:283)
at junit.framework.Assert.assertEquals(Assert.java:64)
at org.apache.hadoop.mapreduce.v2.app.MRApp.waitForState(MRApp.java:236)
at
org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator.testReportedAppProgressWithOnlyMaps(TestRMContainerAllocator.java:564)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3777) used mem and util have negative values after a queue addition
used mem and util have negative values after a queue addition - Key: MAPREDUCE-3777 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3777 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Arun C Murthy Fix For: 0.23.1 After a queue addition to capacity scheduler and submission of an application, root queue utilization and used memory have negative values. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3778) Per-state RM app-pages should have search ala JHS pages
Per-state RM app-pages should have search ala JHS pages --- Key: MAPREDUCE-3778 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3778 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, webapps Affects Versions: 0.23.0 Reporter: Vinod Kumar Vavilapalli -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
