[
https://issues.apache.org/jira/browse/HADOOP-5180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rodrigo Schmidt reassigned HADOOP-5180:
---------------------------------------
Assignee: Rodrigo Schmidt
> FileNotFoundException when finishing a profiled task that doesn't generate an
> output file
> -----------------------------------------------------------------------------------------
>
> Key: HADOOP-5180
> URL: https://issues.apache.org/jira/browse/HADOOP-5180
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Rodrigo Schmidt
> Assignee: Rodrigo Schmidt
> Priority: Trivial
>
> When running a hadoop job with mapred.task.profile=true, hadoop assumes that
> mapred.task.profile.params contains a "%s" that can be changed for a file
> name that will be created by the profiler containing its text output. If the
> profiler doesn't generate such an output file, profiled tasks will raise a
> FileNotFound exception at the end of their execution:
> 09/02/05 15:39:10 INFO mapred.JobClient: map 100% reduce 0%
> 09/02/05 15:39:10 INFO mapred.JobClient: Communication problem with server:
> java.io.FileNotFoundException:
> http://machine1:50060/tasklog?plaintext=true&taskid=attempt_200901221838_0043_m_000000_0&filter=profile
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1239)
> at
> org.apache.hadoop.mapred.JobClient.downloadProfile(JobClient.java:1089)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1166)
> at JavaCat.main(JavaCat.java:46)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:165)
> at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)
> This happens because it's assumed users will use the default hprof profiler,
> which actually allows you to set the output file name. Not all profilers are
> like that and some don't have any option to create a text output log at all
> or don't allow you to change the name of the output file. In my case, I'm a
> profiler that doesn't generate text output and only allows you to set the
> output directory for the snapshot files it generates.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
