[jira] [Resolved] (MAPREDUCE-6910) MapReduceTrackingUriPlugin can not return the right URI of history server with HTTPS
[
https://issues.apache.org/jira/browse/MAPREDUCE-6910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash resolved MAPREDUCE-6910.
-
Resolution: Fixed
> MapReduceTrackingUriPlugin can not return the right URI of history server
> with HTTPS
>
>
> Key: MAPREDUCE-6910
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6910
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.7.3, 2.8.1, 3.0.0-alpha3
>Reporter: Lantao Jin
>Assignee: Lantao Jin
> Fix For: 2.9.0, 3.0.0-beta1, 2.8.2
>
> Attachments: MAPREDUCE-6910.001.patch, MAPREDUCE-6910.002.patch
>
>
> When the {{MapReduceTrackingUriPlugin}} enabled, the URI requests from proxy
> server or RM UI which are also out of
> {{yarn.resourcemanager.max-completed-applications}} should be redirect to the
> history server URI.
> But when I access a HTTPS history server with the properties:
> {quote}
>
> mapreduce.jobhistory.http.policy
> HTTPS_ONLY
>
>
> mapreduce.jobhistory.webapp.https.address
> history.example.com:12345
>
> {quote}
> The {{MapReduceTrackingUriPlugin}} still returns a default HTTP URI:
> {{http://0.0.0.0:19888}}
> or
> {{http://history.example.com:67890}}
> if {{mapreduce.jobhistory.webapp.address}} is engaged at same time.
> {quote}
>
> mapreduce.jobhistory.webapp.address
> history.example.com:67890
>
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Reopened] (MAPREDUCE-6910) MapReduceTrackingUriPlugin can not return the right URI of history server with HTTPS
[
https://issues.apache.org/jira/browse/MAPREDUCE-6910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash reopened MAPREDUCE-6910:
-
> MapReduceTrackingUriPlugin can not return the right URI of history server
> with HTTPS
>
>
> Key: MAPREDUCE-6910
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6910
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.7.3, 2.8.1, 3.0.0-alpha3
>Reporter: Lantao Jin
> Fix For: 2.9.0, 3.0.0-beta1, 2.8.2
>
> Attachments: MAPREDUCE-6910.001.patch, MAPREDUCE-6910.002.patch
>
>
> When the {{MapReduceTrackingUriPlugin}} enabled, the URI requests from proxy
> server or RM UI which are also out of
> {{yarn.resourcemanager.max-completed-applications}} should be redirect to the
> history server URI.
> But when I access a HTTPS history server with the properties:
> {quote}
>
> mapreduce.jobhistory.http.policy
> HTTPS_ONLY
>
>
> mapreduce.jobhistory.webapp.https.address
> history.example.com:12345
>
> {quote}
> The {{MapReduceTrackingUriPlugin}} still returns a default HTTP URI:
> {{http://0.0.0.0:19888}}
> or
> {{http://history.example.com:67890}}
> if {{mapreduce.jobhistory.webapp.address}} is engaged at same time.
> {quote}
>
> mapreduce.jobhistory.webapp.address
> history.example.com:67890
>
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6810) hadoop-mapreduce-client-nativetask compilation broken on GCC-6.2.1
Ravi Prakash created MAPREDUCE-6810:
---
Summary: hadoop-mapreduce-client-nativetask compilation broken on
GCC-6.2.1
Key: MAPREDUCE-6810
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6810
Project: Hadoop Map/Reduce
Issue Type: Task
Affects Versions: 3.0.0-alpha1
Reporter: Ravi Prakash
Assignee: Ravi Prakash
I recently upgraded from Fedora 22 to Fedora 25 (I'm assuming this means the
latest and greatest compilers, cmake etc.) My trunk build failed with this
error:
{code}
[WARNING]
/home/raviprak/Code/hadoop/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/native/src/lib/Log.h:35:67:
error: unable to find string literal operator ‘operator""_fmt_’ with ‘const
char [37]’, ‘long unsigned int’ arguments
[WARNING] fprintf(LOG_DEVICE, "%02d/%02d/%02d %02d:%02d:%02d INFO
"_fmt_"\n", \
{code}
https://access.redhat.com/documentation/en-US/Red_Hat_Developer_Toolset/3/html/User_Guide/sect-Changes_in_Version_3.0-GCC.html
bq.This applies to any string literal followed without white space by some
macro. To fix this, add some white space between the string literal and the
macro name.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Resolved] (MAPREDUCE-6695) Problem accessing /jobtracker.jsp
[ https://issues.apache.org/jira/browse/MAPREDUCE-6695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash resolved MAPREDUCE-6695. - Resolution: Invalid Please send an email to the user mailing list at u...@hadoop.apache.org . JIRA is for reporting bugs. > Problem accessing /jobtracker.jsp > - > > Key: MAPREDUCE-6695 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6695 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobtracker >Affects Versions: 2.4.1 > Environment: MapR Cluster >Reporter: Mina Samir Yacoub > Attachments: screenshot-1.png > > > Problem accessing /jobtracker.jsp. Reason: > Too many counters: 121 max=120 > Caused by: > org.apache.hadoop.mapreduce.counters.LimitExceededException: Too many > counters: 121 max=120 > at > org.apache.hadoop.mapreduce.counters.Limits.checkCounters(Limits.java:61) > at > org.apache.hadoop.mapreduce.counters.Limits.incrCounters(Limits.java:68) > at > org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounter(AbstractCounterGroup.java:77) > at > org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.addCounterImpl(AbstractCounterGroup.java:94) > at > org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounterImpl(AbstractCounterGroup.java:122) > at > org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:112) > at > org.apache.hadoop.mapreduce.counters.AbstractCounterGroup.findCounter(AbstractCounterGroup.java:129) > at > org.apache.hadoop.mapred.Counters$Group.findCounter(Counters.java:323) > at > org.apache.hadoop.mapred.Counters$Group.getCounterForName(Counters.java:268) > at org.apache.hadoop.mapred.Counters.incrAllCounters(Counters.java:533) > at > org.apache.hadoop.mapred.JobInProgress.incrementTaskCounters(JobInProgress.java:1705) > at > org.apache.hadoop.mapred.JobInProgress.getMapCounters(JobInProgress.java:1667) > at org.apache.hadoop.mapred.JSPUtil.generateJobTable(JSPUtil.java:428) > at > org.apache.hadoop.mapred.jobtracker_jsp._jspService(jobtracker_jsp.java:207) > at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) > at > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
[jira] [Resolved] (MAPREDUCE-6683) Execute hadoop 1.0.1 application in hadoop 2.6.0 cause Output directory not set execption
[ https://issues.apache.org/jira/browse/MAPREDUCE-6683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash resolved MAPREDUCE-6683. - Resolution: Invalid > Execute hadoop 1.0.1 application in hadoop 2.6.0 cause Output directory not > set execption > - > > Key: MAPREDUCE-6683 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6683 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 > Environment: Linux Ubuntu 12.04, hadoop 2.6.0 >Reporter: Han Gao >Priority: Minor > > The application can run normally in Hadoop 1.0.1 but can't run in 2.6.0 even > though adapt to use new mapreduce API. > org.apache.hadoop.mapred.InvalidJobConfException: Output directory not set. > at > org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:128) > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:889) > at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093) > at > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850) > at org.apache.hadoop.mapreduce.Job.submit(Job.java:500) > at > org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:336) > at > org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:233) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-3010) ant mvn-install doesn't work on hadoop-mapreduce-project
[ https://issues.apache.org/jira/browse/MAPREDUCE-3010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash resolved MAPREDUCE-3010. - Resolution: Invalid We have moved to maven a long time since > ant mvn-install doesn't work on hadoop-mapreduce-project > > > Key: MAPREDUCE-3010 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3010 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Ravi Prakash > > Even though ant jar works, ant mvn-install fails in the compile-fault-inject > step -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-5074) Remove limits on number of counters and counter groups in MapReduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-5074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash resolved MAPREDUCE-5074. - Resolution: Won't Fix We can re-open this if we find users compelling us to increase the limits > Remove limits on number of counters and counter groups in MapReduce > --- > > Key: MAPREDUCE-5074 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5074 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mr-am, mrv2 >Affects Versions: 3.0.0, 2.0.3-alpha, 0.23.6 >Reporter: Ravi Prakash > > Can we please consider removing limits on the number of counters and counter > groups now that it is all user code? Thanks to the much better architecture > of YARN in which there is no single Job Tracker we have to worry about > overloading, I feel we should do away with this (now arbitrary) constraint on > users' capabilities. Thoughts? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-3663) After submitting a job. If the Runjar process gets killed ,then the job is hanging
[ https://issues.apache.org/jira/browse/MAPREDUCE-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash resolved MAPREDUCE-3663. - Resolution: Cannot Reproduce > After submitting a job. If the Runjar process gets killed ,then the job is > hanging > -- > > Key: MAPREDUCE-3663 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3663 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramgopal N > > When the job is submitted...Runjar process is created and the YarnChild > processes also start running.If at this time ,the RunJar process is getting > killed, the job is hanging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-3201) Even though jobs are getting failed on particular NM, it is not getting blacklisted
[
https://issues.apache.org/jira/browse/MAPREDUCE-3201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash resolved MAPREDUCE-3201.
-
Resolution: Fixed
Please reopen if this is still an issue
> Even though jobs are getting failed on particular NM, it is not getting
> blacklisted
> ---
>
> Key: MAPREDUCE-3201
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3201
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramgopal N
>Priority: Minor
>
> {code:xml}
> The yarnchild process on a particular NM are getting killed continuosly.
> Still the NM is not getting blacklisted
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-3094) org.apache.hadoop.streaming.TestUlimit.testCommandLine fails intermittantly in 20.205.0
[ https://issues.apache.org/jira/browse/MAPREDUCE-3094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash resolved MAPREDUCE-3094. - Resolution: Won't Fix > org.apache.hadoop.streaming.TestUlimit.testCommandLine fails intermittantly > in 20.205.0 > --- > > Key: MAPREDUCE-3094 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3094 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/streaming >Affects Versions: 0.20.205.0 >Reporter: Nathan Roberts > > 11/09/24 00:22:10 INFO mapred.TaskInProgress: Error from > attempt_20110924002157563_0001_m_00_0: java.lang.RuntimeException: > PipeMapRed.waitOutputThreads(): subprocess failed with code 134 > at > org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:311) > at > org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:545) > at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:132) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) > at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) > at org.apache.hadoop.mapred.Child$4.run(Child.java:261) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) > at org.apache.hadoop.mapred.Child.main(Child.java:255) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-6220) To forbid stderr and stdout for MapReduce job
[ https://issues.apache.org/jira/browse/MAPREDUCE-6220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash resolved MAPREDUCE-6220. - Resolution: Not a Problem > To forbid stderr and stdout for MapReduce job > - > > Key: MAPREDUCE-6220 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6220 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.6.0 >Reporter: Yang Hao >Assignee: Yang Hao > Attachments: MAPREDUCE-6220.patch > > > System.out and System.error is a ugly way to print log, and many times it > would do harm to Hadoop cluster. So we can forbid it -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-6028) java.lang.ArithmeticException: / by zero
[ https://issues.apache.org/jira/browse/MAPREDUCE-6028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash resolved MAPREDUCE-6028. - Resolution: Invalid > java.lang.ArithmeticException: / by zero > > > Key: MAPREDUCE-6028 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6028 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 1.2.1 > Environment: hadoop version:1.2.1 >Reporter: eagle > > Run any sql through hive with following > error message: > 2014-08-07 10:22:28,061 INFO org.apache.hadoop.mapred.TaskInProgress: Error > from attempt_201407251033_24476_m_02_0: Error initializing > attempt_201407251033_24476_m_02_0: > java.lang.ArithmeticException: / by zero > and after restart hadoop cluster,the problem resolved。how to find the root > casue for the problem? thks。 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (MAPREDUCE-5317) Stale files left behind for failed jobs
Ravi Prakash created MAPREDUCE-5317:
---
Summary: Stale files left behind for failed jobs
Key: MAPREDUCE-5317
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5317
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.8, 2.0.4-alpha, 3.0.0
Reporter: Ravi Prakash
Assignee: Ravi Prakash
Courtesy [~amar_kamat]!
{quote}
We are seeing _temporary files left behind in the output folder if the job
fails.
The job were failed due to hitting quota issue.
I simply ran the randomwriter (from hadoop examples) with the default setting.
That failed and left behind some stray files.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-3135) Unit test org.apache.hadoop.mapred.TestJobHistoryServer fails intermittently
[ https://issues.apache.org/jira/browse/MAPREDUCE-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash resolved MAPREDUCE-3135. - Resolution: Duplicate Release Note: Presumably TestJobHistoryServer is working fine after MAPREDUCE-4798 > Unit test org.apache.hadoop.mapred.TestJobHistoryServer fails intermittently > > > Key: MAPREDUCE-3135 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3135 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 1.0.0 >Reporter: Ravi Prakash > > Every once in a while org.apache.hadoop.mapred.TestJobHistoryServer fails due > to a timeout. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-5074) Remove limits on number of counters and counter groups in MapReduce
Ravi Prakash created MAPREDUCE-5074: --- Summary: Remove limits on number of counters and counter groups in MapReduce Key: MAPREDUCE-5074 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5074 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mr-am, mrv2 Affects Versions: 2.0.3-alpha, 3.0.0, 0.23.6 Reporter: Ravi Prakash Can we please consider removing limits on the number of counters and counter groups now that it is all user code? Thanks to the much better architecture of YARN in which there is no single Job Tracker we have to worry about overloading, I feel we should do away with this (now arbitrary) constraint on users' capabilities. Thoughts? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-3779) Create hard and soft limits for job counters
[ https://issues.apache.org/jira/browse/MAPREDUCE-3779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash resolved MAPREDUCE-3779. - Resolution: Won't Fix Release Note: I'm marking this JIRA as won't fix. We can consider re-opening it if you propose a compelling use case > Create hard and soft limits for job counters > > > Key: MAPREDUCE-3779 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3779 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: jobtracker, tasktracker >Affects Versions: 0.23.0, 1.0.0 >Reporter: Dave Shine >Priority: Minor > > The mapreduce.job.counters.limit is not overridable at the job level. While > it is necessary to limit the number of counters to reduce overhead, there are > times when exceeding the limit is required. Currently, the only solution is > to increase the limit cluster wide. > I would like to see a soft limit set in the mapred-site.xml that can be > overridden at the job level, in addition to the hard limit that exists today. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4989) JSONify DataTables input data for Attempts page
Ravi Prakash created MAPREDUCE-4989: --- Summary: JSONify DataTables input data for Attempts page Key: MAPREDUCE-4989 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4989 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver, mr-am Affects Versions: 0.23.6 Reporter: Ravi Prakash Assignee: Ravi Prakash Use deferred rendering for the attempts page as was done in MAPREDUCE-4720. I'm sorry I didn't realize earlier that this table could be huge too. Thanks to [~jlowe] for pointing it out. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-3267) MR2 reduce tasks showing >100% complete
[ https://issues.apache.org/jira/browse/MAPREDUCE-3267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash resolved MAPREDUCE-3267. - Resolution: Duplicate Target Version/s: 2.0.0-alpha, 0.23.3, 3.0.0 (was: 0.23.3, 2.0.0-alpha, 3.0.0) > MR2 reduce tasks showing >100% complete > --- > > Key: MAPREDUCE-3267 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3267 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, task >Affects Versions: 0.22.0, 0.23.0 >Reporter: Todd Lipcon >Assignee: Ravi Prakash > > My job is currently showing >100% reduce completion. Some reduce tasks are > much higher than 100% complete. they appear to be in the "last merge pass" > stage -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4836) Elapsed time for running tasks on AM web UI tasks page is 0
Ravi Prakash created MAPREDUCE-4836: --- Summary: Elapsed time for running tasks on AM web UI tasks page is 0 Key: MAPREDUCE-4836 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4836 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.5, 2.0.2-alpha, 3.0.0 Reporter: Ravi Prakash Assignee: Ravi Prakash Yeah! The summary -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4811) JobHistoryServer should show when it was started in WebUI About page
Ravi Prakash created MAPREDUCE-4811: --- Summary: JobHistoryServer should show when it was started in WebUI About page Key: MAPREDUCE-4811 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4811 Project: Hadoop Map/Reduce Issue Type: Improvement Components: jobhistoryserver, mrv2 Affects Versions: 0.23.4, 2.0.2-alpha, 3.0.0 Reporter: Ravi Prakash Priority: Minor Unlike the RM, the JHS doesn't show when it was started. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4802) Takes a long time to load the task list on the AM for large jobs
Ravi Prakash created MAPREDUCE-4802: --- Summary: Takes a long time to load the task list on the AM for large jobs Key: MAPREDUCE-4802 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4802 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mr-am, mrv2, webapps Affects Versions: 0.23.4, 2.0.2-alpha, 3.0.0 Reporter: Ravi Prakash Assignee: Ravi Prakash We should turn on deferred rendering in DataTables as suggested by Luke here: https://issues.apache.org/jira/browse/YARN-151?focusedCommentId=13475811&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13475811 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4787) TestJobMonitorAndPrint is broken
Ravi Prakash created MAPREDUCE-4787: --- Summary: TestJobMonitorAndPrint is broken Key: MAPREDUCE-4787 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4787 Project: Hadoop Map/Reduce Issue Type: Bug Components: test Affects Versions: 3.0.0, 2.0.3-alpha, 0.23.5 Reporter: Ravi Prakash Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.169 sec <<< FAILURE! testJobMonitorAndPrint(org.apache.hadoop.mapreduce.TestJobMonitorAndPrint) Time elapsed: 1105 sec <<< ERROR! java.lang.NullPointerException at org.apache.hadoop.mapreduce.TestJobMonitorAndPrint.testJobMonitorAndPrint(TestJobMonitorAndPrint.java:97) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at junit.framework.TestCase.runTest(TestCase.java:168) at junit.framework.TestCase.runBare(TestCase.java:134) at junit.framework.TestResult$1.protect(TestResult.java:110) at junit.framework.TestResult.runProtected(TestResult.java:128) at junit.framework.TestResult.run(TestResult.java:113) at junit.framework.TestCase.run(TestCase.java:124) at junit.framework.TestSuite.runTest(TestSuite.java:243) at junit.framework.TestSuite.run(TestSuite.java:238) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4786) Job End Notification retry interval is 5 milliseconds by default
Ravi Prakash created MAPREDUCE-4786:
---
Summary: Job End Notification retry interval is 5 milliseconds by
default
Key: MAPREDUCE-4786
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4786
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.4, 2.0.2-alpha, 3.0.0
Reporter: Ravi Prakash
Assignee: Ravi Prakash
Courtesy [~stevenwillis] and [~qwertymaniac]
{quote}
From: Harsh J
I believe the configs of the latter of both of the above
classifications were meant to be added in as replacement names, but
the property names got added in wrong (as the former/older named ones)
in the XML.
the word "seconds" in the description of retries? The code in MR2's
JobEndNotifier seems to expect seconds but uses it directly in
Thread.sleep(…) without making it milliseconds, which may be a bug we
need to fix as well, perhaps in a same issue as the configs ones.
On Fri, Nov 9, 2012 at 11:21 PM, Steven Willis wrote:
> And I noticed that there are some duplicate properties with different values
> and different descriptions:
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4747) Fancy graphs for visualizing task progress
Ravi Prakash created MAPREDUCE-4747: --- Summary: Fancy graphs for visualizing task progress Key: MAPREDUCE-4747 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4747 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.4 Reporter: Ravi Prakash We should think about what kind of map / reduce graphs we want to see in MRv2 to visualize all the task progress / completion information we have. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4721) Task startup time in JHS is same as job startup time.
Ravi Prakash created MAPREDUCE-4721: --- Summary: Task startup time in JHS is same as job startup time. Key: MAPREDUCE-4721 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4721 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Affects Versions: 0.23.3 Reporter: Ravi Prakash As Bobby pointed out in https://issues.apache.org/jira/browse/MAPREDUCE-4711?focusedCommentId=13471696&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13471696 In the Map and Reduce tasks page, it should print the earliest task attempt launch time as TaskImpl:getLaunchTime() does. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4711) Append time elapsed since job-start-time for finished tasks
Ravi Prakash created MAPREDUCE-4711: --- Summary: Append time elapsed since job-start-time for finished tasks Key: MAPREDUCE-4711 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4711 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Affects Versions: 0.23.3 Reporter: Ravi Prakash Assignee: Ravi Prakash In 0.20.x/1.x, the analyze job link gave this information bq. The last Map task task_ finished at (relative to the Job launch time): 5/10 20:23:10 (1hrs, 27mins, 54sec) The time it took for the last task to finish needs to be calculated mentally in 0.23. I believe we should print it next to the finish time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4645) Providing a random seed to Slive should make the sequence of filenames completely deterministic
Ravi Prakash created MAPREDUCE-4645: --- Summary: Providing a random seed to Slive should make the sequence of filenames completely deterministic Key: MAPREDUCE-4645 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4645 Project: Hadoop Map/Reduce Issue Type: Improvement Components: performance, test Affects Versions: 2.0.0-alpha, 0.23.1 Reporter: Ravi Prakash Assignee: Ravi Prakash Using the -random seed option still doesn't produce a deterministic sequence of filenames. Hence there's no way to replicate the performance test. If I'm providing a seed, its obvious that I want the test to be reproducible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4606) TestMRJobs and TestUberAM fail if /mapred/history are not present
Ravi Prakash created MAPREDUCE-4606: --- Summary: TestMRJobs and TestUberAM fail if /mapred/history are not present Key: MAPREDUCE-4606 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4606 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Ravi Prakash This might be related to the test framework rather than those tests themselves. To make them pass, I had to create /mapred/history. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4604) in mapred-default, mapreduce.map.maxattempts & mapreduce.reduce.maxattempts defaults are set to 4 as well as mapreduce.job.maxtaskfailures.per.tracker.
Ravi Prakash created MAPREDUCE-4604: --- Summary: in mapred-default, mapreduce.map.maxattempts & mapreduce.reduce.maxattempts defaults are set to 4 as well as mapreduce.job.maxtaskfailures.per.tracker. Key: MAPREDUCE-4604 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4604 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 2.0.0-alpha, 0.23.3, trunk Reporter: Ravi Prakash Assignee: Ravi Prakash Priority: Critical This causes the AM to fail the job at the same time as it blacklists a node, thus never actually trying another node. Marking as critical because we need this in 0.23.3 before it releases. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-4135) MRAppMaster throws IllegalStateException while shutting down
[
https://issues.apache.org/jira/browse/MAPREDUCE-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash resolved MAPREDUCE-4135.
-
Resolution: Duplicate
Fix Version/s: 0.23.3
Thanks Tucu! HADOOP-8325 seems to have fixed this issue. Duping this jira
> MRAppMaster throws IllegalStateException while shutting down
>
>
> Key: MAPREDUCE-4135
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4135
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 2.0.0-alpha, 3.0.0
>Reporter: Devaraj K
>Assignee: Ravi Prakash
> Fix For: 0.23.3
>
> Attachments: MAPREDUCE-4135.branch-0.23.patch, MAPREDUCE-4135.patch,
> MAPREDUCE-4135.patch
>
>
> Always MRAppMaster throws IllegalStateException in the stderr while shutting
> down. It doesn't look good having this exception in the stderr file of
> MRAppMaster container.
> {code:xml}
> WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use
> org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
> Exception in thread "Thread-1" java.lang.IllegalStateException: Shutdown in
> progress
> at
> java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:55)
> at java.lang.Runtime.removeShutdownHook(Runtime.java:220)
> at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:2148)
> at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2180)
> at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2157)
> at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:361)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1014)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4297) Usersmap file in gridmix should not fail on empty lines
Ravi Prakash created MAPREDUCE-4297: --- Summary: Usersmap file in gridmix should not fail on empty lines Key: MAPREDUCE-4297 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4297 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/gridmix Affects Versions: 0.23.1 Reporter: Ravi Prakash Assignee: Ravi Prakash An empty line (e.g. at the end of the file) in the usersmap file will cause gridmix to fail. Empty lines should be silently ignored. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4227) TimeWindow statistics are not updated for TaskTrackers which have been restarted.
Ravi Prakash created MAPREDUCE-4227: --- Summary: TimeWindow statistics are not updated for TaskTrackers which have been restarted. Key: MAPREDUCE-4227 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4227 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.20.205.0 Reporter: Ravi Prakash Assignee: Ravi Prakash Whenever a TaskTracker is restarted after the JobTracker has been running for a while (an hour / day maybe), the TimeWindow statistics on the JobTracker Active nodes page are stuck at 0. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4197) Include the hsqldb jar in the hadoop-mapreduce tar file
Ravi Prakash created MAPREDUCE-4197:
---
Summary: Include the hsqldb jar in the hadoop-mapreduce tar file
Key: MAPREDUCE-4197
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4197
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 0.23.1
Reporter: Ravi Prakash
Assignee: Ravi Prakash
Courtesy Brahma
{quote}
In the previuos hadoop releases(20.XX) hsqldb was provided.
But in hadoop-2.0.0 it is not present.Is it intentionally deleted or missing?
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3087) CLASSPATH not the same after MAPREDUCE-2880
CLASSPATH not the same after MAPREDUCE-2880 --- Key: MAPREDUCE-3087 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3087 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Ravi Prakash After MAPREDUCE-2880, my classpath was missing key jar files. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2790) [MR-279] Add additional field for storing the AM/job history info on CLI
[
https://issues.apache.org/jira/browse/MAPREDUCE-2790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash resolved MAPREDUCE-2790.
-
Resolution: Duplicate
> [MR-279] Add additional field for storing the AM/job history info on CLI
>
>
> Key: MAPREDUCE-2790
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2790
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Ramya Sunil
>Assignee: Ravi Prakash
>Priority: Critical
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2790.v1.txt, MAPREDUCE-2790.v2.txt,
> MAPREDUCE-2790.v3.txt, MAPREDUCE-2790.v4.txt
>
>
> bin/mapred job [-list [all]] displays the AM or job history location in the
> "SchedulingInfo" field. An additional column has to be added to display the
> AM/job history information. Currently, the output reads:
> {noformat}
> JobId State StartTime UserNameQueue Priority
> SchedulingInfo
> jobID FAILED 0 ramya default NORMAL AM
> information/job history location
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3069) running test-patch gives me 10 warnings for missing 'build.plugins.plugin.version' of org.apache.rat:apache-rat-plugin
running test-patch gives me 10 warnings for missing 'build.plugins.plugin.version' of org.apache.rat:apache-rat-plugin -- Key: MAPREDUCE-3069 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3069 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.0 Reporter: Ravi Prakash Assignee: Ravi Prakash Fix For: 0.23.0 apache-rat-plugin doesn't have a version specified in hadoop-mapreduce-project and hadoop-yarn pom.xml files -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3023) Queue state is not being translated properly (is always assumed to be running)
Queue state is not being translated properly (is always assumed to be running) -- Key: MAPREDUCE-3023 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3023 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.0 Reporter: Ravi Prakash Assignee: Ravi Prakash Fix For: 0.23.0 During translation of QueueInfo, bq. TypeConverter.java:435 : queueInfo.toString(), QueueState.RUNNING, ought to be bq. queueInfo.toString(), QueueState.getState(queueInfo.getQueueState().toString().toLowerCase()), -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3010) ant mvn-install doesn't work on hadoop-mapreduce-project
ant mvn-install doesn't work on hadoop-mapreduce-project Key: MAPREDUCE-3010 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3010 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Ravi Prakash Even though ant jar works, ant mvn-install fails in the compile-fault-inject step -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2907) ResourceManager logs filled with [INFO] debug messages from org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue
ResourceManager logs filled with [INFO] debug messages from org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue Key: MAPREDUCE-2907 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2907 Project: Hadoop Map/Reduce Issue Type: Bug Components: resourcemanager Affects Versions: 0.23.0 Reporter: Ravi Prakash Fix For: 0.23.0 I see a lot of info messages (probably used for debugging during development) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (MAPREDUCE-2550) bin/mapred no longer works from a source checkout
[
https://issues.apache.org/jira/browse/MAPREDUCE-2550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Prakash reopened MAPREDUCE-2550:
-
Commit 103374ed71a22d614e77e79ac816fc72fbf93463 (Revision 161793) on branch-0.23
mapred-config.sh searches for hadoop-config.sh in bin. Whereas after the recent
mavenization and restructuring, if I had done
{noformat} $ mvn -Pdist install {noformat} in the top level directory then,
hadoop-config.sh is put in libexec.
Can we please include this case too?
> bin/mapred no longer works from a source checkout
> -
>
> Key: MAPREDUCE-2550
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2550
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build
>Affects Versions: 0.20.3
> Environment: Java 6, Redhat 5.5
>Reporter: Eric Yang
>Assignee: Eric Yang
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2550-1.patch, MAPREDUCE-2550-2.patch,
> MAPREDUCE-2550.patch
>
>
> Developer may want to run hadoop without extracting tarball. It would be
> nice if existing method to run mapred scripts from source code is preserved
> for developers.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
