[jira] [Created] (MAPREDUCE-7199) HsJobsBlock reuse JobACLsManager for checkAccess
Bibin A Chundatt created MAPREDUCE-7199:
---
Summary: HsJobsBlock reuse JobACLsManager for checkAccess
Key: MAPREDUCE-7199
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7199
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Bibin A Chundatt
Reuse JobAclManager.checkAccess
{code}
private boolean checkAccess(String userName) {
if(!areAclsEnabled) {
return true;
}
// User could see its own job.
if (ugi.getShortUserName().equals(userName)) {
return true;
}
// Admin could also see all jobs
if (adminAclList != null && adminAclList.isUserAllowed(ugi)) {
return true;
}
return false;
}
{code}
{code}
jobACLsManager
.checkAccess(ugi, JobACL.VIEW_JOB, ..
new AccessControlList()))
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6190) If a task stucks before its first heartbeat, it never timeouts and the MR job becomes stuck
[
https://issues.apache.org/jira/browse/MAPREDUCE-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16797892#comment-16797892
]
Bibin A Chundatt commented on MAPREDUCE-6190:
-
[~uranus]
During one of the test found that mapreduce.task.timeout=0 configuration used
to disable timeout doesn't work now.
If the task timeout is configured as zero the task fails with stuck timeout, if
the TaskStatus is null
{code}
if (sendProgress) {
// we need to send progress update
updateCounters();
checkTaskLimits();
taskStatus.statusUpdate(taskProgress.get(),
taskProgress.toString(),
counters);
amFeedback = umbilical.statusUpdate(taskId, taskStatus);
taskFound = amFeedback.getTaskFound();
taskStatus.clearStatus();
}
else {
// send ping
amFeedback = umbilical.statusUpdate(taskId, null);
taskFound = amFeedback.getTaskFound();
}
{code}
> If a task stucks before its first heartbeat, it never timeouts and the MR job
> becomes stuck
> ---
>
> Key: MAPREDUCE-6190
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6190
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.6.0, 2.7.0, 2.8.0, 2.9.0, 3.0.0, 3.1.1
>Reporter: Ankit Malhotra
>Assignee: Zhaohui Xin
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: MAPREDUCE-6190.001.patch, MAPREDUCE-6190.002.patch,
> MAPREDUCE-6190.003.patch, MAPREDUCE-6190.004.patch, MAPREDUCE-6190.005.patch
>
>
> Trying to figure out a weird issue we started seeing on our CDH5.1.0 cluster
> with map reduce jobs on YARN.
> We had a job stuck for hours because one of the mappers never started up
> fully. Basically, the map task had 2 attempts, the first one failed and the
> AM tried to schedule a second one and the second attempt was stuck on STATE:
> STARTING, STATUS: NEW. A node never got assigned and the task along with the
> job was stuck indefinitely.
> The AM logs had this being logged again and again:
> {code}
> 2014-12-09 19:25:12,347 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Ramping down 0
> 2014-12-09 19:25:13,352 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received
> completed container container_1408745633994_450952_02_003807
> 2014-12-09 19:25:13,352 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce preemption
> successful attempt_1408745633994_450952_r_48_1000
> 2014-12-09 19:25:13,352 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Ramping down all
> scheduled reduces:0
> 2014-12-09 19:25:13,352 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Going to preempt 1
> 2014-12-09 19:25:13,353 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Preempting
> attempt_1408745633994_450952_r_50_1000
> 2014-12-09 19:25:13,353 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating
> schedule, headroom=0
> 2014-12-09 19:25:13,353 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator:
> completedMapPercent 0.99968 totalMemLimit:1722880 finalMapMemLimit:2560
> finalReduceMemLimit:1720320 netScheduledMapMem:2560
> netScheduledReduceMem:1722880
> 2014-12-09 19:25:13,353 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Ramping down 0
> 2014-12-09 19:25:13,353 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling:
> PendingReds:77 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:0
> AssignedReds:673 CompletedMaps:3124 CompletedReds:0 ContAlloc:4789
> ContRel:798 HostLocal:2944 RackLocal:155
> 2014-12-09 19:25:14,353 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before
> Scheduling: PendingReds:78 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:0
> AssignedReds:673 CompletedMaps:3124 CompletedReds:0 ContAlloc:4789
> ContRel:798 HostLocal:2944 RackLocal:155
> 2014-12-09 19:25:14,359 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating
> schedule, headroom=0
> {code}
> On killing the task manually, the AM started up the task again, scheduled and
> ran it successfully completing the task and the job with it.
> Some quick code grepping led us here:
>
[jira] [Updated] (MAPREDUCE-7195) Mapreduce task timeout to zero could cause too many status update
[
https://issues.apache.org/jira/browse/MAPREDUCE-7195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-7195:
Description:
* mapreduce.task.timeout=0
Could create too many status update
{code}
public static long getTaskProgressReportInterval(final Configuration conf) {
long taskHeartbeatTimeOut = conf.getLong(
MRJobConfig.TASK_TIMEOUT, MRJobConfig.DEFAULT_TASK_TIMEOUT_MILLIS);
return conf.getLong(MRJobConfig.TASK_PROGRESS_REPORT_INTERVAL,
(long) (TASK_REPORT_INTERVAL_TO_TIMEOUT_RATIO * taskHeartbeatTimeOut));
}
{code}
mapreduce timeout=0 is used to disable timeout feature
was:
* mapreduce.task.timeout=0
Could create too many status update
{code}
public static long getTaskProgressReportInterval(final Configuration conf) {
long taskHeartbeatTimeOut = conf.getLong(
MRJobConfig.TASK_TIMEOUT, MRJobConfig.DEFAULT_TASK_TIMEOUT_MILLIS);
return conf.getLong(MRJobConfig.TASK_PROGRESS_REPORT_INTERVAL,
(long) (TASK_REPORT_INTERVAL_TO_TIMEOUT_RATIO * taskHeartbeatTimeOut));
}
{code}
!screenshot-1.png!
> Mapreduce task timeout to zero could cause too many status update
> -
>
> Key: MAPREDUCE-7195
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7195
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Major
> Attachments: screenshot-1.png
>
>
> * mapreduce.task.timeout=0
> Could create too many status update
> {code}
> public static long getTaskProgressReportInterval(final Configuration conf) {
> long taskHeartbeatTimeOut = conf.getLong(
> MRJobConfig.TASK_TIMEOUT, MRJobConfig.DEFAULT_TASK_TIMEOUT_MILLIS);
> return conf.getLong(MRJobConfig.TASK_PROGRESS_REPORT_INTERVAL,
> (long) (TASK_REPORT_INTERVAL_TO_TIMEOUT_RATIO *
> taskHeartbeatTimeOut));
> }
> {code}
> mapreduce timeout=0 is used to disable timeout feature
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7195) Mapreduce task timeout to zero could cause too many status update
[
https://issues.apache.org/jira/browse/MAPREDUCE-7195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-7195:
Description:
* mapreduce.task.timeout=0
Could create too many status update
{code}
public static long getTaskProgressReportInterval(final Configuration conf) {
long taskHeartbeatTimeOut = conf.getLong(
MRJobConfig.TASK_TIMEOUT, MRJobConfig.DEFAULT_TASK_TIMEOUT_MILLIS);
return conf.getLong(MRJobConfig.TASK_PROGRESS_REPORT_INTERVAL,
(long) (TASK_REPORT_INTERVAL_TO_TIMEOUT_RATIO * taskHeartbeatTimeOut));
}
{code}
!screenshot-1.png!
was:
* mapreduce.task.timeout=0
Could create too many status update
{code}
public static long getTaskProgressReportInterval(final Configuration conf) {
long taskHeartbeatTimeOut = conf.getLong(
MRJobConfig.TASK_TIMEOUT, MRJobConfig.DEFAULT_TASK_TIMEOUT_MILLIS);
return conf.getLong(MRJobConfig.TASK_PROGRESS_REPORT_INTERVAL,
(long) (TASK_REPORT_INTERVAL_TO_TIMEOUT_RATIO * taskHeartbeatTimeOut));
}
{code}
> Mapreduce task timeout to zero could cause too many status update
> -
>
> Key: MAPREDUCE-7195
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7195
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Major
> Attachments: screenshot-1.png
>
>
> * mapreduce.task.timeout=0
> Could create too many status update
> {code}
> public static long getTaskProgressReportInterval(final Configuration conf) {
> long taskHeartbeatTimeOut = conf.getLong(
> MRJobConfig.TASK_TIMEOUT, MRJobConfig.DEFAULT_TASK_TIMEOUT_MILLIS);
> return conf.getLong(MRJobConfig.TASK_PROGRESS_REPORT_INTERVAL,
> (long) (TASK_REPORT_INTERVAL_TO_TIMEOUT_RATIO *
> taskHeartbeatTimeOut));
> }
> {code}
> !screenshot-1.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7195) Mapreduce task timeout to zero could cause too many status update
[
https://issues.apache.org/jira/browse/MAPREDUCE-7195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-7195:
Attachment: screenshot-1.png
> Mapreduce task timeout to zero could cause too many status update
> -
>
> Key: MAPREDUCE-7195
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7195
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Major
> Attachments: screenshot-1.png
>
>
> * mapreduce.task.timeout=0
> Could create too many status update
> {code}
> public static long getTaskProgressReportInterval(final Configuration conf) {
> long taskHeartbeatTimeOut = conf.getLong(
> MRJobConfig.TASK_TIMEOUT, MRJobConfig.DEFAULT_TASK_TIMEOUT_MILLIS);
> return conf.getLong(MRJobConfig.TASK_PROGRESS_REPORT_INTERVAL,
> (long) (TASK_REPORT_INTERVAL_TO_TIMEOUT_RATIO *
> taskHeartbeatTimeOut));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-7195) Mapreduce task timeout to zero could cause too many status update
Bibin A Chundatt created MAPREDUCE-7195:
---
Summary: Mapreduce task timeout to zero could cause too many
status update
Key: MAPREDUCE-7195
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7195
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
* mapreduce.task.timeout=0
Could create too many status update
{code}
public static long getTaskProgressReportInterval(final Configuration conf) {
long taskHeartbeatTimeOut = conf.getLong(
MRJobConfig.TASK_TIMEOUT, MRJobConfig.DEFAULT_TASK_TIMEOUT_MILLIS);
return conf.getLong(MRJobConfig.TASK_PROGRESS_REPORT_INTERVAL,
(long) (TASK_REPORT_INTERVAL_TO_TIMEOUT_RATIO * taskHeartbeatTimeOut));
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7169) Speculative attempts should not run on the same node
[
https://issues.apache.org/jira/browse/MAPREDUCE-7169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16734040#comment-16734040
]
Bibin A Chundatt commented on MAPREDUCE-7169:
-
[~uranus]
Looking at the code . Currently for the new taskAttempt container request, we
use all {{datalocalHosts}}
TaskAttemptImpl#RequestContainerTransition
{code}
taskAttempt.eventHandler.handle(new ContainerRequestEvent(
taskAttempt.attemptId, taskAttempt.resourceCapability,
taskAttempt.dataLocalHosts.toArray(
new String[taskAttempt.dataLocalHosts.size()]),
taskAttempt.dataLocalRacks.toArray(
new String[taskAttempt.dataLocalRacks.size()])));
{code}
In async scheduling high probability for containers getting allocated to same
node.
We should skip the nodes on which previous task attempt was lauched,when
Avataar is *SPECULTIVE*
> Speculative attempts should not run on the same node
>
>
> Key: MAPREDUCE-7169
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7169
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: yarn
>Affects Versions: 2.7.2
>Reporter: Lee chen
>Assignee: Zhaohui Xin
>Priority: Major
> Attachments: image-2018-12-03-09-54-07-859.png
>
>
> I found in all versions of yarn, Speculative Execution may set the
> speculative task to the node of original task.What i have read is only it
> will try to have one more task attempt. haven't seen any place mentioning not
> on same node.It is unreasonable.If the node have some problems lead to tasks
> execution will be very slow. and then placement the speculative task to same
> node cannot help the problematic task.
> In our cluster （version 2.7.2，2700 nodes），this phenomenon appear
> almost everyday.
> !image-2018-12-03-09-54-07-859.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Moved] (MAPREDUCE-7169) Speculative attempts should not run on the same node
[ https://issues.apache.org/jira/browse/MAPREDUCE-7169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bibin A Chundatt moved YARN-9079 to MAPREDUCE-7169: --- Affects Version/s: (was: 2.7.2) 2.7.2 Component/s: (was: yarn) (was: capacity scheduler) (was: api) yarn Key: MAPREDUCE-7169 (was: YARN-9079) Project: Hadoop Map/Reduce (was: Hadoop YARN) > Speculative attempts should not run on the same node > > > Key: MAPREDUCE-7169 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7169 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: yarn >Affects Versions: 2.7.2 >Reporter: Lee chen >Priority: Major > Attachments: image-2018-12-03-09-54-07-859.png > > > I found in all versions of yarn, Speculative Execution may set the > speculative task to the node of original task.What i have read is only it > will try to have one more task attempt. haven't seen any place mentioning not > on same node.It is unreasonable.If the node have some problems lead to tasks > execution will be very slow. and then placement the speculative task to same > node cannot help the problematic task. > In our cluster （version 2.7.2，2700 nodes），this phenomenon appear > almost everyday. > !image-2018-12-03-09-54-07-859.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7151) RMContainerAllocator#handleJobPriorityChange expects application_priority always
[
https://issues.apache.org/jira/browse/MAPREDUCE-7151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-7151:
Fix Version/s: 3.1.2
3.2.1
> RMContainerAllocator#handleJobPriorityChange expects application_priority
> always
>
>
> Key: MAPREDUCE-7151
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7151
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bilwa S T
>Priority: Major
> Fix For: 3.1.2, 3.3.0, 3.2.1
>
> Attachments: MAPREDUCE-7151-001.patch, MAPREDUCE-7151-002.patch
>
>
> As per yarn_service.proto {{AllocateResponseProto}} *application_priority* is
> *optional* field.
> But {{RMContainerAllocator#handleJobPriorityChange}} expects to have
> application priority always in response which is not mandatory.
> {code}
> private void handleJobPriorityChange(AllocateResponse response) {
> Priority priorityFromResponse = Priority.newInstance(response
> .getApplicationPriority().getPriority());
> // Update the job priority to Job directly.
> getJob().setJobPriority(priorityFromResponse);
> }
> {code}
> {code}
> message AllocateResponseProto {
> ..
> optional hadoop.common.TokenProto am_rm_token = 12;
> optional PriorityProto application_priority = 13;
> optional CollectorInfoProto collector_info = 14;
> ..
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7151) RMContainerAllocator#handleJobPriorityChange expects application_priority always
[
https://issues.apache.org/jira/browse/MAPREDUCE-7151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-7151:
Resolution: Fixed
Fix Version/s: 3.3.0
Status: Resolved (was: Patch Available)
Committed to trunk.
> RMContainerAllocator#handleJobPriorityChange expects application_priority
> always
>
>
> Key: MAPREDUCE-7151
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7151
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bilwa S T
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: MAPREDUCE-7151-001.patch, MAPREDUCE-7151-002.patch
>
>
> As per yarn_service.proto {{AllocateResponseProto}} *application_priority* is
> *optional* field.
> But {{RMContainerAllocator#handleJobPriorityChange}} expects to have
> application priority always in response which is not mandatory.
> {code}
> private void handleJobPriorityChange(AllocateResponse response) {
> Priority priorityFromResponse = Priority.newInstance(response
> .getApplicationPriority().getPriority());
> // Update the job priority to Job directly.
> getJob().setJobPriority(priorityFromResponse);
> }
> {code}
> {code}
> message AllocateResponseProto {
> ..
> optional hadoop.common.TokenProto am_rm_token = 12;
> optional PriorityProto application_priority = 13;
> optional CollectorInfoProto collector_info = 14;
> ..
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7151) RMContainerAllocator#handleJobPriorityChange expects application_priority always
[
https://issues.apache.org/jira/browse/MAPREDUCE-7151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16655384#comment-16655384
]
Bibin A Chundatt commented on MAPREDUCE-7151:
-
+ 1 for the latest patch .
Will commit by tomorrow if no one objects.
> RMContainerAllocator#handleJobPriorityChange expects application_priority
> always
>
>
> Key: MAPREDUCE-7151
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7151
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bilwa S T
>Priority: Major
> Attachments: MAPREDUCE-7151-001.patch, MAPREDUCE-7151-002.patch
>
>
> As per yarn_service.proto {{AllocateResponseProto}} *application_priority* is
> *optional* field.
> But {{RMContainerAllocator#handleJobPriorityChange}} expects to have
> application priority always in response which is not mandatory.
> {code}
> private void handleJobPriorityChange(AllocateResponse response) {
> Priority priorityFromResponse = Priority.newInstance(response
> .getApplicationPriority().getPriority());
> // Update the job priority to Job directly.
> getJob().setJobPriority(priorityFromResponse);
> }
> {code}
> {code}
> message AllocateResponseProto {
> ..
> optional hadoop.common.TokenProto am_rm_token = 12;
> optional PriorityProto application_priority = 13;
> optional CollectorInfoProto collector_info = 14;
> ..
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7151) RMContainerAllocator#handleJobPriorityChange expects application_priority always
[
https://issues.apache.org/jira/browse/MAPREDUCE-7151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16653592#comment-16653592
]
Bibin A Chundatt commented on MAPREDUCE-7151:
-
Thank you [~BilwaST] for patch
Minor comment.
# default visibility can be set for {{handleJobPriorityChange}}
> RMContainerAllocator#handleJobPriorityChange expects application_priority
> always
>
>
> Key: MAPREDUCE-7151
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7151
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bilwa S T
>Priority: Major
> Attachments: MAPREDUCE-7151-001.patch
>
>
> As per yarn_service.proto {{AllocateResponseProto}} *application_priority* is
> *optional* field.
> But {{RMContainerAllocator#handleJobPriorityChange}} expects to have
> application priority always in response which is not mandatory.
> {code}
> private void handleJobPriorityChange(AllocateResponse response) {
> Priority priorityFromResponse = Priority.newInstance(response
> .getApplicationPriority().getPriority());
> // Update the job priority to Job directly.
> getJob().setJobPriority(priorityFromResponse);
> }
> {code}
> {code}
> message AllocateResponseProto {
> ..
> optional hadoop.common.TokenProto am_rm_token = 12;
> optional PriorityProto application_priority = 13;
> optional CollectorInfoProto collector_info = 14;
> ..
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-7151) RMContainerAllocator#handleJobPriorityChange expects application_priority always
Bibin A Chundatt created MAPREDUCE-7151:
---
Summary: RMContainerAllocator#handleJobPriorityChange expects
application_priority always
Key: MAPREDUCE-7151
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7151
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
As per yarn_service.proto {{AllocateResponseProto}} *application_priority* is
*optional* field.
But {{RMContainerAllocator#handleJobPriorityChange}} expects to have
application priority always in response which is not mandatory.
{code}
private void handleJobPriorityChange(AllocateResponse response) {
Priority priorityFromResponse = Priority.newInstance(response
.getApplicationPriority().getPriority());
// Update the job priority to Job directly.
getJob().setJobPriority(priorityFromResponse);
}
{code}
{code}
message AllocateResponseProto {
..
optional hadoop.common.TokenProto am_rm_token = 12;
optional PriorityProto application_priority = 13;
optional CollectorInfoProto collector_info = 14;
..
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6826) Job fails with InvalidStateTransitonException: Invalid event: JOB_TASK_COMPLETED at SUCCEEDED
[
https://issues.apache.org/jira/browse/MAPREDUCE-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6826:
Status: Patch Available (was: Open)
> Job fails with InvalidStateTransitonException: Invalid event:
> JOB_TASK_COMPLETED at SUCCEEDED
> -
>
> Key: MAPREDUCE-6826
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6826
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.7.2
>Reporter: Varun Saxena
>Assignee: Bilwa S T
>Priority: Major
> Attachments: MAPREDUCE-6826-001.patch
>
>
> This happens if a container is preempted by scheduler after job starts
> committing.
> And this exception in turn leads to application being marked as FAILED in
> YARN.
> I think we can probably ignore JOB_TASK_COMPLETED event while JobImpl state
> is COMMITTING or SUCCEEDED as job is in the process of finishing.
> Also is there any point in attempting to scheduler another task attempt if
> job is already in COMMITTING or SUCCEEDED state.
> {noformat}
> 2016-12-23 09:10:38,642 INFO [AsyncDispatcher event handler]
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl:
> task_1482404625971_23910_m_04 Task Transitioned from RUNNING to SUCCEEDED
> 2016-12-23 09:10:38,642 INFO [AsyncDispatcher event handler]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 5
> 2016-12-23 09:10:38,643 INFO [AsyncDispatcher event handler]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl:
> job_1482404625971_23910Job Transitioned from RUNNING to COMMITTING
> 2016-12-23 09:10:38,644 INFO [ContainerLauncher #5]
> org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing
> the event EventType: CONTAINER_REMOTE_CLEANUP for container
> container_e55_1482404625971_23910_01_10 taskAttempt
> attempt_1482404625971_23910_m_04_1
> 2016-12-23 09:10:38,644 INFO [ContainerLauncher #5]
> org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING
> attempt_1482404625971_23910_m_04_1
> 2016-12-23 09:10:38,644 INFO [ContainerLauncher #5]
> org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy:
> Opening proxy : linux-19:26009
> 2016-12-23 09:10:38,644 INFO [CommitterEvent Processor #4]
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing
> the event EventType: JOB_COMMIT
> 2016-12-23 09:10:38,724 INFO [IPC Server handler 0 on 27113]
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID :
> jvm_1482404625971_23910_m_60473139527690 asked for a task
> 2016-12-23 09:10:38,724 INFO [IPC Server handler 0 on 27113]
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID:
> jvm_1482404625971_23910_m_60473139527690 is invalid and will be killed.
> 2016-12-23 09:10:38,797 INFO [AsyncDispatcher event handler]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Calling handler for
> JobFinishedEvent
> 2016-12-23 09:10:38,797 INFO [AsyncDispatcher event handler]
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl:
> job_1482404625971_23910Job Transitioned from COMMITTING to SUCCEEDED
> 2016-12-23 09:10:38,798 INFO [Thread-93]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Job finished cleanly,
> recording last MRAppMaster retry
> 2016-12-23 09:10:38,798 INFO [Thread-93]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator
> isAMLastRetry: true
> 2016-12-23 09:10:38,798 INFO [Thread-93]
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: RMCommunicator notified
> that shouldUnregistered is: true
> 2016-12-23 09:10:38,799 INFO [Thread-93]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry:
> true
> 2016-12-23 09:10:38,799 INFO [Thread-93]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler:
> JobHistoryEventHandler notified that forceJobCompletion is true
> 2016-12-23 09:10:38,799 INFO [Thread-93]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the
> services
> 2016-12-23 09:10:38,800 INFO [Thread-93]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping
> JobHistoryEventHandler. Size of the outstanding queue size is 1
> 2016-12-23 09:10:38,989 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before
> Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1
> AssignedReds:0 CompletedMaps:5 CompletedReds:0 ContAlloc:8 ContRel:0
> HostLocal:0 RackLocal:0
> 2016-12-23 09:10:38,993 INFO [RMCommunicator Allocator]
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received
> completed container container_e55_1482404625971_23910_01_10
> 2016-12-23 09:10:38,993 INFO [RMCommunicator Allocator]
>
[jira] [Commented] (MAPREDUCE-7073) Optimize TokenCache#obtainTokensForNamenodesInternal
[
https://issues.apache.org/jira/browse/MAPREDUCE-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433686#comment-16433686
]
Bibin A Chundatt commented on MAPREDUCE-7073:
-
[~jianhe]
Could you please help in review ?
> Optimize TokenCache#obtainTokensForNamenodesInternal
>
>
> Key: MAPREDUCE-7073
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7073
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Major
> Attachments: MAPREDUCE-7073.001.patch, MAPREDUCE-7073.002.patch
>
>
> {{FileInputFormat#listStatus}} is too slow file system cache is disabled.
> {{TokenCache#obtainTokensForNamenodesInternal}} for every filesystem instance
> {{Master.getMasterPrincipal(conf)}} is caled which reloads YarnConfiguration .
> For fileInput with 1k file will reload YarnConfiguration 1k times.
> {{Master.getMasterPrincipal(conf)}} can be passed for
> {{obtainTokensForNamenodesInternal}} per filesystem call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7073) Optimize TokenCache#obtainTokensForNamenodesInternal
[
https://issues.apache.org/jira/browse/MAPREDUCE-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-7073:
Attachment: MAPREDUCE-7073.002.patch
> Optimize TokenCache#obtainTokensForNamenodesInternal
>
>
> Key: MAPREDUCE-7073
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7073
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Major
> Attachments: MAPREDUCE-7073.001.patch, MAPREDUCE-7073.002.patch
>
>
> {{FileInputFormat#listStatus}} is too slow file system cache is disabled.
> {{TokenCache#obtainTokensForNamenodesInternal}} for every filesystem instance
> {{Master.getMasterPrincipal(conf)}} is caled which reloads YarnConfiguration .
> For fileInput with 1k file will reload YarnConfiguration 1k times.
> {{Master.getMasterPrincipal(conf)}} can be passed for
> {{obtainTokensForNamenodesInternal}} per filesystem call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7073) Optimize TokenCache#obtainTokensForNamenodesInternal
[
https://issues.apache.org/jira/browse/MAPREDUCE-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16430324#comment-16430324
]
Bibin A Chundatt commented on MAPREDUCE-7073:
-
Issue credits to [~lindongdong]
> Optimize TokenCache#obtainTokensForNamenodesInternal
>
>
> Key: MAPREDUCE-7073
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7073
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Major
> Attachments: MAPREDUCE-7073.001.patch
>
>
> {{FileInputFormat#listStatus}} is too slow file system cache is disabled.
> {{TokenCache#obtainTokensForNamenodesInternal}} for every filesystem instance
> {{Master.getMasterPrincipal(conf)}} is caled which reloads YarnConfiguration .
> For fileInput with 1k file will reload YarnConfiguration 1k times.
> {{Master.getMasterPrincipal(conf)}} can be passed for
> {{obtainTokensForNamenodesInternal}} per filesystem call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7073) Optimize TokenCache#obtainTokensForNamenodesInternal
[
https://issues.apache.org/jira/browse/MAPREDUCE-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-7073:
Description:
{{FileInputFormat#listStatus}} is too slow file system cache is disabled.
{{TokenCache#obtainTokensForNamenodesInternal}} for every filesystem instance
{{Master.getMasterPrincipal(conf)}} is caled which reloads YarnConfiguration .
For fileInput with 1k file will reload YarnConfiguration 1k times.
{{Master.getMasterPrincipal(conf)}} can be passed for
{{obtainTokensForNamenodesInternal}} per filesystem call.
was:
{{FileInputFormat#listStatus}} is too slow file system cache is disabled.
{{TokenCache#obtainTokensForNamenodesInternal}} for every filesystem instance
{{Master.getMasterPrincipal(conf)}} is caled which reloads YarnConfiguration .
For fileInput with 1k file will reload YarnConfiguration 1k times.
{{Master.getMasterPrincipal(conf)}} can be passed for
{{obtainTokensForNamenodesInternal}} per filesystem call.
> Optimize TokenCache#obtainTokensForNamenodesInternal
>
>
> Key: MAPREDUCE-7073
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7073
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Major
> Attachments: MAPREDUCE-7073.001.patch
>
>
> {{FileInputFormat#listStatus}} is too slow file system cache is disabled.
> {{TokenCache#obtainTokensForNamenodesInternal}} for every filesystem instance
> {{Master.getMasterPrincipal(conf)}} is caled which reloads YarnConfiguration .
> For fileInput with 1k file will reload YarnConfiguration 1k times.
> {{Master.getMasterPrincipal(conf)}} can be passed for
> {{obtainTokensForNamenodesInternal}} per filesystem call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7073) Optimize TokenCache#obtainTokensForNamenodesInternal
[
https://issues.apache.org/jira/browse/MAPREDUCE-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-7073:
Status: Patch Available (was: Open)
> Optimize TokenCache#obtainTokensForNamenodesInternal
>
>
> Key: MAPREDUCE-7073
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7073
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Major
> Attachments: MAPREDUCE-7073.001.patch
>
>
> {{FileInputFormat#listStatus}} is too slow file system cache is disabled.
> {{TokenCache#obtainTokensForNamenodesInternal}} for every filesystem instance
> {{Master.getMasterPrincipal(conf)}} is caled which reloads YarnConfiguration .
> For fileInput with 1k file will reload YarnConfiguration 1k times.
> {{Master.getMasterPrincipal(conf)}} can be passed for
> {{obtainTokensForNamenodesInternal}} per filesystem call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7073) Optimize TokenCache#obtainTokensForNamenodesInternal
[
https://issues.apache.org/jira/browse/MAPREDUCE-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-7073:
Attachment: MAPREDUCE-7073.001.patch
> Optimize TokenCache#obtainTokensForNamenodesInternal
>
>
> Key: MAPREDUCE-7073
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7073
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Major
> Attachments: MAPREDUCE-7073.001.patch
>
>
> {{FileInputFormat#listStatus}} is too slow file system cache is disabled.
> {{TokenCache#obtainTokensForNamenodesInternal}} for every filesystem instance
> {{Master.getMasterPrincipal(conf)}} is caled which reloads YarnConfiguration .
> For fileInput with 1k file will reload YarnConfiguration 1k times.
> {{Master.getMasterPrincipal(conf)}} can be passed for
> {{obtainTokensForNamenodesInternal}} per filesystem call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7073) Optimize TokenCache#obtainTokensForNamenodesInternal
[
https://issues.apache.org/jira/browse/MAPREDUCE-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-7073:
Description:
{{FileInputFormat#listStatus}} is too slow file system cache is disabled.
{{TokenCache#obtainTokensForNamenodesInternal}} for every filesystem instance
{{Master.getMasterPrincipal(conf)}} is caled which reloads YarnConfiguration .
For fileInput with 1k file will reload YarnConfiguration 1k times.
{{Master.getMasterPrincipal(conf)}} can be passed for
{{obtainTokensForNamenodesInternal}} per filesystem call.
was:
FileInputFormat#listStatus is too slow file system cache is disabled.
{{TokenCache#obtainTokensForNamenodesInternal}} for every filesystem instance
{{Master.getMasterPrincipal(conf)}} is caled which reloads YarnConfiguration .
For fileInput with 1k file will reload YarnConfiguration 1k times.
{{Master.getMasterPrincipal(conf)}} can be passed for
{{obtainTokensForNamenodesInternal}} per filesystem call.
> Optimize TokenCache#obtainTokensForNamenodesInternal
>
>
> Key: MAPREDUCE-7073
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7073
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Major
>
> {{FileInputFormat#listStatus}} is too slow file system cache is disabled.
> {{TokenCache#obtainTokensForNamenodesInternal}} for every filesystem instance
> {{Master.getMasterPrincipal(conf)}} is caled which reloads YarnConfiguration .
> For fileInput with 1k file will reload YarnConfiguration 1k times.
> {{Master.getMasterPrincipal(conf)}} can be passed for
> {{obtainTokensForNamenodesInternal}} per filesystem call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Assigned] (MAPREDUCE-7073) Optimize TokenCache#obtainTokensForNamenodesInternal
[
https://issues.apache.org/jira/browse/MAPREDUCE-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt reassigned MAPREDUCE-7073:
---
Assignee: Bibin A Chundatt
> Optimize TokenCache#obtainTokensForNamenodesInternal
>
>
> Key: MAPREDUCE-7073
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7073
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Major
>
> FileInputFormat#listStatus is too slow file system cache is disabled.
> {{TokenCache#obtainTokensForNamenodesInternal}} for every filesystem instance
> {{Master.getMasterPrincipal(conf)}} is caled which reloads YarnConfiguration .
> For fileInput with 1k file will reload YarnConfiguration 1k times.
> {{Master.getMasterPrincipal(conf)}} can be passed for
> {{obtainTokensForNamenodesInternal}} per filesystem call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-7073) Optimize TokenCache#obtainTokensForNamenodesInternal
Bibin A Chundatt created MAPREDUCE-7073:
---
Summary: Optimize TokenCache#obtainTokensForNamenodesInternal
Key: MAPREDUCE-7073
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7073
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
FileInputFormat#listStatus is too slow file system cache is disabled.
{{TokenCache#obtainTokensForNamenodesInternal}} for every filesystem instance
{{Master.getMasterPrincipal(conf)}} is caled which reloads YarnConfiguration .
For fileInput with 1k file will reload YarnConfiguration 1k times.
{{Master.getMasterPrincipal(conf)}} can be passed for
{{obtainTokensForNamenodesInternal}} per filesystem call.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6987) JHS Log Scanner and Cleaner blocked
[
https://issues.apache.org/jira/browse/MAPREDUCE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16216736#comment-16216736
]
Bibin A Chundatt commented on MAPREDUCE-6987:
-
Issue fixed as part of HADOOP-14966
> JHS Log Scanner and Cleaner blocked
> ---
>
> Key: MAPREDUCE-6987
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6987
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.9.0, 3.0.0-alpha1
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Fix For: 2.9.0, 3.0.0, 3.1.0
>
>
> {code}
> "Log Scanner/Cleaner #1" #81 prio=5 os_prio=0 tid=0x7fd6c010f000
> nid=0x11db waiting on condition [0x7fd6aa859000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xd6c88a80> (a
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
> at
> org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> "Log Scanner/Cleaner #0" #80 prio=5 os_prio=0 tid=0x7fd6c010c800
> nid=0x11da waiting on condition [0x7fd6aa95a000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xd6c8> (a
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
> at
> org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Both threads waiting on {{FutureTask.get()}} for infinite time after first
> execution
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Resolved] (MAPREDUCE-6987) JHS Log Scanner and Cleaner blocked
[
https://issues.apache.org/jira/browse/MAPREDUCE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt resolved MAPREDUCE-6987.
-
Resolution: Fixed
Fix Version/s: 3.1.0
3.0.0
2.9.0
> JHS Log Scanner and Cleaner blocked
> ---
>
> Key: MAPREDUCE-6987
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6987
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.9.0, 3.0.0-alpha1
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Fix For: 2.9.0, 3.0.0, 3.1.0
>
>
> {code}
> "Log Scanner/Cleaner #1" #81 prio=5 os_prio=0 tid=0x7fd6c010f000
> nid=0x11db waiting on condition [0x7fd6aa859000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xd6c88a80> (a
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
> at
> org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> "Log Scanner/Cleaner #0" #80 prio=5 os_prio=0 tid=0x7fd6c010c800
> nid=0x11da waiting on condition [0x7fd6aa95a000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xd6c8> (a
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
> at
> org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Both threads waiting on {{FutureTask.get()}} for infinite time after first
> execution
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6987) JHS Log Scanner and Cleaner blocked
[
https://issues.apache.org/jira/browse/MAPREDUCE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6987:
Affects Version/s: 2.9.0
Target Version/s: 2.9.0, 3.0.0, 3.1.0
> JHS Log Scanner and Cleaner blocked
> ---
>
> Key: MAPREDUCE-6987
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6987
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.9.0, 3.0.0-alpha1
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
>
> {code}
> "Log Scanner/Cleaner #1" #81 prio=5 os_prio=0 tid=0x7fd6c010f000
> nid=0x11db waiting on condition [0x7fd6aa859000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xd6c88a80> (a
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
> at
> org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> "Log Scanner/Cleaner #0" #80 prio=5 os_prio=0 tid=0x7fd6c010c800
> nid=0x11da waiting on condition [0x7fd6aa95a000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xd6c8> (a
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
> at
> org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Both threads waiting on {{FutureTask.get()}} for infinite time after first
> execution
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6987) JHS Log Scanner and Cleaner blocked
[
https://issues.apache.org/jira/browse/MAPREDUCE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6987:
Affects Version/s: (was: 2.8.0)
> JHS Log Scanner and Cleaner blocked
> ---
>
> Key: MAPREDUCE-6987
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6987
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 3.0.0-alpha1
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
>
> {code}
> "Log Scanner/Cleaner #1" #81 prio=5 os_prio=0 tid=0x7fd6c010f000
> nid=0x11db waiting on condition [0x7fd6aa859000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xd6c88a80> (a
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
> at
> org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> "Log Scanner/Cleaner #0" #80 prio=5 os_prio=0 tid=0x7fd6c010c800
> nid=0x11da waiting on condition [0x7fd6aa95a000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xd6c8> (a
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
> at
> org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Both threads waiting on {{FutureTask.get()}} for infinite time after first
> execution
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6987) JHS Log Scanner and Cleaner blocked
[
https://issues.apache.org/jira/browse/MAPREDUCE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212387#comment-16212387
]
Bibin A Chundatt commented on MAPREDUCE-6987:
-
Removed 2.8 since its not applicable , Thank you [~jlowe] for pointing out
> JHS Log Scanner and Cleaner blocked
> ---
>
> Key: MAPREDUCE-6987
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6987
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 3.0.0-alpha1
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
>
> {code}
> "Log Scanner/Cleaner #1" #81 prio=5 os_prio=0 tid=0x7fd6c010f000
> nid=0x11db waiting on condition [0x7fd6aa859000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xd6c88a80> (a
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
> at
> org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> "Log Scanner/Cleaner #0" #80 prio=5 os_prio=0 tid=0x7fd6c010c800
> nid=0x11da waiting on condition [0x7fd6aa95a000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xd6c8> (a
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
> at
> org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Both threads waiting on {{FutureTask.get()}} for infinite time after first
> execution
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6987) JHS Log Scanner and Cleaner blocked
[
https://issues.apache.org/jira/browse/MAPREDUCE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6987:
Affects Version/s: (was: 3.0.0)
3.0.0-alpha1
> JHS Log Scanner and Cleaner blocked
> ---
>
> Key: MAPREDUCE-6987
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6987
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.8.0, 3.0.0-alpha1
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
>
> {code}
> "Log Scanner/Cleaner #1" #81 prio=5 os_prio=0 tid=0x7fd6c010f000
> nid=0x11db waiting on condition [0x7fd6aa859000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xd6c88a80> (a
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
> at
> org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> "Log Scanner/Cleaner #0" #80 prio=5 os_prio=0 tid=0x7fd6c010c800
> nid=0x11da waiting on condition [0x7fd6aa95a000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xd6c8> (a
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
> at
> org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Both threads waiting on {{FutureTask.get()}} for infinite time after first
> execution
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6987) JHS Log Scanner and Cleaner blocked
[
https://issues.apache.org/jira/browse/MAPREDUCE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6987:
Affects Version/s: 3.0.0
2.8.0
Description:
{code}
"Log Scanner/Cleaner #1" #81 prio=5 os_prio=0 tid=0x7fd6c010f000 nid=0x11db
waiting on condition [0x7fd6aa859000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xd6c88a80> (a
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
at java.util.concurrent.FutureTask.get(FutureTask.java:191)
at
org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
at
org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
"Log Scanner/Cleaner #0" #80 prio=5 os_prio=0 tid=0x7fd6c010c800 nid=0x11da
waiting on condition [0x7fd6aa95a000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xd6c8> (a
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
at java.util.concurrent.FutureTask.get(FutureTask.java:191)
at
org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
at
org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
Both threads waiting on {{FutureTask.get()}} for infinite time after first
execution
Component/s: jobhistoryserver
> JHS Log Scanner and Cleaner blocked
> ---
>
> Key: MAPREDUCE-6987
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6987
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.8.0, 3.0.0
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
>
> {code}
> "Log Scanner/Cleaner #1" #81 prio=5 os_prio=0 tid=0x7fd6c010f000
> nid=0x11db waiting on condition [0x7fd6aa859000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xd6c88a80> (a
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
> at
> org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> "Log Scanner/Cleaner #0" #80 prio=5 os_prio=0 tid=0x7fd6c010c800
> nid=0x11da waiting on condition [0x7fd6aa95a000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xd6c8> (a
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
> at
> org.apache.hadoop.util.concurrent.ExecutorHelper.logThrowableFromAfterExecute(ExecutorHelper.java:47)
> at
> org.apache.hadoop.util.concurrent.HadoopScheduledThreadPoolExecutor.afterExecute(HadoopScheduledThreadPoolExecutor.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
> at
>
[jira] [Assigned] (MAPREDUCE-6987) JHS Log Scanner and Cleaner blocked
[ https://issues.apache.org/jira/browse/MAPREDUCE-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bibin A Chundatt reassigned MAPREDUCE-6987: --- Assignee: Bibin A Chundatt > JHS Log Scanner and Cleaner blocked > --- > > Key: MAPREDUCE-6987 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6987 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6987) JHS Log Scanner and Cleaner blocked
Bibin A Chundatt created MAPREDUCE-6987: --- Summary: JHS Log Scanner and Cleaner blocked Key: MAPREDUCE-6987 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6987 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Bibin A Chundatt Priority: Critical -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6704:
Attachment: MAPREDUCE-6704.0002.patch
Attaching same patch again
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> ClusterSetup.html, MAPREDUCE-6704.0002.patch, MAPREDUCE-6704.0002.patch,
> MR-6704-branch2.8.tar.gz, MR-6704-trunk-tempPatch.tar.gz,
> MR-6704-trunk.tar.gz, SingleCluster.html, container-whitelist-env-wip.patch,
> temp.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6704:
Status: Patch Available (was: Open)
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> ClusterSetup.html, MAPREDUCE-6704.0002.patch, MAPREDUCE-6704.0002.patch,
> MR-6704-branch2.8.tar.gz, MR-6704-trunk-tempPatch.tar.gz,
> MR-6704-trunk.tar.gz, SingleCluster.html, container-whitelist-env-wip.patch,
> temp.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15753457#comment-15753457
]
Bibin A Chundatt commented on MAPREDUCE-6704:
-
[~andrew.wang]
Yes we are getting close to resolving.Only documentation update is required as
part of this jira.
{{MAPREDUCE-6704.0002}} should get committed after YARN-5877 is resolved.
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> ClusterSetup.html, MAPREDUCE-6704.0002.patch, MR-6704-branch2.8.tar.gz,
> MR-6704-trunk-tempPatch.tar.gz, MR-6704-trunk.tar.gz, SingleCluster.html,
> container-whitelist-env-wip.patch, temp.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15689002#comment-15689002
]
Bibin A Chundatt edited comment on MAPREDUCE-6704 at 11/23/16 5:36 AM:
---
[~tangzhankun] thank you for providing test result. YARN-5877 is added to track
the same.
[~templedf] As per
[discussion|https://issues.apache.org/jira/browse/MAPREDUCE-6704?focusedCommentId=15596876=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15596876]
we had identified that Solution 3 is best fit. But for docker to run by
default we should support change YARN-5877 (Currently the vars to be fetched
from env for docker is fixed we should make it configurable) .[Comment
|https://issues.apache.org/jira/browse/MAPREDUCE-6704?focusedCommentId=15661038=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15661038]
explains in details why we should support.
So combination of both would fix completely.
was (Author: bibinchundatt):
[~tangzhankun] thank you for providing test result. YARN-5877 is added to track
the same.
[~templedf] As per
[discussion|https://issues.apache.org/jira/browse/MAPREDUCE-6704?focusedCommentId=15596876=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15596876]
we had identified that Solution 3 is best fit. But for docker to run by
default we should support change YARN-5877 (Currently the vars to be fetched
from env for docker is fixed we should make it configurable)
.[comment2|https://issues.apache.org/jira/browse/MAPREDUCE-6704?focusedCommentId=15661038=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15661038]
explains in details why we should support.
So combination of both will would fix completely.
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> ClusterSetup.html, MAPREDUCE-6704.0002.patch, MR-6704-branch2.8.tar.gz,
> MR-6704-trunk-tempPatch.tar.gz, MR-6704-trunk.tar.gz, SingleCluster.html,
> container-whitelist-env-wip.patch, temp.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15689002#comment-15689002
]
Bibin A Chundatt commented on MAPREDUCE-6704:
-
[~tangzhankun] thank you for providing test result. YARN-5877 is added to track
the same.
[~templedf] As per
[discussion|https://issues.apache.org/jira/browse/MAPREDUCE-6704?focusedCommentId=15596876=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15596876]
we had identified that Solution 3 is best fit. But for docker to run by
default we should support change YARN-5877 (Currently the vars to be fetched
from env for docker is fixed we should make it configurable)
.[comment2|https://issues.apache.org/jira/browse/MAPREDUCE-6704?focusedCommentId=15661038=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15661038]
explains in details why we should support.
So combination of both will would fix completely.
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> ClusterSetup.html, MAPREDUCE-6704.0002.patch, MR-6704-branch2.8.tar.gz,
> MR-6704-trunk-tempPatch.tar.gz, MR-6704-trunk.tar.gz, SingleCluster.html,
> container-whitelist-env-wip.patch, temp.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15662943#comment-15662943
]
Bibin A Chundatt edited comment on MAPREDUCE-6704 at 11/14/16 6:45 AM:
---
[~tangzhankun] could be related docker mount , YARN-5765 or 2.7 to hadoop.3.0
compatability Sorry i am new to docker so unaware
[~templedf] Added YARN-5877 to track white list case.
was (Author: bibinchundatt):
[~tangzhankun] could be related docker mount , YARN-5765, Sorry i am new to
docker
[~templedf] Added YARN-5877 to track white list case.
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> ClusterSetup.html, MAPREDUCE-6704.0002.patch, MR-6704-branch2.8.tar.gz,
> MR-6704-trunk-tempPatch.tar.gz, MR-6704-trunk.tar.gz, SingleCluster.html,
> container-whitelist-env-wip.patch, temp.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15662943#comment-15662943
]
Bibin A Chundatt commented on MAPREDUCE-6704:
-
[~tangzhankun] could be related docker mount , YARN-5765, Sorry i am new to
docker
[~templedf] Added YARN-5877 to track white list case.
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> ClusterSetup.html, MAPREDUCE-6704.0002.patch, MR-6704-branch2.8.tar.gz,
> MR-6704-trunk-tempPatch.tar.gz, MR-6704-trunk.tar.gz, SingleCluster.html,
> container-whitelist-env-wip.patch, temp.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15662639#comment-15662639
]
Bibin A Chundatt commented on MAPREDUCE-6704:
-
Not able to read token files could be rights related also. Could you check
rights of nm-local-dir is 755.If its a test setup try removing
{{nm-local-dir/usercache/yarn}} dir and run again.
[~templedf]
Once you look into the {{temp.patch}} will file a jira in yarn
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> ClusterSetup.html, MAPREDUCE-6704.0002.patch, MR-6704-branch2.8.tar.gz,
> MR-6704-trunk-tempPatch.tar.gz, MR-6704-trunk.tar.gz, SingleCluster.html,
> container-whitelist-env-wip.patch, temp.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6704:
Attachment: temp.patch
[~tangzhankun]
Thank you for attaching logs. Had quick look are the launch_container.sh
{{HADOOP_MAPRED_HOME}} patch is set based nodemanager which should have worked
in all cases.As per my understanding could fail only when the
{{HADOOP_MAPRED_HOME}} where the container do launch is different.
Could you try based on {{temp.patch}} .Currently i don't have docker setup to
test my analysis.
{code}
+++
b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java
@@ -320,11 +320,11 @@ public void writeLaunchEnv(OutputStream out, Map environment,
ContainerLaunch.ShellScriptBuilder.create();
Set whitelist = new HashSet<>();
-whitelist.add(ApplicationConstants.Environment.HADOOP_YARN_HOME.name());
-whitelist.add(ApplicationConstants.Environment.HADOOP_COMMON_HOME.name());
-whitelist.add(ApplicationConstants.Environment.HADOOP_HDFS_HOME.name());
-whitelist.add(ApplicationConstants.Environment.HADOOP_CONF_DIR.name());
-whitelist.add(ApplicationConstants.Environment.JAVA_HOME.name());
+String[] nmWhiteList = conf.get(YarnConfiguration.NM_ENV_WHITELIST,
+YarnConfiguration.DEFAULT_NM_ENV_WHITELIST).split(",");
+for (String param : nmWhiteList) {
+ whitelist.add(param);
+}
{code}
[~templedf] {{ContainerExecutor#writeLaunchEnv}} is handling only the default
whitelist params as per current code.Looks like bug in yarn.
The above change should allow taking the {{HADOOP_MAPRED_HOME}} from docker
env during container launch.
{{launcher_container.sh}} mapred_home_patch will get exported as following.
{noformat}
export HADOOP_MAPRED_HOME=${HADOOP_MAPRED_HOME:-"/home/hadoopbuild"}
{noformat}
instead of
{noformat}
export HADOOP_MAPRED_HOME="/home/hadoopbuild"
{noformat}
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> ClusterSetup.html, MAPREDUCE-6704.0002.patch, MR-6704-branch2.8.tar.gz,
> MR-6704-trunk.tar.gz, SingleCluster.html, container-whitelist-env-wip.patch,
> temp.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096
[jira] [Updated] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6704:
Status: Open (was: Patch Available)
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> ClusterSetup.html, MAPREDUCE-6704.0002.patch, MR-6704-branch2.8.tar.gz,
> MR-6704-trunk.tar.gz, SingleCluster.html, container-whitelist-env-wip.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654304#comment-15654304
]
Bibin A Chundatt edited comment on MAPREDUCE-6704 at 11/10/16 3:18 PM:
---
[~templedf]
I was thinking of handling through mailing list and analyze the logs . We will
other any other env needs to specified
[~tangzhankun]
As Daniel mentioned after configuring
{{yarn.nodemanager.delete.debug-delay-sec}} please attach the following
# launch script
# node manager logs during docker run.
was (Author: bibinchundatt):
[~templedf]
Was thinking of handling though mailing list and analyze the logs . Will other
any other env needs specified.
[~tangzhankun]
As Daniel mentioned after configuring
{{yarn.nodemanager.delete.debug-delay-sec}} please attach the following
# launch script
# node manager logs during docker run.
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> ClusterSetup.html, MAPREDUCE-6704.0002.patch, SingleCluster.html,
> container-whitelist-env-wip.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654304#comment-15654304
]
Bibin A Chundatt commented on MAPREDUCE-6704:
-
[~templedf]
Was thinking of handling though mailing list and analyze the logs . Will other
any other env needs specified.
[~tangzhankun]
As Daniel mentioned after configuring
{{yarn.nodemanager.delete.debug-delay-sec}} please attach the following
# launch script
# node manager logs during docker run.
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> ClusterSetup.html, MAPREDUCE-6704.0002.patch, SingleCluster.html,
> container-whitelist-env-wip.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653702#comment-15653702
]
Bibin A Chundatt commented on MAPREDUCE-6704:
-
[~tangzhankun]
{quote}
I used "sequenceiq/hadoop-docker" Docker image. The image works in branch-2.8.
Any idea?
{quote}
Sorry,No idea. Check with hadoop mailing list with logs
[~templedf]
Any more changes required for this jira
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> ClusterSetup.html, MAPREDUCE-6704.0002.patch, SingleCluster.html,
> container-whitelist-env-wip.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15643506#comment-15643506
]
Bibin A Chundatt commented on MAPREDUCE-6704:
-
{quote}
However do we need to whitelist JAVA_HOME by default?
{quote}
Its already part of the default list
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> ClusterSetup.html, MAPREDUCE-6704.0002.patch, SingleCluster.html,
> container-whitelist-env-wip.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6704:
Attachment: SingleCluster.html
ClusterSetup.html
Thank you [~templedf] for looking in to issue .
Attaching html files for reference.
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> ClusterSetup.html, MAPREDUCE-6704.0002.patch, SingleCluster.html,
> container-whitelist-env-wip.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6704:
Attachment: MAPREDUCE-6704.0002.patch
Thank you all . Attaching patch based on solution 3
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> MAPREDUCE-6704.0002.patch, container-whitelist-env-wip.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Assigned] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt reassigned MAPREDUCE-6704:
---
Assignee: Bibin A Chundatt
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> container-whitelist-env-wip.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15617802#comment-15617802
]
Bibin A Chundatt edited comment on MAPREDUCE-6704 at 10/29/16 10:56 AM:
Thank you [~templedf] and [~rkanter] for inputs
#3 since for upgrade case when we have running apps and havnt used
distributed cache.
Need to modify only one config during upgrade and everything works as old.
Another case can happen when old clients try to submit apps too rt?
{quote}
client doesn't have the same paths as the NM hosts, it won't work.
{quote}
[~templedf] Thank you for bring this scenarios too had missed out.
Will do document update and upload patch soon.
was (Author: bibinchundatt):
Thank you [~templedf] and [~rkanter] for inputs
#3 since for upgrade case when already application we have running apps and
havnt used distributed cache.
Need to modify only one config during upgrade and everything works as old.
Another case can happen when old clients try to submit apps too rt?
{quote}
client doesn't have the same paths as the NM hosts, it won't work.
{quote}
[~templedf] Thank you for bring this scenarios too had missed out.
Will do document update and upload patch soon.
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> container-whitelist-env-wip.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15617802#comment-15617802
]
Bibin A Chundatt commented on MAPREDUCE-6704:
-
Thank you [~templedf] and [~rkanter] for inputs
#3 since for upgrade case when already application we have running apps and
havnt used distributed cache.
Need to modify only one config during upgrade and everything works as old.
Another case can happen when old clients try to submit apps too rt?
{quote}
client doesn't have the same paths as the NM hosts, it won't work.
{quote}
[~templedf] Thank you for bring this scenarios too had missed out.
Will do document update and upload patch soon.
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> container-whitelist-env-wip.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6704:
Assignee: (was: Bibin A Chundatt)
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> container-whitelist-env-wip.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6440) Duplicate Key in Json Output for Job details
[ https://issues.apache.org/jira/browse/MAPREDUCE-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bibin A Chundatt updated MAPREDUCE-6440: Assignee: (was: Bibin A Chundatt) > Duplicate Key in Json Output for Job details > > > Key: MAPREDUCE-6440 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6440 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: jobhistoryserver >Reporter: Anushri >Priority: Minor > > Duplicate key in Json Output for Job details for the url : > http://:/ws/v1/history/mapreduce/jobs/job_id/tasks/task_id/attempts > If the task type is "REDUCE" the json output for this url contains duplicate > key for "type". -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6704:
Attachment: container-whitelist-env-wip.patch
Solution 4:
We should add option in YARN to support sending addition whitelist ENV as part
of ContainerLaunchContext. Addition patch for solution 4 also
{{container-whitelist-env-wip.patch}} . created a jira YARN-5771 also.
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch,
> container-whitelist-env-wip.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596876#comment-15596876
]
Bibin A Chundatt commented on MAPREDUCE-6704:
-
[~rkanter]
{quote}
Given the difficulty people seem to be having and that there doesn't seem to be
a single fix that works for everyone for some reason, perhaps we should revisit
that decision?
{quote}
IMHO we have to rethink of the same.
Summarizing discussion and solutions
# Add HADOOP_MAPRED_HOME=HADOOP_COMMON_HOME in opts.But its not mandatory that
MAPRED_HOME=HADOOP_COMMON_HOME
# Add HADOOP_MAPRED_HOME to Yarn.Since we want to keep YARN and MAPRED separate
initial solution was not accepted.
# Add documentation to configure yarn.nodemanager.env-whitelist in nodemanager
to run mapred application
{noformat}
yarn.nodemanager.env-whitelist
JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME
{noformat}
Waiting for +1 for any one of above solution or inputs for any other approach
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15582894#comment-15582894
]
Bibin A Chundatt commented on MAPREDUCE-6704:
-
[~yzhangal]
Looks like you have made wrong configuration
{noformat}
=JAVA_HOME
{noformat}
Could please remove {{=}} and try again. I thought of documenting the white
list parameter configuration and closing the issue waiting for +1 which
solution to take.
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6782) JHS task page search based on each individual column not working
[
https://issues.apache.org/jira/browse/MAPREDUCE-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524997#comment-15524997
]
Bibin A Chundatt commented on MAPREDUCE-6782:
-
[~gu chi]
IIUC the change you mentioned was done as part of YARN-237.
> JHS task page search based on each individual column not working
>
>
> Key: MAPREDUCE-6782
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6782
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: gu-chi
>
> Submit mapreduce pi job with 10 maps
> In Jobs history server selection completed job
> Select maps to Task Page for job
> Search in individual column fields
> *Expected*
> Search should be working fine in task page for individual columns
> *Actual*
> Search not working for individual column in task page
> In Attempts page the same search is working fine
> {noformat}
> jquery.dataTables.min.js:109
> Uncaught TypeError: Cannot read property 'oFeatures' of null
> fnFilter @ jquery.dataTables.min.js:109(anonymous function) @ m:49dispatch
> @ jquery-1.8.2.min.js:2h @ jquery-1.8.2.min.js:2
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Assigned] (MAPREDUCE-6782) JHS task page search based on each individual column not working
[
https://issues.apache.org/jira/browse/MAPREDUCE-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt reassigned MAPREDUCE-6782:
---
Assignee: Bibin A Chundatt
> JHS task page search based on each individual column not working
>
>
> Key: MAPREDUCE-6782
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6782
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>
> Submit mapreduce pi job with 10 maps
> In Jobs history server selection completed job
> Select maps to Task Page for job
> Search in individual column fields
> *Expected*
> Search should be working fine in task page for individual columns
> *Actual*
> Search not working for individual column in task page
> In Attempts page the same search is working fine
> {noformat}
> jquery.dataTables.min.js:109
> Uncaught TypeError: Cannot read property 'oFeatures' of null
> fnFilter @ jquery.dataTables.min.js:109(anonymous function) @ m:49dispatch
> @ jquery-1.8.2.min.js:2h @ jquery-1.8.2.min.js:2
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6782) JHS task page search based on each individual column not working
Bibin A Chundatt created MAPREDUCE-6782:
---
Summary: JHS task page search based on each individual column not
working
Key: MAPREDUCE-6782
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6782
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
Submit mapreduce pi job with 10 maps
In Jobs history server selection completed job
Select maps to Task Page for job
Search in individual column fields
*Expected*
Search should be working fine in task page for individual columns
*Actual*
Search not working for individual column in task page
In Attempts page the same search is working fine
{noformat}
jquery.dataTables.min.js:109
Uncaught TypeError: Cannot read property 'oFeatures' of null
fnFilter @ jquery.dataTables.min.js:109(anonymous function) @ m:49dispatch
@ jquery-1.8.2.min.js:2h @ jquery-1.8.2.min.js:2
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509009#comment-15509009
]
Bibin A Chundatt edited comment on MAPREDUCE-6704 at 9/21/16 7:10 AM:
--
[~tangzhankun]
In nodemanager we can add the below config. Last time i checked below solution
was working fine.
yarn.nodemanager.env-whitelist=
"JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME"
was (Author: bibinchundatt):
[~tangzhankun]
In nodemanager we can add we can add the below config last time i checked below
solution was working fine.
yarn.nodemanager.env-whitelist=
"JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME"
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15509009#comment-15509009
]
Bibin A Chundatt commented on MAPREDUCE-6704:
-
[~tangzhankun]
In nodemanager we can add we can add the below config last time i checked below
solution was working fine.
yarn.nodemanager.env-whitelist=
"JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME"
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6779) Mapreduce job failure on submission
[
https://issues.apache.org/jira/browse/MAPREDUCE-6779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6779:
Assignee: (was: Bibin A Chundatt)
> Mapreduce job failure on submission
> ---
>
> Key: MAPREDUCE-6779
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6779
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Configure Hibench
> Try running Enhanced TestDFSIO
> {noformat}
> 2016-09-15 18:20:24,849 INFO mapreduce.Job: Task Id :
> attempt_1473943118844_0001_m_01_0, Status : FAILED
> Error: java.lang.RuntimeException: Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1806)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
> ... 9 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2330)
> at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:1108)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
> ... 14 more
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2298)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2322)
> ... 16 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2202)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2296)
> ... 17 more
> {noformat}
> *mapreduce.JobResourceUploader: No job jar file set. User classes may not be
> found. See Job or Job#setJar(String).*
> Job jar is not getting set and not uploaded to staging dir
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (MAPREDUCE-6779) Mapreduce job failure on submission
[
https://issues.apache.org/jira/browse/MAPREDUCE-6779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6779:
Comment: was deleted
(was: We can close this jira once main jira is reverted)
> Mapreduce job failure on submission
> ---
>
> Key: MAPREDUCE-6779
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6779
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
>
> Configure Hibench
> Try running Enhanced TestDFSIO
> {noformat}
> 2016-09-15 18:20:24,849 INFO mapreduce.Job: Task Id :
> attempt_1473943118844_0001_m_01_0, Status : FAILED
> Error: java.lang.RuntimeException: Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1806)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
> ... 9 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2330)
> at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:1108)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
> ... 14 more
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2298)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2322)
> ... 16 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2202)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2296)
> ... 17 more
> {noformat}
> *mapreduce.JobResourceUploader: No job jar file set. User classes may not be
> found. See Job or Job#setJar(String).*
> Job jar is not getting set and not uploaded to staging dir
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6779) Mapreduce job failure on submission
[
https://issues.apache.org/jira/browse/MAPREDUCE-6779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15494019#comment-15494019
]
Bibin A Chundatt commented on MAPREDUCE-6779:
-
We can close this jira once main jira is reverted
> Mapreduce job failure on submission
> ---
>
> Key: MAPREDUCE-6779
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6779
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
>
> Configure Hibench
> Try running Enhanced TestDFSIO
> {noformat}
> 2016-09-15 18:20:24,849 INFO mapreduce.Job: Task Id :
> attempt_1473943118844_0001_m_01_0, Status : FAILED
> Error: java.lang.RuntimeException: Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1806)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
> ... 9 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2330)
> at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:1108)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
> ... 14 more
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2298)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2322)
> ... 16 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2202)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2296)
> ... 17 more
> {noformat}
> *mapreduce.JobResourceUploader: No job jar file set. User classes may not be
> found. See Job or Job#setJar(String).*
> Job jar is not getting set and not uploaded to staging dir
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6779) Mapreduce job failure on submission
[
https://issues.apache.org/jira/browse/MAPREDUCE-6779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15494018#comment-15494018
]
Bibin A Chundatt commented on MAPREDUCE-6779:
-
We can close this jira once main jira is reverted
> Mapreduce job failure on submission
> ---
>
> Key: MAPREDUCE-6779
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6779
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
>
> Configure Hibench
> Try running Enhanced TestDFSIO
> {noformat}
> 2016-09-15 18:20:24,849 INFO mapreduce.Job: Task Id :
> attempt_1473943118844_0001_m_01_0, Status : FAILED
> Error: java.lang.RuntimeException: Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1806)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
> ... 9 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2330)
> at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:1108)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
> ... 14 more
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2298)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2322)
> ... 16 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2202)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2296)
> ... 17 more
> {noformat}
> *mapreduce.JobResourceUploader: No job jar file set. User classes may not be
> found. See Job or Job#setJar(String).*
> Job jar is not getting set and not uploaded to staging dir
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Assigned] (MAPREDUCE-6779) Mapreduce job failure on submission
[
https://issues.apache.org/jira/browse/MAPREDUCE-6779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt reassigned MAPREDUCE-6779:
---
Assignee: Bibin A Chundatt
> Mapreduce job failure on submission
> ---
>
> Key: MAPREDUCE-6779
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6779
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
>
> Configure Hibench
> Try running Enhanced TestDFSIO
> {noformat}
> 2016-09-15 18:20:24,849 INFO mapreduce.Job: Task Id :
> attempt_1473943118844_0001_m_01_0, Status : FAILED
> Error: java.lang.RuntimeException: Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1806)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
> ... 9 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2330)
> at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:1108)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
> ... 14 more
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2298)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2322)
> ... 16 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2202)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2296)
> ... 17 more
> {noformat}
> *mapreduce.JobResourceUploader: No job jar file set. User classes may not be
> found. See Job or Job#setJar(String).*
> Job jar is not getting set and not uploaded to staging dir
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6779) Mapreduce job failure on submission
[
https://issues.apache.org/jira/browse/MAPREDUCE-6779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15493913#comment-15493913
]
Bibin A Chundatt commented on MAPREDUCE-6779:
-
[~sjlee0]
>From ClassLoader file is removed as per new implementation.Since the jar is
>not available in applcation class path all map execution will fail.
{noformat}
- classPath.add(file.toURI().toURL());
{noformat}
When {{JobConf#setJarByClass}} is set tries to get the jar name from classloader
{noformat}
public void setJarByClass(Class cls) {
String jar = ClassUtil.findContainingJar(cls);
if (jar != null) {
setJar(jar);
}
}
{noformat}
{{ClassUtil#findContainingJar}}
{noformat}
for(final Enumeration itr = loader.getResources(classFile);
itr.hasMoreElements();) {
final URL url = itr.nextElement();
if ("jar".equals(url.getProtocol())) {
String toReturn = url.getPath();
if (toReturn.startsWith("file:")) {
toReturn = toReturn.substring("file:".length());
}
toReturn = URLDecoder.decode(toReturn, "UTF-8");
return toReturn.replaceAll("!.*$", "");
}
}
{noformat}
which returns null. The job jar will not get loaded to staging dir in
{{JobResourceUploader#uploadResources}} since jobjar is {{null}}.
{noformat}
private void uploadJobJar(Job job, String jobJar, Path submitJobDir,
short submitReplication) throws IOException {
if (jobJar != null) {
{noformat}
> Mapreduce job failure on submission
> ---
>
> Key: MAPREDUCE-6779
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6779
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Configure Hibench
> Try running Enhanced TestDFSIO
> {noformat}
> 2016-09-15 18:20:24,849 INFO mapreduce.Job: Task Id :
> attempt_1473943118844_0001_m_01_0, Status : FAILED
> Error: java.lang.RuntimeException: Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1806)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
> ... 9 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2330)
> at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:1108)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
> ... 14 more
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2298)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2322)
> ... 16 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2202)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2296)
> ... 17 more
> {noformat}
> *mapreduce.JobResourceUploader: No job jar file set. User classes may not be
> found. See Job or Job#setJar(String).*
> Job jar is not getting set and not uploaded to staging dir
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands,
[jira] [Commented] (MAPREDUCE-6779) Mapreduce job failure on submission
[
https://issues.apache.org/jira/browse/MAPREDUCE-6779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15493735#comment-15493735
]
Bibin A Chundatt commented on MAPREDUCE-6779:
-
[~sjlee0]
Thank you for looking in to issue. Sorry for not adding as part of defect
description.
We can reproduce the same with sleep job too.
# Move {{hadoop-mapreduce-client-jobclient-3.0.0-alpha2-SNAPSHOT-tests.jar}} to
{{bin}} directory.
# Submit application using below command.
{{./yarn jar hadoop-mapreduce-client-jobclient-3.0.0-alpha2-SNAPSHOT-tests.jar
sleep -m 1 -r 1 -mt 1 -rt 1}}
{noformat}
2016-09-15 21:00:18,922 INFO mapreduce.Job: Task Id :
attempt_1473952300123_0004_m_00_1, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.hadoop.mapreduce.SleepJob$SleepMapper not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2277)
at
org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:187)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:747)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1790)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
Caused by: java.lang.ClassNotFoundException: Class
org.apache.hadoop.mapreduce.SleepJob$SleepMapper not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2181)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2275)
... 8 more
{noformat}
> Mapreduce job failure on submission
> ---
>
> Key: MAPREDUCE-6779
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6779
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Configure Hibench
> Try running Enhanced TestDFSIO
> {noformat}
> 2016-09-15 18:20:24,849 INFO mapreduce.Job: Task Id :
> attempt_1473943118844_0001_m_01_0, Status : FAILED
> Error: java.lang.RuntimeException: Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1806)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
> ... 9 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2330)
> at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:1108)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
> ... 14 more
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2298)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2322)
> ... 16 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2202)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2296)
> ... 17 more
> {noformat}
> *mapreduce.JobResourceUploader: No job jar file set. User
[jira] [Commented] (MAPREDUCE-6779) Mapreduce job failure on submission
[
https://issues.apache.org/jira/browse/MAPREDUCE-6779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15493227#comment-15493227
]
Bibin A Chundatt commented on MAPREDUCE-6779:
-
Looks like impact of HADOOP-13410
> Mapreduce job failure on submission
> ---
>
> Key: MAPREDUCE-6779
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6779
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Configure Hibench
> Try running Enhanced TestDFSIO
> {noformat}
> 2016-09-15 18:20:24,849 INFO mapreduce.Job: Task Id :
> attempt_1473943118844_0001_m_01_0, Status : FAILED
> Error: java.lang.RuntimeException: Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1806)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
> ... 9 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2330)
> at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:1108)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
> ... 14 more
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2298)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2322)
> ... 16 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2202)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2296)
> ... 17 more
> {noformat}
> *mapreduce.JobResourceUploader: No job jar file set. User classes may not be
> found. See Job or Job#setJar(String).*
> Job jar is not getting set and not uploaded to staging dir
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6779) Mapreduce job failure on submission
Bibin A Chundatt created MAPREDUCE-6779:
---
Summary: Mapreduce job failure on submission
Key: MAPREDUCE-6779
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6779
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
Priority: Blocker
Configure Hibench
Try running Enhanced TestDFSIO
{noformat}
2016-09-15 18:20:24,849 INFO mapreduce.Job: Task Id :
attempt_1473943118844_0001_m_01_0, Status : FAILED
Error: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1806)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:169)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
... 9 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException: Class
org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2330)
at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:1108)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
... 14 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2298)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2322)
... 16 more
Caused by: java.lang.ClassNotFoundException: Class
org.apache.hadoop.fs.dfsioe.TestDFSIOEnh$WriteMapperEnh not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2202)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2296)
... 17 more
{noformat}
*mapreduce.JobResourceUploader: No job jar file set. User classes may not be
found. See Job or Job#setJar(String).*
Job jar is not getting set and not uploaded to staging dir
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410998#comment-15410998
]
Bibin A Chundatt edited comment on MAPREDUCE-6704 at 8/8/16 4:39 AM:
-
[~aw]/[~ajisakaa]
Sorry for the delay.
As already mentioned the {{HADOOP_MAPRED_HOME}} is not part of env since not
inherited to launcher.Adding
{code}
yarn.nodemanager.env-whitelist="JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME"
{code}
in yarn-site.xml of NM does solve the problem. But since its a mapreduce
property its not correct to add to the whitelist of yarn .
IIUU {{HADOOP_MAPRED_HOME}} for default installation will be same as
{{HADOOP_COMMON_HOME}} .
For layout change we can configure explicity mapred.child.env paths.
Till jira is fixed ,to make mapreduce job run add params as below
{code}
-Dmapred.child.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}"
-Dyarn.app.mapreduce.am.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}"
{code}
Open to any other solution.
was (Author: bibinchundatt):
[~aw]/[~ajisakaa]
Sorry for the delay.
As already mentioned the {{HADOOP_MAPRED_HOME}} is not part of env since not
inherited to launcher.Adding
{code}
yarn.nodemanager.env-whitelist="JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME"
{code}
in yarn-site.xml of NM does solve the problem. But since its a mapreduce
property its not correct to add to the whitelist of yarn .
IIUU {{HADOOP_MAPRED_HOME}} for default installation will be same as
{{HADOOP_COMMON_HOME}} .
For layout change we can configure explicity mapred.child.env paths.
Will jira is fixed ,to make mapreduce job run add params as below
{code}
-Dmapred.child.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}"
-Dyarn.app.mapreduce.am.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}"
{code}
Open to any other solution.
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15411208#comment-15411208
]
Bibin A Chundatt commented on MAPREDUCE-6704:
-
0001-YARN-5026.patch implementation is based on {{HADOOP_MAPRED_HOME}} added to
whitelist.
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15410998#comment-15410998
]
Bibin A Chundatt commented on MAPREDUCE-6704:
-
[~aw]/[~ajisakaa]
Sorry for the delay.
As already mentioned the {{HADOOP_MAPRED_HOME}} is not part of env since not
inherited to launcher.Adding
{code}
yarn.nodemanager.env-whitelist="JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME"
{code}
in yarn-site.xml of NM does solve the problem. But since its a mapreduce
property its not correct to add to the whitelist of yarn .
IIUU {{HADOOP_MAPRED_HOME}} for default installation will be same as
{{HADOOP_COMMON_HOME}} .
For layout change we can configure explicity mapred.child.env paths.
Will jira is fixed ,to make mapreduce job run add params as below
{code}
-Dmapred.child.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}"
-Dyarn.app.mapreduce.am.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}"
{code}
Open to any other solution.
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6704:
Attachment: 0001-MAPREDUCE-6704.patch
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6704:
Status: Patch Available (was: Open)
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6666) Support MultiThreads in a Map and Distribution of files in NNBench
[ https://issues.apache.org/jira/browse/MAPREDUCE-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15323831#comment-15323831 ] Bibin A Chundatt commented on MAPREDUCE-: - To Executors even if you submit all tasks at the same time it does not imply that they will be executed at the same time. Shouldnt cause too much problem in our case > Support MultiThreads in a Map and Distribution of files in NNBench > -- > > Key: MAPREDUCE- > URL: https://issues.apache.org/jira/browse/MAPREDUCE- > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula > Attachments: MAPREDUCE--01.patch, MAPREDUCE--02.patch, > MAPREDUCE--03.patch, MAPREDUCE--04.patch > > > Support Distribution of files to multiple directories generated by NNBench. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6705) Task failing continuously on trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15316402#comment-15316402
]
Bibin A Chundatt commented on MAPREDUCE-6705:
-
[~drankye]
Since MAPREDUCE-6706 is available to track the {{ProtocolRPCEngine}} change, we
can close this JIRA. Thank you for handling the revert.
> Task failing continuously on trunk
> --
>
> Key: MAPREDUCE-6705
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6705
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Kai Zheng
>Priority: Blocker
>
> Task attempt failing continuously. Submit any mapreduce application
> Run the job as below
> {code}
> ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-examples*.jar pi
> -Dyarn.app.mapreduce.am.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}"
> -Dmapreduce.admin.user.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}" 1 1
> {code}
> {noformat}
> 2016-05-27 11:28:27,148 DEBUG [main] org.apache.hadoop.ipc.Client: getting
> client out of cache: org.apache.hadoop.ipc.Client@291ae
> 2016-05-27 11:28:27,160 DEBUG [main] org.apache.hadoop.mapred.YarnChild: PID:
> 22305
> 2016-05-27 11:28:27,160 INFO [main] org.apache.hadoop.mapred.YarnChild:
> Sleeping for 0ms before retrying again. Got null now.
> 2016-05-27 11:28:27,161 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.reflect.UndeclaredThrowableException
> at com.sun.proxy.$Proxy10.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:136)
> Caused by: com.google.protobuf.ServiceException: Too many or few parameters
> for request. Method: [getTask], Expected: 2, Actual: 1
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> ... 2 more
> 2016-05-27 11:28:27,161 DEBUG [main] org.apache.hadoop.ipc.Client: stopping
> client from cache: org.apache.hadoop.ipc.Client@291ae
> 2016-05-27 11:28:27,161 DEBUG [main] org.apache.hadoop.ipc.Client: removing
> client from cache: org.apache.hadoop.ipc.Client@291ae
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (MAPREDUCE-6705) Task failing continuously on trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304790#comment-15304790
]
Bibin A Chundatt edited comment on MAPREDUCE-6705 at 5/27/16 9:14 PM:
--
{{WritableRpcEngine}} is removed as part of HADOOP-12579 the same is causing
this to fail.The below check is not available as part of {{WritableRpcEngine}} .
{code}
if (args.length != 2) { // RpcController + Message
throw new ServiceException("Too many parameters for request. Method: ["
+ method.getName() + "]" + ", Expected: 2, Actual: "
+ args.length);
}
if (args[1] == null) {
throw new ServiceException("null param while calling Method: ["
+ method.getName() + "]");
}
{code}
was (Author: bibinchundatt):
{{WritableRpcEngine}}s removed as part of HADOOP-12579 the same is causing this
to fail.The below check is not available as part of {{WritableRpcEngine}} .
{code}
if (args.length != 2) { // RpcController + Message
throw new ServiceException("Too many parameters for request. Method: ["
+ method.getName() + "]" + ", Expected: 2, Actual: "
+ args.length);
}
if (args[1] == null) {
throw new ServiceException("null param while calling Method: ["
+ method.getName() + "]");
}
{code}
> Task failing continuously on trunk
> --
>
> Key: MAPREDUCE-6705
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6705
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Task attempt failing continuously. Submit any mapreduce application
> Run the job as below
> {code}
> ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-examples*.jar pi
> -Dyarn.app.mapreduce.am.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}"
> -Dmapreduce.admin.user.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}" 1 1
> {code}
> {noformat}
> 2016-05-27 11:28:27,148 DEBUG [main] org.apache.hadoop.ipc.Client: getting
> client out of cache: org.apache.hadoop.ipc.Client@291ae
> 2016-05-27 11:28:27,160 DEBUG [main] org.apache.hadoop.mapred.YarnChild: PID:
> 22305
> 2016-05-27 11:28:27,160 INFO [main] org.apache.hadoop.mapred.YarnChild:
> Sleeping for 0ms before retrying again. Got null now.
> 2016-05-27 11:28:27,161 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.reflect.UndeclaredThrowableException
> at com.sun.proxy.$Proxy10.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:136)
> Caused by: com.google.protobuf.ServiceException: Too many or few parameters
> for request. Method: [getTask], Expected: 2, Actual: 1
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> ... 2 more
> 2016-05-27 11:28:27,161 DEBUG [main] org.apache.hadoop.ipc.Client: stopping
> client from cache: org.apache.hadoop.ipc.Client@291ae
> 2016-05-27 11:28:27,161 DEBUG [main] org.apache.hadoop.ipc.Client: removing
> client from cache: org.apache.hadoop.ipc.Client@291ae
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6705) Task failing continuously on trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304790#comment-15304790
]
Bibin A Chundatt commented on MAPREDUCE-6705:
-
{{WritableRpcEngine}}s removed as part of HADOOP-12579 the same is causing this
to fail.The below check is not available as part of {{WritableRpcEngine}} .
{code}
if (args.length != 2) { // RpcController + Message
throw new ServiceException("Too many parameters for request. Method: ["
+ method.getName() + "]" + ", Expected: 2, Actual: "
+ args.length);
}
if (args[1] == null) {
throw new ServiceException("null param while calling Method: ["
+ method.getName() + "]");
}
{code}
> Task failing continuously on trunk
> --
>
> Key: MAPREDUCE-6705
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6705
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Task attempt failing continuously. Submit any mapreduce application
> Run the job as below
> {code}
> ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-examples*.jar pi
> -Dyarn.app.mapreduce.am.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}"
> -Dmapreduce.admin.user.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}" 1 1
> {code}
> {noformat}
> 2016-05-27 11:28:27,148 DEBUG [main] org.apache.hadoop.ipc.Client: getting
> client out of cache: org.apache.hadoop.ipc.Client@291ae
> 2016-05-27 11:28:27,160 DEBUG [main] org.apache.hadoop.mapred.YarnChild: PID:
> 22305
> 2016-05-27 11:28:27,160 INFO [main] org.apache.hadoop.mapred.YarnChild:
> Sleeping for 0ms before retrying again. Got null now.
> 2016-05-27 11:28:27,161 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.reflect.UndeclaredThrowableException
> at com.sun.proxy.$Proxy10.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:136)
> Caused by: com.google.protobuf.ServiceException: Too many or few parameters
> for request. Method: [getTask], Expected: 2, Actual: 1
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> ... 2 more
> 2016-05-27 11:28:27,161 DEBUG [main] org.apache.hadoop.ipc.Client: stopping
> client from cache: org.apache.hadoop.ipc.Client@291ae
> 2016-05-27 11:28:27,161 DEBUG [main] org.apache.hadoop.ipc.Client: removing
> client from cache: org.apache.hadoop.ipc.Client@291ae
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6705) Task failing continuously on trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304282#comment-15304282
]
Bibin A Chundatt commented on MAPREDUCE-6705:
-
[~cnauroth]
Using trunk build with secure. Both client and server are of same version. Can
HADOOP-12579 cause this?
> Task failing continuously on trunk
> --
>
> Key: MAPREDUCE-6705
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6705
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Task attempt failing continuously. Submit any mapreduce application
> Run the job as below
> {code}
> ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-examples*.jar pi
> -Dyarn.app.mapreduce.am.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}"
> -Dmapreduce.admin.user.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}" 1 1
> {code}
> {noformat}
> 2016-05-27 11:28:27,148 DEBUG [main] org.apache.hadoop.ipc.Client: getting
> client out of cache: org.apache.hadoop.ipc.Client@291ae
> 2016-05-27 11:28:27,160 DEBUG [main] org.apache.hadoop.mapred.YarnChild: PID:
> 22305
> 2016-05-27 11:28:27,160 INFO [main] org.apache.hadoop.mapred.YarnChild:
> Sleeping for 0ms before retrying again. Got null now.
> 2016-05-27 11:28:27,161 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.reflect.UndeclaredThrowableException
> at com.sun.proxy.$Proxy10.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:136)
> Caused by: com.google.protobuf.ServiceException: Too many or few parameters
> for request. Method: [getTask], Expected: 2, Actual: 1
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> ... 2 more
> 2016-05-27 11:28:27,161 DEBUG [main] org.apache.hadoop.ipc.Client: stopping
> client from cache: org.apache.hadoop.ipc.Client@291ae
> 2016-05-27 11:28:27,161 DEBUG [main] org.apache.hadoop.ipc.Client: removing
> client from cache: org.apache.hadoop.ipc.Client@291ae
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6705) Task failing continuously on trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15304200#comment-15304200
]
Bibin A Chundatt commented on MAPREDUCE-6705:
-
Verified distributed shell its working fine ..
{code}
./hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar
../share/hadoop/yarn/hadoop-yarn-applications-distributedshell-*.jar
-shell_command ls -container_memory 1024 -master_memory 1024
{code}
Only for mapreduce application seems too have problem.
> Task failing continuously on trunk
> --
>
> Key: MAPREDUCE-6705
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6705
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Task attempt failing continuously. Submit any mapreduce application
> Run the job as below
> {code}
> ./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-examples*.jar pi
> -Dyarn.app.mapreduce.am.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}"
> -Dmapreduce.admin.user.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}" 1 1
> {code}
> {noformat}
> 2016-05-27 11:28:27,148 DEBUG [main] org.apache.hadoop.ipc.Client: getting
> client out of cache: org.apache.hadoop.ipc.Client@291ae
> 2016-05-27 11:28:27,160 DEBUG [main] org.apache.hadoop.mapred.YarnChild: PID:
> 22305
> 2016-05-27 11:28:27,160 INFO [main] org.apache.hadoop.mapred.YarnChild:
> Sleeping for 0ms before retrying again. Got null now.
> 2016-05-27 11:28:27,161 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.reflect.UndeclaredThrowableException
> at com.sun.proxy.$Proxy10.getTask(Unknown Source)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:136)
> Caused by: com.google.protobuf.ServiceException: Too many or few parameters
> for request. Method: [getTask], Expected: 2, Actual: 1
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> ... 2 more
> 2016-05-27 11:28:27,161 DEBUG [main] org.apache.hadoop.ipc.Client: stopping
> client from cache: org.apache.hadoop.ipc.Client@291ae
> 2016-05-27 11:28:27,161 DEBUG [main] org.apache.hadoop.ipc.Client: removing
> client from cache: org.apache.hadoop.ipc.Client@291ae
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6705) Task failing continuously on trunk
Bibin A Chundatt created MAPREDUCE-6705:
---
Summary: Task failing continuously on trunk
Key: MAPREDUCE-6705
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6705
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
Priority: Blocker
Task attempt failing continuously. Submit any mapreduce application
Run the job as below
{code}
./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-examples*.jar pi
-Dyarn.app.mapreduce.am.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}"
-Dmapreduce.admin.user.env="HADOOP_MAPRED_HOME={{HADOOP_COMMON_HOME}}" 1 1
{code}
{noformat}
2016-05-27 11:28:27,148 DEBUG [main] org.apache.hadoop.ipc.Client: getting
client out of cache: org.apache.hadoop.ipc.Client@291ae
2016-05-27 11:28:27,160 DEBUG [main] org.apache.hadoop.mapred.YarnChild: PID:
22305
2016-05-27 11:28:27,160 INFO [main] org.apache.hadoop.mapred.YarnChild:
Sleeping for 0ms before retrying again. Got null now.
2016-05-27 11:28:27,161 WARN [main] org.apache.hadoop.mapred.YarnChild:
Exception running child : java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy10.getTask(Unknown Source)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:136)
Caused by: com.google.protobuf.ServiceException: Too many or few parameters for
request. Method: [getTask], Expected: 2, Actual: 1
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
... 2 more
2016-05-27 11:28:27,161 DEBUG [main] org.apache.hadoop.ipc.Client: stopping
client from cache: org.apache.hadoop.ipc.Client@291ae
2016-05-27 11:28:27,161 DEBUG [main] org.apache.hadoop.ipc.Client: removing
client from cache: org.apache.hadoop.ipc.Client@291ae
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Moved] (MAPREDUCE-6704) Container fail to launch for mapred application
[
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt moved YARN-5026 to MAPREDUCE-6704:
---
Key: MAPREDUCE-6704 (was: YARN-5026)
Project: Hadoop Map/Reduce (was: Hadoop YARN)
> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-YARN-5026.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set
> .After
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}}
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with
> state FAILED due to: Application application_1462155939310_0004 failed 2
> times due to AM Container for appattempt_1462155939310_0004_02 exited
> with exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> Error: Could not find or load main class
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier;
> support was removed in 8.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6666) Support MultiThreads in a Map and Distribution of files in NNBench
[ https://issues.apache.org/jira/browse/MAPREDUCE-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290576#comment-15290576 ] Bibin A Chundatt commented on MAPREDUCE-: - [~brahmareddy] Could you check all threads are starting are same time?IIUC the barrier case also we need to handle. > Support MultiThreads in a Map and Distribution of files in NNBench > -- > > Key: MAPREDUCE- > URL: https://issues.apache.org/jira/browse/MAPREDUCE- > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula > Attachments: MAPREDUCE--01.patch, MAPREDUCE--02.patch > > > Support Distribution of files to multiple directories generated by NNBench. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce
[
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15288627#comment-15288627
]
Bibin A Chundatt commented on MAPREDUCE-6700:
-
[~rohithsharma]
{quote}
Bibin A Chundatt would you mind providing rebase patch to branch-2.8 please?
{quote}
Offline have asked Varun to upload don't have rights for to upload in
YARN-3840.
> Jobhistory server attempt and task table not loading maps/reduce
>
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc'
> jquery.dataTables.min.js, line 86 character 179
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Assigned] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce
[
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt reassigned MAPREDUCE-6700:
---
Assignee: Bibin A Chundatt
> Jobhistory server attempt and task table not loading maps/reduce
>
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc'
> jquery.dataTables.min.js, line 86 character 179
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce
[
https://issues.apache.org/jira/browse/MAPREDUCE-6700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6700:
Description:
Browser
===
Chrome
Steps to reproduce
==
# Submit mapreduce application with 20 maps
# Wait till completion of mapreduce application
# Check maps attempts page
{{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
and {{jobhistory/tasks/job_1463446678437_0003/m}} page
Actual
=
Table not loading.
Sort based on any column other than attempt contents are loaded.
Column 0 is of *natural sorting* and not working. So waiting for ever to be
sorted.
{noformat}
SCRIPT438: Object doesn't support property or method 'natural-asc'
jquery.dataTables.min.js, line 86 character 179
{noformat}
was:
Browser
===
Chrome
Steps to reproduce
==
# Submit mapreduce application with 20 maps
# Wait till completion of mapreduce application
# Check maps attempts page
{{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
and {{jobhistory/tasks/job_1463446678437_0003/m}} page
Actual
=
Table not loading.
Sort based on any column other than attempt contents are loaded.
Column 0 is of *natural sorting* and not working. So waiting for ever to be
sorted.
> Jobhistory server attempt and task table not loading maps/reduce
>
>
> Key: MAPREDUCE-6700
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Priority: Blocker
>
> Browser
> ===
> Chrome
> Steps to reproduce
> ==
> # Submit mapreduce application with 20 maps
> # Wait till completion of mapreduce application
> # Check maps attempts page
> {{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
> and {{jobhistory/tasks/job_1463446678437_0003/m}} page
> Actual
> =
> Table not loading.
> Sort based on any column other than attempt contents are loaded.
> Column 0 is of *natural sorting* and not working. So waiting for ever to be
> sorted.
> {noformat}
> SCRIPT438: Object doesn't support property or method 'natural-asc'
> jquery.dataTables.min.js, line 86 character 179
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6700) Jobhistory server attempt and task table not loading maps/reduce
Bibin A Chundatt created MAPREDUCE-6700:
---
Summary: Jobhistory server attempt and task table not loading
maps/reduce
Key: MAPREDUCE-6700
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6700
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
Priority: Blocker
Browser
===
Chrome
Steps to reproduce
==
# Submit mapreduce application with 20 maps
# Wait till completion of mapreduce application
# Check maps attempts page
{{jobhistory/attempts/job_1463446678437_0003/m/SUCCESSFUL}}
and {{jobhistory/tasks/job_1463446678437_0003/m}} page
Actual
=
Table not loading.
Sort based on any column other than attempt contents are loaded.
Column 0 is of *natural sorting* and not working. So waiting for ever to be
sorted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Resolved] (MAPREDUCE-6466) NNBench map task should create files in separate folder
[
https://issues.apache.org/jira/browse/MAPREDUCE-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt resolved MAPREDUCE-6466.
-
Resolution: Duplicate
MAPREDUCE- handling the same.
> NNBench map task should create files in separate folder
> ---
>
> Key: MAPREDUCE-6466
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6466
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>
> In case of NNBench with current implementation all files created using
> different maps are created in same directory.
> In this JIRA will provide a flag to create files in same folder or separate
> folder for each task.
> {{dfs.namenode.fs-limits.max-directory-items}} cannot be set to more than
> 6,400,000 as i understand
> Advantage
> # For scenarios like creating fsimage with large size NNBench can be used
> # Creation of multiple files of same size using NNBench folder based.
> No need to run NNBench multiple time to create FSImage of large size.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6693) Job history entry missing when JOB name is of mapreduce.jobhistory.jobname.limit length
[
https://issues.apache.org/jira/browse/MAPREDUCE-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278249#comment-15278249
]
Bibin A Chundatt commented on MAPREDUCE-6693:
-
[~ajithshetty]
Fix looks good to me too after [~ajisakaa] comments handled. Could you upload a
patch with testcase.
> Job history entry missing when JOB name is of
> mapreduce.jobhistory.jobname.limit length
> ---
>
> Key: MAPREDUCE-6693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Ajith S
>Priority: Critical
>
> Job history entry missing when JOB name is of
> {{mapreduce.jobhistory.jobname.limit}} character
> {noformat}
> 2016-05-10 06:51:00,674 DEBUG [Thread-73]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Interrupting
> Event Handling thread
> 2016-05-10 06:51:00,674 DEBUG [Thread-73]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Waiting for
> Event Handling thread to complete
> 2016-05-10 06:51:00,674 ERROR [eventHandlingThread]
> org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread
> Thread[eventHandlingThread,5,main] threw an Exception.
> java.lang.ArrayIndexOutOfBoundsException: 50
> at
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.trimURLEncodedString(FileNameIndexUtils.java:326)
> at
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:86)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:1147)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:635)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:341)
> at java.lang.Thread.run(Thread.java:745)
> 2016-05-10 06:51:00,675 DEBUG [Thread-73]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down
> timer for Job MetaInfo for job_1462840033869_0009 history file
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,675 DEBUG [Thread-73]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down
> timer Job MetaInfo for job_1462840033869_0009 history file
> hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
> 2016-05-10 06:51:00,676 DEBUG [Thread-73]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Closing Writer
> {noformat}
> Looks like 50 character check is going wrong
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6693) Job history entry missing when JOB name is of mapreduce.jobhistory.jobname.limit length
[
https://issues.apache.org/jira/browse/MAPREDUCE-6693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6693:
Description:
Job history entry missing when JOB name is of
{{mapreduce.jobhistory.jobname.limit}} character
{noformat}
2016-05-10 06:51:00,674 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Interrupting
Event Handling thread
2016-05-10 06:51:00,674 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Waiting for
Event Handling thread to complete
2016-05-10 06:51:00,674 ERROR [eventHandlingThread]
org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread
Thread[eventHandlingThread,5,main] threw an Exception.
java.lang.ArrayIndexOutOfBoundsException: 50
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.trimURLEncodedString(FileNameIndexUtils.java:326)
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:86)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:1147)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:635)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:341)
at java.lang.Thread.run(Thread.java:745)
2016-05-10 06:51:00,675 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down
timer for Job MetaInfo for job_1462840033869_0009 history file
hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
2016-05-10 06:51:00,675 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down
timer Job MetaInfo for job_1462840033869_0009 history file
hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
2016-05-10 06:51:00,676 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Closing Writer
{noformat}
Looks like 50 character check is going wrong
was:
Job entry history missing when JOB name is of 50 character
{noformat}
2016-05-10 06:51:00,674 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Interrupting
Event Handling thread
2016-05-10 06:51:00,674 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Waiting for
Event Handling thread to complete
2016-05-10 06:51:00,674 ERROR [eventHandlingThread]
org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread
Thread[eventHandlingThread,5,main] threw an Exception.
java.lang.ArrayIndexOutOfBoundsException: 50
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.trimURLEncodedString(FileNameIndexUtils.java:326)
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:86)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:1147)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:635)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:341)
at java.lang.Thread.run(Thread.java:745)
2016-05-10 06:51:00,675 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down
timer for Job MetaInfo for job_1462840033869_0009 history file
hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
2016-05-10 06:51:00,675 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down
timer Job MetaInfo for job_1462840033869_0009 history file
hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
2016-05-10 06:51:00,676 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Closing Writer
{noformat}
Looks like 50 character check is going wrong
Summary: Job history entry missing when JOB name is of
mapreduce.jobhistory.jobname.limit length (was: Job history entry missing when
JOB name is of 50 character)
> Job history entry missing when JOB name is of
> mapreduce.jobhistory.jobname.limit length
> ---
>
> Key: MAPREDUCE-6693
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6693
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Ajith S
>Priority: Critical
>
> Job history entry missing when JOB name is of
> {{mapreduce.jobhistory.jobname.limit}} character
> {noformat}
>
[jira] [Created] (MAPREDUCE-6693) Job history entry missing when JOB name is of 50 character
Bibin A Chundatt created MAPREDUCE-6693:
---
Summary: Job history entry missing when JOB name is of 50 character
Key: MAPREDUCE-6693
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6693
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
Priority: Critical
Job entry history missing when JOB name is of 50 character
{noformat}
2016-05-10 06:51:00,674 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Interrupting
Event Handling thread
2016-05-10 06:51:00,674 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Waiting for
Event Handling thread to complete
2016-05-10 06:51:00,674 ERROR [eventHandlingThread]
org.apache.hadoop.yarn.YarnUncaughtExceptionHandler: Thread
Thread[eventHandlingThread,5,main] threw an Exception.
java.lang.ArrayIndexOutOfBoundsException: 50
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.trimURLEncodedString(FileNameIndexUtils.java:326)
at
org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils.getDoneFileName(FileNameIndexUtils.java:86)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.processDoneFiles(JobHistoryEventHandler.java:1147)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:635)
at
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$1.run(JobHistoryEventHandler.java:341)
at java.lang.Thread.run(Thread.java:745)
2016-05-10 06:51:00,675 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down
timer for Job MetaInfo for job_1462840033869_0009 history file
hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
2016-05-10 06:51:00,675 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Shutting down
timer Job MetaInfo for job_1462840033869_0009 history file
hdfs://hacluster:9820/staging-dir/dsperf/.staging/job_1462840033869_0009/job_1462840033869_0009_1.jhist
2016-05-10 06:51:00,676 DEBUG [Thread-73]
org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Closing Writer
{noformat}
Looks like 50 character check is going wrong
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6554) MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory
[
https://issues.apache.org/jira/browse/MAPREDUCE-6554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6554:
Attachment: MAPREDUCE-6554-branch-2.7.patch
[~jlowe]
Thanks you for review . Attaching patch for branch-2.7
> MRAppMaster servicestart failing with NPE in
> MRAppMaster#parsePreviousJobHistory
> -
>
> Key: MAPREDUCE-6554
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6554
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: 0001-MAPREDUCE-6554.patch, 0002-MAPREDUCE-6554.patch,
> 0003-MAPREDUCE-6554.patch, MAPREDUCE-6554-branch-2.7.patch
>
>
> Create scenario so that MR app master gets preempted.
> On next MRAppMaster launch tried to recover previous job history file
> {{MRAppMaster#parsePreviousJobHistory}}
> {noformat}
> 2015-11-21 13:52:27,722 INFO [main]
> org.apache.hadoop.service.AbstractService: Service
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state STARTED;
> cause: java.lang.NullPointerException
> java.lang.NullPointerException
> at java.io.StringReader.(StringReader.java:50)
> at org.apache.avro.Schema$Parser.parse(Schema.java:917)
> at org.apache.avro.Schema.parse(Schema.java:966)
> at
> org.apache.hadoop.mapreduce.jobhistory.EventReader.(EventReader.java:75)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser.parse(JobHistoryParser.java:139)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.parsePreviousJobHistory(MRAppMaster.java:1256)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.processRecovery(MRAppMaster.java:1225)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1087)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1570)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1673)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1566)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1499)
> 2015-11-21 13:52:27,725 INFO [main]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping
> JobHistoryEventHandler. Size of the outstanding queue size is 0
> {noformat}
> EventReader(EventReader stream)
> {noformat}
> this.version = in.readLine();
> ...
> Schema myschema = new
> SpecificData(Event.class.getClassLoader()).getSchema(Event.class);
> this.schema = Schema.parse(in.readLine());
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6554) MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory
[
https://issues.apache.org/jira/browse/MAPREDUCE-6554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6554:
Attachment: MAPREDUCE-6554-branch-2.7.001.patch
> MRAppMaster servicestart failing with NPE in
> MRAppMaster#parsePreviousJobHistory
> -
>
> Key: MAPREDUCE-6554
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6554
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: 0001-MAPREDUCE-6554.patch, 0002-MAPREDUCE-6554.patch,
> 0003-MAPREDUCE-6554.patch, MAPREDUCE-6554-branch-2.7.001.patch,
> MAPREDUCE-6554-branch-2.7.patch
>
>
> Create scenario so that MR app master gets preempted.
> On next MRAppMaster launch tried to recover previous job history file
> {{MRAppMaster#parsePreviousJobHistory}}
> {noformat}
> 2015-11-21 13:52:27,722 INFO [main]
> org.apache.hadoop.service.AbstractService: Service
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state STARTED;
> cause: java.lang.NullPointerException
> java.lang.NullPointerException
> at java.io.StringReader.(StringReader.java:50)
> at org.apache.avro.Schema$Parser.parse(Schema.java:917)
> at org.apache.avro.Schema.parse(Schema.java:966)
> at
> org.apache.hadoop.mapreduce.jobhistory.EventReader.(EventReader.java:75)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser.parse(JobHistoryParser.java:139)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.parsePreviousJobHistory(MRAppMaster.java:1256)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.processRecovery(MRAppMaster.java:1225)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1087)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1570)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1673)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1566)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1499)
> 2015-11-21 13:52:27,725 INFO [main]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping
> JobHistoryEventHandler. Size of the outstanding queue size is 0
> {noformat}
> EventReader(EventReader stream)
> {noformat}
> this.version = in.readLine();
> ...
> Schema myschema = new
> SpecificData(Event.class.getClassLoader()).getSchema(Event.class);
> this.schema = Schema.parse(in.readLine());
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6554) MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory
[
https://issues.apache.org/jira/browse/MAPREDUCE-6554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15096245#comment-15096245
]
Bibin A Chundatt commented on MAPREDUCE-6554:
-
[~jlowe]
Testcase/Findbug failures are not related to patch attached. Could you please
review patch attached.
> MRAppMaster servicestart failing with NPE in
> MRAppMaster#parsePreviousJobHistory
> -
>
> Key: MAPREDUCE-6554
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6554
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: 0001-MAPREDUCE-6554.patch, 0002-MAPREDUCE-6554.patch,
> 0003-MAPREDUCE-6554.patch
>
>
> Create scenario so that MR app master gets preempted.
> On next MRAppMaster launch tried to recover previous job history file
> {{MRAppMaster#parsePreviousJobHistory}}
> {noformat}
> 2015-11-21 13:52:27,722 INFO [main]
> org.apache.hadoop.service.AbstractService: Service
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state STARTED;
> cause: java.lang.NullPointerException
> java.lang.NullPointerException
> at java.io.StringReader.(StringReader.java:50)
> at org.apache.avro.Schema$Parser.parse(Schema.java:917)
> at org.apache.avro.Schema.parse(Schema.java:966)
> at
> org.apache.hadoop.mapreduce.jobhistory.EventReader.(EventReader.java:75)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser.parse(JobHistoryParser.java:139)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.parsePreviousJobHistory(MRAppMaster.java:1256)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.processRecovery(MRAppMaster.java:1225)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1087)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1570)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1673)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1566)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1499)
> 2015-11-21 13:52:27,725 INFO [main]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping
> JobHistoryEventHandler. Size of the outstanding queue size is 0
> {noformat}
> EventReader(EventReader stream)
> {noformat}
> this.version = in.readLine();
> ...
> Schema myschema = new
> SpecificData(Event.class.getClassLoader()).getSchema(Event.class);
> this.schema = Schema.parse(in.readLine());
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers
[ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15096564#comment-15096564 ] Bibin A Chundatt commented on MAPREDUCE-6363: - [~ajisakaa] Testcase failures are not related to patch attached. Could you please help in reviewing patch. > [NNBench] Lease mismatch error when running with multiple mappers > - > > Key: MAPREDUCE-6363 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: benchmarks >Reporter: Brahma Reddy Battula >Assignee: Vlad Sharanhovich >Priority: Critical > Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, > MAPREDUCE-6363-002.patch, MAPREDUCE-6363-003.patch, MAPREDUCE-6363-004.patch, > MAPREDUCE-6363-005.patch, MAPREDUCE-6363-006.patch, MAPREDUCE-6363-007.patch, > MAPREDUCE-6363-008.patch, nnbench.log > > > Command : > ./yarn jar > ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar > nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 > -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` > -replicationFactorPerFile 3 -maps 100 -reduces 10 > Trace : > 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler > 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from > 192.168.105.214:36320: error: > org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch > on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by > DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed > by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 > org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch > on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by > DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed > by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6606) Findbug issue in org.apache.hadoop.mapred.OutputCommitter
Bibin A Chundatt created MAPREDUCE-6606:
---
Summary: Findbug issue in org.apache.hadoop.mapred.OutputCommitter
Key: MAPREDUCE-6606
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6606
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Bibin A Chundatt
{noformat}
org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext)
doesn't override method in superclass because parameter type
org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type
org.apache.hadoop.mapreduce.JobContext
Bug type NM_WRONG_PACKAGE_INTENTIONAL (click for details)
In class org.apache.hadoop.mapred.OutputCommitter
In method
org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext)
superclass is org.apache.hadoop.mapreduce.OutputCommitter
Did you intend to override
org.apache.hadoop.mapreduce.OutputCommitter.isCommitJobRepeatable(JobContext)
Actual type org.apache.hadoop.mapred.JobContext
Expected org.apache.hadoop.mapreduce.JobContext
Overrides
org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext)
At OutputCommitter.java:[line 230]
{noformat}
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6268/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html#Warnings_BAD_PRACTICE
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6554) MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory
[
https://issues.apache.org/jira/browse/MAPREDUCE-6554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bibin A Chundatt updated MAPREDUCE-6554:
Attachment: 0003-MAPREDUCE-6554.patch
[~jlowe]
Thank you for reviewing patch.Sorry to miss the second issue i had mentioned.
Attached patch after handling both the cases
> MRAppMaster servicestart failing with NPE in
> MRAppMaster#parsePreviousJobHistory
> -
>
> Key: MAPREDUCE-6554
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6554
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: 0001-MAPREDUCE-6554.patch, 0002-MAPREDUCE-6554.patch,
> 0003-MAPREDUCE-6554.patch
>
>
> Create scenario so that MR app master gets preempted.
> On next MRAppMaster launch tried to recover previous job history file
> {{MRAppMaster#parsePreviousJobHistory}}
> {noformat}
> 2015-11-21 13:52:27,722 INFO [main]
> org.apache.hadoop.service.AbstractService: Service
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state STARTED;
> cause: java.lang.NullPointerException
> java.lang.NullPointerException
> at java.io.StringReader.(StringReader.java:50)
> at org.apache.avro.Schema$Parser.parse(Schema.java:917)
> at org.apache.avro.Schema.parse(Schema.java:966)
> at
> org.apache.hadoop.mapreduce.jobhistory.EventReader.(EventReader.java:75)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser.parse(JobHistoryParser.java:139)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.parsePreviousJobHistory(MRAppMaster.java:1256)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.processRecovery(MRAppMaster.java:1225)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1087)
> at
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1570)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1673)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1566)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1499)
> 2015-11-21 13:52:27,725 INFO [main]
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping
> JobHistoryEventHandler. Size of the outstanding queue size is 0
> {noformat}
> EventReader(EventReader stream)
> {noformat}
> this.version = in.readLine();
> ...
> Schema myschema = new
> SpecificData(Event.class.getClassLoader()).getSchema(Event.class);
> this.schema = Schema.parse(in.readLine());
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6602) ApplicationResourceUsageReport should return ALL reserved resource
Bibin A Chundatt created MAPREDUCE-6602: --- Summary: ApplicationResourceUsageReport should return ALL reserved resource Key: MAPREDUCE-6602 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6602 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Bibin A Chundatt Assignee: Bibin A Chundatt ApplicationResourceUsageReport reserved resource report is only of default parition should be of all partitions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers
[ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bibin A Chundatt updated MAPREDUCE-6363: Attachment: MAPREDUCE-6363-008.patch [~ajisakaa] Attaching patch after formatting. Thanks for the review > [NNBench] Lease mismatch error when running with multiple mappers > - > > Key: MAPREDUCE-6363 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: benchmarks >Reporter: Brahma Reddy Battula >Assignee: Vlad Sharanhovich >Priority: Critical > Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, > MAPREDUCE-6363-002.patch, MAPREDUCE-6363-003.patch, MAPREDUCE-6363-004.patch, > MAPREDUCE-6363-005.patch, MAPREDUCE-6363-006.patch, MAPREDUCE-6363-007.patch, > MAPREDUCE-6363-008.patch, nnbench.log > > > Command : > ./yarn jar > ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar > nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 > -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` > -replicationFactorPerFile 3 -maps 100 -reduces 10 > Trace : > 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler > 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from > 192.168.105.214:36320: error: > org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch > on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by > DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed > by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 > org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch > on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by > DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed > by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1 > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710) > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
