[jira] [Created] (MAPREDUCE-4647) We should only unjar jobjar if there is a lib directory in it.

2012-09-10 Thread Robert Joseph Evans (JIRA) 
Robert Joseph Evans created MAPREDUCE-4647:
--

 Summary: We should only unjar jobjar if there is a lib directory 
in it.
 Key: MAPREDUCE-4647
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4647
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: mrv2
Affects Versions: 0.23.3
Reporter: Robert Joseph Evans
Assignee: Robert Joseph Evans


For backwards compatibility we recently added made is so we would unjar the 
job.jar and add anything to the classpath in the lib directory of that jar.  
But this also slows job startup down a lot if the jar is large.  We should only 
unjar it if actually doing so would add something new to the classpath.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (MAPREDUCE-4564) Shell timeout mechanism does not work for processes spawned using winutils

2012-09-10 Thread Arun C Murthy (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-4564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arun C Murthy resolved MAPREDUCE-4564.
--

   Resolution: Fixed
Fix Version/s: 1-win

I just committed this. Thanks Bikas (and Chuan for the review).

 Shell timeout mechanism does not work for processes spawned using winutils
 --

 Key: MAPREDUCE-4564
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4564
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Bikas Saha
Assignee: Bikas Saha
 Fix For: 1-win

 Attachments: MAPREDUCE-4564.branch-1-win.1.patch, 
 MAPREDUCE-4564.branch-1-win.2.patch


 Upon timeout, Shell calls Java process.destroy() to terminate the spawned 
 process. This would destroy the winutils process but not the real process 
 spawned by winutils.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (MAPREDUCE-4648) Diagnostics from AM are missing from job history

2012-09-10 Thread Jason Lowe (JIRA) 
Jason Lowe created MAPREDUCE-4648:
-

 Summary: Diagnostics from AM are missing from job history
 Key: MAPREDUCE-4648
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4648
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2
Affects Versions: 2.0.0-alpha, 0.23.0
Reporter: Jason Lowe


When a job fails during setup or commit, any diagnostics from the MapReduce 
ApplicationMaster are not available in the job history.  Currently the 
diagnostics for the job are collected from the diagnostics of tasks run for the 
job, but the AM has no corresponding task record in the job history.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (MAPREDUCE-4338) NodeManager daemon is failing to start.

2012-09-10 Thread Vinod Kumar Vavilapalli (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kumar Vavilapalli resolved MAPREDUCE-4338.


Resolution: Not A Problem

Please check connectivity from your NM machine to RM. You can login into the NM 
node and do a telnet to the RM host at port 8025. Closing this as 
not-a-problem. Create any newer issues in NM at 
https://issues.apache.org/jira/browse/YARN. Tx.

 NodeManager daemon is failing to start.
 ---

 Key: MAPREDUCE-4338
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4338
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: nodemanager
Affects Versions: 0.23.0
 Environment: Ubuntu Server 11.04, 
Reporter: srikanth ayalasomayajulu
  Labels: features, hadoop
 Fix For: 0.23.0

   Original Estimate: 4h
  Remaining Estimate: 4h

 Node manager daemons is not getting started on the slave machines. and giving 
 an error like stated below.
 2012-06-12 19:05:56,172 FATAL nodemanager.NodeManager 
 (NodeManager.java:main(233)) - Error starting NodeManager
 org.apache.hadoop.yarn.YarnException: Failed to Start 
 org.apache.hadoop.yarn.server.nodemanager.NodeManager
 at 
 org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:78)
 at 
 org.apache.hadoop.yarn.server.nodemanager.NodeManager.start(NodeManager.java:163)
 at 
 org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:231)
 Caused by: org.apache.avro.AvroRuntimeException: 
 java.lang.reflect.UndeclaredThrowableException
 at 
 org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:132)
 at 
 org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
 ... 2 more
 Caused by: java.lang.reflect.UndeclaredThrowableException
 at 
 org.apache.hadoop.yarn.server.api.impl.pb.client.ResourceTrackerPBClientImpl.registerNodeManager(ResourceTrackerPBClientImpl.java:66)
 at 
 org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:161)
 at 
 org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:128)
 ... 3 more
 Caused by: com.google.protobuf.ServiceException: java.net.ConnectException: 
 Call From mvm5/192.168.100.177 to mvm4:8025 failed on connection exception: 
 java.net.ConnectException: Connection refused; For more details see:  
 http://wiki.apache.org/hadoop/ConnectionRefused
 at 
 org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:139)
 at $Proxy14.registerNodeManager(Unknown Source)
 at 
 org.apache.hadoop.yarn.server.api.impl.pb.client.ResourceTrackerPBClientImpl.registerNodeManager(ResourceTrackerPBClientImpl.java:59)
 ... 5 more
 Caused by: java.net.ConnectException: Call From mvm5/192.168.100.177 to 
 mvm4:8025 failed on connection exception: java.net.ConnectException: 
 Connection refused; For more details see:  
 http://wiki.apache.org/hadoop/ConnectionRefused
 at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:617)
 at org.apache.hadoop.ipc.Client.call(Client.java:1089)
 at 
 org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:136)
 ... 7 more
 Caused by: java.net.ConnectException: Connection refused
 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
 at 
 sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
 at 
 org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
 at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:419)
 at 
 org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:460)
 at 
 org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:557)
 at 
 org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:205)
 at org.apache.hadoop.ipc.Client.getConnection(Client.java:1195)
 at org.apache.hadoop.ipc.Client.call(Client.java:1065)
 ... 8 more
 2012-06-12 19:05:56,184 INFO  ipc.Server (Server.java:stop(1709)) - Stopping 
 server on 47645
 2012-06-12 19:05:56,184 INFO  ipc.Server (Server.java:stop(1709)) - Stopping 
 server on 4344
 2012-06-12 19:05:56,190 INFO  impl.MetricsSystemImpl 
 (MetricsSystemImpl.java:stop(199)) - Stopping NodeManager metrics system...
 2012-06-12 19:05:56,190 INFO  impl.MetricsSystemImpl 
 (MetricsSystemImpl.java:stopSources(408)) - Stopping metrics source JvmMetrics
 2012-06-12 19:05:56,191 INFO  nodemanager.NodeManager 
 (StringUtils.java:run(605)) - SHUTDOWN_MSG:

--
This message is automatically