Are you using Hive on top of Hadoop or writing a raw Hadoop job? This is a the hive list so I'm going to assumed you're running hive... can you send your HiveQL query along?
Pat From: hadoop n00b [mailto:[email protected]] Sent: Wednesday, March 16, 2011 3:33 AM To: [email protected] Subject: Fwd: Hadoop error 2 while joining two large tables Hello, I am trying to execute a query that joins two large tables (3 million and 20 million records). I am getting the Hadoop error code 2 during execution. This happens mainly while the reducers are running. Sometimes the reducers complete 100% and then the error comes. The logs talk about running out of Heap space and GC overhead limit exceeding. I am running a 6 node cluster with child JVM memory of 1GB. Are there any parameters I could tweak to make them run? Is adding more nodes the solution to such problem? Thanks!
