543 changes logging for local mode. By default, the logs from local mode map-reduce tasks now go to a per-query log file that's written to /tmp/<username>/<queryid>.log
This was very much intended to enable local mode to be a bit friendlier to use. -----Original Message----- From: Mayank Lahiri [mailto:mayank.lah...@facebook.com] Sent: Monday, June 21, 2010 2:54 PM To: John Sichi Cc: hive-dev@hadoop.apache.org Subject: Re: Latest trunk breaks logging? Hi Joydeep, I've confirmed that logging is restored after reverting HIVE-543. The problem is that LOG.warn() calls from inside UDAFs do not generate any output after applying HIVE-543. For example, passing an invalid argument to histogram() causes a general one-line exception instead of the diagnostic HiveException that is supposed to be thrown. This is what HEAD currently returns: ----- snip ------ FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask ---- end snip ---- Was this intended, or are the logs just being written to some other location post-HIVE-543? Thanks, -Mayank On 6/21/10 1:00 PM, "John Sichi" <jsi...@facebook.com> wrote: +hive-dev Looks like Joydeep made some changes to the logging in recently committed HIVE-543; maybe related? Apache JIRA is down at the moment but I found the patch here: http://mail-archives.apache.org/mod_mbox/hadoop-hive-commits/201006.mbox/%3c20100616225037.c67df2388...@eris.apache.org%3e Mayank, if you can confirm that this is the cause, we can check with Joydeep on whether or not this was intentional. The DataNucleus noise I've been seeing for a while now; I think it's harmless, but go ahead and create a JIRA issue to get it cleaned up. JVS On Jun 21, 2010, at 12:34 PM, Mayank Lahiri wrote: Hi John, I just updated trunk and logging seems to be slightly different, and possibly broken. For one, LOG.warn() messages from inside UDAFs don't show up anywhere, and are not printed to console. /tmp/mlahiri/hive.log contains a lot of lines like this: 2010-06-21 12:02:01,486 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c ore" requires "org.eclipse.core.runtime" but it cannot be resolved. 2010-06-21 12:02:01,487 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c ore" requires "org.eclipse.text" but it cannot be resolved. 2010-06-21 12:23:23,576 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c ore" requires "org.eclipse.core.resources" but it cannot be resolved. 2010-06-21 12:23:23,578 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c ore" requires "org.eclipse.core.runtime" but it cannot be resolved. 2010-06-21 12:23:23,579 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c ore" requires "org.eclipse.text" but it cannot be resolved. 2010-06-21 12:27:45,003 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c ore" requires "org.eclipse.core.resources" but it cannot be resolved. 2010-06-21 12:27:45,005 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c ore" requires "org.eclipse.core.runtime" but it cannot be resolved. 2010-06-21 12:27:45,005 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c ore" requires "org.eclipse.text" but it cannot be resolved. 2010-06-21 12:30:13,786 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c ore" requires "org.eclipse.core.resources" but it cannot be resolved. 2010-06-21 12:30:13,788 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c ore" requires "org.eclipse.core.runtime" but it cannot be resolved. 2010-06-21 12:30:13,789 ERROR DataNucleus.Plugin (Log4JLogger.java:error(115)) - Bundle "org.eclipse.jdt.c ore" requires "org.eclipse.text" but it cannot be resolved. ~ Any ideas what I could be doing differently/wrong? - Mayank
