Russell, See if this can help you- http://indoos.wordpress.com/2010/06/24/hive-remote-debugging/
You would have to tweak the old scripts however for your Hadoop and Hive versions Regards, Sanjay Sharma -----Original Message----- From: Russell Melick [mailto:rmel...@hmc.edu] Sent: Wednesday, November 17, 2010 1:04 PM To: hive-...@hadoop.apache.org Cc: j...@hmc.edu; mw...@hmc.edu Subject: Debugging Hive Hi all, Apologies for the long email. I'm attempting to connect to the hive code with a debugger to understand the optimizer. In this example, I would like to pause at the beginning of the createIndex function in DDLTask. After following the directions on the wiki, I'm stumped as to why connecting remotely with jdb is not working. Any help is appreciated. I've detailed my process below. Thanks, Russell Melick First, I check out a fresh copy of trunk: > svn checkout http://svn.apache.org/repos/asf/hive/trunk hive Then, I compile it for debugging, as I found at http://wiki.apache.org/hadoop/Hive/DeveloperGuide#Debugging_Hive_code: > cd hive > ant -Djavac.debug=on package Here are the important variables before I run hive: > echo $HIVE_DEBUG_PORT > 8000 > echo $HIVE_DEBUG > -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y > echo $HADOOP_OPTS > -Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=y I then try to run one of the unit tests: > ant test -Dtestcase=TestCliDriver -Dqfile=index_compact_1.q The test then waits for a debugger to attach: > [junit] Begin query: index_compact_1.q > [junit] Listening for transport dt_socket at address: 8000 However, when I try to attach with jdb, it doesn't work. > jdb -attach 8000 > Set uncaught java.lang.Throwable > Set deferred uncaught java.lang.Throwable > Initializing jdb ... > > > VM Started: No frames on the current call stack > > main[1] stop at DDLTask:317 > Deferring breakpoint DDLTask:317. > It will be set after the class is loaded. > main[1] resume > All threads resumed. > > > The application exited The unit test does not seem to recognize that it received the connection. After jdb prints "application exited", the test says it is listening again, but it didn't confirm the connection. > [junit] Begin query: index_compact_1.q > [junit] Listening for transport dt_socket at address: 8000 > [junit] Deleted > file:/home/rmelick/hive/build/ql/test/data/warehouse/default__src_src_index__ > [junit] Listening for transport dt_socket at address: 8000 > [junit] Listening for transport dt_socket at address: 8000 I have also attempted to start hiveCli, but I have had no success getting the CLI to start after attaching the debugger. This is the only output I get from the cli: > build/dist/bin/hive > Unable to determine Hadoop version information. > 'hadoop version' returned: jdb doesn't seem to be connecting correctly > jdb -attach 8000 > Set uncaught java.lang.Throwable > Set deferred uncaught java.lang.Throwable > Initializing jdb ... > > > VM Started: No frames on the current call stack > > main[1] stop at DDLTask:317 > Deferring breakpoint DDLTask:317. > It will be set after the class is loaded. > main[1] resume > All threads resumed. > > > The application exited I've also tried connecting with Eclipse, after generating the eclipse files, importing the project, and creating the MapRedTask configuration. It either seems to similarly connect without the client recognizing it, or throws a connection error. Again, thanks for any help. Impetus is a proud sponsor for ASCI Tour 2010 (Agile Software Community of India) on Oct 30 in Noida, India. Meet Impetus at the Cloud Computing Expo from Nov 1-4 in Santa Clara. Our Sr. Director of Engineering, Vineet Tyagi will be speaking about ‘Using Hadoop for Deriving Intelligence from Large Data’. Click http://www.impetus.com/ to know more. Follow us on www.twitter.com/impetuscalling NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.
