Based on my experience, remote debugging is used to debug running MapReduce jobs. Seems TestRCFile is used to test RCFile reader and writer individually (I mean it is not related to MapReduce). So you should be able to run it in eclipse. Seems you need to set value for "test.data.dir" and "test.build.data" through run configuration. Also you may need to change the path used in testReadOldFileHeader (i mean "src/test/data/rc-file-v0.rc"), otherwise, this test can fail with a FileNotFoundException. It is a ugly way.... but it can work...
On Fri, Jul 19, 2013 at 2:09 PM, Mainak Ghosh <gho...@us.ibm.com> wrote: > Hey Yin, > > Thanks for all the pointers. I have been able to get the debugger to > attach and the tests to run with errors from unmaskcheck.sh. There is one > last thing that I needed but could not do. I want to run the TestRCFile > code inside a debugger with breakpoints inside the code. Any idea how I can > achieve that? > > The problem here is my debugger attaches to the live instance of hive and > testcase runs using ant. > > Thanks and Regards, > Mainak. > > [image: Inactive hide details for Yin Huai ---07/19/2013 11:25:20 AM---Hi > Mainak, Here are steps:]Yin Huai ---07/19/2013 11:25:20 AM---Hi Mainak, > Here are steps: > > > From: Yin Huai <huaiyin....@gmail.com> > To: dev@hive.apache.org, > Date: 07/19/2013 11:25 AM > > Subject: Re: Trying to run TestRCFile.java > ------------------------------ > > > > Hi Mainak, > > > Here are steps: > 1. go to the src dir in your hive dir. From your log, it seems > /root/hive-0.11.0/src/ > 2. Build hive with 'ant clean package' > 3. Run RCFile test 'ant test -Dtestcase=TestRCFile'. > 4. If you see 'BUILD FAILED' and errors are related to > 'build-support/scripts/umaskcheck.sh', no worry. Run 'ant testreport' to > generate a html test report. You can find it at > '/root/hive-0.11.0/src/build/test/junit-noframes.html'. > In this report, you can see passed and failed cases of TestRCFile. > > I have not dug about the errors since I do not see those when I am working > on trunk. > > You can take a look at this link > https://cwiki.apache.org/confluence/display/Hive/HowToContribute for > information related to build project, unit tests, and debugging. > > Thanks, > > Yin > > > On Fri, Jul 19, 2013 at 10:45 AM, Mainak Ghosh <gho...@us.ibm.com> wrote: > > > Hello Yin, > > > > I tried doing that by downloading and unzipping a fresh tar ball but the > > build fails with the following errors: > > > > [javac] > > > /root/hive-0.11.0/src/ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java:43: > > package org.apache.hadoop.hive.serde2.columnar does not exist > > [javac] import > > org.apache.hadoop.hive.serde2.columnar.BytesRefArrayWritable; > > [javac] ^ > > [javac] > > > /root/hive-0.11.0/src/ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java:44: > > package org.apache.hadoop.hive.serde2.columnar does not exist > > [javac] import > org.apache.hadoop.hive.serde2.columnar.BytesRefWritable; > > [javac] ^ > > [javac] > > > /root/hive-0.11.0/src/ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java:45: > > package org.apache.hadoop.hive.serde2.columnar does not exist > > [javac] import org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe; > > [javac] ^ > > [javac] > > > /root/hive-0.11.0/src/ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java:46: > > package org.apache.hadoop.hive.serde2.io does not exist > > [javac] import org.apache.hadoop.hive.serde2.io.ByteWritable; > > [javac] ^ > > [javac] > > > /root/hive-0.11.0/src/ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java:47: > > package org.apache.hadoop.hive.serde2.io does not exist > > [javac] import org.apache.hadoop.hive.serde2.io.DoubleWritable; > > [javac] ^ > > [javac] > > > /root/hive-0.11.0/src/ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java:48: > > package org.apache.hadoop.hive.serde2.io does not exist > > [javac] import org.apache.hadoop.hive.serde2.io.ShortWritable; > > [javac] ^ > > [javac] > > > /root/hive-0.11.0/src/ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java:49: > > package org.apache.hadoop.hive.serde2.objectinspector does not exist > > [javac] import > > org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils; > > [javac] ^ > > [javac] > > > /root/hive-0.11.0/src/ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java:50: > > package > org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils > > does not exist > > [javac] import > > > org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.ObjectInspectorCopyOption; > > [javac] > > ^ > > [javac] > > > /root/hive-0.11.0/src/ql/src/test/org/apache/hadoop/hive/ql/io/TestRCFile.java:74: > > cannot find symbol > > [javac] symbol : class ColumnarSerDe > > [javac] location: class org.apache.hadoop.hive.ql.io.TestRCFile > > [javac] private static ColumnarSerDe serDe; > > > > Any idea what these are? How to resolve them? > > > > I personally do not have a lot of idea on remote debugging Hive. So a > step > > by step manual if present would be of great help. > > > > Thanks and Regards, > > Mainak. > > > > > > [image: Inactive hide details for Yin Huai ---07/19/2013 10:38:40 AM---Hi > > Mainak, Can you try 'ant test -Dtestcase=TestRCFile'? It shou]Yin Huai > > ---07/19/2013 10:38:40 AM---Hi Mainak, Can you try 'ant test > > -Dtestcase=TestRCFile'? It should work. > > > > From: Yin Huai <huaiyin....@gmail.com> > > To: dev@hive.apache.org, > > Date: 07/19/2013 10:38 AM > > Subject: Re: Trying to run TestRCFile.java > > ------------------------------ > > > > > > > > > Hi Mainak, > > > > Can you try 'ant test -Dtestcase=TestRCFile'? It should work. > > > > Thanks, > > > > Yin > > > > > > On Fri, Jul 19, 2013 at 10:16 AM, Mainak Ghosh <gho...@us.ibm.com> > wrote: > > > > > > > > > > > Hello, > > > > > > I am new to Hive codebase and was hoping to run some of its test code > > > specifically TestRCFile.java. Unfortunately I am not sure what are the > > > steps to execute or debug it. I could not find any resource on the web. > > Any > > > help would be greatly appreciated. > > > > > > Thanks and Regards, > > > Mainak. > > > > > >
