Thanks for getting back on what change worked. :) I've updated the wiki as per your suggestion : https://cwiki.apache.org/confluence/display/Hive/HiveDeveloperFAQ
On Fri, Sep 6, 2013 at 4:12 PM, Eric Chu <[email protected]> wrote: > I found out what I missed - I needed to set JAVA_HOME. It'd be good if we > could add that to the unit test documentation. It's not intuitive to me b/c > some tests did run w/out that. After setting that, the unit tests seem to > run fine. > > Thanks, > > Eric > > > On Fri, Sep 6, 2013 at 2:27 AM, Eric Chu <[email protected]> wrote: >> >> It does, and MiniMrShim is defined in HadoopShims.java... Anyway, the >> problem came up after various experiments to get the unit tests pass, so >> maybe at some point something got corrupted. >> >> My main concern, however, is much broader than this - I can never get all >> the unit tests pass for Hive 11. It's much worse than when I upgraded Hive >> to 0.10. I don't know what to expect about these unit tests. Are they >> supposed to all pass if you just run "ant clean package test" on >> branch-0.11? >> >> I tried the following approach: I installed VirtualBox and Vagrant >> (lucid32), and other required programs (ant, git, java, make) for running >> Hive's unit tests, then git clone apache hive, and ran the following on >> branch-0.11: >> >> export ANT_OPTS="-Xms768m -Xmx1024m -XX:PermSize=128m >> -XX:MaxPermSize=128m" >> ant clean package test -logfile ant.log -Dtest.silent=false >> >> I got lots of errors (attached is the full log) with this approach. They >> are typically in the form of: >> >> Begin query: alter3.q >> [junit] Deleted >> file:/home/vagrant/hive/build/ql/test/data/warehouse/alter3_src >> [junit] /home/vagrant/hive/testutils/hadoop: line 109: /bin/java: No >> such file or directory >> [junit] /home/vagrant/hive/testutils/hadoop: line 109: exec: >> /bin/java: cannot execute: No such file or directory >> [junit] Exception: Client Execution failed with error code = 126 >> [junit] See build/ql/tmp/hive.log, or try "ant test ... >> -Dtest.silent=false" to get more logs. >> [junit] junit.framework.AssertionFailedError: Client Execution failed >> with error code = 126 >> [junit] See build/ql/tmp/hive.log, or try "ant test ... >> -Dtest.silent=false" to get more logs. >> ... >> >> /home/vagrant/hive/testutils/hadoop: line 109: /bin/java: No such file or >> directory >> [junit] /home/vagrant/hive/testutils/hadoop: line 109: exec: >> /bin/java: cannot execute: No such file or directory >> [junit] Execution failed with exit status: 126 >> [junit] Obtaining error information >> [junit] >> [junit] Task failed! >> [junit] Task ID: >> [junit] null >> ... >> Logs: >> [junit] >> [junit] /home/vagrant/hive/build/ql/tmp/hive.log >> [junit] testMapPlan1 execution failed with exit status: 126 >> [junit] junit.framework.AssertionFailedError: expected:<true> but >> was:<false> >> [junit] at junit.framework.Assert.fail(Assert.java:47) >> >> >> I'm trying to make sure unit tests of 11 pass before I upgrade from 10 to >> 11. So the code is straight from branch-0.11, and I followed the >> instructions from the Hive wiki to run the unit tests. I don't understand >> why I'd get these errors, for so many times. If anyone who has successfully >> gotten unit tests to pass (those who regularly commit), can they share some >> insights? The documentation is painfully insufficient. Also, if I take a >> branch like 0.11, what/how many tests would I expect to pass? (A majority of >> tests fail in my experience) >> >> Thanks so much, >> >> Eric >> >> >> On Wed, Sep 4, 2013 at 11:00 AM, Sushanth Sowmyan <[email protected]> >> wrote: >>> >>> This seems to work for me on trunk. This should work in 0.11 as well, >>> given that the feature it is complaining about was introduced in >>> HIVE-4139, which was committed in hive-0.11. >>> >>> Could you please check your /hive/build/shims/hive-shims-0.11.0.jar >>> (should be the same as >>> >>> /Users/echu/.ivy2/local/org.apache.hive/hive-shims/0.11.0/jars/hive-shims.jar >>> ) and run jar tf on it to see if it has the following definitions: >>> >>> org/apache/hadoop/hive/shims/Hadoop20SShims$MiniMrShim.class >>> org/apache/hadoop/hive/shims/Hadoop20Shims$MiniMrShim.class >>> org/apache/hadoop/hive/shims/Hadoop23Shims$MiniMrShim.class >>> org/apache/hadoop/hive/shims/HadoopShims$MiniMrShim.class >>> >>> And failing that, if your codebase has MiniMrShim defined in >>> shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java ? >>> >>> On Tue, Sep 3, 2013 at 2:39 PM, Eric Chu <[email protected]> wrote: >>> > Hi, >>> > >>> > I'm trying to run unit tests on Hive 11 (with a few patches such as >>> > 4619, >>> > 4003, 4403, 4900, 3632, 4942) and encountering compile-test errors. I >>> > got >>> > the same thing when I ran on Trunk or just the 11 branch. Is there >>> > something >>> > I'm missing? Note that I could build and run Hive w/out problems. >>> > >>> > Commands: >>> > export ANT_OPTS=-XX:MaxPermSize=512M >>> > ant very-clean package test -logfile ant.log >>> > >>> > Error (Attached is full log): >>> > compile-test: >>> > [echo] Project: ql >>> > [javac] Compiling 92 source files to /hive/build/ql/test/classes >>> > [javac] >>> > /hive/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java:118: >>> > cannot find symbol >>> > [javac] symbol : class MiniMrShim >>> > [javac] location: interface >>> > org.apache.hadoop.hive.shims.HadoopShims >>> > [javac] private HadoopShims.MiniMrShim mr = null; >>> > [javac] ^ >>> > [javac] >>> > /hive/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java:284: >>> > cannot find symbol >>> > [javac] symbol : method >>> > >>> > getMiniMrCluster(org.apache.hadoop.hive.conf.HiveConf,int,java.lang.String,int) >>> > [javac] location: interface >>> > org.apache.hadoop.hive.shims.HadoopShims >>> > [javac] mr = >>> > ShimLoader.getHadoopShims().getMiniMrCluster(conf, 4, >>> > getHdfsUriString(fs.getUri().toString()), 1); >>> > [javac] ^ >>> > [javac] >>> > >>> > /hive/ql/src/test/org/apache/hadoop/hive/ql/exec/TestFunctionRegistry.java:57: >>> > cannot find symbol >>> > [javac] symbol : variable decimalTypeInfo >>> > [javac] location: class >>> > org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory >>> > [javac] implicit(TypeInfoFactory.intTypeInfo, >>> > TypeInfoFactory.decimalTypeInfo, true); >>> > >>> > .... >>> > >>> > Thanks, >>> > >>> > Eric >>> > >> >> >
