Thanks Zheng, It actually appears not to build with the patch...
$ svn status ? HIVE-624.1.patch $ svn update At revision 793104. $ svn info Path: . URL: http://svn.apache.org/repos/asf/hadoop/hive/trunk Repository Root: http://svn.apache.org/repos/asf Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68 Revision: 793104 Node Kind: directory Schedule: normal Last Changed Author: namit Last Changed Rev: 792772 Last Changed Date: 2009-07-09 21:04:37 -0400 (Thu, 09 Jul 2009) $ patch -p0 < HIVE-624.1.patch ... $ ant clean ... $ ant -Dhadoop.version="0.18.3" package ... compile: [echo] Compiling: hive [javac] Compiling 200 source files to /Users/dlerman/Documents/VideoEgg/repo/vdat/ext_unused/hive_trunk/build/serd e/classes [javac] /Users/dlerman/Documents/VideoEgg/repo/vdat/ext_unused/hive_trunk/serde/src/ java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyByteOb jectInspector.java:30: org.apache.hadoop.hive.serde2.lazy.objectinspector.primitive.LazyByteObjectI nspector is not abstract and does not override abstract method set(java.lang.Object,byte) in org.apache.hadoop.hive.serde2.objectinspector.primitive.ByteObjectInspector [javac] public class LazyByteObjectInspector extends AbstractPrimitiveLazyObjectInspector<ByteWritable> [javac] ^ [javac] /Users/dlerman/Documents/VideoEgg/repo/vdat/ext_unused/hive_trunk/serde/src/ java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaByteObjectI nspector.java:26: org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaByteObjectInspec tor is not abstract and does not override abstract method set(java.lang.Object,byte) in org.apache.hadoop.hive.serde2.objectinspector.primitive.ByteObjectInspector [javac] public class JavaByteObjectInspector extends AbstractPrimitiveJavaObjectInspector [javac] ^ [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. [javac] 2 errors On 7/10/09 4:42 PM, "Zheng Shao" <[email protected]> wrote: > Hi David, > > Please do "ant clean" before running "ant -Dhadoop.version=0.18.x package" > > Zheng > > On Fri, Jul 10, 2009 at 12:45 PM, David Lerman<[email protected]> wrote: >> Thanks Zheng, >> >> Applying the 624 patch changes the Exception to: >> >> java.lang.ClassCastException: java.util.ArrayList cannot be cast to >> java.util.Vector >> at >> org.apache.hadoop.hive.ql.exec.JoinOperator.process(JoinOperator.java:58) >> at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:189) >> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318) >> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198) >> >> Dave >> >> >> On 7/10/09 3:17 PM, "Zheng Shao" <[email protected]> wrote: >> >>> Hi David, >>> >>> Thanks for letting us know. I will take a look now. >>> >>> In the meanwhile, there is a fix related to types >>> https://issues.apache.org/jira/browse/HIVE-624 which might solve the >>> problem. >>> You might want to try it out. >>> >>> Zheng >>> >>> On Fri, Jul 10, 2009 at 11:28 AM, David Lerman<[email protected]> wrote: >>>> Attempting to join three tables is consistently failing with a >>>> ClassCastException using Hive trunk (r792966) and Hadoop 0.18.3. >>>> >>>> The three tables are defined as follows: >>>> >>>> create table foo (foo_id int, foo_name string, foo_a string, foo_b string, >>>> foo_c string, foo_d string) row format delimited fields terminated by ',' >>>> stored as textfile; >>>> >>>> create table bar (bar_id int, bar_0 int, foo_id int, bar_1 int, bar_name >>>> string, bar_a string, bar_b string, bar_c string, bar_d string) row format >>>> delimited fields terminated by ',' stored as textfile; >>>> >>>> create table count (bar_id int, n int) row format delimited fields >>>> terminated by ',' stored as textfile; >>>> >>>> Each table has a single row as follows: >>>> >>>> foo: >>>> 1,foo1,a,b,c,d >>>> >>>> bar: >>>> 10,0,1,1,bar10,a,b,c,d >>>> >>>> counts: >>>> 10,2 >>>> >>>> The failing query is: >>>> >>>> select foo.foo_name, bar.bar_name, n from foo join bar on foo.foo_id = >>>> bar.foo_id join count on count.bar_id = bar.bar_id; >>>> >>>> Interestingly, the query works if you reorder the joins (select >>>> foo.foo_name, bar.bar_name, n from count join bar on count.bar_id = >>>> bar.bar_id join foo on foo.foo_id = bar.foo_id) or if you remove any of the >>>> unused string columns from foo or even just move the unused int columns in >>>> bar to the end. >>>> >>>> The exception is as follows: >>>> >>>> java.lang.ClassCastException: >>>> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectIn>>>> s >>>> pector cannot be cast to >>>> org.apache.hadoop.hive.serde2.objectinspector.primitive.StringObjectInspect>>>> o >>>> r >>>> at >>>> org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDeTypeString.serialize>>>> ( >>>> DynamicSerDeTypeString.java:63) >>>> at >>>> org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDeFieldList.serialize(>>>> D >>>> ynamicSerDeFieldList.java:249) >>>> at >>>> org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDeStructBase.serialize>>>> ( >>>> DynamicSerDeStructBase.java:81) >>>> at >>>> org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe.serialize(DynamicSe>>>> r >>>> De.java:177) >>>> at >>>> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.ja>>>> v >>>> a:180) >>>> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:492) >>>> at >>>> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.createForwardJoinObject(C>>>> o >>>> mmonJoinOperator.java:290) >>>> at >>>> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOpera>>>> t >>>> or.java:533) >>>> at >>>> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOpera>>>> t >>>> or.java:522) >>>> at >>>> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOpera>>>> t >>>> or.java:522) >>>> at >>>> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJ>>>> o >>>> inOperator.java:563) >>>> at >>>> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.endGroup(CommonJoinOperat>>>> o >>>> r.java:545) >>>> at org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:159) >>>> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:318) >>>> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198) >>>> java.lang.ClassCastException: >>>> org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectIn>>>> s >>>> pector cannot be cast to >>>> org.apache.hadoop.hive.serde2.objectinspector.primitive.StringObjectInspect>>>> o >>>> r >>>> at >>>> org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDeTypeString.serialize>>>> ( >>>> DynamicSerDeTypeString.java:63) >>>> at >>>> org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDeFieldList.serialize(>>>> D >>>> ynamicSerDeFieldList.java:249) >>>> at >>>> org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDeStructBase.serialize>>>> ( >>>> DynamicSerDeStructBase.java:81) >>>> at >>>> org.apache.hadoop.hive.serde2.dynamic_type.DynamicSerDe.serialize(DynamicSe>>>> r >>>> De.java:177) >>>> at >>>> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.ja>>>> v >>>> a:180) >>>> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:492) >>>> at >>>> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.createForwardJoinObject(C>>>> o >>>> mmonJoinOperator.java:290) >>>> at >>>> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOpera>>>> t >>>> or.java:533) >>>> at >>>> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOpera>>>> t >>>> or.java:522) >>>> at >>>> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOpera>>>> t >>>> or.java:522) >>>> at >>>> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJ>>>> o >>>> inOperator.java:563) >>>> at >>>> org.apache.hadoop.hive.ql.exec.CommonJoinOperator.endGroup(CommonJoinOperat>>>> o >>>> r.java:545) >>>> at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:236) >>>> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:329) >>>> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198) >>>> >>>> >>>> Thanks for your help! >>>> >>>> >>> >>> >>> >>> -- >>> Yours, >>> Zheng >> >> > > > > -- > Yours, > Zheng
