Cool, figured you guys were on it but couldn't locate the JIRA for some reason. I'll blame user error :)
Thanks for the fix! Nick Sent from my iPhone On Feb 27, 2014, at 6:09 PM, "Prasanth Jayachandran" <[email protected]<mailto:[email protected]>> wrote: Hi Martin This is an known issue and its fixed in hive trunk. It should be available in 0.13 release. https://issues.apache.org/jira/browse/HIVE-5601 Thanks Prasanth Jayachandran On Feb 26, 2014, at 8:55 AM, Martin, Nick <[email protected]<mailto:[email protected]>> wrote: Hi all, (Running Hive 12.0) I have two tables and both are stored as ORC. I attempted to insert via select from tbl1 to tbl2 using ‘BETWEEN’ in my where clause to narrow down some dates. Something like so: “Insert into tbl1 select col1, col2 from tbl2 where col1 between 2 and 4” I kept hitting the error pasted below. So, I switched to a different approach to see if it would work: “Insert into tbl1 select col1,col2 from tbl2 where col1>=2 and col1<=4” Hit the same error. When I just use “where col1>=2” in the where clause the insert will run fine. Is this expected? 2014-02-26 11:22:53,755 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring. 2014-02-26 11:22:53,782 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring. 2014-02-26 11:22:53,902 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2014-02-26 11:22:53,930 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSinkAdapter: Sink ganglia started 2014-02-26 11:22:53,975 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2014-02-26 11:22:53,975 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started 2014-02-26 11:22:53,985 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens: 2014-02-26 11:22:53,985 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1392147432508_1108, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@249c2715) 2014-02-26 11:22:54,057 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now. 2014-02-26 11:22:54,352 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring. 2014-02-26 11:22:54,363 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring. 2014-02-26 11:22:54,409 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /hdfs/01/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/02/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/03/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/04/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/05/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/06/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/07/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/08/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/09/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/10/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/11/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108,/hdfs/12/hadoop/yarn/local/usercache/myusername/appcache/application_1392147432508_1108 2014-02-26 11:22:54,481 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring. 2014-02-26 11:22:54,486 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring. 2014-02-26 11:22:54,542 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id 2014-02-26 11:22:54,542 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap 2014-02-26 11:22:54,543 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir 2014-02-26 11:22:54,543 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.cache.localFiles is deprecated. Instead, use mapreduce.job.cache.local.files 2014-02-26 11:22:54,543 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id 2014-02-26 11:22:54,544 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id 2014-02-26 11:22:54,544 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition 2014-02-26 11:22:54,545 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: job.local.dir is deprecated. Instead, use mapreduce.job.local.dir 2014-02-26 11:22:54,546 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.cache.localArchives is deprecated. Instead, use mapreduce.job.cache.local.archives 2014-02-26 11:22:54,710 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id 2014-02-26 11:22:55,007 INFO [main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2014-02-26 11:22:55,316 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: Paths:/apps/hive/warehouse/edw.db/penopp_fsm/000000_0:0+116456645,/apps/hive/warehouse/edw.db/penopp_fsm/000001_0:0+116484938,/apps/hive/warehouse/edw.db/penopp_fsm/000002_0:0+116473509InputFormatClass: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat 2014-02-26 11:22:55,370 INFO [main] org.apache.hadoop.hive.ql.exec.Utilities: <PERFLOG method=deserializePlan> 2014-02-26 11:22:55,370 INFO [main] org.apache.hadoop.hive.ql.exec.Utilities: Deserializing MapWork via kryo 2014-02-26 11:22:55,574 INFO [main] org.apache.hadoop.hive.ql.exec.Utilities: </PERFLOG method=deserializePlan start=1393431775370 end=1393431775574 duration=204> 2014-02-26 11:22:55,708 INFO [main] org.apache.hadoop.hive.ql.io.orc.OrcInputFormat: included column ids = 0,1,2,3,4,5,6 2014-02-26 11:22:55,708 INFO [main] org.apache.hadoop.hive.ql.io.orc.OrcInputFormat: included columns names = col1,col2,col3,col4,col5,col6,col7 2014-02-26 11:22:55,728 INFO [main] org.apache.hadoop.hive.ql.io.orc.OrcInputFormat: ORC pushdown predicate: leaf-0 = (BETWEEN col7 201301 201312) expr = leaf-0 2014-02-26 11:22:55,897 ERROR [main] org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:myusername (auth:SIMPLE) cause:java.io.IOException: java.lang.reflect.InvocationTargetException 2014-02-26 11:22:55,898 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.io.IOException: java.lang.reflect.InvocationTargetException at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97) at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57) at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:343) at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.<init>(HadoopShimsSecure.java:290) at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader(HadoopShimsSecure.java:404) at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:556) at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:167) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:408) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:329) ... 11 more Caused by: java.lang.NullPointerException at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.planReadPartialDataStreams(RecordReaderImpl.java:1962) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readPartialDataStreams(RecordReaderImpl.java:2118) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.readStripe(RecordReaderImpl.java:1775) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceStripe(RecordReaderImpl.java:2148) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.advanceToNextRow(RecordReaderImpl.java:2190) at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.<init>(RecordReaderImpl.java:130) at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rows(ReaderImpl.java:335) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.<init>(OrcInputFormat.java:100) at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:205) at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:65) ... 16 more 2014-02-26 11:22:55,902 INFO [main] org.apache.hadoop.mapred.Task: Runnning cleanup for the task CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
