Isn't this the JDO issue with mysql? https://issues.apache.org/jira/browse/HCATALOG-209
On 3/13/12 12:24 PM, "Travis Crawford" <[email protected]> wrote: > Here's the stack trace (below). What's interesting is I just verified a job > with this filter statement works: > > f = FILTER data BY part_dt == '20120313T000000Z'; > > Then the stack trace below happens when changing to: > > f = FILTER data BY part_dt >= '20120313T000000Z'; > > This suggests its not a thrift version issue, and I've double-checked and > everything appears to be thrift 0.7.0 (looked inside jars for potential > packaging-related conflicts too). > > Additionally, here's a similar show partitions statement in the Hive shell: > > hive> show partitions foo partition(part_dt='20120313T110000Z'); > OK > part_dt=20120313T110000Z > Time taken: 0.257 seconds > hive> show partitions foo partition(part_dt>='20120313T110000Z'); > FAILED: Parse Error: line 1:39 mismatched input '>=' expecting ) near > 'part_dt' in show statement > > hive> show partitions foo partition(part_dt>'20120313T110000Z'); > FAILED: Parse Error: line 1:39 mismatched input '>' expecting ) near > 'part_dt' in show statement > > Would you expect this sort of range query to work? > > > Pig Stack Trace > --------------- > ERROR 2017: Internal error creating job configuration. > > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to > open iterator for alias l > at org.apache.pig.PigServer.openIterator(PigServer.java:857) > at > org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:655) > at > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.ja > va:303) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164) > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84) > at org.apache.pig.Main.run(Main.java:561) > at org.apache.pig.Main.main(Main.java:111) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.j > ava:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:186) > Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias l > at org.apache.pig.PigServer.storeEx(PigServer.java:956) > at org.apache.pig.PigServer.store(PigServer.java:919) > at org.apache.pig.PigServer.openIterator(PigServer.java:832) > ... 12 more > Caused by: > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationExcept > ion: > ERROR 2017: Internal error creating job configuration. > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompile > r.getJob(JobControlCompiler.java:736) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompile > r.compile(JobControlCompiler.java:261) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > .launchPig(MapReduceLauncher.java:151) > at org.apache.pig.PigServer.launchPlan(PigServer.java:1270) > at > org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1255) > at org.apache.pig.PigServer.storeEx(PigServer.java:952) > ... 14 more > Caused by: java.io.IOException: > org.apache.thrift.transport.TTransportException > at > org.apache.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:42> ) > at > org.apache.hcatalog.pig.HCatLoader.setLocation(HCatLoader.java:90) > at > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompile > r.getJob(JobControlCompiler.java:385) > ... 19 more > Caused by: org.apache.thrift.transport.TTransportException > at > org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:13 > 2) > at > org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) > at > org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:1 > 29) > at > org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101) > at > org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) > at > org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378) > at > org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297) > at > org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.ja > va:204) > at > org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) > at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_parti > tions_by_filter(ThriftHiveMetastore.java:1399) > at > org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_partitions > _by_filter(ThriftHiveMetastore.java:1383) > at > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.listPartitionsByFilter(Hi > veMetaStoreClient.java:680) > at > org.apache.hcatalog.mapreduce.InitializeInput.getSerializedHcatKeyJobInfo(Init > ializeInput.java:100) > at > org.apache.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:76> ) > at > org.apache.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:40> ) > ... 21 more > ============================================================================== > == > > > --Travis > > > > On Tue, Mar 13, 2012 at 11:28 AM, Ashutosh Chauhan > <[email protected]>wrote: > >> Hey Travis, >> >> You should never get an exception. Difference in behavior will be whether >> filter got pushed from Pig into HCat or not, but that should not affect >> correctness. >> Can you paste the stack trace that you are getting? >> >> Ashutosh >> >> On Tue, Mar 13, 2012 at 11:14, Travis Crawford <[email protected] >>> wrote: >> >>> Hey HCat gurus - >>> >>> I'm having an issue getting range filters working and am curious if what >>> I'm trying to do makes sense. When filtering for explicit partitions >> (even >>> multiple explicit partitions) things work as expected. However, when >>> requesting a range it fails with a TTransportException. >>> >>> data = LOAD 'db.table' USING org.apache.hcatalog.pig.HCatLoader(); >>> >>> -- filter statements that work >>> f = FILTER data BY part_dt == '20120313T000000Z'; >>> f = FILTER data BY part_dt == '20120313T000000Z' >>> OR part_dt == '20120313T010000Z'; >>> >>> -- filter statements that do not work >>> f = FILTER data BY part_dt >= '20120313T000000Z'; >>> f = FILTER data BY part_dt >= '20120313T000000Z' >>> AND part_dt < '20120313T010000Z'; >>> >>> When things are working correctly would you expect all four of these >> filter >>> statements to be valid? >>> >>> Thanks! >>> Travis >>> >>
