[
https://issues.apache.org/jira/browse/HCATALOG-347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255771#comment-13255771
]
Thomas Weise commented on HCATALOG-347:
---------------------------------------
Verified that the filter criteria is passed to HCat as expected, and on to
HiveMetaStoreClient.listPartitionsByFilter, which seems to ignore it. Search to
continue in Hive.
At least the hive cli does not recognize "!=" as operator:
hive -e "SHOW PARTITIONS pig_checkin_6 PARTITION(ds!='20110924');"
FAILED: Parse Error: line 1:42 mismatched input '!=' expecting ) near 'ds' in
show statement
> Pig loads all partitions when it is specifically told not to load all
> ---------------------------------------------------------------------
>
> Key: HCATALOG-347
> URL: https://issues.apache.org/jira/browse/HCATALOG-347
> Project: HCatalog
> Issue Type: Bug
> Components: pig
> Affects Versions: 0.2
> Reporter: Viraj Bhat
> Assignee: Thomas Weise
>
> I have Pig script of this nature. It accesses a partitioned table,
> partitioned on gridname and dt (datestamp)
> {code}
> A = LOAD 'mytable' USING org.apache.hcatalog.pig.HCatLoader();
> B = FILTER A BY gridname=='XY' and dt != '2012_03_21';
> C = foreach B generate job_id, user;
> store C into '/user/viraj/test/XY' using PigStorage();
> {code}
> I use this as some partitions of the table have not been populated.
> I get an error:
> Backend error message during job submission
> {quote}
> -------------------------------------------
> org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path
> does not exist:
> hdfs://namenode/warehouse/database_confs/gridname=XY/dt=2012_03_21
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:282)
> at
> org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:962)
> at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979)
> at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:897)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1082)
> {quote}
> I suspect that the filter clause is not pushed up.
> Regards
> Viraj
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
