HCatalogLoader should allow specifying a partition spec as a constructor
argument
---------------------------------------------------------------------------------
Key: HCATALOG-61
URL: https://issues.apache.org/jira/browse/HCATALOG-61
Project: HCatalog
Issue Type: Improvement
Reporter: Julien Le Dem
In some cases we want to pass the list of partitions to read as a parameter to
the Pig Script.
This enables incremental processing. Reading a few new partitions in the input
and generating a new corresponding partition in the output.
something similar to the partspec in the HCatStorer:
{code}
pig -p "INPUT_PARTITIONS=t=4,t=5,t=6" -p "OUTPUT_PARTITION=t=2" myscript.pig
{code}
myscript.pig
{code}
A = LOAD '...' USING HCatLoader('$INPUT_PARTITIONS');
...
STORE Z INTO '...' USING HCatStorer('$OUTPUT_PARTITION');
{code}
The number of partitions could be variable from one run to the other.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
