[
https://issues.apache.org/jira/browse/OOZIE-1741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017657#comment-14017657
]
Satish Mittal commented on OOZIE-1741:
--------------------------------------
Thanks [~thiruvel] for your inputs!
- For 1), we already have an EL dataInPartitionFilter().
- For 2), the proposed EL in this jira dataInPartitions() can serve the purpose.
- For 3), I think we should add new ELs as per the requirement, since a large
number of combinations are possible here (comparator:- =, !=, <, > etc and
separator:- AND, OR).
The use-case of 3) is closer to 1) since it aims to generate a filter criteria,
as against 2) which aims to generate the value of a partition spec to be used
in DDL/import/export queries.
I checked that in Hive, the format of partition spec in various DDL queries
(alter, truncate, drop, show, describe), DML queries (load, insert) is:
(partcol1=val1, partcol2=val2 ...). The format of partition spec for
IMPORT/EXPORT queries is: (partcol1="val1", partcol2="val2" ...). I verified
that the latter format is a super-set, i.e. specifying (partcol1="val1",
partcol2="val2" ...) works for all these queries. That is the format generated
by EL dataInPartitions() in this patch as well.
Regarding partial/full partition spec, even the existing EL dataInPartitionSpec
writes all partition cols specified in HCat URI for dataset. The same approach
is taken for EL dataInPartitions() introduced in this patch.
[~rohini], can you provide your comments here?
> Add new coord EL function to get input partitions value string
> --------------------------------------------------------------
>
> Key: OOZIE-1741
> URL: https://issues.apache.org/jira/browse/OOZIE-1741
> Project: Oozie
> Issue Type: Improvement
> Reporter: Satish Mittal
> Assignee: Satish Mittal
> Attachments: OOZIE-1741-v2.patch, OOZIE-1741-v3.patch,
> OOZIE-1741.patch.txt
>
>
> Currently oozie provides coord EL functions to get partition filter string
> (where key-values are separated by AND) for various action types -
> java/pig/hive. However this doesn't work in other cases, e.g. hive action
> that performs export/import of hive partition. In that case, the partition
> value is expected to be in a different format:
> (col1="value1",col2="value2"...). We need to add a new EL function that
> returns partition value in the above format.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
