[
https://issues.apache.org/jira/browse/LENS-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367103#comment-14367103
]
Amareshwari Sriramadasu commented on LENS-418:
----------------------------------------------
On the metastore service we have partitions under
/facts/{factName}/storages/{storage}/partitions and
/dimtables/{dimTableName}/storages/{storage}/partitions - With this resource
binding, partitions are always tied to fact in a storage, dimtable in a
storage. If we go with this, even add/drop partitions will happen on
fact/dimtable on a storage.
What is put in the jira is whether we allow add/partitions to be across
facts/dimtables and storages.
I would say no user will be registering partitions across facts/dimtables and
storages because usually the ingestion pipelines will be adding partitions for
one fact or one dimtable on one storage at a time. But open to look at other
options here.
bq. Fact name is mentioned in command as well as the xml. What will happen if
both are different?
We can remove usage of redundant fact_or_dimension_table_name from XPartition
with current registration model.
> Design issues in add partitions(s)
> ----------------------------------
>
> Key: LENS-418
> URL: https://issues.apache.org/jira/browse/LENS-418
> Project: Apache Lens
> Issue Type: Bug
> Reporter: Rajat Khandelwal
>
> cli command is:
> {noformat}
> fact add partitions factname storagename filepath
> {noformat}
> example partition list is:
> {noformat}
> <x_partition_list xmlns="uri:lens:cube:0.1"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xsi:schemaLocation="uri:lens:cube:0.1 cube-0.1.xsd ">
> <partition fact_or_dimension_table_name="fact1" location="/tmp/fact1"
> update_period="HOURLY"
> xmlns="uri:lens:cube:0.1"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
> xsi:schemaLocation="uri:lens:cube:0.1 cube-0.1.xsd ">
> <time_partition_spec>
> <part_spec_element key="dt" value="2014-03-27T12:00:00"/>
> </time_partition_spec>
> </partition>
> </x_partition_list>
> {noformat}
> Fact name is mentioned in command as well as the xml. What will happen if
> both are different?
> In my opinion, the design should be the following:
> command:
> {noformat}
> add partitions filepath
> {noformat}
> file will contain x_partition_list. which will contain a lot of `partition`s
> which can belong to different fact/dimtable and storages.
> Thoughts?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
