Current data model is one to many for facts and storage tables. What you're suggesting is a many to many model. We are also having thoughts about revamping the model. For example, storing model as a list of key Value pair. This needs a deeper discussion. Should we create a jira and discuss this?
On Wed, Apr 15, 2015, 17:23 Pranav Kumar Agarwal <[email protected]> wrote: > Hi All, > > Consider following storage table defn inside a fact: > > <x_fact_table cube_name="sample_cube" name="fact2" weight="200.0" > xmlns="uri:lens:cube:0.1" > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; > xsi:schemaLocation="uri:lens:cube:0.1 cube-0.1.xsd "> > ... > <storage_tables> > <storage_table> > <update_periods> > <update_period>HOURLY</update_period> > <update_period>DAILY</update_period> > </update_periods> > <storage_name>local</storage_name> > <table_desc external="true" field_delimiter="," > table_location="/tmp/examples/fact2_local"> > <part_cols> > <column comment="Time column" name="dt" type="STRING"/> > </part_cols> > <time_part_cols>dt</time_part_cols> > </table_desc> > </storage_table> > </storage_tables> > </x_fact_table> > > In an event a new partition is added to the external table location > "/tmp/examples/fact2_local" then I wish to add a new partition on the > fact2, however I have no way to find what all facts are built on > external table location "/tmp/examples/fact2_local". We can possibly do > it by matching the location, however that doesn't seem to be quite > nice.. Consider for a JDBC source the table_location is kind of dummy as > its not really used to query the content from that location. JDBCDriver > expects a table with the name as storageName_factName in the target > datastore, thus no indication on which all facts to be updated. > > Problem Statement: Current storage_table definition doesn't give me > enough detail to find where all partitions needs to be added given a new > partition is added to an external table. > > I propose that we add a table_name property as part of table_desc and > provide following API's: > GET /storages/{storageName}/tableNames/ > GET /storages/{storageName}/tableNames/{tableName}/facts > > Thoughts? > > Regards, > -Pranav. > > >
