And what do you mean lineage search for dag ? I mean user can find ins and outs
of the sql by table name.
For example:
INSERT overwrite TABLE TESTTMP.cust_sku_vis_detl1 SELECT
......
FROM
DW.fct_ordr_detl detl
JOIN DW.fct_ordr ordr ON detl.ordr_id = ordr.ordr_id
AND detl.ds = ordr.ds
JOIN DW.dim_prod prod ON detl.prod_id = prod.prod_id
AND prod.cur_flag = 1
JOIN DW.hier_categ categ ON prod.categ_lvl_id = categ.categ_lvl_id
AND categ.cur_flag = 1
WHERE
detl.ds = '2015-06-01'
AND ordr.ds = '2015-06-01'
GROUP BY
ordr.ds,
ordr.end_user_id,
categ.categ_lvl1_id,
categ.categ_lvl1_name
Tez UI can pick out some sql (dag graph) when user search for
TESTTMP.cust_sku_vis_detl1.
[email protected]
From: Jeff Zhang
Date: 2015-06-12 08:38
To: user
Subject: Re: some advice to tez ui
This might need hive add some app specific info on the dag/vertex. Actually Tez
don't know the concept of table, tez use InputDescriptor/OutputDescriptor to
represent the Input/Output which is not restricted to hive table, it could be
any data source.
And what do you mean lineage search for dag ?
On Fri, Jun 12, 2015 at 8:23 AM, [email protected] <[email protected]> wrote:
I think tez ui can add some feature like data governance capabilities such as
- which source table generate the result table,provide lineage search for dag.
[email protected]
--
Best Regards
Jeff Zhang
