----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73170/#review222555 -----------------------------------------------------------
Ship it! Ship It! - Sarath Subramanian On Feb. 3, 2021, 6:10 p.m., Deep Singh wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73170/ > ----------------------------------------------------------- > > (Updated Feb. 3, 2021, 6:10 p.m.) > > > Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Rajesh > Balamohan, and Sarath Subramanian. > > > Bugs: ATLAS-4127 > https://issues.apache.org/jira/browse/ATLAS-4127 > > > Repository: atlas > > > Description > ------- > > In Atlas Hive bridge, while calculating processQualifiedName, all partitions > are processed. > For each partition, a hive call is made to get table details. Since multiple > partitions can belong to the same table, there is a scope for improvement by > avoiding repeated hive calls for the same table information. > This improvement becomes crucial particularly in the case where there are > several partitions for a single table. > > In DWX, query to populate store_sales takes 15 seconds to compile. 10+ > seconds in just Atlas hook for getting table details. > > > Diffs > ----- > > > addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java > 2b809a637 > > > Diff: https://reviews.apache.org/r/73170/diff/1/ > > > Testing > ------- > > > Thanks, > > Deep Singh > >
