[
https://issues.apache.org/jira/browse/LENS-796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepak Barr updated LENS-796:
-----------------------------
Attachment: LENS-796.patch
> Slow response times for /metastore/nativetables API
> ---------------------------------------------------
>
> Key: LENS-796
> URL: https://issues.apache.org/jira/browse/LENS-796
> Project: Apache Lens
> Issue Type: Improvement
> Reporter: Deepak Barr
> Assignee: Deepak Barr
> Attachments: LENS-796.patch
>
>
> To obtain list of native tables, CubeMetastoreService does the following -
> 1. Fetches the list of tables ( one MetastoreClient call)
> 2. Filters out the cube tables from the list. The filtering happens by
> looking at the table properties from the Table object. This table object is
> obtained with another Metastore call. So, If there are 'n' tables, there will
> be 'n' metastore calls.
> Here is the code snippet :
> private List<String> getTablesFromDB(LensSessionHandle sessionid,
> String dbName, boolean prependDbName)
> throws MetaException, UnknownDBException, HiveSQLException, TException,
> LensException {
> List<String> tables =
> getSession(sessionid).getMetaStoreClient().getAllTables(
> dbName);
> List<String> result = new ArrayList<String>();
> if (tables != null && !tables.isEmpty()) {
> Iterator<String> it = tables.iterator();
> while (it.hasNext()) {
> String tblName = it.next();
> org.apache.hadoop.hive.metastore.api.Table tbl =
> getSession(sessionid).getMetaStoreClient().getTable(dbName,
> tblName);
> if (tbl.getParameters().get(MetastoreConstants.TABLE_TYPE_KEY) ==
> null) {
> if (prependDbName) {
> result.add(dbName + "." + tblName);
> } else {
> result.add(tblName);
> }
> }
> }
> }
> return result;
> }
> Instead of this, we can directly fetch the list of table objects for our list
> of table names in a single API call using
> getMetaStoreClient().getTableObjectsByName() method.
> Currently, one of our databases contain 8000+ tables which leads to very very
> long response times.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
