GitHub user jinfengni opened a pull request:
https://github.com/apache/drill/pull/286
Drill 4127: Reduce Hive metastore client API call in HiveSchema
Also, it has commit for DRILL-4126: Add cache to HiveSchema in order to
reduce long planning time or execution time caused by slow Hive meta store.
Both DRILL-4127 and DRILL-4126 address the long delay caused by slow hive
meta store.
Passed unit, pre-commit regression, and additional impersonation test,
before rebasing onto latest master.
Will re-run the above tests.
@vkorukanti , could you please review the two patches? Thanks.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jinfengni/incubator-drill DRILL-4127
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/286.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #286
----
commit 19a5a4d1c9c23eedcb94c988bd2229680575a118
Author: Jinfeng Ni <[email protected]>
Date: 2015-11-19T04:18:51Z
DRILL-4127: Reduce Hive metastore client API call in HiveSchema.
1) Use lazy loading of tableNames in HiveSchema, in stead of pre-loading
all table names under each HiveSchema.
2) Do not call get_all_databases for subSchema to check existence if the
name comes from getSubSchemaNames() directly.
commit 9570319c227649144d3a14f8d5774fbe4a282bc4
Author: Jinfeng Ni <[email protected]>
Date: 2015-11-30T04:15:07Z
DRILL-4126: Add cache to HiveSchema in order to reduce long planning time
or execution time caused by slow Hive meta store.
1) HiveSchema caching will help in case impersonation is enabled.
2) Use flat level cache for tables in DrillHiveMetaStoreClient.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
