Arjun Mishra created SENTRY-2401:
------------------------------------
Summary: FullUpdateInitializer should do a single fetch from HMS
instead of 1 per task
Key: SENTRY-2401
URL: https://issues.apache.org/jira/browse/SENTRY-2401
Project: Sentry
Issue Type: New Feature
Components: Sentry
Reporter: Arjun Mishra
Assignee: Arjun Mishra
This is an optimization ticket. Currently the way we do fetch is
# Get all database names from HMS. Create a Db Task for each database name. HMS
Request = 1.
# For each database task
## Get all database objects from HMS
## Get all table names, for each database
## Create a table task for "maxTablesPerCall" tables
# For each table task
## For each table task get all table objects
## For each "maxTablesPerCall" tables fetched, get partition names
## For each ""maxTablesPerCall" tables fetched, created partition tasks for
"maxPartitionsPerCall" partitions
# For each partition task
## Get all partition objects for, database, table, and list of partition names
Instead of splitting connecting to HMS and getting bits of information, it is
better to get them all in a single request
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
