[
https://issues.apache.org/jira/browse/ATLAS-4501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Mestry reassigned ATLAS-4501:
--------------------------------------
Assignee: Radhika Kundam
> Table 'Allow list' with a default deny to load only a subset of tables
> ----------------------------------------------------------------------
>
> Key: ATLAS-4501
> URL: https://issues.apache.org/jira/browse/ATLAS-4501
> Project: Atlas
> Issue Type: New Feature
> Components: atlas-core, atlas-intg, hive-integration
> Reporter: Adriano
> Assignee: Radhika Kundam
> Priority: Major
>
> There are some huge environments where the warehouse has a thousand databases
> and hundred thousand tables with many columns and most of them are dropped,
> created, updated at a fast pace. In these environments, the Atlas processing
> time can slow down increasing the backlog as it starts moving slower than the
> changes in the warehouse and the {{prune.pattern}} e/o {{ignore.pattern}} it
> is not suitable.
> It will be nice to have the opportunity to have a default deny behaviour for
> all the tables and then to 'allow' the import of a subset of tables specified
> in a parameter regex (in order to process only some important tables):
> basically that works in the opposite way to the {{prune.pattern}} and
> {{ignore.pattern.}}
> As far as I know, there is a similar feature for S3 and ADLS but not for hive.
> If this is the case, will be nice to get the feature onboarded in your
> backlog.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
