Barbara Eckman created RANGER-4298:
--------------------------------------
Summary: Multi-threaded tagsync needed
Key: RANGER-4298
URL: https://issues.apache.org/jira/browse/RANGER-4298
Project: Ranger
Issue Type: Improvement
Components: Ranger, tagsync
Reporter: Barbara Eckman
We are in the process of migrating our entire large data lakes (S3, MinIO) from
resource-based to tag-based AC. This migration is phased and estimated to
continue through Q3. In this phase we need to import 250K tags into Ranger
from Apache Atlas. At the current, single-threaded rate of 500ms/tag, this
will take 6 to 7 days.
Beyond this migration, we expect to be incorporating multiple external datasets
into our datalakes, which will cause roughly the same quantity of tags to be
imported.
So this is not a one-time thing, it is expected to recur. It would be great if
tagsync could be made multi-threaded.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
