Na Li created SENTRY-1898:
-----------------------------
Summary: Sentry no longer supports creating more than ~15
partitions at once
Key: SENTRY-1898
URL: https://issues.apache.org/jira/browse/SENTRY-1898
Project: Sentry
Issue Type: Bug
Components: Sentry
Affects Versions: 2.0.0
Reporter: Na Li
Assignee: Na Li
Priority: Blocker
Vamsee reported: I notice this problem on our stress cluster where we are
creating a huge amounts of Hive metadata (Databases, Tables, and Partitions).
During this process, multiple partitions of the table are being created at one
go using:
ALTER TABLE <TB> ADD PARTITION <PART1> ADD PARTITION <PART2>...
I notice that when we add more than 20 partitions here in one sql statement,
Sentry fails to process the transaction as we cross the 4000 character
threshold on PATH change table.
Every partition entry is of format:
{"1":{"str":"test_hivemd_db_2017_08_15_08_14_27_2_0_8.test_hivemd_table_2017_08_15_08_14_27_2_0_0"},"2":{"lst":["lst",1,["str",6,"user","hive","warehouse","test_hivemd_db_2017_08_15_08_14_27_2_0_8.db","test_hivemd_table_2017_08_15_08_14_27_2_0_0","part=test_hivemd_partition_2017_08_15_08_14_27_2_0_1"]]},"3":{"lst":["lst",0]}}
~ 327 characters
So to fit in 4000 characters, we are limited to 10-15 partitions to be created
at one go.
We should allow the big path update from hive.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
