[
https://issues.apache.org/jira/browse/SENTRY-2225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471348#comment-16471348
]
Brian Towles commented on SENTRY-2225:
--------------------------------------
The default llocation we can get attributes from can be HMS rather then a
static file.
HMS has the ability to do Table Properties as a free form key value store. We
can very easily set table and column level attributes in the properties using a
name-spaced approach.
e.g.
sentry.attributes.table.tag = Foo // table level "tag" attribute
sentry.attributes.column.ccnum.label = "PCI,PII" // column level "*label*"
attribute for ccnum column
sentry.attributes.column.ccnum.content-descriptor = "credit_card_number" //
column level "content-descriptor" attribute for ccnum column
The current structure for TABLE_PARAMS in HMS is
{noformat}
`TBL_ID` bigint(20) NOT NULL,
`PARAM_KEY` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
`PARAM_VALUE` mediumtext CHARACTER SET latin1 COLLATE latin1_bin DEFAULT
NULL,{noformat}
So there is a very large amount of available space for nested structures (like
JSON) as a param value if necessary.
Entry of these properties is easy and done right in SQL with commands like:
*ALTER TABLE table_name SET TBLPROPERTIES ("propertykey" = "property value");*
In the plugin we will be able to respond to HMS changed events from HMSFollower
and populate out the attributes as needed. Or it could implement its own
retrieval method for attributes based on changes happening in HMS, push or
pull. HMS Follower could send notifications of objects changed. Right now it
uses PubSub to do, but that only sends a string. We could augment or replace
with guavas EventBus and have full EventObject passing internal to Sentry. The
messages that HMSFollower gets from include changes to the Table Properties
when they occur and can be used to update attributes on creates, changes, or
removals.
> Generic Attribute Ingestion and Default Implementation
> ------------------------------------------------------
>
> Key: SENTRY-2225
> URL: https://issues.apache.org/jira/browse/SENTRY-2225
> Project: Sentry
> Issue Type: Sub-task
> Components: Core
> Affects Versions: 2.1.0
> Reporter: Anthony Young-Garner
> Priority: Major
> Labels: ABAC
>
> As discussed in the design document linked on SENTRY-2140, attributes and
> their mapping to columns are created and stored in an external system. In
> order for Sentry to make masking decisions based upon these attributes and
> mappings, this information must be ingested from the external system. The
> scope of this Jira is to :
> # implement the generic extensible framework by which different external
> systems may contribute attributes (the specific details of the design are
> still under discussion on the parent Jira; whether there is a full plugin
> model implemented in Sentry or whether the ingestion process will run
> entirely external to Sentry and send the information to Sentry via Thrift API
> is not yet decided).
> # Implement at least one default implementation (whether this will be an
> example implementation only for reference like a static text file or a
> full-featured implementation more suitable for production use is under
> discussion)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
