[jira] [Updated] (ATLAS-4348) Atlas-Kafka Hook : When a producer publishes messages to multiple topics, the latest relationship is marked ACTIVE , rest are marked DELETED
[
https://issues.apache.org/jira/browse/ATLAS-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Radhika Kundam updated ATLAS-4348:
--
Description:
*Steps to reproduce the issue*:
Created 2 topics : test1, test2
Using console consumer produced messages to test1 and test2.
Relationship between console-producer-lineage and test1 is ACTIVE now.
Relationship between console-producer-lineage and test1 is set to DELETED when
messages are published to test2.
*Root cause of issue:*
When messages published on topic, Atlas hook receives partial_update request
with only one topic as output and by default partial update deletes
relationships between previous topics by considering the only topic(which is in
partial_update request) as the active.
*Proposed Solution:*
Introducing typedef option "appendRelationshipsOnPartialUpdate" which will hold
list of relationship attribute names. While Atlas startup, it will resolve
entity type definition and all the specified relationship attributes
definitions will be updated with typedef property "isAppendOnPartialUpdate"
only for that entity type.
With this solution, we can keep all the previous relationship entries active
along with latest relationship entry for the specific entity by providing
typedef options as below.
"typeDefOptions": { "appendRelationshipsOnPartialUpdate": "[\"inputs\",
\"outputs\"]"}
This solution can be used for any entity type to consider all the existing
relationship entries as active along with new entries from partial_update.
was:
*Steps to reproduce the issue*:
Created 2 topics : test1, test2
Using console consumer produced messages to test1 and test2.
Relationship between console-producer-lineage and test1 is ACTIVE now.
Relationship between console-producer-lineage and test1 is set to DELETED when
messages are published to test2.
*Root cause of issue:*
When messages published on topic, Atlas hook receives partial_update request
with only one topic as output and by default partial update deletes
relationships between previous topics by considering the only topic(which is in
partial_update request) as the active.
*Proposed Solution:*
Introducing typedef option "appendRelationshipsOnPartialUpdate" which will hold
list of relationship attribute names. While Atlas startup, it will resolve
entity type definition and all the specified relationship attributes
definitions will be updated with typedef property "isAppendOnPartialUpdate"
only for that entity type.
With this solution, we can keep all the previous relationship entries active
along with latest relationship entry for the specific entity by providing
typedef options as below.
"typeDefOptions": \{ "appendRelationshipsOnPartialUpdate": "[\"inputs\",
\"outputs\"]"}
> Atlas-Kafka Hook : When a producer publishes messages to multiple topics, the
> latest relationship is marked ACTIVE , rest are marked DELETED
>
>
> Key: ATLAS-4348
> URL: https://issues.apache.org/jira/browse/ATLAS-4348
> Project: Atlas
> Issue Type: Task
> Components: atlas-core
>Affects Versions: 2.1.0
>Reporter: Radhika Kundam
>Assignee: Radhika Kundam
>Priority: Major
>
> *Steps to reproduce the issue*:
> Created 2 topics : test1, test2
> Using console consumer produced messages to test1 and test2.
> Relationship between console-producer-lineage and test1 is ACTIVE now.
> Relationship between console-producer-lineage and test1 is set to DELETED
> when messages are published to test2.
> *Root cause of issue:*
> When messages published on topic, Atlas hook receives partial_update request
> with only one topic as output and by default partial update deletes
> relationships between previous topics by considering the only topic(which is
> in partial_update request) as the active.
> *Proposed Solution:*
> Introducing typedef option "appendRelationshipsOnPartialUpdate" which will
> hold list of relationship attribute names. While Atlas startup, it will
> resolve entity type definition and all the specified relationship attributes
> definitions will be updated with typedef property "isAppendOnPartialUpdate"
> only for that entity type.
> With this solution, we can keep all the previous relationship entries active
> along with latest relationship entry for the specific entity by providing
> typedef options as below.
> "typeDefOptions": { "appendRelationshipsOnPartialUpdate": "[\"inputs\",
> \"outputs\"]"}
>
> This solution can be used for any entity type to consider all the existing
> relationship entries as active along with new entries from partial_update.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-4348) Atlas-Kafka Hook : When a producer publishes messages to multiple topics, the latest relationship is marked ACTIVE , rest are marked DELETED
Radhika Kundam created ATLAS-4348:
-
Summary: Atlas-Kafka Hook : When a producer publishes messages to
multiple topics, the latest relationship is marked ACTIVE , rest are marked
DELETED
Key: ATLAS-4348
URL: https://issues.apache.org/jira/browse/ATLAS-4348
Project: Atlas
Issue Type: Task
Components: atlas-core
Affects Versions: 2.1.0
Reporter: Radhika Kundam
Assignee: Radhika Kundam
*Steps to reproduce the issue*:
Created 2 topics : test1, test2
Using console consumer produced messages to test1 and test2.
Relationship between console-producer-lineage and test1 is ACTIVE now.
Relationship between console-producer-lineage and test1 is set to DELETED when
messages are published to test2.
*Root cause of issue:*
When messages published on topic, Atlas hook receives partial_update request
with only one topic as output and by default partial update deletes
relationships between previous topics by considering the only topic(which is in
partial_update request) as the active.
*Proposed Solution:*
Introducing typedef option "appendRelationshipsOnPartialUpdate" which will hold
list of relationship attribute names. While Atlas startup, it will resolve
entity type definition and all the specified relationship attributes
definitions will be updated with typedef property "isAppendOnPartialUpdate"
only for that entity type.
With this solution, we can keep all the previous relationship entries active
along with latest relationship entry for the specific entity by providing
typedef options as below.
"typeDefOptions": \{ "appendRelationshipsOnPartialUpdate": "[\"inputs\",
\"outputs\"]"}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ATLAS-4039) Audit entry for export operation displays param information which is incorrect
[
https://issues.apache.org/jira/browse/ATLAS-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mandar Ambawane reassigned ATLAS-4039:
--
Assignee: Prasad P. Pawar (was: Mandar Ambawane)
> Audit entry for export operation displays param information which is incorrect
> --
>
> Key: ATLAS-4039
> URL: https://issues.apache.org/jira/browse/ATLAS-4039
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Reporter: Dharshana M Krishnamoorthy
>Assignee: Prasad P. Pawar
>Priority: Major
> Attachments: Screenshot 2020-11-18 at 6.21.04 PM.png,
> image-2020-11-18-18-35-32-867.png
>
>
> On expanding the audit report of EXPORT operation *params(1)* is being
> displayed for all the exports irrespective of no/many options used in the
> exporting
> This is a UI issue and the response via rest call works fine.
> !image-2020-11-18-18-35-32-867.png|width=756,height=216!
> Sample payload that can be used to test
> {code}
> |{
> "itemsToExport": [
> \{ "typeName": "hive_db", "uniqueAttributes": { "qualifiedName":
> "" } },
> \{ "typeName": "hdfs_path", "uniqueAttributes": { "qualifiedName":
> "" } }
> ]
> }|
> |{
> "itemsToExport": [
> \{ "typeName": "hive_db", "uniqueAttributes": { "qualifiedName":
> "" } },
> \{ "typeName": "hdfs_path", "uniqueAttributes": { "qualifiedName":
> "" } }
> ],
> "options": {
> "fetchType": "FULL",
> "matchType": "startsWith"
> }
> }|
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (ATLAS-4196) [Atlas: Audits] Audits are created for the (n-1) th operation in atlas while performing an insert operation
[ https://issues.apache.org/jira/browse/ATLAS-4196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mandar Ambawane resolved ATLAS-4196. Resolution: Resolved > [Atlas: Audits] Audits are created for the (n-1) th operation in atlas while > performing an insert operation > --- > > Key: ATLAS-4196 > URL: https://issues.apache.org/jira/browse/ATLAS-4196 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Dharshana M Krishnamoorthy >Assignee: Mandar Ambawane >Priority: Major > Attachments: Screenshot 2021-03-12 at 11.26.45 AM.png, Screenshot > 2021-03-12 at 11.27.04 AM.png > > > While inserting data into table, an audit entry is created but it displays > info of the n-1 th operation. > Eg: When the first insert operation is performed with 3 rows of data, the > audit entry shows 0 rows . > Now insert 2 more rows in 1 go, the audit data displays 3 rows instead of 5 > Latest audit and latest data in the tabled are shared > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4161) basic/quick search on system attribute Classification(s) is_null returns few entities with classification too
[ https://issues.apache.org/jira/browse/ATLAS-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pinal resolved ATLAS-4161. -- Resolution: Fixed > basic/quick search on system attribute Classification(s) is_null returns few > entities with classification too > -- > > Key: ATLAS-4161 > URL: https://issues.apache.org/jira/browse/ATLAS-4161 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Umesh Padashetty >Assignee: Pinal >Priority: Major > Attachments: (Type _ALL_ENTITY_TYPES (AND (Classification(s) is_null > )) ) AND ( Query table_wtotg).png, (hive_table (AND (Classification(s) > is_null )) ) AND (Query table_wtotg).png > > > Results for: ( Type: hive_table (AND (Classification(s) is_null )) ) AND ( > Query: table_wtotg ) > !(hive_table (AND (Classification(s) is_null )) ) AND (Query > table_wtotg).png|width=1067,height=392! > Results for: ( Type: _ALL_ENTITY_TYPES (AND (Classification(s) is_null )) ) > AND ( Query: table_wtotg ) > !(Type _ALL_ENTITY_TYPES (AND (Classification(s) is_null )) ) AND ( Query > table_wtotg).png|width=925,height=345! > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4161) basic/quick search on system attribute Classification(s) is_null returns few entities with classification too
[ https://issues.apache.org/jira/browse/ATLAS-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17376467#comment-17376467 ] Pinal commented on ATLAS-4161: -- This behaviour is due to tag present in 'Propogated Classification' but not in 'Classification'. The 'Classifications' tab on UI, is combination of both 'Classification' and 'Propogated Classification'. > basic/quick search on system attribute Classification(s) is_null returns few > entities with classification too > -- > > Key: ATLAS-4161 > URL: https://issues.apache.org/jira/browse/ATLAS-4161 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Umesh Padashetty >Assignee: Pinal >Priority: Major > Attachments: (Type _ALL_ENTITY_TYPES (AND (Classification(s) is_null > )) ) AND ( Query table_wtotg).png, (hive_table (AND (Classification(s) > is_null )) ) AND (Query table_wtotg).png > > > Results for: ( Type: hive_table (AND (Classification(s) is_null )) ) AND ( > Query: table_wtotg ) > !(hive_table (AND (Classification(s) is_null )) ) AND (Query > table_wtotg).png|width=1067,height=392! > Results for: ( Type: _ALL_ENTITY_TYPES (AND (Classification(s) is_null )) ) > AND ( Query: table_wtotg ) > !(Type _ALL_ENTITY_TYPES (AND (Classification(s) is_null )) ) AND ( Query > table_wtotg).png|width=925,height=345! > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ATLAS-4161) basic/quick search on system attribute Classification(s) is_null returns few entities with classification too
[ https://issues.apache.org/jira/browse/ATLAS-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pinal reassigned ATLAS-4161: Assignee: Pinal > basic/quick search on system attribute Classification(s) is_null returns few > entities with classification too > -- > > Key: ATLAS-4161 > URL: https://issues.apache.org/jira/browse/ATLAS-4161 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Umesh Padashetty >Assignee: Pinal >Priority: Major > Attachments: (Type _ALL_ENTITY_TYPES (AND (Classification(s) is_null > )) ) AND ( Query table_wtotg).png, (hive_table (AND (Classification(s) > is_null )) ) AND (Query table_wtotg).png > > > Results for: ( Type: hive_table (AND (Classification(s) is_null )) ) AND ( > Query: table_wtotg ) > !(hive_table (AND (Classification(s) is_null )) ) AND (Query > table_wtotg).png|width=1067,height=392! > Results for: ( Type: _ALL_ENTITY_TYPES (AND (Classification(s) is_null )) ) > AND ( Query: table_wtotg ) > !(Type _ALL_ENTITY_TYPES (AND (Classification(s) is_null )) ) AND ( Query > table_wtotg).png|width=925,height=345! > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4158) Atlas authorization for Add/Update/Remove classification on entities.
[ https://issues.apache.org/jira/browse/ATLAS-4158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nixon Rodrigues resolved ATLAS-4158. Fix Version/s: 2.2.0 3.0.0 Resolution: Fixed > Atlas authorization for Add/Update/Remove classification on entities. > - > > Key: ATLAS-4158 > URL: https://issues.apache.org/jira/browse/ATLAS-4158 > Project: Atlas > Issue Type: Bug >Reporter: Nixon Rodrigues >Assignee: Nixon Rodrigues >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > *Requirement* > The new requirement is to provide a way to authorize who can > Add/Remove/Update Classification for an entity even if the entities on which > classification have to be applied do not have classifications already tagged > to it. > > e.g. Any users belonging to user group - 'finance_group' can add > classification - FINANCE_PII to any entities (these entities may not have > any classifications associated) if policy has given access to the user or > user-group. -- This message was sent by Atlassian Jira (v8.3.4#803005)
