[jira] [Commented] (ATLAS-3053) Update notification processing to propagate classifications only when specified
[ https://issues.apache.org/jira/browse/ATLAS-3053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773478#comment-16773478 ] ASF subversion and git services commented on ATLAS-3053: Commit e322dfca4bcfbc5688f554537e50ced7572cf535 in atlas's branch refs/heads/branch-1.0 from Madhan Neethiraj [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=e322dfc ] ATLAS-3053: updated notification processing to propagate classifications only when specified (cherry picked from commit fe1c299ce4387138ec82b0824415949daa93fa6f) > Update notification processing to propagate classifications only when > specified > --- > > Key: ATLAS-3053 > URL: https://issues.apache.org/jira/browse/ATLAS-3053 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 1.0.0, 1.1.0 >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj >Priority: Major > Fix For: 1.2.0, 2.0.0 > > Attachments: ATLAS-3053.patch > > > When a notification results in creation of an entity, Atlas saves the > classifications given in the notification (as entity didn’t exist in Atlas, > this is not considered as an update to classifications). Atlas should > propagate the classifications added here only when "isPropagate" flag is set > to 'true'. In other cases, the classifications should not be propagated. > Please note that notifications are meant to ingest metadata from source > systems, and not an interface to add/update business metadata - such as > classifications. Add/update/remove of classifications via notifications is > not supported - except for the new entities created while processing > notifications. If a notification results in update to an existing entity, > Atlas ignores classifications in the notification. Changes to classifications > must be done via appropriate REST API calls. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-3053) Update notification processing to propagate classifications only when specified
[ https://issues.apache.org/jira/browse/ATLAS-3053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773471#comment-16773471 ] ASF subversion and git services commented on ATLAS-3053: Commit fe1c299ce4387138ec82b0824415949daa93fa6f in atlas's branch refs/heads/master from Madhan Neethiraj [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=fe1c299 ] ATLAS-3053: updated notification processing to propagate classifications only when specified > Update notification processing to propagate classifications only when > specified > --- > > Key: ATLAS-3053 > URL: https://issues.apache.org/jira/browse/ATLAS-3053 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 1.0.0, 1.1.0 >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj >Priority: Major > Fix For: 1.2.0, 2.0.0 > > Attachments: ATLAS-3053.patch > > > When a notification results in creation of an entity, Atlas saves the > classifications given in the notification (as entity didn’t exist in Atlas, > this is not considered as an update to classifications). Atlas should > propagate the classifications added here only when "isPropagate" flag is set > to 'true'. In other cases, the classifications should not be propagated. > Please note that notifications are meant to ingest metadata from source > systems, and not an interface to add/update business metadata - such as > classifications. Add/update/remove of classifications via notifications is > not supported - except for the new entities created while processing > notifications. If a notification results in update to an existing entity, > Atlas ignores classifications in the notification. Changes to classifications > must be done via appropriate REST API calls. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 70027: ATLAS-3053: updated notification processing to propagate classifications only when specified
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70027/#review212993 --- Ship it! Ship It! - Sarath Subramanian On Feb. 20, 2019, 2:16 p.m., madhan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70027/ > --- > > (Updated Feb. 20, 2019, 2:16 p.m.) > > > Review request for atlas, Ashutosh Mestry, keval bhatt, Nixon Rodrigues, and > Sarath Subramanian. > > > Bugs: ATLAS-3053 > https://issues.apache.org/jira/browse/ATLAS-3053 > > > Repository: atlas > > > Description > --- > > Updated notification processing to set isPropagate=false if no value is > specified > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java > a260154c9 > server-api/src/main/java/org/apache/atlas/RequestContext.java f12742472 > > webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java > bd5477051 > > > Diff: https://reviews.apache.org/r/70027/diff/1/ > > > Testing > --- > > Pre-commit tests run: > https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/929/ > > > Thanks, > > madhan > >
Review Request 70027: ATLAS-3053: updated notification processing to propagate classifications only when specified
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70027/ --- Review request for atlas. Bugs: ATLAS-3053 https://issues.apache.org/jira/browse/ATLAS-3053 Repository: atlas Description --- Updated notification processing to set isPropagate=false if no value is specified Diffs - repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java a260154c9 server-api/src/main/java/org/apache/atlas/RequestContext.java f12742472 webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java bd5477051 Diff: https://reviews.apache.org/r/70027/diff/1/ Testing --- Pre-commit tests run: https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/929/ Thanks, madhan
[jira] [Created] (ATLAS-3053) Update notification processing to propagate classifications only when specified
Madhan Neethiraj created ATLAS-3053: --- Summary: Update notification processing to propagate classifications only when specified Key: ATLAS-3053 URL: https://issues.apache.org/jira/browse/ATLAS-3053 Project: Atlas Issue Type: Improvement Components: atlas-core Affects Versions: 1.1.0, 1.0.0 Reporter: Madhan Neethiraj Assignee: Madhan Neethiraj When a notification results in creation of an entity, Atlas saves the classifications given in the notification (as entity didn’t exist in Atlas, this is not considered as an update to classifications). Atlas should propagate the classifications added here only when "isPropagate" flag is set to 'true'. In other cases, the classifications should not be propagated. Please note that notifications are meant to ingest metadata from source systems, and not an interface to add/update business metadata - such as classifications. Add/update/remove of classifications via notifications is not supported - except for the new entities created while processing notifications. If a notification results in update to an existing entity, Atlas ignores classifications in the notification. Changes to classifications must be done via appropriate REST API calls. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-3052) metrics query results caching - to be disabled
[ https://issues.apache.org/jira/browse/ATLAS-3052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773417#comment-16773417 ] ASF subversion and git services commented on ATLAS-3052: Commit 0e4a2f696bd9c72b3b026fa0a28eceb1fcf5c2e5 in atlas's branch refs/heads/branch-1.0 from Nikhil Bonte [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=0e4a2f6 ] ATLAS-3052: removed caching of metrics query results Change-Id: Ifeabb13acb387781a72d00146d4acd4ea9a279d8 Signed-off-by: Madhan Neethiraj (cherry picked from commit ee99930f5203ac321a5da41b9e3ab63df98e2faf) > metrics query results caching - to be disabled > -- > > Key: ATLAS-3052 > URL: https://issues.apache.org/jira/browse/ATLAS-3052 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 1.2.0, 2.0.0 >Reporter: Madhan Neethiraj >Priority: Major > > Metrics API introduced in ATLAS-1436 caches the results to provide quick > response and avoid frequent execution of expensive queries. Improvements > introduced in ATLAS-2959 result in much quicker execution of metrics queries, > hence this caching is no more needed. Removing this caching will help to get > recent metrics quickly i.e. without having to wait for cache rebuild. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-3052) metrics query results caching - to be disabled
[ https://issues.apache.org/jira/browse/ATLAS-3052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773414#comment-16773414 ] ASF subversion and git services commented on ATLAS-3052: Commit ee99930f5203ac321a5da41b9e3ab63df98e2faf in atlas's branch refs/heads/master from Nikhil Bonte [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=ee99930 ] ATLAS-3052: removed caching of metrics query results Change-Id: Ifeabb13acb387781a72d00146d4acd4ea9a279d8 Signed-off-by: Madhan Neethiraj > metrics query results caching - to be disabled > -- > > Key: ATLAS-3052 > URL: https://issues.apache.org/jira/browse/ATLAS-3052 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 1.2.0, 2.0.0 >Reporter: Madhan Neethiraj >Priority: Major > > Metrics API introduced in ATLAS-1436 caches the results to provide quick > response and avoid frequent execution of expensive queries. Improvements > introduced in ATLAS-2959 result in much quicker execution of metrics queries, > hence this caching is no more needed. Removing this caching will help to get > recent metrics quickly i.e. without having to wait for cache rebuild. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-3051) On container entity-delete, contained entities in composition relationship should be deleted
[ https://issues.apache.org/jira/browse/ATLAS-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773410#comment-16773410 ] ASF subversion and git services commented on ATLAS-3051: Commit 8ff1dbb4eeda7a9f4ee29de4ba0089828168a0df in atlas's branch refs/heads/branch-1.0 from Madhan Neethiraj [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=8ff1dbb ] ATLAS-3051: on container entity-delete, contained entities in composition relationship should be deleted Change-Id: I6a5c2d19cd2bc96a32050150e88108fa63623d97 (cherry picked from commit a0e4cd1aacb2047ff5af075a43ff9e38c49e2b85) > On container entity-delete, contained entities in composition relationship > should be deleted > > > Key: ATLAS-3051 > URL: https://issues.apache.org/jira/browse/ATLAS-3051 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 1.0.0, 1.1.0 >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj >Priority: Major > Fix For: 1.2.0, 2.0.0 > > Attachments: ATLAS-3051.patch > > > When a container entity is deleted, all its contained entities in composition > relationship should be deleted. For example, when a hive_db entity is deleted > all hive_table entities in the db should also be deleted - similar to delete > of ownedRef entities i.e. delete of hive_table entity deletes its associated > hive_storagedesc and hive_column entities. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-3051) On container entity-delete, contained entities in composition relationship should be deleted
[ https://issues.apache.org/jira/browse/ATLAS-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773406#comment-16773406 ] ASF subversion and git services commented on ATLAS-3051: Commit a0e4cd1aacb2047ff5af075a43ff9e38c49e2b85 in atlas's branch refs/heads/master from Madhan Neethiraj [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=a0e4cd1 ] ATLAS-3051: on container entity-delete, contained entities in composition relationship should be deleted Change-Id: I6a5c2d19cd2bc96a32050150e88108fa63623d97 > On container entity-delete, contained entities in composition relationship > should be deleted > > > Key: ATLAS-3051 > URL: https://issues.apache.org/jira/browse/ATLAS-3051 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 1.0.0, 1.1.0 >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj >Priority: Major > Fix For: 1.2.0, 2.0.0 > > Attachments: ATLAS-3051.patch > > > When a container entity is deleted, all its contained entities in composition > relationship should be deleted. For example, when a hive_db entity is deleted > all hive_table entities in the db should also be deleted - similar to delete > of ownedRef entities i.e. delete of hive_table entity deletes its associated > hive_storagedesc and hive_column entities. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 70019: ATLAS-3051: on container entity-delete, contained entities in composition relationship should be deleted
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70019/#review212979 --- Ship it! Ship It! - Sarath Subramanian On Feb. 20, 2019, 8:27 a.m., madhan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70019/ > --- > > (Updated Feb. 20, 2019, 8:27 a.m.) > > > Review request for atlas, Ashutosh Mestry, keval bhatt, Nixon Rodrigues, and > Sarath Subramanian. > > > Bugs: ATLAS-3051 > https://issues.apache.org/jira/browse/ATLAS-3051 > > > Repository: atlas > > > Description > --- > > - when a container attribute in COMPOSITION relationship is added to > AtlasEntityType, ownedRef constrinat is added, so that existing behaviour of > ownedRef attribute will be extended to these relationship attributes as well > - added validation for contained entities in COMPOSITION relationship, to > require container is set i.e. mandatory attribute > > > Diffs > - > > intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java > a4d6bedfb > intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 0d63325da > intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java > 264c4fc30 > > repository/src/main/java/org/apache/atlas/repository/ogm/glossary/AtlasGlossaryCategoryDTO.java > 92b8fafdb > > repository/src/main/java/org/apache/atlas/repository/ogm/glossary/AtlasGlossaryTermDTO.java > d1da4e0ac > > repository/src/main/java/org/apache/atlas/repository/ogm/profiles/AtlasSavedSearchDTO.java > 68a874402 > > repository/src/main/java/org/apache/atlas/repository/ogm/profiles/AtlasUserProfileDTO.java > 620b14825 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java > b2bd896b8 > > > Diff: https://reviews.apache.org/r/70019/diff/2/ > > > Testing > --- > > - verified that container entity delete causes its contained entities to be > deleted > - verified that entity-create requires container value be set for contined > entities > - pre-commit tests run: > https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/928/ > > > Thanks, > > madhan > >
[jira] [Commented] (ATLAS-3047) Ability to parse and persist avro schema in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-3047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773280#comment-16773280 ] Ayush Nigam commented on ATLAS-3047: In final stage of internal review,will add back once its done along with test cases > Ability to parse and persist avro schema in Atlas > - > > Key: ATLAS-3047 > URL: https://issues.apache.org/jira/browse/ATLAS-3047 > Project: Atlas > Issue Type: New Feature >Reporter: Ayush Nigam >Assignee: Ayush Nigam >Priority: Major > Labels: AVRO > > As part of our use case we have developed Avro Parser that can parse fully > nested avro schemas and persist them to atlas. > Also we have added capability to add field classifications and have done > enhancements on already existing avro modle to accomodate new types such as > avro_map,avro_union,logical types etc -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-3052) metrics query results caching - to be disabled
Madhan Neethiraj created ATLAS-3052: --- Summary: metrics query results caching - to be disabled Key: ATLAS-3052 URL: https://issues.apache.org/jira/browse/ATLAS-3052 Project: Atlas Issue Type: Improvement Components: atlas-core Affects Versions: 1.2.0, 2.0.0 Reporter: Madhan Neethiraj Assignee: Madhan Neethiraj Metrics API introduced in ATLAS-1436 caches the results to provide quick response and avoid frequent execution of expensive queries. Improvements introduced in ATLAS-2959 result in much quicker execution of metrics queries, hence this caching is no more needed. Removing this caching will help to get recent metrics quickly i.e. without having to wait for cache rebuild. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3051) On container entity-delete, contained entities in composition relationship should be deleted
[ https://issues.apache.org/jira/browse/ATLAS-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Madhan Neethiraj updated ATLAS-3051: Attachment: (was: ATLAS-3051.patch) > On container entity-delete, contained entities in composition relationship > should be deleted > > > Key: ATLAS-3051 > URL: https://issues.apache.org/jira/browse/ATLAS-3051 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 1.0.0, 1.1.0 >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj >Priority: Major > Fix For: 1.2.0, 2.0.0 > > Attachments: ATLAS-3051.patch > > > When a container entity is deleted, all its contained entities in composition > relationship should be deleted. For example, when a hive_db entity is deleted > all hive_table entities in the db should also be deleted - similar to delete > of ownedRef entities i.e. delete of hive_table entity deletes its associated > hive_storagedesc and hive_column entities. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-3051) On container entity-delete, contained entities in composition relationship should be deleted
[ https://issues.apache.org/jira/browse/ATLAS-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Madhan Neethiraj updated ATLAS-3051: Attachment: ATLAS-3051.patch > On container entity-delete, contained entities in composition relationship > should be deleted > > > Key: ATLAS-3051 > URL: https://issues.apache.org/jira/browse/ATLAS-3051 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 1.0.0, 1.1.0 >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj >Priority: Major > Fix For: 1.2.0, 2.0.0 > > Attachments: ATLAS-3051.patch > > > When a container entity is deleted, all its contained entities in composition > relationship should be deleted. For example, when a hive_db entity is deleted > all hive_table entities in the db should also be deleted - similar to delete > of ownedRef entities i.e. delete of hive_table entity deletes its associated > hive_storagedesc and hive_column entities. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: Review Request 70019: ATLAS-3051: on container entity-delete, contained entities in composition relationship should be deleted
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70019/ --- (Updated Feb. 20, 2019, 4:27 p.m.) Review request for atlas, Ashutosh Mestry, keval bhatt, Nixon Rodrigues, and Sarath Subramanian. Changes --- fix to address unit-test failures Bugs: ATLAS-3051 https://issues.apache.org/jira/browse/ATLAS-3051 Repository: atlas Description --- - when a container attribute in COMPOSITION relationship is added to AtlasEntityType, ownedRef constrinat is added, so that existing behaviour of ownedRef attribute will be extended to these relationship attributes as well - added validation for contained entities in COMPOSITION relationship, to require container is set i.e. mandatory attribute Diffs (updated) - intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java a4d6bedfb intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 0d63325da intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 264c4fc30 repository/src/main/java/org/apache/atlas/repository/ogm/glossary/AtlasGlossaryCategoryDTO.java 92b8fafdb repository/src/main/java/org/apache/atlas/repository/ogm/glossary/AtlasGlossaryTermDTO.java d1da4e0ac repository/src/main/java/org/apache/atlas/repository/ogm/profiles/AtlasSavedSearchDTO.java 68a874402 repository/src/main/java/org/apache/atlas/repository/ogm/profiles/AtlasUserProfileDTO.java 620b14825 repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java b2bd896b8 Diff: https://reviews.apache.org/r/70019/diff/2/ Changes: https://reviews.apache.org/r/70019/diff/1-2/ Testing (updated) --- - verified that container entity delete causes its contained entities to be deleted - verified that entity-create requires container value be set for contined entities - pre-commit tests run: https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/928/ Thanks, madhan
