Re: Review Request 73076: Deferred Actions Implementation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73076/ --- (Updated Dec. 16, 2020, 11:42 p.m.) Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, and Sarath Subramanian. Changes --- Updates include: - Additional unit tests. - Refactoring for task arguments. Summary (updated) - Deferred Actions Implementation Bugs: ATLAS-3919 https://issues.apache.org/jira/browse/ATLAS-3919 Repository: atlas Description --- **Approach** Model: _TaskDef_ - Definition of task. - Contains status of the task for querying. _AbstractTask_ - Takes TaskDef as input and calls run. - The concrete tasks has logic for doing real work. _TaskFactory_ - Creates tasks given task type. - Enumerates all the tasks that are supported. _TaskManagement_ - Component/Service. Constructed after all existing components are constructed. - Provides a simple interface for adding tasks. - Processes tasks that are in pending state. - Supports HA mode, where the tasks are not executed on passive instance. _TaskManagement.Registry_ - CRUD operations with TaskDef vertex. _TaskManagement.Processor_ - Provides mechanism for provides single queue for processing tasks in a way that is non-blocking. - Uses task type and the associated TaskFactory to construct concrete task. Changes to _EntityGraphMapper_ There are 3 new methods added that use portion of existing classification propagation logic and create a separate method. New class _EntityGraphMapperWithTasks_ acts as a proxy for EntityGraphMapper. All calls that operate without tasks get channeled to the existing implementation. The ones involving tasks are modified to use the new task framework. **Concurrent Changes to Classifications** Via REST APIs and operations from web UI, it is possible to make changes to the same entity. This situation will be more pronounced in the new setup. The existing framework has _GraphTransactionInterceptor.lockObjectAndReleasePostCommit_. This provides ReentrantLock on an entity that is identified by the supplied GUID. This will cause a lock on the entity and all the impacted vertices. This has potential for adversely impacting scalability. Diffs (updated) - intg/src/main/java/org/apache/atlas/AtlasConfiguration.java ea9f26d47 intg/src/main/java/org/apache/atlas/model/tasks/TaskDef.java PRE-CREATION intg/src/main/java/org/apache/atlas/type/AtlasType.java b30b483e0 intg/src/main/java/org/apache/atlas/utils/AtlasJson.java abeddf640 repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java 86b369fc9 repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java 244072289 repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java 2cfcc0bc1 repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapperWithTasks.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationPropagateTaskFactory.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationPropagationTasks.java PRE-CREATION repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationTask.java PRE-CREATION repository/src/main/java/org/apache/atlas/tasks/AbstractTask.java PRE-CREATION repository/src/main/java/org/apache/atlas/tasks/TaskExecutor.java PRE-CREATION repository/src/main/java/org/apache/atlas/tasks/TaskFactory.java PRE-CREATION repository/src/main/java/org/apache/atlas/tasks/TaskManagement.java PRE-CREATION repository/src/main/java/org/apache/atlas/util/BeanUtilRepo.java PRE-CREATION repository/src/test/java/org/apache/atlas/repository/tagpropagation/ClassificationPropagationTest.java 84e9bfa04 repository/src/test/java/org/apache/atlas/tasks/TaskExecutorTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/tasks/TaskManagementTest.java PRE-CREATION repository/src/test/java/org/apache/atlas/tasks/TaskRegistryTest.java PRE-CREATION server-api/src/main/java/org/apache/atlas/RequestContext.java 216ba08d3 server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java bb7f8fcc4 webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java b20b40474 webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 77422b2a5 Diff: https://reviews.apache.org/r/73076/diff/7/ Changes: https://reviews.apache.org/r/73076/diff/6-7/ Testing --- **Unit tests** New tests. **Manual tests** In-progress. **Volume test** Pending. **Pre-commit build** https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/279/ Thanks, Ashutosh Mestry
[jira] [Updated] (ATLAS-4063) EmbeddedKafkaServer simplification
[ https://issues.apache.org/jira/browse/ATLAS-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4063: -- Fix Version/s: 2.2.0 3.0.0 > EmbeddedKafkaServer simplification > -- > > Key: ATLAS-4063 > URL: https://issues.apache.org/jira/browse/ATLAS-4063 > Project: Atlas > Issue Type: Improvement >Affects Versions: 2.1.0 >Reporter: Andras Katona >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > For easier Kafka dependency upgrading > - getting rid of org.apache.kafka.common.utils.Time implementation - as we > can use one provided by Kafka > - unnecessary usage of scala converter: scala.collection.JavaConversions, > using ArrayBuffer instead directly -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4063) EmbeddedKafkaServer simplification
[ https://issues.apache.org/jira/browse/ATLAS-4063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4063: -- Affects Version/s: 2.1.0 > EmbeddedKafkaServer simplification > -- > > Key: ATLAS-4063 > URL: https://issues.apache.org/jira/browse/ATLAS-4063 > Project: Atlas > Issue Type: Improvement >Affects Versions: 2.1.0 >Reporter: Andras Katona >Priority: Major > > For easier Kafka dependency upgrading > - getting rid of org.apache.kafka.common.utils.Time implementation - as we > can use one provided by Kafka > - unnecessary usage of scala converter: scala.collection.JavaConversions, > using ArrayBuffer instead directly -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4060) Update models with hive_db managedLocation attribute
[ https://issues.apache.org/jira/browse/ATLAS-4060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4060: -- Labels: hive models (was: ) > Update models with hive_db managedLocation attribute > > > Key: ATLAS-4060 > URL: https://issues.apache.org/jira/browse/ATLAS-4060 > Project: Atlas > Issue Type: Task > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Labels: hive, models > Fix For: 3.0.0, 2.2.0 > > > HIVE added support for location for managed tables on database and introduced > new attribute managedLocation. > Atlas Hive-hook should have support for managedLocation in hive_db entity. > Updating hive models with managedLocation. > Need to do hive-hook changes once Apache Hive release managedLocation > feature(ATLAS-4061). > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4060) Update models with hive_db managedLocation attribute
[ https://issues.apache.org/jira/browse/ATLAS-4060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4060: -- Affects Version/s: 2.1.0 > Update models with hive_db managedLocation attribute > > > Key: ATLAS-4060 > URL: https://issues.apache.org/jira/browse/ATLAS-4060 > Project: Atlas > Issue Type: Task >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > > HIVE added support for location for managed tables on database and introduced > new attribute managedLocation. > Atlas Hive-hook should have support for managedLocation in hive_db entity. > Updating hive models with managedLocation. > Need to do hive-hook changes once Apache Hive release managedLocation > feature(ATLAS-4061). > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4060) Update models with hive_db managedLocation attribute
[ https://issues.apache.org/jira/browse/ATLAS-4060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4060: -- Component/s: atlas-core > Update models with hive_db managedLocation attribute > > > Key: ATLAS-4060 > URL: https://issues.apache.org/jira/browse/ATLAS-4060 > Project: Atlas > Issue Type: Task > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > HIVE added support for location for managed tables on database and introduced > new attribute managedLocation. > Atlas Hive-hook should have support for managedLocation in hive_db entity. > Updating hive models with managedLocation. > Need to do hive-hook changes once Apache Hive release managedLocation > feature(ATLAS-4061). > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4060) Update models with hive_db managedLocation attribute
[ https://issues.apache.org/jira/browse/ATLAS-4060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sarath Subramanian updated ATLAS-4060: -- Fix Version/s: 2.2.0 3.0.0 > Update models with hive_db managedLocation attribute > > > Key: ATLAS-4060 > URL: https://issues.apache.org/jira/browse/ATLAS-4060 > Project: Atlas > Issue Type: Task >Affects Versions: 2.1.0 >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > HIVE added support for location for managed tables on database and introduced > new attribute managedLocation. > Atlas Hive-hook should have support for managedLocation in hive_db entity. > Updating hive models with managedLocation. > Need to do hive-hook changes once Apache Hive release managedLocation > feature(ATLAS-4061). > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4060) Update models with hive_db managedLocation attribute
[ https://issues.apache.org/jira/browse/ATLAS-4060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250584#comment-17250584 ] ASF subversion and git services commented on ATLAS-4060: Commit 433ae4470f3a2853ff2319c11571ef15d2319875 in atlas's branch refs/heads/master from Radhika Kundam [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=433ae44 ] ATLAS-4060: Update hive_db model to add new attribute - 'managedLocation' Signed-off-by: Sarath Subramanian > Update models with hive_db managedLocation attribute > > > Key: ATLAS-4060 > URL: https://issues.apache.org/jira/browse/ATLAS-4060 > Project: Atlas > Issue Type: Task >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > > HIVE added support for location for managed tables on database and introduced > new attribute managedLocation. > Atlas Hive-hook should have support for managedLocation in hive_db entity. > Updating hive models with managedLocation. > Need to do hive-hook changes once Apache Hive release managedLocation > feature(ATLAS-4061). > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4060) Update models with hive_db managedLocation attribute
[ https://issues.apache.org/jira/browse/ATLAS-4060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250585#comment-17250585 ] ASF subversion and git services commented on ATLAS-4060: Commit a6ffa146a32b1bfdb43c017c9eec06fa36a01470 in atlas's branch refs/heads/branch-2.0 from Radhika Kundam [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=a6ffa14 ] ATLAS-4060: Update hive_db model to add new attribute - 'managedLocation' Signed-off-by: Sarath Subramanian (cherry picked from commit 433ae4470f3a2853ff2319c11571ef15d2319875) > Update models with hive_db managedLocation attribute > > > Key: ATLAS-4060 > URL: https://issues.apache.org/jira/browse/ATLAS-4060 > Project: Atlas > Issue Type: Task >Reporter: Radhika Kundam >Assignee: Radhika Kundam >Priority: Major > > HIVE added support for location for managed tables on database and introduced > new attribute managedLocation. > Atlas Hive-hook should have support for managedLocation in hive_db entity. > Updating hive models with managedLocation. > Need to do hive-hook changes once Apache Hive release managedLocation > feature(ATLAS-4061). > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4015) Indexes: Add Conditional Re-Indexing Patch
[ https://issues.apache.org/jira/browse/ATLAS-4015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250564#comment-17250564 ] ASF subversion and git services commented on ATLAS-4015: Commit e8dcde528deb0e9d5a9edbc28cb3ccb6f87cba0f in atlas's branch refs/heads/branch-2.0 from Ashutosh Mestry [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=e8dcde5 ] ATLAS-4015: Add Re-indexing as JAVA_PATCH. Part 2 > Indexes: Add Conditional Re-Indexing Patch > -- > > Key: ATLAS-4015 > URL: https://issues.apache.org/jira/browse/ATLAS-4015 > Project: Atlas > Issue Type: Improvement >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Attachments: ATLAS-4015-Add-Re-indexing-as-JAVA_PATCH-master.patch > > > *Background* > Every so often, we encounter cases where the Solr indexes are out of sync > with the data stored in the database. This leads to incorrect basic search > output. Additionally the facet numbers displayed are also incorrect. > *Solution* > It may help to make the ReIndex functionality to be made part of one of > JAVA_PATCH handlers that can be applied only when end user notices a problem > like described above. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4076) Creating an asset from existing asset adds duplicate Atlas audit entries
[ https://issues.apache.org/jira/browse/ATLAS-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh resolved ATLAS-4076. --- Resolution: Fixed > Creating an asset from existing asset adds duplicate Atlas audit entries > - > > Key: ATLAS-4076 > URL: https://issues.apache.org/jira/browse/ATLAS-4076 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Critical > > Steps : > Go to beeline in hive. > Create a new asset which is created from an existing asset, using the query : > create table test_web_site as select * from web_site_five; > Observe that web_site_five contains two managed tags which should be > propagated to the newly created table , i.e : test_web_site > Observe the audit tab for 'test_web_site' table in atlas contains multiple > entries -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4067) In Entity Audits tab, show action as "user-defined properties added/updated/deleted" instead of plain 'Entity Updated'
[ https://issues.apache.org/jira/browse/ATLAS-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deep Singh resolved ATLAS-4067. --- Fix Version/s: 2.1.0 Resolution: Fixed > In Entity Audits tab, show action as "user-defined properties > added/updated/deleted" instead of plain 'Entity Updated' > -- > > Key: ATLAS-4067 > URL: https://issues.apache.org/jira/browse/ATLAS-4067 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 2.1.0 >Reporter: Umesh Padashetty >Assignee: Deep Singh >Priority: Major > Fix For: 2.1.0 > > Attachments: Screenshot 2020-12-09 at 1.03.44 AM.png, Screenshot > 2020-12-09 at 12.57.59 AM.png > > > Currently in the entity audits tab, we show actions as below: > * Business Attribute(s) Updated > * Classification Added > * Label(s) Added > !Screenshot 2020-12-09 at 1.03.44 AM.png|width=1017,height=440! > But for user-defined properties alone, we show "Entity Updated" for all the > udp actions. > !Screenshot 2020-12-09 at 12.57.59 AM.png|width=1028,height=292! > This makes it very difficult to find out from the audits tab in which "Entity > Update" action the udp was updated for an entity. > Though the user can expand the operation and find out what happened with udp, > this is not very intuitive -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4076) Creating an asset from existing asset adds duplicate Atlas audit entries
[ https://issues.apache.org/jira/browse/ATLAS-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250538#comment-17250538 ] ASF subversion and git services commented on ATLAS-4076: Commit 9f20077ec9fc6ea672803b11f037fea1bad168e9 in atlas's branch refs/heads/branch-2.0 from Deep Singh [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=9f20077 ] ATLAS-4076: Propagated-tag-addition entries were appearing multiple times on derived table audit Signed-off-by: Ashutosh Mestry > Creating an asset from existing asset adds duplicate Atlas audit entries > - > > Key: ATLAS-4076 > URL: https://issues.apache.org/jira/browse/ATLAS-4076 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Critical > > Steps : > Go to beeline in hive. > Create a new asset which is created from an existing asset, using the query : > create table test_web_site as select * from web_site_five; > Observe that web_site_five contains two managed tags which should be > propagated to the newly created table , i.e : test_web_site > Observe the audit tab for 'test_web_site' table in atlas contains multiple > entries -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4067) In Entity Audits tab, show action as "user-defined properties added/updated/deleted" instead of plain 'Entity Updated'
[ https://issues.apache.org/jira/browse/ATLAS-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250539#comment-17250539 ] ASF subversion and git services commented on ATLAS-4067: Commit 2a42e7826e715f8b05d27c7d65d9b2c753e12b57 in atlas's branch refs/heads/branch-2.0 from Deep Singh [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=2a42e78 ] ATLAS-4067: Updates in Custom Attributes and Business Attributes must be captured separately in entity audits. Signed-off-by: Ashutosh Mestry > In Entity Audits tab, show action as "user-defined properties > added/updated/deleted" instead of plain 'Entity Updated' > -- > > Key: ATLAS-4067 > URL: https://issues.apache.org/jira/browse/ATLAS-4067 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 2.1.0 >Reporter: Umesh Padashetty >Assignee: Deep Singh >Priority: Major > Attachments: Screenshot 2020-12-09 at 1.03.44 AM.png, Screenshot > 2020-12-09 at 12.57.59 AM.png > > > Currently in the entity audits tab, we show actions as below: > * Business Attribute(s) Updated > * Classification Added > * Label(s) Added > !Screenshot 2020-12-09 at 1.03.44 AM.png|width=1017,height=440! > But for user-defined properties alone, we show "Entity Updated" for all the > udp actions. > !Screenshot 2020-12-09 at 12.57.59 AM.png|width=1028,height=292! > This makes it very difficult to find out from the audits tab in which "Entity > Update" action the udp was updated for an entity. > Though the user can expand the operation and find out what happened with udp, > this is not very intuitive -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4076) Creating an asset from existing asset adds duplicate Atlas audit entries
[ https://issues.apache.org/jira/browse/ATLAS-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250513#comment-17250513 ] ASF subversion and git services commented on ATLAS-4076: Commit 2e40de917083c7c5583a61537b6447ab3af41918 in atlas's branch refs/heads/master from Deep Singh [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=2e40de9 ] ATLAS-4076: Propagated-tag-addition entries were appearing multiple times on derived table audit Signed-off-by: Ashutosh Mestry > Creating an asset from existing asset adds duplicate Atlas audit entries > - > > Key: ATLAS-4076 > URL: https://issues.apache.org/jira/browse/ATLAS-4076 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Deep Singh >Assignee: Deep Singh >Priority: Critical > > Steps : > Go to beeline in hive. > Create a new asset which is created from an existing asset, using the query : > create table test_web_site as select * from web_site_five; > Observe that web_site_five contains two managed tags which should be > propagated to the newly created table , i.e : test_web_site > Observe the audit tab for 'test_web_site' table in atlas contains multiple > entries -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4067) In Entity Audits tab, show action as "user-defined properties added/updated/deleted" instead of plain 'Entity Updated'
[ https://issues.apache.org/jira/browse/ATLAS-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250514#comment-17250514 ] ASF subversion and git services commented on ATLAS-4067: Commit c1c28c12add6bf4b19a8218037b52409970d9857 in atlas's branch refs/heads/master from Deep Singh [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=c1c28c1 ] ATLAS-4067: Updates in Custom Attributes and Business Attributes must be captured separately in entity audits. Signed-off-by: Ashutosh Mestry > In Entity Audits tab, show action as "user-defined properties > added/updated/deleted" instead of plain 'Entity Updated' > -- > > Key: ATLAS-4067 > URL: https://issues.apache.org/jira/browse/ATLAS-4067 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 2.1.0 >Reporter: Umesh Padashetty >Assignee: Deep Singh >Priority: Major > Attachments: Screenshot 2020-12-09 at 1.03.44 AM.png, Screenshot > 2020-12-09 at 12.57.59 AM.png > > > Currently in the entity audits tab, we show actions as below: > * Business Attribute(s) Updated > * Classification Added > * Label(s) Added > !Screenshot 2020-12-09 at 1.03.44 AM.png|width=1017,height=440! > But for user-defined properties alone, we show "Entity Updated" for all the > udp actions. > !Screenshot 2020-12-09 at 12.57.59 AM.png|width=1028,height=292! > This makes it very difficult to find out from the audits tab in which "Entity > Update" action the udp was updated for an entity. > Though the user can expand the operation and find out what happened with udp, > this is not very intuitive -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 73084: ATLAS-4067 In Entity Audits tab, show action as "user-defined properties added/updated/deleted" instead of plain 'Entity Updated'
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73084/#review222358 --- Ship it! Ship It! - Ashutosh Mestry On Dec. 16, 2020, 5:35 p.m., Deep Singh wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73084/ > --- > > (Updated Dec. 16, 2020, 5:35 p.m.) > > > Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath > Subramanian. > > > Bugs: ATLAS-4067 > https://issues.apache.org/jira/browse/ATLAS-4067 > > > Repository: atlas > > > Description > --- > > Unlike business attributes, custom(user-defined) attirbutes doesnot have > exclusive CURD APIs. Custome attributes are modified using Entity update POST > call only.As of today in the post call for modifying CA(custom attributes), > there is no other changes in entity definition besides modified CA. However, > entity update POST call can potentially have other updates as well. To > address this issue, in this fix we try to find out if the update is only > about CA if not we consider it regular Entity update call. > > Ideally there should be a separate APIs for handling CA as we do have for > BA(Business Attributes). In entity update call it is very difficult to find > out if the call was for (CREAT, UPDATE, or DELETE) of a particular CA. Same > is true for BA as well of BA modification are sent using ENtity Update API. > Therefor untill we have separate APIs for handling CA we will be showing > "User-defined Attribute(s) Updated" in the UI for any modification in CA > > > Diffs > - > > dashboardv2/public/js/utils/Enums.js ebfcebab1 > dashboardv3/public/js/utils/Enums.js ebfcebab1 > intg/src/main/java/org/apache/atlas/exception/AtlasBaseException.java > 553870278 > intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java > c37e28238 > > repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java > ca6f373e0 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java > b80e42eed > server-api/src/main/java/org/apache/atlas/RequestContext.java 32ffddde1 > > > Diff: https://reviews.apache.org/r/73084/diff/2/ > > > Testing > --- > > Manual Testing was done. > > > Thanks, > > Deep Singh > >
Re: Review Request 73084: ATLAS-4067 In Entity Audits tab, show action as "user-defined properties added/updated/deleted" instead of plain 'Entity Updated'
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73084/ --- (Updated Dec. 16, 2020, 5:35 p.m.) Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath Subramanian. Bugs: ATLAS-4067 https://issues.apache.org/jira/browse/ATLAS-4067 Repository: atlas Description --- Unlike business attributes, custom(user-defined) attirbutes doesnot have exclusive CURD APIs. Custome attributes are modified using Entity update POST call only.As of today in the post call for modifying CA(custom attributes), there is no other changes in entity definition besides modified CA. However, entity update POST call can potentially have other updates as well. To address this issue, in this fix we try to find out if the update is only about CA if not we consider it regular Entity update call. Ideally there should be a separate APIs for handling CA as we do have for BA(Business Attributes). In entity update call it is very difficult to find out if the call was for (CREAT, UPDATE, or DELETE) of a particular CA. Same is true for BA as well of BA modification are sent using ENtity Update API. Therefor untill we have separate APIs for handling CA we will be showing "User-defined Attribute(s) Updated" in the UI for any modification in CA Diffs (updated) - dashboardv2/public/js/utils/Enums.js ebfcebab1 dashboardv3/public/js/utils/Enums.js ebfcebab1 intg/src/main/java/org/apache/atlas/exception/AtlasBaseException.java 553870278 intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java c37e28238 repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java ca6f373e0 repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java b80e42eed server-api/src/main/java/org/apache/atlas/RequestContext.java 32ffddde1 Diff: https://reviews.apache.org/r/73084/diff/2/ Changes: https://reviews.apache.org/r/73084/diff/1-2/ Testing --- Manual Testing was done. Thanks, Deep Singh
[jira] [Commented] (ATLAS-4084) Atlas-503 error
[
https://issues.apache.org/jira/browse/ATLAS-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250188#comment-17250188
]
vijay commented on ATLAS-4084:
--
*The latest error i found in application .log is:*
Cassandra embedded solution is deprecated and will be removed with JanusGraph
0.5.0. For testing, please use JanusGraph Docker images:
https://github.com/JanusGraph/janusgraph-docker. For production, please use a
side-by-side deployment of Cassandra or a pre-packaged version of Cassandra
with a CQL backend in JanusGraph 0.5.0. For more details, see
https://docs.janusgraph.org/basics/server/. (CassandraEmbeddedStoreManager:81)
2020-12-16 09:05:26,704 WARN - [main:] ~ OpenJDK is not recommended. Please
upgrade to the newest Oracle Java release (CassandraDaemon:173)
2020-12-16 09:05:31,822 WARN - [main:] ~ JMX is not enabled to receive remote
connections. Please see cassandra-env.sh for more info. (CassandraDaemon:81)
2020-12-16 09:05:31,823 ERROR - [main:] ~ cassandra.jmx.local.port missing from
cassandra-env.sh, unable to start local JMX service.null (CassandraDaemon:87)
But i couldn't find cassandra-env.sh file. please share your valuable
suggestions
> Atlas-503 error
> ---
>
> Key: ATLAS-4084
> URL: https://issues.apache.org/jira/browse/ATLAS-4084
> Project: Atlas
> Issue Type: Bug
> Components: atlas-webui
>Affects Versions: 2.1.0
> Environment: Test
>Reporter: vijay
>Priority: Major
> Fix For: 2.1.0
>
>
> Hi Team,
> I have installed/configured atlas with embedded cassandra and solr by using
> official website:
> [https://atlas.apache.org/1.1.0/InstallationSteps.html]
>
> when i start the atlas it is showing started in terminal but when i hit the
> url in webb browser or curl the url is terminal i am getting below error
> HTTP ERROR: 503
> Problem accessing /. Reason:
> {{Service Unavailable}}
> *when i check the logs i found below error in application.log:*
> ERROR - [main:] ~ cassandra.jmx.local.port missing from cassandra-env.sh,
> unable to start local JMX service.null (CassandraDaemon:87)
> 2020-12-15 12:49:12,771 ERROR - [main:] ~ Directory
> /root/apache-atlas-sources-2.1.0/distro/target/apache-atlas-2.1.0-bin/apache-atlas-2.1.0/data/cassandra/data
> doesn't exist (CassandraDaemon:258)
> 2020-12-15 12:49:12,774 ERROR - [main:] ~ Directory
> /root/apache-atlas-sources-2.1.0/distro/target/apache-atlas-2.1.0-bin/apache-atlas-2.1.0/data/cassandra/commitlog
> doesn't exist (CassandraDaemon:258)
> 2020-12-15 12:49:12,774 ERROR - [main:] ~ Directory
> /root/apache-atlas-sources-2.1.0/distro/target/apache-atlas-2.1.0-bin/apache-atlas-2.1.0/data/cassandra/saved_caches
> doesn't exist (CassandraDaemon:258)
> 2020-12-15 12:49:14,982 WARN - [main:] ~ No host ID found, created
> 96f3a962-f1b0-4928-9304-8225af3af649 (Note: This should happen exactly once
> per node). (SystemKeyspace:745)
>
>
> *when i check atlas.20201215.out i found below errors:*
> Caused by: org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'get' defined in
> org.apache.atlas.repository.graph.AtlasGraphProvider: Bean instantiation via
> factory method failed; nested exception is
> org.springframework.beans.BeanInstantiationException: Failed to instantiate
> [org.apache.atlas.repository.graphdb.AtlasGraph]: Factory method 'get' threw
> exception; nested exception is java.lang.NoClassDefFoundError:
> org/apache/cassandra/dht/RandomPartitioner$BigIntegerToken
> at
> org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:599)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1178)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1072)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
> at
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
> at
> org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:208)
> at
>
[jira] [Created] (ATLAS-4084) Atlas-503 error
vijay created ATLAS-4084:
Summary: Atlas-503 error
Key: ATLAS-4084
URL: https://issues.apache.org/jira/browse/ATLAS-4084
Project: Atlas
Issue Type: Bug
Components: atlas-webui
Affects Versions: 2.1.0
Environment: Test
Reporter: vijay
Fix For: 2.1.0
Hi Team,
I have installed/configured atlas with embedded cassandra and solr by using
official website:
[https://atlas.apache.org/1.1.0/InstallationSteps.html]
when i start the atlas it is showing started in terminal but when i hit the url
in webb browser or curl the url is terminal i am getting below error
HTTP ERROR: 503
Problem accessing /. Reason:
{{Service Unavailable}}
*when i check the logs i found below error in application.log:*
ERROR - [main:] ~ cassandra.jmx.local.port missing from cassandra-env.sh,
unable to start local JMX service.null (CassandraDaemon:87)
2020-12-15 12:49:12,771 ERROR - [main:] ~ Directory
/root/apache-atlas-sources-2.1.0/distro/target/apache-atlas-2.1.0-bin/apache-atlas-2.1.0/data/cassandra/data
doesn't exist (CassandraDaemon:258)
2020-12-15 12:49:12,774 ERROR - [main:] ~ Directory
/root/apache-atlas-sources-2.1.0/distro/target/apache-atlas-2.1.0-bin/apache-atlas-2.1.0/data/cassandra/commitlog
doesn't exist (CassandraDaemon:258)
2020-12-15 12:49:12,774 ERROR - [main:] ~ Directory
/root/apache-atlas-sources-2.1.0/distro/target/apache-atlas-2.1.0-bin/apache-atlas-2.1.0/data/cassandra/saved_caches
doesn't exist (CassandraDaemon:258)
2020-12-15 12:49:14,982 WARN - [main:] ~ No host ID found, created
96f3a962-f1b0-4928-9304-8225af3af649 (Note: This should happen exactly once per
node). (SystemKeyspace:745)
*when i check atlas.20201215.out i found below errors:*
Caused by: org.springframework.beans.factory.BeanCreationException: Error
creating bean with name 'get' defined in
org.apache.atlas.repository.graph.AtlasGraphProvider: Bean instantiation via
factory method failed; nested exception is
org.springframework.beans.BeanInstantiationException: Failed to instantiate
[org.apache.atlas.repository.graphdb.AtlasGraph]: Factory method 'get' threw
exception; nested exception is java.lang.NoClassDefFoundError:
org/apache/cassandra/dht/RandomPartitioner$BigIntegerToken
at
org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:599)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1178)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1072)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at
org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:208)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1136)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1064)
at
org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:835)
at
org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:741)
... 59 more
Caused by: org.springframework.beans.BeanInstantiationException: Failed to
instantiate [org.apache.atlas.repository.graphdb.AtlasGraph]: Factory method
'get' threw exception; nested exception is java.lang.NoClassDefFoundError:
org/apache/cassandra/dht/RandomPartitioner$BigIntegerToken
at
org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:189)
at
org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:588)
... 72 more
Caused by: java.lang.NoClassDefFoundError:
org/apache/cassandra/dht/RandomPartitioner$BigIntegerToken
at
org.janusgraph.diskstorage.cassandra.embedded.CassandraEmbeddedStoreManager.openDatabase(CassandraEmbeddedStoreManager.java:141)
at
[jira] [Commented] (ATLAS-4058) UI - On entity details page, for Hive column 'position' is getting set as 0(zero)
[ https://issues.apache.org/jira/browse/ATLAS-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250177#comment-17250177 ] ASF subversion and git services commented on ATLAS-4058: Commit 5aec0b8290d778cff8f59cde580e1576dd40c12e in atlas's branch refs/heads/branch-2.0 from prasad pawar [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=5aec0b8 ] ATLAS-4058: UI: Fix Entity detail page Hive Column Position with 0 is set to NA. (cherry picked from commit de87800c19ea06f53904a2948fcde29a57d2f051) > UI - On entity details page, for Hive column 'position' is getting set as > 0(zero) > - > > Key: ATLAS-4058 > URL: https://issues.apache.org/jira/browse/ATLAS-4058 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 2.1.0 >Reporter: Durga Kadam >Assignee: Prasad P. Pawar >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: > 0001-ATLAS-4058-UI-Entity-detail-page-Hive-Column-Positio.patch, > Atlas-4058.png, Atlas-4058_1.png, > screenshot-issues.apache.org-2020.12.03-10_47_09.png > > > Hive Column position should get set to N/A OR blank instead of 0. > PFA image -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4058) UI - On entity details page, for Hive column 'position' is getting set as 0(zero)
[ https://issues.apache.org/jira/browse/ATLAS-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250172#comment-17250172 ] ASF subversion and git services commented on ATLAS-4058: Commit de87800c19ea06f53904a2948fcde29a57d2f051 in atlas's branch refs/heads/master from prasad pawar [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=de87800 ] ATLAS-4058: UI: Fix Entity detail page Hive Column Position with 0 is set to NA. > UI - On entity details page, for Hive column 'position' is getting set as > 0(zero) > - > > Key: ATLAS-4058 > URL: https://issues.apache.org/jira/browse/ATLAS-4058 > Project: Atlas > Issue Type: Bug > Components: atlas-webui >Affects Versions: 2.1.0 >Reporter: Durga Kadam >Assignee: Prasad P. Pawar >Priority: Minor > Fix For: 3.0.0, 2.2.0 > > Attachments: > 0001-ATLAS-4058-UI-Entity-detail-page-Hive-Column-Positio.patch, > Atlas-4058.png, Atlas-4058_1.png, > screenshot-issues.apache.org-2020.12.03-10_47_09.png > > > Hive Column position should get set to N/A OR blank instead of 0. > PFA image -- This message was sent by Atlassian Jira (v8.3.4#803005)
