Re: Review Request 74608: ATLAS-4797 : Implement custom audit filters in Atlas
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74608/#review226399 --- Ship it! Ship It! - Jayendra Parab On April 4, 2024, 5:14 a.m., Sheetal Shah wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74608/ > --- > > (Updated April 4, 2024, 5:14 a.m.) > > > Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, Prasad > Pawar, Radhika Kundam, and Sidharth Mishra. > > > Bugs: ATLAS-4797 > https://issues.apache.org/jira/browse/ATLAS-4797 > > > Repository: atlas > > > Description > --- > > Currently, Atlas has to persist all audit events without any constraints > which can make audit data grow exponentially after some time. > > This feature is specifically to reduce Atlas audit storage where generation > of audit events can be controlled based on the user's use-case/criteria > > Note: Audit data mentioned here is exclusive of Admin Audit Data which > persists in atlas_janus table. Filtering will be applicable to the entity > audit data which persists in ATLAS_ENTITY_AUDIT_EVENTS table. > > Precommit details : > Latest PC build (PASSED) : > build:https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1592/ > > > Diffs > - > > addons/hbase-bridge/pom.xml 278b6c6d3 > addons/hbase-testing-util/pom.xml c4f3a99ee > addons/hive-bridge-shim/pom.xml 39f16ceee > addons/hive-bridge/pom.xml 356ac9542 > addons/kafka-bridge/pom.xml 093db7229 > addons/models/-Area0/0010-base_model.json a4a9248ec > addons/sqoop-bridge/pom.xml 4b6eac98f > authorization/pom.xml 7a1108487 > distro/src/conf/atlas-application.properties b5734d7a8 > graphdb/api/pom.xml 4ba89b20f > graphdb/janus-hbase2/pom.xml 9ec087dbb > intg/pom.xml 43a172c1b > intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 0868f8b6e > intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 77a6fd8c3 > intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java > 3afd27e7d > repository/pom.xml 8fd744b44 > > repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java > 9b011ab06 > > repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java > b7f0dd5e2 > repository/src/main/java/org/apache/atlas/repository/ogm/AtlasRuleDTO.java > PRE-CREATION > repository/src/main/java/org/apache/atlas/repository/ogm/DataAccess.java > e63152475 > > repository/src/main/java/org/apache/atlas/rulesengine/AtlasEntityAuditFilterService.java > PRE-CREATION > repository/src/main/java/org/apache/atlas/rulesengine/AtlasRule.java > PRE-CREATION > repository/src/main/java/org/apache/atlas/rulesengine/AtlasRuleUtils.java > PRE-CREATION > repository/src/main/java/org/apache/atlas/rulesengine/AtlasRulesEngine.java > PRE-CREATION > repository/src/main/java/org/apache/atlas/rulesengine/RuleAction.java > PRE-CREATION > server-api/pom.xml de311c329 > test-tools/src/main/resources/solr/core-template/solrconfig.xml 7a0e8dd16 > webapp/src/main/java/org/apache/atlas/web/filters/ActiveServerFilter.java > e19beccb1 > webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java > 0025276bb > webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java > 268fca717 > > > Diff: https://reviews.apache.org/r/74608/diff/13/ > > > Testing > --- > > Manual testing done > > > Thanks, > > Sheetal Shah > >
Re: Review Request 74947: ATLAS-4838 : Export/Import : changeMarker is not set to entity's lastupdatetime or its closer timestamp value
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74947/#review226360 --- Ship it! Ship It! - Jayendra Parab On March 28, 2024, 11:20 a.m., Priyanshi Shah wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74947/ > --- > > (Updated March 28, 2024, 11:20 a.m.) > > > Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, and > Sheetal Shah. > > > Bugs: ATLAS-4838 > https://issues.apache.org/jira/browse/ATLAS-4838 > > > Repository: atlas > > > Description > --- > > Steps: > > When an hive table entity is exported using fetchType incremental with > changeMarker 0 , after exporting , the changeMarker in the export response is > not set to recent timestamp. > > Problem: > > The value of changeMarker was set on the basis of the earliest request > recorded on the server. There was some unused pending requests contexts which > were not cleared after its usage and hence, changeMarker the value was > changeMarker was wrongly set. > > Solution: > > Have cleared all the unused pending requests which was found. > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/repository/patches/AtlasPatchManager.java > e72a87713 > webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java > 0025276bb > webapp/src/main/java/org/apache/atlas/web/service/EmbeddedServer.java > 4c3a1c569 > webapp/src/main/java/org/apache/atlas/web/service/ServiceState.java > ea74d21ad > > > Diff: https://reviews.apache.org/r/74947/diff/2/ > > > Testing > --- > > Performed multiple export operations with incremental fetch type, the change > marker values were updated properly on each export operation. > > PC link: > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1578/ > > > Thanks, > > Priyanshi Shah > >
[jira] [Assigned] (ATLAS-4838) Export/Import : changeMarker is not set to entity's lastupdatetime or its closer timestamp value
[
https://issues.apache.org/jira/browse/ATLAS-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayendra Parab reassigned ATLAS-4838:
-
Assignee: Jayendra Parab
> Export/Import : changeMarker is not set to entity's lastupdatetime or its
> closer timestamp value
>
>
> Key: ATLAS-4838
> URL: https://issues.apache.org/jira/browse/ATLAS-4838
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Sharmadha S
> Assignee: Jayendra Parab
>Priority: Major
>
> When an hive table entity is exported using fetchType incremental with
> changeMarker 0 , after exporting , the changeMarker in the export response is
> not set to recent timestamp.
> For example , on exporting iceberg table with fetchType incremental and
> changeMarker as 0 :
> {code:java}
> 2024-02-08 13:28:43,838 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~
> export(item=AtlasObjectId{guid='null', typeName='iceberg_table',
> uniqueAttributes={qualifiedName:default.iceberg_table2@cm}}; matchType=null,
> fetchType=incremental): found 1 entities: options:
> {"itemsToExport":[{"typeName":"iceberg_table","uniqueAttributes":{"qualifiedName":"default.iceberg_table2@cm"}}],"options":{"fetchType":"incremental","changeMarker":0},"fetchTypeOptionValue":"incremental","skipLineageOptionValue":false,"changeTokenFromOptions":0}
> (StartEntityFetchByExportRequest:123)
> 2024-02-08 13:28:48,697 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ Assigned guid =
> f5e6342a-dfdf-4d54-a1b3-b24f987c4ecb to newly created entity (DataAccess:85)
> 2024-02-08 13:28:48,697 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ addAuditEntry: user: admin, source:
> cm, target: , operation: EXPORT (ExportImportAuditService:168)
> 2024-02-08 13:28:48,699 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ <== export(user=admin,
> from=10.19.28.124): status SUCCESS: changeMarker: 1707225183928
> (ExportService:107) {code}
> changeMarker from response is 1707225183928 => Tuesday, February 6, 2024
> 6:43:03.928 PM
>
> But lastUpdatetime of iceberg table is
> {code:java}
> "createTime": 1707398866970 => Thursday, February 8, 2024 6:57:46.970 PM
> "updateTime": 1707398867375. => Thursday, February 8, 2024 6:57:47.375 PM
> {code}
> Value is set to 2 days back. This is not then an incremental export , as it
> will fetch all updates from Tuesday, February 6, 2024 6:43:03.928 PM in next
> export
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Re: Review Request 74713: ATLAS-4803 : Optimize Edge fetch
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74713/#review226011
---
Ship it!
Ship It!
- Jayendra Parab
On Nov. 28, 2023, 3:26 a.m., Paresh Devalia wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74713/
> ---
>
> (Updated Nov. 28, 2023, 3:26 a.m.)
>
>
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, Mandar Ambawane,
> Pinal Shah, Sheetal Shah, and Sidharth Mishra.
>
>
> Bugs: ATLAS-4803
> https://issues.apache.org/jira/browse/ATLAS-4803
>
>
> Repository: atlas
>
>
> Description
> ---
>
> Kafka lag was not decreasing for ATLAS_HOOK topics, create Entity API was
> taking 50-60 sec per request.
>
> Hive_table typename count was 10mn record.
>
> Impala_lineage_column typename count was 26mn count.
>
> Able to reproduce the issue.
>
> Metrics
>
>
> This difference exists because earlier even fromVertex did not have any
> edges, the search would iterate through all the edges of the toVertex and
> timeConsume was high.
>
>
> Before: "getRelationshipEdge":{"count":10,"timeTaken":5}
> After removing if condition for toVertex.hasEdge:
> "getRelationshipEdge":{"count":10,"timeTaken":80}
>
>
> Diffs
> -
>
>
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java
> 0dd573b89
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java
> ef0313e02
>
>
> Diff: https://reviews.apache.org/r/74713/diff/1/
>
>
> Testing
> ---
>
> What was the relationship type?
> __hive_db.table, __hive_table.columns
>
> What entity type was identified and tested , meaning which entity type of
> vertex took time to find edges?
> Impala_column_lineage, impala_process, hive_table, hive_column
>
> What was the count of the edges corresponding to that entity type?
> Hive_column = 28m
> Impala_column_lineage = 24m
>
> Timing before and after
> Before: "getRelationshipEdge":{"count":10,"timeTaken":5}
> After removing if condition for toVertex.hasEdge:
> "getRelationshipEdge":{"count":10,"timeTaken":80}
>
>
> Volume testing
> Initiate kafka dump and lag started decreasing.
>
>
> Thanks,
>
> Paresh Devalia
>
>
Re: Review Request 74724: ATLAS-4809 : Atlas - Upgrade amqp-client to 5.18.0+
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74724/#review225954 --- pom.xml Lines 814 (patched) <https://reviews.apache.org/r/74724/#comment314341> We cannot hardcode the version of dropwizard since Atlas isn't using it, it will cause issues later when we upgrade janusgraph's version - Jayendra Parab On Nov. 8, 2023, 11:47 a.m., Sheetal Shah wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74724/ > --- > > (Updated Nov. 8, 2023, 11:47 a.m.) > > > Review request for atlas, Jayendra Parab and Pinal Shah. > > > Bugs: ATLAS-4809 > https://issues.apache.org/jira/browse/ATLAS-4809 > > > Repository: atlas > > > Description > --- > > Upgrade amqp-client to 5.18.0+ > > > Diffs > - > > graphdb/janus-hbase2/pom.xml c2a2e74f0 > pom.xml c55bec763 > tools/atlas-index-repair/pom.xml ab55f4c7f > > > Diff: https://reviews.apache.org/r/74724/diff/1/ > > > Testing > --- > > 1)mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > Atlas server runnning > 2)Quick started was passed successfully > > > Precommit - > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1502/ > > > Thanks, > > Sheetal Shah > >
Re: Review Request 74712: ATLAS-4807: Fix IT test related to Labels
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74712/#review225916 --- Ship it! Ship It! - Jayendra Parab On Nov. 1, 2023, 9:59 a.m., Pinal Shah wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74712/ > --- > > (Updated Nov. 1, 2023, 9:59 a.m.) > > > Review request for atlas and Jayendra Parab. > > > Bugs: ATLAS-4807 > https://issues.apache.org/jira/browse/ATLAS-4807 > > > Repository: atlas > > > Description > --- > > testSetLabelsByTypeName(org.apache.atlas.web.integration.EntityV2JerseyResourceIT) > test is failing because of ATLAS-4785 > > > Diffs > - > > > webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java > 00f0aab1a > > > Diff: https://reviews.apache.org/r/74712/diff/1/ > > > Testing > --- > > PC run: > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1496/ > > > Thanks, > > Pinal Shah > >
Re: Review Request 74452: ATLAS-4754 : Download Search with Basic Search gives java.io.FileNotFoundException
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74452/#review225895 --- Ship it! Ship It! - Jayendra Parab On May 23, 2023, 7:56 a.m., Mandar Ambawane wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74452/ > --- > > (Updated May 23, 2023, 7:56 a.m.) > > > Review request for atlas, Jayendra Parab, Pinal Shah, and Sheetal Shah. > > > Bugs: ATLAS-4754 > https://issues.apache.org/jira/browse/ATLAS-4754 > > > Repository: atlas > > > Description > --- > > Default value for the download directory will be read from system property > "user.dir" instead of property "atlas.home" > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/searchdownload/SearchResultDownloadTask.java > fd90fd440 > > > Diff: https://reviews.apache.org/r/74452/diff/1/ > > > Testing > --- > > > Thanks, > > Mandar Ambawane > >
Re: Review Request 74549: ATLAS-4784 :- Atlas - Upgrade Spring framework to 5.2.27
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74549/#review225657 --- Ship it! Ship It! - Jayendra Parab On Aug. 17, 2023, 3:40 a.m., Vinayak Marraiya wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74549/ > --- > > (Updated Aug. 17, 2023, 3:40 a.m.) > > > Review request for atlas, Jayendra Parab and Pinal Shah. > > > Bugs: ATLAS-4784 > https://issues.apache.org/jira/browse/ATLAS-4784 > > > Repository: atlas > > > Description > --- > > Currently Atlas is using Spring framework version 5.3.21 upgrading it to > 5.3.27 > > > Diffs > - > > pom.xml 5b914d4bc > > > Diff: https://reviews.apache.org/r/74549/diff/1/ > > > Testing > --- > > 1)mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > Atlas server runnning > 2)Quick started was passed successfully > > Precommit - > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1444/console > > > Thanks, > > Vinayak Marraiya > >
Re: Review Request 74161: ATLAS-4687 : Atlas server doesn't come up due to circular dependency in TaskRegistry class
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74161/#review225418 --- Ship it! Ship It! - Jayendra Parab On April 27, 2023, 7:34 a.m., Sheetal Shah wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74161/ > --- > > (Updated April 27, 2023, 7:34 a.m.) > > > Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, and > Prasad Pawar. > > > Bugs: ATLAS-4687 > https://issues.apache.org/jira/browse/ATLAS-4687 > > > Repository: atlas > > > Description > --- > > Following errors were found intermittently due to which Atlas server won’t > start up. These errors pointed to a circular dependency issue which was > caused by TaskRegistry > class(repository/src/main/java/org/apache/atlas/tasks/TaskRegistry.java) As a > resolution to this, @Lazy annotation is added on @Component definition of > this class. > (FYI : A @Lazy bean is not initialised until referenced by another bean or > explicitly retrieved from BeanFactory . Beans that are not annotated with > @Lazy are initialise eagerly.) > > > > org.springframework.beans.factory.BeanCurrentlyInCreationException: Error > creating bean with name 'taskRegistry': Bean with name 'taskRegistry' has > been injected into other beans [taskManagement] in its raw version as part of > a circular reference, but has eventually been wrapped. This means that said > other beans do not use the final version of the bean. This is often the > result of over-eager type matching - consider using 'getBeanNamesForType' > with the 'allowEagerInit' flag turned off, for example. > > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:649) > ~[spring-beans-5.3.21.jar:5.3.21] > > at > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) > ~[spring-beans-5.3.21.jar:5.3.21] > > at > org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) > ~[spring-beans-5.3.21.jar:5.3.21] > > at > org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) > ~[spring-beans-5.3.21.jar:5.3.21] > > at > org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) > ~[spring-beans-5.3.21.jar:5.3.21] > > at > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) > ~[spring-beans-5.3.21.jar:5.3.21] > > at > org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955) > ~[spring-beans-5.3.21.jar:5.3.21] > > at > org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918) > ~[spring-context-5.3.21.jar:5.3.21] > > at > org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583) > ~[spring-context-5.3.21.jar:5.3.21] > > at > org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:401) > ~[spring-web-5.3.21.jar:5.3.21] > > at > org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:292) > [spring-web-5.3.21.jar:5.3.21] > > at > org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103) > [spring-web-5.3.21.jar:5.3.21] > > at > org.apache.atlas.web.setup.KerberosAwareListener.contextInitialized(KerberosAwareListener.java:31) > [atlas-webapp-3.0.0-SNAPSHOT.jar:?] > > at > org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1013) > [jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723] > > at > org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:553) > [jetty-servlet-9.4.31.v20200723.jar:9.4.31.v20200723] > > at > org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:942) > [jetty-server-9.4.31.v20200723.jar:9.4.31.v20200723] > > at > org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:782) > [jetty-servlet-9.4.31.v202
Re: Review Request 74341: ATLAS-4733 : Download Basic and DSL search results
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74341/#review225384
---
Ship it!
Ship It!
- Jayendra Parab
On April 14, 2023, 8 a.m., Mandar Ambawane wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74341/
> ---
>
> (Updated April 14, 2023, 8 a.m.)
>
>
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Pinal Shah,
> Radhika Kundam, Sheetal Shah, and Sidharth Mishra.
>
>
> Bugs: ATLAS-4733
> https://issues.apache.org/jira/browse/ATLAS-4733
>
>
> Repository: atlas
>
>
> Description
> ---
>
> This ticket tracks the Server side changes required for the Search result
> download task.
>
> From server side, each download request is considered as a separate thread
> which will run in the background and will generate the csv file according to
> the inputs provided.
>
> The generated files will be stored in the specific location (which is
> configurable).
>
> A scheduler will be run to delete the old files. In this case, the scheduler
> interval and file expiry time is also configurable.
>
>
> Diffs
> -
>
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 608342433
>
> intg/src/main/java/org/apache/atlas/model/discovery/AtlasSearchResultDownloadStatus.java
> PRE-CREATION
>
> repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java
> d94110004
>
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
> 582d97542
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/searchdownload/SearchResultDownloadTask.java
> PRE-CREATION
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/searchdownload/SearchResultDownloadTaskFactory.java
> PRE-CREATION
> repository/src/main/java/org/apache/atlas/tasks/TaskManagement.java
> 5b4bf71cc
> repository/src/main/java/org/apache/atlas/tasks/TaskRegistry.java 6f770edb1
> webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java a6ca04f3c
>
>
> Diff: https://reviews.apache.org/r/74341/diff/8/
>
>
> Testing
> ---
>
> PreCommit:
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1338/consoleFull
>
> Request to generate csv file with default columns seen on UI:
>
> curl -u : --request POST
> ":/api/atlas/v2/search/basic/download/create_file" --header
> 'Content-Type: application/json' --data-raw
> '{"searchParameters":{"excludeDeletedEntities":true,"includeSubClassifications":true,"includeSubTypes":true,"includeClassificationAttributes":true,"entityFilters":null,"tagFilters":null,"attributes":[],"limit":25,"offset":0,"typeName":"_ALL_ENTITY_TYPES","classification":null,"termName":null},"attributeLabelMap":{}}'
>
>
> Request to generate csv file with additional columns selected from UI:
>
> curl -u : --request POST
> ":/api/atlas/v2/search/basic/download/create_file" --header
> 'Content-Type: application/json' --data-raw
> '{"searchParameters":{"excludeDeletedEntities":true,"includeSubClassifications":true,"includeSubTypes":true,"includeClassificationAttributes":true,"entityFilters":null,"tagFilters":null,"attributes":["__guid","__state"],"limit":25,"offset":0,"typeName":"_ALL_ENTITY_TYPES","classification":null,"termName":null},"attributeLabelMap":{"Guid":"__guid","Status":"__state"}}'
>
>
> Thanks,
>
> Mandar Ambawane
>
>
Re: Review Request 74341: ATLAS-4733 : Download Basic and DSL search results
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74341/#review225358 --- repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java Lines 491 (patched) <https://reviews.apache.org/r/74341/#comment314006> Consider having a constructor to create AtlasSearchDownloadRecord with the values set here and also in the for-loop below. repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/searchdownload/SearchResultDownloadTask.java Lines 71 (patched) <https://reviews.apache.org/r/74341/#comment314008> Consider prefixing all the download search related properties with atlas.download.search, so that the the properties aren't confused with any other search related properties. repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/searchdownload/SearchResultDownloadTask.java Lines 72 (patched) <https://reviews.apache.org/r/74341/#comment314007> Consider using sub-dir inside atlas.home directory as the default directory path. "/tmp" path is accessible for all users. repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/searchdownload/SearchResultDownloadTaskFactory.java Lines 55 (patched) <https://reviews.apache.org/r/74341/#comment314010> Consider having only the expiry time property, not the property for cleanup repository/src/main/java/org/apache/atlas/tasks/TaskRegistry.java Lines 108 (patched) <https://reviews.apache.org/r/74341/#comment314009> Is graph.commit() required for a read-only operation? - Jayendra Parab On March 30, 2023, 2:37 p.m., Mandar Ambawane wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74341/ > --- > > (Updated March 30, 2023, 2:37 p.m.) > > > Review request for atlas, Jayendra Parab, Madhan Neethiraj, Pinal Shah, > Radhika Kundam, Sheetal Shah, and Sidharth Mishra. > > > Bugs: ATLAS-4733 > https://issues.apache.org/jira/browse/ATLAS-4733 > > > Repository: atlas > > > Description > --- > > This ticket tracks the Server side changes required for the Search result > download task. > > From server side, each download request is considered as a separate thread > which will run in the background and will generate the csv file according to > the inputs provided. > > The generated files will be stored in the specific location (which is > configurable). > > A scheduler will be run to delete the old files. In this case, the scheduler > interval and file expiry time is also configurable. > > > Diffs > - > > intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 608342433 > > intg/src/main/java/org/apache/atlas/model/discovery/AtlasSearchResultDownloadStatus.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java > d94110004 > > repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java > 582d97542 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/searchdownload/SearchResultDownloadTask.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/searchdownload/SearchResultDownloadTaskFactory.java > PRE-CREATION > repository/src/main/java/org/apache/atlas/tasks/TaskManagement.java > 5b4bf71cc > repository/src/main/java/org/apache/atlas/tasks/TaskRegistry.java 6f770edb1 > webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java a6ca04f3c > > > Diff: https://reviews.apache.org/r/74341/diff/4/ > > > Testing > --- > > > Thanks, > > Mandar Ambawane > >
Re: Review Request 74376: ATLAS-4741: updated Storm version to 2.4.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74376/#review225336 --- Ship it! Ship It! - Jayendra Parab On March 31, 2023, 2:36 a.m., Madhan Neethiraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74376/ > --- > > (Updated March 31, 2023, 2:36 a.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, Nikhil Bonte, > Nixon Rodrigues, Pinal Shah, Sarath Subramanian, and Sidharth Mishra. > > > Bugs: ATLAS-4741 > https://issues.apache.org/jira/browse/ATLAS-4741 > > > Repository: atlas > > > Description > --- > > - updated Strom version to 2.4.0 > - excluded storm-clojure from storm-core > > > Diffs > - > > addons/storm-bridge-shim/pom.xml f4e75927f > addons/storm-bridge/pom.xml d6cec279b > pom.xml 15aa57bc4 > > > Diff: https://reviews.apache.org/r/74376/diff/1/ > > > Testing > --- > > - pre-commit tests run > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1336/ > > > Thanks, > > Madhan Neethiraj > >
Re: Review Request 74278: ATLAS-4723 :- Atlas - Upgrade snakeyaml to 1.33
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74278/#review225099 --- Ship it! Ship It! - Jayendra Parab On Jan. 13, 2023, 8:23 a.m., Vinayak Marraiya wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74278/ > --- > > (Updated Jan. 13, 2023, 8:23 a.m.) > > > Review request for atlas, Jayendra Parab and Pinal Shah. > > > Bugs: ATLAS-4723 > https://issues.apache.org/jira/browse/ATLAS-4723 > > > Repository: atlas > > > Description > --- > > Currently Atlas is using snakeyaml version 1.32 > > > Diffs > - > > pom.xml ec6b39802 > > > Diff: https://reviews.apache.org/r/74278/diff/1/ > > > Testing > --- > > 1)mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > Atlas server runnning > 2)Quick started was passed successfully > > Pre-commit > :-https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test//1307/console > > > Thanks, > > Vinayak Marraiya > >
Re: Review Request 74273: ATLAS-4721 :- Atlas - Upgrade jettison to 1.5.2
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74273/#review225084 --- Ship it! Ship It! - Jayendra Parab On Jan. 6, 2023, 11:43 a.m., Vinayak Marraiya wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74273/ > --- > > (Updated Jan. 6, 2023, 11:43 a.m.) > > > Review request for atlas, Jayendra Parab and Pinal Shah. > > > Bugs: ATLAS-4721 > https://issues.apache.org/jira/browse/ATLAS-4721 > > > Repository: atlas > > > Description > --- > > Currently atlas is using jettison version 1.3.7 > > > Diffs > - > > pom.xml f8558cf2a > > > Diff: https://reviews.apache.org/r/74273/diff/1/ > > > Testing > --- > > 1)mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > Atlas server runnning > 2)Quick started was passed successfully > > Pre-commit :- > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1302/console > > > Thanks, > > Vinayak Marraiya > >
Re: Review Request 74269: ATLAS-4720 :- Atlas - Upgrade Netty to 4.1.86.Final
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74269/#review225054 --- Ship it! Ship It! - Jayendra Parab On Jan. 3, 2023, 6:55 p.m., Vinayak Marraiya wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74269/ > --- > > (Updated Jan. 3, 2023, 6:55 p.m.) > > > Review request for atlas, Jayendra Parab and Pinal Shah. > > > Bugs: ATLAS-4720 > https://issues.apache.org/jira/browse/ATLAS-4720 > > > Repository: atlas > > > Description > --- > > Currently atlas is using netty version 4.1.78.Final > > > Diffs > - > > pom.xml 125076def > > > Diff: https://reviews.apache.org/r/74269/diff/1/ > > > Testing > --- > > 1)mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > Atlas server runnning > 2)Quick started was passed successfully > > Pre-commit :- > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1299/console > > > Thanks, > > Vinayak Marraiya > >
Re: Review Request 74264: ATLAS-4718 :- Atlas - Upgrade Spring-security to 5.7.5
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74264/#review225017 --- Ship it! Ship It! - Jayendra Parab On Dec. 21, 2022, 5:39 a.m., Vinayak Marraiya wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74264/ > --- > > (Updated Dec. 21, 2022, 5:39 a.m.) > > > Review request for atlas, Jayendra Parab and Pinal Shah. > > > Bugs: ATLAS-4718 > https://issues.apache.org/jira/browse/ATLAS-4718 > > > Repository: atlas > > > Description > --- > > Currently Atlas is using Spring-security-version 5.7.2 upgrading it to 5.7.5 > > > Diffs > - > > pom.xml 496d2692e > > > Diff: https://reviews.apache.org/r/74264/diff/1/ > > > Testing > --- > > 1)mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > Atlas server runnning > 2)Quick started was passed successfully > > > Pre-commit :- > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1296/console > > > Thanks, > > Vinayak Marraiya > >
Re: [VOTE] Release Apache Atlas 2.3.0 rc1
Thank you Madhan for the release candidate. +1 for Apache Atlas 2.3.0 release candidate #1 Validated the following: - Able to download source, signature, md5 and sha512 files and verified checksum hash - Validated signature of source from release manager (Madhan Neethiraj < mad...@apache.org>) - Built the source successfully using embedded-hbase-solr profile - Started Atlas and ran quickstart script to preload sample types and entities - Validated relation search, basic and advanced search Thanks and regards, Jayendra. On Mon, Dec 5, 2022 at 1:02 PM Nixon Rodrigues wrote: > +1 for the Atlas 2.3.0 RC#1 for release > > The RC bits build looks good. > > Thanks Madhan for putting this together for release voting. > > Thanks and Regards > Nixon Rodrigues > > On Sun, 4 Dec 2022 at 20:58, Nikhil Bonte wrote: > > > +1 for the Apache Atlas 2.3.0 release candidate #1. > > > > Performed Following: > > - Able to download source, signature, md5 and sha512 files and verify the > > checksum hash > > > > - Build the source successfully using embedded-hbase-solr profile > > > > > > Thanks > > Nikhil P. Bonte > > > > On Sun, Dec 4, 2022 at 4:17 AM Ramesh Mani wrote: > > > > > +1 for the Apache Atlas 2.3.0 release candidate #1. > > > > > > Verified Build in CentOS7 > > > /apache-maven-3.8.6/bin/mvn clean -DskipTests > > > -Dmaven.wagon.http.ssl.insecure=true > -Dmaven.wagon.http.ssl.allowall=true > > > -Dmaven.wagon.http.ssl.ignore.validity.dates=true install > > > Verified Signature > > > gpg --verify apache-atlas-2.3.0-sources.tar.gz.asc > > > gpg: Signature made Mon 28 Nov 2022 09:14:22 PM UTC using > RSA > > > key ID B01179D4 > > > gpg: Good signature from "Madhan Neethiraj < > > mad...@apache.org > > > >" > > > gpg: WARNING: This key is not certified with a trusted > > > signature! > > > gpg: There is no indication that the signature > > > belongs to the owner. > > > Primary key fingerprint: 0524 DD1F 7940 6654 6D61 0744 FED4 67D3 B011 > > 79D4 > > > > > > Verified SHA256 and SHA512 > > > openssl dgst -sha256 apache-atlas-2.3.0-sources.tar.gz > > > SHA256(apache-atlas-2.3.0-sources.tar.gz)= > > > d56b89edd0499f109488b1d8394ec3f4f23609b9ea29a5099c473e84ec7e91c0 > > > > > > openssl dgst -sha512 apache-atlas-2.3.0-sources.tar.gz > > >SHA512(apache-atlas-2.3.0-sources.tar.gz)= > > > > > > > > > b24c9785f601b4644090ef0b4c16974fc572100b61f04ef2d6c68fdaa45293e902483f3574342a0e70c54f757c509d3d7ec6a85e494599b1018c65d8126d3480 > > > > > > Thanks, > > > Ramesh > > > > > > > > > [root@rmani-7216-1 tmp]# gpg --verify > > apache-atlas-2.3.0-sources.tar.gz.as > > > > > > On Mon, Nov 28, 2022 at 1:30 PM Madhan Neethiraj > > > wrote: > > > > > > > Atlas team, > > > > > > > > > > > > > > > > Apache Atlas 2.3.0 release candidate #1 is now available for a vote > > > within > > > > dev community. Links to the release artifacts are given below. Please > > > > review and vote. > > > > > > > > > > > > > > > > The vote will be open for at least 72 hours or until necessary votes > > are > > > > reached. > > > > > > > > [ ] +1 approve > > > > > > > > [ ] +0 no opinion > > > > > > > > [ ] -1 disapprove (and reason why) > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Madhan > > > > > > > > > > > > > > > > > > > > > > > > List of issues addressed in this release: > > > > > > > > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20Atlas%20AND%20fixVersion%20%3D%202.3.0%20ORDER%20BY%20key%20DESC > > > > > > > > > > > > > > > > Git tag for the release: > > > > https://github.com/apache/atlas/tree/release-2.3.0-rc1 > > > > > > > > > > > > > > > > Sources for the release: > > > > > > > > > > https://dist.apache.org/repos/dist/dev/atlas/2.3.0-rc1/apache-atlas-2.3.0-sources.tar.gz > > > > > > > > > > > > > > > > Source release verification: > > > > > > > > PGP Signature: > > > > > > > > > > https://dist.apache.org/repos/dist/dev/atlas/2.3.0-rc1/apache-atlas-2.3.0-sources.tar.gz.asc > > > > > > > > MD5 Hash: > > > > > > > > > > https://dist.apache.org/repos/dist/dev/atlas/2.3.0-rc1/apache-atlas-2.3.0-sources.tar.gz.md5 > > > > > > > > > > > > > > > > Keys to verify the signature of the release artifacts are available > at: > > > > https://dist.apache.org/repos/dist/dev/atlas/KEYS > > > > > > > > > > > > > >
Review Request 74186: ATLAS-4692 - Upgrade Apache Commons Text to 1.10.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74186/ --- Review request for atlas, Mandar Ambawane, Pinal Shah, Radhika Kundam, and Sidharth Mishra. Bugs: ATLAS-4692 https://issues.apache.org/jira/browse/ATLAS-4692 Repository: atlas Description --- Upgrading Commons Text library to 1.10.0 Diffs - addons/hbase-bridge/pom.xml d28f13dc3 addons/hbase-testing-util/pom.xml af6eb8e57 common/pom.xml 2562384cb graphdb/api/pom.xml 7c6fff077 graphdb/janus-hbase2/pom.xml 5a813de7e graphdb/janus/pom.xml ae156fc5a pom.xml be4068de9 Diff: https://reviews.apache.org/r/74186/diff/1/ Testing --- Conducted a sanity check Ran Pre-Commit link: https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1268/ Thanks, Jayendra Parab
[jira] [Created] (ATLAS-4692) Upgrade Apache Commons Text to 1.10.0
Jayendra Parab created ATLAS-4692: - Summary: Upgrade Apache Commons Text to 1.10.0 Key: ATLAS-4692 URL: https://issues.apache.org/jira/browse/ATLAS-4692 Project: Atlas Issue Type: Task Components: atlas-core Affects Versions: 2.2.0 Reporter: Jayendra Parab Assignee: Jayendra Parab Upgrade Apache Commons Text to 1.10.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4691) Discrepancy in the atlas debug metrics between the active and the not active servers
[ https://issues.apache.org/jira/browse/ATLAS-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab updated ATLAS-4691: -- Description: There is a discrepancy in the atlas debug metrics between the active and the not active servers. Steps to reproduce: - Setup Atlas in HA mode with two instances of Atlas - add "atlas.debug.metrics.enabled=true" to the Atlas configuration (conf/atlas-application.properties) - restart Atlas - open Atlas and create a new entity (e.g. hdfs_path) - check the debug metrics on both atlas instances using API call: https://:/api/atlas/admin/debug/metrics - The active instance returns correct metrics, whereas passive instance returns empty metrics The reason for this is that the debug metrics are maintained in-memory in the active server. When "admin/debug/metrics" API call is accessed on the passive server, the request isn't redirected to the active server like it does for the other APIs, hence the passive node returns empty metrics. was: There is a discrepancy in the atlas debug metrics between the active and the not active servers. Steps to reproduce: - Setup Atlas in HA mode with two instances of Atlas - add "atlas.debug.metrics.enabled=true" to the Atlas configuration (conf/atlas-application.properties_role_safety_valve) - restart Atlas - open Atlas and create a new entity (e.g. hdfs_path) - check the debug metrics on both atlas instances using API call: https://:/api/atlas/admin/debug/metrics - The active instance returns correct metrics, whereas passive instance returns empty metrics The reason for this is that the debug metrics are maintained in-memory in the active server. When "admin/debug/metrics" API call is accessed on the passive server, the request isn't redirected to the active server like it does for the other APIs, hence the passive node returns empty metrics. > Discrepancy in the atlas debug metrics between the active and the not active > servers > > > Key: ATLAS-4691 > URL: https://issues.apache.org/jira/browse/ATLAS-4691 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.2.0 > Reporter: Jayendra Parab >Assignee: Jayendra Parab >Priority: Major > > There is a discrepancy in the atlas debug metrics between the active and the > not active servers. > Steps to reproduce: > - Setup Atlas in HA mode with two instances of Atlas > - add "atlas.debug.metrics.enabled=true" to the Atlas configuration > (conf/atlas-application.properties) > - restart Atlas > - open Atlas and create a new entity (e.g. hdfs_path) > - check the debug metrics on both atlas instances using API call: > https://:/api/atlas/admin/debug/metrics > - The active instance returns correct metrics, whereas passive instance > returns empty metrics > The reason for this is that the debug metrics are maintained in-memory in the > active server. When "admin/debug/metrics" API call is accessed on the passive > server, the request isn't redirected to the active server like it does for > the other APIs, hence the passive node returns empty metrics. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Review Request 74180: ATLAS-4691 - Discrepancy in the atlas debug metrics between the active and the not active servers
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74180/
---
Review request for atlas, Mandar Ambawane, Pinal Shah, Radhika Kundam, and
Sidharth Mishra.
Bugs: ATLAS-4691
https://issues.apache.org/jira/browse/ATLAS-4691
Repository: atlas
Description
---
In HA setup, there is difference in response when debug metrics API
("admin/debug/metrics") is accessed on active server and passive server.
The reason for this is debug metrics are maintained in-memory in the active
server. When "admin/debug/metrics" API call is accessed on the passive server,
the request isn't redirected to the active server like it does for the other
APIs, hence the passive node returns empty metrics.
Adding "admin/debug/metrics" to the list of adminUriNotFiltered, so that the
API call gets redirected to active server.
Diffs
-
webapp/src/main/java/org/apache/atlas/web/filters/ActiveServerFilter.java
a1d30ba87
Diff: https://reviews.apache.org/r/74180/diff/1/
Testing
---
Tested manually on a HA setup, with this change, when the "admin/debug/metrics"
is sent to passive server, the request is redirected to active server and
correct response is returned
Thanks,
Jayendra Parab
[jira] [Assigned] (ATLAS-4691) Discrepancy in the atlas debug metrics between the active and the not active servers
[ https://issues.apache.org/jira/browse/ATLAS-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab reassigned ATLAS-4691: - Assignee: Jayendra Parab > Discrepancy in the atlas debug metrics between the active and the not active > servers > > > Key: ATLAS-4691 > URL: https://issues.apache.org/jira/browse/ATLAS-4691 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.2.0 > Reporter: Jayendra Parab >Assignee: Jayendra Parab >Priority: Major > > There is a discrepancy in the atlas debug metrics between the active and the > not active servers. > Steps to reproduce: > - Setup Atlas in HA mode with two instances of Atlas > - add "atlas.debug.metrics.enabled=true" to the Atlas configuration > (conf/atlas-application.properties_role_safety_valve) > - restart Atlas > - open Atlas and create a new entity (e.g. hdfs_path) > - check the debug metrics on both atlas instances using API call: > https://:/api/atlas/admin/debug/metrics > - The active instance returns correct metrics, whereas passive instance > returns empty metrics > The reason for this is that the debug metrics are maintained in-memory in the > active server. When "admin/debug/metrics" API call is accessed on the passive > server, the request isn't redirected to the active server like it does for > the other APIs, hence the passive node returns empty metrics. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ATLAS-4691) Discrepancy in the atlas debug metrics between the active and the not active servers
Jayendra Parab created ATLAS-4691: - Summary: Discrepancy in the atlas debug metrics between the active and the not active servers Key: ATLAS-4691 URL: https://issues.apache.org/jira/browse/ATLAS-4691 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 2.2.0 Reporter: Jayendra Parab There is a discrepancy in the atlas debug metrics between the active and the not active servers. Steps to reproduce: - Setup Atlas in HA mode with two instances of Atlas - add "atlas.debug.metrics.enabled=true" to the Atlas configuration (conf/atlas-application.properties_role_safety_valve) - restart Atlas - open Atlas and create a new entity (e.g. hdfs_path) - check the debug metrics on both atlas instances using API call: https://:/api/atlas/admin/debug/metrics - The active instance returns correct metrics, whereas passive instance returns empty metrics The reason for this is that the debug metrics are maintained in-memory in the active server. When "admin/debug/metrics" API call is accessed on the passive server, the request isn't redirected to the active server like it does for the other APIs, hence the passive node returns empty metrics. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: Review Request 74176: ATLAS-4674 - Regression: Classification tagging is not sending appropriate notification to ATLAS_ENTITIES
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74176/#review224817 --- Ship it! Ship It! - Jayendra Parab On Oct. 18, 2022, 7:57 p.m., Disha Talreja wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74176/ > --- > > (Updated Oct. 18, 2022, 7:57 p.m.) > > > Review request for atlas and Jayendra Parab. > > > Bugs: ATLAS-4674 > https://issues.apache.org/jira/browse/ATLAS-4674 > > > Repository: atlas > > > Description > --- > > When we add/delete a classification from an entity, we send notification to > ATLAS_ENTITIES so ranger tag sync works appropriately to facilitate Tag-based > policy. This functionally seems to be broken and is a regression. > > > On further investigation, it is seen that the issue is caused by deferred > action for classification propagation as the right notifications are sent > when deferred action is disabled. > > > Entity details for the notification messages are fetched from the entity > cache which is not updated and lacks the new classification mapping. Hence, > this patch clears the cache so that the entity is newly mapped from it's > vertex and contains all the information. > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationTask.java > 74bed571f > > > Diff: https://reviews.apache.org/r/74176/diff/2/ > > > Testing > --- > > mvn clean -DskipTests install > mvn clean -DskipTests package -Pdist,embedded-hbase-solr > Atlas server running. > Classification add, Classification delete: Verified appropriate messages > being received on ATLAS_ENTITIES. > > > Thanks, > > Disha Talreja > >
Re: Review Request 74176: ATLAS-4674 - Regression: Classification tagging is not sending appropriate notification to ATLAS_ENTITIES
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74176/#review224814 --- repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationTask.java Line 77 (original), 77 (patched) <https://reviews.apache.org/r/74176/#comment313649> Consider clearing the RequestContext at the start of the method, before its used anywhere. - Jayendra Parab On Oct. 18, 2022, 6:45 p.m., Disha Talreja wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74176/ > --- > > (Updated Oct. 18, 2022, 6:45 p.m.) > > > Review request for atlas and Jayendra Parab. > > > Bugs: ATLAS-4674 > https://issues.apache.org/jira/browse/ATLAS-4674 > > > Repository: atlas > > > Description > --- > > When we add/delete a classification from an entity, we send notification to > ATLAS_ENTITIES so ranger tag sync works appropriately to facilitate Tag-based > policy. This functionally seems to be broken and is a regression. > > > On further investigation, it is seen that the issue is caused by deferred > action for classification propagation as the right notifications are sent > when deferred action is disabled. > > > Entity details for the notification messages are fetched from the entity > cache which is not updated and lacks the new classification mapping. Hence, > this patch clears the cache so that the entity is newly mapped from it's > vertex and contains all the information. > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationTask.java > 74bed571f > > > Diff: https://reviews.apache.org/r/74176/diff/1/ > > > Testing > --- > > mvn clean -DskipTests install > mvn clean -DskipTests package -Pdist,embedded-hbase-solr > Atlas server running. > Classification add, Classification delete: Verified appropriate messages > being received on ATLAS_ENTITIES. > > > Thanks, > > Disha Talreja > >
Re: Review Request 74109: ATLAS-4671 : Basic Search : Exclude Header attributes of entities from the response
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74109/#review224706
---
Ship it!
Ship It!
- Jayendra Parab
On Sept. 21, 2022, 9:48 a.m., Pinal Shah wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74109/
> ---
>
> (Updated Sept. 21, 2022, 9:48 a.m.)
>
>
> Review request for atlas, Jayendra Parab, Radhika Kundam, and Sidharth Mishra.
>
>
> Bugs: ATLAS-4671
> https://issues.apache.org/jira/browse/ATLAS-4671
>
>
> Repository: atlas
>
>
> Description
> ---
>
> **Issue** : Basic search has AtlasEntityHeader of each entity in the response.
> AtlasEntityHeader has many attributes including classification and terms.
> hence for each attribute, it will request janusgraph
>
> **Approach** : To overcome, we can add a flag to exclude other attributes and
> add only selected attributes from 'attributes' field in the response
>
> **Working** : This improvement can be seen, if user has **entityTypes**,
> **excludeHeaderAttributes=true** and **valid entity attributes (not
> relationship) in 'attributes' field** , in the request payload.
>
> **Request**: {
> "excludeDeletedEntities": true,
> "includeSubClassifications": true,
> "includeSubTypes": true,
> "includeClassificationAttributes": true,
> "limit": 25,
> "offset": 0,
> "typeName": "hdfs_path",
> "attributes": ["path", "name"],
> "excludeHeaderAttributes": "true"
>}
>
> **Response**: {
> "queryType": "BASIC",
> "searchParameters": {
> "typeName": "hdfs_path",
> "excludeDeletedEntities": true,
> "includeClassificationAttributes": true,
> "includeSubTypes": true,
> "includeSubClassifications": true,
> "limit": 25,
> "offset": 0,
> "attributes": ["path", "name"]
> },
> "attributes": {
> "name": ["path", "name"],
> "values": [
> ["/data/warehouse/customer", "customer"],
> ["/data/warehouse/sales", "sales"]
> ]
> },
> "approximateCount": 2
> }
>
>
> Diffs
> -
>
>
> intg/src/main/java/org/apache/atlas/model/discovery/QuickSearchParameters.java
> 79f5aae0d
> intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java
> 78fb4a48f
>
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
> 8fbc22fa0
> repository/src/main/java/org/apache/atlas/discovery/SearchContext.java
> 01954d07e
>
> repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
> fbc739652
>
>
> Diff: https://reviews.apache.org/r/74109/diff/6/
>
>
> Testing
> ---
>
> Unit testcases added
> Precommit :
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1215/
>
> Performance Readings:
> Basic Search on 20K entities, limit 1000
>
>
> Thanks,
>
> Pinal Shah
>
>
Re: Review Request 74126: ATLAS-4677 : Atlas - Upgrade commons-configuration to 2.8.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74126/#review224679 --- Ship it! Ship It! - Jayendra Parab On Sept. 19, 2022, 5:21 a.m., chaitali wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74126/ > --- > > (Updated Sept. 19, 2022, 5:21 a.m.) > > > Review request for atlas, Jayendra Parab and Pinal Shah. > > > Bugs: ATLAS-4677 > https://issues.apache.org/jira/browse/ATLAS-4677 > > > Repository: atlas > > > Description > --- > > Current version for commons-configuration2 is 2.2 > > > Diffs > - > > pom.xml d5aabbf24 > > > Diff: https://reviews.apache.org/r/74126/diff/1/ > > > Testing > --- > > Testcases passed > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1219/console > > > Thanks, > > chaitali > >
Re: Review Request 74103: ATLAS-4669 : Atlas - Upgrade Junit to 4.13.2
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74103/#review224653 --- Ship it! Ship It! - Jayendra Parab On Sept. 8, 2022, 10:01 a.m., chaitali wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74103/ > --- > > (Updated Sept. 8, 2022, 10:01 a.m.) > > > Review request for atlas, Jayendra Parab and Pinal Shah. > > > Bugs: ATLAS-4669 > https://issues.apache.org/jira/browse/ATLAS-4669 > > > Repository: atlas > > > Description > --- > > currently Junit version is 4.13.1 > > > Diffs > - > > addons/hbase-bridge/pom.xml a6ed51421 > pom.xml a5d4b1f9e > > > Diff: https://reviews.apache.org/r/74103/diff/2/ > > > Testing > --- > > mvn clean -DskipTests package -Pdist,embedded-hbase-solr > mvn clean install -DskipTests -Drat.skip > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1210/consoleFull > > > Thanks, > > chaitali > >
Re: Review Request 74093: ATLAS-4662 : Upgrade POI to 5.2.2
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74093/#review224637 --- Ship it! Ship It! - Jayendra Parab On Aug. 23, 2022, 7:43 a.m., chaitali wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74093/ > --- > > (Updated Aug. 23, 2022, 7:43 a.m.) > > > Review request for atlas, Jayendra Parab and Pinal Shah. > > > Bugs: ATLAS-4662 > https://issues.apache.org/jira/browse/ATLAS-4662 > > > Repository: atlas > > > Description > --- > > POI and POI-ooxml version is 4.1.1 > > > Diffs > - > > pom.xml 548382446 > > > Diff: https://reviews.apache.org/r/74093/diff/1/ > > > Testing > --- > > mvn clean install > apache server running fine > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1198/console > > > Thanks, > > chaitali > >
Re: Review Request 74055: ATLAS-4633: Multiple typos in official Apache Atlas Docs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74055/#review224562 --- Ship it! Ship It! - Jayendra Parab On July 14, 2022, 5:36 a.m., Umesh Padashetty wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74055/ > --- > > (Updated July 14, 2022, 5:36 a.m.) > > > Review request for atlas, Jayendra Parab, keval bhatt, Madhan Neethiraj, > Mandar Ambawane, Nixon Rodrigues, Pinal Shah, Prasad Pawar, and Sarath > Subramanian. > > > Bugs: ATLAS-4633 > https://issues.apache.org/jira/browse/ATLAS-4633 > > > Repository: atlas > > > Description > --- > > Multiple typos in official Apache Atlas Docs > > > Diffs > - > > docs/src/documents/Asf/asfinfo.md 99f7db675 > docs/src/documents/BusinessMetadata.md 65755fb35 > docs/src/documents/Downloads/Downloads.md b8bf09250 > docs/src/documents/Glossary.md e6e1439d9 > docs/src/documents/HighAvailability.md ed0fe8b9f > docs/src/documents/Hook/HookHBase.md cbb8d81f2 > docs/src/documents/Hook/HookHive.md 0fba4f5fa > docs/src/documents/Hook/HookSqoop.md 3f600bb05 > docs/src/documents/Hook/HookStorm.md 37b23e542 > docs/src/documents/Import-Export/ExportAPI.md 849d452ab > docs/src/documents/Import-Export/ImportAPIOptions.md 1f6c8e39d > docs/src/documents/Import-Export/ImportEntityTransforms.md f1676b7c0 > docs/src/documents/Import-Export/ImportExportAPI.md c903c17a3 > docs/src/documents/Migration/Migration-0.8-to-1.0.md 7efd5752b > docs/src/documents/Misc/AtlasServer.md ee33c3cce > docs/src/documents/Misc/SoftReference.md 31ad20321 > docs/src/documents/Notifications.md 34d5c1a4b > docs/src/documents/Overview.md 3a82656ad > docs/src/documents/Project-Info/MailingLists.md 884330363 > docs/src/documents/Project-Info/TeamList.md d260fa353 > docs/src/documents/Search/SearchAdvanced.md c22981e1b > docs/src/documents/Security/AtlasRangerAuthorizer.md 9b019ec12 > docs/src/documents/Security/AtlasSimpleAuthorizer.md fc470d742 > docs/src/documents/Security/Authentication.md 3bdb5f7e8 > docs/src/documents/Security/Security.md af7d80e51 > docs/src/documents/Setup/BuildInstruction.md b067676cd > docs/src/documents/Setup/Configuration.md b72a0cc44 > docs/src/documents/Setup/EclipseSetup.md 1092e8319 > docs/src/documents/Setup/InstallationInstruction.md 100eaf51e > docs/src/documents/Tools/AtlasRepairIndex.md 0161cb1cb > docs/src/documents/TypeSystem.md 7d6125591 > docs/src/documents/Whats-New/WhatsNew-2.0.md 724e8ab31 > docs/src/documents/Whats-New/WhatsNew-2.1.md 82079f9f5 > > > Diff: https://reviews.apache.org/r/74055/diff/1/ > > > Testing > --- > > > Thanks, > > Umesh Padashetty > >
Re: Review Request 74028: ATLAS-4624 : Upgrade gson to 2.9.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74028/#review224528 --- Ship it! Ship It! - Jayendra Parab On June 24, 2022, 8:52 a.m., Vinayak Marraiya wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74028/ > --- > > (Updated June 24, 2022, 8:52 a.m.) > > > Review request for atlas, Jayendra Parab and Pinal Shah. > > > Bugs: ATLAS-4624 > https://issues.apache.org/jira/browse/ATLAS-4624 > > > Repository: atlas > > > Description > --- > > Currently Atlas is using gson version 2.5 upgrading it to 2.9.0 > > > Diffs > - > > pom.xml c83099f9a > > > Diff: https://reviews.apache.org/r/74028/diff/1/ > > > Testing > --- > > 1)mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > Atlas server runnning > 2)Quick started was passed successfully > > Pre-Commit - > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1131/console > > > Thanks, > > Vinayak Marraiya > >
Re: Review Request 74023: ATLAS-4618 :- Atlas - Upgrade netty to 4.1.77
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74023/#review224515 --- Ship it! Ship It! - Jayendra Parab On June 16, 2022, 3:30 p.m., Vinayak Marraiya wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74023/ > --- > > (Updated June 16, 2022, 3:30 p.m.) > > > Review request for atlas, Jayendra Parab and Pinal Shah. > > > Bugs: ATLAS-4618 > https://issues.apache.org/jira/browse/ATLAS-4618 > > > Repository: atlas > > > Description > --- > > Currently Atlas is using netty version 4.1.72 upgrading it to 4.1.77 > > > Diffs > - > > pom.xml faf294e42 > > > Diff: https://reviews.apache.org/r/74023/diff/1/ > > > Testing > --- > > 1)mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > Atlas server runnning > 2)Quick started was passed successfully > > > Pre-commit :- > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1124/console > > > Thanks, > > Vinayak Marraiya > >
Re: Review Request 74021: ATLAS-4617 :- Atlas - Upgrade Spring Framework to 5.3.20
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74021/#review224510 --- Ship it! Ship It! - Jayendra Parab On June 15, 2022, 9:47 a.m., Vinayak Marraiya wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74021/ > --- > > (Updated June 15, 2022, 9:47 a.m.) > > > Review request for atlas, Jayendra Parab and Pinal Shah. > > > Bugs: ATLAS-4617 > https://issues.apache.org/jira/browse/ATLAS-4617 > > > Repository: atlas > > > Description > --- > > Currently Atlas is using Spring framework version 5.3.18 upgrading it to > 5.3.20 > > > Diffs > - > > pom.xml faf294e42 > > > Diff: https://reviews.apache.org/r/74021/diff/1/ > > > Testing > --- > > 1)mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > Atlas server runnning > 2)Quick started was passed successfully > > > Pre-commit :- > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1119/console > > > Thanks, > > Vinayak Marraiya > >
Re: Review Request 73974: ATLAS-4614 Deferred Actions : Support to add classification name instead of classificationVertexId
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73974/#review224508
---
Ship it!
Ship It!
- Jayendra Parab
On June 13, 2022, 6:46 a.m., Paresh Devalia wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73974/
> ---
>
> (Updated June 13, 2022, 6:46 a.m.)
>
>
> Review request for atlas, Jayendra Parab, Mandar Ambawane, and Pinal Shah.
>
>
> Bugs: ATLAS-4614
> https://issues.apache.org/jira/browse/ATLAS-4614
>
>
> Repository: atlas
>
>
> Description
> ---
>
> In /api/atlas/admin/tasks call , when a tag is propagated , the
> classificationVertexId is displayed. It becomes tough to debug when there are
> many pending tasks . classification name can be added in the response in
> addition to/instead of classificationVertexId.
>
>
> Diffs
> -
>
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
> 6964211e9
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
> 9a7f29017
>
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationTask.java
> 0bad84e49
>
>
> Diff: https://reviews.apache.org/r/73974/diff/2/
>
>
> Testing
> ---
>
> Manual testing is done.
>
> Pre-commit :
> https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1096/consoleFull
>
> Created 1000 hive_tables dependant on parent table.
> Hive_tables name will be sequence_1 to sequence_999
>
> Below step are tested for add, update and delete classification.
>
> 1) Adding classification :-
> Create a classification with name test_1.
> Added test_1 classification with checked on "propagate" of
> sequence_1 hive_table, so this classification will be assign to all child
> table.
> So while background process is going on, will check the task
> status API.
> curl k -u : -request GET
> 'https://:/api/atlas/admin/tasks'
> so in response will be
> [
> {
> "type":
> "CLASSIFICATION_PROPAGATION_ADD",
> "guid":
> "39f68061-2c5b-4e0c-9ba6-65b3d8117e99",
> "createdBy": "admin",
> "createdTime": 1651591949394,
> "updatedTime": 1651591949394,
> "parameters": {
> "relationshipGuid": null,
> "classificationName": "test_1",
> "entityGuid":
> "29a5ac89-a3fb-46ff-bbb1-646d3b0717dd",
> "classificationVertexId":
> "122896440"
> },
> "attemptCount": 0,
> "status": "PENDING"
> }
> ]
>
> 2)Updating classification attribute value:-
> Creating attribute for test_1 classification.
> Updating classification attribute value on sequence_1
> hive_table, then in background process it will update all the child table
> attribute value.
> So while background process is going on, will check the task
> status API.
> curl k -u : -request GET
> 'https://:/api/atlas/admin/tasks'
> so in response will be
> [
> {
> "type":
> "CLASSIFICATION_PROPAGATION_ADD",
> "guid":
> "39f68061-2c5b-4e0c-9ba6-65b3d8117e99",
> "createdBy": "admin",
> "createdTime": 1651591949394,
> "updatedTime": 1651591949394,
> "p
Re: Review Request 74005: ATLAS-4609 :- Atlas - Upgrade Spring-security to 5.7.1
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74005/#review224489 --- Ship it! Ship It! - Jayendra Parab On June 9, 2022, 5:55 p.m., Vinayak Marraiya wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74005/ > --- > > (Updated June 9, 2022, 5:55 p.m.) > > > Review request for atlas, Jayendra Parab and Pinal Shah. > > > Bugs: ATLAS-4609 > https://issues.apache.org/jira/browse/ATLAS-4609 > > > Repository: atlas > > > Description > --- > > Currently Atlas is using Spring-security version 5.5.1 upgrading it to 5.7.1 > > > Diffs > - > > pom.xml 29d769296 > > > Diff: https://reviews.apache.org/r/74005/diff/1/ > > > Testing > --- > > 1)mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > Atlas server runnning > 2)Quick started was passed successfully > > Pre-commit :- > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1113/console > > > Thanks, > > Vinayak Marraiya > >
Re: Review Request 73819: ATLAS-4539 :- Atlas - Upgrade Spring framework to 5.3.15
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73819/#review224001 --- Ship it! Ship It! - Jayendra Parab On Jan. 25, 2022, 11:56 a.m., Vinayak Marraiya wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73819/ > --- > > (Updated Jan. 25, 2022, 11:56 a.m.) > > > Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, and > Sarath Subramanian. > > > Bugs: ATLAS-4539 > https://issues.apache.org/jira/browse/ATLAS-4539 > > > Repository: atlas > > > Description > --- > > Currently Atlas is using Spring framework version 5.3.13 upgrading it to > 5.3.15 > > > Diffs > - > > pom.xml 6ebef6024 > > > Diff: https://reviews.apache.org/r/73819/diff/2/ > > > Testing > --- > > 1)mvn clean install > Atlas server runnning > 2)Quick started was passed successfully > > Precommit - > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1025/console > > > Thanks, > > Vinayak Marraiya > >
Re: Review Request 73797: ATLAS - 4527 : Upgrade underscore.js to 1.12.1+
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73797/#review223954 --- Ship it! Ship It! - Jayendra Parab On Jan. 12, 2022, 6:57 a.m., Farhan Khan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73797/ > --- > > (Updated Jan. 12, 2022, 6:57 a.m.) > > > Review request for atlas and Prasad Pawar. > > > Bugs: ATLAS-4527 > https://issues.apache.org/jira/browse/ATLAS-4527 > > > Repository: atlas > > > Description > --- > > Atlas is using underscore 1.8.3. This needs to be upgraded to 1.12.1+ > > > Diffs > - > > dashboardv2/package-lock.json 61fd944c6 > dashboardv2/package.json 2cc1d4b08 > dashboardv2/public/js/utils/Globals.js 8fe8ebc4c > dashboardv3/package-lock.json 2b00b159e > dashboardv3/package.json 49fd9dfac > dashboardv3/public/js/utils/Globals.js df17a683b > > > Diff: https://reviews.apache.org/r/73797/diff/1/ > > > Testing > --- > > Functional testing has been done for all UI pages. > > > Thanks, > > Farhan Khan > >
Re: Review Request 73740: ATLAS-4491 :- Atlas - Upgrade Spring framework to 5.3.10+/5.2.17+
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73740/#review223807 --- Ship it! Ship It! - Jayendra Parab On Dec. 6, 2021, 9:32 a.m., Vinayak Marraiya wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73740/ > --- > > (Updated Dec. 6, 2021, 9:32 a.m.) > > > Review request for atlas, Ashutosh Mestry, chaitali, Jayendra Parab, Mandar > Ambawane, Pinal Shah, and Sarath Subramanian. > > > Repository: atlas > > > Description > --- > > Atlas upgraded to 5.3.10 or higher > > > Diffs > - > > pom.xml 4b3408acc > > > Diff: https://reviews.apache.org/r/73740/diff/2/ > > > Testing > --- > > 1)mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > Atlas server runnning > 2)Quick started was passed successfully > > > Precommit - > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/982/console > > > Thanks, > > Vinayak Marraiya > >
Re: Review Request 73722: ATLAS-4491 :- Atlas - Upgrade Spring framework to 5.3.10+/5.2.17
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73722/#review223806 --- Ship it! Ship It! - Jayendra Parab On Nov. 23, 2021, 6:21 a.m., Vinayak Marraiya wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73722/ > --- > > (Updated Nov. 23, 2021, 6:21 a.m.) > > > Review request for atlas, Ashutosh Mestry, chaitali, Jayendra Parab, Mandar > Ambawane, Pinal Shah, and Sarath Subramanian. > > > Bugs: ATLAS-4491 > https://issues.apache.org/jira/browse/ATLAS-4491 > > > Repository: atlas > > > Description > --- > > Atlas upgraded to 5.3.10 or higher > > > Diffs > - > > pom.xml 86519daba > > > Diff: https://reviews.apache.org/r/73722/diff/1/ > > > Testing > --- > > 1)mvn clean package -Pdist,embedded-hbase-solr -DskipTests -Drat.skip > Atlas server runnning > 2)Quick started was passed successfully > > Precommit - > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/982/console > > > Thanks, > > Vinayak Marraiya > >
[jira] [Commented] (ATLAS-4459) Fix solr indexes for testcases failures
[ https://issues.apache.org/jira/browse/ATLAS-4459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17432254#comment-17432254 ] Jayendra Parab commented on ATLAS-4459: --- +1 > Fix solr indexes for testcases failures > --- > > Key: ATLAS-4459 > URL: https://issues.apache.org/jira/browse/ATLAS-4459 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Pinal Shah >Assignee: Pinal Shah >Priority: Major > Labels: Testing > Attachments: ATLAS-4459.patch > > > Fixing solr indexes which are used for the testcases in solrConfig.xml > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4458) Commons-Logging Exclusion Causes Startup Problems
[ https://issues.apache.org/jira/browse/ATLAS-4458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431443#comment-17431443 ] Jayendra Parab commented on ATLAS-4458: --- +1 > Commons-Logging Exclusion Causes Startup Problems > - > > Key: ATLAS-4458 > URL: https://issues.apache.org/jira/browse/ATLAS-4458 > Project: Atlas > Issue Type: Bug >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Attachments: ATLAS-4458-Commons-logging-reference-fix.patch > > > Earlier commit of ATLAS-4351, caused _commons-logging*.jar_ to be excluded. > This causes startup problem due to missing dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [VOTE] Release Apache Atlas version 2.2.0 - rc1
Thanks Sidharth for putting this together for vote.
+1 for 2.2.0 - rc1 release.
Verified following:
* MD5, SHA512, release signature
* mvn build with embedded-hbase-solr
* Setup Atlas server with embedded HBase and Solr.
* Ran quick_start.py
* Atlas UI loading cleanly.
* Created entities using Atlas UI
* Validated basic and DSL search
On Wed, Aug 11, 2021 at 1:14 PM Mandar Ambawane <
mandar.ambaw...@freestoneinfotech.com> wrote:
> Hi Sidharth,
>
> +1 for Apache Atlas 2.2.0 rc1
>
> Verified following:
>
>- md5, sha512 signatures
>- mvn build with embedded-hbase-solr -DskipTests
>- Started Atlas server and did run quick start
>- Atlas UI loading cleanly
>- Created entities using Atlas UI
>- Validated basic search
>- Validated Admin audits
>
> Thanks,
> Mandar
>
> On Wed, Aug 11, 2021 at 12:06 PM Chaitali Borole <
> chaitali.bor...@freestoneinfotech.com> wrote:
>
> > Hello Sidharth,
> >
> > +1 for Apache Atlas 2.2.0 rc1
> >
> > Checked for below points:
> > * md5, sha512 signatures.
> > * Clean build with mvn clean package -Pdist,embedded-hbase-solr
> > -DskipTests -Drat.skip.
> > * Atlas server up and running.
> > * Re-indexing with JAVA_PATCH.
> > * Sanity checks :
> > quick_search,entities,relationship,tags,glossaries,metrics api.
> > * Quick start is successful.
> >
> > Thanks,
> > Chaitali.
> >
> > On Tue, Aug 10, 2021 at 4:25 AM Sidharth Mishra
> > wrote:
> >
> > > Atlas team,
> > >
> > > Apache Atlas 2.2.0 rc1, with the following fixes since rc0, is now
> > > available for vote within the dev community.
> > >
> > > ATLAS-4390 - Changed Atlas branch to release-2.2.0-rc1
> > > ATLAS-4391 - NoSuchFileException for
> > {atlas.log.dir}/atlas-metrics.out
> > > ATLAS-4343 - [Entity Audits] 'Propagated Classification Added'
> > > Timestamp is < 'Entity
> > > Created' Timestamp
> > > ATLAS-4387 - UI: Propagated Classifications shows wrong count for
> > > entity type on Entity detail
> > > page
> > > ATLAS-4386 - Relationship search: Sorting does not work when
> > > relationship label is passed
> > >
> > > Links to the release artifacts are given below. Please review and vote.
> > >
> > > The vote will be open for at least 72 hours or until necessary votes
> > > are reached.
> > >
> > > [ ] +1 Approve
> > >
> > > [ ] +0 No opinion
> > >
> > > [ ] -1 Disapprove (and reason why)
> > >
> > > Thanks,
> > > Sid
> > >
> > > List of all issues addressed in this release:
> > >
> > >
> >
> https://issues.apache.org/jira/issues/?jql=project%3DATLAS%20AND%20status%3DResolved%20AND%20fixVersion%3D2.2.0%20ORDER%20BY%20key%20DESC
> > >
> > > Git tag for the release:
> > > https://github.com/apache/atlas/tree/release-2.2.0-rc1
> > >
> > > Sources for the release:
> > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/atlas/2.2.0-rc1/apache-atlas-2.2.0-sources.tar.gz
> > >
> > > Source release verification:
> > >
> > > PGP Signature:
> > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/atlas/2.2.0-rc1/apache-atlas-2.2.0-sources.tar.gz.asc
> > >
> > > SHA512 Hash:
> > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/atlas/2.2.0-rc1/apache-atlas-2.2.0-sources.tar.gz.sha512
> > >
> > > MD5 Hash:
> > >
> > >
> >
> https://dist.apache.org/repos/dist/dev/atlas/2.2.0-rc1/apache-atlas-2.2.0-sources.tar.gz.md5
> > >
> > >
> > > Keys to verify the signature of the release artifacts are available
> > > at: https://dist.apache.org/repos/dist/release/atlas/KEYS
> > >
> > >
> > > New features/enhancements:
> > >
> > > - Introduced Deferred Action - Tag propagation will be handled as
> > > a background process (AtlasTask)
> > >
> > > - Added Re-indexing as JAVA_PATCH
> > >
> > > - Created JavaPatch to add new super types to existing entities
> > >
> > > - Supported for Business Metadata in Atlas Export API
> > >
> > > - Supported HA for admin/task API
> > >
> > > - Enhanced export service to export Terms
> > >
> > > - Added new models and addition attributes to Atlas for Apache
> > > Flink, GCP and ADLS Gen 2
> > >
> > > - Added support to Configure HiveServer2 Hook to send Lineage-only
> > > Messages
> > >
> > > - Improved and enhanced Bulk Glossary Import supporting import
> > > with relations
> > >
> > > - Dependency between Atlas and Kafka Zookeeper removed
> > >
> > > - Improved performance of GraphHelper's guid and status getter
> > methods
> > >
> > > - Added new privilege - "admin-audits" for Atlas Admin Audits
> > > authorization
> > >
> > > - Basic search Improvements - search text with special characters,
> > > optimize pagination
> > >
> > > - Updated free-text search processor to support Elasticsearch
> > >
> > > - Refactored and enhanced Atlas Python Client supporting Python 2.7
> > >
> > > - DSL Search enhancements - Support glossary terms and
> > > relationship, can handle null attribute values, Java Tinkerpop
> > >
Re: Planning for Apache Atlas 2.2.0 release
+1 for Apache Atlas 2.2.0 release Thanks Siddharth for volunteering! Regards, Jayendra. On Wed, Jul 28, 2021 at 2:28 PM Nixon Rodrigues < nixon.rodrig...@freestoneinfotech.com> wrote: > +1 for the Atlas 2.2.0 release. > > Thanks Siddharth for volunteering the release. > > Regards, > Nixon Rodrigue > > > On Wed, Jul 28, 2021 at 1:23 PM Madhan Neethiraj > wrote: > >> +1 for Apache Atlas 2.2.0 release. It has been a while now! >> >> >> >> Thanks Sid! >> >> >> >> Madhan >> >> >> >> >> >> *From: *Nikhil Bonte >> *Reply-To: *"u...@atlas.apache.org" >> *Date: *Tuesday, July 27, 2021 at 10:20 PM >> *To: *"u...@atlas.apache.org" >> *Cc: *"dev@atlas.apache.org" >> *Subject: *Re: Planning for Apache Atlas 2.2.0 release >> >> >> >> Thanks for volunteering Sidharth! >> >> >> >> +1 for the release Apache Atlas 2.2. >> >> >> >> >> >> Regards >> >> Nikhil P. Bonte >> >> >> >> >> >> On Wed, Jul 28, 2021 at 10:46 AM Ashutosh Mestry >> wrote: >> >> Sidharth: Thanks for volunteering! >> >> >> >> +1 for the release. >> >> >> >> *~ ashutosh* >> >> *Ashutosh Mestry* * . Staff Software Engineer . >> Cloudera, Inc. . * >> >> ... >> >> *No hurry, no pause. – Tim Ferriss, Life Hacker, Author* >> >> >> >> >> >> *From: *Sarath Subramanian >> *Date: *Tuesday, July 27, 2021 at 10:06 PM >> *To: *dev@atlas.apache.org >> *Cc: *u...@atlas.apache.org >> *Subject: *Re: Planning for Apache Atlas 2.2.0 release >> >> Thanks for volunteering to release Apache Atlas 2.2 >> >> +1 >> >> >> Thanks, >> Sarath >> >> >> On Tue, Jul 27, 2021 at 12:11 PM Sidharth Mishra >> wrote: >> >> > Dear Atlas community, >> > >> > Since Apache Atlas 2.1 was released, there are many new features, >> > improvements and fixes. Here are few features/enhancements since last >> > release, Apache Atlas 2.1: >> > >> > - Introduced Deferred Action - Tag propagation will be handled as >> > a background process (AtlasTask) >> > >> > - Added Re-indexing as JAVA_PATCH >> > >> > - Created JavaPatch to add new super types to existing entities >> > >> > - Supported for Business Metadata in Atlas Export API >> > >> > - Supported HA for admin/task API >> > >> > - Enhanced export service to export Terms >> > >> > - Added new models and addition attributes to Atlas for Apache >> > Flink, GCP and ADLS Gen 2 >> > >> > - Added support to Configure HiveServer2 Hook to send Lineage-only >> > Messages >> > >> > - Improved and enhanced Bulk Glossary Import supporting import >> > with relations >> > >> > - Dependency between Atlas and Kafka Zookeeper removed >> > >> > - Improved performance of GraphHelper's guid and status getter >> methods >> > >> > - Added new privilege - "admin-audits" for Atlas Admin Audits >> > authorization >> > >> > - Basic search Improvements - search text with special characters, >> > optimize pagination >> > >> > - Updated free-text search processor to support Elasticsearch >> > >> > - Refactored and enhanced Atlas Python Client supporting Python 2.7 >> > >> > - DSL Search enhancements - Support glossary terms and >> > relationship, can handle null attribute values, Java Tinkerpop >> > Traversals instead of GremlinScriptEngine, support search by >> > classification and its attribute, Caching mechanism for translated dsl >> > queries >> > >> > - Enhanced Export/Import Service to conditionally Support >> > Simultaneous Operations >> > >> > - Enhanced NotificationHookConsumer for Large Message Processing >> > >> > - Enhanced Atlas authorization for Add/Update/Remove >> > classification on entities >> > >> > - Provided option to add mandatory attribute to existing entity >> > definition >> > >> > - Fixed Atlas UI to load faster in case of large number of >> > classifications & entities >> > >> > - Updated versions of dependent libraries/components: JanusGraph, >> > elasticsearch, JQuery, Http core, Http Client, slf4j, log4j, ant, >> > gremlin, Solr, groovy, netty, Kafka >> > >> > There are around 336 plus commits on branch 2.0 and 205 resolved >> > issues with fix version 2.2.0 after 2.1.0 release. >> > With these significant improvements in place, it is time for the next >> > release of Apache Atlas! >> > >> > I propose to release Apache Atlas 2.2 by early next month. Please >> > review and send your comments. >> > >> > Regards, >> > Sidharth >> > >> >>
Re: Review Request 73341: ATLAS-4278 - Deferred Actions : Deleting a tag after disassociating it from a table which propagated fails
> On May 10, 2021, 3:44 p.m., Ashutosh Mestry wrote: > > Can you please update the jira/review with the error you saw in the log? Exception seen in the logs while trying to delete the classification is : ``` 2021-05-09 01:24:23,250 ERROR - [etp1992550266-179 - 55d97d73-3c4f-4985-81e2-696b41a3df1e:] ~ graph rollback due to exception (GraphTransactionInterceptor:200) org.apache.atlas.exception.AtlasBaseException: Given type class8 has references at org.apache.atlas.repository.store.graph.v2.AtlasClassificationDefStoreV2.preDeleteByGuid(AtlasClassificationDefStoreV2.java:313) at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.deleteTypesDef(AtlasTypeDefGraphStore.java:587) at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.deleteTypeByName(AtlasTypeDefGraphStore.java:713) at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$$FastClassBySpringCGLIB$$5226c80b.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:737) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:111) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:672) at org.apache.atlas.repository.store.graph.v2.AtlasTypeDefGraphStoreV2$$EnhancerBySpringCGLIB$$cc451202.deleteTypeByName() at org.apache.atlas.web.rest.TypesREST.deleteAtlasTypeByName(TypesREST.java:478) at org.apache.atlas.web.rest.TypesREST$$FastClassBySpringCGLIB$$1c2e37c6.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) ``` - Jayendra --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73341/#review222967 --- On May 9, 2021, 1:09 p.m., Jayendra Parab wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73341/ > --- > > (Updated May 9, 2021, 1:09 p.m.) > > > Review request for atlas, Ashutosh Mestry, Nikhil Bonte, Nixon Rodrigues, > Pinal Shah, and Sarath Subramanian. > > > Bugs: ATLAS-4278 > https://issues.apache.org/jira/browse/ATLAS-4278 > > > Repository: atlas > > > Description > --- > > When deferred action is enabled, even after dissociating the tag from all > entities the classification vertex created while associating the tag still > exists. > The error given "tag1 has references" is because of the classification vertex > created is still present in graphdb. > > When deferred action is disable, at the time of dissociating the tag, the > classification vertex is deleted from DeleteHandlerV1.deleteEdgeReference. > But when the deferred action is enabled, the classification vertex isn't > deleted because when DeleteHandlerV1.deleteEdgeReference is invoked, that > instance the classification vertex has references to the propagated entities. > In the background thread, which executes the dissociating of tags from > propagated entities, the classification vertex isn't deleted. > To resolve this issue, the classification vertex is deleted once the tag is > disassoociated from propagated entities. > > > Diffs > - > > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java > 02ba5ef8d > > > Diff: https://reviews.apache.org/r/73341/diff/1/ > > > Testing > --- > > With deferred action enabled: > Test 1: > - Associated a tag to an entity table1 which has lineage like table1 -> > process1 -> table2 > - Waited for tag propagation to complete > - Once tag is propagation is complete, deleted the tag1 from table1 > - Once the tag is removed from all the entities in the lineage above, deleted > the tag > - The tag is deleted properly > > Test 2: > - Associated a tag to an entity table1 which has lineage like table1 -> > process1 -> table2 > - Waited for tag propagation to complete > - Once tag is propagation is complete, deleted the tag1 from table1 > - Associated the same tag to an entity table3 which has lineage like table3 > -> process1 -> table4 > - Once the tag is removed from all the entities in the lineage above, deleted > the tag > - The tag is deleted properly > > > Thanks, > > Jayendra Parab > >
[jira] [Commented] (ATLAS-4278) Deferred Actions : Deleting a tag after disassociating it from a table which propagated fails
[
https://issues.apache.org/jira/browse/ATLAS-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342286#comment-17342286
]
Jayendra Parab commented on ATLAS-4278:
---
The exception seen in the logs at the time of deletion of classification is :
{code:java}
2021-05-09 01:24:23,250 ERROR - [etp1992550266-179 -
55d97d73-3c4f-4985-81e2-696b41a3df1e:] ~ graph rollback due to exception
(GraphTransactionInterceptor:200)
org.apache.atlas.exception.AtlasBaseException: Given type tag1 has references
at
org.apache.atlas.repository.store.graph.v2.AtlasClassificationDefStoreV2.preDeleteByGuid(AtlasClassificationDefStoreV2.java:313)
at
org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.deleteTypesDef(AtlasTypeDefGraphStore.java:587)
at
org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.deleteTypeByName(AtlasTypeDefGraphStore.java:713)
at
org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$$FastClassBySpringCGLIB$$5226c80b.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:737)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at
org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:111)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:672)
at
org.apache.atlas.repository.store.graph.v2.AtlasTypeDefGraphStoreV2$$EnhancerBySpringCGLIB$$cc451202.deleteTypeByName()
at
org.apache.atlas.web.rest.TypesREST.deleteAtlasTypeByName(TypesREST.java:478)
at
org.apache.atlas.web.rest.TypesREST$$FastClassBySpringCGLIB$$1c2e37c6.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
{code}
> Deferred Actions : Deleting a tag after disassociating it from a table which
> propagated fails
> -
>
> Key: ATLAS-4278
> URL: https://issues.apache.org/jira/browse/ATLAS-4278
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 2.1.0
> Reporter: Jayendra Parab
> Assignee: Jayendra Parab
>Priority: Major
>
> # Create a lineage : table1 ---> process1 > table2
> # Add tag tag1 to table1.
> # Wait for task to complete, it propagates to table2
> # Disassociate tag from table1
> # Wait for task to complete , it is removed from all tables and process.
> # Attempt to delete the tag.
> # Tag deletion fails with `tag1` has references , though the tag is not
> associated to any entity.
> If the tag was added to entity without propagate flag set to False , and then
> disassociated and deleted , it is deleted successfully.
> This issue is seen only with deferred actions enabled.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Review Request 73343: ATLAS-4279: Precommit is failing for DebugMetricsIT testcase
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73343/ --- (Updated May 10, 2021, 7:04 a.m.) Review request for atlas, Ashutosh Mestry, Nixon Rodrigues, and Sarath Subramanian. Summary (updated) - ATLAS-4279: Precommit is failing for DebugMetricsIT testcase Bugs: ATLAS-4279 https://issues.apache.org/jira/browse/ATLAS-4279 Repository: atlas Description --- DebugMetricsIT test case in introduced as part of commit for https://issues.apache.org/jira/browse/ATLAS-4106, is failing because the debug metrics are disabled by default and the test case expects the metrics to be enabled So to fix this case, enabled the debug metrics in the webapp/src/test/resources/atlas-application.properties file. Diffs - webapp/src/test/resources/atlas-application.properties adeade800 Diff: https://reviews.apache.org/r/73343/diff/1/ Testing --- Ran the precommit with this fix and the test case has passed https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/537/ Thanks, Jayendra Parab
Review Request 73343: Precommit is failing for DebugMetricsIT testcase
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73343/ --- Review request for atlas, Ashutosh Mestry, Nixon Rodrigues, and Sarath Subramanian. Bugs: ATLAS-4279 https://issues.apache.org/jira/browse/ATLAS-4279 Repository: atlas Description --- DebugMetricsIT test case in introduced as part of commit for https://issues.apache.org/jira/browse/ATLAS-4106, is failing because the debug metrics are disabled by default and the test case expects the metrics to be enabled So to fix this case, enabled the debug metrics in the webapp/src/test/resources/atlas-application.properties file. Diffs - webapp/src/test/resources/atlas-application.properties adeade800 Diff: https://reviews.apache.org/r/73343/diff/1/ Testing --- Ran the precommit with this fix and the test case has passed https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/537/ Thanks, Jayendra Parab
[jira] [Created] (ATLAS-4279) Precommit is failing for DebugMetricsIT testcase
Jayendra Parab created ATLAS-4279: - Summary: Precommit is failing for DebugMetricsIT testcase Key: ATLAS-4279 URL: https://issues.apache.org/jira/browse/ATLAS-4279 Project: Atlas Issue Type: Bug Reporter: Jayendra Parab Assignee: Jayendra Parab DebugMetricsIT test case in introduced as part of commit for https://issues.apache.org/jira/browse/ATLAS-4106, is failing because the debug metrics are disabled by default and the test case expects the metrics to be enabled -- This message was sent by Atlassian Jira (v8.3.4#803005)
Review Request 73341: ATLAS-4278 - Deferred Actions : Deleting a tag after disassociating it from a table which propagated fails
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73341/ --- Review request for atlas, Ashutosh Mestry, Nikhil Bonte, Nixon Rodrigues, Pinal Shah, and Sarath Subramanian. Bugs: ATLAS-4278 https://issues.apache.org/jira/browse/ATLAS-4278 Repository: atlas Description --- When deferred action is enabled, even after dissociating the tag from all entities the classification vertex created while associating the tag still exists. The error given "tag1 has references" is because of the classification vertex created is still present in graphdb. When deferred action is disable, at the time of dissociating the tag, the classification vertex is deleted from DeleteHandlerV1.deleteEdgeReference. But when the deferred action is enabled, the classification vertex isn't deleted because when DeleteHandlerV1.deleteEdgeReference is invoked, that instance the classification vertex has references to the propagated entities. In the background thread, which executes the dissociating of tags from propagated entities, the classification vertex isn't deleted. To resolve this issue, the classification vertex is deleted once the tag is disassoociated from propagated entities. Diffs - repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java 02ba5ef8d Diff: https://reviews.apache.org/r/73341/diff/1/ Testing --- With deferred action enabled: Test 1: - Associated a tag to an entity table1 which has lineage like table1 -> process1 -> table2 - Waited for tag propagation to complete - Once tag is propagation is complete, deleted the tag1 from table1 - Once the tag is removed from all the entities in the lineage above, deleted the tag - The tag is deleted properly Test 2: - Associated a tag to an entity table1 which has lineage like table1 -> process1 -> table2 - Waited for tag propagation to complete - Once tag is propagation is complete, deleted the tag1 from table1 - Associated the same tag to an entity table3 which has lineage like table3 -> process1 -> table4 - Once the tag is removed from all the entities in the lineage above, deleted the tag - The tag is deleted properly Thanks, Jayendra Parab
[jira] [Created] (ATLAS-4278) Deferred Actions : Deleting a tag after disassociating it from a table which propagated fails
Jayendra Parab created ATLAS-4278: - Summary: Deferred Actions : Deleting a tag after disassociating it from a table which propagated fails Key: ATLAS-4278 URL: https://issues.apache.org/jira/browse/ATLAS-4278 Project: Atlas Issue Type: Bug Affects Versions: 2.1.0 Reporter: Jayendra Parab Assignee: Jayendra Parab # Create a lineage : table1 ---> process1 > table2 # Add tag tag1 to table1. # Wait for task to complete, it propagates to table2 # Disassociate tag from table1 # Wait for task to complete , it is removed from all tables and process. # Attempt to delete the tag. # Tag deletion fails with `tag1` has references , though the tag is not associated to any entity. If the tag was added to entity without propagate flag set to False , and then disassociated and deleted , it is deleted successfully. This issue is seen only with deferred actions enabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4263) KafkaUtils sets invalid dynamic JAAS config
[
https://issues.apache.org/jira/browse/ATLAS-4263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338882#comment-17338882
]
Jayendra Parab commented on ATLAS-4263:
---
+1 for the patch
> KafkaUtils sets invalid dynamic JAAS config
> ---
>
> Key: ATLAS-4263
> URL: https://issues.apache.org/jira/browse/ATLAS-4263
> Project: Atlas
> Issue Type: Task
> Components: atlas-core
>Affects Versions: 2.1.0, 3.0.0
>Reporter: Vladislav Glinskiy
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> [KafkaUtils|https://github.com/apache/atlas/blob/8d3c4ab0e8844f04e29f66acb3577e9d40de9a16/common/src/main/java/org/apache/atlas/utils/KafkaUtils.java#L195]
> doesn't always
> [enclose|https://github.com/apache/atlas/blob/8d3c4ab0e8844f04e29f66acb3577e9d40de9a16/common/src/main/java/org/apache/atlas/utils/KafkaUtils.java#L316]
> property values in double-quotes, thus, setting invalid dynamic JAAS config
> for token auth in some cases.
> Faced this issue with Spark Atlas Connector trying to configure Atlas client
> to use delegation tokens. The following configuration is not handled properly:
> {code:java}
> atlas.jaas.KafkaClient.option.username=30CQ4q1hQMy0dB6X0eXfxQ
> atlas.jaas.KafkaClient.option.password=KdaUQ4FlKWlDxwQrAeFGUVbb6sR0P+zoqOZDZjtIRP1wseXbSbhiTjz3QI9Ur9o4LTYZSv8TE1QqUC4FSwnoTA==
> {code}
> and results in the following error:
> {code:java}
> java.lang.IllegalArgumentException: Value not specified for key 'null' in
> JAAS config
> at
> org.apache.kafka.common.security.JaasConfig.parseAppConfigurationEntry(JaasConfig.java:116)
> at
> org.apache.kafka.common.security.JaasConfig.(JaasConfig.java:63)
> at
> org.apache.kafka.common.security.JaasContext.load(JaasContext.java:90)
> at
> org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:84)
> {code}
> [KafkaUtils|https://github.com/apache/atlas/blob/8d3c4ab0e8844f04e29f66acb3577e9d40de9a16/common/src/main/java/org/apache/atlas/utils/KafkaUtils.java#L195]
> should always enclose property values in double-quotes, since unenclosed
> digits and '+' sign can not be parsed by Kafka
> [JaasConfig|https://github.com/apache/kafka/blob/2.0.0/clients/src/main/java/org/apache/kafka/common/security/JaasConfig.java#L116].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4271) When unauthorised user is tries to create a relationship, message thrown is incorrect
[
https://issues.apache.org/jira/browse/ATLAS-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayendra Parab updated ATLAS-4271:
--
Description:
When an unauthorized user (say user1) tries to create a relationship between
two entities (say type1 and type2), the error message thrown is
{code:java}
user1 is not authorized to perform create relationship-def{code}
The expected message should be
{code:java}
user1 is not authorized to perform read type-def of category ENTITY type1{code}
The current message should be thrown when an user with read-only access tries
to create a relationship.
was:
When an unauthorized user (say user1) tries to create a relationship between
two entities (say type1 and type2), the error message thrown is
user1 is not authorized to perform create relationship-def
The expected message should be
{code:java}
user1 is not authorized to perform read type-def of category ENTITY type1{code}
The current message should be thrown when an user with read-only access tries
to create a relationship.
> When unauthorised user is tries to create a relationship, message thrown is
> incorrect
> -
>
> Key: ATLAS-4271
> URL: https://issues.apache.org/jira/browse/ATLAS-4271
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 2.1.0
> Reporter: Jayendra Parab
> Assignee: Jayendra Parab
>Priority: Major
>
> When an unauthorized user (say user1) tries to create a relationship between
> two entities (say type1 and type2), the error message thrown is
> {code:java}
> user1 is not authorized to perform create relationship-def{code}
> The expected message should be
> {code:java}
> user1 is not authorized to perform read type-def of category ENTITY
> type1{code}
> The current message should be thrown when an user with read-only access tries
> to create a relationship.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-4271) When unauthorised user is tries to create a relationship, message thrown is incorrect
Jayendra Parab created ATLAS-4271:
-
Summary: When unauthorised user is tries to create a relationship,
message thrown is incorrect
Key: ATLAS-4271
URL: https://issues.apache.org/jira/browse/ATLAS-4271
Project: Atlas
Issue Type: Bug
Affects Versions: 2.1.0
Reporter: Jayendra Parab
Assignee: Jayendra Parab
When an unauthorized user (say user1) tries to create a relationship between
two entities (say type1 and type2), the error message thrown is
user1 is not authorized to perform create relationship-def
The expected message should be
{code:java}
user1 is not authorized to perform read type-def of category ENTITY type1{code}
The current message should be thrown when an user with read-only access tries
to create a relationship.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ATLAS-4206) Swagger: File upload APIs does not work
[ https://issues.apache.org/jira/browse/ATLAS-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab reassigned ATLAS-4206: - Assignee: Nikhil P Bonte > Swagger: File upload APIs does not work > --- > > Key: ATLAS-4206 > URL: https://issues.apache.org/jira/browse/ATLAS-4206 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Nikhil Bonte >Assignee: Nikhil P Bonte >Priority: Major > Labels: REST_API, swagger > > APIs affected: > POST /v2/glossary/import > POST /v2/entity/businessmetadata/import > > # Swagger UI for the above APIs shows two params _fileDetail_ & > _inputStream_ both of type _file_, the expectation is to have the only > _inputStream_ to upload a file. > # Param _inputStream_ have name as "_inputStream_", expected name is > "_file_". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ATLAS-4259) Swagger: Improve Header validation
[ https://issues.apache.org/jira/browse/ATLAS-4259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab reassigned ATLAS-4259: - Assignee: Nikhil P Bonte > Swagger: Improve Header validation > --- > > Key: ATLAS-4259 > URL: https://issues.apache.org/jira/browse/ATLAS-4259 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Nikhil Bonte >Assignee: Nikhil P Bonte >Priority: Major > Labels: REST_API, swagger > Attachments: ATLAS-4259-Swagger-Improve-Header-validation-v0.patch > > > This will add support for ATLAS-4064 in swagger. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ATLAS-4260) Swagger: click to expand API does not work properly in some cases
[ https://issues.apache.org/jira/browse/ATLAS-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab reassigned ATLAS-4260: - Assignee: Nikhil P Bonte > Swagger: click to expand API does not work properly in some cases > - > > Key: ATLAS-4260 > URL: https://issues.apache.org/jira/browse/ATLAS-4260 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Nikhil Bonte >Assignee: Nikhil P Bonte >Priority: Major > Labels: rest_api, swagger > > Click on API POST /v2/entity also expands API POST /v2/entity/bulk & vice > versa > > Root Cause: > This is due to the same id of div element of both APIs. > The swagger spec has "operationId" same for both API. > > Need to check if any other API has the same issue. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4250) Support HA for admin/task API
[ https://issues.apache.org/jira/browse/ATLAS-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17326467#comment-17326467 ] Jayendra Parab commented on ATLAS-4250: --- +1 for the patch > Support HA for admin/task API > - > > Key: ATLAS-4250 > URL: https://issues.apache.org/jira/browse/ATLAS-4250 > Project: Atlas > Issue Type: Improvement > Components: atlas-webui >Reporter: Nixon Rodrigues >Assignee: Nixon Rodrigues >Priority: Major > Attachments: ATLAS-4250.patch > > > Support HA for newly added *admin/task* API -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 73249: ATLAS-4221 : Solr collection names should be configurable
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73249/#review222759 --- distro/src/bin/atlas_config.py Lines 741 (patched) <https://reviews.apache.org/r/73249/#comment311937> Remove this print msg... distro/src/bin/atlas_start.py Lines 29 (patched) <https://reviews.apache.org/r/73249/#comment311936> We should have these constants in atlas_config.py and should be names as VERTEX_INDEX_NAME_CONF The method which gets the name, should make use of these constants from there distro/src/conf/atlas-application.properties Lines 83 (patched) <https://reviews.apache.org/r/73249/#comment311935> Kee the default index names as vertex_index, edge_index, fulltext_index - Jayendra Parab On March 30, 2021, 10:41 a.m., chaitali wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73249/ > --- > > (Updated March 30, 2021, 10:41 a.m.) > > > Review request for atlas, Jayendra Parab and Nixon Rodrigues. > > > Bugs: ATLAS-4221 > https://issues.apache.org/jira/browse/ATLAS-4221 > > > Repository: atlas > > > Description > --- > > Currently in Atlas, the collection names are not configurable > > We need to provide functionality where in names of the solr collections can > be customized using property configuration > > > Diffs > - > > common/src/main/java/org/apache/atlas/repository/Constants.java 771287f75 > distro/src/bin/atlas_config.py 80b3bfb78 > distro/src/bin/atlas_start.py 7cf35a92a > distro/src/conf/atlas-application.properties e06e74a26 > intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 08d6c9d4a > > > Diff: https://reviews.apache.org/r/73249/diff/2/ > > > Testing > --- > > mvn clean install > atlas server up and running > Collections created with entities linked > > > Thanks, > > chaitali > >
Re: Review Request 73246: ATLAS-4218: Docker support to run Apache Hive and Apache HBase with Atlas hook
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73246/#review222737 --- Ship it! Ship It! - Jayendra Parab On March 20, 2021, 7:24 a.m., Madhan Neethiraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73246/ > --- > > (Updated March 20, 2021, 7:24 a.m.) > > > Review request for atlas, Ashutosh Mestry, Deep Singh, Jayendra Parab, Nikhil > Bonte, Nixon Rodrigues, Pinal Shah, Radhika Kundam, Sarath Subramanian, and > Sidharth Mishra. > > > Bugs: ATLAS-4218 > https://issues.apache.org/jira/browse/ATLAS-4218 > > > Repository: atlas > > > Description > --- > > updated docker setup to run Apache Hive and Apache HBase with Atlas hook > > > Diffs > - > > dev-support/atlas-docker/.dockerignore fa4ce32f7 > dev-support/atlas-docker/.env 79d2b9780 > dev-support/atlas-docker/Dockerfile.atlas 4bafe80f5 > dev-support/atlas-docker/Dockerfile.atlas-db PRE-CREATION > dev-support/atlas-docker/Dockerfile.atlas-hbase 1c5fa282f > dev-support/atlas-docker/Dockerfile.atlas-hive PRE-CREATION > dev-support/atlas-docker/README.md 6ba40586a > dev-support/atlas-docker/config/init_postgres.sh PRE-CREATION > dev-support/atlas-docker/docker-compose.atlas-hbase.yml 715788936 > dev-support/atlas-docker/docker-compose.atlas-hive.yml PRE-CREATION > dev-support/atlas-docker/download-archives.sh dc6d22ca1 > dev-support/atlas-docker/scripts/atlas-build.sh 316ef410c > dev-support/atlas-docker/scripts/atlas-hbase-application.properties > PRE-CREATION > dev-support/atlas-docker/scripts/atlas-hbase-setup.sh c57698963 > dev-support/atlas-docker/scripts/atlas-hive-application.properties > PRE-CREATION > dev-support/atlas-docker/scripts/atlas-hive-setup.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas-hive.sh PRE-CREATION > dev-support/atlas-docker/scripts/hbase-site.xml dd8828fb4 > dev-support/atlas-docker/scripts/hive-site.xml PRE-CREATION > > > Diff: https://reviews.apache.org/r/73246/diff/1/ > > > Testing > --- > > - verified that Atlas hook in HBase publishes metadata events to Atlas via > Kafka notifications > - verified that Atlas hook in Hive published metadata events to Atlas via > Kafka notification > - pre-commit tests run: > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/461/ > > > Thanks, > > Madhan Neethiraj > >
Re: Review Request 73216: ATLAS-4188: Docker setup updated to run Atlas, HBase, Solr, Kafka, Zookeeper in seperate containers
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73216/#review222680 --- Ship it! Ship It! - Jayendra Parab On March 11, 2021, 4:29 p.m., Madhan Neethiraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73216/ > --- > > (Updated March 11, 2021, 4:29 p.m.) > > > Review request for atlas, Ashutosh Mestry, Deep Singh, Jayendra Parab, Nikhil > Bonte, Nixon Rodrigues, Pinal Shah, Radhika Kundam, Sarath Subramanian, > Sharmadha Sainath, and Sidharth Mishra. > > > Bugs: ATLAS-4188 > https://issues.apache.org/jira/browse/ATLAS-4188 > > > Repository: atlas > > > Description > --- > > updated Docker setup to run Atlas, HBase, Hadoop, Solr, Kafka, Zookeeper in > seperate containers > > > Diffs > - > > dev-support/atlas-docker/.dockerignore d6a848020 > dev-support/atlas-docker/.env 19443ad62 > dev-support/atlas-docker/Dockerfile.atlas 76e5c5967 > dev-support/atlas-docker/Dockerfile.atlas-base 36e8f0ff7 > dev-support/atlas-docker/Dockerfile.atlas-hadoop PRE-CREATION > dev-support/atlas-docker/Dockerfile.atlas-hbase PRE-CREATION > dev-support/atlas-docker/Dockerfile.atlas-kafka PRE-CREATION > dev-support/atlas-docker/Dockerfile.atlas-solr PRE-CREATION > dev-support/atlas-docker/Dockerfile.atlas-zk PRE-CREATION > dev-support/atlas-docker/README.md 37caa3a76 > dev-support/atlas-docker/config/solr/currency.xml PRE-CREATION > dev-support/atlas-docker/config/solr/lang/stopwords_en.txt PRE-CREATION > dev-support/atlas-docker/config/solr/protwords.txt PRE-CREATION > dev-support/atlas-docker/config/solr/schema.xml PRE-CREATION > dev-support/atlas-docker/config/solr/solrconfig.xml PRE-CREATION > dev-support/atlas-docker/config/solr/stopwords.txt PRE-CREATION > dev-support/atlas-docker/config/solr/synonyms.txt PRE-CREATION > dev-support/atlas-docker/docker-compose.atlas-build.yml c29588fe1 > dev-support/atlas-docker/docker-compose.atlas-hadoop.yml PRE-CREATION > dev-support/atlas-docker/docker-compose.atlas-hbase.yml PRE-CREATION > dev-support/atlas-docker/docker-compose.atlas-kafka.yml PRE-CREATION > dev-support/atlas-docker/docker-compose.atlas.yml cfaf0d7b4 > dev-support/atlas-docker/download-archives.sh PRE-CREATION > dev-support/atlas-docker/downloads/.gitignore PRE-CREATION > dev-support/atlas-docker/scripts/atlas-hadoop-mkdir.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas-hadoop-setup.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas-hadoop.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas-hbase-setup.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas-hbase.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas-kafka-setup.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas-kafka.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas-solr-create.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas.sh 7abb6f734 > dev-support/atlas-docker/scripts/hbase-site.xml PRE-CREATION > distro/pom.xml c00de3990 > pom.xml df4dec5d4 > webapp/pom.xml 7d6173544 > > > Diff: https://reviews.apache.org/r/73216/diff/4/ > > > Testing > --- > > - using docker-compose, verified that containers for Atlas and dependent > services are built and run successfully > - ran quick_start.py successfully on Atlas instance running in a container > - pre-commit tests run: > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/431 > > > Thanks, > > Madhan Neethiraj > >
Re: Review Request 73216: ATLAS-4188: Docker setup updated to run Atlas, HBase, Solr, Kafka, Zookeeper in seperate containers
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73216/#review222666 --- dev-support/atlas-docker/.env Lines 10 (patched) <https://reviews.apache.org/r/73216/#comment311848> Can we also have solr and zookeeper version which can be used to pull the appropriate docker image? dev-support/atlas-docker/README.md Line 26 (original), 26 (patched) <https://reviews.apache.org/r/73216/#comment311847> Shall we also have the minimum memory allocation for docker as part of the README? - Jayendra Parab On March 7, 2021, 9:42 a.m., Madhan Neethiraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73216/ > --- > > (Updated March 7, 2021, 9:42 a.m.) > > > Review request for atlas, Ashutosh Mestry, Deep Singh, Jayendra Parab, Nikhil > Bonte, Nixon Rodrigues, Pinal Shah, Radhika Kundam, Sarath Subramanian, > Sharmadha Sainath, and Sidharth Mishra. > > > Bugs: ATLAS-4188 > https://issues.apache.org/jira/browse/ATLAS-4188 > > > Repository: atlas > > > Description > --- > > updated Docker setup to run Atlas, HBase, Hadoop, Solr, Kafka, Zookeeper in > seperate containers > > > Diffs > - > > dev-support/atlas-docker/.dockerignore d6a848020 > dev-support/atlas-docker/.env 19443ad62 > dev-support/atlas-docker/Dockerfile.atlas 76e5c5967 > dev-support/atlas-docker/Dockerfile.atlas-base 36e8f0ff7 > dev-support/atlas-docker/Dockerfile.atlas-hadoop PRE-CREATION > dev-support/atlas-docker/Dockerfile.atlas-hbase PRE-CREATION > dev-support/atlas-docker/Dockerfile.atlas-kafka PRE-CREATION > dev-support/atlas-docker/Dockerfile.atlas-solr PRE-CREATION > dev-support/atlas-docker/Dockerfile.atlas-zk PRE-CREATION > dev-support/atlas-docker/README.md 37caa3a76 > dev-support/atlas-docker/config/solr/currency.xml PRE-CREATION > dev-support/atlas-docker/config/solr/lang/stopwords_en.txt PRE-CREATION > dev-support/atlas-docker/config/solr/protwords.txt PRE-CREATION > dev-support/atlas-docker/config/solr/schema.xml PRE-CREATION > dev-support/atlas-docker/config/solr/solrconfig.xml PRE-CREATION > dev-support/atlas-docker/config/solr/stopwords.txt PRE-CREATION > dev-support/atlas-docker/config/solr/synonyms.txt PRE-CREATION > dev-support/atlas-docker/docker-compose.atlas-build.yml c29588fe1 > dev-support/atlas-docker/docker-compose.atlas-hadoop.yml PRE-CREATION > dev-support/atlas-docker/docker-compose.atlas-hbase.yml PRE-CREATION > dev-support/atlas-docker/docker-compose.atlas-kafka.yml PRE-CREATION > dev-support/atlas-docker/docker-compose.atlas.yml cfaf0d7b4 > dev-support/atlas-docker/download-archives.sh PRE-CREATION > dev-support/atlas-docker/downloads/.gitignore PRE-CREATION > dev-support/atlas-docker/scripts/atlas-hadoop-mkdir.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas-hadoop-setup.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas-hadoop.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas-hbase-setup.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas-hbase.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas-kafka-setup.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas-kafka.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas-solr-create.sh PRE-CREATION > dev-support/atlas-docker/scripts/atlas.sh 7abb6f734 > dev-support/atlas-docker/scripts/hbase-site.xml PRE-CREATION > distro/pom.xml c00de3990 > pom.xml df4dec5d4 > webapp/pom.xml 7d6173544 > > > Diff: https://reviews.apache.org/r/73216/diff/3/ > > > Testing > --- > > - using docker-compose, verified that containers for Atlas and dependent > services are built and run successfully > - ran quick_start.py successfully on Atlas instance running in a container > - pre-commit tests run: > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/431 > > > Thanks, > > Madhan Neethiraj > >
Re: Review Request 73076: Deferred Actions Implementation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73076/#review222507 --- repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationTask.java Lines 39 (patched) <https://reviews.apache.org/r/73076/#comment311615> Can we make ARGUMENT_INDEX_CLASSIFICATION_NAMES and ARGUMENT_INDEX_VERTEX_IDS as final as well? repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationTask.java Lines 58 (patched) <https://reviews.apache.org/r/73076/#comment311614> Can we store the error message in graphdb whenever we store the status as COMPLETE_WITH_ERRORS? - Jayendra Parab On Jan. 20, 2021, 11:14 p.m., Ashutosh Mestry wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73076/ > --- > > (Updated Jan. 20, 2021, 11:14 p.m.) > > > Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, > Nixon Rodrigues, and Sarath Subramanian. > > > Bugs: ATLAS-3919 > https://issues.apache.org/jira/browse/ATLAS-3919 > > > Repository: atlas > > > Description > --- > > **Approach** > Model: _TaskDef_ > - Definition of task. > - Contains status of the task for querying. > > _AbstractTask_ > - Takes TaskDef as input and calls run. > - The concrete tasks has logic for doing real work. > > _TaskFactory_ > > - Creates tasks given task type. > - Enumerates all the tasks that are supported. > > _TaskManagement_ > > - Component/Service. Constructed after all existing components are > constructed. > - Provides a simple interface for adding tasks. > - Processes tasks that are in pending state. > - Supports HA mode, where the tasks are not executed on passive instance. > > _TaskManagement.Registry_ > > - CRUD operations with TaskDef vertex. > > _TaskManagement.Processor_ > - Provides mechanism for provides single queue for processing tasks in a way > that is non-blocking. > - Uses task type and the associated TaskFactory to construct concrete task. > > Changes to _EntityGraphMapper_ > There are 3 new methods added that use portion of existing classification > propagation logic and create a separate method. > > New class _EntityGraphMapperWithTasks_ acts as a proxy for EntityGraphMapper. > All calls that operate without tasks get channeled to the existing > implementation. The ones involving tasks are modified to use the new task > framework. > > **Concurrent Changes to Classifications** > Via REST APIs and operations from web UI, it is possible to make changes to > the same entity. This situation will be more pronounced in the new setup. > > The existing framework has > _GraphTransactionInterceptor.lockObjectAndReleasePostCommit_. This provides > ReentrantLock on an entity that is identified by the supplied GUID. This will > cause a lock on the entity and all the impacted vertices. This has potential > for adversely impacting scalability. > > > Diffs > - > > intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 396aad06f > intg/src/main/java/org/apache/atlas/model/tasks/TaskDef.java PRE-CREATION > intg/src/main/java/org/apache/atlas/type/AtlasType.java b30b483e0 > intg/src/main/java/org/apache/atlas/utils/AtlasJson.java abeddf640 > intg/src/test/resources/atlas-application.properties 7e74d5107 > repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java > 86b369fc9 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java > 244072289 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java > 2cfcc0bc1 > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapperWithTasks.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationPropagateTaskFactory.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationPropagationTasks.java > PRE-CREATION > > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/tasks/ClassificationTask.java > PRE-CREATION > repository/src/main/java/org/apache/atlas/tasks/AbstractTask.java > PRE-CREATION > repository/src/main/java/org/apache/atlas/tasks/TaskExecutor.java > PRE-CREATION > repository/src/main/java/org/apache/atlas/task
[jira] [Updated] (ATLAS-4055) Basic search doesn't work if the search text has special characters
[ https://issues.apache.org/jira/browse/ATLAS-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab updated ATLAS-4055: -- Fix Version/s: 2.2.0 3.0.0 > Basic search doesn't work if the search text has special characters > --- > > Key: ATLAS-4055 > URL: https://issues.apache.org/jira/browse/ATLAS-4055 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 > Reporter: Jayendra Parab > Assignee: Jayendra Parab >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > If the user tries to perform a search with text which has special characters > mentioned in [Lucene > documentation|https://lucene.apache.org/core/6_5_1/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Escaping_Special_Characters], > then the results don't include entities matching the search text, even > though the document exists in Solr. > The special characters are escaped as part of fix provided in ATLAS-3983, but > it also needs to be enclosed the search text in double quotes -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4055) Basic search doesn't work if the search text has special characters
[ https://issues.apache.org/jira/browse/ATLAS-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab resolved ATLAS-4055. --- Resolution: Fixed > Basic search doesn't work if the search text has special characters > --- > > Key: ATLAS-4055 > URL: https://issues.apache.org/jira/browse/ATLAS-4055 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 > Reporter: Jayendra Parab > Assignee: Jayendra Parab >Priority: Major > > If the user tries to perform a search with text which has special characters > mentioned in [Lucene > documentation|https://lucene.apache.org/core/6_5_1/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Escaping_Special_Characters], > then the results don't include entities matching the search text, even > though the document exists in Solr. > The special characters are escaped as part of fix provided in ATLAS-3983, but > it also needs to be enclosed the search text in double quotes -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 73046: ATLAS-4055 Basic search doesn't work if the search text has special characters
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73046/
---
(Updated Nov. 26, 2020, 9:46 a.m.)
Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath
Subramanian.
Changes
---
Fixed a testcase failure, removed * from the list of special characters for
which we need to enclose in double quotes. The * char is used in the search
string for wildcard search, so removing it
Bugs: ATLAS-4055
https://issues.apache.org/jira/browse/ATLAS-4055
Repository: atlas
Description
---
If the search text has a special character (i.e, one of + - & | ! ( ) { } [ ] ^
" ~ * ? : \ /), then we need to enclose the search text in double quotes. This
change is required in addition to the changes provided in ATLAS-3983
Diffs (updated)
-
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 28d2e23c4
Diff: https://reviews.apache.org/r/73046/diff/2/
Changes: https://reviews.apache.org/r/73046/diff/1-2/
Testing
---
- Created entities with all the special characters in the name and qualified
name attribute
- Tried basic search with attributes to search the name and qualified name
- The search result included the appropriate entity that was matching the
search criteria
Thanks,
Jayendra Parab
Re: Review Request 73046: ATLAS-4055 Basic search doesn't work if the search text has special characters
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73046/
---
(Updated Nov. 26, 2020, 6 a.m.)
Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath
Subramanian.
Summary (updated)
-
ATLAS-4055 Basic search doesn't work if the search text has special characters
Bugs: ATLAS-4055
https://issues.apache.org/jira/browse/ATLAS-4055
Repository: atlas
Description
---
If the search text has a special character (i.e, one of + - & | ! ( ) { } [ ] ^
" ~ * ? : \ /), then we need to enclose the search text in double quotes. This
change is required in addition to the changes provided in ATLAS-3983
Diffs
-
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 28d2e23c4
Diff: https://reviews.apache.org/r/73046/diff/1/
Testing
---
- Created entities with all the special characters in the name and qualified
name attribute
- Tried basic search with attributes to search the name and qualified name
- The search result included the appropriate entity that was matching the
search criteria
Thanks,
Jayendra Parab
[jira] [Updated] (ATLAS-4055) Basic search doesn't work if the search text has special characters
[ https://issues.apache.org/jira/browse/ATLAS-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab updated ATLAS-4055: -- Summary: Basic search doesn't work if the search text has special characters (was: Basic search doesn't work if the basic search with attributes text has special characters) > Basic search doesn't work if the search text has special characters > --- > > Key: ATLAS-4055 > URL: https://issues.apache.org/jira/browse/ATLAS-4055 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 > Reporter: Jayendra Parab > Assignee: Jayendra Parab >Priority: Major > > If the user tries to perform a search with text which has special characters > mentioned in [Lucene > documentation|https://lucene.apache.org/core/6_5_1/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Escaping_Special_Characters], > then the results don't include entities matching the search text, even > though the document exists in Solr. > The special characters are escaped as part of fix provided in ATLAS-3983, but > it also needs to be enclosed the search text in double quotes -- This message was sent by Atlassian Jira (v8.3.4#803005)
Review Request 73046: ATLAS-4055 Basic search doesn't work if the basic search with attributes text has special characters
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73046/
---
Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath
Subramanian.
Bugs: ATLAS-4055
https://issues.apache.org/jira/browse/ATLAS-4055
Repository: atlas
Description
---
If the search text has a special character (i.e, one of + - & | ! ( ) { } [ ] ^
" ~ * ? : \ /), then we need to enclose the search text in double quotes. This
change is required in addition to the changes provided in ATLAS-3983
Diffs
-
intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 28d2e23c4
Diff: https://reviews.apache.org/r/73046/diff/1/
Testing
---
- Created entities with all the special characters in the name and qualified
name attribute
- Tried basic search with attributes to search the name and qualified name
- The search result included the appropriate entity that was matching the
search criteria
Thanks,
Jayendra Parab
[jira] [Created] (ATLAS-4055) Basic search doesn't work if the basic search with attributes text has special characters
Jayendra Parab created ATLAS-4055: - Summary: Basic search doesn't work if the basic search with attributes text has special characters Key: ATLAS-4055 URL: https://issues.apache.org/jira/browse/ATLAS-4055 Project: Atlas Issue Type: Bug Affects Versions: 2.1.0 Reporter: Jayendra Parab Assignee: Jayendra Parab If the user tries to perform a search with text which has special characters mentioned in [Lucene documentation|https://lucene.apache.org/core/6_5_1/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Escaping_Special_Characters], then the results don't include entities matching the search text, even though the document exists in Solr. The special characters are escaped as part of fix provided in ATLAS-3983, but it also needs to be enclosed the search text in double quotes -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Review Request 73000: ATLAS-4011: While running Atlas Docker container, data should be stored on host
> On Nov. 2, 2020, 7:09 p.m., Madhan Neethiraj wrote: > > dev-support/atlas-docker/README.md > > Lines 42 (patched) > > <https://reviews.apache.org/r/73000/diff/1/?file=2241770#file2241770line42> > > > > Instead of requiring users to create directory 'data', I suggest to > > create this directory and include with this patch. Please make sure to > > commit .gitignore file in this directory. > > > > Once this is done, please remove 4.2 and 5.5. The explicit `mkdir data` step isn't required at all since docker creates the directory during the `docker run` command. So, we don't need to commit the data directory. The user can also choose any other directory for storing the data. Updated the README file with these changes. - Jayendra --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73000/#review222158 ------- On Nov. 5, 2020, 5:48 a.m., Jayendra Parab wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/73000/ > --- > > (Updated Nov. 5, 2020, 5:48 a.m.) > > > Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, > and Sarath Subramanian. > > > Bugs: ATLAS-4011 > https://issues.apache.org/jira/browse/ATLAS-4011 > > > Repository: atlas > > > Description > --- > > - Mounting data directory from host machine into $ATLAS_HOME/data > - To ensure solr collections are created in the data directory, setting the > SOLR_HOME using -s option. Also, copying solr.xml from the solr installation > directory to the solr home > - Changed the command to keep the container from exiting when the container > runs in a pod in Kubernetes > - Updated the documentation with new command for running docker container > > > Diffs > - > > dev-support/atlas-docker/Dockerfile.atlas e2dea7716 > dev-support/atlas-docker/README.md de644472b > dev-support/atlas-docker/docker-compose.atlas.yml 92bd1edd0 > dev-support/atlas-docker/scripts/atlas.sh 0c4a4546a > distro/src/bin/atlas_config.py bda8e2a10 > distro/src/bin/atlas_start.py 54fd2bd47 > distro/src/bin/atlas_stop.py 400a7d23d > distro/src/test/python/scripts/TestMetadata.py f1235f747 > > > Diff: https://reviews.apache.org/r/73000/diff/2/ > > > Testing > --- > > 1. Built the atlas container using `docker build -f Dockerfile.atlas > --build-arg ATLAS_VERSION=3.0.0-SNAPSHOT -t atlas .` > 2. Ran the atlas container using `docker run -it -d --name atlas --hostname > localhost -p 21000:21000 -v $PWD/data:/home/atlas/data atlas` > 3. Atlas was up and running, created some entities > 4. Restarted the container. The entites created in step 3, were still there > 5. Deleted the container using `docker rm -f atlas` > 6. Created the conatiner again using the same command in step 2. The entities > created in step 3 were present. > > > Thanks, > > Jayendra Parab > >
Re: Review Request 73000: ATLAS-4011: While running Atlas Docker container, data should be stored on host
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73000/ --- (Updated Nov. 5, 2020, 5:48 a.m.) Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Changes --- Removed the `mkdir data` instruction from the README file. During testing I found that if the data directory isn't present on host machine, its created during the `docker run` command without user requiring to create it. So that instruction in README file isn't required. Bugs: ATLAS-4011 https://issues.apache.org/jira/browse/ATLAS-4011 Repository: atlas Description --- - Mounting data directory from host machine into $ATLAS_HOME/data - To ensure solr collections are created in the data directory, setting the SOLR_HOME using -s option. Also, copying solr.xml from the solr installation directory to the solr home - Changed the command to keep the container from exiting when the container runs in a pod in Kubernetes - Updated the documentation with new command for running docker container Diffs (updated) - dev-support/atlas-docker/Dockerfile.atlas e2dea7716 dev-support/atlas-docker/README.md de644472b dev-support/atlas-docker/docker-compose.atlas.yml 92bd1edd0 dev-support/atlas-docker/scripts/atlas.sh 0c4a4546a distro/src/bin/atlas_config.py bda8e2a10 distro/src/bin/atlas_start.py 54fd2bd47 distro/src/bin/atlas_stop.py 400a7d23d distro/src/test/python/scripts/TestMetadata.py f1235f747 Diff: https://reviews.apache.org/r/73000/diff/2/ Changes: https://reviews.apache.org/r/73000/diff/1-2/ Testing --- 1. Built the atlas container using `docker build -f Dockerfile.atlas --build-arg ATLAS_VERSION=3.0.0-SNAPSHOT -t atlas .` 2. Ran the atlas container using `docker run -it -d --name atlas --hostname localhost -p 21000:21000 -v $PWD/data:/home/atlas/data atlas` 3. Atlas was up and running, created some entities 4. Restarted the container. The entites created in step 3, were still there 5. Deleted the container using `docker rm -f atlas` 6. Created the conatiner again using the same command in step 2. The entities created in step 3 were present. Thanks, Jayendra Parab
Review Request 73000: ATLAS-4011: While running Atlas Docker container, data should be stored on host
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73000/ --- Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Bugs: ATLAS-4011 https://issues.apache.org/jira/browse/ATLAS-4011 Repository: atlas Description --- - Mounting data directory from host machine into $ATLAS_HOME/data - To ensure solr collections are created in the data directory, setting the SOLR_HOME using -s option. Also, copying solr.xml from the solr installation directory to the solr home - Changed the command to keep the container from exiting when the container runs in a pod in Kubernetes - Updated the documentation with new command for running docker container Diffs - dev-support/atlas-docker/Dockerfile.atlas e2dea7716 dev-support/atlas-docker/README.md de644472b dev-support/atlas-docker/docker-compose.atlas.yml 92bd1edd0 dev-support/atlas-docker/scripts/atlas.sh 0c4a4546a distro/src/bin/atlas_config.py bda8e2a10 distro/src/bin/atlas_start.py 54fd2bd47 distro/src/bin/atlas_stop.py 400a7d23d distro/src/test/python/scripts/TestMetadata.py f1235f747 Diff: https://reviews.apache.org/r/73000/diff/1/ Testing --- 1. Built the atlas container using `docker build -f Dockerfile.atlas --build-arg ATLAS_VERSION=3.0.0-SNAPSHOT -t atlas .` 2. Ran the atlas container using `docker run -it -d --name atlas --hostname localhost -p 21000:21000 -v $PWD/data:/home/atlas/data atlas` 3. Atlas was up and running, created some entities 4. Restarted the container. The entites created in step 3, were still there 5. Deleted the container using `docker rm -f atlas` 6. Created the conatiner again using the same command in step 2. The entities created in step 3 were present. Thanks, Jayendra Parab
[jira] [Assigned] (ATLAS-4011) Docker: While running Atlas Docker container, data should be stored on host
[ https://issues.apache.org/jira/browse/ATLAS-4011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab reassigned ATLAS-4011: - Assignee: Jayendra Parab > Docker: While running Atlas Docker container, data should be stored on host > --- > > Key: ATLAS-4011 > URL: https://issues.apache.org/jira/browse/ATLAS-4011 > Project: Atlas > Issue Type: Improvement > Reporter: Jayendra Parab > Assignee: Jayendra Parab >Priority: Major > > Currently, if Atlas is started with Embedded Solr and HBase in Docker > container, the data is persisted inside the container. So if the container is > destroyed and a new container is started, then its old data is lost. > The data directory should be mounted as a volume, so that data can be > restored across container destroy and restore. > To achieve this, Solr data directory should be configurable and by default it > should be $ATLAS_HOME/data/solr. HBase and Kafka data directory is already > configured to $ATLAS_HOME/data -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-4011) Docker: While running Atlas Docker container, data should be stored on host
Jayendra Parab created ATLAS-4011: - Summary: Docker: While running Atlas Docker container, data should be stored on host Key: ATLAS-4011 URL: https://issues.apache.org/jira/browse/ATLAS-4011 Project: Atlas Issue Type: Improvement Reporter: Jayendra Parab Currently, if Atlas is started with Embedded Solr and HBase in Docker container, the data is persisted inside the container. So if the container is destroyed and a new container is started, then its old data is lost. The data directory should be mounted as a volume, so that data can be restored across container destroy and restore. To achieve this, Solr data directory should be configurable and by default it should be $ATLAS_HOME/data/solr. HBase and Kafka data directory is already configured to $ATLAS_HOME/data -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3864) Atlas should use AdminClient where possible
[ https://issues.apache.org/jira/browse/ATLAS-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab updated ATLAS-3864: -- Affects Version/s: 2.1.0 > Atlas should use AdminClient where possible > --- > > Key: ATLAS-3864 > URL: https://issues.apache.org/jira/browse/ATLAS-3864 > Project: Atlas > Issue Type: Improvement >Affects Versions: 2.1.0 >Reporter: Viktor Somogyi-Vass > Assignee: Jayendra Parab >Priority: Major > > Zookeeper access should be restricted wherever possible as going forward > Kafka won't use Zookeeper (see > [KIP-500|https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum]). > Also the Zookeeper classes in Kafka aren't public apis so usage should be > avoided wherever possible. As a tactical fix (see ATLAS-3910) though it is > possible to continue using KafkaZkClient as that class likely won't change > much until the completion of KIP-500. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3864) Atlas should use AdminClient where possible
[ https://issues.apache.org/jira/browse/ATLAS-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab updated ATLAS-3864: -- Fix Version/s: 2.2.0 3.0.0 > Atlas should use AdminClient where possible > --- > > Key: ATLAS-3864 > URL: https://issues.apache.org/jira/browse/ATLAS-3864 > Project: Atlas > Issue Type: Improvement >Affects Versions: 2.1.0 >Reporter: Viktor Somogyi-Vass > Assignee: Jayendra Parab >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > Zookeeper access should be restricted wherever possible as going forward > Kafka won't use Zookeeper (see > [KIP-500|https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum]). > Also the Zookeeper classes in Kafka aren't public apis so usage should be > avoided wherever possible. As a tactical fix (see ATLAS-3910) though it is > possible to continue using KafkaZkClient as that class likely won't change > much until the completion of KIP-500. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-3864) Atlas should use AdminClient where possible
[ https://issues.apache.org/jira/browse/ATLAS-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab resolved ATLAS-3864. --- Assignee: Jayendra Parab Resolution: Fixed Fixed as part of https://reviews.apache.org/r/72671/ > Atlas should use AdminClient where possible > --- > > Key: ATLAS-3864 > URL: https://issues.apache.org/jira/browse/ATLAS-3864 > Project: Atlas > Issue Type: Improvement >Reporter: Viktor Somogyi-Vass > Assignee: Jayendra Parab >Priority: Major > > Zookeeper access should be restricted wherever possible as going forward > Kafka won't use Zookeeper (see > [KIP-500|https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum]). > Also the Zookeeper classes in Kafka aren't public apis so usage should be > avoided wherever possible. As a tactical fix (see ATLAS-3910) though it is > possible to continue using KafkaZkClient as that class likely won't change > much until the completion of KIP-500. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4005) DSL search gives error if select clause contains attributes with null values
[
https://issues.apache.org/jira/browse/ATLAS-4005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayendra Parab resolved ATLAS-4005.
---
Fix Version/s: 2.2.0
3.0.0
Assignee: Jayendra Parab
Resolution: Fixed
Submitted the fix in this review request
https://reviews.apache.org/r/72987/
> DSL search gives error if select clause contains attributes with null values
>
>
> Key: ATLAS-4005
> URL: https://issues.apache.org/jira/browse/ATLAS-4005
> Project: Atlas
> Issue Type: Bug
> Reporter: Jayendra Parab
> Assignee: Jayendra Parab
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
>
> When we do DSL search in the atlas, if one or more attributes in select
> clause are not set for a vertex then the below given exception is thrown:
>
> {code:java}
> java.lang.IllegalStateException: The property does not exist as the key has
> no associated value for the provided element
> {code}
> Example, in the query below, if owner isn't set for any of the entities, but
> name and description are present then the above exception is thrown
> {code:java}
> from hdfs_path where name="testPath" select name, owner, description{code}
> Expected Behaviour: Query should return name and description of the filtered
> entities with owner as blank
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4005) DSL search gives error if select clause contains attributes with null values
[
https://issues.apache.org/jira/browse/ATLAS-4005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayendra Parab updated ATLAS-4005:
--
Affects Version/s: 2.1.0
> DSL search gives error if select clause contains attributes with null values
>
>
> Key: ATLAS-4005
> URL: https://issues.apache.org/jira/browse/ATLAS-4005
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 2.1.0
> Reporter: Jayendra Parab
> Assignee: Jayendra Parab
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
>
> When we do DSL search in the atlas, if one or more attributes in select
> clause are not set for a vertex then the below given exception is thrown:
>
> {code:java}
> java.lang.IllegalStateException: The property does not exist as the key has
> no associated value for the provided element
> {code}
> Example, in the query below, if owner isn't set for any of the entities, but
> name and description are present then the above exception is thrown
> {code:java}
> from hdfs_path where name="testPath" select name, owner, description{code}
> Expected Behaviour: Query should return name and description of the filtered
> entities with owner as blank
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: Review Request 72987: ATLAS-4005 DSL search gives error if select clause contains attributes with null values
> On Oct. 23, 2020, 5:44 p.m., Ashutosh Mestry wrote: > > Can you please add per-commit build details. The build which passed https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/132/console The build which failed https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/131/console - Jayendra --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72987/#review222121 --- On Oct. 23, 2020, 5:43 p.m., Jayendra Parab wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72987/ > --- > > (Updated Oct. 23, 2020, 5:43 p.m.) > > > Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, > and Sarath Subramanian. > > > Bugs: ATLAS-4005 > https://issues.apache.org/jira/browse/ATLAS-4005 > > > Repository: atlas > > > Description > --- > > - The exception mentioned in the issue comes because the generated gremlin > query uses it.value() to get the values for an attribute of a vertex, If the > value isn't present, then it throws the mentioned exception > - In the fix provided, a check is added in the generated gremlin script, > where property.isPresent(), is checked before getting any value > - In ATLAS-2354, a similar issue was fixed, but because of that change there > are no results fetched for the scenario mentioned in ATLAS-4005. > - As part of fix for ATLAS-2354, whatever columns are present in select > clause, those columns were implicitly added to the where part of query, > because this no entities are selected > - So this review request reverts that change so that selected values are > shown for entities matching the where clause > - Also, after reverting the change one DSL Query test case was failing > because qualifiedName was null for columns which don't exist. So added a > check in GremilinQueryComposer.process() method to fix that testcase failure > > > Diffs > - > > repository/src/main/java/org/apache/atlas/query/GremlinClause.java > 55ccabd23 > repository/src/main/java/org/apache/atlas/query/GremlinQueryComposer.java > 801e89806 > repository/src/main/java/org/apache/atlas/query/SelectClauseComposer.java > fc7414802 > repository/src/test/java/org/apache/atlas/query/DSLQueriesTest.java > 5ace37944 > > repository/src/test/java/org/apache/atlas/query/GremlinQueryComposerTest.java > 959aa1149 > > > Diff: https://reviews.apache.org/r/72987/diff/2/ > > > Testing > --- > > - All the testcases in DSLQueriesTest are passing > - Tested with different queries from UI, like > "from hdfs_path where name="testPath" select name" > "from hdfs_path where name="testPath" select name, owner" > "from hdfs_path where name="testPath" select name, owner, description" > all give expected results with any exceptions > - Tried the above queries different scenarios > with all entites having the selected attributes set > with some entities having the selected attribute set > with all entities having owner value as null > all scenarios worked as expected > > > Thanks, > > Jayendra Parab > >
Re: Review Request 72987: ATLAS-4005 DSL search gives error if select clause contains attributes with null values
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72987/ --- (Updated Oct. 23, 2020, 5:43 p.m.) Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Changes --- - Addressed review comments - Fixed testcases from GremlineQueryComposerTest which failed in precommit. Added isPresent check in the expected generated query value. Bugs: ATLAS-4005 https://issues.apache.org/jira/browse/ATLAS-4005 Repository: atlas Description --- - The exception mentioned in the issue comes because the generated gremlin query uses it.value() to get the values for an attribute of a vertex, If the value isn't present, then it throws the mentioned exception - In the fix provided, a check is added in the generated gremlin script, where property.isPresent(), is checked before getting any value - In ATLAS-2354, a similar issue was fixed, but because of that change there are no results fetched for the scenario mentioned in ATLAS-4005. - As part of fix for ATLAS-2354, whatever columns are present in select clause, those columns were implicitly added to the where part of query, because this no entities are selected - So this review request reverts that change so that selected values are shown for entities matching the where clause - Also, after reverting the change one DSL Query test case was failing because qualifiedName was null for columns which don't exist. So added a check in GremilinQueryComposer.process() method to fix that testcase failure Diffs (updated) - repository/src/main/java/org/apache/atlas/query/GremlinClause.java 55ccabd23 repository/src/main/java/org/apache/atlas/query/GremlinQueryComposer.java 801e89806 repository/src/main/java/org/apache/atlas/query/SelectClauseComposer.java fc7414802 repository/src/test/java/org/apache/atlas/query/DSLQueriesTest.java 5ace37944 repository/src/test/java/org/apache/atlas/query/GremlinQueryComposerTest.java 959aa1149 Diff: https://reviews.apache.org/r/72987/diff/2/ Changes: https://reviews.apache.org/r/72987/diff/1-2/ Testing --- - All the testcases in DSLQueriesTest are passing - Tested with different queries from UI, like "from hdfs_path where name="testPath" select name" "from hdfs_path where name="testPath" select name, owner" "from hdfs_path where name="testPath" select name, owner, description" all give expected results with any exceptions - Tried the above queries different scenarios with all entites having the selected attributes set with some entities having the selected attribute set with all entities having owner value as null all scenarios worked as expected Thanks, Jayendra Parab
Review Request 72987: ATLAS-4005 DSL search gives error if select clause contains attributes with null values
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72987/ --- Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Bugs: ATLAS-4005 https://issues.apache.org/jira/browse/ATLAS-4005 Repository: atlas Description --- - The exception mentioned in the issue comes because the generated gremlin query uses it.value() to get the values for an attribute of a vertex, If the value isn't present, then it throws the mentioned exception - In the fix provided, a check is added in the generated gremlin script, where property.isPresent(), is checked before getting any value - In ATLAS-2354, a similar issue was fixed, but because of that change there are no results fetched for the scenario mentioned in ATLAS-4005. - As part of fix for ATLAS-2354, whatever columns are present in select clause, those columns were implicitly added to the where part of query, because this no entities are selected - So this review request reverts that change so that selected values are shown for entities matching the where clause - Also, after reverting the change one DSL Query test case was failing because qualifiedName was null for columns which don't exist. So added a check in GremilinQueryComposer.process() method to fix that testcase failure Diffs - repository/src/main/java/org/apache/atlas/query/GremlinClause.java 55ccabd23 repository/src/main/java/org/apache/atlas/query/GremlinQueryComposer.java 801e89806 repository/src/main/java/org/apache/atlas/query/SelectClauseComposer.java fc7414802 repository/src/test/java/org/apache/atlas/query/DSLQueriesTest.java 5ace37944 Diff: https://reviews.apache.org/r/72987/diff/1/ Testing --- - All the testcases in DSLQueriesTest are passing - Tested with different queries from UI, like "from hdfs_path where name="testPath" select name" "from hdfs_path where name="testPath" select name, owner" "from hdfs_path where name="testPath" select name, owner, description" all give expected results with any exceptions - Tried the above queries different scenarios with all entites having the selected attributes set with some entities having the selected attribute set with all entities having owner value as null all scenarios worked as expected Thanks, Jayendra Parab
[jira] [Created] (ATLAS-4005) DSL search gives error if select clause contains attributes with null values
Jayendra Parab created ATLAS-4005:
-
Summary: DSL search gives error if select clause contains
attributes with null values
Key: ATLAS-4005
URL: https://issues.apache.org/jira/browse/ATLAS-4005
Project: Atlas
Issue Type: Bug
Reporter: Jayendra Parab
When we do DSL search in the atlas, if one or more attributes in select clause
are not set for a vertex then the below given exception is thrown:
{code:java}
java.lang.IllegalStateException: The property does not exist as the key has no
associated value for the provided element
{code}
Example, in the query below, if owner isn't set for any of the entities, but
name and description are present then the above exception is thrown
{code:java}
from hdfs_path where name="testPath" select name, owner, description{code}
Expected Behaviour: Query should return name and description of the filtered
entities with owner as blank
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-3919) Handling classification propagation as deferred-action
Jayendra Parab created ATLAS-3919: - Summary: Handling classification propagation as deferred-action Key: ATLAS-3919 URL: https://issues.apache.org/jira/browse/ATLAS-3919 Project: Atlas Issue Type: Improvement Components: atlas-core Reporter: Jayendra Parab Assignee: Jayendra Parab Currently, whenever a user assigns a tag or updates a tag on an entity, it gets propagated to all the entities derived from the tagged entity. This operation takes quite a lot of time to complete (sometimes into minutes) and causes usability issues on the UI and other clients invoking the REST API To resolve this issue, tag-propagation needs to be handled as deferred-action, so that time consuming like propagation, audits & notifications can be processed in background threads -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [VOTE] Release Apache Atlas version 2.1.0 - rc3
+1 for Apache Atlas 2.1.0 RC3 release Validated the following: 1. Verified the source SHA512 and MD5 checksum 2. Verified the build with -Pdist,embedded-hbase-solr 3. Started Atlas server successfully 4. Verified quick_start.py, all entities and tag get created successfully 5. Created entities, applied tags from UI 6. Checked Basic and advanced search Regards, Jayendra. On Mon, Jul 13, 2020 at 12:17 PM Sarath Subramanian wrote: > +1 for Apache Atlas 2.1 rc3 > > Validated the following: > - Validated MD5 checksum > - Validated sha512 hash > - Verified PGP signature > - No binaries in extracted source > - Built successfully using embedded-hbase and solr profile > - Ran quick start and tested basic sanity - saved search, basic/advanced > search, Glossary, tag propagation, Admin purge, system attribute search > > > > On Thu, Jul 9, 2020 at 9:59 PM Madhan Neethiraj wrote: > > > Atlas team, > > > > Apache Atlas 2.1.0 rc3, with following fixes since rc2, is now available > > for vote within dev community. > > ATLAS-3877: Purged entity audit throws 404 Entity not found error > > ATLAS-3882: Classic UI: Loader not disappearing while assigning > category > > to term > > > > Links to the release artifacts are given below. Please review and vote. > > > > The vote will be open for at least 72 hours or until necessary votes are > > reached. > > [ ] +1 Approve > > [ ] +0 No opinion > > [ ] -1 Disapprove (and reason why) > > > > Thanks, > > Madhan > > > > > > List of all issues addressed in this release: > > https://issues.apache.org/jira/issues/?jql=project=ATLAS AND > > status=Resolved AND fixVersion=2.1.0 ORDER BY key DESC > > > > Git tag for the release: > > https://github.com/apache/atlas/tree/release-2.1.0-rc3 > > Sources for the release: > > > https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc3/apache-atlas-2.1.0-sources.tar.gz > > > > Source release verification: > > PGP Signature: > > > https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc3/apache-atlas-2.1.0-sources.tar.gz.asc > > SHA512 Hash: > > > https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc3/apache-atlas-2.1.0-sources.tar.gz.sha512 > > MD5 Hash: > > > https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc3/apache-atlas-2.1.0-sources.tar.gz.md5 > > > > Keys to verify the signature of the release artifacts are available at: > > https://dist.apache.org/repos/dist/release/atlas/KEYS > > > > New features/enhancements: > > - Quick Search: provides a simpler search experience with type-ahead > > suggestions > > - Business Metadata: enables augmenting entity-types with additional > > attributes, search entities using these attributes > > - Labels: ability to add/remove labels on entities, and search entities > > using labels > > - Custom Attributes: ability to add entity instance specific custom > > attributes i.e. attributes not defined in entity-def or business metadata > > - Entity Purge: added REST APIs to purge deleted entities > > > > Enhancements: > > - Search: ability to find entities by more than one classification > > - Performance: improvements in lineage retrieval and > > classification-propagation > > - Notification: ability to process notifications from multiple Kafka > > topics > > - Hive Hook: tracks process-executions via hive_process_execution > > entities > > - Hive Hook: captures DDL operations via hive_db_ddl and hive_table_ddl > > entities > > - Notification: introduced shell entities to record references to > > non-existing entities in notifications > > - Spark: added model to capture Spark entities, processes and > > relationships > > - AWS S3: introduced updated model to capture AWS S3 entities and > > relationships > > - ADLS-Gen2: introduced model to capture Azure Data Lake Storage Gen2 > > entities and relationships > > - Dependencies: JanusGraph 0.5.1, Tinkerpop 3.4.6, Spring Framework > > 4.3.20 > > - Authorization: updated to cover new features, like: business > metadata, > > labels, purge > > - UI: multiple UI improvements, including a beta UI > > > > > > >
Re: Review Request 72648: ATLAS-3841 Response Headers: Code refactoring
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72648/#review221143 --- distro/src/conf/atlas-application.properties Lines 285 (patched) <https://reviews.apache.org/r/72648/#comment309933> 1. The name of this property shouldn't be "exclude", because we aren't excluding, just that we aren't overriding Spring's default headers 2. Instead of having list of headers that we won't override Spring's default behaviour, we should have the list of headers whom we are going to customize, like CONTENT_SEC_POLICY_VAL. This list will be smaller than what we will specify here. webapp/src/main/java/org/apache/atlas/web/filters/AtlasHeaderFilter.java Lines 79 (patched) <https://reviews.apache.org/r/72648/#comment309934> 1. Instead of casting again and again, we should put it in a variable and use that variable 2. Instead of passing null, consider passing empty array. - Jayendra Parab On July 7, 2020, 9:30 a.m., Mandar Ambawane wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72648/ > --- > > (Updated July 7, 2020, 9:30 a.m.) > > > Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, > Nixon Rodrigues, and Sarath Subramanian. > > > Bugs: ATLAS-3841 > https://issues.apache.org/jira/browse/ATLAS-3841 > > > Repository: atlas > > > Description > --- > > Currently headers setting code is scattered at different places. > In this patch, code refactoring is done to set response headers from > AtlasHeaderFilter.java > > This patch provides flexibility to select response headers to be excluded. > Also this patch provides flexibility to select file formats to be excluded > from applying response headers. > > > Diffs > - > > distro/src/conf/atlas-application.properties e06e74a > intg/src/main/java/org/apache/atlas/AtlasConfiguration.java 2c007ca > > webapp/src/main/java/org/apache/atlas/web/filters/AtlasAuthenticationFilter.java > d9b1c82 > > webapp/src/main/java/org/apache/atlas/web/filters/AtlasCSRFPreventionFilter.java > df3fce6 > > webapp/src/main/java/org/apache/atlas/web/filters/AtlasDelegatingAuthenticationEntryPoint.java > c629a7e > webapp/src/main/java/org/apache/atlas/web/filters/AtlasHeaderFilter.java > fa7218c > > webapp/src/main/java/org/apache/atlas/web/filters/AtlasKnoxSSOAuthenticationFilter.java > 1944a9f > webapp/src/main/java/org/apache/atlas/web/filters/HeadersUtil.java 1f8845d > > webapp/src/main/java/org/apache/atlas/web/security/AtlasAuthenticationFailureHandler.java > a117df6 > > webapp/src/main/java/org/apache/atlas/web/security/AtlasAuthenticationSuccessHandler.java > e7a5d66 > webapp/src/main/java/org/apache/atlas/web/security/AtlasSecurityConfig.java > e74a9e9 > webapp/src/main/webapp/WEB-INF/web.xml 2595a15 > > > Diff: https://reviews.apache.org/r/72648/diff/1/ > > > Testing > --- > > PreCommit: > https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2015/console > > > Thanks, > > Mandar Ambawane > >
Re: [VOTE] Release Apache Atlas version 2.1.0 - rc2
+1 for Apache Atlas 2.1.0 RC2 release Validated the following: 1. Verified the source SHA512 and MD5 checksum 2. Verified the build with -Pdist,embedded-hbase-solr 3. Started Atlas server successfully 4. Verified quick_start.py, all entities and tag get created successfully 5. Created entities, applied tags from UI 6. Checked Basic and advanced search Regards, Jayendra. On Tue, Jul 7, 2020 at 9:44 AM Nikhil Bonte < nikhil.bo...@freestoneinfotech.com> wrote: > +1 for Apache Atlas 2.1.0 - rc2 release > > Validated the following items. > >- Verified PGP signature. >- Verified md5 and SHA512 digests. >- Packaged with embedded-hbase-solr >- Verified entity creation & tag associations using quick_start.py >- Tested basic, quick & advanced search. >- Tested tag dissociation > > Thanks > Nikhil P. Bonte > > On Mon, Jul 6, 2020 at 12:11 PM Ramesh Mani wrote: > > > +1 for Apache Atlas 2.1.0 rc2 release. > > > > - Build successfully with the source > > > > > https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc2/apache-atlas-2.1.0-sources.tar.gz > > - Verified PGP signature. > > - Verified md5 and SHA512 digests. > > > > Thanks. > > Ramesh > > > > On Sat, Jul 4, 2020 at 11:50 PM Madhan Neethiraj > > wrote: > > > > > Atlas team, > > > > > > Apache Atlas 2.1.0 rc2 is now available for vote within dev community. > > > > > > Links to the release artifacts are given below. Please review and vote. > > > > > > The vote will be open for at least 72 hours or until necessary votes > are > > > reached. > > > [ ] +1 Approve > > > [ ] +0 No opinion > > > [ ] -1 Disapprove (and reason why) > > > > > > Thanks, > > > Madhan > > > > > > > > > List of all issues addressed in this release: > > > https://issues.apache.org/jira/issues/?jql=project=ATLAS AND > > > status=Resolved AND fixVersion=2.1.0 ORDER BY key DESC > > > > > > Git tag for the release: > > > https://github.com/apache/atlas/tree/release-2.1.0-rc2 > > > Sources for the release: > > > > > > https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc2/apache-atlas-2.1.0-sources.tar.gz > > > > > > Source release verification: > > > PGP Signature: > > > > > > https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc2/apache-atlas-2.1.0-sources.tar.gz.asc > > > SHA512 Hash: > > > > > > https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc2/apache-atlas-2.1.0-sources.tar.gz.sha512 > > > MD5 Hash: > > > > > > https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc2/apache-atlas-2.1.0-sources.tar.gz.md5 > > > > > > Keys to verify the signature of the release artifacts are available at: > > > https://dist.apache.org/repos/dist/release/atlas/KEYS > > > > > > New features/enhancements: > > > - Quick Search: provides a simpler search experience with type-ahead > > > suggestions > > > - Business Metadata: enables augmenting entity-types with additional > > > attributes, search entities using these attributes > > > - Labels: ability to add/remove labels on entities, and search > entities > > > using labels > > > - Custom Attributes: ability to add entity instance specific custom > > > attributes i.e. attributes not defined in entity-def or business > metadata > > > - Entity Purge: added REST APIs to purge deleted entities > > > > > > Enhancements: > > > - Search: ability to find entities by more than one classification > > > - Performance: improvements in lineage retrieval and > > > classification-propagation > > > - Notification: ability to process notifications from multiple Kafka > > > topics > > > - Hive Hook: tracks process-executions via hive_process_execution > > > entities > > > - Hive Hook: captures DDL operations via hive_db_ddl and > hive_table_ddl > > > entities > > > - Notification: introduced shell entities to record references to > > > non-existing entities in notifications > > > - Spark: added model to capture Spark entities, processes and > > > relationships > > > - AWS S3: introduced updated model to capture AWS S3 entities and > > > relationships > > > - ADLS-Gen2: introduced model to capture Azure Data Lake Storage Gen2 > > > entities and relationships > > > - Dependencies: JanusGraph 0.5.1, Tinkerpop 3.4.6, Spring Framework > > > 4.3.20 > > > - Authorization: updated to cover new features, like: business > > metadata, > > > labels, purge > > > - UI: multiple UI improvements, including a beta UI > > > > > > > > > > > > > > -- > Regards > Nikhil P. Bonte >
Re: Review Request 72566: ATLAS-1798 : Fix Findbugs problems in repository module
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72566/#review220965 --- repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java Line 309 (original), 312 (patched) <https://reviews.apache.org/r/72566/#comment309702> Remove commented code, its there in multiple places, remove from there as well repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java Line 537 (original), 537 (patched) <https://reviews.apache.org/r/72566/#comment309701> Remove this declaration and move it to line 572 - Jayendra Parab On June 4, 2020, 2:33 a.m., mayank jain wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72566/ > --- > > (Updated June 4, 2020, 2:33 a.m.) > > > Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, > and Sarath Subramanian. > > > Bugs: ATLAS-1798 > https://issues.apache.org/jira/browse/ATLAS-1798 > > > Repository: atlas > > > Description > --- > > Currently Findbugs complaints about some problems (see attachment) in the > repository module. They should be fixed to get the code more reliable. > > > Diffs > - > > repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java > 57e454a > > repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java > dd4d1b4 > > repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java > 56956e6 > > repository/src/main/java/org/apache/atlas/discovery/SearchAggregatorImpl.java > e8f7dbc > repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java > 804c694 > repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java > d630f66 > repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java > 2c84ec7 > repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java > 2a2cebb > repository/src/main/java/org/apache/atlas/query/AtlasDSL.java b8a744b > repository/src/main/java/org/apache/atlas/query/GremlinQueryComposer.java > 801e898 > repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLLexer.java > 142b9ca > repository/src/main/java/org/apache/atlas/query/antlr4/AtlasDSLParser.java > 6b33edb > > repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java > 1aac375 > > repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java > 69d373d > > repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java > 79527ac > > repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java > 9fca744 > > repository/src/main/java/org/apache/atlas/repository/converters/AtlasArrayFormatConverter.java > c335f0a > > repository/src/main/java/org/apache/atlas/repository/converters/AtlasInstanceConverter.java > 6fc0c65 > > repository/src/main/java/org/apache/atlas/repository/converters/AtlasMapFormatConverter.java > 0eacd8e > > repository/src/main/java/org/apache/atlas/repository/converters/AtlasStructFormatConverter.java > ae92b8b > > repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapperV2.java > 497a877 > > repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java > 4a09b08 > repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java > 7b7ec65 > > repository/src/main/java/org/apache/atlas/repository/impexp/AtlasServerService.java > 542106f > > repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java > 0491a85 > > repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java > 1d29bf8 > > repository/src/main/java/org/apache/atlas/repository/impexp/ImportTransforms.java > a2f592c > repository/src/main/java/org/apache/atlas/repository/impexp/ZipSink.java > 6375454 > repository/src/main/java/org/apache/atlas/repository/impexp/ZipSource.java > 812add9 > > repository/src/main/java/org/apache/atlas/repository/impexp/ZipSourceDirect.java > 04342fa > > repository/src/main/java/org/apache/atlas/repository/impexp/ZipSourceWithBackingDirectory.java > 7963800 > > repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationService.java > 0a2257e > > r
Review Request 72558: ATLAS-3779: Added fallback if ticket-basedKafkaClient config is not specified
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72558/ --- Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Bugs: ATLAS-3779 https://issues.apache.org/jira/browse/ATLAS-3779 Repository: atlas Description --- As part of earlier fix for ATLAS-3779, the code to set JAAS configuration for authentication with Kafka was moved to KafkaNotification class. In some scenarios, even though on Atlas Client side, the UserGroupInformation.isLoginTicketBased() is true and UserGroupInformation.isLoginKeytabBased() is false, the atlas.jaas.ticketBased-KafkaClient.* configuration properties are not set in atlas-application.properties file. Because of this, communication with Kafka is affected. Since in the earlier InMemoryJAASConfiguration code, in such scenarios, if JAAS config for atlas.jaas.ticketBased-KafkaClient.* configuration wasn't found then it would try to fallback on the JAAS configuration specified by atlas.jaas.KafkaClient.* properties. This wasn't handled in the earlier fix for ATLAS-3779. This review request provides solution to this problem. NOTE: To add unit testcases for above scenario, changed the access specifier of isLoginKeytabBased() and isLoginTicketBased() methods and made then non-static. Diffs - notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 278b3a742 notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationMockTest.java e345c8bba Diff: https://reviews.apache.org/r/72558/diff/1/ Testing --- Tested in scenario where Atlas client uses ticket based login, but atlas.jaas.ticketBased-KafkaClient.* properties are not specified in atlas-application.properties. Thanks, Jayendra Parab
Contributor Request
Hi All, I would like to contribute to the Apache Atlas project. Can you please add me as a contributor? Regards, Jayendra.
Re: Review Request 72527: Refactoring Kafka JAAS Configuration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72527/ --- (Updated May 27, 2020, 6:10 a.m.) Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Changes --- Addressed the review comment Bugs: ATLAS-3779 https://issues.apache.org/jira/browse/ATLAS-3779 Repository: atlas Description --- The JAAS config set through InMemoryJAASConfiguration overwrites user supplied JAAS configuration which might be required to connect to user's Kakfa instead of Atlas's Kafka and might have different credentials To resolve this we should make use of Kafka's dynamic JAAS configuration mechanism wherein you can set the Kafka JAAS config using sasl.jaas.config property. (https://cwiki.apache.org/confluence/display/KAFKA/KIP-85%3A+Dynamic+JAAS+configuration+for+Kafka+clients) In this patch, we are picking up the atlas.jaas.* properties and setting the sasl.jaas.config property. Since the JAAS configuration is specific only to Kafka, the JAAS property initialization is done while instantiating the KafkaNotification object. Also, since we are using sasl.jaas.config property to set the configuration it's not conflicting with any other JAAS configuration. To keep the code backward compatible for client application which make use of ticketBased-KafkaClient configuration , we are still making use of the earlier mechanism i.e. with the use of UserGroupInformation.isLoginTicketBased() and UserGroupInformation.isLoginKeytabBased() to set the configuration. Diffs (updated) - intg/src/main/java/org/apache/atlas/ApplicationProperties.java 3ba50612e notification/src/main/java/org/apache/atlas/hook/AtlasHook.java cc6546be8 notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 11a29b9e0 notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationMockTest.java 9b5891f27 Diff: https://reviews.apache.org/r/72527/diff/3/ Changes: https://reviews.apache.org/r/72527/diff/2-3/ Testing --- With these changes, tried creating table using CTAS query in Hive. The entities corresponding to the table was updated properly in Atlas server Thanks, Jayendra Parab
Re: Review Request 72527: Refactoring Kafka JAAS Configuration
> On May 19, 2020, 4:26 p.m., Ashutosh Mestry wrote: > > notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java > > Lines 510 (patched) > > <https://reviews.apache.org/r/72527/diff/1/?file=2232586#file2232586line510> > > > > Refactor: Extract constant. This statement checks whether the string is enclosed in double quotes, so can't make use of constants here > On May 19, 2020, 4:26 p.m., Ashutosh Mestry wrote: > > notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java > > Lines 512 (patched) > > <https://reviews.apache.org/r/72527/diff/1/?file=2232586#file2232586line512> > > > > Refactor: Promote to const field. These set of characters are specific to this method, so won't be of much use if we promote to class level field. - Jayendra --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72527/#review220830 --- On May 25, 2020, 9:04 a.m., Jayendra Parab wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72527/ > --- > > (Updated May 25, 2020, 9:04 a.m.) > > > Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath > Subramanian. > > > Bugs: ATLAS-3779 > https://issues.apache.org/jira/browse/ATLAS-3779 > > > Repository: atlas > > > Description > --- > > The JAAS config set through InMemoryJAASConfiguration overwrites user > supplied JAAS configuration which might be required to connect to user's > Kakfa instead of Atlas's Kafka and might have different credentials > > To resolve this we should make use of Kafka's dynamic JAAS configuration > mechanism wherein you can set the Kafka JAAS config using sasl.jaas.config > property. > (https://cwiki.apache.org/confluence/display/KAFKA/KIP-85%3A+Dynamic+JAAS+configuration+for+Kafka+clients) > > In this patch, we are picking up the atlas.jaas.* properties and setting the > sasl.jaas.config property. Since the JAAS configuration is specific only to > Kafka, the JAAS property initialization is done while instantiating the > KafkaNotification object. Also, since we are using sasl.jaas.config property > to set the configuration it's not conflicting with any other JAAS > configuration. > > To keep the code backward compatible for client application which make use of > ticketBased-KafkaClient configuration , we are still making use of the > earlier mechanism i.e. with the use of > UserGroupInformation.isLoginTicketBased() and > UserGroupInformation.isLoginKeytabBased() to set the configuration. > > > Diffs > - > > intg/src/main/java/org/apache/atlas/ApplicationProperties.java 3ba50612e > notification/src/main/java/org/apache/atlas/hook/AtlasHook.java cc6546be8 > notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java > 11a29b9e0 > > notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationMockTest.java > 9b5891f27 > > > Diff: https://reviews.apache.org/r/72527/diff/2/ > > > Testing > --- > > With these changes, tried creating table using CTAS query in Hive. The > entities corresponding to the table was updated properly in Atlas server > > > Thanks, > > Jayendra Parab > >
Re: Review Request 72527: Refactoring Kafka JAAS Configuration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72527/ --- (Updated May 25, 2020, 9:04 a.m.) Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Changes --- Addressed review comments and added test cases Bugs: ATLAS-3779 https://issues.apache.org/jira/browse/ATLAS-3779 Repository: atlas Description --- The JAAS config set through InMemoryJAASConfiguration overwrites user supplied JAAS configuration which might be required to connect to user's Kakfa instead of Atlas's Kafka and might have different credentials To resolve this we should make use of Kafka's dynamic JAAS configuration mechanism wherein you can set the Kafka JAAS config using sasl.jaas.config property. (https://cwiki.apache.org/confluence/display/KAFKA/KIP-85%3A+Dynamic+JAAS+configuration+for+Kafka+clients) In this patch, we are picking up the atlas.jaas.* properties and setting the sasl.jaas.config property. Since the JAAS configuration is specific only to Kafka, the JAAS property initialization is done while instantiating the KafkaNotification object. Also, since we are using sasl.jaas.config property to set the configuration it's not conflicting with any other JAAS configuration. To keep the code backward compatible for client application which make use of ticketBased-KafkaClient configuration , we are still making use of the earlier mechanism i.e. with the use of UserGroupInformation.isLoginTicketBased() and UserGroupInformation.isLoginKeytabBased() to set the configuration. Diffs (updated) - intg/src/main/java/org/apache/atlas/ApplicationProperties.java 3ba50612e notification/src/main/java/org/apache/atlas/hook/AtlasHook.java cc6546be8 notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 11a29b9e0 notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationMockTest.java 9b5891f27 Diff: https://reviews.apache.org/r/72527/diff/2/ Changes: https://reviews.apache.org/r/72527/diff/1-2/ Testing --- With these changes, tried creating table using CTAS query in Hive. The entities corresponding to the table was updated properly in Atlas server Thanks, Jayendra Parab
Re: Review Request 72527: Refactoring Kafka JAAS Configuration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72527/ --- (Updated May 19, 2020, 11:25 a.m.) Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath Subramanian. Bugs: ATLAS-3779 https://issues.apache.org/jira/browse/ATLAS-3779 Repository: atlas Description --- The JAAS config set through InMemoryJAASConfiguration overwrites user supplied JAAS configuration which might be required to connect to user's Kakfa instead of Atlas's Kafka and might have different credentials To resolve this we should make use of Kafka's dynamic JAAS configuration mechanism wherein you can set the Kafka JAAS config using sasl.jaas.config property. (https://cwiki.apache.org/confluence/display/KAFKA/KIP-85%3A+Dynamic+JAAS+configuration+for+Kafka+clients) In this patch, we are picking up the atlas.jaas.* properties and setting the sasl.jaas.config property. Since the JAAS configuration is specific only to Kafka, the JAAS property initialization is done while instantiating the KafkaNotification object. Also, since we are using sasl.jaas.config property to set the configuration it's not conflicting with any other JAAS configuration. To keep the code backward compatible for client application which make use of ticketBased-KafkaClient configuration , we are still making use of the earlier mechanism i.e. with the use of UserGroupInformation.isLoginTicketBased() and UserGroupInformation.isLoginKeytabBased() to set the configuration. Diffs - intg/src/main/java/org/apache/atlas/ApplicationProperties.java 3ba50612e notification/src/main/java/org/apache/atlas/hook/AtlasHook.java cc6546be8 notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 11a29b9e0 Diff: https://reviews.apache.org/r/72527/diff/1/ Testing --- With these changes, tried creating table using CTAS query in Hive. The entities corresponding to the table was updated properly in Atlas server Thanks, Jayendra Parab
