[DISCUSS] merge RANGER-3923 branch into master branch
Rangers, For more than a year now, Apache Ranger community has been adding significant enhancements in RANGER-3923 branch. These enhancements enable business managers to manage access to datasets, instead of having data owners to manage access to individual resources like tables, columns, files, directories. In addition, this approach offers several benefits including: reduced time for users in getting access to data across multiple services, with a single policy update. business managers like project managers to manage access to datasets instead of multiple data owners, which reduces the time taken to get access to data. separation of responsibilities: data owners focus on building datasets, while business managers focus on managing access to datasets. eliminate the need to update access policies for any changes in datasets like add/remove/change resources. Apache Ranger community has built APIs and UI to manage datasets and access to datasets. Apache Ranger policy engine has been updated as well to support datasets. I propose merging these enhancements into master branch this week, and work towards releasing Apache Ranger 3.0 version, by next month. Please share your feedback. Looking forward to Apache Ranger 3.0 release! Thanks, Madhan
[jira] [Resolved] (ATLAS-4754) Download Search with Basic Search gives java.io.FileNotFoundException
[
https://issues.apache.org/jira/browse/ATLAS-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mandar Ambawane resolved ATLAS-4754.
Resolution: Fixed
Default value for the download directory will be read from system property
"user.dir" instead of property "atlas.home"
> Download Search with Basic Search gives java.io.FileNotFoundException
> -
>
> Key: ATLAS-4754
> URL: https://issues.apache.org/jira/browse/ATLAS-4754
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Abhishek Pal
>Assignee: Mandar Ambawane
>Priority: Major
>
> Currently when trying to download search results, the download task is stuck
> on pending.
> Upon checking the logs the following error is being seen:
> {code:java|title=application.log}
> 2023-05-17 22:09:47,321 INFO - [etp1158258131-199 -
> a6302f98-7e3d-4cbc-bd06-883fbfa21d11:] ~ TaskManagement: Processing stats:
> total=1, sinceLastStatsReport=2 completedWithErrors=1, succeded=0
> (TaskExecutor$TaskLogger:166)
> 2023-05-17 22:09:47,330 ERROR - [atlas-task-0-etp1158258131-282 -
> b9a5a4ef-6e3f-440c-9f65-d446a6da9720:] ~ Task:
> f4e26459-746f-4f31-90db-144c31a9696f: Error performing task!
> (SearchResultDownloadTask:126)
> java.io.FileNotFoundException:
> /search_result_downloads//.csv (No such file
> or directory)
> at java.base/java.io.FileOutputStream.open0(Native Method)
> at java.base/java.io.FileOutputStream.open(FileOutputStream.java:298)
> at java.base/java.io.FileOutputStream.(FileOutputStream.java:237)
> at java.base/java.io.FileOutputStream.(FileOutputStream.java:187)
> at java.base/java.io.FileWriter.(FileWriter.java:96)
> at
> org.apache.atlas.repository.store.graph.v2.tasks.searchdownload.SearchResultDownloadTask.generateCSVFileFromSearchResult(SearchResultDownloadTask.java:185)
> at
> org.apache.atlas.repository.store.graph.v2.tasks.searchdownload.SearchResultDownloadTask.run(SearchResultDownloadTask.java:163)
> at
> org.apache.atlas.repository.store.graph.v2.tasks.searchdownload.SearchResultDownloadTask.perform(SearchResultDownloadTask.java:122)
> at org.apache.atlas.tasks.AbstractTask.run(AbstractTask.java:33)
> at
> org.apache.atlas.tasks.TaskExecutor$TaskConsumer.performTask(TaskExecutor.java:150)
> at org.apache.atlas.tasks.TaskExecutor$TaskConsumer.run(TaskExecutor.java:109)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834){code}
> h2. Steps to Reproduce:
> h4. Create type: test_download_entity
> {code:java|title=/api/atlas/v2/types/typedefs}
> {
> "enumDefs": [],
> "structDefs": [],
> "classificationDefs": [],
> "entityDefs": [
> {
> "attributeDefs": [
> {
> "name": "name",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": false,
> "isIndexable": false
> },
> {
> "name": "type_str",
> "typeName": "string",
> "isOptional": true,
> "cardinality": "SINGLE",
> "valuesMinCount": 0,
> "valuesMaxCount": 1,
> "isUnique": false,
> "isIndexable": false
> }
> ],
> "description": "description",
> "name": "test_download_entity",
> "guid": "-32062751815011",
> "category": "ENTITY",
> "superTypes": []
> }
> ],
> "relationshipDefs": [],
> "businessMetadataDefs": []
> }
> {code}
> h4. Create entity of type: test_download_entity
> {code:java|title=/api/atlas/v2/entity/bulk}
> {
> "entities": [{
> "typeName": "test_download_entity",
> "attributes": {
> "name": "downloadable_entity_1",
> "type_str": "str1"
> },
> "guid": "-08727943682221"
> },
> {
> "typeName": "test_download_entity",
> "attributes": {
> "name": "downloadable_entity_2",
> "type_str": "str2"
> },
>
Re: Review Request 74691: ATLAS-4802 : Atlas 'updateTime' parameter is not updated when business metadata is edited.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74691/
---
(Updated Jan. 2, 2024, 10:35 a.m.)
Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, and
Sheetal Shah.
Bugs: ATLAS-4802
https://issues.apache.org/jira/browse/ATLAS-4802
Repository: atlas
Description
---
When updating the business metadata and Labels on an atlas entity, the
'updateTime' system attribute is not changed.
Updates to other aspects, such as technical properties and user defined
attributes does update the timestamp, but not the business metadata and labels.
Diffs
-
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
c670740c9
Diff: https://reviews.apache.org/r/74691/diff/2/
Testing (updated)
---
update time Without patch
before business metadata update
"createTime": 1703221384530,
"updateTime": 1703221386989,
after business metadata update
"createTime": 1703221384530,
"updateTime": 1703221386989,
after applying PATCH
before business metadata update
"createTime": 1704190056433,
"updateTime": 1704190059208,
after business metadata update
"createTime": 1704190056433,
"updateTime": 1704190523323,
Entire json for GET api on entity before applying patch :
http://localhost:21000/api/atlas/v2/entity/guid/dc1baa88-3a07-4577-9b6b-48982253aa3d
{
"referredEntities": {
"a1a27e35-aa0d-472d-8b94-2405d8001a84": {
"typeName": "Column",
"attributes": {
"qualifiedName": "Sales.customer_dim.address@cl1",
"name": "address",
"comment": "customer address"
},
"guid": "a1a27e35-aa0d-472d-8b94-2405d8001a84",
"isIncomplete": false,
"status": "ACTIVE",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1704190056433,
"updateTime": 1704190056433,
"version": 0,
"classifications": [
{
"typeName": "PII",
"entityGuid": "a1a27e35-aa0d-472d-8b94-2405d8001a84",
"entityStatus": "ACTIVE",
"propagate": true,
"removePropagationsOnEntityDelete": false
}
],
"labels": []
},
"834ba687-c8f1-4a48-a6a0-ef3ddeadf7ee": {
"typeName": "Column",
"attributes": {
"qualifiedName": "Sales.customer_dim.name@cl1",
"name": "name",
"comment": "customer name"
},
"guid": "834ba687-c8f1-4a48-a6a0-ef3ddeadf7ee",
"isIncomplete": false,
"status": "ACTIVE",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1704190056433,
"updateTime": 1704190056433,
"version": 0,
"classifications": [
{
"typeName": "PII",
"entityGuid": "834ba687-c8f1-4a48-a6a0-ef3ddeadf7ee",
"entityStatus": "ACTIVE",
"propagate": true,
"removePropagationsOnEntityDelete": false
}
],
"labels": []
},
"f8ded729-7bad-48dd-9969-c5b36a8d5b9c": {
"typeName": "Column",
"attributes": {
"qualifiedName": "Sales.customer_dim.customer_id@cl1",
"name": "customer_id",
"comment": "customer id"
},
"guid": "f8ded729-7bad-48dd-9969-c5b36a8d5b9c",
"isIncomplete": false,
"status": "ACTIVE",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1704190056433,
"updateTime": 1704190056433,
"version": 0,
"classifications": [
{
"typeName": "PII",
"entityGuid": "f8ded729-7bad-48dd-9969-c5b36a8d5b9c",
"entityStatus": "ACTIVE",
"propagate": true,
"removePropagationsOnEntityDelete": false
}
],
"labels": []
},
"06f3f56c-8ed7-4930-93b3-8a4b6f81c505": {
"typeName": "StorageDesc",
"attributes": {
"qualifiedName": "sd:hdfs://host:8000/apps/warehouse/sales@cl1",
"name": "sd:hdfs://host:8000/apps/warehouse/sales"
},
"guid": "06f3f56c-8ed7-4930-93b3-8a4b6f81c505",
"isIncomplete": false,
"status": "ACTIVE",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1704190055353,
"updateTime": 1704190058427,
Re: Review Request 74691: ATLAS-4802 : Atlas 'updateTime' parameter is not updated when business metadata is edited.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74691/ --- (Updated Jan. 2, 2024, 8:35 a.m.) Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, and Sheetal Shah. Bugs: ATLAS-4802 https://issues.apache.org/jira/browse/ATLAS-4802 Repository: atlas Description --- When updating the business metadata and Labels on an atlas entity, the 'updateTime' system attribute is not changed. Updates to other aspects, such as technical properties and user defined attributes does update the timestamp, but not the business metadata and labels. Diffs - repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java c670740c9 Diff: https://reviews.apache.org/r/74691/diff/2/ Testing (updated) --- PC successful Thanks, chaitali
Re: Review Request 74691: ATLAS-4802 : Atlas 'updateTime' parameter is not updated when business metadata is edited.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74691/ --- (Updated Jan. 2, 2024, 8:33 a.m.) Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, and Sheetal Shah. Bugs: ATLAS-4802 https://issues.apache.org/jira/browse/ATLAS-4802 Repository: atlas Description --- When updating the business metadata and Labels on an atlas entity, the 'updateTime' system attribute is not changed. Updates to other aspects, such as technical properties and user defined attributes does update the timestamp, but not the business metadata and labels. Diffs (updated) - repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java c670740c9 Diff: https://reviews.apache.org/r/74691/diff/2/ Changes: https://reviews.apache.org/r/74691/diff/1-2/ Testing --- Thanks, chaitali
Re: Review Request 74691: ATLAS-4802 : Atlas 'updateTime' parameter is not updated when business metadata is edited.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74691/ --- (Updated Jan. 2, 2024, 8:28 a.m.) Review request for atlas, Jayendra Parab, Mandar Ambawane, Pinal Shah, and Sheetal Shah. Bugs: ATLAS-4802 https://issues.apache.org/jira/browse/ATLAS-4802 Repository: atlas Description --- When updating the business metadata and Labels on an atlas entity, the 'updateTime' system attribute is not changed. Updates to other aspects, such as technical properties and user defined attributes does update the timestamp, but not the business metadata and labels. Diffs - repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java c670740c9 Diff: https://reviews.apache.org/r/74691/diff/1/ Testing --- Thanks, chaitali
