date:20200811

Re: Review Request 72756: ATLAS-3918 : Regression : Renaming table has exceptions in Atlas application logs

2020-08-11 Thread Madhan Neethiraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72756/#review221551
---



For some DDLs like 'create view', HMS doesn't have all details. So, it is 
critical that we process DDLs in HiveServer2.

Consider adding a new message-type ENTITY_RENAME, and for such messages Atlas 
should ignore if source entity is not found.

- Madhan Neethiraj


On Aug. 11, 2020, 3:21 p.m., chaitali wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72756/
> ---
> 
> (Updated Aug. 11, 2020, 3:21 p.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3918
> https://issues.apache.org/jira/browse/ATLAS-3918
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Issue : Messages are being sent for DDL operations one from HMS and 
> HiveServer2 which causes duplicate messages hence it throws exception:
> 
>  message 
> {"version":{"version":"1.0.0","versionParts":[1]},"msgCompressionKind":"NONE","msgSplitIdx":1,"msgSplitCount":1,"msgCreationTime":1594802485264,"message":{"type":"ENTITY_PARTIAL_UPDATE_V2","user":"hrt_qa","entityId":{"typeName":"hive_table","uniqueAttributes":{"qualifiedName":"default.t1@cm"}},"entity":{"entity":{"typeName":"hive_table","attributes":{"owner":"hrt_qa","tableType":"MANAGED_TABLE","temporary":false,"lastAccessTime":1594802469000,"aliases":["t1"],"createTime":1594802469000,"qualifiedName":"default.t2@cm","name":"t2","comment":null,"parameters":{"last_modified_time":"1594802481","rawDataSize":"0","numRows":"0","transient_lastDdlTime":"1594802481","bucketing_version":"2","last_modified_by":"hrt_qa","numFilesErasureCoded":"0","totalSize":"0","transactional_properties":"default","COLUMN_STATS_ACCURATE":"{\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"id\":\"true\"}}","numFiles":"0","transactional":"true"},"retention":0},"guid":"-21782793831367970","isIncomplete":fal
 se,"provenanceType":0,"version":0,"proxy":false 
>  
> This patch fix it by providing a property "DDL_MESSAGES_ENABLE" in 
> configuration when enabled the HiveServer2 will send ddl messages and if 
> disabled then it will only send non ddl messages.
> 
> 
> Diffs
> -
> 
>   addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
> 651323490 
> 
> 
> Diff: https://reviews.apache.org/r/72756/diff/3/
> 
> 
> Testing
> ---
> 
> Tested by creating and renaming the hive_table hence ddl messages are not 
> sent from hiveserver2 after the property is set to false and so no exception 
> is thrown: 
> > create table t1(id int);
> 
> > alter table t1 rename to t2;
> 
> 
> Thanks,
> 
> chaitali
> 
>

[jira] [Commented] (ATLAS-3884) Unable to access Apache Atlas UI http://localhost:21000

2020-08-11 Thread Edward Armes (Jira)



[ 
https://issues.apache.org/jira/browse/ATLAS-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175881#comment-17175881
 ] 

Edward Armes commented on ATLAS-3884:
-

Update: I modified conf/atlias_config.py appending -force to the start and 
create calls to pOpen() and that appears to fix the issue with Solr not 
starting correctly.

I also noticed that python script when a log direcrtory is set for Solr it 
appears not bo logging to logs directory

> Unable to access Apache Atlas UI http://localhost:21000
> ---
>
> Key: ATLAS-3884
> URL: https://issues.apache.org/jira/browse/ATLAS-3884
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-webui
>Affects Versions: 2.0.0
> Environment: RHEL
>Reporter: Vidya
>Priority: Major
>
> Hi,I have installed and ran Apache Atlas with embedded Apache HBase & Apache 
> Solr.After bin/atlas_start.py command I see apache atlas started message and 
> I could grep Atlas,Hbase and Solr process.However,when I do `curl -u 
> admin:admin http://localhost:21000/api/atlas/admin/version`, I am getting 
> below error
> ```
> 
> 
> 
> Error 503 
> 
> 
> HTTP ERROR: 503
> Problem accessing /api/atlas/admin/version. Reason:
>  Service Unavailable
> http://eclipse.org/jetty;>Powered by Jetty:// 
> 9.3.14.v20161028
> 
> 
> ```
> I am getting below error in application logs.
> ```
> Caused by: org.apache.solr.common.SolrException: Cannot connect to cluster at 
> localhost:2181: cluster not found/not ready
>  at 
> org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:385)
>  at 
> org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:141)
>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:383)
>  at org.janusgraph.diskstorage.solr.Solr6Index.(Solr6Index.java:218)
>  ... 101 more 
> ```



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 72730: ATLAS-3875: Introduce AtlasClient in Python

2020-08-11 Thread Sidharth Mishra


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72730/#review221546
---




atlas-examples/sample-app/src/main/python/discovery_example.py
Lines 3 (patched)


Remove two extra blank lines. Along with this please add content to 
README.md file as well



atlas-examples/sample-app/src/main/python/discovery_example.py
Lines 12 (patched)


should we rename it to something more meaningful?



atlas-examples/sample-app/src/main/python/utils.py
Lines 11 (patched)


I think it will be good to rename these const to have some prefix like 
CUSTOM or SAMPLE to differentiate between the existing table types.



client/client-v2/src/main/python/atlas_client/base_client.py
Lines 84 (patched)


Should we have custom exceptions like AtlasException instead of Exception 
thrown from most of the python client similar to Java client. This will help 
the users of the client to handle appropriately and distinguish from the 
generic python exceptions



client/client-v2/src/main/python/atlas_client/discovery_client.py
Lines 6 (patched)


Remove one extra blank line



client/client-v2/src/main/python/atlas_client/discovery_client.py
Lines 20 (patched)


Can we have vertical indentation like:

DSL_SEARCH   = API(DSL_SEARCH_URI, HttpMethod.GET, HTTPStatus.OK)
FULL_TEXT_SEARCH = API(FULL_TEXT_SEARCH_URI, HttpMethod.GET, 
HTTPStatus.OK)
BASIC_SEARCH = API(BASIC_SEARCH_URI, HttpMethod.GET, HTTPStatus.OK)
FACETED_SEARCH   = API(FACETED_SEARCH_URI, HttpMethod.POST, 
HTTPStatus.OK)
ATTRIBUTE_SEARCH = API(DISCOVERY_URI + "/attribute", HttpMethod.GET, 
HTTPStatus.OK)

In case you are changing please take care of this at other places



client/client-v2/src/main/python/atlas_client/discovery_model.py
Lines 18 (patched)


Add blank line after the header comment. Please do at other files as well



client/client-v2/src/main/python/atlas_client/discovery_model.py
Lines 22 (patched)


Please remove one extra blank line (we always keep one blank line as 
separator if needed)



client/client-v2/src/main/python/atlas_client/discovery_model.py
Lines 145 (patched)


Please remove if not needed


- Sidharth Mishra


On Aug. 10, 2020, 5:26 p.m., Jyoti Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72730/
> ---
> 
> (Updated Aug. 10, 2020, 5:26 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Sarath 
> Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3875
> https://issues.apache.org/jira/browse/ATLAS-3875
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> - Added basic structure for Python client
> - This includes all classes for Entity and TypeDef
> - Added all endpoints for typedef
> 
> 
> Diffs
> -
> 
>   atlas-examples/sample-app/src/main/python/discovery_example.py PRE-CREATION 
>   atlas-examples/sample-app/src/main/python/entity_example.py PRE-CREATION 
>   atlas-examples/sample-app/src/main/python/glossary_example.py PRE-CREATION 
>   atlas-examples/sample-app/src/main/python/lineage_example.py PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/python/request_json/entity_create_db.json 
> PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/python/request_json/entity_create_process.json
>  PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/python/request_json/entity_create_table_canada.json
>  PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/python/request_json/entity_create_table_us.json
>  PRE-CREATION 
>   atlas-examples/sample-app/src/main/python/request_json/typedef_create.json 
> PRE-CREATION 
>   atlas-examples/sample-app/src/main/python/sample_client.py PRE-CREATION 
>   atlas-examples/sample-app/src/main/python/typedef_example.py PRE-CREATION 
>   atlas-examples/sample-app/src/main/python/utils.py PRE-CREATION 
>   client/client-v2/src/main/python/LICENSE PRE-CREATION 
>   client/client-v2/src/main/python/README.md PRE-CREATION 
>   client/client-v2/src/main/python/atlas_client/__init__.py PRE-CREATION 
>   client/client-v2/src/main/python/atlas_client/base_client.py PRE-CREATION 
>   client/client-v2/src/main/python/atlas_client/discovery_client.py 
> PRE-CREATION 
>

[jira] [Commented] (ATLAS-3916) Get metrics according to the user permissions

2020-08-11 Thread Sarath Subramanian (Jira)



[ 
https://issues.apache.org/jira/browse/ATLAS-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17175783#comment-17175783
 ] 

Sarath Subramanian commented on ATLAS-3916:
---

[~yued], good requirement. The way metrics query works today is we rely on solr 
to provide you with the totalCount of a specific type. 
{code:java}
atlasGraph.indexQuery(VERTEX_INDEX, indexQuery).vertexTotals();
{code}
If metrics query should display the count of only those entities which user has 
read authorization.

We may need to go over individual entities for each type and check if user has 
authorization. 

 

> Get metrics according to the user permissions
> -
>
> Key: ATLAS-3916
> URL: https://issues.apache.org/jira/browse/ATLAS-3916
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Yue Dong
>Priority: Major
> Attachments: Captura-de-pantalla-de-2020-08-11-10-20-06.png
>
>
> I have two user groups: admin who has access to all tables and reader can 
> only see public data and module A tables. So I have configured Atlas to use a 
> simple authorizer with a little variation, which is to hide entities that are 
> not accessible to the user.
> The searches and displaying results work properly.
> The only problem I find is that the metrics. In the elements of search by 
> type, it indicates the number of all the entities of each type in the system. 
> And this is not consistent with the search result of a reader user. 
> !Captura-de-pantalla-de-2020-08-11-10-20-06.png!
>  
> I have verified that these numbers come from the getMetrics method, which is 
> not secured so it does not obtain the numbers according to the users' 
> configuration. Am I missing something? Is there any way to change these 
> numbers?
> Maybe it'd be nice to have something that allows to modify the querys of the 
> metrics based on security and authorization, like 
> AtlasAuthorizer.scrubSearchResults in search methods.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 72477: ATLAS-3583 Use Audit framework to generate audit entries for TypeDefs CREATE, UPDATE and DELETE

2020-08-11 Thread Madhan Neethiraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72477/#review221545
---


Ship it!




Ship It!

- Madhan Neethiraj


On Aug. 11, 2020, 5:25 a.m., Mandar Ambawane wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72477/
> ---
> 
> (Updated Aug. 11, 2020, 5:25 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> Sarath Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3583
> https://issues.apache.org/jira/browse/ATLAS-3583
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-3583 Use Audit framework to generate audit entries for TypeDefs CREATE, 
> UPDATE and DELETE
> 
> 
> Diffs
> -
> 
>   addons/models/-Area0/0010-base_model.json 9777b7a 
>   
> addons/models/-Area0/patches/006-base_model_add_atlas_operation_attributes.json
>  PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
> a95cf4e 
>   intg/src/main/java/org/apache/atlas/model/audit/AuditSearchParameters.java 
> 9120062 
>   intg/src/test/java/org/apache/atlas/TestUtilsV2.java 2b9cf6e 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
>  a0dc816 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/TypeDefAuditListener.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/store/bootstrap/AtlasTypeDefStoreInitializer.java
>  8e7c1b3 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityChangeNotifier.java
>  0dc3193 
>   repository/src/test/java/org/apache/atlas/TestModules.java a298934 
>   test-tools/src/main/resources/solr/core-template/solrconfig.xml 8ebbeff 
>   webapp/src/test/java/org/apache/atlas/web/adapters/TestTypeDefsREST.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/72477/diff/7/
> 
> 
> Testing
> ---
> 
> Basic testing is done.
> 
> Pre-commit: 
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1871/console
> 
> Pre-commit: 
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1885/console
> 
> Pre-commit: 
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1888/console
> 
> Pre-commit: 
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2071/console
> 
> Pre-commit: 
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2082/console
> 
> Pre-commit: 
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2100/console
> 
> 
> Thanks,
> 
> Mandar Ambawane
> 
>

Re: Review Request 72756: ATLAS-3918 : Regression : Renaming table has exceptions in Atlas application logs

2020-08-11 Thread chaitali


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72756/
---

(Updated Aug. 11, 2020, 3:21 p.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, Nixon 
Rodrigues, and Sarath Subramanian.


Bugs: ATLAS-3918
https://issues.apache.org/jira/browse/ATLAS-3918


Repository: atlas


Description (updated)
---

Issue : Messages are being sent for DDL operations one from HMS and HiveServer2 
which causes duplicate messages hence it throws exception:

 message 
{"version":{"version":"1.0.0","versionParts":[1]},"msgCompressionKind":"NONE","msgSplitIdx":1,"msgSplitCount":1,"msgCreationTime":1594802485264,"message":{"type":"ENTITY_PARTIAL_UPDATE_V2","user":"hrt_qa","entityId":{"typeName":"hive_table","uniqueAttributes":{"qualifiedName":"default.t1@cm"}},"entity":{"entity":{"typeName":"hive_table","attributes":{"owner":"hrt_qa","tableType":"MANAGED_TABLE","temporary":false,"lastAccessTime":1594802469000,"aliases":["t1"],"createTime":1594802469000,"qualifiedName":"default.t2@cm","name":"t2","comment":null,"parameters":{"last_modified_time":"1594802481","rawDataSize":"0","numRows":"0","transient_lastDdlTime":"1594802481","bucketing_version":"2","last_modified_by":"hrt_qa","numFilesErasureCoded":"0","totalSize":"0","transactional_properties":"default","COLUMN_STATS_ACCURATE":"{\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"id\":\"true\"}}","numFiles":"0","transactional":"true"},"retention":0},"guid":"-21782793831367970","isIncomplete":false
 ,"provenanceType":0,"version":0,"proxy":false 
 
This patch fix it by providing a property "DDL_MESSAGES_ENABLE" in 
configuration when enabled the HiveServer2 will send ddl messages and if 
disabled then it will only send non ddl messages.


Diffs
-

  addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
651323490 


Diff: https://reviews.apache.org/r/72756/diff/3/


Testing (updated)
---

Tested by creating and renaming the hive_table hence ddl messages are not sent 
from hiveserver2 after the property is set to false and so no exception is 
thrown: 
> create table t1(id int);

> alter table t1 rename to t2;


Thanks,

chaitali

Re: Review Request 72756: ATLAS-3918 : Regression : Renaming table has exceptions in Atlas application logs

2020-08-11 Thread chaitali


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72756/
---

(Updated Aug. 11, 2020, 3:16 p.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, Nixon 
Rodrigues, and Sarath Subramanian.


Bugs: ATLAS-3918
https://issues.apache.org/jira/browse/ATLAS-3918


Repository: atlas


Description (updated)
---

Issue : Messages are being sent for DDL operations one from HMS and 
HiveServer2(hive_on_tez) which causes duplicate messages hence it throws 
exception.

 message 
{"version":{"version":"1.0.0","versionParts":[1]},"msgCompressionKind":"NONE","msgSplitIdx":1,"msgSplitCount":1,"msgCreationTime":1594802485264,"message":{"type":"ENTITY_PARTIAL_UPDATE_V2","user":"hrt_qa","entityId":{"typeName":"hive_table","uniqueAttributes":{"qualifiedName":"default.t1@cm"}},"entity":{"entity":{"typeName":"hive_table","attributes":{"owner":"hrt_qa","tableType":"MANAGED_TABLE","temporary":false,"lastAccessTime":1594802469000,"aliases":["t1"],"createTime":1594802469000,"qualifiedName":"default.t2@cm","name":"t2","comment":null,"parameters":{"last_modified_time":"1594802481","rawDataSize":"0","numRows":"0","transient_lastDdlTime":"1594802481","bucketing_version":"2","last_modified_by":"hrt_qa","numFilesErasureCoded":"0","totalSize":"0","transactional_properties":"default","COLUMN_STATS_ACCURATE":"{\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"id\":\"true\"}}","numFiles":"0","transactional":"true"},"retention":0},"guid":"-21782793831367970","isIncomplete":false
 ,"provenanceType":0,"version":0,"proxy":false 
 
This patch fix it by providing a property in configuration when enabled the 
HiveServer2 will send ddl messages and if disabled then it will only send non 
ddl messages.


Diffs
-

  addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java 
651323490 


Diff: https://reviews.apache.org/r/72756/diff/3/


Testing (updated)
---

Tested by creating and renaming the hive_table and it doesn't throw above error 
message when the property is set to false : 
> create table t1(id int);

> alter table t1 rename to t2;


Thanks,

chaitali

[jira] [Updated] (ATLAS-3917) While deleting parent tag, shows incorrect message.

2020-08-11 Thread chaitali borole (Jira)



 [ 
https://issues.apache.org/jira/browse/ATLAS-3917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chaitali borole updated ATLAS-3917:
---
Affects Version/s: 2.1.0

> While deleting parent tag, shows incorrect message.
> ---
>
> Key: ATLAS-3917
> URL: https://issues.apache.org/jira/browse/ATLAS-3917
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Durga Kadam
>Priority: Minor
> Attachments: tag_with_{0}_reproducible.mkv
>
>
> Actual Behaviour - While deleting parent tag if the tag is associated with 
> any entity shows this message :: "Failed to delete classification 
> 'parent_abc'. Given type \{0} has reference."
> Expected Result - There should be value instead of \{0}
> Steps to generate
>  # Create tag
>  # Associate the created tag with any of the entity
>  # Try to delete the tag
>  # Shows validation message with some encrypted value
>  
> PFA Evidence file



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (ATLAS-3918) Regression : Renaming table has exceptions in Atlas application logs

2020-08-11 Thread chaitali borole (Jira)

chaitali borole created ATLAS-3918:
--

 Summary: Regression : Renaming table has exceptions in Atlas 
application logs
 Key: ATLAS-3918
 URL: https://issues.apache.org/jira/browse/ATLAS-3918
 Project: Atlas
  Issue Type: Improvement
  Components: hive-integration
Affects Versions: 2.1.0
Reporter: chaitali borole
Assignee: chaitali borole
 Fix For: 3.0.0


Reproduce steps :

> create table t1(id int);

> alter table t1 rename to t2;

Exception :
{code:java}
2020-07-15 08:41:25,265 WARN  - [NotificationHookConsumer thread-0:] ~ Max 
retries exceeded for message 
{"version":{"version":"1.0.0","versionParts":[1]},"msgCompressionKind":"NONE","msgSplitIdx":1,"msgSplitCount":1,"msgCreationTime":1594802485264,"message":{"type":"ENTITY_PARTIAL_UPDATE_V2","user":"hrt_qa","entityId":{"typeName":"hive_table","uniqueAttributes":{"qualifiedName":"default.t1@cm"}},"entity":{"entity":{"typeName":"hive_table","attributes":{"owner":"hrt_qa","tableType":"MANAGED_TABLE","temporary":false,"lastAccessTime":1594802469000,"aliases":["t1"],"createTime":1594802469000,"qualifiedName":"default.t2@cm","name":"t2","comment":null,"parameters":{"last_modified_time":"1594802481","rawDataSize":"0","numRows":"0","transient_lastDdlTime":"1594802481","bucketing_version":"2","last_modified_by":"hrt_qa","numFilesErasureCoded":"0","totalSize":"0","transactional_properties":"default","COLUMN_STATS_ACCURATE":"{\"BASIC_STATS\":\"true\",\"COLUMN_STATS\":{\"id\":\"true\"}}","numFiles":"0","transactional":"true"},"retention":0},"guid":"-21782793831367970","isIncomplete":false,"provenanceType":0,"version":0,"proxy":false
 (NotificationHookConsumer$HookConsumer:793) {code}
because , in the message sent to ATLAS_HOOK , in the "attributes" dict , right 
qualifiedName is given, but in "uniqueAttributes" it still has the old 
qualifiedName.

"uniqueAttributes":{"qualifiedName":"*default.t1@cm"*}},"entity":{"entity":{"typeName":"hive_table","attributes":{"owner":"hrt_qa","tableType":"MANAGED_TABLE","temporary":false,"lastAccessTime":1594802469000,"aliases":["t1"],"createTime":1594802469000,"qualifiedName":*"default.t2@cm"*

**But the entities are updated in Atlas as expected.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (ATLAS-3917) While deleting parent tag, shows incorrect message.

2020-08-11 Thread Durga Kadam (Jira)

Durga Kadam created ATLAS-3917:
--

 Summary: While deleting parent tag, shows incorrect message.
 Key: ATLAS-3917
 URL: https://issues.apache.org/jira/browse/ATLAS-3917
 Project: Atlas
  Issue Type: Bug
Reporter: Durga Kadam
 Attachments: tag_with_{0}_reproducible.mkv

Actual Behaviour - While deleting parent tag if the tag is associated with any 
entity shows this message :: "Failed to delete classification 'parent_abc'. 
Given type \{0} has reference."

Expected Result - There should be value instead of \{0}

Steps to generate
 # Create tag
 # Associate the created tag with any of the entity
 # Try to delete the tag
 # Shows validation message with some encrypted value

 

PFA Evidence file



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (ATLAS-3916) Get metrics according to the user permissions

2020-08-11 Thread Yue Dong (Jira)

Yue Dong created ATLAS-3916:
---

 Summary: Get metrics according to the user permissions
 Key: ATLAS-3916
 URL: https://issues.apache.org/jira/browse/ATLAS-3916
 Project: Atlas
  Issue Type: New Feature
  Components:  atlas-core
Affects Versions: 2.1.0, 2.0.0
Reporter: Yue Dong
 Attachments: Captura-de-pantalla-de-2020-08-11-10-20-06.png

I have two user groups: admin who has access to all tables and reader can only 
see public data and module A tables. So I have configured Atlas to use a simple 
authorizer with a little variation, which is to hide entities that are not 
accessible to the user.

The searches and displaying results work properly.

The only problem I find is that the metrics. In the elements of search by type, 
it indicates the number of all the entities of each type in the system. And 
this is not consistent with the search result of a reader user. 

!Captura-de-pantalla-de-2020-08-11-10-20-06.png!

 

I have verified that these numbers come from the getMetrics method, which is 
not secured so it does not obtain the numbers according to the users' 
configuration. Am I missing something? Is there any way to change these numbers?

Maybe it'd be nice to have something that allows to modify the querys of the 
metrics based on security and authorization, like 
AtlasAuthorizer.scrubSearchResults in search methods.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 72729: ATLAS-3910: Use KafkaZkClient instead of ZkUtils

2020-08-11 Thread Viktor Somogyi-Vass


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72729/
---

(Updated Aug. 11, 2020, 7:42 a.m.)


Review request for atlas and Madhan Neethiraj.


Repository: atlas


Description
---

ATLAS-3910: Use KafkaZkClient instead of ZkUtils


Diffs (updated)
-

  
addons/kafka-bridge/src/main/java/org/apache/atlas/kafka/bridge/KafkaBridge.java
 40b1fee71e57175cd7ddf8a3696ef147282295c2 
  
addons/kafka-bridge/src/test/java/org/apache/atlas/kafka/bridge/KafkaBridgeTest.java
 c8cc85cda1f62b2d1a0a49c61abf06d9beb6fdb7 
  notification/src/main/java/org/apache/atlas/hook/AtlasTopicCreator.java 
c69574103e360fcfb7dfb525bb06043a1fd647c3 
  notification/src/test/java/org/apache/atlas/hook/AtlasTopicCreatorTest.java 
293784770502299875177bd4a4a9c4b32c6b5a9a 


Diff: https://reviews.apache.org/r/72729/diff/2/

Changes: https://reviews.apache.org/r/72729/diff/1-2/


Testing
---


Thanks,

Viktor Somogyi-Vass

Re: Review Request 72698: ATLAS-3875: Introduce sample project for AtlasClient

2020-08-11 Thread Sarath Subramanian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72698/#review221539
---



The sample-app fails to run with the following error:

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.0.0:java 
(default-cli) on project sample-app: An exception occured while executing the 
Java class. Metadata service API org.apache.atlas.AtlasClientV2$API_V2@2eff472d 
failed with status 409 (Conflict) Response Body 
({"errorCode":"ATLAS-409-00-002","errorMessage":"Given type sample_Table_DB has 
references"}) -> [Help 1]

can you review.


atlas-examples/sample-app/src/main/java/org/apache/atlas/examples/sampleapp/SampleApp.java
Lines 123 (patched)


the password here is clear text.

public static String[] getBasicAuthenticationInput() {
String username = null;
String password = null;

try {
Console console = System.console();
if (console == null) {
System.err.println("Couldn't get a console object for user 
input");
System.exit(1);
}

username = console.readLine("Enter username for atlas :- ");

char[] pwdChar = console.readPassword("Enter password for atlas 
:- ");
if(pwdChar != null) {
password = new String(pwdChar);
}

} catch (Exception e) {
System.out.print("Error while reading user input");
System.exit(1);
}
return new String[]{username, password};
}


- Sarath Subramanian


On Aug. 4, 2020, 9:27 a.m., Jyoti Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72698/
> ---
> 
> (Updated Aug. 4, 2020, 9:27 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Sarath 
> Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3875
> https://issues.apache.org/jira/browse/ATLAS-3875
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Using this project users can get an idea as how to integrate with Atlas using 
> AtlasCleint. This helps the user to understand the basic rest functionality 
> of Atlas such as
> 
> - EntityRest
> - TypeDefRest
> - DiscoveryRest
> - LineageRest
> - GlossaryRest
> 
> 
> Diffs
> -
> 
>   atlas-examples/pom.xml PRE-CREATION 
>   atlas-examples/sample-app/README.md PRE-CREATION 
>   atlas-examples/sample-app/pom.xml PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/java/org/apache/atlas/examples/sampleapp/DiscoveryExample.java
>  PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/java/org/apache/atlas/examples/sampleapp/EntityExample.java
>  PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/java/org/apache/atlas/examples/sampleapp/GlossaryExample.java
>  PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/java/org/apache/atlas/examples/sampleapp/LineageExample.java
>  PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/java/org/apache/atlas/examples/sampleapp/SampleApp.java
>  PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/java/org/apache/atlas/examples/sampleapp/SampleAppConstants.java
>  PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/java/org/apache/atlas/examples/sampleapp/TypeDefExample.java
>  PRE-CREATION 
>   atlas-examples/sample-app/src/main/resources/atlas-application.properties 
> PRE-CREATION 
>   pom.xml 5e0442ae5 
> 
> 
> Diff: https://reviews.apache.org/r/72698/diff/7/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Jyoti Singh
> 
>

[jira] [Updated] (ATLAS-3915) impala_process startTime and endTime epoch is being set to 0

2020-08-11 Thread chaitali borole (Jira)



 [ 
https://issues.apache.org/jira/browse/ATLAS-3915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chaitali borole updated ATLAS-3915:
---
Description: 
For the queries being run via impala-shell the startTime and endTime epoch of 
impala_process is being set to 0 instead of a valid epoch. 

Commands to reproduce :
create table t3 (name string)
create table t4 as select * from t3

insert into t4 select * from t3

  was:For the queries being run via impala-shell the startTime and endTime 
epoch of impala_process is being set to 0 instead of a valid epoch. 
!atlasattach.png!


> impala_process startTime and endTime epoch is being set to 0
> 
>
> Key: ATLAS-3915
> URL: https://issues.apache.org/jira/browse/ATLAS-3915
> Project: Atlas
>  Issue Type: Improvement
>  Components: impala-integration
>Affects Versions: 2.1.0
>Reporter: chaitali borole
>Assignee: chaitali borole
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: atlasattach.png
>
>
> For the queries being run via impala-shell the startTime and endTime epoch of 
> impala_process is being set to 0 instead of a valid epoch. 
> Commands to reproduce :
> create table t3 (name string)
> create table t4 as select * from t3
> insert into t4 select * from t3



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3915) impala_process startTime and endTime epoch is being set to 0

2020-08-11 Thread chaitali borole (Jira)



 [ 
https://issues.apache.org/jira/browse/ATLAS-3915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chaitali borole updated ATLAS-3915:
---
Attachment: atlasattach.png

> impala_process startTime and endTime epoch is being set to 0
> 
>
> Key: ATLAS-3915
> URL: https://issues.apache.org/jira/browse/ATLAS-3915
> Project: Atlas
>  Issue Type: Improvement
>  Components: impala-integration
>Affects Versions: 2.1.0
>Reporter: chaitali borole
>Assignee: chaitali borole
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: atlasattach.png
>
>
> This patch fix following issue : impala process enitity start time and end 
> time was set 0 in the backend due to which the UI wasn't showing up timings.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3915) impala_process startTime and endTime epoch is being set to 0

2020-08-11 Thread chaitali borole (Jira)



 [ 
https://issues.apache.org/jira/browse/ATLAS-3915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

chaitali borole updated ATLAS-3915:
---
Description: For the queries being run via impala-shell the startTime and 
endTime epoch of impala_process is being set to 0 instead of a valid epoch. 
!atlasattach.png!  (was: This patch fix following issue : impala process 
enitity start time and end time was set 0 in the backend due to which the UI 
wasn't showing up timings.)

> impala_process startTime and endTime epoch is being set to 0
> 
>
> Key: ATLAS-3915
> URL: https://issues.apache.org/jira/browse/ATLAS-3915
> Project: Atlas
>  Issue Type: Improvement
>  Components: impala-integration
>Affects Versions: 2.1.0
>Reporter: chaitali borole
>Assignee: chaitali borole
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: atlasattach.png
>
>
> For the queries being run via impala-shell the startTime and endTime epoch of 
> impala_process is being set to 0 instead of a valid epoch. !atlasattach.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 72756: ATLAS-3918 : Regression : Renaming table has exceptions in Atlas application logs

[jira] [Commented] (ATLAS-3884) Unable to access Apache Atlas UI http://localhost:21000

Re: Review Request 72730: ATLAS-3875: Introduce AtlasClient in Python

[jira] [Commented] (ATLAS-3916) Get metrics according to the user permissions

Re: Review Request 72477: ATLAS-3583 Use Audit framework to generate audit entries for TypeDefs CREATE, UPDATE and DELETE

Re: Review Request 72756: ATLAS-3918 : Regression : Renaming table has exceptions in Atlas application logs

Re: Review Request 72756: ATLAS-3918 : Regression : Renaming table has exceptions in Atlas application logs

[jira] [Updated] (ATLAS-3917) While deleting parent tag, shows incorrect message.

[jira] [Created] (ATLAS-3918) Regression : Renaming table has exceptions in Atlas application logs

[jira] [Created] (ATLAS-3917) While deleting parent tag, shows incorrect message.

[jira] [Created] (ATLAS-3916) Get metrics according to the user permissions

Re: Review Request 72729: ATLAS-3910: Use KafkaZkClient instead of ZkUtils

Re: Review Request 72698: ATLAS-3875: Introduce sample project for AtlasClient

[jira] [Updated] (ATLAS-3915) impala_process startTime and endTime epoch is being set to 0

[jira] [Updated] (ATLAS-3915) impala_process startTime and endTime epoch is being set to 0

[jira] [Updated] (ATLAS-3915) impala_process startTime and endTime epoch is being set to 0

16 matches

Site Navigation

Mail list logo

Footer information