Re: Review Request 73140: ATLAS-4099: Removed slash from adls_gen2_directory

2021-01-12 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73140/
---

(Updated Jan. 13, 2021, 12:44 a.m.)


Review request for atlas, Ashutosh Mestry, Deep Singh, Madhan Neethiraj, 
Radhika Kundam, and Sarath Subramanian.


Bugs: ATLAS-4099
https://issues.apache.org/jira/browse/ATLAS-4099


Repository: atlas


Description
---

ATLAS-4099: Removed slash from adls_gen2_directory


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/utils/AtlasPathExtractorUtil.java 
0e4b507ff 
  common/src/test/java/org/apache/atlas/utils/AtlasPathExtractorUtilTest.java 
2d2ff2c47 


Diff: https://reviews.apache.org/r/73140/diff/2/

Changes: https://reviews.apache.org/r/73140/diff/1-2/


Testing
---

Already tested manually and precommit will be run


Thanks,

Sidharth Mishra



Review Request 73140: ATLAS-4099: Removed slash from adls_gen2_directory

2021-01-12 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73140/
---

Review request for atlas, Ashutosh Mestry, Deep Singh, Madhan Neethiraj, 
Radhika Kundam, and Sarath Subramanian.


Bugs: ATLAS-4099
https://issues.apache.org/jira/browse/ATLAS-4099


Repository: atlas


Description
---

ATLAS-4099: Removed slash from adls_gen2_directory


Diffs
-

  common/src/main/java/org/apache/atlas/utils/AtlasPathExtractorUtil.java 
0e4b507ff 


Diff: https://reviews.apache.org/r/73140/diff/1/


Testing
---

Already tested manually and precommit will be run


Thanks,

Sidharth Mishra



Review Request 73093: ATLAS-4083: Change the existing ADLS Gen2 type with new attributes

2020-12-15 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73093/
---

Review request for atlas, Ashutosh Mestry, Deep Singh, Madhan Neethiraj, 
Radhika Kundam, and Sarath Subramanian.


Bugs: ATLAS-4083
https://issues.apache.org/jira/browse/ATLAS-4083


Repository: atlas


Description
---

ATLAS-4083: Change the existing ADLS Gen2 type with new attributes


Diffs
-

  addons/models/3000-Cloud/3040-azure_adls_typedefs.json 2d44f4e07 


Diff: https://reviews.apache.org/r/73093/diff/1/


Testing
---


Thanks,

Sidharth Mishra



Review Request 73091: ATLAS-4082: Add more attributes like create or last modify time, contentLength etc. to ADLS Gen2 types

2020-12-15 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73091/
---

Review request for atlas, Ashutosh Mestry, Deep Singh, Madhan Neethiraj, 
Radhika Kundam, and Sarath Subramanian.


Bugs: ATLAS-4082
https://issues.apache.org/jira/browse/ATLAS-4082


Repository: atlas


Description
---

ATLAS-4082: Add more attributes like create or last modify time, contentLength 
etc. to ADLS Gen2 types


Diffs
-

  addons/models/3000-Cloud/patches/005-azure_adls_add_attributes.json 
PRE-CREATION 


Diff: https://reviews.apache.org/r/73091/diff/1/


Testing
---

Applied the patch to my cluster successfully


Thanks,

Sidharth Mishra



Re: Review Request 73089: ATLAS-408: Atlas incorrectly reports successful change to glossary term

2020-12-15 Thread Sidharth Mishra


> On Dec. 15, 2020, 6:54 p.m., Deep Singh wrote:
> > repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
> > Line 416 (original), 415 (patched)
> > <https://reviews.apache.org/r/73089/diff/1/?file=2243547#file2243547line416>
> >
> > What Exception is thrown here? Does it throw unauthorized error. And in 
> > such cases, does the API sends 401 response ?

{"errorCode":"ATLAS-403-00-001","errorMessage":"sid is not authorized to 
perform update entity: type=AtlasGlossaryTerm"} is been thrown and Status Code: 
403 Forbidden


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73089/#review222345
-------


On Dec. 15, 2020, 5:45 p.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73089/
> ---
> 
> (Updated Dec. 15, 2020, 5:45 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Deep Singh, Madhan Neethiraj, 
> Radhika Kundam, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-4081
> https://issues.apache.org/jira/browse/ATLAS-4081
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-4081: Atlas incorrectly reports successful change to glossary term
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
> d630f669e 
> 
> 
> Diff: https://reviews.apache.org/r/73089/diff/1/
> 
> 
> Testing
> ---
> 
> Created one more user - sid which is not an admin user and reproduced the 
> issue. Then patched the cluster with the fix and confirmed it works. Now the 
> behaviour is same as GlossaryService.
> 
> 
> Thanks,
> 
> Sidharth Mishra
> 
>



Review Request 73089: ATLAS-408: Atlas incorrectly reports successful change to glossary term

2020-12-15 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73089/
---

Review request for atlas, Ashutosh Mestry, Deep Singh, Madhan Neethiraj, 
Radhika Kundam, and Sarath Subramanian.


Bugs: ATLAS-4081
https://issues.apache.org/jira/browse/ATLAS-4081


Repository: atlas


Description
---

ATLAS-4081: Atlas incorrectly reports successful change to glossary term


Diffs
-

  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
d630f669e 


Diff: https://reviews.apache.org/r/73089/diff/1/


Testing
---

Created one more user - sid which is not an admin user and reproduced the 
issue. Then patched the cluster with the fix and confirmed it works. Now the 
behaviour is same as GlossaryService.


Thanks,

Sidharth Mishra



Re: Review Request 72730: ATLAS-3875: Introduce AtlasClient in Python

2020-08-11 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72730/#review221546
---




atlas-examples/sample-app/src/main/python/discovery_example.py
Lines 3 (patched)
<https://reviews.apache.org/r/72730/#comment310609>

Remove two extra blank lines. Along with this please add content to 
README.md file as well



atlas-examples/sample-app/src/main/python/discovery_example.py
Lines 12 (patched)
<https://reviews.apache.org/r/72730/#comment310601>

should we rename it to something more meaningful?



atlas-examples/sample-app/src/main/python/utils.py
Lines 11 (patched)
<https://reviews.apache.org/r/72730/#comment310600>

I think it will be good to rename these const to have some prefix like 
CUSTOM or SAMPLE to differentiate between the existing table types.



client/client-v2/src/main/python/atlas_client/base_client.py
Lines 84 (patched)
<https://reviews.apache.org/r/72730/#comment310602>

Should we have custom exceptions like AtlasException instead of Exception 
thrown from most of the python client similar to Java client. This will help 
the users of the client to handle appropriately and distinguish from the 
generic python exceptions



client/client-v2/src/main/python/atlas_client/discovery_client.py
Lines 6 (patched)
<https://reviews.apache.org/r/72730/#comment310608>

Remove one extra blank line



client/client-v2/src/main/python/atlas_client/discovery_client.py
Lines 20 (patched)
<https://reviews.apache.org/r/72730/#comment310603>

Can we have vertical indentation like:

DSL_SEARCH   = API(DSL_SEARCH_URI, HttpMethod.GET, HTTPStatus.OK)
FULL_TEXT_SEARCH = API(FULL_TEXT_SEARCH_URI, HttpMethod.GET, 
HTTPStatus.OK)
BASIC_SEARCH = API(BASIC_SEARCH_URI, HttpMethod.GET, HTTPStatus.OK)
FACETED_SEARCH   = API(FACETED_SEARCH_URI, HttpMethod.POST, 
HTTPStatus.OK)
ATTRIBUTE_SEARCH = API(DISCOVERY_URI + "/attribute", HttpMethod.GET, 
HTTPStatus.OK)

In case you are changing please take care of this at other places



client/client-v2/src/main/python/atlas_client/discovery_model.py
Lines 18 (patched)
<https://reviews.apache.org/r/72730/#comment310606>

Add blank line after the header comment. Please do at other files as well



client/client-v2/src/main/python/atlas_client/discovery_model.py
Lines 22 (patched)
<https://reviews.apache.org/r/72730/#comment310604>

Please remove one extra blank line (we always keep one blank line as 
separator if needed)



client/client-v2/src/main/python/atlas_client/discovery_model.py
Lines 145 (patched)
<https://reviews.apache.org/r/72730/#comment310607>

Please remove if not needed


- Sidharth Mishra


On Aug. 10, 2020, 5:26 p.m., Jyoti Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72730/
> ---
> 
> (Updated Aug. 10, 2020, 5:26 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Sarath 
> Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3875
> https://issues.apache.org/jira/browse/ATLAS-3875
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> - Added basic structure for Python client
> - This includes all classes for Entity and TypeDef
> - Added all endpoints for typedef
> 
> 
> Diffs
> -
> 
>   atlas-examples/sample-app/src/main/python/discovery_example.py PRE-CREATION 
>   atlas-examples/sample-app/src/main/python/entity_example.py PRE-CREATION 
>   atlas-examples/sample-app/src/main/python/glossary_example.py PRE-CREATION 
>   atlas-examples/sample-app/src/main/python/lineage_example.py PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/python/request_json/entity_create_db.json 
> PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/python/request_json/entity_create_process.json
>  PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/python/request_json/entity_create_table_canada.json
>  PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/python/request_json/entity_create_table_us.json
>  PRE-CREATION 
>   atlas-examples/sample-app/src/main/python/request_json/typedef_create.json 
> PRE-CREATION 
>   atlas-examples/sample-app/src/main/python/sample_client.py PRE-CREATION 
>   atlas-examples/sample-app/src/main/python/typedef_example.py PRE-CREATION 
>   atlas-examples/sample-app/src/main/python/utils.py PRE-CREATION 
>   client/client-v2/src/main/python/LICENSE PRE-CREATION 
>   client/client-v2/src/main/python/README.md PRE-CREATION 
>   client/client-v2/src/main/pyt

Re: Review Request 72698: ATLAS-3875: Introduce sample project for AtlasClient

2020-07-24 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72698/#review221369
---


Ship it!




Ship It!

- Sidharth Mishra


On July 23, 2020, 6:10 p.m., Jyoti Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72698/
> ---
> 
> (Updated July 23, 2020, 6:10 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Sarath 
> Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3875
> https://issues.apache.org/jira/browse/ATLAS-3875
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Using this project users can get an idea as how to integrate with Atlas using 
> AtlasCleint. This helps the user to understand the basic rest functionality 
> of Atlas such as
> 
> - EntityRest
> - TypeDefRest
> - DiscoveryRest
> - LineageRest
> - GlossaryRest
> 
> 
> Diffs
> -
> 
>   atlas-examples/pom.xml PRE-CREATION 
>   atlas-examples/sample-app/README.md PRE-CREATION 
>   atlas-examples/sample-app/pom.xml PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/java/org/apache/atlas/examples/sampleapp/DiscoveryExample.java
>  PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/java/org/apache/atlas/examples/sampleapp/EntityExample.java
>  PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/java/org/apache/atlas/examples/sampleapp/GlossaryExample.java
>  PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/java/org/apache/atlas/examples/sampleapp/LineageExample.java
>  PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/java/org/apache/atlas/examples/sampleapp/SampleApp.java
>  PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/java/org/apache/atlas/examples/sampleapp/SampleAppConstants.java
>  PRE-CREATION 
>   
> atlas-examples/sample-app/src/main/java/org/apache/atlas/examples/sampleapp/TypeDefExample.java
>  PRE-CREATION 
>   atlas-examples/sample-app/src/main/resources/atlas-application.properties 
> PRE-CREATION 
>   pom.xml 5e0442ae5 
> 
> 
> Diff: https://reviews.apache.org/r/72698/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Jyoti Singh
> 
>



Re: Review Request 72698: ATLAS-3875: Introduce sample project for AtlasClient

2020-07-22 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72698/#review221321
---




atlas-examples/src/main/java/org/apache/atlas/AtlasClientBaseExample.java
Lines 63 (patched)
<https://reviews.apache.org/r/72698/#comment310200>

char[] is always preferred over String - check 
https://stackoverflow.com/questions/8881291/why-is-char-preferred-over-string-for-passwords#:~:text=Since%20Strings%20are%20immutable%20there,2.

Please check if we can add a new contructor at client v2 and do this ow 
lets file a jira and track this for client v2



atlas-examples/src/main/java/org/apache/atlas/AtlasClientBaseExample.java
Lines 75 (patched)
<https://reviews.apache.org/r/72698/#comment310202>

Please check if we should change the code snippet from 72:84 to something 
like:

public static void main(String[] args) throws Exception {
String[] basicAuthUsernamePassword = null;
String[] urls = null;

urls = getServerUrl();
if (!AuthenticationUtil.isKerberosAuthenticationEnabled()) {
basicAuthUsernamePassword = getUserInput();
new AtlasClientBaseExample(urls, basicAuthUsernamePassword);
} else {
new AtlasClientBaseExample(urls);
}

try {
// TypeDef



atlas-examples/src/main/java/org/apache/atlas/AtlasClientBaseExample.java
Lines 83 (patched)
<https://reviews.apache.org/r/72698/#comment310201>

In case of kerberos urls will be null as we populate that if kerberos not 
enabled.



atlas-examples/src/main/java/org/apache/atlas/AtlasClientBaseExample.java
Lines 86 (patched)
<https://reviews.apache.org/r/72698/#comment310203>

Instead of commenting like typedef then entity examples etc. it would be 
good to move these to separate private functions. Please refer - 

FUNCTIONS SHOULD DO ONE THING. THEY SHOULD DO IT WELL. THEY SHOULD DO IT 
ONLY (more details - 
https://learning.oreilly.com/library/view/clean-code/9780136083238/chapter03.html#ch3)


https://softwareengineering.stackexchange.com/questions/137941/should-a-method-do-one-thing-and-be-good-at-it



atlas-examples/src/main/java/org/apache/atlas/AtlasClientBaseExample.java
Lines 140 (patched)
<https://reviews.apache.org/r/72698/#comment310199>

Please use Console.readPassword instead


- Sidharth Mishra


On July 21, 2020, 4:24 p.m., Jyoti Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72698/
> ---
> 
> (Updated July 21, 2020, 4:24 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Sarath 
> Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3875
> https://issues.apache.org/jira/browse/ATLAS-3875
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Using this project users can get an idea as how to integrate with Atlas using 
> AtlasCleint. This helps the user to understand the basic rest functionality 
> of Atlas such as
> 
> - EntityRest
> - TypeDefRest
> - DiscoveryRest
> - LineageRest
> - GlossaryRest
> 
> 
> Diffs
> -
> 
>   atlas-examples/pom.xml PRE-CREATION 
>   atlas-examples/src/main/java/org/apache/atlas/AtlasClientBaseExample.java 
> PRE-CREATION 
>   atlas-examples/src/main/java/org/apache/atlas/DiscoveryExample.java 
> PRE-CREATION 
>   atlas-examples/src/main/java/org/apache/atlas/EntityRestBaseExample.java 
> PRE-CREATION 
>   atlas-examples/src/main/java/org/apache/atlas/GlossaryExample.java 
> PRE-CREATION 
>   atlas-examples/src/main/java/org/apache/atlas/LineageExample.java 
> PRE-CREATION 
>   atlas-examples/src/main/java/org/apache/atlas/TypeDefExample.java 
> PRE-CREATION 
>   pom.xml 5e0442ae5 
> 
> 
> Diff: https://reviews.apache.org/r/72698/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Jyoti Singh
> 
>



Re: Review Request 72638: ATLAS-3875: Enhancing AtlasClient by adding missing API endpoints

2020-07-14 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72638/#review221215
---




webapp/src/test/java/org/apache/atlas/web/integration/BasicSearchIT.java
Lines 38 (patched)
<https://reviews.apache.org/r/72638/#comment310041>

Please remove unused imports.



webapp/src/test/java/org/apache/atlas/web/integration/BasicSearchIT.java
Lines 52 (patched)
<https://reviews.apache.org/r/72638/#comment310042>

Instead of import * use below:

import static org.testng.Assert.assertNotNull;
import static org.testng.Assert.fail;
import static org.testng.Assert.assertNull;
import static org.testng.Assert.assertEquals;


- Sidharth Mishra


On July 10, 2020, 12:05 a.m., Jyoti Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72638/
> ---
> 
> (Updated July 10, 2020, 12:05 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Sarath 
> Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3875
> https://issues.apache.org/jira/browse/ATLAS-3875
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> There are many missing API endpoints in AtlasClientV2. This solution is 
> adding functions corresponding to those missing APIs for the following Rest 
> endpoints.
> 
> 1. TypeRest
> 2. EntityRest
> 3. LineageRest
> 4. DiscoveryRest
> 5. GlossaryRest
> 6. RelationshipRest
> 
> This will enable users to test and integrate with APIs more effectively via 
> AtlasCient.
> 
> 
> Diffs
> -
> 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 7b6d1d0f3 
>   client/common/src/main/java/org/apache/atlas/AtlasBaseClient.java e3d2ebc34 
>   webapp/pom.xml 3c55b4dd4 
>   webapp/src/test/java/org/apache/atlas/web/TestUtils.java e22a1c10d 
>   webapp/src/test/java/org/apache/atlas/web/integration/BaseResourceIT.java 
> 680028892 
>   webapp/src/test/java/org/apache/atlas/web/integration/BasicSearchIT.java 
> 808f62354 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
>  cc883d615 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/GlossaryClientV2IT.java 
> PRE-CREATION 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/LineageClientV2IT.java 
> PRE-CREATION 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/TypedefsJerseyResourceIT.java
>  331ea2c47 
>   webapp/src/test/resources/json/search-parameters/attribute-filters.json 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/72638/diff/4/
> 
> 
> Testing
> ---
> 
> IT has been added for newly created functions in AtlasClientV2.
> Validated by running IT on local system.
> 
> 
> Thanks,
> 
> Jyoti Singh
> 
>



Re: Review Request 72638: ATLAS-3875: Enhancing AtlasClient by adding missing API endpoints

2020-07-14 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72638/#review221214
---




client/common/src/main/java/org/apache/atlas/AtlasBaseClient.java
Lines 84 (patched)
<https://reviews.apache.org/r/72638/#comment310038>

Please remove one extra space added before String



client/common/src/main/java/org/apache/atlas/AtlasBaseClient.java
Lines 107 (patched)
<https://reviews.apache.org/r/72638/#comment310039>

We don't have any v2 API at AtlasBaseClient. Could you please move this to 
AtlasClientV2.



client/common/src/main/java/org/apache/atlas/AtlasBaseClient.java
Lines 489 (patched)
<https://reviews.apache.org/r/72638/#comment310040>

Please move this as well to v2 as this is the base client.


- Sidharth Mishra


On July 10, 2020, 12:05 a.m., Jyoti Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72638/
> ---
> 
> (Updated July 10, 2020, 12:05 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Sarath 
> Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3875
> https://issues.apache.org/jira/browse/ATLAS-3875
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> There are many missing API endpoints in AtlasClientV2. This solution is 
> adding functions corresponding to those missing APIs for the following Rest 
> endpoints.
> 
> 1. TypeRest
> 2. EntityRest
> 3. LineageRest
> 4. DiscoveryRest
> 5. GlossaryRest
> 6. RelationshipRest
> 
> This will enable users to test and integrate with APIs more effectively via 
> AtlasCient.
> 
> 
> Diffs
> -
> 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 7b6d1d0f3 
>   client/common/src/main/java/org/apache/atlas/AtlasBaseClient.java e3d2ebc34 
>   webapp/pom.xml 3c55b4dd4 
>   webapp/src/test/java/org/apache/atlas/web/TestUtils.java e22a1c10d 
>   webapp/src/test/java/org/apache/atlas/web/integration/BaseResourceIT.java 
> 680028892 
>   webapp/src/test/java/org/apache/atlas/web/integration/BasicSearchIT.java 
> 808f62354 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
>  cc883d615 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/GlossaryClientV2IT.java 
> PRE-CREATION 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/LineageClientV2IT.java 
> PRE-CREATION 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/TypedefsJerseyResourceIT.java
>  331ea2c47 
>   webapp/src/test/resources/json/search-parameters/attribute-filters.json 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/72638/diff/4/
> 
> 
> Testing
> ---
> 
> IT has been added for newly created functions in AtlasClientV2.
> Validated by running IT on local system.
> 
> 
> Thanks,
> 
> Jyoti Singh
> 
>



Re: Review Request 72638: ATLAS-3875: Enhancing AtlasClient by adding missing API endpoints

2020-07-14 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72638/#review221213
---




client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java
Line 40 (original), 57 (patched)
<https://reviews.apache.org/r/72638/#comment310029>

Please remove this one as its already imported above



client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java
Lines 103 (patched)
<https://reviews.apache.org/r/72638/#comment310030>

Please maintain the above alignment.



client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java
Lines 112 (patched)
<https://reviews.apache.org/r/72638/#comment310032>

Align below lines like:

//Glossary APIs
private static final String GLOSSARY_URI= BASE_URI + 
"v2/glossary";
private static final String GLOSSARY_TERM   = GLOSSARY_URI + 
"/term";
private static final String GLOSSARY_TERMS  = GLOSSARY_URI + 
"/terms";
private static final String GLOSSARY_CATEGORY   = GLOSSARY_URI + 
"/category";
private static final String GLOSSARY_CATEGORIES = GLOSSARY_URI + 
"/categories";



client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java
Lines 118 (patched)
<https://reviews.apache.org/r/72638/#comment310031>

Please have one blank line instead of two



client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java
Lines 220 (patched)
<https://reviews.apache.org/r/72638/#comment310033>

Please rename getRelationShipDefByName -> getRelationshipDefByName (As 
RelationShip is a single word)



client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java
Lines 224 (patched)
<https://reviews.apache.org/r/72638/#comment310034>

getRelationShipDefByGuid -> getRelationshipDefByGuid



client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java
Lines 597 (patched)
<https://reviews.apache.org/r/72638/#comment310035>

Please remove these two extra blank lines



client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java
Lines 648 (patched)
<https://reviews.apache.org/r/72638/#comment310037>

Please remove the below lines as it seems its not bMultivaluedMap queryParams = new MultivaluedMapImpl();
   queryParams.add("name", name);



client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java
Lines 650 (patched)
<https://reviews.apache.org/r/72638/#comment310036>

Please remove the commented line


- Sidharth Mishra


On July 10, 2020, 12:05 a.m., Jyoti Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72638/
> ---
> 
> (Updated July 10, 2020, 12:05 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Sarath 
> Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3875
> https://issues.apache.org/jira/browse/ATLAS-3875
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> There are many missing API endpoints in AtlasClientV2. This solution is 
> adding functions corresponding to those missing APIs for the following Rest 
> endpoints.
> 
> 1. TypeRest
> 2. EntityRest
> 3. LineageRest
> 4. DiscoveryRest
> 5. GlossaryRest
> 6. RelationshipRest
> 
> This will enable users to test and integrate with APIs more effectively via 
> AtlasCient.
> 
> 
> Diffs
> -
> 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 7b6d1d0f3 
>   client/common/src/main/java/org/apache/atlas/AtlasBaseClient.java e3d2ebc34 
>   webapp/pom.xml 3c55b4dd4 
>   webapp/src/test/java/org/apache/atlas/web/TestUtils.java e22a1c10d 
>   webapp/src/test/java/org/apache/atlas/web/integration/BaseResourceIT.java 
> 680028892 
>   webapp/src/test/java/org/apache/atlas/web/integration/BasicSearchIT.java 
> 808f62354 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
>  cc883d615 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/GlossaryClientV2IT.java 
> PRE-CREATION 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/LineageClientV2IT.java 
> PRE-CREATION 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/TypedefsJerseyResourceIT.java
>  331ea2c47 
>   webapp/src/test/resources/json/search-parameters/attribute-filters.json 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/72638/diff/4/
> 
> 
> Testing
> ---
> 
> IT has been added for newly created functions in AtlasClientV2.
> Validated by running IT on local system.
> 
> 
> Thanks,
> 
> Jyoti Singh
> 
>



Re: [VOTE] Release Apache Atlas version 2.1.0 - rc3

2020-07-14 Thread sidharth mishra
+1 for Apache Atlas 2.1 rc3

Validated the following:

- Verified the MD5/sha512 hash
- Verified PGP signature
- No binaries in extracted source
- Verified the build with embedded-hbase and solr
- Ran quick start and tested some sanity test cases - Basic/advanced
search, Classification, Glossary, tag propagation, Admin purge and audit,
Lineage, Propagated from in case of purged entity

On Tue, Jul 14, 2020 at 4:24 AM Keval Bhatt  wrote:

> +1 (binding) for Apache Atlas 2.1.0 RC3
>
> Validated the following:
>
>- Verified the md5/sha checksum of the source bundle
>- Verified the build with -Pdist,embedded-hbase-solr
>- Created classification, updated the attributes and associated with the
>entity and term
>- Tested basic search with the type, classification, term and their
>attribute filters
>- Tested advance search
>- Saved the search for later use.
>- Created Business metadata and associated with an entity.
>- Create Labels and user-define properties
>- Tested Lineage and applied filter hide/show deleted entities and
>process
>- Checked the relationship diagram
>- Entity create/update test from UI
>- Checked the statistics page.
>- checked the loader hide change for term association.
>
>
> On Tue, Jul 14, 2020 at 12:01 PM vishal suvagia
>  wrote:
>
> > +1 for Apache Atlas 2.1.0 RC#3.
> > Verified clean compilation and rat-checks.Validated the source checksums
> > and hash.Verified quick-start data and UI to be working
> >
> >
> > RegardsVishal Suvagia.
> >On Monday, 13 July, 2020, 9:20:54 pm IST, Sharmadha Sainath
> >  wrote:
> >
> >  +1 for Apache Atlas 2.1.0-rc3 release.
> >
> >   - Validated MD5 and SHA512 checksum of source.
> >   - Verified PGP Signature
> >   - Brought up Atlas using embedded HBase and Solr.
> >   - Verified DSL , Basic Search , Quick search and Suggestions
> >   - Validated CRUD on labels, custom attributes , business metadata
> >   - Verified Basic search using system attributes filters (multiple
> >   classifications , labels , custom attributes , business metadata
> search)
> >   - Added labels , user defined properties (custom attributes) , business
> >   metadata to entities and verified the entity properties and audits.
> >   - Purged Soft Deleted entity  and verified the audits of purged entity.
> >   Deleted tag associated to the purged entity and verified there are no
> >   references.
> >   - Saved the Favorite searches and executed them.
> >   - Tested Atlas UI against XSS strings
> >
> > Thanks,
> > Sharmadha Sainath
> >
> > On Mon, Jul 13, 2020 at 7:15 PM Nixon Rodrigues <
> > nixon.rodrig...@freestoneinfotech.com> wrote:
> >
> > > +1 for Atlas 2.1.0 rc3 release.
> > >
> > > Verified signature and MD5/SHA keys.
> > > Verified release tag on git repo.
> > > Verified build with rc3 bits.
> > > Verified UI with Quick Start data.
> > >
> > > Thanks  & Regards
> > > Nixon
> > >
> > >
> > >
> > > On Mon, Jul 13, 2020 at 12:44 PM Mandar Ambawane <
> > > mandar.ambaw...@freestoneinfotech.com> wrote:
> > >
> > > > +1 for Apache Atlas 2.1.0 RC3 release
> > > >
> > > > Validated the following:
> > > > 1. Verified the source SHA512 and MD5 checksum
> > > > 2. Verified the build with -Pdist,embedded-hbase-solr
> > > > 3. Started Atlas server successfully
> > > > 4. Verified quick_start.py, all entities and tag get created
> > successfully
> > > > 5. Created Business metadata and associated with an entity.
> > > > 6. Checked Basic and advanced search
> > > >
> > > > Thanks,
> > > > Mandar
> > > >
> > > > On Mon, Jul 13, 2020 at 12:39 PM Jayendra Parab <
> > > jayendra.pa...@gmail.com>
> > > > wrote:
> > > >
> > > > > +1 for Apache Atlas 2.1.0 RC3 release
> > > > >
> > > > > Validated the following:
> > > > > 1. Verified the source SHA512 and MD5 checksum
> > > > > 2. Verified the build with -Pdist,embedded-hbase-solr
> > > > > 3. Started Atlas server successfully
> > > > > 4. Verified quick_start.py, all entities and tag get created
> > > successfully
> > > > > 5. Created entities, applied tags from UI
> > > > > 6. Checked Basic and advanced search
> > > > >
> > > > > Regards,
> > > > > Jayendra.
> > > > >
> > > > > On Mon, Jul 13, 2020 at 12:17 PM Sarath Subramanian <
> > sar...@apache.org
> > > >
> > > > > wrote:
> > > > >
> > > > > > +1 for Apache Atlas 2.1 rc3
> > > > > >
> > > > > > Validated the following:
> > > > > > - Validated MD5 checksum
> > > > > > - Validated sha512 hash
> > > > > > - Verified PGP signature
> > > > > > - No binaries in extracted source
> > > > > > - Built successfully using embedded-hbase and solr profile
> > > > > > - Ran quick start and tested basic sanity - saved search,
> > > > basic/advanced
> > > > > > search, Glossary, tag propagation, Admin purge, system attribute
> > > search
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Jul 9, 2020 at 9:59 PM Madhan Neethiraj <
> mad...@apache.org
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Atlas team,
> > > > > > >
> > > > > > 

Re: Review Request 72658: ATLAS-3877: Audit API returns the audit information for an unauthorised entity

2020-07-08 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72658/
---

(Updated July 8, 2020, 7:31 a.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3877
https://issues.apache.org/jira/browse/ATLAS-3877


Repository: atlas


Description
---

ATLAS-3877: Audit Fixed the authorization for audit both in case of non purged 
entity to populate classifcation and purged entity by getting the header from 
HBase Audit


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 717310daf 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
 bf1629cb3 
  webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 88de8b679 


Diff: https://reviews.apache.org/r/72658/diff/2/

Changes: https://reviews.apache.org/r/72658/diff/1-2/


Testing
---

#Deleted Entity
//Allow
curl --location --request GET 
'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/196bf3a9-07c6-4152-82ad-10176c5df5b9/audit?count=1=ENTITY_DELETE'
 \
-u admin:admin123

Response:
[{"entityId":"196bf3a9-07c6-4152-82ad-10176c5df5b9","timestamp":1594192518488,"user":"admin","action":"ENTITY_DELETE","details":"Deleted
 
entity","eventKey":"196bf3a9-07c6-4152-82ad-10176c5df5b9:1594192518488:1:1594192519277","entity":null,"type":null}]

//Deny
curl --location --request GET 
'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/196bf3a9-07c6-4152-82ad-10176c5df5b9/audit?count=1=ENTITY_DELETE'
 \
-u sid:sid

Response:
{"errorCode":"ATLAS-403-00-001","errorMessage":"sid is not authorized to 
perform read entity: guid=196bf3a9-07c6-4152-82ad-10176c5df5b9"}


#Purged Entity
//Allow
curl --location --request GET 
'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/def375bd-9bad-42f3-9fa7-9edc1c7d8d96/audit?count=1=ENTITY_PURGE'
 \
-u admin:admin123

Response:
[{"entityId":"def375bd-9bad-42f3-9fa7-9edc1c7d8d96","timestamp":1594191071761,"user":"admin","action":"ENTITY_PURGE","details":"Purged:
 
{\"typeName\":\"hive_table\",\"attributes\":{\"owner\":\"admin\",\"createTime\":1594177077000,\"qualifiedName\":\"default.test_july_7_2020_ctas_10@cm\",\"name\":\"test_july_7_2020_ctas_10\"},\"guid\":\"def375bd-9bad-42f3-9fa7-9edc1c7d8d96\",\"isIncomplete\":false,\"provenanceType\":0,\"status\":\"DELETED\",\"version\":0,\"classifications\":[{\"typeName\":\"NOT_IMP\",\"entityGuid\":\"5f52ee87-85c8-46ac-ad38-e0e5be935bb2\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"PARENT\",\"entityGuid\":\"73cc5730-494b-4313-8747-9ccf8a1b0227\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false}],\"meanings\":[],\"proxy\":false}","eventKey":"def375bd-9bad-42f3-9fa7-9edc1c7d8d96:1594191071761:3:1594191103634","entity":null
 ,"type":null}]

//Deny
curl --location --request GET 
'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/def375bd-9bad-42f3-9fa7-9edc1c7d8d96/audit?count=1=ENTITY_PURGE'
 \
-u sid:sid

Response:
{"errorCode":"ATLAS-403-00-001","errorMessage":"sid is not authorized to 
perform read entity audit: guid=def375bd-9bad-42f3-9fa7-9edc1c7d8d96"}


Thanks,

Sidharth Mishra



Review Request 72658: ATLAS-3877: Audit API returns the audit information for an unauthorised entity

2020-07-08 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72658/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3877
https://issues.apache.org/jira/browse/ATLAS-3877


Repository: atlas


Description
---

ATLAS-3877: Audit Fixed the authorization for audit both in case of non purged 
entity to populate classifcation and purged entity by getting the header from 
HBase Audit


Diffs
-

  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 717310daf 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
 bf1629cb3 
  webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 88de8b679 


Diff: https://reviews.apache.org/r/72658/diff/1/


Testing
---

#Deleted Entity
//Allow
curl --location --request GET 
'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/196bf3a9-07c6-4152-82ad-10176c5df5b9/audit?count=1=ENTITY_DELETE'
 \
-u admin:admin123

Response:
[{"entityId":"196bf3a9-07c6-4152-82ad-10176c5df5b9","timestamp":1594192518488,"user":"admin","action":"ENTITY_DELETE","details":"Deleted
 
entity","eventKey":"196bf3a9-07c6-4152-82ad-10176c5df5b9:1594192518488:1:1594192519277","entity":null,"type":null}]

//Deny
curl --location --request GET 
'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/196bf3a9-07c6-4152-82ad-10176c5df5b9/audit?count=1=ENTITY_DELETE'
 \
-u sid:sid

Response:
{"errorCode":"ATLAS-403-00-001","errorMessage":"sid is not authorized to 
perform read entity: guid=196bf3a9-07c6-4152-82ad-10176c5df5b9"}


#Purged Entity
//Allow
curl --location --request GET 
'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/def375bd-9bad-42f3-9fa7-9edc1c7d8d96/audit?count=1=ENTITY_PURGE'
 \
-u admin:admin123

Response:
[{"entityId":"def375bd-9bad-42f3-9fa7-9edc1c7d8d96","timestamp":1594191071761,"user":"admin","action":"ENTITY_PURGE","details":"Purged:
 
{\"typeName\":\"hive_table\",\"attributes\":{\"owner\":\"admin\",\"createTime\":1594177077000,\"qualifiedName\":\"default.test_july_7_2020_ctas_10@cm\",\"name\":\"test_july_7_2020_ctas_10\"},\"guid\":\"def375bd-9bad-42f3-9fa7-9edc1c7d8d96\",\"isIncomplete\":false,\"provenanceType\":0,\"status\":\"DELETED\",\"version\":0,\"classifications\":[{\"typeName\":\"NOT_IMP\",\"entityGuid\":\"5f52ee87-85c8-46ac-ad38-e0e5be935bb2\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false},{\"typeName\":\"PARENT\",\"entityGuid\":\"73cc5730-494b-4313-8747-9ccf8a1b0227\",\"entityStatus\":\"ACTIVE\",\"propagate\":true,\"validityPeriods\":[],\"removePropagationsOnEntityDelete\":false}],\"meanings\":[],\"proxy\":false}","eventKey":"def375bd-9bad-42f3-9fa7-9edc1c7d8d96:1594191071761:3:1594191103634","entity":null
 ,"type":null}]

//Deny
curl --location --request GET 
'http://sid-ycloud-atlas1-2.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/def375bd-9bad-42f3-9fa7-9edc1c7d8d96/audit?count=1=ENTITY_PURGE'
 \
-u sid:sid

Response:
{"errorCode":"ATLAS-403-00-001","errorMessage":"sid is not authorized to 
perform read entity audit: guid=def375bd-9bad-42f3-9fa7-9edc1c7d8d96"}


Thanks,

Sidharth Mishra



Re: [VOTE] Release Apache Atlas version 2.1.0 - rc1

2020-06-08 Thread sidharth mishra
+1 for Apache Atlas 2.1 RC1

Validated the following items.

   1. Verified the md5/sha512 checksum of source bundle
   2. Packaging with -Pdist,embedded-hbase-solr works fine
   3. Starting Atlas server works fine
   4. quick_start.py works fine and all the entities and tags get created
successfully at UI
   5. Tested basic and quick search flow
   6. Tested entity and tag delete
   7. Tested entities purge
   8. Checked admin audit for entity purge

Regards,
Sid

On Mon, Jun 8, 2020 at 11:41 AM Madhan Neethiraj  wrote:

> Bolke,
>
> Can you send specific security concerns that need to addressed in 2.1.0
> release? If this is about questions from Melinda Crane, I asked for Melinda
> details to better understand the questions. I suggest we know the details
> before treating them as release blockers.
>
> Thanks,
> Madhan
>
> On 6/8/20, 9:56 AM, "Bolke de Bruin"  wrote:
>
> This is my -1 (non-binding) which I will revert to +1 when the
> security concerns have been addressed/clarified.
>
> Given that Atlas has pretty slow releases I think it smart to address
> them now.
>
> Cheers
> Bolke
>
> Sent from my iPhone
>
> > On 8 Jun 2020, at 18:18, Sharmadha Sainath
>  wrote:
> >
> > +1 for Apache Atlas 2.1 RC1
> >
> >
> >   - Validated MD5 and SHA512 checksum of source.
> >   - Brought up Atlas using embedded HBase and Solr.
> >   - Verified DSL , Basic Search , Quick search and Suggestions
> >   - Verified Basic search using system attributes filters (multiple
> >   classification search , labels , custom attributes , business
> metadata
> >   search)
> >   - Added labels , user defined properties (custom attributes) ,
> business
> >   metadata to entities and verified the entity properties and audits.
> >   - Purged Soft Deleted entity and verified purge audits.
> >   - Saved the Favorite searches and executed them.
> >
> > Thanks,
> > Sharmadha S.
> >
> >
> >
> >> On Mon, Jun 8, 2020 at 6:23 PM Nixon Rodrigues <
> >> nixon.rodrig...@freestoneinfotech.com> wrote:
> >>
> >> Thanks Madhan for putting 2.1 RC1 together for the vote.
> >>
> >> +1 for Apache Atlas 2.1 RC1
> >>
> >> Validated the following items.
> >>
> >>   1. verified the md5/sha checksum of source bundle.
> >>   2. packaging with -Pdist,embedded-hbase-solr is good.
> >>   3. Unit testcase passes.
> >>   4. Run quickstart and all the types and entities created are
> loading
> >>   correctly on the UI.
> >>   5. Create tags, glossary, term and its apply correctly on
> entities.
> >>   6. Created business metadata with attributes and applied it to
> entities.
> >>   7. tested basic and quick search flow.
> >>   8. Verified statistics page details
> >>
> >>
> >> Thanks & Regards
> >> Nixon
> >>
> >>
> >>
> >> On Mon, Jun 8, 2020 at 11:17 AM Sarath Subramanian <
> sar...@apache.org>
> >> wrote:
> >>
> >>> +1 for Apache Atlas 2.1 RC1
> >>>
> >>> Validated the following:
> >>>
> >>>   - No binaries in source
> >>>   - Verified source checksum - md5 and sha512
> >>>   - Verified signature
> >>>   - Build was successful on the source (mvn clean -DskipTests
> package
> >>>   -Pdist,embedded-hbase-solr)
> >>>   - Ran quickStart and validated types and entities were created
> >>>   successfully.
> >>>   - Validated tag propagation works
> >>>   - Added/removed labels to an entity and verified audit is getting
> >>>   updated correctly
> >>>   - Added user-defined properties and business metadata attributes.
> >>>   - Deleted few entities and purged those entities - validated if
> purge
> >>>   audit recorded correctly.
> >>>   - Basic sanity tests - basic search (with system attributes),
> advanced
> >>>   search, quick search.
> >>>
> >>>
> >>>
> >>> Thanks,
> >>> Sarath
> >>>
> >>>
> >>>
> >>> On Sat, Jun 6, 2020 at 10:22 AM Madhan Neethiraj <
> mad...@apache.org>
> >>> wrote:
> >>>
>  (resending with plain-text format)
> 
>  Atlas team,
> 
>  Apache Atlas 2.1.0 rc1, with following fixes since rc0, is now
> >> available
>  for vote within dev community.
> 
> ATLAS-3770: UI(Classic): Active and Deleted hyperlinks for
> certain
>  entities throwing error on click
> ATLAS-3766: Stats modal not close issue #2
> ATLAS-3674: ZipFileMigationImporter: Migration status display
> fix.
>  Part 2
> 
>  Links to the release artifacts are given below. Please review and
> vote.
> 
>  The vote will be open for at least 72 hours or until necessary
> votes
> >> are
>  reached.
>   [ ] +1 Approve
>   [ ] +0 No opinion
>   [ ] -1 Disapprove (and reason why)
> 

Re: Review Request 72540: ATLAS-3806: Classifications information missing in notification events during entity create/update

2020-05-29 Thread Sidharth Mishra
ionsOnEntityDelete": false
},
{
"typeName": "PII",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "Fact",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
},
"operationType": "CLASSIFICATION_ADD",
"eventTime": 1590181673717
}
}
{
"version": {
"version": "1.0.0",
"versionParts": [
1
]
},
"msgCompressionKind": "NONE",
"msgSplitIdx": 1,
"msgSplitCount": 1,
"msgSourceIP": "172.27.168.128",
"msgCreatedBy": "",
"msgCreationTime": 1590181680710,
"message": {
"type": "ENTITY_NOTIFICATION_V2",
"entity": {
"typeName": "hive_process",
"attributes": {
"qualifiedName": 
"sid_hive_db.sid_hive_table_ctas3@bulk:1590120919000",
"name": "create table sid_hive_table_ctas3 as select * from 
sid_hive_table_1"
},
"guid": "36cc1a73-9c55-4a66-9d59-fa1a2f535791",
"status": "ACTIVE",
"displayText": "create table sid_hive_table_ctas3 as select * from 
sid_hive_table_1",
"classificationNames": [
"ETL",
"PII",
"Fact"
],
"classifications": [
{
"typeName": "ETL",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "PII",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "Fact",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
},
"operationType": "CLASSIFICATION_ADD",
"eventTime": 1590181673717
}
}
   
{
"version": {
"version": "1.0.0",
"versionParts": [
1
]
},
"msgCompressionKind": "NONE",
"msgSplitIdx": 1,
"msgSplitCount": 1,
"msgSourceIP": "172.27.168.128",
"msgCreatedBy": "",
"msgCreationTime": 1590181680714,
"message": {
"type": "ENTITY_NOTIFICATION_V2",
"entity": {
"typeName": "hive_table",
"attributes": {
"owner": "anonymous",
"createTime": 1590120919000,
"qualifiedName": "sid_hive_db.sid_hive_table_ctas3@bulk",
"name": "sid_hive_table_ctas3"
},
"guid": "ab302745-a843-4164-8166-017f85c7e15e",
"status": "ACTIVE",
"displayText": "sid_hive_table_ctas3",
"classificationNames": [
"ETL",
"PII",
"Fact"
],
"classifications": [
{
"typeName": "ETL",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "PII",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "Fact",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
},
"operationType": "CLASSIFICATION_ADD",
"eventTime": 1590181673717
}
}


Thanks,

Sidharth Mishra



Re: Review Request 72540: ATLAS-3806: Classifications information missing in notification events during entity create/update

2020-05-28 Thread Sidharth Mishra
ionsOnEntityDelete": false
},
{
"typeName": "PII",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "Fact",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
},
"operationType": "CLASSIFICATION_ADD",
"eventTime": 1590181673717
}
}
{
"version": {
"version": "1.0.0",
"versionParts": [
1
]
},
"msgCompressionKind": "NONE",
"msgSplitIdx": 1,
"msgSplitCount": 1,
"msgSourceIP": "172.27.168.128",
"msgCreatedBy": "",
"msgCreationTime": 1590181680710,
"message": {
"type": "ENTITY_NOTIFICATION_V2",
"entity": {
"typeName": "hive_process",
"attributes": {
"qualifiedName": 
"sid_hive_db.sid_hive_table_ctas3@bulk:1590120919000",
"name": "create table sid_hive_table_ctas3 as select * from 
sid_hive_table_1"
},
"guid": "36cc1a73-9c55-4a66-9d59-fa1a2f535791",
"status": "ACTIVE",
"displayText": "create table sid_hive_table_ctas3 as select * from 
sid_hive_table_1",
"classificationNames": [
"ETL",
"PII",
"Fact"
],
"classifications": [
{
"typeName": "ETL",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "PII",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "Fact",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
},
"operationType": "CLASSIFICATION_ADD",
"eventTime": 1590181673717
}
}
   
{
"version": {
"version": "1.0.0",
"versionParts": [
1
]
},
"msgCompressionKind": "NONE",
"msgSplitIdx": 1,
"msgSplitCount": 1,
"msgSourceIP": "172.27.168.128",
"msgCreatedBy": "",
"msgCreationTime": 1590181680714,
"message": {
"type": "ENTITY_NOTIFICATION_V2",
"entity": {
"typeName": "hive_table",
"attributes": {
"owner": "anonymous",
"createTime": 1590120919000,
"qualifiedName": "sid_hive_db.sid_hive_table_ctas3@bulk",
"name": "sid_hive_table_ctas3"
},
"guid": "ab302745-a843-4164-8166-017f85c7e15e",
"status": "ACTIVE",
"displayText": "sid_hive_table_ctas3",
"classificationNames": [
"ETL",
"PII",
"Fact"
],
"classifications": [
{
"typeName": "ETL",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "PII",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "Fact",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
},
"operationType": "CLASSIFICATION_ADD",
"eventTime": 1590181673717
}
}


Thanks,

Sidharth Mishra



Re: Review Request 72540: ATLAS-3806: Classifications information missing in notification events during entity create/update

2020-05-28 Thread Sidharth Mishra
opagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "PII",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "Fact",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
},
"operationType": "CLASSIFICATION_ADD",
"eventTime": 1590181673717
}
}
{
"version": {
"version": "1.0.0",
"versionParts": [
1
]
},
"msgCompressionKind": "NONE",
"msgSplitIdx": 1,
"msgSplitCount": 1,
"msgSourceIP": "172.27.168.128",
"msgCreatedBy": "",
"msgCreationTime": 1590181680710,
"message": {
"type": "ENTITY_NOTIFICATION_V2",
"entity": {
"typeName": "hive_process",
"attributes": {
"qualifiedName": 
"sid_hive_db.sid_hive_table_ctas3@bulk:1590120919000",
"name": "create table sid_hive_table_ctas3 as select * from 
sid_hive_table_1"
},
"guid": "36cc1a73-9c55-4a66-9d59-fa1a2f535791",
"status": "ACTIVE",
"displayText": "create table sid_hive_table_ctas3 as select * from 
sid_hive_table_1",
"classificationNames": [
"ETL",
"PII",
"Fact"
],
"classifications": [
{
"typeName": "ETL",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "PII",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "Fact",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
},
"operationType": "CLASSIFICATION_ADD",
"eventTime": 1590181673717
}
}
   
{
"version": {
"version": "1.0.0",
"versionParts": [
1
]
},
"msgCompressionKind": "NONE",
"msgSplitIdx": 1,
"msgSplitCount": 1,
"msgSourceIP": "172.27.168.128",
"msgCreatedBy": "",
"msgCreationTime": 1590181680714,
"message": {
"type": "ENTITY_NOTIFICATION_V2",
"entity": {
"typeName": "hive_table",
"attributes": {
"owner": "anonymous",
"createTime": 1590120919000,
"qualifiedName": "sid_hive_db.sid_hive_table_ctas3@bulk",
"name": "sid_hive_table_ctas3"
},
"guid": "ab302745-a843-4164-8166-017f85c7e15e",
"status": "ACTIVE",
"displayText": "sid_hive_table_ctas3",
"classificationNames": [
"ETL",
"PII",
"Fact"
],
"classifications": [
{
"typeName": "ETL",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "PII",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "Fact",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
},
"operationType": "CLASSIFICATION_ADD",
"eventTime": 1590181673717
}
}


Thanks,

Sidharth Mishra



Re: Review Request 72540: ATLAS-3806: Classifications information missing in notification events during entity create/update

2020-05-28 Thread Sidharth Mishra


> On May 26, 2020, 10:45 p.m., Madhan Neethiraj wrote:
> > Sid - I suggest to consider the following alternate approach to address 
> > this issue:
> > 
> >   public class EntityGraphMapper{
> > private AtlasEntityHeader constructHeader(AtlasEntity entity, final 
> > AtlasEntityType type, AtlasVertex vertex) {
> > /*
> >  * replace current implementation with a call to 
> > entityRetriever.toAtlasEntityHeaderWithClassifications()
> >  * 
> > AtlasEntityHeader header = new 
> > AtlasEntityHeader(entity.getTypeName());
> >  
> > header.setGuid(getIdFromVertex(vertex));
> > header.setStatus(entity.getStatus());
> > header.setIsIncomplete(entity.getIsIncomplete());
> >  
> > for (AtlasAttribute attribute : type.getUniqAttributes().values()) {
> > header.setAttribute(attribute.getName(), 
> > entity.getAttribute(attribute.getName()));
> > }
> >  
> > return header;
> >  *
> >  */
> > 
> > return 
> > entityRetriever.toAtlasEntityHeaderWithClassifications(vertex);
> > }
> >   }
> >   
> >   public class EntityMutationResponse {
> > JsonIgnore
> > public void addEntity(EntityOperation op, AtlasEntityHeader header) {
> > // if an entity is already included in CREATE, update the header, 
> > to capture propagated classifications
> > if (op == EntityOperation.UPDATE || op == 
> > EntityOperation.PARTIAL_UPDATE) {
> > if (entityHeaderExists(getCreatedEntities(), header.getGuid())) 
> > {
> > // return;
> > op = EntityOperaton.CREATE;
> > }
> > }
> > 
> > ...
> > }
> >   }

As you suggested I have changed the code. The issue is, we cache the entity at 
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java:285
 and 313 which is not getting updated with classification. At 
org.apache.atlas.repository.store.graph.v2.AtlasEntityChangeNotifier#notifyPropagatedEntities
 when we call fullTextMapperV2.getAndCacheEntity(guid) we still get the old one 
without classification. So along with this change I added code to 
constructHeader so that it will set classification to entity from entityHeader. 
Now the updated solution works.


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72540/#review220873
---


On May 28, 2020, 10:58 p.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72540/
> ---
> 
> (Updated May 28, 2020, 10:58 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3806
> https://issues.apache.org/jira/browse/ATLAS-3806
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Classifications information missing in notification events during entity 
> create/update
> 
> 
> Diffs
> -
> 
>   
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
>  d83f0e0f1 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
>  2ed524ff5 
>   
> webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
>  a677b315c 
> 
> 
> Diff: https://reviews.apache.org/r/72540/diff/2/
> 
> 
> Testing
> ---
> 
> For add new ctas notification looks like (Just shown for ctas entity):
> {
> "version": {
> "version": "1.0.0",
> "versionParts": [
> 1
> ]
> },
> "msgCompressionKind": "NONE",
> "msgSplitIdx": 1,
> "msgSplitCount": 1,
> "msgSourceIP": "172.27.168.128",
> "msgCreatedBy": "",
> "msgCreationTime": 1590176023734,
> "message": {
> "type": "ENTITY_NOTIFICATION_V2",
> "entity": {
> "typeName": "hive_table",
> "attributes": {
> "owner": "anonymous",
> "cre

Review Request 72540: ATLAS-3806: Classifications information missing in notification events during entity create/update

2020-05-22 Thread Sidharth Mishra
23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "Fact",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
},
"operationType": "CLASSIFICATION_ADD",
"eventTime": 1590181673717
}
}
{
"version": {
"version": "1.0.0",
"versionParts": [
1
]
},
"msgCompressionKind": "NONE",
"msgSplitIdx": 1,
"msgSplitCount": 1,
"msgSourceIP": "172.27.168.128",
"msgCreatedBy": "",
"msgCreationTime": 1590181680710,
"message": {
"type": "ENTITY_NOTIFICATION_V2",
"entity": {
"typeName": "hive_process",
"attributes": {
"qualifiedName": 
"sid_hive_db.sid_hive_table_ctas3@bulk:1590120919000",
"name": "create table sid_hive_table_ctas3 as select * from 
sid_hive_table_1"
},
"guid": "36cc1a73-9c55-4a66-9d59-fa1a2f535791",
"status": "ACTIVE",
"displayText": "create table sid_hive_table_ctas3 as select * from 
sid_hive_table_1",
"classificationNames": [
"ETL",
"PII",
"Fact"
],
"classifications": [
{
"typeName": "ETL",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "PII",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "Fact",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
},
"operationType": "CLASSIFICATION_ADD",
"eventTime": 1590181673717
}
}
   
{
"version": {
"version": "1.0.0",
"versionParts": [
1
]
},
"msgCompressionKind": "NONE",
"msgSplitIdx": 1,
"msgSplitCount": 1,
"msgSourceIP": "172.27.168.128",
"msgCreatedBy": "",
"msgCreationTime": 1590181680714,
"message": {
"type": "ENTITY_NOTIFICATION_V2",
"entity": {
"typeName": "hive_table",
"attributes": {
"owner": "anonymous",
"createTime": 1590120919000,
"qualifiedName": "sid_hive_db.sid_hive_table_ctas3@bulk",
"name": "sid_hive_table_ctas3"
},
"guid": "ab302745-a843-4164-8166-017f85c7e15e",
"status": "ACTIVE",
"displayText": "sid_hive_table_ctas3",
"classificationNames": [
"ETL",
"PII",
"Fact"
],
"classifications": [
{
"typeName": "ETL",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "PII",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "Fact",
"entityGuid": "410cb0f3-6df7-4523-ba23-8cd65752a811",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
},
"operationType": "CLASSIFICATION_ADD",
"eventTime": 1590181673717
}
}


Thanks,

Sidharth Mishra



Re: Review Request 72515: ATLAS-3785: Modify the AWS s3 v2 model to make relationship def end points as legacy attributes and objectPrefix as optional

2020-05-15 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72515/
---

(Updated May 15, 2020, 6:50 a.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3785
https://issues.apache.org/jira/browse/ATLAS-3785


Repository: atlas


Description
---

Modify the aws s3 v2 model to make relationship def end points as legacy 
attributes and objectPrefix as optional


Diffs (updated)
-

  addons/models/3000-Cloud/3030-aws_s3_v2_typedefs.json 676a2237c 


Diff: https://reviews.apache.org/r/72515/diff/2/

Changes: https://reviews.apache.org/r/72515/diff/1-2/


Testing
---

Manually added to my cluster


Thanks,

Sidharth Mishra



Re: Review Request 72512: Edge Creation: Refactoring

2020-05-14 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72512/#review220776
---


Ship it!




Ship It!

- Sidharth Mishra


On May 14, 2020, 10:41 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72512/
> ---
> 
> (Updated May 14, 2020, 10:41 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-3797
> https://issues.apache.org/jira/browse/ATLAS-3797
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Approach**
> Modified: _GraphHelper.getOrCreateEdge_ to use 
> _AtlasJanusGraph.getEdgeBetweenVertices_.
> Modified: _AtlasJanusGraph.getEdgeBetweenVertices_ modified fetch.
> 
> 
> Diffs
> -
> 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java
>  35274c9d1 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> b1368e5e9 
> 
> 
> Diff: https://reviews.apache.org/r/72512/diff/2/
> 
> 
> Testing
> ---
> 
> **Pre-commit**
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1896/
> 
> **Volume test**
> Case edge creation involves existing entity where large number of existing 
> edges (200K), this change shows significant improvement.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>



Review Request 72515: ATLAS-3785: Modify the AWS s3 v2 model to make relationship def end points as legacy attributes and objectPrefix as optional

2020-05-14 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72515/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3785
https://issues.apache.org/jira/browse/ATLAS-3785


Repository: atlas


Description
---

Modify the aws s3 v2 model to make relationship def end points as legacy 
attributes and objectPrefix as optional


Diffs
-

  addons/models/3000-Cloud/3030-aws_s3_v2_typedefs.json 676a2237c 


Diff: https://reviews.apache.org/r/72515/diff/1/


Testing
---

Manually added to my cluster


Thanks,

Sidharth Mishra



Re: Review Request 72512: Edge Creation: Refactoring

2020-05-14 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72512/#review220770
---




graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java
Line 177 (original), 179 (patched)
<https://reviews.apache.org/r/72512/#comment309462>

As we are not returning ret, please rename this to something more 
meaningful like gremlinEdge ( As we are trying to find the gremlinEdge here)



graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java
Lines 183 (patched)
<https://reviews.apache.org/r/72512/#comment309461>

Kindly change this to somthing like below as second 
ret.property(STATE_PROPERTY_KEY).isPresent() check is not rquired:

if (ret != null && (!ret.property(STATE_PROPERTY_KEY).isPresent() ||
 
ret.property(STATE_PROPERTY_KEY).value().equals(AtlasEntity.Status.ACTIVE.toString()))



graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java
Lines 195 (patched)
<https://reviews.apache.org/r/72512/#comment309463>

Just to avoid two returns statement as we do other places in atlas, we can 
do something like:

Initialize variable AtlasEdge ret = null;
At 192 line: ret = GraphDbObjectFactory.createEdge(this, ret);

At the end return ret;


- Sidharth Mishra


On May 14, 2020, 6:19 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72512/
> ---
> 
> (Updated May 14, 2020, 6:19 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-3797
> https://issues.apache.org/jira/browse/ATLAS-3797
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Approach**
> Modified: _GraphHelper.getOrCreateEdge_ to use 
> _AtlasJanusGraph.getEdgeBetweenVertices_.
> Modified: _AtlasJanusGraph.getEdgeBetweenVertices_ modified fetch.
> 
> 
> Diffs
> -
> 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java
>  35274c9d1 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> b1368e5e9 
> 
> 
> Diff: https://reviews.apache.org/r/72512/diff/1/
> 
> 
> Testing
> ---
> 
> **Pre-commit**
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1896/
> 
> **Volume test**
> Case edge creation involves existing entity where large number of existing 
> edges (200K), this change shows significant improvement.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>



Re: Review Request 72464: ATLAS-3778: Improve performance during classification delete

2020-05-04 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72464/#review220605
---


Ship it!




Ship It!

- Sidharth Mishra


On May 3, 2020, 11:04 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72464/
> ---
> 
> (Updated May 3, 2020, 11:04 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
> and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3778
> https://issues.apache.org/jira/browse/ATLAS-3778
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> When deleting a classification, we remove the following properties from 
> entity vertex and update these properties with the classification name 
> removed.
> 
> PROPAGATED_TRAIT_NAMES_PROPERTY_KEY
> TRAIT_NAMES_PROPERTY_KEY
> 
> This will be an expensive operation during commit time when dealing with 
> large number of classifications. This improvement will only remove the 
> deleted classification name from both the above property keys and the commit 
> foot print will be less.
> 
> 
> Diffs
> -
> 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasElement.java
>  4af39eded 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusElement.java
>  a9cc5a6ce 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  5643599a4 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
>  5564bc59a 
> 
> 
> Diff: https://reviews.apache.org/r/72464/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested on delete classification with 400 classifications associated 
> to entity.
> 
> Performance improvement seen from 19 mins to 29 secs
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>



Re: Review Request 72452: Efficiently Searching for Edges Between Vertices

2020-05-04 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72452/#review220604
---


Ship it!




Ship It!

- Sidharth Mishra


On April 30, 2020, 4:04 a.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72452/
> ---
> 
> (Updated April 30, 2020, 4:04 a.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, 
> Sarath Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3762
> https://issues.apache.org/jira/browse/ATLAS-3762
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Problem Definition**
> Please refer to JIRA for details.
> 
> **Updates**
> - Modifed: _AtlasJanusGraph.wrapVertices_ and _AtlasJanusGraph.wrapEdges_ : 
> Now uses genuine iterators. This reduces number of elements fetched, since 
> the search is linear.
> - New: _AtlasVertex.getEdgeCount_ fetches edge count using iterator returned 
> from _JanusVertex_. Fetching the count is effecient using stream support.
> 
> 
> **Metrics**
> 
> This difference exists because earlier even _fromVertex_ did not have any 
> edges, the search would iterate through all the edges of the _toVertex_ and 
> return NULL. This condition is detected in the algorithm and used to reduce 
> the search cycle.
> 
> Before: "getRelationshipEdge":{"count":500,"timeTaken":37048}
> After: "getRelationshipEdge":{"count":500,"timeTaken":42}
> 
> 
> Diffs
> -
> 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertex.java
>  9406e26ff 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java
>  eb0206271 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusVertex.java
>  fdc9fd0b5 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 2b8227a7e 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java
>  d1c1f1255 
> 
> 
> Diff: https://reviews.apache.org/r/72452/diff/4/
> 
> 
> Testing
> ---
> 
> **Volume testing**
> High volume testing makes the edge fetching effcient. Cases where incoming 
> edges was in 1000s and outgoing edges was handful.
> 
> Memory footprint has improved since JanusGrpah caches edges and then expires 
> it. Fetching fewer edges will reduce number of items in memory.
> 
> **Pre-commit Build**
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1860/
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>



Re: Review Request 72452: Efficiently Searching for Edges Between Vertices

2020-04-29 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72452/#review220557
---


Ship it!




Ship It!

- Sidharth Mishra


On April 29, 2020, 9:17 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72452/
> ---
> 
> (Updated April 29, 2020, 9:17 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, 
> Sarath Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3762
> https://issues.apache.org/jira/browse/ATLAS-3762
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Problem Definition**
> Please refer to JIRA for details.
> 
> **Updates**
> - Modifed: _AtlasJanusGraph.wrapVertices_ and _AtlasJanusGraph.wrapEdges_ : 
> Now uses genuine iterators. This reduces number of elements fetched, since 
> the search is linear.
> - New: _AtlasVertex.getEdgeCount_ fetches edge count using iterator returned 
> from _JanusVertex_. Fetching the count is effecient using stream support.
> 
> 
> Diffs
> -
> 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertex.java
>  9406e26ff 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java
>  eb0206271 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusVertex.java
>  fdc9fd0b5 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 2b8227a7e 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java
>  d1c1f1255 
> 
> 
> Diff: https://reviews.apache.org/r/72452/diff/3/
> 
> 
> Testing
> ---
> 
> **Volume testing**
> High volume testing makes the edge fetching effcient. Cases where incoming 
> edges was in 1000s and outgoing edges was handful.
> 
> Memory footprint has improved since JanusGrpah caches edges and then expires 
> it. Fetching fewer edges will reduce number of items in memory.
> 
> **Pre-commit Build**
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1858/
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>



Re: Review Request 72452: Efficiently Searching for Edges Between Vertices

2020-04-29 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72452/#review220553
---




repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java
Lines 780 (patched)
<https://reviews.apache.org/r/72452/#comment309045>

Just thinking the below might be more easy to understand if someone is not 
aware of why we do both edge count.

   AtlasPerfMetrics.MetricRecorder metric = 
RequestContext.get().startMetricRecord("getRelationshipEdge");

AtlasEdge   ret = null;
long toVertexIncomingEdgesCount = 
graphHelper.getInComingEdgesByLabelCount(toVertex, relationshipLabel);

if(toVertexIncomingEdgesCount > 0) {
long fromVertexOutgoingEdgesCount = 
graphHelper.getOutGoingEdgesByLabelCount(fromVertex, relationshipLabel);

if(toVertexIncomingEdgesCount < fromVertexOutgoingEdgesCount) {
Iterator edgesIterator = 
graphHelper.getIncomingEdgesByLabel(toVertex, relationshipLabel);
ret = getActiveEdgeFromList(fromVertex.getId(), 
edgesIterator);
} else if(fromVertexOutgoingEdgesCount > 0) {
Iterator edgesIterator = 
graphHelper.getOutGoingEdgesByLabel(fromVertex, relationshipLabel);
ret = getActiveEdgeFromList(toVertex.getId(), 
edgesIterator);
}
}

RequestContext.get().endMetricRecord(metric);
    return ret;


- Sidharth Mishra


On April 29, 2020, 5:51 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72452/
> ---
> 
> (Updated April 29, 2020, 5:51 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, 
> Sarath Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3762
> https://issues.apache.org/jira/browse/ATLAS-3762
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Problem Definition**
> Please refer to JIRA for details.
> 
> **Updates**
> - Modifed: _AtlasJanusGraph.wrapVertices_ and _AtlasJanusGraph.wrapEdges_ : 
> Now uses genuine iterators. This reduces number of elements fetched, since 
> the search is linear.
> - New: _AtlasVertex.getEdgeCount_ fetches edge count using iterator returned 
> from _JanusVertex_. Fetching the count is effecient using stream support.
> 
> 
> Diffs
> -
> 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertex.java
>  9406e26ff 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java
>  eb0206271 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusVertex.java
>  fdc9fd0b5 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 2b8227a7e 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasRelationshipStoreV2.java
>  d1c1f1255 
> 
> 
> Diff: https://reviews.apache.org/r/72452/diff/1/
> 
> 
> Testing
> ---
> 
> **Volume testing**
> High volume testing makes the edge fetching effcient. Cases where incoming 
> edges was in 1000s and outgoing edges was handful.
> 
> Memory footprint has improved since JanusGrpah caches edges and then expires 
> it. Fetching fewer edges will reduce number of items in memory.
> 
> **Pre-commit Build**
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1858/
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>



Re: Review Request 72411: ATLAS-3747: Atlas Admin Purge API should take list of guids as body with operation Put instead of Delete

2020-04-21 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72411/
---

(Updated April 21, 2020, 9:45 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Summary (updated)
-

ATLAS-3747: Atlas Admin Purge API should take list of guids as body with 
operation Put instead of Delete


Bugs: ATLAS-3747
https://issues.apache.org/jira/browse/ATLAS-3747


Repository: atlas


Description
---

ATLAS-3747: Atlas Admin Purge API should take list of guids as query param 
instead of body


Diffs
-

  client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 5a19a0b0e 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
a7d554a36 


Diff: https://reviews.apache.org/r/72411/diff/1/


Testing (updated)
---

Manual Testing:

curl -u admin:admin123 --location --request PUT 
'http://sid-ycloud-atlas2-1.sid-ycloud-atlas2.root.hwx.site:31000/api/atlas/admin/purge/'
 \
--header 'Content-Type: application/json' \
--header 'Content-Type: application/json' \
--header 'Cookie: ATLASSESSIONID=node0g7w7rpfe6o0a1e4e7hk5cth4d2.node0' \
--data-raw '["bd7262f6-fed5-40f3-9f72-f57fba6b1193", 
"24c4df6e-b27a-4c60-b85a-67ed20210446", "24c4df6e-b27a-4c60-b85a-67ed20210446"]'

Note: As local jetty was failing for couple of times, I was not able to run 
EntityV2JerseyResourceIT. precommit run will ensure if this passes.


Thanks,

Sidharth Mishra



Re: Review Request 72411: ATLAS-3747: Atlas Admin Purge API should take list of guids as body with operation Put instead of Delete

2020-04-21 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72411/
---

(Updated April 21, 2020, 9:45 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3747
https://issues.apache.org/jira/browse/ATLAS-3747


Repository: atlas


Description
---

ATLAS-3747: Atlas Admin Purge API should take list of guids as query param 
instead of body


Diffs (updated)
-

  client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 5a19a0b0e 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
a7d554a36 


Diff: https://reviews.apache.org/r/72411/diff/2/

Changes: https://reviews.apache.org/r/72411/diff/1-2/


Testing
---

Manual Testing:

curl -u admin:admin123 --location --request PUT 
'http://sid-ycloud-atlas2-1.sid-ycloud-atlas2.root.hwx.site:31000/api/atlas/admin/purge/'
 \
--header 'Content-Type: application/json' \
--header 'Content-Type: application/json' \
--header 'Cookie: ATLASSESSIONID=node0g7w7rpfe6o0a1e4e7hk5cth4d2.node0' \
--data-raw '["bd7262f6-fed5-40f3-9f72-f57fba6b1193", 
"24c4df6e-b27a-4c60-b85a-67ed20210446", "24c4df6e-b27a-4c60-b85a-67ed20210446"]'

Note: As local jetty was failing for couple of times, I was not able to run 
EntityV2JerseyResourceIT. precommit run will ensure if this passes.


Thanks,

Sidharth Mishra



Review Request 72411: ATLAS-3747: Atlas Admin Purge API should take list of guids as query param instead of body

2020-04-21 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72411/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3747
https://issues.apache.org/jira/browse/ATLAS-3747


Repository: atlas


Description
---

ATLAS-3747: Atlas Admin Purge API should take list of guids as query param 
instead of body


Diffs
-

  client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 5a19a0b0e 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
a7d554a36 


Diff: https://reviews.apache.org/r/72411/diff/1/


Testing
---

Manual Testing:
curl -u admin:admin123 --location --request DELETE 
'http://sid-ycloud-atlas2-1.sid-ycloud-atlas2.root.hwx.site:31000/api/atlas/admin/purge/?guid=82785128-778a-4e0d-aac0-8327b4ca1e87=f0cc9cfd-71af-47fc-9321-2970dfa5825f=f0cc9cfd-71af-47fc-9321-2970dfa5825f'
 --header 'Content-Type: application/json'

Note: As local jetty was failing for couple of times, I was not able to run 
EntityV2JerseyResourceIT. precommit run will ensure if this passes.


Thanks,

Sidharth Mishra



Re: Review Request 72367: ATLAS-3709: fix to address skipped index initialization in earlier commit

2020-04-15 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72367/#review220328
---


Ship it!




Ship It!

- Sidharth Mishra


On April 15, 2020, 5:22 p.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72367/
> ---
> 
> (Updated April 15, 2020, 5:22 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Mandar Ambawane, Mehul Parikh, 
> Nixon Rodrigues, Pinal Shah, Sarath Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3709
> https://issues.apache.org/jira/browse/ATLAS-3709
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> With the changes in earlier commit for ATLAS-3709, 
> notifyInitializationCompletion() was called without any entity-types which 
> resulted in index not getting initiaized. This issue is fixed in this patch.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  f505bee0a 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/SolrIndexHelper.java
>  0670d5000 
> 
> 
> Diff: https://reviews.apache.org/r/72367/diff/1/
> 
> 
> Testing
> ---
> 
> - pre-commit tests run: 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1810/
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>



Re: Review Request 72257: ATLAS-3583 Use Audit framework to generate audit entries for Classification CREATE, UPDATE and DELETE

2020-04-15 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72257/#review220322
---




repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AuditChangeListener.java
Lines 71 (patched)
<https://reviews.apache.org/r/72257/#comment308655>

To get more details for these audit records, we need to change code simlar 
to the patch which I have - https://reviews.apache.org/r/72351/ (Look for 
AtlasAuditEntry.java changes. toEntityAuditActionV2() should return proper 
state which can be used to get more details from entity audit - HBase)


- Sidharth Mishra


On April 7, 2020, 6:22 a.m., Mandar Ambawane wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72257/
> ---
> 
> (Updated April 7, 2020, 6:22 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
> Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3583
> https://issues.apache.org/jira/browse/ATLAS-3583
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Added listener to listen create/update/delete operations of all type defs.
> 
> 
> Diffs
> -
> 
>   addons/models/-Area0/0010-base_model.json 351e07b 
>   intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
> f2a8b3e 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStore.java
>  79f5270 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AuditChangeListener.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/72257/diff/1/
> 
> 
> Testing
> ---
> 
> Pre-commit build: 
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1747/console
> 
> Manual testing is done.
> 
> 
> Thanks,
> 
> Mandar Ambawane
> 
>



Re: Review Request 72351: ATLAS-3727: Add get result details REST for admin audit record

2020-04-13 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72351/
---

(Updated April 14, 2020, 12:46 a.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3727
https://issues.apache.org/jira/browse/ATLAS-3727


Repository: atlas


Description
---

ATLAS-3727: Add get result details REST for admin audit record
ATLAS-3729: Add audit operation as a parameter to v2/entity/{guid}/audit


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
98e56f2bb 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
bcfdd9466 
  
repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
 0deb7295e 
  
repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
 35bf1d07e 
  
repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
 b8131bd39 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 4c1e1a959 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditRepository.java
 2a47e39ed 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 021ca43ab 
  
repository/src/main/java/org/apache/atlas/repository/audit/InMemoryEntityAuditRepository.java
 ad6ec94c0 
  
repository/src/main/java/org/apache/atlas/repository/audit/NoopEntityAuditRepository.java
 4bb68d502 
  
repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
 aa175a229 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
2ab8bdb03 
  webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 402a323dd 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
43b85efc0 


Diff: https://reviews.apache.org/r/72351/diff/3/

Changes: https://reviews.apache.org/r/72351/diff/2-3/


Testing
---

Manual Testing:

curl -u admin:admin123 --location --request GET 
'http://sid-ycloud-atlas1-1.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/admin/audit/706e6e32-b619-43f2-8587-f2877ee27582/details?limit=100=0'
 --header 'Content-Type: application/json'

curl -u admin:admin123 --location --request GET 
'http://sid-ycloud-atlas1-1.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/0b10d06b-6740-4550-a571-5be8ba1ea3ae/audit?count=1=0b10d06b-6740-4550-a571-5be8ba1ea3ae:1586797534411:0:1586797535131=ENTITY_UPDATE'

curl -u admin:admin123 --location --request GET 
'http://sid-ycloud-atlas1-1.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/0b10d06b-6740-4550-a571-5be8ba1ea3ae/audit?count=1=ENTITY_UPDATE'


Thanks,

Sidharth Mishra



Review Request 72358: ATLAS-3730: In case Hard delete and Purge of an Entity we should make the entity status as purged instead of deleted

2020-04-13 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72358/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3730
https://issues.apache.org/jira/browse/ATLAS-3730


Repository: atlas


Description
---

ATLAS-3730: Changed code to make entity status to PURGED at classification in 
case of Hard delete or Purge of an entity


Diffs
-

  intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java c534fd06f 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 7b438600d 


Diff: https://reviews.apache.org/r/72358/diff/1/


Testing
---

1. Before Test when we do get we can see the classification contains entity 
status as DELETED. 
curl -u admin:admin123 --location --request GET 
'http://sid-ycloud-atlas1-1.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/guid/b7d5d07c-fbaf-4a10-9c2e-c360d4ac0828?minExtInfo=true&_=1586810974769'


"classifications": [
{
"typeName": "TEST_TAG_2",
"entityGuid": "b7d5d07c-fbaf-4a10-9c2e-c360d4ac0828",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "TEST_TAG_1",
"entityGuid": "7d5dfc2d-bcb9-429a-b241-a34705777b6b",
"entityStatus": "DELETED",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "TEST_TAG_1",
"entityGuid": "a2d1cad9-d2a1-4fe4-8570-2e58e16877ca",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]
2. Purge the entity:
curl -u admin:admin123 --location --request DELETE 
'http://sid-ycloud-atlas1-1.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/admin/purge/'
 \
--header 'Content-Type: application/json' \
--data-raw ' ["7d5dfc2d-bcb9-429a-b241-a34705777b6b"]'

3. After purge of entity - 7d5dfc2d-bcb9-429a-b241-a34705777b6b we can see the 
entity state changed to PURGED.

curl -u admin:admin123 --location --request GET 
'http://sid-ycloud-atlas1-1.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/guid/b7d5d07c-fbaf-4a10-9c2e-c360d4ac0828?minExtInfo=true&_=1586810974769'

"classifications": [
{
"typeName": "TEST_TAG_2",
"entityGuid": "b7d5d07c-fbaf-4a10-9c2e-c360d4ac0828",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "TEST_TAG_1",
"entityGuid": "7d5dfc2d-bcb9-429a-b241-a34705777b6b",
"entityStatus": "PURGED",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
},
{
"typeName": "TEST_TAG_1",
"entityGuid": "a2d1cad9-d2a1-4fe4-8570-2e58e16877ca",
"entityStatus": "ACTIVE",
"propagate": true,
"validityPeriods": [],
"removePropagationsOnEntityDelete": false
}
]


Thanks,

Sidharth Mishra



Re: Review Request 72351: ATLAS-3727: Add get result details REST for admin audit record

2020-04-13 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72351/
---

(Updated April 13, 2020, 6:17 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3727
https://issues.apache.org/jira/browse/ATLAS-3727


Repository: atlas


Description (updated)
---

ATLAS-3727: Add get result details REST for admin audit record
ATLAS-3729: Add audit operation as a parameter to v2/entity/{guid}/audit


Diffs
-

  intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
98e56f2bb 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
bcfdd9466 
  
repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
 0deb7295e 
  
repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
 35bf1d07e 
  
repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
 b8131bd39 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 4c1e1a959 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditRepository.java
 2a47e39ed 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 021ca43ab 
  
repository/src/main/java/org/apache/atlas/repository/audit/InMemoryEntityAuditRepository.java
 ad6ec94c0 
  
repository/src/main/java/org/apache/atlas/repository/audit/NoopEntityAuditRepository.java
 4bb68d502 
  
repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
 aa175a229 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
2ab8bdb03 
  webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 402a323dd 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
43b85efc0 


Diff: https://reviews.apache.org/r/72351/diff/2/


Testing (updated)
---

Manual Testing:

curl -u admin:admin123 --location --request GET 
'http://sid-ycloud-atlas1-1.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/admin/audit/706e6e32-b619-43f2-8587-f2877ee27582/details?limit=100=0'
 --header 'Content-Type: application/json'

curl -u admin:admin123 --location --request GET 
'http://sid-ycloud-atlas1-1.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/0b10d06b-6740-4550-a571-5be8ba1ea3ae/audit?count=1=0b10d06b-6740-4550-a571-5be8ba1ea3ae:1586797534411:0:1586797535131=ENTITY_UPDATE'

curl -u admin:admin123 --location --request GET 
'http://sid-ycloud-atlas1-1.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/v2/entity/0b10d06b-6740-4550-a571-5be8ba1ea3ae/audit?count=1=ENTITY_UPDATE'


Thanks,

Sidharth Mishra



Re: Review Request 72351: ATLAS-3727: Add get result details REST for admin audit record

2020-04-13 Thread Sidharth Mishra


> On April 11, 2020, 1:35 a.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
> > Lines 231 (patched)
> > <https://reviews.apache.org/r/72351/diff/1/?file=2217432#file2217432line231>
> >
> > caching => limit

I think this parameter is causing confusion and its been used for 
scan.setCaching(MAXPAYLOAD) which will increase the payload size to MAXPAYLOAD 
rows per remote call. I am removing this from function parameter as this will 
be always to a const value.


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72351/#review220287
---


On April 13, 2020, 6:15 p.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72351/
> ---
> 
> (Updated April 13, 2020, 6:15 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3727
> https://issues.apache.org/jira/browse/ATLAS-3727
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-3727: Add get result details REST for admin audit record
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
> 98e56f2bb 
>   intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
> bcfdd9466 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
>  0deb7295e 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
>  35bf1d07e 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
>  b8131bd39 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
>  4c1e1a959 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditRepository.java
>  2a47e39ed 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
>  021ca43ab 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/InMemoryEntityAuditRepository.java
>  ad6ec94c0 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/NoopEntityAuditRepository.java
>  4bb68d502 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
>  aa175a229 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
> 2ab8bdb03 
>   webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 402a323dd 
>   webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
> 43b85efc0 
> 
> 
> Diff: https://reviews.apache.org/r/72351/diff/2/
> 
> 
> Testing
> ---
> 
> Manual Testing:
> 
> curl -u admin:admin123 --location --request GET 
> 'http://sid-ycloud-atlas1-1.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/admin/audit/706e6e32-b619-43f2-8587-f2877ee27582/details?limit=100=0'
>  --header 'Content-Type: application/json'
> 
> 
> Thanks,
> 
> Sidharth Mishra
> 
>



Re: Review Request 72351: ATLAS-3727: Add get result details REST for admin audit record

2020-04-13 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72351/
---

(Updated April 13, 2020, 6:15 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3727
https://issues.apache.org/jira/browse/ATLAS-3727


Repository: atlas


Description
---

ATLAS-3727: Add get result details REST for admin audit record


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
98e56f2bb 
  intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
bcfdd9466 
  
repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
 0deb7295e 
  
repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
 35bf1d07e 
  
repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
 b8131bd39 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 4c1e1a959 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditRepository.java
 2a47e39ed 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 021ca43ab 
  
repository/src/main/java/org/apache/atlas/repository/audit/InMemoryEntityAuditRepository.java
 ad6ec94c0 
  
repository/src/main/java/org/apache/atlas/repository/audit/NoopEntityAuditRepository.java
 4bb68d502 
  
repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
 aa175a229 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
2ab8bdb03 
  webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java 402a323dd 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
43b85efc0 


Diff: https://reviews.apache.org/r/72351/diff/2/

Changes: https://reviews.apache.org/r/72351/diff/1-2/


Testing
---

Manual Testing:

curl -u admin:admin123 --location --request GET 
'http://sid-ycloud-atlas1-1.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/admin/audit/706e6e32-b619-43f2-8587-f2877ee27582/details?limit=100=0'
 --header 'Content-Type: application/json'


Thanks,

Sidharth Mishra



Re: Review Request 72351: ATLAS-3727: Add get result details REST for admin audit record

2020-04-10 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72351/
---

(Updated April 10, 2020, 10:44 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3727
https://issues.apache.org/jira/browse/ATLAS-3727


Repository: atlas


Description
---

ATLAS-3727: Add get result details REST for admin audit record


Diffs
-

  intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
98e56f2bb 
  
repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
 35bf1d07e 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 cab4e1e55 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditRepository.java
 2a47e39ed 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 021ca43ab 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
2ab8bdb03 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
43b85efc0 


Diff: https://reviews.apache.org/r/72351/diff/1/


Testing (updated)
---

Manual Testing:

curl -u admin:admin123 --location --request GET 
'http://sid-ycloud-atlas1-1.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/admin/audit/706e6e32-b619-43f2-8587-f2877ee27582/details?limit=100=0'
 --header 'Content-Type: application/json'


Thanks,

Sidharth Mishra



Review Request 72351: ATLAS-3727: Add get result details REST for admin audit record

2020-04-10 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72351/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3727
https://issues.apache.org/jira/browse/ATLAS-3727


Repository: atlas


Description
---

ATLAS-3727: Add get result details REST for admin audit record


Diffs
-

  intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
98e56f2bb 
  
repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
 35bf1d07e 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
 cab4e1e55 
  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditRepository.java
 2a47e39ed 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 021ca43ab 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
2ab8bdb03 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
43b85efc0 


Diff: https://reviews.apache.org/r/72351/diff/1/


Testing
---

Manual Testing:

curl --location --request GET 
'http://sid-ycloud-atlas1-1.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/admin/audit/706e6e32-b619-43f2-8587-f2877ee27582/details?limit=100=0'
 \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic YWRtaW46YWRtaW4xMjM='


Thanks,

Sidharth Mishra



Re: Review Request 72338: ATLAS-3726: Add result count to Atlas Admin Audit and enable filters on count for get audit

2020-04-09 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72338/
---

(Updated April 9, 2020, 6:26 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3726
https://issues.apache.org/jira/browse/ATLAS-3726


Repository: atlas


Description
---

Added a new field for result count at model, Changed the DTO and service 
accordingly. UT has been modified too. Now sort and filter will also work on 
top of result count in admin audit


Diffs (updated)
-

  addons/models/-Area0/0010-base_model.json 351e07bd3 
  intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
f2a8b3e4d 
  
repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
 590f7a034 
  
repository/src/main/java/org/apache/atlas/repository/ogm/AtlasAuditEntryDTO.java
 8e4bdb523 
  
repository/src/test/java/org/apache/atlas/repository/audit/AdminPurgeTest.java 
0a2fc39a3 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
9bc70400e 


Diff: https://reviews.apache.org/r/72338/diff/2/

Changes: https://reviews.apache.org/r/72338/diff/1-2/


Testing
---

Manual Testing:

curl --location --request POST 
'http://sid-ycloud-atlas1-1.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/admin/audits/'
 \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic YWRtaW46YWRtaW4xMjM=' \
--data-raw '{
"auditFilters": {
"condition": "AND",
"criterion": [
{
"attributeName": "userName",
"operator": "like",
"attributeValue": "admin"
},
{
"attributeName": "operation",
"operator": "like",
"attributeValue": "PURGE"
},
{
"attributeName": "clientId",
"operator": "eq",
"attributeValue": "10.96.94.236"
},
{
"attributeName": "startTime",
"operator": "gte",
"attributeValue": "1575958152162"
},
{
"attributeName": "endTime",
"operator": "gte",
"attributeValue": "1575958152184"
},
{
"attributeName": "resultCount",
"operator": "eq",
"attributeValue": "4"
}
]
},
"limit": 15,
"offset": 0,
"sortBy": "resultCount",
"sortOrder": "DESCENDING"
}'


Thanks,

Sidharth Mishra



Review Request 72338: ATLAS-3726: Add result count to Atlas Admin Audit and enable filters on count for get audit

2020-04-08 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72338/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3726
https://issues.apache.org/jira/browse/ATLAS-3726


Repository: atlas


Description
---

Added a new field for result count at model, Changed the DTO and service 
accordingly. UT has been modified too. Now sort and filter will also work on 
top of result count in admin audit


Diffs
-

  addons/models/-Area0/0010-base_model.json 351e07bd3 
  intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
f2a8b3e4d 
  
repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
 590f7a034 
  
repository/src/main/java/org/apache/atlas/repository/ogm/AtlasAuditEntryDTO.java
 8e4bdb523 
  
repository/src/test/java/org/apache/atlas/repository/audit/AdminPurgeTest.java 
0a2fc39a3 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
9bc70400e 


Diff: https://reviews.apache.org/r/72338/diff/1/


Testing
---

Manual Testing:

curl --location --request POST 
'http://sid-ycloud-atlas1-1.sid-ycloud-atlas1.root.hwx.site:31000/api/atlas/admin/audits/'
 \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic YWRtaW46YWRtaW4xMjM=' \
--data-raw '{
"auditFilters": {
"condition": "AND",
"criterion": [
{
"attributeName": "userName",
"operator": "like",
"attributeValue": "admin"
},
{
"attributeName": "operation",
"operator": "like",
"attributeValue": "PURGE"
},
{
"attributeName": "clientId",
"operator": "eq",
"attributeValue": "10.96.94.236"
},
{
"attributeName": "startTime",
"operator": "gte",
"attributeValue": "1575958152162"
},
{
"attributeName": "endTime",
"operator": "gte",
"attributeValue": "1575958152184"
},
{
"attributeName": "resultCount",
"operator": "eq",
"attributeValue": "4"
}
]
},
"limit": 15,
"offset": 0,
"sortBy": "resultCount",
"sortOrder": "DESCENDING"
}'


Thanks,

Sidharth Mishra



Re: Review Request 72317: ATLAS-3709 Issues with quick search/suggestions in conjunction with Business Metadata attributes

2020-04-07 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72317/#review220236
---


Ship it!




Ship It!

- Sidharth Mishra


On April 7, 2020, 6:20 a.m., Mandar Ambawane wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72317/
> ---
> 
> (Updated April 7, 2020, 6:20 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
> Nixon Rodrigues, Sarath Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3709
> https://issues.apache.org/jira/browse/ATLAS-3709
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Resolved issues in conjunction with Business Metadata attributes.
> Also handled index populate for Business Metadata attributes at the time of 
> start up.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/listener/ChangedTypeDefs.java 58e889a 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeRegistry.java 9df89a2 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  647e304 
> 
> 
> Diff: https://reviews.apache.org/r/72317/diff/1/
> 
> 
> Testing
> ---
> 
> Pre-commit 
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1784/console
> 
> Manual testing done.
> 
> CASE:1
> 1. create Busines meatadata "BM1" 
> 2. add one attribute "attr1" in "BM1" (search weight 9)
> 3. create one entity "hdfsentity1"
> 4. associate "attr1" to "hdfsentity1"
> 5. check in quick search and suggestions for "attr1" value
> 
> 6. add another attribute "attr2" in "BM1" (search weight 9)
> 7. associate "attr2" to "hdfsentity1"
> 8. check in quick search and suggestions
> 
> 
> CASE:2
> 1. create Busines meatadata "BM1" 
> 2. add one attribute "attr1" in "BM1" (search weight 9)
> 3. create one entity "hdfsentity1"
> 4. associate "attr1" to "hdfsentity1"
> 5. check in quick search and suggestions for "attr1" value
> 
> 6. add another attribute "attr2" in "BM1" (search weight 10)
> 7. associate "attr2" to "hdfsentity1"
> 8. check in quick search and suggestions for "attr1" and "attr2" values
> 
> 
> CASE:3
> 1. create Busines meatadata "BM1" 
> 2. add one attribute "attr1" in "BM1" (search weight 9)
> 3. create one entity "hdfsentity1"
> 4. associate "attr1" to "hdfsentity1"
> 5. check in quick search and suggestions for "attr1" value
> 
> 6. create another Busines meatadata "BM2" 
> 7. add attribute "attr2" in "BM2" (search weight 9)
> 8. associate "attr2" to "hdfsentity1"
> 9. check in quick search and suggestions for "attr1" and "attr2" values
> 
> 
> Thanks,
> 
> Mandar Ambawane
> 
>



Review Request 72328: ATLAS-3719: Tag and Type search is not doing index search

2020-04-06 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72328/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3719
https://issues.apache.org/jira/browse/ATLAS-3719


Repository: atlas


Description
---

Changed the condition to add the classification processor in case of 
Classification and Type


Diffs
-

  repository/src/main/java/org/apache/atlas/discovery/SearchContext.java 
2c2888e16 


Diff: https://reviews.apache.org/r/72328/diff/1/


Testing
---

Manual testing - 


curl --location --request POST 
'http://sid-cdp-1-1.gce.cloudera.com:31000/api/atlas/v2/search/basic' \
--header 'Content-Type: application/json' \
--data-raw 
'{"excludeDeletedEntities":true,"includeSubClassifications":true,"includeSubTypes":true,"includeClassificationAttributes":true,"entityFilters":null,"tagFilters":null,"attributes":[],"limit":100,"offset":0,"typeName":"hive_column","classification":"secu_rgpd_dcpc","termName":null}'


curl --location --request POST 
'http://sid-cdp-1-1.gce.cloudera.com:31000/api/atlas/v2/search/basic' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic YWRtaW46YWRtaW4=' \
--data-raw 
'{"excludeDeletedEntities":true,"includeSubClassifications":true,"includeSubTypes":true,"includeClassificationAttributes":true,"limit":25,"offset":0,"typeName":null,"classification":"BEFORE_DEMO","termName":null}'


Thanks,

Sidharth Mishra



Re: Review Request 72284: ATLAS-3707: Atlas Purge entity is generating both purge and delete audit

2020-03-30 Thread Sidharth Mishra


> On March 30, 2020, 7:25 p.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
> > Lines 325 (patched)
> > <https://reviews.apache.org/r/72284/diff/1/?file=2216465#file2216465line325>
> >
> > In case of purge, shouldn't these entities be not added to resp as 
> > PURGEed entities? Like:
> > 
> >   EntityOperation deleteOperation = req.isPurgeRequested() ? PURGE : 
> > DELETE;
> > 
> >   for (AtlasEntityHeader entity : req.getDeletedEntities()) {
> > resp.addEntity(deleteOperation, entity);
> >   }
> > 
> > Alternatively, consider adding RequestContext.purgedEntities and add to 
> > this collection (instead of deletedEntities) while performing purge - from 
> > DeleteHandlerV1.deleteEntities():
> > 
> > for (GraphHelper.VertexInfo vertexInfo : 
> > getOwnedVertices(instanceVertex)) {
> > if (requestContext.isPurgeRequested()) {
> > requestContext.recordEntityPurge(vertexInfo.getEntity());
> > } else {
> > requestContext.recordEntityDelete(vertexInfo.getEntity());
> > }
> > 
> > deletionCandidateVertices.add(vertexInfo.getVertex());
> > }

At org/apache/atlas/web/resources/AdminResource.java:473, 
EntityMutationResponse resp =  entityStore.purgeByIds(guids); keeps the deleted 
entities in requestContext and we get the same and assign it to response as you 
correctly pointed above.
After that we again create atlas audit entity at 
org/apache/atlas/web/resources/AdminResource.java:480 which eventually calls 
org.apache.atlas.repository.store.graph.v2.AtlasEntityStoreV2#createOrUpdate(org.apache.atlas.repository.store.graph.v2.EntityStream,
 boolean, boolean, boolean) where we see if there are deleted entities at 
request context then we generate the audit for those which we need to stop by 
adding the check. To keep the hard delete and purge code same we are still 
using the requestContext deletedEntities instead of having separate one for 
purge.


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72284/#review220121
---


On March 30, 2020, 7:03 p.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72284/
> ---
> 
> (Updated March 30, 2020, 7:03 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3707
> https://issues.apache.org/jira/browse/ATLAS-3707
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> The check is been added, so that when its purge operation there will not be 
> any entity delete notification sent to audit
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
>  75b016cca 
> 
> 
> Diff: https://reviews.apache.org/r/72284/diff/1/
> 
> 
> Testing
> ---
> 
> Manual testing. 
> 
> curl --location --request GET 
> 'http://sid-cdp-3-1.gce.cloudera.com:31000/api/atlas/v2/entity/d7d29f3e-1ef6-4baa-b66c-3f20d2839036/audit'
>  \
> --header 'Authorization: Basic YWRtaW46YWRtaW4='
> 
> 
> Thanks,
> 
> Sidharth Mishra
> 
>



Re: Review Request 72265: ATLAS-3684: Atlas glossary get is taking more time when number of terms increase

2020-03-30 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72265/
---

(Updated March 30, 2020, 7:05 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3684
https://issues.apache.org/jira/browse/ATLAS-3684


Repository: atlas


Description
---

ATLAS-3684: Atlas glossary get is taking more time when number of terms increase


Diffs
-

  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
d630f669e 
  
repository/src/main/java/org/apache/atlas/repository/ogm/glossary/AbstractGlossaryDTO.java
 bee88c61f 


Diff: https://reviews.apache.org/r/72265/diff/1/


Testing
---

Manually tested


Thanks,

Sidharth Mishra



Review Request 72284: ATLAS-3707: Atlas Purge entity is generating both purge and delete audit

2020-03-30 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72284/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-3707
https://issues.apache.org/jira/browse/ATLAS-3707


Repository: atlas


Description
---

The check is been added, so that when its purge operation there will not be any 
entity delete notification sent to audit


Diffs
-

  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
 75b016cca 


Diff: https://reviews.apache.org/r/72284/diff/1/


Testing
---

Manual testing. 

curl --location --request GET 
'http://sid-cdp-3-1.gce.cloudera.com:31000/api/atlas/v2/entity/d7d29f3e-1ef6-4baa-b66c-3f20d2839036/audit'
 \
--header 'Authorization: Basic YWRtaW46YWRtaW4='


Thanks,

Sidharth Mishra



Re: Review Request 72268: ATLAS-3688: updated dependencies to remove unnecessary references

2020-03-25 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72268/#review220081
---


Ship it!




Ship It!

- Sidharth Mishra


On March 25, 2020, 7:35 p.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72268/
> ---
> 
> (Updated March 25, 2020, 7:35 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sarath Subramanian, and Sidharth 
> Mishra.
> 
> 
> Bugs: ATLAS-3688
> https://issues.apache.org/jira/browse/ATLAS-3688
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Unnecessary references to hdfs-model, hbase-common and hbase-server were 
> removed from modules
> 
> 
> Diffs
> -
> 
>   addons/falcon-bridge/pom.xml c53d0ef8b 
>   addons/hbase-bridge/pom.xml ff88c8b1e 
>   addons/hive-bridge/pom.xml 13384cb72 
>   addons/impala-bridge/pom.xml a684dd03d 
>   addons/sqoop-bridge/pom.xml 0d89e98dc 
>   addons/storm-bridge/pom.xml c282625bc 
>   pom.xml 7e9c7e06a 
>   tools/classification-updater/pom.xml 4adfb3d04 
> 
> 
> Diff: https://reviews.apache.org/r/72268/diff/1/
> 
> 
> Testing
> ---
> 
> Pre-commit tests run: 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1758/
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>



Review Request 72265: ATLAS-3684: Atlas glossary get is taking more time when number of terms increase

2020-03-24 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72265/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Repository: atlas


Description
---

ATLAS-3684: Atlas glossary get is taking more time when number of terms increase


Diffs
-

  repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
d630f669e 
  
repository/src/main/java/org/apache/atlas/repository/ogm/glossary/AbstractGlossaryDTO.java
 bee88c61f 


Diff: https://reviews.apache.org/r/72265/diff/1/


Testing
---

Manually tested


Thanks,

Sidharth Mishra



Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

2020-03-20 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review220048
---


Ship it!




Ship It!

- Sidharth Mishra


On March 20, 2020, 2:06 p.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 20, 2020, 2:06 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/import/template
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/atlas/v2/glossary/import
> 
> 
> Note:-
> 
> While populating the data in the  csv file each record should be maintained 
> in single Line (enter command within the record would result in parsing the 
> second line as a new record).
> 
> The downloaded template needs to be saved as whateverTheFileNameIs.csv 
> explicitly.
> 
> If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
> returned or else List of Errors would returned for user to rectify them 
> further.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 1bd2fd0 
>   
> intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
>  660514b 
>   pom.xml a6da9b8 
>   repository/pom.xml 802d587 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
> 9229d2d 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
> cdc3f07 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 
> 9625f94 
>   repository/src/main/java/org/apache/atlas/util/FileUtils.java PRE-CREATION 
>   repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
> 759dcdf 
>   repository/src/test/resources/csvFiles/empty.csv PRE-CREATION 
>   repository/src/test/resources/csvFiles/incorrectFile.csv PRE-CREATION 
>   repository/src/test/resources/csvFiles/template_1.csv PRE-CREATION 
>   repository/src/test/resources/excelFiles/template_1.xlsx PRE-CREATION 
>   webapp/src/main/java/org/apache/atlas/web/rest/GlossaryREST.java 151aa6b 
> 
> 
> Diff: https://reviews.apache.org/r/71482/diff/20/
> 
> 
> Testing
> ---
> 
> Tested both the endpoint with curl call.
> 
> Tested upload term csv with around 100 records from curl.
> 
> 
> Thanks,
> 
> mayank jain
> 
>



Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

2020-03-19 Thread Sidharth Mishra


> On March 18, 2020, 11:55 p.m., Sidharth Mishra wrote:
> > repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
> > Lines 570 (patched)
> > <https://reviews.apache.org/r/71482/diff/18/?file=2213646#file2213646line570>
> >
> > Please remove these kind of extra blank lines from here as well all 
> > other places

I can still see the extra blank lines other places. Kindly let me know if there 
is any specific reason you add them?


> On March 18, 2020, 11:55 p.m., Sidharth Mishra wrote:
> > repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
> > Lines 690 (patched)
> > <https://reviews.apache.org/r/71482/diff/18/?file=2213646#file2213646line690>
> >
> > Instead of \n use System.lineSeparator(). Please do this at all other 
> > places in entire patch

Please mention the reason for drop.


> On March 18, 2020, 11:55 p.m., Sidharth Mishra wrote:
> > repository/src/main/java/org/apache/atlas/util/FileUtils.java
> > Lines 99 (patched)
> > <https://reviews.apache.org/r/71482/diff/18/?file=2213648#file2213648line99>
> >
> > Try to create a const for 0 explaining what does the position contains

Please mention the reason for drop.


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review219988
---


On March 19, 2020, 5:38 a.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 19, 2020, 5:38 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/import/template
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/atlas/v2/glossary/import
> 
> 
> Note:-
> 
> While populating the data in the  csv file each record should be maintained 
> in single Line (enter command within the record would result in parsing the 
> second line as a new record).
> 
> The downloaded template needs to be saved as whateverTheFileNameIs.csv 
> explicitly.
> 
> If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
> returned or else List of Errors would returned for user to rectify them 
> further.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 1bd2fd0 
>   
> intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
>  660514b 
>   pom.xml a6da9b8 
>   repository/pom.xml 802d587 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java 
> 9229d2d 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java 
> cdc3f07 
>   repository/src/main/java/org/apache/atlas/glossary/GlossaryUtils.java 
> 962

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

2020-03-18 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review219988
---




repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 570 (patched)
<https://reviews.apache.org/r/71482/#comment308221>

Please remove these kind of extra blank lines from here as well all other 
places



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 649 (patched)
<https://reviews.apache.org/r/71482/#comment308223>

Do something like:

Set ret = null;

if (StringUtils.isNotBlank(csvRecord)) {
  ...
  ret = new HashSet();
  ...
}

return ret;



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 672 (patched)
<https://reviews.apache.org/r/71482/#comment308224>

there is on extra space after system msg and then you are again doing + " 
". Please remove one. Same is there at other places. please take care of this 
at all other locations



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 682 (patched)
<https://reviews.apache.org/r/71482/#comment308222>

you are already passing atlasGlossaryTerm as input to this function where 
you are using set of this object and then retunring the same object 
atlasGlossaryTerm and assigning it to same atlasGlossaryTerm at 573. Please 
revisit the design and rename the function accordingly. The function should 
always do one thing and should be named accordingly. Depending on that you 
should design the input and output arguments



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 690 (patched)
<https://reviews.apache.org/r/71482/#comment308225>

Instead of \n use System.lineSeparator(). Please do this at all other 
places in entire patch



repository/src/main/java/org/apache/atlas/util/FileUtils.java
Lines 50 (patched)
<https://reviews.apache.org/r/71482/#comment308226>

Please call readCSV function directly instead of using class name as its in 
the same class



repository/src/main/java/org/apache/atlas/util/FileUtils.java
Lines 99 (patched)
<https://reviews.apache.org/r/71482/#comment308227>

Try to create a const for 0 explaining what does the position contains



repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java
Lines 960 (patched)
<https://reviews.apache.org/r/71482/#comment308228>

Please remove the extra space:

assertEquals(e.getMessage(), "No Data found in the uploaded file!");


- Sidharth Mishra


On March 13, 2020, 1 p.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 13, 2020, 1 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/importHeaderRow
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/a

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

2020-03-12 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review219912
---




repository/src/main/java/org/apache/atlas/glossary/GlossaryService.java
Lines 1124 (patched)
<https://reviews.apache.org/r/71482/#comment308149>

Remove extra blank line



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 549 (patched)
<https://reviews.apache.org/r/71482/#comment308151>

Please remove the extra blank lines like this even other places.



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 558 (patched)
<https://reviews.apache.org/r/71482/#comment308150>

Remove the extra blank line


- Sidharth Mishra


On March 12, 2020, 10:37 a.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 12, 2020, 10:37 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, keval bhatt, Madhan Neethiraj, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/importHeaderRow
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/atlas/v2/glossary/import
> 
> 
> Note:-
> 
> While populating the data in the  csv file each record should be maintained 
> in single Line (enter command within the record would result in parsing the 
> second line as a new record).
> 
> The downloaded template needs to be saved as whateverTheFileNameIs.csv 
> explicitly.
> 
> If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
> returned or else List of Errors would returned for user to rectify them 
> further.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd56 
>   dashboardv2/gruntfile.js fef4e08 
>   dashboardv2/package-lock.json 7f25b57 
>   dashboardv2/package.json e90040e 
>   dashboardv2/public/css/scss/theme.scss aad0dfa 
>   dashboardv2/public/index.html.tpl 7cd86bd 
>   dashboardv2/public/js/main.js 41af465 
>   dashboardv2/public/js/templates/glossary/GlossaryLayoutView_tmpl.html 
> 1fa1e35 
>   dashboardv2/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
> PRE-CREATION 
>   dashboardv2/public/js/utils/UrlLinks.js 6c67e8c 
>   dashboardv2/public/js/views/glossary/GlossaryLayoutView.js 9b386f3 
>   dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js 
> PRE-CREATION 
>   dashboardv3/gruntfile.js f55ff0d 
>   dashboardv3/package-lock.json 3918ecc 
>   dashboardv3/package.json 5dc0510 
>   dashboardv3/public/css/scss/leftsidebar.scss bbdc5fb 
>   dashboardv3/public/css/scss/theme.scss 2b0c45d 
>   dashboardv3/public/index.html.tpl 1baeb12 
>   dashboardv3/public/js/main.js 96ef7ae 
>   dashboardv3/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
> PRE-CREATION 
>   
> dashboardv3/public/js/templates/search/tree/GlossaryTreeLayoutView_tmpl.html 
> 83da9

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

2020-03-05 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review219788
---



- Sidharth Mishra


On March 4, 2020, 10:07 a.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 4, 2020, 10:07 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/importHeaderRow
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/atlas/v2/glossary/import
> 
> 
> Note:-
> 
> While populating the data in the  csv file each record should be maintained 
> in single Line (enter command within the record would result in parsing the 
> second line as a new record).
> 
> The downloaded template needs to be saved as whateverTheFileNameIs.csv 
> explicitly.
> 
> If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
> returned or else List of Errors would returned for user to rectify them 
> further.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd5601 
>   dashboardv2/gruntfile.js fef4e08c3 
>   dashboardv2/package-lock.json 7f25b5752 
>   dashboardv2/package.json e90040edb 
>   dashboardv2/public/css/scss/theme.scss 0589e0920 
>   dashboardv2/public/index.html.tpl a6a999e53 
>   dashboardv2/public/js/main.js 75e16c3aa 
>   dashboardv2/public/js/templates/glossary/GlossaryLayoutView_tmpl.html 
> 1fa1e3540 
>   dashboardv2/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
> PRE-CREATION 
>   dashboardv2/public/js/utils/UrlLinks.js 6c67e8c37 
>   dashboardv2/public/js/views/glossary/GlossaryLayoutView.js 9b386f326 
>   dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js 
> PRE-CREATION 
>   dashboardv3/gruntfile.js f55ff0d5e 
>   dashboardv3/package-lock.json 3918eccaa 
>   dashboardv3/package.json 5dc05104f 
>   dashboardv3/public/css/scss/leftsidebar.scss bbdc5fb26 
>   dashboardv3/public/css/scss/theme.scss 2b0c45d6b 
>   dashboardv3/public/index.html.tpl 2edbb659d 
>   dashboardv3/public/js/main.js 26fd70991 
>   dashboardv3/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
> PRE-CREATION 
>   
> dashboardv3/public/js/templates/search/tree/GlossaryTreeLayoutView_tmpl.html 
> 83da9c57c 
>   dashboardv3/public/js/utils/UrlLinks.js 2bbe6796f 
>   dashboardv3/public/js/views/glossary/ImportGlossaryLayoutView.js 
> PRE-CREATION 
>   dashboardv3/public/js/views/search/tree/GlossaryTreeLayoutView.js 28c6a9e4a 
>   intg/src/main/java/org/apache/atlas/AtlasConfiguration.java c5bf50dca 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 04eb4a08e 
>   
> intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
>  660514bc2 
>   pom.xml f76c6a05e 
>   repository/pom.xml 802d587a8 
>  

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

2020-03-05 Thread Sidharth Mishra
id out argument



repository/src/main/java/org/apache/atlas/glossary/GlossaryTermUtils.java
Lines 676 (patched)
<https://reviews.apache.org/r/71482/#comment308010>

Please take care of indentation at most of the places in your patchl. Like 
space before and after '+' etc.


- Sidharth Mishra


On March 4, 2020, 10:07 a.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 4, 2020, 10:07 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/importHeaderRow
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/atlas/v2/glossary/import
> 
> 
> Note:-
> 
> While populating the data in the  csv file each record should be maintained 
> in single Line (enter command within the record would result in parsing the 
> second line as a new record).
> 
> The downloaded template needs to be saved as whateverTheFileNameIs.csv 
> explicitly.
> 
> If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
> returned or else List of Errors would returned for user to rectify them 
> further.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd5601 
>   dashboardv2/gruntfile.js fef4e08c3 
>   dashboardv2/package-lock.json 7f25b5752 
>   dashboardv2/package.json e90040edb 
>   dashboardv2/public/css/scss/theme.scss 0589e0920 
>   dashboardv2/public/index.html.tpl a6a999e53 
>   dashboardv2/public/js/main.js 75e16c3aa 
>   dashboardv2/public/js/templates/glossary/GlossaryLayoutView_tmpl.html 
> 1fa1e3540 
>   dashboardv2/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
> PRE-CREATION 
>   dashboardv2/public/js/utils/UrlLinks.js 6c67e8c37 
>   dashboardv2/public/js/views/glossary/GlossaryLayoutView.js 9b386f326 
>   dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js 
> PRE-CREATION 
>   dashboardv3/gruntfile.js f55ff0d5e 
>   dashboardv3/package-lock.json 3918eccaa 
>   dashboardv3/package.json 5dc05104f 
>   dashboardv3/public/css/scss/leftsidebar.scss bbdc5fb26 
>   dashboardv3/public/css/scss/theme.scss 2b0c45d6b 
>   dashboardv3/public/index.html.tpl 2edbb659d 
>   dashboardv3/public/js/main.js 26fd70991 
>   dashboardv3/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
> PRE-CREATION 
>   
> dashboardv3/public/js/templates/search/tree/GlossaryTreeLayoutView_tmpl.html 
> 83da9c57c 
>   dashboardv3/public/js/utils/UrlLinks.js 2bbe6796f 
>   dashboardv3/public/js/views/glossary/ImportGlossaryLayoutView.js 
> PRE-CREATION 
>   dashboardv3/public/js/views/search/tree/GlossaryTreeLayoutView.js 28c6a9e4a 
>   intg/src/main/java/org/apache/atlas/AtlasConfiguration.java c5bf50dca 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 04eb4a08e 
>   
> intg/src/main/java/org/apache/atlas/model/glossary/relations/AtlasGlossaryHeader.java
>  660514bc2 
>   pom.xml f76c6a05e 

Re: Review Request 71482: ATLAS-3423:-Import Glossary Terms CSV into a Glossary

2020-03-05 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71482/#review219786
---




common/src/main/java/org/apache/atlas/repository/Constants.java
Lines 205 (patched)
<https://reviews.apache.org/r/71482/#comment307986>

Please provide comments which will be useful to get some more context or 
remove the comment. In this case the variable name is more informative than the 
comment.



common/src/main/java/org/apache/atlas/repository/Constants.java
Lines 207 (patched)
<https://reviews.apache.org/r/71482/#comment307985>

Please consider renaming GlossaryImportSupportedFileFormats to 
GlossaryImportSupportedFileExtensions as you have used extensions at other 
places and comments



dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js
Lines 53 (patched)
<https://reviews.apache.org/r/71482/#comment307987>

This comment is not adding any useful information as the initialize block 
is self explanatory.


- Sidharth Mishra


On March 4, 2020, 10:07 a.m., mayank jain wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71482/
> ---
> 
> (Updated March 4, 2020, 10:07 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3423
> https://issues.apache.org/jira/browse/ATLAS-3423
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch consists implementation for 2 end points first for template 
> download and other for csv file upload with term details also the Unit Test 
> cases for both the end points.
> 
> * The 1st endpoint {glossary/template} return template file this would be 
> type of format of data that shows how the data needs to be populated by user 
> in the file.
> 
> http://localhost:21000/api/atlas/v2/glossary/importHeaderRow
> 
> Template structure:-
> 
> GlossaryName, TermName, ShortDescription, LongDescription, Examples, 
> Abbreviation, Usage, AdditionalAttributes, TranslationTerms, ValidValuesFor, 
> Synonyms, ReplacedBy, ValidValues, ReplacementTerms, SeeAlso, 
> TranslatedTerms, IsA, Antonyms, Classifies, PreferredToTerms, PreferredTerms
> Fruits,Apple5,SD4,LD4,"EXAMPLE","ABBREVIATION","USAGE",,"Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4","Footwear:B4"
> 
> 
> * The 2nd endpoint {glossary/importGlossaryData} (file upload) would actually 
> parse the Data into AtlasObjects and further create the AtlasGlossaryTerms 
> inside Glossary.
> 
> curl -v -g POST -u admin:admin -H "Content-Type: multipart/form-data" -H 
> "Cache-Control: no-cache" -F file=@template_6.csv 
> "http://localhost:21000/api/atlas/v2/glossary/import
> 
> 
> Note:-
> 
> While populating the data in the  csv file each record should be maintained 
> in single Line (enter command within the record would result in parsing the 
> second line as a new record).
> 
> The downloaded template needs to be saved as whateverTheFileNameIs.csv 
> explicitly.
> 
> If the file is been succefully uploaded then the AtlasGlossaryTerm would be 
> returned or else List of Errors would returned for user to rectify them 
> further.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 7c0fd5601 
>   dashboardv2/gruntfile.js fef4e08c3 
>   dashboardv2/package-lock.json 7f25b5752 
>   dashboardv2/package.json e90040edb 
>   dashboardv2/public/css/scss/theme.scss 0589e0920 
>   dashboardv2/public/index.html.tpl a6a999e53 
>   dashboardv2/public/js/main.js 75e16c3aa 
>   dashboardv2/public/js/templates/glossary/GlossaryLayoutView_tmpl.html 
> 1fa1e3540 
>   dashboardv2/public/js/templates/glossary/ImportGlossaryLayoutView_tmpl.html 
> PRE-CREATION 
>   dashboardv2/public/js/utils/UrlLinks.js 6c67e8c37 
>   dashboardv2/public/js/views/glossary/GlossaryLayoutView.js 9b386f326 
>   dashboardv2/public/js/views/glossary/ImportGlossaryLayoutView.js 
> PRE-CREATION 
>   dashboardv3/gruntfile.js f55ff0d5e 
>   dashboardv3/package-lock.json 3918eccaa 
>   dashboardv3/package.json 5dc05104f 
>   dashboardv3/public/css/scss/leftsidebar.scss bbdc5fb26 
>   dashboardv3/public/css/scss/theme.scss 2b0c45d6b 
>  

Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2020-03-02 Thread Sidharth Mishra
-search-parameter-purge.json
 PRE-CREATION 
  
repository/src/test/resources/json/auditSearchParameters/audit-search-parameter-without-filter.json
 PRE-CREATION 
  server-api/src/main/java/org/apache/atlas/RequestContext.java f9ca7a26d 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
40fd6efdc 
  webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 
65a9aa328 
  
webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
 a4b7b0d49 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
563a16f64 
  webapp/src/test/resources/json/audit-search-parameter-purge.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71791/diff/18/

Changes: https://reviews.apache.org/r/71791/diff/17-18/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2020-03-02 Thread Sidharth Mishra
-search-parameter-purge.json
 PRE-CREATION 
  
repository/src/test/resources/json/auditSearchParameters/audit-search-parameter-without-filter.json
 PRE-CREATION 
  server-api/src/main/java/org/apache/atlas/RequestContext.java f9ca7a26d 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
40fd6efdc 
  webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 
65a9aa328 
  
webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
 a4b7b0d49 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
563a16f64 
  webapp/src/test/resources/json/audit-search-parameter-purge.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71791/diff/17/

Changes: https://reviews.apache.org/r/71791/diff/16-17/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2020-02-26 Thread Sidharth Mishra
-search-parameter-purge.json
 PRE-CREATION 
  
repository/src/test/resources/json/auditSearchParameters/audit-search-parameter-without-filter.json
 PRE-CREATION 
  server-api/src/main/java/org/apache/atlas/RequestContext.java f9ca7a26d 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
40fd6efdc 
  webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 
65a9aa328 
  
webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
 a4b7b0d49 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
563a16f64 
  webapp/src/test/resources/json/audit-search-parameter-purge.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71791/diff/16/

Changes: https://reviews.apache.org/r/71791/diff/15-16/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2020-02-25 Thread Sidharth Mishra
-search-parameter-purge.json
 PRE-CREATION 
  
repository/src/test/resources/json/auditSearchParameters/audit-search-parameter-without-filter.json
 PRE-CREATION 
  server-api/src/main/java/org/apache/atlas/RequestContext.java f9ca7a26d 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
40fd6efdc 
  webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 
65a9aa328 
  
webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
 a4b7b0d49 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
563a16f64 
  webapp/src/test/resources/json/audit-search-parameter-purge.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71791/diff/15/

Changes: https://reviews.apache.org/r/71791/diff/14-15/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2020-01-27 Thread Sidharth Mishra
-search-parameter-purge.json
 PRE-CREATION 
  
repository/src/test/resources/json/auditSearchParameters/audit-search-parameter-without-filter.json
 PRE-CREATION 
  server-api/src/main/java/org/apache/atlas/RequestContext.java f9ca7a26d 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
40fd6efdc 
  webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 
65a9aa328 
  
webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
 a4b7b0d49 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
563a16f64 
  webapp/src/test/resources/json/audit-search-parameter-purge.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71791/diff/14/

Changes: https://reviews.apache.org/r/71791/diff/13-14/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2020-01-27 Thread Sidharth Mishra


> On Jan. 22, 2020, 10:09 p.m., Sarath Subramanian wrote:
> > addons/models/-Area0/0010-base_model.json
> > Lines 4 (patched)
> > <https://reviews.apache.org/r/71791/diff/13/?file=2183621#file2183621line4>
> >
> > if this enum is used only for aud consider renaming to 
> > atlas_audit_operation

Initially it was atlas_audit_operation, but later changed to atlas_operation as 
suggested by Madhan


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71791/#review219365
---


On Dec. 11, 2019, 1:17 a.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71791/
> ---
> 
> (Updated Dec. 11, 2019, 1:17 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3518
> https://issues.apache.org/jira/browse/ATLAS-3518
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Right now for purge, delete entity, etc. at Atlas we create audit entry at 
> HBase. User can go to entity and click on audit to see the audit information 
> for that particular entity. But if user purge one entity there will not be 
> any vertex at Janus graph and user will not able to get the purge audit which 
> is important for governance purpose.
> 
> As part of fix, implemented Atlas Audit Framework which is being used for 
> Purge operation and can be extended for other operations. In Future once we 
> change the AtlasAudit type as super type for Import export as well we can use 
> this framework for all Atlas operations. 
> 
> admin/audit rest has been added which will return all the Atlas audit 
> operations including import export at Atlas.
> 
> 
> Diffs
> -
> 
>   addons/models/-Area0/0010-base_model.json 2f5fdaf14 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 8c0a6407c 
>   dashboardv2/public/js/utils/Enums.js a7de0d451 
>   dashboardv3/public/js/utils/Enums.js a7de0d451 
>   intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/audit/AuditSearchParameters.java 
> PRE-CREATION 
>   
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
>  b448d510b 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/ogm/AtlasAuditEntryDTO.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  c9ed79750 
>   repository/src/test/java/org/apache/atlas/TestModules.java 94b08a187 
>   repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
> be1698fb4 
>   repository/src/test/java/org/apache/atlas/query/BasicTestSetup.java 
> 02f78b369 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AdminPurgeTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AtlasAuditServiceTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/AtlasServerServiceTest.java
>  78865c569 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportImportAuditServiceTest.java
>  ba7a8a04b 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportImportTestBase.java
>  925b2a007 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportIncrementalTest.java
>  4d438525a 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportServiceTest.java
>  db8c6c83c 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportSkipLineageTest.java
>  25e0a5362 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/HdfsPathEntityCreatorTest.java
>  1863b8d74 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportReactivateTableTest.java
>  d0c06a107 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
>  c14850f43 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportTransformsShaperTest.java
>  78fdaca8d 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/IncrementalExportEntityProviderTest.java
>  42b63535b 
>   
> repository/src/test/java/org/apache/atlas/repositor

Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2020-01-27 Thread Sidharth Mishra


> On Jan. 22, 2020, 10:09 p.m., Sarath Subramanian wrote:
> > client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java
> > Lines 495 (patched)
> > <https://reviews.apache.org/r/71791/diff/13/?file=2183622#file2183622line496>
> >
> > why return type is ArrayNode.class? can we not have return type to be - 
> > List ? if ti works we don't need ExtractOperation() and 
> > ExtractResults()

It didn't work for me. Please let me know if you have any idea which will 
return List directly that way we can really get read of the 
two functions


> On Jan. 22, 2020, 10:09 p.m., Sarath Subramanian wrote:
> > intg/src/main/java/org/apache/atlas/model/audit/AuditSearchParameters.java
> > Lines 36 (patched)
> > <https://reviews.apache.org/r/71791/diff/13/?file=2183626#file2183626line36>
> >
> > AuditSearchParameters class looks like a trimmed down version of 
> > SearchParameters class. Consider using SearchParameters instead and removeA 
> > uditSearchParameters class.

As we initially decided this way user will have less flexibility and will 
understand what is required to be passed for audit filter not the wider 
SearchParameters


> On Jan. 22, 2020, 10:09 p.m., Sarath Subramanian wrote:
> > repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
> > Line 85 (original), 86 (patched)
> > <https://reviews.apache.org/r/71791/diff/13/?file=2183644#file2183644line86>
> >
> > why ImportServiceTest now need to extend from AtlasAuditTestBase? 
> > AtlasAuditTestBase should be specific to only atlas sudit tests. Move 
> > common methods to a new base class - AtlasBaseTest.
> > 
> > Please review all tests which is modified now to extend from 
> > 'AtlasAuditTestBase'

Moved to AtlasTestBase and all the impexp test and audit test are now inherited 
from this. The only function assertExportImportAuditEntry is part of this which 
I might have moved to another inherited class. But we have plan to unify the 
imp/exp audit soon, will take care later


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71791/#review219365
---


On Dec. 11, 2019, 1:17 a.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71791/
> ---
> 
> (Updated Dec. 11, 2019, 1:17 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3518
> https://issues.apache.org/jira/browse/ATLAS-3518
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Right now for purge, delete entity, etc. at Atlas we create audit entry at 
> HBase. User can go to entity and click on audit to see the audit information 
> for that particular entity. But if user purge one entity there will not be 
> any vertex at Janus graph and user will not able to get the purge audit which 
> is important for governance purpose.
> 
> As part of fix, implemented Atlas Audit Framework which is being used for 
> Purge operation and can be extended for other operations. In Future once we 
> change the AtlasAudit type as super type for Import export as well we can use 
> this framework for all Atlas operations. 
> 
> admin/audit rest has been added which will return all the Atlas audit 
> operations including import export at Atlas.
> 
> 
> Diffs
> -
> 
>   addons/models/-Area0/0010-base_model.json 2f5fdaf14 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 8c0a6407c 
>   dashboardv2/public/js/utils/Enums.js a7de0d451 
>   dashboardv3/public/js/utils/Enums.js a7de0d451 
>   intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
> PRE-CREATION 
>   intg/src/main/java/org/apache/atlas/model/audit/AuditSearchParameters.java 
> PRE-CREATION 
>   
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
>  b448d510b 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/ogm/AtlasAuditEntryDTO.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  c9ed79750 
>   repository/src/test/java/org/apache/atlas/TestModules.java 94b08a187 
>   repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
> be1698fb4 

Re: Review Request 71922: ATLAS-3564: New version of AWS S3 model addition

2020-01-13 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71922/
---

(Updated Jan. 13, 2020, 8:06 p.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Madhan Neethiraj, and 
Sarath Subramanian.


Bugs: ATLAS-3564
https://issues.apache.org/jira/browse/ATLAS-3564


Repository: atlas


Description
---

This is the new version) of aws s3 model at atlas to allow the hierarchical 
structure and attributes same as AWS S3 Console. The existing aws s3 models has 
limitations like pseudo directory not containing another pseudo directory, 
pseudo directory doesn't have attributes of aws object (as some case it gets 
populated), both object and bucket doesn't containing all the attributes of AWS 
S3 etc.


Diffs (updated)
-

  addons/models/3000-Cloud/3030-aws_s3_v2_typedefs.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71922/diff/7/

Changes: https://reviews.apache.org/r/71922/diff/6-7/


Testing
---


Thanks,

Sidharth Mishra



Re: Review Request 71922: ATLAS-3564: New version of AWS S3 model addition

2020-01-12 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71922/
---

(Updated Jan. 13, 2020, 5:40 a.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Madhan Neethiraj, and 
Sarath Subramanian.


Bugs: ATLAS-3564
https://issues.apache.org/jira/browse/ATLAS-3564


Repository: atlas


Description
---

This is the new version) of aws s3 model at atlas to allow the hierarchical 
structure and attributes same as AWS S3 Console. The existing aws s3 models has 
limitations like pseudo directory not containing another pseudo directory, 
pseudo directory doesn't have attributes of aws object (as some case it gets 
populated), both object and bucket doesn't containing all the attributes of AWS 
S3 etc.


Diffs (updated)
-

  addons/models/3000-Cloud/3030-aws_s3_v2_typedefs.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71922/diff/6/

Changes: https://reviews.apache.org/r/71922/diff/5-6/


Testing
---


Thanks,

Sidharth Mishra



Re: Review Request 71922: ATLAS-3564: New version of AWS S3 model addition

2020-01-10 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71922/
---

(Updated Jan. 11, 2020, 1:50 a.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Madhan Neethiraj, and 
Sarath Subramanian.


Bugs: ATLAS-3564
https://issues.apache.org/jira/browse/ATLAS-3564


Repository: atlas


Description
---

This is the new version) of aws s3 model at atlas to allow the hierarchical 
structure and attributes same as AWS S3 Console. The existing aws s3 models has 
limitations like pseudo directory not containing another pseudo directory, 
pseudo directory doesn't have attributes of aws object (as some case it gets 
populated), both object and bucket doesn't containing all the attributes of AWS 
S3 etc.


Diffs (updated)
-

  addons/models/3000-Cloud/3030-aws_s3_v2_typedefs.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71922/diff/5/

Changes: https://reviews.apache.org/r/71922/diff/4-5/


Testing
---


Thanks,

Sidharth Mishra



Re: Review Request 71922: ATLAS-3564: New version of AWS S3 model addition

2020-01-10 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71922/
---

(Updated Jan. 10, 2020, 7:13 p.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Madhan Neethiraj, and 
Sarath Subramanian.


Bugs: ATLAS-3564
https://issues.apache.org/jira/browse/ATLAS-3564


Repository: atlas


Description
---

This is the new version) of aws s3 model at atlas to allow the hierarchical 
structure and attributes same as AWS S3 Console. The existing aws s3 models has 
limitations like pseudo directory not containing another pseudo directory, 
pseudo directory doesn't have attributes of aws object (as some case it gets 
populated), both object and bucket doesn't containing all the attributes of AWS 
S3 etc.


Diffs (updated)
-

  addons/models/3000-Cloud/3030-aws_s3_typedefs_v2.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71922/diff/4/

Changes: https://reviews.apache.org/r/71922/diff/3-4/


Testing
---


Thanks,

Sidharth Mishra



Re: Review Request 71922: ATLAS-3564: New version of AWS S3 model addition

2020-01-10 Thread Sidharth Mishra
ud/3030-aws_s3_typedefs_v2.json
> > Lines 238 (patched)
> > <https://reviews.apache.org/r/71922/diff/3/?file=2199453#file2199453line238>
> >
> > This should be part of base class.

same as above.


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71922/#review219215
---


On Jan. 8, 2020, 11:12 p.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71922/
> ---
> 
> (Updated Jan. 8, 2020, 11:12 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Sridhar K, Madhan Neethiraj, and 
> Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3564
> https://issues.apache.org/jira/browse/ATLAS-3564
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This is the new version) of aws s3 model at atlas to allow the hierarchical 
> structure and attributes same as AWS S3 Console. The existing aws s3 models 
> has limitations like pseudo directory not containing another pseudo 
> directory, pseudo directory doesn't have attributes of aws object (as some 
> case it gets populated), both object and bucket doesn't containing all the 
> attributes of AWS S3 etc.
> 
> 
> Diffs
> -
> 
>   addons/models/3000-Cloud/3030-aws_s3_typedefs_v2.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/71922/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sidharth Mishra
> 
>



Re: Review Request 71922: ATLAS-3564: New version of AWS S3 model addition

2020-01-08 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71922/
---

(Updated Jan. 8, 2020, 11:12 p.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Madhan Neethiraj, and 
Sarath Subramanian.


Bugs: ATLAS-3564
https://issues.apache.org/jira/browse/ATLAS-3564


Repository: atlas


Description
---

This is the new version) of aws s3 model at atlas to allow the hierarchical 
structure and attributes same as AWS S3 Console. The existing aws s3 models has 
limitations like pseudo directory not containing another pseudo directory, 
pseudo directory doesn't have attributes of aws object (as some case it gets 
populated), both object and bucket doesn't containing all the attributes of AWS 
S3 etc.


Diffs (updated)
-

  addons/models/3000-Cloud/3030-aws_s3_typedefs_v2.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71922/diff/3/

Changes: https://reviews.apache.org/r/71922/diff/2-3/


Testing
---


Thanks,

Sidharth Mishra



Re: Review Request 71922: ATLAS-3564: New version of AWS S3 model addition

2020-01-08 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71922/
---

(Updated Jan. 8, 2020, 6:54 p.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Madhan Neethiraj, and 
Sarath Subramanian.


Bugs: ATLAS-3564
https://issues.apache.org/jira/browse/ATLAS-3564


Repository: atlas


Description (updated)
---

This is the new version) of aws s3 model at atlas to allow the hierarchical 
structure and attributes same as AWS S3 Console. The existing aws s3 models has 
limitations like pseudo directory not containing another pseudo directory, 
pseudo directory doesn't have attributes of aws object (as some case it gets 
populated), both object and bucket doesn't containing all the attributes of AWS 
S3 etc.


Diffs
-

  addons/models/3000-Cloud/3030-aws_s3_typedefs_v2.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71922/diff/1/


Testing
---


Thanks,

Sidharth Mishra



Re: Review Request 71922: ATLAS-3564: New version of AWS S3 model addition

2020-01-08 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71922/
---

(Updated Jan. 8, 2020, 6:54 p.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Madhan Neethiraj, and 
Sarath Subramanian.


Bugs: ATLAS-3564
https://issues.apache.org/jira/browse/ATLAS-3564


Repository: atlas


Description
---

This is the new version) of aws s3 model at atlas to allow the hierarchical 
structure and attributes same as AWS S3 Console. The existing aws s3 models has 
limitations like pseudo directory not containing another pseudo directory, 
pseudo directory doesn't have attributes of aws object (as some case it gets 
populated), both object and bucket doesn't containing all the attributes of AWS 
S3 etc.


Diffs (updated)
-

  addons/models/3000-Cloud/3030-aws_s3_typedefs_v2.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71922/diff/2/

Changes: https://reviews.apache.org/r/71922/diff/1-2/


Testing
---


Thanks,

Sidharth Mishra



Review Request 71922: ATLAS-3564: New version of AWS S3 model addition

2019-12-17 Thread Sidharth Mishra

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71922/
---

Review request for atlas, Ashutosh Mestry, Sridhar K, Madhan Neethiraj, and 
Sarath Subramanian.


Bugs: ATLAS-3564
https://issues.apache.org/jira/browse/ATLAS-3564


Repository: atlas


Description
---

This is the new version) of aws s3 model at atlas to allow the hierarchical 
structure and attributes same as AWS S3 Console. The existing aws s3 models has 
limitations like pseudo directory not containing another pseudo directory, 
Object doesn't have version information, both object and bucket doesn't 
containing all the attributes of AWS S3 etc.


Diffs
-

  addons/models/3000-Cloud/3030-aws_s3_typedefs_v2.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71922/diff/1/


Testing
---


Thanks,

Sidharth Mishra



Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-12-10 Thread Sidharth Mishra
-parameter-without-filter.json
 PRE-CREATION 
  server-api/src/main/java/org/apache/atlas/RequestContext.java f9ca7a26d 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
40fd6efdc 
  webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 
afc4c50d7 
  
webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
 a4b7b0d49 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
563a16f64 
  webapp/src/test/resources/json/audit-search-parameter-purge.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71791/diff/13/

Changes: https://reviews.apache.org/r/71791/diff/12-13/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-12-10 Thread Sidharth Mishra
-parameter-without-filter.json
 PRE-CREATION 
  server-api/src/main/java/org/apache/atlas/RequestContext.java f9ca7a26d 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
40fd6efdc 
  webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 
afc4c50d7 
  
webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
 a4b7b0d49 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
563a16f64 
  webapp/src/test/resources/json/audit-search-parameter-purge.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71791/diff/12/

Changes: https://reviews.apache.org/r/71791/diff/11-12/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-12-10 Thread Sidharth Mishra
-parameter-without-filter.json
 PRE-CREATION 
  server-api/src/main/java/org/apache/atlas/RequestContext.java f9ca7a26d 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
40fd6efdc 
  webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 
afc4c50d7 
  
webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
 a4b7b0d49 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
563a16f64 
  webapp/src/test/resources/json/audit-search-parameter-purge.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71791/diff/11/

Changes: https://reviews.apache.org/r/71791/diff/10-11/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-12-09 Thread Sidharth Mishra
-parameter-without-filter.json
 PRE-CREATION 
  server-api/src/main/java/org/apache/atlas/RequestContext.java f9ca7a26d 
  webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
40fd6efdc 
  webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 
afc4c50d7 
  
webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
 a4b7b0d49 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
563a16f64 
  webapp/src/test/resources/json/audit-search-parameter-purge.json PRE-CREATION 


Diff: https://reviews.apache.org/r/71791/diff/10/

Changes: https://reviews.apache.org/r/71791/diff/9-10/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-12-04 Thread Sidharth Mishra
/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
 a4b7b0d49 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
563a16f64 


Diff: https://reviews.apache.org/r/71791/diff/8/

Changes: https://reviews.apache.org/r/71791/diff/7-8/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-12-04 Thread Sidharth Mishra


> On Dec. 5, 2019, 12:10 a.m., Sarath Subramanian wrote:
> > intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java
> > Lines 124 (patched)
> > <https://reviews.apache.org/r/71791/diff/7/?file=2182059#file2182059line124>
> >
> > define equals() and hashCode() method as well.

AtlasAuditEntry is derived from AtlasBaseModelObject and the base class 
contains both of these method with guid check. Please let me know do we still 
need these two to be overridden?


> On Dec. 5, 2019, 12:10 a.m., Sarath Subramanian wrote:
> > intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java
> > Lines 125 (patched)
> > <https://reviews.apache.org/r/71791/diff/7/?file=2182059#file2182059line125>
> >
> > incorrect toString() implementation. please format in alignment with 
> > other model classes.

AtlasAuditEntry is derived from AtlasBaseModelObject and the base class 
contains the toString() which calls this toString(sb). At the end it returns 
the correct alignment.


> On Dec. 5, 2019, 12:10 a.m., Sarath Subramanian wrote:
> > intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
> > Lines 128 (patched)
> > <https://reviews.apache.org/r/71791/diff/7/?file=2182060#file2182060line139>
> >
> > => StringUtils.join(entityIds, ","))

Fixed. Could you please let me know the advantage of StringUtils.join over 
String.join


> On Dec. 5, 2019, 12:10 a.m., Sarath Subramanian wrote:
> > repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
> > Lines 76 (patched)
> > <https://reviews.apache.org/r/71791/diff/7/?file=2182062#file2182062line76>
> >
> > move debug log to beginning of method.

This debug msg is to put debug message when we successfully added the audit 
entry


> On Dec. 5, 2019, 12:10 a.m., Sarath Subramanian wrote:
> > webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java
> > Lines 437 (patched)
> > <https://reviews.apache.org/r/71791/diff/7/?file=2182093#file2182093line462>
> >
> > shouldn't purge events be audited only on successful purge on entities? 
> > consider checking EntityMutationResponse for any purge events present and 
> > then audit them.

I think its better to audit admin action purge even if we don't purge any 
entities. Please let me know if you still think we should not audit the 
unsuccessful purge action by admin.


- Sidharth


-------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71791/#review218927
---


On Dec. 4, 2019, 9:57 p.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71791/
> ---
> 
> (Updated Dec. 4, 2019, 9:57 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3518
> https://issues.apache.org/jira/browse/ATLAS-3518
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Right now for purge, delete entity, etc. at Atlas we create audit entry at 
> HBase. User can go to entity and click on audit to see the audit information 
> for that particular entity. But if user purge one entity there will not be 
> any vertex at Janus graph and user will not able to get the purge audit which 
> is important for governance purpose.
> 
> As part of fix, implemented Atlas Audit Framework which is being used for 
> Purge operation and can be extended for other operations. In Future once we 
> change the AtlasAudit type as super type for Import export as well we can use 
> this framework for all Atlas operations. 
> 
> admin/audit rest has been added which will return all the Atlas audit 
> operations including import export at Atlas.
> 
> 
> Diffs
> -
> 
>   addons/models/-Area0/0010-base_model.json 2f5fdaf14 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 8c0a6407c 
>   dashboardv2/public/js/utils/Enums.js 58542ec59 
>   dashboardv3/public/js/utils/Enums.js 58542ec59 
>   intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
> PRE-CREATION 
>   
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
>  b448d510b 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasBaseTypeDef.java 
> eb8634533 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
>  PRE-CREATION 
>

Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-12-04 Thread Sidharth Mishra


> On Dec. 5, 2019, 12:39 a.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
> > Lines 1057 (patched)
> > <https://reviews.apache.org/r/71791/diff/7/?file=2182064#file2182064line1095>
> >
> > This will cause 'DELETED' entities to be skipped even though purge is 
> > in progress. This isn't correct. Please review and update.
> > 
> >   final String guid  = AtlasGraphUtilsV2.getIdFromVertex(vertex);
> >   
> >   ret = guid == null;
> > 
> >   if (!ret) {
> > if (reqContext.isPurgeRequested()) {
> >   ret = requestContext.isPurgedEntity(guid) || getState(vertex) == 
> > ACTIVE;
> > } else {
> >   ret = requestContext.isDeletedEntity(guid) || getState(vertex) == 
> > DELETED;
> > }
> >   }

As the code for purge, hard delete is same, the requestContext still stores the 
purged entities as part of deleteEntity List.


> On Dec. 5, 2019, 12:39 a.m., Madhan Neethiraj wrote:
> > webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java
> > Lines 436 (patched)
> > <https://reviews.apache.org/r/71791/diff/7/?file=2182093#file2182093line461>
> >
> > Audit generation should be done inside entityStore.purgeByIds(), in the 
> > same graph transaction. Avoid spilling such details into REST layer.

As I earlier replied to your previous comment, Moving auditing to inside 
AtlasEntityStore.purgeByIds() is causing circular Injection issue. 
AtlasAuditService depends on DataAccess and DataAccess depends on 
AtlasEntityStore. So AtlasEntityStore can't depends on AtlasAuditService again. 
KIndly suggest is there any other option to avoid this circular injection issue.


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71791/#review218929
---


On Dec. 4, 2019, 9:57 p.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71791/
> ---
> 
> (Updated Dec. 4, 2019, 9:57 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3518
> https://issues.apache.org/jira/browse/ATLAS-3518
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Right now for purge, delete entity, etc. at Atlas we create audit entry at 
> HBase. User can go to entity and click on audit to see the audit information 
> for that particular entity. But if user purge one entity there will not be 
> any vertex at Janus graph and user will not able to get the purge audit which 
> is important for governance purpose.
> 
> As part of fix, implemented Atlas Audit Framework which is being used for 
> Purge operation and can be extended for other operations. In Future once we 
> change the AtlasAudit type as super type for Import export as well we can use 
> this framework for all Atlas operations. 
> 
> admin/audit rest has been added which will return all the Atlas audit 
> operations including import export at Atlas.
> 
> 
> Diffs
> -
> 
>   addons/models/-Area0/0010-base_model.json 2f5fdaf14 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 8c0a6407c 
>   dashboardv2/public/js/utils/Enums.js 58542ec59 
>   dashboardv3/public/js/utils/Enums.js 58542ec59 
>   intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
> PRE-CREATION 
>   
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
>  b448d510b 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasBaseTypeDef.java 
> eb8634533 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/ogm/AtlasAuditEntryDTO.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  c9ed79750 
>   repository/src/test/java/org/apache/atlas/TestModules.java 94b08a187 
>   repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
> be1698fb4 
>   repository/src/test/java/org/apache/atlas/query/BasicTestSetup.java 
> 02f78b369 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AdminPurgeTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AtlasAuditS

Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-12-04 Thread Sidharth Mishra
/atlas/web/adapters/TestEntitiesREST.java 
afc4c50d7 
  
webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
 a4b7b0d49 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
563a16f64 


Diff: https://reviews.apache.org/r/71791/diff/7/

Changes: https://reviews.apache.org/r/71791/diff/6-7/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-12-04 Thread Sidharth Mishra


> On Dec. 1, 2019, 4:37 p.m., Madhan Neethiraj wrote:
> > addons/models/-Area0/0010-base_model.json
> > Lines 5 (patched)
> > <https://reviews.apache.org/r/71791/diff/5/?file=2181198#file2181198line5>
> >
> > "Audit Operation defines the operations we support for admin audit" => 
> > "Defines admin operations in Apache Atlas"

Removed the admin and changed it to "Defines operations in Apache Atlas which 
can be audited"


> On Dec. 1, 2019, 4:37 p.m., Madhan Neethiraj wrote:
> > webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java
> > Lines 436 (patched)
> > <https://reviews.apache.org/r/71791/diff/5/?file=2181235#file2181235line461>
> >
> > Instead of generating audit in a seperate graph transaction here, 
> > consider generating in the transaction used by 
> > AtlasEntityStore.purgeByIds() i.e. move auditing to inside 
> > AtlasEntityStore.purgeByIds().

Moving auditing to inside AtlasEntityStore.purgeByIds() is causing circular 
Injection issue. AtlasAuditService depends on DataAccess and DataAccess depends 
on AtlasEntityStore. So AtlasEntityStore can't depends on AtlasAuditService 
again.


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71791/#review218865
---


On Dec. 4, 2019, 9:47 p.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71791/
> ---
> 
> (Updated Dec. 4, 2019, 9:47 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3518
> https://issues.apache.org/jira/browse/ATLAS-3518
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Right now for purge, delete entity, etc. at Atlas we create audit entry at 
> HBase. User can go to entity and click on audit to see the audit information 
> for that particular entity. But if user purge one entity there will not be 
> any vertex at Janus graph and user will not able to get the purge audit which 
> is important for governance purpose.
> 
> As part of fix, implemented Atlas Audit Framework which is being used for 
> Purge operation and can be extended for other operations. In Future once we 
> change the AtlasAudit type as super type for Import export as well we can use 
> this framework for all Atlas operations. 
> 
> admin/audit rest has been added which will return all the Atlas audit 
> operations including import export at Atlas.
> 
> 
> Diffs
> -
> 
>   addons/models/-Area0/0010-base_model.json 2f5fdaf14 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 8c0a6407c 
>   dashboardv2/public/js/utils/Enums.js 58542ec59 
>   dashboardv3/public/js/utils/Enums.js 58542ec59 
>   intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
> PRE-CREATION 
>   
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
>  b448d510b 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasBaseTypeDef.java 
> eb8634533 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/ogm/AtlasAuditEntryDTO.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  c9ed79750 
>   repository/src/test/java/org/apache/atlas/TestModules.java 94b08a187 
>   repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
> be1698fb4 
>   repository/src/test/java/org/apache/atlas/query/BasicTestSetup.java 
> 02f78b369 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AdminPurgeTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AtlasAuditServiceTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/AtlasServerServiceTest.java
>  78865c569 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportImportAuditServiceTest.java
>  ba7a8a04b 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportImportTestBase.java
>  925b2a007 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportIncrementalTest.java
>  4d438525a 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportServiceTest.java
>  db8c6c83c 

Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-12-04 Thread Sidharth Mishra
/atlas/web/adapters/TestEntitiesREST.java 
afc4c50d7 
  
webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
 a4b7b0d49 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
563a16f64 


Diff: https://reviews.apache.org/r/71791/diff/6/

Changes: https://reviews.apache.org/r/71791/diff/5-6/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-12-03 Thread Sidharth Mishra


> On Dec. 1, 2019, 4:37 p.m., Madhan Neethiraj wrote:
> > addons/models/-Area0/0010-base_model.json
> > Lines 4 (patched)
> > <https://reviews.apache.org/r/71791/diff/5/?file=2181198#file2181198line4>
> >
> > atlas_operation => atlas_admin_operation

I think as you initially suggested to rename it to atlas_operation is more 
flexible than atlas_admin_operation as in future we may allow some more 
operation to be audited like other than admin operations.


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71791/#review218865
---


On Nov. 30, 2019, 7:44 p.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71791/
> ---
> 
> (Updated Nov. 30, 2019, 7:44 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3518
> https://issues.apache.org/jira/browse/ATLAS-3518
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Right now for purge, delete entity, etc. at Atlas we create audit entry at 
> HBase. User can go to entity and click on audit to see the audit information 
> for that particular entity. But if user purge one entity there will not be 
> any vertex at Janus graph and user will not able to get the purge audit which 
> is important for governance purpose.
> 
> As part of fix, implemented Atlas Audit Framework which is being used for 
> Purge operation and can be extended for other operations. In Future once we 
> change the AtlasAudit type as super type for Import export as well we can use 
> this framework for all Atlas operations. 
> 
> admin/audit rest has been added which will return all the Atlas audit 
> operations including import export at Atlas.
> 
> 
> Diffs
> -
> 
>   addons/models/-Area0/0010-base_model.json 2f5fdaf14 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 8c0a6407c 
>   dashboardv2/public/js/utils/Enums.js 58542ec59 
>   dashboardv3/public/js/utils/Enums.js 58542ec59 
>   intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
> PRE-CREATION 
>   
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
>  b448d510b 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/ogm/AtlasAuditEntryDTO.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  621458406 
>   repository/src/test/java/org/apache/atlas/TestModules.java 94b08a187 
>   repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
> be1698fb4 
>   repository/src/test/java/org/apache/atlas/query/BasicTestSetup.java 
> 02f78b369 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AdminPurgeTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AtlasAuditServiceTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/AtlasServerServiceTest.java
>  78865c569 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportImportAuditServiceTest.java
>  ba7a8a04b 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportImportTestBase.java
>  925b2a007 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportIncrementalTest.java
>  4d438525a 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportServiceTest.java
>  db8c6c83c 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportSkipLineageTest.java
>  25e0a5362 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/HdfsPathEntityCreatorTest.java
>  1863b8d74 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportReactivateTableTest.java
>  d0c06a107 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
>  c14850f43 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportTransformsShaperTest.java
>  78fdaca8d 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/IncrementalExportEntityProviderTest.java
>  42b63535b 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/RelationshipAttributesExtractorTest.java
>  a1b512fb

Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-12-03 Thread Sidharth Mishra


> On Nov. 27, 2019, 12:39 a.m., Madhan Neethiraj wrote:
> > addons/models/-Area0/0010-base_model.json
> > Lines 442 (patched)
> > <https://reviews.apache.org/r/71791/diff/4/?file=2179150#file2179150line442>
> >
> > Consider renaming 'clientHost' to 'clientId' - given this field is used 
> > to store AtlasServer name for import/export audit entries.
> > 
> > This rename should be applied to AtlasAuditEntry.clientHost as well.

I changed it to clientId as suggested, But please note this field contains 
Client IP Address. In future we may put host fqdn if available. In both the 
cases do you think client Id makes more sense?


> On Nov. 27, 2019, 12:39 a.m., Madhan Neethiraj wrote:
> > pom.xml
> > Line 695 (original), 695 (patched)
> > <https://reviews.apache.org/r/71791/diff/4/?file=2179156#file2179156line695>
> >
> > I suggest a separate patch for library version changes.

This fix was to fix the precommit build and now the precommit is successful - 
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1559/.
 I am having separate patch for the lib version change and reverted the changes


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71791/#review218808
-------


On Nov. 30, 2019, 7:44 p.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71791/
> ---
> 
> (Updated Nov. 30, 2019, 7:44 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3518
> https://issues.apache.org/jira/browse/ATLAS-3518
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Right now for purge, delete entity, etc. at Atlas we create audit entry at 
> HBase. User can go to entity and click on audit to see the audit information 
> for that particular entity. But if user purge one entity there will not be 
> any vertex at Janus graph and user will not able to get the purge audit which 
> is important for governance purpose.
> 
> As part of fix, implemented Atlas Audit Framework which is being used for 
> Purge operation and can be extended for other operations. In Future once we 
> change the AtlasAudit type as super type for Import export as well we can use 
> this framework for all Atlas operations. 
> 
> admin/audit rest has been added which will return all the Atlas audit 
> operations including import export at Atlas.
> 
> 
> Diffs
> -
> 
>   addons/models/-Area0/0010-base_model.json 2f5fdaf14 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 8c0a6407c 
>   dashboardv2/public/js/utils/Enums.js 58542ec59 
>   dashboardv3/public/js/utils/Enums.js 58542ec59 
>   intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
> PRE-CREATION 
>   
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
>  b448d510b 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/ogm/AtlasAuditEntryDTO.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  621458406 
>   repository/src/test/java/org/apache/atlas/TestModules.java 94b08a187 
>   repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
> be1698fb4 
>   repository/src/test/java/org/apache/atlas/query/BasicTestSetup.java 
> 02f78b369 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AdminPurgeTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AtlasAuditServiceTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/AtlasServerServiceTest.java
>  78865c569 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportImportAuditServiceTest.java
>  ba7a8a04b 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportImportTestBase.java
>  925b2a007 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportIncrementalTest.java
>  4d438525a 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportServiceTest.java
>  db8c6c83c 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportSkipLineageTest.java
>  25e0a5362 
>   
> 

Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-12-03 Thread Sidharth Mishra


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71791/#review218808
---


On Nov. 30, 2019, 7:44 p.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71791/
> ---
> 
> (Updated Nov. 30, 2019, 7:44 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3518
> https://issues.apache.org/jira/browse/ATLAS-3518
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Right now for purge, delete entity, etc. at Atlas we create audit entry at 
> HBase. User can go to entity and click on audit to see the audit information 
> for that particular entity. But if user purge one entity there will not be 
> any vertex at Janus graph and user will not able to get the purge audit which 
> is important for governance purpose.
> 
> As part of fix, implemented Atlas Audit Framework which is being used for 
> Purge operation and can be extended for other operations. In Future once we 
> change the AtlasAudit type as super type for Import export as well we can use 
> this framework for all Atlas operations. 
> 
> admin/audit rest has been added which will return all the Atlas audit 
> operations including import export at Atlas.
> 
> 
> Diffs
> -
> 
>   addons/models/-Area0/0010-base_model.json 2f5fdaf14 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 8c0a6407c 
>   dashboardv2/public/js/utils/Enums.js 58542ec59 
>   dashboardv3/public/js/utils/Enums.js 58542ec59 
>   intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
> PRE-CREATION 
>   
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
>  b448d510b 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/ogm/AtlasAuditEntryDTO.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  621458406 
>   repository/src/test/java/org/apache/atlas/TestModules.java 94b08a187 
>   repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
> be1698fb4 
>   repository/src/test/java/org/apache/atlas/query/BasicTestSetup.java 
> 02f78b369 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AdminPurgeTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AtlasAuditServiceTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/AtlasServerServiceTest.java
>  78865c569 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportImportAuditServiceTest.java
>  ba7a8a04b 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportImportTestBase.java
>  925b2a007 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportIncrementalTest.java
>  4d438525a 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportServiceTest.java
>  db8c6c83c 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportSkipLineageTest.java
>  25e0a5362 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/HdfsPathEntityCreatorTest.java
>  1863b8d74 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportReactivateTableTest.java
>  d0c06a107 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java
>  c14850f43 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportTransformsShaperTest.java
>  78fdaca8d 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/IncrementalExportEntityProviderTest.java
>  42b63535b 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/RelationshipAttributesExtractorTest.java
>  a1b512fb9 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ReplicationEntityAttributeTest.java
>  7a1ed18cb 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/StartEntityFetchByExportRequestTest.java
>  48b958271 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/TableReplicationRequestProcessorTest.java
>  c9bb11c56 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ZipFileResourceTestUtils.java
>  0ffc3d595 
>   
> r

Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-11-30 Thread Sidharth Mishra
/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
 a4b7b0d49 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
563a16f64 


Diff: https://reviews.apache.org/r/71791/diff/5/

Changes: https://reviews.apache.org/r/71791/diff/4-5/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-11-30 Thread Sidharth Mishra


> On Nov. 25, 2019, 5:05 a.m., Madhan Neethiraj wrote:
> > addons/models/-Area0/0010-base_model.json
> > Lines 397 (patched)
> > <https://reviews.apache.org/r/71791/diff/3/?file=2174685#file2174685line397>
> >
> > - I suggest adding attribute 'serverName', that will be set to 
> > AtlasServer.name - similar to 
> > __ExportImportAuditEntry.sourceServerName/__ExportImportAuditEntry.targetServerName
> > - consider adding an unique attribute, like __guid

As we have __guid as system attribute and gets added to entity, not adding 
__guid as part of model. Do you think we need serverName as it will be inserted 
as current host all the time and it may not be usefull. For import export its 
needed and it may differ so we addded that. As part of migration I was planning 
to change the current import export audit model where it will be derived from 
this audit model type and add the source and destination server only for import 
export audit type.


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71791/#review218785
-------


On Nov. 26, 2019, 9:11 a.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71791/
> ---
> 
> (Updated Nov. 26, 2019, 9:11 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3518
> https://issues.apache.org/jira/browse/ATLAS-3518
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Right now for purge, delete entity, etc. at Atlas we create audit entry at 
> HBase. User can go to entity and click on audit to see the audit information 
> for that particular entity. But if user purge one entity there will not be 
> any vertex at Janus graph and user will not able to get the purge audit which 
> is important for governance purpose.
> 
> As part of fix, implemented Atlas Audit Framework which is being used for 
> Purge operation and can be extended for other operations. In Future once we 
> change the AtlasAudit type as super type for Import export as well we can use 
> this framework for all Atlas operations. 
> 
> admin/audit rest has been added which will return all the Atlas audit 
> operations including import export at Atlas.
> 
> 
> Diffs
> -
> 
>   addons/models/-Area0/0010-base_model.json 2f5fdaf14 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 8c0a6407c 
>   dashboardv2/public/js/utils/Enums.js 58542ec59 
>   dashboardv3/public/js/utils/Enums.js 58542ec59 
>   intg/src/main/java/org/apache/atlas/model/audit/AtlasAuditEntry.java 
> PRE-CREATION 
>   
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
>  b448d510b 
>   pom.xml b2506e70e 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AtlasAuditService.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/ogm/AtlasAuditEntryDTO.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  621458406 
>   repository/src/test/java/org/apache/atlas/TestModules.java 94b08a187 
>   repository/src/test/java/org/apache/atlas/glossary/GlossaryServiceTest.java 
> be1698fb4 
>   repository/src/test/java/org/apache/atlas/query/BasicTestSetup.java 
> 02f78b369 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AdminPurgeTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/audit/AtlasAuditServiceTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/AtlasServerServiceTest.java
>  78865c569 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportImportAuditServiceTest.java
>  ba7a8a04b 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportImportTestBase.java
>  925b2a007 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportIncrementalTest.java
>  4d438525a 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportServiceTest.java
>  db8c6c83c 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/ExportSkipLineageTest.java
>  25e0a5362 
>   
> repository/src/test/java/org/apache/atlas/repository/impexp/HdfsPathEntityCreatorTest.java
>  1863b8d74 
>   
> repository/src/test/java/org/apache/atlas/repo

Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-11-26 Thread Sidharth Mishra
 
  webapp/src/test/java/org/apache/atlas/web/resources/AdminResourceTest.java 
563a16f64 


Diff: https://reviews.apache.org/r/71791/diff/4/

Changes: https://reviews.apache.org/r/71791/diff/3-4/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-11-21 Thread Sidharth Mishra
/java/org/apache/atlas/web/resources/AdminResourceTest.java 
563a16f64 


Diff: https://reviews.apache.org/r/71791/diff/3/

Changes: https://reviews.apache.org/r/71791/diff/2-3/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Re: Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-11-19 Thread Sidharth Mishra
/java/org/apache/atlas/web/resources/AdminResourceTest.java 
563a16f64 


Diff: https://reviews.apache.org/r/71791/diff/2/

Changes: https://reviews.apache.org/r/71791/diff/1-2/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Review Request 71791: ATLAS-3518: Create Audit Framework for Atlas Purge Entity

2019-11-19 Thread Sidharth Mishra
/AdminResourceTest.java 
563a16f64 


Diff: https://reviews.apache.org/r/71791/diff/1/


Testing
---

Two UT and one IT has been added to test functionality like:
1. Atlas Audit entry save and retrive
2. Atlas Purge Audit store, retrive and search (UT and IT)


Thanks,

Sidharth Mishra



Re: Review Request 71737: ATLAS-3477: Introduce purging of entity in Atlas

2019-11-14 Thread Sidharth Mishra


> On Nov. 13, 2019, 8:37 p.m., Sarath Subramanian wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
> > Lines 484 (patched)
> > <https://reviews.apache.org/r/71737/diff/4/?file=2173006#file2173006line485>
> >
> > should this be AtlasPrivilege.ADMIN_PURGE ?
> 
> Sidharth Mishra wrote:
> As we don't have the Ranger changes for time being its ADMIN_IMPORT. 
> Later ll change it to ADMIN_PURGE

To track this created a JIRA Item - ATLAS-3524


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71737/#review218622
---


On Nov. 14, 2019, 7:10 p.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71737/
> ---
> 
> (Updated Nov. 14, 2019, 7:10 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3477
> https://issues.apache.org/jira/browse/ATLAS-3477
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> As part of this solution users can purge the entities which are already soft 
> deleted. This will be useful as someone may be interested to purge some list 
> of soft deleted entities from Atlas. Due to governance compliances, only 
> admin can perform purge of entities.
> 
> Purge will also create separate Audit entry at HBase similar to delete entity 
> operation.
> 
> 
> Diffs
> -
> 
>   authorization/src/main/java/org/apache/atlas/authorize/AtlasPrivilege.java 
> 59c596d66 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 199d6bf58 
>   intg/src/main/java/org/apache/atlas/listener/EntityChangeListenerV2.java 
> 444167eae 
>   intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
> e9cc7cd3d 
>   
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
>  7ace00dff 
>   intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 
> daf4ca88c 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
>  43a9b842c 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java
>  b94590bf1 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  d2544df4f 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityChangeNotifier.java
>  3389d2465 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
>  17a96493f 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasGraphUtilsV2.java
>  4d57d8bc8 
>   server-api/src/main/java/org/apache/atlas/RequestContext.java 3d2a18f0c 
>   
> webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
>  48f0cd3c2 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
> 464d46f1e 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
>  5d506bbc2 
> 
> 
> Diff: https://reviews.apache.org/r/71737/diff/6/
> 
> 
> Testing
> ---
> 
> IT has been added. 
> Validated manually by running below mentioned queries and REST:
> 
> 
> a) create table thursday_07_11_1 (id int);
>create table thursday_07_11_1_ctas as select * from thursday_07_11_1;
> 
>create table thursday_07_11_2 (id int);
>create table thursday_07_11_2_ctas as select * from thursday_07_11_2;
> b) The above will create two lineages. Go to Atlas UI and get the vertices 
> for atleast any two.
> 
> 
> c) Using rest delete those two entities or use below query:
> drop table thursday_07_11_1;
> drop table thursday_07_11_1_ctas;
> 
> drop table thursday_07_11_2;
> drop table thursday_07_11_2_ctas;
> 
> REST delete example:
> DELETE 
> http://localhopst:31000/api/atlas/v2/entity/guid/7dd7fd9c-0185-41a5-aff0-4212fa7eb87c
> 
> d) Now use the same guids and call purge REST:
> DELETE 
> http://localhost:31000/api/atlas/admin/purge/?guid=75983487-9e29-41ee-9bae-53f0a0b54c05=98bee463-8810-402a-bcdc-3eb00bf50365
> 
> Check the response which will return all the successfully purged entities
> 
> e) For each guid check if the audit has separate new entry for purge. REST 
> used:
> GET 
> http://localhost:31000/api/atlas/v2/entity/7b6806

Re: Review Request 71737: ATLAS-3477: Introduce purging of entity in Atlas

2019-11-14 Thread Sidharth Mishra
plicatedFrom\":null,\"qualifiedName\":\"default.thursday_07_11_5.id@cm\",\"name\":\"id\",\"description\":null,\"comment\":null,\"position\":0,\"type\":\"int\"},\"guid\":\"7b6806c1-8daf-4e0f-8d10-672a192968f1\",\"isIncomplete\":false,\"status\":\"ACTIVE\",\"createdBy\":\"impala\",\"updatedBy\":\"admin\",\"createTime\":1573161723180,\"updateTime\":1573161727305,\"version\":0,\"labels\":[]}",
"eventKey": "7b6806c1-8daf-4e0f-8d10-672a192968f1:1573161727305:0",
"entity": null,
"type": null
},
{
"entityId": "7b6806c1-8daf-4e0f-8d10-672a192968f1",
"timestamp": 1573161723180,
"user": "impala",
"action": "ENTITY_CREATE",
"details": "Created: 
{\"typeName\":\"hive_column\",\"attributes\":{\"owner\":\"admin\",\"qualifiedName\":\"default.thursday_07_11_5.id@cm\",\"name\":\"id\",\"comment\":null,\"position\":0,\"type\":\"int\"},\"guid\":\"7b6806c1-8daf-4e0f-8d10-672a192968f1\",\"isIncomplete\":false,\"provenanceType\":0,\"version\":0,\"relationshipAttributes\":{\"table\":{\"guid\":\"c5ae4503-b498-493c-86b5-d8d909c6b16a\",\"typeName\":\"hive_table\",\"uniqueAttributes\":{\"qualifiedName\":\"default.thursday_07_11_5@cm\"},\"relationshipType\":\"hive_table_columns\"}},\"proxy\":false}",
"eventKey": "7b6806c1-8daf-4e0f-8d10-672a192968f1:1573161723180:2",
"entity": null,
"type": null
}
]


Thanks,

Sidharth Mishra



Re: Review Request 71737: ATLAS-3477: Introduce purging of entity in Atlas

2019-11-14 Thread Sidharth Mishra


> On Nov. 14, 2019, 12:02 a.m., Ashutosh Mestry wrote:
> > My thinking is that there should be a bulk API that should take JSON 
> > payload of entities to be purged. Generally, there are large number of 
> > deleted entities. There should be a convinent way to purge them. Right now 
> > the queryString has a smaller limit which is quickly exhausted due to 
> > length of GUIDs.

Right now in the current code I have already Addressed this and changed it to 
body from queryString.


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71737/#review218627
---


On Nov. 13, 2019, 9:04 p.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71737/
> ---
> 
> (Updated Nov. 13, 2019, 9:04 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3477
> https://issues.apache.org/jira/browse/ATLAS-3477
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> As part of this solution users can purge the entities which are already soft 
> deleted. This will be useful as someone may be interested to purge some list 
> of soft deleted entities from Atlas. Due to governance compliances, only 
> admin can perform purge of entities.
> 
> Purge will also create separate Audit entry at HBase similar to delete entity 
> operation.
> 
> 
> Diffs
> -
> 
>   authorization/src/main/java/org/apache/atlas/authorize/AtlasPrivilege.java 
> 59c596d66 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 199d6bf58 
>   intg/src/main/java/org/apache/atlas/listener/EntityChangeListenerV2.java 
> 444167eae 
>   intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
> e9cc7cd3d 
>   
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
>  7ace00dff 
>   intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 
> daf4ca88c 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
>  43a9b842c 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java
>  b94590bf1 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  d2544df4f 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityChangeNotifier.java
>  3389d2465 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
>  17a96493f 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasGraphUtilsV2.java
>  4d57d8bc8 
>   server-api/src/main/java/org/apache/atlas/RequestContext.java 3d2a18f0c 
>   
> webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
>  48f0cd3c2 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
> 464d46f1e 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
>  5d506bbc2 
> 
> 
> Diff: https://reviews.apache.org/r/71737/diff/5/
> 
> 
> Testing
> ---
> 
> IT has been added. 
> Validated manually by running below mentioned queries and REST:
> 
> 
> a) create table thursday_07_11_1 (id int);
>create table thursday_07_11_1_ctas as select * from thursday_07_11_1;
> 
>create table thursday_07_11_2 (id int);
>create table thursday_07_11_2_ctas as select * from thursday_07_11_2;
> b) The above will create two lineages. Go to Atlas UI and get the vertices 
> for atleast any two.
> 
> 
> c) Using rest delete those two entities or use below query:
> drop table thursday_07_11_1;
> drop table thursday_07_11_1_ctas;
> 
> drop table thursday_07_11_2;
> drop table thursday_07_11_2_ctas;
> 
> REST delete example:
> DELETE 
> http://localhopst:31000/api/atlas/v2/entity/guid/7dd7fd9c-0185-41a5-aff0-4212fa7eb87c
> 
> d) Now use the same guids and call purge REST:
> DELETE 
> http://localhost:31000/api/atlas/admin/purge/?guid=75983487-9e29-41ee-9bae-53f0a0b54c05=98bee463-8810-402a-bcdc-3eb00bf50365
> 
> Check the response which will return all the successfully purged entities
> 
> e) For each guid check if the audit has separate new entry for purge. REST 
> used:
> GET 
> http://localhost:31000/api/atlas/v2/entity/7b6806c1-8daf-4e0f-8d10-672a192968f1/audit
> 
> Example Response:
&

Re: Review Request 71737: ATLAS-3477: Introduce purging of entity in Atlas

2019-11-14 Thread Sidharth Mishra


> On Nov. 14, 2019, midnight, Ashutosh Mestry wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
> > Lines 159 (patched)
> > <https://reviews.apache.org/r/71737/diff/5/?file=2173080#file2173080line159>
> >
> > Does this mean that if purge is requested, then even active edges are 
> > requested whereas if not deleted edges are requested.
> > 
> > Shouldn't it be that when purge is requested, both active and deleted 
> > edges are requested?

Right now purge will only be allowed if the entity is already soft deleted. 
This check skip the edges or vertices for the same reason.


> On Nov. 14, 2019, midnight, Ashutosh Mestry wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
> > Lines 200 (patched)
> > <https://reviews.apache.org/r/71737/diff/5/?file=2173080#file2173080line200>
> >
> > Same comment as above.

Right now purge will only be allowed if the entity is already soft deleted. 
This check skip the edges or vertices for the same reason.


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71737/#review218626
---


On Nov. 13, 2019, 9:04 p.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71737/
> ---
> 
> (Updated Nov. 13, 2019, 9:04 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3477
> https://issues.apache.org/jira/browse/ATLAS-3477
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> As part of this solution users can purge the entities which are already soft 
> deleted. This will be useful as someone may be interested to purge some list 
> of soft deleted entities from Atlas. Due to governance compliances, only 
> admin can perform purge of entities.
> 
> Purge will also create separate Audit entry at HBase similar to delete entity 
> operation.
> 
> 
> Diffs
> -
> 
>   authorization/src/main/java/org/apache/atlas/authorize/AtlasPrivilege.java 
> 59c596d66 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 199d6bf58 
>   intg/src/main/java/org/apache/atlas/listener/EntityChangeListenerV2.java 
> 444167eae 
>   intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
> e9cc7cd3d 
>   
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
>  7ace00dff 
>   intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 
> daf4ca88c 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
>  43a9b842c 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java
>  b94590bf1 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  d2544df4f 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityChangeNotifier.java
>  3389d2465 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
>  17a96493f 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasGraphUtilsV2.java
>  4d57d8bc8 
>   server-api/src/main/java/org/apache/atlas/RequestContext.java 3d2a18f0c 
>   
> webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
>  48f0cd3c2 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
> 464d46f1e 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
>  5d506bbc2 
> 
> 
> Diff: https://reviews.apache.org/r/71737/diff/5/
> 
> 
> Testing
> ---
> 
> IT has been added. 
> Validated manually by running below mentioned queries and REST:
> 
> 
> a) create table thursday_07_11_1 (id int);
>create table thursday_07_11_1_ctas as select * from thursday_07_11_1;
> 
>create table thursday_07_11_2 (id int);
>create table thursday_07_11_2_ctas as select * from thursday_07_11_2;
> b) The above will create two lineages. Go to Atlas UI and get the vertices 
> for atleast any two.
> 
> 
> c) Using rest delete those two entities or use below query:
> drop table thursday_07_11_1;
> drop table thursday_07_11_1_ctas;
> 
> drop table thursday_07_11_2;
> drop table thursday_07_1

Re: Review Request 71737: ATLAS-3477: Introduce purging of entity in Atlas

2019-11-13 Thread Sidharth Mishra


> On Nov. 13, 2019, 8:37 p.m., Sarath Subramanian wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
> > Lines 484 (patched)
> > <https://reviews.apache.org/r/71737/diff/4/?file=2173006#file2173006line485>
> >
> > should this be AtlasPrivilege.ADMIN_PURGE ?

As we don't have the Ranger changes for time being its ADMIN_IMPORT. Later ll 
change it to ADMIN_PURGE


- Sidharth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71737/#review218622
---


On Nov. 13, 2019, 6:04 a.m., Sidharth Mishra wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71737/
> ---
> 
> (Updated Nov. 13, 2019, 6:04 a.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3477
> https://issues.apache.org/jira/browse/ATLAS-3477
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> As part of this solution users can purge the entities which are already soft 
> deleted. This will be useful as someone may be interested to purge some list 
> of soft deleted entities from Atlas. Due to governance compliances, only 
> admin can perform purge of entities.
> 
> Purge will also create separate Audit entry at HBase similar to delete entity 
> operation.
> 
> 
> Diffs
> -
> 
>   authorization/src/main/java/org/apache/atlas/authorize/AtlasPrivilege.java 
> 59c596d66 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 199d6bf58 
>   intg/src/main/java/org/apache/atlas/listener/EntityChangeListenerV2.java 
> 444167eae 
>   intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java 
> e9cc7cd3d 
>   
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java
>  7ace00dff 
>   intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 
> daf4ca88c 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
>  43a9b842c 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java
>  b94590bf1 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  d2544df4f 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityChangeNotifier.java
>  3389d2465 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
>  17a96493f 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasGraphUtilsV2.java
>  4d57d8bc8 
>   server-api/src/main/java/org/apache/atlas/RequestContext.java 3d2a18f0c 
>   
> webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
>  48f0cd3c2 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 
> 464d46f1e 
>   
> webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
>  5d506bbc2 
> 
> 
> Diff: https://reviews.apache.org/r/71737/diff/4/
> 
> 
> Testing
> ---
> 
> IT has been added. 
> Validated manually by running below mentioned queries and REST:
> 
> 
> a) create table thursday_07_11_1 (id int);
>create table thursday_07_11_1_ctas as select * from thursday_07_11_1;
> 
>create table thursday_07_11_2 (id int);
>create table thursday_07_11_2_ctas as select * from thursday_07_11_2;
> b) The above will create two lineages. Go to Atlas UI and get the vertices 
> for atleast any two.
> 
> 
> c) Using rest delete those two entities or use below query:
> drop table thursday_07_11_1;
> drop table thursday_07_11_1_ctas;
> 
> drop table thursday_07_11_2;
> drop table thursday_07_11_2_ctas;
> 
> REST delete example:
> DELETE 
> http://localhopst:31000/api/atlas/v2/entity/guid/7dd7fd9c-0185-41a5-aff0-4212fa7eb87c
> 
> d) Now use the same guids and call purge REST:
> DELETE 
> http://localhost:31000/api/atlas/admin/purge/?guid=75983487-9e29-41ee-9bae-53f0a0b54c05=98bee463-8810-402a-bcdc-3eb00bf50365
> 
> Check the response which will return all the successfully purged entities
> 
> e) For each guid check if the audit has separate new entry for purge. REST 
> used:
> GET 
> http://localhost:31000/api/atlas/v2/entity/7b6806c1-8daf-4e0f-8d10-672a192968f1/audit
> 
> Example Response:
> 
> [
> {
> "ent

Re: Review Request 71737: ATLAS-3477: Introduce purging of entity in Atlas

2019-11-12 Thread Sidharth Mishra
plicatedFrom\":null,\"qualifiedName\":\"default.thursday_07_11_5.id@cm\",\"name\":\"id\",\"description\":null,\"comment\":null,\"position\":0,\"type\":\"int\"},\"guid\":\"7b6806c1-8daf-4e0f-8d10-672a192968f1\",\"isIncomplete\":false,\"status\":\"ACTIVE\",\"createdBy\":\"impala\",\"updatedBy\":\"admin\",\"createTime\":1573161723180,\"updateTime\":1573161727305,\"version\":0,\"labels\":[]}",
"eventKey": "7b6806c1-8daf-4e0f-8d10-672a192968f1:1573161727305:0",
"entity": null,
"type": null
},
{
"entityId": "7b6806c1-8daf-4e0f-8d10-672a192968f1",
"timestamp": 1573161723180,
"user": "impala",
"action": "ENTITY_CREATE",
"details": "Created: 
{\"typeName\":\"hive_column\",\"attributes\":{\"owner\":\"admin\",\"qualifiedName\":\"default.thursday_07_11_5.id@cm\",\"name\":\"id\",\"comment\":null,\"position\":0,\"type\":\"int\"},\"guid\":\"7b6806c1-8daf-4e0f-8d10-672a192968f1\",\"isIncomplete\":false,\"provenanceType\":0,\"version\":0,\"relationshipAttributes\":{\"table\":{\"guid\":\"c5ae4503-b498-493c-86b5-d8d909c6b16a\",\"typeName\":\"hive_table\",\"uniqueAttributes\":{\"qualifiedName\":\"default.thursday_07_11_5@cm\"},\"relationshipType\":\"hive_table_columns\"}},\"proxy\":false}",
"eventKey": "7b6806c1-8daf-4e0f-8d10-672a192968f1:1573161723180:2",
"entity": null,
"type": null
}
]


Thanks,

Sidharth Mishra



Re: Review Request 71737: ATLAS-3477: Introduce purging of entity in Atlas

2019-11-12 Thread Sidharth Mishra
edTo\":null,\"replicatedFrom\":null,\"qualifiedName\":\"default.thursday_07_11_5.id@cm\",\"name\":\"id\",\"description\":null,\"comment\":null,\"position\":0,\"type\":\"int\"},\"guid\":\"7b6806c1-8daf-4e0f-8d10-672a192968f1\",\"isIncomplete\":false,\"status\":\"ACTIVE\",\"createdBy\":\"impala\",\"updatedBy\":\"admin\",\"createTime\":1573161723180,\"updateTime\":1573161727305,\"version\":0,\"labels\":[]}",
"eventKey": "7b6806c1-8daf-4e0f-8d10-672a192968f1:1573161727305:0",
"entity": null,
"type": null
},
{
"entityId": "7b6806c1-8daf-4e0f-8d10-672a192968f1",
"timestamp": 1573161723180,
"user": "impala",
"action": "ENTITY_CREATE",
"details": "Created: 
{\"typeName\":\"hive_column\",\"attributes\":{\"owner\":\"admin\",\"qualifiedName\":\"default.thursday_07_11_5.id@cm\",\"name\":\"id\",\"comment\":null,\"position\":0,\"type\":\"int\"},\"guid\":\"7b6806c1-8daf-4e0f-8d10-672a192968f1\",\"isIncomplete\":false,\"provenanceType\":0,\"version\":0,\"relationshipAttributes\":{\"table\":{\"guid\":\"c5ae4503-b498-493c-86b5-d8d909c6b16a\",\"typeName\":\"hive_table\",\"uniqueAttributes\":{\"qualifiedName\":\"default.thursday_07_11_5@cm\"},\"relationshipType\":\"hive_table_columns\"}},\"proxy\":false}",
"eventKey": "7b6806c1-8daf-4e0f-8d10-672a192968f1:1573161723180:2",
"entity": null,
"type": null
}
]


Thanks,

Sidharth Mishra



Re: Review Request 71737: ATLAS-3477: Introduce purging of entity in Atlas

2019-11-12 Thread Sidharth Mishra
ot;:\"admin\",\"replicatedTo\":null,\"replicatedFrom\":null,\"qualifiedName\":\"default.thursday_07_11_5.id@cm\",\"name\":\"id\",\"description\":null,\"comment\":null,\"position\":0,\"type\":\"int\"},\"guid\":\"7b6806c1-8daf-4e0f-8d10-672a192968f1\",\"isIncomplete\":false,\"status\":\"ACTIVE\",\"createdBy\":\"impala\",\"updatedBy\":\"admin\",\"createTime\":1573161723180,\"updateTime\":1573161727305,\"version\":0,\"labels\":[]}",
"eventKey": "7b6806c1-8daf-4e0f-8d10-672a192968f1:1573161727305:0",
"entity": null,
"type": null
},
{
"entityId": "7b6806c1-8daf-4e0f-8d10-672a192968f1",
"timestamp": 1573161723180,
"user": "impala",
"action": "ENTITY_CREATE",
"details": "Created: 
{\"typeName\":\"hive_column\",\"attributes\":{\"owner\":\"admin\",\"qualifiedName\":\"default.thursday_07_11_5.id@cm\",\"name\":\"id\",\"comment\":null,\"position\":0,\"type\":\"int\"},\"guid\":\"7b6806c1-8daf-4e0f-8d10-672a192968f1\",\"isIncomplete\":false,\"provenanceType\":0,\"version\":0,\"relationshipAttributes\":{\"table\":{\"guid\":\"c5ae4503-b498-493c-86b5-d8d909c6b16a\",\"typeName\":\"hive_table\",\"uniqueAttributes\":{\"qualifiedName\":\"default.thursday_07_11_5@cm\"},\"relationshipType\":\"hive_table_columns\"}},\"proxy\":false}",
"eventKey": "7b6806c1-8daf-4e0f-8d10-672a192968f1:1573161723180:2",
"entity": null,
"type": null
}
]


Thanks,

Sidharth Mishra



  1   2   >