Re: Review Request 72890: ATLAS-3949 : Relationship search API, add parameter to get classification attributes in search results

2020-09-21 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72890/
---

(Updated Sept. 21, 2020, 1:26 p.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3949
https://issues.apache.org/jira/browse/ATLAS-3949


Repository: atlas


Description
---

**Improvement:** Adding boolean parameter 'includeClassificationAttributes' in 
Relationship search Api to get all attributes of the classification associated 
to the entity.

**Example of api:** 
/api/atlas/v2/search/relationship?guid={guid}=tables=10=true


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java 
4d97a0859 
  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
f8143453a 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java dd9980a82 


Diff: https://reviews.apache.org/r/72890/diff/1/


Testing (updated)
---

Precommit : 
https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/27


Thanks,

Pinal Shah



Review Request 72890: ATLAS-3949 : Relationship search API, add parameter to get classification attributes in search results

2020-09-21 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72890/
---

Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3949
https://issues.apache.org/jira/browse/ATLAS-3949


Repository: atlas


Description
---

**Improvement:** Adding boolean parameter 'includeClassificationAttributes' in 
Relationship search Api to get all attributes of the classification associated 
to the entity.

**Example of api:** 
/api/atlas/v2/search/relationship?guid={guid}=tables=10=true


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java 
4d97a0859 
  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
f8143453a 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java dd9980a82 


Diff: https://reviews.apache.org/r/72890/diff/1/


Testing
---


Thanks,

Pinal Shah



Re: Review Request 72873: ATLAS-3938 : Deletion of non existing hive entities

2020-09-16 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72873/
---

(Updated Sept. 17, 2020, 4:40 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

addressed comment


Bugs: ATLAS-3938
https://issues.apache.org/jira/browse/ATLAS-3938


Repository: atlas


Description
---

**Problem:** Whenever database or table is dropped in hive, and HiveHook is not 
enabled, we dont have anyway to get the database and table sync with hive.

**Workaround:** Added support to delete hive entities in Atlas which are 
dropped in hive.

**Usage:** ./import-hive.sh -deleteNonExisting

**NOTE:** **atlas.hook.hive.page.limit** property is added to configure the 
pageSize/limit while fetching entities from Atlas


Diffs (updated)
-

  addons/hive-bridge/src/bin/import-hive.sh b1660ea65 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 24e06dc2b 
  client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 18811f8c7 


Diff: https://reviews.apache.org/r/72873/diff/2/

Changes: https://reviews.apache.org/r/72873/diff/1-2/


Testing
---

Manually tested


Thanks,

Pinal Shah



Review Request 72873: ATLAS-3938 : Deletion of non existing hive entities

2020-09-16 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72873/
---

Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3938
https://issues.apache.org/jira/browse/ATLAS-3938


Repository: atlas


Description
---

**Problem:** Whenever database or table is dropped in hive, and HiveHook is not 
enabled, we dont have anyway to get the database and table sync with hive.

**Workaround:** Added support to delete hive entities in Atlas which are 
dropped in hive.

**Usage:** ./import-hive.sh -deleteNonExisting

**NOTE:** **atlas.hook.hive.page.limit** property is added to configure the 
pageSize/limit while fetching entities from Atlas


Diffs
-

  addons/hive-bridge/src/bin/import-hive.sh b1660ea65 
  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 24e06dc2b 
  client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 18811f8c7 


Diff: https://reviews.apache.org/r/72873/diff/1/


Testing
---

Manually tested


Thanks,

Pinal Shah



Re: Review Request 72815: Deletion of non existing hive entities

2020-09-11 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72815/
---

(Updated Sept. 11, 2020, 12:27 p.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, Nixon 
Rodrigues, and Sarath Subramanian.


Changes
---

updating patch for branch-0.8


Repository: atlas


Description
---

**Problem:** Whenever database or table is dropped in hive, and HiveHook is not 
enabled, we dont have anyway to get the database and table sync with hive.

**Workaround:** Added support to delete hive entities in Atlas which are 
dropped in hive.

**Usage:** ./import-hive.sh -deleteNonExisting

**NOTE:** **atlas.hook.hive.page.limit** property is added to configure the 
pageSize/limit while fetching entities from Atlas


Diffs (updated)
-

  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 f18d01b7c 
  client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 6968e8358 


Diff: https://reviews.apache.org/r/72815/diff/4/

Changes: https://reviews.apache.org/r/72815/diff/3-4/


Testing
---

Manually tested


Thanks,

Pinal Shah



Re: Review Request 72815: Deletion of non existing hive entities

2020-09-04 Thread Pinal Shah


> On Sept. 4, 2020, 4:04 p.m., Madhan Neethiraj wrote:
> > addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
> > Lines 959 (patched)
> > <https://reviews.apache.org/r/72815/diff/2/?file=2239187#file2239187line968>
> >
> > Returning after deleting one entity? Please review.

thanks for pointing, it is incorrect


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72815/#review221806
---


On Sept. 5, 2020, 5:02 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72815/
> ---
> 
> (Updated Sept. 5, 2020, 5:02 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Problem:** Whenever database or table is dropped in hive, and HiveHook is 
> not enabled, we dont have anyway to get the database and table sync with hive.
> 
> **Workaround:** Added support to delete hive entities in Atlas which are 
> dropped in hive.
> 
> **Usage:** ./import-hive.sh -deleteNonExisting
> 
> **NOTE:** **atlas.hook.hive.page.limit** property is added to configure the 
> pageSize/limit while fetching entities from Atlas
> 
> 
> Diffs
> -
> 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
>  e659ca041 
>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 
> 6968e8358 
> 
> 
> Diff: https://reviews.apache.org/r/72815/diff/3/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72815: Deletion of non existing hive entities

2020-09-04 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72815/
---

(Updated Sept. 5, 2020, 5:02 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, Nixon 
Rodrigues, and Sarath Subramanian.


Changes
---

addressed comments


Repository: atlas


Description
---

**Problem:** Whenever database or table is dropped in hive, and HiveHook is not 
enabled, we dont have anyway to get the database and table sync with hive.

**Workaround:** Added support to delete hive entities in Atlas which are 
dropped in hive.

**Usage:** ./import-hive.sh -deleteNonExisting

**NOTE:** **atlas.hook.hive.page.limit** property is added to configure the 
pageSize/limit while fetching entities from Atlas


Diffs (updated)
-

  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 e659ca041 
  client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 6968e8358 


Diff: https://reviews.apache.org/r/72815/diff/3/

Changes: https://reviews.apache.org/r/72815/diff/2-3/


Testing
---

Manually tested


Thanks,

Pinal Shah



Re: Review Request 72815: Deletion of non existing hive entities

2020-09-03 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72815/
---

(Updated Sept. 3, 2020, 12:21 p.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, Nixon 
Rodrigues, and Sarath Subramanian.


Repository: atlas


Description (updated)
---

**Problem:** Whenever database or table is dropped in hive, and HiveHook is not 
enabled, we dont have anyway to get the database and table sync with hive.

**Workaround:** Added support to delete hive entities in Atlas which are 
dropped in hive.

**Usage:** ./import-hive.sh -deleteNonExisting

**NOTE:** **atlas.hook.hive.page.limit** property is added to configure the 
pageSize/limit while fetching entities from Atlas


Diffs
-

  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 e659ca041 
  client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 6968e8358 


Diff: https://reviews.apache.org/r/72815/diff/2/


Testing
---

Manually tested


Thanks,

Pinal Shah



Re: Review Request 72815: Deletion of non existing hive entities

2020-09-03 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72815/
---

(Updated Sept. 3, 2020, 10:26 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, Nixon 
Rodrigues, and Sarath Subramanian.


Changes
---

Addressed review comments


Repository: atlas


Description
---

**Problem:** Whenever database or table is dropped in hive, and HiveHook is not 
enabled, we dont have anyway to get the database and table sync with hive.

**Workaround:** Added support to delete hive entities in Atlas which are 
dropped in hive.

**Usage:** ./import-hive.sh -deleteNonExisting


Diffs (updated)
-

  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 e659ca041 
  client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 6968e8358 


Diff: https://reviews.apache.org/r/72815/diff/2/

Changes: https://reviews.apache.org/r/72815/diff/1-2/


Testing
---

Manually tested


Thanks,

Pinal Shah



Re: Review Request 72815: Deletion of non existing hive entities

2020-08-31 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72815/
---

(Updated Aug. 31, 2020, 8:51 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, Nixon 
Rodrigues, and Sarath Subramanian.


Summary (updated)
-

Deletion of non existing hive entities


Repository: atlas


Description
---

**Problem:** Whenever database or table is dropped in hive, and HiveHook is not 
enabled, we dont have anyway to get the database and table sync with hive.

**Workaround:** Added support to delete hive entities in Atlas which are 
dropped in hive.

**Usage:** ./import-hive.sh -deleteNonExisting


Diffs
-

  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 e659ca041 
  client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 6968e8358 


Diff: https://reviews.apache.org/r/72815/diff/1/


Testing
---

Manually tested


Thanks,

Pinal Shah



Review Request 72815: ENGESC-3520: Deletion of non existing hive entities

2020-08-28 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72815/
---

Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nikhil Bonte, Nixon 
Rodrigues, and Sarath Subramanian.


Repository: atlas


Description
---

**Problem:** Whenever database or table is dropped in hive, and HiveHook is not 
enabled, we dont have anyway to get the database and table sync with hive.

**Workaround:** Added support to delete hive entities in Atlas which are 
dropped in hive.

**Usage:** ./import-hive.sh -deleteNonExisting


Diffs
-

  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java
 e659ca041 
  client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 6968e8358 


Diff: https://reviews.apache.org/r/72815/diff/1/


Testing
---

Manually tested


Thanks,

Pinal Shah



Re: Review Request 72646: ATLAS-3876 : Relationship Search API not showing correct approximateCount

2020-08-04 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72646/
---

(Updated Aug. 5, 2020, 4:27 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3876
https://issues.apache.org/jira/browse/ATLAS-3876


Repository: atlas


Description
---

**Issue:**
Relationship api doesn't provide approximate count of the related entities in 
the response.

**Workaround:**
Get the total count of related given entity , irrespective of the offset/limit.
Added 'getApproximateCount' as queryParam in Rest api.


Also this patch includes **improvement in the time taken to fetch related 
entities**  .
Average time taken for the Api to search relationship entities having **5000 
end vertices** with limit **500**:
Before: 9seconds
After applying this patch : 3seconds


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 61ee4589f 
  
repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java 
86572598a 
  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
4b9564295 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
044179d44 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
 863a00350 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
5069d78c8 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java cdeb912a4 


Diff: https://reviews.apache.org/r/72646/diff/6/

Changes: https://reviews.apache.org/r/72646/diff/5-6/


Testing (updated)
---

Manually tested
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2084


Thanks,

Pinal Shah



Re: Review Request 72646: ATLAS-3876 : Relationship Search API not showing correct approximateCount

2020-08-04 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72646/
---

(Updated Aug. 4, 2020, 11:01 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

added support for Edge attribute predicate


Bugs: ATLAS-3876
https://issues.apache.org/jira/browse/ATLAS-3876


Repository: atlas


Description
---

**Issue:**
Relationship api doesn't provide approximate count of the related entities in 
the response.

**Workaround:**
Get the total count of related given entity , irrespective of the offset/limit.
Added 'getApproximateCount' as queryParam in Rest api.


Also this patch includes **improvement in the time taken to fetch related 
entities**  .
Average time taken for the Api to search relationship entities having **5000 
end vertices** with limit **500**:
Before: 9seconds
After applying this patch : 3seconds


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 58de4cc47 
  intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 61ee4589f 
  
repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java 
86572598a 
  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
4b9564295 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
044179d44 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
 863a00350 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
5069d78c8 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java cdeb912a4 


Diff: https://reviews.apache.org/r/72646/diff/5/

Changes: https://reviews.apache.org/r/72646/diff/4-5/


Testing
---

Manually tested
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2011 
(Failed in Impala build)


Thanks,

Pinal Shah



Re: Review Request 72646: ATLAS-3876 : Relationship Search API not showing correct approximateCount

2020-07-31 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72646/
---

(Updated July 31, 2020, 12:43 p.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

Addressed review comment


Bugs: ATLAS-3876
https://issues.apache.org/jira/browse/ATLAS-3876


Repository: atlas


Description
---

**Issue:**
Relationship api doesn't provide approximate count of the related entities in 
the response.

**Workaround:**
Get the total count of related given entity , irrespective of the offset/limit.
Added 'getApproximateCount' as queryParam in Rest api.


Also this patch includes **improvement in the time taken to fetch related 
entities**  .
Average time taken for the Api to search relationship entities having **5000 
end vertices** with limit **500**:
Before: 9seconds
After applying this patch : 3seconds


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 58de4cc47 
  intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 61ee4589f 
  
repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java 
86572598a 
  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
4b9564295 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
 863a00350 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
5069d78c8 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java cdeb912a4 


Diff: https://reviews.apache.org/r/72646/diff/4/

Changes: https://reviews.apache.org/r/72646/diff/3-4/


Testing
---

Manually tested
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2011 
(Failed in Impala build)


Thanks,

Pinal Shah



Re: Review Request 72646: ATLAS-3876 : Relationship Search API not showing correct approximateCount

2020-07-29 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72646/
---

(Updated July 29, 2020, 1:41 p.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3876
https://issues.apache.org/jira/browse/ATLAS-3876


Repository: atlas


Description (updated)
---

**Issue:**
Relationship api doesn't provide approximate count of the related entities in 
the response.

**Workaround:**
Get the total count of related given entity , irrespective of the offset/limit.
Added 'getApproximateCount' as queryParam in Rest api.


Also this patch includes **improvement in the time taken to fetch related 
entities**  .
Average time taken for the Api to search relationship entities having **5000 
end vertices** with limit **500**:
Before: 9seconds
After applying this patch : 3seconds


Diffs
-

  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 58de4cc47 
  
repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java 
86572598a 
  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
4b9564295 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
 863a00350 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
5069d78c8 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java cdeb912a4 


Diff: https://reviews.apache.org/r/72646/diff/3/


Testing
---

Manually tested
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2011 
(Failed in Impala build)


Thanks,

Pinal Shah



Re: Review Request 72646: ATLAS-3876 : Relationship Search API not showing correct approximateCount

2020-07-29 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72646/
---

(Updated July 29, 2020, 1:38 p.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

Addressed Review comments


Bugs: ATLAS-3876
https://issues.apache.org/jira/browse/ATLAS-3876


Repository: atlas


Description
---

**Issue:**
Relationship api doesn't provide approximate count of the related entities in 
the response.

**Workaround:**
Get the total count of related given entity , irrespective of the offset/limit.


Also this patch includes **improvement in the time taken to fetch related 
entities**  .
Average time taken for the Api to search relationship entities having **5000 
end vertices** with limit **500**:
Before: 9seconds
After applying this patch : 3seconds


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 58de4cc47 
  
repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java 
86572598a 
  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
4b9564295 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
 863a00350 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
5069d78c8 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java cdeb912a4 


Diff: https://reviews.apache.org/r/72646/diff/3/

Changes: https://reviews.apache.org/r/72646/diff/2-3/


Testing
---

Manually tested
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2011 
(Failed in Impala build)


Thanks,

Pinal Shah



Re: Review Request 72646: ATLAS-3876 : Relationship Search API not showing correct approximateCount

2020-07-23 Thread Pinal Shah


> On July 23, 2020, 10:50 p.m., Sarath Subramanian wrote:
> > repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
> > Line 607 (original), 616 (patched)
> > <https://reviews.apache.org/r/72646/diff/2/?file=2235580#file2235580line618>
> >
> > what if out/in edges size is a lot, do you bring everything into memory 
> > just to get size? 
> > 
> > Do you need entire edges in the list or maybe we can maintain a counter 
> > and increment?

Yes Sarath you are right but,
approximate count is count without limit/offset
I brought it to inmemory to filter out 'ACTIVE' edges incase of excludeDeleted 
flag
Other way is to fire another query with 'ACTIVE'filter


> On July 23, 2020, 10:50 p.m., Sarath Subramanian wrote:
> > repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
> > Lines 682 (patched)
> > <https://reviews.apache.org/r/72646/diff/2/?file=2235580#file2235580line703>
> >
> > Constants.GUID_PROPERTY_KEY).value() can be null (mostly not, but some 
> > bad vertices). Consider checking for null value of guid.
> > 
> > Also, since you already have vertex, do we need additional 
> > 'endVerticesGuid' list? Can we extract 'guid' from Vertex 'v' and add it to 
> > 'resultList' is same loop. 
> > 
> > resultList.add(entityRetriever.toAtlasEntityHeader(endVertexGuid, 
> > attributes));
> > 
> > (or)
> > 
> > Maybe map Vertex to AtlasVertex and directly call:
> > 
> > resultList.add(entityRetriever.toAtlasEntityHeader(atlasVertex, 
> > attributes));

I have added  (v != null && 
v.property(Constants.GUID_PROPERTY_KEY).isPresent()) for the null check
Correct, we dont need extra list, will address it


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72646/#review221343
---


On July 6, 2020, 9:14 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72646/
> ---
> 
> (Updated July 6, 2020, 9:14 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3876
> https://issues.apache.org/jira/browse/ATLAS-3876
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Issue:**
> Relationship api doesn't provide approximate count of the related entities in 
> the response.
> 
> **Workaround:**
> Get the total count of related given entity , irrespective of the 
> offset/limit.
> 
> 
> Also this patch includes **improvement in the time taken to fetch related 
> entities**  .
> Average time taken for the Api to search relationship entities having **5000 
> end vertices** with limit **500**:
> Before: 9seconds
> After applying this patch : 3seconds
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  4b9564295 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
>  863a00350 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> 5069d78c8 
> 
> 
> Diff: https://reviews.apache.org/r/72646/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested
> Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2011 
> (Failed in Impala build)
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72646: ATLAS-3876 : Relationship Search API not showing correct approximateCount

2020-07-21 Thread Pinal Shah


> On July 20, 2020, 10:54 p.m., Ashutosh Mestry wrote:
> > repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
> > Line 649 (original), 662 (patched)
> > <https://reviews.apache.org/r/72646/diff/2/?file=2235580#file2235580line664>
> >
> > Consider replacing this with AtlasGraphTraversal. This should be easy 
> > change but will give good performance mileage.

Thanks Ashutosh for reviewing.

You mean to add these methods, (bothE, otherV) to AtlasGraphTraversal and make 
use from it?
How will it make difference in performance?


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72646/#review221273
---


On July 6, 2020, 9:14 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72646/
> ---
> 
> (Updated July 6, 2020, 9:14 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3876
> https://issues.apache.org/jira/browse/ATLAS-3876
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Issue:**
> Relationship api doesn't provide approximate count of the related entities in 
> the response.
> 
> **Workaround:**
> Get the total count of related given entity , irrespective of the 
> offset/limit.
> 
> 
> Also this patch includes **improvement in the time taken to fetch related 
> entities**  .
> Average time taken for the Api to search relationship entities having **5000 
> end vertices** with limit **500**:
> Before: 9seconds
> After applying this patch : 3seconds
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  4b9564295 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
>  863a00350 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> 5069d78c8 
> 
> 
> Diff: https://reviews.apache.org/r/72646/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested
> Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2011 
> (Failed in Impala build)
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72646: ATLAS-3876 : Relationship Search API not showing correct approximateCount

2020-07-21 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72646/#review221282
---




repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
Lines 705 (patched)
<https://reviews.apache.org/r/72646/#comment310162>

I tested it with 5000 edge vertices.
Will vertexTotal() method be efficient than this?


- Pinal Shah


On July 6, 2020, 9:14 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72646/
> ---
> 
> (Updated July 6, 2020, 9:14 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3876
> https://issues.apache.org/jira/browse/ATLAS-3876
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Issue:**
> Relationship api doesn't provide approximate count of the related entities in 
> the response.
> 
> **Workaround:**
> Get the total count of related given entity , irrespective of the 
> offset/limit.
> 
> 
> Also this patch includes **improvement in the time taken to fetch related 
> entities**  .
> Average time taken for the Api to search relationship entities having **5000 
> end vertices** with limit **500**:
> Before: 9seconds
> After applying this patch : 3seconds
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  4b9564295 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
>  863a00350 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> 5069d78c8 
> 
> 
> Diff: https://reviews.apache.org/r/72646/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested
> Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2011 
> (Failed in Impala build)
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72646: ATLAS-3876 : Relationship Search API not showing correct approximateCount

2020-07-21 Thread Pinal Shah


> On July 20, 2020, 10:54 p.m., Ashutosh Mestry wrote:
> > repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
> > Lines 705 (patched)
> > <https://reviews.apache.org/r/72646/diff/2/?file=2235580#file2235580line732>
> >
> > What is the volume of data you have tested this with?

I tested it with 5000 edge vertices.
Will vertexTotal() method be efficient than this?


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72646/#review221273
-------


On July 6, 2020, 9:14 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72646/
> ---
> 
> (Updated July 6, 2020, 9:14 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3876
> https://issues.apache.org/jira/browse/ATLAS-3876
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Issue:**
> Relationship api doesn't provide approximate count of the related entities in 
> the response.
> 
> **Workaround:**
> Get the total count of related given entity , irrespective of the 
> offset/limit.
> 
> 
> Also this patch includes **improvement in the time taken to fetch related 
> entities**  .
> Average time taken for the Api to search relationship entities having **5000 
> end vertices** with limit **500**:
> Before: 9seconds
> After applying this patch : 3seconds
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  4b9564295 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
>  863a00350 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> 5069d78c8 
> 
> 
> Diff: https://reviews.apache.org/r/72646/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested
> Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2011 
> (Failed in Impala build)
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: [VOTE] Release Apache Atlas version 2.1.0 - rc3

2020-07-13 Thread Pinal Shah
+1 for Apache Atlas 2.1.0 rc3 release

Validated below:
- Build successfully with embedded-hbase-solr.
- Creating entities and classification.
- Basic/Quick search
- Creating Labels and search for them.
- Creating custom attributes and search for them.

Thanks,
Pinal Shah


On Mon, Jul 13, 2020 at 11:00 AM Chaitali Borole <
chaitali.bor...@freestoneinfotech.com> wrote:

> Hello,
>
> +1 for Apache Atlas 2.1.0 - rc3 release
>
> 1. Successful build with -Pdist,embedded-hbase-solr.
> 2. Started Atlas server successfully and quick_Start works too..
> 3. Verified all entities getting tagged successfully also tags are created.
> 4. Successfully created entities, relationships.
> 5. Validated Basic and advanced search.
>
> Thanks,
> Chaitali Borole.
>
> On Fri, Jul 10, 2020 at 10:29 AM Madhan Neethiraj 
> wrote:
>
> > Atlas team,
> >
> > Apache Atlas 2.1.0 rc3, with following fixes since rc2, is now available
> > for vote within dev community.
> >   ATLAS-3877: Purged entity audit throws 404 Entity not found error
> >   ATLAS-3882: Classic UI: Loader not disappearing while assigning
> category
> > to term
> >
> > Links to the release artifacts are given below. Please review and vote.
> >
> > The vote will be open for at least 72 hours or until necessary votes are
> > reached.
> >   [ ] +1 Approve
> >   [ ] +0 No opinion
> >   [ ] -1 Disapprove (and reason why)
> >
> > Thanks,
> > Madhan
> >
> >
> > List of all issues addressed in this release:
> > https://issues.apache.org/jira/issues/?jql=project=ATLAS AND
> > status=Resolved AND fixVersion=2.1.0 ORDER BY key DESC
> >
> > Git tag for the release:
> > https://github.com/apache/atlas/tree/release-2.1.0-rc3
> > Sources for the release:
> >
> https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc3/apache-atlas-2.1.0-sources.tar.gz
> >
> > Source release verification:
> >   PGP Signature:
> >
> https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc3/apache-atlas-2.1.0-sources.tar.gz.asc
> >   SHA512 Hash:
> >
> https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc3/apache-atlas-2.1.0-sources.tar.gz.sha512
> >   MD5 Hash:
> >
> https://dist.apache.org/repos/dist/dev/atlas/2.1.0-rc3/apache-atlas-2.1.0-sources.tar.gz.md5
> >
> > Keys to verify the signature of the release artifacts are available at:
> > https://dist.apache.org/repos/dist/release/atlas/KEYS
> >
> > New features/enhancements:
> >   - Quick Search: provides a simpler search experience with type-ahead
> > suggestions
> >   - Business Metadata: enables augmenting entity-types with additional
> > attributes, search entities using these attributes
> >   - Labels: ability to add/remove labels on entities, and search entities
> > using labels
> >   - Custom Attributes: ability to add entity instance specific custom
> > attributes i.e. attributes not defined in entity-def or business metadata
> >   - Entity Purge: added REST APIs to purge deleted entities
> >
> > Enhancements:
> >   - Search: ability to find entities by more than one classification
> >   - Performance: improvements in lineage retrieval and
> > classification-propagation
> >   - Notification: ability to process notifications from multiple Kafka
> > topics
> >   - Hive Hook: tracks process-executions via hive_process_execution
> > entities
> >   - Hive Hook: captures DDL operations via hive_db_ddl and hive_table_ddl
> > entities
> >   - Notification: introduced shell entities to record references to
> > non-existing entities in notifications
> >   - Spark: added model to capture Spark entities, processes and
> > relationships
> >   - AWS S3: introduced updated model to capture AWS S3 entities and
> > relationships
> >   - ADLS-Gen2: introduced model to capture Azure Data Lake Storage Gen2
> > entities and relationships
> >   - Dependencies: JanusGraph 0.5.1, Tinkerpop 3.4.6, Spring Framework
> > 4.3.20
> >   - Authorization: updated to cover new features, like: business
> metadata,
> > labels, purge
> >   - UI: multiple UI improvements, including a beta UI
> >
> >
> >
>


Review Request 72669: ATLAS-3888 : BasicSearch: Multiple type/tag: [Regression], Restrict when tag filters and regex in tag

2020-07-10 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72669/
---

Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3888
https://issues.apache.org/jira/browse/ATLAS-3888


Repository: atlas


Description
---

**Issue:**
In Basic search ,
tag : PII_*
tag filter : sensitivity < 0.5 
returns 200 OK with no entities in response though there are entities tagged 
with PII_1 with sensitivity < 0.5

**Expected:** 400 Bad request , as tag filters are not allowed when the tag has 
regex.

**Note:** This patch will only resolve tags having regex Eg. "PII_*" not "*", * 
can have System attribute filters.


Diffs
-

  repository/src/main/java/org/apache/atlas/discovery/SearchContext.java 
dd24a8cc4 


Diff: https://reviews.apache.org/r/72669/diff/1/


Testing
---

Manually Tested
Precommit :


Thanks,

Pinal Shah



Re: Review Request 72651: ATLAS-3880 : BasicSearch: Multiple type/tag: Log invalid and allow searching valid type/tag names

2020-07-08 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72651/
---

(Updated July 8, 2020, 4:26 p.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3880
https://issues.apache.org/jira/browse/ATLAS-3880


Repository: atlas


Description
---

**Issue:**
If in basic search, request has combination of valid and invalid typename/tags, 
it throws "errorCode": "ATLAS-400-00-00A" unknown/invalid typeName for the 
invalid typeName.

**WorkAround:**
Log invalid typeNames/tagNames and allow searching for valid

**Example:**
typeName:"valid,invalid"
Allow searching entities for 'valid' type.


Diffs
-

  repository/src/main/java/org/apache/atlas/discovery/SearchContext.java 
04e82186b 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 b7ce97845 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java 140ed9a6e 


Diff: https://reviews.apache.org/r/72651/diff/2/


Testing (updated)
---

Manually tested
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2033 (In 
progress)


Thanks,

Pinal Shah



Review Request 72651: ATLAS-3880 : BasicSearch: Multiple type/tag: Log invalid and allow searching valid type/tag names

2020-07-07 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72651/
---

Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3880
https://issues.apache.org/jira/browse/ATLAS-3880


Repository: atlas


Description
---

**Issue:**
If in basic search, request has combination of valid and invalid typename/tags, 
it throws "errorCode": "ATLAS-400-00-00A" unknown/invalid typeName for the 
invalid typeName.

**WorkAround:**
Log invalid typeNames/tagNames and allow searching for valid

**Example:**
typeName:"valid,invalid"
Allow searching entities for 'valid' type.


Diffs
-

  repository/src/main/java/org/apache/atlas/discovery/SearchContext.java 
04e82186b 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java 140ed9a6e 


Diff: https://reviews.apache.org/r/72651/diff/1/


Testing
---

Manually tested
Precommit :


Thanks,

Pinal Shah



Re: Review Request 72646: ATLAS-3876 : Relationship Search API not showing correct approximateCount

2020-07-06 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72646/
---

(Updated July 6, 2020, 9:14 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3876
https://issues.apache.org/jira/browse/ATLAS-3876


Repository: atlas


Description
---

**Issue:**
Relationship api doesn't provide approximate count of the related entities in 
the response.

**Workaround:**
Get the total count of related given entity , irrespective of the offset/limit.


Also this patch includes **improvement in the time taken to fetch related 
entities**  .
Average time taken for the Api to search relationship entities having **5000 
end vertices** with limit **500**:
Before: 9seconds
After applying this patch : 3seconds


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
4b9564295 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
 863a00350 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
5069d78c8 


Diff: https://reviews.apache.org/r/72646/diff/1/


Testing (updated)
---

Manually tested
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2011 
(Failed in Impala build)


Thanks,

Pinal Shah



Review Request 72646: ATLAS-3876 : Relationship Search API not showing correct approximateCount

2020-07-06 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72646/
---

Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3876
https://issues.apache.org/jira/browse/ATLAS-3876


Repository: atlas


Description
---

**Issue:**
Relationship api doesn't provide approximate count of the related entities in 
the response.

**Workaround:**
Get the total count of related given entity , irrespective of the offset/limit.


Also this patch includes **improvement in the time taken to fetch related 
entities**  .
Average time taken for the Api to search relationship entities having **5000 
end vertices** with limit **500**:
Before: 9seconds
After applying this patch : 3seconds


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
4b9564295 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
 863a00350 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
5069d78c8 


Diff: https://reviews.apache.org/r/72646/diff/1/


Testing
---

Manually tested
Precommit :


Thanks,

Pinal Shah



Re: Review Request 72627: ATLAS-3867 : Relationship search API should have a provision to fetch custom attributes in search results

2020-06-30 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72627/
---

(Updated July 1, 2020, 5:26 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

rebase patch


Bugs: ATLAS-3867
https://issues.apache.org/jira/browse/ATLAS-3867


Repository: atlas


Description
---

**Issue:** Attributes in search result of Relationship api are minimal.

**WorkAround:** Relationship api request should have provision to specify 
attributes to be present in search result.

**Example Request:** 
/v2/search/relationship?guid=ac9e04cc-f927-4334-af08-c83bc3733f5b=columns=name=ASCENDING=dcProfiledData


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java 
e64c31522 
  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
4169e14ef 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java 076284ec4 


Diff: https://reviews.apache.org/r/72627/diff/2/

Changes: https://reviews.apache.org/r/72627/diff/1-2/


Testing
---

Manually tested
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1994


Thanks,

Pinal Shah



Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-06-30 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/
---

(Updated June 30, 2020, 1:09 p.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3782
https://issues.apache.org/jira/browse/ATLAS-3782


Repository: atlas


Description
---

**Issue:**
The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
implemented in SearchProcessors. It would allow a search of entities that do 
not contain in given string in a specified attribute (eg exclude entities from 
search that contain 'temp' in the qualified name). 

**WorkAround:**
JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
inmemory
BasicSearch generates query via three modes
1. Index query -> NOT_CONTAINS will not be supported
2. InMemeory Predicates -> NOT_CONTAINS will be supported, Already handled #123 
SearchProcessor
3. Graph query -> NOT_CONTAINS will not be supported

As in index and graph query wiil not support not_Contains operator, We need to 
apply filter(inMemoryPredicate) after either index/graph query.

To support above, I have modified ClassificationSearchProcessor
+ For both cases index as well as graph, added typeNamePredicate and 
attributePredicate
+ Added these predicate after query
- Removed gremlinQuery block

**Operator Value:**
It can be either "not_contains" or "NOT_CONTAINS"

**Note:**
As part of this jira, not_contains is also added in quick search.


Diffs
-

  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasSolrQueryBuilder.java
 6c06a3cbe 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 9c72cd4a2 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
c9a605355 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
  
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 9b16e919d 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 b7ce97845 


Diff: https://reviews.apache.org/r/72567/diff/4/


Testing (updated)
---

Added testcases
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1999


Thanks,

Pinal Shah



Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-06-30 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/
---

(Updated June 30, 2020, 10:04 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3782
https://issues.apache.org/jira/browse/ATLAS-3782


Repository: atlas


Description (updated)
---

**Issue:**
The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
implemented in SearchProcessors. It would allow a search of entities that do 
not contain in given string in a specified attribute (eg exclude entities from 
search that contain 'temp' in the qualified name). 

**WorkAround:**
JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
inmemory
BasicSearch generates query via three modes
1. Index query -> NOT_CONTAINS will not be supported
2. InMemeory Predicates -> NOT_CONTAINS will be supported, Already handled #123 
SearchProcessor
3. Graph query -> NOT_CONTAINS will not be supported

As in index and graph query wiil not support not_Contains operator, We need to 
apply filter(inMemoryPredicate) after either index/graph query.

To support above, I have modified ClassificationSearchProcessor
+ For both cases index as well as graph, added typeNamePredicate and 
attributePredicate
+ Added these predicate after query
- Removed gremlinQuery block

**Operator Value:**
It can be either "not_contains" or "NOT_CONTAINS"

**Note:**
As part of this jira, not_contains is also added in quick search.


Diffs
-

  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasSolrQueryBuilder.java
 6c06a3cbe 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 9c72cd4a2 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
c9a605355 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
  
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 9b16e919d 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 b7ce97845 


Diff: https://reviews.apache.org/r/72567/diff/4/


Testing
---

Added testcases
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1983


Thanks,

Pinal Shah



Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-06-30 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/
---

(Updated June 30, 2020, 9:54 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

Rebase patch and added not_contains in quicksearch aggregation metrics


Bugs: ATLAS-3782
https://issues.apache.org/jira/browse/ATLAS-3782


Repository: atlas


Description
---

The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
implemented in SearchProcessors. It would allow a search of entities that do 
not contain in given string in a specified attribute (eg exclude entities from 
search that contain 'temp' in the qualified name). 

JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
inmemory
BasicSearch can generates query via three modes
1. Index query -> NOT_CONTAINS will not be supported
2. InMemeory Predicated -> NOT_CONTAINS will be supported, Already handled #123 
SearchProcessor
3. Graph query -> NOT_CONTAINS will not be supported

We need to apply filter(inMemoryPredicate) after either index/graph query.

To support above, i have modified ClassificationSearchProcessor
+ For both cases index as well as graph, added typeNamePredicate and 
attributePredicate
+ Added these predicate after query
- Removed gremlinQuery block


Diffs (updated)
-

  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasSolrQueryBuilder.java
 6c06a3cbe 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 9c72cd4a2 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
c9a605355 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
  
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 9b16e919d 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 b7ce97845 


Diff: https://reviews.apache.org/r/72567/diff/4/

Changes: https://reviews.apache.org/r/72567/diff/3-4/


Testing
---

Added testcases
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1983


Thanks,

Pinal Shah



Review Request 72627: ATLAS-3867 : Relationship search API should have a provision to fetch custom attributes in search results

2020-06-29 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72627/
---

Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3867
https://issues.apache.org/jira/browse/ATLAS-3867


Repository: atlas


Description
---

**Issue:** Attributes in search result of Relationship api are minimal.

**WorkAround:** Relationship api request should have provision to specify 
attributes to be present in search result.

**Example Request:** 
/v2/search/relationship?guid=ac9e04cc-f927-4334-af08-c83bc3733f5b=columns=name=ASCENDING=dcProfiledData


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java 
e64c31522 
  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
b2737af01 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java 076284ec4 


Diff: https://reviews.apache.org/r/72627/diff/1/


Testing
---

Manually tested
Precommit :


Thanks,

Pinal Shah



Re: Review Request 72625: ATLAS-3866 : Relationship search API for hive storage desc throws error code 500

2020-06-29 Thread Pinal Shah


> On June 29, 2020, 6:29 a.m., Sarath Subramanian wrote:
> > repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
> > Lines 607 (patched)
> > <https://reviews.apache.org/r/72625/diff/1/?file=2235134#file2235134line607>
> >
> > Do we really need to compare edge end points?
> > 
> > can we do:
> > 
> > once you get relationshipEdge, you can get the typeName of the other 
> > vertex (not entityVertex - comparing vertexId). 
> > 
> > From this typeName - for e.g. 'hive_storagedesc' you can get entityType
> > 
> > relationshipEndEntityType = 
> > typeRegistry.getEntityTypeByName(otherEntityTypeName);

Thanks Sarath,
Yes we can do the way you suggested


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72625/#review221085
---


On June 29, 2020, 10:49 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72625/
> ---
> 
> (Updated June 29, 2020, 10:49 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3866
> https://issues.apache.org/jira/browse/ATLAS-3866
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Issue:** In relationship api v2/relationship, for hive_storagedesc it 
> throws exception
> 
> **Reason:** it is because, sort attribute is assigned 'name' as default, and 
> 'name' attribute is not in the defination of hive_storagedesc.
> 
> **Workaround:** Validate if attribute is present in relationship end def, if 
> not ignore sorting
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  dd4d1b441 
> 
> 
> Diff: https://reviews.apache.org/r/72625/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested
> Precommit : 
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1992/console
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72625: ATLAS-3866 : Relationship search API for hive storage desc throws error code 500

2020-06-29 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72625/
---

(Updated June 29, 2020, 10:49 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

addressed comment


Bugs: ATLAS-3866
https://issues.apache.org/jira/browse/ATLAS-3866


Repository: atlas


Description
---

**Issue:** In relationship api v2/relationship, for hive_storagedesc it throws 
exception

**Reason:** it is because, sort attribute is assigned 'name' as default, and 
'name' attribute is not in the defination of hive_storagedesc.

**Workaround:** Validate if attribute is present in relationship end def, if 
not ignore sorting


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
dd4d1b441 


Diff: https://reviews.apache.org/r/72625/diff/2/

Changes: https://reviews.apache.org/r/72625/diff/1-2/


Testing (updated)
---

Manually tested
Precommit : 
https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1992/console


Thanks,

Pinal Shah



Review Request 72625: ATLAS-3866 : Relationship search API for hive storage desc throws error code 500

2020-06-28 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72625/
---

Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3866
https://issues.apache.org/jira/browse/ATLAS-3866


Repository: atlas


Description
---

**Issue:** In relationship api v2/relationship, for hive_storagedesc it throws 
exception

**Reason:** it is because, sort attribute is assigned 'name' as default, and 
'name' attribute is not in the defination of hive_storagedesc.

**Workaround:** Validate if attribute is present in relationship end def, if 
not ignore sorting


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
dd4d1b441 


Diff: https://reviews.apache.org/r/72625/diff/1/


Testing
---

Manually tested
Precommit :


Thanks,

Pinal Shah



Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-06-25 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/
---

(Updated June 25, 2020, 9:17 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

commented one testcase, need to look, succesful on local but failed for 
precommit


Bugs: ATLAS-3782
https://issues.apache.org/jira/browse/ATLAS-3782


Repository: atlas


Description
---

The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
implemented in SearchProcessors. It would allow a search of entities that do 
not contain in given string in a specified attribute (eg exclude entities from 
search that contain 'temp' in the qualified name). 

JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
inmemory
BasicSearch can generates query via three modes
1. Index query -> NOT_CONTAINS will not be supported
2. InMemeory Predicated -> NOT_CONTAINS will be supported, Already handled #123 
SearchProcessor
3. Graph query -> NOT_CONTAINS will not be supported

We need to apply filter(inMemoryPredicate) after either index/graph query.

To support above, i have modified ClassificationSearchProcessor
+ For both cases index as well as graph, added typeNamePredicate and 
attributePredicate
+ Added these predicate after query
- Removed gremlinQuery block


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 5dd0d7f97 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
ad48be11b 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
  
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 9b16e919d 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 43f11d1df 


Diff: https://reviews.apache.org/r/72567/diff/3/

Changes: https://reviews.apache.org/r/72567/diff/2-3/


Testing
---

Added testcases
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952


Thanks,

Pinal Shah



Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-06-25 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/
---

(Updated June 25, 2020, 6:57 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

rebase patch, to update indexes from solrConfig.xml and removed one log 
statement


Bugs: ATLAS-3782
https://issues.apache.org/jira/browse/ATLAS-3782


Repository: atlas


Description
---

The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
implemented in SearchProcessors. It would allow a search of entities that do 
not contain in given string in a specified attribute (eg exclude entities from 
search that contain 'temp' in the qualified name). 

JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
inmemory
BasicSearch can generates query via three modes
1. Index query -> NOT_CONTAINS will not be supported
2. InMemeory Predicated -> NOT_CONTAINS will be supported, Already handled #123 
SearchProcessor
3. Graph query -> NOT_CONTAINS will not be supported

We need to apply filter(inMemoryPredicate) after either index/graph query.

To support above, i have modified ClassificationSearchProcessor
+ For both cases index as well as graph, added typeNamePredicate and 
attributePredicate
+ Added these predicate after query
- Removed gremlinQuery block


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 5dd0d7f97 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
ad48be11b 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
  
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 9b16e919d 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 43f11d1df 


Diff: https://reviews.apache.org/r/72567/diff/2/

Changes: https://reviews.apache.org/r/72567/diff/1-2/


Testing
---

Added testcases
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952


Thanks,

Pinal Shah



Re: Review Request 72610: ATLAS-3850 : Regression, Terms are not getting displayed

2020-06-23 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72610/
---

(Updated June 23, 2020, 11:49 a.m.)


Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
Nixon Rodrigues, and Sarath Subramanian.


Bugs: ATLAS-3850
https://issues.apache.org/jira/browse/ATLAS-3850


Repository: atlas


Description (updated)
---

During basic search, terms(meanings) in the entity response header is empty, 
because while fetching terms from the vertex edge, term label is wrong

**Solution: Removed "__" prefix from the term edge label while fetching.**

TO-DO: Find out reason or remove this line
GraphBackedSearchIndexer.initialize(){
 --> createEdgeLabel(management, Constants.TERM_ASSIGNMENT_LABEL);
}

Added below index field names for FreetextSearchProcessor
"Referenceable.qualifiedName" -> "3eh1_t"
"__labels" -> "yrp_t"
"__propagatedClassificationNames" -> "m4l_t"
"__classificationNames" -> "kjp_t"
"__classificationsText" -> "iyt_t"
"__typeName" -> "cn9_t"
"__customAttributes" -> "x6t_t"
"__state" -> "hdx_t"
"Asset.__s_name" -> "3ksl_s"
"Asset.__s_owner" -> "3qbp_s"
"Asset.__s_displayName" -> "3mdh_s"
"Asset.__s_userDescription" -> "3oqt_s"
"Asset.description" -> "3nyd_t"
"hive_db.clusterName" -> "7uv9_t"
"hive_table.tableType" -> "7ldx_t"


Diffs
-

  
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
 4a09b08a4 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
 757fcb116 
  test-tools/src/main/resources/solr/core-template/solrconfig.xml ef74a0a45 


Diff: https://reviews.apache.org/r/72610/diff/3/


Testing
---

Precommit : 
https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1971/console


Thanks,

Pinal Shah



Re: Review Request 72610: ATLAS-3850 : Regression, Terms are not getting displayed

2020-06-23 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72610/
---

(Updated June 23, 2020, 11:47 a.m.)


Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
Nixon Rodrigues, and Sarath Subramanian.


Changes
---

Update index field names for testcases


Bugs: ATLAS-3850
https://issues.apache.org/jira/browse/ATLAS-3850


Repository: atlas


Description
---

During basic search, terms(meanings) in the entity response header is empty, 
because while fetching terms from the vertex edge, term label is wrong

**Solution: Removed "__" prefix from the term edge label while fetching.**

TO-DO: Find out reason or remove this line
GraphBackedSearchIndexer.initialize(){
 --> createEdgeLabel(management, Constants.TERM_ASSIGNMENT_LABEL);
}


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
 4a09b08a4 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
 757fcb116 
  test-tools/src/main/resources/solr/core-template/solrconfig.xml ef74a0a45 


Diff: https://reviews.apache.org/r/72610/diff/3/

Changes: https://reviews.apache.org/r/72610/diff/2-3/


Testing
---

Precommit : 
https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1971/console


Thanks,

Pinal Shah



Re: Review Request 72610: ATLAS-3850 : Regression, Terms are not getting displayed

2020-06-22 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72610/
---

(Updated June 22, 2020, 12:08 p.m.)


Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
Nixon Rodrigues, and Sarath Subramanian.


Bugs: ATLAS-3850
https://issues.apache.org/jira/browse/ATLAS-3850


Repository: atlas


Description
---

During basic search, terms(meanings) in the entity response header is empty, 
because while fetching terms from the vertex edge, term label is wrong

**Solution: Removed "__" prefix from the term edge label while fetching.**

TO-DO: Find out reason or remove this line
GraphBackedSearchIndexer.initialize(){
 --> createEdgeLabel(management, Constants.TERM_ASSIGNMENT_LABEL);
}


Diffs
-

  
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
 4a09b08a4 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
 757fcb116 


Diff: https://reviews.apache.org/r/72610/diff/2/


Testing (updated)
---

Precommit : 
https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1971/console


Thanks,

Pinal Shah



Re: Review Request 72610: ATLAS-3850 : Regression, Terms are not getting displayed

2020-06-22 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72610/
---

(Updated June 22, 2020, 9:23 a.m.)


Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
Nixon Rodrigues, and Sarath Subramanian.


Bugs: ATLAS-3850
https://issues.apache.org/jira/browse/ATLAS-3850


Repository: atlas


Description
---

During basic search, terms(meanings) in the entity response header is empty, 
because while fetching terms from the vertex edge, term label is wrong

**Solution: Removed "__" prefix from the term edge label while fetching.**

TO-DO: Find out reason or remove this line
GraphBackedSearchIndexer.initialize(){
 --> createEdgeLabel(management, Constants.TERM_ASSIGNMENT_LABEL);
}


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
 4a09b08a4 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
 757fcb116 


Diff: https://reviews.apache.org/r/72610/diff/2/

Changes: https://reviews.apache.org/r/72610/diff/1-2/


Testing
---


Thanks,

Pinal Shah



Review Request 72610: ATLAS-3850 : Regression, Terms are not getting displayed

2020-06-22 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72610/
---

Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
Nixon Rodrigues, and Sarath Subramanian.


Bugs: ATLAS-3850
https://issues.apache.org/jira/browse/ATLAS-3850


Repository: atlas


Description
---

During basic search, terms(meanings) in the entity response header is empty, 
because while fetching terms from the vertex edge, term label is wrong

**Solution: Removed "__" prefix from the term edge label while fetching.**

TO-DO: Find out reason or remove this line
GraphBackedSearchIndexer.initialize(){
 --> createEdgeLabel(management, Constants.TERM_ASSIGNMENT_LABEL);
}


Diffs
-

  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
 757fcb116 


Diff: https://reviews.apache.org/r/72610/diff/1/


Testing
---


Thanks,

Pinal Shah



Re: Review Request 72486: ATLAS-3780 : Change 'Status' from Classification System Attributes to EntityStatus

2020-06-08 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72486/
---

(Updated June 8, 2020, 3:14 p.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

Renamed status variable


Bugs: ATLAS-3780
https://issues.apache.org/jira/browse/ATLAS-3780


Repository: atlas


Description
---

We are having 'State' Attribute for Status i.e ACTIVE and DELETED.
Whenever Entity is Deleted,'state' is always ACTIVE for Classification vertices 
but 'entityState' gets reflected as 'DELETED'.

For this UI, needs to handle '_entityStatus' as enum and whenever DELETED is 
selected, showHistoricalEntities needs to be true.


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 
4d0179c44 
  intg/src/main/java/org/apache/atlas/type/Constants.java 3fc13056e 


Diff: https://reviews.apache.org/r/72486/diff/3/

Changes: https://reviews.apache.org/r/72486/diff/2-3/


Testing
---

Manually tested


Thanks,

Pinal Shah



Re: Review Request 72486: ATLAS-3780 : Change 'Status' from Classification System Attributes to EntityStatus

2020-06-08 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72486/
---

(Updated June 8, 2020, 10:48 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

Moved ClassificationSearchProcessor changes, fixed and added test cases in 
https://reviews.apache.org/r/72567/


Bugs: ATLAS-3780
https://issues.apache.org/jira/browse/ATLAS-3780


Repository: atlas


Description
---

We are having 'State' Attribute for Status i.e ACTIVE and DELETED.
Whenever Entity is Deleted,'state' is always ACTIVE for Classification vertices 
but 'entityState' gets reflected as 'DELETED'.

For this UI, needs to handle '_entityStatus' as enum and whenever DELETED is 
selected, showHistoricalEntities needs to be true.


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 
4d0179c44 
  intg/src/main/java/org/apache/atlas/type/Constants.java 3fc13056e 


Diff: https://reviews.apache.org/r/72486/diff/2/

Changes: https://reviews.apache.org/r/72486/diff/1-2/


Testing
---

Manually tested


Thanks,

Pinal Shah



Re: Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-06-04 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/
---

(Updated June 4, 2020, 12:05 p.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3782
https://issues.apache.org/jira/browse/ATLAS-3782


Repository: atlas


Description
---

The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
implemented in SearchProcessors. It would allow a search of entities that do 
not contain in given string in a specified attribute (eg exclude entities from 
search that contain 'temp' in the qualified name). 

JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
inmemory
BasicSearch can generates query via three modes
1. Index query -> NOT_CONTAINS will not be supported
2. InMemeory Predicated -> NOT_CONTAINS will be supported, Already handled #123 
SearchProcessor
3. Graph query -> NOT_CONTAINS will not be supported

We need to apply filter(inMemoryPredicate) after either index/graph query.

To support above, i have modified ClassificationSearchProcessor
+ For both cases index as well as graph, added typeNamePredicate and 
attributePredicate
+ Added these predicate after query
- Removed gremlinQuery block


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 5dd0d7f97 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
2c5f925c7 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
  
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 9b16e919d 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 43f11d1df 
  test-tools/src/main/resources/solr/core-template/solrconfig.xml ef74a0a45 


Diff: https://reviews.apache.org/r/72567/diff/1/


Testing (updated)
---

Added testcases
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1952


Thanks,

Pinal Shah



Review Request 72567: ATLAS-3782 : Support NOT_CONTAINS operator in basic search

2020-06-04 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72567/
---

Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3782
https://issues.apache.org/jira/browse/ATLAS-3782


Repository: atlas


Description
---

The operator 'SearchParameters.Operator.NOT_CONTAINS' is defined and 
implemented in SearchProcessors. It would allow a search of entities that do 
not contain in given string in a specified attribute (eg exclude entities from 
search that contain 'temp' in the qualified name). 

JanusGraph doesn't allow NOT_CONTAINS operator. So we will handle this in 
inmemory
BasicSearch can generates query via three modes
1. Index query -> NOT_CONTAINS will not be supported
2. InMemeory Predicated -> NOT_CONTAINS will be supported, Already handled #123 
SearchProcessor
3. Graph query -> NOT_CONTAINS will not be supported

We need to apply filter(inMemoryPredicate) after either index/graph query.

To support above, i have modified ClassificationSearchProcessor
+ For both cases index as well as graph, added typeNamePredicate and 
attributePredicate
+ Added these predicate after query
- Removed gremlinQuery block


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 5dd0d7f97 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
2c5f925c7 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 8b98b3990 
  
repository/src/test/java/org/apache/atlas/discovery/AtlasDiscoveryServiceTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 9b16e919d 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 43f11d1df 
  test-tools/src/main/resources/solr/core-template/solrconfig.xml ef74a0a45 


Diff: https://reviews.apache.org/r/72567/diff/1/


Testing
---

Added testcases
Precommit :


Thanks,

Pinal Shah



Re: Review Request 72562: ATLAS-3823 : BasicSearch: Warning for the Unsupported operator.

2020-06-03 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72562/
---

(Updated June 3, 2020, 6:58 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3823
https://issues.apache.org/jira/browse/ATLAS-3823


Repository: atlas


Description
---

For Some operator like notNull, isNull, there is incorrect warning message in 
logs 'unsupported operator'.
This log is in preprocessing step (processPipeSeperatedSystemAttributes), So it 
is removed.


Diffs
-

  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
804c69431 


Diff: https://reviews.apache.org/r/72562/diff/1/


Testing (updated)
---

PreCommit: https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1948/console


Thanks,

Pinal Shah



Review Request 72562: ATLAS-3823 : BasicSearch: Warning for the Unsupported operator.

2020-06-03 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72562/
---

Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3823
https://issues.apache.org/jira/browse/ATLAS-3823


Repository: atlas


Description
---

For Some operator like notNull, isNull, there is incorrect warning message in 
logs 'unsupported operator'.
This log is in preprocessing step (processPipeSeperatedSystemAttributes), So it 
is removed.


Diffs
-

  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
804c69431 


Diff: https://reviews.apache.org/r/72562/diff/1/


Testing
---


Thanks,

Pinal Shah



Re: Review Request 72533: ATLAS-3802 : BasicSearch: filter, attribute with NEQ operator doesn't match attribute having null values

2020-05-20 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72533/
---

(Updated May 21, 2020, 3:42 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Bugs: ATLAS-3802
https://issues.apache.org/jira/browse/ATLAS-3802


Repository: atlas


Description
---

Issue: When filter with 'attribute' != 'value' is searched, it doesn't fetch 
entities where attribute is not present.

Workaround: In the current implementation attribute = null is ignored, So now 
we can allow null when Operator is NEQ.


Diffs
-

  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
4408cd449 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 7508f7855 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 95f782abb 
  
repository/src/test/java/org/apache/atlas/discovery/EntitySearchProcessorTest.java
 1951c3937 


Diff: https://reviews.apache.org/r/72533/diff/1/


Testing
---

Added test cases and manually tested NEQ operator for below scenarios:
1) Non String Attribute -> index query
2) String Attributes -> graph query
3) pipeSeperated Attributes (classificationNames,labels) -> NOT_CONTAINS 
operator -> graph query
4) businessMetadata attribute


Thanks,

Pinal Shah



Re: Review Request 72489: ATLAS-3788 : BasicSearch: Classification with System attribute(indexed) filters has pagination issue

2020-05-14 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72489/
---

(Updated May 14, 2020, 8:45 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Bugs: ATLAS-3788
https://issues.apache.org/jira/browse/ATLAS-3788


Repository: atlas


Description
---

When selecting ALL_CLASSIFICATION_TYPES, with system attribute Created Time < 
currentTimeStamp , it results to less or zero number of entities than actual.


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 74c088c0e 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 958781e41 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/72489/diff/3/


Testing (updated)
---

Manually Tested.
Added test cases.
Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1894


Thanks,

Pinal Shah



Re: Review Request 72489: ATLAS-3788 : BasicSearch: Classification with System attribute(indexed) filters has pagination issue

2020-05-13 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72489/
---

(Updated May 14, 2020, 5:46 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

added license to new file


Bugs: ATLAS-3788
https://issues.apache.org/jira/browse/ATLAS-3788


Repository: atlas


Description
---

When selecting ALL_CLASSIFICATION_TYPES, with system attribute Created Time < 
currentTimeStamp , it results to less or zero number of entities than actual.


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 74c088c0e 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 958781e41 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/72489/diff/3/

Changes: https://reviews.apache.org/r/72489/diff/2-3/


Testing
---

Manually Tested.
Will add Precommit Link.


Thanks,

Pinal Shah



Re: Review Request 72489: ATLAS-3788 : BasicSearch: Classification with System attribute(indexed) filters has pagination issue

2020-05-13 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72489/
---

(Updated May 13, 2020, 5:04 p.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

Added unit test cases for basic search of classification


Bugs: ATLAS-3788
https://issues.apache.org/jira/browse/ATLAS-3788


Repository: atlas


Description
---

When selecting ALL_CLASSIFICATION_TYPES, with system attribute Created Time < 
currentTimeStamp , it results to less or zero number of entities than actual.


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 74c088c0e 
  repository/src/test/java/org/apache/atlas/BasicTestSetup.java 958781e41 
  
repository/src/test/java/org/apache/atlas/discovery/BasicSearchClassificationTest.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/72489/diff/2/

Changes: https://reviews.apache.org/r/72489/diff/1-2/


Testing (updated)
---

Manually Tested.
Will add Precommit Link.


Thanks,

Pinal Shah



Re: Review Request 72486: ATLAS-3780 : Change 'Status' from Classification System Attributes to EntityStatus

2020-05-11 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72486/
---

(Updated May 11, 2020, 2:01 p.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Summary (updated)
-

ATLAS-3780 : Change 'Status' from Classification System Attributes to 
EntityStatus


Bugs: ATLAS-3780
https://issues.apache.org/jira/browse/ATLAS-3780


Repository: atlas


Description
---

We are having 'State' Attribute for Status i.e ACTIVE and DELETED.
Whenever Entity is Deleted,'state' is always ACTIVE for Classification vertices 
but 'entityState' gets reflected as 'DELETED'.

For this UI, needs to handle '_entityStatus' as enum and whenever DELETED is 
selected, showHistoricalEntities needs to be true.


Diffs
-

  intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 
4d0179c44 
  intg/src/main/java/org/apache/atlas/type/Constants.java 3fc13056e 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 74c088c0e 


Diff: https://reviews.apache.org/r/72486/diff/1/


Testing
---

Manually tested


Thanks,

Pinal Shah



Review Request 72489: ATLAS-3788 : BasicSearch: Classification with System attribute(indexed) filters has pagination issue

2020-05-11 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72489/
---

Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Bugs: ATLAS-3788
https://issues.apache.org/jira/browse/ATLAS-3788


Repository: atlas


Description
---

When selecting ALL_CLASSIFICATION_TYPES, with system attribute Created Time < 
currentTimeStamp , it results to less or zero number of entities than actual.


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 74c088c0e 


Diff: https://reviews.apache.org/r/72489/diff/1/


Testing
---

Manually Tested


Thanks,

Pinal Shah



Re: Review Request 72486: ATLAS-3680 : Change 'Status' from Classification System Attributes to EntityStatus

2020-05-11 Thread Pinal Shah


> On May 8, 2020, 4:48 p.m., Ashutosh Mestry wrote:
> > repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
> > Line 195 (original), 195 (patched)
> > <https://reviews.apache.org/r/72486/diff/1/?file=2231049#file2231049line195>
> >
> > Prefer GraphTraversal over gremlin. GraphTraversal is order of 
> > magnitude faster than germlin.

Sure Ashutosh, I will update this with GraphTraversal, but i think i should 
create new JIRA for it.
Thanks for reviewing.


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72486/#review220698
---


On May 8, 2020, 11:21 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72486/
> ---
> 
> (Updated May 8, 2020, 11:21 a.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3780
> https://issues.apache.org/jira/browse/ATLAS-3780
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> We are having 'State' Attribute for Status i.e ACTIVE and DELETED.
> Whenever Entity is Deleted,'state' is always ACTIVE for Classification 
> vertices but 'entityState' gets reflected as 'DELETED'.
> 
> For this UI, needs to handle '_entityStatus' as enum and whenever DELETED is 
> selected, showHistoricalEntities needs to be true.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 
> 4d0179c44 
>   intg/src/main/java/org/apache/atlas/type/Constants.java 3fc13056e 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  74c088c0e 
> 
> 
> Diff: https://reviews.apache.org/r/72486/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Review Request 72486: ATLAS-3680 : Change 'Status' from Classification System Attributes to EntityStatus

2020-05-08 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72486/
---

Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Bugs: ATLAS-3780
https://issues.apache.org/jira/browse/ATLAS-3780


Repository: atlas


Description
---

We are having 'State' Attribute for Status i.e ACTIVE and DELETED.
Whenever Entity is Deleted,'state' is always ACTIVE for Classification vertices 
but 'entityState' gets reflected as 'DELETED'.

For this UI, needs to handle '_entityStatus' as enum and whenever DELETED is 
selected, showHistoricalEntities needs to be true.


Diffs
-

  intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 
4d0179c44 
  intg/src/main/java/org/apache/atlas/type/Constants.java 3fc13056e 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 74c088c0e 


Diff: https://reviews.apache.org/r/72486/diff/1/


Testing
---

Manually tested


Thanks,

Pinal Shah



Re: Review Request 72418: ATLAS-3753 : Classification Search with wildcard in between strings causes issues.

2020-04-22 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72418/#review220441
---




repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java
Line 207 (original), 206 (patched)
<https://reviews.apache.org/r/72418/#comment308827>

I have added delimeter '|' here , as _classificationNames can have multiple 
tag names


- Pinal Shah


On April 22, 2020, 6:56 p.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72418/
> ---
> 
> (Updated April 22, 2020, 6:56 p.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3753
> https://issues.apache.org/jira/browse/ATLAS-3753
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Tag `tag_abc_test` is created and associated to an entity.
> 
> Search with test or tag fetches the entity.
> 
> But Search with tag*test doesn't fetch the entity.
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> 5d7c50838 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> 69770c3b4 
> 
> 
> Diff: https://reviews.apache.org/r/72418/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested below combinations for wildcard (CASE_INSENSITIVE)
> 
> 'tag_abc_*'
> '*_abc*'
> 'tag_*_*'
> 'tag*test'
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72418: ATLAS-3753 : Classification Search with wildcard in between strings causes issues.

2020-04-22 Thread Pinal Shah


> On April 22, 2020, 11:48 p.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java
> > Lines 853 (patched)
> > <https://reviews.apache.org/r/72418/diff/1/?file=2225709#file2225709line854>
> >
> > Is it necessary to add '.*' as prefix and suffix? Wouldn't this result 
> > in incorrect results, for example given seach string tag*a*b would match 
> > for tags: 'mytag_abcd' which is incorrect. Please review and update.

Yes Madhan you are correct.
To handle this scenario, i have added prefix and suffix with '|' to the 
classificationName


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72418/#review220437
---


On April 22, 2020, 6:56 p.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72418/
> ---
> 
> (Updated April 22, 2020, 6:56 p.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3753
> https://issues.apache.org/jira/browse/ATLAS-3753
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Tag `tag_abc_test` is created and associated to an entity.
> 
> Search with test or tag fetches the entity.
> 
> But Search with tag*test doesn't fetch the entity.
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> 5d7c50838 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> 69770c3b4 
> 
> 
> Diff: https://reviews.apache.org/r/72418/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested below combinations for wildcard (CASE_INSENSITIVE)
> 
> 'tag_abc_*'
> '*_abc*'
> 'tag_*_*'
> 'tag*test'
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Review Request 72418: ATLAS-3753 : Classification Search with wildcard in between strings causes issues.

2020-04-22 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72418/
---

Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3753
https://issues.apache.org/jira/browse/ATLAS-3753


Repository: atlas


Description
---

Tag `tag_abc_test` is created and associated to an entity.

Search with test or tag fetches the entity.

But Search with tag*test doesn't fetch the entity.


Diffs
-

  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
5d7c50838 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
69770c3b4 


Diff: https://reviews.apache.org/r/72418/diff/1/


Testing
---

Manually tested below combinations for wildcard (CASE_INSENSITIVE)

'tag_abc_*'
'*_abc*'
'tag_*_*'
'tag*test'


Thanks,

Pinal Shah



Review Request 72396: ATLAS-3746 : _NOT_CLASSIFIED doesn't fetch right results

2020-04-21 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72396/
---

Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3746
https://issues.apache.org/jira/browse/ATLAS-3746


Repository: atlas


Description
---

typename = hive_table

classification = _NOT_CLASSIFIED

fetches right results.

But

just classification = _NOT_CLASSIFIED

doesn't fetch any results.


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/GraphIndexQueryBuilder.java 
3f58acb9c 


Diff: https://reviews.apache.org/r/72396/diff/1/


Testing
---

Tested below combination manually

1) classification = _NOT_CLASSIFIED -> working
2) typeName = hive_table, classification = _NOT_CLASSIFIED -> working
3) typeName = hive_table, Entity Filters, classification = _NOT_CLASSIFIED -> 
working
4) typeName = hive_table, classification = _NOT_CLASSIFIED, 
classificationFilters -> **results will not show up because classification 
filter on _NOT_CLASSIFIED classification doesn't make sense**
5) classification = _NOT_CLASSIFIED, classificationFilters -> **results will 
not show up because classification filter on _NOT_CLASSIFIED classification 
doesn't make sense**


Thanks,

Pinal Shah



Re: Review Request 72300: ATLAS-3696 : [Business Metadata] No option to check/uncheck Business Metadata field in search results

2020-04-07 Thread Pinal Shah


> On April 7, 2020, 3:19 p.m., Ashutosh Mestry wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
> > Lines 697 (patched)
> > <https://reviews.apache.org/r/72300/diff/1/?file=2216709#file2216709line697>
> >
> > Use MapUtils.isNotEmpty.

sure


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72300/#review220240
---


On April 1, 2020, 10:43 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72300/
> ---
> 
> (Updated April 1, 2020, 10:43 a.m.)
> 
> 
> Review request for atlas, keval bhatt, Madhan Neethiraj, Mandar Ambawane, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3696
> https://issues.apache.org/jira/browse/ATLAS-3696
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> For instance, there is no way to show "Business Metadata" column if Business 
> Metadata attribute filter is not applied while searching.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java 
> 7d2476a64 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
>  3a0d001bc 
> 
> 
> Diff: https://reviews.apache.org/r/72300/diff/1/
> 
> 
> Testing
> ---
> 
> Selected Entity which has business attribute, the response of basic search 
> will include businessMetadata attributes
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72300: ATLAS-3696 : [Business Metadata] No option to check/uncheck Business Metadata field in search results

2020-04-07 Thread Pinal Shah


> On April 7, 2020, 4:06 p.m., Madhan Neethiraj wrote:
> > intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java
> > Lines 66 (patched)
> > <https://reviews.apache.org/r/72300/diff/1/?file=2216708#file2216708line66>
> >
> > - I suggest to remove businessMetadataNames, as it has the same value 
> > as businessMetadata.keys() i.e. avoid duplicate
> > - rename businessMetadata => businessAttributes, just as it is named in 
> > AtlasEntity.

Yes, businessMetadataNames is duplicate
I will rename businessMetadata => businessAttributes


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72300/#review220241
---


On April 1, 2020, 10:43 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72300/
> ---
> 
> (Updated April 1, 2020, 10:43 a.m.)
> 
> 
> Review request for atlas, keval bhatt, Madhan Neethiraj, Mandar Ambawane, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3696
> https://issues.apache.org/jira/browse/ATLAS-3696
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> For instance, there is no way to show "Business Metadata" column if Business 
> Metadata attribute filter is not applied while searching.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java 
> 7d2476a64 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
>  3a0d001bc 
> 
> 
> Diff: https://reviews.apache.org/r/72300/diff/1/
> 
> 
> Testing
> ---
> 
> Selected Entity which has business attribute, the response of basic search 
> will include businessMetadata attributes
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72300: ATLAS-3696 : [Business Metadata] No option to check/uncheck Business Metadata field in search results

2020-04-07 Thread Pinal Shah


> On April 7, 2020, 4:06 p.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
> > Lines 696 (patched)
> > <https://reviews.apache.org/r/72300/diff/1/?file=2216709#file2216709line696>
> >
> > This returns all available business attributes in the entity. Instead 
> > of including all attribues, only the attributes requested in the search 
> > should be returned in AtlasHeader - refer to block below at #720 below that 
> > retrieves only requested attributes.
> > 
> > In fact the block below at #720 should already handle business 
> > attributes as well. Search request should add desired business attributes 
> > in 'attributes', in the following format: 
> > . - the same format used to in 
> > filter criteria. Please verify that business-attributes are indeeded 
> > included in AtlasEntityHeader.attrbutes. If true, changes in this patch 
> > shouldn't be necessary.

Thanks Madhan for reviewing
The usecase to add businessMetadata at the attribute level was when user 
selects businessMetadata from the "Columns" tab ,that time response should have 
all the businessMetadata attributes. 
In this case can we populate all the businessMetadat attributes inside 
'attributes'?


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72300/#review220241
---


On April 1, 2020, 10:43 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72300/
> ---
> 
> (Updated April 1, 2020, 10:43 a.m.)
> 
> 
> Review request for atlas, keval bhatt, Madhan Neethiraj, Mandar Ambawane, 
> Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3696
> https://issues.apache.org/jira/browse/ATLAS-3696
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> For instance, there is no way to show "Business Metadata" column if Business 
> Metadata attribute filter is not applied while searching.
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java 
> 7d2476a64 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
>  3a0d001bc 
> 
> 
> Diff: https://reviews.apache.org/r/72300/diff/1/
> 
> 
> Testing
> ---
> 
> Selected Entity which has business attribute, the response of basic search 
> will include businessMetadata attributes
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Review Request 72300: ATLAS-3696 : [Business Metadata] No option to check/uncheck Business Metadata field in search results

2020-04-01 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72300/
---

Review request for atlas, keval bhatt, Madhan Neethiraj, Mandar Ambawane, Nixon 
Rodrigues, and Sarath Subramanian.


Bugs: ATLAS-3696
https://issues.apache.org/jira/browse/ATLAS-3696


Repository: atlas


Description
---

For instance, there is no way to show "Business Metadata" column if Business 
Metadata attribute filter is not applied while searching.


Diffs
-

  intg/src/main/java/org/apache/atlas/model/instance/AtlasEntityHeader.java 
7d2476a64 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
 3a0d001bc 


Diff: https://reviews.apache.org/r/72300/diff/1/


Testing
---

Selected Entity which has business attribute, the response of basic search will 
include businessMetadata attributes


Thanks,

Pinal Shah



Re: Review Request 72280: ATLAS-3703 : BasicSearch: Search by terms/Search by text in combination of Classification filter, filter doesn't work

2020-03-31 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72280/
---

(Updated March 31, 2020, 10:49 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Summary (updated)
-

ATLAS-3703 : BasicSearch: Search by terms/Search by text in combination of 
Classification filter, filter doesn't work


Bugs: ATLAS-3703
https://issues.apache.org/jira/browse/ATLAS-3703


Repository: atlas


Description
---

Example of cases where Filters of classification doesn't work
ClassificationType + Classification Filter + Term
ClassificationType + Classification Filter + Text
EntityType + Entity Filter + ClassificationType + Classification Filter + Term


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 1343c9b96 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
ddd3e60d2 


Diff: https://reviews.apache.org/r/72280/diff/3/


Testing
---

Manually tested all above cases, except case where attributes are non indexed


Thanks,

Pinal Shah



Re: Review Request 72253: ATLAS-3672 : Search with Classification without filters and Entity with filters doesn't fetch expected results.

2020-03-31 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72253/
---

(Updated March 31, 2020, 10:45 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

removed unwanted check


Bugs: ATLAS-3672
https://issues.apache.org/jira/browse/ATLAS-3672


Repository: atlas


Description
---

As the vertexPropertyName of "name" attribute is changed to prefix '__s_name' 
for graphdb
During fetching it was taking qualifiedName .name as the propertyKey


Diffs (updated)
-

  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
ddd3e60d2 


Diff: https://reviews.apache.org/r/72253/diff/2/

Changes: https://reviews.apache.org/r/72253/diff/1-2/


Testing
---

Manually tested below scenario:
1) add Entity and Entity Filters
2) add classification


Thanks,

Pinal Shah



Re: Review Request 72253: ATLAS-3672 : Search with Classification without filters and Entity with filters doesn't fetch expected results.

2020-03-31 Thread Pinal Shah


> On March 31, 2020, 9:12 a.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java
> > Line 737 (original), 737 (patched)
> > <https://reviews.apache.org/r/72253/diff/1/?file=2214312#file2214312line737>
> >
> > attr.getVertexPropertyName()  will not be null for any attribute; so no 
> > need to check and fall back to attr.getQualifiedName().

As the vertexPropertyName of "name" attribute is changed to prefix '__s_name' 
for graphdb
attr.getQualifiedName() will not solve purpose.


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72253/#review220143
---


On March 20, 2020, 9:37 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72253/
> ---
> 
> (Updated March 20, 2020, 9:37 a.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3672
> https://issues.apache.org/jira/browse/ATLAS-3672
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> As the vertexPropertyName of "name" attribute is changed to prefix '__s_name' 
> for graphdb
> During fetching it was taking qualifiedName .name as the propertyKey
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> ddd3e60d2 
> 
> 
> Diff: https://reviews.apache.org/r/72253/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested below scenario:
> 1) add Entity and Entity Filters
> 2) add classification
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72280: BasicSearch: Search by terms/Search by text in combination of Classification filter, filter doesn't work

2020-03-30 Thread Pinal Shah


> On March 30, 2020, 6:28 a.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
> > Lines 111 (patched)
> > <https://reviews.apache.org/r/72280/diff/1/?file=2216343#file2216343line119>
> >
> > buildTraitPredict() doesn't handle wildcards in the classification-name 
> > - like  classificationName="*PII". Please review.
> 
> Pinal Shah wrote:
> Thanks Madhan,
> What if i use __classificationNames which String type, instead of 
> __traitNames which is of List type in graphdb.

For the building the traitName incase of wildcard


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72280/#review220100
-------


On March 31, 2020, 5:12 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72280/
> ---
> 
> (Updated March 31, 2020, 5:12 a.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3703
> https://issues.apache.org/jira/browse/ATLAS-3703
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Example of cases where Filters of classification doesn't work
> ClassificationType + Classification Filter + Term
> ClassificationType + Classification Filter + Text
> EntityType + Entity Filter + ClassificationType + Classification Filter + Term
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  1343c9b96 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> ddd3e60d2 
> 
> 
> Diff: https://reviews.apache.org/r/72280/diff/3/
> 
> 
> Testing
> ---
> 
> Manually tested all above cases, except case where attributes are non indexed
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72280: BasicSearch: Search by terms/Search by text in combination of Classification filter, filter doesn't work

2020-03-30 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72280/
---

(Updated March 31, 2020, 5:12 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

Added support to wildcard search for classification


Bugs: ATLAS-3703
https://issues.apache.org/jira/browse/ATLAS-3703


Repository: atlas


Description
---

Example of cases where Filters of classification doesn't work
ClassificationType + Classification Filter + Term
ClassificationType + Classification Filter + Text
EntityType + Entity Filter + ClassificationType + Classification Filter + Term


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 1343c9b96 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
ddd3e60d2 


Diff: https://reviews.apache.org/r/72280/diff/3/

Changes: https://reviews.apache.org/r/72280/diff/2-3/


Testing
---

Manually tested all above cases, except case where attributes are non indexed


Thanks,

Pinal Shah



Re: Review Request 72280: BasicSearch: Search by terms/Search by text in combination of Classification filter, filter doesn't work

2020-03-30 Thread Pinal Shah


> On March 30, 2020, 6:28 a.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
> > Lines 111 (patched)
> > <https://reviews.apache.org/r/72280/diff/1/?file=2216343#file2216343line119>
> >
> > buildTraitPredict() doesn't handle wildcards in the classification-name 
> > - like  classificationName="*PII". Please review.

Thanks Madhan,
What if i use __classificationNames which String type, instead of __traitNames 
which is of List type in graphdb.


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72280/#review220100
-------


On March 30, 2020, 12:45 p.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72280/
> ---
> 
> (Updated March 30, 2020, 12:45 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3703
> https://issues.apache.org/jira/browse/ATLAS-3703
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Example of cases where Filters of classification doesn't work
> ClassificationType + Classification Filter + Term
> ClassificationType + Classification Filter + Text
> EntityType + Entity Filter + ClassificationType + Classification Filter + Term
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  1343c9b96 
> 
> 
> Diff: https://reviews.apache.org/r/72280/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested all above cases, except case where attributes are non indexed
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72280: BasicSearch: Search by terms/Search by text in combination of Classification filter, filter doesn't work

2020-03-30 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72280/
---

(Updated March 30, 2020, 12:45 p.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

Added support to wildcard search


Bugs: ATLAS-3703
https://issues.apache.org/jira/browse/ATLAS-3703


Repository: atlas


Description
---

Example of cases where Filters of classification doesn't work
ClassificationType + Classification Filter + Term
ClassificationType + Classification Filter + Text
EntityType + Entity Filter + ClassificationType + Classification Filter + Term


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 1343c9b96 


Diff: https://reviews.apache.org/r/72280/diff/2/

Changes: https://reviews.apache.org/r/72280/diff/1-2/


Testing
---

Manually tested all above cases, except case where attributes are non indexed


Thanks,

Pinal Shah



Re: Review Request 72280: BasicSearch: Search by terms/Search by text in combination of Classification filter, filter doesn't work

2020-03-30 Thread Pinal Shah


> On March 30, 2020, 6:28 a.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
> > Line 116 (original), 108 (patched)
> > <https://reviews.apache.org/r/72280/diff/1/?file=2216343#file2216343line116>
> >
> > Wouldn't line #108 cause index-search to be not used when no attribute 
> > filter is specified? For example,  when the search filter has only 
> > classification name?

in that case, line #99 will handle


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72280/#review220100
---


On March 30, 2020, 5:50 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72280/
> ---
> 
> (Updated March 30, 2020, 5:50 a.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3703
> https://issues.apache.org/jira/browse/ATLAS-3703
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Example of cases where Filters of classification doesn't work
> ClassificationType + Classification Filter + Term
> ClassificationType + Classification Filter + Text
> EntityType + Entity Filter + ClassificationType + Classification Filter + Term
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  1343c9b96 
> 
> 
> Diff: https://reviews.apache.org/r/72280/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested all above cases, except case where attributes are non indexed
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Review Request 72280: BasicSearch: Search by terms/Search by text in combination of Classification filter, filter doesn't work

2020-03-29 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72280/
---

Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Bugs: ATLAS-3703
https://issues.apache.org/jira/browse/ATLAS-3703


Repository: atlas


Description
---

Example of cases where Filters of classification doesn't work
ClassificationType + Classification Filter + Term
ClassificationType + Classification Filter + Text
EntityType + Entity Filter + ClassificationType + Classification Filter + Term


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 1343c9b96 


Diff: https://reviews.apache.org/r/72280/diff/1/


Testing
---

Manually tested all above cases, except case where attributes are non indexed


Thanks,

Pinal Shah



Review Request 72253: ATLAS-3672 : Search with Classification without filters and Entity with filters doesn't fetch expected results.

2020-03-20 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72253/
---

Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Bugs: ATLAS-3672
https://issues.apache.org/jira/browse/ATLAS-3672


Repository: atlas


Description
---

As the vertexPropertyName of "name" attribute is changed to prefix '__s_name' 
for graphdb
During fetching it was taking qualifiedName .name as the propertyKey


Diffs
-

  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
ddd3e60d2 


Diff: https://reviews.apache.org/r/72253/diff/1/


Testing
---

Manually tested below scenario:
1) add Entity and Entity Filters
2) add classification


Thanks,

Pinal Shah



Re: Review Request 72188: ATLAS-3650 : Basic Search: query of typeName doesn't apply when it has many subTypes(like Asset) in combination with attribute filter

2020-03-16 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72188/
---

(Updated March 16, 2020, 12:45 p.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

retained attributePredicate


Bugs: ATLAS-3650
https://issues.apache.org/jira/browse/ATLAS-3650


Repository: atlas


Description
---

When Asset typeName, which has many subtypes, with some attribute filter is 
searched , 
query formed for typeName is not included in final query


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
c9f3166f1 


Diff: https://reviews.apache.org/r/72188/diff/3/

Changes: https://reviews.apache.org/r/72188/diff/2-3/


Testing
---

1) Select Asset
2) Select Guid notNull
--> Before, all entities with guid notnull including internal 
Type(AtlasGlossary) where coming in result
--> Now, Entities with typeName Asset and allSubtypes with guid notnull is 
coming in result


Thanks,

Pinal Shah



Re: Review Request 72188: ATLAS-3650 : Basic Search: query of typeName doesn't apply when it has many subTypes(like Asset) in combination with attribute filter

2020-03-13 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72188/
---

(Updated March 13, 2020, 11:10 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Bugs: ATLAS-3650
https://issues.apache.org/jira/browse/ATLAS-3650


Repository: atlas


Description
---

When Asset typeName, which has many subtypes, with some attribute filter is 
searched , 
query formed for typeName is not included in final query


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
c9f3166f1 


Diff: https://reviews.apache.org/r/72188/diff/2/

Changes: https://reviews.apache.org/r/72188/diff/1-2/


Testing
---

1) Select Asset
2) Select Guid notNull
--> Before, all entities with guid notnull including internal 
Type(AtlasGlossary) where coming in result
--> Now, Entities with typeName Asset and allSubtypes with guid notnull is 
coming in result


Thanks,

Pinal Shah



Re: Review Request 72156: ATLAS-3618 Entities with no guid appears in search result

2020-03-05 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72156/
---

(Updated March 6, 2020, 6:26 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

Changed typeNamePredicate to EntityPredicate


Bugs: ATLAS-3618
https://issues.apache.org/jira/browse/ATLAS-3618


Repository: atlas


Description
---

1) Entities of struct types appears when ALL_ENTITY_TYPES is selected
2) Entities of internal types like AtlasGlossary etc appears when 
ALL_ENTITY_TYPES is selected


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 6ab0afbf9 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
ebd5992cd 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
b5ede0b82 


Diff: https://reviews.apache.org/r/72156/diff/5/

Changes: https://reviews.apache.org/r/72156/diff/4-5/


Testing
---

1) typeName: ALL_ENTITY_TYPES returns all entities with no struct types(whoes 
guid isnotnull) and no internal types(whoes supertype is not _internal)
2) typeName: ALL_ENTITY_TYPES, filter: guid isnull, returns no result
3) typeName: ALL_ENTITY_TYPES, filter: typeName begins_with Atlas, returns no 
result

Usecase:
-> Added, term1 in Glossary
-> added classification1 to term1
1) In search Panel(showing all entities) -> term1 shouln't appear
2) In classification Panel, showing all entities associated to classification1 
-> term1 shouldn't appear


Thanks,

Pinal Shah



Re: Review Request 72188: ATLAS-3650 : Basic Search: query of typeName doesn't apply when it has many subTypes(like Asset) in combination with attribute filter

2020-03-05 Thread Pinal Shah


> On March 4, 2020, 5:10 a.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
> > Lines 126 (patched)
> > <https://reviews.apache.org/r/72188/diff/1/?file=2212720#file2212720line126>
> >
> > typeNamePredicate could be null - see #100.
> > 
> > Consider alternate fix of moving #114 - #116 outside the 'if' block at 
> > #110.

Thanks Madhan,
I think once ATLAS-3618(https://reviews.apache.org/r/72156) is committed,
I can incooperate changes in it, because this patch is dependent on ATLAS-3618.


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72188/#review219747
-------


On March 4, 2020, 5:05 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72188/
> ---
> 
> (Updated March 4, 2020, 5:05 a.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3650
> https://issues.apache.org/jira/browse/ATLAS-3650
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> When Asset typeName, which has many subtypes, with some attribute filter is 
> searched , 
> query formed for typeName is not included in final query
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
>  8f531876b 
> 
> 
> Diff: https://reviews.apache.org/r/72188/diff/1/
> 
> 
> Testing
> ---
> 
> 1) Select Asset
> 2) Select Guid notNull
> --> Before, all entities with guid notnull including internal 
> Type(AtlasGlossary) where coming in result
> --> Now, Entities with typeName Asset and allSubtypes with guid notnull is 
> coming in result
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72156: ATLAS-3618 Entities with no guid appears in search result

2020-03-05 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72156/
---

(Updated March 6, 2020, 5:37 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

Rebase patch


Bugs: ATLAS-3618
https://issues.apache.org/jira/browse/ATLAS-3618


Repository: atlas


Description
---

1) Entities of struct types appears when ALL_ENTITY_TYPES is selected
2) Entities of internal types like AtlasGlossary etc appears when 
ALL_ENTITY_TYPES is selected


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 6ab0afbf9 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
ebd5992cd 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
b5ede0b82 


Diff: https://reviews.apache.org/r/72156/diff/4/

Changes: https://reviews.apache.org/r/72156/diff/3-4/


Testing
---

1) typeName: ALL_ENTITY_TYPES returns all entities with no struct types(whoes 
guid isnotnull) and no internal types(whoes supertype is not _internal)
2) typeName: ALL_ENTITY_TYPES, filter: guid isnull, returns no result
3) typeName: ALL_ENTITY_TYPES, filter: typeName begins_with Atlas, returns no 
result

Usecase:
-> Added, term1 in Glossary
-> added classification1 to term1
1) In search Panel(showing all entities) -> term1 shouln't appear
2) In classification Panel, showing all entities associated to classification1 
-> term1 shouldn't appear


Thanks,

Pinal Shah



Re: Review Request 72156: ATLAS-3618 Entities with no guid appears in search result

2020-03-05 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72156/
---

(Updated March 5, 2020, 11:35 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

addressed typeNamePredicate null check


Bugs: ATLAS-3618
https://issues.apache.org/jira/browse/ATLAS-3618


Repository: atlas


Description
---

1) Entities of struct types appears when ALL_ENTITY_TYPES is selected
2) Entities of internal types like AtlasGlossary etc appears when 
ALL_ENTITY_TYPES is selected


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 6ab0afbf9 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
8f531876b 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
bb1e9f633 


Diff: https://reviews.apache.org/r/72156/diff/3/

Changes: https://reviews.apache.org/r/72156/diff/2-3/


Testing
---

1) typeName: ALL_ENTITY_TYPES returns all entities with no struct types(whoes 
guid isnotnull) and no internal types(whoes supertype is not _internal)
2) typeName: ALL_ENTITY_TYPES, filter: guid isnull, returns no result
3) typeName: ALL_ENTITY_TYPES, filter: typeName begins_with Atlas, returns no 
result

Usecase:
-> Added, term1 in Glossary
-> added classification1 to term1
1) In search Panel(showing all entities) -> term1 shouln't appear
2) In classification Panel, showing all entities associated to classification1 
-> term1 shouldn't appear


Thanks,

Pinal Shah



Re: Review Request 72156: ATLAS-3618 Entities with no guid appears in search result

2020-03-04 Thread Pinal Shah


> On March 4, 2020, 7:29 a.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
> > Lines 286 (patched)
> > <https://reviews.apache.org/r/72156/diff/2/?file=2212025#file2212025line286>
> >
> > typeNamePredicate can be null here (see #159 above); please review if 
> > #286 should be surrounded with:
> >   if (typeNamePredicate != null)

Thanks Madhan will surely add it


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72156/#review219751
---


On Feb. 25, 2020, 7:11 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72156/
> ---
> 
> (Updated Feb. 25, 2020, 7:11 a.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3618
> https://issues.apache.org/jira/browse/ATLAS-3618
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> 1) Entities of struct types appears when ALL_ENTITY_TYPES is selected
> 2) Entities of internal types like AtlasGlossary etc appears when 
> ALL_ENTITY_TYPES is selected
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  c0a5a46dd 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
>  b5606d046 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> bb1e9f633 
> 
> 
> Diff: https://reviews.apache.org/r/72156/diff/2/
> 
> 
> Testing
> ---
> 
> 1) typeName: ALL_ENTITY_TYPES returns all entities with no struct types(whoes 
> guid isnotnull) and no internal types(whoes supertype is not _internal)
> 2) typeName: ALL_ENTITY_TYPES, filter: guid isnull, returns no result
> 3) typeName: ALL_ENTITY_TYPES, filter: typeName begins_with Atlas, returns no 
> result
> 
> Usecase:
> -> Added, term1 in Glossary
> -> added classification1 to term1
> 1) In search Panel(showing all entities) -> term1 shouln't appear
> 2) In classification Panel, showing all entities associated to 
> classification1 -> term1 shouldn't appear
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72156: ATLAS-3618 Entities with no guid appears in search result

2020-03-04 Thread Pinal Shah


> On March 4, 2020, 7:29 a.m., Madhan Neethiraj wrote:
> > repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
> > Lines 169 (patched)
> > <https://reviews.apache.org/r/72156/diff/2/?file=2212026#file2212026line171>
> >
> > Wouldn't this overwrite earlier assignment to inMemoryPredicate - line 
> > #121, #123? Please review and update.

Thanks Madhan, for reviewing the patch.
Yes, it will overwrite.
When it is GraphQuery, query itself includes type filter(except incase of 
_ALL_ENTITY_TYPES) and attribute filters(except incase when operator is NEQ in 
pipeSeperatedSystemAttributes)
So I think, inMemoryPredicate should only be typeNamePredicate when 
isEntityRootType


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72156/#review219751
-------


On Feb. 25, 2020, 7:11 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72156/
> ---
> 
> (Updated Feb. 25, 2020, 7:11 a.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3618
> https://issues.apache.org/jira/browse/ATLAS-3618
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> 1) Entities of struct types appears when ALL_ENTITY_TYPES is selected
> 2) Entities of internal types like AtlasGlossary etc appears when 
> ALL_ENTITY_TYPES is selected
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  c0a5a46dd 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
>  b5606d046 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> bb1e9f633 
> 
> 
> Diff: https://reviews.apache.org/r/72156/diff/2/
> 
> 
> Testing
> ---
> 
> 1) typeName: ALL_ENTITY_TYPES returns all entities with no struct types(whoes 
> guid isnotnull) and no internal types(whoes supertype is not _internal)
> 2) typeName: ALL_ENTITY_TYPES, filter: guid isnull, returns no result
> 3) typeName: ALL_ENTITY_TYPES, filter: typeName begins_with Atlas, returns no 
> result
> 
> Usecase:
> -> Added, term1 in Glossary
> -> added classification1 to term1
> 1) In search Panel(showing all entities) -> term1 shouln't appear
> 2) In classification Panel, showing all entities associated to 
> classification1 -> term1 shouldn't appear
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72156: ATLAS-3618 Entities with no guid appears in search result

2020-03-03 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72156/#review219750
---




repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
Line 99 (original), 99 (patched)
<https://reviews.apache.org/r/72156/#comment307949>

Handled Non entities (without guid) and internal entities


- Pinal Shah


On Feb. 25, 2020, 7:11 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72156/
> ---
> 
> (Updated Feb. 25, 2020, 7:11 a.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3618
> https://issues.apache.org/jira/browse/ATLAS-3618
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> 1) Entities of struct types appears when ALL_ENTITY_TYPES is selected
> 2) Entities of internal types like AtlasGlossary etc appears when 
> ALL_ENTITY_TYPES is selected
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  c0a5a46dd 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java
>  b5606d046 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> bb1e9f633 
> 
> 
> Diff: https://reviews.apache.org/r/72156/diff/2/
> 
> 
> Testing
> ---
> 
> 1) typeName: ALL_ENTITY_TYPES returns all entities with no struct types(whoes 
> guid isnotnull) and no internal types(whoes supertype is not _internal)
> 2) typeName: ALL_ENTITY_TYPES, filter: guid isnull, returns no result
> 3) typeName: ALL_ENTITY_TYPES, filter: typeName begins_with Atlas, returns no 
> result
> 
> Usecase:
> -> Added, term1 in Glossary
> -> added classification1 to term1
> 1) In search Panel(showing all entities) -> term1 shouln't appear
> 2) In classification Panel, showing all entities associated to 
> classification1 -> term1 shouldn't appear
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Review Request 72188: ATLAS-3650 : Basic Search: query of typeName doesn't apply when it has many subTypes(like Asset) in combination with attribute filter

2020-03-03 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72188/
---

Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Bugs: ATLAS-3650
https://issues.apache.org/jira/browse/ATLAS-3650


Repository: atlas


Description
---

When Asset typeName, which has many subtypes, with some attribute filter is 
searched , 
query formed for typeName is not included in final query


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
8f531876b 


Diff: https://reviews.apache.org/r/72188/diff/1/


Testing
---

1) Select Asset
2) Select Guid notNull
--> Before, all entities with guid notnull including internal 
Type(AtlasGlossary) where coming in result
--> Now, Entities with typeName Asset and allSubtypes with guid notnull is 
coming in result


Thanks,

Pinal Shah



Re: Review Request 72174: ATLAS-3600 :- Some System Attribute of Entity filter doesn't work

2020-03-02 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72174/
---

(Updated March 2, 2020, 12:14 p.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

added one usecase, when in graphquery, filter with not_contains is not added


Bugs: ATLAS-3600
https://issues.apache.org/jira/browse/ATLAS-3600


Repository: atlas


Description
---

System Attributes does not support following 4 search ,

classification search
Propogated Classification Search
User-Defined Attributes 
Labels


Diffs (updated)
-

  intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java 
c74cf5122 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
8f531876b 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
356363db0 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
bb1e9f633 


Diff: https://reviews.apache.org/r/72174/diff/2/

Changes: https://reviews.apache.org/r/72174/diff/1-2/


Testing
---

Tested all above 4 system attributes manually.


Thanks,

Pinal Shah



Re: Review Request 72174: ATLAS-3600 :- Some System Attribute of Entity filter doesn't work

2020-03-01 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72174/
---

(Updated March 2, 2020, 5:41 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Summary (updated)
-

ATLAS-3600 :- Some System Attribute of Entity filter doesn't work


Bugs: ATLAS-3600
https://issues.apache.org/jira/browse/ATLAS-3600


Repository: atlas


Description
---

System Attributes does not support following 4 search ,

classification search
Propogated Classification Search
User-Defined Attributes 
Labels


Diffs
-

  intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java 
c74cf5122 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
356363db0 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
bb1e9f633 


Diff: https://reviews.apache.org/r/72174/diff/1/


Testing
---

Tested all above 4 system attributes manually.


Thanks,

Pinal Shah



Re: Review Request 72174: Some System Attribute of Entity filter doesn't work

2020-03-01 Thread Pinal Shah


> On Feb. 28, 2020, 3:13 p.m., Madhan Neethiraj wrote:
> > intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java
> > Lines 458 (patched)
> > <https://reviews.apache.org/r/72174/diff/1/?file=2212408#file2212408line458>
> >
> > Pinal - good to see addition of 'NOT_CONTAINS' operator! I suggest to 
> > use a separate patch for this enhancement. Please review if UI updates are 
> > necessary for addition of this new operator; for example, the list of 
> > operators shown in UI depends on the type of attribute.

Hi Madhan, 

Thanks for reviewing the patch.
Incase of pipeSeperatedSystemAttributes, when user selects NEQ operator, i 
think we need to use NotContains Predicate, because NotEquals Predicate will 
not solve the purpose because we store multiple values with pipe as a seperator.
So internally(on the server side), in the above case, i have added operator as 
NOT_CONTAINS.

(in #525 to #528 of 
repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java)


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72174/#review219691
-------


On Feb. 28, 2020, 12:04 p.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72174/
> ---
> 
> (Updated Feb. 28, 2020, 12:04 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3600
> https://issues.apache.org/jira/browse/ATLAS-3600
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> System Attributes does not support following 4 search ,
> 
> classification search
> Propogated Classification Search
> User-Defined Attributes 
> Labels
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java 
> c74cf5122 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> 356363db0 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> bb1e9f633 
> 
> 
> Diff: https://reviews.apache.org/r/72174/diff/1/
> 
> 
> Testing
> ---
> 
> Tested all above 4 system attributes manually.
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Review Request 72174: Some System Attribute of Entity filter doesn't work

2020-02-28 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72174/
---

Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Bugs: ATLAS-3600
https://issues.apache.org/jira/browse/ATLAS-3600


Repository: atlas


Description
---

System Attributes does not support following 4 search ,

classification search
Propogated Classification Search
User-Defined Attributes 
Labels


Diffs
-

  intg/src/main/java/org/apache/atlas/model/discovery/SearchParameters.java 
c74cf5122 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
356363db0 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
bb1e9f633 


Diff: https://reviews.apache.org/r/72174/diff/1/


Testing
---

Tested all above 4 system attributes manually.


Thanks,

Pinal Shah



Re: Review Request 72156: ATLAS-3618 Entities with no guid appears in search result

2020-02-24 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72156/
---

(Updated Feb. 25, 2020, 7:11 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

added method to check Nonentity vertices


Bugs: ATLAS-3618
https://issues.apache.org/jira/browse/ATLAS-3618


Repository: atlas


Description
---

1) Entities of struct types appears when ALL_ENTITY_TYPES is selected
2) Entities of internal types like AtlasGlossary etc appears when 
ALL_ENTITY_TYPES is selected


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 c0a5a46dd 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
b5606d046 
  repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
bb1e9f633 


Diff: https://reviews.apache.org/r/72156/diff/2/

Changes: https://reviews.apache.org/r/72156/diff/1-2/


Testing (updated)
---

1) typeName: ALL_ENTITY_TYPES returns all entities with no struct types(whoes 
guid isnotnull) and no internal types(whoes supertype is not _internal)
2) typeName: ALL_ENTITY_TYPES, filter: guid isnull, returns no result
3) typeName: ALL_ENTITY_TYPES, filter: typeName begins_with Atlas, returns no 
result

Usecase:
-> Added, term1 in Glossary
-> added classification1 to term1
1) In search Panel(showing all entities) -> term1 shouln't appear
2) In classification Panel, showing all entities associated to classification1 
-> term1 shouldn't appear


Thanks,

Pinal Shah



Re: Review Request 72156: ATLAS-3618 Entities with no guid appears in search result

2020-02-21 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72156/
---

(Updated Feb. 21, 2020, 12:23 p.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Bugs: ATLAS-3618
https://issues.apache.org/jira/browse/ATLAS-3618


Repository: atlas


Description
---

1) Entities of struct types appears when ALL_ENTITY_TYPES is selected
2) Entities of internal types like AtlasGlossary etc appears when 
ALL_ENTITY_TYPES is selected


Diffs
-

  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 2824feb16 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
b5606d046 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasGraphUtilsV2.java
 bf13338b6 


Diff: https://reviews.apache.org/r/72156/diff/1/


Testing (updated)
---

1) typeName: ALL_ENTITY_TYPES returns all entities with no struct types(whoes 
guid isnotnull) and no internal types(whoes supertype is not _internal)
2) typeName: ALL_ENTITY_TYPES, filter: guid isnull, returns no result
3) typeName: ALL_ENTITY_TYPES, filter: typeName begins_with Atlas, returns no 
result


Thanks,

Pinal Shah



Review Request 72156: ATLAS-3618 Entities with no guid appears in search result

2020-02-21 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72156/
---

Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Bugs: ATLAS-3618
https://issues.apache.org/jira/browse/ATLAS-3618


Repository: atlas


Description
---

1) Entities of struct types appears when ALL_ENTITY_TYPES is selected
2) Entities of internal types like AtlasGlossary etc appears when 
ALL_ENTITY_TYPES is selected


Diffs
-

  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 2824feb16 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
b5606d046 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasGraphUtilsV2.java
 bf13338b6 


Diff: https://reviews.apache.org/r/72156/diff/1/


Testing
---

1) typeName: ALL_ENTITY_TYPES returns all entities with no struct types(whoes 
guid isnotnull) and no internal types(whoes supertype is not _internal)
2) typeName: ALL_ENTITY_TYPES, filter: guid isnull, returns no result
3) typeName: ALL_ENTITY_TYPES, filter: typeName begins_with Atlas


Thanks,

Pinal Shah



Re: Review Request 72127: ATLAS-3615 :- Basic Search : Search with _ALL_ENTITY_TYPES and classification doesn't not fetch any entity associated to that classification

2020-02-20 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72127/
---

(Updated Feb. 21, 2020, 5:24 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

removed empty line and combined if condition


Bugs: ATLAS-3615
https://issues.apache.org/jira/browse/ATLAS-3615


Repository: atlas


Description
---

Issues:
1) Combination of typeName:_ALL_ENTITY_TYPES and classification doesn't return 
result
Why? As _ALL_ENTITY_TYPES value is assigned as [* TO *] results show up when 
searched from solr but not from graphdb

2) Combination of typeName:_ALL_ENTITY_TYPES, classification, 
query(SearchByText) doesn't return result
Why? result from solr, thru "/freetext" is correct, but onsearching if resulted 
entitytype is in typeAndSubtypes of given entityType, it breaks because 
typeAndSubTypes of _ALL_ENTITY_TYPES is [* TO *]

3) Combination of typeName, classification:_ALL_CLASSIFICATION_TYPES doesn't 
return result

4) Combination of typeName:_ALL_ENTITY_TYPES, 
classification:_ALL_CLASSIFICATION_TYPES, query(SearchByText) doesn't return 
correct result
why? query formed with above combination in solr /freetext is 'SAMPLEQUERY' AND 
[* TO *] AND [* TO *]

5) Combination of classification:_ALL_CLASSIFICATION_TYPES and 
classificationFilter doesn't return correct result


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 c0a5a46dd 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
b5606d046 
  
repository/src/main/java/org/apache/atlas/discovery/FreeTextSearchProcessor.java
 d628f1352 
  repository/src/main/java/org/apache/atlas/discovery/SearchContext.java 
353411363 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
8c17a9044 


Diff: https://reviews.apache.org/r/72127/diff/4/

Changes: https://reviews.apache.org/r/72127/diff/3-4/


Testing
---

For issue1:
1) tested with values _ALL_ENTITY_TYPES + EntityFilter + classification -> 
result fetches from gremlin and shows up correct result
2) tested with values _ALL_ENTITY_TYPES + classification -> result fetches from 
gremlin and shows up correct result

For issue2:
1) tested with values _ALL_ENTITY_TYPES + classification + query(SearchByText) 
-> shows up correct result


Thanks,

Pinal Shah



Re: Review Request 72127: ATLAS-3615 :- Basic Search : Search with _ALL_ENTITY_TYPES and classification doesn't not fetch any entity associated to that classification

2020-02-18 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72127/
---

(Updated Feb. 18, 2020, 11:57 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Changes
---

Changed typeAndSubTypes to emptySet


Bugs: ATLAS-3615
https://issues.apache.org/jira/browse/ATLAS-3615


Repository: atlas


Description (updated)
---

Issues:
1) Combination of typeName:_ALL_ENTITY_TYPES and classification doesn't return 
result
Why? As _ALL_ENTITY_TYPES value is assigned as [* TO *] results show up when 
searched from solr but not from graphdb

2) Combination of typeName:_ALL_ENTITY_TYPES, classification, 
query(SearchByText) doesn't return result
Why? result from solr, thru "/freetext" is correct, but onsearching if resulted 
entitytype is in typeAndSubtypes of given entityType, it breaks because 
typeAndSubTypes of _ALL_ENTITY_TYPES is [* TO *]

3) Combination of typeName, classification:_ALL_CLASSIFICATION_TYPES doesn't 
return result

4) Combination of typeName:_ALL_ENTITY_TYPES, 
classification:_ALL_CLASSIFICATION_TYPES, query(SearchByText) doesn't return 
correct result
why? query formed with above combination in solr /freetext is 'SAMPLEQUERY' AND 
[* TO *] AND [* TO *]

5) Combination of classification:_ALL_CLASSIFICATION_TYPES and 
classificationFilter doesn't return correct result


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 c0a5a46dd 
  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
b5606d046 
  
repository/src/main/java/org/apache/atlas/discovery/FreeTextSearchProcessor.java
 d628f1352 
  repository/src/main/java/org/apache/atlas/discovery/SearchContext.java 
353411363 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
8c17a9044 


Diff: https://reviews.apache.org/r/72127/diff/3/

Changes: https://reviews.apache.org/r/72127/diff/2-3/


Testing
---

For issue1:
1) tested with values _ALL_ENTITY_TYPES + EntityFilter + classification -> 
result fetches from gremlin and shows up correct result
2) tested with values _ALL_ENTITY_TYPES + classification -> result fetches from 
gremlin and shows up correct result

For issue2:
1) tested with values _ALL_ENTITY_TYPES + classification + query(SearchByText) 
-> shows up correct result


Thanks,

Pinal Shah



Re: Review Request 72127: ATLAS-3615 :- Basic Search : Search with _ALL_ENTITY_TYPES and classification doesn't not fetch any entity associated to that classification

2020-02-13 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72127/
---

(Updated Feb. 13, 2020, 9:49 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Bugs: ATLAS-3615
https://issues.apache.org/jira/browse/ATLAS-3615


Repository: atlas


Description
---

Issues:
1) Combination of typeName:_ALL_ENTITY_TYPES and classification doesn't return 
result
Why? As _ALL_ENTITY_TYPES value is assigned as [* TO *] results show up when 
searched from solr but not from graphdb
2) Combination of typeName:_ALL_ENTITY_TYPES, classification, 
query(SearchByText) doesn't return result
Why? result from solr, thru "/freetext" is correct, but onsearching if resulted 
entitytype is in typeAndSubtypes of given entityType, it breaks because 
typeAndSubTypes of _ALL_ENTITY_TYPES is [* TO *]


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/EntitySearchProcessor.java 
b5606d046 
  repository/src/main/java/org/apache/atlas/discovery/SearchContext.java 
353411363 


Diff: https://reviews.apache.org/r/72127/diff/2/

Changes: https://reviews.apache.org/r/72127/diff/1-2/


Testing
---

For issue1:
1) tested with values _ALL_ENTITY_TYPES + EntityFilter + classification -> 
result fetches from gremlin and shows up correct result
2) tested with values _ALL_ENTITY_TYPES + classification -> result fetches from 
gremlin and shows up correct result

For issue2:
1) tested with values _ALL_ENTITY_TYPES + classification + query(SearchByText) 
-> shows up correct result


Thanks,

Pinal Shah



Re: Review Request 72076: ATLAS-3606 : Hive Hook: Missing HIVE_PROCESS attribute values

2020-02-03 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72076/
---

(Updated Feb. 3, 2020, 11:53 a.m.)


Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Bugs: ATLAS-3606
https://issues.apache.org/jira/browse/ATLAS-3606


Repository: atlas


Description
---

When we create table in hive, value of attributes cluster_name, owner, 
query_text is missing in the corresponding hive_process entity on Atlas


Diffs
-

  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java
 60dc1147f 


Diff: https://reviews.apache.org/r/72076/diff/1/


Testing (updated)
---

1) CREATETABLE operation
2) CREATETABLE_AS_SELECT operation 
3) CREATEVIEW operation
4) ALTERVIEW_AS operation
5) IMPORT operation
6) EXPORT operation, here owner is N/A

tested by changing user in hive and then creating operation, the owner in 
hive_process is reflected


Thanks,

Pinal Shah



Review Request 72076: ATLAS-3606 : Hive Hook: Missing HIVE_PROCESS attribute values

2020-02-03 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72076/
---

Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
Subramanian.


Bugs: ATLAS-3606
https://issues.apache.org/jira/browse/ATLAS-3606


Repository: atlas


Description
---

When we create table in hive, value of attributes cluster_name, owner, 
query_text is missing in the corresponding hive_process entity on Atlas


Diffs
-

  
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java
 60dc1147f 


Diff: https://reviews.apache.org/r/72076/diff/1/


Testing
---

1) CREATETABLE operation
2) CREATETABLE_AS_SELECT operation 

tested by changing user in hive and then creating operation, the owner in 
hive_process is reflected


Thanks,

Pinal Shah



Re: Review Request 72060: ATLAS-3601 No LABEL_DELETE event is audited when last label is deleted.

2020-01-31 Thread Pinal Shah


> On Jan. 30, 2020, 10:48 p.m., Sarath Subramanian wrote:
> > repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
> > Line 161 (original), 161 (patched)
> > <https://reviews.apache.org/r/72060/diff/1/?file=2210004#file2210004line161>
> >
> > how does adding a new field to hbase rowkey fix the issue? We are 
> > changing the rowkey format here. Will this be backward compatible? will 
> > older rows be retrieved using the new key format? Please review.

**Issue-2: When we add label and delete label from multiple label present, 
simultaneously, Only LABEL_ADD is audited**

*how does adding a new field to hbase rowkey fix the issue?*
=> As row_key(hbase) of both the audits were same, hbase couldnt insert both 
the audits, for this we need unique row_key for each audit,so whenever getKey() 
of AbstractStorageBasedAuditRepository class is called, it will assure to 
return unique Key
https://github.com/apache/atlas/blob/master/repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java#L158

*We are changing the rowkey format here*
=> Yes old row_key = ::
   new row_key = 
:::   
https://github.com/apache/atlas/blob/master/repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java#L158

*Will this be backward compatible? will older rows be retrieved using the new 
key format?*
=> Yes it will be backward compatible and will retrieve older rows.
=> For retrieval, we make use of entityId and from the result, we extract only 
entityId and timestamp from row_key
   below is the snippet.
String[] parts = key.split(FIELD_SEPARATOR);
event.setEntityId(parts[0]);
event.setTimestamp(Long.valueOf(parts[1]));
   
https://github.com/apache/atlas/blob/master/repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java#L464


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72060/#review219442
---


On Jan. 30, 2020, 12:10 p.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72060/
> ---
> 
> (Updated Jan. 30, 2020, 12:10 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3601
> https://issues.apache.org/jira/browse/ATLAS-3601
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> 1) When 1 label among multiple labels is removed , LABEL_DELETE is audited. 
> But when all labels are deleted or the last label is deleted , no 
> LABEL_DELETE is audited.
> 2)When we add label and delete label from multiple label present, 
> simultaneously, Only LABEL_ADD is audited
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
>  f6b741b5e 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
>  e427a59e7 
> 
> 
> Diff: https://reviews.apache.org/r/72060/diff/1/
> 
> 
> Testing
> ---
> 
> 1) Added one label - one audit for LABEL_ADD
> 2) Added multiple labels simultaneously - one audit for LABEL_ADD,with 
> multiple label names in Details
> 3) Deleted one of the label - one audit for LABEL_DELETE
> 4) Deleted multiple labels simultaneously - one audit for LABEL_DELETE,with 
> multiple label names in Details
> 5) deleled last label left - one audit for LABEL_DELETE
> 6) Added one label and deleted two labels simultaneously -two audits, one 
> audit for LABEL_ADD and second for LABEL_DELETE ,with multiple label names in 
> Details
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Contributor Request

2020-01-30 Thread Pinal Shah
Hi Team,

I would like to contribute to the Atlas project, Can you please add me as
contributor.

Thanks,
Pinal Shah


Review Request 72060: ATLAS-3601 No LABEL_DELETE event is audited when last label is deleted.

2020-01-30 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72060/
---

Review request for atlas and Nixon Rodrigues.


Bugs: ATLAS-3601
https://issues.apache.org/jira/browse/ATLAS-3601


Repository: atlas


Description
---

1) When 1 label among multiple labels is removed , LABEL_DELETE is audited. But 
when all labels are deleted or the last label is deleted , no LABEL_DELETE is 
audited.
2)When we add label and delete label from multiple label present, 
simultaneously, Only LABEL_ADD is audited


Diffs
-

  
repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
 f6b741b5e 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphMapper.java
 e427a59e7 


Diff: https://reviews.apache.org/r/72060/diff/1/


Testing
---

1) Added one label - one audit for LABEL_ADD
2) Added multiple labels simultaneously - one audit for LABEL_ADD,with multiple 
label names in Details
3) Deleted one of the label - one audit for LABEL_DELETE
4) Deleted multiple labels simultaneously - one audit for LABEL_DELETE,with 
multiple label names in Details
5) deleled last label left - one audit for LABEL_DELETE
6) Added one label and deleted two labels simultaneously -two audits, one audit 
for LABEL_ADD and second for LABEL_DELETE ,with multiple label names in Details


Thanks,

Pinal Shah