Re: Review Request 72698: ATLAS-3875: Introduce sample project for AtlasClient

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72698/#review221300
---




atlas-examples/src/main/java/org/apache/atlas/AtlasClientBaseExample.java
Lines 18 (patched)


- move all Java files to to org.apache.atlas.examples package
- also, consider moving this to a sub-project named SampleApp, similar to 
Ranger examples here: 
https://github.com/apache/ranger/tree/master/ranger-examples



atlas-examples/src/main/java/org/apache/atlas/AtlasClientBaseExample.java
Lines 31 (patched)


AtlasClientBaseExample => SampleApp



atlas-examples/src/main/java/org/apache/atlas/AtlasClientBaseExample.java
Lines 32 (patched)


CLUSTER_SUFFIX => METADATA_NAMESPACE_SUFFIX



atlas-examples/src/main/java/org/apache/atlas/AtlasClientBaseExample.java
Lines 55 (patched)


empSalaryGlossary is used only in GlossaryExample. Consider moving this 
filed to GlossaryExample. Review other fields for the same.



atlas-examples/src/main/java/org/apache/atlas/AtlasClientBaseExample.java
Lines 62 (patched)


urls => atlasServerUrls
 - also in #66, #73



atlas-examples/src/main/java/org/apache/atlas/DiscoveryExample.java
Lines 28 (patched)


Classes DiscoveryExample, EntityRestBaseExample, GlossaryExample, 
LineageExample and TypeDefExample derive from only to share few constants. I 
suggest to avoid this hierachy. To share constants, consider having all 
constants in a class like 'SampleAppConstants'.


- Madhan Neethiraj


On July 21, 2020, 4:24 p.m., Jyoti Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72698/
> ---
> 
> (Updated July 21, 2020, 4:24 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Sarath 
> Subramanian, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3875
> https://issues.apache.org/jira/browse/ATLAS-3875
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Using this project users can get an idea as how to integrate with Atlas using 
> AtlasCleint. This helps the user to understand the basic rest functionality 
> of Atlas such as
> 
> - EntityRest
> - TypeDefRest
> - DiscoveryRest
> - LineageRest
> - GlossaryRest
> 
> 
> Diffs
> -
> 
>   atlas-examples/pom.xml PRE-CREATION 
>   atlas-examples/src/main/java/org/apache/atlas/AtlasClientBaseExample.java 
> PRE-CREATION 
>   atlas-examples/src/main/java/org/apache/atlas/DiscoveryExample.java 
> PRE-CREATION 
>   atlas-examples/src/main/java/org/apache/atlas/EntityRestBaseExample.java 
> PRE-CREATION 
>   atlas-examples/src/main/java/org/apache/atlas/GlossaryExample.java 
> PRE-CREATION 
>   atlas-examples/src/main/java/org/apache/atlas/LineageExample.java 
> PRE-CREATION 
>   atlas-examples/src/main/java/org/apache/atlas/TypeDefExample.java 
> PRE-CREATION 
>   pom.xml 5e0442ae5 
> 
> 
> Diff: https://reviews.apache.org/r/72698/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Jyoti Singh
> 
>

Re: Review Request 72695: Optional configuration to support locks on JanusGraph to ensure data consitency.

[Madhan Neethiraj]

> On July 21, 2020, 8:38 p.m., Ashutosh Mestry wrote:
> > Here's what I have tried so far:
> > - Concurrent entity creation using my own test rig. This creates entities 
> > concurrently but it does not allow for entity with same qualifiedName to be 
> > created by different workers.
> > - Ingest via Kafka queue. This is to verify that there is not performance 
> > degradation. 
> > 
> > So far results are positive. I did not obseve any performance or impact on 
> > accuracy.
> > 
> > Yet to try:
> > - Migration from 0.8 to latest.
> > - Migration import using ZipDirect format.

Ashutosh - what is the behavior seen in case of simultaneous attempts to entity 
an entity? Do all but 1 attempt fail?


- Madhan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72695/#review221296
---


On July 20, 2020, 9:48 p.m., Damian Warszawski wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72695/
> ---
> 
> (Updated July 20, 2020, 9:48 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Bolke de Bruin, madhan, and Sarath 
> Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Optional configuration to support locks on JanusGraph to ensure data 
> consitency.
> 
> JanusGraph is eventually consistent by default which is efficient but results 
> in duplicates when race condition occurs.
> 
> 
> Reference to jira 
> https://issues.apache.org/jira/projects/ATLAS/issues/ATLAS-3398
> 
> 
> Diffs
> -
> 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphManagement.java
>  fca789027 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphManagement.java
>  6ef9cb76c 
>   
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/AbstractGraphDatabaseTest.java
>  35004157f 
>   
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusDatabaseTest.java
>  5cd55093e 
>   intg/src/main/java/org/apache/atlas/ApplicationProperties.java e662c8fae 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  e35f3594f 
>   
> repository/src/main/java/org/apache/atlas/repository/patches/ConcurrentPatchProcessor.java
>  5a9ac2abe 
>   
> repository/src/main/java/org/apache/atlas/repository/patches/UniqueAttributePatch.java
>  d3111f110 
> 
> 
> Diff: https://reviews.apache.org/r/72695/diff/1/
> 
> 
> Testing
> ---
> 
> Not possible to reproduce the error on local machine. Enable locking on our 
> dev env and have not introduce any regression.
> 
> 
> Thanks,
> 
> Damian Warszawski
> 
>

Re: Review Request 72695: Optional configuration to support locks on JanusGraph to ensure data consitency.

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72695/#review221297
---




graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphManagement.java
Lines 260 (patched)


Consistency lock might be relevant/needed only for unique-index i.e. 
isUnique=true. If this true, consider calling setConsistency(LOCK) when 
isUnique is true, without requiring additional argument. Same applies for #278 
as well.

It will be useful support following configuration, to optionally disable 
consistentcy-lock:
  atlas.graph.storage.unique-key.consitency-lock.enabled

This configuration can be sent to AtlasJanusGraphManagement during 
construction - from AtlasJanusGraph.getManagementSystem().

Above will avoid updates to many methods for the addition of lockEnabled 
argument.


- Madhan Neethiraj


On July 20, 2020, 9:48 p.m., Damian Warszawski wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72695/
> ---
> 
> (Updated July 20, 2020, 9:48 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Bolke de Bruin, madhan, and Sarath 
> Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Optional configuration to support locks on JanusGraph to ensure data 
> consitency.
> 
> JanusGraph is eventually consistent by default which is efficient but results 
> in duplicates when race condition occurs.
> 
> 
> Reference to jira 
> https://issues.apache.org/jira/projects/ATLAS/issues/ATLAS-3398
> 
> 
> Diffs
> -
> 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphManagement.java
>  fca789027 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphManagement.java
>  6ef9cb76c 
>   
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/AbstractGraphDatabaseTest.java
>  35004157f 
>   
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusDatabaseTest.java
>  5cd55093e 
>   intg/src/main/java/org/apache/atlas/ApplicationProperties.java e662c8fae 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  e35f3594f 
>   
> repository/src/main/java/org/apache/atlas/repository/patches/ConcurrentPatchProcessor.java
>  5a9ac2abe 
>   
> repository/src/main/java/org/apache/atlas/repository/patches/UniqueAttributePatch.java
>  d3111f110 
> 
> 
> Diff: https://reviews.apache.org/r/72695/diff/1/
> 
> 
> Testing
> ---
> 
> Not possible to reproduce the error on local machine. Enable locking on our 
> dev env and have not introduce any regression.
> 
> 
> Thanks,
> 
> Damian Warszawski
> 
>

Re: Review Request 72695: Optional configuration to support locks on JanusGraph to ensure data consitency.

[Ashutosh Mestry via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72695/#review221296
---



Here's what I have tried so far:
- Concurrent entity creation using my own test rig. This creates entities 
concurrently but it does not allow for entity with same qualifiedName to be 
created by different workers.
- Ingest via Kafka queue. This is to verify that there is not performance 
degradation. 

So far results are positive. I did not obseve any performance or impact on 
accuracy.

Yet to try:
- Migration from 0.8 to latest.
- Migration import using ZipDirect format.

- Ashutosh Mestry


On July 20, 2020, 9:48 p.m., Damian Warszawski wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72695/
> ---
> 
> (Updated July 20, 2020, 9:48 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Bolke de Bruin, madhan, and Sarath 
> Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Optional configuration to support locks on JanusGraph to ensure data 
> consitency.
> 
> JanusGraph is eventually consistent by default which is efficient but results 
> in duplicates when race condition occurs.
> 
> 
> Reference to jira 
> https://issues.apache.org/jira/projects/ATLAS/issues/ATLAS-3398
> 
> 
> Diffs
> -
> 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphManagement.java
>  fca789027 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphManagement.java
>  6ef9cb76c 
>   
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/AbstractGraphDatabaseTest.java
>  35004157f 
>   
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusDatabaseTest.java
>  5cd55093e 
>   intg/src/main/java/org/apache/atlas/ApplicationProperties.java e662c8fae 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
>  e35f3594f 
>   
> repository/src/main/java/org/apache/atlas/repository/patches/ConcurrentPatchProcessor.java
>  5a9ac2abe 
>   
> repository/src/main/java/org/apache/atlas/repository/patches/UniqueAttributePatch.java
>  d3111f110 
> 
> 
> Diff: https://reviews.apache.org/r/72695/diff/1/
> 
> 
> Testing
> ---
> 
> Not possible to reproduce the error on local machine. Enable locking on our 
> dev env and have not introduce any regression.
> 
> 
> Thanks,
> 
> Damian Warszawski
> 
>

[jira] [Updated] (ATLAS-3900) UI: Allow user to select the date range for date attribute in basic search

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3900:
---
Description: 
In basic search attribute popup if the user selects the date type attribute 
then UI should show a few quick search operator and custom range selection. 
example:

!Screen Shot 2020-07-21 at 10.59.59 PM.png|width=631,height=282!

 

!Screen Shot 2020-07-21 at 11.00.08 PM.png|width=630,height=302!

 

!Screen Shot 2020-07-21 at 11.00.15 PM.png|width=630,height=304!

  was:
In basic search attribute popup if the user selects the date type attribute 
then UI should show a few quick search operator and custom range selection. 
example:

!Screen Shot 2020-07-08 at 11.49.25 AM.png|width=457,height=230!


> UI: Allow user to select the date range for date attribute in basic search
> --
>
> Key: ATLAS-3900
> URL: https://issues.apache.org/jira/browse/ATLAS-3900
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-3900-1.patch, ATLAS-3900.patch, Screen Shot 
> 2020-07-21 at 10.59.59 PM.png, Screen Shot 2020-07-21 at 11.00.08 PM.png, 
> Screen Shot 2020-07-21 at 11.00.15 PM.png
>
>
> In basic search attribute popup if the user selects the date type attribute 
> then UI should show a few quick search operator and custom range selection. 
> example:
> !Screen Shot 2020-07-21 at 10.59.59 PM.png|width=631,height=282!
>  
> !Screen Shot 2020-07-21 at 11.00.08 PM.png|width=630,height=302!
>  
> !Screen Shot 2020-07-21 at 11.00.15 PM.png|width=630,height=304!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3900) UI: Allow user to select the date range for date attribute in basic search

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3900:
---
Attachment: Screen Shot 2020-07-21 at 11.00.15 PM.png
Screen Shot 2020-07-21 at 11.00.08 PM.png
Screen Shot 2020-07-21 at 10.59.59 PM.png

> UI: Allow user to select the date range for date attribute in basic search
> --
>
> Key: ATLAS-3900
> URL: https://issues.apache.org/jira/browse/ATLAS-3900
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-3900-1.patch, ATLAS-3900.patch, Screen Shot 
> 2020-07-21 at 10.59.59 PM.png, Screen Shot 2020-07-21 at 11.00.08 PM.png, 
> Screen Shot 2020-07-21 at 11.00.15 PM.png
>
>
> In basic search attribute popup if the user selects the date type attribute 
> then UI should show a few quick search operator and custom range selection. 
> example:
> !Screen Shot 2020-07-08 at 11.49.25 AM.png|width=457,height=230!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3900) UI: Allow user to select the date range for date attribute in basic search

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3900:
---
Attachment: (was: Screen Shot 2020-07-08 at 11.49.25 AM.png)

> UI: Allow user to select the date range for date attribute in basic search
> --
>
> Key: ATLAS-3900
> URL: https://issues.apache.org/jira/browse/ATLAS-3900
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-3900-1.patch, ATLAS-3900.patch, Screen Shot 
> 2020-07-21 at 10.59.59 PM.png, Screen Shot 2020-07-21 at 11.00.08 PM.png, 
> Screen Shot 2020-07-21 at 11.00.15 PM.png
>
>
> In basic search attribute popup if the user selects the date type attribute 
> then UI should show a few quick search operator and custom range selection. 
> example:
> !Screen Shot 2020-07-08 at 11.49.25 AM.png|width=457,height=230!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3897) UI: Normalize list of propagated classifications

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3897:
---
Attachment: ATLAS-3897-3.patch

> UI: Normalize list of propagated classifications
> 
>
> Key: ATLAS-3897
> URL: https://issues.apache.org/jira/browse/ATLAS-3897
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-3897-1.patch, ATLAS-3897-2.patch, 
> ATLAS-3897-3.patch, ATLAS-3897.patch, Screen Shot 2020-07-17 at 6.13.04 PM.png
>
>
> When an entity has multiple instances of the same propagated classification 
> (e.g. via join of Hive tables), the lineage view will show each of them in 
> the list. The result may be a long list of classifications with identical 
> entries They don't seem to offer any additional information.
> I propose we normalize the list (think _uniq_ command in unix)
>  
> !Screen Shot 2020-07-17 at 6.13.04 PM.png|width=593,height=359!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3900) UI: Allow user to select the date range for date attribute in basic search

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3900:
---
Attachment: ATLAS-3900-1.patch

> UI: Allow user to select the date range for date attribute in basic search
> --
>
> Key: ATLAS-3900
> URL: https://issues.apache.org/jira/browse/ATLAS-3900
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Attachments: ATLAS-3900-1.patch, ATLAS-3900.patch, Screen Shot 
> 2020-07-08 at 11.49.25 AM.png
>
>
> In basic search attribute popup if the user selects the date type attribute 
> then UI should show a few quick search operator and custom range selection. 
> example:
> !Screen Shot 2020-07-08 at 11.49.25 AM.png|width=457,height=230!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3900) UI: Allow user to select the date range for date attribute in basic search

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3900:
---
Fix Version/s: 2.2.0
   3.0.0

> UI: Allow user to select the date range for date attribute in basic search
> --
>
> Key: ATLAS-3900
> URL: https://issues.apache.org/jira/browse/ATLAS-3900
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-3900-1.patch, ATLAS-3900.patch, Screen Shot 
> 2020-07-08 at 11.49.25 AM.png
>
>
> In basic search attribute popup if the user selects the date type attribute 
> then UI should show a few quick search operator and custom range selection. 
> example:
> !Screen Shot 2020-07-08 at 11.49.25 AM.png|width=457,height=230!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 72698: ATLAS-3875: Introduce sample project for AtlasClient

[Jyoti Singh]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72698/
---

(Updated July 21, 2020, 4:24 p.m.)


Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Sarath 
Subramanian, and Sidharth Mishra.


Bugs: ATLAS-3875
https://issues.apache.org/jira/browse/ATLAS-3875


Repository: atlas


Description
---

Using this project users can get an idea as how to integrate with Atlas using 
AtlasCleint. This helps the user to understand the basic rest functionality of 
Atlas such as

- EntityRest
- TypeDefRest
- DiscoveryRest
- LineageRest
- GlossaryRest


Diffs (updated)
-

  atlas-examples/pom.xml PRE-CREATION 
  atlas-examples/src/main/java/org/apache/atlas/AtlasClientBaseExample.java 
PRE-CREATION 
  atlas-examples/src/main/java/org/apache/atlas/DiscoveryExample.java 
PRE-CREATION 
  atlas-examples/src/main/java/org/apache/atlas/EntityRestBaseExample.java 
PRE-CREATION 
  atlas-examples/src/main/java/org/apache/atlas/GlossaryExample.java 
PRE-CREATION 
  atlas-examples/src/main/java/org/apache/atlas/LineageExample.java 
PRE-CREATION 
  atlas-examples/src/main/java/org/apache/atlas/TypeDefExample.java 
PRE-CREATION 
  pom.xml 5e0442ae5 


Diff: https://reviews.apache.org/r/72698/diff/2/

Changes: https://reviews.apache.org/r/72698/diff/1-2/


Testing
---


Thanks,

Jyoti Singh

Review Request 72698: ATLAS-3875: Introduce sample project for AtlasClient

[Jyoti Singh]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72698/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Sarath 
Subramanian, and Sidharth Mishra.


Bugs: ATLAS-3875
https://issues.apache.org/jira/browse/ATLAS-3875


Repository: atlas


Description
---

Using this project users can get an idea as how to integrate with Atlas using 
AtlasCleint. This helps the user to understand the basic rest functionality of 
Atlas such as

- EntityRest
- TypeDefRest
- DiscoveryRest
- LineageRest
- GlossaryRest


Diffs
-

  atlas-examples/pom.xml PRE-CREATION 
  atlas-examples/src/main/java/org/apache/atlas/AtlasClientBaseExample.java 
PRE-CREATION 
  atlas-examples/src/main/java/org/apache/atlas/DiscoveryExample.java 
PRE-CREATION 
  atlas-examples/src/main/java/org/apache/atlas/EntityRestBaseExample.java 
PRE-CREATION 
  atlas-examples/src/main/java/org/apache/atlas/GlossaryExample.java 
PRE-CREATION 
  atlas-examples/src/main/java/org/apache/atlas/LineageExample.java 
PRE-CREATION 
  atlas-examples/src/main/java/org/apache/atlas/TypeDefExample.java 
PRE-CREATION 
  pom.xml 5e0442ae5 


Diff: https://reviews.apache.org/r/72698/diff/1/


Testing
---


Thanks,

Jyoti Singh

[jira] [Updated] (ATLAS-3900) UI: Allow user to select the date range for date attribute in basic search

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3900:
---
Attachment: ATLAS-3900.patch

> UI: Allow user to select the date range for date attribute in basic search
> --
>
> Key: ATLAS-3900
> URL: https://issues.apache.org/jira/browse/ATLAS-3900
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Attachments: ATLAS-3900.patch, Screen Shot 2020-07-08 at 11.49.25 
> AM.png
>
>
> In basic search attribute popup if the user selects the date type attribute 
> then UI should show a few quick search operator and custom range selection. 
> example:
> !Screen Shot 2020-07-08 at 11.49.25 AM.png|width=457,height=230!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 72646: ATLAS-3876 : Relationship Search API not showing correct approximateCount

[Pinal Shah]

> On July 20, 2020, 10:54 p.m., Ashutosh Mestry wrote:
> > repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
> > Line 649 (original), 662 (patched)
> > 
> >
> > Consider replacing this with AtlasGraphTraversal. This should be easy 
> > change but will give good performance mileage.

Thanks Ashutosh for reviewing.

You mean to add these methods, (bothE, otherV) to AtlasGraphTraversal and make 
use from it?
How will it make difference in performance?


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72646/#review221273
---


On July 6, 2020, 9:14 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72646/
> ---
> 
> (Updated July 6, 2020, 9:14 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3876
> https://issues.apache.org/jira/browse/ATLAS-3876
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Issue:**
> Relationship api doesn't provide approximate count of the related entities in 
> the response.
> 
> **Workaround:**
> Get the total count of related given entity , irrespective of the 
> offset/limit.
> 
> 
> Also this patch includes **improvement in the time taken to fetch related 
> entities**  .
> Average time taken for the Api to search relationship entities having **5000 
> end vertices** with limit **500**:
> Before: 9seconds
> After applying this patch : 3seconds
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  4b9564295 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
>  863a00350 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> 5069d78c8 
> 
> 
> Diff: https://reviews.apache.org/r/72646/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested
> Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2011 
> (Failed in Impala build)
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>

Re: Review Request 72646: ATLAS-3876 : Relationship Search API not showing correct approximateCount

[Pinal Shah]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72646/#review221282
---




repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
Lines 705 (patched)


I tested it with 5000 edge vertices.
Will vertexTotal() method be efficient than this?


- Pinal Shah


On July 6, 2020, 9:14 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72646/
> ---
> 
> (Updated July 6, 2020, 9:14 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3876
> https://issues.apache.org/jira/browse/ATLAS-3876
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Issue:**
> Relationship api doesn't provide approximate count of the related entities in 
> the response.
> 
> **Workaround:**
> Get the total count of related given entity , irrespective of the 
> offset/limit.
> 
> 
> Also this patch includes **improvement in the time taken to fetch related 
> entities**  .
> Average time taken for the Api to search relationship entities having **5000 
> end vertices** with limit **500**:
> Before: 9seconds
> After applying this patch : 3seconds
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  4b9564295 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
>  863a00350 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> 5069d78c8 
> 
> 
> Diff: https://reviews.apache.org/r/72646/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested
> Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2011 
> (Failed in Impala build)
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>

Re: Review Request 72646: ATLAS-3876 : Relationship Search API not showing correct approximateCount

[Pinal Shah]

> On July 20, 2020, 10:54 p.m., Ashutosh Mestry wrote:
> > repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
> > Lines 705 (patched)
> > 
> >
> > What is the volume of data you have tested this with?

I tested it with 5000 edge vertices.
Will vertexTotal() method be efficient than this?


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72646/#review221273
---


On July 6, 2020, 9:14 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72646/
> ---
> 
> (Updated July 6, 2020, 9:14 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3876
> https://issues.apache.org/jira/browse/ATLAS-3876
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Issue:**
> Relationship api doesn't provide approximate count of the related entities in 
> the response.
> 
> **Workaround:**
> Get the total count of related given entity , irrespective of the 
> offset/limit.
> 
> 
> Also this patch includes **improvement in the time taken to fetch related 
> entities**  .
> Average time taken for the Api to search relationship entities having **5000 
> end vertices** with limit **500**:
> Before: 9seconds
> After applying this patch : 3seconds
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  4b9564295 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java
>  863a00350 
>   repository/src/main/java/org/apache/atlas/util/SearchPredicateUtil.java 
> 5069d78c8 
> 
> 
> Diff: https://reviews.apache.org/r/72646/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested
> Precommit : https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/2011 
> (Failed in Impala build)
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>

[jira] [Updated] (ATLAS-3900) UI: Allow user to select the date range for date attribute in basic search

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3900:
---
Description: 
In basic search attribute popup if the user selects the date type attribute 
then UI should show a few quick search operator and custom range selection. 
example:

!Screen Shot 2020-07-08 at 11.49.25 AM.png|width=457,height=230!

  was:
In basic search attribute popup if the user selects the date type attribute 
then UI should show a few quick search operator and custom range selection.

!Screen Shot 2020-07-08 at 11.49.25 AM.png|width=457,height=230!


> UI: Allow user to select the date range for date attribute in basic search
> --
>
> Key: ATLAS-3900
> URL: https://issues.apache.org/jira/browse/ATLAS-3900
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Attachments: Screen Shot 2020-07-08 at 11.49.25 AM.png
>
>
> In basic search attribute popup if the user selects the date type attribute 
> then UI should show a few quick search operator and custom range selection. 
> example:
> !Screen Shot 2020-07-08 at 11.49.25 AM.png|width=457,height=230!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3900) UI: Allow user to select the date range for date attribute in basic search

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3900:
---
Description: 
In basic search attribute popup if the user selects the date type attribute 
then UI should show a few quick search operator and custom range selection.

!Screen Shot 2020-07-08 at 11.49.25 AM.png|width=457,height=230!

> UI: Allow user to select the date range for date attribute in basic search
> --
>
> Key: ATLAS-3900
> URL: https://issues.apache.org/jira/browse/ATLAS-3900
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Attachments: Screen Shot 2020-07-08 at 11.49.25 AM.png
>
>
> In basic search attribute popup if the user selects the date type attribute 
> then UI should show a few quick search operator and custom range selection.
> !Screen Shot 2020-07-08 at 11.49.25 AM.png|width=457,height=230!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3900) UI: Allow user to select the date range for date attribute in basic search

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3900:
---
Attachment: Screen Shot 2020-07-08 at 11.49.25 AM.png

> UI: Allow user to select the date range for date attribute in basic search
> --
>
> Key: ATLAS-3900
> URL: https://issues.apache.org/jira/browse/ATLAS-3900
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Attachments: Screen Shot 2020-07-08 at 11.49.25 AM.png
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3900) UI: Allow user to select the date range for date attribute in basic search

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3900:
---
Summary: UI: Allow user to select the date range for date attribute in 
basic search  (was: UI: Allow user to select the date range or quick selection)

> UI: Allow user to select the date range for date attribute in basic search
> --
>
> Key: ATLAS-3900
> URL: https://issues.apache.org/jira/browse/ATLAS-3900
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (ATLAS-3900) UI: Allow user to select the date range or quick selection

[Keval Bhatt (Jira)]

Keval Bhatt created ATLAS-3900:
--

 Summary: UI: Allow user to select the date range or quick selection
 Key: ATLAS-3900
 URL: https://issues.apache.org/jira/browse/ATLAS-3900
 Project: Atlas
  Issue Type: Sub-task
Reporter: Keval Bhatt
Assignee: Keval Bhatt






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3897) UI: Normalize list of propagated classifications

[Keval Bhatt (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-3897:
---
Attachment: ATLAS-3897-2.patch

> UI: Normalize list of propagated classifications
> 
>
> Key: ATLAS-3897
> URL: https://issues.apache.org/jira/browse/ATLAS-3897
> Project: Atlas
>  Issue Type: Bug
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: ATLAS-3897-1.patch, ATLAS-3897-2.patch, 
> ATLAS-3897.patch, Screen Shot 2020-07-17 at 6.13.04 PM.png
>
>
> When an entity has multiple instances of the same propagated classification 
> (e.g. via join of Hive tables), the lineage view will show each of them in 
> the list. The result may be a long list of classifications with identical 
> entries They don't seem to offer any additional information.
> I propose we normalize the list (think _uniq_ command in unix)
>  
> !Screen Shot 2020-07-17 at 6.13.04 PM.png|width=593,height=359!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-3398) Duplicates for unique attributes

[Damian Warszawski (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17161884#comment-17161884
 ] 

Damian Warszawski commented on ATLAS-3398:
--

According to our observations that can be a race condition between hive-hook 
(kafka event) and profiler (org.apache.atlas:atlas-client-v2:2.0.0). It can be 
also the case with atlas-client which make implicit retries while calling Atlas 
API. 

> Duplicates for unique attributes 
> -
>
> Key: ATLAS-3398
> URL: https://issues.apache.org/jira/browse/ATLAS-3398
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0, trunk
>Reporter: Bolke de Bruin
>Priority: Blocker
>  Labels: integrity
> Attachments: zrzut_ekranu_2019-09-03_o_10.28.50.png
>
>
> We are seeing issues with entities being added to Atlas with duplicate 
> "qualifiedName". The guids differ and other attributes do also differ. Below 
> a graph that shows the distribution over time for duplicates. We have 
> difficulty determining which one is the right one (as they are different) in 
> order to clean them up.
> We are also not the only ones encountering this as you can in the linked 
> issue.
> We have noticed that Atlas does not use the 
> [locking|https://docs.janusgraph.org/master/advanced-topics/eventual-consistency/]
>  mechanism of Janus to prevent this:
>  
> !zrzut_ekranu_2019-09-03_o_10.28.50.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Comment Edited] (ATLAS-3833) Packaging for atlas index repair tool

[Nixon Rodrigues (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17161787#comment-17161787
 ] 

Nixon Rodrigues edited comment on ATLAS-3833 at 7/21/20, 7:10 AM:
--

The patch [^ATLAS-3833-1.patch] adds copying and assembling artifacts such jar, 
log4j xml , README and py file into tools/atlas-index-repair directory inside 
atlas-server distro package.  


was (Author: nixonrodrigues):
The patch [^ATLAS-3833-1.patch] adds copying and assembling artefacts such jar, 
log4j xml , README and py file into tools/atlas-index-repair directory inside 
atlas-server distro package.  

> Packaging for atlas index repair tool 
> --
>
> Key: ATLAS-3833
> URL: https://issues.apache.org/jira/browse/ATLAS-3833
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: chaitali borole
>Assignee: chaitali borole
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: ATLAS-3833-1.patch
>
>
> Packaging for atlas repair index was missing in Apache master build:
> Added descriptor in pom.xml for atlas-repair-index-package.xml
> Added readme
> Added atlas-repair-index-package.xml



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Reopened] (ATLAS-3833) Packaging for atlas index repair tool

[Nixon Rodrigues (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nixon Rodrigues reopened ATLAS-3833:


> Packaging for atlas index repair tool 
> --
>
> Key: ATLAS-3833
> URL: https://issues.apache.org/jira/browse/ATLAS-3833
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: chaitali borole
>Assignee: chaitali borole
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: ATLAS-3833-1.patch
>
>
> Packaging for atlas repair index was missing in Apache master build:
> Added descriptor in pom.xml for atlas-repair-index-package.xml
> Added readme
> Added atlas-repair-index-package.xml



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3833) Packaging for atlas index repair tool

[Nixon Rodrigues (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nixon Rodrigues updated ATLAS-3833:
---
Fix Version/s: (was: 2.0.0)

> Packaging for atlas index repair tool 
> --
>
> Key: ATLAS-3833
> URL: https://issues.apache.org/jira/browse/ATLAS-3833
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: chaitali borole
>Assignee: chaitali borole
>Priority: Major
> Attachments: ATLAS-3833-1.patch
>
>
> Packaging for atlas repair index was missing in Apache master build:
> Added descriptor in pom.xml for atlas-repair-index-package.xml
> Added readme
> Added atlas-repair-index-package.xml



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (ATLAS-3833) Packaging for atlas index repair tool

[Nixon Rodrigues (Jira)]

[ 
https://issues.apache.org/jira/browse/ATLAS-3833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17161787#comment-17161787
 ] 

Nixon Rodrigues commented on ATLAS-3833:


The patch [^ATLAS-3833-1.patch] adds copying and assembling artefacts such jar, 
log4j xml , README and py file into tools/atlas-index-repair directory inside 
atlas-server distro package.  

> Packaging for atlas index repair tool 
> --
>
> Key: ATLAS-3833
> URL: https://issues.apache.org/jira/browse/ATLAS-3833
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: chaitali borole
>Assignee: chaitali borole
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: ATLAS-3833-1.patch
>
>
> Packaging for atlas repair index was missing in Apache master build:
> Added descriptor in pom.xml for atlas-repair-index-package.xml
> Added readme
> Added atlas-repair-index-package.xml



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (ATLAS-3833) Packaging for atlas index repair tool

[Nixon Rodrigues (Jira)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-3833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nixon Rodrigues updated ATLAS-3833:
---
Attachment: ATLAS-3833-1.patch

> Packaging for atlas index repair tool 
> --
>
> Key: ATLAS-3833
> URL: https://issues.apache.org/jira/browse/ATLAS-3833
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: chaitali borole
>Assignee: chaitali borole
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: ATLAS-3833-1.patch
>
>
> Packaging for atlas repair index was missing in Apache master build:
> Added descriptor in pom.xml for atlas-repair-index-package.xml
> Added readme
> Added atlas-repair-index-package.xml



--
This message was sent by Atlassian Jira
(v8.3.4#803005)