Re: Review Request 72630: ATLAS-3868: [Regression] removing a term-association doesn't remove classifications propagated from the term

2020-06-29 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72630/#review221094
---




repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
Lines 159 (patched)


Shouldn't removeTagPropagation() be called from inside deleteEdge(edge, 
isForceDelete) implemenation? deleteEdge(edge, isForceDelete) seems to be 
called from multiple places; please review if tag-propagation needs to be 
re-evaluated in these contexts.


- Madhan Neethiraj


On June 29, 2020, 10:28 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72630/
> ---
> 
> (Updated June 29, 2020, 10:28 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
> Nikhil Bonte, Nixon Rodrigues, Pinal Shah, and Sidharth Mishra.
> 
> 
> Bugs: ATLAS-3868
> https://issues.apache.org/jira/browse/ATLAS-3868
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Issue:
> ==
> removing a term-association doesn't remove classifications propagated from 
> the term
> 
> Cause:
> ==
> Regression caused by https://issues.apache.org/jira/browse/ATLAS-3863
> 
> Solution:
> =
> Re-evaluate tag propagation only when relationship (edge) is force deleted or 
> for internal types.
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  717310daf 
> 
> 
> Diff: https://reviews.apache.org/r/72630/diff/1/
> 
> 
> Testing
> ---
> 
> Manually validated - term dissassociation removes all propagated 
> classifications
> 
> Precommit:
> ==
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1996/
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>



Re: Review Request 72630: ATLAS-3868: [Regression] removing a term-association doesn't remove classifications propagated from the term

2020-06-29 Thread Sarath Subramanian

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72630/
---

(Updated June 29, 2020, 10:52 p.m.)


Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
Nikhil Bonte, Nixon Rodrigues, Pinal Shah, and Sidharth Mishra.


Changes
---

moved logic to evaluate removeTagPropagation logic inside hard/soft delete 
handler.


Bugs: ATLAS-3868
https://issues.apache.org/jira/browse/ATLAS-3868


Repository: atlas


Description
---

Issue:
==
removing a term-association doesn't remove classifications propagated from the 
term

Cause:
==
Regression caused by https://issues.apache.org/jira/browse/ATLAS-3863

Solution:
=
Re-evaluate tag propagation only when relationship (edge) is force deleted or 
for internal types.


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/HardDeleteHandlerV1.java
 72dd632f4 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1.java
 59e7cf864 


Diff: https://reviews.apache.org/r/72630/diff/2/

Changes: https://reviews.apache.org/r/72630/diff/1-2/


Testing
---

Manually validated - term dissassociation removes all propagated classifications

Precommit:
==
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1996/


Thanks,

Sarath Subramanian



[jira] [Updated] (ATLAS-3868) [Regression] removing a term-association doesn't remove classifications propagated from the term

2020-06-29 Thread Sarath Subramanian (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3868:
--
Description: 
# Create a business term – *term1*
 # Associate classification *PII* to *term1*
 # Associate *term1* to an entity
 # Entity will now have PII classification propagated from the term
 # Now remove *term1* association with the entity
 # Entity still shows *PII* classification, propagated from the term => this is 
a regression

 

This regression is caused by ATLAS-3863

  was:
# Create a business term – *term1*
 # Associate classification *PII* to *term1*
 # Associate *term1* to an entity
 # Entity will now have PII classification propagated from the term
 # Now remove *term1* association with the entity
 # Entity still shows *PII* classification, propagated from the term => this is 
a regression

 

This regression is caused by ATLAS-3863


> [Regression] removing a term-association doesn't remove classifications 
> propagated from the term
> 
>
> Key: ATLAS-3868
> URL: https://issues.apache.org/jira/browse/ATLAS-3868
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
> Fix For: 2.1.0
>
>
> # Create a business term – *term1*
>  # Associate classification *PII* to *term1*
>  # Associate *term1* to an entity
>  # Entity will now have PII classification propagated from the term
>  # Now remove *term1* association with the entity
>  # Entity still shows *PII* classification, propagated from the term => this 
> is a regression
>  
> This regression is caused by ATLAS-3863



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (ATLAS-3868) [Regression] removing a term-association doesn't remove classifications propagated from the term

2020-06-29 Thread Sarath Subramanian (Jira)
Sarath Subramanian created ATLAS-3868:
-

 Summary: [Regression] removing a term-association doesn't remove 
classifications propagated from the term
 Key: ATLAS-3868
 URL: https://issues.apache.org/jira/browse/ATLAS-3868
 Project: Atlas
  Issue Type: Improvement
  Components:  atlas-core
Affects Versions: 2.0.0
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 2.1.0


# Create a business term – *term1*
 # Associate classification *PII* to *term1*
 # Associate *term1* to an entity
 # Entity will now have PII classification propagated from the term
 # Now remove *term1* association with the entity
 # Entity still shows *PII* classification, propagated from the term => this is 
a regression

 

This regression is caused by ATLAS-3863



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Review Request 72630: ATLAS-3868: [Regression] removing a term-association doesn't remove classifications propagated from the term

2020-06-29 Thread Sarath Subramanian

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72630/
---

Review request for atlas, Ashutosh Mestry, Jayendra Parab, Madhan Neethiraj, 
Nikhil Bonte, Nixon Rodrigues, Pinal Shah, and Sidharth Mishra.


Bugs: ATLAS-3868
https://issues.apache.org/jira/browse/ATLAS-3868


Repository: atlas


Description
---

Issue:
==
removing a term-association doesn't remove classifications propagated from the 
term

Cause:
==
Regression caused by https://issues.apache.org/jira/browse/ATLAS-3863

Solution:
=
Re-evaluate tag propagation only when relationship (edge) is force deleted or 
for internal types.


Diffs
-

  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 717310daf 


Diff: https://reviews.apache.org/r/72630/diff/1/


Testing
---

Manually validated - term dissassociation removes all propagated classifications

Precommit:
==
https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1996/


Thanks,

Sarath Subramanian



[jira] [Updated] (ATLAS-3868) [Regression] removing a term-association doesn't remove classifications propagated from the term

2020-06-29 Thread Sarath Subramanian (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3868:
--
Labels: glossary tagpropagation terms  (was: glossary tagpropagation)

> [Regression] removing a term-association doesn't remove classifications 
> propagated from the term
> 
>
> Key: ATLAS-3868
> URL: https://issues.apache.org/jira/browse/ATLAS-3868
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
>  Labels: glossary, tagpropagation, terms
> Fix For: 2.1.0
>
>
> # Create a business term – *term1*
>  # Associate classification *PII* to *term1*
>  # Associate *term1* to an entity
>  # Entity will now have PII classification propagated from the term
>  # Now remove *term1* association with the entity
>  # Entity still shows *PII* classification, propagated from the term => this 
> is a regression
>  
> This regression is caused by ATLAS-3863



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3868) [Regression] removing a term-association doesn't remove classifications propagated from the term

2020-06-29 Thread Sarath Subramanian (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3868:
--
Labels: glossary tagpropagation  (was: )

> [Regression] removing a term-association doesn't remove classifications 
> propagated from the term
> 
>
> Key: ATLAS-3868
> URL: https://issues.apache.org/jira/browse/ATLAS-3868
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>Priority: Major
>  Labels: glossary, tagpropagation
> Fix For: 2.1.0
>
>
> # Create a business term – *term1*
>  # Associate classification *PII* to *term1*
>  # Associate *term1* to an entity
>  # Entity will now have PII classification propagated from the term
>  # Now remove *term1* association with the entity
>  # Entity still shows *PII* classification, propagated from the term => this 
> is a regression
>  
> This regression is caused by ATLAS-3863



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (ATLAS-3866) Relationship search API for hive storage desc throws error code 500

2020-06-29 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148186#comment-17148186
 ] 

ASF subversion and git services commented on ATLAS-3866:


Commit 5ff3339fc03149e62bbdea9991e954539f262968 in atlas's branch 
refs/heads/branch-2.0 from Pinal Shah
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=5ff3339 ]

ATLAS-3866 : Relationship search API for hive storage desc throws error code 500

Signed-off-by: Sarath Subramanian 
(cherry picked from commit 0221e3dd297057dab344b8d9b6672a37adafabbc)


> Relationship search API for hive storage desc throws error code 500
> ---
>
> Key: ATLAS-3866
> URL: https://issues.apache.org/jira/browse/ATLAS-3866
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Pinal
>Assignee: Pinal
>Priority: Major
>  Labels: Relationships
> Fix For: 2.1.0, 3.0.0
>
>
> Tried below API of ATLAS
> {code:java}
> v2/search/relationship?limit=10=0=3e424245-0676-4d47-96a1-e17228b38367=__hive_table.sd=name=ASCENDING=true{code}
> error Response:
> {code:java}
> {
>   "errorCode": "ATLAS-500-00-001",
>   "errorMessage": "Internal server error Relationship search query failed"
> }
> {code}
>  
> *Cause of issue:* 
>  Above Request contains sortBy=name, 'name' attribute is not in 
> hive_storagedesc definition, hence the issue.
>  Also if sortBy is not passed, default attribute is 'name'



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (ATLAS-3866) Relationship search API for hive storage desc throws error code 500

2020-06-29 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148185#comment-17148185
 ] 

ASF subversion and git services commented on ATLAS-3866:


Commit 0221e3dd297057dab344b8d9b6672a37adafabbc in atlas's branch 
refs/heads/master from Pinal Shah
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=0221e3d ]

ATLAS-3866 : Relationship search API for hive storage desc throws error code 500

Signed-off-by: Sarath Subramanian 


> Relationship search API for hive storage desc throws error code 500
> ---
>
> Key: ATLAS-3866
> URL: https://issues.apache.org/jira/browse/ATLAS-3866
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Pinal
>Assignee: Pinal
>Priority: Major
>  Labels: Relationships
> Fix For: 2.1.0, 3.0.0
>
>
> Tried below API of ATLAS
> {code:java}
> v2/search/relationship?limit=10=0=3e424245-0676-4d47-96a1-e17228b38367=__hive_table.sd=name=ASCENDING=true{code}
> error Response:
> {code:java}
> {
>   "errorCode": "ATLAS-500-00-001",
>   "errorMessage": "Internal server error Relationship search query failed"
> }
> {code}
>  
> *Cause of issue:* 
>  Above Request contains sortBy=name, 'name' attribute is not in 
> hive_storagedesc definition, hence the issue.
>  Also if sortBy is not passed, default attribute is 'name'



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (ATLAS-3833) Packaging for atlas index repair tool

2020-06-29 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147568#comment-17147568
 ] 

ASF subversion and git services commented on ATLAS-3833:


Commit e691a32c2d3888c99757e6aa31694ec8d14c2d48 in atlas's branch 
refs/heads/master from chaitali borole
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=e691a32 ]

ATLAS-3833 : Packaging for atlas index repair tool

Signed-off-by: nixonrodrigues 


> Packaging for atlas index repair tool 
> --
>
> Key: ATLAS-3833
> URL: https://issues.apache.org/jira/browse/ATLAS-3833
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: chaitali borole
>Assignee: chaitali borole
>Priority: Major
>
> Packaging for atlas repair index was missing in Apache master build:
> Added descriptor in pom.xml for atlas-repair-index-package.xml
> Added readme
> Added atlas-repair-index-package.xml



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (ATLAS-3652) Quick Search: API requirement for GET request on multiple entity types

2020-06-29 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147579#comment-17147579
 ] 

ASF subversion and git services commented on ATLAS-3652:


Commit 8b5cb9d9aa9f977ba5dfc455dfe440dd55eace06 in atlas's branch 
refs/heads/master from Pinal Shah
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=8b5cb9d ]

ATLAS-3838: Support multiple tag/classification in basic/quick search API
ATLAS-3652: Quick Search: API requirement for GET request on multiple entity 
types

Signed-off-by: Sarath Subramanian 

both JIRA's addressed in the same commit.


> Quick Search: API requirement for GET request on multiple entity types
> --
>
> Key: ATLAS-3652
> URL: https://issues.apache.org/jira/browse/ATLAS-3652
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Pinal
>Assignee: Pinal
>Priority: Major
>  Labels: quicksearch
> Fix For: 2.1.0, 3.0.0
>
>
> Need an API that can be used to GET information of multiple selected entity 
> types. Currently it seems to support GET request for
>  # One type (typeName=someType)\{by specifying typeName in the call},
> OR 
>  # All types (typeName= )\{by leaving out typeName empty}.
>  
> Need an API that works for multiple selected types while also returning the 
> aggregationMetrics in the responseJSON as opposed to the POST call(where the 
> aggregationMetrics is left out as empty).
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (ATLAS-3838) Support multiple tag/classification in basic/quick search API

2020-06-29 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147578#comment-17147578
 ] 

ASF subversion and git services commented on ATLAS-3838:


Commit 8b5cb9d9aa9f977ba5dfc455dfe440dd55eace06 in atlas's branch 
refs/heads/master from Pinal Shah
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=8b5cb9d ]

ATLAS-3838: Support multiple tag/classification in basic/quick search API
ATLAS-3652: Quick Search: API requirement for GET request on multiple entity 
types

Signed-off-by: Sarath Subramanian 

both JIRA's addressed in the same commit.


> Support multiple tag/classification in basic/quick search API
> -
>
> Key: ATLAS-3838
> URL: https://issues.apache.org/jira/browse/ATLAS-3838
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Pinal
>Assignee: Pinal
>Priority: Major
>  Labels: BasicSearch
> Fix For: 2.1.0, 3.0.0
>
>
> it will allow user to search with multiple tags and the tag attribute filters 
> (attribute should System attributes or attribute which common for all the 
> mentioned tags)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (ATLAS-3652) Quick Search: API requirement for GET request on multiple entity types

2020-06-29 Thread Pinal (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147580#comment-17147580
 ] 

Pinal commented on ATLAS-3652:
--

This patch is included in ATLAS-3838

> Quick Search: API requirement for GET request on multiple entity types
> --
>
> Key: ATLAS-3652
> URL: https://issues.apache.org/jira/browse/ATLAS-3652
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Pinal
>Assignee: Pinal
>Priority: Major
>  Labels: quicksearch
> Fix For: 2.1.0, 3.0.0
>
>
> Need an API that can be used to GET information of multiple selected entity 
> types. Currently it seems to support GET request for
>  # One type (typeName=someType)\{by specifying typeName in the call},
> OR 
>  # All types (typeName= )\{by leaving out typeName empty}.
>  
> Need an API that works for multiple selected types while also returning the 
> aggregationMetrics in the responseJSON as opposed to the POST call(where the 
> aggregationMetrics is left out as empty).
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3838) Support multiple tag/classification in basic/quick search API

2020-06-29 Thread Sarath Subramanian (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-3838:
--
Fix Version/s: (was: 2.1.0)

> Support multiple tag/classification in basic/quick search API
> -
>
> Key: ATLAS-3838
> URL: https://issues.apache.org/jira/browse/ATLAS-3838
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Pinal
>Assignee: Pinal
>Priority: Major
>  Labels: BasicSearch
> Fix For: 3.0.0
>
>
> it will allow user to search with multiple tags and the tag attribute filters 
> (attribute should System attributes or attribute which common for all the 
> mentioned tags)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: Review Request 72625: ATLAS-3866 : Relationship search API for hive storage desc throws error code 500

2020-06-29 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72625/
---

(Updated June 29, 2020, 10:49 a.m.)


Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Changes
---

addressed comment


Bugs: ATLAS-3866
https://issues.apache.org/jira/browse/ATLAS-3866


Repository: atlas


Description
---

**Issue:** In relationship api v2/relationship, for hive_storagedesc it throws 
exception

**Reason:** it is because, sort attribute is assigned 'name' as default, and 
'name' attribute is not in the defination of hive_storagedesc.

**Workaround:** Validate if attribute is present in relationship end def, if 
not ignore sorting


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
dd4d1b441 


Diff: https://reviews.apache.org/r/72625/diff/2/

Changes: https://reviews.apache.org/r/72625/diff/1-2/


Testing (updated)
---

Manually tested
Precommit : 
https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1992/console


Thanks,

Pinal Shah



Re: Seeking advice on Atlas XSS vulnerabilities

2020-06-29 Thread Madhan Neethiraj
Justin,

Thank you for the input and details. I think there is either a disconnect in 
what is meant by 'server-side' and UI; or an assumption that the REST API 
request/response is only for use in HTML browser. I strongly believe that an 
API implementation can't be mandated to accommodate to needs of a specific 
rendering method/library. Different rendering methods might have different 
escaping requirements, hence must be handled by modules specific to the chosen 
rendering method.

If you have specific ideas to enhance Atlas handling of HTML escapes, I suggest 
to provide necessary updates in a patch. This will benefit you and the 
community.

Thanks,
Madhan

On 6/18/20, 5:47 PM, "Justin Kuang"  wrote:

Hi,

Offering my two cents. I'm a security engineer at Snap working with Melinda
to review her service which uses Atlas. While the XSS found here can be
hotfixed, it doesn't provide confidence that it won't occur elsewhere
without a broader solution.

For fixes, I don't think it's unreasonable to filter user inputs
server-side, especially when we know they're rendered as HTML.
Alternatively, on the frontend it would be better if we could sanitize at a
central point, rather than trying to escape fields as they're used. If
there isn't a central point, then somewhere closer to where JSON responses
are fetched and serialized would also work.

The library in its current state would fail to get internal approval for
production usage. XSS like this has been addressed in other frameworks for
years so there's little reason for a modern webapp to still have them. If
these features aren't fit for the public release we'll likely find a
different solution or try to add them internally, but they might be less
elegant than if they were developed by the maintainers.

Justin

On Mon, Jun 15, 2020 at 3:34 PM Melinda Crane  wrote:

>
>
> -- Forwarded message -
> From: Madhan Neethiraj 
> Date: Mon, Jun 15, 2020 at 6:12 PM
> Subject: Re: Seeking advice on Atlas XSS vulnerabilities
> To: dev@atlas.apache.org , Bolke de Bruin <
> bdbr...@gmail.com>, Melinda Crane 
>
>
> Bolke, Melinda,
>
> I think it will help to clarify what I meant by "server-side" and "UI".
>  - server-side: Atlas REST APIs to perform various operations on
> types/entities/glossaries/..
>  - UI: a web-application running in a browser which calls Atlas REST APIs
> to fetch data, render and update data
>
> I completely agree on all UI modules to handle HTML escaping appropriately
> - irrespective of whether they run in browser or server side. Few
> references to 'server side' in the documents referred in this thread seem
> to be about servlets/JSP - which are not used in Apache Atlas.
>
> If you are suggesting all Atlas REST API responses to escape HTML - I
> think it is a very bad idea. BTW, none of the documents referred in this
> thread ask the server REST APIs to perform any escaping for HTML.
>
> Regards,
> Madhan
>
> On 6/15/20, 1:48 PM, "Bolke de Bruin"  wrote:
>
> Hi Keval
>
> I'm happy that you will tackle the UI part of it. However, these are
> not mere UI issues and need to be tackled server side IMHO. This opinion 
is
> supported by what the experts say[1]
>
> Can you elaborate on how you will approach this? I have not seen a
> discussion.
>
> Cheers
> Bolke
>
> [1]
> 
https://urldefense.proofpoint.com/v2/url?u=https-3A__security.stackexchange.com_questions_19524_current-2Dbest-2Dpractices-2Dto-2Dprevent-2Dpersistent-2Dxss-2Dattacks_19536=DwIFaQ=ncDTmphkJTvjIDPh0hpF_w=IkBo18xErS3TGsJ4f2MH4HIFKDcWTAyKWqDieefibV4=W6dQ86TlCtMFcgDcupU4BuWnH1Oh2gxXD7_Y4AUuztA=8OVEzsh4MotsUL__aDfRl_bkm6THou4bBPMP2hDYhOM=
>
>
> Sent from my iPhone
>
> > On 15 Jun 2020, at 19:47, Keval Bhatt  wrote:
> >
> > Hi Bolke,
> >
> > Thanks for the details on the UI issues in rendering entity name and
> > incorrect value storied for user-defined attributes. I am working on
> > updating UI to address these issues, will send a patch shortly.
> >
> > If you see further issues that require a server-side update, can you
> please
> > add specific use case details?
> >
> > Thanks
> >
> >> On Fri, Jun 12, 2020 at 2:15 PM Bolke de Bruin 
> wrote:
> >>
> >> Hi Madhan,
> >>
> >> A second order XSS (I.e. stored) can be executed against Atlas 
quite
> >> easily. If you create an entity with a name like
> >>
> >> “>
> >>
> >> You will get an alert when clicking “edit” and you will get
> artifacts all
> >> over the place stemming from the closing ‘“>’. And there are
> 

Re: Review Request 72625: ATLAS-3866 : Relationship search API for hive storage desc throws error code 500

2020-06-29 Thread Sarath Subramanian

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72625/#review221085
---




repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
Lines 607 (patched)


Do we really need to compare edge end points?

can we do:

once you get relationshipEdge, you can get the typeName of the other vertex 
(not entityVertex - comparing vertexId). 

From this typeName - for e.g. 'hive_storagedesc' you can get entityType

relationshipEndEntityType = 
typeRegistry.getEntityTypeByName(otherEntityTypeName);


- Sarath Subramanian


On June 28, 2020, 10:06 p.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72625/
> ---
> 
> (Updated June 28, 2020, 10:06 p.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3866
> https://issues.apache.org/jira/browse/ATLAS-3866
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Issue:** In relationship api v2/relationship, for hive_storagedesc it 
> throws exception
> 
> **Reason:** it is because, sort attribute is assigned 'name' as default, and 
> 'name' attribute is not in the defination of hive_storagedesc.
> 
> **Workaround:** Validate if attribute is present in relationship end def, if 
> not ignore sorting
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  dd4d1b441 
> 
> 
> Diff: https://reviews.apache.org/r/72625/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested
> Precommit :
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



Re: Review Request 72625: ATLAS-3866 : Relationship search API for hive storage desc throws error code 500

2020-06-29 Thread Pinal Shah


> On June 29, 2020, 6:29 a.m., Sarath Subramanian wrote:
> > repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
> > Lines 607 (patched)
> > 
> >
> > Do we really need to compare edge end points?
> > 
> > can we do:
> > 
> > once you get relationshipEdge, you can get the typeName of the other 
> > vertex (not entityVertex - comparing vertexId). 
> > 
> > From this typeName - for e.g. 'hive_storagedesc' you can get entityType
> > 
> > relationshipEndEntityType = 
> > typeRegistry.getEntityTypeByName(otherEntityTypeName);

Thanks Sarath,
Yes we can do the way you suggested


- Pinal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72625/#review221085
---


On June 29, 2020, 10:49 a.m., Pinal Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72625/
> ---
> 
> (Updated June 29, 2020, 10:49 a.m.)
> 
> 
> Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
> and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3866
> https://issues.apache.org/jira/browse/ATLAS-3866
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Issue:** In relationship api v2/relationship, for hive_storagedesc it 
> throws exception
> 
> **Reason:** it is because, sort attribute is assigned 'name' as default, and 
> 'name' attribute is not in the defination of hive_storagedesc.
> 
> **Workaround:** Validate if attribute is present in relationship end def, if 
> not ignore sorting
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  dd4d1b441 
> 
> 
> Diff: https://reviews.apache.org/r/72625/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested
> Precommit : 
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1992/console
> 
> 
> Thanks,
> 
> Pinal Shah
> 
>



[jira] [Updated] (ATLAS-3867) Relationship search API should have a provision to fetch custom attributes in search results

2020-06-29 Thread Pinal (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pinal updated ATLAS-3867:
-
Issue Type: Improvement  (was: Bug)

> Relationship search API should have a provision to fetch custom attributes in 
> search results
> 
>
> Key: ATLAS-3867
> URL: https://issues.apache.org/jira/browse/ATLAS-3867
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core, atlas-intg
>Affects Versions: 2.0.0
>Reporter: Pinal
>Assignee: Pinal
>Priority: Major
>  Labels: Relationships
>
> {code:java}
> /v2/search/relationship?guid=ac9e04cc-f927-4334-af08-c83bc3733f5b=__hive_table.columns=name=ASCENDING
> {code}
> The response of the  above API contains below attributes only:
> {code:java}
> "attributes": {
> "owner": "hive",
> "qualifiedName": "ho.us_customers.age@cm",
> "name": "age"
>   },{code}
>  Is it possible to fetch a custom attribute "*dcProfiledData*" present in 
> Technical properties of the entity detailed page.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (ATLAS-3724) Qualified name pattern for Column cause ambiguity in Quickstart data.

2020-06-29 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/ATLAS-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147729#comment-17147729
 ] 

ASF subversion and git services commented on ATLAS-3724:


Commit 9d597f249bc222d362dd96921ed3254579e8fc1b in atlas's branch 
refs/heads/master from chaitali borole
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=9d597f2 ]

ATLAS-3724 : Update Qualified name pattern for Column since it cause ambiguity 
in Quickstart data.

Signed-off-by: nixonrodrigues 


> Qualified name pattern for Column cause ambiguity in Quickstart data.
> -
>
> Key: ATLAS-3724
> URL: https://issues.apache.org/jira/browse/ATLAS-3724
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 2.0.0, 2.1.0, 3.0.0
>Reporter: chaitali borole
>Assignee: chaitali borole
>Priority: Major
> Fix For: 2.0.0, 2.1.0
>
> Attachments: ATLAS-3724.patch
>
>
> 1) Create Entity of type Column name = app_id, qualified name = app_id@cl1
> assign Table - logging_fact_monthly_mv
> 2) Again Create same entity with same qualified name
> assign different Table - log_fact_daily_mv
> 3) Check Column Relationship, Only one Table relationship is seen
> 4) Check Table (logging_fact_monthly_mv,log_fact_daily_mv) In Both 
> Relationship, Column app_id is seen



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3652) Quick Search: API requirement for GET request on multiple entity types

2020-06-29 Thread Pinal (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pinal updated ATLAS-3652:
-
Description: 
Need an API that can be used to GET information of multiple selected entity 
types. Currently it seems to support GET request for
 # One type (typeName=someType)\{by specifying typeName in the call},
 OR 
 # All types (typeName= )\{by leaving out typeName empty}.
  
 Need an API that works for multiple selected types while also returning the 
aggregationMetrics in the responseJSON as opposed to the POST call(where the 
aggregationMetrics is left out as empty).

 

*UseCase:* Multiple Entity types with its attribute filters

*How And Validations:*

1. Search Results can be filtered with multiple entity types by 'comma' 
seperated string of entity typeName in the request
Eg. "typeName": "hive_table,hive_db".

2. For the attribute filters, attribute in the criteria should be System 
attribute like "created by user, last modified time" or the attribute should 
present in both the typeDefs of the mentioned entity 'hive_table' and 
'hive_db'. (Common attributes like name).

  was:
Need an API that can be used to GET information of multiple selected entity 
types. Currently it seems to support GET request for
 # One type (typeName=someType)\{by specifying typeName in the call},
OR 
 # All types (typeName= )\{by leaving out typeName empty}.
 
Need an API that works for multiple selected types while also returning the 
aggregationMetrics in the responseJSON as opposed to the POST call(where the 
aggregationMetrics is left out as empty).
 


> Quick Search: API requirement for GET request on multiple entity types
> --
>
> Key: ATLAS-3652
> URL: https://issues.apache.org/jira/browse/ATLAS-3652
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 2.0.0
>Reporter: Pinal
>Assignee: Pinal
>Priority: Major
>  Labels: quicksearch
> Fix For: 3.0.0
>
>
> Need an API that can be used to GET information of multiple selected entity 
> types. Currently it seems to support GET request for
>  # One type (typeName=someType)\{by specifying typeName in the call},
>  OR 
>  # All types (typeName= )\{by leaving out typeName empty}.
>   
>  Need an API that works for multiple selected types while also returning the 
> aggregationMetrics in the responseJSON as opposed to the POST call(where the 
> aggregationMetrics is left out as empty).
>  
> *UseCase:* Multiple Entity types with its attribute filters
> *How And Validations:*
> 1. Search Results can be filtered with multiple entity types by 'comma' 
> seperated string of entity typeName in the request
> Eg. "typeName": "hive_table,hive_db".
> 2. For the attribute filters, attribute in the criteria should be System 
> attribute like "created by user, last modified time" or the attribute should 
> present in both the typeDefs of the mentioned entity 'hive_table' and 
> 'hive_db'. (Common attributes like name).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (ATLAS-3867) Relationship search API should have a provision to fetch custom attributes in search results

2020-06-29 Thread Pinal (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pinal updated ATLAS-3867:
-
Summary: Relationship search API should have a provision to fetch custom 
attributes in search results  (was: Relationship search API should have a 
provision to fetch all the attributes)

> Relationship search API should have a provision to fetch custom attributes in 
> search results
> 
>
> Key: ATLAS-3867
> URL: https://issues.apache.org/jira/browse/ATLAS-3867
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core, atlas-intg
>Affects Versions: 2.0.0
>Reporter: Pinal
>Assignee: Pinal
>Priority: Major
>  Labels: Relationships
>
> {code:java}
> /v2/search/relationship?guid=ac9e04cc-f927-4334-af08-c83bc3733f5b=__hive_table.columns=name=ASCENDING
> {code}
> The response of the  above API contains below attributes only:
> {code:java}
> "attributes": {
> "owner": "hive",
> "qualifiedName": "ho.us_customers.age@cm",
> "name": "age"
>   },{code}
>  Is it possible to fetch a custom attribute "*dcProfiledData*" present in 
> Technical properties of the entity detailed page.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (ATLAS-3867) Relationship search API should have a provision to fetch all the attributes

2020-06-29 Thread Pinal (Jira)
Pinal created ATLAS-3867:


 Summary: Relationship search API should have a provision to fetch 
all the attributes
 Key: ATLAS-3867
 URL: https://issues.apache.org/jira/browse/ATLAS-3867
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core, atlas-intg
Affects Versions: 2.0.0
Reporter: Pinal
Assignee: Pinal


{code:java}
/v2/search/relationship?guid=ac9e04cc-f927-4334-af08-c83bc3733f5b=__hive_table.columns=name=ASCENDING
{code}
The response of the  above API contains below attributes only:
{code:java}
"attributes": {
"owner": "hive",
"qualifiedName": "ho.us_customers.age@cm",
"name": "age"
  },{code}
 Is it possible to fetch a custom attribute "*dcProfiledData*" present in 
Technical properties of the entity detailed page.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Announcing ApacheCon @Home 2020

2020-06-29 Thread Rich Bowen

Hi, Apache enthusiast!

(You’re receiving this because you’re subscribed to one or more dev or 
user mailing lists for an Apache Software Foundation project.)


The ApacheCon Planners and the Apache Software Foundation are pleased to 
announce that ApacheCon @Home will be held online, September 29th 
through October 1st, 2020. We’ll be featuring content from dozens of our 
projects, as well as content about community, how Apache works, business 
models around Apache software, the legal aspects of open source, and 
many other topics.


Full details about the event, and registration, is available at 
https://apachecon.com/acah2020


Due to the confusion around how and where this event was going to be 
held, and in order to open up to presenters from around the world who 
may previously have been unable or unwilling to travel, we’ve reopened 
the Call For Presentations until July 13th. Submit your talks today at 
https://acna2020.jamhosted.net/


We hope to see you at the event!
Rich Bowen, VP Conferences, The Apache Software Foundation


[jira] [Updated] (ATLAS-3838) Support multiple tag/classification in basic/quick search API

2020-06-29 Thread Pinal (Jira)


 [ 
https://issues.apache.org/jira/browse/ATLAS-3838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pinal updated ATLAS-3838:
-
Description: 
it will allow user to search with multiple tags and the tag attribute filters 
(attribute should System attributes or attribute which common for all the 
mentioned tags)

 

*Use Case**: Multiple tags with its attribute filters*


*How And Validations:*
 # Search Results can be filtered with multiple tag by 'comma' seperated string 
of tags in the request
Eg. "classification": "tag1,tag2".
 # For the attribute filters, attribute in the criteria should be System 
attribute like "created by user, last modified time" or the attribute should 
present in both the typeDefs of the mentioned classification 'tag1' and 'tag2'. 
(Common attributes like name).
 # Curently we support Classifications like _ALL_CLASSIFICATION_TYPES, 
_NOT_CLASSIFIED So if the request have any builtin classifications with the 
normal tags, preference will be given to it in the below order
_NOT_CLASSIFIED > _ALL_CLASSIFICATION_TYPES = _CLASSIFIED = 
WILDCARD_CLASSIFICATIONS(*) > tag1 = tag2
Eg. "classification" : "tag1,_CLASSIFIED" , search results will be according to 
_CLASSIFIED
 # Classification supports wildcard search too Eg. tag*, So if the request has 
wildcard tag , tag filters will not apply to it.

  was:it will allow user to search with multiple tags and the tag attribute 
filters (attribute should System attributes or attribute which common for all 
the mentioned tags)


> Support multiple tag/classification in basic/quick search API
> -
>
> Key: ATLAS-3838
> URL: https://issues.apache.org/jira/browse/ATLAS-3838
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Pinal
>Assignee: Pinal
>Priority: Major
>  Labels: BasicSearch
> Fix For: 3.0.0
>
>
> it will allow user to search with multiple tags and the tag attribute filters 
> (attribute should System attributes or attribute which common for all the 
> mentioned tags)
>  
> *Use Case**: Multiple tags with its attribute filters*
> *How And Validations:*
>  # Search Results can be filtered with multiple tag by 'comma' seperated 
> string of tags in the request
> Eg. "classification": "tag1,tag2".
>  # For the attribute filters, attribute in the criteria should be System 
> attribute like "created by user, last modified time" or the attribute should 
> present in both the typeDefs of the mentioned classification 'tag1' and 
> 'tag2'. (Common attributes like name).
>  # Curently we support Classifications like _ALL_CLASSIFICATION_TYPES, 
> _NOT_CLASSIFIED So if the request have any builtin classifications with the 
> normal tags, preference will be given to it in the below order
> _NOT_CLASSIFIED > _ALL_CLASSIFICATION_TYPES = _CLASSIFIED = 
> WILDCARD_CLASSIFICATIONS(*) > tag1 = tag2
> Eg. "classification" : "tag1,_CLASSIFIED" , search results will be according 
> to _CLASSIFIED
>  # Classification supports wildcard search too Eg. tag*, So if the request 
> has wildcard tag , tag filters will not apply to it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Review Request 72627: ATLAS-3867 : Relationship search API should have a provision to fetch custom attributes in search results

2020-06-29 Thread Pinal Shah

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72627/
---

Review request for atlas, Jayendra Parab, Madhan Neethiraj, Nixon Rodrigues, 
and Sarath Subramanian.


Bugs: ATLAS-3867
https://issues.apache.org/jira/browse/ATLAS-3867


Repository: atlas


Description
---

**Issue:** Attributes in search result of Relationship api are minimal.

**WorkAround:** Relationship api request should have provision to specify 
attributes to be present in search result.

**Example Request:** 
/v2/search/relationship?guid=ac9e04cc-f927-4334-af08-c83bc3733f5b=columns=name=ASCENDING=dcProfiledData


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java 
e64c31522 
  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
b2737af01 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java 076284ec4 


Diff: https://reviews.apache.org/r/72627/diff/1/


Testing
---

Manually tested
Precommit :


Thanks,

Pinal Shah



Re: Review Request 72335: ATLAS-3724 : Qualified name pattern for Column cause ambiguity in Quickstart data.

2020-06-29 Thread Nixon Rodrigues

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72335/#review221089
---


Ship it!




Ship It!

- Nixon Rodrigues


On April 13, 2020, 7:24 a.m., chaitali wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72335/
> ---
> 
> (Updated April 13, 2020, 7:24 a.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nixon Rodrigues, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-3724
> https://issues.apache.org/jira/browse/ATLAS-3724
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> When we create entity of type Column (app_id) first with Table 
> log_fact_daily_mv and then same entity with Table logging_fact_monthly_mv 
> using same qualified name both the times.Column app_id relationship reflects 
> only recent assigned Table.
> 
> This patch sets the qualified name format db.table.column@cluster which 
> breaks the ambiguity.
> 
> 
> Diffs
> -
> 
>   webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java b5d2a47ec 
> 
> 
> Diff: https://reviews.apache.org/r/72335/diff/4/
> 
> 
> Testing
> ---
> 
> Pre-commit: 
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/1788/console
> 
> 
> Thanks,
> 
> chaitali
> 
>