Review Request 61100: Hooks to include atlas-intg jar

2017-07-24 Thread Apoorv Naik

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61100/
---

Review request for atlas.


Bugs: ATLAS-1989
https://issues.apache.org/jira/browse/ATLAS-1989


Repository: atlas


Description
---

Atlas Hooks should include the atlas-intg jar for the model classes.


Diffs
-

  addons/falcon-bridge/pom.xml bc71925b 
  addons/hive-bridge/pom.xml 64ffe048 
  addons/sqoop-bridge/pom.xml 33c5089b 
  addons/storm-bridge/pom.xml a4e4d0f9 


Diff: https://reviews.apache.org/r/61100/diff/1/


Testing
---

mvn clean package -DskipTests -Pdist copies the atlas-intg*.jar to the 
dependency folder under target


Thanks,

Apoorv Naik



[jira] [Updated] (ATLAS-1989) Atlas Hooks should package atlas-intg jar

2017-07-24 Thread Apoorv Naik (JIRA)

 [ 
https://issues.apache.org/jira/browse/ATLAS-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apoorv Naik updated ATLAS-1989:
---
Attachment: 0001-ATLAS-1989-Hooks-to-include-atlas-model-classes-atla.patch

> Atlas Hooks should package atlas-intg jar
> -
>
> Key: ATLAS-1989
> URL: https://issues.apache.org/jira/browse/ATLAS-1989
> Project: Atlas
>  Issue Type: Bug
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
> Fix For: 0.8-incubating, 0.9-incubating, 0.8.1-incubating
>
> Attachments: 
> 0001-ATLAS-1989-Hooks-to-include-atlas-model-classes-atla.patch
>
>
> The Hooks should include the atlas-intg jar so that the atlas client and the 
> message producers can use the V2 atlas models



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (ATLAS-1989) Atlas Hooks should package atlas-intg jar

2017-07-24 Thread Apoorv Naik (JIRA)
Apoorv Naik created ATLAS-1989:
--

 Summary: Atlas Hooks should package atlas-intg jar
 Key: ATLAS-1989
 URL: https://issues.apache.org/jira/browse/ATLAS-1989
 Project: Atlas
  Issue Type: Bug
Reporter: Apoorv Naik
Assignee: Apoorv Naik


The Hooks should include the atlas-intg jar so that the atlas client and the 
message producers can use the V2 atlas models



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Review Request 61088: [ATLAS-1988]: Implement REST API to search for related entities

2017-07-24 Thread Sarath Subramanian

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61088/
---

Review request for atlas, keval bhatt and Madhan Neethiraj.


Bugs: ATLAS-1988
https://issues.apache.org/jira/browse/ATLAS-1988


Repository: atlas


Description
---

Add a new endpoint (v2/search/relationship) in DiscoveryREST to search for 
related entities.
Parameters:
guid
attributeLabel
sortBy
'limit
offset


Diffs
-

  intg/src/main/java/org/apache/atlas/model/discovery/AtlasSearchResult.java 
58274403 
  
repository/src/main/java/org/apache/atlas/discovery/AtlasDiscoveryService.java 
764b548f 
  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
b183c72a 
  
repository/src/main/java/org/apache/atlas/util/AtlasGremlin2QueryProvider.java 
1bf03464 
  repository/src/main/java/org/apache/atlas/util/AtlasGremlinQueryProvider.java 
8481a4f6 
  webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java efab72a3 


Diff: https://reviews.apache.org/r/61088/diff/1/


Testing
---


Thanks,

Sarath Subramanian



[jira] [Updated] (ATLAS-1988) Implement REST API to search for related entities

2017-07-24 Thread Sarath Subramanian (JIRA)

 [ 
https://issues.apache.org/jira/browse/ATLAS-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-1988:
--
Description: 
Add a new endpoint in DiscoveryREST to search for related entities.

*v2/search/relationship
** guid
* attributeLabel
* sortBy
* 'limit
* offset

> Implement REST API to search for related entities 
> --
>
> Key: ATLAS-1988
> URL: https://issues.apache.org/jira/browse/ATLAS-1988
> Project: Atlas
>  Issue Type: Sub-task
>  Components:  atlas-core, atlas-intg
>Affects Versions: 0.8.1-incubating
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
> Fix For: 0.8.1-incubating
>
>
> Add a new endpoint in DiscoveryREST to search for related entities.
> *v2/search/relationship
> ** guid
> * attributeLabel
> * sortBy
> * 'limit
> * offset



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (ATLAS-1988) Implement REST API to search for related entities

2017-07-24 Thread Sarath Subramanian (JIRA)

 [ 
https://issues.apache.org/jira/browse/ATLAS-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-1988:
--
Description: 
Add a new endpoint (v2/search/relationship) in DiscoveryREST to search for 
related entities.

Parameters:
* guid
* attributeLabel
* sortBy
* 'limit
* offset

  was:
Add a new endpoint in DiscoveryREST to search for related entities.

*v2/search/relationship
** guid
* attributeLabel
* sortBy
* 'limit
* offset


> Implement REST API to search for related entities 
> --
>
> Key: ATLAS-1988
> URL: https://issues.apache.org/jira/browse/ATLAS-1988
> Project: Atlas
>  Issue Type: Sub-task
>  Components:  atlas-core, atlas-intg
>Affects Versions: 0.8.1-incubating
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
> Fix For: 0.8.1-incubating
>
>
> Add a new endpoint (v2/search/relationship) in DiscoveryREST to search for 
> related entities.
> Parameters:
> * guid
> * attributeLabel
> * sortBy
> * 'limit
> * offset



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (ATLAS-1988) Implement REST API to search for related entities

2017-07-24 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-1988:
-

 Summary: Implement REST API to search for related entities 
 Key: ATLAS-1988
 URL: https://issues.apache.org/jira/browse/ATLAS-1988
 Project: Atlas
  Issue Type: Sub-task
  Components:  atlas-core, atlas-intg
Affects Versions: 0.8.1-incubating
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian
 Fix For: 0.8.1-incubating






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Re: Review Request 61085: ATLAS-1983: Add relationship cardinality validation

2017-07-24 Thread Madhan Neethiraj

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61085/#review181253
---




repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
Lines 128 (patched)


consider using vertex.getEdges(direction, edgeLabel), instead of iterating 
through edges to find the one with the expected label.


- Madhan Neethiraj


On July 24, 2017, 5:18 p.m., David Radley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61085/
> ---
> 
> (Updated July 24, 2017, 5:18 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-1983: Add relationship cardinality validation
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
> b24f99f6f9337aa10f40e9f10024fe5a345c3540 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipDef.java 
> c17e875d10169753b76fcdb483e2ca85195104b2 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
>  49e08a070a803a36253b502666947ab92ffe39d3 
> 
> 
> Diff: https://reviews.apache.org/r/61085/diff/1/
> 
> 
> Testing
> ---
> 
> ran Junits 
> Successfully added one relationship, then added a second which resulted in 
> the error.
> 
> 
> Thanks,
> 
> David Radley
> 
>



[jira] [Commented] (ATLAS-1986) Rename relationshipAttributes in Entity to relationships.

2017-07-24 Thread Sarath Subramanian (JIRA)

[ 
https://issues.apache.org/jira/browse/ATLAS-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098905#comment-16098905
 ] 

Sarath Subramanian commented on ATLAS-1986:
---

Yes, once the change is committed to the right branch we include the commit id 
to the JIRA and mark it resolved.

> Rename relationshipAttributes in Entity to relationships. 
> --
>
> Key: ATLAS-1986
> URL: https://issues.apache.org/jira/browse/ATLAS-1986
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: ATLAS-1986.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (ATLAS-1986) Rename relationshipAttributes in Entity to relationships.

2017-07-24 Thread David Radley (JIRA)

[ 
https://issues.apache.org/jira/browse/ATLAS-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098880#comment-16098880
 ] 

David Radley commented on ATLAS-1986:
-

Hi Sarath,
Sorry about that - I am still quite new to this - when should I resolve 
the Jira - after it has been committed? 
 all the best, David. 



From:   "Sarath Subramanian (JIRA)" 
To: david_rad...@uk.ibm.com
Date:   24/07/2017 17:14
Subject:[jira] [Reopened] (ATLAS-1986) Rename 
relationshipAttributes in Entity to relationships.




 [ 
https://issues.apache.org/jira/browse/ATLAS-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 
]

Sarath Subramanian reopened ATLAS-1986:
---

This change is not committed yet, moving to patch available state





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


> Rename relationshipAttributes in Entity to relationships. 
> --
>
> Key: ATLAS-1986
> URL: https://issues.apache.org/jira/browse/ATLAS-1986
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: ATLAS-1986.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Review Request 61085: ATLAS-1983: Add relationship cardinality validation

2017-07-24 Thread David Radley

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61085/
---

Review request for atlas, Madhan Neethiraj and Sarath Subramanian.


Repository: atlas


Description
---

ATLAS-1983: Add relationship cardinality validation


Diffs
-

  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 
b24f99f6f9337aa10f40e9f10024fe5a345c3540 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipDef.java 
c17e875d10169753b76fcdb483e2ca85195104b2 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
 49e08a070a803a36253b502666947ab92ffe39d3 


Diff: https://reviews.apache.org/r/61085/diff/1/


Testing
---

ran Junits 
Successfully added one relationship, then added a second which resulted in the 
error.


Thanks,

David Radley



[jira] [Reopened] (ATLAS-1986) Rename relationshipAttributes in Entity to relationships.

2017-07-24 Thread Sarath Subramanian (JIRA)

 [ 
https://issues.apache.org/jira/browse/ATLAS-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian reopened ATLAS-1986:
---

This change is not committed yet, moving to patch available state

> Rename relationshipAttributes in Entity to relationships. 
> --
>
> Key: ATLAS-1986
> URL: https://issues.apache.org/jira/browse/ATLAS-1986
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: ATLAS-1986.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Re: Relationship attributes

2017-07-24 Thread Madhan Neethiraj
As I said earlier, I prefer “relatedEntities” – as this name states that the 
values in this attributes are references to entities. I think “relationships” 
is good as well. However, I would prefer to use “AtlasRelatedObjectId” instead 
of “AtlasRelationshipObjectId” – as “RelationshipObjectId” might be confused 
with an instance of a relationship. Also, I think we should have 
“AtlasRelatedObjectId” extend “AtlasObjectId” (instead of embedding as an 
attribute).

 

Thanks,

Madhan

 

 

From: David Radley 
Date: Monday, July 24, 2017 at 3:39 AM
To: "dev@atlas.apache.org" 
Cc: Madhan Neethiraj , Sarath Subramanian 
, Graham Wallis 
Subject: Re: Relationship attributes

 

Hi all, 
I have just had a chat with Graham. we are thinking that we should go with 
relationships as the top level name. We also think we could helpfully add in 
the related entity in the new class like this: 

class AtlasRelationshipObjectId  { 
   AtlasObjectId relatedEntity; 
  String relationshipGuid;
  AtlasStruct relationshipAttributes; 
}

I think this gives us the best of both worlds, 
   regards David. 





From:Graham Wallis  
To:dev@atlas.apache.org 
Cc:Madhan Neethiraj , Sarath Subramanian 
 
Date:24/07/2017 09:58 
Subject:Re: Relationship attributes 




Personally I think 'relatedEntities' is clearer.

Best regards,
 Graham

Graham Wallis
IBM Analytics Emerging Technology Center
Internet: graham_wal...@uk.ibm.com 
IBM Laboratories, Hursley Park, Hursley, Hampshire SO21 2JN
Tel: +44-1962-815356Tie: 7-245356




From:   Madhan Neethiraj 
To: "dev@atlas.apache.org" , Sarath Subramanian 

Date:   24/07/2017 09:04
Subject:Re: Relationship attributes
Sent by:Madhan Neethiraj 



Current name of ‘relationshipAttributes’ makes sense looking from an 
entity point-of-view – it distinguishes regular-attributes of an entity 
from attributes injected by relationships. However, given that 
relationships can themselves might have attributes, it can be confusing.

I was going to suggest ‘relatedEntities’; but ‘relationships’ seems to be 
good choice.

+1 for naming the field as ‘relationships’.

Thanks,
Madhan


On 7/24/17, 12:53 AM, "David Radley"  wrote:

   Hi Sarath,
   Great, personally for the Entity's current relationshipAttributes, I 
   prefer relationships as it is simpler - is there a reason you need 
   attribute in the name?
all the best, David. 



   From:   Sarath Subramanian 
   To: dev@atlas.apache.org
   Cc: Madhan Neethiraj 
   Date:   24/07/2017 07:09
   Subject:Re: Relationship attributes



   Hi David,

   I agree with using the term 'relationship attributes' for attributes 
of
   relationship, I suggest we use "relatedAttributes" for relationship
   attributes of entity.


   Thanks,
   Sarath Subramanian

   On Sun, Jul 23, 2017 at 2:22 AM, David Radley 

   wrote:

   > Hi Madhan,
   > When I see the phrase 'relationship attributes', I am never quite 
sure
   > whether we are referring to the attributes of an entity that relate 
to
   > another entity or the attributes of the relationship instance 
itself. I
   > think the phrase ' relationship attributes' more naturally fits as 
the
   > attributes of the relationship itself; we are using it in the other 
   sense.
   >
   > I suggest we change the relationshipAttributes in the entity to
   > relationships (if you really want attributes in the name we could 
call 
   it
   > relatingAttributes)- and use the term 'relationship attributes' 
purely 
   for
   > the attributes of the relationship itself. What do you think?
   > all the best, David.
   > Unless stated otherwise above:
   > IBM United Kingdom Limited - Registered in England and Wales with 
number
   > 741598.
   > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire 
PO6 
   3AU
   >



   Unless stated otherwise above:
   IBM United Kingdom Limited - Registered in England and Wales with 
number 
   741598. 
   Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 
3AU





Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU




Re: Audit Repository: Hbase, No-op ....

2017-07-24 Thread Mandy Chessell
Hello Nigel,
This makes a lot of sense to me. 

Does the Audit log interface assume a transactional backing store -so that 
data updates and the audit log entry are committed in the same 
transaction?   If not then we might also want to consider a simple file 
based audit log as the entry level service.

I am assuming each of the audit log implementations implement the same 
Java interface?  So it also makes sense to move this interface - and the 
audit log management to the GAF. 

I added a getAuditLog method to the connector interface to allow an 
application to create an audit trail for their use of a connector - this 
could also be used by the gaf plugins.  I was also assuming that this was 
something we need to synch up with between the GAF and the OCF since the 
OCF should really get its audit logs from the GAF.

All the best
Mandy
___
Mandy Chessell CBE FREng CEng FBCS
IBM Distinguished Engineer

Master Inventor
Member of the IBM Academy of Technology
Visiting Professor, Department of Computer Science, University of 
Sheffield

Email: mandy_chess...@uk.ibm.com
LinkedIn: http://www.linkedin.com/pub/mandy-chessell/22/897/a49

Assistant: Janet Brooks - jsbrook...@uk.ibm.com



From:   Nigel Jones 
To: d...@atlas.incubator.apache.org
Date:   24/07/2017 13:40
Subject:Audit Repository: Hbase, No-op 



I was just reading ATLAS-1870 which related to default audit 
repositories. I see the default is HBaseBasedAuditRepository, and that 
there is also NoopEntityAuditRepository, and 
InMemoryEntityAuditRepository.

I'm thinking that when Atlas is used in a non-hadoop oriented 
environment it would be useful to have a non-hbase, persistent (helps a 
little with compliance ;-) ) audit respository. Perhaps another RDB, or 
indeed in solr/elastic search alongside Ranger's audit events.

I'll raise a JIRA on this if it's felt useful & my understanding is 
correct

Nigel.





[jira] [Updated] (ATLAS-1987) Relationship creation for many to many does not work.

2017-07-24 Thread David Radley (JIRA)

 [ 
https://issues.apache.org/jira/browse/ATLAS-1987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-1987:

Attachment: 0005LinkedMediaTypes.json
0210Glossary.json
0230Terms.json
0220CategoryHierarchy.json

> Relationship creation for many to many does not work.
> -
>
> Key: ATLAS-1987
> URL: https://issues.apache.org/jira/browse/ATLAS-1987
> Project: Atlas
>  Issue Type: Bug
>Reporter: David Radley
> Attachments: 0005LinkedMediaTypes.json, 0210Glossary.json, 
> 0220CategoryHierarchy.json, 0230Terms.json
>
>
> I run the supplied json files to add in the types.
> In summary
> I create a category and 2 terms,
> I create 2 termcategoization relationships linking both terms TermA and TermB 
> to Category CatA. All the calls are successfull.
> I get the category and terms and do not see the expected relaitonship.
> In detail
> I perform the following in Postman
> 1) I add a GlossaryCategory using
> {
>   "entity" : {
> "classifications" : [],
> "typeName" : "GlossaryCategory",
> "attributes" : {
>   "qualifiedName": "DomainA.CatA",
>   "displayName" : "CatA",
>   "shortDescription" : "CatA desc",
>   "longDescription" : "CatA desc is described here and we could say alot 
> more about it"
> }
>   }
> }
> and get 
> {
> "mutatedEntities": {
> "CREATE": [
> {
> "typeName": "GlossaryCategory",
> "attributes": {
> "qualifiedName": "DomainA.CatA"
> },
> "guid": "3b2d022b-290b-4809-a140-51fcdb92ec5d",
> "status": "ACTIVE"
> }
> ]
> },
> "guidAssignments": {
> "-1500901345537443002": "3b2d022b-290b-4809-a140-51fcdb92ec5d"
> }
> }
> 2) Create GlossaryTerm Term A
> {
>   "entity" : {
> "classifications" : [],
> "typeName" : "GlossaryTerm",
> "attributes" : {
>   "qualifiedName": "DomainA.TermA",
>   "Abbreviation" : "A",
>   "name" : "TermA",
>   "shortDescription" : "TermA desc",
>   "longDescription" : "termA desc is described here and we could say alot 
> more about it"
> }
>   }
> }
> and get response
> {
> "mutatedEntities": {
> "CREATE": [
> {
> "typeName": "GlossaryTerm",
> "attributes": {
> "qualifiedName": "DomainA.TermA"
> },
> "guid": "9a2cbbc2-efa5-4925-83fb-df2b98b119c9",
> "status": "ACTIVE"
> }
> ]
> },
> "guidAssignments": {
> "-1500901345537443005": "9a2cbbc2-efa5-4925-83fb-df2b98b119c9"
> }
> }
> I then run 
> {
> "end1": {
>
> "guid" : "6dd8d120-fd6d-4998-b650-c041d4450941",
> "typeName" : "GlossaryTerm"
> },
> "end2": {
>  "guid": "3b2d022b-290b-4809-a140-51fcdb92ec5d",
> "typeName" : "GlossaryCategory"
> },
> "typeName"  : "TermCategorization"
> }
> and get
> {
> "typeName": "TermCategorization",
> "guid": "86ea1178-5239-4dca-94cc-488e948078de",
> "end1": {
> "guid": "6dd8d120-fd6d-4998-b650-c041d4450941",
> "typeName": "GlossaryTerm"
> },
> "end2": {
> "guid": "3b2d022b-290b-4809-a140-51fcdb92ec5d",
> "typeName": "GlossaryCategory"
> },
> "label": "r:TermCategorization",
> "status": "ACTIVE",
> "createdBy": "admin",
> "updatedBy": "admin",
> "createTime": 1500903666136,
> "updateTime": 1500903666136,
> "version": 0
> }
> A get of the terms reveals no catergories associated withthem



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (ATLAS-1987) Relationship creation for many to many does not work.

2017-07-24 Thread David Radley (JIRA)
David Radley created ATLAS-1987:
---

 Summary: Relationship creation for many to many does not work.
 Key: ATLAS-1987
 URL: https://issues.apache.org/jira/browse/ATLAS-1987
 Project: Atlas
  Issue Type: Bug
Reporter: David Radley


I run the supplied json files to add in the types.
In summary
I create a category and 2 terms,
I create 2 termcategoization relationships linking both terms TermA and TermB 
to Category CatA. All the calls are successfull.
I get the category and terms and do not see the expected relaitonship.

In detail
I perform the following in Postman

1) I add a GlossaryCategory using
{
  "entity" : {
"classifications" : [],
"typeName" : "GlossaryCategory",
"attributes" : {
  "qualifiedName": "DomainA.CatA",
  "displayName" : "CatA",
  "shortDescription" : "CatA desc",
  "longDescription" : "CatA desc is described here and we could say alot 
more about it"
}
  }
}
and get 
{
"mutatedEntities": {
"CREATE": [
{
"typeName": "GlossaryCategory",
"attributes": {
"qualifiedName": "DomainA.CatA"
},
"guid": "3b2d022b-290b-4809-a140-51fcdb92ec5d",
"status": "ACTIVE"
}
]
},
"guidAssignments": {
"-1500901345537443002": "3b2d022b-290b-4809-a140-51fcdb92ec5d"
}
}

2) Create GlossaryTerm Term A
{
  "entity" : {
"classifications" : [],
"typeName" : "GlossaryTerm",
"attributes" : {
  "qualifiedName": "DomainA.TermA",
  "Abbreviation" : "A",
  "name" : "TermA",
  "shortDescription" : "TermA desc",
  "longDescription" : "termA desc is described here and we could say alot 
more about it"
}
  }
}
and get response
{
"mutatedEntities": {
"CREATE": [
{
"typeName": "GlossaryTerm",
"attributes": {
"qualifiedName": "DomainA.TermA"
},
"guid": "9a2cbbc2-efa5-4925-83fb-df2b98b119c9",
"status": "ACTIVE"
}
]
},
"guidAssignments": {
"-1500901345537443005": "9a2cbbc2-efa5-4925-83fb-df2b98b119c9"
}
}

I then run 
{
"end1": {
   
"guid"   : "6dd8d120-fd6d-4998-b650-c041d4450941",
"typeName"   : "GlossaryTerm"
},
"end2": {
 "guid"  : "3b2d022b-290b-4809-a140-51fcdb92ec5d",
"typeName"   : "GlossaryCategory"
},
"typeName"  : "TermCategorization"
}
and get
{
"typeName": "TermCategorization",
"guid": "86ea1178-5239-4dca-94cc-488e948078de",
"end1": {
"guid": "6dd8d120-fd6d-4998-b650-c041d4450941",
"typeName": "GlossaryTerm"
},
"end2": {
"guid": "3b2d022b-290b-4809-a140-51fcdb92ec5d",
"typeName": "GlossaryCategory"
},
"label": "r:TermCategorization",
"status": "ACTIVE",
"createdBy": "admin",
"updatedBy": "admin",
"createTime": 1500903666136,
"updateTime": 1500903666136,
"version": 0
}
A get of the terms reveals no catergories associated withthem






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (ATLAS-1986) Rename relationshipAttributes in Entity to relationships.

2017-07-24 Thread David Radley (JIRA)

 [ 
https://issues.apache.org/jira/browse/ATLAS-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-1986:

Attachment: ATLAS-1986.patch

> Rename relationshipAttributes in Entity to relationships. 
> --
>
> Key: ATLAS-1986
> URL: https://issues.apache.org/jira/browse/ATLAS-1986
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: ATLAS-1986.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Re: Can be tags inherited?

2017-07-24 Thread Nigel Jones

On 12/07/2017 13:17, David Radley wrote:

> will derive the tags associated with an entity. We have not designed yet
> how to override this behaviour, when the propagation behaviour in the Def
> is not appropriate. I have not got the Jira number for this  : by copy
> Madhan - do you have this for Peter.
Additionally there will be changes in how tags are propogated to ranger 
in that whilst today you just associate a tag (classification) with an 
entity, it's more useful often to associate the classification with a 
business term, and then associate that with the entity.


The changes up from the core atlas, will go through a new API 
(ATLAS-1696) and changes in ranger  (RANGER-1454) - if approved (this is 
still work in progress/unreviewed thus far), so feel free to engage in 
the discussions there too.


In terms of in general inheriting the classification ... overrides will 
IMO be needed (as David mentions) since sometimes a derived table will 
require the same classification, for example around confidentiality, 
whilst other times data could be combined and need a HIGHER 
classification whilst in other cases data may be redacted, requiring a 
LOWER classification ;-)


Regards
Nigel.




[jira] [Commented] (ATLAS-1986) Rename relationshipAttributes in Entity to relationships.

2017-07-24 Thread David Radley (JIRA)

[ 
https://issues.apache.org/jira/browse/ATLAS-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098330#comment-16098330
 ] 

David Radley commented on ATLAS-1986:
-

https://reviews.apache.org/r/61077/

> Rename relationshipAttributes in Entity to relationships. 
> --
>
> Key: ATLAS-1986
> URL: https://issues.apache.org/jira/browse/ATLAS-1986
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Review Request 61077: ATLAS-1986 Rename relationshipAttributes in Entity to relationships.

2017-07-24 Thread David Radley

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61077/
---

Review request for atlas, Madhan Neethiraj and Sarath Subramanian.


Repository: atlas


Description
---

ATLAS-1986 Rename relationshipAttributes in Entity to relationships.


Diffs
-

  intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 
365e548dd4dd9d32f22ecc1ffb8ba42f39ebc208 
  intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipDef.java 
c17e875d10169753b76fcdb483e2ca85195104b2 
  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java 
f89c55688ee1680dede87f0e7cbd5d7300fd0ef9 
  intg/src/main/java/org/apache/atlas/type/AtlasRelationshipType.java 
934dffceab3a8087e7bc3ded6bcf62ef311a6df6 
  intg/src/test/java/org/apache/atlas/type/TestAtlasRelationshipType.java 
8f3ac5b077b610b99b73b6d3e191d9c93e72ccd0 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java
 0210a1189c68d447a80c1676b236ccb7293bfcef 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1.java
 49e08a070a803a36253b502666947ab92ffe39d3 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
 b8fd70e5f42fbbeea76267471920e2764db730fe 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
 31fc8377f8eae0079915661c513ef82e254663f6 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasRelationshipStoreV1Test.java
 3ebda0df5d80f79117432f644d2acac8e800947c 


Diff: https://reviews.apache.org/r/61077/diff/1/


Testing
---

I ran junits.
I created a relationship then did a get on the entity to see the new 
relationships part in the returned json


Thanks,

David Radley



Audit Repository: Hbase, No-op ....

2017-07-24 Thread Nigel Jones
I was just reading ATLAS-1870 which related to default audit 
repositories. I see the default is HBaseBasedAuditRepository, and that 
there is also NoopEntityAuditRepository, and InMemoryEntityAuditRepository.


I'm thinking that when Atlas is used in a non-hadoop oriented 
environment it would be useful to have a non-hbase, persistent (helps a 
little with compliance ;-) ) audit respository. Perhaps another RDB, or 
indeed in solr/elastic search alongside Ranger's audit events.


I'll raise a JIRA on this if it's felt useful & my understanding is correct

Nigel.



[jira] [Commented] (ATLAS-1955) Validation for Attributes

2017-07-24 Thread Israel Varea (JIRA)

[ 
https://issues.apache.org/jira/browse/ATLAS-1955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098207#comment-16098207
 ] 

Israel Varea commented on ATLAS-1955:
-

I think these two validations will cover most of the basic validation cases.
In the second validation, from a reference table, it would be nice to be able 
to provide a reference to a column of a table, since columns are already 
modeled. I think importing a reference table into Atlas enums will duplicate 
data, and you will have to keep a synchronization between the two of them, so I 
think it will be much simple if we just point to a column. However, both of the 
two alternatives can solve succesfully the same modelling use case :)


> Validation for Attributes
> -
>
> Key: ATLAS-1955
> URL: https://issues.apache.org/jira/browse/ATLAS-1955
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Israel Varea
> Fix For: 0.9-incubating
>
>
> It would be very nice that Atlas model could contain a way to represent 
> attribute validation. 
> A simple example is that we would like to model a Person, with attributes 
> Name, Email and Country. Now we would like to specify that Email has to 
> follow a specific regular expression, so it would be nice if we could set 
> Email -> hasValidation -> EmailRegex, with EmailRegex having:
> Name: Email Regular Expresion
> Expression: /[0-9a-z]+@[0-9a-z]+.[0-9a-z]+/
> For more complex types of validation, e.g. checking card number validity, it 
> could be added some external validator function/service.
> Name: Credit Card Number Validator
> Validator: org.apache.atlas.validators.creditcard or 
> https://host:port/creditCardValidator
> For validations from a reference table, for example a country name, it could 
> be:
> Name: Country Name Ref Validator
> Reference Column: 
> where  would be an instance of type Hive_Column or 
> HBase_Column.
> Since this is a kind of Standarization, it could be placed in [Area 
> 5|https://cwiki.apache.org/confluence/display/ATLAS/Area+5+-+Standards].
> A similar approach is followed in software 
> [Kylo|https://github.com/Teradata/kylo/tree/master/integrations/spark/spark-validate-cleanse]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Re: Relationship attributes

2017-07-24 Thread David Radley
Hi all,
I have just had a chat with Graham. we are thinking that we should go with 
relationships as the top level name. We also think we could helpfully add 
in the related entity in the new class like this: 

class AtlasRelationshipObjectId  {
   AtlasObjectId relatedEntity; 
   String relationshipGuid;
   AtlasStruct relationshipAttributes; 
}

I think this gives us the best of both worlds, 
   regards David. 





From:   Graham Wallis 
To: dev@atlas.apache.org
Cc: Madhan Neethiraj , Sarath Subramanian 
 
Date:   24/07/2017 09:58
Subject:Re: Relationship attributes



Personally I think 'relatedEntities' is clearer.

Best regards,
  Graham

Graham Wallis
IBM Analytics Emerging Technology Center
Internet: graham_wal...@uk.ibm.com 
IBM Laboratories, Hursley Park, Hursley, Hampshire SO21 2JN
Tel: +44-1962-815356Tie: 7-245356




From:   Madhan Neethiraj 
To: "dev@atlas.apache.org" , Sarath Subramanian 

Date:   24/07/2017 09:04
Subject:Re: Relationship attributes
Sent by:Madhan Neethiraj 



Current name of ‘relationshipAttributes’ makes sense looking from an 
entity point-of-view – it distinguishes regular-attributes of an entity 
from attributes injected by relationships. However, given that 
relationships can themselves might have attributes, it can be confusing.

I was going to suggest ‘relatedEntities’; but ‘relationships’ seems to be 
good choice.

+1 for naming the field as ‘relationships’.

Thanks,
Madhan


On 7/24/17, 12:53 AM, "David Radley"  wrote:

Hi Sarath,
Great, personally for the Entity's current relationshipAttributes, I 
prefer relationships as it is simpler - is there a reason you need 
attribute in the name?
 all the best, David. 
 
 
 
From:   Sarath Subramanian 
To: dev@atlas.apache.org
Cc: Madhan Neethiraj 
Date:   24/07/2017 07:09
Subject:Re: Relationship attributes
 
 
 
Hi David,
 
I agree with using the term 'relationship attributes' for attributes 
of
relationship, I suggest we use "relatedAttributes" for relationship
attributes of entity.
 
 
Thanks,
Sarath Subramanian
 
On Sun, Jul 23, 2017 at 2:22 AM, David Radley 

wrote:
 
> Hi Madhan,
> When I see the phrase 'relationship attributes', I am never quite 
sure
> whether we are referring to the attributes of an entity that relate 
to
> another entity or the attributes of the relationship instance 
itself. I
> think the phrase ' relationship attributes' more naturally fits as 
the
> attributes of the relationship itself; we are using it in the other 
sense.
>
> I suggest we change the relationshipAttributes in the entity to
> relationships (if you really want attributes in the name we could 
call 
it
> relatingAttributes)- and use the term 'relationship attributes' 
purely 
for
> the attributes of the relationship itself. What do you think?
> all the best, David.
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with 
number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire 
PO6 
3AU
>
 
 
 
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with 
number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 

3AU
 




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU



Re: [jira] [Created] (ATLAS-1982) Update referenced to "incubator" in website

2017-07-24 Thread Nigel Jones

On 22/07/2017 00:00, Madhan Neethiraj (JIRA) wrote:
> Madhan Neethiraj created ATLAS-1982:

>   Summary: Update referenced to "incubator" in website
>
When INFRA update the mailing lists, I presume the old subscriber list 
is still in place?
 -> Users should update any email filters they have to use the new 
addresses (ie for auto filing)

 -> No other explicit action is needed?

Also worth noting is that back in 2015 I had the atlas mailing lists 
replicated onto gmane.org. This allows access via nntp 
(gmane.comp.apache.incubator.atlas.devel etc). For those who don't know, 
gmane has had a chequered recent history. It was down for a while, and 
then moved to a new owner. Whilst the newsgroups are now working I'm 
sceptical any updates will get handled promptly. I will though try and 
contact them to update the lists. I have no idea whether inbound, 
outbound may stop working - nor indeed who finds them useful (I do!). 
User remains absent from this mirror.


Regards
Nigel.



Re: [jira] [Commented] (ATLAS-1955) Validation for Attributes

2017-07-24 Thread Nigel Jones

On 23/07/2017 10:36, David Radley (JIRA) wrote:
>
> [~ivarea] some thoughts:
> I think there is a value in Atlas having the capability to validate 
attribute values conform to a certain pattern.

I think there's 2 forms of validation here..

a) Capturing validation rules in Atlas - ie more metadata that we might 
relate to business terms & apply to assets.. for example to define the 
fact that whenever we refer to a credit card number it needs to be in a 
certain format


b) Validating the metadata itself

Both seem entirely valid.. but I think Israel is referring to a) and 
David - you are referring to b. Two JIRAS? I could be wrong though, if 
so a) is an additional idea :-)




Re: Maven restructure

2017-07-24 Thread Nigel Jones

On 19/07/2017 16:45, David Radley wrote:


for Hadoop 3 to bring in Java 8 and Janus? We should look to see what the
migration considerations are for Hadoop 2 customers to move to Janus.


It's probably worth creating a subtask, or separate JIRA for the 
specific data migration work.


Initially I think we can allow Janus only for new users, with existing 
users continuing to use titan 0.54 ... but at some point - in fact as 
soon as practical.. though this may be after hadoop 3 .. we should 
deprecate 0.54 & so need to get those customer's data forward migrated.


Additionally doing this even earlier than needed for hadoop helps to 
cement JanusGraph as the graph tech of choice in Atlas :-), though I'm 
unsure how many Atlas users there are currently, with production data, 
using titan 0.54 outside hadoop.


NIgel.



Re: Relationship attributes

2017-07-24 Thread Nigel L Jones
I'm good with relatedEntities
 - relationshipAttributes is definately confusing
 - relationships isn't clear to me

other options could include
 - relatedEntityAttributes - but is a little longwinded
 - injectedAttributes - another option but perhaps unclear for some


Nigel Jones, Analytics CTO Office - jon...@uk.ibm.com



From:   Graham Wallis 
To: dev@atlas.apache.org
Cc: Madhan Neethiraj , Sarath Subramanian 

Date:   24/07/2017 09:58
Subject:Re: Relationship attributes



Personally I think 'relatedEntities' is clearer.

Best regards,
  Graham

Graham Wallis
IBM Analytics Emerging Technology Center
Internet: graham_wal...@uk.ibm.com 
IBM Laboratories, Hursley Park, Hursley, Hampshire SO21 2JN
Tel: +44-1962-815356Tie: 7-245356




From:   Madhan Neethiraj 
To: "dev@atlas.apache.org" , Sarath Subramanian 

Date:   24/07/2017 09:04
Subject:Re: Relationship attributes
Sent by:Madhan Neethiraj 



Current name of ‘relationshipAttributes’ makes sense looking from an 
entity point-of-view – it distinguishes regular-attributes of an entity 
from attributes injected by relationships. However, given that 
relationships can themselves might have attributes, it can be confusing.

I was going to suggest ‘relatedEntities’; but ‘relationships’ seems to be 
good choice.

+1 for naming the field as ‘relationships’.

Thanks,
Madhan


On 7/24/17, 12:53 AM, "David Radley"  wrote:

Hi Sarath,
Great, personally for the Entity's current relationshipAttributes, I 
prefer relationships as it is simpler - is there a reason you need 
attribute in the name?
 all the best, David. 
 
 
 
From:   Sarath Subramanian 
To: dev@atlas.apache.org
Cc: Madhan Neethiraj 
Date:   24/07/2017 07:09
Subject:Re: Relationship attributes
 
 
 
Hi David,
 
I agree with using the term 'relationship attributes' for attributes 
of
relationship, I suggest we use "relatedAttributes" for relationship
attributes of entity.
 
 
Thanks,
Sarath Subramanian
 
On Sun, Jul 23, 2017 at 2:22 AM, David Radley 

wrote:
 
> Hi Madhan,
> When I see the phrase 'relationship attributes', I am never quite 
sure
> whether we are referring to the attributes of an entity that relate 
to
> another entity or the attributes of the relationship instance 
itself. I
> think the phrase ' relationship attributes' more naturally fits as 
the
> attributes of the relationship itself; we are using it in the other 
sense.
>
> I suggest we change the relationshipAttributes in the entity to
> relationships (if you really want attributes in the name we could 
call 
it
> relatingAttributes)- and use the term 'relationship attributes' 
purely 
for
> the attributes of the relationship itself. What do you think?
> all the best, David.
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with 
number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire 
PO6 
3AU
>
 
 
 
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with 
number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 

3AU
 




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU



[jira] [Created] (ATLAS-1986) Rename relationshipAttributes in Entity to relationships.

2017-07-24 Thread David Radley (JIRA)
David Radley created ATLAS-1986:
---

 Summary: Rename relationshipAttributes in Entity to relationships. 
 Key: ATLAS-1986
 URL: https://issues.apache.org/jira/browse/ATLAS-1986
 Project: Atlas
  Issue Type: Improvement
Reporter: David Radley
Assignee: David Radley






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Re: Relationship attributes

2017-07-24 Thread Graham Wallis
Personally I think 'relatedEntities' is clearer.

Best regards,
  Graham

Graham Wallis
IBM Analytics Emerging Technology Center
Internet: graham_wal...@uk.ibm.com 
IBM Laboratories, Hursley Park, Hursley, Hampshire SO21 2JN
Tel: +44-1962-815356Tie: 7-245356




From:   Madhan Neethiraj 
To: "dev@atlas.apache.org" , Sarath Subramanian 

Date:   24/07/2017 09:04
Subject:Re: Relationship attributes
Sent by:Madhan Neethiraj 



Current name of ‘relationshipAttributes’ makes sense looking from an 
entity point-of-view – it distinguishes regular-attributes of an entity 
from attributes injected by relationships. However, given that 
relationships can themselves might have attributes, it can be confusing.

I was going to suggest ‘relatedEntities’; but ‘relationships’ seems to be 
good choice.

+1 for naming the field as ‘relationships’.

Thanks,
Madhan


On 7/24/17, 12:53 AM, "David Radley"  wrote:

Hi Sarath,
Great, personally for the Entity's current relationshipAttributes, I 
prefer relationships as it is simpler - is there a reason you need 
attribute in the name?
 all the best, David. 
 
 
 
From:   Sarath Subramanian 
To: dev@atlas.apache.org
Cc: Madhan Neethiraj 
Date:   24/07/2017 07:09
Subject:Re: Relationship attributes
 
 
 
Hi David,
 
I agree with using the term 'relationship attributes' for attributes 
of
relationship, I suggest we use "relatedAttributes" for relationship
attributes of entity.
 
 
Thanks,
Sarath Subramanian
 
On Sun, Jul 23, 2017 at 2:22 AM, David Radley 

wrote:
 
> Hi Madhan,
> When I see the phrase 'relationship attributes', I am never quite 
sure
> whether we are referring to the attributes of an entity that relate 
to
> another entity or the attributes of the relationship instance 
itself. I
> think the phrase ' relationship attributes' more naturally fits as 
the
> attributes of the relationship itself; we are using it in the other 
sense.
>
> I suggest we change the relationshipAttributes in the entity to
> relationships (if you really want attributes in the name we could 
call 
it
> relatingAttributes)- and use the term 'relationship attributes' 
purely 
for
> the attributes of the relationship itself. What do you think?
> all the best, David.
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with 
number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire 
PO6 
3AU
>
 
 
 
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with 
number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 
3AU
 




Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU



[jira] [Commented] (ATLAS-1984) Use AtlasRelatedObjectId to refer to relationship attributes during create/update/delete of entity

2017-07-24 Thread David Radley (JIRA)

[ 
https://issues.apache.org/jira/browse/ATLAS-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16098097#comment-16098097
 ] 

David Radley commented on ATLAS-1984:
-

As per my dev list comment, I like the way of using an existing attribute as a 
display name; I wonder if we should have display name as a flag on 
AttributeDef, if there were multiples specified  then we could concatinate or 
police to only allow one. This would allow people to create a new attribute 
called displayAttribute if they like or use an existing attribute like name in 
the GlossaryCategory case. It would also mean that classifications, 
relationships and structures could have display names.   

I wonder if the displayName is not specified for Entities, then we could 
default it to the first unique attribute value if there are unique attributes, 
if not then we could default to the qualified name for Referenceables. If we 
wanted we could sample the data values and look for the most unique and use 
that! 

If we do this - I  think that it would make sense to enhance AtlasObjectId to 
show the list of display attributes as well. 

Also i suggest we call AtlasRelatedObjectId, AtlasRelationshipObjectId. I think 
we should just use the word 'relationship' in all namings - and only introduce 
new terminology like 'related' if there is a new meaning we are looking to 
show.   

 

> Use AtlasRelatedObjectId to refer to relationship attributes during 
> create/update/delete of entity
> --
>
> Key: ATLAS-1984
> URL: https://issues.apache.org/jira/browse/ATLAS-1984
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: trunk, 0.9-incubating
>Reporter: Sarath Subramanian
>Assignee: Sarath Subramanian
>
> AtlasObjectId is used to refer to entity attributes referring to another 
> entity.
> hive_table.columns => List
> hive_table.db => AtlasObjectId
> Change this to use AtlasRelatedObjectId with the following structure:
> class AtlasRelatedObjectId  {
>String relationshipGuid;
>String displayText;
>AtlasStruct relationshipAttributes; 
> }



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Review Request 61076: ATLAS-1910 : Classpath issue in import-hive.sh script

2017-07-24 Thread Benjamin BONNET

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61076/
---

Review request for atlas.


Repository: atlas


Description
---

https://issues.apache.org/jira/browse/ATLAS-1910


Diffs
-

  addons/hive-bridge/src/bin/import-hive.sh 47581ac8d 


Diff: https://reviews.apache.org/r/61076/diff/1/


Testing
---

tested on hdp 2.6 sandbox.


Thanks,

Benjamin BONNET



Re: Review Request 61040: ATLAS-1981: Cache escaped type-query string to avoid repeated computation

2017-07-24 Thread Nixon Rodrigues

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61040/#review181193
---


Ship it!




Ship It!

- Nixon Rodrigues


On July 24, 2017, 3:50 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61040/
> ---
> 
> (Updated July 24, 2017, 3:50 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-1981
> https://issues.apache.org/jira/browse/ATLAS-1981
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Updated search to use cached type-query string to improve performance
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 049a537b 
>   
> repository/src/test/java/org/apache/atlas/services/EntityDiscoveryServiceTest.java
>  dfb2ee27 
> 
> 
> Diff: https://reviews.apache.org/r/61040/diff/2/
> 
> 
> Testing
> ---
> 
> verified that the search returns correct results after this enhancement
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>



Re: Review Request 61040: ATLAS-1981: Cache escaped type-query string to avoid repeated computation

2017-07-24 Thread Nixon Rodrigues

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61040/#review181192
---


Ship it!




Ship It!

- Nixon Rodrigues


On July 24, 2017, 3:50 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61040/
> ---
> 
> (Updated July 24, 2017, 3:50 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-1981
> https://issues.apache.org/jira/browse/ATLAS-1981
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Updated search to use cached type-query string to improve performance
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/type/AtlasStructType.java 049a537b 
>   
> repository/src/test/java/org/apache/atlas/services/EntityDiscoveryServiceTest.java
>  dfb2ee27 
> 
> 
> Diff: https://reviews.apache.org/r/61040/diff/2/
> 
> 
> Testing
> ---
> 
> verified that the search returns correct results after this enhancement
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>



Re: Relationship attributes

2017-07-24 Thread Madhan Neethiraj
Current name of ‘relationshipAttributes’ makes sense looking from an entity 
point-of-view – it distinguishes regular-attributes of an entity from 
attributes injected by relationships. However, given that relationships can 
themselves might have attributes, it can be confusing.

I was going to suggest ‘relatedEntities’; but ‘relationships’ seems to be good 
choice.

+1 for naming the field as ‘relationships’.

Thanks,
Madhan


On 7/24/17, 12:53 AM, "David Radley"  wrote:

Hi Sarath,
Great, personally for the Entity's current relationshipAttributes, I 
prefer relationships as it is simpler - is there a reason you need 
attribute in the name?
 all the best, David. 



From:   Sarath Subramanian 
To: dev@atlas.apache.org
Cc: Madhan Neethiraj 
Date:   24/07/2017 07:09
Subject:Re: Relationship attributes



Hi David,

I agree with using the term 'relationship attributes' for attributes of
relationship, I suggest we use "relatedAttributes" for relationship
attributes of entity.


Thanks,
Sarath Subramanian

On Sun, Jul 23, 2017 at 2:22 AM, David Radley 
wrote:

> Hi Madhan,
> When I see the phrase 'relationship attributes', I am never quite sure
> whether we are referring to the attributes of an entity that relate to
> another entity or the attributes of the relationship instance itself. I
> think the phrase ' relationship attributes' more naturally fits as the
> attributes of the relationship itself; we are using it in the other 
sense.
>
> I suggest we change the relationshipAttributes in the entity to
> relationships (if you really want attributes in the name we could call 
it
> relatingAttributes)- and use the term 'relationship attributes' purely 
for
> the attributes of the relationship itself. What do you think?
> all the best, David.
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 
3AU
>



Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU





Re: Relationship attributes

2017-07-24 Thread David Radley
Hi Sarath,
Great, personally for the Entity's current relationshipAttributes, I 
prefer relationships as it is simpler - is there a reason you need 
attribute in the name?
 all the best, David. 



From:   Sarath Subramanian 
To: dev@atlas.apache.org
Cc: Madhan Neethiraj 
Date:   24/07/2017 07:09
Subject:Re: Relationship attributes



Hi David,

I agree with using the term 'relationship attributes' for attributes of
relationship, I suggest we use "relatedAttributes" for relationship
attributes of entity.


Thanks,
Sarath Subramanian

On Sun, Jul 23, 2017 at 2:22 AM, David Radley 
wrote:

> Hi Madhan,
> When I see the phrase 'relationship attributes', I am never quite sure
> whether we are referring to the attributes of an entity that relate to
> another entity or the attributes of the relationship instance itself. I
> think the phrase ' relationship attributes' more naturally fits as the
> attributes of the relationship itself; we are using it in the other 
sense.
>
> I suggest we change the relationshipAttributes in the entity to
> relationships (if you really want attributes in the name we could call 
it
> relatingAttributes)- and use the term 'relationship attributes' purely 
for
> the attributes of the relationship itself. What do you think?
> all the best, David.
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 
3AU
>



Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


[jira] [Created] (ATLAS-1985) Regression : Basic/DSL Query fired on PASSIVE server redirects to ACTIVE server , adding an extra "amp;" to the parameters

2017-07-24 Thread Sharmadha Sainath (JIRA)
Sharmadha Sainath created ATLAS-1985:


 Summary: Regression : Basic/DSL Query fired on PASSIVE server 
redirects to ACTIVE server , adding an extra "amp;" to the parameters
 Key: ATLAS-1985
 URL: https://issues.apache.org/jira/browse/ATLAS-1985
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 0.9-incubating, 0.8.1-incubating
Reporter: Sharmadha Sainath
Priority: Critical


1. Fired the query 
{code}
http://PassiveHost:21000/api/atlas/discovery/search/fulltext?limit=100=hive_table
{code}

The query failed with 
{code}
{
error: "dslQuery cannot be null cannot be null"
}
{code}

The redirected URL is :
{code}
http://activehost:21000/api/atlas/discovery/search/fulltext?limit=100query=hive_table
{code}

Redirection adds an extra "amp;" and ignores rest of the query after the first 
parameter.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (ATLAS-1984) Use AtlasRelatedObjectId to refer to relationship attributes during create/update/delete of entity

2017-07-24 Thread Sarath Subramanian (JIRA)
Sarath Subramanian created ATLAS-1984:
-

 Summary: Use AtlasRelatedObjectId to refer to relationship 
attributes during create/update/delete of entity
 Key: ATLAS-1984
 URL: https://issues.apache.org/jira/browse/ATLAS-1984
 Project: Atlas
  Issue Type: Improvement
  Components:  atlas-core
Affects Versions: trunk, 0.9-incubating
Reporter: Sarath Subramanian
Assignee: Sarath Subramanian


AtlasObjectId is used to refer to entity attributes referring to another entity.
hive_table.columns => List
hive_table.db => AtlasObjectId

Change this to use AtlasRelatedObjectId with the following structure:
class AtlasRelatedObjectId  {
   String relationshipGuid;
   String displayText;
   AtlasStruct relationshipAttributes; 
}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


Re: Relationship attributes

2017-07-24 Thread Sarath Subramanian
Hi David,

I agree with using the term 'relationship attributes' for attributes of
relationship, I suggest we use "relatedAttributes" for relationship
attributes of entity.


Thanks,
Sarath Subramanian

On Sun, Jul 23, 2017 at 2:22 AM, David Radley 
wrote:

> Hi Madhan,
> When I see the phrase 'relationship attributes', I am never quite sure
> whether we are referring to the attributes of an entity that relate to
> another entity or the attributes of the relationship instance itself. I
> think the phrase ' relationship attributes' more naturally fits as the
> attributes of the relationship itself; we are using it in the other sense.
>
> I suggest we change the relationshipAttributes in the entity to
> relationships (if you really want attributes in the name we could call it
> relatingAttributes)- and use the term 'relationship attributes' purely for
> the attributes of the relationship itself. What do you think?
> all the best, David.
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>


[jira] [Updated] (ATLAS-1967) Search UI : Render attribute filter based on browser URL

2017-07-24 Thread Keval Bhatt (JIRA)

 [ 
https://issues.apache.org/jira/browse/ATLAS-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Keval Bhatt updated ATLAS-1967:
---
Fix Version/s: 0.8-incubating
   0.9-incubating

> Search UI : Render attribute filter based on browser URL
> 
>
> Key: ATLAS-1967
> URL: https://issues.apache.org/jira/browse/ATLAS-1967
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 0.9-incubating
>Reporter: Keval Bhatt
>Assignee: Keval Bhatt
> Fix For: 0.8-incubating, 0.9-incubating
>
> Attachments: ATLAS-1967.patch
>
>
> Support added for attributes to be rendered based on browser URL.
> Example:
> Search for the Table entity whose name is starting from sales and description 
> contains monthly string.
> http://localhost:21000/#!/search/searchResult?type=Table=name%3A%3Abegins_with%3A%3Asales%3A%3Astring%2Cdescription%3A%3Acontains%3A%3Amonthly%3A%3Astring=basic=db%2Cdescription%2Cname%2Cowner%2Csd%2Cselected%2Ctag%2Cterms%2CtypeName.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)