Re: Review Request 60922: ATLAS-1925: Misc bugfixes for ATLAS-1880

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60922/#review180763
---


Fix it, then Ship it!




Ship It!


repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
Lines 479 (patched)


keep this inside a finally {} block, to ensure that search-context gets 
removed when search method exits.


- Madhan Neethiraj


On July 17, 2017, 9:37 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60922/
> ---
> 
> (Updated July 17, 2017, 9:37 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-1925
> https://issues.apache.org/jira/browse/ATLAS-1925
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Resolves ATLAS-1917, 1922, 1926
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java
>  a4538bda 
>   repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
> 1a2d997b 
> 
> 
> Diff: https://reviews.apache.org/r/60922/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean package executes successfully
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 60927: ATLAS-1957: Multi-vertex atlas graph query

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60927/#review180754
---




repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
Lines 184 (patched)


TODO: The multi-vertex query will be used here


- Apoorv Naik


On July 17, 2017, 10:31 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60927/
> ---
> 
> (Updated July 17, 2017, 10:31 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Sarath 
> Subramanian, and Suma Shivaprasad.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Initial draft for multi-vertex graph query in atlas
> 
> 
> Diffs
> -
> 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraph.java 
> dded76f8 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasMultiVertexQuery.java
>  PRE-CREATION 
>   
> graphdb/common/src/main/java/org/apache/atlas/repository/graphdb/titan/query/NativeTitanMultiVertexQuery.java
>  PRE-CREATION 
>   
> graphdb/common/src/main/java/org/apache/atlas/repository/graphdb/titan/query/NativeTitanQueryFactory.java
>  ac7ff9e8 
>   graphdb/titan0/pom.xml ceda1bb2 
>   
> graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Graph.java
>  2408287b 
>   
> graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/query/Titan0AtlasMultiVertexQuery.java
>  PRE-CREATION 
>   
> graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1Graph.java
>  ffb6b37f 
>   
> graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/query/Titan1AtlasMultiVertexQuery.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
>  77b2c7c2 
> 
> 
> Diff: https://reviews.apache.org/r/60927/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Review Request 60927: ATLAS-1957: Multi-vertex atlas graph query

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60927/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, Sarath 
Subramanian, and Suma Shivaprasad.


Repository: atlas


Description
---

Initial draft for multi-vertex graph query in atlas


Diffs
-

  graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraph.java 
dded76f8 
  
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasMultiVertexQuery.java
 PRE-CREATION 
  
graphdb/common/src/main/java/org/apache/atlas/repository/graphdb/titan/query/NativeTitanMultiVertexQuery.java
 PRE-CREATION 
  
graphdb/common/src/main/java/org/apache/atlas/repository/graphdb/titan/query/NativeTitanQueryFactory.java
 ac7ff9e8 
  graphdb/titan0/pom.xml ceda1bb2 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Graph.java
 2408287b 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/query/Titan0AtlasMultiVertexQuery.java
 PRE-CREATION 
  
graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/Titan1Graph.java
 ffb6b37f 
  
graphdb/titan1/src/main/java/org/apache/atlas/repository/graphdb/titan1/query/Titan1AtlasMultiVertexQuery.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/discovery/ClassificationSearchProcessor.java
 77b2c7c2 


Diff: https://reviews.apache.org/r/60927/diff/1/


Testing
---


Thanks,

Apoorv Naik

[jira] [Created] (ATLAS-1958) Better graph traversal APIs

[Apoorv Naik (JIRA)]

Apoorv Naik created ATLAS-1958:
--

 Summary: Better graph traversal APIs 
 Key: ATLAS-1958
 URL: https://issues.apache.org/jira/browse/ATLAS-1958
 Project: Atlas
  Issue Type: Improvement
Reporter: Apoorv Naik
Assignee: Apoorv Naik


Right now the only way to perform deep/complex graph traversals is through the 
Gremlin API.
Currently atlas has an ExpressionFactory abstraction which internally generates 
groovy expression as per the compatible TinkerPop version. It requires solid 
understanding of the TypeSystem, gremlin query structure, required 
optimizations etc.

Instead, TinkerPop itself supports a Gremlin pipeline (TP2) and a 
GraphTraversal (TP3), which are good abstractions over the gremlin query 
language. We should have a Querying abstraction in Atlas which is agnostic to 
the underlying Graph implementation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (ATLAS-1957) MultiVertex graph query

[Apoorv Naik (JIRA)]

Apoorv Naik created ATLAS-1957:
--

 Summary: MultiVertex graph query
 Key: ATLAS-1957
 URL: https://issues.apache.org/jira/browse/ATLAS-1957
 Project: Atlas
  Issue Type: Improvement
Affects Versions: trunk
Reporter: Apoorv Naik
Assignee: Apoorv Naik
 Fix For: trunk


There are lot of use cases where the graph query needs to be executed against a 
given set of vertices, right now the only way to do it to first determine a 
unique indexed attribute for those and then use that as the very first has 
condition filter.

e.g. graph.query().has("__guid", IN, Set guids)

This makes it difficult to perform hops in the graph traversal using the query 
API alone.

Titan0 and Titan1 has the ability to perform queries on multiple vertices in 
one go, which would benefit us in the future.

This enhancement will track the implementation of the MultiVertexQuery feature 
across Titan0 and Titan1



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1956) AtlasSearchResult to include filter attributes by default

[Apoorv Naik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apoorv Naik updated ATLAS-1956:
---
Issue Type: Improvement  (was: Bug)

> AtlasSearchResult to include filter attributes by default
> -
>
> Key: ATLAS-1956
> URL: https://issues.apache.org/jira/browse/ATLAS-1956
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: trunk
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
> Fix For: trunk
>
>
> If no attributes are specified in the SearchParameters then by default 
> AtlasSearchResult should include the attributes present in the filter criteria



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Review Request 60922: ATLAS-1925: Misc bugfixes for ATLAS-1880

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60922/
---

Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath 
Subramanian.


Bugs: ATLAS-1925
https://issues.apache.org/jira/browse/ATLAS-1925


Repository: atlas


Description
---

Resolves ATLAS-1917, 1922, 1926


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
a4538bda 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
1a2d997b 


Diff: https://reviews.apache.org/r/60922/diff/1/


Testing
---

mvn clean package executes successfully


Thanks,

Apoorv Naik

Review Request 60923: ATLAS-1956: Search results should include the filter criteria attributes

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60923/
---

Review request for atlas, Madhan Neethiraj and Sarath Subramanian.


Bugs: ATLAS-1956
https://issues.apache.org/jira/browse/ATLAS-1956


Repository: atlas


Description
---

If no attributes are specified in the SearchParameters then by default 
AtlasSearchResult should include the attributes present in the filter criteria


Diffs
-

  
repository/src/main/java/org/apache/atlas/discovery/EntityDiscoveryService.java 
a4538bda 
  repository/src/main/java/org/apache/atlas/discovery/SearchContext.java 
2125d610 
  repository/src/main/java/org/apache/atlas/discovery/SearchProcessor.java 
1a2d997b 


Diff: https://reviews.apache.org/r/60923/diff/1/


Testing
---

Tested using curl calls, response contains the attributes present in the filter 
criteria

mvn clean package executes successfully


Thanks,

Apoorv Naik

Re: Testing API...

[Madhan Neethiraj]

“/api/v2/search” is not the correct URI. Can you try  with 
“api/atlas/v2/search”?

Madhan



On 7/17/17, 11:51 AM, "Jean Georges Perrin"  wrote:

Hi,

I installed and run Atlas on my Mac. Everything seems to be working fine in 
the UI, but I cannot get the API to work... I am sure it is really a small 
thing, but...

curl --user admin:admin -v http://localhost:21000/api/v2/search/basic 


returns:

*   Trying ::1...
* TCP_NODELAY set
* Connection failed
* connect to ::1 port 21000 failed: Connection refused
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 21000 (#0)
* Server auth using Basic with user 'admin'
> GET /api/v2/search/basic HTTP/1.1
> Host: localhost:21000
> Authorization: Basic YWRtaW46YWRtaW4=
> User-Agent: curl/7.54.1
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Date: Mon, 17 Jul 2017 18:42:49 GMT
< Set-Cookie: ATLASSESSIONID=14z2v5j9416lcibimeswa5it;Path=/;HttpOnly
< X-Frame-Options: DENY
< Content-Type: text/html; charset=ISO-8859-1
< Cache-Control: must-revalidate,no-cache,no-store
< Content-Length: 303
< Server: Jetty(9.2.12.v20150709)
<



Error 404 Not Found

HTTP ERROR 404
Problem accessing /api/v2/search/basic. Reason:
Not FoundPowered by 
Jetty://



* Connection #0 to host localhost left intact

Thanks!

jg

Testing API...

[Jean Georges Perrin]

Hi,

I installed and run Atlas on my Mac. Everything seems to be working fine in the 
UI, but I cannot get the API to work... I am sure it is really a small thing, 
but...

curl --user admin:admin -v http://localhost:21000/api/v2/search/basic 


returns:

*   Trying ::1...
* TCP_NODELAY set
* Connection failed
* connect to ::1 port 21000 failed: Connection refused
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 21000 (#0)
* Server auth using Basic with user 'admin'
> GET /api/v2/search/basic HTTP/1.1
> Host: localhost:21000
> Authorization: Basic YWRtaW46YWRtaW4=
> User-Agent: curl/7.54.1
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Date: Mon, 17 Jul 2017 18:42:49 GMT
< Set-Cookie: ATLASSESSIONID=14z2v5j9416lcibimeswa5it;Path=/;HttpOnly
< X-Frame-Options: DENY
< Content-Type: text/html; charset=ISO-8859-1
< Cache-Control: must-revalidate,no-cache,no-store
< Content-Length: 303
< Server: Jetty(9.2.12.v20150709)
<



Error 404 Not Found

HTTP ERROR 404
Problem accessing /api/v2/search/basic. Reason:
Not FoundPowered by Jetty://



* Connection #0 to host localhost left intact

Thanks!

jg

[jira] [Created] (ATLAS-1955) Validation for Attributes

[Israel Varea (JIRA)]

Israel Varea created ATLAS-1955:
---

 Summary: Validation for Attributes
 Key: ATLAS-1955
 URL: https://issues.apache.org/jira/browse/ATLAS-1955
 Project: Atlas
  Issue Type: New Feature
  Components:  atlas-core
Affects Versions: 0.9-incubating
Reporter: Israel Varea
 Fix For: 0.9-incubating


It would be very nice that Atlas model could contain a way to represent 
attribute validation. 

A simple example is that we would like to model a Person, with attributes Name, 
Email and Country. Now we would like to specify that Email has to follow a 
specific regular expression, so it would be nice if we could set Email -> 
hasValidation -> EmailRegex, with EmailRegex having:

Name: Email Regular Expresion
Expression: /[0-9a-z]+@[0-9a-z]+.[0-9a-z]+/

For more complex types of validation, e.g. checking card number validity, it 
could be added some external validator function/service.

Name: Credit Card Number Validator
Validator: org.apache.atlas.validators.creditcard or 
https://host:port/creditCardValidator

For validations from a reference table, for example a country name, it could be:

Name: Country Name Ref Validator
Reference Column: 

where  would be an instance of type Hive_Column or 
HBase_Column.

Since this is a kind of Standarization, it could be placed in [Area 
5|https://cwiki.apache.org/confluence/display/ATLAS/Area+5+-+Standards].

A similar approach is followed in software 
[Kylo|https://github.com/Teradata/kylo/tree/master/integrations/spark/spark-validate-cleanse]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (ATLAS-1944) NotificationHookConsumer throws exception while shutting down the consumer thread

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-1944.
-
   Resolution: Fixed
Fix Version/s: 0.8.1-incubating

Committed to the following branches:
 - master: 
http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/18745cf4
 - 0.8-incubating: 
http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/9b63509d

> NotificationHookConsumer throws exception while shutting down the consumer 
> thread
> -
>
> Key: ATLAS-1944
> URL: https://issues.apache.org/jira/browse/ATLAS-1944
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Ayub Pathan
>Assignee: Ashutosh Mestry
>Priority: Critical
> Fix For: 0.9-incubating, 0.8.1-incubating
>
> Attachments: ATLAS-1944.patch
>
>
> NotificationHookConsumer throws below exception while shutting down the 
> consumer thread, this issue is possibly after this commit.
> https://github.com/apache/incubator-atlas/commit/0e7f8ea4603c858cc295259bbd1a22314b732f62
> CC [~nixonrodrigues]
> {noformat}
> 2017-07-12 01:26:09,743 WARN  - [pool-1-thread-1:] ~ Error stopping service 
> org.apache.atlas.notification.NotificationHookConsumer (Services:69)
> java.util.ConcurrentModificationException: KafkaConsumer is not safe for 
> multi-threaded access
> at 
> org.apache.kafka.clients.consumer.KafkaConsumer.acquire(KafkaConsumer.java:1557)
> at 
> org.apache.kafka.clients.consumer.KafkaConsumer.close(KafkaConsumer.java:1467)
> at 
> org.apache.atlas.kafka.AtlasKafkaConsumer.close(AtlasKafkaConsumer.java:88)
> at 
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.stop(NotificationHookConsumer.java:384)
> at 
> org.apache.atlas.notification.NotificationHookConsumer.stopConsumerThreads(NotificationHookConsumer.java:172)
> at 
> org.apache.atlas.notification.NotificationHookConsumer.stop(NotificationHookConsumer.java:155)
> at org.apache.atlas.service.Services.stop(Services.java:67)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at 
> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:366)
> at 
> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeDestroyMethods(InitDestroyAnnotationBeanPostProcessor.java:325)
> at 
> org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeDestruction(InitDestroyAnnotationBeanPostProcessor.java:154)
> at 
> org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:253)
> at 
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:578)
> at 
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:554)
> at 
> org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:961)
> at 
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:523)
> at 
> org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:968)
> at 
> org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1033)
> at 
> org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1009)
> at 
> org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:961)
> at 
> org.springframework.web.context.ContextLoader.closeWebApplicationContext(ContextLoader.java:583)
> at 
> org.springframework.web.context.ContextLoaderListener.contextDestroyed(ContextLoaderListener.java:116)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.callContextDestroyed(ContextHandler.java:808)
> at 
> org.eclipse.jetty.servlet.ServletContextHandler.callContextDestroyed(ServletContextHandler.java:457)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doStop(ContextHandler.java:842)
> at 
> 

[jira] [Updated] (ATLAS-1839) Area 2 of the open metadata model

[David Radley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-1839:

Attachment: (was: 0210Glossary.json)

> Area 2 of the open metadata model
> -
>
> Key: ATLAS-1839
> URL: https://issues.apache.org/jira/browse/ATLAS-1839
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Mandy Chessell
>Assignee: Mandy Chessell
>  Labels: OpenMetadata, VirtualDataConnector
>
> This task delivers the JSON files for the new models that describe types for 
> Area 2 in the open metadata model. This area covers the glossary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1839) Area 2 of the open metadata model

[David Radley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-1839:

Attachment: (was: 0220CategoryHierarchy.json)

> Area 2 of the open metadata model
> -
>
> Key: ATLAS-1839
> URL: https://issues.apache.org/jira/browse/ATLAS-1839
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Mandy Chessell
>Assignee: Mandy Chessell
>  Labels: OpenMetadata, VirtualDataConnector
>
> This task delivers the JSON files for the new models that describe types for 
> Area 2 in the open metadata model. This area covers the glossary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (ATLAS-1768) Create common types for Open Metadata

[David Radley (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089941#comment-16089941
 ] 

David Radley edited comment on ATLAS-1768 at 7/17/17 4:32 PM:
--

[~mandy_chessell] I am looking at the glossary model and I notice that in model 
210-Glossary, the ExternalGlossaryLink has an atribute called source of type 
ExternalReference. Now that ExternalReference is an entity and not a structure, 
I suggest we change ExternalGlossaryLink  to subclass  ExternalReference and 
add the extra organization attribute.  

In model 220 LibraryCategoryReference is a relationship with an attribute 
called source of type  ExternalReference. We need to change this as 
relationshipDefs cannot have relationships. I suggest we explicitly add the 
attributes we need into this relationship. I notice the term version of this 
relationship is not consistent with this category one.   

I model 280 spine objects, there are 2 endpoints called attribute. I suggest we 
use plural names for all "SET" endpoints and have attributes and 
typedAttributes. 


was (Author: davidrad):
[~mandy_chessell] I am looking at the glossary model and I notice that in model 
210-Glossary, the ExternalGlossaryLink has an atribute called source of type 
ExternalReference. Now that ExternalReference is an entity and not a structure, 
I suggest we change ExternalGlossaryLink  to subclass  ExternalReference and 
add the extra organization attribute.  

In model 220 LibraryCategoryReference is a relationship with an attribute 
called source of type  ExternalReference. We need to change this as 
relationshipDefs cannot have relationships. I suggest we explicitly add the 
attributes we need into this relationship. I notice the term version of this 
relationship is not consistent with this category one.   


> Create common types for Open Metadata
> -
>
> Key: ATLAS-1768
> URL: https://issues.apache.org/jira/browse/ATLAS-1768
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Mandy Chessell
>Assignee: Mandy Chessell
>  Labels: OpenMetadata
>
> This JIRA describes a proposal for standard types for open metadata entities 
> and relationships.  For example, glossaries, database definitions, rules, 
> policies, ...
> The value of having standard definitions for metadata is to enable type safe 
> APIs and business level UIs plus be able to exchange metadata between 
> different instances of metadata repositories.
> The implementation of these common types is divided into 8 areas:
> * Area 0 - for extensions to Apache Atlas's base model
> * Area 1 - for definitions of the data-related assets we are governing and 
> using
> * Area 2 - for a glossary of meanings and semantic relationships
> * Area 3 - for information about asset use, crowd-sourced definitions and 
> collaboration around the data-related assets
> * Area 4 - for governance such as policies, rules and classifications
> * Area 5 - for reference models and reference data
> * Area 6 - for metadata discovery processes (see 
> https://issues.apache.org/jira/browse/ATLAS-1748)
> * Area 7 - for lineage
> Adaptation and flexibility are key in metadata environments so these common 
> definitions must be extensible - and we still need to support the ad hoc 
> definition of new types in Atlas.
> Apache Atlas supports meta-types that are used in the definition of new 
> types.  These are currently enumeration, struct, classification and entity.  
> JIRA https://issues.apache.org/jira/browse/ATLAS-1690 adds relationships to 
> this list.  The open metadata models make use of all of these meta-types.  
> These are represented by sterotypes on the classes of the open metadata 
> definitions.
> The Atlas wiki has the models as a set of linked pages which are probably the 
> easiest way to view the models.
> Start here: 
> https://cwiki.apache.org/confluence/display/ATLAS/Building+out+the+Apache+Atlas+Typesystem



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Comment Edited] (ATLAS-1768) Create common types for Open Metadata

[David Radley (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089941#comment-16089941
 ] 

David Radley edited comment on ATLAS-1768 at 7/17/17 3:19 PM:
--

[~mandy_chessell] I am looking at the glossary model and I notice that in model 
210-Glossary, the ExternalGlossaryLink has an atribute called source of type 
ExternalReference. Now that ExternalReference is an entity and not a structure, 
I suggest we change ExternalGlossaryLink  to subclass  ExternalReference and 
add the extra organization attribute.  

In model 220 LibraryCategoryReference is a relationship with an attribute 
called source of type  ExternalReference. We need to change this as 
relationshipDefs cannot have relationships. I suggest we explicitly add the 
attributes we need into this relationship. I notice the term version of this 
relationship is not consistent with this category one.   



was (Author: davidrad):
[~mandy_chessell] I am looking at the glossary model and I notice that in model 
210-Glossary, the ExternalGlossaryLink has an atribute called source of type 
ExternalReference. Now that ExternalReference is an entity and not a structure, 
I suggest we change ExternalGlossaryLink  to subclass  ExternalReference and 
add the extra organization attribute.  

> Create common types for Open Metadata
> -
>
> Key: ATLAS-1768
> URL: https://issues.apache.org/jira/browse/ATLAS-1768
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Mandy Chessell
>Assignee: Mandy Chessell
>  Labels: OpenMetadata
>
> This JIRA describes a proposal for standard types for open metadata entities 
> and relationships.  For example, glossaries, database definitions, rules, 
> policies, ...
> The value of having standard definitions for metadata is to enable type safe 
> APIs and business level UIs plus be able to exchange metadata between 
> different instances of metadata repositories.
> The implementation of these common types is divided into 8 areas:
> * Area 0 - for extensions to Apache Atlas's base model
> * Area 1 - for definitions of the data-related assets we are governing and 
> using
> * Area 2 - for a glossary of meanings and semantic relationships
> * Area 3 - for information about asset use, crowd-sourced definitions and 
> collaboration around the data-related assets
> * Area 4 - for governance such as policies, rules and classifications
> * Area 5 - for reference models and reference data
> * Area 6 - for metadata discovery processes (see 
> https://issues.apache.org/jira/browse/ATLAS-1748)
> * Area 7 - for lineage
> Adaptation and flexibility are key in metadata environments so these common 
> definitions must be extensible - and we still need to support the ad hoc 
> definition of new types in Atlas.
> Apache Atlas supports meta-types that are used in the definition of new 
> types.  These are currently enumeration, struct, classification and entity.  
> JIRA https://issues.apache.org/jira/browse/ATLAS-1690 adds relationships to 
> this list.  The open metadata models make use of all of these meta-types.  
> These are represented by sterotypes on the classes of the open metadata 
> definitions.
> The Atlas wiki has the models as a set of linked pages which are probably the 
> easiest way to view the models.
> Start here: 
> https://cwiki.apache.org/confluence/display/ATLAS/Building+out+the+Apache+Atlas+Typesystem



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1698) Create Glossary OMAS API

[David Radley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-1698:

Attachment: (was: apidocs-3.zip)

> Create Glossary OMAS API
> 
>
> Key: ATLAS-1698
> URL: https://issues.apache.org/jira/browse/ATLAS-1698
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: David Radley
>Assignee: David Radley
>  Labels: VirtualDataConnector
> Attachments: apidocs-4.zip, Atlas Glossary OMAS API proposal v1.0 
> .pdf, Atlas Glossary OMAS API proposal v1.1.pdf
>
>
> The Glossary OMAS provides a specialized API for glossary applications to 
> retrieve and store their glossary metadata and link assets of different types 
> to these glossary entries.
> The Glossary OMAS makes heavy use of the Area 2 open metadata model.  See 
> https://cwiki.apache.org/confluence/display/ATLAS/Area+2+-+Glossary



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (ATLAS-1698) Create Glossary OMAS API

[David Radley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-1698:

Attachment: apidocs-4.zip

Addressed review comments and changed to allow Atlas to start with this code. 
amended the object names to not include Atlas

> Create Glossary OMAS API
> 
>
> Key: ATLAS-1698
> URL: https://issues.apache.org/jira/browse/ATLAS-1698
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: David Radley
>Assignee: David Radley
>  Labels: VirtualDataConnector
> Attachments: apidocs-4.zip, Atlas Glossary OMAS API proposal v1.0 
> .pdf, Atlas Glossary OMAS API proposal v1.1.pdf
>
>
> The Glossary OMAS provides a specialized API for glossary applications to 
> retrieve and store their glossary metadata and link assets of different types 
> to these glossary entries.
> The Glossary OMAS makes heavy use of the Area 2 open metadata model.  See 
> https://cwiki.apache.org/confluence/display/ATLAS/Area+2+-+Glossary



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (ATLAS-1839) Area 2 of the open metadata model

[Israel Varea (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089814#comment-16089814
 ] 

Israel Varea commented on ATLAS-1839:
-

Hi all,

I have been looking at how open metadata model is being defined and I think 
this iniciative is awesome.

In particular, I would like to model some domain using this Area 2 definitions, 
but I can't find a proper way to do so, may be you can have some insights of 
how to do it.

I want to model a Person, so I would include Person as a Term that inherits 
from Catalog. Now I want to add some attributes (concepts) related to a Person, 
such as Person_Name, Person_Surname, Date_of_Birth, Email, so I add them as 
Terms too.
How should I relate these Terms as "attribute_of" Person? I see there is a 0250 
Related Terms section, however there is no relation to specify "attribute_of" 
or similar.

In this model, I would like to specify some features for Terms, such as 
validation/normalization, max and min values, etc. For example, I would like to 
specify that only a specific regex is valid for Emails, or Card_Numbers. 
How could I add these features for the Terms?

Thanks




> Area 2 of the open metadata model
> -
>
> Key: ATLAS-1839
> URL: https://issues.apache.org/jira/browse/ATLAS-1839
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 0.9-incubating
>Reporter: Mandy Chessell
>Assignee: Mandy Chessell
>  Labels: OpenMetadata, VirtualDataConnector
> Attachments: 0210Glossary.json, 0220CategoryHierarchy.json, 
> 0230Terms.json, 0240Dictionary.json, 0250RelatedTerms.json
>
>
> This task delivers the JSON files for the new models that describe types for 
> Area 2 in the open metadata model. This area covers the glossary.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)