Re: Review Request 61736: ATLAS-2049 Document common standards for OMAS interfaces

[Nigel Jones]

Hi all,
 We now have a number of OMAS APIs being designed and developed. There have 
been a number of discussions around best practices/standards, and the document 
below is an initial attempt to get these written down.

I'm sure this will be an evolving list, but it would be really useful to get 
the discussion flowing, so if anyone can spend the time to take a look it would 
be most appreciated.

This started as a list of notes for a discussion so it's far from complete and 
perhaps not detailed enough but we can start somewhere :-)

Note that it's created as a patch against the twiki docs, but we can also 
discuss where this documentation SHOULD go 

Thanks!
Nigel.


On 2017-08-18 15:55, Nigel Jones  wrote: 

> https://reviews.apache.org/r/61736/

> ATLAS-2049 Document common standards for OMAS interfaces
> 
> As we start to build OMAS interfaces it seems having some common standards 
> would be a good idea. I wanted to get this discussion going so opted to 
> document this in twiki format & submit via review board. The intent isn't 
> this first pass is correct, but that we come to a consensus ...

Re: Review Request 61736: ATLAS-2049 Document common standards for OMAS interfaces

[Nigel Jones]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61736/
---

(Updated Aug. 23, 2017, 2:25 p.m.)


Review request for atlas.


Changes
---

Updated with responses from Davids comments (except for common parms). Also 
review board should now be in sync with patch in JIRA


Repository: atlas


Description
---

ATLAS-2049 Document common standards for OMAS interfaces

As we start to build OMAS interfaces it seems having some common standards 
would be a good idea. I wanted to get this discussion going so opted to 
document this in twiki format & submit via review board. The intent isn't this 
first pass is correct, but that we come to a consensus ...

Open to suggestions as to whether this is a useful approach - Alternatives 
include
 - Simply posting on wiki - but I think we want to keep that for more consumer 
style documentation that's been agreed
 - Posting as a word doc or similar within the JIRA

See more info in the JIRA(s)

-- Nigel.


Diffs (updated)
-

  docs/src/site/twiki/OMAS-Standards.twiki PRE-CREATION 
  docs/src/site/twiki/index.twiki a8e7de9fb74bca0548eda5f5832e1e2bff3f7aec 


Diff: https://reviews.apache.org/r/61736/diff/2/

Changes: https://reviews.apache.org/r/61736/diff/1-2/


Testing
---

* Rebuilt twiki documentation to validate formatting, and reviewed in Safari.


Thanks,

Nigel Jones

Re: Review Request 61736: ATLAS-2049 Document common standards for OMAS interfaces

[Nigel Jones]

> On Aug. 18, 2017, 4:30 p.m., David Radley wrote:
> > docs/src/site/twiki/OMAS-Standards.twiki
> > Lines 9 (patched)
> > 
> >
> > I would not be this blunt. All of the existing applicaiton use the 
> > entity API. Instead, I suggest saying something like, 
> > " the OMAS APIs are access APIs that are not chatty and should be 
> > ideally tailors for consumers to access Atlas. As such we are looking to 
> > encourage Atlas application to move away from the lower level APIs and use 
> > the OMAS API instead.

I softened the statement slightly - though not as far as perhaps you suggested, 
to 

* Use of OMAS interfaces by applications is preferred over lower level Atlas 
interfaces

 Explanation: Whilst all existing applications use the existing Atlas API, 
the intent of OMAS is to provide a less chatty, tailored, set of interfaces 
that provide all the interactions applications need.

Is that better? I was trying to make it a specific actionable statement - I do 
think it's an important principle. We are aiming to provide full coverage 
across the suite of OMAS interfaces so that all atlas applications can find the 
function they need in an efficient form. Only when we've achieved this and have 
a critical mass could we even contemplate deprecation as there are many 
applications out there already - so this is more about best practices moving 
forward... but getting that transition started. If it's not felt we could do 
this, then why, does it mean we've missed something? Is the strategy wrong?


> On Aug. 18, 2017, 4:30 p.m., David Radley wrote:
> > docs/src/site/twiki/OMAS-Standards.twiki
> > Lines 12 (patched)
> > 
> >
> > I think you need to explain what OMRS and OMRS connectors are).

I removed this statement. However the document does still refer to OMAS, OMRS 
so the point is valid. I added an initial statement

Further background in relation to Open Metadata including explanations of OMAS, 
OMRS, may be found in the Atlas Wiki at 
https://cwiki.apache.org/confluence/display/ATLAS/Open+Metadata+and+Governance


> On Aug. 18, 2017, 4:30 p.m., David Radley wrote:
> > docs/src/site/twiki/OMAS-Standards.twiki
> > Lines 16 (patched)
> > 
> >
> > When you say "transforming, mapping data from Atlas/OMRS as required" - 
> > I suggest saying the OMAS implmentation will call the lower level resource 
> > specific APIs. I suggest not mentioning that the OMAS call Atlas - it calls 
> > OMRS which could be Atlas behind it.

I will change it to refer to OMRS only, since as you say, we could be talking 
to a non-Atlas based OMRS implementation. By definition (and given the link to 
the wiki above) these are lower level so I think just stopping there makes 
sense?

It now reads

 Explanation: OMAS interfaces are targetted at a particular type of 
consumer with the objective of making it easy for that consumer - transforming, 
mapping data from OMRS as required


> On Aug. 18, 2017, 4:30 p.m., David Radley wrote:
> > docs/src/site/twiki/OMAS-Standards.twiki
> > Lines 20 (patched)
> > 
> >
> > what do you mean by "some duplication is expected"?

I mean that the same, or very similar capability may surface through two 
different OMASs. I think this is inevitable since by definition the OMASs are 
consumer centric, and there will be some things that two different types of 
consumers may need. The chances are the information will be packaged slightly 
differently.

It;s certainly possible some applications could use multiple OMAS interfaces, 
but I think it's generally better if the functionality provided on an OMAS is 
closely tied to the type of application using it. For example a governance 
engine may need notification when an asset classification changes, so you might 
find that notification on the Governance engine omas topic, but also that same 
notification may be seen on the asset omas too, which could be monitored by the 
virtualizer. That to me makes sense ...

I added this statement to the text:

For example a change in an asset classification may be sent out through both 
the governance engine OMAS and the Asset OMAS.


> On Aug. 18, 2017, 4:30 p.m., David Radley wrote:
> > docs/src/site/twiki/OMAS-Standards.twiki
> > Lines 22 (patched)
> > 
> >
> > I think of the interface as the API - maybe we should say the 
> > implmnetaion of the OMAS interface or something like that.

Yes agreed, bad terminology from me there.

Updated to:
* OMAS implementations will use OMRS to manage the underlaying metadata (other 
than in any transitionary phase).


> On Aug. 18, 2017, 4:30 p.m., David Radley wrote:
> > 

Re: Review Request 61736: ATLAS-2049 Document common standards for OMAS interfaces

[David Radley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61736/#review183222
---




docs/src/site/twiki/OMAS-Standards.twiki
Lines 9 (patched)


I would not be this blunt. All of the existing applicaiton use the entity 
API. Instead, I suggest saying something like, 
" the OMAS APIs are access APIs that are not chatty and should be ideally 
tailors for consumers to access Atlas. As such we are looking to encourage 
Atlas application to move away from the lower level APIs and use the OMAS API 
instead.



docs/src/site/twiki/OMAS-Standards.twiki
Lines 12 (patched)


I think you need to explain what OMRS and OMRS connectors are).



docs/src/site/twiki/OMAS-Standards.twiki
Lines 16 (patched)


When you say "transforming, mapping data from Atlas/OMRS as required" - I 
suggest saying the OMAS implmentation will call the lower level resource 
specific APIs. I suggest not mentioning that the OMAS call Atlas - it calls 
OMRS which could be Atlas behind it.



docs/src/site/twiki/OMAS-Standards.twiki
Lines 20 (patched)


what do you mean by "some duplication is expected"?



docs/src/site/twiki/OMAS-Standards.twiki
Lines 22 (patched)


I think of the interface as the API - maybe we should say the implmnetaion 
of the OMAS interface or something like that.



docs/src/site/twiki/OMAS-Standards.twiki
Lines 26 (patched)


You talk of OMAS interfaces and OMAS interface. Is there one or many. Do we 
require separate OMAS build for each OMAS?



docs/src/site/twiki/OMAS-Standards.twiki
Lines 42 (patched)


you refer to the connector OMAS. In this context how would I find that? Do 
you actually mean the asset OMAS to the connector framework?



docs/src/site/twiki/OMAS-Standards.twiki
Lines 46 (patched)


tab



docs/src/site/twiki/OMAS-Standards.twiki
Lines 50 (patched)


you mention Atlas here - when it might not be Atlas



docs/src/site/twiki/OMAS-Standards.twiki
Lines 135 (patched)


tab



docs/src/site/twiki/index.twiki
Lines 61 (patched)


Might be less ambiguous to remove Atlas from the title.


- David Radley


On Aug. 18, 2017, 2:55 p.m., Nigel Jones wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61736/
> ---
> 
> (Updated Aug. 18, 2017, 2:55 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> ATLAS-2049 Document common standards for OMAS interfaces
> 
> As we start to build OMAS interfaces it seems having some common standards 
> would be a good idea. I wanted to get this discussion going so opted to 
> document this in twiki format & submit via review board. The intent isn't 
> this first pass is correct, but that we come to a consensus ...
> 
> Open to suggestions as to whether this is a useful approach - Alternatives 
> include
>  - Simply posting on wiki - but I think we want to keep that for more 
> consumer style documentation that's been agreed
>  - Posting as a word doc or similar within the JIRA
> 
> See more info in the JIRA(s)
> 
> -- Nigel.
> 
> 
> Diffs
> -
> 
>   docs/src/site/twiki/OMAS-Standards.twiki PRE-CREATION 
>   docs/src/site/twiki/index.twiki a8e7de9fb74bca0548eda5f5832e1e2bff3f7aec 
> 
> 
> Diff: https://reviews.apache.org/r/61736/diff/1/
> 
> 
> Testing
> ---
> 
> * Rebuilt twiki documentation to validate formatting, and reviewed in Safari.
> 
> 
> Thanks,
> 
> Nigel Jones
> 
>