Re: [VOTE] Graduate Apache Atlas Project from Incubator

2017-05-24 Thread Srikanth Venkat 
+1 (non-binding)
Huge thanks to Atlas community for your wonderful support and collaboration 
over the past years!

Srikanth Venkat | Senior Director, Product Management | 
Hortonworks Inc 
 

On 5/24/17, 4:23 PM, "Suma Shivaprasad" <sumasai.shivapra...@gmail.com> wrote:

Dear Atlas Community Members,

Following the discussion in Atlas community on graduating Apache Atlas
Project from Incubation to a Top Level Project, starting this
voting thread within the Atlas community. I appreciate mentors and
community members in making this happen and thanks for their
support. Apache Atlas Project has come a long way since entering into
incubation during May 2015 and the project summary shows the
details on this.

Please vote on the Apache Atlas Project resolution that is found in
bottom of this email.

[ ] +1 Graduate Apache Atlas from the Incubator.
[ ] +0 No opinion
[ ] -1 Don't graduate Apache Atlas from the Incubator ( please provide
the reason)

This VOTE will be opened for next 72 hours.

If this VOTE succeeds, a similar VOTE will be done in general@incubator
mailing list. If that succeeds, a resolution will be
included in the next Apache Board Meeting.

Here is my Vote +1 (binding)

Thanks all for your participation
Suma


Project Summary:

=
http://incubator.apache.org/projects/atlas.html

Project website:
=

http://atlas.incubator.apache.org

Project Documentation:
===

http://atlas.incubator.apache.org/index.html
Architecture - http://atlas.incubator.apache.org/Architecture.html
Installation guide - http://atlas.incubator.apache.org/InstallationSteps.
html
QuickStart Guide- http://atlas.incubator.apache.org/QuickStart.html
REST API - http://atlas.incubator.apache.org/api/v2/index.html
Atlas Release Guide - https://cwiki.apache.org/confluence/display/ATLAS/
Developer+Resources

Project maturity Assessment:
===

https://cwiki.apache.org/confluence/display/ATLAS/
Apache+Atlas+Project+Maturity+Model

Proposed PMC size: 38

Total number of committers  : 38 members
Total number of contributors : 62 members

PMC affiliation (* indicated chair)

* Hortonworks (13)
   IBM (3)
   Aetna(4)
   Freestone (3)
   Merck (2)
   Oracle(2)
   Target (2)
   Schlumberger(2)
   Dataguise(1)
   Inmobi(1)
   JPMC(1)
   MPR(1)
   SAP(1)
   SparklineData(1)
   Others(1)

1660 commits on master
80 contributors across all branches
Dev list averaged ~500-800 msgs/month in 2017
User list was created recently in April 2017. Hence not considering stats
for this.
1827 issues created

Committer¹s affiliation:
===
* Hortonworks (13)
   IBM (3)
   Aetna(4)
   Freestone (3)
   Merck (2)
   Oracle(2)
   Target (2)
   Schlumberger(2)
   Dataguise(1)
   Inmobi(1)
   JPMC(1)
   MPR(1)
   SAP(1)
   SparklineData(1)
   Others(1)


Apache Atlas Top Level Project Resolution:


Establish the Apache Atlas Project

WHEREAS, the Board of Directors deems it to be in the best interests of
the Foundation and consistent with the Foundation¹s purpose to establish a
Project Management Committee charged with the creation and maintenance of
open-source software, for distribution at no charge to the public, related
to a data management platform That provides real-time, consistent access
to data-intensive applications throughout widely distributed cloud
architectures.

NOW, THEREFORE, BE IT RESOLVED, that a Project Management Committee
(PMC), to be known as the "Apache Atlas Project", be and hereby is
established pursuant to Bylaws of the Foundation; and be it further
RESOLVED,that the Apache Atlas Project be and hereby is responsible for
the creation and maintenance of software related to a data management
platform that provides real-time, consistent access to data-intensive
applications throughout widely distributed cloud architectures.

RESOLVED, that the office of "Vice President, Apache Atlas" be and
hereby is created, the person holding such office to serve at the
direction of the Board of Directors as the chair of the Apache Atlas
Project, and to have primary responsibility for management of the projects
within the scope of responsibility of the Apache Atlas Project; and be it
Further.

>RESOLVED,that the persons listed immediately below be and hereby are
>appointed to serve as the initial members of the Apache Atlas Project:

Aaron Dossett adoss...@target.c

[jira] [Updated] (ATLAS-1822) Search Query Builder and Faceted Search in Atlas

2017-05-19 Thread Srikanth Venkat (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Srikanth Venkat updated ATLAS-1822:
---
Summary: Search Query Builder and Faceted Search in Atlas  (was: 
QueryBuilder and Faceted Search in Atlas)

> Search Query Builder and Faceted Search in Atlas
> 
>
> Key: ATLAS-1822
> URL: https://issues.apache.org/jira/browse/ATLAS-1822
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core, atlas-webui
>        Reporter: Srikanth Venkat
>
> Current search experience in Atlas UI is pretty basic and any advanced 
> searches require knowledge of DSL that might be difficult for business users.
> As a data steward or data consumer, I need an easy and intuitive way to 
> perform faceted searches on entities, classifications, relationships, and 
> business glossaries and filter the results based on a combination of 
> parameters so I can navigate and locate assets for curation and in order to 
> take any governance actions. I also need to be able to save my searches and 
> filters and rerun them dynamically against the latest state of the enterprise 
> metadata catalog in Atlas.
> Implementation considerations:
> # For business users, there should be default filters provided based on 
> commonly used criteria in the search experience
> # Any search results should only show metadata of data assets that user has 
> permission to view, based upon authorization policies (via Ranger)
> # Search facets need to be based on search conditions and should allow for 
> sequence of AND, OR and NOT operations with multiple search conditions
> # Search facets details (should allow for combinations of any or all of the 
> following):
> * Asset type (entity types)
> * Attribute value checker (should allow for combinations of range checks, 
> ordinality, exact match, numeric, string and date comparisons, single or 
> muliple values, category checks, nulls)
> * Created time
> * Updated time
> * Size (where relevant for e.g. HDFS files)
> * Deleted/Active status
> * Owner
> * Custom property values
> * File format
> * Asset name
> * Physical storage context: Datalake/cluster or datacenter holding the asset
> * Classification
> * Glossary/taxonomy association
> Note:
> # Searches should be able to be saved and scoped to a particular business 
> catalog or asset type or namespace (when multi-tenancy is supported by Atlas 
> in the future) 
> # Once search results are available, user should be able to filter by any of 
> the facets.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ATLAS-1822) QueryBuilder and Faceted Search in Atlas

2017-05-19 Thread Srikanth Venkat (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Srikanth Venkat updated ATLAS-1822:
---
Description: 
Current search experience in Atlas UI is pretty basic and any advanced searches 
require knowledge of DSL that might be difficult for business users.

As a data steward or data consumer, I need an easy and intuitive way to perform 
faceted searches on entities, classifications, relationships, and business 
glossaries and filter the results based on a combination of parameters so I can 
navigate and locate assets for curation and in order to take any governance 
actions. I also need to be able to save my searches and filters and rerun them 
dynamically against the latest state of the enterprise metadata catalog in 
Atlas.
Implementation considerations:
# For business users, there should be default filters provided based on 
commonly used criteria in the search experience
# Any search results should only show metadata of data assets that user has 
permission to view, based upon authorization policies (via Ranger)
# Search facets need to be based on search conditions and should allow for 
sequence of AND, OR and NOT operations with multiple search conditions
# Search facets details (should allow for combinations of any or all of the 
following):
* Asset type (entity types)
* Attribute value checker (should allow for combinations of range checks, 
ordinality, exact match, numeric, string and date comparisons, single or 
muliple values, category checks, nulls)
* Created time
* Updated time
* Size (where relevant for e.g. HDFS files)
* Deleted/Active status
* Owner
* Custom property values
* File format
* Asset name
* Physical storage context: Datalake/cluster or datacenter holding the asset
* Classification
* Glossary/taxonomy association

Note:
# Searches should be able to be saved and scoped to a particular business 
catalog or asset type or namespace (when multi-tenancy is supported by Atlas in 
the future) 
# Once search results are available, user should be able to filter by any of 
the facets.

  was:
Current search experience in Atlas UI is pretty basic and any advanced searches 
require knowledge of DSL that might be difficult for business users.

As a data steward or data consumer, I need an easy and intuitive way to perform 
faceted searches on entities, classifications, relationships, and business 
glossaries and filter the results based on a combination of parameters so I can 
navigate and locate assets for curation and in order to take any governance 
actions. I also need to be able to save my searches and filters and rerun them 
dynamically against the latest state of the enterprise metadata catalog in 
Atlas.
Implementation considerations:
# For business users, there should be default filters provided based on 
commonly used criteria in the search experience
# Any search results should only show metadata of data assets that user has 
permission to view, based upon authorization policies (via Ranger)
# Search facets need to be based on search conditions and should allow for 
sequence of AND, OR and NOT operations with multiple search conditions
# Search facets (should allow for combinations):
* Asset type (entity types)
* Attribute value checker (should allow for combinations of range checks, 
ordinality, exact match, numeric, string and date comparisons, single or 
muliple values, category checks, nulls)
* Created time
* Updated time
* Size (where relevant for e.g. HDFS files)
* Deleted/Active status
* Owner
* Custom property values
* File format
* Asset name
* Physical storage context: Datalake/cluster or datacenter holding the asset
* Classification
* Glossary/taxonomy association

Note:
# Searches should be able to be saved and scoped to a particular business 
catalog or asset type or namespace (when multi-tenancy is supported by Atlas in 
the future) 
# Once search results are available, user should be able to filter by any of 
the facets.


> QueryBuilder and Faceted Search in Atlas
> 
>
> Key: ATLAS-1822
> URL: https://issues.apache.org/jira/browse/ATLAS-1822
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core, atlas-webui
>        Reporter: Srikanth Venkat
>
> Current search experience in Atlas UI is pretty basic and any advanced 
> searches require knowledge of DSL that might be difficult for business users.
> As a data steward or data consumer, I need an easy and intuitive way to 
> perform faceted searches on entities, classifications, relationships, and 
> business glossaries and filter the results based on a combination of 
> parameters so I can navigate and locate assets for curation and in order to 
> take any governance actions. I also need to be able to save my searches and 
> filte

[jira] [Updated] (ATLAS-1822) QueryBuilder and Faceted Search in Atlas

2017-05-19 Thread Srikanth Venkat (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Srikanth Venkat updated ATLAS-1822:
---
Description: 
Current search experience in Atlas UI is pretty basic and any advanced searches 
require knowledge of DSL that might be difficult for business users.

As a data steward or data consumer, I need an easy and intuitive way to perform 
faceted searches on entities, classifications, relationships, and business 
glossaries and filter the results based on a combination of parameters so I can 
navigate and locate assets for curation and in order to take any governance 
actions. I also need to be able to save my searches and filters and rerun them 
dynamically against the latest state of the enterprise metadata catalog in 
Atlas.
Implementation considerations:
# For business users, there should be default filters provided based on 
commonly used criteria in the search experience
# Any search results should only show metadata of data assets that user has 
permission to view, based upon authorization policies (via Ranger)
# Search facets need to be based on search conditions and should allow for 
sequence of AND, OR and NOT operations with multiple search conditions
# Search facets (should allow for combinations):
* Asset type (entity types)
* Attribute value checker (should allow for combinations of range checks, 
ordinality, exact match, numeric, string and date comparisons, single or 
muliple values, category checks, nulls)
* Created time
* Updated time
* Size (where relevant for e.g. HDFS files)
* Deleted/Active status
* Owner
* Custom property values
* File format
* Asset name
* Physical storage context: Datalake/cluster or datacenter holding the asset
* Classification
* Glossary/taxonomy association

Note:
# Searches should be able to be saved and scoped to a particular business 
catalog or asset type or namespace (when multi-tenancy is supported by Atlas in 
the future) 
# Once search results are available, user should be able to filter by any of 
the facets.

  was:
Current search experience in Atlas UI is pretty basic and any advanced searches 
require knowledge of DSL that might be difficult for business users.

As a data steward or data consumer, I need an easy and intuitive way to perform 
faceted searches on entities, classifications, relationships, and business 
glossaries and filter the results based on a combination of parameters so I can 
navigate and locate assets for curation and in order to take any governance 
actions. I also need to be able to save my searches and filters and rerun them 
dynamically against the latest state of the enterprise metadata catalog in 
Atlas.
Implementation considerations:
# For business users, there should be default filters provided based on 
commonly used criteria in the search experience
# Any search results should only show metadata of data assets that user has 
permission to view, based upon authorization policies (via Ranger)
# Search facets need to be based on search conditions and should allow for 
sequence of AND, OR and NOT operations with multiple search conditions
# Search facets (should allow for combinations):
* Asset type (entity types)
* Attribute value checker (should allow for combinations of range checks, 
ordinality, exact match, numeric, string and date comparisons, single or 
muliple values, category checks, nulls)
* Created time
* Updated time
* Size (where relevant for e.g. HDFS files)
* Deleted/Active status
* Owner
* Custom property values
* File format
* Asset name
* Physical storage context: Datalake/cluster or datacenter holding the asset
* Classification
* Glossary/taxonomy association

Note:
# Searches should be able to be saved and scoped to a particular business 
catalog or asset type or tenant/namespace (when this is future) 
# Once search results are available, user should be able to filter by any of 
the facets.


> QueryBuilder and Faceted Search in Atlas
> 
>
> Key: ATLAS-1822
> URL: https://issues.apache.org/jira/browse/ATLAS-1822
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core, atlas-webui
>        Reporter: Srikanth Venkat
>
> Current search experience in Atlas UI is pretty basic and any advanced 
> searches require knowledge of DSL that might be difficult for business users.
> As a data steward or data consumer, I need an easy and intuitive way to 
> perform faceted searches on entities, classifications, relationships, and 
> business glossaries and filter the results based on a combination of 
> parameters so I can navigate and locate assets for curation and in order to 
> take any governance actions. I also need to be able to save my searches and 
> filters and rerun them dynamically against the latest state of the ent

[jira] [Updated] (ATLAS-1822) QueryBuilder and Faceted Search in Atlas

2017-05-19 Thread Srikanth Venkat (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Srikanth Venkat updated ATLAS-1822:
---
Description: 
Current search experience in Atlas UI is pretty basic and any advanced searches 
require knowledge of DSL that might be difficult for business users.

As a data steward or data consumer, I need an easy and intuitive way to perform 
faceted searches on entities, classifications, relationships, and business 
glossaries and filter the results based on a combination of parameters so I can 
navigate and locate assets for curation and in order to take any governance 
actions. I also need to be able to save my searches and filters and rerun them 
dynamically against the latest state of the enterprise metadata catalog in 
Atlas.
Implementation considerations:
# For business users, there should be default filters provided based on 
commonly used criteria in the search experience
# Any search results should only show metadata of data assets that user has 
permission to view, based upon authorization policies (via Ranger)
# Search facets need to be based on search conditions and should allow for 
sequence of AND, OR and NOT operations with multiple search conditions
# Search facets (should allow for combinations):
* Asset type (entity types)
* Attribute value checker (should allow for combinations of range checks, 
ordinality, exact match, numeric, string and date comparisons, single or 
muliple values, category checks, nulls)
* Created time
* Updated time
* Size (where relevant for e.g. HDFS files)
* Deleted/Active status
* Owner
* Custom property values
* File format
* Asset name
* Physical storage context: Datalake/cluster or datacenter holding the asset
* Classification
* Glossary/taxonomy association

Note:
# Searches should be able to be saved and scoped to a particular business 
catalog or asset type or tenant/namespace (when this is future) 
# Once search results are available, user should be able to filter by any of 
the facets.

  was:
Current search experience in Atlas UI is pretty basic and any advanced searches 
require knowledge of DSL that might be difficult for business users.

As a data steward or data consumer, I need an easy and intuitive way to perform 
faceted searches on entities, classifications, relationships, and business 
glossaries and filter the results based on a combination of parameters so I can 
navigate and locate assets for curation and in order to take any governance 
actions. I also need to be able to save my searches and filters and rerun them 
dynamically against the latest state of the enterprise metadata catalog in 
Atlas.
Implementation considerations:
# For business users, there should be default filters provided based on 
commonly used criteria in the search experience
# Any search results should only show metadata of data assets that user has 
permission to view, based upon authorization policies (via Ranger)
# Search facets need to be based on search conditions and should allow for 
sequence of AND, OR and NOT operations with multiple search conditions
# Search facets (should allow for combinations):
* Asset type (entity types)
* Attribute value checker (should allow for combinations of range checks, 
ordinality, exact match, numeric, string and date comparisons, single or 
muliple values, category checks, nulls)
* Created time
* Updated time
* Size (where relevant for e.g. HDFS files)
* Deleted/Active status
* Owner
* Custom property values
* File format
* Asset name
* Physical storage context: Datalake/cluster or datacenter holding the asset
* Classification
* Glossary/taxonomy association

# Searches should be able to be saved and scoped to a particular business 
catalog or asset type 

# Once search results are available, user should be able to filter by any of 
facets.


> QueryBuilder and Faceted Search in Atlas
> 
>
> Key: ATLAS-1822
> URL: https://issues.apache.org/jira/browse/ATLAS-1822
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core, atlas-webui
>        Reporter: Srikanth Venkat
>
> Current search experience in Atlas UI is pretty basic and any advanced 
> searches require knowledge of DSL that might be difficult for business users.
> As a data steward or data consumer, I need an easy and intuitive way to 
> perform faceted searches on entities, classifications, relationships, and 
> business glossaries and filter the results based on a combination of 
> parameters so I can navigate and locate assets for curation and in order to 
> take any governance actions. I also need to be able to save my searches and 
> filters and rerun them dynamically against the latest state of the enterprise 
> metadata catalog in Atlas.
> Implementation considerations:
> # For

[jira] [Created] (ATLAS-1822) QueryBuilder and Faceted Search in Atlas

2017-05-19 Thread Srikanth Venkat (JIRA) 
Srikanth Venkat created ATLAS-1822:
--

 Summary: QueryBuilder and Faceted Search in Atlas
 Key: ATLAS-1822
 URL: https://issues.apache.org/jira/browse/ATLAS-1822
 Project: Atlas
  Issue Type: Improvement
  Components:  atlas-core, atlas-webui
Reporter: Srikanth Venkat


Current search experience in Atlas UI is pretty basic and any advanced searches 
require knowledge of DSL that might be difficult for business users.

As a data steward or data consumer, I need an easy and intuitive way to perform 
faceted searches on entities, classifications, relationships, and business 
glossaries and filter the results based on a combination of parameters so I can 
navigate and locate assets for curation and in order to take any governance 
actions. I also need to be able to save my searches and filters and rerun them 
dynamically against the latest state of the enterprise metadata catalog in 
Atlas.
Implementation considerations:
# For business users, there should be default filters provided based on 
commonly used criteria in the search experience
# Any search results should only show metadata of data assets that user has 
permission to view, based upon authorization policies (via Ranger)
# Search facets need to be based on search conditions and should allow for 
sequence of AND, OR and NOT operations with multiple search conditions
# Search facets (should allow for combinations):
* Asset type (entity types)
* Attribute value checker (should allow for combinations of range checks, 
ordinality, exact match, numeric, string and date comparisons, single or 
muliple values, category checks, nulls)
* Created time
* Updated time
* Size (where relevant for e.g. HDFS files)
* Deleted/Active status
* Owner
* Custom property values
* File format
* Asset name
* Physical storage context: Datalake/cluster or datacenter holding the asset
* Classification
* Glossary/taxonomy association

# Searches should be able to be saved and scoped to a particular business 
catalog or asset type 

# Once search results are available, user should be able to filter by any of 
facets.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ATLAS-1821) Classification propagation from entity to a derivative or child entity

2017-05-19 Thread Srikanth Venkat (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Srikanth Venkat updated ATLAS-1821:
---
Description: 
User Story:
As a data steward, I need a scalable way to quickly and efficiently propagate 
classification across the information supply chain to support efficient 
searches and classification based security for compliance and audit purposes. 
This requires:
1. Classifications for derivative entities should be inherited from the 
originator and to child entities from parent. 
For example, if a Hive column is classified "Confidential" then resulting 
column created from a CTAS operation should also be tagged "Confidential" to 
maintain the classification of the original entity. In the case where 2 or more 
entities are composed, the derivative entity should have the union of all 
classifications of each source entity.
2. Business Terms:
a. Child business terms should inherit the classifications associated with the 
parent term.
b. The option to propagate classification to child business terms in a 
hierarchy should be provided
c. Ability to update the propagated tags manually via UI or through the API
d. Tagging a term should propagate to data assets that are already attached to 
that business term as well
3. Data assets
a. For all supported data asset types in Atlas, if a derivative asset is 
created it should inherit the tags and attributes from the original asset.
b. the option to propagate tags to child entities should be provided (e.g. if 
you tag a folder in HDFS optionally tag all the files within it)
c. Ability to update the propagated tags manually via UI or through the API
d. Tagging a parent object should be inherited after child creation dynamically 
(unless a flag is set not to do this)
e. Derived data assets should have the tags of the original data asset.
Conflict resolution - if there are different values for attributes on tags 
(classifications) on upstream or parent entities used to derive a data asset 
then user needs to be prompted for action to resolve the conflict. Once 
resolved, the resolved value should be carried forth to derived assets.

  was:
User Story:
As a data steward, I need a scalable way to quickly and efficiently propagate 
classification across the information supply chain to support efficient 
searches and classification based security for compliance and audit puproses. 
Classifications for derivative entities should be inherited from the originator 
and to child entities from parent. For example, if a Hive column is classified 
"Confidential" then resulting column created from a CTAS operation should also 
be tagged "Confidential" to maintain the classification of the original entity. 
In the case where 2 or more entities are composed, the derivative entiity 
should have the union of all classifications of each source entity.
Business Terms:
Child business terms should inherit the classifications associated with the 
parent term.
The option to propagate classification to child business terms in a hierarchy 
should be provided
Ability to update the propagated tags manually via UI or through the API
Tagging a term should propagate to data assets that are already attached to 
that business term as well
Data assets
For all supported data asset types in Atlas, if a derivative asset is created 
it should inherit the tags and attributes from the original asset.
the option to propagate tags to child entities should be provided (e.g. if you 
tag a folder in HDFS optionally tag all the files within it)
Ability to update the propagated tags manually via UI or through the API
Tagging a parent object should be inherited after child creation dynamically 
(unless a flag is set not to do this)
Derived data assets should have the tags of the original data asset.
Conflict resolution - if there are different values for attributes on tags 
(classifications) then a dialog user needs to be prompted for action to resolve 
the conflict. Once resolved, the resolved value will be carried forth to 
derivative assets.


> Classification propagation from entity to a derivative or child entity
> --
>
> Key: ATLAS-1821
> URL: https://issues.apache.org/jira/browse/ATLAS-1821
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core, atlas-webui
>Reporter: Srikanth Venkat
> Fix For: 0.9-incubating
>
>
> User Story:
> As a data steward, I need a scalable way to quickly and efficiently propagate 
> classification across the information supply chain to support efficient 
> searches and classification based security for compliance and audit purposes. 
> This requires:
> 1. Classifications for derivative entities should be inheri

Re: [DISCUSS] Ready to graduate?

2017-05-02 Thread Srikanth Venkat 
+1 

Atlas community is active, engaged, and spans organizations, continents, and 
time zones. It is exciting to contribute and be part of this active community 
and I fully support TLP graduation!

I look forward to ongoing collaboration with the Atlas community to further the 
capabilities and make Atlas the open metadata and governance standard.

Srikanth Venkat | Senior Director, Product Management | 
sven...@hortonworks.com
Hortonworks Inc.
 

On 4/28/17, 7:06 AM, "Helvoirt, S.A.G. van (Stefhan)" 
<stefhan.van.helvo...@ing.nl.INVALID> wrote:

+1 From me, really pleased with the recent activities, discussions and 
engagement to mature Atlas.  

Met vriendelijke groet / Kind regard,
Stefhan van Helvoirt


ING Nederland
DB/COO/KIM Tr/CJ Data Governance/CJE
ACT C 03.260, Bijlmerdreef 24
Postbus 1800, 1000 BV Amsterdam
M 06 51 21 59 08 
E stefhan.van.helvo...@ing.nl 

-Original Message-
From: Sarath Subramanian [mailto:sar...@apache.org] 
Sent: donderdag 27 april 2017 8:45
To: dev@atlas.incubator.apache.org
Subject: Re: [DISCUSS] Ready to graduate?

+1 from me.

I see no concerns in Atlas graduating to a top-level project. The community 
has been buzzing with lot of activities and active engagements from people 
across organizations and time zones.


Thanks,
Sarath Subramanian

On Wed, Apr 26, 2017 at 11:30 PM, Ayub Khan Pathan <apat...@hortonworks.com>
wrote:

> +1 for graduation.
>
> The atlas community is very much active and evident by the phenomenal 
> growth & customer acceptance.
>
> Thanks
> Ayub Khan
>
> On 4/27/17, 10:41 AM, "Vimal Sharma" <visha...@hortonworks.com> wrote:
>
> +1 for graduation.
>
> Atlas community has been a pleasure to work with and we have 
> rapidly developed new features and improvements to the project.
>
> Thanks
> Vimal
>
>
>
>
> On 4/27/17, 9:47 AM, "Hemanth Yamijala" 
> <hyamij...@hortonworks.com>
> wrote:
>
> >+1 from me as well.
> >
> >It has been a pleasure  being involved with Atlas and see the 
> product and community grow.
> >
> >Thanks, Chris, for starting this thread.
> >
> >Hemanth
> >
> >From: Jakob Homan <jgho...@gmail.com>
> >Sent: Thursday, April 27, 2017 4:27 AM
> >To: dev@atlas.incubator.apache.org
> >Subject: Re: [DISCUSS] Ready to graduate?
> >
> >+1.
> >
> >I've not been very involved with the day-to-day bits of the project 
as
> >Mentor, so when the new PMC is constituted, there's no need to 
include
> >me.  But I'll stay a fan of Atlas after graduation.
> >
> >-Jakob
> >
> >On 26 April 2017 at 15:29, Suma Shivaprasad
> ><sumasai.shivapra...@gmail.com> wrote:
> >> +1. Agreed
> >>
> >> The community is really vibrant/active with many companies 
> involved in
> >> shaping the roadmap and features.
> >>  It has also grown a lot from past year with a lot of interest 
> from users
> >> and this is a great time to propose for graduation
> >>
> >> I volunteer to start working on the graduation and take this 
> forward if no
> >> one else has any objections.
> >>
> >> Thanks
> >> Suma
> >>
> >>
> >>
> >>
> >>
> >> On Wed, Apr 26, 2017 at 2:03 AM, Nigel L Jones 
> <jon...@uk.ibm.com>
> wrote:
> >>
> >>>
> >>>
> >>> +1 from me.
> >>>
> >>> The Atlas community is vibrant, we have engagement from 
> multiple companies,
> >>> and the industry interest around governance is immense, as 
> witnessed at
> >>> recent conferences. It's exciting to see it evolve and getting 
> it into a
> >>> TLP will be another tick in the box as seen by potential users.
> >>>
> >>> Let's go for it!
> >>>
> >>> -Original Message-
> >>> From: Mandy Chessell [mailto:mandy_chess...@uk.ibm.com]

[jira] [Commented] (ATLAS-1186) Add Glossary Category

2017-02-03 Thread Srikanth Venkat (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852648#comment-15852648
 ] 

Srikanth Venkat commented on ATLAS-1186:


[~davidrad] Do you have any comments on this? It would be helpful if you can 
share your feedback on my questions above.

> Add Glossary Category
> -
>
> Key: ATLAS-1186
> URL: https://issues.apache.org/jira/browse/ATLAS-1186
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 0.8-incubating
>Reporter: David Radley
>Assignee: David Radley
>  Labels: features
> Fix For: 0.8-incubating
>
> Attachments: rb54430(2).patch
>
>
> Add Glossary Category, which would have a name and description and be hung 
> off a taxonomy. The category would contain (composition type) sub categories. 
> Categories can contain terms.  



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ATLAS-1114) Performance improvements for create/update entities

2017-01-26 Thread Srikanth Venkat (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15840088#comment-15840088
 ] 

Srikanth Venkat commented on ATLAS-1114:


Jeff,

It would be useful if along with the patch you can provide test cases or 
scenarios that were used to identify this issue and can be used to baseline and 
can be used to validate/benchmark improvements in these areas moving forward.

> Performance improvements for create/update entities
> ---
>
> Key: ATLAS-1114
> URL: https://issues.apache.org/jira/browse/ATLAS-1114
> Project: Atlas
>  Issue Type: Bug
>Reporter: Jeffrey Hagelberg
>Assignee: Jeffrey Hagelberg
> Attachments: rb51092.patch
>
>
> General Atlas Perforance Improvements
> Apply performance fixes for create/update entities from IBM fork to Atlas.  
> During our performance profiling, we found a number of performance hotspots 
> in JProfiler.  Our main findings were
>- multiple queries were being executed for each instance being 
> created/updated to find matches by unique attribute.  
>   - one query was being executed for each instance being created/updated to 
> find the corresponding vertex if there is one
>   - Calculating the value of the full text property was taking a significant 
> portion of the time to create/update entities, mainly due to its calls to 
> getVertexForGUID
> The changes we put in do the following:
> - batch lookups by guid when create/update entities.  Execute one 
> AtlasGraphQuery to find them all.
> - batch lookups by unique attribute when create/update entities.  Execute one 
> AtlasGraphQuery per class to find unique attribute matches.
> - find all existing vertices up front during create/update entity.  Use those 
> vertices during the graph mapping process to avoid running unnecessary graph 
> queries
> - reuse reference vertices from instance to graph mapping when computing full 
> text property



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Loops in V2 API

2017-01-26 Thread Srikanth Venkat 
Hi David,

Thanks for your input. IMHO, think this would be a good addition so that 
community can leverage the relationships aspects.

Srikanth Venkat 
Senior Director, Product Management 
Hortonworks Inc.
sven...@hortonworks.com |  +1 510 394 0497 (O) 

On 1/26/17, 3:18 AM, "David Radley" <david_rad...@uk.ibm.com> wrote:

Dear all, 
I would like to address these loops. Is the community supportive of me 
coding relationships as  AtlasObjectId (Madhan's idea) and adding in an 
optional refLabel? 
  
I will raise Jiras to track the displaytext enhancement around 
globalization and a separate enhancement to introduce explicit 
identifiers. 
 
   many thanks, David. 


- Forwarded by David Radley/UK/IBM on 26/01/2017 11:06 -

From:   David Radley/UK/IBM
To: dev@atlas.incubator.apache.org
Date:   24/01/2017 10:36
Subject:Fw: Loops in V2 API.


Hi,
Responding to Madhan:
- I think there is a need for a new piece of text to specify the 
relationship label - the uml association label that Mandy talks of. The 
label I am thinking of here is part of the  logical model. 
- in terms of displayText for an entity. 
- This appears to me to be a view rather than a model concept 
- displayText normally is globalized. 
- this brings us onto thinking about how we identify an entity- I 
suspect what we want to to display would be a useful entity identifier. . 
- if there is an obvious attribute that is the identifier 
then your approach would be useful. For example the primary key of a RDB 
asset.
- we have found while working with master data management, 
that is useful to have business keys, these are one or more concatenated 
attributes that are useful to a business user to identify 
an entity. I think that introducing business keys into terms and entities 
is a flexible way of dealing with this issue. For   example 
identifying a person with the national insurance number and first and last 
name - could produce a meaningful label like this "NM111333444555-David 
Radley. 
 
So in summary I like the idea, I would split it into: 
1) introduce business identifiers into entities and terms logical 
model. I think this means attributes could be identifiers (isIdentifier) 
and allowing an attribute to be specified as an identifier and be composed 
of a list of other attributes. 
2) introduce globalization of displayText for terms, entities and 
attributes. 
3) enhance the relationships to include an identifier field(s) 
rather than an arbitrary field. 

It will be quite difficult to add in identifier support as more people use 
the V2 API- it would be good to add it early. 

Am I making sense? 

Responding to Mandy:
I was thinking about how we could combine the need for reverse pointers 
always being there, but only sometimes having the need to name a 
relationship . 
In many cases we want to specify a direction to a relationship, but also 
be able to navigate it backwards.

I think having relationships as top level objects in the type system would 
work; it would allow us to manage relationships with properties in a 
standard entity manner. 
I guess we would need to prevent relationships from having relationships.

I wonder what you think of embedding the relationship definition in the 
source object (as the current Atlas does in the constraintDef) and allow 
it to be found in the target object. 

I think having the constraintDefs along the lines of what Madhan and I 
suggested would be a way to optionally specify the reverse attribute name 
and the association name. 
In order see the reverse relationship - I wonder if we had a section in 
the entity called "inbound relationships". We could easily list the 
inbound relationships, inside the API by looking for IN edges in the graph 
 

For example TypeA could have constraintdefs on the relationship and TypeB 
does not. This allows us to navigate back any inbound relaitonship - but 
not have to model this unless we need to add labels. Something like : 

"inboundRelationships": [
{
"type": "TypeA",
"refAttribute": "children",
attributeName : "parent",
"refguid" : "10101010",
"label" : "cares for" 
},