[jira] [Commented] (ATLAS-1691) OMAS Interfaces for Atlas
[ https://issues.apache.org/jira/browse/ATLAS-1691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127323#comment-16127323 ] Nigel Jones commented on ATLAS-1691: I thought I'd start putting together a set of statements that relate to what OMAS is and how we build it. See https://cwiki.apache.org/confluence/display/ATLAS/Standards+for+OMAS+interfaces Comments welcome ... it's just a start.. > OMAS Interfaces for Atlas > - > > Key: ATLAS-1691 > URL: https://issues.apache.org/jira/browse/ATLAS-1691 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Mandy Chessell > Labels: VirtualDataConnector > > The Open Metadata Access Services (OMAS) provide consumer specific interfaces > to the metadata repository. The hope is to have a community of vendor tools > developing around each OMAS interface. The OMAS interfaces are independent > from one another to allow each community to evolve their own API at their > pace. > Related Jira will cover the list of OMAS interfaces that are oriented to > particular tools & usage. > Each OMAS JIRA needs to cover: > * The Java and REST API for the OMAS call-response API > * The appropriate OMAS Event Publisher and Listener -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1698) Create Glossary OMAS API
[ https://issues.apache.org/jira/browse/ATLAS-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127325#comment-16127325 ] Nigel Jones commented on ATLAS-1698: I thought I'd try and summarise some of my thoughts around OMAS design in general (not specific to glossary or indeed GAF) through a series of statements at https://cwiki.apache.org/confluence/display/ATLAS/Standards+for+OMAS+interfaces Agree or not, hopefully this could be the start of documenting our "standards" of how to define an OMAS interface? > Create Glossary OMAS API > > > Key: ATLAS-1698 > URL: https://issues.apache.org/jira/browse/ATLAS-1698 > Project: Atlas > Issue Type: Sub-task >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: apidocs-5.zip, Atlas Glossary OMAS API proposal v1.0 > .pdf, Atlas Glossary OMAS API proposal v1.1.pdf, Atlas Glossary OMAS API > proposal v1.2.pdf > > > The Glossary OMAS provides a specialized API for glossary applications to > retrieve and store their glossary metadata and link assets of different types > to these glossary entries. > The Glossary OMAS makes heavy use of the Area 2 open metadata model. See > https://cwiki.apache.org/confluence/display/ATLAS/Area+2+-+Glossary -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ATLAS-2049) Document common standards for OMAS interfaces
[ https://issues.apache.org/jira/browse/ATLAS-2049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones reassigned ATLAS-2049: -- Assignee: Nigel Jones > Document common standards for OMAS interfaces > - > > Key: ATLAS-2049 > URL: https://issues.apache.org/jira/browse/ATLAS-2049 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones > > Document (and review) common standards for OMAS interfaces, so as to ensure > some consistency between the multiple OMASs being developed. > As per phone discussion with some of team on 15 Aug will create this list in > a text-like format (perhaps twiki) and review using review board tool. > Since this will then form documentation for ATlas (internal) developers it > may make sense to create as a permanent part of the twiki docs, though if so > we may wish to consider adding other related information there such as > architectural explanations of omrs, omas etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2012) Docker - image & hub - for Atlas
[ https://issues.apache.org/jira/browse/ATLAS-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128931#comment-16128931 ] Nigel Jones commented on ATLAS-2012: Still testing - getting intermittent failures on maven dependency retrievals but not consistent - could be work networking issue. Will update with new patch once tested ok > Docker - image & hub - for Atlas > > > Key: ATLAS-2012 > URL: https://issues.apache.org/jira/browse/ATLAS-2012 > Project: Atlas > Issue Type: New Feature >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: atlas_docker1.patch, atlas_docker_2.patch, > atlas_docker.patch > > > Docker is increasingly become a standard way of easily running components in > a flexible manner, whether for development, production, or test > I feel there are a few things we can do with docker that will aid Atlas's > appeal > 1. We could provide a simple example of how to create a docker image from the > Atlas build. This could be published on the wiki & it would make it easier > for developers to use Atlas within a docker environment - mostly by > addressing how to configure & start up > 2. We could automatically generate a docker image as part of the build > process. This builds on #1 by automating the creation of the image & making > it "just part of the build" > 3. We could publish the docker image from #2, for releases, to dockerhub, > making it near trivial for any developer to easily pull down and experiment > with Atlas. Full source of course would be provided/documented so that a user > could customize as needed for their environment plus of course improve what > is provided with the core project > 4. We could work with other teams especially ranger, to allow similar efforts > in other teams to easily work together & be orchestrated -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2049) Document common standards for OMAS interfaces
[ https://issues.apache.org/jira/browse/ATLAS-2049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2049: --- Attachment: ATLAS_2049_Document_common_standards_for_OMAS_interfaces.patch > Document common standards for OMAS interfaces > - > > Key: ATLAS-2049 > URL: https://issues.apache.org/jira/browse/ATLAS-2049 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: > ATLAS_2049_Document_common_standards_for_OMAS_interfaces.patch > > > Document (and review) common standards for OMAS interfaces, so as to ensure > some consistency between the multiple OMASs being developed. > As per phone discussion with some of team on 15 Aug will create this list in > a text-like format (perhaps twiki) and review using review board tool. > Since this will then form documentation for ATlas (internal) developers it > may make sense to create as a permanent part of the twiki docs, though if so > we may wish to consider adding other related information there such as > architectural explanations of omrs, omas etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2049) Document common standards for OMAS interfaces
[ https://issues.apache.org/jira/browse/ATLAS-2049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128983#comment-16128983 ] Nigel Jones commented on ATLAS-2049: Uploaded patch into twiki with initial bulleted list from wiki Will add new patch tomorrow incorporating davids wiki comments, then submit via RB. > Document common standards for OMAS interfaces > - > > Key: ATLAS-2049 > URL: https://issues.apache.org/jira/browse/ATLAS-2049 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: > ATLAS_2049_Document_common_standards_for_OMAS_interfaces.patch > > > Document (and review) common standards for OMAS interfaces, so as to ensure > some consistency between the multiple OMASs being developed. > As per phone discussion with some of team on 15 Aug will create this list in > a text-like format (perhaps twiki) and review using review board tool. > Since this will then form documentation for ATlas (internal) developers it > may make sense to create as a permanent part of the twiki docs, though if so > we may wish to consider adding other related information there such as > architectural explanations of omrs, omas etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2012) Docker - image & hub - for Atlas
[ https://issues.apache.org/jira/browse/ATLAS-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16125800#comment-16125800 ] Nigel Jones commented on ATLAS-2012: Hi Richard, Thanks for posting that patch. I gave it a go under WIndows 10 x64 (with docker installed!) and the build fell over at : Step 6/21 : RUN mkdir -p /usr/java ---> Running in b9952e21fdd7 ---> 5b1b164a474c Removing intermediate container b9952e21fdd7 Step 7/21 : RUN tar xzf *jdk* --strip-components 1 -C /usr/java ---> Running in 98080c6e748e tar (child): *jdk*: Cannot open: No such file or directory tar (child): Error is not recoverable: exiting now tar: Child returned status 2 tar: Error is not recoverable: exiting now The command '/bin/sh -c tar xzf *jdk* --strip-components 1 -C /usr/java' returned a non-zero code: 2 C:\src\atlas\dev-support\atlas-docker>dir Volume in drive C is Windows Volume Serial Number is CE3F-3B05 Directory of C:\src\atlas\dev-support\atlas-docker 14/08/2017 13:56 . 14/08/2017 13:56 .. 14/08/2017 13:55 8,534,562 apache-maven-3.5.0-bin (1).tar.gz 14/08/2017 13:51 2,208 Dockerfile 14/08/2017 13:55 207,382,584 jdk-8u144-windows-x64 (1).exe 14/08/2017 13:51 2,857 README.md 4 File(s)215,922,211 bytes 2 Dir(s) 31,488,966,656 bytes free Just taking a look now, but I also had another thought -- "openjdk" could be a better option to use instead of oracle java (pretty much the same anyway), and has the benefit of being installing via apt. Similarly maven3 is available at 3.3.9 - at least with Ubuntu 16.10 I updated the patch to install jdk/maven via APT, and my build progressed well and completed just fine. I was able to login to the Atlas UI. I did get the hbase shell though didn't need it. Attaching my modified patch > Docker - image & hub - for Atlas > > > Key: ATLAS-2012 > URL: https://issues.apache.org/jira/browse/ATLAS-2012 > Project: Atlas > Issue Type: New Feature >Reporter: Nigel Jones >Assignee: Péter Gergő Barna > Attachments: atlas_docker.patch > > > Docker is increasingly become a standard way of easily running components in > a flexible manner, whether for development, production, or test > I feel there are a few things we can do with docker that will aid Atlas's > appeal > 1. We could provide a simple example of how to create a docker image from the > Atlas build. This could be published on the wiki & it would make it easier > for developers to use Atlas within a docker environment - mostly by > addressing how to configure & start up > 2. We could automatically generate a docker image as part of the build > process. This builds on #1 by automating the creation of the image & making > it "just part of the build" > 3. We could publish the docker image from #2, for releases, to dockerhub, > making it near trivial for any developer to easily pull down and experiment > with Atlas. Full source of course would be provided/documented so that a user > could customize as needed for their environment plus of course improve what > is provided with the core project > 4. We could work with other teams especially ranger, to allow similar efforts > in other teams to easily work together & be orchestrated -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1975) gaf omas : Implement classification support
Nigel Jones created ATLAS-1975: -- Summary: gaf omas : Implement classification support Key: ATLAS-1975 URL: https://issues.apache.org/jira/browse/ATLAS-1975 Project: Atlas Issue Type: Sub-task Reporter: Nigel Jones This subtask will add the initial implementation for classifications (tags) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1696) Governance Action Framework OMAS
[ https://issues.apache.org/jira/browse/ATLAS-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094883#comment-16094883 ] Nigel Jones commented on ATLAS-1696: [~davidrad] I will update the swagger as per some of our recent discussions. Tags is the term still used by ranger, whilst classifications are used by atlas (there is of course also a distinction between the classification (type), the classification instance, and that association of a classification definition+instance to an asset (which in ranger is termed a resource). I initially went with tag being in a ranger-mindset, plus it's shorter... BUT To answer your question I will change to classification since this is an Atlas API. On your roles question, I will clarify -- it's still dependent on the definition of roles which we haven't worked through in the model yet. I'll go with all roles atlas knows about for now, and elaborate once the model is refined. The GAF implementation will be neutral. There are references in the swagger doc as I hoped it would form a basis for discussion rather than being the final result. I will update references to ensure if ranger is mentioned it's clear it's an example. Further I am creating some subtasks which will include creating a swagger output off a skeleton implementation as you've done for glossary OMAS. > Governance Action Framework OMAS > > > Key: ATLAS-1696 > URL: https://issues.apache.org/jira/browse/ATLAS-1696 > Project: Atlas > Issue Type: New Feature >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > > Governance Action OMAS is one of multiple consumer-centric based interfaces > that will be added to Apache Atlas, & provides the API (REST and messaging) > to support policy enforcement frameworks such as Apache Ranger. Detailed > knowledge of the Atlas data models and structure can then be hidden from > these consumers. > The functionality of gaf includes > - ability to retrieve classifications associated to assets > - restricted to "interesting" classifications > - restricted to interesting assets being managed by the requesting endpoint > - to retrieve a list of interesting roles that relate to enforcement > - to retrieve any template rule definitions/lookup tables that might be used > to construct executable rules > The scoping constructs supported in the API will include > - Only get classifications that are relevant for security enforcement (ie: > only those inheriting from a specified supertype? Verify in ATLAS-1839) > - only get information about assets (resources) in a certain part of the > datalake (Q: HOW. By zone? How to specify? by asset type? By associated > endpoint?) > - pagination > > See ATLAS-1839 for more information on the model and classifications > In the Atlas data model classifications propagate - for example > * An database column DOB has no explicit classification > * It's containing table CDB is classified as "customer personal details" > * The "SPI" classification is attached to this table with the value > "sensitive" > At enforcement time all that an engine such as ranger cares about is that the > column "DOB" is sensitive, how we got there isn't important. In the example > above the propogation occurs > * Along the assigned term relationship > * along the structural containment relationship (table->column) > Therefore gaf omas will "flatten" the structure - so in this case we'll see > table/CDB - SPI:sensitive > column/DOB - SPI:sensitive > There will be cases where multiple classifications (of the same type) can be > navigated to from an asset like DOB. This may not make logical sense, > however, Until precedence is resolved in ATLAS-1839 & related Jiras, OMAS > will pass through multiple classifications > This interface will also support message notifications of changes to managed > resources such as a new role, classification. A single kafka topic will be > used. > > A first pass swagger can be found at > https://app.swaggerhub.com/apis/planetf1/GovernanceActionOMAS/0.1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1974) gas omas : create skeleton implementation + generated swagger
Nigel Jones created ATLAS-1974: -- Summary: gas omas : create skeleton implementation + generated swagger Key: ATLAS-1974 URL: https://issues.apache.org/jira/browse/ATLAS-1974 Project: Atlas Issue Type: Sub-task Reporter: Nigel Jones In this task I'll take the previously generated definition in yaml & take the first step to making it closer to implementation by creating the appropriate directory structure, maven projects & java classes which would generate the interface & generate swagger output -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1978) gas omas : add outbound publishing/kafka
Nigel Jones created ATLAS-1978: -- Summary: gas omas : add outbound publishing/kafka Key: ATLAS-1978 URL: https://issues.apache.org/jira/browse/ATLAS-1978 Project: Atlas Issue Type: Sub-task Reporter: Nigel Jones GAF omas must publish changes on topics for consumers. This subtask will be used to add that support after the initial rest api has been created -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1977) gas omas : add inbound messaging/kafka support
Nigel Jones created ATLAS-1977: -- Summary: gas omas : add inbound messaging/kafka support Key: ATLAS-1977 URL: https://issues.apache.org/jira/browse/ATLAS-1977 Project: Atlas Issue Type: Sub-task Reporter: Nigel Jones -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-1973) gaf omas : high level swagger API + supporting doc
Nigel Jones created ATLAS-1973: -- Summary: gaf omas : high level swagger API + supporting doc Key: ATLAS-1973 URL: https://issues.apache.org/jira/browse/ATLAS-1973 Project: Atlas Issue Type: Sub-task Reporter: Nigel Jones Complete/update swagger.io high level definition of api (yaml) Add supporting notes as documentation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1696) Governance Action Framework OMAS
[ https://issues.apache.org/jira/browse/ATLAS-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094861#comment-16094861 ] Nigel Jones commented on ATLAS-1696: Glossary omas is being defined in https://issues.apache.org/jira/browse/ATLAS-1698 - the discussion includes some general discussion that would apply to all OMAS interfaces. Creating sub-tasks for review and implementation: > Governance Action Framework OMAS > > > Key: ATLAS-1696 > URL: https://issues.apache.org/jira/browse/ATLAS-1696 > Project: Atlas > Issue Type: New Feature >Reporter: Nigel Jones >Assignee: Nigel Jones > Labels: VirtualDataConnector > > Governance Action OMAS is one of multiple consumer-centric based interfaces > that will be added to Apache Atlas, & provides the API (REST and messaging) > to support policy enforcement frameworks such as Apache Ranger. Detailed > knowledge of the Atlas data models and structure can then be hidden from > these consumers. > The functionality of gaf includes > - ability to retrieve classifications associated to assets > - restricted to "interesting" classifications > - restricted to interesting assets being managed by the requesting endpoint > - to retrieve a list of interesting roles that relate to enforcement > - to retrieve any template rule definitions/lookup tables that might be used > to construct executable rules > The scoping constructs supported in the API will include > - Only get classifications that are relevant for security enforcement (ie: > only those inheriting from a specified supertype? Verify in ATLAS-1839) > - only get information about assets (resources) in a certain part of the > datalake (Q: HOW. By zone? How to specify? by asset type? By associated > endpoint?) > - pagination > > See ATLAS-1839 for more information on the model and classifications > In the Atlas data model classifications propagate - for example > * An database column DOB has no explicit classification > * It's containing table CDB is classified as "customer personal details" > * The "SPI" classification is attached to this table with the value > "sensitive" > At enforcement time all that an engine such as ranger cares about is that the > column "DOB" is sensitive, how we got there isn't important. In the example > above the propogation occurs > * Along the assigned term relationship > * along the structural containment relationship (table->column) > Therefore gaf omas will "flatten" the structure - so in this case we'll see > table/CDB - SPI:sensitive > column/DOB - SPI:sensitive > There will be cases where multiple classifications (of the same type) can be > navigated to from an asset like DOB. This may not make logical sense, > however, Until precedence is resolved in ATLAS-1839 & related Jiras, OMAS > will pass through multiple classifications > This interface will also support message notifications of changes to managed > resources such as a new role, classification. A single kafka topic will be > used. > > A first pass swagger can be found at > https://app.swaggerhub.com/apis/planetf1/GovernanceActionOMAS/0.1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1698) Create Glossary OMAS API
[ https://issues.apache.org/jira/browse/ATLAS-1698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093149#comment-16093149 ] Nigel Jones commented on ATLAS-1698: Thanks for sharing. For next steps a few more thoughts (this is more general OMAS discussion) a) WHat does the community think is the best way to review the APIs? Should we be swagger-first? swagger.io? standalone yaml or json? Or generate from a java skeleton? Review with review board tool? wiki? Maybe it doesn';t matter? b) I'm thinking to link best practices (wiki) /discussion (jira/ml) from the parent OMAS JIRA > Create Glossary OMAS API > > > Key: ATLAS-1698 > URL: https://issues.apache.org/jira/browse/ATLAS-1698 > Project: Atlas > Issue Type: Sub-task >Reporter: David Radley >Assignee: David Radley > Labels: VirtualDataConnector > Attachments: apidocs-5.zip, Atlas Glossary OMAS API proposal v1.0 > .pdf, Atlas Glossary OMAS API proposal v1.1.pdf, Atlas Glossary OMAS API > proposal v1.2.pdf > > > The Glossary OMAS provides a specialized API for glossary applications to > retrieve and store their glossary metadata and link assets of different types > to these glossary entries. > The Glossary OMAS makes heavy use of the Area 2 open metadata model. See > https://cwiki.apache.org/confluence/display/ATLAS/Area+2+-+Glossary -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1955) Validation for Attributes
[ https://issues.apache.org/jira/browse/ATLAS-1955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093242#comment-16093242 ] Nigel Jones commented on ATLAS-1955: I believe here you are modeling the fact that an email has to follow a certain format, so that metadata should be captured in atlas However the actual validation for instances of this data, ie a customer record being stored in a DB, would typically be outside atlas. In addition the validation may differ as it would be specific to the data processing system being used - an ETL engine, hbase, a filesystem, different languages. So I think the model is more similar to that of Policies, Rules & how ranger works In atlas we have a business-centric definition of a policy, but the actual implementation sits at the enforcement point (in this case a ranger rule) I'm interesting in being able to add capability to capture metadata from ranger so we can then tie back the rule implementation to the policy, to aid in compliance checks, reporting -as well as allow ranger to query atlas for policies when a security admin is creating a rule So I wonder if the same pattern applies here with validation? > Validation for Attributes > - > > Key: ATLAS-1955 > URL: https://issues.apache.org/jira/browse/ATLAS-1955 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: 0.9-incubating >Reporter: Israel Varea > Fix For: 0.9-incubating > > > It would be very nice that Atlas model could contain a way to represent > attribute validation. > A simple example is that we would like to model a Person, with attributes > Name, Email and Country. Now we would like to specify that Email has to > follow a specific regular expression, so it would be nice if we could set > Email -> hasValidation -> EmailRegex, with EmailRegex having: > Name: Email Regular Expresion > Expression: /[0-9a-z]+@[0-9a-z]+.[0-9a-z]+/ > For more complex types of validation, e.g. checking card number validity, it > could be added some external validator function/service. > Name: Credit Card Number Validator > Validator: org.apache.atlas.validators.creditcard or > https://host:port/creditCardValidator > For validations from a reference table, for example a country name, it could > be: > Name: Country Name Ref Validator > Reference Column: > where would be an instance of type Hive_Column or > HBase_Column. > Since this is a kind of Standarization, it could be placed in [Area > 5|https://cwiki.apache.org/confluence/display/ATLAS/Area+5+-+Standards]. > A similar approach is followed in software > [Kylo|https://github.com/Teradata/kylo/tree/master/integrations/spark/spark-validate-cleanse] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2011) gaf omas : Add OMRS event listener
Nigel Jones created ATLAS-2011: -- Summary: gaf omas : Add OMRS event listener Key: ATLAS-2011 URL: https://issues.apache.org/jira/browse/ATLAS-2011 Project: Atlas Issue Type: Sub-task Reporter: Nigel Jones -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2012) Docker - image & hub - for Atlas
Nigel Jones created ATLAS-2012: -- Summary: Docker - image & hub - for Atlas Key: ATLAS-2012 URL: https://issues.apache.org/jira/browse/ATLAS-2012 Project: Atlas Issue Type: New Feature Reporter: Nigel Jones Docker is increasingly become a standard way of easily running components in a flexible manner, whether for development, production, or test I feel there are a few things we can do with docker that will aid Atlas's appeal 1. We could provide a simple example of how to create a docker image from the Atlas build. This could be published on the wiki & it would make it easier for developers to use Atlas within a docker environment - mostly by addressing how to configure & start up 2. We could automatically generate a docker image as part of the build process. This builds on #1 by automating the creation of the image & making it "just part of the build" 3. We could publish the docker image from #2, for releases, to dockerhub, making it near trivial for any developer to easily pull down and experiment with Atlas. Full source of course would be provided/documented so that a user could customize as needed for their environment plus of course improve what is provided with the core project 4. We could work with other teams especially ranger, to allow similar efforts in other teams to easily work together & be orchestrated -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2013) Audit - add non-hbase, persistent audit repository
Nigel Jones created ATLAS-2013: -- Summary: Audit - add non-hbase, persistent audit repository Key: ATLAS-2013 URL: https://issues.apache.org/jira/browse/ATLAS-2013 Project: Atlas Issue Type: New Feature Reporter: Nigel Jones As per mailing list post on 2017-07-24: I was just reading ATLAS-1870 which related to default audit repositories. I see the default is HBaseBasedAuditRepository, and that there is also NoopEntityAuditRepository, and InMemoryEntityAuditRepository. I'm thinking that when Atlas is used in a non-hadoop oriented environment it would be useful to have a non-hbase, persistent (helps a little with compliance ) audit respository. Perhaps another RDB, or indeed in solr/elastic search alongside Ranger's audit events. I'll raise a JIRA on this if it's felt useful & my understanding is correct Nigel. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1978) gas omas : add outbound publishing/kafka
[ https://issues.apache.org/jira/browse/ATLAS-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112640#comment-16112640 ] Nigel Jones commented on ATLAS-1978: This also requires GAF to listen to relevant OMRS notifications so that it can publish it's own OMAS notifications > gas omas : add outbound publishing/kafka > > > Key: ATLAS-1978 > URL: https://issues.apache.org/jira/browse/ATLAS-1978 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones > > GAF omas must publish changes on topics for consumers. > This subtask will be used to add that support after the initial rest api has > been created -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1869) Atlas "plugin" for Ranger (metadata capture)
[ https://issues.apache.org/jira/browse/ATLAS-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-1869: --- Issue Type: New Feature (was: Bug) > Atlas "plugin" for Ranger (metadata capture) > > > Key: ATLAS-1869 > URL: https://issues.apache.org/jira/browse/ATLAS-1869 > Project: Atlas > Issue Type: New Feature >Reporter: Nigel Jones >Assignee: Nigel Jones > > With a variety of data processing engines in Hadoop such as Hive, we have an > Atlas plugin & hook that captures new & updated metadata from those engines > and pushes it to Atlas. This can then be used to support governance including > lineage. > We already have a "ranger plugin" for Atlas which allows ranger to control > access to metadata in atlas - this is NOT the subject of this Jira, but > rather "the other way around" > Examples might include > * Capture information about the policies that are deployed in a ranger > server - the types of assets they refer to, the classifications that are > used. > * Capture information about the topology of ranger - by this I mean the > plugins that are deployed and active, the nodes they run on, and feed this > back into an operational model in Atlas > In each case the information could be published by Ranger, consumed by Atlas, > and stewardship activities around the atlas metadata could help in tying > things together > The benefit would be > - better end to end view (since we know the endpoints, identifiers in audit > logs) > - optimizing the interfaces (rest & kafka) by being able to better targer > useful information - ie if only a hive plugin is being used & configured for > tags, let's just worry the tags it needs. > I see this of use around our open metadata work & specifically VDC, though > not essential for an initial MVP > Placeholder for now... will elaborate further > At the same time the coupling would be loose, and shouldn't hinder any > existing integrations, or decisions as to what is done in Atlas vs Ranger -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2049) Document common standards for OMAS interfaces
[ https://issues.apache.org/jira/browse/ATLAS-2049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2049: --- External issue URL: https://reviews.apache.org/r/61736 (was: https://reviews.apache.org/r/61736/#comment259238) > Document common standards for OMAS interfaces > - > > Key: ATLAS-2049 > URL: https://issues.apache.org/jira/browse/ATLAS-2049 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: > ATLAS_2049_Document_common_standards_for_OMAS_interfaces1.patch, > ATLAS_2049_Document_common_standards_for_OMAS_interfaces21.patch, > ATLAS_2049_Document_common_standards_for_OMAS_interfaces3.patch, > ATLAS_2049_Document_common_standards_for_OMAS_interfaces.patch > > > Document (and review) common standards for OMAS interfaces, so as to ensure > some consistency between the multiple OMASs being developed. > As per phone discussion with some of team on 15 Aug will create this list in > a text-like format (perhaps twiki) and review using review board tool. > Since this will then form documentation for ATlas (internal) developers it > may make sense to create as a permanent part of the twiki docs, though if so > we may wish to consider adding other related information there such as > architectural explanations of omrs, omas etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2049) Document common standards for OMAS interfaces
[ https://issues.apache.org/jira/browse/ATLAS-2049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138330#comment-16138330 ] Nigel Jones commented on ATLAS-2049: Updated review comments (except 1) and hopefully created a correct patch that now mirrors what's on review board > Document common standards for OMAS interfaces > - > > Key: ATLAS-2049 > URL: https://issues.apache.org/jira/browse/ATLAS-2049 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: ATLAS-2049.patch > > > Document (and review) common standards for OMAS interfaces, so as to ensure > some consistency between the multiple OMASs being developed. > As per phone discussion with some of team on 15 Aug will create this list in > a text-like format (perhaps twiki) and review using review board tool. > Since this will then form documentation for ATlas (internal) developers it > may make sense to create as a permanent part of the twiki docs, though if so > we may wish to consider adding other related information there such as > architectural explanations of omrs, omas etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2049) Document common standards for OMAS interfaces
[ https://issues.apache.org/jira/browse/ATLAS-2049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2049: --- Attachment: (was: ATLAS_2049_Document_common_standards_for_OMAS_interfaces1.patch) > Document common standards for OMAS interfaces > - > > Key: ATLAS-2049 > URL: https://issues.apache.org/jira/browse/ATLAS-2049 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: ATLAS-2049.patch > > > Document (and review) common standards for OMAS interfaces, so as to ensure > some consistency between the multiple OMASs being developed. > As per phone discussion with some of team on 15 Aug will create this list in > a text-like format (perhaps twiki) and review using review board tool. > Since this will then form documentation for ATlas (internal) developers it > may make sense to create as a permanent part of the twiki docs, though if so > we may wish to consider adding other related information there such as > architectural explanations of omrs, omas etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2049) Document common standards for OMAS interfaces
[ https://issues.apache.org/jira/browse/ATLAS-2049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2049: --- Attachment: ATLAS-2049.patch > Document common standards for OMAS interfaces > - > > Key: ATLAS-2049 > URL: https://issues.apache.org/jira/browse/ATLAS-2049 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: ATLAS-2049.patch > > > Document (and review) common standards for OMAS interfaces, so as to ensure > some consistency between the multiple OMASs being developed. > As per phone discussion with some of team on 15 Aug will create this list in > a text-like format (perhaps twiki) and review using review board tool. > Since this will then form documentation for ATlas (internal) developers it > may make sense to create as a permanent part of the twiki docs, though if so > we may wish to consider adding other related information there such as > architectural explanations of omrs, omas etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2049) Document common standards for OMAS interfaces
[ https://issues.apache.org/jira/browse/ATLAS-2049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2049: --- Attachment: (was: ATLAS_2049_Document_common_standards_for_OMAS_interfaces3.patch) > Document common standards for OMAS interfaces > - > > Key: ATLAS-2049 > URL: https://issues.apache.org/jira/browse/ATLAS-2049 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: ATLAS-2049.patch > > > Document (and review) common standards for OMAS interfaces, so as to ensure > some consistency between the multiple OMASs being developed. > As per phone discussion with some of team on 15 Aug will create this list in > a text-like format (perhaps twiki) and review using review board tool. > Since this will then form documentation for ATlas (internal) developers it > may make sense to create as a permanent part of the twiki docs, though if so > we may wish to consider adding other related information there such as > architectural explanations of omrs, omas etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ATLAS-1974) gas omas : create skeleton implementation + generated swagger
[ https://issues.apache.org/jira/browse/ATLAS-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones reassigned ATLAS-1974: -- Assignee: Nigel Jones starting work on this > gas omas : create skeleton implementation + generated swagger > - > > Key: ATLAS-1974 > URL: https://issues.apache.org/jira/browse/ATLAS-1974 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones > > In this task I'll take the previously generated definition in yaml & take the > first step to making it closer to implementation by creating the appropriate > directory structure, maven projects & java classes which would generate the > interface & generate swagger output -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1974) goveng omas : create skeleton implementation + generated swagger
[ https://issues.apache.org/jira/browse/ATLAS-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-1974: --- Summary: goveng omas : create skeleton implementation + generated swagger (was: gas omas : create skeleton implementation + generated swagger) > goveng omas : create skeleton implementation + generated swagger > > > Key: ATLAS-1974 > URL: https://issues.apache.org/jira/browse/ATLAS-1974 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones > > In this task I'll take the previously generated definition in yaml & take the > first step to making it closer to implementation by creating the appropriate > directory structure, maven projects & java classes which would generate the > interface & generate swagger output -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2049) Document common standards for OMAS interfaces
[ https://issues.apache.org/jira/browse/ATLAS-2049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2049: --- External issue URL: https://reviews.apache.org/r/61736/#comment259238 > Document common standards for OMAS interfaces > - > > Key: ATLAS-2049 > URL: https://issues.apache.org/jira/browse/ATLAS-2049 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: > ATLAS_2049_Document_common_standards_for_OMAS_interfaces1.patch, > ATLAS_2049_Document_common_standards_for_OMAS_interfaces21.patch, > ATLAS_2049_Document_common_standards_for_OMAS_interfaces3.patch, > ATLAS_2049_Document_common_standards_for_OMAS_interfaces.patch > > > Document (and review) common standards for OMAS interfaces, so as to ensure > some consistency between the multiple OMASs being developed. > As per phone discussion with some of team on 15 Aug will create this list in > a text-like format (perhaps twiki) and review using review board tool. > Since this will then form documentation for ATlas (internal) developers it > may make sense to create as a permanent part of the twiki docs, though if so > we may wish to consider adding other related information there such as > architectural explanations of omrs, omas etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-1696) Governance Engine OMAS
[ https://issues.apache.org/jira/browse/ATLAS-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-1696: --- Description: Governance Engine OMAS is one of multiple consumer-centric based interfaces that will be added to Apache Atlas, & provides the API (REST and messaging) to support policy enforcement frameworks such as Apache Ranger. Detailed knowledge of the Atlas data models and structure can then be hidden from these consumers. The functionality of gaf includes - ability to retrieve classifications associated to assets - restricted to "interesting" classifications - restricted to interesting assets being managed by the requesting endpoint - to retrieve a list of interesting roles that relate to enforcement - to retrieve any template rule definitions/lookup tables that might be used to construct executable rules The scoping constructs supported in the API will include - Only get classifications that are relevant for security enforcement (ie: only those inheriting from a specified supertype? Verify in ATLAS-1839) - only get information about assets (resources) in a certain part of the datalake (Q: HOW. By zone? How to specify? by asset type? By associated endpoint?) - pagination See ATLAS-1839 for more information on the model and classifications In the Atlas data model classifications propagate - for example * An database column DOB has no explicit classification * It's containing table CDB is classified as "customer personal details" * The "SPI" classification is attached to this table with the value "sensitive" At enforcement time all that an engine such as ranger cares about is that the column "DOB" is sensitive, how we got there isn't important. In the example above the propogation occurs * Along the assigned term relationship * along the structural containment relationship (table->column) Therefore gaf omas will "flatten" the structure - so in this case we'll see table/CDB - SPI:sensitive column/DOB - SPI:sensitive There will be cases where multiple classifications (of the same type) can be navigated to from an asset like DOB. This may not make logical sense, however, Until precedence is resolved in ATLAS-1839 & related Jiras, OMAS will pass through multiple classifications This interface will also support message notifications of changes to managed resources such as a new role, classification. A single kafka topic will be used. A first pass swagger can be found at https://app.swaggerhub.com/apis/planetf1/GovernanceActionOMAS/0.1 NOTE: Updated 23 Aug with new name of GOVERNANCE ENGINE OMAS was: Governance Action OMAS is one of multiple consumer-centric based interfaces that will be added to Apache Atlas, & provides the API (REST and messaging) to support policy enforcement frameworks such as Apache Ranger. Detailed knowledge of the Atlas data models and structure can then be hidden from these consumers. The functionality of gaf includes - ability to retrieve classifications associated to assets - restricted to "interesting" classifications - restricted to interesting assets being managed by the requesting endpoint - to retrieve a list of interesting roles that relate to enforcement - to retrieve any template rule definitions/lookup tables that might be used to construct executable rules The scoping constructs supported in the API will include - Only get classifications that are relevant for security enforcement (ie: only those inheriting from a specified supertype? Verify in ATLAS-1839) - only get information about assets (resources) in a certain part of the datalake (Q: HOW. By zone? How to specify? by asset type? By associated endpoint?) - pagination See ATLAS-1839 for more information on the model and classifications In the Atlas data model classifications propagate - for example * An database column DOB has no explicit classification * It's containing table CDB is classified as "customer personal details" * The "SPI" classification is attached to this table with the value "sensitive" At enforcement time all that an engine such as ranger cares about is that the column "DOB" is sensitive, how we got there isn't important. In the example above the propogation occurs * Along the assigned term relationship * along the structural containment relationship (table->column) Therefore gaf omas will "flatten" the structure - so in this case we'll see table/CDB - SPI:sensitive column/DOB - SPI:sensitive There will be cases where multiple classifications (of the same type) can be navigated to from an asset like DOB. This may not make logical sense, however, Until precedence is resolved in ATLAS-1839 & related Jiras, OMAS will pass through multiple classifications This interface will also support message notifications of changes to managed resources such as a new role, classification. A single kafka topic will be used.
[jira] [Commented] (ATLAS-2049) Document common standards for OMAS interfaces
[ https://issues.apache.org/jira/browse/ATLAS-2049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16130688#comment-16130688 ] Nigel Jones commented on ATLAS-2049: I'm working on an update, will post a new patch when ready > Document common standards for OMAS interfaces > - > > Key: ATLAS-2049 > URL: https://issues.apache.org/jira/browse/ATLAS-2049 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: > ATLAS_2049_Document_common_standards_for_OMAS_interfaces1.patch, > ATLAS_2049_Document_common_standards_for_OMAS_interfaces.patch > > > Document (and review) common standards for OMAS interfaces, so as to ensure > some consistency between the multiple OMASs being developed. > As per phone discussion with some of team on 15 Aug will create this list in > a text-like format (perhaps twiki) and review using review board tool. > Since this will then form documentation for ATlas (internal) developers it > may make sense to create as a permanent part of the twiki docs, though if so > we may wish to consider adding other related information there such as > architectural explanations of omrs, omas etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2049) Document common standards for OMAS interfaces
[ https://issues.apache.org/jira/browse/ATLAS-2049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2049: --- Attachment: ATLAS_2049_Document_common_standards_for_OMAS_interfaces3.patch > Document common standards for OMAS interfaces > - > > Key: ATLAS-2049 > URL: https://issues.apache.org/jira/browse/ATLAS-2049 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: > ATLAS_2049_Document_common_standards_for_OMAS_interfaces1.patch, > ATLAS_2049_Document_common_standards_for_OMAS_interfaces21.patch, > ATLAS_2049_Document_common_standards_for_OMAS_interfaces3.patch, > ATLAS_2049_Document_common_standards_for_OMAS_interfaces.patch > > > Document (and review) common standards for OMAS interfaces, so as to ensure > some consistency between the multiple OMASs being developed. > As per phone discussion with some of team on 15 Aug will create this list in > a text-like format (perhaps twiki) and review using review board tool. > Since this will then form documentation for ATlas (internal) developers it > may make sense to create as a permanent part of the twiki docs, though if so > we may wish to consider adding other related information there such as > architectural explanations of omrs, omas etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2086) Docker - reenable unit tests in build once IT stable
[ https://issues.apache.org/jira/browse/ATLAS-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145729#comment-16145729 ] Nigel Jones commented on ATLAS-2086: In the review comments of ATLAS-2012 it was pointed out the UTs take rather a long time to run, so in consideration I will close this JIRA for now. We'll go without UTs. > Docker - reenable unit tests in build once IT stable > > > Key: ATLAS-2086 > URL: https://issues.apache.org/jira/browse/ATLAS-2086 > Project: Atlas > Issue Type: Bug >Reporter: Nigel Jones > > A build of a docker image for atlas was created under ATLAS-2012. > An initial comment from David was that it would be better to build with tests > enabled. I agree - this makes sense in case there's something different about > the docker (ubuntu) environment from the host, and would be good to spot even > though the build takes longer > However with Jenkins build failing regularly, and with some other > intermittent tests failing as per a recent mailing list discussion I am > instead leaving 2012 without UT, and raising this issue to re-introduce it > when things are stable. That way we get the docker support in, but also don't > forget to read testing as soon as we can. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (ATLAS-2086) Docker - reenable unit tests in build once IT stable
[ https://issues.apache.org/jira/browse/ATLAS-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones resolved ATLAS-2086. Resolution: Won't Do > Docker - reenable unit tests in build once IT stable > > > Key: ATLAS-2086 > URL: https://issues.apache.org/jira/browse/ATLAS-2086 > Project: Atlas > Issue Type: Bug >Reporter: Nigel Jones > > A build of a docker image for atlas was created under ATLAS-2012. > An initial comment from David was that it would be better to build with tests > enabled. I agree - this makes sense in case there's something different about > the docker (ubuntu) environment from the host, and would be good to spot even > though the build takes longer > However with Jenkins build failing regularly, and with some other > intermittent tests failing as per a recent mailing list discussion I am > instead leaving 2012 without UT, and raising this issue to re-introduce it > when things are stable. That way we get the docker support in, but also don't > forget to read testing as soon as we can. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2012) Docker - image & hub - for Atlas
[ https://issues.apache.org/jira/browse/ATLAS-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145723#comment-16145723 ] Nigel Jones commented on ATLAS-2012: thanks. will raise new jira to add sample data, to make available via docker hub, and for thinking about combining atlas with other components (be it hive, ranger) for a more complete experience > Docker - image & hub - for Atlas > > > Key: ATLAS-2012 > URL: https://issues.apache.org/jira/browse/ATLAS-2012 > Project: Atlas > Issue Type: New Feature >Reporter: Nigel Jones >Assignee: Nigel Jones > Fix For: 0.9-incubating > > Attachments: ATLAS-2012-Build-docker-image.patch, > atlas_docker1.patch, atlas_docker_2.patch, atlas_docker.patch > > > Docker is increasingly become a standard way of easily running components in > a flexible manner, whether for development, production, or test > I feel there are a few things we can do with docker that will aid Atlas's > appeal > 1. We could provide a simple example of how to create a docker image from the > Atlas build. This could be published on the wiki & it would make it easier > for developers to use Atlas within a docker environment - mostly by > addressing how to configure & start up > 2. We could automatically generate a docker image as part of the build > process. This builds on #1 by automating the creation of the image & making > it "just part of the build" > 3. We could publish the docker image from #2, for releases, to dockerhub, > making it near trivial for any developer to easily pull down and experiment > with Atlas. Full source of course would be provided/documented so that a user > could customize as needed for their environment plus of course improve what > is provided with the core project > 4. We could work with other teams especially ranger, to allow similar efforts > in other teams to easily work together & be orchestrated -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2096) Integration Test fails: Given Type Dimension already exists
Nigel Jones created ATLAS-2096:
--
Summary: Integration Test fails: Given Type Dimension already
exists
Key: ATLAS-2096
URL: https://issues.apache.org/jira/browse/ATLAS-2096
Project: Atlas
Issue Type: Bug
Reporter: Nigel Jones
When building Atlas for hbase/embedded solo I hit a UT failure during the web
app tests:
127.0.0.1 - - [29/Aug/2017:17:33:22 +] "GET
/api/atlas/entities/6cb20ab1-00c2-4e80-a2dc-dfb7a20c90e3 HTTP/1.1" 200 - "-"
"Java/1.8.0_144"
127.0.0.1 - - [29/Aug/2017:17:33:22 +] "POST
/api/atlas/entities/6cb20ab1-00c2-4e80-a2dc-dfb7a20c90e3?property=invalid_property
HTTP/1.1" 400 - "-" "Java/1.8.0_144"
127.0.0.1 - - [29/Aug/2017:17:33:22 +] "GET
/api/atlas/entities/6cb20ab1-00c2-4e80-a2dc-dfb7a20c90e3 HTTP/1.1" 200 - "-"
"Java/1.8.0_144"
127.0.0.1 - - [29/Aug/2017:17:33:23 +] "POST /api/atlas/v2/types/typedefs/
HTTP/1.1" 409 - "-" "Java/1.8.0_144"
Tests run: 134, Failures: 1, Errors: 0, Skipped: 5, Time elapsed: 80.922 sec
<<< FAILURE! - in TestSuite
runQuickStart(org.apache.atlas.examples.QuickStartV2IT) Time elapsed: 0.387
sec <<< FAILURE!
org.apache.atlas.AtlasServiceException: Metadata service API
org.apache.atlas.AtlasBaseClient$APIInfo@1bd893c7 failed with status 409
(Conflict) Response Body ({"errorCode":"ATLAS-409-00-001","errorMessage":"Given
type Dimension already exists"})
at
org.apache.atlas.examples.QuickStartV2IT.runQuickStart(QuickStartV2IT.java:49)
Results :
Failed tests:
QuickStartV2IT.runQuickStart:49 » AtlasService Metadata service API
org.apache...
Tests run: 134, Failures: 1, Errors: 0, Skipped: 5
[INFO]
[INFO] --- jetty-maven-plugin:9.2.12.v20150709:stop (stop-jetty) @ atlas-webapp
---
This was reported in the mailing list at
https://lists.apache.org/thread.html/9db9d8668f22a2cdd58dd42fb582e026e07703c7ec6e9ae0e7d53f69@%3Cdev.atlas.apache.org%3E
Subsequently noticed this is failing (intermittently) on the Jenkins build too.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ATLAS-1768) Create common types for Open Metadata
[ https://issues.apache.org/jira/browse/ATLAS-1768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147506#comment-16147506 ] Nigel Jones commented on ATLAS-1768: In RANGER-1486 we propose a mechanism to scope the user groups retrieved from LDAP to a subset that is 'interesting' in terms of data users. For example it would be these users that might access a virtualized data resource through gaiandb with it's associated ranger plugin. So far I don't believe we have these groups -- more appropriately let's call them roles here -- captured in any of the existing model definitions. ATLAS-1696 will deliver governance engine OMAS, and one of the proposed APIs is to retrieve user roles which will be added via subtask ATLAS-1796 These definitions probably belong in area 4 as they more broadly define the governance landscape, though individuals themselves (and one could argue roles) feature in area 3. In either case I think we need a new subtask to implement this area - even if it's just a list of user roles. In terms of what we need to know about roles, at this point it's mostly simple a list that we can match against ldap, and report in ranger. For UI use the usual description etc could be useful, but probably won't be used initially. A guid of some form is always good, but probably can't be used for matching since the association with ldap is via alphanumeric match of the name only. All of these do become relevant as we build more information into atlas around the definition of policies and their enforcement which also comes into area 4 & would be influenced by RANGER-1869 along with more general operational infrastructure metadata we capture (servers, nodes etc) > Create common types for Open Metadata > - > > Key: ATLAS-1768 > URL: https://issues.apache.org/jira/browse/ATLAS-1768 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Affects Versions: 0.9-incubating >Reporter: Mandy Chessell >Assignee: Mandy Chessell > Labels: OpenMetadata > > This JIRA describes a proposal for standard types for open metadata entities > and relationships. For example, glossaries, database definitions, rules, > policies, ... > The value of having standard definitions for metadata is to enable type safe > APIs and business level UIs plus be able to exchange metadata between > different instances of metadata repositories. > The implementation of these common types is divided into 8 areas: > * Area 0 - for extensions to Apache Atlas's base model > * Area 1 - for definitions of the data-related assets we are governing and > using > * Area 2 - for a glossary of meanings and semantic relationships > * Area 3 - for information about asset use, crowd-sourced definitions and > collaboration around the data-related assets > * Area 4 - for governance such as policies, rules and classifications > * Area 5 - for reference models and reference data > * Area 6 - for metadata discovery processes (see > https://issues.apache.org/jira/browse/ATLAS-1748) > * Area 7 - for lineage > Adaptation and flexibility are key in metadata environments so these common > definitions must be extensible - and we still need to support the ad hoc > definition of new types in Atlas. > Apache Atlas supports meta-types that are used in the definition of new > types. These are currently enumeration, struct, classification and entity. > JIRA https://issues.apache.org/jira/browse/ATLAS-1690 adds relationships to > this list. The open metadata models make use of all of these meta-types. > These are represented by sterotypes on the classes of the open metadata > definitions. > The Atlas wiki has the models as a set of linked pages which are probably the > easiest way to view the models. > Start here: > https://cwiki.apache.org/confluence/display/ATLAS/Building+out+the+Apache+Atlas+Typesystem -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2095) Docker - add other interesting pieces like hive, ranger
Nigel Jones created ATLAS-2095: -- Summary: Docker - add other interesting pieces like hive, ranger Key: ATLAS-2095 URL: https://issues.apache.org/jira/browse/ATLAS-2095 Project: Atlas Issue Type: New Feature Reporter: Nigel Jones In ATLAS-2012, ATLAS-2093 & ATLAS-2094 we have created docker images for Atlas. However this is 'just' Atlas. It would be interesting to have a proper data engine like hive contributing metadata, maybe even perhaps an enforcement engine like Ranger, enforcing access to the data based on business terms, classifications & real data. This makes it much easier to understand the scope of how Atlas can help. Since this is cross component we should consider if Atlas is the right home, or if there are other external projects which might handle this. Also whilst we could build a single monolithic image, I think it would be preferable instead to orchestrate a number of docker containers. docker-compose is rather primitive, but is simple & could possibly suffice, whilst a kubernetes approach is more real, but requires more setup work to run -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1974) goveng omas : create skeleton implementation + generated swagger
[ https://issues.apache.org/jira/browse/ATLAS-1974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155343#comment-16155343 ] Nigel Jones commented on ATLAS-1974: Suggestion to Also add UML model to facilitate review > goveng omas : create skeleton implementation + generated swagger > > > Key: ATLAS-1974 > URL: https://issues.apache.org/jira/browse/ATLAS-1974 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones > > In this task I'll take the previously generated definition in yaml & take the > first step to making it closer to implementation by creating the appropriate > directory structure, maven projects & java classes which would generate the > interface & generate swagger output -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Issue Comment Deleted] (ATLAS-2012) Docker - image & hub - for Atlas
[ https://issues.apache.org/jira/browse/ATLAS-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2012: --- Comment: was deleted (was: I hit the exact same issue on a standalone build. I therefore suggest a) We continue with the review board post (I'll make the change to have tests enabled) for the docker image (Richard: if you want to do this let me know today else I'll action) b) I'll open up a new JIRA and link to this one (or link if one exists) for the specific compile failure) > Docker - image & hub - for Atlas > > > Key: ATLAS-2012 > URL: https://issues.apache.org/jira/browse/ATLAS-2012 > Project: Atlas > Issue Type: New Feature >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: atlas_docker1.patch, atlas_docker_2.patch, > atlas_docker.patch > > > Docker is increasingly become a standard way of easily running components in > a flexible manner, whether for development, production, or test > I feel there are a few things we can do with docker that will aid Atlas's > appeal > 1. We could provide a simple example of how to create a docker image from the > Atlas build. This could be published on the wiki & it would make it easier > for developers to use Atlas within a docker environment - mostly by > addressing how to configure & start up > 2. We could automatically generate a docker image as part of the build > process. This builds on #1 by automating the creation of the image & making > it "just part of the build" > 3. We could publish the docker image from #2, for releases, to dockerhub, > making it near trivial for any developer to easily pull down and experiment > with Atlas. Full source of course would be provided/documented so that a user > could customize as needed for their environment plus of course improve what > is provided with the core project > 4. We could work with other teams especially ranger, to allow similar efforts > in other teams to easily work together & be orchestrated -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2012) Docker - image & hub - for Atlas
[ https://issues.apache.org/jira/browse/ATLAS-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140056#comment-16140056 ] Nigel Jones commented on ATLAS-2012: I hit the exact same issue on a standalone build. I therefore suggest a) We continue with the review board post (I'll make the change to have tests enabled) for the docker image (Richard: if you want to do this let me know today else I'll action) b) I'll open up a new JIRA and link to this one (or link if one exists) for the specific compile failure > Docker - image & hub - for Atlas > > > Key: ATLAS-2012 > URL: https://issues.apache.org/jira/browse/ATLAS-2012 > Project: Atlas > Issue Type: New Feature >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: atlas_docker1.patch, atlas_docker_2.patch, > atlas_docker.patch > > > Docker is increasingly become a standard way of easily running components in > a flexible manner, whether for development, production, or test > I feel there are a few things we can do with docker that will aid Atlas's > appeal > 1. We could provide a simple example of how to create a docker image from the > Atlas build. This could be published on the wiki & it would make it easier > for developers to use Atlas within a docker environment - mostly by > addressing how to configure & start up > 2. We could automatically generate a docker image as part of the build > process. This builds on #1 by automating the creation of the image & making > it "just part of the build" > 3. We could publish the docker image from #2, for releases, to dockerhub, > making it near trivial for any developer to easily pull down and experiment > with Atlas. Full source of course would be provided/documented so that a user > could customize as needed for their environment plus of course improve what > is provided with the core project > 4. We could work with other teams especially ranger, to allow similar efforts > in other teams to easily work together & be orchestrated -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2012) Docker - image & hub - for Atlas
[ https://issues.apache.org/jira/browse/ATLAS-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139987#comment-16139987 ] Nigel Jones commented on ATLAS-2012: Checked with unit tests active. Currently failing on the Web Application: 127.0.0.1 - - [24/Aug/2017:10:44:08 +] "GET /api/atlas/lineage/hive/table/blah/schema HTTP/1.1" 404 - "-" "Java/1.8.0_131" 127.0.0.1 - - [24/Aug/2017:10:44:08 +] "GET /api/atlas/entities?type=hive_table_v1=qualifiedName=sales_fact_monthly_mvGHl8C8DX58 HTTP/1.1" 200 - "-" "Java/1.8.0_131" 127.0.0.1 - - [24/Aug/2017:10:44:08 +] "GET /api/atlas/lineage/a84a19b9-cba1-4a2e-a0f4-c41471d4b811/inputs/graph HTTP/1.1" 200 - "-" "Java/1.8.0_131" Tests run: 133, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 280.46 sec <<< FAILURE! - in TestSuite testUTF8(org.apache.atlas.web.integration.EntityJerseyResourceIT) Time elapsed: 0.234 sec <<< FAILURE! java.lang.AssertionError: expected: but was: at org.apache.atlas.web.integration.EntityJerseyResourceIT.testUTF8(EntityJerseyResourceIT.java:840) Results : Failed tests: EntityJerseyResourceIT.testUTF8:840 expected: but was: Tests run: 133, Failures: 1, Errors: 0, Skipped: 0 This error was repeated on a second run > Docker - image & hub - for Atlas > > > Key: ATLAS-2012 > URL: https://issues.apache.org/jira/browse/ATLAS-2012 > Project: Atlas > Issue Type: New Feature >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: atlas_docker1.patch, atlas_docker_2.patch, > atlas_docker.patch > > > Docker is increasingly become a standard way of easily running components in > a flexible manner, whether for development, production, or test > I feel there are a few things we can do with docker that will aid Atlas's > appeal > 1. We could provide a simple example of how to create a docker image from the > Atlas build. This could be published on the wiki & it would make it easier > for developers to use Atlas within a docker environment - mostly by > addressing how to configure & start up > 2. We could automatically generate a docker image as part of the build > process. This builds on #1 by automating the creation of the image & making > it "just part of the build" > 3. We could publish the docker image from #2, for releases, to dockerhub, > making it near trivial for any developer to easily pull down and experiment > with Atlas. Full source of course would be provided/documented so that a user > could customize as needed for their environment plus of course improve what > is provided with the core project > 4. We could work with other teams especially ranger, to allow similar efforts > in other teams to easily work together & be orchestrated -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2012) Docker - image & hub - for Atlas
[ https://issues.apache.org/jira/browse/ATLAS-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139850#comment-16139850 ] Nigel Jones commented on ATLAS-2012: Validated the patch again (macOS). Main build within the image took 14:30, whilst the entire docker process (from docker run, in a clean environment, so a full build including all maven downloads) with fast internet connection 200 Mbps on a 2016 macpro took 20 minutes. Launcher the docker image took around 45s (at base shell, with atlas launched) and around another 20-30s for the web UI to be accessible. (there's another atlas issue open on that) I propose we post the patch for review. Richard - would you like to, or I can kick off the review process and update as needed. > Docker - image & hub - for Atlas > > > Key: ATLAS-2012 > URL: https://issues.apache.org/jira/browse/ATLAS-2012 > Project: Atlas > Issue Type: New Feature >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: atlas_docker1.patch, atlas_docker_2.patch, > atlas_docker.patch > > > Docker is increasingly become a standard way of easily running components in > a flexible manner, whether for development, production, or test > I feel there are a few things we can do with docker that will aid Atlas's > appeal > 1. We could provide a simple example of how to create a docker image from the > Atlas build. This could be published on the wiki & it would make it easier > for developers to use Atlas within a docker environment - mostly by > addressing how to configure & start up > 2. We could automatically generate a docker image as part of the build > process. This builds on #1 by automating the creation of the image & making > it "just part of the build" > 3. We could publish the docker image from #2, for releases, to dockerhub, > making it near trivial for any developer to easily pull down and experiment > with Atlas. Full source of course would be provided/documented so that a user > could customize as needed for their environment plus of course improve what > is provided with the core project > 4. We could work with other teams especially ranger, to allow similar efforts > in other teams to easily work together & be orchestrated -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-2012) Docker - image & hub - for Atlas
[ https://issues.apache.org/jira/browse/ATLAS-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139850#comment-16139850 ] Nigel Jones edited comment on ATLAS-2012 at 8/24/17 10:07 AM: -- Validated the patch again (macOS). Main build within the image took 14:30, whilst the entire docker process (from docker run, in a clean environment, so a full build including all maven downloads) with fast internet connection 200 Mbps on a 2016 macpro took 20 minutes. Launcher the docker image took around 45s (at base shell, with atlas launched) and around another 20-30s for the web UI to be accessible. (there's another atlas issue open on that) I propose we post the patch for review. Richard - would you like to, or I can kick off the review process and update as needed. I'm just checking one other change - the impact of removing the test skip was (Author: jonesn): Validated the patch again (macOS). Main build within the image took 14:30, whilst the entire docker process (from docker run, in a clean environment, so a full build including all maven downloads) with fast internet connection 200 Mbps on a 2016 macpro took 20 minutes. Launcher the docker image took around 45s (at base shell, with atlas launched) and around another 20-30s for the web UI to be accessible. (there's another atlas issue open on that) I propose we post the patch for review. Richard - would you like to, or I can kick off the review process and update as needed. > Docker - image & hub - for Atlas > > > Key: ATLAS-2012 > URL: https://issues.apache.org/jira/browse/ATLAS-2012 > Project: Atlas > Issue Type: New Feature >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: atlas_docker1.patch, atlas_docker_2.patch, > atlas_docker.patch > > > Docker is increasingly become a standard way of easily running components in > a flexible manner, whether for development, production, or test > I feel there are a few things we can do with docker that will aid Atlas's > appeal > 1. We could provide a simple example of how to create a docker image from the > Atlas build. This could be published on the wiki & it would make it easier > for developers to use Atlas within a docker environment - mostly by > addressing how to configure & start up > 2. We could automatically generate a docker image as part of the build > process. This builds on #1 by automating the creation of the image & making > it "just part of the build" > 3. We could publish the docker image from #2, for releases, to dockerhub, > making it near trivial for any developer to easily pull down and experiment > with Atlas. Full source of course would be provided/documented so that a user > could customize as needed for their environment plus of course improve what > is provided with the core project > 4. We could work with other teams especially ranger, to allow similar efforts > in other teams to easily work together & be orchestrated -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2012) Docker - image & hub - for Atlas
[ https://issues.apache.org/jira/browse/ATLAS-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2012: --- Attachment: ATLAS-2012-Build-docker-image.patch > Docker - image & hub - for Atlas > > > Key: ATLAS-2012 > URL: https://issues.apache.org/jira/browse/ATLAS-2012 > Project: Atlas > Issue Type: New Feature >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: ATLAS-2012-Build-docker-image.patch, > atlas_docker1.patch, atlas_docker_2.patch, atlas_docker.patch > > > Docker is increasingly become a standard way of easily running components in > a flexible manner, whether for development, production, or test > I feel there are a few things we can do with docker that will aid Atlas's > appeal > 1. We could provide a simple example of how to create a docker image from the > Atlas build. This could be published on the wiki & it would make it easier > for developers to use Atlas within a docker environment - mostly by > addressing how to configure & start up > 2. We could automatically generate a docker image as part of the build > process. This builds on #1 by automating the creation of the image & making > it "just part of the build" > 3. We could publish the docker image from #2, for releases, to dockerhub, > making it near trivial for any developer to easily pull down and experiment > with Atlas. Full source of course would be provided/documented so that a user > could customize as needed for their environment plus of course improve what > is provided with the core project > 4. We could work with other teams especially ranger, to allow similar efforts > in other teams to easily work together & be orchestrated -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (ATLAS-2086) Docker - reenable unit tests in build once IT stable
Nigel Jones created ATLAS-2086: -- Summary: Docker - reenable unit tests in build once IT stable Key: ATLAS-2086 URL: https://issues.apache.org/jira/browse/ATLAS-2086 Project: Atlas Issue Type: Bug Reporter: Nigel Jones A build of a docker image for atlas was created under ATLAS-2012. An initial comment from David was that it would be better to build with tests enabled. I agree - this makes sense in case there's something different about the docker (ubuntu) environment from the host, and would be good to spot even though the build takes longer However with Jenkins build failing regularly, and with some other intermittent tests failing as per a recent mailing list discussion I am instead leaving 2012 without UT, and raising this issue to re-introduce it when things are stable. That way we get the docker support in, but also don't forget to read testing as soon as we can. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2049) Document common standards for OMAS interfaces
[ https://issues.apache.org/jira/browse/ATLAS-2049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2049: --- Attachment: ATLAS_2049_Document_common_standards_for_OMAS_interfaces21.patch > Document common standards for OMAS interfaces > - > > Key: ATLAS-2049 > URL: https://issues.apache.org/jira/browse/ATLAS-2049 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: > ATLAS_2049_Document_common_standards_for_OMAS_interfaces1.patch, > ATLAS_2049_Document_common_standards_for_OMAS_interfaces21.patch, > ATLAS_2049_Document_common_standards_for_OMAS_interfaces.patch > > > Document (and review) common standards for OMAS interfaces, so as to ensure > some consistency between the multiple OMASs being developed. > As per phone discussion with some of team on 15 Aug will create this list in > a text-like format (perhaps twiki) and review using review board tool. > Since this will then form documentation for ATlas (internal) developers it > may make sense to create as a permanent part of the twiki docs, though if so > we may wish to consider adding other related information there such as > architectural explanations of omrs, omas etc. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2092) Failures following concurrent updates
[ https://issues.apache.org/jira/browse/ATLAS-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148957#comment-16148957 ] Nigel Jones commented on ATLAS-2092: thanks for the very thorough write-up and walk through. In terms of resolution a little thinking 'out-loud' * At the point we switch to janus and drop titan 0.54 we ideally don't want any workarounds in that version of the code, but we may need to make sure we address any odd things in the graph during a migration step? * You mention that fixing titan 0.54 is unrealistic. I understand there is no longer any support/activity, it's a dead end. However if it is a small, precise change that is required, I still wonder if could build a patch that can be applied to 0.54, and ship it with atlas. We could also ensure the libraries we supply have it included (needs a build), and perhaps detect on startup, and log an error if patch isn't in place. Though this is a bit tedious, I still wonder if it's safer or more controlled than adding a fair bit of workaround code. may be worth thinking about? * However in the case of any fix (as in previous point) am I right in thinking once this has occurred it may continue to cause functional issues since the key will always be out of the correct range? In which case we'd still either need to be able to detect, and/or self-heal/fix on demand, or have a fix-script available for old atlas users? this may make my previous point somewhat mute. > Failures following concurrent updates > - > > Key: ATLAS-2092 > URL: https://issues.apache.org/jira/browse/ATLAS-2092 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Graham Wallis > Attachments: Investigations and findings relating to concurrent > updates in Atlas.pdf > > > There is a race condition that causes duplication of schema vertices as a > result of concurrent graph updates. This in turn leads to failure of queries > that specify a type such as an edge label used in an attribute that > references another entity. This problem is known to affect Atlas entity refs > – which create graph edges that use edge label schema vertices. It is likely > that it also affects other types in Atlas. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-1978) gas omas : add outbound publishing/kafka
[ https://issues.apache.org/jira/browse/ATLAS-1978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150472#comment-16150472 ] Nigel Jones commented on ATLAS-1978: See ATLAS-2011 for a description of the event listening of OMRS messages This JIRA will use the content of those messages to construct an appropriate outbound message for governance engine OMAS subscribers - That format needs to be defined - As much information as possible needs to be retrieved from the inbound message. Ideally nothing additional will be required from the repository. If more information is required it's important to consider that any REST based query to OMRS could return different information to that in the message either because a) The entity (or similar) being queried hasn't yet been stored OR b) the entity has subsequently been deleted. Sequence numbers/last update need to be checked, and retries may need to be done. - we need to obtain configuration information about the kafka endpoint, topic name to use from 'configuration' - The actual publish needs to be done Note to David - This is now Governance Engine omas. I've not updated all the subtasks of ATLAS-1696 yet... You are correct in that the governance action framework includes a lot more than just the omas interface. > gas omas : add outbound publishing/kafka > > > Key: ATLAS-1978 > URL: https://issues.apache.org/jira/browse/ATLAS-1978 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones > > GAF omas must publish changes on topics for consumers. > This subtask will be used to add that support after the initial rest api has > been created -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2011) Governance Engine OMAS : Add OMAS event listener
[ https://issues.apache.org/jira/browse/ATLAS-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150478#comment-16150478 ] Nigel Jones commented on ATLAS-2011: Thanks :-) * I know what I mean but tripped over on terminology. The phrase 'omrs event listener' could mean the part of omrs that listens to events.. ie from other omrs -> OMRS/Event Listener. OR 'the code in governance engine omas that listens to omrs messages'.OMRS Event/Listener. It was the latter I meant Rather than edit the comment, which would make these comments difficult to follow I'll clarify later.. could do with a picture > Governance Engine OMAS : Add OMAS event listener > > > Key: ATLAS-2011 > URL: https://issues.apache.org/jira/browse/ATLAS-2011 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2011) Governance Engine OMAS : Add OMAS event listener
[ https://issues.apache.org/jira/browse/ATLAS-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150486#comment-16150486 ] Nigel Jones commented on ATLAS-2011: Adding dependencies on the areas of the metadata model that this work depends on. To correctly inform governance engines, we need to know about area 1 (assets), area 2 (terms) and area 4 (classifications). > Governance Engine OMAS : Add OMAS event listener > > > Key: ATLAS-2011 > URL: https://issues.apache.org/jira/browse/ATLAS-2011 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2011) gaf omas : Add OMRS event listener
[ https://issues.apache.org/jira/browse/ATLAS-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150455#comment-16150455 ] Nigel Jones commented on ATLAS-2011: When changes occur in the metadata repository (whether Atlas, or another compliant repository), messages are posted to the OMRS outbound topic so that subscribers can learn about changes in the repository - The format of these messages needs defining. See ATLAS-1829 for this work The OMRS event listener will listen to these, and where there are repository changes that relate to a change in anything governance engine omas cares about -- user roles, classifications themselves, or associates of classifications to entities -- it will formulate the appropriate action to take, so that it can then publish a message to it's outbound kafka topic for downstream subscribers. That publication is covered in ATLAS-1978 I suggest this JIRA is used to setup the event listening process and filter the messages needed, whilst 1978 focusses on building the outbound message for downstream. > gaf omas : Add OMRS event listener > -- > > Key: ATLAS-2011 > URL: https://issues.apache.org/jira/browse/ATLAS-2011 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2258) Fix new Integration test failures introduced due to ATLAS-2251
[ https://issues.apache.org/jira/browse/ATLAS-2258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252255#comment-16252255 ] Nigel Jones commented on ATLAS-2258: I was getting a whole slew of errors relating to the DSL change.. so I applied this patch to master (as of just now) and most of the errors have gone, though I now get this new one: Mentioning it here as this patch isn't in master yet?\ Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 13.277 sec <<< FAILURE! - in org.apache.atlas.web.service.SecureEmbeddedServerTest runOtherSuitesAgainstSecureServer(org.apache.atlas.web.service.SecureEmbeddedServerTest) Time elapsed: 1.509 sec <<< FAILURE! org.testng.TestNGException: org.apache.atlas.web.integration.MetadataDiscoveryJerseyResourceIT.testSearchUsingFullText() is depending on method public void org.apache.atlas.web.integration.MetadataDiscoveryJerseyResourceIT.testSearchDSLLimits() throws java.lang.Exception, which is not annotated with @Test or not included. at org.testng.internal.MethodHelper.findDependedUponMethods(MethodHelper.java:111) at org.testng.internal.MethodHelper.topologicalSort(MethodHelper.java:241) at org.testng.internal.MethodHelper.sortMethods(MethodHelper.java:317) at org.testng.internal.MethodHelper.collectAndOrderMethods(MethodHelper.java:59) at org.testng.TestRunner.initMethods(TestRunner.java:485) at org.testng.TestRunner.init(TestRunner.java:239) at org.testng.TestRunner.init(TestRunner.java:209) at org.testng.TestRunner.(TestRunner.java:157) at org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:546) at org.testng.SuiteRunner.init(SuiteRunner.java:159) at org.testng.SuiteRunner.(SuiteRunner.java:113) at org.testng.TestNG.createSuiteRunner(TestNG.java:1274) at org.testng.TestNG.createSuiteRunners(TestNG.java:1261) at org.testng.TestNG.runSuitesLocally(TestNG.java:1115) at org.testng.TestNG.run(TestNG.java:1032) at org.apache.atlas.web.service.SecureEmbeddedServerTestBase.runOtherSuitesAgainstSecureServer(SecureEmbeddedServerTestBase.java:179) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:85) at org.testng.internal.Invoker.invokeMethod(Invoker.java:659) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:845) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1153) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:108) at org.testng.TestRunner.privateRun(TestRunner.java:771) at org.testng.TestRunner.run(TestRunner.java:621) at org.testng.SuiteRunner.runTest(SuiteRunner.java:357) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:352) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:310) at org.testng.SuiteRunner.run(SuiteRunner.java:259) at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86) at org.testng.TestNG.runSuitesSequentially(TestNG.java:1199) at org.testng.TestNG.runSuitesLocally(TestNG.java:1124) at org.testng.TestNG.run(TestNG.java:1032) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:115) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:129) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:113) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:111) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) (macOS, jdk 1.8 152b16) > Fix new Integration test failures introduced due to ATLAS-2251 > -- > > Key: ATLAS-2258 > URL: https://issues.apache.org/jira/browse/ATLAS-2258 > Project: Atlas > Issue Type: Task > Components: atlas-core >Affects Versions: 1.0.0 >Reporter: Sarath Subramanian >Assignee: Sarath Subramanian > Fix
[jira] [Assigned] (ATLAS-2094) Make Atlas available in docker hub
[ https://issues.apache.org/jira/browse/ATLAS-2094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones reassigned ATLAS-2094: -- Assignee: Nigel Jones > Make Atlas available in docker hub > -- > > Key: ATLAS-2094 > URL: https://issues.apache.org/jira/browse/ATLAS-2094 > Project: Atlas > Issue Type: New Feature >Reporter: Nigel Jones >Assignee: Nigel Jones > > In ATLAS-2012 we introduced a basic docker image for Atlas which is created > in the build. > In ATLAS-2093 this can be extended with some basic sample data > This Jira will take things a step further. We can make Atlas available on the > docker hub or similar, so that just as with many technologies like Cassandra, > mysql, someone interested in taking a peek at what atlas has to offer can > quickly obtain and run atlas in little more than a single line. > We need to check licensing (I imagine it's ok but need to validate), and > consider what other documentation might be needed, and any extra sample data, > so that someone is given a good impression of what the project offers. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2251) Remove/replace use of TypeSystem with AtlasTypeRegistry
[ https://issues.apache.org/jira/browse/ATLAS-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16252738#comment-16252738 ] Nigel Jones commented on ATLAS-2251: The patch may need rebasing - that error I posted now seems fixed. Getting a Hive Hook IT error now but I think that is a seperate issue so once rebased I think patch is good > Remove/replace use of TypeSystem with AtlasTypeRegistry > --- > > Key: ATLAS-2251 > URL: https://issues.apache.org/jira/browse/ATLAS-2251 > Project: Atlas > Issue Type: Task > Components: atlas-core >Affects Versions: 1.0.0 >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj >Priority: Critical > Fix For: 1.0.0 > > Attachments: ATLAS-2251-1.patch, ATLAS-2251-3.patch, > ATLAS-2251-4.patch > > > With introduction of V2 APIs in 0.8 release, Atlas core type-system > implementation moved to use AtlasTypeRegistry (which uses data-structures > introduced in V2 APIs). However, earlier version equivalent, TypeSystem > (which uses V1 data-structures), is still being referenced in few places, > like: > - V1 REST APIs, to covert between V1 and V2 data-structures > - notification processing, which uses V1 data-structures > - DSL query implementation > - AtlasClient API > - unit tests > Having to support legacy TypeSystem causes run time overhead in duplicate > caching of type-information - in AtlasTypeRegistry and TypeSystem. Removing > TypeSystem implementation will help eliminate this overhead and also improve > maintainability of Atlas sources. > This JIRA is to track removal of TypeSystem implementation and its > references; where necessary, references should be updated to use > AtlasTypeRegistry equivalent. > - it is very critical to continue to support V1 REST APIs and processing of > current V1 notifications > - existing unit tests that use V1 data-structures will be removed and will be > replaced with equivalents that use V2 data-structures > - AtlasClient in new release will only support V2 data-structures; however, > older version AtlasClient will continue to work with new version of Atlas > server -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2012) Docker - image & hub - for Atlas
[ https://issues.apache.org/jira/browse/ATLAS-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253009#comment-16253009 ] Nigel Jones commented on ATLAS-2012: Looking at docker hub, for an individual to upload an image is pretty simple. However to allow multiple users (within the atlas community) to update the image would require setting up an organization & then adding members A number of projects though including zookeeper, flink, httpd, solr, storm have 'official' images, guidelines for this are at https://github.com/docker-library/official-images. More review is required, but I think in the medium term this is what we should target, this comes with it requirements around aspects like issue tracking In order to get an image uploaded promptly, a pragmatic approach is probably to create an organization (adding any individual interested) and upload as a contributed image in order to learn/verify the process, and once we have something useful that meets the official requirements to work towards migrating the image there. > Docker - image & hub - for Atlas > > > Key: ATLAS-2012 > URL: https://issues.apache.org/jira/browse/ATLAS-2012 > Project: Atlas > Issue Type: New Feature >Reporter: Nigel Jones >Assignee: Nigel Jones > Fix For: 1.0.0 > > Attachments: ATLAS-2012-Build-docker-image.patch, atlas_docker.patch, > atlas_docker1.patch, atlas_docker_2.patch > > > Docker is increasingly become a standard way of easily running components in > a flexible manner, whether for development, production, or test > I feel there are a few things we can do with docker that will aid Atlas's > appeal > 1. We could provide a simple example of how to create a docker image from the > Atlas build. This could be published on the wiki & it would make it easier > for developers to use Atlas within a docker environment - mostly by > addressing how to configure & start up > 2. We could automatically generate a docker image as part of the build > process. This builds on #1 by automating the creation of the image & making > it "just part of the build" > 3. We could publish the docker image from #2, for releases, to dockerhub, > making it near trivial for any developer to easily pull down and experiment > with Atlas. Full source of course would be provided/documented so that a user > could customize as needed for their environment plus of course improve what > is provided with the core project > 4. We could work with other teams especially ranger, to allow similar efforts > in other teams to easily work together & be orchestrated -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Issue Comment Deleted] (ATLAS-2012) Docker - image & hub - for Atlas
[ https://issues.apache.org/jira/browse/ATLAS-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2012: --- Comment: was deleted (was: Looking at docker hub, for an individual to upload an image is pretty simple. However to allow multiple users (within the atlas community) to update the image would require setting up an organization & then adding members A number of projects though including zookeeper, flink, httpd, solr, storm have 'official' images, guidelines for this are at https://github.com/docker-library/official-images. More review is required, but I think in the medium term this is what we should target, this comes with it requirements around aspects like issue tracking In order to get an image uploaded promptly, a pragmatic approach is probably to create an organization (adding any individual interested) and upload as a contributed image in order to learn/verify the process, and once we have something useful that meets the official requirements to work towards migrating the image there. ) > Docker - image & hub - for Atlas > > > Key: ATLAS-2012 > URL: https://issues.apache.org/jira/browse/ATLAS-2012 > Project: Atlas > Issue Type: New Feature >Reporter: Nigel Jones >Assignee: Nigel Jones > Fix For: 1.0.0 > > Attachments: ATLAS-2012-Build-docker-image.patch, atlas_docker.patch, > atlas_docker1.patch, atlas_docker_2.patch > > > Docker is increasingly become a standard way of easily running components in > a flexible manner, whether for development, production, or test > I feel there are a few things we can do with docker that will aid Atlas's > appeal > 1. We could provide a simple example of how to create a docker image from the > Atlas build. This could be published on the wiki & it would make it easier > for developers to use Atlas within a docker environment - mostly by > addressing how to configure & start up > 2. We could automatically generate a docker image as part of the build > process. This builds on #1 by automating the creation of the image & making > it "just part of the build" > 3. We could publish the docker image from #2, for releases, to dockerhub, > making it near trivial for any developer to easily pull down and experiment > with Atlas. Full source of course would be provided/documented so that a user > could customize as needed for their environment plus of course improve what > is provided with the core project > 4. We could work with other teams especially ranger, to allow similar efforts > in other teams to easily work together & be orchestrated -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2094) Make Atlas available in docker hub
[ https://issues.apache.org/jira/browse/ATLAS-2094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16253011#comment-16253011 ] Nigel Jones commented on ATLAS-2094: Looking at docker hub, for an individual to upload an image is pretty simple. However to allow multiple users (within the atlas community) to update the image would require setting up an organization & then adding members A number of projects though including zookeeper, flink, httpd, solr, storm have 'official' images, guidelines for this are at https://github.com/docker-library/official-images. More review is required, but I think in the medium term this is what we should target, this comes with it requirements around aspects like issue tracking In order to get an image uploaded promptly, a pragmatic approach is probably to create an organization (adding any individual interested) and upload as a contributed image in order to learn/verify the process, and once we have something useful that meets the official requirements to work towards migrating the image there. > Make Atlas available in docker hub > -- > > Key: ATLAS-2094 > URL: https://issues.apache.org/jira/browse/ATLAS-2094 > Project: Atlas > Issue Type: New Feature >Reporter: Nigel Jones >Assignee: Nigel Jones > > In ATLAS-2012 we introduced a basic docker image for Atlas which is created > in the build. > In ATLAS-2093 this can be extended with some basic sample data > This Jira will take things a step further. We can make Atlas available on the > docker hub or similar, so that just as with many technologies like Cassandra, > mysql, someone interested in taking a peek at what atlas has to offer can > quickly obtain and run atlas in little more than a single line. > We need to check licensing (I imagine it's ok but need to validate), and > consider what other documentation might be needed, and any extra sample data, > so that someone is given a good impression of what the project offers. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2245) Update Maven dependency in build to 3.5.0
[ https://issues.apache.org/jira/browse/ATLAS-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2245: --- Attachment: (was: ATLAS-2245-1.patch) > Update Maven dependency in build to 3.5.0 > - > > Key: ATLAS-2245 > URL: https://issues.apache.org/jira/browse/ATLAS-2245 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Trivial > Attachments: ATLAS-2245-3.patch > > > The version of maven on the jenkins build was recently updated to address > some build issues. > It seems a few people have hit maven issues so I now propose we update the > main build dependency to 3.5.0 as this is currently in the top level pom as > > 3.0.4 > > and change to 3.5.0 > The very latest is 3.5.2, but since jenkins is now at 3.5.0 and is known > working, plus homebrew (mac) at least only goes as far as 3.5.0, I suggest we > go with that as a prereq. 3.5.2 will work as well > A search through the source tree shows this is the only occurance, so this is > a one file change > NOTE: There is an impact to users, who need to upgrade to maven 3.5. I will > shortly check the wiki docs and would propose an email to the mailing list > remining developers to pull the latest maven -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2245) Update Maven dependency in build to 3.5.0
[ https://issues.apache.org/jira/browse/ATLAS-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2245: --- Attachment: ATLAS-2245-4.patch Updated with new error message as per review ** JAVA VERSION ERROR ** Java 8 (Update 151) or above is required. > Update Maven dependency in build to 3.5.0 > - > > Key: ATLAS-2245 > URL: https://issues.apache.org/jira/browse/ATLAS-2245 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Trivial > Attachments: ATLAS-2245-4.patch > > > The version of maven on the jenkins build was recently updated to address > some build issues. > It seems a few people have hit maven issues so I now propose we update the > main build dependency to 3.5.0 as this is currently in the top level pom as > > 3.0.4 > > and change to 3.5.0 > The very latest is 3.5.2, but since jenkins is now at 3.5.0 and is known > working, plus homebrew (mac) at least only goes as far as 3.5.0, I suggest we > go with that as a prereq. 3.5.2 will work as well > A search through the source tree shows this is the only occurance, so this is > a one file change > NOTE: There is an impact to users, who need to upgrade to maven 3.5. I will > shortly check the wiki docs and would propose an email to the mailing list > remining developers to pull the latest maven -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2245) Update Maven dependency in build to 3.5.0
[ https://issues.apache.org/jira/browse/ATLAS-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2245: --- Attachment: (was: ATLAS-2245-3.patch) > Update Maven dependency in build to 3.5.0 > - > > Key: ATLAS-2245 > URL: https://issues.apache.org/jira/browse/ATLAS-2245 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Trivial > Attachments: ATLAS-2245-4.patch > > > The version of maven on the jenkins build was recently updated to address > some build issues. > It seems a few people have hit maven issues so I now propose we update the > main build dependency to 3.5.0 as this is currently in the top level pom as > > 3.0.4 > > and change to 3.5.0 > The very latest is 3.5.2, but since jenkins is now at 3.5.0 and is known > working, plus homebrew (mac) at least only goes as far as 3.5.0, I suggest we > go with that as a prereq. 3.5.2 will work as well > A search through the source tree shows this is the only occurance, so this is > a one file change > NOTE: There is an impact to users, who need to upgrade to maven 3.5. I will > shortly check the wiki docs and would propose an email to the mailing list > remining developers to pull the latest maven -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2245) Update Maven dependency in build to 3.5.0
[ https://issues.apache.org/jira/browse/ATLAS-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2245: --- Attachment: ATLAS-2245-3.patch Removed additional dependency check in graph pom > Update Maven dependency in build to 3.5.0 > - > > Key: ATLAS-2245 > URL: https://issues.apache.org/jira/browse/ATLAS-2245 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Trivial > Attachments: ATLAS-2245-3.patch > > > The version of maven on the jenkins build was recently updated to address > some build issues. > It seems a few people have hit maven issues so I now propose we update the > main build dependency to 3.5.0 as this is currently in the top level pom as > > 3.0.4 > > and change to 3.5.0 > The very latest is 3.5.2, but since jenkins is now at 3.5.0 and is known > working, plus homebrew (mac) at least only goes as far as 3.5.0, I suggest we > go with that as a prereq. 3.5.2 will work as well > A search through the source tree shows this is the only occurance, so this is > a one file change > NOTE: There is an impact to users, who need to upgrade to maven 3.5. I will > shortly check the wiki docs and would propose an email to the mailing list > remining developers to pull the latest maven -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2260) Ensure BerkeleyDB works with Janus
[ https://issues.apache.org/jira/browse/ATLAS-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16259579#comment-16259579 ] Nigel Jones commented on ATLAS-2260: See my explanation in ATLAS-2084, I think if I do a fix there, it should work with Janus. For now copy the library manually. Suggestion - I can modify a patch to try and do the packaging .. it will still need testing (which can continue here)? > Ensure BerkeleyDB works with Janus > --- > > Key: ATLAS-2260 > URL: https://issues.apache.org/jira/browse/ATLAS-2260 > Project: Atlas > Issue Type: Bug >Reporter: David Radley > > currently Atlas defaults to JanusGraph in master. The Atlas build appears to > bring in the Apache 2 licensed BerkeleyDB JE 7.3.7. I should be possible for > Atlas to be built and run with Janus using this embedded BerkeleyDB. > Currently building with > mvn clean package -Pdist,berkeley-elasticsearch -DskipTests > is clean but Atlas fails to start - as it cannot find a sleepycat DBException > class. > We should consider supporting running Berkeley with solr as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2260) Ensure BerkeleyDB works with Janus
[ https://issues.apache.org/jira/browse/ATLAS-2260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16260465#comment-16260465 ] Nigel Jones commented on ATLAS-2260: Ok... if and when we get the rest of the build sorted, I still expect the proposal in ATLAS-2084 makes sense for incorporation of berkelydb as a local repo if a) we're ok with the license (seemed ok to me) and b) we can't find the lib elsewhere.. However thanks for the explanation, as you say seems there are bigger issues which are being worked first. > Ensure BerkeleyDB works with Janus > --- > > Key: ATLAS-2260 > URL: https://issues.apache.org/jira/browse/ATLAS-2260 > Project: Atlas > Issue Type: Bug >Reporter: David Radley > > currently Atlas defaults to JanusGraph in master. The Atlas build appears to > bring in the Apache 2 licensed BerkeleyDB JE 7.3.7. I should be possible for > Atlas to be built and run with Janus using this embedded BerkeleyDB. > Currently building with > mvn clean package -Pdist,berkeley-elasticsearch -DskipTests > is clean but Atlas fails to start - as it cannot find a sleepycat DBException > class. > We should consider supporting running Berkeley with solr as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2084) Make berkeley DB backed Atlas available without external download
[ https://issues.apache.org/jira/browse/ATLAS-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16251855#comment-16251855 ] Nigel Jones commented on ATLAS-2084: I noticed when building that the berkeley library is now pulled in by Janusgraph: org.janusgraph.janusgraph-berkeleyje:0.1.1 depends on com.sleepycat.je:7.3.7 If so, a) we need to be comfortable this is ok and b) if so, we can potentally abandon this particular change and just update documentation as to how to configure janus to use berkeleydb Comments [~grahamwallis] > Make berkeley DB backed Atlas available without external download > - > > Key: ATLAS-2084 > URL: https://issues.apache.org/jira/browse/ATLAS-2084 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: ATLAS_2084_Add_BerkeleyDB_libraries.patch > > > In https://issues.apache.org/jira/browse/ATLAS-2012 I proposed providing a > dockerized version of Atlas to aid people getting up to speed with the > technology. During the review it was noted that we had used Hbase for storage > rather than the more lightweight, simpler, BerkeleyDB. My concern had been > around licensing. > Today when we build Atlas following the instructions at > http://atlas.apache.org/InstallationSteps.html the user is directed to > download the berkeleyDB jar from oracle. In ATLAS-2012 David/Graham pointed > out that this DB appears to be Apache 2.0 licensed, which means we could > incorporate it into our build legitimately. > Given the necessity to get licensing correct I've opened up this new JIRA to > focus on that change. We would need to > * Validate we are indeed correct about the license > * Add the DB file into the build > * Update the web instructions > * And optionally as per the original JIRA, consider making B-DB the default > for our docker image (most lightweight). > Please add your thoughts. I'm happy to get and do this change . -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2245) Update Maven dependency in build to 3.5.0
[ https://issues.apache.org/jira/browse/ATLAS-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2245: --- Attachment: ATLAS-2245.patch > Update Maven dependency in build to 3.5.0 > - > > Key: ATLAS-2245 > URL: https://issues.apache.org/jira/browse/ATLAS-2245 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: ATLAS-2245.patch > > > The version of maven on the jenkins build was recently updated to address > some build issues. > It seems a few people have hit maven issues so I now propose we update the > main build dependency to 3.5.0 as this is currently in the top level pom as > > 3.0.4 > > and change to 3.5.0 > The very latest is 3.5.2, but since jenkins is now at 3.5.0 and is known > working, plus homebrew (mac) at least only goes as far as 3.5.0, I suggest we > go with that as a prereq. 3.5.2 will work as well > A search through the source tree shows this is the only occurance, so this is > a one file change > NOTE: There is an impact to users, who need to upgrade to maven 3.5. I will > shortly check the wiki docs and would propose an email to the mailing list > remining developers to pull the latest maven -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ATLAS-2245) Update Maven dependency in build to 3.5.0
[ https://issues.apache.org/jira/browse/ATLAS-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones reassigned ATLAS-2245: -- Assignee: Nigel Jones > Update Maven dependency in build to 3.5.0 > - > > Key: ATLAS-2245 > URL: https://issues.apache.org/jira/browse/ATLAS-2245 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones > > The version of maven on the jenkins build was recently updated to address > some build issues. > It seems a few people have hit maven issues so I now propose we update the > main build dependency to 3.5.0 as this is currently in the top level pom as > > 3.0.4 > > and change to 3.5.0 > The very latest is 3.5.2, but since jenkins is now at 3.5.0 and is known > working, plus homebrew (mac) at least only goes as far as 3.5.0, I suggest we > go with that as a prereq. 3.5.2 will work as well > A search through the source tree shows this is the only occurance, so this is > a one file change > NOTE: There is an impact to users, who need to upgrade to maven 3.5. I will > shortly check the wiki docs and would propose an email to the mailing list > remining developers to pull the latest maven -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2245) Update Maven dependency in build to 3.5.0
[ https://issues.apache.org/jira/browse/ATLAS-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233981#comment-16233981 ] Nigel Jones commented on ATLAS-2245: proposing this as I think it will make our builds more consistent between developers as there have been some unknowns that may well be caused by maven version differences > Update Maven dependency in build to 3.5.0 > - > > Key: ATLAS-2245 > URL: https://issues.apache.org/jira/browse/ATLAS-2245 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: ATLAS-2245.patch > > > The version of maven on the jenkins build was recently updated to address > some build issues. > It seems a few people have hit maven issues so I now propose we update the > main build dependency to 3.5.0 as this is currently in the top level pom as > > 3.0.4 > > and change to 3.5.0 > The very latest is 3.5.2, but since jenkins is now at 3.5.0 and is known > working, plus homebrew (mac) at least only goes as far as 3.5.0, I suggest we > go with that as a prereq. 3.5.2 will work as well > A search through the source tree shows this is the only occurance, so this is > a one file change > NOTE: There is an impact to users, who need to upgrade to maven 3.5. I will > shortly check the wiki docs and would propose an email to the mailing list > remining developers to pull the latest maven -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2245) Update Maven dependency in build to 3.5.0
[ https://issues.apache.org/jira/browse/ATLAS-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2245: --- Attachment: (was: ATLAS-2245.patch) > Update Maven dependency in build to 3.5.0 > - > > Key: ATLAS-2245 > URL: https://issues.apache.org/jira/browse/ATLAS-2245 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones > > The version of maven on the jenkins build was recently updated to address > some build issues. > It seems a few people have hit maven issues so I now propose we update the > main build dependency to 3.5.0 as this is currently in the top level pom as > > 3.0.4 > > and change to 3.5.0 > The very latest is 3.5.2, but since jenkins is now at 3.5.0 and is known > working, plus homebrew (mac) at least only goes as far as 3.5.0, I suggest we > go with that as a prereq. 3.5.2 will work as well > A search through the source tree shows this is the only occurance, so this is > a one file change > NOTE: There is an impact to users, who need to upgrade to maven 3.5. I will > shortly check the wiki docs and would propose an email to the mailing list > remining developers to pull the latest maven -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2245) Update Maven dependency in build to 3.5.0
[ https://issues.apache.org/jira/browse/ATLAS-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234227#comment-16234227 ] Nigel Jones commented on ATLAS-2245: If Maven is not at 3.5.2 above: [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @ apache-atlas --- [WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireMavenVersion failed with message: ** Please upgrade maven to 3.5.2 or above. See https://maven.apache.org/install.html Similar for Java. > Update Maven dependency in build to 3.5.0 > - > > Key: ATLAS-2245 > URL: https://issues.apache.org/jira/browse/ATLAS-2245 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: ATLAS-2245.patch > > > The version of maven on the jenkins build was recently updated to address > some build issues. > It seems a few people have hit maven issues so I now propose we update the > main build dependency to 3.5.0 as this is currently in the top level pom as > > 3.0.4 > > and change to 3.5.0 > The very latest is 3.5.2, but since jenkins is now at 3.5.0 and is known > working, plus homebrew (mac) at least only goes as far as 3.5.0, I suggest we > go with that as a prereq. 3.5.2 will work as well > A search through the source tree shows this is the only occurance, so this is > a one file change > NOTE: There is an impact to users, who need to upgrade to maven 3.5. I will > shortly check the wiki docs and would propose an email to the mailing list > remining developers to pull the latest maven -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2245) Update Maven dependency in build to 3.5.0
[ https://issues.apache.org/jira/browse/ATLAS-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2245: --- Attachment: (was: ATLAS-2245.patch) > Update Maven dependency in build to 3.5.0 > - > > Key: ATLAS-2245 > URL: https://issues.apache.org/jira/browse/ATLAS-2245 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones > > The version of maven on the jenkins build was recently updated to address > some build issues. > It seems a few people have hit maven issues so I now propose we update the > main build dependency to 3.5.0 as this is currently in the top level pom as > > 3.0.4 > > and change to 3.5.0 > The very latest is 3.5.2, but since jenkins is now at 3.5.0 and is known > working, plus homebrew (mac) at least only goes as far as 3.5.0, I suggest we > go with that as a prereq. 3.5.2 will work as well > A search through the source tree shows this is the only occurance, so this is > a one file change > NOTE: There is an impact to users, who need to upgrade to maven 3.5. I will > shortly check the wiki docs and would propose an email to the mailing list > remining developers to pull the latest maven -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (ATLAS-2245) Update Maven dependency in build to 3.5.0
[ https://issues.apache.org/jira/browse/ATLAS-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2245: --- Attachment: ATLAS-2245.patch corrected typo in error message > Update Maven dependency in build to 3.5.0 > - > > Key: ATLAS-2245 > URL: https://issues.apache.org/jira/browse/ATLAS-2245 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones > Attachments: ATLAS-2245.patch > > > The version of maven on the jenkins build was recently updated to address > some build issues. > It seems a few people have hit maven issues so I now propose we update the > main build dependency to 3.5.0 as this is currently in the top level pom as > > 3.0.4 > > and change to 3.5.0 > The very latest is 3.5.2, but since jenkins is now at 3.5.0 and is known > working, plus homebrew (mac) at least only goes as far as 3.5.0, I suggest we > go with that as a prereq. 3.5.2 will work as well > A search through the source tree shows this is the only occurance, so this is > a one file change > NOTE: There is an impact to users, who need to upgrade to maven 3.5. I will > shortly check the wiki docs and would propose an email to the mailing list > remining developers to pull the latest maven -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-2084) Make berkeley DB backed Atlas available without external download
[ https://issues.apache.org/jira/browse/ATLAS-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16218603#comment-16218603 ] Nigel Jones edited comment on ATLAS-2084 at 10/25/17 1:20 PM: -- http://www.oracle.com/technetwork/database/berkeleydb/downloads/maven-087630.html provides references to the maven repo My plan is to modify the POMs appropriately (when b-db is a target) to point to this repo. Hopefully this is ok. An alternative is to include the binary .jar, but I think using maven is clearer, and simpler, and keeps the source tree cleaner I will also add the license/ref into 3party-licenses, and the top level licence file Will create a patch once done. was (Author: jonesn): http://www.oracle.com/technetwork/database/berkeleydb/downloads/maven-087630.html provides references to the maven repo My plan is to modify the POMs appropriately (when b-db is a target) to point to this repo. Hopefully this is ok I will also add the license/ref into 3party-licenses, and the top level licence file Will create a patch once done. > Make berkeley DB backed Atlas available without external download > - > > Key: ATLAS-2084 > URL: https://issues.apache.org/jira/browse/ATLAS-2084 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones > > In https://issues.apache.org/jira/browse/ATLAS-2012 I proposed providing a > dockerized version of Atlas to aid people getting up to speed with the > technology. During the review it was noted that we had used Hbase for storage > rather than the more lightweight, simpler, BerkeleyDB. My concern had been > around licensing. > Today when we build Atlas following the instructions at > http://atlas.apache.org/InstallationSteps.html the user is directed to > download the berkeleyDB jar from oracle. In ATLAS-2012 David/Graham pointed > out that this DB appears to be Apache 2.0 licensed, which means we could > incorporate it into our build legitimately. > Given the necessity to get licensing correct I've opened up this new JIRA to > focus on that change. We would need to > * Validate we are indeed correct about the license > * Add the DB file into the build > * Update the web instructions > * And optionally as per the original JIRA, consider making B-DB the default > for our docker image (most lightweight). > Please add your thoughts. I'm happy to get and do this change . -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2084) Make berkeley DB backed Atlas available without external download
[ https://issues.apache.org/jira/browse/ATLAS-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16218603#comment-16218603 ] Nigel Jones commented on ATLAS-2084: http://www.oracle.com/technetwork/database/berkeleydb/downloads/maven-087630.html provides references to the maven repo My plan is to modify the POMs appropriately (when b-db is a target) to point to this repo. Hopefully this is ok I will also add the license/ref into 3party-licenses, and the top level licence file Will create a patch once done. > Make berkeley DB backed Atlas available without external download > - > > Key: ATLAS-2084 > URL: https://issues.apache.org/jira/browse/ATLAS-2084 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones > > In https://issues.apache.org/jira/browse/ATLAS-2012 I proposed providing a > dockerized version of Atlas to aid people getting up to speed with the > technology. During the review it was noted that we had used Hbase for storage > rather than the more lightweight, simpler, BerkeleyDB. My concern had been > around licensing. > Today when we build Atlas following the instructions at > http://atlas.apache.org/InstallationSteps.html the user is directed to > download the berkeleyDB jar from oracle. In ATLAS-2012 David/Graham pointed > out that this DB appears to be Apache 2.0 licensed, which means we could > incorporate it into our build legitimately. > Given the necessity to get licensing correct I've opened up this new JIRA to > focus on that change. We would need to > * Validate we are indeed correct about the license > * Add the DB file into the build > * Update the web instructions > * And optionally as per the original JIRA, consider making B-DB the default > for our docker image (most lightweight). > Please add your thoughts. I'm happy to get and do this change . -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2084) Make berkeley DB backed Atlas available without external download
[ https://issues.apache.org/jira/browse/ATLAS-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16218648#comment-16218648 ] Nigel Jones commented on ATLAS-2084: Made changes to the distro/pom.xl to define a pom delta specific to the berkeleydb-elasticsearch profile to use Oracle maven repo, but it turns out the repo requires authentication via an oracle id - not acceptable for our generic build. Therefore the only option is to include the .jar (which should be fine as it's ASF 2.0 licensed). Plan B ... > Make berkeley DB backed Atlas available without external download > - > > Key: ATLAS-2084 > URL: https://issues.apache.org/jira/browse/ATLAS-2084 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones > > In https://issues.apache.org/jira/browse/ATLAS-2012 I proposed providing a > dockerized version of Atlas to aid people getting up to speed with the > technology. During the review it was noted that we had used Hbase for storage > rather than the more lightweight, simpler, BerkeleyDB. My concern had been > around licensing. > Today when we build Atlas following the instructions at > http://atlas.apache.org/InstallationSteps.html the user is directed to > download the berkeleyDB jar from oracle. In ATLAS-2012 David/Graham pointed > out that this DB appears to be Apache 2.0 licensed, which means we could > incorporate it into our build legitimately. > Given the necessity to get licensing correct I've opened up this new JIRA to > focus on that change. We would need to > * Validate we are indeed correct about the license > * Add the DB file into the build > * Update the web instructions > * And optionally as per the original JIRA, consider making B-DB the default > for our docker image (most lightweight). > Please add your thoughts. I'm happy to get and do this change . -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2226) ATLAS build fails (20171025) - macos
[ https://issues.apache.org/jira/browse/ATLAS-2226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16218433#comment-16218433 ] Nigel Jones commented on ATLAS-2226: Confirming this occured with mvn clean package -Pdist,embedded-hbase-solr also However a short while later all are now working. No pull/rebase done since earlier attempt. Environment? Caching in relation to maven 3.5.2? > ATLAS build fails (20171025) - macos > > > Key: ATLAS-2226 > URL: https://issues.apache.org/jira/browse/ATLAS-2226 > Project: Atlas > Issue Type: Bug >Reporter: Nigel Jones > > I refreshed my build environment today, and noted that atlas fails to build > (for embedded hbase/solr). I took a fresh git clone to confirm. JDK was > updated to the 152 fix level (java 8) a few days ago. Maven is at 3.5 for me > and may need to be updated. > 11:51 $ git clone git://git.apache.org/atlas.git atlas2 > Cloning into 'atlas2'... > remote: Counting objects: 55084, done. > remote: Compressing objects: 100% (18742/18742), done. > remote: Total 55084 (delta 24065), reused 49458 (delta 21532) > Receiving objects: 100% (55084/55084), 33.37 MiB | 7.29 MiB/s, done. > Resolving deltas: 100% (24065/24065), done. > ✔ ~/src > 11:51 $ cd atlas2 > ✔ ~/src/atlas2 [master|✔] > 11:51 $ mvn clean package -Pdist,external-hbase-solr 2>&1 ~ tee > ../20171025a.log > [INFO] Scanning for projects... > [WARNING] The project org.apache.atlas:apache-atlas:pom:1.0.0-SNAPSHOT uses > prerequisites which is only intended for maven-plugin projects but not for > non maven-plugin projects. For such purposes you should use the > maven-enforcer-plugin. See > https://maven.apache.org/enforcer/enforcer-rules/requireMavenVersion.html > [INFO] > > [INFO] Reactor Build Order: > [INFO] > [INFO] Apache Atlas Server Build Tools > [INFO] apache-atlas > [INFO] Apache Atlas Integration > [INFO] Apache Atlas Common > [INFO] Apache Atlas Typesystem > [INFO] Apache Atlas Client > [INFO] atlas-client-common > [INFO] atlas-client-v1 > [INFO] Apache Atlas Server API > [INFO] Apache Atlas Notification > [INFO] atlas-client-v2 > [INFO] Apache Atlas Graph Database Projects > [INFO] Apache Atlas Graph Database API > [INFO] Graph Database Common Code > [INFO] Apache Atlas Titan 1.0.0 GraphDB Impl > [INFO] Shaded version of Apache hbase client > [INFO] Apache Atlas Titan 0.5.4 Graph DB Impl > [INFO] Apache Atlas Graph Database Implementation Dependencies > [INFO] Shaded version of Apache hbase server > [INFO] Apache Atlas Repository > [INFO] Apache Atlas Authorization > [INFO] Apache Atlas UI > [INFO] Apache Atlas Web Application > [INFO] Apache Atlas Documentation > [INFO] Apache Atlas FileSystem Model > [INFO] Apache Atlas Plugin Classloader > [INFO] Apache Atlas Hive Bridge Shim > [INFO] Apache Atlas Hive Bridge > [INFO] Apache Atlas Falcon Bridge Shim > [INFO] Apache Atlas Falcon Bridge > [INFO] Apache Atlas Sqoop Bridge Shim > [INFO] Apache Atlas Sqoop Bridge > [INFO] Apache Atlas Storm Bridge Shim > [INFO] Apache Atlas Storm Bridge > [INFO] Apache Atlas Hbase Bridge Shim > [INFO] Apache Atlas Hbase Bridge > [INFO] Apache Atlas Distribution > [INFO] > [INFO] > > [INFO] Building Apache Atlas Server Build Tools 1.0 > [INFO] > > [INFO] > > [INFO] Reactor Summary: > [INFO] > [INFO] Apache Atlas Server Build Tools FAILURE [ 0.002 > s] > [INFO] apache-atlas ... SKIPPED > [INFO] Apache Atlas Integration ... SKIPPED > [INFO] Apache Atlas Common SKIPPED > [INFO] Apache Atlas Typesystem SKIPPED > [INFO] Apache Atlas Client SKIPPED > [INFO] atlas-client-common SKIPPED > [INFO] atlas-client-v1 SKIPPED > [INFO] Apache Atlas Server API SKIPPED > [INFO] Apache Atlas Notification .. SKIPPED > [INFO] atlas-client-v2 SKIPPED > [INFO] Apache Atlas Graph Database Projects ... SKIPPED > [INFO] Apache Atlas Graph Database API SKIPPED > [INFO] Graph Database Common Code . SKIPPED > [INFO] Apache Atlas Titan 1.0.0 GraphDB Impl .. SKIPPED > [INFO] Shaded version of Apache hbase client .. SKIPPED > [INFO] Apache Atlas Titan 0.5.4 Graph DB Impl . SKIPPED > [INFO] Apache Atlas Graph Database Implementation Dependencies
[jira] [Reopened] (ATLAS-2084) Make berkeley DB backed Atlas available without external download
[ https://issues.apache.org/jira/browse/ATLAS-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones reopened ATLAS-2084: Reopening. Confirmed je-7.4.5.tar contains an ASF 2.0 license (not gnu) Had opened db-6.2.32 (native) initially and not noticed the duplicate file names in editor -- this native code version is licensed under gnu Reopening to propose addition to source tree & will create patch > Make berkeley DB backed Atlas available without external download > - > > Key: ATLAS-2084 > URL: https://issues.apache.org/jira/browse/ATLAS-2084 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones > > In https://issues.apache.org/jira/browse/ATLAS-2012 I proposed providing a > dockerized version of Atlas to aid people getting up to speed with the > technology. During the review it was noted that we had used Hbase for storage > rather than the more lightweight, simpler, BerkeleyDB. My concern had been > around licensing. > Today when we build Atlas following the instructions at > http://atlas.apache.org/InstallationSteps.html the user is directed to > download the berkeleyDB jar from oracle. In ATLAS-2012 David/Graham pointed > out that this DB appears to be Apache 2.0 licensed, which means we could > incorporate it into our build legitimately. > Given the necessity to get licensing correct I've opened up this new JIRA to > focus on that change. We would need to > * Validate we are indeed correct about the license > * Add the DB file into the build > * Update the web instructions > * And optionally as per the original JIRA, consider making B-DB the default > for our docker image (most lightweight). > Please add your thoughts. I'm happy to get and do this change . -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (ATLAS-2084) Make berkeley DB backed Atlas available without external download
[ https://issues.apache.org/jira/browse/ATLAS-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones reassigned ATLAS-2084: -- Assignee: Nigel Jones > Make berkeley DB backed Atlas available without external download > - > > Key: ATLAS-2084 > URL: https://issues.apache.org/jira/browse/ATLAS-2084 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones > > In https://issues.apache.org/jira/browse/ATLAS-2012 I proposed providing a > dockerized version of Atlas to aid people getting up to speed with the > technology. During the review it was noted that we had used Hbase for storage > rather than the more lightweight, simpler, BerkeleyDB. My concern had been > around licensing. > Today when we build Atlas following the instructions at > http://atlas.apache.org/InstallationSteps.html the user is directed to > download the berkeleyDB jar from oracle. In ATLAS-2012 David/Graham pointed > out that this DB appears to be Apache 2.0 licensed, which means we could > incorporate it into our build legitimately. > Given the necessity to get licensing correct I've opened up this new JIRA to > focus on that change. We would need to > * Validate we are indeed correct about the license > * Add the DB file into the build > * Update the web instructions > * And optionally as per the original JIRA, consider making B-DB the default > for our docker image (most lightweight). > Please add your thoughts. I'm happy to get and do this change . -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2084) Make berkeley DB backed Atlas available without external download
[ https://issues.apache.org/jira/browse/ATLAS-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16218365#comment-16218365 ] Nigel Jones commented on ATLAS-2084: The main Berkeley DB page is at http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/overview/index.html The latest version is 12.1.6.2.32 - pulling the tar apart we see that the library is licensed under the GNU AFFERO GENERAL PUBLIC LICENSE Version 3 Referring to https://www.apache.org/legal/resolved.html it's clear the gnu affero license IS NOT PERMITTED for being bundled with apache projects For this reason I will have to close this JIRA - it seems we cannot include the berkeleyDB libraries within Atlas. Not a huge issue as this is primarily for devs, and other persistance options can be used. Closing. > Make berkeley DB backed Atlas available without external download > - > > Key: ATLAS-2084 > URL: https://issues.apache.org/jira/browse/ATLAS-2084 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones > > In https://issues.apache.org/jira/browse/ATLAS-2012 I proposed providing a > dockerized version of Atlas to aid people getting up to speed with the > technology. During the review it was noted that we had used Hbase for storage > rather than the more lightweight, simpler, BerkeleyDB. My concern had been > around licensing. > Today when we build Atlas following the instructions at > http://atlas.apache.org/InstallationSteps.html the user is directed to > download the berkeleyDB jar from oracle. In ATLAS-2012 David/Graham pointed > out that this DB appears to be Apache 2.0 licensed, which means we could > incorporate it into our build legitimately. > Given the necessity to get licensing correct I've opened up this new JIRA to > focus on that change. We would need to > * Validate we are indeed correct about the license > * Add the DB file into the build > * Update the web instructions > * And optionally as per the original JIRA, consider making B-DB the default > for our docker image (most lightweight). > Please add your thoughts. I'm happy to get and do this change . -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (ATLAS-2084) Make berkeley DB backed Atlas available without external download
[ https://issues.apache.org/jira/browse/ATLAS-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16218365#comment-16218365 ] Nigel Jones edited comment on ATLAS-2084 at 10/25/17 10:40 AM: --- The main Berkeley DB page is at http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/overview/index.html The latest version is 7.4.5 (JE) - pulling the tar apart we see that the library is licensed under the GNU AFFERO GENERAL PUBLIC LICENSE Version 3 Referring to https://www.apache.org/legal/resolved.html it's clear the gnu affero license IS NOT PERMITTED for being bundled with apache projects For this reason I will have to close this JIRA - it seems we cannot include the berkeleyDB libraries within Atlas. Not a huge issue as this is primarily for devs, and other persistance options can be used. Closing. was (Author: jonesn): The main Berkeley DB page is at http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/overview/index.html The latest version is 12.1.6.2.32 - pulling the tar apart we see that the library is licensed under the GNU AFFERO GENERAL PUBLIC LICENSE Version 3 Referring to https://www.apache.org/legal/resolved.html it's clear the gnu affero license IS NOT PERMITTED for being bundled with apache projects For this reason I will have to close this JIRA - it seems we cannot include the berkeleyDB libraries within Atlas. Not a huge issue as this is primarily for devs, and other persistance options can be used. Closing. > Make berkeley DB backed Atlas available without external download > - > > Key: ATLAS-2084 > URL: https://issues.apache.org/jira/browse/ATLAS-2084 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones > > In https://issues.apache.org/jira/browse/ATLAS-2012 I proposed providing a > dockerized version of Atlas to aid people getting up to speed with the > technology. During the review it was noted that we had used Hbase for storage > rather than the more lightweight, simpler, BerkeleyDB. My concern had been > around licensing. > Today when we build Atlas following the instructions at > http://atlas.apache.org/InstallationSteps.html the user is directed to > download the berkeleyDB jar from oracle. In ATLAS-2012 David/Graham pointed > out that this DB appears to be Apache 2.0 licensed, which means we could > incorporate it into our build legitimately. > Given the necessity to get licensing correct I've opened up this new JIRA to > focus on that change. We would need to > * Validate we are indeed correct about the license > * Add the DB file into the build > * Update the web instructions > * And optionally as per the original JIRA, consider making B-DB the default > for our docker image (most lightweight). > Please add your thoughts. I'm happy to get and do this change . -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (ATLAS-2593) Governance Engine OMAS - skeleton based on new omrs/omas patterns/ & framework
[ https://issues.apache.org/jira/browse/ATLAS-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16464024#comment-16464024 ] Nigel Jones commented on ATLAS-2593: Added first patch. - Based on outline provided in omas-connectedasset & omas-assetconsumer - Provides framework for two initial API calls - Not executed/tested/no unit tests so far - No implementation to retrieve actual tags/classifications from metadata store - lots of todos/further reading through/checks will post on review board once API calls work (even returning dummys) > Governance Engine OMAS - skeleton based on new omrs/omas patterns/ & framework > -- > > Key: ATLAS-2593 > URL: https://issues.apache.org/jira/browse/ATLAS-2593 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Attachments: ATLAS-2593.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2593) Governance Engine OMAS - skeleton based on new omrs/omas patterns/ & framework
[ https://issues.apache.org/jira/browse/ATLAS-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2593: --- Attachment: ATLAS-2593.patch > Governance Engine OMAS - skeleton based on new omrs/omas patterns/ & framework > -- > > Key: ATLAS-2593 > URL: https://issues.apache.org/jira/browse/ATLAS-2593 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Attachments: ATLAS-2593.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-2668) Add OMAG Server to distribution
[ https://issues.apache.org/jira/browse/ATLAS-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470379#comment-16470379 ] Nigel Jones commented on ATLAS-2668: Originally used the following pom.xml fragment which has worked before... but for other standalone apps. It doesn't work for spring, so I will use the spring framework version, but adding here as a useful note on making a standalone app with dependencies and main class identified org.apache.maven.plugins maven-shade-plugin 3.1.1 false package shade org.apache.atlas.omag.application.OMAGApplication In this case I'll use org.springframework.boot spring-boot-maven-plugin 1.5.10.RELEASE repackage This fragment will build a standalone jar, so we can now do: ✔ ~/src/atlas/omag-server/target [master|✚ 1⚑ 3] 14:07 $ ls classes maven-shared-archive-resources original-omag-server-1.0.0-SNAPSHOT.jar generated-sources maven-status rat.txt maven-archiver omag-server-1.0.0-SNAPSHOT.jar test-classes ✔ ~/src/atlas/omag-server/target [master|✚ 1⚑ 3] 14:07 $ java -jar ./omag-server-1.0.0-SNAPSHOT.jar . _ __ _ _ /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \ ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \ \\/ ___)| |_)| | | | | || (_| | ) ) ) ) ' || .__|_| |_|_| |_\__, | / / / / =|_|==|___/=/_/_/_/ :: Spring Boot :: (v1.0.0-SNAPSHOT) 2018-05-10 14:07:12.047 INFO 36215 --- [ main] o.a.a.omag.application.OMAGApplication : Starting OMAGApplication v1.0.0-SNAPSHOT on nigels-mbp.hursley.uk.ibm.com with PID 36215 (/Users/jonesn/IdeaProjects/atlas/omag-server/target/omag-server-1.0.0-SNAPSHOT.jar started by jonesn in /Users/jonesn/IdeaProjects/atlas/omag-server/target) 2018-05-10 14:07:12.050 INFO 36215 --- [ main] o.a.a.omag.application.OMAGApplication : No active profile set, falling back to default profiles: default etc ... That's build done. Next part - place somewhere in distribution... > Add OMAG Server to distribution > --- > > Key: ATLAS-2668 > URL: https://issues.apache.org/jira/browse/ATLAS-2668 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > > The standalone Open Metadata server is built under omag-server. > However it is not incorporated into the distribution files (*.gz), and needs > careful attention to the classpath to work (ie launch from within IntelliJ) > The intent of this Jira is > * have OMAG server available 'out of the box' in a binary distribution so > that users can easily launch it without having to check classpaths or run in > an odd environment > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (ATLAS-2569) maven dependency plugin back level - issues with dependency tree check
[ https://issues.apache.org/jira/browse/ATLAS-2569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones closed ATLAS-2569. -- > maven dependency plugin back level - issues with dependency tree check > -- > > Key: ATLAS-2569 > URL: https://issues.apache.org/jira/browse/ATLAS-2569 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Fix For: 1.0.0 > > Attachments: ATLAS-2569.patch > > > The maven dependency plugin is at v2, yet we are using maven v3. This can > lead to incorrect results when checking the dependency tree. > The error producted from 'mvn dependency:tree' includes: > [INFO] --- maven-dependency-plugin:2.10:tree (default-cli) @ atlas-client-v1 > --- > [WARNING] Using Maven 2 dependency tree to get verbose output, which may be > inconsistent with actual Maven 3 resolution > The fix is to update the plugin to a later version. Patch to be added. No > warnings are then produced, and we should be more sure of getting proper > dependency tree reporting -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (ATLAS-2571) Maven: duplicate declarations
[
https://issues.apache.org/jira/browse/ATLAS-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nigel Jones closed ATLAS-2571.
--
> Maven: duplicate declarations
> -
>
> Key: ATLAS-2571
> URL: https://issues.apache.org/jira/browse/ATLAS-2571
> Project: Atlas
> Issue Type: Improvement
>Reporter: Nigel Jones
>Assignee: Nigel Jones
>Priority: Minor
> Fix For: 1.0.0
>
> Attachments: ATLAS-2571.patch, ATLAS-2571.patch
>
>
> [INFO] Scanning for projects...
> [WARNING]
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.atlas:atlas-graphdb-titan0:jar:1.0.0-SNAPSHOT
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must
> be unique: org.apache.solr:solr-solrj:jar -> version (?) vs ${solr.version} @
> org.apache.atlas:atlas-graphdb-titan0:[unknown-version],
> /Users/jonesn/IdeaProjects/atlas/graphdb/titan0/pom.xml, line 194, column 21
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must
> be unique: org.apache.solr:solr-core:jar -> version (?) vs ${solr.version} @
> org.apache.atlas:atlas-graphdb-titan0:[unknown-version],
> /Users/jonesn/IdeaProjects/atlas/graphdb/titan0/pom.xml, line 200, column 21
> [WARNING]
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.atlas:hive-bridge:jar:1.0.0-SNAPSHOT
> [WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but
> found duplicate declaration of plugin
> org.apache.maven.plugins:maven-resources-plugin @
> org.apache.atlas:hive-bridge:[unknown-version],
> /Users/jonesn/IdeaProjects/atlas/addons/hive-bridge/pom.xml, line 484, column
> 21
> [WARNING]
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.atlas:falcon-bridge:jar:1.0.0-SNAPSHOT
> [WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but
> found duplicate declaration of plugin
> org.apache.maven.plugins:maven-resources-plugin @
> org.apache.atlas:falcon-bridge:[unknown-version],
> /Users/jonesn/IdeaProjects/atlas/addons/falcon-bridge/pom.xml, line 370,
> column 21
> [WARNING]
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.atlas:sqoop-bridge:jar:1.0.0-SNAPSHOT
> [WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but
> found duplicate declaration of plugin
> org.apache.maven.plugins:maven-resources-plugin @
> org.apache.atlas:sqoop-bridge:[unknown-version],
> /Users/jonesn/IdeaProjects/atlas/addons/sqoop-bridge/pom.xml, line 433,
> column 21
> [WARNING]
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.atlas:storm-bridge:jar:1.0.0-SNAPSHOT
> [WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but
> found duplicate declaration of plugin
> org.apache.maven.plugins:maven-resources-plugin @
> org.apache.atlas:storm-bridge:[unknown-version],
> /Users/jonesn/IdeaProjects/atlas/addons/storm-bridge/pom.xml, line 536,
> column 21
> [WARNING]
> [WARNING] Some problems were encountered while building the effective model
> for org.apache.atlas:hbase-bridge:jar:1.0.0-SNAPSHOT
> [WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but
> found duplicate declaration of plugin
> org.apache.maven.plugins:maven-resources-plugin @
> org.apache.atlas:hbase-bridge:[unknown-version],
> /Users/jonesn/IdeaProjects/atlas/addons/hbase-bridge/pom.xml, line 508,
> column 21
> [WARNING]
> [WARNING] It is highly recommended to fix these problems because they
> threaten the stability of your build.
> [WARNING]
> [WARNING] For this reason, future Maven versions might no longer support
> building such malformed projects.
> [WARNING]
> [WARNING] The project org.apache.atlas:apache-atlas:pom:1.0.0-SNAPSHOT uses
> prerequisites which is only intended for maven-plugin projects but not for
> non maven-plugin projects. For such purposes you should use the
> maven-enforcer-plugin. See
> [https://maven.apache.org/enforcer/enforcer-rules/requireMavenVersion.html]
> [INFO]
>
>
>
> These are generally caused by duplicates - for example taking the first:
>
>
> org.apache.solr
> solr-solrj
>
>
> org.apache.solr
> solr-solrj
> ${solr.version}
>
>
> So we have it declared twice, once pulling in solr.version - defined in the
> top level pom, and the other blank. In the case of this plugin, the version
> is not declared in any DependencyManagement section (the common way of
> defining default plugin versions which can then be overriden in child poms)
> Cleaning this up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (ATLAS-2668) Add OMAG Server to distribution
[ https://issues.apache.org/jira/browse/ATLAS-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones reassigned ATLAS-2668: -- Assignee: Nigel Jones > Add OMAG Server to distribution > --- > > Key: ATLAS-2668 > URL: https://issues.apache.org/jira/browse/ATLAS-2668 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > > The standalone Open Metadata server is built under omag-server. > However it is not incorporated into the distribution files (*.gz), and needs > careful attention to the classpath to work (ie launch from within IntelliJ) > The intent of this Jira is > * have OMAG server available 'out of the box' in a binary distribution so > that users can easily launch it without having to check classpaths or run in > an odd environment > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-2668) Add OMAG Server to distribution
Nigel Jones created ATLAS-2668: -- Summary: Add OMAG Server to distribution Key: ATLAS-2668 URL: https://issues.apache.org/jira/browse/ATLAS-2668 Project: Atlas Issue Type: Improvement Reporter: Nigel Jones The standalone Open Metadata server is built under omag-server. However it is not incorporated into the distribution files (*.gz), and needs careful attention to the classpath to work (ie launch from within IntelliJ) The intent of this Jira is * have OMAG server available 'out of the box' in a binary distribution so that users can easily launch it without having to check classpaths or run in an odd environment -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-2668) Add OMAG Server to distribution
[ https://issues.apache.org/jira/browse/ATLAS-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470631#comment-16470631 ] Nigel Jones commented on ATLAS-2668: Now added to distribution assemblies. Assembly includes main jar file & a new README.md which just documents how to launch OMAG Server After patch installed new files are created: ie 17:18 $ ls README.md omag-server-1.0.0-SNAPSHOT.jar ✔ ~/src/atlas/distro/target/apache-atlas-1.0.0-SNAPSHOT-omag-server/omag-server-1.0.0-SNAPSHOT [master ↑·1|…1⚑ 3] This mirrors the scheme used for other packages A subsequent Jira could consider adding specific launch/terminate scripts for the omag server. Looking at the atlas scripts these make use of some common python libs which manages atlas config - in python (easier for platform independence) so suggest defer for now to consider what the requirements are. This Jira does make it easier to launch OMAG server with default settings. > Add OMAG Server to distribution > --- > > Key: ATLAS-2668 > URL: https://issues.apache.org/jira/browse/ATLAS-2668 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Attachments: ATLAS-2668.patch > > > The standalone Open Metadata server is built under omag-server. > However it is not incorporated into the distribution files (*.gz), and needs > careful attention to the classpath to work (ie launch from within IntelliJ) > The intent of this Jira is > * have OMAG server available 'out of the box' in a binary distribution so > that users can easily launch it without having to check classpaths or run in > an odd environment > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-2668) Add OMAG Server to distribution
[ https://issues.apache.org/jira/browse/ATLAS-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16473412#comment-16473412 ] Nigel Jones commented on ATLAS-2668: Updated patch to match review board > Add OMAG Server to distribution > --- > > Key: ATLAS-2668 > URL: https://issues.apache.org/jira/browse/ATLAS-2668 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Attachments: ATLAS-2668e.patch > > > The standalone Open Metadata server is built under omag-server. > However it is not incorporated into the distribution files (*.gz), and needs > careful attention to the classpath to work (ie launch from within IntelliJ) > The intent of this Jira is > * have OMAG server available 'out of the box' in a binary distribution so > that users can easily launch it without having to check classpaths or run in > an odd environment > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2668) Add OMAG Server to distribution
[ https://issues.apache.org/jira/browse/ATLAS-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2668: --- Attachment: ATLAS-2668e.patch > Add OMAG Server to distribution > --- > > Key: ATLAS-2668 > URL: https://issues.apache.org/jira/browse/ATLAS-2668 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Attachments: ATLAS-2668e.patch > > > The standalone Open Metadata server is built under omag-server. > However it is not incorporated into the distribution files (*.gz), and needs > careful attention to the classpath to work (ie launch from within IntelliJ) > The intent of this Jira is > * have OMAG server available 'out of the box' in a binary distribution so > that users can easily launch it without having to check classpaths or run in > an odd environment > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2668) Add OMAG Server to distribution
[ https://issues.apache.org/jira/browse/ATLAS-2668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2668: --- Attachment: (was: ATLAS-2668.patch) > Add OMAG Server to distribution > --- > > Key: ATLAS-2668 > URL: https://issues.apache.org/jira/browse/ATLAS-2668 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Attachments: ATLAS-2668e.patch > > > The standalone Open Metadata server is built under omag-server. > However it is not incorporated into the distribution files (*.gz), and needs > careful attention to the classpath to work (ie launch from within IntelliJ) > The intent of this Jira is > * have OMAG server available 'out of the box' in a binary distribution so > that users can easily launch it without having to check classpaths or run in > an odd environment > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-2593) Governance Engine OMAS - skeleton based on new omrs/omas patterns/ & framework
[ https://issues.apache.org/jira/browse/ATLAS-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482708#comment-16482708 ] Nigel Jones commented on ATLAS-2593: Added current patch. Intermediate WIP. Based on older-style OMRS/omas templates, needs refactoring to accomodate recent changes in rest etc. > Governance Engine OMAS - skeleton based on new omrs/omas patterns/ & framework > -- > > Key: ATLAS-2593 > URL: https://issues.apache.org/jira/browse/ATLAS-2593 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Attachments: ATLAS-2593.patch, ATLAS-2593b.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2593) Governance Engine OMAS - skeleton based on new omrs/omas patterns/ & framework
[ https://issues.apache.org/jira/browse/ATLAS-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2593: --- Attachment: ATLAS-2593b.patch > Governance Engine OMAS - skeleton based on new omrs/omas patterns/ & framework > -- > > Key: ATLAS-2593 > URL: https://issues.apache.org/jira/browse/ATLAS-2593 > Project: Atlas > Issue Type: Sub-task >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Attachments: ATLAS-2593.patch, ATLAS-2593b.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2693) Revert ATLAS-2668 for 1.0 branch
[ https://issues.apache.org/jira/browse/ATLAS-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2693: --- Summary: Revert ATLAS-2668 for 1.0 branch (was: Revert ATLAS-2699 for 1.0 branch) > Revert ATLAS-2668 for 1.0 branch > > > Key: ATLAS-2693 > URL: https://issues.apache.org/jira/browse/ATLAS-2693 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Attachments: ATLAS-2699.patch > > > Revert ATLAS-2668 for the 1.0 branch as per review > [https://reviews.apache.org/r/67060/#comment285318] > > We could revert the entire fix, however the simplest minimum change is to > remove the reference to the assembly from the distro pom. This means that the > binary is not placed into the distro directory, though the jar with > dependencies is still built in the component directory, and the assembly xml > file will still exist in distro. > If we need to revert the entire fix please comment here and I'll update with > a new patch :) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ATLAS-2693) Revert ATLAS-2699 for 1.0 branch
Nigel Jones created ATLAS-2693: -- Summary: Revert ATLAS-2699 for 1.0 branch Key: ATLAS-2693 URL: https://issues.apache.org/jira/browse/ATLAS-2693 Project: Atlas Issue Type: Improvement Reporter: Nigel Jones Revert ATLAS-2668 for the 1.0 branch as per review [https://reviews.apache.org/r/67060/#comment285318] We could revert the entire fix, however the simplest minimum change is to remove the reference to the assembly from the distro pom. This means that the binary is not placed into the distro directory, though the jar with dependencies is still built in the component directory, and the assembly xml file will still exist in distro. If we need to revert the entire fix please comment here and I'll update with a new patch :) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (ATLAS-2693) Revert ATLAS-2699 for 1.0 branch
[ https://issues.apache.org/jira/browse/ATLAS-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones reassigned ATLAS-2693: -- Assignee: Nigel Jones > Revert ATLAS-2699 for 1.0 branch > > > Key: ATLAS-2693 > URL: https://issues.apache.org/jira/browse/ATLAS-2693 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > > Revert ATLAS-2668 for the 1.0 branch as per review > [https://reviews.apache.org/r/67060/#comment285318] > > We could revert the entire fix, however the simplest minimum change is to > remove the reference to the assembly from the distro pom. This means that the > binary is not placed into the distro directory, though the jar with > dependencies is still built in the component directory, and the assembly xml > file will still exist in distro. > If we need to revert the entire fix please comment here and I'll update with > a new patch :) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (ATLAS-2693) Revert ATLAS-2699 for 1.0 branch
[ https://issues.apache.org/jira/browse/ATLAS-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478069#comment-16478069 ] Nigel Jones commented on ATLAS-2693: added patch. testing build. > Revert ATLAS-2699 for 1.0 branch > > > Key: ATLAS-2693 > URL: https://issues.apache.org/jira/browse/ATLAS-2693 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Attachments: ATLAS-2699.patch > > > Revert ATLAS-2668 for the 1.0 branch as per review > [https://reviews.apache.org/r/67060/#comment285318] > > We could revert the entire fix, however the simplest minimum change is to > remove the reference to the assembly from the distro pom. This means that the > binary is not placed into the distro directory, though the jar with > dependencies is still built in the component directory, and the assembly xml > file will still exist in distro. > If we need to revert the entire fix please comment here and I'll update with > a new patch :) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2693) Revert ATLAS-2699 for 1.0 branch
[ https://issues.apache.org/jira/browse/ATLAS-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2693: --- Attachment: ATLAS-2699.patch > Revert ATLAS-2699 for 1.0 branch > > > Key: ATLAS-2693 > URL: https://issues.apache.org/jira/browse/ATLAS-2693 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Attachments: ATLAS-2699.patch > > > Revert ATLAS-2668 for the 1.0 branch as per review > [https://reviews.apache.org/r/67060/#comment285318] > > We could revert the entire fix, however the simplest minimum change is to > remove the reference to the assembly from the distro pom. This means that the > binary is not placed into the distro directory, though the jar with > dependencies is still built in the component directory, and the assembly xml > file will still exist in distro. > If we need to revert the entire fix please comment here and I'll update with > a new patch :) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2693) Revert ATLAS-2668 for 1.0 branch
[ https://issues.apache.org/jira/browse/ATLAS-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2693: --- Attachment: (was: ATLAS-2699.patch) > Revert ATLAS-2668 for 1.0 branch > > > Key: ATLAS-2693 > URL: https://issues.apache.org/jira/browse/ATLAS-2693 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Attachments: ATLAS-2693.patch > > > Revert ATLAS-2668 for the 1.0 branch as per review > [https://reviews.apache.org/r/67060/#comment285318] > > We could revert the entire fix, however the simplest minimum change is to > remove the reference to the assembly from the distro pom. This means that the > binary is not placed into the distro directory, though the jar with > dependencies is still built in the component directory, and the assembly xml > file will still exist in distro. > If we need to revert the entire fix please comment here and I'll update with > a new patch :) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2693) Revert ATLAS-2668 for 1.0 branch
[ https://issues.apache.org/jira/browse/ATLAS-2693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Jones updated ATLAS-2693: --- Attachment: ATLAS-2693.patch > Revert ATLAS-2668 for 1.0 branch > > > Key: ATLAS-2693 > URL: https://issues.apache.org/jira/browse/ATLAS-2693 > Project: Atlas > Issue Type: Improvement >Reporter: Nigel Jones >Assignee: Nigel Jones >Priority: Major > Attachments: ATLAS-2693.patch > > > Revert ATLAS-2668 for the 1.0 branch as per review > [https://reviews.apache.org/r/67060/#comment285318] > > We could revert the entire fix, however the simplest minimum change is to > remove the reference to the assembly from the distro pom. This means that the > binary is not placed into the distro directory, though the jar with > dependencies is still built in the component directory, and the assembly xml > file will still exist in distro. > If we need to revert the entire fix please comment here and I'll update with > a new patch :) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
