Re: Review Request 66253: Migration: GraphSON-based Import

2018-03-30 Thread Ashutosh Mestry 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66253/
---

(Updated March 31, 2018, 2:18 a.m.)


Review request for atlas, Apoorv Naik, Madhan Neethiraj, Ruchi Solani, Sarath 
Subramanian, and Vishal Suvagia.


Changes
---

Updates include:
- Scalable migration import.
- Significant refactoring.
- Updated status reporting using _status_ REST end point.


Bugs: ATLAS-2460
https://issues.apache.org/jira/browse/ATLAS-2460


Repository: atlas


Description
---

**Background**
This implementation deals with the 'import into' part of the data migration 
process. 

It assumes:
- Export from older cluster is done.
- Generated file has been moved to newer cluster.

**Implementation**

During _Atlas_ server startup, the configuration parameter is checked, if that 
parameter exists, all services except _DataMigrationService_ is started. 
Migration is started. Atlas server is available in _MIGRATION_ mode. It 
processes REST calls made only to the _AdminResource_.

Here's are the udpates:
- New configuration parameter has been added:
_atlas.migration.mode.filename=_
  This configuration parameter is set by Ambari as part of its migration 
orchestration. 
- _DataMigrationService_: New service that performs async migration as soon as 
Atlas server starts up.
- _MigrationProgressService_: Added. Get progress of import.
- _AdminResource.getStatus()_ Now supplies additional status about migration.
- _ServiceState_ Modified to carry additional status _MIGRATION_. This status 
is set by looking at the configuration parameter above.
- _Services_ modified for special handling of _DataMigrationService_.


**CURL**
Check status using:
```
curl -X GET -u admin:admin -H "Content-Type: application/json" -H 
"Cache-Control: no-cache" http://localhost:21000/api/atlas/admin/status
```

**Migration Status**
The above URL in migration mode yields JSON like:
```
{"Status":"MIGRATION","MigrationStatus":{"operationStatus":"SUCCESS","startTime":1521738357947,"endTime":1521738359272,"currentIndex":48544}}
```


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/AtlasConstants.java f5de1df3 
  common/src/main/java/org/apache/atlas/service/Services.java 1267dc92 
  graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraph.java 
31d20855 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java
 6820a93c 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphDatabase.java
 a0060200 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/AtlasGraphSONReader.java
 PRE-CREATION 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/GraphSONTokensTP2.java
 PRE-CREATION 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/GraphSONUtility.java
 PRE-CREATION 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/JsonNodeParsers.java
 PRE-CREATION 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/JsonNodeProcessManager.java
 PRE-CREATION 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/MappedElementCache.java
 PRE-CREATION 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/PostProcessManager.java
 PRE-CREATION 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/ReaderStatusManager.java
 PRE-CREATION 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/RelationshipTypeCache.java
 PRE-CREATION 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/pc/WorkItemBuilder.java
 PRE-CREATION 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/pc/WorkItemConsumer.java
 PRE-CREATION 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/pc/WorkItemManager.java
 PRE-CREATION 
  
graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/JsonNodeManagerTest.java
 PRE-CREATION 
  
graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/PostProcessManagerTest.java
 PRE-CREATION 
  
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Graph.java
 44090097 
  intg/src/main/java/org/apache/atlas/model/impexp/AtlasExportResult.java 
1ea961d8 
  intg/src/main/java/org/apache/atlas/model/impexp/MigrationStatus.java 
PRE-CREATION 
  intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java c63dc24a 
  pom.xml bfbb9535 
  repository/pom.xml b1d6b1f9 
  
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
 5672d9dc 
  
repository/src/main/java/org/apache/atlas/repository/impexp/MigrationProgressService.java
 PRE-CREATION 
  
reposi

Re: Review Request 66372: ATLAS-2524: fix Hive hook for incorrect handling of 'alter view as' operation

2018-03-30 Thread Ramesh Mani 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66372/#review200250
---


Ship it!




Ship It!

- Ramesh Mani


On March 30, 2018, 12:24 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66372/
> ---
> 
> (Updated March 30, 2018, 12:24 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Ramesh Mani, and 
> Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2524
> https://issues.apache.org/jira/browse/ATLAS-2524
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> removed output entities for create/alter operations from 'known' entity 
> cache, so that complete details of the updated entities will be incldued in 
> notificaitons
> 
> 
> Diffs
> -
> 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/AtlasHiveHookContext.java
>  9105ebe66 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRename.java
>  23db807f2 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java
>  0c38bb371 
> 
> 
> Diff: https://reviews.apache.org/r/66372/diff/1/
> 
> 
> Testing
> ---
> 
> Verfied that the metadata of the altered view is seen correctly in Atlas.
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Build failed in Jenkins: Atlas-0.8-IntegrationTests #193

2018-03-30 Thread Apache Jenkins Server 
See 


Changes:

[madhan] ATLAS-2524: fix Hive hook for incorrect handling of 'alter view as'

--
[...truncated 530.02 KB...]
127.0.0.1 - - [30/Mar/2018:18:48:37 +] "GET 
/api/atlas/v2/entity/guid/15671ae9-b3c8-4285-a024-f2a34ddd1e05 HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:37 +] "POST /api/atlas/v2/entity HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:37 +] "GET 
/api/atlas/v2/entity/guid/f33957b3-2006-4d29-92a6-642419f67df7 HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:37 +] "POST /api/atlas/v2/entity HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:37 +] "GET 
/api/atlas/v2/entity/guid/d93aa6ae-46a1-41b6-934a-bcf81d4d0d08 HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:37 +] "POST /api/atlas/v2/entity HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:37 +] "GET 
/api/atlas/v2/entity/guid/a83fce98-60d5-4fbd-8f41-b4b7e0bf4f8c HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:37 +] "POST /api/atlas/v2/entity HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:37 +] "GET 
/api/atlas/v2/entity/guid/f763c6fb-ea39-4867-9ff5-18fb5c98e43b HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:37 +] "POST /api/atlas/v2/entity HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:37 +] "GET 
/api/atlas/v2/entity/guid/b319cb76-abb9-4c9c-846f-76f5bb133f91 HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:37 +] "POST /api/atlas/v2/entity HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:37 +] "GET 
/api/atlas/v2/entity/guid/9fdc5856-8320-47f9-9481-6508e4bf63ef HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:37 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=from+DB&limit=10 HTTP/1.1" 200 - "-" 
"Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:38 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB&limit=10 HTTP/1.1" 200 - "-" 
"Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:38 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+where+name%3D%22Reporting%22&limit=10
 HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:39 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+where+DB.name%3D%22Reporting%22&limit=10
 HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:40 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+name+%3D+%22Reporting%22&limit=10 
HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:40 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+DB.name+%3D+%22Reporting%22&limit=10 
HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:41 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+where+name%3D%22Reporting%22+select+name,+owner&limit=10
 HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:42 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+where+DB.name%3D%22Reporting%22+select+name,+owner&limit=10
 HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:43 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+has+name&limit=10 HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:44 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+where+DB+has+name&limit=10 HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:45 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB,+Table&limit=10 HTTP/1.1" 200 - "-" 
"Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:45 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+is+JdbcAccess&limit=10 HTTP/1.1" 200 
- "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:46 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=from+Table&limit=10 HTTP/1.1" 200 - "-" 
"Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:46 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=Table&limit=10 HTTP/1.1" 200 - "-" 
"Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:47 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=Table+is+Dimension&limit=10 HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:48 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=Column+where+Column+isa+PII&limit=10 
HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:48 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=View+is+Dimension&limit=10 HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:49 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=Column+select+Column.name&limit=10 
HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:48:50 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=Column+select+name&limit=10 HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [30/Mar/2018:18:

[jira] [Updated] (ATLAS-2525) Atlas Hbase/Hive hook improvement to provide import from file

2018-03-30 Thread Ramesh Mani (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramesh Mani updated ATLAS-2525:
---
Description: 
Atlas Hbase/Hive hook improvement to provide import from file

Hook script should have another option -f to accept a file which contains the 
database/table in hive or namespace/table in hbase and import it into Atlas

Format can be JSON or CSV.

HIVE CASE:

 import-hive.sh -f : This can be JSON or CSV file 
{code:java}
import-hive.sh -f : This can be JSON or CSV file
Imports all databases and tables in the file...
cvs format:
database1,tbl1,tbl2,tbl3,..
database2,tbl1,tbl2,tbl3,..
JSON format:
{
"database1" :["tbl1","tbl2","tbl3",..]
"database2" :["tbl1","tbl2","tbl3",..]
}{code}
JSON format:

{ "namespace1" :["tbl1","tbl2","tbl3",..] "namespace2" 
:["tbl1","tbl2","tbl3",..] }

HBASE CASE:
{code:java}
import-hbase.sh [-t ] : This can be JSON or CSV file
 cvs format:
namespace1,tbl1,tbl2,tbl3,..
namespace2,tbl1,tbl2,tbl3,..
 JSON format:
{
"namespace1" :["tbl1","tbl2","tbl3",..]
"namespace2" :["tbl1","tbl2","tbl3",..]
}
{code:java}
 {code}

  was:
Atlas Hbase/Hive hook improvement to provide import from file

Hook script should have another option -f to accept a file which contains the 
database/table in hive or namespace/table in hbase and import it into Atlas

Format can be JSON or CSV.

HIVE CASE:

 import-hive.sh -f : This can be JSON or CSV file 
{code:java}
{code:java}
import-hive.sh -f : This can be JSON or CSV file
Imports all databases and tables in the file...
cvs format:
database1,tbl1,tbl2,tbl3,..
database2,tbl1,tbl2,tbl3,..
JSON format:
{
"database1" :["tbl1","tbl2","tbl3",..]
"database2" :["tbl1","tbl2","tbl3",..]
}{code}
JSON format:

{ "namespace1" :["tbl1","tbl2","tbl3",..] "namespace2" 
:["tbl1","tbl2","tbl3",..] }

HBASE CASE:
{code:java}
import-hbase.sh [-t ] : This can be JSON or CSV file
 cvs format:
namespace1,tbl1,tbl2,tbl3,..
namespace2,tbl1,tbl2,tbl3,..
 JSON format:
{
"namespace1" :["tbl1","tbl2","tbl3",..]
"namespace2" :["tbl1","tbl2","tbl3",..]
}
{code:java}
 {code}


> Atlas Hbase/Hive hook improvement to provide import from file
> -
>
> Key: ATLAS-2525
> URL: https://issues.apache.org/jira/browse/ATLAS-2525
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: trunk, 0.8.2
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Major
> Fix For: trunk, 0.8.2
>
> Attachments: 
> 0001-ATLAS-2525-Atlas-Hbase-Hive-hook-improvement-to-prov.patch
>
>
> Atlas Hbase/Hive hook improvement to provide import from file
> Hook script should have another option -f to accept a file which contains the 
> database/table in hive or namespace/table in hbase and import it into Atlas
> Format can be JSON or CSV.
> HIVE CASE:
>  import-hive.sh -f : This can be JSON or CSV file 
> {code:java}
> import-hive.sh -f : This can be JSON or CSV file
> Imports all databases and tables in the file...
> cvs format:
> database1,tbl1,tbl2,tbl3,..
> database2,tbl1,tbl2,tbl3,..
> JSON format:
> {
> "database1" :["tbl1","tbl2","tbl3",..]
> "database2" :["tbl1","tbl2","tbl3",..]
> }{code}
> JSON format:
> { "namespace1" :["tbl1","tbl2","tbl3",..] "namespace2" 
> :["tbl1","tbl2","tbl3",..] }
> HBASE CASE:
> {code:java}
> import-hbase.sh [-t ] : This can be JSON or CSV file
>  cvs format:
> namespace1,tbl1,tbl2,tbl3,..
> namespace2,tbl1,tbl2,tbl3,..
>  JSON format:
> {
> "namespace1" :["tbl1","tbl2","tbl3",..]
> "namespace2" :["tbl1","tbl2","tbl3",..]
> }
> {code:java}
>  {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2525) Atlas Hbase/Hive hook improvement to provide import from file

2018-03-30 Thread Ramesh Mani (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramesh Mani updated ATLAS-2525:
---
Description: 
Atlas Hbase/Hive hook improvement to provide import from file

Hook script should have another option -f to accept a file which contains the 
database/table in hive or namespace/table in hbase and import it into Atlas

Format can be JSON or CSV.

HIVE CASE:

 import-hive.sh -f : This can be JSON or CSV file 
{code:java}
{code:java}
import-hive.sh -f : This can be JSON or CSV file
Imports all databases and tables in the file...
cvs format:
database1,tbl1,tbl2,tbl3,..
database2,tbl1,tbl2,tbl3,..
JSON format:
{
"database1" :["tbl1","tbl2","tbl3",..]
"database2" :["tbl1","tbl2","tbl3",..]
}{code}
JSON format:

{ "namespace1" :["tbl1","tbl2","tbl3",..] "namespace2" 
:["tbl1","tbl2","tbl3",..] }

HBASE CASE:
{code:java}
import-hbase.sh [-t ] : This can be JSON or CSV file
 cvs format:
namespace1,tbl1,tbl2,tbl3,..
namespace2,tbl1,tbl2,tbl3,..
 JSON format:
{
"namespace1" :["tbl1","tbl2","tbl3",..]
"namespace2" :["tbl1","tbl2","tbl3",..]
}
{code:java}
 {code}

  was:
Atlas Hbase/Hive hook improvement to provide import from file

Hook script should have another option -f to accept a file which contains the 
database/table in hive or namespace/table in hbase and import it into Atlas

Format can be JSON or CSV.

HIVE CASE:


 import-hive.sh -f : This can be JSON or CSV file Imports all 
databases and tables in the file... cvs format: namespace1,tbl1,tbl2,tbl3,.. 
namespace2,tbl1,tbl2,tbl3,..

{code:java}
{code:java}
import-hive.sh -f : This can be JSON or CSV file
Imports all databases and tables in the file...
cvs format:
database1,tbl1,tbl2,tbl3,..
database2,tbl1,tbl2,tbl3,..
JSON format:
{
"database1" :["tbl1","tbl2","tbl3",..]
"database2" :["tbl1","tbl2","tbl3",..]
}{code}

 JSON format:

{ "namespace1" :["tbl1","tbl2","tbl3",..] "namespace2" 
:["tbl1","tbl2","tbl3",..] }

HBASE CASE:
{code:java}
import-hbase.sh [-t ] : This can be JSON or CSV file
 cvs format:
namespace1,tbl1,tbl2,tbl3,..
namespace2,tbl1,tbl2,tbl3,..
 JSON format:
{
"namespace1" :["tbl1","tbl2","tbl3",..]
"namespace2" :["tbl1","tbl2","tbl3",..]
}
{code:java}
 {code}


> Atlas Hbase/Hive hook improvement to provide import from file
> -
>
> Key: ATLAS-2525
> URL: https://issues.apache.org/jira/browse/ATLAS-2525
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: trunk, 0.8.2
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Major
> Fix For: trunk, 0.8.2
>
> Attachments: 
> 0001-ATLAS-2525-Atlas-Hbase-Hive-hook-improvement-to-prov.patch
>
>
> Atlas Hbase/Hive hook improvement to provide import from file
> Hook script should have another option -f to accept a file which contains the 
> database/table in hive or namespace/table in hbase and import it into Atlas
> Format can be JSON or CSV.
> HIVE CASE:
>  import-hive.sh -f : This can be JSON or CSV file 
> {code:java}
> {code:java}
> import-hive.sh -f : This can be JSON or CSV file
> Imports all databases and tables in the file...
> cvs format:
> database1,tbl1,tbl2,tbl3,..
> database2,tbl1,tbl2,tbl3,..
> JSON format:
> {
> "database1" :["tbl1","tbl2","tbl3",..]
> "database2" :["tbl1","tbl2","tbl3",..]
> }{code}
> JSON format:
> { "namespace1" :["tbl1","tbl2","tbl3",..] "namespace2" 
> :["tbl1","tbl2","tbl3",..] }
> HBASE CASE:
> {code:java}
> import-hbase.sh [-t ] : This can be JSON or CSV file
>  cvs format:
> namespace1,tbl1,tbl2,tbl3,..
> namespace2,tbl1,tbl2,tbl3,..
>  JSON format:
> {
> "namespace1" :["tbl1","tbl2","tbl3",..]
> "namespace2" :["tbl1","tbl2","tbl3",..]
> }
> {code:java}
>  {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2525) Atlas Hbase/Hive hook improvement to provide import from file

2018-03-30 Thread Ramesh Mani (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramesh Mani updated ATLAS-2525:
---
Attachment: (was: 
0001-ATLAS-2525-Atlas-Hbase-Hive-hook-improvement-to-prov.patch)

> Atlas Hbase/Hive hook improvement to provide import from file
> -
>
> Key: ATLAS-2525
> URL: https://issues.apache.org/jira/browse/ATLAS-2525
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: trunk, 0.8.2
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Major
> Fix For: trunk, 0.8.2
>
> Attachments: 
> 0001-ATLAS-2525-Atlas-Hbase-Hive-hook-improvement-to-prov.patch
>
>
> Atlas Hbase/Hive hook improvement to provide import from file
> Hook script should have another option -f to accept a file which contains the 
> database/table in hive or namespace/table in hbase and import it into Atlas
> Format can be JSON or CSV.
> HIVE CASE:
>  import-hive.sh -f : This can be JSON or CSV file Imports all 
> databases and tables in the file... cvs format: namespace1,tbl1,tbl2,tbl3,.. 
> namespace2,tbl1,tbl2,tbl3,..
> {code:java}
> {code:java}
> import-hive.sh -f : This can be JSON or CSV file
> Imports all databases and tables in the file...
> cvs format:
> database1,tbl1,tbl2,tbl3,..
> database2,tbl1,tbl2,tbl3,..
> JSON format:
> {
> "database1" :["tbl1","tbl2","tbl3",..]
> "database2" :["tbl1","tbl2","tbl3",..]
> }{code}
>  JSON format:
> { "namespace1" :["tbl1","tbl2","tbl3",..] "namespace2" 
> :["tbl1","tbl2","tbl3",..] }
> HBASE CASE:
> {code:java}
> import-hbase.sh [-t ] : This can be JSON or CSV file
>  cvs format:
> namespace1,tbl1,tbl2,tbl3,..
> namespace2,tbl1,tbl2,tbl3,..
>  JSON format:
> {
> "namespace1" :["tbl1","tbl2","tbl3",..]
> "namespace2" :["tbl1","tbl2","tbl3",..]
> }
> {code:java}
>  {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66372: ATLAS-2524: fix Hive hook for incorrect handling of 'alter view as' operation

2018-03-30 Thread Nixon Rodrigues 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66372/#review200247
---


Ship it!




Ship It!

- Nixon Rodrigues


On March 30, 2018, 12:24 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66372/
> ---
> 
> (Updated March 30, 2018, 12:24 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Ramesh Mani, and 
> Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2524
> https://issues.apache.org/jira/browse/ATLAS-2524
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> removed output entities for create/alter operations from 'known' entity 
> cache, so that complete details of the updated entities will be incldued in 
> notificaitons
> 
> 
> Diffs
> -
> 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/AtlasHiveHookContext.java
>  9105ebe66 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/AlterTableRename.java
>  23db807f2 
>   
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java
>  0c38bb371 
> 
> 
> Diff: https://reviews.apache.org/r/66372/diff/1/
> 
> 
> Testing
> ---
> 
> Verfied that the metadata of the altered view is seen correctly in Atlas.
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Review Request 66374: ATLAS-2526: misc model updates

2018-03-30 Thread Madhan Neethiraj 


> On March 30, 2018, 9:47 a.m., David Radley wrote:
> > addons/models/-Area0/0010-base_model.json
> > Line 205 (original), 205 (patched)
> > 
> >
> > I suggest there may be migration implications to renaming these base 
> > types. I suggest versioning this change.

Relationships 'dataset_process_inputs' and 'process_dataset_outputs' were 
introduced only in master branch in 1.0. Renaming ends of these relationships 
won't impact migration from earlier version.


> On March 30, 2018, 9:47 a.m., David Radley wrote:
> > addons/models/0300-Area3-SubjectArea/0310-Glossary.json
> > Lines 89 (patched)
> > 
> >
> > I thought that we were thinking of these constraints as legacy and had 
> > assumed that we would not be adding any new ones. Doesn't the owned 
> > constraint imply that this is a one directional relationship, but the 
> > relationship definition is bidirectional. 
> > 
> > What does it mean that there is an attribute that uses the contraint 
> > but is not marked as legacy?
> > 
> > I am wondering what we gain by adding the attribute in this way?

- Without mandatory attribute 'anchor', terms and categories can exist without 
being associated with a glossary. This is incorrect.
- Current relationship model doesn't allow to mark a relationship attribute as 
manadatory. In fact allowing such constraint on relationships will make it 
difficult/impossible to build clients that deal with CRUD of entities in such 
relationships, as they need to be updated whenever a new mandatory relationship 
attribute is introduced later.
- From modelling point of view, attributes like 'anchor' and 'parent' are 
integral part of Term and Category; and these shouldn't have to be injected via 
relationship


> On March 30, 2018, 9:47 a.m., David Radley wrote:
> > addons/models/0300-Area3-SubjectArea/0320-CategoryHierarchy.json
> > Lines 32 (patched)
> > 
> >
> > I wonder why we are adding legacy attributes. I can see it gives us a 
> > manditory parent; I thought the relationship design was such that we were 
> > dropping this capability.

The field name 'legacy' is unfortunate (I think you pointed this earlier when 
this was introduced). It is meant to convey if the entity at this relationship 
end has an attribute that represents this end.


- Madhan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66374/#review200231
---


On March 30, 2018, 8:16 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66374/
> ---
> 
> (Updated March 30, 2018, 8:16 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-2526
> https://issues.apache.org/jira/browse/ATLAS-2526
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> - fixed incorrect syntax/braces in 0120-Collections.json that resulted in 
> relationshipDefs not to be loaded
> - renamed sourceToProcesses as inputToProcess and sinkFromProcesses as 
> outputFromProcess
> - added terms and categories as direct attributes of Glossary
> - added anchor, parentCategory and childrenCategories as direct attributes of 
> GlossaryCategory
> - added anchor as direct attribute of GlossaryTerm
> 
> 
> Diffs
> -
> 
>   addons/models/-Area0/0010-base_model.json aebe955d 
>   addons/models/0100-Area1-Collaboration/0120-Collections.json 4dbd2adb 
>   addons/models/0300-Area3-SubjectArea/0310-Glossary.json cac2ce2a 
>   addons/models/0300-Area3-SubjectArea/0320-CategoryHierarchy.json 10637970 
>   addons/models/0300-Area3-SubjectArea/0330-Terms.json f492ddfd 
> 
> 
> Diff: https://reviews.apache.org/r/66374/diff/1/
> 
> 
> Testing
> ---
> 
> Verified Atlas server starts up successfully with updated models
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

[jira] [Resolved] (ATLAS-1840) Area 3 of the open metadata model

2018-03-30 Thread Mandy Chessell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mandy Chessell resolved ATLAS-1840.
---
   Resolution: Fixed
Fix Version/s: 1.0.0

The area 3 models are generated by OMRS java class: 
org.apache.atlas.omrs.archivemanager.opentypes.OpenMetadataTypesArchive.

> Area 3 of the open metadata model
> -
>
> Key: ATLAS-1840
> URL: https://issues.apache.org/jira/browse/ATLAS-1840
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Mandy Chessell
>Assignee: Mandy Chessell
>Priority: Major
>  Labels: OpenMetadata, VirtualDataConnector
> Fix For: 1.0.0
>
>
> This task delivers the JSON files for the new models that describe types for 
> Area 2 in the open metadata model. This area covers collaboration, projects 
> and communities



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (ATLAS-1841) Area 4 of the open metadata model

2018-03-30 Thread Mandy Chessell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mandy Chessell resolved ATLAS-1841.
---
   Resolution: Fixed
Fix Version/s: 1.0.0

The area 4 models are generated by OMRS java class: 
org.apache.atlas.omrs.archivemanager.opentypes.OpenMetadataTypesArchive.

> Area 4 of the open metadata model
> -
>
> Key: ATLAS-1841
> URL: https://issues.apache.org/jira/browse/ATLAS-1841
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Mandy Chessell
>Assignee: Mandy Chessell
>Priority: Major
>  Labels: OpenMetadata, VirtualDataConnector
> Fix For: 1.0.0
>
>
> This task delivers the JSON files for the new models that describe types for 
> Area 4 in the open metadata model. This area covers governance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (ATLAS-1842) Area 5 of the open metadata model

2018-03-30 Thread Mandy Chessell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mandy Chessell resolved ATLAS-1842.
---
Resolution: Fixed

The area 5 models are generated by OMRS java class: 
org.apache.atlas.omrs.archivemanager.opentypes.OpenMetadataTypesArchive.

> Area 5 of the open metadata model
> -
>
> Key: ATLAS-1842
> URL: https://issues.apache.org/jira/browse/ATLAS-1842
> Project: Atlas
>  Issue Type: Task
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Mandy Chessell
>Assignee: Mandy Chessell
>Priority: Major
>  Labels: OpenMetadata, VirtualDataConnector
>
> This task delivers the JSON files for the new models that describe types for 
> Area 5 in the open metadata model. This area covers standards, reference data 
> and models.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (ATLAS-1837) OCF definitions for Area 1 of the open metadata model

2018-03-30 Thread Mandy Chessell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mandy Chessell resolved ATLAS-1837.
---
Resolution: Fixed

The area 1 models are generated by OMRS java class: 
org.apache.atlas.omrs.archivemanager.opentypes.OpenMetadataTypesArchive.

> OCF definitions for Area 1 of the open metadata model
> -
>
> Key: ATLAS-1837
> URL: https://issues.apache.org/jira/browse/ATLAS-1837
> Project: Atlas
>  Issue Type: Task
>Reporter: Mandy Chessell
>Assignee: Mandy Chessell
>Priority: Major
>  Labels: OpenMetadata, VirtualDataConnector
>
> This task delivers the JSON files for the new models that describe types for 
> the open connector framework in Area 1 in the open metadata model.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (ATLAS-1838) Area 1 of the open metadata model

2018-03-30 Thread Mandy Chessell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mandy Chessell resolved ATLAS-1838.
---
Resolution: Fixed

The area 1 models are generated by OMRS java class: 
org.apache.atlas.omrs.archivemanager.opentypes.OpenMetadataTypesArchive.

> Area 1 of the open metadata model
> -
>
> Key: ATLAS-1838
> URL: https://issues.apache.org/jira/browse/ATLAS-1838
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Mandy Chessell
>Assignee: Mandy Chessell
>Priority: Major
>  Labels: OpenMetadata, VirtualDataConnector
>
> This task delivers the JSON files for the new models that describe types for 
> Area 1 in the open metadata model. This area covers Asset and Connector types.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-1775) Create Enterprise OMRS Connector

2018-03-30 Thread Mandy Chessell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mandy Chessell updated ATLAS-1775:
--
Fix Version/s: 1.0.0

> Create Enterprise OMRS Connector
> 
>
> Key: ATLAS-1775
> URL: https://issues.apache.org/jira/browse/ATLAS-1775
> Project: Atlas
>  Issue Type: New Feature
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Mandy Chessell
>Assignee: Mandy Chessell
>Priority: Major
> Fix For: 1.0.0
>
>
> The Enterprise OMRS Connector provides a connector that implements the OMRS 
> Connector API defined in JIRA ATLAS-1773 that is able to aggregate the 
> metadata from multiple metadata repositories in response to metadata requests.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2527) Delete obsolete open metadata model files

2018-03-30 Thread Mandy Chessell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mandy Chessell updated ATLAS-2527:
--
Attachment: 0002-ATLAS-2527-delete-obsolete-models.patch

> Delete obsolete open metadata model files
> -
>
> Key: ATLAS-2527
> URL: https://issues.apache.org/jira/browse/ATLAS-2527
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 1.0.0-alpha
>Reporter: Mandy Chessell
>Assignee: Mandy Chessell
>Priority: Critical
> Attachments: 0002-ATLAS-2527-delete-obsolete-models.patch
>
>
> The open metadata models under directory addons/model need to be deleted 
> before version 1.0 since they are in conflict with the OMRS.  These files 
> were only created as a temporary work around while the OMRS was being written.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2122) OMAG Server

2018-03-30 Thread Mandy Chessell (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mandy Chessell updated ATLAS-2122:
--
Fix Version/s: 1.0.0

> OMAG Server
> ---
>
> Key: ATLAS-2122
> URL: https://issues.apache.org/jira/browse/ATLAS-2122
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 1.0.0
>Reporter: Mandy Chessell
>Assignee: Mandy Chessell
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: 0001-ATLAS-2122-Basic-OMAG-Server.patch
>
>
> This Jira Assembles the Open Metadata and Governance components into a 
> configurable server to support the OMAS Access Layer, Repository or native 
> metadata repository.
> The OMAG server is a Spring Boot Application with a simple REST API. It takes 
> the user Id of the administrator, the server name and cohort name as path 
> variables, other parameters are passed as request parameters. These are 
> examples of the commands to set up the server using defaults. 
> Query configuration 
> GET http://localhost:8080/omag/admin/{userId}/{serverName}/configuration/ 
> Set server type name: 
> POST 
> http://localhost:8080/omag/admin/{userId}/{serverName}/server-type?typeName={name}
>  
> Set organization name: 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/organization-name?name={organizationName}
>  
> Enable the access services: 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/access-services/mode?serviceMode=ENABLED
>  
> Enable the Atlas graph repository: 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/local-repository/mode?repositoryMode=LOCAL_GRAPH_REPOSITORY
>  
> Enable the in-memory repository: 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/local-repository/mode?repositoryMode=IN_MEMORY_REPOSITORY
>  
> Enable server as a repository proxy: 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/local-repository/mode?repositoryMode=REPOSITORY_PROXY
>  
> To add the local repository connection for the repository proxy 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/local-repository/proxy-details?connectorProvider={javaClassName}&url={nativeServerURL}
>  
> To add the local repository's event mapper: 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/local-repository/event-mapper-details?connectorProvider={javaClassName}&eventSource={resourceName}
>  
> To enable access to a cohort 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/cohort/{cohortName}/mode?serviceMode=ENABLED
>  
> To remove the local repository 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/local-repository/mode?repositoryMode=NO_LOCAL_REPOSITORY
>  
> To disable the access services 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/access-services/mode?serviceMode=DISABLED
>  
> To disconnect from a cohort 
> POST 
> http://localhost:8080/omag/admin//{userId}/{serverName}/cohort/{cohortName}/mode?serviceMode=DISABLED
>  
> To start up OMRS/OMAS services 
> POST http://localhost:8080/omag/admin//{userId}/{serverName}/instance 
> To shutdown OMRS/OMAS services 
> DELETE 
> http://localhost:8080/omag/admin//{userId}/{serverName}/instance?permanent=false
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-2527) Delete obsolete open metadata model files

2018-03-30 Thread Mandy Chessell (JIRA) 
Mandy Chessell created ATLAS-2527:
-

 Summary: Delete obsolete open metadata model files
 Key: ATLAS-2527
 URL: https://issues.apache.org/jira/browse/ATLAS-2527
 Project: Atlas
  Issue Type: Improvement
  Components:  atlas-core
Affects Versions: 1.0.0-alpha
Reporter: Mandy Chessell
Assignee: Mandy Chessell


The open metadata models under directory addons/model need to be deleted before 
version 1.0 since they are in conflict with the OMRS.  These files were only 
created as a temporary work around while the OMRS was being written.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66374: ATLAS-2526: misc model updates

2018-03-30 Thread Mandy Chessell 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66374/#review200233
---



The open metadata model files will be deleted before we ship v1.0.  Could be in 
this Jira, or another one.

- Mandy Chessell


On March 30, 2018, 8:16 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66374/
> ---
> 
> (Updated March 30, 2018, 8:16 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-2526
> https://issues.apache.org/jira/browse/ATLAS-2526
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> - fixed incorrect syntax/braces in 0120-Collections.json that resulted in 
> relationshipDefs not to be loaded
> - renamed sourceToProcesses as inputToProcess and sinkFromProcesses as 
> outputFromProcess
> - added terms and categories as direct attributes of Glossary
> - added anchor, parentCategory and childrenCategories as direct attributes of 
> GlossaryCategory
> - added anchor as direct attribute of GlossaryTerm
> 
> 
> Diffs
> -
> 
>   addons/models/-Area0/0010-base_model.json aebe955d 
>   addons/models/0100-Area1-Collaboration/0120-Collections.json 4dbd2adb 
>   addons/models/0300-Area3-SubjectArea/0310-Glossary.json cac2ce2a 
>   addons/models/0300-Area3-SubjectArea/0320-CategoryHierarchy.json 10637970 
>   addons/models/0300-Area3-SubjectArea/0330-Terms.json f492ddfd 
> 
> 
> Diff: https://reviews.apache.org/r/66374/diff/1/
> 
> 
> Testing
> ---
> 
> Verified Atlas server starts up successfully with updated models
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Review Request 66374: ATLAS-2526: misc model updates

2018-03-30 Thread David Radley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66374/#review200231
---




addons/models/-Area0/0010-base_model.json
Line 205 (original), 205 (patched)


I suggest there may be migration implications to renaming these base types. 
I suggest versioning this change.



addons/models/0300-Area3-SubjectArea/0310-Glossary.json
Lines 89 (patched)


I thought that we were thinking of these constraints as legacy and had 
assumed that we would not be adding any new ones. Doesn't the owned constraint 
imply that this is a one directional relationship, but the relationship 
definition is bidirectional. 

What does it mean that there is an attribute that uses the contraint but is 
not marked as legacy?

I am wondering what we gain by adding the attribute in this way?



addons/models/0300-Area3-SubjectArea/0320-CategoryHierarchy.json
Lines 32 (patched)


I wonder why we are adding legacy attributes. I can see it gives us a 
manditory parent; I thought the relationship design was such that we were 
dropping this capability.


- David Radley


On March 30, 2018, 8:16 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66374/
> ---
> 
> (Updated March 30, 2018, 8:16 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-2526
> https://issues.apache.org/jira/browse/ATLAS-2526
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> - fixed incorrect syntax/braces in 0120-Collections.json that resulted in 
> relationshipDefs not to be loaded
> - renamed sourceToProcesses as inputToProcess and sinkFromProcesses as 
> outputFromProcess
> - added terms and categories as direct attributes of Glossary
> - added anchor, parentCategory and childrenCategories as direct attributes of 
> GlossaryCategory
> - added anchor as direct attribute of GlossaryTerm
> 
> 
> Diffs
> -
> 
>   addons/models/-Area0/0010-base_model.json aebe955d 
>   addons/models/0100-Area1-Collaboration/0120-Collections.json 4dbd2adb 
>   addons/models/0300-Area3-SubjectArea/0310-Glossary.json cac2ce2a 
>   addons/models/0300-Area3-SubjectArea/0320-CategoryHierarchy.json 10637970 
>   addons/models/0300-Area3-SubjectArea/0330-Terms.json f492ddfd 
> 
> 
> Diff: https://reviews.apache.org/r/66374/diff/1/
> 
> 
> Testing
> ---
> 
> Verified Atlas server starts up successfully with updated models
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Review Request 65435: ATLAS-2298 - Review of OCF Database Connector_New

2018-03-30 Thread Yao Li 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65435/
---

(Updated March 30, 2018, 9:33 a.m.)


Review request for atlas and Mandy Chessell.


Repository: atlas


Description
---

This is the new review request for ATLAS-2298 OCF Database Connector. The old 
review on 
[https://reviews.apache.org/r/65123/](https://reviews.apache.org/r/65123/) will 
not be updated anymore.

The OCF Database Connector is the subclass of OCF Connector and it is designed 
especially for connection to database to retrieve data. 
Here we implement a connector for Gaian (GaianOCFConnector) as an example for 
using OCF Database Connector. It is related to Open Connector Framework. The 
JIRA can be found https://issues.apache.org/jira/browse/ATLAS-2298


Diffs (updated)
-

  ocf-database-connector/README.md PRE-CREATION 
  ocf-database-connector/pom.xml PRE-CREATION 
  
ocf-database-connector/src/main/java/org/apache/atlas/ocfdbconnector/connectors/OCFDatabaseConnection.java
 PRE-CREATION 
  
ocf-database-connector/src/main/java/org/apache/atlas/ocfdbconnector/connectors/OCFDatabaseConnector.java
 PRE-CREATION 
  
ocf-database-connector/src/main/java/org/apache/atlas/ocfdbconnector/connectors/OCFDatabaseConnectorProviderBase.java
 PRE-CREATION 
  
ocf-database-connector/src/main/java/org/apache/atlas/ocfdbconnector/connectors/gaian/GaianOCFConnector.java
 PRE-CREATION 
  
ocf-database-connector/src/main/java/org/apache/atlas/ocfdbconnector/connectors/gaian/GaianOCFConnectorProvider.java
 PRE-CREATION 
  
ocf-database-connector/src/main/java/org/apache/atlas/ocfdbconnector/ffdc/DatabaseConnectCheckedException.java
 PRE-CREATION 
  
ocf-database-connector/src/main/java/org/apache/atlas/ocfdbconnector/ffdc/OCFDatabaseCheckedExceptionBase.java
 PRE-CREATION 
  
ocf-database-connector/src/main/java/org/apache/atlas/ocfdbconnector/ffdc/OCFDatabaseConnectorErrorCode.java
 PRE-CREATION 
  
ocf-database-connector/src/main/java/org/apache/atlas/ocfdbconnector/ffdc/OCFDatabaseConnectorRuntimeException.java
 PRE-CREATION 
  
ocf-database-connector/src/test/java/gaianocfconnector/GaianOCFConnectorTest.java
 PRE-CREATION 
  
ocf-database-connector/src/test/java/gaianocfconnector/UseGaianOCFConnector.java
 PRE-CREATION 
  pom.xml ee2746a905f84359d426acfb1fa6d64c6310aaaf 


Diff: https://reviews.apache.org/r/65435/diff/5/

Changes: https://reviews.apache.org/r/65435/diff/4-5/


Testing
---

see test folder.
Gaian has to set up in advance


File Attachments


0005-ATLAS-2298-05-Feb-code-review.patch
  
https://reviews.apache.org/media/uploaded/files/2018/02/05/292c1917-9deb-4ced-8bd6-1689fc45fbd4__0005-ATLAS-2298-05-Feb-code-review.patch


Thanks,

Yao Li

[jira] [Updated] (ATLAS-1831) Virtualiser to dynamically configure an information virtualization layer

2018-03-30 Thread Yao Li (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-1831:
--
Attachment: Virtualiser Summary - JIRA.pdf

> Virtualiser to dynamically configure an information virtualization layer
> 
>
> Key: ATLAS-1831
> URL: https://issues.apache.org/jira/browse/ATLAS-1831
> Project: Atlas
>  Issue Type: New Feature
>  Components: atlas-intg
>Affects Versions: 1.0.0
>Reporter: Mandy Chessell
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0008-ATLAS-1831-29-03-2018.patch, Virtualiser Summary - 
> JIRA.pdf, java doc.zip
>
>
> Code review is available in [https://reviews.apache.org/r/66174/]
> This component listens for events from the Information View OMAS event 
> publisher built by ATLAS-1711 
> (https://issues.apache.org/jira/browse/ATLAS-1711) to detect changes to 
> relational data assets and configures an information virtualization tool 
> (Gaian in the first instance) to create/maintain a view that maps to the 
> physical data asset. It also populates Information View metadata (using the 
> Information View OMAS API) for each of the views it creates.
> RANGER-1485 (https://issues.apache.org/jira/browse/RANGER-1485) will also 
> configure Gaian with enforcement logic to apply data protection rules to the 
> data when queries are made to the views.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-1831) Virtualiser to dynamically configure an information virtualization layer

2018-03-30 Thread Yao Li (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-1831:
--
Attachment: (was: Virtualiser Summary - JIRA.pdf)

> Virtualiser to dynamically configure an information virtualization layer
> 
>
> Key: ATLAS-1831
> URL: https://issues.apache.org/jira/browse/ATLAS-1831
> Project: Atlas
>  Issue Type: New Feature
>  Components: atlas-intg
>Affects Versions: 1.0.0
>Reporter: Mandy Chessell
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0008-ATLAS-1831-29-03-2018.patch, Virtualiser Summary - 
> JIRA.pdf, java doc.zip
>
>
> Code review is available in [https://reviews.apache.org/r/66174/]
> This component listens for events from the Information View OMAS event 
> publisher built by ATLAS-1711 
> (https://issues.apache.org/jira/browse/ATLAS-1711) to detect changes to 
> relational data assets and configures an information virtualization tool 
> (Gaian in the first instance) to create/maintain a view that maps to the 
> physical data asset. It also populates Information View metadata (using the 
> Information View OMAS API) for each of the views it creates.
> RANGER-1485 (https://issues.apache.org/jira/browse/RANGER-1485) will also 
> configure Gaian with enforcement logic to apply data protection rules to the 
> data when queries are made to the views.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2298) OCF Database Connector

2018-03-30 Thread Yao Li (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-2298:
--
Attachment: (was: OCF Database Connector - JIRA.pdf)

> OCF Database Connector
> --
>
> Key: ATLAS-2298
> URL: https://issues.apache.org/jira/browse/ATLAS-2298
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 1.0.0
>Reporter: Maryna Strelchuk
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0006-ATLAS-2298-30-3-2018.patch, OCF Database Connector 
> - JIRA.pdf, java doc.zip
>
>
> Code review is available in https://reviews.apache.org/r/65435/
> This Jira is focused on development of the OCF Database Connector. 
> The OCF Database Connector is the subclass of OCF Connector and it is 
> designed especially for connection to database to retrieve data. It provides 
> the OCFDatabaseConnector as a basic class for other implementations for 
> various databases.
> Here we implement a connector for Gaian (GaianOCFConnector) as an example for 
> using OCF Database Connector.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2298) OCF Database Connector

2018-03-30 Thread Yao Li (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-2298:
--
Attachment: OCF Database Connector - JIRA.pdf

> OCF Database Connector
> --
>
> Key: ATLAS-2298
> URL: https://issues.apache.org/jira/browse/ATLAS-2298
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 1.0.0
>Reporter: Maryna Strelchuk
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0006-ATLAS-2298-30-3-2018.patch, OCF Database Connector 
> - JIRA.pdf, java doc.zip
>
>
> Code review is available in https://reviews.apache.org/r/65435/
> This Jira is focused on development of the OCF Database Connector. 
> The OCF Database Connector is the subclass of OCF Connector and it is 
> designed especially for connection to database to retrieve data. It provides 
> the OCFDatabaseConnector as a basic class for other implementations for 
> various databases.
> Here we implement a connector for Gaian (GaianOCFConnector) as an example for 
> using OCF Database Connector.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Jenkins build is back to normal : PreCommit-ATLAS-Build-Test #233-master-

2018-03-30 Thread Apache Jenkins Server 
See

[jira] [Updated] (ATLAS-2298) OCF Database Connector

2018-03-30 Thread Yao Li (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-2298:
--
Attachment: (was: 0005-ATLAS-2298-29-03-2018.patch)

> OCF Database Connector
> --
>
> Key: ATLAS-2298
> URL: https://issues.apache.org/jira/browse/ATLAS-2298
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 1.0.0
>Reporter: Maryna Strelchuk
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0006-ATLAS-2298-30-3-2018.patch, OCF Database Connector 
> - JIRA.pdf, java doc.zip
>
>
> Code review is available in https://reviews.apache.org/r/65435/
> This Jira is focused on development of the OCF Database Connector. 
> The OCF Database Connector is the subclass of OCF Connector and it is 
> designed especially for connection to database to retrieve data. It provides 
> the OCFDatabaseConnector as a basic class for other implementations for 
> various databases.
> Here we implement a connector for Gaian (GaianOCFConnector) as an example for 
> using OCF Database Connector.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2298) OCF Database Connector

2018-03-30 Thread Yao Li (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-2298:
--
Attachment: (was: java doc.zip)

> OCF Database Connector
> --
>
> Key: ATLAS-2298
> URL: https://issues.apache.org/jira/browse/ATLAS-2298
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 1.0.0
>Reporter: Maryna Strelchuk
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0006-ATLAS-2298-30-3-2018.patch, OCF Database Connector 
> - JIRA.pdf, java doc.zip
>
>
> Code review is available in https://reviews.apache.org/r/65435/
> This Jira is focused on development of the OCF Database Connector. 
> The OCF Database Connector is the subclass of OCF Connector and it is 
> designed especially for connection to database to retrieve data. It provides 
> the OCFDatabaseConnector as a basic class for other implementations for 
> various databases.
> Here we implement a connector for Gaian (GaianOCFConnector) as an example for 
> using OCF Database Connector.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2298) OCF Database Connector

2018-03-30 Thread Yao Li (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-2298:
--
Attachment: java doc.zip

> OCF Database Connector
> --
>
> Key: ATLAS-2298
> URL: https://issues.apache.org/jira/browse/ATLAS-2298
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 1.0.0
>Reporter: Maryna Strelchuk
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0006-ATLAS-2298-30-3-2018.patch, OCF Database Connector 
> - JIRA.pdf, java doc.zip
>
>
> Code review is available in https://reviews.apache.org/r/65435/
> This Jira is focused on development of the OCF Database Connector. 
> The OCF Database Connector is the subclass of OCF Connector and it is 
> designed especially for connection to database to retrieve data. It provides 
> the OCFDatabaseConnector as a basic class for other implementations for 
> various databases.
> Here we implement a connector for Gaian (GaianOCFConnector) as an example for 
> using OCF Database Connector.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2298) OCF Database Connector

2018-03-30 Thread Yao Li (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-2298:
--
Attachment: 0006-ATLAS-2298-30-3-2018.patch

> OCF Database Connector
> --
>
> Key: ATLAS-2298
> URL: https://issues.apache.org/jira/browse/ATLAS-2298
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 1.0.0
>Reporter: Maryna Strelchuk
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0006-ATLAS-2298-30-3-2018.patch, OCF Database Connector 
> - JIRA.pdf, java doc.zip
>
>
> Code review is available in https://reviews.apache.org/r/65435/
> This Jira is focused on development of the OCF Database Connector. 
> The OCF Database Connector is the subclass of OCF Connector and it is 
> designed especially for connection to database to retrieve data. It provides 
> the OCFDatabaseConnector as a basic class for other implementations for 
> various databases.
> Here we implement a connector for Gaian (GaianOCFConnector) as an example for 
> using OCF Database Connector.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: PreCommit-ATLAS-Build-Test #232-master-

2018-03-30 Thread Apache Jenkins Server 
See 


--
Started by user madhan
[EnvInject] - Loading node environment variables.
Building remotely on H0 (Hadoop) in workspace 

[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Done
Cloning the remote Git repository
Cloning repository http://git-wip-us.apache.org/repos/asf/atlas.git
 > git init  # 
 > timeout=10
Fetching upstream changes from http://git-wip-us.apache.org/repos/asf/atlas.git
 > git --version # timeout=10
 > git fetch --tags --progress http://git-wip-us.apache.org/repos/asf/atlas.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git config remote.origin.url 
 > http://git-wip-us.apache.org/repos/asf/atlas.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # 
 > timeout=10
 > git config remote.origin.url 
 > http://git-wip-us.apache.org/repos/asf/atlas.git # timeout=10
Fetching upstream changes from http://git-wip-us.apache.org/repos/asf/atlas.git
 > git fetch --tags --progress http://git-wip-us.apache.org/repos/asf/atlas.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 9ffaaabaaf692a07ea74f929515659fa425feff8 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 9ffaaabaaf692a07ea74f929515659fa425feff8
Commit message: "ATLAS-2515: fix authorization of delete-classification 
operation"
 > git rev-list --no-walk 9ffaaabaaf692a07ea74f929515659fa425feff8 # timeout=10
[EnvInject] - Mask passwords that will be passed as build parameters.
Set build name.
New build name is '#232-master-'
[PreCommit-ATLAS-Build-Test] $ /bin/bash /tmp/jenkins1440802766364993727.sh
+ export TOOLS_HOME=/home/jenkins/tools
+ TOOLS_HOME=/home/jenkins/tools
+ export ANT_HOME=/home/jenkins/tools/ant/latest
+ ANT_HOME=/home/jenkins/tools/ant/latest
+ export FORREST_HOME=/home/jenkins/tools/forrest/latest
+ FORREST_HOME=/home/jenkins/tools/forrest/latest
+ export JAVA5_HOME=/home/jenkins/tools/java/latest1.5
+ JAVA5_HOME=/home/jenkins/tools/java/latest1.5
+ export FINDBUGS_HOME=/home/jenkins/tools/findbugs/latest
+ FINDBUGS_HOME=/home/jenkins/tools/findbugs/latest
+ export CLOVER_HOME=/home/jenkins/tools/clover/latest
+ CLOVER_HOME=/home/jenkins/tools/clover/latest
+ export MAVEN_HOME=/home/jenkins/tools/maven/latest
+ MAVEN_HOME=/home/jenkins/tools/maven/latest
+ export 
PATH=/home/jenkins/tools/java/latest1.8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/jenkins/tools/java/latest1.8/bin:/home/jenkins/tools/ant/latest/bin:/home/jenkins/tools/maven/latest/bin
+ 
PATH=/home/jenkins/tools/java/latest1.8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/jenkins/tools/java/latest1.8/bin:/home/jenkins/tools/ant/latest/bin:/home/jenkins/tools/maven/latest/bin
+ export 'MAVEN_OPTS=-Xmx4096m -XX:MaxPermSize=1024m -Dfile.encoding=UTF-8 
-noverify'
+ MAVEN_OPTS='-Xmx4096m -XX:MaxPermSize=1024m -Dfile.encoding=UTF-8 -noverify'
+ export 'ANT_OPTS=-Xmx4096m -XX:MaxPermSize=1024m -Dfile.encoding=UTF-8 
-noverify'
+ ANT_OPTS='-Xmx4096m -XX:MaxPermSize=1024m -Dfile.encoding=UTF-8 -noverify'
+ env
BUILD_URL=https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/232/
XDG_SESSION_ID=57
SKIP_UT=false
ISSUE_NUM=2526
HUDSON_SERVER_COOKIE=f4ebd1e6b0d976e8
SHELL=/bin/bash
FORREST_HOME=/home/jenkins/tools/forrest/latest
SSH_CLIENT=62.210.60.235 53050 22
TOOLS_HOME=/home/jenkins/tools
BUILD_TAG=jenkins-PreCommit-ATLAS-Build-Test-232
GIT_PREVIOUS_COMMIT=9ffaaabaaf692a07ea74f929515659fa425feff8
ROOT_BUILD_CAUSE=MANUALTRIGGER
ANT_HOME=/home/jenkins/tools/ant/latest
WORKSPACE=
JOB_URL=https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/
ANT_OPTS=-Xmx4096m -XX:MaxPermSize=1024m -Dfile.encoding=UTF-8 -noverify
BRANCH=master
RUN_CHANGES_DISPLAY_URL=https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/232/display/redirect?page=changes
USER=jenkins
GIT_AUTHOR_NAME=jenkins
GIT_COMMITTER_NAME=jenkins
MAVEN_OPTS=-Xmx4096m -XX:MaxPermSize=1024m -Dfile.encoding=UTF-8 -noverify
NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat
GIT_COMMIT=9ffaaabaaf692a07ea74f929515659fa425feff8
JENKINS_HOME=/x1/jenkins/jenkins-home
MAVEN_HOME=/home/jenkins/tools/maven/latest
MAIL=/var/mail/jenkins
PATH=/home/jenkins/tools/java/latest1.8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/jenkins/tools/java/latest1.8/bin:/home/jenkins/tools/ant/latest/bin:/home/jenkins/tools/maven/latest/bin
RUN_DISPLAY_URL=https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/232/display/redirect
BUILD_CAUSE_MANUALTRIGGER=true
GIT_COMMITTER_EMAIL=bui...@apache.org
PWD=

[jira] [Updated] (ATLAS-2526) Misc model updates

2018-03-30 Thread Madhan Neethiraj (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-2526:

Attachment: ATLAS-2526.patch

> Misc model updates
> --
>
> Key: ATLAS-2526
> URL: https://issues.apache.org/jira/browse/ATLAS-2526
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
> Attachments: ATLAS-2526.patch
>
>
> This Jira is to track following minor changes to few models:
> - 0120-Collections.json: fix incorrect syntax/braces that resulted in 
> relationshipDefs not to be loaded
> - rename sourceToProcesses as inputToProcess and sinkFromProcesses as 
> outputFromProcess - as this makes it easier to read relationship diagram
> - add terms and categories as direct attributes of Glossary for convenience. 
> Corresponding relationships can continue to exist
> - add anchor, parentCategory and childrenCategories as direct attributes of 
> GlossaryCategory for convenience. Also, this makes it possible to make anchor 
> as a mandatory attribute for GlossaryCategory
> - add anchor as direct attribute of GlossaryTerm for convenience. Also, this 
> makes it possible to make anchor as a mandatory attribute for GlossaryCategory



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Review Request 66374: ATLAS-2526: misc model updates

2018-03-30 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66374/
---

Review request for atlas.


Summary (updated)
-

ATLAS-2526: misc model updates


Bugs: ATLAS-2526
https://issues.apache.org/jira/browse/ATLAS-2526


Repository: atlas


Description (updated)
---

- fixed incorrect syntax/braces in 0120-Collections.json that resulted in 
relationshipDefs not to be loaded
- renamed sourceToProcesses as inputToProcess and sinkFromProcesses as 
outputFromProcess
- added terms and categories as direct attributes of Glossary
- added anchor, parentCategory and childrenCategories as direct attributes of 
GlossaryCategory
- added anchor as direct attribute of GlossaryTerm


Diffs (updated)
-

  addons/models/-Area0/0010-base_model.json aebe955d 
  addons/models/0100-Area1-Collaboration/0120-Collections.json 4dbd2adb 
  addons/models/0300-Area3-SubjectArea/0310-Glossary.json cac2ce2a 
  addons/models/0300-Area3-SubjectArea/0320-CategoryHierarchy.json 10637970 
  addons/models/0300-Area3-SubjectArea/0330-Terms.json f492ddfd 


Diff: https://reviews.apache.org/r/66374/diff/1/


Testing (updated)
---

Verified Atlas server starts up successfully with updated models


Thanks,

Madhan Neethiraj

[jira] [Updated] (ATLAS-2526) Misc model updates

2018-03-30 Thread Madhan Neethiraj (JIRA) 

 [ 
https://issues.apache.org/jira/browse/ATLAS-2526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-2526:

Description: 
This Jira is to track following minor changes to few models:

- 0120-Collections.json: fix incorrect syntax/braces that resulted in 
relationshipDefs not to be loaded
- rename sourceToProcesses as inputToProcess and sinkFromProcesses as 
outputFromProcess - as this makes it easier to read relationship diagram
- add terms and categories as direct attributes of Glossary for convenience. 
Corresponding relationships can continue to exist
- add anchor, parentCategory and childrenCategories as direct attributes of 
GlossaryCategory for convenience. Also, this makes it possible to make anchor 
as a mandatory attribute for GlossaryCategory
- add anchor as direct attribute of GlossaryTerm for convenience. Also, this 
makes it possible to make anchor as a mandatory attribute for GlossaryCategory


  was:
This Jira is to track following minor changes to few models:

- 0120-Collections.json: fix incorrect syntax/braces that resulted in 
relationshipDefs not to be loaded
- rename sourceToProcesses as inputToProcess and sinkFromProcesses as 
outputFromProcess - as this makes it easier to read relationship diagram
- add terms and categories as direct attributes of Glossary for convenience. 
Corresponding relationships can continue to exist
- add anchor, parentCategory and childrenCategories as direct attributes of 
GlossaryCategory for convenience. Also, this makes it possible to make anchor 
as a mandatory attribute for GlossaryCategory
- add anchor as direct attribute of GlossaryTerm for convenience. Also, this 
makes it possible to make anchor as a mandatory attribute for GlossaryCategory



> Misc model updates
> --
>
> Key: ATLAS-2526
> URL: https://issues.apache.org/jira/browse/ATLAS-2526
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
>Priority: Major
>
> This Jira is to track following minor changes to few models:
> - 0120-Collections.json: fix incorrect syntax/braces that resulted in 
> relationshipDefs not to be loaded
> - rename sourceToProcesses as inputToProcess and sinkFromProcesses as 
> outputFromProcess - as this makes it easier to read relationship diagram
> - add terms and categories as direct attributes of Glossary for convenience. 
> Corresponding relationships can continue to exist
> - add anchor, parentCategory and childrenCategories as direct attributes of 
> GlossaryCategory for convenience. Also, this makes it possible to make anchor 
> as a mandatory attribute for GlossaryCategory
> - add anchor as direct attribute of GlossaryTerm for convenience. Also, this 
> makes it possible to make anchor as a mandatory attribute for GlossaryCategory



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-2526) Misc model updates

2018-03-30 Thread Madhan Neethiraj (JIRA) 
Madhan Neethiraj created ATLAS-2526:
---

 Summary: Misc model updates
 Key: ATLAS-2526
 URL: https://issues.apache.org/jira/browse/ATLAS-2526
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 1.0.0
Reporter: Madhan Neethiraj
Assignee: Madhan Neethiraj


This Jira is to track following minor changes to few models:

- 0120-Collections.json: fix incorrect syntax/braces that resulted in 
relationshipDefs not to be loaded
- rename sourceToProcesses as inputToProcess and sinkFromProcesses as 
outputFromProcess - as this makes it easier to read relationship diagram
- add terms and categories as direct attributes of Glossary for convenience. 
Corresponding relationships can continue to exist
- add anchor, parentCategory and childrenCategories as direct attributes of 
GlossaryCategory for convenience. Also, this makes it possible to make anchor 
as a mandatory attribute for GlossaryCategory
- add anchor as direct attribute of GlossaryTerm for convenience. Also, this 
makes it possible to make anchor as a mandatory attribute for GlossaryCategory




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)