date:20180326

Build failed in Jenkins: Atlas-master-UnitTests #870

2018-03-26 Thread Apache Jenkins Server

See 

--
[...truncated 172.73 KB...]
[INFO] Exclude: **/data/**
[INFO] Exclude: **/maven-eclipse.xml
[INFO] Exclude: **/.externalToolBuilders/**
[INFO] Exclude: **/build.log
[INFO] Exclude: **/.bowerrc
[INFO] Exclude: *.json
[INFO] Exclude: **/overlays/**
[INFO] Exclude: dev-support/**
[INFO] Exclude: **/users-credentials.properties
[INFO] Exclude: **/public/css/animate.min.css
[INFO] Exclude: **/public/css/bootstrap-sidebar.css
[INFO] Exclude: **/public/js/external_lib/**
[INFO] Exclude: **/node_modules/**
[INFO] Exclude: **/public/js/libs/**
[INFO] Exclude: **/atlas.data/**
[INFO] Exclude: **/${sys:atlas.data}/**
[INFO] Exclude: **/policy-store.txt
[INFO] Exclude: **/*rebel*.xml
[INFO] Exclude: **/*rebel*.xml.bak
[INFO] Exclude: **/test/resources/**
[INFO] 148 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 
0, approved: 148 licenses.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:copy-resources (copy-resources) @ 
atlas-repository ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 9 resources
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
atlas-repository ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
atlas-repository ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 2 resources to META-INF
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ 
atlas-repository ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 118 source files to 

[INFO] 
:
 Some input files use or override a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
atlas-repository ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 22 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ 
atlas-repository ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 33 source files to 

[INFO] 
:
 

 uses or overrides a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ atlas-repository 
---
[INFO] Surefire report directory: 

[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider

---
 T E S T S
---
Running org.apache.atlas.query.DSLParsingTest
Running org.apache.atlas.query.DSLQueriesTest
Running 
org.apache.atlas.repository.store.graph.v1.AtlasRelationshipStoreHardDeleteV1Test
Running org.apache.atlas.repository.impexp.TypeAttributeDifferenceTest
Running org.apache.atlas.repository.store.graph.v1.AtlasEntityDefStoreV1Test
Running org.apache.atlas.repository.impexp.ZipSinkTest
Running o

Build failed in Jenkins: Atlas-master-UnitTests #869

2018-03-26 Thread Apache Jenkins Server

See 


Changes:

[ssubramanian] ATLAS-2510: Add support to disable/enable propagated 
classification in

--
[...truncated 173.66 KB...]
[INFO] Exclude: **/data/**
[INFO] Exclude: **/maven-eclipse.xml
[INFO] Exclude: **/.externalToolBuilders/**
[INFO] Exclude: **/build.log
[INFO] Exclude: **/.bowerrc
[INFO] Exclude: *.json
[INFO] Exclude: **/overlays/**
[INFO] Exclude: dev-support/**
[INFO] Exclude: **/users-credentials.properties
[INFO] Exclude: **/public/css/animate.min.css
[INFO] Exclude: **/public/css/bootstrap-sidebar.css
[INFO] Exclude: **/public/js/external_lib/**
[INFO] Exclude: **/node_modules/**
[INFO] Exclude: **/public/js/libs/**
[INFO] Exclude: **/atlas.data/**
[INFO] Exclude: **/${sys:atlas.data}/**
[INFO] Exclude: **/policy-store.txt
[INFO] Exclude: **/*rebel*.xml
[INFO] Exclude: **/*rebel*.xml.bak
[INFO] Exclude: **/test/resources/**
[INFO] 148 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 
0, approved: 148 licenses.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:copy-resources (copy-resources) @ 
atlas-repository ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 9 resources
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
atlas-repository ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
atlas-repository ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 2 resources to META-INF
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ 
atlas-repository ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 118 source files to 

[INFO] 
:
 Some input files use or override a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
atlas-repository ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 22 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ 
atlas-repository ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 33 source files to 

[INFO] 
:
 

 uses or overrides a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ atlas-repository 
---
[INFO] Surefire report directory: 

[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider

---
 T E S T S
---
Running org.apache.atlas.query.DSLParsingTest
Running org.apache.atlas.query.DSLQueriesTest
Running 
org.apache.atlas.repository.store.graph.v1.AtlasRelationshipDefStoreV1Test
Running org.apache.atlas.repository.impexp.TypeAttributeDifferenceTest
Running org.apache.atlas.repository.

Re: Review Request 66203: [ATLAS-2510]: Add support to disable/enable propagated classification in entity

2018-03-26 Thread Sarath Subramanian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66203/
---

(Updated March 26, 2018, 10:26 p.m.)


Review request for atlas, Apoorv Naik, Ashutosh Mestry, and Madhan Neethiraj.


Bugs: ATLAS-2510
https://issues.apache.org/jira/browse/ATLAS-2510


Repository: atlas


Description
---

Currently there is no way to remove propagated classification from an entity. 
This Jira provides capability to disable/enable a propagated classification 
from an entity using REST.

The propagated classification name and its associated entity guid is supplied 
to the REST to disable/enable right propagated classification.


Disable a propagated tag:
curl -X PUT 
'http://localhost:21000/api/atlas/v2/entity/guid/0ea86d75-b92c-4907-b149-4fe3fe341cc3/propagatedClassifications/PII?sourceEntityGuid=fb38e88c-c9a8-4fa2-abaa-6be895b067b3&disablePropagation=true'
 -H 'Authorization: Basic YWRtaW46YWRtaW4='

Enable a propagated tag:
curl -X PUT 
'http://localhost:21000/api/atlas/v2/entity/guid/0ea86d75-b92c-4907-b149-4fe3fe341cc3/propagatedClassifications/PII?sourceEntityGuid=fb38e88c-c9a8-4fa2-abaa-6be895b067b3&disablePropagation=false'
 -H 'Authorization: Basic YWRtaW46YWRtaW4='


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 605742dd 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 4a86670c 
  intg/src/main/java/org/apache/atlas/model/instance/AtlasClassification.java 
008314b3 
  intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java fce46da9 
  repository/src/main/java/org/apache/atlas/query/GremlinClause.java 454b3434 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
bfb5a711 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java
 79e8e3e8 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
 b5461d4b 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 19ac6209 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
 c76f6405 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
 8fe635ae 
  repository/src/test/java/org/apache/atlas/query/GremlinQueryComposerTest.java 
9b4c91d1 
  webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java fdafa2c1 


Diff: https://reviews.apache.org/r/66203/diff/6/

Changes: https://reviews.apache.org/r/66203/diff/5-6/


Testing
---

Validated disable and enable of propagated classification through REST client - 
POSTMAN.

https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/213/console


Thanks,

Sarath Subramanian

[jira] [Updated] (ATLAS-2461) Data Migration: Create Migration Utility to Export Contents from Atlas Server

2018-03-26 Thread Madhan Neethiraj (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-2461:

Fix Version/s: 1.0.0

Committed to following branches:
 - branch-0.8: http://git-wip-us.apache.org/repos/asf/atlas/commit/ef642b4c
 - master: http://git-wip-us.apache.org/repos/asf/atlas/commit/99318030

> Data Migration: Create Migration Utility to Export Contents from Atlas Server
> -
>
> Key: ATLAS-2461
> URL: https://issues.apache.org/jira/browse/ATLAS-2461
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 0.8.2
>Reporter: Ashutosh Mestry
>Assignee: Ashutosh Mestry
>Priority: Major
> Fix For: 1.0.0, 0.8.3
>
> Attachments: ATLAS-2461-Performance-improvement.patch, 
> ATLAS-2461-migration-utility-branch-0.8.patch
>
>
> *Background*
> As part of data migration, earlier versions of Atlas (prior to v 1.x) should 
> be export contents of their database to a ZIP file, that would later be 
> imported into a cluster running new version.
> During this process, the server should not accept any requests or process 
> notifications.
> This process should work on an existing cluster without making it necessary 
> to upgrade.
> *Approach*
> Create a command line utility that would perform export using core components 
> from Atlas. This would allow Atlas to be shutdown, so that it does not 
> process any incoming requests or hook messages.
> This utility should allow for using external configuration JSON so that it 
> can be used on Atlas setup that does not have latest version.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66203: [ATLAS-2510]: Add support to disable/enable propagated classification in entity

2018-03-26 Thread Madhan Neethiraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66203/#review26
---


Fix it, then Ship it!





intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java
Lines 85 (patched)


Consider renaming 'disabledPropagatedClassifications' to 
'propagationDisabledClassifications'


- Madhan Neethiraj


On March 26, 2018, 11:14 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66203/
> ---
> 
> (Updated March 26, 2018, 11:14 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2510
> https://issues.apache.org/jira/browse/ATLAS-2510
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Currently there is no way to remove propagated classification from an entity. 
> This Jira provides capability to disable/enable a propagated classification 
> from an entity using REST.
> 
> The propagated classification name and its associated entity guid is supplied 
> to the REST to disable/enable right propagated classification.
> 
> 
> Disable a propagated tag:
> curl -X PUT 
> 'http://localhost:21000/api/atlas/v2/entity/guid/0ea86d75-b92c-4907-b149-4fe3fe341cc3/propagatedClassifications/PII?sourceEntityGuid=fb38e88c-c9a8-4fa2-abaa-6be895b067b3&disablePropagation=true'
>  -H 'Authorization: Basic YWRtaW46YWRtaW4='
> 
> Enable a propagated tag:
> curl -X PUT 
> 'http://localhost:21000/api/atlas/v2/entity/guid/0ea86d75-b92c-4907-b149-4fe3fe341cc3/propagatedClassifications/PII?sourceEntityGuid=fb38e88c-c9a8-4fa2-abaa-6be895b067b3&disablePropagation=false'
>  -H 'Authorization: Basic YWRtaW46YWRtaW4='
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 605742dd 
>   intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 4a86670c 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasClassification.java 
> 008314b3 
>   intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java 
> fce46da9 
>   repository/src/main/java/org/apache/atlas/query/GremlinClause.java 454b3434 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> bfb5a711 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java
>  79e8e3e8 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
>  b5461d4b 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  19ac6209 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  c76f6405 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
>  8fe635ae 
>   
> repository/src/test/java/org/apache/atlas/query/GremlinQueryComposerTest.java 
> 9b4c91d1 
>   webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java fdafa2c1 
> 
> 
> Diff: https://reviews.apache.org/r/66203/diff/5/
> 
> 
> Testing
> ---
> 
> Validated disable and enable of propagated classification through REST client 
> - POSTMAN.
> 
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/213/console
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

Jenkins build is back to normal : PreCommit-ATLAS-Build-Test #219-master-ATLAS-2461-master.patch

2018-03-26 Thread Apache Jenkins Server

See

Build failed in Jenkins: PreCommit-ATLAS-Build-Test #218-master-ATLAS-2510.3.patch

2018-03-26 Thread Apache Jenkins Server

See 


Changes:

[madhan] ATLAS-2507: updated HBase hook to use : as separator between namespace

[madhan] ATLAS-2508: when adding validityPeriod, default value shown in UI is 
not

[ssubramanian] ATLAS-2514: Deleting a table with lineage should not remove 
propagated

--
[...truncated 221.80 KB...]
[INFO] Exclude: **/*rebel*.xml
[INFO] Exclude: **/*rebel*.xml.bak
[INFO] Exclude: **/test/resources/**
[INFO] 148 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 
0, approved: 148 licenses.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:copy-resources (copy-resources) @ 
atlas-repository ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 9 resources
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
atlas-repository ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
atlas-repository ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 2 resources to META-INF
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ 
atlas-repository ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 118 source files to 

[INFO] 
:
 Some input files use or override a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
atlas-repository ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 22 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ 
atlas-repository ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 33 source files to 

[INFO] 
:
 

 uses or overrides a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-surefire-plugin:2.18.1:test (default-test) @ atlas-repository 
---
[INFO] Surefire report directory: 

[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider

---
 T E S T S
---
Running org.apache.atlas.repository.audit.InMemoryAuditRepositoryTest
Running org.apache.atlas.utils.TestResourceFileUtils
Running org.apache.atlas.repository.impexp.TypeAttributeDifferenceTest
Running org.apache.atlas.query.DSLParsingTest
Running org.apache.atlas.repository.userprofile.UserProfileServiceTest
Running 
org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1BulkImportPercentTest
Running org.apache.atlas.query.DSLQueriesTest
Running org.apache.atlas.repository.impexp.ExportServiceTest
Running org.apache.atlas.utils.ObjectUpdateSynchronizerTest
Running org.apache.atlas.query.GremlinQueryComposerTest
Running org.apache.atlas.repository.store.graph.v1.AtlasEntityDefStoreV1Test
Running org.apache.atlas.reposito

Re: Review Request 66203: [ATLAS-2510]: Add support to disable/enable propagated classification in entity

2018-03-26 Thread Sarath Subramanian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66203/
---

(Updated March 26, 2018, 4:14 p.m.)


Review request for atlas, Apoorv Naik, Ashutosh Mestry, and Madhan Neethiraj.


Changes
---

Added disablePropagatedClassifications in AtlasEntity to retrieve all disabled 
propagated classifications.


Bugs: ATLAS-2510
https://issues.apache.org/jira/browse/ATLAS-2510


Repository: atlas


Description
---

Currently there is no way to remove propagated classification from an entity. 
This Jira provides capability to disable/enable a propagated classification 
from an entity using REST.

The propagated classification name and its associated entity guid is supplied 
to the REST to disable/enable right propagated classification.


Disable a propagated tag:
curl -X PUT 
'http://localhost:21000/api/atlas/v2/entity/guid/0ea86d75-b92c-4907-b149-4fe3fe341cc3/propagatedClassifications/PII?sourceEntityGuid=fb38e88c-c9a8-4fa2-abaa-6be895b067b3&disablePropagation=true'
 -H 'Authorization: Basic YWRtaW46YWRtaW4='

Enable a propagated tag:
curl -X PUT 
'http://localhost:21000/api/atlas/v2/entity/guid/0ea86d75-b92c-4907-b149-4fe3fe341cc3/propagatedClassifications/PII?sourceEntityGuid=fb38e88c-c9a8-4fa2-abaa-6be895b067b3&disablePropagation=false'
 -H 'Authorization: Basic YWRtaW46YWRtaW4='


Diffs (updated)
-

  common/src/main/java/org/apache/atlas/repository/Constants.java 605742dd 
  intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 4a86670c 
  intg/src/main/java/org/apache/atlas/model/instance/AtlasClassification.java 
008314b3 
  intg/src/main/java/org/apache/atlas/model/instance/AtlasEntity.java fce46da9 
  repository/src/main/java/org/apache/atlas/query/GremlinClause.java 454b3434 
  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
bfb5a711 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java
 79e8e3e8 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
 b5461d4b 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 19ac6209 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
 c76f6405 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphRetriever.java
 8fe635ae 
  repository/src/test/java/org/apache/atlas/query/GremlinQueryComposerTest.java 
9b4c91d1 
  webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java fdafa2c1 


Diff: https://reviews.apache.org/r/66203/diff/5/

Changes: https://reviews.apache.org/r/66203/diff/4-5/


Testing
---

Validated disable and enable of propagated classification through REST client - 
POSTMAN.

https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/213/console


Thanks,

Sarath Subramanian

Build failed in Jenkins: Atlas-0.8-IntegrationTests #192

2018-03-26 Thread Apache Jenkins Server

See 


Changes:

[amestry] ATLAS-2461: Migration export utility. Improved for performance using

[madhan] ATLAS-2507: updated HBase hook to use : as separator between namespace

--
[...truncated 531.17 KB...]
127.0.0.1 - - [26/Mar/2018:22:20:43 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:43 +] "GET 
/api/atlas/entities?type=testpxZHzNYO5f HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:43 +] "GET /api/atlas/entities/blah 
HTTP/1.1" 404 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:43 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:43 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "GET 
/api/atlas/v2/types/typedef/name/PII_TraitFymEo74wxO HTTP/1.1" 404 - "-" 
"Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST 
/api/atlas/entities/ae1fe9d9-98d3-4d8d-aa00-6fb9db61dcc0/traits HTTP/1.1" 201 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "GET 
/api/atlas/entities/ae1fe9d9-98d3-4d8d-aa00-6fb9db61dcc0/traitDefinitions/PII_TraitFymEo74wxO
 HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "GET 
/api/atlas/entities/ae1fe9d9-98d3-4d8d-aa00-6fb9db61dcc0/traitDefinitions 
HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "GET 
/api/atlas/entities/d1aaa067-7c2f-4b2a-b11c-4910a4e3b272/traits HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST 
/api/atlas/entities/c35198fe-2ab0-4860-bd25-d9052dc67d6b HTTP/1.1" 200 - "-" 
"Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "GET 
/api/atlas/entities/c35198fe-2ab0-4860-bd25-d9052dc67d6b HTTP/1.1" 200 - "-" 
"Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST 
/api/atlas/entities/qualifiedName?type=hive_table&property=qualifiedName&value=tableODCjytExYq
 HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "GET 
/api/atlas/entities/08d2eb5e-418f-4046-b22c-03eba57ae568 HTTP/1.1" 200 - "-" 
"Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "GET 
/api/atlas/entities/d265f0b0-8dad-4c30-9a46-63c7de63e38a/audit?count=10 
HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST /api/atlas/entities HTTP/1.1" 
400 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "GET 
/api/atlas/v2/types/typedef/name/pRB6AVAaOX HTTP/1.1" 404 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "GET 
/api/atlas/entities/0266bac7-a091-42f3-bee2-c1eeac23f29e HTTP/1.1" 200 - "-" 
"Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:44 +] "GET /api/atlas/admin/version 
HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:49 +] "GET /api/atlas/admin/status 
HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:49 +] "GET /api/atlas/v1/taxonomies 
HTTP/1.1" 403 493 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:49 +] "GET /api/atlas/admin/session 
HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:49 +] "GET /api/atlas/admin/session 
HTTP/1.1" 401 317 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:22:20:49 +] "GET 
/api/atlas/v2/types/typedef/name/database HTTP/1.1" 200 - "-" "

[jira] [Resolved] (ATLAS-2507) HBase hook for Atlas should have QualifiedName with ":" as suffix for Namespace instead of "."

2018-03-26 Thread Madhan Neethiraj (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-2507.
-
   Resolution: Fixed
Fix Version/s: 0.8.3
   1.0.0

Committed to following branches:
 - master: http://git-wip-us.apache.org/repos/asf/atlas/commit/c26e9bcd
 - branch-0.8: http://git-wip-us.apache.org/repos/asf/atlas/commit/8e1c6065

> HBase hook for Atlas should have QualifiedName with ":" as suffix for 
> Namespace instead of "."
> --
>
> Key: ATLAS-2507
> URL: https://issues.apache.org/jira/browse/ATLAS-2507
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-intg
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Critical
> Fix For: 1.0.0, 0.8.3
>
>
> HBase hook for Atlas should have QualifiedName with ":" as suffix for 
> Namespace instead of "."
> This is need as in HBase Namespace association with Table is marked with ":" 
> with HBase and when Tag associated to Namespace / Table is propagated to 
> Ranger, then Qualified name for Table / Column family should have 
> "NAMESPACE:TABLE". 
> Currently it is in format "NAMESPACE.TABLE" in Atlas and this is send to 
> Ranger.Policy is created for this 
> resource name. When HBase send Table with Namespace in the Ranger 
> Authorization request, the format is "NAMESPACE:TABLE" and policy created 
> with TAG is not effective because of this mismatch.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: PreCommit-ATLAS-Build-Test #217-master-0001-Use-relationships-to-retrieve-array-elements-propert.patch

2018-03-26 Thread Apache Jenkins Server

See 


Changes:

[madhan] ATLAS-2462: Sqoop import for all tables throws NPE for no table 
provided

[madhan] ATLAS-2505: add client IP address in authorization requests

[madhan] ATLAS-2511: updated import-hive utility to add options to selectively

[madhan] ATLAS-2362: Fix IT failures in sqoop,falcon bridge module (#2)

[ssubramanian] ATLAS-2513: Notifications are not sent about impacted entities 
when

--
[...truncated 223.67 KB...]
Running 
org.apache.atlas.repository.store.graph.v1.AtlasRelationshipStoreHardDeleteV1Test
Running 
org.apache.atlas.repository.store.graph.v1.InverseReferenceUpdateSoftDeleteV1Test
Running org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1Test
Running org.apache.atlas.repository.impexp.AtlasImportRequestTest
Running org.apache.atlas.utils.TestResourceFileUtils
Running org.apache.atlas.repository.store.graph.v1.AtlasRelationshipStoreV1Test
Running org.apache.atlas.repository.audit.InMemoryAuditRepositoryTest
Running 
org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1BulkImportPercentTest
Running org.apache.atlas.repository.userprofile.UserProfileServiceTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.909 sec - in 
org.apache.atlas.TestModules
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.185 sec - in 
org.apache.atlas.repository.store.graph.v1.InverseReferenceUpdateV1Test
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.072 sec - in 
org.apache.atlas.utils.TestResourceFileUtils
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.556 sec - in 
org.apache.atlas.repository.impexp.UniqueListTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.599 sec - in 
org.apache.atlas.repository.store.graph.v1.AtlasRelationshipStoreV1Test
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.883 sec - in 
org.apache.atlas.repository.impexp.ImportTransformerTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.74 sec - in 
org.apache.atlas.repository.audit.InMemoryAuditRepositoryTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.268 sec - in 
org.apache.atlas.repository.impexp.TypeAttributeDifferenceTest
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.383 sec - in 
org.apache.atlas.query.DSLParsingTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.419 sec - in 
org.apache.atlas.repository.impexp.ImportTransformsTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.645 sec - in 
org.apache.atlas.repository.impexp.AtlasImportRequestTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.618 sec - in 
org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1BulkImportPercentTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.969 sec - in 
org.apache.atlas.repository.impexp.ImportTransformerJSONTest
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.084 sec - in 
org.apache.atlas.repository.impexp.ZipSinkTest
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.295 sec - in 
org.apache.atlas.repository.impexp.ZipSourceTest
Tests run: 39, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.925 sec - in 
org.apache.atlas.query.GremlinQueryComposerTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.11 sec - in 
org.apache.atlas.utils.ObjectUpdateSynchronizerTest
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.649 sec - 
in org.apache.atlas.repository.store.graph.v1.AtlasRelationshipDefStoreV1Test
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 36.508 sec - in 
org.apache.atlas.repository.store.graph.v1.AtlasEntityDefStoreV1Test
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 44.215 sec - in 
org.apache.atlas.services.MetricsServiceTest
Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 47.2 sec <<< 
FAILURE! - in 
org.apache.atlas.repository.store.graph.v1.InverseReferenceUpdateSoftDeleteV1Test
testInverseReferenceAutoUpdate_NonCompositeManyToOne(org.apache.atlas.repository.store.graph.v1.InverseReferenceUpdateSoftDeleteV1Test)
  Time elapsed: 0.506 sec  <<< FAILURE!
java.lang.AssertionError: expected [4] but found [2]
at org.testng.Assert.fail(Assert.java:94)
at org.testng.Assert.failNotEquals(Assert.java:496)
at org.testng.Assert.assertEquals(Assert.java:125)
at org.testng.Assert.assertEquals(Assert.java:372)
at org.testng.Assert.assertEquals(Assert.java:382)
at 
org.apache.atlas.repository.store.graph.v1.InverseReferenceUpdateV1Test.testInverseReferenceAutoUpdate_NonCompositeManyToOne(InverseReferenceUpdateV1Test.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMeth

[jira] [Updated] (ATLAS-2470) Basic support for Cassandra

2018-03-26 Thread Pierre Padovani (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pierre Padovani updated ATLAS-2470:
---
Attachment: ATLAS-2470-5.patch

> Basic support for Cassandra 
> 
>
> Key: ATLAS-2470
> URL: https://issues.apache.org/jira/browse/ATLAS-2470
> Project: Atlas
>  Issue Type: Sub-task
>Affects Versions: 1.0.0
>Reporter: Pierre Padovani
>Assignee: Pierre Padovani
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: ATLAS-2470-1.patch, ATLAS-2470-2.patch, 
> ATLAS-2470-3.patch, ATLAS-2470-5.patch, ATLAS-2470.patch
>
>
> Add the basic build support, and ability to run embedded Cassandra and solr 
> using a profile build:
> -Pdist,embedded-cassandra-solr



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-26 Thread Pierre Padovani


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/
---

(Updated March 26, 2018, 8:07 p.m.)


Review request for atlas and David Radley.


Changes
---

Resolved all indicated issues from review. Of special note, changes were made 
to the storm bridge/shim pom files to force the dropwizard metric jars to match 
the version used by other pieces in Atlas. This avoids random test failures due 
to classpath loading ordering.


Bugs: ATLAS-2470
https://issues.apache.org/jira/browse/ATLAS-2470


Repository: atlas


Description
---

This updates pom's to add the needed cassandra jars, and adds a dist profile 
embedded-cassandra-solr.


Diffs (updated)
-

  addons/storm-bridge-shim/pom.xml c25fbfe5 
  addons/storm-bridge/pom.xml 2627d92e 
  distro/pom.xml 0103bef6 
  distro/src/bin/atlas_config.py e6415cf4 
  distro/src/bin/atlas_start.py 39be6b7c 
  distro/src/bin/atlas_stop.py 66edd904 
  distro/src/conf/atlas-application.properties 10fb7b86 
  distro/src/conf/atlas-env.sh 68b24e93 
  distro/src/conf/cassandra.yml.template PRE-CREATION 
  distro/src/conf/zookeeper/log4j.properties PRE-CREATION 
  distro/src/conf/zookeeper/zoo.cfg.template PRE-CREATION 
  distro/src/main/assemblies/standalone-package.xml 1881082e 
  docs/src/site/twiki/InstallationSteps.twiki 6b9f0313 
  graphdb/janus/pom.xml 143b775f 
  pom.xml 9ddc8e18 
  repository/pom.xml b1d6b1f9 
  
repository/src/main/java/org/apache/atlas/repository/audit/AbstractStorageBasedAuditRepository.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/audit/CassandraBasedAuditRepository.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepository.java
 cee7374e 
  
repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
 4d70b7f7 
  
repository/src/test/java/org/apache/atlas/repository/audit/CassandraAuditRepositoryTest.java
 PRE-CREATION 
  repository/src/test/resources/cassandra_test.yml PRE-CREATION 
  
server-api/src/main/java/org/apache/atlas/listener/ActiveStateChangeHandler.java
 7d282b30 
  test-tools/src/main/resources/solr/solr.xml 041f0058 


Diff: https://reviews.apache.org/r/65885/diff/5/

Changes: https://reviews.apache.org/r/65885/diff/4-5/


Testing
---

Full build with the new embedded-cassandra-solr, and testing to make sure Atlas 
comes up and is functional.

Be aware that we have been running Cassandra backed JanusGraph for months with 
no issues.

This patch now includes support for Cassandra as an audit store. I've also 
refactored the audit unit tests, and added cassandra specific tests.


Thanks,

Pierre Padovani

Re: Review Request 66184: Migration Utility: Branch 0.8: Performance Improvement

2018-03-26 Thread Madhan Neethiraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66184/#review12
---


Ship it!




Ship It!

- Madhan Neethiraj


On March 26, 2018, 7:38 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66184/
> ---
> 
> (Updated March 26, 2018, 7:38 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Ruchi Solani, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2461
> https://issues.apache.org/jira/browse/ATLAS-2461
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Background** 
> The migration utility committed ealier has couple of short comings:
> - Relies on Export service.
>   - Needs _export-options.json_ to be specified.
>   - Exporting everything means meticuloulsy updating the options file. It is 
> likely some specification is missed and hence will lead to less data being 
> migrated. 
> - Suffers from performance problems for large data sets.
> 
> **Approach**
> The new approach uses _Titan's_ _GraphSON_ writer. This is configured to 
> export all data in _EXTENDED_ format.
> 
> The _EXTENDED_ format separates _vertices_ and _edges_. This open other 
> interesting avenues for import.
> 
> **Implementation**
> - Modified _Exporter_ to use _AtlasTypeRegistry_ and _GraphSONWriter_.
> - Produced files: 
>- _atlas-typedef.json_: Contains type definitions of all types.
>- _atlas-migration-data.json_: Contains data from the database.
> 
> 
> Diffs
> -
> 
>   distro/src/main/assemblies/migration-exporter.xml 8f751ff9 
>   tools/atlas-migration-exporter/pom.xml 5c6c61ee 
>   
> tools/atlas-migration-exporter/src/main/java/org/apache/atlas/migration/Exporter.java
>  a9873df0 
>   tools/atlas-migration-exporter/src/main/resources/README 2f2bf3e1 
>   tools/atlas-migration-exporter/src/main/resources/atlas-log4j.xml 
> PRE-CREATION 
>   tools/atlas-migration-exporter/src/main/resources/atlas_migration.py 
> 199cde28 
>   
> tools/atlas-migration-exporter/src/main/resources/migration-export-request.json
>  64002aff 
> 
> 
> Diff: https://reviews.apache.org/r/66184/diff/3/
> 
> 
> Testing
> ---
> 
> **Functional tests**
> Export from repositories with:
> - Custom types.
> - Complex lineages.
> - Created hive entities via beeline.
> - Imported data.
> 
> **Gremlin Shell**
> - Used _Gremlin_ shell to perform export operation.
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>

Re: Review Request 66184: Migration Utility: Branch 0.8: Performance Improvement

2018-03-26 Thread Ashutosh Mestry


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66184/
---

(Updated March 26, 2018, 7:38 p.m.)


Review request for atlas, Madhan Neethiraj, Ruchi Solani, and Sarath 
Subramanian.


Changes
---

Updates include: 
- Separate log4j xml for migration log configuration.
- Updated atlas_migration.py to display log display on screen.
- Minor changes to README.


Bugs: ATLAS-2461
https://issues.apache.org/jira/browse/ATLAS-2461


Repository: atlas


Description
---

**Background** 
The migration utility committed ealier has couple of short comings:
- Relies on Export service.
  - Needs _export-options.json_ to be specified.
  - Exporting everything means meticuloulsy updating the options file. It is 
likely some specification is missed and hence will lead to less data being 
migrated. 
- Suffers from performance problems for large data sets.

**Approach**
The new approach uses _Titan's_ _GraphSON_ writer. This is configured to export 
all data in _EXTENDED_ format.

The _EXTENDED_ format separates _vertices_ and _edges_. This open other 
interesting avenues for import.

**Implementation**
- Modified _Exporter_ to use _AtlasTypeRegistry_ and _GraphSONWriter_.
- Produced files: 
   - _atlas-typedef.json_: Contains type definitions of all types.
   - _atlas-migration-data.json_: Contains data from the database.


Diffs (updated)
-

  distro/src/main/assemblies/migration-exporter.xml 8f751ff9 
  tools/atlas-migration-exporter/pom.xml 5c6c61ee 
  
tools/atlas-migration-exporter/src/main/java/org/apache/atlas/migration/Exporter.java
 a9873df0 
  tools/atlas-migration-exporter/src/main/resources/README 2f2bf3e1 
  tools/atlas-migration-exporter/src/main/resources/atlas-log4j.xml 
PRE-CREATION 
  tools/atlas-migration-exporter/src/main/resources/atlas_migration.py 199cde28 
  
tools/atlas-migration-exporter/src/main/resources/migration-export-request.json 
64002aff 


Diff: https://reviews.apache.org/r/66184/diff/3/

Changes: https://reviews.apache.org/r/66184/diff/2-3/


Testing
---

**Functional tests**
Export from repositories with:
- Custom types.
- Complex lineages.
- Created hive entities via beeline.
- Imported data.

**Gremlin Shell**
- Used _Gremlin_ shell to perform export operation.


Thanks,

Ashutosh Mestry

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-26 Thread Pierre Padovani



> On March 26, 2018, 1:32 p.m., David Radley wrote:
> > pom.xml
> > Line 673 (original), 673 (patched)
> > 
> >
> > I did a build with  :"mvn clean  package -Pdist" without your patch and 
> > it runs through cleanly.
> > 
> > I apply your patch and build with "mvn clean  package 
> > -Pdist,embedded-cassandra-solr" and get a build failure - details below. 
> > During the integration tests, it tries to spin up an Atlas and an embedded 
> > solr. This works slightly differently depending on the build options. I 
> > suspect that there is something in the integration tests for the embedded 
> > cassandra build that is not set up correctly so it is not fidning the 
> > embdedd solr. 
> > 
> >  
> > 
> > [INFO] BUILD FAILURE
> > [INFO] 
> > 
> > [INFO] Total time: 04:31 min
> > [INFO] Finished at: 2018-03-26T13:49:19+01:00
> > [INFO] Final Memory: 146M/1574M
> > [INFO] 
> > 
> > [ERROR] Failed to execute goal 
> > org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) 
> > on project atlas-repository: ExecutionException: 
> > java.lang.RuntimeException: There was an error in the forked process
> > [ERROR] java.lang.ExceptionInInitializerError
> > [ERROR] at java.lang.Class.forName0(Native Method)
> > [ERROR] at java.lang.Class.forName(Class.java:264)
> > [ERROR] at 
> > org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:132)
> > [ERROR] at 
> > org.apache.atlas.util.AtlasRepositoryConfiguration.getDeleteHandlerV1Impl(AtlasRepositoryConfiguration.java:98)
> > [ERROR] at 
> > org.apache.atlas.TestModules$TestOnlyModule.bindDeleteHandler(TestModules.java:160)
> > [ERROR] at 
> > org.apache.atlas.TestModules$TestOnlyModule.configure(TestModules.java:115)
> > [ERROR] at 
> > com.google.inject.AbstractModule.configure(AbstractModule.java:62)
> > [ERROR] at 
> > com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
> > [ERROR] at 
> > com.google.inject.spi.Elements.getElements(Elements.java:110)
> > [ERROR] at 
> > com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
> > [ERROR] at 
> > com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
> > [ERROR] at 
> > com.google.inject.internal.InjectorImpl.createChildInjector(InjectorImpl.java:232)
> > [ERROR] at 
> > org.testng.TestRunner.getInjector(TestRunner.java:1626)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getInstanceFromGuice(ClassImpl.java:132)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getDefaultInstance(ClassImpl.java:112)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getInstances(ClassImpl.java:183)
> > [ERROR] at 
> > org.testng.internal.TestNGClassFinder.(TestNGClassFinder.java:128)
> > [ERROR] at 
> > org.testng.TestRunner.initMethods(TestRunner.java:413)
> > [ERROR] at org.testng.TestRunner.init(TestRunner.java:239)
> > [ERROR] at org.testng.TestRunner.init(TestRunner.java:209)
> > [ERROR] at org.testng.TestRunner.(TestRunner.java:157)
> > [ERROR] at 
> > org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:546)
> > [ERROR] at org.testng.SuiteRunner.init(SuiteRunner.java:159)
> > [ERROR] at org.testng.SuiteRunner.(SuiteRunner.java:113)
> > [ERROR] at org.testng.TestNG.createSuiteRunner(TestNG.java:1274)
> > [ERROR] at 
> > org.testng.TestNG.createSuiteRunners(TestNG.java:1261)
> > [ERROR] at org.testng.TestNG.runSuitesLocally(TestNG.java:1115)
> > [ERROR] at org.testng.TestNG.run(TestNG.java:1032)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:115)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:129)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:113)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:111)
> > [ERROR] at 
> > org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
> > [ERROR] at 
> > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
> > [ERROR] at 
> > org.apache.maven.surefire.booter.ForkedBooter.main(Fo

Re: Review Request 66194: ATLAS-2508 : When adding validityPeriod , default value shown in UI is not taken by the POST request

2018-03-26 Thread Madhan Neethiraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66194/#review199988
---


Ship it!




Ship It!

- Madhan Neethiraj


On March 21, 2018, 12:16 p.m., pratik pandey wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66194/
> ---
> 
> (Updated March 21, 2018, 12:16 p.m.)
> 
> 
> Review request for atlas, keval bhatt, Madhan Neethiraj, Nixon Rodrigues, and 
> Sharmadha Sainath.
> 
> 
> Bugs: ATLAS-2508
> https://issues.apache.org/jira/browse/ATLAS-2508
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> When Apply Validity Period check box is checked , default value for start 
> time and end time is applied 
> Example : 2018/03/20 2:18 PM - 2018/03/20 11:59 PM (i.e) 
> Start time : Today's Date Current time
> End time : Todays's Date 11:59 PM
> 
> and no timezone is selected by default.
> 
> But the POST request is sent as 
> "validityPeriods":[
> 
> {"startTime":"","endTime":"","timeZone":""}
> ]}
> 
> 
> Diffs
> -
> 
>   dashboardv2/public/js/views/tag/AddTimezoneItemView.js 420303f 
> 
> 
> Diff: https://reviews.apache.org/r/66194/diff/3/
> 
> 
> Testing
> ---
> 
> Done one round of sanity testing.
> 
> 
> Thanks,
> 
> pratik pandey
> 
>

Re: Review Request 66261: [ATLAS-2514]: Deleting a table with lineage should not remove propagated classification edges

2018-03-26 Thread Madhan Neethiraj


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66261/#review199987
---


Ship it!




Ship It!

- Madhan Neethiraj


On March 26, 2018, 4:18 p.m., Sarath Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66261/
> ---
> 
> (Updated March 26, 2018, 4:18 p.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, and Madhan Neethiraj.
> 
> 
> Bugs: ATLAS-2514
> https://issues.apache.org/jira/browse/ATLAS-2514
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> When a table with tag assigned is deleted, the propagated classification 
> edges to the downstream entities should not be removed.
> 
> 
> Diffs
> -
> 
>   repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
> 9e8077cb 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  18ed533d 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1.java
>  49c9e0ca 
> 
> 
> Diff: https://reviews.apache.org/r/66261/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sarath Subramanian
> 
>

[jira] [Updated] (ATLAS-1831) Virtualiser to dynamically configure an information virtualization layer

2018-03-26 Thread Yao Li (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-1831:
--
Attachment: Virtualiser Summary - JIRA.pdf

> Virtualiser to dynamically configure an information virtualization layer
> 
>
> Key: ATLAS-1831
> URL: https://issues.apache.org/jira/browse/ATLAS-1831
> Project: Atlas
>  Issue Type: New Feature
>  Components: atlas-intg
>Affects Versions: 1.0.0
>Reporter: Mandy Chessell
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0006-ATLAS-1831-26-03-2018-rbt.patch, Virtualiser 
> Summary - JIRA.pdf, java doc.zip
>
>
> Code review is available in [https://reviews.apache.org/r/66174/]
> This component listens for events from the Information View OMAS event 
> publisher built by ATLAS-1711 
> (https://issues.apache.org/jira/browse/ATLAS-1711) to detect changes to 
> relational data assets and configures an information virtualization tool 
> (Gaian in the first instance) to create/maintain a view that maps to the 
> physical data asset. It also populates Information View metadata (using the 
> Information View OMAS API) for each of the views it creates.
> RANGER-1485 (https://issues.apache.org/jira/browse/RANGER-1485) will also 
> configure Gaian with enforcement logic to apply data protection rules to the 
> data when queries are made to the views.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-1831) Virtualiser to dynamically configure an information virtualization layer

2018-03-26 Thread Yao Li (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-1831:
--
Attachment: (was: Virtualiser Summary - JIRA.pdf)

> Virtualiser to dynamically configure an information virtualization layer
> 
>
> Key: ATLAS-1831
> URL: https://issues.apache.org/jira/browse/ATLAS-1831
> Project: Atlas
>  Issue Type: New Feature
>  Components: atlas-intg
>Affects Versions: 1.0.0
>Reporter: Mandy Chessell
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0006-ATLAS-1831-26-03-2018-rbt.patch, java doc.zip
>
>
> Code review is available in [https://reviews.apache.org/r/66174/]
> This component listens for events from the Information View OMAS event 
> publisher built by ATLAS-1711 
> (https://issues.apache.org/jira/browse/ATLAS-1711) to detect changes to 
> relational data assets and configures an information virtualization tool 
> (Gaian in the first instance) to create/maintain a view that maps to the 
> physical data asset. It also populates Information View metadata (using the 
> Information View OMAS API) for each of the views it creates.
> RANGER-1485 (https://issues.apache.org/jira/browse/RANGER-1485) will also 
> configure Gaian with enforcement logic to apply data protection rules to the 
> data when queries are made to the views.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-1831) Virtualiser to dynamically configure an information virtualization layer

2018-03-26 Thread Yao Li (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-1831:
--
Attachment: 0006-ATLAS-1831-26-03-2018-rbt.patch

> Virtualiser to dynamically configure an information virtualization layer
> 
>
> Key: ATLAS-1831
> URL: https://issues.apache.org/jira/browse/ATLAS-1831
> Project: Atlas
>  Issue Type: New Feature
>  Components: atlas-intg
>Affects Versions: 1.0.0
>Reporter: Mandy Chessell
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0006-ATLAS-1831-26-03-2018-rbt.patch, Virtualiser 
> Summary - JIRA.pdf, java doc.zip
>
>
> Code review is available in [https://reviews.apache.org/r/66174/]
> This component listens for events from the Information View OMAS event 
> publisher built by ATLAS-1711 
> (https://issues.apache.org/jira/browse/ATLAS-1711) to detect changes to 
> relational data assets and configures an information virtualization tool 
> (Gaian in the first instance) to create/maintain a view that maps to the 
> physical data asset. It also populates Information View metadata (using the 
> Information View OMAS API) for each of the views it creates.
> RANGER-1485 (https://issues.apache.org/jira/browse/RANGER-1485) will also 
> configure Gaian with enforcement logic to apply data protection rules to the 
> data when queries are made to the views.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-1831) Virtualiser to dynamically configure an information virtualization layer

2018-03-26 Thread Yao Li (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-1831:
--
Attachment: (was: 0003-ATLAS-1831-26-03-2018.patch)

> Virtualiser to dynamically configure an information virtualization layer
> 
>
> Key: ATLAS-1831
> URL: https://issues.apache.org/jira/browse/ATLAS-1831
> Project: Atlas
>  Issue Type: New Feature
>  Components: atlas-intg
>Affects Versions: 1.0.0
>Reporter: Mandy Chessell
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0006-ATLAS-1831-26-03-2018-rbt.patch, Virtualiser 
> Summary - JIRA.pdf, java doc.zip
>
>
> Code review is available in [https://reviews.apache.org/r/66174/]
> This component listens for events from the Information View OMAS event 
> publisher built by ATLAS-1711 
> (https://issues.apache.org/jira/browse/ATLAS-1711) to detect changes to 
> relational data assets and configures an information virtualization tool 
> (Gaian in the first instance) to create/maintain a view that maps to the 
> physical data asset. It also populates Information View metadata (using the 
> Information View OMAS API) for each of the views it creates.
> RANGER-1485 (https://issues.apache.org/jira/browse/RANGER-1485) will also 
> configure Gaian with enforcement logic to apply data protection rules to the 
> data when queries are made to the views.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-26 Thread Pierre Padovani



> On March 26, 2018, 1:32 p.m., David Radley wrote:
> > docs/src/site/twiki/InstallationSteps.twiki
> > Lines 43 (patched)
> > 
> >
> > I suggest mentioningthat you require the MANAGE_LOCAL_SOLR variable to 
> > be set.

This environment variable is managed for you as part of the build. There is an 
issue: https://issues.apache.org/jira/browse/ATLAS-832 which discusses the 
issue. I found that the environment variables in atlas_config.py were not setup 
correctly.


> On March 26, 2018, 1:32 p.m., David Radley wrote:
> > pom.xml
> > Line 673 (original), 673 (patched)
> > 
> >
> > I did a build with  :"mvn clean  package -Pdist" without your patch and 
> > it runs through cleanly.
> > 
> > I apply your patch and build with "mvn clean  package 
> > -Pdist,embedded-cassandra-solr" and get a build failure - details below. 
> > During the integration tests, it tries to spin up an Atlas and an embedded 
> > solr. This works slightly differently depending on the build options. I 
> > suspect that there is something in the integration tests for the embedded 
> > cassandra build that is not set up correctly so it is not fidning the 
> > embdedd solr. 
> > 
> >  
> > 
> > [INFO] BUILD FAILURE
> > [INFO] 
> > 
> > [INFO] Total time: 04:31 min
> > [INFO] Finished at: 2018-03-26T13:49:19+01:00
> > [INFO] Final Memory: 146M/1574M
> > [INFO] 
> > 
> > [ERROR] Failed to execute goal 
> > org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) 
> > on project atlas-repository: ExecutionException: 
> > java.lang.RuntimeException: There was an error in the forked process
> > [ERROR] java.lang.ExceptionInInitializerError
> > [ERROR] at java.lang.Class.forName0(Native Method)
> > [ERROR] at java.lang.Class.forName(Class.java:264)
> > [ERROR] at 
> > org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:132)
> > [ERROR] at 
> > org.apache.atlas.util.AtlasRepositoryConfiguration.getDeleteHandlerV1Impl(AtlasRepositoryConfiguration.java:98)
> > [ERROR] at 
> > org.apache.atlas.TestModules$TestOnlyModule.bindDeleteHandler(TestModules.java:160)
> > [ERROR] at 
> > org.apache.atlas.TestModules$TestOnlyModule.configure(TestModules.java:115)
> > [ERROR] at 
> > com.google.inject.AbstractModule.configure(AbstractModule.java:62)
> > [ERROR] at 
> > com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
> > [ERROR] at 
> > com.google.inject.spi.Elements.getElements(Elements.java:110)
> > [ERROR] at 
> > com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
> > [ERROR] at 
> > com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
> > [ERROR] at 
> > com.google.inject.internal.InjectorImpl.createChildInjector(InjectorImpl.java:232)
> > [ERROR] at 
> > org.testng.TestRunner.getInjector(TestRunner.java:1626)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getInstanceFromGuice(ClassImpl.java:132)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getDefaultInstance(ClassImpl.java:112)
> > [ERROR] at 
> > org.testng.internal.ClassImpl.getInstances(ClassImpl.java:183)
> > [ERROR] at 
> > org.testng.internal.TestNGClassFinder.(TestNGClassFinder.java:128)
> > [ERROR] at 
> > org.testng.TestRunner.initMethods(TestRunner.java:413)
> > [ERROR] at org.testng.TestRunner.init(TestRunner.java:239)
> > [ERROR] at org.testng.TestRunner.init(TestRunner.java:209)
> > [ERROR] at org.testng.TestRunner.(TestRunner.java:157)
> > [ERROR] at 
> > org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:546)
> > [ERROR] at org.testng.SuiteRunner.init(SuiteRunner.java:159)
> > [ERROR] at org.testng.SuiteRunner.(SuiteRunner.java:113)
> > [ERROR] at org.testng.TestNG.createSuiteRunner(TestNG.java:1274)
> > [ERROR] at 
> > org.testng.TestNG.createSuiteRunners(TestNG.java:1261)
> > [ERROR] at org.testng.TestNG.runSuitesLocally(TestNG.java:1115)
> > [ERROR] at org.testng.TestNG.run(TestNG.java:1032)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:115)
> > [ERROR] at 
> > org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:129)
> > [ERROR] at 
> > o

Re: Review Request 66174: ATLAS-1831 Virtualiser to dynamically configure an information virtualization layer

2018-03-26 Thread Yao Li


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66174/
---

(Updated March 26, 2018, 4:54 p.m.)


Review request for atlas, Kees van de Fliert, Mandy Chessell, and Maryna 
Strelchuk.


Repository: atlas


Description
---

Virtualiser has three main funcitons:
1. listen to Information View OMAS (information-view-out-topic) and retrieve 
the json file;
2. update business logic table and technical logic table, send the updates to 
Gaian;
3. create business view json file and technical view json file, notify 
Information View OMAS the update(information-view-in-topic). 

The JIRA can be found https://issues.apache.org/jira/browse/ATLAS-1831


Diffs (updated)
-

  pom.xml ee2746a905f84359d426acfb1fa6d64c6310aaaf 
  virtualiser/README.md PRE-CREATION 
  virtualiser/pom.xml PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/omas/informationview/events/BusinessTerm.java
 PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/omas/informationview/events/ColumnContextEvent.java
 PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/omas/informationview/events/ColumnDetails.java
 PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/omas/informationview/events/ConnectionDetails.java
 PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/omas/informationview/events/DerivedColumnDetail.java
 PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/omas/informationview/events/InformationViewEvent.java
 PRE-CREATION 
  virtualiser/src/main/java/org/apache/atlas/virtualiser/KafkaVirtualiser.java 
PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/virtualiser/ffdc/VirtualiserCheckedException.java
 PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/virtualiser/ffdc/VirtualiserCheckedExceptionBase.java
 PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/virtualiser/ffdc/VirtualiserErrorCode.java
 PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/virtualiser/ffdc/VirtualiserRuntimeException.java
 PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/virtualiser/gaian/GaianQueryConstructor.java
 PRE-CREATION 
  virtualiser/src/main/java/org/apache/atlas/virtualiser/gaian/LogicTable.java 
PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/virtualiser/gaian/MappedColumn.java 
PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/virtualiser/kafka/KafkaVirtualiserConsumer.java
 PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/virtualiser/kafka/KafkaVirtualiserProducer.java
 PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/virtualiser/util/ExecuteQueryUtil.java
 PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/virtualiser/util/PropertiesHelper.java
 PRE-CREATION 
  
virtualiser/src/main/java/org/apache/atlas/virtualiser/views/ViewsConstructor.java
 PRE-CREATION 
  virtualiser/src/main/resources/json/delete.json PRE-CREATION 
  virtualiser/src/main/resources/json/fullyAssign.json PRE-CREATION 
  virtualiser/src/main/resources/json/technical.json PRE-CREATION 
  virtualiser/src/main/resources/json/testIn.json PRE-CREATION 
  virtualiser/src/main/resources/virtualiser.properties PRE-CREATION 
  
virtualiser/src/test/java/org/apache/atlas/virtualiser/GaianQueryConstructorTest.java
 PRE-CREATION 
  virtualiser/src/test/java/org/apache/atlas/virtualiser/JsonReadHelper.java 
PRE-CREATION 
  
virtualiser/src/test/java/org/apache/atlas/virtualiser/KafkaVirtualiserProducerForTest.java
 PRE-CREATION 
  
virtualiser/src/test/java/org/apache/atlas/virtualiser/KafkaVirtualiserTest.java
 PRE-CREATION 
  
virtualiser/src/test/java/org/apache/atlas/virtualiser/ViewsConstructorTest.java
 PRE-CREATION 
  
virtualiser/src/test/java/org/apache/atlas/virtualiser/gaian/GaianQueryConstructorTest.java
 PRE-CREATION 
  
virtualiser/src/test/java/org/apache/atlas/virtualiser/views/ViewsConstructorTest.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/66174/diff/3/

Changes: https://reviews.apache.org/r/66174/diff/2-3/


Testing
---

see test folder


Thanks,

Yao Li

Re: Review Request 66261: [ATLAS-2514]: Deleting a table with lineage should not remove propagated classification edges

2018-03-26 Thread Sarath Subramanian


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66261/
---

(Updated March 26, 2018, 9:18 a.m.)


Review request for atlas, Apoorv Naik, Ashutosh Mestry, and Madhan Neethiraj.


Changes
---

addressed review comments from Madhan.


Bugs: ATLAS-2514
https://issues.apache.org/jira/browse/ATLAS-2514


Repository: atlas


Description
---

When a table with tag assigned is deleted, the propagated classification edges 
to the downstream entities should not be removed.


Diffs (updated)
-

  repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java 
9e8077cb 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 18ed533d 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/SoftDeleteHandlerV1.java
 49c9e0ca 


Diff: https://reviews.apache.org/r/66261/diff/2/

Changes: https://reviews.apache.org/r/66261/diff/1-2/


Testing
---


Thanks,

Sarath Subramanian

[jira] [Assigned] (ATLAS-2515) add classification to the entity fails if there is no read classification permission

2018-03-26 Thread Nixon Rodrigues (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nixon Rodrigues reassigned ATLAS-2515:
--

Assignee: Nixon Rodrigues

> add classification to the entity fails if there is no read classification 
> permission
> 
>
> Key: ATLAS-2515
> URL: https://issues.apache.org/jira/browse/ATLAS-2515
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: trunk
>Reporter: Deepak Sharma
>Assignee: Nixon Rodrigues
>Priority: Critical
> Fix For: trunk
>
>
> steps:
> 1) create a ranger policy to give read,create,update,delete entity + add 
> classification for admin user.
> 2) add a classification to entity by admin user:
> error:
> admin is not authorized to perform get classification



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2503) Fix authorization while creating/updating EnumDef

2018-03-26 Thread Madhan Neethiraj (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-2503:

Reporter: Deepak Sharma  (was: Nixon Rodrigues)

> Fix authorization while creating/updating EnumDef 
> --
>
> Key: ATLAS-2503
> URL: https://issues.apache.org/jira/browse/ATLAS-2503
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: trunk
>Reporter: Deepak Sharma
>Assignee: Nixon Rodrigues
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: ATLAS-2503.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-2517) denied audit log is missing for read entity operation

2018-03-26 Thread Deepak Sharma (JIRA)

Deepak Sharma created ATLAS-2517:


 Summary: denied audit log is missing for read entity operation
 Key: ATLAS-2517
 URL: https://issues.apache.org/jira/browse/ATLAS-2517
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: trunk
Reporter: Deepak Sharma
 Fix For: trunk


Scenario:
1) remove read entity permission for an user.
2) through that user try to read the entity

Result:
read operation fails due to "admin is not authorized to perform read entity"
but denied audit log is missing for this operation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-2516) issue with delete operation authorisation

2018-03-26 Thread Deepak Sharma (JIRA)

Deepak Sharma created ATLAS-2516:


 Summary: issue with delete operation authorisation
 Key: ATLAS-2516
 URL: https://issues.apache.org/jira/browse/ATLAS-2516
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: trunk
Reporter: Deepak Sharma
 Fix For: trunk


issue with delete operation authorisation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 65435: ATLAS-2298 - Review of OCF Database Connector_New

2018-03-26 Thread David Radley

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65435/#review199965
---

ocf-database-connector/README.md
Lines 40 (patched)

bad eol character

ocf-database-connector/README.md
Lines 42 (patched)

typo sequene

the sentence mentions we and you - I suggest using one or the other.

ocf-database-connector/README.md
Lines 58 (patched)

bad eol character

ocf-database-connector/README.md
Lines 61 (patched)

I thought impersonation would require a patch on top of Gaian. If this is 
the case we need to detail where to get this patch for impersonation to be able 
to be used here.

ocf-database-connector/README.md
Lines 69 (patched)

same

ocf-database-connector/README.md
Lines 71 (patched)

'here' is an extra word
Typo Gian.
We have not mention LDAP, I think we need to be careful mentioning products 
that might not be installed.
We should not mention default passwords here - as in production this would 
not make sense.

ocf-database-connector/README.md
Lines 79 (patched)

This text is assuming that people have access to a Gaian Ranger plugin. 
This is not currently available.

You say "it is better to use **table function**. My understanding is that 
using the Ranger plugin requires the table function to be used. 

You say "You need to change cst to your own table name". What is cst?

ocf-database-connector/src/main/java/org/apache/atlas/ocfdbconnector/connectors/gaian/GaianOCFConnector.java
Lines 35 (patched)

incomplete sentence

ocf-database-connector/src/main/java/org/apache/atlas/ocfdbconnector/connectors/gaian/GaianOCFConnector.java
Lines 159 (patched)

finally is driven for errors as well - so this debug message is not 
correct. I suggst moving this debug to the end of the try section.

ocf-database-connector/src/main/java/org/apache/atlas/ocfdbconnector/connectors/gaian/GaianOCFConnector.java
Lines 212 (patched)

should we return this for an error?

ocf-database-connector/src/main/java/org/apache/atlas/ocfdbconnector/connectors/gaian/GaianOCFConnector.java
Lines 232 (patched)

this if and body is a repeat of an previous if.

ocf-database-connector/src/main/java/org/apache/atlas/ocfdbconnector/ffdc/OCFDatabaseConnectorErrorCode.java
Lines 76 (patched)

what is the user's valid information and valid use of OCF Connectors?

ocf-database-connector/src/main/resources/gaian.properties
Lines 6 (patched)

It does not look right to include default passwords here, without talking 
about poc / dev scenarios and not production scenarios

ocf-database-connector/src/test/java/gaianocfconnector/GaianOCFConnectorTest.java
Lines 56 (patched)

It looks like this class might a be unit test, but it seems that it 
requires Gaian to be running. If this cannot be run as a unit test - I suggest 
including instructions on how to run the tests in this file.

ocf-database-connector/src/test/java/gaianocfconnector/UseGaianOCFConnector.java
Lines 33 (patched)

I suggest the test throws the excpetions rather than gobbles them. In this 
way the test fails if there is an error

- David Radley

On March 22, 2018, 11:23 a.m., Yao Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65435/
> ---
> 
> (Updated March 22, 2018, 11:23 a.m.)
> 
> 
> Review request for atlas and Mandy Chessell.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This is the new review request for ATLAS-2298 OCF Database Connector. The old 
> review on 
> [https://reviews.apache.org/r/65123/](https://reviews.apache.org/r/65123/) 
> will not be updated anymore.
> 
> The OCF Database Connector is the subclass of OCF Connector and it is 
> designed especially for connection to database to retrieve data. 
> Here we implement a connector for Gaian (GaianOCFConnector) as an example for 
> using OCF Database Connector. It is related to Open Connec

Re: Review Request 65885: ATLAS-2470 - Add JanusGraph Cassandra support to Atlas

2018-03-26 Thread David Radley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65885/#review199964
---




distro/src/conf/cassandra.yml.template
Lines 21 (patched)


It is not possible to build atlas with a stand alone Cassandra. Also there 
isa typo STANDALONG.



distro/src/conf/cassandra.yml.template
Lines 27 (patched)


we do not seem to be using a end tag convntion - I suggest removing.



distro/src/conf/zookeeper/zoo.cfg.template
Lines 20 (patched)


bad eol character



docs/src/site/twiki/InstallationSteps.twiki
Lines 41 (patched)


I  do not think we should be recommending building without tests.



docs/src/site/twiki/InstallationSteps.twiki
Lines 43 (patched)


I suggest mentioningthat you require the MANAGE_LOCAL_SOLR variable to be 
set.



pom.xml
Line 673 (original), 673 (patched)


I did a build with  :"mvn clean  package -Pdist" without your patch and it 
runs through cleanly.

I apply your patch and build with "mvn clean  package 
-Pdist,embedded-cassandra-solr" and get a build failure - details below. During 
the integration tests, it tries to spin up an Atlas and an embedded solr. This 
works slightly differently depending on the build options. I suspect that there 
is something in the integration tests for the embedded cassandra build that is 
not set up correctly so it is not fidning the embdedd solr. 

 
[INFO] BUILD FAILURE
[INFO] 

[INFO] Total time: 04:31 min
[INFO] Finished at: 2018-03-26T13:49:19+01:00
[INFO] Final Memory: 146M/1574M
[INFO] 

[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) on 
project atlas-repository: ExecutionException: java.lang.RuntimeException: There 
was an error in the forked process
[ERROR] java.lang.ExceptionInInitializerError
[ERROR] at java.lang.Class.forName0(Native Method)
[ERROR] at java.lang.Class.forName(Class.java:264)
[ERROR] at 
org.apache.atlas.ApplicationProperties.getClass(ApplicationProperties.java:132)
[ERROR] at 
org.apache.atlas.util.AtlasRepositoryConfiguration.getDeleteHandlerV1Impl(AtlasRepositoryConfiguration.java:98)
[ERROR] at 
org.apache.atlas.TestModules$TestOnlyModule.bindDeleteHandler(TestModules.java:160)
[ERROR] at 
org.apache.atlas.TestModules$TestOnlyModule.configure(TestModules.java:115)
[ERROR] at 
com.google.inject.AbstractModule.configure(AbstractModule.java:62)
[ERROR] at 
com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
[ERROR] at com.google.inject.spi.Elements.getElements(Elements.java:110)
[ERROR] at 
com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
[ERROR] at 
com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
[ERROR] at 
com.google.inject.internal.InjectorImpl.createChildInjector(InjectorImpl.java:232)
[ERROR] at org.testng.TestRunner.getInjector(TestRunner.java:1626)
[ERROR] at 
org.testng.internal.ClassImpl.getInstanceFromGuice(ClassImpl.java:132)
[ERROR] at 
org.testng.internal.ClassImpl.getDefaultInstance(ClassImpl.java:112)
[ERROR] at 
org.testng.internal.ClassImpl.getInstances(ClassImpl.java:183)
[ERROR] at 
org.testng.internal.TestNGClassFinder.(TestNGClassFinder.java:128)
[ERROR] at org.testng.TestRunner.initMethods(TestRunner.java:413)
[ERROR] at org.testng.TestRunner.init(TestRunner.java:239)
[ERROR] at org.testng.TestRunner.init(TestRunner.java:209)
[ERROR] at org.testng.TestRunner.(TestRunner.java:157)
[ERROR] at 
org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:546)
[ERROR] at org.testng.SuiteRunner.init(SuiteRunner.java:159)
[ERROR] at org.testng.SuiteRunner.(SuiteRunner.java:113)
[ERROR] at org.testng.TestNG.createSuiteRunner(TestNG.java:1274)
[ERROR] at org.testng.TestNG.createSuiteRunners(TestNG.java:1261)
[ERROR] at org.testng.TestNG.runSuitesLocally(TestNG.java:1115)
[ERROR] at org.testng.TestNG.run(TestNG.java:1032)
[ERROR] at 
org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:115)
[ERROR] at 
org.apache.maven.surefire.testng.TestNGDirectoryTes

[jira] [Updated] (ATLAS-2298) OCF Database Connector

2018-03-26 Thread Yao Li (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-2298:
--
Attachment: 0003-ATLAS-2298-26-3-2018.patch

> OCF Database Connector
> --
>
> Key: ATLAS-2298
> URL: https://issues.apache.org/jira/browse/ATLAS-2298
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 1.0.0
>Reporter: Maryna Strelchuk
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0003-ATLAS-2298-26-3-2018.patch, OCF Database Connector 
> - JIRA.pdf, java doc.zip
>
>
> Code review is available in https://reviews.apache.org/r/65435/
> This Jira is focused on development of the OCF Database Connector. 
> The OCF Database Connector is the subclass of OCF Connector and it is 
> designed especially for connection to database to retrieve data. It provides 
> the OCFDatabaseConnector as a basic class for other implementations for 
> various databases.
> Here we implement a connector for Gaian (GaianOCFConnector) as an example for 
> using OCF Database Connector.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-1831) Virtualiser to dynamically configure an information virtualization layer

2018-03-26 Thread Yao Li (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-1831:
--
Attachment: 0003-ATLAS-1831-26-03-2018.patch

> Virtualiser to dynamically configure an information virtualization layer
> 
>
> Key: ATLAS-1831
> URL: https://issues.apache.org/jira/browse/ATLAS-1831
> Project: Atlas
>  Issue Type: New Feature
>  Components: atlas-intg
>Affects Versions: 1.0.0
>Reporter: Mandy Chessell
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0003-ATLAS-1831-26-03-2018.patch, Virtualiser Summary - 
> JIRA.pdf, java doc.zip
>
>
> Code review is available in [https://reviews.apache.org/r/66174/]
> This component listens for events from the Information View OMAS event 
> publisher built by ATLAS-1711 
> (https://issues.apache.org/jira/browse/ATLAS-1711) to detect changes to 
> relational data assets and configures an information virtualization tool 
> (Gaian in the first instance) to create/maintain a view that maps to the 
> physical data asset. It also populates Information View metadata (using the 
> Information View OMAS API) for each of the views it creates.
> RANGER-1485 (https://issues.apache.org/jira/browse/RANGER-1485) will also 
> configure Gaian with enforcement logic to apply data protection rules to the 
> data when queries are made to the views.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2298) OCF Database Connector

2018-03-26 Thread Yao Li (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-2298:
--
Attachment: (was: java doc.zip)

> OCF Database Connector
> --
>
> Key: ATLAS-2298
> URL: https://issues.apache.org/jira/browse/ATLAS-2298
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 1.0.0
>Reporter: Maryna Strelchuk
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0002-ATLAS-2298-22-3-2018-rbt.patch, OCF Database 
> Connector - JIRA.pdf
>
>
> Code review is available in https://reviews.apache.org/r/65435/
> This Jira is focused on development of the OCF Database Connector. 
> The OCF Database Connector is the subclass of OCF Connector and it is 
> designed especially for connection to database to retrieve data. It provides 
> the OCFDatabaseConnector as a basic class for other implementations for 
> various databases.
> Here we implement a connector for Gaian (GaianOCFConnector) as an example for 
> using OCF Database Connector.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-1831) Virtualiser to dynamically configure an information virtualization layer

2018-03-26 Thread Yao Li (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-1831:
--
Attachment: (was: 0001-ATLAS-1831-22-03-2018-rbt.patch)

> Virtualiser to dynamically configure an information virtualization layer
> 
>
> Key: ATLAS-1831
> URL: https://issues.apache.org/jira/browse/ATLAS-1831
> Project: Atlas
>  Issue Type: New Feature
>  Components: atlas-intg
>Affects Versions: 1.0.0
>Reporter: Mandy Chessell
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0003-ATLAS-1831-26-03-2018.patch, Virtualiser Summary - 
> JIRA.pdf, java doc.zip
>
>
> Code review is available in [https://reviews.apache.org/r/66174/]
> This component listens for events from the Information View OMAS event 
> publisher built by ATLAS-1711 
> (https://issues.apache.org/jira/browse/ATLAS-1711) to detect changes to 
> relational data assets and configures an information virtualization tool 
> (Gaian in the first instance) to create/maintain a view that maps to the 
> physical data asset. It also populates Information View metadata (using the 
> Information View OMAS API) for each of the views it creates.
> RANGER-1485 (https://issues.apache.org/jira/browse/RANGER-1485) will also 
> configure Gaian with enforcement logic to apply data protection rules to the 
> data when queries are made to the views.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2298) OCF Database Connector

2018-03-26 Thread Yao Li (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-2298:
--
Attachment: (was: 0002-ATLAS-2298-22-3-2018-rbt.patch)

> OCF Database Connector
> --
>
> Key: ATLAS-2298
> URL: https://issues.apache.org/jira/browse/ATLAS-2298
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 1.0.0
>Reporter: Maryna Strelchuk
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0003-ATLAS-2298-26-3-2018.patch, OCF Database Connector 
> - JIRA.pdf, java doc.zip
>
>
> Code review is available in https://reviews.apache.org/r/65435/
> This Jira is focused on development of the OCF Database Connector. 
> The OCF Database Connector is the subclass of OCF Connector and it is 
> designed especially for connection to database to retrieve data. It provides 
> the OCFDatabaseConnector as a basic class for other implementations for 
> various databases.
> Here we implement a connector for Gaian (GaianOCFConnector) as an example for 
> using OCF Database Connector.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2298) OCF Database Connector

2018-03-26 Thread Yao Li (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yao Li updated ATLAS-2298:
--
Attachment: java doc.zip

> OCF Database Connector
> --
>
> Key: ATLAS-2298
> URL: https://issues.apache.org/jira/browse/ATLAS-2298
> Project: Atlas
>  Issue Type: New Feature
>Affects Versions: 1.0.0
>Reporter: Maryna Strelchuk
>Assignee: Yao Li
>Priority: Major
>  Labels: VirtualDataConnector
> Attachments: 0002-ATLAS-2298-22-3-2018-rbt.patch, OCF Database 
> Connector - JIRA.pdf, java doc.zip
>
>
> Code review is available in https://reviews.apache.org/r/65435/
> This Jira is focused on development of the OCF Database Connector. 
> The OCF Database Connector is the subclass of OCF Connector and it is 
> designed especially for connection to database to retrieve data. It provides 
> the OCFDatabaseConnector as a basic class for other implementations for 
> various databases.
> Here we implement a connector for Gaian (GaianOCFConnector) as an example for 
> using OCF Database Connector.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Jenkins build is back to normal : Atlas-master-IntegrationTests #864

2018-03-26 Thread Apache Jenkins Server

See

Build failed in Jenkins: Atlas-0.8-IntegrationTests #191

2018-03-26 Thread Apache Jenkins Server

See 


Changes:

[madhan] ATLAS-2511: updated import-hive utility to add options to selectively

--
[...truncated 530.68 KB...]
127.0.0.1 - - [26/Mar/2018:07:50:16 +] "GET 
/api/atlas/v2/entity/guid/accd269a-16f4-4108-85f5-da3a7ab594a6 HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:16 +] "POST /api/atlas/v2/entity HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:16 +] "GET 
/api/atlas/v2/entity/guid/3b65f1bb-4fdd-4c06-8d85-dc0fcea4bfd9 HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:16 +] "POST /api/atlas/v2/entity HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:16 +] "GET 
/api/atlas/v2/entity/guid/d1d8ac15-59a1-4851-b032-62bec26dc049 HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:16 +] "POST /api/atlas/v2/entity HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:16 +] "GET 
/api/atlas/v2/entity/guid/ef33e410-a6b6-4d6e-a94c-c1db40df511f HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:16 +] "POST /api/atlas/v2/entity HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:16 +] "GET 
/api/atlas/v2/entity/guid/dc7b90d0-a72a-4300-8bd9-2ccb5bba8093 HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:16 +] "POST /api/atlas/v2/entity HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:16 +] "GET 
/api/atlas/v2/entity/guid/a4e4ef55-dbb8-4f10-a3b5-374bb25fdd98 HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:16 +] "POST /api/atlas/v2/entity HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:16 +] "GET 
/api/atlas/v2/entity/guid/614aafe3-60fa-47bd-ac76-18d67e65f1a9 HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:16 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=from+DB&limit=10 HTTP/1.1" 200 - "-" 
"Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:17 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB&limit=10 HTTP/1.1" 200 - "-" 
"Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:17 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+where+name%3D%22Reporting%22&limit=10
 HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:18 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+where+DB.name%3D%22Reporting%22&limit=10
 HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:19 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+name+%3D+%22Reporting%22&limit=10 
HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:19 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+DB.name+%3D+%22Reporting%22&limit=10 
HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:20 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+where+name%3D%22Reporting%22+select+name,+owner&limit=10
 HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:22 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+where+DB.name%3D%22Reporting%22+select+name,+owner&limit=10
 HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:23 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+has+name&limit=10 HTTP/1.1" 200 - 
"-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:24 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+where+DB+has+name&limit=10 HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:24 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB,+Table&limit=10 HTTP/1.1" 200 - "-" 
"Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:25 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=DB+is+JdbcAccess&limit=10 HTTP/1.1" 200 
- "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:26 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=from+Table&limit=10 HTTP/1.1" 200 - "-" 
"Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:27 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=Table&limit=10 HTTP/1.1" 200 - "-" 
"Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:27 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=Table+is+Dimension&limit=10 HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:28 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=Column+where+Column+isa+PII&limit=10 
HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:29 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=View+is+Dimension&limit=10 HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:30 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=Column+select+Column.name&limit=10 
HTTP/1.1" 200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:07:50:31 +] "GET 
/api/atlas/v2/search/dsl?offset=0&query=Column+select+name&limit=10 HTTP/1.1" 
200 - "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:0

Jenkins build is back to normal : Atlas-0.8-UnitTests #191

2018-03-26 Thread Apache Jenkins Server

See

[jira] [Assigned] (ATLAS-2511) Provide options to selectively import database / tables from Hive into Atlas

2018-03-26 Thread Madhan Neethiraj (JIRA)


 [ 
https://issues.apache.org/jira/browse/ATLAS-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj reassigned ATLAS-2511:
---

   Resolution: Fixed
 Assignee: Ramesh Mani
Fix Version/s: (was: trunk)
   0.8.3

Committed to following branches:
 - master: http://git-wip-us.apache.org/repos/asf/atlas/commit/bd39a509
 - branch-0.8: http://git-wip-us.apache.org/repos/asf/atlas/commit/5f877dda

> Provide options to selectively import database / tables from Hive into Atlas
> 
>
> Key: ATLAS-2511
> URL: https://issues.apache.org/jira/browse/ATLAS-2511
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-intg
>Affects Versions: trunk, 1.0.0
>Reporter: Ramesh Mani
>Assignee: Ramesh Mani
>Priority: Major
> Fix For: 1.0.0, 0.8.3
>
> Attachments: 
> 0001-ATLAS-2511-Provide-options-to-selectively-import-dat.patch
>
>
> Provide options to selectively import database / tables from Hive into Atlas



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: Atlas-master-IntegrationTests #863

2018-03-26 Thread Apache Jenkins Server

See 


--
[...truncated 655.99 KB...]
[INFO] Started Jetty Server
[INFO] 
[INFO] --- maven-failsafe-plugin:2.18.1:integration-test (integration-test) @ 
falcon-bridge ---
[WARNING] useSystemClassloader setting has no effect when not forking
[INFO] Failsafe report directory: 

Running org.apache.atlas.falcon.hook.FalconHookIT
127.0.0.1 - - [26/Mar/2018:06:51:09 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_cluster&property=qualifiedName&value=clusterdrpkN8IYun
 HTTP/1.1" 404 106 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:10 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_cluster&property=qualifiedName&value=clusterdrpkN8IYun
 HTTP/1.1" 404 106 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:10 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_cluster&property=qualifiedName&value=clusterdrpkN8IYun
 HTTP/1.1" 404 106 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:11 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_cluster&property=qualifiedName&value=clusterdrpkN8IYun
 HTTP/1.1" 404 106 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:11 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_cluster&property=qualifiedName&value=clusterdrpkN8IYun
 HTTP/1.1" 404 106 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:12 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_cluster&property=qualifiedName&value=clusterdrpkN8IYun
 HTTP/1.1" 404 106 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:12 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_feed&property=qualifiedName&value=feedcUqDEFMsiC@clusterj6UdtbbWzD
 HTTP/1.1" 404 118 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:12 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_feed&property=qualifiedName&value=feedcUqDEFMsiC@clusterj6UdtbbWzD
 HTTP/1.1" 404 118 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:13 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_feed&property=qualifiedName&value=feedcUqDEFMsiC@clusterj6UdtbbWzD
 HTTP/1.1" 404 118 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:13 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_feed&property=qualifiedName&value=feedcUqDEFMsiC@clusterj6UdtbbWzD
 HTTP/1.1" 200 923 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:13 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_feed&property=qualifiedName&value=feedcUqDEFMsiC@clusterj6UdtbbWzD
 HTTP/1.1" 200 923 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:13 +] "GET 
//localhost:31000/api/atlas/entities/b6b69390-358f-49ef-98be-a68651aa1a02 
HTTP/1.1" 200 923 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:13 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_feed_creation&property=qualifiedName&value=feedcUqDEFMsiC@clusterj6UdtbbWzD
 HTTP/1.1" 200 1266 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:13 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_feed_creation&property=qualifiedName&value=feedcUqDEFMsiC@clusterj6UdtbbWzD
 HTTP/1.1" 200 1266 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:13 +] "GET 
//localhost:31000/api/atlas/entities/c4bfc27b-3a43-436d-8c4d-7ea7cf6324d2 
HTTP/1.1" 200 1266 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:13 +] "GET 
//localhost:31000/api/atlas/entities/038449b5-e97f-4de4-a1ff-64677c7deb0e 
HTTP/1.1" 200 1018 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:14 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_feed&property=qualifiedName&value=feed5DYFklqOGv@clusterj6UdtbbWzD
 HTTP/1.1" 404 118 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:14 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_feed&property=qualifiedName&value=feed5DYFklqOGv@clusterj6UdtbbWzD
 HTTP/1.1" 200 923 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:14 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_feed&property=qualifiedName&value=feed5DYFklqOGv@clusterj6UdtbbWzD
 HTTP/1.1" 200 923 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:14 +] "GET 
//localhost:31000/api/atlas/entities/db3d46fe-ae36-4faa-b1a0-5aa53f92cd4e 
HTTP/1.1" 200 923 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:14 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_feed_creation&property=qualifiedName&value=feed5DYFklqOGv@clusterj6UdtbbWzD
 HTTP/1.1" 200 1267 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:14 +] "GET 
//localhost:31000/api/atlas/entities?type=falcon_feed_creation&property=qualifiedName&value=feed5DYFklqOGv@clusterj6UdtbbWzD
 HTTP/1.1" 200 1267 "-" "Java/1.8.0_152"
127.0.0.1 - - [26/Mar/2018:06:51:14 +] "GET 
//localhost:31000/api/atlas/entities/21be8a6a-e2fc-483c-b65e-0c57d00260c3 
HTTP/

42 matches

Mail list logo